2024-07-17T06:40:37.944 INFO:root:teuthology version: 0.0.1.dev355+g5bc6960 2024-07-17T06:40:37.945 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_mchangir@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap-schedule} --archive /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141 --name mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi -- /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/orig.config.yaml 2024-07-17T06:40:37.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-17T06:40:38.078 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap-schedule} email: mchangir@redhat.com first_in_suite: false job_id: '7805141' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-qa-fail-mgr-to-avoid-races-debug sha1: 9c8b7e3d36b252529f99a61f43068893847721b3 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3770 sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-qa-fail-mgr-to-avoid-races-debug suite_path: /home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: 9c8b7e3d36b252529f99a61f43068893847721b3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXhPnFUzMjsuB14IfhLrblh3foLrDMnkvX7aB8u8J/2+IW8JQTm+Cs6JvEc9wkLwfEjzC3n32gPMGxNeHcZCpU= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAnsJZIsoX8SOfKl6iExPGZfkuVG4tXTc+v7VrtFDaGeWAa/Hk2cAYFR/0IUwU21MuPuGNPAbhbGHvXOy9V44Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5bc696040f54eb999ca02a20c94bfd735f22b93c timestamp: 2024-07-17_06:32:46 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.621193 2024-07-17T06:40:38.078 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa; will attempt to use it 2024-07-17T06:40:38.079 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks 2024-07-17T06:40:38.079 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-17T06:40:38.081 INFO:teuthology.task.internal:Checking packages... 2024-07-17T06:40:38.101 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0325fd3c11a225c0fd9eb38fab88aad1370ad779' 2024-07-17T06:40:38.102 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-17T06:40:38.102 INFO:teuthology.packaging:ref: None 2024-07-17T06:40:38.102 INFO:teuthology.packaging:tag: None 2024-07-17T06:40:38.102 INFO:teuthology.packaging:branch: main 2024-07-17T06:40:38.102 INFO:teuthology.packaging:sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-17T06:40:38.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-07-17T06:40:38.341 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5040.gf19d3f83 2024-07-17T06:40:38.342 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-17T06:40:38.349 INFO:teuthology.task.internal:no buildpackages task found 2024-07-17T06:40:38.349 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-17T06:40:38.355 INFO:teuthology.task.internal:Saving configuration 2024-07-17T06:40:38.373 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-17T06:40:38.379 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-17T06:40:38.404 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-17 06:35:49.239746', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXhPnFUzMjsuB14IfhLrblh3foLrDMnkvX7aB8u8J/2+IW8JQTm+Cs6JvEc9wkLwfEjzC3n32gPMGxNeHcZCpU='} 2024-07-17T06:40:38.428 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-17 06:35:49.241546', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAnsJZIsoX8SOfKl6iExPGZfkuVG4tXTc+v7VrtFDaGeWAa/Hk2cAYFR/0IUwU21MuPuGNPAbhbGHvXOy9V44Y='} 2024-07-17T06:40:38.428 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-17T06:40:38.434 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-07-17T06:40:38.434 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-07-17T06:40:38.434 INFO:teuthology.run_tasks:Running task console_log... 2024-07-17T06:40:38.494 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f43034fbbe0>, signals=[15]) 2024-07-17T06:40:38.494 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-17T06:40:38.500 INFO:teuthology.task.internal:Opening connections... 2024-07-17T06:40:38.500 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:40:38.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:40:38.586 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:40:38.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:40:38.661 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-17T06:40:38.668 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-07-17T06:40:38.686 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-07-17T06:40:38.686 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-07-17T06:40:38.745 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-17T06:40:38.746 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-17T06:40:38.747 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-07-17T06:40:38.775 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-07-17T06:40:38.793 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-07-17T06:40:38.793 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-07-17T06:40:38.850 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-07-17T06:40:38.853 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-07-17T06:40:38.854 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-17T06:40:38.854 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-07-17T06:40:38.854 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-17T06:40:38.854 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-17T06:40:38.854 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-17T06:40:38.854 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-07-17T06:40:38.875 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-17T06:40:38.884 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-17T06:40:38.987 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-17T06:40:38.987 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-07-17T06:40:38.989 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-07-17T06:40:39.006 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-17T06:40:39.011 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-17T06:40:39.011 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-07-17T06:40:39.048 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-07-17T06:40:39.077 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-17T06:40:39.135 INFO:teuthology.run_tasks:Running task kernel... 2024-07-17T06:40:39.149 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-07-17T06:40:39.149 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-07-17T06:40:39.149 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-07-17T06:40:39.150 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-17T06:40:39.150 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-17T06:40:39.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:40:39.167 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-07-17T06:40:39.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:40:39.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-07-17T06:40:39.382 DEBUG:teuthology.task.kernel:sha1 for client.0 is c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:40:39.382 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "c30fd5683ea85fa16b09ca6926db2200d11edd1d"... 2024-07-17T06:40:39.383 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-07-17T06:40:39.389 DEBUG:teuthology.task.kernel:sha1 for client.1 is c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:40:39.389 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "c30fd5683ea85fa16b09ca6926db2200d11edd1d"... 2024-07-17T06:40:39.390 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-07-17T06:40:39.400 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-07-17T06:40:39.400 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:40:39.401 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-07-17T06:40:39.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum upgrade -y linux-firmware 2024-07-17T06:40:39.407 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-437.el9.x86_64 2024-07-17T06:40:39.407 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:40:39.407 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-07-17T06:40:39.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum upgrade -y linux-firmware 2024-07-17T06:40:41.684 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-07-17T06:40:41.941 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.2 MB 00:00 2024-07-17T06:40:43.577 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-07-17T06:40:45.067 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-07-17T06:40:47.961 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-07-17T06:40:49.887 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 17 kB 00:00 2024-07-17T06:40:50.832 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-07-17T06:40:51.334 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-07-17T06:40:53.151 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 17 kB 00:00 2024-07-17T06:40:54.013 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-07-17T06:40:56.305 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-07-17T06:40:58.520 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 1 Package 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:40:58.534 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 380 M 2024-07-17T06:40:58.535 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-07-17T06:40:59.477 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 7.8 kB/s | 1.7 kB 00:00 2024-07-17T06:41:01.624 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-07-17T06:41:01.630 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-07-17T06:41:01.631 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-07-17T06:41:01.632 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 1 Package 2024-07-17T06:41:01.632 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:41:01.632 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 380 M 2024-07-17T06:41:01.632 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-07-17T06:41:03.932 INFO:teuthology.orchestra.run.smithi044.stdout:linux-firmware-20240603-144.el9.noarch.rpm 74 MB/s | 380 MB 00:05 2024-07-17T06:41:03.932 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-07-17T06:41:03.932 INFO:teuthology.orchestra.run.smithi044.stdout:Total 70 MB/s | 380 MB 00:05 2024-07-17T06:41:07.141 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-07-17T06:41:07.225 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-07-17T06:41:07.225 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-07-17T06:41:07.586 INFO:teuthology.orchestra.run.smithi175.stdout:linux-firmware-20240603-144.el9.noarch.rpm 65 MB/s | 380 MB 00:05 2024-07-17T06:41:07.586 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-17T06:41:07.586 INFO:teuthology.orchestra.run.smithi175.stdout:Total 64 MB/s | 380 MB 00:05 2024-07-17T06:41:10.569 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-07-17T06:41:10.569 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-07-17T06:41:10.659 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-07-17T06:41:10.810 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-07-17T06:41:10.811 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-07-17T06:41:14.040 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-07-17T06:41:14.041 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-07-17T06:41:19.863 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-07-17T06:41:20.691 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-17T06:41:20.910 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:21.569 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:21.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout: linux-firmware-20240603-144.el9.noarch 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:41:22.269 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-07-17T06:41:22.590 INFO:teuthology.task.kernel:Downloading kernel c30fd5683ea85fa16b09ca6926db2200d11edd1d on client.0... 2024-07-17T06:41:22.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:41:22.730 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/ 2024-07-17T06:41:22.731 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.10.0_rc7_gc30fd5683ea8-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/ --input-file=- 2024-07-17T06:41:22.733 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi044.front.sepia.ceph.com to complete... 2024-07-17T06:41:22.973 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-07-17T06:41:23.733 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-17T06:41:23.823 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:24.676 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:24.676 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-17T06:41:25.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-17T06:41:25.420 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:41:25.420 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-07-17T06:41:25.420 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware-20240603-144.el9.noarch 2024-07-17T06:41:25.421 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:41:25.421 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-07-17T06:41:25.727 INFO:teuthology.task.kernel:Downloading kernel c30fd5683ea85fa16b09ca6926db2200d11edd1d on client.1... 2024-07-17T06:41:25.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c30fd5683ea85fa16b09ca6926db2200d11edd1d 2024-07-17T06:41:25.873 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/ 2024-07-17T06:41:25.873 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.10.0_rc7_gc30fd5683ea8-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/ --input-file=- 2024-07-17T06:41:25.876 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi175.front.sepia.ceph.com to complete... 2024-07-17T06:41:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17 06:41:42 URL:https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/kernel-6.10.0_rc7_gc30fd5683ea8-1.x86_64.rpm [358856690/358856690] -> "/tmp/kernel.x86_64.rpm" [1] 2024-07-17T06:41:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:FINISHED --2024-07-17 06:41:42-- 2024-07-17T06:41:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:Total wall clock time: 20s 2024-07-17T06:41:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:Downloaded: 1 files, 342M in 20s (17.5 MB/s) 2024-07-17T06:41:42.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17 06:41:42 URL:https://3.chacra.ceph.com/r/kernel/testing/c30fd5683ea85fa16b09ca6926db2200d11edd1d/centos/9/flavors/default/x86_64/kernel-6.10.0_rc7_gc30fd5683ea8-1.x86_64.rpm [358856690/358856690] -> "/tmp/kernel.x86_64.rpm" [1] 2024-07-17T06:41:42.650 INFO:teuthology.orchestra.run.smithi175.stderr:FINISHED --2024-07-17 06:41:42-- 2024-07-17T06:41:42.650 INFO:teuthology.orchestra.run.smithi175.stderr:Total wall clock time: 17s 2024-07-17T06:41:42.650 INFO:teuthology.orchestra.run.smithi175.stderr:Downloaded: 1 files, 342M in 16s (20.8 MB/s) 2024-07-17T06:41:42.655 INFO:teuthology.task.kernel:Installing kernel c30fd5683ea85fa16b09ca6926db2200d11edd1d on client.0... 2024-07-17T06:41:42.655 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-07-17T06:41:42.655 INFO:teuthology.task.kernel:Installing kernel c30fd5683ea85fa16b09ca6926db2200d11edd1d on client.1... 2024-07-17T06:41:42.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-07-17T06:41:44.705 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying... ######################################## 2024-07-17T06:41:44.815 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying... ######################################## 2024-07-17T06:41:44.853 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing... ######################################## 2024-07-17T06:41:44.869 INFO:teuthology.orchestra.run.smithi044.stdout:Preparing... ######################################## 2024-07-17T06:41:44.904 INFO:teuthology.orchestra.run.smithi175.stdout:Updating / installing... 2024-07-17T06:41:44.922 INFO:teuthology.orchestra.run.smithi044.stdout:Updating / installing... 2024-07-17T06:41:52.862 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-6.10.0_rc7_gc30fd5683ea8-1 ######################################## 2024-07-17T06:41:53.266 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-6.10.0_rc7_gc30fd5683ea8-1 ######################################## 2024-07-17T06:42:32.461 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-07-17T06:42:32.462 DEBUG:teuthology.orchestra.run.smithi175:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/System.map-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/config-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/vmlinuz-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/System.map 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/config 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto 2024-07-17T06:42:32.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crc32c-intel.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-avx2.ko 2024-07-17T06:42:32.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel 2024-07-17T06:42:32.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu/mce 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm-amd.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm-intel.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ansi_cprng.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/anubis.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/arc4.ko 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx 2024-07-17T06:42:32.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_memcpy.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_pq.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_raid6_recov.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_tx.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_xor.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/raid6test.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/authenc.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/authencesn.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blake2b_generic.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blowfish_common.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blowfish_generic.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/camellia_generic.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast5_generic.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast6_generic.ko 2024-07-17T06:42:32.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast_common.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ccm.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cmac.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crc32_generic.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crypto_engine.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crypto_user.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cts.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/des_generic.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/dh_generic.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ecc.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ecdh_generic.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/echainiv.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/essiv.ko 2024-07-17T06:42:32.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/fcrypt.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/khazad.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/lz4.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/lz4hc.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/md4.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/michael_mic.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/pcbc.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/pcrypt.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/rmd160.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/seed.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/serpent_generic.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/tcrypt.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/tea.ko 2024-07-17T06:42:32.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/twofish_common.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/twofish_generic.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/vmac.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/wp512.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xcbc.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xor.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xxhash_generic.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/acpi_ipmi.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/acpi_pad.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/ec_sys.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/platform_profile.ko 2024-07-17T06:42:32.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/sbs.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/sbshc.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/video.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/acard-ahci.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/ahci_platform.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/ata_generic.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/libahci_platform.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_acpi.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ali.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_amd.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_artop.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_atiixp.ko 2024-07-17T06:42:32.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_atp867x.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cmd640.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cmd64x.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cypress.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_efar.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt366.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt37x.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt3x2n.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt3x3.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_it8213.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_it821x.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_jmicron.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_marvell.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_mpiix.ko 2024-07-17T06:42:32.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_netcell.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ninja32.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ns87410.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ns87415.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_oldpiix.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_opti.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_optidma.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pcmcia.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pdc2027x.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_piccolo.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_rdc.ko 2024-07-17T06:42:32.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sch.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_serverworks.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sil680.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sis.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sl82c105.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_triflex.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_via.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pdc_adma.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_inic162x.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_mv.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_nv.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_promise.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_qstor.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sil.ko 2024-07-17T06:42:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sil24.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sis.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_svw.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sx4.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_uli.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_via.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_vsc.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/auxdisplay 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/auxdisplay/ks0108.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap/regmap-i2c.ko 2024-07-17T06:42:32.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap/regmap-mmio.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/bcma 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/bcma/bcma.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/aoe 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/aoe/aoe.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/brd.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/drbd 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/drbd/drbd.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/floppy.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/loop.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/mtip32xx 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/nbd.ko 2024-07-17T06:42:32.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/null_blk 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/null_blk/null_blk.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/pktcdvd.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/rbd.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/virtio_blk.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkback 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkfront.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/zram 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/zram/zram.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hangcheck-timer.ko 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random 2024-07-17T06:42:32.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/amd-rng.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/intel-rng.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/via-rng.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/virtio-rng.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_si.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/lp.ko 2024-07-17T06:42:32.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/mwave 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/mwave/mwave.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ppdev.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tlclk.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_atmel.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_infineon.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_nsc.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_tis.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/uv_mmtimer.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/virtio_console.ko 2024-07-17T06:42:32.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/p4-clockmod.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/powernow-k8.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/speedstep-lib.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp/ccp.ko 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_common 2024-07-17T06:42:32.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/padlock-aes.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/padlock-sha.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/virtio 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dca 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dca/dca.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/devfreq 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/dw 2024-07-17T06:42:32.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/dw/dw_dmac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/ioat 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/ioat/ioatdma.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/amd64_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/e752x_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/edac_mce_amd.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i3000_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i3200_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i5100_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i5400_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i7300_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i7core_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i82975x_edac.ko 2024-07-17T06:42:32.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/ie31200_edac.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/sb_edac.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/x38_edac.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware/edd.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware/iscsi_ibft.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio/gpio-ich.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio/gpio-viperboard.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-appleir.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-aureal.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-axff.ko 2024-07-17T06:42:32.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-dr.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-elecom.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-elo.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-emsff.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gaff.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gt683r.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gyration.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtek-kbd.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtek-mouse.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtekff.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-hyperv.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-icade.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-keytouch.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-kye.ko 2024-07-17T06:42:32.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-lcpower.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-led.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-lenovo.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-logitech-dj.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-multitouch.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-ortek.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-penmount.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-petalynx.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-picolcd.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-pl.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-plantronics.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-primax.ko 2024-07-17T06:42:32.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-rmi.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-arvo.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-common.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-isku.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-kone.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-konepure.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-lua.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-pyra.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-ryos.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-savu.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-saitek.ko 2024-07-17T06:42:32.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-samsung.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sensor-hub.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sjoy.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sony.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-speedlink.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-steelseries.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sunplus.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-thrustmaster.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-tivo.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-tmff.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-topseed.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-twinhan.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-uclogic.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-waltop.ko 2024-07-17T06:42:32.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-wiimote.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-xinmo.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-zpff.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-zydacron.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/uhid.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/wacom.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_balloon.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_utils.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_vmbus.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/abituguru.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/abituguru3.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/acpi_power_meter.ko 2024-07-17T06:42:32.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ad7414.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ad7418.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adc128d818.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1025.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1026.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1029.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1031.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm9240.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ads7828.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7410.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7411.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7462.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7470.ko 2024-07-17T06:42:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7475.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7x10.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/amc6821.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/applesmc.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asb100.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asc7621.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asus_atk0110.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/atxp1.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/coretemp.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/dme1737.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ds1621.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ds620.ko 2024-07-17T06:42:32.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/emc1403.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/emc6w201.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f71805f.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f71882fg.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f75375s.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/fam15h_power.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/fschmd.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/g760a.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/g762.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/gl518sm.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/gl520sm.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/hwmon-vid.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/i5500_temp.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/i5k_amb.ko 2024-07-17T06:42:32.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ibmaem.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ibmpex.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ina209.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ina2xx.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/it87.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/k10temp.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/k8temp.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lineage-pem.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm63.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm73.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm75.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm77.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm78.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm80.ko 2024-07-17T06:42:32.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm83.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm85.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm87.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm90.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm92.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm93.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95234.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95241.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95245.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc2945.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4151.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4215.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4222.ko 2024-07-17T06:42:32.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4245.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4260.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4261.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max16065.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max1619.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max1668.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max197.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6639.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6650.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6697.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/mcp3021.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6683.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6775-core.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6775.ko 2024-07-17T06:42:32.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct7802.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pc87360.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pc87427.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pcf8591.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max16064.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max34440.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max8688.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-07-17T06:42:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/powr1220.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch5627.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch5636.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch56xx-common.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sht15.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sht21.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/shtc1.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sis5595.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47b397.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47m1.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47m192.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/thmc50.ko 2024-07-17T06:42:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp102.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp103.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp401.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp421.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/via-cputemp.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/via686a.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/vt1211.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/vt8231.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83627ehf.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83627hf.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83781d.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83791d.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83792d.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83793.ko 2024-07-17T06:42:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83795.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83l785ts.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83l786ng.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-07-17T06:42:32.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-i801.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-isch.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-parport.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-07-17T06:42:32.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-via.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-dev.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-mux.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-smbus.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-stub.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_cm.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_core.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_umad.ko 2024-07-17T06:42:32.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/iw_cm.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/rdma_cm.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/cxgb4 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx4 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx5 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mthca 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-07-17T06:42:32.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/ipoib 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/iser 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/isert 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srp 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srpt 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-07-17T06:42:32.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/fm801-gp.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/gameport.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/joydev.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/keyboard 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/keyboard/gpio_keys.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/appletouch.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/bcm5974.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/cyapatp.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/elan_i2c.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/sermouse.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/synaptics_usb.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/vsxxxaa.ko 2024-07-17T06:42:32.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/rmi4 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/rmi4/rmi_core.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/altera_ps2.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/arc_ps2.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/serio_raw.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/sparse-keymap.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-blinkm.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-lm3530.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-lp3944.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-ss4200.ko 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger 2024-07-17T06:42:32.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/bcache 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/bcache/bcache.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-bio-prison.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-cache-smq.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-cache.ko 2024-07-17T06:42:32.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-crypt.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-delay.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-flakey.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-log-userspace.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-multipath.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-queue-length.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-raid.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-round-robin.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-service-time.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-switch.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-thin-pool.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-verity.ko 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/persistent-data 2024-07-17T06:42:32.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid0.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid1.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid10.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid456.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptbase.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptctl.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptfc.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptlan.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptsas.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptscsih.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptspi.ko 2024-07-17T06:42:32.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/lpc_ich.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/lpc_sch.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/mfd-core.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/sm501.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/viperboard.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/vx855.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/wl1273-core.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/altera-stapl 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/apds9802als.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/apds990x.ko 2024-07-17T06:42:32.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/bh1770glc.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/cb710 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/cb710/cb710.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/at24.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/max6875.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/enclosure.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/hpilo.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/isl29003.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/isl29020.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-07-17T06:42:32.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei-me.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei-txe.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-gru 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-gru/gru.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xp.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xpc.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tifm_7xx1.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tifm_core.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tsl2550.ko 2024-07-17T06:42:32.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_balloon.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_vmci 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/bonding 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/bonding/bonding.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/b53 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/b53/b53_common.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/bcm-sf2.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6060.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6xxx 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dummy.ko 2024-07-17T06:42:32.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/eql.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/8390.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/adaptec 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-07-17T06:42:32.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/agere 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/agere/et131x.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/alteon 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/altera 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/alx 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-07-17T06:42:32.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1c 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1e 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/genet 2024-07-17T06:42:32.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade/bna 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-07-17T06:42:32.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco/enic 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink 2024-07-17T06:42:32.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dnet.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex/benet 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ethoc.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/fealnx.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e100.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-07-17T06:42:32.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000e 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/fm10k 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/i40e 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/iavf 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igb 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igbvf 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbe 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-07-17T06:42:32.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbevf 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libeth 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libie 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/jme.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/skge.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4 2024-07-17T06:42:32.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/micrel 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom/myri10ge 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/neterion 2024-07-17T06:42:32.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/nvidia 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/netxen 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rdc 2024-07-17T06:42:32.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/atp.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rocker 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sfc 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/silan 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis/sis190.ko 2024-07-17T06:42:32.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis/sis900.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/cassini.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/niu.ko 2024-07-17T06:42:32.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/sungem.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/tehuti 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ti 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ti/tlan.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/xircom 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-07-17T06:42:32.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/geneve.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/hyperv 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ieee802154 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ieee802154/fakelb.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ifb.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ipvlan 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ipvlan/ipvlan.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/macvlan.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/macvtap.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-07-17T06:42:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-regmap.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mii.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/net_failover.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/netconsole.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/nlmon.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs/pcs-lynx.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/amd.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-07-17T06:42:32.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm7xxx.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm87xx.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/broadcom.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/cicada.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/davicom.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/dp83640.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/et1011c.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/icplus.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/lxt.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/marvell.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/micrel.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/national.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/phylink.ko 2024-07-17T06:42:32.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/qsemi.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/realtek.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/smsc.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/ste10Xp.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/vitesse.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/bsd_comp.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_async.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_deflate.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_generic.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_mppe.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_synctty.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pppoe.ko 2024-07-17T06:42:32.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pppox.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pptp.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip/slhc.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip/slip.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/sungem_phy.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/tap.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_activebackup.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_broadcast.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_random.ko 2024-07-17T06:42:32.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/tun.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/veth.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/virtio_net.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vmxnet3 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vxlan 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vxlan/vxlan.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netback 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netback/xen-netback.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netfront.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ntb 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ntb/ntb.ko 2024-07-17T06:42:32.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host/nvme-core.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host/nvme.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_cs.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_pc.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_serial.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/hotplug 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/pcie 2024-07-17T06:42:32.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/pcie/aer_inject.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/xen-pcifront.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/i82092.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/pd6729.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/yenta_socket.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl/intel 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/acer-wmi.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/acerhdf.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/apple-gmux.ko 2024-07-17T06:42:32.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-laptop.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-wmi.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/classmate-laptop.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/hdaps.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel_ips.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/msi-wmi.ko 2024-07-17T06:42:32.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/mxm-wmi.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/samsung-laptop.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/samsung-q10.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/topstar-laptop.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/toshiba_haps.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/wmi-bmof.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/wmi.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap/intel_rapl_common.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap/intel_rapl_msr.ko 2024-07-17T06:42:32.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps-gpio.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps-ldisc.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps_parport.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/pps_core.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp/ptp.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp/ptp_kvm.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/atl 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/fmpm.ko 2024-07-17T06:42:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-bq32k.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1286.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1307.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1374.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1511.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1553.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1672.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1742.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds2404.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds3232.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-em3027.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-fm3130.ko 2024-07-17T06:42:32.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-isl12022.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-isl1208.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m41t80.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m48t35.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m48t59.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-max6900.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-msm6242.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf2127.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf85063.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8523.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8563.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8583.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rp5c01.ko 2024-07-17T06:42:32.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rs5c372.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rx8025.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rx8581.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-x1205.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-9xxx.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-sas.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-xxxx.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/BusLogic.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/a100u2w.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aacraid 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aacraid/aacraid.ko 2024-07-17T06:42:32.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/advansys.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/am53c974.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/arcmsr 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/atp870u.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/be2iscsi 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bfa 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bfa/bfa.ko 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2fc 2024-07-17T06:42:32.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2i 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ch.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/csiostor 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/csiostor/csiostor.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb3i 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb4i 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/dc395x.ko 2024-07-17T06:42:32.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/dmx3191d.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esas2r 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esas2r/esas2r.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esp_scsi.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe/fcoe.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-07-17T06:42:32.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fnic 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fnic/fnic.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hpsa.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hptiop.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hv_storvsc.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/initio.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ipr.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ips.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/isci 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/isci/isci.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/iscsi_tcp.ko 2024-07-17T06:42:32.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libfc 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libfc/libfc.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libiscsi.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libiscsi_tcp.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libsas 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libsas/libsas.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/lpfc 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/lpfc/lpfc.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-07-17T06:42:32.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mpt3sas 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvsas 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvsas/mvsas.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvumi.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pm8001 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pmcraid.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla1280.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla4xxx 2024-07-17T06:42:32.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/raid_class.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_debug.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_fc.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_sas.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_spi.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_srp.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ses.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/st.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/stex.ko 2024-07-17T06:42:32.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/sym53c8xx_2 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/virtio_scsi.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/vmw_pvscsi.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/wd719x.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ssb 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ssb/ssb.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/iscsi 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/loopback 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/loopback/tcm_loop.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_file.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_iblock.ko 2024-07-17T06:42:32.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_mod.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_pscsi.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_user.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/tcm_fc 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-07-17T06:42:32.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-17T06:42:32.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/ipwireless 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/n_gsm.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/n_hdlc.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/nozomi.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/8250 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/arc_uart.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/jsm 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/jsm/jsm.ko 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs 2024-07-17T06:42:32.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/core 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/core/ufshcd-core.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/host 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_aec.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_cif.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_pci_generic.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_sercos3.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/cdc-acm.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/cdc-wdm.ko 2024-07-17T06:42:32.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/usblp.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/usbtmc.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/host 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/host/sl811-hcd.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image/mdc800.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image/microtek.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/adutux.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/appledisplay.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/emi26.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/emi62.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/ezusb.ko 2024-07-17T06:42:32.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/idmouse.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/iowarrior.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/isight_firmware.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/ldusb.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/legousbtower.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/sisusbvga 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/trancevibrator.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usb3503.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usblcd.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usbsevseg.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/uss720.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/yurex.ko 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/phy 2024-07-17T06:42:32.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/phy/phy-generic.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/aircable.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ark3116.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/belkin_sa.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ch341.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cp210x.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cyberjack.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cypress_m8.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/digi_acceleport.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/empeg.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ftdi_sio.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/garmin_gps.ko 2024-07-17T06:42:32.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/io_edgeport.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/io_ti.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ipaq.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ipw.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ir-usb.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/keyspan.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/keyspan_pda.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/kl5kusb105.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/kobil_sct.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mct_u232.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mos7720.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mos7840.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/navman.ko 2024-07-17T06:42:32.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/omninet.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/opticon.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/option.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/oti6858.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/pl2303.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/qcaux.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/qcserial.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/quatech2.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/safe_serial.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/sierra.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/spcp8x5.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ssu100.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/symbolserial.ko 2024-07-17T06:42:32.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb_debug.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb_wwan.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/visor.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/whiteheat.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/xsens_mt.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/uas.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-alauda.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-cypress.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-datafab.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-07-17T06:42:32.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-freecom.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-isd200.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-karma.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-onetouch.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-realtek.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-sddr09.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-sddr55.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-usbat.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/usb-storage.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/usbip-core.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/usbip-host.ko 2024-07-17T06:42:32.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci/vfio-pci.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/vfio.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_iotlb.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_net.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_scsi.ko 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight 2024-07-17T06:42:32.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/apple_bl.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/lcd.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/platform_lcd.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_balloon.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_mmio.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_ring.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters/ds2482.ko 2024-07-17T06:42:32.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters/ds2490.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_smem.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_therm.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/wire.ko 2024-07-17T06:42:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/alim1535_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/alim7101_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/f71808e_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/hpwdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/i6300esb.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/iTCO_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ib700wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ibmasr.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/it8712f_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/it87_wdt.ko 2024-07-17T06:42:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/machzwd.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/nv_tco.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/pcwd_pci.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/pcwd_usb.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sch311x_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/softdog.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sp5100_tco.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/via_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83877f_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83977f_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/wdt_pci.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/xen_wdt.ko 2024-07-17T06:42:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-acpi-processor.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-evtchn.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-gntalloc.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-gntdev.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-pciback 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-privcmd.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xenfs 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xenfs/xenfs.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/9p 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/9p/9p.ko 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/affs 2024-07-17T06:42:32.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/affs/affs.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/befs 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/befs/befs.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/binfmt_misc.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/btrfs 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/btrfs/btrfs.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cachefiles 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cachefiles/cachefiles.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ceph 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ceph/ceph.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/coda 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/coda/coda.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cramfs 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cramfs/cramfs.ko 2024-07-17T06:42:32.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/dlm 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/dlm/dlm.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ecryptfs 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ecryptfs/ecryptfs.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/f2fs 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/f2fs/f2fs.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/fat.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/msdos.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/vfat.ko 2024-07-17T06:42:32.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse/cuse.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse/fuse.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/gfs2 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/gfs2/gfs2.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfs 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfs/hfs.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfsplus 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfsplus/hfsplus.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/isofs 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/isofs/isofs.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/jfs 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/jfs/jfs.ko 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/lockd 2024-07-17T06:42:32.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/lockd/lockd.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/minix 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/minix/minix.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/netfs 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/netfs/netfs.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/blocklayout 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/filelayout 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/flexfilelayout 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfs.ko 2024-07-17T06:42:32.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfsv3.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfsv4.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common/grace.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common/nfs_acl.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfsd 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfsd/nfsd.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nilfs2 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nilfs2/nilfs2.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-celtic.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-centeuro.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-croatian.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-cyrillic.ko 2024-07-17T06:42:32.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-gaelic.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-greek.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-iceland.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-inuit.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-roman.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-romanian.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-turkish.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1250.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1251.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1255.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp737.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp775.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp850.ko 2024-07-17T06:42:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp852.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp855.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp857.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp860.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp861.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp862.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp863.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp864.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp865.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp866.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp869.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp874.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp932.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp936.ko 2024-07-17T06:42:32.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp949.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp950.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_euc-jp.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-1.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-13.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-14.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-15.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-2.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-3.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-4.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-5.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-6.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-7.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-9.ko 2024-07-17T06:42:32.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-r.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-ru.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-u.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_ucs2_utils.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_utf8.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/cluster 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlm 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlmfs 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-07-17T06:42:32.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/overlayfs 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/overlayfs/overlay.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/pstore 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/pstore/ramoops.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/reiserfs 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/reiserfs/reiserfs.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/romfs 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/romfs/romfs.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/client 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/client/cifs.ko 2024-07-17T06:42:32.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common/cifs_arc4.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common/cifs_md4.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/squashfs 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/squashfs/squashfs.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/sysv 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/sysv/sysv.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/udf 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/udf/udf.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ufs 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ufs/ufs.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/xfs 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/xfs/xfs.ko 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel 2024-07-17T06:42:32.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/rcu 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/rcu/rcutorture.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/torture.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/trace 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/asn1_encoder.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crc-itu-t.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crc8.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libaescfb.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libarc4.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libdes.ko 2024-07-17T06:42:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/dim 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/dim/dimlib.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/libcrc32c.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lru_cache.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4/lz4_compress.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4/lz4hc_compress.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/math 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/math/cordic.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/raid6 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/raid6/raid6_pq.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/reed_solomon 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/reed_solomon/reed_solomon.ko 2024-07-17T06:42:32.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_bm.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_fsm.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_kmp.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/zstd 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/zstd/zstd_compress.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/mm 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/mm/hwpoison-inject.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/6lowpan.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_dest.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_fragment.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_hop.ko 2024-07-17T06:42:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_ipv6.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_mobility.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_routing.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_udp.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/garp.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/mrp.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/p8022.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/psnap.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/stp.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/8021q 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/8021q/8021q.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet.ko 2024-07-17T06:42:32.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_fd.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_rdma.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_virtio.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/batman-adv 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/batman-adv/batman-adv.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/br_netfilter.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/bridge.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_among.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_arp.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-07-17T06:42:32.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_ip.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_limit.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_log.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_mark.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_snat.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_stp.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-07-17T06:42:32.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtables.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ceph 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ceph/libceph.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core/failover.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core/pktgen.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_diag.ko 2024-07-17T06:42:32.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_ipv4.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_ipv6.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dns_resolver 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dns_resolver/dns_resolver.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/dsa_core.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_brcm.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_dsa.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_none.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_trailer.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/6lowpan 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/ieee802154.ko 2024-07-17T06:42:32.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/ieee802154_socket.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ah4.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/esp4.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/fou.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/gre.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/inet_diag.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_gre.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_tunnel.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_vti.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ipcomp.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ipip.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arp_tables.ko 2024-07-17T06:42:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_security.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-07-17T06:42:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_bic.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_dctcp.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_diag.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_highspeed.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_htcp.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_hybla.ko 2024-07-17T06:42:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_illinois.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_lp.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_scalable.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_vegas.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_veno.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_westwood.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_yeah.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tunnel4.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/udp_diag.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/udp_tunnel.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/xfrm4_tunnel.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ah6.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/esp6.ko 2024-07-17T06:42:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/fou6.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_tunnel.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_vti.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ipcomp6.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-07-17T06:42:32.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-07-17T06:42:32.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/sit.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/tunnel6.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/xfrm6_tunnel.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/key 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/key/af_key.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_core.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_debugfs.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_eth.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ip.ko 2024-07-17T06:42:32.666 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ip6.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_netlink.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ppp.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/llc 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/llc/llc.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mac802154 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mac802154/mac802154.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mpls 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mpls/mpls_gso.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-07-17T06:42:32.667 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-07-17T06:42:32.668 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-07-17T06:42:32.669 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conncount.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_h323.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_irc.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_sane.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_sip.ko 2024-07-17T06:42:32.670 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_dup_netdev.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_flow_table.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_flow_table_inet.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_log_syslog.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_amanda.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_ftp.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_irc.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_sip.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_tftp.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_synproxy_core.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_tables.ko 2024-07-17T06:42:32.671 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_acct.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_log.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_osf.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_queue.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_chain_nat.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_compat.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_ct.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_dup_netdev.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_fib.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_fwd_netdev.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_hash.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_limit.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_log.ko 2024-07-17T06:42:32.672 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_masq.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_nat.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_numgen.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_queue.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_quota.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_redir.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_reject.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_reject_inet.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_socket.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_tunnel.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_AUDIT.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CHECKSUM.ko 2024-07-17T06:42:32.673 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CLASSIFY.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CT.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_DSCP.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_HL.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_HMARK.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_IDLETIMER.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_LED.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_LOG.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_MASQUERADE.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NETMAP.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NFLOG.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NFQUEUE.ko 2024-07-17T06:42:32.674 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_RATEEST.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_REDIRECT.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_SECMARK.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TCPMSS.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TEE.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TPROXY.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TRACE.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_addrtype.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_bpf.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cgroup.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cluster.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_comment.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connbytes.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connlabel.ko 2024-07-17T06:42:32.675 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connlimit.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connmark.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_conntrack.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cpu.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_dccp.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_devgroup.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_dscp.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ecn.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_esp.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_hashlimit.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_helper.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_hl.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ipcomp.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_iprange.ko 2024-07-17T06:42:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ipvs.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_l2tp.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_length.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_limit.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_mac.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_mark.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_multiport.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_nat.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_nfacct.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_osf.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_owner.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_physdev.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_pkttype.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_policy.ko 2024-07-17T06:42:32.677 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_quota.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_rateest.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_realm.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_recent.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_sctp.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_set.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_socket.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_state.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_statistic.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_string.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_tcpmss.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_time.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_u32.ko 2024-07-17T06:42:32.678 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netlink 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netlink/netlink_diag.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/nsh 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/nsh/nsh.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/openvswitch.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-geneve.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-gre.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-vxlan.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/packet 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/packet/af_packet_diag.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds_rdma.ko 2024-07-17T06:42:32.679 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds_tcp.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_csum.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_gact.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_mirred.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_nat.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_pedit.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_police.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_simple.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_skbedit.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_vlan.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_basic.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_bpf.ko 2024-07-17T06:42:32.680 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_flow.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_fw.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_route.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_u32.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_cmp.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_ipset.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_meta.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_nbyte.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_text.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_u32.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_choke.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_codel.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_drr.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_fq.ko 2024-07-17T06:42:32.681 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_gred.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_hfsc.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_hhf.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_htb.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_ingress.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_mqprio.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_mqprio_lib.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_multiq.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_netem.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_pie.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_plug.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_prio.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_qfq.ko 2024-07-17T06:42:32.682 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_red.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_sfb.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_sfq.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_tbf.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_teql.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp/sctp.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp/sctp_diag.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/sunrpc.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/xprtrdma 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-07-17T06:42:32.683 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc/tipc.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc/tipc_diag.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/unix 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/unix/unix_diag.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock_diag.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock_loopback.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/xfrm 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/xfrm/xfrm_ipcomp.ko 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security 2024-07-17T06:42:32.684 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/encrypted-keys 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/trusted-keys 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/trusted-keys/trusted.ko 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.alias 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.alias.bin 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.alias.bin 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.bin 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.modinfo 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep.bin 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.devname 2024-07-17T06:42:32.685 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.order 2024-07-17T06:42:32.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.softdep 2024-07-17T06:42:32.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.symbols 2024-07-17T06:42:32.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.symbols.bin 2024-07-17T06:42:32.686 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/vmlinuz 2024-07-17T06:42:32.690 DEBUG:teuthology.task.kernel:get_image_version: 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.690 DEBUG:teuthology.orchestra.run.smithi175:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout:postinstall scriptlet (using /bin/sh): 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/bin/kernel-install add 6.10.0-rc7-gc30fd5683ea8 /lib/modules/6.10.0-rc7-gc30fd5683ea8/vmlinuz 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout:for file in vmlinuz System.map config; do 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout: if ! cmp --silent "/lib/modules/6.10.0-rc7-gc30fd5683ea8/${file}" "/boot/${file}-6.10.0-rc7-gc30fd5683ea8"; then 2024-07-17T06:42:32.714 INFO:teuthology.orchestra.run.smithi175.stdout: cp "/lib/modules/6.10.0-rc7-gc30fd5683ea8/${file}" "/boot/${file}-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout: fi 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -d "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" ] && \ 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout: ! diff -rq "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" >/dev/null 2>&1; then 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout: rm -rf "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout: cp -r "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:if [ ! -e "/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep" ]; then 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/sbin/depmod 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:preuninstall scriptlet (using /bin/sh): 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-17T06:42:32.715 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-install remove 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.716 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-07-17T06:42:32.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2024-07-17T06:42:32.788 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-17T06:42:32.789 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-07-17T06:42:32.790 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:42:32.790 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-07-17T06:42:32.790 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-17T06:42:32.791 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:32.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-17T06:42:33.335 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-07-17T06:42:34.322 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-07-17T06:42:34.322 DEBUG:teuthology.orchestra.run.smithi044:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-07-17T06:42:34.395 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/System.map-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.395 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/config-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.395 INFO:teuthology.orchestra.run.smithi044.stdout:/boot/vmlinuz-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/System.map 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/config 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-07-17T06:42:34.396 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crc32c-intel.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-avx2.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-07-17T06:42:34.397 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu/mce 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm-amd.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm-intel.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/arch/x86/kvm/kvm.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ansi_cprng.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/anubis.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/arc4.ko 2024-07-17T06:42:34.398 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_memcpy.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_pq.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_raid6_recov.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_tx.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/async_xor.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/async_tx/raid6test.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/authenc.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/authencesn.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blake2b_generic.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blowfish_common.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/blowfish_generic.ko 2024-07-17T06:42:34.399 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/camellia_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast5_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast6_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cast_common.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ccm.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cmac.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crc32_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crypto_engine.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/crypto_user.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/cts.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/des_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/dh_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ecc.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/ecdh_generic.ko 2024-07-17T06:42:34.400 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/echainiv.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/essiv.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/fcrypt.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/khazad.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/lz4.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/lz4hc.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/md4.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/michael_mic.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/pcbc.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/pcrypt.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/rmd160.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/seed.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/serpent_generic.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/tcrypt.ko 2024-07-17T06:42:34.401 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/tea.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/twofish_common.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/twofish_generic.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/vmac.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/wp512.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xcbc.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xor.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/crypto/xxhash_generic.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/acpi_ipmi.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/acpi_pad.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/ec_sys.ko 2024-07-17T06:42:34.402 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/platform_profile.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/sbs.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/sbshc.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/acpi/video.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/acard-ahci.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/ahci_platform.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/ata_generic.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/libahci_platform.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_acpi.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ali.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_amd.ko 2024-07-17T06:42:34.403 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_artop.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_atiixp.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_atp867x.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cmd640.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cmd64x.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_cypress.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_efar.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt366.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt37x.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt3x2n.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_hpt3x3.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_it8213.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_it821x.ko 2024-07-17T06:42:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_jmicron.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_marvell.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_mpiix.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_netcell.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ninja32.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ns87410.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_ns87415.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_oldpiix.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_opti.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_optidma.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pcmcia.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pdc2027x.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-07-17T06:42:34.405 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_piccolo.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_rdc.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sch.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_serverworks.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sil680.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sis.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_sl82c105.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_triflex.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pata_via.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/pdc_adma.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_inic162x.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_mv.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_nv.ko 2024-07-17T06:42:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_promise.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_qstor.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sil.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sil24.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sis.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_svw.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_sx4.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_uli.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_via.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ata/sata_vsc.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/auxdisplay 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/auxdisplay/ks0108.ko 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base 2024-07-17T06:42:34.407 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap/regmap-i2c.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/base/regmap/regmap-mmio.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/bcma 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/bcma/bcma.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/aoe 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/aoe/aoe.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/brd.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/drbd 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/drbd/drbd.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/floppy.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/loop.ko 2024-07-17T06:42:34.408 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/mtip32xx 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/nbd.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/null_blk 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/null_blk/null_blk.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/pktcdvd.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/rbd.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/virtio_blk.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkback 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/xen-blkfront.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/zram 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/block/zram/zram.ko 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char 2024-07-17T06:42:34.409 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hangcheck-timer.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/amd-rng.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/intel-rng.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/via-rng.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/hw_random/virtio-rng.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_si.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-07-17T06:42:34.410 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/lp.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/mwave 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/mwave/mwave.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/ppdev.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tlclk.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_atmel.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_infineon.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_nsc.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_tis.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-07-17T06:42:34.411 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/uv_mmtimer.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/char/virtio_console.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/p4-clockmod.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/powernow-k8.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/cpufreq/speedstep-lib.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/ccp/ccp.ko 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel 2024-07-17T06:42:34.412 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_common 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/padlock-aes.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/padlock-sha.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/virtio 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dca 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dca/dca.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/devfreq 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-07-17T06:42:34.413 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/dw 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/dw/dw_dmac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/ioat 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/dma/ioat/ioatdma.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/amd64_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/e752x_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/edac_mce_amd.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i3000_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i3200_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i5100_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i5400_edac.ko 2024-07-17T06:42:34.414 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i7300_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i7core_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/i82975x_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/ie31200_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/sb_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/edac/x38_edac.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware/edd.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/firmware/iscsi_ibft.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio/gpio-ich.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/gpio/gpio-viperboard.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-appleir.ko 2024-07-17T06:42:34.415 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-aureal.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-axff.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-dr.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-elecom.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-elo.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-emsff.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gaff.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gt683r.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-gyration.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtek-kbd.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtek-mouse.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-holtekff.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-hyperv.ko 2024-07-17T06:42:34.416 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-icade.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-keytouch.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-kye.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-lcpower.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-led.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-lenovo.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-logitech-dj.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-multitouch.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-ortek.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-penmount.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-petalynx.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-picolcd.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-pl.ko 2024-07-17T06:42:34.417 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-plantronics.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-primax.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-rmi.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-arvo.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-common.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-isku.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-kone.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-konepure.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-lua.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-pyra.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-ryos.ko 2024-07-17T06:42:34.418 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat-savu.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-roccat.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-saitek.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-samsung.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sensor-hub.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sjoy.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sony.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-speedlink.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-steelseries.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-sunplus.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-thrustmaster.ko 2024-07-17T06:42:34.419 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-tivo.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-tmff.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-topseed.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-twinhan.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-uclogic.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-waltop.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-wiimote.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-xinmo.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-zpff.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/hid-zydacron.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/uhid.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hid/wacom.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_balloon.ko 2024-07-17T06:42:34.420 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_utils.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hv/hv_vmbus.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/abituguru.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/abituguru3.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/acpi_power_meter.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ad7414.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ad7418.ko 2024-07-17T06:42:34.421 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adc128d818.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1025.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1026.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1029.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm1031.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adm9240.ko 2024-07-17T06:42:34.422 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ads7828.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7410.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7411.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7462.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7470.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7475.ko 2024-07-17T06:42:34.423 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/adt7x10.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/amc6821.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/applesmc.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asb100.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asc7621.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/asus_atk0110.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/atxp1.ko 2024-07-17T06:42:34.424 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/coretemp.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/dme1737.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ds1621.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ds620.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/emc1403.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/emc6w201.ko 2024-07-17T06:42:34.425 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f71805f.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f71882fg.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/f75375s.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/fam15h_power.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/fschmd.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/g760a.ko 2024-07-17T06:42:34.426 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/g762.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/gl518sm.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/gl520sm.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/hwmon-vid.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/i5500_temp.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/i5k_amb.ko 2024-07-17T06:42:34.427 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ibmaem.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ibmpex.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ina209.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ina2xx.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/it87.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/k10temp.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/k8temp.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lineage-pem.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm63.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm73.ko 2024-07-17T06:42:34.428 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm75.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm77.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm78.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm80.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm83.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm85.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm87.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm90.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm92.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm93.ko 2024-07-17T06:42:34.429 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95234.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95241.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/lm95245.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc2945.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4151.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4215.ko 2024-07-17T06:42:34.430 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4222.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4245.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4260.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/ltc4261.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max16065.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max1619.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max1668.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max197.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6639.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6650.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/max6697.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/mcp3021.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6683.ko 2024-07-17T06:42:34.431 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6775-core.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct6775.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/nct7802.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pc87360.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pc87427.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pcf8591.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max16064.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max34440.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/max8688.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-07-17T06:42:34.432 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/powr1220.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch5627.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch5636.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sch56xx-common.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sht15.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sht21.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/shtc1.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/sis5595.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47b397.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47m1.ko 2024-07-17T06:42:34.433 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/smsc47m192.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/thmc50.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp102.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp103.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp401.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/tmp421.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/via-cputemp.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/via686a.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/vt1211.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/vt8231.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83627ehf.ko 2024-07-17T06:42:34.434 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83627hf.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83781d.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83791d.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83792d.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83793.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83795.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83l785ts.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/hwmon/w83l786ng.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-07-17T06:42:34.435 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-i801.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-isch.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-parport.ko 2024-07-17T06:42:34.436 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-via.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-dev.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-mux.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-smbus.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/i2c/i2c-stub.ko 2024-07-17T06:42:34.437 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_cm.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_core.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_umad.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/iw_cm.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/rdma_cm.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/cxgb4 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-07-17T06:42:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx4 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx5 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mthca 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/ipoib 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/iser 2024-07-17T06:42:34.439 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/isert 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srp 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srpt 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/fm801-gp.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/gameport/gameport.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/joydev.ko 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/keyboard 2024-07-17T06:42:34.440 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/keyboard/gpio_keys.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/appletouch.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/bcm5974.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/cyapatp.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/elan_i2c.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/sermouse.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/synaptics_usb.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/mouse/vsxxxaa.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/rmi4 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/rmi4/rmi_core.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/altera_ps2.ko 2024-07-17T06:42:34.441 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/arc_ps2.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/serio/serio_raw.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/input/sparse-keymap.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-blinkm.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-lm3530.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-lp3944.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/leds-ss4200.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-07-17T06:42:34.442 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/bcache 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/bcache/bcache.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-bio-prison.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-cache-smq.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-cache.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-crypt.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-delay.ko 2024-07-17T06:42:34.443 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-flakey.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-log-userspace.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-multipath.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-queue-length.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-raid.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-round-robin.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-service-time.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-switch.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-thin-pool.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/dm-verity.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/persistent-data 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid0.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid1.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid10.ko 2024-07-17T06:42:34.444 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/md/raid456.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptbase.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptctl.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptfc.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptlan.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptsas.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptscsih.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/message/fusion/mptspi.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/lpc_ich.ko 2024-07-17T06:42:34.445 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/lpc_sch.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/mfd-core.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/sm501.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/viperboard.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/vx855.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/mfd/wl1273-core.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/altera-stapl 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/apds9802als.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/apds990x.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/bh1770glc.ko 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/cb710 2024-07-17T06:42:34.446 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/cb710/cb710.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/at24.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/eeprom/max6875.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/enclosure.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/hpilo.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/isl29003.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/isl29020.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei 2024-07-17T06:42:34.447 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei-me.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei-txe.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/mei/mei.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-gru 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-gru/gru.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xp.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xpc.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tifm_7xx1.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tifm_core.ko 2024-07-17T06:42:34.448 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/tsl2550.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_balloon.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_vmci 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/bonding 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/bonding/bonding.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/b53 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/b53/b53_common.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/bcm-sf2.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6060.ko 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6xxx 2024-07-17T06:42:34.449 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/dummy.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/eql.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/8390.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-07-17T06:42:34.450 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/adaptec 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/agere 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/agere/et131x.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/alteon 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/altera 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros 2024-07-17T06:42:34.451 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/alx 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1c 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1e 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-07-17T06:42:34.452 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-07-17T06:42:34.453 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-07-17T06:42:34.453 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-07-17T06:42:34.453 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-07-17T06:42:34.453 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/genet 2024-07-17T06:42:34.453 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade/bna 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio 2024-07-17T06:42:34.454 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-07-17T06:42:34.455 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco/enic 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-07-17T06:42:34.456 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-07-17T06:42:34.457 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/dnet.ko 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex/benet 2024-07-17T06:42:34.458 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ethoc.ko 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/fealnx.ko 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e100.ko 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000 2024-07-17T06:42:34.459 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000e 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/fm10k 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/i40e 2024-07-17T06:42:34.460 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/iavf 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igb 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igbvf 2024-07-17T06:42:34.461 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbe 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbevf 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libeth 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libie 2024-07-17T06:42:34.462 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-07-17T06:42:34.463 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/jme.ko 2024-07-17T06:42:34.463 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell 2024-07-17T06:42:34.463 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-07-17T06:42:34.467 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/skge.ko 2024-07-17T06:42:34.467 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-07-17T06:42:34.467 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox 2024-07-17T06:42:34.467 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/micrel 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom/myri10ge 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-07-17T06:42:34.468 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/neterion 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/nvidia 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/netxen 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-07-17T06:42:34.469 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rdc 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/atp.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rocker 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sfc 2024-07-17T06:42:34.470 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/silan 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis/sis190.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sis/sis900.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-07-17T06:42:34.471 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/cassini.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/niu.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/sungem.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/tehuti 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ti 2024-07-17T06:42:34.472 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/ti/tlan.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/xircom 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/geneve.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/hyperv 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ieee802154 2024-07-17T06:42:34.473 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ieee802154/fakelb.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ifb.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ipvlan 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ipvlan/ipvlan.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/macvlan.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/macvtap.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mdio/mdio-regmap.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/mii.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/net_failover.ko 2024-07-17T06:42:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/netconsole.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/nlmon.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs/pcs-lynx.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/amd.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm7xxx.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/bcm87xx.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/broadcom.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/cicada.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/davicom.ko 2024-07-17T06:42:34.475 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/dp83640.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/et1011c.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/icplus.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/lxt.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/marvell.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/micrel.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/national.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/phylink.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/qsemi.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/realtek.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/smsc.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/ste10Xp.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/phy/vitesse.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/bsd_comp.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_async.ko 2024-07-17T06:42:34.476 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_deflate.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_generic.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_mppe.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/ppp_synctty.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pppoe.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pppox.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/ppp/pptp.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip/slhc.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/slip/slip.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/sungem_phy.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/tap.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_activebackup.ko 2024-07-17T06:42:34.477 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_broadcast.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_random.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/tun.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/veth.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/virtio_net.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vmxnet3 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vxlan 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/vxlan/vxlan.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netback 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netback/xen-netback.ko 2024-07-17T06:42:34.478 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/net/xen-netfront.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ntb 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ntb/ntb.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host/nvme-core.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/nvme/host/nvme.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_cs.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_pc.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/parport/parport_serial.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/hotplug 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-07-17T06:42:34.479 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/pcie 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/pcie/aer_inject.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pci/xen-pcifront.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/i82092.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/pd6729.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pcmcia/yenta_socket.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl/intel 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/acer-wmi.ko 2024-07-17T06:42:34.480 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/acerhdf.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/apple-gmux.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-laptop.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/asus-wmi.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/classmate-laptop.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/hdaps.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-07-17T06:42:34.481 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/intel_ips.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/msi-wmi.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/mxm-wmi.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/samsung-laptop.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/samsung-q10.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/topstar-laptop.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/toshiba_haps.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/wmi-bmof.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/platform/x86/wmi.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap/intel_rapl_common.ko 2024-07-17T06:42:34.482 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/powercap/intel_rapl_msr.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps-gpio.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps-ldisc.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/clients/pps_parport.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/pps/pps_core.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp/ptp.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ptp/ptp_kvm.ko 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/atl 2024-07-17T06:42:34.483 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ras/amd/fmpm.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-bq32k.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1286.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1307.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1374.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1511.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1553.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1672.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds1742.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds2404.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-ds3232.ko 2024-07-17T06:42:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-em3027.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-fm3130.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-isl12022.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-isl1208.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m41t80.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m48t35.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-m48t59.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-max6900.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-msm6242.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf2127.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf85063.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8523.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8563.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-pcf8583.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rp5c01.ko 2024-07-17T06:42:34.485 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rs5c372.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rx8025.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-rx8581.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/rtc/rtc-x1205.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-9xxx.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-sas.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/3w-xxxx.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/BusLogic.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/a100u2w.ko 2024-07-17T06:42:34.486 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aacraid 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aacraid/aacraid.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/advansys.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/am53c974.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/arcmsr 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/atp870u.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/be2iscsi 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bfa 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bfa/bfa.ko 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2fc 2024-07-17T06:42:34.487 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2i 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ch.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/csiostor 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/csiostor/csiostor.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb3i 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb4i 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/dc395x.ko 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler 2024-07-17T06:42:34.488 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/dmx3191d.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esas2r 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esas2r/esas2r.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/esp_scsi.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe/fcoe.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fnic 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/fnic/fnic.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hpsa.ko 2024-07-17T06:42:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hptiop.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/hv_storvsc.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/initio.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ipr.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ips.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/isci 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/isci/isci.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/iscsi_tcp.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libfc 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libfc/libfc.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libiscsi.ko 2024-07-17T06:42:34.490 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libiscsi_tcp.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libsas 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/libsas/libsas.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/lpfc 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/lpfc/lpfc.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mpt3sas 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvsas 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvsas/mvsas.ko 2024-07-17T06:42:34.491 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/mvumi.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pm8001 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/pmcraid.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla1280.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla4xxx 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/raid_class.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_debug.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_fc.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-07-17T06:42:34.492 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_sas.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_spi.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/scsi_transport_srp.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/ses.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/st.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/stex.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/sym53c8xx_2 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/virtio_scsi.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/vmw_pvscsi.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/scsi/wd719x.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ssb 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ssb/ssb.ko 2024-07-17T06:42:34.493 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/iscsi 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/loopback 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/loopback/tcm_loop.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_file.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_iblock.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_mod.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_pscsi.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/target_core_user.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/tcm_fc 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal 2024-07-17T06:42:34.494 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-07-17T06:42:34.495 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/ipwireless 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/n_gsm.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/n_hdlc.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/nozomi.ko 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial 2024-07-17T06:42:34.496 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/8250 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/arc_uart.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/jsm 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/tty/serial/jsm/jsm.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/core 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/core/ufshcd-core.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/host 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_aec.ko 2024-07-17T06:42:34.497 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_cif.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_pci_generic.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/uio/uio_sercos3.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/cdc-acm.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/cdc-wdm.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/usblp.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/class/usbtmc.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/host 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/host/sl811-hcd.ko 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image 2024-07-17T06:42:34.498 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image/mdc800.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/image/microtek.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/adutux.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/appledisplay.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/emi26.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/emi62.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/ezusb.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/idmouse.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/iowarrior.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/isight_firmware.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/ldusb.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/legousbtower.ko 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/sisusbvga 2024-07-17T06:42:34.499 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/trancevibrator.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usb3503.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usblcd.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/usbsevseg.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/uss720.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/misc/yurex.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/phy 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/phy/phy-generic.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/aircable.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ark3116.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/belkin_sa.ko 2024-07-17T06:42:34.500 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ch341.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cp210x.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cyberjack.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/cypress_m8.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/digi_acceleport.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/empeg.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ftdi_sio.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/garmin_gps.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/io_edgeport.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/io_ti.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ipaq.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ipw.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ir-usb.ko 2024-07-17T06:42:34.501 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/keyspan.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/keyspan_pda.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/kl5kusb105.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/kobil_sct.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mct_u232.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mos7720.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/mos7840.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/navman.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/omninet.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/opticon.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/option.ko 2024-07-17T06:42:34.502 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/oti6858.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/pl2303.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/qcaux.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/qcserial.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/quatech2.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/safe_serial.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/sierra.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/spcp8x5.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ssu100.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/symbolserial.ko 2024-07-17T06:42:34.503 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb_debug.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/usb_wwan.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/visor.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/whiteheat.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/serial/xsens_mt.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/uas.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-alauda.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-cypress.ko 2024-07-17T06:42:34.504 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-datafab.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-freecom.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-isd200.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-karma.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-onetouch.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-realtek.ko 2024-07-17T06:42:34.505 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-sddr09.ko 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-sddr55.ko 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/ums-usbat.ko 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/storage/usb-storage.ko 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/usbip-core.ko 2024-07-17T06:42:34.506 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/usbip-host.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/pci/vfio-pci.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/vfio.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_iotlb.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_net.ko 2024-07-17T06:42:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/vhost/vhost_scsi.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/apple_bl.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/lcd.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/video/backlight/platform_lcd.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_balloon.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_mmio.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/virtio/virtio_ring.ko 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1 2024-07-17T06:42:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters/ds2482.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/masters/ds2490.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_smem.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/slaves/w1_therm.ko 2024-07-17T06:42:34.509 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/w1/wire.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/alim1535_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/alim7101_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/f71808e_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/hpwdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/i6300esb.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/iTCO_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ib700wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ibmasr.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/it8712f_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/it87_wdt.ko 2024-07-17T06:42:34.510 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/machzwd.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/nv_tco.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/pcwd_pci.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/pcwd_usb.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sch311x_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/softdog.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/sp5100_tco.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/via_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83877f_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/w83977f_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/wdt_pci.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/watchdog/xen_wdt.ko 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen 2024-07-17T06:42:34.511 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-acpi-processor.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-evtchn.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-gntalloc.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-gntdev.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-pciback 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xen-privcmd.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xenfs 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/drivers/xen/xenfs/xenfs.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/9p 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/9p/9p.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/affs 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/affs/affs.ko 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/befs 2024-07-17T06:42:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/befs/befs.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/binfmt_misc.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/btrfs 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/btrfs/btrfs.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cachefiles 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cachefiles/cachefiles.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ceph 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ceph/ceph.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/coda 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/coda/coda.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cramfs 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/cramfs/cramfs.ko 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/dlm 2024-07-17T06:42:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/dlm/dlm.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ecryptfs 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ecryptfs/ecryptfs.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/f2fs 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/f2fs/f2fs.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/fat.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/msdos.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fat/vfat.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse/cuse.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/fuse/fuse.ko 2024-07-17T06:42:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/gfs2 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/gfs2/gfs2.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfs 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfs/hfs.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfsplus 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/hfsplus/hfsplus.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/isofs 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/isofs/isofs.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/jfs 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/jfs/jfs.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/lockd 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/lockd/lockd.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/minix 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/minix/minix.ko 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/netfs 2024-07-17T06:42:34.515 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/netfs/netfs.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/blocklayout 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/filelayout 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/flexfilelayout 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfs.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfsv3.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs/nfsv4.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common/grace.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfs_common/nfs_acl.ko 2024-07-17T06:42:34.516 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfsd 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nfsd/nfsd.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nilfs2 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nilfs2/nilfs2.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-celtic.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-centeuro.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-croatian.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-cyrillic.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-gaelic.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-greek.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-iceland.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-inuit.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-roman.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-romanian.ko 2024-07-17T06:42:34.517 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/mac-turkish.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1250.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1251.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp1255.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp737.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp775.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp850.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp852.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp855.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp857.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp860.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp861.ko 2024-07-17T06:42:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp862.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp863.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp864.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp865.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp866.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp869.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp874.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp932.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp936.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp949.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_cp950.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_euc-jp.ko 2024-07-17T06:42:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-1.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-13.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-14.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-15.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-2.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-3.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-4.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-5.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-6.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-7.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_iso8859-9.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-r.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-ru.ko 2024-07-17T06:42:34.520 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_koi8-u.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_ucs2_utils.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/nls/nls_utf8.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/cluster 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlm 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlmfs 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/overlayfs 2024-07-17T06:42:34.521 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/overlayfs/overlay.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/pstore 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/pstore/ramoops.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/reiserfs 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/reiserfs/reiserfs.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/romfs 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/romfs/romfs.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/client 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/client/cifs.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common/cifs_arc4.ko 2024-07-17T06:42:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/smb/common/cifs_md4.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/squashfs 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/squashfs/squashfs.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/sysv 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/sysv/sysv.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/udf 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/udf/udf.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ufs 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/ufs/ufs.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/xfs 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/fs/xfs/xfs.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/rcu 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/rcu/rcutorture.ko 2024-07-17T06:42:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/torture.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/trace 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/asn1_encoder.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crc-itu-t.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crc8.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libaescfb.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libarc4.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/crypto/libdes.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/dim 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/dim/dimlib.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/libcrc32c.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lru_cache.ko 2024-07-17T06:42:34.524 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4/lz4_compress.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/lz4/lz4hc_compress.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/math 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/math/cordic.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/raid6 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/raid6/raid6_pq.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/reed_solomon 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/reed_solomon/reed_solomon.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_bm.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_fsm.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/ts_kmp.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/zstd 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/lib/zstd/zstd_compress.ko 2024-07-17T06:42:34.525 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/mm 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/mm/hwpoison-inject.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/6lowpan.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_dest.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_fragment.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_hop.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_ipv6.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_mobility.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_routing.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/6lowpan/nhc_udp.ko 2024-07-17T06:42:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/garp.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/mrp.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/p8022.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/psnap.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/802/stp.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/8021q 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/8021q/8021q.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_fd.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_rdma.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/9p/9pnet_virtio.ko 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/batman-adv 2024-07-17T06:42:34.527 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/batman-adv/batman-adv.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/br_netfilter.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/bridge.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_among.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_arp.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_ip.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_limit.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_log.ko 2024-07-17T06:42:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_mark.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_snat.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_stp.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/ebtables.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-07-17T06:42:34.529 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ceph 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ceph/libceph.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core/failover.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/core/pktgen.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_diag.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_ipv4.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dccp/dccp_ipv6.ko 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dns_resolver 2024-07-17T06:42:34.530 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dns_resolver/dns_resolver.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/dsa_core.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_brcm.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_dsa.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_none.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/dsa/tag_trailer.ko 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154 2024-07-17T06:42:34.531 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/6lowpan 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/ieee802154.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ieee802154/ieee802154_socket.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ah4.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/esp4.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/fou.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/gre.ko 2024-07-17T06:42:34.532 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/inet_diag.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_gre.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_tunnel.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ip_vti.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ipcomp.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/ipip.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arp_tables.ko 2024-07-17T06:42:34.533 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-07-17T06:42:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/iptable_security.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-07-17T06:42:34.535 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_bic.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_dctcp.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_diag.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_highspeed.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_htcp.ko 2024-07-17T06:42:34.536 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_hybla.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_illinois.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_lp.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_scalable.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_vegas.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_veno.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_westwood.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tcp_yeah.ko 2024-07-17T06:42:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/tunnel4.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/udp_diag.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/udp_tunnel.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv4/xfrm4_tunnel.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ah6.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/esp6.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/fou6.ko 2024-07-17T06:42:34.538 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_tunnel.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ip6_vti.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/ipcomp6.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-07-17T06:42:34.539 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-07-17T06:42:34.540 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-07-17T06:42:34.541 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/sit.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/tunnel6.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/ipv6/xfrm6_tunnel.ko 2024-07-17T06:42:34.542 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/key 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/key/af_key.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_core.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_debugfs.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_eth.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ip.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ip6.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_netlink.ko 2024-07-17T06:42:34.543 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/l2tp/l2tp_ppp.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/llc 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/llc/llc.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mac802154 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mac802154/mac802154.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mpls 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/mpls/mpls_gso.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-07-17T06:42:34.544 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-07-17T06:42:34.545 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-07-17T06:42:34.546 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conncount.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_h323.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_irc.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_sane.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_sip.ko 2024-07-17T06:42:34.547 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_dup_netdev.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_flow_table.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_flow_table_inet.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_log_syslog.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_amanda.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_ftp.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_irc.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_sip.ko 2024-07-17T06:42:34.548 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_nat_tftp.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_synproxy_core.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nf_tables.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_acct.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_log.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_osf.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nfnetlink_queue.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_chain_nat.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_compat.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_ct.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_dup_netdev.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_fib.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_fwd_netdev.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_hash.ko 2024-07-17T06:42:34.549 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_limit.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_log.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_masq.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_nat.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_numgen.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_queue.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_quota.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_redir.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_reject.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_reject_inet.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_socket.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/nft_tunnel.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_AUDIT.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CHECKSUM.ko 2024-07-17T06:42:34.550 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CLASSIFY.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_CT.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_DSCP.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_HL.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_HMARK.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_IDLETIMER.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_LED.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_LOG.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_MASQUERADE.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NETMAP.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NFLOG.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_NFQUEUE.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_RATEEST.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_REDIRECT.ko 2024-07-17T06:42:34.551 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_SECMARK.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TCPMSS.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TEE.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TPROXY.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_TRACE.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_addrtype.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_bpf.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cgroup.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cluster.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_comment.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connbytes.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connlabel.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connlimit.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_connmark.ko 2024-07-17T06:42:34.552 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_conntrack.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_cpu.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_dccp.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_devgroup.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_dscp.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ecn.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_esp.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_hashlimit.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_helper.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_hl.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ipcomp.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_iprange.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_ipvs.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_l2tp.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_length.ko 2024-07-17T06:42:34.553 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_limit.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_mac.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_mark.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_multiport.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_nat.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_nfacct.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_osf.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_owner.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_physdev.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_pkttype.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_policy.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_quota.ko 2024-07-17T06:42:34.554 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_rateest.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_realm.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_recent.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_sctp.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_set.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_socket.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_state.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_statistic.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_string.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_tcpmss.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_time.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netfilter/xt_u32.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netlink 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/netlink/netlink_diag.ko 2024-07-17T06:42:34.555 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/nsh 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/nsh/nsh.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/openvswitch.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-geneve.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-gre.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/openvswitch/vport-vxlan.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/packet 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/packet/af_packet_diag.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds_rdma.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/rds/rds_tcp.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_csum.ko 2024-07-17T06:42:34.556 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_gact.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_mirred.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_nat.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_pedit.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_police.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_simple.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_skbedit.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/act_vlan.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_basic.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_bpf.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_flow.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_fw.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_route.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/cls_u32.ko 2024-07-17T06:42:34.557 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_cmp.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_ipset.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_meta.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_nbyte.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_text.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/em_u32.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_choke.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_codel.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_drr.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_fq.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_gred.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_hfsc.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_hhf.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_htb.ko 2024-07-17T06:42:34.558 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_ingress.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_mqprio.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_mqprio_lib.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_multiq.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_netem.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_pie.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_plug.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_prio.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_qfq.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_red.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_sfb.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_sfq.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_tbf.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sched/sch_teql.ko 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp 2024-07-17T06:42:34.559 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp/sctp.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sctp/sctp_diag.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/sunrpc.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/xprtrdma 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc/tipc.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/tipc/tipc_diag.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/unix 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/unix/unix_diag.ko 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock 2024-07-17T06:42:34.560 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock_diag.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/vmw_vsock/vsock_loopback.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/xfrm 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/net/xfrm/xfrm_ipcomp.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/encrypted-keys 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/trusted-keys 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/kernel/security/keys/trusted-keys/trusted.ko 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.alias 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.alias.bin 2024-07-17T06:42:34.561 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.alias.bin 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.bin 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.builtin.modinfo 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep.bin 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.devname 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.order 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.softdep 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.symbols 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.symbols.bin 2024-07-17T06:42:34.562 INFO:teuthology.orchestra.run.smithi044.stdout:/lib/modules/6.10.0-rc7-gc30fd5683ea8/vmlinuz 2024-07-17T06:42:34.566 DEBUG:teuthology.task.kernel:get_image_version: 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.566 DEBUG:teuthology.orchestra.run.smithi044:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-07-17T06:42:34.589 INFO:teuthology.orchestra.run.smithi044.stdout:postinstall scriptlet (using /bin/sh): 2024-07-17T06:42:34.589 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-17T06:42:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: /usr/bin/kernel-install add 6.10.0-rc7-gc30fd5683ea8 /lib/modules/6.10.0-rc7-gc30fd5683ea8/vmlinuz 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout:for file in vmlinuz System.map config; do 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: if ! cmp --silent "/lib/modules/6.10.0-rc7-gc30fd5683ea8/${file}" "/boot/${file}-6.10.0-rc7-gc30fd5683ea8"; then 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: cp "/lib/modules/6.10.0-rc7-gc30fd5683ea8/${file}" "/boot/${file}-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: fi 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout:done 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -d "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" ] && \ 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: ! diff -rq "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" >/dev/null 2>&1; then 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: rm -rf "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: cp -r "/lib/modules/6.10.0-rc7-gc30fd5683ea8/dtb" "/boot/dtb-6.10.0-rc7-gc30fd5683ea8" 2024-07-17T06:42:34.590 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:if [ ! -e "/lib/modules/6.10.0-rc7-gc30fd5683ea8/modules.dep" ]; then 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: /usr/sbin/depmod 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:preuninstall scriptlet (using /bin/sh): 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-install remove 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.591 INFO:teuthology.orchestra.run.smithi044.stdout:fi 2024-07-17T06:42:34.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-07-17T06:42:34.665 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-07-17T06:42:34.665 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-07-17T06:42:34.665 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-07-17T06:42:34.665 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 68.el9 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8264254 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-17T06:42:34.666 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:42:34.667 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-07-17T06:42:34.668 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-17T06:42:34.668 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:34.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-17T06:42:34.818 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-17T06:42:34.848 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-07-17T06:42:34.850 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-07-17T06:42:34.870 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.gK7BNlSKaX 2024-07-17T06:42:34.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gK7BNlSKaX 2024-07-17T06:42:34.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.gK7BNlSKaX 2024-07-17T06:42:35.097 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.gK7BNlSKaX is 6KB 2024-07-17T06:42:35.146 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.gK7BNlSKaX 2024-07-17T06:42:35.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-07-17T06:42:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-07-17T06:42:35.228 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-17T06:42:35.228 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-17T06:42:35.229 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-17T06:42:35.229 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.10.0-rc7-gc30fd5683ea8.conf 2024-07-17T06:42:35.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:35.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-07-17T06:42:36.573 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-17T06:42:36.589 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-07-17T06:42:36.594 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-07-17T06:42:36.609 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.a8Cl2mfzGw 2024-07-17T06:42:36.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.a8Cl2mfzGw 2024-07-17T06:42:36.675 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.a8Cl2mfzGw 2024-07-17T06:42:36.815 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.a8Cl2mfzGw is 6KB 2024-07-17T06:42:36.863 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.a8Cl2mfzGw 2024-07-17T06:42:36.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-07-17T06:42:36.940 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-17T06:42:36.941 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-17T06:42:36.941 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-17T06:42:36.941 INFO:teuthology.orchestra.run.smithi044.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.10.0-rc7-gc30fd5683ea8.conf 2024-07-17T06:42:36.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:42:37.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-07-17T06:43:05.410 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-07-17T06:43:05.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:43:05.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:43:07.108 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-17T06:43:07.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:43:07.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:43:23.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-07-17T06:43:25.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-07-17T06:43:32.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:43:32.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:43:34.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:43:34.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:43:35.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-07-17T06:43:47.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:43:47.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:44:19.727 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-07-17T06:44:20.323 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:44:20.323 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.10.0-rc7-gc30fd5683ea8"... 2024-07-17T06:44:20.323 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-07-17T06:44:20.360 INFO:teuthology.orchestra.run.smithi175.stdout:6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:44:20.361 DEBUG:teuthology.task.kernel:current kernel version is 6.10.0-rc7-gc30fd5683ea8 vs 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:44:20.361 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-17T06:44:20.361 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-17T06:44:21.362 DEBUG:teuthology.parallel:result is None 2024-07-17T06:44:34.603 DEBUG:teuthology.orchestra.remote:timed out 2024-07-17T06:44:46.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:44:46.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:44:46.915 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-07-17T06:44:47.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:44:47.543 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.10.0-rc7-gc30fd5683ea8"... 2024-07-17T06:44:47.543 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-07-17T06:44:47.597 INFO:teuthology.orchestra.run.smithi044.stdout:6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:44:47.597 DEBUG:teuthology.task.kernel:current kernel version is 6.10.0-rc7-gc30fd5683ea8 vs 6.10.0-rc7-gc30fd5683ea8 2024-07-17T06:44:47.598 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-17T06:44:47.598 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-17T06:44:48.600 DEBUG:teuthology.parallel:result is None 2024-07-17T06:44:48.601 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-17T06:44:48.609 INFO:teuthology.task.internal:Creating test directory... 2024-07-17T06:44:48.609 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-17T06:44:48.617 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-17T06:44:48.676 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-17T06:44:48.683 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-17T06:44:48.776 INFO:teuthology.task.internal:Creating archive directory... 2024-07-17T06:44:48.777 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-17T06:44:48.784 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-17T06:44:48.855 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-17T06:44:48.862 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-17T06:44:48.863 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-17T06:44:48.871 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-17T06:44:48.988 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-17T06:44:49.036 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-17T06:44:49.040 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-17T06:44:49.096 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-17T06:44:49.100 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-17T06:44:49.107 INFO:teuthology.task.internal:Configuring sudo... 2024-07-17T06:44:49.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-17T06:44:49.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-17T06:44:49.200 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-17T06:44:49.212 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-17T06:44:49.212 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-17T06:44:49.219 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-17T06:44:49.270 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-17T06:44:49.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-17T06:44:49.452 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-17T06:44:49.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-17T06:44:49.625 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:44:49.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-17T06:44:49.705 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-17T06:44:49.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-17T06:44:49.891 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-17T06:44:49.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-17T06:44:50.055 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:44:50.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-17T06:44:50.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-07-17T06:44:50.137 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-07-17T06:44:50.245 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-17T06:44:50.287 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-17T06:44:50.800 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-17T06:44:50.808 INFO:teuthology.task.internal:Starting timer... 2024-07-17T06:44:50.808 INFO:teuthology.run_tasks:Running task pcp... 2024-07-17T06:44:50.821 INFO:teuthology.run_tasks:Running task selinux... 2024-07-17T06:44:50.829 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-07-17T06:44:50.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-07-17T06:44:50.933 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-07-17T06:44:50.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-07-17T06:44:51.054 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-07-17T06:44:51.059 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-17T06:44:51.059 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-07-17T06:44:51.120 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-07-17T06:44:51.121 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-07-17T06:44:51.175 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-07-17T06:44:51.176 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-07-17T06:44:51.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-17T06:44:51.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:44:51.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-17T06:44:51.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:44:51.330 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-17T06:44:51.330 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-07-17T06:44:51.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-07-17T06:44:51.505 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-17T06:44:51.515 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-17T06:44:51.530 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-17T06:44:51.531 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-17T06:51:23.586 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-07-17T06:51:23.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:51:23.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:51:23.700 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-07-17T06:51:23.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-07-17T06:51:23.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:51:23.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-17T06:51:23.963 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-07-17T06:51:24.119 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-17T06:51:24.119 INFO:teuthology.run_tasks:Running task clock... 2024-07-17T06:51:24.131 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-17T06:51:24.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-17T06:51:24.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-17T06:51:24.140 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-17T06:51:24.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-17T06:51:24.226 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-17T06:51:24.230 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-17T06:51:24.268 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-17T06:51:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-17T06:51:24.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-07-17T06:51:24.348 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-07-17T06:51:24.369 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-07-17T06:51:24.377 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-07-17T06:51:24.409 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-17T06:51:24.417 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-17T06:51:24.451 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-17T06:51:24.461 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-17T06:51:24.597 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.606 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.611 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-07-17T06:51:24.619 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-17T06:51:24.619 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-07-17T06:51:24.619 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.619 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.619 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.620 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T06:51:24.621 INFO:teuthology.run_tasks:Running task install... 2024-07-17T06:51:24.629 DEBUG:teuthology.task.install:project ceph 2024-07-17T06:51:24.629 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}} 2024-07-17T06:51:24.629 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'} 2024-07-17T06:51:24.630 INFO:teuthology.task.install:Using flavor: default 2024-07-17T06:51:24.635 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-17T06:51:24.635 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-17T06:51:24.636 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'tag': None, 'wait_for_package': False} 2024-07-17T06:51:24.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-17T06:51:24.638 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'tag': None, 'wait_for_package': False} 2024-07-17T06:51:24.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-17T06:51:24.781 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/ 2024-07-17T06:51:24.782 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4937.g0325fd3c 2024-07-17T06:51:24.792 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/ 2024-07-17T06:51:24.793 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4937.g0325fd3c 2024-07-17T06:51:24.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-17T06:51:24.897 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:51:24.897 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-17T06:51:24.905 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/0325fd3c11a225c0fd9eb38fab88aad1370ad779/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-17T06:51:24.906 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:51:24.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-17T06:51:24.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-17T06:51:24.986 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0325fd3c11a225c0fd9eb38fab88aad1370ad779/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-17T06:51:24.994 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-17T06:51:24.994 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0325fd3c11a225c0fd9eb38fab88aad1370ad779/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-17T06:51:25.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-17T06:51:25.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-17T06:51:25.227 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-17T06:51:25.232 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-17T06:51:25.300 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-07-17T06:51:25.304 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-07-17T06:51:25.314 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-07-17T06:51:25.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-07-17T06:51:25.606 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-07-17T06:51:25.638 INFO:teuthology.orchestra.run.smithi044.stdout:45 files removed 2024-07-17T06:51:25.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-17T06:51:25.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-17T06:51:26.447 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 217 kB/s | 83 kB 00:00 2024-07-17T06:51:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 218 kB/s | 83 kB 00:00 2024-07-17T06:51:26.906 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 66 kB/s | 24 kB 00:00 2024-07-17T06:51:26.987 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 61 kB/s | 24 kB 00:00 2024-07-17T06:51:27.365 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-07-17T06:51:27.463 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-07-17T06:51:27.947 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-07-17T06:51:28.061 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-07-17T06:51:30.104 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-07-17T06:51:33.741 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-07-17T06:51:36.263 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-07-17T06:51:37.906 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 17 kB 00:00 2024-07-17T06:51:38.464 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-07-17T06:51:39.105 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:00 2024-07-17T06:51:40.340 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 17 kB 00:00 2024-07-17T06:51:41.539 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:00 2024-07-17T06:51:45.624 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-07-17T06:51:47.858 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-07-17T06:51:48.353 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-17T06:51:48.354 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-17T06:51:48.376 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-17T06:51:48.378 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-17T06:51:48.379 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-17T06:51:48.380 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-17T06:51:48.383 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-17T06:51:48.387 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-17T06:51:48.388 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-17T06:51:48.467 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-07-17T06:51:48.487 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-07-17T06:51:48.487 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-07-17T06:51:48.487 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 6.6 k 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 5.4 M 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 898 k 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 144 k 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 1.5 M 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 149 k 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 14 M 2024-07-17T06:51:48.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 7.4 M 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 49 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 11 M 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 50 M 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 274 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 770 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 227 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 27 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 18 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-17T06:51:48.489 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 32 k 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 754 k 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 126 k 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-17T06:51:48.490 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 162 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 320 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 302 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 100 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 85 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.2 M 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 170 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-07-17T06:51:48.491 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.4 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.2 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 21 M 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 34 k 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 2.3 M 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 278 k 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 4.7 M 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 17 M 2024-07-17T06:51:48.493 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 16 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 25 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 163 k 2024-07-17T06:51:48.494 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 509 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 5.4 M 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-17T06:51:48.495 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-17T06:51:48.496 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 45 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 138 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-17T06:51:48.497 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-17T06:51:48.498 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-17T06:51:48.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-17T06:51:48.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-17T06:51:48.501 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-07-17T06:51:48.502 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-07-17T06:51:48.503 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-07-17T06:51:48.503 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 12 Packages 2024-07-17T06:51:48.503 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:51:48.503 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 240 M 2024-07-17T06:51:48.503 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-07-17T06:51:49.392 INFO:teuthology.orchestra.run.smithi044.stdout:(1/160): ceph-19.0.0-4937.g0325fd3c.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-07-17T06:51:49.601 INFO:teuthology.orchestra.run.smithi044.stdout:(2/160): ceph-fuse-19.0.0-4937.g0325fd3c.el9.x8 4.2 MB/s | 898 kB 00:00 2024-07-17T06:51:49.660 INFO:teuthology.orchestra.run.smithi044.stdout:(3/160): ceph-immutable-object-cache-19.0.0-493 2.5 MB/s | 144 kB 00:00 2024-07-17T06:51:49.777 INFO:teuthology.orchestra.run.smithi044.stdout:(4/160): ceph-base-19.0.0-4937.g0325fd3c.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-07-17T06:51:49.902 INFO:teuthology.orchestra.run.smithi044.stdout:(5/160): ceph-mds-19.0.0-4937.g0325fd3c.el9.x86 9.7 MB/s | 2.3 MB 00:00 2024-07-17T06:51:49.952 INFO:teuthology.orchestra.run.smithi044.stdout:(6/160): ceph-mgr-19.0.0-4937.g0325fd3c.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-07-17T06:51:50.303 INFO:teuthology.orchestra.run.smithi044.stdout:(7/160): ceph-mon-19.0.0-4937.g0325fd3c.el9.x86 12 MB/s | 4.7 MB 00:00 2024-07-17T06:51:50.417 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-17T06:51:50.419 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-17T06:51:50.439 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-17T06:51:50.440 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-17T06:51:50.441 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-17T06:51:50.443 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-17T06:51:50.445 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-17T06:51:50.448 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-17T06:51:50.450 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-17T06:51:50.522 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-07-17T06:51:50.543 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-17T06:51:50.543 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-07-17T06:51:50.543 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-17T06:51:50.543 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-07-17T06:51:50.543 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 6.6 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 5.4 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 898 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 144 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 1.5 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 149 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 14 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 7.4 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 49 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 11 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 50 M 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 274 k 2024-07-17T06:51:50.544 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 770 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 227 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 27 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 18 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 32 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 754 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 126 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-17T06:51:50.545 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 162 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 320 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 302 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 100 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 85 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.2 M 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 170 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-17T06:51:50.546 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.4 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 3.2 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-07-17T06:51:50.547 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 21 M 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 34 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 2.3 M 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 278 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 4.7 M 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 17 M 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4937.g0325fd3c.el9 ceph-noarch 16 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 25 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-17T06:51:50.548 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 163 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-17T06:51:50.549 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 509 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 5.4 M 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-17T06:51:50.550 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 45 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-4937.g0325fd3c.el9 ceph 138 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-17T06:51:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-17T06:51:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-17T06:51:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-17T06:51:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-17T06:51:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 12 Packages 2024-07-17T06:51:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:51:50.557 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 240 M 2024-07-17T06:51:50.557 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-07-17T06:51:50.861 INFO:teuthology.orchestra.run.smithi044.stdout:(8/160): ceph-common-19.0.0-4937.g0325fd3c.el9. 12 MB/s | 21 MB 00:01 2024-07-17T06:51:51.119 INFO:teuthology.orchestra.run.smithi044.stdout:(9/160): ceph-osd-19.0.0-4937.g0325fd3c.el9.x86 14 MB/s | 17 MB 00:01 2024-07-17T06:51:51.145 INFO:teuthology.orchestra.run.smithi044.stdout:(10/160): ceph-selinux-19.0.0-4937.g0325fd3c.el 89 kB/s | 25 kB 00:00 2024-07-17T06:51:51.212 INFO:teuthology.orchestra.run.smithi044.stdout:(11/160): cephfs-mirror-19.0.0-4937.g0325fd3c.e 3.4 MB/s | 227 kB 00:00 2024-07-17T06:51:51.270 INFO:teuthology.orchestra.run.smithi044.stdout:(12/160): cephfs-shell-19.0.0-4937.g0325fd3c.el 471 kB/s | 27 kB 00:00 2024-07-17T06:51:51.321 INFO:teuthology.orchestra.run.smithi044.stdout:(13/160): libcephfs-devel-19.0.0-4937.g0325fd3c 649 kB/s | 32 kB 00:00 2024-07-17T06:51:51.413 INFO:teuthology.orchestra.run.smithi044.stdout:(14/160): libcephfs2-19.0.0-4937.g0325fd3c.el9. 8.1 MB/s | 754 kB 00:00 2024-07-17T06:51:51.559 INFO:teuthology.orchestra.run.smithi175.stdout:(1/160): ceph-19.0.0-4937.g0325fd3c.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-07-17T06:51:51.580 INFO:teuthology.orchestra.run.smithi044.stdout:(15/160): ceph-radosgw-19.0.0-4937.g0325fd3c.el 8.5 MB/s | 11 MB 00:01 2024-07-17T06:51:51.613 INFO:teuthology.orchestra.run.smithi044.stdout:(16/160): libcephsqlite-19.0.0-4937.g0325fd3c.e 819 kB/s | 163 kB 00:00 2024-07-17T06:51:51.655 INFO:teuthology.orchestra.run.smithi044.stdout:(17/160): librados-devel-19.0.0-4937.g0325fd3c. 1.7 MB/s | 126 kB 00:00 2024-07-17T06:51:51.697 INFO:teuthology.orchestra.run.smithi044.stdout:(18/160): libradosstriper1-19.0.0-4937.g0325fd3 6.0 MB/s | 509 kB 00:00 2024-07-17T06:51:51.776 INFO:teuthology.orchestra.run.smithi175.stdout:(2/160): ceph-fuse-19.0.0-4937.g0325fd3c.el9.x8 4.1 MB/s | 898 kB 00:00 2024-07-17T06:51:51.790 INFO:teuthology.orchestra.run.smithi044.stdout:(19/160): python3-ceph-argparse-19.0.0-4937.g03 498 kB/s | 45 kB 00:00 2024-07-17T06:51:51.873 INFO:teuthology.orchestra.run.smithi044.stdout:(20/160): python3-ceph-common-19.0.0-4937.g0325 1.6 MB/s | 138 kB 00:00 2024-07-17T06:51:51.926 INFO:teuthology.orchestra.run.smithi175.stdout:(3/160): ceph-base-19.0.0-4937.g0325fd3c.el9.x8 8.7 MB/s | 5.4 MB 00:00 2024-07-17T06:51:51.948 INFO:teuthology.orchestra.run.smithi044.stdout:(21/160): python3-cephfs-19.0.0-4937.g0325fd3c. 2.1 MB/s | 162 kB 00:00 2024-07-17T06:51:51.960 INFO:teuthology.orchestra.run.smithi175.stdout:(4/160): ceph-immutable-object-cache-19.0.0-493 790 kB/s | 144 kB 00:00 2024-07-17T06:51:52.215 INFO:teuthology.orchestra.run.smithi044.stdout:(22/160): librgw2-19.0.0-4937.g0325fd3c.el9.x86 9.7 MB/s | 5.4 MB 00:00 2024-07-17T06:51:52.282 INFO:teuthology.orchestra.run.smithi044.stdout:(23/160): python3-rbd-19.0.0-4937.g0325fd3c.el9 4.5 MB/s | 302 kB 00:00 2024-07-17T06:51:52.294 INFO:teuthology.orchestra.run.smithi175.stdout:(5/160): ceph-mds-19.0.0-4937.g0325fd3c.el9.x86 6.4 MB/s | 2.3 MB 00:00 2024-07-17T06:51:52.341 INFO:teuthology.orchestra.run.smithi044.stdout:(24/160): python3-rgw-19.0.0-4937.g0325fd3c.el9 1.7 MB/s | 100 kB 00:00 2024-07-17T06:51:52.360 INFO:teuthology.orchestra.run.smithi175.stdout:(6/160): ceph-mgr-19.0.0-4937.g0325fd3c.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-07-17T06:51:52.391 INFO:teuthology.orchestra.run.smithi044.stdout:(25/160): python3-rados-19.0.0-4937.g0325fd3c.e 725 kB/s | 320 kB 00:00 2024-07-17T06:51:52.416 INFO:teuthology.orchestra.run.smithi044.stdout:(26/160): rbd-fuse-19.0.0-4937.g0325fd3c.el9.x8 1.1 MB/s | 85 kB 00:00 2024-07-17T06:51:52.509 INFO:teuthology.orchestra.run.smithi044.stdout:(27/160): rbd-nbd-19.0.0-4937.g0325fd3c.el9.x86 1.8 MB/s | 170 kB 00:00 2024-07-17T06:51:52.600 INFO:teuthology.orchestra.run.smithi044.stdout:(28/160): ceph-grafana-dashboards-19.0.0-4937.g 376 kB/s | 34 kB 00:00 2024-07-17T06:51:52.636 INFO:teuthology.orchestra.run.smithi175.stdout:(7/160): ceph-mon-19.0.0-4937.g0325fd3c.el9.x86 14 MB/s | 4.7 MB 00:00 2024-07-17T06:51:52.684 INFO:teuthology.orchestra.run.smithi044.stdout:(29/160): ceph-mgr-cephadm-19.0.0-4937.g0325fd3 1.8 MB/s | 149 kB 00:00 2024-07-17T06:51:52.809 INFO:teuthology.orchestra.run.smithi044.stdout:(30/160): rbd-mirror-19.0.0-4937.g0325fd3c.el9. 7.6 MB/s | 3.2 MB 00:00 2024-07-17T06:51:53.661 INFO:teuthology.orchestra.run.smithi175.stdout:(8/160): ceph-radosgw-19.0.0-4937.g0325fd3c.el9 11 MB/s | 11 MB 00:01 2024-07-17T06:51:53.686 INFO:teuthology.orchestra.run.smithi044.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 8.5 MB/s | 7.4 MB 00:00 2024-07-17T06:51:53.728 INFO:teuthology.orchestra.run.smithi175.stdout:(9/160): ceph-selinux-19.0.0-4937.g0325fd3c.el9 382 kB/s | 25 kB 00:00 2024-07-17T06:51:53.768 INFO:teuthology.orchestra.run.smithi044.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4937.g03 3.4 MB/s | 278 kB 00:00 2024-07-17T06:51:53.835 INFO:teuthology.orchestra.run.smithi044.stdout:(33/160): ceph-mgr-rook-19.0.0-4937.g0325fd3c.e 750 kB/s | 49 kB 00:00 2024-07-17T06:51:53.910 INFO:teuthology.orchestra.run.smithi044.stdout:(34/160): ceph-prometheus-alerts-19.0.0-4937.g0 222 kB/s | 16 kB 00:00 2024-07-17T06:51:54.028 INFO:teuthology.orchestra.run.smithi175.stdout:(10/160): ceph-common-19.0.0-4937.g0325fd3c.el9 7.8 MB/s | 21 MB 00:02 2024-07-17T06:51:54.087 INFO:teuthology.orchestra.run.smithi175.stdout:(11/160): cephfs-mirror-19.0.0-4937.g0325fd3c.e 3.9 MB/s | 227 kB 00:00 2024-07-17T06:51:54.146 INFO:teuthology.orchestra.run.smithi175.stdout:(12/160): cephfs-shell-19.0.0-4937.g0325fd3c.el 472 kB/s | 27 kB 00:00 2024-07-17T06:51:54.196 INFO:teuthology.orchestra.run.smithi175.stdout:(13/160): libcephfs-devel-19.0.0-4937.g0325fd3c 649 kB/s | 32 kB 00:00 2024-07-17T06:51:54.280 INFO:teuthology.orchestra.run.smithi175.stdout:(14/160): libcephfs2-19.0.0-4937.g0325fd3c.el9. 9.0 MB/s | 754 kB 00:00 2024-07-17T06:51:54.503 INFO:teuthology.orchestra.run.smithi044.stdout:(35/160): ceph-test-19.0.0-4937.g0325fd3c.el9.x 15 MB/s | 50 MB 00:03 2024-07-17T06:51:54.536 INFO:teuthology.orchestra.run.smithi044.stdout:(36/160): ceph-volume-19.0.0-4937.g0325fd3c.el9 439 kB/s | 274 kB 00:00 2024-07-17T06:51:54.538 INFO:teuthology.orchestra.run.smithi175.stdout:(15/160): ceph-osd-19.0.0-4937.g0325fd3c.el9.x8 7.7 MB/s | 17 MB 00:02 2024-07-17T06:51:54.572 INFO:teuthology.orchestra.run.smithi175.stdout:(16/160): libcephsqlite-19.0.0-4937.g0325fd3c.e 561 kB/s | 163 kB 00:00 2024-07-17T06:51:54.595 INFO:teuthology.orchestra.run.smithi044.stdout:(37/160): cephfs-top-19.0.0-4937.g0325fd3c.el9. 317 kB/s | 18 kB 00:00 2024-07-17T06:51:54.605 INFO:teuthology.orchestra.run.smithi175.stdout:(17/160): librados-devel-19.0.0-4937.g0325fd3c. 1.9 MB/s | 126 kB 00:00 2024-07-17T06:51:54.647 INFO:teuthology.orchestra.run.smithi175.stdout:(18/160): libradosstriper1-19.0.0-4937.g0325fd3 6.7 MB/s | 509 kB 00:00 2024-07-17T06:51:54.655 INFO:teuthology.orchestra.run.smithi044.stdout:(38/160): cephadm-19.0.0-4937.g0325fd3c.el9.noa 5.0 MB/s | 770 kB 00:00 2024-07-17T06:51:54.689 INFO:teuthology.orchestra.run.smithi175.stdout:(19/160): python3-ceph-argparse-19.0.0-4937.g03 1.1 MB/s | 45 kB 00:00 2024-07-17T06:51:54.756 INFO:teuthology.orchestra.run.smithi175.stdout:(20/160): python3-ceph-common-19.0.0-4937.g0325 2.1 MB/s | 138 kB 00:00 2024-07-17T06:51:54.818 INFO:teuthology.orchestra.run.smithi175.stdout:(21/160): python3-cephfs-19.0.0-4937.g0325fd3c. 2.8 MB/s | 162 kB 00:00 2024-07-17T06:51:54.871 INFO:teuthology.orchestra.run.smithi044.stdout:(39/160): ceph-mgr-dashboard-19.0.0-4937.g0325f 6.4 MB/s | 14 MB 00:02 2024-07-17T06:51:54.882 INFO:teuthology.orchestra.run.smithi175.stdout:(22/160): python3-rados-19.0.0-4937.g0325fd3c.e 4.8 MB/s | 320 kB 00:00 2024-07-17T06:51:54.904 INFO:teuthology.orchestra.run.smithi044.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 259 kB/s | 80 kB 00:00 2024-07-17T06:51:54.940 INFO:teuthology.orchestra.run.smithi175.stdout:(23/160): python3-rbd-19.0.0-4937.g0325fd3c.el9 5.2 MB/s | 302 kB 00:00 2024-07-17T06:51:54.988 INFO:teuthology.orchestra.run.smithi044.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-07-17T06:51:55.021 INFO:teuthology.orchestra.run.smithi044.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-07-17T06:51:55.040 INFO:teuthology.orchestra.run.smithi175.stdout:(24/160): librgw2-19.0.0-4937.g0325fd3c.el9.x86 13 MB/s | 5.4 MB 00:00 2024-07-17T06:51:55.063 INFO:teuthology.orchestra.run.smithi044.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-07-17T06:51:55.066 INFO:teuthology.orchestra.run.smithi175.stdout:(25/160): python3-rgw-19.0.0-4937.g0325fd3c.el9 800 kB/s | 100 kB 00:00 2024-07-17T06:51:55.083 INFO:teuthology.orchestra.run.smithi175.stdout:(26/160): rbd-fuse-19.0.0-4937.g0325fd3c.el9.x8 2.0 MB/s | 85 kB 00:00 2024-07-17T06:51:55.088 INFO:teuthology.orchestra.run.smithi044.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-07-17T06:51:55.130 INFO:teuthology.orchestra.run.smithi044.stdout:(45/160): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-07-17T06:51:55.141 INFO:teuthology.orchestra.run.smithi175.stdout:(27/160): rbd-nbd-19.0.0-4937.g0325fd3c.el9.x86 2.9 MB/s | 170 kB 00:00 2024-07-17T06:51:55.156 INFO:teuthology.orchestra.run.smithi044.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-07-17T06:51:55.183 INFO:teuthology.orchestra.run.smithi175.stdout:(28/160): ceph-grafana-dashboards-19.0.0-4937.g 834 kB/s | 34 kB 00:00 2024-07-17T06:51:55.223 INFO:teuthology.orchestra.run.smithi044.stdout:(47/160): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2024-07-17T06:51:55.242 INFO:teuthology.orchestra.run.smithi175.stdout:(29/160): ceph-mgr-cephadm-19.0.0-4937.g0325fd3 2.6 MB/s | 149 kB 00:00 2024-07-17T06:51:55.248 INFO:teuthology.orchestra.run.smithi044.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-07-17T06:51:55.334 INFO:teuthology.orchestra.run.smithi175.stdout:(30/160): rbd-mirror-19.0.0-4937.g0325fd3c.el9. 12 MB/s | 3.2 MB 00:00 2024-07-17T06:51:55.382 INFO:teuthology.orchestra.run.smithi044.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 805 kB/s | 107 kB 00:00 2024-07-17T06:51:55.432 INFO:teuthology.orchestra.run.smithi044.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-07-17T06:51:55.457 INFO:teuthology.orchestra.run.smithi044.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 292 kB/s | 22 kB 00:00 2024-07-17T06:51:55.507 INFO:teuthology.orchestra.run.smithi044.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-07-17T06:51:55.533 INFO:teuthology.orchestra.run.smithi044.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 438 kB/s | 33 kB 00:00 2024-07-17T06:51:55.575 INFO:teuthology.orchestra.run.smithi044.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-07-17T06:51:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-07-17T06:51:55.642 INFO:teuthology.orchestra.run.smithi044.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-07-17T06:51:55.676 INFO:teuthology.orchestra.run.smithi044.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.8 MB/s | 160 kB 00:00 2024-07-17T06:51:55.709 INFO:teuthology.orchestra.run.smithi044.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 683 kB/s | 45 kB 00:00 2024-07-17T06:51:55.793 INFO:teuthology.orchestra.run.smithi044.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.7 MB/s | 662 kB 00:00 2024-07-17T06:51:55.835 INFO:teuthology.orchestra.run.smithi044.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 4.6 MB/s | 581 kB 00:00 2024-07-17T06:51:55.860 INFO:teuthology.orchestra.run.smithi044.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-07-17T06:51:55.885 INFO:teuthology.orchestra.run.smithi044.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-07-17T06:51:55.892 INFO:teuthology.orchestra.run.smithi175.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-07-17T06:51:55.959 INFO:teuthology.orchestra.run.smithi175.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4937.g03 4.1 MB/s | 278 kB 00:00 2024-07-17T06:51:55.961 INFO:teuthology.orchestra.run.smithi044.stdout:(63/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-07-17T06:51:56.003 INFO:teuthology.orchestra.run.smithi044.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-07-17T06:51:56.018 INFO:teuthology.orchestra.run.smithi175.stdout:(33/160): ceph-mgr-rook-19.0.0-4937.g0325fd3c.e 855 kB/s | 49 kB 00:00 2024-07-17T06:51:56.045 INFO:teuthology.orchestra.run.smithi044.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 424 kB/s | 35 kB 00:00 2024-07-17T06:51:56.068 INFO:teuthology.orchestra.run.smithi175.stdout:(34/160): ceph-prometheus-alerts-19.0.0-4937.g0 333 kB/s | 16 kB 00:00 2024-07-17T06:51:56.079 INFO:teuthology.orchestra.run.smithi044.stdout:(66/160): lua-5.4.4-4.el9.x86_64.rpm 980 kB/s | 188 kB 00:00 2024-07-17T06:51:56.254 INFO:teuthology.orchestra.run.smithi175.stdout:(35/160): ceph-mgr-dashboard-19.0.0-4937.g0325f 14 MB/s | 14 MB 00:01 2024-07-17T06:51:56.285 INFO:teuthology.orchestra.run.smithi175.stdout:(36/160): ceph-volume-19.0.0-4937.g0325fd3c.el9 1.2 MB/s | 274 kB 00:00 2024-07-17T06:51:56.336 INFO:teuthology.orchestra.run.smithi175.stdout:(37/160): cephfs-top-19.0.0-4937.g0325fd3c.el9. 372 kB/s | 18 kB 00:00 2024-07-17T06:51:56.354 INFO:teuthology.orchestra.run.smithi044.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-07-17T06:51:56.378 INFO:teuthology.orchestra.run.smithi175.stdout:(38/160): cephadm-19.0.0-4937.g0325fd3c.el9.noa 6.0 MB/s | 770 kB 00:00 2024-07-17T06:51:56.513 INFO:teuthology.orchestra.run.smithi044.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-07-17T06:51:56.588 INFO:teuthology.orchestra.run.smithi044.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-07-17T06:51:56.696 INFO:teuthology.orchestra.run.smithi044.stdout:(70/160): openblas-openmp-0.3.21-2.el9.x86_64.r 6.8 MB/s | 4.7 MB 00:00 2024-07-17T06:51:56.797 INFO:teuthology.orchestra.run.smithi044.stdout:(71/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-07-17T06:51:56.856 INFO:teuthology.orchestra.run.smithi044.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-07-17T06:51:56.906 INFO:teuthology.orchestra.run.smithi044.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-07-17T06:51:56.964 INFO:teuthology.orchestra.run.smithi044.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-07-17T06:51:57.031 INFO:teuthology.orchestra.run.smithi044.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-07-17T06:51:57.061 INFO:teuthology.orchestra.run.smithi175.stdout:(39/160): ceph-test-19.0.0-4937.g0325fd3c.el9.x 15 MB/s | 50 MB 00:03 2024-07-17T06:51:57.087 INFO:teuthology.orchestra.run.smithi175.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 106 kB/s | 80 kB 00:00 2024-07-17T06:51:57.090 INFO:teuthology.orchestra.run.smithi044.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-07-17T06:51:57.149 INFO:teuthology.orchestra.run.smithi044.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 445 kB/s | 26 kB 00:00 2024-07-17T06:51:57.207 INFO:teuthology.orchestra.run.smithi044.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-07-17T06:51:57.274 INFO:teuthology.orchestra.run.smithi044.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-07-17T06:51:57.333 INFO:teuthology.orchestra.run.smithi044.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 604 kB/s | 35 kB 00:00 2024-07-17T06:51:57.458 INFO:teuthology.orchestra.run.smithi044.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 6.6 MB/s | 5.8 MB 00:00 2024-07-17T06:51:57.487 INFO:teuthology.orchestra.run.smithi175.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 461 kB/s | 195 kB 00:00 2024-07-17T06:51:57.550 INFO:teuthology.orchestra.run.smithi044.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-07-17T06:51:57.595 INFO:teuthology.orchestra.run.smithi175.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 499 kB/s | 253 kB 00:00 2024-07-17T06:51:57.617 INFO:teuthology.orchestra.run.smithi044.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-07-17T06:51:57.759 INFO:teuthology.orchestra.run.smithi044.stdout:(84/160): python3-babel-2.9.1-2.el9.noarch.rpm 6.6 MB/s | 6.0 MB 00:00 2024-07-17T06:51:57.784 INFO:teuthology.orchestra.run.smithi044.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 464 kB/s | 77 kB 00:00 2024-07-17T06:51:57.818 INFO:teuthology.orchestra.run.smithi044.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-07-17T06:51:57.852 INFO:teuthology.orchestra.run.smithi044.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-07-17T06:51:57.894 INFO:teuthology.orchestra.run.smithi044.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-07-17T06:51:57.927 INFO:teuthology.orchestra.run.smithi044.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-07-17T06:51:57.953 INFO:teuthology.orchestra.run.smithi044.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 886 kB/s | 51 kB 00:00 2024-07-17T06:51:57.987 INFO:teuthology.orchestra.run.smithi044.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 925 kB/s | 54 kB 00:00 2024-07-17T06:51:58.021 INFO:teuthology.orchestra.run.smithi175.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 251 kB/s | 106 kB 00:00 2024-07-17T06:51:58.045 INFO:teuthology.orchestra.run.smithi044.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 725 kB/s | 42 kB 00:00 2024-07-17T06:51:58.104 INFO:teuthology.orchestra.run.smithi044.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 772 kB/s | 44 kB 00:00 2024-07-17T06:51:58.179 INFO:teuthology.orchestra.run.smithi044.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-07-17T06:51:58.271 INFO:teuthology.orchestra.run.smithi044.stdout:(95/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.4 MB/s | 5.1 MB 00:00 2024-07-17T06:51:58.296 INFO:teuthology.orchestra.run.smithi044.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-07-17T06:51:58.322 INFO:teuthology.orchestra.run.smithi044.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 959 kB/s | 47 kB 00:00 2024-07-17T06:51:58.337 INFO:teuthology.orchestra.run.smithi175.stdout:(44/160): libgfortran-11.4.1-3.el9.x86_64.rpm 410 kB/s | 803 kB 00:01 2024-07-17T06:51:58.380 INFO:teuthology.orchestra.run.smithi044.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-07-17T06:51:58.414 INFO:teuthology.orchestra.run.smithi044.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-07-17T06:51:58.446 INFO:teuthology.orchestra.run.smithi175.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 318 kB/s | 135 kB 00:00 2024-07-17T06:51:58.448 INFO:teuthology.orchestra.run.smithi044.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-07-17T06:51:58.498 INFO:teuthology.orchestra.run.smithi044.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 769 kB/s | 64 kB 00:00 2024-07-17T06:51:58.523 INFO:teuthology.orchestra.run.smithi044.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-07-17T06:51:58.599 INFO:teuthology.orchestra.run.smithi044.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-07-17T06:51:58.732 INFO:teuthology.orchestra.run.smithi044.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.1 MB/s | 548 kB 00:00 2024-07-17T06:51:58.774 INFO:teuthology.orchestra.run.smithi044.stdout:(105/160): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-07-17T06:51:58.788 INFO:teuthology.orchestra.run.smithi175.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 278 kB/s | 125 kB 00:00 2024-07-17T06:51:58.874 INFO:teuthology.orchestra.run.smithi044.stdout:(106/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-07-17T06:51:58.908 INFO:teuthology.orchestra.run.smithi044.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-07-17T06:51:58.934 INFO:teuthology.orchestra.run.smithi044.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-17T06:51:58.967 INFO:teuthology.orchestra.run.smithi044.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-07-17T06:51:58.988 INFO:teuthology.orchestra.run.smithi175.stdout:(47/160): python3-cryptography-36.0.1-4.el9.x86 836 kB/s | 1.2 MB 00:01 2024-07-17T06:51:59.000 INFO:teuthology.orchestra.run.smithi044.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-07-17T06:51:59.043 INFO:teuthology.orchestra.run.smithi044.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-07-17T06:51:59.105 INFO:teuthology.orchestra.run.smithi175.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 326 kB/s | 215 kB 00:00 2024-07-17T06:51:59.126 INFO:teuthology.orchestra.run.smithi044.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-07-17T06:51:59.193 INFO:teuthology.orchestra.run.smithi044.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-07-17T06:51:59.210 INFO:teuthology.orchestra.run.smithi044.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.7 MB/s | 43 kB 00:00 2024-07-17T06:51:59.227 INFO:teuthology.orchestra.run.smithi044.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 2.0 MB/s | 32 kB 00:00 2024-07-17T06:51:59.244 INFO:teuthology.orchestra.run.smithi044.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 894 kB/s | 14 kB 00:00 2024-07-17T06:51:59.286 INFO:teuthology.orchestra.run.smithi044.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-07-17T06:51:59.345 INFO:teuthology.orchestra.run.smithi044.stdout:(118/160): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-07-17T06:51:59.395 INFO:teuthology.orchestra.run.smithi044.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-07-17T06:51:59.462 INFO:teuthology.orchestra.run.smithi044.stdout:(120/160): python3-google-auth-2.30.0-1.el9.noa 3.2 MB/s | 213 kB 00:00 2024-07-17T06:51:59.604 INFO:teuthology.orchestra.run.smithi044.stdout:(121/160): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-07-17T06:51:59.638 INFO:teuthology.orchestra.run.smithi044.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-07-17T06:51:59.663 INFO:teuthology.orchestra.run.smithi044.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 441 kB/s | 11 kB 00:00 2024-07-17T06:51:59.689 INFO:teuthology.orchestra.run.smithi044.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 732 kB/s | 18 kB 00:00 2024-07-17T06:51:59.714 INFO:teuthology.orchestra.run.smithi044.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 960 kB/s | 23 kB 00:00 2024-07-17T06:51:59.730 INFO:teuthology.orchestra.run.smithi175.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 732 kB/s | 689 kB 00:00 2024-07-17T06:51:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 806 kB/s | 19 kB 00:00 2024-07-17T06:51:59.764 INFO:teuthology.orchestra.run.smithi044.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 814 kB/s | 20 kB 00:00 2024-07-17T06:51:59.889 INFO:teuthology.orchestra.run.smithi175.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 136 kB/s | 107 kB 00:00 2024-07-17T06:51:59.998 INFO:teuthology.orchestra.run.smithi044.stdout:(128/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:02 2024-07-17T06:52:00.081 INFO:teuthology.orchestra.run.smithi175.stdout:(51/160): bison-3.7.4-5.el9.x86_64.rpm 864 kB/s | 943 kB 00:01 2024-07-17T06:52:00.107 INFO:teuthology.orchestra.run.smithi044.stdout:(129/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-07-17T06:52:00.132 INFO:teuthology.orchestra.run.smithi044.stdout:(130/160): python3-logutils-0.3.5-21.el9.noarch 350 kB/s | 46 kB 00:00 2024-07-17T06:52:00.158 INFO:teuthology.orchestra.run.smithi044.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-07-17T06:52:00.183 INFO:teuthology.orchestra.run.smithi044.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-07-17T06:52:00.208 INFO:teuthology.orchestra.run.smithi044.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 682 kB/s | 16 kB 00:00 2024-07-17T06:52:00.242 INFO:teuthology.orchestra.run.smithi044.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.7 MB/s | 90 kB 00:00 2024-07-17T06:52:00.267 INFO:teuthology.orchestra.run.smithi044.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-17T06:52:00.292 INFO:teuthology.orchestra.run.smithi044.stdout:(136/160): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-07-17T06:52:00.326 INFO:teuthology.orchestra.run.smithi044.stdout:(137/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-07-17T06:52:00.360 INFO:teuthology.orchestra.run.smithi044.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-07-17T06:52:00.407 INFO:teuthology.orchestra.run.smithi044.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 764 kB/s | 36 kB 00:00 2024-07-17T06:52:00.414 INFO:teuthology.orchestra.run.smithi175.stdout:(52/160): elfutils-libelf-devel-0.191-4.el9.x86 32 kB/s | 22 kB 00:00 2024-07-17T06:52:00.444 INFO:teuthology.orchestra.run.smithi044.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-17T06:52:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 849 kB/s | 51 kB 00:00 2024-07-17T06:52:00.502 INFO:teuthology.orchestra.run.smithi044.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.7 MB/s | 90 kB 00:00 2024-07-17T06:52:00.544 INFO:teuthology.orchestra.run.smithi044.stdout:(143/160): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-07-17T06:52:00.570 INFO:teuthology.orchestra.run.smithi044.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-07-17T06:52:00.604 INFO:teuthology.orchestra.run.smithi044.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-07-17T06:52:00.637 INFO:teuthology.orchestra.run.smithi044.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-07-17T06:52:00.679 INFO:teuthology.orchestra.run.smithi044.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-07-17T06:52:00.756 INFO:teuthology.orchestra.run.smithi175.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-07-17T06:52:00.821 INFO:teuthology.orchestra.run.smithi044.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-07-17T06:52:00.881 INFO:teuthology.orchestra.run.smithi175.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 317 kB/s | 313 kB 00:00 2024-07-17T06:52:01.065 INFO:teuthology.orchestra.run.smithi044.stdout:(149/160): librados2-19.0.0-4937.g0325fd3c.el9. 8.0 MB/s | 3.4 MB 00:00 2024-07-17T06:52:01.264 INFO:teuthology.orchestra.run.smithi044.stdout:(150/160): librbd1-19.0.0-4937.g0325fd3c.el9.x8 5.5 MB/s | 3.2 MB 00:00 2024-07-17T06:52:01.368 INFO:teuthology.orchestra.run.smithi044.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 395 kB/s | 41 kB 00:00 2024-07-17T06:52:01.390 INFO:teuthology.orchestra.run.smithi175.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 29 kB/s | 18 kB 00:00 2024-07-17T06:52:01.432 INFO:teuthology.orchestra.run.smithi175.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-07-17T06:52:01.464 INFO:teuthology.orchestra.run.smithi044.stdout:(152/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.6 MB/s | 2.3 MB 00:00 2024-07-17T06:52:01.498 INFO:teuthology.orchestra.run.smithi044.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.9 MB/s | 247 kB 00:00 2024-07-17T06:52:01.523 INFO:teuthology.orchestra.run.smithi044.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 635 kB/s | 37 kB 00:00 2024-07-17T06:52:01.574 INFO:teuthology.orchestra.run.smithi175.stdout:(57/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-07-17T06:52:01.581 INFO:teuthology.orchestra.run.smithi044.stdout:(155/160): NetworkManager-libnm-1.48.2-2.el9.x8 3.5 MB/s | 1.8 MB 00:00 2024-07-17T06:52:01.615 INFO:teuthology.orchestra.run.smithi044.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-07-17T06:52:01.657 INFO:teuthology.orchestra.run.smithi044.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.9 MB/s | 260 kB 00:00 2024-07-17T06:52:01.974 INFO:teuthology.orchestra.run.smithi044.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 3.4 MB/s | 1.1 MB 00:00 2024-07-17T06:52:02.024 INFO:teuthology.orchestra.run.smithi044.stdout:(159/160): openssl-3.2.2-2.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-07-17T06:52:02.208 INFO:teuthology.orchestra.run.smithi044.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 4.0 MB/s | 2.4 MB 00:00 2024-07-17T06:52:02.221 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-07-17T06:52:02.221 INFO:teuthology.orchestra.run.smithi044.stdout:Total 17 MB/s | 240 MB 00:13 2024-07-17T06:52:02.224 INFO:teuthology.orchestra.run.smithi175.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 192 kB/s | 160 kB 00:00 2024-07-17T06:52:02.241 INFO:teuthology.orchestra.run.smithi175.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 56 kB/s | 45 kB 00:00 2024-07-17T06:52:02.675 INFO:teuthology.orchestra.run.smithi175.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 602 kB/s | 662 kB 00:01 2024-07-17T06:52:02.891 INFO:teuthology.orchestra.run.smithi175.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 39 kB/s | 25 kB 00:00 2024-07-17T06:52:03.192 INFO:teuthology.orchestra.run.smithi175.stdout:(62/160): libtool-2.4.6-46.el9.x86_64.rpm 602 kB/s | 581 kB 00:00 2024-07-17T06:52:03.342 INFO:teuthology.orchestra.run.smithi175.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-07-17T06:52:03.901 INFO:teuthology.orchestra.run.smithi175.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 313 kB/s | 292 kB 00:00 2024-07-17T06:52:04.051 INFO:teuthology.orchestra.run.smithi175.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 50 kB/s | 35 kB 00:00 2024-07-17T06:52:04.217 INFO:teuthology.orchestra.run.smithi175.stdout:(66/160): lua-5.4.4-4.el9.x86_64.rpm 197 kB/s | 188 kB 00:00 2024-07-17T06:52:05.201 INFO:teuthology.orchestra.run.smithi175.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-07-17T06:52:05.335 INFO:teuthology.orchestra.run.smithi175.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-07-17T06:52:05.394 INFO:teuthology.orchestra.run.smithi175.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-07-17T06:52:05.593 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-07-17T06:52:05.901 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-07-17T06:52:05.901 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-07-17T06:52:06.119 INFO:teuthology.orchestra.run.smithi175.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 170 kB/s | 133 kB 00:00 2024-07-17T06:52:06.360 INFO:teuthology.orchestra.run.smithi175.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 787 kB/s | 911 kB 00:01 2024-07-17T06:52:06.894 INFO:teuthology.orchestra.run.smithi175.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 3.9 MB/s | 5.8 MB 00:01 2024-07-17T06:52:07.211 INFO:teuthology.orchestra.run.smithi175.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 112 kB/s | 87 kB 00:00 2024-07-17T06:52:07.295 INFO:teuthology.orchestra.run.smithi175.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 880 kB/s | 1.0 MB 00:01 2024-07-17T06:52:07.920 INFO:teuthology.orchestra.run.smithi175.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 86 kB/s | 61 kB 00:00 2024-07-17T06:52:08.145 INFO:teuthology.orchestra.run.smithi175.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-07-17T06:52:08.586 INFO:teuthology.orchestra.run.smithi175.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 264 kB/s | 244 kB 00:00 2024-07-17T06:52:08.676 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-07-17T06:52:08.676 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-07-17T06:52:08.837 INFO:teuthology.orchestra.run.smithi175.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 271 kB/s | 248 kB 00:00 2024-07-17T06:52:08.871 INFO:teuthology.orchestra.run.smithi175.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 35 kB/s | 26 kB 00:00 2024-07-17T06:52:08.896 INFO:teuthology.orchestra.run.smithi175.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 32 kB/s | 19 kB 00:00 2024-07-17T06:52:09.529 INFO:teuthology.orchestra.run.smithi175.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-07-17T06:52:09.688 INFO:teuthology.orchestra.run.smithi175.stdout:(82/160): python3-mako-1.1.4-6.el9.noarch.rpm 204 kB/s | 172 kB 00:00 2024-07-17T06:52:10.280 INFO:teuthology.orchestra.run.smithi175.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-07-17T06:52:10.472 INFO:teuthology.orchestra.run.smithi175.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 266 kB/s | 250 kB 00:00 2024-07-17T06:52:10.680 INFO:teuthology.orchestra.run.smithi175.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 224 kB/s | 222 kB 00:00 2024-07-17T06:52:11.064 INFO:teuthology.orchestra.run.smithi175.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 99 kB/s | 77 kB 00:00 2024-07-17T06:52:11.139 INFO:teuthology.orchestra.run.smithi175.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-07-17T06:52:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 302 kB/s | 269 kB 00:00 2024-07-17T06:52:11.890 INFO:teuthology.orchestra.run.smithi175.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 192 kB/s | 159 kB 00:00 2024-07-17T06:52:12.056 INFO:teuthology.orchestra.run.smithi175.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 304 kB/s | 279 kB 00:00 2024-07-17T06:52:12.248 INFO:teuthology.orchestra.run.smithi175.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-07-17T06:52:12.674 INFO:teuthology.orchestra.run.smithi175.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 68 kB/s | 54 kB 00:00 2024-07-17T06:52:12.999 INFO:teuthology.orchestra.run.smithi175.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 56 kB/s | 42 kB 00:00 2024-07-17T06:52:13.341 INFO:teuthology.orchestra.run.smithi175.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 66 kB/s | 44 kB 00:00 2024-07-17T06:52:13.808 INFO:teuthology.orchestra.run.smithi175.stdout:(95/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.3 MB/s | 16 MB 00:01 2024-07-17T06:52:13.841 INFO:teuthology.orchestra.run.smithi175.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 262 kB/s | 221 kB 00:00 2024-07-17T06:52:14.183 INFO:teuthology.orchestra.run.smithi175.stdout:(97/160): qatlib-service-24.02.0-1.el9.x86_64.r 43 kB/s | 36 kB 00:00 2024-07-17T06:52:14.516 INFO:teuthology.orchestra.run.smithi175.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 67 kB/s | 47 kB 00:00 2024-07-17T06:52:14.733 INFO:teuthology.orchestra.run.smithi175.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 341 kB/s | 304 kB 00:00 2024-07-17T06:52:14.942 INFO:teuthology.orchestra.run.smithi175.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 76 kB/s | 58 kB 00:00 2024-07-17T06:52:15.276 INFO:teuthology.orchestra.run.smithi175.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 126 kB/s | 96 kB 00:00 2024-07-17T06:52:15.384 INFO:teuthology.orchestra.run.smithi175.stdout:(102/160): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-07-17T06:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout:(103/160): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-07-17T06:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout:(104/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 74 kB/s | 64 kB 00:00 2024-07-17T06:52:15.734 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-07-17T06:52:15.743 INFO:teuthology.orchestra.run.smithi175.stdout:(105/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 56 kB/s | 45 kB 00:00 2024-07-17T06:52:15.831 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-17T06:52:15.835 INFO:teuthology.orchestra.run.smithi175.stdout:(106/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-07-17T06:52:15.852 INFO:teuthology.orchestra.run.smithi175.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 188 kB/s | 19 kB 00:00 2024-07-17T06:52:15.869 INFO:teuthology.orchestra.run.smithi175.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-07-17T06:52:15.895 INFO:teuthology.orchestra.run.smithi175.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-17T06:52:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-17T06:52:15.928 INFO:teuthology.orchestra.run.smithi175.stdout:(110/160): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-07-17T06:52:15.962 INFO:teuthology.orchestra.run.smithi175.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-07-17T06:52:15.995 INFO:teuthology.orchestra.run.smithi175.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-07-17T06:52:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-17T06:52:16.062 INFO:teuthology.orchestra.run.smithi175.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-07-17T06:52:16.079 INFO:teuthology.orchestra.run.smithi175.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.7 MB/s | 43 kB 00:00 2024-07-17T06:52:16.096 INFO:teuthology.orchestra.run.smithi175.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 2.0 MB/s | 32 kB 00:00 2024-07-17T06:52:16.113 INFO:teuthology.orchestra.run.smithi175.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 894 kB/s | 14 kB 00:00 2024-07-17T06:52:16.114 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-17T06:52:16.151 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-17T06:52:16.155 INFO:teuthology.orchestra.run.smithi175.stdout:(117/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-07-17T06:52:16.189 INFO:teuthology.orchestra.run.smithi175.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-07-17T06:52:16.230 INFO:teuthology.orchestra.run.smithi175.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-07-17T06:52:16.258 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-17T06:52:16.264 INFO:teuthology.orchestra.run.smithi175.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-07-17T06:52:16.398 INFO:teuthology.orchestra.run.smithi175.stdout:(121/160): python3-google-auth-2.30.0-1.el9.noa 1.3 MB/s | 213 kB 00:00 2024-07-17T06:52:16.540 INFO:teuthology.orchestra.run.smithi175.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-07-17T06:52:16.565 INFO:teuthology.orchestra.run.smithi175.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 443 kB/s | 11 kB 00:00 2024-07-17T06:52:16.590 INFO:teuthology.orchestra.run.smithi175.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 732 kB/s | 18 kB 00:00 2024-07-17T06:52:16.616 INFO:teuthology.orchestra.run.smithi175.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 961 kB/s | 23 kB 00:00 2024-07-17T06:52:16.641 INFO:teuthology.orchestra.run.smithi175.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 805 kB/s | 19 kB 00:00 2024-07-17T06:52:16.666 INFO:teuthology.orchestra.run.smithi175.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 816 kB/s | 20 kB 00:00 2024-07-17T06:52:16.699 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-17T06:52:16.759 INFO:teuthology.orchestra.run.smithi175.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-07-17T06:52:16.784 INFO:teuthology.orchestra.run.smithi175.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-17T06:52:16.806 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:52:16.817 INFO:teuthology.orchestra.run.smithi175.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.4 MB/s | 79 kB 00:00 2024-07-17T06:52:16.851 INFO:teuthology.orchestra.run.smithi175.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-07-17T06:52:16.893 INFO:teuthology.orchestra.run.smithi175.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 6.5 MB/s | 272 kB 00:00 2024-07-17T06:52:16.952 INFO:teuthology.orchestra.run.smithi175.stdout:(133/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-07-17T06:52:16.977 INFO:teuthology.orchestra.run.smithi175.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-07-17T06:52:17.002 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:52:17.010 INFO:teuthology.orchestra.run.smithi175.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 883 kB/s | 28 kB 00:00 2024-07-17T06:52:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:52:17.036 INFO:teuthology.orchestra.run.smithi175.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-07-17T06:52:17.061 INFO:teuthology.orchestra.run.smithi175.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-07-17T06:52:17.095 INFO:teuthology.orchestra.run.smithi175.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-07-17T06:52:17.120 INFO:teuthology.orchestra.run.smithi175.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-07-17T06:52:17.153 INFO:teuthology.orchestra.run.smithi175.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-07-17T06:52:17.173 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:52:17.179 INFO:teuthology.orchestra.run.smithi175.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 882 kB/s | 51 kB 00:00 2024-07-17T06:52:17.213 INFO:teuthology.orchestra.run.smithi175.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.7 MB/s | 90 kB 00:00 2024-07-17T06:52:17.215 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 10/172 2024-07-17T06:52:17.246 INFO:teuthology.orchestra.run.smithi175.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-07-17T06:52:17.271 INFO:teuthology.orchestra.run.smithi175.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 818 kB/s | 20 kB 00:00 2024-07-17T06:52:17.280 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-17T06:52:17.380 INFO:teuthology.orchestra.run.smithi175.stdout:(145/160): libarrow-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-07-17T06:52:17.413 INFO:teuthology.orchestra.run.smithi175.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-07-17T06:52:17.455 INFO:teuthology.orchestra.run.smithi175.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-07-17T06:52:17.556 INFO:teuthology.orchestra.run.smithi175.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-07-17T06:52:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-17T06:52:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-17T06:52:17.671 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:52:17.739 INFO:teuthology.orchestra.run.smithi175.stdout:(149/160): librbd1-19.0.0-4937.g0325fd3c.el9.x8 11 MB/s | 3.2 MB 00:00 2024-07-17T06:52:17.815 INFO:teuthology.orchestra.run.smithi175.stdout:(150/160): librados2-19.0.0-4937.g0325fd3c.el9. 8.6 MB/s | 3.4 MB 00:00 2024-07-17T06:52:17.838 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:52:17.872 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 15/172 2024-07-17T06:52:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 15/172 2024-07-17T06:52:17.981 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-17T06:52:18.053 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-17T06:52:18.081 INFO:teuthology.orchestra.run.smithi175.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 153 kB/s | 41 kB 00:00 2024-07-17T06:52:18.334 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-17T06:52:18.357 INFO:teuthology.orchestra.run.smithi175.stdout:(152/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.9 MB/s | 2.3 MB 00:00 2024-07-17T06:52:18.507 INFO:teuthology.orchestra.run.smithi175.stdout:(153/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-07-17T06:52:18.510 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-17T06:52:18.582 INFO:teuthology.orchestra.run.smithi175.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 495 kB/s | 247 kB 00:00 2024-07-17T06:52:18.616 INFO:teuthology.orchestra.run.smithi175.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 142 kB/s | 37 kB 00:00 2024-07-17T06:52:18.922 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-17T06:52:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-17T06:52:19.174 INFO:teuthology.orchestra.run.smithi175.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 310 kB/s | 207 kB 00:00 2024-07-17T06:52:19.187 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-17T06:52:19.307 INFO:teuthology.orchestra.run.smithi175.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 2.0 MB/s | 1.4 MB 00:00 2024-07-17T06:52:19.333 INFO:teuthology.orchestra.run.smithi175.stdout:(158/160): elfutils-libs-0.191-4.el9.x86_64.rpm 346 kB/s | 260 kB 00:00 2024-07-17T06:52:19.380 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:19.476 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:19.778 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-17T06:52:20.001 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-17T06:52:20.056 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-17T06:52:20.187 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-17T06:52:20.258 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-17T06:52:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-17T06:52:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-17T06:52:20.428 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-17T06:52:20.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-17T06:52:20.590 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-17T06:52:20.670 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-17T06:52:20.720 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-17T06:52:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-17T06:52:20.875 INFO:teuthology.orchestra.run.smithi175.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 1.4 MB/s | 2.4 MB 00:01 2024-07-17T06:52:20.909 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-17T06:52:21.065 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9. 35/172 2024-07-17T06:52:21.142 INFO:teuthology.orchestra.run.smithi175.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 604 kB/s | 1.1 MB 00:01 2024-07-17T06:52:21.156 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-17T06:52:21.157 INFO:teuthology.orchestra.run.smithi175.stdout:Total 7.8 MB/s | 240 MB 00:30 2024-07-17T06:52:21.281 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el 36/172 2024-07-17T06:52:21.317 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_6 37/172 2024-07-17T06:52:21.453 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-17T06:52:21.522 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-17T06:52:21.639 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-17T06:52:21.833 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-17T06:52:21.904 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-17T06:52:22.636 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-17T06:52:22.745 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-17T06:52:22.778 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-17T06:52:22.807 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-17T06:52:22.870 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-17T06:52:22.919 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_6 48/172 2024-07-17T06:52:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-17T06:52:23.239 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-17T06:52:23.325 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-17T06:52:23.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-17T06:52:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-17T06:52:23.673 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-17T06:52:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-17T06:52:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-17T06:52:23.831 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-17T06:52:23.881 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-17T06:52:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-17T06:52:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-17T06:52:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-17T06:52:24.264 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-17T06:52:24.309 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-17T06:52:24.381 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-07-17T06:52:24.443 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-17T06:52:24.489 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-17T06:52:24.552 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-17T06:52:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-17T06:52:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-17T06:52:24.665 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-07-17T06:52:24.665 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-07-17T06:52:24.835 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-17T06:52:25.026 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-17T06:52:25.423 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-17T06:52:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-17T06:52:25.551 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-17T06:52:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-17T06:52:25.688 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-17T06:52:25.744 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-17T06:52:25.807 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-17T06:52:25.857 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-17T06:52:25.942 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-17T06:52:25.985 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-17T06:52:26.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-17T06:52:26.198 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-17T06:52:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-17T06:52:26.404 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-17T06:52:26.493 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-17T06:52:27.340 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-07-17T06:52:27.340 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-07-17T06:52:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-17T06:52:27.571 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-17T06:52:27.604 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-17T06:52:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-17T06:52:27.797 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-17T06:52:27.859 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-17T06:52:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-17T06:52:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-17T06:52:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-17T06:52:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-17T06:52:28.403 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-17T06:52:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-17T06:52:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-17T06:52:31.179 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-17T06:52:31.302 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-17T06:52:31.339 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-17T06:52:31.374 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-17T06:52:31.775 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-17T06:52:31.832 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 104/172 2024-07-17T06:52:31.958 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 104/172 2024-07-17T06:52:32.006 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 105/172 2024-07-17T06:52:32.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-17T06:52:32.669 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-17T06:52:32.710 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 108/172 2024-07-17T06:52:32.810 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 108/172 2024-07-17T06:52:32.837 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 109/172 2024-07-17T06:52:33.993 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-07-17T06:52:34.090 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-17T06:52:34.167 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-17T06:52:34.260 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-17T06:52:34.314 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-17T06:52:34.348 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-17T06:52:34.450 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-17T06:52:34.898 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-17T06:52:35.013 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:52:35.202 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:52:35.231 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:52:35.358 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:52:35.397 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 10/172 2024-07-17T06:52:35.461 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-17T06:52:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:35.692 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:35.759 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-17T06:52:35.823 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-17T06:52:35.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:52:36.015 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:52:36.045 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 15/172 2024-07-17T06:52:36.130 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 15/172 2024-07-17T06:52:36.162 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-17T06:52:36.241 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-17T06:52:36.379 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:36.421 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 111/172 2024-07-17T06:52:36.550 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-17T06:52:36.599 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 111/172 2024-07-17T06:52:36.599 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-17T06:52:36.599 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-17T06:52:36.599 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:36.638 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 112/172 2024-07-17T06:52:36.715 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-17T06:52:37.086 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-17T06:52:37.263 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-17T06:52:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-17T06:52:37.515 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:37.624 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-17T06:52:38.148 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-17T06:52:38.187 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-17T06:52:38.242 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-17T06:52:38.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-17T06:52:38.415 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-17T06:52:38.459 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-17T06:52:38.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-17T06:52:38.571 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-17T06:52:38.610 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-17T06:52:38.726 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-17T06:52:38.789 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-17T06:52:38.838 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-17T06:52:38.892 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-17T06:52:39.003 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-17T06:52:39.041 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9. 35/172 2024-07-17T06:52:39.090 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el 36/172 2024-07-17T06:52:39.130 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_6 37/172 2024-07-17T06:52:39.246 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-17T06:52:39.310 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-17T06:52:39.420 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-17T06:52:39.593 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-17T06:52:39.653 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-17T06:52:40.338 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-17T06:52:40.433 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-17T06:52:40.478 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-17T06:52:40.516 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-17T06:52:40.581 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-17T06:52:40.628 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_6 48/172 2024-07-17T06:52:40.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-17T06:52:40.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-17T06:52:40.980 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-17T06:52:41.100 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-17T06:52:41.198 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-17T06:52:41.287 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-17T06:52:41.325 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-17T06:52:41.363 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-17T06:52:41.435 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-17T06:52:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-17T06:52:41.541 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-17T06:52:41.600 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-17T06:52:41.796 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-17T06:52:41.840 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-17T06:52:41.886 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-17T06:52:42.014 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-17T06:52:42.065 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-17T06:52:42.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-17T06:52:42.178 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-17T06:52:42.214 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-17T06:52:42.414 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-17T06:52:42.585 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-17T06:52:42.636 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-17T06:52:42.673 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-17T06:52:42.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-17T06:52:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-17T06:52:42.822 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-17T06:52:42.865 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-17T06:52:42.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-17T06:52:42.942 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-17T06:52:43.007 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-17T06:52:43.049 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-17T06:52:43.187 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-17T06:52:43.250 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-17T06:52:43.300 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-17T06:52:43.452 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-17T06:52:43.520 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-17T06:52:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-17T06:52:44.528 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-17T06:52:44.562 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-17T06:52:44.679 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-17T06:52:44.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-17T06:52:44.789 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-17T06:52:44.896 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-17T06:52:44.938 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-17T06:52:45.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-17T06:52:45.202 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-17T06:52:45.307 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-17T06:52:46.119 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-17T06:52:46.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-17T06:52:47.904 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-17T06:52:48.000 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-17T06:52:48.036 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-17T06:52:48.071 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-17T06:52:48.443 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-17T06:52:48.488 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 104/172 2024-07-17T06:52:48.609 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 104/172 2024-07-17T06:52:48.645 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 105/172 2024-07-17T06:52:48.723 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-17T06:52:49.269 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-17T06:52:49.308 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 108/172 2024-07-17T06:52:49.432 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 108/172 2024-07-17T06:52:49.468 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 109/172 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 112/172 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-07-17T06:52:49.689 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-07-17T06:52:49.690 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-07-17T06:52:49.690 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:49.913 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 113/172 2024-07-17T06:52:49.965 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 113/172 2024-07-17T06:52:49.965 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:52:49.965 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-17T06:52:49.965 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:51.631 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 114/172 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 114/172 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-17T06:52:51.714 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:51.965 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 115/172 2024-07-17T06:52:52.034 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 115/172 2024-07-17T06:52:52.035 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:52:52.035 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-17T06:52:52.035 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-17T06:52:52.035 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-17T06:52:52.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:52.110 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:52.164 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:52.557 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 116/172 2024-07-17T06:52:52.630 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 116/172 2024-07-17T06:52:52.630 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:52:52.631 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-17T06:52:52.631 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-17T06:52:52.631 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-17T06:52:52.631 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:52:52.784 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-17T06:52:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 110/172 2024-07-17T06:52:52.850 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-17T06:52:52.933 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-17T06:52:53.002 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 111/172 2024-07-17T06:52:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 111/172 2024-07-17T06:52:53.161 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-17T06:52:53.161 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-17T06:52:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:52:53.194 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 112/172 2024-07-17T06:52:53.203 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-17T06:52:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-17T06:52:53.357 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-17T06:52:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-17T06:52:53.671 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-17T06:52:53.729 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-17T06:52:53.769 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-17T06:52:53.801 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-17T06:52:54.474 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 128/172 2024-07-17T06:52:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 128/172 2024-07-17T06:52:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.e 129/172 2024-07-17T06:52:54.696 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c. 130/172 2024-07-17T06:52:54.740 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 131/172 2024-07-17T06:52:58.666 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 131/172 2024-07-17T06:52:58.739 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 132/172 2024-07-17T06:52:59.905 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 132/172 2024-07-17T06:52:59.955 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 133/172 2024-07-17T06:53:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 133/172 2024-07-17T06:53:00.313 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el 134/172 2024-07-17T06:53:00.357 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 135/172 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 135/172 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-17T06:53:00.441 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:00.516 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 136/172 2024-07-17T06:53:00.557 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 136/172 2024-07-17T06:53:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 137/172 2024-07-17T06:53:00.745 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 138/172 2024-07-17T06:53:01.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-17T06:53:01.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 140/172 2024-07-17T06:53:01.950 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 140/172 2024-07-17T06:53:01.950 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:01.950 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-17T06:53:01.950 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-17T06:53:01.950 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-17T06:53:01.951 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:01.994 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4937.g0325f 141/172 2024-07-17T06:53:02.064 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4937.g0325f 141/172 2024-07-17T06:53:02.064 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:02.064 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-17T06:53:02.064 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:02.144 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 142/172 2024-07-17T06:53:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 142/172 2024-07-17T06:53:02.222 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:02.222 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-17T06:53:02.222 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-17T06:53:02.222 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-17T06:53:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 143/172 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 143/172 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-17T06:53:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:05.832 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 112/172 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-07-17T06:53:05.833 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:06.073 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 113/172 2024-07-17T06:53:06.123 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 113/172 2024-07-17T06:53:06.123 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:06.123 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-17T06:53:06.124 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:07.744 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 114/172 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 114/172 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-17T06:53:07.820 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:08.051 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 144/172 2024-07-17T06:53:08.085 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 115/172 2024-07-17T06:53:08.111 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 145/172 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 115/172 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-17T06:53:08.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:08.167 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 146/172 2024-07-17T06:53:08.200 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-17T06:53:08.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-17T06:53:08.549 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 149/172 2024-07-17T06:53:08.606 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-17T06:53:08.638 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_ 151/172 2024-07-17T06:53:08.644 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 116/172 2024-07-17T06:53:08.678 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-17T06:53:08.710 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 116/172 2024-07-17T06:53:08.710 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:08.711 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-17T06:53:08.711 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-17T06:53:08.711 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-17T06:53:08.711 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:08.731 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-17T06:53:08.779 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-17T06:53:08.844 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-17T06:53:08.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-17T06:53:08.988 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-17T06:53:09.233 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-17T06:53:09.290 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-17T06:53:09.374 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-17T06:53:09.483 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-17T06:53:09.632 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:09.648 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-17T06:53:09.684 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:09.697 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-17T06:53:09.755 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:09.756 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-17T06:53:09.788 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-17T06:53:10.370 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 128/172 2024-07-17T06:53:10.408 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 128/172 2024-07-17T06:53:10.463 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.e 129/172 2024-07-17T06:53:10.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c. 130/172 2024-07-17T06:53:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 131/172 2024-07-17T06:53:11.199 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 156/172 2024-07-17T06:53:11.302 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-17T06:53:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-17T06:53:11.510 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-17T06:53:11.553 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-17T06:53:11.586 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-17T06:53:11.621 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-17T06:53:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-17T06:53:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-17T06:53:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-17T06:53:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-17T06:53:11.825 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-17T06:53:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-17T06:53:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-17T06:53:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:12.372 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:12.372 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-17T06:53:12.517 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-17T06:53:12.569 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-17T06:53:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-17T06:53:12.752 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:13.034 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:13.034 INFO:teuthology.orchestra.run.smithi044.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-07-17T06:53:13.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:14.221 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 131/172 2024-07-17T06:53:14.291 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 132/172 2024-07-17T06:53:15.384 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 132/172 2024-07-17T06:53:15.415 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 133/172 2024-07-17T06:53:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 133/172 2024-07-17T06:53:15.746 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el 134/172 2024-07-17T06:53:15.792 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 135/172 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 135/172 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-17T06:53:15.860 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:15.927 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 136/172 2024-07-17T06:53:15.964 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 136/172 2024-07-17T06:53:16.109 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 137/172 2024-07-17T06:53:16.191 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 138/172 2024-07-17T06:53:17.263 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-17T06:53:17.295 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 140/172 2024-07-17T06:53:17.361 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 140/172 2024-07-17T06:53:17.361 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:17.361 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-17T06:53:17.362 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-17T06:53:17.362 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-17T06:53:17.362 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:17.412 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4937.g0325f 141/172 2024-07-17T06:53:17.477 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4937.g0325f 141/172 2024-07-17T06:53:17.478 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:17.478 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-17T06:53:17.478 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:17.546 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 142/172 2024-07-17T06:53:17.620 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 142/172 2024-07-17T06:53:17.620 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:17.620 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-17T06:53:17.620 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-17T06:53:17.620 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-17T06:53:17.621 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:18.022 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 143/172 2024-07-17T06:53:18.091 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 143/172 2024-07-17T06:53:18.092 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-17T06:53:18.092 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-17T06:53:18.092 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-17T06:53:18.092 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-17T06:53:18.092 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 1/172 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 2/172 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 3/172 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 4/172 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4937.g0325f 5/172 2024-07-17T06:53:19.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 6/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 7/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 10/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 11/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 12/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 13/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_ 15/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 16/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 17/172 2024-07-17T06:53:19.391 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_6 18/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 19/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 20/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el 21/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9. 22/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_6 23/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 24/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 25/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 26/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 27/172 2024-07-17T06:53:19.392 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 28/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 29/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c. 30/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 31/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 32/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 33/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el 34/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 35/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.e 36/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 37/172 2024-07-17T06:53:19.393 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 38/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 39/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-17T06:53:19.394 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-17T06:53:19.395 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-17T06:53:19.396 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-17T06:53:19.397 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-17T06:53:19.398 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-17T06:53:19.399 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-17T06:53:19.400 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-17T06:53:19.401 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-17T06:53:19.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 149/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 151/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-17T06:53:19.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-17T06:53:19.404 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-17T06:53:19.405 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-17T06:53:19.405 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-17T06:53:19.405 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-17T06:53:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-17T06:53:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-17T06:53:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-17T06:53:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-17T06:53:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-17T06:53:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-17T06:53:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-17T06:53:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-17T06:53:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-17T06:53:21.737 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:21.737 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-07-17T06:53:22.421 DEBUG:teuthology.parallel:result is None 2024-07-17T06:53:23.155 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 144/172 2024-07-17T06:53:23.211 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 145/172 2024-07-17T06:53:23.276 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 146/172 2024-07-17T06:53:23.312 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-17T06:53:23.350 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-17T06:53:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 149/172 2024-07-17T06:53:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-17T06:53:23.740 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_ 151/172 2024-07-17T06:53:23.781 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-17T06:53:23.830 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-17T06:53:23.882 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-17T06:53:24.782 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:24.844 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:24.899 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-17T06:53:26.248 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 156/172 2024-07-17T06:53:26.362 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-17T06:53:26.418 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-17T06:53:26.545 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-17T06:53:26.587 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-17T06:53:26.620 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-17T06:53:26.654 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-17T06:53:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-17T06:53:26.686 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-17T06:53:26.809 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-17T06:53:26.850 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-17T06:53:26.850 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-17T06:53:27.012 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-17T06:53:27.042 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-17T06:53:27.076 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:27.076 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:27.413 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-17T06:53:27.414 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-17T06:53:27.559 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-17T06:53:27.604 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-17T06:53:27.651 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-17T06:53:27.781 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:28.023 INFO:teuthology.orchestra.run.smithi175.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-07-17T06:53:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 1/172 2024-07-17T06:53:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 2/172 2024-07-17T06:53:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 3/172 2024-07-17T06:53:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 4/172 2024-07-17T06:53:34.169 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4937.g0325f 5/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 6/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 7/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 8/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 9/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 10/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 11/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 12/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 13/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 14/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_ 15/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 16/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 17/172 2024-07-17T06:53:34.170 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_6 18/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86 19/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 20/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el 21/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9. 22/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_6 23/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 24/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 25/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 26/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 27/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 28/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 29/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c. 30/172 2024-07-17T06:53:34.171 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noa 31/172 2024-07-17T06:53:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.n 32/172 2024-07-17T06:53:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4937.g032 33/172 2024-07-17T06:53:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el 34/172 2024-07-17T06:53:34.172 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 35/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.e 36/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 37/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 38/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 39/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-17T06:53:34.183 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-17T06:53:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-17T06:53:34.185 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-17T06:53:34.186 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-17T06:53:34.187 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-17T06:53:34.188 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-17T06:53:34.189 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-17T06:53:34.190 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-17T06:53:34.191 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 149/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 151/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-17T06:53:34.192 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-17T06:53:34.193 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-17T06:53:36.388 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-17T06:53:36.389 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.390 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.391 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-4937.g0325fd3c.el9.noarch 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-17T06:53:36.392 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.393 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-17T06:53:36.394 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.395 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-17T06:53:36.396 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-17T06:53:36.397 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.398 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-17T06:53:36.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-4937.g0325fd3c.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-17T06:53:36.400 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-17T06:53:36.401 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-17T06:53:36.401 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:36.401 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-07-17T06:53:37.039 DEBUG:teuthology.parallel:result is None 2024-07-17T06:53:37.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-17T06:53:37.190 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-17T06:53:37.266 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-4937.g0325fd3c.el9 2024-07-17T06:53:37.267 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4937.g0325fd3c.el9 2024-07-17T06:53:37.267 INFO:teuthology.task.install:The correct ceph version 19.0.0-4937.g0325fd3c is installed. 2024-07-17T06:53:37.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-07-17T06:53:37.412 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-17T06:53:37.465 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-4937.g0325fd3c.el9 2024-07-17T06:53:37.465 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4937.g0325fd3c.el9 2024-07-17T06:53:37.466 INFO:teuthology.task.install:The correct ceph version 19.0.0-4937.g0325fd3c is installed. 2024-07-17T06:53:37.467 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-17T06:53:37.468 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:37.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-17T06:53:37.566 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:37.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-17T06:53:37.635 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-17T06:53:37.635 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:37.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-07-17T06:53:37.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-17T06:53:37.811 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:37.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-07-17T06:53:37.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-17T06:53:37.988 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-17T06:53:37.989 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:37.989 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-17T06:53:38.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-17T06:53:38.215 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:38.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-17T06:53:38.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-17T06:53:38.388 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-17T06:53:38.388 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:38.388 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-07-17T06:53:38.473 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-17T06:53:38.555 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:38.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-07-17T06:53:38.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-17T06:53:38.721 INFO:teuthology.run_tasks:Running task ceph... 2024-07-17T06:53:38.802 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-07-17T06:53:38.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-07-17T06:53:38.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-07-17T06:53:38.891 INFO:tasks.ceph:Disabling ceph logrotate... 2024-07-17T06:53:38.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-07-17T06:53:38.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-07-17T06:53:38.976 INFO:tasks.ceph:Creating extra log directories... 2024-07-17T06:53:38.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-07-17T06:53:38.981 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-07-17T06:53:39.066 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-07-17T06:53:39.066 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-07-17T06:53:39.066 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-07-17T06:53:39.066 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:39.066 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-07-17T06:53:39.120 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-07-17T06:53:39.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-07-17T06:53:39.459 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:39.459 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-07-17T06:53:39.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-07-17T06:53:39.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-07-17T06:53:39.759 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-07-17T06:53:39.760 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-07-17T06:53:39.760 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap-schedule}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7805141', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-qa-fail-mgr-to-avoid-races-debug', 'sha1': '9c8b7e3d36b252529f99a61f43068893847721b3'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3770, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-qa-fail-mgr-to-avoid-races-debug', 'suite_path': '/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mchangir/ceph.git', 'suite_sha1': '9c8b7e3d36b252529f99a61f43068893847721b3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXhPnFUzMjsuB14IfhLrblh3foLrDMnkvX7aB8u8J/2+IW8JQTm+Cs6JvEc9wkLwfEjzC3n32gPMGxNeHcZCpU=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAnsJZIsoX8SOfKl6iExPGZfkuVG4tXTc+v7VrtFDaGeWAa/Hk2cAYFR/0IUwU21MuPuGNPAbhbGHvXOy9V44Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '5bc696040f54eb999ca02a20c94bfd735f22b93c', 'timestamp': '2024-07-17_06:32:46', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.621193'} 2024-07-17T06:53:39.761 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-07-17T06:53:39.768 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-07-17T06:53:39.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-07-17T06:53:39.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-07-17T06:53:39.920 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:39.921 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-07-17T06:53:39.977 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-17T06:53:39.977 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-07-17T06:53:40.033 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-17T06:53:40.033 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-07-17 06:53:13.507799027 +0000 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-07-17 06:50:25.165973332 +0000 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-07-17 06:50:25.165973332 +0000 2024-07-17T06:53:40.034 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-07-17 06:50:25.165973332 +0000 2024-07-17T06:53:40.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-17T06:53:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-07-17T06:53:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-07-17T06:53:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000342184 s, 1.5 MB/s 2024-07-17T06:53:40.121 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-17T06:53:40.177 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-07-17 06:53:13.508799011 +0000 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-07-17 06:50:25.756964112 +0000 2024-07-17T06:53:40.223 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-07-17 06:50:25.756964112 +0000 2024-07-17T06:53:40.224 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-07-17 06:50:25.756964112 +0000 2024-07-17T06:53:40.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-17T06:53:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-07-17T06:53:40.349 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-07-17T06:53:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000421104 s, 1.2 MB/s 2024-07-17T06:53:40.353 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-17T06:53:40.412 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-07-17 06:53:13.509798995 +0000 2024-07-17T06:53:40.464 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-07-17 06:50:26.379954393 +0000 2024-07-17T06:53:40.465 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-07-17 06:50:26.379954393 +0000 2024-07-17T06:53:40.465 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-07-17 06:50:26.379954393 +0000 2024-07-17T06:53:40.465 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-17T06:53:40.546 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-07-17T06:53:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-07-17T06:53:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000388094 s, 1.3 MB/s 2024-07-17T06:53:40.551 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-17T06:53:40.607 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-07-17 06:53:13.509798995 +0000 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-07-17 06:50:27.007944595 +0000 2024-07-17T06:53:40.660 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-07-17 06:50:27.007944595 +0000 2024-07-17T06:53:40.661 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-07-17 06:50:27.007944595 +0000 2024-07-17T06:53:40.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-17T06:53:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-07-17T06:53:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-07-17T06:53:40.746 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000346211 s, 1.5 MB/s 2024-07-17T06:53:40.750 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-17T06:53:40.806 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:40.806 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:40.806 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-07-17T06:53:40.861 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-17T06:53:40.862 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-17 06:53:28.452837974 +0000 2024-07-17T06:53:40.907 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-17 06:50:33.502552649 +0000 2024-07-17T06:53:40.908 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-17 06:50:33.502552649 +0000 2024-07-17T06:53:40.908 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-17 06:50:33.502552649 +0000 2024-07-17T06:53:40.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-17T06:53:41.022 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-17T06:53:41.022 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-17T06:53:41.023 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000344971 s, 1.5 MB/s 2024-07-17T06:53:41.026 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-17T06:53:41.078 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:41.125 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-17 06:53:28.453837959 +0000 2024-07-17T06:53:41.126 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-17 06:50:34.068543676 +0000 2024-07-17T06:53:41.126 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-17 06:50:34.068543676 +0000 2024-07-17T06:53:41.126 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-17 06:50:34.068543676 +0000 2024-07-17T06:53:41.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-17T06:53:41.195 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-17T06:53:41.195 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-17T06:53:41.195 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000324252 s, 1.6 MB/s 2024-07-17T06:53:41.199 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-17T06:53:41.247 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-07-17T06:53:41.294 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-17 06:53:28.453837959 +0000 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-17 06:50:34.668534165 +0000 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-17 06:50:34.668534165 +0000 2024-07-17T06:53:41.295 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-17 06:50:34.668534165 +0000 2024-07-17T06:53:41.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-17T06:53:41.372 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-17T06:53:41.372 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-17T06:53:41.372 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000406731 s, 1.3 MB/s 2024-07-17T06:53:41.376 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-17T06:53:41.430 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-17 06:53:28.454837943 +0000 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-17 06:50:35.267524670 +0000 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-17 06:50:35.268524654 +0000 2024-07-17T06:53:41.480 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-17 06:50:35.267524670 +0000 2024-07-17T06:53:41.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-17T06:53:41.557 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-17T06:53:41.557 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-17T06:53:41.557 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000316293 s, 1.6 MB/s 2024-07-17T06:53:41.561 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-17T06:53:41.611 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:41.611 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-07-17T06:53:41.611 INFO:tasks.ceph:Generating config... 2024-07-17T06:53:41.612 INFO:tasks.ceph:[client] pid file = 2024-07-17T06:53:41.612 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mgr] debug client = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mgr] debug ms = 1 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mon] debug mon = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mon] debug ms = 1 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mon] debug paxos = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-07-17T06:53:41.613 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-07-17T06:53:41.613 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-07-17T06:53:41.613 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-07-17T06:53:41.613 INFO:tasks.ceph:[osd] debug ms = 1 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] debug osd = 20 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-07-17T06:53:41.614 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-07-17T06:53:41.614 INFO:tasks.ceph:Setting up mon.a... 2024-07-17T06:53:41.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-07-17T06:53:41.748 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-07-17T06:53:41.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-07-17T06:53:41.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-07-17T06:53:41.949 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2024-07-17T06:53:41.949 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-07-17T06:53:41.951 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:41.951 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-07-17T06:53:42.000 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid 4797a2db-a688-4e60-bfb8-d8a9e86b84a3 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 4797a2db-a688-4e60-bfb8-d8a9e86b84a3 2024-07-17T06:53:42.160 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-07-17T06:53:42.161955+0000 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-07-17T06:53:42.161955+0000 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 17 (quincy) 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-07-17T06:53:42.161 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-07-17T06:53:42.164 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-07-17T06:53:42.215 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4797a2db-a688-4e60-bfb8-d8a9e86b84a3... 2024-07-17T06:53:42.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-07-17T06:53:42.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-07-17T06:53:42.355 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-07-17T06:53:42.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on skipped message = False 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 4797a2db-a688-4e60-bfb8-d8a9e86b84a3 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2024-07-17T06:53:42.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug trim objects = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-07-17T06:53:42.360 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 20 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 20 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 10 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-07-17T06:53:42.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: client mount timeout = 30 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug finisher = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-07-17T06:53:42.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-07-17T06:53:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon down mkfs grace = 300 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-07-17T06:53:42.364 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = "" 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-07-17T06:53:42.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-07-17T06:53:42.366 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-07-17T06:53:42.367 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-07-17T06:53:42.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.44,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on skipped message = False 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 4797a2db-a688-4e60-bfb8-d8a9e86b84a3 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-07-17T06:53:42.369 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug trim objects = true 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.370 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-07-17T06:53:42.371 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 20 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 20 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 10 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.372 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 30 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug finisher = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-07-17T06:53:42.373 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.374 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: mon down mkfs grace = 300 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-07-17T06:53:42.375 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-07-17T06:53:42.376 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-07-17T06:53:42.398 INFO:tasks.ceph:Creating admin key on mon.a... 2024-07-17T06:53:42.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-07-17T06:53:42.536 INFO:tasks.ceph:Copying monmap to all nodes... 2024-07-17T06:53:42.536 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:42.536 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-07-17T06:53:42.587 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:42.587 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-07-17T06:53:42.631 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:53:42.631 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:42.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-07-17T06:53:42.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-07-17T06:53:42.806 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:53:42.806 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:42.904 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:53:42.904 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:42.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-07-17T06:53:42.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-07-17T06:53:43.035 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:53:43.035 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:43.131 INFO:tasks.ceph:Setting up mon nodes... 2024-07-17T06:53:43.131 INFO:tasks.ceph:Setting up mgr nodes... 2024-07-17T06:53:43.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-07-17T06:53:43.297 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-07-17T06:53:43.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-07-17T06:53:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-07-17T06:53:43.481 INFO:tasks.ceph:Setting up mds nodes... 2024-07-17T06:53:43.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-07-17T06:53:43.657 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-07-17T06:53:43.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-07-17T06:53:43.750 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-07-17T06:53:43.913 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-07-17T06:53:43.923 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-07-17T06:53:44.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-07-17T06:53:44.156 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-07-17T06:53:44.164 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-07-17T06:53:44.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-07-17T06:53:44.396 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-07-17T06:53:44.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-07-17T06:53:44.479 INFO:tasks.ceph_client:Setting up client nodes... 2024-07-17T06:53:44.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-17T06:53:44.603 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-07-17T06:53:44.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-07-17T06:53:44.769 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-07-17T06:53:44.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-07-17T06:53:44.932 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-07-17T06:53:44.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-07-17T06:53:45.087 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-07-17T06:53:45.125 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-07-17T06:53:45.126 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-07-17T06:53:45.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-07-17T06:53:45.216 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:45.216 INFO:tasks.ceph:role: osd.0 2024-07-17T06:53:45.216 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:53:45.217 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-07-17T06:53:45.312 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:53:45.313 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:53:45.314 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:53:45.314 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:53:45.720 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-07-17T06:53:45.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-07-17T06:53:45.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-07-17T06:53:45.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-07-17T06:53:45.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-07-17T06:53:46.053 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:46.053 INFO:tasks.ceph:role: osd.1 2024-07-17T06:53:46.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:53:46.053 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:53:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:53:46.142 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:53:46.142 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:53:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:53:46.142 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:53:46.497 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-07-17T06:53:46.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-07-17T06:53:46.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-07-17T06:53:46.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-07-17T06:53:46.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-07-17T06:53:46.803 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:46.803 INFO:tasks.ceph:role: osd.2 2024-07-17T06:53:46.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:53:46.803 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-07-17T06:53:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:53:46.898 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:53:46.899 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:53:47.289 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-07-17T06:53:47.296 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-07-17T06:53:47.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-07-17T06:53:47.463 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-07-17T06:53:47.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-07-17T06:53:47.643 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-07-17T06:53:47.643 INFO:tasks.ceph:role: osd.3 2024-07-17T06:53:47.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:53:47.643 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-07-17T06:53:47.733 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:53:47.734 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:53:48.135 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-07-17T06:53:48.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-07-17T06:53:48.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-07-17T06:53:48.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-07-17T06:53:48.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:48.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.471+0000 7f022b65d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-07-17T06:53:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.472+0000 7f022b65d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-07-17T06:53:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.472+0000 7f022b65d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-17T06:53:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.472+0000 7f022b65d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-17T06:53:48.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.472+0000 7f022b65d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-17T06:53:48.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:48.475+0000 7f022b65d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-07-17T06:53:51.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-17T06:53:51.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:51.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.930+0000 7f0a6462e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-07-17T06:53:51.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.931+0000 7f0a6462e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-07-17T06:53:51.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.931+0000 7f0a6462e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-17T06:53:51.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.931+0000 7f0a6462e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-17T06:53:51.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.931+0000 7f0a6462e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-17T06:53:51.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:51.933+0000 7f0a6462e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-07-17T06:53:54.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-17T06:53:54.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.130+0000 7f7c0e3a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-07-17T06:53:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.130+0000 7f7c0e3a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-07-17T06:53:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.130+0000 7f7c0e3a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-17T06:53:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.130+0000 7f7c0e3a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-17T06:53:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.130+0000 7f7c0e3a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-17T06:53:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:55.133+0000 7f7c0e3a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-07-17T06:53:58.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-17T06:53:58.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:53:58.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.551+0000 7f5f1dc16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-07-17T06:53:58.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.552+0000 7f5f1dc16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-07-17T06:53:58.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.552+0000 7f5f1dc16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-07-17T06:53:58.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.552+0000 7f5f1dc16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-07-17T06:53:58.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.552+0000 7f5f1dc16640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-07-17T06:53:58.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:53:58.554+0000 7f5f1dc16640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-07-17T06:54:01.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-17T06:54:01.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-07-17T06:54:01.717 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-07-17T06:54:01.717 INFO:tasks.ceph:role: osd.4 2024-07-17T06:54:01.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:54:01.717 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:54:01.804 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:54:02.217 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-07-17T06:54:02.225 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-07-17T06:54:02.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-07-17T06:54:02.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-07-17T06:54:02.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-07-17T06:54:02.496 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-07-17T06:54:02.496 INFO:tasks.ceph:role: osd.5 2024-07-17T06:54:02.496 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:54:02.496 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-07-17T06:54:02.570 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:54:02.571 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:54:02.929 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-07-17T06:54:02.934 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-07-17T06:54:02.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-07-17T06:54:03.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-07-17T06:54:03.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-07-17T06:54:03.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-07-17T06:54:03.160 INFO:tasks.ceph:role: osd.6 2024-07-17T06:54:03.160 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:54:03.160 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-07-17T06:54:03.232 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:54:03.233 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:54:03.613 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-07-17T06:54:03.618 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-07-17T06:54:03.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-07-17T06:54:03.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-07-17T06:54:03.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-07-17T06:54:03.887 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-07-17T06:54:03.887 INFO:tasks.ceph:role: osd.7 2024-07-17T06:54:03.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:54:03.887 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-07-17T06:54:03.969 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-17T06:54:03.970 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-17T06:54:04.393 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-07-17T06:54:04.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-07-17T06:54:04.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-07-17T06:54:04.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-07-17T06:54:04.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:04.797 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.798+0000 7fe67993b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-07-17T06:54:04.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.799+0000 7fe67993b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-07-17T06:54:04.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.799+0000 7fe67993b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-07-17T06:54:04.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.799+0000 7fe67993b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-07-17T06:54:04.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.800+0000 7fe67993b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-07-17T06:54:04.802 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:04.803+0000 7fe67993b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-07-17T06:54:08.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-17T06:54:08.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:08.256 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.257+0000 7f25ef418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-07-17T06:54:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.257+0000 7f25ef418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-07-17T06:54:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.257+0000 7f25ef418640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-07-17T06:54:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.257+0000 7f25ef418640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-07-17T06:54:08.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.257+0000 7f25ef418640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-07-17T06:54:08.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:08.260+0000 7f25ef418640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-07-17T06:54:09.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:54:09.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:54:11.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-17T06:54:11.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:11.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.442+0000 7f6fc91cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-07-17T06:54:11.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.443+0000 7f6fc91cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-07-17T06:54:11.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.443+0000 7f6fc91cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-07-17T06:54:11.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.443+0000 7f6fc91cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-07-17T06:54:11.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.443+0000 7f6fc91cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-07-17T06:54:11.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:11.446+0000 7f6fc91cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-07-17T06:54:14.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-07-17T06:54:14.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:14.900 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.901+0000 7f69af933640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-07-17T06:54:14.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.901+0000 7f69af933640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-07-17T06:54:14.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.901+0000 7f69af933640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-07-17T06:54:14.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.901+0000 7f69af933640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-07-17T06:54:14.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.901+0000 7f69af933640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-07-17T06:54:14.902 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:14.904+0000 7f69af933640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-07-17T06:54:17.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-07-17T06:54:17.958 INFO:tasks.ceph:Reading keys from all nodes... 2024-07-17T06:54:17.959 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:17.959 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-07-17T06:54:18.051 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-07-17T06:54:18.139 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.139 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-07-17T06:54:18.227 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-07-17T06:54:18.313 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.313 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-07-17T06:54:18.400 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-07-17T06:54:18.491 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:18.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-07-17T06:54:18.576 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:18.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-07-17T06:54:18.658 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:18.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-07-17T06:54:18.732 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:18.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-07-17T06:54:18.856 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:18.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-07-17T06:54:18.938 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:18.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-07-17T06:54:19.064 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:19.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-07-17T06:54:19.145 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:19.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-07-17T06:54:19.222 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:19.222 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-07-17T06:54:19.279 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:19.279 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-07-17T06:54:19.335 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:19.335 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-07-17T06:54:19.387 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:19.387 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-07-17T06:54:19.431 INFO:tasks.ceph:Adding keys to all mons... 2024-07-17T06:54:19.432 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-07-17T06:54:19.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-07-17T06:54:19.514 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2024-07-17T06:54:19.514 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg== 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw== 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg== 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q== 2024-07-17T06:54:19.515 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ== 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ== 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w== 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ== 2024-07-17T06:54:19.516 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q== 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ== 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw== 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ== 2024-07-17T06:54:19.517 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw== 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg== 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout:[client.2] 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA== 2024-07-17T06:54:19.518 INFO:teuthology.orchestra.run.smithi044.stdout:[client.1] 2024-07-17T06:54:19.519 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA== 2024-07-17T06:54:19.519 INFO:teuthology.orchestra.run.smithi044.stdout:[client.3] 2024-07-17T06:54:19.519 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ== 2024-07-17T06:54:19.552 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-07-17T06:54:19.552 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg== 2024-07-17T06:54:19.552 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw== 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg== 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q== 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-07-17T06:54:19.553 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ== 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ== 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w== 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-07-17T06:54:19.554 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ== 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q== 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ== 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-07-17T06:54:19.555 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw== 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ== 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw== 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-07-17T06:54:19.556 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg== 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout:[client.2] 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA== 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA== 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout:[client.3] 2024-07-17T06:54:19.557 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ== 2024-07-17T06:54:19.558 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-07-17T06:54:19.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-07-17T06:54:19.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:19.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:19.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:19.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:20.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-07-17T06:54:20.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-07-17T06:54:20.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:20.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:20.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:20.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-17T06:54:20.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:20.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-17T06:54:21.523 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.662 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:21.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-17T06:54:22.086 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-07-17T06:54:22.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-07-17T06:54:22.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-07-17T06:54:22.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-07-17T06:54:22.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-07-17T06:54:22.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-07-17T06:54:22.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-07-17T06:54:22.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-07-17T06:54:22.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-07-17T06:54:22.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-07-17T06:54:22.983 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:22.990 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-07-17T06:54:23.041 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-07-17T06:54:23.041 INFO:tasks.ceph.mon.a:Restarting daemon 2024-07-17T06:54:23.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-07-17T06:54:23.046 INFO:tasks.ceph.mon.a:Started 2024-07-17T06:54:23.046 INFO:tasks.ceph.mon.b:Restarting daemon 2024-07-17T06:54:23.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-07-17T06:54:23.084 INFO:tasks.ceph.mon.b:Started 2024-07-17T06:54:23.084 INFO:tasks.ceph.mon.c:Restarting daemon 2024-07-17T06:54:23.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-07-17T06:54:23.089 INFO:tasks.ceph.mon.c:Started 2024-07-17T06:54:23.089 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-07-17T06:54:23.089 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-07-17T06:54:23.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-07-17T06:54:23.095 INFO:tasks.ceph.mgr.y:Started 2024-07-17T06:54:23.095 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-07-17T06:54:23.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-07-17T06:54:23.100 INFO:tasks.ceph.mgr.x:Started 2024-07-17T06:54:23.100 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-07-17T06:54:23.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-07-17T06:54:23.105 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-07-17T06:54:23.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client debug_client 20 2024-07-17T06:54:23.151 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-07-17T06:54:23.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client debug_ms 1 2024-07-17T06:54:23.198 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-07-17T06:54:23.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-07-17T06:54:23.242 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-07-17T06:54:23.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-07-17T06:54:23.287 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-07-17T06:54:23.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-07-17T06:54:23.336 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-07-17T06:54:23.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-07-17T06:54:23.382 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2024-07-17T06:54:23.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds debug_ms 1 2024-07-17T06:54:23.429 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-07-17T06:54:23.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-07-17T06:54:23.475 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-07-17T06:54:23.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-07-17T06:54:23.519 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-07-17T06:54:23.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-07-17T06:54:23.566 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-07-17T06:54:23.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-07-17T06:54:23.611 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-07-17T06:54:23.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-07-17T06:54:23.657 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-07-17T06:54:23.657 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-07-17T06:54:23.704 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-07-17T06:54:23.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-07-17T06:54:23.751 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-07-17T06:54:23.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-07-17T06:54:23.796 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-17T06:54:23.796 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-07-17T06:54:23.843 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-07-17T06:54:23.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-07-17T06:54:23.888 DEBUG:tasks.ceph:set 18 configs 2024-07-17T06:54:23.888 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi044') 2024-07-17T06:54:28.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 -- 172.21.15.44:0/1807934977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf4003d20 msgr2=0x7f6d140cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 --2- 172.21.15.44:0/1807934977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf4003d20 0x7f6d140cd680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6d08012a70 tx=0x7f6d08012f30 comp rx=0 tx=0).stop 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 -- 172.21.15.44:0/1807934977 shutdown_connections 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 --2- 172.21.15.44:0/1807934977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cf40099b0 0x7f6d14159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 --2- 172.21.15.44:0/1807934977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cf4004870 0x7f6d141f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 --2- 172.21.15.44:0/1807934977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cf4003d20 0x7f6d140cd680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.911+0000 7f6d1b800640 1 -- 172.21.15.44:0/1807934977 >> 172.21.15.44:0/1807934977 conn(0x7f6d140d3700 msgr2=0x7f6d1415c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:28.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.912+0000 7f6d1b800640 1 -- 172.21.15.44:0/1807934977 shutdown_connections 2024-07-17T06:54:28.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.912+0000 7f6d1b800640 1 -- 172.21.15.44:0/1807934977 wait complete. 2024-07-17T06:54:28.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.913+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f15440050e0 con 0x7f157015a5a0 2024-07-17T06:54:28.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.917+0000 7f156cc00640 1 -- 172.21.15.44:0/2120370765 <== mon.2 v2:172.21.15.175:3301/0 9 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f155c0037a0 con 0x7f157015a5a0 2024-07-17T06:54:28.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.920+0000 7f79eaa00640 1 -- 172.21.15.44:0/3712938024 <== mon.1 v2:172.21.15.175:3300/0 9 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f79e401f800 con 0x7f79d4003690 2024-07-17T06:54:28.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.920+0000 7f79eaa00640 1 -- 172.21.15.44:0/3712938024 <== mon.1 v2:172.21.15.175:3300/0 10 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f79e401fab0 con 0x7f79d4003690 2024-07-17T06:54:28.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 -- 172.21.15.44:0/3712938024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79d4003690 msgr2=0x7f79f4142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:28.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 --2- 172.21.15.44:0/3712938024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79d4003690 0x7f79f4142b90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f79e4012af0 tx=0x7f79e4012ef0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 -- 172.21.15.44:0/3712938024 shutdown_connections 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 --2- 172.21.15.44:0/3712938024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79d8004800 0x7f79f40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 --2- 172.21.15.44:0/3712938024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79d8003a50 0x7f79f4145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 --2- 172.21.15.44:0/3712938024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79d4003690 0x7f79f4142b90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 -- 172.21.15.44:0/3712938024 >> 172.21.15.44:0/3712938024 conn(0x7f79f406d050 msgr2=0x7f79f40b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 -- 172.21.15.44:0/3712938024 shutdown_connections 2024-07-17T06:54:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.922+0000 7f79f9a00640 1 -- 172.21.15.44:0/3712938024 wait complete. 2024-07-17T06:54:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.926+0000 7f156cc00640 1 -- 172.21.15.44:0/2120370765 <== mon.2 v2:172.21.15.175:3301/0 10 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f155c025030 con 0x7f157015a5a0 2024-07-17T06:54:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f157015a5a0 msgr2=0x7f1570093e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 --2- 172.21.15.44:0/2120370765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f157015a5a0 0x7f1570093e30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1550009d50 tx=0x7f155c014df0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 shutdown_connections 2024-07-17T06:54:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 --2- 172.21.15.44:0/2120370765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1570097cf0 0x7f157014ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 --2- 172.21.15.44:0/2120370765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f157015a5a0 0x7f1570093e30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 --2- 172.21.15.44:0/2120370765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1558004d90 0x7f1570150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 >> 172.21.15.44:0/2120370765 conn(0x7f15700b0eb0 msgr2=0x7f1570152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 shutdown_connections 2024-07-17T06:54:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:28.927+0000 7f1577600640 1 -- 172.21.15.44:0/2120370765 wait complete. 2024-07-17T06:54:28.942 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 -- 172.21.15.44:0/2044931100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac003b30 msgr2=0x7f74ac0066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 --2- 172.21.15.44:0/2044931100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac003b30 0x7f74ac0066a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0060d0 tx=0x7f74bc02b7f0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 -- 172.21.15.44:0/2044931100 shutdown_connections 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 --2- 172.21.15.44:0/2044931100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac006be0 0x7f74ac007020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 --2- 172.21.15.44:0/2044931100 >> v2:172.21.15.175:3300/0 conn(0x7f74ac007560 0x7f74ac0079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 --2- 172.21.15.44:0/2044931100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac003b30 0x7f74ac0066a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.381+0000 7f74d2400640 1 -- 172.21.15.44:0/2044931100 >> 172.21.15.44:0/2044931100 conn(0x7f74cc0d3700 msgr2=0x7f74cc0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.382+0000 7f74d2400640 1 -- 172.21.15.44:0/2044931100 shutdown_connections 2024-07-17T06:54:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.382+0000 7f74d2400640 1 -- 172.21.15.44:0/2044931100 wait complete. 2024-07-17T06:54:29.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.385+0000 7f74d2400640 1 Processor -- start 2024-07-17T06:54:29.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.385+0000 7f74d2400640 1 -- start start 2024-07-17T06:54:29.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac003b30 0x7f74cc1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74ac006be0 0x7f74cc0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0ce9a0 con 0x7f74ac007560 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0ce820 con 0x7f74ac006be0 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74d2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74cc0ceb20 con 0x7f74ac003b30 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53554/0 (socket says 172.21.15.44:53554) 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74cbe00640 1 -- 172.21.15.44:0/2514559620 learned_addr learned my addr 172.21.15.44:0/2514559620 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.387+0000 7f74cb400640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac003b30 0x7f74cc1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74caa00640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74ac006be0 0x7f74cc0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74cbe00640 1 -- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac003b30 msgr2=0x7f74cc1f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74cbe00640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac003b30 0x7f74cc1f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74cbe00640 1 -- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74ac006be0 msgr2=0x7f74cc0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74cbe00640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74ac006be0 0x7f74cc0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.388+0000 7f74cbe00640 1 -- 172.21.15.44:0/2514559620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc0cd2c0 con 0x7f74ac007560 2024-07-17T06:54:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74cbe00640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005fd0 tx=0x7f74c400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400ede0 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f74c400fe50 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004d80 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc0cd5a0 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.389+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc1f45e0 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.390+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f74c4022020 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.390+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f74c4004730 con 0x7f74ac007560 2024-07-17T06:54:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.390+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc161770 con 0x7f74ac007560 2024-07-17T06:54:29.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.393+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f74c402a020 con 0x7f74ac007560 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 -- 172.21.15.44:0/2100635546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 msgr2=0x7f77280081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 --2- 172.21.15.44:0/2100635546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77280081c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7748161550 tx=0x7f773402c560 comp rx=0 tx=0).stop 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 -- 172.21.15.44:0/2100635546 shutdown_connections 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 --2- 172.21.15.44:0/2100635546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77280081c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 -- 172.21.15.44:0/2100635546 >> 172.21.15.44:0/2100635546 conn(0x7f77480b0eb0 msgr2=0x7f77480b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 -- 172.21.15.44:0/2100635546 shutdown_connections 2024-07-17T06:54:29.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.450+0000 7f774e600640 1 -- 172.21.15.44:0/2100635546 wait complete. 2024-07-17T06:54:29.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.453+0000 7f774e600640 1 Processor -- start 2024-07-17T06:54:29.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.453+0000 7f774e600640 1 -- start start 2024-07-17T06:54:29.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.454+0000 7f774e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7728003b30 0x7f7748073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.454+0000 7f774e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f774e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 0x7f774809c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f774e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77480b3ce0 con 0x7f7728007d10 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f774e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f774806d990 con 0x7f7728008700 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f774e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77480b3e60 con 0x7f7728003b30 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53574/0 (socket says 172.21.15.44:53574) 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 -- 172.21.15.44:0/4281258456 learned_addr learned my addr 172.21.15.44:0/4281258456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7747400640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7728003b30 0x7f7748073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 -- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7728003b30 msgr2=0x7f7748073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7728003b30 0x7f7748073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 -- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 msgr2=0x7f774809c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7747e00640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 0x7f774809c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 0x7f774809c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7746a00640 1 -- 172.21.15.44:0/4281258456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7748073a90 con 0x7f7728007d10 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.455+0000 7f7747e00640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 0x7f774809c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f7746a00640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7734012d60 tx=0x7f7734005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734041020 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f7734003ba0 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734038040 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7748073d70 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.456+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7748074330 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.457+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7734044020 con 0x7f7728007d10 2024-07-17T06:54:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.457+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f773403dd90 con 0x7f7728007d10 2024-07-17T06:54:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.457+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774806a1a0 con 0x7f7728007d10 2024-07-17T06:54:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.461+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f774806a1a0 con 0x7f7728007d10 2024-07-17T06:54:29.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.465+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f74cc0b42d0 con 0x7f74ac007560 2024-07-17T06:54:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.468+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v16)=0 v16) ==== 114+0+0 (secure 0 0 0) 0x7f74c4018070 con 0x7f74ac007560 2024-07-17T06:54:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.468+0000 7f74c3400640 1 -- 172.21.15.44:0/2514559620 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f74c4003b10 con 0x7f74ac007560 2024-07-17T06:54:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.468+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f7734004230 con 0x7f7728007d10 2024-07-17T06:54:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 msgr2=0x7f74cc0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005fd0 tx=0x7f74c400fc00 comp rx=0 tx=0).stop 2024-07-17T06:54:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 shutdown_connections 2024-07-17T06:54:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74ac007560 0x7f74cc0ca7d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74ac006be0 0x7f74cc0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 --2- 172.21.15.44:0/2514559620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74ac003b30 0x7f74cc1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 >> 172.21.15.44:0/2514559620 conn(0x7f74cc0d3700 msgr2=0x7f74cc1e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 shutdown_connections 2024-07-17T06:54:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.469+0000 7f74d2400640 1 -- 172.21.15.44:0/2514559620 wait complete. 2024-07-17T06:54:29.486 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.487 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.533+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f77480b1290 con 0x7f7728007d10 2024-07-17T06:54:29.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.538+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) ==== 123+0+0 (secure 0 0 0) 0x7f77480b1290 con 0x7f7728007d10 2024-07-17T06:54:29.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.539+0000 7f773f400640 1 -- 172.21.15.44:0/4281258456 <== mon.0 v2:172.21.15.44:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f773403d4c0 con 0x7f7728007d10 2024-07-17T06:54:29.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.539+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 msgr2=0x7f77481f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.539+0000 7f774e600640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7734012d60 tx=0x7f7734005f90 comp rx=0 tx=0).stop 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 shutdown_connections 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7728008700 0x7f774809c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7728007d10 0x7f77481f7950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 --2- 172.21.15.44:0/4281258456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7728003b30 0x7f7748073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 >> 172.21.15.44:0/4281258456 conn(0x7f77480b0eb0 msgr2=0x7f77480aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 shutdown_connections 2024-07-17T06:54:29.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.540+0000 7f774e600640 1 -- 172.21.15.44:0/4281258456 wait complete. 2024-07-17T06:54:29.557 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.557 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.557 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.557 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 -- 172.21.15.44:0/1956556645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c004530 msgr2=0x7f836c008630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 --2- 172.21.15.44:0/1956556645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c004530 0x7f836c008630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f838c161550 tx=0x7f837802b920 comp rx=0 tx=0).stop 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 -- 172.21.15.44:0/1956556645 shutdown_connections 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 --2- 172.21.15.44:0/1956556645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c004530 0x7f836c008630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 --2- 172.21.15.44:0/1956556645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c003b70 0x7f836c003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 -- 172.21.15.44:0/1956556645 >> 172.21.15.44:0/1956556645 conn(0x7f838c0b0eb0 msgr2=0x7f838c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 -- 172.21.15.44:0/1956556645 shutdown_connections 2024-07-17T06:54:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.619+0000 7f8393e00640 1 -- 172.21.15.44:0/1956556645 wait complete. 2024-07-17T06:54:29.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.623+0000 7f8393e00640 1 Processor -- start 2024-07-17T06:54:29.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.623+0000 7f8393e00640 1 -- start start 2024-07-17T06:54:29.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.625+0000 7f8393e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.625+0000 7f8393e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c004530 0x7f838c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8393e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f836c0049f0 0x7f838c146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:29.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8393e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f838c0ac990 con 0x7f836c0049f0 2024-07-17T06:54:29.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8393e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f838c0ac810 con 0x7f836c003b70 2024-07-17T06:54:29.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8393e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f838c15dbe0 con 0x7f836c004530 2024-07-17T06:54:29.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8391400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8391e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f836c0049f0 0x7f838c146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8390a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c004530 0x7f838c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:29.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8391400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:52246/0 (socket says 172.21.15.44:52246) 2024-07-17T06:54:29.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.626+0000 7f8391400640 1 -- 172.21.15.44:0/3445479274 learned_addr learned my addr 172.21.15.44:0/3445479274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 -- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c004530 msgr2=0x7f838c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c004530 0x7f838c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 -- 172.21.15.44:0/3445479274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f836c0049f0 msgr2=0x7f838c146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f836c0049f0 0x7f838c146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 -- 172.21.15.44:0/3445479274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c1f4450 con 0x7f836c003b70 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8391400640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8380003fd0 tx=0x7f8380006670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800126b0 con 0x7f836c003b70 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8380012850 con 0x7f836c003b70 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380012c40 con 0x7f836c003b70 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.627+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838c1f4730 con 0x7f836c003b70 2024-07-17T06:54:29.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.628+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838c1f4cf0 con 0x7f836c003b70 2024-07-17T06:54:29.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.628+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f838001e420 con 0x7f836c003b70 2024-07-17T06:54:29.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.628+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83800279e0 con 0x7f836c003b70 2024-07-17T06:54:29.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.629+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838c0b4060 con 0x7f836c003b70 2024-07-17T06:54:29.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.633+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f8380027be0 con 0x7f836c003b70 2024-07-17T06:54:29.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.703+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f838c160e90 con 0x7f836c003b70 2024-07-17T06:54:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.710+0000 7f8389600640 1 -- 172.21.15.44:0/3445479274 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v18)=0 v18) ==== 106+0+0 (secure 0 0 0) 0x7f8380027dc0 con 0x7f836c003b70 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 msgr2=0x7f838c09c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8380003fd0 tx=0x7f8380006670 comp rx=0 tx=0).stop 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 shutdown_connections 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f836c0049f0 0x7f838c146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f836c004530 0x7f838c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 --2- 172.21.15.44:0/3445479274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f836c003b70 0x7f838c09c9e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 >> 172.21.15.44:0/3445479274 conn(0x7f838c0b0eb0 msgr2=0x7f838c0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 shutdown_connections 2024-07-17T06:54:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:29.712+0000 7f8393e00640 1 -- 172.21.15.44:0/3445479274 wait complete. 2024-07-17T06:54:29.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.730 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.730 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.730 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.730 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi044') 2024-07-17T06:54:29.730 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi044') 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.013+0000 7f69fc000640 1 -- 172.21.15.44:0/3425219328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 msgr2=0x7f69d4004480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.013+0000 7f69fc000640 1 --2- 172.21.15.44:0/3425219328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69d4004480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f69f4070b60 tx=0x7f69ec02a480 comp rx=0 tx=0).stop 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 -- 172.21.15.44:0/3425219328 shutdown_connections 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 --2- 172.21.15.44:0/3425219328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69d4004480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 --2- 172.21.15.44:0/3425219328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69d4003ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 -- 172.21.15.44:0/3425219328 >> 172.21.15.44:0/3425219328 conn(0x7f69f40b0eb0 msgr2=0x7f69f40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 -- 172.21.15.44:0/3425219328 shutdown_connections 2024-07-17T06:54:30.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.014+0000 7f69fc000640 1 -- 172.21.15.44:0/3425219328 wait complete. 2024-07-17T06:54:30.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.016+0000 7f69fc000640 1 Processor -- start 2024-07-17T06:54:30.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.017+0000 7f69fc000640 1 -- start start 2024-07-17T06:54:30.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69f41f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69f4073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69f415c470 con 0x7f69f40cbab0 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69f415c2f0 con 0x7f69d4004000 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fc000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69f415c5f0 con 0x7f69d4006610 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53584/0 (socket says 172.21.15.44:53584) 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.018+0000 7f69fa000640 1 -- 172.21.15.44:0/1407201442 learned_addr learned my addr 172.21.15.44:0/1407201442 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69f9600640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69f41f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69f8c00640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69f4073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 -- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 msgr2=0x7f69f4073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69f4073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 -- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 msgr2=0x7f69f41f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69f41f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 -- 172.21.15.44:0/1407201442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f4159400 con 0x7f69f40cbab0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69f8c00640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69f4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69f9600640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69f41f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.019+0000 7f69fa000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02aa70 tx=0x7f69ec006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec044070 con 0x7f69f40cbab0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ec033480 con 0x7f69f40cbab0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0338b0 con 0x7f69f40cbab0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f41f45e0 con 0x7f69f40cbab0 2024-07-17T06:54:30.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f41f4ba0 con 0x7f69f40cbab0 2024-07-17T06:54:30.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69ec003c70 con 0x7f69f40cbab0 2024-07-17T06:54:30.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.020+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f69ec03f930 con 0x7f69f40cbab0 2024-07-17T06:54:30.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.021+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f40cb4c0 con 0x7f69f40cbab0 2024-07-17T06:54:30.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.023+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f69ec04d020 con 0x7f69f40cbab0 2024-07-17T06:54:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.091+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f69f40cdfd0 con 0x7f69f40cbab0 2024-07-17T06:54:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.096+0000 7f69f1600640 1 -- 172.21.15.44:0/1407201442 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7f69ec03f450 con 0x7f69f40cbab0 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 msgr2=0x7f69f4074100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f69ec02aa70 tx=0x7f69ec006290 comp rx=0 tx=0).stop 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 shutdown_connections 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69f40cbab0 0x7f69f4074100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69d4006610 0x7f69f4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 --2- 172.21.15.44:0/1407201442 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d4004000 0x7f69f41f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 >> 172.21.15.44:0/1407201442 conn(0x7f69f40b0eb0 msgr2=0x7f69f40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 shutdown_connections 2024-07-17T06:54:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.098+0000 7f69fc000640 1 -- 172.21.15.44:0/1407201442 wait complete. 2024-07-17T06:54:30.114 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-07-17T06:54:30.114 INFO:tasks.ceph:Setting crush tunables to default 2024-07-17T06:54:30.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-07-17T06:54:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.303+0000 7f1eb9e00640 1 -- 172.21.15.44:0/4211265949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b29e0 msgr2=0x7f1eb4168020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.303+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/4211265949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b29e0 0x7f1eb4168020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009830 tx=0x7f1eac02a640 comp rx=0 tx=0).stop 2024-07-17T06:54:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.304+0000 7f1eb9e00640 1 -- 172.21.15.44:0/4211265949 shutdown_connections 2024-07-17T06:54:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.304+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/4211265949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b29e0 0x7f1eb4168020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.304+0000 7f1eb9e00640 1 -- 172.21.15.44:0/4211265949 >> 172.21.15.44:0/4211265949 conn(0x7f1eb40d3700 msgr2=0x7f1eb40d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.304+0000 7f1eb9e00640 1 -- 172.21.15.44:0/4211265949 shutdown_connections 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.304+0000 7f1eb9e00640 1 -- 172.21.15.44:0/4211265949 wait complete. 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.305+0000 7f1eb9e00640 1 Processor -- start 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.305+0000 7f1eb9e00640 1 -- start start 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb40b29e0 0x7f1eb41f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ce1d0 0x7f1eb41556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41688f0 con 0x7f1eb40b6790 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4168770 con 0x7f1eb40b29e0 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eb9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb4168a70 con 0x7f1eb40ce1d0 2024-07-17T06:54:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.306+0000 7f1eaa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53600/0 (socket says 172.21.15.44:53600) 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 -- 172.21.15.44:0/2096591297 learned_addr learned my addr 172.21.15.44:0/2096591297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eb2a00640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb40b29e0 0x7f1eb41f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eb3400640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ce1d0 0x7f1eb41556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 -- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ce1d0 msgr2=0x7f1eb41556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ce1d0 0x7f1eb41556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 -- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb40b29e0 msgr2=0x7f1eb41f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb40b29e0 0x7f1eb41f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 -- 172.21.15.44:0/2096591297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb4158c00 con 0x7f1eb40b6790 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.307+0000 7f1eaa000640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c450 tx=0x7f1e9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:30.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0052d0 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e9c005470 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c012870 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb4158ec0 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb4159480 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e9c012a10 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.308+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e9c01b8a0 con 0x7f1eb40b6790 2024-07-17T06:54:30.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.309+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb40ce5b0 con 0x7f1eb40b6790 2024-07-17T06:54:30.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.311+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f1e9c01baa0 con 0x7f1eb40b6790 2024-07-17T06:54:30.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.382+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1eb40d29c0 con 0x7f1eb40b6790 2024-07-17T06:54:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.386+0000 7f1eabe00640 1 -- 172.21.15.44:0/2096591297 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1e9c01bc80 con 0x7f1eb40b6790 2024-07-17T06:54:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-07-17T06:54:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.387+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 msgr2=0x7f1eb41586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.387+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c450 tx=0x7f1e9c00c910 comp rx=0 tx=0).stop 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 shutdown_connections 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ce1d0 0x7f1eb41556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40b6790 0x7f1eb41586c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 --2- 172.21.15.44:0/2096591297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb40b29e0 0x7f1eb41f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 >> 172.21.15.44:0/2096591297 conn(0x7f1eb40d3700 msgr2=0x7f1eb40b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 shutdown_connections 2024-07-17T06:54:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:30.388+0000 7f1eb9e00640 1 -- 172.21.15.44:0/2096591297 wait complete. 2024-07-17T06:54:30.405 INFO:tasks.ceph:check_enable_crimson: False 2024-07-17T06:54:30.405 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-07-17T06:54:30.406 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:30.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-07-17T06:54:30.475 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:30.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-07-17T06:54:30.589 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:30.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-07-17T06:54:30.711 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:54:30.711 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-07-17T06:54:30.792 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:30.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-07-17T06:54:30.864 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:30.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-07-17T06:54:30.975 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:30.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-07-17T06:54:31.088 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:54:31.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-07-17T06:54:31.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c5762a28-1148-44aa-b365-c565843eb11c 0 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.435+0000 7f29cf200640 1 -- 172.21.15.175:0/3457812876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80d1440 msgr2=0x7f29c80d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.435+0000 7f29cf200640 1 --2- 172.21.15.175:0/3457812876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80d1440 0x7f29c80d1820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005fd0 tx=0x7f29b802a490 comp rx=0 tx=0).stop 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.435+0000 7f29cf200640 1 -- 172.21.15.175:0/3457812876 shutdown_connections 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.435+0000 7f29cf200640 1 --2- 172.21.15.175:0/3457812876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c80d1440 0x7f29c80d1820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.435+0000 7f29cf200640 1 --2- 172.21.15.175:0/3457812876 >> v2:172.21.15.44:3300/0 conn(0x7f29c815d350 0x7f29c8161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.436+0000 7f29cf200640 1 -- 172.21.15.175:0/3457812876 >> 172.21.15.175:0/3457812876 conn(0x7f29c80d04c0 msgr2=0x7f29c80d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.436+0000 7f29cf200640 1 -- 172.21.15.175:0/3457812876 shutdown_connections 2024-07-17T06:54:31.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.436+0000 7f29cf200640 1 -- 172.21.15.175:0/3457812876 wait complete. 2024-07-17T06:54:31.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.437+0000 7f29cf200640 1 Processor -- start 2024-07-17T06:54:31.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.437+0000 7f29cf200640 1 -- start start 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cdca0 0x7f29c81f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c80d1440 0x7f29c81f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29c806ead0 con 0x7f29c80d1440 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29c806e950 con 0x7f29c80cdca0 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.438+0000 7f29cf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29c80ce150 con 0x7f29c815d350 2024-07-17T06:54:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29cde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cdca0 0x7f29c81f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:48088/0 (socket says 172.21.15.175:48088) 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 -- 172.21.15.175:0/3099332356 learned_addr learned my addr 172.21.15.175:0/3099332356 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29cd400640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c80d1440 0x7f29c81f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 -- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cdca0 msgr2=0x7f29c81f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cdca0 0x7f29c81f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 -- 172.21.15.175:0/3099332356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c80d1440 msgr2=0x7f29c81f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c80d1440 0x7f29c81f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 -- 172.21.15.175:0/3099332356 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c8146a40 con 0x7f29c815d350 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.439+0000 7f29ce800640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c780 tx=0x7f29c400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004b10 con 0x7f29c815d350 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29c4004da0 con 0x7f29c815d350 2024-07-17T06:54:31.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c400aab0 con 0x7f29c815d350 2024-07-17T06:54:31.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c81f4f90 con 0x7f29c815d350 2024-07-17T06:54:31.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c80cd870 con 0x7f29c815d350 2024-07-17T06:54:31.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29c4006b50 con 0x7f29c815d350 2024-07-17T06:54:31.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.440+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f29c40075e0 con 0x7f29c815d350 2024-07-17T06:54:31.442 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.441+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c80d1820 con 0x7f29c815d350 2024-07-17T06:54:31.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.443+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f29c4020070 con 0x7f29c815d350 2024-07-17T06:54:31.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.522+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c5762a28-1148-44aa-b365-c565843eb11c", "id": 0} v 0) -- 0x7f29c80cd2d0 con 0x7f29c815d350 2024-07-17T06:54:31.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.530+0000 7f29c2000640 1 -- 172.21.15.175:0/3099332356 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c5762a28-1148-44aa-b365-c565843eb11c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f29c81f4f90 con 0x7f29c815d350 2024-07-17T06:54:31.530 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-07-17T06:54:31.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.531+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 msgr2=0x7f29c81587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.531 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.531+0000 7f29cf200640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c780 tx=0x7f29c400cc40 comp rx=0 tx=0).stop 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 shutdown_connections 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29c815d350 0x7f29c81587b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29c80d1440 0x7f29c81f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 --2- 172.21.15.175:0/3099332356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29c80cdca0 0x7f29c81f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 >> 172.21.15.175:0/3099332356 conn(0x7f29c80d04c0 msgr2=0x7f29c80b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 shutdown_connections 2024-07-17T06:54:31.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.532+0000 7f29cf200640 1 -- 172.21.15.175:0/3099332356 wait complete. 2024-07-17T06:54:31.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 72f19265-9816-4d99-a295-93f784f36b66 1 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 -- 172.21.15.175:0/297474360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0073c50 msgr2=0x7f2ae00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/297474360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0073c50 0x7f2ae00b4e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0074750 tx=0x7f2ad802a820 comp rx=0 tx=0).stop 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 -- 172.21.15.175:0/297474360 shutdown_connections 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/297474360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae0073c50 0x7f2ae00b4e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/297474360 >> v2:172.21.15.44:3300/0 conn(0x7f2ae015e470 0x7f2ae00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 -- 172.21.15.175:0/297474360 >> 172.21.15.175:0/297474360 conn(0x7f2ae00d11b0 msgr2=0x7f2ae00d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.733+0000 7f2ae5a00640 1 -- 172.21.15.175:0/297474360 shutdown_connections 2024-07-17T06:54:31.734 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.734+0000 7f2ae5a00640 1 -- 172.21.15.175:0/297474360 wait complete. 2024-07-17T06:54:31.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.736+0000 7f2ae5a00640 1 Processor -- start 2024-07-17T06:54:31.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.736+0000 7f2ae5a00640 1 -- start start 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae015e470 0x7f2ae01f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 0x7f2ae01f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00b5700 con 0x7f2ae015e470 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00b5580 con 0x7f2ae015e850 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae00b5880 con 0x7f2ae0073c50 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2ae5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 0x7f2ae01f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:48104/0 (socket says 172.21.15.175:48104) 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.738+0000 7f2adfe00640 1 -- 172.21.15.175:0/2311783505 learned_addr learned my addr 172.21.15.175:0/2311783505 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2ad7e00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae015e470 0x7f2ae01f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 -- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 msgr2=0x7f2ae01f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 0x7f2ae01f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 -- 172.21.15.175:0/2311783505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae015e470 msgr2=0x7f2ae01f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae015e470 0x7f2ae01f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 -- 172.21.15.175:0/2311783505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae01585f0 con 0x7f2ae0073c50 2024-07-17T06:54:31.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2ae5000640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 0x7f2ae01f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:54:31.740 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adfe00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000d980 tx=0x7f2ad000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0017070 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ad0009d20 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00106c0 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae0153b30 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.739+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae0153820 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.740+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ad0010860 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.740+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2ad000f3c0 con 0x7f2ae0073c50 2024-07-17T06:54:31.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.740+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae00b5da0 con 0x7f2ae0073c50 2024-07-17T06:54:31.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.743+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f2ad0010d30 con 0x7f2ae0073c50 2024-07-17T06:54:31.821 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.821+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "72f19265-9816-4d99-a295-93f784f36b66", "id": 1} v 0) -- 0x7f2ae00cf990 con 0x7f2ae0073c50 2024-07-17T06:54:31.827 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.827+0000 7f2adcc00640 1 -- 172.21.15.175:0/2311783505 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72f19265-9816-4d99-a295-93f784f36b66", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2ad001b400 con 0x7f2ae0073c50 2024-07-17T06:54:31.827 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.828+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 msgr2=0x7f2ae01f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.828+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000d980 tx=0x7f2ad000de40 comp rx=0 tx=0).stop 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 shutdown_connections 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ae015e850 0x7f2ae01f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae015e470 0x7f2ae01f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 --2- 172.21.15.175:0/2311783505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ae0073c50 0x7f2ae01f55a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 >> 172.21.15.175:0/2311783505 conn(0x7f2ae00d11b0 msgr2=0x7f2ae015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 shutdown_connections 2024-07-17T06:54:31.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:31.829+0000 7f2ae5a00640 1 -- 172.21.15.175:0/2311783505 wait complete. 2024-07-17T06:54:31.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 45d9e4e1-1225-4892-a141-48e9874da9f5 2 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 -- 172.21.15.175:0/2877354527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f188806c480 msgr2=0x7f18880726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 --2- 172.21.15.175:0/2877354527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f188806c480 0x7f18880726f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1880005fd0 tx=0x7f188002a8b0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 -- 172.21.15.175:0/2877354527 shutdown_connections 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 --2- 172.21.15.175:0/2877354527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f188806c480 0x7f18880726f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 --2- 172.21.15.175:0/2877354527 >> v2:172.21.15.44:3300/0 conn(0x7f188813e860 0x7f188806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 -- 172.21.15.175:0/2877354527 >> 172.21.15.175:0/2877354527 conn(0x7f188806d050 msgr2=0x7f188806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 -- 172.21.15.175:0/2877354527 shutdown_connections 2024-07-17T06:54:32.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.019+0000 7f1890000640 1 -- 172.21.15.175:0/2877354527 wait complete. 2024-07-17T06:54:32.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.021+0000 7f1890000640 1 Processor -- start 2024-07-17T06:54:32.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.021+0000 7f1890000640 1 -- start start 2024-07-17T06:54:32.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18880afa90 0x7f18881f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f188813e860 0x7f18880ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1888072c30 con 0x7f188813e860 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f188815efa0 con 0x7f188806c480 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f1890000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1888072db0 con 0x7f18880afa90 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f188d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f188cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18880afa90 0x7f18881f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f188d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44340/0 (socket says 172.21.15.175:44340) 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.022+0000 7f188d600640 1 -- 172.21.15.175:0/4100707005 learned_addr learned my addr 172.21.15.175:0/4100707005 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188e000640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f188813e860 0x7f18880ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 -- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18880afa90 msgr2=0x7f18881f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18880afa90 0x7f18881f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 -- 172.21.15.175:0/4100707005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f188813e860 msgr2=0x7f18880ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f188813e860 0x7f18880ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 -- 172.21.15.175:0/4100707005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18881f2830 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f188d600640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c760 tx=0x7f187c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c016020 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f187c00a930 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00ad80 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.023+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18881f2b10 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.024+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18880ad6d0 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.024+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f187c019020 con 0x7f188806c480 2024-07-17T06:54:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.024+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f187c01bc30 con 0x7f188806c480 2024-07-17T06:54:32.026 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.025+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1888073280 con 0x7f188806c480 2024-07-17T06:54:32.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.028+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f187c020020 con 0x7f188806c480 2024-07-17T06:54:32.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.107+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45d9e4e1-1225-4892-a141-48e9874da9f5", "id": 2} v 0) -- 0x7f18880ad8a0 con 0x7f188806c480 2024-07-17T06:54:32.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.114+0000 7f1885600640 1 -- 172.21.15.175:0/4100707005 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45d9e4e1-1225-4892-a141-48e9874da9f5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f18880cf170 con 0x7f188806c480 2024-07-17T06:54:32.114 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-07-17T06:54:32.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 msgr2=0x7f18880a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c760 tx=0x7f187c00cc20 comp rx=0 tx=0).stop 2024-07-17T06:54:32.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 shutdown_connections 2024-07-17T06:54:32.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f188813e860 0x7f18880ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18880afa90 0x7f18881f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 --2- 172.21.15.175:0/4100707005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f188806c480 0x7f18880a9490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 >> 172.21.15.175:0/4100707005 conn(0x7f188806d050 msgr2=0x7f188806ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 shutdown_connections 2024-07-17T06:54:32.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.116+0000 7f1890000640 1 -- 172.21.15.175:0/4100707005 wait complete. 2024-07-17T06:54:32.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c27fa4b5-eeda-4985-b474-37c30575bb4f 3 2024-07-17T06:54:32.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.301+0000 7f9371a00640 1 -- 172.21.15.175:0/3467162980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0b29e0 msgr2=0x7f936c168020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.301+0000 7f9371a00640 1 --2- 172.21.15.175:0/3467162980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0b29e0 0x7f936c168020 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9364009be0 tx=0x7f936402a9b0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.302+0000 7f9371a00640 1 -- 172.21.15.175:0/3467162980 shutdown_connections 2024-07-17T06:54:32.302 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.302+0000 7f9371a00640 1 --2- 172.21.15.175:0/3467162980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0b29e0 0x7f936c168020 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.302+0000 7f9371a00640 1 -- 172.21.15.175:0/3467162980 >> 172.21.15.175:0/3467162980 conn(0x7f936c0d3700 msgr2=0x7f936c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.302+0000 7f9371a00640 1 -- 172.21.15.175:0/3467162980 shutdown_connections 2024-07-17T06:54:32.303 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.302+0000 7f9371a00640 1 -- 172.21.15.175:0/3467162980 wait complete. 2024-07-17T06:54:32.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.303+0000 7f9371a00640 1 Processor -- start 2024-07-17T06:54:32.304 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.304+0000 7f9371a00640 1 -- start start 2024-07-17T06:54:32.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.304+0000 7f9371a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f9371a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f936c0b6790 0x7f936c1f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f9371a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0ce1d0 0x7f936c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.305 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f9371a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1688f0 con 0x7f936c0b6790 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f9371a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f936c168770 con 0x7f936c0b29e0 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f9371a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f936c168a70 con 0x7f936c0ce1d0 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44344/0 (socket says 172.21.15.175:44344) 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 -- 172.21.15.175:0/1399954289 learned_addr learned my addr 172.21.15.175:0/1399954289 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936b400640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0ce1d0 0x7f936c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936a000640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f936c0b6790 0x7f936c1f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 -- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0ce1d0 msgr2=0x7f936c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0ce1d0 0x7f936c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 -- 172.21.15.175:0/1399954289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f936c0b6790 msgr2=0x7f936c1f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.306 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f936c0b6790 0x7f936c1f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.305+0000 7f936aa00640 1 -- 172.21.15.175:0/1399954289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c0c8510 con 0x7f936c0b29e0 2024-07-17T06:54:32.307 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f936aa00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c430 tx=0x7f935c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c011790 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f935c017070 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c1f4950 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c011ce0 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1f87d0 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f935c01c460 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.306+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f935c012c20 con 0x7f936c0b29e0 2024-07-17T06:54:32.308 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.307+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c0d0fa0 con 0x7f936c0b29e0 2024-07-17T06:54:32.309 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.309+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f936c0d0fa0 con 0x7f936c0b29e0 2024-07-17T06:54:32.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.387+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c27fa4b5-eeda-4985-b474-37c30575bb4f", "id": 3} v 0) -- 0x7f936c0ce5b0 con 0x7f936c0b29e0 2024-07-17T06:54:32.392 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.392+0000 7f9362a00640 1 -- 172.21.15.175:0/1399954289 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c27fa4b5-eeda-4985-b474-37c30575bb4f", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f935c012790 con 0x7f936c0b29e0 2024-07-17T06:54:32.392 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-07-17T06:54:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.393+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 msgr2=0x7f936c1f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.393 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.393+0000 7f9371a00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c430 tx=0x7f935c00c8f0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 shutdown_connections 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f936c0ce1d0 0x7f936c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f936c0b6790 0x7f936c1f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 --2- 172.21.15.175:0/1399954289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f936c0b29e0 0x7f936c1f4300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 >> 172.21.15.175:0/1399954289 conn(0x7f936c0d3700 msgr2=0x7f936c1600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 shutdown_connections 2024-07-17T06:54:32.394 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.394+0000 7f9371a00640 1 -- 172.21.15.175:0/1399954289 wait complete. 2024-07-17T06:54:32.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 237302da-caf3-4c4b-a188-f73614d97790 4 2024-07-17T06:54:32.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 -- 172.21.15.175:0/1562340474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 msgr2=0x7fa9a0166e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/1562340474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0166e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9a008c1e0 tx=0x7fa99802ab40 comp rx=0 tx=0).stop 2024-07-17T06:54:32.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 -- 172.21.15.175:0/1562340474 shutdown_connections 2024-07-17T06:54:32.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/1562340474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0166e40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/1562340474 >> v2:172.21.15.44:3300/0 conn(0x7fa9a00b6790 0x7fa9a01648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.578+0000 7fa9a5e00640 1 -- 172.21.15.175:0/1562340474 >> 172.21.15.175:0/1562340474 conn(0x7fa9a00d3700 msgr2=0x7fa9a00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.579+0000 7fa9a5e00640 1 -- 172.21.15.175:0/1562340474 shutdown_connections 2024-07-17T06:54:32.580 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.579+0000 7fa9a5e00640 1 -- 172.21.15.175:0/1562340474 wait complete. 2024-07-17T06:54:32.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.581+0000 7fa9a5e00640 1 Processor -- start 2024-07-17T06:54:32.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.581+0000 7fa9a5e00640 1 -- start start 2024-07-17T06:54:32.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 0x7fa9a015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 0x7fa9a0159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0167710 con 0x7fa9a00b6790 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0167590 con 0x7fa9a00b29e0 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa9a5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a0167890 con 0x7fa9a00ce1d0 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa99ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa99f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 0x7fa9a0159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa99f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 0x7fa9a0159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:48160/0 (socket says 172.21.15.175:48160) 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa99ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44352/0 (socket says 172.21.15.175:44352) 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa99f400640 1 -- 172.21.15.175:0/3653898343 learned_addr learned my addr 172.21.15.175:0/3653898343 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.583+0000 7fa997e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 0x7fa9a015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 -- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 msgr2=0x7fa9a0159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 0x7fa9a0159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 -- 172.21.15.175:0/3653898343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 msgr2=0x7fa9a015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 0x7fa9a015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 -- 172.21.15.175:0/3653898343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a0152db0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa997e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 0x7fa9a015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa99ea00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c3c0 tx=0x7fa99000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990017070 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9900053e0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01558d0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990003850 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.584+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a0156170 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.585+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa99001a070 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.585+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fa990011cc0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.585+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a00ce5b0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.589+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fa990021020 con 0x7fa9a00b29e0 2024-07-17T06:54:32.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.687+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "237302da-caf3-4c4b-a188-f73614d97790", "id": 4} v 0) -- 0x7fa9a015b8e0 con 0x7fa9a00b29e0 2024-07-17T06:54:32.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.694+0000 7fa997400640 1 -- 172.21.15.175:0/3653898343 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "237302da-caf3-4c4b-a188-f73614d97790", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fa990003b30 con 0x7fa9a00b29e0 2024-07-17T06:54:32.694 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-07-17T06:54:32.695 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 msgr2=0x7fa9a0156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c3c0 tx=0x7fa99000c880 comp rx=0 tx=0).stop 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 shutdown_connections 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a00ce1d0 0x7fa9a0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a00b6790 0x7fa9a015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 --2- 172.21.15.175:0/3653898343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a00b29e0 0x7fa9a0156fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.695+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 >> 172.21.15.175:0/3653898343 conn(0x7fa9a00d3700 msgr2=0x7fa9a00b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.696+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 shutdown_connections 2024-07-17T06:54:32.696 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.696+0000 7fa9a5e00640 1 -- 172.21.15.175:0/3653898343 wait complete. 2024-07-17T06:54:32.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 893f49bf-b33f-4a70-b75a-0448b881671d 5 2024-07-17T06:54:32.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.881+0000 7f6032000640 1 -- 172.21.15.175:0/1839775480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6010005210 msgr2=0x7f601000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.881+0000 7f6032000640 1 --2- 172.21.15.175:0/1839775480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6010005210 0x7f601000b9c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f602400c610 tx=0x7f6010005830 comp rx=0 tx=0).stop 2024-07-17T06:54:32.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.882+0000 7f6032000640 1 -- 172.21.15.175:0/1839775480 shutdown_connections 2024-07-17T06:54:32.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.882+0000 7f6032000640 1 --2- 172.21.15.175:0/1839775480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6010005210 0x7f601000b9c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.882+0000 7f6032000640 1 -- 172.21.15.175:0/1839775480 >> 172.21.15.175:0/1839775480 conn(0x7f602c0d3700 msgr2=0x7f602c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.882+0000 7f6032000640 1 -- 172.21.15.175:0/1839775480 shutdown_connections 2024-07-17T06:54:32.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.882+0000 7f6032000640 1 -- 172.21.15.175:0/1839775480 wait complete. 2024-07-17T06:54:32.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.883+0000 7f6032000640 1 Processor -- start 2024-07-17T06:54:32.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 -- start start 2024-07-17T06:54:32.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6010005210 0x7f602c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.884 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 0x7f602c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:32.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0ce940 con 0x7f6010005210 2024-07-17T06:54:32.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0ce7c0 con 0x7f602c1f4730 2024-07-17T06:54:32.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.884+0000 7f6032000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f602c0ceac0 con 0x7f602c1f4b90 2024-07-17T06:54:32.885 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f6023e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 0x7f602c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:48168/0 (socket says 172.21.15.175:48168) 2024-07-17T06:54:32.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f6023e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 0x7f602c157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44370/0 (socket says 172.21.15.175:44370) 2024-07-17T06:54:32.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 -- 172.21.15.175:0/3765499630 learned_addr learned my addr 172.21.15.175:0/3765499630 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:32.886 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602b400640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6010005210 0x7f602c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 -- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 msgr2=0x7f602c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 0x7f602c157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 -- 172.21.15.175:0/3765499630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6010005210 msgr2=0x7f602c156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6010005210 0x7f602c156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 -- 172.21.15.175:0/3765499630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c1592d0 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.885+0000 7f602be00640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f602400c700 tx=0x7f6024001aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024039070 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f602402b9d0 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602402be00 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c1595b0 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c159b70 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f602403c070 con 0x7f602c1f4b90 2024-07-17T06:54:32.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.886+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f6024034c00 con 0x7f602c1f4b90 2024-07-17T06:54:32.888 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.887+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c1617c0 con 0x7f602c1f4b90 2024-07-17T06:54:32.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.890+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f602400c2d0 con 0x7f602c1f4b90 2024-07-17T06:54:32.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.966+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "893f49bf-b33f-4a70-b75a-0448b881671d", "id": 5} v 0) -- 0x7f602c0b3010 con 0x7f602c1f4b90 2024-07-17T06:54:32.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.973+0000 7f6023400640 1 -- 172.21.15.175:0/3765499630 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "893f49bf-b33f-4a70-b75a-0448b881671d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f6024034680 con 0x7f602c1f4b90 2024-07-17T06:54:32.973 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.974+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 msgr2=0x7f602c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.974+0000 7f6032000640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f602400c700 tx=0x7f6024001aa0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 shutdown_connections 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f602c1f4b90 0x7f602c157780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f602c1f4730 0x7f602c157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 --2- 172.21.15.175:0/3765499630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6010005210 0x7f602c156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 >> 172.21.15.175:0/3765499630 conn(0x7f602c0d3700 msgr2=0x7f602c15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 shutdown_connections 2024-07-17T06:54:32.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:32.975+0000 7f6032000640 1 -- 172.21.15.175:0/3765499630 wait complete. 2024-07-17T06:54:32.997 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b 6 2024-07-17T06:54:33.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 -- 172.21.15.175:0/2153353071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 msgr2=0x7f22cc06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 --2- 172.21.15.175:0/2153353071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc06c320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005fd0 tx=0x7f22c002a530 comp rx=0 tx=0).stop 2024-07-17T06:54:33.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 -- 172.21.15.175:0/2153353071 shutdown_connections 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 --2- 172.21.15.175:0/2153353071 >> v2:172.21.15.175:3300/0 conn(0x7f22cc0b3f20 0x7f22cc161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 --2- 172.21.15.175:0/2153353071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc06c320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 -- 172.21.15.175:0/2153353071 >> 172.21.15.175:0/2153353071 conn(0x7f22cc06d050 msgr2=0x7f22cc06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 -- 172.21.15.175:0/2153353071 shutdown_connections 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.167+0000 7f22d4000640 1 -- 172.21.15.175:0/2153353071 wait complete. 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.169+0000 7f22d4000640 1 Processor -- start 2024-07-17T06:54:33.168 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.169+0000 7f22d4000640 1 -- start start 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.169+0000 7f22d4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc13faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.169+0000 7f22d4000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.169+0000 7f22d4000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22cc0b4360 0x7f22cc0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d4000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc161c70 con 0x7f22cc0b4360 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d4000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc161df0 con 0x7f22cc0b3f20 2024-07-17T06:54:33.170 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22cc06f8f0 con 0x7f22cc06bf40 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc13faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44384/0 (socket says 172.21.15.175:44384) 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 -- 172.21.15.175:0/1601600388 learned_addr learned my addr 172.21.15.175:0/1601600388 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d2000640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22cc0b4360 0x7f22cc0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 -- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 msgr2=0x7f22cc13faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc13faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 -- 172.21.15.175:0/1601600388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22cc0b4360 msgr2=0x7f22cc0af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22cc0b4360 0x7f22cc0af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.171 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.170+0000 7f22d0c00640 1 -- 172.21.15.175:0/1601600388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc0aedd0 con 0x7f22cc0b3f20 2024-07-17T06:54:33.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22d0c00640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e9f0 tx=0x7f22b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:33.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b800cd80 con 0x7f22cc0b3f20 2024-07-17T06:54:33.172 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22b8004020 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc1f89f0 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8004430 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc0af320 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.171+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22b8003980 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.172+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f22b8018d90 con 0x7f22cc0b3f20 2024-07-17T06:54:33.173 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.172+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc074770 con 0x7f22cc0b3f20 2024-07-17T06:54:33.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.175+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f22cc074770 con 0x7f22cc0b3f20 2024-07-17T06:54:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.251+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b", "id": 6} v 0) -- 0x7f22cc0704a0 con 0x7f22cc0b3f20 2024-07-17T06:54:33.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.260+0000 7f22c9600640 1 -- 172.21.15.175:0/1601600388 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f22cc0704a0 con 0x7f22cc0b3f20 2024-07-17T06:54:33.260 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-07-17T06:54:33.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 msgr2=0x7f22cc1f84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e9f0 tx=0x7f22b800eeb0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 shutdown_connections 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22cc0b4360 0x7f22cc0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22cc0b3f20 0x7f22cc1f84b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 --2- 172.21.15.175:0/1601600388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22cc06bf40 0x7f22cc13faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 >> 172.21.15.175:0/1601600388 conn(0x7f22cc06d050 msgr2=0x7f22cc154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.261+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 shutdown_connections 2024-07-17T06:54:33.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.262+0000 7f22d4000640 1 -- 172.21.15.175:0/1601600388 wait complete. 2024-07-17T06:54:33.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new bd85a235-c448-4dcc-9f46-b3ab6fe9be64 7 2024-07-17T06:54:33.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 -- 172.21.15.175:0/1544872739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 msgr2=0x7fa62c161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 --2- 172.21.15.175:0/1544872739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c161770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0b6240 tx=0x7fa61c02a560 comp rx=0 tx=0).stop 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 -- 172.21.15.175:0/1544872739 shutdown_connections 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 --2- 172.21.15.175:0/1544872739 >> v2:172.21.15.44:3300/0 conn(0x7fa62c0d1440 0x7fa62c0d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 --2- 172.21.15.175:0/1544872739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c161770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 -- 172.21.15.175:0/1544872739 >> 172.21.15.175:0/1544872739 conn(0x7fa62c0d04c0 msgr2=0x7fa62c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 -- 172.21.15.175:0/1544872739 shutdown_connections 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.448+0000 7fa630e00640 1 -- 172.21.15.175:0/1544872739 wait complete. 2024-07-17T06:54:33.449 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.450+0000 7fa630e00640 1 Processor -- start 2024-07-17T06:54:33.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.450+0000 7fa630e00640 1 -- start start 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 0x7fa62c1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa62c0d1440 0x7fa62c0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c06ead0 con 0x7fa62c0cdca0 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c06e950 con 0x7fa62c15d350 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa630e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c0ce130 con 0x7fa62c0d1440 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa62be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa623e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa62c0d1440 0x7fa62c0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa62be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44394/0 (socket says 172.21.15.175:44394) 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa62be00640 1 -- 172.21.15.175:0/2673330811 learned_addr learned my addr 172.21.15.175:0/2673330811 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.451+0000 7fa62b400640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 0x7fa62c1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 -- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa62c0d1440 msgr2=0x7fa62c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa62c0d1440 0x7fa62c0caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 -- 172.21.15.175:0/2673330811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 msgr2=0x7fa62c1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 0x7fa62c1f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 -- 172.21.15.175:0/2673330811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c0cb9a0 con 0x7fa62c15d350 2024-07-17T06:54:33.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62b400640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 0x7fa62c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:54:33.454 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa62be00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c6b0 tx=0x7fa62400cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003b70 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa624003d50 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624005330 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c0cbc80 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.452+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c0cc240 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.453+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6240054d0 con 0x7fa62c15d350 2024-07-17T06:54:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.453+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fa6240057c0 con 0x7fa62c15d350 2024-07-17T06:54:33.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.454+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c0d1820 con 0x7fa62c15d350 2024-07-17T06:54:33.457 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.457+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fa62400a640 con 0x7fa62c15d350 2024-07-17T06:54:33.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.533+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd85a235-c448-4dcc-9f46-b3ab6fe9be64", "id": 7} v 0) -- 0x7fa62c0b29f0 con 0x7fa62c15d350 2024-07-17T06:54:33.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.540+0000 7fa623400640 1 -- 172.21.15.175:0/2673330811 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd85a235-c448-4dcc-9f46-b3ab6fe9be64", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fa62400a820 con 0x7fa62c15d350 2024-07-17T06:54:33.540 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-07-17T06:54:33.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 msgr2=0x7fa62c0c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c6b0 tx=0x7fa62400cb70 comp rx=0 tx=0).stop 2024-07-17T06:54:33.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 shutdown_connections 2024-07-17T06:54:33.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62c15d350 0x7fa62c0c92a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa62c0d1440 0x7fa62c0caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 --2- 172.21.15.175:0/2673330811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa62c0cdca0 0x7fa62c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.541+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 >> 172.21.15.175:0/2673330811 conn(0x7fa62c0d04c0 msgr2=0x7fa62c0d3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.542+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 shutdown_connections 2024-07-17T06:54:33.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:54:33.542+0000 7fa630e00640 1 -- 172.21.15.175:0/2673330811 wait complete. 2024-07-17T06:54:33.560 INFO:tasks.ceph.osd.0:Restarting daemon 2024-07-17T06:54:33.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-07-17T06:54:33.568 INFO:tasks.ceph.osd.0:Started 2024-07-17T06:54:33.568 INFO:tasks.ceph.osd.1:Restarting daemon 2024-07-17T06:54:33.568 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-07-17T06:54:33.575 INFO:tasks.ceph.osd.1:Started 2024-07-17T06:54:33.575 INFO:tasks.ceph.osd.2:Restarting daemon 2024-07-17T06:54:33.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-07-17T06:54:33.580 INFO:tasks.ceph.osd.2:Started 2024-07-17T06:54:33.580 INFO:tasks.ceph.osd.3:Restarting daemon 2024-07-17T06:54:33.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-07-17T06:54:33.586 INFO:tasks.ceph.osd.3:Started 2024-07-17T06:54:33.586 INFO:tasks.ceph.osd.4:Restarting daemon 2024-07-17T06:54:33.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-07-17T06:54:33.591 INFO:tasks.ceph.osd.4:Started 2024-07-17T06:54:33.591 INFO:tasks.ceph.osd.5:Restarting daemon 2024-07-17T06:54:33.591 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-07-17T06:54:33.598 INFO:tasks.ceph.osd.5:Started 2024-07-17T06:54:33.599 INFO:tasks.ceph.osd.6:Restarting daemon 2024-07-17T06:54:33.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-07-17T06:54:33.604 INFO:tasks.ceph.osd.6:Started 2024-07-17T06:54:33.604 INFO:tasks.ceph.osd.7:Restarting daemon 2024-07-17T06:54:33.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-07-17T06:54:33.609 INFO:tasks.ceph.osd.7:Started 2024-07-17T06:54:33.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 -- 172.21.15.44:0/560859540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e00b5910 msgr2=0x7f61e00cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 --2- 172.21.15.44:0/560859540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e00b5910 0x7f61e00cdcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61d8005fd0 tx=0x7f61d802a940 comp rx=0 tx=0).stop 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 -- 172.21.15.44:0/560859540 shutdown_connections 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 --2- 172.21.15.44:0/560859540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e00b5910 0x7f61e00cdcf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 --2- 172.21.15.44:0/560859540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e015d600 0x7f61e006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.835+0000 7f61e4a00640 1 -- 172.21.15.44:0/560859540 >> 172.21.15.44:0/560859540 conn(0x7f61e00d00a0 msgr2=0x7f61e00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.836+0000 7f61e4a00640 1 -- 172.21.15.44:0/560859540 shutdown_connections 2024-07-17T06:54:33.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.836+0000 7f61e4a00640 1 -- 172.21.15.44:0/560859540 wait complete. 2024-07-17T06:54:33.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.837+0000 7f61e4a00640 1 Processor -- start 2024-07-17T06:54:33.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.837+0000 7f61e4a00640 1 -- start start 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 0x7f61e00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e00b5910 0x7f61e00b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00ce6b0 con 0x7f61e015d600 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00ce530 con 0x7f61e00b2ad0 2024-07-17T06:54:33.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61e4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61e00ce830 con 0x7f61e00b5910 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.838+0000 7f61de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53660/0 (socket says 172.21.15.44:53660) 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61d7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e00b5910 0x7f61e00b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 -- 172.21.15.44:0/522522930 learned_addr learned my addr 172.21.15.44:0/522522930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61dde00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 0x7f61e00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 -- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e00b5910 msgr2=0x7f61e00b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e00b5910 0x7f61e00b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 -- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 msgr2=0x7f61e00cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 0x7f61e00cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 -- 172.21.15.44:0/522522930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e00cc470 con 0x7f61e015d600 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61dde00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 0x7f61e00cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.839+0000 7f61de800640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009cd0 tx=0x7f61d8031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.840+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8030430 con 0x7f61e015d600 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.840+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61d8004240 con 0x7f61e015d600 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.840+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e00b73d0 con 0x7f61e015d600 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.840+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8003e60 con 0x7f61e015d600 2024-07-17T06:54:33.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.840+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e00b68f0 con 0x7f61e015d600 2024-07-17T06:54:33.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.841+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e00b70b0 con 0x7f61e015d600 2024-07-17T06:54:33.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.841+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 2) ==== 49436+0+0 (secure 0 0 0) 0x7f61d8004670 con 0x7f61e015d600 2024-07-17T06:54:33.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.842+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f61d8072880 con 0x7f61e015d600 2024-07-17T06:54:33.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.844+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f61e00b70b0 con 0x7f61e015d600 2024-07-17T06:54:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.909+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61e00cece0 con 0x7f61e015d600 2024-07-17T06:54:33.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.910+0000 7f61d6000640 1 -- 172.21.15.44:0/522522930 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f61d804bd70 con 0x7f61e015d600 2024-07-17T06:54:33.910 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:33.910 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:33.536877+0000","last_up_change":"0.000000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:33.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 msgr2=0x7f61e015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:33.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009cd0 tx=0x7f61d8031ea0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 shutdown_connections 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61e015d600 0x7f61e015fcb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e00b5910 0x7f61e00b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 --2- 172.21.15.44:0/522522930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e00b2ad0 0x7f61e00cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 >> 172.21.15.44:0/522522930 conn(0x7f61e00d00a0 msgr2=0x7f61e00c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:33.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 shutdown_connections 2024-07-17T06:54:33.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:33.912+0000 7f61e4a00640 1 -- 172.21.15.44:0/522522930 wait complete. 2024-07-17T06:54:33.927 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-07-17T06:54:33.928 INFO:tasks.ceph:Waiting for OSDs to come up 2024-07-17T06:54:34.292 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-07-17T06:54:34.292+0000 7fe8e27c4640 -1 Falling back to public interface 2024-07-17T06:54:34.293 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-07-17T06:54:34.292+0000 7effa71c0640 -1 Falling back to public interface 2024-07-17T06:54:34.294 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-07-17T06:54:34.295+0000 7fadeb6d5640 -1 Falling back to public interface 2024-07-17T06:54:34.294 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-07-17T06:54:34.295+0000 7f0cfe933640 -1 Falling back to public interface 2024-07-17T06:54:34.294 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-07-17T06:54:34.295+0000 7f0cfd400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-07-17T06:54:34.295 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-07-17T06:54:34.296+0000 7fadea200640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6805/0: (98) Address already in use 2024-07-17T06:54:34.296 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:34.297+0000 7f26ba42b640 -1 Falling back to public interface 2024-07-17T06:54:34.297 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:34.298+0000 7f26b8e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2024-07-17T06:54:34.297 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:34.299+0000 7f26b8e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6811/0: (98) Address already in use 2024-07-17T06:54:34.301 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-07-17T06:54:34.301+0000 7f2a1b71f640 -1 Falling back to public interface 2024-07-17T06:54:34.304 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-07-17T06:54:34.306+0000 7fcc38f50640 -1 Falling back to public interface 2024-07-17T06:54:34.316 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-07-17T06:54:34.317+0000 7f5380544640 -1 Falling back to public interface 2024-07-17T06:54:34.735 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:34.738 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:54:34.739+0000 7fca34e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-07-17T06:54:34.739 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:54:34.739+0000 7fca34e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-07-17T06:54:34.925 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fb1200640 1 -- 172.21.15.44:0/696439181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 msgr2=0x7f5f9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fb1200640 1 --2- 172.21.15.44:0/696439181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5f9000b9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4001d90 tx=0x7f5f90005830 comp rx=0 tx=0).stop 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fabe00640 1 -- 172.21.15.44:0/696439181 reap_dead start 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fb1200640 1 -- 172.21.15.44:0/696439181 shutdown_connections 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fb1200640 1 -- 172.21.15.44:0/696439181 >> 172.21.15.44:0/696439181 conn(0x7f5fac06d990 msgr2=0x7f5fac06dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.926+0000 7f5fb1200640 1 -- 172.21.15.44:0/696439181 shutdown_connections 2024-07-17T06:54:34.926 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.927+0000 7f5fb1200640 1 -- 172.21.15.44:0/696439181 wait complete. 2024-07-17T06:54:34.927 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.928+0000 7f5fb1200640 1 Processor -- start 2024-07-17T06:54:34.927 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.928+0000 7f5fb1200640 1 -- start start 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0acf40 0x7f5fac0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fac0c8c50 0x7f5fac0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0b6e50 con 0x7f5f90004e60 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0d0f90 con 0x7f5fac0acf40 2024-07-17T06:54:34.929 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.929+0000 7f5fb1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac0b6fd0 con 0x7f5fac0c8c50 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53690/0 (socket says 172.21.15.44:53690) 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 -- 172.21.15.44:0/2084612753 learned_addr learned my addr 172.21.15.44:0/2084612753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fabe00640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fac0c8c50 0x7f5fac0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fa3e00640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0acf40 0x7f5fac0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 -- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fac0c8c50 msgr2=0x7f5fac0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fac0c8c50 0x7f5fac0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 -- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0acf40 msgr2=0x7f5fac0ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0acf40 0x7f5fac0ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 -- 172.21.15.44:0/2084612753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac150d40 con 0x7f5f90004e60 2024-07-17T06:54:34.930 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.930+0000 7f5fab400640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00fa00 tx=0x7f5f9c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.931+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00bcb0 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.931+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f9c011440 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.931+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c011870 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.931+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac152c30 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.931+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1531f0 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.932+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac1f3e10 con 0x7f5f90004e60 2024-07-17T06:54:34.932 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.933+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 3) ==== 98667+0+0 (secure 0 0 0) 0x7f5f9c019020 con 0x7f5f90004e60 2024-07-17T06:54:34.933 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.934+0000 7f5fa3400640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 0x7f5f88078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:34.933 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.934+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5f9c097a10 con 0x7f5f90004e60 2024-07-17T06:54:34.933 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.935+0000 7f5fa3e00640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 0x7f5f88078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:34.934 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.936+0000 7f5fa3e00640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 0x7f5f88078ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f94000fd0 tx=0x7f5f94001d10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:34.937 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:34.939+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f9c062130 con 0x7f5f90004e60 2024-07-17T06:54:35.159 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.160+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac0d2cb0 con 0x7f5f90004e60 2024-07-17T06:54:35.159 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.161+0000 7f5fa3400640 1 -- 172.21.15.44:0/2084612753 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5f9c061a30 con 0x7f5f90004e60 2024-07-17T06:54:35.160 INFO:teuthology.misc.health.smithi044.stdout: 2024-07-17T06:54:35.160 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:33.536877+0000","last_up_change":"0.000000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:35.162 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 msgr2=0x7f5f88078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:35.162 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 0x7f5f88078ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f94000fd0 tx=0x7f5f94001d10 comp rx=0 tx=0).stop 2024-07-17T06:54:35.162 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 msgr2=0x7f5fac150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00fa00 tx=0x7f5f9c00fec0 comp rx=0 tx=0).stop 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 shutdown_connections 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f88076a00 0x7f5f88078ec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fac0c8c50 0x7f5fac0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fac0acf40 0x7f5fac0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 --2- 172.21.15.44:0/2084612753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f90004e60 0x7f5fac150800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 >> 172.21.15.44:0/2084612753 conn(0x7f5fac06d990 msgr2=0x7f5fac0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 shutdown_connections 2024-07-17T06:54:35.163 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:35.163+0000 7f5fb1200640 1 -- 172.21.15.44:0/2084612753 wait complete. 2024-07-17T06:54:35.176 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-07-17T06:54:37.651 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-07-17T06:54:37.651+0000 7fe8e27c4640 -1 osd.4 0 log_to_monitors true 2024-07-17T06:54:37.699 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-07-17T06:54:37.700+0000 7fcc38f50640 -1 osd.2 0 log_to_monitors true 2024-07-17T06:54:37.710 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-07-17T06:54:37.710+0000 7f5380544640 -1 osd.7 0 log_to_monitors true 2024-07-17T06:54:37.893 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-07-17T06:54:37.893+0000 7f2a1b71f640 -1 osd.5 0 log_to_monitors true 2024-07-17T06:54:37.918 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-07-17T06:54:37.918+0000 7effa71c0640 -1 osd.6 0 log_to_monitors true 2024-07-17T06:54:37.924 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-07-17T06:54:37.925+0000 7fadeb6d5640 -1 osd.3 0 log_to_monitors true 2024-07-17T06:54:37.945 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-07-17T06:54:37.946+0000 7f0cfe933640 -1 osd.1 0 log_to_monitors true 2024-07-17T06:54:37.946 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:37.947+0000 7f26ba42b640 -1 osd.0 0 log_to_monitors true 2024-07-17T06:54:39.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:54:39.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:54:39.950 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-07-17T06:54:39.951+0000 7fcc33e00640 -1 osd.2 0 waiting for initial osdmap 2024-07-17T06:54:39.966 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-07-17T06:54:39.967+0000 7fcc29e00640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:40.423 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-07-17T06:54:40.423+0000 7f537be00640 -1 osd.7 0 waiting for initial osdmap 2024-07-17T06:54:40.432 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-07-17T06:54:40.432+0000 7f5371400640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:40.508 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-07-17T06:54:40.508+0000 7fe8dd600640 -1 osd.4 0 waiting for initial osdmap 2024-07-17T06:54:40.517 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-07-17T06:54:40.517+0000 7fe8d3600640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.195 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-07-17T06:54:41.195+0000 7f2a16600640 -1 osd.5 0 waiting for initial osdmap 2024-07-17T06:54:41.197 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-07-17T06:54:41.197+0000 7effa2000640 -1 osd.6 0 waiting for initial osdmap 2024-07-17T06:54:41.201 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-07-17T06:54:41.202+0000 7f2a0c600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.209 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-07-17T06:54:41.209+0000 7eff98000640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.567 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-07-17T06:54:41.569+0000 7fade6600640 -1 osd.3 0 waiting for initial osdmap 2024-07-17T06:54:41.577 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-07-17T06:54:41.578+0000 7faddc600640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.677 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:41.679+0000 7f26b5200640 -1 osd.0 0 waiting for initial osdmap 2024-07-17T06:54:41.688 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-07-17T06:54:41.689+0000 7f26ab200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.691 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-07-17T06:54:41.692+0000 7f0cf9800640 -1 osd.1 0 waiting for initial osdmap 2024-07-17T06:54:41.701 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-07-17T06:54:41.702+0000 7f0cef800640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-17T06:54:41.982 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 -- 172.21.15.44:0/1315819139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e4004d90 msgr2=0x7ff5e4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 --2- 172.21.15.44:0/1315819139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e4004d90 0x7ff5e4005210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005fd0 tx=0x7ff5e4005830 comp rx=0 tx=0).stop 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff5ffe00640 1 -- 172.21.15.44:0/1315819139 reap_dead start 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 -- 172.21.15.44:0/1315819139 shutdown_connections 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 -- 172.21.15.44:0/1315819139 >> 172.21.15.44:0/1315819139 conn(0x7ff6000b0f30 msgr2=0x7ff6000b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 -- 172.21.15.44:0/1315819139 shutdown_connections 2024-07-17T06:54:42.175 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.176+0000 7ff605000640 1 -- 172.21.15.44:0/1315819139 wait complete. 2024-07-17T06:54:42.176 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.177+0000 7ff605000640 1 Processor -- start 2024-07-17T06:54:42.176 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.178+0000 7ff605000640 1 -- start start 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 0x7ff6001f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6000ca5e0 0x7ff6000caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff60006f5f0 con 0x7ff60014b070 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff60006f470 con 0x7ff5e4004d90 2024-07-17T06:54:42.179 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff605000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff60015cef0 con 0x7ff6000ca5e0 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36192/0 (socket says 172.21.15.44:36192) 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 -- 172.21.15.44:0/2497153405 learned_addr learned my addr 172.21.15.44:0/2497153405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5ff400640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 0x7ff6001f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5ffe00640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6000ca5e0 0x7ff6000caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 -- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6000ca5e0 msgr2=0x7ff6000caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6000ca5e0 0x7ff6000caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 -- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 msgr2=0x7ff6001f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 0x7ff6001f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5f7e00640 1 -- 172.21.15.44:0/2497153405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60014bff0 con 0x7ff60014b070 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.179+0000 7ff5ff400640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 0x7ff6001f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff5f7e00640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d620 tx=0x7ff5e800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:42.180 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80062e0 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5e8015020 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8003870 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60014b750 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.180+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60014bd10 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.181+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60006e8f0 con 0x7ff60014b070 2024-07-17T06:54:42.181 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.183+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 98773+0+0 (secure 0 0 0) 0x7ff5e8003a10 con 0x7ff60014b070 2024-07-17T06:54:42.182 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.184+0000 7ff5f7400640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 0x7ff5dc078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.182 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.184+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3071+0+0 (secure 0 0 0) 0x7ff5e8097d00 con 0x7ff60014b070 2024-07-17T06:54:42.183 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.184+0000 7ff5ff400640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 0x7ff5dc078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.184 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.185+0000 7ff5ff400640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 0x7ff5dc078fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0006a30 tx=0x7ff5f00015e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:42.189 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.189+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff5e801f020 con 0x7ff60014b070 2024-07-17T06:54:42.408 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.409+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6000ac240 con 0x7ff60014b070 2024-07-17T06:54:42.410 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.411+0000 7ff5f7400640 1 -- 172.21.15.44:0/2497153405 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12761 (secure 0 0 0) 0x7ff5e8061cf0 con 0x7ff60014b070 2024-07-17T06:54:42.410 INFO:teuthology.misc.health.smithi044.stdout: 2024-07-17T06:54:42.410 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":15,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:42.202505+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.413+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 msgr2=0x7ff5dc078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.413+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 0x7ff5dc078fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0006a30 tx=0x7ff5f00015e0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.413+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 msgr2=0x7ff60014ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.413+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d620 tx=0x7ff5e800dae0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 shutdown_connections 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff5dc076b20 0x7ff5dc078fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6000ca5e0 0x7ff6000caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60014b070 0x7ff60014ee00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 --2- 172.21.15.44:0/2497153405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5e4004d90 0x7ff6001f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 >> 172.21.15.44:0/2497153405 conn(0x7ff6000b0f30 msgr2=0x7ff6000cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 shutdown_connections 2024-07-17T06:54:42.413 INFO:teuthology.misc.health.smithi044.stderr:2024-07-17T06:54:42.414+0000 7ff605000640 1 -- 172.21.15.44:0/2497153405 wait complete. 2024-07-17T06:54:42.426 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-07-17T06:54:42.426 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-07-17T06:54:42.427 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T06:54:42.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T06:54:42.435 INFO:tasks.ceph.mds.a:Started 2024-07-17T06:54:42.435 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T06:54:42.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T06:54:42.442 INFO:tasks.ceph.mds.c:Started 2024-07-17T06:54:42.442 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T06:54:42.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T06:54:42.450 INFO:tasks.ceph.mds.b:Started 2024-07-17T06:54:42.450 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T06:54:42.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T06:54:42.458 INFO:tasks.ceph.mds.d:Started 2024-07-17T06:54:42.458 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-07-17T06:54:42.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:54:42.581 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T06:54:42.591 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T06:54:42.602 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T06:54:42.604 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T06:54:42.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 -- 172.21.15.44:0/2753580653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c06d6d0 msgr2=0x7f078c06db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 --2- 172.21.15.44:0/2753580653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c06d6d0 0x7f078c06db80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f078c072210 tx=0x7f078402aac0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 -- 172.21.15.44:0/2753580653 shutdown_connections 2024-07-17T06:54:42.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 --2- 172.21.15.44:0/2753580653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c06d6d0 0x7f078c06db80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 --2- 172.21.15.44:0/2753580653 >> v2:172.21.15.175:3300/0 conn(0x7f078c0c6290 0x7f078c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 -- 172.21.15.44:0/2753580653 >> 172.21.15.44:0/2753580653 conn(0x7f078c0b3020 msgr2=0x7f078c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:42.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 -- 172.21.15.44:0/2753580653 shutdown_connections 2024-07-17T06:54:42.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.632+0000 7f0792600640 1 -- 172.21.15.44:0/2753580653 wait complete. 2024-07-17T06:54:42.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.634+0000 7f0792600640 1 Processor -- start 2024-07-17T06:54:42.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.634+0000 7f0792600640 1 -- start start 2024-07-17T06:54:42.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f078c06d2f0 0x7f078c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f078c06d6d0 0x7f078c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0c7ba0 con 0x7f078c0c6290 2024-07-17T06:54:42.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0c7a20 con 0x7f078c06d2f0 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0792600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f078c0c7d20 con 0x7f078c06d6d0 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36242/0 (socket says 172.21.15.44:36242) 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078be00640 1 -- 172.21.15.44:0/356471881 learned_addr learned my addr 172.21.15.44:0/356471881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f0783e00640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f078c06d6d0 0x7f078c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078b400640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f078c06d2f0 0x7f078c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078be00640 1 -- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f078c06d6d0 msgr2=0x7f078c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.635+0000 7f078be00640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f078c06d6d0 0x7f078c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f078be00640 1 -- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f078c06d2f0 msgr2=0x7f078c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f078be00640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f078c06d2f0 0x7f078c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f078be00640 1 -- 172.21.15.44:0/356471881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c15e150 con 0x7f078c0c6290 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f078be00640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0784009d60 tx=0x7f0784003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784044070 con 0x7f078c0c6290 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0784004250 con 0x7f078c0c6290 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078c15e430 con 0x7f078c0c6290 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784004680 con 0x7f078c0c6290 2024-07-17T06:54:42.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.636+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078c15e9f0 con 0x7f078c0c6290 2024-07-17T06:54:42.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.637+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078c0c81d0 con 0x7f078c0c6290 2024-07-17T06:54:42.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.638+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 98773+0+0 (secure 0 0 0) 0x7f078402c840 con 0x7f078c0c6290 2024-07-17T06:54:42.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.639+0000 7f0783400640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 0x7f0768078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:42.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.639+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f07840b8470 con 0x7f078c0c6290 2024-07-17T06:54:42.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.640+0000 7f078b400640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 0x7f0768078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.640+0000 7f078b400640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 0x7f0768078fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f077c0015e0 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:42.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.644+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0784032030 con 0x7f078c0c6290 2024-07-17T06:54:42.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.889+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f078c15ebc0 con 0x7f078c0c6290 2024-07-17T06:54:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.890+0000 7f0783400640 1 -- 172.21.15.44:0/356471881 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f0784004820 con 0x7f078c0c6290 2024-07-17T06:54:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:42.889 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"btime":"2024-07-17T06:54:42:592017+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-17T06:54:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 msgr2=0x7f0768078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 0x7f0768078fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f077c0015e0 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 msgr2=0x7f078c154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0784009d60 tx=0x7f0784003f10 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 shutdown_connections 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f078c0c6290 0x7f078c154200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0768076b20 0x7f0768078fe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f078c06d6d0 0x7f078c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 --2- 172.21.15.44:0/356471881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f078c06d2f0 0x7f078c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 >> 172.21.15.44:0/356471881 conn(0x7f078c0b3020 msgr2=0x7f078c0b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:42.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 shutdown_connections 2024-07-17T06:54:42.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:42.893+0000 7f0792600640 1 -- 172.21.15.44:0/356471881 wait complete. 2024-07-17T06:54:44.909 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.129+0000 7f51a1a00640 1 -- 172.21.15.44:0/1488484554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3f10 msgr2=0x7f519c0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.129+0000 7f51a1a00640 1 --2- 172.21.15.44:0/1488484554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3f10 0x7f519c0b4a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f519c06e930 tx=0x7f518802aa30 comp rx=0 tx=0).stop 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 -- 172.21.15.44:0/1488484554 shutdown_connections 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 --2- 172.21.15.44:0/1488484554 >> v2:172.21.15.175:3300/0 conn(0x7f519c0cded0 0x7f519c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 --2- 172.21.15.44:0/1488484554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3f10 0x7f519c0b4a30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 -- 172.21.15.44:0/1488484554 >> 172.21.15.44:0/1488484554 conn(0x7f519c08ca20 msgr2=0x7f519c08ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 -- 172.21.15.44:0/1488484554 shutdown_connections 2024-07-17T06:54:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.130+0000 7f51a1a00640 1 -- 172.21.15.44:0/1488484554 wait complete. 2024-07-17T06:54:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.132+0000 7f51a1a00640 1 Processor -- start 2024-07-17T06:54:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.132+0000 7f51a1a00640 1 -- start start 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f519c0cded0 0x7f519c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f519c0d3f10 0x7f519c15f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0ce880 con 0x7f519c0d3420 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f519c0cea00 con 0x7f519c0d3f10 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f519c0ceb80 con 0x7f519c0cded0 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f519b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f519b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36260/0 (socket says 172.21.15.44:36260) 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f519b400640 1 -- 172.21.15.44:0/3738091221 learned_addr learned my addr 172.21.15.44:0/3738091221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f519be00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f519c0cded0 0x7f519c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.133+0000 7f51a1000640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f519c0d3f10 0x7f519c15f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 -- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f519c0cded0 msgr2=0x7f519c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f519c0cded0 0x7f519c0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 -- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f519c0d3f10 msgr2=0x7f519c15f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f519c0d3f10 0x7f519c15f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 -- 172.21.15.44:0/3738091221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c1f4fd0 con 0x7f519c0d3420 2024-07-17T06:54:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f519b400640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5188009bf0 tx=0x7f5188003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188033050 con 0x7f519c0d3420 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51880044d0 con 0x7f519c0d3420 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c1f4a10 con 0x7f519c0d3420 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.134+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188038450 con 0x7f519c0d3420 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.135+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c15ebd0 con 0x7f519c0d3420 2024-07-17T06:54:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.136+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c0ce340 con 0x7f519c0d3420 2024-07-17T06:54:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.137+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 98773+0+0 (secure 0 0 0) 0x7f5188036070 con 0x7f519c0d3420 2024-07-17T06:54:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.138+0000 7f5193e00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 0x7f517c078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.138+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f51880b91e0 con 0x7f519c0d3420 2024-07-17T06:54:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.138+0000 7f519be00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 0x7f517c078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.139+0000 7f519be00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 0x7f517c078fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f518c0017c0 tx=0x7f518c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.143+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5188082ad0 con 0x7f519c0d3420 2024-07-17T06:54:45.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.403+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f519c0cf3f0 con 0x7f519c0d3420 2024-07-17T06:54:45.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.404+0000 7f5193e00640 1 -- 172.21.15.44:0/3738091221 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7f51880823d0 con 0x7f519c0d3420 2024-07-17T06:54:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:45.404 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"btime":"2024-07-17T06:54:43:593644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-17T06:54:45.404 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.407+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 msgr2=0x7f517c078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.407+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 0x7f517c078fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f518c0017c0 tx=0x7f518c009040 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.407+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 msgr2=0x7f519c1f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.407+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5188009bf0 tx=0x7f5188003f90 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 shutdown_connections 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f519c0d3f10 0x7f519c15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f519c0d3420 0x7f519c1f4070 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f517c076b20 0x7f517c078fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 --2- 172.21.15.44:0/3738091221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f519c0cded0 0x7f519c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 >> 172.21.15.44:0/3738091221 conn(0x7f519c08ca20 msgr2=0x7f519c0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:45.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 shutdown_connections 2024-07-17T06:54:45.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.408+0000 7f51a1a00640 1 -- 172.21.15.44:0/3738091221 wait complete. 2024-07-17T06:54:45.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T06:54:45.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 -- 172.21.15.44:0/68944408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 msgr2=0x7f5c5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 --2- 172.21.15.44:0/68944408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c5c003de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00bb20 tx=0x7f5c5c005830 comp rx=0 tx=0).stop 2024-07-17T06:54:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c7be00640 1 -- 172.21.15.44:0/68944408 reap_dead start 2024-07-17T06:54:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 -- 172.21.15.44:0/68944408 shutdown_connections 2024-07-17T06:54:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 -- 172.21.15.44:0/68944408 >> 172.21.15.44:0/68944408 conn(0x7f5c7c0b0fb0 msgr2=0x7f5c7c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 -- 172.21.15.44:0/68944408 shutdown_connections 2024-07-17T06:54:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.644+0000 7f5c82400640 1 -- 172.21.15.44:0/68944408 wait complete. 2024-07-17T06:54:45.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.646+0000 7f5c82400640 1 Processor -- start 2024-07-17T06:54:45.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.646+0000 7f5c82400640 1 -- start start 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c7c0afd40 0x7f5c7c09b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c7c0b01c0 0x7f5c7c09b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c06f010 con 0x7f5c5c0039e0 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c06ee90 con 0x7f5c7c0b01c0 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c82400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c15e510 con 0x7f5c7c0afd40 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.647+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36282/0 (socket says 172.21.15.44:36282) 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 -- 172.21.15.44:0/2237487522 learned_addr learned my addr 172.21.15.44:0/2237487522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 -- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c7c0afd40 msgr2=0x7f5c7c09b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c7c0afd40 0x7f5c7c09b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 -- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c7c0b01c0 msgr2=0x7f5c7c09b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c7c0b01c0 0x7f5c7c09b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 -- 172.21.15.44:0/2237487522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c1f80a0 con 0x7f5c5c0039e0 2024-07-17T06:54:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.648+0000 7f5c7b400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00bc40 tx=0x7f5c6c00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.649+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c040070 con 0x7f5c5c0039e0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.649+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c6c03a830 con 0x7f5c5c0039e0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.649+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c035040 con 0x7f5c5c0039e0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.649+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1f8340 con 0x7f5c5c0039e0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.649+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1f8900 con 0x7f5c5c0039e0 2024-07-17T06:54:45.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.650+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c0ac220 con 0x7f5c5c0039e0 2024-07-17T06:54:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.652+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 98773+0+0 (secure 0 0 0) 0x7f5c6c038020 con 0x7f5c5c0039e0 2024-07-17T06:54:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.653+0000 7f5c73400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 0x7f5c58079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:45.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.653+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5c6c0b8ce0 con 0x7f5c5c0039e0 2024-07-17T06:54:45.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.653+0000 7f5c7aa00640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 0x7f5c58079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:45.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.654+0000 7f5c7aa00640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 0x7f5c58079030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b4210 tx=0x7f5c64006d40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:45.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.656+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5c6c0825c0 con 0x7f5c5c0039e0 2024-07-17T06:54:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.869+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5c7c154300 con 0x7f5c5c0039e0 2024-07-17T06:54:45.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.870+0000 7f5c73400640 1 -- 172.21.15.44:0/2237487522 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f5c6c081ec0 con 0x7f5c5c0039e0 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:54:45.869 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.872+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 msgr2=0x7f5c58079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.872+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 0x7f5c58079030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b4210 tx=0x7f5c64006d40 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.872+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 msgr2=0x7f5c7c1522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.872+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00bc40 tx=0x7f5c6c00b6f0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 shutdown_connections 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c7c0b01c0 0x7f5c7c09b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5c58076b70 0x7f5c58079030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c7c0afd40 0x7f5c7c09b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 --2- 172.21.15.44:0/2237487522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c5c0039e0 0x7f5c7c1522c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:45.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 >> 172.21.15.44:0/2237487522 conn(0x7f5c7c0b0fb0 msgr2=0x7f5c7c156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 shutdown_connections 2024-07-17T06:54:45.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:45.873+0000 7f5c82400640 1 -- 172.21.15.44:0/2237487522 wait complete. 2024-07-17T06:54:45.891 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T06:54:45.891 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T06:54:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.112+0000 7fe45b600640 1 -- 172.21.15.44:0/3336563629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 msgr2=0x7fe4540ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.112+0000 7fe45b600640 1 --2- 172.21.15.44:0/3336563629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4540ca7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4540b1940 tx=0x7fe43c02a510 comp rx=0 tx=0).stop 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.113+0000 7fe45b600640 1 -- 172.21.15.44:0/3336563629 shutdown_connections 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.113+0000 7fe45b600640 1 --2- 172.21.15.44:0/3336563629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4540ca7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.113+0000 7fe45b600640 1 -- 172.21.15.44:0/3336563629 >> 172.21.15.44:0/3336563629 conn(0x7fe4540b3120 msgr2=0x7fe4540b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.113+0000 7fe45b600640 1 -- 172.21.15.44:0/3336563629 shutdown_connections 2024-07-17T06:54:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.113+0000 7fe45b600640 1 -- 172.21.15.44:0/3336563629 wait complete. 2024-07-17T06:54:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.114+0000 7fe45b600640 1 Processor -- start 2024-07-17T06:54:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.114+0000 7fe45b600640 1 -- start start 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4540b3c40 0x7fe4541f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe454160a80 0x7fe454152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe45406eb80 con 0x7fe4540b1c50 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4540cb680 con 0x7fe4540b3c40 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.115+0000 7fe45b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe45406ed00 con 0x7fe454160a80 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36314/0 (socket says 172.21.15.44:36314) 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 -- 172.21.15.44:0/3967498482 learned_addr learned my addr 172.21.15.44:0/3967498482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 -- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe454160a80 msgr2=0x7fe454152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe454160a80 0x7fe454152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 -- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4540b3c40 msgr2=0x7fe4541f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe453e00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4540b3c40 0x7fe4541f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:46.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4540b3c40 0x7fe4541f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 -- 172.21.15.44:0/3967498482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541f8e40 con 0x7fe4540b1c50 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.116+0000 7fe458c00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c7a0 tx=0x7fe44400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.117+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444017070 con 0x7fe4540b1c50 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.117+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe444005070 con 0x7fe4540b1c50 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.117+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4440054a0 con 0x7fe4540b1c50 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.117+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45415d960 con 0x7fe4540b1c50 2024-07-17T06:54:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.117+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe454156cf0 con 0x7fe4540b1c50 2024-07-17T06:54:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.118+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45406a1a0 con 0x7fe4540b1c50 2024-07-17T06:54:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.119+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 98773+0+0 (secure 0 0 0) 0x7fe44401a070 con 0x7fe4540b1c50 2024-07-17T06:54:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.120+0000 7fe450c00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 0x7fe430078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.120+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe4440975b0 con 0x7fe4540b1c50 2024-07-17T06:54:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.121+0000 7fe453e00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 0x7fe430078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.122+0000 7fe453e00640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 0x7fe430078fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe43c034000 tx=0x7fe43c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.125+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe45406a1a0 con 0x7fe4540b1c50 2024-07-17T06:54:46.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.228+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe44405ca30 con 0x7fe4540b1c50 2024-07-17T06:54:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:46.357+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4540b0860 con 0x7fe4540b1c50 2024-07-17T06:54:47.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.230+0000 7fe450c00640 1 -- 172.21.15.44:0/3967498482 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fe4540b0860 con 0x7fe4540b1c50 2024-07-17T06:54:47.230 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T06:54:47.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.235+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 msgr2=0x7fe430078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:47.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.235+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 0x7fe430078fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe43c034000 tx=0x7fe43c004730 comp rx=0 tx=0).stop 2024-07-17T06:54:47.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.235+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 msgr2=0x7fe4541567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:47.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.235+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe44400c7a0 tx=0x7fe44400cc60 comp rx=0 tx=0).stop 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 shutdown_connections 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe430076b20 0x7fe430078fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe454160a80 0x7fe454152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4540b3c40 0x7fe4541f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 --2- 172.21.15.44:0/3967498482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4540b1c50 0x7fe4541567b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.236+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 >> 172.21.15.44:0/3967498482 conn(0x7fe4540b3120 msgr2=0x7fe4540b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:47.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.237+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 shutdown_connections 2024-07-17T06:54:47.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.237+0000 7fe45b600640 1 -- 172.21.15.44:0/3967498482 wait complete. 2024-07-17T06:54:47.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 -- 172.21.15.44:0/2750568509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 msgr2=0x7efd9006d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 --2- 172.21.15.44:0/2750568509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd9006d850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd90070830 tx=0x7efd8802b910 comp rx=0 tx=0).stop 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 -- 172.21.15.44:0/2750568509 shutdown_connections 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 --2- 172.21.15.44:0/2750568509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd9006d850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 --2- 172.21.15.44:0/2750568509 >> v2:172.21.15.175:3300/0 conn(0x7efd9006f4d0 0x7efd9015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 -- 172.21.15.44:0/2750568509 >> 172.21.15.44:0/2750568509 conn(0x7efd900cf0d0 msgr2=0x7efd900cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:47.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 -- 172.21.15.44:0/2750568509 shutdown_connections 2024-07-17T06:54:47.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.463+0000 7efd95600640 1 -- 172.21.15.44:0/2750568509 wait complete. 2024-07-17T06:54:47.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.465+0000 7efd95600640 1 Processor -- start 2024-07-17T06:54:47.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.465+0000 7efd95600640 1 -- start start 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 0x7efd90158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd901503c0 0x7efd901f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd900b3320 con 0x7efd9006d420 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd900b31a0 con 0x7efd901503c0 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd95600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd900b34a0 con 0x7efd9006f4d0 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36332/0 (socket says 172.21.15.44:36332) 2024-07-17T06:54:47.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd8fe00640 1 -- 172.21.15.44:0/1946487621 learned_addr learned my addr 172.21.15.44:0/1946487621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:47.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.466+0000 7efd8f400640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 0x7efd90158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd94c00640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd901503c0 0x7efd901f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 -- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 msgr2=0x7efd90158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 0x7efd90158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 -- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd901503c0 msgr2=0x7efd901f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd901503c0 0x7efd901f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 -- 172.21.15.44:0/1946487621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd90151900 con 0x7efd9006d420 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8f400640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 0x7efd90158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd8fe00640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd8000d980 tx=0x7efd8000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80017070 con 0x7efd9006d420 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd80009d20 con 0x7efd9006d420 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800106c0 con 0x7efd9006d420 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901f4510 con 0x7efd9006d420 2024-07-17T06:54:47.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.467+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd90150bc0 con 0x7efd9006d420 2024-07-17T06:54:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.468+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9006d850 con 0x7efd9006d420 2024-07-17T06:54:47.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.470+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7efd80010860 con 0x7efd9006d420 2024-07-17T06:54:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.471+0000 7efd87e00640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 0x7efd70079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.471+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7efd800984b0 con 0x7efd9006d420 2024-07-17T06:54:47.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.471+0000 7efd8f400640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 0x7efd70079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:47.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.472+0000 7efd8f400640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 0x7efd70079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd9014f660 tx=0x7efd7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.476+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efd80061b50 con 0x7efd9006d420 2024-07-17T06:54:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:47.688+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7efd900cffe0 con 0x7efd9006d420 2024-07-17T06:54:48.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.237+0000 7efd87e00640 1 -- 172.21.15.44:0/1946487621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7efd80061450 con 0x7efd9006d420 2024-07-17T06:54:48.236 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.240+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 msgr2=0x7efd70079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.240+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 0x7efd70079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd9014f660 tx=0x7efd7c008040 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.240+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 msgr2=0x7efd901513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.240+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd8000d980 tx=0x7efd8000de40 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 shutdown_connections 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd901503c0 0x7efd901f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efd70076bb0 0x7efd70079070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd9006f4d0 0x7efd90158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 --2- 172.21.15.44:0/1946487621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd9006d420 0x7efd901513c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 >> 172.21.15.44:0/1946487621 conn(0x7efd900cf0d0 msgr2=0x7efd9015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 shutdown_connections 2024-07-17T06:54:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.241+0000 7efd95600640 1 -- 172.21.15.44:0/1946487621 wait complete. 2024-07-17T06:54:48.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T06:54:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1070356966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4154600 msgr2=0x7f4cc40b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1070356966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4154600 0x7f4cc40b3ce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005fd0 tx=0x7f4cb402a540 comp rx=0 tx=0).stop 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1070356966 shutdown_connections 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1070356966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 0x7f4cc40ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1070356966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4154600 0x7f4cc40b3ce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1070356966 >> 172.21.15.44:0/1070356966 conn(0x7f4cc40b0eb0 msgr2=0x7f4cc40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1070356966 shutdown_connections 2024-07-17T06:54:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.452+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1070356966 wait complete. 2024-07-17T06:54:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.454+0000 7f4cc8c00640 1 Processor -- start 2024-07-17T06:54:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.454+0000 7f4cc8c00640 1 -- start start 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 0x7f4cc4156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cc4154600 0x7f4cc41515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc415e220 con 0x7f4cc4161120 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc415e0a0 con 0x7f4cc4154600 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc415e3a0 con 0x7f4cc40ac190 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36352/0 (socket says 172.21.15.44:36352) 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.455+0000 7f4cc2800640 1 -- 172.21.15.44:0/1389244939 learned_addr learned my addr 172.21.15.44:0/1389244939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc1e00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 0x7f4cc4156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cbbe00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cc4154600 0x7f4cc41515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 -- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 msgr2=0x7f4cc4156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 0x7f4cc4156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 -- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cc4154600 msgr2=0x7f4cc41515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cc4154600 0x7f4cc41515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 -- 172.21.15.44:0/1389244939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc41f5170 con 0x7f4cc4161120 2024-07-17T06:54:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cc2800640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b9a0 tx=0x7f4cbc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.456+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc019070 con 0x7f4cc4161120 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.457+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cbc00ccb0 con 0x7f4cc4161120 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.457+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc4151f90 con 0x7f4cc4161120 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.457+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0146a0 con 0x7f4cc4161120 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.457+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc4152550 con 0x7f4cc4161120 2024-07-17T06:54:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.458+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc415e9b0 con 0x7f4cc4161120 2024-07-17T06:54:48.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.459+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f4cbc01c070 con 0x7f4cc4161120 2024-07-17T06:54:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.460+0000 7f4cba000640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 0x7f4ca0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.460+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f4cbc098130 con 0x7f4cc4161120 2024-07-17T06:54:48.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.461+0000 7f4cc1e00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 0x7f4ca0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.462+0000 7f4cc1e00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 0x7f4ca0079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4033000 tx=0x7f4cb4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:48.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.468+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4cbc061540 con 0x7f4cc4161120 2024-07-17T06:54:48.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:48.716+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4cc41f5340 con 0x7f4cc4161120 2024-07-17T06:54:49.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.281+0000 7f4cba000640 1 -- 172.21.15.44:0/1389244939 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f4cbc0820c0 con 0x7f4cc4161120 2024-07-17T06:54:49.281 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-07-17T06:54:49.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.293+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 msgr2=0x7f4ca0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.293+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 0x7f4ca0079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4033000 tx=0x7f4cb4004730 comp rx=0 tx=0).stop 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.293+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 msgr2=0x7f4cc415f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.293+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00b9a0 tx=0x7f4cbc00be60 comp rx=0 tx=0).stop 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 shutdown_connections 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc4161120 0x7f4cc415f960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cc4154600 0x7f4cc41515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4ca0076bb0 0x7f4ca0079070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 --2- 172.21.15.44:0/1389244939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cc40ac190 0x7f4cc4156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 >> 172.21.15.44:0/1389244939 conn(0x7f4cc40b0eb0 msgr2=0x7f4cc40ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 shutdown_connections 2024-07-17T06:54:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.294+0000 7f4cc8c00640 1 -- 172.21.15.44:0/1389244939 wait complete. 2024-07-17T06:54:49.326 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-07-17T06:54:49.327 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 -- 172.21.15.44:0/2786666836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc0015b060 msgr2=0x7fdc0006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2786666836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc0015b060 0x7fdc0006d620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009830 tx=0x7fdbf002a4d0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 -- 172.21.15.44:0/2786666836 shutdown_connections 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2786666836 >> v2:172.21.15.175:3300/0 conn(0x7fdc000cb880 0x7fdc0006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2786666836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc0015b060 0x7fdc0006d620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 -- 172.21.15.44:0/2786666836 >> 172.21.15.44:0/2786666836 conn(0x7fdc000b0eb0 msgr2=0x7fdc000b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.504+0000 7fdc05c00640 1 -- 172.21.15.44:0/2786666836 shutdown_connections 2024-07-17T06:54:49.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.505+0000 7fdc05c00640 1 -- 172.21.15.44:0/2786666836 wait complete. 2024-07-17T06:54:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.506+0000 7fdc05c00640 1 Processor -- start 2024-07-17T06:54:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.506+0000 7fdc05c00640 1 -- start start 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc0015b060 0x7fdc000c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc0015c220 0x7fdc000c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0015b5c0 con 0x7fdc000cb880 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0015b440 con 0x7fdc0015c220 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdc05c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0015f4a0 con 0x7fdc0015b060 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdbfea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdbfea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46764/0 (socket says 172.21.15.44:46764) 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.507+0000 7fdbfea00640 1 -- 172.21.15.44:0/2478916127 learned_addr learned my addr 172.21.15.44:0/2478916127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 -- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc0015b060 msgr2=0x7fdc000c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc0015b060 0x7fdc000c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 -- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc0015c220 msgr2=0x7fdc000c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc0015c220 0x7fdc000c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 -- 172.21.15.44:0/2478916127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc001f42b0 con 0x7fdc000cb880 2024-07-17T06:54:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbfea00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0040070 con 0x7fdc000cb880 2024-07-17T06:54:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbf0003af0 con 0x7fdc000cb880 2024-07-17T06:54:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0045460 con 0x7fdc000cb880 2024-07-17T06:54:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.508+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc001f4590 con 0x7fdc000cb880 2024-07-17T06:54:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.509+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001f3fa0 con 0x7fdc000cb880 2024-07-17T06:54:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.510+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0015f9c0 con 0x7fdc000cb880 2024-07-17T06:54:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.511+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fdbf0039040 con 0x7fdc000cb880 2024-07-17T06:54:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.512+0000 7fdbf6a00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 0x7fdbdc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.512+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbf00b9a10 con 0x7fdc000cb880 2024-07-17T06:54:49.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.512+0000 7fdbfe000640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 0x7fdbdc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:49.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.513+0000 7fdbfe000640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 0x7fdbdc079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc0015a180 tx=0x7fdbe8005c70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.516+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdbf0082ed0 con 0x7fdc000cb880 2024-07-17T06:54:49.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:49.726+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdc0015d0e0 con 0x7fdc000cb880 2024-07-17T06:54:50.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.293+0000 7fdbf6a00640 1 -- 172.21.15.44:0/2478916127 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fdbf00827d0 con 0x7fdc000cb880 2024-07-17T06:54:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 msgr2=0x7fdbdc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 0x7fdbdc079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc0015a180 tx=0x7fdbe8005c70 comp rx=0 tx=0).stop 2024-07-17T06:54:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 msgr2=0x7fdc0015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf0032d50 comp rx=0 tx=0).stop 2024-07-17T06:54:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 shutdown_connections 2024-07-17T06:54:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc0015c220 0x7fdc000c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdbdc076bb0 0x7fdbdc079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc0015b060 0x7fdc000c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 --2- 172.21.15.44:0/2478916127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc000cb880 0x7fdc0015d850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 >> 172.21.15.44:0/2478916127 conn(0x7fdc000b0eb0 msgr2=0x7fdc0006d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:50.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.297+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 shutdown_connections 2024-07-17T06:54:50.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.298+0000 7fdc05c00640 1 -- 172.21.15.44:0/2478916127 wait complete. 2024-07-17T06:54:50.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T06:54:50.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 -- 172.21.15.44:0/2253285324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0040051c0 msgr2=0x7fe00400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 --2- 172.21.15.44:0/2253285324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0040051c0 0x7fe00400b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe018002230 tx=0x7fe004005830 comp rx=0 tx=0).stop 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe025800640 1 -- 172.21.15.44:0/2253285324 reap_dead start 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 -- 172.21.15.44:0/2253285324 shutdown_connections 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 -- 172.21.15.44:0/2253285324 >> 172.21.15.44:0/2253285324 conn(0x7fe02008ca20 msgr2=0x7fe02008ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 -- 172.21.15.44:0/2253285324 shutdown_connections 2024-07-17T06:54:50.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.536+0000 7fe026200640 1 -- 172.21.15.44:0/2253285324 wait complete. 2024-07-17T06:54:50.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.538+0000 7fe026200640 1 Processor -- start 2024-07-17T06:54:50.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.538+0000 7fe026200640 1 -- start start 2024-07-17T06:54:50.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 0x7fe0201f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0200b8c70 0x7fe02015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200d2190 con 0x7fe02015f840 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200cf080 con 0x7fe0040051c0 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe026200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0200d2310 con 0x7fe0200b8c70 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe017e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe017e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46786/0 (socket says 172.21.15.44:46786) 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.539+0000 7fe017e00640 1 -- 172.21.15.44:0/1990075885 learned_addr learned my addr 172.21.15.44:0/1990075885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe024e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 0x7fe0201f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe025800640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0200b8c70 0x7fe02015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:50.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 -- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0200b8c70 msgr2=0x7fe02015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0200b8c70 0x7fe02015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 -- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 msgr2=0x7fe0201f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 0x7fe0201f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 -- 172.21.15.44:0/1990075885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0200b64c0 con 0x7fe02015f840 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe024e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 0x7fe0201f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe017e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe008011a80 tx=0x7fe008011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00800fd50 con 0x7fe02015f840 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe008004020 con 0x7fe02015f840 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008004490 con 0x7fe02015f840 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.540+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0200b67a0 con 0x7fe02015f840 2024-07-17T06:54:50.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.541+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0201f46d0 con 0x7fe02015f840 2024-07-17T06:54:50.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.542+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0200d2dd0 con 0x7fe02015f840 2024-07-17T06:54:50.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.543+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe0080035f0 con 0x7fe02015f840 2024-07-17T06:54:50.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.544+0000 7fe01d600640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 0x7fdffc079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:50.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.544+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fe008099290 con 0x7fe02015f840 2024-07-17T06:54:50.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.545+0000 7fe024e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 0x7fdffc079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:50.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.546+0000 7fe024e00640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 0x7fdffc079210 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe010005d00 tx=0x7fe010005f30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:50.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.548+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0200d2dd0 con 0x7fe02015f840 2024-07-17T06:54:50.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:50.753+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe0200cfc80 con 0x7fe02015f840 2024-07-17T06:54:52.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.310+0000 7fe01d600640 1 -- 172.21.15.44:0/1990075885 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fe008062590 con 0x7fe02015f840 2024-07-17T06:54:52.310 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' created 2024-07-17T06:54:52.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 msgr2=0x7fdffc079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 0x7fdffc079210 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe010005d00 tx=0x7fe010005f30 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 msgr2=0x7fe0200b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe008011a80 tx=0x7fe008011f40 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 shutdown_connections 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdffc076d50 0x7fdffc079210 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0200b8c70 0x7fe02015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02015f840 0x7fe0200b8730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 --2- 172.21.15.44:0/1990075885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0040051c0 0x7fe0201f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.316+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 >> 172.21.15.44:0/1990075885 conn(0x7fe02008ca20 msgr2=0x7fe0200cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:52.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.317+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 shutdown_connections 2024-07-17T06:54:52.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.317+0000 7fe026200640 1 -- 172.21.15.44:0/1990075885 wait complete. 2024-07-17T06:54:52.344 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-07-17T06:54:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 -- 172.21.15.44:0/41864253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f869015e370 msgr2=0x7f86900ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 --2- 172.21.15.44:0/41864253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f869015e370 0x7f86900ce500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8690160e60 tx=0x7f868002a410 comp rx=0 tx=0).stop 2024-07-17T06:54:52.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 -- 172.21.15.44:0/41864253 shutdown_connections 2024-07-17T06:54:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 --2- 172.21.15.44:0/41864253 >> v2:172.21.15.175:3300/0 conn(0x7f86900d0b40 0x7f86900b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 --2- 172.21.15.44:0/41864253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f869015e370 0x7f86900ce500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 -- 172.21.15.44:0/41864253 >> 172.21.15.44:0/41864253 conn(0x7f8690073a60 msgr2=0x7f8690073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 -- 172.21.15.44:0/41864253 shutdown_connections 2024-07-17T06:54:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.535+0000 7f868fe00640 1 -- 172.21.15.44:0/41864253 wait complete. 2024-07-17T06:54:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.537+0000 7f868fe00640 1 Processor -- start 2024-07-17T06:54:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.537+0000 7f868fe00640 1 -- start start 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900d0b40 0x7f869014f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f869015e370 0x7f86900cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86900b8d30 con 0x7f8690161b50 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86900b8bb0 con 0x7f86900d0b40 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f86900b8eb0 con 0x7f869015e370 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46802/0 (socket says 172.21.15.44:46802) 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.538+0000 7f868f400640 1 -- 172.21.15.44:0/1188954756 learned_addr learned my addr 172.21.15.44:0/1188954756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868e000640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f869015e370 0x7f86900cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868ea00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900d0b40 0x7f869014f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 -- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f869015e370 msgr2=0x7f86900cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f869015e370 0x7f86900cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 -- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900d0b40 msgr2=0x7f869014f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900d0b40 0x7f869014f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 -- 172.21.15.44:0/1188954756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86900ca730 con 0x7f8690161b50 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.539+0000 7f868f400640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f868800eaa0 tx=0x7f868800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.540+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868800cdb0 con 0x7f8690161b50 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.540+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8688004c10 con 0x7f8690161b50 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.540+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86900caa10 con 0x7f8690161b50 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.540+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688005020 con 0x7f8690161b50 2024-07-17T06:54:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.540+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86900cd7e0 con 0x7f8690161b50 2024-07-17T06:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.541+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86900cac50 con 0x7f8690161b50 2024-07-17T06:54:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.543+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f86880051c0 con 0x7f8690161b50 2024-07-17T06:54:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.543+0000 7f8686a00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 0x7f8670079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.544+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f8688098fe0 con 0x7f8690161b50 2024-07-17T06:54:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.544+0000 7f868ea00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 0x7f8670079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.545+0000 7f868ea00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 0x7f8670079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8680034000 tx=0x7f8680004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:52.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.548+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f86900cac50 con 0x7f8690161b50 2024-07-17T06:54:52.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:52.759+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f86900b9360 con 0x7f8690161b50 2024-07-17T06:54:53.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.315+0000 7f8686a00640 1 -- 172.21.15.44:0/1188954756 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f8688016030 con 0x7f8690161b50 2024-07-17T06:54:53.314 INFO:teuthology.orchestra.run.smithi044.stderr:set pool 4 allow_ec_overwrites to true 2024-07-17T06:54:53.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.322+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 msgr2=0x7f8670079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:53.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.322+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 0x7f8670079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8680034000 tx=0x7f8680004730 comp rx=0 tx=0).stop 2024-07-17T06:54:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.322+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 msgr2=0x7f86900ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:53.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.322+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f868800eaa0 tx=0x7f868800ef60 comp rx=0 tx=0).stop 2024-07-17T06:54:53.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.328+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 shutdown_connections 2024-07-17T06:54:53.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.329+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690161b50 0x7f86900ccef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.329+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8670076bb0 0x7f8670079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.329+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f869015e370 0x7f86900cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.329+0000 7f868fe00640 1 --2- 172.21.15.44:0/1188954756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86900d0b40 0x7f869014f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.329+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 >> 172.21.15.44:0/1188954756 conn(0x7f8690073a60 msgr2=0x7f869015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:53.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.330+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 shutdown_connections 2024-07-17T06:54:53.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.330+0000 7f868fe00640 1 -- 172.21.15.44:0/1188954756 wait complete. 2024-07-17T06:54:53.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-07-17T06:54:53.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 -- 172.21.15.44:0/902214412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b3cb0 msgr2=0x7f6b980b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/902214412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b3cb0 0x7f6b980b29e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b3c10 tx=0x7f6b8402a560 comp rx=0 tx=0).stop 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 -- 172.21.15.44:0/902214412 shutdown_connections 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/902214412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980b2f20 0x7f6b9815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/902214412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b3cb0 0x7f6b980b29e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/902214412 >> v2:172.21.15.175:3300/0 conn(0x7f6b980ac390 0x7f6b9815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 -- 172.21.15.44:0/902214412 >> 172.21.15.44:0/902214412 conn(0x7f6b9806c120 msgr2=0x7f6b9806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 -- 172.21.15.44:0/902214412 shutdown_connections 2024-07-17T06:54:53.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.546+0000 7f6b9ea00640 1 -- 172.21.15.44:0/902214412 wait complete. 2024-07-17T06:54:53.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.548+0000 7f6b9ea00640 1 Processor -- start 2024-07-17T06:54:53.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.548+0000 7f6b9ea00640 1 -- start start 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980ac390 0x7f6b9809be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980b3cb0 0x7f6b9815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9815b090 con 0x7f6b980b2f20 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9815af10 con 0x7f6b980b3cb0 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.549+0000 7f6b9ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9806d690 con 0x7f6b980ac390 2024-07-17T06:54:53.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46822/0 (socket says 172.21.15.44:46822) 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 -- 172.21.15.44:0/612617185 learned_addr learned my addr 172.21.15.44:0/612617185 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9e000640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980b3cb0 0x7f6b9815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9d600640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980ac390 0x7f6b9809be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 -- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980ac390 msgr2=0x7f6b9809be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980ac390 0x7f6b9809be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 -- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980b3cb0 msgr2=0x7f6b9815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980b3cb0 0x7f6b9815d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 -- 172.21.15.44:0/612617185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9809c380 con 0x7f6b980b2f20 2024-07-17T06:54:53.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.550+0000 7f6b9cc00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b84034040 tx=0x7f6b840039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.551+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8403e030 con 0x7f6b980b2f20 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.551+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b840047a0 con 0x7f6b980b2f20 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.551+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84035040 con 0x7f6b980b2f20 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.551+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b981f5bd0 con 0x7f6b980b2f20 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.551+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b981f87c0 con 0x7f6b980b2f20 2024-07-17T06:54:53.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.552+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b981f5da0 con 0x7f6b980b2f20 2024-07-17T06:54:53.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.553+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f6b84041030 con 0x7f6b980b2f20 2024-07-17T06:54:53.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.554+0000 7f6b95600640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 0x7f6b78079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:53.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.554+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f6b840b8dc0 con 0x7f6b980b2f20 2024-07-17T06:54:53.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.555+0000 7f6b9d600640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 0x7f6b78079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:53.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.556+0000 7f6b9d600640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 0x7f6b78079070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0058c0 tx=0x7f6b8c00a4c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:53.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.559+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6b84081f60 con 0x7f6b980b2f20 2024-07-17T06:54:53.616 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T06:54:53.617+0000 7f872a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T06:54:53.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:53.819+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6b9806f7e0 con 0x7f6b980b2f20 2024-07-17T06:54:54.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.345+0000 7f6b95600640 1 -- 172.21.15.44:0/612617185 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f6b8408a1a0 con 0x7f6b980b2f20 2024-07-17T06:54:54.345 INFO:teuthology.orchestra.run.smithi044.stderr:added data pool 4 to fsmap 2024-07-17T06:54:54.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 msgr2=0x7f6b78079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 0x7f6b78079070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0058c0 tx=0x7f6b8c00a4c0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 msgr2=0x7f6b9815cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b84034040 tx=0x7f6b840039e0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 shutdown_connections 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b980b3cb0 0x7f6b9815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b980b2f20 0x7f6b9815cb90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6b78076bb0 0x7f6b78079070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 --2- 172.21.15.44:0/612617185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b980ac390 0x7f6b9809be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 >> 172.21.15.44:0/612617185 conn(0x7f6b9806c120 msgr2=0x7f6b980734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:54.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 shutdown_connections 2024-07-17T06:54:54.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.349+0000 7f6b9ea00640 1 -- 172.21.15.44:0/612617185 wait complete. 2024-07-17T06:54:54.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 -- 172.21.15.44:0/1263036622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 msgr2=0x7fef8c0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 --2- 172.21.15.44:0/1263036622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0b5200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0d0000 tx=0x7fef7402a5d0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 -- 172.21.15.44:0/1263036622 shutdown_connections 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 --2- 172.21.15.44:0/1263036622 >> v2:172.21.15.175:3300/0 conn(0x7fef8c0b5740 0x7fef8c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 --2- 172.21.15.44:0/1263036622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0b5200 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 -- 172.21.15.44:0/1263036622 >> 172.21.15.44:0/1263036622 conn(0x7fef8c0d00a0 msgr2=0x7fef8c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 -- 172.21.15.44:0/1263036622 shutdown_connections 2024-07-17T06:54:54.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.581+0000 7fef92e00640 1 -- 172.21.15.44:0/1263036622 wait complete. 2024-07-17T06:54:54.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.582+0000 7fef92e00640 1 Processor -- start 2024-07-17T06:54:54.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.583+0000 7fef92e00640 1 -- start start 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef8c0b5740 0x7fef8c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c15d600 0x7fef8c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0ce670 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c0ce4f0 con 0x7fef8c15d600 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef92e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c0ce7f0 con 0x7fef8c0b5740 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46848/0 (socket says 172.21.15.44:46848) 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 -- 172.21.15.44:0/331535340 learned_addr learned my addr 172.21.15.44:0/331535340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 -- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef8c0b5740 msgr2=0x7fef8c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef8c0b5740 0x7fef8c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 -- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c15d600 msgr2=0x7fef8c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c15d600 0x7fef8c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.584+0000 7fef8be00640 1 -- 172.21.15.44:0/331535340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c0ca700 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef8be00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00cbf0 tx=0x7fef7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c016020 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef7c00ad90 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c012680 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c0ca970 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.585+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c0b3280 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.586+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c1f4b40 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.587+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fef7c019020 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.588+0000 7fef88c00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 0x7fef68079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:54.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.589+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fef7c098ce0 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.589+0000 7fef83400640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 0x7fef68079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:54.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.590+0000 7fef83400640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 0x7fef68079070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef74034000 tx=0x7fef740046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.592+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fef7c020020 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.848+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef8c0ced10 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.849+0000 7fef88c00640 1 -- 172.21.15.44:0/331535340 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5219 (secure 0 0 0) 0x7fef7c061e60 con 0x7fef8c0b2ad0 2024-07-17T06:54:54.849 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:54.849 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"btime":"2024-07-17T06:54:54:329445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:54:53.821186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:54:54.849 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 msgr2=0x7fef68079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 0x7fef68079070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef74034000 tx=0x7fef740046c0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 msgr2=0x7fef8c0cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00cbf0 tx=0x7fef7c00a060 comp rx=0 tx=0).stop 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 shutdown_connections 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef8c15d600 0x7fef8c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fef68076bb0 0x7fef68079070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef8c0b5740 0x7fef8c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 --2- 172.21.15.44:0/331535340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef8c0b2ad0 0x7fef8c0cb050 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:54.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 >> 172.21.15.44:0/331535340 conn(0x7fef8c0d00a0 msgr2=0x7fef8c0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:54.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 shutdown_connections 2024-07-17T06:54:54.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:54.853+0000 7fef92e00640 1 -- 172.21.15.44:0/331535340 wait complete. 2024-07-17T06:54:54.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:55.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 -- 172.21.15.44:0/2365389915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9341594b0 msgr2=0x7fa9340abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 --2- 172.21.15.44:0/2365389915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9341594b0 0x7fa9340abf50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa93409c550 tx=0x7fa92402a4a0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 -- 172.21.15.44:0/2365389915 shutdown_connections 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 --2- 172.21.15.44:0/2365389915 >> v2:172.21.15.175:3300/0 conn(0x7fa93409b7c0 0x7fa9340b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 --2- 172.21.15.44:0/2365389915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9341594b0 0x7fa9340abf50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 -- 172.21.15.44:0/2365389915 >> 172.21.15.44:0/2365389915 conn(0x7fa9340b0e80 msgr2=0x7fa9340b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 -- 172.21.15.44:0/2365389915 shutdown_connections 2024-07-17T06:54:55.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.069+0000 7fa93aa00640 1 -- 172.21.15.44:0/2365389915 wait complete. 2024-07-17T06:54:55.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.071+0000 7fa93aa00640 1 Processor -- start 2024-07-17T06:54:55.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.071+0000 7fa93aa00640 1 -- start start 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa93409b7c0 0x7fa93414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9341594b0 0x7fa934158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340cf7f0 con 0x7fa934159890 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340cf670 con 0x7fa9341594b0 2024-07-17T06:54:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9340cf970 con 0x7fa93409b7c0 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.072+0000 7fa93a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46876/0 (socket says 172.21.15.44:46876) 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 -- 172.21.15.44:0/1708569527 learned_addr learned my addr 172.21.15.44:0/1708569527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa939600640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa93409b7c0 0x7fa93414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa938c00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9341594b0 0x7fa934158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 -- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa93409b7c0 msgr2=0x7fa93414db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa93409b7c0 0x7fa93414db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 -- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9341594b0 msgr2=0x7fa934158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9341594b0 0x7fa934158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 -- 172.21.15.44:0/1708569527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9341f7140 con 0x7fa934159890 2024-07-17T06:54:55.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.073+0000 7fa93a000640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00db40 tx=0x7fa92c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.074+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c004020 con 0x7fa934159890 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.074+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92c0042b0 con 0x7fa934159890 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.074+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c003850 con 0x7fa934159890 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.074+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9341f7420 con 0x7fa934159890 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.074+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341f79e0 con 0x7fa934159890 2024-07-17T06:54:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.075+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93415c2d0 con 0x7fa934159890 2024-07-17T06:54:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.076+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fa92c0039f0 con 0x7fa934159890 2024-07-17T06:54:55.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.077+0000 7fa931600640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 0x7fa9180790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.078+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa92c098cd0 con 0x7fa934159890 2024-07-17T06:54:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.078+0000 7fa939600640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 0x7fa9180790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.079+0000 7fa939600640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 0x7fa9180790c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa924034000 tx=0x7fa924004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:55.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.081+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa92c061e40 con 0x7fa934159890 2024-07-17T06:54:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.293+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9340b30a0 con 0x7fa934159890 2024-07-17T06:54:55.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.296+0000 7fa931600640 1 -- 172.21.15.44:0/1708569527 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fa92c061740 con 0x7fa934159890 2024-07-17T06:54:55.295 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:55.295 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:55.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 msgr2=0x7fa9180790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 0x7fa9180790c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa924034000 tx=0x7fa924004730 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 msgr2=0x7fa9341e6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00db40 tx=0x7fa92c00a250 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 shutdown_connections 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa934159890 0x7fa9341e6300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9341594b0 0x7fa934158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa918076c00 0x7fa9180790c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 --2- 172.21.15.44:0/1708569527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa93409b7c0 0x7fa93414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.299+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 >> 172.21.15.44:0/1708569527 conn(0x7fa9340b0e80 msgr2=0x7fa9340733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.300+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 shutdown_connections 2024-07-17T06:54:55.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.300+0000 7fa93aa00640 1 -- 172.21.15.44:0/1708569527 wait complete. 2024-07-17T06:54:55.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:55.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 -- 172.21.15.44:0/335514005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d81522c0 msgr2=0x7f51d80ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 --2- 172.21.15.44:0/335514005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d81522c0 0x7f51d80ad790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005fd0 tx=0x7f51c802a530 comp rx=0 tx=0).stop 2024-07-17T06:54:55.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 -- 172.21.15.44:0/335514005 shutdown_connections 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 --2- 172.21.15.44:0/335514005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 0x7f51d80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 --2- 172.21.15.44:0/335514005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d81522c0 0x7f51d80ad790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 -- 172.21.15.44:0/335514005 >> 172.21.15.44:0/335514005 conn(0x7f51d80b0fb0 msgr2=0x7f51d80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 -- 172.21.15.44:0/335514005 shutdown_connections 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.520+0000 7f51dde00640 1 -- 172.21.15.44:0/335514005 wait complete. 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.522+0000 7f51dde00640 1 Processor -- start 2024-07-17T06:54:55.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.522+0000 7f51dde00640 1 -- start start 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 0x7f51d80737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d81522c0 0x7f51d81e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80cdf60 con 0x7f51d80adcd0 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80cdde0 con 0x7f51d81522c0 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51dde00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51d80ce0e0 con 0x7f51d80ac930 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46882/0 (socket says 172.21.15.44:46882) 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51cfe00640 1 -- 172.21.15.44:0/3349502690 learned_addr learned my addr 172.21.15.44:0/3349502690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.523+0000 7f51d7400640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d81522c0 0x7f51d81e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51d6a00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 0x7f51d80737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 -- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 msgr2=0x7f51d80737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 0x7f51d80737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 -- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d81522c0 msgr2=0x7f51d81e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d81522c0 0x7f51d81e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 -- 172.21.15.44:0/3349502690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d8073dd0 con 0x7f51d80adcd0 2024-07-17T06:54:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cfe00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c460 tx=0x7f51c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011790 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011d00 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.524+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d8074110 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.525+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d81f3c10 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.526+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d80ce630 con 0x7f51d80adcd0 2024-07-17T06:54:55.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.527+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f51c001c460 con 0x7f51d80adcd0 2024-07-17T06:54:55.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.528+0000 7f51cf400640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 0x7f51b4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.528+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51c0098630 con 0x7f51d80adcd0 2024-07-17T06:54:55.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.528+0000 7f51d6a00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 0x7f51b4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.530+0000 7f51d6a00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 0x7f51b4079070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51c8033000 tx=0x7f51c8004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:55.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.532+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f51d8074110 con 0x7f51d80adcd0 2024-07-17T06:54:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.747+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51d815d780 con 0x7f51d80adcd0 2024-07-17T06:54:55.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.749+0000 7f51cf400640 1 -- 172.21.15.44:0/3349502690 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f51c0061720 con 0x7f51d80adcd0 2024-07-17T06:54:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:55.749 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 msgr2=0x7f51b4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 0x7f51b4079070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51c8033000 tx=0x7f51c8004730 comp rx=0 tx=0).stop 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 msgr2=0x7f51d81553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c460 tx=0x7f51c000c920 comp rx=0 tx=0).stop 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 shutdown_connections 2024-07-17T06:54:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51d81522c0 0x7f51d81e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51d80adcd0 0x7f51d81553e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f51b4076bb0 0x7f51b4079070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 --2- 172.21.15.44:0/3349502690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51d80ac930 0x7f51d80737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 >> 172.21.15.44:0/3349502690 conn(0x7f51d80b0fb0 msgr2=0x7f51d815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 shutdown_connections 2024-07-17T06:54:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.752+0000 7f51dde00640 1 -- 172.21.15.44:0/3349502690 wait complete. 2024-07-17T06:54:55.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:54:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 -- 172.21.15.44:0/524444365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b80acb90 msgr2=0x7f82b806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 --2- 172.21.15.44:0/524444365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b80acb90 0x7f82b806f020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009830 tx=0x7f82a002a510 comp rx=0 tx=0).stop 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 -- 172.21.15.44:0/524444365 shutdown_connections 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 --2- 172.21.15.44:0/524444365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b80acb90 0x7f82b806f020 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 --2- 172.21.15.44:0/524444365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b815b2e0 0x7f82b80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 -- 172.21.15.44:0/524444365 >> 172.21.15.44:0/524444365 conn(0x7f82b80b0fb0 msgr2=0x7f82b80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 -- 172.21.15.44:0/524444365 shutdown_connections 2024-07-17T06:54:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.995+0000 7f82bcc00640 1 -- 172.21.15.44:0/524444365 wait complete. 2024-07-17T06:54:55.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.997+0000 7f82bcc00640 1 Processor -- start 2024-07-17T06:54:55.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.997+0000 7f82bcc00640 1 -- start start 2024-07-17T06:54:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82b806f560 0x7f82b8073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b80acb90 0x7f82b8152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82b81548e0 con 0x7f82b815b2e0 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82b8154760 con 0x7f82b806f560 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82bcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82b8154a60 con 0x7f82b80acb90 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.998+0000 7f82b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46904/0 (socket says 172.21.15.44:46904) 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 -- 172.21.15.44:0/1837306853 learned_addr learned my addr 172.21.15.44:0/1837306853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b5e00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82b806f560 0x7f82b8073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82afe00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b80acb90 0x7f82b8152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 -- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b80acb90 msgr2=0x7f82b8152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b80acb90 0x7f82b8152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 -- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82b806f560 msgr2=0x7f82b8073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82b806f560 0x7f82b8073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 -- 172.21.15.44:0/1837306853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b81f7ad0 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:55.999+0000 7f82b6800640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82b000ca70 tx=0x7f82b000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.000+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0016020 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.000+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b000acc0 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.000+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0012680 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.000+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b81f7db0 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.000+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b81f8370 con 0x7f82b815b2e0 2024-07-17T06:54:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.001+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b806f020 con 0x7f82b815b2e0 2024-07-17T06:54:56.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.002+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f82b0019020 con 0x7f82b815b2e0 2024-07-17T06:54:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.003+0000 7f82ae000640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 0x7f8294079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.003+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f82b0098c20 con 0x7f82b815b2e0 2024-07-17T06:54:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.004+0000 7f82b5e00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 0x7f8294079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.005+0000 7f82b5e00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 0x7f8294079070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82a800bb50 tx=0x7f82a800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.008+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f82b0020020 con 0x7f82b815b2e0 2024-07-17T06:54:56.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.256+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82b806dbf0 con 0x7f82b815b2e0 2024-07-17T06:54:56.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.257+0000 7f82ae000640 1 -- 172.21.15.44:0/1837306853 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5219 (secure 0 0 0) 0x7f82b0061da0 con 0x7f82b815b2e0 2024-07-17T06:54:56.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:56.256 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"btime":"2024-07-17T06:54:54:329445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:54:53.821186+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:54:56.257 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 msgr2=0x7f8294079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 0x7f8294079070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82a800bb50 tx=0x7f82a800b2e0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 msgr2=0x7f82b8158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82b000ca70 tx=0x7f82b000cf30 comp rx=0 tx=0).stop 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 shutdown_connections 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b815b2e0 0x7f82b8158230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8294076bb0 0x7f8294079070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b80acb90 0x7f82b8152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 --2- 172.21.15.44:0/1837306853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82b806f560 0x7f82b8073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 >> 172.21.15.44:0/1837306853 conn(0x7f82b80b0fb0 msgr2=0x7f82b80b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 shutdown_connections 2024-07-17T06:54:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.260+0000 7f82bcc00640 1 -- 172.21.15.44:0/1837306853 wait complete. 2024-07-17T06:54:56.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:54:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 -- 172.21.15.44:0/3575336373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 msgr2=0x7f79300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 --2- 172.21.15.44:0/3575336373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f79300051c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7944009e20 tx=0x7f7930005830 comp rx=0 tx=0).stop 2024-07-17T06:54:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794b400640 1 -- 172.21.15.44:0/3575336373 reap_dead start 2024-07-17T06:54:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 -- 172.21.15.44:0/3575336373 shutdown_connections 2024-07-17T06:54:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 -- 172.21.15.44:0/3575336373 >> 172.21.15.44:0/3575336373 conn(0x7f794c0cfec0 msgr2=0x7f794c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:56.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 -- 172.21.15.44:0/3575336373 shutdown_connections 2024-07-17T06:54:56.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.488+0000 7f794be00640 1 -- 172.21.15.44:0/3575336373 wait complete. 2024-07-17T06:54:56.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.490+0000 7f794be00640 1 Processor -- start 2024-07-17T06:54:56.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.490+0000 7f794be00640 1 -- start start 2024-07-17T06:54:56.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:56.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f794c1f7520 0x7f794c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f794c0b6890 0x7f794c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0b4ec0 con 0x7f7930004d40 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0b9270 con 0x7f794c0b6890 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0b5040 con 0x7f794c1f7520 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46926/0 (socket says 172.21.15.44:46926) 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.491+0000 7f794aa00640 1 -- 172.21.15.44:0/3885808376 learned_addr learned my addr 172.21.15.44:0/3885808376 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794a000640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f794c1f7520 0x7f794c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 -- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f794c1f7520 msgr2=0x7f794c0b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f794c1f7520 0x7f794c0b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 -- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f794c0b6890 msgr2=0x7f794c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f794c0b6890 0x7f794c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 -- 172.21.15.44:0/3885808376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0b7080 con 0x7f7930004d40 2024-07-17T06:54:56.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794aa00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e740 tx=0x7f793c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:56.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00cbe0 con 0x7f7930004d40 2024-07-17T06:54:56.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f793c00ee50 con 0x7f7930004d40 2024-07-17T06:54:56.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c005280 con 0x7f7930004d40 2024-07-17T06:54:56.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.492+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c0b7360 con 0x7f7930004d40 2024-07-17T06:54:56.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.493+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c0b7920 con 0x7f7930004d40 2024-07-17T06:54:56.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.494+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c06d9b0 con 0x7f7930004d40 2024-07-17T06:54:56.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.495+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f793c005420 con 0x7f7930004d40 2024-07-17T06:54:56.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.496+0000 7f7943400640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 0x7f792c0790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:56.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.496+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f793c098de0 con 0x7f7930004d40 2024-07-17T06:54:56.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.496+0000 7f794a000640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 0x7f792c0790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:56.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.497+0000 7f794a000640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 0x7f792c0790d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7938000fb0 tx=0x7f7938001620 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:56.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.500+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f793c016030 con 0x7f7930004d40 2024-07-17T06:54:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.713+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f794c1f7900 con 0x7f7930004d40 2024-07-17T06:54:56.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.716+0000 7f7943400640 1 -- 172.21.15.44:0/3885808376 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f793c061ed0 con 0x7f7930004d40 2024-07-17T06:54:56.715 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:54:56.715 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 msgr2=0x7f792c0790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 0x7f792c0790d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7938000fb0 tx=0x7f7938001620 comp rx=0 tx=0).stop 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 msgr2=0x7f794c1f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e740 tx=0x7f793c00ec00 comp rx=0 tx=0).stop 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 shutdown_connections 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f794c0b6890 0x7f794c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f792c076c10 0x7f792c0790d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f794c1f7520 0x7f794c0b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 --2- 172.21.15.44:0/3885808376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930004d40 0x7f794c1f6fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 >> 172.21.15.44:0/3885808376 conn(0x7f794c0cfec0 msgr2=0x7f794c0d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 shutdown_connections 2024-07-17T06:54:56.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:56.719+0000 7f794be00640 1 -- 172.21.15.44:0/3885808376 wait complete. 2024-07-17T06:54:57.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T06:54:48.717423+0000', 'modified': '2024-07-17T06:54:53.821186+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4329}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4329': {'gid': 4329, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/910842174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 910842174}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 910842174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4329, 'qdb_cluster': [4329]} 2024-07-17T06:54:57.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T06:54:57.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.366+0000 7f951a400640 1 -- 172.21.15.44:0/3395448367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f951415d300 msgr2=0x7f95140b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.366+0000 7f951a400640 1 --2- 172.21.15.44:0/3395448367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f951415d300 0x7f95140b4ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9508009830 tx=0x7f950802a480 comp rx=0 tx=0).stop 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.367+0000 7f951a400640 1 -- 172.21.15.44:0/3395448367 shutdown_connections 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.367+0000 7f951a400640 1 --2- 172.21.15.44:0/3395448367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f951415d300 0x7f95140b4ec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.367+0000 7f951a400640 1 -- 172.21.15.44:0/3395448367 >> 172.21.15.44:0/3395448367 conn(0x7f95140cfec0 msgr2=0x7f95140d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.367+0000 7f951a400640 1 -- 172.21.15.44:0/3395448367 shutdown_connections 2024-07-17T06:54:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.367+0000 7f951a400640 1 -- 172.21.15.44:0/3395448367 wait complete. 2024-07-17T06:54:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.368+0000 7f951a400640 1 Processor -- start 2024-07-17T06:54:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.369+0000 7f951a400640 1 -- start start 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95140d2900 0x7f9514156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 0x7f9514149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95140d1190 con 0x7f95140d2fa0 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95140d1010 con 0x7f951415d300 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f951a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95140d1310 con 0x7f95140d2900 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46950/0 (socket says 172.21.15.44:46950) 2024-07-17T06:54:57.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 -- 172.21.15.44:0/68667690 learned_addr learned my addr 172.21.15.44:0/68667690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9519a00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 0x7f9514149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9519000640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95140d2900 0x7f9514156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 -- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95140d2900 msgr2=0x7f9514156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95140d2900 0x7f9514156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 -- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 msgr2=0x7f9514149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 0x7f9514149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9513e00640 1 -- 172.21.15.44:0/68667690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141558b0 con 0x7f95140d2fa0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.370+0000 7f9519a00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 0x7f9514149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f9513e00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f950000c430 tx=0x7f950000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f9510c00640 1 -- 172.21.15.44:0/68667690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500011790 con 0x7f95140d2fa0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f9510c00640 1 -- 172.21.15.44:0/68667690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9500017070 con 0x7f95140d2fa0 2024-07-17T06:54:57.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f9510c00640 1 -- 172.21.15.44:0/68667690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500011d00 con 0x7f95140d2fa0 2024-07-17T06:54:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9514155b90 con 0x7f95140d2fa0 2024-07-17T06:54:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.371+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9514156150 con 0x7f95140d2fa0 2024-07-17T06:54:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.373+0000 7f9510c00640 1 -- 172.21.15.44:0/68667690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f950001c460 con 0x7f95140d2fa0 2024-07-17T06:54:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.374+0000 7f9510c00640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 0x7f94f40790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.375+0000 7f9510c00640 1 -- 172.21.15.44:0/68667690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f95000985c0 con 0x7f95140d2fa0 2024-07-17T06:54:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.375+0000 7f951e81c740 1 Processor -- start 2024-07-17T06:54:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.375+0000 7f9519000640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 0x7f94f40790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.375+0000 7f951e81c740 1 -- start start 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad23cfc0 0x5591ad23d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad239c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5591ad184d60 con 0x5591ad214280 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591ad174260 con 0x5591ad23cfc0 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5591ad1782a0 con 0x5591ad239790 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9519000640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 0x7f94f40790a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f9508004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46954/0 (socket says 172.21.15.44:46954) 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9513e00640 1 -- 172.21.15.44:0/2377052389 learned_addr learned my addr 172.21.15.44:0/2377052389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9519000640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad23cfc0 0x5591ad23d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.376+0000 7f9519a00640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad239c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 -- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 msgr2=0x5591ad239c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad239c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 -- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad23cfc0 msgr2=0x5591ad23d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad23cfc0 0x5591ad23d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 -- 172.21.15.44:0/2377052389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591aced8850 con 0x5591ad214280 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9519a00640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad239c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9513e00640 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9500060fd0 tx=0x7f9500060aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed76a34859ee825 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9505600640 1 -- 172.21.15.44:0/2377052389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500065400 con 0x5591ad214280 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9505600640 1 -- 172.21.15.44:0/2377052389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9500061040 con 0x5591ad214280 2024-07-17T06:54:57.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.377+0000 7f9505600640 1 -- 172.21.15.44:0/2377052389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500098300 con 0x5591ad214280 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 -- 172.21.15.44:0/2377052389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 msgr2=0x5591ad239250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9500060fd0 tx=0x7f9500060aa0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 -- 172.21.15.44:0/2377052389 shutdown_connections 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad239c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad214280 0x5591ad239250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 --2- 172.21.15.44:0/2377052389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad23cfc0 0x5591ad23d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 -- 172.21.15.44:0/2377052389 >> 172.21.15.44:0/2377052389 conn(0x5591ad20f460 msgr2=0x5591ad212e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 -- 172.21.15.44:0/2377052389 shutdown_connections 2024-07-17T06:54:57.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.378+0000 7f951e81c740 1 -- 172.21.15.44:0/2377052389 wait complete. 2024-07-17T06:54:57.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.379+0000 7f951e81c740 1 -- start start 2024-07-17T06:54:57.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.379+0000 7f951e81c740 1 Processor -- start 2024-07-17T06:54:57.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad239790 0x5591ad24e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad26bb20 0x5591ad2513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5591acee7620 con 0x5591ad23cfc0 2024-07-17T06:54:57.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591ad11bf90 con 0x5591ad239790 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5591ad20db60 con 0x5591ad26bb20 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f951e81c740 10 client.0 _ll_register_callbacks cb 0x5591ad141f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46968/0 (socket says 172.21.15.44:46968) 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9513e00640 1 -- 172.21.15.44:0/3717694761 learned_addr learned my addr 172.21.15.44:0/3717694761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9519000640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad239790 0x5591ad24e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9513e00640 1 -- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad26bb20 msgr2=0x5591ad2513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.380+0000 7f9513e00640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad26bb20 0x5591ad2513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f9513e00640 1 -- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad239790 msgr2=0x5591ad24e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f9513e00640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad239790 0x5591ad24e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f9513e00640 1 -- 172.21.15.44:0/3717694761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591acf4f6b0 con 0x5591ad23cfc0 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f9513e00640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f950005d0c0 tx=0x7f950005d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bd9b76bb5f387d server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f94fa000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500094060 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f95000980c0 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591ad14b900 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f950007f050 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f951e81c740 10 client.4464 fetch_fsmap learned FSMap version 7 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5591ad267f40 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.381+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5591ad14b900 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f95000962b0 con 0x5591ad23cfc0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 10 client.4464 fetch_fsmap finished waiting for FSMap version 7 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 10 client.4464 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 20 client.4464 populate_metadata read hostname 'smithi044' 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad23b9a0 0x5591ad23bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 4 client.4464 mds_command: new command op to 4329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 --> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591ad186cb0 con 0x5591ad23b9a0 2024-07-17T06:54:57.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.382+0000 7f9519000640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad23b9a0 0x5591ad23bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.383+0000 7f9519000640 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad23b9a0 0x5591ad23bd80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce2df6671e97a31 server_cookie=ff8be54bf4ca043b in_seq=0 out_seq=0 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.383+0000 7f94fa000640 10 client.4464 ms_handle_connect on v2:172.21.15.175:6834/910842174 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.385+0000 7f94fa000640 1 -- 172.21.15.44:0/3717694761 <== mds.0 v2:172.21.15.175:6834/910842174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5591ad186cb0 con 0x5591ad23b9a0 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.385+0000 7f94fa000640 10 client.4464 handle_command_reply: tid=0 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.385+0000 7f951e81c740 1 client.4464 shutdown 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 msgr2=0x5591ad26b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f950005d0c0 tx=0x7f950005d580 comp rx=0 tx=0).stop 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 shutdown_connections 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad23b9a0 0x5591ad23bd80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad26bb20 0x5591ad2513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad26b5e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 --2- 172.21.15.44:0/3717694761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad239790 0x5591ad24e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f94fa000640 0 client.4464 ms_handle_reset on v2:172.21.15.175:6834/910842174 2024-07-17T06:54:57.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 >> 172.21.15.44:0/3717694761 conn(0x5591ad214280 msgr2=0x5591ad212b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 shutdown_connections 2024-07-17T06:54:57.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 1 -- 172.21.15.44:0/3717694761 wait complete. 2024-07-17T06:54:57.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.386+0000 7f951e81c740 20 client.4464 trim_cache size 0 max 16384 2024-07-17T06:54:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.407+0000 7f951e81c740 1 Processor -- start 2024-07-17T06:54:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.407+0000 7f951e81c740 1 -- start start 2024-07-17T06:54:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad2675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9514200c20 con 0x5591ad23cfc0 2024-07-17T06:54:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95142000d0 con 0x5591ad26bb20 2024-07-17T06:54:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95142005d0 con 0x5591ad239790 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f9513e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46976/0 (socket says 172.21.15.44:46976) 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f9513e00640 1 -- 172.21.15.44:0/1376382801 learned_addr learned my addr 172.21.15.44:0/1376382801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f9519a00640 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.408+0000 7f9519000640 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad2675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 -- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 msgr2=0x5591ad2675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad2675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 -- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 msgr2=0x5591ad23bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 -- 172.21.15.44:0/1376382801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ad211870 con 0x5591ad23cfc0 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9513e00640 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9500060fd0 tx=0x7f9500096670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2077d341b3be3a09 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9504c00640 1 -- 172.21.15.44:0/1376382801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500065400 con 0x5591ad23cfc0 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9504c00640 1 -- 172.21.15.44:0/1376382801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95000962b0 con 0x5591ad23cfc0 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.409+0000 7f9504c00640 1 -- 172.21.15.44:0/1376382801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000602a0 con 0x5591ad23cfc0 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- 172.21.15.44:0/1376382801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 msgr2=0x5591ad27bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9500060fd0 tx=0x7f9500096670 comp rx=0 tx=0).stop 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- 172.21.15.44:0/1376382801 shutdown_connections 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad23cfc0 0x5591ad27bfe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 --2- 172.21.15.44:0/1376382801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad239790 0x5591ad2675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- 172.21.15.44:0/1376382801 >> 172.21.15.44:0/1376382801 conn(0x5591ad214280 msgr2=0x5591ad22e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- 172.21.15.44:0/1376382801 shutdown_connections 2024-07-17T06:54:57.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- 172.21.15.44:0/1376382801 wait complete. 2024-07-17T06:54:57.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.410+0000 7f951e81c740 1 -- start start 2024-07-17T06:54:57.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.411+0000 7f951e81c740 1 Processor -- start 2024-07-17T06:54:57.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.411+0000 7f951e81c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad23cfc0 0x5591ad23bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5591acee7620 con 0x5591ad239790 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591ad20db60 con 0x5591ad26bb20 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5591ad11bf90 con 0x5591ad23cfc0 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f951e81c740 10 client.0 _ll_register_callbacks cb 0x7f95000607b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f9519000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f9519000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46990/0 (socket says 172.21.15.44:46990) 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f9519000640 1 -- 172.21.15.44:0/2699932590 learned_addr learned my addr 172.21.15.44:0/2699932590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.412+0000 7f9519a00640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9513e00640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad23cfc0 0x5591ad23bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 -- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad23cfc0 msgr2=0x5591ad23bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad23cfc0 0x5591ad23bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 -- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 msgr2=0x5591ad23c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 -- 172.21.15.44:0/2699932590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ad27c060 con 0x5591ad239790 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f9519000640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f95080043f0 tx=0x7f9508005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2628c047e258af8 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f94fa000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080038c0 con 0x5591ad239790 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9508003a60 con 0x5591ad239790 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.413+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95000962b0 con 0x5591ad239790 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f950805e050 con 0x5591ad239790 2024-07-17T06:54:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 10 client.4470 fetch_fsmap learned FSMap version 7 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5591ad236340 con 0x5591ad239790 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95000962b0 con 0x5591ad239790 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f9508061020 con 0x5591ad239790 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 10 client.4470 fetch_fsmap finished waiting for FSMap version 7 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 10 client.4470 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 20 client.4470 populate_metadata read hostname 'smithi044' 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad266f00 0x5591ad2de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 4 client.4470 mds_command: new command op to 4329 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.414+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 --> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] -- command(tid 0: {"prefix": "status"}) -- 0x5591ad182750 con 0x5591ad266f00 2024-07-17T06:54:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.415+0000 7f9513e00640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad266f00 0x5591ad2de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f9513e00640 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad266f00 0x5591ad2de130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe495e510cae8ee server_cookie=da0243242d06862a in_seq=0 out_seq=0 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f94fa000640 10 client.4470 ms_handle_connect on v2:172.21.15.175:6834/910842174 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f94fa000640 1 -- 172.21.15.44:0/2699932590 <== mds.0 v2:172.21.15.175:6834/910842174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5591ad182750 con 0x5591ad266f00 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f94fa000640 10 client.4470 handle_command_reply: tid=0 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f951e81c740 1 client.4470 shutdown 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 msgr2=0x5591ad2dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.416+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f95080043f0 tx=0x7f9508005f90 comp rx=0 tx=0).stop 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 shutdown_connections 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x5591ad266f00 0x5591ad2de130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5591ad26bb20 0x5591ad23c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5591ad23cfc0 0x5591ad23bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 --2- 172.21.15.44:0/2699932590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591ad239790 0x5591ad2dd9e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f94fa000640 0 client.4470 ms_handle_reset on v2:172.21.15.175:6834/910842174 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 >> 172.21.15.44:0/2699932590 conn(0x5591ad214280 msgr2=0x5591ad213160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 shutdown_connections 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 1 -- 172.21.15.44:0/2699932590 wait complete. 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951e81c740 20 client.4470 trim_cache size 0 max 16384 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4329, 2024-07-17T06:54:57.419 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 8.1323553230000005, 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 7, 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 25, 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 14.807870192999999 2024-07-17T06:54:57.420 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.417+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 msgr2=0x7f94f40790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 0x7f94f40790a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9508034000 tx=0x7f9508004730 comp rx=0 tx=0).stop 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 msgr2=0x7f95141f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f950000c430 tx=0x7f950000c8f0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 shutdown_connections 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f951415d300 0x7f9514149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95140d2fa0 0x7f95141f7e40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f94f4076be0 0x7f94f40790a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 --2- 172.21.15.44:0/68667690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95140d2900 0x7f9514156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 >> 172.21.15.44:0/68667690 conn(0x7f95140cfec0 msgr2=0x7f95140d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 shutdown_connections 2024-07-17T06:54:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.418+0000 7f951a400640 1 -- 172.21.15.44:0/68667690 wait complete. 2024-07-17T06:54:57.432 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-17T06:54:57.432 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:54:57.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 -- 172.21.15.44:0/3013041895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 msgr2=0x7f8ef415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 --2- 172.21.15.44:0/3013041895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef415e510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4005fd0 tx=0x7f8ee402aa00 comp rx=0 tx=0).stop 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 -- 172.21.15.44:0/3013041895 shutdown_connections 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 --2- 172.21.15.44:0/3013041895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef40ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 --2- 172.21.15.44:0/3013041895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef415e510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 -- 172.21.15.44:0/3013041895 >> 172.21.15.44:0/3013041895 conn(0x7f8ef40b0fb0 msgr2=0x7f8ef40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.625+0000 7f8efc400640 1 -- 172.21.15.44:0/3013041895 shutdown_connections 2024-07-17T06:54:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.626+0000 7f8efc400640 1 -- 172.21.15.44:0/3013041895 wait complete. 2024-07-17T06:54:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.627+0000 7f8efc400640 1 Processor -- start 2024-07-17T06:54:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.627+0000 7f8efc400640 1 -- start start 2024-07-17T06:54:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef4140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ef406ee90 0x7f8ef4143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef40b2e60 con 0x7f8ef40ae810 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef40b2ce0 con 0x7f8ef406ee90 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.628+0000 7f8efc400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef40b2fe0 con 0x7f8ef406ea50 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46996/0 (socket says 172.21.15.44:46996) 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 -- 172.21.15.44:0/601583116 learned_addr learned my addr 172.21.15.44:0/601583116 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:54:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef4140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8ef9000640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ef406ee90 0x7f8ef4143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 -- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 msgr2=0x7f8ef4140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef4140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 -- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ef406ee90 msgr2=0x7f8ef4143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ef406ee90 0x7f8ef4143bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8efa400640 1 -- 172.21.15.44:0/601583116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef41f4560 con 0x7f8ef40ae810 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.629+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef4140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8efa400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c410 tx=0x7f8eec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec017070 con 0x7f8ef40ae810 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8eec0035f0 con 0x7f8ef40ae810 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41f4840 con 0x7f8ef40ae810 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec003a40 con 0x7f8ef40ae810 2024-07-17T06:54:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.630+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef4156ed0 con 0x7f8ef40ae810 2024-07-17T06:54:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.631+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef40b2b00 con 0x7f8ef40ae810 2024-07-17T06:54:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.632+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f8eec01a070 con 0x7f8ef40ae810 2024-07-17T06:54:57.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.633+0000 7f8ef1600640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 0x7f8ed40790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:54:57.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.634+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8eec0979c0 con 0x7f8ef40ae810 2024-07-17T06:54:57.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.634+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 0x7f8ed40790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:54:57.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.635+0000 7f8ef9a00640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 0x7f8ed40790a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4035000 tx=0x7f8ee4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:54:57.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.638+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8ef40b2b00 con 0x7f8ef40ae810 2024-07-17T06:54:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.889+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ef40ac840 con 0x7f8ef40ae810 2024-07-17T06:54:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.889+0000 7f8ef1600640 1 -- 172.21.15.44:0/601583116 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f8eec060ab0 con 0x7f8ef40ae810 2024-07-17T06:54:57.888 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-17T06:54:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 msgr2=0x7f8ed40790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 0x7f8ed40790a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4035000 tx=0x7f8ee4004ab0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 msgr2=0x7f8ef40737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:54:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c410 tx=0x7f8eec00c8d0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 shutdown_connections 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.892+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ef40ae810 0x7f8ef40737f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ef406ee90 0x7f8ef4143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8ed4076be0 0x7f8ed40790a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 --2- 172.21.15.44:0/601583116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ef406ea50 0x7f8ef4140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 >> 172.21.15.44:0/601583116 conn(0x7f8ef40b0fb0 msgr2=0x7f8ef4160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 shutdown_connections 2024-07-17T06:54:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:57.893+0000 7f8efc400640 1 -- 172.21.15.44:0/601583116 wait complete. 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-17T06:54:57.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:54:57.911 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-07-17T06:54:57.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:54:57.911 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T06:54:57.960 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:54:57.960 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:54:57.960 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:54:57.960 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T06:54:57.961 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-07-17T06:54:57.962 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T06:54:57.962 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:54:57.962 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:54:57.962 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:54:57.962 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T06:54:57.963 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-17T06:54:57.963 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:54:57.963 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:54:57.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-07-17T06:54:57.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-07-17T06:54:57.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-07-17T06:54:57.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-17T06:54:57.964 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:54:58.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:54:58.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:54:58.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:54:58.470 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:54:58.535 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T06:54:58.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:54:58.539 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T06:54:58.595 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T06:54:58.596 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:54:58.596 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T06:54:58.596 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T06:54:58.596 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:54:58.596 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:54:58.596 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:54:58.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T06:54:58.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T06:54:58.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T06:54:58.597 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:54:58.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:54:59.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:54:59.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:54:59.417 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:54:59.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:54:59.462 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-07-17T06:54:59.550 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:54:59.551 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:54:59.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:54:59.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-07-17T06:54:59.551 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:54:59.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:54:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:54:59.729 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-07-17T06:54:59.729 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:54:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:54:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:00.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:00.016 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:00.017 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:00.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T06:55:00.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-07-17T06:55:00.017 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:00.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:00.208 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-07-17T06:55:00.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:00.208 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:00.248 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-07-17T06:55:00.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:00.248 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:00.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-07-17T06:55:00.408 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-07-17T06:55:00.450 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-07-17T06:55:00.451 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:00.452 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:00.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:00.453 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-07-17T06:55:00.544 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-17T06:55:00.544 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-07-17T06:55:00.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-07-17T06:55:00.618 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-07-17T06:55:00.665 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-07-17T06:55:00.665 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-07-17T06:55:00.665 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-07-17T06:55:00.666 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-07-17T06:55:00.667 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-07-17T06:55:00.668 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-07-17T06:55:00.668 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-07-17T06:55:00.668 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-07-17T06:55:00.668 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:00.668 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:00.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:00.669 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-07-17T06:55:00.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:2024-07-17T06:55:00.763+0000 7f0836383480 -1 init, newargv = 0x55c9545674c0 newargc=15 2024-07-17T06:55:00.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[44112]: starting ceph client 2024-07-17T06:55:00.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[44112]: starting fuse 2024-07-17T06:55:01.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-07-17T06:55:01.902 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-07-17T06:55:01.959 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-07-17T06:55:01.960 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-07-17T06:55:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:01.962 INFO:teuthology.orchestra.run.smithi044.stdout:/home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:01.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:01.963 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-07-17T06:55:02.017 INFO:teuthology.orchestra.run.smithi044.stdout:42 2024-07-17T06:55:02.017 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-07-17T06:55:02.017 DEBUG:teuthology.orchestra.run.smithi044:> 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-17T06:55:02.018 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.admin")) 2024-07-17T06:55:02.019 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-07-17T06:55:02.201 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.admin.44112.asok 2024-07-17T06:55:02.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:02.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44112.asok 2024-07-17T06:55:02.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:02.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44112.asok status 2024-07-17T06:55:02.452 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "0325fd3c11a225c0fd9eb38fab88aad1370ad779", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-4937-g0325fd3c (0325fd3c11a225c0fd9eb38fab88aad1370ad779) squid (dev)", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "admin", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "44112", 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4482, 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-07-17T06:55:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4482 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1256316458 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-07-17T06:55:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1256316458 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.4482 192.168.0.1:0/1256316458", 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/1256316458", 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 7, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 25, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-07-17T06:55:02.455 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T06:55:02.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:02.467 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-07-17T06:55:02.467 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:02.518 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-07-17T06:55:02.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:02.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:02.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:02.601 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-07-17T06:55:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:02.730 INFO:teuthology.orchestra.run.smithi044.stdout:# file: . 2024-07-17T06:55:02.730 INFO:teuthology.orchestra.run.smithi044.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-07-17T06:55:02.730 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:02.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:02.740 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:02.741 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:02.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:03.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 -- 172.21.15.44:0/1333493253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 msgr2=0x7fd70415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 --2- 172.21.15.44:0/1333493253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd70415c140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009830 tx=0x7fd6f402a480 comp rx=0 tx=0).stop 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 -- 172.21.15.44:0/1333493253 shutdown_connections 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 --2- 172.21.15.44:0/1333493253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd70415c140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 -- 172.21.15.44:0/1333493253 >> 172.21.15.44:0/1333493253 conn(0x7fd7040cf0d0 msgr2=0x7fd7040cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 -- 172.21.15.44:0/1333493253 shutdown_connections 2024-07-17T06:55:03.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.016+0000 7fd708c00640 1 -- 172.21.15.44:0/1333493253 wait complete. 2024-07-17T06:55:03.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.018+0000 7fd708c00640 1 Processor -- start 2024-07-17T06:55:03.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.018+0000 7fd708c00640 1 -- start start 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd70406f380 0x7fd7041f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040cd0b0 0x7fd704073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd70406e730 con 0x7fd704151f20 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd704152510 con 0x7fd70406f380 2024-07-17T06:55:03.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.019+0000 7fd708c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040cb790 con 0x7fd7040cd0b0 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50488/0 (socket says 172.21.15.44:50488) 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 -- 172.21.15.44:0/3889419292 learned_addr learned my addr 172.21.15.44:0/3889419292 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703400640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd70406f380 0x7fd7041f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd702a00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040cd0b0 0x7fd704073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 -- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040cd0b0 msgr2=0x7fd704073e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040cd0b0 0x7fd704073e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 -- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd70406f380 msgr2=0x7fd7041f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd70406f380 0x7fd7041f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.020+0000 7fd703e00640 1 -- 172.21.15.44:0/3889419292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd704072d90 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd703e00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cbc0 tx=0x7fd6fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc016020 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6fc00ae50 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc012620 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd704073070 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.021+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd704073630 con 0x7fd704151f20 2024-07-17T06:55:03.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.022+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70415c140 con 0x7fd704151f20 2024-07-17T06:55:03.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.024+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fd6fc019020 con 0x7fd704151f20 2024-07-17T06:55:03.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.024+0000 7fd6fb400640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 0x7fd6e4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.025+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6fc098d00 con 0x7fd704151f20 2024-07-17T06:55:03.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.025+0000 7fd703400640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 0x7fd6e4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.026+0000 7fd703400640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 0x7fd6e4079070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4034000 tx=0x7fd6f4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:03.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.029+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd6fc020020 con 0x7fd704151f20 2024-07-17T06:55:03.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.255+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7040cff20 con 0x7fd704151f20 2024-07-17T06:55:03.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.256+0000 7fd6fb400640 1 -- 172.21.15.44:0/3889419292 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fd70415c680 con 0x7fd704151f20 2024-07-17T06:55:03.254 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 msgr2=0x7fd6e4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 0x7fd6e4079070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4034000 tx=0x7fd6f4004730 comp rx=0 tx=0).stop 2024-07-17T06:55:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 msgr2=0x7fd704150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00cbc0 tx=0x7fd6fc00a430 comp rx=0 tx=0).stop 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 shutdown_connections 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd704151f20 0x7fd704150830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd6e4076bb0 0x7fd6e4079070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040cd0b0 0x7fd704073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.258+0000 7fd708c00640 1 --2- 172.21.15.44:0/3889419292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd70406f380 0x7fd7041f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.259+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 >> 172.21.15.44:0/3889419292 conn(0x7fd7040cf0d0 msgr2=0x7fd70406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.259+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 shutdown_connections 2024-07-17T06:55:03.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.259+0000 7fd708c00640 1 -- 172.21.15.44:0/3889419292 wait complete. 2024-07-17T06:55:03.279 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:03.279 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:03.335 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:03.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 -- 172.21.15.44:0/254693464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 msgr2=0x7fce1415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 --2- 172.21.15.44:0/254693464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce1415d510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fce140cf340 tx=0x7fce0c02a980 comp rx=0 tx=0).stop 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 -- 172.21.15.44:0/254693464 shutdown_connections 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 --2- 172.21.15.44:0/254693464 >> v2:172.21.15.175:3300/0 conn(0x7fce140ce690 0x7fce140cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 --2- 172.21.15.44:0/254693464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce1415d510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.548+0000 7fce1b200640 1 -- 172.21.15.44:0/254693464 >> 172.21.15.44:0/254693464 conn(0x7fce140b4ff0 msgr2=0x7fce140b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.549+0000 7fce1b200640 1 -- 172.21.15.44:0/254693464 shutdown_connections 2024-07-17T06:55:03.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.549+0000 7fce1b200640 1 -- 172.21.15.44:0/254693464 wait complete. 2024-07-17T06:55:03.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.550+0000 7fce1b200640 1 Processor -- start 2024-07-17T06:55:03.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.550+0000 7fce1b200640 1 -- start start 2024-07-17T06:55:03.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 0x7fce140cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 0x7fce140c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce140d19d0 con 0x7fce1415d130 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce140d1850 con 0x7fce14161be0 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.551+0000 7fce1b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce140d1b50 con 0x7fce140ce690 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50496/0 (socket says 172.21.15.44:50496) 2024-07-17T06:55:03.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 -- 172.21.15.44:0/844083836 learned_addr learned my addr 172.21.15.44:0/844083836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce1a800640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 0x7fce140c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19e00640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 0x7fce140cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 -- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 msgr2=0x7fce140cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 0x7fce140cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 -- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 msgr2=0x7fce140c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 0x7fce140c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 -- 172.21.15.44:0/844083836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1415e350 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce1a800640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 0x7fce140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19e00640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 0x7fce140cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.552+0000 7fce19400640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c440 tx=0x7fce0400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.553+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04017070 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.553+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce040035f0 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.553+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04003a40 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.553+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1415e630 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.553+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1415ebf0 con 0x7fce1415d130 2024-07-17T06:55:03.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.554+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce140cfdf0 con 0x7fce1415d130 2024-07-17T06:55:03.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.555+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fce04003be0 con 0x7fce1415d130 2024-07-17T06:55:03.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.556+0000 7fce0a000640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 0x7fcdf8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:03.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.556+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce04097a50 con 0x7fce1415d130 2024-07-17T06:55:03.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.557+0000 7fce19e00640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 0x7fcdf8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:03.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.558+0000 7fce19e00640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 0x7fcdf8079070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce0c035000 tx=0x7fce0c004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:03.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.561+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fce04060b40 con 0x7fce1415d130 2024-07-17T06:55:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.787+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce140b26b0 con 0x7fce1415d130 2024-07-17T06:55:03.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.788+0000 7fce0a000640 1 -- 172.21.15.44:0/844083836 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fce040010c0 con 0x7fce1415d130 2024-07-17T06:55:03.787 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 msgr2=0x7fcdf8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 0x7fcdf8079070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce0c035000 tx=0x7fce0c004ab0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 msgr2=0x7fce140c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c440 tx=0x7fce0400c900 comp rx=0 tx=0).stop 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 shutdown_connections 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce14161be0 0x7fce140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1415d130 0x7fce140c7f10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fcdf8076bb0 0x7fcdf8079070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 --2- 172.21.15.44:0/844083836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce140ce690 0x7fce140cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 >> 172.21.15.44:0/844083836 conn(0x7fce140b4ff0 msgr2=0x7fce140b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 shutdown_connections 2024-07-17T06:55:03.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:03.791+0000 7fce1b200640 1 -- 172.21.15.44:0/844083836 wait complete. 2024-07-17T06:55:03.810 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-07-17T06:55:03.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:03.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-07-17T06:55:03.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi044.stderr:ceph-fuse[44112]: fuse finished with error 0 and tester_r 0 2024-07-17T06:55:03.899 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T06:55:03.950 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:55:03.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:03.950 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-07-17T06:55:04.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:04.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 -- 172.21.15.44:0/2466790316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf920 msgr2=0x7f01b40ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 --2- 172.21.15.44:0/2466790316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf920 0x7f01b40ade20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f01b406f770 tx=0x7f01ac02aa10 comp rx=0 tx=0).stop 2024-07-17T06:55:04.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 -- 172.21.15.44:0/2466790316 shutdown_connections 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 --2- 172.21.15.44:0/2466790316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf920 0x7f01b40ade20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 --2- 172.21.15.44:0/2466790316 >> v2:172.21.15.175:3300/0 conn(0x7f01b40b3db0 0x7f01b40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 --2- 172.21.15.44:0/2466790316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40cf540 0x7f01b40b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 -- 172.21.15.44:0/2466790316 >> 172.21.15.44:0/2466790316 conn(0x7f01b40b0e80 msgr2=0x7f01b40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 -- 172.21.15.44:0/2466790316 shutdown_connections 2024-07-17T06:55:04.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.224+0000 7f01ba400640 1 -- 172.21.15.44:0/2466790316 wait complete. 2024-07-17T06:55:04.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.227+0000 7f01ba400640 1 Processor -- start 2024-07-17T06:55:04.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.227+0000 7f01ba400640 1 -- start start 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40b3db0 0x7f01b413b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b40cf920 0x7f01b41f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40ae6f0 con 0x7f01b40cf540 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40ae570 con 0x7f01b40cf920 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01ba400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40ae870 con 0x7f01b40b3db0 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.228+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50506/0 (socket says 172.21.15.44:50506) 2024-07-17T06:55:04.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 -- 172.21.15.44:0/4193422695 learned_addr learned my addr 172.21.15.44:0/4193422695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b9000640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40b3db0 0x7f01b413b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b9a00640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b40cf920 0x7f01b41f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 -- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40b3db0 msgr2=0x7f01b413b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40b3db0 0x7f01b413b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 -- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b40cf920 msgr2=0x7f01b41f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b40cf920 0x7f01b41f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 -- 172.21.15.44:0/4193422695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41592b0 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.229+0000 7f01b3e00640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c9a0 tx=0x7f01a400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.230+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4007bb0 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.230+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a4007e40 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.230+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4005260 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.230+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b4159590 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.230+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b4159b50 con 0x7f01b40cf540 2024-07-17T06:55:04.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.231+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b40b3870 con 0x7f01b40cf540 2024-07-17T06:55:04.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.232+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f01a4005400 con 0x7f01b40cf540 2024-07-17T06:55:04.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.233+0000 7f01b0c00640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 0x7f01980790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.233+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01a4098a60 con 0x7f01b40cf540 2024-07-17T06:55:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.234+0000 7f01b9000640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 0x7f01980790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.235+0000 7f01b9000640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 0x7f01980790a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01a800cac0 tx=0x7f01a800c4a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:04.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.238+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01a4014030 con 0x7f01b40cf540 2024-07-17T06:55:04.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.458+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01b413f1f0 con 0x7f01b40cf540 2024-07-17T06:55:04.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.461+0000 7f01b0c00640 1 -- 172.21.15.44:0/4193422695 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f01a4061be0 con 0x7f01b40cf540 2024-07-17T06:55:04.461 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:04.461 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:04.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.464+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 msgr2=0x7f01980790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.464+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 0x7f01980790a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01a800cac0 tx=0x7f01a800c4a0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.464+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 msgr2=0x7f01b415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.464+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c9a0 tx=0x7f01a400ce60 comp rx=0 tx=0).stop 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 shutdown_connections 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b40cf920 0x7f01b41f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40cf540 0x7f01b415a350 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0198076be0 0x7f01980790a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 --2- 172.21.15.44:0/4193422695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40b3db0 0x7f01b413b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 >> 172.21.15.44:0/4193422695 conn(0x7f01b40b0e80 msgr2=0x7f01b40afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 shutdown_connections 2024-07-17T06:55:04.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.465+0000 7f01ba400640 1 -- 172.21.15.44:0/4193422695 wait complete. 2024-07-17T06:55:04.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 -- 172.21.15.44:0/3025601275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2340003af0 msgr2=0x7f2340003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 --2- 172.21.15.44:0/3025601275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2340003af0 0x7f2340003f10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f234c009ed0 tx=0x7f2340005830 comp rx=0 tx=0).stop 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235e000640 1 -- 172.21.15.44:0/3025601275 reap_dead start 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 -- 172.21.15.44:0/3025601275 shutdown_connections 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 -- 172.21.15.44:0/3025601275 >> 172.21.15.44:0/3025601275 conn(0x7f23580b0e80 msgr2=0x7f23580b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:04.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 -- 172.21.15.44:0/3025601275 shutdown_connections 2024-07-17T06:55:04.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.711+0000 7f235ea00640 1 -- 172.21.15.44:0/3025601275 wait complete. 2024-07-17T06:55:04.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.713+0000 7f235ea00640 1 Processor -- start 2024-07-17T06:55:04.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.713+0000 7f235ea00640 1 -- start start 2024-07-17T06:55:04.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 0x7f235809c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 0x7f23580713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23580adff0 con 0x7f2358155600 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23580b3b00 con 0x7f2340003af0 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23580ae170 con 0x7f235813d170 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50520/0 (socket says 172.21.15.44:50520) 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 -- 172.21.15.44:0/304535236 learned_addr learned my addr 172.21.15.44:0/304535236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235d600640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 0x7f235809c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235e000640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 0x7f23580713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 -- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 msgr2=0x7f23580713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 0x7f23580713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 -- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 msgr2=0x7f235809c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 0x7f235809c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.714+0000 7f235cc00640 1 -- 172.21.15.44:0/304535236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358155a80 con 0x7f2358155600 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f235d600640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 0x7f235809c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f235e000640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 0x7f23580713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f235cc00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2344004cf0 tx=0x7f23440015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344011660 con 0x7f2358155600 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2344016070 con 0x7f2358155600 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344011b50 con 0x7f2358155600 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2358156c50 con 0x7f2358155600 2024-07-17T06:55:04.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.715+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581563b0 con 0x7f2358155600 2024-07-17T06:55:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.716+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23580ae700 con 0x7f2358155600 2024-07-17T06:55:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.718+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f234401b460 con 0x7f2358155600 2024-07-17T06:55:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.718+0000 7f2355600640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 0x7f23380790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:04.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.719+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2344098f20 con 0x7f2358155600 2024-07-17T06:55:04.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.719+0000 7f235d600640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 0x7f23380790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:04.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.720+0000 7f235d600640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 0x7f23380790a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f234c035000 tx=0x7f234c02d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:04.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.723+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2344061a50 con 0x7f2358155600 2024-07-17T06:55:04.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.935+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f235813d550 con 0x7f2358155600 2024-07-17T06:55:04.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.937+0000 7f2355600640 1 -- 172.21.15.44:0/304535236 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f2344065fa0 con 0x7f2358155600 2024-07-17T06:55:04.937 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:04.937 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:04.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 msgr2=0x7f23380790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 0x7f23380790a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f234c035000 tx=0x7f234c02d040 comp rx=0 tx=0).stop 2024-07-17T06:55:04.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 msgr2=0x7f235813cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:04.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2344004cf0 tx=0x7f23440015d0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 shutdown_connections 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2338076be0 0x7f23380790a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f235813d170 0x7f23580713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358155600 0x7f235813cc30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 --2- 172.21.15.44:0/304535236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2340003af0 0x7f235809c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 >> 172.21.15.44:0/304535236 conn(0x7f23580b0e80 msgr2=0x7f2358140f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 shutdown_connections 2024-07-17T06:55:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:04.940+0000 7f235ea00640 1 -- 172.21.15.44:0/304535236 wait complete. 2024-07-17T06:55:04.958 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.181+0000 7f981c800640 1 -- 172.21.15.44:0/966737255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 msgr2=0x7f97fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.181+0000 7f981c800640 1 --2- 172.21.15.44:0/966737255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f97fc003e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9808009b50 tx=0x7f97fc005830 comp rx=0 tx=0).stop 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.182+0000 7f981a800640 1 -- 172.21.15.44:0/966737255 reap_dead start 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.182+0000 7f981c800640 1 -- 172.21.15.44:0/966737255 shutdown_connections 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.182+0000 7f981c800640 1 -- 172.21.15.44:0/966737255 >> 172.21.15.44:0/966737255 conn(0x7f98140b0eb0 msgr2=0x7f98140b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.182+0000 7f981c800640 1 -- 172.21.15.44:0/966737255 shutdown_connections 2024-07-17T06:55:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.182+0000 7f981c800640 1 -- 172.21.15.44:0/966737255 wait complete. 2024-07-17T06:55:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.184+0000 7f981c800640 1 Processor -- start 2024-07-17T06:55:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.184+0000 7f981c800640 1 -- start start 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 0x7f9814144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9814073770 0x7f9814073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98140b3e60 con 0x7f97fc003a00 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98140b3ce0 con 0x7f9814144180 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981c800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98140b3fe0 con 0x7f9814073770 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50554/0 (socket says 172.21.15.44:50554) 2024-07-17T06:55:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 -- 172.21.15.44:0/691707494 learned_addr learned my addr 172.21.15.44:0/691707494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:05.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 -- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9814073770 msgr2=0x7f9814073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f981a800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9814073770 0x7f9814073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819400640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 0x7f9814144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9814073770 0x7f9814073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 -- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 msgr2=0x7f9814144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 0x7f9814144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819e00640 1 -- 172.21.15.44:0/691707494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98140726d0 con 0x7f97fc003a00 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.185+0000 7f9819400640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 0x7f9814144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f9819e00640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f980802b4f0 tx=0x7f9808033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808043070 con 0x7f97fc003a00 2024-07-17T06:55:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9808004320 con 0x7f97fc003a00 2024-07-17T06:55:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808038040 con 0x7f97fc003a00 2024-07-17T06:55:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98140729b0 con 0x7f97fc003a00 2024-07-17T06:55:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.186+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9814072f70 con 0x7f97fc003a00 2024-07-17T06:55:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.188+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98140b0840 con 0x7f97fc003a00 2024-07-17T06:55:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.189+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9808046070 con 0x7f97fc003a00 2024-07-17T06:55:05.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.189+0000 7f980e000640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 0x7f97f8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.190+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f98080b8ad0 con 0x7f97fc003a00 2024-07-17T06:55:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.190+0000 7f9819400640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 0x7f97f8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:05.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.191+0000 7f9819400640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 0x7f97f8079070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9804009460 tx=0x7f9804009390 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:05.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.194+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98140b0840 con 0x7f97fc003a00 2024-07-17T06:55:05.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:05.442+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f981406d620 con 0x7f97fc003a00 2024-07-17T06:55:06.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.401+0000 7f980e000640 1 -- 172.21.15.44:0/691707494 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f9808081610 con 0x7f97fc003a00 2024-07-17T06:55:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 msgr2=0x7f97f8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 0x7f97f8079070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9804009460 tx=0x7f9804009390 comp rx=0 tx=0).stop 2024-07-17T06:55:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 msgr2=0x7f98141f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f980802b4f0 tx=0x7f9808033670 comp rx=0 tx=0).stop 2024-07-17T06:55:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 shutdown_connections 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f97f8076bb0 0x7f97f8079070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9814073770 0x7f9814073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9814144180 0x7f9814144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 --2- 172.21.15.44:0/691707494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc003a00 0x7f98141f4ea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 >> 172.21.15.44:0/691707494 conn(0x7f98140b0eb0 msgr2=0x7f98140b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 shutdown_connections 2024-07-17T06:55:06.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.406+0000 7f981c800640 1 -- 172.21.15.44:0/691707494 wait complete. 2024-07-17T06:55:06.429 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-07-17T06:55:06.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T06:55:06.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 -- 172.21.15.44:0/3444903244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 msgr2=0x7fbdf406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3444903244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf406d620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005fd0 tx=0x7fbdec02a940 comp rx=0 tx=0).stop 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 -- 172.21.15.44:0/3444903244 shutdown_connections 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3444903244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf406d620 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 -- 172.21.15.44:0/3444903244 >> 172.21.15.44:0/3444903244 conn(0x7fbdf40b0eb0 msgr2=0x7fbdf40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 -- 172.21.15.44:0/3444903244 shutdown_connections 2024-07-17T06:55:06.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.645+0000 7fbdfa000640 1 -- 172.21.15.44:0/3444903244 wait complete. 2024-07-17T06:55:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.647+0000 7fbdfa000640 1 Processor -- start 2024-07-17T06:55:06.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.647+0000 7fbdfa000640 1 -- start start 2024-07-17T06:55:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf40cbab0 0x7fbdf41f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf4152220 0x7fbdf4147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf40cd9f0 con 0x7fbdf406ed30 2024-07-17T06:55:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf415c7d0 con 0x7fbdf4152220 2024-07-17T06:55:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.648+0000 7fbdfa000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf40cdb70 con 0x7fbdf40cbab0 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50560/0 (socket says 172.21.15.44:50560) 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 -- 172.21.15.44:0/3368312194 learned_addr learned my addr 172.21.15.44:0/3368312194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3e00640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf4152220 0x7fbdf4147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf2a00640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf40cbab0 0x7fbdf41f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 -- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf40cbab0 msgr2=0x7fbdf41f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf40cbab0 0x7fbdf41f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 -- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf4152220 msgr2=0x7fbdf4147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf4152220 0x7fbdf4147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 -- 172.21.15.44:0/3368312194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf414ca40 con 0x7fbdf406ed30 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.649+0000 7fbdf3400640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbde400c3d0 tx=0x7fbde400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.650+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4017070 con 0x7fbdf406ed30 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.650+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbde40053c0 con 0x7fbdf406ed30 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.650+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4003850 con 0x7fbdf406ed30 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.650+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf414cd20 con 0x7fbdf406ed30 2024-07-17T06:55:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.650+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf40ac6c0 con 0x7fbdf406ed30 2024-07-17T06:55:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.651+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf40ce040 con 0x7fbdf406ed30 2024-07-17T06:55:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.652+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fbde401a070 con 0x7fbdf406ed30 2024-07-17T06:55:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.653+0000 7fbdeb400640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 0x7fbdd0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.654+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbde4097a60 con 0x7fbdf406ed30 2024-07-17T06:55:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.654+0000 7fbdf2a00640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 0x7fbdd0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.655+0000 7fbdf2a00640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 0x7fbdd0079070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40724f0 tx=0x7fbddc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.657+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbde4060be0 con 0x7fbdf406ed30 2024-07-17T06:55:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:06.913+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbdf415d830 con 0x7fbdf406ed30 2024-07-17T06:55:07.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.405+0000 7fbdeb400640 1 -- 172.21.15.44:0/3368312194 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fbde40604e0 con 0x7fbdf406ed30 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 msgr2=0x7fbdd0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 0x7fbdd0079070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40724f0 tx=0x7fbddc009040 comp rx=0 tx=0).stop 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 msgr2=0x7fbdf414c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbde400c3d0 tx=0x7fbde400c890 comp rx=0 tx=0).stop 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 shutdown_connections 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf4152220 0x7fbdf4147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbdd0076bb0 0x7fbdd0079070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf40cbab0 0x7fbdf41f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 --2- 172.21.15.44:0/3368312194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf406ed30 0x7fbdf414c3f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 >> 172.21.15.44:0/3368312194 conn(0x7fbdf40b0eb0 msgr2=0x7fbdf415e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 shutdown_connections 2024-07-17T06:55:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.409+0000 7fbdfa000640 1 -- 172.21.15.44:0/3368312194 wait complete. 2024-07-17T06:55:07.432 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T06:55:07.432 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa423200640 1 -- 172.21.15.44:0/3102381375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa408003ac0 msgr2=0x7fa408003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa423200640 1 --2- 172.21.15.44:0/3102381375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa408003ac0 0x7fa408003ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa414009b50 tx=0x7fa408005830 comp rx=0 tx=0).stop 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa422800640 1 -- 172.21.15.44:0/3102381375 reap_dead start 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa423200640 1 -- 172.21.15.44:0/3102381375 shutdown_connections 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa423200640 1 -- 172.21.15.44:0/3102381375 >> 172.21.15.44:0/3102381375 conn(0x7fa4240b0e80 msgr2=0x7fa4240b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.644+0000 7fa423200640 1 -- 172.21.15.44:0/3102381375 shutdown_connections 2024-07-17T06:55:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.645+0000 7fa423200640 1 -- 172.21.15.44:0/3102381375 wait complete. 2024-07-17T06:55:07.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.646+0000 7fa423200640 1 Processor -- start 2024-07-17T06:55:07.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.646+0000 7fa423200640 1 -- start start 2024-07-17T06:55:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa408003ac0 0x7fa4241f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 0x7fa4241f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa42409b690 con 0x7fa4241427e0 2024-07-17T06:55:07.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa42409b510 con 0x7fa4241f4ef0 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.647+0000 7fa423200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa42409b810 con 0x7fa408003ac0 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50578/0 (socket says 172.21.15.44:50578) 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 -- 172.21.15.44:0/203292341 learned_addr learned my addr 172.21.15.44:0/203292341 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa422800640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 0x7fa4241f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa421e00640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa408003ac0 0x7fa4241f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 -- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa408003ac0 msgr2=0x7fa4241f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa408003ac0 0x7fa4241f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 -- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 msgr2=0x7fa4241f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 0x7fa4241f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 -- 172.21.15.44:0/203292341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4241f21a0 con 0x7fa4241427e0 2024-07-17T06:55:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa422800640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 0x7fa4241f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.648+0000 7fa419400640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00ead0 tx=0x7fa40c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.649+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c01c070 con 0x7fa4241427e0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.649+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa40c00ddb0 con 0x7fa4241427e0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.649+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c017650 con 0x7fa4241427e0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.649+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4241f2480 con 0x7fa4241427e0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.649+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa424072410 con 0x7fa4241427e0 2024-07-17T06:55:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.650+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42406e060 con 0x7fa4241427e0 2024-07-17T06:55:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.652+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fa40c01f070 con 0x7fa4241427e0 2024-07-17T06:55:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.653+0000 7fa41aa00640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 0x7fa400079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.653+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa40c098b10 con 0x7fa4241427e0 2024-07-17T06:55:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.654+0000 7fa421e00640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 0x7fa400079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.655+0000 7fa421e00640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 0x7fa400079070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa414035000 tx=0x7fa41402d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:07.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.658+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa40c0615b0 con 0x7fa4241427e0 2024-07-17T06:55:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.901+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa424153a80 con 0x7fa4241427e0 2024-07-17T06:55:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.902+0000 7fa41aa00640 1 -- 172.21.15.44:0/203292341 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7fa40c065b00 con 0x7fa4241427e0 2024-07-17T06:55:07.902 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:07.902 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.905+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 msgr2=0x7fa400079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.905+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 0x7fa400079070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa414035000 tx=0x7fa41402d040 comp rx=0 tx=0).stop 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.905+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 msgr2=0x7fa42413f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.905+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00ead0 tx=0x7fa40c00ef90 comp rx=0 tx=0).stop 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 shutdown_connections 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4241f4ef0 0x7fa4241f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4241427e0 0x7fa42413f1c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa400076bb0 0x7fa400079070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 --2- 172.21.15.44:0/203292341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa408003ac0 0x7fa4241f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 >> 172.21.15.44:0/203292341 conn(0x7fa4240b0e80 msgr2=0x7fa4240b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 shutdown_connections 2024-07-17T06:55:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:07.906+0000 7fa423200640 1 -- 172.21.15.44:0/203292341 wait complete. 2024-07-17T06:55:07.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 -- 172.21.15.44:0/3599342928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 msgr2=0x7ff04800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 --2- 172.21.15.44:0/3599342928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff04800b9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c7a0 tx=0x7ff048005830 comp rx=0 tx=0).stop 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 -- 172.21.15.44:0/3599342928 shutdown_connections 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 --2- 172.21.15.44:0/3599342928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff04800b9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 -- 172.21.15.44:0/3599342928 >> 172.21.15.44:0/3599342928 conn(0x7ff0640b0e80 msgr2=0x7ff0640b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 -- 172.21.15.44:0/3599342928 shutdown_connections 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.134+0000 7ff063200640 1 -- 172.21.15.44:0/3599342928 wait complete. 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.136+0000 7ff063200640 1 Processor -- start 2024-07-17T06:55:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.136+0000 7ff063200640 1 -- start start 2024-07-17T06:55:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 0x7ff064152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0641f0fe0 0x7ff0641f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640cf660 con 0x7ff0480052c0 2024-07-17T06:55:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640cf4e0 con 0x7ff0641f0fe0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff063200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640b1fa0 con 0x7ff064151da0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff061e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff061e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50594/0 (socket says 172.21.15.44:50594) 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff061e00640 1 -- 172.21.15.44:0/3152021636 learned_addr learned my addr 172.21.15.44:0/3152021636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff062800640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0641f0fe0 0x7ff0641f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.137+0000 7ff061400640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 0x7ff064152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 -- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 msgr2=0x7ff064152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 0x7ff064152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 -- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0641f0fe0 msgr2=0x7ff0641f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0641f0fe0 0x7ff0641f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 -- 172.21.15.44:0/3152021636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0641f1990 con 0x7ff0480052c0 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061400640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 0x7ff064152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff061e00640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff054004e20 tx=0x7ff054001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0540107d0 con 0x7ff0480052c0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff054016070 con 0x7ff0480052c0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054010cc0 con 0x7ff0480052c0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff064150d00 con 0x7ff0480052c0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.138+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0641512c0 con 0x7ff0480052c0 2024-07-17T06:55:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.139+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064160de0 con 0x7ff0480052c0 2024-07-17T06:55:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.140+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ff05401b460 con 0x7ff0480052c0 2024-07-17T06:55:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.141+0000 7ff05a000640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 0x7ff044079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.142+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff054098e80 con 0x7ff0480052c0 2024-07-17T06:55:08.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.142+0000 7ff061400640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 0x7ff044079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.143+0000 7ff061400640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 0x7ff044079070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff064152940 tx=0x7ff050005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.146+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff054061f70 con 0x7ff0480052c0 2024-07-17T06:55:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.348+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0640b0240 con 0x7ff0480052c0 2024-07-17T06:55:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.351+0000 7ff05a000640 1 -- 172.21.15.44:0/3152021636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7ff054061870 con 0x7ff0480052c0 2024-07-17T06:55:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:08.351 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:08.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 msgr2=0x7ff044079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 0x7ff044079070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff064152940 tx=0x7ff050005f90 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 msgr2=0x7ff0641f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff054004e20 tx=0x7ff054001630 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 shutdown_connections 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0641f0fe0 0x7ff0641f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff044076bb0 0x7ff044079070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff064151da0 0x7ff064152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 --2- 172.21.15.44:0/3152021636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480052c0 0x7ff0641f4aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.353+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 >> 172.21.15.44:0/3152021636 conn(0x7ff0640b0e80 msgr2=0x7ff064158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.354+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 shutdown_connections 2024-07-17T06:55:08.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.354+0000 7ff063200640 1 -- 172.21.15.44:0/3152021636 wait complete. 2024-07-17T06:55:08.375 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T06:55:08.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.590+0000 7fca81e00640 1 -- 172.21.15.44:0/2767989174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c1522c0 msgr2=0x7fca7c0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.590+0000 7fca81e00640 1 --2- 172.21.15.44:0/2767989174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c1522c0 0x7fca7c0ad790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca7c15edd0 tx=0x7fca6c02aa10 comp rx=0 tx=0).stop 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 -- 172.21.15.44:0/2767989174 shutdown_connections 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 --2- 172.21.15.44:0/2767989174 >> v2:172.21.15.175:3300/0 conn(0x7fca7c0ac930 0x7fca7c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 --2- 172.21.15.44:0/2767989174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c1522c0 0x7fca7c0ad790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 -- 172.21.15.44:0/2767989174 >> 172.21.15.44:0/2767989174 conn(0x7fca7c0b0fb0 msgr2=0x7fca7c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:08.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 -- 172.21.15.44:0/2767989174 shutdown_connections 2024-07-17T06:55:08.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.591+0000 7fca81e00640 1 -- 172.21.15.44:0/2767989174 wait complete. 2024-07-17T06:55:08.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.593+0000 7fca81e00640 1 Processor -- start 2024-07-17T06:55:08.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.593+0000 7fca81e00640 1 -- start start 2024-07-17T06:55:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 0x7fca7c072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca7c1522c0 0x7fca7c06f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c0cdf60 con 0x7fca7c0adcd0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c0cdde0 con 0x7fca7c0ac930 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.594+0000 7fca81e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca7c0ce0e0 con 0x7fca7c1522c0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50612/0 (socket says 172.21.15.44:50612) 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 -- 172.21.15.44:0/255554760 learned_addr learned my addr 172.21.15.44:0/255554760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca7aa00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 0x7fca7c072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca7b400640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca7c1522c0 0x7fca7c06f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 -- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca7c1522c0 msgr2=0x7fca7c06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca7c1522c0 0x7fca7c06f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 -- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 msgr2=0x7fca7c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 0x7fca7c072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca72000640 1 -- 172.21.15.44:0/255554760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c1f3db0 con 0x7fca7c0adcd0 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.595+0000 7fca7aa00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 0x7fca7c072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:08.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca72000640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c430 tx=0x7fca6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64011790 con 0x7fca7c0adcd0 2024-07-17T06:55:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca64017070 con 0x7fca7c0adcd0 2024-07-17T06:55:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64011cc0 con 0x7fca7c0adcd0 2024-07-17T06:55:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c1f7bb0 con 0x7fca7c0adcd0 2024-07-17T06:55:08.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.596+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c1f8170 con 0x7fca7c0adcd0 2024-07-17T06:55:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.597+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7c0cd690 con 0x7fca7c0adcd0 2024-07-17T06:55:08.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.598+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fca6401c460 con 0x7fca7c0adcd0 2024-07-17T06:55:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.599+0000 7fca73e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 0x7fca58079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:08.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.599+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fca64098590 con 0x7fca7c0adcd0 2024-07-17T06:55:08.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.600+0000 7fca7aa00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 0x7fca58079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:08.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.601+0000 7fca7aa00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 0x7fca58079070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca6c035000 tx=0x7fca6c004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:08.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.604+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fca64061710 con 0x7fca7c0adcd0 2024-07-17T06:55:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.815+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca7c15d050 con 0x7fca58076bb0 2024-07-17T06:55:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.837+0000 7fca73e00640 1 -- 172.21.15.44:0/255554760 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332559 (secure 0 0 0) 0x7fca7c15d050 con 0x7fca58076bb0 2024-07-17T06:55:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:55:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 msgr2=0x7fca58079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 0x7fca58079070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca6c035000 tx=0x7fca6c004ab0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 msgr2=0x7fca7c1f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c430 tx=0x7fca6400c8f0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 shutdown_connections 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fca58076bb0 0x7fca58079070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca7c1522c0 0x7fca7c06f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca7c0adcd0 0x7fca7c1f73a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 --2- 172.21.15.44:0/255554760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca7c0ac930 0x7fca7c072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 >> 172.21.15.44:0/255554760 conn(0x7fca7c0b0fb0 msgr2=0x7fca7c0acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 shutdown_connections 2024-07-17T06:55:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:08.846+0000 7fca81e00640 1 -- 172.21.15.44:0/255554760 wait complete. 2024-07-17T06:55:08.866 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-17T06:55:07.716449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219448,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755272,"statfs":{"total":773094113280,"available":772869398528,"internally_reserved":0,"allocated":3317760,"data_stored":2001358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005148"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569050+0000","last_change":"2024-07-17T06:54:48.289026+0000","last_active":"2024-07-17T06:54:54.569050+0000","last_peered":"2024-07-17T06:54:54.569050+0000","last_clean":"2024-07-17T06:54:54.569050+0000","last_became_active":"2024-07-17T06:54:48.288767+0000","last_became_peered":"2024-07-17T06:54:48.288767+0000","last_unstale":"2024-07-17T06:54:54.569050+0000","last_undegraded":"2024-07-17T06:54:54.569050+0000","last_fullsized":"2024-07-17T06:54:54.569050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572074+0000","last_change":"2024-07-17T06:54:49.320600+0000","last_active":"2024-07-17T06:54:54.572074+0000","last_peered":"2024-07-17T06:54:54.572074+0000","last_clean":"2024-07-17T06:54:54.572074+0000","last_became_active":"2024-07-17T06:54:49.319878+0000","last_became_peered":"2024-07-17T06:54:49.319878+0000","last_unstale":"2024-07-17T06:54:54.572074+0000","last_undegraded":"2024-07-17T06:54:54.572074+0000","last_fullsized":"2024-07-17T06:54:54.572074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569764+0000","last_change":"2024-07-17T06:54:53.670763+0000","last_active":"2024-07-17T06:54:54.569764+0000","last_peered":"2024-07-17T06:54:54.569764+0000","last_clean":"2024-07-17T06:54:54.569764+0000","last_became_active":"2024-07-17T06:54:53.670345+0000","last_became_peered":"2024-07-17T06:54:53.670345+0000","last_unstale":"2024-07-17T06:54:54.569764+0000","last_undegraded":"2024-07-17T06:54:54.569764+0000","last_fullsized":"2024-07-17T06:54:54.569764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:19.165767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572200+0000","last_change":"2024-07-17T06:54:48.267830+0000","last_active":"2024-07-17T06:54:54.572200+0000","last_peered":"2024-07-17T06:54:54.572200+0000","last_clean":"2024-07-17T06:54:54.572200+0000","last_became_active":"2024-07-17T06:54:48.267458+0000","last_became_peered":"2024-07-17T06:54:48.267458+0000","last_unstale":"2024-07-17T06:54:54.572200+0000","last_undegraded":"2024-07-17T06:54:54.572200+0000","last_fullsized":"2024-07-17T06:54:54.572200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566732+0000","last_change":"2024-07-17T06:54:49.313799+0000","last_active":"2024-07-17T06:54:54.566732+0000","last_peered":"2024-07-17T06:54:54.566732+0000","last_clean":"2024-07-17T06:54:54.566732+0000","last_became_active":"2024-07-17T06:54:49.313240+0000","last_became_peered":"2024-07-17T06:54:49.313240+0000","last_unstale":"2024-07-17T06:54:54.566732+0000","last_undegraded":"2024-07-17T06:54:54.566732+0000","last_fullsized":"2024-07-17T06:54:54.566732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340034+0000","last_change":"2024-07-17T06:54:53.356992+0000","last_active":"2024-07-17T06:54:54.340034+0000","last_peered":"2024-07-17T06:54:54.340034+0000","last_clean":"2024-07-17T06:54:54.340034+0000","last_became_active":"2024-07-17T06:54:53.356521+0000","last_became_peered":"2024-07-17T06:54:53.356521+0000","last_unstale":"2024-07-17T06:54:54.340034+0000","last_undegraded":"2024-07-17T06:54:54.340034+0000","last_fullsized":"2024-07-17T06:54:54.340034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:33.058963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569075+0000","last_change":"2024-07-17T06:54:48.285421+0000","last_active":"2024-07-17T06:54:54.569075+0000","last_peered":"2024-07-17T06:54:54.569075+0000","last_clean":"2024-07-17T06:54:54.569075+0000","last_became_active":"2024-07-17T06:54:48.285132+0000","last_became_peered":"2024-07-17T06:54:48.285132+0000","last_unstale":"2024-07-17T06:54:54.569075+0000","last_undegraded":"2024-07-17T06:54:54.569075+0000","last_fullsized":"2024-07-17T06:54:54.569075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:38.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566825+0000","last_change":"2024-07-17T06:54:49.311977+0000","last_active":"2024-07-17T06:54:54.566825+0000","last_peered":"2024-07-17T06:54:54.566825+0000","last_clean":"2024-07-17T06:54:54.566825+0000","last_became_active":"2024-07-17T06:54:49.311364+0000","last_became_peered":"2024-07-17T06:54:49.311364+0000","last_unstale":"2024-07-17T06:54:54.566825+0000","last_undegraded":"2024-07-17T06:54:54.566825+0000","last_fullsized":"2024-07-17T06:54:54.566825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:21.034101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341182+0000","last_change":"2024-07-17T06:54:53.362264+0000","last_active":"2024-07-17T06:54:54.341182+0000","last_peered":"2024-07-17T06:54:54.341182+0000","last_clean":"2024-07-17T06:54:54.341182+0000","last_became_active":"2024-07-17T06:54:53.361863+0000","last_became_peered":"2024-07-17T06:54:53.361863+0000","last_unstale":"2024-07-17T06:54:54.341182+0000","last_undegraded":"2024-07-17T06:54:54.341182+0000","last_fullsized":"2024-07-17T06:54:54.341182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:22.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567855+0000","last_change":"2024-07-17T06:54:48.288847+0000","last_active":"2024-07-17T06:54:54.567855+0000","last_peered":"2024-07-17T06:54:54.567855+0000","last_clean":"2024-07-17T06:54:54.567855+0000","last_became_active":"2024-07-17T06:54:48.288504+0000","last_became_peered":"2024-07-17T06:54:48.288504+0000","last_unstale":"2024-07-17T06:54:54.567855+0000","last_undegraded":"2024-07-17T06:54:54.567855+0000","last_fullsized":"2024-07-17T06:54:54.567855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:42.640200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567324+0000","last_change":"2024-07-17T06:54:49.311645+0000","last_active":"2024-07-17T06:54:54.567324+0000","last_peered":"2024-07-17T06:54:54.567324+0000","last_clean":"2024-07-17T06:54:54.567324+0000","last_became_active":"2024-07-17T06:54:49.311336+0000","last_became_peered":"2024-07-17T06:54:49.311336+0000","last_unstale":"2024-07-17T06:54:54.567324+0000","last_undegraded":"2024-07-17T06:54:54.567324+0000","last_fullsized":"2024-07-17T06:54:54.567324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342111+0000","last_change":"2024-07-17T06:54:53.357717+0000","last_active":"2024-07-17T06:54:54.342111+0000","last_peered":"2024-07-17T06:54:54.342111+0000","last_clean":"2024-07-17T06:54:54.342111+0000","last_became_active":"2024-07-17T06:54:53.356799+0000","last_became_peered":"2024-07-17T06:54:53.356799+0000","last_unstale":"2024-07-17T06:54:54.342111+0000","last_undegraded":"2024-07-17T06:54:54.342111+0000","last_fullsized":"2024-07-17T06:54:54.342111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:29.584528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341883+0000","last_change":"2024-07-17T06:54:49.299990+0000","last_active":"2024-07-17T06:54:54.341883+0000","last_peered":"2024-07-17T06:54:54.341883+0000","last_clean":"2024-07-17T06:54:54.341883+0000","last_became_active":"2024-07-17T06:54:49.299679+0000","last_became_peered":"2024-07-17T06:54:49.299679+0000","last_unstale":"2024-07-17T06:54:54.341883+0000","last_undegraded":"2024-07-17T06:54:54.341883+0000","last_fullsized":"2024-07-17T06:54:54.341883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568387+0000","last_change":"2024-07-17T06:54:48.285371+0000","last_active":"2024-07-17T06:54:54.568387+0000","last_peered":"2024-07-17T06:54:54.568387+0000","last_clean":"2024-07-17T06:54:54.568387+0000","last_became_active":"2024-07-17T06:54:48.284925+0000","last_became_peered":"2024-07-17T06:54:48.284925+0000","last_unstale":"2024-07-17T06:54:54.568387+0000","last_undegraded":"2024-07-17T06:54:54.568387+0000","last_fullsized":"2024-07-17T06:54:54.568387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:21.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344985+0000","last_change":"2024-07-17T06:54:53.353487+0000","last_active":"2024-07-17T06:54:54.344985+0000","last_peered":"2024-07-17T06:54:54.344985+0000","last_clean":"2024-07-17T06:54:54.344985+0000","last_became_active":"2024-07-17T06:54:53.353037+0000","last_became_peered":"2024-07-17T06:54:53.353037+0000","last_unstale":"2024-07-17T06:54:54.344985+0000","last_undegraded":"2024-07-17T06:54:54.344985+0000","last_fullsized":"2024-07-17T06:54:54.344985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:51.743135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570785+0000","last_change":"2024-07-17T06:54:49.300898+0000","last_active":"2024-07-17T06:54:54.570785+0000","last_peered":"2024-07-17T06:54:54.570785+0000","last_clean":"2024-07-17T06:54:54.570785+0000","last_became_active":"2024-07-17T06:54:49.300631+0000","last_became_peered":"2024-07-17T06:54:49.300631+0000","last_unstale":"2024-07-17T06:54:54.570785+0000","last_undegraded":"2024-07-17T06:54:54.570785+0000","last_fullsized":"2024-07-17T06:54:54.570785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568669+0000","last_change":"2024-07-17T06:54:48.274040+0000","last_active":"2024-07-17T06:54:54.568669+0000","last_peered":"2024-07-17T06:54:54.568669+0000","last_clean":"2024-07-17T06:54:54.568669+0000","last_became_active":"2024-07-17T06:54:48.272837+0000","last_became_peered":"2024-07-17T06:54:48.272837+0000","last_unstale":"2024-07-17T06:54:54.568669+0000","last_undegraded":"2024-07-17T06:54:54.568669+0000","last_fullsized":"2024-07-17T06:54:54.568669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344238+0000","last_change":"2024-07-17T06:54:53.356025+0000","last_active":"2024-07-17T06:54:54.344238+0000","last_peered":"2024-07-17T06:54:54.344238+0000","last_clean":"2024-07-17T06:54:54.344238+0000","last_became_active":"2024-07-17T06:54:53.355539+0000","last_became_peered":"2024-07-17T06:54:53.355539+0000","last_unstale":"2024-07-17T06:54:54.344238+0000","last_undegraded":"2024-07-17T06:54:54.344238+0000","last_fullsized":"2024-07-17T06:54:54.344238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:14.244752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567230+0000","last_change":"2024-07-17T06:54:49.303989+0000","last_active":"2024-07-17T06:54:54.567230+0000","last_peered":"2024-07-17T06:54:54.567230+0000","last_clean":"2024-07-17T06:54:54.567230+0000","last_became_active":"2024-07-17T06:54:49.303723+0000","last_became_peered":"2024-07-17T06:54:49.303723+0000","last_unstale":"2024-07-17T06:54:54.567230+0000","last_undegraded":"2024-07-17T06:54:54.567230+0000","last_fullsized":"2024-07-17T06:54:54.567230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568506+0000","last_change":"2024-07-17T06:54:48.280459+0000","last_active":"2024-07-17T06:54:54.568506+0000","last_peered":"2024-07-17T06:54:54.568506+0000","last_clean":"2024-07-17T06:54:54.568506+0000","last_became_active":"2024-07-17T06:54:48.280196+0000","last_became_peered":"2024-07-17T06:54:48.280196+0000","last_unstale":"2024-07-17T06:54:54.568506+0000","last_undegraded":"2024-07-17T06:54:54.568506+0000","last_fullsized":"2024-07-17T06:54:54.568506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571786+0000","last_change":"2024-07-17T06:54:53.672128+0000","last_active":"2024-07-17T06:54:54.571786+0000","last_peered":"2024-07-17T06:54:54.571786+0000","last_clean":"2024-07-17T06:54:54.571786+0000","last_became_active":"2024-07-17T06:54:53.671083+0000","last_became_peered":"2024-07-17T06:54:53.671083+0000","last_unstale":"2024-07-17T06:54:54.571786+0000","last_undegraded":"2024-07-17T06:54:54.571786+0000","last_fullsized":"2024-07-17T06:54:54.571786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341192+0000","last_change":"2024-07-17T06:54:49.301897+0000","last_active":"2024-07-17T06:54:54.341192+0000","last_peered":"2024-07-17T06:54:54.341192+0000","last_clean":"2024-07-17T06:54:54.341192+0000","last_became_active":"2024-07-17T06:54:49.301602+0000","last_became_peered":"2024-07-17T06:54:49.301602+0000","last_unstale":"2024-07-17T06:54:54.341192+0000","last_undegraded":"2024-07-17T06:54:54.341192+0000","last_fullsized":"2024-07-17T06:54:54.341192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:39.268743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568075+0000","last_change":"2024-07-17T06:54:48.280905+0000","last_active":"2024-07-17T06:54:54.568075+0000","last_peered":"2024-07-17T06:54:54.568075+0000","last_clean":"2024-07-17T06:54:54.568075+0000","last_became_active":"2024-07-17T06:54:48.280538+0000","last_became_peered":"2024-07-17T06:54:48.280538+0000","last_unstale":"2024-07-17T06:54:54.568075+0000","last_undegraded":"2024-07-17T06:54:54.568075+0000","last_fullsized":"2024-07-17T06:54:54.568075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343510+0000","last_change":"2024-07-17T06:54:53.356455+0000","last_active":"2024-07-17T06:54:54.343510+0000","last_peered":"2024-07-17T06:54:54.343510+0000","last_clean":"2024-07-17T06:54:54.343510+0000","last_became_active":"2024-07-17T06:54:53.355594+0000","last_became_peered":"2024-07-17T06:54:53.355594+0000","last_unstale":"2024-07-17T06:54:54.343510+0000","last_undegraded":"2024-07-17T06:54:54.343510+0000","last_fullsized":"2024-07-17T06:54:54.343510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:37.799189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569678+0000","last_change":"2024-07-17T06:54:49.306398+0000","last_active":"2024-07-17T06:54:54.569678+0000","last_peered":"2024-07-17T06:54:54.569678+0000","last_clean":"2024-07-17T06:54:54.569678+0000","last_became_active":"2024-07-17T06:54:49.305914+0000","last_became_peered":"2024-07-17T06:54:49.305914+0000","last_unstale":"2024-07-17T06:54:54.569678+0000","last_undegraded":"2024-07-17T06:54:54.569678+0000","last_fullsized":"2024-07-17T06:54:54.569678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343534+0000","last_change":"2024-07-17T06:54:48.281630+0000","last_active":"2024-07-17T06:54:54.343534+0000","last_peered":"2024-07-17T06:54:54.343534+0000","last_clean":"2024-07-17T06:54:54.343534+0000","last_became_active":"2024-07-17T06:54:48.281368+0000","last_became_peered":"2024-07-17T06:54:48.281368+0000","last_unstale":"2024-07-17T06:54:54.343534+0000","last_undegraded":"2024-07-17T06:54:54.343534+0000","last_fullsized":"2024-07-17T06:54:54.343534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:09.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340594+0000","last_change":"2024-07-17T06:54:53.685324+0000","last_active":"2024-07-17T06:54:54.340594+0000","last_peered":"2024-07-17T06:54:54.340594+0000","last_clean":"2024-07-17T06:54:54.340594+0000","last_became_active":"2024-07-17T06:54:53.683906+0000","last_became_peered":"2024-07-17T06:54:53.683906+0000","last_unstale":"2024-07-17T06:54:54.340594+0000","last_undegraded":"2024-07-17T06:54:54.340594+0000","last_fullsized":"2024-07-17T06:54:54.340594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:31.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572498+0000","last_change":"2024-07-17T06:54:49.307258+0000","last_active":"2024-07-17T06:54:54.572498+0000","last_peered":"2024-07-17T06:54:54.572498+0000","last_clean":"2024-07-17T06:54:54.572498+0000","last_became_active":"2024-07-17T06:54:49.306989+0000","last_became_peered":"2024-07-17T06:54:49.306989+0000","last_unstale":"2024-07-17T06:54:54.572498+0000","last_undegraded":"2024-07-17T06:54:54.572498+0000","last_fullsized":"2024-07-17T06:54:54.572498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571324+0000","last_change":"2024-07-17T06:54:48.273115+0000","last_active":"2024-07-17T06:54:54.571324+0000","last_peered":"2024-07-17T06:54:54.571324+0000","last_clean":"2024-07-17T06:54:54.571324+0000","last_became_active":"2024-07-17T06:54:48.272804+0000","last_became_peered":"2024-07-17T06:54:48.272804+0000","last_unstale":"2024-07-17T06:54:54.571324+0000","last_undegraded":"2024-07-17T06:54:54.571324+0000","last_fullsized":"2024-07-17T06:54:54.571324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566623+0000","last_change":"2024-07-17T06:54:53.667515+0000","last_active":"2024-07-17T06:54:54.566623+0000","last_peered":"2024-07-17T06:54:54.566623+0000","last_clean":"2024-07-17T06:54:54.566623+0000","last_became_active":"2024-07-17T06:54:53.665920+0000","last_became_peered":"2024-07-17T06:54:53.665920+0000","last_unstale":"2024-07-17T06:54:54.566623+0000","last_undegraded":"2024-07-17T06:54:54.566623+0000","last_fullsized":"2024-07-17T06:54:54.566623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:20.348618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569843+0000","last_change":"2024-07-17T06:54:49.296691+0000","last_active":"2024-07-17T06:54:54.569843+0000","last_peered":"2024-07-17T06:54:54.569843+0000","last_clean":"2024-07-17T06:54:54.569843+0000","last_became_active":"2024-07-17T06:54:49.296340+0000","last_became_peered":"2024-07-17T06:54:49.296340+0000","last_unstale":"2024-07-17T06:54:54.569843+0000","last_undegraded":"2024-07-17T06:54:54.569843+0000","last_fullsized":"2024-07-17T06:54:54.569843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569522+0000","last_change":"2024-07-17T06:54:48.288182+0000","last_active":"2024-07-17T06:54:54.569522+0000","last_peered":"2024-07-17T06:54:54.569522+0000","last_clean":"2024-07-17T06:54:54.569522+0000","last_became_active":"2024-07-17T06:54:48.287723+0000","last_became_peered":"2024-07-17T06:54:48.287723+0000","last_unstale":"2024-07-17T06:54:54.569522+0000","last_undegraded":"2024-07-17T06:54:54.569522+0000","last_fullsized":"2024-07-17T06:54:54.569522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:54.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565754+0000","last_change":"2024-07-17T06:54:53.672017+0000","last_active":"2024-07-17T06:54:54.565754+0000","last_peered":"2024-07-17T06:54:54.565754+0000","last_clean":"2024-07-17T06:54:54.565754+0000","last_became_active":"2024-07-17T06:54:53.671036+0000","last_became_peered":"2024-07-17T06:54:53.671036+0000","last_unstale":"2024-07-17T06:54:54.565754+0000","last_undegraded":"2024-07-17T06:54:54.565754+0000","last_fullsized":"2024-07-17T06:54:54.565754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567594+0000","last_change":"2024-07-17T06:54:49.320403+0000","last_active":"2024-07-17T06:54:54.567594+0000","last_peered":"2024-07-17T06:54:54.567594+0000","last_clean":"2024-07-17T06:54:54.567594+0000","last_became_active":"2024-07-17T06:54:49.320119+0000","last_became_peered":"2024-07-17T06:54:49.320119+0000","last_unstale":"2024-07-17T06:54:54.567594+0000","last_undegraded":"2024-07-17T06:54:54.567594+0000","last_fullsized":"2024-07-17T06:54:54.567594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:41.108822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569854+0000","last_change":"2024-07-17T06:54:48.271506+0000","last_active":"2024-07-17T06:54:54.569854+0000","last_peered":"2024-07-17T06:54:54.569854+0000","last_clean":"2024-07-17T06:54:54.569854+0000","last_became_active":"2024-07-17T06:54:48.271181+0000","last_became_peered":"2024-07-17T06:54:48.271181+0000","last_unstale":"2024-07-17T06:54:54.569854+0000","last_undegraded":"2024-07-17T06:54:54.569854+0000","last_fullsized":"2024-07-17T06:54:54.569854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568214+0000","last_change":"2024-07-17T06:54:53.358600+0000","last_active":"2024-07-17T06:54:54.568214+0000","last_peered":"2024-07-17T06:54:54.568214+0000","last_clean":"2024-07-17T06:54:54.568214+0000","last_became_active":"2024-07-17T06:54:53.355359+0000","last_became_peered":"2024-07-17T06:54:53.355359+0000","last_unstale":"2024-07-17T06:54:54.568214+0000","last_undegraded":"2024-07-17T06:54:54.568214+0000","last_fullsized":"2024-07-17T06:54:54.568214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344036+0000","last_change":"2024-07-17T06:54:49.305202+0000","last_active":"2024-07-17T06:54:54.344036+0000","last_peered":"2024-07-17T06:54:54.344036+0000","last_clean":"2024-07-17T06:54:54.344036+0000","last_became_active":"2024-07-17T06:54:49.304856+0000","last_became_peered":"2024-07-17T06:54:49.304856+0000","last_unstale":"2024-07-17T06:54:54.344036+0000","last_undegraded":"2024-07-17T06:54:54.344036+0000","last_fullsized":"2024-07-17T06:54:54.344036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:00.298763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566783+0000","last_change":"2024-07-17T06:54:48.279231+0000","last_active":"2024-07-17T06:54:54.566783+0000","last_peered":"2024-07-17T06:54:54.566783+0000","last_clean":"2024-07-17T06:54:54.566783+0000","last_became_active":"2024-07-17T06:54:48.277422+0000","last_became_peered":"2024-07-17T06:54:48.277422+0000","last_unstale":"2024-07-17T06:54:54.566783+0000","last_undegraded":"2024-07-17T06:54:54.566783+0000","last_fullsized":"2024-07-17T06:54:54.566783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569758+0000","last_change":"2024-07-17T06:54:53.350972+0000","last_active":"2024-07-17T06:54:54.569758+0000","last_peered":"2024-07-17T06:54:54.569758+0000","last_clean":"2024-07-17T06:54:54.569758+0000","last_became_active":"2024-07-17T06:54:53.347571+0000","last_became_peered":"2024-07-17T06:54:53.347571+0000","last_unstale":"2024-07-17T06:54:54.569758+0000","last_undegraded":"2024-07-17T06:54:54.569758+0000","last_fullsized":"2024-07-17T06:54:54.569758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567608+0000","last_change":"2024-07-17T06:54:49.302823+0000","last_active":"2024-07-17T06:54:54.567608+0000","last_peered":"2024-07-17T06:54:54.567608+0000","last_clean":"2024-07-17T06:54:54.567608+0000","last_became_active":"2024-07-17T06:54:49.302434+0000","last_became_peered":"2024-07-17T06:54:49.302434+0000","last_unstale":"2024-07-17T06:54:54.567608+0000","last_undegraded":"2024-07-17T06:54:54.567608+0000","last_fullsized":"2024-07-17T06:54:54.567608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339350+0000","last_change":"2024-07-17T06:54:48.268185+0000","last_active":"2024-07-17T06:54:54.339350+0000","last_peered":"2024-07-17T06:54:54.339350+0000","last_clean":"2024-07-17T06:54:54.339350+0000","last_became_active":"2024-07-17T06:54:48.267807+0000","last_became_peered":"2024-07-17T06:54:48.267807+0000","last_unstale":"2024-07-17T06:54:54.339350+0000","last_undegraded":"2024-07-17T06:54:54.339350+0000","last_fullsized":"2024-07-17T06:54:54.339350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:28.391194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572146+0000","last_change":"2024-07-17T06:54:53.345628+0000","last_active":"2024-07-17T06:54:54.572146+0000","last_peered":"2024-07-17T06:54:54.572146+0000","last_clean":"2024-07-17T06:54:54.572146+0000","last_became_active":"2024-07-17T06:54:53.345269+0000","last_became_peered":"2024-07-17T06:54:53.345269+0000","last_unstale":"2024-07-17T06:54:54.572146+0000","last_undegraded":"2024-07-17T06:54:54.572146+0000","last_fullsized":"2024-07-17T06:54:54.572146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565847+0000","last_change":"2024-07-17T06:54:49.302929+0000","last_active":"2024-07-17T06:54:54.565847+0000","last_peered":"2024-07-17T06:54:54.565847+0000","last_clean":"2024-07-17T06:54:54.565847+0000","last_became_active":"2024-07-17T06:54:49.302598+0000","last_became_peered":"2024-07-17T06:54:49.302598+0000","last_unstale":"2024-07-17T06:54:54.565847+0000","last_undegraded":"2024-07-17T06:54:54.565847+0000","last_fullsized":"2024-07-17T06:54:54.565847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568955+0000","last_change":"2024-07-17T06:54:48.284164+0000","last_active":"2024-07-17T06:54:54.568955+0000","last_peered":"2024-07-17T06:54:54.568955+0000","last_clean":"2024-07-17T06:54:54.568955+0000","last_became_active":"2024-07-17T06:54:48.282885+0000","last_became_peered":"2024-07-17T06:54:48.282885+0000","last_unstale":"2024-07-17T06:54:54.568955+0000","last_undegraded":"2024-07-17T06:54:54.568955+0000","last_fullsized":"2024-07-17T06:54:54.568955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:05.674177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569179+0000","last_change":"2024-07-17T06:54:53.677311+0000","last_active":"2024-07-17T06:54:54.569179+0000","last_peered":"2024-07-17T06:54:54.569179+0000","last_clean":"2024-07-17T06:54:54.569179+0000","last_became_active":"2024-07-17T06:54:53.670958+0000","last_became_peered":"2024-07-17T06:54:53.670958+0000","last_unstale":"2024-07-17T06:54:54.569179+0000","last_undegraded":"2024-07-17T06:54:54.569179+0000","last_fullsized":"2024-07-17T06:54:54.569179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566963+0000","last_change":"2024-07-17T06:54:49.312042+0000","last_active":"2024-07-17T06:54:54.566963+0000","last_peered":"2024-07-17T06:54:54.566963+0000","last_clean":"2024-07-17T06:54:54.566963+0000","last_became_active":"2024-07-17T06:54:49.311741+0000","last_became_peered":"2024-07-17T06:54:49.311741+0000","last_unstale":"2024-07-17T06:54:54.566963+0000","last_undegraded":"2024-07-17T06:54:54.566963+0000","last_fullsized":"2024-07-17T06:54:54.566963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567568+0000","last_change":"2024-07-17T06:54:48.277447+0000","last_active":"2024-07-17T06:54:54.567568+0000","last_peered":"2024-07-17T06:54:54.567568+0000","last_clean":"2024-07-17T06:54:54.567568+0000","last_became_active":"2024-07-17T06:54:48.273874+0000","last_became_peered":"2024-07-17T06:54:48.273874+0000","last_unstale":"2024-07-17T06:54:54.567568+0000","last_undegraded":"2024-07-17T06:54:54.567568+0000","last_fullsized":"2024-07-17T06:54:54.567568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568469+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:54:54.568469+0000","last_peered":"2024-07-17T06:54:54.568469+0000","last_clean":"2024-07-17T06:54:54.568469+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:54:54.568469+0000","last_undegraded":"2024-07-17T06:54:54.568469+0000","last_fullsized":"2024-07-17T06:54:54.568469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340950+0000","last_change":"2024-07-17T06:54:53.680957+0000","last_active":"2024-07-17T06:54:54.340950+0000","last_peered":"2024-07-17T06:54:54.340950+0000","last_clean":"2024-07-17T06:54:54.340950+0000","last_became_active":"2024-07-17T06:54:53.679463+0000","last_became_peered":"2024-07-17T06:54:53.679463+0000","last_unstale":"2024-07-17T06:54:54.340950+0000","last_undegraded":"2024-07-17T06:54:54.340950+0000","last_fullsized":"2024-07-17T06:54:54.340950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568688+0000","last_change":"2024-07-17T06:54:49.313201+0000","last_active":"2024-07-17T06:54:54.568688+0000","last_peered":"2024-07-17T06:54:54.568688+0000","last_clean":"2024-07-17T06:54:54.568688+0000","last_became_active":"2024-07-17T06:54:49.312937+0000","last_became_peered":"2024-07-17T06:54:49.312937+0000","last_unstale":"2024-07-17T06:54:54.568688+0000","last_undegraded":"2024-07-17T06:54:54.568688+0000","last_fullsized":"2024-07-17T06:54:54.568688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:16.869857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339687+0000","last_change":"2024-07-17T06:54:48.276023+0000","last_active":"2024-07-17T06:54:54.339687+0000","last_peered":"2024-07-17T06:54:54.339687+0000","last_clean":"2024-07-17T06:54:54.339687+0000","last_became_active":"2024-07-17T06:54:48.275283+0000","last_became_peered":"2024-07-17T06:54:48.275283+0000","last_unstale":"2024-07-17T06:54:54.339687+0000","last_undegraded":"2024-07-17T06:54:54.339687+0000","last_fullsized":"2024-07-17T06:54:54.339687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:52.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341589+0000","last_change":"2024-07-17T06:54:53.674509+0000","last_active":"2024-07-17T06:54:54.341589+0000","last_peered":"2024-07-17T06:54:54.341589+0000","last_clean":"2024-07-17T06:54:54.341589+0000","last_became_active":"2024-07-17T06:54:53.672806+0000","last_became_peered":"2024-07-17T06:54:53.672806+0000","last_unstale":"2024-07-17T06:54:54.341589+0000","last_undegraded":"2024-07-17T06:54:54.341589+0000","last_fullsized":"2024-07-17T06:54:54.341589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567064+0000","last_change":"2024-07-17T06:54:49.324849+0000","last_active":"2024-07-17T06:54:54.567064+0000","last_peered":"2024-07-17T06:54:54.567064+0000","last_clean":"2024-07-17T06:54:54.567064+0000","last_became_active":"2024-07-17T06:54:49.324195+0000","last_became_peered":"2024-07-17T06:54:49.324195+0000","last_unstale":"2024-07-17T06:54:54.567064+0000","last_undegraded":"2024-07-17T06:54:54.567064+0000","last_fullsized":"2024-07-17T06:54:54.567064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:45.753022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570696+0000","last_change":"2024-07-17T06:54:48.276038+0000","last_active":"2024-07-17T06:54:54.570696+0000","last_peered":"2024-07-17T06:54:54.570696+0000","last_clean":"2024-07-17T06:54:54.570696+0000","last_became_active":"2024-07-17T06:54:48.275729+0000","last_became_peered":"2024-07-17T06:54:48.275729+0000","last_unstale":"2024-07-17T06:54:54.570696+0000","last_undegraded":"2024-07-17T06:54:54.570696+0000","last_fullsized":"2024-07-17T06:54:54.570696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565953+0000","last_change":"2024-07-17T06:54:53.356746+0000","last_active":"2024-07-17T06:54:54.565953+0000","last_peered":"2024-07-17T06:54:54.565953+0000","last_clean":"2024-07-17T06:54:54.565953+0000","last_became_active":"2024-07-17T06:54:53.353358+0000","last_became_peered":"2024-07-17T06:54:53.353358+0000","last_unstale":"2024-07-17T06:54:54.565953+0000","last_undegraded":"2024-07-17T06:54:54.565953+0000","last_fullsized":"2024-07-17T06:54:54.565953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569283+0000","last_change":"2024-07-17T06:54:49.318930+0000","last_active":"2024-07-17T06:54:54.569283+0000","last_peered":"2024-07-17T06:54:54.569283+0000","last_clean":"2024-07-17T06:54:54.569283+0000","last_became_active":"2024-07-17T06:54:49.318606+0000","last_became_peered":"2024-07-17T06:54:49.318606+0000","last_unstale":"2024-07-17T06:54:54.569283+0000","last_undegraded":"2024-07-17T06:54:54.569283+0000","last_fullsized":"2024-07-17T06:54:54.569283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:10:28.608769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339970+0000","last_change":"2024-07-17T06:54:48.269152+0000","last_active":"2024-07-17T06:54:54.339970+0000","last_peered":"2024-07-17T06:54:54.339970+0000","last_clean":"2024-07-17T06:54:54.339970+0000","last_became_active":"2024-07-17T06:54:48.268914+0000","last_became_peered":"2024-07-17T06:54:48.268914+0000","last_unstale":"2024-07-17T06:54:54.339970+0000","last_undegraded":"2024-07-17T06:54:54.339970+0000","last_fullsized":"2024-07-17T06:54:54.339970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565453+0000","last_change":"2024-07-17T06:54:53.672373+0000","last_active":"2024-07-17T06:54:54.565453+0000","last_peered":"2024-07-17T06:54:54.565453+0000","last_clean":"2024-07-17T06:54:54.565453+0000","last_became_active":"2024-07-17T06:54:53.671288+0000","last_became_peered":"2024-07-17T06:54:53.671288+0000","last_unstale":"2024-07-17T06:54:54.565453+0000","last_undegraded":"2024-07-17T06:54:54.565453+0000","last_fullsized":"2024-07-17T06:54:54.565453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568889+0000","last_change":"2024-07-17T06:54:49.320151+0000","last_active":"2024-07-17T06:54:54.568889+0000","last_peered":"2024-07-17T06:54:54.568889+0000","last_clean":"2024-07-17T06:54:54.568889+0000","last_became_active":"2024-07-17T06:54:49.319845+0000","last_became_peered":"2024-07-17T06:54:49.319845+0000","last_unstale":"2024-07-17T06:54:54.568889+0000","last_undegraded":"2024-07-17T06:54:54.568889+0000","last_fullsized":"2024-07-17T06:54:54.568889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:12:25.836163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343920+0000","last_change":"2024-07-17T06:54:48.268964+0000","last_active":"2024-07-17T06:54:54.343920+0000","last_peered":"2024-07-17T06:54:54.343920+0000","last_clean":"2024-07-17T06:54:54.343920+0000","last_became_active":"2024-07-17T06:54:48.268564+0000","last_became_peered":"2024-07-17T06:54:48.268564+0000","last_unstale":"2024-07-17T06:54:54.343920+0000","last_undegraded":"2024-07-17T06:54:54.343920+0000","last_fullsized":"2024-07-17T06:54:54.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:47.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340002+0000","last_change":"2024-07-17T06:54:53.362347+0000","last_active":"2024-07-17T06:54:54.340002+0000","last_peered":"2024-07-17T06:54:54.340002+0000","last_clean":"2024-07-17T06:54:54.340002+0000","last_became_active":"2024-07-17T06:54:53.361896+0000","last_became_peered":"2024-07-17T06:54:53.361896+0000","last_unstale":"2024-07-17T06:54:54.340002+0000","last_undegraded":"2024-07-17T06:54:54.340002+0000","last_fullsized":"2024-07-17T06:54:54.340002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:22.522023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566168+0000","last_change":"2024-07-17T06:54:49.316448+0000","last_active":"2024-07-17T06:54:54.566168+0000","last_peered":"2024-07-17T06:54:54.566168+0000","last_clean":"2024-07-17T06:54:54.566168+0000","last_became_active":"2024-07-17T06:54:49.316042+0000","last_became_peered":"2024-07-17T06:54:49.316042+0000","last_unstale":"2024-07-17T06:54:54.566168+0000","last_undegraded":"2024-07-17T06:54:54.566168+0000","last_fullsized":"2024-07-17T06:54:54.566168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:33.731836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.338832+0000","last_change":"2024-07-17T06:54:48.270227+0000","last_active":"2024-07-17T06:54:54.338832+0000","last_peered":"2024-07-17T06:54:54.338832+0000","last_clean":"2024-07-17T06:54:54.338832+0000","last_became_active":"2024-07-17T06:54:48.269954+0000","last_became_peered":"2024-07-17T06:54:48.269954+0000","last_unstale":"2024-07-17T06:54:54.338832+0000","last_undegraded":"2024-07-17T06:54:54.338832+0000","last_fullsized":"2024-07-17T06:54:54.338832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.390020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568918+0000","last_change":"2024-07-17T06:54:53.356983+0000","last_active":"2024-07-17T06:54:54.568918+0000","last_peered":"2024-07-17T06:54:54.568918+0000","last_clean":"2024-07-17T06:54:54.568918+0000","last_became_active":"2024-07-17T06:54:53.356288+0000","last_became_peered":"2024-07-17T06:54:53.356288+0000","last_unstale":"2024-07-17T06:54:54.568918+0000","last_undegraded":"2024-07-17T06:54:54.568918+0000","last_fullsized":"2024-07-17T06:54:54.568918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339405+0000","last_change":"2024-07-17T06:54:49.294433+0000","last_active":"2024-07-17T06:54:54.339405+0000","last_peered":"2024-07-17T06:54:54.339405+0000","last_clean":"2024-07-17T06:54:54.339405+0000","last_became_active":"2024-07-17T06:54:49.293644+0000","last_became_peered":"2024-07-17T06:54:49.293644+0000","last_unstale":"2024-07-17T06:54:54.339405+0000","last_undegraded":"2024-07-17T06:54:54.339405+0000","last_fullsized":"2024-07-17T06:54:54.339405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340443+0000","last_change":"2024-07-17T06:54:48.267002+0000","last_active":"2024-07-17T06:54:54.340443+0000","last_peered":"2024-07-17T06:54:54.340443+0000","last_clean":"2024-07-17T06:54:54.340443+0000","last_became_active":"2024-07-17T06:54:48.266711+0000","last_became_peered":"2024-07-17T06:54:48.266711+0000","last_unstale":"2024-07-17T06:54:54.340443+0000","last_undegraded":"2024-07-17T06:54:54.340443+0000","last_fullsized":"2024-07-17T06:54:54.340443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572427+0000","last_change":"2024-07-17T06:54:53.678483+0000","last_active":"2024-07-17T06:54:54.572427+0000","last_peered":"2024-07-17T06:54:54.572427+0000","last_clean":"2024-07-17T06:54:54.572427+0000","last_became_active":"2024-07-17T06:54:53.677707+0000","last_became_peered":"2024-07-17T06:54:53.677707+0000","last_unstale":"2024-07-17T06:54:54.572427+0000","last_undegraded":"2024-07-17T06:54:54.572427+0000","last_fullsized":"2024-07-17T06:54:54.572427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:21.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568071+0000","last_change":"2024-07-17T06:54:49.319059+0000","last_active":"2024-07-17T06:54:54.568071+0000","last_peered":"2024-07-17T06:54:54.568071+0000","last_clean":"2024-07-17T06:54:54.568071+0000","last_became_active":"2024-07-17T06:54:49.318532+0000","last_became_peered":"2024-07-17T06:54:49.318532+0000","last_unstale":"2024-07-17T06:54:54.568071+0000","last_undegraded":"2024-07-17T06:54:54.568071+0000","last_fullsized":"2024-07-17T06:54:54.568071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:47.675783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566475+0000","last_change":"2024-07-17T06:54:48.279991+0000","last_active":"2024-07-17T06:54:54.566475+0000","last_peered":"2024-07-17T06:54:54.566475+0000","last_clean":"2024-07-17T06:54:54.566475+0000","last_became_active":"2024-07-17T06:54:48.279691+0000","last_became_peered":"2024-07-17T06:54:48.279691+0000","last_unstale":"2024-07-17T06:54:54.566475+0000","last_undegraded":"2024-07-17T06:54:54.566475+0000","last_fullsized":"2024-07-17T06:54:54.566475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571709+0000","last_change":"2024-07-17T06:54:53.677451+0000","last_active":"2024-07-17T06:54:54.571709+0000","last_peered":"2024-07-17T06:54:54.571709+0000","last_clean":"2024-07-17T06:54:54.571709+0000","last_became_active":"2024-07-17T06:54:53.677080+0000","last_became_peered":"2024-07-17T06:54:53.677080+0000","last_unstale":"2024-07-17T06:54:54.571709+0000","last_undegraded":"2024-07-17T06:54:54.571709+0000","last_fullsized":"2024-07-17T06:54:54.571709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567583+0000","last_change":"2024-07-17T06:54:49.302477+0000","last_active":"2024-07-17T06:54:54.567583+0000","last_peered":"2024-07-17T06:54:54.567583+0000","last_clean":"2024-07-17T06:54:54.567583+0000","last_became_active":"2024-07-17T06:54:49.302168+0000","last_became_peered":"2024-07-17T06:54:49.302168+0000","last_unstale":"2024-07-17T06:54:54.567583+0000","last_undegraded":"2024-07-17T06:54:54.567583+0000","last_fullsized":"2024-07-17T06:54:54.567583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:20.279165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566160+0000","last_change":"2024-07-17T06:54:48.278091+0000","last_active":"2024-07-17T06:54:54.566160+0000","last_peered":"2024-07-17T06:54:54.566160+0000","last_clean":"2024-07-17T06:54:54.566160+0000","last_became_active":"2024-07-17T06:54:48.277769+0000","last_became_peered":"2024-07-17T06:54:48.277769+0000","last_unstale":"2024-07-17T06:54:54.566160+0000","last_undegraded":"2024-07-17T06:54:54.566160+0000","last_fullsized":"2024-07-17T06:54:54.566160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567004+0000","last_change":"2024-07-17T06:54:53.672346+0000","last_active":"2024-07-17T06:54:54.567004+0000","last_peered":"2024-07-17T06:54:54.567004+0000","last_clean":"2024-07-17T06:54:54.567004+0000","last_became_active":"2024-07-17T06:54:53.670973+0000","last_became_peered":"2024-07-17T06:54:53.670973+0000","last_unstale":"2024-07-17T06:54:54.567004+0000","last_undegraded":"2024-07-17T06:54:54.567004+0000","last_fullsized":"2024-07-17T06:54:54.567004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568419+0000","last_change":"2024-07-17T06:54:49.314009+0000","last_active":"2024-07-17T06:54:54.568419+0000","last_peered":"2024-07-17T06:54:54.568419+0000","last_clean":"2024-07-17T06:54:54.568419+0000","last_became_active":"2024-07-17T06:54:49.313238+0000","last_became_peered":"2024-07-17T06:54:49.313238+0000","last_unstale":"2024-07-17T06:54:54.568419+0000","last_undegraded":"2024-07-17T06:54:54.568419+0000","last_fullsized":"2024-07-17T06:54:54.568419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:16.115918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:02.616138+0000","last_change":"2024-07-17T06:54:48.274173+0000","last_active":"2024-07-17T06:55:02.616138+0000","last_peered":"2024-07-17T06:55:02.616138+0000","last_clean":"2024-07-17T06:55:02.616138+0000","last_became_active":"2024-07-17T06:54:48.270354+0000","last_became_peered":"2024-07-17T06:54:48.270354+0000","last_unstale":"2024-07-17T06:55:02.616138+0000","last_undegraded":"2024-07-17T06:55:02.616138+0000","last_fullsized":"2024-07-17T06:55:02.616138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568505+0000","last_change":"2024-07-17T06:54:53.347808+0000","last_active":"2024-07-17T06:54:54.568505+0000","last_peered":"2024-07-17T06:54:54.568505+0000","last_clean":"2024-07-17T06:54:54.568505+0000","last_became_active":"2024-07-17T06:54:53.347369+0000","last_became_peered":"2024-07-17T06:54:53.347369+0000","last_unstale":"2024-07-17T06:54:54.568505+0000","last_undegraded":"2024-07-17T06:54:54.568505+0000","last_fullsized":"2024-07-17T06:54:54.568505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344599+0000","last_change":"2024-07-17T06:54:49.308852+0000","last_active":"2024-07-17T06:54:54.344599+0000","last_peered":"2024-07-17T06:54:54.344599+0000","last_clean":"2024-07-17T06:54:54.344599+0000","last_became_active":"2024-07-17T06:54:49.308337+0000","last_became_peered":"2024-07-17T06:54:49.308337+0000","last_unstale":"2024-07-17T06:54:54.344599+0000","last_undegraded":"2024-07-17T06:54:54.344599+0000","last_fullsized":"2024-07-17T06:54:54.344599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:06.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570694+0000","last_change":"2024-07-17T06:54:48.282475+0000","last_active":"2024-07-17T06:54:54.570694+0000","last_peered":"2024-07-17T06:54:54.570694+0000","last_clean":"2024-07-17T06:54:54.570694+0000","last_became_active":"2024-07-17T06:54:48.282039+0000","last_became_peered":"2024-07-17T06:54:48.282039+0000","last_unstale":"2024-07-17T06:54:54.570694+0000","last_undegraded":"2024-07-17T06:54:54.570694+0000","last_fullsized":"2024-07-17T06:54:54.570694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344160+0000","last_change":"2024-07-17T06:54:53.353664+0000","last_active":"2024-07-17T06:54:54.344160+0000","last_peered":"2024-07-17T06:54:54.344160+0000","last_clean":"2024-07-17T06:54:54.344160+0000","last_became_active":"2024-07-17T06:54:53.351779+0000","last_became_peered":"2024-07-17T06:54:53.351779+0000","last_unstale":"2024-07-17T06:54:54.344160+0000","last_undegraded":"2024-07-17T06:54:54.344160+0000","last_fullsized":"2024-07-17T06:54:54.344160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:20.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567440+0000","last_change":"2024-07-17T06:54:49.313557+0000","last_active":"2024-07-17T06:54:54.567440+0000","last_peered":"2024-07-17T06:54:54.567440+0000","last_clean":"2024-07-17T06:54:54.567440+0000","last_became_active":"2024-07-17T06:54:49.312875+0000","last_became_peered":"2024-07-17T06:54:49.312875+0000","last_unstale":"2024-07-17T06:54:54.567440+0000","last_undegraded":"2024-07-17T06:54:54.567440+0000","last_fullsized":"2024-07-17T06:54:54.567440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567055+0000","last_change":"2024-07-17T06:54:48.286081+0000","last_active":"2024-07-17T06:54:54.567055+0000","last_peered":"2024-07-17T06:54:54.567055+0000","last_clean":"2024-07-17T06:54:54.567055+0000","last_became_active":"2024-07-17T06:54:48.285327+0000","last_became_peered":"2024-07-17T06:54:48.285327+0000","last_unstale":"2024-07-17T06:54:54.567055+0000","last_undegraded":"2024-07-17T06:54:54.567055+0000","last_fullsized":"2024-07-17T06:54:54.567055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567082+0000","last_change":"2024-07-17T06:54:53.353255+0000","last_active":"2024-07-17T06:54:54.567082+0000","last_peered":"2024-07-17T06:54:54.567082+0000","last_clean":"2024-07-17T06:54:54.567082+0000","last_became_active":"2024-07-17T06:54:53.352945+0000","last_became_peered":"2024-07-17T06:54:53.352945+0000","last_unstale":"2024-07-17T06:54:54.567082+0000","last_undegraded":"2024-07-17T06:54:54.567082+0000","last_fullsized":"2024-07-17T06:54:54.567082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:06.889450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566804+0000","last_change":"2024-07-17T06:54:49.318904+0000","last_active":"2024-07-17T06:54:54.566804+0000","last_peered":"2024-07-17T06:54:54.566804+0000","last_clean":"2024-07-17T06:54:54.566804+0000","last_became_active":"2024-07-17T06:54:49.316753+0000","last_became_peered":"2024-07-17T06:54:49.316753+0000","last_unstale":"2024-07-17T06:54:54.566804+0000","last_undegraded":"2024-07-17T06:54:54.566804+0000","last_fullsized":"2024-07-17T06:54:54.566804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:45.913897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343278+0000","last_change":"2024-07-17T06:54:48.286978+0000","last_active":"2024-07-17T06:54:54.343278+0000","last_peered":"2024-07-17T06:54:54.343278+0000","last_clean":"2024-07-17T06:54:54.343278+0000","last_became_active":"2024-07-17T06:54:48.286613+0000","last_became_peered":"2024-07-17T06:54:48.286613+0000","last_unstale":"2024-07-17T06:54:54.343278+0000","last_undegraded":"2024-07-17T06:54:54.343278+0000","last_fullsized":"2024-07-17T06:54:54.343278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570149+0000","last_change":"2024-07-17T06:54:53.671456+0000","last_active":"2024-07-17T06:54:54.570149+0000","last_peered":"2024-07-17T06:54:54.570149+0000","last_clean":"2024-07-17T06:54:54.570149+0000","last_became_active":"2024-07-17T06:54:53.670354+0000","last_became_peered":"2024-07-17T06:54:53.670354+0000","last_unstale":"2024-07-17T06:54:54.570149+0000","last_undegraded":"2024-07-17T06:54:54.570149+0000","last_fullsized":"2024-07-17T06:54:54.570149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571489+0000","last_change":"2024-07-17T06:54:53.682714+0000","last_active":"2024-07-17T06:54:54.571489+0000","last_peered":"2024-07-17T06:54:54.571489+0000","last_clean":"2024-07-17T06:54:54.571489+0000","last_became_active":"2024-07-17T06:54:53.681884+0000","last_became_peered":"2024-07-17T06:54:53.681884+0000","last_unstale":"2024-07-17T06:54:54.571489+0000","last_undegraded":"2024-07-17T06:54:54.571489+0000","last_fullsized":"2024-07-17T06:54:54.571489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568224+0000","last_change":"2024-07-17T06:54:49.305906+0000","last_active":"2024-07-17T06:54:54.568224+0000","last_peered":"2024-07-17T06:54:54.568224+0000","last_clean":"2024-07-17T06:54:54.568224+0000","last_became_active":"2024-07-17T06:54:49.305633+0000","last_became_peered":"2024-07-17T06:54:49.305633+0000","last_unstale":"2024-07-17T06:54:54.568224+0000","last_undegraded":"2024-07-17T06:54:54.568224+0000","last_fullsized":"2024-07-17T06:54:54.568224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565775+0000","last_change":"2024-07-17T06:54:48.275763+0000","last_active":"2024-07-17T06:54:54.565775+0000","last_peered":"2024-07-17T06:54:54.565775+0000","last_clean":"2024-07-17T06:54:54.565775+0000","last_became_active":"2024-07-17T06:54:48.275073+0000","last_became_peered":"2024-07-17T06:54:48.275073+0000","last_unstale":"2024-07-17T06:54:54.565775+0000","last_undegraded":"2024-07-17T06:54:54.565775+0000","last_fullsized":"2024-07-17T06:54:54.565775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343084+0000","last_change":"2024-07-17T06:54:53.347306+0000","last_active":"2024-07-17T06:54:54.343084+0000","last_peered":"2024-07-17T06:54:54.343084+0000","last_clean":"2024-07-17T06:54:54.343084+0000","last_became_active":"2024-07-17T06:54:53.346541+0000","last_became_peered":"2024-07-17T06:54:53.346541+0000","last_unstale":"2024-07-17T06:54:54.343084+0000","last_undegraded":"2024-07-17T06:54:54.343084+0000","last_fullsized":"2024-07-17T06:54:54.343084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567129+0000","last_change":"2024-07-17T06:54:49.299691+0000","last_active":"2024-07-17T06:54:54.567129+0000","last_peered":"2024-07-17T06:54:54.567129+0000","last_clean":"2024-07-17T06:54:54.567129+0000","last_became_active":"2024-07-17T06:54:49.299370+0000","last_became_peered":"2024-07-17T06:54:49.299370+0000","last_unstale":"2024-07-17T06:54:54.567129+0000","last_undegraded":"2024-07-17T06:54:54.567129+0000","last_fullsized":"2024-07-17T06:54:54.567129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570373+0000","last_change":"2024-07-17T06:54:48.276504+0000","last_active":"2024-07-17T06:54:54.570373+0000","last_peered":"2024-07-17T06:54:54.570373+0000","last_clean":"2024-07-17T06:54:54.570373+0000","last_became_active":"2024-07-17T06:54:48.276041+0000","last_became_peered":"2024-07-17T06:54:48.276041+0000","last_unstale":"2024-07-17T06:54:54.570373+0000","last_undegraded":"2024-07-17T06:54:54.570373+0000","last_fullsized":"2024-07-17T06:54:54.570373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342215+0000","last_change":"2024-07-17T06:54:48.267874+0000","last_active":"2024-07-17T06:54:54.342215+0000","last_peered":"2024-07-17T06:54:54.342215+0000","last_clean":"2024-07-17T06:54:54.342215+0000","last_became_active":"2024-07-17T06:54:48.267487+0000","last_became_peered":"2024-07-17T06:54:48.267487+0000","last_unstale":"2024-07-17T06:54:54.342215+0000","last_undegraded":"2024-07-17T06:54:54.342215+0000","last_fullsized":"2024-07-17T06:54:54.342215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566178+0000","last_change":"2024-07-17T06:54:49.306648+0000","last_active":"2024-07-17T06:54:54.566178+0000","last_peered":"2024-07-17T06:54:54.566178+0000","last_clean":"2024-07-17T06:54:54.566178+0000","last_became_active":"2024-07-17T06:54:49.305001+0000","last_became_peered":"2024-07-17T06:54:49.305001+0000","last_unstale":"2024-07-17T06:54:54.566178+0000","last_undegraded":"2024-07-17T06:54:54.566178+0000","last_fullsized":"2024-07-17T06:54:54.566178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339673+0000","last_change":"2024-07-17T06:54:53.360122+0000","last_active":"2024-07-17T06:54:54.339673+0000","last_peered":"2024-07-17T06:54:54.339673+0000","last_clean":"2024-07-17T06:54:54.339673+0000","last_became_active":"2024-07-17T06:54:53.356530+0000","last_became_peered":"2024-07-17T06:54:53.356530+0000","last_unstale":"2024-07-17T06:54:54.339673+0000","last_undegraded":"2024-07-17T06:54:54.339673+0000","last_fullsized":"2024-07-17T06:54:54.339673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:41.753064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570947+0000","last_change":"2024-07-17T06:54:48.277409+0000","last_active":"2024-07-17T06:54:54.570947+0000","last_peered":"2024-07-17T06:54:54.570947+0000","last_clean":"2024-07-17T06:54:54.570947+0000","last_became_active":"2024-07-17T06:54:48.277087+0000","last_became_peered":"2024-07-17T06:54:48.277087+0000","last_unstale":"2024-07-17T06:54:54.570947+0000","last_undegraded":"2024-07-17T06:54:54.570947+0000","last_fullsized":"2024-07-17T06:54:54.570947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:56:17.744184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342397+0000","last_change":"2024-07-17T06:54:49.300524+0000","last_active":"2024-07-17T06:54:54.342397+0000","last_peered":"2024-07-17T06:54:54.342397+0000","last_clean":"2024-07-17T06:54:54.342397+0000","last_became_active":"2024-07-17T06:54:49.300136+0000","last_became_peered":"2024-07-17T06:54:49.300136+0000","last_unstale":"2024-07-17T06:54:54.342397+0000","last_undegraded":"2024-07-17T06:54:54.342397+0000","last_fullsized":"2024-07-17T06:54:54.342397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:29.406070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571827+0000","last_change":"2024-07-17T06:54:53.355705+0000","last_active":"2024-07-17T06:54:54.571827+0000","last_peered":"2024-07-17T06:54:54.571827+0000","last_clean":"2024-07-17T06:54:54.571827+0000","last_became_active":"2024-07-17T06:54:53.355294+0000","last_became_peered":"2024-07-17T06:54:53.355294+0000","last_unstale":"2024-07-17T06:54:54.571827+0000","last_undegraded":"2024-07-17T06:54:54.571827+0000","last_fullsized":"2024-07-17T06:54:54.571827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:32:32.729442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339117+0000","last_change":"2024-07-17T06:54:49.294765+0000","last_active":"2024-07-17T06:54:54.339117+0000","last_peered":"2024-07-17T06:54:54.339117+0000","last_clean":"2024-07-17T06:54:54.339117+0000","last_became_active":"2024-07-17T06:54:49.294095+0000","last_became_peered":"2024-07-17T06:54:49.294095+0000","last_unstale":"2024-07-17T06:54:54.339117+0000","last_undegraded":"2024-07-17T06:54:54.339117+0000","last_fullsized":"2024-07-17T06:54:54.339117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339066+0000","last_change":"2024-07-17T06:54:49.299295+0000","last_active":"2024-07-17T06:54:54.339066+0000","last_peered":"2024-07-17T06:54:54.339066+0000","last_clean":"2024-07-17T06:54:54.339066+0000","last_became_active":"2024-07-17T06:54:49.298385+0000","last_became_peered":"2024-07-17T06:54:49.298385+0000","last_unstale":"2024-07-17T06:54:54.339066+0000","last_undegraded":"2024-07-17T06:54:54.339066+0000","last_fullsized":"2024-07-17T06:54:54.339066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:55.920311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566243+0000","last_change":"2024-07-17T06:54:49.325974+0000","last_active":"2024-07-17T06:54:54.566243+0000","last_peered":"2024-07-17T06:54:54.566243+0000","last_clean":"2024-07-17T06:54:54.566243+0000","last_became_active":"2024-07-17T06:54:49.322413+0000","last_became_peered":"2024-07-17T06:54:49.322413+0000","last_unstale":"2024-07-17T06:54:54.566243+0000","last_undegraded":"2024-07-17T06:54:54.566243+0000","last_fullsized":"2024-07-17T06:54:54.566243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:18.496012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566055+0000","last_change":"2024-07-17T06:54:49.308310+0000","last_active":"2024-07-17T06:54:54.566055+0000","last_peered":"2024-07-17T06:54:54.566055+0000","last_clean":"2024-07-17T06:54:54.566055+0000","last_became_active":"2024-07-17T06:54:49.304819+0000","last_became_peered":"2024-07-17T06:54:49.304819+0000","last_unstale":"2024-07-17T06:54:54.566055+0000","last_undegraded":"2024-07-17T06:54:54.566055+0000","last_fullsized":"2024-07-17T06:54:54.566055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571870+0000","last_change":"2024-07-17T06:54:49.312400+0000","last_active":"2024-07-17T06:54:54.571870+0000","last_peered":"2024-07-17T06:54:54.571870+0000","last_clean":"2024-07-17T06:54:54.571870+0000","last_became_active":"2024-07-17T06:54:49.311958+0000","last_became_peered":"2024-07-17T06:54:49.311958+0000","last_unstale":"2024-07-17T06:54:54.571870+0000","last_undegraded":"2024-07-17T06:54:54.571870+0000","last_fullsized":"2024-07-17T06:54:54.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.321280+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.320912+0000","last_became_peered":"2024-07-17T06:54:49.320912+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341006+0000","last_change":"2024-07-17T06:54:49.297628+0000","last_active":"2024-07-17T06:54:54.341006+0000","last_peered":"2024-07-17T06:54:54.341006+0000","last_clean":"2024-07-17T06:54:54.341006+0000","last_became_active":"2024-07-17T06:54:49.296871+0000","last_became_peered":"2024-07-17T06:54:49.296871+0000","last_unstale":"2024-07-17T06:54:54.341006+0000","last_undegraded":"2024-07-17T06:54:54.341006+0000","last_fullsized":"2024-07-17T06:54:54.341006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572513+0000","last_change":"2024-07-17T06:54:49.303253+0000","last_active":"2024-07-17T06:54:54.572513+0000","last_peered":"2024-07-17T06:54:54.572513+0000","last_clean":"2024-07-17T06:54:54.572513+0000","last_became_active":"2024-07-17T06:54:49.302956+0000","last_became_peered":"2024-07-17T06:54:49.302956+0000","last_unstale":"2024-07-17T06:54:54.572513+0000","last_undegraded":"2024-07-17T06:54:54.572513+0000","last_fullsized":"2024-07-17T06:54:54.572513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565697+0000","last_change":"2024-07-17T06:54:49.306156+0000","last_active":"2024-07-17T06:54:54.565697+0000","last_peered":"2024-07-17T06:54:54.565697+0000","last_clean":"2024-07-17T06:54:54.565697+0000","last_became_active":"2024-07-17T06:54:49.305858+0000","last_became_peered":"2024-07-17T06:54:49.305858+0000","last_unstale":"2024-07-17T06:54:54.565697+0000","last_undegraded":"2024-07-17T06:54:54.565697+0000","last_fullsized":"2024-07-17T06:54:54.565697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.301568+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.301244+0000","last_became_peered":"2024-07-17T06:54:49.301244+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567719+0000","last_change":"2024-07-17T06:54:49.316422+0000","last_active":"2024-07-17T06:54:54.567719+0000","last_peered":"2024-07-17T06:54:54.567719+0000","last_clean":"2024-07-17T06:54:54.567719+0000","last_became_active":"2024-07-17T06:54:49.316133+0000","last_became_peered":"2024-07-17T06:54:49.316133+0000","last_unstale":"2024-07-17T06:54:54.567719+0000","last_undegraded":"2024-07-17T06:54:54.567719+0000","last_fullsized":"2024-07-17T06:54:54.567719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565901+0000","last_change":"2024-07-17T06:54:49.305491+0000","last_active":"2024-07-17T06:54:54.565901+0000","last_peered":"2024-07-17T06:54:54.565901+0000","last_clean":"2024-07-17T06:54:54.565901+0000","last_became_active":"2024-07-17T06:54:49.304629+0000","last_became_peered":"2024-07-17T06:54:49.304629+0000","last_unstale":"2024-07-17T06:54:54.565901+0000","last_undegraded":"2024-07-17T06:54:54.565901+0000","last_fullsized":"2024-07-17T06:54:54.565901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343504+0000","last_change":"2024-07-17T06:54:49.309637+0000","last_active":"2024-07-17T06:54:54.343504+0000","last_peered":"2024-07-17T06:54:54.343504+0000","last_clean":"2024-07-17T06:54:54.343504+0000","last_became_active":"2024-07-17T06:54:49.309243+0000","last_became_peered":"2024-07-17T06:54:49.309243+0000","last_unstale":"2024-07-17T06:54:54.343504+0000","last_undegraded":"2024-07-17T06:54:54.343504+0000","last_fullsized":"2024-07-17T06:54:54.343504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569329+0000","last_change":"2024-07-17T06:54:49.323709+0000","last_active":"2024-07-17T06:54:54.569329+0000","last_peered":"2024-07-17T06:54:54.569329+0000","last_clean":"2024-07-17T06:54:54.569329+0000","last_became_active":"2024-07-17T06:54:49.323324+0000","last_became_peered":"2024-07-17T06:54:49.323324+0000","last_unstale":"2024-07-17T06:54:54.569329+0000","last_undegraded":"2024-07-17T06:54:54.569329+0000","last_fullsized":"2024-07-17T06:54:54.569329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339810+0000","last_change":"2024-07-17T06:54:49.286063+0000","last_active":"2024-07-17T06:54:54.339810+0000","last_peered":"2024-07-17T06:54:54.339810+0000","last_clean":"2024-07-17T06:54:54.339810+0000","last_became_active":"2024-07-17T06:54:49.283989+0000","last_became_peered":"2024-07-17T06:54:49.283989+0000","last_unstale":"2024-07-17T06:54:54.339810+0000","last_undegraded":"2024-07-17T06:54:54.339810+0000","last_fullsized":"2024-07-17T06:54:54.339810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:43.655076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567362+0000","last_change":"2024-07-17T06:54:49.319544+0000","last_active":"2024-07-17T06:54:54.567362+0000","last_peered":"2024-07-17T06:54:54.567362+0000","last_clean":"2024-07-17T06:54:54.567362+0000","last_became_active":"2024-07-17T06:54:49.319285+0000","last_became_peered":"2024-07-17T06:54:49.319285+0000","last_unstale":"2024-07-17T06:54:54.567362+0000","last_undegraded":"2024-07-17T06:54:54.567362+0000","last_fullsized":"2024-07-17T06:54:54.567362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:58.821222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565429+0000","last_change":"2024-07-17T06:54:49.305371+0000","last_active":"2024-07-17T06:54:54.565429+0000","last_peered":"2024-07-17T06:54:54.565429+0000","last_clean":"2024-07-17T06:54:54.565429+0000","last_became_active":"2024-07-17T06:54:49.305069+0000","last_became_peered":"2024-07-17T06:54:49.305069+0000","last_unstale":"2024-07-17T06:54:54.565429+0000","last_undegraded":"2024-07-17T06:54:54.565429+0000","last_fullsized":"2024-07-17T06:54:54.565429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568226+0000","last_change":"2024-07-17T06:54:49.314419+0000","last_active":"2024-07-17T06:54:54.568226+0000","last_peered":"2024-07-17T06:54:54.568226+0000","last_clean":"2024-07-17T06:54:54.568226+0000","last_became_active":"2024-07-17T06:54:49.314106+0000","last_became_peered":"2024-07-17T06:54:49.314106+0000","last_unstale":"2024-07-17T06:54:54.568226+0000","last_undegraded":"2024-07-17T06:54:54.568226+0000","last_fullsized":"2024-07-17T06:54:54.568226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566591+0000","last_change":"2024-07-17T06:54:49.297635+0000","last_active":"2024-07-17T06:54:54.566591+0000","last_peered":"2024-07-17T06:54:54.566591+0000","last_clean":"2024-07-17T06:54:54.566591+0000","last_became_active":"2024-07-17T06:54:49.297252+0000","last_became_peered":"2024-07-17T06:54:49.297252+0000","last_unstale":"2024-07-17T06:54:54.566591+0000","last_undegraded":"2024-07-17T06:54:54.566591+0000","last_fullsized":"2024-07-17T06:54:54.566591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:02.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570670+0000","last_change":"2024-07-17T06:54:49.311510+0000","last_active":"2024-07-17T06:54:54.570670+0000","last_peered":"2024-07-17T06:54:54.570670+0000","last_clean":"2024-07-17T06:54:54.570670+0000","last_became_active":"2024-07-17T06:54:49.311208+0000","last_became_peered":"2024-07-17T06:54:49.311208+0000","last_unstale":"2024-07-17T06:54:54.570670+0000","last_undegraded":"2024-07-17T06:54:54.570670+0000","last_fullsized":"2024-07-17T06:54:54.570670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344453+0000","last_change":"2024-07-17T06:54:49.300592+0000","last_active":"2024-07-17T06:54:54.344453+0000","last_peered":"2024-07-17T06:54:54.344453+0000","last_clean":"2024-07-17T06:54:54.344453+0000","last_became_active":"2024-07-17T06:54:49.300233+0000","last_became_peered":"2024-07-17T06:54:49.300233+0000","last_unstale":"2024-07-17T06:54:54.344453+0000","last_undegraded":"2024-07-17T06:54:54.344453+0000","last_fullsized":"2024-07-17T06:54:54.344453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:53.264056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341453+0000","last_change":"2024-07-17T06:54:49.304204+0000","last_active":"2024-07-17T06:54:54.341453+0000","last_peered":"2024-07-17T06:54:54.341453+0000","last_clean":"2024-07-17T06:54:54.341453+0000","last_became_active":"2024-07-17T06:54:49.303907+0000","last_became_peered":"2024-07-17T06:54:49.303907+0000","last_unstale":"2024-07-17T06:54:54.341453+0000","last_undegraded":"2024-07-17T06:54:54.341453+0000","last_fullsized":"2024-07-17T06:54:54.341453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340757+0000","last_change":"2024-07-17T06:54:49.297834+0000","last_active":"2024-07-17T06:54:54.340757+0000","last_peered":"2024-07-17T06:54:54.340757+0000","last_clean":"2024-07-17T06:54:54.340757+0000","last_became_active":"2024-07-17T06:54:49.297296+0000","last_became_peered":"2024-07-17T06:54:49.297296+0000","last_unstale":"2024-07-17T06:54:54.340757+0000","last_undegraded":"2024-07-17T06:54:54.340757+0000","last_fullsized":"2024-07-17T06:54:54.340757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569209+0000","last_change":"2024-07-17T06:54:49.318654+0000","last_active":"2024-07-17T06:54:54.569209+0000","last_peered":"2024-07-17T06:54:54.569209+0000","last_clean":"2024-07-17T06:54:54.569209+0000","last_became_active":"2024-07-17T06:54:49.318380+0000","last_became_peered":"2024-07-17T06:54:49.318380+0000","last_unstale":"2024-07-17T06:54:54.569209+0000","last_undegraded":"2024-07-17T06:54:54.569209+0000","last_fullsized":"2024-07-17T06:54:54.569209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:12.140489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566890+0000","last_change":"2024-07-17T06:54:49.297152+0000","last_active":"2024-07-17T06:54:54.566890+0000","last_peered":"2024-07-17T06:54:54.566890+0000","last_clean":"2024-07-17T06:54:54.566890+0000","last_became_active":"2024-07-17T06:54:49.296830+0000","last_became_peered":"2024-07-17T06:54:49.296830+0000","last_unstale":"2024-07-17T06:54:54.566890+0000","last_undegraded":"2024-07-17T06:54:54.566890+0000","last_fullsized":"2024-07-17T06:54:54.566890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566123+0000","last_change":"2024-07-17T06:54:49.313713+0000","last_active":"2024-07-17T06:54:54.566123+0000","last_peered":"2024-07-17T06:54:54.566123+0000","last_clean":"2024-07-17T06:54:54.566123+0000","last_became_active":"2024-07-17T06:54:49.313419+0000","last_became_peered":"2024-07-17T06:54:49.313419+0000","last_unstale":"2024-07-17T06:54:54.566123+0000","last_undegraded":"2024-07-17T06:54:54.566123+0000","last_fullsized":"2024-07-17T06:54:54.566123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:25.517176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565979+0000","last_change":"2024-07-17T06:54:49.304993+0000","last_active":"2024-07-17T06:54:54.565979+0000","last_peered":"2024-07-17T06:54:54.565979+0000","last_clean":"2024-07-17T06:54:54.565979+0000","last_became_active":"2024-07-17T06:54:49.304475+0000","last_became_peered":"2024-07-17T06:54:49.304475+0000","last_unstale":"2024-07-17T06:54:54.565979+0000","last_undegraded":"2024-07-17T06:54:54.565979+0000","last_fullsized":"2024-07-17T06:54:54.565979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565951+0000","last_change":"2024-07-17T06:54:49.319946+0000","last_active":"2024-07-17T06:54:54.565951+0000","last_peered":"2024-07-17T06:54:54.565951+0000","last_clean":"2024-07-17T06:54:54.565951+0000","last_became_active":"2024-07-17T06:54:49.319685+0000","last_became_peered":"2024-07-17T06:54:49.319685+0000","last_unstale":"2024-07-17T06:54:54.565951+0000","last_undegraded":"2024-07-17T06:54:54.565951+0000","last_fullsized":"2024-07-17T06:54:54.565951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:10.177399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566540+0000","last_change":"2024-07-17T06:54:49.324468+0000","last_active":"2024-07-17T06:54:54.566540+0000","last_peered":"2024-07-17T06:54:54.566540+0000","last_clean":"2024-07-17T06:54:54.566540+0000","last_became_active":"2024-07-17T06:54:49.323700+0000","last_became_peered":"2024-07-17T06:54:49.323700+0000","last_unstale":"2024-07-17T06:54:54.566540+0000","last_undegraded":"2024-07-17T06:54:54.566540+0000","last_fullsized":"2024-07-17T06:54:54.566540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:05.797436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567150+0000","last_change":"2024-07-17T06:54:49.290953+0000","last_active":"2024-07-17T06:54:54.567150+0000","last_peered":"2024-07-17T06:54:54.567150+0000","last_clean":"2024-07-17T06:54:54.567150+0000","last_became_active":"2024-07-17T06:54:49.290069+0000","last_became_peered":"2024-07-17T06:54:49.290069+0000","last_unstale":"2024-07-17T06:54:54.567150+0000","last_undegraded":"2024-07-17T06:54:54.567150+0000","last_fullsized":"2024-07-17T06:54:54.567150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571210+0000","last_change":"2024-07-17T06:54:49.301304+0000","last_active":"2024-07-17T06:54:54.571210+0000","last_peered":"2024-07-17T06:54:54.571210+0000","last_clean":"2024-07-17T06:54:54.571210+0000","last_became_active":"2024-07-17T06:54:49.300865+0000","last_became_peered":"2024-07-17T06:54:49.300865+0000","last_unstale":"2024-07-17T06:54:54.571210+0000","last_undegraded":"2024-07-17T06:54:54.571210+0000","last_fullsized":"2024-07-17T06:54:54.571210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569004+0000","last_change":"2024-07-17T06:54:49.324041+0000","last_active":"2024-07-17T06:54:54.569004+0000","last_peered":"2024-07-17T06:54:54.569004+0000","last_clean":"2024-07-17T06:54:54.569004+0000","last_became_active":"2024-07-17T06:54:49.323747+0000","last_became_peered":"2024-07-17T06:54:49.323747+0000","last_unstale":"2024-07-17T06:54:54.569004+0000","last_undegraded":"2024-07-17T06:54:54.569004+0000","last_fullsized":"2024-07-17T06:54:54.569004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:04:46.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":101953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":692869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":692813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":101989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":101863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:55:08.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:55:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 -- 172.21.15.44:0/2363402494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 msgr2=0x7f23c80ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 --2- 172.21.15.44:0/2363402494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c80ac220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23c8070bf0 tx=0x7f23b402a540 comp rx=0 tx=0).stop 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 -- 172.21.15.44:0/2363402494 shutdown_connections 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 --2- 172.21.15.44:0/2363402494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c80ac220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 --2- 172.21.15.44:0/2363402494 >> v2:172.21.15.175:3300/0 conn(0x7f23c81522c0 0x7f23c80ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 -- 172.21.15.44:0/2363402494 >> 172.21.15.44:0/2363402494 conn(0x7f23c80b0fb0 msgr2=0x7f23c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 -- 172.21.15.44:0/2363402494 shutdown_connections 2024-07-17T06:55:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.122+0000 7f23cd200640 1 -- 172.21.15.44:0/2363402494 wait complete. 2024-07-17T06:55:09.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.124+0000 7f23cd200640 1 Processor -- start 2024-07-17T06:55:09.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.124+0000 7f23cd200640 1 -- start start 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c80ac760 0x7f23c81f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c81522c0 0x7f23c8073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80ce820 con 0x7f23c80adcd0 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23c80ce6a0 con 0x7f23c80ac760 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.125+0000 7f23cd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23c80ce9a0 con 0x7f23c81522c0 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37496/0 (socket says 172.21.15.44:37496) 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 -- 172.21.15.44:0/1430606007 learned_addr learned my addr 172.21.15.44:0/1430606007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5e00640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c80ac760 0x7f23c81f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c6800640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c81522c0 0x7f23c8073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 -- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c81522c0 msgr2=0x7f23c8073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c81522c0 0x7f23c8073610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 -- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c80ac760 msgr2=0x7f23c81f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c80ac760 0x7f23c81f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.126+0000 7f23c5400640 1 -- 172.21.15.44:0/1430606007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c81f4030 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23c5400640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009920 tx=0x7f23b4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4040070 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23b4005310 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40047a0 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c81f42d0 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.127+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c81f4890 con 0x7f23c80adcd0 2024-07-17T06:55:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.128+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c80ac220 con 0x7f23c80adcd0 2024-07-17T06:55:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.130+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f23b4039040 con 0x7f23c80adcd0 2024-07-17T06:55:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.131+0000 7f23be000640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 0x7f23a8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.131+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f23b40b97c0 con 0x7f23c80adcd0 2024-07-17T06:55:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.131+0000 7f23c5e00640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 0x7f23a8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.132+0000 7f23c5e00640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 0x7f23a8079070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23b8005fd0 tx=0x7f23b8005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:09.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.135+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f23b40828b0 con 0x7f23c80adcd0 2024-07-17T06:55:09.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.332+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23c80b23b0 con 0x7f23a8076bb0 2024-07-17T06:55:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.347+0000 7f23be000640 1 -- 172.21.15.44:0/1430606007 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332559 (secure 0 0 0) 0x7f23b800fad0 con 0x7f23a8076bb0 2024-07-17T06:55:09.351 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:09.355 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 msgr2=0x7f23a8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 0x7f23a8079070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23b8005fd0 tx=0x7f23b8005850 comp rx=0 tx=0).stop 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 msgr2=0x7f23c8071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009920 tx=0x7f23b4032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 shutdown_connections 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f23a8076bb0 0x7f23a8079070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23c81522c0 0x7f23c8073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c80adcd0 0x7f23c8071890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 --2- 172.21.15.44:0/1430606007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23c80ac760 0x7f23c81f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 >> 172.21.15.44:0/1430606007 conn(0x7f23c80b0fb0 msgr2=0x7f23c80ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 shutdown_connections 2024-07-17T06:55:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.358+0000 7f23cd200640 1 -- 172.21.15.44:0/1430606007 wait complete. 2024-07-17T06:55:09.376 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-17T06:55:07.716449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219448,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755272,"statfs":{"total":773094113280,"available":772869398528,"internally_reserved":0,"allocated":3317760,"data_stored":2001358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005148"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569050+0000","last_change":"2024-07-17T06:54:48.289026+0000","last_active":"2024-07-17T06:54:54.569050+0000","last_peered":"2024-07-17T06:54:54.569050+0000","last_clean":"2024-07-17T06:54:54.569050+0000","last_became_active":"2024-07-17T06:54:48.288767+0000","last_became_peered":"2024-07-17T06:54:48.288767+0000","last_unstale":"2024-07-17T06:54:54.569050+0000","last_undegraded":"2024-07-17T06:54:54.569050+0000","last_fullsized":"2024-07-17T06:54:54.569050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572074+0000","last_change":"2024-07-17T06:54:49.320600+0000","last_active":"2024-07-17T06:54:54.572074+0000","last_peered":"2024-07-17T06:54:54.572074+0000","last_clean":"2024-07-17T06:54:54.572074+0000","last_became_active":"2024-07-17T06:54:49.319878+0000","last_became_peered":"2024-07-17T06:54:49.319878+0000","last_unstale":"2024-07-17T06:54:54.572074+0000","last_undegraded":"2024-07-17T06:54:54.572074+0000","last_fullsized":"2024-07-17T06:54:54.572074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569764+0000","last_change":"2024-07-17T06:54:53.670763+0000","last_active":"2024-07-17T06:54:54.569764+0000","last_peered":"2024-07-17T06:54:54.569764+0000","last_clean":"2024-07-17T06:54:54.569764+0000","last_became_active":"2024-07-17T06:54:53.670345+0000","last_became_peered":"2024-07-17T06:54:53.670345+0000","last_unstale":"2024-07-17T06:54:54.569764+0000","last_undegraded":"2024-07-17T06:54:54.569764+0000","last_fullsized":"2024-07-17T06:54:54.569764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:19.165767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572200+0000","last_change":"2024-07-17T06:54:48.267830+0000","last_active":"2024-07-17T06:54:54.572200+0000","last_peered":"2024-07-17T06:54:54.572200+0000","last_clean":"2024-07-17T06:54:54.572200+0000","last_became_active":"2024-07-17T06:54:48.267458+0000","last_became_peered":"2024-07-17T06:54:48.267458+0000","last_unstale":"2024-07-17T06:54:54.572200+0000","last_undegraded":"2024-07-17T06:54:54.572200+0000","last_fullsized":"2024-07-17T06:54:54.572200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566732+0000","last_change":"2024-07-17T06:54:49.313799+0000","last_active":"2024-07-17T06:54:54.566732+0000","last_peered":"2024-07-17T06:54:54.566732+0000","last_clean":"2024-07-17T06:54:54.566732+0000","last_became_active":"2024-07-17T06:54:49.313240+0000","last_became_peered":"2024-07-17T06:54:49.313240+0000","last_unstale":"2024-07-17T06:54:54.566732+0000","last_undegraded":"2024-07-17T06:54:54.566732+0000","last_fullsized":"2024-07-17T06:54:54.566732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340034+0000","last_change":"2024-07-17T06:54:53.356992+0000","last_active":"2024-07-17T06:54:54.340034+0000","last_peered":"2024-07-17T06:54:54.340034+0000","last_clean":"2024-07-17T06:54:54.340034+0000","last_became_active":"2024-07-17T06:54:53.356521+0000","last_became_peered":"2024-07-17T06:54:53.356521+0000","last_unstale":"2024-07-17T06:54:54.340034+0000","last_undegraded":"2024-07-17T06:54:54.340034+0000","last_fullsized":"2024-07-17T06:54:54.340034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:33.058963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569075+0000","last_change":"2024-07-17T06:54:48.285421+0000","last_active":"2024-07-17T06:54:54.569075+0000","last_peered":"2024-07-17T06:54:54.569075+0000","last_clean":"2024-07-17T06:54:54.569075+0000","last_became_active":"2024-07-17T06:54:48.285132+0000","last_became_peered":"2024-07-17T06:54:48.285132+0000","last_unstale":"2024-07-17T06:54:54.569075+0000","last_undegraded":"2024-07-17T06:54:54.569075+0000","last_fullsized":"2024-07-17T06:54:54.569075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:38.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566825+0000","last_change":"2024-07-17T06:54:49.311977+0000","last_active":"2024-07-17T06:54:54.566825+0000","last_peered":"2024-07-17T06:54:54.566825+0000","last_clean":"2024-07-17T06:54:54.566825+0000","last_became_active":"2024-07-17T06:54:49.311364+0000","last_became_peered":"2024-07-17T06:54:49.311364+0000","last_unstale":"2024-07-17T06:54:54.566825+0000","last_undegraded":"2024-07-17T06:54:54.566825+0000","last_fullsized":"2024-07-17T06:54:54.566825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:21.034101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341182+0000","last_change":"2024-07-17T06:54:53.362264+0000","last_active":"2024-07-17T06:54:54.341182+0000","last_peered":"2024-07-17T06:54:54.341182+0000","last_clean":"2024-07-17T06:54:54.341182+0000","last_became_active":"2024-07-17T06:54:53.361863+0000","last_became_peered":"2024-07-17T06:54:53.361863+0000","last_unstale":"2024-07-17T06:54:54.341182+0000","last_undegraded":"2024-07-17T06:54:54.341182+0000","last_fullsized":"2024-07-17T06:54:54.341182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:22.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567855+0000","last_change":"2024-07-17T06:54:48.288847+0000","last_active":"2024-07-17T06:54:54.567855+0000","last_peered":"2024-07-17T06:54:54.567855+0000","last_clean":"2024-07-17T06:54:54.567855+0000","last_became_active":"2024-07-17T06:54:48.288504+0000","last_became_peered":"2024-07-17T06:54:48.288504+0000","last_unstale":"2024-07-17T06:54:54.567855+0000","last_undegraded":"2024-07-17T06:54:54.567855+0000","last_fullsized":"2024-07-17T06:54:54.567855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:42.640200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567324+0000","last_change":"2024-07-17T06:54:49.311645+0000","last_active":"2024-07-17T06:54:54.567324+0000","last_peered":"2024-07-17T06:54:54.567324+0000","last_clean":"2024-07-17T06:54:54.567324+0000","last_became_active":"2024-07-17T06:54:49.311336+0000","last_became_peered":"2024-07-17T06:54:49.311336+0000","last_unstale":"2024-07-17T06:54:54.567324+0000","last_undegraded":"2024-07-17T06:54:54.567324+0000","last_fullsized":"2024-07-17T06:54:54.567324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342111+0000","last_change":"2024-07-17T06:54:53.357717+0000","last_active":"2024-07-17T06:54:54.342111+0000","last_peered":"2024-07-17T06:54:54.342111+0000","last_clean":"2024-07-17T06:54:54.342111+0000","last_became_active":"2024-07-17T06:54:53.356799+0000","last_became_peered":"2024-07-17T06:54:53.356799+0000","last_unstale":"2024-07-17T06:54:54.342111+0000","last_undegraded":"2024-07-17T06:54:54.342111+0000","last_fullsized":"2024-07-17T06:54:54.342111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:29.584528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341883+0000","last_change":"2024-07-17T06:54:49.299990+0000","last_active":"2024-07-17T06:54:54.341883+0000","last_peered":"2024-07-17T06:54:54.341883+0000","last_clean":"2024-07-17T06:54:54.341883+0000","last_became_active":"2024-07-17T06:54:49.299679+0000","last_became_peered":"2024-07-17T06:54:49.299679+0000","last_unstale":"2024-07-17T06:54:54.341883+0000","last_undegraded":"2024-07-17T06:54:54.341883+0000","last_fullsized":"2024-07-17T06:54:54.341883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568387+0000","last_change":"2024-07-17T06:54:48.285371+0000","last_active":"2024-07-17T06:54:54.568387+0000","last_peered":"2024-07-17T06:54:54.568387+0000","last_clean":"2024-07-17T06:54:54.568387+0000","last_became_active":"2024-07-17T06:54:48.284925+0000","last_became_peered":"2024-07-17T06:54:48.284925+0000","last_unstale":"2024-07-17T06:54:54.568387+0000","last_undegraded":"2024-07-17T06:54:54.568387+0000","last_fullsized":"2024-07-17T06:54:54.568387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:21.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344985+0000","last_change":"2024-07-17T06:54:53.353487+0000","last_active":"2024-07-17T06:54:54.344985+0000","last_peered":"2024-07-17T06:54:54.344985+0000","last_clean":"2024-07-17T06:54:54.344985+0000","last_became_active":"2024-07-17T06:54:53.353037+0000","last_became_peered":"2024-07-17T06:54:53.353037+0000","last_unstale":"2024-07-17T06:54:54.344985+0000","last_undegraded":"2024-07-17T06:54:54.344985+0000","last_fullsized":"2024-07-17T06:54:54.344985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:51.743135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570785+0000","last_change":"2024-07-17T06:54:49.300898+0000","last_active":"2024-07-17T06:54:54.570785+0000","last_peered":"2024-07-17T06:54:54.570785+0000","last_clean":"2024-07-17T06:54:54.570785+0000","last_became_active":"2024-07-17T06:54:49.300631+0000","last_became_peered":"2024-07-17T06:54:49.300631+0000","last_unstale":"2024-07-17T06:54:54.570785+0000","last_undegraded":"2024-07-17T06:54:54.570785+0000","last_fullsized":"2024-07-17T06:54:54.570785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568669+0000","last_change":"2024-07-17T06:54:48.274040+0000","last_active":"2024-07-17T06:54:54.568669+0000","last_peered":"2024-07-17T06:54:54.568669+0000","last_clean":"2024-07-17T06:54:54.568669+0000","last_became_active":"2024-07-17T06:54:48.272837+0000","last_became_peered":"2024-07-17T06:54:48.272837+0000","last_unstale":"2024-07-17T06:54:54.568669+0000","last_undegraded":"2024-07-17T06:54:54.568669+0000","last_fullsized":"2024-07-17T06:54:54.568669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344238+0000","last_change":"2024-07-17T06:54:53.356025+0000","last_active":"2024-07-17T06:54:54.344238+0000","last_peered":"2024-07-17T06:54:54.344238+0000","last_clean":"2024-07-17T06:54:54.344238+0000","last_became_active":"2024-07-17T06:54:53.355539+0000","last_became_peered":"2024-07-17T06:54:53.355539+0000","last_unstale":"2024-07-17T06:54:54.344238+0000","last_undegraded":"2024-07-17T06:54:54.344238+0000","last_fullsized":"2024-07-17T06:54:54.344238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:14.244752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567230+0000","last_change":"2024-07-17T06:54:49.303989+0000","last_active":"2024-07-17T06:54:54.567230+0000","last_peered":"2024-07-17T06:54:54.567230+0000","last_clean":"2024-07-17T06:54:54.567230+0000","last_became_active":"2024-07-17T06:54:49.303723+0000","last_became_peered":"2024-07-17T06:54:49.303723+0000","last_unstale":"2024-07-17T06:54:54.567230+0000","last_undegraded":"2024-07-17T06:54:54.567230+0000","last_fullsized":"2024-07-17T06:54:54.567230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568506+0000","last_change":"2024-07-17T06:54:48.280459+0000","last_active":"2024-07-17T06:54:54.568506+0000","last_peered":"2024-07-17T06:54:54.568506+0000","last_clean":"2024-07-17T06:54:54.568506+0000","last_became_active":"2024-07-17T06:54:48.280196+0000","last_became_peered":"2024-07-17T06:54:48.280196+0000","last_unstale":"2024-07-17T06:54:54.568506+0000","last_undegraded":"2024-07-17T06:54:54.568506+0000","last_fullsized":"2024-07-17T06:54:54.568506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571786+0000","last_change":"2024-07-17T06:54:53.672128+0000","last_active":"2024-07-17T06:54:54.571786+0000","last_peered":"2024-07-17T06:54:54.571786+0000","last_clean":"2024-07-17T06:54:54.571786+0000","last_became_active":"2024-07-17T06:54:53.671083+0000","last_became_peered":"2024-07-17T06:54:53.671083+0000","last_unstale":"2024-07-17T06:54:54.571786+0000","last_undegraded":"2024-07-17T06:54:54.571786+0000","last_fullsized":"2024-07-17T06:54:54.571786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341192+0000","last_change":"2024-07-17T06:54:49.301897+0000","last_active":"2024-07-17T06:54:54.341192+0000","last_peered":"2024-07-17T06:54:54.341192+0000","last_clean":"2024-07-17T06:54:54.341192+0000","last_became_active":"2024-07-17T06:54:49.301602+0000","last_became_peered":"2024-07-17T06:54:49.301602+0000","last_unstale":"2024-07-17T06:54:54.341192+0000","last_undegraded":"2024-07-17T06:54:54.341192+0000","last_fullsized":"2024-07-17T06:54:54.341192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:39.268743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568075+0000","last_change":"2024-07-17T06:54:48.280905+0000","last_active":"2024-07-17T06:54:54.568075+0000","last_peered":"2024-07-17T06:54:54.568075+0000","last_clean":"2024-07-17T06:54:54.568075+0000","last_became_active":"2024-07-17T06:54:48.280538+0000","last_became_peered":"2024-07-17T06:54:48.280538+0000","last_unstale":"2024-07-17T06:54:54.568075+0000","last_undegraded":"2024-07-17T06:54:54.568075+0000","last_fullsized":"2024-07-17T06:54:54.568075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343510+0000","last_change":"2024-07-17T06:54:53.356455+0000","last_active":"2024-07-17T06:54:54.343510+0000","last_peered":"2024-07-17T06:54:54.343510+0000","last_clean":"2024-07-17T06:54:54.343510+0000","last_became_active":"2024-07-17T06:54:53.355594+0000","last_became_peered":"2024-07-17T06:54:53.355594+0000","last_unstale":"2024-07-17T06:54:54.343510+0000","last_undegraded":"2024-07-17T06:54:54.343510+0000","last_fullsized":"2024-07-17T06:54:54.343510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:37.799189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569678+0000","last_change":"2024-07-17T06:54:49.306398+0000","last_active":"2024-07-17T06:54:54.569678+0000","last_peered":"2024-07-17T06:54:54.569678+0000","last_clean":"2024-07-17T06:54:54.569678+0000","last_became_active":"2024-07-17T06:54:49.305914+0000","last_became_peered":"2024-07-17T06:54:49.305914+0000","last_unstale":"2024-07-17T06:54:54.569678+0000","last_undegraded":"2024-07-17T06:54:54.569678+0000","last_fullsized":"2024-07-17T06:54:54.569678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343534+0000","last_change":"2024-07-17T06:54:48.281630+0000","last_active":"2024-07-17T06:54:54.343534+0000","last_peered":"2024-07-17T06:54:54.343534+0000","last_clean":"2024-07-17T06:54:54.343534+0000","last_became_active":"2024-07-17T06:54:48.281368+0000","last_became_peered":"2024-07-17T06:54:48.281368+0000","last_unstale":"2024-07-17T06:54:54.343534+0000","last_undegraded":"2024-07-17T06:54:54.343534+0000","last_fullsized":"2024-07-17T06:54:54.343534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:09.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340594+0000","last_change":"2024-07-17T06:54:53.685324+0000","last_active":"2024-07-17T06:54:54.340594+0000","last_peered":"2024-07-17T06:54:54.340594+0000","last_clean":"2024-07-17T06:54:54.340594+0000","last_became_active":"2024-07-17T06:54:53.683906+0000","last_became_peered":"2024-07-17T06:54:53.683906+0000","last_unstale":"2024-07-17T06:54:54.340594+0000","last_undegraded":"2024-07-17T06:54:54.340594+0000","last_fullsized":"2024-07-17T06:54:54.340594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:31.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572498+0000","last_change":"2024-07-17T06:54:49.307258+0000","last_active":"2024-07-17T06:54:54.572498+0000","last_peered":"2024-07-17T06:54:54.572498+0000","last_clean":"2024-07-17T06:54:54.572498+0000","last_became_active":"2024-07-17T06:54:49.306989+0000","last_became_peered":"2024-07-17T06:54:49.306989+0000","last_unstale":"2024-07-17T06:54:54.572498+0000","last_undegraded":"2024-07-17T06:54:54.572498+0000","last_fullsized":"2024-07-17T06:54:54.572498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571324+0000","last_change":"2024-07-17T06:54:48.273115+0000","last_active":"2024-07-17T06:54:54.571324+0000","last_peered":"2024-07-17T06:54:54.571324+0000","last_clean":"2024-07-17T06:54:54.571324+0000","last_became_active":"2024-07-17T06:54:48.272804+0000","last_became_peered":"2024-07-17T06:54:48.272804+0000","last_unstale":"2024-07-17T06:54:54.571324+0000","last_undegraded":"2024-07-17T06:54:54.571324+0000","last_fullsized":"2024-07-17T06:54:54.571324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566623+0000","last_change":"2024-07-17T06:54:53.667515+0000","last_active":"2024-07-17T06:54:54.566623+0000","last_peered":"2024-07-17T06:54:54.566623+0000","last_clean":"2024-07-17T06:54:54.566623+0000","last_became_active":"2024-07-17T06:54:53.665920+0000","last_became_peered":"2024-07-17T06:54:53.665920+0000","last_unstale":"2024-07-17T06:54:54.566623+0000","last_undegraded":"2024-07-17T06:54:54.566623+0000","last_fullsized":"2024-07-17T06:54:54.566623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:20.348618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569843+0000","last_change":"2024-07-17T06:54:49.296691+0000","last_active":"2024-07-17T06:54:54.569843+0000","last_peered":"2024-07-17T06:54:54.569843+0000","last_clean":"2024-07-17T06:54:54.569843+0000","last_became_active":"2024-07-17T06:54:49.296340+0000","last_became_peered":"2024-07-17T06:54:49.296340+0000","last_unstale":"2024-07-17T06:54:54.569843+0000","last_undegraded":"2024-07-17T06:54:54.569843+0000","last_fullsized":"2024-07-17T06:54:54.569843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569522+0000","last_change":"2024-07-17T06:54:48.288182+0000","last_active":"2024-07-17T06:54:54.569522+0000","last_peered":"2024-07-17T06:54:54.569522+0000","last_clean":"2024-07-17T06:54:54.569522+0000","last_became_active":"2024-07-17T06:54:48.287723+0000","last_became_peered":"2024-07-17T06:54:48.287723+0000","last_unstale":"2024-07-17T06:54:54.569522+0000","last_undegraded":"2024-07-17T06:54:54.569522+0000","last_fullsized":"2024-07-17T06:54:54.569522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:54.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565754+0000","last_change":"2024-07-17T06:54:53.672017+0000","last_active":"2024-07-17T06:54:54.565754+0000","last_peered":"2024-07-17T06:54:54.565754+0000","last_clean":"2024-07-17T06:54:54.565754+0000","last_became_active":"2024-07-17T06:54:53.671036+0000","last_became_peered":"2024-07-17T06:54:53.671036+0000","last_unstale":"2024-07-17T06:54:54.565754+0000","last_undegraded":"2024-07-17T06:54:54.565754+0000","last_fullsized":"2024-07-17T06:54:54.565754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567594+0000","last_change":"2024-07-17T06:54:49.320403+0000","last_active":"2024-07-17T06:54:54.567594+0000","last_peered":"2024-07-17T06:54:54.567594+0000","last_clean":"2024-07-17T06:54:54.567594+0000","last_became_active":"2024-07-17T06:54:49.320119+0000","last_became_peered":"2024-07-17T06:54:49.320119+0000","last_unstale":"2024-07-17T06:54:54.567594+0000","last_undegraded":"2024-07-17T06:54:54.567594+0000","last_fullsized":"2024-07-17T06:54:54.567594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:41.108822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569854+0000","last_change":"2024-07-17T06:54:48.271506+0000","last_active":"2024-07-17T06:54:54.569854+0000","last_peered":"2024-07-17T06:54:54.569854+0000","last_clean":"2024-07-17T06:54:54.569854+0000","last_became_active":"2024-07-17T06:54:48.271181+0000","last_became_peered":"2024-07-17T06:54:48.271181+0000","last_unstale":"2024-07-17T06:54:54.569854+0000","last_undegraded":"2024-07-17T06:54:54.569854+0000","last_fullsized":"2024-07-17T06:54:54.569854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568214+0000","last_change":"2024-07-17T06:54:53.358600+0000","last_active":"2024-07-17T06:54:54.568214+0000","last_peered":"2024-07-17T06:54:54.568214+0000","last_clean":"2024-07-17T06:54:54.568214+0000","last_became_active":"2024-07-17T06:54:53.355359+0000","last_became_peered":"2024-07-17T06:54:53.355359+0000","last_unstale":"2024-07-17T06:54:54.568214+0000","last_undegraded":"2024-07-17T06:54:54.568214+0000","last_fullsized":"2024-07-17T06:54:54.568214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344036+0000","last_change":"2024-07-17T06:54:49.305202+0000","last_active":"2024-07-17T06:54:54.344036+0000","last_peered":"2024-07-17T06:54:54.344036+0000","last_clean":"2024-07-17T06:54:54.344036+0000","last_became_active":"2024-07-17T06:54:49.304856+0000","last_became_peered":"2024-07-17T06:54:49.304856+0000","last_unstale":"2024-07-17T06:54:54.344036+0000","last_undegraded":"2024-07-17T06:54:54.344036+0000","last_fullsized":"2024-07-17T06:54:54.344036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:00.298763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566783+0000","last_change":"2024-07-17T06:54:48.279231+0000","last_active":"2024-07-17T06:54:54.566783+0000","last_peered":"2024-07-17T06:54:54.566783+0000","last_clean":"2024-07-17T06:54:54.566783+0000","last_became_active":"2024-07-17T06:54:48.277422+0000","last_became_peered":"2024-07-17T06:54:48.277422+0000","last_unstale":"2024-07-17T06:54:54.566783+0000","last_undegraded":"2024-07-17T06:54:54.566783+0000","last_fullsized":"2024-07-17T06:54:54.566783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569758+0000","last_change":"2024-07-17T06:54:53.350972+0000","last_active":"2024-07-17T06:54:54.569758+0000","last_peered":"2024-07-17T06:54:54.569758+0000","last_clean":"2024-07-17T06:54:54.569758+0000","last_became_active":"2024-07-17T06:54:53.347571+0000","last_became_peered":"2024-07-17T06:54:53.347571+0000","last_unstale":"2024-07-17T06:54:54.569758+0000","last_undegraded":"2024-07-17T06:54:54.569758+0000","last_fullsized":"2024-07-17T06:54:54.569758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567608+0000","last_change":"2024-07-17T06:54:49.302823+0000","last_active":"2024-07-17T06:54:54.567608+0000","last_peered":"2024-07-17T06:54:54.567608+0000","last_clean":"2024-07-17T06:54:54.567608+0000","last_became_active":"2024-07-17T06:54:49.302434+0000","last_became_peered":"2024-07-17T06:54:49.302434+0000","last_unstale":"2024-07-17T06:54:54.567608+0000","last_undegraded":"2024-07-17T06:54:54.567608+0000","last_fullsized":"2024-07-17T06:54:54.567608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339350+0000","last_change":"2024-07-17T06:54:48.268185+0000","last_active":"2024-07-17T06:54:54.339350+0000","last_peered":"2024-07-17T06:54:54.339350+0000","last_clean":"2024-07-17T06:54:54.339350+0000","last_became_active":"2024-07-17T06:54:48.267807+0000","last_became_peered":"2024-07-17T06:54:48.267807+0000","last_unstale":"2024-07-17T06:54:54.339350+0000","last_undegraded":"2024-07-17T06:54:54.339350+0000","last_fullsized":"2024-07-17T06:54:54.339350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:28.391194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572146+0000","last_change":"2024-07-17T06:54:53.345628+0000","last_active":"2024-07-17T06:54:54.572146+0000","last_peered":"2024-07-17T06:54:54.572146+0000","last_clean":"2024-07-17T06:54:54.572146+0000","last_became_active":"2024-07-17T06:54:53.345269+0000","last_became_peered":"2024-07-17T06:54:53.345269+0000","last_unstale":"2024-07-17T06:54:54.572146+0000","last_undegraded":"2024-07-17T06:54:54.572146+0000","last_fullsized":"2024-07-17T06:54:54.572146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565847+0000","last_change":"2024-07-17T06:54:49.302929+0000","last_active":"2024-07-17T06:54:54.565847+0000","last_peered":"2024-07-17T06:54:54.565847+0000","last_clean":"2024-07-17T06:54:54.565847+0000","last_became_active":"2024-07-17T06:54:49.302598+0000","last_became_peered":"2024-07-17T06:54:49.302598+0000","last_unstale":"2024-07-17T06:54:54.565847+0000","last_undegraded":"2024-07-17T06:54:54.565847+0000","last_fullsized":"2024-07-17T06:54:54.565847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568955+0000","last_change":"2024-07-17T06:54:48.284164+0000","last_active":"2024-07-17T06:54:54.568955+0000","last_peered":"2024-07-17T06:54:54.568955+0000","last_clean":"2024-07-17T06:54:54.568955+0000","last_became_active":"2024-07-17T06:54:48.282885+0000","last_became_peered":"2024-07-17T06:54:48.282885+0000","last_unstale":"2024-07-17T06:54:54.568955+0000","last_undegraded":"2024-07-17T06:54:54.568955+0000","last_fullsized":"2024-07-17T06:54:54.568955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:05.674177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569179+0000","last_change":"2024-07-17T06:54:53.677311+0000","last_active":"2024-07-17T06:54:54.569179+0000","last_peered":"2024-07-17T06:54:54.569179+0000","last_clean":"2024-07-17T06:54:54.569179+0000","last_became_active":"2024-07-17T06:54:53.670958+0000","last_became_peered":"2024-07-17T06:54:53.670958+0000","last_unstale":"2024-07-17T06:54:54.569179+0000","last_undegraded":"2024-07-17T06:54:54.569179+0000","last_fullsized":"2024-07-17T06:54:54.569179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566963+0000","last_change":"2024-07-17T06:54:49.312042+0000","last_active":"2024-07-17T06:54:54.566963+0000","last_peered":"2024-07-17T06:54:54.566963+0000","last_clean":"2024-07-17T06:54:54.566963+0000","last_became_active":"2024-07-17T06:54:49.311741+0000","last_became_peered":"2024-07-17T06:54:49.311741+0000","last_unstale":"2024-07-17T06:54:54.566963+0000","last_undegraded":"2024-07-17T06:54:54.566963+0000","last_fullsized":"2024-07-17T06:54:54.566963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567568+0000","last_change":"2024-07-17T06:54:48.277447+0000","last_active":"2024-07-17T06:54:54.567568+0000","last_peered":"2024-07-17T06:54:54.567568+0000","last_clean":"2024-07-17T06:54:54.567568+0000","last_became_active":"2024-07-17T06:54:48.273874+0000","last_became_peered":"2024-07-17T06:54:48.273874+0000","last_unstale":"2024-07-17T06:54:54.567568+0000","last_undegraded":"2024-07-17T06:54:54.567568+0000","last_fullsized":"2024-07-17T06:54:54.567568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568469+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:54:54.568469+0000","last_peered":"2024-07-17T06:54:54.568469+0000","last_clean":"2024-07-17T06:54:54.568469+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:54:54.568469+0000","last_undegraded":"2024-07-17T06:54:54.568469+0000","last_fullsized":"2024-07-17T06:54:54.568469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340950+0000","last_change":"2024-07-17T06:54:53.680957+0000","last_active":"2024-07-17T06:54:54.340950+0000","last_peered":"2024-07-17T06:54:54.340950+0000","last_clean":"2024-07-17T06:54:54.340950+0000","last_became_active":"2024-07-17T06:54:53.679463+0000","last_became_peered":"2024-07-17T06:54:53.679463+0000","last_unstale":"2024-07-17T06:54:54.340950+0000","last_undegraded":"2024-07-17T06:54:54.340950+0000","last_fullsized":"2024-07-17T06:54:54.340950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568688+0000","last_change":"2024-07-17T06:54:49.313201+0000","last_active":"2024-07-17T06:54:54.568688+0000","last_peered":"2024-07-17T06:54:54.568688+0000","last_clean":"2024-07-17T06:54:54.568688+0000","last_became_active":"2024-07-17T06:54:49.312937+0000","last_became_peered":"2024-07-17T06:54:49.312937+0000","last_unstale":"2024-07-17T06:54:54.568688+0000","last_undegraded":"2024-07-17T06:54:54.568688+0000","last_fullsized":"2024-07-17T06:54:54.568688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:16.869857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339687+0000","last_change":"2024-07-17T06:54:48.276023+0000","last_active":"2024-07-17T06:54:54.339687+0000","last_peered":"2024-07-17T06:54:54.339687+0000","last_clean":"2024-07-17T06:54:54.339687+0000","last_became_active":"2024-07-17T06:54:48.275283+0000","last_became_peered":"2024-07-17T06:54:48.275283+0000","last_unstale":"2024-07-17T06:54:54.339687+0000","last_undegraded":"2024-07-17T06:54:54.339687+0000","last_fullsized":"2024-07-17T06:54:54.339687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:52.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341589+0000","last_change":"2024-07-17T06:54:53.674509+0000","last_active":"2024-07-17T06:54:54.341589+0000","last_peered":"2024-07-17T06:54:54.341589+0000","last_clean":"2024-07-17T06:54:54.341589+0000","last_became_active":"2024-07-17T06:54:53.672806+0000","last_became_peered":"2024-07-17T06:54:53.672806+0000","last_unstale":"2024-07-17T06:54:54.341589+0000","last_undegraded":"2024-07-17T06:54:54.341589+0000","last_fullsized":"2024-07-17T06:54:54.341589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567064+0000","last_change":"2024-07-17T06:54:49.324849+0000","last_active":"2024-07-17T06:54:54.567064+0000","last_peered":"2024-07-17T06:54:54.567064+0000","last_clean":"2024-07-17T06:54:54.567064+0000","last_became_active":"2024-07-17T06:54:49.324195+0000","last_became_peered":"2024-07-17T06:54:49.324195+0000","last_unstale":"2024-07-17T06:54:54.567064+0000","last_undegraded":"2024-07-17T06:54:54.567064+0000","last_fullsized":"2024-07-17T06:54:54.567064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:45.753022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570696+0000","last_change":"2024-07-17T06:54:48.276038+0000","last_active":"2024-07-17T06:54:54.570696+0000","last_peered":"2024-07-17T06:54:54.570696+0000","last_clean":"2024-07-17T06:54:54.570696+0000","last_became_active":"2024-07-17T06:54:48.275729+0000","last_became_peered":"2024-07-17T06:54:48.275729+0000","last_unstale":"2024-07-17T06:54:54.570696+0000","last_undegraded":"2024-07-17T06:54:54.570696+0000","last_fullsized":"2024-07-17T06:54:54.570696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565953+0000","last_change":"2024-07-17T06:54:53.356746+0000","last_active":"2024-07-17T06:54:54.565953+0000","last_peered":"2024-07-17T06:54:54.565953+0000","last_clean":"2024-07-17T06:54:54.565953+0000","last_became_active":"2024-07-17T06:54:53.353358+0000","last_became_peered":"2024-07-17T06:54:53.353358+0000","last_unstale":"2024-07-17T06:54:54.565953+0000","last_undegraded":"2024-07-17T06:54:54.565953+0000","last_fullsized":"2024-07-17T06:54:54.565953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569283+0000","last_change":"2024-07-17T06:54:49.318930+0000","last_active":"2024-07-17T06:54:54.569283+0000","last_peered":"2024-07-17T06:54:54.569283+0000","last_clean":"2024-07-17T06:54:54.569283+0000","last_became_active":"2024-07-17T06:54:49.318606+0000","last_became_peered":"2024-07-17T06:54:49.318606+0000","last_unstale":"2024-07-17T06:54:54.569283+0000","last_undegraded":"2024-07-17T06:54:54.569283+0000","last_fullsized":"2024-07-17T06:54:54.569283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:10:28.608769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339970+0000","last_change":"2024-07-17T06:54:48.269152+0000","last_active":"2024-07-17T06:54:54.339970+0000","last_peered":"2024-07-17T06:54:54.339970+0000","last_clean":"2024-07-17T06:54:54.339970+0000","last_became_active":"2024-07-17T06:54:48.268914+0000","last_became_peered":"2024-07-17T06:54:48.268914+0000","last_unstale":"2024-07-17T06:54:54.339970+0000","last_undegraded":"2024-07-17T06:54:54.339970+0000","last_fullsized":"2024-07-17T06:54:54.339970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565453+0000","last_change":"2024-07-17T06:54:53.672373+0000","last_active":"2024-07-17T06:54:54.565453+0000","last_peered":"2024-07-17T06:54:54.565453+0000","last_clean":"2024-07-17T06:54:54.565453+0000","last_became_active":"2024-07-17T06:54:53.671288+0000","last_became_peered":"2024-07-17T06:54:53.671288+0000","last_unstale":"2024-07-17T06:54:54.565453+0000","last_undegraded":"2024-07-17T06:54:54.565453+0000","last_fullsized":"2024-07-17T06:54:54.565453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568889+0000","last_change":"2024-07-17T06:54:49.320151+0000","last_active":"2024-07-17T06:54:54.568889+0000","last_peered":"2024-07-17T06:54:54.568889+0000","last_clean":"2024-07-17T06:54:54.568889+0000","last_became_active":"2024-07-17T06:54:49.319845+0000","last_became_peered":"2024-07-17T06:54:49.319845+0000","last_unstale":"2024-07-17T06:54:54.568889+0000","last_undegraded":"2024-07-17T06:54:54.568889+0000","last_fullsized":"2024-07-17T06:54:54.568889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:12:25.836163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343920+0000","last_change":"2024-07-17T06:54:48.268964+0000","last_active":"2024-07-17T06:54:54.343920+0000","last_peered":"2024-07-17T06:54:54.343920+0000","last_clean":"2024-07-17T06:54:54.343920+0000","last_became_active":"2024-07-17T06:54:48.268564+0000","last_became_peered":"2024-07-17T06:54:48.268564+0000","last_unstale":"2024-07-17T06:54:54.343920+0000","last_undegraded":"2024-07-17T06:54:54.343920+0000","last_fullsized":"2024-07-17T06:54:54.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:47.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340002+0000","last_change":"2024-07-17T06:54:53.362347+0000","last_active":"2024-07-17T06:54:54.340002+0000","last_peered":"2024-07-17T06:54:54.340002+0000","last_clean":"2024-07-17T06:54:54.340002+0000","last_became_active":"2024-07-17T06:54:53.361896+0000","last_became_peered":"2024-07-17T06:54:53.361896+0000","last_unstale":"2024-07-17T06:54:54.340002+0000","last_undegraded":"2024-07-17T06:54:54.340002+0000","last_fullsized":"2024-07-17T06:54:54.340002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:22.522023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566168+0000","last_change":"2024-07-17T06:54:49.316448+0000","last_active":"2024-07-17T06:54:54.566168+0000","last_peered":"2024-07-17T06:54:54.566168+0000","last_clean":"2024-07-17T06:54:54.566168+0000","last_became_active":"2024-07-17T06:54:49.316042+0000","last_became_peered":"2024-07-17T06:54:49.316042+0000","last_unstale":"2024-07-17T06:54:54.566168+0000","last_undegraded":"2024-07-17T06:54:54.566168+0000","last_fullsized":"2024-07-17T06:54:54.566168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:33.731836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.338832+0000","last_change":"2024-07-17T06:54:48.270227+0000","last_active":"2024-07-17T06:54:54.338832+0000","last_peered":"2024-07-17T06:54:54.338832+0000","last_clean":"2024-07-17T06:54:54.338832+0000","last_became_active":"2024-07-17T06:54:48.269954+0000","last_became_peered":"2024-07-17T06:54:48.269954+0000","last_unstale":"2024-07-17T06:54:54.338832+0000","last_undegraded":"2024-07-17T06:54:54.338832+0000","last_fullsized":"2024-07-17T06:54:54.338832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.390020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568918+0000","last_change":"2024-07-17T06:54:53.356983+0000","last_active":"2024-07-17T06:54:54.568918+0000","last_peered":"2024-07-17T06:54:54.568918+0000","last_clean":"2024-07-17T06:54:54.568918+0000","last_became_active":"2024-07-17T06:54:53.356288+0000","last_became_peered":"2024-07-17T06:54:53.356288+0000","last_unstale":"2024-07-17T06:54:54.568918+0000","last_undegraded":"2024-07-17T06:54:54.568918+0000","last_fullsized":"2024-07-17T06:54:54.568918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339405+0000","last_change":"2024-07-17T06:54:49.294433+0000","last_active":"2024-07-17T06:54:54.339405+0000","last_peered":"2024-07-17T06:54:54.339405+0000","last_clean":"2024-07-17T06:54:54.339405+0000","last_became_active":"2024-07-17T06:54:49.293644+0000","last_became_peered":"2024-07-17T06:54:49.293644+0000","last_unstale":"2024-07-17T06:54:54.339405+0000","last_undegraded":"2024-07-17T06:54:54.339405+0000","last_fullsized":"2024-07-17T06:54:54.339405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340443+0000","last_change":"2024-07-17T06:54:48.267002+0000","last_active":"2024-07-17T06:54:54.340443+0000","last_peered":"2024-07-17T06:54:54.340443+0000","last_clean":"2024-07-17T06:54:54.340443+0000","last_became_active":"2024-07-17T06:54:48.266711+0000","last_became_peered":"2024-07-17T06:54:48.266711+0000","last_unstale":"2024-07-17T06:54:54.340443+0000","last_undegraded":"2024-07-17T06:54:54.340443+0000","last_fullsized":"2024-07-17T06:54:54.340443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572427+0000","last_change":"2024-07-17T06:54:53.678483+0000","last_active":"2024-07-17T06:54:54.572427+0000","last_peered":"2024-07-17T06:54:54.572427+0000","last_clean":"2024-07-17T06:54:54.572427+0000","last_became_active":"2024-07-17T06:54:53.677707+0000","last_became_peered":"2024-07-17T06:54:53.677707+0000","last_unstale":"2024-07-17T06:54:54.572427+0000","last_undegraded":"2024-07-17T06:54:54.572427+0000","last_fullsized":"2024-07-17T06:54:54.572427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:21.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568071+0000","last_change":"2024-07-17T06:54:49.319059+0000","last_active":"2024-07-17T06:54:54.568071+0000","last_peered":"2024-07-17T06:54:54.568071+0000","last_clean":"2024-07-17T06:54:54.568071+0000","last_became_active":"2024-07-17T06:54:49.318532+0000","last_became_peered":"2024-07-17T06:54:49.318532+0000","last_unstale":"2024-07-17T06:54:54.568071+0000","last_undegraded":"2024-07-17T06:54:54.568071+0000","last_fullsized":"2024-07-17T06:54:54.568071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:47.675783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566475+0000","last_change":"2024-07-17T06:54:48.279991+0000","last_active":"2024-07-17T06:54:54.566475+0000","last_peered":"2024-07-17T06:54:54.566475+0000","last_clean":"2024-07-17T06:54:54.566475+0000","last_became_active":"2024-07-17T06:54:48.279691+0000","last_became_peered":"2024-07-17T06:54:48.279691+0000","last_unstale":"2024-07-17T06:54:54.566475+0000","last_undegraded":"2024-07-17T06:54:54.566475+0000","last_fullsized":"2024-07-17T06:54:54.566475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571709+0000","last_change":"2024-07-17T06:54:53.677451+0000","last_active":"2024-07-17T06:54:54.571709+0000","last_peered":"2024-07-17T06:54:54.571709+0000","last_clean":"2024-07-17T06:54:54.571709+0000","last_became_active":"2024-07-17T06:54:53.677080+0000","last_became_peered":"2024-07-17T06:54:53.677080+0000","last_unstale":"2024-07-17T06:54:54.571709+0000","last_undegraded":"2024-07-17T06:54:54.571709+0000","last_fullsized":"2024-07-17T06:54:54.571709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567583+0000","last_change":"2024-07-17T06:54:49.302477+0000","last_active":"2024-07-17T06:54:54.567583+0000","last_peered":"2024-07-17T06:54:54.567583+0000","last_clean":"2024-07-17T06:54:54.567583+0000","last_became_active":"2024-07-17T06:54:49.302168+0000","last_became_peered":"2024-07-17T06:54:49.302168+0000","last_unstale":"2024-07-17T06:54:54.567583+0000","last_undegraded":"2024-07-17T06:54:54.567583+0000","last_fullsized":"2024-07-17T06:54:54.567583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:20.279165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566160+0000","last_change":"2024-07-17T06:54:48.278091+0000","last_active":"2024-07-17T06:54:54.566160+0000","last_peered":"2024-07-17T06:54:54.566160+0000","last_clean":"2024-07-17T06:54:54.566160+0000","last_became_active":"2024-07-17T06:54:48.277769+0000","last_became_peered":"2024-07-17T06:54:48.277769+0000","last_unstale":"2024-07-17T06:54:54.566160+0000","last_undegraded":"2024-07-17T06:54:54.566160+0000","last_fullsized":"2024-07-17T06:54:54.566160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567004+0000","last_change":"2024-07-17T06:54:53.672346+0000","last_active":"2024-07-17T06:54:54.567004+0000","last_peered":"2024-07-17T06:54:54.567004+0000","last_clean":"2024-07-17T06:54:54.567004+0000","last_became_active":"2024-07-17T06:54:53.670973+0000","last_became_peered":"2024-07-17T06:54:53.670973+0000","last_unstale":"2024-07-17T06:54:54.567004+0000","last_undegraded":"2024-07-17T06:54:54.567004+0000","last_fullsized":"2024-07-17T06:54:54.567004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568419+0000","last_change":"2024-07-17T06:54:49.314009+0000","last_active":"2024-07-17T06:54:54.568419+0000","last_peered":"2024-07-17T06:54:54.568419+0000","last_clean":"2024-07-17T06:54:54.568419+0000","last_became_active":"2024-07-17T06:54:49.313238+0000","last_became_peered":"2024-07-17T06:54:49.313238+0000","last_unstale":"2024-07-17T06:54:54.568419+0000","last_undegraded":"2024-07-17T06:54:54.568419+0000","last_fullsized":"2024-07-17T06:54:54.568419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:16.115918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:02.616138+0000","last_change":"2024-07-17T06:54:48.274173+0000","last_active":"2024-07-17T06:55:02.616138+0000","last_peered":"2024-07-17T06:55:02.616138+0000","last_clean":"2024-07-17T06:55:02.616138+0000","last_became_active":"2024-07-17T06:54:48.270354+0000","last_became_peered":"2024-07-17T06:54:48.270354+0000","last_unstale":"2024-07-17T06:55:02.616138+0000","last_undegraded":"2024-07-17T06:55:02.616138+0000","last_fullsized":"2024-07-17T06:55:02.616138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568505+0000","last_change":"2024-07-17T06:54:53.347808+0000","last_active":"2024-07-17T06:54:54.568505+0000","last_peered":"2024-07-17T06:54:54.568505+0000","last_clean":"2024-07-17T06:54:54.568505+0000","last_became_active":"2024-07-17T06:54:53.347369+0000","last_became_peered":"2024-07-17T06:54:53.347369+0000","last_unstale":"2024-07-17T06:54:54.568505+0000","last_undegraded":"2024-07-17T06:54:54.568505+0000","last_fullsized":"2024-07-17T06:54:54.568505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344599+0000","last_change":"2024-07-17T06:54:49.308852+0000","last_active":"2024-07-17T06:54:54.344599+0000","last_peered":"2024-07-17T06:54:54.344599+0000","last_clean":"2024-07-17T06:54:54.344599+0000","last_became_active":"2024-07-17T06:54:49.308337+0000","last_became_peered":"2024-07-17T06:54:49.308337+0000","last_unstale":"2024-07-17T06:54:54.344599+0000","last_undegraded":"2024-07-17T06:54:54.344599+0000","last_fullsized":"2024-07-17T06:54:54.344599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:06.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570694+0000","last_change":"2024-07-17T06:54:48.282475+0000","last_active":"2024-07-17T06:54:54.570694+0000","last_peered":"2024-07-17T06:54:54.570694+0000","last_clean":"2024-07-17T06:54:54.570694+0000","last_became_active":"2024-07-17T06:54:48.282039+0000","last_became_peered":"2024-07-17T06:54:48.282039+0000","last_unstale":"2024-07-17T06:54:54.570694+0000","last_undegraded":"2024-07-17T06:54:54.570694+0000","last_fullsized":"2024-07-17T06:54:54.570694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344160+0000","last_change":"2024-07-17T06:54:53.353664+0000","last_active":"2024-07-17T06:54:54.344160+0000","last_peered":"2024-07-17T06:54:54.344160+0000","last_clean":"2024-07-17T06:54:54.344160+0000","last_became_active":"2024-07-17T06:54:53.351779+0000","last_became_peered":"2024-07-17T06:54:53.351779+0000","last_unstale":"2024-07-17T06:54:54.344160+0000","last_undegraded":"2024-07-17T06:54:54.344160+0000","last_fullsized":"2024-07-17T06:54:54.344160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:20.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567440+0000","last_change":"2024-07-17T06:54:49.313557+0000","last_active":"2024-07-17T06:54:54.567440+0000","last_peered":"2024-07-17T06:54:54.567440+0000","last_clean":"2024-07-17T06:54:54.567440+0000","last_became_active":"2024-07-17T06:54:49.312875+0000","last_became_peered":"2024-07-17T06:54:49.312875+0000","last_unstale":"2024-07-17T06:54:54.567440+0000","last_undegraded":"2024-07-17T06:54:54.567440+0000","last_fullsized":"2024-07-17T06:54:54.567440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567055+0000","last_change":"2024-07-17T06:54:48.286081+0000","last_active":"2024-07-17T06:54:54.567055+0000","last_peered":"2024-07-17T06:54:54.567055+0000","last_clean":"2024-07-17T06:54:54.567055+0000","last_became_active":"2024-07-17T06:54:48.285327+0000","last_became_peered":"2024-07-17T06:54:48.285327+0000","last_unstale":"2024-07-17T06:54:54.567055+0000","last_undegraded":"2024-07-17T06:54:54.567055+0000","last_fullsized":"2024-07-17T06:54:54.567055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567082+0000","last_change":"2024-07-17T06:54:53.353255+0000","last_active":"2024-07-17T06:54:54.567082+0000","last_peered":"2024-07-17T06:54:54.567082+0000","last_clean":"2024-07-17T06:54:54.567082+0000","last_became_active":"2024-07-17T06:54:53.352945+0000","last_became_peered":"2024-07-17T06:54:53.352945+0000","last_unstale":"2024-07-17T06:54:54.567082+0000","last_undegraded":"2024-07-17T06:54:54.567082+0000","last_fullsized":"2024-07-17T06:54:54.567082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:06.889450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566804+0000","last_change":"2024-07-17T06:54:49.318904+0000","last_active":"2024-07-17T06:54:54.566804+0000","last_peered":"2024-07-17T06:54:54.566804+0000","last_clean":"2024-07-17T06:54:54.566804+0000","last_became_active":"2024-07-17T06:54:49.316753+0000","last_became_peered":"2024-07-17T06:54:49.316753+0000","last_unstale":"2024-07-17T06:54:54.566804+0000","last_undegraded":"2024-07-17T06:54:54.566804+0000","last_fullsized":"2024-07-17T06:54:54.566804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:45.913897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343278+0000","last_change":"2024-07-17T06:54:48.286978+0000","last_active":"2024-07-17T06:54:54.343278+0000","last_peered":"2024-07-17T06:54:54.343278+0000","last_clean":"2024-07-17T06:54:54.343278+0000","last_became_active":"2024-07-17T06:54:48.286613+0000","last_became_peered":"2024-07-17T06:54:48.286613+0000","last_unstale":"2024-07-17T06:54:54.343278+0000","last_undegraded":"2024-07-17T06:54:54.343278+0000","last_fullsized":"2024-07-17T06:54:54.343278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570149+0000","last_change":"2024-07-17T06:54:53.671456+0000","last_active":"2024-07-17T06:54:54.570149+0000","last_peered":"2024-07-17T06:54:54.570149+0000","last_clean":"2024-07-17T06:54:54.570149+0000","last_became_active":"2024-07-17T06:54:53.670354+0000","last_became_peered":"2024-07-17T06:54:53.670354+0000","last_unstale":"2024-07-17T06:54:54.570149+0000","last_undegraded":"2024-07-17T06:54:54.570149+0000","last_fullsized":"2024-07-17T06:54:54.570149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571489+0000","last_change":"2024-07-17T06:54:53.682714+0000","last_active":"2024-07-17T06:54:54.571489+0000","last_peered":"2024-07-17T06:54:54.571489+0000","last_clean":"2024-07-17T06:54:54.571489+0000","last_became_active":"2024-07-17T06:54:53.681884+0000","last_became_peered":"2024-07-17T06:54:53.681884+0000","last_unstale":"2024-07-17T06:54:54.571489+0000","last_undegraded":"2024-07-17T06:54:54.571489+0000","last_fullsized":"2024-07-17T06:54:54.571489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568224+0000","last_change":"2024-07-17T06:54:49.305906+0000","last_active":"2024-07-17T06:54:54.568224+0000","last_peered":"2024-07-17T06:54:54.568224+0000","last_clean":"2024-07-17T06:54:54.568224+0000","last_became_active":"2024-07-17T06:54:49.305633+0000","last_became_peered":"2024-07-17T06:54:49.305633+0000","last_unstale":"2024-07-17T06:54:54.568224+0000","last_undegraded":"2024-07-17T06:54:54.568224+0000","last_fullsized":"2024-07-17T06:54:54.568224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565775+0000","last_change":"2024-07-17T06:54:48.275763+0000","last_active":"2024-07-17T06:54:54.565775+0000","last_peered":"2024-07-17T06:54:54.565775+0000","last_clean":"2024-07-17T06:54:54.565775+0000","last_became_active":"2024-07-17T06:54:48.275073+0000","last_became_peered":"2024-07-17T06:54:48.275073+0000","last_unstale":"2024-07-17T06:54:54.565775+0000","last_undegraded":"2024-07-17T06:54:54.565775+0000","last_fullsized":"2024-07-17T06:54:54.565775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343084+0000","last_change":"2024-07-17T06:54:53.347306+0000","last_active":"2024-07-17T06:54:54.343084+0000","last_peered":"2024-07-17T06:54:54.343084+0000","last_clean":"2024-07-17T06:54:54.343084+0000","last_became_active":"2024-07-17T06:54:53.346541+0000","last_became_peered":"2024-07-17T06:54:53.346541+0000","last_unstale":"2024-07-17T06:54:54.343084+0000","last_undegraded":"2024-07-17T06:54:54.343084+0000","last_fullsized":"2024-07-17T06:54:54.343084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567129+0000","last_change":"2024-07-17T06:54:49.299691+0000","last_active":"2024-07-17T06:54:54.567129+0000","last_peered":"2024-07-17T06:54:54.567129+0000","last_clean":"2024-07-17T06:54:54.567129+0000","last_became_active":"2024-07-17T06:54:49.299370+0000","last_became_peered":"2024-07-17T06:54:49.299370+0000","last_unstale":"2024-07-17T06:54:54.567129+0000","last_undegraded":"2024-07-17T06:54:54.567129+0000","last_fullsized":"2024-07-17T06:54:54.567129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570373+0000","last_change":"2024-07-17T06:54:48.276504+0000","last_active":"2024-07-17T06:54:54.570373+0000","last_peered":"2024-07-17T06:54:54.570373+0000","last_clean":"2024-07-17T06:54:54.570373+0000","last_became_active":"2024-07-17T06:54:48.276041+0000","last_became_peered":"2024-07-17T06:54:48.276041+0000","last_unstale":"2024-07-17T06:54:54.570373+0000","last_undegraded":"2024-07-17T06:54:54.570373+0000","last_fullsized":"2024-07-17T06:54:54.570373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342215+0000","last_change":"2024-07-17T06:54:48.267874+0000","last_active":"2024-07-17T06:54:54.342215+0000","last_peered":"2024-07-17T06:54:54.342215+0000","last_clean":"2024-07-17T06:54:54.342215+0000","last_became_active":"2024-07-17T06:54:48.267487+0000","last_became_peered":"2024-07-17T06:54:48.267487+0000","last_unstale":"2024-07-17T06:54:54.342215+0000","last_undegraded":"2024-07-17T06:54:54.342215+0000","last_fullsized":"2024-07-17T06:54:54.342215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566178+0000","last_change":"2024-07-17T06:54:49.306648+0000","last_active":"2024-07-17T06:54:54.566178+0000","last_peered":"2024-07-17T06:54:54.566178+0000","last_clean":"2024-07-17T06:54:54.566178+0000","last_became_active":"2024-07-17T06:54:49.305001+0000","last_became_peered":"2024-07-17T06:54:49.305001+0000","last_unstale":"2024-07-17T06:54:54.566178+0000","last_undegraded":"2024-07-17T06:54:54.566178+0000","last_fullsized":"2024-07-17T06:54:54.566178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339673+0000","last_change":"2024-07-17T06:54:53.360122+0000","last_active":"2024-07-17T06:54:54.339673+0000","last_peered":"2024-07-17T06:54:54.339673+0000","last_clean":"2024-07-17T06:54:54.339673+0000","last_became_active":"2024-07-17T06:54:53.356530+0000","last_became_peered":"2024-07-17T06:54:53.356530+0000","last_unstale":"2024-07-17T06:54:54.339673+0000","last_undegraded":"2024-07-17T06:54:54.339673+0000","last_fullsized":"2024-07-17T06:54:54.339673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:41.753064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570947+0000","last_change":"2024-07-17T06:54:48.277409+0000","last_active":"2024-07-17T06:54:54.570947+0000","last_peered":"2024-07-17T06:54:54.570947+0000","last_clean":"2024-07-17T06:54:54.570947+0000","last_became_active":"2024-07-17T06:54:48.277087+0000","last_became_peered":"2024-07-17T06:54:48.277087+0000","last_unstale":"2024-07-17T06:54:54.570947+0000","last_undegraded":"2024-07-17T06:54:54.570947+0000","last_fullsized":"2024-07-17T06:54:54.570947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:56:17.744184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342397+0000","last_change":"2024-07-17T06:54:49.300524+0000","last_active":"2024-07-17T06:54:54.342397+0000","last_peered":"2024-07-17T06:54:54.342397+0000","last_clean":"2024-07-17T06:54:54.342397+0000","last_became_active":"2024-07-17T06:54:49.300136+0000","last_became_peered":"2024-07-17T06:54:49.300136+0000","last_unstale":"2024-07-17T06:54:54.342397+0000","last_undegraded":"2024-07-17T06:54:54.342397+0000","last_fullsized":"2024-07-17T06:54:54.342397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:29.406070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571827+0000","last_change":"2024-07-17T06:54:53.355705+0000","last_active":"2024-07-17T06:54:54.571827+0000","last_peered":"2024-07-17T06:54:54.571827+0000","last_clean":"2024-07-17T06:54:54.571827+0000","last_became_active":"2024-07-17T06:54:53.355294+0000","last_became_peered":"2024-07-17T06:54:53.355294+0000","last_unstale":"2024-07-17T06:54:54.571827+0000","last_undegraded":"2024-07-17T06:54:54.571827+0000","last_fullsized":"2024-07-17T06:54:54.571827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:32:32.729442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339117+0000","last_change":"2024-07-17T06:54:49.294765+0000","last_active":"2024-07-17T06:54:54.339117+0000","last_peered":"2024-07-17T06:54:54.339117+0000","last_clean":"2024-07-17T06:54:54.339117+0000","last_became_active":"2024-07-17T06:54:49.294095+0000","last_became_peered":"2024-07-17T06:54:49.294095+0000","last_unstale":"2024-07-17T06:54:54.339117+0000","last_undegraded":"2024-07-17T06:54:54.339117+0000","last_fullsized":"2024-07-17T06:54:54.339117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339066+0000","last_change":"2024-07-17T06:54:49.299295+0000","last_active":"2024-07-17T06:54:54.339066+0000","last_peered":"2024-07-17T06:54:54.339066+0000","last_clean":"2024-07-17T06:54:54.339066+0000","last_became_active":"2024-07-17T06:54:49.298385+0000","last_became_peered":"2024-07-17T06:54:49.298385+0000","last_unstale":"2024-07-17T06:54:54.339066+0000","last_undegraded":"2024-07-17T06:54:54.339066+0000","last_fullsized":"2024-07-17T06:54:54.339066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:55.920311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566243+0000","last_change":"2024-07-17T06:54:49.325974+0000","last_active":"2024-07-17T06:54:54.566243+0000","last_peered":"2024-07-17T06:54:54.566243+0000","last_clean":"2024-07-17T06:54:54.566243+0000","last_became_active":"2024-07-17T06:54:49.322413+0000","last_became_peered":"2024-07-17T06:54:49.322413+0000","last_unstale":"2024-07-17T06:54:54.566243+0000","last_undegraded":"2024-07-17T06:54:54.566243+0000","last_fullsized":"2024-07-17T06:54:54.566243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:18.496012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566055+0000","last_change":"2024-07-17T06:54:49.308310+0000","last_active":"2024-07-17T06:54:54.566055+0000","last_peered":"2024-07-17T06:54:54.566055+0000","last_clean":"2024-07-17T06:54:54.566055+0000","last_became_active":"2024-07-17T06:54:49.304819+0000","last_became_peered":"2024-07-17T06:54:49.304819+0000","last_unstale":"2024-07-17T06:54:54.566055+0000","last_undegraded":"2024-07-17T06:54:54.566055+0000","last_fullsized":"2024-07-17T06:54:54.566055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571870+0000","last_change":"2024-07-17T06:54:49.312400+0000","last_active":"2024-07-17T06:54:54.571870+0000","last_peered":"2024-07-17T06:54:54.571870+0000","last_clean":"2024-07-17T06:54:54.571870+0000","last_became_active":"2024-07-17T06:54:49.311958+0000","last_became_peered":"2024-07-17T06:54:49.311958+0000","last_unstale":"2024-07-17T06:54:54.571870+0000","last_undegraded":"2024-07-17T06:54:54.571870+0000","last_fullsized":"2024-07-17T06:54:54.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.321280+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.320912+0000","last_became_peered":"2024-07-17T06:54:49.320912+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341006+0000","last_change":"2024-07-17T06:54:49.297628+0000","last_active":"2024-07-17T06:54:54.341006+0000","last_peered":"2024-07-17T06:54:54.341006+0000","last_clean":"2024-07-17T06:54:54.341006+0000","last_became_active":"2024-07-17T06:54:49.296871+0000","last_became_peered":"2024-07-17T06:54:49.296871+0000","last_unstale":"2024-07-17T06:54:54.341006+0000","last_undegraded":"2024-07-17T06:54:54.341006+0000","last_fullsized":"2024-07-17T06:54:54.341006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572513+0000","last_change":"2024-07-17T06:54:49.303253+0000","last_active":"2024-07-17T06:54:54.572513+0000","last_peered":"2024-07-17T06:54:54.572513+0000","last_clean":"2024-07-17T06:54:54.572513+0000","last_became_active":"2024-07-17T06:54:49.302956+0000","last_became_peered":"2024-07-17T06:54:49.302956+0000","last_unstale":"2024-07-17T06:54:54.572513+0000","last_undegraded":"2024-07-17T06:54:54.572513+0000","last_fullsized":"2024-07-17T06:54:54.572513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565697+0000","last_change":"2024-07-17T06:54:49.306156+0000","last_active":"2024-07-17T06:54:54.565697+0000","last_peered":"2024-07-17T06:54:54.565697+0000","last_clean":"2024-07-17T06:54:54.565697+0000","last_became_active":"2024-07-17T06:54:49.305858+0000","last_became_peered":"2024-07-17T06:54:49.305858+0000","last_unstale":"2024-07-17T06:54:54.565697+0000","last_undegraded":"2024-07-17T06:54:54.565697+0000","last_fullsized":"2024-07-17T06:54:54.565697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.301568+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.301244+0000","last_became_peered":"2024-07-17T06:54:49.301244+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567719+0000","last_change":"2024-07-17T06:54:49.316422+0000","last_active":"2024-07-17T06:54:54.567719+0000","last_peered":"2024-07-17T06:54:54.567719+0000","last_clean":"2024-07-17T06:54:54.567719+0000","last_became_active":"2024-07-17T06:54:49.316133+0000","last_became_peered":"2024-07-17T06:54:49.316133+0000","last_unstale":"2024-07-17T06:54:54.567719+0000","last_undegraded":"2024-07-17T06:54:54.567719+0000","last_fullsized":"2024-07-17T06:54:54.567719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565901+0000","last_change":"2024-07-17T06:54:49.305491+0000","last_active":"2024-07-17T06:54:54.565901+0000","last_peered":"2024-07-17T06:54:54.565901+0000","last_clean":"2024-07-17T06:54:54.565901+0000","last_became_active":"2024-07-17T06:54:49.304629+0000","last_became_peered":"2024-07-17T06:54:49.304629+0000","last_unstale":"2024-07-17T06:54:54.565901+0000","last_undegraded":"2024-07-17T06:54:54.565901+0000","last_fullsized":"2024-07-17T06:54:54.565901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343504+0000","last_change":"2024-07-17T06:54:49.309637+0000","last_active":"2024-07-17T06:54:54.343504+0000","last_peered":"2024-07-17T06:54:54.343504+0000","last_clean":"2024-07-17T06:54:54.343504+0000","last_became_active":"2024-07-17T06:54:49.309243+0000","last_became_peered":"2024-07-17T06:54:49.309243+0000","last_unstale":"2024-07-17T06:54:54.343504+0000","last_undegraded":"2024-07-17T06:54:54.343504+0000","last_fullsized":"2024-07-17T06:54:54.343504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569329+0000","last_change":"2024-07-17T06:54:49.323709+0000","last_active":"2024-07-17T06:54:54.569329+0000","last_peered":"2024-07-17T06:54:54.569329+0000","last_clean":"2024-07-17T06:54:54.569329+0000","last_became_active":"2024-07-17T06:54:49.323324+0000","last_became_peered":"2024-07-17T06:54:49.323324+0000","last_unstale":"2024-07-17T06:54:54.569329+0000","last_undegraded":"2024-07-17T06:54:54.569329+0000","last_fullsized":"2024-07-17T06:54:54.569329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339810+0000","last_change":"2024-07-17T06:54:49.286063+0000","last_active":"2024-07-17T06:54:54.339810+0000","last_peered":"2024-07-17T06:54:54.339810+0000","last_clean":"2024-07-17T06:54:54.339810+0000","last_became_active":"2024-07-17T06:54:49.283989+0000","last_became_peered":"2024-07-17T06:54:49.283989+0000","last_unstale":"2024-07-17T06:54:54.339810+0000","last_undegraded":"2024-07-17T06:54:54.339810+0000","last_fullsized":"2024-07-17T06:54:54.339810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:43.655076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567362+0000","last_change":"2024-07-17T06:54:49.319544+0000","last_active":"2024-07-17T06:54:54.567362+0000","last_peered":"2024-07-17T06:54:54.567362+0000","last_clean":"2024-07-17T06:54:54.567362+0000","last_became_active":"2024-07-17T06:54:49.319285+0000","last_became_peered":"2024-07-17T06:54:49.319285+0000","last_unstale":"2024-07-17T06:54:54.567362+0000","last_undegraded":"2024-07-17T06:54:54.567362+0000","last_fullsized":"2024-07-17T06:54:54.567362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:58.821222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565429+0000","last_change":"2024-07-17T06:54:49.305371+0000","last_active":"2024-07-17T06:54:54.565429+0000","last_peered":"2024-07-17T06:54:54.565429+0000","last_clean":"2024-07-17T06:54:54.565429+0000","last_became_active":"2024-07-17T06:54:49.305069+0000","last_became_peered":"2024-07-17T06:54:49.305069+0000","last_unstale":"2024-07-17T06:54:54.565429+0000","last_undegraded":"2024-07-17T06:54:54.565429+0000","last_fullsized":"2024-07-17T06:54:54.565429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568226+0000","last_change":"2024-07-17T06:54:49.314419+0000","last_active":"2024-07-17T06:54:54.568226+0000","last_peered":"2024-07-17T06:54:54.568226+0000","last_clean":"2024-07-17T06:54:54.568226+0000","last_became_active":"2024-07-17T06:54:49.314106+0000","last_became_peered":"2024-07-17T06:54:49.314106+0000","last_unstale":"2024-07-17T06:54:54.568226+0000","last_undegraded":"2024-07-17T06:54:54.568226+0000","last_fullsized":"2024-07-17T06:54:54.568226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566591+0000","last_change":"2024-07-17T06:54:49.297635+0000","last_active":"2024-07-17T06:54:54.566591+0000","last_peered":"2024-07-17T06:54:54.566591+0000","last_clean":"2024-07-17T06:54:54.566591+0000","last_became_active":"2024-07-17T06:54:49.297252+0000","last_became_peered":"2024-07-17T06:54:49.297252+0000","last_unstale":"2024-07-17T06:54:54.566591+0000","last_undegraded":"2024-07-17T06:54:54.566591+0000","last_fullsized":"2024-07-17T06:54:54.566591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:02.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570670+0000","last_change":"2024-07-17T06:54:49.311510+0000","last_active":"2024-07-17T06:54:54.570670+0000","last_peered":"2024-07-17T06:54:54.570670+0000","last_clean":"2024-07-17T06:54:54.570670+0000","last_became_active":"2024-07-17T06:54:49.311208+0000","last_became_peered":"2024-07-17T06:54:49.311208+0000","last_unstale":"2024-07-17T06:54:54.570670+0000","last_undegraded":"2024-07-17T06:54:54.570670+0000","last_fullsized":"2024-07-17T06:54:54.570670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344453+0000","last_change":"2024-07-17T06:54:49.300592+0000","last_active":"2024-07-17T06:54:54.344453+0000","last_peered":"2024-07-17T06:54:54.344453+0000","last_clean":"2024-07-17T06:54:54.344453+0000","last_became_active":"2024-07-17T06:54:49.300233+0000","last_became_peered":"2024-07-17T06:54:49.300233+0000","last_unstale":"2024-07-17T06:54:54.344453+0000","last_undegraded":"2024-07-17T06:54:54.344453+0000","last_fullsized":"2024-07-17T06:54:54.344453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:53.264056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341453+0000","last_change":"2024-07-17T06:54:49.304204+0000","last_active":"2024-07-17T06:54:54.341453+0000","last_peered":"2024-07-17T06:54:54.341453+0000","last_clean":"2024-07-17T06:54:54.341453+0000","last_became_active":"2024-07-17T06:54:49.303907+0000","last_became_peered":"2024-07-17T06:54:49.303907+0000","last_unstale":"2024-07-17T06:54:54.341453+0000","last_undegraded":"2024-07-17T06:54:54.341453+0000","last_fullsized":"2024-07-17T06:54:54.341453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340757+0000","last_change":"2024-07-17T06:54:49.297834+0000","last_active":"2024-07-17T06:54:54.340757+0000","last_peered":"2024-07-17T06:54:54.340757+0000","last_clean":"2024-07-17T06:54:54.340757+0000","last_became_active":"2024-07-17T06:54:49.297296+0000","last_became_peered":"2024-07-17T06:54:49.297296+0000","last_unstale":"2024-07-17T06:54:54.340757+0000","last_undegraded":"2024-07-17T06:54:54.340757+0000","last_fullsized":"2024-07-17T06:54:54.340757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569209+0000","last_change":"2024-07-17T06:54:49.318654+0000","last_active":"2024-07-17T06:54:54.569209+0000","last_peered":"2024-07-17T06:54:54.569209+0000","last_clean":"2024-07-17T06:54:54.569209+0000","last_became_active":"2024-07-17T06:54:49.318380+0000","last_became_peered":"2024-07-17T06:54:49.318380+0000","last_unstale":"2024-07-17T06:54:54.569209+0000","last_undegraded":"2024-07-17T06:54:54.569209+0000","last_fullsized":"2024-07-17T06:54:54.569209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:12.140489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566890+0000","last_change":"2024-07-17T06:54:49.297152+0000","last_active":"2024-07-17T06:54:54.566890+0000","last_peered":"2024-07-17T06:54:54.566890+0000","last_clean":"2024-07-17T06:54:54.566890+0000","last_became_active":"2024-07-17T06:54:49.296830+0000","last_became_peered":"2024-07-17T06:54:49.296830+0000","last_unstale":"2024-07-17T06:54:54.566890+0000","last_undegraded":"2024-07-17T06:54:54.566890+0000","last_fullsized":"2024-07-17T06:54:54.566890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566123+0000","last_change":"2024-07-17T06:54:49.313713+0000","last_active":"2024-07-17T06:54:54.566123+0000","last_peered":"2024-07-17T06:54:54.566123+0000","last_clean":"2024-07-17T06:54:54.566123+0000","last_became_active":"2024-07-17T06:54:49.313419+0000","last_became_peered":"2024-07-17T06:54:49.313419+0000","last_unstale":"2024-07-17T06:54:54.566123+0000","last_undegraded":"2024-07-17T06:54:54.566123+0000","last_fullsized":"2024-07-17T06:54:54.566123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:25.517176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565979+0000","last_change":"2024-07-17T06:54:49.304993+0000","last_active":"2024-07-17T06:54:54.565979+0000","last_peered":"2024-07-17T06:54:54.565979+0000","last_clean":"2024-07-17T06:54:54.565979+0000","last_became_active":"2024-07-17T06:54:49.304475+0000","last_became_peered":"2024-07-17T06:54:49.304475+0000","last_unstale":"2024-07-17T06:54:54.565979+0000","last_undegraded":"2024-07-17T06:54:54.565979+0000","last_fullsized":"2024-07-17T06:54:54.565979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565951+0000","last_change":"2024-07-17T06:54:49.319946+0000","last_active":"2024-07-17T06:54:54.565951+0000","last_peered":"2024-07-17T06:54:54.565951+0000","last_clean":"2024-07-17T06:54:54.565951+0000","last_became_active":"2024-07-17T06:54:49.319685+0000","last_became_peered":"2024-07-17T06:54:49.319685+0000","last_unstale":"2024-07-17T06:54:54.565951+0000","last_undegraded":"2024-07-17T06:54:54.565951+0000","last_fullsized":"2024-07-17T06:54:54.565951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:10.177399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566540+0000","last_change":"2024-07-17T06:54:49.324468+0000","last_active":"2024-07-17T06:54:54.566540+0000","last_peered":"2024-07-17T06:54:54.566540+0000","last_clean":"2024-07-17T06:54:54.566540+0000","last_became_active":"2024-07-17T06:54:49.323700+0000","last_became_peered":"2024-07-17T06:54:49.323700+0000","last_unstale":"2024-07-17T06:54:54.566540+0000","last_undegraded":"2024-07-17T06:54:54.566540+0000","last_fullsized":"2024-07-17T06:54:54.566540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:05.797436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567150+0000","last_change":"2024-07-17T06:54:49.290953+0000","last_active":"2024-07-17T06:54:54.567150+0000","last_peered":"2024-07-17T06:54:54.567150+0000","last_clean":"2024-07-17T06:54:54.567150+0000","last_became_active":"2024-07-17T06:54:49.290069+0000","last_became_peered":"2024-07-17T06:54:49.290069+0000","last_unstale":"2024-07-17T06:54:54.567150+0000","last_undegraded":"2024-07-17T06:54:54.567150+0000","last_fullsized":"2024-07-17T06:54:54.567150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571210+0000","last_change":"2024-07-17T06:54:49.301304+0000","last_active":"2024-07-17T06:54:54.571210+0000","last_peered":"2024-07-17T06:54:54.571210+0000","last_clean":"2024-07-17T06:54:54.571210+0000","last_became_active":"2024-07-17T06:54:49.300865+0000","last_became_peered":"2024-07-17T06:54:49.300865+0000","last_unstale":"2024-07-17T06:54:54.571210+0000","last_undegraded":"2024-07-17T06:54:54.571210+0000","last_fullsized":"2024-07-17T06:54:54.571210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569004+0000","last_change":"2024-07-17T06:54:49.324041+0000","last_active":"2024-07-17T06:54:54.569004+0000","last_peered":"2024-07-17T06:54:54.569004+0000","last_clean":"2024-07-17T06:54:54.569004+0000","last_became_active":"2024-07-17T06:54:49.323747+0000","last_became_peered":"2024-07-17T06:54:49.323747+0000","last_unstale":"2024-07-17T06:54:54.569004+0000","last_undegraded":"2024-07-17T06:54:54.569004+0000","last_fullsized":"2024-07-17T06:54:54.569004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:04:46.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":101953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":692869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":692813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":101989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":101863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:55:09.381 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T06:55:09.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-07-17T06:55:09.382 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-07-17T06:55:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 -- 172.21.15.44:0/3826256765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 msgr2=0x7fe9a40ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/3826256765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a40ac290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe994009830 tx=0x7fe99402a480 comp rx=0 tx=0).stop 2024-07-17T06:55:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 -- 172.21.15.44:0/3826256765 shutdown_connections 2024-07-17T06:55:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/3826256765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 0x7fe9a415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/3826256765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a40ac290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 -- 172.21.15.44:0/3826256765 >> 172.21.15.44:0/3826256765 conn(0x7fe9a40cf0d0 msgr2=0x7fe9a40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 -- 172.21.15.44:0/3826256765 shutdown_connections 2024-07-17T06:55:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.626+0000 7fe9a9c00640 1 -- 172.21.15.44:0/3826256765 wait complete. 2024-07-17T06:55:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.628+0000 7fe9a9c00640 1 Processor -- start 2024-07-17T06:55:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.628+0000 7fe9a9c00640 1 -- start start 2024-07-17T06:55:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 0x7fe9a41f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 0x7fe9a414c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a406f670 con 0x7fe9a4150ed0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a406f4f0 con 0x7fe9a406edc0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a415e2a0 con 0x7fe9a41512b0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37506/0 (socket says 172.21.15.44:37506) 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.629+0000 7fe9a3400640 1 -- 172.21.15.44:0/397539176 learned_addr learned my addr 172.21.15.44:0/397539176 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3e00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 0x7fe9a41f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a9200640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 0x7fe9a414c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 -- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 msgr2=0x7fe9a414c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 0x7fe9a414c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 -- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 msgr2=0x7fe9a41f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 0x7fe9a41f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 -- 172.21.15.44:0/397539176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a41f44c0 con 0x7fe9a4150ed0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3e00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 0x7fe9a41f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a3400640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe990005fd0 tx=0x7fe99000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990017070 con 0x7fe9a4150ed0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe990009d20 con 0x7fe9a4150ed0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a41f47a0 con 0x7fe9a4150ed0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.630+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990004280 con 0x7fe9a4150ed0 2024-07-17T06:55:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.631+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41f4d60 con 0x7fe9a4150ed0 2024-07-17T06:55:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.632+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a415e750 con 0x7fe9a4150ed0 2024-07-17T06:55:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.633+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe9900035f0 con 0x7fe9a4150ed0 2024-07-17T06:55:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.634+0000 7fe99be00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 0x7fe9840790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.634+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9900984b0 con 0x7fe9a4150ed0 2024-07-17T06:55:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.635+0000 7fe9a3e00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 0x7fe9840790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.636+0000 7fe9a3e00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 0x7fe9840790a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe994034000 tx=0x7fe994004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.638+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe990061520 con 0x7fe9a4150ed0 2024-07-17T06:55:09.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.859+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe9a406d520 con 0x7fe9a4150ed0 2024-07-17T06:55:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.860+0000 7fe99be00640 1 -- 172.21.15.44:0/397539176 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fe990060e20 con 0x7fe9a4150ed0 2024-07-17T06:55:09.860 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 msgr2=0x7fe9840790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 0x7fe9840790a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe994034000 tx=0x7fe994004730 comp rx=0 tx=0).stop 2024-07-17T06:55:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 msgr2=0x7fe9a41f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe990005fd0 tx=0x7fe99000dde0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 shutdown_connections 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe984076be0 0x7fe9840790a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a41512b0 0x7fe9a414c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4150ed0 0x7fe9a41f7e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 --2- 172.21.15.44:0/397539176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a406edc0 0x7fe9a41f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 >> 172.21.15.44:0/397539176 conn(0x7fe9a40cf0d0 msgr2=0x7fe9a414d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 shutdown_connections 2024-07-17T06:55:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:09.863+0000 7fe9a9c00640 1 -- 172.21.15.44:0/397539176 wait complete. 2024-07-17T06:55:09.885 INFO:teuthology.orchestra.run.smithi044.stdout:[{"version":27,"timestamp":"2024-07-17T06:54:41.694056+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"3933.278847"}]}] 2024-07-17T06:55:09.885 INFO:tasks.ceph_manager:config epoch is 27 2024-07-17T06:55:09.885 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-17T06:55:09.886 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-07-17T06:55:09.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-07-17T06:55:10.033 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:55:10.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:55:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 -- 172.21.15.44:0/3872361046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ee90 msgr2=0x7f54700ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 --2- 172.21.15.44:0/3872361046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ee90 0x7f54700ac760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5468009830 tx=0x7f546802a510 comp rx=0 tx=0).stop 2024-07-17T06:55:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 -- 172.21.15.44:0/3872361046 shutdown_connections 2024-07-17T06:55:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 --2- 172.21.15.44:0/3872361046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ee90 0x7f54700ac760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 --2- 172.21.15.44:0/3872361046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f547006ea50 0x7f54700ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 -- 172.21.15.44:0/3872361046 >> 172.21.15.44:0/3872361046 conn(0x7f54700b0fb0 msgr2=0x7f54700b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 -- 172.21.15.44:0/3872361046 shutdown_connections 2024-07-17T06:55:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.097+0000 7f5477a00640 1 -- 172.21.15.44:0/3872361046 wait complete. 2024-07-17T06:55:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.099+0000 7f5477a00640 1 Processor -- start 2024-07-17T06:55:10.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.099+0000 7f5477a00640 1 -- start start 2024-07-17T06:55:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ea50 0x7f54701614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f547006ee90 0x7f54701f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54700b2750 con 0x7f54700ae810 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54700b25d0 con 0x7f547006ea50 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.100+0000 7f5477a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54700b28d0 con 0x7f547006ee90 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37510/0 (socket says 172.21.15.44:37510) 2024-07-17T06:55:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 -- 172.21.15.44:0/1184358535 learned_addr learned my addr 172.21.15.44:0/1184358535 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475000640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ea50 0x7f54701614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 -- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f547006ee90 msgr2=0x7f54701f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f547006ee90 0x7f54701f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 -- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ea50 msgr2=0x7f54701614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ea50 0x7f54701614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.101+0000 7f5475a00640 1 -- 172.21.15.44:0/1184358535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5470071fb0 con 0x7f54700ae810 2024-07-17T06:55:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f5475a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f546802ab00 tx=0x7f5468032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468040070 con 0x7f54700ae810 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5468005310 con 0x7f54700ae810 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5470160c40 con 0x7f54700ae810 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468005c90 con 0x7f54700ae810 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.102+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5470160960 con 0x7f54700ae810 2024-07-17T06:55:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.104+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54700b2e20 con 0x7f54700ae810 2024-07-17T06:55:10.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.105+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f5468039040 con 0x7f54700ae810 2024-07-17T06:55:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.106+0000 7f546d600640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 0x7f544c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.106+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54680b97a0 con 0x7f54700ae810 2024-07-17T06:55:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.107+0000 7f5475000640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 0x7f544c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.108+0000 7f5475000640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 0x7f544c0790a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54600017a0 tx=0x7f5460001730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.110+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5468082800 con 0x7f54700ae810 2024-07-17T06:55:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.382+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f54700b23b0 con 0x7f54700ae810 2024-07-17T06:55:10.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.386+0000 7f546d600640 1 -- 172.21.15.44:0/1184358535 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+190588 (secure 0 0 0) 0x7f5468082100 con 0x7f54700ae810 2024-07-17T06:55:10.390 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.392+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 msgr2=0x7f544c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.392+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 0x7f544c0790a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54600017a0 tx=0x7f5460001730 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.392+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 msgr2=0x7f5470156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.392+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f546802ab00 tx=0x7f5468032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 shutdown_connections 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54700ae810 0x7f5470156d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f544c076be0 0x7f544c0790a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f547006ee90 0x7f54701f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 --2- 172.21.15.44:0/1184358535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f547006ea50 0x7f54701614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 >> 172.21.15.44:0/1184358535 conn(0x7f54700b0fb0 msgr2=0x7f5470074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:10.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 shutdown_connections 2024-07-17T06:55:10.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.393+0000 7f5477a00640 1 -- 172.21.15.44:0/1184358535 wait complete. 2024-07-17T06:55:10.414 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":557887042},{"type":"v1","addr":"172.21.15.175:6801","nonce":557887042}]},"active_addr":"172.21.15.175:6801/557887042","active_change":"2024-07-17T06:54:33.695836+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4140,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2038460701}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":733122570}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3025199526}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2225237696}]}]} 2024-07-17T06:55:10.419 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-07-17T06:55:10.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-07-17T06:55:10.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:10.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 -- 172.21.15.44:0/1237583142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2700cfa90 msgr2=0x7fb27006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 --2- 172.21.15.44:0/1237583142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2700cfa90 0x7fb27006e060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c02b860 comp rx=0 tx=0).stop 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 -- 172.21.15.44:0/1237583142 shutdown_connections 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 --2- 172.21.15.44:0/1237583142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 0x7fb270159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 --2- 172.21.15.44:0/1237583142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2700cfa90 0x7fb27006e060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 -- 172.21.15.44:0/1237583142 >> 172.21.15.44:0/1237583142 conn(0x7fb2700b0e80 msgr2=0x7fb2700b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.623+0000 7fb276000640 1 -- 172.21.15.44:0/1237583142 shutdown_connections 2024-07-17T06:55:10.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.624+0000 7fb276000640 1 -- 172.21.15.44:0/1237583142 wait complete. 2024-07-17T06:55:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.625+0000 7fb276000640 1 Processor -- start 2024-07-17T06:55:10.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.625+0000 7fb276000640 1 -- start start 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 0x7fb2700ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 0x7fb270147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2700b2120 con 0x7fb27006f4b0 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2700b1fa0 con 0x7fb2700cfa90 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.626+0000 7fb276000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700b22a0 con 0x7fb270159800 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37528/0 (socket says 172.21.15.44:37528) 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 -- 172.21.15.44:0/901882414 learned_addr learned my addr 172.21.15.44:0/901882414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb26fe00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 0x7fb2700ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb275600640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 0x7fb270147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 -- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 msgr2=0x7fb270147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 0x7fb270147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 -- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 msgr2=0x7fb2700ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 0x7fb2700ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb274c00640 1 -- 172.21.15.44:0/901882414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2700ce740 con 0x7fb27006f4b0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.627+0000 7fb26fe00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 0x7fb2700ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb274c00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c430 tx=0x7fb26400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264011790 con 0x7fb27006f4b0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb264017070 con 0x7fb27006f4b0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264011d00 con 0x7fb27006f4b0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb270147c60 con 0x7fb27006f4b0 2024-07-17T06:55:10.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.628+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2701f45e0 con 0x7fb27006f4b0 2024-07-17T06:55:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.629+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27006e060 con 0x7fb27006f4b0 2024-07-17T06:55:10.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.630+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fb26401c460 con 0x7fb27006f4b0 2024-07-17T06:55:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.631+0000 7fb26cc00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 0x7fb250079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.631+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb2640985c0 con 0x7fb27006f4b0 2024-07-17T06:55:10.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.632+0000 7fb26fe00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 0x7fb250079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:10.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.633+0000 7fb26fe00640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 0x7fb250079070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb25c035000 tx=0x7fb25c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:10.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.635+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb264061740 con 0x7fb27006f4b0 2024-07-17T06:55:10.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.846+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb270159fd0 con 0x7fb27006f4b0 2024-07-17T06:55:10.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.849+0000 7fb26cc00640 1 -- 172.21.15.44:0/901882414 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fb264061040 con 0x7fb27006f4b0 2024-07-17T06:55:10.849 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:10.849 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.851+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 msgr2=0x7fb250079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.851+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 0x7fb250079070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb25c035000 tx=0x7fb25c004730 comp rx=0 tx=0).stop 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.851+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 msgr2=0x7fb270146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.851+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb26400c430 tx=0x7fb26400c8f0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 shutdown_connections 2024-07-17T06:55:10.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb250076bb0 0x7fb250079070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270159800 0x7fb270147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2700cfa90 0x7fb2700ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 --2- 172.21.15.44:0/901882414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27006f4b0 0x7fb270146f40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 >> 172.21.15.44:0/901882414 conn(0x7fb2700b0e80 msgr2=0x7fb2700ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 shutdown_connections 2024-07-17T06:55:10.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:10.852+0000 7fb276000640 1 -- 172.21.15.44:0/901882414 wait complete. 2024-07-17T06:55:10.869 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-07-17T06:55:10.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:11.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 -- 172.21.15.44:0/3392804567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26340d1240 msgr2=0x7f26340d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 --2- 172.21.15.44:0/3392804567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26340d1240 0x7f26340d16b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2634075c80 tx=0x7f262c02a560 comp rx=0 tx=0).stop 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 -- 172.21.15.44:0/3392804567 shutdown_connections 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 --2- 172.21.15.44:0/3392804567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26340d1240 0x7f26340d16b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 --2- 172.21.15.44:0/3392804567 >> v2:172.21.15.175:3300/0 conn(0x7f26340b56b0 0x7f26340b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 --2- 172.21.15.44:0/3392804567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634158e70 0x7f263415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 -- 172.21.15.44:0/3392804567 >> 172.21.15.44:0/3392804567 conn(0x7f263406bf40 msgr2=0x7f263406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.081+0000 7f263ac00640 1 -- 172.21.15.44:0/3392804567 shutdown_connections 2024-07-17T06:55:11.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.082+0000 7f263ac00640 1 -- 172.21.15.44:0/3392804567 wait complete. 2024-07-17T06:55:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.084+0000 7f263ac00640 1 Processor -- start 2024-07-17T06:55:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.084+0000 7f263ac00640 1 -- start start 2024-07-17T06:55:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26340b56b0 0x7f2634150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340d1240 0x7f26340ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f263415bb50 con 0x7f2634158e70 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f263415b9d0 con 0x7f26340d1240 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.085+0000 7f263ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f263415bcd0 con 0x7f26340b56b0 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37544/0 (socket says 172.21.15.44:37544) 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 -- 172.21.15.44:0/1559071479 learned_addr learned my addr 172.21.15.44:0/1559071479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f2639800640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26340b56b0 0x7f2634150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f2638e00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340d1240 0x7f26340ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 -- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26340b56b0 msgr2=0x7f2634150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26340b56b0 0x7f2634150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 -- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340d1240 msgr2=0x7f26340ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340d1240 0x7f26340ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.086+0000 7f263a200640 1 -- 172.21.15.44:0/1559071479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26340ae1a0 con 0x7f2634158e70 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f263a200640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f262c0099c0 tx=0x7f262c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c03f070 con 0x7f2634158e70 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f262c0047a0 con 0x7f2634158e70 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26340ae480 con 0x7f2634158e70 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c035030 con 0x7f2634158e70 2024-07-17T06:55:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.087+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26340aea40 con 0x7f2634158e70 2024-07-17T06:55:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.088+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634074770 con 0x7f2634158e70 2024-07-17T06:55:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.090+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f262c042070 con 0x7f2634158e70 2024-07-17T06:55:11.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.091+0000 7f2631600640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 0x7f2614079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.091+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f262c0b9820 con 0x7f2634158e70 2024-07-17T06:55:11.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.092+0000 7f2639800640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 0x7f2614079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.093+0000 7f2639800640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 0x7f2614079070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2624005fd0 tx=0x7f26240097d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.095+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f262c082a90 con 0x7f2634158e70 2024-07-17T06:55:11.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.306+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26340d16b0 con 0x7f2634158e70 2024-07-17T06:55:11.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.309+0000 7f2631600640 1 -- 172.21.15.44:0/1559071479 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f262c082390 con 0x7f2634158e70 2024-07-17T06:55:11.309 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:11.309 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 msgr2=0x7f2614079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 0x7f2614079070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2624005fd0 tx=0x7f26240097d0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 msgr2=0x7f263415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f262c0099c0 tx=0x7f262c003910 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 shutdown_connections 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634158e70 0x7f263415a4b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340d1240 0x7f26340ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2614076bb0 0x7f2614079070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 --2- 172.21.15.44:0/1559071479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26340b56b0 0x7f2634150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 >> 172.21.15.44:0/1559071479 conn(0x7f263406bf40 msgr2=0x7f2634153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 shutdown_connections 2024-07-17T06:55:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.312+0000 7f263ac00640 1 -- 172.21.15.44:0/1559071479 wait complete. 2024-07-17T06:55:11.332 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-07-17T06:55:11.332 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-07-17T06:55:11.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-07-17T06:55:11.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-07-17T06:55:11.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-07-17T06:55:11.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-07-17T06:55:11.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-07-17T06:55:11.335 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-07-17T06:55:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 -- 172.21.15.44:0/2823958624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a400aa90 msgr2=0x7fe9a401cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 --2- 172.21.15.44:0/2823958624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a400aa90 0x7fe9a401cd60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009830 tx=0x7fe9a002a510 comp rx=0 tx=0).stop 2024-07-17T06:55:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 -- 172.21.15.44:0/2823958624 shutdown_connections 2024-07-17T06:55:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 --2- 172.21.15.44:0/2823958624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a400aa90 0x7fe9a401cd60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 -- 172.21.15.44:0/2823958624 >> 172.21.15.44:0/2823958624 conn(0x7fe9a4054590 msgr2=0x7fe9a4054970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 -- 172.21.15.44:0/2823958624 shutdown_connections 2024-07-17T06:55:11.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.636+0000 7fe822200640 1 -- 172.21.15.44:0/3895670663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81403ab40 msgr2=0x7fe814006770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.636+0000 7fe822200640 1 --2- 172.21.15.44:0/3895670663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81403ab40 0x7fe814006770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe810009830 tx=0x7fe81002a4a0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.634+0000 7fe9b4000640 1 -- 172.21.15.44:0/2823958624 wait complete. 2024-07-17T06:55:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 -- 172.21.15.44:0/3895670663 shutdown_connections 2024-07-17T06:55:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 --2- 172.21.15.44:0/3895670663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8140379e0 0x7fe814056d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 --2- 172.21.15.44:0/3895670663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81403ab40 0x7fe814006770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 -- 172.21.15.44:0/3895670663 >> 172.21.15.44:0/3895670663 conn(0x7fe814055570 msgr2=0x7fe814055950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe9b4000640 1 Processor -- start 2024-07-17T06:55:11.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 -- 172.21.15.44:0/3895670663 shutdown_connections 2024-07-17T06:55:11.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe822200640 1 -- 172.21.15.44:0/3895670663 wait complete. 2024-07-17T06:55:11.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.637+0000 7fe9b4000640 1 -- start start 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.638+0000 7fddaf200640 1 -- 172.21.15.44:0/3949076621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 msgr2=0x7fddb0071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.638+0000 7fddaf200640 1 --2- 172.21.15.44:0/3949076621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 0x7fddb0071920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009830 tx=0x7fdd9c02a4a0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a400aa90 0x7fe9a404dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 0x7fe9a40ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4036720 0x7fe9a40de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a40ebe80 con 0x7fe9a4036720 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a40ebd00 con 0x7fe9a400aa90 2024-07-17T06:55:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a40ec000 con 0x7fe9a400ba30 2024-07-17T06:55:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 -- 172.21.15.44:0/3949076621 shutdown_connections 2024-07-17T06:55:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7f167f600640 1 -- 172.21.15.44:0/737812561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16780b6c20 msgr2=0x7f16780d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7f167f600640 1 --2- 172.21.15.44:0/737812561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16780b6c20 0x7f16780d2bc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1670009ca0 tx=0x7f167002a8c0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 --2- 172.21.15.44:0/3949076621 >> v2:172.21.15.175:3300/0 conn(0x7fddb01619e0 0x7fddb015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 0x7fe9a40ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 0x7fe9a40ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58040/0 (socket says 172.21.15.44:58040) 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.639+0000 7fe9b0c00640 1 -- 172.21.15.44:0/1329040821 learned_addr learned my addr 172.21.15.44:0/1329040821 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 --2- 172.21.15.44:0/3949076621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 0x7fddb0071920 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 -- 172.21.15.44:0/3949076621 >> 172.21.15.44:0/3949076621 conn(0x7fddb006cc30 msgr2=0x7fddb00b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7f167f600640 1 -- 172.21.15.44:0/737812561 shutdown_connections 2024-07-17T06:55:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 -- 172.21.15.44:0/3949076621 shutdown_connections 2024-07-17T06:55:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b2000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4036720 0x7fe9a40de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b1600640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a400aa90 0x7fe9a404dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7f167f600640 1 --2- 172.21.15.44:0/737812561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16780b6c20 0x7f16780d2bc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7f167f600640 1 -- 172.21.15.44:0/737812561 >> 172.21.15.44:0/737812561 conn(0x7f16780d3440 msgr2=0x7f16780d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.641+0000 7fe822200640 1 Processor -- start 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b0c00640 1 -- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a400aa90 msgr2=0x7fe9a404dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b0c00640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a400aa90 0x7fe9a404dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b0c00640 1 -- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4036720 msgr2=0x7fe9a40de1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b0c00640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9a4036720 0x7fe9a40de1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fe9b0c00640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a404e3b0 con 0x7fe9a400ba30 2024-07-17T06:55:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.641+0000 7f167f600640 1 -- 172.21.15.44:0/737812561 shutdown_connections 2024-07-17T06:55:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.641+0000 7f167f600640 1 -- 172.21.15.44:0/737812561 wait complete. 2024-07-17T06:55:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.641+0000 7fe822200640 1 -- start start 2024-07-17T06:55:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.640+0000 7fddaf200640 1 -- 172.21.15.44:0/3949076621 wait complete. 2024-07-17T06:55:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.642+0000 7fe9b0c00640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 0x7fe9a40ddc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b410 tx=0x7fe99800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.642+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998015020 con 0x7fe9a400ba30 2024-07-17T06:55:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.642+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe998011920 con 0x7fe9a400ba30 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.642+0000 7fe822200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 0x7fe814180730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe822200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe81403ab40 0x7fe8140e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe822200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8140e2b20 0x7fe8140e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fddaf200640 1 Processor -- start 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.642+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998011d70 con 0x7fe9a400ba30 2024-07-17T06:55:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a4050310 con 0x7fe9a400ba30 2024-07-17T06:55:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fddaf200640 1 -- start start 2024-07-17T06:55:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a40508d0 con 0x7fe9a400ba30 2024-07-17T06:55:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe822200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe814057630 con 0x7fe8140379e0 2024-07-17T06:55:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe822200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140574b0 con 0x7fe81403ab40 2024-07-17T06:55:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.643+0000 7fe822200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140577b0 con 0x7fe8140e2b20 2024-07-17T06:55:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7f167f600640 1 Processor -- start 2024-07-17T06:55:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe81b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 0x7fe814180730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe81b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 0x7fe814180730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37586/0 (socket says 172.21.15.44:37586) 2024-07-17T06:55:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7f167f600640 1 -- start start 2024-07-17T06:55:11.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe81403ab40 0x7fe8140e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe81b400640 1 -- 172.21.15.44:0/2609273691 learned_addr learned my addr 172.21.15.44:0/2609273691 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9a4036010 con 0x7fe9a400ba30 2024-07-17T06:55:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fe81be00640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8140e2b20 0x7fe8140e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.644+0000 7fddaf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 0x7fddb01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddaf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 0x7fddb00cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddaf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb01619e0 0x7fddb014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddaf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddb015d860 con 0x7fddb01619e0 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddaf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddb015d6e0 con 0x7fddb0160ef0 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddaf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddb015d9e0 con 0x7fddb00737e0 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7f167f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 0x7f16781f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.645+0000 7fddad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 0x7fddb00cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fddad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 0x7fddb00cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44492/0 (socket says 172.21.15.44:44492) 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7f167f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678160a20 0x7f167814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7fddad400640 1 -- 172.21.15.44:0/1205075123 learned_addr learned my addr 172.21.15.44:0/1205075123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fe81b400640 1 -- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8140e2b20 msgr2=0x7fe8140e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fe81b400640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8140e2b20 0x7fe8140e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fe81b400640 1 -- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe81403ab40 msgr2=0x7fe8140e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fe81b400640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe81403ab40 0x7fe8140e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.646+0000 7fe81b400640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81404f950 con 0x7fe8140379e0 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678160e00 0x7f16780c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1678168970 con 0x7f16780b6c20 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16781687f0 con 0x7f1678160e00 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1678168af0 con 0x7f1678160a20 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 0x7f16781f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 0x7f16781f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37602/0 (socket says 172.21.15.44:37602) 2024-07-17T06:55:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167cc00640 1 -- 172.21.15.44:0/109690073 learned_addr learned my addr 172.21.15.44:0/109690073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7f167d600640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678160e00 0x7f16780c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7fddae800640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb01619e0 0x7fddb014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.647+0000 7fddade00640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 0x7fddb01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.648+0000 7f166fe00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678160a20 0x7f167814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.648+0000 7fe81b400640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 0x7fe814180730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c730 tx=0x7fe80800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.648+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe998018020 con 0x7fe9a400ba30 2024-07-17T06:55:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808017070 con 0x7fe8140379e0 2024-07-17T06:55:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe808005290 con 0x7fe8140379e0 2024-07-17T06:55:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800a8a0 con 0x7fe8140379e0 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7f167cc00640 1 -- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678160a20 msgr2=0x7f167814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7f167cc00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678160a20 0x7f167814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7f167cc00640 1 -- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678160e00 msgr2=0x7f16780c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7f167cc00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678160e00 0x7f16780c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7f167cc00640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167814d850 con 0x7f16780b6c20 2024-07-17T06:55:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81404fc30 con 0x7fe8140379e0 2024-07-17T06:55:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8140501f0 con 0x7fe8140379e0 2024-07-17T06:55:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fddad400640 1 -- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 msgr2=0x7fddb01592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fddad400640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00737e0 0x7fddb01592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fddad400640 1 -- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb01619e0 msgr2=0x7fddb014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fddad400640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb01619e0 0x7fddb014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fe9a9600640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe98c076c00 0x7fe98c0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe998098c20 con 0x7fe9a400ba30 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9a9600640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7fe98c080380 0x7fe98c0827c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe998018ec0 con 0x7fe98c080380 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe998098fd0 con 0x7fe9a400ba30 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9b1600640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe98c076c00 0x7fe98c0790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fe9b2000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7fe98c080380 0x7fe98c0827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7f167cc00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 0x7f16781f84d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f166000c410 tx=0x7f166000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.649+0000 7fddad400640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0159c10 con 0x7fddb0160ef0 2024-07-17T06:55:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.650+0000 7fddae800640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb01619e0 0x7fddb014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:55:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7fddad400640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 0x7fddb00cf340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009950 tx=0x7fdd9c032c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660017070 con 0x7f16780b6c20 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16600035f0 con 0x7f16780b6c20 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660003a40 con 0x7f16780b6c20 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.652+0000 7f167f600640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1678156780 con 0x7f16780b6c20 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.652+0000 7f167f600640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16781540a0 con 0x7f16780b6c20 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c040070 con 0x7fddb0160ef0 2024-07-17T06:55:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.653+0000 7fe9b1600640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe98c076c00 0x7fe98c0790c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06af30 tx=0x7fe9ac06b9a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.651+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb0159ef0 con 0x7fddb0160ef0 2024-07-17T06:55:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.652+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb015e100 con 0x7fddb0160ef0 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.653+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2375526381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc06c950 msgr2=0x7fc5dc0d28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.653+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2375526381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc06c950 0x7fc5dc0d28d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc06d640 tx=0x7fc5d402a8f0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2375526381 shutdown_connections 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2375526381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc06c950 0x7fc5dc0d28d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2375526381 >> v2:172.21.15.44:3300/0 conn(0x7fc5dc15d740 0x7fc5dc06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1648001030 con 0x7f16780b6c20 2024-07-17T06:55:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd9c005220 con 0x7fddb0160ef0 2024-07-17T06:55:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c005650 con 0x7fddb0160ef0 2024-07-17T06:55:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2375526381 >> 172.21.15.44:0/2375526381 conn(0x7fc5dc0d00f0 msgr2=0x7fc5dc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2375526381 shutdown_connections 2024-07-17T06:55:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.655+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd88001030 con 0x7fddb0160ef0 2024-07-17T06:55:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.655+0000 7fe9b2000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7fe98c080380 0x7fe98c0827c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8140078e0 con 0x7fe8140379e0 2024-07-17T06:55:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.656+0000 7f1696c00640 1 -- 172.21.15.44:0/2082435629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901608a0 msgr2=0x7f1690160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.656+0000 7f1696c00640 1 --2- 172.21.15.44:0/2082435629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901608a0 0x7f1690160d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f16900cdce0 tx=0x7f167802a6a0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.656+0000 7f1696c00640 1 -- 172.21.15.44:0/2082435629 shutdown_connections 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.654+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2375526381 wait complete. 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.656+0000 7f1696c00640 1 --2- 172.21.15.44:0/2082435629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16901608a0 0x7f1690160d30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7f1696c00640 1 --2- 172.21.15.44:0/2082435629 >> v2:172.21.15.44:3300/0 conn(0x7f16900b9410 0x7f1690160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7f1696c00640 1 -- 172.21.15.44:0/2082435629 >> 172.21.15.44:0/2082435629 conn(0x7f16900d3700 msgr2=0x7f16900d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.656+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe80801a070 con 0x7fe8140379e0 2024-07-17T06:55:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fdd9c039040 con 0x7fddb0160ef0 2024-07-17T06:55:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7fc5e0a00640 1 Processor -- start 2024-07-17T06:55:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7f1696c00640 1 -- 172.21.15.44:0/2082435629 shutdown_connections 2024-07-17T06:55:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7fc5e0a00640 1 -- start start 2024-07-17T06:55:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.658+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== osd.2 v2:172.21.15.44:6821/2788335734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe998018ec0 con 0x7fe98c080380 2024-07-17T06:55:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.658+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f166001a070 con 0x7f16780b6c20 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.658+0000 7fe80f400640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe7fc076c30 0x7fe7fc0790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.658+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe808097970 con 0x7fe8140379e0 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fe80f400640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7fe7fc0803b0 0x7fe7fc0827f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe808016ec0 con 0x7fe7fc0803b0 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe80809c2c0 con 0x7fe8140379e0 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fe81aa00640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe7fc076c30 0x7fe7fc0790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7f7571000640 1 -- 172.21.15.44:0/541462351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0b6790 msgr2=0x7f756c1648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7f7571000640 1 --2- 172.21.15.44:0/541462351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0b6790 0x7f756c1648f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f755c009830 tx=0x7f755c02a5a0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5dc06c950 0x7fc5dc0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc0ce710 0x7fc5dc150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 0x7fc5dc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc15dca0 con 0x7fc5dc15d740 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc15db20 con 0x7fc5dc0ce710 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7fc5e0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc0ca370 con 0x7fc5dc06c950 2024-07-17T06:55:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7f29c1000640 1 -- 172.21.15.44:0/1469667540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b29e0 msgr2=0x7f29bc161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.659+0000 7f29c1000640 1 --2- 172.21.15.44:0/1469667540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b29e0 0x7f29bc161b90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f29bc074630 tx=0x7f29b402a8c0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f1675600640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1654076c80 0x7f1654079140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.657+0000 7f1696c00640 1 -- 172.21.15.44:0/2082435629 wait complete. 2024-07-17T06:55:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fe81be00640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7fe7fc0803b0 0x7fe7fc0827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f7571000640 1 -- 172.21.15.44:0/541462351 shutdown_connections 2024-07-17T06:55:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f7571000640 1 --2- 172.21.15.44:0/541462351 >> v2:172.21.15.44:3300/0 conn(0x7f756c0b29e0 0x7f756c166e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f7571000640 1 --2- 172.21.15.44:0/541462351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0b6790 0x7f756c1648f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f7571000640 1 -- 172.21.15.44:0/541462351 >> 172.21.15.44:0/541462351 conn(0x7f756c0d3700 msgr2=0x7f756c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fc5da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 0x7fc5dc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fc5d9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5dc06c950 0x7fc5dc0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fc5da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 0x7fc5dc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37658/0 (socket says 172.21.15.44:37658) 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f7571000640 1 -- 172.21.15.44:0/541462351 shutdown_connections 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1660098180 con 0x7f16780b6c20 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f1675600640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7f1654080400 0x7f1654082840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1654082f10 con 0x7f1654080400 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1660098570 con 0x7f16780b6c20 2024-07-17T06:55:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fc5da800640 1 -- 172.21.15.44:0/2500157212 learned_addr learned my addr 172.21.15.44:0/2500157212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fc5d9400640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc0ce710 0x7fc5dc150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f29c1000640 1 -- 172.21.15.44:0/1469667540 shutdown_connections 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fdda6000640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdd90076bb0 0x7fdd90079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd9c0b9600 con 0x7fddb0160ef0 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fdda6000640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fdd90080330 0x7fdd90082770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.660+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd90082e40 con 0x7fdd90080330 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c0b99f0 con 0x7fddb0160ef0 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f29c1000640 1 --2- 172.21.15.44:0/1469667540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b6790 0x7f29bc161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f29c1000640 1 --2- 172.21.15.44:0/1469667540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b29e0 0x7f29bc161b90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7fc5da800640 1 -- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5dc06c950 msgr2=0x7fc5dc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7fc5da800640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5dc06c950 0x7fc5dc0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7fc5da800640 1 -- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc0ce710 msgr2=0x7fc5dc150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f29c1000640 1 --2- 172.21.15.44:0/1469667540 >> v2:172.21.15.44:3300/0 conn(0x7f29bc0ce1d0 0x7f29bc0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f167d600640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7f1654080400 0x7f1654082840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f29c1000640 1 -- 172.21.15.44:0/1469667540 >> 172.21.15.44:0/1469667540 conn(0x7f29bc0d3700 msgr2=0x7f29bc0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f1696c00640 1 Processor -- start 2024-07-17T06:55:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f1696c00640 1 -- start start 2024-07-17T06:55:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f166fe00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1654076c80 0x7f1654079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7fddae800640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fdd90080330 0x7fdd90082770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fddade00640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdd90076bb0 0x7fdd90079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fe81aa00640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe7fc076c30 0x7fe7fc0790f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe810034000 tx=0x7fe810004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fc5da800640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc0ce710 0x7fc5dc150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fc5da800640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1fc840 con 0x7fc5dc15d740 2024-07-17T06:55:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fe81be00640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7fe7fc0803b0 0x7fe7fc0827f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7fc5d9400640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5dc0ce710 0x7fc5dc150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7f29c1000640 1 -- 172.21.15.44:0/1469667540 shutdown_connections 2024-07-17T06:55:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.661+0000 7f7571000640 1 -- 172.21.15.44:0/541462351 wait complete. 2024-07-17T06:55:11.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7f167d600640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7f1654080400 0x7f1654082840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.662+0000 7f29c1000640 1 -- 172.21.15.44:0/1469667540 wait complete. 2024-07-17T06:55:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7fc5da800640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 0x7fc5dc15ecb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009bf0 tx=0x7fc5d4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f166fe00640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1654076c80 0x7f1654079140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16781561f0 tx=0x7f1668009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16900b9410 0x7f169015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 0x7f16900b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16901608a0 0x7f16900b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16901613f0 con 0x7f16900b9410 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1690161270 con 0x7f16900ce1d0 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7f1696c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1690161570 con 0x7f16901608a0 2024-07-17T06:55:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7fddae800640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fdd90080330 0x7fdd90082770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004040 con 0x7fc5dc15d740 2024-07-17T06:55:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f168f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 0x7f16900b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.663+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5d402d070 con 0x7fc5dc15d740 2024-07-17T06:55:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004420 con 0x7fc5dc15d740 2024-07-17T06:55:11.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7fddade00640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdd90076bb0 0x7fdd90079070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdda0004ea0 tx=0x7fdda0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc0b3b40 con 0x7fc5dc15d740 2024-07-17T06:55:11.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== osd.3 v2:172.21.15.44:6800/2705860470 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1654082f10 con 0x7f1654080400 2024-07-17T06:55:11.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== osd.0 v2:172.21.15.44:6812/429081112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe808016ec0 con 0x7fe7fc0803b0 2024-07-17T06:55:11.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f29c1000640 1 Processor -- start 2024-07-17T06:55:11.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f1694c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16901608a0 0x7f16900b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f168f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 0x7f16900b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44538/0 (socket says 172.21.15.44:44538) 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f168f400640 1 -- 172.21.15.44:0/2771463657 learned_addr learned my addr 172.21.15.44:0/2771463657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.664+0000 7f168fe00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16900b9410 0x7f169015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 -- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16901608a0 msgr2=0x7f16900b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16901608a0 0x7f16900b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 -- 172.21.15.44:0/2771463657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16900b9410 msgr2=0x7f169015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16900b9410 0x7f169015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16901fc7b0 con 0x7f16900ce1d0 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f1694c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16901608a0 0x7f16900b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:55:11.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f7571000640 1 Processor -- start 2024-07-17T06:55:11.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc0b4100 con 0x7fc5dc15d740 2024-07-17T06:55:11.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f7571000640 1 -- start start 2024-07-17T06:55:11.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f168f400640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 0x7f16900b3940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f168000cd10 tx=0x7f168000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000f650 con 0x7f16900ce1d0 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f168000f8e0 con 0x7f16900ce1d0 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000fd30 con 0x7f16900ce1d0 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.665+0000 7f29c1000640 1 -- start start 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 0x7f29bc1f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b6790 0x7f29bc0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ce1d0 0x7f29bc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc162250 con 0x7f29bc0ce1d0 2024-07-17T06:55:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1620d0 con 0x7f29bc0b6790 2024-07-17T06:55:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.666+0000 7f29c1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc1623d0 con 0x7f29bc0b29e0 2024-07-17T06:55:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== osd.1 v2:172.21.15.44:6802/3861011633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd90082e40 con 0x7fdd90080330 2024-07-17T06:55:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f29ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ce1d0 0x7f29bc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f7571000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756c0b29e0 0x7f756c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5dc0762e0 con 0x7fc5dc15d740 2024-07-17T06:55:11.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f7571000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f756c0b6790 0x7f756c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f7571000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0ce1d0 0x7f756c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f29b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 0x7f29bc1f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f29b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 0x7f29bc1f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58116/0 (socket says 172.21.15.44:58116) 2024-07-17T06:55:11.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.667+0000 7f29b9e00640 1 -- 172.21.15.44:0/1079770623 learned_addr learned my addr 172.21.15.44:0/1079770623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7571000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f756c167710 con 0x7f756c0b29e0 2024-07-17T06:55:11.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7571000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f756c167590 con 0x7f756c0b6790 2024-07-17T06:55:11.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7571000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f756c167890 con 0x7f756c0ce1d0 2024-07-17T06:55:11.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f29b3e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b6790 0x7f29bc0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f756a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0ce1d0 0x7f756c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f29b9e00640 1 -- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b6790 msgr2=0x7f29bc0b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f29b9e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b6790 0x7f29bc0b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f29b9e00640 1 -- 172.21.15.44:0/1079770623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ce1d0 msgr2=0x7f29bc15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7569400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f756c0b6790 0x7f756c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7569400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f756c0b6790 0x7f756c0c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44544/0 (socket says 172.21.15.44:44544) 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7569400640 1 -- 172.21.15.44:0/919616609 learned_addr learned my addr 172.21.15.44:0/919616609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.668+0000 7f7569e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756c0b29e0 0x7f756c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f29b9e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ce1d0 0x7f29bc15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f29b9e00640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1f9640 con 0x7f29bc0b29e0 2024-07-17T06:55:11.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f29ba800640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29bc0ce1d0 0x7f29bc15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:11.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc5d40045c0 con 0x7fc5dc15d740 2024-07-17T06:55:11.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f29b3e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29bc0b6790 0x7f29bc0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:11.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1690149160 con 0x7f16900ce1d0 2024-07-17T06:55:11.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.669+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1690149720 con 0x7f16900ce1d0 2024-07-17T06:55:11.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f29b9e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 0x7f29bc1f9100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00d750 tx=0x7f29ac00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f7569e00640 1 -- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0ce1d0 msgr2=0x7f756c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f7569e00640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f756c0ce1d0 0x7f756c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f7569e00640 1 -- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f756c0b6790 msgr2=0x7f756c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f7569e00640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f756c0b6790 0x7f756c0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.670+0000 7f7569e00640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c0ca270 con 0x7f756c0b29e0 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7569e00640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756c0b29e0 0x7f756c1f4800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f755400c390 tx=0x7f7554001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac017070 con 0x7f29bc0b29e0 2024-07-17T06:55:11.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554017070 con 0x7f756c0b29e0 2024-07-17T06:55:11.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29ac004310 con 0x7f29bc0b29e0 2024-07-17T06:55:11.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75540035f0 con 0x7f756c0b29e0 2024-07-17T06:55:11.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554003ac0 con 0x7f756c0b29e0 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00de60 con 0x7f29bc0b29e0 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7fc5d2000640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5bc076c00 0x7fc5bc0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc5d40ba170 con 0x7fc5dc15d740 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7fc5d2000640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fc5bc080370 0x7fc5bc0827b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5bc082e80 con 0x7fc5bc080370 2024-07-17T06:55:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40c1050 con 0x7fc5dc15d740 2024-07-17T06:55:11.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c155b40 con 0x7f756c0b29e0 2024-07-17T06:55:11.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.671+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c156100 con 0x7f756c0b29e0 2024-07-17T06:55:11.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.673+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7544001030 con 0x7f756c0b29e0 2024-07-17T06:55:11.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.674+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc0c9a60 con 0x7f29bc0b29e0 2024-07-17T06:55:11.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.674+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc0ca020 con 0x7f29bc0b29e0 2024-07-17T06:55:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.674+0000 7fc5d9400640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fc5bc080370 0x7fc5bc0827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.674+0000 7fc5d9e00640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5bc076c00 0x7fc5bc0790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.674+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f1680018400 con 0x7f16900ce1d0 2024-07-17T06:55:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.675+0000 7fc5d9400640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fc5bc080370 0x7fc5bc0827b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.676+0000 7fc5d9e00640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5bc076c00 0x7fc5bc0790c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005e50 tx=0x7fc5c8005de0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.676+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1690160d30 con 0x7f16900ce1d0 2024-07-17T06:55:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.676+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29bc0ce5b0 con 0x7f29bc0b29e0 2024-07-17T06:55:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.676+0000 7f1687e00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f166c076bb0 0x7f166c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.675+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f755401a070 con 0x7f756c0b29e0 2024-07-17T06:55:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.676+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f16800990f0 con 0x7f16900ce1d0 2024-07-17T06:55:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7562000640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7548076c30 0x7f75480790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7554097700 con 0x7f756c0b29e0 2024-07-17T06:55:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f1687e00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f166c0802a0 0x7f166c0826e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f166c082db0 con 0x7f166c0802a0 2024-07-17T06:55:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16800a0050 con 0x7f16900ce1d0 2024-07-17T06:55:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f168fe00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f166c076bb0 0x7f166c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7562000640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f75480803b0 0x7f75480827f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f755401be50 con 0x7f75480803b0 2024-07-17T06:55:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f755409e050 con 0x7f756c0b29e0 2024-07-17T06:55:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.677+0000 7f7569400640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7548076c30 0x7f75480790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.678+0000 7f1694c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f166c0802a0 0x7f166c0826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.678+0000 7f756a800640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f75480803b0 0x7f75480827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.679+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== osd.4 v2:172.21.15.175:6802/2954959872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5bc082e80 con 0x7fc5bc080370 2024-07-17T06:55:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.679+0000 7f7569400640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7548076c30 0x7f75480790f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f756c0d3230 tx=0x7f755c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.679+0000 7f756a800640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f75480803b0 0x7f75480827f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.679+0000 7f168fe00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f166c076bb0 0x7f166c079070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1688000f60 tx=0x7f1688009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.679+0000 7f1694c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f166c0802a0 0x7f166c0826e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.680+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f29ac009c30 con 0x7f29bc0b29e0 2024-07-17T06:55:11.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.680+0000 7f29b2000640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2998076c00 0x7f29980790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.681+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f29ac05e080 con 0x7f29bc0b29e0 2024-07-17T06:55:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.681+0000 7f29b2000640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f2998081020 0x7f29980834c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.681+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2998083b90 con 0x7f2998081020 2024-07-17T06:55:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.681+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29ac05e280 con 0x7f29bc0b29e0 2024-07-17T06:55:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.681+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== osd.7 v2:172.21.15.175:6826/2516790419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f755401be50 con 0x7f75480803b0 2024-07-17T06:55:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.682+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== osd.6 v2:172.21.15.175:6803/302428054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f166c082db0 con 0x7f166c0802a0 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.683+0000 7f29b3e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2998076c00 0x7f29980790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.683+0000 7f29ba800640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f2998081020 0x7f29980834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.684+0000 7f29ba800640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f2998081020 0x7f29980834c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.686+0000 7f29b3e00640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2998076c00 0x7f29980790c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0c6bc0 tx=0x7f29a4009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.686+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== osd.5 v2:172.21.15.175:6818/1413905252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2998083b90 con 0x7f2998081020 2024-07-17T06:55:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.689+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 --> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe814057c60 con 0x7fe7fc0803b0 2024-07-17T06:55:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.694+0000 7fe80f400640 1 -- 172.21.15.44:0/2609273691 <== osd.0 v2:172.21.15.44:6812/429081112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe814057c60 con 0x7fe7fc0803b0 2024-07-17T06:55:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7fe7fc0803b0 msgr2=0x7fe7fc0827f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7fe7fc0803b0 0x7fe7fc0827f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe7fc076c30 msgr2=0x7fe7fc0790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe7fc076c30 0x7fe7fc0790f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe810034000 tx=0x7fe810004730 comp rx=0 tx=0).stop 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 msgr2=0x7fe814180730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe822200640 1 --2- 172.21.15.44:0/2609273691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8140379e0 0x7fe814180730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c730 tx=0x7fe80800cbf0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.695+0000 7fe81be00640 1 -- 172.21.15.44:0/2609273691 reap_dead start 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.696+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 shutdown_connections 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.696+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 >> 172.21.15.44:0/2609273691 conn(0x7fe814055570 msgr2=0x7fe81400e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.696+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 --> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1648002dd0 con 0x7f1654080400 2024-07-17T06:55:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.696+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 shutdown_connections 2024-07-17T06:55:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.697+0000 7fe822200640 1 -- 172.21.15.44:0/2609273691 wait complete. 2024-07-17T06:55:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.697+0000 7f1675600640 1 -- 172.21.15.44:0/109690073 <== osd.3 v2:172.21.15.44:6800/2705860470 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1648002dd0 con 0x7f1654080400 2024-07-17T06:55:11.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.697+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 --> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9a401d9d0 con 0x7fe98c080380 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.697+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7f1654080400 msgr2=0x7f1654082840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7f1654080400 0x7f1654082840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1654076c80 msgr2=0x7f1654079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1654076c80 0x7f1654079140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16781561f0 tx=0x7f1668009060 comp rx=0 tx=0).stop 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 msgr2=0x7f16781f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 --2- 172.21.15.44:0/109690073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16780b6c20 0x7f16781f84d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f166000c410 tx=0x7f166000c8d0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f167d600640 1 -- 172.21.15.44:0/109690073 reap_dead start 2024-07-17T06:55:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 shutdown_connections 2024-07-17T06:55:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 >> 172.21.15.44:0/109690073 conn(0x7f16780d3440 msgr2=0x7f1678068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7fe9a9600640 1 -- 172.21.15.44:0/1329040821 <== osd.2 v2:172.21.15.44:6821/2788335734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe9a401d9d0 con 0x7fe98c080380 2024-07-17T06:55:11.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.698+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 shutdown_connections 2024-07-17T06:55:11.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7f166e000640 1 -- 172.21.15.44:0/109690073 wait complete. 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7fe98c080380 msgr2=0x7fe98c0827c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7fe98c080380 0x7fe98c0827c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe98c076c00 msgr2=0x7fe98c0790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe98c076c00 0x7fe98c0790c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06af30 tx=0x7fe9ac06b9a0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 msgr2=0x7fe9a40ddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.699+0000 7fe9b4000640 1 --2- 172.21.15.44:0/1329040821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9a400ba30 0x7fe9a40ddc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b410 tx=0x7fe99800b8d0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.700+0000 7fe9b2000640 1 -- 172.21.15.44:0/1329040821 reap_dead start 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.700+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 shutdown_connections 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.700+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 >> 172.21.15.44:0/1329040821 conn(0x7fe9a4054590 msgr2=0x7fe9a40386b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.701+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 shutdown_connections 2024-07-17T06:55:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.701+0000 7fe9b4000640 1 -- 172.21.15.44:0/1329040821 wait complete. 2024-07-17T06:55:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.702+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 --> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd88002dd0 con 0x7fdd90080330 2024-07-17T06:55:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.703+0000 7fdda6000640 1 -- 172.21.15.44:0/1205075123 <== osd.1 v2:172.21.15.44:6802/3861011633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd88002dd0 con 0x7fdd90080330 2024-07-17T06:55:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.703+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fdd90080330 msgr2=0x7fdd90082770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.703+0000 7fddaf200640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fdd90080330 0x7fdd90082770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdd90076bb0 msgr2=0x7fdd90079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fdd90076bb0 0x7fdd90079070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdda0004ea0 tx=0x7fdda0009040 comp rx=0 tx=0).stop 2024-07-17T06:55:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 msgr2=0x7fddb00cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 --2- 172.21.15.44:0/1205075123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb0160ef0 0x7fddb00cf340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009950 tx=0x7fdd9c032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddae800640 1 -- 172.21.15.44:0/1205075123 reap_dead start 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 shutdown_connections 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.706+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 >> 172.21.15.44:0/1205075123 conn(0x7fddb006cc30 msgr2=0x7fddb00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.708+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 shutdown_connections 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.708+0000 7fddaf200640 1 -- 172.21.15.44:0/1205075123 wait complete. 2024-07-17T06:55:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.713+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 --> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5dc0d2ef0 con 0x7fc5bc080370 2024-07-17T06:55:11.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.714+0000 7fc5d2000640 1 -- 172.21.15.44:0/2500157212 <== osd.4 v2:172.21.15.175:6802/2954959872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5dc0d2ef0 con 0x7fc5bc080370 2024-07-17T06:55:11.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.715+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fc5bc080370 msgr2=0x7fc5bc0827b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.715+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fc5bc080370 0x7fc5bc0827b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5bc076c00 msgr2=0x7fc5bc0790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5bc076c00 0x7fc5bc0790c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005e50 tx=0x7fc5c8005de0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 msgr2=0x7fc5dc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 --2- 172.21.15.44:0/2500157212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5dc15d740 0x7fc5dc15ecb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009bf0 tx=0x7fc5d4003ba0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5da800640 1 -- 172.21.15.44:0/2500157212 reap_dead start 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 shutdown_connections 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.716+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 >> 172.21.15.44:0/2500157212 conn(0x7fc5dc0d00f0 msgr2=0x7fc5dc160040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.717+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 shutdown_connections 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.717+0000 7fc5e0a00640 1 -- 172.21.15.44:0/2500157212 wait complete. 2024-07-17T06:55:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.717+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 --> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1690149920 con 0x7f166c0802a0 2024-07-17T06:55:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.719+0000 7f1687e00640 1 -- 172.21.15.44:0/2771463657 <== osd.6 v2:172.21.15.175:6803/302428054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1690149920 con 0x7f166c0802a0 2024-07-17T06:55:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.719+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 --> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7544002dd0 con 0x7f75480803b0 2024-07-17T06:55:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.720+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 --> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29bc15daa0 con 0x7f2998081020 2024-07-17T06:55:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.721+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f166c0802a0 msgr2=0x7f166c0826e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.721+0000 7f1696c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f166c0802a0 0x7f166c0826e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.720+0000 7f7562000640 1 -- 172.21.15.44:0/919616609 <== osd.7 v2:172.21.15.175:6826/2516790419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7544002dd0 con 0x7f75480803b0 2024-07-17T06:55:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.721+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f166c076bb0 msgr2=0x7f166c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.721+0000 7f1696c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f166c076bb0 0x7f166c079070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1688000f60 tx=0x7f1688009040 comp rx=0 tx=0).stop 2024-07-17T06:55:11.725 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:11.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-07-17T06:55:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f75480803b0 msgr2=0x7f75480827f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f75480803b0 0x7f75480827f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 msgr2=0x7f16900b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f1696c00640 1 --2- 172.21.15.44:0/2771463657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16900ce1d0 0x7f16900b3940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f168000cd10 tx=0x7f168000a620 comp rx=0 tx=0).stop 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f1694c00640 1 -- 172.21.15.44:0/2771463657 reap_dead start 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 shutdown_connections 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 >> 172.21.15.44:0/2771463657 conn(0x7f16900d3700 msgr2=0x7f1690160180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f29b2000640 1 -- 172.21.15.44:0/1079770623 <== osd.5 v2:172.21.15.175:6818/1413905252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f29bc15daa0 con 0x7f2998081020 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7548076c30 msgr2=0x7f75480790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7548076c30 0x7f75480790f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f756c0d3230 tx=0x7f755c004730 comp rx=0 tx=0).stop 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756c0b29e0 msgr2=0x7f756c1f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 --2- 172.21.15.44:0/919616609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f756c0b29e0 0x7f756c1f4800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f755400c390 tx=0x7f7554001e70 comp rx=0 tx=0).stop 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f756a800640 1 -- 172.21.15.44:0/919616609 reap_dead start 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 shutdown_connections 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.722+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 >> 172.21.15.44:0/919616609 conn(0x7f756c0d3700 msgr2=0x7f756c0b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 shutdown_connections 2024-07-17T06:55:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f7571000640 1 -- 172.21.15.44:0/919616609 wait complete. 2024-07-17T06:55:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 shutdown_connections 2024-07-17T06:55:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f1696c00640 1 -- 172.21.15.44:0/2771463657 wait complete. 2024-07-17T06:55:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f2998081020 msgr2=0x7f29980834c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.723+0000 7f29c1000640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f2998081020 0x7f29980834c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2998076c00 msgr2=0x7f29980790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2998076c00 0x7f29980790c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0c6bc0 tx=0x7f29a4009260 comp rx=0 tx=0).stop 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 msgr2=0x7f29bc1f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 --2- 172.21.15.44:0/1079770623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29bc0b29e0 0x7f29bc1f9100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00d750 tx=0x7f29ac00dc10 comp rx=0 tx=0).stop 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29ba800640 1 -- 172.21.15.44:0/1079770623 reap_dead start 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 shutdown_connections 2024-07-17T06:55:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.724+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 >> 172.21.15.44:0/1079770623 conn(0x7f29bc0d3700 msgr2=0x7f29bc0d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.731 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:11.731 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-07-17T06:55:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.728+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 shutdown_connections 2024-07-17T06:55:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.728+0000 7f29c1000640 1 -- 172.21.15.44:0/1079770623 wait complete. 2024-07-17T06:55:11.733 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2024-07-17T06:55:11.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-07-17T06:55:11.734 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-07-17T06:55:11.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-07-17T06:55:11.750 INFO:teuthology.orchestra.run.smithi044.stdout:60129542153 2024-07-17T06:55:11.750 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:11.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-07-17T06:55:11.751 INFO:teuthology.orchestra.run.smithi044.stdout:60129542152 2024-07-17T06:55:11.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-07-17T06:55:11.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-07-17T06:55:11.754 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-07-17T06:55:11.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-07-17T06:55:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.986+0000 7f7738600640 1 -- 172.21.15.44:0/1454511308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 msgr2=0x7f77300ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.986+0000 7f7738600640 1 --2- 172.21.15.44:0/1454511308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300ac830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7728005fd0 tx=0x7f772802aaa0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.987+0000 7f7738600640 1 -- 172.21.15.44:0/1454511308 shutdown_connections 2024-07-17T06:55:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.987+0000 7f7738600640 1 --2- 172.21.15.44:0/1454511308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300ac830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.987+0000 7f7738600640 1 --2- 172.21.15.44:0/1454511308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f773006eb90 0x7f773015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.987+0000 7f7738600640 1 -- 172.21.15.44:0/1454511308 >> 172.21.15.44:0/1454511308 conn(0x7f77300b0eb0 msgr2=0x7f77300b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.988+0000 7f7738600640 1 -- 172.21.15.44:0/1454511308 shutdown_connections 2024-07-17T06:55:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.988+0000 7f7738600640 1 -- 172.21.15.44:0/1454511308 wait complete. 2024-07-17T06:55:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.990+0000 7f7738600640 1 Processor -- start 2024-07-17T06:55:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.991+0000 7f7738600640 1 -- start start 2024-07-17T06:55:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.991+0000 7fb216000640 1 -- 172.21.15.44:0/343136527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204006bc0 con 0x7fb1fc0051c0 2024-07-17T06:55:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.992+0000 7f7738600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.992+0000 7fb21e200640 1 -- 172.21.15.44:0/343136527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 msgr2=0x7fb1fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.992+0000 7fb21e200640 1 --2- 172.21.15.44:0/343136527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb1fc00b9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb210009ed0 tx=0x7fb1fc005830 comp rx=0 tx=0).stop 2024-07-17T06:55:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7f7738600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7fb21e200640 1 -- 172.21.15.44:0/343136527 shutdown_connections 2024-07-17T06:55:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7fb21e200640 1 --2- 172.21.15.44:0/343136527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb1fc00b9c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7fb21e200640 1 -- 172.21.15.44:0/343136527 >> 172.21.15.44:0/343136527 conn(0x7fb2180d3700 msgr2=0x7fb2180d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7f7738600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 0x7f77300737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7f7735200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7f7735200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37726/0 (socket says 172.21.15.44:37726) 2024-07-17T06:55:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.993+0000 7f7735c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.994+0000 7fb21e200640 1 -- 172.21.15.44:0/343136527 shutdown_connections 2024-07-17T06:55:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.994+0000 7f7735c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44582/0 (socket says 172.21.15.44:44582) 2024-07-17T06:55:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.994+0000 7fb21e200640 1 -- 172.21.15.44:0/343136527 wait complete. 2024-07-17T06:55:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7738600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f773014fd60 con 0x7f77300ac3a0 2024-07-17T06:55:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7738600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f773006f8e0 con 0x7f773006eb90 2024-07-17T06:55:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7738600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f773014fee0 con 0x7f77300b3280 2024-07-17T06:55:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7735200640 1 -- 172.21.15.44:0/2113182489 learned_addr learned my addr 172.21.15.44:0/2113182489 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7736600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 0x7f77300737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7735200640 1 -- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 msgr2=0x7f77300737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7735200640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 0x7f77300737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.995+0000 7f7735200640 1 -- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 msgr2=0x7f77301567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.997+0000 7f7735200640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.997+0000 7f7735200640 1 -- 172.21.15.44:0/2113182489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77301f81b0 con 0x7f77300ac3a0 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.997+0000 7f7735200640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f772400c390 tx=0x7f772400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.997+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724017070 con 0x7f77300ac3a0 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.998+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7730156ee0 con 0x7f77300ac3a0 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.998+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301f8a10 con 0x7f77300ac3a0 2024-07-17T06:55:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.997+0000 7fb21e200640 1 Processor -- start 2024-07-17T06:55:11.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.998+0000 7f7735c00640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:55:11.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.998+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77240053c0 con 0x7f77300ac3a0 2024-07-17T06:55:11.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.998+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724003850 con 0x7f77300ac3a0 2024-07-17T06:55:11.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:11.999+0000 7f7736600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 0x7f77300737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.000+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c005290 con 0x7f77300ac3a0 2024-07-17T06:55:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.001+0000 7fb21e200640 1 -- start start 2024-07-17T06:55:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.002+0000 7f7dca600640 1 -- 172.21.15.44:0/3871751189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7da8004800 msgr2=0x7f7da8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.002+0000 7f7dca600640 1 --2- 172.21.15.44:0/3871751189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7da8004800 0x7f7da8004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40ceac0 tx=0x7f7da8005830 comp rx=0 tx=0).stop 2024-07-17T06:55:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.002+0000 7f7dc3e00640 1 -- 172.21.15.44:0/3871751189 reap_dead start 2024-07-17T06:55:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.002+0000 7f7dca600640 1 -- 172.21.15.44:0/3871751189 shutdown_connections 2024-07-17T06:55:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.002+0000 7f7dca600640 1 -- 172.21.15.44:0/3871751189 >> 172.21.15.44:0/3871751189 conn(0x7f7dc40d04e0 msgr2=0x7f7dc40d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb2180cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2180c95c0 0x7fb218153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 0x7fb2180c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180cea50 con 0x7fb1fc0051c0 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2180ce8d0 con 0x7fb2180cb2b0 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.003+0000 7fb21e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2180cebd0 con 0x7fb2180c95c0 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7fb217e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 0x7fb2180c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7fb217e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 0x7fb2180c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44590/0 (socket says 172.21.15.44:44590) 2024-07-17T06:55:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7f7dca600640 1 -- 172.21.15.44:0/3871751189 shutdown_connections 2024-07-17T06:55:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7f7dca600640 1 -- 172.21.15.44:0/3871751189 wait complete. 2024-07-17T06:55:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7fb217e00640 1 -- 172.21.15.44:0/1610808325 learned_addr learned my addr 172.21.15.44:0/1610808325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.004+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f772401a070 con 0x7f77300ac3a0 2024-07-17T06:55:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.005+0000 7fb217400640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb2180cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.006+0000 7fb217400640 1 -- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2180c95c0 msgr2=0x7fb218153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.006+0000 7fb217400640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2180c95c0 0x7fb218153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.006+0000 7fb217400640 1 -- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 msgr2=0x7fb2180c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.006+0000 7fb217400640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 0x7fb2180c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.006+0000 7fb217400640 1 -- 172.21.15.44:0/1610808325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2180c91c0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.007+0000 7fb217400640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb2180cbd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb2040065c0 tx=0x7fb2040039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.007+0000 7f7dca600640 1 Processor -- start 2024-07-17T06:55:12.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.008+0000 7f7dca600640 1 -- start start 2024-07-17T06:55:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f772d600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 0x7f7710079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f77240035f0 con 0x7f77300ac3a0 2024-07-17T06:55:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7735c00640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 0x7f7710079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 0x7f7dc415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 0x7f7dc41f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc40c7ca0 con 0x7f7dc40cd1e0 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc40c7b20 con 0x7f7da8004800 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.009+0000 7f7dca600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc40ce150 con 0x7f7dc415dfc0 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7dc3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7dc3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:44604/0 (socket says 172.21.15.44:44604) 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7dc3400640 1 -- 172.21.15.44:0/669347984 learned_addr learned my addr 172.21.15.44:0/669347984 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7dc3e00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 0x7f7dc41f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.010+0000 7f7dc2a00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 0x7f7dc415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.011+0000 7fc662a00640 1 -- 172.21.15.44:0/2193672553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6640ad8f0 con 0x7fc65c036520 2024-07-17T06:55:12.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7724021020 con 0x7f77300ac3a0 2024-07-17T06:55:12.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204015070 con 0x7fb1fc0051c0 2024-07-17T06:55:12.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb204011900 con 0x7fb1fc0051c0 2024-07-17T06:55:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7fc66b000640 1 -- 172.21.15.44:0/2193672553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c036520 msgr2=0x7fc65c01cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7fc66b000640 1 --2- 172.21.15.44:0/2193672553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c036520 0x7fc65c01cd30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc664065cb0 tx=0x7fc6640a5290 comp rx=0 tx=0).stop 2024-07-17T06:55:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.012+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204011d10 con 0x7fb1fc0051c0 2024-07-17T06:55:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 -- 172.21.15.44:0/2193672553 shutdown_connections 2024-07-17T06:55:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 --2- 172.21.15.44:0/2193672553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c036520 0x7fc65c01cd30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 --2- 172.21.15.44:0/2193672553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c00ba00 0x7fc65c035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 -- 172.21.15.44:0/2193672553 >> 172.21.15.44:0/2193672553 conn(0x7fc65c054560 msgr2=0x7fc65c054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.011+0000 7f7dc3e00640 1 -- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 msgr2=0x7f7dc40ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3e00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3e00640 1 -- 172.21.15.44:0/669347984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 msgr2=0x7f7dc415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3e00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 0x7f7dc415da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3e00640 1 -- 172.21.15.44:0/669347984 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc40cbc00 con 0x7f7dc415dfc0 2024-07-17T06:55:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7735c00640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 0x7f7710079270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f771c00bfd0 tx=0x7f771c00b390 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3400640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:55:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc2a00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 0x7f7dc415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T06:55:12.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2180cc4c0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7f7dc3e00640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 0x7f7dc41f84e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4069800 tx=0x7f7dbc00eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.015+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc00c970 con 0x7f7dc415dfc0 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.015+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dbc017070 con 0x7f7dc415dfc0 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.015+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011460 con 0x7f7dc415dfc0 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 -- 172.21.15.44:0/2193672553 shutdown_connections 2024-07-17T06:55:12.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.013+0000 7fc66b000640 1 -- 172.21.15.44:0/2193672553 wait complete. 2024-07-17T06:55:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.016+0000 7fc66b000640 1 Processor -- start 2024-07-17T06:55:12.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.016+0000 7fc66b000640 1 -- start start 2024-07-17T06:55:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.018+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fb204010420 con 0x7fb1fc0051c0 2024-07-17T06:55:12.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.018+0000 7fc66b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.019+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2180ca7d0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.019+0000 7fc66b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c00ba00 0x7fc65c04d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.020+0000 7fc66b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc65c036520 0x7fc65c04dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.020+0000 7fc663e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.021+0000 7fc66b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c009fd0 con 0x7fc65c00ba00 2024-07-17T06:55:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.021+0000 7fc66b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c009e50 con 0x7fc65c036520 2024-07-17T06:55:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.021+0000 7fc66b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c00a150 con 0x7fc65c005a60 2024-07-17T06:55:12.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.022+0000 7fc663e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58216/0 (socket says 172.21.15.44:58216) 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.022+0000 7fc663e00640 1 -- 172.21.15.44:0/3578425396 learned_addr learned my addr 172.21.15.44:0/3578425396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.019+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc40cbee0 con 0x7f7dc415dfc0 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.019+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc40cc4a0 con 0x7f7dc415dfc0 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.022+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c005290 con 0x7f7dc415dfc0 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.020+0000 7fb20f400640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 0x7fb1f80790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.020+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb218161910 con 0x7fb1fc0051c0 2024-07-17T06:55:12.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.020+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb2040983c0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fc663e00640 1 -- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc65c036520 msgr2=0x7fc65c04dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fc663e00640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc65c036520 0x7fc65c04dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fc663e00640 1 -- 172.21.15.44:0/3578425396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c00ba00 msgr2=0x7fc65c04d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fc663e00640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c00ba00 0x7fc65c04d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fc663e00640 1 -- 172.21.15.44:0/3578425396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c0dc440 con 0x7fc65c005a60 2024-07-17T06:55:12.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.023+0000 7fb216a00640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 0x7fb1f80790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.024+0000 7fc663e00640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e9f0 tx=0x7fc65800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.024+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65800cde0 con 0x7fc65c005a60 2024-07-17T06:55:12.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.024+0000 7fb216a00640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 0x7fb1f80790c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb218073640 tx=0x7fb208001b60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.024+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c0dba10 con 0x7fc65c005a60 2024-07-17T06:55:12.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.024+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c0dc6e0 con 0x7fc65c005a60 2024-07-17T06:55:12.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.026+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc658004020 con 0x7fc65c005a60 2024-07-17T06:55:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.026+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658004450 con 0x7fc65c005a60 2024-07-17T06:55:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.026+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c0e2260 con 0x7fc65c005a60 2024-07-17T06:55:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.027+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f7dbc01c460 con 0x7f7dc415dfc0 2024-07-17T06:55:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.028+0000 7f7dbb400640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 0x7f7da4079140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.028+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7dbc098700 con 0x7f7dc415dfc0 2024-07-17T06:55:12.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.029+0000 7f7dc3400640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 0x7f7da4079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.030+0000 7f7dc3400640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 0x7f7da4079140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fd0 tx=0x7f7db0005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.031+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb2040616e0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.042+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7dbc061220 con 0x7f7dc415dfc0 2024-07-17T06:55:12.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.044+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc658005030 con 0x7fc65c005a60 2024-07-17T06:55:12.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.046+0000 7fc657e00640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 0x7fc6440790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.047+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc65805e690 con 0x7fc65c005a60 2024-07-17T06:55:12.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.047+0000 7fc663400640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 0x7fc6440790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.051+0000 7fc663400640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 0x7fc6440790c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc65c039460 tx=0x7fc650007970 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.052+0000 7f490b000640 1 -- 172.21.15.44:0/3220927948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406eb90 msgr2=0x7f490415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.052+0000 7f490b000640 1 --2- 172.21.15.44:0/3220927948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406eb90 0x7f490415a5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49040cb540 tx=0x7f48fc02a420 comp rx=0 tx=0).stop 2024-07-17T06:55:12.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.053+0000 7f490b000640 1 -- 172.21.15.44:0/3220927948 shutdown_connections 2024-07-17T06:55:12.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.053+0000 7f490b000640 1 --2- 172.21.15.44:0/3220927948 >> v2:172.21.15.175:3300/0 conn(0x7f49040b3280 0x7f49040ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.053+0000 7f490b000640 1 --2- 172.21.15.44:0/3220927948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406eb90 0x7f490415a5a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.053+0000 7f490b000640 1 -- 172.21.15.44:0/3220927948 >> 172.21.15.44:0/3220927948 conn(0x7f49040b0eb0 msgr2=0x7f49040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.054+0000 7f490b000640 1 -- 172.21.15.44:0/3220927948 shutdown_connections 2024-07-17T06:55:12.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.054+0000 7f490b000640 1 -- 172.21.15.44:0/3220927948 wait complete. 2024-07-17T06:55:12.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.056+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc658026020 con 0x7fc65c005a60 2024-07-17T06:55:12.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.057+0000 7f490b000640 1 Processor -- start 2024-07-17T06:55:12.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.058+0000 7f490b000640 1 -- start start 2024-07-17T06:55:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 0x7f490415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49040ac3a0 0x7f4904072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 0x7f4904152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f490406f850 con 0x7f49040b3280 2024-07-17T06:55:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f490406f6d0 con 0x7f49040ac3a0 2024-07-17T06:55:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.060+0000 7f490b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f490414fd60 con 0x7f490406eb90 2024-07-17T06:55:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.061+0000 7f4903e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 0x7f490415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.061+0000 7f4909000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 0x7f4904152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.061+0000 7f4903e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 0x7f490415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58230/0 (socket says 172.21.15.44:58230) 2024-07-17T06:55:12.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.061+0000 7f4903e00640 1 -- 172.21.15.44:0/1821631709 learned_addr learned my addr 172.21.15.44:0/1821631709 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.062+0000 7f4909000640 1 -- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 msgr2=0x7f490415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.062+0000 7f4909000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 0x7f490415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.062+0000 7f4909000640 1 -- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49040ac3a0 msgr2=0x7f4904072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.062+0000 7f4909000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49040ac3a0 0x7f4904072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.062+0000 7f4909000640 1 -- 172.21.15.44:0/1821631709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49040cdbd0 con 0x7f49040b3280 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.063+0000 7f4909000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 0x7f4904152600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c340 tx=0x7f48f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.063+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0011790 con 0x7f49040b3280 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.063+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f0017070 con 0x7f49040b3280 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.063+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0011ce0 con 0x7f49040b3280 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.064+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49040cdeb0 con 0x7f49040b3280 2024-07-17T06:55:12.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.065+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041508d0 con 0x7f49040b3280 2024-07-17T06:55:12.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.070+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f48f001c460 con 0x7f49040b3280 2024-07-17T06:55:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.071+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904150ad0 con 0x7f49040b3280 2024-07-17T06:55:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.071+0000 7f48fbe00640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 0x7f48e4079140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.071+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48f0098440 con 0x7f49040b3280 2024-07-17T06:55:12.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.072+0000 7f4903e00640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 0x7f48e4079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.073+0000 7f4903e00640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 0x7f48e4079140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007510 tx=0x7f48fc004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.077+0000 7f65a3e00640 1 -- 172.21.15.44:0/2347940072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40d1440 msgr2=0x7f65a40d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.077+0000 7f65a3e00640 1 --2- 172.21.15.44:0/2347940072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40d1440 0x7f65a40d1820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65a40b2610 tx=0x7f659002a3d0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 -- 172.21.15.44:0/2347940072 shutdown_connections 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 --2- 172.21.15.44:0/2347940072 >> v2:172.21.15.44:3300/0 conn(0x7f65a40cdca0 0x7f65a40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 --2- 172.21.15.44:0/2347940072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40d1440 0x7f65a40d1820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 -- 172.21.15.44:0/2347940072 >> 172.21.15.44:0/2347940072 conn(0x7f65a40d04c0 msgr2=0x7f65a40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 -- 172.21.15.44:0/2347940072 shutdown_connections 2024-07-17T06:55:12.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.079+0000 7f65a3e00640 1 -- 172.21.15.44:0/2347940072 wait complete. 2024-07-17T06:55:12.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.082+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4904150ad0 con 0x7f49040b3280 2024-07-17T06:55:12.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.083+0000 7f1dad400640 1 -- 172.21.15.44:0/876543381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da806ed30 msgr2=0x7f1da815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.083+0000 7f1dad400640 1 --2- 172.21.15.44:0/876543381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da806ed30 0x7f1da815c2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009830 tx=0x7f1d9402a4b0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.083+0000 7f65a3e00640 1 Processor -- start 2024-07-17T06:55:12.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.083+0000 7f65a3e00640 1 -- start start 2024-07-17T06:55:12.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.085+0000 7f65a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40cdca0 0x7f65a414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.085+0000 7f65a3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a40d1440 0x7f65a4151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 -- 172.21.15.44:0/876543381 shutdown_connections 2024-07-17T06:55:12.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 --2- 172.21.15.44:0/876543381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da806ed30 0x7f1da815c2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 --2- 172.21.15.44:0/876543381 >> v2:172.21.15.44:3300/0 conn(0x7f1da8152220 0x7f1da806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 -- 172.21.15.44:0/876543381 >> 172.21.15.44:0/876543381 conn(0x7f1da80b0eb0 msgr2=0x7f1da80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 -- 172.21.15.44:0/876543381 shutdown_connections 2024-07-17T06:55:12.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.085+0000 7f65a3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37798/0 (socket says 172.21.15.44:37798) 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3400640 1 -- 172.21.15.44:0/663469928 learned_addr learned my addr 172.21.15.44:0/663469928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a2000640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a40d1440 0x7f65a4151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65a406e3c0 con 0x7f65a415d350 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65a406e240 con 0x7f65a40cdca0 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65a406e540 con 0x7f65a40d1440 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.086+0000 7f1dad400640 1 -- 172.21.15.44:0/876543381 wait complete. 2024-07-17T06:55:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.087+0000 7f65a3400640 1 -- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a40d1440 msgr2=0x7f65a4151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.087+0000 7f65a3400640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a40d1440 0x7f65a4151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.087+0000 7f65a3400640 1 -- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40cdca0 msgr2=0x7f65a414bea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.087+0000 7f65a3400640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40cdca0 0x7f65a414bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.087+0000 7f65a3400640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a415c490 con 0x7f65a415d350 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.088+0000 7f65a3400640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eee0 tx=0x7f659c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.088+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004b20 con 0x7f65a415d350 2024-07-17T06:55:12.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.089+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a415b810 con 0x7f65a415d350 2024-07-17T06:55:12.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.089+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a415bdd0 con 0x7f65a415d350 2024-07-17T06:55:12.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.089+0000 7f1dad400640 1 Processor -- start 2024-07-17T06:55:12.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.090+0000 7f1dad400640 1 -- start start 2024-07-17T06:55:12.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.090+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f659c004db0 con 0x7f65a415d350 2024-07-17T06:55:12.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.090+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004b20 con 0x7f65a415d350 2024-07-17T06:55:12.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.093+0000 7f1dad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1da806ed30 0x7f1da8071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.093+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657c005290 con 0x7f65a415d350 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f1dad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 0x7f1da8072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f1dad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 0x7f1da8098290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f1dad400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1da80cbbd0 con 0x7f1da806ed30 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f1dad400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1da80cba50 con 0x7f1da80cb3a0 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f1dad400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1da80cbd50 con 0x7f1da8152220 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 0x7f1da8072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f1da7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 0x7f1da8098290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f1da6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1da806ed30 0x7f1da8071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f1da7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 0x7f1da8098290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58260/0 (socket says 172.21.15.44:58260) 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f1da7200640 1 -- 172.21.15.44:0/2710930841 learned_addr learned my addr 172.21.15.44:0/2710930841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.094+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f659c01d020 con 0x7f65a415d350 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.095+0000 7f659aa00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 0x7f6584079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f1da5e00640 1 -- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 msgr2=0x7f1da8098290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f1da5e00640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 0x7f1da8098290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f1da5e00640 1 -- 172.21.15.44:0/2710930841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1da806ed30 msgr2=0x7f1da8071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f1da5e00640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1da806ed30 0x7f1da8071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f1da5e00640 1 -- 172.21.15.44:0/2710930841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da8072b60 con 0x7f1da80cb3a0 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f65a2a00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 0x7f6584079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.096+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f659c05d750 con 0x7f65a415d350 2024-07-17T06:55:12.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.097+0000 7f1da5e00640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 0x7f1da8072530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d94031000 tx=0x7f1d94032c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.098+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94040070 con 0x7f1da80cb3a0 2024-07-17T06:55:12.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.098+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d94003900 con 0x7f1da80cb3a0 2024-07-17T06:55:12.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.098+0000 7f65a2a00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 0x7f6584079270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65940095f0 tx=0x7f6594006b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.101+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94003d50 con 0x7f1da80cb3a0 2024-07-17T06:55:12.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.101+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81538e0 con 0x7f1da80cb3a0 2024-07-17T06:55:12.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.101+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da8153ea0 con 0x7f1da80cb3a0 2024-07-17T06:55:12.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.103+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da815f940 con 0x7f1da80cb3a0 2024-07-17T06:55:12.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.107+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f659c061860 con 0x7f65a415d350 2024-07-17T06:55:12.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.108+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f1d94039040 con 0x7f1da80cb3a0 2024-07-17T06:55:12.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.110+0000 7f1d9ea00640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 0x7f1d88079140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.111+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1d940b9eb0 con 0x7f1da80cb3a0 2024-07-17T06:55:12.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.111+0000 7f1da6800640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 0x7f1d88079140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.113+0000 7f1da6800640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 0x7f1d88079140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1da8151f30 tx=0x7f1d9800b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.126+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1d94082b80 con 0x7f1da80cb3a0 2024-07-17T06:55:12.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.146+0000 7f1091e00640 1 -- 172.21.15.44:0/1630622984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0cbab0 msgr2=0x7f108c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.146+0000 7f1091e00640 1 --2- 172.21.15.44:0/1630622984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0cbab0 0x7f108c06d620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1078009830 tx=0x7f107802a3f0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.147+0000 7f1091e00640 1 -- 172.21.15.44:0/1630622984 shutdown_connections 2024-07-17T06:55:12.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.147+0000 7f1091e00640 1 --2- 172.21.15.44:0/1630622984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0cbab0 0x7f108c06d620 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.147+0000 7f1091e00640 1 -- 172.21.15.44:0/1630622984 >> 172.21.15.44:0/1630622984 conn(0x7f108c0b0eb0 msgr2=0x7f108c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.148+0000 7f1091e00640 1 -- 172.21.15.44:0/1630622984 shutdown_connections 2024-07-17T06:55:12.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.148+0000 7f1091e00640 1 -- 172.21.15.44:0/1630622984 wait complete. 2024-07-17T06:55:12.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.152+0000 7f1091e00640 1 Processor -- start 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.152+0000 7f1091e00640 1 -- start start 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 0x7f108c1ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f108c0cbab0 0x7f108c09c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0cd9f0 con 0x7f108c06ed30 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f108c15c800 con 0x7f108c0cbab0 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f1091e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f108c0cdb70 con 0x7f108c152220 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.154+0000 7f108b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:58270/0 (socket says 172.21.15.44:58270) 2024-07-17T06:55:12.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 -- 172.21.15.44:0/3586973253 learned_addr learned my addr 172.21.15.44:0/3586973253 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 -- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f108c0cbab0 msgr2=0x7f108c09c180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f108c0cbab0 0x7f108c09c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108aa00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 0x7f108c1ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 -- 172.21.15.44:0/3586973253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 msgr2=0x7f108c1ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 0x7f108c1ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.155+0000 7f108b400640 1 -- 172.21.15.44:0/3586973253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c1f3090 con 0x7f108c152220 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.156+0000 7f108b400640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f108400c780 tx=0x7f108400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.156+0000 7f108aa00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 0x7f108c1ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.163+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084017070 con 0x7f108c152220 2024-07-17T06:55:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.163+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1084005070 con 0x7f108c152220 2024-07-17T06:55:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.163+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084005500 con 0x7f108c152220 2024-07-17T06:55:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.163+0000 7f1091e00640 1 -- 172.21.15.44:0/3586973253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c0997c0 con 0x7f108c152220 2024-07-17T06:55:12.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.164+0000 7f1091e00640 1 -- 172.21.15.44:0/3586973253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c099d80 con 0x7f108c152220 2024-07-17T06:55:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.165+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c0ce090 con 0x7f108c152220 2024-07-17T06:55:12.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.168+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f108401a070 con 0x7f108c152220 2024-07-17T06:55:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.170+0000 7f1083400640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 0x7f106c07d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.170+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1084097c90 con 0x7f108c152220 2024-07-17T06:55:12.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.172+0000 7f108aa00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 0x7f106c07d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:12.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.173+0000 7f108aa00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 0x7f106c07d550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f107c000f60 tx=0x7f107c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.184+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1084060840 con 0x7f108c152220 2024-07-17T06:55:12.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.450+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb2180b3d00 con 0x7fb1fc0051c0 2024-07-17T06:55:12.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.454+0000 7fb20f400640 1 -- 172.21.15.44:0/1610808325 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb204060fe0 con 0x7fb1fc0051c0 2024-07-17T06:55:12.462 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 msgr2=0x7fb1f80790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 0x7fb1f80790c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb218073640 tx=0x7fb208001b60 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 msgr2=0x7fb2180cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb2180cbd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb2040065c0 tx=0x7fb2040039c0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 shutdown_connections 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.459+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2180cb2b0 0x7fb2180c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fb1f8076c00 0x7fb1f80790c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2180c95c0 0x7fb218153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 --2- 172.21.15.44:0/1610808325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1fc0051c0 0x7fb2180cbd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 >> 172.21.15.44:0/1610808325 conn(0x7fb2180d3700 msgr2=0x7fb2180b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 shutdown_connections 2024-07-17T06:55:12.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.460+0000 7fb21e200640 1 -- 172.21.15.44:0/1610808325 wait complete. 2024-07-17T06:55:12.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.475+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f770c005850 con 0x7f77300ac3a0 2024-07-17T06:55:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.476+0000 7f772d600640 1 -- 172.21.15.44:0/2113182489 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7724097020 con 0x7f77300ac3a0 2024-07-17T06:55:12.476 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.480+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 msgr2=0x7f7710079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.480+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 0x7f7710079270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f771c00bfd0 tx=0x7f771c00b390 comp rx=0 tx=0).stop 2024-07-17T06:55:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.480+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 msgr2=0x7f77300b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.480+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f772400c390 tx=0x7f772400c850 comp rx=0 tx=0).stop 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 shutdown_connections 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7710076db0 0x7f7710079270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77300b3280 0x7f77300737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77300ac3a0 0x7f77300b1500 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 --2- 172.21.15.44:0/2113182489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f773006eb90 0x7f77301567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.481+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 >> 172.21.15.44:0/2113182489 conn(0x7f77300b0eb0 msgr2=0x7f77300b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.482+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 shutdown_connections 2024-07-17T06:55:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.483+0000 7f7738600640 1 -- 172.21.15.44:0/2113182489 wait complete. 2024-07-17T06:55:12.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-07-17T06:55:12.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-07-17T06:55:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.501+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d9c005580 con 0x7f7dc415dfc0 2024-07-17T06:55:12.501 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:12.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.502+0000 7f7dbb400640 1 -- 172.21.15.44:0/669347984 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dbc065770 con 0x7f7dc415dfc0 2024-07-17T06:55:12.502 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 msgr2=0x7f7da4079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 0x7f7da4079140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fd0 tx=0x7f7db0005f90 comp rx=0 tx=0).stop 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 msgr2=0x7f7dc41f84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 0x7f7dc41f84e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4069800 tx=0x7f7dbc00eae0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 shutdown_connections 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7da4076c80 0x7f7da4079140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dc415dfc0 0x7f7dc41f84e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc40cd1e0 0x7f7dc415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 --2- 172.21.15.44:0/669347984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7da8004800 0x7f7dc40ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 >> 172.21.15.44:0/669347984 conn(0x7f7dc40d04e0 msgr2=0x7f7dc40c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 shutdown_connections 2024-07-17T06:55:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.506+0000 7f7dca600640 1 -- 172.21.15.44:0/669347984 wait complete. 2024-07-17T06:55:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.516+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f490415a5a0 con 0x7f49040b3280 2024-07-17T06:55:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.517+0000 7f48fbe00640 1 -- 172.21.15.44:0/1821631709 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48f0061520 con 0x7f49040b3280 2024-07-17T06:55:12.517 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 msgr2=0x7f48e4079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 0x7f48e4079140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007510 tx=0x7f48fc004ab0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 msgr2=0x7f4904152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 0x7f4904152600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c340 tx=0x7f48f000c800 comp rx=0 tx=0).stop 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 shutdown_connections 2024-07-17T06:55:12.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040b3280 0x7f4904152600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49040ac3a0 0x7f4904072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f48e4076c80 0x7f48e4079140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 --2- 172.21.15.44:0/1821631709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406eb90 0x7f490415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.520+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 >> 172.21.15.44:0/1821631709 conn(0x7f49040b0eb0 msgr2=0x7f49040cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.521+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc65c035fe0 con 0x7fc65c005a60 2024-07-17T06:55:12.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.521+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 shutdown_connections 2024-07-17T06:55:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.521+0000 7f490b000640 1 -- 172.21.15.44:0/1821631709 wait complete. 2024-07-17T06:55:12.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.521+0000 7fc657e00640 1 -- 172.21.15.44:0/3578425396 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6580627a0 con 0x7fc65c005a60 2024-07-17T06:55:12.522 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-07-17T06:55:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 msgr2=0x7fc6440790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 0x7fc6440790c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc65c039460 tx=0x7fc650007970 comp rx=0 tx=0).stop 2024-07-17T06:55:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 msgr2=0x7fc65c0e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e9f0 tx=0x7fc65800eeb0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 shutdown_connections 2024-07-17T06:55:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc65c036520 0x7fc65c04dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc65c00ba00 0x7fc65c04d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc644076c00 0x7fc6440790c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 --2- 172.21.15.44:0/3578425396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc65c005a60 0x7fc65c0e1d20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 >> 172.21.15.44:0/3578425396 conn(0x7fc65c054560 msgr2=0x7fc65c034a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.524+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 shutdown_connections 2024-07-17T06:55:12.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.525+0000 7fc66b000640 1 -- 172.21.15.44:0/3578425396 wait complete. 2024-07-17T06:55:12.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-07-17T06:55:12.533 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:12.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-07-17T06:55:12.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-07-17T06:55:12.550 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.555+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f657c005850 con 0x7f65a415d350 2024-07-17T06:55:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.556+0000 7f659aa00640 1 -- 172.21.15.44:0/663469928 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f659c061160 con 0x7f65a415d350 2024-07-17T06:55:12.556 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.558+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 msgr2=0x7f6584079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.558+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 0x7f6584079270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65940095f0 tx=0x7f6594006b50 comp rx=0 tx=0).stop 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.558+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 msgr2=0x7f65a41521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.558+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f659c00eee0 tx=0x7f659c00c740 comp rx=0 tx=0).stop 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 shutdown_connections 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65a415d350 0x7f65a41521a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6584076db0 0x7f6584079270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a40d1440 0x7f65a4151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 --2- 172.21.15.44:0/663469928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a40cdca0 0x7f65a414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 >> 172.21.15.44:0/663469928 conn(0x7f65a40d04c0 msgr2=0x7f65a40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 shutdown_connections 2024-07-17T06:55:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.559+0000 7f65a3e00640 1 -- 172.21.15.44:0/663469928 wait complete. 2024-07-17T06:55:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.560+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1da809d490 con 0x7f1da80cb3a0 2024-07-17T06:55:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.561+0000 7f1d9ea00640 1 -- 172.21.15.44:0/2710930841 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d940870d0 con 0x7f1da80cb3a0 2024-07-17T06:55:12.561 INFO:teuthology.orchestra.run.smithi044.stdout:60129542153 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 msgr2=0x7f1d88079140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 0x7f1d88079140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1da8151f30 tx=0x7f1d9800b2e0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 msgr2=0x7f1da8072530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 0x7f1da8072530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d94031000 tx=0x7f1d94032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 shutdown_connections 2024-07-17T06:55:12.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1d88076c80 0x7f1d88079140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1da8152220 0x7f1da8098290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1da80cb3a0 0x7f1da8072530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 --2- 172.21.15.44:0/2710930841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1da806ed30 0x7f1da8071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 >> 172.21.15.44:0/2710930841 conn(0x7f1da80b0eb0 msgr2=0x7f1da815bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 shutdown_connections 2024-07-17T06:55:12.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.565+0000 7f1dad400640 1 -- 172.21.15.44:0/2710930841 wait complete. 2024-07-17T06:55:12.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.7 2024-07-17T06:55:12.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.4 2024-07-17T06:55:12.591 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:12.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.597+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f108c0afa60 con 0x7f108c152220 2024-07-17T06:55:12.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.597+0000 7f1083400640 1 -- 172.21.15.44:0/3586973253 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1084064d90 con 0x7f108c152220 2024-07-17T06:55:12.598 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:12.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 msgr2=0x7f106c07d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 0x7f106c07d550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f107c000f60 tx=0x7f107c009040 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 msgr2=0x7f108c09c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f108400c780 tx=0x7f108400cc40 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 shutdown_connections 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f106c07b090 0x7f106c07d550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f108c152220 0x7f108c09c6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f108c0cbab0 0x7f108c09c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 --2- 172.21.15.44:0/3586973253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c06ed30 0x7f108c1ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 >> 172.21.15.44:0/3586973253 conn(0x7f108c0b0eb0 msgr2=0x7f108c0b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 shutdown_connections 2024-07-17T06:55:12.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:12.600+0000 7f1080c00640 1 -- 172.21.15.44:0/3586973253 wait complete. 2024-07-17T06:55:12.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-07-17T06:55:13.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-07-17T06:55:13.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-07-17T06:55:13.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-07-17T06:55:13.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.701+0000 7f46f6e00640 1 -- 172.21.15.44:0/2332787837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00ce1d0 msgr2=0x7f46f00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.701+0000 7f46f6e00640 1 --2- 172.21.15.44:0/2332787837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00ce1d0 0x7f46f00ce5b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009830 tx=0x7f46e002a5a0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 -- 172.21.15.44:0/2332787837 shutdown_connections 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 --2- 172.21.15.44:0/2332787837 >> v2:172.21.15.175:3300/0 conn(0x7f46f00b29e0 0x7f46f0161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 --2- 172.21.15.44:0/2332787837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00ce1d0 0x7f46f00ce5b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 -- 172.21.15.44:0/2332787837 >> 172.21.15.44:0/2332787837 conn(0x7f46f00d3700 msgr2=0x7f46f00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 -- 172.21.15.44:0/2332787837 shutdown_connections 2024-07-17T06:55:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.702+0000 7f46f6e00640 1 -- 172.21.15.44:0/2332787837 wait complete. 2024-07-17T06:55:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.704+0000 7f46f6e00640 1 Processor -- start 2024-07-17T06:55:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.704+0000 7f46f6e00640 1 -- start start 2024-07-17T06:55:13.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00b29e0 0x7f46f014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f00b6790 0x7f46f01f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0162420 con 0x7f46f00ce1d0 2024-07-17T06:55:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46f01622a0 con 0x7f46f00b6790 2024-07-17T06:55:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46f0167910 con 0x7f46f00b29e0 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37832/0 (socket says 172.21.15.44:37832) 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46f4e00640 1 -- 172.21.15.44:0/4127943943 learned_addr learned my addr 172.21.15.44:0/4127943943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46ef400640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f00b6790 0x7f46f01f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.705+0000 7f46efe00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00b29e0 0x7f46f014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 -- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00b29e0 msgr2=0x7f46f014c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00b29e0 0x7f46f014c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 -- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f00b6790 msgr2=0x7f46f01f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f00b6790 0x7f46f01f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 -- 172.21.15.44:0/4127943943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f00ccd80 con 0x7f46f00ce1d0 2024-07-17T06:55:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46f4e00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46e800efc0 tx=0x7f46e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004b20 con 0x7f46f00ce1d0 2024-07-17T06:55:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.706+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e8004dd0 con 0x7f46f00ce1d0 2024-07-17T06:55:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.707+0000 7f46f6e00640 1 -- 172.21.15.44:0/4127943943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f00cd060 con 0x7f46f00ce1d0 2024-07-17T06:55:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.707+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8003bd0 con 0x7f46f00ce1d0 2024-07-17T06:55:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.707+0000 7f46f6e00640 1 -- 172.21.15.44:0/4127943943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f00cd620 con 0x7f46f00ce1d0 2024-07-17T06:55:13.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.708+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f00ce5b0 con 0x7f46f00ce1d0 2024-07-17T06:55:13.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.709+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f46e801d020 con 0x7f46f00ce1d0 2024-07-17T06:55:13.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.710+0000 7f46e7e00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 0x7f46d00790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.710+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f46e8098c50 con 0x7f46f00ce1d0 2024-07-17T06:55:13.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.711+0000 7f46efe00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 0x7f46d00790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.713+0000 7f46efe00640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 0x7f46d00790c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46e0034000 tx=0x7f46e0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.719+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f46e8061dc0 con 0x7f46f00ce1d0 2024-07-17T06:55:13.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 -- 172.21.15.44:0/1565721551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 msgr2=0x7f87a8161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 --2- 172.21.15.44:0/1565721551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a8161020 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87a8069980 tx=0x7f879402a540 comp rx=0 tx=0).stop 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 -- 172.21.15.44:0/1565721551 shutdown_connections 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 --2- 172.21.15.44:0/1565721551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a8161020 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 --2- 172.21.15.44:0/1565721551 >> v2:172.21.15.175:3300/0 conn(0x7f87a806bf40 0x7f87a806c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 -- 172.21.15.44:0/1565721551 >> 172.21.15.44:0/1565721551 conn(0x7f87a806d050 msgr2=0x7f87a806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.738+0000 7f87aee00640 1 -- 172.21.15.44:0/1565721551 shutdown_connections 2024-07-17T06:55:13.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.739+0000 7f87aee00640 1 -- 172.21.15.44:0/1565721551 wait complete. 2024-07-17T06:55:13.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.740+0000 7f87aee00640 1 Processor -- start 2024-07-17T06:55:13.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.740+0000 7f87aee00640 1 -- start start 2024-07-17T06:55:13.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87a806bf40 0x7f87a8144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87a80b4360 0x7f87a815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87a806cdb0 con 0x7f87a80b3f20 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87a806cc30 con 0x7f87a806bf40 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87aee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87a8161c70 con 0x7f87a80b4360 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37860/0 (socket says 172.21.15.44:37860) 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.742+0000 7f87a7400640 1 -- 172.21.15.44:0/3076458530 learned_addr learned my addr 172.21.15.44:0/3076458530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7e00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87a806bf40 0x7f87a8144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 -- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87a80b4360 msgr2=0x7f87a815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87a80b4360 0x7f87a815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 -- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87a806bf40 msgr2=0x7f87a8144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87a806bf40 0x7f87a8144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 -- 172.21.15.44:0/3076458530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a8144d70 con 0x7f87a80b3f20 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.743+0000 7f87a7400640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8794009780 tx=0x7f8794032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.744+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794040070 con 0x7f87a80b3f20 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.744+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8794005310 con 0x7f87a80b3f20 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.744+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940047a0 con 0x7f87a80b3f20 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.744+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a8145050 con 0x7f87a80b3f20 2024-07-17T06:55:13.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.744+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a8145610 con 0x7f87a80b3f20 2024-07-17T06:55:13.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.746+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a806fa60 con 0x7f87a80b3f20 2024-07-17T06:55:13.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.747+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f8794039040 con 0x7f87a80b3f20 2024-07-17T06:55:13.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.749+0000 7f879fe00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 0x7f8788079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.749+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f87940b94a0 con 0x7f87a80b3f20 2024-07-17T06:55:13.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.749+0000 7f87a7e00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 0x7f8788079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.750+0000 7f87a7e00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 0x7f8788079070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.752+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f879404b070 con 0x7f87a80b3f20 2024-07-17T06:55:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 -- 172.21.15.44:0/470450783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c161be0 msgr2=0x7f9f3c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 --2- 172.21.15.44:0/470450783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c161be0 0x7f9f3c15d510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009860 tx=0x7f9f2802a5a0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 -- 172.21.15.44:0/470450783 shutdown_connections 2024-07-17T06:55:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 --2- 172.21.15.44:0/470450783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c161be0 0x7f9f3c15d510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 -- 172.21.15.44:0/470450783 >> 172.21.15.44:0/470450783 conn(0x7f9f3c0b4ff0 msgr2=0x7f9f3c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.776+0000 7f9f41800640 1 -- 172.21.15.44:0/470450783 shutdown_connections 2024-07-17T06:55:13.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.777+0000 7f9f41800640 1 -- 172.21.15.44:0/470450783 wait complete. 2024-07-17T06:55:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.779+0000 7f9f41800640 1 Processor -- start 2024-07-17T06:55:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.779+0000 7f9f41800640 1 -- start start 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 0x7f9f3c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f3c15d130 0x7f9f3c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c168810 con 0x7f9f3c161be0 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c168690 con 0x7f9f3c15d130 2024-07-17T06:55:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f41800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c168990 con 0x7f9f3c0ce690 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f40e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f40e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37866/0 (socket says 172.21.15.44:37866) 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.781+0000 7f9f40e00640 1 -- 172.21.15.44:0/1380298278 learned_addr learned my addr 172.21.15.44:0/1380298278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f3be00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 0x7f9f3c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f3b400640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f3c15d130 0x7f9f3c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 -- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 msgr2=0x7f9f3c1f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 0x7f9f3c1f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 -- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f3c15d130 msgr2=0x7f9f3c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f3c15d130 0x7f9f3c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 -- 172.21.15.44:0/1380298278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c15af00 con 0x7f9f3c161be0 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f3be00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 0x7f9f3c1f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.782+0000 7f9f40e00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400caa0 tx=0x7f9f3400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.783+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34016020 con 0x7f9f3c161be0 2024-07-17T06:55:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.783+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f3400acf0 con 0x7f9f3c161be0 2024-07-17T06:55:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.783+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34012680 con 0x7f9f3c161be0 2024-07-17T06:55:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.783+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c15b1e0 con 0x7f9f3c161be0 2024-07-17T06:55:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.783+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c15b7a0 con 0x7f9f3c161be0 2024-07-17T06:55:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.785+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c0762e0 con 0x7f9f3c161be0 2024-07-17T06:55:13.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.787+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9f34019020 con 0x7f9f3c161be0 2024-07-17T06:55:13.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.788+0000 7f9f33e00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 0x7f9f1c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.788+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f34098640 con 0x7f9f3c161be0 2024-07-17T06:55:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.788+0000 7f9f3be00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 0x7f9f1c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.790+0000 7f9f3be00640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 0x7f9f1c079070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000f60 tx=0x7f9f2c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.796+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9f3c0762e0 con 0x7f9f3c161be0 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 -- 172.21.15.44:0/3037838009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 msgr2=0x7f56000cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 --2- 172.21.15.44:0/3037838009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000cf510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5600074bc0 tx=0x7f55f802aa30 comp rx=0 tx=0).stop 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 -- 172.21.15.44:0/3037838009 shutdown_connections 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 --2- 172.21.15.44:0/3037838009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000cf510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 --2- 172.21.15.44:0/3037838009 >> v2:172.21.15.175:3300/0 conn(0x7f5600073780 0x7f560006ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 -- 172.21.15.44:0/3037838009 >> 172.21.15.44:0/3037838009 conn(0x7f560006bf40 msgr2=0x7f560006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 -- 172.21.15.44:0/3037838009 shutdown_connections 2024-07-17T06:55:13.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.823+0000 7f5607c00640 1 -- 172.21.15.44:0/3037838009 wait complete. 2024-07-17T06:55:13.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.825+0000 7f5607c00640 1 Processor -- start 2024-07-17T06:55:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.825+0000 7f5607c00640 1 -- start start 2024-07-17T06:55:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5600073780 0x7f56001f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f560015c5f0 0x7f56001f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56000cd8e0 con 0x7f56000cf060 2024-07-17T06:55:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56000cd760 con 0x7f560015c5f0 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5607c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56000cda60 con 0x7f5600073780 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37876/0 (socket says 172.21.15.44:37876) 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f55ffe00640 1 -- 172.21.15.44:0/644131893 learned_addr learned my addr 172.21.15.44:0/644131893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5605200640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5600073780 0x7f56001f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.826+0000 7f5605c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f560015c5f0 0x7f56001f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 -- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5600073780 msgr2=0x7f56001f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5600073780 0x7f56001f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 -- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f560015c5f0 msgr2=0x7f56001f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f560015c5f0 0x7f56001f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 -- 172.21.15.44:0/644131893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5600155490 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55ffe00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b3c0 tx=0x7f55ec00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011830 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55ec016070 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011da0 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001f4ac0 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.827+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001f5080 con 0x7f56000cf060 2024-07-17T06:55:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.828+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56000cd340 con 0x7f56000cf060 2024-07-17T06:55:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.830+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f55ec004020 con 0x7f56000cf060 2024-07-17T06:55:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.831+0000 7f55fcc00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 0x7f55e00790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.831+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f55ec098b60 con 0x7f56000cf060 2024-07-17T06:55:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.831+0000 7f5605200640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 0x7f55e00790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:13.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.832+0000 7f5605200640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 0x7f55e00790c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005fd0 tx=0x7f55f4005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.838+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f56000cd340 con 0x7f56000cf060 2024-07-17T06:55:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.972+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f87a8070000 con 0x7f87a80b3f20 2024-07-17T06:55:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.972+0000 7f879fe00640 1 -- 172.21.15.44:0/3076458530 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8794082500 con 0x7f87a80b3f20 2024-07-17T06:55:13.973 INFO:teuthology.orchestra.run.smithi044.stdout:64424509446 2024-07-17T06:55:13.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.973+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f46f00c8b70 con 0x7f46f00ce1d0 2024-07-17T06:55:13.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.974+0000 7f46e7e00640 1 -- 172.21.15.44:0/4127943943 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46e80616c0 con 0x7f46f00ce1d0 2024-07-17T06:55:13.974 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 msgr2=0x7f8788079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 0x7f8788079070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8798005fd0 tx=0x7f8798005850 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 msgr2=0x7f87a81f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8794009780 tx=0x7f8794032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 shutdown_connections 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8788076bb0 0x7f8788079070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87a80b4360 0x7f87a815f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a80b3f20 0x7f87a81f3a90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 --2- 172.21.15.44:0/3076458530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87a806bf40 0x7f87a8144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 >> 172.21.15.44:0/3076458530 conn(0x7f87a806d050 msgr2=0x7f87a8160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 shutdown_connections 2024-07-17T06:55:13.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.975+0000 7f87aee00640 1 -- 172.21.15.44:0/3076458530 wait complete. 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 msgr2=0x7f46d00790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 0x7f46d00790c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46e0034000 tx=0x7f46e0004730 comp rx=0 tx=0).stop 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 msgr2=0x7f46f0157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46e800efc0 tx=0x7f46e800c800 comp rx=0 tx=0).stop 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 shutdown_connections 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46f00ce1d0 0x7f46f0157730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46f00b6790 0x7f46f01f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f46d0076c00 0x7f46d00790c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 --2- 172.21.15.44:0/4127943943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f00b29e0 0x7f46f014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 >> 172.21.15.44:0/4127943943 conn(0x7f46f00d3700 msgr2=0x7f46f0156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 shutdown_connections 2024-07-17T06:55:13.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:13.978+0000 7f46e5600640 1 -- 172.21.15.44:0/4127943943 wait complete. 2024-07-17T06:55:13.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-07-17T06:55:14.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-07-17T06:55:14.002 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.044+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f560006ea90 con 0x7f56000cf060 2024-07-17T06:55:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.045+0000 7f55fcc00640 1 -- 172.21.15.44:0/644131893 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55ec061cd0 con 0x7f56000cf060 2024-07-17T06:55:14.045 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-07-17T06:55:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 msgr2=0x7f55e00790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 0x7f55e00790c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005fd0 tx=0x7f55f4005850 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 msgr2=0x7f56000b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b3c0 tx=0x7f55ec00b880 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 shutdown_connections 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f560015c5f0 0x7f56001f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.047+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56000cf060 0x7f56000b3f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.048+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f55e0076c00 0x7f55e00790c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.048+0000 7f5607c00640 1 --2- 172.21.15.44:0/644131893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5600073780 0x7f56001f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.048+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 >> 172.21.15.44:0/644131893 conn(0x7f560006bf40 msgr2=0x7f560015bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.048+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 shutdown_connections 2024-07-17T06:55:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.048+0000 7f5607c00640 1 -- 172.21.15.44:0/644131893 wait complete. 2024-07-17T06:55:14.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.051+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9f3c0d0e10 con 0x7f9f3c161be0 2024-07-17T06:55:14.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.052+0000 7f9f33e00640 1 -- 172.21.15.44:0/1380298278 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f34020030 con 0x7f9f3c161be0 2024-07-17T06:55:14.052 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2024-07-17T06:55:14.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 msgr2=0x7f9f1c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 0x7f9f1c079070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000f60 tx=0x7f9f2c009040 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 msgr2=0x7f9f3c1f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400caa0 tx=0x7f9f3400cf60 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 shutdown_connections 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f3c161be0 0x7f9f3c1f7a90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f3c15d130 0x7f9f3c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9f1c076bb0 0x7f9f1c079070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 --2- 172.21.15.44:0/1380298278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f3c0ce690 0x7f9f3c1f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.054+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 >> 172.21.15.44:0/1380298278 conn(0x7f9f3c0b4ff0 msgr2=0x7f9f3c0d3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.055+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 shutdown_connections 2024-07-17T06:55:14.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:14.055+0000 7f9f41800640 1 -- 172.21.15.44:0/1380298278 wait complete. 2024-07-17T06:55:14.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-07-17T06:55:14.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.7 2024-07-17T06:55:14.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-07-17T06:55:15.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-07-17T06:55:15.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-07-17T06:55:15.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 -- 172.21.15.44:0/3636869524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e8005910 msgr2=0x7f49e8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 --2- 172.21.15.44:0/3636869524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e8005910 0x7f49e8005d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49f00016c0 tx=0x7f49e8006940 comp rx=0 tx=0).stop 2024-07-17T06:55:15.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a02800640 1 -- 172.21.15.44:0/3636869524 reap_dead start 2024-07-17T06:55:15.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 -- 172.21.15.44:0/3636869524 shutdown_connections 2024-07-17T06:55:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 -- 172.21.15.44:0/3636869524 >> 172.21.15.44:0/3636869524 conn(0x7f4a040b0eb0 msgr2=0x7f4a040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 -- 172.21.15.44:0/3636869524 shutdown_connections 2024-07-17T06:55:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.212+0000 7f4a08e00640 1 -- 172.21.15.44:0/3636869524 wait complete. 2024-07-17T06:55:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.214+0000 7f4a08e00640 1 Processor -- start 2024-07-17T06:55:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.214+0000 7f4a08e00640 1 -- start start 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 0x7f4a041e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a0414e3c0 0x7f4a0414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040b3610 con 0x7f4a0414f0c0 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040b3490 con 0x7f49e8005910 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a08e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a040b3790 con 0x7f4a0414e3c0 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37902/0 (socket says 172.21.15.44:37902) 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 -- 172.21.15.44:0/421983041 learned_addr learned my addr 172.21.15.44:0/421983041 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 0x7f4a041e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a02800640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a0414e3c0 0x7f4a0414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 -- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a0414e3c0 msgr2=0x7f4a0414e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a0414e3c0 0x7f4a0414e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 -- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 msgr2=0x7f4a041e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 0x7f4a041e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01400640 1 -- 172.21.15.44:0/421983041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a041e3c30 con 0x7f4a0414f0c0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.215+0000 7f4a01e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 0x7f4a041e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f4a01400640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49f002a930 tx=0x7f49f0006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0041070 con 0x7f4a0414f0c0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f00038e0 con 0x7f4a0414f0c0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0038070 con 0x7f4a0414f0c0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a041e3f10 con 0x7f4a0414f0c0 2024-07-17T06:55:15.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.216+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0414ed70 con 0x7f4a0414f0c0 2024-07-17T06:55:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.218+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0406ee80 con 0x7f4a0414f0c0 2024-07-17T06:55:15.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.219+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f49f0044070 con 0x7f4a0414f0c0 2024-07-17T06:55:15.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.220+0000 7f49fa000640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 0x7f49e4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.220+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f49f00b7eb0 con 0x7f4a0414f0c0 2024-07-17T06:55:15.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.221+0000 7f4a01e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 0x7f49e4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.222+0000 7f4a01e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 0x7f49e4079070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49f4001860 tx=0x7f49f4001200 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.224+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f49f0080fa0 con 0x7f4a0414f0c0 2024-07-17T06:55:15.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 -- 172.21.15.44:0/313760271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a500cbab0 msgr2=0x7f9a5006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 --2- 172.21.15.44:0/313760271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a500cbab0 0x7f9a5006d620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005fd0 tx=0x7f9a3c02a540 comp rx=0 tx=0).stop 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 -- 172.21.15.44:0/313760271 shutdown_connections 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 --2- 172.21.15.44:0/313760271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5006ed30 0x7f9a5015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 --2- 172.21.15.44:0/313760271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a500cbab0 0x7f9a5006d620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 -- 172.21.15.44:0/313760271 >> 172.21.15.44:0/313760271 conn(0x7f9a500b0eb0 msgr2=0x7f9a500b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 -- 172.21.15.44:0/313760271 shutdown_connections 2024-07-17T06:55:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.254+0000 7f9a55600640 1 -- 172.21.15.44:0/313760271 wait complete. 2024-07-17T06:55:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.256+0000 7f9a55600640 1 Processor -- start 2024-07-17T06:55:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.256+0000 7f9a55600640 1 -- start start 2024-07-17T06:55:15.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a500cbab0 0x7f9a5014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a50152220 0x7f9a5014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500cd750 con 0x7f9a5006ed30 2024-07-17T06:55:15.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500cd5d0 con 0x7f9a500cbab0 2024-07-17T06:55:15.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a55600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a500cd8d0 con 0x7f9a50152220 2024-07-17T06:55:15.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a4ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a4ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37922/0 (socket says 172.21.15.44:37922) 2024-07-17T06:55:15.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.257+0000 7f9a4ea00640 1 -- 172.21.15.44:0/273426232 learned_addr learned my addr 172.21.15.44:0/273426232 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:15.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4f400640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a50152220 0x7f9a5014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4e000640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a500cbab0 0x7f9a5014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 -- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a50152220 msgr2=0x7f9a5014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a50152220 0x7f9a5014d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 -- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a500cbab0 msgr2=0x7f9a5014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a500cbab0 0x7f9a5014d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 -- 172.21.15.44:0/273426232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501575f0 con 0x7f9a5006ed30 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.258+0000 7f9a4ea00640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c3f0 tx=0x7f9a4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.259+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40017070 con 0x7f9a5006ed30 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.259+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a400035f0 con 0x7f9a5006ed30 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.259+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40003a40 con 0x7f9a5006ed30 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.259+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a500ae1f0 con 0x7f9a5006ed30 2024-07-17T06:55:15.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.259+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a500add00 con 0x7f9a5006ed30 2024-07-17T06:55:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.260+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a500cddf0 con 0x7f9a5006ed30 2024-07-17T06:55:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.262+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9a4001a070 con 0x7f9a5006ed30 2024-07-17T06:55:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.263+0000 7f9a46a00640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 0x7f9a30079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.264+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9a40097e40 con 0x7f9a5006ed30 2024-07-17T06:55:15.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.264+0000 7f9a4e000640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 0x7f9a30079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.265+0000 7f9a4e000640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 0x7f9a30079070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c033000 tx=0x7f9a3c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.266+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9a40021020 con 0x7f9a5006ed30 2024-07-17T06:55:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 -- 172.21.15.44:0/3714614855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58e4003ac0 msgr2=0x7f58e4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 --2- 172.21.15.44:0/3714614855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58e4003ac0 0x7f58e4003ec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c710 tx=0x7f58e4005830 comp rx=0 tx=0).stop 2024-07-17T06:55:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 -- 172.21.15.44:0/3714614855 shutdown_connections 2024-07-17T06:55:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 --2- 172.21.15.44:0/3714614855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58e4003ac0 0x7f58e4003ec0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 -- 172.21.15.44:0/3714614855 >> 172.21.15.44:0/3714614855 conn(0x7f58fc0b4ff0 msgr2=0x7f58fc0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 -- 172.21.15.44:0/3714614855 shutdown_connections 2024-07-17T06:55:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.292+0000 7f5902000640 1 -- 172.21.15.44:0/3714614855 wait complete. 2024-07-17T06:55:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.294+0000 7f5902000640 1 Processor -- start 2024-07-17T06:55:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.294+0000 7f5902000640 1 -- start start 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58e4003ac0 0x7f58fc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc1f74d0 0x7f58fc14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc0b2860 con 0x7f58fc1f4650 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc0b26e0 con 0x7f58fc1f74d0 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5902000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58fc0cfdf0 con 0x7f58e4003ac0 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5901600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5901600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37942/0 (socket says 172.21.15.44:37942) 2024-07-17T06:55:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5901600640 1 -- 172.21.15.44:0/2310511302 learned_addr learned my addr 172.21.15.44:0/2310511302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f5900c00640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58e4003ac0 0x7f58fc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.295+0000 7f58fbe00640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc1f74d0 0x7f58fc14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 -- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58e4003ac0 msgr2=0x7f58fc15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58e4003ac0 0x7f58fc15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 -- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc1f74d0 msgr2=0x7f58fc14d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc1f74d0 0x7f58fc14d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 -- 172.21.15.44:0/2310511302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc1f7950 con 0x7f58fc1f4650 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5901600640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4004b20 con 0x7f58fc1f4650 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58f4004db0 con 0x7f58fc1f4650 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f400e9e0 con 0x7f58fc1f4650 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.296+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc158d40 con 0x7f58fc1f4650 2024-07-17T06:55:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.297+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc1f7eb0 con 0x7f58fc1f4650 2024-07-17T06:55:15.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.298+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc0d02c0 con 0x7f58fc1f4650 2024-07-17T06:55:15.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.299+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f58f401a020 con 0x7f58fc1f4650 2024-07-17T06:55:15.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.300+0000 7f58f8c00640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 0x7f58dc0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.300+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58f4098630 con 0x7f58fc1f4650 2024-07-17T06:55:15.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.301+0000 7f5900c00640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 0x7f58dc0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.302+0000 7f5900c00640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 0x7f58dc0790a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58f0032000 tx=0x7f58f0003760 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.305+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f58fc1f7950 con 0x7f58fc1f4650 2024-07-17T06:55:15.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.440+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4a041e40e0 con 0x7f4a0414f0c0 2024-07-17T06:55:15.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.441+0000 7f49fa000640 1 -- 172.21.15.44:0/421983041 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49f00891e0 con 0x7f4a0414f0c0 2024-07-17T06:55:15.441 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-07-17T06:55:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 msgr2=0x7f49e4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 0x7f49e4079070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49f4001860 tx=0x7f49f4001200 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 msgr2=0x7f4a0415cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49f002a930 tx=0x7f49f0006290 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 shutdown_connections 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f49e4076bb0 0x7f49e4079070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a0414e3c0 0x7f4a0414e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a0414f0c0 0x7f4a0415cff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 --2- 172.21.15.44:0/421983041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49e8005910 0x7f4a041e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 >> 172.21.15.44:0/421983041 conn(0x7f4a040b0eb0 msgr2=0x7f4a040ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.444+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 shutdown_connections 2024-07-17T06:55:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.445+0000 7f4a08e00640 1 -- 172.21.15.44:0/421983041 wait complete. 2024-07-17T06:55:15.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.6 2024-07-17T06:55:15.469 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:15.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.477+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9a50151970 con 0x7f9a5006ed30 2024-07-17T06:55:15.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.478+0000 7f9a46a00640 1 -- 172.21.15.44:0/273426232 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a40060fb0 con 0x7f9a5006ed30 2024-07-17T06:55:15.478 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-07-17T06:55:15.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 msgr2=0x7f9a30079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 0x7f9a30079070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c033000 tx=0x7f9a3c004730 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 msgr2=0x7f9a501f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c3f0 tx=0x7f9a4000c8b0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 shutdown_connections 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a30076bb0 0x7f9a30079070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a50152220 0x7f9a5014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a500cbab0 0x7f9a5014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 --2- 172.21.15.44:0/273426232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5006ed30 0x7f9a501f44e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.480+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 >> 172.21.15.44:0/273426232 conn(0x7f9a500b0eb0 msgr2=0x7f9a500b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.481+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 shutdown_connections 2024-07-17T06:55:15.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.481+0000 7f9a55600640 1 -- 172.21.15.44:0/273426232 wait complete. 2024-07-17T06:55:15.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509449 for osd.5 2024-07-17T06:55:15.498 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:15.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.521+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f58fc0d2e90 con 0x7f58fc1f4650 2024-07-17T06:55:15.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.521+0000 7f58f8c00640 1 -- 172.21.15.44:0/2310511302 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58f40617c0 con 0x7f58fc1f4650 2024-07-17T06:55:15.521 INFO:teuthology.orchestra.run.smithi044.stdout:60129542153 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.524+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 msgr2=0x7f58dc0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.524+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 0x7f58dc0790a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58f0032000 tx=0x7f58f0003760 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.524+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 msgr2=0x7f58fc0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.524+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f4009b00 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 shutdown_connections 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc1f4650 0x7f58fc0cbcd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc1f74d0 0x7f58fc14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58dc076be0 0x7f58dc0790a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 --2- 172.21.15.44:0/2310511302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58e4003ac0 0x7f58fc15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 >> 172.21.15.44:0/2310511302 conn(0x7f58fc0b4ff0 msgr2=0x7f58fc0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 shutdown_connections 2024-07-17T06:55:15.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.525+0000 7f5902000640 1 -- 172.21.15.44:0/2310511302 wait complete. 2024-07-17T06:55:15.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.7 2024-07-17T06:55:15.542 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:15.542 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T06:55:15.542 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:55:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 -- 172.21.15.44:0/2703387802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b5400 msgr2=0x7f24a815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 --2- 172.21.15.44:0/2703387802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b5400 0x7f24a815e2d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009830 tx=0x7f24a002a560 comp rx=0 tx=0).stop 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 -- 172.21.15.44:0/2703387802 shutdown_connections 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 --2- 172.21.15.44:0/2703387802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b5400 0x7f24a815e2d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 --2- 172.21.15.44:0/2703387802 >> v2:172.21.15.175:3300/0 conn(0x7f24a80b8c80 0x7f24a80b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 -- 172.21.15.44:0/2703387802 >> 172.21.15.44:0/2703387802 conn(0x7f24a80cfec0 msgr2=0x7f24a80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.797+0000 7f24a7e00640 1 -- 172.21.15.44:0/2703387802 shutdown_connections 2024-07-17T06:55:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.798+0000 7f24a7e00640 1 -- 172.21.15.44:0/2703387802 wait complete. 2024-07-17T06:55:15.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.799+0000 7f24a7e00640 1 Processor -- start 2024-07-17T06:55:15.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.799+0000 7f24a7e00640 1 -- start start 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 0x7f24a81f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 0x7f24a81f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24a806e0c0 con 0x7f24a80b8c80 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24a80b9380 con 0x7f24a80ce5c0 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.800+0000 7f24a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24a806e240 con 0x7f24a80b5400 2024-07-17T06:55:15.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37960/0 (socket says 172.21.15.44:37960) 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 -- 172.21.15.44:0/1796744059 learned_addr learned my addr 172.21.15.44:0/1796744059 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6a00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 0x7f24a81f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a7400640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 0x7f24a81f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 -- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 msgr2=0x7f24a81f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 0x7f24a81f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 -- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 msgr2=0x7f24a81f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 0x7f24a81f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 -- 172.21.15.44:0/1796744059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80cb560 con 0x7f24a80b8c80 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6a00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 0x7f24a81f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a7400640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 0x7f24a81f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:15.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.801+0000 7f24a6000640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f249000cbb0 tx=0x7f249000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.802+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490016020 con 0x7f24a80b8c80 2024-07-17T06:55:15.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.802+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249000ae30 con 0x7f24a80b8c80 2024-07-17T06:55:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.802+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490012680 con 0x7f24a80b8c80 2024-07-17T06:55:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.802+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a815c0f0 con 0x7f24a80b8c80 2024-07-17T06:55:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.802+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a80cbb00 con 0x7f24a80b8c80 2024-07-17T06:55:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.803+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a806e6f0 con 0x7f24a80b8c80 2024-07-17T06:55:15.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.804+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f2490019020 con 0x7f24a80b8c80 2024-07-17T06:55:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.805+0000 7f249ea00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 0x7f2484079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.805+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2490098d50 con 0x7f24a80b8c80 2024-07-17T06:55:15.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.806+0000 7f24a6a00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 0x7f2484079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:15.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.807+0000 7f24a6a00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 0x7f2484079070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2498009b40 tx=0x7f2498005e50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:15.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:15.809+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2490020020 con 0x7f24a80b8c80 2024-07-17T06:55:16.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.009+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24a80d25d0 con 0x7f2484076bb0 2024-07-17T06:55:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.026+0000 7f249ea00640 1 -- 172.21.15.44:0/1796744059 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332526 (secure 0 0 0) 0x7f24a80d25d0 con 0x7f2484076bb0 2024-07-17T06:55:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:16.034 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.034+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 msgr2=0x7f2484079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.034+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 0x7f2484079070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2498009b40 tx=0x7f2498005e50 comp rx=0 tx=0).stop 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.034+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 msgr2=0x7f24a81f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.034+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f249000cbb0 tx=0x7f249000a430 comp rx=0 tx=0).stop 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 shutdown_connections 2024-07-17T06:55:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24a80ce5c0 0x7f24a81f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24a80b8c80 0x7f24a81f5090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2484076bb0 0x7f2484079070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 --2- 172.21.15.44:0/1796744059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24a80b5400 0x7f24a81f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 >> 172.21.15.44:0/1796744059 conn(0x7f24a80cfec0 msgr2=0x7f24a80b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 shutdown_connections 2024-07-17T06:55:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.035+0000 7f24a7e00640 1 -- 172.21.15.44:0/1796744059 wait complete. 2024-07-17T06:55:16.055 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-07-17T06:55:15.720992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":154,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":3317760,"data_stored":2004920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006374"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569050+0000","last_change":"2024-07-17T06:54:48.289026+0000","last_active":"2024-07-17T06:54:54.569050+0000","last_peered":"2024-07-17T06:54:54.569050+0000","last_clean":"2024-07-17T06:54:54.569050+0000","last_became_active":"2024-07-17T06:54:48.288767+0000","last_became_peered":"2024-07-17T06:54:48.288767+0000","last_unstale":"2024-07-17T06:54:54.569050+0000","last_undegraded":"2024-07-17T06:54:54.569050+0000","last_fullsized":"2024-07-17T06:54:54.569050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572074+0000","last_change":"2024-07-17T06:54:49.320600+0000","last_active":"2024-07-17T06:54:54.572074+0000","last_peered":"2024-07-17T06:54:54.572074+0000","last_clean":"2024-07-17T06:54:54.572074+0000","last_became_active":"2024-07-17T06:54:49.319878+0000","last_became_peered":"2024-07-17T06:54:49.319878+0000","last_unstale":"2024-07-17T06:54:54.572074+0000","last_undegraded":"2024-07-17T06:54:54.572074+0000","last_fullsized":"2024-07-17T06:54:54.572074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569764+0000","last_change":"2024-07-17T06:54:53.670763+0000","last_active":"2024-07-17T06:54:54.569764+0000","last_peered":"2024-07-17T06:54:54.569764+0000","last_clean":"2024-07-17T06:54:54.569764+0000","last_became_active":"2024-07-17T06:54:53.670345+0000","last_became_peered":"2024-07-17T06:54:53.670345+0000","last_unstale":"2024-07-17T06:54:54.569764+0000","last_undegraded":"2024-07-17T06:54:54.569764+0000","last_fullsized":"2024-07-17T06:54:54.569764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:19.165767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572200+0000","last_change":"2024-07-17T06:54:48.267830+0000","last_active":"2024-07-17T06:54:54.572200+0000","last_peered":"2024-07-17T06:54:54.572200+0000","last_clean":"2024-07-17T06:54:54.572200+0000","last_became_active":"2024-07-17T06:54:48.267458+0000","last_became_peered":"2024-07-17T06:54:48.267458+0000","last_unstale":"2024-07-17T06:54:54.572200+0000","last_undegraded":"2024-07-17T06:54:54.572200+0000","last_fullsized":"2024-07-17T06:54:54.572200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566732+0000","last_change":"2024-07-17T06:54:49.313799+0000","last_active":"2024-07-17T06:54:54.566732+0000","last_peered":"2024-07-17T06:54:54.566732+0000","last_clean":"2024-07-17T06:54:54.566732+0000","last_became_active":"2024-07-17T06:54:49.313240+0000","last_became_peered":"2024-07-17T06:54:49.313240+0000","last_unstale":"2024-07-17T06:54:54.566732+0000","last_undegraded":"2024-07-17T06:54:54.566732+0000","last_fullsized":"2024-07-17T06:54:54.566732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340034+0000","last_change":"2024-07-17T06:54:53.356992+0000","last_active":"2024-07-17T06:54:54.340034+0000","last_peered":"2024-07-17T06:54:54.340034+0000","last_clean":"2024-07-17T06:54:54.340034+0000","last_became_active":"2024-07-17T06:54:53.356521+0000","last_became_peered":"2024-07-17T06:54:53.356521+0000","last_unstale":"2024-07-17T06:54:54.340034+0000","last_undegraded":"2024-07-17T06:54:54.340034+0000","last_fullsized":"2024-07-17T06:54:54.340034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:33.058963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569075+0000","last_change":"2024-07-17T06:54:48.285421+0000","last_active":"2024-07-17T06:54:54.569075+0000","last_peered":"2024-07-17T06:54:54.569075+0000","last_clean":"2024-07-17T06:54:54.569075+0000","last_became_active":"2024-07-17T06:54:48.285132+0000","last_became_peered":"2024-07-17T06:54:48.285132+0000","last_unstale":"2024-07-17T06:54:54.569075+0000","last_undegraded":"2024-07-17T06:54:54.569075+0000","last_fullsized":"2024-07-17T06:54:54.569075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:38.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566825+0000","last_change":"2024-07-17T06:54:49.311977+0000","last_active":"2024-07-17T06:54:54.566825+0000","last_peered":"2024-07-17T06:54:54.566825+0000","last_clean":"2024-07-17T06:54:54.566825+0000","last_became_active":"2024-07-17T06:54:49.311364+0000","last_became_peered":"2024-07-17T06:54:49.311364+0000","last_unstale":"2024-07-17T06:54:54.566825+0000","last_undegraded":"2024-07-17T06:54:54.566825+0000","last_fullsized":"2024-07-17T06:54:54.566825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:21.034101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341182+0000","last_change":"2024-07-17T06:54:53.362264+0000","last_active":"2024-07-17T06:54:54.341182+0000","last_peered":"2024-07-17T06:54:54.341182+0000","last_clean":"2024-07-17T06:54:54.341182+0000","last_became_active":"2024-07-17T06:54:53.361863+0000","last_became_peered":"2024-07-17T06:54:53.361863+0000","last_unstale":"2024-07-17T06:54:54.341182+0000","last_undegraded":"2024-07-17T06:54:54.341182+0000","last_fullsized":"2024-07-17T06:54:54.341182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:22.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567855+0000","last_change":"2024-07-17T06:54:48.288847+0000","last_active":"2024-07-17T06:54:54.567855+0000","last_peered":"2024-07-17T06:54:54.567855+0000","last_clean":"2024-07-17T06:54:54.567855+0000","last_became_active":"2024-07-17T06:54:48.288504+0000","last_became_peered":"2024-07-17T06:54:48.288504+0000","last_unstale":"2024-07-17T06:54:54.567855+0000","last_undegraded":"2024-07-17T06:54:54.567855+0000","last_fullsized":"2024-07-17T06:54:54.567855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:42.640200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567324+0000","last_change":"2024-07-17T06:54:49.311645+0000","last_active":"2024-07-17T06:54:54.567324+0000","last_peered":"2024-07-17T06:54:54.567324+0000","last_clean":"2024-07-17T06:54:54.567324+0000","last_became_active":"2024-07-17T06:54:49.311336+0000","last_became_peered":"2024-07-17T06:54:49.311336+0000","last_unstale":"2024-07-17T06:54:54.567324+0000","last_undegraded":"2024-07-17T06:54:54.567324+0000","last_fullsized":"2024-07-17T06:54:54.567324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342111+0000","last_change":"2024-07-17T06:54:53.357717+0000","last_active":"2024-07-17T06:54:54.342111+0000","last_peered":"2024-07-17T06:54:54.342111+0000","last_clean":"2024-07-17T06:54:54.342111+0000","last_became_active":"2024-07-17T06:54:53.356799+0000","last_became_peered":"2024-07-17T06:54:53.356799+0000","last_unstale":"2024-07-17T06:54:54.342111+0000","last_undegraded":"2024-07-17T06:54:54.342111+0000","last_fullsized":"2024-07-17T06:54:54.342111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:29.584528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341883+0000","last_change":"2024-07-17T06:54:49.299990+0000","last_active":"2024-07-17T06:54:54.341883+0000","last_peered":"2024-07-17T06:54:54.341883+0000","last_clean":"2024-07-17T06:54:54.341883+0000","last_became_active":"2024-07-17T06:54:49.299679+0000","last_became_peered":"2024-07-17T06:54:49.299679+0000","last_unstale":"2024-07-17T06:54:54.341883+0000","last_undegraded":"2024-07-17T06:54:54.341883+0000","last_fullsized":"2024-07-17T06:54:54.341883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568387+0000","last_change":"2024-07-17T06:54:48.285371+0000","last_active":"2024-07-17T06:54:54.568387+0000","last_peered":"2024-07-17T06:54:54.568387+0000","last_clean":"2024-07-17T06:54:54.568387+0000","last_became_active":"2024-07-17T06:54:48.284925+0000","last_became_peered":"2024-07-17T06:54:48.284925+0000","last_unstale":"2024-07-17T06:54:54.568387+0000","last_undegraded":"2024-07-17T06:54:54.568387+0000","last_fullsized":"2024-07-17T06:54:54.568387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:21.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344985+0000","last_change":"2024-07-17T06:54:53.353487+0000","last_active":"2024-07-17T06:54:54.344985+0000","last_peered":"2024-07-17T06:54:54.344985+0000","last_clean":"2024-07-17T06:54:54.344985+0000","last_became_active":"2024-07-17T06:54:53.353037+0000","last_became_peered":"2024-07-17T06:54:53.353037+0000","last_unstale":"2024-07-17T06:54:54.344985+0000","last_undegraded":"2024-07-17T06:54:54.344985+0000","last_fullsized":"2024-07-17T06:54:54.344985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:51.743135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570785+0000","last_change":"2024-07-17T06:54:49.300898+0000","last_active":"2024-07-17T06:54:54.570785+0000","last_peered":"2024-07-17T06:54:54.570785+0000","last_clean":"2024-07-17T06:54:54.570785+0000","last_became_active":"2024-07-17T06:54:49.300631+0000","last_became_peered":"2024-07-17T06:54:49.300631+0000","last_unstale":"2024-07-17T06:54:54.570785+0000","last_undegraded":"2024-07-17T06:54:54.570785+0000","last_fullsized":"2024-07-17T06:54:54.570785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568669+0000","last_change":"2024-07-17T06:54:48.274040+0000","last_active":"2024-07-17T06:54:54.568669+0000","last_peered":"2024-07-17T06:54:54.568669+0000","last_clean":"2024-07-17T06:54:54.568669+0000","last_became_active":"2024-07-17T06:54:48.272837+0000","last_became_peered":"2024-07-17T06:54:48.272837+0000","last_unstale":"2024-07-17T06:54:54.568669+0000","last_undegraded":"2024-07-17T06:54:54.568669+0000","last_fullsized":"2024-07-17T06:54:54.568669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344238+0000","last_change":"2024-07-17T06:54:53.356025+0000","last_active":"2024-07-17T06:54:54.344238+0000","last_peered":"2024-07-17T06:54:54.344238+0000","last_clean":"2024-07-17T06:54:54.344238+0000","last_became_active":"2024-07-17T06:54:53.355539+0000","last_became_peered":"2024-07-17T06:54:53.355539+0000","last_unstale":"2024-07-17T06:54:54.344238+0000","last_undegraded":"2024-07-17T06:54:54.344238+0000","last_fullsized":"2024-07-17T06:54:54.344238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:14.244752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567230+0000","last_change":"2024-07-17T06:54:49.303989+0000","last_active":"2024-07-17T06:54:54.567230+0000","last_peered":"2024-07-17T06:54:54.567230+0000","last_clean":"2024-07-17T06:54:54.567230+0000","last_became_active":"2024-07-17T06:54:49.303723+0000","last_became_peered":"2024-07-17T06:54:49.303723+0000","last_unstale":"2024-07-17T06:54:54.567230+0000","last_undegraded":"2024-07-17T06:54:54.567230+0000","last_fullsized":"2024-07-17T06:54:54.567230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568506+0000","last_change":"2024-07-17T06:54:48.280459+0000","last_active":"2024-07-17T06:54:54.568506+0000","last_peered":"2024-07-17T06:54:54.568506+0000","last_clean":"2024-07-17T06:54:54.568506+0000","last_became_active":"2024-07-17T06:54:48.280196+0000","last_became_peered":"2024-07-17T06:54:48.280196+0000","last_unstale":"2024-07-17T06:54:54.568506+0000","last_undegraded":"2024-07-17T06:54:54.568506+0000","last_fullsized":"2024-07-17T06:54:54.568506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571786+0000","last_change":"2024-07-17T06:54:53.672128+0000","last_active":"2024-07-17T06:54:54.571786+0000","last_peered":"2024-07-17T06:54:54.571786+0000","last_clean":"2024-07-17T06:54:54.571786+0000","last_became_active":"2024-07-17T06:54:53.671083+0000","last_became_peered":"2024-07-17T06:54:53.671083+0000","last_unstale":"2024-07-17T06:54:54.571786+0000","last_undegraded":"2024-07-17T06:54:54.571786+0000","last_fullsized":"2024-07-17T06:54:54.571786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341192+0000","last_change":"2024-07-17T06:54:49.301897+0000","last_active":"2024-07-17T06:54:54.341192+0000","last_peered":"2024-07-17T06:54:54.341192+0000","last_clean":"2024-07-17T06:54:54.341192+0000","last_became_active":"2024-07-17T06:54:49.301602+0000","last_became_peered":"2024-07-17T06:54:49.301602+0000","last_unstale":"2024-07-17T06:54:54.341192+0000","last_undegraded":"2024-07-17T06:54:54.341192+0000","last_fullsized":"2024-07-17T06:54:54.341192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:39.268743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568075+0000","last_change":"2024-07-17T06:54:48.280905+0000","last_active":"2024-07-17T06:54:54.568075+0000","last_peered":"2024-07-17T06:54:54.568075+0000","last_clean":"2024-07-17T06:54:54.568075+0000","last_became_active":"2024-07-17T06:54:48.280538+0000","last_became_peered":"2024-07-17T06:54:48.280538+0000","last_unstale":"2024-07-17T06:54:54.568075+0000","last_undegraded":"2024-07-17T06:54:54.568075+0000","last_fullsized":"2024-07-17T06:54:54.568075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343510+0000","last_change":"2024-07-17T06:54:53.356455+0000","last_active":"2024-07-17T06:54:54.343510+0000","last_peered":"2024-07-17T06:54:54.343510+0000","last_clean":"2024-07-17T06:54:54.343510+0000","last_became_active":"2024-07-17T06:54:53.355594+0000","last_became_peered":"2024-07-17T06:54:53.355594+0000","last_unstale":"2024-07-17T06:54:54.343510+0000","last_undegraded":"2024-07-17T06:54:54.343510+0000","last_fullsized":"2024-07-17T06:54:54.343510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:37.799189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569678+0000","last_change":"2024-07-17T06:54:49.306398+0000","last_active":"2024-07-17T06:54:54.569678+0000","last_peered":"2024-07-17T06:54:54.569678+0000","last_clean":"2024-07-17T06:54:54.569678+0000","last_became_active":"2024-07-17T06:54:49.305914+0000","last_became_peered":"2024-07-17T06:54:49.305914+0000","last_unstale":"2024-07-17T06:54:54.569678+0000","last_undegraded":"2024-07-17T06:54:54.569678+0000","last_fullsized":"2024-07-17T06:54:54.569678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343534+0000","last_change":"2024-07-17T06:54:48.281630+0000","last_active":"2024-07-17T06:54:54.343534+0000","last_peered":"2024-07-17T06:54:54.343534+0000","last_clean":"2024-07-17T06:54:54.343534+0000","last_became_active":"2024-07-17T06:54:48.281368+0000","last_became_peered":"2024-07-17T06:54:48.281368+0000","last_unstale":"2024-07-17T06:54:54.343534+0000","last_undegraded":"2024-07-17T06:54:54.343534+0000","last_fullsized":"2024-07-17T06:54:54.343534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:09.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340594+0000","last_change":"2024-07-17T06:54:53.685324+0000","last_active":"2024-07-17T06:54:54.340594+0000","last_peered":"2024-07-17T06:54:54.340594+0000","last_clean":"2024-07-17T06:54:54.340594+0000","last_became_active":"2024-07-17T06:54:53.683906+0000","last_became_peered":"2024-07-17T06:54:53.683906+0000","last_unstale":"2024-07-17T06:54:54.340594+0000","last_undegraded":"2024-07-17T06:54:54.340594+0000","last_fullsized":"2024-07-17T06:54:54.340594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:31.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572498+0000","last_change":"2024-07-17T06:54:49.307258+0000","last_active":"2024-07-17T06:54:54.572498+0000","last_peered":"2024-07-17T06:54:54.572498+0000","last_clean":"2024-07-17T06:54:54.572498+0000","last_became_active":"2024-07-17T06:54:49.306989+0000","last_became_peered":"2024-07-17T06:54:49.306989+0000","last_unstale":"2024-07-17T06:54:54.572498+0000","last_undegraded":"2024-07-17T06:54:54.572498+0000","last_fullsized":"2024-07-17T06:54:54.572498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:03.300938+0000","last_change":"2024-07-17T06:54:48.273115+0000","last_active":"2024-07-17T06:55:03.300938+0000","last_peered":"2024-07-17T06:55:03.300938+0000","last_clean":"2024-07-17T06:55:03.300938+0000","last_became_active":"2024-07-17T06:54:48.272804+0000","last_became_peered":"2024-07-17T06:54:48.272804+0000","last_unstale":"2024-07-17T06:55:03.300938+0000","last_undegraded":"2024-07-17T06:55:03.300938+0000","last_fullsized":"2024-07-17T06:55:03.300938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566623+0000","last_change":"2024-07-17T06:54:53.667515+0000","last_active":"2024-07-17T06:54:54.566623+0000","last_peered":"2024-07-17T06:54:54.566623+0000","last_clean":"2024-07-17T06:54:54.566623+0000","last_became_active":"2024-07-17T06:54:53.665920+0000","last_became_peered":"2024-07-17T06:54:53.665920+0000","last_unstale":"2024-07-17T06:54:54.566623+0000","last_undegraded":"2024-07-17T06:54:54.566623+0000","last_fullsized":"2024-07-17T06:54:54.566623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:20.348618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569843+0000","last_change":"2024-07-17T06:54:49.296691+0000","last_active":"2024-07-17T06:54:54.569843+0000","last_peered":"2024-07-17T06:54:54.569843+0000","last_clean":"2024-07-17T06:54:54.569843+0000","last_became_active":"2024-07-17T06:54:49.296340+0000","last_became_peered":"2024-07-17T06:54:49.296340+0000","last_unstale":"2024-07-17T06:54:54.569843+0000","last_undegraded":"2024-07-17T06:54:54.569843+0000","last_fullsized":"2024-07-17T06:54:54.569843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569522+0000","last_change":"2024-07-17T06:54:48.288182+0000","last_active":"2024-07-17T06:54:54.569522+0000","last_peered":"2024-07-17T06:54:54.569522+0000","last_clean":"2024-07-17T06:54:54.569522+0000","last_became_active":"2024-07-17T06:54:48.287723+0000","last_became_peered":"2024-07-17T06:54:48.287723+0000","last_unstale":"2024-07-17T06:54:54.569522+0000","last_undegraded":"2024-07-17T06:54:54.569522+0000","last_fullsized":"2024-07-17T06:54:54.569522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:54.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565754+0000","last_change":"2024-07-17T06:54:53.672017+0000","last_active":"2024-07-17T06:54:54.565754+0000","last_peered":"2024-07-17T06:54:54.565754+0000","last_clean":"2024-07-17T06:54:54.565754+0000","last_became_active":"2024-07-17T06:54:53.671036+0000","last_became_peered":"2024-07-17T06:54:53.671036+0000","last_unstale":"2024-07-17T06:54:54.565754+0000","last_undegraded":"2024-07-17T06:54:54.565754+0000","last_fullsized":"2024-07-17T06:54:54.565754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567594+0000","last_change":"2024-07-17T06:54:49.320403+0000","last_active":"2024-07-17T06:54:54.567594+0000","last_peered":"2024-07-17T06:54:54.567594+0000","last_clean":"2024-07-17T06:54:54.567594+0000","last_became_active":"2024-07-17T06:54:49.320119+0000","last_became_peered":"2024-07-17T06:54:49.320119+0000","last_unstale":"2024-07-17T06:54:54.567594+0000","last_undegraded":"2024-07-17T06:54:54.567594+0000","last_fullsized":"2024-07-17T06:54:54.567594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:41.108822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569854+0000","last_change":"2024-07-17T06:54:48.271506+0000","last_active":"2024-07-17T06:54:54.569854+0000","last_peered":"2024-07-17T06:54:54.569854+0000","last_clean":"2024-07-17T06:54:54.569854+0000","last_became_active":"2024-07-17T06:54:48.271181+0000","last_became_peered":"2024-07-17T06:54:48.271181+0000","last_unstale":"2024-07-17T06:54:54.569854+0000","last_undegraded":"2024-07-17T06:54:54.569854+0000","last_fullsized":"2024-07-17T06:54:54.569854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568214+0000","last_change":"2024-07-17T06:54:53.358600+0000","last_active":"2024-07-17T06:54:54.568214+0000","last_peered":"2024-07-17T06:54:54.568214+0000","last_clean":"2024-07-17T06:54:54.568214+0000","last_became_active":"2024-07-17T06:54:53.355359+0000","last_became_peered":"2024-07-17T06:54:53.355359+0000","last_unstale":"2024-07-17T06:54:54.568214+0000","last_undegraded":"2024-07-17T06:54:54.568214+0000","last_fullsized":"2024-07-17T06:54:54.568214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344036+0000","last_change":"2024-07-17T06:54:49.305202+0000","last_active":"2024-07-17T06:54:54.344036+0000","last_peered":"2024-07-17T06:54:54.344036+0000","last_clean":"2024-07-17T06:54:54.344036+0000","last_became_active":"2024-07-17T06:54:49.304856+0000","last_became_peered":"2024-07-17T06:54:49.304856+0000","last_unstale":"2024-07-17T06:54:54.344036+0000","last_undegraded":"2024-07-17T06:54:54.344036+0000","last_fullsized":"2024-07-17T06:54:54.344036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:00.298763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566783+0000","last_change":"2024-07-17T06:54:48.279231+0000","last_active":"2024-07-17T06:54:54.566783+0000","last_peered":"2024-07-17T06:54:54.566783+0000","last_clean":"2024-07-17T06:54:54.566783+0000","last_became_active":"2024-07-17T06:54:48.277422+0000","last_became_peered":"2024-07-17T06:54:48.277422+0000","last_unstale":"2024-07-17T06:54:54.566783+0000","last_undegraded":"2024-07-17T06:54:54.566783+0000","last_fullsized":"2024-07-17T06:54:54.566783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569758+0000","last_change":"2024-07-17T06:54:53.350972+0000","last_active":"2024-07-17T06:54:54.569758+0000","last_peered":"2024-07-17T06:54:54.569758+0000","last_clean":"2024-07-17T06:54:54.569758+0000","last_became_active":"2024-07-17T06:54:53.347571+0000","last_became_peered":"2024-07-17T06:54:53.347571+0000","last_unstale":"2024-07-17T06:54:54.569758+0000","last_undegraded":"2024-07-17T06:54:54.569758+0000","last_fullsized":"2024-07-17T06:54:54.569758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567608+0000","last_change":"2024-07-17T06:54:49.302823+0000","last_active":"2024-07-17T06:54:54.567608+0000","last_peered":"2024-07-17T06:54:54.567608+0000","last_clean":"2024-07-17T06:54:54.567608+0000","last_became_active":"2024-07-17T06:54:49.302434+0000","last_became_peered":"2024-07-17T06:54:49.302434+0000","last_unstale":"2024-07-17T06:54:54.567608+0000","last_undegraded":"2024-07-17T06:54:54.567608+0000","last_fullsized":"2024-07-17T06:54:54.567608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339350+0000","last_change":"2024-07-17T06:54:48.268185+0000","last_active":"2024-07-17T06:54:54.339350+0000","last_peered":"2024-07-17T06:54:54.339350+0000","last_clean":"2024-07-17T06:54:54.339350+0000","last_became_active":"2024-07-17T06:54:48.267807+0000","last_became_peered":"2024-07-17T06:54:48.267807+0000","last_unstale":"2024-07-17T06:54:54.339350+0000","last_undegraded":"2024-07-17T06:54:54.339350+0000","last_fullsized":"2024-07-17T06:54:54.339350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:28.391194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572146+0000","last_change":"2024-07-17T06:54:53.345628+0000","last_active":"2024-07-17T06:54:54.572146+0000","last_peered":"2024-07-17T06:54:54.572146+0000","last_clean":"2024-07-17T06:54:54.572146+0000","last_became_active":"2024-07-17T06:54:53.345269+0000","last_became_peered":"2024-07-17T06:54:53.345269+0000","last_unstale":"2024-07-17T06:54:54.572146+0000","last_undegraded":"2024-07-17T06:54:54.572146+0000","last_fullsized":"2024-07-17T06:54:54.572146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565847+0000","last_change":"2024-07-17T06:54:49.302929+0000","last_active":"2024-07-17T06:54:54.565847+0000","last_peered":"2024-07-17T06:54:54.565847+0000","last_clean":"2024-07-17T06:54:54.565847+0000","last_became_active":"2024-07-17T06:54:49.302598+0000","last_became_peered":"2024-07-17T06:54:49.302598+0000","last_unstale":"2024-07-17T06:54:54.565847+0000","last_undegraded":"2024-07-17T06:54:54.565847+0000","last_fullsized":"2024-07-17T06:54:54.565847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568955+0000","last_change":"2024-07-17T06:54:48.284164+0000","last_active":"2024-07-17T06:54:54.568955+0000","last_peered":"2024-07-17T06:54:54.568955+0000","last_clean":"2024-07-17T06:54:54.568955+0000","last_became_active":"2024-07-17T06:54:48.282885+0000","last_became_peered":"2024-07-17T06:54:48.282885+0000","last_unstale":"2024-07-17T06:54:54.568955+0000","last_undegraded":"2024-07-17T06:54:54.568955+0000","last_fullsized":"2024-07-17T06:54:54.568955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:05.674177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569179+0000","last_change":"2024-07-17T06:54:53.677311+0000","last_active":"2024-07-17T06:54:54.569179+0000","last_peered":"2024-07-17T06:54:54.569179+0000","last_clean":"2024-07-17T06:54:54.569179+0000","last_became_active":"2024-07-17T06:54:53.670958+0000","last_became_peered":"2024-07-17T06:54:53.670958+0000","last_unstale":"2024-07-17T06:54:54.569179+0000","last_undegraded":"2024-07-17T06:54:54.569179+0000","last_fullsized":"2024-07-17T06:54:54.569179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566963+0000","last_change":"2024-07-17T06:54:49.312042+0000","last_active":"2024-07-17T06:54:54.566963+0000","last_peered":"2024-07-17T06:54:54.566963+0000","last_clean":"2024-07-17T06:54:54.566963+0000","last_became_active":"2024-07-17T06:54:49.311741+0000","last_became_peered":"2024-07-17T06:54:49.311741+0000","last_unstale":"2024-07-17T06:54:54.566963+0000","last_undegraded":"2024-07-17T06:54:54.566963+0000","last_fullsized":"2024-07-17T06:54:54.566963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567568+0000","last_change":"2024-07-17T06:54:48.277447+0000","last_active":"2024-07-17T06:54:54.567568+0000","last_peered":"2024-07-17T06:54:54.567568+0000","last_clean":"2024-07-17T06:54:54.567568+0000","last_became_active":"2024-07-17T06:54:48.273874+0000","last_became_peered":"2024-07-17T06:54:48.273874+0000","last_unstale":"2024-07-17T06:54:54.567568+0000","last_undegraded":"2024-07-17T06:54:54.567568+0000","last_fullsized":"2024-07-17T06:54:54.567568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568469+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:54:54.568469+0000","last_peered":"2024-07-17T06:54:54.568469+0000","last_clean":"2024-07-17T06:54:54.568469+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:54:54.568469+0000","last_undegraded":"2024-07-17T06:54:54.568469+0000","last_fullsized":"2024-07-17T06:54:54.568469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340950+0000","last_change":"2024-07-17T06:54:53.680957+0000","last_active":"2024-07-17T06:54:54.340950+0000","last_peered":"2024-07-17T06:54:54.340950+0000","last_clean":"2024-07-17T06:54:54.340950+0000","last_became_active":"2024-07-17T06:54:53.679463+0000","last_became_peered":"2024-07-17T06:54:53.679463+0000","last_unstale":"2024-07-17T06:54:54.340950+0000","last_undegraded":"2024-07-17T06:54:54.340950+0000","last_fullsized":"2024-07-17T06:54:54.340950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568688+0000","last_change":"2024-07-17T06:54:49.313201+0000","last_active":"2024-07-17T06:54:54.568688+0000","last_peered":"2024-07-17T06:54:54.568688+0000","last_clean":"2024-07-17T06:54:54.568688+0000","last_became_active":"2024-07-17T06:54:49.312937+0000","last_became_peered":"2024-07-17T06:54:49.312937+0000","last_unstale":"2024-07-17T06:54:54.568688+0000","last_undegraded":"2024-07-17T06:54:54.568688+0000","last_fullsized":"2024-07-17T06:54:54.568688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:16.869857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339687+0000","last_change":"2024-07-17T06:54:48.276023+0000","last_active":"2024-07-17T06:54:54.339687+0000","last_peered":"2024-07-17T06:54:54.339687+0000","last_clean":"2024-07-17T06:54:54.339687+0000","last_became_active":"2024-07-17T06:54:48.275283+0000","last_became_peered":"2024-07-17T06:54:48.275283+0000","last_unstale":"2024-07-17T06:54:54.339687+0000","last_undegraded":"2024-07-17T06:54:54.339687+0000","last_fullsized":"2024-07-17T06:54:54.339687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:52.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341589+0000","last_change":"2024-07-17T06:54:53.674509+0000","last_active":"2024-07-17T06:54:54.341589+0000","last_peered":"2024-07-17T06:54:54.341589+0000","last_clean":"2024-07-17T06:54:54.341589+0000","last_became_active":"2024-07-17T06:54:53.672806+0000","last_became_peered":"2024-07-17T06:54:53.672806+0000","last_unstale":"2024-07-17T06:54:54.341589+0000","last_undegraded":"2024-07-17T06:54:54.341589+0000","last_fullsized":"2024-07-17T06:54:54.341589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567064+0000","last_change":"2024-07-17T06:54:49.324849+0000","last_active":"2024-07-17T06:54:54.567064+0000","last_peered":"2024-07-17T06:54:54.567064+0000","last_clean":"2024-07-17T06:54:54.567064+0000","last_became_active":"2024-07-17T06:54:49.324195+0000","last_became_peered":"2024-07-17T06:54:49.324195+0000","last_unstale":"2024-07-17T06:54:54.567064+0000","last_undegraded":"2024-07-17T06:54:54.567064+0000","last_fullsized":"2024-07-17T06:54:54.567064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:45.753022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570696+0000","last_change":"2024-07-17T06:54:48.276038+0000","last_active":"2024-07-17T06:54:54.570696+0000","last_peered":"2024-07-17T06:54:54.570696+0000","last_clean":"2024-07-17T06:54:54.570696+0000","last_became_active":"2024-07-17T06:54:48.275729+0000","last_became_peered":"2024-07-17T06:54:48.275729+0000","last_unstale":"2024-07-17T06:54:54.570696+0000","last_undegraded":"2024-07-17T06:54:54.570696+0000","last_fullsized":"2024-07-17T06:54:54.570696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565953+0000","last_change":"2024-07-17T06:54:53.356746+0000","last_active":"2024-07-17T06:54:54.565953+0000","last_peered":"2024-07-17T06:54:54.565953+0000","last_clean":"2024-07-17T06:54:54.565953+0000","last_became_active":"2024-07-17T06:54:53.353358+0000","last_became_peered":"2024-07-17T06:54:53.353358+0000","last_unstale":"2024-07-17T06:54:54.565953+0000","last_undegraded":"2024-07-17T06:54:54.565953+0000","last_fullsized":"2024-07-17T06:54:54.565953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569283+0000","last_change":"2024-07-17T06:54:49.318930+0000","last_active":"2024-07-17T06:54:54.569283+0000","last_peered":"2024-07-17T06:54:54.569283+0000","last_clean":"2024-07-17T06:54:54.569283+0000","last_became_active":"2024-07-17T06:54:49.318606+0000","last_became_peered":"2024-07-17T06:54:49.318606+0000","last_unstale":"2024-07-17T06:54:54.569283+0000","last_undegraded":"2024-07-17T06:54:54.569283+0000","last_fullsized":"2024-07-17T06:54:54.569283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:10:28.608769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339970+0000","last_change":"2024-07-17T06:54:48.269152+0000","last_active":"2024-07-17T06:54:54.339970+0000","last_peered":"2024-07-17T06:54:54.339970+0000","last_clean":"2024-07-17T06:54:54.339970+0000","last_became_active":"2024-07-17T06:54:48.268914+0000","last_became_peered":"2024-07-17T06:54:48.268914+0000","last_unstale":"2024-07-17T06:54:54.339970+0000","last_undegraded":"2024-07-17T06:54:54.339970+0000","last_fullsized":"2024-07-17T06:54:54.339970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565453+0000","last_change":"2024-07-17T06:54:53.672373+0000","last_active":"2024-07-17T06:54:54.565453+0000","last_peered":"2024-07-17T06:54:54.565453+0000","last_clean":"2024-07-17T06:54:54.565453+0000","last_became_active":"2024-07-17T06:54:53.671288+0000","last_became_peered":"2024-07-17T06:54:53.671288+0000","last_unstale":"2024-07-17T06:54:54.565453+0000","last_undegraded":"2024-07-17T06:54:54.565453+0000","last_fullsized":"2024-07-17T06:54:54.565453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568889+0000","last_change":"2024-07-17T06:54:49.320151+0000","last_active":"2024-07-17T06:54:54.568889+0000","last_peered":"2024-07-17T06:54:54.568889+0000","last_clean":"2024-07-17T06:54:54.568889+0000","last_became_active":"2024-07-17T06:54:49.319845+0000","last_became_peered":"2024-07-17T06:54:49.319845+0000","last_unstale":"2024-07-17T06:54:54.568889+0000","last_undegraded":"2024-07-17T06:54:54.568889+0000","last_fullsized":"2024-07-17T06:54:54.568889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:12:25.836163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343920+0000","last_change":"2024-07-17T06:54:48.268964+0000","last_active":"2024-07-17T06:54:54.343920+0000","last_peered":"2024-07-17T06:54:54.343920+0000","last_clean":"2024-07-17T06:54:54.343920+0000","last_became_active":"2024-07-17T06:54:48.268564+0000","last_became_peered":"2024-07-17T06:54:48.268564+0000","last_unstale":"2024-07-17T06:54:54.343920+0000","last_undegraded":"2024-07-17T06:54:54.343920+0000","last_fullsized":"2024-07-17T06:54:54.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:47.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340002+0000","last_change":"2024-07-17T06:54:53.362347+0000","last_active":"2024-07-17T06:54:54.340002+0000","last_peered":"2024-07-17T06:54:54.340002+0000","last_clean":"2024-07-17T06:54:54.340002+0000","last_became_active":"2024-07-17T06:54:53.361896+0000","last_became_peered":"2024-07-17T06:54:53.361896+0000","last_unstale":"2024-07-17T06:54:54.340002+0000","last_undegraded":"2024-07-17T06:54:54.340002+0000","last_fullsized":"2024-07-17T06:54:54.340002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:22.522023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566168+0000","last_change":"2024-07-17T06:54:49.316448+0000","last_active":"2024-07-17T06:54:54.566168+0000","last_peered":"2024-07-17T06:54:54.566168+0000","last_clean":"2024-07-17T06:54:54.566168+0000","last_became_active":"2024-07-17T06:54:49.316042+0000","last_became_peered":"2024-07-17T06:54:49.316042+0000","last_unstale":"2024-07-17T06:54:54.566168+0000","last_undegraded":"2024-07-17T06:54:54.566168+0000","last_fullsized":"2024-07-17T06:54:54.566168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:33.731836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.338832+0000","last_change":"2024-07-17T06:54:48.270227+0000","last_active":"2024-07-17T06:54:54.338832+0000","last_peered":"2024-07-17T06:54:54.338832+0000","last_clean":"2024-07-17T06:54:54.338832+0000","last_became_active":"2024-07-17T06:54:48.269954+0000","last_became_peered":"2024-07-17T06:54:48.269954+0000","last_unstale":"2024-07-17T06:54:54.338832+0000","last_undegraded":"2024-07-17T06:54:54.338832+0000","last_fullsized":"2024-07-17T06:54:54.338832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.390020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568918+0000","last_change":"2024-07-17T06:54:53.356983+0000","last_active":"2024-07-17T06:54:54.568918+0000","last_peered":"2024-07-17T06:54:54.568918+0000","last_clean":"2024-07-17T06:54:54.568918+0000","last_became_active":"2024-07-17T06:54:53.356288+0000","last_became_peered":"2024-07-17T06:54:53.356288+0000","last_unstale":"2024-07-17T06:54:54.568918+0000","last_undegraded":"2024-07-17T06:54:54.568918+0000","last_fullsized":"2024-07-17T06:54:54.568918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339405+0000","last_change":"2024-07-17T06:54:49.294433+0000","last_active":"2024-07-17T06:54:54.339405+0000","last_peered":"2024-07-17T06:54:54.339405+0000","last_clean":"2024-07-17T06:54:54.339405+0000","last_became_active":"2024-07-17T06:54:49.293644+0000","last_became_peered":"2024-07-17T06:54:49.293644+0000","last_unstale":"2024-07-17T06:54:54.339405+0000","last_undegraded":"2024-07-17T06:54:54.339405+0000","last_fullsized":"2024-07-17T06:54:54.339405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340443+0000","last_change":"2024-07-17T06:54:48.267002+0000","last_active":"2024-07-17T06:54:54.340443+0000","last_peered":"2024-07-17T06:54:54.340443+0000","last_clean":"2024-07-17T06:54:54.340443+0000","last_became_active":"2024-07-17T06:54:48.266711+0000","last_became_peered":"2024-07-17T06:54:48.266711+0000","last_unstale":"2024-07-17T06:54:54.340443+0000","last_undegraded":"2024-07-17T06:54:54.340443+0000","last_fullsized":"2024-07-17T06:54:54.340443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572427+0000","last_change":"2024-07-17T06:54:53.678483+0000","last_active":"2024-07-17T06:54:54.572427+0000","last_peered":"2024-07-17T06:54:54.572427+0000","last_clean":"2024-07-17T06:54:54.572427+0000","last_became_active":"2024-07-17T06:54:53.677707+0000","last_became_peered":"2024-07-17T06:54:53.677707+0000","last_unstale":"2024-07-17T06:54:54.572427+0000","last_undegraded":"2024-07-17T06:54:54.572427+0000","last_fullsized":"2024-07-17T06:54:54.572427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:21.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568071+0000","last_change":"2024-07-17T06:54:49.319059+0000","last_active":"2024-07-17T06:54:54.568071+0000","last_peered":"2024-07-17T06:54:54.568071+0000","last_clean":"2024-07-17T06:54:54.568071+0000","last_became_active":"2024-07-17T06:54:49.318532+0000","last_became_peered":"2024-07-17T06:54:49.318532+0000","last_unstale":"2024-07-17T06:54:54.568071+0000","last_undegraded":"2024-07-17T06:54:54.568071+0000","last_fullsized":"2024-07-17T06:54:54.568071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:47.675783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566475+0000","last_change":"2024-07-17T06:54:48.279991+0000","last_active":"2024-07-17T06:54:54.566475+0000","last_peered":"2024-07-17T06:54:54.566475+0000","last_clean":"2024-07-17T06:54:54.566475+0000","last_became_active":"2024-07-17T06:54:48.279691+0000","last_became_peered":"2024-07-17T06:54:48.279691+0000","last_unstale":"2024-07-17T06:54:54.566475+0000","last_undegraded":"2024-07-17T06:54:54.566475+0000","last_fullsized":"2024-07-17T06:54:54.566475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571709+0000","last_change":"2024-07-17T06:54:53.677451+0000","last_active":"2024-07-17T06:54:54.571709+0000","last_peered":"2024-07-17T06:54:54.571709+0000","last_clean":"2024-07-17T06:54:54.571709+0000","last_became_active":"2024-07-17T06:54:53.677080+0000","last_became_peered":"2024-07-17T06:54:53.677080+0000","last_unstale":"2024-07-17T06:54:54.571709+0000","last_undegraded":"2024-07-17T06:54:54.571709+0000","last_fullsized":"2024-07-17T06:54:54.571709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567583+0000","last_change":"2024-07-17T06:54:49.302477+0000","last_active":"2024-07-17T06:54:54.567583+0000","last_peered":"2024-07-17T06:54:54.567583+0000","last_clean":"2024-07-17T06:54:54.567583+0000","last_became_active":"2024-07-17T06:54:49.302168+0000","last_became_peered":"2024-07-17T06:54:49.302168+0000","last_unstale":"2024-07-17T06:54:54.567583+0000","last_undegraded":"2024-07-17T06:54:54.567583+0000","last_fullsized":"2024-07-17T06:54:54.567583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:20.279165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566160+0000","last_change":"2024-07-17T06:54:48.278091+0000","last_active":"2024-07-17T06:54:54.566160+0000","last_peered":"2024-07-17T06:54:54.566160+0000","last_clean":"2024-07-17T06:54:54.566160+0000","last_became_active":"2024-07-17T06:54:48.277769+0000","last_became_peered":"2024-07-17T06:54:48.277769+0000","last_unstale":"2024-07-17T06:54:54.566160+0000","last_undegraded":"2024-07-17T06:54:54.566160+0000","last_fullsized":"2024-07-17T06:54:54.566160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567004+0000","last_change":"2024-07-17T06:54:53.672346+0000","last_active":"2024-07-17T06:54:54.567004+0000","last_peered":"2024-07-17T06:54:54.567004+0000","last_clean":"2024-07-17T06:54:54.567004+0000","last_became_active":"2024-07-17T06:54:53.670973+0000","last_became_peered":"2024-07-17T06:54:53.670973+0000","last_unstale":"2024-07-17T06:54:54.567004+0000","last_undegraded":"2024-07-17T06:54:54.567004+0000","last_fullsized":"2024-07-17T06:54:54.567004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568419+0000","last_change":"2024-07-17T06:54:49.314009+0000","last_active":"2024-07-17T06:54:54.568419+0000","last_peered":"2024-07-17T06:54:54.568419+0000","last_clean":"2024-07-17T06:54:54.568419+0000","last_became_active":"2024-07-17T06:54:49.313238+0000","last_became_peered":"2024-07-17T06:54:49.313238+0000","last_unstale":"2024-07-17T06:54:54.568419+0000","last_undegraded":"2024-07-17T06:54:54.568419+0000","last_fullsized":"2024-07-17T06:54:54.568419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:16.115918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:02.724870+0000","last_change":"2024-07-17T06:54:48.274173+0000","last_active":"2024-07-17T06:55:02.724870+0000","last_peered":"2024-07-17T06:55:02.724870+0000","last_clean":"2024-07-17T06:55:02.724870+0000","last_became_active":"2024-07-17T06:54:48.270354+0000","last_became_peered":"2024-07-17T06:54:48.270354+0000","last_unstale":"2024-07-17T06:55:02.724870+0000","last_undegraded":"2024-07-17T06:55:02.724870+0000","last_fullsized":"2024-07-17T06:55:02.724870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568505+0000","last_change":"2024-07-17T06:54:53.347808+0000","last_active":"2024-07-17T06:54:54.568505+0000","last_peered":"2024-07-17T06:54:54.568505+0000","last_clean":"2024-07-17T06:54:54.568505+0000","last_became_active":"2024-07-17T06:54:53.347369+0000","last_became_peered":"2024-07-17T06:54:53.347369+0000","last_unstale":"2024-07-17T06:54:54.568505+0000","last_undegraded":"2024-07-17T06:54:54.568505+0000","last_fullsized":"2024-07-17T06:54:54.568505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344599+0000","last_change":"2024-07-17T06:54:49.308852+0000","last_active":"2024-07-17T06:54:54.344599+0000","last_peered":"2024-07-17T06:54:54.344599+0000","last_clean":"2024-07-17T06:54:54.344599+0000","last_became_active":"2024-07-17T06:54:49.308337+0000","last_became_peered":"2024-07-17T06:54:49.308337+0000","last_unstale":"2024-07-17T06:54:54.344599+0000","last_undegraded":"2024-07-17T06:54:54.344599+0000","last_fullsized":"2024-07-17T06:54:54.344599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:06.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570694+0000","last_change":"2024-07-17T06:54:48.282475+0000","last_active":"2024-07-17T06:54:54.570694+0000","last_peered":"2024-07-17T06:54:54.570694+0000","last_clean":"2024-07-17T06:54:54.570694+0000","last_became_active":"2024-07-17T06:54:48.282039+0000","last_became_peered":"2024-07-17T06:54:48.282039+0000","last_unstale":"2024-07-17T06:54:54.570694+0000","last_undegraded":"2024-07-17T06:54:54.570694+0000","last_fullsized":"2024-07-17T06:54:54.570694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344160+0000","last_change":"2024-07-17T06:54:53.353664+0000","last_active":"2024-07-17T06:54:54.344160+0000","last_peered":"2024-07-17T06:54:54.344160+0000","last_clean":"2024-07-17T06:54:54.344160+0000","last_became_active":"2024-07-17T06:54:53.351779+0000","last_became_peered":"2024-07-17T06:54:53.351779+0000","last_unstale":"2024-07-17T06:54:54.344160+0000","last_undegraded":"2024-07-17T06:54:54.344160+0000","last_fullsized":"2024-07-17T06:54:54.344160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:20.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567440+0000","last_change":"2024-07-17T06:54:49.313557+0000","last_active":"2024-07-17T06:54:54.567440+0000","last_peered":"2024-07-17T06:54:54.567440+0000","last_clean":"2024-07-17T06:54:54.567440+0000","last_became_active":"2024-07-17T06:54:49.312875+0000","last_became_peered":"2024-07-17T06:54:49.312875+0000","last_unstale":"2024-07-17T06:54:54.567440+0000","last_undegraded":"2024-07-17T06:54:54.567440+0000","last_fullsized":"2024-07-17T06:54:54.567440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567055+0000","last_change":"2024-07-17T06:54:48.286081+0000","last_active":"2024-07-17T06:54:54.567055+0000","last_peered":"2024-07-17T06:54:54.567055+0000","last_clean":"2024-07-17T06:54:54.567055+0000","last_became_active":"2024-07-17T06:54:48.285327+0000","last_became_peered":"2024-07-17T06:54:48.285327+0000","last_unstale":"2024-07-17T06:54:54.567055+0000","last_undegraded":"2024-07-17T06:54:54.567055+0000","last_fullsized":"2024-07-17T06:54:54.567055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567082+0000","last_change":"2024-07-17T06:54:53.353255+0000","last_active":"2024-07-17T06:54:54.567082+0000","last_peered":"2024-07-17T06:54:54.567082+0000","last_clean":"2024-07-17T06:54:54.567082+0000","last_became_active":"2024-07-17T06:54:53.352945+0000","last_became_peered":"2024-07-17T06:54:53.352945+0000","last_unstale":"2024-07-17T06:54:54.567082+0000","last_undegraded":"2024-07-17T06:54:54.567082+0000","last_fullsized":"2024-07-17T06:54:54.567082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:06.889450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566804+0000","last_change":"2024-07-17T06:54:49.318904+0000","last_active":"2024-07-17T06:54:54.566804+0000","last_peered":"2024-07-17T06:54:54.566804+0000","last_clean":"2024-07-17T06:54:54.566804+0000","last_became_active":"2024-07-17T06:54:49.316753+0000","last_became_peered":"2024-07-17T06:54:49.316753+0000","last_unstale":"2024-07-17T06:54:54.566804+0000","last_undegraded":"2024-07-17T06:54:54.566804+0000","last_fullsized":"2024-07-17T06:54:54.566804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:45.913897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343278+0000","last_change":"2024-07-17T06:54:48.286978+0000","last_active":"2024-07-17T06:54:54.343278+0000","last_peered":"2024-07-17T06:54:54.343278+0000","last_clean":"2024-07-17T06:54:54.343278+0000","last_became_active":"2024-07-17T06:54:48.286613+0000","last_became_peered":"2024-07-17T06:54:48.286613+0000","last_unstale":"2024-07-17T06:54:54.343278+0000","last_undegraded":"2024-07-17T06:54:54.343278+0000","last_fullsized":"2024-07-17T06:54:54.343278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570149+0000","last_change":"2024-07-17T06:54:53.671456+0000","last_active":"2024-07-17T06:54:54.570149+0000","last_peered":"2024-07-17T06:54:54.570149+0000","last_clean":"2024-07-17T06:54:54.570149+0000","last_became_active":"2024-07-17T06:54:53.670354+0000","last_became_peered":"2024-07-17T06:54:53.670354+0000","last_unstale":"2024-07-17T06:54:54.570149+0000","last_undegraded":"2024-07-17T06:54:54.570149+0000","last_fullsized":"2024-07-17T06:54:54.570149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571489+0000","last_change":"2024-07-17T06:54:53.682714+0000","last_active":"2024-07-17T06:54:54.571489+0000","last_peered":"2024-07-17T06:54:54.571489+0000","last_clean":"2024-07-17T06:54:54.571489+0000","last_became_active":"2024-07-17T06:54:53.681884+0000","last_became_peered":"2024-07-17T06:54:53.681884+0000","last_unstale":"2024-07-17T06:54:54.571489+0000","last_undegraded":"2024-07-17T06:54:54.571489+0000","last_fullsized":"2024-07-17T06:54:54.571489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568224+0000","last_change":"2024-07-17T06:54:49.305906+0000","last_active":"2024-07-17T06:54:54.568224+0000","last_peered":"2024-07-17T06:54:54.568224+0000","last_clean":"2024-07-17T06:54:54.568224+0000","last_became_active":"2024-07-17T06:54:49.305633+0000","last_became_peered":"2024-07-17T06:54:49.305633+0000","last_unstale":"2024-07-17T06:54:54.568224+0000","last_undegraded":"2024-07-17T06:54:54.568224+0000","last_fullsized":"2024-07-17T06:54:54.568224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565775+0000","last_change":"2024-07-17T06:54:48.275763+0000","last_active":"2024-07-17T06:54:54.565775+0000","last_peered":"2024-07-17T06:54:54.565775+0000","last_clean":"2024-07-17T06:54:54.565775+0000","last_became_active":"2024-07-17T06:54:48.275073+0000","last_became_peered":"2024-07-17T06:54:48.275073+0000","last_unstale":"2024-07-17T06:54:54.565775+0000","last_undegraded":"2024-07-17T06:54:54.565775+0000","last_fullsized":"2024-07-17T06:54:54.565775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343084+0000","last_change":"2024-07-17T06:54:53.347306+0000","last_active":"2024-07-17T06:54:54.343084+0000","last_peered":"2024-07-17T06:54:54.343084+0000","last_clean":"2024-07-17T06:54:54.343084+0000","last_became_active":"2024-07-17T06:54:53.346541+0000","last_became_peered":"2024-07-17T06:54:53.346541+0000","last_unstale":"2024-07-17T06:54:54.343084+0000","last_undegraded":"2024-07-17T06:54:54.343084+0000","last_fullsized":"2024-07-17T06:54:54.343084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567129+0000","last_change":"2024-07-17T06:54:49.299691+0000","last_active":"2024-07-17T06:54:54.567129+0000","last_peered":"2024-07-17T06:54:54.567129+0000","last_clean":"2024-07-17T06:54:54.567129+0000","last_became_active":"2024-07-17T06:54:49.299370+0000","last_became_peered":"2024-07-17T06:54:49.299370+0000","last_unstale":"2024-07-17T06:54:54.567129+0000","last_undegraded":"2024-07-17T06:54:54.567129+0000","last_fullsized":"2024-07-17T06:54:54.567129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570373+0000","last_change":"2024-07-17T06:54:48.276504+0000","last_active":"2024-07-17T06:54:54.570373+0000","last_peered":"2024-07-17T06:54:54.570373+0000","last_clean":"2024-07-17T06:54:54.570373+0000","last_became_active":"2024-07-17T06:54:48.276041+0000","last_became_peered":"2024-07-17T06:54:48.276041+0000","last_unstale":"2024-07-17T06:54:54.570373+0000","last_undegraded":"2024-07-17T06:54:54.570373+0000","last_fullsized":"2024-07-17T06:54:54.570373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342215+0000","last_change":"2024-07-17T06:54:48.267874+0000","last_active":"2024-07-17T06:54:54.342215+0000","last_peered":"2024-07-17T06:54:54.342215+0000","last_clean":"2024-07-17T06:54:54.342215+0000","last_became_active":"2024-07-17T06:54:48.267487+0000","last_became_peered":"2024-07-17T06:54:48.267487+0000","last_unstale":"2024-07-17T06:54:54.342215+0000","last_undegraded":"2024-07-17T06:54:54.342215+0000","last_fullsized":"2024-07-17T06:54:54.342215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566178+0000","last_change":"2024-07-17T06:54:49.306648+0000","last_active":"2024-07-17T06:54:54.566178+0000","last_peered":"2024-07-17T06:54:54.566178+0000","last_clean":"2024-07-17T06:54:54.566178+0000","last_became_active":"2024-07-17T06:54:49.305001+0000","last_became_peered":"2024-07-17T06:54:49.305001+0000","last_unstale":"2024-07-17T06:54:54.566178+0000","last_undegraded":"2024-07-17T06:54:54.566178+0000","last_fullsized":"2024-07-17T06:54:54.566178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339673+0000","last_change":"2024-07-17T06:54:53.360122+0000","last_active":"2024-07-17T06:54:54.339673+0000","last_peered":"2024-07-17T06:54:54.339673+0000","last_clean":"2024-07-17T06:54:54.339673+0000","last_became_active":"2024-07-17T06:54:53.356530+0000","last_became_peered":"2024-07-17T06:54:53.356530+0000","last_unstale":"2024-07-17T06:54:54.339673+0000","last_undegraded":"2024-07-17T06:54:54.339673+0000","last_fullsized":"2024-07-17T06:54:54.339673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:41.753064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570947+0000","last_change":"2024-07-17T06:54:48.277409+0000","last_active":"2024-07-17T06:54:54.570947+0000","last_peered":"2024-07-17T06:54:54.570947+0000","last_clean":"2024-07-17T06:54:54.570947+0000","last_became_active":"2024-07-17T06:54:48.277087+0000","last_became_peered":"2024-07-17T06:54:48.277087+0000","last_unstale":"2024-07-17T06:54:54.570947+0000","last_undegraded":"2024-07-17T06:54:54.570947+0000","last_fullsized":"2024-07-17T06:54:54.570947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:56:17.744184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342397+0000","last_change":"2024-07-17T06:54:49.300524+0000","last_active":"2024-07-17T06:54:54.342397+0000","last_peered":"2024-07-17T06:54:54.342397+0000","last_clean":"2024-07-17T06:54:54.342397+0000","last_became_active":"2024-07-17T06:54:49.300136+0000","last_became_peered":"2024-07-17T06:54:49.300136+0000","last_unstale":"2024-07-17T06:54:54.342397+0000","last_undegraded":"2024-07-17T06:54:54.342397+0000","last_fullsized":"2024-07-17T06:54:54.342397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:29.406070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571827+0000","last_change":"2024-07-17T06:54:53.355705+0000","last_active":"2024-07-17T06:54:54.571827+0000","last_peered":"2024-07-17T06:54:54.571827+0000","last_clean":"2024-07-17T06:54:54.571827+0000","last_became_active":"2024-07-17T06:54:53.355294+0000","last_became_peered":"2024-07-17T06:54:53.355294+0000","last_unstale":"2024-07-17T06:54:54.571827+0000","last_undegraded":"2024-07-17T06:54:54.571827+0000","last_fullsized":"2024-07-17T06:54:54.571827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:32:32.729442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339117+0000","last_change":"2024-07-17T06:54:49.294765+0000","last_active":"2024-07-17T06:54:54.339117+0000","last_peered":"2024-07-17T06:54:54.339117+0000","last_clean":"2024-07-17T06:54:54.339117+0000","last_became_active":"2024-07-17T06:54:49.294095+0000","last_became_peered":"2024-07-17T06:54:49.294095+0000","last_unstale":"2024-07-17T06:54:54.339117+0000","last_undegraded":"2024-07-17T06:54:54.339117+0000","last_fullsized":"2024-07-17T06:54:54.339117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339066+0000","last_change":"2024-07-17T06:54:49.299295+0000","last_active":"2024-07-17T06:54:54.339066+0000","last_peered":"2024-07-17T06:54:54.339066+0000","last_clean":"2024-07-17T06:54:54.339066+0000","last_became_active":"2024-07-17T06:54:49.298385+0000","last_became_peered":"2024-07-17T06:54:49.298385+0000","last_unstale":"2024-07-17T06:54:54.339066+0000","last_undegraded":"2024-07-17T06:54:54.339066+0000","last_fullsized":"2024-07-17T06:54:54.339066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:55.920311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566243+0000","last_change":"2024-07-17T06:54:49.325974+0000","last_active":"2024-07-17T06:54:54.566243+0000","last_peered":"2024-07-17T06:54:54.566243+0000","last_clean":"2024-07-17T06:54:54.566243+0000","last_became_active":"2024-07-17T06:54:49.322413+0000","last_became_peered":"2024-07-17T06:54:49.322413+0000","last_unstale":"2024-07-17T06:54:54.566243+0000","last_undegraded":"2024-07-17T06:54:54.566243+0000","last_fullsized":"2024-07-17T06:54:54.566243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:18.496012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566055+0000","last_change":"2024-07-17T06:54:49.308310+0000","last_active":"2024-07-17T06:54:54.566055+0000","last_peered":"2024-07-17T06:54:54.566055+0000","last_clean":"2024-07-17T06:54:54.566055+0000","last_became_active":"2024-07-17T06:54:49.304819+0000","last_became_peered":"2024-07-17T06:54:49.304819+0000","last_unstale":"2024-07-17T06:54:54.566055+0000","last_undegraded":"2024-07-17T06:54:54.566055+0000","last_fullsized":"2024-07-17T06:54:54.566055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571870+0000","last_change":"2024-07-17T06:54:49.312400+0000","last_active":"2024-07-17T06:54:54.571870+0000","last_peered":"2024-07-17T06:54:54.571870+0000","last_clean":"2024-07-17T06:54:54.571870+0000","last_became_active":"2024-07-17T06:54:49.311958+0000","last_became_peered":"2024-07-17T06:54:49.311958+0000","last_unstale":"2024-07-17T06:54:54.571870+0000","last_undegraded":"2024-07-17T06:54:54.571870+0000","last_fullsized":"2024-07-17T06:54:54.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.321280+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.320912+0000","last_became_peered":"2024-07-17T06:54:49.320912+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341006+0000","last_change":"2024-07-17T06:54:49.297628+0000","last_active":"2024-07-17T06:54:54.341006+0000","last_peered":"2024-07-17T06:54:54.341006+0000","last_clean":"2024-07-17T06:54:54.341006+0000","last_became_active":"2024-07-17T06:54:49.296871+0000","last_became_peered":"2024-07-17T06:54:49.296871+0000","last_unstale":"2024-07-17T06:54:54.341006+0000","last_undegraded":"2024-07-17T06:54:54.341006+0000","last_fullsized":"2024-07-17T06:54:54.341006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572513+0000","last_change":"2024-07-17T06:54:49.303253+0000","last_active":"2024-07-17T06:54:54.572513+0000","last_peered":"2024-07-17T06:54:54.572513+0000","last_clean":"2024-07-17T06:54:54.572513+0000","last_became_active":"2024-07-17T06:54:49.302956+0000","last_became_peered":"2024-07-17T06:54:49.302956+0000","last_unstale":"2024-07-17T06:54:54.572513+0000","last_undegraded":"2024-07-17T06:54:54.572513+0000","last_fullsized":"2024-07-17T06:54:54.572513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565697+0000","last_change":"2024-07-17T06:54:49.306156+0000","last_active":"2024-07-17T06:54:54.565697+0000","last_peered":"2024-07-17T06:54:54.565697+0000","last_clean":"2024-07-17T06:54:54.565697+0000","last_became_active":"2024-07-17T06:54:49.305858+0000","last_became_peered":"2024-07-17T06:54:49.305858+0000","last_unstale":"2024-07-17T06:54:54.565697+0000","last_undegraded":"2024-07-17T06:54:54.565697+0000","last_fullsized":"2024-07-17T06:54:54.565697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.301568+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.301244+0000","last_became_peered":"2024-07-17T06:54:49.301244+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567719+0000","last_change":"2024-07-17T06:54:49.316422+0000","last_active":"2024-07-17T06:54:54.567719+0000","last_peered":"2024-07-17T06:54:54.567719+0000","last_clean":"2024-07-17T06:54:54.567719+0000","last_became_active":"2024-07-17T06:54:49.316133+0000","last_became_peered":"2024-07-17T06:54:49.316133+0000","last_unstale":"2024-07-17T06:54:54.567719+0000","last_undegraded":"2024-07-17T06:54:54.567719+0000","last_fullsized":"2024-07-17T06:54:54.567719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565901+0000","last_change":"2024-07-17T06:54:49.305491+0000","last_active":"2024-07-17T06:54:54.565901+0000","last_peered":"2024-07-17T06:54:54.565901+0000","last_clean":"2024-07-17T06:54:54.565901+0000","last_became_active":"2024-07-17T06:54:49.304629+0000","last_became_peered":"2024-07-17T06:54:49.304629+0000","last_unstale":"2024-07-17T06:54:54.565901+0000","last_undegraded":"2024-07-17T06:54:54.565901+0000","last_fullsized":"2024-07-17T06:54:54.565901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343504+0000","last_change":"2024-07-17T06:54:49.309637+0000","last_active":"2024-07-17T06:54:54.343504+0000","last_peered":"2024-07-17T06:54:54.343504+0000","last_clean":"2024-07-17T06:54:54.343504+0000","last_became_active":"2024-07-17T06:54:49.309243+0000","last_became_peered":"2024-07-17T06:54:49.309243+0000","last_unstale":"2024-07-17T06:54:54.343504+0000","last_undegraded":"2024-07-17T06:54:54.343504+0000","last_fullsized":"2024-07-17T06:54:54.343504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569329+0000","last_change":"2024-07-17T06:54:49.323709+0000","last_active":"2024-07-17T06:54:54.569329+0000","last_peered":"2024-07-17T06:54:54.569329+0000","last_clean":"2024-07-17T06:54:54.569329+0000","last_became_active":"2024-07-17T06:54:49.323324+0000","last_became_peered":"2024-07-17T06:54:49.323324+0000","last_unstale":"2024-07-17T06:54:54.569329+0000","last_undegraded":"2024-07-17T06:54:54.569329+0000","last_fullsized":"2024-07-17T06:54:54.569329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339810+0000","last_change":"2024-07-17T06:54:49.286063+0000","last_active":"2024-07-17T06:54:54.339810+0000","last_peered":"2024-07-17T06:54:54.339810+0000","last_clean":"2024-07-17T06:54:54.339810+0000","last_became_active":"2024-07-17T06:54:49.283989+0000","last_became_peered":"2024-07-17T06:54:49.283989+0000","last_unstale":"2024-07-17T06:54:54.339810+0000","last_undegraded":"2024-07-17T06:54:54.339810+0000","last_fullsized":"2024-07-17T06:54:54.339810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:43.655076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567362+0000","last_change":"2024-07-17T06:54:49.319544+0000","last_active":"2024-07-17T06:54:54.567362+0000","last_peered":"2024-07-17T06:54:54.567362+0000","last_clean":"2024-07-17T06:54:54.567362+0000","last_became_active":"2024-07-17T06:54:49.319285+0000","last_became_peered":"2024-07-17T06:54:49.319285+0000","last_unstale":"2024-07-17T06:54:54.567362+0000","last_undegraded":"2024-07-17T06:54:54.567362+0000","last_fullsized":"2024-07-17T06:54:54.567362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:58.821222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565429+0000","last_change":"2024-07-17T06:54:49.305371+0000","last_active":"2024-07-17T06:54:54.565429+0000","last_peered":"2024-07-17T06:54:54.565429+0000","last_clean":"2024-07-17T06:54:54.565429+0000","last_became_active":"2024-07-17T06:54:49.305069+0000","last_became_peered":"2024-07-17T06:54:49.305069+0000","last_unstale":"2024-07-17T06:54:54.565429+0000","last_undegraded":"2024-07-17T06:54:54.565429+0000","last_fullsized":"2024-07-17T06:54:54.565429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568226+0000","last_change":"2024-07-17T06:54:49.314419+0000","last_active":"2024-07-17T06:54:54.568226+0000","last_peered":"2024-07-17T06:54:54.568226+0000","last_clean":"2024-07-17T06:54:54.568226+0000","last_became_active":"2024-07-17T06:54:49.314106+0000","last_became_peered":"2024-07-17T06:54:49.314106+0000","last_unstale":"2024-07-17T06:54:54.568226+0000","last_undegraded":"2024-07-17T06:54:54.568226+0000","last_fullsized":"2024-07-17T06:54:54.568226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566591+0000","last_change":"2024-07-17T06:54:49.297635+0000","last_active":"2024-07-17T06:54:54.566591+0000","last_peered":"2024-07-17T06:54:54.566591+0000","last_clean":"2024-07-17T06:54:54.566591+0000","last_became_active":"2024-07-17T06:54:49.297252+0000","last_became_peered":"2024-07-17T06:54:49.297252+0000","last_unstale":"2024-07-17T06:54:54.566591+0000","last_undegraded":"2024-07-17T06:54:54.566591+0000","last_fullsized":"2024-07-17T06:54:54.566591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:02.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570670+0000","last_change":"2024-07-17T06:54:49.311510+0000","last_active":"2024-07-17T06:54:54.570670+0000","last_peered":"2024-07-17T06:54:54.570670+0000","last_clean":"2024-07-17T06:54:54.570670+0000","last_became_active":"2024-07-17T06:54:49.311208+0000","last_became_peered":"2024-07-17T06:54:49.311208+0000","last_unstale":"2024-07-17T06:54:54.570670+0000","last_undegraded":"2024-07-17T06:54:54.570670+0000","last_fullsized":"2024-07-17T06:54:54.570670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344453+0000","last_change":"2024-07-17T06:54:49.300592+0000","last_active":"2024-07-17T06:54:54.344453+0000","last_peered":"2024-07-17T06:54:54.344453+0000","last_clean":"2024-07-17T06:54:54.344453+0000","last_became_active":"2024-07-17T06:54:49.300233+0000","last_became_peered":"2024-07-17T06:54:49.300233+0000","last_unstale":"2024-07-17T06:54:54.344453+0000","last_undegraded":"2024-07-17T06:54:54.344453+0000","last_fullsized":"2024-07-17T06:54:54.344453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:53.264056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341453+0000","last_change":"2024-07-17T06:54:49.304204+0000","last_active":"2024-07-17T06:54:54.341453+0000","last_peered":"2024-07-17T06:54:54.341453+0000","last_clean":"2024-07-17T06:54:54.341453+0000","last_became_active":"2024-07-17T06:54:49.303907+0000","last_became_peered":"2024-07-17T06:54:49.303907+0000","last_unstale":"2024-07-17T06:54:54.341453+0000","last_undegraded":"2024-07-17T06:54:54.341453+0000","last_fullsized":"2024-07-17T06:54:54.341453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340757+0000","last_change":"2024-07-17T06:54:49.297834+0000","last_active":"2024-07-17T06:54:54.340757+0000","last_peered":"2024-07-17T06:54:54.340757+0000","last_clean":"2024-07-17T06:54:54.340757+0000","last_became_active":"2024-07-17T06:54:49.297296+0000","last_became_peered":"2024-07-17T06:54:49.297296+0000","last_unstale":"2024-07-17T06:54:54.340757+0000","last_undegraded":"2024-07-17T06:54:54.340757+0000","last_fullsized":"2024-07-17T06:54:54.340757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569209+0000","last_change":"2024-07-17T06:54:49.318654+0000","last_active":"2024-07-17T06:54:54.569209+0000","last_peered":"2024-07-17T06:54:54.569209+0000","last_clean":"2024-07-17T06:54:54.569209+0000","last_became_active":"2024-07-17T06:54:49.318380+0000","last_became_peered":"2024-07-17T06:54:49.318380+0000","last_unstale":"2024-07-17T06:54:54.569209+0000","last_undegraded":"2024-07-17T06:54:54.569209+0000","last_fullsized":"2024-07-17T06:54:54.569209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:12.140489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566890+0000","last_change":"2024-07-17T06:54:49.297152+0000","last_active":"2024-07-17T06:54:54.566890+0000","last_peered":"2024-07-17T06:54:54.566890+0000","last_clean":"2024-07-17T06:54:54.566890+0000","last_became_active":"2024-07-17T06:54:49.296830+0000","last_became_peered":"2024-07-17T06:54:49.296830+0000","last_unstale":"2024-07-17T06:54:54.566890+0000","last_undegraded":"2024-07-17T06:54:54.566890+0000","last_fullsized":"2024-07-17T06:54:54.566890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566123+0000","last_change":"2024-07-17T06:54:49.313713+0000","last_active":"2024-07-17T06:54:54.566123+0000","last_peered":"2024-07-17T06:54:54.566123+0000","last_clean":"2024-07-17T06:54:54.566123+0000","last_became_active":"2024-07-17T06:54:49.313419+0000","last_became_peered":"2024-07-17T06:54:49.313419+0000","last_unstale":"2024-07-17T06:54:54.566123+0000","last_undegraded":"2024-07-17T06:54:54.566123+0000","last_fullsized":"2024-07-17T06:54:54.566123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:25.517176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565979+0000","last_change":"2024-07-17T06:54:49.304993+0000","last_active":"2024-07-17T06:54:54.565979+0000","last_peered":"2024-07-17T06:54:54.565979+0000","last_clean":"2024-07-17T06:54:54.565979+0000","last_became_active":"2024-07-17T06:54:49.304475+0000","last_became_peered":"2024-07-17T06:54:49.304475+0000","last_unstale":"2024-07-17T06:54:54.565979+0000","last_undegraded":"2024-07-17T06:54:54.565979+0000","last_fullsized":"2024-07-17T06:54:54.565979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565951+0000","last_change":"2024-07-17T06:54:49.319946+0000","last_active":"2024-07-17T06:54:54.565951+0000","last_peered":"2024-07-17T06:54:54.565951+0000","last_clean":"2024-07-17T06:54:54.565951+0000","last_became_active":"2024-07-17T06:54:49.319685+0000","last_became_peered":"2024-07-17T06:54:49.319685+0000","last_unstale":"2024-07-17T06:54:54.565951+0000","last_undegraded":"2024-07-17T06:54:54.565951+0000","last_fullsized":"2024-07-17T06:54:54.565951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:10.177399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566540+0000","last_change":"2024-07-17T06:54:49.324468+0000","last_active":"2024-07-17T06:54:54.566540+0000","last_peered":"2024-07-17T06:54:54.566540+0000","last_clean":"2024-07-17T06:54:54.566540+0000","last_became_active":"2024-07-17T06:54:49.323700+0000","last_became_peered":"2024-07-17T06:54:49.323700+0000","last_unstale":"2024-07-17T06:54:54.566540+0000","last_undegraded":"2024-07-17T06:54:54.566540+0000","last_fullsized":"2024-07-17T06:54:54.566540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:05.797436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567150+0000","last_change":"2024-07-17T06:54:49.290953+0000","last_active":"2024-07-17T06:54:54.567150+0000","last_peered":"2024-07-17T06:54:54.567150+0000","last_clean":"2024-07-17T06:54:54.567150+0000","last_became_active":"2024-07-17T06:54:49.290069+0000","last_became_peered":"2024-07-17T06:54:49.290069+0000","last_unstale":"2024-07-17T06:54:54.567150+0000","last_undegraded":"2024-07-17T06:54:54.567150+0000","last_fullsized":"2024-07-17T06:54:54.567150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571210+0000","last_change":"2024-07-17T06:54:49.301304+0000","last_active":"2024-07-17T06:54:54.571210+0000","last_peered":"2024-07-17T06:54:54.571210+0000","last_clean":"2024-07-17T06:54:54.571210+0000","last_became_active":"2024-07-17T06:54:49.300865+0000","last_became_peered":"2024-07-17T06:54:49.300865+0000","last_unstale":"2024-07-17T06:54:54.571210+0000","last_undegraded":"2024-07-17T06:54:54.571210+0000","last_fullsized":"2024-07-17T06:54:54.571210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569004+0000","last_change":"2024-07-17T06:54:49.324041+0000","last_active":"2024-07-17T06:54:54.569004+0000","last_peered":"2024-07-17T06:54:54.569004+0000","last_clean":"2024-07-17T06:54:54.569004+0000","last_became_active":"2024-07-17T06:54:49.323747+0000","last_became_peered":"2024-07-17T06:54:49.323747+0000","last_unstale":"2024-07-17T06:54:54.569004+0000","last_undegraded":"2024-07-17T06:54:54.569004+0000","last_fullsized":"2024-07-17T06:54:54.569004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:04:46.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":101953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":102023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":692869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":692813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":101989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":101863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:55:16.060 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:55:16.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 -- 172.21.15.44:0/2682337392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc1522c0 msgr2=0x7f07dc0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2682337392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc1522c0 0x7f07dc0ad790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009900 tx=0x7f07cc02a600 comp rx=0 tx=0).stop 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 -- 172.21.15.44:0/2682337392 shutdown_connections 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2682337392 >> v2:172.21.15.175:3300/0 conn(0x7f07dc0ac930 0x7f07dc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2682337392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc1522c0 0x7f07dc0ad790 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 -- 172.21.15.44:0/2682337392 >> 172.21.15.44:0/2682337392 conn(0x7f07dc0b0fb0 msgr2=0x7f07dc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.307+0000 7f07e3a00640 1 -- 172.21.15.44:0/2682337392 shutdown_connections 2024-07-17T06:55:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.308+0000 7f07e3a00640 1 -- 172.21.15.44:0/2682337392 wait complete. 2024-07-17T06:55:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.310+0000 7f07e3a00640 1 Processor -- start 2024-07-17T06:55:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.310+0000 7f07e3a00640 1 -- start start 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc0ac930 0x7f07dc14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07dc1522c0 0x7f07dc0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cdf90 con 0x7f07dc0adcd0 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cde10 con 0x7f07dc1522c0 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07e3a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0ce110 con 0x7f07dc0ac930 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37962/0 (socket says 172.21.15.44:37962) 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.311+0000 7f07dbe00640 1 -- 172.21.15.44:0/2209133367 learned_addr learned my addr 172.21.15.44:0/2209133367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07e1000640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc0ac930 0x7f07dc14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07e1a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07dc1522c0 0x7f07dc0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 -- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc0ac930 msgr2=0x7f07dc14bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc0ac930 0x7f07dc14bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 -- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07dc1522c0 msgr2=0x7f07dc0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07dc1522c0 0x7f07dc0b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 -- 172.21.15.44:0/2209133367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0b2750 con 0x7f07dc0adcd0 2024-07-17T06:55:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.312+0000 7f07dbe00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c420 tx=0x7f07c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.313+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4017070 con 0x7f07dc0adcd0 2024-07-17T06:55:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.313+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c40053c0 con 0x7f07dc0adcd0 2024-07-17T06:55:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.313+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4003850 con 0x7f07dc0adcd0 2024-07-17T06:55:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.313+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc0b2a30 con 0x7f07dc0adcd0 2024-07-17T06:55:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.313+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc0b2ff0 con 0x7f07dc0adcd0 2024-07-17T06:55:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.314+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc0ce660 con 0x7f07dc0adcd0 2024-07-17T06:55:16.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.315+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f07c401a070 con 0x7f07dc0adcd0 2024-07-17T06:55:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.316+0000 7f07d8c00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 0x7f07b80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.316+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f07c4097a30 con 0x7f07dc0adcd0 2024-07-17T06:55:16.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.317+0000 7f07e1000640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 0x7f07b80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.318+0000 7f07e1000640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 0x7f07b80790a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07cc033000 tx=0x7f07cc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:16.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.320+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f07c4060bb0 con 0x7f07dc0adcd0 2024-07-17T06:55:16.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.527+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07dc0ac470 con 0x7f07b8076be0 2024-07-17T06:55:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.543+0000 7f07d8c00640 1 -- 172.21.15.44:0/2209133367 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332526 (secure 0 0 0) 0x7f07dc0ac470 con 0x7f07b8076be0 2024-07-17T06:55:16.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:16.550 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:55:16.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 msgr2=0x7f07b80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 0x7f07b80790a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07cc033000 tx=0x7f07cc004730 comp rx=0 tx=0).stop 2024-07-17T06:55:16.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 msgr2=0x7f07dc098b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c420 tx=0x7f07c400c8e0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 shutdown_connections 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07dc1522c0 0x7f07dc0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07dc0adcd0 0x7f07dc098b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f07b8076be0 0x7f07b80790a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 --2- 172.21.15.44:0/2209133367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07dc0ac930 0x7f07dc14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 >> 172.21.15.44:0/2209133367 conn(0x7f07dc0b0fb0 msgr2=0x7f07dc0ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 shutdown_connections 2024-07-17T06:55:16.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.551+0000 7f07e3a00640 1 -- 172.21.15.44:0/2209133367 wait complete. 2024-07-17T06:55:16.571 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-07-17T06:55:15.720992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":154,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":3317760,"data_stored":2004920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006374"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569050+0000","last_change":"2024-07-17T06:54:48.289026+0000","last_active":"2024-07-17T06:54:54.569050+0000","last_peered":"2024-07-17T06:54:54.569050+0000","last_clean":"2024-07-17T06:54:54.569050+0000","last_became_active":"2024-07-17T06:54:48.288767+0000","last_became_peered":"2024-07-17T06:54:48.288767+0000","last_unstale":"2024-07-17T06:54:54.569050+0000","last_undegraded":"2024-07-17T06:54:54.569050+0000","last_fullsized":"2024-07-17T06:54:54.569050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572074+0000","last_change":"2024-07-17T06:54:49.320600+0000","last_active":"2024-07-17T06:54:54.572074+0000","last_peered":"2024-07-17T06:54:54.572074+0000","last_clean":"2024-07-17T06:54:54.572074+0000","last_became_active":"2024-07-17T06:54:49.319878+0000","last_became_peered":"2024-07-17T06:54:49.319878+0000","last_unstale":"2024-07-17T06:54:54.572074+0000","last_undegraded":"2024-07-17T06:54:54.572074+0000","last_fullsized":"2024-07-17T06:54:54.572074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569764+0000","last_change":"2024-07-17T06:54:53.670763+0000","last_active":"2024-07-17T06:54:54.569764+0000","last_peered":"2024-07-17T06:54:54.569764+0000","last_clean":"2024-07-17T06:54:54.569764+0000","last_became_active":"2024-07-17T06:54:53.670345+0000","last_became_peered":"2024-07-17T06:54:53.670345+0000","last_unstale":"2024-07-17T06:54:54.569764+0000","last_undegraded":"2024-07-17T06:54:54.569764+0000","last_fullsized":"2024-07-17T06:54:54.569764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:19.165767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572200+0000","last_change":"2024-07-17T06:54:48.267830+0000","last_active":"2024-07-17T06:54:54.572200+0000","last_peered":"2024-07-17T06:54:54.572200+0000","last_clean":"2024-07-17T06:54:54.572200+0000","last_became_active":"2024-07-17T06:54:48.267458+0000","last_became_peered":"2024-07-17T06:54:48.267458+0000","last_unstale":"2024-07-17T06:54:54.572200+0000","last_undegraded":"2024-07-17T06:54:54.572200+0000","last_fullsized":"2024-07-17T06:54:54.572200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:15.343691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566732+0000","last_change":"2024-07-17T06:54:49.313799+0000","last_active":"2024-07-17T06:54:54.566732+0000","last_peered":"2024-07-17T06:54:54.566732+0000","last_clean":"2024-07-17T06:54:54.566732+0000","last_became_active":"2024-07-17T06:54:49.313240+0000","last_became_peered":"2024-07-17T06:54:49.313240+0000","last_unstale":"2024-07-17T06:54:54.566732+0000","last_undegraded":"2024-07-17T06:54:54.566732+0000","last_fullsized":"2024-07-17T06:54:54.566732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340034+0000","last_change":"2024-07-17T06:54:53.356992+0000","last_active":"2024-07-17T06:54:54.340034+0000","last_peered":"2024-07-17T06:54:54.340034+0000","last_clean":"2024-07-17T06:54:54.340034+0000","last_became_active":"2024-07-17T06:54:53.356521+0000","last_became_peered":"2024-07-17T06:54:53.356521+0000","last_unstale":"2024-07-17T06:54:54.340034+0000","last_undegraded":"2024-07-17T06:54:54.340034+0000","last_fullsized":"2024-07-17T06:54:54.340034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:33.058963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569075+0000","last_change":"2024-07-17T06:54:48.285421+0000","last_active":"2024-07-17T06:54:54.569075+0000","last_peered":"2024-07-17T06:54:54.569075+0000","last_clean":"2024-07-17T06:54:54.569075+0000","last_became_active":"2024-07-17T06:54:48.285132+0000","last_became_peered":"2024-07-17T06:54:48.285132+0000","last_unstale":"2024-07-17T06:54:54.569075+0000","last_undegraded":"2024-07-17T06:54:54.569075+0000","last_fullsized":"2024-07-17T06:54:54.569075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:38.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566825+0000","last_change":"2024-07-17T06:54:49.311977+0000","last_active":"2024-07-17T06:54:54.566825+0000","last_peered":"2024-07-17T06:54:54.566825+0000","last_clean":"2024-07-17T06:54:54.566825+0000","last_became_active":"2024-07-17T06:54:49.311364+0000","last_became_peered":"2024-07-17T06:54:49.311364+0000","last_unstale":"2024-07-17T06:54:54.566825+0000","last_undegraded":"2024-07-17T06:54:54.566825+0000","last_fullsized":"2024-07-17T06:54:54.566825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:21.034101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341182+0000","last_change":"2024-07-17T06:54:53.362264+0000","last_active":"2024-07-17T06:54:54.341182+0000","last_peered":"2024-07-17T06:54:54.341182+0000","last_clean":"2024-07-17T06:54:54.341182+0000","last_became_active":"2024-07-17T06:54:53.361863+0000","last_became_peered":"2024-07-17T06:54:53.361863+0000","last_unstale":"2024-07-17T06:54:54.341182+0000","last_undegraded":"2024-07-17T06:54:54.341182+0000","last_fullsized":"2024-07-17T06:54:54.341182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:22.498997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567855+0000","last_change":"2024-07-17T06:54:48.288847+0000","last_active":"2024-07-17T06:54:54.567855+0000","last_peered":"2024-07-17T06:54:54.567855+0000","last_clean":"2024-07-17T06:54:54.567855+0000","last_became_active":"2024-07-17T06:54:48.288504+0000","last_became_peered":"2024-07-17T06:54:48.288504+0000","last_unstale":"2024-07-17T06:54:54.567855+0000","last_undegraded":"2024-07-17T06:54:54.567855+0000","last_fullsized":"2024-07-17T06:54:54.567855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:42.640200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567324+0000","last_change":"2024-07-17T06:54:49.311645+0000","last_active":"2024-07-17T06:54:54.567324+0000","last_peered":"2024-07-17T06:54:54.567324+0000","last_clean":"2024-07-17T06:54:54.567324+0000","last_became_active":"2024-07-17T06:54:49.311336+0000","last_became_peered":"2024-07-17T06:54:49.311336+0000","last_unstale":"2024-07-17T06:54:54.567324+0000","last_undegraded":"2024-07-17T06:54:54.567324+0000","last_fullsized":"2024-07-17T06:54:54.567324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:15.496069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342111+0000","last_change":"2024-07-17T06:54:53.357717+0000","last_active":"2024-07-17T06:54:54.342111+0000","last_peered":"2024-07-17T06:54:54.342111+0000","last_clean":"2024-07-17T06:54:54.342111+0000","last_became_active":"2024-07-17T06:54:53.356799+0000","last_became_peered":"2024-07-17T06:54:53.356799+0000","last_unstale":"2024-07-17T06:54:54.342111+0000","last_undegraded":"2024-07-17T06:54:54.342111+0000","last_fullsized":"2024-07-17T06:54:54.342111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:29.584528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341883+0000","last_change":"2024-07-17T06:54:49.299990+0000","last_active":"2024-07-17T06:54:54.341883+0000","last_peered":"2024-07-17T06:54:54.341883+0000","last_clean":"2024-07-17T06:54:54.341883+0000","last_became_active":"2024-07-17T06:54:49.299679+0000","last_became_peered":"2024-07-17T06:54:49.299679+0000","last_unstale":"2024-07-17T06:54:54.341883+0000","last_undegraded":"2024-07-17T06:54:54.341883+0000","last_fullsized":"2024-07-17T06:54:54.341883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568387+0000","last_change":"2024-07-17T06:54:48.285371+0000","last_active":"2024-07-17T06:54:54.568387+0000","last_peered":"2024-07-17T06:54:54.568387+0000","last_clean":"2024-07-17T06:54:54.568387+0000","last_became_active":"2024-07-17T06:54:48.284925+0000","last_became_peered":"2024-07-17T06:54:48.284925+0000","last_unstale":"2024-07-17T06:54:54.568387+0000","last_undegraded":"2024-07-17T06:54:54.568387+0000","last_fullsized":"2024-07-17T06:54:54.568387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:21.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344985+0000","last_change":"2024-07-17T06:54:53.353487+0000","last_active":"2024-07-17T06:54:54.344985+0000","last_peered":"2024-07-17T06:54:54.344985+0000","last_clean":"2024-07-17T06:54:54.344985+0000","last_became_active":"2024-07-17T06:54:53.353037+0000","last_became_peered":"2024-07-17T06:54:53.353037+0000","last_unstale":"2024-07-17T06:54:54.344985+0000","last_undegraded":"2024-07-17T06:54:54.344985+0000","last_fullsized":"2024-07-17T06:54:54.344985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:51.743135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570785+0000","last_change":"2024-07-17T06:54:49.300898+0000","last_active":"2024-07-17T06:54:54.570785+0000","last_peered":"2024-07-17T06:54:54.570785+0000","last_clean":"2024-07-17T06:54:54.570785+0000","last_became_active":"2024-07-17T06:54:49.300631+0000","last_became_peered":"2024-07-17T06:54:49.300631+0000","last_unstale":"2024-07-17T06:54:54.570785+0000","last_undegraded":"2024-07-17T06:54:54.570785+0000","last_fullsized":"2024-07-17T06:54:54.570785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568669+0000","last_change":"2024-07-17T06:54:48.274040+0000","last_active":"2024-07-17T06:54:54.568669+0000","last_peered":"2024-07-17T06:54:54.568669+0000","last_clean":"2024-07-17T06:54:54.568669+0000","last_became_active":"2024-07-17T06:54:48.272837+0000","last_became_peered":"2024-07-17T06:54:48.272837+0000","last_unstale":"2024-07-17T06:54:54.568669+0000","last_undegraded":"2024-07-17T06:54:54.568669+0000","last_fullsized":"2024-07-17T06:54:54.568669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344238+0000","last_change":"2024-07-17T06:54:53.356025+0000","last_active":"2024-07-17T06:54:54.344238+0000","last_peered":"2024-07-17T06:54:54.344238+0000","last_clean":"2024-07-17T06:54:54.344238+0000","last_became_active":"2024-07-17T06:54:53.355539+0000","last_became_peered":"2024-07-17T06:54:53.355539+0000","last_unstale":"2024-07-17T06:54:54.344238+0000","last_undegraded":"2024-07-17T06:54:54.344238+0000","last_fullsized":"2024-07-17T06:54:54.344238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:14.244752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567230+0000","last_change":"2024-07-17T06:54:49.303989+0000","last_active":"2024-07-17T06:54:54.567230+0000","last_peered":"2024-07-17T06:54:54.567230+0000","last_clean":"2024-07-17T06:54:54.567230+0000","last_became_active":"2024-07-17T06:54:49.303723+0000","last_became_peered":"2024-07-17T06:54:49.303723+0000","last_unstale":"2024-07-17T06:54:54.567230+0000","last_undegraded":"2024-07-17T06:54:54.567230+0000","last_fullsized":"2024-07-17T06:54:54.567230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568506+0000","last_change":"2024-07-17T06:54:48.280459+0000","last_active":"2024-07-17T06:54:54.568506+0000","last_peered":"2024-07-17T06:54:54.568506+0000","last_clean":"2024-07-17T06:54:54.568506+0000","last_became_active":"2024-07-17T06:54:48.280196+0000","last_became_peered":"2024-07-17T06:54:48.280196+0000","last_unstale":"2024-07-17T06:54:54.568506+0000","last_undegraded":"2024-07-17T06:54:54.568506+0000","last_fullsized":"2024-07-17T06:54:54.568506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571786+0000","last_change":"2024-07-17T06:54:53.672128+0000","last_active":"2024-07-17T06:54:54.571786+0000","last_peered":"2024-07-17T06:54:54.571786+0000","last_clean":"2024-07-17T06:54:54.571786+0000","last_became_active":"2024-07-17T06:54:53.671083+0000","last_became_peered":"2024-07-17T06:54:53.671083+0000","last_unstale":"2024-07-17T06:54:54.571786+0000","last_undegraded":"2024-07-17T06:54:54.571786+0000","last_fullsized":"2024-07-17T06:54:54.571786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341192+0000","last_change":"2024-07-17T06:54:49.301897+0000","last_active":"2024-07-17T06:54:54.341192+0000","last_peered":"2024-07-17T06:54:54.341192+0000","last_clean":"2024-07-17T06:54:54.341192+0000","last_became_active":"2024-07-17T06:54:49.301602+0000","last_became_peered":"2024-07-17T06:54:49.301602+0000","last_unstale":"2024-07-17T06:54:54.341192+0000","last_undegraded":"2024-07-17T06:54:54.341192+0000","last_fullsized":"2024-07-17T06:54:54.341192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:39.268743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568075+0000","last_change":"2024-07-17T06:54:48.280905+0000","last_active":"2024-07-17T06:54:54.568075+0000","last_peered":"2024-07-17T06:54:54.568075+0000","last_clean":"2024-07-17T06:54:54.568075+0000","last_became_active":"2024-07-17T06:54:48.280538+0000","last_became_peered":"2024-07-17T06:54:48.280538+0000","last_unstale":"2024-07-17T06:54:54.568075+0000","last_undegraded":"2024-07-17T06:54:54.568075+0000","last_fullsized":"2024-07-17T06:54:54.568075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343510+0000","last_change":"2024-07-17T06:54:53.356455+0000","last_active":"2024-07-17T06:54:54.343510+0000","last_peered":"2024-07-17T06:54:54.343510+0000","last_clean":"2024-07-17T06:54:54.343510+0000","last_became_active":"2024-07-17T06:54:53.355594+0000","last_became_peered":"2024-07-17T06:54:53.355594+0000","last_unstale":"2024-07-17T06:54:54.343510+0000","last_undegraded":"2024-07-17T06:54:54.343510+0000","last_fullsized":"2024-07-17T06:54:54.343510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:37.799189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569678+0000","last_change":"2024-07-17T06:54:49.306398+0000","last_active":"2024-07-17T06:54:54.569678+0000","last_peered":"2024-07-17T06:54:54.569678+0000","last_clean":"2024-07-17T06:54:54.569678+0000","last_became_active":"2024-07-17T06:54:49.305914+0000","last_became_peered":"2024-07-17T06:54:49.305914+0000","last_unstale":"2024-07-17T06:54:54.569678+0000","last_undegraded":"2024-07-17T06:54:54.569678+0000","last_fullsized":"2024-07-17T06:54:54.569678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343534+0000","last_change":"2024-07-17T06:54:48.281630+0000","last_active":"2024-07-17T06:54:54.343534+0000","last_peered":"2024-07-17T06:54:54.343534+0000","last_clean":"2024-07-17T06:54:54.343534+0000","last_became_active":"2024-07-17T06:54:48.281368+0000","last_became_peered":"2024-07-17T06:54:48.281368+0000","last_unstale":"2024-07-17T06:54:54.343534+0000","last_undegraded":"2024-07-17T06:54:54.343534+0000","last_fullsized":"2024-07-17T06:54:54.343534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:09.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340594+0000","last_change":"2024-07-17T06:54:53.685324+0000","last_active":"2024-07-17T06:54:54.340594+0000","last_peered":"2024-07-17T06:54:54.340594+0000","last_clean":"2024-07-17T06:54:54.340594+0000","last_became_active":"2024-07-17T06:54:53.683906+0000","last_became_peered":"2024-07-17T06:54:53.683906+0000","last_unstale":"2024-07-17T06:54:54.340594+0000","last_undegraded":"2024-07-17T06:54:54.340594+0000","last_fullsized":"2024-07-17T06:54:54.340594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:31.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572498+0000","last_change":"2024-07-17T06:54:49.307258+0000","last_active":"2024-07-17T06:54:54.572498+0000","last_peered":"2024-07-17T06:54:54.572498+0000","last_clean":"2024-07-17T06:54:54.572498+0000","last_became_active":"2024-07-17T06:54:49.306989+0000","last_became_peered":"2024-07-17T06:54:49.306989+0000","last_unstale":"2024-07-17T06:54:54.572498+0000","last_undegraded":"2024-07-17T06:54:54.572498+0000","last_fullsized":"2024-07-17T06:54:54.572498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:03.300938+0000","last_change":"2024-07-17T06:54:48.273115+0000","last_active":"2024-07-17T06:55:03.300938+0000","last_peered":"2024-07-17T06:55:03.300938+0000","last_clean":"2024-07-17T06:55:03.300938+0000","last_became_active":"2024-07-17T06:54:48.272804+0000","last_became_peered":"2024-07-17T06:54:48.272804+0000","last_unstale":"2024-07-17T06:55:03.300938+0000","last_undegraded":"2024-07-17T06:55:03.300938+0000","last_fullsized":"2024-07-17T06:55:03.300938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566623+0000","last_change":"2024-07-17T06:54:53.667515+0000","last_active":"2024-07-17T06:54:54.566623+0000","last_peered":"2024-07-17T06:54:54.566623+0000","last_clean":"2024-07-17T06:54:54.566623+0000","last_became_active":"2024-07-17T06:54:53.665920+0000","last_became_peered":"2024-07-17T06:54:53.665920+0000","last_unstale":"2024-07-17T06:54:54.566623+0000","last_undegraded":"2024-07-17T06:54:54.566623+0000","last_fullsized":"2024-07-17T06:54:54.566623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:20.348618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569843+0000","last_change":"2024-07-17T06:54:49.296691+0000","last_active":"2024-07-17T06:54:54.569843+0000","last_peered":"2024-07-17T06:54:54.569843+0000","last_clean":"2024-07-17T06:54:54.569843+0000","last_became_active":"2024-07-17T06:54:49.296340+0000","last_became_peered":"2024-07-17T06:54:49.296340+0000","last_unstale":"2024-07-17T06:54:54.569843+0000","last_undegraded":"2024-07-17T06:54:54.569843+0000","last_fullsized":"2024-07-17T06:54:54.569843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569522+0000","last_change":"2024-07-17T06:54:48.288182+0000","last_active":"2024-07-17T06:54:54.569522+0000","last_peered":"2024-07-17T06:54:54.569522+0000","last_clean":"2024-07-17T06:54:54.569522+0000","last_became_active":"2024-07-17T06:54:48.287723+0000","last_became_peered":"2024-07-17T06:54:48.287723+0000","last_unstale":"2024-07-17T06:54:54.569522+0000","last_undegraded":"2024-07-17T06:54:54.569522+0000","last_fullsized":"2024-07-17T06:54:54.569522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:54.905435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565754+0000","last_change":"2024-07-17T06:54:53.672017+0000","last_active":"2024-07-17T06:54:54.565754+0000","last_peered":"2024-07-17T06:54:54.565754+0000","last_clean":"2024-07-17T06:54:54.565754+0000","last_became_active":"2024-07-17T06:54:53.671036+0000","last_became_peered":"2024-07-17T06:54:53.671036+0000","last_unstale":"2024-07-17T06:54:54.565754+0000","last_undegraded":"2024-07-17T06:54:54.565754+0000","last_fullsized":"2024-07-17T06:54:54.565754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567594+0000","last_change":"2024-07-17T06:54:49.320403+0000","last_active":"2024-07-17T06:54:54.567594+0000","last_peered":"2024-07-17T06:54:54.567594+0000","last_clean":"2024-07-17T06:54:54.567594+0000","last_became_active":"2024-07-17T06:54:49.320119+0000","last_became_peered":"2024-07-17T06:54:49.320119+0000","last_unstale":"2024-07-17T06:54:54.567594+0000","last_undegraded":"2024-07-17T06:54:54.567594+0000","last_fullsized":"2024-07-17T06:54:54.567594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:41.108822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569854+0000","last_change":"2024-07-17T06:54:48.271506+0000","last_active":"2024-07-17T06:54:54.569854+0000","last_peered":"2024-07-17T06:54:54.569854+0000","last_clean":"2024-07-17T06:54:54.569854+0000","last_became_active":"2024-07-17T06:54:48.271181+0000","last_became_peered":"2024-07-17T06:54:48.271181+0000","last_unstale":"2024-07-17T06:54:54.569854+0000","last_undegraded":"2024-07-17T06:54:54.569854+0000","last_fullsized":"2024-07-17T06:54:54.569854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568214+0000","last_change":"2024-07-17T06:54:53.358600+0000","last_active":"2024-07-17T06:54:54.568214+0000","last_peered":"2024-07-17T06:54:54.568214+0000","last_clean":"2024-07-17T06:54:54.568214+0000","last_became_active":"2024-07-17T06:54:53.355359+0000","last_became_peered":"2024-07-17T06:54:53.355359+0000","last_unstale":"2024-07-17T06:54:54.568214+0000","last_undegraded":"2024-07-17T06:54:54.568214+0000","last_fullsized":"2024-07-17T06:54:54.568214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344036+0000","last_change":"2024-07-17T06:54:49.305202+0000","last_active":"2024-07-17T06:54:54.344036+0000","last_peered":"2024-07-17T06:54:54.344036+0000","last_clean":"2024-07-17T06:54:54.344036+0000","last_became_active":"2024-07-17T06:54:49.304856+0000","last_became_peered":"2024-07-17T06:54:49.304856+0000","last_unstale":"2024-07-17T06:54:54.344036+0000","last_undegraded":"2024-07-17T06:54:54.344036+0000","last_fullsized":"2024-07-17T06:54:54.344036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:00.298763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566783+0000","last_change":"2024-07-17T06:54:48.279231+0000","last_active":"2024-07-17T06:54:54.566783+0000","last_peered":"2024-07-17T06:54:54.566783+0000","last_clean":"2024-07-17T06:54:54.566783+0000","last_became_active":"2024-07-17T06:54:48.277422+0000","last_became_peered":"2024-07-17T06:54:48.277422+0000","last_unstale":"2024-07-17T06:54:54.566783+0000","last_undegraded":"2024-07-17T06:54:54.566783+0000","last_fullsized":"2024-07-17T06:54:54.566783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569758+0000","last_change":"2024-07-17T06:54:53.350972+0000","last_active":"2024-07-17T06:54:54.569758+0000","last_peered":"2024-07-17T06:54:54.569758+0000","last_clean":"2024-07-17T06:54:54.569758+0000","last_became_active":"2024-07-17T06:54:53.347571+0000","last_became_peered":"2024-07-17T06:54:53.347571+0000","last_unstale":"2024-07-17T06:54:54.569758+0000","last_undegraded":"2024-07-17T06:54:54.569758+0000","last_fullsized":"2024-07-17T06:54:54.569758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567608+0000","last_change":"2024-07-17T06:54:49.302823+0000","last_active":"2024-07-17T06:54:54.567608+0000","last_peered":"2024-07-17T06:54:54.567608+0000","last_clean":"2024-07-17T06:54:54.567608+0000","last_became_active":"2024-07-17T06:54:49.302434+0000","last_became_peered":"2024-07-17T06:54:49.302434+0000","last_unstale":"2024-07-17T06:54:54.567608+0000","last_undegraded":"2024-07-17T06:54:54.567608+0000","last_fullsized":"2024-07-17T06:54:54.567608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339350+0000","last_change":"2024-07-17T06:54:48.268185+0000","last_active":"2024-07-17T06:54:54.339350+0000","last_peered":"2024-07-17T06:54:54.339350+0000","last_clean":"2024-07-17T06:54:54.339350+0000","last_became_active":"2024-07-17T06:54:48.267807+0000","last_became_peered":"2024-07-17T06:54:48.267807+0000","last_unstale":"2024-07-17T06:54:54.339350+0000","last_undegraded":"2024-07-17T06:54:54.339350+0000","last_fullsized":"2024-07-17T06:54:54.339350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:28.391194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572146+0000","last_change":"2024-07-17T06:54:53.345628+0000","last_active":"2024-07-17T06:54:54.572146+0000","last_peered":"2024-07-17T06:54:54.572146+0000","last_clean":"2024-07-17T06:54:54.572146+0000","last_became_active":"2024-07-17T06:54:53.345269+0000","last_became_peered":"2024-07-17T06:54:53.345269+0000","last_unstale":"2024-07-17T06:54:54.572146+0000","last_undegraded":"2024-07-17T06:54:54.572146+0000","last_fullsized":"2024-07-17T06:54:54.572146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565847+0000","last_change":"2024-07-17T06:54:49.302929+0000","last_active":"2024-07-17T06:54:54.565847+0000","last_peered":"2024-07-17T06:54:54.565847+0000","last_clean":"2024-07-17T06:54:54.565847+0000","last_became_active":"2024-07-17T06:54:49.302598+0000","last_became_peered":"2024-07-17T06:54:49.302598+0000","last_unstale":"2024-07-17T06:54:54.565847+0000","last_undegraded":"2024-07-17T06:54:54.565847+0000","last_fullsized":"2024-07-17T06:54:54.565847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568955+0000","last_change":"2024-07-17T06:54:48.284164+0000","last_active":"2024-07-17T06:54:54.568955+0000","last_peered":"2024-07-17T06:54:54.568955+0000","last_clean":"2024-07-17T06:54:54.568955+0000","last_became_active":"2024-07-17T06:54:48.282885+0000","last_became_peered":"2024-07-17T06:54:48.282885+0000","last_unstale":"2024-07-17T06:54:54.568955+0000","last_undegraded":"2024-07-17T06:54:54.568955+0000","last_fullsized":"2024-07-17T06:54:54.568955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:05.674177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569179+0000","last_change":"2024-07-17T06:54:53.677311+0000","last_active":"2024-07-17T06:54:54.569179+0000","last_peered":"2024-07-17T06:54:54.569179+0000","last_clean":"2024-07-17T06:54:54.569179+0000","last_became_active":"2024-07-17T06:54:53.670958+0000","last_became_peered":"2024-07-17T06:54:53.670958+0000","last_unstale":"2024-07-17T06:54:54.569179+0000","last_undegraded":"2024-07-17T06:54:54.569179+0000","last_fullsized":"2024-07-17T06:54:54.569179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:09.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566963+0000","last_change":"2024-07-17T06:54:49.312042+0000","last_active":"2024-07-17T06:54:54.566963+0000","last_peered":"2024-07-17T06:54:54.566963+0000","last_clean":"2024-07-17T06:54:54.566963+0000","last_became_active":"2024-07-17T06:54:49.311741+0000","last_became_peered":"2024-07-17T06:54:49.311741+0000","last_unstale":"2024-07-17T06:54:54.566963+0000","last_undegraded":"2024-07-17T06:54:54.566963+0000","last_fullsized":"2024-07-17T06:54:54.566963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567568+0000","last_change":"2024-07-17T06:54:48.277447+0000","last_active":"2024-07-17T06:54:54.567568+0000","last_peered":"2024-07-17T06:54:54.567568+0000","last_clean":"2024-07-17T06:54:54.567568+0000","last_became_active":"2024-07-17T06:54:48.273874+0000","last_became_peered":"2024-07-17T06:54:48.273874+0000","last_unstale":"2024-07-17T06:54:54.567568+0000","last_undegraded":"2024-07-17T06:54:54.567568+0000","last_fullsized":"2024-07-17T06:54:54.567568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":118,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568469+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:54:54.568469+0000","last_peered":"2024-07-17T06:54:54.568469+0000","last_clean":"2024-07-17T06:54:54.568469+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:54:54.568469+0000","last_undegraded":"2024-07-17T06:54:54.568469+0000","last_fullsized":"2024-07-17T06:54:54.568469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340950+0000","last_change":"2024-07-17T06:54:53.680957+0000","last_active":"2024-07-17T06:54:54.340950+0000","last_peered":"2024-07-17T06:54:54.340950+0000","last_clean":"2024-07-17T06:54:54.340950+0000","last_became_active":"2024-07-17T06:54:53.679463+0000","last_became_peered":"2024-07-17T06:54:53.679463+0000","last_unstale":"2024-07-17T06:54:54.340950+0000","last_undegraded":"2024-07-17T06:54:54.340950+0000","last_fullsized":"2024-07-17T06:54:54.340950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:02.888574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568688+0000","last_change":"2024-07-17T06:54:49.313201+0000","last_active":"2024-07-17T06:54:54.568688+0000","last_peered":"2024-07-17T06:54:54.568688+0000","last_clean":"2024-07-17T06:54:54.568688+0000","last_became_active":"2024-07-17T06:54:49.312937+0000","last_became_peered":"2024-07-17T06:54:49.312937+0000","last_unstale":"2024-07-17T06:54:54.568688+0000","last_undegraded":"2024-07-17T06:54:54.568688+0000","last_fullsized":"2024-07-17T06:54:54.568688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:16.869857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339687+0000","last_change":"2024-07-17T06:54:48.276023+0000","last_active":"2024-07-17T06:54:54.339687+0000","last_peered":"2024-07-17T06:54:54.339687+0000","last_clean":"2024-07-17T06:54:54.339687+0000","last_became_active":"2024-07-17T06:54:48.275283+0000","last_became_peered":"2024-07-17T06:54:48.275283+0000","last_unstale":"2024-07-17T06:54:54.339687+0000","last_undegraded":"2024-07-17T06:54:54.339687+0000","last_fullsized":"2024-07-17T06:54:54.339687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:52.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341589+0000","last_change":"2024-07-17T06:54:53.674509+0000","last_active":"2024-07-17T06:54:54.341589+0000","last_peered":"2024-07-17T06:54:54.341589+0000","last_clean":"2024-07-17T06:54:54.341589+0000","last_became_active":"2024-07-17T06:54:53.672806+0000","last_became_peered":"2024-07-17T06:54:53.672806+0000","last_unstale":"2024-07-17T06:54:54.341589+0000","last_undegraded":"2024-07-17T06:54:54.341589+0000","last_fullsized":"2024-07-17T06:54:54.341589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567064+0000","last_change":"2024-07-17T06:54:49.324849+0000","last_active":"2024-07-17T06:54:54.567064+0000","last_peered":"2024-07-17T06:54:54.567064+0000","last_clean":"2024-07-17T06:54:54.567064+0000","last_became_active":"2024-07-17T06:54:49.324195+0000","last_became_peered":"2024-07-17T06:54:49.324195+0000","last_unstale":"2024-07-17T06:54:54.567064+0000","last_undegraded":"2024-07-17T06:54:54.567064+0000","last_fullsized":"2024-07-17T06:54:54.567064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:45.753022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570696+0000","last_change":"2024-07-17T06:54:48.276038+0000","last_active":"2024-07-17T06:54:54.570696+0000","last_peered":"2024-07-17T06:54:54.570696+0000","last_clean":"2024-07-17T06:54:54.570696+0000","last_became_active":"2024-07-17T06:54:48.275729+0000","last_became_peered":"2024-07-17T06:54:48.275729+0000","last_unstale":"2024-07-17T06:54:54.570696+0000","last_undegraded":"2024-07-17T06:54:54.570696+0000","last_fullsized":"2024-07-17T06:54:54.570696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565953+0000","last_change":"2024-07-17T06:54:53.356746+0000","last_active":"2024-07-17T06:54:54.565953+0000","last_peered":"2024-07-17T06:54:54.565953+0000","last_clean":"2024-07-17T06:54:54.565953+0000","last_became_active":"2024-07-17T06:54:53.353358+0000","last_became_peered":"2024-07-17T06:54:53.353358+0000","last_unstale":"2024-07-17T06:54:54.565953+0000","last_undegraded":"2024-07-17T06:54:54.565953+0000","last_fullsized":"2024-07-17T06:54:54.565953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569283+0000","last_change":"2024-07-17T06:54:49.318930+0000","last_active":"2024-07-17T06:54:54.569283+0000","last_peered":"2024-07-17T06:54:54.569283+0000","last_clean":"2024-07-17T06:54:54.569283+0000","last_became_active":"2024-07-17T06:54:49.318606+0000","last_became_peered":"2024-07-17T06:54:49.318606+0000","last_unstale":"2024-07-17T06:54:54.569283+0000","last_undegraded":"2024-07-17T06:54:54.569283+0000","last_fullsized":"2024-07-17T06:54:54.569283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:10:28.608769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339970+0000","last_change":"2024-07-17T06:54:48.269152+0000","last_active":"2024-07-17T06:54:54.339970+0000","last_peered":"2024-07-17T06:54:54.339970+0000","last_clean":"2024-07-17T06:54:54.339970+0000","last_became_active":"2024-07-17T06:54:48.268914+0000","last_became_peered":"2024-07-17T06:54:48.268914+0000","last_unstale":"2024-07-17T06:54:54.339970+0000","last_undegraded":"2024-07-17T06:54:54.339970+0000","last_fullsized":"2024-07-17T06:54:54.339970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:59.283887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565453+0000","last_change":"2024-07-17T06:54:53.672373+0000","last_active":"2024-07-17T06:54:54.565453+0000","last_peered":"2024-07-17T06:54:54.565453+0000","last_clean":"2024-07-17T06:54:54.565453+0000","last_became_active":"2024-07-17T06:54:53.671288+0000","last_became_peered":"2024-07-17T06:54:53.671288+0000","last_unstale":"2024-07-17T06:54:54.565453+0000","last_undegraded":"2024-07-17T06:54:54.565453+0000","last_fullsized":"2024-07-17T06:54:54.565453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568889+0000","last_change":"2024-07-17T06:54:49.320151+0000","last_active":"2024-07-17T06:54:54.568889+0000","last_peered":"2024-07-17T06:54:54.568889+0000","last_clean":"2024-07-17T06:54:54.568889+0000","last_became_active":"2024-07-17T06:54:49.319845+0000","last_became_peered":"2024-07-17T06:54:49.319845+0000","last_unstale":"2024-07-17T06:54:54.568889+0000","last_undegraded":"2024-07-17T06:54:54.568889+0000","last_fullsized":"2024-07-17T06:54:54.568889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:12:25.836163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343920+0000","last_change":"2024-07-17T06:54:48.268964+0000","last_active":"2024-07-17T06:54:54.343920+0000","last_peered":"2024-07-17T06:54:54.343920+0000","last_clean":"2024-07-17T06:54:54.343920+0000","last_became_active":"2024-07-17T06:54:48.268564+0000","last_became_peered":"2024-07-17T06:54:48.268564+0000","last_unstale":"2024-07-17T06:54:54.343920+0000","last_undegraded":"2024-07-17T06:54:54.343920+0000","last_fullsized":"2024-07-17T06:54:54.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:47.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340002+0000","last_change":"2024-07-17T06:54:53.362347+0000","last_active":"2024-07-17T06:54:54.340002+0000","last_peered":"2024-07-17T06:54:54.340002+0000","last_clean":"2024-07-17T06:54:54.340002+0000","last_became_active":"2024-07-17T06:54:53.361896+0000","last_became_peered":"2024-07-17T06:54:53.361896+0000","last_unstale":"2024-07-17T06:54:54.340002+0000","last_undegraded":"2024-07-17T06:54:54.340002+0000","last_fullsized":"2024-07-17T06:54:54.340002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:22.522023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566168+0000","last_change":"2024-07-17T06:54:49.316448+0000","last_active":"2024-07-17T06:54:54.566168+0000","last_peered":"2024-07-17T06:54:54.566168+0000","last_clean":"2024-07-17T06:54:54.566168+0000","last_became_active":"2024-07-17T06:54:49.316042+0000","last_became_peered":"2024-07-17T06:54:49.316042+0000","last_unstale":"2024-07-17T06:54:54.566168+0000","last_undegraded":"2024-07-17T06:54:54.566168+0000","last_fullsized":"2024-07-17T06:54:54.566168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:33.731836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.338832+0000","last_change":"2024-07-17T06:54:48.270227+0000","last_active":"2024-07-17T06:54:54.338832+0000","last_peered":"2024-07-17T06:54:54.338832+0000","last_clean":"2024-07-17T06:54:54.338832+0000","last_became_active":"2024-07-17T06:54:48.269954+0000","last_became_peered":"2024-07-17T06:54:48.269954+0000","last_unstale":"2024-07-17T06:54:54.338832+0000","last_undegraded":"2024-07-17T06:54:54.338832+0000","last_fullsized":"2024-07-17T06:54:54.338832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.390020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568918+0000","last_change":"2024-07-17T06:54:53.356983+0000","last_active":"2024-07-17T06:54:54.568918+0000","last_peered":"2024-07-17T06:54:54.568918+0000","last_clean":"2024-07-17T06:54:54.568918+0000","last_became_active":"2024-07-17T06:54:53.356288+0000","last_became_peered":"2024-07-17T06:54:53.356288+0000","last_unstale":"2024-07-17T06:54:54.568918+0000","last_undegraded":"2024-07-17T06:54:54.568918+0000","last_fullsized":"2024-07-17T06:54:54.568918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339405+0000","last_change":"2024-07-17T06:54:49.294433+0000","last_active":"2024-07-17T06:54:54.339405+0000","last_peered":"2024-07-17T06:54:54.339405+0000","last_clean":"2024-07-17T06:54:54.339405+0000","last_became_active":"2024-07-17T06:54:49.293644+0000","last_became_peered":"2024-07-17T06:54:49.293644+0000","last_unstale":"2024-07-17T06:54:54.339405+0000","last_undegraded":"2024-07-17T06:54:54.339405+0000","last_fullsized":"2024-07-17T06:54:54.339405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340443+0000","last_change":"2024-07-17T06:54:48.267002+0000","last_active":"2024-07-17T06:54:54.340443+0000","last_peered":"2024-07-17T06:54:54.340443+0000","last_clean":"2024-07-17T06:54:54.340443+0000","last_became_active":"2024-07-17T06:54:48.266711+0000","last_became_peered":"2024-07-17T06:54:48.266711+0000","last_unstale":"2024-07-17T06:54:54.340443+0000","last_undegraded":"2024-07-17T06:54:54.340443+0000","last_fullsized":"2024-07-17T06:54:54.340443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572427+0000","last_change":"2024-07-17T06:54:53.678483+0000","last_active":"2024-07-17T06:54:54.572427+0000","last_peered":"2024-07-17T06:54:54.572427+0000","last_clean":"2024-07-17T06:54:54.572427+0000","last_became_active":"2024-07-17T06:54:53.677707+0000","last_became_peered":"2024-07-17T06:54:53.677707+0000","last_unstale":"2024-07-17T06:54:54.572427+0000","last_undegraded":"2024-07-17T06:54:54.572427+0000","last_fullsized":"2024-07-17T06:54:54.572427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:21.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568071+0000","last_change":"2024-07-17T06:54:49.319059+0000","last_active":"2024-07-17T06:54:54.568071+0000","last_peered":"2024-07-17T06:54:54.568071+0000","last_clean":"2024-07-17T06:54:54.568071+0000","last_became_active":"2024-07-17T06:54:49.318532+0000","last_became_peered":"2024-07-17T06:54:49.318532+0000","last_unstale":"2024-07-17T06:54:54.568071+0000","last_undegraded":"2024-07-17T06:54:54.568071+0000","last_fullsized":"2024-07-17T06:54:54.568071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:47.675783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566475+0000","last_change":"2024-07-17T06:54:48.279991+0000","last_active":"2024-07-17T06:54:54.566475+0000","last_peered":"2024-07-17T06:54:54.566475+0000","last_clean":"2024-07-17T06:54:54.566475+0000","last_became_active":"2024-07-17T06:54:48.279691+0000","last_became_peered":"2024-07-17T06:54:48.279691+0000","last_unstale":"2024-07-17T06:54:54.566475+0000","last_undegraded":"2024-07-17T06:54:54.566475+0000","last_fullsized":"2024-07-17T06:54:54.566475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571709+0000","last_change":"2024-07-17T06:54:53.677451+0000","last_active":"2024-07-17T06:54:54.571709+0000","last_peered":"2024-07-17T06:54:54.571709+0000","last_clean":"2024-07-17T06:54:54.571709+0000","last_became_active":"2024-07-17T06:54:53.677080+0000","last_became_peered":"2024-07-17T06:54:53.677080+0000","last_unstale":"2024-07-17T06:54:54.571709+0000","last_undegraded":"2024-07-17T06:54:54.571709+0000","last_fullsized":"2024-07-17T06:54:54.571709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567583+0000","last_change":"2024-07-17T06:54:49.302477+0000","last_active":"2024-07-17T06:54:54.567583+0000","last_peered":"2024-07-17T06:54:54.567583+0000","last_clean":"2024-07-17T06:54:54.567583+0000","last_became_active":"2024-07-17T06:54:49.302168+0000","last_became_peered":"2024-07-17T06:54:49.302168+0000","last_unstale":"2024-07-17T06:54:54.567583+0000","last_undegraded":"2024-07-17T06:54:54.567583+0000","last_fullsized":"2024-07-17T06:54:54.567583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:20.279165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566160+0000","last_change":"2024-07-17T06:54:48.278091+0000","last_active":"2024-07-17T06:54:54.566160+0000","last_peered":"2024-07-17T06:54:54.566160+0000","last_clean":"2024-07-17T06:54:54.566160+0000","last_became_active":"2024-07-17T06:54:48.277769+0000","last_became_peered":"2024-07-17T06:54:48.277769+0000","last_unstale":"2024-07-17T06:54:54.566160+0000","last_undegraded":"2024-07-17T06:54:54.566160+0000","last_fullsized":"2024-07-17T06:54:54.566160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567004+0000","last_change":"2024-07-17T06:54:53.672346+0000","last_active":"2024-07-17T06:54:54.567004+0000","last_peered":"2024-07-17T06:54:54.567004+0000","last_clean":"2024-07-17T06:54:54.567004+0000","last_became_active":"2024-07-17T06:54:53.670973+0000","last_became_peered":"2024-07-17T06:54:53.670973+0000","last_unstale":"2024-07-17T06:54:54.567004+0000","last_undegraded":"2024-07-17T06:54:54.567004+0000","last_fullsized":"2024-07-17T06:54:54.567004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:35.203323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568419+0000","last_change":"2024-07-17T06:54:49.314009+0000","last_active":"2024-07-17T06:54:54.568419+0000","last_peered":"2024-07-17T06:54:54.568419+0000","last_clean":"2024-07-17T06:54:54.568419+0000","last_became_active":"2024-07-17T06:54:49.313238+0000","last_became_peered":"2024-07-17T06:54:49.313238+0000","last_unstale":"2024-07-17T06:54:54.568419+0000","last_undegraded":"2024-07-17T06:54:54.568419+0000","last_fullsized":"2024-07-17T06:54:54.568419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:16.115918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:55:02.724870+0000","last_change":"2024-07-17T06:54:48.274173+0000","last_active":"2024-07-17T06:55:02.724870+0000","last_peered":"2024-07-17T06:55:02.724870+0000","last_clean":"2024-07-17T06:55:02.724870+0000","last_became_active":"2024-07-17T06:54:48.270354+0000","last_became_peered":"2024-07-17T06:54:48.270354+0000","last_unstale":"2024-07-17T06:55:02.724870+0000","last_undegraded":"2024-07-17T06:55:02.724870+0000","last_fullsized":"2024-07-17T06:55:02.724870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:15.529832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568505+0000","last_change":"2024-07-17T06:54:53.347808+0000","last_active":"2024-07-17T06:54:54.568505+0000","last_peered":"2024-07-17T06:54:54.568505+0000","last_clean":"2024-07-17T06:54:54.568505+0000","last_became_active":"2024-07-17T06:54:53.347369+0000","last_became_peered":"2024-07-17T06:54:53.347369+0000","last_unstale":"2024-07-17T06:54:54.568505+0000","last_undegraded":"2024-07-17T06:54:54.568505+0000","last_fullsized":"2024-07-17T06:54:54.568505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:22.563365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344599+0000","last_change":"2024-07-17T06:54:49.308852+0000","last_active":"2024-07-17T06:54:54.344599+0000","last_peered":"2024-07-17T06:54:54.344599+0000","last_clean":"2024-07-17T06:54:54.344599+0000","last_became_active":"2024-07-17T06:54:49.308337+0000","last_became_peered":"2024-07-17T06:54:49.308337+0000","last_unstale":"2024-07-17T06:54:54.344599+0000","last_undegraded":"2024-07-17T06:54:54.344599+0000","last_fullsized":"2024-07-17T06:54:54.344599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:06.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570694+0000","last_change":"2024-07-17T06:54:48.282475+0000","last_active":"2024-07-17T06:54:54.570694+0000","last_peered":"2024-07-17T06:54:54.570694+0000","last_clean":"2024-07-17T06:54:54.570694+0000","last_became_active":"2024-07-17T06:54:48.282039+0000","last_became_peered":"2024-07-17T06:54:48.282039+0000","last_unstale":"2024-07-17T06:54:54.570694+0000","last_undegraded":"2024-07-17T06:54:54.570694+0000","last_fullsized":"2024-07-17T06:54:54.570694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:44.240643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344160+0000","last_change":"2024-07-17T06:54:53.353664+0000","last_active":"2024-07-17T06:54:54.344160+0000","last_peered":"2024-07-17T06:54:54.344160+0000","last_clean":"2024-07-17T06:54:54.344160+0000","last_became_active":"2024-07-17T06:54:53.351779+0000","last_became_peered":"2024-07-17T06:54:53.351779+0000","last_unstale":"2024-07-17T06:54:54.344160+0000","last_undegraded":"2024-07-17T06:54:54.344160+0000","last_fullsized":"2024-07-17T06:54:54.344160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:20.183270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567440+0000","last_change":"2024-07-17T06:54:49.313557+0000","last_active":"2024-07-17T06:54:54.567440+0000","last_peered":"2024-07-17T06:54:54.567440+0000","last_clean":"2024-07-17T06:54:54.567440+0000","last_became_active":"2024-07-17T06:54:49.312875+0000","last_became_peered":"2024-07-17T06:54:49.312875+0000","last_unstale":"2024-07-17T06:54:54.567440+0000","last_undegraded":"2024-07-17T06:54:54.567440+0000","last_fullsized":"2024-07-17T06:54:54.567440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567055+0000","last_change":"2024-07-17T06:54:48.286081+0000","last_active":"2024-07-17T06:54:54.567055+0000","last_peered":"2024-07-17T06:54:54.567055+0000","last_clean":"2024-07-17T06:54:54.567055+0000","last_became_active":"2024-07-17T06:54:48.285327+0000","last_became_peered":"2024-07-17T06:54:48.285327+0000","last_unstale":"2024-07-17T06:54:54.567055+0000","last_undegraded":"2024-07-17T06:54:54.567055+0000","last_fullsized":"2024-07-17T06:54:54.567055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:58.978782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567082+0000","last_change":"2024-07-17T06:54:53.353255+0000","last_active":"2024-07-17T06:54:54.567082+0000","last_peered":"2024-07-17T06:54:54.567082+0000","last_clean":"2024-07-17T06:54:54.567082+0000","last_became_active":"2024-07-17T06:54:53.352945+0000","last_became_peered":"2024-07-17T06:54:53.352945+0000","last_unstale":"2024-07-17T06:54:54.567082+0000","last_undegraded":"2024-07-17T06:54:54.567082+0000","last_fullsized":"2024-07-17T06:54:54.567082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:06.889450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566804+0000","last_change":"2024-07-17T06:54:49.318904+0000","last_active":"2024-07-17T06:54:54.566804+0000","last_peered":"2024-07-17T06:54:54.566804+0000","last_clean":"2024-07-17T06:54:54.566804+0000","last_became_active":"2024-07-17T06:54:49.316753+0000","last_became_peered":"2024-07-17T06:54:49.316753+0000","last_unstale":"2024-07-17T06:54:54.566804+0000","last_undegraded":"2024-07-17T06:54:54.566804+0000","last_fullsized":"2024-07-17T06:54:54.566804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:45.913897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343278+0000","last_change":"2024-07-17T06:54:48.286978+0000","last_active":"2024-07-17T06:54:54.343278+0000","last_peered":"2024-07-17T06:54:54.343278+0000","last_clean":"2024-07-17T06:54:54.343278+0000","last_became_active":"2024-07-17T06:54:48.286613+0000","last_became_peered":"2024-07-17T06:54:48.286613+0000","last_unstale":"2024-07-17T06:54:54.343278+0000","last_undegraded":"2024-07-17T06:54:54.343278+0000","last_fullsized":"2024-07-17T06:54:54.343278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:36.633169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570149+0000","last_change":"2024-07-17T06:54:53.671456+0000","last_active":"2024-07-17T06:54:54.570149+0000","last_peered":"2024-07-17T06:54:54.570149+0000","last_clean":"2024-07-17T06:54:54.570149+0000","last_became_active":"2024-07-17T06:54:53.670354+0000","last_became_peered":"2024-07-17T06:54:53.670354+0000","last_unstale":"2024-07-17T06:54:54.570149+0000","last_undegraded":"2024-07-17T06:54:54.570149+0000","last_fullsized":"2024-07-17T06:54:54.570149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.111420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571489+0000","last_change":"2024-07-17T06:54:53.682714+0000","last_active":"2024-07-17T06:54:54.571489+0000","last_peered":"2024-07-17T06:54:54.571489+0000","last_clean":"2024-07-17T06:54:54.571489+0000","last_became_active":"2024-07-17T06:54:53.681884+0000","last_became_peered":"2024-07-17T06:54:53.681884+0000","last_unstale":"2024-07-17T06:54:54.571489+0000","last_undegraded":"2024-07-17T06:54:54.571489+0000","last_fullsized":"2024-07-17T06:54:54.571489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:50.691076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568224+0000","last_change":"2024-07-17T06:54:49.305906+0000","last_active":"2024-07-17T06:54:54.568224+0000","last_peered":"2024-07-17T06:54:54.568224+0000","last_clean":"2024-07-17T06:54:54.568224+0000","last_became_active":"2024-07-17T06:54:49.305633+0000","last_became_peered":"2024-07-17T06:54:49.305633+0000","last_unstale":"2024-07-17T06:54:54.568224+0000","last_undegraded":"2024-07-17T06:54:54.568224+0000","last_fullsized":"2024-07-17T06:54:54.568224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565775+0000","last_change":"2024-07-17T06:54:48.275763+0000","last_active":"2024-07-17T06:54:54.565775+0000","last_peered":"2024-07-17T06:54:54.565775+0000","last_clean":"2024-07-17T06:54:54.565775+0000","last_became_active":"2024-07-17T06:54:48.275073+0000","last_became_peered":"2024-07-17T06:54:48.275073+0000","last_unstale":"2024-07-17T06:54:54.565775+0000","last_undegraded":"2024-07-17T06:54:54.565775+0000","last_fullsized":"2024-07-17T06:54:54.565775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:15.690651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343084+0000","last_change":"2024-07-17T06:54:53.347306+0000","last_active":"2024-07-17T06:54:54.343084+0000","last_peered":"2024-07-17T06:54:54.343084+0000","last_clean":"2024-07-17T06:54:54.343084+0000","last_became_active":"2024-07-17T06:54:53.346541+0000","last_became_peered":"2024-07-17T06:54:53.346541+0000","last_unstale":"2024-07-17T06:54:54.343084+0000","last_undegraded":"2024-07-17T06:54:54.343084+0000","last_fullsized":"2024-07-17T06:54:54.343084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:49.820374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567129+0000","last_change":"2024-07-17T06:54:49.299691+0000","last_active":"2024-07-17T06:54:54.567129+0000","last_peered":"2024-07-17T06:54:54.567129+0000","last_clean":"2024-07-17T06:54:54.567129+0000","last_became_active":"2024-07-17T06:54:49.299370+0000","last_became_peered":"2024-07-17T06:54:49.299370+0000","last_unstale":"2024-07-17T06:54:54.567129+0000","last_undegraded":"2024-07-17T06:54:54.567129+0000","last_fullsized":"2024-07-17T06:54:54.567129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570373+0000","last_change":"2024-07-17T06:54:48.276504+0000","last_active":"2024-07-17T06:54:54.570373+0000","last_peered":"2024-07-17T06:54:54.570373+0000","last_clean":"2024-07-17T06:54:54.570373+0000","last_became_active":"2024-07-17T06:54:48.276041+0000","last_became_peered":"2024-07-17T06:54:48.276041+0000","last_unstale":"2024-07-17T06:54:54.570373+0000","last_undegraded":"2024-07-17T06:54:54.570373+0000","last_fullsized":"2024-07-17T06:54:54.570373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:54.192194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342215+0000","last_change":"2024-07-17T06:54:48.267874+0000","last_active":"2024-07-17T06:54:54.342215+0000","last_peered":"2024-07-17T06:54:54.342215+0000","last_clean":"2024-07-17T06:54:54.342215+0000","last_became_active":"2024-07-17T06:54:48.267487+0000","last_became_peered":"2024-07-17T06:54:48.267487+0000","last_unstale":"2024-07-17T06:54:54.342215+0000","last_undegraded":"2024-07-17T06:54:54.342215+0000","last_fullsized":"2024-07-17T06:54:54.342215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:00.779635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566178+0000","last_change":"2024-07-17T06:54:49.306648+0000","last_active":"2024-07-17T06:54:54.566178+0000","last_peered":"2024-07-17T06:54:54.566178+0000","last_clean":"2024-07-17T06:54:54.566178+0000","last_became_active":"2024-07-17T06:54:49.305001+0000","last_became_peered":"2024-07-17T06:54:49.305001+0000","last_unstale":"2024-07-17T06:54:54.566178+0000","last_undegraded":"2024-07-17T06:54:54.566178+0000","last_fullsized":"2024-07-17T06:54:54.566178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:51.241330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339673+0000","last_change":"2024-07-17T06:54:53.360122+0000","last_active":"2024-07-17T06:54:54.339673+0000","last_peered":"2024-07-17T06:54:54.339673+0000","last_clean":"2024-07-17T06:54:54.339673+0000","last_became_active":"2024-07-17T06:54:53.356530+0000","last_became_peered":"2024-07-17T06:54:53.356530+0000","last_unstale":"2024-07-17T06:54:54.339673+0000","last_undegraded":"2024-07-17T06:54:54.339673+0000","last_fullsized":"2024-07-17T06:54:54.339673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:41.753064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570947+0000","last_change":"2024-07-17T06:54:48.277409+0000","last_active":"2024-07-17T06:54:54.570947+0000","last_peered":"2024-07-17T06:54:54.570947+0000","last_clean":"2024-07-17T06:54:54.570947+0000","last_became_active":"2024-07-17T06:54:48.277087+0000","last_became_peered":"2024-07-17T06:54:48.277087+0000","last_unstale":"2024-07-17T06:54:54.570947+0000","last_undegraded":"2024-07-17T06:54:54.570947+0000","last_fullsized":"2024-07-17T06:54:54.570947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:47.215960+0000","last_clean_scrub_stamp":"2024-07-17T06:54:47.215960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:56:17.744184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.342397+0000","last_change":"2024-07-17T06:54:49.300524+0000","last_active":"2024-07-17T06:54:54.342397+0000","last_peered":"2024-07-17T06:54:54.342397+0000","last_clean":"2024-07-17T06:54:54.342397+0000","last_became_active":"2024-07-17T06:54:49.300136+0000","last_became_peered":"2024-07-17T06:54:49.300136+0000","last_unstale":"2024-07-17T06:54:54.342397+0000","last_undegraded":"2024-07-17T06:54:54.342397+0000","last_fullsized":"2024-07-17T06:54:54.342397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:29.406070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571827+0000","last_change":"2024-07-17T06:54:53.355705+0000","last_active":"2024-07-17T06:54:54.571827+0000","last_peered":"2024-07-17T06:54:54.571827+0000","last_clean":"2024-07-17T06:54:54.571827+0000","last_became_active":"2024-07-17T06:54:53.355294+0000","last_became_peered":"2024-07-17T06:54:53.355294+0000","last_unstale":"2024-07-17T06:54:54.571827+0000","last_undegraded":"2024-07-17T06:54:54.571827+0000","last_fullsized":"2024-07-17T06:54:54.571827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:52.298189+0000","last_clean_scrub_stamp":"2024-07-17T06:54:52.298189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:32:32.729442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339117+0000","last_change":"2024-07-17T06:54:49.294765+0000","last_active":"2024-07-17T06:54:54.339117+0000","last_peered":"2024-07-17T06:54:54.339117+0000","last_clean":"2024-07-17T06:54:54.339117+0000","last_became_active":"2024-07-17T06:54:49.294095+0000","last_became_peered":"2024-07-17T06:54:49.294095+0000","last_unstale":"2024-07-17T06:54:54.339117+0000","last_undegraded":"2024-07-17T06:54:54.339117+0000","last_fullsized":"2024-07-17T06:54:54.339117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339066+0000","last_change":"2024-07-17T06:54:49.299295+0000","last_active":"2024-07-17T06:54:54.339066+0000","last_peered":"2024-07-17T06:54:54.339066+0000","last_clean":"2024-07-17T06:54:54.339066+0000","last_became_active":"2024-07-17T06:54:49.298385+0000","last_became_peered":"2024-07-17T06:54:49.298385+0000","last_unstale":"2024-07-17T06:54:54.339066+0000","last_undegraded":"2024-07-17T06:54:54.339066+0000","last_fullsized":"2024-07-17T06:54:54.339066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:55.920311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566243+0000","last_change":"2024-07-17T06:54:49.325974+0000","last_active":"2024-07-17T06:54:54.566243+0000","last_peered":"2024-07-17T06:54:54.566243+0000","last_clean":"2024-07-17T06:54:54.566243+0000","last_became_active":"2024-07-17T06:54:49.322413+0000","last_became_peered":"2024-07-17T06:54:49.322413+0000","last_unstale":"2024-07-17T06:54:54.566243+0000","last_undegraded":"2024-07-17T06:54:54.566243+0000","last_fullsized":"2024-07-17T06:54:54.566243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:18.496012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566055+0000","last_change":"2024-07-17T06:54:49.308310+0000","last_active":"2024-07-17T06:54:54.566055+0000","last_peered":"2024-07-17T06:54:54.566055+0000","last_clean":"2024-07-17T06:54:54.566055+0000","last_became_active":"2024-07-17T06:54:49.304819+0000","last_became_peered":"2024-07-17T06:54:49.304819+0000","last_unstale":"2024-07-17T06:54:54.566055+0000","last_undegraded":"2024-07-17T06:54:54.566055+0000","last_fullsized":"2024-07-17T06:54:54.566055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:51.607323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571870+0000","last_change":"2024-07-17T06:54:49.312400+0000","last_active":"2024-07-17T06:54:54.571870+0000","last_peered":"2024-07-17T06:54:54.571870+0000","last_clean":"2024-07-17T06:54:54.571870+0000","last_became_active":"2024-07-17T06:54:49.311958+0000","last_became_peered":"2024-07-17T06:54:49.311958+0000","last_unstale":"2024-07-17T06:54:54.571870+0000","last_undegraded":"2024-07-17T06:54:54.571870+0000","last_fullsized":"2024-07-17T06:54:54.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.321280+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.320912+0000","last_became_peered":"2024-07-17T06:54:49.320912+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341006+0000","last_change":"2024-07-17T06:54:49.297628+0000","last_active":"2024-07-17T06:54:54.341006+0000","last_peered":"2024-07-17T06:54:54.341006+0000","last_clean":"2024-07-17T06:54:54.341006+0000","last_became_active":"2024-07-17T06:54:49.296871+0000","last_became_peered":"2024-07-17T06:54:49.296871+0000","last_unstale":"2024-07-17T06:54:54.341006+0000","last_undegraded":"2024-07-17T06:54:54.341006+0000","last_fullsized":"2024-07-17T06:54:54.341006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.572513+0000","last_change":"2024-07-17T06:54:49.303253+0000","last_active":"2024-07-17T06:54:54.572513+0000","last_peered":"2024-07-17T06:54:54.572513+0000","last_clean":"2024-07-17T06:54:54.572513+0000","last_became_active":"2024-07-17T06:54:49.302956+0000","last_became_peered":"2024-07-17T06:54:49.302956+0000","last_unstale":"2024-07-17T06:54:54.572513+0000","last_undegraded":"2024-07-17T06:54:54.572513+0000","last_fullsized":"2024-07-17T06:54:54.572513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565697+0000","last_change":"2024-07-17T06:54:49.306156+0000","last_active":"2024-07-17T06:54:54.565697+0000","last_peered":"2024-07-17T06:54:54.565697+0000","last_clean":"2024-07-17T06:54:54.565697+0000","last_became_active":"2024-07-17T06:54:49.305858+0000","last_became_peered":"2024-07-17T06:54:49.305858+0000","last_unstale":"2024-07-17T06:54:54.565697+0000","last_undegraded":"2024-07-17T06:54:54.565697+0000","last_fullsized":"2024-07-17T06:54:54.565697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570174+0000","last_change":"2024-07-17T06:54:49.301568+0000","last_active":"2024-07-17T06:54:54.570174+0000","last_peered":"2024-07-17T06:54:54.570174+0000","last_clean":"2024-07-17T06:54:54.570174+0000","last_became_active":"2024-07-17T06:54:49.301244+0000","last_became_peered":"2024-07-17T06:54:49.301244+0000","last_unstale":"2024-07-17T06:54:54.570174+0000","last_undegraded":"2024-07-17T06:54:54.570174+0000","last_fullsized":"2024-07-17T06:54:54.570174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:40:50.724841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567719+0000","last_change":"2024-07-17T06:54:49.316422+0000","last_active":"2024-07-17T06:54:54.567719+0000","last_peered":"2024-07-17T06:54:54.567719+0000","last_clean":"2024-07-17T06:54:54.567719+0000","last_became_active":"2024-07-17T06:54:49.316133+0000","last_became_peered":"2024-07-17T06:54:49.316133+0000","last_unstale":"2024-07-17T06:54:54.567719+0000","last_undegraded":"2024-07-17T06:54:54.567719+0000","last_fullsized":"2024-07-17T06:54:54.567719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565901+0000","last_change":"2024-07-17T06:54:49.305491+0000","last_active":"2024-07-17T06:54:54.565901+0000","last_peered":"2024-07-17T06:54:54.565901+0000","last_clean":"2024-07-17T06:54:54.565901+0000","last_became_active":"2024-07-17T06:54:49.304629+0000","last_became_peered":"2024-07-17T06:54:49.304629+0000","last_unstale":"2024-07-17T06:54:54.565901+0000","last_undegraded":"2024-07-17T06:54:54.565901+0000","last_fullsized":"2024-07-17T06:54:54.565901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.343504+0000","last_change":"2024-07-17T06:54:49.309637+0000","last_active":"2024-07-17T06:54:54.343504+0000","last_peered":"2024-07-17T06:54:54.343504+0000","last_clean":"2024-07-17T06:54:54.343504+0000","last_became_active":"2024-07-17T06:54:49.309243+0000","last_became_peered":"2024-07-17T06:54:49.309243+0000","last_unstale":"2024-07-17T06:54:54.343504+0000","last_undegraded":"2024-07-17T06:54:54.343504+0000","last_fullsized":"2024-07-17T06:54:54.343504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569329+0000","last_change":"2024-07-17T06:54:49.323709+0000","last_active":"2024-07-17T06:54:54.569329+0000","last_peered":"2024-07-17T06:54:54.569329+0000","last_clean":"2024-07-17T06:54:54.569329+0000","last_became_active":"2024-07-17T06:54:49.323324+0000","last_became_peered":"2024-07-17T06:54:49.323324+0000","last_unstale":"2024-07-17T06:54:54.569329+0000","last_undegraded":"2024-07-17T06:54:54.569329+0000","last_fullsized":"2024-07-17T06:54:54.569329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.339810+0000","last_change":"2024-07-17T06:54:49.286063+0000","last_active":"2024-07-17T06:54:54.339810+0000","last_peered":"2024-07-17T06:54:54.339810+0000","last_clean":"2024-07-17T06:54:54.339810+0000","last_became_active":"2024-07-17T06:54:49.283989+0000","last_became_peered":"2024-07-17T06:54:49.283989+0000","last_unstale":"2024-07-17T06:54:54.339810+0000","last_undegraded":"2024-07-17T06:54:54.339810+0000","last_fullsized":"2024-07-17T06:54:54.339810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:43.655076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567362+0000","last_change":"2024-07-17T06:54:49.319544+0000","last_active":"2024-07-17T06:54:54.567362+0000","last_peered":"2024-07-17T06:54:54.567362+0000","last_clean":"2024-07-17T06:54:54.567362+0000","last_became_active":"2024-07-17T06:54:49.319285+0000","last_became_peered":"2024-07-17T06:54:49.319285+0000","last_unstale":"2024-07-17T06:54:54.567362+0000","last_undegraded":"2024-07-17T06:54:54.567362+0000","last_fullsized":"2024-07-17T06:54:54.567362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:58.821222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565429+0000","last_change":"2024-07-17T06:54:49.305371+0000","last_active":"2024-07-17T06:54:54.565429+0000","last_peered":"2024-07-17T06:54:54.565429+0000","last_clean":"2024-07-17T06:54:54.565429+0000","last_became_active":"2024-07-17T06:54:49.305069+0000","last_became_peered":"2024-07-17T06:54:49.305069+0000","last_unstale":"2024-07-17T06:54:54.565429+0000","last_undegraded":"2024-07-17T06:54:54.565429+0000","last_fullsized":"2024-07-17T06:54:54.565429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.568226+0000","last_change":"2024-07-17T06:54:49.314419+0000","last_active":"2024-07-17T06:54:54.568226+0000","last_peered":"2024-07-17T06:54:54.568226+0000","last_clean":"2024-07-17T06:54:54.568226+0000","last_became_active":"2024-07-17T06:54:49.314106+0000","last_became_peered":"2024-07-17T06:54:49.314106+0000","last_unstale":"2024-07-17T06:54:54.568226+0000","last_undegraded":"2024-07-17T06:54:54.568226+0000","last_fullsized":"2024-07-17T06:54:54.568226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566591+0000","last_change":"2024-07-17T06:54:49.297635+0000","last_active":"2024-07-17T06:54:54.566591+0000","last_peered":"2024-07-17T06:54:54.566591+0000","last_clean":"2024-07-17T06:54:54.566591+0000","last_became_active":"2024-07-17T06:54:49.297252+0000","last_became_peered":"2024-07-17T06:54:49.297252+0000","last_unstale":"2024-07-17T06:54:54.566591+0000","last_undegraded":"2024-07-17T06:54:54.566591+0000","last_fullsized":"2024-07-17T06:54:54.566591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:02.636028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.570670+0000","last_change":"2024-07-17T06:54:49.311510+0000","last_active":"2024-07-17T06:54:54.570670+0000","last_peered":"2024-07-17T06:54:54.570670+0000","last_clean":"2024-07-17T06:54:54.570670+0000","last_became_active":"2024-07-17T06:54:49.311208+0000","last_became_peered":"2024-07-17T06:54:49.311208+0000","last_unstale":"2024-07-17T06:54:54.570670+0000","last_undegraded":"2024-07-17T06:54:54.570670+0000","last_fullsized":"2024-07-17T06:54:54.570670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:23.053361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.344453+0000","last_change":"2024-07-17T06:54:49.300592+0000","last_active":"2024-07-17T06:54:54.344453+0000","last_peered":"2024-07-17T06:54:54.344453+0000","last_clean":"2024-07-17T06:54:54.344453+0000","last_became_active":"2024-07-17T06:54:49.300233+0000","last_became_peered":"2024-07-17T06:54:49.300233+0000","last_unstale":"2024-07-17T06:54:54.344453+0000","last_undegraded":"2024-07-17T06:54:54.344453+0000","last_fullsized":"2024-07-17T06:54:54.344453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:53.264056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.341453+0000","last_change":"2024-07-17T06:54:49.304204+0000","last_active":"2024-07-17T06:54:54.341453+0000","last_peered":"2024-07-17T06:54:54.341453+0000","last_clean":"2024-07-17T06:54:54.341453+0000","last_became_active":"2024-07-17T06:54:49.303907+0000","last_became_peered":"2024-07-17T06:54:49.303907+0000","last_unstale":"2024-07-17T06:54:54.341453+0000","last_undegraded":"2024-07-17T06:54:54.341453+0000","last_fullsized":"2024-07-17T06:54:54.341453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:25.894171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.340757+0000","last_change":"2024-07-17T06:54:49.297834+0000","last_active":"2024-07-17T06:54:54.340757+0000","last_peered":"2024-07-17T06:54:54.340757+0000","last_clean":"2024-07-17T06:54:54.340757+0000","last_became_active":"2024-07-17T06:54:49.297296+0000","last_became_peered":"2024-07-17T06:54:49.297296+0000","last_unstale":"2024-07-17T06:54:54.340757+0000","last_undegraded":"2024-07-17T06:54:54.340757+0000","last_fullsized":"2024-07-17T06:54:54.340757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:44:06.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569209+0000","last_change":"2024-07-17T06:54:49.318654+0000","last_active":"2024-07-17T06:54:54.569209+0000","last_peered":"2024-07-17T06:54:54.569209+0000","last_clean":"2024-07-17T06:54:54.569209+0000","last_became_active":"2024-07-17T06:54:49.318380+0000","last_became_peered":"2024-07-17T06:54:49.318380+0000","last_unstale":"2024-07-17T06:54:54.569209+0000","last_undegraded":"2024-07-17T06:54:54.569209+0000","last_fullsized":"2024-07-17T06:54:54.569209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:12.140489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566890+0000","last_change":"2024-07-17T06:54:49.297152+0000","last_active":"2024-07-17T06:54:54.566890+0000","last_peered":"2024-07-17T06:54:54.566890+0000","last_clean":"2024-07-17T06:54:54.566890+0000","last_became_active":"2024-07-17T06:54:49.296830+0000","last_became_peered":"2024-07-17T06:54:49.296830+0000","last_unstale":"2024-07-17T06:54:54.566890+0000","last_undegraded":"2024-07-17T06:54:54.566890+0000","last_fullsized":"2024-07-17T06:54:54.566890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:11.834051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566123+0000","last_change":"2024-07-17T06:54:49.313713+0000","last_active":"2024-07-17T06:54:54.566123+0000","last_peered":"2024-07-17T06:54:54.566123+0000","last_clean":"2024-07-17T06:54:54.566123+0000","last_became_active":"2024-07-17T06:54:49.313419+0000","last_became_peered":"2024-07-17T06:54:49.313419+0000","last_unstale":"2024-07-17T06:54:54.566123+0000","last_undegraded":"2024-07-17T06:54:54.566123+0000","last_fullsized":"2024-07-17T06:54:54.566123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:25.517176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565979+0000","last_change":"2024-07-17T06:54:49.304993+0000","last_active":"2024-07-17T06:54:54.565979+0000","last_peered":"2024-07-17T06:54:54.565979+0000","last_clean":"2024-07-17T06:54:54.565979+0000","last_became_active":"2024-07-17T06:54:49.304475+0000","last_became_peered":"2024-07-17T06:54:49.304475+0000","last_unstale":"2024-07-17T06:54:54.565979+0000","last_undegraded":"2024-07-17T06:54:54.565979+0000","last_fullsized":"2024-07-17T06:54:54.565979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:53.036574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.565951+0000","last_change":"2024-07-17T06:54:49.319946+0000","last_active":"2024-07-17T06:54:54.565951+0000","last_peered":"2024-07-17T06:54:54.565951+0000","last_clean":"2024-07-17T06:54:54.565951+0000","last_became_active":"2024-07-17T06:54:49.319685+0000","last_became_peered":"2024-07-17T06:54:49.319685+0000","last_unstale":"2024-07-17T06:54:54.565951+0000","last_undegraded":"2024-07-17T06:54:54.565951+0000","last_fullsized":"2024-07-17T06:54:54.565951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:10.177399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.566540+0000","last_change":"2024-07-17T06:54:49.324468+0000","last_active":"2024-07-17T06:54:54.566540+0000","last_peered":"2024-07-17T06:54:54.566540+0000","last_clean":"2024-07-17T06:54:54.566540+0000","last_became_active":"2024-07-17T06:54:49.323700+0000","last_became_peered":"2024-07-17T06:54:49.323700+0000","last_unstale":"2024-07-17T06:54:54.566540+0000","last_undegraded":"2024-07-17T06:54:54.566540+0000","last_fullsized":"2024-07-17T06:54:54.566540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:05.797436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.567150+0000","last_change":"2024-07-17T06:54:49.290953+0000","last_active":"2024-07-17T06:54:54.567150+0000","last_peered":"2024-07-17T06:54:54.567150+0000","last_clean":"2024-07-17T06:54:54.567150+0000","last_became_active":"2024-07-17T06:54:49.290069+0000","last_became_peered":"2024-07-17T06:54:49.290069+0000","last_unstale":"2024-07-17T06:54:54.567150+0000","last_undegraded":"2024-07-17T06:54:54.567150+0000","last_fullsized":"2024-07-17T06:54:54.567150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:15.100108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.571210+0000","last_change":"2024-07-17T06:54:49.301304+0000","last_active":"2024-07-17T06:54:54.571210+0000","last_peered":"2024-07-17T06:54:54.571210+0000","last_clean":"2024-07-17T06:54:54.571210+0000","last_became_active":"2024-07-17T06:54:49.300865+0000","last_became_peered":"2024-07-17T06:54:49.300865+0000","last_unstale":"2024-07-17T06:54:54.571210+0000","last_undegraded":"2024-07-17T06:54:54.571210+0000","last_fullsized":"2024-07-17T06:54:54.571210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:56.255932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-07-17T06:54:54.569004+0000","last_change":"2024-07-17T06:54:49.324041+0000","last_active":"2024-07-17T06:54:54.569004+0000","last_peered":"2024-07-17T06:54:54.569004+0000","last_clean":"2024-07-17T06:54:54.569004+0000","last_became_active":"2024-07-17T06:54:49.323747+0000","last_became_peered":"2024-07-17T06:54:49.323747+0000","last_unstale":"2024-07-17T06:54:54.569004+0000","last_undegraded":"2024-07-17T06:54:54.569004+0000","last_fullsized":"2024-07-17T06:54:54.569004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:48.230836+0000","last_clean_scrub_stamp":"2024-07-17T06:54:48.230836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:04:46.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":101953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":102023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":692869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":692813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":101989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":101863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:55:16.576 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T06:55:16.576 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-17T06:55:16.576 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-07-17T06:55:16.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-07-17T06:55:16.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 -- 172.21.15.44:0/2456127261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f4004800 msgr2=0x7fc5f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 --2- 172.21.15.44:0/2456127261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f4004800 0x7fc5f4004c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc6100b41e0 tx=0x7fc5f4005830 comp rx=0 tx=0).stop 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc60fe00640 1 -- 172.21.15.44:0/2456127261 reap_dead start 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 -- 172.21.15.44:0/2456127261 shutdown_connections 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 -- 172.21.15.44:0/2456127261 >> 172.21.15.44:0/2456127261 conn(0x7fc6100b0e80 msgr2=0x7fc6100b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 -- 172.21.15.44:0/2456127261 shutdown_connections 2024-07-17T06:55:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.822+0000 7fc614e00640 1 -- 172.21.15.44:0/2456127261 wait complete. 2024-07-17T06:55:16.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.824+0000 7fc614e00640 1 Processor -- start 2024-07-17T06:55:16.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.824+0000 7fc614e00640 1 -- start start 2024-07-17T06:55:16.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f4004800 0x7fc610071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6100cdc00 0x7fc6100ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101411c0 con 0x7fc610157ef0 2024-07-17T06:55:16.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc610141040 con 0x7fc6100cdc00 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.825+0000 7fc614e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc610141340 con 0x7fc5f4004800 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37984/0 (socket says 172.21.15.44:37984) 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 -- 172.21.15.44:0/2813685301 learned_addr learned my addr 172.21.15.44:0/2813685301 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60ea00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6100cdc00 0x7fc6100ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60f400640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f4004800 0x7fc610071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 -- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f4004800 msgr2=0x7fc610071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f4004800 0x7fc610071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 -- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6100cdc00 msgr2=0x7fc6100ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6100cdc00 0x7fc6100ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 -- 172.21.15.44:0/2813685301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc610071fe0 con 0x7fc610157ef0 2024-07-17T06:55:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.826+0000 7fc60fe00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e890 tx=0x7fc60800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.827+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60800cc60 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.827+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc608019070 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.827+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080116d0 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.827+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6100722c0 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.827+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc610072880 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.828+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6100b3870 con 0x7fc610157ef0 2024-07-17T06:55:16.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.829+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc608011870 con 0x7fc610157ef0 2024-07-17T06:55:16.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.830+0000 7fc607400640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 0x7fc5f00790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:16.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.830+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc608098970 con 0x7fc610157ef0 2024-07-17T06:55:16.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.831+0000 7fc60f400640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 0x7fc5f00790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:16.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.832+0000 7fc60f400640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 0x7fc5f00790a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc60000cca0 tx=0x7fc60000c2d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:16.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:16.834+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc608061a60 con 0x7fc610157ef0 2024-07-17T06:55:17.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.078+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc610140340 con 0x7fc610157ef0 2024-07-17T06:55:17.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.079+0000 7fc607400640 1 -- 172.21.15.44:0/2813685301 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc608061360 con 0x7fc610157ef0 2024-07-17T06:55:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:17.079 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-17T06:55:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 msgr2=0x7fc5f00790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 0x7fc5f00790a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc60000cca0 tx=0x7fc60000c2d0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 msgr2=0x7fc610158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e890 tx=0x7fc60800ed50 comp rx=0 tx=0).stop 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 shutdown_connections 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc610157ef0 0x7fc610158330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6100cdc00 0x7fc6100ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc5f0076be0 0x7fc5f00790a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 --2- 172.21.15.44:0/2813685301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f4004800 0x7fc610071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.081+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 >> 172.21.15.44:0/2813685301 conn(0x7fc6100b0e80 msgr2=0x7fc610140840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.082+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 shutdown_connections 2024-07-17T06:55:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.082+0000 7fc614e00640 1 -- 172.21.15.44:0/2813685301 wait complete. 2024-07-17T06:55:17.099 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-07-17T06:55:17.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 -- 172.21.15.44:0/2256074548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f80ac3e0 msgr2=0x7f36f80ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 --2- 172.21.15.44:0/2256074548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f80ac3e0 0x7f36f80ac870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36e400a940 tx=0x7f36e402b850 comp rx=0 tx=0).stop 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 -- 172.21.15.44:0/2256074548 shutdown_connections 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 --2- 172.21.15.44:0/2256074548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f80ac3e0 0x7f36f80ac870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 -- 172.21.15.44:0/2256074548 >> 172.21.15.44:0/2256074548 conn(0x7f36f80b0fb0 msgr2=0x7f36f80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 -- 172.21.15.44:0/2256074548 shutdown_connections 2024-07-17T06:55:17.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.309+0000 7f3700000640 1 -- 172.21.15.44:0/2256074548 wait complete. 2024-07-17T06:55:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.311+0000 7f3700000640 1 Processor -- start 2024-07-17T06:55:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.311+0000 7f3700000640 1 -- start start 2024-07-17T06:55:17.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 0x7f36f814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36f80ac3e0 0x7f36f8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8153fb0 con 0x7f36f8153a50 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8153e30 con 0x7f36f806e090 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f3700000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36f806f710 con 0x7f36f80ac3e0 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37996/0 (socket says 172.21.15.44:37996) 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 -- 172.21.15.44:0/3013937738 learned_addr learned my addr 172.21.15.44:0/3013937738 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fd600640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 0x7f36f814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 -- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36f80ac3e0 msgr2=0x7f36f8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fcc00640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36f80ac3e0 0x7f36f8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36f80ac3e0 0x7f36f8074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 -- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 msgr2=0x7f36f814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 0x7f36f814f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fe000640 1 -- 172.21.15.44:0/3013937738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f81f41f0 con 0x7f36f8153a50 2024-07-17T06:55:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.312+0000 7f36fd600640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 0x7f36f814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f36fe000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c7b0 tx=0x7f36f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000ce60 con 0x7f36f8153a50 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f0004c00 con 0x7f36f8153a50 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f814fbc0 con 0x7f36f8153a50 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000a850 con 0x7f36f8153a50 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.313+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f80745f0 con 0x7f36f8153a50 2024-07-17T06:55:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.315+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f80b0370 con 0x7f36f8153a50 2024-07-17T06:55:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.316+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f36f000a9f0 con 0x7f36f8153a50 2024-07-17T06:55:17.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.317+0000 7f36f5600640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 0x7f36d8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.318+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f36f00989d0 con 0x7f36f8153a50 2024-07-17T06:55:17.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.318+0000 7f36fd600640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 0x7f36d8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.319+0000 7f36fd600640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 0x7f36d8079070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0013f0 tx=0x7f36ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:17.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.322+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f36f80b0370 con 0x7f36f8153a50 2024-07-17T06:55:17.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.579+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36f80ac870 con 0x7f36f8153a50 2024-07-17T06:55:17.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.580+0000 7f36f5600640 1 -- 172.21.15.44:0/3013937738 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f36f0020070 con 0x7f36f8153a50 2024-07-17T06:55:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:17.580 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:17.581 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:17.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 msgr2=0x7f36d8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 0x7f36d8079070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0013f0 tx=0x7f36ec009040 comp rx=0 tx=0).stop 2024-07-17T06:55:17.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 msgr2=0x7f36f81f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c7b0 tx=0x7f36f000cc70 comp rx=0 tx=0).stop 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 shutdown_connections 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36f8153a50 0x7f36f81f8000 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36d8076bb0 0x7f36d8079070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36f80ac3e0 0x7f36f8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 --2- 172.21.15.44:0/3013937738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36f806e090 0x7f36f814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.583+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 >> 172.21.15.44:0/3013937738 conn(0x7f36f80b0fb0 msgr2=0x7f36f80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.584+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 shutdown_connections 2024-07-17T06:55:17.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.584+0000 7f3700000640 1 -- 172.21.15.44:0/3013937738 wait complete. 2024-07-17T06:55:17.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 -- 172.21.15.44:0/2491162885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc15eff0 msgr2=0x7fd9bc06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 --2- 172.21.15.44:0/2491162885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc06f7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0cdf30 tx=0x7fd9b402a970 comp rx=0 tx=0).stop 2024-07-17T06:55:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 -- 172.21.15.44:0/2491162885 shutdown_connections 2024-07-17T06:55:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 --2- 172.21.15.44:0/2491162885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc06f7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 --2- 172.21.15.44:0/2491162885 >> v2:172.21.15.175:3300/0 conn(0x7fd9bc1444d0 0x7fd9bc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.810+0000 7fd9c3400640 1 -- 172.21.15.44:0/2491162885 >> 172.21.15.44:0/2491162885 conn(0x7fd9bc072730 msgr2=0x7fd9bc072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.811+0000 7fd9c3400640 1 -- 172.21.15.44:0/2491162885 shutdown_connections 2024-07-17T06:55:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.811+0000 7fd9c3400640 1 -- 172.21.15.44:0/2491162885 wait complete. 2024-07-17T06:55:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.812+0000 7fd9c3400640 1 Processor -- start 2024-07-17T06:55:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.812+0000 7fd9c3400640 1 -- start start 2024-07-17T06:55:17.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9bc1444d0 0x7fd9bc0aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc1ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0700a0 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc06ff20 con 0x7fd9bc15eff0 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.813+0000 7fd9c3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0b5850 con 0x7fd9bc1444d0 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.814+0000 7fd9c0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.814+0000 7fd9c0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38008/0 (socket says 172.21.15.44:38008) 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.814+0000 7fd9c0a00640 1 -- 172.21.15.44:0/3875521406 learned_addr learned my addr 172.21.15.44:0/3875521406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.814+0000 7fd9c1400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc1ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.814+0000 7fd9bbe00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9bc1444d0 0x7fd9bc0aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 -- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9bc1444d0 msgr2=0x7fd9bc0aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9bc1444d0 0x7fd9bc0aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 -- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc15eff0 msgr2=0x7fd9bc1ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc1ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 -- 172.21.15.44:0/3875521406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc1ef880 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c0a00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d770 tx=0x7fd9ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac017070 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ac009d20 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac005280 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc0a9e90 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.815+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc0aa450 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.816+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc1448b0 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.818+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fd9ac005420 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.819+0000 7fd9b8c00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 0x7fd99c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:17.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.819+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd9ac0986b0 con 0x7fd9bc0cd8e0 2024-07-17T06:55:17.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.819+0000 7fd9bbe00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 0x7fd99c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:17.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.820+0000 7fd9bbe00640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 0x7fd99c079070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006fd0 tx=0x7fd9a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:17.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:17.823+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd9ac0611e0 con 0x7fd9bc0cd8e0 2024-07-17T06:55:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.074+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9bc072bb0 con 0x7fd9bc0cd8e0 2024-07-17T06:55:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.075+0000 7fd9b8c00640 1 -- 172.21.15.44:0/3875521406 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7fd9ac065730 con 0x7fd9bc0cd8e0 2024-07-17T06:55:18.075 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:18.076 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:18.076 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:18.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 msgr2=0x7fd99c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 0x7fd99c079070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006fd0 tx=0x7fd9a8008040 comp rx=0 tx=0).stop 2024-07-17T06:55:18.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 msgr2=0x7fd9bc0a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d770 tx=0x7fd9ac00dc30 comp rx=0 tx=0).stop 2024-07-17T06:55:18.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 shutdown_connections 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9bc15eff0 0x7fd9bc1ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd99c076bb0 0x7fd99c079070 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9bc1444d0 0x7fd9bc0aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 --2- 172.21.15.44:0/3875521406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9bc0cd8e0 0x7fd9bc0a98b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.078+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 >> 172.21.15.44:0/3875521406 conn(0x7fd9bc072730 msgr2=0x7fd9bc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.079+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 shutdown_connections 2024-07-17T06:55:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.079+0000 7fd9c3400640 1 -- 172.21.15.44:0/3875521406 wait complete. 2024-07-17T06:55:18.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:18.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 -- 172.21.15.44:0/2203872235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 msgr2=0x7faf30003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 --2- 172.21.15.44:0/2203872235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf30003f10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0016c0 tx=0x7faf30005830 comp rx=0 tx=0).stop 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf4be00640 1 -- 172.21.15.44:0/2203872235 reap_dead start 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 -- 172.21.15.44:0/2203872235 shutdown_connections 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 -- 172.21.15.44:0/2203872235 >> 172.21.15.44:0/2203872235 conn(0x7faf4c0cfec0 msgr2=0x7faf4c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 -- 172.21.15.44:0/2203872235 shutdown_connections 2024-07-17T06:55:18.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.301+0000 7faf51200640 1 -- 172.21.15.44:0/2203872235 wait complete. 2024-07-17T06:55:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.303+0000 7faf51200640 1 Processor -- start 2024-07-17T06:55:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.303+0000 7faf51200640 1 -- start start 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c1f76d0 0x7faf4c1f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c1f8090 0x7faf4c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c15dd60 con 0x7faf30003af0 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c15dbe0 con 0x7faf4c1f8090 2024-07-17T06:55:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf51200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c15dee0 con 0x7faf4c1f76d0 2024-07-17T06:55:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf4b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf4b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38032/0 (socket says 172.21.15.44:38032) 2024-07-17T06:55:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.304+0000 7faf4b400640 1 -- 172.21.15.44:0/837289670 learned_addr learned my addr 172.21.15.44:0/837289670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 -- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c1f76d0 msgr2=0x7faf4c1f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf43e00640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c1f76d0 0x7faf4c1f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c1f76d0 0x7faf4c1f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 -- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c1f8090 msgr2=0x7faf4c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4be00640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c1f8090 0x7faf4c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c1f8090 0x7faf4c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 -- 172.21.15.44:0/837289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c0cc030 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf4b400640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0017e0 tx=0x7faf3c0063e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c041070 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.305+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf3c003af0 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.306+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c1f8540 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.306+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c038070 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.306+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c0cbd50 con 0x7faf30003af0 2024-07-17T06:55:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.307+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c15e470 con 0x7faf30003af0 2024-07-17T06:55:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.309+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7faf3c044070 con 0x7faf30003af0 2024-07-17T06:55:18.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.310+0000 7faf43400640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 0x7faf280790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.310+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf3c0b7d10 con 0x7faf30003af0 2024-07-17T06:55:18.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.310+0000 7faf43e00640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 0x7faf280790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.311+0000 7faf43e00640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 0x7faf280790a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf34001860 tx=0x7faf34000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:18.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.317+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faf3c0bd020 con 0x7faf30003af0 2024-07-17T06:55:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.532+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf4c0b5d70 con 0x7faf30003af0 2024-07-17T06:55:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.534+0000 7faf43400640 1 -- 172.21.15.44:0/837289670 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7faf4c0cc030 con 0x7faf30003af0 2024-07-17T06:55:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:18.535 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:54:53.820795+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.537+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 msgr2=0x7faf280790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.537+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 0x7faf280790a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf34001860 tx=0x7faf34000ed0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.537+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 msgr2=0x7faf4c156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.537+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0017e0 tx=0x7faf3c0063e0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 shutdown_connections 2024-07-17T06:55:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4c1f8090 0x7faf4c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7faf28076be0 0x7faf280790a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4c1f76d0 0x7faf4c1f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 --2- 172.21.15.44:0/837289670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf30003af0 0x7faf4c156d30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 >> 172.21.15.44:0/837289670 conn(0x7faf4c0cfec0 msgr2=0x7faf4c15d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 shutdown_connections 2024-07-17T06:55:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.538+0000 7faf51200640 1 -- 172.21.15.44:0/837289670 wait complete. 2024-07-17T06:55:18.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:18.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 -- 172.21.15.44:0/1714129085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 msgr2=0x7f323c0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 --2- 172.21.15.44:0/1714129085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c0b3870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f322c009830 tx=0x7f322c02a510 comp rx=0 tx=0).stop 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 -- 172.21.15.44:0/1714129085 shutdown_connections 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 --2- 172.21.15.44:0/1714129085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0b3db0 0x7f323c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 --2- 172.21.15.44:0/1714129085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c0b3870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.768+0000 7f323be00640 1 -- 172.21.15.44:0/1714129085 >> 172.21.15.44:0/1714129085 conn(0x7f323c0b0e80 msgr2=0x7f323c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.769+0000 7f323be00640 1 -- 172.21.15.44:0/1714129085 shutdown_connections 2024-07-17T06:55:18.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.769+0000 7f323be00640 1 -- 172.21.15.44:0/1714129085 wait complete. 2024-07-17T06:55:18.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.770+0000 7f323be00640 1 Processor -- start 2024-07-17T06:55:18.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.770+0000 7f323be00640 1 -- start start 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323c0b3db0 0x7f323c1e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0cf920 0x7f323c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f323c0ae940 con 0x7f323c0cf540 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f323c0ae7c0 con 0x7f323c0b3db0 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.771+0000 7f323be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f323c141040 con 0x7f323c0cf920 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38040/0 (socket says 172.21.15.44:38040) 2024-07-17T06:55:18.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 -- 172.21.15.44:0/4293355136 learned_addr learned my addr 172.21.15.44:0/4293355136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323aa00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323c0b3db0 0x7f323c1e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323b400640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0cf920 0x7f323c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 -- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0cf920 msgr2=0x7f323c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0cf920 0x7f323c154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 -- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323c0b3db0 msgr2=0x7f323c1e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323c0b3db0 0x7f323c1e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:18.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.772+0000 7f323a000640 1 -- 172.21.15.44:0/4293355136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c15c620 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f323a000640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f322800c890 tx=0x7f322800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228004220 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32280079b0 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228007de0 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c15c900 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.773+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c15cec0 con 0x7f323c0cf540 2024-07-17T06:55:18.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.774+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323c1414c0 con 0x7f323c0cf540 2024-07-17T06:55:18.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.775+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f3228005020 con 0x7f323c0cf540 2024-07-17T06:55:18.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.776+0000 7f3232a00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 0x7f321c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:18.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.776+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3228098b30 con 0x7f323c0cf540 2024-07-17T06:55:18.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.777+0000 7f323aa00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 0x7f321c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:18.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.778+0000 7f323aa00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 0x7f321c079070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f322c034000 tx=0x7f322c004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:18.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:18.780+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3228014030 con 0x7f323c0cf540 2024-07-17T06:55:19.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.028+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f323c06ec40 con 0x7f323c0cf540 2024-07-17T06:55:19.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.029+0000 7f3232a00640 1 -- 172.21.15.44:0/4293355136 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f3228061c20 con 0x7f323c0cf540 2024-07-17T06:55:19.029 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:19.029 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:19.030 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:19.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 msgr2=0x7f321c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 0x7f321c079070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f322c034000 tx=0x7f322c004730 comp rx=0 tx=0).stop 2024-07-17T06:55:19.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 msgr2=0x7f323c1e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f322800c890 tx=0x7f322800cd50 comp rx=0 tx=0).stop 2024-07-17T06:55:19.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 shutdown_connections 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f321c076bb0 0x7f321c079070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323c0cf920 0x7f323c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f323c0cf540 0x7f323c1e6810 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 --2- 172.21.15.44:0/4293355136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323c0b3db0 0x7f323c1e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 >> 172.21.15.44:0/4293355136 conn(0x7f323c0b0e80 msgr2=0x7f323c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.032+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 shutdown_connections 2024-07-17T06:55:19.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.033+0000 7f323be00640 1 -- 172.21.15.44:0/4293355136 wait complete. 2024-07-17T06:55:19.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T06:54:48.717423+0000', 'modified': '2024-07-17T06:55:06.915431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4329}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4329': {'gid': 4329, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/910842174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 910842174}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 910842174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4329, 'qdb_cluster': [4329]} 2024-07-17T06:55:19.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T06:55:19.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-17T06:55:19.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 -- 172.21.15.44:0/2214730323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640b3280 msgr2=0x7ff16406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 --2- 172.21.15.44:0/2214730323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640b3280 0x7ff16406eb90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff150005fd0 tx=0x7ff15002a530 comp rx=0 tx=0).stop 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 -- 172.21.15.44:0/2214730323 shutdown_connections 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 --2- 172.21.15.44:0/2214730323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640b3280 0x7ff16406eb90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 --2- 172.21.15.44:0/2214730323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff16406f6d0 0x7ff16415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 -- 172.21.15.44:0/2214730323 >> 172.21.15.44:0/2214730323 conn(0x7ff1640b0eb0 msgr2=0x7ff1640b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.680+0000 7ff16c400640 1 -- 172.21.15.44:0/2214730323 shutdown_connections 2024-07-17T06:55:19.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.681+0000 7ff16c400640 1 -- 172.21.15.44:0/2214730323 wait complete. 2024-07-17T06:55:19.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.682+0000 7ff16c400640 1 Processor -- start 2024-07-17T06:55:19.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.682+0000 7ff16c400640 1 -- start start 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16406f6d0 0x7ff1640ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 0x7ff1640734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640accd0 con 0x7ff1640ae560 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640aeba0 con 0x7ff16406f6d0 2024-07-17T06:55:19.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.683+0000 7ff16c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff16414fd60 con 0x7ff1640b3280 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43578/0 (socket says 172.21.15.44:43578) 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 -- 172.21.15.44:0/3665863031 learned_addr learned my addr 172.21.15.44:0/3665863031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169a00640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16406f6d0 0x7ff1640ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff16a400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 0x7ff1640734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 -- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 msgr2=0x7ff1640734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 0x7ff1640734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 -- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16406f6d0 msgr2=0x7ff1640ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16406f6d0 0x7ff1640ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff169000640 1 -- 172.21.15.44:0/3665863031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1640c8530 con 0x7ff1640ae560 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.684+0000 7ff16a400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 0x7ff1640734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff169000640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff150034040 tx=0x7ff1500039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff161600640 1 -- 172.21.15.44:0/3665863031 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15003e030 con 0x7ff1640ae560 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff161600640 1 -- 172.21.15.44:0/3665863031 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1500047a0 con 0x7ff1640ae560 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1640ca8f0 con 0x7ff1640ae560 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff161600640 1 -- 172.21.15.44:0/3665863031 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150035040 con 0x7ff1640ae560 2024-07-17T06:55:19.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.685+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1640c8a80 con 0x7ff1640ae560 2024-07-17T06:55:19.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.688+0000 7ff161600640 1 -- 172.21.15.44:0/3665863031 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ff150041030 con 0x7ff1640ae560 2024-07-17T06:55:19.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.689+0000 7ff161600640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 0x7ff144079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.689+0000 7ff161600640 1 -- 172.21.15.44:0/3665863031 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff1500b8de0 con 0x7ff1640ae560 2024-07-17T06:55:19.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.690+0000 7ff16f1c1740 1 Processor -- start 2024-07-17T06:55:19.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.690+0000 7ff16f1c1740 1 -- start start 2024-07-17T06:55:19.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.690+0000 7ff169a00640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 0x7ff144079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbffde0 0x562d9dc001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbfca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d9db47d60 con 0x562d9dbd70a0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562d9db37260 con 0x562d9dbffde0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562d9db3b2a0 con 0x562d9dbfc5b0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff169a00640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 0x7ff144079070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff158005e10 tx=0x7ff158005da0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43590/0 (socket says 172.21.15.44:43590) 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff169000640 1 -- 172.21.15.44:0/3522704468 learned_addr learned my addr 172.21.15.44:0/3522704468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.691+0000 7ff169a00640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbffde0 0x562d9dc001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff16a400640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbfca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 -- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 msgr2=0x562d9dbfca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbfca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 -- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbffde0 msgr2=0x562d9dc001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbffde0 0x562d9dc001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 -- 172.21.15.44:0/3522704468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9d89b850 con 0x562d9dbd70a0 2024-07-17T06:55:19.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff16a400640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbfca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff169000640 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff15007d8e0 tx=0x7ff150081270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0cb055f9ea5ca7 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff156000640 1 -- 172.21.15.44:0/3522704468 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150048070 con 0x562d9dbd70a0 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff156000640 1 -- 172.21.15.44:0/3522704468 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff150081370 con 0x562d9dbd70a0 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.692+0000 7ff156000640 1 -- 172.21.15.44:0/3522704468 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15008a5f0 con 0x562d9dbd70a0 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3522704468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 msgr2=0x562d9dbfc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff15007d8e0 tx=0x7ff150081270 comp rx=0 tx=0).stop 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3522704468 shutdown_connections 2024-07-17T06:55:19.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbfca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbd70a0 0x562d9dbfc070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3522704468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbffde0 0x562d9dc001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3522704468 >> 172.21.15.44:0/3522704468 conn(0x562d9dbd2280 msgr2=0x562d9dbd5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3522704468 shutdown_connections 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.693+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3522704468 wait complete. 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.694+0000 7ff16f1c1740 1 -- start start 2024-07-17T06:55:19.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.694+0000 7ff16f1c1740 1 Processor -- start 2024-07-17T06:55:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbf5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbe3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d9d8aa620 con 0x562d9dbffde0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562d9dadef90 con 0x562d9dbf4320 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562d9db37260 con 0x562d9dbfc5b0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.695+0000 7ff16f1c1740 10 client.0 _ll_register_callbacks cb 0x562d9db04f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43596/0 (socket says 172.21.15.44:43596) 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 -- 172.21.15.44:0/3261172598 learned_addr learned my addr 172.21.15.44:0/3261172598 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169a00640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbf5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff16a400640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbe3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 -- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 msgr2=0x562d9dbf5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbf5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 -- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 msgr2=0x562d9dbe3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbe3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 -- 172.21.15.44:0/3261172598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9d9126b0 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff169000640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff15002ab20 tx=0x7ff15008a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae6f90738339263 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.696+0000 7ff14aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150048040 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff150082da0 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d9db0e900 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7ff150081f60 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 10 client.4782 fetch_fsmap learned FSMap version 9 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x562d9dbfbe00 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562d9db0e900 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7ff150080ac0 con 0x562d9dbffde0 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 10 client.4782 fetch_fsmap finished waiting for FSMap version 9 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 10 client.4782 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T06:55:19.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 20 client.4782 populate_metadata read hostname 'smithi044' 2024-07-17T06:55:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc10f90 0x562d9dc2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 4 client.4782 mds_command: new command op to 4329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T06:55:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.697+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 --> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d9db49cb0 con 0x562d9dc10f90 2024-07-17T06:55:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.698+0000 7ff169a00640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc10f90 0x562d9dc2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.699+0000 7ff169a00640 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc10f90 0x562d9dc2de20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2470c6223e32282 server_cookie=cbaafad25dc999c0 in_seq=0 out_seq=0 2024-07-17T06:55:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.699+0000 7ff14aa00640 10 client.4782 ms_handle_connect on v2:172.21.15.175:6834/910842174 2024-07-17T06:55:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff14aa00640 1 -- 172.21.15.44:0/3261172598 <== mds.0 v2:172.21.15.175:6834/910842174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d9db49cb0 con 0x562d9dc10f90 2024-07-17T06:55:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff14aa00640 10 client.4782 handle_command_reply: tid=0 2024-07-17T06:55:19.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 client.4782 shutdown 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 msgr2=0x562d9dbf3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff15002ab20 tx=0x7ff15008a680 comp rx=0 tx=0).stop 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 shutdown_connections 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc10f90 0x562d9dc2de20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbe3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dbf3de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/3261172598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dbf5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff14aa00640 0 client.4782 ms_handle_reset on v2:172.21.15.175:6834/910842174 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 >> 172.21.15.44:0/3261172598 conn(0x562d9dbd70a0 msgr2=0x562d9dbcf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 shutdown_connections 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 1 -- 172.21.15.44:0/3261172598 wait complete. 2024-07-17T06:55:19.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.701+0000 7ff16f1c1740 20 client.4782 trim_cache size 0 max 16384 2024-07-17T06:55:19.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.722+0000 7ff16f1c1740 1 Processor -- start 2024-07-17T06:55:19.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.722+0000 7ff16f1c1740 1 -- start start 2024-07-17T06:55:19.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbfc5b0 0x562d9dbf2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbffde0 0x562d9dbcf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff164200d60 con 0x562d9dbf4320 2024-07-17T06:55:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641f48f0 con 0x562d9dbfc5b0 2024-07-17T06:55:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.723+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff164200710 con 0x562d9dbffde0 2024-07-17T06:55:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43606/0 (socket says 172.21.15.44:43606) 2024-07-17T06:55:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 -- 172.21.15.44:0/2387511420 learned_addr learned my addr 172.21.15.44:0/2387511420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 -- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbffde0 msgr2=0x562d9dbcf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbffde0 0x562d9dbcf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 -- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbfc5b0 msgr2=0x562d9dbf2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbfc5b0 0x562d9dbf2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 -- 172.21.15.44:0/2387511420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9dc106a0 con 0x562d9dbf4320 2024-07-17T06:55:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.724+0000 7ff169a00640 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff15800efc0 tx=0x7ff158014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d0ed0d79b3156a server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff155600640 1 -- 172.21.15.44:0/2387511420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580285f0 con 0x562d9dbf4320 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff155600640 1 -- 172.21.15.44:0/2387511420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff15802e070 con 0x562d9dbf4320 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff155600640 1 -- 172.21.15.44:0/2387511420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158028b20 con 0x562d9dbf4320 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2387511420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 msgr2=0x562d9dc001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff15800efc0 tx=0x7ff158014000 comp rx=0 tx=0).stop 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2387511420 shutdown_connections 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbffde0 0x562d9dbcf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbfc5b0 0x562d9dbf2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2387511420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbf4320 0x562d9dc001c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.725+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2387511420 >> 172.21.15.44:0/2387511420 conn(0x562d9dbd70a0 msgr2=0x562d9dbe3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.726+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2387511420 shutdown_connections 2024-07-17T06:55:19.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.726+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2387511420 wait complete. 2024-07-17T06:55:19.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.726+0000 7ff16f1c1740 1 -- start start 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.726+0000 7ff16f1c1740 1 Processor -- start 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbd4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dc1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562d9dbfbe00 con 0x562d9dbffde0 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562d9db47d60 con 0x562d9dbf4320 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.727+0000 7ff16f1c1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562d9dc11390 con 0x562d9dbfc5b0 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16f1c1740 10 client.0 _ll_register_callbacks cb 0x7ff150081630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43612/0 (socket says 172.21.15.44:43612) 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 -- 172.21.15.44:0/2619477726 learned_addr learned my addr 172.21.15.44:0/2619477726 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:19.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff169000640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dc1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff169a00640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbd4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 -- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 msgr2=0x562d9dc1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dc1d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 -- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 msgr2=0x562d9dbd4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbd4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff16a400640 1 -- 172.21.15.44:0/2619477726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9dc009e0 con 0x562d9dbffde0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.728+0000 7ff169a00640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbd4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff16a400640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0104f0 tx=0x7ff15c0109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65949c4ee8f30aa1 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff14aa00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c018020 con 0x562d9dbffde0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff15c007050 con 0x562d9dbffde0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15802e070 con 0x562d9dbffde0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7ff15c022050 con 0x562d9dbffde0 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff16f1c1740 10 client.4788 fetch_fsmap learned FSMap version 9 2024-07-17T06:55:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.729+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x562d9dbfa950 con 0x562d9dbffde0 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff15802e070 con 0x562d9dbffde0 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7ff15c025020 con 0x562d9dbffde0 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 10 client.4788 fetch_fsmap finished waiting for FSMap version 9 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 10 client.4788 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 20 client.4788 populate_metadata read hostname 'smithi044' 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc98830 0x562d9dc98c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 4 client.4788 mds_command: new command op to 4329 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 --> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] -- command(tid 0: {"prefix": "status"}) -- 0x562d9db45750 con 0x562d9dc98830 2024-07-17T06:55:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.730+0000 7ff169a00640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc98830 0x562d9dc98c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.731+0000 7ff169a00640 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc98830 0x562d9dc98c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20ad6c5630c9df1 server_cookie=6c318a8a2038611d in_seq=0 out_seq=0 2024-07-17T06:55:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.731+0000 7ff14aa00640 10 client.4788 ms_handle_connect on v2:172.21.15.175:6834/910842174 2024-07-17T06:55:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff14aa00640 1 -- 172.21.15.44:0/2619477726 <== mds.0 v2:172.21.15.175:6834/910842174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562d9db45750 con 0x562d9dc98830 2024-07-17T06:55:19.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff14aa00640 10 client.4788 handle_command_reply: tid=0 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 client.4788 shutdown 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 msgr2=0x562d9dc1d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0104f0 tx=0x7ff15c0109b0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 shutdown_connections 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:6834/910842174,v1:172.21.15.175:6835/910842174] conn(0x562d9dc98830 0x562d9dc98c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562d9dbffde0 0x562d9dc1d690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562d9dbfc5b0 0x562d9dc1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 --2- 172.21.15.44:0/2619477726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562d9dbf4320 0x562d9dbd4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff14aa00640 0 client.4788 ms_handle_reset on v2:172.21.15.175:6834/910842174 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 >> 172.21.15.44:0/2619477726 conn(0x562d9dbd70a0 msgr2=0x562d9dbcf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 shutdown_connections 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.732+0000 7ff16f1c1740 1 -- 172.21.15.44:0/2619477726 wait complete. 2024-07-17T06:55:19.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16f1c1740 20 client.4788 trim_cache size 0 max 16384 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4329, 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 30.449041114, 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 9, 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-07-17T06:55:19.735 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 25, 2024-07-17T06:55:19.736 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 37.124551551000003 2024-07-17T06:55:19.736 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T06:55:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 msgr2=0x7ff144079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 0x7ff144079070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff158005e10 tx=0x7ff158005da0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 msgr2=0x7ff1641f4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff150034040 tx=0x7ff1500039e0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 shutdown_connections 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff144076bb0 0x7ff144079070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1640b3280 0x7ff1640734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1640ae560 0x7ff1641f4f60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 --2- 172.21.15.44:0/3665863031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff16406f6d0 0x7ff1640ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 >> 172.21.15.44:0/3665863031 conn(0x7ff1640b0eb0 msgr2=0x7ff1640ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 shutdown_connections 2024-07-17T06:55:19.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:19.733+0000 7ff16c400640 1 -- 172.21.15.44:0/3665863031 wait complete. 2024-07-17T06:55:19.752 INFO:teuthology.run_tasks:Running task sequential... 2024-07-17T06:55:19.761 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-17T06:55:19.761 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-17T06:55:19.762 INFO:teuthology.task.print:Enabling mgr modules 2024-07-17T06:55:19.762 INFO:teuthology.run_tasks:Running task kclient... 2024-07-17T06:55:19.774 INFO:tasks.kclient:Mounting kernel clients... 2024-07-17T06:55:19.774 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-07-17T06:55:19.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:19.774 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:55:19.827 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-07-17T06:55:19.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:19.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-17T06:55:19.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:19.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-07-17T06:55:20.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.361 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:55:20.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-07-17T06:55:20.534 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-07-17T06:55:20.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:55:20.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.538 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T06:55:20.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-17T06:55:20.680 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-07-17T06:55:20.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:55:20.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.685 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T06:55:20.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:20.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-17T06:55:20.819 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-07-17T06:55:20.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T06:55:20.822 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-07-17T06:55:20.822 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-17T06:55:20.822 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T06:55:20.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:55:21.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 -- 172.21.15.44:0/2962750160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 msgr2=0x7f2d000b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 --2- 172.21.15.44:0/2962750160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d000b4a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d000cdad0 tx=0x7f2cf402a980 comp rx=0 tx=0).stop 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 -- 172.21.15.44:0/2962750160 shutdown_connections 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 --2- 172.21.15.44:0/2962750160 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 0x7f2d000d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 --2- 172.21.15.44:0/2962750160 >> v2:172.21.15.175:3300/0 conn(0x7f2d000b3420 0x7f2d000b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 --2- 172.21.15.44:0/2962750160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d000b4a30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 -- 172.21.15.44:0/2962750160 >> 172.21.15.44:0/2962750160 conn(0x7f2d0008ca20 msgr2=0x7f2d0008ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 -- 172.21.15.44:0/2962750160 shutdown_connections 2024-07-17T06:55:21.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.044+0000 7f2d06c00640 1 -- 172.21.15.44:0/2962750160 wait complete. 2024-07-17T06:55:21.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.046+0000 7f2d06c00640 1 Processor -- start 2024-07-17T06:55:21.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.046+0000 7f2d06c00640 1 -- start start 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d000b3420 0x7f2d0009d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 0x7f2d001f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000d2310 con 0x7f2d000d3420 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000d2190 con 0x7f2d000b3420 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.047+0000 7f2d06c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d000b3e10 con 0x7f2d000d3800 2024-07-17T06:55:21.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43626/0 (socket says 172.21.15.44:43626) 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 -- 172.21.15.44:0/3691472006 learned_addr learned my addr 172.21.15.44:0/3691472006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d06200640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 0x7f2d001f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d05800640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d000b3420 0x7f2d0009d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 -- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 msgr2=0x7f2d001f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 0x7f2d001f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 -- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d000b3420 msgr2=0x7f2d0009d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d000b3420 0x7f2d0009d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.048+0000 7f2d04e00640 1 -- 172.21.15.44:0/3691472006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0009fad0 con 0x7f2d000d3420 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2d04e00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000d780 tx=0x7f2cf000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0017070 con 0x7f2d000d3420 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cf0009b10 con 0x7f2d000d3420 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0010460 con 0x7f2d000d3420 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0009fdb0 con 0x7f2d000d3420 2024-07-17T06:55:21.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.049+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0009d680 con 0x7f2d000d3420 2024-07-17T06:55:21.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.051+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d000b4a30 con 0x7f2d000d3420 2024-07-17T06:55:21.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.052+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f2cf001a070 con 0x7f2d000d3420 2024-07-17T06:55:21.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.052+0000 7f2cfd600640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 0x7f2ce4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:21.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.053+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2cf0098690 con 0x7f2d000d3420 2024-07-17T06:55:21.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.053+0000 7f2d05800640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 0x7f2ce4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:21.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.054+0000 7f2d05800640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 0x7f2ce4079070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4035000 tx=0x7f2cf4004ab0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:21.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.057+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2cf00611c0 con 0x7f2d000d3420 2024-07-17T06:55:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.316+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2d0008cf80 con 0x7f2d000d3420 2024-07-17T06:55:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.317+0000 7f2cfd600640 1 -- 172.21.15.44:0/3691472006 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f2cf0065710 con 0x7f2d000d3420 2024-07-17T06:55:21.318 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-17T06:55:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 msgr2=0x7f2ce4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 0x7f2ce4079070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4035000 tx=0x7f2cf4004ab0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 msgr2=0x7f2d001f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000d780 tx=0x7f2cf000dc40 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 shutdown_connections 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2ce4076bb0 0x7f2ce4079070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d000d3800 0x7f2d001f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d000d3420 0x7f2d001f7120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 --2- 172.21.15.44:0/3691472006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d000b3420 0x7f2d0009d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 >> 172.21.15.44:0/3691472006 conn(0x7f2d0008ca20 msgr2=0x7f2d000b8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.320+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 shutdown_connections 2024-07-17T06:55:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21.321+0000 7f2d06c00640 1 -- 172.21.15.44:0/3691472006 wait complete. 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:55:21.341 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T06:55:21.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:21.342 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T06:55:21.396 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42537sec preferred_lft 42537sec 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:21.397 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T06:55:21.398 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-07-17T06:55:21.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-17T06:55:21.399 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:21.651 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:55:21.719 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T06:55:21.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:21.724 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T06:55:21.774 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T06:55:21.774 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:55:21.775 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T06:55:21.775 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T06:55:21.775 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T06:55:21.776 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:21.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:21.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:21.986 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:55:22.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:22.036 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-07-17T06:55:22.090 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:22.090 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:22.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T06:55:22.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-17T06:55:22.090 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:22.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:22.279 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-17T06:55:22.279 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:22.279 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:22.279 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-17T06:55:22.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-17T06:55:22.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-17T06:55:22.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-17T06:55:22.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-17T06:55:22.280 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:22.547 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:22.547 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:22.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T06:55:22.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-07-17T06:55:22.547 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:22.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:22.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:22.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:22.746 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T06:55:22.784 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T06:55:22.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:22.785 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T06:55:22.872 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T06:55:22.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:22.872 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:55:23.026 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:55:23.026 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:55:23.026 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:23.767 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:55:23.768 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:55:23.768 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:23.768 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:23.768 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T06:55:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T06:55:23.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T06:55:23.809 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T06:55:23.809 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T06:55:23.809 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T06:55:23.809 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T06:55:23.815 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T06:55:23.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:23.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T06:55:23.891 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:23.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:23.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:24.006 INFO:teuthology.orchestra.run.smithi044.stdout:client4526 2024-07-17T06:55:24.007 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:24.007 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:24.057 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:24.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:24.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:24.224 INFO:teuthology.orchestra.run.smithi044.stdout:client4526 2024-07-17T06:55:24.224 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:24.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:24.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:24.340 INFO:teuthology.orchestra.run.smithi044.stdout:client4526 2024-07-17T06:55:24.341 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4797a2db-a688-4e60-bfb8-d8a9e86b84a3.client4526/status') 2024-07-17T06:55:24.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:24.469 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.4526 (3)172.21.15.44:0/3348435759 2024-07-17T06:55:24.469 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-07-17T06:55:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-07-17T06:55:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-07-17T06:55:24.469 INFO:teuthology.orchestra.run.smithi044.stderr:70 bytes copied, 6.9609e-05 s, 1.0 MB/s 2024-07-17T06:55:24.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:24.482 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-07-17T06:55:24.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-17T06:55:24.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-07-17T06:55:24.483 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 -- 172.21.15.44:0/934147476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8ac004800 msgr2=0x7fe8ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 --2- 172.21.15.44:0/934147476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8ac004800 0x7fe8ac004c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001690 tx=0x7fe8ac005830 comp rx=0 tx=0).stop 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 -- 172.21.15.44:0/934147476 shutdown_connections 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 --2- 172.21.15.44:0/934147476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8ac004800 0x7fe8ac004c80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 -- 172.21.15.44:0/934147476 >> 172.21.15.44:0/934147476 conn(0x7fe8c80b0fb0 msgr2=0x7fe8c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:24.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 -- 172.21.15.44:0/934147476 shutdown_connections 2024-07-17T06:55:24.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.687+0000 7fe8ce200640 1 -- 172.21.15.44:0/934147476 wait complete. 2024-07-17T06:55:24.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.689+0000 7fe8ce200640 1 Processor -- start 2024-07-17T06:55:24.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.689+0000 7fe8ce200640 1 -- start start 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 0x7fe8c80993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c809cad0 0x7fe8c8154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c80ad910 con 0x7fe8c8150bb0 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c80ad790 con 0x7fe8c809cad0 2024-07-17T06:55:24.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.690+0000 7fe8ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c80ada90 con 0x7fe8ac004800 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43642/0 (socket says 172.21.15.44:43642) 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 -- 172.21.15.44:0/125705486 learned_addr learned my addr 172.21.15.44:0/125705486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8c7400640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 0x7fe8c80993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8c7e00640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c809cad0 0x7fe8c8154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 -- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 msgr2=0x7fe8c80993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 0x7fe8c80993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 -- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c809cad0 msgr2=0x7fe8c8154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c809cad0 0x7fe8c8154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 -- 172.21.15.44:0/125705486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8151030 con 0x7fe8c8150bb0 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8c7400640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 0x7fe8c80993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.691+0000 7fe8bfe00640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00017b0 tx=0x7fe8b0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:24.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.692+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0003a00 con 0x7fe8c8150bb0 2024-07-17T06:55:24.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.692+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b0039070 con 0x7fe8c8150bb0 2024-07-17T06:55:24.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.692+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0033420 con 0x7fe8c8150bb0 2024-07-17T06:55:24.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.692+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c809d130 con 0x7fe8c8150bb0 2024-07-17T06:55:24.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.692+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c8152d40 con 0x7fe8c8150bb0 2024-07-17T06:55:24.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.693+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c80afa70 con 0x7fe8c8150bb0 2024-07-17T06:55:24.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.694+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe8b003b030 con 0x7fe8c8150bb0 2024-07-17T06:55:24.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.695+0000 7fe8bf400640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 0x7fe8a4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:24.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.695+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8b00b7a50 con 0x7fe8c8150bb0 2024-07-17T06:55:24.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.696+0000 7fe8c7400640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 0x7fe8a4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:24.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.697+0000 7fe8c7400640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 0x7fe8a4079070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8001860 tx=0x7fe8b8001110 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:24.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.699+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe8c80afa70 con 0x7fe8c8150bb0 2024-07-17T06:55:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.954+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8c809ceb0 con 0x7fe8c8150bb0 2024-07-17T06:55:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.955+0000 7fe8bf400640 1 -- 172.21.15.44:0/125705486 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe8c809ceb0 con 0x7fe8c8150bb0 2024-07-17T06:55:24.955 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.957+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 msgr2=0x7fe8a4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.957+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 0x7fe8a4079070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8001860 tx=0x7fe8b8001110 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.957+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 msgr2=0x7fe8c809c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.957+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00017b0 tx=0x7fe8b0006430 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 shutdown_connections 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c809cad0 0x7fe8c8154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c8150bb0 0x7fe8c809c590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe8a4076bb0 0x7fe8a4079070 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 --2- 172.21.15.44:0/125705486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8ac004800 0x7fe8c80993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 >> 172.21.15.44:0/125705486 conn(0x7fe8c80b0fb0 msgr2=0x7fe8c80cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 shutdown_connections 2024-07-17T06:55:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:24.958+0000 7fe8ce200640 1 -- 172.21.15.44:0/125705486 wait complete. 2024-07-17T06:55:24.978 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-17T06:55:24.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:55:24.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:55:24.979 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-07-17T06:55:24.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:24.979 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T06:55:25.032 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T06:55:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T06:55:25.034 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T06:55:25.035 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.036 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:55:25.120 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T06:55:25.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.124 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T06:55:25.175 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T06:55:25.175 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:55:25.175 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T06:55:25.175 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T06:55:25.175 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T06:55:25.176 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:25.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:25.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:25.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.395 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:55:25.441 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T06:55:25.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.442 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-07-17T06:55:25.536 INFO:teuthology.orchestra.run.smithi044.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-07-17T06:55:25.538 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:25.538 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:25.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T06:55:25.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-07-17T06:55:25.538 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:25.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-07-17T06:55:25.800 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 8e:be:a8:f3:d2:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::8cbe:a8ff:fef3:d228/64 scope link 2024-07-17T06:55:25.801 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:25.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 8e:be:a8:f3:d2:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-17T06:55:25.888 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.889 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::8cbe:a8ff:fef3:d228/64 scope link 2024-07-17T06:55:25.889 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:25.892 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-07-17T06:55:25.892 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:25.892 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:25.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-07-17T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-07-17T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-07-17T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-07-17T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-07-17T06:55:25.893 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:26.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:26.186 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:26.186 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:55:26.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.1 up 2024-07-17T06:55:26.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.1 master ceph-brx 2024-07-17T06:55:26.187 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:55:26.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:26.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:26.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:26.403 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-07-17T06:55:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-07-17T06:55:26.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:26.448 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-07-17T06:55:26.545 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T06:55:26.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:26.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-07-17T06:55:26.679 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-07-17T06:55:26.679 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-07-17T06:55:26.679 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:26.703 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-07-17T06:55:26.703 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-07-17T06:55:26.703 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:26.703 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:26.703 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:26.735 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-07-17T06:55:26.735 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-07-17T06:55:26.735 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:55:26.736 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:26.736 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:26.736 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T06:55:26.736 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T06:55:26.736 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T06:55:26.742 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-07-17T06:55:26.742 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T06:55:26.742 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T06:55:26.742 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T06:55:26.742 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T06:55:26.748 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T06:55:26.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:26.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-07-17T06:55:26.825 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:26.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:26.950 INFO:teuthology.orchestra.run.smithi044.stdout:client4532 2024-07-17T06:55:26.950 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:26.951 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:27.002 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:27.169 INFO:teuthology.orchestra.run.smithi044.stdout:client4532 2024-07-17T06:55:27.169 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:27.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:27.291 INFO:teuthology.orchestra.run.smithi044.stdout:client4532 2024-07-17T06:55:27.291 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4797a2db-a688-4e60-bfb8-d8a9e86b84a3.client4532/status') 2024-07-17T06:55:27.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:27.423 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.4532 (3)172.21.15.44:0/2486258163 2024-07-17T06:55:27.423 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-07-17T06:55:27.424 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-07-17T06:55:27.424 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-07-17T06:55:27.424 INFO:teuthology.orchestra.run.smithi044.stderr:70 bytes copied, 0.000106724 s, 656 kB/s 2024-07-17T06:55:27.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:27.438 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-07-17T06:55:27.438 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-17T06:55:27.438 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-17T06:55:27.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 -- 172.21.15.44:0/3696675894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 msgr2=0x7f3c480b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 --2- 172.21.15.44:0/3696675894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c480b3870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c480cfdb0 tx=0x7f3c3802a560 comp rx=0 tx=0).stop 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 -- 172.21.15.44:0/3696675894 shutdown_connections 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 --2- 172.21.15.44:0/3696675894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c480ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 --2- 172.21.15.44:0/3696675894 >> v2:172.21.15.175:3300/0 conn(0x7f3c480cf920 0x7f3c480ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 --2- 172.21.15.44:0/3696675894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c480b3870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 -- 172.21.15.44:0/3696675894 >> 172.21.15.44:0/3696675894 conn(0x7f3c480b0e80 msgr2=0x7f3c480b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.651+0000 7f3c47200640 1 -- 172.21.15.44:0/3696675894 shutdown_connections 2024-07-17T06:55:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.652+0000 7f3c47200640 1 -- 172.21.15.44:0/3696675894 wait complete. 2024-07-17T06:55:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.653+0000 7f3c47200640 1 Processor -- start 2024-07-17T06:55:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.653+0000 7f3c47200640 1 -- start start 2024-07-17T06:55:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.654+0000 7f3c47200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c4806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.654+0000 7f3c47200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c47200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 0x7f3c4815d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c47200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480ae6f0 con 0x7f3c480cf540 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c47200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480ae570 con 0x7f3c480cf920 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c47200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480ae870 con 0x7f3c480b3db0 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43648/0 (socket says 172.21.15.44:43648) 2024-07-17T06:55:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 -- 172.21.15.44:0/3965967099 learned_addr learned my addr 172.21.15.44:0/3965967099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c45e00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c4806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c46800640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 0x7f3c4815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 -- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 msgr2=0x7f3c4806ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c4806ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 -- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 msgr2=0x7f3c4815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 0x7f3c4815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c3fe00640 1 -- 172.21.15.44:0/3965967099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480730d0 con 0x7f3c480cf540 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c45e00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c4806ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.655+0000 7f3c46800640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 0x7f3c4815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c3fe00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000d770 tx=0x7f3c3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30017070 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c30009d20 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30005280 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c480733b0 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.656+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c48073970 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.657+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c48155b40 con 0x7f3c480cf540 2024-07-17T06:55:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.658+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f3c30005420 con 0x7f3c480cf540 2024-07-17T06:55:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.659+0000 7f3c3e000640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 0x7f3c24079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.660+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c300987f0 con 0x7f3c480cf540 2024-07-17T06:55:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.660+0000 7f3c45e00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 0x7f3c24079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:27.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.661+0000 7f3c45e00640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 0x7f3c24079070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c38033000 tx=0x7f3c38004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.664+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3c48155b40 con 0x7f3c480cf540 2024-07-17T06:55:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.916+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c480b3870 con 0x7f3c480cf540 2024-07-17T06:55:27.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.917+0000 7f3c3e000640 1 -- 172.21.15.44:0/3965967099 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f3c300618e0 con 0x7f3c480cf540 2024-07-17T06:55:27.917 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.919+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 msgr2=0x7f3c24079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.919+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 0x7f3c24079070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c38033000 tx=0x7f3c38004730 comp rx=0 tx=0).stop 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.919+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 msgr2=0x7f3c4815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.919+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000d770 tx=0x7f3c3000dc30 comp rx=0 tx=0).stop 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 shutdown_connections 2024-07-17T06:55:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c480cf920 0x7f3c4815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c480cf540 0x7f3c4815cdc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f3c24076bb0 0x7f3c24079070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 --2- 172.21.15.44:0/3965967099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c480b3db0 0x7f3c4806ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 >> 172.21.15.44:0/3965967099 conn(0x7f3c480b0e80 msgr2=0x7f3c481401a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 shutdown_connections 2024-07-17T06:55:27.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:27.920+0000 7f3c47200640 1 -- 172.21.15.44:0/3965967099 wait complete. 2024-07-17T06:55:27.938 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-17T06:55:27.938 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:55:27.938 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-07-17T06:55:27.938 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-17T06:55:27.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-17T06:55:27.939 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-17T06:55:27.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-17T06:55:27.939 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:55:27.939 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-17T06:55:27.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:27.939 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-07-17T06:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42525sec preferred_lft 42525sec 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-07-17T06:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:27.999 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-17T06:55:27.999 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-17T06:55:28.000 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:28.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:28.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:28.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:28.479 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:55:28.541 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-07-17T06:55:28.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:28.545 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-07-17T06:55:28.598 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-07-17T06:55:28.598 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:55:28.598 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-17T06:55:28.598 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-17T06:55:28.598 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-17T06:55:28.599 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:28.694 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:29.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:29.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:29.389 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T06:55:29.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:29.440 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-07-17T06:55:29.490 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:29.490 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:29.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-17T06:55:29.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-07-17T06:55:29.490 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:29.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:29.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:29.672 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-07-17T06:55:29.672 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:29.672 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:29.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-07-17T06:55:29.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-17T06:55:29.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-07-17T06:55:29.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-07-17T06:55:29.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-07-17T06:55:29.673 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:29.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:29.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:29.976 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:29.977 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:29.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-07-17T06:55:29.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-07-17T06:55:29.977 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:30.065 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:30.159 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:30.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:30.169 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-17T06:55:30.213 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-17T06:55:30.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:30.213 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-17T06:55:30.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T06:55:30.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:30.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-17T06:55:30.443 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-17T06:55:30.444 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-17T06:55:30.444 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:31.150 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-17T06:55:31.153 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-17T06:55:31.153 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:31.153 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:31.153 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T06:55:31.187 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T06:55:31.192 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-07-17T06:55:31.192 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-07-17T06:55:31.192 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T06:55:31.192 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T06:55:31.192 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T06:55:31.196 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T06:55:31.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:31.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-17T06:55:31.278 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:31.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:31.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:31.394 INFO:teuthology.orchestra.run.smithi175.stdout:client4639 2024-07-17T06:55:31.394 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:31.394 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:31.444 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:31.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:31.595 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:31.605 INFO:teuthology.orchestra.run.smithi175.stdout:client4639 2024-07-17T06:55:31.606 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:31.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:31.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:31.721 INFO:teuthology.orchestra.run.smithi175.stdout:client4639 2024-07-17T06:55:31.722 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4797a2db-a688-4e60-bfb8-d8a9e86b84a3.client4639/status') 2024-07-17T06:55:31.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:31.854 INFO:teuthology.orchestra.run.smithi175.stdout:instance: client.4639 (3)192.168.0.1:0/2039953243 2024-07-17T06:55:31.855 INFO:teuthology.orchestra.run.smithi175.stdout:blocklisted: false 2024-07-17T06:55:31.855 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2024-07-17T06:55:31.855 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2024-07-17T06:55:31.855 INFO:teuthology.orchestra.run.smithi175.stderr:69 bytes copied, 0.000100196 s, 689 kB/s 2024-07-17T06:55:31.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:31.868 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-07-17T06:55:31.868 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-17T06:55:31.868 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-07-17T06:55:31.868 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:55:32.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.088+0000 7ff1bee00640 1 -- 172.21.15.44:0/811700361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b9060 msgr2=0x7ff1b815d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.088+0000 7ff1bee00640 1 --2- 172.21.15.44:0/811700361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b9060 0x7ff1b815d9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009c10 tx=0x7ff1b002a940 comp rx=0 tx=0).stop 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.088+0000 7ff1bee00640 1 -- 172.21.15.44:0/811700361 shutdown_connections 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.088+0000 7ff1bee00640 1 --2- 172.21.15.44:0/811700361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b9060 0x7ff1b815d9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.088+0000 7ff1bee00640 1 -- 172.21.15.44:0/811700361 >> 172.21.15.44:0/811700361 conn(0x7ff1b80cfec0 msgr2=0x7ff1b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.089+0000 7ff1bee00640 1 -- 172.21.15.44:0/811700361 shutdown_connections 2024-07-17T06:55:32.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.089+0000 7ff1bee00640 1 -- 172.21.15.44:0/811700361 wait complete. 2024-07-17T06:55:32.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.090+0000 7ff1bee00640 1 Processor -- start 2024-07-17T06:55:32.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.090+0000 7ff1bee00640 1 -- start start 2024-07-17T06:55:32.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8c80 0x7ff1b815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b80ce5c0 0x7ff1b81f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b815e2a0 con 0x7ff1b80b9060 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b815e120 con 0x7ff1b80ce5c0 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.091+0000 7ff1bee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b815e420 con 0x7ff1b80b8c80 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33460/0 (socket says 172.21.15.44:33460) 2024-07-17T06:55:32.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 -- 172.21.15.44:0/2096718425 learned_addr learned my addr 172.21.15.44:0/2096718425 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bda00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8c80 0x7ff1b815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1be400640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b80ce5c0 0x7ff1b81f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:32.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 -- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8c80 msgr2=0x7ff1b815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8c80 0x7ff1b815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 -- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b80ce5c0 msgr2=0x7ff1b81f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b80ce5c0 0x7ff1b81f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.092+0000 7ff1bd000640 1 -- 172.21.15.44:0/2096718425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b80caa50 con 0x7ff1b80b9060 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1bd000640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4001eb0 tx=0x7ff1a400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011790 con 0x7ff1b80b9060 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1a4017070 con 0x7ff1b80b9060 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011d00 con 0x7ff1b80b9060 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81f4860 con 0x7ff1b80b9060 2024-07-17T06:55:32.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.093+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b80aa1c0 con 0x7ff1b80b9060 2024-07-17T06:55:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.094+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b80b4ec0 con 0x7ff1b80b9060 2024-07-17T06:55:32.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.096+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ff1a401c460 con 0x7ff1b80b9060 2024-07-17T06:55:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.097+0000 7ff1b5600640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 0x7ff198079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.097+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff1a40984e0 con 0x7ff1b80b9060 2024-07-17T06:55:32.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.098+0000 7ff1bda00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 0x7ff198079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:32.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.099+0000 7ff1bda00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 0x7ff198079070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:32.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.101+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff1a40615d0 con 0x7ff1b80b9060 2024-07-17T06:55:32.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.354+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1b80b5d70 con 0x7ff1b80b9060 2024-07-17T06:55:32.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.354+0000 7ff1b5600640 1 -- 172.21.15.44:0/2096718425 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff1a4060ed0 con 0x7ff1b80b9060 2024-07-17T06:55:32.354 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-17T06:55:32.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 msgr2=0x7ff198079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:32.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 0x7ff198079070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 msgr2=0x7ff1b80ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4001eb0 tx=0x7ff1a400c820 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 shutdown_connections 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1b80ce5c0 0x7ff1b81f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1b80b9060 0x7ff1b80ca510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff198076bb0 0x7ff198079070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 --2- 172.21.15.44:0/2096718425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1b80b8c80 0x7ff1b815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 >> 172.21.15.44:0/2096718425 conn(0x7ff1b80cfec0 msgr2=0x7ff1b815eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.357+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 shutdown_connections 2024-07-17T06:55:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:32.358+0000 7ff1bee00640 1 -- 172.21.15.44:0/2096718425 wait complete. 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:55:32.379 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-07-17T06:55:32.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:32.380 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-07-17T06:55:32.436 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:32.436 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:32.436 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:32.436 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.436 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-07-17T06:55:32.437 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 52:f8:8a:75:a3:05 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::80b9:d6ff:fe1b:c6ea/64 scope link 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 52:f8:8a:75:a3:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::50f8:8aff:fe75:a305/64 scope link 2024-07-17T06:55:32.438 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:32.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:32.439 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:55:32.519 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-07-17T06:55:32.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:32.524 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-07-17T06:55:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-07-17T06:55:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:55:32.573 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-17T06:55:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-17T06:55:32.574 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-17T06:55:32.575 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:32.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:32.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:32.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:32.782 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T06:55:32.834 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-07-17T06:55:32.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:32.836 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-07-17T06:55:32.885 INFO:teuthology.orchestra.run.smithi175.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-07-17T06:55:32.886 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:32.886 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:32.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-07-17T06:55:32.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-07-17T06:55:32.886 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:32.986 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:33.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:33.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:33.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-07-17T06:55:33.142 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.142 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether be:88:e3:b3:6c:d0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::bc88:e3ff:feb3:6cd0/64 scope link 2024-07-17T06:55:33.143 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:33.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-07-17T06:55:33.224 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:55:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:55:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:55:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether be:88:e3:b3:6c:d0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::bc88:e3ff:feb3:6cd0/64 scope link 2024-07-17T06:55:33.225 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:55:33.228 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-07-17T06:55:33.229 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:33.332 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:33.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:33.502 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:55:33.502 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-07-17T06:55:33.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.1 up 2024-07-17T06:55:33.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.1 master ceph-brx 2024-07-17T06:55:33.502 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-07-17T06:55:33.601 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:33.697 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:33.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:33.707 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-07-17T06:55:33.754 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-07-17T06:55:33.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:55:33.754 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-07-17T06:55:33.844 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T06:55:33.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:33.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-07-17T06:55:33.950 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-07-17T06:55:33.950 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-07-17T06:55:33.950 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:33.970 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-07-17T06:55:33.970 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-07-17T06:55:33.970 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:33.970 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:33.970 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:55:33.996 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T06:55:33.997 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T06:55:33.997 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T06:55:33.999 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-07-17T06:55:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-07-17T06:55:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T06:55:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T06:55:33.999 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T06:55:34.003 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T06:55:34.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:34.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-07-17T06:55:34.072 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:34.174 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:34.178 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:34.188 INFO:teuthology.orchestra.run.smithi175.stdout:client4544 2024-07-17T06:55:34.188 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:34.188 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:34.235 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:34.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:34.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:34.392 INFO:teuthology.orchestra.run.smithi175.stdout:client4544 2024-07-17T06:55:34.392 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:55:34.494 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:55:34.498 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:34.508 INFO:teuthology.orchestra.run.smithi175.stdout:client4544 2024-07-17T06:55:34.509 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4797a2db-a688-4e60-bfb8-d8a9e86b84a3.client4544/status') 2024-07-17T06:55:34.608 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:55:34.637 INFO:teuthology.orchestra.run.smithi175.stdout:instance: client.4544 (3)192.168.0.2:0/4107089189 2024-07-17T06:55:34.638 INFO:teuthology.orchestra.run.smithi175.stdout:blocklisted: false 2024-07-17T06:55:34.638 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2024-07-17T06:55:34.638 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2024-07-17T06:55:34.638 INFO:teuthology.orchestra.run.smithi175.stderr:69 bytes copied, 8.9702e-05 s, 769 kB/s 2024-07-17T06:55:34.641 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-17T06:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:55:34.651 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-07-17T06:55:34.684 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2024-07-17T06:55:34.685 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T06:55:34.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 -- 172.21.15.44:0/4268637896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c15a040 msgr2=0x7fbc1c0ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 --2- 172.21.15.44:0/4268637896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c15a040 0x7fbc1c0ac3e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c070a70 tx=0x7fbc0402ade0 comp rx=0 tx=0).stop 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 -- 172.21.15.44:0/4268637896 shutdown_connections 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 --2- 172.21.15.44:0/4268637896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c15a040 0x7fbc1c0ac3e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 --2- 172.21.15.44:0/4268637896 >> v2:172.21.15.175:3300/0 conn(0x7fbc1c153a50 0x7fbc1c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 -- 172.21.15.44:0/4268637896 >> 172.21.15.44:0/4268637896 conn(0x7fbc1c0b0fb0 msgr2=0x7fbc1c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 -- 172.21.15.44:0/4268637896 shutdown_connections 2024-07-17T06:55:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.918+0000 7fbc22400640 1 -- 172.21.15.44:0/4268637896 wait complete. 2024-07-17T06:55:34.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.919+0000 7fbc22400640 1 Processor -- start 2024-07-17T06:55:34.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.920+0000 7fbc22400640 1 -- start start 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0ac920 0x7fbc1c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15a040 0x7fbc1c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c153e30 con 0x7fbc1c153a50 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c06f1b0 con 0x7fbc1c0ac920 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc22400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c153fb0 con 0x7fbc1c15a040 2024-07-17T06:55:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:34.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33482/0 (socket says 172.21.15.44:33482) 2024-07-17T06:55:34.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 -- 172.21.15.44:0/79762136 learned_addr learned my addr 172.21.15.44:0/79762136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:34.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 -- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15a040 msgr2=0x7fbc1c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15a040 0x7fbc1c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 -- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0ac920 msgr2=0x7fbc1c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0ac920 0x7fbc1c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.921+0000 7fbc13e00640 1 -- 172.21.15.44:0/79762136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0b23e0 con 0x7fbc1c153a50 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc13e00640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc04035040 tx=0x7fbc04009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04040020 con 0x7fbc1c153a50 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc0402b930 con 0x7fbc1c153a50 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0402bd20 con 0x7fbc1c153a50 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c098b40 con 0x7fbc1c153a50 2024-07-17T06:55:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.922+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c0b2a50 con 0x7fbc1c153a50 2024-07-17T06:55:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.923+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c159b00 con 0x7fbc1c153a50 2024-07-17T06:55:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.924+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fbc04043020 con 0x7fbc1c153a50 2024-07-17T06:55:34.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.925+0000 7fbc13400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 0x7fbbf8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:34.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.925+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fbc040b8c70 con 0x7fbc1c153a50 2024-07-17T06:55:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.926+0000 7fbc1b400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 0x7fbbf8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.927+0000 7fbc1b400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 0x7fbbf8079070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009710 tx=0x7fbc0c006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:34.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:34.930+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbc0404a030 con 0x7fbc1c153a50 2024-07-17T06:55:35.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.139+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc1c0af030 con 0x7fbc1c153a50 2024-07-17T06:55:35.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.142+0000 7fbc13400640 1 -- 172.21.15.44:0/79762136 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7fbc1c15acc0 con 0x7fbc1c153a50 2024-07-17T06:55:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:35.141 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:34.800937+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 msgr2=0x7fbbf8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 0x7fbbf8079070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009710 tx=0x7fbc0c006c90 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 msgr2=0x7fbc1c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc04035040 tx=0x7fbc04009850 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 shutdown_connections 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbbf8076bb0 0x7fbbf8079070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc1c15a040 0x7fbc1c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc1c153a50 0x7fbc1c0b2e10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 --2- 172.21.15.44:0/79762136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc1c0ac920 0x7fbc1c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.145+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 >> 172.21.15.44:0/79762136 conn(0x7fbc1c0b0fb0 msgr2=0x7fbc1c15d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.146+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 shutdown_connections 2024-07-17T06:55:35.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.146+0000 7fbc22400640 1 -- 172.21.15.44:0/79762136 wait complete. 2024-07-17T06:55:35.166 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-17T06:54:46.375450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-17T06:54:47.702732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-17T06:54:51.319737+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-17T06:55:35.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.384+0000 7f98cb800640 1 -- 172.21.15.44:0/3507482019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40ac8a0 msgr2=0x7f98c40b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.384+0000 7f98cb800640 1 --2- 172.21.15.44:0/3507482019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40ac8a0 0x7f98c40b21b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009830 tx=0x7f98b402a510 comp rx=0 tx=0).stop 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.385+0000 7f98cb800640 1 -- 172.21.15.44:0/3507482019 shutdown_connections 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.385+0000 7f98cb800640 1 --2- 172.21.15.44:0/3507482019 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40ac8a0 0x7f98c40b21b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.385+0000 7f98cb800640 1 -- 172.21.15.44:0/3507482019 >> 172.21.15.44:0/3507482019 conn(0x7f98c40b0eb0 msgr2=0x7f98c40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.385+0000 7f98cb800640 1 -- 172.21.15.44:0/3507482019 shutdown_connections 2024-07-17T06:55:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.385+0000 7f98cb800640 1 -- 172.21.15.44:0/3507482019 wait complete. 2024-07-17T06:55:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.386+0000 7f98cb800640 1 Processor -- start 2024-07-17T06:55:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.387+0000 7f98cb800640 1 -- start start 2024-07-17T06:55:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 0x7f98c4151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c415a280 0x7f98c41f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98c415b320 con 0x7f98c40b1af0 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98c406f1d0 con 0x7f98c40ac8a0 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98cb800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98c415b4a0 con 0x7f98c415a280 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33506/0 (socket says 172.21.15.44:33506) 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 -- 172.21.15.44:0/80228643 learned_addr learned my addr 172.21.15.44:0/80228643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c8e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 0x7f98c4151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c9800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c415a280 0x7f98c41f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 -- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c415a280 msgr2=0x7f98c41f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c415a280 0x7f98c41f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 -- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 msgr2=0x7f98c4151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 0x7f98c4151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.388+0000 7f98c3e00640 1 -- 172.21.15.44:0/80228643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c4073c30 con 0x7f98c40b1af0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98c8e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 0x7f98c4151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98c3e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98b4000c00 tx=0x7f98b4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4040070 con 0x7f98c40b1af0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98b4003af0 con 0x7f98c40b1af0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4045460 con 0x7f98c40b1af0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c4157330 con 0x7f98c40b1af0 2024-07-17T06:55:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.389+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c41578f0 con 0x7f98c40b1af0 2024-07-17T06:55:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.390+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c4157130 con 0x7f98c40b1af0 2024-07-17T06:55:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.392+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f98b4039040 con 0x7f98c40b1af0 2024-07-17T06:55:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.393+0000 7f98c0c00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 0x7f98a80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.393+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f98b40b9d90 con 0x7f98c40b1af0 2024-07-17T06:55:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.394+0000 7f98c8e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 0x7f98a80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.395+0000 7f98c8e00640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 0x7f98a80790a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007fd0 tx=0x7f98b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.397+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98b4082db0 con 0x7f98c40b1af0 2024-07-17T06:55:35.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.611+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98c40b21b0 con 0x7f98c40b1af0 2024-07-17T06:55:35.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.612+0000 7f98c0c00640 1 -- 172.21.15.44:0/80228643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f98b40826b0 con 0x7f98c40b1af0 2024-07-17T06:55:35.611 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 msgr2=0x7f98a80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 0x7f98a80790a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007fd0 tx=0x7f98b8009040 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 msgr2=0x7f98c414e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98b4000c00 tx=0x7f98b4032d50 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 shutdown_connections 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f98a8076be0 0x7f98a80790a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c415a280 0x7f98c41f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c40b1af0 0x7f98c414e730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 --2- 172.21.15.44:0/80228643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c40ac8a0 0x7f98c4151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 >> 172.21.15.44:0/80228643 conn(0x7f98c40b0eb0 msgr2=0x7f98c40ad190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.615+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 shutdown_connections 2024-07-17T06:55:35.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.616+0000 7f98cb800640 1 -- 172.21.15.44:0/80228643 wait complete. 2024-07-17T06:55:35.634 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-17T06:55:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 -- 172.21.15.44:0/2788597958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 msgr2=0x7f53f00d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 --2- 172.21.15.44:0/2788597958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f00d0aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005fd0 tx=0x7f53e002aa90 comp rx=0 tx=0).stop 2024-07-17T06:55:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 -- 172.21.15.44:0/2788597958 shutdown_connections 2024-07-17T06:55:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 --2- 172.21.15.44:0/2788597958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f00d0aa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 --2- 172.21.15.44:0/2788597958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 0x7f53f006e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 -- 172.21.15.44:0/2788597958 >> 172.21.15.44:0/2788597958 conn(0x7f53f00b5560 msgr2=0x7f53f00b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 -- 172.21.15.44:0/2788597958 shutdown_connections 2024-07-17T06:55:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.837+0000 7f53efe00640 1 -- 172.21.15.44:0/2788597958 wait complete. 2024-07-17T06:55:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.839+0000 7f53efe00640 1 Processor -- start 2024-07-17T06:55:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.839+0000 7f53efe00640 1 -- start start 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 0x7f53f01f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 0x7f53f0151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53f006c650 con 0x7f53f006f570 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53f015e840 con 0x7f53f015e1c0 2024-07-17T06:55:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.840+0000 7f53efe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53f006c7d0 con 0x7f53f006e570 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33532/0 (socket says 172.21.15.44:33532) 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 -- 172.21.15.44:0/3452092070 learned_addr learned my addr 172.21.15.44:0/3452092070 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53eea00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 0x7f53f01f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 -- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 msgr2=0x7f53f01f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 0x7f53f01f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ef400640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 0x7f53f0151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 -- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 msgr2=0x7f53f0151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 0x7f53f0151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ee000640 1 -- 172.21.15.44:0/3452092070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0144750 con 0x7f53f006f570 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.841+0000 7f53ef400640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 0x7f53f0151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53ee000640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000f80 tx=0x7f53e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0046070 con 0x7f53f006f570 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53e0004250 con 0x7f53f006f570 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f01f46a0 con 0x7f53f006f570 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004680 con 0x7f53f006f570 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.842+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01f4c60 con 0x7f53f006f570 2024-07-17T06:55:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.843+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f006cc30 con 0x7f53f006f570 2024-07-17T06:55:35.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.844+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f53e0049070 con 0x7f53f006f570 2024-07-17T06:55:35.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.845+0000 7f53e6a00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 0x7f53cc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:35.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.845+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f53e00b9590 con 0x7f53f006f570 2024-07-17T06:55:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.846+0000 7f53eea00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 0x7f53cc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.847+0000 7f53eea00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 0x7f53cc079070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005e10 tx=0x7f53d8005da0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:35.850+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f53e0032030 con 0x7f53f006f570 2024-07-17T06:55:36.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.058+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53f015a5c0 con 0x7f53f006f570 2024-07-17T06:55:36.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.059+0000 7f53e6a00640 1 -- 172.21.15.44:0/3452092070 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f53e0004820 con 0x7f53f006f570 2024-07-17T06:55:36.057 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 msgr2=0x7f53cc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 0x7f53cc079070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005e10 tx=0x7f53d8005da0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 msgr2=0x7f53f01f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000f80 tx=0x7f53e0003f10 comp rx=0 tx=0).stop 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 shutdown_connections 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f015e1c0 0x7f53f0151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53f006f570 0x7f53f01f3de0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f53cc076bb0 0x7f53cc079070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 --2- 172.21.15.44:0/3452092070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f006e570 0x7f53f01f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 >> 172.21.15.44:0/3452092070 conn(0x7f53f00b5560 msgr2=0x7f53f015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 shutdown_connections 2024-07-17T06:55:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.062+0000 7f53efe00640 1 -- 172.21.15.44:0/3452092070 wait complete. 2024-07-17T06:55:36.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 -- 172.21.15.44:0/2933553453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806f4d0 msgr2=0x7f874815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 --2- 172.21.15.44:0/2933553453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806f4d0 0x7f874815a450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87480cb730 tx=0x7f873802b880 comp rx=0 tx=0).stop 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 -- 172.21.15.44:0/2933553453 shutdown_connections 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 --2- 172.21.15.44:0/2933553453 >> v2:172.21.15.175:3300/0 conn(0x7f87481503c0 0x7f87480ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 --2- 172.21.15.44:0/2933553453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806f4d0 0x7f874815a450 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 -- 172.21.15.44:0/2933553453 >> 172.21.15.44:0/2933553453 conn(0x7f87480cf0d0 msgr2=0x7f87480cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 -- 172.21.15.44:0/2933553453 shutdown_connections 2024-07-17T06:55:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.298+0000 7f8747e00640 1 -- 172.21.15.44:0/2933553453 wait complete. 2024-07-17T06:55:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.300+0000 7f8747e00640 1 Processor -- start 2024-07-17T06:55:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.300+0000 7f8747e00640 1 -- start start 2024-07-17T06:55:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 0x7f87481e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 0x7f87481f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f874806d850 con 0x7f874806d420 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87480ac930 con 0x7f87481503c0 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f874806d9d0 con 0x7f874806f4d0 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33538/0 (socket says 172.21.15.44:33538) 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 -- 172.21.15.44:0/2068356184 learned_addr learned my addr 172.21.15.44:0/2068356184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 -- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 msgr2=0x7f87481e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746000640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 0x7f87481e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8747400640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 0x7f87481f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 0x7f87481e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 -- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 msgr2=0x7f87481f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.301+0000 7f8746a00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 0x7f87481f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8746a00640 1 -- 172.21.15.44:0/2068356184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87480cd030 con 0x7f874806d420 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8746000640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 0x7f87481e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8747400640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 0x7f87481f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8746a00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8738034040 tx=0x7f8738003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738040030 con 0x7f874806d420 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87380047a0 con 0x7f874806d420 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87380049f0 con 0x7f874806d420 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87480cd310 con 0x7f874806d420 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.302+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87481f79c0 con 0x7f874806d420 2024-07-17T06:55:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.304+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f874815a450 con 0x7f874806d420 2024-07-17T06:55:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.305+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f8738043030 con 0x7f874806d420 2024-07-17T06:55:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.306+0000 7f873ea00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 0x7f87280790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.306+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f87380b9550 con 0x7f874806d420 2024-07-17T06:55:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.306+0000 7f8746000640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 0x7f87280790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.308+0000 7f8746000640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 0x7f87280790a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8748159440 tx=0x7f8734005c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.310+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f873802f050 con 0x7f874806d420 2024-07-17T06:55:36.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.521+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f874814fb00 con 0x7f874806d420 2024-07-17T06:55:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.522+0000 7f873ea00640 1 -- 172.21.15.44:0/2068356184 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f8738082600 con 0x7f874806d420 2024-07-17T06:55:36.521 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-07-17T06:55:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 msgr2=0x7f87280790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 0x7f87280790a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8748159440 tx=0x7f8734005c90 comp rx=0 tx=0).stop 2024-07-17T06:55:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 msgr2=0x7f874814d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8738034040 tx=0x7f8738003970 comp rx=0 tx=0).stop 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 shutdown_connections 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87481503c0 0x7f87481f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8728076be0 0x7f87280790a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f874806f4d0 0x7f87481e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 --2- 172.21.15.44:0/2068356184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f874806d420 0x7f874814d360 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.525+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 >> 172.21.15.44:0/2068356184 conn(0x7f87480cf0d0 msgr2=0x7f87480cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.526+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 shutdown_connections 2024-07-17T06:55:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.526+0000 7f8747e00640 1 -- 172.21.15.44:0/2068356184 wait complete. 2024-07-17T06:55:36.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-17T06:55:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 -- 172.21.15.44:0/1673546235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 msgr2=0x7fc2e815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 --2- 172.21.15.44:0/1673546235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815d510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc005fd0 tx=0x7fc2dc02a5d0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 -- 172.21.15.44:0/1673546235 shutdown_connections 2024-07-17T06:55:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 --2- 172.21.15.44:0/1673546235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815d510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 -- 172.21.15.44:0/1673546235 >> 172.21.15.44:0/1673546235 conn(0x7fc2e80b4ff0 msgr2=0x7fc2e80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 -- 172.21.15.44:0/1673546235 shutdown_connections 2024-07-17T06:55:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.753+0000 7fc2ef200640 1 -- 172.21.15.44:0/1673546235 wait complete. 2024-07-17T06:55:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.755+0000 7fc2ef200640 1 Processor -- start 2024-07-17T06:55:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.755+0000 7fc2ef200640 1 -- start start 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2e80ce690 0x7fc2e814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 0x7fc2e815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8168860 con 0x7fc2e815d130 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81686e0 con 0x7fc2e8161be0 2024-07-17T06:55:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e81689e0 con 0x7fc2e80ce690 2024-07-17T06:55:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33552/0 (socket says 172.21.15.44:33552) 2024-07-17T06:55:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 -- 172.21.15.44:0/3804009509 learned_addr learned my addr 172.21.15.44:0/3804009509 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ede00640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2e80ce690 0x7fc2e814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ee800640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 0x7fc2e815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 -- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2e80ce690 msgr2=0x7fc2e814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2e80ce690 0x7fc2e814f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.757+0000 7fc2ed400640 1 -- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 msgr2=0x7fc2e815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ed400640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 0x7fc2e815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ed400640 1 -- 172.21.15.44:0/3804009509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e815be50 con 0x7fc2e815d130 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ee800640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 0x7fc2e815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ed400640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c430 tx=0x7fc2d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011790 con 0x7fc2e815d130 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2d8017070 con 0x7fc2e815d130 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011d00 con 0x7fc2e815d130 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e815c130 con 0x7fc2e815d130 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.758+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e815c6f0 con 0x7fc2e815d130 2024-07-17T06:55:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.760+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e815c8c0 con 0x7fc2e815d130 2024-07-17T06:55:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.761+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc2d801c460 con 0x7fc2e815d130 2024-07-17T06:55:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.762+0000 7fc2e2000640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 0x7fc2cc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.762+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fc2d8098670 con 0x7fc2e815d130 2024-07-17T06:55:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.763+0000 7fc2ede00640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 0x7fc2cc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.764+0000 7fc2ede00640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 0x7fc2cc079070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc033000 tx=0x7fc2dc004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:36.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.766+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc2d8061720 con 0x7fc2e815d130 2024-07-17T06:55:36.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.980+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2e80b3800 con 0x7fc2e815d130 2024-07-17T06:55:36.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.981+0000 7fc2e2000640 1 -- 172.21.15.44:0/3804009509 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc2d8061020 con 0x7fc2e815d130 2024-07-17T06:55:36.980 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-07-17T06:55:36.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 msgr2=0x7fc2cc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 0x7fc2cc079070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc033000 tx=0x7fc2dc004730 comp rx=0 tx=0).stop 2024-07-17T06:55:36.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 msgr2=0x7fc2e815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:36.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c430 tx=0x7fc2d800c8f0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 shutdown_connections 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2e8161be0 0x7fc2e815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2e815d130 0x7fc2e815b500 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc2cc076bb0 0x7fc2cc079070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.984+0000 7fc2ef200640 1 --2- 172.21.15.44:0/3804009509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2e80ce690 0x7fc2e814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:36.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.985+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 >> 172.21.15.44:0/3804009509 conn(0x7fc2e80b4ff0 msgr2=0x7fc2e80b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:36.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.985+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 shutdown_connections 2024-07-17T06:55:36.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:36.985+0000 7fc2ef200640 1 -- 172.21.15.44:0/3804009509 wait complete. 2024-07-17T06:55:37.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 -- 172.21.15.44:0/3447808740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 msgr2=0x7f28f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 --2- 172.21.15.44:0/3447808740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f28f000b9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f290400c290 tx=0x7f28f0005830 comp rx=0 tx=0).stop 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 -- 172.21.15.44:0/3447808740 shutdown_connections 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 --2- 172.21.15.44:0/3447808740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f28f000b9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 -- 172.21.15.44:0/3447808740 >> 172.21.15.44:0/3447808740 conn(0x7f290c0cfec0 msgr2=0x7f290c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:37.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 -- 172.21.15.44:0/3447808740 shutdown_connections 2024-07-17T06:55:37.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.219+0000 7f2912000640 1 -- 172.21.15.44:0/3447808740 wait complete. 2024-07-17T06:55:37.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.221+0000 7f2912000640 1 Processor -- start 2024-07-17T06:55:37.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.221+0000 7f2912000640 1 -- start start 2024-07-17T06:55:37.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 0x7f290c0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f290c1f4500 0x7f290c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f290c0b54c0 con 0x7f28f0005290 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f290c0b5340 con 0x7f290c0c8940 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.222+0000 7f2912000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f290c0b5640 con 0x7f290c1f4500 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33558/0 (socket says 172.21.15.44:33558) 2024-07-17T06:55:37.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 -- 172.21.15.44:0/2171226706 learned_addr learned my addr 172.21.15.44:0/2171226706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:37.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 -- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f290c1f4500 msgr2=0x7f290c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:37.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f290c1f4500 0x7f290c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 -- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 msgr2=0x7f290c0c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f290be00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 0x7f290c0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 0x7f290c0c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f2910c00640 1 -- 172.21.15.44:0/2171226706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c0c78a0 con 0x7f28f0005290 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.223+0000 7f290be00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 0x7f290c0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2910c00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00d630 tx=0x7f28fc00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc014070 con 0x7f28f0005290 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28fc00f970 con 0x7f28f0005290 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290c0c7b80 con 0x7f28f0005290 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc00fdc0 con 0x7f28f0005290 2024-07-17T06:55:37.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.224+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290c0c8140 con 0x7f28f0005290 2024-07-17T06:55:37.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.225+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c06dca0 con 0x7f28f0005290 2024-07-17T06:55:37.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.226+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f28fc017070 con 0x7f28f0005290 2024-07-17T06:55:37.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.227+0000 7f2908c00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 0x7f28ec0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:37.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.227+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f28fc098930 con 0x7f28f0005290 2024-07-17T06:55:37.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.228+0000 7f290be00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 0x7f28ec0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:37.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.229+0000 7f290be00640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 0x7f28ec0790a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f290c15f820 tx=0x7f28f80093e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:37.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.232+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f28fc061950 con 0x7f28f0005290 2024-07-17T06:55:37.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.527+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) -- 0x7f290c0b5d70 con 0x7f28f0005290 2024-07-17T06:55:37.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.835+0000 7f2908c00640 1 -- 172.21.15.44:0/2171226706 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v62) ==== 161+0+0 (secure 0 0 0) 0x7f28fc061250 con 0x7f28f0005290 2024-07-17T06:55:37.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 msgr2=0x7f28ec0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:37.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 0x7f28ec0790a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f290c15f820 tx=0x7f28f80093e0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 msgr2=0x7f290c15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00d630 tx=0x7f28fc00da30 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 shutdown_connections 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f28ec076be0 0x7f28ec0790a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f290c1f4500 0x7f290c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f290c0c8940 0x7f290c0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 --2- 172.21.15.44:0/2171226706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0005290 0x7f290c15a790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.838+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 >> 172.21.15.44:0/2171226706 conn(0x7f290c0cfec0 msgr2=0x7f290c0b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.839+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 shutdown_connections 2024-07-17T06:55:37.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:37.839+0000 7f2912000640 1 -- 172.21.15.44:0/2171226706 wait complete. 2024-07-17T06:55:37.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T06:55:38.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 -- 172.21.15.44:0/416354198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 msgr2=0x7f18bc15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 --2- 172.21.15.44:0/416354198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc15be70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009830 tx=0x7f18ac02a510 comp rx=0 tx=0).stop 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 -- 172.21.15.44:0/416354198 shutdown_connections 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 --2- 172.21.15.44:0/416354198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc15b910 0x7f18bc06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 --2- 172.21.15.44:0/416354198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc15be70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 -- 172.21.15.44:0/416354198 >> 172.21.15.44:0/416354198 conn(0x7f18bc06bf40 msgr2=0x7f18bc06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 -- 172.21.15.44:0/416354198 shutdown_connections 2024-07-17T06:55:38.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.079+0000 7f18c3000640 1 -- 172.21.15.44:0/416354198 wait complete. 2024-07-17T06:55:38.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.081+0000 7f18c3000640 1 Processor -- start 2024-07-17T06:55:38.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.081+0000 7f18c3000640 1 -- start start 2024-07-17T06:55:38.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18bc15b910 0x7f18bc0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc1f4160 0x7f18bc157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc06df90 con 0x7f18bc0aea80 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc06de10 con 0x7f18bc15b910 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.082+0000 7f18c3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc06e110 con 0x7f18bc1f4160 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33576/0 (socket says 172.21.15.44:33576) 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 -- 172.21.15.44:0/3040480408 learned_addr learned my addr 172.21.15.44:0/3040480408 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bb400640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18bc15b910 0x7f18bc0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18c1000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc1f4160 0x7f18bc157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 -- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc1f4160 msgr2=0x7f18bc157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc1f4160 0x7f18bc157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 -- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18bc15b910 msgr2=0x7f18bc0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18bc15b910 0x7f18bc0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.083+0000 7f18bbe00640 1 -- 172.21.15.44:0/3040480408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc1f4660 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18bbe00640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac040070 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18ac003af0 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac045460 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc1f4940 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.084+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1f4f00 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.085+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc071720 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.086+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f18ac039040 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.087+0000 7f18b3e00640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 0x7f189c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.087+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f18ac0b9d90 con 0x7f18bc0aea80 2024-07-17T06:55:38.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.088+0000 7f18bb400640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 0x7f189c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.089+0000 7f18bb400640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 0x7f189c079070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18bc15ae80 tx=0x7f18a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:38.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.092+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f18bc071720 con 0x7f18bc0aea80 2024-07-17T06:55:38.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.304+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f18bc06e5c0 con 0x7f18bc0aea80 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.305+0000 7f18b3e00640 1 -- 172.21.15.44:0/3040480408 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f18ac082db0 con 0x7f18bc0aea80 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T06:55:38.304 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T06:55:38.305 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T06:55:38.306 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.307+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 msgr2=0x7f189c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.307+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 0x7f189c079070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18bc15ae80 tx=0x7f18a8009040 comp rx=0 tx=0).stop 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.307+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 msgr2=0x7f18bc156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.307+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac032d50 comp rx=0 tx=0).stop 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 shutdown_connections 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f189c076bb0 0x7f189c079070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18bc1f4160 0x7f18bc157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18bc15b910 0x7f18bc0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 --2- 172.21.15.44:0/3040480408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18bc0aea80 0x7f18bc156130 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 >> 172.21.15.44:0/3040480408 conn(0x7f18bc06bf40 msgr2=0x7f18bc0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:38.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 shutdown_connections 2024-07-17T06:55:38.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.308+0000 7f18c3000640 1 -- 172.21.15.44:0/3040480408 wait complete. 2024-07-17T06:55:38.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 -- 172.21.15.44:0/1905188883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500b6790 msgr2=0x7ff1501648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 --2- 172.21.15.44:0/1905188883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500b6790 0x7ff1501648f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff138005fd0 tx=0x7ff13802a670 comp rx=0 tx=0).stop 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 -- 172.21.15.44:0/1905188883 shutdown_connections 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 --2- 172.21.15.44:0/1905188883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 0x7ff150166e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 --2- 172.21.15.44:0/1905188883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500b6790 0x7ff1501648f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 -- 172.21.15.44:0/1905188883 >> 172.21.15.44:0/1905188883 conn(0x7ff1500d3700 msgr2=0x7ff1500d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 -- 172.21.15.44:0/1905188883 shutdown_connections 2024-07-17T06:55:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.543+0000 7ff157400640 1 -- 172.21.15.44:0/1905188883 wait complete. 2024-07-17T06:55:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.545+0000 7ff157400640 1 Processor -- start 2024-07-17T06:55:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.545+0000 7ff157400640 1 -- start start 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 0x7ff150155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1500b6790 0x7ff1500c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff150167710 con 0x7ff1500ce1d0 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff150167590 con 0x7ff1500b6790 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff157400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff150167890 con 0x7ff1500b29e0 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33606/0 (socket says 172.21.15.44:33606) 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 -- 172.21.15.44:0/3108548358 learned_addr learned my addr 172.21.15.44:0/3108548358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:38.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff154a00640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 0x7ff150155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 -- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 msgr2=0x7ff150155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 0x7ff150155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 -- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1500b6790 msgr2=0x7ff1500c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1500b6790 0x7ff1500c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.546+0000 7ff155400640 1 -- 172.21.15.44:0/3108548358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1501f7b70 con 0x7ff1500ce1d0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff155400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d750 tx=0x7ff14800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148017070 con 0x7ff1500ce1d0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff148009cc0 con 0x7ff1500ce1d0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14800de60 con 0x7ff1500ce1d0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1501f7e50 con 0x7ff1500ce1d0 2024-07-17T06:55:38.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.547+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1501f8410 con 0x7ff1500ce1d0 2024-07-17T06:55:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.549+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1501f8650 con 0x7ff1500ce1d0 2024-07-17T06:55:38.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.550+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ff148005280 con 0x7ff1500ce1d0 2024-07-17T06:55:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.551+0000 7ff14cc00640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 0x7ff12c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:38.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.551+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff148098830 con 0x7ff1500ce1d0 2024-07-17T06:55:38.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.551+0000 7ff154a00640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 0x7ff12c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:38.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.552+0000 7ff154a00640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 0x7ff12c079070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff140001ce0 tx=0x7ff140009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:38.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.555+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff1501f8650 con 0x7ff1500ce1d0 2024-07-17T06:55:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.766+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff1500d2950 con 0x7ff1500ce1d0 2024-07-17T06:55:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.773+0000 7ff14cc00640 1 -- 172.21.15.44:0/3108548358 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7ff148061850 con 0x7ff1500ce1d0 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.775+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 msgr2=0x7ff12c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.775+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 0x7ff12c079070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff140001ce0 tx=0x7ff140009040 comp rx=0 tx=0).stop 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.775+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 msgr2=0x7ff15015f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.775+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d750 tx=0x7ff14800dc10 comp rx=0 tx=0).stop 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 shutdown_connections 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1500ce1d0 0x7ff15015f550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1500b6790 0x7ff1500c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff12c076bb0 0x7ff12c079070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 --2- 172.21.15.44:0/3108548358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1500b29e0 0x7ff150155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 >> 172.21.15.44:0/3108548358 conn(0x7ff1500d3700 msgr2=0x7ff1500b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 shutdown_connections 2024-07-17T06:55:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:38.776+0000 7ff157400640 1 -- 172.21.15.44:0/3108548358 wait complete. 2024-07-17T06:55:38.797 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T06:55:38.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T06:55:38.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T06:55:39.036 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T06:55:39.036 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:39.036 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:39.097 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:39.097 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:39.150 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:39.150 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:39.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:39.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 -- 172.21.15.44:0/2178768242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f538006db30 msgr2=0x7f53801503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:39.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 --2- 172.21.15.44:0/2178768242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f538006db30 0x7f53801503c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5378005fd0 tx=0x7f537802bbc0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 -- 172.21.15.44:0/2178768242 shutdown_connections 2024-07-17T06:55:39.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 --2- 172.21.15.44:0/2178768242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f538006db30 0x7f53801503c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 -- 172.21.15.44:0/2178768242 >> 172.21.15.44:0/2178768242 conn(0x7f53800cf0d0 msgr2=0x7f53800cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:39.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 -- 172.21.15.44:0/2178768242 shutdown_connections 2024-07-17T06:55:39.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.420+0000 7f537f200640 1 -- 172.21.15.44:0/2178768242 wait complete. 2024-07-17T06:55:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.422+0000 7f537f200640 1 Processor -- start 2024-07-17T06:55:39.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.422+0000 7f537f200640 1 -- start start 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538006f4d0 0x7f53801f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 0x7f5380152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53800b34e0 con 0x7f53801f1e30 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53800b3360 con 0x7f538006f4d0 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53800ac010 con 0x7f538015a450 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.423+0000 7f537e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56776/0 (socket says 172.21.15.44:56776) 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 -- 172.21.15.44:0/2042437761 learned_addr learned my addr 172.21.15.44:0/2042437761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537de00640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538006f4d0 0x7f53801f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 -- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 msgr2=0x7f5380152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:39.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537d400640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 0x7f5380152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 0x7f5380152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 -- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538006f4d0 msgr2=0x7f53801f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538006f4d0 0x7f53801f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 -- 172.21.15.44:0/2042437761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53800cbed0 con 0x7f53801f1e30 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537d400640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 0x7f5380152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.424+0000 7f537e800640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f537800ade0 tx=0x7f537802d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.425+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378042020 con 0x7f53801f1e30 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.425+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5378004330 con 0x7f53801f1e30 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.425+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53800cc1b0 con 0x7f53801f1e30 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.425+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537803a040 con 0x7f53801f1e30 2024-07-17T06:55:39.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.425+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538015d4e0 con 0x7f53801f1e30 2024-07-17T06:55:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.426+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53801503c0 con 0x7f53801f1e30 2024-07-17T06:55:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.427+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f5378045020 con 0x7f53801f1e30 2024-07-17T06:55:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.428+0000 7f5376000640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 0x7f535c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.428+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f53780b9fe0 con 0x7f53801f1e30 2024-07-17T06:55:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.429+0000 7f537de00640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 0x7f535c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:39.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.430+0000 7f537de00640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 0x7f535c079070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f53800acf00 tx=0x7f5370009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.433+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f537804c020 con 0x7f53801f1e30 2024-07-17T06:55:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.658+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f538015d7a0 con 0x7f53801f1e30 2024-07-17T06:55:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.659+0000 7f5376000640 1 -- 172.21.15.44:0/2042437761 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5378083020 con 0x7f53801f1e30 2024-07-17T06:55:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:39.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 msgr2=0x7f535c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 0x7f535c079070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f53800acf00 tx=0x7f5370009040 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 msgr2=0x7f5380152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f537800ade0 tx=0x7f537802d9d0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 shutdown_connections 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53801f1e30 0x7f5380152f90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f535c076bb0 0x7f535c079070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538015a450 0x7f5380152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 --2- 172.21.15.44:0/2042437761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538006f4d0 0x7f53801f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.662+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 >> 172.21.15.44:0/2042437761 conn(0x7f53800cf0d0 msgr2=0x7f53800aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.663+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 shutdown_connections 2024-07-17T06:55:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:39.663+0000 7f537f200640 1 -- 172.21.15.44:0/2042437761 wait complete. 2024-07-17T06:55:39.680 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T06:55:39.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:39.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-07-17T06:55:39.770 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:55:39.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:39.770 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T06:55:39.822 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:39.822 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:39.878 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:39.878 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:39.926 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:39.926 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:40.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:40.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:55:40.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:55:40.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 -- 172.21.15.44:0/1479208536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 msgr2=0x7f570c06bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:40.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 --2- 172.21.15.44:0/1479208536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c06bf40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009830 tx=0x7f56f802a540 comp rx=0 tx=0).stop 2024-07-17T06:55:40.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 -- 172.21.15.44:0/1479208536 shutdown_connections 2024-07-17T06:55:40.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 --2- 172.21.15.44:0/1479208536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c06bf40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 -- 172.21.15.44:0/1479208536 >> 172.21.15.44:0/1479208536 conn(0x7f570c06d050 msgr2=0x7f570c06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:40.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 -- 172.21.15.44:0/1479208536 shutdown_connections 2024-07-17T06:55:40.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.252+0000 7f5713600640 1 -- 172.21.15.44:0/1479208536 wait complete. 2024-07-17T06:55:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.254+0000 7f5713600640 1 Processor -- start 2024-07-17T06:55:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.254+0000 7f5713600640 1 -- start start 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570c0afa90 0x7f570c1f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f570c072e00 con 0x7f570c06c480 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f570c15efa0 con 0x7f570c13e860 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5713600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f570c072f80 con 0x7f570c0afa90 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5710c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5710c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56778/0 (socket says 172.21.15.44:56778) 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.255+0000 7f5710c00640 1 -- 172.21.15.44:0/2514934078 learned_addr learned my addr 172.21.15.44:0/2514934078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f570be00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570c0afa90 0x7f570c1f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5711600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:40.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 -- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570c0afa90 msgr2=0x7f570c1f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570c0afa90 0x7f570c1f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 -- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 msgr2=0x7f570c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 -- 172.21.15.44:0/2514934078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c1f51d0 con 0x7f570c06c480 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5711600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:40.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5710c00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f570000c500 tx=0x7f570000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000052d0 con 0x7f570c06c480 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5700005470 con 0x7f570c06c480 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700012850 con 0x7f570c06c480 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.256+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c1f7990 con 0x7f570c06c480 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.257+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c0a99c0 con 0x7f570c06c480 2024-07-17T06:55:40.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.258+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570c0733c0 con 0x7f570c06c480 2024-07-17T06:55:40.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.259+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f57000129f0 con 0x7f570c06c480 2024-07-17T06:55:40.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.260+0000 7f5708c00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 0x7f56ec079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:40.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.260+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f5700098440 con 0x7f570c06c480 2024-07-17T06:55:40.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.261+0000 7f570be00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 0x7f56ec079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:40.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.262+0000 7f570be00640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 0x7f56ec079070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f8033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.265+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f57000615b0 con 0x7f570c06c480 2024-07-17T06:55:40.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.493+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f570c06d6e0 con 0x7f570c06c480 2024-07-17T06:55:40.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.495+0000 7f5708c00640 1 -- 172.21.15.44:0/2514934078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5700060eb0 con 0x7f570c06c480 2024-07-17T06:55:40.494 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:40.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.497+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 msgr2=0x7f56ec079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:40.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.497+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 0x7f56ec079070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f8033040 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.497+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 msgr2=0x7f570c1f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.497+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f570000c500 tx=0x7f570000c9c0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 shutdown_connections 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f570c13e860 0x7f570c0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f56ec076bb0 0x7f56ec079070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570c0afa90 0x7f570c1f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 --2- 172.21.15.44:0/2514934078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f570c06c480 0x7f570c1f4c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:40.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 >> 172.21.15.44:0/2514934078 conn(0x7f570c06d050 msgr2=0x7f570c15a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:40.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 shutdown_connections 2024-07-17T06:55:40.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:40.498+0000 7f5713600640 1 -- 172.21.15.44:0/2514934078 wait complete. 2024-07-17T06:55:40.518 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-07-17T06:55:40.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:40.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-07-17T06:55:40.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:55:40.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:40.618 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-17T06:55:40.664 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:40.664 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:40.723 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:40.723 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:40.778 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:55:40.778 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:40.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.048+0000 7fd19fa00640 1 -- 172.21.15.44:0/392905777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004800 msgr2=0x7fd178004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.048+0000 7fd19fa00640 1 --2- 172.21.15.44:0/392905777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004800 0x7fd178004c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd180001d90 tx=0x7fd178005830 comp rx=0 tx=0).stop 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.048+0000 7fd19da00640 1 -- 172.21.15.44:0/392905777 reap_dead start 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.048+0000 7fd19fa00640 1 -- 172.21.15.44:0/392905777 shutdown_connections 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.048+0000 7fd19fa00640 1 -- 172.21.15.44:0/392905777 >> 172.21.15.44:0/392905777 conn(0x7fd1980b0eb0 msgr2=0x7fd1980b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.049+0000 7fd19fa00640 1 -- 172.21.15.44:0/392905777 shutdown_connections 2024-07-17T06:55:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.049+0000 7fd19fa00640 1 -- 172.21.15.44:0/392905777 wait complete. 2024-07-17T06:55:41.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.050+0000 7fd19fa00640 1 Processor -- start 2024-07-17T06:55:41.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.051+0000 7fd19fa00640 1 -- start start 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.051+0000 7fd19fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 0x7fd1981f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19815e7d0 0x7fd19815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980ac9d0 con 0x7fd1981f76f0 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980ac850 con 0x7fd19815e7d0 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd19815e0a0 con 0x7fd178004800 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56806/0 (socket says 172.21.15.44:56806) 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 -- 172.21.15.44:0/3626547860 learned_addr learned my addr 172.21.15.44:0/3626547860 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19d000640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 0x7fd1981f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19da00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19815e7d0 0x7fd19815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 -- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 msgr2=0x7fd1981f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 0x7fd1981f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 -- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19815e7d0 msgr2=0x7fd19815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19815e7d0 0x7fd19815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd197e00640 1 -- 172.21.15.44:0/3626547860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981f85d0 con 0x7fd1981f76f0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.052+0000 7fd19d000640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 0x7fd1981f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd197e00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd18002ac80 tx=0x7fd180003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180006250 con 0x7fd1981f76f0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1800064e0 con 0x7fd1981f76f0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180033480 con 0x7fd1981f76f0 2024-07-17T06:55:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981f4a00 con 0x7fd1981f76f0 2024-07-17T06:55:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.053+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981f4fc0 con 0x7fd1981f76f0 2024-07-17T06:55:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.054+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19815e5c0 con 0x7fd1981f76f0 2024-07-17T06:55:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.055+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fd180033620 con 0x7fd1981f76f0 2024-07-17T06:55:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.056+0000 7fd194c00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 0x7fd174079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.057+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fd1800b8d20 con 0x7fd1981f76f0 2024-07-17T06:55:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.057+0000 7fd19d000640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 0x7fd174079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.058+0000 7fd19d000640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 0x7fd174079070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd188001a30 tx=0x7fd188001e50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:41.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.062+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd1981f85d0 con 0x7fd1981f76f0 2024-07-17T06:55:41.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.288+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd198160e90 con 0x7fd1981f76f0 2024-07-17T06:55:41.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.288+0000 7fd194c00640 1 -- 172.21.15.44:0/3626547860 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fd180003b60 con 0x7fd1981f76f0 2024-07-17T06:55:41.287 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:41.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 msgr2=0x7fd174079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 0x7fd174079070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd188001a30 tx=0x7fd188001e50 comp rx=0 tx=0).stop 2024-07-17T06:55:41.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 msgr2=0x7fd1981f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd18002ac80 tx=0x7fd180003970 comp rx=0 tx=0).stop 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 shutdown_connections 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd19815e7d0 0x7fd19815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1981f76f0 0x7fd1981f4090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd174076bb0 0x7fd174079070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 --2- 172.21.15.44:0/3626547860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178004800 0x7fd1981f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 >> 172.21.15.44:0/3626547860 conn(0x7fd1980b0eb0 msgr2=0x7fd19806d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.291+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 shutdown_connections 2024-07-17T06:55:41.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.292+0000 7fd19fa00640 1 -- 172.21.15.44:0/3626547860 wait complete. 2024-07-17T06:55:41.309 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-07-17T06:55:41.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:41.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-07-17T06:55:41.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T06:55:41.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:41.400 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-07-17T06:55:41.449 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:41.449 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:41.498 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:41.499 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:41.583 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T06:55:41.583 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:55:41.671 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T06:55:41.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 -- 172.21.15.44:0/3607817795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 msgr2=0x7feba815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 --2- 172.21.15.44:0/3607817795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba815ac10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feba80d0260 tx=0x7feba002aa30 comp rx=0 tx=0).stop 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 -- 172.21.15.44:0/3607817795 shutdown_connections 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 --2- 172.21.15.44:0/3607817795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba815ac10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 --2- 172.21.15.44:0/3607817795 >> v2:172.21.15.175:3300/0 conn(0x7feba815a280 0x7feba80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 -- 172.21.15.44:0/3607817795 >> 172.21.15.44:0/3607817795 conn(0x7feba80b0eb0 msgr2=0x7feba80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.892+0000 7febaca00640 1 -- 172.21.15.44:0/3607817795 shutdown_connections 2024-07-17T06:55:41.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.893+0000 7febaca00640 1 -- 172.21.15.44:0/3607817795 wait complete. 2024-07-17T06:55:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.894+0000 7febaca00640 1 Processor -- start 2024-07-17T06:55:41.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.894+0000 7febaca00640 1 -- start start 2024-07-17T06:55:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba80b1af0 0x7feba8157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba815a280 0x7feba80c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feba806d760 con 0x7feba80ac8a0 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feba806f1d0 con 0x7feba815a280 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.895+0000 7febaca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feba815b150 con 0x7feba80b1af0 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56826/0 (socket says 172.21.15.44:56826) 2024-07-17T06:55:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 -- 172.21.15.44:0/788355775 learned_addr learned my addr 172.21.15.44:0/788355775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba6800640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba815a280 0x7feba80c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 -- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba80b1af0 msgr2=0x7feba8157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba80b1af0 0x7feba8157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 -- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba815a280 msgr2=0x7feba80c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba815a280 0x7feba80c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.896+0000 7feba5e00640 1 -- 172.21.15.44:0/788355775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8156b20 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7feba5e00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d6d0 tx=0x7feb9800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98017070 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb98005330 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98009820 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba81f74f0 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.897+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81f7ab0 con 0x7feba80ac8a0 2024-07-17T06:55:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.898+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba80cd910 con 0x7feba80ac8a0 2024-07-17T06:55:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.899+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7feb98009bd0 con 0x7feba80ac8a0 2024-07-17T06:55:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.900+0000 7feb9ea00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 0x7feb840790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.900+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7feb98098980 con 0x7feba80ac8a0 2024-07-17T06:55:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.901+0000 7feb9d400640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 0x7feb840790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:41.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.902+0000 7feb9d400640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 0x7feb840790c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feba8074a70 tx=0x7feb900079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:41.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:41.905+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feba80cd910 con 0x7feba80ac8a0 2024-07-17T06:55:42.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.133+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feba815b6e0 con 0x7feba80ac8a0 2024-07-17T06:55:42.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.134+0000 7feb9ea00640 1 -- 172.21.15.44:0/788355775 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7feb98061ab0 con 0x7feba80ac8a0 2024-07-17T06:55:42.133 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T06:55:42.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 msgr2=0x7feb840790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 0x7feb840790c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feba8074a70 tx=0x7feb900079b0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 msgr2=0x7feba81556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d6d0 tx=0x7feb9800db90 comp rx=0 tx=0).stop 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 shutdown_connections 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba815a280 0x7feba80c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7feb84076c00 0x7feb840790c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba80b1af0 0x7feba8157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 --2- 172.21.15.44:0/788355775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feba80ac8a0 0x7feba81556a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.137+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 >> 172.21.15.44:0/788355775 conn(0x7feba80b0eb0 msgr2=0x7feba806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.138+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 shutdown_connections 2024-07-17T06:55:42.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.138+0000 7febaca00640 1 -- 172.21.15.44:0/788355775 wait complete. 2024-07-17T06:55:42.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-07-17T06:55:42.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:42.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-07-17T06:55:42.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T06:55:42.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:55:42.241 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-07-17T06:55:42.287 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T06:55:42.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:42.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 -- 172.21.15.44:0/4206158196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 msgr2=0x7f27000cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 --2- 172.21.15.44:0/4206158196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27000cf2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009830 tx=0x7f26ec02a480 comp rx=0 tx=0).stop 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 -- 172.21.15.44:0/4206158196 shutdown_connections 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 --2- 172.21.15.44:0/4206158196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27000cf2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 --2- 172.21.15.44:0/4206158196 >> v2:172.21.15.175:3300/0 conn(0x7f270015d240 0x7f270009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 -- 172.21.15.44:0/4206158196 >> 172.21.15.44:0/4206158196 conn(0x7f27000b0e80 msgr2=0x7f27000b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 -- 172.21.15.44:0/4206158196 shutdown_connections 2024-07-17T06:55:42.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.510+0000 7f2706800640 1 -- 172.21.15.44:0/4206158196 wait complete. 2024-07-17T06:55:42.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.512+0000 7f2706800640 1 Processor -- start 2024-07-17T06:55:42.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.512+0000 7f2706800640 1 -- start start 2024-07-17T06:55:42.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27001f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 0x7f270015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27001614f0 con 0x7f270015d240 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27000ad780 con 0x7f27000cf810 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.513+0000 7f2706800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2700161670 con 0x7f27000b1fa0 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56840/0 (socket says 172.21.15.44:56840) 2024-07-17T06:55:42.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 -- 172.21.15.44:0/3393456189 learned_addr learned my addr 172.21.15.44:0/3393456189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:42.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705400640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27001f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2704a00640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 0x7f270015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 -- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 msgr2=0x7f27001f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27001f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 -- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 msgr2=0x7f270015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 0x7f270015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2705e00640 1 -- 172.21.15.44:0/3393456189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27001f47f0 con 0x7f270015d240 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.514+0000 7f2704a00640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 0x7f270015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f2705e00640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c7a0 tx=0x7f26f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8003b70 con 0x7f270015d240 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26f8003d30 con 0x7f270015d240 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800aab0 con 0x7f270015d240 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2700071730 con 0x7f270015d240 2024-07-17T06:55:42.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.515+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2700071cf0 con 0x7f270015d240 2024-07-17T06:55:42.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.516+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2700161b90 con 0x7f270015d240 2024-07-17T06:55:42.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.517+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f26f800ac50 con 0x7f270015d240 2024-07-17T06:55:42.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.518+0000 7f26fd600640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 0x7f26e0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.518+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f26f80985a0 con 0x7f270015d240 2024-07-17T06:55:42.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.519+0000 7f2705400640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 0x7f26e0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:42.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.520+0000 7f2705400640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 0x7f26e0079070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f270014ba80 tx=0x7f26f4009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:42.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.523+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f26f8061650 con 0x7f270015d240 2024-07-17T06:55:42.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.774+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27000b15e0 con 0x7f270015d240 2024-07-17T06:55:42.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.775+0000 7f26fd600640 1 -- 172.21.15.44:0/3393456189 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f26f8060f50 con 0x7f270015d240 2024-07-17T06:55:42.774 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:42.775 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:42.775 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 msgr2=0x7f26e0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 0x7f26e0079070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f270014ba80 tx=0x7f26f4009260 comp rx=0 tx=0).stop 2024-07-17T06:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 msgr2=0x7f27001603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:42.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c7a0 tx=0x7f26f800cc60 comp rx=0 tx=0).stop 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 shutdown_connections 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f270015d240 0x7f27001603d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27000cf810 0x7f270015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f26e0076bb0 0x7f26e0079070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.778+0000 7f2706800640 1 --2- 172.21.15.44:0/3393456189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27000b1fa0 0x7f27001f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.779+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 >> 172.21.15.44:0/3393456189 conn(0x7f27000b0e80 msgr2=0x7f27000af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.779+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 shutdown_connections 2024-07-17T06:55:42.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:42.779+0000 7f2706800640 1 -- 172.21.15.44:0/3393456189 wait complete. 2024-07-17T06:55:42.800 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 -- 172.21.15.44:0/1526407205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab806f020 msgr2=0x7f9ab806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 --2- 172.21.15.44:0/1526407205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab806f020 0x7f9ab806f480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009830 tx=0x7f9aa402a510 comp rx=0 tx=0).stop 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 -- 172.21.15.44:0/1526407205 shutdown_connections 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 --2- 172.21.15.44:0/1526407205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab8153e40 0x7f9ab81542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 --2- 172.21.15.44:0/1526407205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab806f020 0x7f9ab806f480 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 -- 172.21.15.44:0/1526407205 >> 172.21.15.44:0/1526407205 conn(0x7f9ab80b0fb0 msgr2=0x7f9ab80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 -- 172.21.15.44:0/1526407205 shutdown_connections 2024-07-17T06:55:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.007+0000 7f9ac0200640 1 -- 172.21.15.44:0/1526407205 wait complete. 2024-07-17T06:55:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.009+0000 7f9ac0200640 1 Processor -- start 2024-07-17T06:55:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.009+0000 7f9ac0200640 1 -- start start 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab806f020 0x7f9ab8147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab815b2e0 0x7f9ab815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8154810 con 0x7f9ab8153e40 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab815b8d0 con 0x7f9ab815b2e0 2024-07-17T06:55:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9ac0200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab8154990 con 0x7f9ab806f020 2024-07-17T06:55:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9abce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9abce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56862/0 (socket says 172.21.15.44:56862) 2024-07-17T06:55:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9abce00640 1 -- 172.21.15.44:0/861598792 learned_addr learned my addr 172.21.15.44:0/861598792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.010+0000 7f9abd800640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab806f020 0x7f9ab8147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abe200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab815b2e0 0x7f9ab815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 -- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab806f020 msgr2=0x7f9ab8147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab806f020 0x7f9ab8147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 -- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab815b2e0 msgr2=0x7f9ab815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab815b2e0 0x7f9ab815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 -- 172.21.15.44:0/861598792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab80b2350 con 0x7f9ab8153e40 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9abce00640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009950 tx=0x7f9aa4032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4040070 con 0x7f9ab8153e40 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aa4032e70 con 0x7f9ab8153e40 2024-07-17T06:55:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40055b0 con 0x7f9ab8153e40 2024-07-17T06:55:43.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.011+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab80b2630 con 0x7f9ab8153e40 2024-07-17T06:55:43.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.012+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab80b2bf0 con 0x7f9ab8153e40 2024-07-17T06:55:43.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.013+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab806f480 con 0x7f9ab8153e40 2024-07-17T06:55:43.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.014+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9aa4039040 con 0x7f9ab8153e40 2024-07-17T06:55:43.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.015+0000 7f9ab5600640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 0x7f9a98079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.015+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f9aa40b97e0 con 0x7f9ab8153e40 2024-07-17T06:55:43.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.016+0000 7f9abd800640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 0x7f9a98079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.017+0000 7f9abd800640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 0x7f9a98079070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80cf220 tx=0x7f9aac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.019+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9aa4082890 con 0x7f9ab8153e40 2024-07-17T06:55:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.270+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ab814daa0 con 0x7f9ab8153e40 2024-07-17T06:55:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.272+0000 7f9ab5600640 1 -- 172.21.15.44:0/861598792 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f9aa4082190 con 0x7f9ab8153e40 2024-07-17T06:55:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:43.271 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:43.271 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 msgr2=0x7f9a98079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 0x7f9a98079070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80cf220 tx=0x7f9aac009040 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 msgr2=0x7f9ab814dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009950 tx=0x7f9aa4032c80 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 shutdown_connections 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ab815b2e0 0x7f9ab815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ab8153e40 0x7f9ab814dcc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a98076bb0 0x7f9a98079070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 --2- 172.21.15.44:0/861598792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ab806f020 0x7f9ab8147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 >> 172.21.15.44:0/861598792 conn(0x7f9ab80b0fb0 msgr2=0x7f9ab80b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:43.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 shutdown_connections 2024-07-17T06:55:43.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.275+0000 7f9ac0200640 1 -- 172.21.15.44:0/861598792 wait complete. 2024-07-17T06:55:43.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:43.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 -- 172.21.15.44:0/1128954645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880b9060 msgr2=0x7f7b8815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 --2- 172.21.15.44:0/1128954645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880b9060 0x7f7b8815d800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009860 tx=0x7f7b8002a540 comp rx=0 tx=0).stop 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 -- 172.21.15.44:0/1128954645 shutdown_connections 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 --2- 172.21.15.44:0/1128954645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880b9060 0x7f7b8815d800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 --2- 172.21.15.44:0/1128954645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880ce5c0 0x7f7b8815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.504+0000 7f7b8d800640 1 -- 172.21.15.44:0/1128954645 >> 172.21.15.44:0/1128954645 conn(0x7f7b880cfec0 msgr2=0x7f7b880d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.505+0000 7f7b8d800640 1 -- 172.21.15.44:0/1128954645 shutdown_connections 2024-07-17T06:55:43.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.505+0000 7f7b8d800640 1 -- 172.21.15.44:0/1128954645 wait complete. 2024-07-17T06:55:43.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.506+0000 7f7b8d800640 1 Processor -- start 2024-07-17T06:55:43.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.506+0000 7f7b8d800640 1 -- start start 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.507+0000 7f7b8d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b880b8c80 0x7f7b8814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.507+0000 7f7b8d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880b9060 0x7f7b880ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8815e0d0 con 0x7f7b880ce5c0 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8815df50 con 0x7f7b880b8c80 2024-07-17T06:55:43.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8815e250 con 0x7f7b880b9060 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56884/0 (socket says 172.21.15.44:56884) 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 -- 172.21.15.44:0/2076929829 learned_addr learned my addr 172.21.15.44:0/2076929829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b87400640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880b9060 0x7f7b880ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 -- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880b9060 msgr2=0x7f7b880ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b87e00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b880b8c80 0x7f7b8814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880b9060 0x7f7b880ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 -- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b880b8c80 msgr2=0x7f7b8814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b880b8c80 0x7f7b8814d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.508+0000 7f7b8ce00640 1 -- 172.21.15.44:0/2076929829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880cb610 con 0x7f7b880ce5c0 2024-07-17T06:55:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b8ce00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002ab30 tx=0x7f7b80032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80040070 con 0x7f7b880ce5c0 2024-07-17T06:55:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b80005310 con 0x7f7b880ce5c0 2024-07-17T06:55:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800047a0 con 0x7f7b880ce5c0 2024-07-17T06:55:43.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b880cb8f0 con 0x7f7b880ce5c0 2024-07-17T06:55:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.509+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8814d870 con 0x7f7b880ce5c0 2024-07-17T06:55:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.510+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8815e700 con 0x7f7b880ce5c0 2024-07-17T06:55:43.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.511+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f7b80039040 con 0x7f7b880ce5c0 2024-07-17T06:55:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.512+0000 7f7b7fe00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 0x7f7b64079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.512+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f7b800b9a40 con 0x7f7b880ce5c0 2024-07-17T06:55:43.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.513+0000 7f7b87e00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 0x7f7b64079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.514+0000 7f7b87e00640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 0x7f7b64079070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7b700015e0 tx=0x7f7b70009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.516+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7b80082a60 con 0x7f7b880ce5c0 2024-07-17T06:55:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.722+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b880b5d70 con 0x7f7b880ce5c0 2024-07-17T06:55:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.724+0000 7f7b7fe00640 1 -- 172.21.15.44:0/2076929829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f7b80082360 con 0x7f7b880ce5c0 2024-07-17T06:55:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:43.724 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:34.800937+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.726+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 msgr2=0x7f7b64079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.726+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 0x7f7b64079070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7b700015e0 tx=0x7f7b70009040 comp rx=0 tx=0).stop 2024-07-17T06:55:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.726+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 msgr2=0x7f7b881f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.726+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002ab30 tx=0x7f7b80032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 shutdown_connections 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b880ce5c0 0x7f7b881f45b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7b64076bb0 0x7f7b64079070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b880b9060 0x7f7b880ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 --2- 172.21.15.44:0/2076929829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b880b8c80 0x7f7b8814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 >> 172.21.15.44:0/2076929829 conn(0x7f7b880cfec0 msgr2=0x7f7b8806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 shutdown_connections 2024-07-17T06:55:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.727+0000 7f7b8d800640 1 -- 172.21.15.44:0/2076929829 wait complete. 2024-07-17T06:55:43.744 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T06:55:43.744 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:43.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 -- 172.21.15.44:0/2390392564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80ce5c0 msgr2=0x7fe6e806d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 --2- 172.21.15.44:0/2390392564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80ce5c0 0x7fe6e806d9b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009830 tx=0x7fe6d002a560 comp rx=0 tx=0).stop 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 -- 172.21.15.44:0/2390392564 shutdown_connections 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 --2- 172.21.15.44:0/2390392564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80ce5c0 0x7fe6e806d9b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 --2- 172.21.15.44:0/2390392564 >> v2:172.21.15.175:3300/0 conn(0x7fe6e80b8c80 0x7fe6e80b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 -- 172.21.15.44:0/2390392564 >> 172.21.15.44:0/2390392564 conn(0x7fe6e80cfec0 msgr2=0x7fe6e80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 -- 172.21.15.44:0/2390392564 shutdown_connections 2024-07-17T06:55:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.954+0000 7fe6e7200640 1 -- 172.21.15.44:0/2390392564 wait complete. 2024-07-17T06:55:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.956+0000 7fe6e7200640 1 Processor -- start 2024-07-17T06:55:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.956+0000 7fe6e7200640 1 -- start start 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6e80b8c80 0x7fe6e815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6e80ce5c0 0x7fe6e80c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e806e780 con 0x7fe6e80b5400 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e806e600 con 0x7fe6e80b8c80 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.957+0000 7fe6e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e806e900 con 0x7fe6e80ce5c0 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56904/0 (socket says 172.21.15.44:56904) 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 -- 172.21.15.44:0/439362980 learned_addr learned my addr 172.21.15.44:0/439362980 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6dfe00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6e80b8c80 0x7fe6e815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e6800640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6e80ce5c0 0x7fe6e80c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 -- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6e80ce5c0 msgr2=0x7fe6e80c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6e80ce5c0 0x7fe6e80c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 -- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6e80b8c80 msgr2=0x7fe6e815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6e80b8c80 0x7fe6e815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.958+0000 7fe6e5e00640 1 -- 172.21.15.44:0/439362980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e80c77a0 con 0x7fe6e80b5400 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6e5e00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800d950 tx=0x7fe6d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8017070 con 0x7fe6e80b5400 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d8009d20 con 0x7fe6e80b5400 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80106c0 con 0x7fe6e80b5400 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e80c7a80 con 0x7fe6e80b5400 2024-07-17T06:55:43.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.959+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e80c8040 con 0x7fe6e80b5400 2024-07-17T06:55:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.960+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e815f6f0 con 0x7fe6e80b5400 2024-07-17T06:55:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.961+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe6d8010860 con 0x7fe6e80b5400 2024-07-17T06:55:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.962+0000 7fe6de000640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 0x7fe6c4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.962+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fe6d8098b60 con 0x7fe6e80b5400 2024-07-17T06:55:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.963+0000 7fe6dfe00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 0x7fe6c4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.964+0000 7fe6dfe00640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 0x7fe6c4079070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0034000 tx=0x7fe6d0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:43.967+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe6d8061b80 con 0x7fe6e80b5400 2024-07-17T06:55:44.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.218+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6e80b5d70 con 0x7fe6e80b5400 2024-07-17T06:55:44.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.220+0000 7fe6de000640 1 -- 172.21.15.44:0/439362980 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7fe6d8061480 con 0x7fe6e80b5400 2024-07-17T06:55:44.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:44.219 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:44.219 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.222+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 msgr2=0x7fe6c4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.222+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 0x7fe6c4079070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0034000 tx=0x7fe6d0004730 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.222+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 msgr2=0x7fe6e81f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.222+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800d950 tx=0x7fe6d800de10 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 shutdown_connections 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe6c4076bb0 0x7fe6c4079070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6e80ce5c0 0x7fe6e80c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6e80b8c80 0x7fe6e815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 --2- 172.21.15.44:0/439362980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6e80b5400 0x7fe6e81f3bb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 >> 172.21.15.44:0/439362980 conn(0x7fe6e80cfec0 msgr2=0x7fe6e80b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:44.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 shutdown_connections 2024-07-17T06:55:44.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.223+0000 7fe6e7200640 1 -- 172.21.15.44:0/439362980 wait complete. 2024-07-17T06:55:44.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:55:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 -- 172.21.15.44:0/10233168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00cded0 msgr2=0x7f86f00ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 --2- 172.21.15.44:0/10233168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00cded0 0x7f86f00ce380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009830 tx=0x7f86e802a480 comp rx=0 tx=0).stop 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 -- 172.21.15.44:0/10233168 shutdown_connections 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 --2- 172.21.15.44:0/10233168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00cded0 0x7f86f00ce380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 --2- 172.21.15.44:0/10233168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00d3f10 0x7f86f00b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.452+0000 7f86f5600640 1 -- 172.21.15.44:0/10233168 >> 172.21.15.44:0/10233168 conn(0x7f86f008ca20 msgr2=0x7f86f008ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.453+0000 7f86f5600640 1 -- 172.21.15.44:0/10233168 shutdown_connections 2024-07-17T06:55:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.453+0000 7f86f5600640 1 -- 172.21.15.44:0/10233168 wait complete. 2024-07-17T06:55:44.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.454+0000 7f86f5600640 1 Processor -- start 2024-07-17T06:55:44.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.454+0000 7f86f5600640 1 -- start start 2024-07-17T06:55:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00cded0 0x7f86f0158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f00d3420 0x7f86f00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00ce8c0 con 0x7f86f00d3f10 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00cea40 con 0x7f86f00d3420 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.455+0000 7f86f5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f86f00cebc0 con 0x7f86f00cded0 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56934/0 (socket says 172.21.15.44:56934) 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 -- 172.21.15.44:0/1342695669 learned_addr learned my addr 172.21.15.44:0/1342695669 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86efe00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00cded0 0x7f86f0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86ef400640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f00d3420 0x7f86f00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 -- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00cded0 msgr2=0x7f86f0158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00cded0 0x7f86f0158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 -- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f00d3420 msgr2=0x7f86f00caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f00d3420 0x7f86f00caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.456+0000 7f86f4c00640 1 -- 172.21.15.44:0/1342695669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0159740 con 0x7f86f00d3f10 2024-07-17T06:55:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86f4c00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009950 tx=0x7f86e8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8040070 con 0x7f86f00d3f10 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86e8005310 con 0x7f86f00d3f10 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80047a0 con 0x7f86f00d3f10 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01599e0 con 0x7f86f00d3f10 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.457+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f00b35c0 con 0x7f86f00d3f10 2024-07-17T06:55:44.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.458+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f00ce380 con 0x7f86f00d3f10 2024-07-17T06:55:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.459+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f86e8039040 con 0x7f86f00d3f10 2024-07-17T06:55:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.461+0000 7f86e7e00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 0x7f86cc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.461+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f86e80b9840 con 0x7f86f00d3f10 2024-07-17T06:55:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.462+0000 7f86efe00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 0x7f86cc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.463+0000 7f86efe00640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 0x7f86cc079070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f86d80016b0 tx=0x7f86d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.466+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f86e80828f0 con 0x7f86f00d3f10 2024-07-17T06:55:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.712+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f00b3790 con 0x7f86f00d3f10 2024-07-17T06:55:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.714+0000 7f86e7e00640 1 -- 172.21.15.44:0/1342695669 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5219 (secure 0 0 0) 0x7f86e80821f0 con 0x7f86f00d3f10 2024-07-17T06:55:44.713 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:44.713 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-07-17T06:55:07:402940+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3043418549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":3043418549},{"type":"v1","addr":"172.21.15.175:6837","nonce":3043418549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2461175984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2461175984},{"type":"v1","addr":"172.21.15.44:6833","nonce":2461175984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/425837014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":425837014},{"type":"v1","addr":"172.21.15.44:6835","nonce":425837014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:54:48.717423+0000","modified":"2024-07-17T06:55:06.915431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4329":{"gid":4329,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/910842174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":910842174},{"type":"v1","addr":"172.21.15.175:6835","nonce":910842174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4329,"qdb_cluster":[4329]},"id":1}]} 2024-07-17T06:55:44.713 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-07-17T06:55:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 msgr2=0x7f86cc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 0x7f86cc079070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f86d80016b0 tx=0x7f86d8009040 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 msgr2=0x7f86f00cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009950 tx=0x7f86e8032b90 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 shutdown_connections 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f00d3f10 0x7f86f00cca50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86f00d3420 0x7f86f00caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f86cc076bb0 0x7f86cc079070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 --2- 172.21.15.44:0/1342695669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86f00cded0 0x7f86f0158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.716+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 >> 172.21.15.44:0/1342695669 conn(0x7f86f008ca20 msgr2=0x7f86f008ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.717+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 shutdown_connections 2024-07-17T06:55:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.717+0000 7f86f5600640 1 -- 172.21.15.44:0/1342695669 wait complete. 2024-07-17T06:55:44.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.932+0000 7f530c800640 1 -- 172.21.15.44:0/721872549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ec0052e0 msgr2=0x7f52ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.932+0000 7f530c800640 1 --2- 172.21.15.44:0/721872549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ec0052e0 0x7f52ec00b9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53000043b0 tx=0x7f52ec005830 comp rx=0 tx=0).stop 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.932+0000 7f530c800640 1 -- 172.21.15.44:0/721872549 shutdown_connections 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.932+0000 7f530c800640 1 --2- 172.21.15.44:0/721872549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52ec0052e0 0x7f52ec00b9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.932+0000 7f530c800640 1 -- 172.21.15.44:0/721872549 >> 172.21.15.44:0/721872549 conn(0x7f53040b0fb0 msgr2=0x7f53040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.933+0000 7f530c800640 1 -- 172.21.15.44:0/721872549 shutdown_connections 2024-07-17T06:55:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.933+0000 7f530c800640 1 -- 172.21.15.44:0/721872549 wait complete. 2024-07-17T06:55:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.934+0000 7f530c800640 1 Processor -- start 2024-07-17T06:55:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.935+0000 7f530c800640 1 -- start start 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 0x7f53041f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f530408bfc0 0x7f530408c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f530415a320 con 0x7f53040c8630 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f530415a1a0 con 0x7f52ec0052e0 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530c800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53040ae910 con 0x7f530408bfc0 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56942/0 (socket says 172.21.15.44:56942) 2024-07-17T06:55:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 -- 172.21.15.44:0/347126829 learned_addr learned my addr 172.21.15.44:0/347126829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:44.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309e00640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 0x7f53041f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f530a800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f530408bfc0 0x7f530408c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 -- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f530408bfc0 msgr2=0x7f530408c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f530408bfc0 0x7f530408c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 -- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 msgr2=0x7f53041f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 0x7f53041f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309400640 1 -- 172.21.15.44:0/347126829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53040caec0 con 0x7f53040c8630 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.936+0000 7f5309e00640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 0x7f53041f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f5309400640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e7a0 tx=0x7f52f400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400cc00 con 0x7f53040c8630 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52f400ce90 con 0x7f53040c8630 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53040cb1a0 con 0x7f53040c8630 2024-07-17T06:55:44.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4005280 con 0x7f53040c8630 2024-07-17T06:55:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.937+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53040cb760 con 0x7f53040c8630 2024-07-17T06:55:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.938+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53040b0750 con 0x7f53040c8630 2024-07-17T06:55:44.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.939+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f52f4005420 con 0x7f53040c8630 2024-07-17T06:55:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.940+0000 7f52fe000640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 0x7f52e8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:44.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.941+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f52f4098ee0 con 0x7f53040c8630 2024-07-17T06:55:44.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.941+0000 7f5309e00640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 0x7f52e8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:44.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.942+0000 7f5309e00640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 0x7f52e8079070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52f8001eb0 tx=0x7f52f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:44.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:44.944+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f53040b0750 con 0x7f53040c8630 2024-07-17T06:55:45.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.155+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5304153cd0 con 0x7f53040c8630 2024-07-17T06:55:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.157+0000 7f52fe000640 1 -- 172.21.15.44:0/347126829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f52f4016030 con 0x7f53040c8630 2024-07-17T06:55:45.157 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:45.157 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:34.800937+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:54:46.375450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-17T06:54:47.702732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-17T06:54:51.319737+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.17","mappings":[{"from":0,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:45.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.160+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 msgr2=0x7f52e8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.160+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 0x7f52e8079070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52f8001eb0 tx=0x7f52f8009040 comp rx=0 tx=0).stop 2024-07-17T06:55:45.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.160+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 msgr2=0x7f53040c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.160+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e7a0 tx=0x7f52f400ec60 comp rx=0 tx=0).stop 2024-07-17T06:55:45.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 shutdown_connections 2024-07-17T06:55:45.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f52e8076bb0 0x7f52e8079070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f530408bfc0 0x7f530408c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53040c8630 0x7f53040c8ab0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 --2- 172.21.15.44:0/347126829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52ec0052e0 0x7f53041f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 >> 172.21.15.44:0/347126829 conn(0x7f53040b0fb0 msgr2=0x7f53040b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:45.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 shutdown_connections 2024-07-17T06:55:45.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.161+0000 7f530c800640 1 -- 172.21.15.44:0/347126829 wait complete. 2024-07-17T06:55:45.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T06:55:45.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 -- 172.21.15.44:0/2520437342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0005210 msgr2=0x7efde000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 --2- 172.21.15.44:0/2520437342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0005210 0x7efde000b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efdf80043e0 tx=0x7efde0005830 comp rx=0 tx=0).stop 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 -- 172.21.15.44:0/2520437342 shutdown_connections 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 --2- 172.21.15.44:0/2520437342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0005210 0x7efde000b9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 -- 172.21.15.44:0/2520437342 >> 172.21.15.44:0/2520437342 conn(0x7efe000d3700 msgr2=0x7efe000d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.394+0000 7efe06600640 1 -- 172.21.15.44:0/2520437342 shutdown_connections 2024-07-17T06:55:45.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.395+0000 7efe06600640 1 -- 172.21.15.44:0/2520437342 wait complete. 2024-07-17T06:55:45.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.396+0000 7efe06600640 1 Processor -- start 2024-07-17T06:55:45.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.396+0000 7efe06600640 1 -- start start 2024-07-17T06:55:45.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 0x7efe00153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe000cbaf0 0x7efe000cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe000ce940 con 0x7efe000cc0b0 2024-07-17T06:55:45.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe000ce7c0 con 0x7efde0005210 2024-07-17T06:55:45.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.397+0000 7efe06600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe000ceac0 con 0x7efe000cbaf0 2024-07-17T06:55:45.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56958/0 (socket says 172.21.15.44:56958) 2024-07-17T06:55:45.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 -- 172.21.15.44:0/3307604394 learned_addr learned my addr 172.21.15.44:0/3307604394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdfea00640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe000cbaf0 0x7efe000cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdff400640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 0x7efe00153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 -- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe000cbaf0 msgr2=0x7efe000cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe000cbaf0 0x7efe000cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 -- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 msgr2=0x7efe00153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 0x7efe00153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 -- 172.21.15.44:0/3307604394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe000c9850 con 0x7efe000cc0b0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdff400640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 0x7efe00153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.398+0000 7efdffe00640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efdf80044d0 tx=0x7efdf8004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.399+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8041070 con 0x7efe000cc0b0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.399+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdf8006250 con 0x7efe000cc0b0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.399+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8037040 con 0x7efe000cc0b0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.399+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe000c9b30 con 0x7efe000cc0b0 2024-07-17T06:55:45.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.399+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe000cca70 con 0x7efe000cc0b0 2024-07-17T06:55:45.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.400+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe001616a0 con 0x7efe000cc0b0 2024-07-17T06:55:45.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.401+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7efdf8044070 con 0x7efe000cc0b0 2024-07-17T06:55:45.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.402+0000 7efdf7400640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 0x7efddc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.402+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7efdf80b7ed0 con 0x7efe000cc0b0 2024-07-17T06:55:45.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.403+0000 7efdff400640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 0x7efddc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.404+0000 7efdff400640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 0x7efddc079070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009b60 tx=0x7efdf0005e70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:45.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.407+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efdf802ae20 con 0x7efe000cc0b0 2024-07-17T06:55:45.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.659+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe000ca450 con 0x7efe000cc0b0 2024-07-17T06:55:45.659 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T06:55:45.661+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T06:55:45.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.665+0000 7efdf7400640 1 -- 172.21.15.44:0/3307604394 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7efdf8080f00 con 0x7efe000cc0b0 2024-07-17T06:55:45.664 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T06:55:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 msgr2=0x7efddc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 0x7efddc079070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009b60 tx=0x7efdf0005e70 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 msgr2=0x7efe000cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efdf80044d0 tx=0x7efdf8004a10 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 shutdown_connections 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe000cc0b0 0x7efe000cc530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7efddc076bb0 0x7efddc079070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe000cbaf0 0x7efe000cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 --2- 172.21.15.44:0/3307604394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde0005210 0x7efe00153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.669+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 >> 172.21.15.44:0/3307604394 conn(0x7efe000d3700 msgr2=0x7efe000b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.670+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 shutdown_connections 2024-07-17T06:55:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.670+0000 7efe06600640 1 -- 172.21.15.44:0/3307604394 wait complete. 2024-07-17T06:55:45.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T06:55:45.696 INFO:tasks.ceph.mds.b.smithi175.stderr: -4525> 2024-07-17T06:54:53.617+0000 7f872a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T06:55:45.770 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.898+0000 7f33dd200640 1 -- 172.21.15.44:0/2461878000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 msgr2=0x7f33d815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.898+0000 7f33dd200640 1 --2- 172.21.15.44:0/2461878000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d815c2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f33d80b43d0 tx=0x7f33c402a4d0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 -- 172.21.15.44:0/2461878000 shutdown_connections 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 --2- 172.21.15.44:0/2461878000 >> v2:172.21.15.175:3300/0 conn(0x7f33d80cb3a0 0x7f33d80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 --2- 172.21.15.44:0/2461878000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d815c2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 --2- 172.21.15.44:0/2461878000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d8152220 0x7f33d806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 -- 172.21.15.44:0/2461878000 >> 172.21.15.44:0/2461878000 conn(0x7f33d80b0eb0 msgr2=0x7f33d80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 -- 172.21.15.44:0/2461878000 shutdown_connections 2024-07-17T06:55:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.899+0000 7f33dd200640 1 -- 172.21.15.44:0/2461878000 wait complete. 2024-07-17T06:55:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.901+0000 7f33dd200640 1 Processor -- start 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.901+0000 7f33dd200640 1 -- start start 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d80cb3a0 0x7f33d81f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33d8152220 0x7f33d8153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33d815f1f0 con 0x7f33d806ed30 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33d815f070 con 0x7f33d8152220 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33dd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33d815f370 con 0x7f33d80cb3a0 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.902+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56966/0 (socket says 172.21.15.44:56966) 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 -- 172.21.15.44:0/2083784756 learned_addr learned my addr 172.21.15.44:0/2083784756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d6800640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33d8152220 0x7f33d8153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5400640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d80cb3a0 0x7f33d81f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 -- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d80cb3a0 msgr2=0x7f33d81f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d80cb3a0 0x7f33d81f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 -- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33d8152220 msgr2=0x7f33d8153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33d8152220 0x7f33d8153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.903+0000 7f33d5e00640 1 -- 172.21.15.44:0/2083784756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d81f4da0 con 0x7f33d806ed30 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33d5e00640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d6d0 tx=0x7f33c800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8017070 con 0x7f33d806ed30 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33c8005330 con 0x7f33d806ed30 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d81529c0 con 0x7f33d806ed30 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8009820 con 0x7f33d806ed30 2024-07-17T06:55:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.904+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d8152f80 con 0x7f33d806ed30 2024-07-17T06:55:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.905+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d80cb830 con 0x7f33d806ed30 2024-07-17T06:55:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.906+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f33c8009bd0 con 0x7f33d806ed30 2024-07-17T06:55:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.907+0000 7f33ce000640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 0x7f33b8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.908+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6829+0+0 (secure 0 0 0) 0x7f33c8098980 con 0x7f33d806ed30 2024-07-17T06:55:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.908+0000 7f33d5400640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 0x7f33b8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:45.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.909+0000 7f33d5400640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 0x7f33b8079070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f33c4033000 tx=0x7f33c4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:45.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:45.911+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f33c80619e0 con 0x7f33d806ed30 2024-07-17T06:55:46.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.162+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f33d80b3e10 con 0x7f33d806ed30 2024-07-17T06:55:46.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.679+0000 7f33ce000640 1 -- 172.21.15.44:0/2083784756 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f33c80612e0 con 0x7f33d806ed30 2024-07-17T06:55:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.683+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 msgr2=0x7f33b8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.683+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 0x7f33b8079070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f33c4033000 tx=0x7f33c4004730 comp rx=0 tx=0).stop 2024-07-17T06:55:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.683+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 msgr2=0x7f33d81534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d6d0 tx=0x7f33c800db90 comp rx=0 tx=0).stop 2024-07-17T06:55:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 shutdown_connections 2024-07-17T06:55:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33d8152220 0x7f33d8153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f33b8076bb0 0x7f33b8079070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33d80cb3a0 0x7f33d81f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 --2- 172.21.15.44:0/2083784756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33d806ed30 0x7f33d81534a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 >> 172.21.15.44:0/2083784756 conn(0x7f33d80b0eb0 msgr2=0x7f33d80b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:46.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 shutdown_connections 2024-07-17T06:55:46.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.684+0000 7f33dd200640 1 -- 172.21.15.44:0/2083784756 wait complete. 2024-07-17T06:55:46.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T06:55:46.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 -- 172.21.15.44:0/1619861509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27640b21b0 msgr2=0x7f27640b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 --2- 172.21.15.44:0/1619861509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27640b21b0 0x7f27640b2620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2764070850 tx=0x7f275c02b9a0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 -- 172.21.15.44:0/1619861509 shutdown_connections 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 --2- 172.21.15.44:0/1619861509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27640b21b0 0x7f27640b2620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 --2- 172.21.15.44:0/1619861509 >> v2:172.21.15.175:3300/0 conn(0x7f276406f6d0 0x7f27641539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 -- 172.21.15.44:0/1619861509 >> 172.21.15.44:0/1619861509 conn(0x7f27640b0eb0 msgr2=0x7f27640b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 -- 172.21.15.44:0/1619861509 shutdown_connections 2024-07-17T06:55:46.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.929+0000 7f2769c00640 1 -- 172.21.15.44:0/1619861509 wait complete. 2024-07-17T06:55:46.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.931+0000 7f2769c00640 1 Processor -- start 2024-07-17T06:55:46.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.931+0000 7f2769c00640 1 -- start start 2024-07-17T06:55:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27640b21b0 0x7f2764141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f276415a5d0 0x7f2764142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27640b3090 con 0x7f276406f6d0 2024-07-17T06:55:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27640b2f10 con 0x7f276415a5d0 2024-07-17T06:55:46.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2769c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27640b3210 con 0x7f27640b21b0 2024-07-17T06:55:46.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2762a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:46.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2762a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56994/0 (socket says 172.21.15.44:56994) 2024-07-17T06:55:46.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.932+0000 7f2762a00640 1 -- 172.21.15.44:0/1901909837 learned_addr learned my addr 172.21.15.44:0/1901909837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:46.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 -- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27640b21b0 msgr2=0x7f2764141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:46.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27640b21b0 0x7f2764141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 -- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f276415a5d0 msgr2=0x7f2764142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:46.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f276415a5d0 0x7f2764142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:46.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 -- 172.21.15.44:0/1901909837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27641f3f50 con 0x7f276406f6d0 2024-07-17T06:55:46.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2762a00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9f0 tx=0x7f275400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:46.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275400cd80 con 0x7f276406f6d0 2024-07-17T06:55:46.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2754010430 con 0x7f276406f6d0 2024-07-17T06:55:46.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754010840 con 0x7f276406f6d0 2024-07-17T06:55:46.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.933+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641f4230 con 0x7f276406f6d0 2024-07-17T06:55:46.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.934+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641f47f0 con 0x7f276406f6d0 2024-07-17T06:55:46.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.935+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27641539f0 con 0x7f276406f6d0 2024-07-17T06:55:46.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.936+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f27540109e0 con 0x7f276406f6d0 2024-07-17T06:55:46.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.937+0000 7f275aa00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 0x7f2744079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:46.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.937+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6829+0+0 (secure 0 0 0) 0x7f2754099290 con 0x7f276406f6d0 2024-07-17T06:55:46.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.938+0000 7f2762000640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 0x7f2744079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:46.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.939+0000 7f2762000640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 0x7f2744079070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2764153cb0 tx=0x7f2750007970 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:46.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:46.941+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f27540623b0 con 0x7f276406f6d0 2024-07-17T06:55:47.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.153+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2764153560 con 0x7f276406f6d0 2024-07-17T06:55:47.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.899+0000 7f275aa00640 1 -- 172.21.15.44:0/1901909837 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f2754061cb0 con 0x7f276406f6d0 2024-07-17T06:55:47.899 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T06:55:47.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.905+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 msgr2=0x7f2744079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:47.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.905+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 0x7f2744079070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2764153cb0 tx=0x7f2750007970 comp rx=0 tx=0).stop 2024-07-17T06:55:47.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.905+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 msgr2=0x7f27641f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:47.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.905+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9f0 tx=0x7f275400eeb0 comp rx=0 tx=0).stop 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 shutdown_connections 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f276415a5d0 0x7f2764142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2744076bb0 0x7f2744079070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27640b21b0 0x7f2764141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 --2- 172.21.15.44:0/1901909837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f276406f6d0 0x7f27641f4ff0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.907+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 >> 172.21.15.44:0/1901909837 conn(0x7f27640b0eb0 msgr2=0x7f276406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:47.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.908+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 shutdown_connections 2024-07-17T06:55:47.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:47.908+0000 7f2769c00640 1 -- 172.21.15.44:0/1901909837 wait complete. 2024-07-17T06:55:47.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.134+0000 7fed09000640 1 -- 172.21.15.44:0/280000422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed04154560 msgr2=0x7fed040ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.134+0000 7fed09000640 1 --2- 172.21.15.44:0/280000422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed04154560 0x7fed040ae1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005fd0 tx=0x7fecf402a540 comp rx=0 tx=0).stop 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.134+0000 7fed09000640 1 -- 172.21.15.44:0/280000422 shutdown_connections 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.134+0000 7fed09000640 1 --2- 172.21.15.44:0/280000422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed04154560 0x7fed040ae1a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.134+0000 7fed09000640 1 -- 172.21.15.44:0/280000422 >> 172.21.15.44:0/280000422 conn(0x7fed040b0eb0 msgr2=0x7fed040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.135+0000 7fed09000640 1 -- 172.21.15.44:0/280000422 shutdown_connections 2024-07-17T06:55:48.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.135+0000 7fed09000640 1 -- 172.21.15.44:0/280000422 wait complete. 2024-07-17T06:55:48.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.136+0000 7fed09000640 1 Processor -- start 2024-07-17T06:55:48.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.136+0000 7fed09000640 1 -- start start 2024-07-17T06:55:48.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.137+0000 7fed09000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0406e690 0x7fed04152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.137+0000 7fed09000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 0x7fed04152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.137+0000 7fed09000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed09000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b28c0 con 0x7fed0415e1e0 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed09000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b2a40 con 0x7fed04154560 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed09000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed040b2bc0 con 0x7fed0406e690 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57010/0 (socket says 172.21.15.44:57010) 2024-07-17T06:55:48.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 -- 172.21.15.44:0/2747935904 learned_addr learned my addr 172.21.15.44:0/2747935904 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:48.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fecf9400640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 0x7fed04152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:48.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed01e00640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0406e690 0x7fed04152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 -- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0406e690 msgr2=0x7fed04152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0406e690 0x7fed04152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 -- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 msgr2=0x7fed04152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 0x7fed04152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fed02800640 1 -- 172.21.15.44:0/2747935904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0409c9a0 con 0x7fed0415e1e0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.138+0000 7fecf9400640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 0x7fed04152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fed02800640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c700 tx=0x7fecfc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc003b70 con 0x7fed0415e1e0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecfc003d10 con 0x7fed0415e1e0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00aab0 con 0x7fed0415e1e0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0409cc80 con 0x7fed0415e1e0 2024-07-17T06:55:48.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.139+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041532a0 con 0x7fed0415e1e0 2024-07-17T06:55:48.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.141+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed040ae1a0 con 0x7fed0415e1e0 2024-07-17T06:55:48.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.141+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fecfc00ac50 con 0x7fed0415e1e0 2024-07-17T06:55:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.143+0000 7fecfaa00640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 0x7fece0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.143+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6391+0+0 (secure 0 0 0) 0x7fecfc098460 con 0x7fed0415e1e0 2024-07-17T06:55:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.144+0000 7fed01e00640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 0x7fece0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:48.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.145+0000 7fed01e00640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 0x7fece0079070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fecf4033000 tx=0x7fecf4004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:48.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.148+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fecfc061670 con 0x7fed0415e1e0 2024-07-17T06:55:48.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.356+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed040b21b0 con 0x7fed0415e1e0 2024-07-17T06:55:48.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.913+0000 7fecfaa00640 1 -- 172.21.15.44:0/2747935904 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fecfc060f70 con 0x7fed0415e1e0 2024-07-17T06:55:48.913 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T06:55:48.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.919+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 msgr2=0x7fece0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:48.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.919+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 0x7fece0079070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fecf4033000 tx=0x7fecf4004730 comp rx=0 tx=0).stop 2024-07-17T06:55:48.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.919+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 msgr2=0x7fed04156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:48.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.919+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c700 tx=0x7fecfc00cbc0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 shutdown_connections 2024-07-17T06:55:48.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0415e1e0 0x7fed04156b40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed04154560 0x7fed04152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fece0076bb0 0x7fece0079070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 --2- 172.21.15.44:0/2747935904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed0406e690 0x7fed04152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.920+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 >> 172.21.15.44:0/2747935904 conn(0x7fed040b0eb0 msgr2=0x7fed040b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.921+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 shutdown_connections 2024-07-17T06:55:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:48.921+0000 7fed09000640 1 -- 172.21.15.44:0/2747935904 wait complete. 2024-07-17T06:55:48.948 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-07-17T06:55:49.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 -- 172.21.15.44:0/3768741694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc15e370 msgr2=0x7f25dc0ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 --2- 172.21.15.44:0/3768741694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc15e370 0x7f25dc0ce500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f25dc08c210 tx=0x7f25cc02a420 comp rx=0 tx=0).stop 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 -- 172.21.15.44:0/3768741694 shutdown_connections 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 --2- 172.21.15.44:0/3768741694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc0d0970 0x7f25dc0b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 --2- 172.21.15.44:0/3768741694 >> v2:172.21.15.175:3300/0 conn(0x7f25dc161b50 0x7f25dc0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 --2- 172.21.15.44:0/3768741694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc15e370 0x7f25dc0ce500 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 -- 172.21.15.44:0/3768741694 >> 172.21.15.44:0/3768741694 conn(0x7f25dc073a60 msgr2=0x7f25dc073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 -- 172.21.15.44:0/3768741694 shutdown_connections 2024-07-17T06:55:49.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.151+0000 7f25e0c00640 1 -- 172.21.15.44:0/3768741694 wait complete. 2024-07-17T06:55:49.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.153+0000 7f25e0c00640 1 Processor -- start 2024-07-17T06:55:49.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.153+0000 7f25e0c00640 1 -- start start 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 0x7f25dc1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc161b50 0x7f25dc157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc15e8d0 con 0x7f25dc0d0970 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc15e750 con 0x7f25dc15e370 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25e0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc0b8bb0 con 0x7f25dc161b50 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:49.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57216/0 (socket says 172.21.15.44:57216) 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.154+0000 7f25db400640 1 -- 172.21.15.44:0/973915358 learned_addr learned my addr 172.21.15.44:0/973915358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25d3e00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 0x7f25dc1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 -- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc161b50 msgr2=0x7f25dc157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25dbe00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc161b50 0x7f25dc157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc161b50 0x7f25dc157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 -- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 msgr2=0x7f25dc1f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 0x7f25dc1f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 -- 172.21.15.44:0/973915358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc157e30 con 0x7f25dc0d0970 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25d3e00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 0x7f25dc1f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:49.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.155+0000 7f25db400640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009850 tx=0x7f25cc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.156+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc03f070 con 0x7f25dc0d0970 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.156+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25cc0047a0 con 0x7f25dc0d0970 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.156+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc0ccb70 con 0x7f25dc0d0970 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.156+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc035030 con 0x7f25dc0d0970 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.156+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc1f3d00 con 0x7f25dc0d0970 2024-07-17T06:55:49.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.157+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc0ccf60 con 0x7f25dc0d0970 2024-07-17T06:55:49.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.158+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f25cc042070 con 0x7f25dc0d0970 2024-07-17T06:55:49.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.159+0000 7f25d3400640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 0x7f25b8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:49.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.159+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f25cc0b9270 con 0x7f25dc0d0970 2024-07-17T06:55:49.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.160+0000 7f25d3e00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 0x7f25b8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:49.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.161+0000 7f25d3e00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 0x7f25b8079070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25dc15ac50 tx=0x7f25c4005db0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:49.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.164+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f25dc0ccf60 con 0x7f25dc0d0970 2024-07-17T06:55:49.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.382+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25dc0d1090 con 0x7f25dc0d0970 2024-07-17T06:55:49.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.924+0000 7f25d3400640 1 -- 172.21.15.44:0/973915358 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f25dc157e30 con 0x7f25dc0d0970 2024-07-17T06:55:49.923 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data_ec' removed 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 msgr2=0x7f25b8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 0x7f25b8079070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25dc15ac50 tx=0x7f25c4005db0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 msgr2=0x7f25dc0c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009850 tx=0x7f25cc003910 comp rx=0 tx=0).stop 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 shutdown_connections 2024-07-17T06:55:49.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f25b8076bb0 0x7f25b8079070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc161b50 0x7f25dc157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc15e370 0x7f25dc1f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 --2- 172.21.15.44:0/973915358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0d0970 0x7f25dc0c9820 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:49.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.931+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 >> 172.21.15.44:0/973915358 conn(0x7f25dc073a60 msgr2=0x7f25dc15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:49.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.932+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 shutdown_connections 2024-07-17T06:55:49.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:49.932+0000 7f25e0c00640 1 -- 172.21.15.44:0/973915358 wait complete. 2024-07-17T06:55:49.954 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T06:55:49.954 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T06:55:49.955 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T06:55:49.955 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T06:55:49.955 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T06:55:49.955 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T06:55:49.955 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T06:55:49.955 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T06:55:49.955 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T06:55:49.955 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T06:55:49.956 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T06:55:49.956 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T06:55:49.956 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T06:55:49.956 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T06:55:49.956 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T06:55:49.956 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T06:55:49.978 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T06:55:49.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T06:55:49.982 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T06:55:49.982 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T06:55:49.983 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T06:55:49.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T06:55:49.984 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T06:55:49.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T06:55:50.028 INFO:tasks.ceph.mds.d:Started 2024-07-17T06:55:50.029 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:50.031 INFO:tasks.ceph.mds.c:Started 2024-07-17T06:55:50.031 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:50.032 INFO:tasks.ceph.mds.b:Started 2024-07-17T06:55:50.033 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:50.037 INFO:tasks.ceph.mds.a:Started 2024-07-17T06:55:50.037 DEBUG:teuthology.parallel:result is None 2024-07-17T06:55:50.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T06:55:50.171 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T06:55:50.171 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T06:55:50.184 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T06:55:50.187 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 -- 172.21.15.44:0/228669702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458161120 msgr2=0x7fd45806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 --2- 172.21.15.44:0/228669702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458161120 0x7fd45806d620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd440009860 tx=0x7fd44002a540 comp rx=0 tx=0).stop 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 -- 172.21.15.44:0/228669702 shutdown_connections 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 --2- 172.21.15.44:0/228669702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458161120 0x7fd45806d620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 --2- 172.21.15.44:0/228669702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458154600 0x7fd4580b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 -- 172.21.15.44:0/228669702 >> 172.21.15.44:0/228669702 conn(0x7fd4580b0eb0 msgr2=0x7fd4580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 -- 172.21.15.44:0/228669702 shutdown_connections 2024-07-17T06:55:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.226+0000 7fd45e000640 1 -- 172.21.15.44:0/228669702 wait complete. 2024-07-17T06:55:50.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.228+0000 7fd45e000640 1 Processor -- start 2024-07-17T06:55:50.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.228+0000 7fd45e000640 1 -- start start 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 0x7fd458157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4581594e0 0x7fd45815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd45815e220 con 0x7fd458154600 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd45815e0a0 con 0x7fd4581594e0 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.229+0000 7fd45e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd45815e3a0 con 0x7fd458161120 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57264/0 (socket says 172.21.15.44:57264) 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 -- 172.21.15.44:0/71045308 learned_addr learned my addr 172.21.15.44:0/71045308 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457e00640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4581594e0 0x7fd45815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd44fe00640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 0x7fd458157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 -- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 msgr2=0x7fd458157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 0x7fd458157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 -- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4581594e0 msgr2=0x7fd45815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4581594e0 0x7fd45815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 -- 172.21.15.44:0/71045308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd458158160 con 0x7fd458154600 2024-07-17T06:55:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd44fe00640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 0x7fd458157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.230+0000 7fd457400640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d950 tx=0x7fd44800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.231+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448017070 con 0x7fd458154600 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.231+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd448009d20 con 0x7fd458154600 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.231+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480106c0 con 0x7fd458154600 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.231+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd458158440 con 0x7fd458154600 2024-07-17T06:55:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.231+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd458158a00 con 0x7fd458154600 2024-07-17T06:55:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.232+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45815e8c0 con 0x7fd458154600 2024-07-17T06:55:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.233+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fd448010860 con 0x7fd458154600 2024-07-17T06:55:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.234+0000 7fd44f400640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 0x7fd434079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.234+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7fd448098500 con 0x7fd458154600 2024-07-17T06:55:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.235+0000 7fd44fe00640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 0x7fd434079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:50.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.236+0000 7fd44fe00640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 0x7fd434079070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd440004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:50.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.239+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd448061450 con 0x7fd458154600 2024-07-17T06:55:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.463+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd458160e90 con 0x7fd458154600 2024-07-17T06:55:50.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.947+0000 7fd44f400640 1 -- 172.21.15.44:0/71045308 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7fd4480659a0 con 0x7fd458154600 2024-07-17T06:55:50.947 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T06:55:50.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.951+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 msgr2=0x7fd434079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:50.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.951+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 0x7fd434079070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd440004730 comp rx=0 tx=0).stop 2024-07-17T06:55:50.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.951+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 msgr2=0x7fd45814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:50.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.951+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d950 tx=0x7fd44800de10 comp rx=0 tx=0).stop 2024-07-17T06:55:50.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 shutdown_connections 2024-07-17T06:55:50.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4581594e0 0x7fd45815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd434076bb0 0x7fd434079070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd458161120 0x7fd458157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 --2- 172.21.15.44:0/71045308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd458154600 0x7fd45814e640 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:50.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 >> 172.21.15.44:0/71045308 conn(0x7fd4580b0eb0 msgr2=0x7fd4580ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:50.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 shutdown_connections 2024-07-17T06:55:50.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:50.952+0000 7fd45e000640 1 -- 172.21.15.44:0/71045308 wait complete. 2024-07-17T06:55:50.975 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 -- 172.21.15.44:0/3833068999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f708c003aa0 msgr2=0x7f708c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 --2- 172.21.15.44:0/3833068999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f708c003aa0 0x7f708c003ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f70980016f0 tx=0x7f708c005830 comp rx=0 tx=0).stop 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70a7e00640 1 -- 172.21.15.44:0/3833068999 reap_dead start 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 -- 172.21.15.44:0/3833068999 shutdown_connections 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 -- 172.21.15.44:0/3833068999 >> 172.21.15.44:0/3833068999 conn(0x7f70a80b0e80 msgr2=0x7f70a80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 -- 172.21.15.44:0/3833068999 shutdown_connections 2024-07-17T06:55:51.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.194+0000 7f70aca00640 1 -- 172.21.15.44:0/3833068999 wait complete. 2024-07-17T06:55:51.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.196+0000 7f70aca00640 1 Processor -- start 2024-07-17T06:55:51.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.196+0000 7f70aca00640 1 -- start start 2024-07-17T06:55:51.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f708c003aa0 0x7f70a81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 0x7f70a80acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70a815a2a0 con 0x7f70a80cbdf0 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70a815a120 con 0x7f708c003aa0 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70aca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70a80b1fa0 con 0x7f70a80ce640 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57270/0 (socket says 172.21.15.44:57270) 2024-07-17T06:55:51.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70a6a00640 1 -- 172.21.15.44:0/1971228136 learned_addr learned my addr 172.21.15.44:0/1971228136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.197+0000 7f70a7400640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f708c003aa0 0x7f70a81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a7e00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 0x7f70a80acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 -- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 msgr2=0x7f70a80acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 0x7f70a80acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 -- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f708c003aa0 msgr2=0x7f70a81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f708c003aa0 0x7f70a81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 -- 172.21.15.44:0/1971228136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a80cb480 con 0x7f70a80cbdf0 2024-07-17T06:55:51.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a7e00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 0x7f70a80acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70a6a00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7094007860 tx=0x7f7094007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094017020 con 0x7f70a80cbdf0 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f709400da90 con 0x7f70a80cbdf0 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094013400 con 0x7f70a80cbdf0 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.198+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a80cd940 con 0x7f70a80cbdf0 2024-07-17T06:55:51.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.199+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a80cdbe0 con 0x7f70a80cbdf0 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.200+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a80cf380 con 0x7f70a80cbdf0 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.201+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f709401a020 con 0x7f70a80cbdf0 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.202+0000 7f709f400640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 0x7f70880790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.202+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f7094098210 con 0x7f70a80cbdf0 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.203+0000 7f70a7400640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 0x7f70880790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.204+0000 7f70a7400640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 0x7f70880790a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7098032000 tx=0x7f7098001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:51.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.206+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7094021030 con 0x7f70a80cbdf0 2024-07-17T06:55:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.497+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f70a8074ee0 con 0x7f70a80cbdf0 2024-07-17T06:55:51.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.498+0000 7f709f400640 1 -- 172.21.15.44:0/1971228136 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+6645 (secure 0 0 0) 0x7f7094061800 con 0x7f70a80cbdf0 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T06:55:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T06:55:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T06:55:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T06:55:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T06:55:51.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-mds", 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdm5Ec1HRAAygcLWCf9cdmsFxy/mWRuXw==", 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-mds" 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.510 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-mgr", 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdmrmQ1HRAAa9uUYJtgwDFFpjyKDIDW0g==", 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-mgr" 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-osd", 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdmmYU1HRAAGW4qBH/XJS5yMf9RCc+Hsw==", 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-osd" 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rbd", 2024-07-17T06:55:51.511 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdm8aE1HRAAdP633HIaE8THz6Hfx/x/7A==", 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rbd" 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdmArs1HRAABuYB9oA4V/M5oguxJ9USLQ==", 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.bootstrap-rgw", 2024-07-17T06:55:51.512 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCkapdmEtQ1HRAAMUXIpTVcmS2mHXQzE/YyWA==", 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile bootstrap-rgw" 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:51.513 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T06:55:51.514 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 msgr2=0x7f70880790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 0x7f70880790a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7098032000 tx=0x7f7098001190 comp rx=0 tx=0).stop 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 msgr2=0x7f70a80ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7094007860 tx=0x7f7094007d20 comp rx=0 tx=0).stop 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 shutdown_connections 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7088076be0 0x7f70880790a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a80ce640 0x7f70a80acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70a80cbdf0 0x7f70a80ce100 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 --2- 172.21.15.44:0/1971228136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f708c003aa0 0x7f70a81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 >> 172.21.15.44:0/1971228136 conn(0x7f70a80b0e80 msgr2=0x7f70a815a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 shutdown_connections 2024-07-17T06:55:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.501+0000 7f70aca00640 1 -- 172.21.15.44:0/1971228136 wait complete. 2024-07-17T06:55:51.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-07-17T06:55:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 -- 172.21.15.44:0/3238107011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680ce5c0 msgr2=0x7f0e6815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 --2- 172.21.15.44:0/3238107011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680ce5c0 0x7f0e6815d2c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009860 tx=0x7f0e5002a540 comp rx=0 tx=0).stop 2024-07-17T06:55:51.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 -- 172.21.15.44:0/3238107011 shutdown_connections 2024-07-17T06:55:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 --2- 172.21.15.44:0/3238107011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e680b9060 0x7f0e6815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 --2- 172.21.15.44:0/3238107011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680ce5c0 0x7f0e6815d2c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 -- 172.21.15.44:0/3238107011 >> 172.21.15.44:0/3238107011 conn(0x7f0e680cfec0 msgr2=0x7f0e680d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 -- 172.21.15.44:0/3238107011 shutdown_connections 2024-07-17T06:55:51.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.730+0000 7f0e6d400640 1 -- 172.21.15.44:0/3238107011 wait complete. 2024-07-17T06:55:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.732+0000 7f0e6d400640 1 Processor -- start 2024-07-17T06:55:51.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.732+0000 7f0e6d400640 1 -- start start 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 0x7f0e680ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e680ce5c0 0x7f0e680aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6815e0d0 con 0x7f0e680b9060 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6815df50 con 0x7f0e680b8c80 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.733+0000 7f0e6d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6815e250 con 0x7f0e680ce5c0 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57278/0 (socket says 172.21.15.44:57278) 2024-07-17T06:55:51.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 -- 172.21.15.44:0/1750240125 learned_addr learned my addr 172.21.15.44:0/1750240125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 -- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e680ce5c0 msgr2=0x7f0e680aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e680ce5c0 0x7f0e680aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 -- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 msgr2=0x7f0e680ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e67e00640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 0x7f0e680ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 0x7f0e680ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e5f400640 1 -- 172.21.15.44:0/1750240125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e680aa4b0 con 0x7f0e680b9060 2024-07-17T06:55:51.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.734+0000 7f0e67e00640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 0x7f0e680ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:51.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e5f400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009980 tx=0x7f0e50033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:51.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031d60 con 0x7f0e680b9060 2024-07-17T06:55:51.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e50033e10 con 0x7f0e680b9060 2024-07-17T06:55:51.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e681f3e00 con 0x7f0e680b9060 2024-07-17T06:55:51.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50005280 con 0x7f0e680b9060 2024-07-17T06:55:51.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.735+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e681f43c0 con 0x7f0e680b9060 2024-07-17T06:55:51.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.736+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6815e700 con 0x7f0e680b9060 2024-07-17T06:55:51.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.737+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f0e50005420 con 0x7f0e680b9060 2024-07-17T06:55:51.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.738+0000 7f0e64c00640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 0x7f0e44079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:51.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.739+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f0e500b9130 con 0x7f0e680b9060 2024-07-17T06:55:51.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.739+0000 7f0e67e00640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 0x7f0e44079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:51.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.740+0000 7f0e67e00640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 0x7f0e44079070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e58000f60 tx=0x7f0e58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:51.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:51.743+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0e5003f030 con 0x7f0e680b9060 2024-07-17T06:55:52.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.036+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0e680b5d70 con 0x7f0e680b9060 2024-07-17T06:55:52.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.040+0000 7f0e64c00640 1 -- 172.21.15.44:0/1750240125 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v23) ==== 90+0+0 (secure 0 0 0) 0x7f0e500827b0 con 0x7f0e680b9060 2024-07-17T06:55:52.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 msgr2=0x7f0e44079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 0x7f0e44079070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e58000f60 tx=0x7f0e58009040 comp rx=0 tx=0).stop 2024-07-17T06:55:52.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 msgr2=0x7f0e680a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009980 tx=0x7f0e50033c20 comp rx=0 tx=0).stop 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 shutdown_connections 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0e44076bb0 0x7f0e44079070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e680ce5c0 0x7f0e680aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e680b9060 0x7f0e680a9f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 --2- 172.21.15.44:0/1750240125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e680b8c80 0x7f0e680ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.043+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 >> 172.21.15.44:0/1750240125 conn(0x7f0e680cfec0 msgr2=0x7f0e680b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.044+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 shutdown_connections 2024-07-17T06:55:52.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.044+0000 7f0e6d400640 1 -- 172.21.15.44:0/1750240125 wait complete. 2024-07-17T06:55:52.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-07-17T06:55:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 -- 172.21.15.44:0/2154459606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 msgr2=0x7f8c44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/2154459606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c44004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8c60162020 tx=0x7f8c44005830 comp rx=0 tx=0).stop 2024-07-17T06:55:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5f400640 1 -- 172.21.15.44:0/2154459606 reap_dead start 2024-07-17T06:55:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 -- 172.21.15.44:0/2154459606 shutdown_connections 2024-07-17T06:55:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 -- 172.21.15.44:0/2154459606 >> 172.21.15.44:0/2154459606 conn(0x7f8c600b4ff0 msgr2=0x7f8c600b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 -- 172.21.15.44:0/2154459606 shutdown_connections 2024-07-17T06:55:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.274+0000 7f8c5fe00640 1 -- 172.21.15.44:0/2154459606 wait complete. 2024-07-17T06:55:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.276+0000 7f8c5fe00640 1 Processor -- start 2024-07-17T06:55:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.276+0000 7f8c5fe00640 1 -- start start 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 0x7f8c60157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 0x7f8c60152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6015d8a0 con 0x7f8c44004800 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6015d720 con 0x7f8c601527e0 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.277+0000 7f8c5fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c600d15c0 con 0x7f8c60157700 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57302/0 (socket says 172.21.15.44:57302) 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 -- 172.21.15.44:0/363654532 learned_addr learned my addr 172.21.15.44:0/363654532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c57e00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 0x7f8c60157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5f400640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 0x7f8c60152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 -- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 msgr2=0x7f8c60157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 0x7f8c60157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 -- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 msgr2=0x7f8c60152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 0x7f8c60152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5ea00640 1 -- 172.21.15.44:0/363654532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c600ca360 con 0x7f8c44004800 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c57e00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 0x7f8c60157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.278+0000 7f8c5f400640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 0x7f8c60152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c5ea00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c50011a50 tx=0x7f8c50011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5001f070 con 0x7f8c44004800 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c5000fdc0 con 0x7f8c44004800 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500126b0 con 0x7f8c44004800 2024-07-17T06:55:52.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c601f7440 con 0x7f8c44004800 2024-07-17T06:55:52.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.279+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601f7a00 con 0x7f8c44004800 2024-07-17T06:55:52.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.280+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c600cf9d0 con 0x7f8c44004800 2024-07-17T06:55:52.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.281+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f8c50022070 con 0x7f8c44004800 2024-07-17T06:55:52.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.282+0000 7f8c56a00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 0x7f8c3c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.282+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f8c50098a60 con 0x7f8c44004800 2024-07-17T06:55:52.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.283+0000 7f8c57e00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 0x7f8c3c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.284+0000 7f8c57e00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 0x7f8c3c0790a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035000 tx=0x7f8c4802d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:52.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.287+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8c600cf9d0 con 0x7f8c44004800 2024-07-17T06:55:52.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.576+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8c600d0e10 con 0x7f8c44004800 2024-07-17T06:55:52.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.581+0000 7f8c56a00640 1 -- 172.21.15.44:0/363654532 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7f8c50061a00 con 0x7f8c44004800 2024-07-17T06:55:52.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.583+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 msgr2=0x7f8c3c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.583+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 0x7f8c3c0790a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035000 tx=0x7f8c4802d040 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.583+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 msgr2=0x7f8c600b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.583+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c50011a50 tx=0x7f8c50011f10 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 shutdown_connections 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c601527e0 0x7f8c60152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8c3c076be0 0x7f8c3c0790a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c60157700 0x7f8c60157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 --2- 172.21.15.44:0/363654532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c44004800 0x7f8c600b41e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 >> 172.21.15.44:0/363654532 conn(0x7f8c600b4ff0 msgr2=0x7f8c600b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 shutdown_connections 2024-07-17T06:55:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.584+0000 7f8c5fe00640 1 -- 172.21.15.44:0/363654532 wait complete. 2024-07-17T06:55:52.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-07-17T06:55:52.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2825521065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 msgr2=0x7fbba40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2825521065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbba40051a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c560 tx=0x7fbba4005830 comp rx=0 tx=0).stop 2024-07-17T06:55:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbbe800640 1 -- 172.21.15.44:0/2825521065 reap_dead start 2024-07-17T06:55:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2825521065 shutdown_connections 2024-07-17T06:55:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2825521065 >> 172.21.15.44:0/2825521065 conn(0x7fbbc00b0eb0 msgr2=0x7fbbc00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2825521065 shutdown_connections 2024-07-17T06:55:52.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.796+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2825521065 wait complete. 2024-07-17T06:55:52.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.798+0000 7fbbc4a00640 1 Processor -- start 2024-07-17T06:55:52.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.798+0000 7fbbc4a00640 1 -- start start 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbc01f7f30 0x7fbbc01550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbc00ad630 0x7fbbc014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc015c470 con 0x7fbba4004d40 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc015c2f0 con 0x7fbbc01f7f30 2024-07-17T06:55:52.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbc4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc015c5f0 con 0x7fbbc00ad630 2024-07-17T06:55:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57320/0 (socket says 172.21.15.44:57320) 2024-07-17T06:55:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.799+0000 7fbbbde00640 1 -- 172.21.15.44:0/2527519057 learned_addr learned my addr 172.21.15.44:0/2527519057 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:52.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 -- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbc00ad630 msgr2=0x7fbbc014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbe800640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbc00ad630 0x7fbbc014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbd400640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbc01f7f30 0x7fbbc01550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbc00ad630 0x7fbbc014ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 -- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbc01f7f30 msgr2=0x7fbbc01550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbc01f7f30 0x7fbbc01550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 -- 172.21.15.44:0/2527519057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00ac5d0 con 0x7fbba4004d40 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.800+0000 7fbbbde00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0004e20 tx=0x7fbbb0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.801+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0015020 con 0x7fbba4004d40 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.801+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbb0011960 con 0x7fbba4004d40 2024-07-17T06:55:52.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.801+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc0155610 con 0x7fbba4004d40 2024-07-17T06:55:52.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.801+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0011d70 con 0x7fbba4004d40 2024-07-17T06:55:52.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.801+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc00acb50 con 0x7fbba4004d40 2024-07-17T06:55:52.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.802+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc00cd2e0 con 0x7fbba4004d40 2024-07-17T06:55:52.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.803+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fbbb0018020 con 0x7fbba4004d40 2024-07-17T06:55:52.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.804+0000 7fbbb6000640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 0x7fbba00790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:52.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.804+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fbbb0098c00 con 0x7fbba4004d40 2024-07-17T06:55:52.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.805+0000 7fbbbd400640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 0x7fbba00790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:52.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.806+0000 7fbbbd400640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 0x7fbba00790a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00ac560 tx=0x7fbbac005f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:52.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:52.809+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbbb001f020 con 0x7fbba4004d40 2024-07-17T06:55:53.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.098+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fbbc00ad280 con 0x7fbba4004d40 2024-07-17T06:55:53.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.102+0000 7fbbb6000640 1 -- 172.21.15.44:0/2527519057 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7fbbc00ad280 con 0x7fbba4004d40 2024-07-17T06:55:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 msgr2=0x7fbba00790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 0x7fbba00790a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00ac560 tx=0x7fbbac005f90 comp rx=0 tx=0).stop 2024-07-17T06:55:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 msgr2=0x7fbbc0150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0004e20 tx=0x7fbbb0001630 comp rx=0 tx=0).stop 2024-07-17T06:55:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 shutdown_connections 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fbba0076be0 0x7fbba00790a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbc00ad630 0x7fbbc014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbc01f7f30 0x7fbbc01550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 --2- 172.21.15.44:0/2527519057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbba4004d40 0x7fbbc0150470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.104+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 >> 172.21.15.44:0/2527519057 conn(0x7fbbc00b0eb0 msgr2=0x7fbbc015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.105+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 shutdown_connections 2024-07-17T06:55:53.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.105+0000 7fbbc4a00640 1 -- 172.21.15.44:0/2527519057 wait complete. 2024-07-17T06:55:53.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-07-17T06:55:53.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 -- 172.21.15.44:0/1196089428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 msgr2=0x7f36fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 --2- 172.21.15.44:0/1196089428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f36fc00b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c640 tx=0x7f36fc005830 comp rx=0 tx=0).stop 2024-07-17T06:55:53.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f3719600640 1 -- 172.21.15.44:0/1196089428 reap_dead start 2024-07-17T06:55:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 -- 172.21.15.44:0/1196089428 shutdown_connections 2024-07-17T06:55:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 -- 172.21.15.44:0/1196089428 >> 172.21.15.44:0/1196089428 conn(0x7f37140d3700 msgr2=0x7f37140d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 -- 172.21.15.44:0/1196089428 shutdown_connections 2024-07-17T06:55:53.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.294+0000 7f371b600640 1 -- 172.21.15.44:0/1196089428 wait complete. 2024-07-17T06:55:53.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.296+0000 7f371b600640 1 Processor -- start 2024-07-17T06:55:53.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.296+0000 7f371b600640 1 -- start start 2024-07-17T06:55:53.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3714160750 0x7f37141f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 0x7f37141f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3714162cf0 con 0x7f36fc004e60 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3714162b70 con 0x7f3714160bd0 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.297+0000 7f371b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3714162e70 con 0x7f3714160750 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57340/0 (socket says 172.21.15.44:57340) 2024-07-17T06:55:53.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 -- 172.21.15.44:0/703274265 learned_addr learned my addr 172.21.15.44:0/703274265 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:53.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3719600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 0x7f37141f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 -- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3714160750 msgr2=0x7f37141f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3714160750 0x7f37141f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 -- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 msgr2=0x7f37141f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 0x7f37141f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 -- 172.21.15.44:0/703274265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37141f49f0 con 0x7f36fc004e60 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3719600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 0x7f37141f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.298+0000 7f3718c00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3708004eb0 tx=0x7f3708001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.299+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708015020 con 0x7f36fc004e60 2024-07-17T06:55:53.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.299+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3708011960 con 0x7f36fc004e60 2024-07-17T06:55:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.299+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708011d90 con 0x7f36fc004e60 2024-07-17T06:55:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.299+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37141f4cd0 con 0x7f36fc004e60 2024-07-17T06:55:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.299+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37140c87d0 con 0x7f36fc004e60 2024-07-17T06:55:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.300+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371406a1a0 con 0x7f36fc004e60 2024-07-17T06:55:53.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.301+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f3708018020 con 0x7f36fc004e60 2024-07-17T06:55:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.302+0000 7f3710c00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 0x7f36f4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.302+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3708098bc0 con 0x7f36fc004e60 2024-07-17T06:55:53.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.303+0000 7f3713e00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 0x7f36f4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.304+0000 7f3713e00640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 0x7f36f4079070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3700005fd0 tx=0x7f3700005d40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:53.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.307+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f371406a1a0 con 0x7f36fc004e60 2024-07-17T06:55:53.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.627+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f37140ce5b0 con 0x7f36fc004e60 2024-07-17T06:55:53.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.631+0000 7f3710c00640 1 -- 172.21.15.44:0/703274265 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7f370801f020 con 0x7f36fc004e60 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.633+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 msgr2=0x7f36f4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 0x7f36f4079070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3700005fd0 tx=0x7f3700005d40 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 msgr2=0x7f37140c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3708004eb0 tx=0x7f3708001630 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 shutdown_connections 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3714160bd0 0x7f37141f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f36f4076bb0 0x7f36f4079070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3714160750 0x7f37141f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 --2- 172.21.15.44:0/703274265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36fc004e60 0x7f37140c8fd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 >> 172.21.15.44:0/703274265 conn(0x7f37140d3700 msgr2=0x7f3714161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 shutdown_connections 2024-07-17T06:55:53.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.634+0000 7f371b600640 1 -- 172.21.15.44:0/703274265 wait complete. 2024-07-17T06:55:53.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-07-17T06:55:53.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 -- 172.21.15.44:0/2699140602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c157d10 msgr2=0x7f591c15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 --2- 172.21.15.44:0/2699140602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c157d10 0x7f591c15b180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f591c06deb0 tx=0x7f590c02a530 comp rx=0 tx=0).stop 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 -- 172.21.15.44:0/2699140602 shutdown_connections 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 --2- 172.21.15.44:0/2699140602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c0afb30 0x7f591c06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 --2- 172.21.15.44:0/2699140602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c157d10 0x7f591c15b180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 -- 172.21.15.44:0/2699140602 >> 172.21.15.44:0/2699140602 conn(0x7f591c0b2240 msgr2=0x7f591c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 -- 172.21.15.44:0/2699140602 shutdown_connections 2024-07-17T06:55:53.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.832+0000 7f5921200640 1 -- 172.21.15.44:0/2699140602 wait complete. 2024-07-17T06:55:53.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.834+0000 7f5921200640 1 Processor -- start 2024-07-17T06:55:53.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.834+0000 7f5921200640 1 -- start start 2024-07-17T06:55:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 0x7f591c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 0x7f591c1479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0cebb0 con 0x7f591c0702d0 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0cea30 con 0x7f591c0afb30 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5921200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f591c070740 con 0x7f591c157d10 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5919e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5919e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57370/0 (socket says 172.21.15.44:57370) 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.835+0000 7f5919e00640 1 -- 172.21.15.44:0/4154669380 learned_addr learned my addr 172.21.15.44:0/4154669380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919400640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 0x7f591c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 -- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 msgr2=0x7f591c1479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f591a800640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 0x7f591c1479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 0x7f591c1479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 -- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 msgr2=0x7f591c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 0x7f591c0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 -- 172.21.15.44:0/4154669380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c14bc30 con 0x7f591c0702d0 2024-07-17T06:55:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919400640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 0x7f591c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:53.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f591a800640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 0x7f591c1479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5919e00640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f590c009850 tx=0x7f590c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c03e030 con 0x7f591c0702d0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f590c0047a0 con 0x7f591c0702d0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.836+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1535c0 con 0x7f591c0702d0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.837+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c035040 con 0x7f591c0702d0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.837+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c0b3eb0 con 0x7f591c0702d0 2024-07-17T06:55:53.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.838+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c06f6b0 con 0x7f591c0702d0 2024-07-17T06:55:53.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.839+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f590c041030 con 0x7f591c0702d0 2024-07-17T06:55:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.840+0000 7f5912000640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 0x7f58fc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.840+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f590c0b8c80 con 0x7f591c0702d0 2024-07-17T06:55:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.841+0000 7f5919400640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 0x7f58fc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.842+0000 7f5919400640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 0x7f58fc079070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5908006fd0 tx=0x7f5908008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:53.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:53.843+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f590c082380 con 0x7f591c0702d0 2024-07-17T06:55:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.120+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f591c0cf1b0 con 0x7f591c0702d0 2024-07-17T06:55:54.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.125+0000 7f5912000640 1 -- 172.21.15.44:0/4154669380 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v27) ==== 97+0+0 (secure 0 0 0) 0x7f590c081c80 con 0x7f591c0702d0 2024-07-17T06:55:54.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 msgr2=0x7f58fc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 0x7f58fc079070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5908006fd0 tx=0x7f5908008040 comp rx=0 tx=0).stop 2024-07-17T06:55:54.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 msgr2=0x7f591c1492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f590c009850 tx=0x7f590c0039e0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 shutdown_connections 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f58fc076bb0 0x7f58fc079070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f591c157d10 0x7f591c1479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f591c0afb30 0x7f591c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 --2- 172.21.15.44:0/4154669380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f591c0702d0 0x7f591c1492b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 >> 172.21.15.44:0/4154669380 conn(0x7f591c0b2240 msgr2=0x7f591c0cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.128+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 shutdown_connections 2024-07-17T06:55:54.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.129+0000 7f5921200640 1 -- 172.21.15.44:0/4154669380 wait complete. 2024-07-17T06:55:54.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 -- 172.21.15.44:0/575352244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 msgr2=0x7f1f5415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 --2- 172.21.15.44:0/575352244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5415a5a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005fd0 tx=0x7f1f4402b860 comp rx=0 tx=0).stop 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 -- 172.21.15.44:0/575352244 shutdown_connections 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 --2- 172.21.15.44:0/575352244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f540b3280 0x7f1f5406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 --2- 172.21.15.44:0/575352244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5415a5a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 -- 172.21.15.44:0/575352244 >> 172.21.15.44:0/575352244 conn(0x7f1f540b0eb0 msgr2=0x7f1f540b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:54.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 -- 172.21.15.44:0/575352244 shutdown_connections 2024-07-17T06:55:54.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.331+0000 7f1f5c000640 1 -- 172.21.15.44:0/575352244 wait complete. 2024-07-17T06:55:54.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.333+0000 7f1f5c000640 1 Processor -- start 2024-07-17T06:55:54.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.333+0000 7f1f5c000640 1 -- start start 2024-07-17T06:55:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f5406f6d0 0x7f1f541f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f540b3280 0x7f1f54073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540aea80 con 0x7f1f540acab0 2024-07-17T06:55:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f540ae900 con 0x7f1f5406f6d0 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f5c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f540aec00 con 0x7f1f540b3280 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57386/0 (socket says 172.21.15.44:57386) 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 -- 172.21.15.44:0/412412915 learned_addr learned my addr 172.21.15.44:0/412412915 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 -- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f540b3280 msgr2=0x7f1f54073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f540b3280 0x7f1f54073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 -- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f5406f6d0 msgr2=0x7f1f541f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f5406f6d0 0x7f1f541f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.334+0000 7f1f58c00640 1 -- 172.21.15.44:0/412412915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f54151990 con 0x7f1f540acab0 2024-07-17T06:55:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f58c00640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f44034040 tx=0x7f1f440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44040030 con 0x7f1f540acab0 2024-07-17T06:55:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f44003d90 con 0x7f1f540acab0 2024-07-17T06:55:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f440047a0 con 0x7f1f540acab0 2024-07-17T06:55:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f54151c70 con 0x7f1f540acab0 2024-07-17T06:55:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.335+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f54152230 con 0x7f1f540acab0 2024-07-17T06:55:54.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.336+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5415a5a0 con 0x7f1f540acab0 2024-07-17T06:55:54.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.337+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f1f44043030 con 0x7f1f540acab0 2024-07-17T06:55:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.338+0000 7f1f51600640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 0x7f1f38079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.339+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1f440b8d10 con 0x7f1f540acab0 2024-07-17T06:55:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.339+0000 7f1f59600640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 0x7f1f38079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.340+0000 7f1f59600640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 0x7f1f38079070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f48001560 tx=0x7f1f48007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:54.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.343+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1f440be020 con 0x7f1f540acab0 2024-07-17T06:55:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.641+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f1f540b4200 con 0x7f1f540acab0 2024-07-17T06:55:54.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.646+0000 7f1f51600640 1 -- 172.21.15.44:0/412412915 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7f1f44082390 con 0x7f1f540acab0 2024-07-17T06:55:54.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 msgr2=0x7f1f38079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 0x7f1f38079070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f48001560 tx=0x7f1f48007990 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 msgr2=0x7f1f5414c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f44034040 tx=0x7f1f440038a0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 shutdown_connections 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1f38076bb0 0x7f1f38079070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f540b3280 0x7f1f54073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f540acab0 0x7f1f5414c2e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 --2- 172.21.15.44:0/412412915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f5406f6d0 0x7f1f541f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.649+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 >> 172.21.15.44:0/412412915 conn(0x7f1f540b0eb0 msgr2=0x7f1f540afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.650+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 shutdown_connections 2024-07-17T06:55:54.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.650+0000 7f1f5c000640 1 -- 172.21.15.44:0/412412915 wait complete. 2024-07-17T06:55:54.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T06:55:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.884+0000 7fca10a00640 1 -- 172.21.15.44:0/1177825047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f0004d40 msgr2=0x7fc9f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.884+0000 7fca10a00640 1 --2- 172.21.15.44:0/1177825047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f0004d40 0x7fc9f00051c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fca0400f460 tx=0x7fc9f0005830 comp rx=0 tx=0).stop 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.884+0000 7fca0a800640 1 -- 172.21.15.44:0/1177825047 reap_dead start 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.884+0000 7fca10a00640 1 -- 172.21.15.44:0/1177825047 shutdown_connections 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.884+0000 7fca10a00640 1 -- 172.21.15.44:0/1177825047 >> 172.21.15.44:0/1177825047 conn(0x7fca0c0b0fb0 msgr2=0x7fca0c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.885+0000 7fca10a00640 1 -- 172.21.15.44:0/1177825047 shutdown_connections 2024-07-17T06:55:54.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.885+0000 7fca10a00640 1 -- 172.21.15.44:0/1177825047 wait complete. 2024-07-17T06:55:54.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.887+0000 7fca10a00640 1 Processor -- start 2024-07-17T06:55:54.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.887+0000 7fca10a00640 1 -- start start 2024-07-17T06:55:54.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f0004d40 0x7fca0c1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1f26a0 0x7fca0c1f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c159b00 con 0x7fca0c15d070 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c0acd90 con 0x7fca0c1f26a0 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.888+0000 7fca10a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca0c159c80 con 0x7fc9f0004d40 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57402/0 (socket says 172.21.15.44:57402) 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 -- 172.21.15.44:0/4265561904 learned_addr learned my addr 172.21.15.44:0/4265561904 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca0a800640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1f26a0 0x7fca0c1f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09e00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f0004d40 0x7fca0c1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 -- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f0004d40 msgr2=0x7fca0c1f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f0004d40 0x7fca0c1f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 -- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1f26a0 msgr2=0x7fca0c1f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1f26a0 0x7fca0c1f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:54.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.889+0000 7fca09400640 1 -- 172.21.15.44:0/4265561904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c154360 con 0x7fca0c15d070 2024-07-17T06:55:54.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca09400640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800c530 tx=0x7fc9f800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80052d0 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f8005490 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c154640 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8012890 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.890+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c154c00 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.891+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0c15da10 con 0x7fca0c15d070 2024-07-17T06:55:54.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.893+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc9f80035f0 con 0x7fca0c15d070 2024-07-17T06:55:54.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.894+0000 7fca02000640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 0x7fc9ec079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:54.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.894+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc9f8097de0 con 0x7fca0c15d070 2024-07-17T06:55:54.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.895+0000 7fca09e00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 0x7fc9ec079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:54.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.896+0000 7fca09e00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 0x7fc9ec079070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001eb0 tx=0x7fc9fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:54.898+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc9f80613d0 con 0x7fca0c15d070 2024-07-17T06:55:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.114+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fca0c0b44c0 con 0x7fca0c15d070 2024-07-17T06:55:55.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.114+0000 7fca02000640 1 -- 172.21.15.44:0/4265561904 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7fca0c0b44c0 con 0x7fca0c15d070 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:55:55.114 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T06:55:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 msgr2=0x7fc9ec079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 0x7fc9ec079070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001eb0 tx=0x7fc9fc009040 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 msgr2=0x7fca0c15d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800c530 tx=0x7fc9f800c9f0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 shutdown_connections 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca0c1f26a0 0x7fca0c1f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca0c15d070 0x7fca0c15d4d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc9ec076bb0 0x7fc9ec079070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 --2- 172.21.15.44:0/4265561904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f0004d40 0x7fca0c1f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.117+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 >> 172.21.15.44:0/4265561904 conn(0x7fca0c0b0fb0 msgr2=0x7fca0c14e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.118+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 shutdown_connections 2024-07-17T06:55:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.118+0000 7fca10a00640 1 -- 172.21.15.44:0/4265561904 wait complete. 2024-07-17T06:55:55.136 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T06:55:55.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T06:55:55.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 -- 172.21.15.44:0/1624866593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4480ae820 msgr2=0x7fc4480abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 --2- 172.21.15.44:0/1624866593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4480ae820 0x7fc4480abfa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc440005fd0 tx=0x7fc44002a8b0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 -- 172.21.15.44:0/1624866593 shutdown_connections 2024-07-17T06:55:55.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 --2- 172.21.15.44:0/1624866593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4480ae820 0x7fc4480abfa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 -- 172.21.15.44:0/1624866593 >> 172.21.15.44:0/1624866593 conn(0x7fc4480cf050 msgr2=0x7fc4480cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:55.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 -- 172.21.15.44:0/1624866593 shutdown_connections 2024-07-17T06:55:55.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.354+0000 7fc447e00640 1 -- 172.21.15.44:0/1624866593 wait complete. 2024-07-17T06:55:55.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.356+0000 7fc447e00640 1 Processor -- start 2024-07-17T06:55:55.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.356+0000 7fc447e00640 1 -- start start 2024-07-17T06:55:55.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc44806e6d0 0x7fc4481f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:55.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:55.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ae820 0x7fc4480aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:55.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4480ac870 con 0x7fc44806f460 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4480ac6f0 con 0x7fc44806e6d0 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4480b26c0 con 0x7fc4480ae820 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc446000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc446000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57434/0 (socket says 172.21.15.44:57434) 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc446000640 1 -- 172.21.15.44:0/2134768374 learned_addr learned my addr 172.21.15.44:0/2134768374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc446a00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc44806e6d0 0x7fc4481f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.357+0000 7fc447400640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ae820 0x7fc4480aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 -- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ae820 msgr2=0x7fc4480aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ae820 0x7fc4480aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 -- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc44806e6d0 msgr2=0x7fc4481f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc44806e6d0 0x7fc4481f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 -- 172.21.15.44:0/2134768374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4481f46a0 con 0x7fc44806f460 2024-07-17T06:55:55.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc446000640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c410 tx=0x7fc43400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:55.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434017070 con 0x7fc44806f460 2024-07-17T06:55:55.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4340035f0 con 0x7fc44806f460 2024-07-17T06:55:55.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434003a80 con 0x7fc44806f460 2024-07-17T06:55:55.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4481f4980 con 0x7fc44806f460 2024-07-17T06:55:55.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.358+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481f4380 con 0x7fc44806f460 2024-07-17T06:55:55.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.359+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44815e310 con 0x7fc44806f460 2024-07-17T06:55:55.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.361+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fc43401a070 con 0x7fc44806f460 2024-07-17T06:55:55.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.362+0000 7fc43ea00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 0x7fc4280790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:55.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.362+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc4340975c0 con 0x7fc44806f460 2024-07-17T06:55:55.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.362+0000 7fc446a00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 0x7fc4280790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:55.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.363+0000 7fc446a00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 0x7fc4280790a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc438001700 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:55.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.366+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc44815e310 con 0x7fc44806f460 2024-07-17T06:55:55.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.579+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc4481e7c90 con 0x7fc44806f460 2024-07-17T06:55:55.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.977+0000 7fc43ea00640 1 -- 172.21.15.44:0/2134768374 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fc434060c40 con 0x7fc44806f460 2024-07-17T06:55:55.977 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T06:55:55.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.981+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 msgr2=0x7fc4280790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.981+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 0x7fc4280790a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc438001700 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-07-17T06:55:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.981+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 msgr2=0x7fc448149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.981+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c410 tx=0x7fc43400c8d0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 shutdown_connections 2024-07-17T06:55:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fc428076be0 0x7fc4280790a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ae820 0x7fc4480aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc44806f460 0x7fc448149ff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 --2- 172.21.15.44:0/2134768374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc44806e6d0 0x7fc4481f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:55.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 >> 172.21.15.44:0/2134768374 conn(0x7fc4480cf050 msgr2=0x7fc44815ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:55.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 shutdown_connections 2024-07-17T06:55:55.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:55.982+0000 7fc447e00640 1 -- 172.21.15.44:0/2134768374 wait complete. 2024-07-17T06:55:56.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 -- 172.21.15.44:0/4151209810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0806d420 msgr2=0x7ffa0806d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/4151209810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0806d420 0x7ffa0806d8b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005fd0 tx=0x7ffa0002bbc0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 -- 172.21.15.44:0/4151209810 shutdown_connections 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/4151209810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0806d420 0x7ffa0806d8b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/4151209810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0806f4d0 0x7ffa0815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 -- 172.21.15.44:0/4151209810 >> 172.21.15.44:0/4151209810 conn(0x7ffa080cf0d0 msgr2=0x7ffa080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 -- 172.21.15.44:0/4151209810 shutdown_connections 2024-07-17T06:55:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.216+0000 7ffa0ce00640 1 -- 172.21.15.44:0/4151209810 wait complete. 2024-07-17T06:55:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.218+0000 7ffa0ce00640 1 Processor -- start 2024-07-17T06:55:56.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.218+0000 7ffa0ce00640 1 -- start start 2024-07-17T06:55:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0806d420 0x7ffa081f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:56.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0806f4d0 0x7ffa081e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa080b33e0 con 0x7ffa081503c0 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa080b3260 con 0x7ffa0806d420 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa0ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa080b3560 con 0x7ffa0806f4d0 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.219+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57454/0 (socket says 172.21.15.44:57454) 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 -- 172.21.15.44:0/1540640644 learned_addr learned my addr 172.21.15.44:0/1540640644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 -- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0806f4d0 msgr2=0x7ffa081e4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:56.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0806f4d0 0x7ffa081e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 -- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0806d420 msgr2=0x7ffa081f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0806d420 0x7ffa081f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 -- 172.21.15.44:0/1540640644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa081f7b30 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.220+0000 7ffa07e00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ade0 tx=0x7ffa0002d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.221+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00042020 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.221+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa00004330 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.221+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0003a040 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.221+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa08152050 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.221+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081517b0 con 0x7ffa081503c0 2024-07-17T06:55:56.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.222+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0806d8b0 con 0x7ffa081503c0 2024-07-17T06:55:56.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.223+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ffa00045020 con 0x7ffa081503c0 2024-07-17T06:55:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.225+0000 7ff9ffe00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 0x7ff9e4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.225+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7ffa000b9800 con 0x7ffa081503c0 2024-07-17T06:55:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.226+0000 7ffa07400640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 0x7ff9e4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:56.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.227+0000 7ffa07400640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 0x7ff9e4079070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005fd0 tx=0x7ff9f8005850 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:56.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.230+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffa0004c030 con 0x7ffa081503c0 2024-07-17T06:55:56.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.442+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffa080cffe0 con 0x7ffa081503c0 2024-07-17T06:55:56.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.985+0000 7ff9ffe00640 1 -- 172.21.15.44:0/1540640644 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7ffa00082d10 con 0x7ffa081503c0 2024-07-17T06:55:56.984 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T06:55:56.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.988+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 msgr2=0x7ff9e4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:56.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.988+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 0x7ff9e4079070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8005fd0 tx=0x7ff9f8005850 comp rx=0 tx=0).stop 2024-07-17T06:55:56.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.988+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 msgr2=0x7ffa08152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:56.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.988+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ade0 tx=0x7ffa0002d950 comp rx=0 tx=0).stop 2024-07-17T06:55:56.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 shutdown_connections 2024-07-17T06:55:56.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa081503c0 0x7ffa08152260 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff9e4076bb0 0x7ff9e4079070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0806f4d0 0x7ffa081e4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 --2- 172.21.15.44:0/1540640644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa0806d420 0x7ffa081f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:56.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 >> 172.21.15.44:0/1540640644 conn(0x7ffa080cf0d0 msgr2=0x7ffa08068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:56.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 shutdown_connections 2024-07-17T06:55:56.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:56.989+0000 7ffa0ce00640 1 -- 172.21.15.44:0/1540640644 wait complete. 2024-07-17T06:55:57.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T06:55:57.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 -- 172.21.15.44:0/510246428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 msgr2=0x7f8c00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:57.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 --2- 172.21.15.44:0/510246428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c00004800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009b50 tx=0x7f8c00005830 comp rx=0 tx=0).stop 2024-07-17T06:55:57.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c1a800640 1 -- 172.21.15.44:0/510246428 reap_dead start 2024-07-17T06:55:57.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 -- 172.21.15.44:0/510246428 shutdown_connections 2024-07-17T06:55:57.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 -- 172.21.15.44:0/510246428 >> 172.21.15.44:0/510246428 conn(0x7f8c1c0b0eb0 msgr2=0x7f8c1c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:57.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 -- 172.21.15.44:0/510246428 shutdown_connections 2024-07-17T06:55:57.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.209+0000 7f8c21000640 1 -- 172.21.15.44:0/510246428 wait complete. 2024-07-17T06:55:57.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.211+0000 7f8c21000640 1 Processor -- start 2024-07-17T06:55:57.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.211+0000 7f8c21000640 1 -- start start 2024-07-17T06:55:57.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1c073cc0 0x7f8c1c144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1c144d40 0x7f8c1c1427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15c7c0 con 0x7f8c00003fd0 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15c640 con 0x7f8c1c073cc0 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c21000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c0cb810 con 0x7f8c1c144d40 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57480/0 (socket says 172.21.15.44:57480) 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.212+0000 7f8c19e00640 1 -- 172.21.15.44:0/2976940590 learned_addr learned my addr 172.21.15.44:0/2976940590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c11400640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1c073cc0 0x7f8c1c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:57.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 -- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1c144d40 msgr2=0x7f8c1c1427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:57.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1c144d40 0x7f8c1c1427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:57.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 -- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1c073cc0 msgr2=0x7f8c1c144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:57.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1c073cc0 0x7f8c1c144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:57.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 -- 172.21.15.44:0/2976940590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c145180 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c19e00640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c010ca0 tx=0x7f8c0c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c017070 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.213+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c0c011640 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.214+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c152900 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.214+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c011a90 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.214+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c152ec0 con 0x7f8c00003fd0 2024-07-17T06:55:57.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.215+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c073900 con 0x7f8c00003fd0 2024-07-17T06:55:57.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.216+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f8c0c011c30 con 0x7f8c00003fd0 2024-07-17T06:55:57.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.217+0000 7f8c12a00640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 0x7f8bf8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:57.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.217+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f8c0c0993e0 con 0x7f8c00003fd0 2024-07-17T06:55:57.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.218+0000 7f8c11400640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 0x7f8bf8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:57.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.219+0000 7f8c11400640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 0x7f8bf8079070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c04035000 tx=0x7f8c040046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:57.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.222+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8c0c0626e0 con 0x7f8c00003fd0 2024-07-17T06:55:57.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:57.476+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8c1c06d620 con 0x7f8c00003fd0 2024-07-17T06:55:58.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.009+0000 7f8c12a00640 1 -- 172.21.15.44:0/2976940590 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8c0c061fe0 con 0x7f8c00003fd0 2024-07-17T06:55:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 5 and data pool 6 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 msgr2=0x7f8bf8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 0x7f8bf8079070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c04035000 tx=0x7f8c040046c0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 msgr2=0x7f8c1c1507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c010ca0 tx=0x7f8c0c00e720 comp rx=0 tx=0).stop 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 shutdown_connections 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f8bf8076bb0 0x7f8bf8079070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1c144d40 0x7f8c1c1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1c073cc0 0x7f8c1c144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 --2- 172.21.15.44:0/2976940590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c00003fd0 0x7f8c1c1507c0 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c010ca0 tx=0x7f8c0c00e720 comp rx=0 tx=0).stop 2024-07-17T06:55:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.013+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 >> 172.21.15.44:0/2976940590 conn(0x7f8c1c0b0eb0 msgr2=0x7f8c1c15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.014+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 shutdown_connections 2024-07-17T06:55:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.014+0000 7f8c21000640 1 -- 172.21.15.44:0/2976940590 wait complete. 2024-07-17T06:55:58.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 -- 172.21.15.44:0/3972294440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4003a00 msgr2=0x7fe4b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 --2- 172.21.15.44:0/3972294440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4003a00 0x7fe4b4003e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c470 tx=0x7fe4b4005830 comp rx=0 tx=0).stop 2024-07-17T06:55:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 -- 172.21.15.44:0/3972294440 shutdown_connections 2024-07-17T06:55:58.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 --2- 172.21.15.44:0/3972294440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4003a00 0x7fe4b4003e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 -- 172.21.15.44:0/3972294440 >> 172.21.15.44:0/3972294440 conn(0x7fe4d00b0fb0 msgr2=0x7fe4d00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:58.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 -- 172.21.15.44:0/3972294440 shutdown_connections 2024-07-17T06:55:58.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.236+0000 7fe4d7600640 1 -- 172.21.15.44:0/3972294440 wait complete. 2024-07-17T06:55:58.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.238+0000 7fe4d7600640 1 Processor -- start 2024-07-17T06:55:58.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.238+0000 7fe4d7600640 1 -- start start 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4b4003a00 0x7fe4d0097780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d01f3a20 0x7fe4d0072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00ae910 con 0x7fe4d01f7d70 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0154040 con 0x7fe4d01f3a20 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4d7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d006f710 con 0x7fe4b4003a00 2024-07-17T06:55:58.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57500/0 (socket says 172.21.15.44:57500) 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.239+0000 7fe4cfe00640 1 -- 172.21.15.44:0/4033147262 learned_addr learned my addr 172.21.15.44:0/4033147262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4d5600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d01f3a20 0x7fe4d0072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4d4c00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4b4003a00 0x7fe4d0097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 -- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4b4003a00 msgr2=0x7fe4d0097780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4b4003a00 0x7fe4d0097780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 -- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d01f3a20 msgr2=0x7fe4d0072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d01f3a20 0x7fe4d0072e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 -- 172.21.15.44:0/4033147262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d01f8270 con 0x7fe4d01f7d70 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4cfe00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800d440 tx=0x7fe4b800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8017070 con 0x7fe4d01f7d70 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4b8005030 con 0x7fe4d01f7d70 2024-07-17T06:55:58.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8005460 con 0x7fe4d01f7d70 2024-07-17T06:55:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d0072740 con 0x7fe4d01f7d70 2024-07-17T06:55:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.240+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d0142f20 con 0x7fe4d01f7d70 2024-07-17T06:55:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.242+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d0159b50 con 0x7fe4d01f7d70 2024-07-17T06:55:58.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.243+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fe4b801a070 con 0x7fe4d01f7d70 2024-07-17T06:55:58.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.244+0000 7fe4ccc00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 0x7fe4ac079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.244+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe4b8097650 con 0x7fe4d01f7d70 2024-07-17T06:55:58.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.244+0000 7fe4d4c00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 0x7fe4ac079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.246+0000 7fe4d4c00640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 0x7fe4ac079070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0033000 tx=0x7fe4c002d070 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.248+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe4b8060980 con 0x7fe4d01f7d70 2024-07-17T06:55:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.462+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d0098900 con 0x7fe4d01f7d70 2024-07-17T06:55:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.465+0000 7fe4ccc00640 1 -- 172.21.15.44:0/4033147262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7fe4b8060280 con 0x7fe4d01f7d70 2024-07-17T06:55:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:58.465 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:57.477114+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:55:55.596688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-07-17T06:55:56.460663+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 msgr2=0x7fe4ac079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 0x7fe4ac079070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0033000 tx=0x7fe4c002d070 comp rx=0 tx=0).stop 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 msgr2=0x7fe4d01f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800d440 tx=0x7fe4b800d900 comp rx=0 tx=0).stop 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 shutdown_connections 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4d01f3a20 0x7fe4d0072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4d01f7d70 0x7fe4d01f34e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fe4ac076bb0 0x7fe4ac079070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 --2- 172.21.15.44:0/4033147262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4b4003a00 0x7fe4d0097780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 >> 172.21.15.44:0/4033147262 conn(0x7fe4d00b0fb0 msgr2=0x7fe4d015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:58.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 shutdown_connections 2024-07-17T06:55:58.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.468+0000 7fe4d7600640 1 -- 172.21.15.44:0/4033147262 wait complete. 2024-07-17T06:55:58.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:55:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 -- 172.21.15.44:0/3375818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 msgr2=0x7f69880ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 --2- 172.21.15.44:0/3375818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f69880ad8e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6974009830 tx=0x7f697402a520 comp rx=0 tx=0).stop 2024-07-17T06:55:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 -- 172.21.15.44:0/3375818971 shutdown_connections 2024-07-17T06:55:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 --2- 172.21.15.44:0/3375818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f69880ad8e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 --2- 172.21.15.44:0/3375818971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf540 0x7f69880b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 -- 172.21.15.44:0/3375818971 >> 172.21.15.44:0/3375818971 conn(0x7f69880b0e80 msgr2=0x7f69880b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 -- 172.21.15.44:0/3375818971 shutdown_connections 2024-07-17T06:55:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.686+0000 7f698e800640 1 -- 172.21.15.44:0/3375818971 wait complete. 2024-07-17T06:55:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.688+0000 7f698e800640 1 Processor -- start 2024-07-17T06:55:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.688+0000 7f698e800640 1 -- start start 2024-07-17T06:55:58.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f698813d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf920 0x7f69881f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69880ae940 con 0x7f69880cf540 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69880ae7c0 con 0x7f69880b3db0 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6988141040 con 0x7f69880cf920 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57502/0 (socket says 172.21.15.44:57502) 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698ca00640 1 -- 172.21.15.44:0/4220499609 learned_addr learned my addr 172.21.15.44:0/4220499609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698d400640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f698813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 -- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf920 msgr2=0x7f69881f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.690+0000 7f698de00640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf920 0x7f69881f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf920 0x7f69881f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 -- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 msgr2=0x7f698813d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f698813d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 -- 172.21.15.44:0/4220499609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6988073be0 con 0x7f69880cf540 2024-07-17T06:55:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698ca00640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6974009950 tx=0x7f6974032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974040070 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69740038e0 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974003d30 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881f8010 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.691+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6988074130 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.692+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69881414c0 con 0x7f69880cf540 2024-07-17T06:55:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.693+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f6974039040 con 0x7f69880cf540 2024-07-17T06:55:58.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.695+0000 7f6985600640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 0x7f6968079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:58.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.695+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f69740b9ca0 con 0x7f69880cf540 2024-07-17T06:55:58.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.696+0000 7f698d400640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 0x7f6968079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:58.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.697+0000 7f698d400640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 0x7f6968079070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f698815aff0 tx=0x7f697c009260 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:58.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.700+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f69881f8010 con 0x7f69880cf540 2024-07-17T06:55:58.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.902+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69881e9780 con 0x7f69880cf540 2024-07-17T06:55:58.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.905+0000 7f6985600640 1 -- 172.21.15.44:0/4220499609 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7f6974082f40 con 0x7f69880cf540 2024-07-17T06:55:58.905 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:55:58.905 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":38,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:57.477114+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:55:55.596688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-07-17T06:55:56.460663+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 msgr2=0x7f6968079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 0x7f6968079070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f698815aff0 tx=0x7f697c009260 comp rx=0 tx=0).stop 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 msgr2=0x7f698813caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6974009950 tx=0x7f6974032c10 comp rx=0 tx=0).stop 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 shutdown_connections 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6968076bb0 0x7f6968079070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69880cf920 0x7f69881f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69880cf540 0x7f698813caf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 --2- 172.21.15.44:0/4220499609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69880b3db0 0x7f698813d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 >> 172.21.15.44:0/4220499609 conn(0x7f69880b0e80 msgr2=0x7f69880b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 shutdown_connections 2024-07-17T06:55:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:58.908+0000 7f698e800640 1 -- 172.21.15.44:0/4220499609 wait complete. 2024-07-17T06:55:58.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T06:55:59.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 -- 172.21.15.44:0/3433180513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 msgr2=0x7ff85c005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 --2- 172.21.15.44:0/3433180513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff85c005910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff86000bb20 tx=0x7ff85c006940 comp rx=0 tx=0).stop 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877400640 1 -- 172.21.15.44:0/3433180513 reap_dead start 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 -- 172.21.15.44:0/3433180513 shutdown_connections 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 -- 172.21.15.44:0/3433180513 >> 172.21.15.44:0/3433180513 conn(0x7ff8780cf0d0 msgr2=0x7ff8780cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 -- 172.21.15.44:0/3433180513 shutdown_connections 2024-07-17T06:55:59.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.145+0000 7ff877e00640 1 -- 172.21.15.44:0/3433180513 wait complete. 2024-07-17T06:55:59.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.147+0000 7ff877e00640 1 Processor -- start 2024-07-17T06:55:59.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.147+0000 7ff877e00640 1 -- start start 2024-07-17T06:55:59.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:59.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87814e570 0x7ff878149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:59.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff878072840 0x7ff8780739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:59.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff87815a5d0 con 0x7ff85c0051a0 2024-07-17T06:55:59.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff87815a450 con 0x7ff87814e570 2024-07-17T06:55:59.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.148+0000 7ff877e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff87815a750 con 0x7ff878072840 2024-07-17T06:55:59.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:59.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43212/0 (socket says 172.21.15.44:43212) 2024-07-17T06:55:59.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 -- 172.21.15.44:0/1410151955 learned_addr learned my addr 172.21.15.44:0/1410151955 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 -- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff878072840 msgr2=0x7ff8780739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff878072840 0x7ff8780739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 -- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87814e570 msgr2=0x7ff878149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87814e570 0x7ff878149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 -- 172.21.15.44:0/1410151955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff878149a50 con 0x7ff85c0051a0 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.149+0000 7ff876a00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff868010a40 tx=0x7ff868010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.150+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86800ecc0 con 0x7ff85c0051a0 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.150+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff86800ee60 con 0x7ff85c0051a0 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.150+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868005280 con 0x7ff85c0051a0 2024-07-17T06:55:59.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.150+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8780717a0 con 0x7ff85c0051a0 2024-07-17T06:55:59.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.150+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff878071d60 con 0x7ff85c0051a0 2024-07-17T06:55:59.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.151+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8780ac3f0 con 0x7ff85c0051a0 2024-07-17T06:55:59.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.152+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7ff868005420 con 0x7ff85c0051a0 2024-07-17T06:55:59.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.154+0000 7ff86ea00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 0x7ff8540790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:55:59.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.154+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff868099100 con 0x7ff85c0051a0 2024-07-17T06:55:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.154+0000 7ff86fe00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 0x7ff8540790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:55:59.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.155+0000 7ff86fe00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 0x7ff8540790a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff860036000 tx=0x7ff860004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:55:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.158+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff8780ac3f0 con 0x7ff85c0051a0 2024-07-17T06:55:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:55:59.400+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff8780cffe0 con 0x7ff85c0051a0 2024-07-17T06:56:00.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.056+0000 7ff86ea00640 1 -- 172.21.15.44:0/1410151955 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7ff878149a50 con 0x7ff85c0051a0 2024-07-17T06:56:00.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 msgr2=0x7ff8540790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:00.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 0x7ff8540790a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff860036000 tx=0x7ff860004730 comp rx=0 tx=0).stop 2024-07-17T06:56:00.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 msgr2=0x7ff8780cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:00.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff868010a40 tx=0x7ff868010f00 comp rx=0 tx=0).stop 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 shutdown_connections 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7ff854076be0 0x7ff8540790a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff878072840 0x7ff8780739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87814e570 0x7ff878149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 --2- 172.21.15.44:0/1410151955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff85c0051a0 0x7ff8780cdaa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 >> 172.21.15.44:0/1410151955 conn(0x7ff8780cf0d0 msgr2=0x7ff87815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 shutdown_connections 2024-07-17T06:56:00.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.060+0000 7ff877e00640 1 -- 172.21.15.44:0/1410151955 wait complete. 2024-07-17T06:56:00.080 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T06:56:00.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T06:56:00.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.299+0000 7f047b200640 1 -- 172.21.15.44:0/1130146942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0460004800 msgr2=0x7f0460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:00.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.299+0000 7f047b200640 1 --2- 172.21.15.44:0/1130146942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0460004800 0x7f0460004c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f046400bb20 tx=0x7f0460005830 comp rx=0 tx=0).stop 2024-07-17T06:56:00.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.299+0000 7f047a800640 1 -- 172.21.15.44:0/1130146942 reap_dead start 2024-07-17T06:56:00.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.300+0000 7f047b200640 1 -- 172.21.15.44:0/1130146942 shutdown_connections 2024-07-17T06:56:00.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.300+0000 7f047b200640 1 -- 172.21.15.44:0/1130146942 >> 172.21.15.44:0/1130146942 conn(0x7f047c0cf0d0 msgr2=0x7f047c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:00.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.300+0000 7f047b200640 1 -- 172.21.15.44:0/1130146942 shutdown_connections 2024-07-17T06:56:00.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.300+0000 7f047b200640 1 -- 172.21.15.44:0/1130146942 wait complete. 2024-07-17T06:56:00.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.302+0000 7f047b200640 1 Processor -- start 2024-07-17T06:56:00.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.302+0000 7f047b200640 1 -- start start 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 0x7f047c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f047c1506b0 0x7f047c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f047c0cb200 con 0x7f047c0ad190 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f047c0cb080 con 0x7f0460004800 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f047c0cb380 con 0x7f047c1506b0 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43220/0 (socket says 172.21.15.44:43220) 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f047a800640 1 -- 172.21.15.44:0/2067055270 learned_addr learned my addr 172.21.15.44:0/2067055270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f0479e00640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 0x7f047c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.303+0000 7f0479400640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f047c1506b0 0x7f047c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:00.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 -- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f047c1506b0 msgr2=0x7f047c14a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f047c1506b0 0x7f047c14a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 -- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 msgr2=0x7f047c1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 0x7f047c1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 -- 172.21.15.44:0/2067055270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c152d00 con 0x7f047c0ad190 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f0479e00640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 0x7f047c1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:00.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047a800640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f047c15c110 tx=0x7f047400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474003b70 con 0x7f047c0ad190 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0474003e20 con 0x7f047c0ad190 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400bb80 con 0x7f047c0ad190 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.304+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c152fa0 con 0x7f047c0ad190 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.305+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c153560 con 0x7f047c0ad190 2024-07-17T06:56:00.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.306+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c0cb910 con 0x7f047c0ad190 2024-07-17T06:56:00.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.307+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f047400bd20 con 0x7f047c0ad190 2024-07-17T06:56:00.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.308+0000 7f0472000640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 0x7f04580790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:00.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.308+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f04740985a0 con 0x7f047c0ad190 2024-07-17T06:56:00.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.309+0000 7f0479e00640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 0x7f04580790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.310+0000 7f0479e00640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 0x7f04580790a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f046c001860 tx=0x7f046c001650 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:00.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.313+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f04740618d0 con 0x7f047c0ad190 2024-07-17T06:56:00.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:00.563+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f047c0af7a0 con 0x7f047c0ad190 2024-07-17T06:56:01.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.060+0000 7f0472000640 1 -- 172.21.15.44:0/2067055270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f04740611d0 con 0x7f047c0ad190 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 msgr2=0x7f04580790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 0x7f04580790a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f046c001860 tx=0x7f046c001650 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 msgr2=0x7f047c0ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f047c15c110 tx=0x7f047400dd10 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 shutdown_connections 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c0ad190 0x7f047c0ad640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f0458076be0 0x7f04580790a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f047c1506b0 0x7f047c14a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 --2- 172.21.15.44:0/2067055270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0460004800 0x7f047c1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 >> 172.21.15.44:0/2067055270 conn(0x7f047c0cf0d0 msgr2=0x7f047c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:01.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.064+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 shutdown_connections 2024-07-17T06:56:01.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.065+0000 7f047b200640 1 -- 172.21.15.44:0/2067055270 wait complete. 2024-07-17T06:56:01.084 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T06:56:01.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.287+0000 7f9ebec00640 1 -- 172.21.15.44:0/2576633115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 msgr2=0x7f9eb80ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.287+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2576633115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb80ac690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009830 tx=0x7f9ea002a5c0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.287+0000 7f9ebec00640 1 -- 172.21.15.44:0/2576633115 shutdown_connections 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.287+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2576633115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb80ac690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.287+0000 7f9ebec00640 1 -- 172.21.15.44:0/2576633115 >> 172.21.15.44:0/2576633115 conn(0x7f9eb80b0fb0 msgr2=0x7f9eb80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.288+0000 7f9ebec00640 1 -- 172.21.15.44:0/2576633115 shutdown_connections 2024-07-17T06:56:01.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.288+0000 7f9ebec00640 1 -- 172.21.15.44:0/2576633115 wait complete. 2024-07-17T06:56:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.289+0000 7f9ebec00640 1 Processor -- start 2024-07-17T06:56:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.289+0000 7f9ebec00640 1 -- start start 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb80cd690 0x7f9eb80b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 0x7f9eb80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80ce370 con 0x7f9eb80ac220 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb80ce1f0 con 0x7f9eb81522c0 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.290+0000 7f9ebec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb80ce4f0 con 0x7f9eb80cd690 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43240/0 (socket says 172.21.15.44:43240) 2024-07-17T06:56:01.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 -- 172.21.15.44:0/2084569339 learned_addr learned my addr 172.21.15.44:0/2084569339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:01.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9ebcc00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 0x7f9eb80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eafe00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb80cd690 0x7f9eb80b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 -- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb80cd690 msgr2=0x7f9eb80b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb80cd690 0x7f9eb80b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 -- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 msgr2=0x7f9eb80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 0x7f9eb80cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9eb7e00640 1 -- 172.21.15.44:0/2084569339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb80b3d60 con 0x7f9eb80ac220 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.291+0000 7f9ebcc00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 0x7f9eb80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9eb7e00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c3a0 tx=0x7f9ea800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8017070 con 0x7f9eb80ac220 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea80053c0 con 0x7f9eb80ac220 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb81f7710 con 0x7f9eb80ac220 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8003850 con 0x7f9eb80ac220 2024-07-17T06:56:01.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.292+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81f79b0 con 0x7f9eb80ac220 2024-07-17T06:56:01.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.293+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb80cbfc0 con 0x7f9eb80ac220 2024-07-17T06:56:01.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.294+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9ea801a070 con 0x7f9eb80ac220 2024-07-17T06:56:01.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.295+0000 7f9eb4c00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 0x7f9e94079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.296+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9ea80978e0 con 0x7f9eb80ac220 2024-07-17T06:56:01.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.296+0000 7f9eafe00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 0x7f9e94079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.297+0000 7f9eafe00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 0x7f9e94079070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0034000 tx=0x7f9ea0004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:01.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.300+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9eb80cbfc0 con 0x7f9eb80ac220 2024-07-17T06:56:01.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.544+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eb80ac690 con 0x7f9eb80ac220 2024-07-17T06:56:01.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.545+0000 7f9eb4c00640 1 -- 172.21.15.44:0/2084569339 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f9ea8060c10 con 0x7f9eb80ac220 2024-07-17T06:56:01.544 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:01.544 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T06:56:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 msgr2=0x7f9e94079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 0x7f9e94079070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0034000 tx=0x7f9ea0004730 comp rx=0 tx=0).stop 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 msgr2=0x7f9eb81476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c3a0 tx=0x7f9ea800c860 comp rx=0 tx=0).stop 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 shutdown_connections 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb81522c0 0x7f9eb80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9e94076bb0 0x7f9e94079070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb80cd690 0x7f9eb80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 --2- 172.21.15.44:0/2084569339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb80ac220 0x7f9eb81476d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 >> 172.21.15.44:0/2084569339 conn(0x7f9eb80b0fb0 msgr2=0x7f9eb80ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 shutdown_connections 2024-07-17T06:56:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.548+0000 7f9ebec00640 1 -- 172.21.15.44:0/2084569339 wait complete. 2024-07-17T06:56:01.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T06:56:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.775+0000 7f774a200640 1 -- 172.21.15.44:0/1818238656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 msgr2=0x7f774415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.775+0000 7f774a200640 1 --2- 172.21.15.44:0/1818238656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f774415f2d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7734005fd0 tx=0x7f773402a8b0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 -- 172.21.15.44:0/1818238656 shutdown_connections 2024-07-17T06:56:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 --2- 172.21.15.44:0/1818238656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77440714f0 0x7f77440cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 --2- 172.21.15.44:0/1818238656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f774415f2d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 -- 172.21.15.44:0/1818238656 >> 172.21.15.44:0/1818238656 conn(0x7f77440b3090 msgr2=0x7f77440b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 -- 172.21.15.44:0/1818238656 shutdown_connections 2024-07-17T06:56:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.776+0000 7f774a200640 1 -- 172.21.15.44:0/1818238656 wait complete. 2024-07-17T06:56:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.777+0000 7f774a200640 1 Processor -- start 2024-07-17T06:56:01.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.777+0000 7f774a200640 1 -- start start 2024-07-17T06:56:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77440714f0 0x7f77441358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774415f810 0x7f77440afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77440d06f0 con 0x7f7744140070 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77440d0570 con 0x7f77440714f0 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.778+0000 7f774a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77440d0870 con 0x7f774415f810 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43250/0 (socket says 172.21.15.44:43250) 2024-07-17T06:56:01.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 -- 172.21.15.44:0/2111263212 learned_addr learned my addr 172.21.15.44:0/2111263212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:01.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7749800640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774415f810 0x7f77440afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 -- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774415f810 msgr2=0x7f77440afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774415f810 0x7f77440afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 -- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77440714f0 msgr2=0x7f77441358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77440714f0 0x7f77441358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 -- 172.21.15.44:0/2111263212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7744130920 con 0x7f7744140070 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.779+0000 7f7743e00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c3d0 tx=0x7f772c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.780+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c011790 con 0x7f7744140070 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.780+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772c017070 con 0x7f7744140070 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.780+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c011cc0 con 0x7f7744140070 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.780+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7744130c00 con 0x7f7744140070 2024-07-17T06:56:01.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.780+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441311c0 con 0x7f7744140070 2024-07-17T06:56:01.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.781+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77440d0dc0 con 0x7f7744140070 2024-07-17T06:56:01.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.782+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f772c01c460 con 0x7f7744140070 2024-07-17T06:56:01.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.783+0000 7f7740c00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 0x7f7720079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:01.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.783+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f772c098340 con 0x7f7744140070 2024-07-17T06:56:01.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.783+0000 7f7748e00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 0x7f7720079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:01.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.784+0000 7f7748e00640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 0x7f7720079070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7734034000 tx=0x7f7734004a40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:01.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.787+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f772c0615e0 con 0x7f7744140070 2024-07-17T06:56:01.970 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T06:56:01.971+0000 7f8c04c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T06:56:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.998+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f774415b900 con 0x7f7744140070 2024-07-17T06:56:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:01.999+0000 7f7740c00640 1 -- 172.21.15.44:0/2111263212 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16930 (secure 0 0 0) 0x7f772c060ee0 con 0x7f7744140070 2024-07-17T06:56:01.999 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:01.999 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":39,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T06:55:59.027938+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:55:55.596688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-07-17T06:55:56.460663+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.002+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 msgr2=0x7f7720079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.002+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 0x7f7720079070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7734034000 tx=0x7f7734004a40 comp rx=0 tx=0).stop 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.002+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 msgr2=0x7f77441300d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.002+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c3d0 tx=0x7f772c00c890 comp rx=0 tx=0).stop 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 shutdown_connections 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7720076bb0 0x7f7720079070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774415f810 0x7f77440afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7744140070 0x7f77441300d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 --2- 172.21.15.44:0/2111263212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77440714f0 0x7f77441358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 >> 172.21.15.44:0/2111263212 conn(0x7f77440b3090 msgr2=0x7f774415da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 shutdown_connections 2024-07-17T06:56:02.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.003+0000 7f774a200640 1 -- 172.21.15.44:0/2111263212 wait complete. 2024-07-17T06:56:02.021 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T06:56:02.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 -- 172.21.15.44:0/618751183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 msgr2=0x7f645c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 --2- 172.21.15.44:0/618751183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c15d800 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6454009ca0 tx=0x7f645402aa30 comp rx=0 tx=0).stop 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 -- 172.21.15.44:0/618751183 shutdown_connections 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 --2- 172.21.15.44:0/618751183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c15d800 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 --2- 172.21.15.44:0/618751183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0ce5c0 0x7f645c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 -- 172.21.15.44:0/618751183 >> 172.21.15.44:0/618751183 conn(0x7f645c0cfec0 msgr2=0x7f645c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 -- 172.21.15.44:0/618751183 shutdown_connections 2024-07-17T06:56:02.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.222+0000 7f6460a00640 1 -- 172.21.15.44:0/618751183 wait complete. 2024-07-17T06:56:02.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.224+0000 7f6460a00640 1 Processor -- start 2024-07-17T06:56:02.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.225+0000 7f6460a00640 1 -- start start 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0b8c80 0x7f645c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c0ce5c0 0x7f645c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f645c15e0d0 con 0x7f645c0b9060 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f645c15df50 con 0x7f645c0ce5c0 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.226+0000 7f6460a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f645c15e250 con 0x7f645c0b8c80 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43266/0 (socket says 172.21.15.44:43266) 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 -- 172.21.15.44:0/978481673 learned_addr learned my addr 172.21.15.44:0/978481673 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645b400640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0b8c80 0x7f645c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645be00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c0ce5c0 0x7f645c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 -- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0b8c80 msgr2=0x7f645c0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0b8c80 0x7f645c0a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 -- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c0ce5c0 msgr2=0x7f645c0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c0ce5c0 0x7f645c0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 -- 172.21.15.44:0/978481673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c14b580 con 0x7f645c0b9060 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.227+0000 7f645aa00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f644800c970 tx=0x7f644800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.228+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448007bb0 con 0x7f645c0b9060 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.228+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6448007e40 con 0x7f645c0b9060 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.228+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644800f690 con 0x7f645c0b9060 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.228+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c14b860 con 0x7f645c0b9060 2024-07-17T06:56:02.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.228+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1f4fc0 con 0x7f645c0b9060 2024-07-17T06:56:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.229+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c06e100 con 0x7f645c0b9060 2024-07-17T06:56:02.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.230+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f644800f830 con 0x7f645c0b9060 2024-07-17T06:56:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.231+0000 7f6453400640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 0x7f643c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.231+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6448098c70 con 0x7f645c0b9060 2024-07-17T06:56:02.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.232+0000 7f645b400640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 0x7f643c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.233+0000 7f645b400640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 0x7f643c079070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f644c004ea0 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:02.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.236+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f645c06e100 con 0x7f645c0b9060 2024-07-17T06:56:02.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.442+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f645c15e700 con 0x7f643c076bb0 2024-07-17T06:56:02.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.455+0000 7f6453400640 1 -- 172.21.15.44:0/978481673 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244116 (secure 0 0 0) 0x7f645c15e700 con 0x7f643c076bb0 2024-07-17T06:56:02.458 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:02.459 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.460+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 msgr2=0x7f643c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.460+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 0x7f643c079070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f644c004ea0 tx=0x7f644c009040 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 msgr2=0x7f645c1f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f644800c970 tx=0x7f644800ce30 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 shutdown_connections 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f645c0ce5c0 0x7f645c0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f645c0b9060 0x7f645c1f4a80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f643c076bb0 0x7f643c079070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 --2- 172.21.15.44:0/978481673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645c0b8c80 0x7f645c0a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 >> 172.21.15.44:0/978481673 conn(0x7f645c0cfec0 msgr2=0x7f645c0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:02.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 shutdown_connections 2024-07-17T06:56:02.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.461+0000 7f6460a00640 1 -- 172.21.15.44:0/978481673 wait complete. 2024-07-17T06:56:02.481 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-07-17T06:56:01.738383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220556,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754164,"statfs":{"total":773094113280,"available":772868263936,"internally_reserved":0,"allocated":4481024,"data_stored":2577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756276"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008688+0000","last_change":"2024-07-17T06:55:57.008688+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008688+0000","last_undegraded":"2024-07-17T06:55:57.008688+0000","last_fullsized":"2024-07-17T06:55:57.008688+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005414+0000","last_change":"2024-07-17T06:55:57.005414+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005414+0000","last_undegraded":"2024-07-17T06:55:57.005414+0000","last_fullsized":"2024-07-17T06:55:57.005414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.016491+0000","last_change":"2024-07-17T06:55:57.016491+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.016491+0000","last_undegraded":"2024-07-17T06:55:57.016491+0000","last_fullsized":"2024-07-17T06:55:57.016491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003049+0000","last_change":"2024-07-17T06:55:57.003049+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003049+0000","last_undegraded":"2024-07-17T06:55:57.003049+0000","last_fullsized":"2024-07-17T06:55:57.003049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008287+0000","last_change":"2024-07-17T06:55:57.008287+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008287+0000","last_undegraded":"2024-07-17T06:55:57.008287+0000","last_fullsized":"2024-07-17T06:55:57.008287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999693+0000","last_change":"2024-07-17T06:55:56.999693+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999693+0000","last_undegraded":"2024-07-17T06:55:56.999693+0000","last_fullsized":"2024-07-17T06:55:56.999693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007070+0000","last_change":"2024-07-17T06:55:57.007070+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007070+0000","last_undegraded":"2024-07-17T06:55:57.007070+0000","last_fullsized":"2024-07-17T06:55:57.007070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018563+0000","last_change":"2024-07-17T06:55:57.018563+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018563+0000","last_undegraded":"2024-07-17T06:55:57.018563+0000","last_fullsized":"2024-07-17T06:55:57.018563+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019159+0000","last_change":"2024-07-17T06:55:57.019159+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019159+0000","last_undegraded":"2024-07-17T06:55:57.019159+0000","last_fullsized":"2024-07-17T06:55:57.019159+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007239+0000","last_change":"2024-07-17T06:55:57.007239+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007239+0000","last_undegraded":"2024-07-17T06:55:57.007239+0000","last_fullsized":"2024-07-17T06:55:57.007239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019874+0000","last_change":"2024-07-17T06:55:57.019874+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019874+0000","last_undegraded":"2024-07-17T06:55:57.019874+0000","last_fullsized":"2024-07-17T06:55:57.019874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007767+0000","last_change":"2024-07-17T06:55:57.007767+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007767+0000","last_undegraded":"2024-07-17T06:55:57.007767+0000","last_fullsized":"2024-07-17T06:55:57.007767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998191+0000","last_change":"2024-07-17T06:55:56.998191+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998191+0000","last_undegraded":"2024-07-17T06:55:56.998191+0000","last_fullsized":"2024-07-17T06:55:56.998191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996430+0000","last_change":"2024-07-17T06:55:56.996430+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996430+0000","last_undegraded":"2024-07-17T06:55:56.996430+0000","last_fullsized":"2024-07-17T06:55:56.996430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019375+0000","last_change":"2024-07-17T06:55:57.019375+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019375+0000","last_undegraded":"2024-07-17T06:55:57.019375+0000","last_fullsized":"2024-07-17T06:55:57.019375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.997729+0000","last_change":"2024-07-17T06:55:56.997729+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.997729+0000","last_undegraded":"2024-07-17T06:55:56.997729+0000","last_fullsized":"2024-07-17T06:55:56.997729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005523+0000","last_change":"2024-07-17T06:55:57.005523+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005523+0000","last_undegraded":"2024-07-17T06:55:57.005523+0000","last_fullsized":"2024-07-17T06:55:57.005523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996319+0000","last_change":"2024-07-17T06:55:56.996319+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996319+0000","last_undegraded":"2024-07-17T06:55:56.996319+0000","last_fullsized":"2024-07-17T06:55:56.996319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.012155+0000","last_change":"2024-07-17T06:55:57.012155+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.012155+0000","last_undegraded":"2024-07-17T06:55:57.012155+0000","last_fullsized":"2024-07-17T06:55:57.012155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994916+0000","last_change":"2024-07-17T06:55:56.994916+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994916+0000","last_undegraded":"2024-07-17T06:55:56.994916+0000","last_fullsized":"2024-07-17T06:55:56.994916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994476+0000","last_change":"2024-07-17T06:55:56.994476+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994476+0000","last_undegraded":"2024-07-17T06:55:56.994476+0000","last_fullsized":"2024-07-17T06:55:56.994476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.004910+0000","last_change":"2024-07-17T06:55:57.004910+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.004910+0000","last_undegraded":"2024-07-17T06:55:57.004910+0000","last_fullsized":"2024-07-17T06:55:57.004910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028124+0000","last_change":"2024-07-17T06:55:57.024271+0000","last_active":"2024-07-17T06:55:57.028124+0000","last_peered":"2024-07-17T06:55:57.028124+0000","last_clean":"2024-07-17T06:55:57.028124+0000","last_became_active":"2024-07-17T06:55:57.022997+0000","last_became_peered":"2024-07-17T06:55:57.022997+0000","last_unstale":"2024-07-17T06:55:57.028124+0000","last_undegraded":"2024-07-17T06:55:57.028124+0000","last_fullsized":"2024-07-17T06:55:57.028124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.002945+0000","last_change":"2024-07-17T06:55:57.002945+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.002945+0000","last_undegraded":"2024-07-17T06:55:57.002945+0000","last_fullsized":"2024-07-17T06:55:57.002945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026074+0000","last_change":"2024-07-17T06:55:57.023703+0000","last_active":"2024-07-17T06:55:57.026074+0000","last_peered":"2024-07-17T06:55:57.026074+0000","last_clean":"2024-07-17T06:55:57.026074+0000","last_became_active":"2024-07-17T06:55:57.023430+0000","last_became_peered":"2024-07-17T06:55:57.023430+0000","last_unstale":"2024-07-17T06:55:57.026074+0000","last_undegraded":"2024-07-17T06:55:57.026074+0000","last_fullsized":"2024-07-17T06:55:57.026074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.027830+0000","last_change":"2024-07-17T06:55:57.025336+0000","last_active":"2024-07-17T06:55:57.027830+0000","last_peered":"2024-07-17T06:55:57.027830+0000","last_clean":"2024-07-17T06:55:57.027830+0000","last_became_active":"2024-07-17T06:55:57.024773+0000","last_became_peered":"2024-07-17T06:55:57.024773+0000","last_unstale":"2024-07-17T06:55:57.027830+0000","last_undegraded":"2024-07-17T06:55:57.027830+0000","last_fullsized":"2024-07-17T06:55:57.027830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.006970+0000","last_change":"2024-07-17T06:55:57.006970+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.006970+0000","last_undegraded":"2024-07-17T06:55:57.006970+0000","last_fullsized":"2024-07-17T06:55:57.006970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991195+0000","last_change":"2024-07-17T06:55:55.991195+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991195+0000","last_undegraded":"2024-07-17T06:55:55.991195+0000","last_fullsized":"2024-07-17T06:55:55.991195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009696+0000","last_change":"2024-07-17T06:55:57.009696+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009696+0000","last_undegraded":"2024-07-17T06:55:57.009696+0000","last_fullsized":"2024-07-17T06:55:57.009696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.030344+0000","last_change":"2024-07-17T06:55:57.027827+0000","last_active":"2024-07-17T06:55:57.030344+0000","last_peered":"2024-07-17T06:55:57.030344+0000","last_clean":"2024-07-17T06:55:57.030344+0000","last_became_active":"2024-07-17T06:55:57.027549+0000","last_became_peered":"2024-07-17T06:55:57.027549+0000","last_unstale":"2024-07-17T06:55:57.030344+0000","last_undegraded":"2024-07-17T06:55:57.030344+0000","last_fullsized":"2024-07-17T06:55:57.030344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:20.727281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007048+0000","last_change":"2024-07-17T06:55:57.007048+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007048+0000","last_undegraded":"2024-07-17T06:55:57.007048+0000","last_fullsized":"2024-07-17T06:55:57.007048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.000923+0000","last_change":"2024-07-17T06:55:56.000923+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.000923+0000","last_undegraded":"2024-07-17T06:55:56.000923+0000","last_fullsized":"2024-07-17T06:55:56.000923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026289+0000","last_change":"2024-07-17T06:55:57.023183+0000","last_active":"2024-07-17T06:55:57.026289+0000","last_peered":"2024-07-17T06:55:57.026289+0000","last_clean":"2024-07-17T06:55:57.026289+0000","last_became_active":"2024-07-17T06:55:57.022952+0000","last_became_peered":"2024-07-17T06:55:57.022952+0000","last_unstale":"2024-07-17T06:55:57.026289+0000","last_undegraded":"2024-07-17T06:55:57.026289+0000","last_fullsized":"2024-07-17T06:55:57.026289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001923+0000","last_change":"2024-07-17T06:55:57.001923+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001923+0000","last_undegraded":"2024-07-17T06:55:57.001923+0000","last_fullsized":"2024-07-17T06:55:57.001923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991226+0000","last_change":"2024-07-17T06:55:55.991226+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991226+0000","last_undegraded":"2024-07-17T06:55:55.991226+0000","last_fullsized":"2024-07-17T06:55:55.991226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028321+0000","last_change":"2024-07-17T06:55:57.026571+0000","last_active":"2024-07-17T06:55:57.028321+0000","last_peered":"2024-07-17T06:55:57.028321+0000","last_clean":"2024-07-17T06:55:57.028321+0000","last_became_active":"2024-07-17T06:55:57.026249+0000","last_became_peered":"2024-07-17T06:55:57.026249+0000","last_unstale":"2024-07-17T06:55:57.028321+0000","last_undegraded":"2024-07-17T06:55:57.028321+0000","last_fullsized":"2024-07-17T06:55:57.028321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:47.719938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028649+0000","last_change":"2024-07-17T06:55:57.025366+0000","last_active":"2024-07-17T06:55:57.028649+0000","last_peered":"2024-07-17T06:55:57.028649+0000","last_clean":"2024-07-17T06:55:57.028649+0000","last_became_active":"2024-07-17T06:55:57.024821+0000","last_became_peered":"2024-07-17T06:55:57.024821+0000","last_unstale":"2024-07-17T06:55:57.028649+0000","last_undegraded":"2024-07-17T06:55:57.028649+0000","last_fullsized":"2024-07-17T06:55:57.028649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008333+0000","last_change":"2024-07-17T06:55:57.008333+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008333+0000","last_undegraded":"2024-07-17T06:55:57.008333+0000","last_fullsized":"2024-07-17T06:55:57.008333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028462+0000","last_change":"2024-07-17T06:55:57.025608+0000","last_active":"2024-07-17T06:55:57.028462+0000","last_peered":"2024-07-17T06:55:57.028462+0000","last_clean":"2024-07-17T06:55:57.028462+0000","last_became_active":"2024-07-17T06:55:57.024617+0000","last_became_peered":"2024-07-17T06:55:57.024617+0000","last_unstale":"2024-07-17T06:55:57.028462+0000","last_undegraded":"2024-07-17T06:55:57.028462+0000","last_fullsized":"2024-07-17T06:55:57.028462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021233+0000","last_change":"2024-07-17T06:55:57.016760+0000","last_active":"2024-07-17T06:55:57.021233+0000","last_peered":"2024-07-17T06:55:57.021233+0000","last_clean":"2024-07-17T06:55:57.021233+0000","last_became_active":"2024-07-17T06:55:57.016398+0000","last_became_peered":"2024-07-17T06:55:57.016398+0000","last_unstale":"2024-07-17T06:55:57.021233+0000","last_undegraded":"2024-07-17T06:55:57.021233+0000","last_fullsized":"2024-07-17T06:55:57.021233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009235+0000","last_change":"2024-07-17T06:55:57.009235+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009235+0000","last_undegraded":"2024-07-17T06:55:57.009235+0000","last_fullsized":"2024-07-17T06:55:57.009235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.019212+0000","last_change":"2024-07-17T06:55:57.014743+0000","last_active":"2024-07-17T06:55:57.019212+0000","last_peered":"2024-07-17T06:55:57.019212+0000","last_clean":"2024-07-17T06:55:57.019212+0000","last_became_active":"2024-07-17T06:55:57.013321+0000","last_became_peered":"2024-07-17T06:55:57.013321+0000","last_unstale":"2024-07-17T06:55:57.019212+0000","last_undegraded":"2024-07-17T06:55:57.019212+0000","last_fullsized":"2024-07-17T06:55:57.019212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998505+0000","last_change":"2024-07-17T06:55:56.998505+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998505+0000","last_undegraded":"2024-07-17T06:55:56.998505+0000","last_fullsized":"2024-07-17T06:55:56.998505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026415+0000","last_change":"2024-07-17T06:55:57.023971+0000","last_active":"2024-07-17T06:55:57.026415+0000","last_peered":"2024-07-17T06:55:57.026415+0000","last_clean":"2024-07-17T06:55:57.026415+0000","last_became_active":"2024-07-17T06:55:57.023779+0000","last_became_peered":"2024-07-17T06:55:57.023779+0000","last_unstale":"2024-07-17T06:55:57.026415+0000","last_undegraded":"2024-07-17T06:55:57.026415+0000","last_fullsized":"2024-07-17T06:55:57.026415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023971+0000","last_change":"2024-07-17T06:55:57.015366+0000","last_active":"2024-07-17T06:55:57.023971+0000","last_peered":"2024-07-17T06:55:57.023971+0000","last_clean":"2024-07-17T06:55:57.023971+0000","last_became_active":"2024-07-17T06:55:57.015091+0000","last_became_peered":"2024-07-17T06:55:57.015091+0000","last_unstale":"2024-07-17T06:55:57.023971+0000","last_undegraded":"2024-07-17T06:55:57.023971+0000","last_fullsized":"2024-07-17T06:55:57.023971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001451+0000","last_change":"2024-07-17T06:55:57.001451+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001451+0000","last_undegraded":"2024-07-17T06:55:57.001451+0000","last_fullsized":"2024-07-17T06:55:57.001451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001437+0000","last_change":"2024-07-17T06:55:56.001437+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001437+0000","last_undegraded":"2024-07-17T06:55:56.001437+0000","last_fullsized":"2024-07-17T06:55:56.001437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007875+0000","last_change":"2024-07-17T06:55:57.007875+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007875+0000","last_undegraded":"2024-07-17T06:55:57.007875+0000","last_fullsized":"2024-07-17T06:55:57.007875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031034+0000","last_change":"2024-07-17T06:55:57.030109+0000","last_active":"2024-07-17T06:55:57.031034+0000","last_peered":"2024-07-17T06:55:57.031034+0000","last_clean":"2024-07-17T06:55:57.031034+0000","last_became_active":"2024-07-17T06:55:57.029434+0000","last_became_peered":"2024-07-17T06:55:57.029434+0000","last_unstale":"2024-07-17T06:55:57.031034+0000","last_undegraded":"2024-07-17T06:55:57.031034+0000","last_fullsized":"2024-07-17T06:55:57.031034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.014177+0000","last_change":"2024-07-17T06:55:57.014177+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.014177+0000","last_undegraded":"2024-07-17T06:55:57.014177+0000","last_fullsized":"2024-07-17T06:55:57.014177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.992431+0000","last_change":"2024-07-17T06:55:55.992431+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.992431+0000","last_undegraded":"2024-07-17T06:55:55.992431+0000","last_fullsized":"2024-07-17T06:55:55.992431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988522+0000","last_change":"2024-07-17T06:55:55.988522+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988522+0000","last_undegraded":"2024-07-17T06:55:55.988522+0000","last_fullsized":"2024-07-17T06:55:55.988522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.022671+0000","last_change":"2024-07-17T06:55:57.020021+0000","last_active":"2024-07-17T06:55:57.022671+0000","last_peered":"2024-07-17T06:55:57.022671+0000","last_clean":"2024-07-17T06:55:57.022671+0000","last_became_active":"2024-07-17T06:55:57.019599+0000","last_became_peered":"2024-07-17T06:55:57.019599+0000","last_unstale":"2024-07-17T06:55:57.022671+0000","last_undegraded":"2024-07-17T06:55:57.022671+0000","last_fullsized":"2024-07-17T06:55:57.022671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988691+0000","last_change":"2024-07-17T06:55:55.988691+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988691+0000","last_undegraded":"2024-07-17T06:55:55.988691+0000","last_fullsized":"2024-07-17T06:55:55.988691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007055+0000","last_change":"2024-07-17T06:55:57.007055+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007055+0000","last_undegraded":"2024-07-17T06:55:57.007055+0000","last_fullsized":"2024-07-17T06:55:57.007055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988678+0000","last_change":"2024-07-17T06:55:55.988678+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988678+0000","last_undegraded":"2024-07-17T06:55:55.988678+0000","last_fullsized":"2024-07-17T06:55:55.988678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.013267+0000","last_change":"2024-07-17T06:55:57.013267+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.013267+0000","last_undegraded":"2024-07-17T06:55:57.013267+0000","last_fullsized":"2024-07-17T06:55:57.013267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001021+0000","last_change":"2024-07-17T06:55:56.001021+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001021+0000","last_undegraded":"2024-07-17T06:55:56.001021+0000","last_fullsized":"2024-07-17T06:55:56.001021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.000933+0000","last_change":"2024-07-17T06:55:57.000933+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.000933+0000","last_undegraded":"2024-07-17T06:55:57.000933+0000","last_fullsized":"2024-07-17T06:55:57.000933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988719+0000","last_change":"2024-07-17T06:55:55.988719+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988719+0000","last_undegraded":"2024-07-17T06:55:55.988719+0000","last_fullsized":"2024-07-17T06:55:55.988719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003230+0000","last_change":"2024-07-17T06:55:57.003230+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003230+0000","last_undegraded":"2024-07-17T06:55:57.003230+0000","last_fullsized":"2024-07-17T06:55:57.003230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031012+0000","last_change":"2024-07-17T06:55:57.029338+0000","last_active":"2024-07-17T06:55:57.031012+0000","last_peered":"2024-07-17T06:55:57.031012+0000","last_clean":"2024-07-17T06:55:57.031012+0000","last_became_active":"2024-07-17T06:55:57.028513+0000","last_became_peered":"2024-07-17T06:55:57.028513+0000","last_unstale":"2024-07-17T06:55:57.031012+0000","last_undegraded":"2024-07-17T06:55:57.031012+0000","last_fullsized":"2024-07-17T06:55:57.031012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988267+0000","last_change":"2024-07-17T06:55:55.988267+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988267+0000","last_undegraded":"2024-07-17T06:55:55.988267+0000","last_fullsized":"2024-07-17T06:55:55.988267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999415+0000","last_change":"2024-07-17T06:55:56.999415+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999415+0000","last_undegraded":"2024-07-17T06:55:56.999415+0000","last_fullsized":"2024-07-17T06:55:56.999415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.025686+0000","last_change":"2024-07-17T06:55:57.020427+0000","last_active":"2024-07-17T06:55:57.025686+0000","last_peered":"2024-07-17T06:55:57.025686+0000","last_clean":"2024-07-17T06:55:57.025686+0000","last_became_active":"2024-07-17T06:55:57.020107+0000","last_became_peered":"2024-07-17T06:55:57.020107+0000","last_unstale":"2024-07-17T06:55:57.025686+0000","last_undegraded":"2024-07-17T06:55:57.025686+0000","last_fullsized":"2024-07-17T06:55:57.025686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008761+0000","last_change":"2024-07-17T06:55:57.008761+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008761+0000","last_undegraded":"2024-07-17T06:55:57.008761+0000","last_fullsized":"2024-07-17T06:55:57.008761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023491+0000","last_change":"2024-07-17T06:55:57.018030+0000","last_active":"2024-07-17T06:55:57.023491+0000","last_peered":"2024-07-17T06:55:57.023491+0000","last_clean":"2024-07-17T06:55:57.023491+0000","last_became_active":"2024-07-17T06:55:57.017102+0000","last_became_peered":"2024-07-17T06:55:57.017102+0000","last_unstale":"2024-07-17T06:55:57.023491+0000","last_undegraded":"2024-07-17T06:55:57.023491+0000","last_fullsized":"2024-07-17T06:55:57.023491+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005417+0000","last_change":"2024-07-17T06:55:57.005417+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005417+0000","last_undegraded":"2024-07-17T06:55:57.005417+0000","last_fullsized":"2024-07-17T06:55:57.005417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026967+0000","last_change":"2024-07-17T06:55:57.017786+0000","last_active":"2024-07-17T06:55:57.026967+0000","last_peered":"2024-07-17T06:55:57.026967+0000","last_clean":"2024-07-17T06:55:57.026967+0000","last_became_active":"2024-07-17T06:55:57.017469+0000","last_became_peered":"2024-07-17T06:55:57.017469+0000","last_unstale":"2024-07-17T06:55:57.026967+0000","last_undegraded":"2024-07-17T06:55:57.026967+0000","last_fullsized":"2024-07-17T06:55:57.026967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.989062+0000","last_change":"2024-07-17T06:55:55.989062+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.989062+0000","last_undegraded":"2024-07-17T06:55:55.989062+0000","last_fullsized":"2024-07-17T06:55:55.989062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021919+0000","last_change":"2024-07-17T06:55:57.020044+0000","last_active":"2024-07-17T06:55:57.021919+0000","last_peered":"2024-07-17T06:55:57.021919+0000","last_clean":"2024-07-17T06:55:57.021919+0000","last_became_active":"2024-07-17T06:55:57.018728+0000","last_became_peered":"2024-07-17T06:55:57.018728+0000","last_unstale":"2024-07-17T06:55:57.021919+0000","last_undegraded":"2024-07-17T06:55:57.021919+0000","last_fullsized":"2024-07-17T06:55:57.021919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:48.777690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018789+0000","last_change":"2024-07-17T06:55:57.018789+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018789+0000","last_undegraded":"2024-07-17T06:55:57.018789+0000","last_fullsized":"2024-07-17T06:55:57.018789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.999523+0000","last_change":"2024-07-17T06:55:55.999523+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.999523+0000","last_undegraded":"2024-07-17T06:55:55.999523+0000","last_fullsized":"2024-07-17T06:55:55.999523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":135,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-07-17T06:55:56.002635+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:55:56.002635+0000","last_peered":"2024-07-17T06:55:56.002635+0000","last_clean":"2024-07-17T06:55:56.002635+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:55:56.002635+0000","last_undegraded":"2024-07-17T06:55:56.002635+0000","last_fullsized":"2024-07-17T06:55:56.002635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":768192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":761525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":164948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":171157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:56:02.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:56:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 -- 172.21.15.44:0/520940072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 msgr2=0x7f4dc000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 --2- 172.21.15.44:0/520940072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4dc000b9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0b9860 tx=0x7f4dc0005830 comp rx=0 tx=0).stop 2024-07-17T06:56:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 -- 172.21.15.44:0/520940072 shutdown_connections 2024-07-17T06:56:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 --2- 172.21.15.44:0/520940072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4dc000b9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 -- 172.21.15.44:0/520940072 >> 172.21.15.44:0/520940072 conn(0x7f4ddc0d0350 msgr2=0x7f4ddc0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 -- 172.21.15.44:0/520940072 shutdown_connections 2024-07-17T06:56:02.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.742+0000 7f4de3600640 1 -- 172.21.15.44:0/520940072 wait complete. 2024-07-17T06:56:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.744+0000 7f4de3600640 1 Processor -- start 2024-07-17T06:56:02.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.744+0000 7f4de3600640 1 -- start start 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 0x7f4ddc0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 0x7f4ddc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc0b6640 con 0x7f4dc0005210 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc0b25a0 con 0x7f4ddc0cce80 2024-07-17T06:56:02.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc0b67c0 con 0x7f4ddc1f3c20 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.745+0000 7f4de0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43284/0 (socket says 172.21.15.44:43284) 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 -- 172.21.15.44:0/2560355603 learned_addr learned my addr 172.21.15.44:0/2560355603 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de1600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 0x7f4ddc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 -- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 msgr2=0x7f4ddc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4ddbe00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 0x7f4ddc0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 0x7f4ddc0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 -- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 msgr2=0x7f4ddc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 0x7f4ddc15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 -- 172.21.15.44:0/2560355603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc0cb810 con 0x7f4dc0005210 2024-07-17T06:56:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de1600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 0x7f4ddc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4ddbe00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 0x7f4ddc0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.746+0000 7f4de0c00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e900 tx=0x7f4dcc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:02.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.747+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc015070 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.747+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dcc00cd00 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.747+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0116c0 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.747+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc0cbaf0 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.747+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc0cc0b0 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.748+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc073c30 con 0x7f4dc0005210 2024-07-17T06:56:02.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.749+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f4dcc018070 con 0x7f4dc0005210 2024-07-17T06:56:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.750+0000 7f4dd8c00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 0x7f4db80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:02.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.750+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4dcc098a60 con 0x7f4dc0005210 2024-07-17T06:56:02.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.751+0000 7f4ddbe00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 0x7f4db80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:02.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.752+0000 7f4ddbe00640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 0x7f4db80790a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc1f4000 tx=0x7f4dc4006a90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:02.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.755+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4dcc061d00 con 0x7f4dc0005210 2024-07-17T06:56:02.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.966+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ddc0d3190 con 0x7f4db8076be0 2024-07-17T06:56:02.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.978+0000 7f4dd8c00640 1 -- 172.21.15.44:0/2560355603 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244116 (secure 0 0 0) 0x7f4ddc0d3190 con 0x7f4db8076be0 2024-07-17T06:56:02.981 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:02.982 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 msgr2=0x7f4db80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 0x7f4db80790a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc1f4000 tx=0x7f4dc4006a90 comp rx=0 tx=0).stop 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 msgr2=0x7f4ddc1f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e900 tx=0x7f4dcc00edc0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 shutdown_connections 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc0cce80 0x7f4ddc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f4db8076be0 0x7f4db80790a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ddc1f3c20 0x7f4ddc0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 --2- 172.21.15.44:0/2560355603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc0005210 0x7f4ddc1f4700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:02.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 >> 172.21.15.44:0/2560355603 conn(0x7f4ddc0d0350 msgr2=0x7f4ddc0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:02.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 shutdown_connections 2024-07-17T06:56:02.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:02.985+0000 7f4de3600640 1 -- 172.21.15.44:0/2560355603 wait complete. 2024-07-17T06:56:03.002 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-07-17T06:56:01.738383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220556,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754164,"statfs":{"total":773094113280,"available":772868263936,"internally_reserved":0,"allocated":4481024,"data_stored":2577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756276"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008688+0000","last_change":"2024-07-17T06:55:57.008688+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008688+0000","last_undegraded":"2024-07-17T06:55:57.008688+0000","last_fullsized":"2024-07-17T06:55:57.008688+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005414+0000","last_change":"2024-07-17T06:55:57.005414+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005414+0000","last_undegraded":"2024-07-17T06:55:57.005414+0000","last_fullsized":"2024-07-17T06:55:57.005414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.016491+0000","last_change":"2024-07-17T06:55:57.016491+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.016491+0000","last_undegraded":"2024-07-17T06:55:57.016491+0000","last_fullsized":"2024-07-17T06:55:57.016491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003049+0000","last_change":"2024-07-17T06:55:57.003049+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003049+0000","last_undegraded":"2024-07-17T06:55:57.003049+0000","last_fullsized":"2024-07-17T06:55:57.003049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008287+0000","last_change":"2024-07-17T06:55:57.008287+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008287+0000","last_undegraded":"2024-07-17T06:55:57.008287+0000","last_fullsized":"2024-07-17T06:55:57.008287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999693+0000","last_change":"2024-07-17T06:55:56.999693+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999693+0000","last_undegraded":"2024-07-17T06:55:56.999693+0000","last_fullsized":"2024-07-17T06:55:56.999693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007070+0000","last_change":"2024-07-17T06:55:57.007070+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007070+0000","last_undegraded":"2024-07-17T06:55:57.007070+0000","last_fullsized":"2024-07-17T06:55:57.007070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018563+0000","last_change":"2024-07-17T06:55:57.018563+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018563+0000","last_undegraded":"2024-07-17T06:55:57.018563+0000","last_fullsized":"2024-07-17T06:55:57.018563+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019159+0000","last_change":"2024-07-17T06:55:57.019159+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019159+0000","last_undegraded":"2024-07-17T06:55:57.019159+0000","last_fullsized":"2024-07-17T06:55:57.019159+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007239+0000","last_change":"2024-07-17T06:55:57.007239+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007239+0000","last_undegraded":"2024-07-17T06:55:57.007239+0000","last_fullsized":"2024-07-17T06:55:57.007239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019874+0000","last_change":"2024-07-17T06:55:57.019874+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019874+0000","last_undegraded":"2024-07-17T06:55:57.019874+0000","last_fullsized":"2024-07-17T06:55:57.019874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007767+0000","last_change":"2024-07-17T06:55:57.007767+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007767+0000","last_undegraded":"2024-07-17T06:55:57.007767+0000","last_fullsized":"2024-07-17T06:55:57.007767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998191+0000","last_change":"2024-07-17T06:55:56.998191+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998191+0000","last_undegraded":"2024-07-17T06:55:56.998191+0000","last_fullsized":"2024-07-17T06:55:56.998191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996430+0000","last_change":"2024-07-17T06:55:56.996430+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996430+0000","last_undegraded":"2024-07-17T06:55:56.996430+0000","last_fullsized":"2024-07-17T06:55:56.996430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019375+0000","last_change":"2024-07-17T06:55:57.019375+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019375+0000","last_undegraded":"2024-07-17T06:55:57.019375+0000","last_fullsized":"2024-07-17T06:55:57.019375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.997729+0000","last_change":"2024-07-17T06:55:56.997729+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.997729+0000","last_undegraded":"2024-07-17T06:55:56.997729+0000","last_fullsized":"2024-07-17T06:55:56.997729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005523+0000","last_change":"2024-07-17T06:55:57.005523+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005523+0000","last_undegraded":"2024-07-17T06:55:57.005523+0000","last_fullsized":"2024-07-17T06:55:57.005523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996319+0000","last_change":"2024-07-17T06:55:56.996319+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996319+0000","last_undegraded":"2024-07-17T06:55:56.996319+0000","last_fullsized":"2024-07-17T06:55:56.996319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.012155+0000","last_change":"2024-07-17T06:55:57.012155+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.012155+0000","last_undegraded":"2024-07-17T06:55:57.012155+0000","last_fullsized":"2024-07-17T06:55:57.012155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994916+0000","last_change":"2024-07-17T06:55:56.994916+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994916+0000","last_undegraded":"2024-07-17T06:55:56.994916+0000","last_fullsized":"2024-07-17T06:55:56.994916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994476+0000","last_change":"2024-07-17T06:55:56.994476+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994476+0000","last_undegraded":"2024-07-17T06:55:56.994476+0000","last_fullsized":"2024-07-17T06:55:56.994476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.004910+0000","last_change":"2024-07-17T06:55:57.004910+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.004910+0000","last_undegraded":"2024-07-17T06:55:57.004910+0000","last_fullsized":"2024-07-17T06:55:57.004910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028124+0000","last_change":"2024-07-17T06:55:57.024271+0000","last_active":"2024-07-17T06:55:57.028124+0000","last_peered":"2024-07-17T06:55:57.028124+0000","last_clean":"2024-07-17T06:55:57.028124+0000","last_became_active":"2024-07-17T06:55:57.022997+0000","last_became_peered":"2024-07-17T06:55:57.022997+0000","last_unstale":"2024-07-17T06:55:57.028124+0000","last_undegraded":"2024-07-17T06:55:57.028124+0000","last_fullsized":"2024-07-17T06:55:57.028124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.002945+0000","last_change":"2024-07-17T06:55:57.002945+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.002945+0000","last_undegraded":"2024-07-17T06:55:57.002945+0000","last_fullsized":"2024-07-17T06:55:57.002945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026074+0000","last_change":"2024-07-17T06:55:57.023703+0000","last_active":"2024-07-17T06:55:57.026074+0000","last_peered":"2024-07-17T06:55:57.026074+0000","last_clean":"2024-07-17T06:55:57.026074+0000","last_became_active":"2024-07-17T06:55:57.023430+0000","last_became_peered":"2024-07-17T06:55:57.023430+0000","last_unstale":"2024-07-17T06:55:57.026074+0000","last_undegraded":"2024-07-17T06:55:57.026074+0000","last_fullsized":"2024-07-17T06:55:57.026074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.027830+0000","last_change":"2024-07-17T06:55:57.025336+0000","last_active":"2024-07-17T06:55:57.027830+0000","last_peered":"2024-07-17T06:55:57.027830+0000","last_clean":"2024-07-17T06:55:57.027830+0000","last_became_active":"2024-07-17T06:55:57.024773+0000","last_became_peered":"2024-07-17T06:55:57.024773+0000","last_unstale":"2024-07-17T06:55:57.027830+0000","last_undegraded":"2024-07-17T06:55:57.027830+0000","last_fullsized":"2024-07-17T06:55:57.027830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.006970+0000","last_change":"2024-07-17T06:55:57.006970+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.006970+0000","last_undegraded":"2024-07-17T06:55:57.006970+0000","last_fullsized":"2024-07-17T06:55:57.006970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991195+0000","last_change":"2024-07-17T06:55:55.991195+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991195+0000","last_undegraded":"2024-07-17T06:55:55.991195+0000","last_fullsized":"2024-07-17T06:55:55.991195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009696+0000","last_change":"2024-07-17T06:55:57.009696+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009696+0000","last_undegraded":"2024-07-17T06:55:57.009696+0000","last_fullsized":"2024-07-17T06:55:57.009696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.030344+0000","last_change":"2024-07-17T06:55:57.027827+0000","last_active":"2024-07-17T06:55:57.030344+0000","last_peered":"2024-07-17T06:55:57.030344+0000","last_clean":"2024-07-17T06:55:57.030344+0000","last_became_active":"2024-07-17T06:55:57.027549+0000","last_became_peered":"2024-07-17T06:55:57.027549+0000","last_unstale":"2024-07-17T06:55:57.030344+0000","last_undegraded":"2024-07-17T06:55:57.030344+0000","last_fullsized":"2024-07-17T06:55:57.030344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:20.727281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007048+0000","last_change":"2024-07-17T06:55:57.007048+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007048+0000","last_undegraded":"2024-07-17T06:55:57.007048+0000","last_fullsized":"2024-07-17T06:55:57.007048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.000923+0000","last_change":"2024-07-17T06:55:56.000923+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.000923+0000","last_undegraded":"2024-07-17T06:55:56.000923+0000","last_fullsized":"2024-07-17T06:55:56.000923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026289+0000","last_change":"2024-07-17T06:55:57.023183+0000","last_active":"2024-07-17T06:55:57.026289+0000","last_peered":"2024-07-17T06:55:57.026289+0000","last_clean":"2024-07-17T06:55:57.026289+0000","last_became_active":"2024-07-17T06:55:57.022952+0000","last_became_peered":"2024-07-17T06:55:57.022952+0000","last_unstale":"2024-07-17T06:55:57.026289+0000","last_undegraded":"2024-07-17T06:55:57.026289+0000","last_fullsized":"2024-07-17T06:55:57.026289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001923+0000","last_change":"2024-07-17T06:55:57.001923+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001923+0000","last_undegraded":"2024-07-17T06:55:57.001923+0000","last_fullsized":"2024-07-17T06:55:57.001923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991226+0000","last_change":"2024-07-17T06:55:55.991226+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991226+0000","last_undegraded":"2024-07-17T06:55:55.991226+0000","last_fullsized":"2024-07-17T06:55:55.991226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028321+0000","last_change":"2024-07-17T06:55:57.026571+0000","last_active":"2024-07-17T06:55:57.028321+0000","last_peered":"2024-07-17T06:55:57.028321+0000","last_clean":"2024-07-17T06:55:57.028321+0000","last_became_active":"2024-07-17T06:55:57.026249+0000","last_became_peered":"2024-07-17T06:55:57.026249+0000","last_unstale":"2024-07-17T06:55:57.028321+0000","last_undegraded":"2024-07-17T06:55:57.028321+0000","last_fullsized":"2024-07-17T06:55:57.028321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:47.719938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028649+0000","last_change":"2024-07-17T06:55:57.025366+0000","last_active":"2024-07-17T06:55:57.028649+0000","last_peered":"2024-07-17T06:55:57.028649+0000","last_clean":"2024-07-17T06:55:57.028649+0000","last_became_active":"2024-07-17T06:55:57.024821+0000","last_became_peered":"2024-07-17T06:55:57.024821+0000","last_unstale":"2024-07-17T06:55:57.028649+0000","last_undegraded":"2024-07-17T06:55:57.028649+0000","last_fullsized":"2024-07-17T06:55:57.028649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008333+0000","last_change":"2024-07-17T06:55:57.008333+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008333+0000","last_undegraded":"2024-07-17T06:55:57.008333+0000","last_fullsized":"2024-07-17T06:55:57.008333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028462+0000","last_change":"2024-07-17T06:55:57.025608+0000","last_active":"2024-07-17T06:55:57.028462+0000","last_peered":"2024-07-17T06:55:57.028462+0000","last_clean":"2024-07-17T06:55:57.028462+0000","last_became_active":"2024-07-17T06:55:57.024617+0000","last_became_peered":"2024-07-17T06:55:57.024617+0000","last_unstale":"2024-07-17T06:55:57.028462+0000","last_undegraded":"2024-07-17T06:55:57.028462+0000","last_fullsized":"2024-07-17T06:55:57.028462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021233+0000","last_change":"2024-07-17T06:55:57.016760+0000","last_active":"2024-07-17T06:55:57.021233+0000","last_peered":"2024-07-17T06:55:57.021233+0000","last_clean":"2024-07-17T06:55:57.021233+0000","last_became_active":"2024-07-17T06:55:57.016398+0000","last_became_peered":"2024-07-17T06:55:57.016398+0000","last_unstale":"2024-07-17T06:55:57.021233+0000","last_undegraded":"2024-07-17T06:55:57.021233+0000","last_fullsized":"2024-07-17T06:55:57.021233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009235+0000","last_change":"2024-07-17T06:55:57.009235+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009235+0000","last_undegraded":"2024-07-17T06:55:57.009235+0000","last_fullsized":"2024-07-17T06:55:57.009235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.019212+0000","last_change":"2024-07-17T06:55:57.014743+0000","last_active":"2024-07-17T06:55:57.019212+0000","last_peered":"2024-07-17T06:55:57.019212+0000","last_clean":"2024-07-17T06:55:57.019212+0000","last_became_active":"2024-07-17T06:55:57.013321+0000","last_became_peered":"2024-07-17T06:55:57.013321+0000","last_unstale":"2024-07-17T06:55:57.019212+0000","last_undegraded":"2024-07-17T06:55:57.019212+0000","last_fullsized":"2024-07-17T06:55:57.019212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998505+0000","last_change":"2024-07-17T06:55:56.998505+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998505+0000","last_undegraded":"2024-07-17T06:55:56.998505+0000","last_fullsized":"2024-07-17T06:55:56.998505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026415+0000","last_change":"2024-07-17T06:55:57.023971+0000","last_active":"2024-07-17T06:55:57.026415+0000","last_peered":"2024-07-17T06:55:57.026415+0000","last_clean":"2024-07-17T06:55:57.026415+0000","last_became_active":"2024-07-17T06:55:57.023779+0000","last_became_peered":"2024-07-17T06:55:57.023779+0000","last_unstale":"2024-07-17T06:55:57.026415+0000","last_undegraded":"2024-07-17T06:55:57.026415+0000","last_fullsized":"2024-07-17T06:55:57.026415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023971+0000","last_change":"2024-07-17T06:55:57.015366+0000","last_active":"2024-07-17T06:55:57.023971+0000","last_peered":"2024-07-17T06:55:57.023971+0000","last_clean":"2024-07-17T06:55:57.023971+0000","last_became_active":"2024-07-17T06:55:57.015091+0000","last_became_peered":"2024-07-17T06:55:57.015091+0000","last_unstale":"2024-07-17T06:55:57.023971+0000","last_undegraded":"2024-07-17T06:55:57.023971+0000","last_fullsized":"2024-07-17T06:55:57.023971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001451+0000","last_change":"2024-07-17T06:55:57.001451+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001451+0000","last_undegraded":"2024-07-17T06:55:57.001451+0000","last_fullsized":"2024-07-17T06:55:57.001451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001437+0000","last_change":"2024-07-17T06:55:56.001437+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001437+0000","last_undegraded":"2024-07-17T06:55:56.001437+0000","last_fullsized":"2024-07-17T06:55:56.001437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007875+0000","last_change":"2024-07-17T06:55:57.007875+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007875+0000","last_undegraded":"2024-07-17T06:55:57.007875+0000","last_fullsized":"2024-07-17T06:55:57.007875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031034+0000","last_change":"2024-07-17T06:55:57.030109+0000","last_active":"2024-07-17T06:55:57.031034+0000","last_peered":"2024-07-17T06:55:57.031034+0000","last_clean":"2024-07-17T06:55:57.031034+0000","last_became_active":"2024-07-17T06:55:57.029434+0000","last_became_peered":"2024-07-17T06:55:57.029434+0000","last_unstale":"2024-07-17T06:55:57.031034+0000","last_undegraded":"2024-07-17T06:55:57.031034+0000","last_fullsized":"2024-07-17T06:55:57.031034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.014177+0000","last_change":"2024-07-17T06:55:57.014177+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.014177+0000","last_undegraded":"2024-07-17T06:55:57.014177+0000","last_fullsized":"2024-07-17T06:55:57.014177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.992431+0000","last_change":"2024-07-17T06:55:55.992431+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.992431+0000","last_undegraded":"2024-07-17T06:55:55.992431+0000","last_fullsized":"2024-07-17T06:55:55.992431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988522+0000","last_change":"2024-07-17T06:55:55.988522+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988522+0000","last_undegraded":"2024-07-17T06:55:55.988522+0000","last_fullsized":"2024-07-17T06:55:55.988522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.022671+0000","last_change":"2024-07-17T06:55:57.020021+0000","last_active":"2024-07-17T06:55:57.022671+0000","last_peered":"2024-07-17T06:55:57.022671+0000","last_clean":"2024-07-17T06:55:57.022671+0000","last_became_active":"2024-07-17T06:55:57.019599+0000","last_became_peered":"2024-07-17T06:55:57.019599+0000","last_unstale":"2024-07-17T06:55:57.022671+0000","last_undegraded":"2024-07-17T06:55:57.022671+0000","last_fullsized":"2024-07-17T06:55:57.022671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988691+0000","last_change":"2024-07-17T06:55:55.988691+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988691+0000","last_undegraded":"2024-07-17T06:55:55.988691+0000","last_fullsized":"2024-07-17T06:55:55.988691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007055+0000","last_change":"2024-07-17T06:55:57.007055+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007055+0000","last_undegraded":"2024-07-17T06:55:57.007055+0000","last_fullsized":"2024-07-17T06:55:57.007055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988678+0000","last_change":"2024-07-17T06:55:55.988678+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988678+0000","last_undegraded":"2024-07-17T06:55:55.988678+0000","last_fullsized":"2024-07-17T06:55:55.988678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.013267+0000","last_change":"2024-07-17T06:55:57.013267+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.013267+0000","last_undegraded":"2024-07-17T06:55:57.013267+0000","last_fullsized":"2024-07-17T06:55:57.013267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001021+0000","last_change":"2024-07-17T06:55:56.001021+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001021+0000","last_undegraded":"2024-07-17T06:55:56.001021+0000","last_fullsized":"2024-07-17T06:55:56.001021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.000933+0000","last_change":"2024-07-17T06:55:57.000933+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.000933+0000","last_undegraded":"2024-07-17T06:55:57.000933+0000","last_fullsized":"2024-07-17T06:55:57.000933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988719+0000","last_change":"2024-07-17T06:55:55.988719+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988719+0000","last_undegraded":"2024-07-17T06:55:55.988719+0000","last_fullsized":"2024-07-17T06:55:55.988719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003230+0000","last_change":"2024-07-17T06:55:57.003230+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003230+0000","last_undegraded":"2024-07-17T06:55:57.003230+0000","last_fullsized":"2024-07-17T06:55:57.003230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031012+0000","last_change":"2024-07-17T06:55:57.029338+0000","last_active":"2024-07-17T06:55:57.031012+0000","last_peered":"2024-07-17T06:55:57.031012+0000","last_clean":"2024-07-17T06:55:57.031012+0000","last_became_active":"2024-07-17T06:55:57.028513+0000","last_became_peered":"2024-07-17T06:55:57.028513+0000","last_unstale":"2024-07-17T06:55:57.031012+0000","last_undegraded":"2024-07-17T06:55:57.031012+0000","last_fullsized":"2024-07-17T06:55:57.031012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988267+0000","last_change":"2024-07-17T06:55:55.988267+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988267+0000","last_undegraded":"2024-07-17T06:55:55.988267+0000","last_fullsized":"2024-07-17T06:55:55.988267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999415+0000","last_change":"2024-07-17T06:55:56.999415+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999415+0000","last_undegraded":"2024-07-17T06:55:56.999415+0000","last_fullsized":"2024-07-17T06:55:56.999415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.025686+0000","last_change":"2024-07-17T06:55:57.020427+0000","last_active":"2024-07-17T06:55:57.025686+0000","last_peered":"2024-07-17T06:55:57.025686+0000","last_clean":"2024-07-17T06:55:57.025686+0000","last_became_active":"2024-07-17T06:55:57.020107+0000","last_became_peered":"2024-07-17T06:55:57.020107+0000","last_unstale":"2024-07-17T06:55:57.025686+0000","last_undegraded":"2024-07-17T06:55:57.025686+0000","last_fullsized":"2024-07-17T06:55:57.025686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008761+0000","last_change":"2024-07-17T06:55:57.008761+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008761+0000","last_undegraded":"2024-07-17T06:55:57.008761+0000","last_fullsized":"2024-07-17T06:55:57.008761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023491+0000","last_change":"2024-07-17T06:55:57.018030+0000","last_active":"2024-07-17T06:55:57.023491+0000","last_peered":"2024-07-17T06:55:57.023491+0000","last_clean":"2024-07-17T06:55:57.023491+0000","last_became_active":"2024-07-17T06:55:57.017102+0000","last_became_peered":"2024-07-17T06:55:57.017102+0000","last_unstale":"2024-07-17T06:55:57.023491+0000","last_undegraded":"2024-07-17T06:55:57.023491+0000","last_fullsized":"2024-07-17T06:55:57.023491+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005417+0000","last_change":"2024-07-17T06:55:57.005417+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005417+0000","last_undegraded":"2024-07-17T06:55:57.005417+0000","last_fullsized":"2024-07-17T06:55:57.005417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026967+0000","last_change":"2024-07-17T06:55:57.017786+0000","last_active":"2024-07-17T06:55:57.026967+0000","last_peered":"2024-07-17T06:55:57.026967+0000","last_clean":"2024-07-17T06:55:57.026967+0000","last_became_active":"2024-07-17T06:55:57.017469+0000","last_became_peered":"2024-07-17T06:55:57.017469+0000","last_unstale":"2024-07-17T06:55:57.026967+0000","last_undegraded":"2024-07-17T06:55:57.026967+0000","last_fullsized":"2024-07-17T06:55:57.026967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.989062+0000","last_change":"2024-07-17T06:55:55.989062+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.989062+0000","last_undegraded":"2024-07-17T06:55:55.989062+0000","last_fullsized":"2024-07-17T06:55:55.989062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021919+0000","last_change":"2024-07-17T06:55:57.020044+0000","last_active":"2024-07-17T06:55:57.021919+0000","last_peered":"2024-07-17T06:55:57.021919+0000","last_clean":"2024-07-17T06:55:57.021919+0000","last_became_active":"2024-07-17T06:55:57.018728+0000","last_became_peered":"2024-07-17T06:55:57.018728+0000","last_unstale":"2024-07-17T06:55:57.021919+0000","last_undegraded":"2024-07-17T06:55:57.021919+0000","last_fullsized":"2024-07-17T06:55:57.021919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:48.777690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018789+0000","last_change":"2024-07-17T06:55:57.018789+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018789+0000","last_undegraded":"2024-07-17T06:55:57.018789+0000","last_fullsized":"2024-07-17T06:55:57.018789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.999523+0000","last_change":"2024-07-17T06:55:55.999523+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.999523+0000","last_undegraded":"2024-07-17T06:55:55.999523+0000","last_fullsized":"2024-07-17T06:55:55.999523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":135,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-07-17T06:55:56.002635+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:55:56.002635+0000","last_peered":"2024-07-17T06:55:56.002635+0000","last_clean":"2024-07-17T06:55:56.002635+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:55:56.002635+0000","last_undegraded":"2024-07-17T06:55:56.002635+0000","last_fullsized":"2024-07-17T06:55:56.002635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":768192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":761525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":164948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":171157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:56:03.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:56:03.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 -- 172.21.15.44:0/1433797552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240cfa90 msgr2=0x7f182406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 --2- 172.21.15.44:0/1433797552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240cfa90 0x7f182406e060 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f181000a940 tx=0x7f181002b850 comp rx=0 tx=0).stop 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 -- 172.21.15.44:0/1433797552 shutdown_connections 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 --2- 172.21.15.44:0/1433797552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240cfa90 0x7f182406e060 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 -- 172.21.15.44:0/1433797552 >> 172.21.15.44:0/1433797552 conn(0x7f18240b0e80 msgr2=0x7f18240b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.261+0000 7f1829600640 1 -- 172.21.15.44:0/1433797552 shutdown_connections 2024-07-17T06:56:03.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.262+0000 7f1829600640 1 -- 172.21.15.44:0/1433797552 wait complete. 2024-07-17T06:56:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.263+0000 7f1829600640 1 Processor -- start 2024-07-17T06:56:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.263+0000 7f1829600640 1 -- start start 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18240cfa90 0x7f182414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 0x7f182414c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18240b2330 con 0x7f182406f4b0 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18240b21b0 con 0x7f1824159800 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.264+0000 7f1829600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18240b24b0 con 0x7f18240cfa90 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43304/0 (socket says 172.21.15.44:43304) 2024-07-17T06:56:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 -- 172.21.15.44:0/3619275928 learned_addr learned my addr 172.21.15.44:0/3619275928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823400640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18240cfa90 0x7f182414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 -- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18240cfa90 msgr2=0x7f182414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1828c00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 0x7f182414c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18240cfa90 0x7f182414c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 -- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 msgr2=0x7f182414c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 0x7f182414c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1823e00640 1 -- 172.21.15.44:0/3619275928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824147420 con 0x7f182406f4b0 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.265+0000 7f1828c00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 0x7f182414c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f1823e00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f181400c490 tx=0x7f181400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814016020 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1814012940 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814012d90 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1824147700 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.266+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1824147cc0 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.267+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182406e060 con 0x7f182406f4b0 2024-07-17T06:56:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.268+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f1814019020 con 0x7f182406f4b0 2024-07-17T06:56:03.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.269+0000 7f181be00640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 0x7f1804079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.269+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1814098240 con 0x7f182406f4b0 2024-07-17T06:56:03.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.270+0000 7f1823400640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 0x7f1804079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.271+0000 7f1823400640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 0x7f1804079070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1810036000 tx=0x7f1810004730 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:03.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.274+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1814020030 con 0x7f182406f4b0 2024-07-17T06:56:03.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.481+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18241f3ad0 con 0x7f1804076bb0 2024-07-17T06:56:03.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.493+0000 7f181be00640 1 -- 172.21.15.44:0/3619275928 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244116 (secure 0 0 0) 0x7f18241f3ad0 con 0x7f1804076bb0 2024-07-17T06:56:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:03.497 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:56:03.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 msgr2=0x7f1804079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 0x7f1804079070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1810036000 tx=0x7f1810004730 comp rx=0 tx=0).stop 2024-07-17T06:56:03.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 msgr2=0x7f18241f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f181400c490 tx=0x7f181400c950 comp rx=0 tx=0).stop 2024-07-17T06:56:03.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 shutdown_connections 2024-07-17T06:56:03.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1824159800 0x7f182414c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f1804076bb0 0x7f1804079070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18240cfa90 0x7f182414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 --2- 172.21.15.44:0/3619275928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f182406f4b0 0x7f18241f3590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 >> 172.21.15.44:0/3619275928 conn(0x7f18240b0e80 msgr2=0x7f18240aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:03.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 shutdown_connections 2024-07-17T06:56:03.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.499+0000 7f1829600640 1 -- 172.21.15.44:0/3619275928 wait complete. 2024-07-17T06:56:03.518 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-07-17T06:56:01.738383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220556,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754164,"statfs":{"total":773094113280,"available":772868263936,"internally_reserved":0,"allocated":4481024,"data_stored":2577118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756276"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008688+0000","last_change":"2024-07-17T06:55:57.008688+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008688+0000","last_undegraded":"2024-07-17T06:55:57.008688+0000","last_fullsized":"2024-07-17T06:55:57.008688+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005414+0000","last_change":"2024-07-17T06:55:57.005414+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005414+0000","last_undegraded":"2024-07-17T06:55:57.005414+0000","last_fullsized":"2024-07-17T06:55:57.005414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.016491+0000","last_change":"2024-07-17T06:55:57.016491+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.016491+0000","last_undegraded":"2024-07-17T06:55:57.016491+0000","last_fullsized":"2024-07-17T06:55:57.016491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003049+0000","last_change":"2024-07-17T06:55:57.003049+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003049+0000","last_undegraded":"2024-07-17T06:55:57.003049+0000","last_fullsized":"2024-07-17T06:55:57.003049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008287+0000","last_change":"2024-07-17T06:55:57.008287+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008287+0000","last_undegraded":"2024-07-17T06:55:57.008287+0000","last_fullsized":"2024-07-17T06:55:57.008287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999693+0000","last_change":"2024-07-17T06:55:56.999693+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999693+0000","last_undegraded":"2024-07-17T06:55:56.999693+0000","last_fullsized":"2024-07-17T06:55:56.999693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007070+0000","last_change":"2024-07-17T06:55:57.007070+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007070+0000","last_undegraded":"2024-07-17T06:55:57.007070+0000","last_fullsized":"2024-07-17T06:55:57.007070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018563+0000","last_change":"2024-07-17T06:55:57.018563+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018563+0000","last_undegraded":"2024-07-17T06:55:57.018563+0000","last_fullsized":"2024-07-17T06:55:57.018563+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019159+0000","last_change":"2024-07-17T06:55:57.019159+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019159+0000","last_undegraded":"2024-07-17T06:55:57.019159+0000","last_fullsized":"2024-07-17T06:55:57.019159+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007239+0000","last_change":"2024-07-17T06:55:57.007239+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007239+0000","last_undegraded":"2024-07-17T06:55:57.007239+0000","last_fullsized":"2024-07-17T06:55:57.007239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019874+0000","last_change":"2024-07-17T06:55:57.019874+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019874+0000","last_undegraded":"2024-07-17T06:55:57.019874+0000","last_fullsized":"2024-07-17T06:55:57.019874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007767+0000","last_change":"2024-07-17T06:55:57.007767+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007767+0000","last_undegraded":"2024-07-17T06:55:57.007767+0000","last_fullsized":"2024-07-17T06:55:57.007767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998191+0000","last_change":"2024-07-17T06:55:56.998191+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998191+0000","last_undegraded":"2024-07-17T06:55:56.998191+0000","last_fullsized":"2024-07-17T06:55:56.998191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996430+0000","last_change":"2024-07-17T06:55:56.996430+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996430+0000","last_undegraded":"2024-07-17T06:55:56.996430+0000","last_fullsized":"2024-07-17T06:55:56.996430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.019375+0000","last_change":"2024-07-17T06:55:57.019375+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.019375+0000","last_undegraded":"2024-07-17T06:55:57.019375+0000","last_fullsized":"2024-07-17T06:55:57.019375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.997729+0000","last_change":"2024-07-17T06:55:56.997729+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.997729+0000","last_undegraded":"2024-07-17T06:55:56.997729+0000","last_fullsized":"2024-07-17T06:55:56.997729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005523+0000","last_change":"2024-07-17T06:55:57.005523+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005523+0000","last_undegraded":"2024-07-17T06:55:57.005523+0000","last_fullsized":"2024-07-17T06:55:57.005523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.996319+0000","last_change":"2024-07-17T06:55:56.996319+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.996319+0000","last_undegraded":"2024-07-17T06:55:56.996319+0000","last_fullsized":"2024-07-17T06:55:56.996319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.012155+0000","last_change":"2024-07-17T06:55:57.012155+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.012155+0000","last_undegraded":"2024-07-17T06:55:57.012155+0000","last_fullsized":"2024-07-17T06:55:57.012155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994916+0000","last_change":"2024-07-17T06:55:56.994916+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994916+0000","last_undegraded":"2024-07-17T06:55:56.994916+0000","last_fullsized":"2024-07-17T06:55:56.994916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.994476+0000","last_change":"2024-07-17T06:55:56.994476+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.994476+0000","last_undegraded":"2024-07-17T06:55:56.994476+0000","last_fullsized":"2024-07-17T06:55:56.994476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.004910+0000","last_change":"2024-07-17T06:55:57.004910+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.004910+0000","last_undegraded":"2024-07-17T06:55:57.004910+0000","last_fullsized":"2024-07-17T06:55:57.004910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028124+0000","last_change":"2024-07-17T06:55:57.024271+0000","last_active":"2024-07-17T06:55:57.028124+0000","last_peered":"2024-07-17T06:55:57.028124+0000","last_clean":"2024-07-17T06:55:57.028124+0000","last_became_active":"2024-07-17T06:55:57.022997+0000","last_became_peered":"2024-07-17T06:55:57.022997+0000","last_unstale":"2024-07-17T06:55:57.028124+0000","last_undegraded":"2024-07-17T06:55:57.028124+0000","last_fullsized":"2024-07-17T06:55:57.028124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.002945+0000","last_change":"2024-07-17T06:55:57.002945+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.002945+0000","last_undegraded":"2024-07-17T06:55:57.002945+0000","last_fullsized":"2024-07-17T06:55:57.002945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026074+0000","last_change":"2024-07-17T06:55:57.023703+0000","last_active":"2024-07-17T06:55:57.026074+0000","last_peered":"2024-07-17T06:55:57.026074+0000","last_clean":"2024-07-17T06:55:57.026074+0000","last_became_active":"2024-07-17T06:55:57.023430+0000","last_became_peered":"2024-07-17T06:55:57.023430+0000","last_unstale":"2024-07-17T06:55:57.026074+0000","last_undegraded":"2024-07-17T06:55:57.026074+0000","last_fullsized":"2024-07-17T06:55:57.026074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.027830+0000","last_change":"2024-07-17T06:55:57.025336+0000","last_active":"2024-07-17T06:55:57.027830+0000","last_peered":"2024-07-17T06:55:57.027830+0000","last_clean":"2024-07-17T06:55:57.027830+0000","last_became_active":"2024-07-17T06:55:57.024773+0000","last_became_peered":"2024-07-17T06:55:57.024773+0000","last_unstale":"2024-07-17T06:55:57.027830+0000","last_undegraded":"2024-07-17T06:55:57.027830+0000","last_fullsized":"2024-07-17T06:55:57.027830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.006970+0000","last_change":"2024-07-17T06:55:57.006970+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.006970+0000","last_undegraded":"2024-07-17T06:55:57.006970+0000","last_fullsized":"2024-07-17T06:55:57.006970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991195+0000","last_change":"2024-07-17T06:55:55.991195+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991195+0000","last_undegraded":"2024-07-17T06:55:55.991195+0000","last_fullsized":"2024-07-17T06:55:55.991195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009696+0000","last_change":"2024-07-17T06:55:57.009696+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009696+0000","last_undegraded":"2024-07-17T06:55:57.009696+0000","last_fullsized":"2024-07-17T06:55:57.009696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.030344+0000","last_change":"2024-07-17T06:55:57.027827+0000","last_active":"2024-07-17T06:55:57.030344+0000","last_peered":"2024-07-17T06:55:57.030344+0000","last_clean":"2024-07-17T06:55:57.030344+0000","last_became_active":"2024-07-17T06:55:57.027549+0000","last_became_peered":"2024-07-17T06:55:57.027549+0000","last_unstale":"2024-07-17T06:55:57.030344+0000","last_undegraded":"2024-07-17T06:55:57.030344+0000","last_fullsized":"2024-07-17T06:55:57.030344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:20.727281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007048+0000","last_change":"2024-07-17T06:55:57.007048+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007048+0000","last_undegraded":"2024-07-17T06:55:57.007048+0000","last_fullsized":"2024-07-17T06:55:57.007048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.000923+0000","last_change":"2024-07-17T06:55:56.000923+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.000923+0000","last_undegraded":"2024-07-17T06:55:56.000923+0000","last_fullsized":"2024-07-17T06:55:56.000923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026289+0000","last_change":"2024-07-17T06:55:57.023183+0000","last_active":"2024-07-17T06:55:57.026289+0000","last_peered":"2024-07-17T06:55:57.026289+0000","last_clean":"2024-07-17T06:55:57.026289+0000","last_became_active":"2024-07-17T06:55:57.022952+0000","last_became_peered":"2024-07-17T06:55:57.022952+0000","last_unstale":"2024-07-17T06:55:57.026289+0000","last_undegraded":"2024-07-17T06:55:57.026289+0000","last_fullsized":"2024-07-17T06:55:57.026289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001923+0000","last_change":"2024-07-17T06:55:57.001923+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001923+0000","last_undegraded":"2024-07-17T06:55:57.001923+0000","last_fullsized":"2024-07-17T06:55:57.001923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.991226+0000","last_change":"2024-07-17T06:55:55.991226+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.991226+0000","last_undegraded":"2024-07-17T06:55:55.991226+0000","last_fullsized":"2024-07-17T06:55:55.991226+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028321+0000","last_change":"2024-07-17T06:55:57.026571+0000","last_active":"2024-07-17T06:55:57.028321+0000","last_peered":"2024-07-17T06:55:57.028321+0000","last_clean":"2024-07-17T06:55:57.028321+0000","last_became_active":"2024-07-17T06:55:57.026249+0000","last_became_peered":"2024-07-17T06:55:57.026249+0000","last_unstale":"2024-07-17T06:55:57.028321+0000","last_undegraded":"2024-07-17T06:55:57.028321+0000","last_fullsized":"2024-07-17T06:55:57.028321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:47.719938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028649+0000","last_change":"2024-07-17T06:55:57.025366+0000","last_active":"2024-07-17T06:55:57.028649+0000","last_peered":"2024-07-17T06:55:57.028649+0000","last_clean":"2024-07-17T06:55:57.028649+0000","last_became_active":"2024-07-17T06:55:57.024821+0000","last_became_peered":"2024-07-17T06:55:57.024821+0000","last_unstale":"2024-07-17T06:55:57.028649+0000","last_undegraded":"2024-07-17T06:55:57.028649+0000","last_fullsized":"2024-07-17T06:55:57.028649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008333+0000","last_change":"2024-07-17T06:55:57.008333+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008333+0000","last_undegraded":"2024-07-17T06:55:57.008333+0000","last_fullsized":"2024-07-17T06:55:57.008333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.028462+0000","last_change":"2024-07-17T06:55:57.025608+0000","last_active":"2024-07-17T06:55:57.028462+0000","last_peered":"2024-07-17T06:55:57.028462+0000","last_clean":"2024-07-17T06:55:57.028462+0000","last_became_active":"2024-07-17T06:55:57.024617+0000","last_became_peered":"2024-07-17T06:55:57.024617+0000","last_unstale":"2024-07-17T06:55:57.028462+0000","last_undegraded":"2024-07-17T06:55:57.028462+0000","last_fullsized":"2024-07-17T06:55:57.028462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021233+0000","last_change":"2024-07-17T06:55:57.016760+0000","last_active":"2024-07-17T06:55:57.021233+0000","last_peered":"2024-07-17T06:55:57.021233+0000","last_clean":"2024-07-17T06:55:57.021233+0000","last_became_active":"2024-07-17T06:55:57.016398+0000","last_became_peered":"2024-07-17T06:55:57.016398+0000","last_unstale":"2024-07-17T06:55:57.021233+0000","last_undegraded":"2024-07-17T06:55:57.021233+0000","last_fullsized":"2024-07-17T06:55:57.021233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.009235+0000","last_change":"2024-07-17T06:55:57.009235+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.009235+0000","last_undegraded":"2024-07-17T06:55:57.009235+0000","last_fullsized":"2024-07-17T06:55:57.009235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.019212+0000","last_change":"2024-07-17T06:55:57.014743+0000","last_active":"2024-07-17T06:55:57.019212+0000","last_peered":"2024-07-17T06:55:57.019212+0000","last_clean":"2024-07-17T06:55:57.019212+0000","last_became_active":"2024-07-17T06:55:57.013321+0000","last_became_peered":"2024-07-17T06:55:57.013321+0000","last_unstale":"2024-07-17T06:55:57.019212+0000","last_undegraded":"2024-07-17T06:55:57.019212+0000","last_fullsized":"2024-07-17T06:55:57.019212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.998505+0000","last_change":"2024-07-17T06:55:56.998505+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.998505+0000","last_undegraded":"2024-07-17T06:55:56.998505+0000","last_fullsized":"2024-07-17T06:55:56.998505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026415+0000","last_change":"2024-07-17T06:55:57.023971+0000","last_active":"2024-07-17T06:55:57.026415+0000","last_peered":"2024-07-17T06:55:57.026415+0000","last_clean":"2024-07-17T06:55:57.026415+0000","last_became_active":"2024-07-17T06:55:57.023779+0000","last_became_peered":"2024-07-17T06:55:57.023779+0000","last_unstale":"2024-07-17T06:55:57.026415+0000","last_undegraded":"2024-07-17T06:55:57.026415+0000","last_fullsized":"2024-07-17T06:55:57.026415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023971+0000","last_change":"2024-07-17T06:55:57.015366+0000","last_active":"2024-07-17T06:55:57.023971+0000","last_peered":"2024-07-17T06:55:57.023971+0000","last_clean":"2024-07-17T06:55:57.023971+0000","last_became_active":"2024-07-17T06:55:57.015091+0000","last_became_peered":"2024-07-17T06:55:57.015091+0000","last_unstale":"2024-07-17T06:55:57.023971+0000","last_undegraded":"2024-07-17T06:55:57.023971+0000","last_fullsized":"2024-07-17T06:55:57.023971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.001451+0000","last_change":"2024-07-17T06:55:57.001451+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.001451+0000","last_undegraded":"2024-07-17T06:55:57.001451+0000","last_fullsized":"2024-07-17T06:55:57.001451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001437+0000","last_change":"2024-07-17T06:55:56.001437+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001437+0000","last_undegraded":"2024-07-17T06:55:56.001437+0000","last_fullsized":"2024-07-17T06:55:56.001437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007875+0000","last_change":"2024-07-17T06:55:57.007875+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007875+0000","last_undegraded":"2024-07-17T06:55:57.007875+0000","last_fullsized":"2024-07-17T06:55:57.007875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031034+0000","last_change":"2024-07-17T06:55:57.030109+0000","last_active":"2024-07-17T06:55:57.031034+0000","last_peered":"2024-07-17T06:55:57.031034+0000","last_clean":"2024-07-17T06:55:57.031034+0000","last_became_active":"2024-07-17T06:55:57.029434+0000","last_became_peered":"2024-07-17T06:55:57.029434+0000","last_unstale":"2024-07-17T06:55:57.031034+0000","last_undegraded":"2024-07-17T06:55:57.031034+0000","last_fullsized":"2024-07-17T06:55:57.031034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:51.412813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.014177+0000","last_change":"2024-07-17T06:55:57.014177+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.014177+0000","last_undegraded":"2024-07-17T06:55:57.014177+0000","last_fullsized":"2024-07-17T06:55:57.014177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.992431+0000","last_change":"2024-07-17T06:55:55.992431+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.992431+0000","last_undegraded":"2024-07-17T06:55:55.992431+0000","last_fullsized":"2024-07-17T06:55:55.992431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988522+0000","last_change":"2024-07-17T06:55:55.988522+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988522+0000","last_undegraded":"2024-07-17T06:55:55.988522+0000","last_fullsized":"2024-07-17T06:55:55.988522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.022671+0000","last_change":"2024-07-17T06:55:57.020021+0000","last_active":"2024-07-17T06:55:57.022671+0000","last_peered":"2024-07-17T06:55:57.022671+0000","last_clean":"2024-07-17T06:55:57.022671+0000","last_became_active":"2024-07-17T06:55:57.019599+0000","last_became_peered":"2024-07-17T06:55:57.019599+0000","last_unstale":"2024-07-17T06:55:57.022671+0000","last_undegraded":"2024-07-17T06:55:57.022671+0000","last_fullsized":"2024-07-17T06:55:57.022671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:29.876784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988691+0000","last_change":"2024-07-17T06:55:55.988691+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988691+0000","last_undegraded":"2024-07-17T06:55:55.988691+0000","last_fullsized":"2024-07-17T06:55:55.988691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.007055+0000","last_change":"2024-07-17T06:55:57.007055+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.007055+0000","last_undegraded":"2024-07-17T06:55:57.007055+0000","last_fullsized":"2024-07-17T06:55:57.007055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988678+0000","last_change":"2024-07-17T06:55:55.988678+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988678+0000","last_undegraded":"2024-07-17T06:55:55.988678+0000","last_fullsized":"2024-07-17T06:55:55.988678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.013267+0000","last_change":"2024-07-17T06:55:57.013267+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.013267+0000","last_undegraded":"2024-07-17T06:55:57.013267+0000","last_fullsized":"2024-07-17T06:55:57.013267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.001021+0000","last_change":"2024-07-17T06:55:56.001021+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.001021+0000","last_undegraded":"2024-07-17T06:55:56.001021+0000","last_fullsized":"2024-07-17T06:55:56.001021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.000933+0000","last_change":"2024-07-17T06:55:57.000933+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.000933+0000","last_undegraded":"2024-07-17T06:55:57.000933+0000","last_fullsized":"2024-07-17T06:55:57.000933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988719+0000","last_change":"2024-07-17T06:55:55.988719+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988719+0000","last_undegraded":"2024-07-17T06:55:55.988719+0000","last_fullsized":"2024-07-17T06:55:55.988719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.003230+0000","last_change":"2024-07-17T06:55:57.003230+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.003230+0000","last_undegraded":"2024-07-17T06:55:57.003230+0000","last_fullsized":"2024-07-17T06:55:57.003230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.031012+0000","last_change":"2024-07-17T06:55:57.029338+0000","last_active":"2024-07-17T06:55:57.031012+0000","last_peered":"2024-07-17T06:55:57.031012+0000","last_clean":"2024-07-17T06:55:57.031012+0000","last_became_active":"2024-07-17T06:55:57.028513+0000","last_became_peered":"2024-07-17T06:55:57.028513+0000","last_unstale":"2024-07-17T06:55:57.031012+0000","last_undegraded":"2024-07-17T06:55:57.031012+0000","last_fullsized":"2024-07-17T06:55:57.031012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.128677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.988267+0000","last_change":"2024-07-17T06:55:55.988267+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.988267+0000","last_undegraded":"2024-07-17T06:55:55.988267+0000","last_fullsized":"2024-07-17T06:55:55.988267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:56.999415+0000","last_change":"2024-07-17T06:55:56.999415+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.999415+0000","last_undegraded":"2024-07-17T06:55:56.999415+0000","last_fullsized":"2024-07-17T06:55:56.999415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.025686+0000","last_change":"2024-07-17T06:55:57.020427+0000","last_active":"2024-07-17T06:55:57.025686+0000","last_peered":"2024-07-17T06:55:57.025686+0000","last_clean":"2024-07-17T06:55:57.025686+0000","last_became_active":"2024-07-17T06:55:57.020107+0000","last_became_peered":"2024-07-17T06:55:57.020107+0000","last_unstale":"2024-07-17T06:55:57.025686+0000","last_undegraded":"2024-07-17T06:55:57.025686+0000","last_fullsized":"2024-07-17T06:55:57.025686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:04.971712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.008761+0000","last_change":"2024-07-17T06:55:57.008761+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.008761+0000","last_undegraded":"2024-07-17T06:55:57.008761+0000","last_fullsized":"2024-07-17T06:55:57.008761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.023491+0000","last_change":"2024-07-17T06:55:57.018030+0000","last_active":"2024-07-17T06:55:57.023491+0000","last_peered":"2024-07-17T06:55:57.023491+0000","last_clean":"2024-07-17T06:55:57.023491+0000","last_became_active":"2024-07-17T06:55:57.017102+0000","last_became_peered":"2024-07-17T06:55:57.017102+0000","last_unstale":"2024-07-17T06:55:57.023491+0000","last_undegraded":"2024-07-17T06:55:57.023491+0000","last_fullsized":"2024-07-17T06:55:57.023491+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.005417+0000","last_change":"2024-07-17T06:55:57.005417+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.005417+0000","last_undegraded":"2024-07-17T06:55:57.005417+0000","last_fullsized":"2024-07-17T06:55:57.005417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.026967+0000","last_change":"2024-07-17T06:55:57.017786+0000","last_active":"2024-07-17T06:55:57.026967+0000","last_peered":"2024-07-17T06:55:57.026967+0000","last_clean":"2024-07-17T06:55:57.026967+0000","last_became_active":"2024-07-17T06:55:57.017469+0000","last_became_peered":"2024-07-17T06:55:57.017469+0000","last_unstale":"2024-07-17T06:55:57.026967+0000","last_undegraded":"2024-07-17T06:55:57.026967+0000","last_fullsized":"2024-07-17T06:55:57.026967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:50.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.989062+0000","last_change":"2024-07-17T06:55:55.989062+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.989062+0000","last_undegraded":"2024-07-17T06:55:55.989062+0000","last_fullsized":"2024-07-17T06:55:55.989062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T06:55:56.977863+0000","last_change":"2024-07-17T06:55:56.977863+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:56.977863+0000","last_undegraded":"2024-07-17T06:55:56.977863+0000","last_fullsized":"2024-07-17T06:55:56.977863+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-07-17T06:55:57.021919+0000","last_change":"2024-07-17T06:55:57.020044+0000","last_active":"2024-07-17T06:55:57.021919+0000","last_peered":"2024-07-17T06:55:57.021919+0000","last_clean":"2024-07-17T06:55:57.021919+0000","last_became_active":"2024-07-17T06:55:57.018728+0000","last_became_peered":"2024-07-17T06:55:57.018728+0000","last_unstale":"2024-07-17T06:55:57.021919+0000","last_undegraded":"2024-07-17T06:55:57.021919+0000","last_fullsized":"2024-07-17T06:55:57.021919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:48.777690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-07-17T06:55:57.018789+0000","last_change":"2024-07-17T06:55:57.018789+0000","last_active":"2024-07-17T06:55:56.977863+0000","last_peered":"2024-07-17T06:55:56.977863+0000","last_clean":"2024-07-17T06:55:56.977863+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:57.018789+0000","last_undegraded":"2024-07-17T06:55:57.018789+0000","last_fullsized":"2024-07-17T06:55:57.018789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-07-17T06:55:55.999523+0000","last_change":"2024-07-17T06:55:55.999523+0000","last_active":"2024-07-17T06:55:55.964774+0000","last_peered":"2024-07-17T06:55:55.964774+0000","last_clean":"2024-07-17T06:55:55.964774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T06:55:55.999523+0000","last_undegraded":"2024-07-17T06:55:55.999523+0000","last_fullsized":"2024-07-17T06:55:55.999523+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":135,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-07-17T06:55:56.002635+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:55:56.002635+0000","last_peered":"2024-07-17T06:55:56.002635+0000","last_clean":"2024-07-17T06:55:56.002635+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:55:56.002635+0000","last_undegraded":"2024-07-17T06:55:56.002635+0000","last_fullsized":"2024-07-17T06:55:56.002635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":177824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1011712,"data_stored":768192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":761525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":164948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":171157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:56:03.522 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-07-17T06:56:03.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.008688+0000', 'last_change': '2024-07-17T06:55:57.008688+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.008688+0000', 'last_undegraded': '2024-07-17T06:55:57.008688+0000', 'last_fullsized': '2024-07-17T06:55:57.008688+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.522 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.005414+0000', 'last_change': '2024-07-17T06:55:57.005414+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.005414+0000', 'last_undegraded': '2024-07-17T06:55:57.005414+0000', 'last_fullsized': '2024-07-17T06:55:57.005414+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.016491+0000', 'last_change': '2024-07-17T06:55:57.016491+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.016491+0000', 'last_undegraded': '2024-07-17T06:55:57.016491+0000', 'last_fullsized': '2024-07-17T06:55:57.016491+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.003049+0000', 'last_change': '2024-07-17T06:55:57.003049+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.003049+0000', 'last_undegraded': '2024-07-17T06:55:57.003049+0000', 'last_fullsized': '2024-07-17T06:55:57.003049+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.523 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.008287+0000', 'last_change': '2024-07-17T06:55:57.008287+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.008287+0000', 'last_undegraded': '2024-07-17T06:55:57.008287+0000', 'last_fullsized': '2024-07-17T06:55:57.008287+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.999693+0000', 'last_change': '2024-07-17T06:55:56.999693+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.999693+0000', 'last_undegraded': '2024-07-17T06:55:56.999693+0000', 'last_fullsized': '2024-07-17T06:55:56.999693+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007070+0000', 'last_change': '2024-07-17T06:55:57.007070+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007070+0000', 'last_undegraded': '2024-07-17T06:55:57.007070+0000', 'last_fullsized': '2024-07-17T06:55:57.007070+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.018563+0000', 'last_change': '2024-07-17T06:55:57.018563+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.018563+0000', 'last_undegraded': '2024-07-17T06:55:57.018563+0000', 'last_fullsized': '2024-07-17T06:55:57.018563+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-07-17T06:56:03.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.019159+0000', 'last_change': '2024-07-17T06:55:57.019159+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.019159+0000', 'last_undegraded': '2024-07-17T06:55:57.019159+0000', 'last_fullsized': '2024-07-17T06:55:57.019159+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007239+0000', 'last_change': '2024-07-17T06:55:57.007239+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007239+0000', 'last_undegraded': '2024-07-17T06:55:57.007239+0000', 'last_fullsized': '2024-07-17T06:55:57.007239+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.019874+0000', 'last_change': '2024-07-17T06:55:57.019874+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.019874+0000', 'last_undegraded': '2024-07-17T06:55:57.019874+0000', 'last_fullsized': '2024-07-17T06:55:57.019874+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007767+0000', 'last_change': '2024-07-17T06:55:57.007767+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007767+0000', 'last_undegraded': '2024-07-17T06:55:57.007767+0000', 'last_fullsized': '2024-07-17T06:55:57.007767+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-07-17T06:56:03.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.998191+0000', 'last_change': '2024-07-17T06:55:56.998191+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.998191+0000', 'last_undegraded': '2024-07-17T06:55:56.998191+0000', 'last_fullsized': '2024-07-17T06:55:56.998191+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.996430+0000', 'last_change': '2024-07-17T06:55:56.996430+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.996430+0000', 'last_undegraded': '2024-07-17T06:55:56.996430+0000', 'last_fullsized': '2024-07-17T06:55:56.996430+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.526 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.019375+0000', 'last_change': '2024-07-17T06:55:57.019375+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.019375+0000', 'last_undegraded': '2024-07-17T06:55:57.019375+0000', 'last_fullsized': '2024-07-17T06:55:57.019375+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-07-17T06:56:03.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.997729+0000', 'last_change': '2024-07-17T06:55:56.997729+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.997729+0000', 'last_undegraded': '2024-07-17T06:55:56.997729+0000', 'last_fullsized': '2024-07-17T06:55:56.997729+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.005523+0000', 'last_change': '2024-07-17T06:55:57.005523+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.005523+0000', 'last_undegraded': '2024-07-17T06:55:57.005523+0000', 'last_fullsized': '2024-07-17T06:55:57.005523+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.996319+0000', 'last_change': '2024-07-17T06:55:56.996319+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.996319+0000', 'last_undegraded': '2024-07-17T06:55:56.996319+0000', 'last_fullsized': '2024-07-17T06:55:56.996319+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.012155+0000', 'last_change': '2024-07-17T06:55:57.012155+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.012155+0000', 'last_undegraded': '2024-07-17T06:55:57.012155+0000', 'last_fullsized': '2024-07-17T06:55:57.012155+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.994916+0000', 'last_change': '2024-07-17T06:55:56.994916+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.994916+0000', 'last_undegraded': '2024-07-17T06:55:56.994916+0000', 'last_fullsized': '2024-07-17T06:55:56.994916+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-07-17T06:56:03.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.994476+0000', 'last_change': '2024-07-17T06:55:56.994476+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.994476+0000', 'last_undegraded': '2024-07-17T06:55:56.994476+0000', 'last_fullsized': '2024-07-17T06:55:56.994476+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.004910+0000', 'last_change': '2024-07-17T06:55:57.004910+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.004910+0000', 'last_undegraded': '2024-07-17T06:55:57.004910+0000', 'last_fullsized': '2024-07-17T06:55:57.004910+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.002945+0000', 'last_change': '2024-07-17T06:55:57.002945+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.002945+0000', 'last_undegraded': '2024-07-17T06:55:57.002945+0000', 'last_fullsized': '2024-07-17T06:55:57.002945+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-07-17T06:56:03.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.006970+0000', 'last_change': '2024-07-17T06:55:57.006970+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.006970+0000', 'last_undegraded': '2024-07-17T06:55:57.006970+0000', 'last_fullsized': '2024-07-17T06:55:57.006970+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.991195+0000', 'last_change': '2024-07-17T06:55:55.991195+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.991195+0000', 'last_undegraded': '2024-07-17T06:55:55.991195+0000', 'last_fullsized': '2024-07-17T06:55:55.991195+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.009696+0000', 'last_change': '2024-07-17T06:55:57.009696+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.009696+0000', 'last_undegraded': '2024-07-17T06:55:57.009696+0000', 'last_fullsized': '2024-07-17T06:55:57.009696+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007048+0000', 'last_change': '2024-07-17T06:55:57.007048+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007048+0000', 'last_undegraded': '2024-07-17T06:55:57.007048+0000', 'last_fullsized': '2024-07-17T06:55:57.007048+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.530 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.000923+0000', 'last_change': '2024-07-17T06:55:56.000923+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.000923+0000', 'last_undegraded': '2024-07-17T06:55:56.000923+0000', 'last_fullsized': '2024-07-17T06:55:56.000923+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.001923+0000', 'last_change': '2024-07-17T06:55:57.001923+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.001923+0000', 'last_undegraded': '2024-07-17T06:55:57.001923+0000', 'last_fullsized': '2024-07-17T06:55:57.001923+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.991226+0000', 'last_change': '2024-07-17T06:55:55.991226+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.991226+0000', 'last_undegraded': '2024-07-17T06:55:55.991226+0000', 'last_fullsized': '2024-07-17T06:55:55.991226+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.531 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.008333+0000', 'last_change': '2024-07-17T06:55:57.008333+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.008333+0000', 'last_undegraded': '2024-07-17T06:55:57.008333+0000', 'last_fullsized': '2024-07-17T06:55:57.008333+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.009235+0000', 'last_change': '2024-07-17T06:55:57.009235+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.009235+0000', 'last_undegraded': '2024-07-17T06:55:57.009235+0000', 'last_fullsized': '2024-07-17T06:55:57.009235+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-07-17T06:56:03.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.998505+0000', 'last_change': '2024-07-17T06:55:56.998505+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.998505+0000', 'last_undegraded': '2024-07-17T06:55:56.998505+0000', 'last_fullsized': '2024-07-17T06:55:56.998505+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.001451+0000', 'last_change': '2024-07-17T06:55:57.001451+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.001451+0000', 'last_undegraded': '2024-07-17T06:55:57.001451+0000', 'last_fullsized': '2024-07-17T06:55:57.001451+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.001437+0000', 'last_change': '2024-07-17T06:55:56.001437+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.001437+0000', 'last_undegraded': '2024-07-17T06:55:56.001437+0000', 'last_fullsized': '2024-07-17T06:55:56.001437+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007875+0000', 'last_change': '2024-07-17T06:55:57.007875+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007875+0000', 'last_undegraded': '2024-07-17T06:55:57.007875+0000', 'last_fullsized': '2024-07-17T06:55:57.007875+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-07-17T06:56:03.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.014177+0000', 'last_change': '2024-07-17T06:55:57.014177+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.014177+0000', 'last_undegraded': '2024-07-17T06:55:57.014177+0000', 'last_fullsized': '2024-07-17T06:55:57.014177+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.992431+0000', 'last_change': '2024-07-17T06:55:55.992431+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.992431+0000', 'last_undegraded': '2024-07-17T06:55:55.992431+0000', 'last_fullsized': '2024-07-17T06:55:55.992431+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.988522+0000', 'last_change': '2024-07-17T06:55:55.988522+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.988522+0000', 'last_undegraded': '2024-07-17T06:55:55.988522+0000', 'last_fullsized': '2024-07-17T06:55:55.988522+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-07-17T06:56:03.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.988691+0000', 'last_change': '2024-07-17T06:55:55.988691+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.988691+0000', 'last_undegraded': '2024-07-17T06:55:55.988691+0000', 'last_fullsized': '2024-07-17T06:55:55.988691+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.007055+0000', 'last_change': '2024-07-17T06:55:57.007055+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.007055+0000', 'last_undegraded': '2024-07-17T06:55:57.007055+0000', 'last_fullsized': '2024-07-17T06:55:57.007055+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.988678+0000', 'last_change': '2024-07-17T06:55:55.988678+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.988678+0000', 'last_undegraded': '2024-07-17T06:55:55.988678+0000', 'last_fullsized': '2024-07-17T06:55:55.988678+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.013267+0000', 'last_change': '2024-07-17T06:55:57.013267+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.013267+0000', 'last_undegraded': '2024-07-17T06:55:57.013267+0000', 'last_fullsized': '2024-07-17T06:55:57.013267+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.535 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.001021+0000', 'last_change': '2024-07-17T06:55:56.001021+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.001021+0000', 'last_undegraded': '2024-07-17T06:55:56.001021+0000', 'last_fullsized': '2024-07-17T06:55:56.001021+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.000933+0000', 'last_change': '2024-07-17T06:55:57.000933+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.000933+0000', 'last_undegraded': '2024-07-17T06:55:57.000933+0000', 'last_fullsized': '2024-07-17T06:55:57.000933+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.988719+0000', 'last_change': '2024-07-17T06:55:55.988719+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.988719+0000', 'last_undegraded': '2024-07-17T06:55:55.988719+0000', 'last_fullsized': '2024-07-17T06:55:55.988719+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.003230+0000', 'last_change': '2024-07-17T06:55:57.003230+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.003230+0000', 'last_undegraded': '2024-07-17T06:55:57.003230+0000', 'last_fullsized': '2024-07-17T06:55:57.003230+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.536 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.988267+0000', 'last_change': '2024-07-17T06:55:55.988267+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.988267+0000', 'last_undegraded': '2024-07-17T06:55:55.988267+0000', 'last_fullsized': '2024-07-17T06:55:55.988267+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:56.999415+0000', 'last_change': '2024-07-17T06:55:56.999415+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.999415+0000', 'last_undegraded': '2024-07-17T06:55:56.999415+0000', 'last_fullsized': '2024-07-17T06:55:56.999415+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.008761+0000', 'last_change': '2024-07-17T06:55:57.008761+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.008761+0000', 'last_undegraded': '2024-07-17T06:55:57.008761+0000', 'last_fullsized': '2024-07-17T06:55:57.008761+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.005417+0000', 'last_change': '2024-07-17T06:55:57.005417+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.005417+0000', 'last_undegraded': '2024-07-17T06:55:57.005417+0000', 'last_fullsized': '2024-07-17T06:55:57.005417+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.537 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.989062+0000', 'last_change': '2024-07-17T06:55:55.989062+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.989062+0000', 'last_undegraded': '2024-07-17T06:55:55.989062+0000', 'last_fullsized': '2024-07-17T06:55:55.989062+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-07-17T06:55:56.977863+0000', 'last_change': '2024-07-17T06:55:56.977863+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:56.977863+0000', 'last_undegraded': '2024-07-17T06:55:56.977863+0000', 'last_fullsized': '2024-07-17T06:55:56.977863+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:57.018789+0000', 'last_change': '2024-07-17T06:55:57.018789+0000', 'last_active': '2024-07-17T06:55:56.977863+0000', 'last_peered': '2024-07-17T06:55:56.977863+0000', 'last_clean': '2024-07-17T06:55:56.977863+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:57.018789+0000', 'last_undegraded': '2024-07-17T06:55:57.018789+0000', 'last_fullsized': '2024-07-17T06:55:57.018789+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:56.977863+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-07-17T06:56:03.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-07-17T06:55:55.999523+0000', 'last_change': '2024-07-17T06:55:55.999523+0000', 'last_active': '2024-07-17T06:55:55.964774+0000', 'last_peered': '2024-07-17T06:55:55.964774+0000', 'last_clean': '2024-07-17T06:55:55.964774+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T06:55:55.999523+0000', 'last_undegraded': '2024-07-17T06:55:55.999523+0000', 'last_fullsized': '2024-07-17T06:55:55.999523+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'last_clean_scrub_stamp': '2024-07-17T06:55:55.964774+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T06:56:03.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-07-17T06:56:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 -- 172.21.15.44:0/1176915052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 msgr2=0x7f45c4074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 --2- 172.21.15.44:0/1176915052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c4074210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45c40725c0 tx=0x7f45bc02a980 comp rx=0 tx=0).stop 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 -- 172.21.15.44:0/1176915052 shutdown_connections 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 --2- 172.21.15.44:0/1176915052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c40b2a00 0x7f45c40738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 --2- 172.21.15.44:0/1176915052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c4074210 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 --2- 172.21.15.44:0/1176915052 >> v2:172.21.15.175:3300/0 conn(0x7f45c415b1d0 0x7f45c40add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 -- 172.21.15.44:0/1176915052 >> 172.21.15.44:0/1176915052 conn(0x7f45c40b5890 msgr2=0x7f45c40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 -- 172.21.15.44:0/1176915052 shutdown_connections 2024-07-17T06:56:03.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.781+0000 7f45c8a00640 1 -- 172.21.15.44:0/1176915052 wait complete. 2024-07-17T06:56:03.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.783+0000 7f45c8a00640 1 Processor -- start 2024-07-17T06:56:03.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.783+0000 7f45c8a00640 1 -- start start 2024-07-17T06:56:03.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c40b2a00 0x7f45c40cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c415b1d0 0x7f45c41563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45c415f140 con 0x7f45c4073e00 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45c415efc0 con 0x7f45c415b1d0 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.784+0000 7f45c8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45c415f2c0 con 0x7f45c40b2a00 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43310/0 (socket says 172.21.15.44:43310) 2024-07-17T06:56:03.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 -- 172.21.15.44:0/2758695450 learned_addr learned my addr 172.21.15.44:0/2758695450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c3200640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c415b1d0 0x7f45c41563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 -- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c40b2a00 msgr2=0x7f45c40cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c40b2a00 0x7f45c40cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 -- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c415b1d0 msgr2=0x7f45c41563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c415b1d0 0x7f45c41563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.785+0000 7f45c2800640 1 -- 172.21.15.44:0/2758695450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c4158770 con 0x7f45c4073e00 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45c2800640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00d740 tx=0x7f45ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac017070 con 0x7f45c4073e00 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ac009d20 con 0x7f45c4073e00 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0106c0 con 0x7f45c4073e00 2024-07-17T06:56:03.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c4158a50 con 0x7f45c4073e00 2024-07-17T06:56:03.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.786+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c41583b0 con 0x7f45c4073e00 2024-07-17T06:56:03.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.787+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c4074210 con 0x7f45c4073e00 2024-07-17T06:56:03.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.788+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f45ac01a070 con 0x7f45c4073e00 2024-07-17T06:56:03.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.790+0000 7f45baa00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 0x7f45a0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:03.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.790+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f45ac098540 con 0x7f45c4073e00 2024-07-17T06:56:03.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.790+0000 7f45c1e00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 0x7f45a0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:03.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.792+0000 7f45c1e00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 0x7f45a0079070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45c415aca0 tx=0x7f45b4005c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:03.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:03.794+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f45ac061870 con 0x7f45c4073e00 2024-07-17T06:56:04.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.100+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f45c40b4500 con 0x7f45c4073e00 2024-07-17T06:56:04.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.102+0000 7f45baa00640 1 -- 172.21.15.44:0/2758695450 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1054 (secure 0 0 0) 0x7f45ac061170 con 0x7f45c4073e00 2024-07-17T06:56:04.101 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:04.101 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":95,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1721199282,"num_in_osds":8,"osd_in_since":1721199273,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":226168832,"bytes_avail":772867944448,"bytes_total":773094113280,"write_bytes_sec":2123,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4738}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-07-17T06:55:45.733148+0000","services":{}},"progress_events":{}} 2024-07-17T06:56:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.104+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 msgr2=0x7f45a0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.104+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 0x7f45a0079070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45c415aca0 tx=0x7f45b4005c00 comp rx=0 tx=0).stop 2024-07-17T06:56:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.104+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 msgr2=0x7f45c415e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.104+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00d740 tx=0x7f45ac00dc00 comp rx=0 tx=0).stop 2024-07-17T06:56:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 shutdown_connections 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c415b1d0 0x7f45c41563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f45a0076bb0 0x7f45a0079070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c40b2a00 0x7f45c40cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 --2- 172.21.15.44:0/2758695450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45c4073e00 0x7f45c415e090 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 >> 172.21.15.44:0/2758695450 conn(0x7f45c40b5890 msgr2=0x7f45c406f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 shutdown_connections 2024-07-17T06:56:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.105+0000 7f45c8a00640 1 -- 172.21.15.44:0/2758695450 wait complete. 2024-07-17T06:56:04.123 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-07-17T06:56:04.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:56:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 -- 172.21.15.44:0/2755817162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c15b2e0 msgr2=0x7f9a6c0ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 --2- 172.21.15.44:0/2755817162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c0ac650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009830 tx=0x7f9a5c02a510 comp rx=0 tx=0).stop 2024-07-17T06:56:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 -- 172.21.15.44:0/2755817162 shutdown_connections 2024-07-17T06:56:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 --2- 172.21.15.44:0/2755817162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c0ac650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 -- 172.21.15.44:0/2755817162 >> 172.21.15.44:0/2755817162 conn(0x7f9a6c0b0fb0 msgr2=0x7f9a6c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 -- 172.21.15.44:0/2755817162 shutdown_connections 2024-07-17T06:56:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.317+0000 7f9a71000640 1 -- 172.21.15.44:0/2755817162 wait complete. 2024-07-17T06:56:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.319+0000 7f9a71000640 1 Processor -- start 2024-07-17T06:56:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.320+0000 7f9a71000640 1 -- start start 2024-07-17T06:56:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6c0acb90 0x7f9a6c1f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c1f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c154550 con 0x7f9a6c06f730 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c15cf80 con 0x7f9a6c15b2e0 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a71000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c1546d0 con 0x7f9a6c0acb90 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a69e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a69e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43330/0 (socket says 172.21.15.44:43330) 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.321+0000 7f9a69e00640 1 -- 172.21.15.44:0/723264158 learned_addr learned my addr 172.21.15.44:0/723264158 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a6a800640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c1f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69400640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6c0acb90 0x7f9a6c1f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 -- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6c0acb90 msgr2=0x7f9a6c1f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6c0acb90 0x7f9a6c1f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 -- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6c15b2e0 msgr2=0x7f9a6c1f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c1f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 -- 172.21.15.44:0/723264158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c1f1e30 con 0x7f9a6c06f730 2024-07-17T06:56:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a69e00640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c000c00 tx=0x7f9a5c033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c031d60 con 0x7f9a6c06f730 2024-07-17T06:56:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a5c0330d0 con 0x7f9a6c06f730 2024-07-17T06:56:04.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c033520 con 0x7f9a6c06f730 2024-07-17T06:56:04.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.322+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6c072860 con 0x7f9a6c06f730 2024-07-17T06:56:04.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.323+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6c072e20 con 0x7f9a6c06f730 2024-07-17T06:56:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.324+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6c0ac650 con 0x7f9a6c06f730 2024-07-17T06:56:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.325+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f9a5c0035f0 con 0x7f9a6c06f730 2024-07-17T06:56:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.326+0000 7f9a62000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 0x7f9a4c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.326+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9a5c0b9860 con 0x7f9a6c06f730 2024-07-17T06:56:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.326+0000 7f9a69400640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 0x7f9a4c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.328+0000 7f9a69400640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 0x7f9a4c079070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c14b780 tx=0x7f9a58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:04.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.330+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9a5c03f030 con 0x7f9a6c06f730 2024-07-17T06:56:04.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.528+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a6c06dc60 con 0x7f9a4c076bb0 2024-07-17T06:56:04.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.542+0000 7f9a62000640 1 -- 172.21.15.44:0/723264158 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f9a6c06dc60 con 0x7f9a4c076bb0 2024-07-17T06:56:04.546 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:04.547 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:56:04.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.548+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 msgr2=0x7f9a4c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.548+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 0x7f9a4c079070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c14b780 tx=0x7f9a58009040 comp rx=0 tx=0).stop 2024-07-17T06:56:04.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.548+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 msgr2=0x7f9a6c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.548+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c000c00 tx=0x7f9a5c033d60 comp rx=0 tx=0).stop 2024-07-17T06:56:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 shutdown_connections 2024-07-17T06:56:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6c15b2e0 0x7f9a6c1f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9a4c076bb0 0x7f9a4c079070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6c0acb90 0x7f9a6c1f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 --2- 172.21.15.44:0/723264158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a6c06f730 0x7f9a6c152270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 >> 172.21.15.44:0/723264158 conn(0x7f9a6c0b0fb0 msgr2=0x7f9a6c159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 shutdown_connections 2024-07-17T06:56:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:04.549+0000 7f9a71000640 1 -- 172.21.15.44:0/723264158 wait complete. 2024-07-17T06:56:04.566 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-07-17T06:56:03.739030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220868,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753852,"statfs":{"total":773094113280,"available":772867944448,"internally_reserved":0,"allocated":4792320,"data_stored":2719020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.750835"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659819+0000","last_change":"2024-07-17T06:55:58.045274+0000","last_active":"2024-07-17T06:55:59.659819+0000","last_peered":"2024-07-17T06:55:59.659819+0000","last_clean":"2024-07-17T06:55:59.659819+0000","last_became_active":"2024-07-17T06:55:58.044983+0000","last_became_peered":"2024-07-17T06:55:58.044983+0000","last_unstale":"2024-07-17T06:55:59.659819+0000","last_undegraded":"2024-07-17T06:55:59.659819+0000","last_fullsized":"2024-07-17T06:55:59.659819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:56.755380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658713+0000","last_change":"2024-07-17T06:55:58.035807+0000","last_active":"2024-07-17T06:55:59.658713+0000","last_peered":"2024-07-17T06:55:59.658713+0000","last_clean":"2024-07-17T06:55:59.658713+0000","last_became_active":"2024-07-17T06:55:58.035523+0000","last_became_peered":"2024-07-17T06:55:58.035523+0000","last_unstale":"2024-07-17T06:55:59.658713+0000","last_undegraded":"2024-07-17T06:55:59.658713+0000","last_fullsized":"2024-07-17T06:55:59.658713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:29.515079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660443+0000","last_change":"2024-07-17T06:55:58.063675+0000","last_active":"2024-07-17T06:55:59.660443+0000","last_peered":"2024-07-17T06:55:59.660443+0000","last_clean":"2024-07-17T06:55:59.660443+0000","last_became_active":"2024-07-17T06:55:58.063343+0000","last_became_peered":"2024-07-17T06:55:58.063343+0000","last_unstale":"2024-07-17T06:55:59.660443+0000","last_undegraded":"2024-07-17T06:55:59.660443+0000","last_fullsized":"2024-07-17T06:55:59.660443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:55.758472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036936+0000","last_change":"2024-07-17T06:55:58.043654+0000","last_active":"2024-07-17T06:55:59.036936+0000","last_peered":"2024-07-17T06:55:59.036936+0000","last_clean":"2024-07-17T06:55:59.036936+0000","last_became_active":"2024-07-17T06:55:58.042825+0000","last_became_peered":"2024-07-17T06:55:58.042825+0000","last_unstale":"2024-07-17T06:55:59.036936+0000","last_undegraded":"2024-07-17T06:55:59.036936+0000","last_fullsized":"2024-07-17T06:55:59.036936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:03.993952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035998+0000","last_change":"2024-07-17T06:55:58.040071+0000","last_active":"2024-07-17T06:55:59.035998+0000","last_peered":"2024-07-17T06:55:59.035998+0000","last_clean":"2024-07-17T06:55:59.035998+0000","last_became_active":"2024-07-17T06:55:58.039796+0000","last_became_peered":"2024-07-17T06:55:58.039796+0000","last_unstale":"2024-07-17T06:55:59.035998+0000","last_undegraded":"2024-07-17T06:55:59.035998+0000","last_fullsized":"2024-07-17T06:55:59.035998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:00.706573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655654+0000","last_change":"2024-07-17T06:55:58.033469+0000","last_active":"2024-07-17T06:55:59.655654+0000","last_peered":"2024-07-17T06:55:59.655654+0000","last_clean":"2024-07-17T06:55:59.655654+0000","last_became_active":"2024-07-17T06:55:58.032268+0000","last_became_peered":"2024-07-17T06:55:58.032268+0000","last_unstale":"2024-07-17T06:55:59.655654+0000","last_undegraded":"2024-07-17T06:55:59.655654+0000","last_fullsized":"2024-07-17T06:55:59.655654+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043885+0000","last_change":"2024-07-17T06:55:58.057539+0000","last_active":"2024-07-17T06:55:59.043885+0000","last_peered":"2024-07-17T06:55:59.043885+0000","last_clean":"2024-07-17T06:55:59.043885+0000","last_became_active":"2024-07-17T06:55:58.057288+0000","last_became_peered":"2024-07-17T06:55:58.057288+0000","last_unstale":"2024-07-17T06:55:59.043885+0000","last_undegraded":"2024-07-17T06:55:59.043885+0000","last_fullsized":"2024-07-17T06:55:59.043885+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:50:35.078043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657003+0000","last_change":"2024-07-17T06:55:58.060879+0000","last_active":"2024-07-17T06:55:59.657003+0000","last_peered":"2024-07-17T06:55:59.657003+0000","last_clean":"2024-07-17T06:55:59.657003+0000","last_became_active":"2024-07-17T06:55:58.060658+0000","last_became_peered":"2024-07-17T06:55:58.060658+0000","last_unstale":"2024-07-17T06:55:59.657003+0000","last_undegraded":"2024-07-17T06:55:59.657003+0000","last_fullsized":"2024-07-17T06:55:59.657003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:10:07.906199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.661028+0000","last_change":"2024-07-17T06:55:58.063719+0000","last_active":"2024-07-17T06:55:59.661028+0000","last_peered":"2024-07-17T06:55:59.661028+0000","last_clean":"2024-07-17T06:55:59.661028+0000","last_became_active":"2024-07-17T06:55:58.063435+0000","last_became_peered":"2024-07-17T06:55:58.063435+0000","last_unstale":"2024-07-17T06:55:59.661028+0000","last_undegraded":"2024-07-17T06:55:59.661028+0000","last_fullsized":"2024-07-17T06:55:59.661028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:18.810801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047019+0000","last_change":"2024-07-17T06:55:58.066563+0000","last_active":"2024-07-17T06:55:59.047019+0000","last_peered":"2024-07-17T06:55:59.047019+0000","last_clean":"2024-07-17T06:55:59.047019+0000","last_became_active":"2024-07-17T06:55:58.066294+0000","last_became_peered":"2024-07-17T06:55:58.066294+0000","last_unstale":"2024-07-17T06:55:59.047019+0000","last_undegraded":"2024-07-17T06:55:59.047019+0000","last_fullsized":"2024-07-17T06:55:59.047019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:56.755380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047255+0000","last_change":"2024-07-17T06:55:58.053366+0000","last_active":"2024-07-17T06:55:59.047255+0000","last_peered":"2024-07-17T06:55:59.047255+0000","last_clean":"2024-07-17T06:55:59.047255+0000","last_became_active":"2024-07-17T06:55:58.053049+0000","last_became_peered":"2024-07-17T06:55:58.053049+0000","last_unstale":"2024-07-17T06:55:59.047255+0000","last_undegraded":"2024-07-17T06:55:59.047255+0000","last_fullsized":"2024-07-17T06:55:59.047255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:00:10.987843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660422+0000","last_change":"2024-07-17T06:55:58.034823+0000","last_active":"2024-07-17T06:55:59.660422+0000","last_peered":"2024-07-17T06:55:59.660422+0000","last_clean":"2024-07-17T06:55:59.660422+0000","last_became_active":"2024-07-17T06:55:58.034497+0000","last_became_peered":"2024-07-17T06:55:58.034497+0000","last_unstale":"2024-07-17T06:55:59.660422+0000","last_undegraded":"2024-07-17T06:55:59.660422+0000","last_fullsized":"2024-07-17T06:55:59.660422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:27.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045419+0000","last_change":"2024-07-17T06:55:58.050205+0000","last_active":"2024-07-17T06:55:59.045419+0000","last_peered":"2024-07-17T06:55:59.045419+0000","last_clean":"2024-07-17T06:55:59.045419+0000","last_became_active":"2024-07-17T06:55:58.049877+0000","last_became_peered":"2024-07-17T06:55:58.049877+0000","last_unstale":"2024-07-17T06:55:59.045419+0000","last_undegraded":"2024-07-17T06:55:59.045419+0000","last_fullsized":"2024-07-17T06:55:59.045419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:29.515079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042997+0000","last_change":"2024-07-17T06:55:58.055343+0000","last_active":"2024-07-17T06:55:59.042997+0000","last_peered":"2024-07-17T06:55:59.042997+0000","last_clean":"2024-07-17T06:55:59.042997+0000","last_became_active":"2024-07-17T06:55:58.054518+0000","last_became_peered":"2024-07-17T06:55:58.054518+0000","last_unstale":"2024-07-17T06:55:59.042997+0000","last_undegraded":"2024-07-17T06:55:59.042997+0000","last_fullsized":"2024-07-17T06:55:59.042997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:54:56.542624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044115+0000","last_change":"2024-07-17T06:55:58.058751+0000","last_active":"2024-07-17T06:55:59.044115+0000","last_peered":"2024-07-17T06:55:59.044115+0000","last_clean":"2024-07-17T06:55:59.044115+0000","last_became_active":"2024-07-17T06:55:58.058406+0000","last_became_peered":"2024-07-17T06:55:58.058406+0000","last_unstale":"2024-07-17T06:55:59.044115+0000","last_undegraded":"2024-07-17T06:55:59.044115+0000","last_fullsized":"2024-07-17T06:55:59.044115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:42.383602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656349+0000","last_change":"2024-07-17T06:55:58.063260+0000","last_active":"2024-07-17T06:55:59.656349+0000","last_peered":"2024-07-17T06:55:59.656349+0000","last_clean":"2024-07-17T06:55:59.656349+0000","last_became_active":"2024-07-17T06:55:58.063013+0000","last_became_peered":"2024-07-17T06:55:58.063013+0000","last_unstale":"2024-07-17T06:55:59.656349+0000","last_undegraded":"2024-07-17T06:55:59.656349+0000","last_fullsized":"2024-07-17T06:55:59.656349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:25.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657077+0000","last_change":"2024-07-17T06:55:58.060726+0000","last_active":"2024-07-17T06:55:59.657077+0000","last_peered":"2024-07-17T06:55:59.657077+0000","last_clean":"2024-07-17T06:55:59.657077+0000","last_became_active":"2024-07-17T06:55:58.060398+0000","last_became_peered":"2024-07-17T06:55:58.060398+0000","last_unstale":"2024-07-17T06:55:59.657077+0000","last_undegraded":"2024-07-17T06:55:59.657077+0000","last_fullsized":"2024-07-17T06:55:59.657077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:44.785341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046069+0000","last_change":"2024-07-17T06:55:58.058568+0000","last_active":"2024-07-17T06:55:59.046069+0000","last_peered":"2024-07-17T06:55:59.046069+0000","last_clean":"2024-07-17T06:55:59.046069+0000","last_became_active":"2024-07-17T06:55:58.058044+0000","last_became_peered":"2024-07-17T06:55:58.058044+0000","last_unstale":"2024-07-17T06:55:59.046069+0000","last_undegraded":"2024-07-17T06:55:59.046069+0000","last_fullsized":"2024-07-17T06:55:59.046069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036288+0000","last_change":"2024-07-17T06:55:58.032879+0000","last_active":"2024-07-17T06:55:59.036288+0000","last_peered":"2024-07-17T06:55:59.036288+0000","last_clean":"2024-07-17T06:55:59.036288+0000","last_became_active":"2024-07-17T06:55:58.032598+0000","last_became_peered":"2024-07-17T06:55:58.032598+0000","last_unstale":"2024-07-17T06:55:59.036288+0000","last_undegraded":"2024-07-17T06:55:59.036288+0000","last_fullsized":"2024-07-17T06:55:59.036288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:34.827013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044726+0000","last_change":"2024-07-17T06:55:58.051885+0000","last_active":"2024-07-17T06:55:59.044726+0000","last_peered":"2024-07-17T06:55:59.044726+0000","last_clean":"2024-07-17T06:55:59.044726+0000","last_became_active":"2024-07-17T06:55:58.051588+0000","last_became_peered":"2024-07-17T06:55:58.051588+0000","last_unstale":"2024-07-17T06:55:59.044726+0000","last_undegraded":"2024-07-17T06:55:59.044726+0000","last_fullsized":"2024-07-17T06:55:59.044726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:31.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045597+0000","last_change":"2024-07-17T06:55:58.057046+0000","last_active":"2024-07-17T06:55:59.045597+0000","last_peered":"2024-07-17T06:55:59.045597+0000","last_clean":"2024-07-17T06:55:59.045597+0000","last_became_active":"2024-07-17T06:55:58.055803+0000","last_became_peered":"2024-07-17T06:55:58.055803+0000","last_unstale":"2024-07-17T06:55:59.045597+0000","last_undegraded":"2024-07-17T06:55:59.045597+0000","last_fullsized":"2024-07-17T06:55:59.045597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:31:51.390519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042568+0000","last_change":"2024-07-17T06:55:58.033523+0000","last_active":"2024-07-17T06:55:59.042568+0000","last_peered":"2024-07-17T06:55:59.042568+0000","last_clean":"2024-07-17T06:55:59.042568+0000","last_became_active":"2024-07-17T06:55:58.032980+0000","last_became_peered":"2024-07-17T06:55:58.032980+0000","last_unstale":"2024-07-17T06:55:59.042568+0000","last_undegraded":"2024-07-17T06:55:59.042568+0000","last_fullsized":"2024-07-17T06:55:59.042568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:24.777827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656328+0000","last_change":"2024-07-17T06:55:58.045860+0000","last_active":"2024-07-17T06:55:59.656328+0000","last_peered":"2024-07-17T06:55:59.656328+0000","last_clean":"2024-07-17T06:55:59.656328+0000","last_became_active":"2024-07-17T06:55:58.045527+0000","last_became_peered":"2024-07-17T06:55:58.045527+0000","last_unstale":"2024-07-17T06:55:59.656328+0000","last_undegraded":"2024-07-17T06:55:59.656328+0000","last_fullsized":"2024-07-17T06:55:59.656328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:16.686880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035342+0000","last_change":"2024-07-17T06:55:58.040719+0000","last_active":"2024-07-17T06:55:59.035342+0000","last_peered":"2024-07-17T06:55:59.035342+0000","last_clean":"2024-07-17T06:55:59.035342+0000","last_became_active":"2024-07-17T06:55:58.040212+0000","last_became_peered":"2024-07-17T06:55:58.040212+0000","last_unstale":"2024-07-17T06:55:59.035342+0000","last_undegraded":"2024-07-17T06:55:59.035342+0000","last_fullsized":"2024-07-17T06:55:59.035342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:57.540694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046697+0000","last_change":"2024-07-17T06:55:58.059600+0000","last_active":"2024-07-17T06:55:59.046697+0000","last_peered":"2024-07-17T06:55:59.046697+0000","last_clean":"2024-07-17T06:55:59.046697+0000","last_became_active":"2024-07-17T06:55:58.053612+0000","last_became_peered":"2024-07-17T06:55:58.053612+0000","last_unstale":"2024-07-17T06:55:59.046697+0000","last_undegraded":"2024-07-17T06:55:59.046697+0000","last_fullsized":"2024-07-17T06:55:59.046697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658271+0000","last_change":"2024-07-17T06:55:58.053299+0000","last_active":"2024-07-17T06:55:59.658271+0000","last_peered":"2024-07-17T06:55:59.658271+0000","last_clean":"2024-07-17T06:55:59.658271+0000","last_became_active":"2024-07-17T06:55:58.053020+0000","last_became_peered":"2024-07-17T06:55:58.053020+0000","last_unstale":"2024-07-17T06:55:59.658271+0000","last_undegraded":"2024-07-17T06:55:59.658271+0000","last_fullsized":"2024-07-17T06:55:59.658271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:55.758472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045182+0000","last_change":"2024-07-17T06:55:58.054772+0000","last_active":"2024-07-17T06:55:59.045182+0000","last_peered":"2024-07-17T06:55:59.045182+0000","last_clean":"2024-07-17T06:55:59.045182+0000","last_became_active":"2024-07-17T06:55:58.054003+0000","last_became_peered":"2024-07-17T06:55:58.054003+0000","last_unstale":"2024-07-17T06:55:59.045182+0000","last_undegraded":"2024-07-17T06:55:59.045182+0000","last_fullsized":"2024-07-17T06:55:59.045182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046375+0000","last_change":"2024-07-17T06:55:58.056445+0000","last_active":"2024-07-17T06:55:59.046375+0000","last_peered":"2024-07-17T06:55:59.046375+0000","last_clean":"2024-07-17T06:55:59.046375+0000","last_became_active":"2024-07-17T06:55:58.056142+0000","last_became_peered":"2024-07-17T06:55:58.056142+0000","last_unstale":"2024-07-17T06:55:59.046375+0000","last_undegraded":"2024-07-17T06:55:59.046375+0000","last_fullsized":"2024-07-17T06:55:59.046375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:42.902826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043098+0000","last_change":"2024-07-17T06:55:58.062202+0000","last_active":"2024-07-17T06:55:59.043098+0000","last_peered":"2024-07-17T06:55:59.043098+0000","last_clean":"2024-07-17T06:55:59.043098+0000","last_became_active":"2024-07-17T06:55:58.060959+0000","last_became_peered":"2024-07-17T06:55:58.060959+0000","last_unstale":"2024-07-17T06:55:59.043098+0000","last_undegraded":"2024-07-17T06:55:59.043098+0000","last_fullsized":"2024-07-17T06:55:59.043098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:19.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043379+0000","last_change":"2024-07-17T06:55:58.053301+0000","last_active":"2024-07-17T06:55:59.043379+0000","last_peered":"2024-07-17T06:55:59.043379+0000","last_clean":"2024-07-17T06:55:59.043379+0000","last_became_active":"2024-07-17T06:55:58.052826+0000","last_became_peered":"2024-07-17T06:55:58.052826+0000","last_unstale":"2024-07-17T06:55:59.043379+0000","last_undegraded":"2024-07-17T06:55:59.043379+0000","last_fullsized":"2024-07-17T06:55:59.043379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:29.604009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656526+0000","last_change":"2024-07-17T06:55:58.048345+0000","last_active":"2024-07-17T06:55:59.656526+0000","last_peered":"2024-07-17T06:55:59.656526+0000","last_clean":"2024-07-17T06:55:59.656526+0000","last_became_active":"2024-07-17T06:55:58.047431+0000","last_became_peered":"2024-07-17T06:55:58.047431+0000","last_unstale":"2024-07-17T06:55:59.656526+0000","last_undegraded":"2024-07-17T06:55:59.656526+0000","last_fullsized":"2024-07-17T06:55:59.656526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:48:47.315173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655700+0000","last_change":"2024-07-17T06:55:58.031898+0000","last_active":"2024-07-17T06:55:59.655700+0000","last_peered":"2024-07-17T06:55:59.655700+0000","last_clean":"2024-07-17T06:55:59.655700+0000","last_became_active":"2024-07-17T06:55:58.031564+0000","last_became_peered":"2024-07-17T06:55:58.031564+0000","last_unstale":"2024-07-17T06:55:59.655700+0000","last_undegraded":"2024-07-17T06:55:59.655700+0000","last_fullsized":"2024-07-17T06:55:59.655700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656710+0000","last_change":"2024-07-17T06:55:58.066942+0000","last_active":"2024-07-17T06:55:59.656710+0000","last_peered":"2024-07-17T06:55:59.656710+0000","last_clean":"2024-07-17T06:55:59.656710+0000","last_became_active":"2024-07-17T06:55:58.066637+0000","last_became_peered":"2024-07-17T06:55:58.066637+0000","last_unstale":"2024-07-17T06:55:59.656710+0000","last_undegraded":"2024-07-17T06:55:59.656710+0000","last_fullsized":"2024-07-17T06:55:59.656710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:57.766247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046800+0000","last_change":"2024-07-17T06:55:57.024271+0000","last_active":"2024-07-17T06:55:59.046800+0000","last_peered":"2024-07-17T06:55:59.046800+0000","last_clean":"2024-07-17T06:55:59.046800+0000","last_became_active":"2024-07-17T06:55:57.022997+0000","last_became_peered":"2024-07-17T06:55:57.022997+0000","last_unstale":"2024-07-17T06:55:59.046800+0000","last_undegraded":"2024-07-17T06:55:59.046800+0000","last_fullsized":"2024-07-17T06:55:59.046800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657147+0000","last_change":"2024-07-17T06:55:58.058629+0000","last_active":"2024-07-17T06:55:59.657147+0000","last_peered":"2024-07-17T06:55:59.657147+0000","last_clean":"2024-07-17T06:55:59.657147+0000","last_became_active":"2024-07-17T06:55:58.058309+0000","last_became_peered":"2024-07-17T06:55:58.058309+0000","last_unstale":"2024-07-17T06:55:59.657147+0000","last_undegraded":"2024-07-17T06:55:59.657147+0000","last_fullsized":"2024-07-17T06:55:59.657147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:27.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660481+0000","last_change":"2024-07-17T06:55:57.023703+0000","last_active":"2024-07-17T06:55:59.660481+0000","last_peered":"2024-07-17T06:55:59.660481+0000","last_clean":"2024-07-17T06:55:59.660481+0000","last_became_active":"2024-07-17T06:55:57.023430+0000","last_became_peered":"2024-07-17T06:55:57.023430+0000","last_unstale":"2024-07-17T06:55:59.660481+0000","last_undegraded":"2024-07-17T06:55:59.660481+0000","last_fullsized":"2024-07-17T06:55:59.660481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045747+0000","last_change":"2024-07-17T06:55:58.054862+0000","last_active":"2024-07-17T06:55:59.045747+0000","last_peered":"2024-07-17T06:55:59.045747+0000","last_clean":"2024-07-17T06:55:59.045747+0000","last_became_active":"2024-07-17T06:55:58.054367+0000","last_became_peered":"2024-07-17T06:55:58.054367+0000","last_unstale":"2024-07-17T06:55:59.045747+0000","last_undegraded":"2024-07-17T06:55:59.045747+0000","last_fullsized":"2024-07-17T06:55:59.045747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:35.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657759+0000","last_change":"2024-07-17T06:55:57.025336+0000","last_active":"2024-07-17T06:55:59.657759+0000","last_peered":"2024-07-17T06:55:59.657759+0000","last_clean":"2024-07-17T06:55:59.657759+0000","last_became_active":"2024-07-17T06:55:57.024773+0000","last_became_peered":"2024-07-17T06:55:57.024773+0000","last_unstale":"2024-07-17T06:55:59.657759+0000","last_undegraded":"2024-07-17T06:55:59.657759+0000","last_fullsized":"2024-07-17T06:55:59.657759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659324+0000","last_change":"2024-07-17T06:55:58.053903+0000","last_active":"2024-07-17T06:55:59.659324+0000","last_peered":"2024-07-17T06:55:59.659324+0000","last_clean":"2024-07-17T06:55:59.659324+0000","last_became_active":"2024-07-17T06:55:58.053559+0000","last_became_peered":"2024-07-17T06:55:58.053559+0000","last_unstale":"2024-07-17T06:55:59.659324+0000","last_undegraded":"2024-07-17T06:55:59.659324+0000","last_fullsized":"2024-07-17T06:55:59.659324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:42.902826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044073+0000","last_change":"2024-07-17T06:55:57.021580+0000","last_active":"2024-07-17T06:55:59.044073+0000","last_peered":"2024-07-17T06:55:59.044073+0000","last_clean":"2024-07-17T06:55:59.044073+0000","last_became_active":"2024-07-17T06:55:57.021284+0000","last_became_peered":"2024-07-17T06:55:57.021284+0000","last_unstale":"2024-07-17T06:55:59.044073+0000","last_undegraded":"2024-07-17T06:55:59.044073+0000","last_fullsized":"2024-07-17T06:55:59.044073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:56.527606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656373+0000","last_change":"2024-07-17T06:55:58.039345+0000","last_active":"2024-07-17T06:55:59.656373+0000","last_peered":"2024-07-17T06:55:59.656373+0000","last_clean":"2024-07-17T06:55:59.656373+0000","last_became_active":"2024-07-17T06:55:58.039091+0000","last_became_peered":"2024-07-17T06:55:58.039091+0000","last_unstale":"2024-07-17T06:55:59.656373+0000","last_undegraded":"2024-07-17T06:55:59.656373+0000","last_fullsized":"2024-07-17T06:55:59.656373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045718+0000","last_change":"2024-07-17T06:55:57.027827+0000","last_active":"2024-07-17T06:55:59.045718+0000","last_peered":"2024-07-17T06:55:59.045718+0000","last_clean":"2024-07-17T06:55:59.045718+0000","last_became_active":"2024-07-17T06:55:57.027549+0000","last_became_peered":"2024-07-17T06:55:57.027549+0000","last_unstale":"2024-07-17T06:55:59.045718+0000","last_undegraded":"2024-07-17T06:55:59.045718+0000","last_fullsized":"2024-07-17T06:55:59.045718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:58.711643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044332+0000","last_change":"2024-07-17T06:55:58.053549+0000","last_active":"2024-07-17T06:55:59.044332+0000","last_peered":"2024-07-17T06:55:59.044332+0000","last_clean":"2024-07-17T06:55:59.044332+0000","last_became_active":"2024-07-17T06:55:58.053086+0000","last_became_peered":"2024-07-17T06:55:58.053086+0000","last_unstale":"2024-07-17T06:55:59.044332+0000","last_undegraded":"2024-07-17T06:55:59.044332+0000","last_fullsized":"2024-07-17T06:55:59.044332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:16.112850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656956+0000","last_change":"2024-07-17T06:55:57.023729+0000","last_active":"2024-07-17T06:55:59.656956+0000","last_peered":"2024-07-17T06:55:59.656956+0000","last_clean":"2024-07-17T06:55:59.656956+0000","last_became_active":"2024-07-17T06:55:57.023443+0000","last_became_peered":"2024-07-17T06:55:57.023443+0000","last_unstale":"2024-07-17T06:55:59.656956+0000","last_undegraded":"2024-07-17T06:55:59.656956+0000","last_fullsized":"2024-07-17T06:55:59.656956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:55.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043715+0000","last_change":"2024-07-17T06:55:58.052096+0000","last_active":"2024-07-17T06:55:59.043715+0000","last_peered":"2024-07-17T06:55:59.043715+0000","last_clean":"2024-07-17T06:55:59.043715+0000","last_became_active":"2024-07-17T06:55:58.050834+0000","last_became_peered":"2024-07-17T06:55:58.050834+0000","last_unstale":"2024-07-17T06:55:59.043715+0000","last_undegraded":"2024-07-17T06:55:59.043715+0000","last_fullsized":"2024-07-17T06:55:59.043715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:32.286503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656020+0000","last_change":"2024-07-17T06:55:57.023183+0000","last_active":"2024-07-17T06:55:59.656020+0000","last_peered":"2024-07-17T06:55:59.656020+0000","last_clean":"2024-07-17T06:55:59.656020+0000","last_became_active":"2024-07-17T06:55:57.022952+0000","last_became_peered":"2024-07-17T06:55:57.022952+0000","last_unstale":"2024-07-17T06:55:59.656020+0000","last_undegraded":"2024-07-17T06:55:59.656020+0000","last_fullsized":"2024-07-17T06:55:59.656020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:09:23.205075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657799+0000","last_change":"2024-07-17T06:55:58.040402+0000","last_active":"2024-07-17T06:55:59.657799+0000","last_peered":"2024-07-17T06:55:59.657799+0000","last_clean":"2024-07-17T06:55:59.657799+0000","last_became_active":"2024-07-17T06:55:58.040131+0000","last_became_peered":"2024-07-17T06:55:58.040131+0000","last_unstale":"2024-07-17T06:55:59.657799+0000","last_undegraded":"2024-07-17T06:55:59.657799+0000","last_fullsized":"2024-07-17T06:55:59.657799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:42.200860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042325+0000","last_change":"2024-07-17T06:55:57.015368+0000","last_active":"2024-07-17T06:55:59.042325+0000","last_peered":"2024-07-17T06:55:59.042325+0000","last_clean":"2024-07-17T06:55:59.042325+0000","last_became_active":"2024-07-17T06:55:57.013915+0000","last_became_peered":"2024-07-17T06:55:57.013915+0000","last_unstale":"2024-07-17T06:55:59.042325+0000","last_undegraded":"2024-07-17T06:55:59.042325+0000","last_fullsized":"2024-07-17T06:55:59.042325+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:04.895014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036930+0000","last_change":"2024-07-17T06:55:58.042878+0000","last_active":"2024-07-17T06:55:59.036930+0000","last_peered":"2024-07-17T06:55:59.036930+0000","last_clean":"2024-07-17T06:55:59.036930+0000","last_became_active":"2024-07-17T06:55:58.042538+0000","last_became_peered":"2024-07-17T06:55:58.042538+0000","last_unstale":"2024-07-17T06:55:59.036930+0000","last_undegraded":"2024-07-17T06:55:59.036930+0000","last_fullsized":"2024-07-17T06:55:59.036930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:09:46.207572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659668+0000","last_change":"2024-07-17T06:55:57.026571+0000","last_active":"2024-07-17T06:55:59.659668+0000","last_peered":"2024-07-17T06:55:59.659668+0000","last_clean":"2024-07-17T06:55:59.659668+0000","last_became_active":"2024-07-17T06:55:57.026249+0000","last_became_peered":"2024-07-17T06:55:57.026249+0000","last_unstale":"2024-07-17T06:55:59.659668+0000","last_undegraded":"2024-07-17T06:55:59.659668+0000","last_fullsized":"2024-07-17T06:55:59.659668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:30.625072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657481+0000","last_change":"2024-07-17T06:55:58.058048+0000","last_active":"2024-07-17T06:55:59.657481+0000","last_peered":"2024-07-17T06:55:59.657481+0000","last_clean":"2024-07-17T06:55:59.657481+0000","last_became_active":"2024-07-17T06:55:58.057750+0000","last_became_peered":"2024-07-17T06:55:58.057750+0000","last_unstale":"2024-07-17T06:55:59.657481+0000","last_undegraded":"2024-07-17T06:55:59.657481+0000","last_fullsized":"2024-07-17T06:55:59.657481+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:27:04.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047425+0000","last_change":"2024-07-17T06:55:57.025366+0000","last_active":"2024-07-17T06:55:59.047425+0000","last_peered":"2024-07-17T06:55:59.047425+0000","last_clean":"2024-07-17T06:55:59.047425+0000","last_became_active":"2024-07-17T06:55:57.024821+0000","last_became_peered":"2024-07-17T06:55:57.024821+0000","last_unstale":"2024-07-17T06:55:59.047425+0000","last_undegraded":"2024-07-17T06:55:59.047425+0000","last_fullsized":"2024-07-17T06:55:59.047425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:48.777690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660626+0000","last_change":"2024-07-17T06:55:58.044118+0000","last_active":"2024-07-17T06:55:59.660626+0000","last_peered":"2024-07-17T06:55:59.660626+0000","last_clean":"2024-07-17T06:55:59.660626+0000","last_became_active":"2024-07-17T06:55:58.043853+0000","last_became_peered":"2024-07-17T06:55:58.043853+0000","last_unstale":"2024-07-17T06:55:59.660626+0000","last_undegraded":"2024-07-17T06:55:59.660626+0000","last_fullsized":"2024-07-17T06:55:59.660626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:52.345032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047514+0000","last_change":"2024-07-17T06:55:57.025608+0000","last_active":"2024-07-17T06:55:59.047514+0000","last_peered":"2024-07-17T06:55:59.047514+0000","last_clean":"2024-07-17T06:55:59.047514+0000","last_became_active":"2024-07-17T06:55:57.024617+0000","last_became_peered":"2024-07-17T06:55:57.024617+0000","last_unstale":"2024-07-17T06:55:59.047514+0000","last_undegraded":"2024-07-17T06:55:59.047514+0000","last_fullsized":"2024-07-17T06:55:59.047514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044747+0000","last_change":"2024-07-17T06:55:58.037543+0000","last_active":"2024-07-17T06:55:59.044747+0000","last_peered":"2024-07-17T06:55:59.044747+0000","last_clean":"2024-07-17T06:55:59.044747+0000","last_became_active":"2024-07-17T06:55:58.033368+0000","last_became_peered":"2024-07-17T06:55:58.033368+0000","last_unstale":"2024-07-17T06:55:59.044747+0000","last_undegraded":"2024-07-17T06:55:59.044747+0000","last_fullsized":"2024-07-17T06:55:59.044747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:26:05.155740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658598+0000","last_change":"2024-07-17T06:55:57.016760+0000","last_active":"2024-07-17T06:55:59.658598+0000","last_peered":"2024-07-17T06:55:59.658598+0000","last_clean":"2024-07-17T06:55:59.658598+0000","last_became_active":"2024-07-17T06:55:57.016398+0000","last_became_peered":"2024-07-17T06:55:57.016398+0000","last_unstale":"2024-07-17T06:55:59.658598+0000","last_undegraded":"2024-07-17T06:55:59.658598+0000","last_fullsized":"2024-07-17T06:55:59.658598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:58.711643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656834+0000","last_change":"2024-07-17T06:55:58.063954+0000","last_active":"2024-07-17T06:55:59.656834+0000","last_peered":"2024-07-17T06:55:59.656834+0000","last_clean":"2024-07-17T06:55:59.656834+0000","last_became_active":"2024-07-17T06:55:58.063630+0000","last_became_peered":"2024-07-17T06:55:58.063630+0000","last_unstale":"2024-07-17T06:55:59.656834+0000","last_undegraded":"2024-07-17T06:55:59.656834+0000","last_fullsized":"2024-07-17T06:55:59.656834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658155+0000","last_change":"2024-07-17T06:55:57.014743+0000","last_active":"2024-07-17T06:55:59.658155+0000","last_peered":"2024-07-17T06:55:59.658155+0000","last_clean":"2024-07-17T06:55:59.658155+0000","last_became_active":"2024-07-17T06:55:57.013321+0000","last_became_peered":"2024-07-17T06:55:57.013321+0000","last_unstale":"2024-07-17T06:55:59.658155+0000","last_undegraded":"2024-07-17T06:55:59.658155+0000","last_fullsized":"2024-07-17T06:55:59.658155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.590920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658277+0000","last_change":"2024-07-17T06:55:58.032791+0000","last_active":"2024-07-17T06:55:59.658277+0000","last_peered":"2024-07-17T06:55:59.658277+0000","last_clean":"2024-07-17T06:55:59.658277+0000","last_became_active":"2024-07-17T06:55:58.032467+0000","last_became_peered":"2024-07-17T06:55:58.032467+0000","last_unstale":"2024-07-17T06:55:59.658277+0000","last_undegraded":"2024-07-17T06:55:59.658277+0000","last_fullsized":"2024-07-17T06:55:59.658277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:42.383602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655477+0000","last_change":"2024-07-17T06:55:57.023971+0000","last_active":"2024-07-17T06:55:59.655477+0000","last_peered":"2024-07-17T06:55:59.655477+0000","last_clean":"2024-07-17T06:55:59.655477+0000","last_became_active":"2024-07-17T06:55:57.023779+0000","last_became_peered":"2024-07-17T06:55:57.023779+0000","last_unstale":"2024-07-17T06:55:59.655477+0000","last_undegraded":"2024-07-17T06:55:59.655477+0000","last_fullsized":"2024-07-17T06:55:59.655477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656802+0000","last_change":"2024-07-17T06:55:58.045964+0000","last_active":"2024-07-17T06:55:59.656802+0000","last_peered":"2024-07-17T06:55:59.656802+0000","last_clean":"2024-07-17T06:55:59.656802+0000","last_became_active":"2024-07-17T06:55:58.045590+0000","last_became_peered":"2024-07-17T06:55:58.045590+0000","last_unstale":"2024-07-17T06:55:59.656802+0000","last_undegraded":"2024-07-17T06:55:59.656802+0000","last_fullsized":"2024-07-17T06:55:59.656802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:45:34.831048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042918+0000","last_change":"2024-07-17T06:55:57.015366+0000","last_active":"2024-07-17T06:55:59.042918+0000","last_peered":"2024-07-17T06:55:59.042918+0000","last_clean":"2024-07-17T06:55:59.042918+0000","last_became_active":"2024-07-17T06:55:57.015091+0000","last_became_peered":"2024-07-17T06:55:57.015091+0000","last_unstale":"2024-07-17T06:55:59.042918+0000","last_undegraded":"2024-07-17T06:55:59.042918+0000","last_fullsized":"2024-07-17T06:55:59.042918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657772+0000","last_change":"2024-07-17T06:55:58.033692+0000","last_active":"2024-07-17T06:55:59.657772+0000","last_peered":"2024-07-17T06:55:59.657772+0000","last_clean":"2024-07-17T06:55:59.657772+0000","last_became_active":"2024-07-17T06:55:58.032538+0000","last_became_peered":"2024-07-17T06:55:58.032538+0000","last_unstale":"2024-07-17T06:55:59.657772+0000","last_undegraded":"2024-07-17T06:55:59.657772+0000","last_fullsized":"2024-07-17T06:55:59.657772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:31.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656219+0000","last_change":"2024-07-17T06:55:57.025095+0000","last_active":"2024-07-17T06:55:59.656219+0000","last_peered":"2024-07-17T06:55:59.656219+0000","last_clean":"2024-07-17T06:55:59.656219+0000","last_became_active":"2024-07-17T06:55:57.023655+0000","last_became_peered":"2024-07-17T06:55:57.023655+0000","last_unstale":"2024-07-17T06:55:59.656219+0000","last_undegraded":"2024-07-17T06:55:59.656219+0000","last_fullsized":"2024-07-17T06:55:59.656219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:28:33.365794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658639+0000","last_change":"2024-07-17T06:55:58.050981+0000","last_active":"2024-07-17T06:55:59.658639+0000","last_peered":"2024-07-17T06:55:59.658639+0000","last_clean":"2024-07-17T06:55:59.658639+0000","last_became_active":"2024-07-17T06:55:58.050714+0000","last_became_peered":"2024-07-17T06:55:58.050714+0000","last_unstale":"2024-07-17T06:55:59.658639+0000","last_undegraded":"2024-07-17T06:55:59.658639+0000","last_fullsized":"2024-07-17T06:55:59.658639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:16.112850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043600+0000","last_change":"2024-07-17T06:55:57.030109+0000","last_active":"2024-07-17T06:55:59.043600+0000","last_peered":"2024-07-17T06:55:59.043600+0000","last_clean":"2024-07-17T06:55:59.043600+0000","last_became_active":"2024-07-17T06:55:57.029434+0000","last_became_peered":"2024-07-17T06:55:57.029434+0000","last_unstale":"2024-07-17T06:55:59.043600+0000","last_undegraded":"2024-07-17T06:55:59.043600+0000","last_fullsized":"2024-07-17T06:55:59.043600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:47.719938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047275+0000","last_change":"2024-07-17T06:55:58.056458+0000","last_active":"2024-07-17T06:55:59.047275+0000","last_peered":"2024-07-17T06:55:59.047275+0000","last_clean":"2024-07-17T06:55:59.047275+0000","last_became_active":"2024-07-17T06:55:58.056080+0000","last_became_peered":"2024-07-17T06:55:58.056080+0000","last_unstale":"2024-07-17T06:55:59.047275+0000","last_undegraded":"2024-07-17T06:55:59.047275+0000","last_fullsized":"2024-07-17T06:55:59.047275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:39.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045886+0000","last_change":"2024-07-17T06:55:57.027880+0000","last_active":"2024-07-17T06:55:59.045886+0000","last_peered":"2024-07-17T06:55:59.045886+0000","last_clean":"2024-07-17T06:55:59.045886+0000","last_became_active":"2024-07-17T06:55:57.027361+0000","last_became_peered":"2024-07-17T06:55:57.027361+0000","last_unstale":"2024-07-17T06:55:59.045886+0000","last_undegraded":"2024-07-17T06:55:59.045886+0000","last_fullsized":"2024-07-17T06:55:59.045886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.693484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045341+0000","last_change":"2024-07-17T06:55:58.034283+0000","last_active":"2024-07-17T06:55:59.045341+0000","last_peered":"2024-07-17T06:55:59.045341+0000","last_clean":"2024-07-17T06:55:59.045341+0000","last_became_active":"2024-07-17T06:55:58.033852+0000","last_became_peered":"2024-07-17T06:55:58.033852+0000","last_unstale":"2024-07-17T06:55:59.045341+0000","last_undegraded":"2024-07-17T06:55:59.045341+0000","last_fullsized":"2024-07-17T06:55:59.045341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:23.626316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036423+0000","last_change":"2024-07-17T06:55:57.018791+0000","last_active":"2024-07-17T06:55:59.036423+0000","last_peered":"2024-07-17T06:55:59.036423+0000","last_clean":"2024-07-17T06:55:59.036423+0000","last_became_active":"2024-07-17T06:55:57.016819+0000","last_became_peered":"2024-07-17T06:55:57.016819+0000","last_unstale":"2024-07-17T06:55:59.036423+0000","last_undegraded":"2024-07-17T06:55:59.036423+0000","last_fullsized":"2024-07-17T06:55:59.036423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:37.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036056+0000","last_change":"2024-07-17T06:55:58.029485+0000","last_active":"2024-07-17T06:55:59.036056+0000","last_peered":"2024-07-17T06:55:59.036056+0000","last_clean":"2024-07-17T06:55:59.036056+0000","last_became_active":"2024-07-17T06:55:58.029204+0000","last_became_peered":"2024-07-17T06:55:58.029204+0000","last_unstale":"2024-07-17T06:55:59.036056+0000","last_undegraded":"2024-07-17T06:55:59.036056+0000","last_fullsized":"2024-07-17T06:55:59.036056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:36.108890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656113+0000","last_change":"2024-07-17T06:55:57.020021+0000","last_active":"2024-07-17T06:55:59.656113+0000","last_peered":"2024-07-17T06:55:59.656113+0000","last_clean":"2024-07-17T06:55:59.656113+0000","last_became_active":"2024-07-17T06:55:57.019599+0000","last_became_peered":"2024-07-17T06:55:57.019599+0000","last_unstale":"2024-07-17T06:55:59.656113+0000","last_undegraded":"2024-07-17T06:55:59.656113+0000","last_fullsized":"2024-07-17T06:55:59.656113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:20.727281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657092+0000","last_change":"2024-07-17T06:55:58.051497+0000","last_active":"2024-07-17T06:55:59.657092+0000","last_peered":"2024-07-17T06:55:59.657092+0000","last_clean":"2024-07-17T06:55:59.657092+0000","last_became_active":"2024-07-17T06:55:58.051167+0000","last_became_peered":"2024-07-17T06:55:58.051167+0000","last_unstale":"2024-07-17T06:55:59.657092+0000","last_undegraded":"2024-07-17T06:55:59.657092+0000","last_fullsized":"2024-07-17T06:55:59.657092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:20.126847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035964+0000","last_change":"2024-07-17T06:55:57.014314+0000","last_active":"2024-07-17T06:55:59.035964+0000","last_peered":"2024-07-17T06:55:59.035964+0000","last_clean":"2024-07-17T06:55:59.035964+0000","last_became_active":"2024-07-17T06:55:57.014009+0000","last_became_peered":"2024-07-17T06:55:57.014009+0000","last_unstale":"2024-07-17T06:55:59.035964+0000","last_undegraded":"2024-07-17T06:55:59.035964+0000","last_fullsized":"2024-07-17T06:55:59.035964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:03.980067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659453+0000","last_change":"2024-07-17T06:55:58.050671+0000","last_active":"2024-07-17T06:55:59.659453+0000","last_peered":"2024-07-17T06:55:59.659453+0000","last_clean":"2024-07-17T06:55:59.659453+0000","last_became_active":"2024-07-17T06:55:58.049737+0000","last_became_peered":"2024-07-17T06:55:58.049737+0000","last_unstale":"2024-07-17T06:55:59.659453+0000","last_undegraded":"2024-07-17T06:55:59.659453+0000","last_fullsized":"2024-07-17T06:55:59.659453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:10.010707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035633+0000","last_change":"2024-07-17T06:55:57.014718+0000","last_active":"2024-07-17T06:55:59.035633+0000","last_peered":"2024-07-17T06:55:59.035633+0000","last_clean":"2024-07-17T06:55:59.035633+0000","last_became_active":"2024-07-17T06:55:57.014397+0000","last_became_peered":"2024-07-17T06:55:57.014397+0000","last_unstale":"2024-07-17T06:55:59.035633+0000","last_undegraded":"2024-07-17T06:55:59.035633+0000","last_fullsized":"2024-07-17T06:55:59.035633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:25.053286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042498+0000","last_change":"2024-07-17T06:55:58.059706+0000","last_active":"2024-07-17T06:55:59.042498+0000","last_peered":"2024-07-17T06:55:59.042498+0000","last_clean":"2024-07-17T06:55:59.042498+0000","last_became_active":"2024-07-17T06:55:58.057842+0000","last_became_peered":"2024-07-17T06:55:58.057842+0000","last_unstale":"2024-07-17T06:55:59.042498+0000","last_undegraded":"2024-07-17T06:55:59.042498+0000","last_fullsized":"2024-07-17T06:55:59.042498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:31.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656493+0000","last_change":"2024-07-17T06:55:57.026804+0000","last_active":"2024-07-17T06:55:59.656493+0000","last_peered":"2024-07-17T06:55:59.656493+0000","last_clean":"2024-07-17T06:55:59.656493+0000","last_became_active":"2024-07-17T06:55:57.024887+0000","last_became_peered":"2024-07-17T06:55:57.024887+0000","last_unstale":"2024-07-17T06:55:59.656493+0000","last_undegraded":"2024-07-17T06:55:59.656493+0000","last_fullsized":"2024-07-17T06:55:59.656493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:20:31.619884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658214+0000","last_change":"2024-07-17T06:55:58.041686+0000","last_active":"2024-07-17T06:55:59.658214+0000","last_peered":"2024-07-17T06:55:59.658214+0000","last_clean":"2024-07-17T06:55:59.658214+0000","last_became_active":"2024-07-17T06:55:58.040003+0000","last_became_peered":"2024-07-17T06:55:58.040003+0000","last_unstale":"2024-07-17T06:55:59.658214+0000","last_undegraded":"2024-07-17T06:55:59.658214+0000","last_fullsized":"2024-07-17T06:55:59.658214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:50:35.078043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035215+0000","last_change":"2024-07-17T06:55:57.015139+0000","last_active":"2024-07-17T06:55:59.035215+0000","last_peered":"2024-07-17T06:55:59.035215+0000","last_clean":"2024-07-17T06:55:59.035215+0000","last_became_active":"2024-07-17T06:55:57.012240+0000","last_became_peered":"2024-07-17T06:55:57.012240+0000","last_unstale":"2024-07-17T06:55:59.035215+0000","last_undegraded":"2024-07-17T06:55:59.035215+0000","last_fullsized":"2024-07-17T06:55:59.035215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:09.873719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655851+0000","last_change":"2024-07-17T06:55:58.059105+0000","last_active":"2024-07-17T06:55:59.655851+0000","last_peered":"2024-07-17T06:55:59.655851+0000","last_clean":"2024-07-17T06:55:59.655851+0000","last_became_active":"2024-07-17T06:55:58.058356+0000","last_became_peered":"2024-07-17T06:55:58.058356+0000","last_unstale":"2024-07-17T06:55:59.655851+0000","last_undegraded":"2024-07-17T06:55:59.655851+0000","last_fullsized":"2024-07-17T06:55:59.655851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044173+0000","last_change":"2024-07-17T06:55:57.029338+0000","last_active":"2024-07-17T06:55:59.044173+0000","last_peered":"2024-07-17T06:55:59.044173+0000","last_clean":"2024-07-17T06:55:59.044173+0000","last_became_active":"2024-07-17T06:55:57.028513+0000","last_became_peered":"2024-07-17T06:55:57.028513+0000","last_unstale":"2024-07-17T06:55:59.044173+0000","last_undegraded":"2024-07-17T06:55:59.044173+0000","last_fullsized":"2024-07-17T06:55:59.044173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.590920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657491+0000","last_change":"2024-07-17T06:55:58.057664+0000","last_active":"2024-07-17T06:55:59.657491+0000","last_peered":"2024-07-17T06:55:59.657491+0000","last_clean":"2024-07-17T06:55:59.657491+0000","last_became_active":"2024-07-17T06:55:58.057223+0000","last_became_peered":"2024-07-17T06:55:58.057223+0000","last_unstale":"2024-07-17T06:55:59.657491+0000","last_undegraded":"2024-07-17T06:55:59.657491+0000","last_fullsized":"2024-07-17T06:55:59.657491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:44.455462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036321+0000","last_change":"2024-07-17T06:55:57.027451+0000","last_active":"2024-07-17T06:55:59.036321+0000","last_peered":"2024-07-17T06:55:59.036321+0000","last_clean":"2024-07-17T06:55:59.036321+0000","last_became_active":"2024-07-17T06:55:57.027142+0000","last_became_peered":"2024-07-17T06:55:57.027142+0000","last_unstale":"2024-07-17T06:55:59.036321+0000","last_undegraded":"2024-07-17T06:55:59.036321+0000","last_fullsized":"2024-07-17T06:55:59.036321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:27:05.080931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658942+0000","last_change":"2024-07-17T06:55:58.041665+0000","last_active":"2024-07-17T06:55:59.658942+0000","last_peered":"2024-07-17T06:55:59.658942+0000","last_clean":"2024-07-17T06:55:59.658942+0000","last_became_active":"2024-07-17T06:55:58.041353+0000","last_became_peered":"2024-07-17T06:55:58.041353+0000","last_unstale":"2024-07-17T06:55:59.658942+0000","last_undegraded":"2024-07-17T06:55:59.658942+0000","last_fullsized":"2024-07-17T06:55:59.658942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:31:51.390519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660361+0000","last_change":"2024-07-17T06:55:57.020427+0000","last_active":"2024-07-17T06:55:59.660361+0000","last_peered":"2024-07-17T06:55:59.660361+0000","last_clean":"2024-07-17T06:55:59.660361+0000","last_became_active":"2024-07-17T06:55:57.020107+0000","last_became_peered":"2024-07-17T06:55:57.020107+0000","last_unstale":"2024-07-17T06:55:59.660361+0000","last_undegraded":"2024-07-17T06:55:59.660361+0000","last_fullsized":"2024-07-17T06:55:59.660361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:09:23.205075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657241+0000","last_change":"2024-07-17T06:55:58.038842+0000","last_active":"2024-07-17T06:55:59.657241+0000","last_peered":"2024-07-17T06:55:59.657241+0000","last_clean":"2024-07-17T06:55:59.657241+0000","last_became_active":"2024-07-17T06:55:58.038214+0000","last_became_peered":"2024-07-17T06:55:58.038214+0000","last_unstale":"2024-07-17T06:55:59.657241+0000","last_undegraded":"2024-07-17T06:55:59.657241+0000","last_fullsized":"2024-07-17T06:55:59.657241+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660159+0000","last_change":"2024-07-17T06:55:57.018030+0000","last_active":"2024-07-17T06:55:59.660159+0000","last_peered":"2024-07-17T06:55:59.660159+0000","last_clean":"2024-07-17T06:55:59.660159+0000","last_became_active":"2024-07-17T06:55:57.017102+0000","last_became_peered":"2024-07-17T06:55:57.017102+0000","last_unstale":"2024-07-17T06:55:59.660159+0000","last_undegraded":"2024-07-17T06:55:59.660159+0000","last_fullsized":"2024-07-17T06:55:59.660159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655990+0000","last_change":"2024-07-17T06:55:58.049117+0000","last_active":"2024-07-17T06:55:59.655990+0000","last_peered":"2024-07-17T06:55:59.655990+0000","last_clean":"2024-07-17T06:55:59.655990+0000","last_became_active":"2024-07-17T06:55:58.048851+0000","last_became_peered":"2024-07-17T06:55:58.048851+0000","last_unstale":"2024-07-17T06:55:59.655990+0000","last_undegraded":"2024-07-17T06:55:59.655990+0000","last_fullsized":"2024-07-17T06:55:59.655990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:10.010707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043510+0000","last_change":"2024-07-17T06:55:57.017786+0000","last_active":"2024-07-17T06:55:59.043510+0000","last_peered":"2024-07-17T06:55:59.043510+0000","last_clean":"2024-07-17T06:55:59.043510+0000","last_became_active":"2024-07-17T06:55:57.017469+0000","last_became_peered":"2024-07-17T06:55:57.017469+0000","last_unstale":"2024-07-17T06:55:59.043510+0000","last_undegraded":"2024-07-17T06:55:59.043510+0000","last_fullsized":"2024-07-17T06:55:59.043510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.041987+0000","last_change":"2024-07-17T06:55:58.034550+0000","last_active":"2024-07-17T06:55:59.041987+0000","last_peered":"2024-07-17T06:55:59.041987+0000","last_clean":"2024-07-17T06:55:59.041987+0000","last_became_active":"2024-07-17T06:55:58.034058+0000","last_became_peered":"2024-07-17T06:55:58.034058+0000","last_unstale":"2024-07-17T06:55:59.041987+0000","last_undegraded":"2024-07-17T06:55:59.041987+0000","last_fullsized":"2024-07-17T06:55:59.041987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:28:34.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.041295+0000","last_change":"2024-07-17T06:55:57.014519+0000","last_active":"2024-07-17T06:55:59.041295+0000","last_peered":"2024-07-17T06:55:59.041295+0000","last_clean":"2024-07-17T06:55:59.041295+0000","last_became_active":"2024-07-17T06:55:57.014241+0000","last_became_peered":"2024-07-17T06:55:57.014241+0000","last_unstale":"2024-07-17T06:55:59.041295+0000","last_undegraded":"2024-07-17T06:55:59.041295+0000","last_fullsized":"2024-07-17T06:55:59.041295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:20.293369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657143+0000","last_change":"2024-07-17T06:55:58.058633+0000","last_active":"2024-07-17T06:55:59.657143+0000","last_peered":"2024-07-17T06:55:59.657143+0000","last_clean":"2024-07-17T06:55:59.657143+0000","last_became_active":"2024-07-17T06:55:58.058282+0000","last_became_peered":"2024-07-17T06:55:58.058282+0000","last_unstale":"2024-07-17T06:55:59.657143+0000","last_undegraded":"2024-07-17T06:55:59.657143+0000","last_fullsized":"2024-07-17T06:55:59.657143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:13.537454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657049+0000","last_change":"2024-07-17T06:55:57.020044+0000","last_active":"2024-07-17T06:55:59.657049+0000","last_peered":"2024-07-17T06:55:59.657049+0000","last_clean":"2024-07-17T06:55:59.657049+0000","last_became_active":"2024-07-17T06:55:57.018728+0000","last_became_peered":"2024-07-17T06:55:57.018728+0000","last_unstale":"2024-07-17T06:55:59.657049+0000","last_undegraded":"2024-07-17T06:55:59.657049+0000","last_fullsized":"2024-07-17T06:55:59.657049+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043144+0000","last_change":"2024-07-17T06:55:58.057996+0000","last_active":"2024-07-17T06:55:59.043144+0000","last_peered":"2024-07-17T06:55:59.043144+0000","last_clean":"2024-07-17T06:55:59.043144+0000","last_became_active":"2024-07-17T06:55:58.056857+0000","last_became_peered":"2024-07-17T06:55:58.056857+0000","last_unstale":"2024-07-17T06:55:59.043144+0000","last_undegraded":"2024-07-17T06:55:59.043144+0000","last_fullsized":"2024-07-17T06:55:59.043144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:42.200860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657063+0000","last_change":"2024-07-17T06:55:57.021175+0000","last_active":"2024-07-17T06:55:59.657063+0000","last_peered":"2024-07-17T06:55:59.657063+0000","last_clean":"2024-07-17T06:55:59.657063+0000","last_became_active":"2024-07-17T06:55:57.020798+0000","last_became_peered":"2024-07-17T06:55:57.020798+0000","last_unstale":"2024-07-17T06:55:59.657063+0000","last_undegraded":"2024-07-17T06:55:59.657063+0000","last_fullsized":"2024-07-17T06:55:59.657063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:44.639430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":141,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656575+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:55:59.656575+0000","last_peered":"2024-07-17T06:55:59.656575+0000","last_clean":"2024-07-17T06:55:59.656575+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:55:59.656575+0000","last_undegraded":"2024-07-17T06:55:59.656575+0000","last_fullsized":"2024-07-17T06:55:59.656575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":192265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":191795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1040384,"data_stored":782085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":783733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":192287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":191753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:56:07.574 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T06:56:07.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.795+0000 7fa9e2e00640 1 -- 172.21.15.44:0/3306528831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0cf540 msgr2=0x7fa9dc0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.795+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/3306528831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0cf540 0x7fa9dc0b3870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0cfdb0 tx=0x7fa9d002a560 comp rx=0 tx=0).stop 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.795+0000 7fa9e2e00640 1 -- 172.21.15.44:0/3306528831 shutdown_connections 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/3306528831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0b3db0 0x7fa9dc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/3306528831 >> v2:172.21.15.175:3300/0 conn(0x7fa9dc0cf920 0x7fa9dc0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/3306528831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0cf540 0x7fa9dc0b3870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 -- 172.21.15.44:0/3306528831 >> 172.21.15.44:0/3306528831 conn(0x7fa9dc0b0e80 msgr2=0x7fa9dc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 -- 172.21.15.44:0/3306528831 shutdown_connections 2024-07-17T06:56:07.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.796+0000 7fa9e2e00640 1 -- 172.21.15.44:0/3306528831 wait complete. 2024-07-17T06:56:07.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.797+0000 7fa9e2e00640 1 Processor -- start 2024-07-17T06:56:07.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.798+0000 7fa9e2e00640 1 -- start start 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.798+0000 7fa9e2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.798+0000 7fa9e2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0cf540 0x7fa9dc072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.798+0000 7fa9e2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0cf920 0x7fa9dc15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0ae6f0 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0ae570 con 0x7fa9dc0cf920 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9dc0ae870 con 0x7fa9dc0cf540 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43352/0 (socket says 172.21.15.44:43352) 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 -- 172.21.15.44:0/394516220 learned_addr learned my addr 172.21.15.44:0/394516220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1000640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0cf540 0x7fa9dc072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:07.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e2400640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0cf920 0x7fa9dc15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 -- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0cf540 msgr2=0x7fa9dc072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0cf540 0x7fa9dc072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 -- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0cf920 msgr2=0x7fa9dc15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0cf920 0x7fa9dc15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.799+0000 7fa9e1a00640 1 -- 172.21.15.44:0/394516220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc156130 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9e1a00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000c00 tx=0x7fa9d0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d003f070 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d00047a0 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc156410 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0035030 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.800+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc15bad0 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.801+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc0b3870 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.803+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7fa9d0042070 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.804+0000 7fa9d9600640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 0x7fa9bc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:07.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.804+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa9d00b92d0 con 0x7fa9dc0b3db0 2024-07-17T06:56:07.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.804+0000 7fa9e1000640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 0x7fa9bc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.805+0000 7fa9e1000640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 0x7fa9bc079070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc1f5f10 tx=0x7fa9c800a4c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:07.808+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa9d0082600 con 0x7fa9dc0b3db0 2024-07-17T06:56:08.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.012+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9dc13f530 con 0x7fa9bc076bb0 2024-07-17T06:56:08.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.026+0000 7fa9d9600640 1 -- 172.21.15.44:0/394516220 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251912 (secure 0 0 0) 0x7fa9dc13f530 con 0x7fa9bc076bb0 2024-07-17T06:56:08.028 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T06:56:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 msgr2=0x7fa9bc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 0x7fa9bc079070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc1f5f10 tx=0x7fa9c800a4c0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 msgr2=0x7fa9dc1f12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000c00 tx=0x7fa9d0003910 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 shutdown_connections 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9dc0cf920 0x7fa9dc15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fa9bc076bb0 0x7fa9bc079070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9dc0cf540 0x7fa9dc072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 --2- 172.21.15.44:0/394516220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9dc0b3db0 0x7fa9dc1f12f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.033+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 >> 172.21.15.44:0/394516220 conn(0x7fa9dc0b0e80 msgr2=0x7fa9dc09ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.034+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 shutdown_connections 2024-07-17T06:56:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.034+0000 7fa9e2e00640 1 -- 172.21.15.44:0/394516220 wait complete. 2024-07-17T06:56:08.054 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-07-17T06:56:07.740720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220868,"kb_used_data":4680,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753852,"statfs":{"total":773094113280,"available":772867944448,"internally_reserved":0,"allocated":4792320,"data_stored":2719020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.727189"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659819+0000","last_change":"2024-07-17T06:55:58.045274+0000","last_active":"2024-07-17T06:55:59.659819+0000","last_peered":"2024-07-17T06:55:59.659819+0000","last_clean":"2024-07-17T06:55:59.659819+0000","last_became_active":"2024-07-17T06:55:58.044983+0000","last_became_peered":"2024-07-17T06:55:58.044983+0000","last_unstale":"2024-07-17T06:55:59.659819+0000","last_undegraded":"2024-07-17T06:55:59.659819+0000","last_fullsized":"2024-07-17T06:55:59.659819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:56.755380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658713+0000","last_change":"2024-07-17T06:55:58.035807+0000","last_active":"2024-07-17T06:55:59.658713+0000","last_peered":"2024-07-17T06:55:59.658713+0000","last_clean":"2024-07-17T06:55:59.658713+0000","last_became_active":"2024-07-17T06:55:58.035523+0000","last_became_peered":"2024-07-17T06:55:58.035523+0000","last_unstale":"2024-07-17T06:55:59.658713+0000","last_undegraded":"2024-07-17T06:55:59.658713+0000","last_fullsized":"2024-07-17T06:55:59.658713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:29.515079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660443+0000","last_change":"2024-07-17T06:55:58.063675+0000","last_active":"2024-07-17T06:55:59.660443+0000","last_peered":"2024-07-17T06:55:59.660443+0000","last_clean":"2024-07-17T06:55:59.660443+0000","last_became_active":"2024-07-17T06:55:58.063343+0000","last_became_peered":"2024-07-17T06:55:58.063343+0000","last_unstale":"2024-07-17T06:55:59.660443+0000","last_undegraded":"2024-07-17T06:55:59.660443+0000","last_fullsized":"2024-07-17T06:55:59.660443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:55.758472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036936+0000","last_change":"2024-07-17T06:55:58.043654+0000","last_active":"2024-07-17T06:55:59.036936+0000","last_peered":"2024-07-17T06:55:59.036936+0000","last_clean":"2024-07-17T06:55:59.036936+0000","last_became_active":"2024-07-17T06:55:58.042825+0000","last_became_peered":"2024-07-17T06:55:58.042825+0000","last_unstale":"2024-07-17T06:55:59.036936+0000","last_undegraded":"2024-07-17T06:55:59.036936+0000","last_fullsized":"2024-07-17T06:55:59.036936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:03.993952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035998+0000","last_change":"2024-07-17T06:55:58.040071+0000","last_active":"2024-07-17T06:55:59.035998+0000","last_peered":"2024-07-17T06:55:59.035998+0000","last_clean":"2024-07-17T06:55:59.035998+0000","last_became_active":"2024-07-17T06:55:58.039796+0000","last_became_peered":"2024-07-17T06:55:58.039796+0000","last_unstale":"2024-07-17T06:55:59.035998+0000","last_undegraded":"2024-07-17T06:55:59.035998+0000","last_fullsized":"2024-07-17T06:55:59.035998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:00.706573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655654+0000","last_change":"2024-07-17T06:55:58.033469+0000","last_active":"2024-07-17T06:55:59.655654+0000","last_peered":"2024-07-17T06:55:59.655654+0000","last_clean":"2024-07-17T06:55:59.655654+0000","last_became_active":"2024-07-17T06:55:58.032268+0000","last_became_peered":"2024-07-17T06:55:58.032268+0000","last_unstale":"2024-07-17T06:55:59.655654+0000","last_undegraded":"2024-07-17T06:55:59.655654+0000","last_fullsized":"2024-07-17T06:55:59.655654+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043885+0000","last_change":"2024-07-17T06:55:58.057539+0000","last_active":"2024-07-17T06:55:59.043885+0000","last_peered":"2024-07-17T06:55:59.043885+0000","last_clean":"2024-07-17T06:55:59.043885+0000","last_became_active":"2024-07-17T06:55:58.057288+0000","last_became_peered":"2024-07-17T06:55:58.057288+0000","last_unstale":"2024-07-17T06:55:59.043885+0000","last_undegraded":"2024-07-17T06:55:59.043885+0000","last_fullsized":"2024-07-17T06:55:59.043885+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:50:35.078043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657003+0000","last_change":"2024-07-17T06:55:58.060879+0000","last_active":"2024-07-17T06:55:59.657003+0000","last_peered":"2024-07-17T06:55:59.657003+0000","last_clean":"2024-07-17T06:55:59.657003+0000","last_became_active":"2024-07-17T06:55:58.060658+0000","last_became_peered":"2024-07-17T06:55:58.060658+0000","last_unstale":"2024-07-17T06:55:59.657003+0000","last_undegraded":"2024-07-17T06:55:59.657003+0000","last_fullsized":"2024-07-17T06:55:59.657003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:10:07.906199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.661028+0000","last_change":"2024-07-17T06:55:58.063719+0000","last_active":"2024-07-17T06:55:59.661028+0000","last_peered":"2024-07-17T06:55:59.661028+0000","last_clean":"2024-07-17T06:55:59.661028+0000","last_became_active":"2024-07-17T06:55:58.063435+0000","last_became_peered":"2024-07-17T06:55:58.063435+0000","last_unstale":"2024-07-17T06:55:59.661028+0000","last_undegraded":"2024-07-17T06:55:59.661028+0000","last_fullsized":"2024-07-17T06:55:59.661028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:18.810801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047019+0000","last_change":"2024-07-17T06:55:58.066563+0000","last_active":"2024-07-17T06:55:59.047019+0000","last_peered":"2024-07-17T06:55:59.047019+0000","last_clean":"2024-07-17T06:55:59.047019+0000","last_became_active":"2024-07-17T06:55:58.066294+0000","last_became_peered":"2024-07-17T06:55:58.066294+0000","last_unstale":"2024-07-17T06:55:59.047019+0000","last_undegraded":"2024-07-17T06:55:59.047019+0000","last_fullsized":"2024-07-17T06:55:59.047019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:56.755380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047255+0000","last_change":"2024-07-17T06:55:58.053366+0000","last_active":"2024-07-17T06:55:59.047255+0000","last_peered":"2024-07-17T06:55:59.047255+0000","last_clean":"2024-07-17T06:55:59.047255+0000","last_became_active":"2024-07-17T06:55:58.053049+0000","last_became_peered":"2024-07-17T06:55:58.053049+0000","last_unstale":"2024-07-17T06:55:59.047255+0000","last_undegraded":"2024-07-17T06:55:59.047255+0000","last_fullsized":"2024-07-17T06:55:59.047255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:00:10.987843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660422+0000","last_change":"2024-07-17T06:55:58.034823+0000","last_active":"2024-07-17T06:55:59.660422+0000","last_peered":"2024-07-17T06:55:59.660422+0000","last_clean":"2024-07-17T06:55:59.660422+0000","last_became_active":"2024-07-17T06:55:58.034497+0000","last_became_peered":"2024-07-17T06:55:58.034497+0000","last_unstale":"2024-07-17T06:55:59.660422+0000","last_undegraded":"2024-07-17T06:55:59.660422+0000","last_fullsized":"2024-07-17T06:55:59.660422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:27.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045419+0000","last_change":"2024-07-17T06:55:58.050205+0000","last_active":"2024-07-17T06:55:59.045419+0000","last_peered":"2024-07-17T06:55:59.045419+0000","last_clean":"2024-07-17T06:55:59.045419+0000","last_became_active":"2024-07-17T06:55:58.049877+0000","last_became_peered":"2024-07-17T06:55:58.049877+0000","last_unstale":"2024-07-17T06:55:59.045419+0000","last_undegraded":"2024-07-17T06:55:59.045419+0000","last_fullsized":"2024-07-17T06:55:59.045419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:29.515079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042997+0000","last_change":"2024-07-17T06:55:58.055343+0000","last_active":"2024-07-17T06:55:59.042997+0000","last_peered":"2024-07-17T06:55:59.042997+0000","last_clean":"2024-07-17T06:55:59.042997+0000","last_became_active":"2024-07-17T06:55:58.054518+0000","last_became_peered":"2024-07-17T06:55:58.054518+0000","last_unstale":"2024-07-17T06:55:59.042997+0000","last_undegraded":"2024-07-17T06:55:59.042997+0000","last_fullsized":"2024-07-17T06:55:59.042997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:54:56.542624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044115+0000","last_change":"2024-07-17T06:55:58.058751+0000","last_active":"2024-07-17T06:55:59.044115+0000","last_peered":"2024-07-17T06:55:59.044115+0000","last_clean":"2024-07-17T06:55:59.044115+0000","last_became_active":"2024-07-17T06:55:58.058406+0000","last_became_peered":"2024-07-17T06:55:58.058406+0000","last_unstale":"2024-07-17T06:55:59.044115+0000","last_undegraded":"2024-07-17T06:55:59.044115+0000","last_fullsized":"2024-07-17T06:55:59.044115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:42.383602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656349+0000","last_change":"2024-07-17T06:55:58.063260+0000","last_active":"2024-07-17T06:55:59.656349+0000","last_peered":"2024-07-17T06:55:59.656349+0000","last_clean":"2024-07-17T06:55:59.656349+0000","last_became_active":"2024-07-17T06:55:58.063013+0000","last_became_peered":"2024-07-17T06:55:58.063013+0000","last_unstale":"2024-07-17T06:55:59.656349+0000","last_undegraded":"2024-07-17T06:55:59.656349+0000","last_fullsized":"2024-07-17T06:55:59.656349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:25.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657077+0000","last_change":"2024-07-17T06:55:58.060726+0000","last_active":"2024-07-17T06:55:59.657077+0000","last_peered":"2024-07-17T06:55:59.657077+0000","last_clean":"2024-07-17T06:55:59.657077+0000","last_became_active":"2024-07-17T06:55:58.060398+0000","last_became_peered":"2024-07-17T06:55:58.060398+0000","last_unstale":"2024-07-17T06:55:59.657077+0000","last_undegraded":"2024-07-17T06:55:59.657077+0000","last_fullsized":"2024-07-17T06:55:59.657077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:44.785341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046069+0000","last_change":"2024-07-17T06:55:58.058568+0000","last_active":"2024-07-17T06:55:59.046069+0000","last_peered":"2024-07-17T06:55:59.046069+0000","last_clean":"2024-07-17T06:55:59.046069+0000","last_became_active":"2024-07-17T06:55:58.058044+0000","last_became_peered":"2024-07-17T06:55:58.058044+0000","last_unstale":"2024-07-17T06:55:59.046069+0000","last_undegraded":"2024-07-17T06:55:59.046069+0000","last_fullsized":"2024-07-17T06:55:59.046069+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036288+0000","last_change":"2024-07-17T06:55:58.032879+0000","last_active":"2024-07-17T06:55:59.036288+0000","last_peered":"2024-07-17T06:55:59.036288+0000","last_clean":"2024-07-17T06:55:59.036288+0000","last_became_active":"2024-07-17T06:55:58.032598+0000","last_became_peered":"2024-07-17T06:55:58.032598+0000","last_unstale":"2024-07-17T06:55:59.036288+0000","last_undegraded":"2024-07-17T06:55:59.036288+0000","last_fullsized":"2024-07-17T06:55:59.036288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:34.827013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044726+0000","last_change":"2024-07-17T06:55:58.051885+0000","last_active":"2024-07-17T06:55:59.044726+0000","last_peered":"2024-07-17T06:55:59.044726+0000","last_clean":"2024-07-17T06:55:59.044726+0000","last_became_active":"2024-07-17T06:55:58.051588+0000","last_became_peered":"2024-07-17T06:55:58.051588+0000","last_unstale":"2024-07-17T06:55:59.044726+0000","last_undegraded":"2024-07-17T06:55:59.044726+0000","last_fullsized":"2024-07-17T06:55:59.044726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:31.638160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045597+0000","last_change":"2024-07-17T06:55:58.057046+0000","last_active":"2024-07-17T06:55:59.045597+0000","last_peered":"2024-07-17T06:55:59.045597+0000","last_clean":"2024-07-17T06:55:59.045597+0000","last_became_active":"2024-07-17T06:55:58.055803+0000","last_became_peered":"2024-07-17T06:55:58.055803+0000","last_unstale":"2024-07-17T06:55:59.045597+0000","last_undegraded":"2024-07-17T06:55:59.045597+0000","last_fullsized":"2024-07-17T06:55:59.045597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:31:51.390519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042568+0000","last_change":"2024-07-17T06:55:58.033523+0000","last_active":"2024-07-17T06:55:59.042568+0000","last_peered":"2024-07-17T06:55:59.042568+0000","last_clean":"2024-07-17T06:55:59.042568+0000","last_became_active":"2024-07-17T06:55:58.032980+0000","last_became_peered":"2024-07-17T06:55:58.032980+0000","last_unstale":"2024-07-17T06:55:59.042568+0000","last_undegraded":"2024-07-17T06:55:59.042568+0000","last_fullsized":"2024-07-17T06:55:59.042568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:24.777827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656328+0000","last_change":"2024-07-17T06:55:58.045860+0000","last_active":"2024-07-17T06:55:59.656328+0000","last_peered":"2024-07-17T06:55:59.656328+0000","last_clean":"2024-07-17T06:55:59.656328+0000","last_became_active":"2024-07-17T06:55:58.045527+0000","last_became_peered":"2024-07-17T06:55:58.045527+0000","last_unstale":"2024-07-17T06:55:59.656328+0000","last_undegraded":"2024-07-17T06:55:59.656328+0000","last_fullsized":"2024-07-17T06:55:59.656328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:16.686880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035342+0000","last_change":"2024-07-17T06:55:58.040719+0000","last_active":"2024-07-17T06:55:59.035342+0000","last_peered":"2024-07-17T06:55:59.035342+0000","last_clean":"2024-07-17T06:55:59.035342+0000","last_became_active":"2024-07-17T06:55:58.040212+0000","last_became_peered":"2024-07-17T06:55:58.040212+0000","last_unstale":"2024-07-17T06:55:59.035342+0000","last_undegraded":"2024-07-17T06:55:59.035342+0000","last_fullsized":"2024-07-17T06:55:59.035342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:57.540694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046697+0000","last_change":"2024-07-17T06:55:58.059600+0000","last_active":"2024-07-17T06:55:59.046697+0000","last_peered":"2024-07-17T06:55:59.046697+0000","last_clean":"2024-07-17T06:55:59.046697+0000","last_became_active":"2024-07-17T06:55:58.053612+0000","last_became_peered":"2024-07-17T06:55:58.053612+0000","last_unstale":"2024-07-17T06:55:59.046697+0000","last_undegraded":"2024-07-17T06:55:59.046697+0000","last_fullsized":"2024-07-17T06:55:59.046697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658271+0000","last_change":"2024-07-17T06:55:58.053299+0000","last_active":"2024-07-17T06:55:59.658271+0000","last_peered":"2024-07-17T06:55:59.658271+0000","last_clean":"2024-07-17T06:55:59.658271+0000","last_became_active":"2024-07-17T06:55:58.053020+0000","last_became_peered":"2024-07-17T06:55:58.053020+0000","last_unstale":"2024-07-17T06:55:59.658271+0000","last_undegraded":"2024-07-17T06:55:59.658271+0000","last_fullsized":"2024-07-17T06:55:59.658271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:55.758472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045182+0000","last_change":"2024-07-17T06:55:58.054772+0000","last_active":"2024-07-17T06:55:59.045182+0000","last_peered":"2024-07-17T06:55:59.045182+0000","last_clean":"2024-07-17T06:55:59.045182+0000","last_became_active":"2024-07-17T06:55:58.054003+0000","last_became_peered":"2024-07-17T06:55:58.054003+0000","last_unstale":"2024-07-17T06:55:59.045182+0000","last_undegraded":"2024-07-17T06:55:59.045182+0000","last_fullsized":"2024-07-17T06:55:59.045182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046375+0000","last_change":"2024-07-17T06:55:58.056445+0000","last_active":"2024-07-17T06:55:59.046375+0000","last_peered":"2024-07-17T06:55:59.046375+0000","last_clean":"2024-07-17T06:55:59.046375+0000","last_became_active":"2024-07-17T06:55:58.056142+0000","last_became_peered":"2024-07-17T06:55:58.056142+0000","last_unstale":"2024-07-17T06:55:59.046375+0000","last_undegraded":"2024-07-17T06:55:59.046375+0000","last_fullsized":"2024-07-17T06:55:59.046375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:42.902826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043098+0000","last_change":"2024-07-17T06:55:58.062202+0000","last_active":"2024-07-17T06:55:59.043098+0000","last_peered":"2024-07-17T06:55:59.043098+0000","last_clean":"2024-07-17T06:55:59.043098+0000","last_became_active":"2024-07-17T06:55:58.060959+0000","last_became_peered":"2024-07-17T06:55:58.060959+0000","last_unstale":"2024-07-17T06:55:59.043098+0000","last_undegraded":"2024-07-17T06:55:59.043098+0000","last_fullsized":"2024-07-17T06:55:59.043098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:19.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043379+0000","last_change":"2024-07-17T06:55:58.053301+0000","last_active":"2024-07-17T06:55:59.043379+0000","last_peered":"2024-07-17T06:55:59.043379+0000","last_clean":"2024-07-17T06:55:59.043379+0000","last_became_active":"2024-07-17T06:55:58.052826+0000","last_became_peered":"2024-07-17T06:55:58.052826+0000","last_unstale":"2024-07-17T06:55:59.043379+0000","last_undegraded":"2024-07-17T06:55:59.043379+0000","last_fullsized":"2024-07-17T06:55:59.043379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:29.604009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656526+0000","last_change":"2024-07-17T06:55:58.048345+0000","last_active":"2024-07-17T06:55:59.656526+0000","last_peered":"2024-07-17T06:55:59.656526+0000","last_clean":"2024-07-17T06:55:59.656526+0000","last_became_active":"2024-07-17T06:55:58.047431+0000","last_became_peered":"2024-07-17T06:55:58.047431+0000","last_unstale":"2024-07-17T06:55:59.656526+0000","last_undegraded":"2024-07-17T06:55:59.656526+0000","last_fullsized":"2024-07-17T06:55:59.656526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:48:47.315173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655700+0000","last_change":"2024-07-17T06:55:58.031898+0000","last_active":"2024-07-17T06:55:59.655700+0000","last_peered":"2024-07-17T06:55:59.655700+0000","last_clean":"2024-07-17T06:55:59.655700+0000","last_became_active":"2024-07-17T06:55:58.031564+0000","last_became_peered":"2024-07-17T06:55:58.031564+0000","last_unstale":"2024-07-17T06:55:59.655700+0000","last_undegraded":"2024-07-17T06:55:59.655700+0000","last_fullsized":"2024-07-17T06:55:59.655700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656710+0000","last_change":"2024-07-17T06:55:58.066942+0000","last_active":"2024-07-17T06:55:59.656710+0000","last_peered":"2024-07-17T06:55:59.656710+0000","last_clean":"2024-07-17T06:55:59.656710+0000","last_became_active":"2024-07-17T06:55:58.066637+0000","last_became_peered":"2024-07-17T06:55:58.066637+0000","last_unstale":"2024-07-17T06:55:59.656710+0000","last_undegraded":"2024-07-17T06:55:59.656710+0000","last_fullsized":"2024-07-17T06:55:59.656710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:57.766247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.046800+0000","last_change":"2024-07-17T06:55:57.024271+0000","last_active":"2024-07-17T06:55:59.046800+0000","last_peered":"2024-07-17T06:55:59.046800+0000","last_clean":"2024-07-17T06:55:59.046800+0000","last_became_active":"2024-07-17T06:55:57.022997+0000","last_became_peered":"2024-07-17T06:55:57.022997+0000","last_unstale":"2024-07-17T06:55:59.046800+0000","last_undegraded":"2024-07-17T06:55:59.046800+0000","last_fullsized":"2024-07-17T06:55:59.046800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657147+0000","last_change":"2024-07-17T06:55:58.058629+0000","last_active":"2024-07-17T06:55:59.657147+0000","last_peered":"2024-07-17T06:55:59.657147+0000","last_clean":"2024-07-17T06:55:59.657147+0000","last_became_active":"2024-07-17T06:55:58.058309+0000","last_became_peered":"2024-07-17T06:55:58.058309+0000","last_unstale":"2024-07-17T06:55:59.657147+0000","last_undegraded":"2024-07-17T06:55:59.657147+0000","last_fullsized":"2024-07-17T06:55:59.657147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:27.020997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660481+0000","last_change":"2024-07-17T06:55:57.023703+0000","last_active":"2024-07-17T06:55:59.660481+0000","last_peered":"2024-07-17T06:55:59.660481+0000","last_clean":"2024-07-17T06:55:59.660481+0000","last_became_active":"2024-07-17T06:55:57.023430+0000","last_became_peered":"2024-07-17T06:55:57.023430+0000","last_unstale":"2024-07-17T06:55:59.660481+0000","last_undegraded":"2024-07-17T06:55:59.660481+0000","last_fullsized":"2024-07-17T06:55:59.660481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045747+0000","last_change":"2024-07-17T06:55:58.054862+0000","last_active":"2024-07-17T06:55:59.045747+0000","last_peered":"2024-07-17T06:55:59.045747+0000","last_clean":"2024-07-17T06:55:59.045747+0000","last_became_active":"2024-07-17T06:55:58.054367+0000","last_became_peered":"2024-07-17T06:55:58.054367+0000","last_unstale":"2024-07-17T06:55:59.045747+0000","last_undegraded":"2024-07-17T06:55:59.045747+0000","last_fullsized":"2024-07-17T06:55:59.045747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:35.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657759+0000","last_change":"2024-07-17T06:55:57.025336+0000","last_active":"2024-07-17T06:55:59.657759+0000","last_peered":"2024-07-17T06:55:59.657759+0000","last_clean":"2024-07-17T06:55:59.657759+0000","last_became_active":"2024-07-17T06:55:57.024773+0000","last_became_peered":"2024-07-17T06:55:57.024773+0000","last_unstale":"2024-07-17T06:55:59.657759+0000","last_undegraded":"2024-07-17T06:55:59.657759+0000","last_fullsized":"2024-07-17T06:55:59.657759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659324+0000","last_change":"2024-07-17T06:55:58.053903+0000","last_active":"2024-07-17T06:55:59.659324+0000","last_peered":"2024-07-17T06:55:59.659324+0000","last_clean":"2024-07-17T06:55:59.659324+0000","last_became_active":"2024-07-17T06:55:58.053559+0000","last_became_peered":"2024-07-17T06:55:58.053559+0000","last_unstale":"2024-07-17T06:55:59.659324+0000","last_undegraded":"2024-07-17T06:55:59.659324+0000","last_fullsized":"2024-07-17T06:55:59.659324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:42.902826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044073+0000","last_change":"2024-07-17T06:55:57.021580+0000","last_active":"2024-07-17T06:55:59.044073+0000","last_peered":"2024-07-17T06:55:59.044073+0000","last_clean":"2024-07-17T06:55:59.044073+0000","last_became_active":"2024-07-17T06:55:57.021284+0000","last_became_peered":"2024-07-17T06:55:57.021284+0000","last_unstale":"2024-07-17T06:55:59.044073+0000","last_undegraded":"2024-07-17T06:55:59.044073+0000","last_fullsized":"2024-07-17T06:55:59.044073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:56.527606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656373+0000","last_change":"2024-07-17T06:55:58.039345+0000","last_active":"2024-07-17T06:55:59.656373+0000","last_peered":"2024-07-17T06:55:59.656373+0000","last_clean":"2024-07-17T06:55:59.656373+0000","last_became_active":"2024-07-17T06:55:58.039091+0000","last_became_peered":"2024-07-17T06:55:58.039091+0000","last_unstale":"2024-07-17T06:55:59.656373+0000","last_undegraded":"2024-07-17T06:55:59.656373+0000","last_fullsized":"2024-07-17T06:55:59.656373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045718+0000","last_change":"2024-07-17T06:55:57.027827+0000","last_active":"2024-07-17T06:55:59.045718+0000","last_peered":"2024-07-17T06:55:59.045718+0000","last_clean":"2024-07-17T06:55:59.045718+0000","last_became_active":"2024-07-17T06:55:57.027549+0000","last_became_peered":"2024-07-17T06:55:57.027549+0000","last_unstale":"2024-07-17T06:55:59.045718+0000","last_undegraded":"2024-07-17T06:55:59.045718+0000","last_fullsized":"2024-07-17T06:55:59.045718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:58.711643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044332+0000","last_change":"2024-07-17T06:55:58.053549+0000","last_active":"2024-07-17T06:55:59.044332+0000","last_peered":"2024-07-17T06:55:59.044332+0000","last_clean":"2024-07-17T06:55:59.044332+0000","last_became_active":"2024-07-17T06:55:58.053086+0000","last_became_peered":"2024-07-17T06:55:58.053086+0000","last_unstale":"2024-07-17T06:55:59.044332+0000","last_undegraded":"2024-07-17T06:55:59.044332+0000","last_fullsized":"2024-07-17T06:55:59.044332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:16.112850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656956+0000","last_change":"2024-07-17T06:55:57.023729+0000","last_active":"2024-07-17T06:55:59.656956+0000","last_peered":"2024-07-17T06:55:59.656956+0000","last_clean":"2024-07-17T06:55:59.656956+0000","last_became_active":"2024-07-17T06:55:57.023443+0000","last_became_peered":"2024-07-17T06:55:57.023443+0000","last_unstale":"2024-07-17T06:55:59.656956+0000","last_undegraded":"2024-07-17T06:55:59.656956+0000","last_fullsized":"2024-07-17T06:55:59.656956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:55.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043715+0000","last_change":"2024-07-17T06:55:58.052096+0000","last_active":"2024-07-17T06:55:59.043715+0000","last_peered":"2024-07-17T06:55:59.043715+0000","last_clean":"2024-07-17T06:55:59.043715+0000","last_became_active":"2024-07-17T06:55:58.050834+0000","last_became_peered":"2024-07-17T06:55:58.050834+0000","last_unstale":"2024-07-17T06:55:59.043715+0000","last_undegraded":"2024-07-17T06:55:59.043715+0000","last_fullsized":"2024-07-17T06:55:59.043715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:32.286503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656020+0000","last_change":"2024-07-17T06:55:57.023183+0000","last_active":"2024-07-17T06:55:59.656020+0000","last_peered":"2024-07-17T06:55:59.656020+0000","last_clean":"2024-07-17T06:55:59.656020+0000","last_became_active":"2024-07-17T06:55:57.022952+0000","last_became_peered":"2024-07-17T06:55:57.022952+0000","last_unstale":"2024-07-17T06:55:59.656020+0000","last_undegraded":"2024-07-17T06:55:59.656020+0000","last_fullsized":"2024-07-17T06:55:59.656020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:09:23.205075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657799+0000","last_change":"2024-07-17T06:55:58.040402+0000","last_active":"2024-07-17T06:55:59.657799+0000","last_peered":"2024-07-17T06:55:59.657799+0000","last_clean":"2024-07-17T06:55:59.657799+0000","last_became_active":"2024-07-17T06:55:58.040131+0000","last_became_peered":"2024-07-17T06:55:58.040131+0000","last_unstale":"2024-07-17T06:55:59.657799+0000","last_undegraded":"2024-07-17T06:55:59.657799+0000","last_fullsized":"2024-07-17T06:55:59.657799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:42.200860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042325+0000","last_change":"2024-07-17T06:55:57.015368+0000","last_active":"2024-07-17T06:55:59.042325+0000","last_peered":"2024-07-17T06:55:59.042325+0000","last_clean":"2024-07-17T06:55:59.042325+0000","last_became_active":"2024-07-17T06:55:57.013915+0000","last_became_peered":"2024-07-17T06:55:57.013915+0000","last_unstale":"2024-07-17T06:55:59.042325+0000","last_undegraded":"2024-07-17T06:55:59.042325+0000","last_fullsized":"2024-07-17T06:55:59.042325+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:04.895014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036930+0000","last_change":"2024-07-17T06:55:58.042878+0000","last_active":"2024-07-17T06:55:59.036930+0000","last_peered":"2024-07-17T06:55:59.036930+0000","last_clean":"2024-07-17T06:55:59.036930+0000","last_became_active":"2024-07-17T06:55:58.042538+0000","last_became_peered":"2024-07-17T06:55:58.042538+0000","last_unstale":"2024-07-17T06:55:59.036930+0000","last_undegraded":"2024-07-17T06:55:59.036930+0000","last_fullsized":"2024-07-17T06:55:59.036930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:09:46.207572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659668+0000","last_change":"2024-07-17T06:55:57.026571+0000","last_active":"2024-07-17T06:55:59.659668+0000","last_peered":"2024-07-17T06:55:59.659668+0000","last_clean":"2024-07-17T06:55:59.659668+0000","last_became_active":"2024-07-17T06:55:57.026249+0000","last_became_peered":"2024-07-17T06:55:57.026249+0000","last_unstale":"2024-07-17T06:55:59.659668+0000","last_undegraded":"2024-07-17T06:55:59.659668+0000","last_fullsized":"2024-07-17T06:55:59.659668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:30.625072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657481+0000","last_change":"2024-07-17T06:55:58.058048+0000","last_active":"2024-07-17T06:55:59.657481+0000","last_peered":"2024-07-17T06:55:59.657481+0000","last_clean":"2024-07-17T06:55:59.657481+0000","last_became_active":"2024-07-17T06:55:58.057750+0000","last_became_peered":"2024-07-17T06:55:58.057750+0000","last_unstale":"2024-07-17T06:55:59.657481+0000","last_undegraded":"2024-07-17T06:55:59.657481+0000","last_fullsized":"2024-07-17T06:55:59.657481+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:27:04.587398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047425+0000","last_change":"2024-07-17T06:55:57.025366+0000","last_active":"2024-07-17T06:55:59.047425+0000","last_peered":"2024-07-17T06:55:59.047425+0000","last_clean":"2024-07-17T06:55:59.047425+0000","last_became_active":"2024-07-17T06:55:57.024821+0000","last_became_peered":"2024-07-17T06:55:57.024821+0000","last_unstale":"2024-07-17T06:55:59.047425+0000","last_undegraded":"2024-07-17T06:55:59.047425+0000","last_fullsized":"2024-07-17T06:55:59.047425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:48.777690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660626+0000","last_change":"2024-07-17T06:55:58.044118+0000","last_active":"2024-07-17T06:55:59.660626+0000","last_peered":"2024-07-17T06:55:59.660626+0000","last_clean":"2024-07-17T06:55:59.660626+0000","last_became_active":"2024-07-17T06:55:58.043853+0000","last_became_peered":"2024-07-17T06:55:58.043853+0000","last_unstale":"2024-07-17T06:55:59.660626+0000","last_undegraded":"2024-07-17T06:55:59.660626+0000","last_fullsized":"2024-07-17T06:55:59.660626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:52.345032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047514+0000","last_change":"2024-07-17T06:55:57.025608+0000","last_active":"2024-07-17T06:55:59.047514+0000","last_peered":"2024-07-17T06:55:59.047514+0000","last_clean":"2024-07-17T06:55:59.047514+0000","last_became_active":"2024-07-17T06:55:57.024617+0000","last_became_peered":"2024-07-17T06:55:57.024617+0000","last_unstale":"2024-07-17T06:55:59.047514+0000","last_undegraded":"2024-07-17T06:55:59.047514+0000","last_fullsized":"2024-07-17T06:55:59.047514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044747+0000","last_change":"2024-07-17T06:55:58.037543+0000","last_active":"2024-07-17T06:55:59.044747+0000","last_peered":"2024-07-17T06:55:59.044747+0000","last_clean":"2024-07-17T06:55:59.044747+0000","last_became_active":"2024-07-17T06:55:58.033368+0000","last_became_peered":"2024-07-17T06:55:58.033368+0000","last_unstale":"2024-07-17T06:55:59.044747+0000","last_undegraded":"2024-07-17T06:55:59.044747+0000","last_fullsized":"2024-07-17T06:55:59.044747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:26:05.155740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658598+0000","last_change":"2024-07-17T06:55:57.016760+0000","last_active":"2024-07-17T06:55:59.658598+0000","last_peered":"2024-07-17T06:55:59.658598+0000","last_clean":"2024-07-17T06:55:59.658598+0000","last_became_active":"2024-07-17T06:55:57.016398+0000","last_became_peered":"2024-07-17T06:55:57.016398+0000","last_unstale":"2024-07-17T06:55:59.658598+0000","last_undegraded":"2024-07-17T06:55:59.658598+0000","last_fullsized":"2024-07-17T06:55:59.658598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:58.711643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656834+0000","last_change":"2024-07-17T06:55:58.063954+0000","last_active":"2024-07-17T06:55:59.656834+0000","last_peered":"2024-07-17T06:55:59.656834+0000","last_clean":"2024-07-17T06:55:59.656834+0000","last_became_active":"2024-07-17T06:55:58.063630+0000","last_became_peered":"2024-07-17T06:55:58.063630+0000","last_unstale":"2024-07-17T06:55:59.656834+0000","last_undegraded":"2024-07-17T06:55:59.656834+0000","last_fullsized":"2024-07-17T06:55:59.656834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:58.834429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658155+0000","last_change":"2024-07-17T06:55:57.014743+0000","last_active":"2024-07-17T06:55:59.658155+0000","last_peered":"2024-07-17T06:55:59.658155+0000","last_clean":"2024-07-17T06:55:59.658155+0000","last_became_active":"2024-07-17T06:55:57.013321+0000","last_became_peered":"2024-07-17T06:55:57.013321+0000","last_unstale":"2024-07-17T06:55:59.658155+0000","last_undegraded":"2024-07-17T06:55:59.658155+0000","last_fullsized":"2024-07-17T06:55:59.658155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.590920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658277+0000","last_change":"2024-07-17T06:55:58.032791+0000","last_active":"2024-07-17T06:55:59.658277+0000","last_peered":"2024-07-17T06:55:59.658277+0000","last_clean":"2024-07-17T06:55:59.658277+0000","last_became_active":"2024-07-17T06:55:58.032467+0000","last_became_peered":"2024-07-17T06:55:58.032467+0000","last_unstale":"2024-07-17T06:55:59.658277+0000","last_undegraded":"2024-07-17T06:55:59.658277+0000","last_fullsized":"2024-07-17T06:55:59.658277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:42.383602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655477+0000","last_change":"2024-07-17T06:55:57.023971+0000","last_active":"2024-07-17T06:55:59.655477+0000","last_peered":"2024-07-17T06:55:59.655477+0000","last_clean":"2024-07-17T06:55:59.655477+0000","last_became_active":"2024-07-17T06:55:57.023779+0000","last_became_peered":"2024-07-17T06:55:57.023779+0000","last_unstale":"2024-07-17T06:55:59.655477+0000","last_undegraded":"2024-07-17T06:55:59.655477+0000","last_fullsized":"2024-07-17T06:55:59.655477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:15.562098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656802+0000","last_change":"2024-07-17T06:55:58.045964+0000","last_active":"2024-07-17T06:55:59.656802+0000","last_peered":"2024-07-17T06:55:59.656802+0000","last_clean":"2024-07-17T06:55:59.656802+0000","last_became_active":"2024-07-17T06:55:58.045590+0000","last_became_peered":"2024-07-17T06:55:58.045590+0000","last_unstale":"2024-07-17T06:55:59.656802+0000","last_undegraded":"2024-07-17T06:55:59.656802+0000","last_fullsized":"2024-07-17T06:55:59.656802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:45:34.831048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042918+0000","last_change":"2024-07-17T06:55:57.015366+0000","last_active":"2024-07-17T06:55:59.042918+0000","last_peered":"2024-07-17T06:55:59.042918+0000","last_clean":"2024-07-17T06:55:59.042918+0000","last_became_active":"2024-07-17T06:55:57.015091+0000","last_became_peered":"2024-07-17T06:55:57.015091+0000","last_unstale":"2024-07-17T06:55:59.042918+0000","last_undegraded":"2024-07-17T06:55:59.042918+0000","last_fullsized":"2024-07-17T06:55:59.042918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:22.754750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657772+0000","last_change":"2024-07-17T06:55:58.033692+0000","last_active":"2024-07-17T06:55:59.657772+0000","last_peered":"2024-07-17T06:55:59.657772+0000","last_clean":"2024-07-17T06:55:59.657772+0000","last_became_active":"2024-07-17T06:55:58.032538+0000","last_became_peered":"2024-07-17T06:55:58.032538+0000","last_unstale":"2024-07-17T06:55:59.657772+0000","last_undegraded":"2024-07-17T06:55:59.657772+0000","last_fullsized":"2024-07-17T06:55:59.657772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:31.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656219+0000","last_change":"2024-07-17T06:55:57.025095+0000","last_active":"2024-07-17T06:55:59.656219+0000","last_peered":"2024-07-17T06:55:59.656219+0000","last_clean":"2024-07-17T06:55:59.656219+0000","last_became_active":"2024-07-17T06:55:57.023655+0000","last_became_peered":"2024-07-17T06:55:57.023655+0000","last_unstale":"2024-07-17T06:55:59.656219+0000","last_undegraded":"2024-07-17T06:55:59.656219+0000","last_fullsized":"2024-07-17T06:55:59.656219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:28:33.365794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658639+0000","last_change":"2024-07-17T06:55:58.050981+0000","last_active":"2024-07-17T06:55:59.658639+0000","last_peered":"2024-07-17T06:55:59.658639+0000","last_clean":"2024-07-17T06:55:59.658639+0000","last_became_active":"2024-07-17T06:55:58.050714+0000","last_became_peered":"2024-07-17T06:55:58.050714+0000","last_unstale":"2024-07-17T06:55:59.658639+0000","last_undegraded":"2024-07-17T06:55:59.658639+0000","last_fullsized":"2024-07-17T06:55:59.658639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:16.112850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043600+0000","last_change":"2024-07-17T06:55:57.030109+0000","last_active":"2024-07-17T06:55:59.043600+0000","last_peered":"2024-07-17T06:55:59.043600+0000","last_clean":"2024-07-17T06:55:59.043600+0000","last_became_active":"2024-07-17T06:55:57.029434+0000","last_became_peered":"2024-07-17T06:55:57.029434+0000","last_unstale":"2024-07-17T06:55:59.043600+0000","last_undegraded":"2024-07-17T06:55:59.043600+0000","last_fullsized":"2024-07-17T06:55:59.043600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:47.719938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.047275+0000","last_change":"2024-07-17T06:55:58.056458+0000","last_active":"2024-07-17T06:55:59.047275+0000","last_peered":"2024-07-17T06:55:59.047275+0000","last_clean":"2024-07-17T06:55:59.047275+0000","last_became_active":"2024-07-17T06:55:58.056080+0000","last_became_peered":"2024-07-17T06:55:58.056080+0000","last_unstale":"2024-07-17T06:55:59.047275+0000","last_undegraded":"2024-07-17T06:55:59.047275+0000","last_fullsized":"2024-07-17T06:55:59.047275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:39.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045886+0000","last_change":"2024-07-17T06:55:57.027880+0000","last_active":"2024-07-17T06:55:59.045886+0000","last_peered":"2024-07-17T06:55:59.045886+0000","last_clean":"2024-07-17T06:55:59.045886+0000","last_became_active":"2024-07-17T06:55:57.027361+0000","last_became_peered":"2024-07-17T06:55:57.027361+0000","last_unstale":"2024-07-17T06:55:59.045886+0000","last_undegraded":"2024-07-17T06:55:59.045886+0000","last_fullsized":"2024-07-17T06:55:59.045886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.693484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.045341+0000","last_change":"2024-07-17T06:55:58.034283+0000","last_active":"2024-07-17T06:55:59.045341+0000","last_peered":"2024-07-17T06:55:59.045341+0000","last_clean":"2024-07-17T06:55:59.045341+0000","last_became_active":"2024-07-17T06:55:58.033852+0000","last_became_peered":"2024-07-17T06:55:58.033852+0000","last_unstale":"2024-07-17T06:55:59.045341+0000","last_undegraded":"2024-07-17T06:55:59.045341+0000","last_fullsized":"2024-07-17T06:55:59.045341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:23.626316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036423+0000","last_change":"2024-07-17T06:55:57.018791+0000","last_active":"2024-07-17T06:55:59.036423+0000","last_peered":"2024-07-17T06:55:59.036423+0000","last_clean":"2024-07-17T06:55:59.036423+0000","last_became_active":"2024-07-17T06:55:57.016819+0000","last_became_peered":"2024-07-17T06:55:57.016819+0000","last_unstale":"2024-07-17T06:55:59.036423+0000","last_undegraded":"2024-07-17T06:55:59.036423+0000","last_fullsized":"2024-07-17T06:55:59.036423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:37.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036056+0000","last_change":"2024-07-17T06:55:58.029485+0000","last_active":"2024-07-17T06:55:59.036056+0000","last_peered":"2024-07-17T06:55:59.036056+0000","last_clean":"2024-07-17T06:55:59.036056+0000","last_became_active":"2024-07-17T06:55:58.029204+0000","last_became_peered":"2024-07-17T06:55:58.029204+0000","last_unstale":"2024-07-17T06:55:59.036056+0000","last_undegraded":"2024-07-17T06:55:59.036056+0000","last_fullsized":"2024-07-17T06:55:59.036056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:36.108890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656113+0000","last_change":"2024-07-17T06:55:57.020021+0000","last_active":"2024-07-17T06:55:59.656113+0000","last_peered":"2024-07-17T06:55:59.656113+0000","last_clean":"2024-07-17T06:55:59.656113+0000","last_became_active":"2024-07-17T06:55:57.019599+0000","last_became_peered":"2024-07-17T06:55:57.019599+0000","last_unstale":"2024-07-17T06:55:59.656113+0000","last_undegraded":"2024-07-17T06:55:59.656113+0000","last_fullsized":"2024-07-17T06:55:59.656113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:20.727281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657092+0000","last_change":"2024-07-17T06:55:58.051497+0000","last_active":"2024-07-17T06:55:59.657092+0000","last_peered":"2024-07-17T06:55:59.657092+0000","last_clean":"2024-07-17T06:55:59.657092+0000","last_became_active":"2024-07-17T06:55:58.051167+0000","last_became_peered":"2024-07-17T06:55:58.051167+0000","last_unstale":"2024-07-17T06:55:59.657092+0000","last_undegraded":"2024-07-17T06:55:59.657092+0000","last_fullsized":"2024-07-17T06:55:59.657092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:20.126847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035964+0000","last_change":"2024-07-17T06:55:57.014314+0000","last_active":"2024-07-17T06:55:59.035964+0000","last_peered":"2024-07-17T06:55:59.035964+0000","last_clean":"2024-07-17T06:55:59.035964+0000","last_became_active":"2024-07-17T06:55:57.014009+0000","last_became_peered":"2024-07-17T06:55:57.014009+0000","last_unstale":"2024-07-17T06:55:59.035964+0000","last_undegraded":"2024-07-17T06:55:59.035964+0000","last_fullsized":"2024-07-17T06:55:59.035964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:03.980067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.659453+0000","last_change":"2024-07-17T06:55:58.050671+0000","last_active":"2024-07-17T06:55:59.659453+0000","last_peered":"2024-07-17T06:55:59.659453+0000","last_clean":"2024-07-17T06:55:59.659453+0000","last_became_active":"2024-07-17T06:55:58.049737+0000","last_became_peered":"2024-07-17T06:55:58.049737+0000","last_unstale":"2024-07-17T06:55:59.659453+0000","last_undegraded":"2024-07-17T06:55:59.659453+0000","last_fullsized":"2024-07-17T06:55:59.659453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:10.010707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035633+0000","last_change":"2024-07-17T06:55:57.014718+0000","last_active":"2024-07-17T06:55:59.035633+0000","last_peered":"2024-07-17T06:55:59.035633+0000","last_clean":"2024-07-17T06:55:59.035633+0000","last_became_active":"2024-07-17T06:55:57.014397+0000","last_became_peered":"2024-07-17T06:55:57.014397+0000","last_unstale":"2024-07-17T06:55:59.035633+0000","last_undegraded":"2024-07-17T06:55:59.035633+0000","last_fullsized":"2024-07-17T06:55:59.035633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:25.053286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.042498+0000","last_change":"2024-07-17T06:55:58.059706+0000","last_active":"2024-07-17T06:55:59.042498+0000","last_peered":"2024-07-17T06:55:59.042498+0000","last_clean":"2024-07-17T06:55:59.042498+0000","last_became_active":"2024-07-17T06:55:58.057842+0000","last_became_peered":"2024-07-17T06:55:58.057842+0000","last_unstale":"2024-07-17T06:55:59.042498+0000","last_undegraded":"2024-07-17T06:55:59.042498+0000","last_fullsized":"2024-07-17T06:55:59.042498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:31.059724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656493+0000","last_change":"2024-07-17T06:55:57.026804+0000","last_active":"2024-07-17T06:55:59.656493+0000","last_peered":"2024-07-17T06:55:59.656493+0000","last_clean":"2024-07-17T06:55:59.656493+0000","last_became_active":"2024-07-17T06:55:57.024887+0000","last_became_peered":"2024-07-17T06:55:57.024887+0000","last_unstale":"2024-07-17T06:55:59.656493+0000","last_undegraded":"2024-07-17T06:55:59.656493+0000","last_fullsized":"2024-07-17T06:55:59.656493+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:20:31.619884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658214+0000","last_change":"2024-07-17T06:55:58.041686+0000","last_active":"2024-07-17T06:55:59.658214+0000","last_peered":"2024-07-17T06:55:59.658214+0000","last_clean":"2024-07-17T06:55:59.658214+0000","last_became_active":"2024-07-17T06:55:58.040003+0000","last_became_peered":"2024-07-17T06:55:58.040003+0000","last_unstale":"2024-07-17T06:55:59.658214+0000","last_undegraded":"2024-07-17T06:55:59.658214+0000","last_fullsized":"2024-07-17T06:55:59.658214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:50:35.078043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.035215+0000","last_change":"2024-07-17T06:55:57.015139+0000","last_active":"2024-07-17T06:55:59.035215+0000","last_peered":"2024-07-17T06:55:59.035215+0000","last_clean":"2024-07-17T06:55:59.035215+0000","last_became_active":"2024-07-17T06:55:57.012240+0000","last_became_peered":"2024-07-17T06:55:57.012240+0000","last_unstale":"2024-07-17T06:55:59.035215+0000","last_undegraded":"2024-07-17T06:55:59.035215+0000","last_fullsized":"2024-07-17T06:55:59.035215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:09.873719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655851+0000","last_change":"2024-07-17T06:55:58.059105+0000","last_active":"2024-07-17T06:55:59.655851+0000","last_peered":"2024-07-17T06:55:59.655851+0000","last_clean":"2024-07-17T06:55:59.655851+0000","last_became_active":"2024-07-17T06:55:58.058356+0000","last_became_peered":"2024-07-17T06:55:58.058356+0000","last_unstale":"2024-07-17T06:55:59.655851+0000","last_undegraded":"2024-07-17T06:55:59.655851+0000","last_fullsized":"2024-07-17T06:55:59.655851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.044173+0000","last_change":"2024-07-17T06:55:57.029338+0000","last_active":"2024-07-17T06:55:59.044173+0000","last_peered":"2024-07-17T06:55:59.044173+0000","last_clean":"2024-07-17T06:55:59.044173+0000","last_became_active":"2024-07-17T06:55:57.028513+0000","last_became_peered":"2024-07-17T06:55:57.028513+0000","last_unstale":"2024-07-17T06:55:59.044173+0000","last_undegraded":"2024-07-17T06:55:59.044173+0000","last_fullsized":"2024-07-17T06:55:59.044173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.590920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657491+0000","last_change":"2024-07-17T06:55:58.057664+0000","last_active":"2024-07-17T06:55:59.657491+0000","last_peered":"2024-07-17T06:55:59.657491+0000","last_clean":"2024-07-17T06:55:59.657491+0000","last_became_active":"2024-07-17T06:55:58.057223+0000","last_became_peered":"2024-07-17T06:55:58.057223+0000","last_unstale":"2024-07-17T06:55:59.657491+0000","last_undegraded":"2024-07-17T06:55:59.657491+0000","last_fullsized":"2024-07-17T06:55:59.657491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:44.455462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.036321+0000","last_change":"2024-07-17T06:55:57.027451+0000","last_active":"2024-07-17T06:55:59.036321+0000","last_peered":"2024-07-17T06:55:59.036321+0000","last_clean":"2024-07-17T06:55:59.036321+0000","last_became_active":"2024-07-17T06:55:57.027142+0000","last_became_peered":"2024-07-17T06:55:57.027142+0000","last_unstale":"2024-07-17T06:55:59.036321+0000","last_undegraded":"2024-07-17T06:55:59.036321+0000","last_fullsized":"2024-07-17T06:55:59.036321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:27:05.080931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.658942+0000","last_change":"2024-07-17T06:55:58.041665+0000","last_active":"2024-07-17T06:55:59.658942+0000","last_peered":"2024-07-17T06:55:59.658942+0000","last_clean":"2024-07-17T06:55:59.658942+0000","last_became_active":"2024-07-17T06:55:58.041353+0000","last_became_peered":"2024-07-17T06:55:58.041353+0000","last_unstale":"2024-07-17T06:55:59.658942+0000","last_undegraded":"2024-07-17T06:55:59.658942+0000","last_fullsized":"2024-07-17T06:55:59.658942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:31:51.390519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660361+0000","last_change":"2024-07-17T06:55:57.020427+0000","last_active":"2024-07-17T06:55:59.660361+0000","last_peered":"2024-07-17T06:55:59.660361+0000","last_clean":"2024-07-17T06:55:59.660361+0000","last_became_active":"2024-07-17T06:55:57.020107+0000","last_became_peered":"2024-07-17T06:55:57.020107+0000","last_unstale":"2024-07-17T06:55:59.660361+0000","last_undegraded":"2024-07-17T06:55:59.660361+0000","last_fullsized":"2024-07-17T06:55:59.660361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:09:23.205075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657241+0000","last_change":"2024-07-17T06:55:58.038842+0000","last_active":"2024-07-17T06:55:59.657241+0000","last_peered":"2024-07-17T06:55:59.657241+0000","last_clean":"2024-07-17T06:55:59.657241+0000","last_became_active":"2024-07-17T06:55:58.038214+0000","last_became_peered":"2024-07-17T06:55:58.038214+0000","last_unstale":"2024-07-17T06:55:59.657241+0000","last_undegraded":"2024-07-17T06:55:59.657241+0000","last_fullsized":"2024-07-17T06:55:59.657241+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:36.495720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.660159+0000","last_change":"2024-07-17T06:55:57.018030+0000","last_active":"2024-07-17T06:55:59.660159+0000","last_peered":"2024-07-17T06:55:59.660159+0000","last_clean":"2024-07-17T06:55:59.660159+0000","last_became_active":"2024-07-17T06:55:57.017102+0000","last_became_peered":"2024-07-17T06:55:57.017102+0000","last_unstale":"2024-07-17T06:55:59.660159+0000","last_undegraded":"2024-07-17T06:55:59.660159+0000","last_fullsized":"2024-07-17T06:55:59.660159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.655990+0000","last_change":"2024-07-17T06:55:58.049117+0000","last_active":"2024-07-17T06:55:59.655990+0000","last_peered":"2024-07-17T06:55:59.655990+0000","last_clean":"2024-07-17T06:55:59.655990+0000","last_became_active":"2024-07-17T06:55:58.048851+0000","last_became_peered":"2024-07-17T06:55:58.048851+0000","last_unstale":"2024-07-17T06:55:59.655990+0000","last_undegraded":"2024-07-17T06:55:59.655990+0000","last_fullsized":"2024-07-17T06:55:59.655990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:10.010707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043510+0000","last_change":"2024-07-17T06:55:57.017786+0000","last_active":"2024-07-17T06:55:59.043510+0000","last_peered":"2024-07-17T06:55:59.043510+0000","last_clean":"2024-07-17T06:55:59.043510+0000","last_became_active":"2024-07-17T06:55:57.017469+0000","last_became_peered":"2024-07-17T06:55:57.017469+0000","last_unstale":"2024-07-17T06:55:59.043510+0000","last_undegraded":"2024-07-17T06:55:59.043510+0000","last_fullsized":"2024-07-17T06:55:59.043510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:53.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.041987+0000","last_change":"2024-07-17T06:55:58.034550+0000","last_active":"2024-07-17T06:55:59.041987+0000","last_peered":"2024-07-17T06:55:59.041987+0000","last_clean":"2024-07-17T06:55:59.041987+0000","last_became_active":"2024-07-17T06:55:58.034058+0000","last_became_peered":"2024-07-17T06:55:58.034058+0000","last_unstale":"2024-07-17T06:55:59.041987+0000","last_undegraded":"2024-07-17T06:55:59.041987+0000","last_fullsized":"2024-07-17T06:55:59.041987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:28:34.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.041295+0000","last_change":"2024-07-17T06:55:57.014519+0000","last_active":"2024-07-17T06:55:59.041295+0000","last_peered":"2024-07-17T06:55:59.041295+0000","last_clean":"2024-07-17T06:55:59.041295+0000","last_became_active":"2024-07-17T06:55:57.014241+0000","last_became_peered":"2024-07-17T06:55:57.014241+0000","last_unstale":"2024-07-17T06:55:59.041295+0000","last_undegraded":"2024-07-17T06:55:59.041295+0000","last_fullsized":"2024-07-17T06:55:59.041295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:20.293369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657143+0000","last_change":"2024-07-17T06:55:58.058633+0000","last_active":"2024-07-17T06:55:59.657143+0000","last_peered":"2024-07-17T06:55:59.657143+0000","last_clean":"2024-07-17T06:55:59.657143+0000","last_became_active":"2024-07-17T06:55:58.058282+0000","last_became_peered":"2024-07-17T06:55:58.058282+0000","last_unstale":"2024-07-17T06:55:59.657143+0000","last_undegraded":"2024-07-17T06:55:59.657143+0000","last_fullsized":"2024-07-17T06:55:59.657143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:13.537454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657049+0000","last_change":"2024-07-17T06:55:57.020044+0000","last_active":"2024-07-17T06:55:59.657049+0000","last_peered":"2024-07-17T06:55:59.657049+0000","last_clean":"2024-07-17T06:55:59.657049+0000","last_became_active":"2024-07-17T06:55:57.018728+0000","last_became_peered":"2024-07-17T06:55:57.018728+0000","last_unstale":"2024-07-17T06:55:59.657049+0000","last_undegraded":"2024-07-17T06:55:59.657049+0000","last_fullsized":"2024-07-17T06:55:59.657049+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.280390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.043144+0000","last_change":"2024-07-17T06:55:58.057996+0000","last_active":"2024-07-17T06:55:59.043144+0000","last_peered":"2024-07-17T06:55:59.043144+0000","last_clean":"2024-07-17T06:55:59.043144+0000","last_became_active":"2024-07-17T06:55:58.056857+0000","last_became_peered":"2024-07-17T06:55:58.056857+0000","last_unstale":"2024-07-17T06:55:59.043144+0000","last_undegraded":"2024-07-17T06:55:59.043144+0000","last_fullsized":"2024-07-17T06:55:59.043144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:56.977863+0000","last_clean_scrub_stamp":"2024-07-17T06:55:56.977863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:42.200860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.657063+0000","last_change":"2024-07-17T06:55:57.021175+0000","last_active":"2024-07-17T06:55:59.657063+0000","last_peered":"2024-07-17T06:55:59.657063+0000","last_clean":"2024-07-17T06:55:59.657063+0000","last_became_active":"2024-07-17T06:55:57.020798+0000","last_became_peered":"2024-07-17T06:55:57.020798+0000","last_unstale":"2024-07-17T06:55:59.657063+0000","last_undegraded":"2024-07-17T06:55:59.657063+0000","last_fullsized":"2024-07-17T06:55:59.657063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:55:55.964774+0000","last_clean_scrub_stamp":"2024-07-17T06:55:55.964774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:44.639430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":141,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-07-17T06:55:59.656575+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T06:55:59.656575+0000","last_peered":"2024-07-17T06:55:59.656575+0000","last_clean":"2024-07-17T06:55:59.656575+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T06:55:59.656575+0000","last_undegraded":"2024-07-17T06:55:59.656575+0000","last_fullsized":"2024-07-17T06:55:59.656575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":192265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":191795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":193329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1040384,"data_stored":782085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":783733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":192287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":191753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T06:56:08.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T06:56:08.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 -- 172.21.15.44:0/3200692649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d800ac3a0 msgr2=0x7f6d800ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 --2- 172.21.15.44:0/3200692649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ac810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d78005fd0 tx=0x7f6d7802b860 comp rx=0 tx=0).stop 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 -- 172.21.15.44:0/3200692649 shutdown_connections 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 --2- 172.21.15.44:0/3200692649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ac810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 --2- 172.21.15.44:0/3200692649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 0x7f6d800b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 -- 172.21.15.44:0/3200692649 >> 172.21.15.44:0/3200692649 conn(0x7f6d800b0eb0 msgr2=0x7f6d800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:08.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 -- 172.21.15.44:0/3200692649 shutdown_connections 2024-07-17T06:56:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.317+0000 7f6d85e00640 1 -- 172.21.15.44:0/3200692649 wait complete. 2024-07-17T06:56:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.319+0000 7f6d85e00640 1 Processor -- start 2024-07-17T06:56:08.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.319+0000 7f6d85e00640 1 -- start start 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 0x7f6d800cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8006f850 con 0x7f6d8006eb90 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8006f6d0 con 0x7f6d800ac3a0 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d85e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8014fd60 con 0x7f6d8015a5a0 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43372/0 (socket says 172.21.15.44:43372) 2024-07-17T06:56:08.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d7ea00640 1 -- 172.21.15.44:0/537035882 learned_addr learned my addr 172.21.15.44:0/537035882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d7f400640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 0x7f6d800cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.320+0000 7f6d7ea00640 1 -- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 msgr2=0x7f6d800cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7ea00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 0x7f6d800cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7e000640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7ea00640 1 -- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d800ac3a0 msgr2=0x7f6d800ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7ea00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7ea00640 1 -- 172.21.15.44:0/537035882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d80073c90 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d7ea00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c430 tx=0x7f6d7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70011790 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d70017070 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d80073f70 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70011d00 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.321+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d80074530 con 0x7f6d8006eb90 2024-07-17T06:56:08.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.322+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d800ac810 con 0x7f6d8006eb90 2024-07-17T06:56:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.324+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f6d7001c460 con 0x7f6d8006eb90 2024-07-17T06:56:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.325+0000 7f6d77400640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 0x7f6d60079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.325+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6d70098480 con 0x7f6d8006eb90 2024-07-17T06:56:08.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.325+0000 7f6d7e000640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 0x7f6d60079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.327+0000 7f6d7e000640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 0x7f6d60079070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005fd0 tx=0x7f6d6c005f60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:08.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.330+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6d70061720 con 0x7f6d8006eb90 2024-07-17T06:56:08.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.617+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6d800b4200 con 0x7f6d8006eb90 2024-07-17T06:56:08.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.622+0000 7f6d77400640 1 -- 172.21.15.44:0/537035882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7f6d70061020 con 0x7f6d8006eb90 2024-07-17T06:56:08.622 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T06:56:08.622 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T06:56:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T06:56:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T06:56:08.623 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T06:56:08.623 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.625+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 msgr2=0x7f6d60079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.625+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 0x7f6d60079070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005fd0 tx=0x7f6d6c005f60 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.625+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 msgr2=0x7f6d8015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.625+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c430 tx=0x7f6d7000c8f0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 shutdown_connections 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6d60076bb0 0x7f6d60079070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d8015a5a0 0x7f6d800cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d800ac3a0 0x7f6d800ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 --2- 172.21.15.44:0/537035882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d8006eb90 0x7f6d8015c100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 >> 172.21.15.44:0/537035882 conn(0x7f6d800b0eb0 msgr2=0x7f6d800cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 shutdown_connections 2024-07-17T06:56:08.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.626+0000 7f6d85e00640 1 -- 172.21.15.44:0/537035882 wait complete. 2024-07-17T06:56:08.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.854+0000 7f6559200640 1 -- 172.21.15.44:0/392297440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 msgr2=0x7f65540ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.854+0000 7f6559200640 1 --2- 172.21.15.44:0/392297440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f65540ac560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65540b35b0 tx=0x7f654c02aa30 comp rx=0 tx=0).stop 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 -- 172.21.15.44:0/392297440 shutdown_connections 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 --2- 172.21.15.44:0/392297440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f65540ac560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 --2- 172.21.15.44:0/392297440 >> v2:172.21.15.175:3300/0 conn(0x7f655409ca20 0x7f65540cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 -- 172.21.15.44:0/392297440 >> 172.21.15.44:0/392297440 conn(0x7f65540b0e80 msgr2=0x7f65540b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 -- 172.21.15.44:0/392297440 shutdown_connections 2024-07-17T06:56:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.855+0000 7f6559200640 1 -- 172.21.15.44:0/392297440 wait complete. 2024-07-17T06:56:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.856+0000 7f6559200640 1 Processor -- start 2024-07-17T06:56:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 -- start start 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 0x7f65541f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f655415d240 0x7f6554073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65540ace30 con 0x7f65540cf810 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65540accb0 con 0x7f655409ca20 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.857+0000 7f6559200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65540acfb0 con 0x7f655415d240 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39042/0 (socket says 172.21.15.44:39042) 2024-07-17T06:56:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 -- 172.21.15.44:0/1556104291 learned_addr learned my addr 172.21.15.44:0/1556104291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6553e00640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f655415d240 0x7f6554073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6553400640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 0x7f65541f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 -- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f655415d240 msgr2=0x7f6554073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f655415d240 0x7f6554073540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 -- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 msgr2=0x7f65541f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 0x7f65541f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6552a00640 1 -- 172.21.15.44:0/1556104291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655414b010 con 0x7f65540cf810 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.858+0000 7f6553400640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 0x7f65541f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f6552a00640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f654000b4a0 tx=0x7f654000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540014070 con 0x7f65540cf810 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6540010700 con 0x7f65540cf810 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540010b50 con 0x7f65540cf810 2024-07-17T06:56:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6554073e90 con 0x7f65540cf810 2024-07-17T06:56:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.859+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541f78e0 con 0x7f65540cf810 2024-07-17T06:56:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.860+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65540ad500 con 0x7f65540cf810 2024-07-17T06:56:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.861+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f6540017070 con 0x7f65540cf810 2024-07-17T06:56:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.862+0000 7f654b400640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 0x7f65340790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.863+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6540097f10 con 0x7f65540cf810 2024-07-17T06:56:08.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.863+0000 7f6553400640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 0x7f65340790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.864+0000 7f6553400640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 0x7f65340790a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65440016a0 tx=0x7f6544009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:08.867+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6540061240 con 0x7f65540cf810 2024-07-17T06:56:09.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.125+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65540b23b0 con 0x7f65540cf810 2024-07-17T06:56:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.126+0000 7f654b400640 1 -- 172.21.15.44:0/1556104291 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f6540060b40 con 0x7f65540cf810 2024-07-17T06:56:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T06:56:09.127 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T06:56:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 msgr2=0x7f65340790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 0x7f65340790a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65440016a0 tx=0x7f6544009040 comp rx=0 tx=0).stop 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 msgr2=0x7f655414aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f654000b4a0 tx=0x7f654000b960 comp rx=0 tx=0).stop 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 shutdown_connections 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f6534076be0 0x7f65340790a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f655415d240 0x7f6554073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65540cf810 0x7f655414aad0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 --2- 172.21.15.44:0/1556104291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655409ca20 0x7f65541f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 >> 172.21.15.44:0/1556104291 conn(0x7f65540b0e80 msgr2=0x7f6554157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 shutdown_connections 2024-07-17T06:56:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.130+0000 7f6559200640 1 -- 172.21.15.44:0/1556104291 wait complete. 2024-07-17T06:56:09.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T06:55:57.477076+0000', 'modified': '2024-07-17T06:56:00.564713+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4738': {'gid': 4738, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/1597326512', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 1597326512}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1597326512}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4738, 'qdb_cluster': [4738]} 2024-07-17T06:56:09.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T06:56:09.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T06:56:09.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 -- 172.21.15.44:0/1430288563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 msgr2=0x7f5f840cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 --2- 172.21.15.44:0/1430288563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f840cfdf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c02a820 comp rx=0 tx=0).stop 2024-07-17T06:56:09.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 -- 172.21.15.44:0/1430288563 shutdown_connections 2024-07-17T06:56:09.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 --2- 172.21.15.44:0/1430288563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f840cfdf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 --2- 172.21.15.44:0/1430288563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 0x7f5f8415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 -- 172.21.15.44:0/1430288563 >> 172.21.15.44:0/1430288563 conn(0x7f5f840b4ff0 msgr2=0x7f5f840b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 -- 172.21.15.44:0/1430288563 shutdown_connections 2024-07-17T06:56:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.775+0000 7f5f83200640 1 -- 172.21.15.44:0/1430288563 wait complete. 2024-07-17T06:56:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.777+0000 7f5f83200640 1 Processor -- start 2024-07-17T06:56:09.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.777+0000 7f5f83200640 1 -- start start 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 0x7f5f840aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f84161be0 0x7f5f840b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f840d1a90 con 0x7f5f840ce690 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f840d1910 con 0x7f5f84161be0 2024-07-17T06:56:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f83200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f840d1c10 con 0x7f5f8415d130 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39052/0 (socket says 172.21.15.44:39052) 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.778+0000 7f5f81e00640 1 -- 172.21.15.44:0/743452149 learned_addr learned my addr 172.21.15.44:0/743452149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81400640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 0x7f5f840aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f82800640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f84161be0 0x7f5f840b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 -- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 msgr2=0x7f5f840aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 0x7f5f840aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 -- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f84161be0 msgr2=0x7f5f840b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f84161be0 0x7f5f840b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 -- 172.21.15.44:0/743452149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f840b1c20 con 0x7f5f840ce690 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f81e00640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400d7a0 tx=0x7f5f7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f7a000640 1 -- 172.21.15.44:0/743452149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74017070 con 0x7f5f840ce690 2024-07-17T06:56:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f7a000640 1 -- 172.21.15.44:0/743452149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f74009d20 con 0x7f5f840ce690 2024-07-17T06:56:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f7a000640 1 -- 172.21.15.44:0/743452149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74005280 con 0x7f5f840ce690 2024-07-17T06:56:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f841f49e0 con 0x7f5f840ce690 2024-07-17T06:56:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.779+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f840aaad0 con 0x7f5f840ce690 2024-07-17T06:56:09.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.782+0000 7f5f7a000640 1 -- 172.21.15.44:0/743452149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f5f74005420 con 0x7f5f840ce690 2024-07-17T06:56:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.783+0000 7f5f7a000640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 0x7f5f64079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.783+0000 7f5f7a000640 1 -- 172.21.15.44:0/743452149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f74098600 con 0x7f5f840ce690 2024-07-17T06:56:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.784+0000 7f5f8c038740 1 Processor -- start 2024-07-17T06:56:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.784+0000 7f5f8c038740 1 -- start start 2024-07-17T06:56:09.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.784+0000 7f5f81400640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 0x7f5f64079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1a2fc0 0x55abbc1a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc17a280 0x55abbc19f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55abbc0ead60 con 0x55abbc19f790 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abbc0da260 con 0x55abbc1a2fc0 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55abbc0de2a0 con 0x55abbc17a280 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f81400640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 0x7f5f64079070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f70007fd0 tx=0x7f5f70009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f82800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f82800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39056/0 (socket says 172.21.15.44:39056) 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f82800640 1 -- 172.21.15.44:0/2494871329 learned_addr learned my addr 172.21.15.44:0/2494871329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f81e00640 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1a2fc0 0x55abbc1a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f82800640 1 -- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc17a280 msgr2=0x55abbc19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.785+0000 7f5f81400640 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc17a280 0x55abbc19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f82800640 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc17a280 0x55abbc19f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f82800640 1 -- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1a2fc0 msgr2=0x55abbc1a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f82800640 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1a2fc0 0x55abbc1a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f82800640 1 -- 172.21.15.44:0/2494871329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbbe3e850 con 0x55abbc19f790 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f82800640 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c035040 tx=0x7f5f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dddec44120e6e20 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f6ea00640 1 -- 172.21.15.44:0/2494871329 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c037050 con 0x55abbc19f790 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f6ea00640 1 -- 172.21.15.44:0/2494871329 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f7c043070 con 0x55abbc19f790 2024-07-17T06:56:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.786+0000 7f5f6ea00640 1 -- 172.21.15.44:0/2494871329 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c003d30 con 0x55abbc19f790 2024-07-17T06:56:09.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 -- 172.21.15.44:0/2494871329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 msgr2=0x55abbc19fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c035040 tx=0x7f5f7c004a40 comp rx=0 tx=0).stop 2024-07-17T06:56:09.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 -- 172.21.15.44:0/2494871329 shutdown_connections 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc19f790 0x55abbc19fc40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc17a280 0x55abbc19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2494871329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1a2fc0 0x55abbc1a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 -- 172.21.15.44:0/2494871329 >> 172.21.15.44:0/2494871329 conn(0x55abbc175460 msgr2=0x55abbc178e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 -- 172.21.15.44:0/2494871329 shutdown_connections 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.787+0000 7f5f8c038740 1 -- 172.21.15.44:0/2494871329 wait complete. 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.788+0000 7f5f8c038740 1 -- start start 2024-07-17T06:56:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.788+0000 7f5f8c038740 1 Processor -- start 2024-07-17T06:56:09.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 0x55abbc1b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55abbbe4d620 con 0x55abbc1a2fc0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abbc081f90 con 0x55abbc1cdaa0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55abbc173b60 con 0x55abbc19f790 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f8c038740 10 client.0 _ll_register_callbacks cb 0x55abbc0a7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.789+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39060/0 (socket says 172.21.15.44:39060) 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 -- 172.21.15.44:0/2924388379 learned_addr learned my addr 172.21.15.44:0/2924388379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f82800640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 0x55abbc1b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81e00640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 -- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 msgr2=0x55abbc19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 -- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 msgr2=0x55abbc1b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 0x55abbc1b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 -- 172.21.15.44:0/2924388379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbbeb56b0 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f82800640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 0x55abbc1b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f81400640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000db80 tx=0x7f5f70001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8724f8c6fe3505 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.790+0000 7f5f63400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7001f410 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55abbc0b1900 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f7001f5b0 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5f7002c050 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 10 client.5199 fetch_fsmap learned FSMap version 17 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55abbc1976f0 con 0x55abbc1a2fc0 2024-07-17T06:56:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abbc0b1900 con 0x55abbc1a2fc0 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5f7002f020 con 0x55abbc1a2fc0 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 10 client.5199 fetch_fsmap finished waiting for FSMap version 17 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 10 client.5199 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 20 client.5199 populate_metadata read hostname 'smithi044' 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc1b8960 0x55abbc199990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 4 client.5199 mds_command: new command op to 4738 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.791+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 --> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abbc0eccb0 con 0x55abbc1b8960 2024-07-17T06:56:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.792+0000 7f5f81e00640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc1b8960 0x55abbc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.792+0000 7f5f81e00640 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc1b8960 0x55abbc199990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e15dc61224d70f server_cookie=7c86545f3c030a15 in_seq=0 out_seq=0 2024-07-17T06:56:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.792+0000 7f5f63400640 10 client.5199 ms_handle_connect on v2:172.21.15.44:6834/1597326512 2024-07-17T06:56:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f63400640 1 -- 172.21.15.44:0/2924388379 <== mds.0 v2:172.21.15.44:6834/1597326512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abbc0eccb0 con 0x55abbc1b8960 2024-07-17T06:56:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f63400640 10 client.5199 handle_command_reply: tid=0 2024-07-17T06:56:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 client.5199 shutdown 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 msgr2=0x55abbc197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000db80 tx=0x7f5f70001060 comp rx=0 tx=0).stop 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 shutdown_connections 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc1b8960 0x55abbc199990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc1cdaa0 0x55abbc1b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc197120 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2924388379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f63400640 0 client.5199 ms_handle_reset on v2:172.21.15.44:6834/1597326512 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.793+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 >> 172.21.15.44:0/2924388379 conn(0x55abbc17a280 msgr2=0x55abbc172240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.794+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 shutdown_connections 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.794+0000 7f5f8c038740 1 -- 172.21.15.44:0/2924388379 wait complete. 2024-07-17T06:56:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.794+0000 7f5f8c038740 20 client.5199 trim_cache size 0 max 16384 2024-07-17T06:56:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.814+0000 7f5f8c038740 1 Processor -- start 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 -- start start 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc19f790 0x55abbc1bb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc1cdaa0 0x55abbc179010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f841ffa10 con 0x55abbc1a2fc0 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f841f6540 con 0x55abbc19f790 2024-07-17T06:56:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.815+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f841ff3c0 con 0x55abbc1cdaa0 2024-07-17T06:56:09.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39074/0 (socket says 172.21.15.44:39074) 2024-07-17T06:56:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 -- 172.21.15.44:0/2824552298 learned_addr learned my addr 172.21.15.44:0/2824552298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 -- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc1cdaa0 msgr2=0x55abbc179010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc1cdaa0 0x55abbc179010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 -- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc19f790 msgr2=0x55abbc1bb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc19f790 0x55abbc1bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.816+0000 7f5f81400640 1 -- 172.21.15.44:0/2824552298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abbc1750f0 con 0x55abbc1a2fc0 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.817+0000 7f5f81400640 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001140 tx=0x7f5f70014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc0937db1248af4 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.817+0000 7f5f6e000640 1 -- 172.21.15.44:0/2824552298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70020650 con 0x55abbc1a2fc0 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.817+0000 7f5f6e000640 1 -- 172.21.15.44:0/2824552298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f70035070 con 0x55abbc1a2fc0 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.817+0000 7f5f6e000640 1 -- 172.21.15.44:0/2824552298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70020a50 con 0x55abbc1a2fc0 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 -- 172.21.15.44:0/2824552298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 msgr2=0x55abbc1c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f70001140 tx=0x7f5f70014e40 comp rx=0 tx=0).stop 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 -- 172.21.15.44:0/2824552298 shutdown_connections 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc1cdaa0 0x55abbc179010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1a2fc0 0x55abbc1c1600 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 --2- 172.21.15.44:0/2824552298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc19f790 0x55abbc1bb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 -- 172.21.15.44:0/2824552298 >> 172.21.15.44:0/2824552298 conn(0x55abbc17a280 msgr2=0x55abbc199960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 -- 172.21.15.44:0/2824552298 shutdown_connections 2024-07-17T06:56:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.818+0000 7f5f8c038740 1 -- 172.21.15.44:0/2824552298 wait complete. 2024-07-17T06:56:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.819+0000 7f5f8c038740 1 -- start start 2024-07-17T06:56:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.819+0000 7f5f8c038740 1 Processor -- start 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc1dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc199930 0x55abbc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55abbc1b87e0 con 0x55abbc1cdaa0 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abbc1b8390 con 0x55abbc199930 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55abbc1b8e00 con 0x55abbc19f790 2024-07-17T06:56:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f8c038740 10 client.0 _ll_register_callbacks cb 0x7f5f7000d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f81400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39076/0 (socket says 172.21.15.44:39076) 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.820+0000 7f5f81400640 1 -- 172.21.15.44:0/209006757 learned_addr learned my addr 172.21.15.44:0/209006757 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81e00640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc1dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 -- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 msgr2=0x55abbc1dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc1dd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 -- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc199930 msgr2=0x55abbc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc199930 0x55abbc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 -- 172.21.15.44:0/209006757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abbc23f730 con 0x55abbc1cdaa0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f81400640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000db80 tx=0x7f5f70020af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40986be4ab466668 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f63400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7003f070 con 0x55abbc1cdaa0 2024-07-17T06:56:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f70039620 con 0x55abbc1cdaa0 2024-07-17T06:56:09.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.821+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f70035070 con 0x55abbc1cdaa0 2024-07-17T06:56:09.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5f70046050 con 0x55abbc1cdaa0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 10 client.5205 fetch_fsmap learned FSMap version 17 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55abbc1dc440 con 0x55abbc1cdaa0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f70035070 con 0x55abbc1cdaa0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5f70049020 con 0x55abbc1cdaa0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 10 client.5205 fetch_fsmap finished waiting for FSMap version 17 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 10 client.5205 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 20 client.5205 populate_metadata read hostname 'smithi044' 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc173720 0x55abbc173b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 4 client.5205 mds_command: new command op to 4738 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 --> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] -- command(tid 0: {"prefix": "status"}) -- 0x55abbc0e8750 con 0x55abbc173720 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.822+0000 7f5f81e00640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc173720 0x55abbc173b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f81e00640 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc173720 0x55abbc173b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ac3f3814283da server_cookie=beb4ce8aa5f7a5fd in_seq=0 out_seq=0 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f63400640 10 client.5205 ms_handle_connect on v2:172.21.15.44:6834/1597326512 2024-07-17T06:56:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f63400640 1 -- 172.21.15.44:0/209006757 <== mds.0 v2:172.21.15.44:6834/1597326512 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55abbc0e8750 con 0x55abbc173720 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f63400640 10 client.5205 handle_command_reply: tid=0 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 client.5205 shutdown 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 msgr2=0x55abbc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000db80 tx=0x7f5f70020af0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 shutdown_connections 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:6834/1597326512,v1:172.21.15.44:6835/1597326512] conn(0x55abbc173720 0x55abbc173b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abbc199930 0x55abbc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55abbc1cdaa0 0x55abbc19a400 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 --2- 172.21.15.44:0/209006757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55abbc19f790 0x55abbc1dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f63400640 0 client.5205 ms_handle_reset on v2:172.21.15.44:6834/1597326512 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.823+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 >> 172.21.15.44:0/209006757 conn(0x55abbc17a280 msgr2=0x55abbc1e1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 shutdown_connections 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f8c038740 1 -- 172.21.15.44:0/209006757 wait complete. 2024-07-17T06:56:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f8c038740 20 client.5205 trim_cache size 0 max 16384 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4738, 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 11.811270667000001, 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-07-17T06:56:09.827 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 39, 2024-07-17T06:56:09.828 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 39, 2024-07-17T06:56:09.828 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 19.630803985 2024-07-17T06:56:09.828 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T06:56:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 msgr2=0x7f5f64079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 0x7f5f64079070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f70007fd0 tx=0x7f5f70009040 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 msgr2=0x7f5f841f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.824+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400d7a0 tx=0x7f5f7400dc60 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 shutdown_connections 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f84161be0 0x7f5f840b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f5f64076bb0 0x7f5f64079070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f8415d130 0x7f5f840aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 --2- 172.21.15.44:0/743452149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f840ce690 0x7f5f841f4310 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 >> 172.21.15.44:0/743452149 conn(0x7f5f840b4ff0 msgr2=0x7f5f840b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 shutdown_connections 2024-07-17T06:56:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:09.825+0000 7f5f83200640 1 -- 172.21.15.44:0/743452149 wait complete. 2024-07-17T06:56:09.842 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T06:56:10.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 -- 172.21.15.44:0/2863560648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 msgr2=0x7f7ab8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:10.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 --2- 172.21.15.44:0/2863560648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ab8005210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009ed0 tx=0x7f7ab8005830 comp rx=0 tx=0).stop 2024-07-17T06:56:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad5c00640 1 -- 172.21.15.44:0/2863560648 reap_dead start 2024-07-17T06:56:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 -- 172.21.15.44:0/2863560648 shutdown_connections 2024-07-17T06:56:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 -- 172.21.15.44:0/2863560648 >> 172.21.15.44:0/2863560648 conn(0x7f7ad00cfec0 msgr2=0x7f7ad00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 -- 172.21.15.44:0/2863560648 shutdown_connections 2024-07-17T06:56:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.036+0000 7f7ad6600640 1 -- 172.21.15.44:0/2863560648 wait complete. 2024-07-17T06:56:10.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.038+0000 7f7ad6600640 1 Processor -- start 2024-07-17T06:56:10.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.038+0000 7f7ad6600640 1 -- start start 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad015c7c0 0x7f7ad00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 0x7f7ad00b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad015dd60 con 0x7f7ab8004d90 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad015dbe0 con 0x7f7ad00b0230 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad015dee0 con 0x7f7ad015c7c0 2024-07-17T06:56:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:10.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39084/0 (socket says 172.21.15.44:39084) 2024-07-17T06:56:10.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.039+0000 7f7ad5200640 1 -- 172.21.15.44:0/4224901307 learned_addr learned my addr 172.21.15.44:0/4224901307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5c00640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 0x7f7ad00b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 -- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad015c7c0 msgr2=0x7f7ad00afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad015c7c0 0x7f7ad00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 -- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 msgr2=0x7f7ad00b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 0x7f7ad00b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 -- 172.21.15.44:0/4224901307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad015c510 con 0x7f7ab8004d90 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5c00640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 0x7f7ad00b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:10.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7ad5200640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400d780 tx=0x7f7ac400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4018070 con 0x7f7ab8004d90 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.040+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ac400de30 con 0x7f7ab8004d90 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.041+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad00b06b0 con 0x7f7ab8004d90 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.041+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400b6c0 con 0x7f7ab8004d90 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.041+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad00b0c70 con 0x7f7ab8004d90 2024-07-17T06:56:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.042+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad015e470 con 0x7f7ab8004d90 2024-07-17T06:56:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.043+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f7ac401b070 con 0x7f7ab8004d90 2024-07-17T06:56:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.044+0000 7f7accc00640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 0x7f7ab4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.044+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7ac4097bc0 con 0x7f7ab8004d90 2024-07-17T06:56:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.045+0000 7f7acfe00640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 0x7f7ab4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.046+0000 7f7acfe00640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 0x7f7ab4079070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ad015fe20 tx=0x7f7ac0006d90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:10.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.048+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7ac4060810 con 0x7f7ab8004d90 2024-07-17T06:56:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.292+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ad00cdcc0 con 0x7f7ab8004d90 2024-07-17T06:56:10.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.293+0000 7f7accc00640 1 -- 172.21.15.44:0/4224901307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f7ac40690a0 con 0x7f7ab8004d90 2024-07-17T06:56:10.293 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 msgr2=0x7f7ab4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 0x7f7ab4079070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7ad015fe20 tx=0x7f7ac0006d90 comp rx=0 tx=0).stop 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 msgr2=0x7f7ad015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400d780 tx=0x7f7ac400dc40 comp rx=0 tx=0).stop 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 shutdown_connections 2024-07-17T06:56:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00b0230 0x7f7ad00b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f7ab4076bb0 0x7f7ab4079070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad015c7c0 0x7f7ad00afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 --2- 172.21.15.44:0/4224901307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab8004d90 0x7f7ad015bfd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 >> 172.21.15.44:0/4224901307 conn(0x7f7ad00cfec0 msgr2=0x7f7ad00d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.296+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 shutdown_connections 2024-07-17T06:56:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10.297+0000 7f7ad6600640 1 -- 172.21.15.44:0/4224901307 wait complete. 2024-07-17T06:56:10.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T06:56:10.314 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T06:56:10.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:10.315 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T06:56:10.360 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:56:10.404 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T06:56:10.404 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T06:56:10.404 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T06:56:10.404 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.404 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42487sec preferred_lft 42487sec 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T06:56:10.405 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T06:56:10.406 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.407 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T06:56:10.407 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T06:56:10.407 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T06:56:10.407 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T06:56:10.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:10.407 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T06:56:10.521 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T06:56:10.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:10.525 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T06:56:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T06:56:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T06:56:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T06:56:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T06:56:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T06:56:10.575 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T06:56:10.575 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T06:56:10.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T06:56:10.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T06:56:10.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T06:56:10.576 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T06:56:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T06:56:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:56:10.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:10.793 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T06:56:10.843 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T06:56:10.843 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T06:56:10.844 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T06:56:10.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:10.844 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T06:56:10.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:56:10.926 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T06:56:10.979 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T06:56:10.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T06:56:10.979 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T06:56:11.027 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T06:56:11.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:11.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:56:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:56:11.196 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:56:11.196 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:56:11.218 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:56:11.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:56:11.218 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:56:11.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:56:11.218 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T06:56:11.248 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T06:56:11.250 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T06:56:11.250 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T06:56:11.250 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T06:56:11.250 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T06:56:11.251 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T06:56:11.256 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T06:56:11.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:11.256 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T06:56:11.331 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T06:56:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T06:56:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T06:56:11.460 INFO:teuthology.orchestra.run.smithi044.stdout:client4807 2024-07-17T06:56:11.460 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T06:56:11.460 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T06:56:11.515 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T06:56:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 -- 172.21.15.44:0/2637738640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2608003a00 msgr2=0x7f2608003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 --2- 172.21.15.44:0/2637738640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2608003a00 0x7f2608003e00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f261400bb20 tx=0x7f2608005830 comp rx=0 tx=0).stop 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 -- 172.21.15.44:0/2637738640 shutdown_connections 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 --2- 172.21.15.44:0/2637738640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2608003a00 0x7f2608003e00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 -- 172.21.15.44:0/2637738640 >> 172.21.15.44:0/2637738640 conn(0x7f26240b0eb0 msgr2=0x7f26240b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 -- 172.21.15.44:0/2637738640 shutdown_connections 2024-07-17T06:56:11.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.723+0000 7f2629400640 1 -- 172.21.15.44:0/2637738640 wait complete. 2024-07-17T06:56:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.726+0000 7f2629400640 1 Processor -- start 2024-07-17T06:56:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.727+0000 7f2629400640 1 -- start start 2024-07-17T06:56:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 0x7f26240acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26240715e0 0x7f26240adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f262415c470 con 0x7f26241f75a0 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f262415c2f0 con 0x7f26240715e0 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.728+0000 7f2629400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f262415c5f0 con 0x7f2608003a00 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39106/0 (socket says 172.21.15.44:39106) 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 -- 172.21.15.44:0/928310185 learned_addr learned my addr 172.21.15.44:0/928310185 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2622800640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 0x7f26240acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2623200640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26240715e0 0x7f26240adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 -- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 msgr2=0x7f26240acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 0x7f26240acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 -- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26240715e0 msgr2=0x7f26240adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26240715e0 0x7f26240adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2621e00640 1 -- 172.21.15.44:0/928310185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26241f7b60 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.729+0000 7f2622800640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 0x7f26240acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f2621e00640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f261000d5d0 tx=0x7f261000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610014070 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f261000f990 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261000fde0 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26241f7e40 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.730+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241f40c0 con 0x7f26241f75a0 2024-07-17T06:56:11.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.731+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26241f42c0 con 0x7f26241f75a0 2024-07-17T06:56:11.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.732+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f2610017070 con 0x7f26241f75a0 2024-07-17T06:56:11.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.733+0000 7f261aa00640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 0x7f2604079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:11.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.733+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2610098770 con 0x7f26241f75a0 2024-07-17T06:56:11.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.734+0000 7f2622800640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 0x7f2604079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:11.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.735+0000 7f2622800640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 0x7f2604079070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f26140046c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:11.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.738+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f26241f42c0 con 0x7f26241f75a0 2024-07-17T06:56:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.979+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f26240cb4d0 con 0x7f2604076bb0 2024-07-17T06:56:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.983+0000 7f261aa00640 1 -- 172.21.15.44:0/928310185 <== mgr.4109 v2:172.21.15.175:6800/557887042 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f26240cb4d0 con 0x7f2604076bb0 2024-07-17T06:56:11.983 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T06:56:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T06:56:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T06:56:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T06:56:11.983 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T06:56:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.985+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 msgr2=0x7f2604079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.985+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 0x7f2604079070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f26140046c0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.985+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 msgr2=0x7f2624072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.985+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f261000d5d0 tx=0x7f261000da90 comp rx=0 tx=0).stop 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 shutdown_connections 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26240715e0 0x7f26240adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26241f75a0 0x7f2624072340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f2604076bb0 0x7f2604079070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 --2- 172.21.15.44:0/928310185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2608003a00 0x7f26240acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 >> 172.21.15.44:0/928310185 conn(0x7f26240b0eb0 msgr2=0x7f26240aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 shutdown_connections 2024-07-17T06:56:11.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:11.986+0000 7f2629400640 1 -- 172.21.15.44:0/928310185 wait complete. 2024-07-17T06:56:12.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T06:56:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 -- 172.21.15.44:0/427327103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 msgr2=0x7f96b80aef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 --2- 172.21.15.44:0/427327103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b80aef10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f96b8068860 tx=0x7f96b002a8b0 comp rx=0 tx=0).stop 2024-07-17T06:56:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 -- 172.21.15.44:0/427327103 shutdown_connections 2024-07-17T06:56:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 --2- 172.21.15.44:0/427327103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b80aef10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 --2- 172.21.15.44:0/427327103 >> v2:172.21.15.175:3300/0 conn(0x7f96b81542a0 0x7f96b8161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 -- 172.21.15.44:0/427327103 >> 172.21.15.44:0/427327103 conn(0x7f96b80b1ce0 msgr2=0x7f96b80b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 -- 172.21.15.44:0/427327103 shutdown_connections 2024-07-17T06:56:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.202+0000 7f96bd400640 1 -- 172.21.15.44:0/427327103 wait complete. 2024-07-17T06:56:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.204+0000 7f96bd400640 1 Processor -- start 2024-07-17T06:56:12.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.204+0000 7f96bd400640 1 -- start start 2024-07-17T06:56:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96b81542a0 0x7f96b81f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96b815e5f0 0x7f96b81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96b806c2d0 con 0x7f96b80aeac0 2024-07-17T06:56:12.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96b806c150 con 0x7f96b815e5f0 2024-07-17T06:56:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96bd400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f96b806c450 con 0x7f96b81542a0 2024-07-17T06:56:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.205+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39126/0 (socket says 172.21.15.44:39126) 2024-07-17T06:56:12.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 -- 172.21.15.44:0/1694218420 learned_addr learned my addr 172.21.15.44:0/1694218420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96b81542a0 0x7f96b81f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96bca00640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96b815e5f0 0x7f96b81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 -- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96b81542a0 msgr2=0x7f96b81f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96b81542a0 0x7f96b81f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 -- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96b815e5f0 msgr2=0x7f96b81f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96b815e5f0 0x7f96b81f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 -- 172.21.15.44:0/1694218420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b81f7b00 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.206+0000 7f96b7e00640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f96a800d950 tx=0x7f96a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.207+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8017070 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.207+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a8009d20 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.207+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80106c0 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.207+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b81f7de0 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.207+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81f3d40 con 0x7f96b80aeac0 2024-07-17T06:56:12.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.208+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b80aef10 con 0x7f96b80aeac0 2024-07-17T06:56:12.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.210+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 98829+0+0 (secure 0 0 0) 0x7f96a8010860 con 0x7f96b80aeac0 2024-07-17T06:56:12.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.211+0000 7f96afe00640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 0x7f9698079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.211+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f96a80988e0 con 0x7f96b80aeac0 2024-07-17T06:56:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.211+0000 7f96b7400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 0x7f9698079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.213+0000 7f96b7400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 0x7f9698079070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96b80b4830 tx=0x7f96a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.219+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f96a8061b80 con 0x7f96b80aeac0 2024-07-17T06:56:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:12.489+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f96b80b0020 con 0x7f96b80aeac0 2024-07-17T06:56:13.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.127+0000 7f96afe00640 1 -- 172.21.15.44:0/1694218420 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) ==== 92+0+0 (secure 0 0 0) 0x7f96a8061480 con 0x7f96b80aeac0 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 msgr2=0x7f9698079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 0x7f9698079070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96b80b4830 tx=0x7f96a4009040 comp rx=0 tx=0).stop 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 msgr2=0x7f96b813ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f96a800d950 tx=0x7f96a800de10 comp rx=0 tx=0).stop 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 shutdown_connections 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96b815e5f0 0x7f96b81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9698076bb0 0x7f9698079070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96b81542a0 0x7f96b81f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 --2- 172.21.15.44:0/1694218420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96b80aeac0 0x7f96b813ed70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 >> 172.21.15.44:0/1694218420 conn(0x7f96b80b1ce0 msgr2=0x7f96b80ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:13.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 shutdown_connections 2024-07-17T06:56:13.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.130+0000 7f96bd400640 1 -- 172.21.15.44:0/1694218420 wait complete. 2024-07-17T06:56:13.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 -- 172.21.15.44:0/4143860388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f00d41a0 msgr2=0x7f83f00d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 --2- 172.21.15.44:0/4143860388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f00d41a0 0x7f83f00d0580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f83f00d0e10 tx=0x7f83e802a980 comp rx=0 tx=0).stop 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 -- 172.21.15.44:0/4143860388 shutdown_connections 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 --2- 172.21.15.44:0/4143860388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f00d41a0 0x7f83f00d0580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 --2- 172.21.15.44:0/4143860388 >> v2:172.21.15.175:3300/0 conn(0x7f83f01611d0 0x7f83f00d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.338+0000 7f83f5a00640 1 -- 172.21.15.44:0/4143860388 >> 172.21.15.44:0/4143860388 conn(0x7f83f006c830 msgr2=0x7f83f006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.339+0000 7f83f5a00640 1 -- 172.21.15.44:0/4143860388 shutdown_connections 2024-07-17T06:56:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.339+0000 7f83f5a00640 1 -- 172.21.15.44:0/4143860388 wait complete. 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.340+0000 7f83f5a00640 1 Processor -- start 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.340+0000 7f83f5a00640 1 -- start start 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 0x7f83f01f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00d41a0 0x7f83f014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00d1910 con 0x7f83f01611d0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00b2690 con 0x7f83f00ceae0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.341+0000 7f83f5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00d1a90 con 0x7f83f00d41a0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39146/0 (socket says 172.21.15.44:39146) 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 -- 172.21.15.44:0/1108724865 learned_addr learned my addr 172.21.15.44:0/1108724865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83e7e00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00d41a0 0x7f83f014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83eea00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 0x7f83f01f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00d41a0 msgr2=0x7f83f014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00d41a0 0x7f83f014efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 msgr2=0x7f83f01f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 0x7f83f01f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83ef400640 1 -- 172.21.15.44:0/1108724865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f00ca4e0 con 0x7f83f01611d0 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.342+0000 7f83eea00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 0x7f83f01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:56:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83ef400640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009d60 tx=0x7f83e8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8034050 con 0x7f83f01611d0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83e8004450 con 0x7f83f01611d0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f01f5310 con 0x7f83f01611d0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8043440 con 0x7f83f01611d0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.343+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f00cb880 con 0x7f83f01611d0 2024-07-17T06:56:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.344+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f00d1f40 con 0x7f83f01611d0 2024-07-17T06:56:13.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.347+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 6) ==== 98846+0+0 (secure 0 0 0) 0x7f83e8037020 con 0x7f83f01611d0 2024-07-17T06:56:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.348+0000 7f83e6a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 0x7f83cc0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.348+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f83e80b9900 con 0x7f83f01611d0 2024-07-17T06:56:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.348+0000 7f83eea00640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 msgr2=0x7f83cc0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.348+0000 7f83eea00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 0x7f83cc0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T06:56:13.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.350+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f83e8082c30 con 0x7f83f01611d0 2024-07-17T06:56:13.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.549+0000 7f83eea00640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 msgr2=0x7f83cc0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:13.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.549+0000 7f83eea00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 0x7f83cc0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T06:56:13.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.553+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f83f00b95b0 con 0x7f83f01611d0 2024-07-17T06:56:13.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.557+0000 7f83e6a00640 1 -- 172.21.15.44:0/1108724865 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f83e8089e20 con 0x7f83f01611d0 2024-07-17T06:56:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 msgr2=0x7f83cc0790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T06:56:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 0x7f83cc0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 msgr2=0x7f83f014f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009d60 tx=0x7f83e8003f10 comp rx=0 tx=0).stop 2024-07-17T06:56:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 shutdown_connections 2024-07-17T06:56:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83f01611d0 0x7f83f014f510 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f83cc076c00 0x7f83cc0790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00d41a0 0x7f83f014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 --2- 172.21.15.44:0/1108724865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f00ceae0 0x7f83f01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.560+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 >> 172.21.15.44:0/1108724865 conn(0x7f83f006c830 msgr2=0x7f83f015cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.561+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 shutdown_connections 2024-07-17T06:56:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.561+0000 7f83f5a00640 1 -- 172.21.15.44:0/1108724865 wait complete. 2024-07-17T06:56:13.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T06:56:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 -- 172.21.15.44:0/3134094882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540702d0 msgr2=0x7fd754070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 --2- 172.21.15.44:0/3134094882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540702d0 0x7fd754070740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd74c005fd0 tx=0x7fd74c02a8b0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 -- 172.21.15.44:0/3134094882 shutdown_connections 2024-07-17T06:56:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 --2- 172.21.15.44:0/3134094882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540702d0 0x7fd754070740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 --2- 172.21.15.44:0/3134094882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540afb30 0x7fd75415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 -- 172.21.15.44:0/3134094882 >> 172.21.15.44:0/3134094882 conn(0x7fd7540b2240 msgr2=0x7fd7540b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 -- 172.21.15.44:0/3134094882 shutdown_connections 2024-07-17T06:56:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.784+0000 7fd758a00640 1 -- 172.21.15.44:0/3134094882 wait complete. 2024-07-17T06:56:13.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.786+0000 7fd758a00640 1 Processor -- start 2024-07-17T06:56:13.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.786+0000 7fd758a00640 1 -- start start 2024-07-17T06:56:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540702d0 0x7fd75413f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd754157d10 0x7fd754158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7540ce350 con 0x7fd7540afb30 2024-07-17T06:56:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd75406fdc0 con 0x7fd754157d10 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.787+0000 7fd758a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7540ce4d0 con 0x7fd7540702d0 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39174/0 (socket says 172.21.15.44:39174) 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 -- 172.21.15.44:0/1700283284 learned_addr learned my addr 172.21.15.44:0/1700283284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751e00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540702d0 0x7fd75413f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd752800640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd754157d10 0x7fd754158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540702d0 msgr2=0x7fd75413f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540702d0 0x7fd75413f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd754157d10 msgr2=0x7fd754158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd754157d10 0x7fd754158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 -- 172.21.15.44:0/1700283284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd754145de0 con 0x7fd7540afb30 2024-07-17T06:56:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.788+0000 7fd751400640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d950 tx=0x7fd74000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.789+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740017070 con 0x7fd7540afb30 2024-07-17T06:56:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.789+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd740009d20 con 0x7fd7540afb30 2024-07-17T06:56:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.789+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400106c0 con 0x7fd7540afb30 2024-07-17T06:56:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.789+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7541460c0 con 0x7fd7540afb30 2024-07-17T06:56:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.789+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd754146680 con 0x7fd7540afb30 2024-07-17T06:56:13.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.790+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7540cd730 con 0x7fd7540afb30 2024-07-17T06:56:13.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.791+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 6) ==== 98846+0+0 (secure 0 0 0) 0x7fd740010860 con 0x7fd7540afb30 2024-07-17T06:56:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.792+0000 7fd74a000640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 0x7fd7340790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.792+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd7400987f0 con 0x7fd7540afb30 2024-07-17T06:56:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.792+0000 7fd751e00640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 msgr2=0x7fd7340790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.792+0000 7fd751e00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 0x7fd7340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T06:56:13.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.796+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd7540cd730 con 0x7fd7540afb30 2024-07-17T06:56:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.993+0000 7fd751e00640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 msgr2=0x7fd7340790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:13.993+0000 7fd751e00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 0x7fd7340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T06:56:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.022+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd7540cea50 con 0x7fd7540afb30 2024-07-17T06:56:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.026+0000 7fd74a000640 1 -- 172.21.15.44:0/1700283284 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fd740061b20 con 0x7fd7540afb30 2024-07-17T06:56:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.029+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 msgr2=0x7fd7340790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.029+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 0x7fd7340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.029+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 msgr2=0x7fd754147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.029+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d950 tx=0x7fd74000de10 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 shutdown_connections 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd754157d10 0x7fd754158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7540afb30 0x7fd754147e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7fd734076c00 0x7fd7340790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 --2- 172.21.15.44:0/1700283284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7540702d0 0x7fd75413f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 >> 172.21.15.44:0/1700283284 conn(0x7fd7540b2240 msgr2=0x7fd7540b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 shutdown_connections 2024-07-17T06:56:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.030+0000 7fd758a00640 1 -- 172.21.15.44:0/1700283284 wait complete. 2024-07-17T06:56:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:14.049 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T06:56:14.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:14.102 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2024-07-17T06:56:14.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2024-07-17T06:56:14.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 -- 172.21.15.44:0/4230142153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd015a450 msgr2=0x7f9fd01503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 --2- 172.21.15.44:0/4230142153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd015a450 0x7f9fd01503c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009830 tx=0x7f9fb802a510 comp rx=0 tx=0).stop 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 -- 172.21.15.44:0/4230142153 shutdown_connections 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 --2- 172.21.15.44:0/4230142153 >> v2:172.21.15.175:3300/0 conn(0x7f9fd006d420 0x7f9fd006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 --2- 172.21.15.44:0/4230142153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd015a450 0x7f9fd01503c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 -- 172.21.15.44:0/4230142153 >> 172.21.15.44:0/4230142153 conn(0x7f9fd00cf0d0 msgr2=0x7f9fd00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 -- 172.21.15.44:0/4230142153 shutdown_connections 2024-07-17T06:56:14.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.384+0000 7f9fd5800640 1 -- 172.21.15.44:0/4230142153 wait complete. 2024-07-17T06:56:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.386+0000 7f9fd5800640 1 Processor -- start 2024-07-17T06:56:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.386+0000 7f9fd5800640 1 -- start start 2024-07-17T06:56:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd006d420 0x7f9fd014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd006f4d0 0x7f9fd014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 0x7f9fd01f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd00b3380 con 0x7f9fd015a450 2024-07-17T06:56:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd00b3200 con 0x7f9fd006f4d0 2024-07-17T06:56:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.387+0000 7f9fd5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd00b3500 con 0x7f9fd006d420 2024-07-17T06:56:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 0x7f9fd01f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd006d420 0x7f9fd014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 0x7f9fd01f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39180/0 (socket says 172.21.15.44:39180) 2024-07-17T06:56:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 -- 172.21.15.44:0/2917724594 learned_addr learned my addr 172.21.15.44:0/2917724594 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fcf400640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd006f4d0 0x7f9fd014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd006d420 msgr2=0x7f9fd014e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:14.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd006d420 0x7f9fd014e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd006f4d0 msgr2=0x7f9fd014e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd006f4d0 0x7f9fd014e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.388+0000 7f9fd4e00640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd00cb9a0 con 0x7f9fd015a450 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fd4e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 0x7f9fd01f74e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800a640 tx=0x7f9fc800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800cd40 con 0x7f9fd015a450 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc8004030 con 0x7f9fd015a450 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004440 con 0x7f9fd015a450 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd00cbc80 con 0x7f9fd015a450 2024-07-17T06:56:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.389+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd01f3c80 con 0x7f9fd015a450 2024-07-17T06:56:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.390+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd00cb730 con 0x7f9fd015a450 2024-07-17T06:56:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.391+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 6) ==== 98846+0+0 (secure 0 0 0) 0x7f9fc80035f0 con 0x7f9fd015a450 2024-07-17T06:56:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.392+0000 7f9fc7e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.393+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9fc80989b0 con 0x7f9fd015a450 2024-07-17T06:56:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.393+0000 7f9fcfe00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 msgr2=0x7f9fac0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.393+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T06:56:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.397+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9fc8061ce0 con 0x7f9fd015a450 2024-07-17T06:56:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.594+0000 7f9fcfe00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 msgr2=0x7f9fac0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.595+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T06:56:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.624+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9fd00cffe0 con 0x7f9fac076c00 2024-07-17T06:56:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.997+0000 7f9fcfe00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 msgr2=0x7f9fac0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:14.997+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T06:56:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:15.798+0000 7f9fcfe00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 msgr2=0x7f9fac0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/557887042 2024-07-17T06:56:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:15.798+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T06:56:17.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:17.349+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 7) ==== 98557+0+0 (secure 0 0 0) 0x7f9fc805d870 con 0x7f9fd015a450 2024-07-17T06:56:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:17.350+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 msgr2=0x7f9fac0790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T06:56:17.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:17.350+0000 7f9fc7e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/557887042,v1:172.21.15.175:6801/557887042] conn(0x7f9fac076c00 0x7f9fac0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:17.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.678+0000 7fa337600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.678+0000 7fa337600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.678+0000 7fa337600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.678+0000 7fa337600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.678+0000 7fa337600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.678 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.679+0000 7fa334400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.678 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.679+0000 7fa334400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.678 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.679+0000 7fa334400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.678 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.679+0000 7fa334400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:17.678 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:56:17.679+0000 7fa334400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:56:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.359+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 8) ==== 98684+0+0 (secure 0 0 0) 0x7f9fc806c4b0 con 0x7f9fd015a450 2024-07-17T06:56:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.360+0000 7f9fc7e00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9fac07f0f0 0x7f9fac0814e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.360+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9fc8068130 con 0x7f9fac07f0f0 2024-07-17T06:56:18.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.363+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9fac07f0f0 0x7f9fac0814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.366+0000 7f9fcfe00640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9fac07f0f0 0x7f9fac0814e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9fd015d6c0 tx=0x7f9fc00016b0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.581+0000 7f9fc7e00640 1 -- 172.21.15.44:0/2917724594 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9fc8068130 con 0x7f9fac07f0f0 2024-07-17T06:56:18.581 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir0 2024-07-17T06:56:18.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.584+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9fac07f0f0 msgr2=0x7f9fac0814e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.584+0000 7f9fd5800640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9fac07f0f0 0x7f9fac0814e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9fd015d6c0 tx=0x7f9fc00016b0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.584+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 msgr2=0x7f9fd01f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.584+0000 7f9fd5800640 1 --2- 172.21.15.44:0/2917724594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd015a450 0x7f9fd01f74e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800a640 tx=0x7f9fc800ef00 comp rx=0 tx=0).stop 2024-07-17T06:56:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.585+0000 7f9fd4e00640 1 -- 172.21.15.44:0/2917724594 reap_dead start 2024-07-17T06:56:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.585+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 shutdown_connections 2024-07-17T06:56:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.585+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 >> 172.21.15.44:0/2917724594 conn(0x7f9fd00cf0d0 msgr2=0x7f9fd00afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.585+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 shutdown_connections 2024-07-17T06:56:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.585+0000 7f9fd5800640 1 -- 172.21.15.44:0/2917724594 wait complete. 2024-07-17T06:56:18.607 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2024-07-17T06:56:18.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:18.607 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2024-07-17T06:56:18.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 -- 172.21.15.44:0/3282599221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c40710c0 msgr2=0x7f20c415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 --2- 172.21.15.44:0/3282599221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c40710c0 0x7f20c415c240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f20c40b76c0 tx=0x7f20b802a560 comp rx=0 tx=0).stop 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 -- 172.21.15.44:0/3282599221 shutdown_connections 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 --2- 172.21.15.44:0/3282599221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c415f180 0x7f20c40b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 --2- 172.21.15.44:0/3282599221 >> v2:172.21.15.175:3300/0 conn(0x7f20c40714a0 0x7f20c415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 --2- 172.21.15.44:0/3282599221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c40710c0 0x7f20c415c240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 -- 172.21.15.44:0/3282599221 >> 172.21.15.44:0/3282599221 conn(0x7f20c40d1350 msgr2=0x7f20c40d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 -- 172.21.15.44:0/3282599221 shutdown_connections 2024-07-17T06:56:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.868+0000 7f20cb000640 1 -- 172.21.15.44:0/3282599221 wait complete. 2024-07-17T06:56:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.870+0000 7f20cb000640 1 Processor -- start 2024-07-17T06:56:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.870+0000 7f20cb000640 1 -- start start 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40710c0 0x7f20c41586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40714a0 0x7f20c40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40b5e00 con 0x7f20c415f180 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40b54e0 con 0x7f20c40710c0 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20cb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20c40b5f80 con 0x7f20c40714a0 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20ca600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20ca600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49558/0 (socket says 172.21.15.44:49558) 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20ca600640 1 -- 172.21.15.44:0/236871515 learned_addr learned my addr 172.21.15.44:0/236871515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20c9c00640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40710c0 0x7f20c41586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.871+0000 7f20c9200640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40714a0 0x7f20c40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 -- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40714a0 msgr2=0x7f20c40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40714a0 0x7f20c40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 -- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40710c0 msgr2=0x7f20c41586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40710c0 0x7f20c41586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 -- 172.21.15.44:0/236871515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40b2890 con 0x7f20c415f180 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20ca600640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00b970 tx=0x7f20bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc019070 con 0x7f20c415f180 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20bc00cc50 con 0x7f20c415f180 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c40b2b70 con 0x7f20c415f180 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0146a0 con 0x7f20c415f180 2024-07-17T06:56:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.872+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c40b3130 con 0x7f20c415f180 2024-07-17T06:56:18.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.873+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c4149a20 con 0x7f20c415f180 2024-07-17T06:56:18.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.875+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 8) ==== 98684+0+0 (secure 0 0 0) 0x7f20bc01c070 con 0x7f20c415f180 2024-07-17T06:56:18.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.876+0000 7f20c1600640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 0x7f20a4078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:18.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.876+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f20bc098080 con 0x7f20c415f180 2024-07-17T06:56:18.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.876+0000 7f20c9c00640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 0x7f20a4078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:18.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.878+0000 7f20c9c00640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 0x7f20a4078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20b8033000 tx=0x7f20b8004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:18.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:18.881+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f20bc061530 con 0x7f20c415f180 2024-07-17T06:56:19.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.117+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f20c415ec40 con 0x7f20a4076a50 2024-07-17T06:56:19.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.299+0000 7f20c1600640 1 -- 172.21.15.44:0/236871515 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f20c415ec40 con 0x7f20a4076a50 2024-07-17T06:56:19.299 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir1 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 msgr2=0x7f20a4078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 0x7f20a4078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20b8033000 tx=0x7f20b8004730 comp rx=0 tx=0).stop 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 msgr2=0x7f20c4148e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00b970 tx=0x7f20bc00be30 comp rx=0 tx=0).stop 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 shutdown_connections 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f20a4076a50 0x7f20a4078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20c415f180 0x7f20c4148e30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40714a0 0x7f20c40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 --2- 172.21.15.44:0/236871515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c40710c0 0x7f20c41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 >> 172.21.15.44:0/236871515 conn(0x7f20c40d1350 msgr2=0x7f20c4159270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:19.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.303+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 shutdown_connections 2024-07-17T06:56:19.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.304+0000 7f20cb000640 1 -- 172.21.15.44:0/236871515 wait complete. 2024-07-17T06:56:19.325 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2024-07-17T06:56:19.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:19.326 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2024-07-17T06:56:19.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 -- 172.21.15.44:0/470092025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760150ed0 msgr2=0x7fd7600ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 --2- 172.21.15.44:0/470092025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760150ed0 0x7fd7600ac290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd754005fd0 tx=0x7fd75402a4a0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 -- 172.21.15.44:0/470092025 shutdown_connections 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 --2- 172.21.15.44:0/470092025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 0x7fd76015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 --2- 172.21.15.44:0/470092025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd760150ed0 0x7fd7600ac290 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 -- 172.21.15.44:0/470092025 >> 172.21.15.44:0/470092025 conn(0x7fd7600cf0d0 msgr2=0x7fd7600cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 -- 172.21.15.44:0/470092025 shutdown_connections 2024-07-17T06:56:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.597+0000 7fd765e00640 1 -- 172.21.15.44:0/470092025 wait complete. 2024-07-17T06:56:19.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.599+0000 7fd765e00640 1 Processor -- start 2024-07-17T06:56:19.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.599+0000 7fd765e00640 1 -- start start 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.600+0000 7fd765e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 0x7fd76009b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760150ed0 0x7fd7601f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b26d0 con 0x7fd7601512b0 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600b2550 con 0x7fd760150ed0 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7600b2850 con 0x7fd76006f4d0 2024-07-17T06:56:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49562/0 (socket says 172.21.15.44:49562) 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 -- 172.21.15.44:0/3637487802 learned_addr learned my addr 172.21.15.44:0/3637487802 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd764a00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 0x7fd76009b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd75fe00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760150ed0 0x7fd7601f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 -- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 msgr2=0x7fd76009b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 0x7fd76009b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 -- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760150ed0 msgr2=0x7fd7601f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760150ed0 0x7fd7601f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.601+0000 7fd765400640 1 -- 172.21.15.44:0/3637487802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76009a540 con 0x7fd7601512b0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd765400640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e960 tx=0x7fd75800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758004d20 con 0x7fd7601512b0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd75800cb20 con 0x7fd7601512b0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758005020 con 0x7fd7601512b0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76009acf0 con 0x7fd7601512b0 2024-07-17T06:56:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.602+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76009b2b0 con 0x7fd7601512b0 2024-07-17T06:56:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.604+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76009b4b0 con 0x7fd7601512b0 2024-07-17T06:56:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.605+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 98949+0+0 (secure 0 0 0) 0x7fd7580051c0 con 0x7fd7601512b0 2024-07-17T06:56:19.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.607+0000 7fd75cc00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 0x7fd744079240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:19.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.607+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fd758099170 con 0x7fd7601512b0 2024-07-17T06:56:19.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.607+0000 7fd764a00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 0x7fd744079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.609+0000 7fd764a00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 0x7fd744079240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd754033000 tx=0x7fd754004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.614+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd76009b4b0 con 0x7fd7601512b0 2024-07-17T06:56:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:19.851+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd7600b2d20 con 0x7fd744076d80 2024-07-17T06:56:20.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.041+0000 7fd75cc00640 1 -- 172.21.15.44:0/3637487802 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd7600b2d20 con 0x7fd744076d80 2024-07-17T06:56:20.041 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir2 2024-07-17T06:56:20.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 msgr2=0x7fd744079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 0x7fd744079240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd754033000 tx=0x7fd754004730 comp rx=0 tx=0).stop 2024-07-17T06:56:20.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 msgr2=0x7fd7601f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e960 tx=0x7fd75800ee20 comp rx=0 tx=0).stop 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 shutdown_connections 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd744076d80 0x7fd744079240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7601512b0 0x7fd7601f5830 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd760150ed0 0x7fd7601f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 --2- 172.21.15.44:0/3637487802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd76006f4d0 0x7fd76009b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.044+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 >> 172.21.15.44:0/3637487802 conn(0x7fd7600cf0d0 msgr2=0x7fd76006dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.045+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 shutdown_connections 2024-07-17T06:56:20.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.045+0000 7fd765e00640 1 -- 172.21.15.44:0/3637487802 wait complete. 2024-07-17T06:56:20.064 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2024-07-17T06:56:20.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:20.065 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2024-07-17T06:56:20.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.337+0000 7fad87a00640 1 -- 172.21.15.44:0/1408406411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad68004000 msgr2=0x7fad68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.337+0000 7fad87a00640 1 --2- 172.21.15.44:0/1408406411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad68004000 0x7fad68004800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fad8006f190 tx=0x7fad68005830 comp rx=0 tx=0).stop 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.337+0000 7fad85a00640 1 -- 172.21.15.44:0/1408406411 reap_dead start 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.338+0000 7fad87a00640 1 -- 172.21.15.44:0/1408406411 shutdown_connections 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.338+0000 7fad87a00640 1 -- 172.21.15.44:0/1408406411 >> 172.21.15.44:0/1408406411 conn(0x7fad800b0eb0 msgr2=0x7fad800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.338+0000 7fad87a00640 1 -- 172.21.15.44:0/1408406411 shutdown_connections 2024-07-17T06:56:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.338+0000 7fad87a00640 1 -- 172.21.15.44:0/1408406411 wait complete. 2024-07-17T06:56:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.340+0000 7fad87a00640 1 Processor -- start 2024-07-17T06:56:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.340+0000 7fad87a00640 1 -- start start 2024-07-17T06:56:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.341+0000 7fad87a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad68004000 0x7fad80158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad87a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad87a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad801f0f70 0x7fad801f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad87a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad8015c470 con 0x7fad80073f00 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad87a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad8015c2f0 con 0x7fad68004000 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad87a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad8015c5f0 con 0x7fad801f0f70 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49584/0 (socket says 172.21.15.44:49584) 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad7fe00640 1 -- 172.21.15.44:0/878647602 learned_addr learned my addr 172.21.15.44:0/878647602 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad85a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad801f0f70 0x7fad801f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.342+0000 7fad85000640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad68004000 0x7fad80158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 -- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad801f0f70 msgr2=0x7fad801f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad801f0f70 0x7fad801f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 -- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad68004000 msgr2=0x7fad80158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad68004000 0x7fad80158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 -- 172.21.15.44:0/878647602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801587b0 con 0x7fad80073f00 2024-07-17T06:56:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7fe00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fad6c02b0a0 tx=0x7fad6c0064a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003a00 con 0x7fad80073f00 2024-07-17T06:56:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c003c90 con 0x7fad80073f00 2024-07-17T06:56:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02e6e0 con 0x7fad80073f00 2024-07-17T06:56:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad80158a90 con 0x7fad80073f00 2024-07-17T06:56:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.343+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801f1920 con 0x7fad80073f00 2024-07-17T06:56:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.345+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad800cb3a0 con 0x7fad80073f00 2024-07-17T06:56:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.346+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 98949+0+0 (secure 0 0 0) 0x7fad6c035020 con 0x7fad80073f00 2024-07-17T06:56:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.347+0000 7fad7cc00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 0x7fad60079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.347+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fad6c0b8cc0 con 0x7fad80073f00 2024-07-17T06:56:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.348+0000 7fad85000640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 0x7fad60079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.349+0000 7fad85000640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 0x7fad60079210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad74009b30 tx=0x7fad74005f90 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:20.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.352+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fad6c0bf080 con 0x7fad80073f00 2024-07-17T06:56:20.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.592+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fad8015cdc0 con 0x7fad60076d50 2024-07-17T06:56:20.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.785+0000 7fad7cc00640 1 -- 172.21.15.44:0/878647602 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fad8015cdc0 con 0x7fad60076d50 2024-07-17T06:56:20.785 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir3 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.788+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 msgr2=0x7fad60079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.788+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 0x7fad60079210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad74009b30 tx=0x7fad74005f90 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 msgr2=0x7fad801f77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fad6c02b0a0 tx=0x7fad6c0064a0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 shutdown_connections 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fad60076d50 0x7fad60079210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad801f0f70 0x7fad801f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad80073f00 0x7fad801f77a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 --2- 172.21.15.44:0/878647602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad68004000 0x7fad80158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:20.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 >> 172.21.15.44:0/878647602 conn(0x7fad800b0eb0 msgr2=0x7fad800b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 shutdown_connections 2024-07-17T06:56:20.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:20.789+0000 7fad87a00640 1 -- 172.21.15.44:0/878647602 wait complete. 2024-07-17T06:56:20.810 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2024-07-17T06:56:20.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:20.810 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2024-07-17T06:56:20.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.071+0000 7f906be00640 1 -- 172.21.15.44:0/3485412318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c151f20 msgr2=0x7f906c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.071+0000 7f906be00640 1 --2- 172.21.15.44:0/3485412318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c151f20 0x7f906c15c140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f905c005fd0 tx=0x7f905c02a540 comp rx=0 tx=0).stop 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.071+0000 7f906be00640 1 -- 172.21.15.44:0/3485412318 shutdown_connections 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.072+0000 7f906be00640 1 --2- 172.21.15.44:0/3485412318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c151f20 0x7f906c15c140 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.072+0000 7f906be00640 1 -- 172.21.15.44:0/3485412318 >> 172.21.15.44:0/3485412318 conn(0x7f906c0cf0d0 msgr2=0x7f906c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.072+0000 7f906be00640 1 -- 172.21.15.44:0/3485412318 shutdown_connections 2024-07-17T06:56:21.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.072+0000 7f906be00640 1 -- 172.21.15.44:0/3485412318 wait complete. 2024-07-17T06:56:21.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.073+0000 7f906be00640 1 Processor -- start 2024-07-17T06:56:21.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.074+0000 7f906be00640 1 -- start start 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 0x7f906c1f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f906c151f20 0x7f906c1e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f906c06e730 con 0x7f906c0cd0b0 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f906c152510 con 0x7f906c151f20 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f906c0cb790 con 0x7f906c06f380 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49602/0 (socket says 172.21.15.44:49602) 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906a000640 1 -- 172.21.15.44:0/2653624833 learned_addr learned my addr 172.21.15.44:0/2653624833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:21.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906aa00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 0x7f906c1f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.075+0000 7f906b400640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f906c151f20 0x7f906c1e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 -- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 msgr2=0x7f906c1f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 0x7f906c1f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 -- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f906c151f20 msgr2=0x7f906c1e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f906c151f20 0x7f906c1e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 -- 172.21.15.44:0/2653624833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c14dcf0 con 0x7f906c0cd0b0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906aa00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 0x7f906c1f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906a000640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f905800c430 tx=0x7f905800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058011790 con 0x7f906c0cd0b0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9058017070 con 0x7f906c0cd0b0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c0ac010 con 0x7f906c0cd0b0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058011d00 con 0x7f906c0cd0b0 2024-07-17T06:56:21.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.076+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c0ac5b0 con 0x7f906c0cd0b0 2024-07-17T06:56:21.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.078+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c0cca10 con 0x7f906c0cd0b0 2024-07-17T06:56:21.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.079+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 9) ==== 98949+0+0 (secure 0 0 0) 0x7f905801c460 con 0x7f906c0cd0b0 2024-07-17T06:56:21.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.080+0000 7f9062a00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 0x7f904c079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.080+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f90580986d0 con 0x7f906c0cd0b0 2024-07-17T06:56:21.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.080+0000 7f906aa00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 0x7f904c079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.082+0000 7f906aa00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 0x7f904c079210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c033000 tx=0x7f905c004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:21.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.084+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f906c0cca10 con 0x7f906c0cd0b0 2024-07-17T06:56:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.318+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f906c14d410 con 0x7f904c076d50 2024-07-17T06:56:21.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.448+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f905805d380 con 0x7f906c0cd0b0 2024-07-17T06:56:21.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.523+0000 7f9062a00640 1 -- 172.21.15.44:0/2653624833 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f906c14d410 con 0x7f904c076d50 2024-07-17T06:56:21.524 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir4 2024-07-17T06:56:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 msgr2=0x7f904c079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 0x7f904c079210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c033000 tx=0x7f905c004730 comp rx=0 tx=0).stop 2024-07-17T06:56:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 msgr2=0x7f906c14ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f905800c430 tx=0x7f905800c8f0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 shutdown_connections 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f904c076d50 0x7f904c079210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f906c151f20 0x7f906c1e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f906c0cd0b0 0x7f906c14ced0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 --2- 172.21.15.44:0/2653624833 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f906c06f380 0x7f906c1f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 >> 172.21.15.44:0/2653624833 conn(0x7f906c0cf0d0 msgr2=0x7f906c0aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 shutdown_connections 2024-07-17T06:56:21.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.527+0000 7f906be00640 1 -- 172.21.15.44:0/2653624833 wait complete. 2024-07-17T06:56:21.547 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2024-07-17T06:56:21.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:21.548 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2024-07-17T06:56:21.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2024-07-17T06:56:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 -- 172.21.15.44:0/4000316473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 msgr2=0x7f938415f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 --2- 172.21.15.44:0/4000316473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f938415f4b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9384144ed0 tx=0x7f937802a560 comp rx=0 tx=0).stop 2024-07-17T06:56:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 -- 172.21.15.44:0/4000316473 shutdown_connections 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 --2- 172.21.15.44:0/4000316473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f938406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 --2- 172.21.15.44:0/4000316473 >> v2:172.21.15.175:3300/0 conn(0x7f938413e860 0x7f938406c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 --2- 172.21.15.44:0/4000316473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f938415f4b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 -- 172.21.15.44:0/4000316473 >> 172.21.15.44:0/4000316473 conn(0x7f938406d050 msgr2=0x7f938406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 -- 172.21.15.44:0/4000316473 shutdown_connections 2024-07-17T06:56:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.808+0000 7f938b600640 1 -- 172.21.15.44:0/4000316473 wait complete. 2024-07-17T06:56:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.811+0000 7f938b600640 1 Processor -- start 2024-07-17T06:56:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.811+0000 7f938b600640 1 -- start start 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938413e860 0x7f9384142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f9384142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9384140d60 con 0x7f93840afa90 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9384140be0 con 0x7f938413e860 2024-07-17T06:56:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f938b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9384140ee0 con 0x7f938415f9f0 2024-07-17T06:56:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f9388c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f9388c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49632/0 (socket says 172.21.15.44:49632) 2024-07-17T06:56:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.812+0000 7f9388c00640 1 -- 172.21.15.44:0/3094186036 learned_addr learned my addr 172.21.15.44:0/3094186036 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9389600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f9384142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9383e00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938413e860 0x7f9384142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 -- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 msgr2=0x7f9384142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f9384142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 -- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938413e860 msgr2=0x7f9384142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938413e860 0x7f9384142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 -- 172.21.15.44:0/3094186036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93841e4f10 con 0x7f93840afa90 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9389600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f9384142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.813+0000 7f9388c00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f93780039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.814+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937803f070 con 0x7f93840afa90 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.814+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9378004890 con 0x7f93840afa90 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.814+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378035030 con 0x7f93840afa90 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.814+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938414a820 con 0x7f93840afa90 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.814+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9384144250 con 0x7f93840afa90 2024-07-17T06:56:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.815+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938406d960 con 0x7f93840afa90 2024-07-17T06:56:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.816+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f9378042070 con 0x7f93840afa90 2024-07-17T06:56:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.817+0000 7f9380c00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 0x7f9364079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.817+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f93780b99e0 con 0x7f93840afa90 2024-07-17T06:56:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.818+0000 7f9383e00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 0x7f9364079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.819+0000 7f9383e00640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 0x7f9364079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93700059a0 tx=0x7f93700058f0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:21.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:21.822+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f938406d960 con 0x7f93840afa90 2024-07-17T06:56:22.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.065+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f938414b020 con 0x7f9364076db0 2024-07-17T06:56:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.263+0000 7f9380c00640 1 -- 172.21.15.44:0/3094186036 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f937000e9a0 con 0x7f9364076db0 2024-07-17T06:56:22.264 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir5 2024-07-17T06:56:22.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 msgr2=0x7f9364079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:22.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 0x7f9364079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93700059a0 tx=0x7f93700058f0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 msgr2=0x7f9384143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9378000c00 tx=0x7f93780039e0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 shutdown_connections 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9364076db0 0x7f9364079270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f938415f9f0 0x7f9384142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f938413e860 0x7f9384142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 --2- 172.21.15.44:0/3094186036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93840afa90 0x7f9384143d10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 >> 172.21.15.44:0/3094186036 conn(0x7f938406d050 msgr2=0x7f93840b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:22.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 shutdown_connections 2024-07-17T06:56:22.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.267+0000 7f938b600640 1 -- 172.21.15.44:0/3094186036 wait complete. 2024-07-17T06:56:22.291 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2024-07-17T06:56:22.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:22.292 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2024-07-17T06:56:22.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2024-07-17T06:56:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 -- 172.21.15.44:0/2438109259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 msgr2=0x7fc11c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 --2- 172.21.15.44:0/2438109259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c15d510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0cf650 tx=0x7fc11002a4d0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 -- 172.21.15.44:0/2438109259 shutdown_connections 2024-07-17T06:56:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 --2- 172.21.15.44:0/2438109259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c0ce690 0x7fc11c0cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 --2- 172.21.15.44:0/2438109259 >> v2:172.21.15.175:3300/0 conn(0x7fc11c161be0 0x7fc11c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 --2- 172.21.15.44:0/2438109259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c15d510 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 -- 172.21.15.44:0/2438109259 >> 172.21.15.44:0/2438109259 conn(0x7fc11c0b4ff0 msgr2=0x7fc11c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 -- 172.21.15.44:0/2438109259 shutdown_connections 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.566+0000 7fc122c00640 1 -- 172.21.15.44:0/2438109259 wait complete. 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.568+0000 7fc122c00640 1 Processor -- start 2024-07-17T06:56:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.568+0000 7fc122c00640 1 -- start start 2024-07-17T06:56:22.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 0x7fc11c1e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 0x7fc11c1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c06c7a0 con 0x7fc11c15d130 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c06c620 con 0x7fc11c0ce690 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c06c920 con 0x7fc11c161be0 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc120e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc120e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49646/0 (socket says 172.21.15.44:49646) 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc120e00640 1 -- 172.21.15.44:0/1778458482 learned_addr learned my addr 172.21.15.44:0/1778458482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:22.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc121800640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 0x7fc11c1e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc122200640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 0x7fc11c1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.569+0000 7fc120e00640 1 -- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 msgr2=0x7fc11c1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc120e00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 0x7fc11c1f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc120e00640 1 -- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 msgr2=0x7fc11c1e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc120e00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 0x7fc11c1e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc120e00640 1 -- 172.21.15.44:0/1778458482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c15e560 con 0x7fc11c15d130 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc122200640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 0x7fc11c1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc121800640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 0x7fc11c1e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:22.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc120e00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c8c0 tx=0x7fc10800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108004220 con 0x7fc11c15d130 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1080079b0 con 0x7fc11c15d130 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108007de0 con 0x7fc11c15d130 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c1e7340 con 0x7fc11c15d130 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.570+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c15eb00 con 0x7fc11c15d130 2024-07-17T06:56:22.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.572+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c0cfdf0 con 0x7fc11c15d130 2024-07-17T06:56:22.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.573+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7fc108005020 con 0x7fc11c15d130 2024-07-17T06:56:22.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.574+0000 7fc119600640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 0x7fc0fc079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:22.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.575+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fc108098eb0 con 0x7fc11c15d130 2024-07-17T06:56:22.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.575+0000 7fc121800640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 0x7fc0fc079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:22.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.576+0000 7fc121800640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 0x7fc0fc079270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc110033000 tx=0x7fc110004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.579+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc108014030 con 0x7fc11c15d130 2024-07-17T06:56:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:22.814+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc11c0d0e10 con 0x7fc0fc076db0 2024-07-17T06:56:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.005+0000 7fc119600640 1 -- 172.21.15.44:0/1778458482 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc11c0d0e10 con 0x7fc0fc076db0 2024-07-17T06:56:23.005 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir6 2024-07-17T06:56:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 msgr2=0x7fc0fc079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 0x7fc0fc079270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc110033000 tx=0x7fc110004730 comp rx=0 tx=0).stop 2024-07-17T06:56:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 msgr2=0x7fc11c1f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c8c0 tx=0x7fc10800cd80 comp rx=0 tx=0).stop 2024-07-17T06:56:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 shutdown_connections 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc0fc076db0 0x7fc0fc079270 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc11c161be0 0x7fc11c1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc11c15d130 0x7fc11c1f4630 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 --2- 172.21.15.44:0/1778458482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc11c0ce690 0x7fc11c1e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 >> 172.21.15.44:0/1778458482 conn(0x7fc11c0b4ff0 msgr2=0x7fc11c0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 shutdown_connections 2024-07-17T06:56:23.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.009+0000 7fc122c00640 1 -- 172.21.15.44:0/1778458482 wait complete. 2024-07-17T06:56:23.033 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2024-07-17T06:56:23.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:23.034 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2024-07-17T06:56:23.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.309+0000 7f7dd4000640 1 -- 172.21.15.44:0/3574611005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 msgr2=0x7f7dcc15c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.309+0000 7f7dd4000640 1 --2- 172.21.15.44:0/3574611005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc15c2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005fd0 tx=0x7f7dc402aa10 comp rx=0 tx=0).stop 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 -- 172.21.15.44:0/3574611005 shutdown_connections 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 --2- 172.21.15.44:0/3574611005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc15c2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 --2- 172.21.15.44:0/3574611005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 0x7f7dcc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 -- 172.21.15.44:0/3574611005 >> 172.21.15.44:0/3574611005 conn(0x7f7dcc0b0eb0 msgr2=0x7f7dcc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 -- 172.21.15.44:0/3574611005 shutdown_connections 2024-07-17T06:56:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.310+0000 7f7dd4000640 1 -- 172.21.15.44:0/3574611005 wait complete. 2024-07-17T06:56:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.312+0000 7f7dd4000640 1 Processor -- start 2024-07-17T06:56:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.312+0000 7f7dd4000640 1 -- start start 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dcc06ed30 0x7f7dcc0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 0x7f7dcc0ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc0cd750 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc0cd5d0 con 0x7f7dcc06ed30 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd4000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc0cd8d0 con 0x7f7dcc152220 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49666/0 (socket says 172.21.15.44:49666) 2024-07-17T06:56:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd0c00640 1 -- 172.21.15.44:0/1810925688 learned_addr learned my addr 172.21.15.44:0/1810925688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd1600640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dcc06ed30 0x7f7dcc0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.313+0000 7f7dd2000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 0x7f7dcc0ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 -- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 msgr2=0x7f7dcc0ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 0x7f7dcc0ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 -- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dcc06ed30 msgr2=0x7f7dcc0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dcc06ed30 0x7f7dcc0cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 -- 172.21.15.44:0/1810925688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc0cb870 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd0c00640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c430 tx=0x7f7db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8017070 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db80035f0 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8003ac0 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1f3700 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.314+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1f3f60 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.316+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc0cddf0 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.317+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f7db801a070 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.318+0000 7f7dc9600640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 0x7f7dac0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.318+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f7db8097e10 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.318+0000 7f7dd1600640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 0x7f7dac0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.319+0000 7f7dd1600640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 0x7f7dac0792a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc0005850 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.322+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7db809b030 con 0x7f7dcc0cbab0 2024-07-17T06:56:23.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.564+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7dcc1f47f0 con 0x7f7dac076de0 2024-07-17T06:56:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.759+0000 7f7dc9600640 1 -- 172.21.15.44:0/1810925688 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7dc000fad0 con 0x7f7dac076de0 2024-07-17T06:56:23.759 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir7 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.762+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 msgr2=0x7f7dac0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 0x7f7dac0792a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005fd0 tx=0x7f7dc0005850 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 msgr2=0x7f7dcc14f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c430 tx=0x7f7db800c8f0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 shutdown_connections 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7dac076de0 0x7f7dac0792a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dcc152220 0x7f7dcc0ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dcc0cbab0 0x7f7dcc14f480 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 --2- 172.21.15.44:0/1810925688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dcc06ed30 0x7f7dcc0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 >> 172.21.15.44:0/1810925688 conn(0x7f7dcc0b0eb0 msgr2=0x7f7dcc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:23.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 shutdown_connections 2024-07-17T06:56:23.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:23.763+0000 7f7dd4000640 1 -- 172.21.15.44:0/1810925688 wait complete. 2024-07-17T06:56:23.784 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2024-07-17T06:56:23.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:23.785 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2024-07-17T06:56:23.839 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2024-07-17T06:56:24.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 -- 172.21.15.44:0/2664388526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48dc003a30 msgr2=0x7f48dc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 --2- 172.21.15.44:0/2664388526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48dc003a30 0x7f48dc003e50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c780 tx=0x7f48dc005830 comp rx=0 tx=0).stop 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48fda00640 1 -- 172.21.15.44:0/2664388526 reap_dead start 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 -- 172.21.15.44:0/2664388526 shutdown_connections 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 -- 172.21.15.44:0/2664388526 >> 172.21.15.44:0/2664388526 conn(0x7f48f80b0eb0 msgr2=0x7f48f80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 -- 172.21.15.44:0/2664388526 shutdown_connections 2024-07-17T06:56:24.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.056+0000 7f48ffa00640 1 -- 172.21.15.44:0/2664388526 wait complete. 2024-07-17T06:56:24.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.058+0000 7f48ffa00640 1 Processor -- start 2024-07-17T06:56:24.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.058+0000 7f48ffa00640 1 -- start start 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 0x7f48f809ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f815d210 0x7f48f81535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48f806d9b0 con 0x7f48f80b27c0 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48f806d830 con 0x7f48f815d210 2024-07-17T06:56:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48ffa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48f815e0a0 con 0x7f48dc003a30 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.059+0000 7f48fda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49686/0 (socket says 172.21.15.44:49686) 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 -- 172.21.15.44:0/659062372 learned_addr learned my addr 172.21.15.44:0/659062372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fd000640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 0x7f48f809ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48f7e00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f815d210 0x7f48f81535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 -- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 msgr2=0x7f48f809ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 0x7f48f809ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 -- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f815d210 msgr2=0x7f48f81535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f815d210 0x7f48f81535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 -- 172.21.15.44:0/659062372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f809b630 con 0x7f48f80b27c0 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fd000640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 0x7f48f809ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.060+0000 7f48fda00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f48f80b23a0 tx=0x7f48f000e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.061+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0018070 con 0x7f48f80b27c0 2024-07-17T06:56:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.061+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f00135f0 con 0x7f48f80b27c0 2024-07-17T06:56:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.061+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f000f040 con 0x7f48f80b27c0 2024-07-17T06:56:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.061+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f809b910 con 0x7f48f80b27c0 2024-07-17T06:56:24.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.061+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f809bed0 con 0x7f48f80b27c0 2024-07-17T06:56:24.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.062+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f815e600 con 0x7f48f80b27c0 2024-07-17T06:56:24.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.063+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f48f001b070 con 0x7f48f80b27c0 2024-07-17T06:56:24.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.064+0000 7f48f4c00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 0x7f48d8079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.064+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f48f00978d0 con 0x7f48f80b27c0 2024-07-17T06:56:24.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.065+0000 7f48fd000640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 0x7f48d8079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.066+0000 7f48fd000640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 0x7f48d8079270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48e8033000 tx=0x7f48e802d070 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:24.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.068+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48f0050f50 con 0x7f48f80b27c0 2024-07-17T06:56:24.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.304+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f48f809c0a0 con 0x7f48d8076db0 2024-07-17T06:56:24.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.500+0000 7f48f4c00640 1 -- 172.21.15.44:0/659062372 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f48f809c0a0 con 0x7f48d8076db0 2024-07-17T06:56:24.500 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir8 2024-07-17T06:56:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.503+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 msgr2=0x7f48d8079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.503+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 0x7f48d8079270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48e8033000 tx=0x7f48e802d070 comp rx=0 tx=0).stop 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 msgr2=0x7f48f80b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f48f80b23a0 tx=0x7f48f000e680 comp rx=0 tx=0).stop 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 shutdown_connections 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f48d8076db0 0x7f48d8079270 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f80b27c0 0x7f48f80b2c40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f815d210 0x7f48f81535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 --2- 172.21.15.44:0/659062372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48dc003a30 0x7f48f809ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 >> 172.21.15.44:0/659062372 conn(0x7f48f80b0eb0 msgr2=0x7f48f80735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 shutdown_connections 2024-07-17T06:56:24.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.504+0000 7f48ffa00640 1 -- 172.21.15.44:0/659062372 wait complete. 2024-07-17T06:56:24.526 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2024-07-17T06:56:24.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T06:56:24.526 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2024-07-17T06:56:24.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2024-07-17T06:56:24.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 -- 172.21.15.44:0/2454238131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957406db30 msgr2=0x7f95741503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 --2- 172.21.15.44:0/2454238131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957406db30 0x7f95741503c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f956c005fd0 tx=0x7f956c02b980 comp rx=0 tx=0).stop 2024-07-17T06:56:24.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 -- 172.21.15.44:0/2454238131 shutdown_connections 2024-07-17T06:56:24.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 --2- 172.21.15.44:0/2454238131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957406db30 0x7f95741503c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 -- 172.21.15.44:0/2454238131 >> 172.21.15.44:0/2454238131 conn(0x7f95740cf0d0 msgr2=0x7f95740cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:24.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 -- 172.21.15.44:0/2454238131 shutdown_connections 2024-07-17T06:56:24.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.801+0000 7f957a000640 1 -- 172.21.15.44:0/2454238131 wait complete. 2024-07-17T06:56:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.803+0000 7f957a000640 1 Processor -- start 2024-07-17T06:56:24.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.803+0000 7f957a000640 1 -- start start 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957406f4d0 0x7f95740cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f957414e370 0x7f95741e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95740b34e0 con 0x7f957415a450 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95740b3360 con 0x7f957406f4d0 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f957a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95740ac010 con 0x7f957414e370 2024-07-17T06:56:24.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f956be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f956be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49712/0 (socket says 172.21.15.44:49712) 2024-07-17T06:56:24.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.804+0000 7f956be00640 1 -- 172.21.15.44:0/871733671 learned_addr learned my addr 172.21.15.44:0/871733671 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f9578c00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957406f4d0 0x7f95740cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 -- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f957414e370 msgr2=0x7f95741e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f957414e370 0x7f95741e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 -- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957406f4d0 msgr2=0x7f95740cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957406f4d0 0x7f95740cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 -- 172.21.15.44:0/871733671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957414eae0 con 0x7f957415a450 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f956be00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f955c00b420 tx=0x7f955c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c011830 con 0x7f957415a450 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f955c016070 con 0x7f957415a450 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9574151740 con 0x7f957415a450 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c011d60 con 0x7f957415a450 2024-07-17T06:56:24.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.805+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9574151d00 con 0x7f957415a450 2024-07-17T06:56:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.807+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95741503c0 con 0x7f957415a450 2024-07-17T06:56:24.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.808+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f955c01b460 con 0x7f957415a450 2024-07-17T06:56:24.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.809+0000 7f9571600640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 0x7f95500792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.809+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f955c098330 con 0x7f957415a450 2024-07-17T06:56:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.809+0000 7f9578c00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 0x7f95500792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:24.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.810+0000 7f9578c00640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 0x7f95500792a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9564001640 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:24.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:24.813+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f955c020020 con 0x7f957415a450 2024-07-17T06:56:25.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.048+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f95741514f0 con 0x7f9550076de0 2024-07-17T06:56:25.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.242+0000 7f9571600640 1 -- 172.21.15.44:0/871733671 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f95741514f0 con 0x7f9550076de0 2024-07-17T06:56:25.242 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/dir9 2024-07-17T06:56:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 msgr2=0x7f95500792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 0x7f95500792a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9564001640 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 msgr2=0x7f957414dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f955c00b420 tx=0x7f955c00b8e0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 shutdown_connections 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9550076de0 0x7f95500792a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f957414e370 0x7f95741e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f957415a450 0x7f957414dcc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 --2- 172.21.15.44:0/871733671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f957406f4d0 0x7f95740cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 >> 172.21.15.44:0/871733671 conn(0x7f95740cf0d0 msgr2=0x7f9574073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.245+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 shutdown_connections 2024-07-17T06:56:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.246+0000 7f957a000640 1 -- 172.21.15.44:0/871733671 wait complete. 2024-07-17T06:56:25.268 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2024-07-17T06:56:25.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2024-07-17T06:56:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 -- 172.21.15.44:0/547278272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f450406eb90 msgr2=0x7f45040ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 --2- 172.21.15.44:0/547278272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f450406eb90 0x7f45040ac3a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f44fc005fd0 tx=0x7f44fc02a5d0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 -- 172.21.15.44:0/547278272 shutdown_connections 2024-07-17T06:56:25.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 --2- 172.21.15.44:0/547278272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f450406eb90 0x7f45040ac3a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 --2- 172.21.15.44:0/547278272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406f6d0 0x7f450415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.481+0000 7f450b200640 1 -- 172.21.15.44:0/547278272 >> 172.21.15.44:0/547278272 conn(0x7f45040b0eb0 msgr2=0x7f45040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.482+0000 7f450b200640 1 -- 172.21.15.44:0/547278272 shutdown_connections 2024-07-17T06:56:25.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.482+0000 7f450b200640 1 -- 172.21.15.44:0/547278272 wait complete. 2024-07-17T06:56:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.484+0000 7f450b200640 1 Processor -- start 2024-07-17T06:56:25.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.484+0000 7f450b200640 1 -- start start 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406eb90 0x7f45040cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f450406f6d0 0x7f45040ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f45040acc70 con 0x7f45040b3280 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45040acaf0 con 0x7f450406f6d0 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f450b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f450414fd60 con 0x7f450406eb90 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f4509200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f4509200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49742/0 (socket says 172.21.15.44:49742) 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f4509200640 1 -- 172.21.15.44:0/2155238754 learned_addr learned my addr 172.21.15.44:0/2155238754 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f4503e00640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406eb90 0x7f45040cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:25.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.486+0000 7f4503400640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f450406f6d0 0x7f45040ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 -- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406eb90 msgr2=0x7f45040cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406eb90 0x7f45040cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 -- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f450406f6d0 msgr2=0x7f45040ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f450406f6d0 0x7f45040ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 -- 172.21.15.44:0/2155238754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040cbad0 con 0x7f45040b3280 2024-07-17T06:56:25.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f4509200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009230 tx=0x7f44fc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc03f070 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44fc0047a0 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc035030 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.487+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45040cd9f0 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.488+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45040cdfb0 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.489+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45040ac3a0 con 0x7f45040b3280 2024-07-17T06:56:25.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.490+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f44fc042070 con 0x7f45040b3280 2024-07-17T06:56:25.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.491+0000 7f44fbe00640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 0x7f44e0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:25.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.491+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f44fc0b9690 con 0x7f45040b3280 2024-07-17T06:56:25.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.492+0000 7f4503e00640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 0x7f44e0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:25.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.493+0000 7f4503e00640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 0x7f44e0079270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f44f4005c40 tx=0x7f44f400a780 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:25.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.495+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f44fc082540 con 0x7f45040b3280 2024-07-17T06:56:25.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.734+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f45040b4200 con 0x7f44e0076db0 2024-07-17T06:56:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.902+0000 7f44fbe00640 1 -- 172.21.15.44:0/2155238754 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f45040b4200 con 0x7f44e0076db0 2024-07-17T06:56:25.902 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir0 2024-07-17T06:56:25.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 msgr2=0x7f44e0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 0x7f44e0079270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f44f4005c40 tx=0x7f44f400a780 comp rx=0 tx=0).stop 2024-07-17T06:56:25.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 msgr2=0x7f45040cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:25.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009230 tx=0x7f44fc003820 comp rx=0 tx=0).stop 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 shutdown_connections 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f44e0076db0 0x7f44e0079270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f45040b3280 0x7f45040cd3e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f450406f6d0 0x7f45040ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 --2- 172.21.15.44:0/2155238754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450406eb90 0x7f45040cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.905+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 >> 172.21.15.44:0/2155238754 conn(0x7f45040b0eb0 msgr2=0x7f450415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.906+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 shutdown_connections 2024-07-17T06:56:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:25.906+0000 7f450b200640 1 -- 172.21.15.44:0/2155238754 wait complete. 2024-07-17T06:56:25.925 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2024-07-17T06:56:25.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2024-07-17T06:56:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 -- 172.21.15.44:0/2379049837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c40ac3a0 msgr2=0x7f59c40ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 --2- 172.21.15.44:0/2379049837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c40ac3a0 0x7f59c40ac830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005fd0 tx=0x7f59bc02b000 comp rx=0 tx=0).stop 2024-07-17T06:56:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 -- 172.21.15.44:0/2379049837 shutdown_connections 2024-07-17T06:56:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 --2- 172.21.15.44:0/2379049837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c40ac3a0 0x7f59c40ac830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 --2- 172.21.15.44:0/2379049837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c406eb90 0x7f59c415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 -- 172.21.15.44:0/2379049837 >> 172.21.15.44:0/2379049837 conn(0x7f59c40b0eb0 msgr2=0x7f59c40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 -- 172.21.15.44:0/2379049837 shutdown_connections 2024-07-17T06:56:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.123+0000 7f59c9000640 1 -- 172.21.15.44:0/2379049837 wait complete. 2024-07-17T06:56:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.125+0000 7f59c9000640 1 Processor -- start 2024-07-17T06:56:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.125+0000 7f59c9000640 1 -- start start 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c40ac3a0 0x7f59c40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c40b3280 0x7f59c40cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59c406f850 con 0x7f59c406eb90 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c406f6d0 con 0x7f59c40b3280 2024-07-17T06:56:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c9000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59c414fd60 con 0x7f59c40ac3a0 2024-07-17T06:56:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49756/0 (socket says 172.21.15.44:49756) 2024-07-17T06:56:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.126+0000 7f59c1e00640 1 -- 172.21.15.44:0/2140373583 learned_addr learned my addr 172.21.15.44:0/2140373583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 -- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c40ac3a0 msgr2=0x7f59c40cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c40ac3a0 0x7f59c40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c2800640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c40b3280 0x7f59c40cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 -- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c40b3280 msgr2=0x7f59c40cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c40b3280 0x7f59c40cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 -- 172.21.15.44:0/2140373583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4157aa0 con 0x7f59c406eb90 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c1e00640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c920 tx=0x7f59b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:26.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4007c70 con 0x7f59c406eb90 2024-07-17T06:56:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59b400f450 con 0x7f59c406eb90 2024-07-17T06:56:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b400f8a0 con 0x7f59c406eb90 2024-07-17T06:56:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.127+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c4157d80 con 0x7f59c406eb90 2024-07-17T06:56:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.128+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c4158340 con 0x7f59c406eb90 2024-07-17T06:56:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.129+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c406ff90 con 0x7f59c406eb90 2024-07-17T06:56:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.130+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f59b400fa40 con 0x7f59c406eb90 2024-07-17T06:56:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.131+0000 7f59ba000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 0x7f59a0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.131+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f59b40991f0 con 0x7f59c406eb90 2024-07-17T06:56:26.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.132+0000 7f59bbe00640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 0x7f59a0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.133+0000 7f59bbe00640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 0x7f59a0079270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac0079b0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:26.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.136+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f59c406ff90 con 0x7f59c406eb90 2024-07-17T06:56:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.378+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f59c415a5a0 con 0x7f59a0076db0 2024-07-17T06:56:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.547+0000 7f59ba000640 1 -- 172.21.15.44:0/2140373583 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f59c415a5a0 con 0x7f59a0076db0 2024-07-17T06:56:26.547 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir1 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.550+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 msgr2=0x7f59a0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.550+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 0x7f59a0079270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac0079b0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.550+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 msgr2=0x7f59c4148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.550+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c920 tx=0x7f59b400cde0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 shutdown_connections 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f59a0076db0 0x7f59a0079270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c40b3280 0x7f59c40cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c40ac3a0 0x7f59c40cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 --2- 172.21.15.44:0/2140373583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c406eb90 0x7f59c4148860 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 >> 172.21.15.44:0/2140373583 conn(0x7f59c40b0eb0 msgr2=0x7f59c415ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 shutdown_connections 2024-07-17T06:56:26.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.551+0000 7f59c9000640 1 -- 172.21.15.44:0/2140373583 wait complete. 2024-07-17T06:56:26.573 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2024-07-17T06:56:26.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.794+0000 7fd07f200640 1 -- 172.21.15.44:0/1482402852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 msgr2=0x7fd064003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.794+0000 7fd07f200640 1 --2- 172.21.15.44:0/1482402852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd064003eb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd070009e20 tx=0x7fd064005830 comp rx=0 tx=0).stop 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.794+0000 7fd07e800640 1 -- 172.21.15.44:0/1482402852 reap_dead start 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.795+0000 7fd07f200640 1 -- 172.21.15.44:0/1482402852 shutdown_connections 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.795+0000 7fd07f200640 1 -- 172.21.15.44:0/1482402852 >> 172.21.15.44:0/1482402852 conn(0x7fd0800cfec0 msgr2=0x7fd0800d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.795+0000 7fd07f200640 1 -- 172.21.15.44:0/1482402852 shutdown_connections 2024-07-17T06:56:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.795+0000 7fd07f200640 1 -- 172.21.15.44:0/1482402852 wait complete. 2024-07-17T06:56:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.796+0000 7fd07f200640 1 Processor -- start 2024-07-17T06:56:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.797+0000 7fd07f200640 1 -- start start 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08015f3e0 0x7fd08015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800aab90 0x7fd0800ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800d1220 con 0x7fd064003a70 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800b5400 con 0x7fd0800aab90 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd080159f60 con 0x7fd08015f3e0 2024-07-17T06:56:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49774/0 (socket says 172.21.15.44:49774) 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 -- 172.21.15.44:0/160340208 learned_addr learned my addr 172.21.15.44:0/160340208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07d400640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08015f3e0 0x7fd08015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 -- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08015f3e0 msgr2=0x7fd08015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07e800640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800aab90 0x7fd0800ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08015f3e0 0x7fd08015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 -- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800aab90 msgr2=0x7fd0800ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800aab90 0x7fd0800ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.798+0000 7fd07de00640 1 -- 172.21.15.44:0/160340208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0800cb6e0 con 0x7fd064003a70 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd07de00640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd070036040 tx=0x7fd0700336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070038040 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd070041070 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070033b20 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0800ca640 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.799+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0800cac00 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.801+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0800b4ec0 con 0x7fd064003a70 2024-07-17T06:56:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.802+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7fd070033cc0 con 0x7fd064003a70 2024-07-17T06:56:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.803+0000 7fd076000640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 0x7fd0600792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.804+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fd0700b8ff0 con 0x7fd064003a70 2024-07-17T06:56:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.804+0000 7fd07d400640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 0x7fd0600792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.805+0000 7fd07d400640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 0x7fd0600792a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000f60 tx=0x7fd06c009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:26.808+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd070082000 con 0x7fd064003a70 2024-07-17T06:56:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.046+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd0800b93c0 con 0x7fd060076de0 2024-07-17T06:56:27.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.213+0000 7fd076000640 1 -- 172.21.15.44:0/160340208 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd0800b93c0 con 0x7fd060076de0 2024-07-17T06:56:27.214 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir2 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 msgr2=0x7fd0600792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 0x7fd0600792a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000f60 tx=0x7fd06c009040 comp rx=0 tx=0).stop 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 msgr2=0x7fd080159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd070036040 tx=0x7fd0700336f0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 shutdown_connections 2024-07-17T06:56:27.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd060076de0 0x7fd0600792a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0800aab90 0x7fd0800ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08015f3e0 0x7fd08015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 --2- 172.21.15.44:0/160340208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd064003a70 0x7fd080159a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 >> 172.21.15.44:0/160340208 conn(0x7fd0800cfec0 msgr2=0x7fd08006d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 shutdown_connections 2024-07-17T06:56:27.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.217+0000 7fd07f200640 1 -- 172.21.15.44:0/160340208 wait complete. 2024-07-17T06:56:27.240 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2024-07-17T06:56:27.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2024-07-17T06:56:27.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 -- 172.21.15.44:0/491772329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c003a70 msgr2=0x7f678c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 --2- 172.21.15.44:0/491772329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c003a70 0x7f678c003e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f679800c610 tx=0x7f678c005830 comp rx=0 tx=0).stop 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a6800640 1 -- 172.21.15.44:0/491772329 reap_dead start 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 -- 172.21.15.44:0/491772329 shutdown_connections 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 -- 172.21.15.44:0/491772329 >> 172.21.15.44:0/491772329 conn(0x7f67a80b4ff0 msgr2=0x7f67a80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 -- 172.21.15.44:0/491772329 shutdown_connections 2024-07-17T06:56:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.455+0000 7f67a7200640 1 -- 172.21.15.44:0/491772329 wait complete. 2024-07-17T06:56:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.457+0000 7f67a7200640 1 Processor -- start 2024-07-17T06:56:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.457+0000 7f67a7200640 1 -- start start 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f678c003a70 0x7f67a80cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a80ab9f0 0x7f67a80abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67a80d15c0 con 0x7f67a81f7380 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67a80d0740 con 0x7f67a80ab9f0 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.458+0000 7f67a7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67a80d1740 con 0x7f678c003a70 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49794/0 (socket says 172.21.15.44:49794) 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 -- 172.21.15.44:0/1169505989 learned_addr learned my addr 172.21.15.44:0/1169505989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a5400640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a80ab9f0 0x7f67a80abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a5e00640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f678c003a70 0x7f67a80cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 -- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f678c003a70 msgr2=0x7f67a80cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f678c003a70 0x7f67a80cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 -- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a80ab9f0 msgr2=0x7f67a80abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a80ab9f0 0x7f67a80abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 -- 172.21.15.44:0/1169505989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a81f80d0 con 0x7f67a81f7380 2024-07-17T06:56:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.459+0000 7f67a6800640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009460 tx=0x7f67a000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.460+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0005220 con 0x7f67a81f7380 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.460+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a00054b0 con 0x7f67a81f7380 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.460+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004a00 con 0x7f67a81f7380 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.460+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a81f4070 con 0x7f67a81f7380 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.460+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81f4630 con 0x7f67a81f7380 2024-07-17T06:56:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.461+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a815d510 con 0x7f67a81f7380 2024-07-17T06:56:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.462+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f67a0004ba0 con 0x7f67a81f7380 2024-07-17T06:56:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.463+0000 7f679e000640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 0x7f67840792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.464+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f67a0098bb0 con 0x7f67a81f7380 2024-07-17T06:56:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.464+0000 7f67a5e00640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 0x7f67840792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.465+0000 7f67a5e00640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 0x7f67840792a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6798033000 tx=0x7f6798038040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.468+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f67a0061a50 con 0x7f67a81f7380 2024-07-17T06:56:27.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.709+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f67a80cd9c0 con 0x7f6784076de0 2024-07-17T06:56:27.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.881+0000 7f679e000640 1 -- 172.21.15.44:0/1169505989 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f67a80cd9c0 con 0x7f6784076de0 2024-07-17T06:56:27.881 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir3 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.884+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 msgr2=0x7f67840792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.884+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 0x7f67840792a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6798033000 tx=0x7f6798038040 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 msgr2=0x7f67a81f7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009460 tx=0x7f67a000ea80 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 shutdown_connections 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6784076de0 0x7f67840792a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a81f7380 0x7f67a81f7830 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a80ab9f0 0x7f67a80abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 --2- 172.21.15.44:0/1169505989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f678c003a70 0x7f67a80cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:27.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 >> 172.21.15.44:0/1169505989 conn(0x7f67a80b4ff0 msgr2=0x7f67a80d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:27.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 shutdown_connections 2024-07-17T06:56:27.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:27.885+0000 7f67a7200640 1 -- 172.21.15.44:0/1169505989 wait complete. 2024-07-17T06:56:27.906 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2024-07-17T06:56:27.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 -- 172.21.15.44:0/1066991841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8152220 msgr2=0x7f94a806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 --2- 172.21.15.44:0/1066991841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8152220 0x7f94a806d620 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9498009830 tx=0x7f949802a480 comp rx=0 tx=0).stop 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 -- 172.21.15.44:0/1066991841 shutdown_connections 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 --2- 172.21.15.44:0/1066991841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a80cbab0 0x7f94a815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 --2- 172.21.15.44:0/1066991841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8152220 0x7f94a806d620 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 -- 172.21.15.44:0/1066991841 >> 172.21.15.44:0/1066991841 conn(0x7f94a80b0eb0 msgr2=0x7f94a80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.116+0000 7f94ad200640 1 -- 172.21.15.44:0/1066991841 shutdown_connections 2024-07-17T06:56:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.117+0000 7f94ad200640 1 -- 172.21.15.44:0/1066991841 wait complete. 2024-07-17T06:56:28.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.118+0000 7f94ad200640 1 Processor -- start 2024-07-17T06:56:28.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.118+0000 7f94ad200640 1 -- start start 2024-07-17T06:56:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80cbab0 0x7f94a81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a8152220 0x7f94a8158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80cd750 con 0x7f94a806ed30 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80cd5d0 con 0x7f94a80cbab0 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.119+0000 7f94ad200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80cd8d0 con 0x7f94a8152220 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49810/0 (socket says 172.21.15.44:49810) 2024-07-17T06:56:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 -- 172.21.15.44:0/1093153812 learned_addr learned my addr 172.21.15.44:0/1093153812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 -- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a8152220 msgr2=0x7f94a8158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a8152220 0x7f94a8158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 -- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80cbab0 msgr2=0x7f94a81588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80cbab0 0x7f94a81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 -- 172.21.15.44:0/1093153812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a8156180 con 0x7f94a806ed30 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.120+0000 7f94a5e00640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9498000c00 tx=0x7f9498033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.121+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498031d60 con 0x7f94a806ed30 2024-07-17T06:56:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.121+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9498033e10 con 0x7f94a806ed30 2024-07-17T06:56:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.121+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498033310 con 0x7f94a806ed30 2024-07-17T06:56:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.121+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81553a0 con 0x7f94a806ed30 2024-07-17T06:56:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.121+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81583a0 con 0x7f94a806ed30 2024-07-17T06:56:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.122+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a80cddf0 con 0x7f94a806ed30 2024-07-17T06:56:28.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.123+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f94980334b0 con 0x7f94a806ed30 2024-07-17T06:56:28.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.124+0000 7f949ea00640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 0x7f94840792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.124+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f94980b9bf0 con 0x7f94a806ed30 2024-07-17T06:56:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.125+0000 7f949d400640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 0x7f94840792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.126+0000 7f949d400640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 0x7f94840792a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9490007fd0 tx=0x7f9490007900 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.129+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f949803f030 con 0x7f94a806ed30 2024-07-17T06:56:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.365+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f94a81490a0 con 0x7f9484076de0 2024-07-17T06:56:28.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.534+0000 7f949ea00640 1 -- 172.21.15.44:0/1093153812 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f94a81490a0 con 0x7f9484076de0 2024-07-17T06:56:28.534 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir4 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.537+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 msgr2=0x7f94840792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 0x7f94840792a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9490007fd0 tx=0x7f9490007900 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 msgr2=0x7f94a8071a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9498000c00 tx=0x7f9498033c20 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 shutdown_connections 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9484076de0 0x7f94840792a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a8152220 0x7f94a8158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a80cbab0 0x7f94a81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 --2- 172.21.15.44:0/1093153812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a806ed30 0x7f94a8071a90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 >> 172.21.15.44:0/1093153812 conn(0x7f94a80b0eb0 msgr2=0x7f94a80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:28.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 shutdown_connections 2024-07-17T06:56:28.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.538+0000 7f94ad200640 1 -- 172.21.15.44:0/1093153812 wait complete. 2024-07-17T06:56:28.562 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2024-07-17T06:56:28.562 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2024-07-17T06:56:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 -- 172.21.15.44:0/167101247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 msgr2=0x7f138c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 --2- 172.21.15.44:0/167101247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f138c004c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1390009b50 tx=0x7f138c005830 comp rx=0 tx=0).stop 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 -- 172.21.15.44:0/167101247 shutdown_connections 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 --2- 172.21.15.44:0/167101247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f138c004c80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 -- 172.21.15.44:0/167101247 >> 172.21.15.44:0/167101247 conn(0x7f13a80b0eb0 msgr2=0x7f13a80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 -- 172.21.15.44:0/167101247 shutdown_connections 2024-07-17T06:56:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.781+0000 7f13acc00640 1 -- 172.21.15.44:0/167101247 wait complete. 2024-07-17T06:56:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.783+0000 7f13acc00640 1 Processor -- start 2024-07-17T06:56:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.783+0000 7f13acc00640 1 -- start start 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 0x7f13a80caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a80cafa0 0x7f13a8149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13a80b36c0 con 0x7f138c004800 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13a80b3540 con 0x7f13a80c8530 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.784+0000 7f13acc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13a80ae560 con 0x7f13a80cafa0 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49830/0 (socket says 172.21.15.44:49830) 2024-07-17T06:56:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 -- 172.21.15.44:0/2292258809 learned_addr learned my addr 172.21.15.44:0/2292258809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5400640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 0x7f13a80caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a6800640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a80cafa0 0x7f13a8149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 -- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a80cafa0 msgr2=0x7f13a8149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a80cafa0 0x7f13a8149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 -- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 msgr2=0x7f13a80caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 0x7f13a80caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5e00640 1 -- 172.21.15.44:0/2292258809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a814eb40 con 0x7f138c004800 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.785+0000 7f13a5400640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 0x7f13a80caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f13a5e00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f139800d920 tx=0x7f139800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398014070 con 0x7f138c004800 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1398004190 con 0x7f138c004800 2024-07-17T06:56:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13980045e0 con 0x7f138c004800 2024-07-17T06:56:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a8072190 con 0x7f138c004800 2024-07-17T06:56:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.786+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a8072710 con 0x7f138c004800 2024-07-17T06:56:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.787+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a806a1a0 con 0x7f138c004800 2024-07-17T06:56:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.788+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f1398017070 con 0x7f138c004800 2024-07-17T06:56:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.789+0000 7f139e000640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 0x7f1384079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.789+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f1398098f50 con 0x7f138c004800 2024-07-17T06:56:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.790+0000 7f13a5400640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 0x7f1384079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:28.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.791+0000 7f13a5400640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 0x7f1384079270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1390035000 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:28.794+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f13a814eb40 con 0x7f138c004800 2024-07-17T06:56:29.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.034+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f13a80b1d00 con 0x7f1384076db0 2024-07-17T06:56:29.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.186+0000 7f139e000640 1 -- 172.21.15.44:0/2292258809 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f13a80b1d00 con 0x7f1384076db0 2024-07-17T06:56:29.187 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir5 2024-07-17T06:56:29.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 msgr2=0x7f1384079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 0x7f1384079270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1390035000 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 msgr2=0x7f13a8073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f139800d920 tx=0x7f139800dde0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 shutdown_connections 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1384076db0 0x7f1384079270 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a80cafa0 0x7f13a8149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a80c8530 0x7f13a80caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 --2- 172.21.15.44:0/2292258809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c004800 0x7f13a8073ab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 >> 172.21.15.44:0/2292258809 conn(0x7f13a80b0eb0 msgr2=0x7f13a80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.190+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 shutdown_connections 2024-07-17T06:56:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.191+0000 7f13acc00640 1 -- 172.21.15.44:0/2292258809 wait complete. 2024-07-17T06:56:29.212 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2024-07-17T06:56:29.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.428+0000 7fc249600640 1 -- 172.21.15.44:0/1285478661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 msgr2=0x7fc24406ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.428+0000 7fc249600640 1 --2- 172.21.15.44:0/1285478661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc24406ec70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc230005fd0 tx=0x7fc23002a4f0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 -- 172.21.15.44:0/1285478661 shutdown_connections 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 --2- 172.21.15.44:0/1285478661 >> v2:172.21.15.175:3300/0 conn(0x7fc244152300 0x7fc24406f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 --2- 172.21.15.44:0/1285478661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc24406ec70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 -- 172.21.15.44:0/1285478661 >> 172.21.15.44:0/1285478661 conn(0x7fc2440cf0d0 msgr2=0x7fc2440cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 -- 172.21.15.44:0/1285478661 shutdown_connections 2024-07-17T06:56:29.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.429+0000 7fc249600640 1 -- 172.21.15.44:0/1285478661 wait complete. 2024-07-17T06:56:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.430+0000 7fc249600640 1 Processor -- start 2024-07-17T06:56:29.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.431+0000 7fc249600640 1 -- start start 2024-07-17T06:56:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.431+0000 7fc249600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.431+0000 7fc249600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 0x7fc244072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc249600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc244152300 0x7fc24414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc249600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2440cb6e0 con 0x7fc2440cd0b0 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc249600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2440cb560 con 0x7fc244152300 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc249600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2440cb860 con 0x7fc244151f20 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51834/0 (socket says 172.21.15.44:51834) 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 -- 172.21.15.44:0/614422917 learned_addr learned my addr 172.21.15.44:0/614422917 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc248c00640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc244152300 0x7fc24414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243400640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 0x7fc244072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 -- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 msgr2=0x7fc244072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 0x7fc244072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 -- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc244152300 msgr2=0x7fc24414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc244152300 0x7fc24414f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243e00640 1 -- 172.21.15.44:0/614422917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc244072c80 con 0x7fc2440cd0b0 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.432+0000 7fc243400640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 0x7fc244072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:29.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc243e00640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c410 tx=0x7fc23400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:29.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234017070 con 0x7fc2440cd0b0 2024-07-17T06:56:29.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2340035f0 con 0x7fc2440cd0b0 2024-07-17T06:56:29.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234003a80 con 0x7fc2440cd0b0 2024-07-17T06:56:29.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc244072f60 con 0x7fc2440cd0b0 2024-07-17T06:56:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.433+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc244073520 con 0x7fc2440cd0b0 2024-07-17T06:56:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.434+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24415c140 con 0x7fc2440cd0b0 2024-07-17T06:56:29.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.436+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7fc234003c20 con 0x7fc2440cd0b0 2024-07-17T06:56:29.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.437+0000 7fc23be00640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 0x7fc224079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:29.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.437+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fc234097c70 con 0x7fc2440cd0b0 2024-07-17T06:56:29.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.438+0000 7fc243400640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 0x7fc224079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:29.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.439+0000 7fc243400640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 0x7fc224079270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc230033000 tx=0x7fc230004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:29.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.442+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc234060b90 con 0x7fc2440cd0b0 2024-07-17T06:56:29.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.684+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc24406f2d0 con 0x7fc224076db0 2024-07-17T06:56:29.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.843+0000 7fc23be00640 1 -- 172.21.15.44:0/614422917 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc24406f2d0 con 0x7fc224076db0 2024-07-17T06:56:29.844 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir6 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 msgr2=0x7fc224079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 0x7fc224079270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc230033000 tx=0x7fc230004730 comp rx=0 tx=0).stop 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 msgr2=0x7fc2441f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c410 tx=0x7fc23400c8d0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 shutdown_connections 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc224076db0 0x7fc224079270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc244152300 0x7fc24414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc244151f20 0x7fc244072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 --2- 172.21.15.44:0/614422917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2440cd0b0 0x7fc2441f2500 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:29.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 >> 172.21.15.44:0/614422917 conn(0x7fc2440cf0d0 msgr2=0x7fc24415a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:29.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 shutdown_connections 2024-07-17T06:56:29.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:29.847+0000 7fc249600640 1 -- 172.21.15.44:0/614422917 wait complete. 2024-07-17T06:56:29.868 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2024-07-17T06:56:29.868 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2024-07-17T06:56:30.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 -- 172.21.15.44:0/3720479435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c0acab0 msgr2=0x7f196c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 --2- 172.21.15.44:0/3720479435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c0acab0 0x7f196c15a5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f195400a940 tx=0x7f195402b840 comp rx=0 tx=0).stop 2024-07-17T06:56:30.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 -- 172.21.15.44:0/3720479435 shutdown_connections 2024-07-17T06:56:30.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 --2- 172.21.15.44:0/3720479435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c0acab0 0x7f196c15a5a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 -- 172.21.15.44:0/3720479435 >> 172.21.15.44:0/3720479435 conn(0x7f196c0b0eb0 msgr2=0x7f196c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:30.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 -- 172.21.15.44:0/3720479435 shutdown_connections 2024-07-17T06:56:30.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.074+0000 7f1973600640 1 -- 172.21.15.44:0/3720479435 wait complete. 2024-07-17T06:56:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.076+0000 7f1973600640 1 Processor -- start 2024-07-17T06:56:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.076+0000 7f1973600640 1 -- start start 2024-07-17T06:56:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f196c0acab0 0x7f196c073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c0b3280 0x7f196c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06ed10 con 0x7f196c06f6d0 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f196c06eb90 con 0x7f196c0b3280 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1973600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f196c06ee90 con 0x7f196c0acab0 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1970c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1970c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51860/0 (socket says 172.21.15.44:51860) 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.077+0000 7f1970c00640 1 -- 172.21.15.44:0/3660101436 learned_addr learned my addr 172.21.15.44:0/3660101436 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1971600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c0b3280 0x7f196c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 -- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f196c0acab0 msgr2=0x7f196c073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f196c0acab0 0x7f196c073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 -- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c0b3280 msgr2=0x7f196c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c0b3280 0x7f196c154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 -- 172.21.15.44:0/3660101436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0b08e0 con 0x7f196c06f6d0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1970c00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c460 tx=0x7f195c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c016020 con 0x7f196c06f6d0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f195c012960 con 0x7f196c06f6d0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c012db0 con 0x7f196c06f6d0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.078+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c0b0600 con 0x7f196c06f6d0 2024-07-17T06:56:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.079+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c0733f0 con 0x7f196c06f6d0 2024-07-17T06:56:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.080+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c0b3f00 con 0x7f196c06f6d0 2024-07-17T06:56:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.081+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f195c019020 con 0x7f196c06f6d0 2024-07-17T06:56:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.082+0000 7f1968c00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 0x7f1948079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.082+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f195c098470 con 0x7f196c06f6d0 2024-07-17T06:56:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.083+0000 7f196be00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 0x7f1948079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.084+0000 7f196be00640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 0x7f1948079270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1954036000 tx=0x7f19540046c0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:30.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.087+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f196c0b3f00 con 0x7f196c06f6d0 2024-07-17T06:56:30.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.332+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f196c15a5a0 con 0x7f1948076db0 2024-07-17T06:56:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.480+0000 7f1968c00640 1 -- 172.21.15.44:0/3660101436 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f196c15a5a0 con 0x7f1948076db0 2024-07-17T06:56:30.480 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir7 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.483+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 msgr2=0x7f1948079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.483+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 0x7f1948079270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1954036000 tx=0x7f19540046c0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.483+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 msgr2=0x7f196c1f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.483+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c460 tx=0x7f195c00c920 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 shutdown_connections 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f1948076db0 0x7f1948079270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f196c0b3280 0x7f196c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f196c0acab0 0x7f196c073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 --2- 172.21.15.44:0/3660101436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f196c06f6d0 0x7f196c1f7480 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 >> 172.21.15.44:0/3660101436 conn(0x7f196c0b0eb0 msgr2=0x7f196c157590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 shutdown_connections 2024-07-17T06:56:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.484+0000 7f1973600640 1 -- 172.21.15.44:0/3660101436 wait complete. 2024-07-17T06:56:30.505 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2024-07-17T06:56:30.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 -- 172.21.15.44:0/2139671359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 msgr2=0x7f005c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 --2- 172.21.15.44:0/2139671359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c06dfd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0048009830 tx=0x7f004802a520 comp rx=0 tx=0).stop 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 -- 172.21.15.44:0/2139671359 shutdown_connections 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 --2- 172.21.15.44:0/2139671359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 0x7f005c0ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 --2- 172.21.15.44:0/2139671359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c06dfd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.720+0000 7f0062e00640 1 -- 172.21.15.44:0/2139671359 >> 172.21.15.44:0/2139671359 conn(0x7f005c0cf0d0 msgr2=0x7f005c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.721+0000 7f0062e00640 1 -- 172.21.15.44:0/2139671359 shutdown_connections 2024-07-17T06:56:30.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.721+0000 7f0062e00640 1 -- 172.21.15.44:0/2139671359 wait complete. 2024-07-17T06:56:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.722+0000 7f0062e00640 1 Processor -- start 2024-07-17T06:56:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.722+0000 7f0062e00640 1 -- start start 2024-07-17T06:56:30.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.723+0000 7f0062e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 0x7f005c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 0x7f005c1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f005c0b2220 con 0x7f005c06f4f0 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f005c0b20a0 con 0x7f005c154490 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f005c0b23a0 con 0x7f005c06e510 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51874/0 (socket says 172.21.15.44:51874) 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 -- 172.21.15.44:0/841367323 learned_addr learned my addr 172.21.15.44:0/841367323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061a00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 0x7f005c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062400640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 0x7f005c1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 -- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 msgr2=0x7f005c1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 0x7f005c1f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 -- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 msgr2=0x7f005c1f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 0x7f005c1f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0061000640 1 -- 172.21.15.44:0/841367323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c1f7a80 con 0x7f005c06f4f0 2024-07-17T06:56:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.724+0000 7f0062400640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 0x7f005c1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T06:56:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0061000640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0048000c00 tx=0x7f0048032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048040070 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00480038e0 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c1f7d60 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048003d30 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.725+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1f39f0 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.726+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c0b2930 con 0x7f005c06f4f0 2024-07-17T06:56:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.728+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f0048039040 con 0x7f005c06f4f0 2024-07-17T06:56:30.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.729+0000 7f0059600640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 0x7f003c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:30.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.729+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f00480ba0a0 con 0x7f005c06f4f0 2024-07-17T06:56:30.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.730+0000 7f0061a00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 0x7f003c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:30.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.731+0000 7f0061a00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 0x7f003c079270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0050001610 tx=0x7f0050009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:30.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.734+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0048082f40 con 0x7f005c06f4f0 2024-07-17T06:56:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:30.971+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f005c0af010 con 0x7f003c076db0 2024-07-17T06:56:31.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.129+0000 7f0059600640 1 -- 172.21.15.44:0/841367323 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f005c0af010 con 0x7f003c076db0 2024-07-17T06:56:31.129 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir8 2024-07-17T06:56:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 msgr2=0x7f003c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 0x7f003c079270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0050001610 tx=0x7f0050009040 comp rx=0 tx=0).stop 2024-07-17T06:56:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 msgr2=0x7f005c09aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0048000c00 tx=0x7f0048032c10 comp rx=0 tx=0).stop 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 shutdown_connections 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f003c076db0 0x7f003c079270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f005c154490 0x7f005c1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c06f4f0 0x7f005c09aa00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 --2- 172.21.15.44:0/841367323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f005c06e510 0x7f005c1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 >> 172.21.15.44:0/841367323 conn(0x7f005c0cf0d0 msgr2=0x7f005c15e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.132+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 shutdown_connections 2024-07-17T06:56:31.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.133+0000 7f0062e00640 1 -- 172.21.15.44:0/841367323 wait complete. 2024-07-17T06:56:31.155 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2024-07-17T06:56:31.155 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.367+0000 7f4383000640 1 -- 172.21.15.44:0/3202563838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c157d10 msgr2=0x7f437c15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.367+0000 7f4383000640 1 --2- 172.21.15.44:0/3202563838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c157d10 0x7f437c15b180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f437c073bd0 tx=0x7f436c02a560 comp rx=0 tx=0).stop 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 -- 172.21.15.44:0/3202563838 shutdown_connections 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 --2- 172.21.15.44:0/3202563838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 0x7f437c070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 --2- 172.21.15.44:0/3202563838 >> v2:172.21.15.175:3300/0 conn(0x7f437c0afb30 0x7f437c06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 --2- 172.21.15.44:0/3202563838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c157d10 0x7f437c15b180 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 -- 172.21.15.44:0/3202563838 >> 172.21.15.44:0/3202563838 conn(0x7f437c0b2240 msgr2=0x7f437c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:31.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 -- 172.21.15.44:0/3202563838 shutdown_connections 2024-07-17T06:56:31.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.368+0000 7f4383000640 1 -- 172.21.15.44:0/3202563838 wait complete. 2024-07-17T06:56:31.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.370+0000 7f4383000640 1 Processor -- start 2024-07-17T06:56:31.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.370+0000 7f4383000640 1 -- start start 2024-07-17T06:56:31.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.371+0000 7f4383000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 0x7f437c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.371+0000 7f4383000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.371+0000 7f4383000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c157d10 0x7f437c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f4383000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0ce290 con 0x7f437c0afb30 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f4383000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0ce110 con 0x7f437c157d10 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f4383000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0ce410 con 0x7f437c0702d0 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:31.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51900/0 (socket says 172.21.15.44:51900) 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437b400640 1 -- 172.21.15.44:0/3521295320 learned_addr learned my addr 172.21.15.44:0/3521295320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437be00640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 0x7f437c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f4381000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c157d10 0x7f437c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437b400640 1 -- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 msgr2=0x7f437c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.372+0000 7f437b400640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 0x7f437c15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f437b400640 1 -- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c157d10 msgr2=0x7f437c14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f437b400640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c157d10 0x7f437c14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f437b400640 1 -- 172.21.15.44:0/3521295320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c15cb90 con 0x7f437c0afb30 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f437b400640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f436400c890 tx=0x7f436400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364004220 con 0x7f437c0afb30 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43640079b0 con 0x7f437c0afb30 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007de0 con 0x7f437c0afb30 2024-07-17T06:56:31.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.373+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c15ce70 con 0x7f437c0afb30 2024-07-17T06:56:31.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.374+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c15d430 con 0x7f437c0afb30 2024-07-17T06:56:31.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.375+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c06f8a0 con 0x7f437c0afb30 2024-07-17T06:56:31.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.376+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 10) ==== 99005+0+0 (secure 0 0 0) 0x7f4364005020 con 0x7f437c0afb30 2024-07-17T06:56:31.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.377+0000 7f4373e00640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 0x7f4358079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T06:56:31.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.377+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f4364098e80 con 0x7f437c0afb30 2024-07-17T06:56:31.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.378+0000 7f437be00640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 0x7f4358079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T06:56:31.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.379+0000 7f437be00640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 0x7f4358079270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f436c033000 tx=0x7f436c004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T06:56:31.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.381+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f437c06f8a0 con 0x7f437c0afb30 2024-07-17T06:56:31.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.623+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f437c070710 con 0x7f4358076db0 2024-07-17T06:56:31.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.786+0000 7f4373e00640 1 -- 172.21.15.44:0/3521295320 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f437c070710 con 0x7f4358076db0 2024-07-17T06:56:31.786 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/dir9 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 msgr2=0x7f4358079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 0x7f4358079270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f436c033000 tx=0x7f436c004730 comp rx=0 tx=0).stop 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 msgr2=0x7f437c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f436400c890 tx=0x7f436400cd50 comp rx=0 tx=0).stop 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 shutdown_connections 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4358076db0 0x7f4358079270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f437c157d10 0x7f437c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f437c0afb30 0x7f437c0b4eb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 --2- 172.21.15.44:0/3521295320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f437c0702d0 0x7f437c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T06:56:31.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.789+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 >> 172.21.15.44:0/3521295320 conn(0x7f437c0b2240 msgr2=0x7f437c0b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T06:56:31.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.790+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 shutdown_connections 2024-07-17T06:56:31.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T06:56:31.790+0000 7f4383000640 1 -- 172.21.15.44:0/3521295320 wait complete. 2024-07-17T06:56:31.812 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2024-07-17T06:56:40.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:56:40.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:57:00.154 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:57:00.154+0000 7fa31a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:57:00.154 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:57:00.154+0000 7fa31a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:57:00.154 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:57:00.154+0000 7fa31a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:57:00.154 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:57:00.154+0000 7fa31a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:57:00.155 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T06:57:00.154+0000 7fa31a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T06:57:10.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:57:10.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:57:40.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:57:40.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:58:11.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:58:11.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:58:41.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:58:41.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:59:11.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:59:11.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:59:41.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T06:59:41.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:00:11.537 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:00:11.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:00:41.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:00:41.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:01:00.379 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:01:00.379+0000 7fa31b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:01:00.379 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:01:00.379+0000 7fa31b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:01:00.379 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:01:00.379+0000 7fa31b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:01:00.379 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:01:00.379+0000 7fa31b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:01:00.379 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:01:00.379+0000 7fa31b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:01:11.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:01:11.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:01:42.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:01:42.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:02:12.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:02:12.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:02:42.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:02:42.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:03:12.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:03:12.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:03:42.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:03:42.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:04:13.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:04:13.049 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:04:43.384 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:04:43.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:05:13.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:05:13.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:05:43.641 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:05:43.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:06:13.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:06:13.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:06:44.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:06:44.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:07:14.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:07:14.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:07:21.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 -- 172.21.15.44:0/569956793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4410004d90 msgr2=0x7f4410005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 --2- 172.21.15.44:0/569956793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4410004d90 0x7f4410005210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4424005fd0 tx=0x7f4410005830 comp rx=0 tx=0).stop 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4431000640 1 -- 172.21.15.44:0/569956793 reap_dead start 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 -- 172.21.15.44:0/569956793 shutdown_connections 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 -- 172.21.15.44:0/569956793 >> 172.21.15.44:0/569956793 conn(0x7f442c0b0eb0 msgr2=0x7f442c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:22.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 -- 172.21.15.44:0/569956793 shutdown_connections 2024-07-17T07:07:22.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.031+0000 7f4433000640 1 -- 172.21.15.44:0/569956793 wait complete. 2024-07-17T07:07:22.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.033+0000 7f4433000640 1 Processor -- start 2024-07-17T07:07:22.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.033+0000 7f4433000640 1 -- start start 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4410004d90 0x7f442c149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f442c0b2cd0 0x7f442c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f442c15c470 con 0x7f442c150fb0 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f442c15c2f0 con 0x7f442c0b2cd0 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4433000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f442c15c5f0 con 0x7f4410004d90 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4423e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4423e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53712/0 (socket says 172.21.15.44:53712) 2024-07-17T07:07:22.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4423e00640 1 -- 172.21.15.44:0/3604295606 learned_addr learned my addr 172.21.15.44:0/3604295606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:22.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4431000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f442c0b2cd0 0x7f442c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f4423e00640 1 -- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4410004d90 msgr2=0x7f442c149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.034+0000 7f442be00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4410004d90 0x7f442c149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4423e00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4410004d90 0x7f442c149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4423e00640 1 -- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f442c0b2cd0 msgr2=0x7f442c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4423e00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f442c0b2cd0 0x7f442c0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4423e00640 1 -- 172.21.15.44:0/3604295606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c149b90 con 0x7f442c150fb0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4423e00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f441400c4c0 tx=0x7f441400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414016020 con 0x7f442c150fb0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44140053e0 con 0x7f442c150fb0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c073ed0 con 0x7f442c150fb0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414007280 con 0x7f442c150fb0 2024-07-17T07:07:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.035+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c152ad0 con 0x7f442c150fb0 2024-07-17T07:07:22.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.036+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c0cd2e0 con 0x7f442c150fb0 2024-07-17T07:07:22.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.038+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f4414007420 con 0x7f442c150fb0 2024-07-17T07:07:22.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.039+0000 7f4428c00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 0x7f4408079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.039+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f44140984c0 con 0x7f442c150fb0 2024-07-17T07:07:22.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.039+0000 7f442be00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 0x7f4408079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.040+0000 7f442be00640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 0x7f4408079270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f441c001860 tx=0x7f441c001190 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:22.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.044+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4414020020 con 0x7f442c150fb0 2024-07-17T07:07:22.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.276+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f442c153750 con 0x7f4408076db0 2024-07-17T07:07:22.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.485+0000 7f4428c00640 1 -- 172.21.15.44:0/3604295606 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f442c153750 con 0x7f4408076db0 2024-07-17T07:07:22.484 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.488+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 msgr2=0x7f4408079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.488+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 0x7f4408079270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f441c001860 tx=0x7f441c001190 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 msgr2=0x7f442c073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f441400c4c0 tx=0x7f441400c980 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 shutdown_connections 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4408076db0 0x7f4408079270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f442c0b2cd0 0x7f442c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442c150fb0 0x7f442c073990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 --2- 172.21.15.44:0/3604295606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4410004d90 0x7f442c149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 >> 172.21.15.44:0/3604295606 conn(0x7f442c0b0eb0 msgr2=0x7f442c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:22.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 shutdown_connections 2024-07-17T07:07:22.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.489+0000 7f4433000640 1 -- 172.21.15.44:0/3604295606 wait complete. 2024-07-17T07:07:22.507 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2024-07-17T07:07:22.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 -- 172.21.15.44:0/2007912982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f222c004120 msgr2=0x7f222c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 --2- 172.21.15.44:0/2007912982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f222c004120 0x7f222c004800 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f22300016c0 tx=0x7f222c005830 comp rx=0 tx=0).stop 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f2247200640 1 -- 172.21.15.44:0/2007912982 reap_dead start 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 -- 172.21.15.44:0/2007912982 shutdown_connections 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 -- 172.21.15.44:0/2007912982 >> 172.21.15.44:0/2007912982 conn(0x7f22480b3090 msgr2=0x7f22480b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 -- 172.21.15.44:0/2007912982 shutdown_connections 2024-07-17T07:07:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.720+0000 7f224ca00640 1 -- 172.21.15.44:0/2007912982 wait complete. 2024-07-17T07:07:22.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.722+0000 7f224ca00640 1 Processor -- start 2024-07-17T07:07:22.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.722+0000 7f224ca00640 1 -- start start 2024-07-17T07:07:22.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f222c004120 0x7f22481f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2248158290 0x7f2248158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22481661e0 con 0x7f224814b7e0 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22480b5690 con 0x7f222c004120 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f224ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2248166360 con 0x7f2248158290 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f223fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f223fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53740/0 (socket says 172.21.15.44:53740) 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.723+0000 7f223fe00640 1 -- 172.21.15.44:0/317068954 learned_addr learned my addr 172.21.15.44:0/317068954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 -- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2248158290 msgr2=0x7f2248158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2248158290 0x7f2248158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 -- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f222c004120 msgr2=0x7f22481f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f2246800640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f222c004120 0x7f22481f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f222c004120 0x7f22481f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 -- 172.21.15.44:0/317068954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22481f4130 con 0x7f224814b7e0 2024-07-17T07:07:22.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223fe00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f223002a930 tx=0x7f2230006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230038070 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2230003af0 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230033420 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.724+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224814bfa0 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.725+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224814c560 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.726+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2248158c50 con 0x7f224814b7e0 2024-07-17T07:07:22.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.727+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f2230043020 con 0x7f224814b7e0 2024-07-17T07:07:22.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.728+0000 7f223ea00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 0x7f2224079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:22.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.728+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f22300b8090 con 0x7f224814b7e0 2024-07-17T07:07:22.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.728+0000 7f2246800640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 0x7f2224079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:22.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.730+0000 7f2246800640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 0x7f2224079270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2238001410 tx=0x7f2238001e00 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:22.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.733+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2230081020 con 0x7f224814b7e0 2024-07-17T07:07:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:22.971+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f22480713c0 con 0x7f2224076db0 2024-07-17T07:07:23.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.180+0000 7f223ea00640 1 -- 172.21.15.44:0/317068954 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f22480713c0 con 0x7f2224076db0 2024-07-17T07:07:23.180 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2024-07-17T07:07:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 msgr2=0x7f2224079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 0x7f2224079270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2238001410 tx=0x7f2238001e00 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 msgr2=0x7f22481f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f223002a930 tx=0x7f2230006360 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 shutdown_connections 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2224076db0 0x7f2224079270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2248158290 0x7f2248158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f224814b7e0 0x7f22481f3bf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 --2- 172.21.15.44:0/317068954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f222c004120 0x7f22481f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.184+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 >> 172.21.15.44:0/317068954 conn(0x7f22480b3090 msgr2=0x7f2248153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.185+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 shutdown_connections 2024-07-17T07:07:23.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.185+0000 7f224ca00640 1 -- 172.21.15.44:0/317068954 wait complete. 2024-07-17T07:07:23.206 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2024-07-17T07:07:23.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2024-07-17T07:07:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 -- 172.21.15.44:0/1418034337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7015b3a0 msgr2=0x7fdd700d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1418034337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7015b3a0 0x7fdd700d0ca0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009830 tx=0x7fdd6402a510 comp rx=0 tx=0).stop 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 -- 172.21.15.44:0/1418034337 shutdown_connections 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1418034337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 0x7fdd700b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1418034337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd7015b3a0 0x7fdd700d0ca0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 -- 172.21.15.44:0/1418034337 >> 172.21.15.44:0/1418034337 conn(0x7fdd700b5560 msgr2=0x7fdd700b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 -- 172.21.15.44:0/1418034337 shutdown_connections 2024-07-17T07:07:23.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.425+0000 7fdd76c00640 1 -- 172.21.15.44:0/1418034337 wait complete. 2024-07-17T07:07:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.427+0000 7fdd76c00640 1 Processor -- start 2024-07-17T07:07:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.427+0000 7fdd76c00640 1 -- start start 2024-07-17T07:07:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd700b2210 0x7fdd70157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:23.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 0x7fdd7006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7006c650 con 0x7fdd7015b3a0 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd700af940 con 0x7fdd700b2210 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.428+0000 7fdd76c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7006c7d0 con 0x7fdd700b3040 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53756/0 (socket says 172.21.15.44:53756) 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 -- 172.21.15.44:0/1103872782 learned_addr learned my addr 172.21.15.44:0/1103872782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd75800640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd700b2210 0x7fdd70157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd74e00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 0x7fdd7006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 -- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 msgr2=0x7fdd7006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 0x7fdd7006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 -- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd700b2210 msgr2=0x7fdd70157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd700b2210 0x7fdd70157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.429+0000 7fdd76200640 1 -- 172.21.15.44:0/1103872782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd701f4c20 con 0x7fdd7015b3a0 2024-07-17T07:07:23.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd76200640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800cbb0 tx=0x7fdd6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68016020 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd6800add0 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68012680 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd701f3e60 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.430+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd701f4420 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.431+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7006cc80 con 0x7fdd7015b3a0 2024-07-17T07:07:23.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.432+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fdd68019020 con 0x7fdd7015b3a0 2024-07-17T07:07:23.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.433+0000 7fdd6d600640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 0x7fdd50079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:23.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.434+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7fdd68098fe0 con 0x7fdd7015b3a0 2024-07-17T07:07:23.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.434+0000 7fdd75800640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 0x7fdd50079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:23.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.435+0000 7fdd75800640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 0x7fdd50079270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd64034000 tx=0x7fdd64004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:23.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.438+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdd68020020 con 0x7fdd7015b3a0 2024-07-17T07:07:23.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.673+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdd70158480 con 0x7fdd50076db0 2024-07-17T07:07:23.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.888+0000 7fdd6d600640 1 -- 172.21.15.44:0/1103872782 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdd70158480 con 0x7fdd50076db0 2024-07-17T07:07:23.887 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.891+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 msgr2=0x7fdd50079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.891+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 0x7fdd50079270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd64034000 tx=0x7fdd64004730 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.891+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 msgr2=0x7fdd700700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.891+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800cbb0 tx=0x7fdd6800a430 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 shutdown_connections 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdd50076db0 0x7fdd50079270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd7015b3a0 0x7fdd700700a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd700b3040 0x7fdd7006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 --2- 172.21.15.44:0/1103872782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd700b2210 0x7fdd70157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 >> 172.21.15.44:0/1103872782 conn(0x7fdd700b5560 msgr2=0x7fdd700afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 shutdown_connections 2024-07-17T07:07:23.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:23.892+0000 7fdd76c00640 1 -- 172.21.15.44:0/1103872782 wait complete. 2024-07-17T07:07:23.914 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2024-07-17T07:07:23.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2024-07-17T07:07:24.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 -- 172.21.15.44:0/2780575380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22006eb90 msgr2=0x7fe22015a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 --2- 172.21.15.44:0/2780575380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22006eb90 0x7fe22015a5a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe210009fd0 tx=0x7fe21002a8c0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 -- 172.21.15.44:0/2780575380 shutdown_connections 2024-07-17T07:07:24.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 --2- 172.21.15.44:0/2780575380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22006eb90 0x7fe22015a5a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 -- 172.21.15.44:0/2780575380 >> 172.21.15.44:0/2780575380 conn(0x7fe2200b0eb0 msgr2=0x7fe2200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:24.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 -- 172.21.15.44:0/2780575380 shutdown_connections 2024-07-17T07:07:24.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.134+0000 7fe226600640 1 -- 172.21.15.44:0/2780575380 wait complete. 2024-07-17T07:07:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.136+0000 7fe226600640 1 Processor -- start 2024-07-17T07:07:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.136+0000 7fe226600640 1 -- start start 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22006eb90 0x7fe2201f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 0x7fe220150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe22014fd60 con 0x7fe2200b3280 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe22006f8e0 con 0x7fe22006eb90 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe226600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe22014fee0 con 0x7fe2200ac3a0 2024-07-17T07:07:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:24.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53784/0 (socket says 172.21.15.44:53784) 2024-07-17T07:07:24.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.137+0000 7fe21fe00640 1 -- 172.21.15.44:0/674315117 learned_addr learned my addr 172.21.15.44:0/674315117 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21ea00640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 0x7fe220150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21f400640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22006eb90 0x7fe2201f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 -- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 msgr2=0x7fe220150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 0x7fe220150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 -- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22006eb90 msgr2=0x7fe2201f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22006eb90 0x7fe2201f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 -- 172.21.15.44:0/674315117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe220072880 con 0x7fe2200b3280 2024-07-17T07:07:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21ea00640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 0x7fe220150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe21fe00640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe21800c4b0 tx=0x7fe21800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218016020 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe218012940 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.138+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218012d90 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.139+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe220072b60 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.139+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22014f9e0 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.140+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22015a5a0 con 0x7fe2200b3280 2024-07-17T07:07:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.141+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fe218019020 con 0x7fe2200b3280 2024-07-17T07:07:24.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.142+0000 7fe217400640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 0x7fe200079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.142+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7fe218098700 con 0x7fe2200b3280 2024-07-17T07:07:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.142+0000 7fe21f400640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 0x7fe200079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.144+0000 7fe21f400640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 0x7fe200079270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe210033000 tx=0x7fe210004ab0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.146+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe218020030 con 0x7fe2200b3280 2024-07-17T07:07:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.383+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe220072630 con 0x7fe200076db0 2024-07-17T07:07:24.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.759+0000 7fe217400640 1 -- 172.21.15.44:0/674315117 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe220072630 con 0x7fe200076db0 2024-07-17T07:07:24.758 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2024-07-17T07:07:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 msgr2=0x7fe200079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 0x7fe200079270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe210033000 tx=0x7fe210004ab0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 msgr2=0x7fe2200ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe21800c4b0 tx=0x7fe21800c970 comp rx=0 tx=0).stop 2024-07-17T07:07:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 shutdown_connections 2024-07-17T07:07:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe200076db0 0x7fe200079270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200b3280 0x7fe2200ce4f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2200ac3a0 0x7fe220150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 --2- 172.21.15.44:0/674315117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22006eb90 0x7fe2201f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 >> 172.21.15.44:0/674315117 conn(0x7fe2200b0eb0 msgr2=0x7fe220156040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 shutdown_connections 2024-07-17T07:07:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.763+0000 7fe226600640 1 -- 172.21.15.44:0/674315117 wait complete. 2024-07-17T07:07:24.782 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2024-07-17T07:07:24.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2024-07-17T07:07:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 -- 172.21.15.44:0/1285653642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaf0004800 msgr2=0x7faaf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 --2- 172.21.15.44:0/1285653642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaf0004800 0x7faaf0004c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faaf800bbb0 tx=0x7faaf0005830 comp rx=0 tx=0).stop 2024-07-17T07:07:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab0be00640 1 -- 172.21.15.44:0/1285653642 reap_dead start 2024-07-17T07:07:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 -- 172.21.15.44:0/1285653642 shutdown_connections 2024-07-17T07:07:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 -- 172.21.15.44:0/1285653642 >> 172.21.15.44:0/1285653642 conn(0x7fab0c06bf40 msgr2=0x7fab0c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 -- 172.21.15.44:0/1285653642 shutdown_connections 2024-07-17T07:07:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:24.999+0000 7fab12400640 1 -- 172.21.15.44:0/1285653642 wait complete. 2024-07-17T07:07:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.001+0000 7fab12400640 1 Processor -- start 2024-07-17T07:07:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.001+0000 7fab12400640 1 -- start start 2024-07-17T07:07:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 0x7fab0c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab0c1f3aa0 0x7fab0c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c072e90 con 0x7fab0c06ef60 2024-07-17T07:07:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c072d10 con 0x7fab0c1f3aa0 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.002+0000 7fab12400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c073010 con 0x7faaf0004800 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53804/0 (socket says 172.21.15.44:53804) 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 -- 172.21.15.44:0/1275991849 learned_addr learned my addr 172.21.15.44:0/1275991849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0b400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 0x7fab0c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 -- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 msgr2=0x7fab0c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 0x7fab0c1437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 -- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab0c1f3aa0 msgr2=0x7fab0c1f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab0c1f3aa0 0x7fab0c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0aa00640 1 -- 172.21.15.44:0/1275991849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c1514c0 con 0x7fab0c06ef60 2024-07-17T07:07:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.003+0000 7fab0b400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 0x7fab0c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab0aa00640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faaf800bb00 tx=0x7faaf8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8034040 con 0x7fab0c06ef60 2024-07-17T07:07:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf803c070 con 0x7fab0c06ef60 2024-07-17T07:07:25.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80416d0 con 0x7fab0c06ef60 2024-07-17T07:07:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c1f4890 con 0x7fab0c06ef60 2024-07-17T07:07:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.004+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c143ce0 con 0x7fab0c06ef60 2024-07-17T07:07:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.005+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c0b3110 con 0x7fab0c06ef60 2024-07-17T07:07:25.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.007+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7faaf8030420 con 0x7fab0c06ef60 2024-07-17T07:07:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.008+0000 7fab03400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 0x7faaec0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.008+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7faaf80b94c0 con 0x7fab0c06ef60 2024-07-17T07:07:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.009+0000 7fab0b400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 0x7faaec0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.010+0000 7fab0b400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 0x7faaec0792a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faafc005b60 tx=0x7faafc005cf0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.013+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faaf8082450 con 0x7fab0c06ef60 2024-07-17T07:07:25.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.256+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fab0c151b60 con 0x7faaec076de0 2024-07-17T07:07:25.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.469+0000 7fab03400640 1 -- 172.21.15.44:0/1275991849 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fab0c151b60 con 0x7faaec076de0 2024-07-17T07:07:25.468 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 msgr2=0x7faaec0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 0x7faaec0792a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faafc005b60 tx=0x7faafc005cf0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 msgr2=0x7fab0c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faaf800bb00 tx=0x7faaf8004a30 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 shutdown_connections 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faaec076de0 0x7faaec0792a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab0c1f3aa0 0x7fab0c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab0c06ef60 0x7fab0c06f3e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 --2- 172.21.15.44:0/1275991849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaf0004800 0x7fab0c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 >> 172.21.15.44:0/1275991849 conn(0x7fab0c06bf40 msgr2=0x7fab0c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 shutdown_connections 2024-07-17T07:07:25.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.474+0000 7fab12400640 1 -- 172.21.15.44:0/1275991849 wait complete. 2024-07-17T07:07:25.495 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2024-07-17T07:07:25.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2024-07-17T07:07:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 -- 172.21.15.44:0/4163850871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 msgr2=0x7fe0cc15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/4163850871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc15d510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009830 tx=0x7fe0b802a5c0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 -- 172.21.15.44:0/4163850871 shutdown_connections 2024-07-17T07:07:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/4163850871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc15d510 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 -- 172.21.15.44:0/4163850871 >> 172.21.15.44:0/4163850871 conn(0x7fe0cc0b4ff0 msgr2=0x7fe0cc0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 -- 172.21.15.44:0/4163850871 shutdown_connections 2024-07-17T07:07:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.712+0000 7fe0cbe00640 1 -- 172.21.15.44:0/4163850871 wait complete. 2024-07-17T07:07:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.714+0000 7fe0cbe00640 1 Processor -- start 2024-07-17T07:07:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.714+0000 7fe0cbe00640 1 -- start start 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 0x7fe0cc1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc15d130 0x7fe0cc1f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc168860 con 0x7fe0cc161be0 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1686e0 con 0x7fe0cc15d130 2024-07-17T07:07:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.715+0000 7fe0cbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc1689e0 con 0x7fe0cc0ce690 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53810/0 (socket says 172.21.15.44:53810) 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 -- 172.21.15.44:0/3575103891 learned_addr learned my addr 172.21.15.44:0/3575103891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0ca000640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc15d130 0x7fe0cc1f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 -- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 msgr2=0x7fe0cc1f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0caa00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 0x7fe0cc1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 0x7fe0cc1f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 -- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc15d130 msgr2=0x7fe0cc1f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc15d130 0x7fe0cc1f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0cb400640 1 -- 172.21.15.44:0/3575103891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc15cb00 con 0x7fe0cc161be0 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.716+0000 7fe0caa00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 0x7fe0cc1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0cb400640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c730 tx=0x7fe0c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4003b70 con 0x7fe0cc161be0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0c4003d10 con 0x7fe0cc161be0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c400aab0 con 0x7fe0cc161be0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc15e1d0 con 0x7fe0cc161be0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.717+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc15e790 con 0x7fe0cc161be0 2024-07-17T07:07:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.718+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0cc15d510 con 0x7fe0cc161be0 2024-07-17T07:07:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.720+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fe0c400ac50 con 0x7fe0cc161be0 2024-07-17T07:07:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.721+0000 7fe0c2a00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 0x7fe0ac079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.721+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7fe0c4098880 con 0x7fe0cc161be0 2024-07-17T07:07:25.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.721+0000 7fe0caa00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 0x7fe0ac079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:25.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.722+0000 7fe0caa00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 0x7fe0ac079270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc001220 tx=0x7fe0bc009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:25.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.725+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0c4061780 con 0x7fe0cc161be0 2024-07-17T07:07:25.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:25.968+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe0cc159920 con 0x7fe0ac076db0 2024-07-17T07:07:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.200+0000 7fe0c2a00640 1 -- 172.21.15.44:0/3575103891 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe0cc159920 con 0x7fe0ac076db0 2024-07-17T07:07:26.199 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 msgr2=0x7fe0ac079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 0x7fe0ac079270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc001220 tx=0x7fe0bc009040 comp rx=0 tx=0).stop 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 msgr2=0x7fe0cc1f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c730 tx=0x7fe0c400cbf0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 shutdown_connections 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe0ac076db0 0x7fe0ac079270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc161be0 0x7fe0cc1f7cb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc15d130 0x7fe0cc1f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 --2- 172.21.15.44:0/3575103891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc0ce690 0x7fe0cc1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 >> 172.21.15.44:0/3575103891 conn(0x7fe0cc0b4ff0 msgr2=0x7fe0cc0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 shutdown_connections 2024-07-17T07:07:26.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.204+0000 7fe0cbe00640 1 -- 172.21.15.44:0/3575103891 wait complete. 2024-07-17T07:07:26.224 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2024-07-17T07:07:26.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2024-07-17T07:07:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 -- 172.21.15.44:0/1909196786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 msgr2=0x7f6f7c0b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 --2- 172.21.15.44:0/1909196786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c0b5400 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c08c0e0 tx=0x7f6f6802a560 comp rx=0 tx=0).stop 2024-07-17T07:07:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 -- 172.21.15.44:0/1909196786 shutdown_connections 2024-07-17T07:07:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 --2- 172.21.15.44:0/1909196786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c0b5400 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 --2- 172.21.15.44:0/1909196786 >> v2:172.21.15.175:3300/0 conn(0x7f6f7c15d300 0x7f6f7c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 -- 172.21.15.44:0/1909196786 >> 172.21.15.44:0/1909196786 conn(0x7f6f7c0cfec0 msgr2=0x7f6f7c0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 -- 172.21.15.44:0/1909196786 shutdown_connections 2024-07-17T07:07:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.442+0000 7f6f82800640 1 -- 172.21.15.44:0/1909196786 wait complete. 2024-07-17T07:07:26.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.444+0000 7f6f82800640 1 Processor -- start 2024-07-17T07:07:26.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.444+0000 7f6f82800640 1 -- start start 2024-07-17T07:07:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 0x7f6f7c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 0x7f6f7c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c160fd0 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c0d2b10 con 0x7f6f7c0d1010 2024-07-17T07:07:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f82800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c06e0c0 con 0x7f6f7c15d300 2024-07-17T07:07:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.445+0000 7f6f80a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53824/0 (socket says 172.21.15.44:53824) 2024-07-17T07:07:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 -- 172.21.15.44:0/2379500497 learned_addr learned my addr 172.21.15.44:0/2379500497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f81400640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 0x7f6f7c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f81e00640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 0x7f6f7c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 -- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 msgr2=0x7f6f7c1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 0x7f6f7c1594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 -- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 msgr2=0x7f6f7c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 0x7f6f7c15e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 -- 172.21.15.44:0/2379500497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c0b1ae0 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f81e00640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 0x7f6f7c1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f81400640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 0x7f6f7c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.446+0000 7f6f80a00640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009850 tx=0x7f6f680039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.447+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6803f070 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.447+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f68004890 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.447+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68035030 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.447+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c0b1dc0 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.447+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c1f3d30 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.448+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c06e570 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.449+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f6f68042070 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.450+0000 7f6f79600640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 0x7f6f5c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.450+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f6f680b99e0 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.451+0000 7f6f81400640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 0x7f6f5c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:26.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.452+0000 7f6f81400640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 0x7f6f5c079270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ba60 tx=0x7f6f6c005cf0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:26.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.455+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6f68082870 con 0x7f6f7c0d2fa0 2024-07-17T07:07:26.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.692+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6f7c159a30 con 0x7f6f5c076db0 2024-07-17T07:07:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.908+0000 7f6f79600640 1 -- 172.21.15.44:0/2379500497 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6f6c00da80 con 0x7f6f5c076db0 2024-07-17T07:07:26.908 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2024-07-17T07:07:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 msgr2=0x7f6f5c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 0x7f6f5c079270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ba60 tx=0x7f6f6c005cf0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 msgr2=0x7f6f7c1f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009850 tx=0x7f6f680039e0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 shutdown_connections 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6f5c076db0 0x7f6f5c079270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f7c15d300 0x7f6f7c1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f7c0d2fa0 0x7f6f7c1f4250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 --2- 172.21.15.44:0/2379500497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f7c0d1010 0x7f6f7c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.912+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 >> 172.21.15.44:0/2379500497 conn(0x7f6f7c0cfec0 msgr2=0x7f6f7c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.913+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 shutdown_connections 2024-07-17T07:07:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:26.913+0000 7f6f82800640 1 -- 172.21.15.44:0/2379500497 wait complete. 2024-07-17T07:07:26.934 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2024-07-17T07:07:26.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2024-07-17T07:07:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 -- 172.21.15.44:0/54401951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71201512b0 msgr2=0x7f71200ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 --2- 172.21.15.44:0/54401951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71201512b0 0x7f71200ac290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f710c005fd0 tx=0x7f710c02a540 comp rx=0 tx=0).stop 2024-07-17T07:07:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 -- 172.21.15.44:0/54401951 shutdown_connections 2024-07-17T07:07:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 --2- 172.21.15.44:0/54401951 >> v2:172.21.15.175:3300/0 conn(0x7f712006f4d0 0x7f712015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 --2- 172.21.15.44:0/54401951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71201512b0 0x7f71200ac290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 -- 172.21.15.44:0/54401951 >> 172.21.15.44:0/54401951 conn(0x7f71200cf0d0 msgr2=0x7f71200cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 -- 172.21.15.44:0/54401951 shutdown_connections 2024-07-17T07:07:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.150+0000 7f711fe00640 1 -- 172.21.15.44:0/54401951 wait complete. 2024-07-17T07:07:27.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.152+0000 7f711fe00640 1 Processor -- start 2024-07-17T07:07:27.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.153+0000 7f711fe00640 1 -- start start 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 0x7f71201f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 0x7f71201495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71200b26d0 con 0x7f71201512b0 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71200b2550 con 0x7f7120150ed0 2024-07-17T07:07:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.154+0000 7f711fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71200b2850 con 0x7f712006f4d0 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53828/0 (socket says 172.21.15.44:53828) 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 -- 172.21.15.44:0/2581667837 learned_addr learned my addr 172.21.15.44:0/2581667837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711ea00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 0x7f71201f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711e000640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 0x7f71201495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 -- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 msgr2=0x7f71201f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 0x7f71201f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 -- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 msgr2=0x7f71201495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 0x7f71201495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711f400640 1 -- 172.21.15.44:0/2581667837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7120149b30 con 0x7f71201512b0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711ea00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 0x7f71201f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.155+0000 7f711e000640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 0x7f71201495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f711f400640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f711800e6d0 tx=0x7f711800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711800cab0 con 0x7f71201512b0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f711800cc50 con 0x7f71201512b0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005280 con 0x7f71201512b0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201f80b0 con 0x7f71201512b0 2024-07-17T07:07:27.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.156+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7120072640 con 0x7f71201512b0 2024-07-17T07:07:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.157+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120072430 con 0x7f71201512b0 2024-07-17T07:07:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.159+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f7118005420 con 0x7f71201512b0 2024-07-17T07:07:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.160+0000 7f7116a00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 0x7f7100079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.160+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f71180993e0 con 0x7f71201512b0 2024-07-17T07:07:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.161+0000 7f711ea00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 0x7f7100079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.162+0000 7f711ea00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 0x7f7100079270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7110001e40 tx=0x7f7110009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.165+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f711800ce20 con 0x7f71201512b0 2024-07-17T07:07:27.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.400+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f71200b2d00 con 0x7f7100076db0 2024-07-17T07:07:27.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.609+0000 7f7116a00640 1 -- 172.21.15.44:0/2581667837 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f71200b2d00 con 0x7f7100076db0 2024-07-17T07:07:27.608 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.612+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 msgr2=0x7f7100079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.612+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 0x7f7100079270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7110001e40 tx=0x7f7110009040 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.612+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 msgr2=0x7f71201f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.612+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f711800e6d0 tx=0x7f711800eb90 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 shutdown_connections 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7100076db0 0x7f7100079270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71201512b0 0x7f71201f75f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7120150ed0 0x7f71201495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 --2- 172.21.15.44:0/2581667837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f712006f4d0 0x7f71201f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 >> 172.21.15.44:0/2581667837 conn(0x7f71200cf0d0 msgr2=0x7f712014f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:27.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 shutdown_connections 2024-07-17T07:07:27.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.613+0000 7f711fe00640 1 -- 172.21.15.44:0/2581667837 wait complete. 2024-07-17T07:07:27.631 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2024-07-17T07:07:27.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.855+0000 7f3503e00640 1 -- 172.21.15.44:0/1442086714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 msgr2=0x7f34fc0ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.855+0000 7f3503e00640 1 --2- 172.21.15.44:0/1442086714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc0ac640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009be0 tx=0x7f34f402a920 comp rx=0 tx=0).stop 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.855+0000 7f3503e00640 1 -- 172.21.15.44:0/1442086714 shutdown_connections 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.855+0000 7f3503e00640 1 --2- 172.21.15.44:0/1442086714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc0ac640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.855+0000 7f3503e00640 1 -- 172.21.15.44:0/1442086714 >> 172.21.15.44:0/1442086714 conn(0x7f34fc0b0eb0 msgr2=0x7f34fc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.856+0000 7f3503e00640 1 -- 172.21.15.44:0/1442086714 shutdown_connections 2024-07-17T07:07:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.856+0000 7f3503e00640 1 -- 172.21.15.44:0/1442086714 wait complete. 2024-07-17T07:07:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.857+0000 7f3503e00640 1 Processor -- start 2024-07-17T07:07:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.857+0000 7f3503e00640 1 -- start start 2024-07-17T07:07:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc161120 0x7f34fc156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc15e2b0 con 0x7f34fc154600 2024-07-17T07:07:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc15dc40 con 0x7f34fc161120 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.858+0000 7f3503e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc15e430 con 0x7f34fc0ac190 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53838/0 (socket says 172.21.15.44:53838) 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 -- 172.21.15.44:0/1356167710 learned_addr learned my addr 172.21.15.44:0/1356167710 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3501e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc161120 0x7f34fc156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3501400640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 -- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 msgr2=0x7f34fc152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 -- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc161120 msgr2=0x7f34fc156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc161120 0x7f34fc156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.859+0000 7f3500a00640 1 -- 172.21.15.44:0/1356167710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc071830 con 0x7f34fc154600 2024-07-17T07:07:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f3500a00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c370 tx=0x7f34e800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:27.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8017070 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e80053c0 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003850 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc071b10 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.860+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc0720d0 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.861+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc0722d0 con 0x7f34fc154600 2024-07-17T07:07:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.862+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f34e801a070 con 0x7f34fc154600 2024-07-17T07:07:27.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.863+0000 7f34f9600640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 0x7f34dc0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:27.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.863+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f34e8097cc0 con 0x7f34fc154600 2024-07-17T07:07:27.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.864+0000 7f3501400640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 0x7f34dc0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:27.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.865+0000 7f3501400640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 0x7f34dc0792a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34f00046b0 tx=0x7f34f0009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:27.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:27.868+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34fc0722d0 con 0x7f34fc154600 2024-07-17T07:07:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.110+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f34fc0715e0 con 0x7f34dc076de0 2024-07-17T07:07:28.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.325+0000 7f34f9600640 1 -- 172.21.15.44:0/1356167710 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f34fc0715e0 con 0x7f34dc076de0 2024-07-17T07:07:28.324 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2024-07-17T07:07:28.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 msgr2=0x7f34dc0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:28.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 0x7f34dc0792a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34f00046b0 tx=0x7f34f0009040 comp rx=0 tx=0).stop 2024-07-17T07:07:28.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 msgr2=0x7f34fc15ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:28.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c370 tx=0x7f34e800c830 comp rx=0 tx=0).stop 2024-07-17T07:07:28.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 shutdown_connections 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f34dc076de0 0x7f34dc0792a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34fc161120 0x7f34fc156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34fc154600 0x7f34fc15ad60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 --2- 172.21.15.44:0/1356167710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34fc0ac190 0x7f34fc152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 >> 172.21.15.44:0/1356167710 conn(0x7f34fc0b0eb0 msgr2=0x7f34fc06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.328+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 shutdown_connections 2024-07-17T07:07:28.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.329+0000 7f3503e00640 1 -- 172.21.15.44:0/1356167710 wait complete. 2024-07-17T07:07:28.349 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2024-07-17T07:07:28.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 -- 172.21.15.44:0/3594518496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaf4003ac0 msgr2=0x7fdaf4003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/3594518496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaf4003ac0 0x7fdaf4003ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800bb20 tx=0x7fdaf4005830 comp rx=0 tx=0).stop 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 -- 172.21.15.44:0/3594518496 shutdown_connections 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/3594518496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaf4003ac0 0x7fdaf4003ee0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 -- 172.21.15.44:0/3594518496 >> 172.21.15.44:0/3594518496 conn(0x7fdb100cfec0 msgr2=0x7fdb100d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 -- 172.21.15.44:0/3594518496 shutdown_connections 2024-07-17T07:07:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.573+0000 7fdb0fe00640 1 -- 172.21.15.44:0/3594518496 wait complete. 2024-07-17T07:07:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.575+0000 7fdb0fe00640 1 Processor -- start 2024-07-17T07:07:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.575+0000 7fdb0fe00640 1 -- start start 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaf4003ac0 0x7fdb101f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb100c8720 0x7fdb100c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb100d1190 con 0x7fdb10152a90 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb100d1010 con 0x7fdb100c8720 2024-07-17T07:07:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb100d1310 con 0x7fdaf4003ac0 2024-07-17T07:07:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53860/0 (socket says 172.21.15.44:53860) 2024-07-17T07:07:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0e000640 1 -- 172.21.15.44:0/561016538 learned_addr learned my addr 172.21.15.44:0/561016538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.576+0000 7fdb0ea00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaf4003ac0 0x7fdb101f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0f400640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb100c8720 0x7fdb100c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 -- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaf4003ac0 msgr2=0x7fdb101f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaf4003ac0 0x7fdb101f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 -- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb100c8720 msgr2=0x7fdb100c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb100c8720 0x7fdb100c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 -- 172.21.15.44:0/561016538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb101543d0 con 0x7fdb10152a90 2024-07-17T07:07:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0e000640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c8c0 tx=0x7fdb0000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00017070 con 0x7fdb10152a90 2024-07-17T07:07:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb00005070 con 0x7fdb10152a90 2024-07-17T07:07:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1015e920 con 0x7fdb10152a90 2024-07-17T07:07:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.577+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb000054c0 con 0x7fdb10152a90 2024-07-17T07:07:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.578+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1015eee0 con 0x7fdb10152a90 2024-07-17T07:07:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.579+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1015f0e0 con 0x7fdb10152a90 2024-07-17T07:07:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.580+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fdb0001a070 con 0x7fdb10152a90 2024-07-17T07:07:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.581+0000 7fdb06a00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 0x7fdaec0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.581+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7fdb00097fe0 con 0x7fdb10152a90 2024-07-17T07:07:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.582+0000 7fdb0ea00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 0x7fdaec0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.583+0000 7fdb0ea00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 0x7fdaec0792a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8036000 tx=0x7fdaf80046c0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:28.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.585+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdb1015f0e0 con 0x7fdb10152a90 2024-07-17T07:07:28.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:28.826+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdb100b51d0 con 0x7fdaec076de0 2024-07-17T07:07:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.048+0000 7fdb06a00640 1 -- 172.21.15.44:0/561016538 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdb100b51d0 con 0x7fdaec076de0 2024-07-17T07:07:29.048 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.052+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 msgr2=0x7fdaec0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.052+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 0x7fdaec0792a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8036000 tx=0x7fdaf80046c0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.052+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 msgr2=0x7fdb100c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.052+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c8c0 tx=0x7fdb0000cd80 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 shutdown_connections 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdaec076de0 0x7fdaec0792a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb100c8720 0x7fdb100c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb10152a90 0x7fdb100c81e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 --2- 172.21.15.44:0/561016538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaf4003ac0 0x7fdb101f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 >> 172.21.15.44:0/561016538 conn(0x7fdb100cfec0 msgr2=0x7fdb100ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 shutdown_connections 2024-07-17T07:07:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.053+0000 7fdb0fe00640 1 -- 172.21.15.44:0/561016538 wait complete. 2024-07-17T07:07:29.074 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2024-07-17T07:07:29.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:29.074 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-07-17T07:07:29.133 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:29.133 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:29.133 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:29.134 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:29.135 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_01_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_01_UTC', 'scheduled-2024-07-17-07_00_01_UTC', 'scheduled-2024-07-17-07_01_01_UTC', 'scheduled-2024-07-17-07_02_00_UTC', 'scheduled-2024-07-17-07_03_01_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_01_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:29.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.358+0000 7f28aa000640 1 -- 172.21.15.44:0/2504387359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 msgr2=0x7f28a4161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.358+0000 7f28aa000640 1 --2- 172.21.15.44:0/2504387359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a4161770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f28a40cf640 tx=0x7f289402a560 comp rx=0 tx=0).stop 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.358+0000 7f28aa000640 1 -- 172.21.15.44:0/2504387359 shutdown_connections 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.358+0000 7f28aa000640 1 --2- 172.21.15.44:0/2504387359 >> v2:172.21.15.175:3300/0 conn(0x7f28a40d1440 0x7f28a40d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.358+0000 7f28aa000640 1 --2- 172.21.15.44:0/2504387359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a4161770 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.359+0000 7f28aa000640 1 -- 172.21.15.44:0/2504387359 >> 172.21.15.44:0/2504387359 conn(0x7f28a40d04c0 msgr2=0x7f28a40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.359+0000 7f28aa000640 1 -- 172.21.15.44:0/2504387359 shutdown_connections 2024-07-17T07:07:29.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.359+0000 7f28aa000640 1 -- 172.21.15.44:0/2504387359 wait complete. 2024-07-17T07:07:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.361+0000 7f28aa000640 1 Processor -- start 2024-07-17T07:07:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.361+0000 7f28aa000640 1 -- start start 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28a40cdca0 0x7f28a41f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a40d1440 0x7f28a40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28a406ead0 con 0x7f28a415d350 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28a406e950 con 0x7f28a40d1440 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.362+0000 7f28aa000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28a40ce130 con 0x7f28a40cdca0 2024-07-17T07:07:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38178/0 (socket says 172.21.15.44:38178) 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 -- 172.21.15.44:0/67747497 learned_addr learned my addr 172.21.15.44:0/67747497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a3e00640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a40d1440 0x7f28a40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a8c00640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28a40cdca0 0x7f28a41f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 -- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28a40cdca0 msgr2=0x7f28a41f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28a40cdca0 0x7f28a41f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 -- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a40d1440 msgr2=0x7f28a40ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a40d1440 0x7f28a40ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.363+0000 7f28a9600640 1 -- 172.21.15.44:0/67747497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a40cc7c0 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28a9600640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c760 tx=0x7f289c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c016020 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f289c00ce10 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c006e60 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a41f5200 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.364+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a40cd000 con 0x7f28a415d350 2024-07-17T07:07:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.366+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a40d1820 con 0x7f28a415d350 2024-07-17T07:07:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.367+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f289c003bf0 con 0x7f28a415d350 2024-07-17T07:07:29.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.368+0000 7f28a0c00640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 0x7f2880079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.368+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f289c098640 con 0x7f28a415d350 2024-07-17T07:07:29.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.369+0000 7f28a8c00640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 0x7f2880079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.370+0000 7f28a8c00640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 0x7f2880079270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2894033000 tx=0x7f2894004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.374+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f289c020020 con 0x7f28a415d350 2024-07-17T07:07:29.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.609+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28a40b29f0 con 0x7f2880076db0 2024-07-17T07:07:29.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.631+0000 7f28a0c00640 1 -- 172.21.15.44:0/67747497 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f28a40b29f0 con 0x7f2880076db0 2024-07-17T07:07:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:29.630 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:18", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 msgr2=0x7f2880079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 0x7f2880079270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2894033000 tx=0x7f2894004730 comp rx=0 tx=0).stop 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 msgr2=0x7f28a40c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c760 tx=0x7f289c00cc20 comp rx=0 tx=0).stop 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 shutdown_connections 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2880076db0 0x7f2880079270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a415d350 0x7f28a40c9020 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28a40d1440 0x7f28a40ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 --2- 172.21.15.44:0/67747497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28a40cdca0 0x7f28a41f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 >> 172.21.15.44:0/67747497 conn(0x7f28a40d04c0 msgr2=0x7f28a40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:29.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 shutdown_connections 2024-07-17T07:07:29.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.634+0000 7f28aa000640 1 -- 172.21.15.44:0/67747497 wait complete. 2024-07-17T07:07:29.653 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:18", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:29.653 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:18', 'first': '2024-07-17T06:57:01', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:29.654 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:29.654 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:29.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:29.654 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-07-17T07:07:29.709 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:29.710 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:29.711 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_01_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_00_UTC', 'scheduled-2024-07-17-07_03_01_UTC', 'scheduled-2024-07-17-07_04_01_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:29.711 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2024-07-17T07:07:29.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 -- 172.21.15.44:0/1507946902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 msgr2=0x7f518c0ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 --2- 172.21.15.44:0/1507946902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c0ac5c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f518c0708b0 tx=0x7f518402aa50 comp rx=0 tx=0).stop 2024-07-17T07:07:29.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 -- 172.21.15.44:0/1507946902 shutdown_connections 2024-07-17T07:07:29.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 --2- 172.21.15.44:0/1507946902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c0ac5c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 --2- 172.21.15.44:0/1507946902 >> v2:172.21.15.175:3300/0 conn(0x7f518c154600 0x7f518c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 -- 172.21.15.44:0/1507946902 >> 172.21.15.44:0/1507946902 conn(0x7f518c0b0eb0 msgr2=0x7f518c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:29.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.927+0000 7f5192000640 1 -- 172.21.15.44:0/1507946902 shutdown_connections 2024-07-17T07:07:29.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.928+0000 7f5192000640 1 -- 172.21.15.44:0/1507946902 wait complete. 2024-07-17T07:07:29.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.929+0000 7f5192000640 1 Processor -- start 2024-07-17T07:07:29.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.929+0000 7f5192000640 1 -- start start 2024-07-17T07:07:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c154600 0x7f518c074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c161120 0x7f518c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15e220 con 0x7f518c0ac190 2024-07-17T07:07:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15e0a0 con 0x7f518c154600 2024-07-17T07:07:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.930+0000 7f5192000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f518c15e3a0 con 0x7f518c161120 2024-07-17T07:07:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38196/0 (socket says 172.21.15.44:38196) 2024-07-17T07:07:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 -- 172.21.15.44:0/392257162 learned_addr learned my addr 172.21.15.44:0/392257162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:29.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518aa00640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c154600 0x7f518c074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518be00640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c161120 0x7f518c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 -- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c161120 msgr2=0x7f518c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c161120 0x7f518c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 -- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c154600 msgr2=0x7f518c074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:29.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c154600 0x7f518c074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.931+0000 7f518b400640 1 -- 172.21.15.44:0/392257162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1f7890 con 0x7f518c0ac190 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f518b400640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f517c00d770 tx=0x7f517c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c017070 con 0x7f518c0ac190 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f517c009d20 con 0x7f518c0ac190 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1566b0 con 0x7f518c0ac190 2024-07-17T07:07:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c005280 con 0x7f518c0ac190 2024-07-17T07:07:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.932+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1f7b50 con 0x7f518c0ac190 2024-07-17T07:07:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.933+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c15e970 con 0x7f518c0ac190 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.934+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f517c005420 con 0x7f518c0ac190 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.935+0000 7f5183400640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 0x7f516c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.935+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f517c098b70 con 0x7f518c0ac190 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.936+0000 7f518aa00640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 0x7f516c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.937+0000 7f518aa00640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 0x7f516c079270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:29.940+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f517c0619f0 con 0x7f518c0ac190 2024-07-17T07:07:30.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.173+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f518c160e90 con 0x7f516c076db0 2024-07-17T07:07:30.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.194+0000 7f5183400640 1 -- 172.21.15.44:0/392257162 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f518c160e90 con 0x7f516c076db0 2024-07-17T07:07:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:30.193 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:19", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:30.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 msgr2=0x7f516c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 0x7f516c079270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178008040 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 msgr2=0x7f518c1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f517c00d770 tx=0x7f517c00dc30 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 shutdown_connections 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f516c076db0 0x7f516c079270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c161120 0x7f518c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c154600 0x7f518c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 --2- 172.21.15.44:0/392257162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f518c0ac190 0x7f518c1f8350 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.197+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 >> 172.21.15.44:0/392257162 conn(0x7f518c0b0eb0 msgr2=0x7f518c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.198+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 shutdown_connections 2024-07-17T07:07:30.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.198+0000 7f5192000640 1 -- 172.21.15.44:0/392257162 wait complete. 2024-07-17T07:07:30.217 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:19", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:30.217 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:19', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:30.217 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:30.217 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:30.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:30.217 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-07-17T07:07:30.268 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:30.269 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:30.270 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_01_UTC', 'scheduled-2024-07-17-06_59_01_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_01_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_01_UTC'] 2024-07-17T07:07:30.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2024-07-17T07:07:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 -- 172.21.15.44:0/503165556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e415d350 msgr2=0x7f01e4161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 --2- 172.21.15.44:0/503165556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e415d350 0x7f01e4161770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f01e40cf640 tx=0x7f01d402aa30 comp rx=0 tx=0).stop 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 -- 172.21.15.44:0/503165556 shutdown_connections 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 --2- 172.21.15.44:0/503165556 >> v2:172.21.15.175:3300/0 conn(0x7f01e40d1440 0x7f01e40d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 --2- 172.21.15.44:0/503165556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e415d350 0x7f01e4161770 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 -- 172.21.15.44:0/503165556 >> 172.21.15.44:0/503165556 conn(0x7f01e40d04c0 msgr2=0x7f01e40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 -- 172.21.15.44:0/503165556 shutdown_connections 2024-07-17T07:07:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.480+0000 7f01e3e00640 1 -- 172.21.15.44:0/503165556 wait complete. 2024-07-17T07:07:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.482+0000 7f01e3e00640 1 Processor -- start 2024-07-17T07:07:30.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.482+0000 7f01e3e00640 1 -- start start 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e40d1440 0x7f01e4154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e415d350 0x7f01e40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01e406ead0 con 0x7f01e40cdca0 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01e406e950 con 0x7f01e415d350 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.483+0000 7f01e3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40ce130 con 0x7f01e40d1440 2024-07-17T07:07:30.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38220/0 (socket says 172.21.15.44:38220) 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 -- 172.21.15.44:0/1079689961 learned_addr learned my addr 172.21.15.44:0/1079689961 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 -- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e40d1440 msgr2=0x7f01e4154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e40d1440 0x7f01e4154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 -- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e415d350 msgr2=0x7f01e40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e415d350 0x7f01e40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 -- 172.21.15.44:0/1079689961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41f87b0 con 0x7f01e40cdca0 2024-07-17T07:07:30.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.484+0000 7f01e2a00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009bf0 tx=0x7f01d4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.485+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4033050 con 0x7f01e40cdca0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.485+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d40044d0 con 0x7f01e40cdca0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.485+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e4154f90 con 0x7f01e40cdca0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.485+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4038450 con 0x7f01e40cdca0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.485+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e4155550 con 0x7f01e40cdca0 2024-07-17T07:07:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.486+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40d1820 con 0x7f01e40cdca0 2024-07-17T07:07:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.487+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f01d4036070 con 0x7f01e40cdca0 2024-07-17T07:07:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.488+0000 7f01db400640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 0x7f01c0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:30.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.489+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f01d40b9e20 con 0x7f01e40cdca0 2024-07-17T07:07:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.489+0000 7f01da000640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 0x7f01c0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:30.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.490+0000 7f01da000640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 0x7f01c0079270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005fd0 tx=0x7f01cc005850 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:30.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.492+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01d4082ca0 con 0x7f01e40cdca0 2024-07-17T07:07:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.727+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01e40b29f0 con 0x7f01c0076db0 2024-07-17T07:07:30.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.749+0000 7f01db400640 1 -- 172.21.15.44:0/1079689961 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f01e40b29f0 con 0x7f01c0076db0 2024-07-17T07:07:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:30.748 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:19", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:30.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.751+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 msgr2=0x7f01c0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.751+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 0x7f01c0079270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005fd0 tx=0x7f01cc005850 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.751+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 msgr2=0x7f01e41f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.751+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009bf0 tx=0x7f01d4003f10 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 shutdown_connections 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f01c0076db0 0x7f01c0079270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01e415d350 0x7f01e40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01e40d1440 0x7f01e4154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 --2- 172.21.15.44:0/1079689961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40cdca0 0x7f01e41f8270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 >> 172.21.15.44:0/1079689961 conn(0x7f01e40d04c0 msgr2=0x7f01e40d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 shutdown_connections 2024-07-17T07:07:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:30.752+0000 7f01e3e00640 1 -- 172.21.15.44:0/1079689961 wait complete. 2024-07-17T07:07:30.769 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:19", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:30.769 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:19', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:30.769 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:30.769 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:30.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:30.770 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:30.820 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:30.820 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:30.820 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_01_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_01_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_01_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_01_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_01_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:30.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2024-07-17T07:07:31.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 -- 172.21.15.44:0/2076972352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a580ac3a0 msgr2=0x7f9a580ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 --2- 172.21.15.44:0/2076972352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a580ac3a0 0x7f9a580ac810 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009fd0 tx=0x7f9a5002a890 comp rx=0 tx=0).stop 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 -- 172.21.15.44:0/2076972352 shutdown_connections 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 --2- 172.21.15.44:0/2076972352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a580ac3a0 0x7f9a580ac810 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 --2- 172.21.15.44:0/2076972352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5815a5a0 0x7f9a580b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 -- 172.21.15.44:0/2076972352 >> 172.21.15.44:0/2076972352 conn(0x7f9a580b0eb0 msgr2=0x7f9a580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 -- 172.21.15.44:0/2076972352 shutdown_connections 2024-07-17T07:07:31.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.036+0000 7f9a5d600640 1 -- 172.21.15.44:0/2076972352 wait complete. 2024-07-17T07:07:31.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.038+0000 7f9a5d600640 1 Processor -- start 2024-07-17T07:07:31.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.038+0000 7f9a5d600640 1 -- start start 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a580ac3a0 0x7f9a58152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5815a5a0 0x7f9a58152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5806f080 con 0x7f9a5806eb90 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5806f8e0 con 0x7f9a580ac3a0 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a5d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5814fd60 con 0x7f9a5815a5a0 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38228/0 (socket says 172.21.15.44:38228) 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.039+0000 7f9a56a00640 1 -- 172.21.15.44:0/4056717162 learned_addr learned my addr 172.21.15.44:0/4056717162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56000640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a580ac3a0 0x7f9a58152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 -- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5815a5a0 msgr2=0x7f9a58152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5815a5a0 0x7f9a58152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 -- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a580ac3a0 msgr2=0x7f9a58152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a580ac3a0 0x7f9a58152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 -- 172.21.15.44:0/4056717162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a581f1a50 con 0x7f9a5806eb90 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a56a00640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c490 tx=0x7f9a4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48016020 con 0x7f9a5806eb90 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a48012940 con 0x7f9a5806eb90 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48012d90 con 0x7f9a5806eb90 2024-07-17T07:07:31.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.040+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a580742e0 con 0x7f9a5806eb90 2024-07-17T07:07:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.041+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a58150f20 con 0x7f9a5806eb90 2024-07-17T07:07:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.042+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a580ac810 con 0x7f9a5806eb90 2024-07-17T07:07:31.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.043+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f9a48019020 con 0x7f9a5806eb90 2024-07-17T07:07:31.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.044+0000 7f9a4ea00640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 0x7f9a340792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.044+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f9a48098680 con 0x7f9a5806eb90 2024-07-17T07:07:31.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.045+0000 7f9a56000640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 0x7f9a340792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.046+0000 7f9a56000640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 0x7f9a340792a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a5815ba90 tx=0x7f9a40005c20 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:31.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.049+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9a48020030 con 0x7f9a5806eb90 2024-07-17T07:07:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.281+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a580b4200 con 0x7f9a34076de0 2024-07-17T07:07:31.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.302+0000 7f9a4ea00640 1 -- 172.21.15.44:0/4056717162 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f9a580b4200 con 0x7f9a34076de0 2024-07-17T07:07:31.301 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:31.301 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:20", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.304+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 msgr2=0x7f9a340792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.304+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 0x7f9a340792a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a5815ba90 tx=0x7f9a40005c20 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.304+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 msgr2=0x7f9a5814dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.304+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c490 tx=0x7f9a4800c950 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 shutdown_connections 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9a34076de0 0x7f9a340792a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5815a5a0 0x7f9a58152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a580ac3a0 0x7f9a58152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 --2- 172.21.15.44:0/4056717162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a5806eb90 0x7f9a5814dfb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 >> 172.21.15.44:0/4056717162 conn(0x7f9a580b0eb0 msgr2=0x7f9a58153820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:31.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 shutdown_connections 2024-07-17T07:07:31.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.305+0000 7f9a5d600640 1 -- 172.21.15.44:0/4056717162 wait complete. 2024-07-17T07:07:31.322 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:20", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:31.322 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:20', 'first': '2024-07-17T06:57:01', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:31.322 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:31.323 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:31.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:31.323 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:31.374 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:31.375 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:31.376 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_01_UTC', 'scheduled-2024-07-17-07_03_01_UTC', 'scheduled-2024-07-17-07_04_01_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_01_UTC'] 2024-07-17T07:07:31.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.576+0000 7f643b800640 1 -- 172.21.15.44:0/3621525174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 msgr2=0x7f643415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.576+0000 7f643b800640 1 --2- 172.21.15.44:0/3621525174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415c2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f642c005fd0 tx=0x7f642c02a820 comp rx=0 tx=0).stop 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 -- 172.21.15.44:0/3621525174 shutdown_connections 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 --2- 172.21.15.44:0/3621525174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415c2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 --2- 172.21.15.44:0/3621525174 >> v2:172.21.15.175:3300/0 conn(0x7f6434152220 0x7f643406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 -- 172.21.15.44:0/3621525174 >> 172.21.15.44:0/3621525174 conn(0x7f64340b0eb0 msgr2=0x7f64340b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 -- 172.21.15.44:0/3621525174 shutdown_connections 2024-07-17T07:07:31.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.577+0000 7f643b800640 1 -- 172.21.15.44:0/3621525174 wait complete. 2024-07-17T07:07:31.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.578+0000 7f643b800640 1 Processor -- start 2024-07-17T07:07:31.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 -- start start 2024-07-17T07:07:31.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 0x7f64341f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64340cd7b0 con 0x7f643406ed30 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64340cd630 con 0x7f6434152220 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.579+0000 7f643b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64340cd930 con 0x7f64340cbab0 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38244/0 (socket says 172.21.15.44:38244) 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 -- 172.21.15.44:0/1869233994 learned_addr learned my addr 172.21.15.44:0/1869233994 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6433e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6439800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 0x7f64341f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 -- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 msgr2=0x7f643415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 -- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 msgr2=0x7f64341f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 0x7f64341f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6438e00640 1 -- 172.21.15.44:0/1869233994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6434159dc0 con 0x7f643406ed30 2024-07-17T07:07:31.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.580+0000 7f6439800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 0x7f64341f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:31.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f6438e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f642800e9f0 tx=0x7f642800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642800cd80 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6428004020 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428004470 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643415a040 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.581+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643415a600 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.582+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64340cde50 con 0x7f643406ed30 2024-07-17T07:07:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.584+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f64280035f0 con 0x7f643406ed30 2024-07-17T07:07:31.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.584+0000 7f6430c00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 0x7f6414079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:31.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.585+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f6428099240 con 0x7f643406ed30 2024-07-17T07:07:31.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.585+0000 7f6433e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 0x7f6414079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:31.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.586+0000 7f6433e00640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 0x7f6414079270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f643415b650 tx=0x7f6420009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.589+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6434159dc0 con 0x7f643406ed30 2024-07-17T07:07:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.822+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64340b0b30 con 0x7f6414076db0 2024-07-17T07:07:31.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.844+0000 7f6430c00640 1 -- 172.21.15.44:0/1869233994 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f64340b0b30 con 0x7f6414076db0 2024-07-17T07:07:31.843 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:31.843 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:21", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:31.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 msgr2=0x7f6414079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 0x7f6414079270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f643415b650 tx=0x7f6420009040 comp rx=0 tx=0).stop 2024-07-17T07:07:31.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 msgr2=0x7f6434145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:31.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f642800e9f0 tx=0x7f642800eeb0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 shutdown_connections 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6414076db0 0x7f6414079270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434152220 0x7f64341f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64340cbab0 0x7f643415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 --2- 172.21.15.44:0/1869233994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f643406ed30 0x7f6434145e90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.847+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 >> 172.21.15.44:0/1869233994 conn(0x7f64340b0eb0 msgr2=0x7f64340b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.848+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 shutdown_connections 2024-07-17T07:07:31.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:31.848+0000 7f643b800640 1 -- 172.21.15.44:0/1869233994 wait complete. 2024-07-17T07:07:31.866 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:21", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:31.866 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:21', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:31.866 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:31.866 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:31.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:31.867 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:31.915 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:31.916 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:31.916 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:31.917 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_01_UTC', 'scheduled-2024-07-17-07_03_01_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_01_UTC', 'scheduled-2024-07-17-07_06_01_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:31.917 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2024-07-17T07:07:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 -- 172.21.15.44:0/2056812245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940ac770 msgr2=0x7f519415a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 --2- 172.21.15.44:0/2056812245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940ac770 0x7f519415a7c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f51940b3c10 tx=0x7f518c02a8b0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 -- 172.21.15.44:0/2056812245 shutdown_connections 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 --2- 172.21.15.44:0/2056812245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940ac770 0x7f519415a7c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 --2- 172.21.15.44:0/2056812245 >> v2:172.21.15.175:3300/0 conn(0x7f51940ac390 0x7f519415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 -- 172.21.15.44:0/2056812245 >> 172.21.15.44:0/2056812245 conn(0x7f519406c120 msgr2=0x7f519406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 -- 172.21.15.44:0/2056812245 shutdown_connections 2024-07-17T07:07:32.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.120+0000 7f5198a00640 1 -- 172.21.15.44:0/2056812245 wait complete. 2024-07-17T07:07:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.122+0000 7f5198a00640 1 Processor -- start 2024-07-17T07:07:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.122+0000 7f5198a00640 1 -- start start 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51940ac390 0x7f51941544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 0x7f51941f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519415b180 con 0x7f51940b3cb0 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f519415b000 con 0x7f51940ac390 2024-07-17T07:07:32.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.123+0000 7f5198a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f519406d690 con 0x7f51940ac770 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38268/0 (socket says 172.21.15.44:38268) 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 -- 172.21.15.44:0/3793006054 learned_addr learned my addr 172.21.15.44:0/3793006054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5192800640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51940ac390 0x7f51941544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f518be00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 0x7f51941f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 -- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 msgr2=0x7f51941f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 0x7f51941f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 -- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51940ac390 msgr2=0x7f51941544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51940ac390 0x7f51941544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f5193200640 1 -- 172.21.15.44:0/3793006054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5194157420 con 0x7f51940b3cb0 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.124+0000 7f518be00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 0x7f51941f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f5193200640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f518c02aea0 tx=0x7f518c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c040070 con 0x7f51940b3cb0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f518c004150 con 0x7f51940b3cb0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0045a0 con 0x7f51940b3cb0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5194157700 con 0x7f51940b3cb0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.125+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5194157cc0 con 0x7f51940b3cb0 2024-07-17T07:07:32.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.126+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51940b3330 con 0x7f51940b3cb0 2024-07-17T07:07:32.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.127+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f518c039040 con 0x7f51940b3cb0 2024-07-17T07:07:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.128+0000 7f518aa00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 0x7f5170079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.128+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f518c0ba090 con 0x7f51940b3cb0 2024-07-17T07:07:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.129+0000 7f5192800640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 0x7f5170079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.130+0000 7f5192800640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 0x7f5170079270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5184005fd0 tx=0x7f5184005850 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.133+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f518c082f10 con 0x7f51940b3cb0 2024-07-17T07:07:32.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.373+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51941571d0 con 0x7f5170076db0 2024-07-17T07:07:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.394+0000 7f518aa00640 1 -- 172.21.15.44:0/3793006054 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f51941571d0 con 0x7f5170076db0 2024-07-17T07:07:32.393 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:32.393 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:22", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 msgr2=0x7f5170079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 0x7f5170079270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5184005fd0 tx=0x7f5184005850 comp rx=0 tx=0).stop 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 msgr2=0x7f51941f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f518c02aea0 tx=0x7f518c0039f0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 shutdown_connections 2024-07-17T07:07:32.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f5170076db0 0x7f5170079270 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5184005fd0 tx=0x7f5184005850 comp rx=0 tx=0).stop 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51940b3cb0 0x7f51941f7550 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51940ac770 0x7f51941f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 --2- 172.21.15.44:0/3793006054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51940ac390 0x7f51941544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 >> 172.21.15.44:0/3793006054 conn(0x7f519406c120 msgr2=0x7f51940ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 shutdown_connections 2024-07-17T07:07:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.397+0000 7f5198a00640 1 -- 172.21.15.44:0/3793006054 wait complete. 2024-07-17T07:07:32.415 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:22", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:32.416 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:22', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:32.416 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:32.416 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:32.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:32.416 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-07-17T07:07:32.470 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:32.470 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:32.470 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:32.472 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_01_UTC', 'scheduled-2024-07-17-06_58_01_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_00_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_01_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:32.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2024-07-17T07:07:32.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 -- 172.21.15.44:0/846569704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c004800 msgr2=0x7f003c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 --2- 172.21.15.44:0/846569704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c004800 0x7f003c004c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0040006fd0 tx=0x7f003c005830 comp rx=0 tx=0).stop 2024-07-17T07:07:32.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f0057400640 1 -- 172.21.15.44:0/846569704 reap_dead start 2024-07-17T07:07:32.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 -- 172.21.15.44:0/846569704 shutdown_connections 2024-07-17T07:07:32.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 -- 172.21.15.44:0/846569704 >> 172.21.15.44:0/846569704 conn(0x7f00580b0eb0 msgr2=0x7f00580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:32.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 -- 172.21.15.44:0/846569704 shutdown_connections 2024-07-17T07:07:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.708+0000 7f005da00640 1 -- 172.21.15.44:0/846569704 wait complete. 2024-07-17T07:07:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.710+0000 7f005da00640 1 Processor -- start 2024-07-17T07:07:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.710+0000 7f005da00640 1 -- start start 2024-07-17T07:07:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003c004800 0x7f00581f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 0x7f00581f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f005806ed10 con 0x7f00581f1780 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f005806eb90 con 0x7f00581403b0 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f005da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f005806ee90 con 0x7f003c004800 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.711+0000 7f0057400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38282/0 (socket says 172.21.15.44:38282) 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 -- 172.21.15.44:0/801010124 learned_addr learned my addr 172.21.15.44:0/801010124 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f004e000640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 0x7f00581f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0056a00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003c004800 0x7f00581f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 -- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003c004800 msgr2=0x7f00581f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003c004800 0x7f00581f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 -- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 msgr2=0x7f00581f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 0x7f00581f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f0057400640 1 -- 172.21.15.44:0/801010124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00581f85d0 con 0x7f00581f1780 2024-07-17T07:07:32.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.712+0000 7f004e000640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 0x7f00581f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:07:32.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f0057400640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f005806faf0 tx=0x7f005000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:32.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050004b20 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0050004dd0 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000ca80 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0058140950 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.713+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581411f0 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.714+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00580ae560 con 0x7f00581f1780 2024-07-17T07:07:32.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.716+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f005001b020 con 0x7f00581f1780 2024-07-17T07:07:32.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.716+0000 7f004f400640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 0x7f0034079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:32.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.717+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f00500986a0 con 0x7f00581f1780 2024-07-17T07:07:32.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.717+0000 7f0056a00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 0x7f0034079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:32.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.718+0000 7f0056a00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 0x7f0034079270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006b10 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.722+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f00581f85d0 con 0x7f00581f1780 2024-07-17T07:07:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.956+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00580b4200 con 0x7f0034076db0 2024-07-17T07:07:32.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.978+0000 7f004f400640 1 -- 172.21.15.44:0/801010124 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f00580b4200 con 0x7f0034076db0 2024-07-17T07:07:32.977 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:32.978 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:22", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:32.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 msgr2=0x7f0034079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 0x7f0034079270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0048006fd0 tx=0x7f0048006b10 comp rx=0 tx=0).stop 2024-07-17T07:07:32.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 msgr2=0x7f00580af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:32.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f005806faf0 tx=0x7f005000eb10 comp rx=0 tx=0).stop 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 shutdown_connections 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0034076db0 0x7f0034079270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00581f1780 0x7f00580af3b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00581403b0 0x7f00581f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 --2- 172.21.15.44:0/801010124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003c004800 0x7f00581f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.981+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 >> 172.21.15.44:0/801010124 conn(0x7f00580b0eb0 msgr2=0x7f00580ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.982+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 shutdown_connections 2024-07-17T07:07:32.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:32.982+0000 7f005da00640 1 -- 172.21.15.44:0/801010124 wait complete. 2024-07-17T07:07:33.001 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:22", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:33.001 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:22', 'first': '2024-07-17T06:57:01', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:33.001 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:33.001 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:33.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:33.001 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:33.050 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:33.051 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_01_UTC', 'scheduled-2024-07-17-07_00_01_UTC', 'scheduled-2024-07-17-07_01_00_UTC', 'scheduled-2024-07-17-07_02_00_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_01_UTC', 'scheduled-2024-07-17-07_06_00_UTC', 'scheduled-2024-07-17-07_07_01_UTC'] 2024-07-17T07:07:33.051 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.260+0000 7f7c65200640 1 -- 172.21.15.44:0/3683544841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 msgr2=0x7f7c600b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.260+0000 7f7c65200640 1 --2- 172.21.15.44:0/3683544841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c600b3ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009830 tx=0x7f7c5002a510 comp rx=0 tx=0).stop 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 -- 172.21.15.44:0/3683544841 shutdown_connections 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 --2- 172.21.15.44:0/3683544841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 0x7f7c600ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 --2- 172.21.15.44:0/3683544841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c600b3ce0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 -- 172.21.15.44:0/3683544841 >> 172.21.15.44:0/3683544841 conn(0x7f7c600b0eb0 msgr2=0x7f7c600b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 -- 172.21.15.44:0/3683544841 shutdown_connections 2024-07-17T07:07:33.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.261+0000 7f7c65200640 1 -- 172.21.15.44:0/3683544841 wait complete. 2024-07-17T07:07:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.263+0000 7f7c65200640 1 Processor -- start 2024-07-17T07:07:33.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.263+0000 7f7c65200640 1 -- start start 2024-07-17T07:07:33.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 0x7f7c601f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c60155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6015e220 con 0x7f7c60161120 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6015e0a0 con 0x7f7c60154600 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c65200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6015e3a0 con 0x7f7c600ac190 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c5e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c5e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38298/0 (socket says 172.21.15.44:38298) 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c5e800640 1 -- 172.21.15.44:0/197566138 learned_addr learned my addr 172.21.15.44:0/197566138 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c5de00640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 0x7f7c601f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.264+0000 7f7c5d400640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c60155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 -- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 msgr2=0x7f7c601f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 0x7f7c601f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 -- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 msgr2=0x7f7c60155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c60155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 -- 172.21.15.44:0/197566138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c601f5160 con 0x7f7c60161120 2024-07-17T07:07:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c5e800640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800eb10 tx=0x7f7c5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004b20 con 0x7f7c60161120 2024-07-17T07:07:33.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c58004ce0 con 0x7f7c60161120 2024-07-17T07:07:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58005280 con 0x7f7c60161120 2024-07-17T07:07:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.265+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c601563c0 con 0x7f7c60161120 2024-07-17T07:07:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.266+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c60156980 con 0x7f7c60161120 2024-07-17T07:07:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.267+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6015e930 con 0x7f7c60161120 2024-07-17T07:07:33.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.268+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f7c58005420 con 0x7f7c60161120 2024-07-17T07:07:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.269+0000 7f7c56000640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 0x7f7c40079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.269+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f7c58099810 con 0x7f7c60161120 2024-07-17T07:07:33.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.270+0000 7f7c5de00640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 0x7f7c40079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.271+0000 7f7c5de00640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 0x7f7c40079270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c50004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:33.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.273+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7c58062580 con 0x7f7c60161120 2024-07-17T07:07:33.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.508+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c60157180 con 0x7f7c40076db0 2024-07-17T07:07:33.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.529+0000 7f7c56000640 1 -- 172.21.15.44:0/197566138 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f7c500041c0 con 0x7f7c40076db0 2024-07-17T07:07:33.528 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:33.528 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:23", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:33.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 msgr2=0x7f7c40079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 0x7f7c40079270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c50004730 comp rx=0 tx=0).stop 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 msgr2=0x7f7c601f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800eb10 tx=0x7f7c5800c2d0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 shutdown_connections 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7c40076db0 0x7f7c40079270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c60161120 0x7f7c601f4c20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c60154600 0x7f7c60155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 --2- 172.21.15.44:0/197566138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600ac190 0x7f7c601f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 >> 172.21.15.44:0/197566138 conn(0x7f7c600b0eb0 msgr2=0x7f7c6015c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 shutdown_connections 2024-07-17T07:07:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.532+0000 7f7c65200640 1 -- 172.21.15.44:0/197566138 wait complete. 2024-07-17T07:07:33.551 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:23", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:33.551 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:23', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:33.551 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:33.551 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:33.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:33.551 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:33.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:33.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:33.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:33.603 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_00_UTC', 'scheduled-2024-07-17-06_58_00_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_01_UTC', 'scheduled-2024-07-17-07_02_00_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_01_UTC', 'scheduled-2024-07-17-07_07_00_UTC'] 2024-07-17T07:07:33.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.824+0000 7f013ce00640 1 -- 172.21.15.44:0/4039876166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 msgr2=0x7f011c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.824+0000 7f013ce00640 1 --2- 172.21.15.44:0/4039876166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f011c003e70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f011c005830 comp rx=0 tx=0).stop 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.824+0000 7f0137e00640 1 -- 172.21.15.44:0/4039876166 reap_dead start 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.825+0000 7f013ce00640 1 -- 172.21.15.44:0/4039876166 shutdown_connections 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.825+0000 7f013ce00640 1 -- 172.21.15.44:0/4039876166 >> 172.21.15.44:0/4039876166 conn(0x7f01380d04c0 msgr2=0x7f01380d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.825+0000 7f013ce00640 1 -- 172.21.15.44:0/4039876166 shutdown_connections 2024-07-17T07:07:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.825+0000 7f013ce00640 1 -- 172.21.15.44:0/4039876166 wait complete. 2024-07-17T07:07:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.826+0000 7f013ce00640 1 Processor -- start 2024-07-17T07:07:33.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.827+0000 7f013ce00640 1 -- start start 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01381f7ba0 0x7f013815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01381f8020 0x7f013815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f013806ead0 con 0x7f011c003a50 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f013806e950 con 0x7f01381f7ba0 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f013ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01380d1820 con 0x7f01381f8020 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38324/0 (socket says 172.21.15.44:38324) 2024-07-17T07:07:33.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 -- 172.21.15.44:0/3804588498 learned_addr learned my addr 172.21.15.44:0/3804588498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:33.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 -- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01381f8020 msgr2=0x7f013815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01381f8020 0x7f013815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 -- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01381f7ba0 msgr2=0x7f013815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:33.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0136a00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01381f7ba0 0x7f013815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01381f7ba0 0x7f013815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.828+0000 7f0137400640 1 -- 172.21.15.44:0/3804588498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01381530c0 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f0137400640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128033c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128043070 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0128033e30 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128038040 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013815c9c0 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.829+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0138160be0 con 0x7f011c003a50 2024-07-17T07:07:33.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.831+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01380b6030 con 0x7f011c003a50 2024-07-17T07:07:33.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.832+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f0128003bb0 con 0x7f011c003a50 2024-07-17T07:07:33.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.833+0000 7f012f400640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 0x7f01140792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:33.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.833+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f01280b9140 con 0x7f011c003a50 2024-07-17T07:07:33.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.833+0000 7f0136a00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 0x7f01140792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:33.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.835+0000 7f0136a00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 0x7f01140792a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01200058c0 tx=0x7f0120005850 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:33.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:33.837+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01381530c0 con 0x7f011c003a50 2024-07-17T07:07:34.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.075+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01380b29f0 con 0x7f0114076de0 2024-07-17T07:07:34.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.097+0000 7f012f400640 1 -- 172.21.15.44:0/3804588498 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f01380b29f0 con 0x7f0114076de0 2024-07-17T07:07:34.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:34.096 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:24", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 msgr2=0x7f01140792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 0x7f01140792a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01200058c0 tx=0x7f0120005850 comp rx=0 tx=0).stop 2024-07-17T07:07:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 msgr2=0x7f01381616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128033c40 comp rx=0 tx=0).stop 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 shutdown_connections 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f0114076de0 0x7f01140792a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01381f8020 0x7f013815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01381f7ba0 0x7f013815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 --2- 172.21.15.44:0/3804588498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f011c003a50 0x7f01381616c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.100+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 >> 172.21.15.44:0/3804588498 conn(0x7f01380d04c0 msgr2=0x7f01380d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.101+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 shutdown_connections 2024-07-17T07:07:34.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.101+0000 7f013ce00640 1 -- 172.21.15.44:0/3804588498 wait complete. 2024-07-17T07:07:34.120 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:24", "first": "2024-07-17T06:57:00", "last": "2024-07-17T07:07:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:34.120 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:24', 'first': '2024-07-17T06:57:00', 'last': '2024-07-17T07:07:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:34.121 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:34.121 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:34.121 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-07-17T07:07:34.170 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:34.170 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:34.170 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:34.170 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:34.170 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:34.171 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:34.172 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-06_57_01_UTC', 'scheduled-2024-07-17-06_58_01_UTC', 'scheduled-2024-07-17-06_59_00_UTC', 'scheduled-2024-07-17-07_00_00_UTC', 'scheduled-2024-07-17-07_01_01_UTC', 'scheduled-2024-07-17-07_02_01_UTC', 'scheduled-2024-07-17-07_03_00_UTC', 'scheduled-2024-07-17-07_04_00_UTC', 'scheduled-2024-07-17-07_05_00_UTC', 'scheduled-2024-07-17-07_06_01_UTC', 'scheduled-2024-07-17-07_07_01_UTC'] 2024-07-17T07:07:34.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2024-07-17T07:07:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 -- 172.21.15.44:0/2917827031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc146bb0 msgr2=0x7f2dfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 --2- 172.21.15.44:0/2917827031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc06e9f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0b6080 tx=0x7f2dec02a980 comp rx=0 tx=0).stop 2024-07-17T07:07:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 -- 172.21.15.44:0/2917827031 shutdown_connections 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 --2- 172.21.15.44:0/2917827031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 0x7f2dfc160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 --2- 172.21.15.44:0/2917827031 >> v2:172.21.15.175:3300/0 conn(0x7f2dfc06e510 0x7f2dfc06f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 --2- 172.21.15.44:0/2917827031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc06e9f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 -- 172.21.15.44:0/2917827031 >> 172.21.15.44:0/2917827031 conn(0x7f2dfc068a40 msgr2=0x7f2dfc068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 -- 172.21.15.44:0/2917827031 shutdown_connections 2024-07-17T07:07:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.378+0000 7f2dfb200640 1 -- 172.21.15.44:0/2917827031 wait complete. 2024-07-17T07:07:34.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.380+0000 7f2dfb200640 1 Processor -- start 2024-07-17T07:07:34.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.380+0000 7f2dfb200640 1 -- start start 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 0x7f2dfc1392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1615f0 con 0x7f2dfc06e510 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc161470 con 0x7f2dfc146bb0 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2dfb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc161770 con 0x7f2dfc06fbb0 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2df9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38336/0 (socket says 172.21.15.44:38336) 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.381+0000 7f2df9e00640 1 -- 172.21.15.44:0/1291316797 learned_addr learned my addr 172.21.15.44:0/1291316797 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df1400640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 0x7f2dfc1392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 -- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 msgr2=0x7f2dfc1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 0x7f2dfc1392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 -- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc146bb0 msgr2=0x7f2dfc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2dfa800640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 -- 172.21.15.44:0/1291316797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc154820 con 0x7f2dfc06e510 2024-07-17T07:07:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.382+0000 7f2df9e00640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000f80 tx=0x7f2dec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.383+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec034040 con 0x7f2dfc06e510 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.383+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dec004640 con 0x7f2dfc06e510 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.383+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec045480 con 0x7f2dfc06e510 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.383+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc0afb00 con 0x7f2dfc06e510 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.383+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc0b00c0 con 0x7f2dfc06e510 2024-07-17T07:07:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.384+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc161c20 con 0x7f2dfc06e510 2024-07-17T07:07:34.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.385+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f2dec037020 con 0x7f2dfc06e510 2024-07-17T07:07:34.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.386+0000 7f2df2a00640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 0x7f2dd80792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.386+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f2dec0ba380 con 0x7f2dfc06e510 2024-07-17T07:07:34.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.387+0000 7f2df1400640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 0x7f2dd80792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.388+0000 7f2df1400640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 0x7f2dd80792a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc1f7470 tx=0x7f2de4005c60 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:34.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.390+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2dec083330 con 0x7f2dfc06e510 2024-07-17T07:07:34.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.625+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2dfc0cf4f0 con 0x7f2dd8076de0 2024-07-17T07:07:34.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.645+0000 7f2df2a00640 1 -- 172.21.15.44:0/1291316797 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f2dfc0cf4f0 con 0x7f2dd8076de0 2024-07-17T07:07:34.644 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:34.644 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:25", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 msgr2=0x7f2dd80792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 0x7f2dd80792a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc1f7470 tx=0x7f2de4005c60 comp rx=0 tx=0).stop 2024-07-17T07:07:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 msgr2=0x7f2dfc1f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2dec000f80 tx=0x7f2dec003f90 comp rx=0 tx=0).stop 2024-07-17T07:07:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 shutdown_connections 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2dd8076de0 0x7f2dd80792a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dfc146bb0 0x7f2dfc0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dfc06fbb0 0x7f2dfc1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 --2- 172.21.15.44:0/1291316797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dfc06e510 0x7f2dfc1f49a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 >> 172.21.15.44:0/1291316797 conn(0x7f2dfc068a40 msgr2=0x7f2dfc0b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.648+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 shutdown_connections 2024-07-17T07:07:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.649+0000 7f2dfb200640 1 -- 172.21.15.44:0/1291316797 wait complete. 2024-07-17T07:07:34.667 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T06:56:25", "first": "2024-07-17T06:57:01", "last": "2024-07-17T07:07:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2024-07-17T07:07:34.668 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T06:56:25', 'first': '2024-07-17T06:57:01', 'last': '2024-07-17T07:07:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2024-07-17T07:07:34.668 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T07:07:34.668 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T07:07:34.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.885+0000 7f6860c00640 1 -- 172.21.15.44:0/971948037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c15d130 msgr2=0x7f685c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.885+0000 7f6860c00640 1 --2- 172.21.15.44:0/971948037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c15d130 0x7f685c15d510 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f685c0d0de0 tx=0x7f684c02a4d0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 -- 172.21.15.44:0/971948037 shutdown_connections 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 --2- 172.21.15.44:0/971948037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c161be0 0x7f685c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 --2- 172.21.15.44:0/971948037 >> v2:172.21.15.175:3300/0 conn(0x7f685c0ce690 0x7f685c0cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 --2- 172.21.15.44:0/971948037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c15d130 0x7f685c15d510 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 -- 172.21.15.44:0/971948037 >> 172.21.15.44:0/971948037 conn(0x7f685c0b4ff0 msgr2=0x7f685c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 -- 172.21.15.44:0/971948037 shutdown_connections 2024-07-17T07:07:34.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.886+0000 7f6860c00640 1 -- 172.21.15.44:0/971948037 wait complete. 2024-07-17T07:07:34.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.887+0000 7f6860c00640 1 Processor -- start 2024-07-17T07:07:34.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.888+0000 7f6860c00640 1 -- start start 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c15d130 0x7f685c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685c161be0 0x7f685c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06c7a0 con 0x7f685c0ce690 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06c620 con 0x7f685c161be0 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f6860c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f685c06c920 con 0x7f685c15d130 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f685b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f685b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38366/0 (socket says 172.21.15.44:38366) 2024-07-17T07:07:34.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f685b400640 1 -- 172.21.15.44:0/307099682 learned_addr learned my addr 172.21.15.44:0/307099682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f685be00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685c161be0 0x7f685c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.889+0000 7f685aa00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c15d130 0x7f685c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 -- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c15d130 msgr2=0x7f685c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c15d130 0x7f685c1580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 -- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685c161be0 msgr2=0x7f685c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:34.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685c161be0 0x7f685c15f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:34.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 -- 172.21.15.44:0/307099682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c15e670 con 0x7f685c0ce690 2024-07-17T07:07:34.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.890+0000 7f685b400640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f684c009850 tx=0x7f684c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.893+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c03f070 con 0x7f685c0ce690 2024-07-17T07:07:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.893+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f684c0047a0 con 0x7f685c0ce690 2024-07-17T07:07:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.893+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c035030 con 0x7f685c0ce690 2024-07-17T07:07:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.893+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c15e950 con 0x7f685c0ce690 2024-07-17T07:07:34.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.893+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c0aa110 con 0x7f685c0ce690 2024-07-17T07:07:34.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.894+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c0cfdf0 con 0x7f685c0ce690 2024-07-17T07:07:34.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.895+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f684c042070 con 0x7f685c0ce690 2024-07-17T07:07:34.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.896+0000 7f6853400640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 0x7f683c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:34.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.896+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f684c0b9930 con 0x7f685c0ce690 2024-07-17T07:07:34.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.897+0000 7f685aa00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 0x7f683c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:34.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.898+0000 7f685aa00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 0x7f683c079270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6848005d80 tx=0x7f6848005c90 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:34.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:34.901+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f684c0827b0 con 0x7f685c0ce690 2024-07-17T07:07:35.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.138+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f685c15ee70 con 0x7f683c076db0 2024-07-17T07:07:35.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.383+0000 7f6853400640 1 -- 172.21.15.44:0/307099682 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f685c15ee70 con 0x7f683c076db0 2024-07-17T07:07:35.382 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir0 2024-07-17T07:07:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 msgr2=0x7f683c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 0x7f683c079270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6848005d80 tx=0x7f6848005c90 comp rx=0 tx=0).stop 2024-07-17T07:07:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 msgr2=0x7f685c15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f684c009850 tx=0x7f684c003910 comp rx=0 tx=0).stop 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 shutdown_connections 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f683c076db0 0x7f683c079270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f685c161be0 0x7f685c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f685c15d130 0x7f685c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 --2- 172.21.15.44:0/307099682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f685c0ce690 0x7f685c15df20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.386+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 >> 172.21.15.44:0/307099682 conn(0x7f685c0b4ff0 msgr2=0x7f685c0b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.387+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 shutdown_connections 2024-07-17T07:07:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:35.387+0000 7f6860c00640 1 -- 172.21.15.44:0/307099682 wait complete. 2024-07-17T07:07:35.406 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2024-07-17T07:07:35.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.407 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2024-07-17T07:07:35.461 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:35.462 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:35.463 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:35.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.463 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_57_01_UTC) 2024-07-17T07:07:35.535 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:35.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.535 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:35.608 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:35.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.608 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-06_59_01_UTC) 2024-07-17T07:07:35.681 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:35.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.681 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_00_01_UTC) 2024-07-17T07:07:35.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:35.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.755 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_01_01_UTC) 2024-07-17T07:07:35.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:35.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.827 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_02_00_UTC) 2024-07-17T07:07:35.899 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:35.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.900 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_03_01_UTC) 2024-07-17T07:07:35.964 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:35.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:35.965 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:36.034 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:36.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:36.034 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_05_01_UTC) 2024-07-17T07:07:36.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:36.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:36.105 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:36.173 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:36.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:36.174 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:36.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:36.243 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2024-07-17T07:07:36.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 -- 172.21.15.44:0/1374061797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0b1af0 msgr2=0x7f3cbc15ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/1374061797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc15ac10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4005fd0 tx=0x7f3cb402aaa0 comp rx=0 tx=0).stop 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 -- 172.21.15.44:0/1374061797 shutdown_connections 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/1374061797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc15ac10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/1374061797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc0ac8a0 0x7f3cbc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 -- 172.21.15.44:0/1374061797 >> 172.21.15.44:0/1374061797 conn(0x7f3cbc0b0eb0 msgr2=0x7f3cbc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 -- 172.21.15.44:0/1374061797 shutdown_connections 2024-07-17T07:07:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.527+0000 7f3cc2a00640 1 -- 172.21.15.44:0/1374061797 wait complete. 2024-07-17T07:07:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.529+0000 7f3cc2a00640 1 Processor -- start 2024-07-17T07:07:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.529+0000 7f3cc2a00640 1 -- start start 2024-07-17T07:07:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc1f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 0x7f3cbc1f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc15b150 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc06f1d0 con 0x7f3cbc0b1af0 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3cbc15b2d0 con 0x7f3cbc15a280 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38384/0 (socket says 172.21.15.44:38384) 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 -- 172.21.15.44:0/605629312 learned_addr learned my addr 172.21.15.44:0/605629312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cb3e00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc1f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cc0a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 0x7f3cbc1f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 -- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 msgr2=0x7f3cbc1f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 0x7f3cbc1f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:36.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 -- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 msgr2=0x7f3cbc1f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc1f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.530+0000 7f3cbbe00640 1 -- 172.21.15.44:0/605629312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc158890 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cb3e00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc1f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cc0a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 0x7f3cbc1f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:36.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cbbe00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c3a0 tx=0x7f3cac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac017070 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cac0053c0 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac003850 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc158b70 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.531+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc159130 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.533+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cbc0b21b0 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.534+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f3cac01a070 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.534+0000 7f3cb8c00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 0x7f3c98079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:36.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.535+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f3cac097d90 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.535+0000 7f3cb3e00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 0x7f3c98079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:36.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.536+0000 7f3cb3e00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 0x7f3c98079270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc074590 tx=0x7f3ca4005e50 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:36.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.540+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3cac060c90 con 0x7f3cbc0ac8a0 2024-07-17T07:07:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:36.778+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3cbc0b3bd0 con 0x7f3c98076db0 2024-07-17T07:07:37.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.028+0000 7f3cb8c00640 1 -- 172.21.15.44:0/605629312 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f3cbc0b3bd0 con 0x7f3c98076db0 2024-07-17T07:07:37.027 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir1 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.031+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 msgr2=0x7f3c98079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.031+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 0x7f3c98079270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc074590 tx=0x7f3ca4005e50 comp rx=0 tx=0).stop 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.031+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 msgr2=0x7f3cbc1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.031+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c3a0 tx=0x7f3cac00c860 comp rx=0 tx=0).stop 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 shutdown_connections 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c98076db0 0x7f3c98079270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cbc15a280 0x7f3cbc1f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cbc0b1af0 0x7f3cbc1f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:37.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 --2- 172.21.15.44:0/605629312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cbc0ac8a0 0x7f3cbc1569b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 >> 172.21.15.44:0/605629312 conn(0x7f3cbc0b0eb0 msgr2=0x7f3cbc0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 shutdown_connections 2024-07-17T07:07:37.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:37.032+0000 7f3cc2a00640 1 -- 172.21.15.44:0/605629312 wait complete. 2024-07-17T07:07:37.052 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2024-07-17T07:07:37.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.053 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2024-07-17T07:07:37.106 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:37.106 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:37.106 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:37.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:37.108 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:37.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.108 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:37.180 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:37.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.181 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:37.251 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:37.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.251 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-06_59_01_UTC) 2024-07-17T07:07:37.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:37.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.321 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:37.390 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:37.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.390 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:37.460 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:37.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.460 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_02_00_UTC) 2024-07-17T07:07:37.526 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:37.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.527 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_03_01_UTC) 2024-07-17T07:07:37.594 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:37.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.595 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_04_01_UTC) 2024-07-17T07:07:37.663 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:37.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.663 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:37.735 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:37.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.736 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:37.801 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:37.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.802 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:37.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:37.871 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2024-07-17T07:07:37.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 -- 172.21.15.44:0/271933163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad80053c0 msgr2=0x7fcad800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 --2- 172.21.15.44:0/271933163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad80053c0 0x7fcad800b9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcae8001dc0 tx=0x7fcad8005830 comp rx=0 tx=0).stop 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 -- 172.21.15.44:0/271933163 shutdown_connections 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 --2- 172.21.15.44:0/271933163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcad80053c0 0x7fcad800b9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 -- 172.21.15.44:0/271933163 >> 172.21.15.44:0/271933163 conn(0x7fcaf00b3090 msgr2=0x7fcaf00b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 -- 172.21.15.44:0/271933163 shutdown_connections 2024-07-17T07:07:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.153+0000 7fcaf6800640 1 -- 172.21.15.44:0/271933163 wait complete. 2024-07-17T07:07:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.155+0000 7fcaf6800640 1 Processor -- start 2024-07-17T07:07:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.155+0000 7fcaf6800640 1 -- start start 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad80053c0 0x7fcaf0147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf0158c40 0x7fcaf015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0070ef0 con 0x7fcaf01500d0 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0070d70 con 0x7fcaf0158c40 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00b3d90 con 0x7fcad80053c0 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38406/0 (socket says 172.21.15.44:38406) 2024-07-17T07:07:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 -- 172.21.15.44:0/1429483181 learned_addr learned my addr 172.21.15.44:0/1429483181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf5400640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad80053c0 0x7fcaf0147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf5e00640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf0158c40 0x7fcaf015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 -- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad80053c0 msgr2=0x7fcaf0147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad80053c0 0x7fcaf0147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 -- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf0158c40 msgr2=0x7fcaf015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf0158c40 0x7fcaf015d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.156+0000 7fcaf4a00640 1 -- 172.21.15.44:0/1429483181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf0147550 con 0x7fcaf01500d0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaf4a00640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c700 tx=0x7fcadc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc017070 con 0x7fcaf01500d0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcadc005360 con 0x7fcaf01500d0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00a8a0 con 0x7fcaf01500d0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf0159ec0 con 0x7fcaf01500d0 2024-07-17T07:07:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.157+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf015a480 con 0x7fcaf01500d0 2024-07-17T07:07:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.159+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00b55c0 con 0x7fcaf01500d0 2024-07-17T07:07:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.160+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fcadc01a070 con 0x7fcaf01500d0 2024-07-17T07:07:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.161+0000 7fcaed600640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 0x7fcad00792c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.161+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7fcadc097cf0 con 0x7fcaf01500d0 2024-07-17T07:07:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.162+0000 7fcaf5400640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 0x7fcad00792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.163+0000 7fcaf5400640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 0x7fcad00792c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005e00 tx=0x7fcae4005c50 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.165+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcadc060d00 con 0x7fcaf01500d0 2024-07-17T07:07:38.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.409+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fcaf00cf3c0 con 0x7fcad0076e00 2024-07-17T07:07:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.656+0000 7fcaed600640 1 -- 172.21.15.44:0/1429483181 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fcaf00cf3c0 con 0x7fcad0076e00 2024-07-17T07:07:38.655 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir2 2024-07-17T07:07:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.659+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 msgr2=0x7fcad00792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.659+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 0x7fcad00792c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcae4005e00 tx=0x7fcae4005c50 comp rx=0 tx=0).stop 2024-07-17T07:07:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.659+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 msgr2=0x7fcaf0158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.659+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c700 tx=0x7fcadc00cbc0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 shutdown_connections 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fcad0076e00 0x7fcad00792c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf0158c40 0x7fcaf015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf01500d0 0x7fcaf0158700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 --2- 172.21.15.44:0/1429483181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcad80053c0 0x7fcaf0147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 >> 172.21.15.44:0/1429483181 conn(0x7fcaf00b3090 msgr2=0x7fcaf00d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 shutdown_connections 2024-07-17T07:07:38.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:38.660+0000 7fcaf6800640 1 -- 172.21.15.44:0/1429483181 wait complete. 2024-07-17T07:07:38.681 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2024-07-17T07:07:38.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:38.681 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2024-07-17T07:07:38.735 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:38.735 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:38.735 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:38.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:38.737 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:38.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:38.737 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:38.809 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:38.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:38.809 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_58_01_UTC) 2024-07-17T07:07:38.878 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:38.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:38.879 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-06_59_01_UTC) 2024-07-17T07:07:38.947 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:38.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:38.948 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:39.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:39.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.011 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:39.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:39.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.079 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_02_01_UTC) 2024-07-17T07:07:39.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:39.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.149 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:39.217 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:39.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.218 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:39.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:39.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.283 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:39.351 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:39.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.351 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:39.420 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:39.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.421 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2024-07-17-07_07_01_UTC) 2024-07-17T07:07:39.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:39.486 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2024-07-17T07:07:39.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2024-07-17T07:07:39.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 -- 172.21.15.44:0/164081257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 msgr2=0x7faeb00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 --2- 172.21.15.44:0/164081257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb00b4ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faeb00ca490 tx=0x7faea002a580 comp rx=0 tx=0).stop 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 -- 172.21.15.44:0/164081257 shutdown_connections 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 --2- 172.21.15.44:0/164081257 >> v2:172.21.15.175:3300/0 conn(0x7faeb00d2900 0x7faeb00b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 --2- 172.21.15.44:0/164081257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb00b4ec0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.761+0000 7faeb6200640 1 -- 172.21.15.44:0/164081257 >> 172.21.15.44:0/164081257 conn(0x7faeb00cfec0 msgr2=0x7faeb00d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.762+0000 7faeb6200640 1 -- 172.21.15.44:0/164081257 shutdown_connections 2024-07-17T07:07:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.762+0000 7faeb6200640 1 -- 172.21.15.44:0/164081257 wait complete. 2024-07-17T07:07:39.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.763+0000 7faeb6200640 1 Processor -- start 2024-07-17T07:07:39.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.763+0000 7faeb6200640 1 -- start start 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 0x7faeb00b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faeb00d2fa0 0x7faeb01f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faeb00d1220 con 0x7faeb015d300 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faeb006e0c0 con 0x7faeb00d2900 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.764+0000 7faeb6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faeb00b2240 con 0x7faeb00d2fa0 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:39.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59394/0 (socket says 172.21.15.44:59394) 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 -- 172.21.15.44:0/1182061774 learned_addr learned my addr 172.21.15.44:0/1182061774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb4e00640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 0x7faeb00b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 -- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faeb00d2fa0 msgr2=0x7faeb01f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faeb00d2fa0 0x7faeb01f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 -- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 msgr2=0x7faeb00b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 0x7faeb00b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb5800640 1 -- 172.21.15.44:0/1182061774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb00b7430 con 0x7faeb015d300 2024-07-17T07:07:39.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.765+0000 7faeb4e00640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 0x7faeb00b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeb5800640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faea800c970 tx=0x7faea800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8007bb0 con 0x7faeb015d300 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea8007e40 con 0x7faeb015d300 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8005280 con 0x7faeb015d300 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb00b7710 con 0x7faeb015d300 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.766+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb00b7cd0 con 0x7faeb015d300 2024-07-17T07:07:39.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.767+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb006e5e0 con 0x7faeb015d300 2024-07-17T07:07:39.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.768+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7faea8005420 con 0x7faeb015d300 2024-07-17T07:07:39.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.769+0000 7faeacc00640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 0x7fae8c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:39.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.769+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7faea8098f00 con 0x7faeb015d300 2024-07-17T07:07:39.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.770+0000 7faeb4e00640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 0x7fae8c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:39.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.771+0000 7faeb4e00640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 0x7fae8c079270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea0033000 tx=0x7faea0004730 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:39.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:39.774+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faea8014030 con 0x7faeb015d300 2024-07-17T07:07:40.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.014+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7faeb00d13a0 con 0x7fae8c076db0 2024-07-17T07:07:40.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.260+0000 7faeacc00640 1 -- 172.21.15.44:0/1182061774 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7faea0004390 con 0x7fae8c076db0 2024-07-17T07:07:40.259 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir3 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.263+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 msgr2=0x7fae8c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.263+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 0x7fae8c079270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faea0033000 tx=0x7faea0004730 comp rx=0 tx=0).stop 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 msgr2=0x7faeb01f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faea800c970 tx=0x7faea800ce30 comp rx=0 tx=0).stop 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 shutdown_connections 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fae8c076db0 0x7fae8c079270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faeb015d300 0x7faeb01f7910 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faeb00d2fa0 0x7faeb01f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 --2- 172.21.15.44:0/1182061774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faeb00d2900 0x7faeb00b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 >> 172.21.15.44:0/1182061774 conn(0x7faeb00cfec0 msgr2=0x7faeb00cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 shutdown_connections 2024-07-17T07:07:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:40.264+0000 7faeb6200640 1 -- 172.21.15.44:0/1182061774 wait complete. 2024-07-17T07:07:40.284 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2024-07-17T07:07:40.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.285 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2024-07-17T07:07:40.335 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:40.335 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:40.335 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:40.336 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:40.337 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:40.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.337 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_57_01_UTC) 2024-07-17T07:07:40.404 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:40.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.405 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:40.477 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:40.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.477 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:40.547 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:40.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.547 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_00_01_UTC) 2024-07-17T07:07:40.616 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:40.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.616 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:40.689 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:40.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.689 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_02_01_UTC) 2024-07-17T07:07:40.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:40.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.758 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:40.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:40.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.825 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_04_01_UTC) 2024-07-17T07:07:40.892 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:40.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.893 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:40.979 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:40.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:40.979 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_06_01_UTC) 2024-07-17T07:07:41.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:41.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:41.077 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:41.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:41.189 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2024-07-17T07:07:41.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2024-07-17T07:07:41.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.493+0000 7f6981a00640 1 -- 172.21.15.44:0/849703983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c06ed30 msgr2=0x7f697c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:41.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.493+0000 7f6981a00640 1 --2- 172.21.15.44:0/849703983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c06ed30 0x7f697c06d620 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6974005fd0 tx=0x7f697402aa90 comp rx=0 tx=0).stop 2024-07-17T07:07:41.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.493+0000 7f6981a00640 1 -- 172.21.15.44:0/849703983 shutdown_connections 2024-07-17T07:07:41.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.494+0000 7f6981a00640 1 --2- 172.21.15.44:0/849703983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c06ed30 0x7f697c06d620 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:41.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.494+0000 7f6981a00640 1 -- 172.21.15.44:0/849703983 >> 172.21.15.44:0/849703983 conn(0x7f697c0b0eb0 msgr2=0x7f697c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:41.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.494+0000 7f6981a00640 1 -- 172.21.15.44:0/849703983 shutdown_connections 2024-07-17T07:07:41.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.494+0000 7f6981a00640 1 -- 172.21.15.44:0/849703983 wait complete. 2024-07-17T07:07:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.495+0000 7f6981a00640 1 Processor -- start 2024-07-17T07:07:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.495+0000 7f6981a00640 1 -- start start 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c06ed30 0x7f697c1f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c152220 0x7f697c1f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f697c0cd9f0 con 0x7f697c0cbab0 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f697c15c800 con 0x7f697c152220 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.496+0000 7f6981a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f697c0cdb70 con 0x7f697c06ed30 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59408/0 (socket says 172.21.15.44:59408) 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 -- 172.21.15.44:0/1145595549 learned_addr learned my addr 172.21.15.44:0/1145595549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f697aa00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c06ed30 0x7f697c1f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f697b400640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c152220 0x7f697c1f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 -- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c06ed30 msgr2=0x7f697c1f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c06ed30 0x7f697c1f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 -- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c152220 msgr2=0x7f697c1f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c152220 0x7f697c1f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.497+0000 7f6973e00640 1 -- 172.21.15.44:0/1145595549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c14e360 con 0x7f697c0cbab0 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6973e00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f696400c480 tx=0x7f696400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69640052d0 con 0x7f697c0cbab0 2024-07-17T07:07:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6964005470 con 0x7f697c0cbab0 2024-07-17T07:07:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964012870 con 0x7f697c0cbab0 2024-07-17T07:07:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c14e640 con 0x7f697c0cbab0 2024-07-17T07:07:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.498+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c14ec00 con 0x7f697c0cbab0 2024-07-17T07:07:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.499+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c06a1a0 con 0x7f697c0cbab0 2024-07-17T07:07:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.501+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f69640035f0 con 0x7f697c0cbab0 2024-07-17T07:07:41.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.502+0000 7f6972a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 0x7f69580792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:41.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.502+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 7244+0+0 (secure 0 0 0) 0x7f69640988d0 con 0x7f697c0cbab0 2024-07-17T07:07:41.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.503+0000 7f697aa00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 0x7f69580792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:41.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.504+0000 7f697aa00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 0x7f69580792a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f696c006fd0 tx=0x7f696c008040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.507+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f697c06a1a0 con 0x7f697c0cbab0 2024-07-17T07:07:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:41.733+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f697c0ce090 con 0x7f6958076de0 2024-07-17T07:07:42.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.012+0000 7f6972a00640 1 -- 172.21.15.44:0/1145595549 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f697c0ce090 con 0x7f6958076de0 2024-07-17T07:07:42.012 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir4 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 msgr2=0x7f69580792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 0x7f69580792a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f696c006fd0 tx=0x7f696c008040 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 msgr2=0x7f697c14daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f696400c480 tx=0x7f696400c940 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 shutdown_connections 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f6958076de0 0x7f69580792a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f697c152220 0x7f697c1f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f697c0cbab0 0x7f697c14daf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 --2- 172.21.15.44:0/1145595549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697c06ed30 0x7f697c1f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.015+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 >> 172.21.15.44:0/1145595549 conn(0x7f697c0b0eb0 msgr2=0x7f697c0add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:42.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.016+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 shutdown_connections 2024-07-17T07:07:42.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:42.016+0000 7f6981a00640 1 -- 172.21.15.44:0/1145595549 wait complete. 2024-07-17T07:07:42.039 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2024-07-17T07:07:42.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.039 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2024-07-17T07:07:42.098 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:42.098 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:42.098 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:42.099 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:42.100 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:42.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.100 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:42.145 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:42.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.145 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:42.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:42.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.239 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:42.297 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:42.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.297 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:42.358 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:42.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.359 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:42.423 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:42.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.423 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_02_01_UTC) 2024-07-17T07:07:42.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:42.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.487 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_03_01_UTC) 2024-07-17T07:07:42.557 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_04_01_UTC 2024-07-17T07:07:42.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.558 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_04_01_UTC) 2024-07-17T07:07:42.616 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:42.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.617 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:42.670 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:42.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.670 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:42.727 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:42.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.728 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2024-07-17-07_07_01_UTC) 2024-07-17T07:07:42.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:42.789 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2024-07-17T07:07:42.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2024-07-17T07:07:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 -- 172.21.15.44:0/2607583807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc152220 msgr2=0x7f4bbc06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 --2- 172.21.15.44:0/2607583807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc152220 0x7f4bbc06d620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0cce50 tx=0x7f4bac02a4d0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 -- 172.21.15.44:0/2607583807 shutdown_connections 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 --2- 172.21.15.44:0/2607583807 >> v2:172.21.15.175:3300/0 conn(0x7f4bbc0cbab0 0x7f4bbc15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 --2- 172.21.15.44:0/2607583807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc152220 0x7f4bbc06d620 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 -- 172.21.15.44:0/2607583807 >> 172.21.15.44:0/2607583807 conn(0x7f4bbc0b0eb0 msgr2=0x7f4bbc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 -- 172.21.15.44:0/2607583807 shutdown_connections 2024-07-17T07:07:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.073+0000 7f4bc2800640 1 -- 172.21.15.44:0/2607583807 wait complete. 2024-07-17T07:07:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.075+0000 7f4bc2800640 1 Processor -- start 2024-07-17T07:07:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.075+0000 7f4bc2800640 1 -- start start 2024-07-17T07:07:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 0x7f4bbc152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 0x7f4bbc141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc0cd7b0 con 0x7f4bbc06ed30 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc0cd630 con 0x7f4bbc0cbab0 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.076+0000 7f4bc2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc0cd930 con 0x7f4bbc152220 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59438/0 (socket says 172.21.15.44:59438) 2024-07-17T07:07:43.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 -- 172.21.15.44:0/225817372 learned_addr learned my addr 172.21.15.44:0/225817372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 -- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 msgr2=0x7f4bbc141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbbe00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 0x7f4bbc141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bb2a00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 0x7f4bbc152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 0x7f4bbc141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 -- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 msgr2=0x7f4bbc152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 0x7f4bbc152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbb400640 1 -- 172.21.15.44:0/225817372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc1533e0 con 0x7f4bbc06ed30 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bbbe00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 0x7f4bbc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.077+0000 7f4bb2a00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 0x7f4bbc152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bbb400640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4bac034040 tx=0x7f4bac0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac03e030 con 0x7f4bbc06ed30 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bac0047a0 con 0x7f4bbc06ed30 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac035040 con 0x7f4bbc06ed30 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc1f2150 con 0x7f4bbc06ed30 2024-07-17T07:07:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.078+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1f2710 con 0x7f4bbc06ed30 2024-07-17T07:07:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.079+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bbc0cde50 con 0x7f4bbc06ed30 2024-07-17T07:07:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.080+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f4bac041030 con 0x7f4bbc06ed30 2024-07-17T07:07:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.081+0000 7f4bb3e00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 0x7f4b98079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.081+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6940+0+0 (secure 0 0 0) 0x7f4bac0b9360 con 0x7f4bbc06ed30 2024-07-17T07:07:43.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.082+0000 7f4bb2a00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 0x7f4b98079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.083+0000 7f4bb2a00640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 0x7f4b98079270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0b05f0 tx=0x7f4ba4009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.086+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4bac081fc0 con 0x7f4bbc06ed30 2024-07-17T07:07:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.336+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4bbc141700 con 0x7f4b98076db0 2024-07-17T07:07:43.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.578+0000 7f4bb3e00640 1 -- 172.21.15.44:0/225817372 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f4bbc141700 con 0x7f4b98076db0 2024-07-17T07:07:43.577 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir5 2024-07-17T07:07:43.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 msgr2=0x7f4b98079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:43.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 0x7f4b98079270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0b05f0 tx=0x7f4ba4009040 comp rx=0 tx=0).stop 2024-07-17T07:07:43.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 msgr2=0x7f4bbc1411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:43.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4bac034040 tx=0x7f4bac0039e0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 shutdown_connections 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f4b98076db0 0x7f4b98079270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bbc152220 0x7f4bbc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bbc0cbab0 0x7f4bbc152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 --2- 172.21.15.44:0/225817372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bbc06ed30 0x7f4bbc1411c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 >> 172.21.15.44:0/225817372 conn(0x7f4bbc0b0eb0 msgr2=0x7f4bbc15f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.582+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 shutdown_connections 2024-07-17T07:07:43.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:43.583+0000 7f4bc2800640 1 -- 172.21.15.44:0/225817372 wait complete. 2024-07-17T07:07:43.603 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2024-07-17T07:07:43.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.603 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:43.660 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:43.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:43.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.661 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:43.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:43.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.731 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:43.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:43.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.792 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:43.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:43.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.852 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:43.917 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:43.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.918 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:43.987 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:43.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:43.988 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_02_01_UTC) 2024-07-17T07:07:44.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_03_01_UTC 2024-07-17T07:07:44.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.060 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_03_01_UTC) 2024-07-17T07:07:44.115 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:44.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.116 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:44.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:44.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.178 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_05_01_UTC) 2024-07-17T07:07:44.223 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:44.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.224 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_06_01_UTC) 2024-07-17T07:07:44.323 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:44.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.323 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:44.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:07:44.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:07:44.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:44.390 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2024-07-17T07:07:44.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.725+0000 7ff48f200640 1 -- 172.21.15.44:0/823587284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49015b3a0 msgr2=0x7ff4900d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.725+0000 7ff48f200640 1 --2- 172.21.15.44:0/823587284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49015b3a0 0x7ff4900d0ca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff4900b5510 tx=0x7ff48002a510 comp rx=0 tx=0).stop 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 -- 172.21.15.44:0/823587284 shutdown_connections 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 --2- 172.21.15.44:0/823587284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b23e0 0x7ff4900ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 --2- 172.21.15.44:0/823587284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49015b3a0 0x7ff4900d0ca0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 -- 172.21.15.44:0/823587284 >> 172.21.15.44:0/823587284 conn(0x7ff4900b5560 msgr2=0x7ff4900b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 -- 172.21.15.44:0/823587284 shutdown_connections 2024-07-17T07:07:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.726+0000 7ff48f200640 1 -- 172.21.15.44:0/823587284 wait complete. 2024-07-17T07:07:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.728+0000 7ff48f200640 1 Processor -- start 2024-07-17T07:07:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.728+0000 7ff48f200640 1 -- start start 2024-07-17T07:07:44.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.729+0000 7ff48f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.729+0000 7ff48f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b3040 0x7ff49009c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 0x7ff49009c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900aeff0 con 0x7ff4900b23e0 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900aee70 con 0x7ff49015b3a0 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900af170 con 0x7ff4900b3040 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59452/0 (socket says 172.21.15.44:59452) 2024-07-17T07:07:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 -- 172.21.15.44:0/1582264377 learned_addr learned my addr 172.21.15.44:0/1582264377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:44.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48d400640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b3040 0x7ff49009c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48e800640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 0x7ff49009c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 -- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b3040 msgr2=0x7ff49009c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b3040 0x7ff49009c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 -- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 msgr2=0x7ff49009c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 0x7ff49009c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48de00640 1 -- 172.21.15.44:0/1582264377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900ce880 con 0x7ff4900b23e0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.730+0000 7ff48e800640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 0x7ff49009c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff48de00640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff480034040 tx=0x7ff4800039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48003e030 con 0x7ff4900b23e0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4800047a0 con 0x7ff4900b23e0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480035040 con 0x7ff4900b23e0 2024-07-17T07:07:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900ceb60 con 0x7ff4900b23e0 2024-07-17T07:07:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.731+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4900cf120 con 0x7ff4900b23e0 2024-07-17T07:07:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.733+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490151cc0 con 0x7ff4900b23e0 2024-07-17T07:07:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.734+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7ff480041030 con 0x7ff4900b23e0 2024-07-17T07:07:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.735+0000 7ff486000640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 0x7ff470079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.735+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 7116+0+0 (secure 0 0 0) 0x7ff4800b9430 con 0x7ff4900b23e0 2024-07-17T07:07:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.736+0000 7ff48d400640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 0x7ff470079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:44.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.737+0000 7ff48d400640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 0x7ff470079270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff49015e620 tx=0x7ff47c009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:44.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.740+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff480081fe0 con 0x7ff4900b23e0 2024-07-17T07:07:44.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:44.974+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff49015e650 con 0x7ff470076db0 2024-07-17T07:07:45.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.256+0000 7ff486000640 1 -- 172.21.15.44:0/1582264377 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7ff49015e650 con 0x7ff470076db0 2024-07-17T07:07:45.257 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir6 2024-07-17T07:07:45.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.263+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 msgr2=0x7ff470079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:45.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.263+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 0x7ff470079270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff49015e620 tx=0x7ff47c009040 comp rx=0 tx=0).stop 2024-07-17T07:07:45.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.263+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 msgr2=0x7ff490151780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:45.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.263+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff480034040 tx=0x7ff4800039e0 comp rx=0 tx=0).stop 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 shutdown_connections 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7ff470076db0 0x7ff470079270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49015b3a0 0x7ff49009c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4900b3040 0x7ff49009c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 --2- 172.21.15.44:0/1582264377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4900b23e0 0x7ff490151780 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 >> 172.21.15.44:0/1582264377 conn(0x7ff4900b5560 msgr2=0x7ff4900b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 shutdown_connections 2024-07-17T07:07:45.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:45.264+0000 7ff48f200640 1 -- 172.21.15.44:0/1582264377 wait complete. 2024-07-17T07:07:45.286 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2024-07-17T07:07:45.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.287 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2024-07-17T07:07:45.327 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:45.327 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:45.328 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:45.329 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:45.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.329 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_57_01_UTC) 2024-07-17T07:07:45.432 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:45.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.432 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_58_01_UTC) 2024-07-17T07:07:45.498 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:45.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.498 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:45.563 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:45.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.563 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:45.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:45.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.631 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:45.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:45.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.698 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_02_00_UTC) 2024-07-17T07:07:45.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:45.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.762 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:45.828 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.828 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:45.898 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:45.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.898 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_05_01_UTC) 2024-07-17T07:07:45.964 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:45.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:45.965 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:46.061 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:46.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:46.061 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:46.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:46.128 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2024-07-17T07:07:46.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 -- 172.21.15.44:0/33952551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a8003a70 msgr2=0x7f56a8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 --2- 172.21.15.44:0/33952551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a8003a70 0x7f56a8003e90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f56c4070830 tx=0x7f56a8005830 comp rx=0 tx=0).stop 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56c9600640 1 -- 172.21.15.44:0/33952551 reap_dead start 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 -- 172.21.15.44:0/33952551 shutdown_connections 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 -- 172.21.15.44:0/33952551 >> 172.21.15.44:0/33952551 conn(0x7f56c40cf0d0 msgr2=0x7f56c40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 -- 172.21.15.44:0/33952551 shutdown_connections 2024-07-17T07:07:46.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.411+0000 7f56ca000640 1 -- 172.21.15.44:0/33952551 wait complete. 2024-07-17T07:07:46.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.413+0000 7f56ca000640 1 Processor -- start 2024-07-17T07:07:46.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.413+0000 7f56ca000640 1 -- start start 2024-07-17T07:07:46.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a8003a70 0x7f56c4098b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c41e4420 0x7f56c41e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56c406f650 con 0x7f56c414c810 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56c406f4d0 con 0x7f56c41e4420 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56ca000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56c415a450 con 0x7f56a8003a70 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59466/0 (socket says 172.21.15.44:59466) 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.414+0000 7f56c3e00640 1 -- 172.21.15.44:0/890838741 learned_addr learned my addr 172.21.15.44:0/890838741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c8c00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a8003a70 0x7f56c4098b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c9600640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c41e4420 0x7f56c41e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 -- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a8003a70 msgr2=0x7f56c4098b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:46.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a8003a70 0x7f56c4098b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 -- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c41e4420 msgr2=0x7f56c41e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:46.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c41e4420 0x7f56c41e48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 -- 172.21.15.44:0/890838741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c414abc0 con 0x7f56c414c810 2024-07-17T07:07:46.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c3e00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d8f0 tx=0x7f56ac00ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00b830 con 0x7f56c414c810 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56ac00bac0 con 0x7f56c414c810 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0035f0 con 0x7f56c414c810 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.415+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c4073140 con 0x7f56c414c810 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.416+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c4073700 con 0x7f56c414c810 2024-07-17T07:07:46.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.417+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c406d8d0 con 0x7f56c414c810 2024-07-17T07:07:46.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.418+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f56ac003790 con 0x7f56c414c810 2024-07-17T07:07:46.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.420+0000 7f56c0c00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 0x7f56a0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:46.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.420+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 7020+0+0 (secure 0 0 0) 0x7f56ac0990e0 con 0x7f56c414c810 2024-07-17T07:07:46.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.420+0000 7f56c8c00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 0x7f56a0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.421+0000 7f56c8c00640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 0x7f56a0079270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56b4036000 tx=0x7f56b40046c0 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:46.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.424+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f56ac061e00 con 0x7f56c414c810 2024-07-17T07:07:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.661+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f56c40cffe0 con 0x7f56a0076db0 2024-07-17T07:07:46.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.919+0000 7f56c0c00640 1 -- 172.21.15.44:0/890838741 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f56c40cffe0 con 0x7f56a0076db0 2024-07-17T07:07:46.919 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir7 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 msgr2=0x7f56a0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 0x7f56a0079270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56b4036000 tx=0x7f56b40046c0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 msgr2=0x7f56c414cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d8f0 tx=0x7f56ac00ddb0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 shutdown_connections 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f56a0076db0 0x7f56a0079270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56c41e4420 0x7f56c41e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56c414c810 0x7f56c414cc90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 --2- 172.21.15.44:0/890838741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56a8003a70 0x7f56c4098b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 >> 172.21.15.44:0/890838741 conn(0x7f56c40cf0d0 msgr2=0x7f56c414f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 shutdown_connections 2024-07-17T07:07:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:46.922+0000 7f56ca000640 1 -- 172.21.15.44:0/890838741 wait complete. 2024-07-17T07:07:46.942 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2024-07-17T07:07:46.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:46.942 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2024-07-17T07:07:47.005 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:47.007 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:47.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.008 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:47.071 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:47.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.071 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:47.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_59_01_UTC 2024-07-17T07:07:47.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.132 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-06_59_01_UTC) 2024-07-17T07:07:47.195 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_00_01_UTC 2024-07-17T07:07:47.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.195 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_00_01_UTC) 2024-07-17T07:07:47.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_01_00_UTC 2024-07-17T07:07:47.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.242 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_01_00_UTC) 2024-07-17T07:07:47.339 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:47.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.340 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_02_00_UTC) 2024-07-17T07:07:47.394 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:47.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.394 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:47.453 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:47.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.454 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:47.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_05_01_UTC 2024-07-17T07:07:47.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.513 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_05_01_UTC) 2024-07-17T07:07:47.573 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_06_00_UTC 2024-07-17T07:07:47.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.573 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_06_00_UTC) 2024-07-17T07:07:47.635 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:47.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.635 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2024-07-17-07_07_01_UTC) 2024-07-17T07:07:47.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:47.697 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2024-07-17T07:07:47.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 -- 172.21.15.44:0/4116766292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 msgr2=0x7fd110150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 --2- 172.21.15.44:0/4116766292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd110150470 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd108005fd0 tx=0x7fd10802b980 comp rx=0 tx=0).stop 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 -- 172.21.15.44:0/4116766292 shutdown_connections 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 --2- 172.21.15.44:0/4116766292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd110150470 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 -- 172.21.15.44:0/4116766292 >> 172.21.15.44:0/4116766292 conn(0x7fd1100b0eb0 msgr2=0x7fd1100b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.968+0000 7fd114e00640 1 -- 172.21.15.44:0/4116766292 shutdown_connections 2024-07-17T07:07:47.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.969+0000 7fd114e00640 1 -- 172.21.15.44:0/4116766292 wait complete. 2024-07-17T07:07:47.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.971+0000 7fd114e00640 1 Processor -- start 2024-07-17T07:07:47.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.971+0000 7fd114e00640 1 -- start start 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.972+0000 7fd114e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd11006f6d0 0x7fd1100cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd114e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 0x7fd1100cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd114e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd114e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd11006ed10 con 0x7fd11015a5a0 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd114e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd11006eb90 con 0x7fd1100acab0 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd114e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd11006ee90 con 0x7fd11006f6d0 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59482/0 (socket says 172.21.15.44:59482) 2024-07-17T07:07:47.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 -- 172.21.15.44:0/2483894911 learned_addr learned my addr 172.21.15.44:0/2483894911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10de00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd11006f6d0 0x7fd1100cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:47.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10d400640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 0x7fd1100cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 -- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd11006f6d0 msgr2=0x7fd1100cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd11006f6d0 0x7fd1100cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 -- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 msgr2=0x7fd1100cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.973+0000 7fd10e800640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 0x7fd1100cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd10e800640 1 -- 172.21.15.44:0/2483894911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101f7bc0 con 0x7fd11015a5a0 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd10d400640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 0x7fd1100cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd10e800640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd10800aea0 tx=0x7fd108003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108035050 con 0x7fd11015a5a0 2024-07-17T07:07:47.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1080045e0 con 0x7fd11015a5a0 2024-07-17T07:07:47.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1100ce860 con 0x7fd11015a5a0 2024-07-17T07:07:47.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108045480 con 0x7fd11015a5a0 2024-07-17T07:07:47.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.974+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1100ce580 con 0x7fd11015a5a0 2024-07-17T07:07:47.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.976+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1101f7d90 con 0x7fd11015a5a0 2024-07-17T07:07:47.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.977+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fd108038020 con 0x7fd11015a5a0 2024-07-17T07:07:47.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.978+0000 7fd106000640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 0x7fd0ec079260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:47.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.978+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 7052+0+0 (secure 0 0 0) 0x7fd1080ba400 con 0x7fd11015a5a0 2024-07-17T07:07:47.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.979+0000 7fd10de00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 0x7fd0ec079260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:47.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.980+0000 7fd10de00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 0x7fd0ec079260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd100009040 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:47.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:47.983+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd108083080 con 0x7fd11015a5a0 2024-07-17T07:07:48.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.247+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd1100ac3a0 con 0x7fd0ec076da0 2024-07-17T07:07:48.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.507+0000 7fd106000640 1 -- 172.21.15.44:0/2483894911 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fd1100ac3a0 con 0x7fd0ec076da0 2024-07-17T07:07:48.509 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir8 2024-07-17T07:07:48.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.513+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 msgr2=0x7fd0ec079260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:48.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.513+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 0x7fd0ec079260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd100009040 comp rx=0 tx=0).stop 2024-07-17T07:07:48.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.513+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 msgr2=0x7fd11014dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:48.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.513+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd10800aea0 tx=0x7fd108003f90 comp rx=0 tx=0).stop 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 shutdown_connections 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fd0ec076da0 0x7fd0ec079260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd11015a5a0 0x7fd11014dd70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1100acab0 0x7fd1100cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 --2- 172.21.15.44:0/2483894911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd11006f6d0 0x7fd1100cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 >> 172.21.15.44:0/2483894911 conn(0x7fd1100b0eb0 msgr2=0x7fd1100b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.514+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 shutdown_connections 2024-07-17T07:07:48.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:48.515+0000 7fd114e00640 1 -- 172.21.15.44:0/2483894911 wait complete. 2024-07-17T07:07:48.549 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2024-07-17T07:07:48.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.549 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2024-07-17T07:07:48.602 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:48.603 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:48.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_57_00_UTC 2024-07-17T07:07:48.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.604 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_57_00_UTC) 2024-07-17T07:07:48.671 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_58_00_UTC 2024-07-17T07:07:48.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.672 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_58_00_UTC) 2024-07-17T07:07:48.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:48.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.727 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:48.783 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:48.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.784 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:48.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:48.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.852 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_01_01_UTC) 2024-07-17T07:07:48.913 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_02_00_UTC 2024-07-17T07:07:48.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.913 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_02_00_UTC) 2024-07-17T07:07:48.977 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:48.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:48.978 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:49.046 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:49.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:49.046 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:49.102 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:49.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:49.102 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:49.159 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:49.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:49.159 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_06_01_UTC) 2024-07-17T07:07:49.221 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_07_00_UTC 2024-07-17T07:07:49.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:49.221 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2024-07-17-07_07_00_UTC) 2024-07-17T07:07:49.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:49.296 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2024-07-17T07:07:49.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 -- 172.21.15.44:0/877371399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c240cb3a0 msgr2=0x7f3c240cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 --2- 172.21.15.44:0/877371399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c240cb3a0 0x7f3c240cb830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3c24070b60 tx=0x7f3c1c02a980 comp rx=0 tx=0).stop 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 -- 172.21.15.44:0/877371399 shutdown_connections 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 --2- 172.21.15.44:0/877371399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c240cb3a0 0x7f3c240cb830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 --2- 172.21.15.44:0/877371399 >> v2:172.21.15.175:3300/0 conn(0x7f3c2406ed30 0x7f3c2415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 --2- 172.21.15.44:0/877371399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c24152220 0x7f3c2406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:49.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 -- 172.21.15.44:0/877371399 >> 172.21.15.44:0/877371399 conn(0x7f3c240b0eb0 msgr2=0x7f3c240b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:49.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 -- 172.21.15.44:0/877371399 shutdown_connections 2024-07-17T07:07:49.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.572+0000 7f3c2c600640 1 -- 172.21.15.44:0/877371399 wait complete. 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.574+0000 7f3c2c600640 1 Processor -- start 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.574+0000 7f3c2c600640 1 -- start start 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240cb3a0 0x7f3c241f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c24152220 0x7f3c241565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:49.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2415f1f0 con 0x7f3c2406ed30 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2415f070 con 0x7f3c24152220 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2415f370 con 0x7f3c240cb3a0 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36994/0 (socket says 172.21.15.44:36994) 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c29c00640 1 -- 172.21.15.44:0/2249763031 learned_addr learned my addr 172.21.15.44:0/2249763031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c29200640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240cb3a0 0x7f3c241f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.575+0000 7f3c2a600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c24152220 0x7f3c241565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 -- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240cb3a0 msgr2=0x7f3c241f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240cb3a0 0x7f3c241f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 -- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c24152220 msgr2=0x7f3c241565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c24152220 0x7f3c241565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 -- 172.21.15.44:0/2249763031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c241f8820 con 0x7f3c2406ed30 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c29c00640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800d770 tx=0x7f3c1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:49.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18017070 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c18009d20 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c24157890 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.576+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180106c0 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.577+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c24157e50 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.578+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c240cb830 con 0x7f3c2406ed30 2024-07-17T07:07:49.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.579+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7f3c1801a070 con 0x7f3c2406ed30 2024-07-17T07:07:49.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.580+0000 7f3c21600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 0x7f3c080792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:49.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.580+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 7560+0+0 (secure 0 0 0) 0x7f3c18098f80 con 0x7f3c2406ed30 2024-07-17T07:07:49.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.581+0000 7f3c29200640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 0x7f3c080792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:49.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.582+0000 7f3c29200640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 0x7f3c080792a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005fd0 tx=0x7f3c14005850 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:49.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.585+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3c180618f0 con 0x7f3c2406ed30 2024-07-17T07:07:49.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:49.822+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 --> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3c240b3c00 con 0x7f3c08076de0 2024-07-17T07:07:50.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.067+0000 7f3c21600640 1 -- 172.21.15.44:0/2249763031 <== mgr.4700 v2:172.21.15.175:6800/2568152110 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f3c240b3c00 con 0x7f3c08076de0 2024-07-17T07:07:50.068 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/dir9 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 msgr2=0x7f3c080792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 0x7f3c080792a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005fd0 tx=0x7f3c14005850 comp rx=0 tx=0).stop 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 msgr2=0x7f3c240740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800d770 tx=0x7f3c1800dc30 comp rx=0 tx=0).stop 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 shutdown_connections 2024-07-17T07:07:50.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f3c08076de0 0x7f3c080792a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c24152220 0x7f3c241565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c240cb3a0 0x7f3c241f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 --2- 172.21.15.44:0/2249763031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2406ed30 0x7f3c240740a0 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800d770 tx=0x7f3c1800dc30 comp rx=0 tx=0).stop 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 >> 172.21.15.44:0/2249763031 conn(0x7f3c240b0eb0 msgr2=0x7f3c240b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.072+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 shutdown_connections 2024-07-17T07:07:50.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:50.073+0000 7f3c2c600640 1 -- 172.21.15.44:0/2249763031 wait complete. 2024-07-17T07:07:50.094 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2024-07-17T07:07:50.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.094 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:50.146 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_57_01_UTC 2024-07-17T07:07:50.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.146 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_57_01_UTC) 2024-07-17T07:07:50.222 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_58_01_UTC 2024-07-17T07:07:50.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.222 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_58_01_UTC) 2024-07-17T07:07:50.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_59_00_UTC 2024-07-17T07:07:50.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.295 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-06_59_00_UTC) 2024-07-17T07:07:50.368 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_00_00_UTC 2024-07-17T07:07:50.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.369 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_00_00_UTC) 2024-07-17T07:07:50.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_01_01_UTC 2024-07-17T07:07:50.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.442 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_01_01_UTC) 2024-07-17T07:07:50.514 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_02_01_UTC 2024-07-17T07:07:50.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.515 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_02_01_UTC) 2024-07-17T07:07:50.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_03_00_UTC 2024-07-17T07:07:50.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.589 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_03_00_UTC) 2024-07-17T07:07:50.664 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_04_00_UTC 2024-07-17T07:07:50.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.664 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_04_00_UTC) 2024-07-17T07:07:50.738 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_05_00_UTC 2024-07-17T07:07:50.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.739 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_05_00_UTC) 2024-07-17T07:07:50.807 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_06_01_UTC 2024-07-17T07:07:50.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.807 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_06_01_UTC) 2024-07-17T07:07:50.870 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_07_01_UTC 2024-07-17T07:07:50.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.870 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2024-07-17-07_07_01_UTC) 2024-07-17T07:07:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:50.944 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2024-07-17T07:07:51.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:07:51.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.266+0000 7fdf54a00640 1 -- 172.21.15.44:0/2744975380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf500b3280 msgr2=0x7fdf500ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:51.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.266+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2744975380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf500b3280 0x7fdf500ae560 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf500b41a0 tx=0x7fdf3c02a6f0 comp rx=0 tx=0).stop 2024-07-17T07:07:51.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.267+0000 7fdf54a00640 1 -- 172.21.15.44:0/2744975380 shutdown_connections 2024-07-17T07:07:51.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.267+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2744975380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf500b3280 0x7fdf500ae560 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:51.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.267+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2744975380 >> v2:172.21.15.44:3300/0 conn(0x7fdf5006f6d0 0x7fdf5015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:51.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.267+0000 7fdf54a00640 1 -- 172.21.15.44:0/2744975380 >> 172.21.15.44:0/2744975380 conn(0x7fdf500b0eb0 msgr2=0x7fdf500b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:51.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.267+0000 7fdf54a00640 1 -- 172.21.15.44:0/2744975380 shutdown_connections 2024-07-17T07:07:51.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.268+0000 7fdf54a00640 1 -- 172.21.15.44:0/2744975380 wait complete. 2024-07-17T07:07:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.273+0000 7fdf54a00640 1 Processor -- start 2024-07-17T07:07:51.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.273+0000 7fdf54a00640 1 -- start start 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf5006e920 0x7fdf501494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 0x7fdf500cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 0x7fdf501513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5006f0b0 con 0x7fdf500b3280 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5006ef30 con 0x7fdf5006f6d0 2024-07-17T07:07:51.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.275+0000 7fdf54a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5014fd60 con 0x7fdf5006e920 2024-07-17T07:07:51.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.276+0000 7fdf4de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf5006e920 0x7fdf501494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:51.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.276+0000 7fdf4d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 0x7fdf500cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:51.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.276+0000 7fdf4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 0x7fdf501513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:51.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.276+0000 7fdf4d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 0x7fdf500cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:36032/0 (socket says 172.21.15.44:36032) 2024-07-17T07:07:51.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.277+0000 7fdf4d400640 1 -- 172.21.15.44:0/2385156884 learned_addr learned my addr 172.21.15.44:0/2385156884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.278+0000 7fdf4e800640 1 -- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf5006e920 msgr2=0x7fdf501494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.278+0000 7fdf4e800640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf5006e920 0x7fdf501494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.278+0000 7fdf4e800640 1 -- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 msgr2=0x7fdf500cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.278+0000 7fdf4e800640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 0x7fdf500cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.278+0000 7fdf4e800640 1 -- 172.21.15.44:0/2385156884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf50150310 con 0x7fdf500b3280 2024-07-17T07:07:51.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.279+0000 7fdf4e800640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 0x7fdf501513b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c680 tx=0x7fdf4800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:51.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.279+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48003b70 con 0x7fdf500b3280 2024-07-17T07:07:51.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.280+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf48003d10 con 0x7fdf500b3280 2024-07-17T07:07:51.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.280+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48005330 con 0x7fdf500b3280 2024-07-17T07:07:51.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.280+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501505f0 con 0x7fdf500b3280 2024-07-17T07:07:51.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.281+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf50150bb0 con 0x7fdf500b3280 2024-07-17T07:07:51.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.284+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf28005290 con 0x7fdf500b3280 2024-07-17T07:07:51.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.289+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) ==== 99017+0+0 (secure 0 0 0) 0x7fdf480054d0 con 0x7fdf500b3280 2024-07-17T07:07:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.291+0000 7fdf46000640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 0x7fdf300792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:51.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.291+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 7084+0+0 (secure 0 0 0) 0x7fdf48098870 con 0x7fdf500b3280 2024-07-17T07:07:51.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.291+0000 7fdf4de00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 0x7fdf300792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.295+0000 7fdf4de00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 0x7fdf300792a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bbe0 tx=0x7fdf4000b390 comp rx=0 tx=0).ready entity=mgr.4700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.298+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdf48061550 con 0x7fdf500b3280 2024-07-17T07:07:51.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:51.564+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fdf28005850 con 0x7fdf500b3280 2024-07-17T07:07:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.065+0000 7fdf46000640 1 -- 172.21.15.44:0/2385156884 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) ==== 93+0+0 (secure 0 0 0) 0x7fdf48060e50 con 0x7fdf500b3280 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.069+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 msgr2=0x7fdf300792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.069+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 0x7fdf300792a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000bbe0 tx=0x7fdf4000b390 comp rx=0 tx=0).stop 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.069+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 msgr2=0x7fdf501513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.069+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 0x7fdf501513b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c680 tx=0x7fdf4800cb40 comp rx=0 tx=0).stop 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 shutdown_connections 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fdf30076de0 0x7fdf300792a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf500b3280 0x7fdf501513b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf5006f6d0 0x7fdf500cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 --2- 172.21.15.44:0/2385156884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf5006e920 0x7fdf501494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 >> 172.21.15.44:0/2385156884 conn(0x7fdf500b0eb0 msgr2=0x7fdf5015c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:52.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.070+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 shutdown_connections 2024-07-17T07:07:52.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.071+0000 7fdf54a00640 1 -- 172.21.15.44:0/2385156884 wait complete. 2024-07-17T07:07:52.104 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:07:52.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:07:52.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:07:52.351 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:07:52.351 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:07:52.351 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:07:52.351 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:07:52.389 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:07:52.390 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:07:52.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:07:52.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 -- 172.21.15.44:0/733204339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c84003a80 msgr2=0x7f8c84003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 --2- 172.21.15.44:0/733204339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c84003a80 0x7f8c84003ea0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c900016c0 tx=0x7f8c84005830 comp rx=0 tx=0).stop 2024-07-17T07:07:52.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8c9f400640 1 -- 172.21.15.44:0/733204339 reap_dead start 2024-07-17T07:07:52.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 -- 172.21.15.44:0/733204339 shutdown_connections 2024-07-17T07:07:52.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 -- 172.21.15.44:0/733204339 >> 172.21.15.44:0/733204339 conn(0x7f8ca00b0eb0 msgr2=0x7f8ca00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:52.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 -- 172.21.15.44:0/733204339 shutdown_connections 2024-07-17T07:07:52.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.726+0000 7f8ca5600640 1 -- 172.21.15.44:0/733204339 wait complete. 2024-07-17T07:07:52.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.728+0000 7f8ca5600640 1 Processor -- start 2024-07-17T07:07:52.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.728+0000 7f8ca5600640 1 -- start start 2024-07-17T07:07:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c84003a80 0x7f8ca0147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca0147560 0x7f8ca01f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca015c470 con 0x7f8ca0149ca0 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca015c2f0 con 0x7f8ca0147560 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8ca5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca015c5f0 con 0x7f8c84003a80 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8c9f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8c9f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37034/0 (socket says 172.21.15.44:37034) 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.729+0000 7f8c9f400640 1 -- 172.21.15.44:0/2205175281 learned_addr learned my addr 172.21.15.44:0/2205175281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9ea00640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c84003a80 0x7f8ca0147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c97e00640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca0147560 0x7f8ca01f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c84003a80 msgr2=0x7f8ca0147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c84003a80 0x7f8ca0147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca0147560 msgr2=0x7f8ca01f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca0147560 0x7f8ca01f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 -- 172.21.15.44:0/2205175281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca014ab20 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c9f400640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005fd0 tx=0x7f8c9800fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.730+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800dda0 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.731+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c98011440 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.731+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98011890 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.731+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca014ae00 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.731+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca014c3c0 con 0x7f8ca0149ca0 2024-07-17T07:07:52.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.732+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca00cd2e0 con 0x7f8ca0149ca0 2024-07-17T07:07:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.734+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7f8c98011a30 con 0x7f8ca0149ca0 2024-07-17T07:07:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.735+0000 7f8c96a00640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 0x7f8c7c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.735+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 7068+0+0 (secure 0 0 0) 0x7f8c98098e60 con 0x7f8ca0149ca0 2024-07-17T07:07:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.735+0000 7f8c9ea00640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 msgr2=0x7f8c7c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.735+0000 7f8c9ea00640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 0x7f8c7c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:52.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.739+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8c98061360 con 0x7f8ca0149ca0 2024-07-17T07:07:52.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.937+0000 7f8c9ea00640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 msgr2=0x7f8c7c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:52.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.937+0000 7f8c9ea00640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 0x7f8c7c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:52.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.962+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ca014fa20 con 0x7f8ca0149ca0 2024-07-17T07:07:52.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.963+0000 7f8c96a00640 1 -- 172.21.15.44:0/2205175281 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v53) ==== 80+0+357 (secure 0 0 0) 0x7f8c980658b0 con 0x7f8ca0149ca0 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2225237696 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3025199526 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/557887042 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2038460701 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/557887042 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/733122570 2024-07-18T06:56:17.331799+0000 2024-07-17T07:07:52.963 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T07:07:52.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 msgr2=0x7f8c7c079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:52.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 0x7f8c7c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 msgr2=0x7f8ca014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:52.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005fd0 tx=0x7f8c9800fe00 comp rx=0 tx=0).stop 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 shutdown_connections 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f8c7c076d60 0x7f8c7c079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0149ca0 0x7f8ca014a120 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ca0147560 0x7f8ca01f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 --2- 172.21.15.44:0/2205175281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c84003a80 0x7f8ca0147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 >> 172.21.15.44:0/2205175281 conn(0x7f8ca00b0eb0 msgr2=0x7f8ca00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.966+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 shutdown_connections 2024-07-17T07:07:52.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:52.967+0000 7f8ca5600640 1 -- 172.21.15.44:0/2205175281 wait complete. 2024-07-17T07:07:52.985 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:07:52.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:52.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:07:53.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:07:53.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:07:53.070 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:07:53.111 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:07:53.111 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:07:53.111 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:07:53.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:07:53.154 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:07:53.154 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:07:53.154 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:07:53.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:07:53.204 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:07:53.204 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:07:53.204 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:07:53.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:07:53.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 -- 172.21.15.44:0/1461554544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb230004e60 msgr2=0x7fb23000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 --2- 172.21.15.44:0/1461554544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb230004e60 0x7fb23000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb240009e50 tx=0x7fb230005830 comp rx=0 tx=0).stop 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb24e000640 1 -- 172.21.15.44:0/1461554544 reap_dead start 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 -- 172.21.15.44:0/1461554544 shutdown_connections 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 -- 172.21.15.44:0/1461554544 >> 172.21.15.44:0/1461554544 conn(0x7fb2480b3120 msgr2=0x7fb2480b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:53.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 -- 172.21.15.44:0/1461554544 shutdown_connections 2024-07-17T07:07:53.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.468+0000 7fb250000640 1 -- 172.21.15.44:0/1461554544 wait complete. 2024-07-17T07:07:53.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.470+0000 7fb250000640 1 Processor -- start 2024-07-17T07:07:53.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.470+0000 7fb250000640 1 -- start start 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb230004e60 0x7fb2481520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2481529b0 0x7fb24806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481641c0 con 0x7fb2480af990 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb248164040 con 0x7fb230004e60 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb250000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb248164340 con 0x7fb2481529b0 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb24cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:53.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb24cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37042/0 (socket says 172.21.15.44:37042) 2024-07-17T07:07:53.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.471+0000 7fb24cc00640 1 -- 172.21.15.44:0/1317533246 learned_addr learned my addr 172.21.15.44:0/1317533246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:53.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2481529b0 msgr2=0x7fb24806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2481529b0 0x7fb24806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb230004e60 msgr2=0x7fb2481520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb230004e60 0x7fb2481520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 -- 172.21.15.44:0/1317533246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2480afe90 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb24cc00640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb234004e50 tx=0x7fb234001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234015020 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb234011980 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.472+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234011dd0 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.473+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2480b0170 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.473+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2480ae530 con 0x7fb2480af990 2024-07-17T07:07:53.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.474+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2480cf6a0 con 0x7fb2480af990 2024-07-17T07:07:53.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.475+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7fb234018020 con 0x7fb2480af990 2024-07-17T07:07:53.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.476+0000 7fb245600640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 0x7fb228079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.476+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7292+0+0 (secure 0 0 0) 0x7fb234099840 con 0x7fb2480af990 2024-07-17T07:07:53.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.476+0000 7fb24d600640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 msgr2=0x7fb228079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:53.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.476+0000 7fb24d600640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 0x7fb228079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.480+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb23401f020 con 0x7fb2480af990 2024-07-17T07:07:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.678+0000 7fb24d600640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 msgr2=0x7fb228079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.678+0000 7fb24d600640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 0x7fb228079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:53.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.740+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb248156700 con 0x7fb2480af990 2024-07-17T07:07:53.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.742+0000 7fb245600640 1 -- 172.21.15.44:0/1317533246 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7fb2340622b0 con 0x7fb2480af990 2024-07-17T07:07:53.742 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:53.742 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T07:07:53.742 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.745+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 msgr2=0x7fb228079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.745+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 0x7fb228079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.745+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 msgr2=0x7fb2480aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.745+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb234004e50 tx=0x7fb234001630 comp rx=0 tx=0).stop 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 shutdown_connections 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fb228076d90 0x7fb228079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2481529b0 0x7fb24806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2480af990 0x7fb2480aea50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 --2- 172.21.15.44:0/1317533246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb230004e60 0x7fb2481520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 >> 172.21.15.44:0/1317533246 conn(0x7fb2480b3120 msgr2=0x7fb248070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:53.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 shutdown_connections 2024-07-17T07:07:53.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.746+0000 7fb250000640 1 -- 172.21.15.44:0/1317533246 wait complete. 2024-07-17T07:07:53.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.958+0000 7f2ae2400640 1 -- 172.21.15.44:0/271945010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0b8c80 msgr2=0x7f2adc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.958+0000 7f2ae2400640 1 --2- 172.21.15.44:0/271945010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0b8c80 0x7f2adc0b4ec0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009860 tx=0x7f2ad002a540 comp rx=0 tx=0).stop 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 -- 172.21.15.44:0/271945010 shutdown_connections 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 --2- 172.21.15.44:0/271945010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 0x7f2adc15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 --2- 172.21.15.44:0/271945010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0b8c80 0x7f2adc0b4ec0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 -- 172.21.15.44:0/271945010 >> 172.21.15.44:0/271945010 conn(0x7f2adc0cfec0 msgr2=0x7f2adc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 -- 172.21.15.44:0/271945010 shutdown_connections 2024-07-17T07:07:53.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.959+0000 7f2ae2400640 1 -- 172.21.15.44:0/271945010 wait complete. 2024-07-17T07:07:53.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.961+0000 7f2ae2400640 1 Processor -- start 2024-07-17T07:07:53.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.961+0000 7f2ae2400640 1 -- start start 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 0x7f2adc1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 0x7f2adc1e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc06e0c0 con 0x7f2adc0b8c80 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0b92f0 con 0x7f2adc0ce5c0 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2ae2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc06e240 con 0x7f2adc0b5400 2024-07-17T07:07:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.962+0000 7f2adbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37058/0 (socket says 172.21.15.44:37058) 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 -- 172.21.15.44:0/2362089501 learned_addr learned my addr 172.21.15.44:0/2362089501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2ae1000640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 0x7f2adc1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2ae1a00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 0x7f2adc1e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 msgr2=0x7f2adc1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 0x7f2adc1f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 msgr2=0x7f2adc1e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 0x7f2adc1e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 -- 172.21.15.44:0/2362089501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc15ded0 con 0x7f2adc0b8c80 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2ae1a00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 0x7f2adc1e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.963+0000 7f2adbe00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c400 tx=0x7f2acc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:53.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.964+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.964+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2acc0053c0 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.964+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc003830 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.964+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc1f7d30 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.964+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc1f82f0 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.965+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc06e6f0 con 0x7f2adc0b8c80 2024-07-17T07:07:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.967+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7f2acc01a070 con 0x7f2adc0b8c80 2024-07-17T07:07:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.968+0000 7f2ad8c00640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 0x7f2ac0079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.968+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7292+0+0 (secure 0 0 0) 0x7f2acc098100 con 0x7f2adc0b8c80 2024-07-17T07:07:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.968+0000 7f2ae1000640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 msgr2=0x7f2ac0079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.968+0000 7f2ae1000640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 0x7f2ac0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:53.972+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2acc060c90 con 0x7f2adc0b8c80 2024-07-17T07:07:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.170+0000 7f2ae1000640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 msgr2=0x7f2ac0079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:54.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.170+0000 7f2ae1000640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 0x7f2ac0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:54.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.218+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2adc0b5d70 con 0x7f2adc0b8c80 2024-07-17T07:07:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.219+0000 7f2ad8c00640 1 -- 172.21.15.44:0/2362089501 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f2acc060590 con 0x7f2adc0b8c80 2024-07-17T07:07:54.219 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:54.219 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T07:07:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 msgr2=0x7f2ac0079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 0x7f2ac0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 msgr2=0x7f2adc15cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c400 tx=0x7f2acc00c8c0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 shutdown_connections 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f2ac0076d60 0x7f2ac0079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2adc0ce5c0 0x7f2adc1e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2adc0b8c80 0x7f2adc15cec0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 --2- 172.21.15.44:0/2362089501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2adc0b5400 0x7f2adc1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 >> 172.21.15.44:0/2362089501 conn(0x7f2adc0cfec0 msgr2=0x7f2adc0d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 shutdown_connections 2024-07-17T07:07:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.222+0000 7f2ae2400640 1 -- 172.21.15.44:0/2362089501 wait complete. 2024-07-17T07:07:54.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:07:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 -- 172.21.15.44:0/4141087835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 msgr2=0x7f7e1c15e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 --2- 172.21.15.44:0/4141087835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c15e2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009ca0 tx=0x7f7e1402aac0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 -- 172.21.15.44:0/4141087835 shutdown_connections 2024-07-17T07:07:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 --2- 172.21.15.44:0/4141087835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c15e2d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 --2- 172.21.15.44:0/4141087835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 0x7f7e1c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 -- 172.21.15.44:0/4141087835 >> 172.21.15.44:0/4141087835 conn(0x7f7e1c0cfec0 msgr2=0x7f7e1c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 -- 172.21.15.44:0/4141087835 shutdown_connections 2024-07-17T07:07:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.434+0000 7f7e21400640 1 -- 172.21.15.44:0/4141087835 wait complete. 2024-07-17T07:07:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.437+0000 7f7e21400640 1 Processor -- start 2024-07-17T07:07:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.437+0000 7f7e21400640 1 -- start start 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 0x7f7e1c14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c06e0c0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0b9380 con 0x7f7e1c0b5400 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e21400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1c06e240 con 0x7f7e1c0b8c80 2024-07-17T07:07:54.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e20a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e20a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37062/0 (socket says 172.21.15.44:37062) 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e20a00640 1 -- 172.21.15.44:0/1013511225 learned_addr learned my addr 172.21.15.44:0/1013511225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e1be00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.439+0000 7f7e13e00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 0x7f7e1c14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 msgr2=0x7f7e1c14f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 0x7f7e1c14f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 msgr2=0x7f7e1c14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 -- 172.21.15.44:0/1013511225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c14fdf0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e20a00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009ca0 tx=0x7f7e14003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14046070 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e14003e50 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140044b0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c1500d0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.440+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c150690 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.441+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c06e6f0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.443+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7f7e14049070 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.445+0000 7f7e18c00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 0x7f7df8079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.445+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 7292+0+0 (secure 0 0 0) 0x7f7e140b9980 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.445+0000 7f7e1be00640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 msgr2=0x7f7df8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.445+0000 7f7e1be00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 0x7f7df8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.448+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7e14032030 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.646+0000 7f7e1be00640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 msgr2=0x7f7df8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:54.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.646+0000 7f7e1be00640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 0x7f7df8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:54.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.674+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e1c15e5f0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.676+0000 7f7e18c00640 1 -- 172.21.15.44:0/1013511225 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+18138 (secure 0 0 0) 0x7f7e14081ea0 con 0x7f7e1c0ce5c0 2024-07-17T07:07:54.676 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:54.676 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:07:54.460656+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:55:55.596688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-07-17T06:55:56.460663+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":55,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.7","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2225237696":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/3025199526":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6801/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/2038460701":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6800/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/733122570":"2024-07-18T06:56:17.331799+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":110,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 msgr2=0x7f7df8079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 0x7f7df8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 msgr2=0x7f7e1c0aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009ca0 tx=0x7f7e14003ce0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 shutdown_connections 2024-07-17T07:07:54.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f7df8076d90 0x7f7df8079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1c0ce5c0 0x7f7e1c0aa8f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e1c0b8c80 0x7f7e1c14f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 --2- 172.21.15.44:0/1013511225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e1c0b5400 0x7f7e1c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 >> 172.21.15.44:0/1013511225 conn(0x7f7e1c0cfec0 msgr2=0x7f7e1c0b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 shutdown_connections 2024-07-17T07:07:54.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.679+0000 7f7e21400640 1 -- 172.21.15.44:0/1013511225 wait complete. 2024-07-17T07:07:54.698 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:07:54.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:07:54.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 -- 172.21.15.44:0/2665492375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 msgr2=0x7f9de4168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 --2- 172.21.15.44:0/2665492375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4168160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005fd0 tx=0x7f9ddc02a940 comp rx=0 tx=0).stop 2024-07-17T07:07:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 -- 172.21.15.44:0/2665492375 shutdown_connections 2024-07-17T07:07:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 --2- 172.21.15.44:0/2665492375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4168160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 -- 172.21.15.44:0/2665492375 >> 172.21.15.44:0/2665492375 conn(0x7f9de40d00a0 msgr2=0x7f9de40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:54.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 -- 172.21.15.44:0/2665492375 shutdown_connections 2024-07-17T07:07:54.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.896+0000 7f9dec200640 1 -- 172.21.15.44:0/2665492375 wait complete. 2024-07-17T07:07:54.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.898+0000 7f9dec200640 1 Processor -- start 2024-07-17T07:07:54.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.898+0000 7f9dec200640 1 -- start start 2024-07-17T07:07:54.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de40b2ad0 0x7f9de406c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 0x7f9de4151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9de4168820 con 0x7f9de40b5910 2024-07-17T07:07:54.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41686a0 con 0x7f9de415d600 2024-07-17T07:07:54.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.899+0000 7f9dec200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9de41689a0 con 0x7f9de40b2ad0 2024-07-17T07:07:54.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9dea200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 0x7f9de4151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37080/0 (socket says 172.21.15.44:37080) 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 -- 172.21.15.44:0/2668422663 learned_addr learned my addr 172.21.15.44:0/2668422663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9dea200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 0x7f9de4151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:36122/0 (socket says 172.21.15.44:36122) 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de9800640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de40b2ad0 0x7f9de406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de40b2ad0 msgr2=0x7f9de406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de40b2ad0 0x7f9de406c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 msgr2=0x7f9de4151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:54.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 0x7f9de4151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:54.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 -- 172.21.15.44:0/2668422663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de406d030 con 0x7f9de40b5910 2024-07-17T07:07:54.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.900+0000 7f9de8e00640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c450 tx=0x7f9dd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:54.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.901+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0011790 con 0x7f9de40b5910 2024-07-17T07:07:54.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.901+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd0017070 con 0x7f9de40b5910 2024-07-17T07:07:54.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.901+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0011cc0 con 0x7f9de40b5910 2024-07-17T07:07:54.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.901+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de41f4410 con 0x7f9de40b5910 2024-07-17T07:07:54.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.901+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de41f4990 con 0x7f9de40b5910 2024-07-17T07:07:54.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.902+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de40b5200 con 0x7f9de40b5910 2024-07-17T07:07:54.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.903+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7f9dd001c460 con 0x7f9de40b5910 2024-07-17T07:07:54.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.904+0000 7f9de1600640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 0x7f9dc4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:54.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.904+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6924+0+0 (secure 0 0 0) 0x7f9dd00988f0 con 0x7f9de40b5910 2024-07-17T07:07:54.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.904+0000 7f9de9800640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 msgr2=0x7f9dc4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:54.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.904+0000 7f9de9800640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 0x7f9dc4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:54.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:54.912+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9dd0060f90 con 0x7f9de40b5910 2024-07-17T07:07:55.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.105+0000 7f9de9800640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 msgr2=0x7f9dc4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:55.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.105+0000 7f9de9800640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 0x7f9dc4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:55.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.164+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9de4150fa0 con 0x7f9de40b5910 2024-07-17T07:07:55.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.166+0000 7f9de1600640 1 -- 172.21.15.44:0/2668422663 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f9dd00654e0 con 0x7f9de40b5910 2024-07-17T07:07:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:55.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T07:07:55.167 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T07:07:55.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.168+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 msgr2=0x7f9dc4079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.168+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 0x7f9dc4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.168+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 msgr2=0x7f9de4151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.168+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c450 tx=0x7f9dd000c910 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 shutdown_connections 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7f9dc4076d60 0x7f9dc4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de415d600 0x7f9de4151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9de40b5910 0x7f9de4151410 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 --2- 172.21.15.44:0/2668422663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de40b2ad0 0x7f9de406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 >> 172.21.15.44:0/2668422663 conn(0x7f9de40d00a0 msgr2=0x7f9de40b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:55.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 shutdown_connections 2024-07-17T07:07:55.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.169+0000 7f9dec200640 1 -- 172.21.15.44:0/2668422663 wait complete. 2024-07-17T07:07:55.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:07:55.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 -- 172.21.15.44:0/3034207059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc09ca20 msgr2=0x7fe1fc0cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 --2- 172.21.15.44:0/3034207059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc09ca20 0x7fe1fc0cf2d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009830 tx=0x7fe1e802a480 comp rx=0 tx=0).stop 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 -- 172.21.15.44:0/3034207059 shutdown_connections 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 --2- 172.21.15.44:0/3034207059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc09ca20 0x7fe1fc0cf2d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 -- 172.21.15.44:0/3034207059 >> 172.21.15.44:0/3034207059 conn(0x7fe1fc0b0e80 msgr2=0x7fe1fc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 -- 172.21.15.44:0/3034207059 shutdown_connections 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.379+0000 7fe203000640 1 -- 172.21.15.44:0/3034207059 wait complete. 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.380+0000 7fe203000640 1 Processor -- start 2024-07-17T07:07:55.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.381+0000 7fe203000640 1 -- start start 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.381+0000 7fe203000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1fc09ca20 0x7fe1fc14fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.381+0000 7fe203000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1fc0cf810 0x7fe1fc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe203000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe203000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc0ad0e0 con 0x7fe1fc15d240 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe203000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc0acf60 con 0x7fe1fc0cf810 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe203000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc0ad260 con 0x7fe1fc09ca20 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37084/0 (socket says 172.21.15.44:37084) 2024-07-17T07:07:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 -- 172.21.15.44:0/333554089 learned_addr learned my addr 172.21.15.44:0/333554089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe201c00640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1fc09ca20 0x7fe1fc14fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe201200640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1fc0cf810 0x7fe1fc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1fc09ca20 msgr2=0x7fe1fc14fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1fc09ca20 0x7fe1fc14fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1fc0cf810 msgr2=0x7fe1fc15ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1fc0cf810 0x7fe1fc15ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.382+0000 7fe202600640 1 -- 172.21.15.44:0/333554089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc1f7de0 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe202600640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c7b0 tx=0x7fe1f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4004b60 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1f4004df0 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400aa90 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc1f3cf0 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.383+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc1f47d0 con 0x7fe1fc15d240 2024-07-17T07:07:55.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.384+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1fc09c320 con 0x7fe1fc15d240 2024-07-17T07:07:55.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.386+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7fe1f400ac30 con 0x7fe1fc15d240 2024-07-17T07:07:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.387+0000 7fe1f9600640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 0x7fe1dc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.387+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6924+0+0 (secure 0 0 0) 0x7fe1f4098d10 con 0x7fe1fc15d240 2024-07-17T07:07:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.387+0000 7fe201c00640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 msgr2=0x7fe1dc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:55.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.387+0000 7fe201c00640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 0x7fe1dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:55.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.392+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe1fc09c320 con 0x7fe1fc15d240 2024-07-17T07:07:55.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.588+0000 7fe201c00640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 msgr2=0x7fe1dc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:55.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.588+0000 7fe201c00640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 0x7fe1dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:55.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.646+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1fc0ad770 con 0x7fe1fc15d240 2024-07-17T07:07:55.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.647+0000 7fe1f9600640 1 -- 172.21.15.44:0/333554089 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7fe1f4020070 con 0x7fe1fc15d240 2024-07-17T07:07:55.648 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:55.648 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"btime":"2024-07-17T06:56:01:057591+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T06:55:57.477076+0000","modified":"2024-07-17T06:56:00.564713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4738":{"gid":4738,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1597326512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1597326512},{"type":"v1","addr":"172.21.15.44:6835","nonce":1597326512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4738,"qdb_cluster":[4738]},"id":2}]} 2024-07-17T07:07:55.648 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-07-17T07:07:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 msgr2=0x7fe1dc079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 0x7fe1dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 msgr2=0x7fe1fc1604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c7b0 tx=0x7fe1f400cc70 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 shutdown_connections 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fe1dc076d60 0x7fe1dc079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1fc15d240 0x7fe1fc1604f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1fc0cf810 0x7fe1fc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 --2- 172.21.15.44:0/333554089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1fc09ca20 0x7fe1fc14fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.650+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 >> 172.21.15.44:0/333554089 conn(0x7fe1fc0b0e80 msgr2=0x7fe1fc159700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.651+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 shutdown_connections 2024-07-17T07:07:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.651+0000 7fe203000640 1 -- 172.21.15.44:0/333554089 wait complete. 2024-07-17T07:07:55.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:07:55.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.851+0000 7fc762c00640 1 -- 172.21.15.44:0/2281224978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc740004800 msgr2=0x7fc740004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.851+0000 7fc762c00640 1 --2- 172.21.15.44:0/2281224978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc740004800 0x7fc740004c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc74c005fd0 tx=0x7fc740005830 comp rx=0 tx=0).stop 2024-07-17T07:07:55.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.852+0000 7fc762c00640 1 -- 172.21.15.44:0/2281224978 shutdown_connections 2024-07-17T07:07:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.852+0000 7fc762c00640 1 --2- 172.21.15.44:0/2281224978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc740004800 0x7fc740004c80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.852+0000 7fc762c00640 1 -- 172.21.15.44:0/2281224978 >> 172.21.15.44:0/2281224978 conn(0x7fc75c0b0fb0 msgr2=0x7fc75c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.852+0000 7fc762c00640 1 -- 172.21.15.44:0/2281224978 shutdown_connections 2024-07-17T07:07:55.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.852+0000 7fc762c00640 1 -- 172.21.15.44:0/2281224978 wait complete. 2024-07-17T07:07:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.853+0000 7fc762c00640 1 Processor -- start 2024-07-17T07:07:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.854+0000 7fc762c00640 1 -- start start 2024-07-17T07:07:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.854+0000 7fc762c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc740004800 0x7fc75c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc762c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75c143590 0x7fc75c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc762c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc762c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c06f480 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc762c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c15b8d0 con 0x7fc740004800 2024-07-17T07:07:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc762c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c06f600 con 0x7fc75c143590 2024-07-17T07:07:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc760c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc760c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37102/0 (socket says 172.21.15.44:37102) 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc760c00640 1 -- 172.21.15.44:0/406709879 learned_addr learned my addr 172.21.15.44:0/406709879 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc75b400640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75c143590 0x7fc75c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc75be00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc740004800 0x7fc75c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.855+0000 7fc760c00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75c143590 msgr2=0x7fc75c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc760c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75c143590 0x7fc75c0cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc760c00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc740004800 msgr2=0x7fc75c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc760c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc740004800 0x7fc75c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc760c00640 1 -- 172.21.15.44:0/406709879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c157120 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc760c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d410 tx=0x7fc754006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754005320 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7540035f0 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754003a40 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1f7cf0 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.856+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1f82b0 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.857+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c0ac730 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.860+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7fc754003be0 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.861+0000 7fc753e00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 0x7fc738079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.861+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6924+0+0 (secure 0 0 0) 0x7fc754097f70 con 0x7fc75c0ce2c0 2024-07-17T07:07:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.861+0000 7fc75be00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 msgr2=0x7fc738079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:55.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.861+0000 7fc75be00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 0x7fc738079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:55.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:55.866+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc754060750 con 0x7fc75c0ce2c0 2024-07-17T07:07:56.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.062+0000 7fc75be00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 msgr2=0x7fc738079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:56.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.062+0000 7fc75be00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 0x7fc738079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:07:56.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.076+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc75c06dbf0 con 0x7fc75c0ce2c0 2024-07-17T07:07:56.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.079+0000 7fc753e00640 1 -- 172.21.15.44:0/406709879 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+18138 (secure 0 0 0) 0x7fc754064ca0 con 0x7fc75c0ce2c0 2024-07-17T07:07:56.079 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:07:56.079 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":55,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:07:54.460656+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-07-17T06:55:55.596688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-07-17T06:55:56.460663+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":55,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.7","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2225237696":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/3025199526":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6801/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/2038460701":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6800/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/733122570":"2024-07-18T06:56:17.331799+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":110,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 msgr2=0x7fc738079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 0x7fc738079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 msgr2=0x7fc75c1e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d410 tx=0x7fc754006f90 comp rx=0 tx=0).stop 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 shutdown_connections 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7fc738076d60 0x7fc738079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75c0ce2c0 0x7fc75c1e6e40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75c143590 0x7fc75c0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 --2- 172.21.15.44:0/406709879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc740004800 0x7fc75c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 >> 172.21.15.44:0/406709879 conn(0x7fc75c0b0fb0 msgr2=0x7fc75c0afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:56.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 shutdown_connections 2024-07-17T07:07:56.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.082+0000 7fc762c00640 1 -- 172.21.15.44:0/406709879 wait complete. 2024-07-17T07:07:56.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:07:56.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.297+0000 7faf9f000640 1 -- 172.21.15.44:0/3043166626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf80005310 msgr2=0x7faf8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.297+0000 7faf9f000640 1 --2- 172.21.15.44:0/3043166626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf80005310 0x7faf8000b9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faf90009ed0 tx=0x7faf80005830 comp rx=0 tx=0).stop 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.297+0000 7faf9f000640 1 -- 172.21.15.44:0/3043166626 shutdown_connections 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.297+0000 7faf9f000640 1 --2- 172.21.15.44:0/3043166626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf80005310 0x7faf8000b9c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.297+0000 7faf9f000640 1 -- 172.21.15.44:0/3043166626 >> 172.21.15.44:0/3043166626 conn(0x7faf980cfec0 msgr2=0x7faf980d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.298+0000 7faf9f000640 1 -- 172.21.15.44:0/3043166626 shutdown_connections 2024-07-17T07:07:56.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.298+0000 7faf9f000640 1 -- 172.21.15.44:0/3043166626 wait complete. 2024-07-17T07:07:56.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.299+0000 7faf9f000640 1 Processor -- start 2024-07-17T07:07:56.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.300+0000 7faf9f000640 1 -- start start 2024-07-17T07:07:56.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 0x7faf98144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf981538d0 0x7faf98153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf980d1220 con 0x7faf9815fb60 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf980b4ec0 con 0x7faf981538d0 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf981435d0 con 0x7faf80005310 2024-07-17T07:07:56.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37110/0 (socket says 172.21.15.44:37110) 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9d200640 1 -- 172.21.15.44:0/3002762500 learned_addr learned my addr 172.21.15.44:0/3002762500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9dc00640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 0x7faf98144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.301+0000 7faf9e600640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf981538d0 0x7faf98153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 -- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 msgr2=0x7faf98144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 0x7faf98144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 -- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf981538d0 msgr2=0x7faf98153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf981538d0 0x7faf98153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 -- 172.21.15.44:0/3002762500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf98145790 con 0x7faf9815fb60 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9dc00640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 0x7faf98144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9d200640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf8400c730 tx=0x7faf8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:56.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84017070 con 0x7faf9815fb60 2024-07-17T07:07:56.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf84005360 con 0x7faf9815fb60 2024-07-17T07:07:56.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf98145a70 con 0x7faf9815fb60 2024-07-17T07:07:56.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400a8a0 con 0x7faf9815fb60 2024-07-17T07:07:56.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.302+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf98146030 con 0x7faf9815fb60 2024-07-17T07:07:56.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.304+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf98160520 con 0x7faf9815fb60 2024-07-17T07:07:56.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.305+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 14) ==== 99000+0+0 (secure 0 0 0) 0x7faf8401a070 con 0x7faf9815fb60 2024-07-17T07:07:56.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.306+0000 7faf95600640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 0x7faf78079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.306+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6924+0+0 (secure 0 0 0) 0x7faf84098280 con 0x7faf9815fb60 2024-07-17T07:07:56.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.306+0000 7faf9dc00640 1 -- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 msgr2=0x7faf78079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2568152110 2024-07-17T07:07:56.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.306+0000 7faf9dc00640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 0x7faf78079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:07:56.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.314+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faf84060f70 con 0x7faf9815fb60 2024-07-17T07:07:56.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.435+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 15) ==== 98540+0+0 (secure 0 0 0) 0x7faf8405cb00 con 0x7faf9815fb60 2024-07-17T07:07:56.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.435+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 msgr2=0x7faf78079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:07:56.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.435+0000 7faf95600640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 0x7faf78079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.511+0000 7fcf9e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.511+0000 7fcf9e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.511+0000 7fcf9e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.511+0000 7fcf9e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.511+0000 7fcf9e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.512+0000 7fcf9aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.512+0000 7fcf9aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.512+0000 7fcf9aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.512 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.512+0000 7fcf9aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.513 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:07:56.512+0000 7fcf9aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:07:56.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.576+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7faf980b5390 con 0x7faf9815fb60 2024-07-17T07:07:56.577 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:07:56.577+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:07:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.581+0000 7faf95600640 1 -- 172.21.15.44:0/3002762500 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7faf8401b3b0 con 0x7faf9815fb60 2024-07-17T07:07:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:07:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 msgr2=0x7faf9815ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faf8400c730 tx=0x7faf8400cbf0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 shutdown_connections 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:6800/2568152110,v1:172.21.15.175:6801/2568152110] conn(0x7faf78076d60 0x7faf78079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf981538d0 0x7faf98153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf9815fb60 0x7faf9815ffe0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 --2- 172.21.15.44:0/3002762500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf80005310 0x7faf98144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.584+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 >> 172.21.15.44:0/3002762500 conn(0x7faf980cfec0 msgr2=0x7faf980b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.585+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 shutdown_connections 2024-07-17T07:07:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.585+0000 7faf9f000640 1 -- 172.21.15.44:0/3002762500 wait complete. 2024-07-17T07:07:56.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:07:56.697 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:07:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 -- 172.21.15.44:0/3476411117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73cc003a50 msgr2=0x7f73cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 --2- 172.21.15.44:0/3476411117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73cc003a50 0x7f73cc003e70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f73e40d1160 tx=0x7f73cc005830 comp rx=0 tx=0).stop 2024-07-17T07:07:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73e9c00640 1 -- 172.21.15.44:0/3476411117 reap_dead start 2024-07-17T07:07:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 -- 172.21.15.44:0/3476411117 shutdown_connections 2024-07-17T07:07:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 -- 172.21.15.44:0/3476411117 >> 172.21.15.44:0/3476411117 conn(0x7f73e40d00f0 msgr2=0x7f73e40d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 -- 172.21.15.44:0/3476411117 shutdown_connections 2024-07-17T07:07:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.787+0000 7f73ebc00640 1 -- 172.21.15.44:0/3476411117 wait complete. 2024-07-17T07:07:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.789+0000 7f73ebc00640 1 Processor -- start 2024-07-17T07:07:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.789+0000 7f73ebc00640 1 -- start start 2024-07-17T07:07:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73cc003a50 0x7f73e40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73e40b49c0 0x7f73e40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73e40ca370 con 0x7f73e41492a0 2024-07-17T07:07:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73e40d34d0 con 0x7f73cc003a50 2024-07-17T07:07:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73ebc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73e40ca4f0 con 0x7f73e40b49c0 2024-07-17T07:07:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.790+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37166/0 (socket says 172.21.15.44:37166) 2024-07-17T07:07:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 -- 172.21.15.44:0/1785679490 learned_addr learned my addr 172.21.15.44:0/1785679490 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e9c00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73e40b49c0 0x7f73e40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e9200640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73cc003a50 0x7f73e40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 -- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73e40b49c0 msgr2=0x7f73e40b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73e40b49c0 0x7f73e40b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 -- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73cc003a50 msgr2=0x7f73e40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73cc003a50 0x7f73e40ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 -- 172.21.15.44:0/1785679490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e41f4280 con 0x7f73e41492a0 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e3e00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e990 tx=0x7f73d000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.791+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000cbe0 con 0x7f73e41492a0 2024-07-17T07:07:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.792+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73d0005020 con 0x7f73e41492a0 2024-07-17T07:07:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.792+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0005430 con 0x7f73e41492a0 2024-07-17T07:07:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.792+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e414bea0 con 0x7f73e41492a0 2024-07-17T07:07:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.792+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e40cb700 con 0x7f73e41492a0 2024-07-17T07:07:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.793+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e40b74b0 con 0x7f73e41492a0 2024-07-17T07:07:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.794+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 98540+0+0 (secure 0 0 0) 0x7f73d00047a0 con 0x7f73e41492a0 2024-07-17T07:07:56.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.796+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7353+0+0 (secure 0 0 0) 0x7f73d0098bf0 con 0x7f73e41492a0 2024-07-17T07:07:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:56.800+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f73d0016030 con 0x7f73e41492a0 2024-07-17T07:07:57.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.059+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f73e415f410 con 0x7f73e41492a0 2024-07-17T07:07:57.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.448+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 16) ==== 98667+0+0 (secure 0 0 0) 0x7f73d005d020 con 0x7f73e41492a0 2024-07-17T07:07:57.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.449+0000 7f73e0c00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 0x7f73c407f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:57.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.450+0000 7f73e9200640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 0x7f73c407f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:57.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.453+0000 7f73e9200640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 0x7f73c407f410 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f73d8033000 tx=0x7f73d8001950 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:57.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.585+0000 7f73e0c00640 1 -- 172.21.15.44:0/1785679490 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f73d0061db0 con 0x7f73e41492a0 2024-07-17T07:07:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 msgr2=0x7f73c407f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 0x7f73c407f410 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f73d8033000 tx=0x7f73d8001950 comp rx=0 tx=0).stop 2024-07-17T07:07:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 msgr2=0x7f73e40acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e990 tx=0x7f73d000ee50 comp rx=0 tx=0).stop 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 shutdown_connections 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f73c407d020 0x7f73c407f410 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73e40b49c0 0x7f73e40b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73e41492a0 0x7f73e40acdd0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 --2- 172.21.15.44:0/1785679490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73cc003a50 0x7f73e40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 >> 172.21.15.44:0/1785679490 conn(0x7f73e40d00f0 msgr2=0x7f73e415f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 shutdown_connections 2024-07-17T07:07:57.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.589+0000 7f73ebc00640 1 -- 172.21.15.44:0/1785679490 wait complete. 2024-07-17T07:07:57.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:07:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 -- 172.21.15.44:0/4031780577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de415b060 msgr2=0x7f2de406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 --2- 172.21.15.44:0/4031780577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de415b060 0x7f2de406d620 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005fd0 tx=0x7f2dd802a5a0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 -- 172.21.15.44:0/4031780577 shutdown_connections 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 --2- 172.21.15.44:0/4031780577 >> v2:172.21.15.175:3300/0 conn(0x7f2de40cb880 0x7f2de406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 --2- 172.21.15.44:0/4031780577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de415b060 0x7f2de406d620 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.825+0000 7f2deba00640 1 -- 172.21.15.44:0/4031780577 >> 172.21.15.44:0/4031780577 conn(0x7f2de40b0eb0 msgr2=0x7f2de40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.826+0000 7f2deba00640 1 -- 172.21.15.44:0/4031780577 shutdown_connections 2024-07-17T07:07:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.826+0000 7f2deba00640 1 -- 172.21.15.44:0/4031780577 wait complete. 2024-07-17T07:07:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.827+0000 7f2deba00640 1 Processor -- start 2024-07-17T07:07:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.828+0000 7f2deba00640 1 -- start start 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2de415b060 0x7f2de40ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 0x7f2de40b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2de415b5c0 con 0x7f2de40cb880 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2de415b440 con 0x7f2de415c220 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2deba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2de415f4a0 con 0x7f2de415b060 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37182/0 (socket says 172.21.15.44:37182) 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 -- 172.21.15.44:0/1367831140 learned_addr learned my addr 172.21.15.44:0/1367831140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de3e00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2de415b060 0x7f2de40ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9a00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 0x7f2de40b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 -- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2de415b060 msgr2=0x7f2de40ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2de415b060 0x7f2de40ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 -- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 msgr2=0x7f2de40b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 0x7f2de40b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9000640 1 -- 172.21.15.44:0/1367831140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40adc10 con 0x7f2de40cb880 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.829+0000 7f2de9a00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 0x7f2de40b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2de9000640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009850 tx=0x7f2dd80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd803f070 con 0x7f2de40cb880 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd80047a0 con 0x7f2de40cb880 2024-07-17T07:07:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de40adef0 con 0x7f2de40cb880 2024-07-17T07:07:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8035030 con 0x7f2de40cb880 2024-07-17T07:07:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.830+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de4158e50 con 0x7f2de40cb880 2024-07-17T07:07:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.831+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de40af8d0 con 0x7f2de40cb880 2024-07-17T07:07:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.832+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 98667+0+0 (secure 0 0 0) 0x7f2dd8042070 con 0x7f2de40cb880 2024-07-17T07:07:57.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.833+0000 7f2de0c00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 0x7f2dc4078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:57.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.834+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 7353+0+0 (secure 0 0 0) 0x7f2dd80b93f0 con 0x7f2de40cb880 2024-07-17T07:07:57.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.834+0000 7f2de3e00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 0x7f2dc4078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:57.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.835+0000 7f2de3e00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 0x7f2dc4078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd0005850 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:57.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:57.837+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2de40af8d0 con 0x7f2de40cb880 2024-07-17T07:07:58.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.052+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2de415f9c0 con 0x7f2de40cb880 2024-07-17T07:07:58.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.589+0000 7f2de0c00640 1 -- 172.21.15.44:0/1367831140 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f2dd80824c0 con 0x7f2de40cb880 2024-07-17T07:07:58.590 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.594+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 msgr2=0x7f2dc4078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.594+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 0x7f2dc4078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0005fd0 tx=0x7f2dd0005850 comp rx=0 tx=0).stop 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.594+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 msgr2=0x7f2de414f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.594+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009850 tx=0x7f2dd80038a0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 shutdown_connections 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2dc4076a00 0x7f2dc4078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2de415c220 0x7f2de40b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2de415b060 0x7f2de40ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 --2- 172.21.15.44:0/1367831140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2de40cb880 0x7f2de414f640 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.595+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 >> 172.21.15.44:0/1367831140 conn(0x7f2de40b0eb0 msgr2=0x7f2de40ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:58.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.596+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 shutdown_connections 2024-07-17T07:07:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.597+0000 7f2deba00640 1 -- 172.21.15.44:0/1367831140 wait complete. 2024-07-17T07:07:58.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.827+0000 7fc017e00640 1 -- 172.21.15.44:0/555801207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc018155860 msgr2=0x7fc0180ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.827+0000 7fc017e00640 1 --2- 172.21.15.44:0/555801207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc018155860 0x7fc0180ad250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc0180d0050 tx=0x7fc01002aa10 comp rx=0 tx=0).stop 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 -- 172.21.15.44:0/555801207 shutdown_connections 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 --2- 172.21.15.44:0/555801207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0180ae2c0 0x7fc0180abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 --2- 172.21.15.44:0/555801207 >> v2:172.21.15.175:3300/0 conn(0x7fc0180ad790 0x7fc0180add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 --2- 172.21.15.44:0/555801207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc018155860 0x7fc0180ad250 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 -- 172.21.15.44:0/555801207 >> 172.21.15.44:0/555801207 conn(0x7fc0180cf050 msgr2=0x7fc0180cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 -- 172.21.15.44:0/555801207 shutdown_connections 2024-07-17T07:07:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.828+0000 7fc017e00640 1 -- 172.21.15.44:0/555801207 wait complete. 2024-07-17T07:07:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.830+0000 7fc017e00640 1 Processor -- start 2024-07-17T07:07:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.830+0000 7fc017e00640 1 -- start start 2024-07-17T07:07:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0180ad790 0x7fc018151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc018155860 0x7fc0181f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0180ac660 con 0x7fc0180ae2c0 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0180ac4e0 con 0x7fc0180ad790 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.831+0000 7fc017e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc0180ac7e0 con 0x7fc018155860 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56942/0 (socket says 172.21.15.44:56942) 2024-07-17T07:07:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 -- 172.21.15.44:0/721593249 learned_addr learned my addr 172.21.15.44:0/721593249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc017400640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc018155860 0x7fc0181f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016a00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0180ad790 0x7fc018151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 -- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc018155860 msgr2=0x7fc0181f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc018155860 0x7fc0181f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 -- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0180ad790 msgr2=0x7fc018151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0180ad790 0x7fc018151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.832+0000 7fc016000640 1 -- 172.21.15.44:0/721593249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc018152a20 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc016000640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b4a0 tx=0x7fc00000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000014070 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc000010700 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000010b50 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc018152d00 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.833+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0181532c0 con 0x7fc0180ae2c0 2024-07-17T07:07:58.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.834+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0180acd20 con 0x7fc0180ae2c0 2024-07-17T07:07:58.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.835+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 98875+0+0 (secure 0 0 0) 0x7fc000017070 con 0x7fc0180ae2c0 2024-07-17T07:07:58.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.837+0000 7fc00f400640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 0x7fbff4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:58.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.837+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6915+0+0 (secure 0 0 0) 0x7fc000098350 con 0x7fc0180ae2c0 2024-07-17T07:07:58.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.837+0000 7fc016a00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 0x7fbff4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:58.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.839+0000 7fc016a00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 0x7fbff4079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc010035000 tx=0x7fc010004ab0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:58.841+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc000061300 con 0x7fc0180ae2c0 2024-07-17T07:07:59.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.053+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0180ac1b0 con 0x7fc0180ae2c0 2024-07-17T07:07:59.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.612+0000 7fc00f400640 1 -- 172.21.15.44:0/721593249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7fc000060c00 con 0x7fc0180ae2c0 2024-07-17T07:07:59.612 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.616+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 msgr2=0x7fbff4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 0x7fbff4079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc010035000 tx=0x7fc010004ab0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 msgr2=0x7fc0181f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b4a0 tx=0x7fc00000b960 comp rx=0 tx=0).stop 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 shutdown_connections 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fbff4076c40 0x7fbff4079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc018155860 0x7fc0181f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0180ae2c0 0x7fc0181f4620 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 --2- 172.21.15.44:0/721593249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0180ad790 0x7fc018151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 >> 172.21.15.44:0/721593249 conn(0x7fc0180cf050 msgr2=0x7fc0180afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:59.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 shutdown_connections 2024-07-17T07:07:59.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.617+0000 7fc017e00640 1 -- 172.21.15.44:0/721593249 wait complete. 2024-07-17T07:07:59.640 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:07:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.843+0000 7f1f1f200640 1 -- 172.21.15.44:0/481709103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 msgr2=0x7f1f1815b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.843+0000 7f1f1f200640 1 --2- 172.21.15.44:0/481709103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f1815b180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009830 tx=0x7f1f1002a510 comp rx=0 tx=0).stop 2024-07-17T07:07:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.843+0000 7f1f1f200640 1 -- 172.21.15.44:0/481709103 shutdown_connections 2024-07-17T07:07:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.843+0000 7f1f1f200640 1 --2- 172.21.15.44:0/481709103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f1815b180 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.843+0000 7f1f1f200640 1 -- 172.21.15.44:0/481709103 >> 172.21.15.44:0/481709103 conn(0x7f1f180b2240 msgr2=0x7f1f180b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:07:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.844+0000 7f1f1f200640 1 -- 172.21.15.44:0/481709103 shutdown_connections 2024-07-17T07:07:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.844+0000 7f1f1f200640 1 -- 172.21.15.44:0/481709103 wait complete. 2024-07-17T07:07:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.845+0000 7f1f1f200640 1 Processor -- start 2024-07-17T07:07:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.845+0000 7f1f1f200640 1 -- start start 2024-07-17T07:07:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.846+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.846+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 0x7f1f18143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f18157d10 0x7f1f18143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f1f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180ce590 con 0x7f1f180709e0 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f1f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806fdc0 con 0x7f1f18157d10 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f1f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f180ce710 con 0x7f1f180afb30 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56960/0 (socket says 172.21.15.44:56960) 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 -- 172.21.15.44:0/854958214 learned_addr learned my addr 172.21.15.44:0/854958214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f1d200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f18157d10 0x7f1f18143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17400640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 0x7f1f18143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 -- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 msgr2=0x7f1f18143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 0x7f1f18143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 -- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f18157d10 msgr2=0x7f1f18143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f18157d10 0x7f1f18143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17e00640 1 -- 172.21.15.44:0/854958214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181f8730 con 0x7f1f180709e0 2024-07-17T07:07:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.847+0000 7f1f17400640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 0x7f1f18143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:07:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f17e00640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800cb30 tx=0x7f1f0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08016020 con 0x7f1f180709e0 2024-07-17T07:07:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f0800ad90 con 0x7f1f180709e0 2024-07-17T07:07:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08012680 con 0x7f1f180709e0 2024-07-17T07:07:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f181f4640 con 0x7f1f180709e0 2024-07-17T07:07:59.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.848+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181f4c00 con 0x7f1f180709e0 2024-07-17T07:07:59.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.850+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1806cc40 con 0x7f1f180709e0 2024-07-17T07:07:59.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.851+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 98875+0+0 (secure 0 0 0) 0x7f1f08019020 con 0x7f1f180709e0 2024-07-17T07:07:59.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.852+0000 7f1f0fe00640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 0x7f1ef4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:07:59.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.852+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7f1f08098e00 con 0x7f1f180709e0 2024-07-17T07:07:59.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.852+0000 7f1f17400640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 0x7f1ef4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:07:59.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.854+0000 7f1f17400640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 0x7f1ef4079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f181f2020 tx=0x7f1f00009040 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:07:59.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:07:59.857+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1f1806cc40 con 0x7f1f180709e0 2024-07-17T07:08:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.070+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f1f181f4430 con 0x7f1f180709e0 2024-07-17T07:08:00.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.077+0000 7f1f0fe00640 1 -- 172.21.15.44:0/854958214 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7f1f08020020 con 0x7f1f180709e0 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.080+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 msgr2=0x7f1ef4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.080+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 0x7f1ef4079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f181f2020 tx=0x7f1f00009040 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.080+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 msgr2=0x7f1f18156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.080+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800cb30 tx=0x7f1f0800a060 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 shutdown_connections 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1ef4076c40 0x7f1ef4079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f18157d10 0x7f1f18143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f180afb30 0x7f1f18143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 --2- 172.21.15.44:0/854958214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f180709e0 0x7f1f18156b00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 >> 172.21.15.44:0/854958214 conn(0x7f1f180b2240 msgr2=0x7f1f1814dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:00.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 shutdown_connections 2024-07-17T07:08:00.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.081+0000 7f1f1f200640 1 -- 172.21.15.44:0/854958214 wait complete. 2024-07-17T07:08:00.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 -- 172.21.15.44:0/2125863891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3420 msgr2=0x7fb9240b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 --2- 172.21.15.44:0/2125863891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3420 0x7fb9240b4a30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb918005fd0 tx=0x7fb91802a980 comp rx=0 tx=0).stop 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 -- 172.21.15.44:0/2125863891 shutdown_connections 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 --2- 172.21.15.44:0/2125863891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240cded0 0x7fb9240ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 --2- 172.21.15.44:0/2125863891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3420 0x7fb9240b4a30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.308+0000 7fb92aa00640 1 -- 172.21.15.44:0/2125863891 >> 172.21.15.44:0/2125863891 conn(0x7fb92408ca20 msgr2=0x7fb92408ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.309+0000 7fb92aa00640 1 -- 172.21.15.44:0/2125863891 shutdown_connections 2024-07-17T07:08:00.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.309+0000 7fb92aa00640 1 -- 172.21.15.44:0/2125863891 wait complete. 2024-07-17T07:08:00.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.310+0000 7fb92aa00640 1 Processor -- start 2024-07-17T07:08:00.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 -- start start 2024-07-17T07:08:00.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9240cded0 0x7fb9241f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:00.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240d3420 0x7fb92414e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:00.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:00.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240ce8a0 con 0x7fb9240d3800 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240cea20 con 0x7fb9240cded0 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.311+0000 7fb92aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240ceba0 con 0x7fb9240d3420 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56972/0 (socket says 172.21.15.44:56972) 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 -- 172.21.15.44:0/1554892231 learned_addr learned my addr 172.21.15.44:0/1554892231 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb929600640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9240cded0 0x7fb9241f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb928c00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240d3420 0x7fb92414e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 -- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240d3420 msgr2=0x7fb92414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240d3420 0x7fb92414e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 -- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9240cded0 msgr2=0x7fb9241f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9240cded0 0x7fb9241f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.312+0000 7fb92a000640 1 -- 172.21.15.44:0/1554892231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241f86b0 con 0x7fb9240d3800 2024-07-17T07:08:00.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb92a000640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d750 tx=0x7fb91c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:00.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017070 con 0x7fb9240d3800 2024-07-17T07:08:00.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb91c009cc0 con 0x7fb9240d3800 2024-07-17T07:08:00.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00de60 con 0x7fb9240d3800 2024-07-17T07:08:00.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92414ee80 con 0x7fb9240d3800 2024-07-17T07:08:00.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.313+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9240b8350 con 0x7fb9240d3800 2024-07-17T07:08:00.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.314+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb924050670 con 0x7fb9240d3800 2024-07-17T07:08:00.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.316+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 98875+0+0 (secure 0 0 0) 0x7fb91c005280 con 0x7fb9240d3800 2024-07-17T07:08:00.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.317+0000 7fb921600640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 0x7fb904079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:00.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.317+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7fb91c0987d0 con 0x7fb9240d3800 2024-07-17T07:08:00.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.318+0000 7fb929600640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 0x7fb904079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:00.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.319+0000 7fb929600640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 0x7fb904079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb918035000 tx=0x7fb918004ab0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:00.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.322+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb924050670 con 0x7fb9240d3800 2024-07-17T07:08:00.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:00.614+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) -- 0x7fb9240d04a0 con 0x7fb9240d3800 2024-07-17T07:08:01.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.106+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fb91c005570 con 0x7fb9240d3800 2024-07-17T07:08:01.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.618+0000 7fb921600640 1 -- 172.21.15.44:0/1554892231 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v477) ==== 158+0+0 (secure 0 0 0) 0x7fb9240d04a0 con 0x7fb9240d3800 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 msgr2=0x7fb904079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 0x7fb904079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb918035000 tx=0x7fb918004ab0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 msgr2=0x7fb9240b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d750 tx=0x7fb91c00dc10 comp rx=0 tx=0).stop 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 shutdown_connections 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076c40 0x7fb904079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9240d3800 0x7fb9240b11f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9240d3420 0x7fb92414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 --2- 172.21.15.44:0/1554892231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9240cded0 0x7fb9241f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 >> 172.21.15.44:0/1554892231 conn(0x7fb92408ca20 msgr2=0x7fb92406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:01.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 shutdown_connections 2024-07-17T07:08:01.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.623+0000 7fb92aa00640 1 -- 172.21.15.44:0/1554892231 wait complete. 2024-07-17T07:08:01.644 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2024-07-17T07:08:01.645 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2024-07-17T07:08:01.645 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:08:01.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.856+0000 7ff7bbe00640 1 -- 172.21.15.44:0/2276797657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 msgr2=0x7ff7bc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.856+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/2276797657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc06e060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac005fd0 tx=0x7ff7ac02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 -- 172.21.15.44:0/2276797657 shutdown_connections 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/2276797657 >> v2:172.21.15.175:3300/0 conn(0x7ff7bc09b6e0 0x7ff7bc0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/2276797657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc06e060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 -- 172.21.15.44:0/2276797657 >> 172.21.15.44:0/2276797657 conn(0x7ff7bc0b0e80 msgr2=0x7ff7bc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 -- 172.21.15.44:0/2276797657 shutdown_connections 2024-07-17T07:08:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.857+0000 7ff7bbe00640 1 -- 172.21.15.44:0/2276797657 wait complete. 2024-07-17T07:08:01.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.859+0000 7ff7bbe00640 1 Processor -- start 2024-07-17T07:08:01.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.859+0000 7ff7bbe00640 1 -- start start 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc09b6e0 0x7ff7bc1f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b2120 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b1fa0 con 0x7ff7bc09b6e0 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc0b22a0 con 0x7ff7bc09b300 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56984/0 (socket says 172.21.15.44:56984) 2024-07-17T07:08:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.860+0000 7ff7bb400640 1 -- 172.21.15.44:0/1838894363 learned_addr learned my addr 172.21.15.44:0/1838894363 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7baa00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7b3e00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc09b6e0 0x7ff7bc1f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 -- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 msgr2=0x7ff7bc1f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc1f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 -- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc09b6e0 msgr2=0x7ff7bc1f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc09b6e0 0x7ff7bc1f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 -- 172.21.15.44:0/1838894363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc14e010 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7bb400640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c6b0 tx=0x7ff7b400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:01.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.861+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003b70 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.862+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b4003d30 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.862+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4005330 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.862+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc14e2f0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.862+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1f4580 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.863+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc0b04b0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.864+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7ff7b40054d0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.865+0000 7ff7b2a00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 0x7ff798079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:01.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.865+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7ff7b40985b0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.866+0000 7ff7baa00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 0x7ff798079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:01.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.867+0000 7ff7baa00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 0x7ff798079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac033000 tx=0x7ff7ac004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:01.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:01.870+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff7bc0b04b0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:02.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.083+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7bc06e060 con 0x7ff7bc0cf8c0 2024-07-17T07:08:02.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.085+0000 7ff7b2a00640 1 -- 172.21.15.44:0/1838894363 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+14977 (secure 0 0 0) 0x7ff7b40615d0 con 0x7ff7bc0cf8c0 2024-07-17T07:08:02.086 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:02.086 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":59,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:07:59.609619+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1597326512":"2024-07-18T07:07:56.578776+0000","172.21.15.44:6834/1597326512":"2024-07-18T07:07:56.578776+0000","172.21.15.175:0/2427812812":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/3091472317":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/733122570":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6800/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/2038460701":"2024-07-18T06:56:17.331799+0000","172.21.15.175:6801/557887042":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/3902401046":"2024-07-18T07:07:56.422481+0000","172.21.15.175:6801/2568152110":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/3025199526":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/2626487684":"2024-07-18T07:07:56.422481+0000","172.21.15.175:6800/2568152110":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/4015639457":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/1066293572":"2024-07-18T07:07:56.422481+0000","172.21.15.175:0/2225237696":"2024-07-18T06:56:17.331799+0000","172.21.15.175:0/2406941271":"2024-07-18T07:07:56.422481+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:08:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 msgr2=0x7ff798079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 0x7ff798079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac033000 tx=0x7ff7ac004730 comp rx=0 tx=0).stop 2024-07-17T07:08:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 msgr2=0x7ff7bc14d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c6b0 tx=0x7ff7b400cb70 comp rx=0 tx=0).stop 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 shutdown_connections 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff798076cd0 0x7ff798079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7bc0cf8c0 0x7ff7bc14d8e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7bc09b6e0 0x7ff7bc1f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 --2- 172.21.15.44:0/1838894363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7bc09b300 0x7ff7bc1f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 >> 172.21.15.44:0/1838894363 conn(0x7ff7bc0b0e80 msgr2=0x7ff7bc15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.088+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 shutdown_connections 2024-07-17T07:08:02.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.089+0000 7ff7bbe00640 1 -- 172.21.15.44:0/1838894363 wait complete. 2024-07-17T07:08:02.110 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:08:02.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:08:02.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd37a400640 1 -- 172.21.15.44:0/2614134831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd358004800 msgr2=0x7fd358004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd37a400640 1 --2- 172.21.15.44:0/2614134831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd358004800 0x7fd358004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c260 tx=0x7fd358005830 comp rx=0 tx=0).stop 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd373e00640 1 -- 172.21.15.44:0/2614134831 reap_dead start 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd37a400640 1 -- 172.21.15.44:0/2614134831 shutdown_connections 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd37a400640 1 -- 172.21.15.44:0/2614134831 >> 172.21.15.44:0/2614134831 conn(0x7fd3740b0eb0 msgr2=0x7fd3740b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.327+0000 7fd37a400640 1 -- 172.21.15.44:0/2614134831 shutdown_connections 2024-07-17T07:08:02.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.328+0000 7fd37a400640 1 -- 172.21.15.44:0/2614134831 wait complete. 2024-07-17T07:08:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.329+0000 7fd37a400640 1 Processor -- start 2024-07-17T07:08:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.329+0000 7fd37a400640 1 -- start start 2024-07-17T07:08:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd358004800 0x7fd3741505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3741f47c0 0x7fd3741e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd37406d9b0 con 0x7fd3741e5b90 2024-07-17T07:08:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd37406d830 con 0x7fd3741f47c0 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.330+0000 7fd37a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd37406db30 con 0x7fd358004800 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56998/0 (socket says 172.21.15.44:56998) 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 -- 172.21.15.44:0/307711441 learned_addr learned my addr 172.21.15.44:0/307711441 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd358004800 0x7fd3741505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd36be00640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3741f47c0 0x7fd3741e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 -- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd358004800 msgr2=0x7fd3741505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd358004800 0x7fd3741505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 -- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3741f47c0 msgr2=0x7fd3741e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3741f47c0 0x7fd3741e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.331+0000 7fd373e00640 1 -- 172.21.15.44:0/307711441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741e6030 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd373e00640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd3740ad6c0 tx=0x7fd36c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c004b20 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd36c004db0 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00cd90 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741f3ce0 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.332+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741f42a0 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.333+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3741f4470 con 0x7fd3741e5b90 2024-07-17T07:08:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.335+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fd36c01b030 con 0x7fd3741e5b90 2024-07-17T07:08:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.336+0000 7fd36b400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 0x7fd3500791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.336+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7fd36c0981f0 con 0x7fd3741e5b90 2024-07-17T07:08:02.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.336+0000 7fd373400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 0x7fd3500791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.338+0000 7fd373400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 0x7fd3500791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3640099c0 tx=0x7fd364005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:02.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.340+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd36c022060 con 0x7fd3741e5b90 2024-07-17T07:08:02.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.553+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd374075550 con 0x7fd3741e5b90 2024-07-17T07:08:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.554+0000 7fd36b400640 1 -- 172.21.15.44:0/307711441 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fd36c061310 con 0x7fd3741e5b90 2024-07-17T07:08:02.554 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 msgr2=0x7fd3500791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 0x7fd3500791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3640099c0 tx=0x7fd364005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 msgr2=0x7fd374158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd3740ad6c0 tx=0x7fd36c00eec0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 shutdown_connections 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fd350076d20 0x7fd3500791e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3741e5b90 0x7fd374158a40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3741f47c0 0x7fd3741e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 --2- 172.21.15.44:0/307711441 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd358004800 0x7fd3741505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 >> 172.21.15.44:0/307711441 conn(0x7fd3740b0eb0 msgr2=0x7fd37415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 shutdown_connections 2024-07-17T07:08:02.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.557+0000 7fd37a400640 1 -- 172.21.15.44:0/307711441 wait complete. 2024-07-17T07:08:02.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 -- 172.21.15.44:0/2226783929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb80052c0 msgr2=0x7effb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 --2- 172.21.15.44:0/2226783929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb80052c0 0x7effb800b9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c630 tx=0x7effb8005830 comp rx=0 tx=0).stop 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 -- 172.21.15.44:0/2226783929 shutdown_connections 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 --2- 172.21.15.44:0/2226783929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb80052c0 0x7effb800b9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 -- 172.21.15.44:0/2226783929 >> 172.21.15.44:0/2226783929 conn(0x7effd40b0e80 msgr2=0x7effd40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 -- 172.21.15.44:0/2226783929 shutdown_connections 2024-07-17T07:08:02.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.787+0000 7effd9a00640 1 -- 172.21.15.44:0/2226783929 wait complete. 2024-07-17T07:08:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.789+0000 7effd9a00640 1 Processor -- start 2024-07-17T07:08:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.789+0000 7effd9a00640 1 -- start start 2024-07-17T07:08:02.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80052c0 0x7effd41e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd40aebc0 0x7effd413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effd40b3a80 con 0x7effd409cbc0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effd40ac270 con 0x7effb80052c0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effd40b3c00 con 0x7effd40aebc0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57016/0 (socket says 172.21.15.44:57016) 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd9000640 1 -- 172.21.15.44:0/1330487139 learned_addr learned my addr 172.21.15.44:0/1330487139 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effd3e00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80052c0 0x7effd41e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.791+0000 7effcbe00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd40aebc0 0x7effd413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 -- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd40aebc0 msgr2=0x7effd413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd40aebc0 0x7effd413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 -- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80052c0 msgr2=0x7effd41e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80052c0 0x7effd41e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:02.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 -- 172.21.15.44:0/1330487139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd41e82e0 con 0x7effd409cbc0 2024-07-17T07:08:02.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9000640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c750 tx=0x7effcc003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:02.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc039070 con 0x7effd409cbc0 2024-07-17T07:08:02.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effcc02bae0 con 0x7effd409cbc0 2024-07-17T07:08:02.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd40ae0e0 con 0x7effd409cbc0 2024-07-17T07:08:02.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.792+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc00c280 con 0x7effd409cbc0 2024-07-17T07:08:02.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.793+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd40ae6a0 con 0x7effd409cbc0 2024-07-17T07:08:02.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.794+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd415e7f0 con 0x7effd409cbc0 2024-07-17T07:08:02.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.795+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7effcc006140 con 0x7effd409cbc0 2024-07-17T07:08:02.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.796+0000 7effd0c00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 0x7effb0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:02.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.796+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7effcc0b81e0 con 0x7effd409cbc0 2024-07-17T07:08:02.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.797+0000 7effd3e00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 0x7effb0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:02.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.798+0000 7effd3e00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 0x7effb0079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effd40ce9a0 tx=0x7effc4005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:02.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:02.802+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7effcc081330 con 0x7effd409cbc0 2024-07-17T07:08:03.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.090+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) -- 0x7effd41410e0 con 0x7effd409cbc0 2024-07-17T07:08:03.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.632+0000 7effd0c00640 1 -- 172.21.15.44:0/1330487139 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v479) ==== 157+0+0 (secure 0 0 0) 0x7effcc080c30 con 0x7effd409cbc0 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.635+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 msgr2=0x7effb0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.635+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 0x7effb0079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effd40ce9a0 tx=0x7effc4005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.635+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 msgr2=0x7effd40af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.635+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c750 tx=0x7effcc003c00 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 shutdown_connections 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7effb0076cd0 0x7effb0079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd409cbc0 0x7effd40af590 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effd40aebc0 0x7effd413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 --2- 172.21.15.44:0/1330487139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80052c0 0x7effd41e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 >> 172.21.15.44:0/1330487139 conn(0x7effd40b0e80 msgr2=0x7effd40b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:03.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 shutdown_connections 2024-07-17T07:08:03.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.636+0000 7effd9a00640 1 -- 172.21.15.44:0/1330487139 wait complete. 2024-07-17T07:08:03.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:08:03.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 -- 172.21.15.44:0/1687021805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e980051c0 msgr2=0x7f4e9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1687021805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e980051c0 0x7f4e9800b9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c630 tx=0x7f4e98005830 comp rx=0 tx=0).stop 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 -- 172.21.15.44:0/1687021805 shutdown_connections 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1687021805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e980051c0 0x7f4e9800b9c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 -- 172.21.15.44:0/1687021805 >> 172.21.15.44:0/1687021805 conn(0x7f4eb806bf40 msgr2=0x7f4eb806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.867+0000 7f4ebe800640 1 -- 172.21.15.44:0/1687021805 shutdown_connections 2024-07-17T07:08:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.868+0000 7f4ebe800640 1 -- 172.21.15.44:0/1687021805 wait complete. 2024-07-17T07:08:03.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.869+0000 7f4ebe800640 1 Processor -- start 2024-07-17T07:08:03.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.869+0000 7f4ebe800640 1 -- start start 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e980051c0 0x7f4eb80ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eb80af090 0x7f4eb80af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80cbfb0 con 0x7f4eb80abdc0 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80cbe30 con 0x7f4e980051c0 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.870+0000 7f4ebe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80cc130 con 0x7f4eb80af090 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57032/0 (socket says 172.21.15.44:57032) 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 -- 172.21.15.44:0/1804111890 learned_addr learned my addr 172.21.15.44:0/1804111890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb6a00640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eb80af090 0x7f4eb80af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7400640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e980051c0 0x7f4eb80ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 -- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eb80af090 msgr2=0x7f4eb80af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eb80af090 0x7f4eb80af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 -- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e980051c0 msgr2=0x7f4eb80ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e980051c0 0x7f4eb80ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.871+0000 7f4eb7e00640 1 -- 172.21.15.44:0/1804111890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb8132f30 con 0x7f4eb80abdc0 2024-07-17T07:08:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4eb7e00640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c720 tx=0x7f4eb0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0039070 con 0x7f4eb80abdc0 2024-07-17T07:08:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eb002bcf0 con 0x7f4eb80abdc0 2024-07-17T07:08:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb8133210 con 0x7f4eb80abdc0 2024-07-17T07:08:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000c280 con 0x7f4eb80abdc0 2024-07-17T07:08:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.872+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81337d0 con 0x7f4eb80abdc0 2024-07-17T07:08:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.873+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80cc690 con 0x7f4eb80abdc0 2024-07-17T07:08:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.874+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f4eb00063a0 con 0x7f4eb80abdc0 2024-07-17T07:08:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.875+0000 7f4eaf400640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 0x7f4e94079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.875+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7f4eb00b81c0 con 0x7f4eb80abdc0 2024-07-17T07:08:03.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.876+0000 7f4eb7400640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 0x7f4e94079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:03.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.877+0000 7f4eb7400640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 0x7f4e94079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009aa0 tx=0x7f4ea8005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:03.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:03.880+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4eb0081310 con 0x7f4eb80abdc0 2024-07-17T07:08:04.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.092+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4eb8132c70 con 0x7f4eb80abdc0 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.093+0000 7f4eaf400640 1 -- 172.21.15.44:0/1804111890 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7f4eb0003e20 con 0x7f4eb80abdc0 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:08:04.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:08:04.094 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:08:04.095 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:08:04.095 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:08:04.095 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:08:04.095 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.095+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 msgr2=0x7f4e94079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.095+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 0x7f4e94079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009aa0 tx=0x7f4ea8005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 msgr2=0x7f4eb80ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c720 tx=0x7f4eb0003c00 comp rx=0 tx=0).stop 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 shutdown_connections 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4e94076cd0 0x7f4e94079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb80abdc0 0x7f4eb80ac240 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4eb80af090 0x7f4eb80af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 --2- 172.21.15.44:0/1804111890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e980051c0 0x7f4eb80ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 >> 172.21.15.44:0/1804111890 conn(0x7f4eb806bf40 msgr2=0x7f4eb80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 shutdown_connections 2024-07-17T07:08:04.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.096+0000 7f4ebe800640 1 -- 172.21.15.44:0/1804111890 wait complete. 2024-07-17T07:08:04.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:08:04.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 -- 172.21.15.44:0/1040129929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 msgr2=0x7f2e1815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1040129929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e1815a450 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2e1815ce40 tx=0x7f2e0802a560 comp rx=0 tx=0).stop 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 -- 172.21.15.44:0/1040129929 shutdown_connections 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1040129929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e181503c0 0x7f2e180ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1040129929 >> v2:172.21.15.175:3300/0 conn(0x7f2e1806d420 0x7f2e1806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1040129929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e1815a450 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 -- 172.21.15.44:0/1040129929 >> 172.21.15.44:0/1040129929 conn(0x7f2e180cf0d0 msgr2=0x7f2e180cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 -- 172.21.15.44:0/1040129929 shutdown_connections 2024-07-17T07:08:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.326+0000 7f2e1d600640 1 -- 172.21.15.44:0/1040129929 wait complete. 2024-07-17T07:08:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.328+0000 7f2e1d600640 1 Processor -- start 2024-07-17T07:08:04.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.329+0000 7f2e1d600640 1 -- start start 2024-07-17T07:08:04.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1806d420 0x7f2e181f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:04.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:04.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e181503c0 0x7f2e18158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:04.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180b3240 con 0x7f2e1806f4d0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180b30c0 con 0x7f2e181503c0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e1d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e180b33c0 con 0x7f2e1806d420 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57056/0 (socket says 172.21.15.44:57056) 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.330+0000 7f2e0fe00640 1 -- 172.21.15.44:0/1702229131 learned_addr learned my addr 172.21.15.44:0/1702229131 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e17e00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1806d420 0x7f2e181f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 -- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1806d420 msgr2=0x7f2e181f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1806d420 0x7f2e181f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 -- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e181503c0 msgr2=0x7f2e18158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e1cc00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e181503c0 0x7f2e18158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e181503c0 0x7f2e18158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 -- 172.21.15.44:0/1702229131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e180ce560 con 0x7f2e1806f4d0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e0fe00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000dae0 tx=0x7f2e0000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:04.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.331+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000ae30 con 0x7f2e1806f4d0 2024-07-17T07:08:04.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.332+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e00005110 con 0x7f2e1806f4d0 2024-07-17T07:08:04.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.332+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00005540 con 0x7f2e1806f4d0 2024-07-17T07:08:04.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.332+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e180c9610 con 0x7f2e1806f4d0 2024-07-17T07:08:04.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.332+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e18146a30 con 0x7f2e1806f4d0 2024-07-17T07:08:04.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.333+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1815a450 con 0x7f2e1806f4d0 2024-07-17T07:08:04.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.334+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f2e000047a0 con 0x7f2e1806f4d0 2024-07-17T07:08:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.335+0000 7f2e14c00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 0x7f2df4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.335+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7f2e00098e70 con 0x7f2e1806f4d0 2024-07-17T07:08:04.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.336+0000 7f2e17e00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 0x7f2df4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:04.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.337+0000 7f2e17e00640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 0x7f2df4079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2e08033000 tx=0x7f2e08004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:04.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.341+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2e00061e90 con 0x7f2e1806f4d0 2024-07-17T07:08:04.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.552+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e180cffe0 con 0x7f2e1806f4d0 2024-07-17T07:08:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.560+0000 7f2e14c00640 1 -- 172.21.15.44:0/1702229131 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f2e0006a0d0 con 0x7f2e1806f4d0 2024-07-17T07:08:04.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 msgr2=0x7f2df4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 0x7f2df4079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2e08033000 tx=0x7f2e08004730 comp rx=0 tx=0).stop 2024-07-17T07:08:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 msgr2=0x7f2e181f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000dae0 tx=0x7f2e0000a250 comp rx=0 tx=0).stop 2024-07-17T07:08:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 shutdown_connections 2024-07-17T07:08:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2df4076cd0 0x7f2df4079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e181503c0 0x7f2e18158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e1806f4d0 0x7f2e181f4470 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 --2- 172.21.15.44:0/1702229131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e1806d420 0x7f2e181f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 >> 172.21.15.44:0/1702229131 conn(0x7f2e180cf0d0 msgr2=0x7f2e1815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.563+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 shutdown_connections 2024-07-17T07:08:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:04.564+0000 7f2e1d600640 1 -- 172.21.15.44:0/1702229131 wait complete. 2024-07-17T07:08:04.583 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:08:04.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:08:04.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:08:04.813 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:08:04.813 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:08:04.813 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:08:04.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:08:04.869 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:08:04.869 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:08:04.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:08:04.925 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:08:04.925 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:08:04.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:08:04.979 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:08:04.979 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:08:05.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:08:05.035 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:08:05.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.258+0000 7fb92de00640 1 -- 172.21.15.44:0/3963787862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf540 msgr2=0x7fb9280b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.258+0000 7fb92de00640 1 --2- 172.21.15.44:0/3963787862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf540 0x7fb9280b3870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb918009830 tx=0x7fb91802a480 comp rx=0 tx=0).stop 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.258+0000 7fb92de00640 1 -- 172.21.15.44:0/3963787862 shutdown_connections 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.258+0000 7fb92de00640 1 --2- 172.21.15.44:0/3963787862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf540 0x7fb9280b3870 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.258+0000 7fb92de00640 1 -- 172.21.15.44:0/3963787862 >> 172.21.15.44:0/3963787862 conn(0x7fb9280b0e80 msgr2=0x7fb9280b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.259+0000 7fb92de00640 1 -- 172.21.15.44:0/3963787862 shutdown_connections 2024-07-17T07:08:05.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.259+0000 7fb92de00640 1 -- 172.21.15.44:0/3963787862 wait complete. 2024-07-17T07:08:05.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.260+0000 7fb92de00640 1 Processor -- start 2024-07-17T07:08:05.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.260+0000 7fb92de00640 1 -- start start 2024-07-17T07:08:05.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9280b3db0 0x7fb9281f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 0x7fb928073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280ae380 con 0x7fb9280cf920 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9280ae200 con 0x7fb9280cf540 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.261+0000 7fb92de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9280ae500 con 0x7fb9280b3db0 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57076/0 (socket says 172.21.15.44:57076) 2024-07-17T07:08:05.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 -- 172.21.15.44:0/2722178961 learned_addr learned my addr 172.21.15.44:0/2722178961 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:05.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb927e00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 0x7fb928073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 -- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9280b3db0 msgr2=0x7fb9281f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92ca00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9280b3db0 0x7fb9281f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9280b3db0 0x7fb9281f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 -- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 msgr2=0x7fb928073920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 0x7fb928073920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb92d400640 1 -- 172.21.15.44:0/2722178961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928072880 con 0x7fb9280cf920 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.262+0000 7fb927e00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 0x7fb928073920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb92d400640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c6c0 tx=0x7fb92000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920017070 con 0x7fb9280cf920 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb920003c60 con 0x7fb9280cf920 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200052d0 con 0x7fb9280cf920 2024-07-17T07:08:05.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb928072b60 con 0x7fb9280cf920 2024-07-17T07:08:05.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.263+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928073120 con 0x7fb9280cf920 2024-07-17T07:08:05.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.264+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92813f670 con 0x7fb9280cf920 2024-07-17T07:08:05.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.265+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fb92001a070 con 0x7fb9280cf920 2024-07-17T07:08:05.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.267+0000 7fb924c00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 0x7fb904079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.267+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7fb920097e30 con 0x7fb9280cf920 2024-07-17T07:08:05.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.268+0000 7fb92ca00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 0x7fb904079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.269+0000 7fb92ca00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 0x7fb904079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb918034000 tx=0x7fb918004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:05.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.271+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb92813f670 con 0x7fb9280cf920 2024-07-17T07:08:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.528+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb92813dd70 con 0x7fb9280cf920 2024-07-17T07:08:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.529+0000 7fb924c00640 1 -- 172.21.15.44:0/2722178961 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3911 (secure 0 0 0) 0x7fb920060e50 con 0x7fb9280cf920 2024-07-17T07:08:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:05.529 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"btime":"2024-07-17T07:07:57:583075+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/232097644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":232097644},{"type":"v1","addr":"172.21.15.175:6836","nonce":232097644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2380824577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2380824577},{"type":"v1","addr":"172.21.15.175:6837","nonce":2380824577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/664356210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":664356210},{"type":"v1","addr":"172.21.15.44:6833","nonce":664356210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2014516048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2014516048},{"type":"v1","addr":"172.21.15.44:6835","nonce":2014516048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-07-17T07:08:05.529 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 19 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 msgr2=0x7fb904079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 0x7fb904079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb918034000 tx=0x7fb918004730 comp rx=0 tx=0).stop 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 msgr2=0x7fb928156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c6c0 tx=0x7fb92000cb80 comp rx=0 tx=0).stop 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 shutdown_connections 2024-07-17T07:08:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb904076cd0 0x7fb904079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9280cf920 0x7fb928156ca0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9280cf540 0x7fb928073920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 --2- 172.21.15.44:0/2722178961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9280b3db0 0x7fb9281f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 >> 172.21.15.44:0/2722178961 conn(0x7fb9280b0e80 msgr2=0x7fb92813fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 shutdown_connections 2024-07-17T07:08:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.532+0000 7fb92de00640 1 -- 172.21.15.44:0/2722178961 wait complete. 2024-07-17T07:08:05.551 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:08:05.551 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:08:05.551 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:08:05.552 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:08:05.552 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:08:05.552 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:08:05.552 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:08:05.552 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:08:05.552 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:08:05.553 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:08:05.553 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:08:05.553 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:08:05.553 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:08:05.553 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:08:05.553 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:08:05.553 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:08:05.571 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:08:05.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:08:05.572 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:08:05.572 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:08:05.578 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:08:05.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:08:05.580 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:08:05.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:08:05.619 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:08:05.619 DEBUG:teuthology.parallel:result is None 2024-07-17T07:08:05.625 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:08:05.625 DEBUG:teuthology.parallel:result is None 2024-07-17T07:08:05.626 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:08:05.626 DEBUG:teuthology.parallel:result is None 2024-07-17T07:08:05.630 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:08:05.631 DEBUG:teuthology.parallel:result is None 2024-07-17T07:08:05.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:08:05.767 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:08:05.768 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:08:05.778 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:08:05.779 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 -- 172.21.15.44:0/2794491608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d806c650 msgr2=0x7fe7d80726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2794491608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d806c650 0x7fe7d80726f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8075990 tx=0x7fe7d402a8b0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 -- 172.21.15.44:0/2794491608 shutdown_connections 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2794491608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d806c650 0x7fe7d80726f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2794491608 >> v2:172.21.15.175:3300/0 conn(0x7fe7d80afa90 0x7fe7d815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 -- 172.21.15.44:0/2794491608 >> 172.21.15.44:0/2794491608 conn(0x7fe7d806d050 msgr2=0x7fe7d806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 -- 172.21.15.44:0/2794491608 shutdown_connections 2024-07-17T07:08:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.813+0000 7fe7e0800640 1 -- 172.21.15.44:0/2794491608 wait complete. 2024-07-17T07:08:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.815+0000 7fe7e0800640 1 Processor -- start 2024-07-17T07:08:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.815+0000 7fe7e0800640 1 -- start start 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 0x7fe7d81446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 0x7fe7d8138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8072c30 con 0x7fe7d815f9f0 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d813ea70 con 0x7fe7d80afa90 2024-07-17T07:08:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7e0800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d8072db0 con 0x7fe7d806c650 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57110/0 (socket says 172.21.15.44:57110) 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.816+0000 7fe7de800640 1 -- 172.21.15.44:0/2936359219 learned_addr learned my addr 172.21.15.44:0/2936359219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7dd400640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 0x7fe7d8138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7dde00640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 0x7fe7d81446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 -- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 msgr2=0x7fe7d81446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 0x7fe7d81446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 -- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 msgr2=0x7fe7d8138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 0x7fe7d8138480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 -- 172.21.15.44:0/2936359219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d813c460 con 0x7fe7d815f9f0 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7dd400640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 0x7fe7d8138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7dde00640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 0x7fe7d81446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7de800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009ca0 tx=0x7fe7d4031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4040070 con 0x7fe7d815f9f0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7d4004040 con 0x7fe7d815f9f0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.817+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4004470 con 0x7fe7d815f9f0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.818+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d813c740 con 0x7fe7d815f9f0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.818+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81e4d50 con 0x7fe7d815f9f0 2024-07-17T07:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.819+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d81e5a10 con 0x7fe7d815f9f0 2024-07-17T07:08:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.820+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fe7d4039040 con 0x7fe7d815f9f0 2024-07-17T07:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.821+0000 7fe7d2000640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 0x7fe7bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.821+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6472+0+0 (secure 0 0 0) 0x7fe7d40b9f30 con 0x7fe7d815f9f0 2024-07-17T07:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.822+0000 7fe7dde00640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 0x7fe7bc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.823+0000 7fe7dde00640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 0x7fe7bc0791c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009b50 tx=0x7fe7cc005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:05.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:05.826+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7d81e5a10 con 0x7fe7d815f9f0 2024-07-17T07:08:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.047+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe7d8073280 con 0x7fe7d815f9f0 2024-07-17T07:08:06.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.671+0000 7fe7d2000640 1 -- 172.21.15.44:0/2936359219 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7fe7d4082e40 con 0x7fe7d815f9f0 2024-07-17T07:08:06.671 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:08:06.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.674+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 msgr2=0x7fe7bc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:06.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.674+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 0x7fe7bc0791c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009b50 tx=0x7fe7cc005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.674+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 msgr2=0x7fe7d81e63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.674+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009ca0 tx=0x7fe7d4031720 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 shutdown_connections 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fe7bc076d00 0x7fe7bc0791c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d815f9f0 0x7fe7d81e63f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d80afa90 0x7fe7d8138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 --2- 172.21.15.44:0/2936359219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d806c650 0x7fe7d81446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 >> 172.21.15.44:0/2936359219 conn(0x7fe7d806d050 msgr2=0x7fe7d806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 shutdown_connections 2024-07-17T07:08:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.675+0000 7fe7e0800640 1 -- 172.21.15.44:0/2936359219 wait complete. 2024-07-17T07:08:06.697 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 -- 172.21.15.44:0/975895308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 msgr2=0x7f25f00b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 --2- 172.21.15.44:0/975895308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f00b3870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f25f013e5e0 tx=0x7f25e002a4d0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 -- 172.21.15.44:0/975895308 shutdown_connections 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 --2- 172.21.15.44:0/975895308 >> v2:172.21.15.175:3300/0 conn(0x7f25f00b3db0 0x7f25f00ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 --2- 172.21.15.44:0/975895308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f00b3870 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 -- 172.21.15.44:0/975895308 >> 172.21.15.44:0/975895308 conn(0x7f25f00b0e80 msgr2=0x7f25f00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:06.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 -- 172.21.15.44:0/975895308 shutdown_connections 2024-07-17T07:08:06.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.902+0000 7f25f5800640 1 -- 172.21.15.44:0/975895308 wait complete. 2024-07-17T07:08:06.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.904+0000 7f25f5800640 1 Processor -- start 2024-07-17T07:08:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.904+0000 7f25f5800640 1 -- start start 2024-07-17T07:08:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25f00b3db0 0x7f25f0156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 0x7f25f01f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00ae8c0 con 0x7f25f00cf540 2024-07-17T07:08:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25f00ae740 con 0x7f25f00cf920 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25f5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f25f00aea40 con 0x7f25f00b3db0 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57138/0 (socket says 172.21.15.44:57138) 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.905+0000 7f25e7400640 1 -- 172.21.15.44:0/2258050457 learned_addr learned my addr 172.21.15.44:0/2258050457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25f4e00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 0x7f25f01f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25efe00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25f00b3db0 0x7f25f0156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 -- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25f00b3db0 msgr2=0x7f25f0156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25f00b3db0 0x7f25f0156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 -- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 msgr2=0x7f25f01f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 0x7f25f01f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 -- 172.21.15.44:0/2258050457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f0155a60 con 0x7f25f00cf540 2024-07-17T07:08:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25f4e00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 0x7f25f01f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25e7400640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f25d800d740 tx=0x7f25d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8017070 con 0x7f25f00cf540 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d800de50 con 0x7f25f00cf540 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8005020 con 0x7f25f00cf540 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.906+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f01568c0 con 0x7f25f00cf540 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.907+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f0072400 con 0x7f25f00cf540 2024-07-17T07:08:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.908+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f01412e0 con 0x7f25f00cf540 2024-07-17T07:08:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.909+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f25d80051c0 con 0x7f25f00cf540 2024-07-17T07:08:06.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.910+0000 7f25ecc00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 0x7f25cc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:06.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.910+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5741+0+0 (secure 0 0 0) 0x7f25d80986f0 con 0x7f25f00cf540 2024-07-17T07:08:06.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.911+0000 7f25efe00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 0x7f25cc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:06.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.912+0000 7f25efe00640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 0x7f25cc079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f25e0033000 tx=0x7f25e0004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:06.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:06.914+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f25d8061970 con 0x7f25f00cf540 2024-07-17T07:08:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.206+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f25f015a490 con 0x7f25f00cf540 2024-07-17T07:08:07.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.207+0000 7f25ecc00640 1 -- 172.21.15.44:0/2258050457 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7f25d8061270 con 0x7f25f00cf540 2024-07-17T07:08:07.207 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:07.207 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.208 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:08:07.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:08:07.212 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.214 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.215 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.216 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.217 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:08:07.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.219 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:08:07.220 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:08:07.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 msgr2=0x7f25cc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 0x7f25cc079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f25e0033000 tx=0x7f25e0004730 comp rx=0 tx=0).stop 2024-07-17T07:08:07.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 msgr2=0x7f25f0158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f25d800d740 tx=0x7f25d800dc00 comp rx=0 tx=0).stop 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 shutdown_connections 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f25cc076cd0 0x7f25cc079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25f00cf920 0x7f25f01f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25f00cf540 0x7f25f0158470 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.210+0000 7f25f5800640 1 --2- 172.21.15.44:0/2258050457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25f00b3db0 0x7f25f0156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.211+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 >> 172.21.15.44:0/2258050457 conn(0x7f25f00b0e80 msgr2=0x7f25f015ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.211+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 shutdown_connections 2024-07-17T07:08:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.211+0000 7f25f5800640 1 -- 172.21.15.44:0/2258050457 wait complete. 2024-07-17T07:08:07.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 -- 172.21.15.44:0/1176556462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f897c004800 msgr2=0x7f897c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 --2- 172.21.15.44:0/1176556462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f897c004800 0x7f897c004c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8980001eb0 tx=0x7f897c005830 comp rx=0 tx=0).stop 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899a200640 1 -- 172.21.15.44:0/1176556462 reap_dead start 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 -- 172.21.15.44:0/1176556462 shutdown_connections 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 -- 172.21.15.44:0/1176556462 >> 172.21.15.44:0/1176556462 conn(0x7f89940b3120 msgr2=0x7f89940b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:07.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 -- 172.21.15.44:0/1176556462 shutdown_connections 2024-07-17T07:08:07.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.455+0000 7f899c200640 1 -- 172.21.15.44:0/1176556462 wait complete. 2024-07-17T07:08:07.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.457+0000 7f899c200640 1 Processor -- start 2024-07-17T07:08:07.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.457+0000 7f899c200640 1 -- start start 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c004800 0x7f8994151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8994154f40 0x7f89940ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89940d11c0 con 0x7f89940afcf0 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89940cd200 con 0x7f897c004800 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.458+0000 7f899c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89940ccb60 con 0x7f8994154f40 2024-07-17T07:08:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57158/0 (socket says 172.21.15.44:57158) 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 -- 172.21.15.44:0/2707807480 learned_addr learned my addr 172.21.15.44:0/2707807480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f8999800640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c004800 0x7f8994151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f8998e00640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8994154f40 0x7f89940ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 -- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8994154f40 msgr2=0x7f89940ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8994154f40 0x7f89940ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 -- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c004800 msgr2=0x7f8994151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c004800 0x7f8994151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.459+0000 7f899a200640 1 -- 172.21.15.44:0/2707807480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940aec50 con 0x7f89940afcf0 2024-07-17T07:08:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f899a200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f89940cdac0 tx=0x7f898c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00ea00 con 0x7f89940afcf0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f898c00ec90 con 0x7f89940afcf0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c005280 con 0x7f89940afcf0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89940aef30 con 0x7f89940afcf0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.460+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89940af4f0 con 0x7f89940afcf0 2024-07-17T07:08:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.461+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8994070460 con 0x7f89940afcf0 2024-07-17T07:08:07.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.462+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f898c01d020 con 0x7f89940afcf0 2024-07-17T07:08:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.463+0000 7f8991600640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 0x7f89740791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.463+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5741+0+0 (secure 0 0 0) 0x7f898c098490 con 0x7f89940afcf0 2024-07-17T07:08:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.464+0000 7f8999800640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 0x7f89740791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.465+0000 7f8999800640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 0x7f89740791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f8988009500 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:07.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.468+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f898c061680 con 0x7f89940afcf0 2024-07-17T07:08:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.689+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f89940b1c90 con 0x7f89940afcf0 2024-07-17T07:08:07.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.689+0000 7f8991600640 1 -- 172.21.15.44:0/2707807480 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f898c060f80 con 0x7f89940afcf0 2024-07-17T07:08:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:07.689 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:08:07.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:08:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:08:07.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:08:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.692+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 msgr2=0x7f89740791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.692+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 0x7f89740791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f8988009500 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.692+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 msgr2=0x7f89940b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.692+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f89940cdac0 tx=0x7f898c010da0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 shutdown_connections 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8974076d00 0x7f89740791c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89940afcf0 0x7f89940b0170 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8994154f40 0x7f89940ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 --2- 172.21.15.44:0/2707807480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f897c004800 0x7f8994151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 >> 172.21.15.44:0/2707807480 conn(0x7f89940b3120 msgr2=0x7f89940b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 shutdown_connections 2024-07-17T07:08:07.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.693+0000 7f899c200640 1 -- 172.21.15.44:0/2707807480 wait complete. 2024-07-17T07:08:07.713 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:08:07.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:08:07.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.929+0000 7fb393e00640 1 -- 172.21.15.44:0/4243623990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 msgr2=0x7fb378004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.929+0000 7fb393e00640 1 --2- 172.21.15.44:0/4243623990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb378004c80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb3940d0050 tx=0x7fb378005830 comp rx=0 tx=0).stop 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.929+0000 7fb393400640 1 -- 172.21.15.44:0/4243623990 reap_dead start 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.929+0000 7fb393e00640 1 -- 172.21.15.44:0/4243623990 shutdown_connections 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.929+0000 7fb393e00640 1 -- 172.21.15.44:0/4243623990 >> 172.21.15.44:0/4243623990 conn(0x7fb3940cf050 msgr2=0x7fb3940cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.930+0000 7fb393e00640 1 -- 172.21.15.44:0/4243623990 shutdown_connections 2024-07-17T07:08:07.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.930+0000 7fb393e00640 1 -- 172.21.15.44:0/4243623990 wait complete. 2024-07-17T07:08:07.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.931+0000 7fb393e00640 1 Processor -- start 2024-07-17T07:08:07.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.932+0000 7fb393e00640 1 -- start start 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.932+0000 7fb393e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb39414ce80 0x7fb39413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39413d5f0 0x7fb394153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb394072ef0 con 0x7fb378004800 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3940ad4e0 con 0x7fb39413d5f0 2024-07-17T07:08:07.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb394073070 con 0x7fb39414ce80 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57178/0 (socket says 172.21.15.44:57178) 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 -- 172.21.15.44:0/2512566851 learned_addr learned my addr 172.21.15.44:0/2512566851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392000640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb39414ce80 0x7fb39413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb393400640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39413d5f0 0x7fb394153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 -- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb39414ce80 msgr2=0x7fb39413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb39414ce80 0x7fb39413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 -- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39413d5f0 msgr2=0x7fb394153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39413d5f0 0x7fb394153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.933+0000 7fb392a00640 1 -- 172.21.15.44:0/2512566851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb394140090 con 0x7fb378004800 2024-07-17T07:08:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb392a00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3840119b0 tx=0x7fb384011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38400fcf0 con 0x7fb378004800 2024-07-17T07:08:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb384005020 con 0x7fb378004800 2024-07-17T07:08:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384005450 con 0x7fb378004800 2024-07-17T07:08:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39409ac60 con 0x7fb378004800 2024-07-17T07:08:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.934+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39409a980 con 0x7fb378004800 2024-07-17T07:08:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.935+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3940729b0 con 0x7fb378004800 2024-07-17T07:08:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.936+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fb3840047a0 con 0x7fb378004800 2024-07-17T07:08:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.937+0000 7fb38aa00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 0x7fb374079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:07.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.938+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5741+0+0 (secure 0 0 0) 0x7fb384099260 con 0x7fb378004800 2024-07-17T07:08:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.938+0000 7fb392000640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 0x7fb374079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.940+0000 7fb392000640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 0x7fb374079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3940d0050 tx=0x7fb380001cb0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:07.942+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb394140090 con 0x7fb378004800 2024-07-17T07:08:08.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.160+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb39415b1b0 con 0x7fb378004800 2024-07-17T07:08:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.689+0000 7fb38aa00640 1 -- 172.21.15.44:0/2512566851 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7fb384062560 con 0x7fb378004800 2024-07-17T07:08:08.690 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:08:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.693+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 msgr2=0x7fb374079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.693+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 0x7fb374079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3940d0050 tx=0x7fb380001cb0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.693+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 msgr2=0x7fb39413fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.693+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3840119b0 tx=0x7fb384011e70 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 shutdown_connections 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fb374076cd0 0x7fb374079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb39413d5f0 0x7fb394153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb39414ce80 0x7fb39413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 --2- 172.21.15.44:0/2512566851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb378004800 0x7fb39413fb50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 >> 172.21.15.44:0/2512566851 conn(0x7fb3940cf050 msgr2=0x7fb39415db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 shutdown_connections 2024-07-17T07:08:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.694+0000 7fb393e00640 1 -- 172.21.15.44:0/2512566851 wait complete. 2024-07-17T07:08:08.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:08:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 -- 172.21.15.44:0/3942540808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0b55d0 msgr2=0x7f359c0d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 --2- 172.21.15.44:0/3942540808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0b55d0 0x7f359c0d18f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3584005fd0 tx=0x7f358402a540 comp rx=0 tx=0).stop 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 -- 172.21.15.44:0/3942540808 shutdown_connections 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 --2- 172.21.15.44:0/3942540808 >> v2:172.21.15.175:3300/0 conn(0x7f359c0d2fa0 0x7f359c0d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 --2- 172.21.15.44:0/3942540808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0b55d0 0x7f359c0d18f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 -- 172.21.15.44:0/3942540808 >> 172.21.15.44:0/3942540808 conn(0x7f359c0cfec0 msgr2=0x7f359c0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 -- 172.21.15.44:0/3942540808 shutdown_connections 2024-07-17T07:08:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.925+0000 7f35a1a00640 1 -- 172.21.15.44:0/3942540808 wait complete. 2024-07-17T07:08:08.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.927+0000 7f35a1a00640 1 Processor -- start 2024-07-17T07:08:08.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.927+0000 7f35a1a00640 1 -- start start 2024-07-17T07:08:08.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c0b55d0 0x7f359c0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:08.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0d2fa0 0x7f359c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0d1220 con 0x7f359c15d300 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0d2370 con 0x7f359c0b55d0 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.928+0000 7f35a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f359c0cb5e0 con 0x7f359c0d2fa0 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50936/0 (socket says 172.21.15.44:50936) 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 -- 172.21.15.44:0/2591302762 learned_addr learned my addr 172.21.15.44:0/2591302762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f3593e00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0d2fa0 0x7f359c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f359be00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c0b55d0 0x7f359c0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 -- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0d2fa0 msgr2=0x7f359c14b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0d2fa0 0x7f359c14b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 -- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c0b55d0 msgr2=0x7f359c0c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c0b55d0 0x7f359c0c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.929+0000 7f35a1000640 1 -- 172.21.15.44:0/2591302762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c0cb810 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f35a1000640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f359400eb40 tx=0x7f359400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594004b20 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3594004db0 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594005280 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c0cbaf0 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.930+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c0cc0b0 con 0x7f359c15d300 2024-07-17T07:08:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.931+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c0d2850 con 0x7f359c15d300 2024-07-17T07:08:08.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.933+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f3594005420 con 0x7f359c15d300 2024-07-17T07:08:08.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.933+0000 7f3598c00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 0x7f3578079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:08.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.934+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6111+0+0 (secure 0 0 0) 0x7f3594099640 con 0x7f359c15d300 2024-07-17T07:08:08.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.934+0000 7f359be00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 0x7f3578079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:08.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.935+0000 7f359be00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 0x7f3578079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f358c0015e0 tx=0x7f358c009040 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:08.939+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f35940626c0 con 0x7f359c15d300 2024-07-17T07:08:09.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.154+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f359c0ca2d0 con 0x7f359c15d300 2024-07-17T07:08:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.709+0000 7f3598c00640 1 -- 172.21.15.44:0/2591302762 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7f3594061fc0 con 0x7f359c15d300 2024-07-17T07:08:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:08:09.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.712+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 msgr2=0x7f3578079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.712+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 0x7f3578079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f358c0015e0 tx=0x7f358c009040 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.712+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 msgr2=0x7f359c0cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.712+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f359400eb40 tx=0x7f359400c2d0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 shutdown_connections 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f3578076cd0 0x7f3578079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359c15d300 0x7f359c0cc5d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f359c0d2fa0 0x7f359c14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 --2- 172.21.15.44:0/2591302762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f359c0b55d0 0x7f359c0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.713+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 >> 172.21.15.44:0/2591302762 conn(0x7f359c0cfec0 msgr2=0x7f359c0b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.714+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 shutdown_connections 2024-07-17T07:08:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.715+0000 7f35a1a00640 1 -- 172.21.15.44:0/2591302762 wait complete. 2024-07-17T07:08:09.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.934+0000 7f8183c00640 1 -- 172.21.15.44:0/180067470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f815c003a80 msgr2=0x7f815c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.934+0000 7f8183c00640 1 --2- 172.21.15.44:0/180067470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f815c003a80 0x7f815c003ea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f816c00bb50 tx=0x7f815c005830 comp rx=0 tx=0).stop 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.934+0000 7f8181c00640 1 -- 172.21.15.44:0/180067470 reap_dead start 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.935+0000 7f8183c00640 1 -- 172.21.15.44:0/180067470 shutdown_connections 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.935+0000 7f8183c00640 1 -- 172.21.15.44:0/180067470 >> 172.21.15.44:0/180067470 conn(0x7f817c0b0eb0 msgr2=0x7f817c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.935+0000 7f8183c00640 1 -- 172.21.15.44:0/180067470 shutdown_connections 2024-07-17T07:08:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.935+0000 7f8183c00640 1 -- 172.21.15.44:0/180067470 wait complete. 2024-07-17T07:08:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.937+0000 7f8183c00640 1 Processor -- start 2024-07-17T07:08:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.937+0000 7f8183c00640 1 -- start start 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c003a80 0x7f817c148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f817c072bc0 0x7f817c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f817c15c470 con 0x7f817c15a7c0 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f817c15c2f0 con 0x7f817c072bc0 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8183c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f817c15c5f0 con 0x7f815c003a80 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50954/0 (socket says 172.21.15.44:50954) 2024-07-17T07:08:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 -- 172.21.15.44:0/1950275654 learned_addr learned my addr 172.21.15.44:0/1950275654 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f8181c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f817c072bc0 0x7f817c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 -- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c003a80 msgr2=0x7f817c148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c003a80 0x7f817c148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 -- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f817c072bc0 msgr2=0x7f817c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f817c072bc0 0x7f817c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.938+0000 7f817be00640 1 -- 172.21.15.44:0/1950275654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c073580 con 0x7f817c15a7c0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f817be00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8164006d10 tx=0x7f8164001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164015070 con 0x7f817c15a7c0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f816400e9e0 con 0x7f817c15a7c0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400ee30 con 0x7f817c15a7c0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c0725f0 con 0x7f817c15a7c0 2024-07-17T07:08:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.939+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c071b10 con 0x7f817c15a7c0 2024-07-17T07:08:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.940+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817c158190 con 0x7f817c15a7c0 2024-07-17T07:08:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.942+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f8164018070 con 0x7f817c15a7c0 2024-07-17T07:08:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.943+0000 7f8178c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 0x7f8158079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.943+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6493+0+0 (secure 0 0 0) 0x7f81640995f0 con 0x7f817c15a7c0 2024-07-17T07:08:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.944+0000 7f8181200640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 0x7f8158079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.945+0000 7f8181200640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 0x7f8158079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f816c036000 tx=0x7f816c004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:09.947+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8164062570 con 0x7f817c15a7c0 2024-07-17T07:08:10.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.202+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f817c150520 con 0x7f817c15a7c0 2024-07-17T07:08:10.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.737+0000 7f8178c00640 1 -- 172.21.15.44:0/1950275654 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7f8164061e70 con 0x7f817c15a7c0 2024-07-17T07:08:10.739 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 7 and data pool 8 2024-07-17T07:08:10.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.747+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 msgr2=0x7f8158079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:10.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.747+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 0x7f8158079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f816c036000 tx=0x7f816c004730 comp rx=0 tx=0).stop 2024-07-17T07:08:10.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.747+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 msgr2=0x7f817c15ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:10.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.747+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8164006d10 tx=0x7f8164001d20 comp rx=0 tx=0).stop 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 shutdown_connections 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8158076cd0 0x7f8158079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f817c072bc0 0x7f817c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f817c15a7c0 0x7f817c15ac40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 --2- 172.21.15.44:0/1950275654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c003a80 0x7f817c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.751+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 >> 172.21.15.44:0/1950275654 conn(0x7f817c0b0eb0 msgr2=0x7f817c0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:10.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.753+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 shutdown_connections 2024-07-17T07:08:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.754+0000 7f8183c00640 1 -- 172.21.15.44:0/1950275654 wait complete. 2024-07-17T07:08:10.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:08:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 -- 172.21.15.44:0/503066398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 msgr2=0x7fdaa406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 --2- 172.21.15.44:0/503066398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa406e480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40cf2f0 tx=0x7fda9402a5f0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 -- 172.21.15.44:0/503066398 shutdown_connections 2024-07-17T07:08:10.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 --2- 172.21.15.44:0/503066398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b2ad0 0x7fdaa40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 --2- 172.21.15.44:0/503066398 >> v2:172.21.15.175:3300/0 conn(0x7fdaa40b5740 0x7fdaa4161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 --2- 172.21.15.44:0/503066398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa406e480 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 -- 172.21.15.44:0/503066398 >> 172.21.15.44:0/503066398 conn(0x7fdaa40d00a0 msgr2=0x7fdaa40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 -- 172.21.15.44:0/503066398 shutdown_connections 2024-07-17T07:08:10.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.966+0000 7fdaa9600640 1 -- 172.21.15.44:0/503066398 wait complete. 2024-07-17T07:08:10.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.968+0000 7fdaa9600640 1 Processor -- start 2024-07-17T07:08:10.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.968+0000 7fdaa9600640 1 -- start start 2024-07-17T07:08:10.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaa40b2ad0 0x7fdaa41f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b5740 0x7fdaa4152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa41618f0 con 0x7fdaa415d600 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4161770 con 0x7fdaa40b2ad0 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.969+0000 7fdaa9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa4161a70 con 0x7fdaa40b5740 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50976/0 (socket says 172.21.15.44:50976) 2024-07-17T07:08:10.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 -- 172.21.15.44:0/408519250 learned_addr learned my addr 172.21.15.44:0/408519250 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:10.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa2a00640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaa40b2ad0 0x7fdaa41f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:10.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa2000640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b5740 0x7fdaa4152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:10.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 -- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b5740 msgr2=0x7fdaa4152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b5740 0x7fdaa4152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 -- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaa40b2ad0 msgr2=0x7fdaa41f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaa40b2ad0 0x7fdaa41f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.970+0000 7fdaa3400640 1 -- 172.21.15.44:0/408519250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa415ab10 con 0x7fdaa415d600 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fdaa3400640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00cb00 tx=0x7fda9c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c004b20 con 0x7fdaa415d600 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda9c004db0 con 0x7fdaa415d600 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa41f4960 con 0x7fdaa415d600 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c005280 con 0x7fdaa415d600 2024-07-17T07:08:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.971+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41f4f20 con 0x7fdaa415d600 2024-07-17T07:08:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.972+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4161020 con 0x7fdaa415d600 2024-07-17T07:08:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.974+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fda9c005420 con 0x7fdaa415d600 2024-07-17T07:08:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.974+0000 7fda9aa00640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 0x7fda80079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:10.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.975+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6593+0+0 (secure 0 0 0) 0x7fda9c099130 con 0x7fdaa415d600 2024-07-17T07:08:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.975+0000 7fdaa2a00640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 0x7fda80079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.976+0000 7fdaa2a00640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 0x7fda80079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda94033000 tx=0x7fda94004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:10.979+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fda9c014030 con 0x7fdaa415d600 2024-07-17T07:08:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.188+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaa40cdf60 con 0x7fdaa415d600 2024-07-17T07:08:11.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.191+0000 7fda9aa00640 1 -- 172.21.15.44:0/408519250 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17685 (secure 0 0 0) 0x7fda9c06a020 con 0x7fdaa415d600 2024-07-17T07:08:11.192 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:08:10.205042+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:08:08.174976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-07-17T07:08:09.173646+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 msgr2=0x7fda80079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 0x7fda80079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda94033000 tx=0x7fda94004730 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 msgr2=0x7fdaa415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00cb00 tx=0x7fda9c00a250 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 shutdown_connections 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fda80076cd0 0x7fda80079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.194+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdaa415d600 0x7fdaa415b2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.195+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdaa40b5740 0x7fdaa4152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.195+0000 7fdaa9600640 1 --2- 172.21.15.44:0/408519250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaa40b2ad0 0x7fdaa41f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.195+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 >> 172.21.15.44:0/408519250 conn(0x7fdaa40d00a0 msgr2=0x7fdaa40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.195+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 shutdown_connections 2024-07-17T07:08:11.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.195+0000 7fdaa9600640 1 -- 172.21.15.44:0/408519250 wait complete. 2024-07-17T07:08:11.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1302724412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 msgr2=0x7ff0a40ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1302724412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a40ac560 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fd0 tx=0x7ff09c02a820 comp rx=0 tx=0).stop 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1302724412 shutdown_connections 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1302724412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a40ac560 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1302724412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a409ca20 0x7ff0a40cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.421+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1302724412 >> 172.21.15.44:0/1302724412 conn(0x7ff0a40b0e80 msgr2=0x7ff0a40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.422+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1302724412 shutdown_connections 2024-07-17T07:08:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.422+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1302724412 wait complete. 2024-07-17T07:08:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.423+0000 7ff0a3e00640 1 Processor -- start 2024-07-17T07:08:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.423+0000 7ff0a3e00640 1 -- start start 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0a409ca20 0x7ff0a41555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a415d240 0x7ff0a4159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a40ace70 con 0x7ff0a40cf810 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a40accf0 con 0x7ff0a409ca20 2024-07-17T07:08:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.424+0000 7ff0a3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a40acff0 con 0x7ff0a415d240 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50994/0 (socket says 172.21.15.44:50994) 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 -- 172.21.15.44:0/1704627947 learned_addr learned my addr 172.21.15.44:0/1704627947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a3400640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a415d240 0x7ff0a4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2a00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0a409ca20 0x7ff0a41555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 -- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a415d240 msgr2=0x7ff0a4159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a415d240 0x7ff0a4159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 -- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0a409ca20 msgr2=0x7ff0a41555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0a409ca20 0x7ff0a41555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 -- 172.21.15.44:0/1704627947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a4155f00 con 0x7ff0a40cf810 2024-07-17T07:08:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.425+0000 7ff0a2000640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d950 tx=0x7ff08c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.426+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017070 con 0x7ff0a40cf810 2024-07-17T07:08:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.426+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff08c009d20 con 0x7ff0a40cf810 2024-07-17T07:08:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.426+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0106c0 con 0x7ff0a40cf810 2024-07-17T07:08:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.426+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a4155c20 con 0x7ff0a40cf810 2024-07-17T07:08:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.426+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a415ec10 con 0x7ff0a40cf810 2024-07-17T07:08:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.427+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a409bd60 con 0x7ff0a40cf810 2024-07-17T07:08:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.428+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7ff08c010860 con 0x7ff0a40cf810 2024-07-17T07:08:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.429+0000 7ff09aa00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 0x7ff080079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.430+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6593+0+0 (secure 0 0 0) 0x7ff08c098d30 con 0x7ff0a40cf810 2024-07-17T07:08:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.430+0000 7ff0a2a00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 0x7ff080079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.431+0000 7ff0a2a00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 0x7ff080079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff094001610 tx=0x7ff094009040 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.434+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff0a409bd60 con 0x7ff0a40cf810 2024-07-17T07:08:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.648+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0a40ceb70 con 0x7ff0a40cf810 2024-07-17T07:08:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.650+0000 7ff09aa00640 1 -- 172.21.15.44:0/1704627947 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17685 (secure 0 0 0) 0x7ff08c09e020 con 0x7ff0a40cf810 2024-07-17T07:08:11.650 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:11.651 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:08:10.205042+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:08:08.174976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-07-17T07:08:09.173646+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:08:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 msgr2=0x7ff080079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 0x7ff080079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff094001610 tx=0x7ff094009040 comp rx=0 tx=0).stop 2024-07-17T07:08:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 msgr2=0x7ff0a41543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00d950 tx=0x7ff08c00de10 comp rx=0 tx=0).stop 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 shutdown_connections 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff080076cd0 0x7ff080079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0a415d240 0x7ff0a4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a40cf810 0x7ff0a41543d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 --2- 172.21.15.44:0/1704627947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0a409ca20 0x7ff0a41555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 >> 172.21.15.44:0/1704627947 conn(0x7ff0a40b0e80 msgr2=0x7ff0a40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.653+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 shutdown_connections 2024-07-17T07:08:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.654+0000 7ff0a3e00640 1 -- 172.21.15.44:0/1704627947 wait complete. 2024-07-17T07:08:11.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.879+0000 7f039c600640 1 -- 172.21.15.44:0/427288494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c004d40 msgr2=0x7f037c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.879+0000 7f039c600640 1 --2- 172.21.15.44:0/427288494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c004d40 0x7f037c0051a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f038c001dc0 tx=0x7f037c005830 comp rx=0 tx=0).stop 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.879+0000 7f039a600640 1 -- 172.21.15.44:0/427288494 reap_dead start 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.880+0000 7f039c600640 1 -- 172.21.15.44:0/427288494 shutdown_connections 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.880+0000 7f039c600640 1 -- 172.21.15.44:0/427288494 >> 172.21.15.44:0/427288494 conn(0x7f03940b0eb0 msgr2=0x7f03940b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.880+0000 7f039c600640 1 -- 172.21.15.44:0/427288494 shutdown_connections 2024-07-17T07:08:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.880+0000 7f039c600640 1 -- 172.21.15.44:0/427288494 wait complete. 2024-07-17T07:08:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.881+0000 7f039c600640 1 Processor -- start 2024-07-17T07:08:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.882+0000 7f039c600640 1 -- start start 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f037c004d40 0x7f03941f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 0x7f039409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f039415a5a0 con 0x7f039415bc30 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03940accc0 con 0x7f039409cf30 2024-07-17T07:08:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f039415a720 con 0x7f037c004d40 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51002/0 (socket says 172.21.15.44:51002) 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 -- 172.21.15.44:0/2960633968 learned_addr learned my addr 172.21.15.44:0/2960633968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f0399c00640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f037c004d40 0x7f03941f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f0399200640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 0x7f039409aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 -- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f037c004d40 msgr2=0x7f03941f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f037c004d40 0x7f03941f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 -- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 msgr2=0x7f039409aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 0x7f039409aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f039a600640 1 -- 172.21.15.44:0/2960633968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03941f5720 con 0x7f039415bc30 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.883+0000 7f0399200640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 0x7f039409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f039a600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f038c02acc0 tx=0x7f038c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0062b0 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f038c006450 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0068a0 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039409afa0 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.884+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03941f5990 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.886+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03940ac3a0 con 0x7f039415bc30 2024-07-17T07:08:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.887+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f038c003a30 con 0x7f039415bc30 2024-07-17T07:08:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.888+0000 7f0391600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 0x7f0374079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.888+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f038c0b8320 con 0x7f039415bc30 2024-07-17T07:08:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.888+0000 7f0399c00640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 0x7f0374079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.890+0000 7f0399c00640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 0x7f0374079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0388009aa0 tx=0x7f0388005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:11.892+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f038c0812c0 con 0x7f039415bc30 2024-07-17T07:08:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.153+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03940b4200 con 0x7f039415bc30 2024-07-17T07:08:12.685 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T07:08:12.686+0000 7fce27e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:08:12.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.760+0000 7f0391600640 1 -- 172.21.15.44:0/2960633968 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f038c080bc0 con 0x7f039415bc30 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 msgr2=0x7f0374079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 0x7f0374079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0388009aa0 tx=0x7f0388005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 msgr2=0x7f039414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f038c02acc0 tx=0x7f038c001b10 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 shutdown_connections 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0374076cd0 0x7f0374079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f039415bc30 0x7f039414d540 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f039409cf30 0x7f039409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 --2- 172.21.15.44:0/2960633968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f037c004d40 0x7f03941f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.764+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 >> 172.21.15.44:0/2960633968 conn(0x7f03940b0eb0 msgr2=0x7f03940aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.765+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 shutdown_connections 2024-07-17T07:08:12.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:12.765+0000 7f039c600640 1 -- 172.21.15.44:0/2960633968 wait complete. 2024-07-17T07:08:12.784 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:08:12.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.000+0000 7f872ee00640 1 -- 172.21.15.44:0/406908747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c004800 msgr2=0x7f870c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.000+0000 7f872ee00640 1 --2- 172.21.15.44:0/406908747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c004800 0x7f870c004c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f871800afd0 tx=0x7f870c005830 comp rx=0 tx=0).stop 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.001+0000 7f872ee00640 1 -- 172.21.15.44:0/406908747 shutdown_connections 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.001+0000 7f872ee00640 1 --2- 172.21.15.44:0/406908747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f870c004800 0x7f870c004c80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.001+0000 7f872ee00640 1 -- 172.21.15.44:0/406908747 >> 172.21.15.44:0/406908747 conn(0x7f87280b0eb0 msgr2=0x7f87280b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.001+0000 7f872ee00640 1 -- 172.21.15.44:0/406908747 shutdown_connections 2024-07-17T07:08:13.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.001+0000 7f872ee00640 1 -- 172.21.15.44:0/406908747 wait complete. 2024-07-17T07:08:13.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.003+0000 7f872ee00640 1 Processor -- start 2024-07-17T07:08:13.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.003+0000 7f872ee00640 1 -- start start 2024-07-17T07:08:13.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870c004800 0x7f87280ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:13.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 0x7f87281524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87280b3280 con 0x7f87281f8200 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f872806f8e0 con 0x7f8728152020 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87280b3400 con 0x7f870c004800 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51034/0 (socket says 172.21.15.44:51034) 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 -- 172.21.15.44:0/1270612410 learned_addr learned my addr 172.21.15.44:0/1270612410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f872ce00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 0x7f87281524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 -- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870c004800 msgr2=0x7f87280ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727e00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870c004800 0x7f87280ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870c004800 0x7f87280ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 -- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 msgr2=0x7f87281524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:13.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 0x7f87281524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.004+0000 7f8727400640 1 -- 172.21.15.44:0/1270612410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87280ca910 con 0x7f87281f8200 2024-07-17T07:08:13.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f872ce00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 0x7f87281524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:13.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f8727400640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f871800aa60 tx=0x7f8718032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871803f030 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f871800a6f0 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718004250 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87280cabf0 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.005+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8728154c50 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.007+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87280ae560 con 0x7f87281f8200 2024-07-17T07:08:13.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.008+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f8718042030 con 0x7f87281f8200 2024-07-17T07:08:13.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.009+0000 7f871fe00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 0x7f8704079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:13.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.009+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f87180b9300 con 0x7f87281f8200 2024-07-17T07:08:13.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.010+0000 7f8727e00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 0x7f8704079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:13.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.011+0000 7f8727e00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 0x7f8704079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f872806b660 tx=0x7f8710006a00 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:13.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.014+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8718082110 con 0x7f87281f8200 2024-07-17T07:08:13.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.276+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f87280b4200 con 0x7f87281f8200 2024-07-17T07:08:13.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.764+0000 7f871fe00640 1 -- 172.21.15.44:0/1270612410 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f8718081a10 con 0x7f87281f8200 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 msgr2=0x7f8704079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 0x7f8704079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f872806b660 tx=0x7f8710006a00 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 msgr2=0x7f8728149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f871800aa60 tx=0x7f8718032d30 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 shutdown_connections 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f8704076cd0 0x7f8704079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8728152020 0x7f87281524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87281f8200 0x7f8728149570 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 --2- 172.21.15.44:0/1270612410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870c004800 0x7f87280ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 >> 172.21.15.44:0/1270612410 conn(0x7f87280b0eb0 msgr2=0x7f87280b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 shutdown_connections 2024-07-17T07:08:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:13.769+0000 7f872ee00640 1 -- 172.21.15.44:0/1270612410 wait complete. 2024-07-17T07:08:13.790 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:08:13.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.005+0000 7f26b7800640 1 -- 172.21.15.44:0/1221916333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 msgr2=0x7f26b006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.005+0000 7f26b7800640 1 --2- 172.21.15.44:0/1221916333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b006e4a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26b01576d0 tx=0x7f26a802b880 comp rx=0 tx=0).stop 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 -- 172.21.15.44:0/1221916333 shutdown_connections 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 --2- 172.21.15.44:0/1221916333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b006e4a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 --2- 172.21.15.44:0/1221916333 >> v2:172.21.15.175:3300/0 conn(0x7f26b00ac3e0 0x7f26b00ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 -- 172.21.15.44:0/1221916333 >> 172.21.15.44:0/1221916333 conn(0x7f26b00b0fb0 msgr2=0x7f26b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 -- 172.21.15.44:0/1221916333 shutdown_connections 2024-07-17T07:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.006+0000 7f26b7800640 1 -- 172.21.15.44:0/1221916333 wait complete. 2024-07-17T07:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.007+0000 7f26b7800640 1 Processor -- start 2024-07-17T07:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.008+0000 7f26b7800640 1 -- start start 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26b006f710 0x7f26b01f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26b00ac3e0 0x7f26b01e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0159c80 con 0x7f26b006e090 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0159b00 con 0x7f26b00ac3e0 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26b0159e00 con 0x7f26b006f710 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51058/0 (socket says 172.21.15.44:51058) 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 -- 172.21.15.44:0/3584084344 learned_addr learned my addr 172.21.15.44:0/3584084344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 -- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26b006f710 msgr2=0x7f26b01f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26afe00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26b006f710 0x7f26b01f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26b006f710 0x7f26b01f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 -- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26b00ac3e0 msgr2=0x7f26b01e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26b00ac3e0 0x7f26b01e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.009+0000 7f26b4e00640 1 -- 172.21.15.44:0/3584084344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b00958e0 con 0x7f26b006e090 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26b4e00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26a000c7a0 tx=0x7f26a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0017070 con 0x7f26b006e090 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26a0005070 con 0x7f26b006e090 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b0095bc0 con 0x7f26b006e090 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00054c0 con 0x7f26b006e090 2024-07-17T07:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.010+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b0151040 con 0x7f26b006e090 2024-07-17T07:08:14.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.011+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0050670 con 0x7f26b006e090 2024-07-17T07:08:14.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.013+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f26a001a070 con 0x7f26b006e090 2024-07-17T07:08:14.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.014+0000 7f26acc00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 0x7f2690079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.014+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f26a0097fe0 con 0x7f26b006e090 2024-07-17T07:08:14.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.014+0000 7f26afe00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 0x7f2690079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.016+0000 7f26afe00640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 0x7f2690079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b4460 tx=0x7f269c005c20 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.018+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f26b0050670 con 0x7f26b006e090 2024-07-17T07:08:14.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.275+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26b00ac870 con 0x7f26b006e090 2024-07-17T07:08:14.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.277+0000 7f26acc00640 1 -- 172.21.15.44:0/3584084344 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7f26a0061010 con 0x7f26b006e090 2024-07-17T07:08:14.276 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:14.276 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:08:14.276 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 msgr2=0x7f2690079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 0x7f2690079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b4460 tx=0x7f269c005c20 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 msgr2=0x7f26b0098730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26a000c7a0 tx=0x7f26a000cc60 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 shutdown_connections 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f2690076cd0 0x7f2690079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26b00ac3e0 0x7f26b01e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26b006f710 0x7f26b01f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 --2- 172.21.15.44:0/3584084344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26b006e090 0x7f26b0098730 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 >> 172.21.15.44:0/3584084344 conn(0x7f26b00b0fb0 msgr2=0x7f26b015bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 shutdown_connections 2024-07-17T07:08:14.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.280+0000 7f26b7800640 1 -- 172.21.15.44:0/3584084344 wait complete. 2024-07-17T07:08:14.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:08:14.473 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:08:14.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 -- 172.21.15.44:0/2995080064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 msgr2=0x7f5ac006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2995080064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac006e480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00b30b0 tx=0x7f5aa802a5f0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 -- 172.21.15.44:0/2995080064 shutdown_connections 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2995080064 >> v2:172.21.15.175:3300/0 conn(0x7f5ac00b2ad0 0x7f5ac00b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2995080064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac006e480 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 -- 172.21.15.44:0/2995080064 >> 172.21.15.44:0/2995080064 conn(0x7f5ac00d00a0 msgr2=0x7f5ac00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 -- 172.21.15.44:0/2995080064 shutdown_connections 2024-07-17T07:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.511+0000 7f5ac6000640 1 -- 172.21.15.44:0/2995080064 wait complete. 2024-07-17T07:08:14.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.513+0000 7f5ac6000640 1 Processor -- start 2024-07-17T07:08:14.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.513+0000 7f5ac6000640 1 -- start start 2024-07-17T07:08:14.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac00b2ad0 0x7f5ac015f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac00b5740 0x7f5ac00cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0161730 con 0x7f5ac015d600 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00cef10 con 0x7f5ac00b2ad0 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5ac6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac01618b0 con 0x7f5ac00b5740 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.514+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51078/0 (socket says 172.21.15.44:51078) 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 -- 172.21.15.44:0/2630686392 learned_addr learned my addr 172.21.15.44:0/2630686392 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac00b2ad0 0x7f5ac015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abea00640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac00b5740 0x7f5ac00cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 -- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac00b5740 msgr2=0x7f5ac00cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac00b5740 0x7f5ac00cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 -- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac00b2ad0 msgr2=0x7f5ac015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac00b2ad0 0x7f5ac015f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 -- 172.21.15.44:0/2630686392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac01f43e0 con 0x7f5ac015d600 2024-07-17T07:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.515+0000 7f5abfe00640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c390 tx=0x7f5ab800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.516+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8017070 con 0x7f5ac015d600 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.516+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ab80035f0 con 0x7f5ac015d600 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.516+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8003a40 con 0x7f5ac015d600 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.516+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01f46c0 con 0x7f5ac015d600 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.516+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01f40b0 con 0x7f5ac015d600 2024-07-17T07:08:14.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.517+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac00b5200 con 0x7f5ac015d600 2024-07-17T07:08:14.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.519+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f5ab8003be0 con 0x7f5ac015d600 2024-07-17T07:08:14.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.520+0000 7f5ab7400640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 0x7f5a9c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.520+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f5ab8097a60 con 0x7f5ac015d600 2024-07-17T07:08:14.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.521+0000 7f5abf400640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 0x7f5a9c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.521+0000 7f5abf400640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 0x7f5a9c079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8033000 tx=0x7f5aa8004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.525+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5ab8060450 con 0x7f5ac015d600 2024-07-17T07:08:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.739+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ac00c9740 con 0x7f5ac015d600 2024-07-17T07:08:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.742+0000 7f5ab7400640 1 -- 172.21.15.44:0/2630686392 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17672 (secure 0 0 0) 0x7f5ab80649a0 con 0x7f5ac015d600 2024-07-17T07:08:14.741 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:14.741 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":67,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:08:11.738906+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:08:08.174976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-07-17T07:08:09.173646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:08:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 msgr2=0x7f5a9c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 0x7f5a9c079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8033000 tx=0x7f5aa8004730 comp rx=0 tx=0).stop 2024-07-17T07:08:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 msgr2=0x7f5ac00ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c390 tx=0x7f5ab800c850 comp rx=0 tx=0).stop 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 shutdown_connections 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f5a9c076cd0 0x7f5a9c079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ac015d600 0x7f5ac00ccc90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac00b5740 0x7f5ac00cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 --2- 172.21.15.44:0/2630686392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac00b2ad0 0x7f5ac015f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.745+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 >> 172.21.15.44:0/2630686392 conn(0x7f5ac00d00a0 msgr2=0x7f5ac00cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.746+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 shutdown_connections 2024-07-17T07:08:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.746+0000 7f5ac6000640 1 -- 172.21.15.44:0/2630686392 wait complete. 2024-07-17T07:08:14.764 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:08:14.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:08:14.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 -- 172.21.15.44:0/4126518126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40ac3e0 msgr2=0x7f1eb40ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 --2- 172.21.15.44:0/4126518126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40ac3e0 0x7f1eb40ac840 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0005fd0 tx=0x7f1ea002b910 comp rx=0 tx=0).stop 2024-07-17T07:08:14.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 -- 172.21.15.44:0/4126518126 shutdown_connections 2024-07-17T07:08:14.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 --2- 172.21.15.44:0/4126518126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb40ac3e0 0x7f1eb40ac840 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 --2- 172.21.15.44:0/4126518126 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb406f710 0x7f1eb4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 -- 172.21.15.44:0/4126518126 >> 172.21.15.44:0/4126518126 conn(0x7f1eb40b0fb0 msgr2=0x7f1eb40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:14.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 -- 172.21.15.44:0/4126518126 shutdown_connections 2024-07-17T07:08:14.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.975+0000 7f1eb9400640 1 -- 172.21.15.44:0/4126518126 wait complete. 2024-07-17T07:08:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.977+0000 7f1eb9400640 1 Processor -- start 2024-07-17T07:08:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.977+0000 7f1eb9400640 1 -- start start 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ac3e0 0x7f1eb4098970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb413ed60 0x7f1eb4072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb40ae800 con 0x7f1eb406f710 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb406f0a0 con 0x7f1eb413ed60 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb40ae980 con 0x7f1eb40ac3e0 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.978+0000 7f1eb2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51098/0 (socket says 172.21.15.44:51098) 2024-07-17T07:08:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 -- 172.21.15.44:0/252229323 learned_addr learned my addr 172.21.15.44:0/252229323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 -- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ac3e0 msgr2=0x7f1eb4098970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb1e00640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ac3e0 0x7f1eb4098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb3200640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb413ed60 0x7f1eb4072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ac3e0 0x7f1eb4098970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 -- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb413ed60 msgr2=0x7f1eb4072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb413ed60 0x7f1eb4072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 -- 172.21.15.44:0/252229323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb4098eb0 con 0x7f1eb406f710 2024-07-17T07:08:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.979+0000 7f1eb2800640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400d950 tx=0x7f1ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.980+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1eb406f710 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.980+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ea4009d20 con 0x7f1eb406f710 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.980+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb4140cb0 con 0x7f1eb406f710 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.980+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40106c0 con 0x7f1eb406f710 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.980+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb4141770 con 0x7f1eb406f710 2024-07-17T07:08:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.981+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb41411d0 con 0x7f1eb406f710 2024-07-17T07:08:14.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.982+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f1ea4010860 con 0x7f1eb406f710 2024-07-17T07:08:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.983+0000 7f1eaaa00640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 0x7f1e94079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.983+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f1ea4098d30 con 0x7f1eb406f710 2024-07-17T07:08:14.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.984+0000 7f1eb1e00640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 0x7f1e94079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:14.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.985+0000 7f1eb1e00640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 0x7f1e94079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0035000 tx=0x7f1ea0004730 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:14.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:14.987+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1eb4098eb0 con 0x7f1eb406f710 2024-07-17T07:08:15.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.194+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 --> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1eb413fad0 con 0x7f1e94076cd0 2024-07-17T07:08:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.210+0000 7f1eaaa00640 1 -- 172.21.15.44:0/252229323 <== mgr.5071 v2:172.21.15.175:6800/533048536 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f1eb413fad0 con 0x7f1e94076cd0 2024-07-17T07:08:15.211 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 msgr2=0x7f1e94079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 0x7f1e94079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0035000 tx=0x7f1ea0004730 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 msgr2=0x7f1eb414f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400d950 tx=0x7f1ea400de10 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 shutdown_connections 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f1e94076cd0 0x7f1e94079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eb413ed60 0x7f1eb4072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1eb40ac3e0 0x7f1eb4098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 --2- 172.21.15.44:0/252229323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eb406f710 0x7f1eb414f020 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 >> 172.21.15.44:0/252229323 conn(0x7f1eb40b0fb0 msgr2=0x7f1eb414ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 shutdown_connections 2024-07-17T07:08:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.217+0000 7f1eb9400640 1 -- 172.21.15.44:0/252229323 wait complete. 2024-07-17T07:08:15.236 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:08:14.462761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":170,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313732,"kb_used_data":7384,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754660988,"statfs":{"total":773094113280,"available":772772851712,"internally_reserved":0,"allocated":7561216,"data_stored":4340972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":313577008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.768645"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255817+0000","last_change":"2024-07-17T07:08:10.760706+0000","last_active":"2024-07-17T07:08:12.255817+0000","last_peered":"2024-07-17T07:08:12.255817+0000","last_clean":"2024-07-17T07:08:12.255817+0000","last_became_active":"2024-07-17T07:08:10.760376+0000","last_became_peered":"2024-07-17T07:08:10.760376+0000","last_unstale":"2024-07-17T07:08:12.255817+0000","last_undegraded":"2024-07-17T07:08:12.255817+0000","last_fullsized":"2024-07-17T07:08:12.255817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:59.006584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257331+0000","last_change":"2024-07-17T07:08:10.756315+0000","last_active":"2024-07-17T07:08:12.257331+0000","last_peered":"2024-07-17T07:08:12.257331+0000","last_clean":"2024-07-17T07:08:12.257331+0000","last_became_active":"2024-07-17T07:08:10.755396+0000","last_became_peered":"2024-07-17T07:08:10.755396+0000","last_unstale":"2024-07-17T07:08:12.257331+0000","last_undegraded":"2024-07-17T07:08:12.257331+0000","last_fullsized":"2024-07-17T07:08:12.257331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:59.545790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260209+0000","last_change":"2024-07-17T07:08:10.763526+0000","last_active":"2024-07-17T07:08:12.260209+0000","last_peered":"2024-07-17T07:08:12.260209+0000","last_clean":"2024-07-17T07:08:12.260209+0000","last_became_active":"2024-07-17T07:08:10.763263+0000","last_became_peered":"2024-07-17T07:08:10.763263+0000","last_unstale":"2024-07-17T07:08:12.260209+0000","last_undegraded":"2024-07-17T07:08:12.260209+0000","last_fullsized":"2024-07-17T07:08:12.260209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:33.521693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258125+0000","last_change":"2024-07-17T07:08:10.754687+0000","last_active":"2024-07-17T07:08:12.258125+0000","last_peered":"2024-07-17T07:08:12.258125+0000","last_clean":"2024-07-17T07:08:12.258125+0000","last_became_active":"2024-07-17T07:08:10.754315+0000","last_became_peered":"2024-07-17T07:08:10.754315+0000","last_unstale":"2024-07-17T07:08:12.258125+0000","last_undegraded":"2024-07-17T07:08:12.258125+0000","last_fullsized":"2024-07-17T07:08:12.258125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:14.190805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758577+0000","last_change":"2024-07-17T07:08:10.753494+0000","last_active":"2024-07-17T07:08:11.758577+0000","last_peered":"2024-07-17T07:08:11.758577+0000","last_clean":"2024-07-17T07:08:11.758577+0000","last_became_active":"2024-07-17T07:08:10.753118+0000","last_became_peered":"2024-07-17T07:08:10.753118+0000","last_unstale":"2024-07-17T07:08:11.758577+0000","last_undegraded":"2024-07-17T07:08:11.758577+0000","last_fullsized":"2024-07-17T07:08:11.758577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:24.759613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254666+0000","last_change":"2024-07-17T07:08:10.760944+0000","last_active":"2024-07-17T07:08:12.254666+0000","last_peered":"2024-07-17T07:08:12.254666+0000","last_clean":"2024-07-17T07:08:12.254666+0000","last_became_active":"2024-07-17T07:08:10.758562+0000","last_became_peered":"2024-07-17T07:08:10.758562+0000","last_unstale":"2024-07-17T07:08:12.254666+0000","last_undegraded":"2024-07-17T07:08:12.254666+0000","last_fullsized":"2024-07-17T07:08:12.254666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:07.650347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260989+0000","last_change":"2024-07-17T07:08:10.760012+0000","last_active":"2024-07-17T07:08:12.260989+0000","last_peered":"2024-07-17T07:08:12.260989+0000","last_clean":"2024-07-17T07:08:12.260989+0000","last_became_active":"2024-07-17T07:08:10.759412+0000","last_became_peered":"2024-07-17T07:08:10.759412+0000","last_unstale":"2024-07-17T07:08:12.260989+0000","last_undegraded":"2024-07-17T07:08:12.260989+0000","last_fullsized":"2024-07-17T07:08:12.260989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:04.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256780+0000","last_change":"2024-07-17T07:08:10.767772+0000","last_active":"2024-07-17T07:08:12.256780+0000","last_peered":"2024-07-17T07:08:12.256780+0000","last_clean":"2024-07-17T07:08:12.256780+0000","last_became_active":"2024-07-17T07:08:10.767391+0000","last_became_peered":"2024-07-17T07:08:10.767391+0000","last_unstale":"2024-07-17T07:08:12.256780+0000","last_undegraded":"2024-07-17T07:08:12.256780+0000","last_fullsized":"2024-07-17T07:08:12.256780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:54.024791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258758+0000","last_change":"2024-07-17T07:08:10.775251+0000","last_active":"2024-07-17T07:08:12.258758+0000","last_peered":"2024-07-17T07:08:12.258758+0000","last_clean":"2024-07-17T07:08:12.258758+0000","last_became_active":"2024-07-17T07:08:10.773551+0000","last_became_peered":"2024-07-17T07:08:10.773551+0000","last_unstale":"2024-07-17T07:08:12.258758+0000","last_undegraded":"2024-07-17T07:08:12.258758+0000","last_fullsized":"2024-07-17T07:08:12.258758+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:16:21.029101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256279+0000","last_change":"2024-07-17T07:08:10.764393+0000","last_active":"2024-07-17T07:08:12.256279+0000","last_peered":"2024-07-17T07:08:12.256279+0000","last_clean":"2024-07-17T07:08:12.256279+0000","last_became_active":"2024-07-17T07:08:10.764108+0000","last_became_peered":"2024-07-17T07:08:10.764108+0000","last_unstale":"2024-07-17T07:08:12.256279+0000","last_undegraded":"2024-07-17T07:08:12.256279+0000","last_fullsized":"2024-07-17T07:08:12.256279+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:39.293210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753510+0000","last_change":"2024-07-17T07:08:10.754997+0000","last_active":"2024-07-17T07:08:11.753510+0000","last_peered":"2024-07-17T07:08:11.753510+0000","last_clean":"2024-07-17T07:08:11.753510+0000","last_became_active":"2024-07-17T07:08:10.754723+0000","last_became_peered":"2024-07-17T07:08:10.754723+0000","last_unstale":"2024-07-17T07:08:11.753510+0000","last_undegraded":"2024-07-17T07:08:11.753510+0000","last_fullsized":"2024-07-17T07:08:11.753510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:23.297065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261031+0000","last_change":"2024-07-17T07:08:10.756370+0000","last_active":"2024-07-17T07:08:12.261031+0000","last_peered":"2024-07-17T07:08:12.261031+0000","last_clean":"2024-07-17T07:08:12.261031+0000","last_became_active":"2024-07-17T07:08:10.753680+0000","last_became_peered":"2024-07-17T07:08:10.753680+0000","last_unstale":"2024-07-17T07:08:12.261031+0000","last_undegraded":"2024-07-17T07:08:12.261031+0000","last_fullsized":"2024-07-17T07:08:12.261031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:17.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254107+0000","last_change":"2024-07-17T07:08:10.758747+0000","last_active":"2024-07-17T07:08:12.254107+0000","last_peered":"2024-07-17T07:08:12.254107+0000","last_clean":"2024-07-17T07:08:12.254107+0000","last_became_active":"2024-07-17T07:08:10.758437+0000","last_became_peered":"2024-07-17T07:08:10.758437+0000","last_unstale":"2024-07-17T07:08:12.254107+0000","last_undegraded":"2024-07-17T07:08:12.254107+0000","last_fullsized":"2024-07-17T07:08:12.254107+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:00.223747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258726+0000","last_change":"2024-07-17T07:08:10.761568+0000","last_active":"2024-07-17T07:08:12.258726+0000","last_peered":"2024-07-17T07:08:12.258726+0000","last_clean":"2024-07-17T07:08:12.258726+0000","last_became_active":"2024-07-17T07:08:10.761157+0000","last_became_peered":"2024-07-17T07:08:10.761157+0000","last_unstale":"2024-07-17T07:08:12.258726+0000","last_undegraded":"2024-07-17T07:08:12.258726+0000","last_fullsized":"2024-07-17T07:08:12.258726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:08:26.039843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257244+0000","last_change":"2024-07-17T07:08:10.752425+0000","last_active":"2024-07-17T07:08:12.257244+0000","last_peered":"2024-07-17T07:08:12.257244+0000","last_clean":"2024-07-17T07:08:12.257244+0000","last_became_active":"2024-07-17T07:08:10.752082+0000","last_became_peered":"2024-07-17T07:08:10.752082+0000","last_unstale":"2024-07-17T07:08:12.257244+0000","last_undegraded":"2024-07-17T07:08:12.257244+0000","last_fullsized":"2024-07-17T07:08:12.257244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:54:14.323739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.253903+0000","last_change":"2024-07-17T07:08:10.766172+0000","last_active":"2024-07-17T07:08:12.253903+0000","last_peered":"2024-07-17T07:08:12.253903+0000","last_clean":"2024-07-17T07:08:12.253903+0000","last_became_active":"2024-07-17T07:08:10.765884+0000","last_became_peered":"2024-07-17T07:08:10.765884+0000","last_unstale":"2024-07-17T07:08:12.253903+0000","last_undegraded":"2024-07-17T07:08:12.253903+0000","last_fullsized":"2024-07-17T07:08:12.253903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:59.370717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260574+0000","last_change":"2024-07-17T07:08:10.769180+0000","last_active":"2024-07-17T07:08:12.260574+0000","last_peered":"2024-07-17T07:08:12.260574+0000","last_clean":"2024-07-17T07:08:12.260574+0000","last_became_active":"2024-07-17T07:08:10.768735+0000","last_became_peered":"2024-07-17T07:08:10.768735+0000","last_unstale":"2024-07-17T07:08:12.260574+0000","last_undegraded":"2024-07-17T07:08:12.260574+0000","last_fullsized":"2024-07-17T07:08:12.260574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:37.078982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257327+0000","last_change":"2024-07-17T07:08:10.757038+0000","last_active":"2024-07-17T07:08:12.257327+0000","last_peered":"2024-07-17T07:08:12.257327+0000","last_clean":"2024-07-17T07:08:12.257327+0000","last_became_active":"2024-07-17T07:08:10.754395+0000","last_became_peered":"2024-07-17T07:08:10.754395+0000","last_unstale":"2024-07-17T07:08:12.257327+0000","last_undegraded":"2024-07-17T07:08:12.257327+0000","last_fullsized":"2024-07-17T07:08:12.257327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:59.824078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260654+0000","last_change":"2024-07-17T07:08:10.762982+0000","last_active":"2024-07-17T07:08:12.260654+0000","last_peered":"2024-07-17T07:08:12.260654+0000","last_clean":"2024-07-17T07:08:12.260654+0000","last_became_active":"2024-07-17T07:08:10.762718+0000","last_became_peered":"2024-07-17T07:08:10.762718+0000","last_unstale":"2024-07-17T07:08:12.260654+0000","last_undegraded":"2024-07-17T07:08:12.260654+0000","last_fullsized":"2024-07-17T07:08:12.260654+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:54.024791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760640+0000","last_change":"2024-07-17T07:08:10.747477+0000","last_active":"2024-07-17T07:08:11.760640+0000","last_peered":"2024-07-17T07:08:11.760640+0000","last_clean":"2024-07-17T07:08:11.760640+0000","last_became_active":"2024-07-17T07:08:10.747194+0000","last_became_peered":"2024-07-17T07:08:10.747194+0000","last_unstale":"2024-07-17T07:08:11.760640+0000","last_undegraded":"2024-07-17T07:08:11.760640+0000","last_fullsized":"2024-07-17T07:08:11.760640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:00:38.786026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760090+0000","last_change":"2024-07-17T07:08:10.754453+0000","last_active":"2024-07-17T07:08:11.760090+0000","last_peered":"2024-07-17T07:08:11.760090+0000","last_clean":"2024-07-17T07:08:11.760090+0000","last_became_active":"2024-07-17T07:08:10.754150+0000","last_became_peered":"2024-07-17T07:08:10.754150+0000","last_unstale":"2024-07-17T07:08:11.760090+0000","last_undegraded":"2024-07-17T07:08:11.760090+0000","last_fullsized":"2024-07-17T07:08:11.760090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:25.475619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257383+0000","last_change":"2024-07-17T07:08:10.761910+0000","last_active":"2024-07-17T07:08:12.257383+0000","last_peered":"2024-07-17T07:08:12.257383+0000","last_clean":"2024-07-17T07:08:12.257383+0000","last_became_active":"2024-07-17T07:08:10.761639+0000","last_became_peered":"2024-07-17T07:08:10.761639+0000","last_unstale":"2024-07-17T07:08:12.257383+0000","last_undegraded":"2024-07-17T07:08:12.257383+0000","last_fullsized":"2024-07-17T07:08:12.257383+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:16.995593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753164+0000","last_change":"2024-07-17T07:08:10.752177+0000","last_active":"2024-07-17T07:08:11.753164+0000","last_peered":"2024-07-17T07:08:11.753164+0000","last_clean":"2024-07-17T07:08:11.753164+0000","last_became_active":"2024-07-17T07:08:10.751487+0000","last_became_peered":"2024-07-17T07:08:10.751487+0000","last_unstale":"2024-07-17T07:08:11.753164+0000","last_undegraded":"2024-07-17T07:08:11.753164+0000","last_fullsized":"2024-07-17T07:08:11.753164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:00.895482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257699+0000","last_change":"2024-07-17T07:08:10.763826+0000","last_active":"2024-07-17T07:08:12.257699+0000","last_peered":"2024-07-17T07:08:12.257699+0000","last_clean":"2024-07-17T07:08:12.257699+0000","last_became_active":"2024-07-17T07:08:10.763519+0000","last_became_peered":"2024-07-17T07:08:10.763519+0000","last_unstale":"2024-07-17T07:08:12.257699+0000","last_undegraded":"2024-07-17T07:08:12.257699+0000","last_fullsized":"2024-07-17T07:08:12.257699+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:01.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261214+0000","last_change":"2024-07-17T07:08:10.773733+0000","last_active":"2024-07-17T07:08:12.261214+0000","last_peered":"2024-07-17T07:08:12.261214+0000","last_clean":"2024-07-17T07:08:12.261214+0000","last_became_active":"2024-07-17T07:08:10.773448+0000","last_became_peered":"2024-07-17T07:08:10.773448+0000","last_unstale":"2024-07-17T07:08:12.261214+0000","last_undegraded":"2024-07-17T07:08:12.261214+0000","last_fullsized":"2024-07-17T07:08:12.261214+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:56.497703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258185+0000","last_change":"2024-07-17T07:08:10.747938+0000","last_active":"2024-07-17T07:08:12.258185+0000","last_peered":"2024-07-17T07:08:12.258185+0000","last_clean":"2024-07-17T07:08:12.258185+0000","last_became_active":"2024-07-17T07:08:10.747287+0000","last_became_peered":"2024-07-17T07:08:10.747287+0000","last_unstale":"2024-07-17T07:08:12.258185+0000","last_undegraded":"2024-07-17T07:08:12.258185+0000","last_fullsized":"2024-07-17T07:08:12.258185+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:56.497703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255935+0000","last_change":"2024-07-17T07:08:10.766063+0000","last_active":"2024-07-17T07:08:12.255935+0000","last_peered":"2024-07-17T07:08:12.255935+0000","last_clean":"2024-07-17T07:08:12.255935+0000","last_became_active":"2024-07-17T07:08:10.765203+0000","last_became_peered":"2024-07-17T07:08:10.765203+0000","last_unstale":"2024-07-17T07:08:12.255935+0000","last_undegraded":"2024-07-17T07:08:12.255935+0000","last_fullsized":"2024-07-17T07:08:12.255935+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:01.977362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759360+0000","last_change":"2024-07-17T07:08:10.734972+0000","last_active":"2024-07-17T07:08:11.759360+0000","last_peered":"2024-07-17T07:08:11.759360+0000","last_clean":"2024-07-17T07:08:11.759360+0000","last_became_active":"2024-07-17T07:08:10.734306+0000","last_became_peered":"2024-07-17T07:08:10.734306+0000","last_unstale":"2024-07-17T07:08:11.759360+0000","last_undegraded":"2024-07-17T07:08:11.759360+0000","last_fullsized":"2024-07-17T07:08:11.759360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:54:51.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257897+0000","last_change":"2024-07-17T07:08:10.769631+0000","last_active":"2024-07-17T07:08:12.257897+0000","last_peered":"2024-07-17T07:08:12.257897+0000","last_clean":"2024-07-17T07:08:12.257897+0000","last_became_active":"2024-07-17T07:08:10.761864+0000","last_became_peered":"2024-07-17T07:08:10.761864+0000","last_unstale":"2024-07-17T07:08:12.257897+0000","last_undegraded":"2024-07-17T07:08:12.257897+0000","last_fullsized":"2024-07-17T07:08:12.257897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:35.175390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257128+0000","last_change":"2024-07-17T07:08:10.770111+0000","last_active":"2024-07-17T07:08:12.257128+0000","last_peered":"2024-07-17T07:08:12.257128+0000","last_clean":"2024-07-17T07:08:12.257128+0000","last_became_active":"2024-07-17T07:08:10.769184+0000","last_became_peered":"2024-07-17T07:08:10.769184+0000","last_unstale":"2024-07-17T07:08:12.257128+0000","last_undegraded":"2024-07-17T07:08:12.257128+0000","last_fullsized":"2024-07-17T07:08:12.257128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:26.796734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260055+0000","last_change":"2024-07-17T07:08:10.759763+0000","last_active":"2024-07-17T07:08:12.260055+0000","last_peered":"2024-07-17T07:08:12.260055+0000","last_clean":"2024-07-17T07:08:12.260055+0000","last_became_active":"2024-07-17T07:08:10.759089+0000","last_became_peered":"2024-07-17T07:08:10.759089+0000","last_unstale":"2024-07-17T07:08:12.260055+0000","last_undegraded":"2024-07-17T07:08:12.260055+0000","last_fullsized":"2024-07-17T07:08:12.260055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:24.336579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255952+0000","last_change":"2024-07-17T07:08:10.764492+0000","last_active":"2024-07-17T07:08:12.255952+0000","last_peered":"2024-07-17T07:08:12.255952+0000","last_clean":"2024-07-17T07:08:12.255952+0000","last_became_active":"2024-07-17T07:08:10.764199+0000","last_became_peered":"2024-07-17T07:08:10.764199+0000","last_unstale":"2024-07-17T07:08:12.255952+0000","last_undegraded":"2024-07-17T07:08:12.255952+0000","last_fullsized":"2024-07-17T07:08:12.255952+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:40.732954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256065+0000","last_change":"2024-07-17T07:08:10.753617+0000","last_active":"2024-07-17T07:08:12.256065+0000","last_peered":"2024-07-17T07:08:12.256065+0000","last_clean":"2024-07-17T07:08:12.256065+0000","last_became_active":"2024-07-17T07:08:10.751488+0000","last_became_peered":"2024-07-17T07:08:10.751488+0000","last_unstale":"2024-07-17T07:08:12.256065+0000","last_undegraded":"2024-07-17T07:08:12.256065+0000","last_fullsized":"2024-07-17T07:08:12.256065+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:45.787269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759527+0000","last_change":"2024-07-17T07:08:09.748112+0000","last_active":"2024-07-17T07:08:11.759527+0000","last_peered":"2024-07-17T07:08:11.759527+0000","last_clean":"2024-07-17T07:08:11.759527+0000","last_became_active":"2024-07-17T07:08:09.747826+0000","last_became_peered":"2024-07-17T07:08:09.747826+0000","last_unstale":"2024-07-17T07:08:11.759527+0000","last_undegraded":"2024-07-17T07:08:11.759527+0000","last_fullsized":"2024-07-17T07:08:11.759527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:30:31.156916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256967+0000","last_change":"2024-07-17T07:08:10.755973+0000","last_active":"2024-07-17T07:08:12.256967+0000","last_peered":"2024-07-17T07:08:12.256967+0000","last_clean":"2024-07-17T07:08:12.256967+0000","last_became_active":"2024-07-17T07:08:10.755665+0000","last_became_peered":"2024-07-17T07:08:10.755665+0000","last_unstale":"2024-07-17T07:08:12.256967+0000","last_undegraded":"2024-07-17T07:08:12.256967+0000","last_fullsized":"2024-07-17T07:08:12.256967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:44.248719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256771+0000","last_change":"2024-07-17T07:08:09.725232+0000","last_active":"2024-07-17T07:08:12.256771+0000","last_peered":"2024-07-17T07:08:12.256771+0000","last_clean":"2024-07-17T07:08:12.256771+0000","last_became_active":"2024-07-17T07:08:09.724883+0000","last_became_peered":"2024-07-17T07:08:09.724883+0000","last_unstale":"2024-07-17T07:08:12.256771+0000","last_undegraded":"2024-07-17T07:08:12.256771+0000","last_fullsized":"2024-07-17T07:08:12.256771+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:55.802501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760287+0000","last_change":"2024-07-17T07:08:10.754781+0000","last_active":"2024-07-17T07:08:11.760287+0000","last_peered":"2024-07-17T07:08:11.760287+0000","last_clean":"2024-07-17T07:08:11.760287+0000","last_became_active":"2024-07-17T07:08:10.753044+0000","last_became_peered":"2024-07-17T07:08:10.753044+0000","last_unstale":"2024-07-17T07:08:11.760287+0000","last_undegraded":"2024-07-17T07:08:11.760287+0000","last_fullsized":"2024-07-17T07:08:11.760287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:33.809004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255996+0000","last_change":"2024-07-17T07:08:09.745870+0000","last_active":"2024-07-17T07:08:12.255996+0000","last_peered":"2024-07-17T07:08:12.255996+0000","last_clean":"2024-07-17T07:08:12.255996+0000","last_became_active":"2024-07-17T07:08:09.745501+0000","last_became_peered":"2024-07-17T07:08:09.745501+0000","last_unstale":"2024-07-17T07:08:12.255996+0000","last_undegraded":"2024-07-17T07:08:12.255996+0000","last_fullsized":"2024-07-17T07:08:12.255996+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:19.774520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258974+0000","last_change":"2024-07-17T07:08:10.748013+0000","last_active":"2024-07-17T07:08:12.258974+0000","last_peered":"2024-07-17T07:08:12.258974+0000","last_clean":"2024-07-17T07:08:12.258974+0000","last_became_active":"2024-07-17T07:08:10.747536+0000","last_became_peered":"2024-07-17T07:08:10.747536+0000","last_unstale":"2024-07-17T07:08:12.258974+0000","last_undegraded":"2024-07-17T07:08:12.258974+0000","last_fullsized":"2024-07-17T07:08:12.258974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:28.064514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256481+0000","last_change":"2024-07-17T07:08:09.742894+0000","last_active":"2024-07-17T07:08:12.256481+0000","last_peered":"2024-07-17T07:08:12.256481+0000","last_clean":"2024-07-17T07:08:12.256481+0000","last_became_active":"2024-07-17T07:08:09.742451+0000","last_became_peered":"2024-07-17T07:08:09.742451+0000","last_unstale":"2024-07-17T07:08:12.256481+0000","last_undegraded":"2024-07-17T07:08:12.256481+0000","last_fullsized":"2024-07-17T07:08:12.256481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:05.368434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759147+0000","last_change":"2024-07-17T07:08:10.775319+0000","last_active":"2024-07-17T07:08:11.759147+0000","last_peered":"2024-07-17T07:08:11.759147+0000","last_clean":"2024-07-17T07:08:11.759147+0000","last_became_active":"2024-07-17T07:08:10.774879+0000","last_became_peered":"2024-07-17T07:08:10.774879+0000","last_unstale":"2024-07-17T07:08:11.759147+0000","last_undegraded":"2024-07-17T07:08:11.759147+0000","last_fullsized":"2024-07-17T07:08:11.759147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:42.053666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258189+0000","last_change":"2024-07-17T07:08:09.747776+0000","last_active":"2024-07-17T07:08:12.258189+0000","last_peered":"2024-07-17T07:08:12.258189+0000","last_clean":"2024-07-17T07:08:12.258189+0000","last_became_active":"2024-07-17T07:08:09.747467+0000","last_became_peered":"2024-07-17T07:08:09.747467+0000","last_unstale":"2024-07-17T07:08:12.258189+0000","last_undegraded":"2024-07-17T07:08:12.258189+0000","last_fullsized":"2024-07-17T07:08:12.258189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:36.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257938+0000","last_change":"2024-07-17T07:08:10.757533+0000","last_active":"2024-07-17T07:08:12.257938+0000","last_peered":"2024-07-17T07:08:12.257938+0000","last_clean":"2024-07-17T07:08:12.257938+0000","last_became_active":"2024-07-17T07:08:10.757147+0000","last_became_peered":"2024-07-17T07:08:10.757147+0000","last_unstale":"2024-07-17T07:08:12.257938+0000","last_undegraded":"2024-07-17T07:08:12.257938+0000","last_fullsized":"2024-07-17T07:08:12.257938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:38.039706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760129+0000","last_change":"2024-07-17T07:08:09.743703+0000","last_active":"2024-07-17T07:08:11.760129+0000","last_peered":"2024-07-17T07:08:11.760129+0000","last_clean":"2024-07-17T07:08:11.760129+0000","last_became_active":"2024-07-17T07:08:09.743417+0000","last_became_peered":"2024-07-17T07:08:09.743417+0000","last_unstale":"2024-07-17T07:08:11.760129+0000","last_undegraded":"2024-07-17T07:08:11.760129+0000","last_fullsized":"2024-07-17T07:08:11.760129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:54.306958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259048+0000","last_change":"2024-07-17T07:08:10.748750+0000","last_active":"2024-07-17T07:08:12.259048+0000","last_peered":"2024-07-17T07:08:12.259048+0000","last_clean":"2024-07-17T07:08:12.259048+0000","last_became_active":"2024-07-17T07:08:10.747669+0000","last_became_peered":"2024-07-17T07:08:10.747669+0000","last_unstale":"2024-07-17T07:08:12.259048+0000","last_undegraded":"2024-07-17T07:08:12.259048+0000","last_fullsized":"2024-07-17T07:08:12.259048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:30.205187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759277+0000","last_change":"2024-07-17T07:08:09.741251+0000","last_active":"2024-07-17T07:08:11.759277+0000","last_peered":"2024-07-17T07:08:11.759277+0000","last_clean":"2024-07-17T07:08:11.759277+0000","last_became_active":"2024-07-17T07:08:09.739627+0000","last_became_peered":"2024-07-17T07:08:09.739627+0000","last_unstale":"2024-07-17T07:08:11.759277+0000","last_undegraded":"2024-07-17T07:08:11.759277+0000","last_fullsized":"2024-07-17T07:08:11.759277+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:15:09.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258237+0000","last_change":"2024-07-17T07:08:10.775266+0000","last_active":"2024-07-17T07:08:12.258237+0000","last_peered":"2024-07-17T07:08:12.258237+0000","last_clean":"2024-07-17T07:08:12.258237+0000","last_became_active":"2024-07-17T07:08:10.774915+0000","last_became_peered":"2024-07-17T07:08:10.774915+0000","last_unstale":"2024-07-17T07:08:12.258237+0000","last_undegraded":"2024-07-17T07:08:12.258237+0000","last_fullsized":"2024-07-17T07:08:12.258237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:47.362334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257978+0000","last_change":"2024-07-17T07:08:09.748292+0000","last_active":"2024-07-17T07:08:12.257978+0000","last_peered":"2024-07-17T07:08:12.257978+0000","last_clean":"2024-07-17T07:08:12.257978+0000","last_became_active":"2024-07-17T07:08:09.747941+0000","last_became_peered":"2024-07-17T07:08:09.747941+0000","last_unstale":"2024-07-17T07:08:12.257978+0000","last_undegraded":"2024-07-17T07:08:12.257978+0000","last_fullsized":"2024-07-17T07:08:12.257978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:48.591292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758140+0000","last_change":"2024-07-17T07:08:10.749130+0000","last_active":"2024-07-17T07:08:11.758140+0000","last_peered":"2024-07-17T07:08:11.758140+0000","last_clean":"2024-07-17T07:08:11.758140+0000","last_became_active":"2024-07-17T07:08:10.748767+0000","last_became_peered":"2024-07-17T07:08:10.748767+0000","last_unstale":"2024-07-17T07:08:11.758140+0000","last_undegraded":"2024-07-17T07:08:11.758140+0000","last_fullsized":"2024-07-17T07:08:11.758140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:10.021984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258399+0000","last_change":"2024-07-17T07:08:09.747136+0000","last_active":"2024-07-17T07:08:12.258399+0000","last_peered":"2024-07-17T07:08:12.258399+0000","last_clean":"2024-07-17T07:08:12.258399+0000","last_became_active":"2024-07-17T07:08:09.746646+0000","last_became_peered":"2024-07-17T07:08:09.746646+0000","last_unstale":"2024-07-17T07:08:12.258399+0000","last_undegraded":"2024-07-17T07:08:12.258399+0000","last_fullsized":"2024-07-17T07:08:12.258399+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:33.988749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255854+0000","last_change":"2024-07-17T07:08:10.746503+0000","last_active":"2024-07-17T07:08:12.255854+0000","last_peered":"2024-07-17T07:08:12.255854+0000","last_clean":"2024-07-17T07:08:12.255854+0000","last_became_active":"2024-07-17T07:08:10.746150+0000","last_became_peered":"2024-07-17T07:08:10.746150+0000","last_unstale":"2024-07-17T07:08:12.255854+0000","last_undegraded":"2024-07-17T07:08:12.255854+0000","last_fullsized":"2024-07-17T07:08:12.255854+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:17.623143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759947+0000","last_change":"2024-07-17T07:08:09.753015+0000","last_active":"2024-07-17T07:08:11.759947+0000","last_peered":"2024-07-17T07:08:11.759947+0000","last_clean":"2024-07-17T07:08:11.759947+0000","last_became_active":"2024-07-17T07:08:09.752780+0000","last_became_peered":"2024-07-17T07:08:09.752780+0000","last_unstale":"2024-07-17T07:08:11.759947+0000","last_undegraded":"2024-07-17T07:08:11.759947+0000","last_fullsized":"2024-07-17T07:08:11.759947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:53.967547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259126+0000","last_change":"2024-07-17T07:08:10.761901+0000","last_active":"2024-07-17T07:08:12.259126+0000","last_peered":"2024-07-17T07:08:12.259126+0000","last_clean":"2024-07-17T07:08:12.259126+0000","last_became_active":"2024-07-17T07:08:10.761579+0000","last_became_peered":"2024-07-17T07:08:10.761579+0000","last_unstale":"2024-07-17T07:08:12.259126+0000","last_undegraded":"2024-07-17T07:08:12.259126+0000","last_fullsized":"2024-07-17T07:08:12.259126+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:40.732954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258219+0000","last_change":"2024-07-17T07:08:09.744248+0000","last_active":"2024-07-17T07:08:12.258219+0000","last_peered":"2024-07-17T07:08:12.258219+0000","last_clean":"2024-07-17T07:08:12.258219+0000","last_became_active":"2024-07-17T07:08:09.743820+0000","last_became_peered":"2024-07-17T07:08:09.743820+0000","last_unstale":"2024-07-17T07:08:12.258219+0000","last_undegraded":"2024-07-17T07:08:12.258219+0000","last_fullsized":"2024-07-17T07:08:12.258219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:27:23.752927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759541+0000","last_change":"2024-07-17T07:08:10.735876+0000","last_active":"2024-07-17T07:08:11.759541+0000","last_peered":"2024-07-17T07:08:11.759541+0000","last_clean":"2024-07-17T07:08:11.759541+0000","last_became_active":"2024-07-17T07:08:10.734802+0000","last_became_peered":"2024-07-17T07:08:10.734802+0000","last_unstale":"2024-07-17T07:08:11.759541+0000","last_undegraded":"2024-07-17T07:08:11.759541+0000","last_fullsized":"2024-07-17T07:08:11.759541+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:03:40.789206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261313+0000","last_change":"2024-07-17T07:08:09.740905+0000","last_active":"2024-07-17T07:08:12.261313+0000","last_peered":"2024-07-17T07:08:12.261313+0000","last_clean":"2024-07-17T07:08:12.261313+0000","last_became_active":"2024-07-17T07:08:09.740411+0000","last_became_peered":"2024-07-17T07:08:09.740411+0000","last_unstale":"2024-07-17T07:08:12.261313+0000","last_undegraded":"2024-07-17T07:08:12.261313+0000","last_fullsized":"2024-07-17T07:08:12.261313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:18.279032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255726+0000","last_change":"2024-07-17T07:08:10.755784+0000","last_active":"2024-07-17T07:08:12.255726+0000","last_peered":"2024-07-17T07:08:12.255726+0000","last_clean":"2024-07-17T07:08:12.255726+0000","last_became_active":"2024-07-17T07:08:10.755486+0000","last_became_peered":"2024-07-17T07:08:10.755486+0000","last_unstale":"2024-07-17T07:08:12.255726+0000","last_undegraded":"2024-07-17T07:08:12.255726+0000","last_fullsized":"2024-07-17T07:08:12.255726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:08:26.039843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258463+0000","last_change":"2024-07-17T07:08:09.747203+0000","last_active":"2024-07-17T07:08:12.258463+0000","last_peered":"2024-07-17T07:08:12.258463+0000","last_clean":"2024-07-17T07:08:12.258463+0000","last_became_active":"2024-07-17T07:08:09.746845+0000","last_became_peered":"2024-07-17T07:08:09.746845+0000","last_unstale":"2024-07-17T07:08:12.258463+0000","last_undegraded":"2024-07-17T07:08:12.258463+0000","last_fullsized":"2024-07-17T07:08:12.258463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:44.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261053+0000","last_change":"2024-07-17T07:08:10.760491+0000","last_active":"2024-07-17T07:08:12.261053+0000","last_peered":"2024-07-17T07:08:12.261053+0000","last_clean":"2024-07-17T07:08:12.261053+0000","last_became_active":"2024-07-17T07:08:10.760117+0000","last_became_peered":"2024-07-17T07:08:10.760117+0000","last_unstale":"2024-07-17T07:08:12.261053+0000","last_undegraded":"2024-07-17T07:08:12.261053+0000","last_fullsized":"2024-07-17T07:08:12.261053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:16.995593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760109+0000","last_change":"2024-07-17T07:08:09.741397+0000","last_active":"2024-07-17T07:08:11.760109+0000","last_peered":"2024-07-17T07:08:11.760109+0000","last_clean":"2024-07-17T07:08:11.760109+0000","last_became_active":"2024-07-17T07:08:09.740048+0000","last_became_peered":"2024-07-17T07:08:09.740048+0000","last_unstale":"2024-07-17T07:08:11.760109+0000","last_undegraded":"2024-07-17T07:08:11.760109+0000","last_fullsized":"2024-07-17T07:08:11.760109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:58.353292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258644+0000","last_change":"2024-07-17T07:08:10.759461+0000","last_active":"2024-07-17T07:08:12.258644+0000","last_peered":"2024-07-17T07:08:12.258644+0000","last_clean":"2024-07-17T07:08:12.258644+0000","last_became_active":"2024-07-17T07:08:10.759076+0000","last_became_peered":"2024-07-17T07:08:10.759076+0000","last_unstale":"2024-07-17T07:08:12.258644+0000","last_undegraded":"2024-07-17T07:08:12.258644+0000","last_fullsized":"2024-07-17T07:08:12.258644+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:03:25.310816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257456+0000","last_change":"2024-07-17T07:08:09.744113+0000","last_active":"2024-07-17T07:08:12.257456+0000","last_peered":"2024-07-17T07:08:12.257456+0000","last_clean":"2024-07-17T07:08:12.257456+0000","last_became_active":"2024-07-17T07:08:09.743765+0000","last_became_peered":"2024-07-17T07:08:09.743765+0000","last_unstale":"2024-07-17T07:08:12.257456+0000","last_undegraded":"2024-07-17T07:08:12.257456+0000","last_fullsized":"2024-07-17T07:08:12.257456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:44.661813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260106+0000","last_change":"2024-07-17T07:08:10.771627+0000","last_active":"2024-07-17T07:08:12.260106+0000","last_peered":"2024-07-17T07:08:12.260106+0000","last_clean":"2024-07-17T07:08:12.260106+0000","last_became_active":"2024-07-17T07:08:10.768762+0000","last_became_peered":"2024-07-17T07:08:10.768762+0000","last_unstale":"2024-07-17T07:08:12.260106+0000","last_undegraded":"2024-07-17T07:08:12.260106+0000","last_fullsized":"2024-07-17T07:08:12.260106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:44.248719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259442+0000","last_change":"2024-07-17T07:08:09.741836+0000","last_active":"2024-07-17T07:08:12.259442+0000","last_peered":"2024-07-17T07:08:12.259442+0000","last_clean":"2024-07-17T07:08:12.259442+0000","last_became_active":"2024-07-17T07:08:09.741507+0000","last_became_peered":"2024-07-17T07:08:09.741507+0000","last_unstale":"2024-07-17T07:08:12.259442+0000","last_undegraded":"2024-07-17T07:08:12.259442+0000","last_fullsized":"2024-07-17T07:08:12.259442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:40.306847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257213+0000","last_change":"2024-07-17T07:08:10.754073+0000","last_active":"2024-07-17T07:08:12.257213+0000","last_peered":"2024-07-17T07:08:12.257213+0000","last_clean":"2024-07-17T07:08:12.257213+0000","last_became_active":"2024-07-17T07:08:10.753806+0000","last_became_peered":"2024-07-17T07:08:10.753806+0000","last_unstale":"2024-07-17T07:08:12.257213+0000","last_undegraded":"2024-07-17T07:08:12.257213+0000","last_fullsized":"2024-07-17T07:08:12.257213+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:04.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261144+0000","last_change":"2024-07-17T07:08:09.741699+0000","last_active":"2024-07-17T07:08:12.261144+0000","last_peered":"2024-07-17T07:08:12.261144+0000","last_clean":"2024-07-17T07:08:12.261144+0000","last_became_active":"2024-07-17T07:08:09.740992+0000","last_became_peered":"2024-07-17T07:08:09.740992+0000","last_unstale":"2024-07-17T07:08:12.261144+0000","last_undegraded":"2024-07-17T07:08:12.261144+0000","last_fullsized":"2024-07-17T07:08:12.261144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:27:29.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.253977+0000","last_change":"2024-07-17T07:08:10.765540+0000","last_active":"2024-07-17T07:08:12.253977+0000","last_peered":"2024-07-17T07:08:12.253977+0000","last_clean":"2024-07-17T07:08:12.253977+0000","last_became_active":"2024-07-17T07:08:10.765233+0000","last_became_peered":"2024-07-17T07:08:10.765233+0000","last_unstale":"2024-07-17T07:08:12.253977+0000","last_undegraded":"2024-07-17T07:08:12.253977+0000","last_fullsized":"2024-07-17T07:08:12.253977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:40.967425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.754072+0000","last_change":"2024-07-17T07:08:09.747524+0000","last_active":"2024-07-17T07:08:11.754072+0000","last_peered":"2024-07-17T07:08:11.754072+0000","last_clean":"2024-07-17T07:08:11.754072+0000","last_became_active":"2024-07-17T07:08:09.747250+0000","last_became_peered":"2024-07-17T07:08:09.747250+0000","last_unstale":"2024-07-17T07:08:11.754072+0000","last_undegraded":"2024-07-17T07:08:11.754072+0000","last_fullsized":"2024-07-17T07:08:11.754072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:34.201482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760308+0000","last_change":"2024-07-17T07:08:10.764798+0000","last_active":"2024-07-17T07:08:11.760308+0000","last_peered":"2024-07-17T07:08:11.760308+0000","last_clean":"2024-07-17T07:08:11.760308+0000","last_became_active":"2024-07-17T07:08:10.763561+0000","last_became_peered":"2024-07-17T07:08:10.763561+0000","last_unstale":"2024-07-17T07:08:11.760308+0000","last_undegraded":"2024-07-17T07:08:11.760308+0000","last_fullsized":"2024-07-17T07:08:11.760308+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:32.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759048+0000","last_change":"2024-07-17T07:08:09.743721+0000","last_active":"2024-07-17T07:08:11.759048+0000","last_peered":"2024-07-17T07:08:11.759048+0000","last_clean":"2024-07-17T07:08:11.759048+0000","last_became_active":"2024-07-17T07:08:09.743441+0000","last_became_peered":"2024-07-17T07:08:09.743441+0000","last_unstale":"2024-07-17T07:08:11.759048+0000","last_undegraded":"2024-07-17T07:08:11.759048+0000","last_fullsized":"2024-07-17T07:08:11.759048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:26.771335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255125+0000","last_change":"2024-07-17T07:08:10.756956+0000","last_active":"2024-07-17T07:08:12.255125+0000","last_peered":"2024-07-17T07:08:12.255125+0000","last_clean":"2024-07-17T07:08:12.255125+0000","last_became_active":"2024-07-17T07:08:10.756561+0000","last_became_peered":"2024-07-17T07:08:10.756561+0000","last_unstale":"2024-07-17T07:08:12.255125+0000","last_undegraded":"2024-07-17T07:08:12.255125+0000","last_fullsized":"2024-07-17T07:08:12.255125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:38.308202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759597+0000","last_change":"2024-07-17T07:08:09.733776+0000","last_active":"2024-07-17T07:08:11.759597+0000","last_peered":"2024-07-17T07:08:11.759597+0000","last_clean":"2024-07-17T07:08:11.759597+0000","last_became_active":"2024-07-17T07:08:09.732904+0000","last_became_peered":"2024-07-17T07:08:09.732904+0000","last_unstale":"2024-07-17T07:08:11.759597+0000","last_undegraded":"2024-07-17T07:08:11.759597+0000","last_fullsized":"2024-07-17T07:08:11.759597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:43:04.739401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256171+0000","last_change":"2024-07-17T07:08:10.748038+0000","last_active":"2024-07-17T07:08:12.256171+0000","last_peered":"2024-07-17T07:08:12.256171+0000","last_clean":"2024-07-17T07:08:12.256171+0000","last_became_active":"2024-07-17T07:08:10.746165+0000","last_became_peered":"2024-07-17T07:08:10.746165+0000","last_unstale":"2024-07-17T07:08:12.256171+0000","last_undegraded":"2024-07-17T07:08:12.256171+0000","last_fullsized":"2024-07-17T07:08:12.256171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:24.336579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255055+0000","last_change":"2024-07-17T07:08:09.744357+0000","last_active":"2024-07-17T07:08:12.255055+0000","last_peered":"2024-07-17T07:08:12.255055+0000","last_clean":"2024-07-17T07:08:12.255055+0000","last_became_active":"2024-07-17T07:08:09.744078+0000","last_became_peered":"2024-07-17T07:08:09.744078+0000","last_unstale":"2024-07-17T07:08:12.255055+0000","last_undegraded":"2024-07-17T07:08:12.255055+0000","last_fullsized":"2024-07-17T07:08:12.255055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:53:04.976791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758738+0000","last_change":"2024-07-17T07:08:10.750628+0000","last_active":"2024-07-17T07:08:11.758738+0000","last_peered":"2024-07-17T07:08:11.758738+0000","last_clean":"2024-07-17T07:08:11.758738+0000","last_became_active":"2024-07-17T07:08:10.747925+0000","last_became_peered":"2024-07-17T07:08:10.747925+0000","last_unstale":"2024-07-17T07:08:11.758738+0000","last_undegraded":"2024-07-17T07:08:11.758738+0000","last_fullsized":"2024-07-17T07:08:11.758738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:39.293210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257680+0000","last_change":"2024-07-17T07:08:09.738575+0000","last_active":"2024-07-17T07:08:12.257680+0000","last_peered":"2024-07-17T07:08:12.257680+0000","last_clean":"2024-07-17T07:08:12.257680+0000","last_became_active":"2024-07-17T07:08:09.738292+0000","last_became_peered":"2024-07-17T07:08:09.738292+0000","last_unstale":"2024-07-17T07:08:12.257680+0000","last_undegraded":"2024-07-17T07:08:12.257680+0000","last_fullsized":"2024-07-17T07:08:12.257680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:20.496733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259191+0000","last_change":"2024-07-17T07:08:10.763480+0000","last_active":"2024-07-17T07:08:12.259191+0000","last_peered":"2024-07-17T07:08:12.259191+0000","last_clean":"2024-07-17T07:08:12.259191+0000","last_became_active":"2024-07-17T07:08:10.763193+0000","last_became_peered":"2024-07-17T07:08:10.763193+0000","last_unstale":"2024-07-17T07:08:12.259191+0000","last_undegraded":"2024-07-17T07:08:12.259191+0000","last_fullsized":"2024-07-17T07:08:12.259191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:05.303830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260432+0000","last_change":"2024-07-17T07:08:09.743966+0000","last_active":"2024-07-17T07:08:12.260432+0000","last_peered":"2024-07-17T07:08:12.260432+0000","last_clean":"2024-07-17T07:08:12.260432+0000","last_became_active":"2024-07-17T07:08:09.743675+0000","last_became_peered":"2024-07-17T07:08:09.743675+0000","last_unstale":"2024-07-17T07:08:12.260432+0000","last_undegraded":"2024-07-17T07:08:12.260432+0000","last_fullsized":"2024-07-17T07:08:12.260432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.605297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256507+0000","last_change":"2024-07-17T07:08:10.776033+0000","last_active":"2024-07-17T07:08:12.256507+0000","last_peered":"2024-07-17T07:08:12.256507+0000","last_clean":"2024-07-17T07:08:12.256507+0000","last_became_active":"2024-07-17T07:08:10.773056+0000","last_became_peered":"2024-07-17T07:08:10.773056+0000","last_unstale":"2024-07-17T07:08:12.256507+0000","last_undegraded":"2024-07-17T07:08:12.256507+0000","last_fullsized":"2024-07-17T07:08:12.256507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:56.778454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753013+0000","last_change":"2024-07-17T07:08:09.746576+0000","last_active":"2024-07-17T07:08:11.753013+0000","last_peered":"2024-07-17T07:08:11.753013+0000","last_clean":"2024-07-17T07:08:11.753013+0000","last_became_active":"2024-07-17T07:08:09.746008+0000","last_became_peered":"2024-07-17T07:08:09.746008+0000","last_unstale":"2024-07-17T07:08:11.753013+0000","last_undegraded":"2024-07-17T07:08:11.753013+0000","last_fullsized":"2024-07-17T07:08:11.753013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:59.696724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256382+0000","last_change":"2024-07-17T07:08:10.755144+0000","last_active":"2024-07-17T07:08:12.256382+0000","last_peered":"2024-07-17T07:08:12.256382+0000","last_clean":"2024-07-17T07:08:12.256382+0000","last_became_active":"2024-07-17T07:08:10.753484+0000","last_became_peered":"2024-07-17T07:08:10.753484+0000","last_unstale":"2024-07-17T07:08:12.256382+0000","last_undegraded":"2024-07-17T07:08:12.256382+0000","last_fullsized":"2024-07-17T07:08:12.256382+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:33.521693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759857+0000","last_change":"2024-07-17T07:08:09.738826+0000","last_active":"2024-07-17T07:08:11.759857+0000","last_peered":"2024-07-17T07:08:11.759857+0000","last_clean":"2024-07-17T07:08:11.759857+0000","last_became_active":"2024-07-17T07:08:09.737129+0000","last_became_peered":"2024-07-17T07:08:09.737129+0000","last_unstale":"2024-07-17T07:08:11.759857+0000","last_undegraded":"2024-07-17T07:08:11.759857+0000","last_fullsized":"2024-07-17T07:08:11.759857+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:01.832881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254733+0000","last_change":"2024-07-17T07:08:10.761514+0000","last_active":"2024-07-17T07:08:12.254733+0000","last_peered":"2024-07-17T07:08:12.254733+0000","last_clean":"2024-07-17T07:08:12.254733+0000","last_became_active":"2024-07-17T07:08:10.761248+0000","last_became_peered":"2024-07-17T07:08:10.761248+0000","last_unstale":"2024-07-17T07:08:12.254733+0000","last_undegraded":"2024-07-17T07:08:12.254733+0000","last_fullsized":"2024-07-17T07:08:12.254733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:58.645982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258490+0000","last_change":"2024-07-17T07:08:09.747032+0000","last_active":"2024-07-17T07:08:12.258490+0000","last_peered":"2024-07-17T07:08:12.258490+0000","last_clean":"2024-07-17T07:08:12.258490+0000","last_became_active":"2024-07-17T07:08:09.746759+0000","last_became_peered":"2024-07-17T07:08:09.746759+0000","last_unstale":"2024-07-17T07:08:12.258490+0000","last_undegraded":"2024-07-17T07:08:12.258490+0000","last_fullsized":"2024-07-17T07:08:12.258490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:50.063060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259385+0000","last_change":"2024-07-17T07:08:10.742687+0000","last_active":"2024-07-17T07:08:12.259385+0000","last_peered":"2024-07-17T07:08:12.259385+0000","last_clean":"2024-07-17T07:08:12.259385+0000","last_became_active":"2024-07-17T07:08:10.742230+0000","last_became_peered":"2024-07-17T07:08:10.742230+0000","last_unstale":"2024-07-17T07:08:12.259385+0000","last_undegraded":"2024-07-17T07:08:12.259385+0000","last_fullsized":"2024-07-17T07:08:12.259385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:02:06.644866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260663+0000","last_change":"2024-07-17T07:08:09.748408+0000","last_active":"2024-07-17T07:08:12.260663+0000","last_peered":"2024-07-17T07:08:12.260663+0000","last_clean":"2024-07-17T07:08:12.260663+0000","last_became_active":"2024-07-17T07:08:09.748028+0000","last_became_peered":"2024-07-17T07:08:09.748028+0000","last_unstale":"2024-07-17T07:08:12.260663+0000","last_undegraded":"2024-07-17T07:08:12.260663+0000","last_fullsized":"2024-07-17T07:08:12.260663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:01.504944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260369+0000","last_change":"2024-07-17T07:08:10.764199+0000","last_active":"2024-07-17T07:08:12.260369+0000","last_peered":"2024-07-17T07:08:12.260369+0000","last_clean":"2024-07-17T07:08:12.260369+0000","last_became_active":"2024-07-17T07:08:10.762314+0000","last_became_peered":"2024-07-17T07:08:10.762314+0000","last_unstale":"2024-07-17T07:08:12.260369+0000","last_undegraded":"2024-07-17T07:08:12.260369+0000","last_fullsized":"2024-07-17T07:08:12.260369+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:59.666091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259670+0000","last_change":"2024-07-17T07:08:09.739221+0000","last_active":"2024-07-17T07:08:12.259670+0000","last_peered":"2024-07-17T07:08:12.259670+0000","last_clean":"2024-07-17T07:08:12.259670+0000","last_became_active":"2024-07-17T07:08:09.738952+0000","last_became_peered":"2024-07-17T07:08:09.738952+0000","last_unstale":"2024-07-17T07:08:12.259670+0000","last_undegraded":"2024-07-17T07:08:12.259670+0000","last_fullsized":"2024-07-17T07:08:12.259670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:17.033750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255944+0000","last_change":"2024-07-17T07:08:10.755979+0000","last_active":"2024-07-17T07:08:12.255944+0000","last_peered":"2024-07-17T07:08:12.255944+0000","last_clean":"2024-07-17T07:08:12.255944+0000","last_became_active":"2024-07-17T07:08:10.755540+0000","last_became_peered":"2024-07-17T07:08:10.755540+0000","last_unstale":"2024-07-17T07:08:12.255944+0000","last_undegraded":"2024-07-17T07:08:12.255944+0000","last_fullsized":"2024-07-17T07:08:12.255944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:05.303830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257946+0000","last_change":"2024-07-17T07:08:09.732886+0000","last_active":"2024-07-17T07:08:12.257946+0000","last_peered":"2024-07-17T07:08:12.257946+0000","last_clean":"2024-07-17T07:08:12.257946+0000","last_became_active":"2024-07-17T07:08:09.732574+0000","last_became_peered":"2024-07-17T07:08:09.732574+0000","last_unstale":"2024-07-17T07:08:12.257946+0000","last_undegraded":"2024-07-17T07:08:12.257946+0000","last_fullsized":"2024-07-17T07:08:12.257946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:10.147524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258800+0000","last_change":"2024-07-17T07:08:10.772034+0000","last_active":"2024-07-17T07:08:12.258800+0000","last_peered":"2024-07-17T07:08:12.258800+0000","last_clean":"2024-07-17T07:08:12.258800+0000","last_became_active":"2024-07-17T07:08:10.771746+0000","last_became_peered":"2024-07-17T07:08:10.771746+0000","last_unstale":"2024-07-17T07:08:12.258800+0000","last_undegraded":"2024-07-17T07:08:12.258800+0000","last_fullsized":"2024-07-17T07:08:12.258800+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:17.623143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758259+0000","last_change":"2024-07-17T07:08:09.747721+0000","last_active":"2024-07-17T07:08:11.758259+0000","last_peered":"2024-07-17T07:08:11.758259+0000","last_clean":"2024-07-17T07:08:11.758259+0000","last_became_active":"2024-07-17T07:08:09.747406+0000","last_became_peered":"2024-07-17T07:08:09.747406+0000","last_unstale":"2024-07-17T07:08:11.758259+0000","last_undegraded":"2024-07-17T07:08:11.758259+0000","last_fullsized":"2024-07-17T07:08:11.758259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:09.538020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259175+0000","last_change":"2024-07-17T07:08:10.752355+0000","last_active":"2024-07-17T07:08:12.259175+0000","last_peered":"2024-07-17T07:08:12.259175+0000","last_clean":"2024-07-17T07:08:12.259175+0000","last_became_active":"2024-07-17T07:08:10.751114+0000","last_became_peered":"2024-07-17T07:08:10.751114+0000","last_unstale":"2024-07-17T07:08:12.259175+0000","last_undegraded":"2024-07-17T07:08:12.259175+0000","last_fullsized":"2024-07-17T07:08:12.259175+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:58.231064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256391+0000","last_change":"2024-07-17T07:08:09.741701+0000","last_active":"2024-07-17T07:08:12.256391+0000","last_peered":"2024-07-17T07:08:12.256391+0000","last_clean":"2024-07-17T07:08:12.256391+0000","last_became_active":"2024-07-17T07:08:09.739373+0000","last_became_peered":"2024-07-17T07:08:09.739373+0000","last_unstale":"2024-07-17T07:08:12.256391+0000","last_undegraded":"2024-07-17T07:08:12.256391+0000","last_fullsized":"2024-07-17T07:08:12.256391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:35:25.743533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758738+0000","last_change":"2024-07-17T07:08:10.746705+0000","last_active":"2024-07-17T07:08:11.758738+0000","last_peered":"2024-07-17T07:08:11.758738+0000","last_clean":"2024-07-17T07:08:11.758738+0000","last_became_active":"2024-07-17T07:08:10.746417+0000","last_became_peered":"2024-07-17T07:08:10.746417+0000","last_unstale":"2024-07-17T07:08:11.758738+0000","last_undegraded":"2024-07-17T07:08:11.758738+0000","last_fullsized":"2024-07-17T07:08:11.758738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:56.778454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259480+0000","last_change":"2024-07-17T07:08:09.749287+0000","last_active":"2024-07-17T07:08:12.259480+0000","last_peered":"2024-07-17T07:08:12.259480+0000","last_clean":"2024-07-17T07:08:12.259480+0000","last_became_active":"2024-07-17T07:08:09.743978+0000","last_became_peered":"2024-07-17T07:08:09.743978+0000","last_unstale":"2024-07-17T07:08:12.259480+0000","last_undegraded":"2024-07-17T07:08:12.259480+0000","last_fullsized":"2024-07-17T07:08:12.259480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:11.147593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"57'86","reported_seq":251,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255626+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:08:12.255626+0000","last_peered":"2024-07-17T07:08:12.255626+0000","last_clean":"2024-07-17T07:08:12.255626+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:08:12.255626+0000","last_undegraded":"2024-07-17T07:08:12.255626+0000","last_fullsized":"2024-07-17T07:08:12.255626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54748,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317092,"statfs":{"total":96636764160,"available":96580702208,"internally_reserved":0,"allocated":1384448,"data_stored":981327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":790528,"data_stored":391487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":794624,"data_stored":398019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":798720,"data_stored":398601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509603,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":798720,"data_stored":392469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542307,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58336,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313504,"statfs":{"total":96636764160,"available":96577028096,"internally_reserved":0,"allocated":806912,"data_stored":399631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509604,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":398179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":981259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:08:15.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:08:15.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 -- 172.21.15.44:0/900421858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06eec0 msgr2=0x7f464c0ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 --2- 172.21.15.44:0/900421858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06eec0 0x7f464c0ac760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4644005fd0 tx=0x7f464402a4f0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 -- 172.21.15.44:0/900421858 shutdown_connections 2024-07-17T07:08:15.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 --2- 172.21.15.44:0/900421858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06eec0 0x7f464c0ac760 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 --2- 172.21.15.44:0/900421858 >> v2:172.21.15.175:3300/0 conn(0x7f464c06ea50 0x7f464c0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 -- 172.21.15.44:0/900421858 >> 172.21.15.44:0/900421858 conn(0x7f464c0b0fb0 msgr2=0x7f464c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:15.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 -- 172.21.15.44:0/900421858 shutdown_connections 2024-07-17T07:08:15.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.493+0000 7f4654200640 1 -- 172.21.15.44:0/900421858 wait complete. 2024-07-17T07:08:15.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.495+0000 7f4654200640 1 Processor -- start 2024-07-17T07:08:15.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.495+0000 7f4654200640 1 -- start start 2024-07-17T07:08:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464c06eec0 0x7f464c1f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464c0ae810 0x7f464c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f464c0b2750 con 0x7f464c06ea50 2024-07-17T07:08:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f464c0b25d0 con 0x7f464c0ae810 2024-07-17T07:08:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4654200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f464c0b28d0 con 0x7f464c06eec0 2024-07-17T07:08:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.496+0000 7f4651800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51106/0 (socket says 172.21.15.44:51106) 2024-07-17T07:08:15.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 -- 172.21.15.44:0/1222964539 learned_addr learned my addr 172.21.15.44:0/1222964539 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4652200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464c0ae810 0x7f464c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4650e00640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464c06eec0 0x7f464c1f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 -- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464c06eec0 msgr2=0x7f464c1f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464c06eec0 0x7f464c1f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 -- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464c0ae810 msgr2=0x7f464c1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464c0ae810 0x7f464c1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 -- 172.21.15.44:0/1222964539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c144980 con 0x7f464c06ea50 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.497+0000 7f4651800640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f464000b420 tx=0x7f464000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.498+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640011830 con 0x7f464c06ea50 2024-07-17T07:08:15.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.498+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4640016070 con 0x7f464c06ea50 2024-07-17T07:08:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.498+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640011d60 con 0x7f464c06ea50 2024-07-17T07:08:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.498+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c144c00 con 0x7f464c06ea50 2024-07-17T07:08:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.498+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1451c0 con 0x7f464c06ea50 2024-07-17T07:08:15.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.499+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c0b35e0 con 0x7f464c06ea50 2024-07-17T07:08:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.500+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f464001b460 con 0x7f464c06ea50 2024-07-17T07:08:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.502+0000 7f4649600640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 0x7f4630079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.502+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f4640098380 con 0x7f464c06ea50 2024-07-17T07:08:15.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.502+0000 7f4650e00640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 0x7f4630079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:15.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.503+0000 7f4650e00640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 0x7f4630079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f464c160e20 tx=0x7f463c005bd0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:15.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.505+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4640020020 con 0x7f464c06ea50 2024-07-17T07:08:15.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.715+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 --> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f464c0b23b0 con 0x7f4630076cd0 2024-07-17T07:08:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.728+0000 7f4649600640 1 -- 172.21.15.44:0/1222964539 <== mgr.5071 v2:172.21.15.175:6800/533048536 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f464c0b23b0 con 0x7f4630076cd0 2024-07-17T07:08:15.730 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:08:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 msgr2=0x7f4630079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 0x7f4630079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f464c160e20 tx=0x7f463c005bd0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 msgr2=0x7f464c098a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f464000b420 tx=0x7f464000b8e0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 shutdown_connections 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f4630076cd0 0x7f4630079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464c0ae810 0x7f464c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464c06eec0 0x7f464c1f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 --2- 172.21.15.44:0/1222964539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f464c06ea50 0x7f464c098a70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 >> 172.21.15.44:0/1222964539 conn(0x7f464c0b0fb0 msgr2=0x7f464c160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 shutdown_connections 2024-07-17T07:08:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:15.733+0000 7f4654200640 1 -- 172.21.15.44:0/1222964539 wait complete. 2024-07-17T07:08:15.753 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:08:14.462761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":170,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313732,"kb_used_data":7384,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754660988,"statfs":{"total":773094113280,"available":772772851712,"internally_reserved":0,"allocated":7561216,"data_stored":4340972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":313577008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.768645"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255817+0000","last_change":"2024-07-17T07:08:10.760706+0000","last_active":"2024-07-17T07:08:12.255817+0000","last_peered":"2024-07-17T07:08:12.255817+0000","last_clean":"2024-07-17T07:08:12.255817+0000","last_became_active":"2024-07-17T07:08:10.760376+0000","last_became_peered":"2024-07-17T07:08:10.760376+0000","last_unstale":"2024-07-17T07:08:12.255817+0000","last_undegraded":"2024-07-17T07:08:12.255817+0000","last_fullsized":"2024-07-17T07:08:12.255817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:59.006584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257331+0000","last_change":"2024-07-17T07:08:10.756315+0000","last_active":"2024-07-17T07:08:12.257331+0000","last_peered":"2024-07-17T07:08:12.257331+0000","last_clean":"2024-07-17T07:08:12.257331+0000","last_became_active":"2024-07-17T07:08:10.755396+0000","last_became_peered":"2024-07-17T07:08:10.755396+0000","last_unstale":"2024-07-17T07:08:12.257331+0000","last_undegraded":"2024-07-17T07:08:12.257331+0000","last_fullsized":"2024-07-17T07:08:12.257331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:59.545790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260209+0000","last_change":"2024-07-17T07:08:10.763526+0000","last_active":"2024-07-17T07:08:12.260209+0000","last_peered":"2024-07-17T07:08:12.260209+0000","last_clean":"2024-07-17T07:08:12.260209+0000","last_became_active":"2024-07-17T07:08:10.763263+0000","last_became_peered":"2024-07-17T07:08:10.763263+0000","last_unstale":"2024-07-17T07:08:12.260209+0000","last_undegraded":"2024-07-17T07:08:12.260209+0000","last_fullsized":"2024-07-17T07:08:12.260209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:33.521693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258125+0000","last_change":"2024-07-17T07:08:10.754687+0000","last_active":"2024-07-17T07:08:12.258125+0000","last_peered":"2024-07-17T07:08:12.258125+0000","last_clean":"2024-07-17T07:08:12.258125+0000","last_became_active":"2024-07-17T07:08:10.754315+0000","last_became_peered":"2024-07-17T07:08:10.754315+0000","last_unstale":"2024-07-17T07:08:12.258125+0000","last_undegraded":"2024-07-17T07:08:12.258125+0000","last_fullsized":"2024-07-17T07:08:12.258125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:14.190805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758577+0000","last_change":"2024-07-17T07:08:10.753494+0000","last_active":"2024-07-17T07:08:11.758577+0000","last_peered":"2024-07-17T07:08:11.758577+0000","last_clean":"2024-07-17T07:08:11.758577+0000","last_became_active":"2024-07-17T07:08:10.753118+0000","last_became_peered":"2024-07-17T07:08:10.753118+0000","last_unstale":"2024-07-17T07:08:11.758577+0000","last_undegraded":"2024-07-17T07:08:11.758577+0000","last_fullsized":"2024-07-17T07:08:11.758577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:24.759613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254666+0000","last_change":"2024-07-17T07:08:10.760944+0000","last_active":"2024-07-17T07:08:12.254666+0000","last_peered":"2024-07-17T07:08:12.254666+0000","last_clean":"2024-07-17T07:08:12.254666+0000","last_became_active":"2024-07-17T07:08:10.758562+0000","last_became_peered":"2024-07-17T07:08:10.758562+0000","last_unstale":"2024-07-17T07:08:12.254666+0000","last_undegraded":"2024-07-17T07:08:12.254666+0000","last_fullsized":"2024-07-17T07:08:12.254666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:07.650347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260989+0000","last_change":"2024-07-17T07:08:10.760012+0000","last_active":"2024-07-17T07:08:12.260989+0000","last_peered":"2024-07-17T07:08:12.260989+0000","last_clean":"2024-07-17T07:08:12.260989+0000","last_became_active":"2024-07-17T07:08:10.759412+0000","last_became_peered":"2024-07-17T07:08:10.759412+0000","last_unstale":"2024-07-17T07:08:12.260989+0000","last_undegraded":"2024-07-17T07:08:12.260989+0000","last_fullsized":"2024-07-17T07:08:12.260989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:04.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256780+0000","last_change":"2024-07-17T07:08:10.767772+0000","last_active":"2024-07-17T07:08:12.256780+0000","last_peered":"2024-07-17T07:08:12.256780+0000","last_clean":"2024-07-17T07:08:12.256780+0000","last_became_active":"2024-07-17T07:08:10.767391+0000","last_became_peered":"2024-07-17T07:08:10.767391+0000","last_unstale":"2024-07-17T07:08:12.256780+0000","last_undegraded":"2024-07-17T07:08:12.256780+0000","last_fullsized":"2024-07-17T07:08:12.256780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:54.024791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258758+0000","last_change":"2024-07-17T07:08:10.775251+0000","last_active":"2024-07-17T07:08:12.258758+0000","last_peered":"2024-07-17T07:08:12.258758+0000","last_clean":"2024-07-17T07:08:12.258758+0000","last_became_active":"2024-07-17T07:08:10.773551+0000","last_became_peered":"2024-07-17T07:08:10.773551+0000","last_unstale":"2024-07-17T07:08:12.258758+0000","last_undegraded":"2024-07-17T07:08:12.258758+0000","last_fullsized":"2024-07-17T07:08:12.258758+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:16:21.029101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256279+0000","last_change":"2024-07-17T07:08:10.764393+0000","last_active":"2024-07-17T07:08:12.256279+0000","last_peered":"2024-07-17T07:08:12.256279+0000","last_clean":"2024-07-17T07:08:12.256279+0000","last_became_active":"2024-07-17T07:08:10.764108+0000","last_became_peered":"2024-07-17T07:08:10.764108+0000","last_unstale":"2024-07-17T07:08:12.256279+0000","last_undegraded":"2024-07-17T07:08:12.256279+0000","last_fullsized":"2024-07-17T07:08:12.256279+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:39.293210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753510+0000","last_change":"2024-07-17T07:08:10.754997+0000","last_active":"2024-07-17T07:08:11.753510+0000","last_peered":"2024-07-17T07:08:11.753510+0000","last_clean":"2024-07-17T07:08:11.753510+0000","last_became_active":"2024-07-17T07:08:10.754723+0000","last_became_peered":"2024-07-17T07:08:10.754723+0000","last_unstale":"2024-07-17T07:08:11.753510+0000","last_undegraded":"2024-07-17T07:08:11.753510+0000","last_fullsized":"2024-07-17T07:08:11.753510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:23.297065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261031+0000","last_change":"2024-07-17T07:08:10.756370+0000","last_active":"2024-07-17T07:08:12.261031+0000","last_peered":"2024-07-17T07:08:12.261031+0000","last_clean":"2024-07-17T07:08:12.261031+0000","last_became_active":"2024-07-17T07:08:10.753680+0000","last_became_peered":"2024-07-17T07:08:10.753680+0000","last_unstale":"2024-07-17T07:08:12.261031+0000","last_undegraded":"2024-07-17T07:08:12.261031+0000","last_fullsized":"2024-07-17T07:08:12.261031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:17.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254107+0000","last_change":"2024-07-17T07:08:10.758747+0000","last_active":"2024-07-17T07:08:12.254107+0000","last_peered":"2024-07-17T07:08:12.254107+0000","last_clean":"2024-07-17T07:08:12.254107+0000","last_became_active":"2024-07-17T07:08:10.758437+0000","last_became_peered":"2024-07-17T07:08:10.758437+0000","last_unstale":"2024-07-17T07:08:12.254107+0000","last_undegraded":"2024-07-17T07:08:12.254107+0000","last_fullsized":"2024-07-17T07:08:12.254107+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:00.223747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258726+0000","last_change":"2024-07-17T07:08:10.761568+0000","last_active":"2024-07-17T07:08:12.258726+0000","last_peered":"2024-07-17T07:08:12.258726+0000","last_clean":"2024-07-17T07:08:12.258726+0000","last_became_active":"2024-07-17T07:08:10.761157+0000","last_became_peered":"2024-07-17T07:08:10.761157+0000","last_unstale":"2024-07-17T07:08:12.258726+0000","last_undegraded":"2024-07-17T07:08:12.258726+0000","last_fullsized":"2024-07-17T07:08:12.258726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:08:26.039843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257244+0000","last_change":"2024-07-17T07:08:10.752425+0000","last_active":"2024-07-17T07:08:12.257244+0000","last_peered":"2024-07-17T07:08:12.257244+0000","last_clean":"2024-07-17T07:08:12.257244+0000","last_became_active":"2024-07-17T07:08:10.752082+0000","last_became_peered":"2024-07-17T07:08:10.752082+0000","last_unstale":"2024-07-17T07:08:12.257244+0000","last_undegraded":"2024-07-17T07:08:12.257244+0000","last_fullsized":"2024-07-17T07:08:12.257244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:54:14.323739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.253903+0000","last_change":"2024-07-17T07:08:10.766172+0000","last_active":"2024-07-17T07:08:12.253903+0000","last_peered":"2024-07-17T07:08:12.253903+0000","last_clean":"2024-07-17T07:08:12.253903+0000","last_became_active":"2024-07-17T07:08:10.765884+0000","last_became_peered":"2024-07-17T07:08:10.765884+0000","last_unstale":"2024-07-17T07:08:12.253903+0000","last_undegraded":"2024-07-17T07:08:12.253903+0000","last_fullsized":"2024-07-17T07:08:12.253903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:59.370717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260574+0000","last_change":"2024-07-17T07:08:10.769180+0000","last_active":"2024-07-17T07:08:12.260574+0000","last_peered":"2024-07-17T07:08:12.260574+0000","last_clean":"2024-07-17T07:08:12.260574+0000","last_became_active":"2024-07-17T07:08:10.768735+0000","last_became_peered":"2024-07-17T07:08:10.768735+0000","last_unstale":"2024-07-17T07:08:12.260574+0000","last_undegraded":"2024-07-17T07:08:12.260574+0000","last_fullsized":"2024-07-17T07:08:12.260574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:37.078982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257327+0000","last_change":"2024-07-17T07:08:10.757038+0000","last_active":"2024-07-17T07:08:12.257327+0000","last_peered":"2024-07-17T07:08:12.257327+0000","last_clean":"2024-07-17T07:08:12.257327+0000","last_became_active":"2024-07-17T07:08:10.754395+0000","last_became_peered":"2024-07-17T07:08:10.754395+0000","last_unstale":"2024-07-17T07:08:12.257327+0000","last_undegraded":"2024-07-17T07:08:12.257327+0000","last_fullsized":"2024-07-17T07:08:12.257327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:59.824078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260654+0000","last_change":"2024-07-17T07:08:10.762982+0000","last_active":"2024-07-17T07:08:12.260654+0000","last_peered":"2024-07-17T07:08:12.260654+0000","last_clean":"2024-07-17T07:08:12.260654+0000","last_became_active":"2024-07-17T07:08:10.762718+0000","last_became_peered":"2024-07-17T07:08:10.762718+0000","last_unstale":"2024-07-17T07:08:12.260654+0000","last_undegraded":"2024-07-17T07:08:12.260654+0000","last_fullsized":"2024-07-17T07:08:12.260654+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:54.024791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760640+0000","last_change":"2024-07-17T07:08:10.747477+0000","last_active":"2024-07-17T07:08:11.760640+0000","last_peered":"2024-07-17T07:08:11.760640+0000","last_clean":"2024-07-17T07:08:11.760640+0000","last_became_active":"2024-07-17T07:08:10.747194+0000","last_became_peered":"2024-07-17T07:08:10.747194+0000","last_unstale":"2024-07-17T07:08:11.760640+0000","last_undegraded":"2024-07-17T07:08:11.760640+0000","last_fullsized":"2024-07-17T07:08:11.760640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:00:38.786026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760090+0000","last_change":"2024-07-17T07:08:10.754453+0000","last_active":"2024-07-17T07:08:11.760090+0000","last_peered":"2024-07-17T07:08:11.760090+0000","last_clean":"2024-07-17T07:08:11.760090+0000","last_became_active":"2024-07-17T07:08:10.754150+0000","last_became_peered":"2024-07-17T07:08:10.754150+0000","last_unstale":"2024-07-17T07:08:11.760090+0000","last_undegraded":"2024-07-17T07:08:11.760090+0000","last_fullsized":"2024-07-17T07:08:11.760090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:25.475619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257383+0000","last_change":"2024-07-17T07:08:10.761910+0000","last_active":"2024-07-17T07:08:12.257383+0000","last_peered":"2024-07-17T07:08:12.257383+0000","last_clean":"2024-07-17T07:08:12.257383+0000","last_became_active":"2024-07-17T07:08:10.761639+0000","last_became_peered":"2024-07-17T07:08:10.761639+0000","last_unstale":"2024-07-17T07:08:12.257383+0000","last_undegraded":"2024-07-17T07:08:12.257383+0000","last_fullsized":"2024-07-17T07:08:12.257383+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:16.995593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753164+0000","last_change":"2024-07-17T07:08:10.752177+0000","last_active":"2024-07-17T07:08:11.753164+0000","last_peered":"2024-07-17T07:08:11.753164+0000","last_clean":"2024-07-17T07:08:11.753164+0000","last_became_active":"2024-07-17T07:08:10.751487+0000","last_became_peered":"2024-07-17T07:08:10.751487+0000","last_unstale":"2024-07-17T07:08:11.753164+0000","last_undegraded":"2024-07-17T07:08:11.753164+0000","last_fullsized":"2024-07-17T07:08:11.753164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:00.895482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257699+0000","last_change":"2024-07-17T07:08:10.763826+0000","last_active":"2024-07-17T07:08:12.257699+0000","last_peered":"2024-07-17T07:08:12.257699+0000","last_clean":"2024-07-17T07:08:12.257699+0000","last_became_active":"2024-07-17T07:08:10.763519+0000","last_became_peered":"2024-07-17T07:08:10.763519+0000","last_unstale":"2024-07-17T07:08:12.257699+0000","last_undegraded":"2024-07-17T07:08:12.257699+0000","last_fullsized":"2024-07-17T07:08:12.257699+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:01.441368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261214+0000","last_change":"2024-07-17T07:08:10.773733+0000","last_active":"2024-07-17T07:08:12.261214+0000","last_peered":"2024-07-17T07:08:12.261214+0000","last_clean":"2024-07-17T07:08:12.261214+0000","last_became_active":"2024-07-17T07:08:10.773448+0000","last_became_peered":"2024-07-17T07:08:10.773448+0000","last_unstale":"2024-07-17T07:08:12.261214+0000","last_undegraded":"2024-07-17T07:08:12.261214+0000","last_fullsized":"2024-07-17T07:08:12.261214+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:56.497703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258185+0000","last_change":"2024-07-17T07:08:10.747938+0000","last_active":"2024-07-17T07:08:12.258185+0000","last_peered":"2024-07-17T07:08:12.258185+0000","last_clean":"2024-07-17T07:08:12.258185+0000","last_became_active":"2024-07-17T07:08:10.747287+0000","last_became_peered":"2024-07-17T07:08:10.747287+0000","last_unstale":"2024-07-17T07:08:12.258185+0000","last_undegraded":"2024-07-17T07:08:12.258185+0000","last_fullsized":"2024-07-17T07:08:12.258185+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:56.497703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255935+0000","last_change":"2024-07-17T07:08:10.766063+0000","last_active":"2024-07-17T07:08:12.255935+0000","last_peered":"2024-07-17T07:08:12.255935+0000","last_clean":"2024-07-17T07:08:12.255935+0000","last_became_active":"2024-07-17T07:08:10.765203+0000","last_became_peered":"2024-07-17T07:08:10.765203+0000","last_unstale":"2024-07-17T07:08:12.255935+0000","last_undegraded":"2024-07-17T07:08:12.255935+0000","last_fullsized":"2024-07-17T07:08:12.255935+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:01.977362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759360+0000","last_change":"2024-07-17T07:08:10.734972+0000","last_active":"2024-07-17T07:08:11.759360+0000","last_peered":"2024-07-17T07:08:11.759360+0000","last_clean":"2024-07-17T07:08:11.759360+0000","last_became_active":"2024-07-17T07:08:10.734306+0000","last_became_peered":"2024-07-17T07:08:10.734306+0000","last_unstale":"2024-07-17T07:08:11.759360+0000","last_undegraded":"2024-07-17T07:08:11.759360+0000","last_fullsized":"2024-07-17T07:08:11.759360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:54:51.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257897+0000","last_change":"2024-07-17T07:08:10.769631+0000","last_active":"2024-07-17T07:08:12.257897+0000","last_peered":"2024-07-17T07:08:12.257897+0000","last_clean":"2024-07-17T07:08:12.257897+0000","last_became_active":"2024-07-17T07:08:10.761864+0000","last_became_peered":"2024-07-17T07:08:10.761864+0000","last_unstale":"2024-07-17T07:08:12.257897+0000","last_undegraded":"2024-07-17T07:08:12.257897+0000","last_fullsized":"2024-07-17T07:08:12.257897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:35.175390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257128+0000","last_change":"2024-07-17T07:08:10.770111+0000","last_active":"2024-07-17T07:08:12.257128+0000","last_peered":"2024-07-17T07:08:12.257128+0000","last_clean":"2024-07-17T07:08:12.257128+0000","last_became_active":"2024-07-17T07:08:10.769184+0000","last_became_peered":"2024-07-17T07:08:10.769184+0000","last_unstale":"2024-07-17T07:08:12.257128+0000","last_undegraded":"2024-07-17T07:08:12.257128+0000","last_fullsized":"2024-07-17T07:08:12.257128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:26.796734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260055+0000","last_change":"2024-07-17T07:08:10.759763+0000","last_active":"2024-07-17T07:08:12.260055+0000","last_peered":"2024-07-17T07:08:12.260055+0000","last_clean":"2024-07-17T07:08:12.260055+0000","last_became_active":"2024-07-17T07:08:10.759089+0000","last_became_peered":"2024-07-17T07:08:10.759089+0000","last_unstale":"2024-07-17T07:08:12.260055+0000","last_undegraded":"2024-07-17T07:08:12.260055+0000","last_fullsized":"2024-07-17T07:08:12.260055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:24.336579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255952+0000","last_change":"2024-07-17T07:08:10.764492+0000","last_active":"2024-07-17T07:08:12.255952+0000","last_peered":"2024-07-17T07:08:12.255952+0000","last_clean":"2024-07-17T07:08:12.255952+0000","last_became_active":"2024-07-17T07:08:10.764199+0000","last_became_peered":"2024-07-17T07:08:10.764199+0000","last_unstale":"2024-07-17T07:08:12.255952+0000","last_undegraded":"2024-07-17T07:08:12.255952+0000","last_fullsized":"2024-07-17T07:08:12.255952+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:40.732954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256065+0000","last_change":"2024-07-17T07:08:10.753617+0000","last_active":"2024-07-17T07:08:12.256065+0000","last_peered":"2024-07-17T07:08:12.256065+0000","last_clean":"2024-07-17T07:08:12.256065+0000","last_became_active":"2024-07-17T07:08:10.751488+0000","last_became_peered":"2024-07-17T07:08:10.751488+0000","last_unstale":"2024-07-17T07:08:12.256065+0000","last_undegraded":"2024-07-17T07:08:12.256065+0000","last_fullsized":"2024-07-17T07:08:12.256065+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:45.787269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759527+0000","last_change":"2024-07-17T07:08:09.748112+0000","last_active":"2024-07-17T07:08:11.759527+0000","last_peered":"2024-07-17T07:08:11.759527+0000","last_clean":"2024-07-17T07:08:11.759527+0000","last_became_active":"2024-07-17T07:08:09.747826+0000","last_became_peered":"2024-07-17T07:08:09.747826+0000","last_unstale":"2024-07-17T07:08:11.759527+0000","last_undegraded":"2024-07-17T07:08:11.759527+0000","last_fullsized":"2024-07-17T07:08:11.759527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:30:31.156916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256967+0000","last_change":"2024-07-17T07:08:10.755973+0000","last_active":"2024-07-17T07:08:12.256967+0000","last_peered":"2024-07-17T07:08:12.256967+0000","last_clean":"2024-07-17T07:08:12.256967+0000","last_became_active":"2024-07-17T07:08:10.755665+0000","last_became_peered":"2024-07-17T07:08:10.755665+0000","last_unstale":"2024-07-17T07:08:12.256967+0000","last_undegraded":"2024-07-17T07:08:12.256967+0000","last_fullsized":"2024-07-17T07:08:12.256967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:44.248719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256771+0000","last_change":"2024-07-17T07:08:09.725232+0000","last_active":"2024-07-17T07:08:12.256771+0000","last_peered":"2024-07-17T07:08:12.256771+0000","last_clean":"2024-07-17T07:08:12.256771+0000","last_became_active":"2024-07-17T07:08:09.724883+0000","last_became_peered":"2024-07-17T07:08:09.724883+0000","last_unstale":"2024-07-17T07:08:12.256771+0000","last_undegraded":"2024-07-17T07:08:12.256771+0000","last_fullsized":"2024-07-17T07:08:12.256771+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:55.802501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760287+0000","last_change":"2024-07-17T07:08:10.754781+0000","last_active":"2024-07-17T07:08:11.760287+0000","last_peered":"2024-07-17T07:08:11.760287+0000","last_clean":"2024-07-17T07:08:11.760287+0000","last_became_active":"2024-07-17T07:08:10.753044+0000","last_became_peered":"2024-07-17T07:08:10.753044+0000","last_unstale":"2024-07-17T07:08:11.760287+0000","last_undegraded":"2024-07-17T07:08:11.760287+0000","last_fullsized":"2024-07-17T07:08:11.760287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:33.809004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255996+0000","last_change":"2024-07-17T07:08:09.745870+0000","last_active":"2024-07-17T07:08:12.255996+0000","last_peered":"2024-07-17T07:08:12.255996+0000","last_clean":"2024-07-17T07:08:12.255996+0000","last_became_active":"2024-07-17T07:08:09.745501+0000","last_became_peered":"2024-07-17T07:08:09.745501+0000","last_unstale":"2024-07-17T07:08:12.255996+0000","last_undegraded":"2024-07-17T07:08:12.255996+0000","last_fullsized":"2024-07-17T07:08:12.255996+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:19.774520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258974+0000","last_change":"2024-07-17T07:08:10.748013+0000","last_active":"2024-07-17T07:08:12.258974+0000","last_peered":"2024-07-17T07:08:12.258974+0000","last_clean":"2024-07-17T07:08:12.258974+0000","last_became_active":"2024-07-17T07:08:10.747536+0000","last_became_peered":"2024-07-17T07:08:10.747536+0000","last_unstale":"2024-07-17T07:08:12.258974+0000","last_undegraded":"2024-07-17T07:08:12.258974+0000","last_fullsized":"2024-07-17T07:08:12.258974+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:28.064514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256481+0000","last_change":"2024-07-17T07:08:09.742894+0000","last_active":"2024-07-17T07:08:12.256481+0000","last_peered":"2024-07-17T07:08:12.256481+0000","last_clean":"2024-07-17T07:08:12.256481+0000","last_became_active":"2024-07-17T07:08:09.742451+0000","last_became_peered":"2024-07-17T07:08:09.742451+0000","last_unstale":"2024-07-17T07:08:12.256481+0000","last_undegraded":"2024-07-17T07:08:12.256481+0000","last_fullsized":"2024-07-17T07:08:12.256481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:05.368434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759147+0000","last_change":"2024-07-17T07:08:10.775319+0000","last_active":"2024-07-17T07:08:11.759147+0000","last_peered":"2024-07-17T07:08:11.759147+0000","last_clean":"2024-07-17T07:08:11.759147+0000","last_became_active":"2024-07-17T07:08:10.774879+0000","last_became_peered":"2024-07-17T07:08:10.774879+0000","last_unstale":"2024-07-17T07:08:11.759147+0000","last_undegraded":"2024-07-17T07:08:11.759147+0000","last_fullsized":"2024-07-17T07:08:11.759147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:42.053666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258189+0000","last_change":"2024-07-17T07:08:09.747776+0000","last_active":"2024-07-17T07:08:12.258189+0000","last_peered":"2024-07-17T07:08:12.258189+0000","last_clean":"2024-07-17T07:08:12.258189+0000","last_became_active":"2024-07-17T07:08:09.747467+0000","last_became_peered":"2024-07-17T07:08:09.747467+0000","last_unstale":"2024-07-17T07:08:12.258189+0000","last_undegraded":"2024-07-17T07:08:12.258189+0000","last_fullsized":"2024-07-17T07:08:12.258189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:36.892544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257938+0000","last_change":"2024-07-17T07:08:10.757533+0000","last_active":"2024-07-17T07:08:12.257938+0000","last_peered":"2024-07-17T07:08:12.257938+0000","last_clean":"2024-07-17T07:08:12.257938+0000","last_became_active":"2024-07-17T07:08:10.757147+0000","last_became_peered":"2024-07-17T07:08:10.757147+0000","last_unstale":"2024-07-17T07:08:12.257938+0000","last_undegraded":"2024-07-17T07:08:12.257938+0000","last_fullsized":"2024-07-17T07:08:12.257938+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:38.039706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760129+0000","last_change":"2024-07-17T07:08:09.743703+0000","last_active":"2024-07-17T07:08:11.760129+0000","last_peered":"2024-07-17T07:08:11.760129+0000","last_clean":"2024-07-17T07:08:11.760129+0000","last_became_active":"2024-07-17T07:08:09.743417+0000","last_became_peered":"2024-07-17T07:08:09.743417+0000","last_unstale":"2024-07-17T07:08:11.760129+0000","last_undegraded":"2024-07-17T07:08:11.760129+0000","last_fullsized":"2024-07-17T07:08:11.760129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:54.306958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259048+0000","last_change":"2024-07-17T07:08:10.748750+0000","last_active":"2024-07-17T07:08:12.259048+0000","last_peered":"2024-07-17T07:08:12.259048+0000","last_clean":"2024-07-17T07:08:12.259048+0000","last_became_active":"2024-07-17T07:08:10.747669+0000","last_became_peered":"2024-07-17T07:08:10.747669+0000","last_unstale":"2024-07-17T07:08:12.259048+0000","last_undegraded":"2024-07-17T07:08:12.259048+0000","last_fullsized":"2024-07-17T07:08:12.259048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:30.205187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759277+0000","last_change":"2024-07-17T07:08:09.741251+0000","last_active":"2024-07-17T07:08:11.759277+0000","last_peered":"2024-07-17T07:08:11.759277+0000","last_clean":"2024-07-17T07:08:11.759277+0000","last_became_active":"2024-07-17T07:08:09.739627+0000","last_became_peered":"2024-07-17T07:08:09.739627+0000","last_unstale":"2024-07-17T07:08:11.759277+0000","last_undegraded":"2024-07-17T07:08:11.759277+0000","last_fullsized":"2024-07-17T07:08:11.759277+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:15:09.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258237+0000","last_change":"2024-07-17T07:08:10.775266+0000","last_active":"2024-07-17T07:08:12.258237+0000","last_peered":"2024-07-17T07:08:12.258237+0000","last_clean":"2024-07-17T07:08:12.258237+0000","last_became_active":"2024-07-17T07:08:10.774915+0000","last_became_peered":"2024-07-17T07:08:10.774915+0000","last_unstale":"2024-07-17T07:08:12.258237+0000","last_undegraded":"2024-07-17T07:08:12.258237+0000","last_fullsized":"2024-07-17T07:08:12.258237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:47.362334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257978+0000","last_change":"2024-07-17T07:08:09.748292+0000","last_active":"2024-07-17T07:08:12.257978+0000","last_peered":"2024-07-17T07:08:12.257978+0000","last_clean":"2024-07-17T07:08:12.257978+0000","last_became_active":"2024-07-17T07:08:09.747941+0000","last_became_peered":"2024-07-17T07:08:09.747941+0000","last_unstale":"2024-07-17T07:08:12.257978+0000","last_undegraded":"2024-07-17T07:08:12.257978+0000","last_fullsized":"2024-07-17T07:08:12.257978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:48.591292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758140+0000","last_change":"2024-07-17T07:08:10.749130+0000","last_active":"2024-07-17T07:08:11.758140+0000","last_peered":"2024-07-17T07:08:11.758140+0000","last_clean":"2024-07-17T07:08:11.758140+0000","last_became_active":"2024-07-17T07:08:10.748767+0000","last_became_peered":"2024-07-17T07:08:10.748767+0000","last_unstale":"2024-07-17T07:08:11.758140+0000","last_undegraded":"2024-07-17T07:08:11.758140+0000","last_fullsized":"2024-07-17T07:08:11.758140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:10.021984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258399+0000","last_change":"2024-07-17T07:08:09.747136+0000","last_active":"2024-07-17T07:08:12.258399+0000","last_peered":"2024-07-17T07:08:12.258399+0000","last_clean":"2024-07-17T07:08:12.258399+0000","last_became_active":"2024-07-17T07:08:09.746646+0000","last_became_peered":"2024-07-17T07:08:09.746646+0000","last_unstale":"2024-07-17T07:08:12.258399+0000","last_undegraded":"2024-07-17T07:08:12.258399+0000","last_fullsized":"2024-07-17T07:08:12.258399+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:33.988749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255854+0000","last_change":"2024-07-17T07:08:10.746503+0000","last_active":"2024-07-17T07:08:12.255854+0000","last_peered":"2024-07-17T07:08:12.255854+0000","last_clean":"2024-07-17T07:08:12.255854+0000","last_became_active":"2024-07-17T07:08:10.746150+0000","last_became_peered":"2024-07-17T07:08:10.746150+0000","last_unstale":"2024-07-17T07:08:12.255854+0000","last_undegraded":"2024-07-17T07:08:12.255854+0000","last_fullsized":"2024-07-17T07:08:12.255854+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:17.623143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759947+0000","last_change":"2024-07-17T07:08:09.753015+0000","last_active":"2024-07-17T07:08:11.759947+0000","last_peered":"2024-07-17T07:08:11.759947+0000","last_clean":"2024-07-17T07:08:11.759947+0000","last_became_active":"2024-07-17T07:08:09.752780+0000","last_became_peered":"2024-07-17T07:08:09.752780+0000","last_unstale":"2024-07-17T07:08:11.759947+0000","last_undegraded":"2024-07-17T07:08:11.759947+0000","last_fullsized":"2024-07-17T07:08:11.759947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:53.967547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259126+0000","last_change":"2024-07-17T07:08:10.761901+0000","last_active":"2024-07-17T07:08:12.259126+0000","last_peered":"2024-07-17T07:08:12.259126+0000","last_clean":"2024-07-17T07:08:12.259126+0000","last_became_active":"2024-07-17T07:08:10.761579+0000","last_became_peered":"2024-07-17T07:08:10.761579+0000","last_unstale":"2024-07-17T07:08:12.259126+0000","last_undegraded":"2024-07-17T07:08:12.259126+0000","last_fullsized":"2024-07-17T07:08:12.259126+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:40.732954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258219+0000","last_change":"2024-07-17T07:08:09.744248+0000","last_active":"2024-07-17T07:08:12.258219+0000","last_peered":"2024-07-17T07:08:12.258219+0000","last_clean":"2024-07-17T07:08:12.258219+0000","last_became_active":"2024-07-17T07:08:09.743820+0000","last_became_peered":"2024-07-17T07:08:09.743820+0000","last_unstale":"2024-07-17T07:08:12.258219+0000","last_undegraded":"2024-07-17T07:08:12.258219+0000","last_fullsized":"2024-07-17T07:08:12.258219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:27:23.752927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759541+0000","last_change":"2024-07-17T07:08:10.735876+0000","last_active":"2024-07-17T07:08:11.759541+0000","last_peered":"2024-07-17T07:08:11.759541+0000","last_clean":"2024-07-17T07:08:11.759541+0000","last_became_active":"2024-07-17T07:08:10.734802+0000","last_became_peered":"2024-07-17T07:08:10.734802+0000","last_unstale":"2024-07-17T07:08:11.759541+0000","last_undegraded":"2024-07-17T07:08:11.759541+0000","last_fullsized":"2024-07-17T07:08:11.759541+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:03:40.789206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261313+0000","last_change":"2024-07-17T07:08:09.740905+0000","last_active":"2024-07-17T07:08:12.261313+0000","last_peered":"2024-07-17T07:08:12.261313+0000","last_clean":"2024-07-17T07:08:12.261313+0000","last_became_active":"2024-07-17T07:08:09.740411+0000","last_became_peered":"2024-07-17T07:08:09.740411+0000","last_unstale":"2024-07-17T07:08:12.261313+0000","last_undegraded":"2024-07-17T07:08:12.261313+0000","last_fullsized":"2024-07-17T07:08:12.261313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:18.279032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255726+0000","last_change":"2024-07-17T07:08:10.755784+0000","last_active":"2024-07-17T07:08:12.255726+0000","last_peered":"2024-07-17T07:08:12.255726+0000","last_clean":"2024-07-17T07:08:12.255726+0000","last_became_active":"2024-07-17T07:08:10.755486+0000","last_became_peered":"2024-07-17T07:08:10.755486+0000","last_unstale":"2024-07-17T07:08:12.255726+0000","last_undegraded":"2024-07-17T07:08:12.255726+0000","last_fullsized":"2024-07-17T07:08:12.255726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:08:26.039843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258463+0000","last_change":"2024-07-17T07:08:09.747203+0000","last_active":"2024-07-17T07:08:12.258463+0000","last_peered":"2024-07-17T07:08:12.258463+0000","last_clean":"2024-07-17T07:08:12.258463+0000","last_became_active":"2024-07-17T07:08:09.746845+0000","last_became_peered":"2024-07-17T07:08:09.746845+0000","last_unstale":"2024-07-17T07:08:12.258463+0000","last_undegraded":"2024-07-17T07:08:12.258463+0000","last_fullsized":"2024-07-17T07:08:12.258463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:44.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261053+0000","last_change":"2024-07-17T07:08:10.760491+0000","last_active":"2024-07-17T07:08:12.261053+0000","last_peered":"2024-07-17T07:08:12.261053+0000","last_clean":"2024-07-17T07:08:12.261053+0000","last_became_active":"2024-07-17T07:08:10.760117+0000","last_became_peered":"2024-07-17T07:08:10.760117+0000","last_unstale":"2024-07-17T07:08:12.261053+0000","last_undegraded":"2024-07-17T07:08:12.261053+0000","last_fullsized":"2024-07-17T07:08:12.261053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:16.995593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760109+0000","last_change":"2024-07-17T07:08:09.741397+0000","last_active":"2024-07-17T07:08:11.760109+0000","last_peered":"2024-07-17T07:08:11.760109+0000","last_clean":"2024-07-17T07:08:11.760109+0000","last_became_active":"2024-07-17T07:08:09.740048+0000","last_became_peered":"2024-07-17T07:08:09.740048+0000","last_unstale":"2024-07-17T07:08:11.760109+0000","last_undegraded":"2024-07-17T07:08:11.760109+0000","last_fullsized":"2024-07-17T07:08:11.760109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:58.353292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258644+0000","last_change":"2024-07-17T07:08:10.759461+0000","last_active":"2024-07-17T07:08:12.258644+0000","last_peered":"2024-07-17T07:08:12.258644+0000","last_clean":"2024-07-17T07:08:12.258644+0000","last_became_active":"2024-07-17T07:08:10.759076+0000","last_became_peered":"2024-07-17T07:08:10.759076+0000","last_unstale":"2024-07-17T07:08:12.258644+0000","last_undegraded":"2024-07-17T07:08:12.258644+0000","last_fullsized":"2024-07-17T07:08:12.258644+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:03:25.310816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257456+0000","last_change":"2024-07-17T07:08:09.744113+0000","last_active":"2024-07-17T07:08:12.257456+0000","last_peered":"2024-07-17T07:08:12.257456+0000","last_clean":"2024-07-17T07:08:12.257456+0000","last_became_active":"2024-07-17T07:08:09.743765+0000","last_became_peered":"2024-07-17T07:08:09.743765+0000","last_unstale":"2024-07-17T07:08:12.257456+0000","last_undegraded":"2024-07-17T07:08:12.257456+0000","last_fullsized":"2024-07-17T07:08:12.257456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:44.661813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260106+0000","last_change":"2024-07-17T07:08:10.771627+0000","last_active":"2024-07-17T07:08:12.260106+0000","last_peered":"2024-07-17T07:08:12.260106+0000","last_clean":"2024-07-17T07:08:12.260106+0000","last_became_active":"2024-07-17T07:08:10.768762+0000","last_became_peered":"2024-07-17T07:08:10.768762+0000","last_unstale":"2024-07-17T07:08:12.260106+0000","last_undegraded":"2024-07-17T07:08:12.260106+0000","last_fullsized":"2024-07-17T07:08:12.260106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:30:44.248719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259442+0000","last_change":"2024-07-17T07:08:09.741836+0000","last_active":"2024-07-17T07:08:12.259442+0000","last_peered":"2024-07-17T07:08:12.259442+0000","last_clean":"2024-07-17T07:08:12.259442+0000","last_became_active":"2024-07-17T07:08:09.741507+0000","last_became_peered":"2024-07-17T07:08:09.741507+0000","last_unstale":"2024-07-17T07:08:12.259442+0000","last_undegraded":"2024-07-17T07:08:12.259442+0000","last_fullsized":"2024-07-17T07:08:12.259442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:40.306847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257213+0000","last_change":"2024-07-17T07:08:10.754073+0000","last_active":"2024-07-17T07:08:12.257213+0000","last_peered":"2024-07-17T07:08:12.257213+0000","last_clean":"2024-07-17T07:08:12.257213+0000","last_became_active":"2024-07-17T07:08:10.753806+0000","last_became_peered":"2024-07-17T07:08:10.753806+0000","last_unstale":"2024-07-17T07:08:12.257213+0000","last_undegraded":"2024-07-17T07:08:12.257213+0000","last_fullsized":"2024-07-17T07:08:12.257213+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:04.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.261144+0000","last_change":"2024-07-17T07:08:09.741699+0000","last_active":"2024-07-17T07:08:12.261144+0000","last_peered":"2024-07-17T07:08:12.261144+0000","last_clean":"2024-07-17T07:08:12.261144+0000","last_became_active":"2024-07-17T07:08:09.740992+0000","last_became_peered":"2024-07-17T07:08:09.740992+0000","last_unstale":"2024-07-17T07:08:12.261144+0000","last_undegraded":"2024-07-17T07:08:12.261144+0000","last_fullsized":"2024-07-17T07:08:12.261144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:27:29.177800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.253977+0000","last_change":"2024-07-17T07:08:10.765540+0000","last_active":"2024-07-17T07:08:12.253977+0000","last_peered":"2024-07-17T07:08:12.253977+0000","last_clean":"2024-07-17T07:08:12.253977+0000","last_became_active":"2024-07-17T07:08:10.765233+0000","last_became_peered":"2024-07-17T07:08:10.765233+0000","last_unstale":"2024-07-17T07:08:12.253977+0000","last_undegraded":"2024-07-17T07:08:12.253977+0000","last_fullsized":"2024-07-17T07:08:12.253977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:40.967425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.754072+0000","last_change":"2024-07-17T07:08:09.747524+0000","last_active":"2024-07-17T07:08:11.754072+0000","last_peered":"2024-07-17T07:08:11.754072+0000","last_clean":"2024-07-17T07:08:11.754072+0000","last_became_active":"2024-07-17T07:08:09.747250+0000","last_became_peered":"2024-07-17T07:08:09.747250+0000","last_unstale":"2024-07-17T07:08:11.754072+0000","last_undegraded":"2024-07-17T07:08:11.754072+0000","last_fullsized":"2024-07-17T07:08:11.754072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:34.201482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.760308+0000","last_change":"2024-07-17T07:08:10.764798+0000","last_active":"2024-07-17T07:08:11.760308+0000","last_peered":"2024-07-17T07:08:11.760308+0000","last_clean":"2024-07-17T07:08:11.760308+0000","last_became_active":"2024-07-17T07:08:10.763561+0000","last_became_peered":"2024-07-17T07:08:10.763561+0000","last_unstale":"2024-07-17T07:08:11.760308+0000","last_undegraded":"2024-07-17T07:08:11.760308+0000","last_fullsized":"2024-07-17T07:08:11.760308+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:32.312700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759048+0000","last_change":"2024-07-17T07:08:09.743721+0000","last_active":"2024-07-17T07:08:11.759048+0000","last_peered":"2024-07-17T07:08:11.759048+0000","last_clean":"2024-07-17T07:08:11.759048+0000","last_became_active":"2024-07-17T07:08:09.743441+0000","last_became_peered":"2024-07-17T07:08:09.743441+0000","last_unstale":"2024-07-17T07:08:11.759048+0000","last_undegraded":"2024-07-17T07:08:11.759048+0000","last_fullsized":"2024-07-17T07:08:11.759048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:26.771335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255125+0000","last_change":"2024-07-17T07:08:10.756956+0000","last_active":"2024-07-17T07:08:12.255125+0000","last_peered":"2024-07-17T07:08:12.255125+0000","last_clean":"2024-07-17T07:08:12.255125+0000","last_became_active":"2024-07-17T07:08:10.756561+0000","last_became_peered":"2024-07-17T07:08:10.756561+0000","last_unstale":"2024-07-17T07:08:12.255125+0000","last_undegraded":"2024-07-17T07:08:12.255125+0000","last_fullsized":"2024-07-17T07:08:12.255125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:38.308202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759597+0000","last_change":"2024-07-17T07:08:09.733776+0000","last_active":"2024-07-17T07:08:11.759597+0000","last_peered":"2024-07-17T07:08:11.759597+0000","last_clean":"2024-07-17T07:08:11.759597+0000","last_became_active":"2024-07-17T07:08:09.732904+0000","last_became_peered":"2024-07-17T07:08:09.732904+0000","last_unstale":"2024-07-17T07:08:11.759597+0000","last_undegraded":"2024-07-17T07:08:11.759597+0000","last_fullsized":"2024-07-17T07:08:11.759597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:43:04.739401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256171+0000","last_change":"2024-07-17T07:08:10.748038+0000","last_active":"2024-07-17T07:08:12.256171+0000","last_peered":"2024-07-17T07:08:12.256171+0000","last_clean":"2024-07-17T07:08:12.256171+0000","last_became_active":"2024-07-17T07:08:10.746165+0000","last_became_peered":"2024-07-17T07:08:10.746165+0000","last_unstale":"2024-07-17T07:08:12.256171+0000","last_undegraded":"2024-07-17T07:08:12.256171+0000","last_fullsized":"2024-07-17T07:08:12.256171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:24.336579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255055+0000","last_change":"2024-07-17T07:08:09.744357+0000","last_active":"2024-07-17T07:08:12.255055+0000","last_peered":"2024-07-17T07:08:12.255055+0000","last_clean":"2024-07-17T07:08:12.255055+0000","last_became_active":"2024-07-17T07:08:09.744078+0000","last_became_peered":"2024-07-17T07:08:09.744078+0000","last_unstale":"2024-07-17T07:08:12.255055+0000","last_undegraded":"2024-07-17T07:08:12.255055+0000","last_fullsized":"2024-07-17T07:08:12.255055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:53:04.976791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758738+0000","last_change":"2024-07-17T07:08:10.750628+0000","last_active":"2024-07-17T07:08:11.758738+0000","last_peered":"2024-07-17T07:08:11.758738+0000","last_clean":"2024-07-17T07:08:11.758738+0000","last_became_active":"2024-07-17T07:08:10.747925+0000","last_became_peered":"2024-07-17T07:08:10.747925+0000","last_unstale":"2024-07-17T07:08:11.758738+0000","last_undegraded":"2024-07-17T07:08:11.758738+0000","last_fullsized":"2024-07-17T07:08:11.758738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:39.293210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257680+0000","last_change":"2024-07-17T07:08:09.738575+0000","last_active":"2024-07-17T07:08:12.257680+0000","last_peered":"2024-07-17T07:08:12.257680+0000","last_clean":"2024-07-17T07:08:12.257680+0000","last_became_active":"2024-07-17T07:08:09.738292+0000","last_became_peered":"2024-07-17T07:08:09.738292+0000","last_unstale":"2024-07-17T07:08:12.257680+0000","last_undegraded":"2024-07-17T07:08:12.257680+0000","last_fullsized":"2024-07-17T07:08:12.257680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:20.496733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259191+0000","last_change":"2024-07-17T07:08:10.763480+0000","last_active":"2024-07-17T07:08:12.259191+0000","last_peered":"2024-07-17T07:08:12.259191+0000","last_clean":"2024-07-17T07:08:12.259191+0000","last_became_active":"2024-07-17T07:08:10.763193+0000","last_became_peered":"2024-07-17T07:08:10.763193+0000","last_unstale":"2024-07-17T07:08:12.259191+0000","last_undegraded":"2024-07-17T07:08:12.259191+0000","last_fullsized":"2024-07-17T07:08:12.259191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:05.303830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260432+0000","last_change":"2024-07-17T07:08:09.743966+0000","last_active":"2024-07-17T07:08:12.260432+0000","last_peered":"2024-07-17T07:08:12.260432+0000","last_clean":"2024-07-17T07:08:12.260432+0000","last_became_active":"2024-07-17T07:08:09.743675+0000","last_became_peered":"2024-07-17T07:08:09.743675+0000","last_unstale":"2024-07-17T07:08:12.260432+0000","last_undegraded":"2024-07-17T07:08:12.260432+0000","last_fullsized":"2024-07-17T07:08:12.260432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.605297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256507+0000","last_change":"2024-07-17T07:08:10.776033+0000","last_active":"2024-07-17T07:08:12.256507+0000","last_peered":"2024-07-17T07:08:12.256507+0000","last_clean":"2024-07-17T07:08:12.256507+0000","last_became_active":"2024-07-17T07:08:10.773056+0000","last_became_peered":"2024-07-17T07:08:10.773056+0000","last_unstale":"2024-07-17T07:08:12.256507+0000","last_undegraded":"2024-07-17T07:08:12.256507+0000","last_fullsized":"2024-07-17T07:08:12.256507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:56.778454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.753013+0000","last_change":"2024-07-17T07:08:09.746576+0000","last_active":"2024-07-17T07:08:11.753013+0000","last_peered":"2024-07-17T07:08:11.753013+0000","last_clean":"2024-07-17T07:08:11.753013+0000","last_became_active":"2024-07-17T07:08:09.746008+0000","last_became_peered":"2024-07-17T07:08:09.746008+0000","last_unstale":"2024-07-17T07:08:11.753013+0000","last_undegraded":"2024-07-17T07:08:11.753013+0000","last_fullsized":"2024-07-17T07:08:11.753013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:59.696724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256382+0000","last_change":"2024-07-17T07:08:10.755144+0000","last_active":"2024-07-17T07:08:12.256382+0000","last_peered":"2024-07-17T07:08:12.256382+0000","last_clean":"2024-07-17T07:08:12.256382+0000","last_became_active":"2024-07-17T07:08:10.753484+0000","last_became_peered":"2024-07-17T07:08:10.753484+0000","last_unstale":"2024-07-17T07:08:12.256382+0000","last_undegraded":"2024-07-17T07:08:12.256382+0000","last_fullsized":"2024-07-17T07:08:12.256382+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:33.521693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.759857+0000","last_change":"2024-07-17T07:08:09.738826+0000","last_active":"2024-07-17T07:08:11.759857+0000","last_peered":"2024-07-17T07:08:11.759857+0000","last_clean":"2024-07-17T07:08:11.759857+0000","last_became_active":"2024-07-17T07:08:09.737129+0000","last_became_peered":"2024-07-17T07:08:09.737129+0000","last_unstale":"2024-07-17T07:08:11.759857+0000","last_undegraded":"2024-07-17T07:08:11.759857+0000","last_fullsized":"2024-07-17T07:08:11.759857+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:01.832881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.254733+0000","last_change":"2024-07-17T07:08:10.761514+0000","last_active":"2024-07-17T07:08:12.254733+0000","last_peered":"2024-07-17T07:08:12.254733+0000","last_clean":"2024-07-17T07:08:12.254733+0000","last_became_active":"2024-07-17T07:08:10.761248+0000","last_became_peered":"2024-07-17T07:08:10.761248+0000","last_unstale":"2024-07-17T07:08:12.254733+0000","last_undegraded":"2024-07-17T07:08:12.254733+0000","last_fullsized":"2024-07-17T07:08:12.254733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:58.645982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258490+0000","last_change":"2024-07-17T07:08:09.747032+0000","last_active":"2024-07-17T07:08:12.258490+0000","last_peered":"2024-07-17T07:08:12.258490+0000","last_clean":"2024-07-17T07:08:12.258490+0000","last_became_active":"2024-07-17T07:08:09.746759+0000","last_became_peered":"2024-07-17T07:08:09.746759+0000","last_unstale":"2024-07-17T07:08:12.258490+0000","last_undegraded":"2024-07-17T07:08:12.258490+0000","last_fullsized":"2024-07-17T07:08:12.258490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:50.063060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259385+0000","last_change":"2024-07-17T07:08:10.742687+0000","last_active":"2024-07-17T07:08:12.259385+0000","last_peered":"2024-07-17T07:08:12.259385+0000","last_clean":"2024-07-17T07:08:12.259385+0000","last_became_active":"2024-07-17T07:08:10.742230+0000","last_became_peered":"2024-07-17T07:08:10.742230+0000","last_unstale":"2024-07-17T07:08:12.259385+0000","last_undegraded":"2024-07-17T07:08:12.259385+0000","last_fullsized":"2024-07-17T07:08:12.259385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:02:06.644866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260663+0000","last_change":"2024-07-17T07:08:09.748408+0000","last_active":"2024-07-17T07:08:12.260663+0000","last_peered":"2024-07-17T07:08:12.260663+0000","last_clean":"2024-07-17T07:08:12.260663+0000","last_became_active":"2024-07-17T07:08:09.748028+0000","last_became_peered":"2024-07-17T07:08:09.748028+0000","last_unstale":"2024-07-17T07:08:12.260663+0000","last_undegraded":"2024-07-17T07:08:12.260663+0000","last_fullsized":"2024-07-17T07:08:12.260663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:01.504944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.260369+0000","last_change":"2024-07-17T07:08:10.764199+0000","last_active":"2024-07-17T07:08:12.260369+0000","last_peered":"2024-07-17T07:08:12.260369+0000","last_clean":"2024-07-17T07:08:12.260369+0000","last_became_active":"2024-07-17T07:08:10.762314+0000","last_became_peered":"2024-07-17T07:08:10.762314+0000","last_unstale":"2024-07-17T07:08:12.260369+0000","last_undegraded":"2024-07-17T07:08:12.260369+0000","last_fullsized":"2024-07-17T07:08:12.260369+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:59.666091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259670+0000","last_change":"2024-07-17T07:08:09.739221+0000","last_active":"2024-07-17T07:08:12.259670+0000","last_peered":"2024-07-17T07:08:12.259670+0000","last_clean":"2024-07-17T07:08:12.259670+0000","last_became_active":"2024-07-17T07:08:09.738952+0000","last_became_peered":"2024-07-17T07:08:09.738952+0000","last_unstale":"2024-07-17T07:08:12.259670+0000","last_undegraded":"2024-07-17T07:08:12.259670+0000","last_fullsized":"2024-07-17T07:08:12.259670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:17.033750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255944+0000","last_change":"2024-07-17T07:08:10.755979+0000","last_active":"2024-07-17T07:08:12.255944+0000","last_peered":"2024-07-17T07:08:12.255944+0000","last_clean":"2024-07-17T07:08:12.255944+0000","last_became_active":"2024-07-17T07:08:10.755540+0000","last_became_peered":"2024-07-17T07:08:10.755540+0000","last_unstale":"2024-07-17T07:08:12.255944+0000","last_undegraded":"2024-07-17T07:08:12.255944+0000","last_fullsized":"2024-07-17T07:08:12.255944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:05.303830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.257946+0000","last_change":"2024-07-17T07:08:09.732886+0000","last_active":"2024-07-17T07:08:12.257946+0000","last_peered":"2024-07-17T07:08:12.257946+0000","last_clean":"2024-07-17T07:08:12.257946+0000","last_became_active":"2024-07-17T07:08:09.732574+0000","last_became_peered":"2024-07-17T07:08:09.732574+0000","last_unstale":"2024-07-17T07:08:12.257946+0000","last_undegraded":"2024-07-17T07:08:12.257946+0000","last_fullsized":"2024-07-17T07:08:12.257946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:10.147524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.258800+0000","last_change":"2024-07-17T07:08:10.772034+0000","last_active":"2024-07-17T07:08:12.258800+0000","last_peered":"2024-07-17T07:08:12.258800+0000","last_clean":"2024-07-17T07:08:12.258800+0000","last_became_active":"2024-07-17T07:08:10.771746+0000","last_became_peered":"2024-07-17T07:08:10.771746+0000","last_unstale":"2024-07-17T07:08:12.258800+0000","last_undegraded":"2024-07-17T07:08:12.258800+0000","last_fullsized":"2024-07-17T07:08:12.258800+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:17.623143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758259+0000","last_change":"2024-07-17T07:08:09.747721+0000","last_active":"2024-07-17T07:08:11.758259+0000","last_peered":"2024-07-17T07:08:11.758259+0000","last_clean":"2024-07-17T07:08:11.758259+0000","last_became_active":"2024-07-17T07:08:09.747406+0000","last_became_peered":"2024-07-17T07:08:09.747406+0000","last_unstale":"2024-07-17T07:08:11.758259+0000","last_undegraded":"2024-07-17T07:08:11.758259+0000","last_fullsized":"2024-07-17T07:08:11.758259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:09.538020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259175+0000","last_change":"2024-07-17T07:08:10.752355+0000","last_active":"2024-07-17T07:08:12.259175+0000","last_peered":"2024-07-17T07:08:12.259175+0000","last_clean":"2024-07-17T07:08:12.259175+0000","last_became_active":"2024-07-17T07:08:10.751114+0000","last_became_peered":"2024-07-17T07:08:10.751114+0000","last_unstale":"2024-07-17T07:08:12.259175+0000","last_undegraded":"2024-07-17T07:08:12.259175+0000","last_fullsized":"2024-07-17T07:08:12.259175+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:58.231064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.256391+0000","last_change":"2024-07-17T07:08:09.741701+0000","last_active":"2024-07-17T07:08:12.256391+0000","last_peered":"2024-07-17T07:08:12.256391+0000","last_clean":"2024-07-17T07:08:12.256391+0000","last_became_active":"2024-07-17T07:08:09.739373+0000","last_became_peered":"2024-07-17T07:08:09.739373+0000","last_unstale":"2024-07-17T07:08:12.256391+0000","last_undegraded":"2024-07-17T07:08:12.256391+0000","last_fullsized":"2024-07-17T07:08:12.256391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:35:25.743533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:11.758738+0000","last_change":"2024-07-17T07:08:10.746705+0000","last_active":"2024-07-17T07:08:11.758738+0000","last_peered":"2024-07-17T07:08:11.758738+0000","last_clean":"2024-07-17T07:08:11.758738+0000","last_became_active":"2024-07-17T07:08:10.746417+0000","last_became_peered":"2024-07-17T07:08:10.746417+0000","last_unstale":"2024-07-17T07:08:11.758738+0000","last_undegraded":"2024-07-17T07:08:11.758738+0000","last_fullsized":"2024-07-17T07:08:11.758738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:09.692106+0000","last_clean_scrub_stamp":"2024-07-17T07:08:09.692106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:56.778454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.259480+0000","last_change":"2024-07-17T07:08:09.749287+0000","last_active":"2024-07-17T07:08:12.259480+0000","last_peered":"2024-07-17T07:08:12.259480+0000","last_clean":"2024-07-17T07:08:12.259480+0000","last_became_active":"2024-07-17T07:08:09.743978+0000","last_became_peered":"2024-07-17T07:08:09.743978+0000","last_unstale":"2024-07-17T07:08:12.259480+0000","last_undegraded":"2024-07-17T07:08:12.259480+0000","last_fullsized":"2024-07-17T07:08:12.259480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:08:08.674681+0000","last_clean_scrub_stamp":"2024-07-17T07:08:08.674681+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:11.147593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"57'86","reported_seq":251,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-07-17T07:08:12.255626+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:08:12.255626+0000","last_peered":"2024-07-17T07:08:12.255626+0000","last_clean":"2024-07-17T07:08:12.255626+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:08:12.255626+0000","last_undegraded":"2024-07-17T07:08:12.255626+0000","last_fullsized":"2024-07-17T07:08:12.255626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54748,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317092,"statfs":{"total":96636764160,"available":96580702208,"internally_reserved":0,"allocated":1384448,"data_stored":981327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":790528,"data_stored":391487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":794624,"data_stored":398019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":798720,"data_stored":398601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509603,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":798720,"data_stored":392469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542307,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58336,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313504,"statfs":{"total":96636764160,"available":96577028096,"internally_reserved":0,"allocated":806912,"data_stored":399631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509604,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":398179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":981259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:08:15.758 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:08:15.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:08:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 -- 172.21.15.44:0/1314844239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 msgr2=0x7f62ac06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 --2- 172.21.15.44:0/1314844239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac06dfd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6298005fd0 tx=0x7f629802a5d0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 -- 172.21.15.44:0/1314844239 shutdown_connections 2024-07-17T07:08:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 --2- 172.21.15.44:0/1314844239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 0x7f62ac0ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 --2- 172.21.15.44:0/1314844239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac06dfd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 -- 172.21.15.44:0/1314844239 >> 172.21.15.44:0/1314844239 conn(0x7f62ac0cf0d0 msgr2=0x7f62ac0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 -- 172.21.15.44:0/1314844239 shutdown_connections 2024-07-17T07:08:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.023+0000 7f62b2800640 1 -- 172.21.15.44:0/1314844239 wait complete. 2024-07-17T07:08:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.025+0000 7f62b2800640 1 Processor -- start 2024-07-17T07:08:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.025+0000 7f62b2800640 1 -- start start 2024-07-17T07:08:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 0x7f62ac072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac0b2220 con 0x7f62ac154490 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac0b20a0 con 0x7f62ac06f4f0 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac0b23a0 con 0x7f62ac06e510 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51118/0 (socket says 172.21.15.44:51118) 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.026+0000 7f62b1e00640 1 -- 172.21.15.44:0/1099078792 learned_addr learned my addr 172.21.15.44:0/1099078792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1400640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 0x7f62ac072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b0a00640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 -- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 msgr2=0x7f62ac072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 0x7f62ac072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 -- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 msgr2=0x7f62ac152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 -- 172.21.15.44:0/1099078792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac09adc0 con 0x7f62ac154490 2024-07-17T07:08:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62b1e00640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009cc0 tx=0x7f62a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.027+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4010630 con 0x7f62ac154490 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.028+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62a40107d0 con 0x7f62ac154490 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.028+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4010be0 con 0x7f62ac154490 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.028+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac1f7570 con 0x7f62ac154490 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.028+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1f7b30 con 0x7f62ac154490 2024-07-17T07:08:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.029+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac0ce980 con 0x7f62ac154490 2024-07-17T07:08:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.030+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f62a4021440 con 0x7f62ac154490 2024-07-17T07:08:16.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.032+0000 7f62a9600640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 0x7f628c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.032+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f62a4099860 con 0x7f62ac154490 2024-07-17T07:08:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.032+0000 7f62b1400640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 0x7f628c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.034+0000 7f62b1400640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 0x7f628c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007fd0 tx=0x7f62a0009040 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:16.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.036+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f62ac0ce980 con 0x7f62ac154490 2024-07-17T07:08:16.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.324+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f62ac0b2930 con 0x7f62ac154490 2024-07-17T07:08:16.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.329+0000 7f62a9600640 1 -- 172.21.15.44:0/1099078792 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7f62a4062780 con 0x7f62ac154490 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:08:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:08:16.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.331+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 msgr2=0x7f628c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.331+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 0x7f628c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007fd0 tx=0x7f62a0009040 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.331+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 msgr2=0x7f62ac073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.331+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009cc0 tx=0x7f62a400c6a0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 shutdown_connections 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f628c076cd0 0x7f628c079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62ac154490 0x7f62ac073f50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62ac06f4f0 0x7f62ac152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 --2- 172.21.15.44:0/1099078792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62ac06e510 0x7f62ac072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 >> 172.21.15.44:0/1099078792 conn(0x7f62ac0cf0d0 msgr2=0x7f62ac1532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 shutdown_connections 2024-07-17T07:08:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.332+0000 7f62b2800640 1 -- 172.21.15.44:0/1099078792 wait complete. 2024-07-17T07:08:16.350 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:08:16.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 -- 172.21.15.44:0/824024891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c80009ff0 msgr2=0x7f9c8000c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 --2- 172.21.15.44:0/824024891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c80009ff0 0x7f9c8000c480 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c610 tx=0x7f9c80003810 comp rx=0 tx=0).stop 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9a800640 1 -- 172.21.15.44:0/824024891 reap_dead start 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 -- 172.21.15.44:0/824024891 shutdown_connections 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 -- 172.21.15.44:0/824024891 >> 172.21.15.44:0/824024891 conn(0x7f9c9c0b0e80 msgr2=0x7f9c9c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 -- 172.21.15.44:0/824024891 shutdown_connections 2024-07-17T07:08:16.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.558+0000 7f9c9b200640 1 -- 172.21.15.44:0/824024891 wait complete. 2024-07-17T07:08:16.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.560+0000 7f9c9b200640 1 Processor -- start 2024-07-17T07:08:16.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.560+0000 7f9c9b200640 1 -- start start 2024-07-17T07:08:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c80009ff0 0x7f9c9c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c9c0aed80 0x7f9c9c0af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c15a2a0 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c15a120 con 0x7f9c9c0aed80 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9c0b1fa0 con 0x7f9c80009ff0 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51138/0 (socket says 172.21.15.44:51138) 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 -- 172.21.15.44:0/1513393825 learned_addr learned my addr 172.21.15.44:0/1513393825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c99e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c80009ff0 0x7f9c9c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c9a800640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c9c0aed80 0x7f9c9c0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 -- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c80009ff0 msgr2=0x7f9c9c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c80009ff0 0x7f9c9c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 -- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c9c0aed80 msgr2=0x7f9c9c0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c9c0aed80 0x7f9c9c0af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.562+0000 7f9c93e00640 1 -- 172.21.15.44:0/1513393825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c1f7910 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c93e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400d6f0 tx=0x7f9c8400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:16.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400de00 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c84015020 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8400b830 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c1f7bf0 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.563+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c14e730 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.564+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c0cf380 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.566+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f9c8400b9d0 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.567+0000 7f9c92000640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 0x7f9c78079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:16.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.567+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f9c840990b0 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.567+0000 7f9c99e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 0x7f9c78079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.569+0000 7f9c99e00640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 0x7f9c78079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009b30 tx=0x7f9c8c005f90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:16.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.571+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9c8401f020 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.824+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c9c14e200 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.825+0000 7f9c92000640 1 -- 172.21.15.44:0/1513393825 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7f9c84061f40 con 0x7f9c9c1f6e50 2024-07-17T07:08:16.824 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:16.824 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:08:16.824 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:08:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 msgr2=0x7f9c78079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 0x7f9c78079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009b30 tx=0x7f9c8c005f90 comp rx=0 tx=0).stop 2024-07-17T07:08:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 msgr2=0x7f9c9c1f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:16.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400d6f0 tx=0x7f9c8400dbb0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 shutdown_connections 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f9c78076cd0 0x7f9c78079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c9c0aed80 0x7f9c9c0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9c1f6e50 0x7f9c9c1f72d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 --2- 172.21.15.44:0/1513393825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c80009ff0 0x7f9c9c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.828+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 >> 172.21.15.44:0/1513393825 conn(0x7f9c9c0b0e80 msgr2=0x7f9c9c15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.829+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 shutdown_connections 2024-07-17T07:08:16.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:16.829+0000 7f9c9b200640 1 -- 172.21.15.44:0/1513393825 wait complete. 2024-07-17T07:08:17.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:08:10.204958+0000', 'modified': '2024-07-17T07:08:13.277327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5823}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5823': {'gid': 5823, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2772368935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2772368935}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2772368935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5823, 'qdb_cluster': [5823]} 2024-07-17T07:08:17.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:08:17.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 -- 172.21.15.44:0/1265978489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406eb90 msgr2=0x7f6b640ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1265978489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406eb90 0x7f6b640ac3a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b64070880 tx=0x7f6b5c02aa30 comp rx=0 tx=0).stop 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 -- 172.21.15.44:0/1265978489 shutdown_connections 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1265978489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406eb90 0x7f6b640ac3a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1265978489 >> v2:172.21.15.175:3300/0 conn(0x7f6b6406f6d0 0x7f6b6415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 -- 172.21.15.44:0/1265978489 >> 172.21.15.44:0/1265978489 conn(0x7f6b640b0eb0 msgr2=0x7f6b640b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 -- 172.21.15.44:0/1265978489 shutdown_connections 2024-07-17T07:08:17.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.477+0000 7f6b6b200640 1 -- 172.21.15.44:0/1265978489 wait complete. 2024-07-17T07:08:17.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.479+0000 7f6b6b200640 1 Processor -- start 2024-07-17T07:08:17.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.480+0000 7f6b6b200640 1 -- start start 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6406eb90 0x7f6b640988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 0x7f6b641f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640acc70 con 0x7f6b6406f6d0 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640acaf0 con 0x7f6b6406eb90 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b6b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6414fd60 con 0x7f6b640b3280 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51166/0 (socket says 172.21.15.44:51166) 2024-07-17T07:08:17.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.481+0000 7f6b5be00640 1 -- 172.21.15.44:0/1206060498 learned_addr learned my addr 172.21.15.44:0/1206060498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6406eb90 0x7f6b640988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b69200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 0x7f6b641f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 -- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 msgr2=0x7f6b641f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 0x7f6b641f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 -- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6406eb90 msgr2=0x7f6b640988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6406eb90 0x7f6b640988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 -- 172.21.15.44:0/1206060498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b640720c0 con 0x7f6b6406f6d0 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b69200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 0x7f6b641f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b5be00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00b3f0 tx=0x7f6b4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.482+0000 7f6b60c00640 1 -- 172.21.15.44:0/1206060498 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c015020 con 0x7f6b6406f6d0 2024-07-17T07:08:17.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.483+0000 7f6b60c00640 1 -- 172.21.15.44:0/1206060498 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b4c011920 con 0x7f6b6406f6d0 2024-07-17T07:08:17.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.483+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b640723a0 con 0x7f6b6406f6d0 2024-07-17T07:08:17.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.483+0000 7f6b60c00640 1 -- 172.21.15.44:0/1206060498 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c011d70 con 0x7f6b6406f6d0 2024-07-17T07:08:17.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.483+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b64072960 con 0x7f6b6406f6d0 2024-07-17T07:08:17.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.485+0000 7f6b60c00640 1 -- 172.21.15.44:0/1206060498 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f6b4c018020 con 0x7f6b6406f6d0 2024-07-17T07:08:17.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.486+0000 7f6b60c00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 0x7f6b40079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b6e011740 1 Processor -- start 2024-07-17T07:08:17.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b60c00640 1 -- 172.21.15.44:0/1206060498 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f6b4c098500 con 0x7f6b6406f6d0 2024-07-17T07:08:17.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b6e011740 1 -- start start 2024-07-17T07:08:17.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 0x7f6b40079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.487+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54b1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54ad41e0 0x562a54ad4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b6e011740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562a54a4ad60 con 0x562a54afeda0 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a54a3a260 con 0x562a54ad41e0 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562a54a3e2a0 con 0x562a54b1e9e0 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51178/0 (socket says 172.21.15.44:51178) 2024-07-17T07:08:17.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 -- 172.21.15.44:0/3598687687 learned_addr learned my addr 172.21.15.44:0/3598687687 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 -- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 msgr2=0x562a54b1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54b1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 -- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54ad41e0 msgr2=0x562a54ad4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54ad41e0 0x562a54ad4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.488+0000 7f6b63e00640 1 -- 172.21.15.44:0/3598687687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5479e850 con 0x562a54afeda0 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.489+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6b54014ba0 tx=0x7f6b54011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf7fdbce687c10f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.489+0000 7f6b58c00640 1 -- 172.21.15.44:0/3598687687 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54023070 con 0x562a54afeda0 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.489+0000 7f6b58c00640 1 -- 172.21.15.44:0/3598687687 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b54014300 con 0x562a54afeda0 2024-07-17T07:08:17.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.489+0000 7f6b58c00640 1 -- 172.21.15.44:0/3598687687 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400b490 con 0x562a54afeda0 2024-07-17T07:08:17.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 -- 172.21.15.44:0/3598687687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 msgr2=0x562a54ad6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 0x7f6b40079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b54011c50 tx=0x7f6b54018000 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6b54014ba0 tx=0x7f6b54011750 comp rx=0 tx=0).stop 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 -- 172.21.15.44:0/3598687687 shutdown_connections 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54ad41e0 0x562a54ad4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54b1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3598687687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54afeda0 0x562a54ad6710 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 -- 172.21.15.44:0/3598687687 >> 172.21.15.44:0/3598687687 conn(0x562a54ad5290 msgr2=0x562a54ad8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 -- 172.21.15.44:0/3598687687 shutdown_connections 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.490+0000 7f6b6e011740 1 -- 172.21.15.44:0/3598687687 wait complete. 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.491+0000 7f6b6e011740 1 -- start start 2024-07-17T07:08:17.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.491+0000 7f6b6e011740 1 Processor -- start 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54afeda0 0x562a54b319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54af6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562a547ad620 con 0x562a54af7fe0 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a549e1f90 con 0x562a54afeda0 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562a54ad34b0 con 0x562a54b1e9e0 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b6e011740 10 client.0 _ll_register_callbacks cb 0x562a54a07f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b69200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b69200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51180/0 (socket says 172.21.15.44:51180) 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.492+0000 7f6b69200640 1 -- 172.21.15.44:0/3610568456 learned_addr learned my addr 172.21.15.44:0/3610568456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54afeda0 0x562a54b319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b5be00640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54af6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 -- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 msgr2=0x562a54af6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54af6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 -- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54afeda0 msgr2=0x562a54b319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54afeda0 0x562a54b319e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 -- 172.21.15.44:0/3610568456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a548156b0 con 0x562a54af7fe0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b5be00640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54af6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b69200640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005fd0 tx=0x7f6b5c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452ce0fe761c929f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b51600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c033070 con 0x562a54af7fe0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f6b5c02f8a0 con 0x562a54af7fe0 2024-07-17T07:08:17.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.493+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b54014300 con 0x562a54af7fe0 2024-07-17T07:08:17.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b5c02faf0 con 0x562a54af7fe0 2024-07-17T07:08:17.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 10 client.5940 fetch_fsmap learned FSMap version 26 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x562a54b021f0 con 0x562a54af7fe0 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b54014300 con 0x562a54af7fe0 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f6b5c04a020 con 0x562a54af7fe0 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 10 client.5940 fetch_fsmap finished waiting for FSMap version 26 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 10 client.5940 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 20 client.5940 populate_metadata read hostname 'smithi044' 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b00610 0x562a54b2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 4 client.5940 mds_command: new command op to 5823 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.494+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 --> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a54a4ccb0 con 0x562a54b00610 2024-07-17T07:08:17.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.495+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b00610 0x562a54b2e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.495+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b00610 0x562a54b2e1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e43a28b1bf7a8 server_cookie=bd0aa68d18515c81 in_seq=0 out_seq=0 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.495+0000 7f6b51600640 10 client.5940 ms_handle_connect on v2:172.21.15.44:6833/2772368935 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b51600640 1 -- 172.21.15.44:0/3610568456 <== mds.0 v2:172.21.15.44:6833/2772368935 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a54a4ccb0 con 0x562a54b00610 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b51600640 10 client.5940 handle_command_reply: tid=0 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 client.5940 shutdown 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 msgr2=0x562a54b02710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005fd0 tx=0x7f6b5c003f40 comp rx=0 tx=0).stop 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 shutdown_connections 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b00610 0x562a54b2e1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b02710 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54b1e9e0 0x562a54af6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3610568456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54afeda0 0x562a54b319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b51600640 0 client.5940 ms_handle_reset on v2:172.21.15.44:6833/2772368935 2024-07-17T07:08:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.496+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 >> 172.21.15.44:0/3610568456 conn(0x562a54ad41e0 msgr2=0x562a54ad8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.497+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 shutdown_connections 2024-07-17T07:08:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.497+0000 7f6b6e011740 1 -- 172.21.15.44:0/3610568456 wait complete. 2024-07-17T07:08:17.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.497+0000 7f6b6e011740 20 client.5940 trim_cache size 0 max 16384 2024-07-17T07:08:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.518+0000 7f6b6e011740 1 Processor -- start 2024-07-17T07:08:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.518+0000 7f6b6e011740 1 -- start start 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54ad2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54ad2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64200440 con 0x562a54af7fe0 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b641f62f0 con 0x562a54b1e9e0 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b641ffdf0 con 0x562a54afeda0 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51188/0 (socket says 172.21.15.44:51188) 2024-07-17T07:08:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 -- 172.21.15.44:0/1032257516 learned_addr learned my addr 172.21.15.44:0/1032257516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 -- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 msgr2=0x562a54ad2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54ad2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 -- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 msgr2=0x562a54ad2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54ad2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.519+0000 7f6b63e00640 1 -- 172.21.15.44:0/1032257516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a54b20b70 con 0x562a54af7fe0 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.520+0000 7f6b63e00640 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400bfd0 tx=0x7f6b54005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7a5826d47d380e server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.520+0000 7f6b53e00640 1 -- 172.21.15.44:0/1032257516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54023040 con 0x562a54af7fe0 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.520+0000 7f6b53e00640 1 -- 172.21.15.44:0/1032257516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b54037070 con 0x562a54af7fe0 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.520+0000 7f6b53e00640 1 -- 172.21.15.44:0/1032257516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400b470 con 0x562a54af7fe0 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 -- 172.21.15.44:0/1032257516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 msgr2=0x562a54ad7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400bfd0 tx=0x7f6b54005850 comp rx=0 tx=0).stop 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 -- 172.21.15.44:0/1032257516 shutdown_connections 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54ad2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54ad2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 --2- 172.21.15.44:0/1032257516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54ad7c40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 -- 172.21.15.44:0/1032257516 >> 172.21.15.44:0/1032257516 conn(0x562a54ad41e0 msgr2=0x562a54b1f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 -- 172.21.15.44:0/1032257516 shutdown_connections 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.521+0000 7f6b6e011740 1 -- 172.21.15.44:0/1032257516 wait complete. 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.522+0000 7f6b6e011740 1 -- start start 2024-07-17T07:08:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.522+0000 7f6b6e011740 1 Processor -- start 2024-07-17T07:08:17.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54b16fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54b1fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562a54b00490 con 0x562a54af7fe0 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562a54af71a0 con 0x562a54b1e9e0 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x562a54b00ae0 con 0x562a54afeda0 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b6e011740 10 client.0 _ll_register_callbacks cb 0x7f6b5c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51190/0 (socket says 172.21.15.44:51190) 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.523+0000 7f6b63e00640 1 -- 172.21.15.44:0/3074751487 learned_addr learned my addr 172.21.15.44:0/3074751487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 -- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 msgr2=0x562a54b16fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54b16fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 -- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 msgr2=0x562a54b1fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:08:17.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54b1fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 -- 172.21.15.44:0/3074751487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a54b418a0 con 0x562a54af7fe0 2024-07-17T07:08:17.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b63e00640 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b54006010 tx=0x7f6b54011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a255e1bcdf5f4fa server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b51600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54042020 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f6b54014710 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.524+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b54037070 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b54023040 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 10 client.5946 fetch_fsmap learned FSMap version 26 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x562a54b174f0 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b54037070 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f6b54031070 con 0x562a54af7fe0 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 10 client.5946 fetch_fsmap finished waiting for FSMap version 26 2024-07-17T07:08:17.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 10 client.5946 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 20 client.5946 populate_metadata read hostname 'smithi044' 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b92a90 0x562a54afd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 4 client.5946 mds_command: new command op to 5823 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 --> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] -- command(tid 0: {"prefix": "status"}) -- 0x562a54b186b0 con 0x562a54b92a90 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.525+0000 7f6b5be00640 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b92a90 0x562a54afd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b5be00640 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b92a90 0x562a54afd350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266a4893fddd617c server_cookie=56cab4c1d118e5b4 in_seq=0 out_seq=0 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b51600640 10 client.5946 ms_handle_connect on v2:172.21.15.44:6833/2772368935 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b51600640 1 -- 172.21.15.44:0/3074751487 <== mds.0 v2:172.21.15.44:6833/2772368935 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562a54b186b0 con 0x562a54b92a90 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b51600640 10 client.5946 handle_command_reply: tid=0 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 client.5946 shutdown 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 msgr2=0x562a54b9f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b54006010 tx=0x7f6b54011d10 comp rx=0 tx=0).stop 2024-07-17T07:08:17.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 shutdown_connections 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:6833/2772368935,v1:172.21.15.44:6835/2772368935] conn(0x562a54b92a90 0x562a54afd350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562a54b1e9e0 0x562a54b1fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x562a54afeda0 0x562a54b16fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b6e011740 1 --2- 172.21.15.44:0/3074751487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a54af7fe0 0x562a54b9f800 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.526+0000 7f6b51600640 0 client.5946 ms_handle_reset on v2:172.21.15.44:6833/2772368935 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 >> 172.21.15.44:0/3074751487 conn(0x562a54ad41e0 msgr2=0x562a54af9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 shutdown_connections 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6e011740 1 -- 172.21.15.44:0/3074751487 wait complete. 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6e011740 20 client.5946 trim_cache size 0 max 16384 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:08:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 5823, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7852196789999999, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 26, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 67, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 67, 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.750800141999999 2024-07-17T07:08:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:08:17.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 msgr2=0x7f6b40079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 0x7f6b40079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b54011c50 tx=0x7f6b54018000 comp rx=0 tx=0).stop 2024-07-17T07:08:17.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 msgr2=0x7f6b640715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.527+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00b3f0 tx=0x7f6b4c00b8b0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 shutdown_connections 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f6b40076cd0 0x7f6b40079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b640b3280 0x7f6b641f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b6406f6d0 0x7f6b640715e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 --2- 172.21.15.44:0/1206060498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6406eb90 0x7f6b640988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 >> 172.21.15.44:0/1206060498 conn(0x7f6b640b0eb0 msgr2=0x7f6b6415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 shutdown_connections 2024-07-17T07:08:17.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.528+0000 7f6b6b200640 1 -- 172.21.15.44:0/1206060498 wait complete. 2024-07-17T07:08:17.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:08:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 -- 172.21.15.44:0/2262626229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 msgr2=0x7fba88003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 --2- 172.21.15.44:0/2262626229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fba88003e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fba94009b50 tx=0x7fba88005830 comp rx=0 tx=0).stop 2024-07-17T07:08:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa6200640 1 -- 172.21.15.44:0/2262626229 reap_dead start 2024-07-17T07:08:17.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 -- 172.21.15.44:0/2262626229 shutdown_connections 2024-07-17T07:08:17.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 -- 172.21.15.44:0/2262626229 >> 172.21.15.44:0/2262626229 conn(0x7fbaa00b0fb0 msgr2=0x7fbaa00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:17.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 -- 172.21.15.44:0/2262626229 shutdown_connections 2024-07-17T07:08:17.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.748+0000 7fbaa8200640 1 -- 172.21.15.44:0/2262626229 wait complete. 2024-07-17T07:08:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.750+0000 7fbaa8200640 1 Processor -- start 2024-07-17T07:08:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.750+0000 7fbaa8200640 1 -- start start 2024-07-17T07:08:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa014b9d0 0x7fbaa00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaa00b4030 0x7fbaa009a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa00ad790 con 0x7fba88003a50 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa00ce8b0 con 0x7fbaa014b9d0 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa8200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa00ad910 con 0x7fbaa00b4030 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.751+0000 7fbaa5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51208/0 (socket says 172.21.15.44:51208) 2024-07-17T07:08:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 -- 172.21.15.44:0/1606439632 learned_addr learned my addr 172.21.15.44:0/1606439632 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa6200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaa00b4030 0x7fbaa009a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa4e00640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa014b9d0 0x7fbaa00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 -- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaa00b4030 msgr2=0x7fbaa009a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaa00b4030 0x7fbaa009a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 -- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa014b9d0 msgr2=0x7fbaa00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa014b9d0 0x7fbaa00b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 -- 172.21.15.44:0/1606439632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa0151cc0 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.752+0000 7fbaa5800640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba94036040 tx=0x7fba94033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.753+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94038040 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.753+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba94004020 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.753+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94004470 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.753+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa01555a0 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.753+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa00b32f0 con 0x7fba88003a50 2024-07-17T07:08:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.754+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa00b0a10 con 0x7fba88003a50 2024-07-17T07:08:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.755+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7fba94045020 con 0x7fba88003a50 2024-07-17T07:08:17.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.756+0000 7fba9d600640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 0x7fba80079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:17.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.756+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7fba940b9120 con 0x7fba88003a50 2024-07-17T07:08:17.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.757+0000 7fbaa4e00640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 0x7fba80079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.758+0000 7fbaa4e00640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 0x7fba80079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbaa015ae60 tx=0x7fba8c00d3f0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:17.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:17.761+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fba94082030 con 0x7fba88003a50 2024-07-17T07:08:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.015+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbaa00b0730 con 0x7fba88003a50 2024-07-17T07:08:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.016+0000 7fba9d600640 1 -- 172.21.15.44:0/1606439632 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fba9408a300 con 0x7fba88003a50 2024-07-17T07:08:18.014 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.018+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 msgr2=0x7fba80079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.018+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 0x7fba80079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbaa015ae60 tx=0x7fba8c00d3f0 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.018+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 msgr2=0x7fbaa015a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.018+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba94036040 tx=0x7fba94033670 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 shutdown_connections 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fba80076cd0 0x7fba80079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaa00b4030 0x7fbaa009a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa014b9d0 0x7fbaa00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 --2- 172.21.15.44:0/1606439632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba88003a50 0x7fbaa015a510 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 >> 172.21.15.44:0/1606439632 conn(0x7fbaa00b0fb0 msgr2=0x7fbaa00aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 shutdown_connections 2024-07-17T07:08:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18.019+0000 7fbaa8200640 1 -- 172.21.15.44:0/1606439632 wait complete. 2024-07-17T07:08:18.037 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:08:18.037 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:08:18.037 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:08:18.037 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:08:18.037 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:08:18.038 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:08:18.038 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:08:18.038 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:08:18.038 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:08:18.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.038 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:08:18.088 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:08:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:08:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41760sec preferred_lft 41760sec 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:08:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:08:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:08:18.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.092 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:08:18.169 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:08:18.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.174 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:08:18.227 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:08:18.227 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:08:18.227 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:08:18.227 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:08:18.227 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:08:18.228 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:08:18.228 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:08:18.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:08:18.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:08:18.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:08:18.229 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:08:18.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:08:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:08:18.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.446 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:08:18.492 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:08:18.492 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:08:18.492 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:08:18.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:08:18.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:08:18.625 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:08:18.679 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:08:18.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:08:18.680 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:08:18.732 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:08:18.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:08:18.870 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:08:18.870 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:08:18.871 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:08:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:08:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:08:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:08:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:08:18.893 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:08:18.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:08:18.920 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:08:18.920 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:08:18.920 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:08:18.920 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:08:18.920 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:08:18.924 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:08:18.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:18.924 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:08:18.992 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:08:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:08:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:08:19.131 INFO:teuthology.orchestra.run.smithi044.stdout:client5299 2024-07-17T07:08:19.131 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:08:19.131 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:08:19.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.474+0000 7f0b62200640 1 -- 172.21.15.44:0/3470544102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b40005e50 msgr2=0x7f0b400062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.474+0000 7f0b62200640 1 --2- 172.21.15.44:0/3470544102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b40005e50 0x7f0b400062d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0016c0 tx=0x7f0b40006940 comp rx=0 tx=0).stop 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.474+0000 7f0b61800640 1 -- 172.21.15.44:0/3470544102 reap_dead start 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.474+0000 7f0b62200640 1 -- 172.21.15.44:0/3470544102 shutdown_connections 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.474+0000 7f0b62200640 1 -- 172.21.15.44:0/3470544102 >> 172.21.15.44:0/3470544102 conn(0x7f0b5c0cf0d0 msgr2=0x7f0b5c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.475+0000 7f0b62200640 1 -- 172.21.15.44:0/3470544102 shutdown_connections 2024-07-17T07:08:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.475+0000 7f0b62200640 1 -- 172.21.15.44:0/3470544102 wait complete. 2024-07-17T07:08:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.477+0000 7f0b62200640 1 Processor -- start 2024-07-17T07:08:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.478+0000 7f0b62200640 1 -- start start 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 0x7f0b5c1f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 0x7f0b5c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c06dc40 con 0x7f0b5c1f1340 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c06dac0 con 0x7f0b5c0c90a0 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.479+0000 7f0b62200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c1503c0 con 0x7f0b40005e50 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51182/0 (socket says 172.21.15.44:51182) 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 -- 172.21.15.44:0/3973167583 learned_addr learned my addr 172.21.15.44:0/3973167583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b60e00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 0x7f0b5c1f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b5be00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 0x7f0b5c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 -- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 msgr2=0x7f0b5c1f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 0x7f0b5c1f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 -- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 msgr2=0x7f0b5c0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 0x7f0b5c0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 -- 172.21.15.44:0/3973167583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c0ca3c0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b5be00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 0x7f0b5c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b60e00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 0x7f0b5c1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:19.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.480+0000 7f0b61800640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0062e0 tx=0x7f0b4c006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.481+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c006920 con 0x7f0b5c1f1340 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.481+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b4c0038e0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.481+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c003cf0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.481+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c0ca6a0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.481+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c0c88a0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.482+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c0ac010 con 0x7f0b5c1f1340 2024-07-17T07:08:19.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.483+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f0b4c030050 con 0x7f0b5c1f1340 2024-07-17T07:08:19.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.485+0000 7f0b58c00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 0x7f0b38079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.485+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f0b4c0b7990 con 0x7f0b5c1f1340 2024-07-17T07:08:19.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.486+0000 7f0b60e00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 0x7f0b38079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.487+0000 7f0b60e00640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 0x7f0b38079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b540015c0 tx=0x7f0b540018f0 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:19.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.489+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0b4c0809c0 con 0x7f0b5c1f1340 2024-07-17T07:08:19.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.724+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 --> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0b5c0cffe0 con 0x7f0b38076cd0 2024-07-17T07:08:19.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.728+0000 7f0b58c00640 1 -- 172.21.15.44:0/3973167583 <== mgr.5071 v2:172.21.15.175:6800/533048536 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0b5c0cffe0 con 0x7f0b38076cd0 2024-07-17T07:08:19.728 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:08:19.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:08:19.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:08:19.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:08:19.728 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.731+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 msgr2=0x7f0b38079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.731+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 0x7f0b38079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b540015c0 tx=0x7f0b540018f0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.731+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 msgr2=0x7f0b5c15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.731+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0062e0 tx=0x7f0b4c006310 comp rx=0 tx=0).stop 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 shutdown_connections 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f0b38076cd0 0x7f0b38079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b5c1f1340 0x7f0b5c15afa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b5c0c90a0 0x7f0b5c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 --2- 172.21.15.44:0/3973167583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b40005e50 0x7f0b5c1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 >> 172.21.15.44:0/3973167583 conn(0x7f0b5c0cf0d0 msgr2=0x7f0b5c073df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 shutdown_connections 2024-07-17T07:08:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.732+0000 7f0b62200640 1 -- 172.21.15.44:0/3973167583 wait complete. 2024-07-17T07:08:19.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 -- 172.21.15.44:0/2905522050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 msgr2=0x7f48ec0b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2905522050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0b2140 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f48ec06e790 tx=0x7f48dc02a4d0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 -- 172.21.15.44:0/2905522050 shutdown_connections 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2905522050 >> v2:172.21.15.175:3300/0 conn(0x7f48ec156d40 0x7f48ec0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2905522050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0b2140 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 -- 172.21.15.44:0/2905522050 >> 172.21.15.44:0/2905522050 conn(0x7f48ec06c120 msgr2=0x7f48ec06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 -- 172.21.15.44:0/2905522050 shutdown_connections 2024-07-17T07:08:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.968+0000 7f48ebe00640 1 -- 172.21.15.44:0/2905522050 wait complete. 2024-07-17T07:08:19.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.970+0000 7f48ebe00640 1 Processor -- start 2024-07-17T07:08:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.970+0000 7f48ebe00640 1 -- start start 2024-07-17T07:08:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 0x7f48ec1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 0x7f48ec0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec06d610 con 0x7f48ec06f5a0 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cf900 con 0x7f48ec156d40 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48ebe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec06d790 con 0x7f48ec0b1690 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51210/0 (socket says 172.21.15.44:51210) 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.971+0000 7f48eaa00640 1 -- 172.21.15.44:0/2636006530 learned_addr learned my addr 172.21.15.44:0/2636006530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48ea000640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 0x7f48ec1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 -- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 msgr2=0x7f48ec0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eb400640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 0x7f48ec0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 0x7f48ec0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 -- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 msgr2=0x7f48ec1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 0x7f48ec1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 -- 172.21.15.44:0/2636006530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec150e30 con 0x7f48ec06f5a0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48ea000640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 0x7f48ec1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eb400640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 0x7f48ec0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48eaa00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc03f070 con 0x7f48ec06f5a0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48dc004890 con 0x7f48ec06f5a0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc035030 con 0x7f48ec06f5a0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.972+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1f1b70 con 0x7f48ec06f5a0 2024-07-17T07:08:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.973+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1f1e30 con 0x7f48ec06f5a0 2024-07-17T07:08:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.974+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0b2140 con 0x7f48ec06f5a0 2024-07-17T07:08:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.975+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) ==== 98931+0+0 (secure 0 0 0) 0x7f48dc042070 con 0x7f48ec06f5a0 2024-07-17T07:08:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.976+0000 7f48e3400640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 0x7f48cc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.976+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f48dc0b97f0 con 0x7f48ec06f5a0 2024-07-17T07:08:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.977+0000 7f48ea000640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 0x7f48cc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.978+0000 7f48ea000640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 0x7f48cc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0b4240 tx=0x7f48d8005c90 comp rx=0 tx=0).ready entity=mgr.5071 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:19.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:19.980+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48ec150e30 con 0x7f48ec06f5a0 2024-07-17T07:08:20.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.250+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f48ec15bfc0 con 0x7f48ec06f5a0 2024-07-17T07:08:20.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.813+0000 7f48e3400640 1 -- 172.21.15.44:0/2636006530 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f48dc082710 con 0x7f48ec06f5a0 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 msgr2=0x7f48cc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 0x7f48cc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0b4240 tx=0x7f48d8005c90 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 msgr2=0x7f48ec0aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc0039e0 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 shutdown_connections 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f48cc076cd0 0x7f48cc079190 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0b4240 tx=0x7f48d8005c90 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec0b1690 0x7f48ec0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec156d40 0x7f48ec1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 --2- 172.21.15.44:0/2636006530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec06f5a0 0x7f48ec0aefc0 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc0039e0 comp rx=0 tx=0).stop 2024-07-17T07:08:20.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 >> 172.21.15.44:0/2636006530 conn(0x7f48ec06c120 msgr2=0x7f48ec1516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:20.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.817+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 shutdown_connections 2024-07-17T07:08:20.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:20.818+0000 7f48ebe00640 1 -- 172.21.15.44:0/2636006530 wait complete. 2024-07-17T07:08:20.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:08:21.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 -- 172.21.15.44:0/988613717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 msgr2=0x7fc94c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 --2- 172.21.15.44:0/988613717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c06d620 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0708b0 tx=0x7fc93402a530 comp rx=0 tx=0).stop 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 -- 172.21.15.44:0/988613717 shutdown_connections 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 --2- 172.21.15.44:0/988613717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c06d620 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 --2- 172.21.15.44:0/988613717 >> v2:172.21.15.175:3300/0 conn(0x7fc94c154600 0x7fc94c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 -- 172.21.15.44:0/988613717 >> 172.21.15.44:0/988613717 conn(0x7fc94c0b0eb0 msgr2=0x7fc94c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 -- 172.21.15.44:0/988613717 shutdown_connections 2024-07-17T07:08:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.030+0000 7fc952e00640 1 -- 172.21.15.44:0/988613717 wait complete. 2024-07-17T07:08:21.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.032+0000 7fc952e00640 1 Processor -- start 2024-07-17T07:08:21.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.032+0000 7fc952e00640 1 -- start start 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc94c154600 0x7fc94c148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c09a060 0x7fc94c09bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c15e0a0 con 0x7fc94c161120 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c0acab0 con 0x7fc94c09a060 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.033+0000 7fc952e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c15e220 con 0x7fc94c154600 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51254/0 (socket says 172.21.15.44:51254) 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 -- 172.21.15.44:0/3969743474 learned_addr learned my addr 172.21.15.44:0/3969743474 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc950e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c09a060 0x7fc94c09bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc94be00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc94c154600 0x7fc94c148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc94c154600 msgr2=0x7fc94c148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc94c154600 0x7fc94c148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c09a060 msgr2=0x7fc94c09bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c09a060 0x7fc94c09bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.034+0000 7fc943400640 1 -- 172.21.15.44:0/3969743474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c09a510 con 0x7fc94c161120 2024-07-17T07:08:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc943400640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc9340098f0 tx=0x7fc934033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934005c90 con 0x7fc94c161120 2024-07-17T07:08:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc934031e50 con 0x7fc94c161120 2024-07-17T07:08:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934005260 con 0x7fc94c161120 2024-07-17T07:08:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c072920 con 0x7fc94c161120 2024-07-17T07:08:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.035+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c09c8c0 con 0x7fc94c161120 2024-07-17T07:08:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.036+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c0b3ce0 con 0x7fc94c161120 2024-07-17T07:08:21.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.037+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 98948+0+0 (secure 0 0 0) 0x7fc934005400 con 0x7fc94c161120 2024-07-17T07:08:21.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.039+0000 7fc948c00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 0x7fc928079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.039+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7fc9340b9810 con 0x7fc94c161120 2024-07-17T07:08:21.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.039+0000 7fc94be00640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 msgr2=0x7fc928079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:21.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.039+0000 7fc94be00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 0x7fc928079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:08:21.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.044+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc94c09a510 con 0x7fc94c161120 2024-07-17T07:08:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.241+0000 7fc94be00640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 msgr2=0x7fc928079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.241+0000 7fc94be00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 0x7fc928079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:08:21.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.259+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc94c072af0 con 0x7fc94c161120 2024-07-17T07:08:21.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.264+0000 7fc948c00640 1 -- 172.21.15.44:0/3969743474 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7fc93403f030 con 0x7fc94c161120 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 msgr2=0x7fc928079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 0x7fc928079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 msgr2=0x7fc94c099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc9340098f0 tx=0x7fc934033c20 comp rx=0 tx=0).stop 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 shutdown_connections 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7fc928076d50 0x7fc928079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc94c09a060 0x7fc94c09bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc94c161120 0x7fc94c099b20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 --2- 172.21.15.44:0/3969743474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc94c154600 0x7fc94c148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 >> 172.21.15.44:0/3969743474 conn(0x7fc94c0b0eb0 msgr2=0x7fc94c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 shutdown_connections 2024-07-17T07:08:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.269+0000 7fc952e00640 1 -- 172.21.15.44:0/3969743474 wait complete. 2024-07-17T07:08:21.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.492+0000 7f9710600640 1 -- 172.21.15.44:0/2573756517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 msgr2=0x7f970815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.492+0000 7f9710600640 1 --2- 172.21.15.44:0/2573756517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f970815a5a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f96fc005fd0 tx=0x7f96fc02adb0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 -- 172.21.15.44:0/2573756517 shutdown_connections 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 --2- 172.21.15.44:0/2573756517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080ac3a0 0x7f97080ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 --2- 172.21.15.44:0/2573756517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f970815a5a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 -- 172.21.15.44:0/2573756517 >> 172.21.15.44:0/2573756517 conn(0x7f97080b0eb0 msgr2=0x7f97080b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 -- 172.21.15.44:0/2573756517 shutdown_connections 2024-07-17T07:08:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.493+0000 7f9710600640 1 -- 172.21.15.44:0/2573756517 wait complete. 2024-07-17T07:08:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.494+0000 7f9710600640 1 Processor -- start 2024-07-17T07:08:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.495+0000 7f9710600640 1 -- start start 2024-07-17T07:08:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.495+0000 7f9710600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f9710600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080ac3a0 0x7f970814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f9710600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080b3280 0x7f97080afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f9710600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f970806f850 con 0x7f970806eb90 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f9710600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f970806f6d0 con 0x7f97080ac3a0 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f9710600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f970814fd60 con 0x7f97080b3280 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51262/0 (socket says 172.21.15.44:51262) 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 -- 172.21.15.44:0/3163709713 learned_addr learned my addr 172.21.15.44:0/3163709713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970e600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080b3280 0x7f97080afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970d200640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080ac3a0 0x7f970814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080b3280 msgr2=0x7f97080afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080b3280 0x7f97080afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080ac3a0 msgr2=0x7f970814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080ac3a0 0x7f970814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.496+0000 7f970dc00640 1 -- 172.21.15.44:0/3163709713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708071ff0 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.497+0000 7f970dc00640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006010 tx=0x7f96fc0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.497+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc041020 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.497+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96fc02b9f0 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.497+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc02be40 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.497+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080722d0 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.498+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708072890 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.499+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970815a5a0 con 0x7f970806eb90 2024-07-17T07:08:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.500+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 98948+0+0 (secure 0 0 0) 0x7f96fc044020 con 0x7f970806eb90 2024-07-17T07:08:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.501+0000 7f9705600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 0x7f96e80791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.501+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7f96fc0b92d0 con 0x7f970806eb90 2024-07-17T07:08:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.501+0000 7f970d200640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 msgr2=0x7f96e80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.501+0000 7f970d200640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 0x7f96e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:08:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.505+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f96fc04b030 con 0x7f970806eb90 2024-07-17T07:08:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.703+0000 7f970d200640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 msgr2=0x7f96e80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.703+0000 7f970d200640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 0x7f96e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:08:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.720+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f97080b0c70 con 0x7f970806eb90 2024-07-17T07:08:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.726+0000 7f9705600640 1 -- 172.21.15.44:0/3163709713 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f96fc0821e0 con 0x7f970806eb90 2024-07-17T07:08:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 msgr2=0x7f96e80791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:08:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 0x7f96e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 msgr2=0x7f9708073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006010 tx=0x7f96fc0097a0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 shutdown_connections 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7f96e8076d20 0x7f96e80791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97080b3280 0x7f97080afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080ac3a0 0x7f970814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 --2- 172.21.15.44:0/3163709713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f970806eb90 0x7f9708073090 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 >> 172.21.15.44:0/3163709713 conn(0x7f97080b0eb0 msgr2=0x7f970815bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 shutdown_connections 2024-07-17T07:08:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.729+0000 7f9710600640 1 -- 172.21.15.44:0/3163709713 wait complete. 2024-07-17T07:08:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:21.746 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T07:08:21.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-07-17T07:08:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 -- 172.21.15.44:0/3486116238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff64415d130 msgr2=0x7ff64415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 --2- 172.21.15.44:0/3486116238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff64415d130 0x7ff64415d510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff634009830 tx=0x7ff63402a5c0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 -- 172.21.15.44:0/3486116238 shutdown_connections 2024-07-17T07:08:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 --2- 172.21.15.44:0/3486116238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff64415d130 0x7ff64415d510 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 -- 172.21.15.44:0/3486116238 >> 172.21.15.44:0/3486116238 conn(0x7ff6440b4ff0 msgr2=0x7ff6440b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 -- 172.21.15.44:0/3486116238 shutdown_connections 2024-07-17T07:08:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.973+0000 7ff649200640 1 -- 172.21.15.44:0/3486116238 wait complete. 2024-07-17T07:08:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.975+0000 7ff649200640 1 Processor -- start 2024-07-17T07:08:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.975+0000 7ff649200640 1 -- start start 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6440ce690 0x7ff64413f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64415d130 0x7ff6441588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 0x7ff64415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff644168860 con 0x7ff644161be0 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441686e0 con 0x7ff64415d130 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff649200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6441689e0 con 0x7ff6440ce690 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.976+0000 7ff643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 0x7ff64415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 0x7ff64415f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51284/0 (socket says 172.21.15.44:51284) 2024-07-17T07:08:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 -- 172.21.15.44:0/431442037 learned_addr learned my addr 172.21.15.44:0/431442037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff642a00640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64415d130 0x7ff6441588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6440ce690 0x7ff64413f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6440ce690 msgr2=0x7ff64413f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6440ce690 0x7ff64413f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64415d130 msgr2=0x7ff6441588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64415d130 0x7ff6441588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff644159390 con 0x7ff644161be0 2024-07-17T07:08:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.977+0000 7ff643e00640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 0x7ff64415f9d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c730 tx=0x7ff63c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.978+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c003b70 con 0x7ff644161be0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.978+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff63c003d10 con 0x7ff644161be0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.978+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00aab0 con 0x7ff644161be0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.978+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64413fcf0 con 0x7ff644161be0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.978+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64414da10 con 0x7ff644161be0 2024-07-17T07:08:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.979+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64415d510 con 0x7ff644161be0 2024-07-17T07:08:21.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.980+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 98948+0+0 (secure 0 0 0) 0x7ff63c00ac50 con 0x7ff644161be0 2024-07-17T07:08:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.981+0000 7ff63b400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.981+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6593+0+0 (secure 0 0 0) 0x7ff63c098730 con 0x7ff644161be0 2024-07-17T07:08:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.981+0000 7ff643400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.981+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:08:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:21.989+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff63c0616d0 con 0x7ff644161be0 2024-07-17T07:08:22.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:22.183+0000 7ff643400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:22.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:22.183+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:08:22.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:22.223+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff6440d0e10 con 0x7ff624076d20 2024-07-17T07:08:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:22.585+0000 7ff643400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:22.585+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:08:23.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:23.386+0000 7ff643400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:23.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:23.386+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:08:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:24.988+0000 7ff643400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/533048536 2024-07-17T07:08:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:24.988+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:08:25.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:25.038+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 20) ==== 98557+0+0 (secure 0 0 0) 0x7ff63c05d260 con 0x7ff644161be0 2024-07-17T07:08:25.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:25.038+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 msgr2=0x7ff6240791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:08:25.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:25.038+0000 7ff63b400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/533048536,v1:172.21.15.175:6801/533048536] conn(0x7ff624076d20 0x7ff6240791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.349 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:25.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:08:25.350+0000 7fdac2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:08:26.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.049+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 21) ==== 98684+0+0 (secure 0 0 0) 0x7ff63c06c170 con 0x7ff644161be0 2024-07-17T07:08:26.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.050+0000 7ff63b400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff62407f7c0 0x7ff624081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:26.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.050+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 --> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff62400c880 con 0x7ff62407f7c0 2024-07-17T07:08:26.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.052+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff62407f7c0 0x7ff624081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:26.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.061+0000 7ff643400640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff62407f7c0 0x7ff624081bb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff634034000 tx=0x7ff634004730 comp rx=0 tx=0).ready entity=mgr.5314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:26.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.264+0000 7ff63b400640 1 -- 172.21.15.44:0/431442037 <== mgr.5314 v2:172.21.15.175:6800/1668760217 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff62400c880 con 0x7ff62407f7c0 2024-07-17T07:08:26.263 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.267+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff62407f7c0 msgr2=0x7ff624081bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.267+0000 7ff649200640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff62407f7c0 0x7ff624081bb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff634034000 tx=0x7ff634004730 comp rx=0 tx=0).stop 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 msgr2=0x7ff64415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 --2- 172.21.15.44:0/431442037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff644161be0 0x7ff64415f9d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c730 tx=0x7ff63c00cbf0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff643e00640 1 -- 172.21.15.44:0/431442037 reap_dead start 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 shutdown_connections 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 >> 172.21.15.44:0/431442037 conn(0x7ff6440b4ff0 msgr2=0x7ff6440d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 shutdown_connections 2024-07-17T07:08:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.268+0000 7ff649200640 1 -- 172.21.15.44:0/431442037 wait complete. 2024-07-17T07:08:26.287 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T07:08:26.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2024-07-17T07:08:26.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 -- 172.21.15.44:0/2082058229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d400b33f0 msgr2=0x7f1d4006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 --2- 172.21.15.44:0/2082058229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d400b33f0 0x7f1d4006d420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1d40070a70 tx=0x7f1d3802aa10 comp rx=0 tx=0).stop 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 -- 172.21.15.44:0/2082058229 shutdown_connections 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 --2- 172.21.15.44:0/2082058229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d400b33f0 0x7f1d4006d420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 --2- 172.21.15.44:0/2082058229 >> v2:172.21.15.175:3300/0 conn(0x7f1d400ae340 0x7f1d400b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 --2- 172.21.15.44:0/2082058229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d401503c0 0x7f1d4015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 -- 172.21.15.44:0/2082058229 >> 172.21.15.44:0/2082058229 conn(0x7f1d400cf0d0 msgr2=0x7f1d400cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 -- 172.21.15.44:0/2082058229 shutdown_connections 2024-07-17T07:08:26.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.503+0000 7f1d45400640 1 -- 172.21.15.44:0/2082058229 wait complete. 2024-07-17T07:08:26.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.505+0000 7f1d45400640 1 Processor -- start 2024-07-17T07:08:26.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.505+0000 7f1d45400640 1 -- start start 2024-07-17T07:08:26.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d400ae340 0x7f1d40073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:26.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d400b33f0 0x7f1d4014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4006dae0 con 0x7f1d401503c0 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4006d960 con 0x7f1d400ae340 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.506+0000 7f1d45400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4006dc60 con 0x7f1d400b33f0 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51294/0 (socket says 172.21.15.44:51294) 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 -- 172.21.15.44:0/2950407949 learned_addr learned my addr 172.21.15.44:0/2950407949 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d400ae340 0x7f1d40073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:26.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d3f400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d400b33f0 0x7f1d4014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:26.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 -- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d400b33f0 msgr2=0x7f1d4014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d400b33f0 0x7f1d4014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 -- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d400ae340 msgr2=0x7f1d40073fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d400ae340 0x7f1d40073fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.507+0000 7f1d44a00640 1 -- 172.21.15.44:0/2950407949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4009b910 con 0x7f1d401503c0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d44a00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009d60 tx=0x7f1d38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38047070 con 0x7f1d401503c0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d38004250 con 0x7f1d401503c0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38033050 con 0x7f1d401503c0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4009bbf0 con 0x7f1d401503c0 2024-07-17T07:08:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.508+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4014bcc0 con 0x7f1d401503c0 2024-07-17T07:08:26.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.509+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d400cffe0 con 0x7f1d401503c0 2024-07-17T07:08:26.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.510+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 98684+0+0 (secure 0 0 0) 0x7f1d3804a070 con 0x7f1d401503c0 2024-07-17T07:08:26.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.511+0000 7f1d37e00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 0x7f1d20078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:26.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.511+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6937+0+0 (secure 0 0 0) 0x7f1d380b9750 con 0x7f1d401503c0 2024-07-17T07:08:26.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.512+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 0x7f1d20078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:26.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.513+0000 7f1d3fe00640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 0x7f1d20078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1d300058c0 tx=0x7f1d3000c440 comp rx=0 tx=0).ready entity=mgr.5314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:26.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.516+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1d380829c0 con 0x7f1d401503c0 2024-07-17T07:08:26.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.750+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 --> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) -- 0x7f1d400c4600 con 0x7f1d20076a50 2024-07-17T07:08:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.916+0000 7f1d37e00640 1 -- 172.21.15.44:0/2950407949 <== mgr.5314 v2:172.21.15.175:6800/1668760217 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d400c4600 con 0x7f1d20076a50 2024-07-17T07:08:26.915 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T07:08:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.919+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 msgr2=0x7f1d20078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.919+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 0x7f1d20078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1d300058c0 tx=0x7f1d3000c440 comp rx=0 tx=0).stop 2024-07-17T07:08:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.919+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 msgr2=0x7f1d400c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.919+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009d60 tx=0x7f1d38003f10 comp rx=0 tx=0).stop 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 shutdown_connections 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f1d20076a50 0x7f1d20078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d401503c0 0x7f1d400c40c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d400b33f0 0x7f1d4014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 --2- 172.21.15.44:0/2950407949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d400ae340 0x7f1d40073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 >> 172.21.15.44:0/2950407949 conn(0x7f1d400cf0d0 msgr2=0x7f1d40158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:26.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 shutdown_connections 2024-07-17T07:08:26.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:26.920+0000 7f1d45400640 1 -- 172.21.15.44:0/2950407949 wait complete. 2024-07-17T07:08:26.940 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T07:08:26.940 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-07-17-07_09_00 in ~34s... 2024-07-17T07:08:26.941 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-07-17-07_10_00 in ~94s... 2024-07-17T07:08:26.941 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2024-07-17T07:08:26.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 -- 172.21.15.44:0/2083214755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f9806f6d0 msgr2=0x7f5f9815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 --2- 172.21.15.44:0/2083214755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f9806f6d0 0x7f5f9815a5a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00a940 tx=0x7f5f8c02b840 comp rx=0 tx=0).stop 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 -- 172.21.15.44:0/2083214755 shutdown_connections 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 --2- 172.21.15.44:0/2083214755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f9806f6d0 0x7f5f9815a5a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 -- 172.21.15.44:0/2083214755 >> 172.21.15.44:0/2083214755 conn(0x7f5f980b0eb0 msgr2=0x7f5f980b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 -- 172.21.15.44:0/2083214755 shutdown_connections 2024-07-17T07:08:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.172+0000 7f5f9f600640 1 -- 172.21.15.44:0/2083214755 wait complete. 2024-07-17T07:08:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.174+0000 7f5f9f600640 1 Processor -- start 2024-07-17T07:08:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.174+0000 7f5f9f600640 1 -- start start 2024-07-17T07:08:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 0x7f5f9813ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980acab0 0x7f5f9814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9806ed10 con 0x7f5f980b3280 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9806eb90 con 0x7f5f9806f6d0 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9806ee90 con 0x7f5f980acab0 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51314/0 (socket says 172.21.15.44:51314) 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9d600640 1 -- 172.21.15.44:0/1903542916 learned_addr learned my addr 172.21.15.44:0/1903542916 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f97e00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980acab0 0x7f5f9814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.176+0000 7f5f9cc00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 0x7f5f9813ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 -- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980acab0 msgr2=0x7f5f9814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980acab0 0x7f5f9814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 -- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 msgr2=0x7f5f9813ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 0x7f5f9813ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 -- 172.21.15.44:0/1903542916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9813d3f0 con 0x7f5f980b3280 2024-07-17T07:08:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9cc00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 0x7f5f9813ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:08:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9d600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c780 tx=0x7f5f9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90004b60 con 0x7f5f980b3280 2024-07-17T07:08:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f90004df0 con 0x7f5f980b3280 2024-07-17T07:08:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000aa70 con 0x7f5f980b3280 2024-07-17T07:08:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.177+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9814dd90 con 0x7f5f980b3280 2024-07-17T07:08:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.178+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9814e630 con 0x7f5f980b3280 2024-07-17T07:08:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.179+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9815a5a0 con 0x7f5f980b3280 2024-07-17T07:08:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.180+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 98949+0+0 (secure 0 0 0) 0x7f5f9000ac10 con 0x7f5f980b3280 2024-07-17T07:08:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.181+0000 7f5f94c00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 0x7f5f780791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:08:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.181+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6937+0+0 (secure 0 0 0) 0x7f5f90098bb0 con 0x7f5f980b3280 2024-07-17T07:08:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.182+0000 7f5f9cc00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 0x7f5f780791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:08:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.183+0000 7f5f9cc00640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 0x7f5f780791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c036000 tx=0x7f5f8c004730 comp rx=0 tx=0).ready entity=mgr.5314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:08:27.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.186+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f90020070 con 0x7f5f980b3280 2024-07-17T07:08:27.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.424+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 --> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f9815a2c0 con 0x7f5f78076d20 2024-07-17T07:08:27.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.442+0000 7f5f94c00640 1 -- 172.21.15.44:0/1903542916 <== mgr.5314 v2:172.21.15.175:6800/1668760217 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+76 (secure 0 0 0) 0x7f5f9815a2c0 con 0x7f5f78076d20 2024-07-17T07:08:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:08:27.441 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2024-07-17T07:08:27.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 msgr2=0x7f5f780791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:27.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 0x7f5f780791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c036000 tx=0x7f5f8c004730 comp rx=0 tx=0).stop 2024-07-17T07:08:27.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 msgr2=0x7f5f9815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:08:27.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c780 tx=0x7f5f9000cc40 comp rx=0 tx=0).stop 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 shutdown_connections 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5f78076d20 0x7f5f780791e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f980b3280 0x7f5f9815bec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f980acab0 0x7f5f9814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 --2- 172.21.15.44:0/1903542916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f9806f6d0 0x7f5f9813ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 >> 172.21.15.44:0/1903542916 conn(0x7f5f980b0eb0 msgr2=0x7f5f9815ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 shutdown_connections 2024-07-17T07:08:27.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:08:27.445+0000 7f5f9f600640 1 -- 172.21.15.44:0/1903542916 wait complete. 2024-07-17T07:08:27.464 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2024-07-17T07:08:27.464 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2024-07-17T07:08:27.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:27.464 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:27.518 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:27.519 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:27.519 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:28.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:28.520 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:28.582 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:28.582 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:28.582 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:29.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:29.584 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:29.646 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:29.646 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:29.646 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:30.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:30.647 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:30.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:30.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:30.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:31.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:31.709 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:31.770 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:31.770 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:31.770 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:32.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:32.771 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:32.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:32.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:32.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:33.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:33.836 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:33.898 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:33.898 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:33.898 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:34.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:34.899 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:34.961 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:34.961 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:34.961 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:35.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:35.962 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:36.023 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:36.023 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:36.023 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:37.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:37.024 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:37.084 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:37.084 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:37.084 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:38.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:38.085 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:38.147 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:38.147 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:38.147 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:39.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:39.148 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:39.211 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:39.211 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:39.211 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:40.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:40.212 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:40.273 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:40.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:40.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:41.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:41.275 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:41.336 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:41.336 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:41.336 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:42.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:42.337 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:42.398 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:42.399 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:42.399 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:43.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:43.400 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:43.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:43.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:43.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:44.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:44.464 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:44.522 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:44.523 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:44.523 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:44.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:08:44.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:08:45.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:45.524 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:45.586 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:45.586 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:45.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:46.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:46.587 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:46.648 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:46.648 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:46.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:47.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:47.650 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:47.710 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:47.710 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:47.710 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:48.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:48.712 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:48.774 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:48.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:48.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:49.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:49.775 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:49.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:49.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:49.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:50.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:50.835 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:50.897 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:50.897 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:50.898 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:51.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:51.899 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:51.961 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:51.961 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:51.961 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:52.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:52.962 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:53.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:53.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:53.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:54.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:54.025 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:54.087 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:54.087 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:54.087 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:55.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:55.089 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:55.150 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:55.150 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:55.150 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:56.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:56.151 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:56.214 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:56.215 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:56.215 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:57.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:57.216 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:57.278 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:57.278 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:57.279 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:58.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:58.280 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:58.341 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:58.341 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:58.341 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:08:59.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:08:59.342 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:08:59.404 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:08:59.404 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:08:59.404 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:00.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:00.406 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:00.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:09:00.464 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:00.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:00.940 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:09:00.940+0000 7fdab2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:09:00.940 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:09:00.940+0000 7fdab2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:09:00.940 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:09:00.940+0000 7fdab2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:09:00.940 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:09:00.940+0000 7fdab2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:09:00.940 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:09:00.940+0000 7fdab2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:09:01.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:01.465 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:01.525 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:01.526 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:01.526 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:01.526 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:01.526 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-07-17-07_09_00_UTC'] 2024-07-17T07:09:01.526 DEBUG:tasks.cephfs.test_snap_schedules:exec=1721200106.940811, now = 1721200141.5269244, timo = 34 2024-07-17T07:09:02.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:02.527 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:02.588 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:02.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:02.590 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:02.590 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:03.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:03.591 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:03.649 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:03.650 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:03.650 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:03.650 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:04.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:04.651 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:04.711 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:04.712 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:04.712 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:04.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:05.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:05.714 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:05.774 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:05.775 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:05.775 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:05.776 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:06.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:06.777 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:06.835 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:06.837 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:06.837 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:06.837 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:07.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:07.838 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:07.898 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:07.900 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:07.900 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:07.900 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:08.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:08.901 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:08.963 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:08.965 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:08.965 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:08.965 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:09.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:09.966 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:10.028 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:10.029 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:10.030 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:10.030 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:11.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:11.031 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:11.087 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:11.088 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:11.089 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:11.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:12.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:12.090 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:12.151 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:12.152 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:12.152 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:12.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:13.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:13.154 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:13.215 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:13.216 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:13.216 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:13.216 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:14.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:14.218 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:14.279 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:14.280 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:14.280 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:14.280 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:14.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:09:14.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:09:15.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:15.281 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:15.341 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:15.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:15.342 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:15.342 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:16.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:16.344 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:16.404 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:16.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:16.406 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:16.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:17.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:17.407 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:17.468 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:17.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:17.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:17.470 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:18.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:18.471 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:18.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:18.533 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:18.533 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:18.533 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:19.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:19.534 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:19.595 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:19.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:19.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:19.597 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:20.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:20.598 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:20.657 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:20.659 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:20.659 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:20.659 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:21.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:21.660 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:21.721 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:21.722 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:21.722 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:22.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:22.723 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:22.782 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:22.783 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:22.783 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:22.783 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:23.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:23.784 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:23.844 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:23.846 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:23.846 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:23.846 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:24.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:24.847 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:24.909 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:24.910 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:24.910 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:24.910 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:25.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:25.911 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:25.971 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:25.972 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:25.972 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:25.972 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:26.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:26.973 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:27.033 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:27.035 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:27.035 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:27.035 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:28.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:28.036 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:28.095 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:28.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:28.097 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:28.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:29.098 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:29.157 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:29.159 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:29.159 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:29.159 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:30.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:30.161 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:30.221 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:30.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:30.222 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:30.222 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:31.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:31.224 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:31.284 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:31.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:31.286 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:31.286 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:32.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:32.287 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:32.347 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:32.348 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:32.348 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:32.348 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:33.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:33.349 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:33.410 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:33.411 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:33.411 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:33.411 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:34.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:34.413 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:34.474 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:34.475 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:34.475 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:34.475 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:35.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:35.477 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:35.536 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:35.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:35.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:35.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:36.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:36.539 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:36.596 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:36.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:36.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:36.597 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:37.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:37.599 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:37.659 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:37.661 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:37.661 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:37.661 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:38.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:38.662 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:38.721 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:38.723 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:38.723 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:38.723 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:39.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:39.724 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:39.784 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:39.785 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:39.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:39.785 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:40.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:40.786 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:40.847 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:40.849 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:40.849 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:40.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:41.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:41.850 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:41.911 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:41.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:41.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:41.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:42.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:42.913 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:42.973 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:42.974 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:42.974 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:42.974 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:43.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:43.975 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:44.033 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:44.035 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:44.035 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:44.035 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:44.854 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:09:44.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:09:45.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:45.036 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:45.096 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:45.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:45.097 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:45.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:46.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:46.099 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:46.157 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:46.159 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:46.159 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:46.159 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:47.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:47.160 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:47.219 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:47.220 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:47.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:47.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:48.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:48.221 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:48.281 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:48.282 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:48.283 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:48.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:49.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:49.283 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:49.342 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:49.344 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:49.344 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:49.344 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:50.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:50.345 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:50.405 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:50.407 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:50.407 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:50.407 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:51.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:51.408 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:51.468 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:51.469 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:51.469 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:51.469 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:52.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:52.471 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:52.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:52.532 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:52.532 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:52.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:53.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:53.533 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:53.595 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:53.596 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:53.596 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:53.596 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:54.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:54.597 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:54.658 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:54.663 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:54.663 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:54.663 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:55.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:55.664 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:55.724 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:55.726 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:55.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:55.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:56.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:56.727 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:56.788 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:56.790 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:56.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:56.791 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:57.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:57.792 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:57.851 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:57.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:57.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:57.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:58.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:58.853 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:58.913 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:58.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:58.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:58.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:09:59.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:09:59.916 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:09:59.975 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:09:59.977 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC'} 2024-07-17T07:09:59.977 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:09:59.977 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:10:00.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:00.978 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:10:01.038 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:10:01.038 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_10_00_UTC 2024-07-17T07:10:01.039 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_09_00_UTC', 'scheduled-2024-07-17-07_10_00_UTC'} 2024-07-17T07:10:01.039 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2024-07-17-07_10_00_UTC'} 2024-07-17T07:10:01.039 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:10:01.039 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-07-17-07_10_00_UTC'] 2024-07-17T07:10:01.040 DEBUG:tasks.cephfs.test_snap_schedules:exec=1721200106.940811, now = 1721200201.0400784, timo = 94 2024-07-17T07:10:02.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.268+0000 7f01bc000640 1 -- 172.21.15.44:0/2558566541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40ac8a0 msgr2=0x7f01b415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.268+0000 7f01bc000640 1 --2- 172.21.15.44:0/2558566541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40ac8a0 0x7f01b415ac10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01b40d0260 tx=0x7f01a802a540 comp rx=0 tx=0).stop 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 -- 172.21.15.44:0/2558566541 shutdown_connections 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 --2- 172.21.15.44:0/2558566541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40ac8a0 0x7f01b415ac10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 --2- 172.21.15.44:0/2558566541 >> v2:172.21.15.175:3300/0 conn(0x7f01b415a280 0x7f01b40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 -- 172.21.15.44:0/2558566541 >> 172.21.15.44:0/2558566541 conn(0x7f01b40b0eb0 msgr2=0x7f01b40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 -- 172.21.15.44:0/2558566541 shutdown_connections 2024-07-17T07:10:02.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.269+0000 7f01bc000640 1 -- 172.21.15.44:0/2558566541 wait complete. 2024-07-17T07:10:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.270+0000 7f01bc000640 1 Processor -- start 2024-07-17T07:10:02.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.271+0000 7f01bc000640 1 -- start start 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.271+0000 7f01bc000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40ac8a0 0x7f01b40c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01bc000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01bc000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b415a280 0x7f01b414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01bc000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01b406d760 con 0x7f01b40b1af0 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01bc000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01b406f1d0 con 0x7f01b415a280 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01bc000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01b415b150 con 0x7f01b40ac8a0 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35062/0 (socket says 172.21.15.44:35062) 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 -- 172.21.15.44:0/554218607 learned_addr learned my addr 172.21.15.44:0/554218607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b9600640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40ac8a0 0x7f01b40c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01ba000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b415a280 0x7f01b414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 -- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40ac8a0 msgr2=0x7f01b40c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40ac8a0 0x7f01b40c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 -- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b415a280 msgr2=0x7f01b414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b415a280 0x7f01b414b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.272+0000 7f01b8c00640 1 -- 172.21.15.44:0/554218607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41509f0 con 0x7f01b40b1af0 2024-07-17T07:10:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01b8c00640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c490 tx=0x7f01a000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0016020 con 0x7f01b40b1af0 2024-07-17T07:10:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a0012940 con 0x7f01b40b1af0 2024-07-17T07:10:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0012d90 con 0x7f01b40b1af0 2024-07-17T07:10:02.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b40c8b80 con 0x7f01b40b1af0 2024-07-17T07:10:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.273+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b40c9140 con 0x7f01b40b1af0 2024-07-17T07:10:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.274+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b415b6e0 con 0x7f01b40b1af0 2024-07-17T07:10:02.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.275+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 98960+0+0 (secure 0 0 0) 0x7f01a0019020 con 0x7f01b40b1af0 2024-07-17T07:10:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.276+0000 7f01b1600640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 0x7f01940791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.276+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7f01a0098730 con 0x7f01b40b1af0 2024-07-17T07:10:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.277+0000 7f01b9600640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 0x7f01940791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.278+0000 7f01b9600640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 0x7f01940791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac009500 comp rx=0 tx=0).ready entity=mgr.5314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.281+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01a0020030 con 0x7f01b40b1af0 2024-07-17T07:10:02.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.516+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 --> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f01b40c88a0 con 0x7f0194076d20 2024-07-17T07:10:02.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.718+0000 7f01b1600640 1 -- 172.21.15.44:0/554218607 <== mgr.5314 v2:172.21.15.175:6800/1668760217 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f01b40c88a0 con 0x7f0194076d20 2024-07-17T07:10:02.718 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path snap_test_dir1 2024-07-17T07:10:02.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 msgr2=0x7f01940791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:02.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 0x7f01940791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac009500 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 msgr2=0x7f01b4158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f01a000c490 tx=0x7f01a000c950 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 shutdown_connections 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f0194076d20 0x7f01940791e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01b415a280 0x7f01b414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b40b1af0 0x7f01b4158940 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.721+0000 7f01bc000640 1 --2- 172.21.15.44:0/554218607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01b40ac8a0 0x7f01b40c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.722+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 >> 172.21.15.44:0/554218607 conn(0x7f01b40b0eb0 msgr2=0x7f01b4154610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.722+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 shutdown_connections 2024-07-17T07:10:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:02.722+0000 7f01bc000640 1 -- 172.21.15.44:0/554218607 wait complete. 2024-07-17T07:10:02.743 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-07-17T07:10:02.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:02.743 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:10:02.798 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:10:02.799 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_10_00_UTC 2024-07-17T07:10:02.800 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_09_00_UTC 2024-07-17T07:10:02.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:02.800 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_09_00_UTC) 2024-07-17T07:10:02.861 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_10_00_UTC 2024-07-17T07:10:02.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:02.861 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_10_00_UTC) 2024-07-17T07:10:02.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:02.923 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T07:10:02.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:10:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 -- 172.21.15.44:0/2736397438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50b4004d40 msgr2=0x7f50b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 --2- 172.21.15.44:0/2736397438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50b4004d40 0x7f50b40051c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f50c8005fd0 tx=0x7f50b4005830 comp rx=0 tx=0).stop 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50cfe00640 1 -- 172.21.15.44:0/2736397438 reap_dead start 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 -- 172.21.15.44:0/2736397438 shutdown_connections 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 -- 172.21.15.44:0/2736397438 >> 172.21.15.44:0/2736397438 conn(0x7f50d00d3460 msgr2=0x7f50d00d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 -- 172.21.15.44:0/2736397438 shutdown_connections 2024-07-17T07:10:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.199+0000 7f50d5000640 1 -- 172.21.15.44:0/2736397438 wait complete. 2024-07-17T07:10:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.201+0000 7f50d5000640 1 Processor -- start 2024-07-17T07:10:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.201+0000 7f50d5000640 1 -- start start 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50b4004d40 0x7f50d0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 0x7f50d0151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50d00b8f70 con 0x7f50d00cb630 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50d00b8df0 con 0x7f50d0159970 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.202+0000 7f50d5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50d00d21f0 con 0x7f50b4004d40 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35082/0 (socket says 172.21.15.44:35082) 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 -- 172.21.15.44:0/2754024988 learned_addr learned my addr 172.21.15.44:0/2754024988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cea00640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 0x7f50d0151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cf400640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50b4004d40 0x7f50d0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 -- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50b4004d40 msgr2=0x7f50d0159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50b4004d40 0x7f50d0159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 -- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 msgr2=0x7f50d0151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 0x7f50d0151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 -- 172.21.15.44:0/2754024988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d00ca5d0 con 0x7f50d00cb630 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cea00640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 0x7f50d0151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.203+0000 7f50cfe00640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002230 tx=0x7f50c8034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.204+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8043070 con 0x7f50d00cb630 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.204+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50c8034ca0 con 0x7f50d00cb630 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.204+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8038040 con 0x7f50d00cb630 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.204+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d00ca870 con 0x7f50d00cb630 2024-07-17T07:10:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.204+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d00cae30 con 0x7f50d00cb630 2024-07-17T07:10:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.205+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d006d260 con 0x7f50d00cb630 2024-07-17T07:10:03.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.206+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 98960+0+0 (secure 0 0 0) 0x7f50c8003bb0 con 0x7f50d00cb630 2024-07-17T07:10:03.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.207+0000 7f50c7400640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 0x7f50b00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:03.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.208+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7f50c80b8a10 con 0x7f50d00cb630 2024-07-17T07:10:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.208+0000 7f50cf400640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 0x7f50b00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.209+0000 7f50cf400640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 0x7f50b00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099e0 tx=0x7f50c0005f90 comp rx=0 tx=0).ready entity=mgr.5314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.212+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f50d006d260 con 0x7f50d00cb630 2024-07-17T07:10:03.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.487+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f50d00cecf0 con 0x7f50d00cb630 2024-07-17T07:10:03.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.531+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) ==== 93+0+0 (secure 0 0 0) 0x7f50d00cecf0 con 0x7f50d00cb630 2024-07-17T07:10:03.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.536+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 msgr2=0x7f50b00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:03.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.536+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 0x7f50b00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099e0 tx=0x7f50c0005f90 comp rx=0 tx=0).stop 2024-07-17T07:10:03.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.536+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 msgr2=0x7f50d00cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.537+0000 7f50c7400640 1 -- 172.21.15.44:0/2754024988 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7f50c807d3d0 con 0x7f50d00cb630 2024-07-17T07:10:03.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.537+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f50c8002230 tx=0x7f50c8034bb0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 shutdown_connections 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f50b0076d20 0x7f50b00791e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50d00cb630 0x7f50d00cbab0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50d0159970 0x7f50d0151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 --2- 172.21.15.44:0/2754024988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50b4004d40 0x7f50d0159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:03.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.538+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 >> 172.21.15.44:0/2754024988 conn(0x7f50d00d3460 msgr2=0x7f50d015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:03.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.539+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 shutdown_connections 2024-07-17T07:10:03.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:03.539+0000 7f50d5000640 1 -- 172.21.15.44:0/2754024988 wait complete. 2024-07-17T07:10:03.568 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:10:03.568 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:10:03.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:10:03.758 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:10:03.758 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:10:03.758 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:03.759 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:03.807 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:03.808 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:03.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:10:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 -- 172.21.15.44:0/3590687302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740152220 msgr2=0x7f374006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 --2- 172.21.15.44:0/3590687302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740152220 0x7f374006d620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3730005fd0 tx=0x7f373002a540 comp rx=0 tx=0).stop 2024-07-17T07:10:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 -- 172.21.15.44:0/3590687302 shutdown_connections 2024-07-17T07:10:04.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 --2- 172.21.15.44:0/3590687302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740152220 0x7f374006d620 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 -- 172.21.15.44:0/3590687302 >> 172.21.15.44:0/3590687302 conn(0x7f37400b0eb0 msgr2=0x7f37400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 -- 172.21.15.44:0/3590687302 shutdown_connections 2024-07-17T07:10:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.101+0000 7f3746400640 1 -- 172.21.15.44:0/3590687302 wait complete. 2024-07-17T07:10:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.102+0000 7f3746400640 1 Processor -- start 2024-07-17T07:10:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.103+0000 7f3746400640 1 -- start start 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.103+0000 7f3746400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006ed30 0x7f3740072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.103+0000 7f3746400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.103+0000 7f3746400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 0x7f374015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3746400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37400cd9f0 con 0x7f37400cbab0 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3746400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f374015c7e0 con 0x7f374006ed30 2024-07-17T07:10:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3746400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37400cdb70 con 0x7f3740152220 2024-07-17T07:10:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:04.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35120/0 (socket says 172.21.15.44:35120) 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 -- 172.21.15.44:0/1421764550 learned_addr learned my addr 172.21.15.44:0/1421764550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f373f400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006ed30 0x7f3740072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f373fe00640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 0x7f374015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 msgr2=0x7f374015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 0x7f374015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006ed30 msgr2=0x7f3740072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006ed30 0x7f3740072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f3737e00640 1 -- 172.21.15.44:0/1421764550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37400728d0 con 0x7f37400cbab0 2024-07-17T07:10:04.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.104+0000 7f373fe00640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 0x7f374015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:10:04.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3737e00640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f372800c480 tx=0x7f372800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:04.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37280052d0 con 0x7f37400cbab0 2024-07-17T07:10:04.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3728005470 con 0x7f37400cbab0 2024-07-17T07:10:04.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728012870 con 0x7f37400cbab0 2024-07-17T07:10:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3740072b70 con 0x7f37400cbab0 2024-07-17T07:10:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.105+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401f7f70 con 0x7f37400cbab0 2024-07-17T07:10:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.106+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37400ce090 con 0x7f37400cbab0 2024-07-17T07:10:04.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.108+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7f37280035f0 con 0x7f37400cbab0 2024-07-17T07:10:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.108+0000 7f3737400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 0x7f371c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.109+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7f3728098550 con 0x7f37400cbab0 2024-07-17T07:10:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.109+0000 7f373f400640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 msgr2=0x7f371c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.109+0000 7f373f400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 0x7f371c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:04.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.117+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f37280613f0 con 0x7f37400cbab0 2024-07-17T07:10:04.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.311+0000 7f373f400640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 msgr2=0x7f371c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:04.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.311+0000 7f373f400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 0x7f371c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:04.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.344+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f374015be80 con 0x7f37400cbab0 2024-07-17T07:10:04.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.345+0000 7f3737400640 1 -- 172.21.15.44:0/1421764550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v70) ==== 80+0+474 (secure 0 0 0) 0x7f3728060cf0 con 0x7f37400cbab0 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2333555199 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/533048536 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/533048536 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2491988681 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2071475375 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3198186364 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/707652649 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/109925158 2024-07-18T07:08:25.020867+0000 2024-07-17T07:10:04.345 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T07:10:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 msgr2=0x7f371c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 0x7f371c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 msgr2=0x7f37401587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f372800c480 tx=0x7f372800c940 comp rx=0 tx=0).stop 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 shutdown_connections 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f371c076cd0 0x7f371c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740152220 0x7f374015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37400cbab0 0x7f37401587d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 --2- 172.21.15.44:0/1421764550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006ed30 0x7f3740072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.347+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 >> 172.21.15.44:0/1421764550 conn(0x7f37400b0eb0 msgr2=0x7f37400afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.348+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 shutdown_connections 2024-07-17T07:10:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.348+0000 7f3746400640 1 -- 172.21.15.44:0/1421764550 wait complete. 2024-07-17T07:10:04.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:10:04.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:04.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:10:04.446 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:10:04.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:04.446 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:10:04.485 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:10:04.485 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:10:04.485 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:04.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:10:04.525 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:10:04.525 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:04.525 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:04.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:10:04.566 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:10:04.566 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:10:04.566 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:04.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:10:04.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.821+0000 7fde05200640 1 -- 172.21.15.44:0/3806342503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 msgr2=0x7fdde4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.821+0000 7fde05200640 1 --2- 172.21.15.44:0/3806342503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fdde4003ea0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006f80 tx=0x7fdde4005830 comp rx=0 tx=0).stop 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.821+0000 7fddfe800640 1 -- 172.21.15.44:0/3806342503 reap_dead start 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.822+0000 7fde05200640 1 -- 172.21.15.44:0/3806342503 shutdown_connections 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.822+0000 7fde05200640 1 -- 172.21.15.44:0/3806342503 >> 172.21.15.44:0/3806342503 conn(0x7fde000af940 msgr2=0x7fde000afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.822+0000 7fde05200640 1 -- 172.21.15.44:0/3806342503 shutdown_connections 2024-07-17T07:10:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.822+0000 7fde05200640 1 -- 172.21.15.44:0/3806342503 wait complete. 2024-07-17T07:10:04.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.825+0000 7fde05200640 1 Processor -- start 2024-07-17T07:10:04.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.825+0000 7fde05200640 1 -- start start 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.826+0000 7fde05200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.826+0000 7fde05200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde00159a60 0x7fde00159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.826+0000 7fde05200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde0015a420 0x7fde001453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fde05200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fde0006f5e0 con 0x7fdde4003a80 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fde05200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde000cf860 con 0x7fde0015a420 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fde05200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde0006f760 con 0x7fde00159a60 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:04.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35146/0 (socket says 172.21.15.44:35146) 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 -- 172.21.15.44:0/230622195 learned_addr learned my addr 172.21.15.44:0/230622195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfe800640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde0015a420 0x7fde001453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde00159a60 msgr2=0x7fde00159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde00159a60 0x7fde00159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde0015a420 msgr2=0x7fde001453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde0015a420 0x7fde001453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:04.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.827+0000 7fddfde00640 1 -- 172.21.15.44:0/230622195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde00158470 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fddfde00640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fddf002b090 tx=0x7fddf0006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0032030 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf0003af0 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0038430 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde00158750 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.828+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde00158d10 con 0x7fdde4003a80 2024-07-17T07:10:04.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.829+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde000b5510 con 0x7fdde4003a80 2024-07-17T07:10:04.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.831+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7fddf0035020 con 0x7fdde4003a80 2024-07-17T07:10:04.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.832+0000 7fddf6000640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 0x7fdddc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:04.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.833+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7fddf00b8fc0 con 0x7fdde4003a80 2024-07-17T07:10:04.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.833+0000 7fddf7e00640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 msgr2=0x7fdddc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:04.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.833+0000 7fddf7e00640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 0x7fdddc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:04.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:04.837+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fde000b5510 con 0x7fdde4003a80 2024-07-17T07:10:05.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.034+0000 7fddf7e00640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 msgr2=0x7fdddc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:05.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.034+0000 7fddf7e00640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 0x7fdddc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.090+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde0006fcf0 con 0x7fdde4003a80 2024-07-17T07:10:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.091+0000 7fddf6000640 1 -- 172.21.15.44:0/230622195 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fddf0047020 con 0x7fdde4003a80 2024-07-17T07:10:05.091 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:05.091 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:10:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 msgr2=0x7fdddc079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 0x7fdddc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 msgr2=0x7fde00142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fddf002b090 tx=0x7fddf0006ca0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 shutdown_connections 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fdddc076cd0 0x7fdddc079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde0015a420 0x7fde001453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde00159a60 0x7fde00159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 --2- 172.21.15.44:0/230622195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdde4003a80 0x7fde00142c10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 >> 172.21.15.44:0/230622195 conn(0x7fde000af940 msgr2=0x7fde00156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 shutdown_connections 2024-07-17T07:10:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.094+0000 7fde05200640 1 -- 172.21.15.44:0/230622195 wait complete. 2024-07-17T07:10:05.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.306+0000 7f2a65c00640 1 -- 172.21.15.44:0/1644425475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 msgr2=0x7f2a44005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a65c00640 1 --2- 172.21.15.44:0/1644425475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a44005910 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2a48006f10 tx=0x7f2a44006940 comp rx=0 tx=0).stop 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a5f400640 1 -- 172.21.15.44:0/1644425475 reap_dead start 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a65c00640 1 -- 172.21.15.44:0/1644425475 shutdown_connections 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a65c00640 1 -- 172.21.15.44:0/1644425475 >> 172.21.15.44:0/1644425475 conn(0x7f2a6006bf40 msgr2=0x7f2a6006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a65c00640 1 -- 172.21.15.44:0/1644425475 shutdown_connections 2024-07-17T07:10:05.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.307+0000 7f2a65c00640 1 -- 172.21.15.44:0/1644425475 wait complete. 2024-07-17T07:10:05.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.308+0000 7f2a65c00640 1 Processor -- start 2024-07-17T07:10:05.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.309+0000 7f2a65c00640 1 -- start start 2024-07-17T07:10:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60159260 0x7f2a601596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 0x7f2a600c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601541b0 con 0x7f2a44005320 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60154030 con 0x7f2a60159260 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a65c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a60068860 con 0x7f2a600c64d0 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35152/0 (socket says 172.21.15.44:35152) 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 -- 172.21.15.44:0/4267485561 learned_addr learned my addr 172.21.15.44:0/4267485561 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a57e00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60159260 0x7f2a601596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 msgr2=0x7f2a600c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5f400640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 0x7f2a600c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 0x7f2a600c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60159260 msgr2=0x7f2a601596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60159260 0x7f2a601596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5ea00640 1 -- 172.21.15.44:0/4267485561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a60159c20 con 0x7f2a44005320 2024-07-17T07:10:05.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.310+0000 7f2a5f400640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 0x7f2a600c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a5ea00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a50004cc0 tx=0x7f2a50001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50015020 con 0x7f2a44005320 2024-07-17T07:10:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a50011980 con 0x7f2a44005320 2024-07-17T07:10:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a60157930 con 0x7f2a44005320 2024-07-17T07:10:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011dd0 con 0x7f2a44005320 2024-07-17T07:10:05.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.311+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a60157ef0 con 0x7f2a44005320 2024-07-17T07:10:05.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.312+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a60068d70 con 0x7f2a44005320 2024-07-17T07:10:05.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.314+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7f2a50018020 con 0x7f2a44005320 2024-07-17T07:10:05.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.315+0000 7f2a56a00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 0x7f2a3c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.315+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7f2a50099590 con 0x7f2a44005320 2024-07-17T07:10:05.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.315+0000 7f2a57e00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 msgr2=0x7f2a3c0791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:05.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.315+0000 7f2a57e00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 0x7f2a3c0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:05.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.319+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2a5001f020 con 0x7f2a44005320 2024-07-17T07:10:05.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.516+0000 7f2a57e00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 msgr2=0x7f2a3c0791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:05.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.516+0000 7f2a57e00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 0x7f2a3c0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.565+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a60153770 con 0x7f2a44005320 2024-07-17T07:10:05.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.566+0000 7f2a56a00640 1 -- 172.21.15.44:0/4267485561 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7f2a50062290 con 0x7f2a44005320 2024-07-17T07:10:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:05.566 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:10:05.566 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 msgr2=0x7f2a3c0791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 0x7f2a3c0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 msgr2=0x7f2a60152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a50004cc0 tx=0x7f2a50001800 comp rx=0 tx=0).stop 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 shutdown_connections 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f2a3c076d00 0x7f2a3c0791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a600c64d0 0x7f2a600c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a60159260 0x7f2a601596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 --2- 172.21.15.44:0/4267485561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a44005320 0x7f2a60152070 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 >> 172.21.15.44:0/4267485561 conn(0x7f2a6006bf40 msgr2=0x7f2a6014ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:05.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 shutdown_connections 2024-07-17T07:10:05.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.569+0000 7f2a65c00640 1 -- 172.21.15.44:0/4267485561 wait complete. 2024-07-17T07:10:05.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:05.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 -- 172.21.15.44:0/1743959135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba500ac220 msgr2=0x7fba500ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 --2- 172.21.15.44:0/1743959135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba500ac220 0x7fba500ac690 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fba3c005fd0 tx=0x7fba3c02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 -- 172.21.15.44:0/1743959135 shutdown_connections 2024-07-17T07:10:05.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 --2- 172.21.15.44:0/1743959135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500cd690 0x7fba500cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 --2- 172.21.15.44:0/1743959135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba500ac220 0x7fba500ac690 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 -- 172.21.15.44:0/1743959135 >> 172.21.15.44:0/1743959135 conn(0x7fba500b0fb0 msgr2=0x7fba500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:05.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 -- 172.21.15.44:0/1743959135 shutdown_connections 2024-07-17T07:10:05.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.777+0000 7fba57e00640 1 -- 172.21.15.44:0/1743959135 wait complete. 2024-07-17T07:10:05.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.779+0000 7fba57e00640 1 Processor -- start 2024-07-17T07:10:05.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.779+0000 7fba57e00640 1 -- start start 2024-07-17T07:10:05.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500ac220 0x7fba50148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba500cd690 0x7fba501f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba500ce1e0 con 0x7fba501522c0 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba500ce060 con 0x7fba500cd690 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba57e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba500ce360 con 0x7fba500ac220 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.780+0000 7fba55e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35180/0 (socket says 172.21.15.44:35180) 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 -- 172.21.15.44:0/1637599343 learned_addr learned my addr 172.21.15.44:0/1637599343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba54a00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba500cd690 0x7fba501f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55400640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500ac220 0x7fba50148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500ac220 msgr2=0x7fba50148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500ac220 0x7fba50148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba500cd690 msgr2=0x7fba501f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba500cd690 0x7fba501f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 -- 172.21.15.44:0/1637599343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5014d0a0 con 0x7fba501522c0 2024-07-17T07:10:05.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.781+0000 7fba55e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fba4800cab0 tx=0x7fba4800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.782+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4800adc0 con 0x7fba501522c0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.782+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba4800f450 con 0x7fba501522c0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.782+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4800f8c0 con 0x7fba501522c0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.782+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba501f2c70 con 0x7fba501522c0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.782+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba501f1f80 con 0x7fba501522c0 2024-07-17T07:10:05.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.783+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba500ce880 con 0x7fba501522c0 2024-07-17T07:10:05.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.785+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7fba4800fa60 con 0x7fba501522c0 2024-07-17T07:10:05.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.786+0000 7fba4d600640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 0x7fba30079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:05.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.786+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7fba480994f0 con 0x7fba501522c0 2024-07-17T07:10:05.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.786+0000 7fba55400640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 msgr2=0x7fba30079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:05.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.787+0000 7fba55400640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 0x7fba30079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:05.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.790+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fba48014030 con 0x7fba501522c0 2024-07-17T07:10:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.988+0000 7fba55400640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 msgr2=0x7fba30079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.988+0000 7fba55400640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 0x7fba30079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:05.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:05.998+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba500ac9b0 con 0x7fba501522c0 2024-07-17T07:10:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.000+0000 7fba4d600640 1 -- 172.21.15.44:0/1637599343 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18272 (secure 0 0 0) 0x7fba48062280 con 0x7fba501522c0 2024-07-17T07:10:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:06.001 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:09:26.318506+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:08:08.174976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-07-17T07:08:09.173646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.27","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2333555199":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6801/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6800/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/2491988681":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/2071475375":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/3198186364":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/707652649":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/109925158":"2024-07-18T07:08:25.020867+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:06.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.003+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 msgr2=0x7fba30079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.003+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 0x7fba30079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.003+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 msgr2=0x7fba501f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.003+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fba4800cab0 tx=0x7fba4800cf70 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 shutdown_connections 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7fba30076cd0 0x7fba30079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba501522c0 0x7fba501f5740 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba500cd690 0x7fba501f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 --2- 172.21.15.44:0/1637599343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba500ac220 0x7fba50148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 >> 172.21.15.44:0/1637599343 conn(0x7fba500b0fb0 msgr2=0x7fba50150e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 shutdown_connections 2024-07-17T07:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.004+0000 7fba57e00640 1 -- 172.21.15.44:0/1637599343 wait complete. 2024-07-17T07:10:06.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:10:06.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:06.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 -- 172.21.15.44:0/3244071682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b2ad0 msgr2=0x7ff4c40b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 --2- 172.21.15.44:0/3244071682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40b5200 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005fd0 tx=0x7ff4b002a670 comp rx=0 tx=0).stop 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 -- 172.21.15.44:0/3244071682 shutdown_connections 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 --2- 172.21.15.44:0/3244071682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40b5200 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 --2- 172.21.15.44:0/3244071682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 0x7ff4c406e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 -- 172.21.15.44:0/3244071682 >> 172.21.15.44:0/3244071682 conn(0x7ff4c40d00a0 msgr2=0x7ff4c40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.208+0000 7ff4ca800640 1 -- 172.21.15.44:0/3244071682 shutdown_connections 2024-07-17T07:10:06.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.209+0000 7ff4ca800640 1 -- 172.21.15.44:0/3244071682 wait complete. 2024-07-17T07:10:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.210+0000 7ff4ca800640 1 Processor -- start 2024-07-17T07:10:06.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.210+0000 7ff4ca800640 1 -- start start 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 0x7ff4c40b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4161730 con 0x7ff4c40b5740 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c40cef10 con 0x7ff4c40b2ad0 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4ca800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c41618b0 con 0x7ff4c415d600 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.211+0000 7ff4c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35198/0 (socket says 172.21.15.44:35198) 2024-07-17T07:10:06.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 -- 172.21.15.44:0/1836356619 learned_addr learned my addr 172.21.15.44:0/1836356619 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c3400640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c3e00640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 0x7ff4c40b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 msgr2=0x7ff4c40b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 0x7ff4c40b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c40b2ad0 msgr2=0x7ff4c40cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 -- 172.21.15.44:0/1836356619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c40b67c0 con 0x7ff4c40b5740 2024-07-17T07:10:06.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.212+0000 7ff4c2a00640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006010 tx=0x7ff4b00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.213+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b003f070 con 0x7ff4c40b5740 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.213+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b00047a0 con 0x7ff4c40b5740 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.213+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c40b6aa0 con 0x7ff4c40b5740 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.213+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0035030 con 0x7ff4c40b5740 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.213+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c40b7060 con 0x7ff4c40b5740 2024-07-17T07:10:06.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.214+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c4161d60 con 0x7ff4c40b5740 2024-07-17T07:10:06.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.215+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7ff4b0042070 con 0x7ff4c40b5740 2024-07-17T07:10:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.216+0000 7ff4bb400640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 0x7ff4a4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.216+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6995+0+0 (secure 0 0 0) 0x7ff4b00b9860 con 0x7ff4c40b5740 2024-07-17T07:10:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.216+0000 7ff4c3400640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 msgr2=0x7ff4a4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:06.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.216+0000 7ff4c3400640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 0x7ff4a4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.220+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4b0082560 con 0x7ff4c40b5740 2024-07-17T07:10:06.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.417+0000 7ff4c3400640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 msgr2=0x7ff4a4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:06.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.417+0000 7ff4c3400640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 0x7ff4a4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:06.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.470+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c406cae0 con 0x7ff4c40b5740 2024-07-17T07:10:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.471+0000 7ff4bb400640 1 -- 172.21.15.44:0/1836356619 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7ff4b0081e60 con 0x7ff4c40b5740 2024-07-17T07:10:06.471 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:06.471 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:10:06.471 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 msgr2=0x7ff4a4079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 0x7ff4a4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 msgr2=0x7ff4c40b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006010 tx=0x7ff4b00038a0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 shutdown_connections 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff4a4076cd0 0x7ff4a4079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4c415d600 0x7ff4c40b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c40b5740 0x7ff4c40b4230 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 --2- 172.21.15.44:0/1836356619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c40b2ad0 0x7ff4c40cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 >> 172.21.15.44:0/1836356619 conn(0x7ff4c40d00a0 msgr2=0x7ff4c40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 shutdown_connections 2024-07-17T07:10:06.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.474+0000 7ff4ca800640 1 -- 172.21.15.44:0/1836356619 wait complete. 2024-07-17T07:10:06.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.697+0000 7ff6fac00640 1 -- 172.21.15.44:0/3284301588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40b8c80 msgr2=0x7ff6f40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.697+0000 7ff6fac00640 1 --2- 172.21.15.44:0/3284301588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40b8c80 0x7ff6f40b4ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009900 tx=0x7ff6e802a560 comp rx=0 tx=0).stop 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 -- 172.21.15.44:0/3284301588 shutdown_connections 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 --2- 172.21.15.44:0/3284301588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40ce5c0 0x7ff6f406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 --2- 172.21.15.44:0/3284301588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40b8c80 0x7ff6f40b4ec0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 -- 172.21.15.44:0/3284301588 >> 172.21.15.44:0/3284301588 conn(0x7ff6f40cfec0 msgr2=0x7ff6f40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:06.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 -- 172.21.15.44:0/3284301588 shutdown_connections 2024-07-17T07:10:06.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.698+0000 7ff6fac00640 1 -- 172.21.15.44:0/3284301588 wait complete. 2024-07-17T07:10:06.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.700+0000 7ff6fac00640 1 Processor -- start 2024-07-17T07:10:06.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.700+0000 7ff6fac00640 1 -- start start 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f40b5400 0x7ff6f4160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40b8c80 0x7ff6f415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f406e780 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f406e600 con 0x7ff6f40b5400 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.701+0000 7ff6fac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f406e900 con 0x7ff6f40b8c80 2024-07-17T07:10:06.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35210/0 (socket says 172.21.15.44:35210) 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 -- 172.21.15.44:0/4027931614 learned_addr learned my addr 172.21.15.44:0/4027931614 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6f8e00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40b8c80 0x7ff6f415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6f9800640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f40b5400 0x7ff6f4160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40b8c80 msgr2=0x7ff6f415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40b8c80 0x7ff6f415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f40b5400 msgr2=0x7ff6f4160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f40b5400 0x7ff6f4160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 -- 172.21.15.44:0/4027931614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41f4c40 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.702+0000 7ff6fa200640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c780 tx=0x7ff6ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:06.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.703+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec004b60 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.703+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6ec004df0 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.703+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f40ca5b0 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.703+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00aa90 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.703+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f40cab70 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.704+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f415f6a0 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.705+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7ff6ec00ac30 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.706+0000 7ff6f1600640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 0x7ff6d4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:06.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.707+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7051+0+0 (secure 0 0 0) 0x7ff6ec098ef0 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.707+0000 7ff6f9800640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 msgr2=0x7ff6d4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:06.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.707+0000 7ff6f9800640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 0x7ff6d4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.715+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff6ec061d60 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.909+0000 7ff6f9800640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 msgr2=0x7ff6d4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.909+0000 7ff6f9800640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 0x7ff6d4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:06.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.992+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6f40b5d70 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.993+0000 7ff6f1600640 1 -- 172.21.15.44:0/4027931614 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7ff6ec061660 con 0x7ff6f40ce5c0 2024-07-17T07:10:06.993 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:06.993 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"btime":"2024-07-17T07:08:13:760942+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:08:10.204958+0000","modified":"2024-07-17T07:08:13.277327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5823},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5823":{"gid":5823,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2772368935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2772368935},{"type":"v1","addr":"172.21.15.44:6835","nonce":2772368935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5823,"qdb_cluster":[5823]},"id":3}]} 2024-07-17T07:10:06.993 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 26 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 msgr2=0x7ff6d4079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 0x7ff6d4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 msgr2=0x7ff6f415ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c780 tx=0x7ff6ec00cc40 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 shutdown_connections 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7ff6d4076cd0 0x7ff6d4079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6f40ce5c0 0x7ff6f415ac20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f40b8c80 0x7ff6f415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 --2- 172.21.15.44:0/4027931614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f40b5400 0x7ff6f4160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.995+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 >> 172.21.15.44:0/4027931614 conn(0x7ff6f40cfec0 msgr2=0x7ff6f40b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:06.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.996+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 shutdown_connections 2024-07-17T07:10:06.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:06.996+0000 7ff6fac00640 1 -- 172.21.15.44:0/4027931614 wait complete. 2024-07-17T07:10:07.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 -- 172.21.15.44:0/3515296971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 msgr2=0x7f5af006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 --2- 172.21.15.44:0/3515296971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af006e060 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5af009c3c0 tx=0x7f5ae002ae80 comp rx=0 tx=0).stop 2024-07-17T07:10:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 -- 172.21.15.44:0/3515296971 shutdown_connections 2024-07-17T07:10:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 --2- 172.21.15.44:0/3515296971 >> v2:172.21.15.175:3300/0 conn(0x7f5af00cfa90 0x7f5af0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 --2- 172.21.15.44:0/3515296971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af006e060 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 -- 172.21.15.44:0/3515296971 >> 172.21.15.44:0/3515296971 conn(0x7f5af00b0e80 msgr2=0x7f5af00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 -- 172.21.15.44:0/3515296971 shutdown_connections 2024-07-17T07:10:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.193+0000 7f5af4c00640 1 -- 172.21.15.44:0/3515296971 wait complete. 2024-07-17T07:10:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.195+0000 7f5af4c00640 1 Processor -- start 2024-07-17T07:10:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.195+0000 7f5af4c00640 1 -- start start 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af009b6e0 0x7f5af00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af00cfa90 0x7f5af00b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5af00b2120 con 0x7f5af009b300 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5af00b1fa0 con 0x7f5af00cfa90 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5af4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5af00b22a0 con 0x7f5af009b6e0 2024-07-17T07:10:07.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5aef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5aef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35228/0 (socket says 172.21.15.44:35228) 2024-07-17T07:10:07.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5aef400640 1 -- 172.21.15.44:0/1729164285 learned_addr learned my addr 172.21.15.44:0/1729164285 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:07.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.196+0000 7f5ae6a00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af009b6e0 0x7f5af00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aefe00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af00cfa90 0x7f5af00b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af009b6e0 msgr2=0x7f5af00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af009b6e0 0x7f5af00b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af00cfa90 msgr2=0x7f5af00b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af00cfa90 0x7f5af00b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 -- 172.21.15.44:0/1729164285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af00cc970 con 0x7f5af009b300 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5aef400640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b4b0 tx=0x7f5ae002cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0041020 con 0x7f5af009b300 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ae0009660 con 0x7f5af009b300 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00035f0 con 0x7f5af009b300 2024-07-17T07:10:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.197+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01f76b0 con 0x7f5af009b300 2024-07-17T07:10:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.198+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01f7c70 con 0x7f5af009b300 2024-07-17T07:10:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.199+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af01f3d40 con 0x7f5af009b300 2024-07-17T07:10:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.200+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7f5ae0044020 con 0x7f5af009b300 2024-07-17T07:10:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.201+0000 7f5ae7e00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 0x7f5acc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.201+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7051+0+0 (secure 0 0 0) 0x7f5ae00b9560 con 0x7f5af009b300 2024-07-17T07:10:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.202+0000 7f5ae6a00640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 msgr2=0x7f5acc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.202+0000 7f5ae6a00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 0x7f5acc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.206+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5af01f3d40 con 0x7f5af009b300 2024-07-17T07:10:07.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.403+0000 7f5ae6a00640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 msgr2=0x7f5acc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:07.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.403+0000 7f5ae6a00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 0x7f5acc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:07.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.419+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5af006e060 con 0x7f5af009b300 2024-07-17T07:10:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.422+0000 7f5ae7e00640 1 -- 172.21.15.44:0/1729164285 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18381 (secure 0 0 0) 0x7f5ae004b070 con 0x7f5af009b300 2024-07-17T07:10:07.422 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:07.422 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":71,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:10:06.536295+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:08:08.174976+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-07-17T07:08:09.173646+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.27","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2333555199":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6801/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6800/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/2491988681":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/2071475375":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/3198186364":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/707652649":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/109925158":"2024-07-18T07:08:25.020867+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:07.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 msgr2=0x7f5acc079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:07.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 0x7f5acc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 msgr2=0x7f5af00cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b4b0 tx=0x7f5ae002cdf0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 shutdown_connections 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f5acc076cd0 0x7f5acc079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af00cfa90 0x7f5af00b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af009b6e0 0x7f5af00b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 --2- 172.21.15.44:0/1729164285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5af009b300 0x7f5af00cc430 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 >> 172.21.15.44:0/1729164285 conn(0x7f5af00b0e80 msgr2=0x7f5af015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.425+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 shutdown_connections 2024-07-17T07:10:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.426+0000 7f5af4c00640 1 -- 172.21.15.44:0/1729164285 wait complete. 2024-07-17T07:10:07.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 -- 172.21.15.44:0/2712479362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e006f6d0 msgr2=0x7f95e015a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 --2- 172.21.15.44:0/2712479362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e006f6d0 0x7f95e015a5a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f95d0005fd0 tx=0x7f95d002b910 comp rx=0 tx=0).stop 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 -- 172.21.15.44:0/2712479362 shutdown_connections 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 --2- 172.21.15.44:0/2712479362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00b3280 0x7f95e006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 --2- 172.21.15.44:0/2712479362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e006f6d0 0x7f95e015a5a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 -- 172.21.15.44:0/2712479362 >> 172.21.15.44:0/2712479362 conn(0x7f95e00b0eb0 msgr2=0x7f95e00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 -- 172.21.15.44:0/2712479362 shutdown_connections 2024-07-17T07:10:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.637+0000 7f95e5800640 1 -- 172.21.15.44:0/2712479362 wait complete. 2024-07-17T07:10:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.639+0000 7f95e5800640 1 Processor -- start 2024-07-17T07:10:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.640+0000 7f95e5800640 1 -- start start 2024-07-17T07:10:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95e006f6d0 0x7f95e0153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 0x7f95e01f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95e00ac810 con 0x7f95e00b3280 2024-07-17T07:10:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95e0150810 con 0x7f95e006f6d0 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.641+0000 7f95e5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95e00ac990 con 0x7f95e00ac3a0 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35238/0 (socket says 172.21.15.44:35238) 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 -- 172.21.15.44:0/4208487322 learned_addr learned my addr 172.21.15.44:0/4208487322 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95de000640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 0x7f95e01f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95dea00640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95e006f6d0 0x7f95e0153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 -- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 msgr2=0x7f95e01f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 0x7f95e01f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 -- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95e006f6d0 msgr2=0x7f95e0153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95e006f6d0 0x7f95e0153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 -- 172.21.15.44:0/4208487322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e0152380 con 0x7f95e00b3280 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95de000640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 0x7f95e01f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.642+0000 7f95df400640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c7a0 tx=0x7f95d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.643+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003b70 con 0x7f95e00b3280 2024-07-17T07:10:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.643+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95d8003d30 con 0x7f95e00b3280 2024-07-17T07:10:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.643+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800aab0 con 0x7f95e00b3280 2024-07-17T07:10:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.643+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e0152660 con 0x7f95e00b3280 2024-07-17T07:10:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.643+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e0152f00 con 0x7f95e00b3280 2024-07-17T07:10:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.644+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e01530d0 con 0x7f95e00b3280 2024-07-17T07:10:07.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.646+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 98943+0+0 (secure 0 0 0) 0x7f95d800ac50 con 0x7f95e00b3280 2024-07-17T07:10:07.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.647+0000 7f95d6a00640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 0x7f95c0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:07.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.647+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7051+0+0 (secure 0 0 0) 0x7f95d8098850 con 0x7f95e00b3280 2024-07-17T07:10:07.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.647+0000 7f95dea00640 1 -- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 msgr2=0x7f95c0079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1668760217 2024-07-17T07:10:07.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.647+0000 7f95dea00640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 0x7f95c0079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.651+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f95d80616c0 con 0x7f95e00b3280 2024-07-17T07:10:07.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.839+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 27) ==== 98540+0+0 (secure 0 0 0) 0x7f95d805d250 con 0x7f95e00b3280 2024-07-17T07:10:07.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.839+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 msgr2=0x7f95c0079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:07.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.839+0000 7f95d6a00640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 0x7f95c0079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.912+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95e00b4200 con 0x7f95e00b3280 2024-07-17T07:10:07.912 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:10:07.912+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:10:07.914 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.914 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.914 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.914 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.914 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.915 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.915 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.915 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.915 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.915 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:07.914+0000 7f54e1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:07.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.918+0000 7f95d6a00640 1 -- 172.21.15.44:0/4208487322 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f95d806d020 con 0x7f95e00b3280 2024-07-17T07:10:07.918 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 msgr2=0x7f95e01f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f95d800c7a0 tx=0x7f95d800cc60 comp rx=0 tx=0).stop 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 shutdown_connections 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:6800/1668760217,v1:172.21.15.175:6801/1668760217] conn(0x7f95c0076cd0 0x7f95c0079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e00b3280 0x7f95e01f45f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95e00ac3a0 0x7f95e01f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 --2- 172.21.15.44:0/4208487322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95e006f6d0 0x7f95e0153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.920+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 >> 172.21.15.44:0/4208487322 conn(0x7f95e00b0eb0 msgr2=0x7f95e00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.921+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 shutdown_connections 2024-07-17T07:10:07.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:07.921+0000 7f95e5800640 1 -- 172.21.15.44:0/4208487322 wait complete. 2024-07-17T07:10:07.947 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:10:08.009 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:10:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 -- 172.21.15.44:0/2770436522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d8144d70 msgr2=0x7f66d806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 --2- 172.21.15.44:0/2770436522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d8144d70 0x7f66d806ec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f66d8070730 tx=0x7f66c802a560 comp rx=0 tx=0).stop 2024-07-17T07:10:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 -- 172.21.15.44:0/2770436522 shutdown_connections 2024-07-17T07:10:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 --2- 172.21.15.44:0/2770436522 >> v2:172.21.15.175:3300/0 conn(0x7f66d80726f0 0x7f66d806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 --2- 172.21.15.44:0/2770436522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d8144d70 0x7f66d806ec40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 -- 172.21.15.44:0/2770436522 >> 172.21.15.44:0/2770436522 conn(0x7f66d806d050 msgr2=0x7f66d806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 -- 172.21.15.44:0/2770436522 shutdown_connections 2024-07-17T07:10:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.133+0000 7f66de200640 1 -- 172.21.15.44:0/2770436522 wait complete. 2024-07-17T07:10:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.135+0000 7f66de200640 1 Processor -- start 2024-07-17T07:10:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.135+0000 7f66de200640 1 -- start start 2024-07-17T07:10:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d80726f0 0x7f66d813fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66d8144d70 0x7f66d813f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80cff40 con 0x7f66d815eec0 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80d00c0 con 0x7f66d80726f0 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66de200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66d806c480 con 0x7f66d8144d70 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35282/0 (socket says 172.21.15.44:35282) 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.136+0000 7f66d7e00640 1 -- 172.21.15.44:0/1057887882 learned_addr learned my addr 172.21.15.44:0/1057887882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7400640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d80726f0 0x7f66d813fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 -- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66d8144d70 msgr2=0x7f66d813f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66d8144d70 0x7f66d813f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 -- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d80726f0 msgr2=0x7f66d813fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d80726f0 0x7f66d813fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 -- 172.21.15.44:0/1057887882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81e50e0 con 0x7f66d815eec0 2024-07-17T07:10:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66d7e00640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f66d000b990 tx=0x7f66d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0019070 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66d000cc90 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d81e53c0 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00146a0 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.137+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81e5c60 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.138+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d81e5e30 con 0x7f66d815eec0 2024-07-17T07:10:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.140+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 98540+0+0 (secure 0 0 0) 0x7f66d001c070 con 0x7f66d815eec0 2024-07-17T07:10:08.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.141+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7453+0+0 (secure 0 0 0) 0x7f66d00983a0 con 0x7f66d815eec0 2024-07-17T07:10:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.145+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f66d0061500 con 0x7f66d815eec0 2024-07-17T07:10:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.390+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f66d81e5590 con 0x7f66d815eec0 2024-07-17T07:10:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.850+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 28) ==== 98667+0+0 (secure 0 0 0) 0x7f66d005c770 con 0x7f66d815eec0 2024-07-17T07:10:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.851+0000 7f66cf400640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 0x7f66b407f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.853+0000 7f66d7400640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 0x7f66b407f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.857+0000 7f66d7400640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 0x7f66b407f530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66c8033000 tx=0x7f66c8004730 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.921+0000 7f66cf400640 1 -- 172.21.15.44:0/1057887882 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f66d006d370 con 0x7f66d815eec0 2024-07-17T07:10:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 msgr2=0x7f66b407f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 0x7f66b407f530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66c8033000 tx=0x7f66c8004730 comp rx=0 tx=0).stop 2024-07-17T07:10:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 msgr2=0x7f66d8142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f66d000b990 tx=0x7f66d000be50 comp rx=0 tx=0).stop 2024-07-17T07:10:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 shutdown_connections 2024-07-17T07:10:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66d815eec0 0x7f66d8142d80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66d8144d70 0x7f66d813f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f66b407d140 0x7f66b407f530 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 --2- 172.21.15.44:0/1057887882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66d80726f0 0x7f66d813fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 >> 172.21.15.44:0/1057887882 conn(0x7f66d806d050 msgr2=0x7f66d80b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 shutdown_connections 2024-07-17T07:10:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:08.925+0000 7f66de200640 1 -- 172.21.15.44:0/1057887882 wait complete. 2024-07-17T07:10:08.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 -- 172.21.15.44:0/596470456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc0cb880 msgr2=0x7faccc06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 --2- 172.21.15.44:0/596470456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc0cb880 0x7faccc06db60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faccc06eca0 tx=0x7facb402a4b0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 -- 172.21.15.44:0/596470456 shutdown_connections 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 --2- 172.21.15.44:0/596470456 >> v2:172.21.15.175:3300/0 conn(0x7faccc15c220 0x7faccc15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 --2- 172.21.15.44:0/596470456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc0cb880 0x7faccc06db60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 --2- 172.21.15.44:0/596470456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc15b060 0x7faccc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 -- 172.21.15.44:0/596470456 >> 172.21.15.44:0/596470456 conn(0x7faccc0b0eb0 msgr2=0x7faccc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 -- 172.21.15.44:0/596470456 shutdown_connections 2024-07-17T07:10:09.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.144+0000 7facd2000640 1 -- 172.21.15.44:0/596470456 wait complete. 2024-07-17T07:10:09.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.146+0000 7facd2000640 1 Processor -- start 2024-07-17T07:10:09.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.146+0000 7facd2000640 1 -- start start 2024-07-17T07:10:09.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc0cb880 0x7faccc0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:09.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc15c220 0x7faccc158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faccc15f2d0 con 0x7faccc15b060 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faccc15b690 con 0x7faccc15c220 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7facd2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faccc15f450 con 0x7faccc0cb880 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7faccaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7faccaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48020/0 (socket says 172.21.15.44:48020) 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7faccaa00640 1 -- 172.21.15.44:0/3809721573 learned_addr learned my addr 172.21.15.44:0/3809721573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7faccbe00640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc15c220 0x7faccc158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.147+0000 7faccb400640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc0cb880 0x7faccc0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 -- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc0cb880 msgr2=0x7faccc0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc0cb880 0x7faccc0cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 -- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc15c220 msgr2=0x7faccc158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc15c220 0x7faccc158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 -- 172.21.15.44:0/3809721573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc156eb0 con 0x7faccc15b060 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7faccaa00640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7facb4005fd0 tx=0x7facb40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb403f070 con 0x7faccc15b060 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb4004890 con 0x7faccc15b060 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.148+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc157190 con 0x7faccc15b060 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.149+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4035030 con 0x7faccc15b060 2024-07-17T07:10:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.149+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc156b30 con 0x7faccc15b060 2024-07-17T07:10:09.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.150+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faccc15f920 con 0x7faccc15b060 2024-07-17T07:10:09.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.151+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) ==== 98667+0+0 (secure 0 0 0) 0x7facb4042070 con 0x7faccc15b060 2024-07-17T07:10:09.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.152+0000 7facc3400640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 0x7faca8078ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:09.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.153+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7453+0+0 (secure 0 0 0) 0x7facb40b9410 con 0x7faccc15b060 2024-07-17T07:10:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.153+0000 7faccb400640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 0x7faca8078ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:09.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.154+0000 7faccb400640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 0x7faca8078ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7facbc005fd0 tx=0x7facbc005850 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.157+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7facb4082480 con 0x7faccc15b060 2024-07-17T07:10:09.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.370+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faccc0c9400 con 0x7faccc15b060 2024-07-17T07:10:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.925+0000 7facc3400640 1 -- 172.21.15.44:0/3809721573 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7facb4081d80 con 0x7faccc15b060 2024-07-17T07:10:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 msgr2=0x7faca8078ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 0x7faca8078ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7facbc005fd0 tx=0x7facbc005850 comp rx=0 tx=0).stop 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 msgr2=0x7faccc1f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7facb4005fd0 tx=0x7facb40039e0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 shutdown_connections 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7faca8076a30 0x7faca8078ef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faccc15c220 0x7faccc158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faccc15b060 0x7faccc1f3100 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 --2- 172.21.15.44:0/3809721573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faccc0cb880 0x7faccc0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:09.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.931+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 >> 172.21.15.44:0/3809721573 conn(0x7faccc0b0eb0 msgr2=0x7faccc0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:09.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.932+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 shutdown_connections 2024-07-17T07:10:09.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:09.932+0000 7facd2000640 1 -- 172.21.15.44:0/3809721573 wait complete. 2024-07-17T07:10:09.959 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:10:10.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 -- 172.21.15.44:0/2156566767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 msgr2=0x7f500c0ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:10.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 --2- 172.21.15.44:0/2156566767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0ce500 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005fd0 tx=0x7f4ffc02a8d0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 -- 172.21.15.44:0/2156566767 shutdown_connections 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 --2- 172.21.15.44:0/2156566767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 --2- 172.21.15.44:0/2156566767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0ce500 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 -- 172.21.15.44:0/2156566767 >> 172.21.15.44:0/2156566767 conn(0x7f500c073a60 msgr2=0x7f500c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 -- 172.21.15.44:0/2156566767 shutdown_connections 2024-07-17T07:10:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.291+0000 7f5010c00640 1 -- 172.21.15.44:0/2156566767 wait complete. 2024-07-17T07:10:10.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.293+0000 7f5010c00640 1 Processor -- start 2024-07-17T07:10:10.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.293+0000 7f5010c00640 1 -- start start 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f500c0d0970 0x7f500c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c14f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f500c15e8d0 con 0x7f500c15e370 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f500c15e750 con 0x7f500c0d0970 2024-07-17T07:10:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.294+0000 7f5010c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f500c0b8d80 con 0x7f500c161b50 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48040/0 (socket says 172.21.15.44:48040) 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 -- 172.21.15.44:0/3322844806 learned_addr learned my addr 172.21.15.44:0/3322844806 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500b400640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f500c0d0970 0x7f500c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500be00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c14f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 -- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 msgr2=0x7f500c14f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c14f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 -- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f500c0d0970 msgr2=0x7f500c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f500c0d0970 0x7f500c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500aa00640 1 -- 172.21.15.44:0/3322844806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c1f74d0 con 0x7f500c15e370 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.295+0000 7f500be00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c14f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f500aa00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b410 tx=0x7f4ff400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4015020 con 0x7f500c15e370 2024-07-17T07:10:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff4011940 con 0x7f500c15e370 2024-07-17T07:10:10.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4011d90 con 0x7f500c15e370 2024-07-17T07:10:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c1f7770 con 0x7f500c15e370 2024-07-17T07:10:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.296+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c1f7d30 con 0x7f500c15e370 2024-07-17T07:10:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.297+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c0b91f0 con 0x7f500c15e370 2024-07-17T07:10:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.298+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98773+0+0 (secure 0 0 0) 0x7f4ff4018020 con 0x7f500c15e370 2024-07-17T07:10:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.299+0000 7f5003400640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 0x7f4fe8078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.300+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7015+0+0 (secure 0 0 0) 0x7f4ff40986b0 con 0x7f500c15e370 2024-07-17T07:10:10.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.300+0000 7f500b400640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 0x7f4fe8078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.301+0000 7f500b400640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 0x7f4fe8078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc035000 tx=0x7f4ffc004ab0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.304+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4ff401f020 con 0x7f500c15e370 2024-07-17T07:10:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.515+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f500c0d1090 con 0x7f500c15e370 2024-07-17T07:10:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.941+0000 7f5003400640 1 -- 172.21.15.44:0/3322844806 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f4ff4061730 con 0x7f500c15e370 2024-07-17T07:10:10.941 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:10:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.946+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 msgr2=0x7f4fe8078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.946+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 0x7f4fe8078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc035000 tx=0x7f4ffc004ab0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.946+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 msgr2=0x7f500c0c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:10.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.946+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b410 tx=0x7f4ff400b8d0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 shutdown_connections 2024-07-17T07:10:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f500c161b50 0x7f500c14f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f500c15e370 0x7f500c0c91d0 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b410 tx=0x7f4ff400b8d0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4fe8076b20 0x7f4fe8078fe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 --2- 172.21.15.44:0/3322844806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f500c0d0970 0x7f500c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 >> 172.21.15.44:0/3322844806 conn(0x7f500c073a60 msgr2=0x7f500c0d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 shutdown_connections 2024-07-17T07:10:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:10.947+0000 7f5010c00640 1 -- 172.21.15.44:0/3322844806 wait complete. 2024-07-17T07:10:10.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:10:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 -- 172.21.15.44:0/1741159372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0b3280 msgr2=0x7f621c06eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 --2- 172.21.15.44:0/1741159372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0b3280 0x7f621c06eb90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f621c0aedf0 tx=0x7f621402b910 comp rx=0 tx=0).stop 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 -- 172.21.15.44:0/1741159372 shutdown_connections 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 --2- 172.21.15.44:0/1741159372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0b3280 0x7f621c06eb90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 --2- 172.21.15.44:0/1741159372 >> v2:172.21.15.175:3300/0 conn(0x7f621c0acab0 0x7f621c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 -- 172.21.15.44:0/1741159372 >> 172.21.15.44:0/1741159372 conn(0x7f621c0b0eb0 msgr2=0x7f621c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 -- 172.21.15.44:0/1741159372 shutdown_connections 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.167+0000 7f6222600640 1 -- 172.21.15.44:0/1741159372 wait complete. 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.169+0000 7f6222600640 1 Processor -- start 2024-07-17T07:10:11.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.169+0000 7f6222600640 1 -- start start 2024-07-17T07:10:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06f6d0 0x7f621c073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c0b3280 0x7f621c1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f621c0aea00 con 0x7f621c0acab0 2024-07-17T07:10:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f621c0ae880 con 0x7f621c06f6d0 2024-07-17T07:10:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6222600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f621c0aeb80 con 0x7f621c0b3280 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6213e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6213e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48046/0 (socket says 172.21.15.44:48046) 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f6213e00640 1 -- 172.21.15.44:0/405386076 learned_addr learned my addr 172.21.15.44:0/405386076 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f621b400640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06f6d0 0x7f621c073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.170+0000 7f621be00640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c0b3280 0x7f621c1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 -- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c0b3280 msgr2=0x7f621c1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c0b3280 0x7f621c1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 -- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06f6d0 msgr2=0x7f621c073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06f6d0 0x7f621c073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 -- 172.21.15.44:0/405386076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c072530 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213e00640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f620400c8c0 tx=0x7f620400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204004220 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62040079b0 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c072800 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.171+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204007de0 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.172+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c072dc0 con 0x7f621c0acab0 2024-07-17T07:10:11.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.173+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c15a5a0 con 0x7f621c0acab0 2024-07-17T07:10:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.174+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98773+0+0 (secure 0 0 0) 0x7f6204005020 con 0x7f621c0acab0 2024-07-17T07:10:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.175+0000 7f6213400640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 0x7f61f8078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.175+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f6204098930 con 0x7f621c0acab0 2024-07-17T07:10:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.176+0000 7f621b400640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 0x7f61f8078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.177+0000 7f621b400640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 0x7f61f8078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f621c1f1cb0 tx=0x7f620c009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.179+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6204014030 con 0x7f621c0acab0 2024-07-17T07:10:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.395+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f621c0b4200 con 0x7f621c0acab0 2024-07-17T07:10:11.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.404+0000 7f6213400640 1 -- 172.21.15.44:0/405386076 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f6204061b90 con 0x7f621c0acab0 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.406+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 msgr2=0x7f61f8078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.406+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 0x7f61f8078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f621c1f1cb0 tx=0x7f620c009040 comp rx=0 tx=0).stop 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.406+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 msgr2=0x7f621c071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.406+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f620400c8c0 tx=0x7f620400cd80 comp rx=0 tx=0).stop 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 shutdown_connections 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c0b3280 0x7f621c1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621c0acab0 0x7f621c071ff0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f61f8076b20 0x7f61f8078fe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 --2- 172.21.15.44:0/405386076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06f6d0 0x7f621c073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 >> 172.21.15.44:0/405386076 conn(0x7f621c0b0eb0 msgr2=0x7f621c0cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 shutdown_connections 2024-07-17T07:10:11.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.407+0000 7f6222600640 1 -- 172.21.15.44:0/405386076 wait complete. 2024-07-17T07:10:11.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.638+0000 7fb797200640 1 -- 172.21.15.44:0/1847294932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b8c80 msgr2=0x7fb7980b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.638+0000 7fb797200640 1 --2- 172.21.15.44:0/1847294932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b8c80 0x7fb7980b4ec0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb788005fd0 tx=0x7fb78802a580 comp rx=0 tx=0).stop 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 -- 172.21.15.44:0/1847294932 shutdown_connections 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 --2- 172.21.15.44:0/1847294932 >> v2:172.21.15.175:3300/0 conn(0x7fb7980b5400 0x7fb79815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 --2- 172.21.15.44:0/1847294932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b8c80 0x7fb7980b4ec0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 -- 172.21.15.44:0/1847294932 >> 172.21.15.44:0/1847294932 conn(0x7fb7980cfec0 msgr2=0x7fb7980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 -- 172.21.15.44:0/1847294932 shutdown_connections 2024-07-17T07:10:11.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.639+0000 7fb797200640 1 -- 172.21.15.44:0/1847294932 wait complete. 2024-07-17T07:10:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.640+0000 7fb797200640 1 Processor -- start 2024-07-17T07:10:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 -- start start 2024-07-17T07:10:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7980b8c80 0x7fb7981596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 0x7fb798159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb79806e0c0 con 0x7fb7980b5400 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7980b9380 con 0x7fb7980b8c80 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.641+0000 7fb797200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb79806e240 con 0x7fb7980ce5c0 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48064/0 (socket says 172.21.15.44:48064) 2024-07-17T07:10:11.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 -- 172.21.15.44:0/3469544237 learned_addr learned my addr 172.21.15.44:0/3469544237 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb796800640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 0x7fb798159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795400640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7980b8c80 0x7fb7981596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 -- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 msgr2=0x7fb798159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 0x7fb798159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 -- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7980b8c80 msgr2=0x7fb7981596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7980b8c80 0x7fb7981596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb795e00640 1 -- 172.21.15.44:0/3469544237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7980cbb40 con 0x7fb7980b5400 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.642+0000 7fb796800640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 0x7fb798159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb795e00640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb788006010 tx=0x7fb7880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78803f070 con 0x7fb7980b5400 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb788004890 con 0x7fb7980b5400 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7980cbe20 con 0x7fb7980b5400 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788035030 con 0x7fb7980b5400 2024-07-17T07:10:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.643+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7980cc3e0 con 0x7fb7980b5400 2024-07-17T07:10:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.644+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79806e730 con 0x7fb7980b5400 2024-07-17T07:10:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.646+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) ==== 98773+0+0 (secure 0 0 0) 0x7fb788042070 con 0x7fb7980b5400 2024-07-17T07:10:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.647+0000 7fb78e000640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 0x7fb778078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.647+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7fb7880b93f0 con 0x7fb7980b5400 2024-07-17T07:10:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.647+0000 7fb795400640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 0x7fb778078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:11.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.649+0000 7fb795400640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 0x7fb778078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb79815fc90 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.651+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb788082650 con 0x7fb7980b5400 2024-07-17T07:10:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:11.942+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) -- 0x7fb7980b5d70 con 0x7fb7980b5400 2024-07-17T07:10:12.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.418+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fb78807e1e0 con 0x7fb7980b5400 2024-07-17T07:10:12.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.947+0000 7fb78e000640 1 -- 172.21.15.44:0/3469544237 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v561) ==== 154+0+0 (secure 0 0 0) 0x7fb7880b3a50 con 0x7fb7980b5400 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.950+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 msgr2=0x7fb778078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.950+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 0x7fb778078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb79815fc90 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.950+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 msgr2=0x7fb7980cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.950+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb788006010 tx=0x7fb7880039e0 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 shutdown_connections 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7980ce5c0 0x7fb798159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fb778076b20 0x7fb778078fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7980b8c80 0x7fb7981596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 --2- 172.21.15.44:0/3469544237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7980b5400 0x7fb7980cb560 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 >> 172.21.15.44:0/3469544237 conn(0x7fb7980cfec0 msgr2=0x7fb7980b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 shutdown_connections 2024-07-17T07:10:12.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:12.951+0000 7fb797200640 1 -- 172.21.15.44:0/3469544237 wait complete. 2024-07-17T07:10:12.973 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2024-07-17T07:10:12.973 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2024-07-17T07:10:12.974 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:10:12.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 -- 172.21.15.44:0/4027076543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8005910 msgr2=0x7f94a8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 --2- 172.21.15.44:0/4027076543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8005910 0x7f94a8005d90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c4a0 tx=0x7f94a8006940 comp rx=0 tx=0).stop 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c5800640 1 -- 172.21.15.44:0/4027076543 reap_dead start 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 -- 172.21.15.44:0/4027076543 shutdown_connections 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 -- 172.21.15.44:0/4027076543 >> 172.21.15.44:0/4027076543 conn(0x7f94c00b0fb0 msgr2=0x7f94c00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:13.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 -- 172.21.15.44:0/4027076543 shutdown_connections 2024-07-17T07:10:13.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.191+0000 7f94c7800640 1 -- 172.21.15.44:0/4027076543 wait complete. 2024-07-17T07:10:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.193+0000 7f94c7800640 1 Processor -- start 2024-07-17T07:10:13.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.193+0000 7f94c7800640 1 -- start start 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 0x7f94c0098520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94c0150920 0x7f94c0151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94c006f0a0 con 0x7f94c00b3880 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00acc70 con 0x7f94a8005910 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94c00ae800 con 0x7f94c0150920 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48086/0 (socket says 172.21.15.44:48086) 2024-07-17T07:10:13.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.194+0000 7f94c5800640 1 -- 172.21.15.44:0/1621681502 learned_addr learned my addr 172.21.15.44:0/1621681502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94bfe00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94c0150920 0x7f94c0151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c4e00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 0x7f94c0098520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 -- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94c0150920 msgr2=0x7f94c0151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94c0150920 0x7f94c0151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 -- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 msgr2=0x7f94c0098520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 0x7f94c0098520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 -- 172.21.15.44:0/1621681502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c0150e00 con 0x7f94c00b3880 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c4e00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 0x7f94c0098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.195+0000 7f94c5800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f94c00d0340 tx=0x7f94b80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.196+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b80107d0 con 0x7f94c00b3880 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.196+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94b8016070 con 0x7f94c00b3880 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.196+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8010d00 con 0x7f94c00b3880 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.196+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c01510e0 con 0x7f94c00b3880 2024-07-17T07:10:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.196+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01516a0 con 0x7f94c00b3880 2024-07-17T07:10:13.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.197+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c0153a50 con 0x7f94c00b3880 2024-07-17T07:10:13.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.198+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f94b801b460 con 0x7f94c00b3880 2024-07-17T07:10:13.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.199+0000 7f94bcc00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 0x7f94a40790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.199+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f94b8098c70 con 0x7f94c00b3880 2024-07-17T07:10:13.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.200+0000 7f94c4e00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 0x7f94a40790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.201+0000 7f94c4e00640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 0x7f94a40790f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009b60 tx=0x7f94b4005e70 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:13.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.203+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f94b8061de0 con 0x7f94c00b3880 2024-07-17T07:10:13.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.412+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94c01e6fd0 con 0x7f94c00b3880 2024-07-17T07:10:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.414+0000 7f94bcc00640 1 -- 172.21.15.44:0/1621681502 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+15080 (secure 0 0 0) 0x7f94b80616e0 con 0x7f94c00b3880 2024-07-17T07:10:13.414 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:13.414 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:10:10.938603+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/2772368935":"2024-07-18T07:10:07.914081+0000","172.21.15.175:6801/1668760217":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/954638229":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/455290005":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/2612450616":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/1230248632":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/2333555199":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6801/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.44:6833/2772368935":"2024-07-18T07:10:07.914081+0000","172.21.15.175:0/4117350255":"2024-07-18T07:10:07.825598+0000","172.21.15.175:6800/533048536":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/2491988681":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/812108565":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/2071475375":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/3198186364":"2024-07-18T07:08:25.020867+0000","172.21.15.175:6800/1668760217":"2024-07-18T07:10:07.825598+0000","172.21.15.175:0/707652649":"2024-07-18T07:08:25.020867+0000","172.21.15.175:0/109925158":"2024-07-18T07:08:25.020867+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 msgr2=0x7f94a40790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 0x7f94a40790f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009b60 tx=0x7f94b4005e70 comp rx=0 tx=0).stop 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 msgr2=0x7f94c00b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f94c00d0340 tx=0x7f94b80019b0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 shutdown_connections 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94c00b3880 0x7f94c00b3d10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94c0150920 0x7f94c0151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f94a4076c30 0x7f94a40790f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 --2- 172.21.15.44:0/1621681502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a8005910 0x7f94c0098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 >> 172.21.15.44:0/1621681502 conn(0x7f94c00b0fb0 msgr2=0x7f94c00b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 shutdown_connections 2024-07-17T07:10:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.417+0000 7f94c7800640 1 -- 172.21.15.44:0/1621681502 wait complete. 2024-07-17T07:10:13.437 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:10:13.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:10:13.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 -- 172.21.15.44:0/24903346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 msgr2=0x7f423800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 --2- 172.21.15.44:0/24903346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f423800cad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f424c005fd0 tx=0x7f4238006940 comp rx=0 tx=0).stop 2024-07-17T07:10:13.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 -- 172.21.15.44:0/24903346 shutdown_connections 2024-07-17T07:10:13.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 --2- 172.21.15.44:0/24903346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f423800cad0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 -- 172.21.15.44:0/24903346 >> 172.21.15.44:0/24903346 conn(0x7f42500cf0d0 msgr2=0x7f42500cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:13.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 -- 172.21.15.44:0/24903346 shutdown_connections 2024-07-17T07:10:13.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.655+0000 7f4257200640 1 -- 172.21.15.44:0/24903346 wait complete. 2024-07-17T07:10:13.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.657+0000 7f4257200640 1 Processor -- start 2024-07-17T07:10:13.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.657+0000 7f4257200640 1 -- start start 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 0x7f425009a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42501f2c10 0x7f42501f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f425006dc20 con 0x7f42380063f0 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f425006daa0 con 0x7f42501f2c10 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4257200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f425006dda0 con 0x7f4250149920 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4255e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4255e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48104/0 (socket says 172.21.15.44:48104) 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.658+0000 7f4255e00640 1 -- 172.21.15.44:0/476765370 learned_addr learned my addr 172.21.15.44:0/476765370 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255400640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 0x7f425009a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 -- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 msgr2=0x7f425009a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 0x7f425009a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 -- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42501f2c10 msgr2=0x7f42501f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42501f2c10 0x7f42501f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 -- 172.21.15.44:0/476765370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4250099500 con 0x7f42380063f0 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255400640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 0x7f425009a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4255e00640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f424400e5c0 tx=0x7f424400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244015070 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f424400ca60 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244011460 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.659+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425009caf0 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.660+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425009d0b0 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.661+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425009c900 con 0x7f42380063f0 2024-07-17T07:10:13.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.662+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f4244018070 con 0x7f42380063f0 2024-07-17T07:10:13.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.663+0000 7f424a000640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 0x7f4234079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.663+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f4244098a70 con 0x7f42380063f0 2024-07-17T07:10:13.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.664+0000 7f4255400640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 0x7f4234079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:13.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.665+0000 7f4255400640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 0x7f4234079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f425014f850 tx=0x7f4240006210 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:13.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.667+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4244061bf0 con 0x7f42380063f0 2024-07-17T07:10:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.880+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42500ae5c0 con 0x7f42380063f0 2024-07-17T07:10:13.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.881+0000 7f424a000640 1 -- 172.21.15.44:0/476765370 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f42440614f0 con 0x7f42380063f0 2024-07-17T07:10:13.881 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 msgr2=0x7f4234079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 0x7f4234079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f425014f850 tx=0x7f4240006210 comp rx=0 tx=0).stop 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 msgr2=0x7f42501588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f424400e5c0 tx=0x7f424400ea80 comp rx=0 tx=0).stop 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 shutdown_connections 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4234076bb0 0x7f4234079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42501f2c10 0x7f42501f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4250149920 0x7f425009a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 --2- 172.21.15.44:0/476765370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42380063f0 0x7f42501588c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:13.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 >> 172.21.15.44:0/476765370 conn(0x7f42500cf0d0 msgr2=0x7f425006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:13.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 shutdown_connections 2024-07-17T07:10:13.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:13.884+0000 7f4257200640 1 -- 172.21.15.44:0/476765370 wait complete. 2024-07-17T07:10:13.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 -- 172.21.15.44:0/4233543076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec003a70 msgr2=0x7effec003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 --2- 172.21.15.44:0/4233543076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec003a70 0x7effec003e90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efff800c290 tx=0x7effec005830 comp rx=0 tx=0).stop 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007400640 1 -- 172.21.15.44:0/4233543076 reap_dead start 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 -- 172.21.15.44:0/4233543076 shutdown_connections 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 -- 172.21.15.44:0/4233543076 >> 172.21.15.44:0/4233543076 conn(0x7f00080b0e80 msgr2=0x7f00080b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 -- 172.21.15.44:0/4233543076 shutdown_connections 2024-07-17T07:10:14.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.115+0000 7f0007e00640 1 -- 172.21.15.44:0/4233543076 wait complete. 2024-07-17T07:10:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.117+0000 7f0007e00640 1 Processor -- start 2024-07-17T07:10:14.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.117+0000 7f0007e00640 1 -- start start 2024-07-17T07:10:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 0x7f00080ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00080cde00 0x7f00081f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f000815a020 con 0x7f00080cac60 2024-07-17T07:10:14.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0008159ea0 con 0x7f00080cde00 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f000815a1a0 con 0x7effec003a70 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48112/0 (socket says 172.21.15.44:48112) 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 -- 172.21.15.44:0/1981460543 learned_addr learned my addr 172.21.15.44:0/1981460543 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006a00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 0x7f00080ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0007400640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00080cde00 0x7f00081f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 -- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 msgr2=0x7f00080ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 0x7f00080ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.118+0000 7f0006000640 1 -- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00080cde00 msgr2=0x7f00081f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0006000640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00080cde00 0x7f00081f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0006000640 1 -- 172.21.15.44:0/1981460543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00080ce2b0 con 0x7f00080cac60 2024-07-17T07:10:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0006a00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 0x7f00080ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:14.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0006000640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efff400d770 tx=0x7efff400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:14.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4017070 con 0x7f00080cac60 2024-07-17T07:10:14.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff4009d20 con 0x7f00080cac60 2024-07-17T07:10:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff40106c0 con 0x7f00080cac60 2024-07-17T07:10:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000814a6e0 con 0x7f00080cac60 2024-07-17T07:10:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.119+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0008071610 con 0x7f00080cac60 2024-07-17T07:10:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.121+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0008071420 con 0x7f00080cac60 2024-07-17T07:10:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.122+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7efff401a070 con 0x7f00080cac60 2024-07-17T07:10:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.123+0000 7efffea00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 0x7effe8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.123+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7efff40987a0 con 0x7f00080cac60 2024-07-17T07:10:14.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.123+0000 7f0006a00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 0x7effe8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.125+0000 7f0006a00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 0x7effe8079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efff800c290 tx=0x7f000815a770 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:14.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.127+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efff4061890 con 0x7f00080cac60 2024-07-17T07:10:14.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.415+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) -- 0x7f000806e060 con 0x7f00080cac60 2024-07-17T07:10:14.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.960+0000 7efffea00640 1 -- 172.21.15.44:0/1981460543 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v563) ==== 159+0+0 (secure 0 0 0) 0x7efff4061190 con 0x7f00080cac60 2024-07-17T07:10:14.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 msgr2=0x7effe8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:14.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 0x7effe8079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efff800c290 tx=0x7f000815a770 comp rx=0 tx=0).stop 2024-07-17T07:10:14.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 msgr2=0x7f00080cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:14.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efff400d770 tx=0x7efff400dc30 comp rx=0 tx=0).stop 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 shutdown_connections 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7effe8076bb0 0x7effe8079070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00080cde00 0x7f00081f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00080cac60 0x7f00080cd8c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 --2- 172.21.15.44:0/1981460543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec003a70 0x7f00080ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 >> 172.21.15.44:0/1981460543 conn(0x7f00080b0e80 msgr2=0x7f00080adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 shutdown_connections 2024-07-17T07:10:14.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:14.965+0000 7f0007e00640 1 -- 172.21.15.44:0/1981460543 wait complete. 2024-07-17T07:10:14.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:10:15.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:10:15.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:10:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 -- 172.21.15.44:0/1541910246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 msgr2=0x7f75d00ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 --2- 172.21.15.44:0/1541910246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d00ce240 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f75bc005fd0 tx=0x7f75bc02a5c0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 -- 172.21.15.44:0/1541910246 shutdown_connections 2024-07-17T07:10:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 --2- 172.21.15.44:0/1541910246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d00ce240 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 --2- 172.21.15.44:0/1541910246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d01579d0 0x7f75d00cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 -- 172.21.15.44:0/1541910246 >> 172.21.15.44:0/1541910246 conn(0x7f75d006c390 msgr2=0x7f75d006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 -- 172.21.15.44:0/1541910246 shutdown_connections 2024-07-17T07:10:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.207+0000 7f75d4a00640 1 -- 172.21.15.44:0/1541910246 wait complete. 2024-07-17T07:10:15.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.209+0000 7f75d4a00640 1 Processor -- start 2024-07-17T07:10:15.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.209+0000 7f75d4a00640 1 -- start start 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 0x7f75d00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d0160d40 0x7f75d01f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00d1800 con 0x7f75d015e3d0 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00d1680 con 0x7f75d01579d0 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.210+0000 7f75d4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75d00d1980 con 0x7f75d0160d40 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48140/0 (socket says 172.21.15.44:48140) 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 -- 172.21.15.44:0/3222592425 learned_addr learned my addr 172.21.15.44:0/3222592425 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cf400640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 0x7f75d00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cfe00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d0160d40 0x7f75d01f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 -- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d0160d40 msgr2=0x7f75d01f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d0160d40 0x7f75d01f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 -- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 msgr2=0x7f75d00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 0x7f75d00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cea00640 1 -- 172.21.15.44:0/3222592425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d00cb950 con 0x7f75d015e3d0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.211+0000 7f75cf400640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 0x7f75d00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75cea00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009850 tx=0x7f75bc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc03f070 con 0x7f75d015e3d0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75bc004890 con 0x7f75d015e3d0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d00ca8b0 con 0x7f75d015e3d0 2024-07-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc035030 con 0x7f75d015e3d0 2024-07-17T07:10:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.212+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d00cae70 con 0x7f75d015e3d0 2024-07-17T07:10:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.213+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00b5860 con 0x7f75d015e3d0 2024-07-17T07:10:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.215+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f75bc042070 con 0x7f75d015e3d0 2024-07-17T07:10:15.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.216+0000 7f75c7400640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 0x7f75b0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.216+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f75bc0b94e0 con 0x7f75d015e3d0 2024-07-17T07:10:15.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.216+0000 7f75cf400640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 0x7f75b0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.218+0000 7f75cf400640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 0x7f75b0079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75c00058c0 tx=0x7f75c0005ec0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:15.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.220+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f75bc082660 con 0x7f75d015e3d0 2024-07-17T07:10:15.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.433+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f75d015fca0 con 0x7f75d015e3d0 2024-07-17T07:10:15.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.434+0000 7f75c7400640 1 -- 172.21.15.44:0/3222592425 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7f75bc081f60 con 0x7f75d015e3d0 2024-07-17T07:10:15.434 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:10:15.434 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:10:15.434 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:10:15.434 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:10:15.435 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:10:15.436 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 msgr2=0x7f75b0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 0x7f75b0079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75c00058c0 tx=0x7f75c0005ec0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 msgr2=0x7f75d01f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009850 tx=0x7f75bc0039e0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 shutdown_connections 2024-07-17T07:10:15.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75d0160d40 0x7f75d01f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75d015e3d0 0x7f75d01f4660 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f75b0076bb0 0x7f75b0079070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 --2- 172.21.15.44:0/3222592425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75d01579d0 0x7f75d00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 >> 172.21.15.44:0/3222592425 conn(0x7f75d006c390 msgr2=0x7f75d006c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 shutdown_connections 2024-07-17T07:10:15.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.438+0000 7f75d4a00640 1 -- 172.21.15.44:0/3222592425 wait complete. 2024-07-17T07:10:15.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.725+0000 7f11bf400640 1 -- 172.21.15.44:0/4103183375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11a0004d40 msgr2=0x7f11a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.725+0000 7f11bf400640 1 --2- 172.21.15.44:0/4103183375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11a0004d40 0x7f11a00051c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009e20 tx=0x7f11a0005830 comp rx=0 tx=0).stop 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.725+0000 7f11bd400640 1 -- 172.21.15.44:0/4103183375 reap_dead start 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.725+0000 7f11bf400640 1 -- 172.21.15.44:0/4103183375 shutdown_connections 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.725+0000 7f11bf400640 1 -- 172.21.15.44:0/4103183375 >> 172.21.15.44:0/4103183375 conn(0x7f11b80b0eb0 msgr2=0x7f11b80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.726+0000 7f11bf400640 1 -- 172.21.15.44:0/4103183375 shutdown_connections 2024-07-17T07:10:15.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.726+0000 7f11bf400640 1 -- 172.21.15.44:0/4103183375 wait complete. 2024-07-17T07:10:15.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.727+0000 7f11bf400640 1 Processor -- start 2024-07-17T07:10:15.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.728+0000 7f11bf400640 1 -- start start 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.728+0000 7f11bf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 0x7f11b8156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.728+0000 7f11bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11b81f5350 0x7f11b80c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bf400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11b806f0b0 con 0x7f11b80ca620 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11b806f8e0 con 0x7f11b81f5350 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11b80b3280 con 0x7f11a0004d40 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48154/0 (socket says 172.21.15.44:48154) 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 -- 172.21.15.44:0/3169493204 learned_addr learned my addr 172.21.15.44:0/3169493204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bca00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 0x7f11b8156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 -- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 msgr2=0x7f11b8156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 0x7f11b8156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 -- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11b81f5350 msgr2=0x7f11b80c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11b81f5350 0x7f11b80c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11b7e00640 1 -- 172.21.15.44:0/3169493204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b81e8960 con 0x7f11b80ca620 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.729+0000 7f11bca00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 0x7f11b8156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11b7e00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f11a4007870 tx=0x7f11a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4019020 con 0x7f11b80ca620 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11a4010b20 con 0x7f11b80ca620 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4015450 con 0x7f11b80ca620 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b80c90e0 con 0x7f11b80ca620 2024-07-17T07:10:15.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.730+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b8156f90 con 0x7f11b80ca620 2024-07-17T07:10:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.731+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b80ac810 con 0x7f11b80ca620 2024-07-17T07:10:15.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.732+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f11a401c020 con 0x7f11b80ca620 2024-07-17T07:10:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.733+0000 7f11b4c00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 0x7f11980790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:15.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.733+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f11a4099050 con 0x7f11b80ca620 2024-07-17T07:10:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.734+0000 7f11bca00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 0x7f11980790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.735+0000 7f11bca00640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 0x7f11980790a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0069c0 tx=0x7f11ac001c70 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:15.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.738+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f11a4023020 con 0x7f11b80ca620 2024-07-17T07:10:15.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.948+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f11b80b4200 con 0x7f11b80ca620 2024-07-17T07:10:15.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.954+0000 7f11b4c00640 1 -- 172.21.15.44:0/3169493204 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f11a4011040 con 0x7f11b80ca620 2024-07-17T07:10:15.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.957+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 msgr2=0x7f11980790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.957+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 0x7f11980790a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0069c0 tx=0x7f11ac001c70 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.957+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 msgr2=0x7f11b81f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.957+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f11a4007870 tx=0x7f11a4007d30 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 shutdown_connections 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f1198076be0 0x7f11980790a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11b81f5350 0x7f11b80c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11b80ca620 0x7f11b81f4e10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 --2- 172.21.15.44:0/3169493204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11a0004d40 0x7f11b8156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 >> 172.21.15.44:0/3169493204 conn(0x7f11b80b0eb0 msgr2=0x7f11b80aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 shutdown_connections 2024-07-17T07:10:15.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:15.958+0000 7f11bf400640 1 -- 172.21.15.44:0/3169493204 wait complete. 2024-07-17T07:10:15.979 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:10:15.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:10:16.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:10:16.225 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:10:16.225 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:16.225 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:16.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:10:16.285 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:10:16.285 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:16.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:10:16.340 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:16.340 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:16.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:10:16.397 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:10:16.397 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:16.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:10:16.454 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:10:16.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:16.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 -- 172.21.15.44:0/3186919476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 msgr2=0x7fbac0159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:16.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 --2- 172.21.15.44:0/3186919476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0159b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005fd0 tx=0x7fbab002ae70 comp rx=0 tx=0).stop 2024-07-17T07:10:16.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 -- 172.21.15.44:0/3186919476 shutdown_connections 2024-07-17T07:10:16.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 --2- 172.21.15.44:0/3186919476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0159b00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 -- 172.21.15.44:0/3186919476 >> 172.21.15.44:0/3186919476 conn(0x7fbac00b0fb0 msgr2=0x7fbac00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 -- 172.21.15.44:0/3186919476 shutdown_connections 2024-07-17T07:10:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.676+0000 7fbac6800640 1 -- 172.21.15.44:0/3186919476 wait complete. 2024-07-17T07:10:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.678+0000 7fbac6800640 1 Processor -- start 2024-07-17T07:10:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.678+0000 7fbac6800640 1 -- start start 2024-07-17T07:10:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00acaf0 0x7fbac01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015a040 0x7fbac00742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006f890 con 0x7fbac0153a50 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006f710 con 0x7fbac00acaf0 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbac6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbac006e7a0 con 0x7fbac015a040 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbab6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbab6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48164/0 (socket says 172.21.15.44:48164) 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbab6a00640 1 -- 172.21.15.44:0/3114248956 learned_addr learned my addr 172.21.15.44:0/3114248956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbab6a00640 1 -- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015a040 msgr2=0x7fbac00742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.679+0000 7fbab6a00640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015a040 0x7fbac00742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab6a00640 1 -- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00acaf0 msgr2=0x7fbac01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab6a00640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00acaf0 0x7fbac01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab6a00640 1 -- 172.21.15.44:0/3114248956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac0150ac0 con 0x7fbac0153a50 2024-07-17T07:10:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab6a00640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c3c0 tx=0x7fbaa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8017070 con 0x7fbac0153a50 2024-07-17T07:10:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa80035f0 con 0x7fbac0153a50 2024-07-17T07:10:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8003a40 con 0x7fbac0153a50 2024-07-17T07:10:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac0150820 con 0x7fbac0153a50 2024-07-17T07:10:16.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.680+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac0156a50 con 0x7fbac0153a50 2024-07-17T07:10:16.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.681+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac015dfa0 con 0x7fbac0153a50 2024-07-17T07:10:16.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.683+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fbaa801a070 con 0x7fbac0153a50 2024-07-17T07:10:16.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.684+0000 7fbab7e00640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 0x7fba9c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:16.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.684+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7fbaa8097a60 con 0x7fbac0153a50 2024-07-17T07:10:16.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.684+0000 7fbabf400640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 0x7fba9c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:16.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.686+0000 7fbabf400640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 0x7fba9c079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbab0034000 tx=0x7fbab00046c0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:16.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.688+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbac015dfa0 con 0x7fbac0153a50 2024-07-17T07:10:16.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.946+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac0150580 con 0x7fbac0153a50 2024-07-17T07:10:16.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.947+0000 7fbab7e00640 1 -- 172.21.15.44:0/3114248956 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3917 (secure 0 0 0) 0x7fbac015acc0 con 0x7fbac0153a50 2024-07-17T07:10:16.947 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:16.947 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-07-17T07:10:08:919115+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1666192665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1666192665},{"type":"v1","addr":"172.21.15.44:6834","nonce":1666192665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1604440979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1604440979},{"type":"v1","addr":"172.21.15.175:6836","nonce":1604440979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3515968140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3515968140},{"type":"v1","addr":"172.21.15.175:6837","nonce":3515968140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":6165,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1952713482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1952713482},{"type":"v1","addr":"172.21.15.44:6835","nonce":1952713482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-07-17T07:10:16.947 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-07-17T07:10:16.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 msgr2=0x7fba9c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:16.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 0x7fba9c079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbab0034000 tx=0x7fbab00046c0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 msgr2=0x7fbac0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c3c0 tx=0x7fbaa800c880 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 shutdown_connections 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbac015a040 0x7fbac00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbac0153a50 0x7fbac0073d90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fba9c076bb0 0x7fba9c079070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 --2- 172.21.15.44:0/3114248956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbac00acaf0 0x7fbac01516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:16.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 >> 172.21.15.44:0/3114248956 conn(0x7fbac00b0fb0 msgr2=0x7fbac015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:16.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 shutdown_connections 2024-07-17T07:10:16.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:16.950+0000 7fbac6800640 1 -- 172.21.15.44:0/3114248956 wait complete. 2024-07-17T07:10:16.971 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:10:16.971 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:10:16.971 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:10:16.971 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:10:16.971 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:10:16.972 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:10:16.972 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:10:16.972 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:10:16.972 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:10:16.972 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:10:16.972 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:10:16.972 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:10:16.973 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:10:16.973 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:10:16.973 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:10:16.973 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:10:16.995 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:10:16.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:10:17.000 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:10:17.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:10:17.001 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:10:17.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:10:17.002 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:10:17.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:10:17.044 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:10:17.045 DEBUG:teuthology.parallel:result is None 2024-07-17T07:10:17.049 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:10:17.049 DEBUG:teuthology.parallel:result is None 2024-07-17T07:10:17.050 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:10:17.051 DEBUG:teuthology.parallel:result is None 2024-07-17T07:10:17.055 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:10:17.055 DEBUG:teuthology.parallel:result is None 2024-07-17T07:10:17.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:10:17.197 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:10:17.199 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:10:17.200 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:10:17.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-07-17T07:10:17.203 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:10:17.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 -- 172.21.15.44:0/4139527361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 msgr2=0x7f232806f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 --2- 172.21.15.44:0/4139527361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f232806f950 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2310009830 tx=0x7f231002a510 comp rx=0 tx=0).stop 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 -- 172.21.15.44:0/4139527361 shutdown_connections 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 --2- 172.21.15.44:0/4139527361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f23280cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 --2- 172.21.15.44:0/4139527361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f232806f950 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.244+0000 7f232f400640 1 -- 172.21.15.44:0/4139527361 >> 172.21.15.44:0/4139527361 conn(0x7f232806bf40 msgr2=0x7f232806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.245+0000 7f232f400640 1 -- 172.21.15.44:0/4139527361 shutdown_connections 2024-07-17T07:10:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.245+0000 7f232f400640 1 -- 172.21.15.44:0/4139527361 wait complete. 2024-07-17T07:10:17.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.246+0000 7f232f400640 1 Processor -- start 2024-07-17T07:10:17.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.246+0000 7f232f400640 1 -- start start 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.247+0000 7f232f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2328073780 0x7f232814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.247+0000 7f232f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f232814b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.247+0000 7f232f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23280cd8e0 con 0x7f232815c5f0 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23280cd760 con 0x7f2328073780 2024-07-17T07:10:17.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23280cda60 con 0x7f23280cf060 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48214/0 (socket says 172.21.15.44:48214) 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f231fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f232814b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 -- 172.21.15.44:0/1691747952 learned_addr learned my addr 172.21.15.44:0/1691747952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232ca00640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2328073780 0x7f232814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 -- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 msgr2=0x7f232814b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f232814b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 -- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2328073780 msgr2=0x7f232814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2328073780 0x7f232814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f232d400640 1 -- 172.21.15.44:0/1691747952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23280afd00 con 0x7f232815c5f0 2024-07-17T07:10:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.248+0000 7f231fe00640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f232814b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-17T07:10:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f232d400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f232000eb10 tx=0x7f232000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320004b20 con 0x7f232815c5f0 2024-07-17T07:10:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2320004ce0 con 0x7f232815c5f0 2024-07-17T07:10:17.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320005280 con 0x7f232815c5f0 2024-07-17T07:10:17.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23280affd0 con 0x7f232815c5f0 2024-07-17T07:10:17.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.249+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23280b0590 con 0x7f232815c5f0 2024-07-17T07:10:17.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.250+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23280cf510 con 0x7f232815c5f0 2024-07-17T07:10:17.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.252+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f2320005420 con 0x7f232815c5f0 2024-07-17T07:10:17.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.253+0000 7f2325600640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 0x7f2304079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:17.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.254+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6543+0+0 (secure 0 0 0) 0x7f2320099220 con 0x7f232815c5f0 2024-07-17T07:10:17.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.254+0000 7f232ca00640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 0x7f2304079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:17.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.255+0000 7f232ca00640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 0x7f2304079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f23180017c0 tx=0x7f2318007e90 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:17.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.258+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2320061cc0 con 0x7f232815c5f0 2024-07-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.479+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f232815b300 con 0x7f232815c5f0 2024-07-17T07:10:17.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:17.997+0000 7f2325600640 1 -- 172.21.15.44:0/1691747952 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f2320066210 con 0x7f232815c5f0 2024-07-17T07:10:17.997 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 msgr2=0x7f2304079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 0x7f2304079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f23180017c0 tx=0x7f2318007e90 comp rx=0 tx=0).stop 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 msgr2=0x7f23280af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f232000eb10 tx=0x7f232000c2d0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 shutdown_connections 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232815c5f0 0x7f23280af7c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23280cf060 0x7f232814b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2304076bb0 0x7f2304079070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 --2- 172.21.15.44:0/1691747952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2328073780 0x7f232814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 >> 172.21.15.44:0/1691747952 conn(0x7f232806bf40 msgr2=0x7f2328158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 shutdown_connections 2024-07-17T07:10:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.002+0000 7f232f400640 1 -- 172.21.15.44:0/1691747952 wait complete. 2024-07-17T07:10:18.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:10:18.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 -- 172.21.15.44:0/4018556860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 msgr2=0x7fe25415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 --2- 172.21.15.44:0/4018556860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe25415d800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe24c005fd0 tx=0x7fe24c02aa50 comp rx=0 tx=0).stop 2024-07-17T07:10:18.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 -- 172.21.15.44:0/4018556860 shutdown_connections 2024-07-17T07:10:18.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 --2- 172.21.15.44:0/4018556860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe25415d800 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 --2- 172.21.15.44:0/4018556860 >> v2:172.21.15.175:3300/0 conn(0x7fe2540ce5c0 0x7fe25415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 -- 172.21.15.44:0/4018556860 >> 172.21.15.44:0/4018556860 conn(0x7fe2540cfec0 msgr2=0x7fe2540d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:18.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 -- 172.21.15.44:0/4018556860 shutdown_connections 2024-07-17T07:10:18.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.226+0000 7fe259e00640 1 -- 172.21.15.44:0/4018556860 wait complete. 2024-07-17T07:10:18.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.228+0000 7fe259e00640 1 Processor -- start 2024-07-17T07:10:18.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.229+0000 7fe259e00640 1 -- start start 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2540b8c80 0x7fe25414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 0x7fe2540ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe25415e0d0 con 0x7fe2540b9060 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe25415df50 con 0x7fe2540b8c80 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe259e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe25415e250 con 0x7fe2540ce5c0 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe253e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe253e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48218/0 (socket says 172.21.15.44:48218) 2024-07-17T07:10:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.230+0000 7fe253e00640 1 -- 172.21.15.44:0/866096939 learned_addr learned my addr 172.21.15.44:0/866096939 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe258a00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2540b8c80 0x7fe25414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe259400640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 0x7fe2540ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 -- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 msgr2=0x7fe2540ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 0x7fe2540ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 -- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2540b8c80 msgr2=0x7fe25414d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2540b8c80 0x7fe25414d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 -- 172.21.15.44:0/866096939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2540cb700 con 0x7fe2540b9060 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe259400640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 0x7fe2540ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.231+0000 7fe253e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b410 tx=0x7fe24000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.232+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240015020 con 0x7fe2540b9060 2024-07-17T07:10:18.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.232+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe240011940 con 0x7fe2540b9060 2024-07-17T07:10:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.232+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240011d90 con 0x7fe2540b9060 2024-07-17T07:10:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.232+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2540cb9e0 con 0x7fe2540b9060 2024-07-17T07:10:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.232+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2540cbfa0 con 0x7fe2540b9060 2024-07-17T07:10:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.233+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25415f900 con 0x7fe2540b9060 2024-07-17T07:10:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.234+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fe240018020 con 0x7fe2540b9060 2024-07-17T07:10:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.236+0000 7fe250c00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 0x7fe234079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.236+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5769+0+0 (secure 0 0 0) 0x7fe2400983d0 con 0x7fe2540b9060 2024-07-17T07:10:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.236+0000 7fe258a00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 0x7fe234079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.238+0000 7fe258a00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 0x7fe234079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe248007fd0 tx=0x7fe248009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:18.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.240+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe24001f020 con 0x7fe2540b9060 2024-07-17T07:10:18.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.530+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe2540b5d70 con 0x7fe2540b9060 2024-07-17T07:10:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.531+0000 7fe250c00640 1 -- 172.21.15.44:0/866096939 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) ==== 80+0+5304 (secure 0 0 0) 0x7fe2400617c0 con 0x7fe2540b9060 2024-07-17T07:10:18.530 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:18.530 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:10:18.530 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:10:18.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.531 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:10:18.532 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.533 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.534 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.535 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:10:18.536 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.537 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:10:18.540 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:10:18.542 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:10:18.543 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:10:18.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:10:18.545 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:10:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 msgr2=0x7fe234079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 0x7fe234079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe248007fd0 tx=0x7fe248009040 comp rx=0 tx=0).stop 2024-07-17T07:10:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 msgr2=0x7fe2541f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b410 tx=0x7fe24000b8d0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 shutdown_connections 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2540ce5c0 0x7fe2540ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2540b9060 0x7fe2541f45c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe234076bb0 0x7fe234079070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 --2- 172.21.15.44:0/866096939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2540b8c80 0x7fe25414d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 >> 172.21.15.44:0/866096939 conn(0x7fe2540cfec0 msgr2=0x7fe2540b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.534+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 shutdown_connections 2024-07-17T07:10:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.535+0000 7fe259e00640 1 -- 172.21.15.44:0/866096939 wait complete. 2024-07-17T07:10:18.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 -- 172.21.15.44:0/2532475971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 msgr2=0x7f9cac15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 --2- 172.21.15.44:0/2532475971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac15d800 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8005fd0 tx=0x7f9ca802a580 comp rx=0 tx=0).stop 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 -- 172.21.15.44:0/2532475971 shutdown_connections 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 --2- 172.21.15.44:0/2532475971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac15d800 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 --2- 172.21.15.44:0/2532475971 >> v2:172.21.15.175:3300/0 conn(0x7f9cac0ce5c0 0x7f9cac15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 -- 172.21.15.44:0/2532475971 >> 172.21.15.44:0/2532475971 conn(0x7f9cac0cfec0 msgr2=0x7f9cac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:18.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 -- 172.21.15.44:0/2532475971 shutdown_connections 2024-07-17T07:10:18.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.745+0000 7f9cb3200640 1 -- 172.21.15.44:0/2532475971 wait complete. 2024-07-17T07:10:18.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.747+0000 7f9cb3200640 1 Processor -- start 2024-07-17T07:10:18.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.747+0000 7f9cb3200640 1 -- start start 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8c80 0x7f9cac0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0ce5c0 0x7f9cac0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac15e0d0 con 0x7f9cac0b9060 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac15df50 con 0x7f9cac0ce5c0 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac15e250 con 0x7f9cac0b8c80 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48238/0 (socket says 172.21.15.44:48238) 2024-07-17T07:10:18.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb1400640 1 -- 172.21.15.44:0/3762001921 learned_addr learned my addr 172.21.15.44:0/3762001921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:18.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb1e00640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8c80 0x7f9cac0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb2800640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0ce5c0 0x7f9cac0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.748+0000 7f9cb1400640 1 -- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8c80 msgr2=0x7f9cac0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb1400640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8c80 0x7f9cac0a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb1400640 1 -- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0ce5c0 msgr2=0x7f9cac0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb1400640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0ce5c0 0x7f9cac0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb1400640 1 -- 172.21.15.44:0/3762001921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac0aa4b0 con 0x7f9cac0b9060 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb1400640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c7a0 tx=0x7f9c9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c007bb0 con 0x7f9cac0b9060 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c9c005020 con 0x7f9cac0b9060 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c007e60 con 0x7f9cac0b9060 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac0aa730 con 0x7f9cac0b9060 2024-07-17T07:10:18.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.749+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac0aacf0 con 0x7f9cac0b9060 2024-07-17T07:10:18.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.751+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac15e720 con 0x7f9cac0b9060 2024-07-17T07:10:18.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.752+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f9c9c005470 con 0x7f9cac0b9060 2024-07-17T07:10:18.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.753+0000 7f9ca6000640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 0x7f9c90079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:18.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.754+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5769+0+0 (secure 0 0 0) 0x7f9c9c098790 con 0x7f9cac0b9060 2024-07-17T07:10:18.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.754+0000 7f9cb1e00640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 0x7f9c90079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:18.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.755+0000 7f9cb1e00640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 0x7f9c90079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007fd0 tx=0x7f9ca0009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:18.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.758+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9cac0aa4b0 con 0x7f9cac0b9060 2024-07-17T07:10:18.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.961+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9cac0b5d70 con 0x7f9cac0b9060 2024-07-17T07:10:18.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.962+0000 7f9ca6000640 1 -- 172.21.15.44:0/3762001921 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f9c9c014030 con 0x7f9cac0b9060 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:18.961 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:10:18.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 msgr2=0x7f9c90079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 0x7f9c90079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0007fd0 tx=0x7f9ca0009040 comp rx=0 tx=0).stop 2024-07-17T07:10:18.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 msgr2=0x7f9cac14cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c7a0 tx=0x7f9c9c00cc60 comp rx=0 tx=0).stop 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 shutdown_connections 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f9c90076bb0 0x7f9c90079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cac0ce5c0 0x7f9cac0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0b9060 0x7f9cac14cc00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 --2- 172.21.15.44:0/3762001921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac0b8c80 0x7f9cac0a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 >> 172.21.15.44:0/3762001921 conn(0x7f9cac0cfec0 msgr2=0x7f9cac0b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 shutdown_connections 2024-07-17T07:10:18.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:18.964+0000 7f9cb3200640 1 -- 172.21.15.44:0/3762001921 wait complete. 2024-07-17T07:10:18.982 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:10:18.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:10:19.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 -- 172.21.15.44:0/2082248779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5540cb880 msgr2=0x7fe5540cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 --2- 172.21.15.44:0/2082248779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5540cb880 0x7fe5540cda10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe54c005fd0 tx=0x7fe54c02a820 comp rx=0 tx=0).stop 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 -- 172.21.15.44:0/2082248779 shutdown_connections 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 --2- 172.21.15.44:0/2082248779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5540cb880 0x7fe5540cda10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 -- 172.21.15.44:0/2082248779 >> 172.21.15.44:0/2082248779 conn(0x7fe5540b0eb0 msgr2=0x7fe5540b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 -- 172.21.15.44:0/2082248779 shutdown_connections 2024-07-17T07:10:19.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.172+0000 7fe55be00640 1 -- 172.21.15.44:0/2082248779 wait complete. 2024-07-17T07:10:19.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.174+0000 7fe55be00640 1 Processor -- start 2024-07-17T07:10:19.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.174+0000 7fe55be00640 1 -- start start 2024-07-17T07:10:19.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:19.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5540cb880 0x7fe5540b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:19.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe55415b060 0x7fe554155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:19.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe55415c430 con 0x7fe55406dd30 2024-07-17T07:10:19.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe55415b690 con 0x7fe5540cb880 2024-07-17T07:10:19.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe55be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540cdf50 con 0x7fe55415b060 2024-07-17T07:10:19.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.175+0000 7fe559400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:19.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37762/0 (socket says 172.21.15.44:37762) 2024-07-17T07:10:19.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 -- 172.21.15.44:0/2223287449 learned_addr learned my addr 172.21.15.44:0/2223287449 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 -- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe55415b060 msgr2=0x7fe554155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe55415b060 0x7fe554155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 -- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5540cb880 msgr2=0x7fe5540b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5540cb880 0x7fe5540b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 -- 172.21.15.44:0/2223287449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540b2570 con 0x7fe55406dd30 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe559400640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe54800b3f0 tx=0x7fe54800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:19.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548015020 con 0x7fe55406dd30 2024-07-17T07:10:19.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe548011920 con 0x7fe55406dd30 2024-07-17T07:10:19.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.176+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548011d70 con 0x7fe55406dd30 2024-07-17T07:10:19.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.177+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5540b2850 con 0x7fe55406dd30 2024-07-17T07:10:19.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.177+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5540b2e10 con 0x7fe55406dd30 2024-07-17T07:10:19.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.178+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5540ce440 con 0x7fe55406dd30 2024-07-17T07:10:19.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.179+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fe548018020 con 0x7fe55406dd30 2024-07-17T07:10:19.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.180+0000 7fe551600640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 0x7fe534079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:19.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.180+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5769+0+0 (secure 0 0 0) 0x7fe548097fa0 con 0x7fe55406dd30 2024-07-17T07:10:19.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.180+0000 7fe558a00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 0x7fe534079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:19.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.181+0000 7fe558a00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 0x7fe534079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe554074050 tx=0x7fe5400078d0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:19.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.184+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe54801f030 con 0x7fe55406dd30 2024-07-17T07:10:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:19.394+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe5540cca50 con 0x7fe55406dd30 2024-07-17T07:10:20.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.008+0000 7fe551600640 1 -- 172.21.15.44:0/2223287449 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7fe5480614c0 con 0x7fe55406dd30 2024-07-17T07:10:20.007 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 msgr2=0x7fe534079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 0x7fe534079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe554074050 tx=0x7fe5400078d0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 msgr2=0x7fe554154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe54800b3f0 tx=0x7fe54800b8b0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 shutdown_connections 2024-07-17T07:10:20.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe55415b060 0x7fe554155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe534076bb0 0x7fe534079070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5540cb880 0x7fe5540b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 --2- 172.21.15.44:0/2223287449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe55406dd30 0x7fe554154920 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 >> 172.21.15.44:0/2223287449 conn(0x7fe5540b0eb0 msgr2=0x7fe5540ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 shutdown_connections 2024-07-17T07:10:20.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.012+0000 7fe55be00640 1 -- 172.21.15.44:0/2223287449 wait complete. 2024-07-17T07:10:20.038 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 -- 172.21.15.44:0/2381116508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5154003af0 msgr2=0x7f5154003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 --2- 172.21.15.44:0/2381116508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5154003af0 0x7f5154003f10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5160006fb0 tx=0x7f5154005830 comp rx=0 tx=0).stop 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516f400640 1 -- 172.21.15.44:0/2381116508 reap_dead start 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 -- 172.21.15.44:0/2381116508 shutdown_connections 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 -- 172.21.15.44:0/2381116508 >> 172.21.15.44:0/2381116508 conn(0x7f51700b4ff0 msgr2=0x7f51700b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 -- 172.21.15.44:0/2381116508 shutdown_connections 2024-07-17T07:10:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.247+0000 7f516fe00640 1 -- 172.21.15.44:0/2381116508 wait complete. 2024-07-17T07:10:20.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.250+0000 7f516fe00640 1 Processor -- start 2024-07-17T07:10:20.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.250+0000 7f516fe00640 1 -- start start 2024-07-17T07:10:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 0x7f51700c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51700ca5b0 0x7f517015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5170166400 con 0x7f517015ca20 2024-07-17T07:10:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5170166280 con 0x7f5154003af0 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5170166580 con 0x7f51700ca5b0 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.251+0000 7f516f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37780/0 (socket says 172.21.15.44:37780) 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 -- 172.21.15.44:0/261063713 learned_addr learned my addr 172.21.15.44:0/261063713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516e000640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51700ca5b0 0x7f517015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516ea00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 0x7f51700c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 -- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51700ca5b0 msgr2=0x7f517015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51700ca5b0 0x7f517015c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 -- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 msgr2=0x7f51700c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 0x7f51700c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516f400640 1 -- 172.21.15.44:0/261063713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51700cb200 con 0x7f517015ca20 2024-07-17T07:10:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.252+0000 7f516ea00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 0x7f51700c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:10:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f516f400640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5168009670 tx=0x7f5168009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168004b20 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5168004db0 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800fba0 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51700cb470 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.253+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51700cba30 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.254+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517015cea0 con 0x7f517015ca20 2024-07-17T07:10:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.256+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f516801b020 con 0x7f517015ca20 2024-07-17T07:10:20.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.257+0000 7f5166a00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 0x7f5150079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.257+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6139+0+0 (secure 0 0 0) 0x7f5168098df0 con 0x7f517015ca20 2024-07-17T07:10:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.258+0000 7f516ea00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 0x7f5150079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:20.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.259+0000 7f516ea00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 0x7f5150079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5160033000 tx=0x7f51600019a0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:20.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.261+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f51680620f0 con 0x7f517015ca20 2024-07-17T07:10:20.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:20.478+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f51700d0e10 con 0x7f517015ca20 2024-07-17T07:10:21.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.015+0000 7f5166a00640 1 -- 172.21.15.44:0/261063713 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7f51680619f0 con 0x7f517015ca20 2024-07-17T07:10:21.014 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:10:21.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 msgr2=0x7f5150079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 0x7f5150079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5160033000 tx=0x7f51600019a0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 msgr2=0x7f51701f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5168009670 tx=0x7f5168009b30 comp rx=0 tx=0).stop 2024-07-17T07:10:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 shutdown_connections 2024-07-17T07:10:21.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517015ca20 0x7f51701f7480 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51700ca5b0 0x7f517015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f5150076bb0 0x7f5150079070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 --2- 172.21.15.44:0/261063713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5154003af0 0x7f51700c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.019+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 >> 172.21.15.44:0/261063713 conn(0x7f51700b4ff0 msgr2=0x7f51700b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.020+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 shutdown_connections 2024-07-17T07:10:21.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.020+0000 7f516fe00640 1 -- 172.21.15.44:0/261063713 wait complete. 2024-07-17T07:10:21.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:10:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 -- 172.21.15.44:0/577010351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 msgr2=0x7f880c0acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 --2- 172.21.15.44:0/577010351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c0acb30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f87fc005fd0 tx=0x7f87fc02b860 comp rx=0 tx=0).stop 2024-07-17T07:10:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 -- 172.21.15.44:0/577010351 shutdown_connections 2024-07-17T07:10:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 --2- 172.21.15.44:0/577010351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0ad070 0x7f880c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 --2- 172.21.15.44:0/577010351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c0acb30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 -- 172.21.15.44:0/577010351 >> 172.21.15.44:0/577010351 conn(0x7f880c0b0eb0 msgr2=0x7f880c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 -- 172.21.15.44:0/577010351 shutdown_connections 2024-07-17T07:10:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.238+0000 7f8811000640 1 -- 172.21.15.44:0/577010351 wait complete. 2024-07-17T07:10:21.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.240+0000 7f8811000640 1 Processor -- start 2024-07-17T07:10:21.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.240+0000 7f8811000640 1 -- start start 2024-07-17T07:10:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0ad070 0x7f880c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0b3c20 0x7f880c15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f880c06d9b0 con 0x7f880c06e5c0 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f880c06d830 con 0x7f880c0ad070 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8811000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f880c06db30 con 0x7f880c0b3c20 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8809e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8809e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37802/0 (socket says 172.21.15.44:37802) 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.241+0000 7f8809e00640 1 -- 172.21.15.44:0/905419683 learned_addr learned my addr 172.21.15.44:0/905419683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f880a800640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0b3c20 0x7f880c15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:21.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809400640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0ad070 0x7f880c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 -- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0b3c20 msgr2=0x7f880c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:21.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0b3c20 0x7f880c15cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 -- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0ad070 msgr2=0x7f880c153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0ad070 0x7f880c153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 -- 172.21.15.44:0/905419683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1f2390 con 0x7f880c06e5c0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.242+0000 7f8809e00640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b3e0 tx=0x7f87f400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.243+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4015020 con 0x7f880c06e5c0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.243+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87f4011940 con 0x7f880c06e5c0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.243+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4011d90 con 0x7f880c06e5c0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.243+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1f2630 con 0x7f880c06e5c0 2024-07-17T07:10:21.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.243+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1f2bf0 con 0x7f880c06e5c0 2024-07-17T07:10:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.244+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0acb30 con 0x7f880c06e5c0 2024-07-17T07:10:21.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.246+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f87f4018020 con 0x7f880c06e5c0 2024-07-17T07:10:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.247+0000 7f8802000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 0x7f87e80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.247+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6521+0+0 (secure 0 0 0) 0x7f87f4098670 con 0x7f880c06e5c0 2024-07-17T07:10:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.247+0000 7f8809400640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 0x7f87e80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:21.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.248+0000 7f8809400640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 0x7f87e80790a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87fc035000 tx=0x7f87fc004730 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:21.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.252+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f87f401f020 con 0x7f880c06e5c0 2024-07-17T07:10:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:21.507+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f880c06e9e0 con 0x7f880c06e5c0 2024-07-17T07:10:22.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.067+0000 7f8802000640 1 -- 172.21.15.44:0/905419683 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f87f4061770 con 0x7f880c06e5c0 2024-07-17T07:10:22.069 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 9 and data pool 10 2024-07-17T07:10:22.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 msgr2=0x7f87e80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 0x7f87e80790a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87fc035000 tx=0x7f87fc004730 comp rx=0 tx=0).stop 2024-07-17T07:10:22.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 msgr2=0x7f880c14f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b3e0 tx=0x7f87f400b8a0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 shutdown_connections 2024-07-17T07:10:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f880c0b3c20 0x7f880c15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f87e8076be0 0x7f87e80790a0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87fc035000 tx=0x7f87fc004730 comp rx=0 tx=0).stop 2024-07-17T07:10:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0ad070 0x7f880c153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 --2- 172.21.15.44:0/905419683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f880c06e5c0 0x7f880c14f740 secure :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b3e0 tx=0x7f87f400b8a0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.077+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 >> 172.21.15.44:0/905419683 conn(0x7f880c0b0eb0 msgr2=0x7f880c15de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:22.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.082+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 shutdown_connections 2024-07-17T07:10:22.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.083+0000 7f8811000640 1 -- 172.21.15.44:0/905419683 wait complete. 2024-07-17T07:10:22.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 -- 172.21.15.44:0/1043267634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406dde0 msgr2=0x7f6194070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 --2- 172.21.15.44:0/1043267634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406dde0 0x7f6194070de0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f618c005fd0 tx=0x7f618c02a8b0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 -- 172.21.15.44:0/1043267634 shutdown_connections 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 --2- 172.21.15.44:0/1043267634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406dde0 0x7f6194070de0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 -- 172.21.15.44:0/1043267634 >> 172.21.15.44:0/1043267634 conn(0x7f61940b3020 msgr2=0x7f61940b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:22.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 -- 172.21.15.44:0/1043267634 shutdown_connections 2024-07-17T07:10:22.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.292+0000 7f619bc00640 1 -- 172.21.15.44:0/1043267634 wait complete. 2024-07-17T07:10:22.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.294+0000 7f619bc00640 1 Processor -- start 2024-07-17T07:10:22.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.294+0000 7f619bc00640 1 -- start start 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f619406dde0 0x7f619409d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61940c6290 0x7f61940b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61940c7c20 con 0x7f619406d2f0 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61940c7aa0 con 0x7f619406dde0 2024-07-17T07:10:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f619bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61940c7da0 con 0x7f61940c6290 2024-07-17T07:10:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f6199200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f6199200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37822/0 (socket says 172.21.15.44:37822) 2024-07-17T07:10:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f6199200640 1 -- 172.21.15.44:0/2556768422 learned_addr learned my addr 172.21.15.44:0/2556768422 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.295+0000 7f6193e00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f619406dde0 0x7f619409d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199c00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61940c6290 0x7f61940b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 -- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61940c6290 msgr2=0x7f61940b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61940c6290 0x7f61940b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 -- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f619406dde0 msgr2=0x7f619409d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f619406dde0 0x7f619409d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 -- 172.21.15.44:0/2556768422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619409e150 con 0x7f619406d2f0 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6199200640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f618800c3a0 tx=0x7f618800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188011790 con 0x7f619406d2f0 2024-07-17T07:10:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6188017070 con 0x7f619406d2f0 2024-07-17T07:10:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188011cc0 con 0x7f619406d2f0 2024-07-17T07:10:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619409b410 con 0x7f619406d2f0 2024-07-17T07:10:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.296+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619409b9d0 con 0x7f619406d2f0 2024-07-17T07:10:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.298+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61940d21a0 con 0x7f619406d2f0 2024-07-17T07:10:22.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.299+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f618801c460 con 0x7f619406d2f0 2024-07-17T07:10:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.300+0000 7f6190c00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 0x7f6174079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.300+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6621+0+0 (secure 0 0 0) 0x7f6188098490 con 0x7f619406d2f0 2024-07-17T07:10:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.301+0000 7f6193e00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 0x7f6174079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.302+0000 7f6193e00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 0x7f6174079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f619406d0c0 tx=0x7f6180009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.304+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f61940d21a0 con 0x7f619406d2f0 2024-07-17T07:10:22.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.516+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f619406d110 con 0x7f619406d2f0 2024-07-17T07:10:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.518+0000 7f6190c00640 1 -- 172.21.15.44:0/2556768422 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17671 (secure 0 0 0) 0x7f619406d110 con 0x7f619406d2f0 2024-07-17T07:10:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:22.517 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":82,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:10:21.508995+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:10:19.413303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-07-17T07:10:20.496271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.520+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 msgr2=0x7f6174079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.520+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 0x7f6174079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f619406d0c0 tx=0x7f6180009040 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.520+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 msgr2=0x7f619414e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.520+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f618800c3a0 tx=0x7f618800c860 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 shutdown_connections 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61940c6290 0x7f61940b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f6174076bb0 0x7f6174079070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f619406dde0 0x7f619409d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 --2- 172.21.15.44:0/2556768422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f619406d2f0 0x7f619414e280 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 >> 172.21.15.44:0/2556768422 conn(0x7f61940b3020 msgr2=0x7f61940b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 shutdown_connections 2024-07-17T07:10:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.521+0000 7f619bc00640 1 -- 172.21.15.44:0/2556768422 wait complete. 2024-07-17T07:10:22.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.750+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3901010326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac0cc890 msgr2=0x7fc0ac0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.750+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3901010326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac0cc890 0x7fc0ac0cf6a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc098009830 tx=0x7fc09802a5a0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.750+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3901010326 shutdown_connections 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.750+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3901010326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cfbe0 0x7fc0ac0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.751+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3901010326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac0cc890 0x7fc0ac0cf6a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.751+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3901010326 >> 172.21.15.44:0/3901010326 conn(0x7fc0ac0b3120 msgr2=0x7fc0ac0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.751+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3901010326 shutdown_connections 2024-07-17T07:10:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.751+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3901010326 wait complete. 2024-07-17T07:10:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.753+0000 7fc0b2a00640 1 Processor -- start 2024-07-17T07:10:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.753+0000 7fc0b2a00640 1 -- start start 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cc890 0x7fc0ac14b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0ac0cfbe0 0x7fc0ac156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b2570 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b23f0 con 0x7fc0ac0cfbe0 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b26f0 con 0x7fc0ac0cc890 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37834/0 (socket says 172.21.15.44:37834) 2024-07-17T07:10:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0b0a00640 1 -- 172.21.15.44:0/3618310444 learned_addr learned my addr 172.21.15.44:0/3618310444 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0ab400640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0ac0cfbe0 0x7fc0ac156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.754+0000 7fc0abe00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cc890 0x7fc0ac14b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 -- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cc890 msgr2=0x7fc0ac14b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cc890 0x7fc0ac14b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 -- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0ac0cfbe0 msgr2=0x7fc0ac156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0ac0cfbe0 0x7fc0ac156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 -- 172.21.15.44:0/3618310444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac14bb90 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b0a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400ead0 tx=0x7fc0a400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400cd80 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a4004c10 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4005020 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac0af320 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.755+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac0aecd0 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.757+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0cb250 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.758+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fc0a40051c0 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.759+0000 7fc0a3e00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 0x7fc08c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.759+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6621+0+0 (secure 0 0 0) 0x7fc0a4099040 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.760+0000 7fc0abe00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 0x7fc08c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.761+0000 7fc0abe00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 0x7fc08c079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc09c001540 tx=0x7fc09c009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.763+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc0ac14bb90 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.976+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0ac0cf380 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.979+0000 7fc0a3e00640 1 -- 172.21.15.44:0/3618310444 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+17671 (secure 0 0 0) 0x7fc0a4016030 con 0x7fc0ac15b5e0 2024-07-17T07:10:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:22.978 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":82,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:10:21.508995+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:10:19.413303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-07-17T07:10:20.496271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 msgr2=0x7fc08c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 0x7fc08c079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc09c001540 tx=0x7fc09c009040 comp rx=0 tx=0).stop 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 msgr2=0x7fc0ac147f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400ead0 tx=0x7fc0a400ef90 comp rx=0 tx=0).stop 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 shutdown_connections 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0ac15b5e0 0x7fc0ac147f80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fc08c076bb0 0x7fc08c079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0ac0cfbe0 0x7fc0ac156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 --2- 172.21.15.44:0/3618310444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0ac0cc890 0x7fc0ac14b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 >> 172.21.15.44:0/3618310444 conn(0x7fc0ac0b3120 msgr2=0x7fc0ac0ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 shutdown_connections 2024-07-17T07:10:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:22.982+0000 7fc0b2a00640 1 -- 172.21.15.44:0/3618310444 wait complete. 2024-07-17T07:10:23.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:10:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 -- 172.21.15.44:0/111138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 msgr2=0x7feebc06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 --2- 172.21.15.44:0/111138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc06ec70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7feea4009860 tx=0x7feea402a560 comp rx=0 tx=0).stop 2024-07-17T07:10:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 -- 172.21.15.44:0/111138740 shutdown_connections 2024-07-17T07:10:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 --2- 172.21.15.44:0/111138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc06ec70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 -- 172.21.15.44:0/111138740 >> 172.21.15.44:0/111138740 conn(0x7feebc0cf0d0 msgr2=0x7feebc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 -- 172.21.15.44:0/111138740 shutdown_connections 2024-07-17T07:10:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.214+0000 7feec0a00640 1 -- 172.21.15.44:0/111138740 wait complete. 2024-07-17T07:10:23.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.216+0000 7feec0a00640 1 Processor -- start 2024-07-17T07:10:23.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.217+0000 7feec0a00640 1 -- start start 2024-07-17T07:10:23.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:23.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feebc151f20 0x7feebc1f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:23.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feebc152300 0x7feebc1f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:23.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feebc06f540 con 0x7feebc0cd0b0 2024-07-17T07:10:23.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feebc06f3c0 con 0x7feebc152300 2024-07-17T07:10:23.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feec0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feebc0cb790 con 0x7feebc151f20 2024-07-17T07:10:23.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feebb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:23.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feebb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37852/0 (socket says 172.21.15.44:37852) 2024-07-17T07:10:23.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.218+0000 7feebb400640 1 -- 172.21.15.44:0/1959402946 learned_addr learned my addr 172.21.15.44:0/1959402946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 -- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feebc151f20 msgr2=0x7feebc1f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feebc151f20 0x7feebc1f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 -- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feebc152300 msgr2=0x7feebc1f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feebc152300 0x7feebc1f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:23.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 -- 172.21.15.44:0/1959402946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc1f4bd0 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feebb400640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c870 tx=0x7feeac00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.219+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac003b70 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.220+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeac003d10 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.220+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00aab0 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.220+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc0ac660 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.220+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc0acc20 con 0x7feebc0cd0b0 2024-07-17T07:10:23.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.221+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc06ec70 con 0x7feebc0cd0b0 2024-07-17T07:10:23.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.222+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7feeac00ac50 con 0x7feebc0cd0b0 2024-07-17T07:10:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.223+0000 7feeb3e00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 0x7fee98079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:23.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.223+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7feeac0986c0 con 0x7feebc0cd0b0 2024-07-17T07:10:23.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.224+0000 7feeb2a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 0x7fee98079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:23.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.225+0000 7feeb2a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 0x7fee98079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feea4034000 tx=0x7feea40046c0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:23.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.228+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feeac061760 con 0x7feebc0cd0b0 2024-07-17T07:10:23.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:23.481+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feebc151500 con 0x7feebc0cd0b0 2024-07-17T07:10:23.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-07-17T07:10:24.000 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T07:10:24.001+0000 7f9a37c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:10:24.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.079+0000 7feeb3e00640 1 -- 172.21.15.44:0/1959402946 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7feeac061060 con 0x7feebc0cd0b0 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.082+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 msgr2=0x7fee98079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.082+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 0x7fee98079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feea4034000 tx=0x7feea40046c0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.082+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 msgr2=0x7feebc0ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.082+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c870 tx=0x7feeac00cd30 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 shutdown_connections 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fee98076bb0 0x7fee98079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feebc152300 0x7feebc1f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feebc151f20 0x7feebc1f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 --2- 172.21.15.44:0/1959402946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc0cd0b0 0x7feebc0ac010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 >> 172.21.15.44:0/1959402946 conn(0x7feebc0cf0d0 msgr2=0x7feebc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:24.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 shutdown_connections 2024-07-17T07:10:24.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.083+0000 7feec0a00640 1 -- 172.21.15.44:0/1959402946 wait complete. 2024-07-17T07:10:24.105 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:10:24.106 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:10:24.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 -- 172.21.15.44:0/3678114040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 msgr2=0x7fdf8815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 --2- 172.21.15.44:0/3678114040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf8815a5a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdf880b41a0 tx=0x7fdf7802a5f0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 -- 172.21.15.44:0/3678114040 shutdown_connections 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 --2- 172.21.15.44:0/3678114040 >> v2:172.21.15.175:3300/0 conn(0x7fdf880b3280 0x7fdf8806eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 --2- 172.21.15.44:0/3678114040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf8815a5a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 -- 172.21.15.44:0/3678114040 >> 172.21.15.44:0/3678114040 conn(0x7fdf880b0eb0 msgr2=0x7fdf880b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 -- 172.21.15.44:0/3678114040 shutdown_connections 2024-07-17T07:10:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.312+0000 7fdf8d800640 1 -- 172.21.15.44:0/3678114040 wait complete. 2024-07-17T07:10:24.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.314+0000 7fdf8d800640 1 Processor -- start 2024-07-17T07:10:24.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.314+0000 7fdf8d800640 1 -- start start 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 0x7fdf8815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 0x7fdf8815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880aeb10 con 0x7fdf8806f6d0 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880ae990 con 0x7fdf880ae560 2024-07-17T07:10:24.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf8d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf881f4fd0 con 0x7fdf880b3280 2024-07-17T07:10:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf86a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37872/0 (socket says 172.21.15.44:37872) 2024-07-17T07:10:24.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.315+0000 7fdf86a00640 1 -- 172.21.15.44:0/374929562 learned_addr learned my addr 172.21.15.44:0/374929562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86000640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 0x7fdf8815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf87400640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 0x7fdf8815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 -- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 msgr2=0x7fdf8815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 0x7fdf8815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 -- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 msgr2=0x7fdf8815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 0x7fdf8815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:24.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 -- 172.21.15.44:0/374929562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf881f52b0 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86000640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 0x7fdf8815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf87400640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 0x7fdf8815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.316+0000 7fdf86a00640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009850 tx=0x7fdf78003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.317+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7803f070 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.317+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf780047a0 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.317+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78035030 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.317+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf881f5590 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.317+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881f5b50 con 0x7fdf8806f6d0 2024-07-17T07:10:24.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.318+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8815a5a0 con 0x7fdf8806f6d0 2024-07-17T07:10:24.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.319+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fdf78042070 con 0x7fdf8806f6d0 2024-07-17T07:10:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.320+0000 7fdf7ea00640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 0x7fdf68079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:24.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.320+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fdf780b9510 con 0x7fdf8806f6d0 2024-07-17T07:10:24.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.321+0000 7fdf86000640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 0x7fdf68079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:24.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.322+0000 7fdf86000640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 0x7fdf68079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf881544e0 tx=0x7fdf74005db0 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:24.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.325+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdf780825b0 con 0x7fdf8806f6d0 2024-07-17T07:10:24.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:24.576+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdf880b4200 con 0x7fdf8806f6d0 2024-07-17T07:10:25.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.090+0000 7fdf7ea00640 1 -- 172.21.15.44:0/374929562 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fdf78081eb0 con 0x7fdf8806f6d0 2024-07-17T07:10:25.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 msgr2=0x7fdf68079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 0x7fdf68079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf881544e0 tx=0x7fdf74005db0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 msgr2=0x7fdf88151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009850 tx=0x7fdf78003910 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 shutdown_connections 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf880b3280 0x7fdf8815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdf68076bb0 0x7fdf68079070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf880ae560 0x7fdf8815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 --2- 172.21.15.44:0/374929562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8806f6d0 0x7fdf88151f50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.094+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 >> 172.21.15.44:0/374929562 conn(0x7fdf880b0eb0 msgr2=0x7fdf8815c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.095+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 shutdown_connections 2024-07-17T07:10:25.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.095+0000 7fdf8d800640 1 -- 172.21.15.44:0/374929562 wait complete. 2024-07-17T07:10:25.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:10:25.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 -- 172.21.15.44:0/1396133694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc004d40 msgr2=0x7fa5cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 --2- 172.21.15.44:0/1396133694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc004d40 0x7fa5cc0051c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009b50 tx=0x7fa5cc005830 comp rx=0 tx=0).stop 2024-07-17T07:10:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e6800640 1 -- 172.21.15.44:0/1396133694 reap_dead start 2024-07-17T07:10:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 -- 172.21.15.44:0/1396133694 shutdown_connections 2024-07-17T07:10:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 -- 172.21.15.44:0/1396133694 >> 172.21.15.44:0/1396133694 conn(0x7fa5e80cfec0 msgr2=0x7fa5e80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 -- 172.21.15.44:0/1396133694 shutdown_connections 2024-07-17T07:10:25.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.329+0000 7fa5e7200640 1 -- 172.21.15.44:0/1396133694 wait complete. 2024-07-17T07:10:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.331+0000 7fa5e7200640 1 Processor -- start 2024-07-17T07:10:25.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.331+0000 7fa5e7200640 1 -- start start 2024-07-17T07:10:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5cc004d40 0x7fa5e80a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 0x7fa5e80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e815e450 con 0x7fa5e815aa40 2024-07-17T07:10:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e815e2d0 con 0x7fa5e815a2b0 2024-07-17T07:10:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e815e5d0 con 0x7fa5cc004d40 2024-07-17T07:10:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37894/0 (socket says 172.21.15.44:37894) 2024-07-17T07:10:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.332+0000 7fa5e5400640 1 -- 172.21.15.44:0/29685513 learned_addr learned my addr 172.21.15.44:0/29685513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5e00640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5cc004d40 0x7fa5e80a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e6800640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 0x7fa5e80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 -- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5cc004d40 msgr2=0x7fa5e80a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5cc004d40 0x7fa5e80a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 -- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 msgr2=0x7fa5e80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 0x7fa5e80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 -- 172.21.15.44:0/29685513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e81f8250 con 0x7fa5e815aa40 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e6800640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 0x7fa5e80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.333+0000 7fa5e5400640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000d7d0 tx=0x7fa5d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.334+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0019070 con 0x7fa5e815aa40 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.334+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5d00052e0 con 0x7fa5e815aa40 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.334+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000b8a0 con 0x7fa5e815aa40 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.334+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e80aab40 con 0x7fa5e815aa40 2024-07-17T07:10:25.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.334+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e80aa860 con 0x7fa5e815aa40 2024-07-17T07:10:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.335+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e80b4ec0 con 0x7fa5e815aa40 2024-07-17T07:10:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.336+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fa5d001c070 con 0x7fa5e815aa40 2024-07-17T07:10:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.337+0000 7fa5de000640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 0x7fa5c4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.337+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fa5d0097fe0 con 0x7fa5e815aa40 2024-07-17T07:10:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.338+0000 7fa5e5e00640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 0x7fa5c4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.339+0000 7fa5e5e00640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 0x7fa5c4079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005fd0 tx=0x7fa5d8009500 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:25.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.341+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa5e81f8250 con 0x7fa5e815aa40 2024-07-17T07:10:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.594+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e80b5d70 con 0x7fa5e815aa40 2024-07-17T07:10:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.595+0000 7fa5de000640 1 -- 172.21.15.44:0/29685513 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7fa5d00610c0 con 0x7fa5e815aa40 2024-07-17T07:10:25.594 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:25.595 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:10:25.595 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:10:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 msgr2=0x7fa5c4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 0x7fa5c4079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005fd0 tx=0x7fa5d8009500 comp rx=0 tx=0).stop 2024-07-17T07:10:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 msgr2=0x7fa5e81f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000d7d0 tx=0x7fa5d000dc90 comp rx=0 tx=0).stop 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 shutdown_connections 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fa5c4076bb0 0x7fa5c4079070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5e815a2b0 0x7fa5e80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5e815aa40 0x7fa5e81f7d10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 --2- 172.21.15.44:0/29685513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5cc004d40 0x7fa5e80a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 >> 172.21.15.44:0/29685513 conn(0x7fa5e80cfec0 msgr2=0x7fa5e815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.598+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 shutdown_connections 2024-07-17T07:10:25.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.599+0000 7fa5e7200640 1 -- 172.21.15.44:0/29685513 wait complete. 2024-07-17T07:10:25.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:10:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 -- 172.21.15.44:0/140021010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c005210 msgr2=0x7ff95c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 --2- 172.21.15.44:0/140021010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c005210 0x7ff95c00b9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff970001d90 tx=0x7ff95c005830 comp rx=0 tx=0).stop 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 -- 172.21.15.44:0/140021010 shutdown_connections 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 --2- 172.21.15.44:0/140021010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c005210 0x7ff95c00b9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 -- 172.21.15.44:0/140021010 >> 172.21.15.44:0/140021010 conn(0x7ff9780b0fb0 msgr2=0x7ff9780b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.834+0000 7ff97cc00640 1 -- 172.21.15.44:0/140021010 shutdown_connections 2024-07-17T07:10:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.835+0000 7ff97cc00640 1 -- 172.21.15.44:0/140021010 wait complete. 2024-07-17T07:10:25.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.836+0000 7ff97cc00640 1 Processor -- start 2024-07-17T07:10:25.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.837+0000 7ff97cc00640 1 -- start start 2024-07-17T07:10:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff95c005210 0x7ff978072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff978152970 0x7ff97814f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780adb20 con 0x7ff9781f2590 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9780ad9a0 con 0x7ff978152970 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff97cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780ac220 con 0x7ff95c005210 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff976800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff976800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37916/0 (socket says 172.21.15.44:37916) 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff976800640 1 -- 172.21.15.44:0/1855239744 learned_addr learned my addr 172.21.15.44:0/1855239744 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff975e00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff95c005210 0x7ff978072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.838+0000 7ff96d400640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff978152970 0x7ff97814f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 -- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff95c005210 msgr2=0x7ff978072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff95c005210 0x7ff978072c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 -- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff978152970 msgr2=0x7ff97814f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:25.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff978152970 0x7ff97814f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:25.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 -- 172.21.15.44:0/1855239744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978073510 con 0x7ff9781f2590 2024-07-17T07:10:25.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff976800640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff970001eb0 tx=0x7ff970006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:25.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970003d80 con 0x7ff9781f2590 2024-07-17T07:10:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff970005c20 con 0x7ff9781f2590 2024-07-17T07:10:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9781f1ab0 con 0x7ff9781f2590 2024-07-17T07:10:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970006070 con 0x7ff9781f2590 2024-07-17T07:10:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.839+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781f2070 con 0x7ff9781f2590 2024-07-17T07:10:25.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.841+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97815ee90 con 0x7ff9781f2590 2024-07-17T07:10:25.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.842+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7ff9700050d0 con 0x7ff9781f2590 2024-07-17T07:10:25.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.843+0000 7ff96ea00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 0x7ff954079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:25.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.843+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7ff9700b8680 con 0x7ff9781f2590 2024-07-17T07:10:25.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.844+0000 7ff975e00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 0x7ff954079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.845+0000 7ff975e00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 0x7ff954079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9680057c0 tx=0x7ff968009630 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:25.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:25.847+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff970043060 con 0x7ff9781f2590 2024-07-17T07:10:26.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.059+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9781519a0 con 0x7ff9781f2590 2024-07-17T07:10:26.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.062+0000 7ff96ea00640 1 -- 172.21.15.44:0/1855239744 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+17658 (secure 0 0 0) 0x7ff970081830 con 0x7ff9781f2590 2024-07-17T07:10:26.061 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:26.061 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":83,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:10:23.064200+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:10:19.413303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-07-17T07:10:20.496271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:10:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.065+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 msgr2=0x7ff954079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.065+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 0x7ff954079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9680057c0 tx=0x7ff968009630 comp rx=0 tx=0).stop 2024-07-17T07:10:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.065+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 msgr2=0x7ff978148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.065+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff970001eb0 tx=0x7ff970006d80 comp rx=0 tx=0).stop 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 shutdown_connections 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9781f2590 0x7ff978148140 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7ff954076bb0 0x7ff954079070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff978152970 0x7ff97814f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 --2- 172.21.15.44:0/1855239744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff95c005210 0x7ff978072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 >> 172.21.15.44:0/1855239744 conn(0x7ff9780b0fb0 msgr2=0x7ff97815e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 shutdown_connections 2024-07-17T07:10:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.066+0000 7ff97cc00640 1 -- 172.21.15.44:0/1855239744 wait complete. 2024-07-17T07:10:26.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:10:26.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.288+0000 7f2032200640 1 -- 172.21.15.44:0/3508016560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2010003a00 msgr2=0x7f2010003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.288+0000 7f2032200640 1 --2- 172.21.15.44:0/3508016560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2010003a00 0x7f2010003e00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f201c001eb0 tx=0x7f2010005830 comp rx=0 tx=0).stop 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.288+0000 7f2032200640 1 -- 172.21.15.44:0/3508016560 shutdown_connections 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.288+0000 7f2032200640 1 --2- 172.21.15.44:0/3508016560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2010003a00 0x7f2010003e00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.288+0000 7f2032200640 1 -- 172.21.15.44:0/3508016560 >> 172.21.15.44:0/3508016560 conn(0x7f202c0b0fb0 msgr2=0x7f202c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:26.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.289+0000 7f2032200640 1 -- 172.21.15.44:0/3508016560 shutdown_connections 2024-07-17T07:10:26.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.289+0000 7f2032200640 1 -- 172.21.15.44:0/3508016560 wait complete. 2024-07-17T07:10:26.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.290+0000 7f2032200640 1 Processor -- start 2024-07-17T07:10:26.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.290+0000 7f2032200640 1 -- start start 2024-07-17T07:10:26.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.291+0000 7f2032200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 0x7f202c1e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.291+0000 7f2032200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.291+0000 7f2032200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f202c14f330 0x7f202c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f2032200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0ac810 con 0x7f202c1e8380 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f2032200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0ac690 con 0x7f2010003a00 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f2032200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f202c0ac990 con 0x7f202c14f330 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37934/0 (socket says 172.21.15.44:37934) 2024-07-17T07:10:26.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 -- 172.21.15.44:0/1182576681 learned_addr learned my addr 172.21.15.44:0/1182576681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:26.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202b400640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 0x7f202c1e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202be00640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f202c14f330 0x7f202c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 -- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f202c14f330 msgr2=0x7f202c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f202c14f330 0x7f202c1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 -- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 msgr2=0x7f202c1e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 0x7f202c1e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202aa00640 1 -- 172.21.15.44:0/1182576681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c14f7e0 con 0x7f202c1e8380 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.292+0000 7f202b400640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 0x7f202c1e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f202aa00640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f201800e690 tx=0x7f201800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800ca50 con 0x7f202c1e8380 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201800cce0 con 0x7f202c1e8380 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018005280 con 0x7f202c1e8380 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c154df0 con 0x7f202c1e8380 2024-07-17T07:10:26.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.293+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c1553b0 con 0x7f202c1e8380 2024-07-17T07:10:26.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.294+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c0adaa0 con 0x7f202c1e8380 2024-07-17T07:10:26.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.295+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f2018005420 con 0x7f202c1e8380 2024-07-17T07:10:26.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.296+0000 7f2023400640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 0x7f200c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.297+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f2018098c80 con 0x7f202c1e8380 2024-07-17T07:10:26.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.297+0000 7f202b400640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 0x7f200c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.298+0000 7f202b400640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 0x7f200c0790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f201c033000 tx=0x7f201c001a30 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:26.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.301+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2018016030 con 0x7f202c1e8380 2024-07-17T07:10:26.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.509+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 --> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f202c0b2660 con 0x7f200c076be0 2024-07-17T07:10:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.525+0000 7f2023400640 1 -- 172.21.15.44:0/1182576681 <== mgr.5404 v2:172.21.15.175:6800/2949346159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251997 (secure 0 0 0) 0x7f202c0b2660 con 0x7f200c076be0 2024-07-17T07:10:26.528 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 msgr2=0x7f200c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 0x7f200c0790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f201c033000 tx=0x7f201c001a30 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 msgr2=0x7f202c14edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f201800e690 tx=0x7f201800eb50 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 shutdown_connections 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f202c14f330 0x7f202c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c1e8380 0x7f202c14edf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f200c076be0 0x7f200c0790a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 --2- 172.21.15.44:0/1182576681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2010003a00 0x7f202c1e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 >> 172.21.15.44:0/1182576681 conn(0x7f202c0b0fb0 msgr2=0x7f202c15df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 shutdown_connections 2024-07-17T07:10:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.531+0000 7f2032200640 1 -- 172.21.15.44:0/1182576681 wait complete. 2024-07-17T07:10:26.550 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:10:25.866547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":194,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341724,"kb_used_data":8996,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754632996,"statfs":{"total":773094113280,"available":772744187904,"internally_reserved":0,"allocated":9211904,"data_stored":5261704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.855326"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292828+0000","last_change":"2024-07-17T07:10:22.084417+0000","last_active":"2024-07-17T07:10:23.292828+0000","last_peered":"2024-07-17T07:10:23.292828+0000","last_clean":"2024-07-17T07:10:23.292828+0000","last_became_active":"2024-07-17T07:10:22.083923+0000","last_became_peered":"2024-07-17T07:10:22.083923+0000","last_unstale":"2024-07-17T07:10:23.292828+0000","last_undegraded":"2024-07-17T07:10:23.292828+0000","last_fullsized":"2024-07-17T07:10:23.292828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:12:36.081261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292633+0000","last_change":"2024-07-17T07:10:22.094856+0000","last_active":"2024-07-17T07:10:23.292633+0000","last_peered":"2024-07-17T07:10:23.292633+0000","last_clean":"2024-07-17T07:10:23.292633+0000","last_became_active":"2024-07-17T07:10:22.093442+0000","last_became_peered":"2024-07-17T07:10:22.093442+0000","last_unstale":"2024-07-17T07:10:23.292633+0000","last_undegraded":"2024-07-17T07:10:23.292633+0000","last_fullsized":"2024-07-17T07:10:23.292633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:52.525267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.096143+0000","last_change":"2024-07-17T07:10:22.093941+0000","last_active":"2024-07-17T07:10:22.096143+0000","last_peered":"2024-07-17T07:10:22.096143+0000","last_clean":"2024-07-17T07:10:22.096143+0000","last_became_active":"2024-07-17T07:10:22.093039+0000","last_became_peered":"2024-07-17T07:10:22.093039+0000","last_unstale":"2024-07-17T07:10:22.096143+0000","last_undegraded":"2024-07-17T07:10:22.096143+0000","last_fullsized":"2024-07-17T07:10:22.096143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:25:36.240909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089056+0000","last_change":"2024-07-17T07:10:22.085109+0000","last_active":"2024-07-17T07:10:22.089056+0000","last_peered":"2024-07-17T07:10:22.089056+0000","last_clean":"2024-07-17T07:10:22.089056+0000","last_became_active":"2024-07-17T07:10:22.084749+0000","last_became_peered":"2024-07-17T07:10:22.084749+0000","last_unstale":"2024-07-17T07:10:22.089056+0000","last_undegraded":"2024-07-17T07:10:22.089056+0000","last_fullsized":"2024-07-17T07:10:22.089056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:20.680613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083701+0000","last_change":"2024-07-17T07:10:22.074773+0000","last_active":"2024-07-17T07:10:23.083701+0000","last_peered":"2024-07-17T07:10:23.083701+0000","last_clean":"2024-07-17T07:10:23.083701+0000","last_became_active":"2024-07-17T07:10:22.074125+0000","last_became_peered":"2024-07-17T07:10:22.074125+0000","last_unstale":"2024-07-17T07:10:23.083701+0000","last_undegraded":"2024-07-17T07:10:23.083701+0000","last_fullsized":"2024-07-17T07:10:23.083701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:52.525267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082097+0000","last_change":"2024-07-17T07:10:22.081553+0000","last_active":"2024-07-17T07:10:23.082097+0000","last_peered":"2024-07-17T07:10:23.082097+0000","last_clean":"2024-07-17T07:10:23.082097+0000","last_became_active":"2024-07-17T07:10:22.081143+0000","last_became_peered":"2024-07-17T07:10:22.081143+0000","last_unstale":"2024-07-17T07:10:23.082097+0000","last_undegraded":"2024-07-17T07:10:23.082097+0000","last_fullsized":"2024-07-17T07:10:23.082097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:56.953433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080434+0000","last_change":"2024-07-17T07:10:22.077756+0000","last_active":"2024-07-17T07:10:23.080434+0000","last_peered":"2024-07-17T07:10:23.080434+0000","last_clean":"2024-07-17T07:10:23.080434+0000","last_became_active":"2024-07-17T07:10:22.076956+0000","last_became_peered":"2024-07-17T07:10:22.076956+0000","last_unstale":"2024-07-17T07:10:23.080434+0000","last_undegraded":"2024-07-17T07:10:23.080434+0000","last_fullsized":"2024-07-17T07:10:23.080434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:38.932663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089281+0000","last_change":"2024-07-17T07:10:22.070662+0000","last_active":"2024-07-17T07:10:23.089281+0000","last_peered":"2024-07-17T07:10:23.089281+0000","last_clean":"2024-07-17T07:10:23.089281+0000","last_became_active":"2024-07-17T07:10:22.070373+0000","last_became_peered":"2024-07-17T07:10:22.070373+0000","last_unstale":"2024-07-17T07:10:23.089281+0000","last_undegraded":"2024-07-17T07:10:23.089281+0000","last_fullsized":"2024-07-17T07:10:23.089281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:03.156146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.083839+0000","last_change":"2024-07-17T07:10:22.081650+0000","last_active":"2024-07-17T07:10:22.083839+0000","last_peered":"2024-07-17T07:10:22.083839+0000","last_clean":"2024-07-17T07:10:22.083839+0000","last_became_active":"2024-07-17T07:10:22.081363+0000","last_became_peered":"2024-07-17T07:10:22.081363+0000","last_unstale":"2024-07-17T07:10:22.083839+0000","last_undegraded":"2024-07-17T07:10:22.083839+0000","last_fullsized":"2024-07-17T07:10:22.083839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:41:04.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.094006+0000","last_change":"2024-07-17T07:10:22.092470+0000","last_active":"2024-07-17T07:10:22.094006+0000","last_peered":"2024-07-17T07:10:22.094006+0000","last_clean":"2024-07-17T07:10:22.094006+0000","last_became_active":"2024-07-17T07:10:22.092022+0000","last_became_peered":"2024-07-17T07:10:22.092022+0000","last_unstale":"2024-07-17T07:10:22.094006+0000","last_undegraded":"2024-07-17T07:10:22.094006+0000","last_fullsized":"2024-07-17T07:10:22.094006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:48:56.465765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093390+0000","last_change":"2024-07-17T07:10:22.091246+0000","last_active":"2024-07-17T07:10:22.093390+0000","last_peered":"2024-07-17T07:10:22.093390+0000","last_clean":"2024-07-17T07:10:22.093390+0000","last_became_active":"2024-07-17T07:10:22.090989+0000","last_became_peered":"2024-07-17T07:10:22.090989+0000","last_unstale":"2024-07-17T07:10:22.093390+0000","last_undegraded":"2024-07-17T07:10:22.093390+0000","last_fullsized":"2024-07-17T07:10:22.093390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:28:10.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088982+0000","last_change":"2024-07-17T07:10:22.077477+0000","last_active":"2024-07-17T07:10:23.088982+0000","last_peered":"2024-07-17T07:10:23.088982+0000","last_clean":"2024-07-17T07:10:23.088982+0000","last_became_active":"2024-07-17T07:10:22.077193+0000","last_became_peered":"2024-07-17T07:10:22.077193+0000","last_unstale":"2024-07-17T07:10:23.088982+0000","last_undegraded":"2024-07-17T07:10:23.088982+0000","last_fullsized":"2024-07-17T07:10:23.088982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:24.612656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089335+0000","last_change":"2024-07-17T07:10:22.081524+0000","last_active":"2024-07-17T07:10:23.089335+0000","last_peered":"2024-07-17T07:10:23.089335+0000","last_clean":"2024-07-17T07:10:23.089335+0000","last_became_active":"2024-07-17T07:10:22.081232+0000","last_became_peered":"2024-07-17T07:10:22.081232+0000","last_unstale":"2024-07-17T07:10:23.089335+0000","last_undegraded":"2024-07-17T07:10:23.089335+0000","last_fullsized":"2024-07-17T07:10:23.089335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:15.343132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093991+0000","last_change":"2024-07-17T07:10:22.089321+0000","last_active":"2024-07-17T07:10:22.093991+0000","last_peered":"2024-07-17T07:10:22.093991+0000","last_clean":"2024-07-17T07:10:22.093991+0000","last_became_active":"2024-07-17T07:10:22.087490+0000","last_became_peered":"2024-07-17T07:10:22.087490+0000","last_unstale":"2024-07-17T07:10:22.093991+0000","last_undegraded":"2024-07-17T07:10:22.093991+0000","last_fullsized":"2024-07-17T07:10:22.093991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:07.344298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093885+0000","last_change":"2024-07-17T07:10:22.091099+0000","last_active":"2024-07-17T07:10:22.093885+0000","last_peered":"2024-07-17T07:10:22.093885+0000","last_clean":"2024-07-17T07:10:22.093885+0000","last_became_active":"2024-07-17T07:10:22.090809+0000","last_became_peered":"2024-07-17T07:10:22.090809+0000","last_unstale":"2024-07-17T07:10:22.093885+0000","last_undegraded":"2024-07-17T07:10:22.093885+0000","last_fullsized":"2024-07-17T07:10:22.093885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:25:51.470485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080184+0000","last_change":"2024-07-17T07:10:22.064491+0000","last_active":"2024-07-17T07:10:23.080184+0000","last_peered":"2024-07-17T07:10:23.080184+0000","last_clean":"2024-07-17T07:10:23.080184+0000","last_became_active":"2024-07-17T07:10:22.063760+0000","last_became_peered":"2024-07-17T07:10:22.063760+0000","last_unstale":"2024-07-17T07:10:23.080184+0000","last_undegraded":"2024-07-17T07:10:23.080184+0000","last_fullsized":"2024-07-17T07:10:23.080184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:15.744790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292949+0000","last_change":"2024-07-17T07:10:22.083829+0000","last_active":"2024-07-17T07:10:23.292949+0000","last_peered":"2024-07-17T07:10:23.292949+0000","last_clean":"2024-07-17T07:10:23.292949+0000","last_became_active":"2024-07-17T07:10:22.083486+0000","last_became_peered":"2024-07-17T07:10:22.083486+0000","last_unstale":"2024-07-17T07:10:23.292949+0000","last_undegraded":"2024-07-17T07:10:23.292949+0000","last_fullsized":"2024-07-17T07:10:23.292949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:55.839853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.096293+0000","last_change":"2024-07-17T07:10:22.092565+0000","last_active":"2024-07-17T07:10:22.096293+0000","last_peered":"2024-07-17T07:10:22.096293+0000","last_clean":"2024-07-17T07:10:22.096293+0000","last_became_active":"2024-07-17T07:10:22.092217+0000","last_became_peered":"2024-07-17T07:10:22.092217+0000","last_unstale":"2024-07-17T07:10:22.096293+0000","last_undegraded":"2024-07-17T07:10:22.096293+0000","last_fullsized":"2024-07-17T07:10:22.096293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:49.355947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083822+0000","last_change":"2024-07-17T07:10:22.063575+0000","last_active":"2024-07-17T07:10:23.083822+0000","last_peered":"2024-07-17T07:10:23.083822+0000","last_clean":"2024-07-17T07:10:23.083822+0000","last_became_active":"2024-07-17T07:10:22.063220+0000","last_became_peered":"2024-07-17T07:10:22.063220+0000","last_unstale":"2024-07-17T07:10:23.083822+0000","last_undegraded":"2024-07-17T07:10:23.083822+0000","last_fullsized":"2024-07-17T07:10:23.083822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:55.839853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080877+0000","last_change":"2024-07-17T07:10:22.069716+0000","last_active":"2024-07-17T07:10:22.080877+0000","last_peered":"2024-07-17T07:10:22.080877+0000","last_clean":"2024-07-17T07:10:22.080877+0000","last_became_active":"2024-07-17T07:10:22.069028+0000","last_became_peered":"2024-07-17T07:10:22.069028+0000","last_unstale":"2024-07-17T07:10:22.080877+0000","last_undegraded":"2024-07-17T07:10:22.080877+0000","last_fullsized":"2024-07-17T07:10:22.080877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:34:11.542173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088961+0000","last_change":"2024-07-17T07:10:22.088829+0000","last_active":"2024-07-17T07:10:23.088961+0000","last_peered":"2024-07-17T07:10:23.088961+0000","last_clean":"2024-07-17T07:10:23.088961+0000","last_became_active":"2024-07-17T07:10:22.088031+0000","last_became_peered":"2024-07-17T07:10:22.088031+0000","last_unstale":"2024-07-17T07:10:23.088961+0000","last_undegraded":"2024-07-17T07:10:23.088961+0000","last_fullsized":"2024-07-17T07:10:23.088961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:55:48.373358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080831+0000","last_change":"2024-07-17T07:10:22.072100+0000","last_active":"2024-07-17T07:10:22.080831+0000","last_peered":"2024-07-17T07:10:22.080831+0000","last_clean":"2024-07-17T07:10:22.080831+0000","last_became_active":"2024-07-17T07:10:22.071707+0000","last_became_peered":"2024-07-17T07:10:22.071707+0000","last_unstale":"2024-07-17T07:10:22.080831+0000","last_undegraded":"2024-07-17T07:10:22.080831+0000","last_fullsized":"2024-07-17T07:10:22.080831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:56.953433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.081118+0000","last_change":"2024-07-17T07:10:22.076942+0000","last_active":"2024-07-17T07:10:22.081118+0000","last_peered":"2024-07-17T07:10:22.081118+0000","last_clean":"2024-07-17T07:10:22.081118+0000","last_became_active":"2024-07-17T07:10:22.076661+0000","last_became_peered":"2024-07-17T07:10:22.076661+0000","last_unstale":"2024-07-17T07:10:22.081118+0000","last_undegraded":"2024-07-17T07:10:22.081118+0000","last_fullsized":"2024-07-17T07:10:22.081118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:22.814127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.091327+0000","last_change":"2024-07-17T07:10:22.088627+0000","last_active":"2024-07-17T07:10:22.091327+0000","last_peered":"2024-07-17T07:10:22.091327+0000","last_clean":"2024-07-17T07:10:22.091327+0000","last_became_active":"2024-07-17T07:10:22.088033+0000","last_became_peered":"2024-07-17T07:10:22.088033+0000","last_unstale":"2024-07-17T07:10:22.091327+0000","last_undegraded":"2024-07-17T07:10:22.091327+0000","last_fullsized":"2024-07-17T07:10:22.091327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:11:37.188561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089641+0000","last_change":"2024-07-17T07:10:22.064537+0000","last_active":"2024-07-17T07:10:23.089641+0000","last_peered":"2024-07-17T07:10:23.089641+0000","last_clean":"2024-07-17T07:10:23.089641+0000","last_became_active":"2024-07-17T07:10:22.064167+0000","last_became_peered":"2024-07-17T07:10:22.064167+0000","last_unstale":"2024-07-17T07:10:23.089641+0000","last_undegraded":"2024-07-17T07:10:23.089641+0000","last_fullsized":"2024-07-17T07:10:23.089641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:09.620340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088966+0000","last_change":"2024-07-17T07:10:22.065980+0000","last_active":"2024-07-17T07:10:23.088966+0000","last_peered":"2024-07-17T07:10:23.088966+0000","last_clean":"2024-07-17T07:10:23.088966+0000","last_became_active":"2024-07-17T07:10:22.065658+0000","last_became_peered":"2024-07-17T07:10:22.065658+0000","last_unstale":"2024-07-17T07:10:23.088966+0000","last_undegraded":"2024-07-17T07:10:23.088966+0000","last_fullsized":"2024-07-17T07:10:23.088966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:37.872709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.088872+0000","last_change":"2024-07-17T07:10:22.085039+0000","last_active":"2024-07-17T07:10:22.088872+0000","last_peered":"2024-07-17T07:10:22.088872+0000","last_clean":"2024-07-17T07:10:22.088872+0000","last_became_active":"2024-07-17T07:10:22.084638+0000","last_became_peered":"2024-07-17T07:10:22.084638+0000","last_unstale":"2024-07-17T07:10:22.088872+0000","last_undegraded":"2024-07-17T07:10:22.088872+0000","last_fullsized":"2024-07-17T07:10:22.088872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:29.228050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292298+0000","last_change":"2024-07-17T07:10:22.089390+0000","last_active":"2024-07-17T07:10:23.292298+0000","last_peered":"2024-07-17T07:10:23.292298+0000","last_clean":"2024-07-17T07:10:23.292298+0000","last_became_active":"2024-07-17T07:10:22.089046+0000","last_became_peered":"2024-07-17T07:10:22.089046+0000","last_unstale":"2024-07-17T07:10:23.292298+0000","last_undegraded":"2024-07-17T07:10:23.292298+0000","last_fullsized":"2024-07-17T07:10:23.292298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:26.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.084553+0000","last_change":"2024-07-17T07:10:22.083024+0000","last_active":"2024-07-17T07:10:22.084553+0000","last_peered":"2024-07-17T07:10:22.084553+0000","last_clean":"2024-07-17T07:10:22.084553+0000","last_became_active":"2024-07-17T07:10:22.082533+0000","last_became_peered":"2024-07-17T07:10:22.082533+0000","last_unstale":"2024-07-17T07:10:22.084553+0000","last_undegraded":"2024-07-17T07:10:22.084553+0000","last_fullsized":"2024-07-17T07:10:22.084553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:31.848232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089371+0000","last_change":"2024-07-17T07:10:22.074465+0000","last_active":"2024-07-17T07:10:23.089371+0000","last_peered":"2024-07-17T07:10:23.089371+0000","last_clean":"2024-07-17T07:10:23.089371+0000","last_became_active":"2024-07-17T07:10:22.072940+0000","last_became_peered":"2024-07-17T07:10:22.072940+0000","last_unstale":"2024-07-17T07:10:23.089371+0000","last_undegraded":"2024-07-17T07:10:23.089371+0000","last_fullsized":"2024-07-17T07:10:23.089371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:42.938529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089290+0000","last_change":"2024-07-17T07:10:22.086891+0000","last_active":"2024-07-17T07:10:22.089290+0000","last_peered":"2024-07-17T07:10:22.089290+0000","last_clean":"2024-07-17T07:10:22.089290+0000","last_became_active":"2024-07-17T07:10:22.084680+0000","last_became_peered":"2024-07-17T07:10:22.084680+0000","last_unstale":"2024-07-17T07:10:22.089290+0000","last_undegraded":"2024-07-17T07:10:22.089290+0000","last_fullsized":"2024-07-17T07:10:22.089290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:38.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081588+0000","last_change":"2024-07-17T07:10:22.086956+0000","last_active":"2024-07-17T07:10:23.081588+0000","last_peered":"2024-07-17T07:10:23.081588+0000","last_clean":"2024-07-17T07:10:23.081588+0000","last_became_active":"2024-07-17T07:10:22.086678+0000","last_became_peered":"2024-07-17T07:10:22.086678+0000","last_unstale":"2024-07-17T07:10:23.081588+0000","last_undegraded":"2024-07-17T07:10:23.081588+0000","last_fullsized":"2024-07-17T07:10:23.081588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:34:11.542173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083715+0000","last_change":"2024-07-17T07:10:22.090902+0000","last_active":"2024-07-17T07:10:23.083715+0000","last_peered":"2024-07-17T07:10:23.083715+0000","last_clean":"2024-07-17T07:10:23.083715+0000","last_became_active":"2024-07-17T07:10:22.089365+0000","last_became_peered":"2024-07-17T07:10:22.089365+0000","last_unstale":"2024-07-17T07:10:23.083715+0000","last_undegraded":"2024-07-17T07:10:23.083715+0000","last_fullsized":"2024-07-17T07:10:23.083715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:12:36.081261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.293083+0000","last_change":"2024-07-17T07:10:21.052818+0000","last_active":"2024-07-17T07:10:23.293083+0000","last_peered":"2024-07-17T07:10:23.293083+0000","last_clean":"2024-07-17T07:10:23.293083+0000","last_became_active":"2024-07-17T07:10:21.051600+0000","last_became_peered":"2024-07-17T07:10:21.051600+0000","last_unstale":"2024-07-17T07:10:23.293083+0000","last_undegraded":"2024-07-17T07:10:23.293083+0000","last_fullsized":"2024-07-17T07:10:23.293083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:04.326536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.086940+0000","last_change":"2024-07-17T07:10:22.081846+0000","last_active":"2024-07-17T07:10:22.086940+0000","last_peered":"2024-07-17T07:10:22.086940+0000","last_clean":"2024-07-17T07:10:22.086940+0000","last_became_active":"2024-07-17T07:10:22.079659+0000","last_became_peered":"2024-07-17T07:10:22.079659+0000","last_unstale":"2024-07-17T07:10:22.086940+0000","last_undegraded":"2024-07-17T07:10:22.086940+0000","last_fullsized":"2024-07-17T07:10:22.086940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:53.886348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082682+0000","last_change":"2024-07-17T07:10:21.047045+0000","last_active":"2024-07-17T07:10:23.082682+0000","last_peered":"2024-07-17T07:10:23.082682+0000","last_clean":"2024-07-17T07:10:23.082682+0000","last_became_active":"2024-07-17T07:10:21.046763+0000","last_became_peered":"2024-07-17T07:10:21.046763+0000","last_unstale":"2024-07-17T07:10:23.082682+0000","last_undegraded":"2024-07-17T07:10:23.082682+0000","last_fullsized":"2024-07-17T07:10:23.082682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:02.141650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082470+0000","last_change":"2024-07-17T07:10:22.086215+0000","last_active":"2024-07-17T07:10:23.082470+0000","last_peered":"2024-07-17T07:10:23.082470+0000","last_clean":"2024-07-17T07:10:23.082470+0000","last_became_active":"2024-07-17T07:10:22.085742+0000","last_became_peered":"2024-07-17T07:10:22.085742+0000","last_unstale":"2024-07-17T07:10:23.082470+0000","last_undegraded":"2024-07-17T07:10:23.082470+0000","last_fullsized":"2024-07-17T07:10:23.082470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:30.279694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082470+0000","last_change":"2024-07-17T07:10:21.044817+0000","last_active":"2024-07-17T07:10:23.082470+0000","last_peered":"2024-07-17T07:10:23.082470+0000","last_clean":"2024-07-17T07:10:23.082470+0000","last_became_active":"2024-07-17T07:10:21.044467+0000","last_became_peered":"2024-07-17T07:10:21.044467+0000","last_unstale":"2024-07-17T07:10:23.082470+0000","last_undegraded":"2024-07-17T07:10:23.082470+0000","last_fullsized":"2024-07-17T07:10:23.082470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:26.514197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.095425+0000","last_change":"2024-07-17T07:10:22.093252+0000","last_active":"2024-07-17T07:10:22.095425+0000","last_peered":"2024-07-17T07:10:22.095425+0000","last_clean":"2024-07-17T07:10:22.095425+0000","last_became_active":"2024-07-17T07:10:22.091863+0000","last_became_peered":"2024-07-17T07:10:22.091863+0000","last_unstale":"2024-07-17T07:10:22.095425+0000","last_undegraded":"2024-07-17T07:10:22.095425+0000","last_fullsized":"2024-07-17T07:10:22.095425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:48:41.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082412+0000","last_change":"2024-07-17T07:10:21.038105+0000","last_active":"2024-07-17T07:10:23.082412+0000","last_peered":"2024-07-17T07:10:23.082412+0000","last_clean":"2024-07-17T07:10:23.082412+0000","last_became_active":"2024-07-17T07:10:21.037781+0000","last_became_peered":"2024-07-17T07:10:21.037781+0000","last_unstale":"2024-07-17T07:10:23.082412+0000","last_undegraded":"2024-07-17T07:10:23.082412+0000","last_fullsized":"2024-07-17T07:10:23.082412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:55:47.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.079843+0000","last_change":"2024-07-17T07:10:22.089262+0000","last_active":"2024-07-17T07:10:23.079843+0000","last_peered":"2024-07-17T07:10:23.079843+0000","last_clean":"2024-07-17T07:10:23.079843+0000","last_became_active":"2024-07-17T07:10:22.088968+0000","last_became_peered":"2024-07-17T07:10:22.088968+0000","last_unstale":"2024-07-17T07:10:23.079843+0000","last_undegraded":"2024-07-17T07:10:23.079843+0000","last_fullsized":"2024-07-17T07:10:23.079843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:59:54.573674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103283+0000","last_change":"2024-07-17T07:10:21.046295+0000","last_active":"2024-07-17T07:10:22.103283+0000","last_peered":"2024-07-17T07:10:22.103283+0000","last_clean":"2024-07-17T07:10:22.103283+0000","last_became_active":"2024-07-17T07:10:21.045951+0000","last_became_peered":"2024-07-17T07:10:21.045951+0000","last_unstale":"2024-07-17T07:10:22.103283+0000","last_undegraded":"2024-07-17T07:10:22.103283+0000","last_fullsized":"2024-07-17T07:10:22.103283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:14.328636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291526+0000","last_change":"2024-07-17T07:10:22.090718+0000","last_active":"2024-07-17T07:10:23.291526+0000","last_peered":"2024-07-17T07:10:23.291526+0000","last_clean":"2024-07-17T07:10:23.291526+0000","last_became_active":"2024-07-17T07:10:22.090409+0000","last_became_peered":"2024-07-17T07:10:22.090409+0000","last_unstale":"2024-07-17T07:10:23.291526+0000","last_undegraded":"2024-07-17T07:10:23.291526+0000","last_fullsized":"2024-07-17T07:10:23.291526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:14.654859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.102458+0000","last_change":"2024-07-17T07:10:21.040347+0000","last_active":"2024-07-17T07:10:22.102458+0000","last_peered":"2024-07-17T07:10:22.102458+0000","last_clean":"2024-07-17T07:10:22.102458+0000","last_became_active":"2024-07-17T07:10:21.040026+0000","last_became_peered":"2024-07-17T07:10:21.040026+0000","last_unstale":"2024-07-17T07:10:22.102458+0000","last_undegraded":"2024-07-17T07:10:22.102458+0000","last_fullsized":"2024-07-17T07:10:22.102458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:02.141650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082539+0000","last_change":"2024-07-17T07:10:22.077794+0000","last_active":"2024-07-17T07:10:23.082539+0000","last_peered":"2024-07-17T07:10:23.082539+0000","last_clean":"2024-07-17T07:10:23.082539+0000","last_became_active":"2024-07-17T07:10:22.077522+0000","last_became_peered":"2024-07-17T07:10:22.077522+0000","last_unstale":"2024-07-17T07:10:23.082539+0000","last_undegraded":"2024-07-17T07:10:23.082539+0000","last_fullsized":"2024-07-17T07:10:23.082539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:20.893102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055160+0000","last_change":"2024-07-17T07:10:21.048982+0000","last_active":"2024-07-17T07:10:22.055160+0000","last_peered":"2024-07-17T07:10:22.055160+0000","last_clean":"2024-07-17T07:10:22.055160+0000","last_became_active":"2024-07-17T07:10:21.048689+0000","last_became_peered":"2024-07-17T07:10:21.048689+0000","last_unstale":"2024-07-17T07:10:22.055160+0000","last_undegraded":"2024-07-17T07:10:22.055160+0000","last_fullsized":"2024-07-17T07:10:22.055160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:23.598160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089460+0000","last_change":"2024-07-17T07:10:22.086890+0000","last_active":"2024-07-17T07:10:22.089460+0000","last_peered":"2024-07-17T07:10:22.089460+0000","last_clean":"2024-07-17T07:10:22.089460+0000","last_became_active":"2024-07-17T07:10:22.084049+0000","last_became_peered":"2024-07-17T07:10:22.084049+0000","last_unstale":"2024-07-17T07:10:22.089460+0000","last_undegraded":"2024-07-17T07:10:22.089460+0000","last_fullsized":"2024-07-17T07:10:22.089460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:30.279694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082335+0000","last_change":"2024-07-17T07:10:21.051458+0000","last_active":"2024-07-17T07:10:23.082335+0000","last_peered":"2024-07-17T07:10:23.082335+0000","last_clean":"2024-07-17T07:10:23.082335+0000","last_became_active":"2024-07-17T07:10:21.050443+0000","last_became_peered":"2024-07-17T07:10:21.050443+0000","last_unstale":"2024-07-17T07:10:23.082335+0000","last_undegraded":"2024-07-17T07:10:23.082335+0000","last_fullsized":"2024-07-17T07:10:23.082335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:47.380726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292517+0000","last_change":"2024-07-17T07:10:22.084767+0000","last_active":"2024-07-17T07:10:23.292517+0000","last_peered":"2024-07-17T07:10:23.292517+0000","last_clean":"2024-07-17T07:10:23.292517+0000","last_became_active":"2024-07-17T07:10:22.084412+0000","last_became_peered":"2024-07-17T07:10:22.084412+0000","last_unstale":"2024-07-17T07:10:23.292517+0000","last_undegraded":"2024-07-17T07:10:23.292517+0000","last_fullsized":"2024-07-17T07:10:23.292517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:47.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.067274+0000","last_change":"2024-07-17T07:10:21.056692+0000","last_active":"2024-07-17T07:10:22.067274+0000","last_peered":"2024-07-17T07:10:22.067274+0000","last_clean":"2024-07-17T07:10:22.067274+0000","last_became_active":"2024-07-17T07:10:21.056258+0000","last_became_peered":"2024-07-17T07:10:21.056258+0000","last_unstale":"2024-07-17T07:10:22.067274+0000","last_undegraded":"2024-07-17T07:10:22.067274+0000","last_fullsized":"2024-07-17T07:10:22.067274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:41.081662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080462+0000","last_change":"2024-07-17T07:10:22.072045+0000","last_active":"2024-07-17T07:10:22.080462+0000","last_peered":"2024-07-17T07:10:22.080462+0000","last_clean":"2024-07-17T07:10:22.080462+0000","last_became_active":"2024-07-17T07:10:22.069225+0000","last_became_peered":"2024-07-17T07:10:22.069225+0000","last_unstale":"2024-07-17T07:10:22.080462+0000","last_undegraded":"2024-07-17T07:10:22.080462+0000","last_fullsized":"2024-07-17T07:10:22.080462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:49.193369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080808+0000","last_change":"2024-07-17T07:10:21.049788+0000","last_active":"2024-07-17T07:10:22.080808+0000","last_peered":"2024-07-17T07:10:22.080808+0000","last_clean":"2024-07-17T07:10:22.080808+0000","last_became_active":"2024-07-17T07:10:21.049228+0000","last_became_peered":"2024-07-17T07:10:21.049228+0000","last_unstale":"2024-07-17T07:10:22.080808+0000","last_undegraded":"2024-07-17T07:10:22.080808+0000","last_fullsized":"2024-07-17T07:10:22.080808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:15:47.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084025+0000","last_change":"2024-07-17T07:10:22.074026+0000","last_active":"2024-07-17T07:10:23.084025+0000","last_peered":"2024-07-17T07:10:23.084025+0000","last_clean":"2024-07-17T07:10:23.084025+0000","last_became_active":"2024-07-17T07:10:22.070134+0000","last_became_peered":"2024-07-17T07:10:22.070134+0000","last_unstale":"2024-07-17T07:10:23.084025+0000","last_undegraded":"2024-07-17T07:10:23.084025+0000","last_fullsized":"2024-07-17T07:10:23.084025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:12.187590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103821+0000","last_change":"2024-07-17T07:10:21.057272+0000","last_active":"2024-07-17T07:10:22.103821+0000","last_peered":"2024-07-17T07:10:22.103821+0000","last_clean":"2024-07-17T07:10:22.103821+0000","last_became_active":"2024-07-17T07:10:21.056200+0000","last_became_peered":"2024-07-17T07:10:21.056200+0000","last_unstale":"2024-07-17T07:10:22.103821+0000","last_undegraded":"2024-07-17T07:10:22.103821+0000","last_fullsized":"2024-07-17T07:10:22.103821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:45:51.092604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089216+0000","last_change":"2024-07-17T07:10:22.086812+0000","last_active":"2024-07-17T07:10:22.089216+0000","last_peered":"2024-07-17T07:10:22.089216+0000","last_clean":"2024-07-17T07:10:22.089216+0000","last_became_active":"2024-07-17T07:10:22.084782+0000","last_became_peered":"2024-07-17T07:10:22.084782+0000","last_unstale":"2024-07-17T07:10:22.089216+0000","last_undegraded":"2024-07-17T07:10:22.089216+0000","last_fullsized":"2024-07-17T07:10:22.089216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:57.103510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292702+0000","last_change":"2024-07-17T07:10:21.046340+0000","last_active":"2024-07-17T07:10:23.292702+0000","last_peered":"2024-07-17T07:10:23.292702+0000","last_clean":"2024-07-17T07:10:23.292702+0000","last_became_active":"2024-07-17T07:10:21.046007+0000","last_became_peered":"2024-07-17T07:10:21.046007+0000","last_unstale":"2024-07-17T07:10:23.292702+0000","last_undegraded":"2024-07-17T07:10:23.292702+0000","last_fullsized":"2024-07-17T07:10:23.292702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:28:09.967835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.091740+0000","last_change":"2024-07-17T07:10:22.088895+0000","last_active":"2024-07-17T07:10:22.091740+0000","last_peered":"2024-07-17T07:10:22.091740+0000","last_clean":"2024-07-17T07:10:22.091740+0000","last_became_active":"2024-07-17T07:10:22.087194+0000","last_became_peered":"2024-07-17T07:10:22.087194+0000","last_unstale":"2024-07-17T07:10:22.091740+0000","last_undegraded":"2024-07-17T07:10:22.091740+0000","last_fullsized":"2024-07-17T07:10:22.091740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:11.140319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.098504+0000","last_change":"2024-07-17T07:10:21.057461+0000","last_active":"2024-07-17T07:10:22.098504+0000","last_peered":"2024-07-17T07:10:22.098504+0000","last_clean":"2024-07-17T07:10:22.098504+0000","last_became_active":"2024-07-17T07:10:21.056630+0000","last_became_peered":"2024-07-17T07:10:21.056630+0000","last_unstale":"2024-07-17T07:10:22.098504+0000","last_undegraded":"2024-07-17T07:10:22.098504+0000","last_fullsized":"2024-07-17T07:10:22.098504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:40.370255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088605+0000","last_change":"2024-07-17T07:10:22.079352+0000","last_active":"2024-07-17T07:10:23.088605+0000","last_peered":"2024-07-17T07:10:23.088605+0000","last_clean":"2024-07-17T07:10:23.088605+0000","last_became_active":"2024-07-17T07:10:22.078209+0000","last_became_peered":"2024-07-17T07:10:22.078209+0000","last_unstale":"2024-07-17T07:10:23.088605+0000","last_undegraded":"2024-07-17T07:10:23.088605+0000","last_fullsized":"2024-07-17T07:10:23.088605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:17.152776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088514+0000","last_change":"2024-07-17T07:10:21.052329+0000","last_active":"2024-07-17T07:10:23.088514+0000","last_peered":"2024-07-17T07:10:23.088514+0000","last_clean":"2024-07-17T07:10:23.088514+0000","last_became_active":"2024-07-17T07:10:21.052038+0000","last_became_peered":"2024-07-17T07:10:21.052038+0000","last_unstale":"2024-07-17T07:10:23.088514+0000","last_undegraded":"2024-07-17T07:10:23.088514+0000","last_fullsized":"2024-07-17T07:10:23.088514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:11.822024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083137+0000","last_change":"2024-07-17T07:10:22.064026+0000","last_active":"2024-07-17T07:10:23.083137+0000","last_peered":"2024-07-17T07:10:23.083137+0000","last_clean":"2024-07-17T07:10:23.083137+0000","last_became_active":"2024-07-17T07:10:22.063482+0000","last_became_peered":"2024-07-17T07:10:22.063482+0000","last_unstale":"2024-07-17T07:10:23.083137+0000","last_undegraded":"2024-07-17T07:10:23.083137+0000","last_fullsized":"2024-07-17T07:10:23.083137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:26.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.069498+0000","last_change":"2024-07-17T07:10:21.054502+0000","last_active":"2024-07-17T07:10:22.069498+0000","last_peered":"2024-07-17T07:10:22.069498+0000","last_clean":"2024-07-17T07:10:22.069498+0000","last_became_active":"2024-07-17T07:10:21.054163+0000","last_became_peered":"2024-07-17T07:10:21.054163+0000","last_unstale":"2024-07-17T07:10:22.069498+0000","last_undegraded":"2024-07-17T07:10:22.069498+0000","last_fullsized":"2024-07-17T07:10:22.069498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:08.019938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292058+0000","last_change":"2024-07-17T07:10:22.087884+0000","last_active":"2024-07-17T07:10:23.292058+0000","last_peered":"2024-07-17T07:10:23.292058+0000","last_clean":"2024-07-17T07:10:23.292058+0000","last_became_active":"2024-07-17T07:10:22.083772+0000","last_became_peered":"2024-07-17T07:10:22.083772+0000","last_unstale":"2024-07-17T07:10:23.292058+0000","last_undegraded":"2024-07-17T07:10:23.292058+0000","last_fullsized":"2024-07-17T07:10:23.292058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:20.893102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.104951+0000","last_change":"2024-07-17T07:10:21.052821+0000","last_active":"2024-07-17T07:10:22.104951+0000","last_peered":"2024-07-17T07:10:22.104951+0000","last_clean":"2024-07-17T07:10:22.104951+0000","last_became_active":"2024-07-17T07:10:21.052494+0000","last_became_peered":"2024-07-17T07:10:21.052494+0000","last_unstale":"2024-07-17T07:10:22.104951+0000","last_undegraded":"2024-07-17T07:10:22.104951+0000","last_fullsized":"2024-07-17T07:10:22.104951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:34.947536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.083684+0000","last_change":"2024-07-17T07:10:22.081150+0000","last_active":"2024-07-17T07:10:22.083684+0000","last_peered":"2024-07-17T07:10:22.083684+0000","last_clean":"2024-07-17T07:10:22.083684+0000","last_became_active":"2024-07-17T07:10:22.080891+0000","last_became_peered":"2024-07-17T07:10:22.080891+0000","last_unstale":"2024-07-17T07:10:22.083684+0000","last_undegraded":"2024-07-17T07:10:22.083684+0000","last_fullsized":"2024-07-17T07:10:22.083684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:30.821998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089560+0000","last_change":"2024-07-17T07:10:21.053977+0000","last_active":"2024-07-17T07:10:23.089560+0000","last_peered":"2024-07-17T07:10:23.089560+0000","last_clean":"2024-07-17T07:10:23.089560+0000","last_became_active":"2024-07-17T07:10:21.053336+0000","last_became_peered":"2024-07-17T07:10:21.053336+0000","last_unstale":"2024-07-17T07:10:23.089560+0000","last_undegraded":"2024-07-17T07:10:23.089560+0000","last_fullsized":"2024-07-17T07:10:23.089560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:39.050407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083913+0000","last_change":"2024-07-17T07:10:22.088610+0000","last_active":"2024-07-17T07:10:23.083913+0000","last_peered":"2024-07-17T07:10:23.083913+0000","last_clean":"2024-07-17T07:10:23.083913+0000","last_became_active":"2024-07-17T07:10:22.088314+0000","last_became_peered":"2024-07-17T07:10:22.088314+0000","last_unstale":"2024-07-17T07:10:23.083913+0000","last_undegraded":"2024-07-17T07:10:23.083913+0000","last_fullsized":"2024-07-17T07:10:23.083913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:34:44.274509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.097537+0000","last_change":"2024-07-17T07:10:21.059797+0000","last_active":"2024-07-17T07:10:22.097537+0000","last_peered":"2024-07-17T07:10:22.097537+0000","last_clean":"2024-07-17T07:10:22.097537+0000","last_became_active":"2024-07-17T07:10:21.059483+0000","last_became_peered":"2024-07-17T07:10:21.059483+0000","last_unstale":"2024-07-17T07:10:22.097537+0000","last_undegraded":"2024-07-17T07:10:22.097537+0000","last_fullsized":"2024-07-17T07:10:22.097537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:39.370581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089591+0000","last_change":"2024-07-17T07:10:22.088063+0000","last_active":"2024-07-17T07:10:22.089591+0000","last_peered":"2024-07-17T07:10:22.089591+0000","last_clean":"2024-07-17T07:10:22.089591+0000","last_became_active":"2024-07-17T07:10:22.087748+0000","last_became_peered":"2024-07-17T07:10:22.087748+0000","last_unstale":"2024-07-17T07:10:22.089591+0000","last_undegraded":"2024-07-17T07:10:22.089591+0000","last_fullsized":"2024-07-17T07:10:22.089591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:25.639980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103682+0000","last_change":"2024-07-17T07:10:21.057507+0000","last_active":"2024-07-17T07:10:22.103682+0000","last_peered":"2024-07-17T07:10:22.103682+0000","last_clean":"2024-07-17T07:10:22.103682+0000","last_became_active":"2024-07-17T07:10:21.056767+0000","last_became_peered":"2024-07-17T07:10:21.056767+0000","last_unstale":"2024-07-17T07:10:22.103682+0000","last_undegraded":"2024-07-17T07:10:22.103682+0000","last_fullsized":"2024-07-17T07:10:22.103682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:29.213844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084016+0000","last_change":"2024-07-17T07:10:22.076189+0000","last_active":"2024-07-17T07:10:23.084016+0000","last_peered":"2024-07-17T07:10:23.084016+0000","last_clean":"2024-07-17T07:10:23.084016+0000","last_became_active":"2024-07-17T07:10:22.075901+0000","last_became_peered":"2024-07-17T07:10:22.075901+0000","last_unstale":"2024-07-17T07:10:23.084016+0000","last_undegraded":"2024-07-17T07:10:23.084016+0000","last_fullsized":"2024-07-17T07:10:23.084016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:52.205565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.056051+0000","last_change":"2024-07-17T07:10:21.052952+0000","last_active":"2024-07-17T07:10:22.056051+0000","last_peered":"2024-07-17T07:10:22.056051+0000","last_clean":"2024-07-17T07:10:22.056051+0000","last_became_active":"2024-07-17T07:10:21.052672+0000","last_became_peered":"2024-07-17T07:10:21.052672+0000","last_unstale":"2024-07-17T07:10:22.056051+0000","last_undegraded":"2024-07-17T07:10:22.056051+0000","last_fullsized":"2024-07-17T07:10:22.056051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:41.924033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081697+0000","last_change":"2024-07-17T07:10:22.086145+0000","last_active":"2024-07-17T07:10:23.081697+0000","last_peered":"2024-07-17T07:10:23.081697+0000","last_clean":"2024-07-17T07:10:23.081697+0000","last_became_active":"2024-07-17T07:10:22.085844+0000","last_became_peered":"2024-07-17T07:10:22.085844+0000","last_unstale":"2024-07-17T07:10:23.081697+0000","last_undegraded":"2024-07-17T07:10:23.081697+0000","last_fullsized":"2024-07-17T07:10:23.081697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:49.193369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082108+0000","last_change":"2024-07-17T07:10:21.049022+0000","last_active":"2024-07-17T07:10:23.082108+0000","last_peered":"2024-07-17T07:10:23.082108+0000","last_clean":"2024-07-17T07:10:23.082108+0000","last_became_active":"2024-07-17T07:10:21.048631+0000","last_became_peered":"2024-07-17T07:10:21.048631+0000","last_unstale":"2024-07-17T07:10:23.082108+0000","last_undegraded":"2024-07-17T07:10:23.082108+0000","last_fullsized":"2024-07-17T07:10:23.082108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:36.858213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088751+0000","last_change":"2024-07-17T07:10:22.060115+0000","last_active":"2024-07-17T07:10:23.088751+0000","last_peered":"2024-07-17T07:10:23.088751+0000","last_clean":"2024-07-17T07:10:23.088751+0000","last_became_active":"2024-07-17T07:10:22.059556+0000","last_became_peered":"2024-07-17T07:10:22.059556+0000","last_unstale":"2024-07-17T07:10:23.088751+0000","last_undegraded":"2024-07-17T07:10:23.088751+0000","last_fullsized":"2024-07-17T07:10:23.088751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:27.528693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088903+0000","last_change":"2024-07-17T07:10:21.050445+0000","last_active":"2024-07-17T07:10:23.088903+0000","last_peered":"2024-07-17T07:10:23.088903+0000","last_clean":"2024-07-17T07:10:23.088903+0000","last_became_active":"2024-07-17T07:10:21.050162+0000","last_became_peered":"2024-07-17T07:10:21.050162+0000","last_unstale":"2024-07-17T07:10:23.088903+0000","last_undegraded":"2024-07-17T07:10:23.088903+0000","last_fullsized":"2024-07-17T07:10:23.088903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:52.259420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083241+0000","last_change":"2024-07-17T07:10:22.077771+0000","last_active":"2024-07-17T07:10:23.083241+0000","last_peered":"2024-07-17T07:10:23.083241+0000","last_clean":"2024-07-17T07:10:23.083241+0000","last_became_active":"2024-07-17T07:10:22.077467+0000","last_became_peered":"2024-07-17T07:10:22.077467+0000","last_unstale":"2024-07-17T07:10:23.083241+0000","last_undegraded":"2024-07-17T07:10:23.083241+0000","last_fullsized":"2024-07-17T07:10:23.083241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:47.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081734+0000","last_change":"2024-07-17T07:10:21.048119+0000","last_active":"2024-07-17T07:10:23.081734+0000","last_peered":"2024-07-17T07:10:23.081734+0000","last_clean":"2024-07-17T07:10:23.081734+0000","last_became_active":"2024-07-17T07:10:21.046894+0000","last_became_peered":"2024-07-17T07:10:21.046894+0000","last_unstale":"2024-07-17T07:10:23.081734+0000","last_undegraded":"2024-07-17T07:10:23.081734+0000","last_fullsized":"2024-07-17T07:10:23.081734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:50:40.506854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082250+0000","last_change":"2024-07-17T07:10:22.089208+0000","last_active":"2024-07-17T07:10:23.082250+0000","last_peered":"2024-07-17T07:10:23.082250+0000","last_clean":"2024-07-17T07:10:23.082250+0000","last_became_active":"2024-07-17T07:10:22.088948+0000","last_became_peered":"2024-07-17T07:10:22.088948+0000","last_unstale":"2024-07-17T07:10:23.082250+0000","last_undegraded":"2024-07-17T07:10:23.082250+0000","last_fullsized":"2024-07-17T07:10:23.082250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:22.814127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089968+0000","last_change":"2024-07-17T07:10:21.054727+0000","last_active":"2024-07-17T07:10:23.089968+0000","last_peered":"2024-07-17T07:10:23.089968+0000","last_clean":"2024-07-17T07:10:23.089968+0000","last_became_active":"2024-07-17T07:10:21.053692+0000","last_became_peered":"2024-07-17T07:10:21.053692+0000","last_unstale":"2024-07-17T07:10:23.089968+0000","last_undegraded":"2024-07-17T07:10:23.089968+0000","last_fullsized":"2024-07-17T07:10:23.089968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:01.790345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080616+0000","last_change":"2024-07-17T07:10:22.093054+0000","last_active":"2024-07-17T07:10:23.080616+0000","last_peered":"2024-07-17T07:10:23.080616+0000","last_clean":"2024-07-17T07:10:23.080616+0000","last_became_active":"2024-07-17T07:10:22.092768+0000","last_became_peered":"2024-07-17T07:10:22.092768+0000","last_unstale":"2024-07-17T07:10:23.080616+0000","last_undegraded":"2024-07-17T07:10:23.080616+0000","last_fullsized":"2024-07-17T07:10:23.080616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:01.377047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.091633+0000","last_change":"2024-07-17T07:10:21.047992+0000","last_active":"2024-07-17T07:10:23.091633+0000","last_peered":"2024-07-17T07:10:23.091633+0000","last_clean":"2024-07-17T07:10:23.091633+0000","last_became_active":"2024-07-17T07:10:21.047697+0000","last_became_peered":"2024-07-17T07:10:21.047697+0000","last_unstale":"2024-07-17T07:10:23.091633+0000","last_undegraded":"2024-07-17T07:10:23.091633+0000","last_fullsized":"2024-07-17T07:10:23.091633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:23.419050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089326+0000","last_change":"2024-07-17T07:10:22.078141+0000","last_active":"2024-07-17T07:10:23.089326+0000","last_peered":"2024-07-17T07:10:23.089326+0000","last_clean":"2024-07-17T07:10:23.089326+0000","last_became_active":"2024-07-17T07:10:22.077782+0000","last_became_peered":"2024-07-17T07:10:22.077782+0000","last_unstale":"2024-07-17T07:10:23.089326+0000","last_undegraded":"2024-07-17T07:10:23.089326+0000","last_fullsized":"2024-07-17T07:10:23.089326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:53.804147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082457+0000","last_change":"2024-07-17T07:10:21.044077+0000","last_active":"2024-07-17T07:10:23.082457+0000","last_peered":"2024-07-17T07:10:23.082457+0000","last_clean":"2024-07-17T07:10:23.082457+0000","last_became_active":"2024-07-17T07:10:21.043774+0000","last_became_peered":"2024-07-17T07:10:21.043774+0000","last_unstale":"2024-07-17T07:10:23.082457+0000","last_undegraded":"2024-07-17T07:10:23.082457+0000","last_fullsized":"2024-07-17T07:10:23.082457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:23.598160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084223+0000","last_change":"2024-07-17T07:10:22.074734+0000","last_active":"2024-07-17T07:10:23.084223+0000","last_peered":"2024-07-17T07:10:23.084223+0000","last_clean":"2024-07-17T07:10:23.084223+0000","last_became_active":"2024-07-17T07:10:22.074330+0000","last_became_peered":"2024-07-17T07:10:22.074330+0000","last_unstale":"2024-07-17T07:10:23.084223+0000","last_undegraded":"2024-07-17T07:10:23.084223+0000","last_fullsized":"2024-07-17T07:10:23.084223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:32.605393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292805+0000","last_change":"2024-07-17T07:10:21.045945+0000","last_active":"2024-07-17T07:10:23.292805+0000","last_peered":"2024-07-17T07:10:23.292805+0000","last_clean":"2024-07-17T07:10:23.292805+0000","last_became_active":"2024-07-17T07:10:21.045641+0000","last_became_peered":"2024-07-17T07:10:21.045641+0000","last_unstale":"2024-07-17T07:10:23.292805+0000","last_undegraded":"2024-07-17T07:10:23.292805+0000","last_fullsized":"2024-07-17T07:10:23.292805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:47.380726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082410+0000","last_change":"2024-07-17T07:10:22.085585+0000","last_active":"2024-07-17T07:10:23.082410+0000","last_peered":"2024-07-17T07:10:23.082410+0000","last_clean":"2024-07-17T07:10:23.082410+0000","last_became_active":"2024-07-17T07:10:22.084470+0000","last_became_peered":"2024-07-17T07:10:22.084470+0000","last_unstale":"2024-07-17T07:10:23.082410+0000","last_undegraded":"2024-07-17T07:10:23.082410+0000","last_fullsized":"2024-07-17T07:10:23.082410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:46.007961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.067211+0000","last_change":"2024-07-17T07:10:21.055213+0000","last_active":"2024-07-17T07:10:22.067211+0000","last_peered":"2024-07-17T07:10:22.067211+0000","last_clean":"2024-07-17T07:10:22.067211+0000","last_became_active":"2024-07-17T07:10:21.054927+0000","last_became_peered":"2024-07-17T07:10:21.054927+0000","last_unstale":"2024-07-17T07:10:22.067211+0000","last_undegraded":"2024-07-17T07:10:22.067211+0000","last_fullsized":"2024-07-17T07:10:22.067211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:00.485092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.087050+0000","last_change":"2024-07-17T07:10:22.080907+0000","last_active":"2024-07-17T07:10:22.087050+0000","last_peered":"2024-07-17T07:10:22.087050+0000","last_clean":"2024-07-17T07:10:22.087050+0000","last_became_active":"2024-07-17T07:10:22.080601+0000","last_became_peered":"2024-07-17T07:10:22.080601+0000","last_unstale":"2024-07-17T07:10:22.087050+0000","last_undegraded":"2024-07-17T07:10:22.087050+0000","last_fullsized":"2024-07-17T07:10:22.087050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:07.606628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.056129+0000","last_change":"2024-07-17T07:10:21.035972+0000","last_active":"2024-07-17T07:10:22.056129+0000","last_peered":"2024-07-17T07:10:22.056129+0000","last_clean":"2024-07-17T07:10:22.056129+0000","last_became_active":"2024-07-17T07:10:21.035653+0000","last_became_peered":"2024-07-17T07:10:21.035653+0000","last_unstale":"2024-07-17T07:10:22.056129+0000","last_undegraded":"2024-07-17T07:10:22.056129+0000","last_fullsized":"2024-07-17T07:10:22.056129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:08.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291643+0000","last_change":"2024-07-17T07:10:22.091068+0000","last_active":"2024-07-17T07:10:23.291643+0000","last_peered":"2024-07-17T07:10:23.291643+0000","last_clean":"2024-07-17T07:10:23.291643+0000","last_became_active":"2024-07-17T07:10:22.085844+0000","last_became_peered":"2024-07-17T07:10:22.085844+0000","last_unstale":"2024-07-17T07:10:23.291643+0000","last_undegraded":"2024-07-17T07:10:23.291643+0000","last_fullsized":"2024-07-17T07:10:23.291643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:46.007961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055653+0000","last_change":"2024-07-17T07:10:21.048954+0000","last_active":"2024-07-17T07:10:22.055653+0000","last_peered":"2024-07-17T07:10:22.055653+0000","last_clean":"2024-07-17T07:10:22.055653+0000","last_became_active":"2024-07-17T07:10:21.048658+0000","last_became_peered":"2024-07-17T07:10:21.048658+0000","last_unstale":"2024-07-17T07:10:22.055653+0000","last_undegraded":"2024-07-17T07:10:22.055653+0000","last_fullsized":"2024-07-17T07:10:22.055653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:52.789651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080093+0000","last_change":"2024-07-17T07:10:22.072465+0000","last_active":"2024-07-17T07:10:23.080093+0000","last_peered":"2024-07-17T07:10:23.080093+0000","last_clean":"2024-07-17T07:10:23.080093+0000","last_became_active":"2024-07-17T07:10:22.071969+0000","last_became_peered":"2024-07-17T07:10:22.071969+0000","last_unstale":"2024-07-17T07:10:23.080093+0000","last_undegraded":"2024-07-17T07:10:23.080093+0000","last_fullsized":"2024-07-17T07:10:23.080093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:33.219013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291331+0000","last_change":"2024-07-17T07:10:21.043741+0000","last_active":"2024-07-17T07:10:23.291331+0000","last_peered":"2024-07-17T07:10:23.291331+0000","last_clean":"2024-07-17T07:10:23.291331+0000","last_became_active":"2024-07-17T07:10:21.043445+0000","last_became_peered":"2024-07-17T07:10:21.043445+0000","last_unstale":"2024-07-17T07:10:23.291331+0000","last_undegraded":"2024-07-17T07:10:23.291331+0000","last_fullsized":"2024-07-17T07:10:23.291331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:14.898452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083956+0000","last_change":"2024-07-17T07:10:22.086624+0000","last_active":"2024-07-17T07:10:23.083956+0000","last_peered":"2024-07-17T07:10:23.083956+0000","last_clean":"2024-07-17T07:10:23.083956+0000","last_became_active":"2024-07-17T07:10:22.086161+0000","last_became_peered":"2024-07-17T07:10:22.086161+0000","last_unstale":"2024-07-17T07:10:23.083956+0000","last_undegraded":"2024-07-17T07:10:23.083956+0000","last_fullsized":"2024-07-17T07:10:23.083956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:01.564545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055085+0000","last_change":"2024-07-17T07:10:21.043557+0000","last_active":"2024-07-17T07:10:22.055085+0000","last_peered":"2024-07-17T07:10:22.055085+0000","last_clean":"2024-07-17T07:10:22.055085+0000","last_became_active":"2024-07-17T07:10:21.043237+0000","last_became_peered":"2024-07-17T07:10:21.043237+0000","last_unstale":"2024-07-17T07:10:22.055085+0000","last_undegraded":"2024-07-17T07:10:22.055085+0000","last_fullsized":"2024-07-17T07:10:22.055085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:36.858213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'100","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.063831+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:10:22.063831+0000","last_peered":"2024-07-17T07:10:22.063831+0000","last_clean":"2024-07-17T07:10:22.063831+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:10:22.063831+0000","last_undegraded":"2024-07-17T07:10:22.063831+0000","last_fullsized":"2024-07-17T07:10:22.063831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542333,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59108,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312732,"statfs":{"total":96636764160,"available":96576237568,"internally_reserved":0,"allocated":1597440,"data_stored":1096155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509628,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":1003520,"data_stored":512257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50208,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321632,"statfs":{"total":96636764160,"available":96585351168,"internally_reserved":0,"allocated":1003520,"data_stored":512245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":506623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":987136,"data_stored":505011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542332,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58528,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313312,"statfs":{"total":96636764160,"available":96576831488,"internally_reserved":0,"allocated":999424,"data_stored":512223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509629,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":514599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509628,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1593344,"data_stored":1102591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:10:26.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:10:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.805+0000 7f2b33000640 1 -- 172.21.15.44:0/3050840705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 msgr2=0x7f2b10003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.805+0000 7f2b33000640 1 --2- 172.21.15.44:0/3050840705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b10003e90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009b20 tx=0x7f2b10005830 comp rx=0 tx=0).stop 2024-07-17T07:10:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.805+0000 7f2b32600640 1 -- 172.21.15.44:0/3050840705 reap_dead start 2024-07-17T07:10:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.806+0000 7f2b33000640 1 -- 172.21.15.44:0/3050840705 shutdown_connections 2024-07-17T07:10:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.806+0000 7f2b33000640 1 -- 172.21.15.44:0/3050840705 >> 172.21.15.44:0/3050840705 conn(0x7f2b2c06ce80 msgr2=0x7f2b2c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.806+0000 7f2b33000640 1 -- 172.21.15.44:0/3050840705 shutdown_connections 2024-07-17T07:10:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.806+0000 7f2b33000640 1 -- 172.21.15.44:0/3050840705 wait complete. 2024-07-17T07:10:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.807+0000 7f2b33000640 1 Processor -- start 2024-07-17T07:10:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.807+0000 7f2b33000640 1 -- start start 2024-07-17T07:10:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 0x7f2b2c1f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b2c1f3de0 0x7f2b2c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0b40c0 con 0x7f2b10003a70 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0ae030 con 0x7f2b2c1f74b0 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.808+0000 7f2b33000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c0b4240 con 0x7f2b2c1f3de0 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37946/0 (socket says 172.21.15.44:37946) 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 -- 172.21.15.44:0/1241267168 learned_addr learned my addr 172.21.15.44:0/1241267168 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31200640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 0x7f2b2c1f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 -- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b2c1f3de0 msgr2=0x7f2b2c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b2c1f3de0 0x7f2b2c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 -- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 msgr2=0x7f2b2c1f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 0x7f2b2c1f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31c00640 1 -- 172.21.15.44:0/1241267168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c1f4750 con 0x7f2b10003a70 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.809+0000 7f2b31200640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 0x7f2b2c1f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b31c00640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036040 tx=0x7f2b1c033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:26.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c038040 con 0x7f2b10003a70 2024-07-17T07:10:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b1c004020 con 0x7f2b10003a70 2024-07-17T07:10:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c004450 con 0x7f2b10003a70 2024-07-17T07:10:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c1f4a30 con 0x7f2b10003a70 2024-07-17T07:10:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.810+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c154660 con 0x7f2b10003a70 2024-07-17T07:10:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.811+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c06e7f0 con 0x7f2b10003a70 2024-07-17T07:10:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.812+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f2b1c045020 con 0x7f2b10003a70 2024-07-17T07:10:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.813+0000 7f2b29600640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 0x7f2b0c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.813+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f2b1c0b8e40 con 0x7f2b10003a70 2024-07-17T07:10:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.814+0000 7f2b31200640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 0x7f2b0c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.815+0000 7f2b31200640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 0x7f2b0c079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1f78b0 tx=0x7f2b18001620 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:26.818+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2b1c081ee0 con 0x7f2b10003a70 2024-07-17T07:10:27.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.020+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 --> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b2c0b1070 con 0x7f2b0c076bb0 2024-07-17T07:10:27.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.034+0000 7f2b29600640 1 -- 172.21.15.44:0/1241267168 <== mgr.5404 v2:172.21.15.175:6800/2949346159 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251997 (secure 0 0 0) 0x7f2b2c0b1070 con 0x7f2b0c076bb0 2024-07-17T07:10:27.037 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:27.039 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 msgr2=0x7f2b0c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 0x7f2b0c079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1f78b0 tx=0x7f2b18001620 comp rx=0 tx=0).stop 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 msgr2=0x7f2b2c1f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036040 tx=0x7f2b1c033650 comp rx=0 tx=0).stop 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 shutdown_connections 2024-07-17T07:10:27.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b2c1f3de0 0x7f2b2c0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f2b0c076bb0 0x7f2b0c079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2c1f74b0 0x7f2b2c1f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 --2- 172.21.15.44:0/1241267168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b10003a70 0x7f2b2c1f4fa0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 >> 172.21.15.44:0/1241267168 conn(0x7f2b2c06ce80 msgr2=0x7f2b2c072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 shutdown_connections 2024-07-17T07:10:27.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.041+0000 7f2b33000640 1 -- 172.21.15.44:0/1241267168 wait complete. 2024-07-17T07:10:27.058 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:10:25.866547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":194,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341724,"kb_used_data":8996,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754632996,"statfs":{"total":773094113280,"available":772744187904,"internally_reserved":0,"allocated":9211904,"data_stored":5261704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.855326"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292828+0000","last_change":"2024-07-17T07:10:22.084417+0000","last_active":"2024-07-17T07:10:23.292828+0000","last_peered":"2024-07-17T07:10:23.292828+0000","last_clean":"2024-07-17T07:10:23.292828+0000","last_became_active":"2024-07-17T07:10:22.083923+0000","last_became_peered":"2024-07-17T07:10:22.083923+0000","last_unstale":"2024-07-17T07:10:23.292828+0000","last_undegraded":"2024-07-17T07:10:23.292828+0000","last_fullsized":"2024-07-17T07:10:23.292828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:12:36.081261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292633+0000","last_change":"2024-07-17T07:10:22.094856+0000","last_active":"2024-07-17T07:10:23.292633+0000","last_peered":"2024-07-17T07:10:23.292633+0000","last_clean":"2024-07-17T07:10:23.292633+0000","last_became_active":"2024-07-17T07:10:22.093442+0000","last_became_peered":"2024-07-17T07:10:22.093442+0000","last_unstale":"2024-07-17T07:10:23.292633+0000","last_undegraded":"2024-07-17T07:10:23.292633+0000","last_fullsized":"2024-07-17T07:10:23.292633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:52.525267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.096143+0000","last_change":"2024-07-17T07:10:22.093941+0000","last_active":"2024-07-17T07:10:22.096143+0000","last_peered":"2024-07-17T07:10:22.096143+0000","last_clean":"2024-07-17T07:10:22.096143+0000","last_became_active":"2024-07-17T07:10:22.093039+0000","last_became_peered":"2024-07-17T07:10:22.093039+0000","last_unstale":"2024-07-17T07:10:22.096143+0000","last_undegraded":"2024-07-17T07:10:22.096143+0000","last_fullsized":"2024-07-17T07:10:22.096143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:25:36.240909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089056+0000","last_change":"2024-07-17T07:10:22.085109+0000","last_active":"2024-07-17T07:10:22.089056+0000","last_peered":"2024-07-17T07:10:22.089056+0000","last_clean":"2024-07-17T07:10:22.089056+0000","last_became_active":"2024-07-17T07:10:22.084749+0000","last_became_peered":"2024-07-17T07:10:22.084749+0000","last_unstale":"2024-07-17T07:10:22.089056+0000","last_undegraded":"2024-07-17T07:10:22.089056+0000","last_fullsized":"2024-07-17T07:10:22.089056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:20.680613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083701+0000","last_change":"2024-07-17T07:10:22.074773+0000","last_active":"2024-07-17T07:10:23.083701+0000","last_peered":"2024-07-17T07:10:23.083701+0000","last_clean":"2024-07-17T07:10:23.083701+0000","last_became_active":"2024-07-17T07:10:22.074125+0000","last_became_peered":"2024-07-17T07:10:22.074125+0000","last_unstale":"2024-07-17T07:10:23.083701+0000","last_undegraded":"2024-07-17T07:10:23.083701+0000","last_fullsized":"2024-07-17T07:10:23.083701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:52.525267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082097+0000","last_change":"2024-07-17T07:10:22.081553+0000","last_active":"2024-07-17T07:10:23.082097+0000","last_peered":"2024-07-17T07:10:23.082097+0000","last_clean":"2024-07-17T07:10:23.082097+0000","last_became_active":"2024-07-17T07:10:22.081143+0000","last_became_peered":"2024-07-17T07:10:22.081143+0000","last_unstale":"2024-07-17T07:10:23.082097+0000","last_undegraded":"2024-07-17T07:10:23.082097+0000","last_fullsized":"2024-07-17T07:10:23.082097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:56.953433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080434+0000","last_change":"2024-07-17T07:10:22.077756+0000","last_active":"2024-07-17T07:10:23.080434+0000","last_peered":"2024-07-17T07:10:23.080434+0000","last_clean":"2024-07-17T07:10:23.080434+0000","last_became_active":"2024-07-17T07:10:22.076956+0000","last_became_peered":"2024-07-17T07:10:22.076956+0000","last_unstale":"2024-07-17T07:10:23.080434+0000","last_undegraded":"2024-07-17T07:10:23.080434+0000","last_fullsized":"2024-07-17T07:10:23.080434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:38.932663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089281+0000","last_change":"2024-07-17T07:10:22.070662+0000","last_active":"2024-07-17T07:10:23.089281+0000","last_peered":"2024-07-17T07:10:23.089281+0000","last_clean":"2024-07-17T07:10:23.089281+0000","last_became_active":"2024-07-17T07:10:22.070373+0000","last_became_peered":"2024-07-17T07:10:22.070373+0000","last_unstale":"2024-07-17T07:10:23.089281+0000","last_undegraded":"2024-07-17T07:10:23.089281+0000","last_fullsized":"2024-07-17T07:10:23.089281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:03.156146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.083839+0000","last_change":"2024-07-17T07:10:22.081650+0000","last_active":"2024-07-17T07:10:22.083839+0000","last_peered":"2024-07-17T07:10:22.083839+0000","last_clean":"2024-07-17T07:10:22.083839+0000","last_became_active":"2024-07-17T07:10:22.081363+0000","last_became_peered":"2024-07-17T07:10:22.081363+0000","last_unstale":"2024-07-17T07:10:22.083839+0000","last_undegraded":"2024-07-17T07:10:22.083839+0000","last_fullsized":"2024-07-17T07:10:22.083839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:41:04.793750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.094006+0000","last_change":"2024-07-17T07:10:22.092470+0000","last_active":"2024-07-17T07:10:22.094006+0000","last_peered":"2024-07-17T07:10:22.094006+0000","last_clean":"2024-07-17T07:10:22.094006+0000","last_became_active":"2024-07-17T07:10:22.092022+0000","last_became_peered":"2024-07-17T07:10:22.092022+0000","last_unstale":"2024-07-17T07:10:22.094006+0000","last_undegraded":"2024-07-17T07:10:22.094006+0000","last_fullsized":"2024-07-17T07:10:22.094006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:48:56.465765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093390+0000","last_change":"2024-07-17T07:10:22.091246+0000","last_active":"2024-07-17T07:10:22.093390+0000","last_peered":"2024-07-17T07:10:22.093390+0000","last_clean":"2024-07-17T07:10:22.093390+0000","last_became_active":"2024-07-17T07:10:22.090989+0000","last_became_peered":"2024-07-17T07:10:22.090989+0000","last_unstale":"2024-07-17T07:10:22.093390+0000","last_undegraded":"2024-07-17T07:10:22.093390+0000","last_fullsized":"2024-07-17T07:10:22.093390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:28:10.862031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088982+0000","last_change":"2024-07-17T07:10:22.077477+0000","last_active":"2024-07-17T07:10:23.088982+0000","last_peered":"2024-07-17T07:10:23.088982+0000","last_clean":"2024-07-17T07:10:23.088982+0000","last_became_active":"2024-07-17T07:10:22.077193+0000","last_became_peered":"2024-07-17T07:10:22.077193+0000","last_unstale":"2024-07-17T07:10:23.088982+0000","last_undegraded":"2024-07-17T07:10:23.088982+0000","last_fullsized":"2024-07-17T07:10:23.088982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:24.612656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089335+0000","last_change":"2024-07-17T07:10:22.081524+0000","last_active":"2024-07-17T07:10:23.089335+0000","last_peered":"2024-07-17T07:10:23.089335+0000","last_clean":"2024-07-17T07:10:23.089335+0000","last_became_active":"2024-07-17T07:10:22.081232+0000","last_became_peered":"2024-07-17T07:10:22.081232+0000","last_unstale":"2024-07-17T07:10:23.089335+0000","last_undegraded":"2024-07-17T07:10:23.089335+0000","last_fullsized":"2024-07-17T07:10:23.089335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:15.343132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093991+0000","last_change":"2024-07-17T07:10:22.089321+0000","last_active":"2024-07-17T07:10:22.093991+0000","last_peered":"2024-07-17T07:10:22.093991+0000","last_clean":"2024-07-17T07:10:22.093991+0000","last_became_active":"2024-07-17T07:10:22.087490+0000","last_became_peered":"2024-07-17T07:10:22.087490+0000","last_unstale":"2024-07-17T07:10:22.093991+0000","last_undegraded":"2024-07-17T07:10:22.093991+0000","last_fullsized":"2024-07-17T07:10:22.093991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:07.344298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.093885+0000","last_change":"2024-07-17T07:10:22.091099+0000","last_active":"2024-07-17T07:10:22.093885+0000","last_peered":"2024-07-17T07:10:22.093885+0000","last_clean":"2024-07-17T07:10:22.093885+0000","last_became_active":"2024-07-17T07:10:22.090809+0000","last_became_peered":"2024-07-17T07:10:22.090809+0000","last_unstale":"2024-07-17T07:10:22.093885+0000","last_undegraded":"2024-07-17T07:10:22.093885+0000","last_fullsized":"2024-07-17T07:10:22.093885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:25:51.470485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080184+0000","last_change":"2024-07-17T07:10:22.064491+0000","last_active":"2024-07-17T07:10:23.080184+0000","last_peered":"2024-07-17T07:10:23.080184+0000","last_clean":"2024-07-17T07:10:23.080184+0000","last_became_active":"2024-07-17T07:10:22.063760+0000","last_became_peered":"2024-07-17T07:10:22.063760+0000","last_unstale":"2024-07-17T07:10:23.080184+0000","last_undegraded":"2024-07-17T07:10:23.080184+0000","last_fullsized":"2024-07-17T07:10:23.080184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:15.744790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292949+0000","last_change":"2024-07-17T07:10:22.083829+0000","last_active":"2024-07-17T07:10:23.292949+0000","last_peered":"2024-07-17T07:10:23.292949+0000","last_clean":"2024-07-17T07:10:23.292949+0000","last_became_active":"2024-07-17T07:10:22.083486+0000","last_became_peered":"2024-07-17T07:10:22.083486+0000","last_unstale":"2024-07-17T07:10:23.292949+0000","last_undegraded":"2024-07-17T07:10:23.292949+0000","last_fullsized":"2024-07-17T07:10:23.292949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:55.839853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.096293+0000","last_change":"2024-07-17T07:10:22.092565+0000","last_active":"2024-07-17T07:10:22.096293+0000","last_peered":"2024-07-17T07:10:22.096293+0000","last_clean":"2024-07-17T07:10:22.096293+0000","last_became_active":"2024-07-17T07:10:22.092217+0000","last_became_peered":"2024-07-17T07:10:22.092217+0000","last_unstale":"2024-07-17T07:10:22.096293+0000","last_undegraded":"2024-07-17T07:10:22.096293+0000","last_fullsized":"2024-07-17T07:10:22.096293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:49.355947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083822+0000","last_change":"2024-07-17T07:10:22.063575+0000","last_active":"2024-07-17T07:10:23.083822+0000","last_peered":"2024-07-17T07:10:23.083822+0000","last_clean":"2024-07-17T07:10:23.083822+0000","last_became_active":"2024-07-17T07:10:22.063220+0000","last_became_peered":"2024-07-17T07:10:22.063220+0000","last_unstale":"2024-07-17T07:10:23.083822+0000","last_undegraded":"2024-07-17T07:10:23.083822+0000","last_fullsized":"2024-07-17T07:10:23.083822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:55.839853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080877+0000","last_change":"2024-07-17T07:10:22.069716+0000","last_active":"2024-07-17T07:10:22.080877+0000","last_peered":"2024-07-17T07:10:22.080877+0000","last_clean":"2024-07-17T07:10:22.080877+0000","last_became_active":"2024-07-17T07:10:22.069028+0000","last_became_peered":"2024-07-17T07:10:22.069028+0000","last_unstale":"2024-07-17T07:10:22.080877+0000","last_undegraded":"2024-07-17T07:10:22.080877+0000","last_fullsized":"2024-07-17T07:10:22.080877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:34:11.542173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088961+0000","last_change":"2024-07-17T07:10:22.088829+0000","last_active":"2024-07-17T07:10:23.088961+0000","last_peered":"2024-07-17T07:10:23.088961+0000","last_clean":"2024-07-17T07:10:23.088961+0000","last_became_active":"2024-07-17T07:10:22.088031+0000","last_became_peered":"2024-07-17T07:10:22.088031+0000","last_unstale":"2024-07-17T07:10:23.088961+0000","last_undegraded":"2024-07-17T07:10:23.088961+0000","last_fullsized":"2024-07-17T07:10:23.088961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:55:48.373358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080831+0000","last_change":"2024-07-17T07:10:22.072100+0000","last_active":"2024-07-17T07:10:22.080831+0000","last_peered":"2024-07-17T07:10:22.080831+0000","last_clean":"2024-07-17T07:10:22.080831+0000","last_became_active":"2024-07-17T07:10:22.071707+0000","last_became_peered":"2024-07-17T07:10:22.071707+0000","last_unstale":"2024-07-17T07:10:22.080831+0000","last_undegraded":"2024-07-17T07:10:22.080831+0000","last_fullsized":"2024-07-17T07:10:22.080831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:56.953433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.081118+0000","last_change":"2024-07-17T07:10:22.076942+0000","last_active":"2024-07-17T07:10:22.081118+0000","last_peered":"2024-07-17T07:10:22.081118+0000","last_clean":"2024-07-17T07:10:22.081118+0000","last_became_active":"2024-07-17T07:10:22.076661+0000","last_became_peered":"2024-07-17T07:10:22.076661+0000","last_unstale":"2024-07-17T07:10:22.081118+0000","last_undegraded":"2024-07-17T07:10:22.081118+0000","last_fullsized":"2024-07-17T07:10:22.081118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:22.814127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.091327+0000","last_change":"2024-07-17T07:10:22.088627+0000","last_active":"2024-07-17T07:10:22.091327+0000","last_peered":"2024-07-17T07:10:22.091327+0000","last_clean":"2024-07-17T07:10:22.091327+0000","last_became_active":"2024-07-17T07:10:22.088033+0000","last_became_peered":"2024-07-17T07:10:22.088033+0000","last_unstale":"2024-07-17T07:10:22.091327+0000","last_undegraded":"2024-07-17T07:10:22.091327+0000","last_fullsized":"2024-07-17T07:10:22.091327+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:11:37.188561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089641+0000","last_change":"2024-07-17T07:10:22.064537+0000","last_active":"2024-07-17T07:10:23.089641+0000","last_peered":"2024-07-17T07:10:23.089641+0000","last_clean":"2024-07-17T07:10:23.089641+0000","last_became_active":"2024-07-17T07:10:22.064167+0000","last_became_peered":"2024-07-17T07:10:22.064167+0000","last_unstale":"2024-07-17T07:10:23.089641+0000","last_undegraded":"2024-07-17T07:10:23.089641+0000","last_fullsized":"2024-07-17T07:10:23.089641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:09.620340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088966+0000","last_change":"2024-07-17T07:10:22.065980+0000","last_active":"2024-07-17T07:10:23.088966+0000","last_peered":"2024-07-17T07:10:23.088966+0000","last_clean":"2024-07-17T07:10:23.088966+0000","last_became_active":"2024-07-17T07:10:22.065658+0000","last_became_peered":"2024-07-17T07:10:22.065658+0000","last_unstale":"2024-07-17T07:10:23.088966+0000","last_undegraded":"2024-07-17T07:10:23.088966+0000","last_fullsized":"2024-07-17T07:10:23.088966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:37.872709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.088872+0000","last_change":"2024-07-17T07:10:22.085039+0000","last_active":"2024-07-17T07:10:22.088872+0000","last_peered":"2024-07-17T07:10:22.088872+0000","last_clean":"2024-07-17T07:10:22.088872+0000","last_became_active":"2024-07-17T07:10:22.084638+0000","last_became_peered":"2024-07-17T07:10:22.084638+0000","last_unstale":"2024-07-17T07:10:22.088872+0000","last_undegraded":"2024-07-17T07:10:22.088872+0000","last_fullsized":"2024-07-17T07:10:22.088872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:29.228050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292298+0000","last_change":"2024-07-17T07:10:22.089390+0000","last_active":"2024-07-17T07:10:23.292298+0000","last_peered":"2024-07-17T07:10:23.292298+0000","last_clean":"2024-07-17T07:10:23.292298+0000","last_became_active":"2024-07-17T07:10:22.089046+0000","last_became_peered":"2024-07-17T07:10:22.089046+0000","last_unstale":"2024-07-17T07:10:23.292298+0000","last_undegraded":"2024-07-17T07:10:23.292298+0000","last_fullsized":"2024-07-17T07:10:23.292298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:26.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.084553+0000","last_change":"2024-07-17T07:10:22.083024+0000","last_active":"2024-07-17T07:10:22.084553+0000","last_peered":"2024-07-17T07:10:22.084553+0000","last_clean":"2024-07-17T07:10:22.084553+0000","last_became_active":"2024-07-17T07:10:22.082533+0000","last_became_peered":"2024-07-17T07:10:22.082533+0000","last_unstale":"2024-07-17T07:10:22.084553+0000","last_undegraded":"2024-07-17T07:10:22.084553+0000","last_fullsized":"2024-07-17T07:10:22.084553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:31.848232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089371+0000","last_change":"2024-07-17T07:10:22.074465+0000","last_active":"2024-07-17T07:10:23.089371+0000","last_peered":"2024-07-17T07:10:23.089371+0000","last_clean":"2024-07-17T07:10:23.089371+0000","last_became_active":"2024-07-17T07:10:22.072940+0000","last_became_peered":"2024-07-17T07:10:22.072940+0000","last_unstale":"2024-07-17T07:10:23.089371+0000","last_undegraded":"2024-07-17T07:10:23.089371+0000","last_fullsized":"2024-07-17T07:10:23.089371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:42.938529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089290+0000","last_change":"2024-07-17T07:10:22.086891+0000","last_active":"2024-07-17T07:10:22.089290+0000","last_peered":"2024-07-17T07:10:22.089290+0000","last_clean":"2024-07-17T07:10:22.089290+0000","last_became_active":"2024-07-17T07:10:22.084680+0000","last_became_peered":"2024-07-17T07:10:22.084680+0000","last_unstale":"2024-07-17T07:10:22.089290+0000","last_undegraded":"2024-07-17T07:10:22.089290+0000","last_fullsized":"2024-07-17T07:10:22.089290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:38.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081588+0000","last_change":"2024-07-17T07:10:22.086956+0000","last_active":"2024-07-17T07:10:23.081588+0000","last_peered":"2024-07-17T07:10:23.081588+0000","last_clean":"2024-07-17T07:10:23.081588+0000","last_became_active":"2024-07-17T07:10:22.086678+0000","last_became_peered":"2024-07-17T07:10:22.086678+0000","last_unstale":"2024-07-17T07:10:23.081588+0000","last_undegraded":"2024-07-17T07:10:23.081588+0000","last_fullsized":"2024-07-17T07:10:23.081588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:34:11.542173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083715+0000","last_change":"2024-07-17T07:10:22.090902+0000","last_active":"2024-07-17T07:10:23.083715+0000","last_peered":"2024-07-17T07:10:23.083715+0000","last_clean":"2024-07-17T07:10:23.083715+0000","last_became_active":"2024-07-17T07:10:22.089365+0000","last_became_peered":"2024-07-17T07:10:22.089365+0000","last_unstale":"2024-07-17T07:10:23.083715+0000","last_undegraded":"2024-07-17T07:10:23.083715+0000","last_fullsized":"2024-07-17T07:10:23.083715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:12:36.081261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.293083+0000","last_change":"2024-07-17T07:10:21.052818+0000","last_active":"2024-07-17T07:10:23.293083+0000","last_peered":"2024-07-17T07:10:23.293083+0000","last_clean":"2024-07-17T07:10:23.293083+0000","last_became_active":"2024-07-17T07:10:21.051600+0000","last_became_peered":"2024-07-17T07:10:21.051600+0000","last_unstale":"2024-07-17T07:10:23.293083+0000","last_undegraded":"2024-07-17T07:10:23.293083+0000","last_fullsized":"2024-07-17T07:10:23.293083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:04.326536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.086940+0000","last_change":"2024-07-17T07:10:22.081846+0000","last_active":"2024-07-17T07:10:22.086940+0000","last_peered":"2024-07-17T07:10:22.086940+0000","last_clean":"2024-07-17T07:10:22.086940+0000","last_became_active":"2024-07-17T07:10:22.079659+0000","last_became_peered":"2024-07-17T07:10:22.079659+0000","last_unstale":"2024-07-17T07:10:22.086940+0000","last_undegraded":"2024-07-17T07:10:22.086940+0000","last_fullsized":"2024-07-17T07:10:22.086940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:53.886348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082682+0000","last_change":"2024-07-17T07:10:21.047045+0000","last_active":"2024-07-17T07:10:23.082682+0000","last_peered":"2024-07-17T07:10:23.082682+0000","last_clean":"2024-07-17T07:10:23.082682+0000","last_became_active":"2024-07-17T07:10:21.046763+0000","last_became_peered":"2024-07-17T07:10:21.046763+0000","last_unstale":"2024-07-17T07:10:23.082682+0000","last_undegraded":"2024-07-17T07:10:23.082682+0000","last_fullsized":"2024-07-17T07:10:23.082682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:02.141650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082470+0000","last_change":"2024-07-17T07:10:22.086215+0000","last_active":"2024-07-17T07:10:23.082470+0000","last_peered":"2024-07-17T07:10:23.082470+0000","last_clean":"2024-07-17T07:10:23.082470+0000","last_became_active":"2024-07-17T07:10:22.085742+0000","last_became_peered":"2024-07-17T07:10:22.085742+0000","last_unstale":"2024-07-17T07:10:23.082470+0000","last_undegraded":"2024-07-17T07:10:23.082470+0000","last_fullsized":"2024-07-17T07:10:23.082470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:30.279694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082470+0000","last_change":"2024-07-17T07:10:21.044817+0000","last_active":"2024-07-17T07:10:23.082470+0000","last_peered":"2024-07-17T07:10:23.082470+0000","last_clean":"2024-07-17T07:10:23.082470+0000","last_became_active":"2024-07-17T07:10:21.044467+0000","last_became_peered":"2024-07-17T07:10:21.044467+0000","last_unstale":"2024-07-17T07:10:23.082470+0000","last_undegraded":"2024-07-17T07:10:23.082470+0000","last_fullsized":"2024-07-17T07:10:23.082470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:26.514197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.095425+0000","last_change":"2024-07-17T07:10:22.093252+0000","last_active":"2024-07-17T07:10:22.095425+0000","last_peered":"2024-07-17T07:10:22.095425+0000","last_clean":"2024-07-17T07:10:22.095425+0000","last_became_active":"2024-07-17T07:10:22.091863+0000","last_became_peered":"2024-07-17T07:10:22.091863+0000","last_unstale":"2024-07-17T07:10:22.095425+0000","last_undegraded":"2024-07-17T07:10:22.095425+0000","last_fullsized":"2024-07-17T07:10:22.095425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:48:41.374483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082412+0000","last_change":"2024-07-17T07:10:21.038105+0000","last_active":"2024-07-17T07:10:23.082412+0000","last_peered":"2024-07-17T07:10:23.082412+0000","last_clean":"2024-07-17T07:10:23.082412+0000","last_became_active":"2024-07-17T07:10:21.037781+0000","last_became_peered":"2024-07-17T07:10:21.037781+0000","last_unstale":"2024-07-17T07:10:23.082412+0000","last_undegraded":"2024-07-17T07:10:23.082412+0000","last_fullsized":"2024-07-17T07:10:23.082412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:55:47.358861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.079843+0000","last_change":"2024-07-17T07:10:22.089262+0000","last_active":"2024-07-17T07:10:23.079843+0000","last_peered":"2024-07-17T07:10:23.079843+0000","last_clean":"2024-07-17T07:10:23.079843+0000","last_became_active":"2024-07-17T07:10:22.088968+0000","last_became_peered":"2024-07-17T07:10:22.088968+0000","last_unstale":"2024-07-17T07:10:23.079843+0000","last_undegraded":"2024-07-17T07:10:23.079843+0000","last_fullsized":"2024-07-17T07:10:23.079843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:59:54.573674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103283+0000","last_change":"2024-07-17T07:10:21.046295+0000","last_active":"2024-07-17T07:10:22.103283+0000","last_peered":"2024-07-17T07:10:22.103283+0000","last_clean":"2024-07-17T07:10:22.103283+0000","last_became_active":"2024-07-17T07:10:21.045951+0000","last_became_peered":"2024-07-17T07:10:21.045951+0000","last_unstale":"2024-07-17T07:10:22.103283+0000","last_undegraded":"2024-07-17T07:10:22.103283+0000","last_fullsized":"2024-07-17T07:10:22.103283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:14.328636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291526+0000","last_change":"2024-07-17T07:10:22.090718+0000","last_active":"2024-07-17T07:10:23.291526+0000","last_peered":"2024-07-17T07:10:23.291526+0000","last_clean":"2024-07-17T07:10:23.291526+0000","last_became_active":"2024-07-17T07:10:22.090409+0000","last_became_peered":"2024-07-17T07:10:22.090409+0000","last_unstale":"2024-07-17T07:10:23.291526+0000","last_undegraded":"2024-07-17T07:10:23.291526+0000","last_fullsized":"2024-07-17T07:10:23.291526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:14.654859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.102458+0000","last_change":"2024-07-17T07:10:21.040347+0000","last_active":"2024-07-17T07:10:22.102458+0000","last_peered":"2024-07-17T07:10:22.102458+0000","last_clean":"2024-07-17T07:10:22.102458+0000","last_became_active":"2024-07-17T07:10:21.040026+0000","last_became_peered":"2024-07-17T07:10:21.040026+0000","last_unstale":"2024-07-17T07:10:22.102458+0000","last_undegraded":"2024-07-17T07:10:22.102458+0000","last_fullsized":"2024-07-17T07:10:22.102458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:02.141650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082539+0000","last_change":"2024-07-17T07:10:22.077794+0000","last_active":"2024-07-17T07:10:23.082539+0000","last_peered":"2024-07-17T07:10:23.082539+0000","last_clean":"2024-07-17T07:10:23.082539+0000","last_became_active":"2024-07-17T07:10:22.077522+0000","last_became_peered":"2024-07-17T07:10:22.077522+0000","last_unstale":"2024-07-17T07:10:23.082539+0000","last_undegraded":"2024-07-17T07:10:23.082539+0000","last_fullsized":"2024-07-17T07:10:23.082539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:20.893102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055160+0000","last_change":"2024-07-17T07:10:21.048982+0000","last_active":"2024-07-17T07:10:22.055160+0000","last_peered":"2024-07-17T07:10:22.055160+0000","last_clean":"2024-07-17T07:10:22.055160+0000","last_became_active":"2024-07-17T07:10:21.048689+0000","last_became_peered":"2024-07-17T07:10:21.048689+0000","last_unstale":"2024-07-17T07:10:22.055160+0000","last_undegraded":"2024-07-17T07:10:22.055160+0000","last_fullsized":"2024-07-17T07:10:22.055160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:23.598160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089460+0000","last_change":"2024-07-17T07:10:22.086890+0000","last_active":"2024-07-17T07:10:22.089460+0000","last_peered":"2024-07-17T07:10:22.089460+0000","last_clean":"2024-07-17T07:10:22.089460+0000","last_became_active":"2024-07-17T07:10:22.084049+0000","last_became_peered":"2024-07-17T07:10:22.084049+0000","last_unstale":"2024-07-17T07:10:22.089460+0000","last_undegraded":"2024-07-17T07:10:22.089460+0000","last_fullsized":"2024-07-17T07:10:22.089460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:30.279694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082335+0000","last_change":"2024-07-17T07:10:21.051458+0000","last_active":"2024-07-17T07:10:23.082335+0000","last_peered":"2024-07-17T07:10:23.082335+0000","last_clean":"2024-07-17T07:10:23.082335+0000","last_became_active":"2024-07-17T07:10:21.050443+0000","last_became_peered":"2024-07-17T07:10:21.050443+0000","last_unstale":"2024-07-17T07:10:23.082335+0000","last_undegraded":"2024-07-17T07:10:23.082335+0000","last_fullsized":"2024-07-17T07:10:23.082335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:47.380726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292517+0000","last_change":"2024-07-17T07:10:22.084767+0000","last_active":"2024-07-17T07:10:23.292517+0000","last_peered":"2024-07-17T07:10:23.292517+0000","last_clean":"2024-07-17T07:10:23.292517+0000","last_became_active":"2024-07-17T07:10:22.084412+0000","last_became_peered":"2024-07-17T07:10:22.084412+0000","last_unstale":"2024-07-17T07:10:23.292517+0000","last_undegraded":"2024-07-17T07:10:23.292517+0000","last_fullsized":"2024-07-17T07:10:23.292517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:47.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.067274+0000","last_change":"2024-07-17T07:10:21.056692+0000","last_active":"2024-07-17T07:10:22.067274+0000","last_peered":"2024-07-17T07:10:22.067274+0000","last_clean":"2024-07-17T07:10:22.067274+0000","last_became_active":"2024-07-17T07:10:21.056258+0000","last_became_peered":"2024-07-17T07:10:21.056258+0000","last_unstale":"2024-07-17T07:10:22.067274+0000","last_undegraded":"2024-07-17T07:10:22.067274+0000","last_fullsized":"2024-07-17T07:10:22.067274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:33:41.081662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080462+0000","last_change":"2024-07-17T07:10:22.072045+0000","last_active":"2024-07-17T07:10:22.080462+0000","last_peered":"2024-07-17T07:10:22.080462+0000","last_clean":"2024-07-17T07:10:22.080462+0000","last_became_active":"2024-07-17T07:10:22.069225+0000","last_became_peered":"2024-07-17T07:10:22.069225+0000","last_unstale":"2024-07-17T07:10:22.080462+0000","last_undegraded":"2024-07-17T07:10:22.080462+0000","last_fullsized":"2024-07-17T07:10:22.080462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:49.193369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.080808+0000","last_change":"2024-07-17T07:10:21.049788+0000","last_active":"2024-07-17T07:10:22.080808+0000","last_peered":"2024-07-17T07:10:22.080808+0000","last_clean":"2024-07-17T07:10:22.080808+0000","last_became_active":"2024-07-17T07:10:21.049228+0000","last_became_peered":"2024-07-17T07:10:21.049228+0000","last_unstale":"2024-07-17T07:10:22.080808+0000","last_undegraded":"2024-07-17T07:10:22.080808+0000","last_fullsized":"2024-07-17T07:10:22.080808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:15:47.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084025+0000","last_change":"2024-07-17T07:10:22.074026+0000","last_active":"2024-07-17T07:10:23.084025+0000","last_peered":"2024-07-17T07:10:23.084025+0000","last_clean":"2024-07-17T07:10:23.084025+0000","last_became_active":"2024-07-17T07:10:22.070134+0000","last_became_peered":"2024-07-17T07:10:22.070134+0000","last_unstale":"2024-07-17T07:10:23.084025+0000","last_undegraded":"2024-07-17T07:10:23.084025+0000","last_fullsized":"2024-07-17T07:10:23.084025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:12.187590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103821+0000","last_change":"2024-07-17T07:10:21.057272+0000","last_active":"2024-07-17T07:10:22.103821+0000","last_peered":"2024-07-17T07:10:22.103821+0000","last_clean":"2024-07-17T07:10:22.103821+0000","last_became_active":"2024-07-17T07:10:21.056200+0000","last_became_peered":"2024-07-17T07:10:21.056200+0000","last_unstale":"2024-07-17T07:10:22.103821+0000","last_undegraded":"2024-07-17T07:10:22.103821+0000","last_fullsized":"2024-07-17T07:10:22.103821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:45:51.092604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089216+0000","last_change":"2024-07-17T07:10:22.086812+0000","last_active":"2024-07-17T07:10:22.089216+0000","last_peered":"2024-07-17T07:10:22.089216+0000","last_clean":"2024-07-17T07:10:22.089216+0000","last_became_active":"2024-07-17T07:10:22.084782+0000","last_became_peered":"2024-07-17T07:10:22.084782+0000","last_unstale":"2024-07-17T07:10:22.089216+0000","last_undegraded":"2024-07-17T07:10:22.089216+0000","last_fullsized":"2024-07-17T07:10:22.089216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:57.103510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292702+0000","last_change":"2024-07-17T07:10:21.046340+0000","last_active":"2024-07-17T07:10:23.292702+0000","last_peered":"2024-07-17T07:10:23.292702+0000","last_clean":"2024-07-17T07:10:23.292702+0000","last_became_active":"2024-07-17T07:10:21.046007+0000","last_became_peered":"2024-07-17T07:10:21.046007+0000","last_unstale":"2024-07-17T07:10:23.292702+0000","last_undegraded":"2024-07-17T07:10:23.292702+0000","last_fullsized":"2024-07-17T07:10:23.292702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:28:09.967835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.091740+0000","last_change":"2024-07-17T07:10:22.088895+0000","last_active":"2024-07-17T07:10:22.091740+0000","last_peered":"2024-07-17T07:10:22.091740+0000","last_clean":"2024-07-17T07:10:22.091740+0000","last_became_active":"2024-07-17T07:10:22.087194+0000","last_became_peered":"2024-07-17T07:10:22.087194+0000","last_unstale":"2024-07-17T07:10:22.091740+0000","last_undegraded":"2024-07-17T07:10:22.091740+0000","last_fullsized":"2024-07-17T07:10:22.091740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:11.140319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.098504+0000","last_change":"2024-07-17T07:10:21.057461+0000","last_active":"2024-07-17T07:10:22.098504+0000","last_peered":"2024-07-17T07:10:22.098504+0000","last_clean":"2024-07-17T07:10:22.098504+0000","last_became_active":"2024-07-17T07:10:21.056630+0000","last_became_peered":"2024-07-17T07:10:21.056630+0000","last_unstale":"2024-07-17T07:10:22.098504+0000","last_undegraded":"2024-07-17T07:10:22.098504+0000","last_fullsized":"2024-07-17T07:10:22.098504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:40.370255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088605+0000","last_change":"2024-07-17T07:10:22.079352+0000","last_active":"2024-07-17T07:10:23.088605+0000","last_peered":"2024-07-17T07:10:23.088605+0000","last_clean":"2024-07-17T07:10:23.088605+0000","last_became_active":"2024-07-17T07:10:22.078209+0000","last_became_peered":"2024-07-17T07:10:22.078209+0000","last_unstale":"2024-07-17T07:10:23.088605+0000","last_undegraded":"2024-07-17T07:10:23.088605+0000","last_fullsized":"2024-07-17T07:10:23.088605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:17.152776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088514+0000","last_change":"2024-07-17T07:10:21.052329+0000","last_active":"2024-07-17T07:10:23.088514+0000","last_peered":"2024-07-17T07:10:23.088514+0000","last_clean":"2024-07-17T07:10:23.088514+0000","last_became_active":"2024-07-17T07:10:21.052038+0000","last_became_peered":"2024-07-17T07:10:21.052038+0000","last_unstale":"2024-07-17T07:10:23.088514+0000","last_undegraded":"2024-07-17T07:10:23.088514+0000","last_fullsized":"2024-07-17T07:10:23.088514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:11.822024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083137+0000","last_change":"2024-07-17T07:10:22.064026+0000","last_active":"2024-07-17T07:10:23.083137+0000","last_peered":"2024-07-17T07:10:23.083137+0000","last_clean":"2024-07-17T07:10:23.083137+0000","last_became_active":"2024-07-17T07:10:22.063482+0000","last_became_peered":"2024-07-17T07:10:22.063482+0000","last_unstale":"2024-07-17T07:10:23.083137+0000","last_undegraded":"2024-07-17T07:10:23.083137+0000","last_fullsized":"2024-07-17T07:10:23.083137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:26.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.069498+0000","last_change":"2024-07-17T07:10:21.054502+0000","last_active":"2024-07-17T07:10:22.069498+0000","last_peered":"2024-07-17T07:10:22.069498+0000","last_clean":"2024-07-17T07:10:22.069498+0000","last_became_active":"2024-07-17T07:10:21.054163+0000","last_became_peered":"2024-07-17T07:10:21.054163+0000","last_unstale":"2024-07-17T07:10:22.069498+0000","last_undegraded":"2024-07-17T07:10:22.069498+0000","last_fullsized":"2024-07-17T07:10:22.069498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:08.019938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292058+0000","last_change":"2024-07-17T07:10:22.087884+0000","last_active":"2024-07-17T07:10:23.292058+0000","last_peered":"2024-07-17T07:10:23.292058+0000","last_clean":"2024-07-17T07:10:23.292058+0000","last_became_active":"2024-07-17T07:10:22.083772+0000","last_became_peered":"2024-07-17T07:10:22.083772+0000","last_unstale":"2024-07-17T07:10:23.292058+0000","last_undegraded":"2024-07-17T07:10:23.292058+0000","last_fullsized":"2024-07-17T07:10:23.292058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:20.893102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.104951+0000","last_change":"2024-07-17T07:10:21.052821+0000","last_active":"2024-07-17T07:10:22.104951+0000","last_peered":"2024-07-17T07:10:22.104951+0000","last_clean":"2024-07-17T07:10:22.104951+0000","last_became_active":"2024-07-17T07:10:21.052494+0000","last_became_peered":"2024-07-17T07:10:21.052494+0000","last_unstale":"2024-07-17T07:10:22.104951+0000","last_undegraded":"2024-07-17T07:10:22.104951+0000","last_fullsized":"2024-07-17T07:10:22.104951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:34.947536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.083684+0000","last_change":"2024-07-17T07:10:22.081150+0000","last_active":"2024-07-17T07:10:22.083684+0000","last_peered":"2024-07-17T07:10:22.083684+0000","last_clean":"2024-07-17T07:10:22.083684+0000","last_became_active":"2024-07-17T07:10:22.080891+0000","last_became_peered":"2024-07-17T07:10:22.080891+0000","last_unstale":"2024-07-17T07:10:22.083684+0000","last_undegraded":"2024-07-17T07:10:22.083684+0000","last_fullsized":"2024-07-17T07:10:22.083684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:30.821998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089560+0000","last_change":"2024-07-17T07:10:21.053977+0000","last_active":"2024-07-17T07:10:23.089560+0000","last_peered":"2024-07-17T07:10:23.089560+0000","last_clean":"2024-07-17T07:10:23.089560+0000","last_became_active":"2024-07-17T07:10:21.053336+0000","last_became_peered":"2024-07-17T07:10:21.053336+0000","last_unstale":"2024-07-17T07:10:23.089560+0000","last_undegraded":"2024-07-17T07:10:23.089560+0000","last_fullsized":"2024-07-17T07:10:23.089560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:39.050407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083913+0000","last_change":"2024-07-17T07:10:22.088610+0000","last_active":"2024-07-17T07:10:23.083913+0000","last_peered":"2024-07-17T07:10:23.083913+0000","last_clean":"2024-07-17T07:10:23.083913+0000","last_became_active":"2024-07-17T07:10:22.088314+0000","last_became_peered":"2024-07-17T07:10:22.088314+0000","last_unstale":"2024-07-17T07:10:23.083913+0000","last_undegraded":"2024-07-17T07:10:23.083913+0000","last_fullsized":"2024-07-17T07:10:23.083913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:34:44.274509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.097537+0000","last_change":"2024-07-17T07:10:21.059797+0000","last_active":"2024-07-17T07:10:22.097537+0000","last_peered":"2024-07-17T07:10:22.097537+0000","last_clean":"2024-07-17T07:10:22.097537+0000","last_became_active":"2024-07-17T07:10:21.059483+0000","last_became_peered":"2024-07-17T07:10:21.059483+0000","last_unstale":"2024-07-17T07:10:22.097537+0000","last_undegraded":"2024-07-17T07:10:22.097537+0000","last_fullsized":"2024-07-17T07:10:22.097537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:39.370581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.089591+0000","last_change":"2024-07-17T07:10:22.088063+0000","last_active":"2024-07-17T07:10:22.089591+0000","last_peered":"2024-07-17T07:10:22.089591+0000","last_clean":"2024-07-17T07:10:22.089591+0000","last_became_active":"2024-07-17T07:10:22.087748+0000","last_became_peered":"2024-07-17T07:10:22.087748+0000","last_unstale":"2024-07-17T07:10:22.089591+0000","last_undegraded":"2024-07-17T07:10:22.089591+0000","last_fullsized":"2024-07-17T07:10:22.089591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:25.639980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.103682+0000","last_change":"2024-07-17T07:10:21.057507+0000","last_active":"2024-07-17T07:10:22.103682+0000","last_peered":"2024-07-17T07:10:22.103682+0000","last_clean":"2024-07-17T07:10:22.103682+0000","last_became_active":"2024-07-17T07:10:21.056767+0000","last_became_peered":"2024-07-17T07:10:21.056767+0000","last_unstale":"2024-07-17T07:10:22.103682+0000","last_undegraded":"2024-07-17T07:10:22.103682+0000","last_fullsized":"2024-07-17T07:10:22.103682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:29.213844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084016+0000","last_change":"2024-07-17T07:10:22.076189+0000","last_active":"2024-07-17T07:10:23.084016+0000","last_peered":"2024-07-17T07:10:23.084016+0000","last_clean":"2024-07-17T07:10:23.084016+0000","last_became_active":"2024-07-17T07:10:22.075901+0000","last_became_peered":"2024-07-17T07:10:22.075901+0000","last_unstale":"2024-07-17T07:10:23.084016+0000","last_undegraded":"2024-07-17T07:10:23.084016+0000","last_fullsized":"2024-07-17T07:10:23.084016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:52.205565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.056051+0000","last_change":"2024-07-17T07:10:21.052952+0000","last_active":"2024-07-17T07:10:22.056051+0000","last_peered":"2024-07-17T07:10:22.056051+0000","last_clean":"2024-07-17T07:10:22.056051+0000","last_became_active":"2024-07-17T07:10:21.052672+0000","last_became_peered":"2024-07-17T07:10:21.052672+0000","last_unstale":"2024-07-17T07:10:22.056051+0000","last_undegraded":"2024-07-17T07:10:22.056051+0000","last_fullsized":"2024-07-17T07:10:22.056051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:41.924033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081697+0000","last_change":"2024-07-17T07:10:22.086145+0000","last_active":"2024-07-17T07:10:23.081697+0000","last_peered":"2024-07-17T07:10:23.081697+0000","last_clean":"2024-07-17T07:10:23.081697+0000","last_became_active":"2024-07-17T07:10:22.085844+0000","last_became_peered":"2024-07-17T07:10:22.085844+0000","last_unstale":"2024-07-17T07:10:23.081697+0000","last_undegraded":"2024-07-17T07:10:23.081697+0000","last_fullsized":"2024-07-17T07:10:23.081697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:49.193369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082108+0000","last_change":"2024-07-17T07:10:21.049022+0000","last_active":"2024-07-17T07:10:23.082108+0000","last_peered":"2024-07-17T07:10:23.082108+0000","last_clean":"2024-07-17T07:10:23.082108+0000","last_became_active":"2024-07-17T07:10:21.048631+0000","last_became_peered":"2024-07-17T07:10:21.048631+0000","last_unstale":"2024-07-17T07:10:23.082108+0000","last_undegraded":"2024-07-17T07:10:23.082108+0000","last_fullsized":"2024-07-17T07:10:23.082108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:36.858213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088751+0000","last_change":"2024-07-17T07:10:22.060115+0000","last_active":"2024-07-17T07:10:23.088751+0000","last_peered":"2024-07-17T07:10:23.088751+0000","last_clean":"2024-07-17T07:10:23.088751+0000","last_became_active":"2024-07-17T07:10:22.059556+0000","last_became_peered":"2024-07-17T07:10:22.059556+0000","last_unstale":"2024-07-17T07:10:23.088751+0000","last_undegraded":"2024-07-17T07:10:23.088751+0000","last_fullsized":"2024-07-17T07:10:23.088751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:27.528693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.088903+0000","last_change":"2024-07-17T07:10:21.050445+0000","last_active":"2024-07-17T07:10:23.088903+0000","last_peered":"2024-07-17T07:10:23.088903+0000","last_clean":"2024-07-17T07:10:23.088903+0000","last_became_active":"2024-07-17T07:10:21.050162+0000","last_became_peered":"2024-07-17T07:10:21.050162+0000","last_unstale":"2024-07-17T07:10:23.088903+0000","last_undegraded":"2024-07-17T07:10:23.088903+0000","last_fullsized":"2024-07-17T07:10:23.088903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:52.259420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083241+0000","last_change":"2024-07-17T07:10:22.077771+0000","last_active":"2024-07-17T07:10:23.083241+0000","last_peered":"2024-07-17T07:10:23.083241+0000","last_clean":"2024-07-17T07:10:23.083241+0000","last_became_active":"2024-07-17T07:10:22.077467+0000","last_became_peered":"2024-07-17T07:10:22.077467+0000","last_unstale":"2024-07-17T07:10:23.083241+0000","last_undegraded":"2024-07-17T07:10:23.083241+0000","last_fullsized":"2024-07-17T07:10:23.083241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:47.480804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.081734+0000","last_change":"2024-07-17T07:10:21.048119+0000","last_active":"2024-07-17T07:10:23.081734+0000","last_peered":"2024-07-17T07:10:23.081734+0000","last_clean":"2024-07-17T07:10:23.081734+0000","last_became_active":"2024-07-17T07:10:21.046894+0000","last_became_peered":"2024-07-17T07:10:21.046894+0000","last_unstale":"2024-07-17T07:10:23.081734+0000","last_undegraded":"2024-07-17T07:10:23.081734+0000","last_fullsized":"2024-07-17T07:10:23.081734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:50:40.506854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082250+0000","last_change":"2024-07-17T07:10:22.089208+0000","last_active":"2024-07-17T07:10:23.082250+0000","last_peered":"2024-07-17T07:10:23.082250+0000","last_clean":"2024-07-17T07:10:23.082250+0000","last_became_active":"2024-07-17T07:10:22.088948+0000","last_became_peered":"2024-07-17T07:10:22.088948+0000","last_unstale":"2024-07-17T07:10:23.082250+0000","last_undegraded":"2024-07-17T07:10:23.082250+0000","last_fullsized":"2024-07-17T07:10:23.082250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:22.814127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089968+0000","last_change":"2024-07-17T07:10:21.054727+0000","last_active":"2024-07-17T07:10:23.089968+0000","last_peered":"2024-07-17T07:10:23.089968+0000","last_clean":"2024-07-17T07:10:23.089968+0000","last_became_active":"2024-07-17T07:10:21.053692+0000","last_became_peered":"2024-07-17T07:10:21.053692+0000","last_unstale":"2024-07-17T07:10:23.089968+0000","last_undegraded":"2024-07-17T07:10:23.089968+0000","last_fullsized":"2024-07-17T07:10:23.089968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:01.790345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080616+0000","last_change":"2024-07-17T07:10:22.093054+0000","last_active":"2024-07-17T07:10:23.080616+0000","last_peered":"2024-07-17T07:10:23.080616+0000","last_clean":"2024-07-17T07:10:23.080616+0000","last_became_active":"2024-07-17T07:10:22.092768+0000","last_became_peered":"2024-07-17T07:10:22.092768+0000","last_unstale":"2024-07-17T07:10:23.080616+0000","last_undegraded":"2024-07-17T07:10:23.080616+0000","last_fullsized":"2024-07-17T07:10:23.080616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:01.377047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.091633+0000","last_change":"2024-07-17T07:10:21.047992+0000","last_active":"2024-07-17T07:10:23.091633+0000","last_peered":"2024-07-17T07:10:23.091633+0000","last_clean":"2024-07-17T07:10:23.091633+0000","last_became_active":"2024-07-17T07:10:21.047697+0000","last_became_peered":"2024-07-17T07:10:21.047697+0000","last_unstale":"2024-07-17T07:10:23.091633+0000","last_undegraded":"2024-07-17T07:10:23.091633+0000","last_fullsized":"2024-07-17T07:10:23.091633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:23.419050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.089326+0000","last_change":"2024-07-17T07:10:22.078141+0000","last_active":"2024-07-17T07:10:23.089326+0000","last_peered":"2024-07-17T07:10:23.089326+0000","last_clean":"2024-07-17T07:10:23.089326+0000","last_became_active":"2024-07-17T07:10:22.077782+0000","last_became_peered":"2024-07-17T07:10:22.077782+0000","last_unstale":"2024-07-17T07:10:23.089326+0000","last_undegraded":"2024-07-17T07:10:23.089326+0000","last_fullsized":"2024-07-17T07:10:23.089326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:53.804147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082457+0000","last_change":"2024-07-17T07:10:21.044077+0000","last_active":"2024-07-17T07:10:23.082457+0000","last_peered":"2024-07-17T07:10:23.082457+0000","last_clean":"2024-07-17T07:10:23.082457+0000","last_became_active":"2024-07-17T07:10:21.043774+0000","last_became_peered":"2024-07-17T07:10:21.043774+0000","last_unstale":"2024-07-17T07:10:23.082457+0000","last_undegraded":"2024-07-17T07:10:23.082457+0000","last_fullsized":"2024-07-17T07:10:23.082457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:23.598160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.084223+0000","last_change":"2024-07-17T07:10:22.074734+0000","last_active":"2024-07-17T07:10:23.084223+0000","last_peered":"2024-07-17T07:10:23.084223+0000","last_clean":"2024-07-17T07:10:23.084223+0000","last_became_active":"2024-07-17T07:10:22.074330+0000","last_became_peered":"2024-07-17T07:10:22.074330+0000","last_unstale":"2024-07-17T07:10:23.084223+0000","last_undegraded":"2024-07-17T07:10:23.084223+0000","last_fullsized":"2024-07-17T07:10:23.084223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:32.605393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.292805+0000","last_change":"2024-07-17T07:10:21.045945+0000","last_active":"2024-07-17T07:10:23.292805+0000","last_peered":"2024-07-17T07:10:23.292805+0000","last_clean":"2024-07-17T07:10:23.292805+0000","last_became_active":"2024-07-17T07:10:21.045641+0000","last_became_peered":"2024-07-17T07:10:21.045641+0000","last_unstale":"2024-07-17T07:10:23.292805+0000","last_undegraded":"2024-07-17T07:10:23.292805+0000","last_fullsized":"2024-07-17T07:10:23.292805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:47.380726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.082410+0000","last_change":"2024-07-17T07:10:22.085585+0000","last_active":"2024-07-17T07:10:23.082410+0000","last_peered":"2024-07-17T07:10:23.082410+0000","last_clean":"2024-07-17T07:10:23.082410+0000","last_became_active":"2024-07-17T07:10:22.084470+0000","last_became_peered":"2024-07-17T07:10:22.084470+0000","last_unstale":"2024-07-17T07:10:23.082410+0000","last_undegraded":"2024-07-17T07:10:23.082410+0000","last_fullsized":"2024-07-17T07:10:23.082410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:46.007961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.067211+0000","last_change":"2024-07-17T07:10:21.055213+0000","last_active":"2024-07-17T07:10:22.067211+0000","last_peered":"2024-07-17T07:10:22.067211+0000","last_clean":"2024-07-17T07:10:22.067211+0000","last_became_active":"2024-07-17T07:10:21.054927+0000","last_became_peered":"2024-07-17T07:10:21.054927+0000","last_unstale":"2024-07-17T07:10:22.067211+0000","last_undegraded":"2024-07-17T07:10:22.067211+0000","last_fullsized":"2024-07-17T07:10:22.067211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:00.485092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.087050+0000","last_change":"2024-07-17T07:10:22.080907+0000","last_active":"2024-07-17T07:10:22.087050+0000","last_peered":"2024-07-17T07:10:22.087050+0000","last_clean":"2024-07-17T07:10:22.087050+0000","last_became_active":"2024-07-17T07:10:22.080601+0000","last_became_peered":"2024-07-17T07:10:22.080601+0000","last_unstale":"2024-07-17T07:10:22.087050+0000","last_undegraded":"2024-07-17T07:10:22.087050+0000","last_fullsized":"2024-07-17T07:10:22.087050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:07.606628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.056129+0000","last_change":"2024-07-17T07:10:21.035972+0000","last_active":"2024-07-17T07:10:22.056129+0000","last_peered":"2024-07-17T07:10:22.056129+0000","last_clean":"2024-07-17T07:10:22.056129+0000","last_became_active":"2024-07-17T07:10:21.035653+0000","last_became_peered":"2024-07-17T07:10:21.035653+0000","last_unstale":"2024-07-17T07:10:22.056129+0000","last_undegraded":"2024-07-17T07:10:22.056129+0000","last_fullsized":"2024-07-17T07:10:22.056129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:08.605844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291643+0000","last_change":"2024-07-17T07:10:22.091068+0000","last_active":"2024-07-17T07:10:23.291643+0000","last_peered":"2024-07-17T07:10:23.291643+0000","last_clean":"2024-07-17T07:10:23.291643+0000","last_became_active":"2024-07-17T07:10:22.085844+0000","last_became_peered":"2024-07-17T07:10:22.085844+0000","last_unstale":"2024-07-17T07:10:23.291643+0000","last_undegraded":"2024-07-17T07:10:23.291643+0000","last_fullsized":"2024-07-17T07:10:23.291643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:46.007961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055653+0000","last_change":"2024-07-17T07:10:21.048954+0000","last_active":"2024-07-17T07:10:22.055653+0000","last_peered":"2024-07-17T07:10:22.055653+0000","last_clean":"2024-07-17T07:10:22.055653+0000","last_became_active":"2024-07-17T07:10:21.048658+0000","last_became_peered":"2024-07-17T07:10:21.048658+0000","last_unstale":"2024-07-17T07:10:22.055653+0000","last_undegraded":"2024-07-17T07:10:22.055653+0000","last_fullsized":"2024-07-17T07:10:22.055653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:52.789651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.080093+0000","last_change":"2024-07-17T07:10:22.072465+0000","last_active":"2024-07-17T07:10:23.080093+0000","last_peered":"2024-07-17T07:10:23.080093+0000","last_clean":"2024-07-17T07:10:23.080093+0000","last_became_active":"2024-07-17T07:10:22.071969+0000","last_became_peered":"2024-07-17T07:10:22.071969+0000","last_unstale":"2024-07-17T07:10:23.080093+0000","last_undegraded":"2024-07-17T07:10:23.080093+0000","last_fullsized":"2024-07-17T07:10:23.080093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:33.219013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.291331+0000","last_change":"2024-07-17T07:10:21.043741+0000","last_active":"2024-07-17T07:10:23.291331+0000","last_peered":"2024-07-17T07:10:23.291331+0000","last_clean":"2024-07-17T07:10:23.291331+0000","last_became_active":"2024-07-17T07:10:21.043445+0000","last_became_peered":"2024-07-17T07:10:21.043445+0000","last_unstale":"2024-07-17T07:10:23.291331+0000","last_undegraded":"2024-07-17T07:10:23.291331+0000","last_fullsized":"2024-07-17T07:10:23.291331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:14.898452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-07-17T07:10:23.083956+0000","last_change":"2024-07-17T07:10:22.086624+0000","last_active":"2024-07-17T07:10:23.083956+0000","last_peered":"2024-07-17T07:10:23.083956+0000","last_clean":"2024-07-17T07:10:23.083956+0000","last_became_active":"2024-07-17T07:10:22.086161+0000","last_became_peered":"2024-07-17T07:10:22.086161+0000","last_unstale":"2024-07-17T07:10:23.083956+0000","last_undegraded":"2024-07-17T07:10:23.083956+0000","last_fullsized":"2024-07-17T07:10:23.083956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:21.008347+0000","last_clean_scrub_stamp":"2024-07-17T07:10:21.008347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:01.564545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.055085+0000","last_change":"2024-07-17T07:10:21.043557+0000","last_active":"2024-07-17T07:10:22.055085+0000","last_peered":"2024-07-17T07:10:22.055085+0000","last_clean":"2024-07-17T07:10:22.055085+0000","last_became_active":"2024-07-17T07:10:21.043237+0000","last_became_peered":"2024-07-17T07:10:21.043237+0000","last_unstale":"2024-07-17T07:10:22.055085+0000","last_undegraded":"2024-07-17T07:10:22.055085+0000","last_fullsized":"2024-07-17T07:10:22.055085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:10:19.993851+0000","last_clean_scrub_stamp":"2024-07-17T07:10:19.993851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:01:36.858213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'100","reported_seq":329,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-07-17T07:10:22.063831+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:10:22.063831+0000","last_peered":"2024-07-17T07:10:22.063831+0000","last_clean":"2024-07-17T07:10:22.063831+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:10:22.063831+0000","last_undegraded":"2024-07-17T07:10:22.063831+0000","last_fullsized":"2024-07-17T07:10:22.063831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":759,"num_write":149,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542333,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59108,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312732,"statfs":{"total":96636764160,"available":96576237568,"internally_reserved":0,"allocated":1597440,"data_stored":1096155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509628,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":1003520,"data_stored":512257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50208,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321632,"statfs":{"total":96636764160,"available":96585351168,"internally_reserved":0,"allocated":1003520,"data_stored":512245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":506623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":987136,"data_stored":505011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542332,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58528,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313312,"statfs":{"total":96636764160,"available":96576831488,"internally_reserved":0,"allocated":999424,"data_stored":512223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509629,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":514599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509628,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1593344,"data_stored":1102591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:10:27.062 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:10:27.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.324+0000 7f48f0c00640 1 -- 172.21.15.44:0/1103783928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 msgr2=0x7f48ec0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.324+0000 7f48f0c00640 1 --2- 172.21.15.44:0/1103783928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec0ccf30 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f48ec073410 tx=0x7f48e402aaa0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 -- 172.21.15.44:0/1103783928 shutdown_connections 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 --2- 172.21.15.44:0/1103783928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec0ccf30 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 --2- 172.21.15.44:0/1103783928 >> v2:172.21.15.175:3300/0 conn(0x7f48ec15b720 0x7f48ec0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 -- 172.21.15.44:0/1103783928 >> 172.21.15.44:0/1103783928 conn(0x7f48ec09aa30 msgr2=0x7f48ec0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 -- 172.21.15.44:0/1103783928 shutdown_connections 2024-07-17T07:10:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.325+0000 7f48f0c00640 1 -- 172.21.15.44:0/1103783928 wait complete. 2024-07-17T07:10:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.326+0000 7f48f0c00640 1 Processor -- start 2024-07-17T07:10:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.327+0000 7f48f0c00640 1 -- start start 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 0x7f48ec14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 0x7f48ec1e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cf2e0 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cd770 con 0x7f48ec15b720 2024-07-17T07:10:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48f0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0cf460 con 0x7f48ec15b340 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37968/0 (socket says 172.21.15.44:37968) 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 -- 172.21.15.44:0/512133653 learned_addr learned my addr 172.21.15.44:0/512133653 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48ea800640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 0x7f48ec1e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9400640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 0x7f48ec14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 -- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 msgr2=0x7f48ec14d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 0x7f48ec14d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 -- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 msgr2=0x7f48ec1e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 0x7f48ec1e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9e00640 1 -- 172.21.15.44:0/512133653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec06d050 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48ea800640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 0x7f48ec1e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:27.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.328+0000 7f48e9400640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 0x7f48ec14d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:27.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48e9e00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c440 tx=0x7f48dc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:27.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc017070 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48dc0035f0 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec14e230 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc003a40 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.329+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1f4bd0 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.330+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0cf950 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.332+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f48dc01a070 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.333+0000 7f48e2000640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 0x7f48cc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.333+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f48dc097b20 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.333+0000 7f48e9400640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 0x7f48cc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.334+0000 7f48e9400640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 0x7f48cc079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48ec151e10 tx=0x7f48d8005c90 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:27.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.337+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48dc060c60 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.621+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f48ec06d220 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.626+0000 7f48e2000640 1 -- 172.21.15.44:0/512133653 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7f48dc0010c0 con 0x7f48ec0ae8d0 2024-07-17T07:10:27.625 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:10:27.626 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:10:27.626 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:10:27.626 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:10:27.626 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:10:27.626 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:10:27.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 msgr2=0x7f48cc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 0x7f48cc079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48ec151e10 tx=0x7f48d8005c90 comp rx=0 tx=0).stop 2024-07-17T07:10:27.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 msgr2=0x7f48ec1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c440 tx=0x7f48dc00c900 comp rx=0 tx=0).stop 2024-07-17T07:10:27.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 shutdown_connections 2024-07-17T07:10:27.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f48cc076bb0 0x7f48cc079070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48ec15b720 0x7f48ec1e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48ec15b340 0x7f48ec14d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 --2- 172.21.15.44:0/512133653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48ec0ae8d0 0x7f48ec1f8350 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.629+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 >> 172.21.15.44:0/512133653 conn(0x7f48ec09aa30 msgr2=0x7f48ec155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:27.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.630+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 shutdown_connections 2024-07-17T07:10:27.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.630+0000 7f48f0c00640 1 -- 172.21.15.44:0/512133653 wait complete. 2024-07-17T07:10:27.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.851+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1819361927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd980ac930 msgr2=0x7fbd980b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.851+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1819361927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd980ac930 0x7fbd980b25d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbd90005fd0 tx=0x7fbd9002aaa0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1819361927 shutdown_connections 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1819361927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd980ac930 0x7fbd980b25d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1819361927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 0x7fbd9815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1819361927 >> 172.21.15.44:0/1819361927 conn(0x7fbd980b0fb0 msgr2=0x7fbd980b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1819361927 shutdown_connections 2024-07-17T07:10:27.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.852+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1819361927 wait complete. 2024-07-17T07:10:27.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.853+0000 7fbd9dc00640 1 Processor -- start 2024-07-17T07:10:27.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.853+0000 7fbd9dc00640 1 -- start start 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd980ac930 0x7fbd98152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 0x7fbd98148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9806f010 con 0x7fbd9806ea50 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9806ee90 con 0x7fbd980ac930 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.854+0000 7fbd9dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd980b2b10 con 0x7fbd980ae810 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37988/0 (socket says 172.21.15.44:37988) 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 -- 172.21.15.44:0/1564555615 learned_addr learned my addr 172.21.15.44:0/1564555615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd97400640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 0x7fbd98148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd8fe00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd980ac930 0x7fbd98152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 -- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 msgr2=0x7fbd98148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 0x7fbd98148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 -- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd980ac930 msgr2=0x7fbd98152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd980ac930 0x7fbd98152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.855+0000 7fbd96a00640 1 -- 172.21.15.44:0/1564555615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd98073080 con 0x7fbd9806ea50 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd96a00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b410 tx=0x7fbd8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88015020 con 0x7fbd9806ea50 2024-07-17T07:10:27.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd88011940 con 0x7fbd9806ea50 2024-07-17T07:10:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011d90 con 0x7fbd9806ea50 2024-07-17T07:10:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981f7b30 con 0x7fbd9806ea50 2024-07-17T07:10:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.856+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981f80f0 con 0x7fbd9806ea50 2024-07-17T07:10:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.857+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd980b0c20 con 0x7fbd9806ea50 2024-07-17T07:10:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.858+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fbd88018020 con 0x7fbd9806ea50 2024-07-17T07:10:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.859+0000 7fbd8ea00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 0x7fbd74079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.860+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fbd880986c0 con 0x7fbd9806ea50 2024-07-17T07:10:27.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.861+0000 7fbd8fe00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 0x7fbd74079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:27.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.862+0000 7fbd8fe00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 0x7fbd74079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd980ce900 tx=0x7fbd80009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:27.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:27.864+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbd8801f020 con 0x7fbd9806ea50 2024-07-17T07:10:28.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.116+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd980af9f0 con 0x7fbd9806ea50 2024-07-17T07:10:28.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.117+0000 7fbd8ea00640 1 -- 172.21.15.44:0/1564555615 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7fbd88061760 con 0x7fbd9806ea50 2024-07-17T07:10:28.117 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:10:28.117 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:10:28.118 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:10:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 msgr2=0x7fbd74079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 0x7fbd74079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd980ce900 tx=0x7fbd80009040 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 msgr2=0x7fbd98072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b410 tx=0x7fbd8800b8d0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 shutdown_connections 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd980ae810 0x7fbd98148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fbd74076bb0 0x7fbd74079070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd980ac930 0x7fbd98152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 --2- 172.21.15.44:0/1564555615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd9806ea50 0x7fbd98072b40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.120+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 >> 172.21.15.44:0/1564555615 conn(0x7fbd980b0fb0 msgr2=0x7fbd980ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.121+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 shutdown_connections 2024-07-17T07:10:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.121+0000 7fbd9dc00640 1 -- 172.21.15.44:0/1564555615 wait complete. 2024-07-17T07:10:28.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:10:21.508926+0000', 'modified': '2024-07-17T07:10:24.577528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6261': {'gid': 6261, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2638137978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 2638137978}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2638137978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6261, 'qdb_cluster': [6261]} 2024-07-17T07:10:28.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:10:28.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.767+0000 7f489f200640 1 -- 172.21.15.44:0/304260656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a00ce690 msgr2=0x7f48a015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.767+0000 7f489f200640 1 --2- 172.21.15.44:0/304260656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a00ce690 0x7f48a015d510 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4898005fd0 tx=0x7f489802aa10 comp rx=0 tx=0).stop 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.768+0000 7f489f200640 1 -- 172.21.15.44:0/304260656 shutdown_connections 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.768+0000 7f489f200640 1 --2- 172.21.15.44:0/304260656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a00ce690 0x7f48a015d510 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.768+0000 7f489f200640 1 -- 172.21.15.44:0/304260656 >> 172.21.15.44:0/304260656 conn(0x7f48a00b4ff0 msgr2=0x7f48a00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.768+0000 7f489f200640 1 -- 172.21.15.44:0/304260656 shutdown_connections 2024-07-17T07:10:28.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.768+0000 7f489f200640 1 -- 172.21.15.44:0/304260656 wait complete. 2024-07-17T07:10:28.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.769+0000 7f489f200640 1 Processor -- start 2024-07-17T07:10:28.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.770+0000 7f489f200640 1 -- start start 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a00ce690 0x7f48a015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 0x7f48a0153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0168860 con 0x7f48a015d130 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01686e0 con 0x7f48a0161be0 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48a01689e0 con 0x7f48a00ce690 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38000/0 (socket says 172.21.15.44:38000) 2024-07-17T07:10:28.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489d400640 1 -- 172.21.15.44:0/970875099 learned_addr learned my addr 172.21.15.44:0/970875099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.771+0000 7f489de00640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a00ce690 0x7f48a015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489e800640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 0x7f48a0153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 -- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a00ce690 msgr2=0x7f48a015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a00ce690 0x7f48a015c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 -- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 msgr2=0x7f48a0153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 0x7f48a0153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 -- 172.21.15.44:0/970875099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a015cf20 con 0x7f48a015d130 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489e800640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 0x7f48a0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489d400640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c480 tx=0x7f488c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f4896000640 1 -- 172.21.15.44:0/970875099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0052d0 con 0x7f48a015d130 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f4896000640 1 -- 172.21.15.44:0/970875099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488c005470 con 0x7f48a015d130 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f4896000640 1 -- 172.21.15.44:0/970875099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c012870 con 0x7f48a015d130 2024-07-17T07:10:28.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.772+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a01540c0 con 0x7f48a015d130 2024-07-17T07:10:28.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.773+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a0154680 con 0x7f48a015d130 2024-07-17T07:10:28.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.775+0000 7f4896000640 1 -- 172.21.15.44:0/970875099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f488c0035f0 con 0x7f48a015d130 2024-07-17T07:10:28.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.776+0000 7f4896000640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 0x7f4880079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.776+0000 7f4896000640 1 -- 172.21.15.44:0/970875099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f488c0982e0 con 0x7f48a015d130 2024-07-17T07:10:28.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.776+0000 7f48a781e740 1 Processor -- start 2024-07-17T07:10:28.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.777+0000 7f48a781e740 1 -- start start 2024-07-17T07:10:28.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.777+0000 7f489de00640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 0x7f4880079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.777+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 0x561d35f6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.777+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f42280 0x561d35f67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f48a781e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f48a781e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561d35eb2d60 con 0x561d35f67790 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561d35ea2260 con 0x561d35f42280 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561d35ea62a0 con 0x561d35f6afc0 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38010/0 (socket says 172.21.15.44:38010) 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489e800640 1 -- 172.21.15.44:0/3798389868 learned_addr learned my addr 172.21.15.44:0/3798389868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489d400640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f42280 0x561d35f67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489de00640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 0x561d35f6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489de00640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 0x7f4880079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4890007fd0 tx=0x7f4890009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.778+0000 7f489e800640 1 -- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 msgr2=0x561d35f6b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489e800640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 0x561d35f6b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489e800640 1 -- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f42280 msgr2=0x561d35f67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489e800640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f42280 0x561d35f67250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489e800640 1 -- 172.21.15.44:0/3798389868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d35c06850 con 0x561d35f67790 2024-07-17T07:10:28.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489de00640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 0x561d35f6b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:10:28.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f489e800640 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f489802b000 tx=0x7f4898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd96b3298b788c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f488a600640 1 -- 172.21.15.44:0/3798389868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898046070 con 0x561d35f67790 2024-07-17T07:10:28.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f488a600640 1 -- 172.21.15.44:0/3798389868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4898003b20 con 0x561d35f67790 2024-07-17T07:10:28.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.779+0000 7f488a600640 1 -- 172.21.15.44:0/3798389868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898033040 con 0x561d35f67790 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 -- 172.21.15.44:0/3798389868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 msgr2=0x561d35f67c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f489802b000 tx=0x7f4898004a40 comp rx=0 tx=0).stop 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 -- 172.21.15.44:0/3798389868 shutdown_connections 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f67c40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f42280 0x561d35f67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 --2- 172.21.15.44:0/3798389868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f6afc0 0x561d35f6b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 -- 172.21.15.44:0/3798389868 >> 172.21.15.44:0/3798389868 conn(0x561d35f3d460 msgr2=0x561d35f40e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 -- 172.21.15.44:0/3798389868 shutdown_connections 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.780+0000 7f48a781e740 1 -- 172.21.15.44:0/3798389868 wait complete. 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.781+0000 7f48a781e740 1 -- start start 2024-07-17T07:10:28.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.781+0000 7f48a781e740 1 Processor -- start 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35f66950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f60730 0x561d35f60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561d35c15620 con 0x561d35f6afc0 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561d35e49f90 con 0x561d35f60730 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561d35f3bb60 con 0x561d35f67790 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f48a781e740 10 client.0 _ll_register_callbacks cb 0x561d35e6ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38026/0 (socket says 172.21.15.44:38026) 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.782+0000 7f489d400640 1 -- 172.21.15.44:0/2072806773 learned_addr learned my addr 172.21.15.44:0/2072806773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:28.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 -- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 msgr2=0x561d35f66950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489de00640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35f66950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489e800640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f60730 0x561d35f60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35f66950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 -- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f60730 msgr2=0x561d35f60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f60730 0x561d35f60bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 -- 172.21.15.44:0/2072806773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d35c7d6b0 con 0x561d35f6afc0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f489d400640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f488c05cd90 tx=0x7f488c0608e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7581cfbb660981 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f487ea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c060d10 con 0x561d35f6afc0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f488c05ff70 con 0x561d35f6afc0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.783+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561d35e79900 con 0x561d35f6afc0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f488c065310 con 0x561d35f6afc0 2024-07-17T07:10:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 10 client.6390 fetch_fsmap learned FSMap version 35 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561d35f62dc0 con 0x561d35f6afc0 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561d35e79900 con 0x561d35f6afc0 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f488c05c680 con 0x561d35f6afc0 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 10 client.6390 fetch_fsmap finished waiting for FSMap version 35 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 10 client.6390 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 20 client.6390 populate_metadata read hostname 'smithi044' 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35f7e660 0x561d35f7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 4 client.6390 mds_command: new command op to 6261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 --> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d35eb4cb0 con 0x561d35f7e660 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.784+0000 7f489de00640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35f7e660 0x561d35f7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.785+0000 7f489de00640 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35f7e660 0x561d35f7ea40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef463df61c0692ae server_cookie=c6d739569c9132f4 in_seq=0 out_seq=0 2024-07-17T07:10:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.785+0000 7f487ea00640 10 client.6390 ms_handle_connect on v2:172.21.15.44:6834/2638137978 2024-07-17T07:10:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f487ea00640 1 -- 172.21.15.44:0/2072806773 <== mds.0 v2:172.21.15.44:6834/2638137978 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561d35eb4cb0 con 0x561d35f7e660 2024-07-17T07:10:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f487ea00640 10 client.6390 handle_command_reply: tid=0 2024-07-17T07:10:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 client.6390 shutdown 2024-07-17T07:10:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 msgr2=0x561d35f4fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f488c05cd90 tx=0x7f488c0608e0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 shutdown_connections 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35f7e660 0x561d35f7ea40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f60730 0x561d35f60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f6afc0 0x561d35f4fe30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f487ea00640 0 client.6390 ms_handle_reset on v2:172.21.15.44:6834/2638137978 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 --2- 172.21.15.44:0/2072806773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35f66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.786+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 >> 172.21.15.44:0/2072806773 conn(0x561d35f42280 msgr2=0x561d35f3a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.787+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 shutdown_connections 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.787+0000 7f48a781e740 1 -- 172.21.15.44:0/2072806773 wait complete. 2024-07-17T07:10:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.787+0000 7f48a781e740 20 client.6390 trim_cache size 0 max 16384 2024-07-17T07:10:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.807+0000 7f48a781e740 1 Processor -- start 2024-07-17T07:10:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 -- start start 2024-07-17T07:10:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35fa96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35fa9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0201490 con 0x561d35f60730 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a02004b0 con 0x561d35f6afc0 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.808+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48a0151dc0 con 0x561d35f67790 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38038/0 (socket says 172.21.15.44:38038) 2024-07-17T07:10:28.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 -- 172.21.15.44:0/2634172743 learned_addr learned my addr 172.21.15.44:0/2634172743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 -- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 msgr2=0x561d35fa96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35fa96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 -- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 msgr2=0x561d35fa9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35fa9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 -- 172.21.15.44:0/2634172743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d35f99620 con 0x561d35f60730 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.809+0000 7f489de00640 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x561d35f7f370 tx=0x7f4890014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f1c54ad7dae28b server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f4889c00640 1 -- 172.21.15.44:0/2634172743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489002f070 con 0x561d35f60730 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f4889c00640 1 -- 172.21.15.44:0/2634172743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f489000ee10 con 0x561d35f60730 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f4889c00640 1 -- 172.21.15.44:0/2634172743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489001a040 con 0x561d35f60730 2024-07-17T07:10:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 -- 172.21.15.44:0/2634172743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 msgr2=0x561d35f40d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x561d35f7f370 tx=0x7f4890014000 comp rx=0 tx=0).stop 2024-07-17T07:10:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 -- 172.21.15.44:0/2634172743 shutdown_connections 2024-07-17T07:10:28.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35fa9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f67790 0x561d35fa96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 --2- 172.21.15.44:0/2634172743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f60730 0x561d35f40d70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.810+0000 7f48a781e740 1 -- 172.21.15.44:0/2634172743 >> 172.21.15.44:0/2634172743 conn(0x561d35f42280 msgr2=0x561d35f36740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.811+0000 7f48a781e740 1 -- 172.21.15.44:0/2634172743 shutdown_connections 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.811+0000 7f48a781e740 1 -- 172.21.15.44:0/2634172743 wait complete. 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.811+0000 7f48a781e740 1 -- start start 2024-07-17T07:10:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.811+0000 7f48a781e740 1 Processor -- start 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 0x561d360049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35f97f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561d35c15620 con 0x561d35f67790 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01ffe60 con 0x561d35f6afc0 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.812+0000 7f48a781e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561d35f63380 con 0x561d35f60730 2024-07-17T07:10:28.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f48a781e740 10 client.0 _ll_register_callbacks cb 0x7f488c060ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38046/0 (socket says 172.21.15.44:38046) 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 -- 172.21.15.44:0/315780775 learned_addr learned my addr 172.21.15.44:0/315780775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489de00640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 0x561d360049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489e800640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35f97f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 -- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 msgr2=0x561d360049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 0x561d360049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 -- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 msgr2=0x561d35f97f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35f97f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489d400640 1 -- 172.21.15.44:0/315780775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d35f7f7b0 con 0x561d35f67790 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489e800640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35f97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:28.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.813+0000 7f489de00640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 0x561d360049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f489d400640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f488c05df80 tx=0x7f488c0604f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3766ec0faec92a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c060170 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f488c05f9e0 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f489000ee10 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f488c07f070 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 10 client.6396 fetch_fsmap learned FSMap version 35 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x561d35f7f7b0 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f489000ee10 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f488c05fc00 con 0x561d35f67790 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 10 client.6396 fetch_fsmap finished waiting for FSMap version 35 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 10 client.6396 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:10:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 20 client.6396 populate_metadata read hostname 'smithi044' 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35ffe1d0 0x561d35f7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 4 client.6396 mds_command: new command op to 6261 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.814+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 --> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] -- command(tid 0: {"prefix": "status"}) -- 0x561d35eb0750 con 0x561d35ffe1d0 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.815+0000 7f489de00640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35ffe1d0 0x561d35f7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.815+0000 7f489de00640 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35ffe1d0 0x561d35f7c200 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa07aea3b49a3a03 server_cookie=85be9ae1b25c7e6a in_seq=0 out_seq=0 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.815+0000 7f487ea00640 10 client.6396 ms_handle_connect on v2:172.21.15.44:6834/2638137978 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f487ea00640 1 -- 172.21.15.44:0/315780775 <== mds.0 v2:172.21.15.44:6834/2638137978 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561d35eb0750 con 0x561d35ffe1d0 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f487ea00640 10 client.6396 handle_command_reply: tid=0 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 client.6396 shutdown 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 msgr2=0x561d35f97a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f488c05df80 tx=0x7f488c0604f0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 shutdown_connections 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:6834/2638137978,v1:172.21.15.44:6835/2638137978] conn(0x561d35ffe1d0 0x561d35f7c200 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561d35f6afc0 0x561d35f97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d35f67790 0x561d35f97a30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 --2- 172.21.15.44:0/315780775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561d35f60730 0x561d360049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 >> 172.21.15.44:0/315780775 conn(0x561d35f42280 msgr2=0x561d35f40d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f487ea00640 0 client.6396 ms_handle_reset on v2:172.21.15.44:6834/2638137978 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 shutdown_connections 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 1 -- 172.21.15.44:0/315780775 wait complete. 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.816+0000 7f48a781e740 20 client.6396 trim_cache size 0 max 16384 2024-07-17T07:10:28.820 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6261, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7438910749999996, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 35, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 83, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 83, 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.605598235 2024-07-17T07:10:28.821 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:10:28.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 msgr2=0x7f4880079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 0x7f4880079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4890007fd0 tx=0x7f4890009040 comp rx=0 tx=0).stop 2024-07-17T07:10:28.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 msgr2=0x7f48a0153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:28.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c480 tx=0x7f488c00c940 comp rx=0 tx=0).stop 2024-07-17T07:10:28.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 shutdown_connections 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f4880076bb0 0x7f4880079070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a0161be0 0x7f48a0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48a015d130 0x7f48a0153550 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 --2- 172.21.15.44:0/970875099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a00ce690 0x7f48a015c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 >> 172.21.15.44:0/970875099 conn(0x7f48a00b4ff0 msgr2=0x7f48a00d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 shutdown_connections 2024-07-17T07:10:28.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:28.817+0000 7f489f200640 1 -- 172.21.15.44:0/970875099 wait complete. 2024-07-17T07:10:28.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:10:29.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 -- 172.21.15.44:0/709165289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 msgr2=0x7f527c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 --2- 172.21.15.44:0/709165289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c153a50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f526c00a940 tx=0x7f526c02b840 comp rx=0 tx=0).stop 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 -- 172.21.15.44:0/709165289 shutdown_connections 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 --2- 172.21.15.44:0/709165289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 0x7f527c13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 --2- 172.21.15.44:0/709165289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c153a50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 -- 172.21.15.44:0/709165289 >> 172.21.15.44:0/709165289 conn(0x7f527c0b0fb0 msgr2=0x7f527c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.045+0000 7f5281200640 1 -- 172.21.15.44:0/709165289 shutdown_connections 2024-07-17T07:10:29.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.046+0000 7f5281200640 1 -- 172.21.15.44:0/709165289 wait complete. 2024-07-17T07:10:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.047+0000 7f5281200640 1 Processor -- start 2024-07-17T07:10:29.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.047+0000 7f5281200640 1 -- start start 2024-07-17T07:10:29.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.048+0000 7f5281200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.048+0000 7f5281200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0ac3e0 0x7f527c0741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5281200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 0x7f527c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5281200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f527c0ae800 con 0x7f527c06f710 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5281200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f527c06f0a0 con 0x7f527c0ac3e0 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5281200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f527c0ae980 con 0x7f527c0ac870 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55010/0 (socket says 172.21.15.44:55010) 2024-07-17T07:10:29.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 -- 172.21.15.44:0/2349835588 learned_addr learned my addr 172.21.15.44:0/2349835588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f527a800640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 0x7f527c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279400640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0ac3e0 0x7f527c0741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 -- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 msgr2=0x7f527c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 0x7f527c151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 -- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0ac3e0 msgr2=0x7f527c0741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0ac3e0 0x7f527c0741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.049+0000 7f5279e00640 1 -- 172.21.15.44:0/2349835588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c074730 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5279e00640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f526c000c00 tx=0x7f526c031e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c041030 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f526c003800 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c003c50 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c1f3570 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.050+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c1f3b30 con 0x7f527c06f710 2024-07-17T07:10:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.051+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527c050670 con 0x7f527c06f710 2024-07-17T07:10:29.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.053+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7f526c044030 con 0x7f527c06f710 2024-07-17T07:10:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.054+0000 7f5272000640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 0x7f525c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.054+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f526c0b9520 con 0x7f527c06f710 2024-07-17T07:10:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.054+0000 7f5279400640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 0x7f525c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.056+0000 7f5279400640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 0x7f525c079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f527c13f820 tx=0x7f5268009040 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:29.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.058+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f527c050670 con 0x7f527c06f710 2024-07-17T07:10:29.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.310+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f527c13fad0 con 0x7f527c06f710 2024-07-17T07:10:29.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.311+0000 7f5272000640 1 -- 172.21.15.44:0/2349835588 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f526c04b030 con 0x7f527c06f710 2024-07-17T07:10:29.311 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:10:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 msgr2=0x7f525c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 0x7f525c079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f527c13f820 tx=0x7f5268009040 comp rx=0 tx=0).stop 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 msgr2=0x7f527c071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f526c000c00 tx=0x7f526c031e90 comp rx=0 tx=0).stop 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 shutdown_connections 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f527c0ac870 0x7f527c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f525c076bb0 0x7f525c079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c0ac3e0 0x7f527c0741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 --2- 172.21.15.44:0/2349835588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f527c06f710 0x7f527c071620 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 >> 172.21.15.44:0/2349835588 conn(0x7f527c0b0fb0 msgr2=0x7f527c13ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 shutdown_connections 2024-07-17T07:10:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29.314+0000 7f5281200640 1 -- 172.21.15.44:0/2349835588 wait complete. 2024-07-17T07:10:29.333 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:10:29.334 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:10:29.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:29.335 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:10:29.383 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:10:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:10:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:10:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:10:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41628sec preferred_lft 41628sec 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:10:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:10:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:10:29.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:29.387 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:10:29.505 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:10:29.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:29.510 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:10:29.559 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:10:29.559 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:10:29.559 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:10:29.559 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:10:29.559 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:10:29.560 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:10:29.560 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:10:29.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:10:29.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:10:29.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:10:29.561 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:10:29.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:10:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:10:29.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:29.781 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:10:29.830 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:10:29.830 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:10:29.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:10:29.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:29.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:10:29.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:10:29.914 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:10:29.971 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:10:29.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:10:29.971 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:10:30.025 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:10:30.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:30.026 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:10:30.153 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:10:30.153 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:10:30.153 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:10:30.174 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:10:30.174 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:10:30.174 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:10:30.174 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:10:30.174 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:10:30.203 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:10:30.206 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:10:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:10:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:10:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:10:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:10:30.212 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:10:30.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:30.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:10:30.289 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:10:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:10:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:10:30.416 INFO:teuthology.orchestra.run.smithi044.stdout:client5339 2024-07-17T07:10:30.416 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:10:30.416 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:10:30.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.722+0000 7fdfd2000640 1 -- 172.21.15.44:0/1542606359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06f4d0 msgr2=0x7fdfcc15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.722+0000 7fdfd2000640 1 --2- 172.21.15.44:0/1542606359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06f4d0 0x7fdfcc15a450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00a940 tx=0x7fdfbc02b840 comp rx=0 tx=0).stop 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.723+0000 7fdfd2000640 1 -- 172.21.15.44:0/1542606359 shutdown_connections 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.723+0000 7fdfd2000640 1 --2- 172.21.15.44:0/1542606359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06f4d0 0x7fdfcc15a450 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.723+0000 7fdfd2000640 1 -- 172.21.15.44:0/1542606359 >> 172.21.15.44:0/1542606359 conn(0x7fdfcc0cf0d0 msgr2=0x7fdfcc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.723+0000 7fdfd2000640 1 -- 172.21.15.44:0/1542606359 shutdown_connections 2024-07-17T07:10:30.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.723+0000 7fdfd2000640 1 -- 172.21.15.44:0/1542606359 wait complete. 2024-07-17T07:10:30.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.725+0000 7fdfd2000640 1 Processor -- start 2024-07-17T07:10:30.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.725+0000 7fdfd2000640 1 -- start start 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfcc06f4d0 0x7fdfcc14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfcc1503c0 0x7fdfcc14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc06dbe0 con 0x7fdfcc06d420 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc06da60 con 0x7fdfcc06f4d0 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc06dd60 con 0x7fdfcc1503c0 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:30.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55020/0 (socket says 172.21.15.44:55020) 2024-07-17T07:10:30.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.726+0000 7fdfd0c00640 1 -- 172.21.15.44:0/2648578495 learned_addr learned my addr 172.21.15.44:0/2648578495 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 -- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfcc1503c0 msgr2=0x7fdfcc14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfcc1503c0 0x7fdfcc14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 -- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfcc06f4d0 msgr2=0x7fdfcc14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfcc06f4d0 0x7fdfcc14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 -- 172.21.15.44:0/2648578495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc0cee90 con 0x7fdfcc06d420 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd0c00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc031e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc041030 con 0x7fdfcc06d420 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc003910 con 0x7fdfcc06d420 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003d60 con 0x7fdfcc06d420 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.727+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc0cdf50 con 0x7fdfcc06d420 2024-07-17T07:10:30.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.728+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc0cc7b0 con 0x7fdfcc06d420 2024-07-17T07:10:30.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.729+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc15cc50 con 0x7fdfcc06d420 2024-07-17T07:10:30.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.730+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fdfbc044030 con 0x7fdfcc06d420 2024-07-17T07:10:30.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.732+0000 7fdfc8c00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 0x7fdfac079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:30.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.732+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fdfbc0b9660 con 0x7fdfcc06d420 2024-07-17T07:10:30.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.733+0000 7fdfcbe00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 0x7fdfac079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:30.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.734+0000 7fdfcbe00640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 0x7fdfac079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc09d300 tx=0x7fdfb8007930 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:30.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.736+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdfbc04b030 con 0x7fdfcc06d420 2024-07-17T07:10:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:30.976+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 --> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fdfcc15a450 con 0x7fdfac076bb0 2024-07-17T07:10:37.944 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.945+0000 7f54e3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.945 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.945+0000 7f54e3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.946 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.945+0000 7f54e3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.946 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.945+0000 7f54e3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.946 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.945+0000 7f54e3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.946 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.947+0000 7f54e1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.946 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.947+0000 7f54e1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.947 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.947+0000 7f54e1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.947 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.947+0000 7f54e1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.947 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:37.947+0000 7f54e1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:37.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.962+0000 7fdfc8c00640 1 -- 172.21.15.44:0/2648578495 <== mgr.5404 v2:172.21.15.175:6800/2949346159 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fdfcc15a450 con 0x7fdfac076bb0 2024-07-17T07:10:37.961 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:10:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:10:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:10:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:10:37.962 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 msgr2=0x7fdfac079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 0x7fdfac079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc09d300 tx=0x7fdfb8007930 comp rx=0 tx=0).stop 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 msgr2=0x7fdfcc0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc031e90 comp rx=0 tx=0).stop 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 shutdown_connections 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfcc1503c0 0x7fdfcc14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdfac076bb0 0x7fdfac079070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:37.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfcc06f4d0 0x7fdfcc14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:37.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 --2- 172.21.15.44:0/2648578495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfcc06d420 0x7fdfcc0cd290 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:37.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.966+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 >> 172.21.15.44:0/2648578495 conn(0x7fdfcc0cf0d0 msgr2=0x7fdfcc15ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:37.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.967+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 shutdown_connections 2024-07-17T07:10:37.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:37.967+0000 7fdfd2000640 1 -- 172.21.15.44:0/2648578495 wait complete. 2024-07-17T07:10:37.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:10:38.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 -- 172.21.15.44:0/3974993184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 msgr2=0x7fdb5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 --2- 172.21.15.44:0/3974993184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb5c0051c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005fd0 tx=0x7fdb5c005830 comp rx=0 tx=0).stop 2024-07-17T07:10:38.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb76800640 1 -- 172.21.15.44:0/3974993184 reap_dead start 2024-07-17T07:10:38.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 -- 172.21.15.44:0/3974993184 shutdown_connections 2024-07-17T07:10:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 -- 172.21.15.44:0/3974993184 >> 172.21.15.44:0/3974993184 conn(0x7fdb7806c120 msgr2=0x7fdb7806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 -- 172.21.15.44:0/3974993184 shutdown_connections 2024-07-17T07:10:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.205+0000 7fdb77200640 1 -- 172.21.15.44:0/3974993184 wait complete. 2024-07-17T07:10:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.207+0000 7fdb77200640 1 Processor -- start 2024-07-17T07:10:38.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.207+0000 7fdb77200640 1 -- start start 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 0x7fdb7814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 0x7fdb780cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb780b24d0 con 0x7fdb5c004d40 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb780b2350 con 0x7fdb780b02d0 2024-07-17T07:10:38.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb77200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb780ae480 con 0x7fdb780cc580 2024-07-17T07:10:38.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb75e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb75e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55030/0 (socket says 172.21.15.44:55030) 2024-07-17T07:10:38.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.208+0000 7fdb75e00640 1 -- 172.21.15.44:0/3070248450 learned_addr learned my addr 172.21.15.44:0/3070248450 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:38.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 -- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 msgr2=0x7fdb7814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb76800640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 0x7fdb780cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75400640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 0x7fdb7814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 0x7fdb7814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 -- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 msgr2=0x7fdb780cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 0x7fdb780cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 -- 172.21.15.44:0/3070248450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7814ea90 con 0x7fdb5c004d40 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb76800640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 0x7fdb780cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75400640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 0x7fdb7814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb75e00640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800e990 tx=0x7fdb6800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800cbe0 con 0x7fdb5c004d40 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb68005020 con 0x7fdb5c004d40 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68005450 con 0x7fdb5c004d40 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.209+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb78154910 con 0x7fdb5c004d40 2024-07-17T07:10:38.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.210+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb78154ed0 con 0x7fdb5c004d40 2024-07-17T07:10:38.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.211+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb78155310 con 0x7fdb5c004d40 2024-07-17T07:10:38.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.212+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) ==== 98829+0+0 (secure 0 0 0) 0x7fdb680047a0 con 0x7fdb5c004d40 2024-07-17T07:10:38.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.213+0000 7fdb6e000640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 0x7fdb54079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.213+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fdb68098f20 con 0x7fdb5c004d40 2024-07-17T07:10:38.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.214+0000 7fdb75400640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 0x7fdb54079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.215+0000 7fdb75400640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 0x7fdb54079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b1f00 tx=0x7fdb60001660 comp rx=0 tx=0).ready entity=mgr.5404 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:38.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.218+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdb7814ea90 con 0x7fdb5c004d40 2024-07-17T07:10:38.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.482+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fdb780aea10 con 0x7fdb5c004d40 2024-07-17T07:10:38.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.541+0000 7fdb6e000640 1 -- 172.21.15.44:0/3070248450 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) ==== 92+0+0 (secure 0 0 0) 0x7fdb6806a020 con 0x7fdb5c004d40 2024-07-17T07:10:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 msgr2=0x7fdb54079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 0x7fdb54079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b1f00 tx=0x7fdb60001660 comp rx=0 tx=0).stop 2024-07-17T07:10:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 msgr2=0x7fdb780af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800e990 tx=0x7fdb6800ee50 comp rx=0 tx=0).stop 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 shutdown_connections 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fdb54076bb0 0x7fdb54079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb780b02d0 0x7fdb780cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb780cc580 0x7fdb7814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.544+0000 7fdb77200640 1 --2- 172.21.15.44:0/3070248450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb5c004d40 0x7fdb780af940 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.545+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 >> 172.21.15.44:0/3070248450 conn(0x7fdb7806c120 msgr2=0x7fdb780b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.545+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 shutdown_connections 2024-07-17T07:10:38.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.545+0000 7fdb77200640 1 -- 172.21.15.44:0/3070248450 wait complete. 2024-07-17T07:10:38.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.775+0000 7fe39b000640 1 -- 172.21.15.44:0/4055925278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe394161be0 msgr2=0x7fe3940b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.775+0000 7fe39b000640 1 --2- 172.21.15.44:0/4055925278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe394161be0 0x7fe3940b2200 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3940d0de0 tx=0x7fe38002a980 comp rx=0 tx=0).stop 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 -- 172.21.15.44:0/4055925278 shutdown_connections 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 --2- 172.21.15.44:0/4055925278 >> v2:172.21.15.175:3300/0 conn(0x7fe3940ce690 0x7fe3940cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 --2- 172.21.15.44:0/4055925278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe394161be0 0x7fe3940b2200 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 --2- 172.21.15.44:0/4055925278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 0x7fe39415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 -- 172.21.15.44:0/4055925278 >> 172.21.15.44:0/4055925278 conn(0x7fe3940b4ff0 msgr2=0x7fe3940b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 -- 172.21.15.44:0/4055925278 shutdown_connections 2024-07-17T07:10:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.776+0000 7fe39b000640 1 -- 172.21.15.44:0/4055925278 wait complete. 2024-07-17T07:10:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.777+0000 7fe39b000640 1 Processor -- start 2024-07-17T07:10:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.778+0000 7fe39b000640 1 -- start start 2024-07-17T07:10:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 0x7fe394158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe394161be0 0x7fe3940d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe39406c7a0 con 0x7fe3940ce690 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe39406c620 con 0x7fe394161be0 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe39406c920 con 0x7fe39415d130 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe399c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe399c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55058/0 (socket says 172.21.15.44:55058) 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe399c00640 1 -- 172.21.15.44:0/987302228 learned_addr learned my addr 172.21.15.44:0/987302228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe399200640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 0x7fe394158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.779+0000 7fe39a600640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe394161be0 0x7fe3940d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 msgr2=0x7fe394158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 0x7fe394158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe394161be0 msgr2=0x7fe3940d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe394161be0 0x7fe3940d1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 -- 172.21.15.44:0/987302228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3941f70c0 con 0x7fe3940ce690 2024-07-17T07:10:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe399c00640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d950 tx=0x7fe38400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384017070 con 0x7fe3940ce690 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe384009d20 con 0x7fe3940ce690 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3941f73a0 con 0x7fe3940ce690 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3840106c0 con 0x7fe3940ce690 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.780+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941f7960 con 0x7fe3940ce690 2024-07-17T07:10:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.782+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3940cfdf0 con 0x7fe3940ce690 2024-07-17T07:10:38.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.784+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98846+0+0 (secure 0 0 0) 0x7fe384010860 con 0x7fe3940ce690 2024-07-17T07:10:38.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.785+0000 7fe391600640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 0x7fe3740790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:38.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.785+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7fe3840989f0 con 0x7fe3940ce690 2024-07-17T07:10:38.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.785+0000 7fe399200640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 msgr2=0x7fe3740790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2949346159 2024-07-17T07:10:38.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.785+0000 7fe399200640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 0x7fe3740790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:38.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.790+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe384061b20 con 0x7fe3940ce690 2024-07-17T07:10:38.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.987+0000 7fe399200640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 msgr2=0x7fe3740790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2949346159 2024-07-17T07:10:38.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:38.987+0000 7fe399200640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 0x7fe3740790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.003+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fe39406d0b0 con 0x7fe3940ce690 2024-07-17T07:10:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.011+0000 7fe391600640 1 -- 172.21.15.44:0/987302228 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v37)=0 v37) ==== 138+0+0 (secure 0 0 0) 0x7fe384061420 con 0x7fe3940ce690 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 msgr2=0x7fe3740790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 0x7fe3740790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 msgr2=0x7fe39415fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d950 tx=0x7fe38400de10 comp rx=0 tx=0).stop 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 shutdown_connections 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7fe374076c00 0x7fe3740790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe394161be0 0x7fe3940d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe39415d130 0x7fe394158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 --2- 172.21.15.44:0/987302228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3940ce690 0x7fe39415fc40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 >> 172.21.15.44:0/987302228 conn(0x7fe3940b4ff0 msgr2=0x7fe3940b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 shutdown_connections 2024-07-17T07:10:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.014+0000 7fe39b000640 1 -- 172.21.15.44:0/987302228 wait complete. 2024-07-17T07:10:39.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:10:39.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 -- 172.21.15.44:0/938381895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b415b3a0 msgr2=0x7f34b40d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:39.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 --2- 172.21.15.44:0/938381895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b415b3a0 0x7f34b40d0ca0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005fd0 tx=0x7f34a802a5d0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 -- 172.21.15.44:0/938381895 shutdown_connections 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 --2- 172.21.15.44:0/938381895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b40b3040 0x7f34b40b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 --2- 172.21.15.44:0/938381895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b415b3a0 0x7f34b40d0ca0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 -- 172.21.15.44:0/938381895 >> 172.21.15.44:0/938381895 conn(0x7f34b40b5560 msgr2=0x7f34b40b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 -- 172.21.15.44:0/938381895 shutdown_connections 2024-07-17T07:10:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.224+0000 7f34ba000640 1 -- 172.21.15.44:0/938381895 wait complete. 2024-07-17T07:10:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.226+0000 7f34ba000640 1 Processor -- start 2024-07-17T07:10:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.226+0000 7f34ba000640 1 -- start start 2024-07-17T07:10:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b40b2210 0x7f34b40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b40b3040 0x7f34b41f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34b406c650 con 0x7f34b415b3a0 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34b40af940 con 0x7f34b40b2210 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.227+0000 7f34ba000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34b406c7d0 con 0x7f34b40b3040 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53612/0 (socket says 172.21.15.44:53612) 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 -- 172.21.15.44:0/672190182 learned_addr learned my addr 172.21.15.44:0/672190182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:10:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b8c00640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b40b2210 0x7f34b40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:10:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b40b3040 msgr2=0x7f34b41f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b40b3040 0x7f34b41f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b40b2210 msgr2=0x7f34b40ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b40b2210 0x7f34b40ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.228+0000 7f34b9600640 1 -- 172.21.15.44:0/672190182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b40ce5d0 con 0x7f34b415b3a0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34b9600640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c730 tx=0x7f34ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac003b90 con 0x7f34b415b3a0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34ac003d30 con 0x7f34b415b3a0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b41423a0 con 0x7f34b415b3a0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00aa50 con 0x7f34b415b3a0 2024-07-17T07:10:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.229+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b40ce840 con 0x7f34b415b3a0 2024-07-17T07:10:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.230+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b406f040 con 0x7f34b415b3a0 2024-07-17T07:10:39.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.232+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) ==== 98846+0+0 (secure 0 0 0) 0x7f34ac00abf0 con 0x7f34b415b3a0 2024-07-17T07:10:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.232+0000 7f34b0c00640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 0x7f34940790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:10:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.233+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6621+0+0 (secure 0 0 0) 0x7f34ac0984b0 con 0x7f34b415b3a0 2024-07-17T07:10:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.233+0000 7f34b8c00640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 msgr2=0x7f34940790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2949346159 2024-07-17T07:10:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.233+0000 7f34b8c00640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 0x7f34940790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:10:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.237+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34b406f040 con 0x7f34b415b3a0 2024-07-17T07:10:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.434+0000 7f34b8c00640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 msgr2=0x7f34940790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2949346159 2024-07-17T07:10:39.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.435+0000 7f34b8c00640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 0x7f34940790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:10:39.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.443+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f34b406cc80 con 0x7f34b415b3a0 2024-07-17T07:10:39.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.451+0000 7f34b0c00640 1 -- 172.21.15.44:0/672190182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v38)=0 v38) ==== 133+0+0 (secure 0 0 0) 0x7f34ac061600 con 0x7f34b415b3a0 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 msgr2=0x7f34940790f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 0x7f34940790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 msgr2=0x7f34b4141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c730 tx=0x7f34ac00cbf0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 shutdown_connections 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34b415b3a0 0x7f34b4141e60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b40b3040 0x7f34b41f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:6800/2949346159,v1:172.21.15.175:6801/2949346159] conn(0x7f3494076c30 0x7f34940790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 --2- 172.21.15.44:0/672190182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b40b2210 0x7f34b40ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 >> 172.21.15.44:0/672190182 conn(0x7f34b40b5560 msgr2=0x7f34b4157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 shutdown_connections 2024-07-17T07:10:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:10:39.454+0000 7f34ba000640 1 -- 172.21.15.44:0/672190182 wait complete. 2024-07-17T07:10:39.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:10:39.470 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T07:10:43.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.066+0000 7fa833a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.066+0000 7fa833a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.066+0000 7fa833a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.066+0000 7fa833a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.066+0000 7fa833a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.067+0000 7fa82f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.067+0000 7fa82f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.067+0000 7fa82f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.067+0000 7fa82f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:43.066 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:10:43.067+0000 7fa82f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:10:45.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:10:45.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:11:15.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:11:15.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:11:42.225 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 -- 172.21.15.44:0/4005491754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae0161120 msgr2=0x7faae006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 --2- 172.21.15.44:0/4005491754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae0161120 0x7faae006d620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faacc009860 tx=0x7faacc02a540 comp rx=0 tx=0).stop 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 -- 172.21.15.44:0/4005491754 shutdown_connections 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 --2- 172.21.15.44:0/4005491754 >> v2:172.21.15.175:3300/0 conn(0x7faae006db60 0x7faae00ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 --2- 172.21.15.44:0/4005491754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae0161120 0x7faae006d620 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:42.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 -- 172.21.15.44:0/4005491754 >> 172.21.15.44:0/4005491754 conn(0x7faae00b0eb0 msgr2=0x7faae00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:42.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 -- 172.21.15.44:0/4005491754 shutdown_connections 2024-07-17T07:11:42.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.450+0000 7faae5a00640 1 -- 172.21.15.44:0/4005491754 wait complete. 2024-07-17T07:11:42.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.452+0000 7faae5a00640 1 Processor -- start 2024-07-17T07:11:42.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.452+0000 7faae5a00640 1 -- start start 2024-07-17T07:11:42.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.453+0000 7faae5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae0154600 0x7faae0150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:42.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.453+0000 7faae5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae0161120 0x7faae0156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:42.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faae5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae01591b0 0x7faae0154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faae5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faae015e220 con 0x7faae0154600 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faae5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faae015e0a0 con 0x7faae0161120 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faae5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faae015e3a0 con 0x7faae01591b0 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae0154600 0x7faae0150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae0154600 0x7faae0150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60106/0 (socket says 172.21.15.44:60106) 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 -- 172.21.15.44:0/2140265836 learned_addr learned my addr 172.21.15.44:0/2140265836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadf400640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae01591b0 0x7faae0154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faade000640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae0161120 0x7faae0156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 -- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae01591b0 msgr2=0x7faae0154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:42.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae01591b0 0x7faae0154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:42.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 -- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae0161120 msgr2=0x7faae0156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:42.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae0161120 0x7faae0156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:42.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.454+0000 7faadea00640 1 -- 172.21.15.44:0/2140265836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae0159870 con 0x7faae0154600 2024-07-17T07:11:42.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faadea00640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faae0154600 0x7faae0150fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faad000c480 tx=0x7faad000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00052d0 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad0005470 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0012870 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faae5a00640 1 -- 172.21.15.44:0/2140265836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae0159b50 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.455+0000 7faae5a00640 1 -- 172.21.15.44:0/2140265836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae015a110 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.456+0000 7faae5a00640 1 -- 172.21.15.44:0/2140265836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0157f90 con 0x7faae0154600 2024-07-17T07:11:42.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.458+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 99005+0+0 (secure 0 0 0) 0x7faad00035f0 con 0x7faae0154600 2024-07-17T07:11:42.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.459+0000 7faad6a00640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7faac0076de0 0x7faac00792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:42.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.459+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6879+0+0 (secure 0 0 0) 0x7faad0098750 con 0x7faae0154600 2024-07-17T07:11:42.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.460+0000 7faade000640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7faac0076de0 0x7faac00792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:42.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.461+0000 7faade000640 1 --2- 172.21.15.44:0/2140265836 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7faac0076de0 0x7faac00792a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faacc034000 tx=0x7faacc004730 comp rx=0 tx=0).ready entity=mgr.5602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:42.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.463+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faae01549e0 con 0x7faae0154600 2024-07-17T07:11:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.695+0000 7faae5a00640 1 -- 172.21.15.44:0/2140265836 --> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7faae015e8c0 con 0x7faac0076de0 2024-07-17T07:11:42.711 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:42.712+0000 7fa83c600640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-07-17T07:11:42.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.713+0000 7faad6a00640 1 -- 172.21.15.44:0/2140265836 <== mgr.5602 v2:172.21.15.175:6800/1065445688 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) ==== 49+0+0 (secure 0 0 0) 0x7faae015e8c0 con 0x7faac0076de0 2024-07-17T07:11:42.712 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-07-17T07:11:42.727 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:11:42.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:11:42.727 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T07:11:42.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:11:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.999+0000 7fcc03200640 1 -- 172.21.15.44:0/1584274013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc040cefe0 msgr2=0x7fcc040cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:42.999+0000 7fcc03200640 1 --2- 172.21.15.44:0/1584274013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc040cefe0 0x7fcc040cf450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406e790 tx=0x7fcbfc02a980 comp rx=0 tx=0).stop 2024-07-17T07:11:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 -- 172.21.15.44:0/1584274013 shutdown_connections 2024-07-17T07:11:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 --2- 172.21.15.44:0/1584274013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc040cefe0 0x7fcc040cf450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 --2- 172.21.15.44:0/1584274013 >> v2:172.21.15.175:3300/0 conn(0x7fcc040b2140 0x7fcc04156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 -- 172.21.15.44:0/1584274013 >> 172.21.15.44:0/1584274013 conn(0x7fcc0406c120 msgr2=0x7fcc0406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 -- 172.21.15.44:0/1584274013 shutdown_connections 2024-07-17T07:11:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.000+0000 7fcc03200640 1 -- 172.21.15.44:0/1584274013 wait complete. 2024-07-17T07:11:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.001+0000 7fcc03200640 1 Processor -- start 2024-07-17T07:11:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.002+0000 7fcc03200640 1 -- start start 2024-07-17T07:11:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc040b2140 0x7fcc041e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc040cefe0 0x7fcc040aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406f210 con 0x7fcc0406f5a0 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406f090 con 0x7fcc040cefe0 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0406d610 con 0x7fcc040b2140 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60116/0 (socket says 172.21.15.44:60116) 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 -- 172.21.15.44:0/3803767549 learned_addr learned my addr 172.21.15.44:0/3803767549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcbfbe00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc040b2140 0x7fcc041e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 -- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc040b2140 msgr2=0x7fcc041e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc02800640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc040cefe0 0x7fcc040aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc040b2140 0x7fcc041e4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 -- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc040cefe0 msgr2=0x7fcc040aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc040cefe0 0x7fcc040aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.003+0000 7fcc01e00640 1 -- 172.21.15.44:0/3803767549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc041e4d90 con 0x7fcc0406f5a0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcc01e00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c430 tx=0x7fcbf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4011790 con 0x7fcc0406f5a0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbf4017070 con 0x7fcc0406f5a0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4011cc0 con 0x7fcc0406f5a0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc040af2f0 con 0x7fcc0406f5a0 2024-07-17T07:11:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.004+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0415e500 con 0x7fcc0406f5a0 2024-07-17T07:11:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.005+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc04156d40 con 0x7fcc0406f5a0 2024-07-17T07:11:43.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.006+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 35) ==== 99005+0+0 (secure 0 0 0) 0x7fcbf401c460 con 0x7fcc0406f5a0 2024-07-17T07:11:43.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.007+0000 7fcbfa000640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 0x7fcbe0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:43.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.008+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6879+0+0 (secure 0 0 0) 0x7fcbf4098b20 con 0x7fcc0406f5a0 2024-07-17T07:11:43.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.008+0000 7fcbfbe00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 0x7fcbe0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:43.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.009+0000 7fcbfbe00640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 0x7fcbe0079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbec005fd0 tx=0x7fcbec005850 comp rx=0 tx=0).ready entity=mgr.5602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:43.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.012+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcbf4061840 con 0x7fcc0406f5a0 2024-07-17T07:11:43.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:43.285+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fcc040ae520 con 0x7fcc0406f5a0 2024-07-17T07:11:44.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.075+0000 7fcbfa000640 1 -- 172.21.15.44:0/3803767549 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v36) ==== 93+0+0 (secure 0 0 0) 0x7fcbf4061140 con 0x7fcc0406f5a0 2024-07-17T07:11:44.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 msgr2=0x7fcbe0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 0x7fcbe0079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbec005fd0 tx=0x7fcbec005850 comp rx=0 tx=0).stop 2024-07-17T07:11:44.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 msgr2=0x7fcc0414ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c430 tx=0x7fcbf400c8f0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 shutdown_connections 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fcbe0076db0 0x7fcbe0079270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc040cefe0 0x7fcc040aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc040b2140 0x7fcc041e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 --2- 172.21.15.44:0/3803767549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0406f5a0 0x7fcc0414ce50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.078+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 >> 172.21.15.44:0/3803767549 conn(0x7fcc0406c120 msgr2=0x7fcc041e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:44.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.079+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 shutdown_connections 2024-07-17T07:11:44.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.079+0000 7fcc03200640 1 -- 172.21.15.44:0/3803767549 wait complete. 2024-07-17T07:11:44.102 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:11:44.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:11:44.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:11:44.297 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:11:44.297 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:11:44.297 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:11:44.297 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:44.346 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:11:44.346 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:44.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:11:44.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 -- 172.21.15.44:0/1551553193 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 msgr2=0x7f73b8161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 --2- 172.21.15.44:0/1551553193 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8161650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009830 tx=0x7f73a402a5a0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 -- 172.21.15.44:0/1551553193 shutdown_connections 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 --2- 172.21.15.44:0/1551553193 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8161650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 --2- 172.21.15.44:0/1551553193 >> v2:172.21.15.175:3300/0 conn(0x7f73b80ce1d0 0x7f73b80ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 -- 172.21.15.44:0/1551553193 >> 172.21.15.44:0/1551553193 conn(0x7f73b80d3700 msgr2=0x7f73b80d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 -- 172.21.15.44:0/1551553193 shutdown_connections 2024-07-17T07:11:44.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.682+0000 7f73c0400640 1 -- 172.21.15.44:0/1551553193 wait complete. 2024-07-17T07:11:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.684+0000 7f73c0400640 1 Processor -- start 2024-07-17T07:11:44.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.684+0000 7f73c0400640 1 -- start start 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 0x7f73b80ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8162420 con 0x7f73b80ce1d0 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81622a0 con 0x7f73b80b29e0 2024-07-17T07:11:44.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73c0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8167910 con 0x7f73b80b6790 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73be400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73be400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60156/0 (socket says 172.21.15.44:60156) 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.685+0000 7f73be400640 1 -- 172.21.15.44:0/1692833290 learned_addr learned my addr 172.21.15.44:0/1692833290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73bda00640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 0x7f73b80ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73bd000640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 msgr2=0x7f73b8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 msgr2=0x7f73b80ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 0x7f73b80ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 -- 172.21.15.44:0/1692833290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b815aa40 con 0x7f73b80ce1d0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73bda00640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 0x7f73b80ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.686+0000 7f73be400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f73b000cbc0 tx=0x7f73b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.687+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0016020 con 0x7f73b80ce1d0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.687+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73b000add0 con 0x7f73b80ce1d0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.687+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0012680 con 0x7f73b80ce1d0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.687+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b815ad20 con 0x7f73b80ce1d0 2024-07-17T07:11:44.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.687+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b80caf40 con 0x7f73b80ce1d0 2024-07-17T07:11:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.688+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b80ce5b0 con 0x7f73b80ce1d0 2024-07-17T07:11:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.689+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7f73b0019020 con 0x7f73b80ce1d0 2024-07-17T07:11:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.691+0000 7f73b5600640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 0x7f7398079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.691+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7f73b0099090 con 0x7f73b80ce1d0 2024-07-17T07:11:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.691+0000 7f73bda00640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 msgr2=0x7f7398079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:44.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.691+0000 7f73bda00640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 0x7f7398079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.696+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f73b0020030 con 0x7f73b80ce1d0 2024-07-17T07:11:44.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.893+0000 7f73bda00640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 msgr2=0x7f7398079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:44.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.893+0000 7f73bda00640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 0x7f7398079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:44.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.918+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73b8157df0 con 0x7f73b80ce1d0 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.919+0000 7f73b5600640 1 -- 172.21.15.44:0/1692833290 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v86) ==== 80+0+360 (secure 0 0 0) 0x7f73b0061d30 con 0x7f73b80ce1d0 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2788898012 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3824917389 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/2949346159 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/2949346159 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4085204748 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1723723265 2024-07-18T07:10:42.755496+0000 2024-07-17T07:11:44.919 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T07:11:44.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 msgr2=0x7f7398079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:44.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 0x7f7398079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 msgr2=0x7f73b8156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:44.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f73b000cbc0 tx=0x7f73b000a430 comp rx=0 tx=0).stop 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 shutdown_connections 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f7398076d60 0x7f7398079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b80ce1d0 0x7f73b8156630 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80b6790 0x7f73b8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 --2- 172.21.15.44:0/1692833290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73b80b29e0 0x7f73b80ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 >> 172.21.15.44:0/1692833290 conn(0x7f73b80d3700 msgr2=0x7f73b80d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 shutdown_connections 2024-07-17T07:11:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:44.922+0000 7f73c0400640 1 -- 172.21.15.44:0/1692833290 wait complete. 2024-07-17T07:11:44.939 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:11:44.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:11:44.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:11:45.033 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:11:45.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:11:45.033 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:11:45.076 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:11:45.077 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:11:45.077 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:45.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:11:45.128 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:11:45.128 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:11:45.128 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:45.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:11:45.179 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:11:45.179 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:11:45.179 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:45.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:11:45.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:11:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 -- 172.21.15.44:0/3783997084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b5740 msgr2=0x7f5b20161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 --2- 172.21.15.44:0/3783997084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b5740 0x7f5b20161020 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5b20069c00 tx=0x7f5b1802aaa0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 -- 172.21.15.44:0/3783997084 shutdown_connections 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 --2- 172.21.15.44:0/3783997084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b2ad0 0x7f5b200b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 --2- 172.21.15.44:0/3783997084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b5740 0x7f5b20161020 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 --2- 172.21.15.44:0/3783997084 >> v2:172.21.15.175:3300/0 conn(0x7f5b2015d600 0x7f5b2006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 -- 172.21.15.44:0/3783997084 >> 172.21.15.44:0/3783997084 conn(0x7f5b200d00a0 msgr2=0x7f5b200d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 -- 172.21.15.44:0/3783997084 shutdown_connections 2024-07-17T07:11:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.431+0000 7f5b25e00640 1 -- 172.21.15.44:0/3783997084 wait complete. 2024-07-17T07:11:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.433+0000 7f5b25e00640 1 Processor -- start 2024-07-17T07:11:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.433+0000 7f5b25e00640 1 -- start start 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b5740 0x7f5b200cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 0x7f5b2015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201618f0 con 0x7f5b200b2ad0 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20161770 con 0x7f5b2015d600 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b25e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5b20161a70 con 0x7f5b200b5740 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.434+0000 7f5b1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60170/0 (socket says 172.21.15.44:60170) 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 -- 172.21.15.44:0/221337841 learned_addr learned my addr 172.21.15.44:0/221337841 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:45.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1f400640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 0x7f5b2015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b17e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b5740 0x7f5b200cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b5740 msgr2=0x7f5b200cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b5740 0x7f5b200cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 msgr2=0x7f5b2015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 0x7f5b2015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 -- 172.21.15.44:0/221337841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b200b7920 con 0x7f5b200b2ad0 2024-07-17T07:11:45.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1f400640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 0x7f5b2015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.435+0000 7f5b1ea00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c970 tx=0x7f5b1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.436+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10007bb0 con 0x7f5b200b2ad0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.436+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b10007e40 con 0x7f5b200b2ad0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.436+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000f690 con 0x7f5b200b2ad0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.436+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b200b7c00 con 0x7f5b200b2ad0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.436+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b200cbf30 con 0x7f5b200b2ad0 2024-07-17T07:11:45.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.437+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b20161020 con 0x7f5b200b2ad0 2024-07-17T07:11:45.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.438+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7f5b1000f830 con 0x7f5b200b2ad0 2024-07-17T07:11:45.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.439+0000 7f5b17400640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 0x7f5afc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.439+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7f5b10099300 con 0x7f5b200b2ad0 2024-07-17T07:11:45.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.440+0000 7f5b17e00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 msgr2=0x7f5afc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:45.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.440+0000 7f5b17e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 0x7f5afc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:45.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.444+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5b10014030 con 0x7f5b200b2ad0 2024-07-17T07:11:45.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:11:45.527 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:11:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.641+0000 7f5b17e00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 msgr2=0x7f5afc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.641+0000 7f5b17e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 0x7f5afc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:45.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.693+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b20160d30 con 0x7f5b200b2ad0 2024-07-17T07:11:45.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.694+0000 7f5b17400640 1 -- 172.21.15.44:0/221337841 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f5b10061f20 con 0x7f5b200b2ad0 2024-07-17T07:11:45.694 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:45.694 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:11:45.694 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:11:45.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 msgr2=0x7f5afc079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:45.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 0x7f5afc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 msgr2=0x7f5b2014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c970 tx=0x7f5b1000ce30 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 shutdown_connections 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5afc076d60 0x7f5afc079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b2015d600 0x7f5b2015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b200b5740 0x7f5b200cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 --2- 172.21.15.44:0/221337841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b200b2ad0 0x7f5b2014f320 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.697+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 >> 172.21.15.44:0/221337841 conn(0x7f5b200d00a0 msgr2=0x7f5b200b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.698+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 shutdown_connections 2024-07-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.698+0000 7f5b25e00640 1 -- 172.21.15.44:0/221337841 wait complete. 2024-07-17T07:11:45.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:11:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 -- 172.21.15.44:0/495586424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c15a380 msgr2=0x7fdf3c0cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 --2- 172.21.15.44:0/495586424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c15a380 0x7fdf3c0cece0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0d0430 tx=0x7fdf2402a510 comp rx=0 tx=0).stop 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 -- 172.21.15.44:0/495586424 shutdown_connections 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 --2- 172.21.15.44:0/495586424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 0x7fdf3c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 --2- 172.21.15.44:0/495586424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c15a380 0x7fdf3c0cece0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 -- 172.21.15.44:0/495586424 >> 172.21.15.44:0/495586424 conn(0x7fdf3c0b2240 msgr2=0x7fdf3c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 -- 172.21.15.44:0/495586424 shutdown_connections 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.955+0000 7fdf42400640 1 -- 172.21.15.44:0/495586424 wait complete. 2024-07-17T07:11:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.957+0000 7fdf42400640 1 Processor -- start 2024-07-17T07:11:45.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.957+0000 7fdf42400640 1 -- start start 2024-07-17T07:11:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 0x7fdf3c1e7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 0x7fdf3c1f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c06f740 con 0x7fdf3c159fa0 2024-07-17T07:11:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c0b6260 con 0x7fdf3c15a380 2024-07-17T07:11:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf42400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c06f8c0 con 0x7fdf3c0cf220 2024-07-17T07:11:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf32a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf32a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60182/0 (socket says 172.21.15.44:60182) 2024-07-17T07:11:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.958+0000 7fdf32a00640 1 -- 172.21.15.44:0/2251362623 learned_addr learned my addr 172.21.15.44:0/2251362623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:45.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf3be00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 0x7fdf3c1f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf3b400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 0x7fdf3c1e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:45.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 msgr2=0x7fdf3c1e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 0x7fdf3c1e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 msgr2=0x7fdf3c1f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 0x7fdf3c1f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 -- 172.21.15.44:0/2251362623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c1e76a0 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf3be00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 0x7fdf3c1f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf32a00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdf24034040 tx=0x7fdf240039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2403e030 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf240047a0 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c1e7980 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.959+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24035040 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.960+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c1e7f40 con 0x7fdf3c159fa0 2024-07-17T07:11:45.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.961+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c0b5940 con 0x7fdf3c159fa0 2024-07-17T07:11:45.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.962+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7fdf24041030 con 0x7fdf3c159fa0 2024-07-17T07:11:45.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.963+0000 7fdf33e00640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 0x7fdf18079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:45.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.964+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7fdf240b9420 con 0x7fdf3c159fa0 2024-07-17T07:11:45.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.964+0000 7fdf3b400640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 msgr2=0x7fdf18079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:45.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.964+0000 7fdf3b400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 0x7fdf18079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:45.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:45.968+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdf24081fb0 con 0x7fdf3c159fa0 2024-07-17T07:11:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.165+0000 7fdf3b400640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 msgr2=0x7fdf18079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:46.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.165+0000 7fdf3b400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 0x7fdf18079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:46.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.221+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf3c1e87d0 con 0x7fdf3c159fa0 2024-07-17T07:11:46.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.222+0000 7fdf33e00640 1 -- 172.21.15.44:0/2251362623 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7fdf2408a1f0 con 0x7fdf3c159fa0 2024-07-17T07:11:46.221 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:46.222 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:11:46.222 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 msgr2=0x7fdf18079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 0x7fdf18079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 msgr2=0x7fdf3c0aa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdf24034040 tx=0x7fdf240039e0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 shutdown_connections 2024-07-17T07:11:46.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fdf18076d60 0x7fdf18079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf3c15a380 0x7fdf3c1f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf3c159fa0 0x7fdf3c0aa0b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 --2- 172.21.15.44:0/2251362623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf3c0cf220 0x7fdf3c1e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 >> 172.21.15.44:0/2251362623 conn(0x7fdf3c0b2240 msgr2=0x7fdf3c15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 shutdown_connections 2024-07-17T07:11:46.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.225+0000 7fdf42400640 1 -- 172.21.15.44:0/2251362623 wait complete. 2024-07-17T07:11:46.243 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:11:46.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 -- 172.21.15.44:0/3141158259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 msgr2=0x7ff46c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 --2- 172.21.15.44:0/3141158259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff46c003f30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff478009b50 tx=0x7ff46c005830 comp rx=0 tx=0).stop 2024-07-17T07:11:46.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 -- 172.21.15.44:0/3141158259 shutdown_connections 2024-07-17T07:11:46.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 --2- 172.21.15.44:0/3141158259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff46c003f30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 -- 172.21.15.44:0/3141158259 >> 172.21.15.44:0/3141158259 conn(0x7ff4840cfec0 msgr2=0x7ff4840d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:46.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 -- 172.21.15.44:0/3141158259 shutdown_connections 2024-07-17T07:11:46.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.438+0000 7ff48a400640 1 -- 172.21.15.44:0/3141158259 wait complete. 2024-07-17T07:11:46.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.440+0000 7ff48a400640 1 Processor -- start 2024-07-17T07:11:46.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.440+0000 7ff48a400640 1 -- start start 2024-07-17T07:11:46.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ccfa0 0x7ff48415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 0x7ff4840c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840b5040 con 0x7ff46c003b10 2024-07-17T07:11:46.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840b4ec0 con 0x7ff48415d4c0 2024-07-17T07:11:46.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff48a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4840b51c0 con 0x7ff4840ccfa0 2024-07-17T07:11:46.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60206/0 (socket says 172.21.15.44:60206) 2024-07-17T07:11:46.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 -- 172.21.15.44:0/49831933 learned_addr learned my addr 172.21.15.44:0/49831933 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff483e00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ccfa0 0x7ff48415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489a00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 0x7ff4840c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ccfa0 msgr2=0x7ff48415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ccfa0 0x7ff48415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.441+0000 7ff489000640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 msgr2=0x7ff4840c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff489000640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 0x7ff4840c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff489000640 1 -- 172.21.15.44:0/49831933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4840c8360 con 0x7ff46c003b10 2024-07-17T07:11:46.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff489a00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 0x7ff4840c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:46.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff489000640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff478036040 tx=0x7ff4780336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:46.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478038040 con 0x7ff46c003b10 2024-07-17T07:11:46.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff478004230 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4780338e0 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4840c8640 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.442+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4840c8c00 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.444+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4841f73e0 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.445+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7ff47802b8a0 con 0x7ff46c003b10 2024-07-17T07:11:46.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.446+0000 7ff480c00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 0x7ff464079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.446+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7ff4780b9060 con 0x7ff46c003b10 2024-07-17T07:11:46.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.446+0000 7ff483e00640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 msgr2=0x7ff464079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:46.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.446+0000 7ff483e00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 0x7ff464079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:46.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.450+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff478081740 con 0x7ff46c003b10 2024-07-17T07:11:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.648+0000 7ff483e00640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 msgr2=0x7ff464079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:46.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.648+0000 7ff483e00640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 0x7ff464079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:46.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.662+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff48406dca0 con 0x7ff46c003b10 2024-07-17T07:11:46.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.665+0000 7ff480c00640 1 -- 172.21.15.44:0/49831933 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+18234 (secure 0 0 0) 0x7ff478086e20 con 0x7ff46c003b10 2024-07-17T07:11:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:46.664 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:11:44.053098+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:10:19.413303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-07-17T07:10:20.496271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.3d","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2788898012":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/3824917389":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6801/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6800/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/4085204748":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/1723723265":"2024-07-18T07:10:42.755496+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 msgr2=0x7ff464079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 0x7ff464079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 msgr2=0x7ff4841f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff478036040 tx=0x7ff4780336f0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 shutdown_connections 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7ff464076d90 0x7ff464079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff48415d4c0 0x7ff4840c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ccfa0 0x7ff48415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 --2- 172.21.15.44:0/49831933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff46c003b10 0x7ff4841f4eb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 >> 172.21.15.44:0/49831933 conn(0x7ff4840cfec0 msgr2=0x7ff4840b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 shutdown_connections 2024-07-17T07:11:46.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.669+0000 7ff48a400640 1 -- 172.21.15.44:0/49831933 wait complete. 2024-07-17T07:11:46.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:11:46.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.894+0000 7f5967200640 1 -- 172.21.15.44:0/3616510802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 msgr2=0x7f5944004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.894+0000 7f5967200640 1 --2- 172.21.15.44:0/3616510802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5944004800 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5948009b20 tx=0x7f5944005830 comp rx=0 tx=0).stop 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.894+0000 7f5965200640 1 -- 172.21.15.44:0/3616510802 reap_dead start 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.895+0000 7f5967200640 1 -- 172.21.15.44:0/3616510802 shutdown_connections 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.895+0000 7f5967200640 1 -- 172.21.15.44:0/3616510802 >> 172.21.15.44:0/3616510802 conn(0x7f59600b0fb0 msgr2=0x7f59600b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.895+0000 7f5967200640 1 -- 172.21.15.44:0/3616510802 shutdown_connections 2024-07-17T07:11:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.895+0000 7f5967200640 1 -- 172.21.15.44:0/3616510802 wait complete. 2024-07-17T07:11:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.896+0000 7f5967200640 1 Processor -- start 2024-07-17T07:11:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 -- start start 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5960073b00 0x7f59601f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 0x7f59600caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f596006f480 con 0x7f5944004020 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f596015b8d0 con 0x7f5960073f80 2024-07-17T07:11:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.897+0000 7f5967200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f596006f600 con 0x7f5960073b00 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60232/0 (socket says 172.21.15.44:60232) 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 -- 172.21.15.44:0/3221680860 learned_addr learned my addr 172.21.15.44:0/3221680860 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f5965200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 0x7f59600caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f5957400640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5960073b00 0x7f59601f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5960073b00 msgr2=0x7f59601f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5960073b00 0x7f59601f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 msgr2=0x7f59600caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 0x7f59600caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 -- 172.21.15.44:0/3221680860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59600c9600 con 0x7f5944004020 2024-07-17T07:11:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f5965200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 0x7f59600caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.898+0000 7f595fe00640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f595000d7d0 tx=0x7f595000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.899+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950018070 con 0x7f5944004020 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.899+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59500052e0 con 0x7f5944004020 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.899+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595000b900 con 0x7f5944004020 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.899+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59601f4c20 con 0x7f5944004020 2024-07-17T07:11:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.899+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59600c9b60 con 0x7f5944004020 2024-07-17T07:11:46.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.900+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59601549f0 con 0x7f5944004020 2024-07-17T07:11:46.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.902+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7f595001b070 con 0x7f5944004020 2024-07-17T07:11:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.903+0000 7f595cc00640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 0x7f593c079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.904+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7f59500987e0 con 0x7f5944004020 2024-07-17T07:11:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.904+0000 7f5957400640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 msgr2=0x7f593c079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.904+0000 7f5957400640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 0x7f593c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:46.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:46.907+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5950022070 con 0x7f5944004020 2024-07-17T07:11:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.105+0000 7f5957400640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 msgr2=0x7f593c079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:47.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.105+0000 7f5957400640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 0x7f593c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:47.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.161+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596006dbf0 con 0x7f5944004020 2024-07-17T07:11:47.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.162+0000 7f595cc00640 1 -- 172.21.15.44:0/3221680860 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f59600c9600 con 0x7f5944004020 2024-07-17T07:11:47.161 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:47.161 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:11:47.161 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:11:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.164+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 msgr2=0x7f593c079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.164+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 0x7f593c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.164+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 msgr2=0x7f5960071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.164+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f595000d7d0 tx=0x7f595000dc90 comp rx=0 tx=0).stop 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 shutdown_connections 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f593c076d90 0x7f593c079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5960073f80 0x7f59600caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5960073b00 0x7f59601f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 --2- 172.21.15.44:0/3221680860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5944004020 0x7f5960071620 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 >> 172.21.15.44:0/3221680860 conn(0x7f59600b0fb0 msgr2=0x7f59601598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 shutdown_connections 2024-07-17T07:11:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.165+0000 7f5967200640 1 -- 172.21.15.44:0/3221680860 wait complete. 2024-07-17T07:11:47.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.372+0000 7f58b9a00640 1 -- 172.21.15.44:0/1595801687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac870 msgr2=0x7f58b413f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.372+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1595801687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac870 0x7f58b413f470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005fd0 tx=0x7f58ac02b980 comp rx=0 tx=0).stop 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 -- 172.21.15.44:0/1595801687 shutdown_connections 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1595801687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac870 0x7f58b413f470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1595801687 >> v2:172.21.15.175:3300/0 conn(0x7f58b406f710 0x7f58b4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 -- 172.21.15.44:0/1595801687 >> 172.21.15.44:0/1595801687 conn(0x7f58b40b0fb0 msgr2=0x7f58b40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 -- 172.21.15.44:0/1595801687 shutdown_connections 2024-07-17T07:11:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.373+0000 7f58b9a00640 1 -- 172.21.15.44:0/1595801687 wait complete. 2024-07-17T07:11:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.374+0000 7f58b9a00640 1 Processor -- start 2024-07-17T07:11:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.374+0000 7f58b9a00640 1 -- start start 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b406f710 0x7f58b41f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac3e0 0x7f58b40982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40ae800 con 0x7f58b40ac870 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58b406f0a0 con 0x7f58b406f710 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.375+0000 7f58b9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40ae980 con 0x7f58b40ac3e0 2024-07-17T07:11:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac3e0 0x7f58b40982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60240/0 (socket says 172.21.15.44:60240) 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 -- 172.21.15.44:0/1772222236 learned_addr learned my addr 172.21.15.44:0/1772222236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b2a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b406f710 0x7f58b41f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac3e0 msgr2=0x7f58b40982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac3e0 0x7f58b40982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b406f710 msgr2=0x7f58b41f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b406f710 0x7f58b41f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.376+0000 7f58b3400640 1 -- 172.21.15.44:0/1772222236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40b3930 con 0x7f58b40ac870 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58b3400640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f58ac006010 tx=0x7f58ac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac035050 con 0x7f58b40ac870 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58ac004340 con 0x7f58b40ac870 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004710 con 0x7f58b40ac870 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b4098d30 con 0x7f58b40ac870 2024-07-17T07:11:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.377+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b4073a20 con 0x7f58b40ac870 2024-07-17T07:11:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.378+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4073810 con 0x7f58b40ac870 2024-07-17T07:11:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.379+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7f58ac038020 con 0x7f58b40ac870 2024-07-17T07:11:47.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.380+0000 7f58aaa00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 0x7f5894079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.381+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7f58ac0ba050 con 0x7f58b40ac870 2024-07-17T07:11:47.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.381+0000 7f58b2a00640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 msgr2=0x7f5894079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:47.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.381+0000 7f58b2a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 0x7f5894079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:47.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.384+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f58ac082cf0 con 0x7f58b40ac870 2024-07-17T07:11:47.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.582+0000 7f58b2a00640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 msgr2=0x7f5894079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:47.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.582+0000 7f58b2a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 0x7f5894079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:47.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.631+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58b413fad0 con 0x7f58b40ac870 2024-07-17T07:11:47.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.632+0000 7f58aaa00640 1 -- 172.21.15.44:0/1772222236 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f58ac0825f0 con 0x7f58b40ac870 2024-07-17T07:11:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:47.631 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":35,"btime":"2024-07-17T07:10:25:087651+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:10:21.508926+0000","modified":"2024-07-17T07:10:24.577528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6261":{"gid":6261,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2638137978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2638137978},{"type":"v1","addr":"172.21.15.44:6835","nonce":2638137978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6261,"qdb_cluster":[6261]},"id":4}]} 2024-07-17T07:11:47.632 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 35 2024-07-17T07:11:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 msgr2=0x7f5894079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 0x7f5894079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 msgr2=0x7f58b40987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f58ac006010 tx=0x7f58ac003f10 comp rx=0 tx=0).stop 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 shutdown_connections 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7f5894076d90 0x7f5894079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58b40ac870 0x7f58b40987f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58b40ac3e0 0x7f58b40982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 --2- 172.21.15.44:0/1772222236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58b406f710 0x7f58b41f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 >> 172.21.15.44:0/1772222236 conn(0x7f58b40b0fb0 msgr2=0x7f58b415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 shutdown_connections 2024-07-17T07:11:47.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.635+0000 7f58b9a00640 1 -- 172.21.15.44:0/1772222236 wait complete. 2024-07-17T07:11:47.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.846+0000 7fadbb000640 1 -- 172.21.15.44:0/1882304894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb40b3c70 msgr2=0x7fadb40b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.846+0000 7fadbb000640 1 --2- 172.21.15.44:0/1882304894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb40b3c70 0x7fadb40b4100 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fadac005fd0 tx=0x7fadac02aa00 comp rx=0 tx=0).stop 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 -- 172.21.15.44:0/1882304894 shutdown_connections 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 --2- 172.21.15.44:0/1882304894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb40b3c70 0x7fadb40b4100 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 --2- 172.21.15.44:0/1882304894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb415d410 0x7fadb406e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 -- 172.21.15.44:0/1882304894 >> 172.21.15.44:0/1882304894 conn(0x7fadb406ce80 msgr2=0x7fadb406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 -- 172.21.15.44:0/1882304894 shutdown_connections 2024-07-17T07:11:47.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.847+0000 7fadbb000640 1 -- 172.21.15.44:0/1882304894 wait complete. 2024-07-17T07:11:47.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.848+0000 7fadbb000640 1 Processor -- start 2024-07-17T07:11:47.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.848+0000 7fadbb000640 1 -- start start 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.849+0000 7fadbb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.849+0000 7fadbb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb40b3c70 0x7fadb4152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadbb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 0x7fadb415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadbb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4072dd0 con 0x7fadb406f810 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadbb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fadb4072c50 con 0x7fadb415d410 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadbb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fadb4072f50 con 0x7fadb40b3c70 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60250/0 (socket says 172.21.15.44:60250) 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 -- 172.21.15.44:0/165976181 learned_addr learned my addr 172.21.15.44:0/165976181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9200640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb40b3c70 0x7fadb4152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadba600640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 0x7fadb415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb40b3c70 msgr2=0x7fadb4152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb40b3c70 0x7fadb4152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 msgr2=0x7fadb415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 0x7fadb415d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadb9c00640 1 -- 172.21.15.44:0/165976181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb41e6d70 con 0x7fadb406f810 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.850+0000 7fadba600640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 0x7fadb415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadb9c00640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fada800d6d0 tx=0x7fada800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8017070 con 0x7fadb406f810 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada8005330 con 0x7fadb406f810 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8009820 con 0x7fadb406f810 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb41f5130 con 0x7fadb406f810 2024-07-17T07:11:47.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.851+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb41e6fe0 con 0x7fadb406f810 2024-07-17T07:11:47.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.852+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb406e3d0 con 0x7fadb406f810 2024-07-17T07:11:47.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.853+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7fada8009bd0 con 0x7fadb406f810 2024-07-17T07:11:47.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.854+0000 7fadb1600640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 0x7fad94079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:47.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.854+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7fada8098f50 con 0x7fadb406f810 2024-07-17T07:11:47.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.854+0000 7fadb9200640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 msgr2=0x7fad94079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:47.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.854+0000 7fadb9200640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 0x7fad94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:47.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:47.858+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fadb406e3d0 con 0x7fadb406f810 2024-07-17T07:11:48.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.055+0000 7fadb9200640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 msgr2=0x7fad94079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:48.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.055+0000 7fadb9200640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 0x7fad94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:11:48.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.071+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb40734b0 con 0x7fadb406f810 2024-07-17T07:11:48.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.074+0000 7fadb1600640 1 -- 172.21.15.44:0/165976181 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+18234 (secure 0 0 0) 0x7fadb41f5130 con 0x7fadb406f810 2024-07-17T07:11:48.073 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:48.073 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":86,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:11:44.053098+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:10:19.413303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-07-17T07:10:20.496271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.3d","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2788898012":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/3824917389":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6801/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6800/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/4085204748":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/1723723265":"2024-07-18T07:10:42.755496+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 msgr2=0x7fad94079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 0x7fad94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 msgr2=0x7fadb41f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fada800d6d0 tx=0x7fada800db90 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 shutdown_connections 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fad94076d60 0x7fad94079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadb415d410 0x7fadb415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadb40b3c70 0x7fadb4152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 --2- 172.21.15.44:0/165976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadb406f810 0x7fadb41f4bf0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 >> 172.21.15.44:0/165976181 conn(0x7fadb406ce80 msgr2=0x7fadb40b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 shutdown_connections 2024-07-17T07:11:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.077+0000 7fadbb000640 1 -- 172.21.15.44:0/165976181 wait complete. 2024-07-17T07:11:48.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.274+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4120687353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e4161be0 msgr2=0x7fd8e40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.274+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4120687353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e4161be0 0x7fd8e40b2200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40d0de0 tx=0x7fd8d002a4d0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4120687353 shutdown_connections 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4120687353 >> v2:172.21.15.175:3300/0 conn(0x7fd8e40ce690 0x7fd8e40cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4120687353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e4161be0 0x7fd8e40b2200 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4120687353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 0x7fd8e415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4120687353 >> 172.21.15.44:0/4120687353 conn(0x7fd8e40b4ff0 msgr2=0x7fd8e40b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4120687353 shutdown_connections 2024-07-17T07:11:48.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.275+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4120687353 wait complete. 2024-07-17T07:11:48.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.276+0000 7fd8e9e00640 1 Processor -- start 2024-07-17T07:11:48.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.276+0000 7fd8e9e00640 1 -- start start 2024-07-17T07:11:48.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 0x7fd8e415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 0x7fd8e40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e406c7a0 con 0x7fd8e40ce690 2024-07-17T07:11:48.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e406c620 con 0x7fd8e4161be0 2024-07-17T07:11:48.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.277+0000 7fd8e9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e406c920 con 0x7fd8e415d130 2024-07-17T07:11:48.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:48.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60276/0 (socket says 172.21.15.44:60276) 2024-07-17T07:11:48.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 -- 172.21.15.44:0/4260008508 learned_addr learned my addr 172.21.15.44:0/4260008508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e9400640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 0x7fd8e40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e3e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 0x7fd8e415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 -- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 msgr2=0x7fd8e415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 0x7fd8e415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 -- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 msgr2=0x7fd8e40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 0x7fd8e40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 -- 172.21.15.44:0/4260008508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e415d510 con 0x7fd8e40ce690 2024-07-17T07:11:48.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e9400640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 0x7fd8e40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.278+0000 7fd8e8a00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800d770 tx=0x7fd8d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.279+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8017070 con 0x7fd8e40ce690 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.279+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8d8009d20 con 0x7fd8e40ce690 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.279+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e415d7f0 con 0x7fd8e40ce690 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.279+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8005280 con 0x7fd8e40ce690 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.279+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e415ddb0 con 0x7fd8e40ce690 2024-07-17T07:11:48.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.280+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e40cfdf0 con 0x7fd8e40ce690 2024-07-17T07:11:48.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.281+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 36) ==== 98988+0+0 (secure 0 0 0) 0x7fd8d8005420 con 0x7fd8e40ce690 2024-07-17T07:11:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.282+0000 7fd8e0c00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 0x7fd8c4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.282+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6995+0+0 (secure 0 0 0) 0x7fd8d8098dc0 con 0x7fd8e40ce690 2024-07-17T07:11:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.282+0000 7fd8e3e00640 1 -- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 msgr2=0x7fd8c4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1065445688 2024-07-17T07:11:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.282+0000 7fd8e3e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 0x7fd8c4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:11:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.286+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd8d8061950 con 0x7fd8e40ce690 2024-07-17T07:11:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.337+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 37) ==== 98540+0+0 (secure 0 0 0) 0x7fd8d805d4e0 con 0x7fd8e40ce690 2024-07-17T07:11:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.337+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 msgr2=0x7fd8c4079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:11:48.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.337+0000 7fd8e0c00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 0x7fd8c4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.406 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa84c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa84c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa84c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa84c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa84c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa847c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa847c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa847c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.407 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa847c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:11:48.408+0000 7fa847c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:11:48.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.589+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd8e40d0e10 con 0x7fd8e40ce690 2024-07-17T07:11:48.589 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:11:48.590+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:11:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.593+0000 7fd8e0c00640 1 -- 172.21.15.44:0/4260008508 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7fd8d8010460 con 0x7fd8e40ce690 2024-07-17T07:11:48.592 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:11:48.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.597+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 msgr2=0x7fd8e41f4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.597+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800d770 tx=0x7fd8d800dc30 comp rx=0 tx=0).stop 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 shutdown_connections 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:6800/1065445688,v1:172.21.15.175:6801/1065445688] conn(0x7fd8c4076d60 0x7fd8c4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4161be0 0x7fd8e40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e415d130 0x7fd8e415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 --2- 172.21.15.44:0/4260008508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40ce690 0x7fd8e41f4690 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 >> 172.21.15.44:0/4260008508 conn(0x7fd8e40b4ff0 msgr2=0x7fd8e40b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 shutdown_connections 2024-07-17T07:11:48.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.598+0000 7fd8e9e00640 1 -- 172.21.15.44:0/4260008508 wait complete. 2024-07-17T07:11:48.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:11:48.616 INFO:tasks.ceph.mds.a.smithi044.stderr: -8930> 2024-07-17T07:10:24.001+0000 7f9a37c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:11:48.675 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:11:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 -- 172.21.15.44:0/518820621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e34154180 msgr2=0x7f9e3406f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 --2- 172.21.15.44:0/518820621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e34154180 0x7f9e3406f2d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c005fd0 tx=0x7f9e2c02a8b0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 -- 172.21.15.44:0/518820621 shutdown_connections 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 --2- 172.21.15.44:0/518820621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e3406e8a0 0x7f9e3415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 --2- 172.21.15.44:0/518820621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e34154180 0x7f9e3406f2d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 -- 172.21.15.44:0/518820621 >> 172.21.15.44:0/518820621 conn(0x7f9e340b1ce0 msgr2=0x7f9e340b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 -- 172.21.15.44:0/518820621 shutdown_connections 2024-07-17T07:11:48.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.805+0000 7f9e38a00640 1 -- 172.21.15.44:0/518820621 wait complete. 2024-07-17T07:11:48.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.807+0000 7f9e38a00640 1 Processor -- start 2024-07-17T07:11:48.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.807+0000 7f9e38a00640 1 -- start start 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340cf690 0x7f9e341f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e34154180 0x7f9e34152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3415e1c0 con 0x7f9e3406e8a0 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3415e040 con 0x7f9e34154180 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.808+0000 7f9e38a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3415e340 con 0x7f9e340cf690 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60322/0 (socket says 172.21.15.44:60322) 2024-07-17T07:11:48.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 -- 172.21.15.44:0/2058363703 learned_addr learned my addr 172.21.15.44:0/2058363703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 -- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340cf690 msgr2=0x7f9e341f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340cf690 0x7f9e341f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 -- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e34154180 msgr2=0x7f9e34152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e34154180 0x7f9e34152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 -- 172.21.15.44:0/2058363703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e341f4770 con 0x7f9e3406e8a0 2024-07-17T07:11:48.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.809+0000 7f9e32800640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c000f80 tx=0x7f9e2c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.810+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c031990 con 0x7f9e3406e8a0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.810+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e2c031c20 con 0x7f9e3406e8a0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.810+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c033330 con 0x7f9e3406e8a0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.810+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e341f4a50 con 0x7f9e3406e8a0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.810+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e34071ec0 con 0x7f9e3406e8a0 2024-07-17T07:11:48.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.811+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3415e790 con 0x7f9e3406e8a0 2024-07-17T07:11:48.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.812+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 37) ==== 98540+0+0 (secure 0 0 0) 0x7f9e2c0334d0 con 0x7f9e3406e8a0 2024-07-17T07:11:48.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.813+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7468+0+0 (secure 0 0 0) 0x7f9e2c0b98a0 con 0x7f9e3406e8a0 2024-07-17T07:11:48.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:48.818+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9e2c03f030 con 0x7f9e3406e8a0 2024-07-17T07:11:49.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.075+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9e34071c90 con 0x7f9e3406e8a0 2024-07-17T07:11:49.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.350+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 38) ==== 98667+0+0 (secure 0 0 0) 0x7f9e2c07dc60 con 0x7f9e3406e8a0 2024-07-17T07:11:49.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.351+0000 7f9e2aa00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 0x7f9e1007f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:49.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.352+0000 7f9e31e00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 0x7f9e1007f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:49.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.356+0000 7f9e31e00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 0x7f9e1007f5e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c007fd0 tx=0x7f9e1c007900 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:49.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.597+0000 7f9e2aa00640 1 -- 172.21.15.44:0/2058363703 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f9e2c0829f0 con 0x7f9e3406e8a0 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 msgr2=0x7f9e1007f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 0x7f9e1007f5e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c007fd0 tx=0x7f9e1c007900 comp rx=0 tx=0).stop 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 msgr2=0x7f9e34073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c000f80 tx=0x7f9e2c003970 comp rx=0 tx=0).stop 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 shutdown_connections 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f9e1007d1f0 0x7f9e1007f5e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e34154180 0x7f9e34152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e340cf690 0x7f9e341f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 --2- 172.21.15.44:0/2058363703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e3406e8a0 0x7f9e34073a50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 >> 172.21.15.44:0/2058363703 conn(0x7f9e340b1ce0 msgr2=0x7f9e340b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 shutdown_connections 2024-07-17T07:11:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.601+0000 7f9e38a00640 1 -- 172.21.15.44:0/2058363703 wait complete. 2024-07-17T07:11:49.621 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:11:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 -- 172.21.15.44:0/1320895158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410074020 msgr2=0x7ff4100ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 --2- 172.21.15.44:0/1320895158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410074020 0x7ff4100ac0f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff400009830 tx=0x7ff40002a510 comp rx=0 tx=0).stop 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 -- 172.21.15.44:0/1320895158 shutdown_connections 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 --2- 172.21.15.44:0/1320895158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410074020 0x7ff4100ac0f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 -- 172.21.15.44:0/1320895158 >> 172.21.15.44:0/1320895158 conn(0x7ff4100cf050 msgr2=0x7ff4100cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 -- 172.21.15.44:0/1320895158 shutdown_connections 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.840+0000 7ff416200640 1 -- 172.21.15.44:0/1320895158 wait complete. 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.842+0000 7ff416200640 1 Processor -- start 2024-07-17T07:11:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.842+0000 7ff416200640 1 -- start start 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 0x7ff4101f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41015d930 0x7ff410159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100ae350 con 0x7ff410154570 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff410074610 con 0x7ff41015d930 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff416200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100ae4d0 con 0x7ff410074020 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55008/0 (socket says 172.21.15.44:55008) 2024-07-17T07:11:49.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff407e00640 1 -- 172.21.15.44:0/2238037343 learned_addr learned my addr 172.21.15.44:0/2238037343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff414e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 0x7ff4101f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.843+0000 7ff415800640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41015d930 0x7ff410159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 -- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 msgr2=0x7ff4101f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 0x7ff4101f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 -- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41015d930 msgr2=0x7ff410159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41015d930 0x7ff410159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 -- 172.21.15.44:0/2238037343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4101f85f0 con 0x7ff410154570 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff414e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 0x7ff4101f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff407e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c390 tx=0x7ff3f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:49.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8017070 con 0x7ff410154570 2024-07-17T07:11:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3f80035f0 con 0x7ff410154570 2024-07-17T07:11:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff410153c50 con 0x7ff410154570 2024-07-17T07:11:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003a20 con 0x7ff410154570 2024-07-17T07:11:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.844+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff410142ae0 con 0x7ff410154570 2024-07-17T07:11:49.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.845+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4100ac0f0 con 0x7ff410154570 2024-07-17T07:11:49.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.847+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 38) ==== 98667+0+0 (secure 0 0 0) 0x7ff3f801a070 con 0x7ff410154570 2024-07-17T07:11:49.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.848+0000 7ff40d600640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 0x7ff3ec078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:49.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.848+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7468+0+0 (secure 0 0 0) 0x7ff3f8097a60 con 0x7ff410154570 2024-07-17T07:11:49.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.849+0000 7ff414e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 0x7ff3ec078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:49.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.850+0000 7ff414e00640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 0x7ff3ec078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff400034000 tx=0x7ff400004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:49.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:49.853+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4101f85f0 con 0x7ff410154570 2024-07-17T07:11:50.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.058+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff41006cd20 con 0x7ff410154570 2024-07-17T07:11:50.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.613+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 39) ==== 98875+0+0 (secure 0 0 0) 0x7ff3f805c650 con 0x7ff410154570 2024-07-17T07:11:50.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.623+0000 7ff40d600640 1 -- 172.21.15.44:0/2238037343 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7ff3f801b3b0 con 0x7ff410154570 2024-07-17T07:11:50.622 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:11:50.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.626+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 msgr2=0x7ff3ec078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:50.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.626+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 0x7ff3ec078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff400034000 tx=0x7ff400004730 comp rx=0 tx=0).stop 2024-07-17T07:11:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.626+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 msgr2=0x7ff4101e83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.626+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c390 tx=0x7ff3f800c850 comp rx=0 tx=0).stop 2024-07-17T07:11:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 shutdown_connections 2024-07-17T07:11:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3ec076a00 0x7ff3ec078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff41015d930 0x7ff410159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff410154570 0x7ff4101e83f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 --2- 172.21.15.44:0/2238037343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff410074020 0x7ff4101f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.627+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 >> 172.21.15.44:0/2238037343 conn(0x7ff4100cf050 msgr2=0x7ff41015f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.628+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 shutdown_connections 2024-07-17T07:11:50.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.628+0000 7ff416200640 1 -- 172.21.15.44:0/2238037343 wait complete. 2024-07-17T07:11:50.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:11:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 -- 172.21.15.44:0/2543396451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910003aa0 msgr2=0x7f2910003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 --2- 172.21.15.44:0/2543396451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910003aa0 0x7f2910003ec0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f29200016c0 tx=0x7f2910005830 comp rx=0 tx=0).stop 2024-07-17T07:11:50.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 -- 172.21.15.44:0/2543396451 shutdown_connections 2024-07-17T07:11:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 --2- 172.21.15.44:0/2543396451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2910003aa0 0x7f2910003ec0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 -- 172.21.15.44:0/2543396451 >> 172.21.15.44:0/2543396451 conn(0x7f2930073a60 msgr2=0x7f2930073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 -- 172.21.15.44:0/2543396451 shutdown_connections 2024-07-17T07:11:50.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.866+0000 7f2934a00640 1 -- 172.21.15.44:0/2543396451 wait complete. 2024-07-17T07:11:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.868+0000 7f2934a00640 1 Processor -- start 2024-07-17T07:11:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.868+0000 7f2934a00640 1 -- start start 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2910003aa0 0x7f29301e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2930158a20 0x7f2930159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29300ce680 con 0x7f29301597a0 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29300ce500 con 0x7f2910003aa0 2024-07-17T07:11:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f2934a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29300ce800 con 0x7f2930158a20 2024-07-17T07:11:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f292f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f292f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55030/0 (socket says 172.21.15.44:55030) 2024-07-17T07:11:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.869+0000 7f292f200640 1 -- 172.21.15.44:0/966322054 learned_addr learned my addr 172.21.15.44:0/966322054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292de00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2930158a20 0x7f2930159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 -- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2930158a20 msgr2=0x7f2930159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2930158a20 0x7f2930159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 -- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2910003aa0 msgr2=0x7f29301e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2910003aa0 0x7f29301e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 -- 172.21.15.44:0/966322054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29300b3f20 con 0x7f29301597a0 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f292f200640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f293006f1b0 tx=0x7f2928011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928005020 con 0x7f29301597a0 2024-07-17T07:11:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2928016070 con 0x7f29301597a0 2024-07-17T07:11:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928005530 con 0x7f29301597a0 2024-07-17T07:11:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.870+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29300b44c0 con 0x7f29301597a0 2024-07-17T07:11:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.871+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29300b8890 con 0x7f29301597a0 2024-07-17T07:11:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.872+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29300d05a0 con 0x7f29301597a0 2024-07-17T07:11:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.873+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 98875+0+0 (secure 0 0 0) 0x7f29280047a0 con 0x7f29301597a0 2024-07-17T07:11:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.875+0000 7f2926a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 0x7f290c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.875+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7001+0+0 (secure 0 0 0) 0x7f2928098f80 con 0x7f29301597a0 2024-07-17T07:11:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.876+0000 7f292e800640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 0x7f290c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.877+0000 7f292e800640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 0x7f290c079130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2920032000 tx=0x7f2920001120 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:50.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:50.879+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2928061ee0 con 0x7f29301597a0 2024-07-17T07:11:51.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.087+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2930069460 con 0x7f29301597a0 2024-07-17T07:11:51.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.626+0000 7f2926a00640 1 -- 172.21.15.44:0/966322054 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f292806a120 con 0x7f29301597a0 2024-07-17T07:11:51.626 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:11:51.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.630+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 msgr2=0x7f290c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:51.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.630+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 0x7f290c079130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2920032000 tx=0x7f2920001120 comp rx=0 tx=0).stop 2024-07-17T07:11:51.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.630+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 msgr2=0x7f29300cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:51.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.630+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f293006f1b0 tx=0x7f2928011af0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 shutdown_connections 2024-07-17T07:11:51.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f290c076c70 0x7f290c079130 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29301597a0 0x7f29300cc760 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2930158a20 0x7f2930159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 --2- 172.21.15.44:0/966322054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2910003aa0 0x7f29301e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 >> 172.21.15.44:0/966322054 conn(0x7f2930073a60 msgr2=0x7f2930073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:51.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.631+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 shutdown_connections 2024-07-17T07:11:51.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.632+0000 7f2934a00640 1 -- 172.21.15.44:0/966322054 wait complete. 2024-07-17T07:11:51.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:11:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 -- 172.21.15.44:0/3273997058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 msgr2=0x7f6ce40ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 --2- 172.21.15.44:0/3273997058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce40ac830 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc005fd0 tx=0x7f6cdc02a8a0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 -- 172.21.15.44:0/3273997058 shutdown_connections 2024-07-17T07:11:51.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 --2- 172.21.15.44:0/3273997058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce40ac830 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 --2- 172.21.15.44:0/3273997058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce406eb90 0x7f6ce415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 -- 172.21.15.44:0/3273997058 >> 172.21.15.44:0/3273997058 conn(0x7f6ce40b0eb0 msgr2=0x7f6ce40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 -- 172.21.15.44:0/3273997058 shutdown_connections 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.863+0000 7f6cec200640 1 -- 172.21.15.44:0/3273997058 wait complete. 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.865+0000 7f6cec200640 1 Processor -- start 2024-07-17T07:11:51.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.865+0000 7f6cec200640 1 -- start start 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce406eb90 0x7f6ce41f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b3280 0x7f6ce40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce406f850 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce406f6d0 con 0x7f6ce406eb90 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6cec200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce41505f0 con 0x7f6ce40b3280 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6ce8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6ce8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55052/0 (socket says 172.21.15.44:55052) 2024-07-17T07:11:51.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.866+0000 7f6ce8e00640 1 -- 172.21.15.44:0/2833878835 learned_addr learned my addr 172.21.15.44:0/2833878835 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6cea200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b3280 0x7f6ce40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce9800640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce406eb90 0x7f6ce41f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 -- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b3280 msgr2=0x7f6ce40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b3280 0x7f6ce40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 -- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce406eb90 msgr2=0x7f6ce41f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce406eb90 0x7f6ce41f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 -- 172.21.15.44:0/2833878835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce41f4c20 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce8e00640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c3e0 tx=0x7f6cd000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0017070 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cd00035f0 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0003a60 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.867+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce4072cd0 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.868+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce4073290 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.869+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce415a5a0 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.870+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 98875+0+0 (secure 0 0 0) 0x7f6cd001a070 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.871+0000 7f6ce1600640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 0x7f6cc4079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.871+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f6cd00979d0 con 0x7f6ce40ac3a0 2024-07-17T07:11:51.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.872+0000 7f6ce9800640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 0x7f6cc4079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:51.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.873+0000 7f6ce9800640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 0x7f6cc4079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40058c0 tx=0x7f6cd4005c80 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:51.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:51.875+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6ce41f4c20 con 0x7f6ce40ac3a0 2024-07-17T07:11:52.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.088+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f6ce40af670 con 0x7f6ce40ac3a0 2024-07-17T07:11:52.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.096+0000 7f6ce1600640 1 -- 172.21.15.44:0/2833878835 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f6cd0060b20 con 0x7f6ce40ac3a0 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.099+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 msgr2=0x7f6cc4079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.099+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 0x7f6cc4079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40058c0 tx=0x7f6cd4005c80 comp rx=0 tx=0).stop 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.099+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 msgr2=0x7f6ce4158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.099+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c3e0 tx=0x7f6cd000c8a0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 shutdown_connections 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6cc4076c70 0x7f6cc4079130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b3280 0x7f6ce40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ce40ac3a0 0x7f6ce4158fe0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 --2- 172.21.15.44:0/2833878835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce406eb90 0x7f6ce41f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 >> 172.21.15.44:0/2833878835 conn(0x7f6ce40b0eb0 msgr2=0x7f6ce415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 shutdown_connections 2024-07-17T07:11:52.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.100+0000 7f6cec200640 1 -- 172.21.15.44:0/2833878835 wait complete. 2024-07-17T07:11:52.119 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 -- 172.21.15.44:0/1209272794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c14fb80 msgr2=0x7f7a4c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 --2- 172.21.15.44:0/1209272794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c14fb80 0x7f7a4c14ffd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005fd0 tx=0x7f7a4402bbc0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 -- 172.21.15.44:0/1209272794 shutdown_connections 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 --2- 172.21.15.44:0/1209272794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c14fb80 0x7f7a4c14ffd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 -- 172.21.15.44:0/1209272794 >> 172.21.15.44:0/1209272794 conn(0x7f7a4c06c120 msgr2=0x7f7a4c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:52.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 -- 172.21.15.44:0/1209272794 shutdown_connections 2024-07-17T07:11:52.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.328+0000 7f7a4b200640 1 -- 172.21.15.44:0/1209272794 wait complete. 2024-07-17T07:11:52.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.330+0000 7f7a4b200640 1 Processor -- start 2024-07-17T07:11:52.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.331+0000 7f7a4b200640 1 -- start start 2024-07-17T07:11:52.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a4c14fb80 0x7f7a4c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a4c15d2f0 0x7f7a4c1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0ad460 con 0x7f7a4c06f580 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c150720 con 0x7f7a4c15d2f0 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a4b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c0ad5e0 con 0x7f7a4c14fb80 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55068/0 (socket says 172.21.15.44:55068) 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.332+0000 7f7a49e00640 1 -- 172.21.15.44:0/4170778468 learned_addr learned my addr 172.21.15.44:0/4170778468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 -- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a4c14fb80 msgr2=0x7f7a4c148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a4c14fb80 0x7f7a4c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a4a800640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a4c15d2f0 0x7f7a4c1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:52.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 -- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a4c15d2f0 msgr2=0x7f7a4c1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:52.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a4c15d2f0 0x7f7a4c1f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:52.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 -- 172.21.15.44:0/4170778468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c149880 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a49e00640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00c460 tx=0x7f7a3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.333+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c016020 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.334+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a3c012960 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.334+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c012db0 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.334+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c1f7b60 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.334+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c14a1a0 con 0x7f7a4c06f580 2024-07-17T07:11:52.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.335+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c14ffd0 con 0x7f7a4c06f580 2024-07-17T07:11:52.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.336+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 39) ==== 98875+0+0 (secure 0 0 0) 0x7f7a3c019020 con 0x7f7a4c06f580 2024-07-17T07:11:52.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.337+0000 7f7a42000640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 0x7f7a2c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:52.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.337+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f7a3c0984e0 con 0x7f7a4c06f580 2024-07-17T07:11:52.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.338+0000 7f7a49400640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 0x7f7a2c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:52.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.339+0000 7f7a49400640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 0x7f7a2c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007fd0 tx=0x7f7a38007900 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:52.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.342+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7a3c020030 con 0x7f7a4c06f580 2024-07-17T07:11:52.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:52.648+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) -- 0x7f7a4c15dac0 con 0x7f7a4c06f580 2024-07-17T07:11:53.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.117+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f7a3c05d130 con 0x7f7a4c06f580 2024-07-17T07:11:53.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.632+0000 7f7a42000640 1 -- 172.21.15.44:0/4170778468 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v628) ==== 156+0+0 (secure 0 0 0) 0x7f7a3c06b930 con 0x7f7a4c06f580 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.637+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 msgr2=0x7f7a2c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.637+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 0x7f7a2c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007fd0 tx=0x7f7a38007900 comp rx=0 tx=0).stop 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.637+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 msgr2=0x7f7a4c14ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.637+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00c460 tx=0x7f7a3c00c920 comp rx=0 tx=0).stop 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 shutdown_connections 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7a2c076c40 0x7f7a2c079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a4c15d2f0 0x7f7a4c1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a4c14fb80 0x7f7a4c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 --2- 172.21.15.44:0/4170778468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a4c06f580 0x7f7a4c14ac80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 >> 172.21.15.44:0/4170778468 conn(0x7f7a4c06c120 msgr2=0x7f7a4c15bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 shutdown_connections 2024-07-17T07:11:53.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.638+0000 7f7a4b200640 1 -- 172.21.15.44:0/4170778468 wait complete. 2024-07-17T07:11:53.658 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2024-07-17T07:11:53.659 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2024-07-17T07:11:53.659 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:11:53.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:11:53.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 -- 172.21.15.44:0/2009363128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec0cfc70 msgr2=0x7f00ec06fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 --2- 172.21.15.44:0/2009363128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec0cfc70 0x7f00ec06fde0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009830 tx=0x7f00d402a490 comp rx=0 tx=0).stop 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 -- 172.21.15.44:0/2009363128 shutdown_connections 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 --2- 172.21.15.44:0/2009363128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec070320 0x7f00ec0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 --2- 172.21.15.44:0/2009363128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec0cfc70 0x7f00ec06fde0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 -- 172.21.15.44:0/2009363128 >> 172.21.15.44:0/2009363128 conn(0x7f00ec0b3020 msgr2=0x7f00ec0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.865+0000 7f00f2e00640 1 -- 172.21.15.44:0/2009363128 shutdown_connections 2024-07-17T07:11:53.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.866+0000 7f00f2e00640 1 -- 172.21.15.44:0/2009363128 wait complete. 2024-07-17T07:11:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.867+0000 7f00f2e00640 1 Processor -- start 2024-07-17T07:11:53.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.867+0000 7f00f2e00640 1 -- start start 2024-07-17T07:11:53.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.868+0000 7f00f2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.868+0000 7f00f2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 0x7f00ec15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.868+0000 7f00f2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 0x7f00ec15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00f2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec0aebd0 con 0x7f00ec070320 2024-07-17T07:11:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00f2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec0aea50 con 0x7f00ec0cfc70 2024-07-17T07:11:53.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00f2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec0aed50 con 0x7f00ec15b560 2024-07-17T07:11:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55080/0 (socket says 172.21.15.44:55080) 2024-07-17T07:11:53.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 -- 172.21.15.44:0/1014565397 learned_addr learned my addr 172.21.15.44:0/1014565397 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00eb400640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 0x7f00ec15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00f0e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 0x7f00ec15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 -- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 msgr2=0x7f00ec15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 0x7f00ec15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 -- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 msgr2=0x7f00ec15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 0x7f00ec15bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00ebe00640 1 -- 172.21.15.44:0/1014565397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec15e5c0 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00eb400640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 0x7f00ec15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.869+0000 7f00f0e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 0x7f00ec15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00ebe00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00cbf0 tx=0x7f00dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc016020 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00dc00ad30 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc012680 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec15e8a0 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.870+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1f7880 con 0x7f00ec070320 2024-07-17T07:11:53.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.871+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec0af270 con 0x7f00ec070320 2024-07-17T07:11:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.872+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f00dc019020 con 0x7f00ec070320 2024-07-17T07:11:53.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.873+0000 7f00e3e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 0x7f00c8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.874+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f00dc098ec0 con 0x7f00ec070320 2024-07-17T07:11:53.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.874+0000 7f00eb400640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 0x7f00c8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:53.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.875+0000 7f00eb400640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 0x7f00c8079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00d4034000 tx=0x7f00d4004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:53.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:53.878+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f00dc020020 con 0x7f00ec070320 2024-07-17T07:11:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.089+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00ec0d1200 con 0x7f00ec070320 2024-07-17T07:11:54.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.091+0000 7f00e3e00640 1 -- 172.21.15.44:0/1014565397 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+15023 (secure 0 0 0) 0x7f00dc061ec0 con 0x7f00ec070320 2024-07-17T07:11:54.090 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:54.091 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:11:51.623278+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/2638137978":"2024-07-18T07:11:48.590589+0000","172.21.15.175:0/2876904945":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/1716388817":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/3530090701":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/3593986976":"2024-07-18T07:11:48.323636+0000","172.21.15.175:6801/1065445688":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/111607085":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/934943506":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/2788898012":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6800/1065445688":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/3824917389":"2024-07-18T07:10:42.755496+0000","172.21.15.175:6801/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.44:6835/2638137978":"2024-07-18T07:11:48.590589+0000","172.21.15.175:6800/2949346159":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/2089782025":"2024-07-18T07:11:48.323636+0000","172.21.15.175:0/4085204748":"2024-07-18T07:10:42.755496+0000","172.21.15.175:0/1723723265":"2024-07-18T07:10:42.755496+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 msgr2=0x7f00c8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 0x7f00c8079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00d4034000 tx=0x7f00d4004730 comp rx=0 tx=0).stop 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 msgr2=0x7f00ec1f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00cbf0 tx=0x7f00dc00a060 comp rx=0 tx=0).stop 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 shutdown_connections 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f00c8076cd0 0x7f00c8079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00ec15b560 0x7f00ec15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00ec0cfc70 0x7f00ec15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 --2- 172.21.15.44:0/1014565397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00ec070320 0x7f00ec1f83a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 >> 172.21.15.44:0/1014565397 conn(0x7f00ec0b3020 msgr2=0x7f00ec158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 shutdown_connections 2024-07-17T07:11:54.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.094+0000 7f00f2e00640 1 -- 172.21.15.44:0/1014565397 wait complete. 2024-07-17T07:11:54.112 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:11:54.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:11:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f307e400640 1 -- 172.21.15.44:0/2411611883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c003a00 msgr2=0x7f305c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f307e400640 1 --2- 172.21.15.44:0/2411611883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f305c003a00 0x7f305c003e20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f306800c260 tx=0x7f305c005830 comp rx=0 tx=0).stop 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f3077e00640 1 -- 172.21.15.44:0/2411611883 reap_dead start 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f307e400640 1 -- 172.21.15.44:0/2411611883 shutdown_connections 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f307e400640 1 -- 172.21.15.44:0/2411611883 >> 172.21.15.44:0/2411611883 conn(0x7f30780b0eb0 msgr2=0x7f30780b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.334+0000 7f307e400640 1 -- 172.21.15.44:0/2411611883 shutdown_connections 2024-07-17T07:11:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.335+0000 7f307e400640 1 -- 172.21.15.44:0/2411611883 wait complete. 2024-07-17T07:11:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.336+0000 7f307e400640 1 Processor -- start 2024-07-17T07:11:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.337+0000 7f307e400640 1 -- start start 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f305c003a00 0x7f30781f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30780c8530 0x7f3078152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30780ac730 con 0x7f30781f58b0 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30780ac5b0 con 0x7f305c003a00 2024-07-17T07:11:54.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f307e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30780ac8b0 con 0x7f30780c8530 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f3077e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f3077e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55096/0 (socket says 172.21.15.44:55096) 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f3077e00640 1 -- 172.21.15.44:0/756719842 learned_addr learned my addr 172.21.15.44:0/756719842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f3077400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f305c003a00 0x7f30781f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.338+0000 7f3076a00640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30780c8530 0x7f3078152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 -- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30780c8530 msgr2=0x7f3078152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30780c8530 0x7f3078152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 -- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f305c003a00 msgr2=0x7f30781f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f305c003a00 0x7f30781f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 -- 172.21.15.44:0/756719842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780cb120 con 0x7f30781f58b0 2024-07-17T07:11:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f3077e00640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f305c004780 tx=0x7f307000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070004b20 con 0x7f30781f58b0 2024-07-17T07:11:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3070004db0 con 0x7f30781f58b0 2024-07-17T07:11:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30780c89f0 con 0x7f30781f58b0 2024-07-17T07:11:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000da90 con 0x7f30781f58b0 2024-07-17T07:11:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.339+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30780c8fb0 con 0x7f30781f58b0 2024-07-17T07:11:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.341+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30780b3280 con 0x7f30781f58b0 2024-07-17T07:11:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.342+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f307001c020 con 0x7f30781f58b0 2024-07-17T07:11:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.343+0000 7f306f400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 0x7f30580791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.343+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f30700983e0 con 0x7f30781f58b0 2024-07-17T07:11:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.344+0000 7f3077400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 0x7f30580791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.345+0000 7f3077400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 0x7f30580791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3068033000 tx=0x7f306802d000 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:54.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.348+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f30780cb120 con 0x7f30781f58b0 2024-07-17T07:11:54.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.560+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30780b4200 con 0x7f30781f58b0 2024-07-17T07:11:54.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.561+0000 7f306f400640 1 -- 172.21.15.44:0/756719842 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f3070061370 con 0x7f30781f58b0 2024-07-17T07:11:54.560 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 msgr2=0x7f30580791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 0x7f30580791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3068033000 tx=0x7f306802d000 comp rx=0 tx=0).stop 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 msgr2=0x7f30781ee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f305c004780 tx=0x7f307000fb60 comp rx=0 tx=0).stop 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 shutdown_connections 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3058076d00 0x7f30580791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30781f58b0 0x7f30781ee210 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30780c8530 0x7f3078152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 --2- 172.21.15.44:0/756719842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f305c003a00 0x7f30781f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 >> 172.21.15.44:0/756719842 conn(0x7f30780b0eb0 msgr2=0x7f30780b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:54.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 shutdown_connections 2024-07-17T07:11:54.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.564+0000 7f307e400640 1 -- 172.21.15.44:0/756719842 wait complete. 2024-07-17T07:11:54.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2024-07-17T07:11:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 -- 172.21.15.44:0/293814553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c80003aa0 msgr2=0x7f3c80003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 --2- 172.21.15.44:0/293814553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c80003aa0 0x7f3c80003ec0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c001eb0 tx=0x7f3c80005830 comp rx=0 tx=0).stop 2024-07-17T07:11:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 -- 172.21.15.44:0/293814553 shutdown_connections 2024-07-17T07:11:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 --2- 172.21.15.44:0/293814553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c80003aa0 0x7f3c80003ec0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 -- 172.21.15.44:0/293814553 >> 172.21.15.44:0/293814553 conn(0x7f3c9c0b0fb0 msgr2=0x7f3c9c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 -- 172.21.15.44:0/293814553 shutdown_connections 2024-07-17T07:11:54.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.800+0000 7f3ca3200640 1 -- 172.21.15.44:0/293814553 wait complete. 2024-07-17T07:11:54.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.802+0000 7f3ca3200640 1 Processor -- start 2024-07-17T07:11:54.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.802+0000 7f3ca3200640 1 -- start start 2024-07-17T07:11:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80003aa0 0x7f3c9c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c9c1e69d0 0x7f3c9c1e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c06f830 con 0x7f3c9c151c50 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c06f6b0 con 0x7f3c80003aa0 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3ca3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0b38a0 con 0x7f3c9c1e69d0 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55104/0 (socket says 172.21.15.44:55104) 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3c9b400640 1 -- 172.21.15.44:0/873338994 learned_addr learned my addr 172.21.15.44:0/873338994 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.804+0000 7f3c9be00640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80003aa0 0x7f3c9c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3ca1200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c9c1e69d0 0x7f3c9c1e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 -- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c9c1e69d0 msgr2=0x7f3c9c1e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c9c1e69d0 0x7f3c9c1e6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 -- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80003aa0 msgr2=0x7f3c9c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80003aa0 0x7f3c9c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 -- 172.21.15.44:0/873338994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1f5200 con 0x7f3c9c151c50 2024-07-17T07:11:54.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c9b400640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ea90 tx=0x7f3c8800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:54.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8801c070 con 0x7f3c9c151c50 2024-07-17T07:11:54.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c8800ddf0 con 0x7f3c9c151c50 2024-07-17T07:11:54.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88017670 con 0x7f3c9c151c50 2024-07-17T07:11:54.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.805+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1e5c10 con 0x7f3c9c151c50 2024-07-17T07:11:54.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.806+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1e61d0 con 0x7f3c9c151c50 2024-07-17T07:11:54.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.807+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9c0b2e80 con 0x7f3c9c151c50 2024-07-17T07:11:54.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.808+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f3c8801f070 con 0x7f3c9c151c50 2024-07-17T07:11:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.809+0000 7f3c93e00640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 0x7f3c7c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.809+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f3c88098c60 con 0x7f3c9c151c50 2024-07-17T07:11:54.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.810+0000 7f3c9be00640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 0x7f3c7c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:54.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.811+0000 7f3c9be00640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 0x7f3c7c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c033000 tx=0x7f3c9c0ad2a0 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:54.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:54.814+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3c88061500 con 0x7f3c9c151c50 2024-07-17T07:11:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.102+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) -- 0x7f3c9c0b27a0 con 0x7f3c9c151c50 2024-07-17T07:11:55.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.644+0000 7f3c93e00640 1 -- 172.21.15.44:0/873338994 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v630) ==== 169+0+0 (secure 0 0 0) 0x7f3c88065a50 con 0x7f3c9c151c50 2024-07-17T07:11:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 msgr2=0x7f3c7c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 0x7f3c7c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c033000 tx=0x7f3c9c0ad2a0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 msgr2=0x7f3c9c071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ea90 tx=0x7f3c8800ef50 comp rx=0 tx=0).stop 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 shutdown_connections 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f3c7c076cd0 0x7f3c7c079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c9c1e69d0 0x7f3c9c1e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c9c151c50 0x7f3c9c071620 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 --2- 172.21.15.44:0/873338994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80003aa0 0x7f3c9c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 >> 172.21.15.44:0/873338994 conn(0x7f3c9c0b0fb0 msgr2=0x7f3c9c15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.649+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 shutdown_connections 2024-07-17T07:11:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.650+0000 7f3ca3200640 1 -- 172.21.15.44:0/873338994 wait complete. 2024-07-17T07:11:55.671 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:11:55.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 -- 172.21.15.44:0/3519675398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b31d0 msgr2=0x7f661c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 --2- 172.21.15.44:0/3519675398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b31d0 0x7f661c071a30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f661000b330 tx=0x7f661002abf0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 -- 172.21.15.44:0/3519675398 shutdown_connections 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 --2- 172.21.15.44:0/3519675398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b31d0 0x7f661c071a30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 -- 172.21.15.44:0/3519675398 >> 172.21.15.44:0/3519675398 conn(0x7f661c06bf40 msgr2=0x7f661c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.881+0000 7f6624400640 1 -- 172.21.15.44:0/3519675398 shutdown_connections 2024-07-17T07:11:55.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.882+0000 7f6624400640 1 -- 172.21.15.44:0/3519675398 wait complete. 2024-07-17T07:11:55.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.883+0000 7f6624400640 1 Processor -- start 2024-07-17T07:11:55.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.883+0000 7f6624400640 1 -- start start 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661c0b31d0 0x7f661c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f661c15d710 0x7f661c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f661c06f8d0 con 0x7f661c0b35b0 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f661c06f750 con 0x7f661c15d710 2024-07-17T07:11:55.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.884+0000 7f6624400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f661c0ceb70 con 0x7f661c0b31d0 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55124/0 (socket says 172.21.15.44:55124) 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 -- 172.21.15.44:0/1227648946 learned_addr learned my addr 172.21.15.44:0/1227648946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621a00640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661c0b31d0 0x7f661c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6622400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f661c15d710 0x7f661c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:55.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 -- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661c0b31d0 msgr2=0x7f661c14cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661c0b31d0 0x7f661c14cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 -- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f661c15d710 msgr2=0x7f661c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f661c15d710 0x7f661c0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.885+0000 7f6621000640 1 -- 172.21.15.44:0/1227648946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c0a9d50 con 0x7f661c0b35b0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6621000640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f660800c460 tx=0x7f660800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608016020 con 0x7f661c0b35b0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6608012960 con 0x7f661c0b35b0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608012db0 con 0x7f661c0b35b0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c0aa030 con 0x7f661c0b35b0 2024-07-17T07:11:55.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.886+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c0aa5f0 con 0x7f661c0b35b0 2024-07-17T07:11:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.887+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c0cefc0 con 0x7f661c0b35b0 2024-07-17T07:11:55.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.889+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f6608019020 con 0x7f661c0b35b0 2024-07-17T07:11:55.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.890+0000 7f6619600640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 0x7f65fc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:55.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.890+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f66080988d0 con 0x7f661c0b35b0 2024-07-17T07:11:55.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.891+0000 7f6621a00640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 0x7f65fc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:55.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.892+0000 7f6621a00640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 0x7f65fc0791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6610035000 tx=0x7f6610004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:55.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:55.895+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6608020020 con 0x7f661c0b35b0 2024-07-17T07:11:56.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.112+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f661c15a620 con 0x7f661c0b35b0 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.113+0000 7f6619600640 1 -- 172.21.15.44:0/1227648946 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+1998 (secure 0 0 0) 0x7f66080618d0 con 0x7f661c0b35b0 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:11:56.113 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:11:56.114 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:11:56.115 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:11:56.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 msgr2=0x7f65fc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 0x7f65fc0791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6610035000 tx=0x7f6610004730 comp rx=0 tx=0).stop 2024-07-17T07:11:56.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 msgr2=0x7f661c0a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f660800c460 tx=0x7f660800c920 comp rx=0 tx=0).stop 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 shutdown_connections 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f65fc076d00 0x7f65fc0791c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f661c15d710 0x7f661c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f661c0b35b0 0x7f661c0a9490 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 --2- 172.21.15.44:0/1227648946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f661c0b31d0 0x7f661c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 >> 172.21.15.44:0/1227648946 conn(0x7f661c06bf40 msgr2=0x7f661c0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 shutdown_connections 2024-07-17T07:11:56.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.116+0000 7f6624400640 1 -- 172.21.15.44:0/1227648946 wait complete. 2024-07-17T07:11:56.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.353+0000 7f0ba2c00640 1 -- 172.21.15.44:0/3694351823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 msgr2=0x7f0b9c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.353+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/3694351823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c0b3110 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0b94005fd0 tx=0x7f0b9402a820 comp rx=0 tx=0).stop 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.353+0000 7f0ba2c00640 1 -- 172.21.15.44:0/3694351823 shutdown_connections 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.353+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/3694351823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c0b3110 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.354+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/3694351823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 0x7f0b9c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.354+0000 7f0ba2c00640 1 -- 172.21.15.44:0/3694351823 >> 172.21.15.44:0/3694351823 conn(0x7f0b9c06bf40 msgr2=0x7f0b9c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.354+0000 7f0ba2c00640 1 -- 172.21.15.44:0/3694351823 shutdown_connections 2024-07-17T07:11:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.354+0000 7f0ba2c00640 1 -- 172.21.15.44:0/3694351823 wait complete. 2024-07-17T07:11:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.355+0000 7f0ba2c00640 1 Processor -- start 2024-07-17T07:11:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.356+0000 7f0ba2c00640 1 -- start start 2024-07-17T07:11:56.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.356+0000 7f0ba2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 0x7f0b9c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:56.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 0x7f0b9c1f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c0ada20 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c0ad8a0 con 0x7f0b9c0b2cd0 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9c0adba0 con 0x7f0b9c1544f0 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55138/0 (socket says 172.21.15.44:55138) 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 -- 172.21.15.44:0/923500031 learned_addr learned my addr 172.21.15.44:0/923500031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0b93e00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 0x7f0b9c1f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:56.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0b9be00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 0x7f0b9c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 -- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 msgr2=0x7f0b9c1f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 0x7f0b9c1f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 -- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 msgr2=0x7f0b9c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 0x7f0b9c159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0ba0c00640 1 -- 172.21.15.44:0/923500031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c06df90 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.357+0000 7f0b9be00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 0x7f0b9c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:56.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0ba0c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b94035040 tx=0x7f0b94032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94037040 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b94042070 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94003e60 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c06e270 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.358+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c159c80 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.360+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c072d10 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.361+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f0b94004260 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.362+0000 7f0b98c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 0x7f0b780791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:56.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.362+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7f0b940b94e0 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.362+0000 7f0b9be00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 0x7f0b780791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:56.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.364+0000 7f0b9be00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 0x7f0b780791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1583a0 tx=0x7f0b8c005f90 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:56.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.366+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0b940823d0 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.584+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0b9c0af7f0 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.590+0000 7f0b98c00640 1 -- 172.21.15.44:0/923500031 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f0b94081cd0 con 0x7f0b9c15e1d0 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 msgr2=0x7f0b780791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 0x7f0b780791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1583a0 tx=0x7f0b8c005f90 comp rx=0 tx=0).stop 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 msgr2=0x7f0b9c1f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b94035040 tx=0x7f0b94032c60 comp rx=0 tx=0).stop 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 shutdown_connections 2024-07-17T07:11:56.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0b78076d00 0x7f0b780791c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b9c15e1d0 0x7f0b9c1f7910 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b9c1544f0 0x7f0b9c1f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 --2- 172.21.15.44:0/923500031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b9c0b2cd0 0x7f0b9c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.593+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 >> 172.21.15.44:0/923500031 conn(0x7f0b9c06bf40 msgr2=0x7f0b9c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.594+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 shutdown_connections 2024-07-17T07:11:56.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:56.594+0000 7f0ba2c00640 1 -- 172.21.15.44:0/923500031 wait complete. 2024-07-17T07:11:56.615 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:11:56.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:11:56.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:11:56.844 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:11:56.844 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:11:56.844 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:56.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:11:56.903 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:11:56.903 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:56.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:11:56.957 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:11:56.957 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:57.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:11:57.018 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:11:57.018 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:11:57.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:11:57.074 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:11:57.074 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:11:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 -- 172.21.15.44:0/3695806329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf920 msgr2=0x7fe6b4141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/3695806329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf920 0x7fe6b4141040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40d00a0 tx=0x7fe6ac02a820 comp rx=0 tx=0).stop 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 -- 172.21.15.44:0/3695806329 shutdown_connections 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/3695806329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf920 0x7fe6b4141040 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/3695806329 >> v2:172.21.15.175:3300/0 conn(0x7fe6b40cf540 0x7fe6b40b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 -- 172.21.15.44:0/3695806329 >> 172.21.15.44:0/3695806329 conn(0x7fe6b40b0e80 msgr2=0x7fe6b40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 -- 172.21.15.44:0/3695806329 shutdown_connections 2024-07-17T07:11:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.295+0000 7fe6b3e00640 1 -- 172.21.15.44:0/3695806329 wait complete. 2024-07-17T07:11:57.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.297+0000 7fe6b3e00640 1 Processor -- start 2024-07-17T07:11:57.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.297+0000 7fe6b3e00640 1 -- start start 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b40b3db0 0x7fe6b4072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b40cf920 0x7fe6b4073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b40ae170 con 0x7fe6b40cf540 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b40adff0 con 0x7fe6b40cf920 2024-07-17T07:11:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.298+0000 7fe6b3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b40ae2f0 con 0x7fe6b40b3db0 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55152/0 (socket says 172.21.15.44:55152) 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 -- 172.21.15.44:0/2087497324 learned_addr learned my addr 172.21.15.44:0/2087497324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6b2a00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b40b3db0 0x7fe6b4072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6b3400640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b40cf920 0x7fe6b4073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 -- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b40b3db0 msgr2=0x7fe6b4072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b40b3db0 0x7fe6b4072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 -- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b40cf920 msgr2=0x7fe6b4073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b40cf920 0x7fe6b4073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 -- 172.21.15.44:0/2087497324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b41f4730 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.299+0000 7fe6abe00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c420 tx=0x7fe69c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.300+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c011790 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.300+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe69c017070 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.300+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c011c80 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.300+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b41f4a10 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.300+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41f4450 con 0x7fe6b40cf540 2024-07-17T07:11:57.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.301+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b4156520 con 0x7fe6b40cf540 2024-07-17T07:11:57.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.302+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fe69c01c460 con 0x7fe6b40cf540 2024-07-17T07:11:57.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.303+0000 7fe6aaa00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 0x7fe690079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.303+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7fe69c098620 con 0x7fe6b40cf540 2024-07-17T07:11:57.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.304+0000 7fe6b2a00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 0x7fe690079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.305+0000 7fe6b2a00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 0x7fe690079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005fd0 tx=0x7fe6a4005850 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:57.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.308+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe69c0616b0 con 0x7fe6b40cf540 2024-07-17T07:11:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.556+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b40ae7a0 con 0x7fe6b40cf540 2024-07-17T07:11:57.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.558+0000 7fe6aaa00640 1 -- 172.21.15.44:0/2087497324 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3914 (secure 0 0 0) 0x7fe69c060fb0 con 0x7fe6b40cf540 2024-07-17T07:11:57.558 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:11:57.558 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":37,"btime":"2024-07-17T07:11:49:593965+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/762385539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":762385539},{"type":"v1","addr":"172.21.15.175:6835","nonce":762385539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5527,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1514419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1514419520},{"type":"v1","addr":"172.21.15.175:6837","nonce":1514419520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1254061104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1254061104},{"type":"v1","addr":"172.21.15.44:6833","nonce":1254061104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6552,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4031690091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4031690091},{"type":"v1","addr":"172.21.15.44:6835","nonce":4031690091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-07-17T07:11:57.558 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 37 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 msgr2=0x7fe690079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 0x7fe690079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4005fd0 tx=0x7fe6a4005850 comp rx=0 tx=0).stop 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 msgr2=0x7fe6b41f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c420 tx=0x7fe69c00c8e0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 shutdown_connections 2024-07-17T07:11:57.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fe690076cd0 0x7fe690079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b40cf920 0x7fe6b4073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b40cf540 0x7fe6b41f3f10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 --2- 172.21.15.44:0/2087497324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b40b3db0 0x7fe6b4072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 >> 172.21.15.44:0/2087497324 conn(0x7fe6b40b0e80 msgr2=0x7fe6b4068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 shutdown_connections 2024-07-17T07:11:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.561+0000 7fe6b3e00640 1 -- 172.21.15.44:0/2087497324 wait complete. 2024-07-17T07:11:57.582 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:11:57.582 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:11:57.582 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:11:57.583 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:11:57.583 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:11:57.583 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:11:57.583 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:11:57.583 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:11:57.583 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:11:57.583 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:11:57.584 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:11:57.584 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:11:57.584 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:11:57.584 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:11:57.584 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:11:57.584 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:11:57.601 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:11:57.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:11:57.602 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:11:57.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:11:57.615 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:11:57.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:11:57.616 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:11:57.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:11:57.648 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:11:57.649 DEBUG:teuthology.parallel:result is None 2024-07-17T07:11:57.654 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:11:57.654 DEBUG:teuthology.parallel:result is None 2024-07-17T07:11:57.664 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:11:57.664 DEBUG:teuthology.parallel:result is None 2024-07-17T07:11:57.670 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:11:57.670 DEBUG:teuthology.parallel:result is None 2024-07-17T07:11:57.671 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:11:57.797 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:11:57.806 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:11:57.815 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:11:57.816 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:11:57.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-07-17T07:11:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 -- 172.21.15.44:0/2366174155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 msgr2=0x7fce5c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 --2- 172.21.15.44:0/2366174155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0ce5b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009830 tx=0x7fce4c02a5a0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 -- 172.21.15.44:0/2366174155 shutdown_connections 2024-07-17T07:11:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 --2- 172.21.15.44:0/2366174155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 0x7fce5c161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 --2- 172.21.15.44:0/2366174155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0ce5b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 -- 172.21.15.44:0/2366174155 >> 172.21.15.44:0/2366174155 conn(0x7fce5c0d3700 msgr2=0x7fce5c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 -- 172.21.15.44:0/2366174155 shutdown_connections 2024-07-17T07:11:57.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.863+0000 7fce60a00640 1 -- 172.21.15.44:0/2366174155 wait complete. 2024-07-17T07:11:57.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.865+0000 7fce60a00640 1 Processor -- start 2024-07-17T07:11:57.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.865+0000 7fce60a00640 1 -- start start 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 0x7fce5c14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce5c0b6790 0x7fce5c1f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c162420 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1622a0 con 0x7fce5c0b6790 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce60a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c167910 con 0x7fce5c0b29e0 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55212/0 (socket says 172.21.15.44:55212) 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce5a800640 1 -- 172.21.15.44:0/2870503959 learned_addr learned my addr 172.21.15.44:0/2870503959 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.866+0000 7fce59e00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 0x7fce5c14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce51400640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce5c0b6790 0x7fce5c1f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 -- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 msgr2=0x7fce5c14c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 0x7fce5c14c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 -- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce5c0b6790 msgr2=0x7fce5c1f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce5c0b6790 0x7fce5c1f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 -- 172.21.15.44:0/2870503959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1f8290 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce5a800640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fce5400eb10 tx=0x7fce5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.867+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54004b20 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.868+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce54004ce0 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.868+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54005280 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.868+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1513a0 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.868+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1505e0 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.869+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c0ce5b0 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.870+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fce54005420 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.871+0000 7fce52a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 0x7fce38079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.871+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6500+0+0 (secure 0 0 0) 0x7fce54099470 con 0x7fce5c0ce1d0 2024-07-17T07:11:57.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.872+0000 7fce59e00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 0x7fce38079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:57.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.872+0000 7fce59e00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 0x7fce38079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034000 tx=0x7fce4c004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:57.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:57.875+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fce54061d10 con 0x7fce5c0ce1d0 2024-07-17T07:11:58.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:58.095+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fce5c0b8b20 con 0x7fce5c0ce1d0 2024-07-17T07:11:59.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.591+0000 7fce52a00640 1 -- 172.21.15.44:0/2870503959 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) ==== 97+0+0 (secure 0 0 0) 0x7fce54066260 con 0x7fce5c0ce1d0 2024-07-17T07:11:59.592 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.595+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 msgr2=0x7fce38079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.595+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 0x7fce38079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034000 tx=0x7fce4c004730 comp rx=0 tx=0).stop 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.595+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 msgr2=0x7fce5c0d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.595+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fce5400eb10 tx=0x7fce5400c2d0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 shutdown_connections 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fce38076cd0 0x7fce38079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce5c0ce1d0 0x7fce5c0d1b50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce5c0b6790 0x7fce5c1f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 --2- 172.21.15.44:0/2870503959 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce5c0b29e0 0x7fce5c14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 >> 172.21.15.44:0/2870503959 conn(0x7fce5c0d3700 msgr2=0x7fce5c0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:59.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 shutdown_connections 2024-07-17T07:11:59.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.596+0000 7fce60a00640 1 -- 172.21.15.44:0/2870503959 wait complete. 2024-07-17T07:11:59.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:11:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 -- 172.21.15.44:0/2723923338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac072a10 msgr2=0x7f54ac0af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 --2- 172.21.15.44:0/2723923338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac072a10 0x7f54ac0af3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 -- 172.21.15.44:0/2723923338 shutdown_connections 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 --2- 172.21.15.44:0/2723923338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac072a10 0x7f54ac0af3f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 --2- 172.21.15.44:0/2723923338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 0x7f54ac06c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 -- 172.21.15.44:0/2723923338 >> 172.21.15.44:0/2723923338 conn(0x7f54ac0b5890 msgr2=0x7f54ac0b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.819+0000 7f54b2800640 1 -- 172.21.15.44:0/2723923338 shutdown_connections 2024-07-17T07:11:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.820+0000 7f54b2800640 1 -- 172.21.15.44:0/2723923338 wait complete. 2024-07-17T07:11:59.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.821+0000 7f54b2800640 1 Processor -- start 2024-07-17T07:11:59.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.821+0000 7f54b2800640 1 -- start start 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 0x7f54ac14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 0x7f54ac15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac0afcc0 con 0x7f54ac072a10 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac0afb40 con 0x7f54ac15d620 2024-07-17T07:11:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.822+0000 7f54b2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54ac0afe40 con 0x7f54ac0b1330 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34920/0 (socket says 172.21.15.44:34920) 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 -- 172.21.15.44:0/4006315182 learned_addr learned my addr 172.21.15.44:0/4006315182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1e00640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 0x7f54ac15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b0a00640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 0x7f54ac14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 -- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 msgr2=0x7f54ac14b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 0x7f54ac14b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 -- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 msgr2=0x7f54ac15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 0x7f54ac15f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1400640 1 -- 172.21.15.44:0/4006315182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac157e80 con 0x7f54ac072a10 2024-07-17T07:11:59.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.823+0000 7f54b1e00640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 0x7f54ac15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:11:59.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54b1400640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c3d0 tx=0x7f54a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0017070 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54a00053c0 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0003850 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac15ee20 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.824+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac155870 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.825+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ac160b40 con 0x7f54ac072a10 2024-07-17T07:11:59.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.827+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f54a001a070 con 0x7f54ac072a10 2024-07-17T07:11:59.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.828+0000 7f54a9600640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 0x7f5490079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:11:59.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.828+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5769+0+0 (secure 0 0 0) 0x7f54a0097800 con 0x7f54ac072a10 2024-07-17T07:11:59.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.829+0000 7f54b0a00640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 0x7f5490079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:11:59.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.830+0000 7f54b0a00640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 0x7f5490079190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54ac1f78a0 tx=0x7f549c009500 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:11:59.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:11:59.833+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f54ac160b40 con 0x7f54ac072a10 2024-07-17T07:12:00.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.121+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f54ac0b03d0 con 0x7f54ac072a10 2024-07-17T07:12:00.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.123+0000 7f54a9600640 1 -- 172.21.15.44:0/4006315182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7f54a0060c10 con 0x7f54ac072a10 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:00.123 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:00.125 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:12:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:12:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.127 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.128 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:12:00.129 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:12:00.129 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.129 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.130 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.131 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.132 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.133 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:12:00.134 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.135 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:00.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:00.137 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:00.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:12:00.139 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:12:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 msgr2=0x7f5490079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 0x7f5490079190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54ac1f78a0 tx=0x7f549c009500 comp rx=0 tx=0).stop 2024-07-17T07:12:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 msgr2=0x7f54ac155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f54a000c3d0 tx=0x7f54a000c890 comp rx=0 tx=0).stop 2024-07-17T07:12:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 shutdown_connections 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5490076cd0 0x7f5490079190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54ac15d620 0x7f54ac15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54ac0b1330 0x7f54ac14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 --2- 172.21.15.44:0/4006315182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54ac072a10 0x7f54ac155330 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 >> 172.21.15.44:0/4006315182 conn(0x7f54ac0b5890 msgr2=0x7f54ac159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 shutdown_connections 2024-07-17T07:12:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.126+0000 7f54b2800640 1 -- 172.21.15.44:0/4006315182 wait complete. 2024-07-17T07:12:00.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:12:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 -- 172.21.15.44:0/3648916553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 msgr2=0x7f9384005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 --2- 172.21.15.44:0/3648916553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f9384005210 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f939800c2e0 tx=0x7f9384005830 comp rx=0 tx=0).stop 2024-07-17T07:12:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a4a00640 1 -- 172.21.15.44:0/3648916553 reap_dead start 2024-07-17T07:12:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 -- 172.21.15.44:0/3648916553 shutdown_connections 2024-07-17T07:12:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 -- 172.21.15.44:0/3648916553 >> 172.21.15.44:0/3648916553 conn(0x7f93a00af940 msgr2=0x7f93a00afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 -- 172.21.15.44:0/3648916553 shutdown_connections 2024-07-17T07:12:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.352+0000 7f93a6a00640 1 -- 172.21.15.44:0/3648916553 wait complete. 2024-07-17T07:12:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.354+0000 7f93a6a00640 1 Processor -- start 2024-07-17T07:12:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.354+0000 7f93a6a00640 1 -- start start 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a014d290 0x7f93a01532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a0153830 0x7f93a0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f93a006f5e0 con 0x7f9384004d90 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93a00cf7e0 con 0x7f93a0153830 2024-07-17T07:12:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f93a6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93a006f760 con 0x7f93a014d290 2024-07-17T07:12:00.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.355+0000 7f939fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34936/0 (socket says 172.21.15.44:34936) 2024-07-17T07:12:00.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 -- 172.21.15.44:0/1438671783 learned_addr learned my addr 172.21.15.44:0/1438671783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f93a4a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a0153830 0x7f93a0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939f400640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a014d290 0x7f93a01532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 -- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a014d290 msgr2=0x7f93a01532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a014d290 0x7f93a01532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 -- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a0153830 msgr2=0x7f93a0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a0153830 0x7f93a0072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 -- 172.21.15.44:0/1438671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a0153cb0 con 0x7f9384004d90 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.356+0000 7f939fe00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f939000d8f0 tx=0x7f939000ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.357+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390016020 con 0x7f9384004d90 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.357+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f939000b8d0 con 0x7f9384004d90 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.357+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939000bd20 con 0x7f9384004d90 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.357+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a0153f90 con 0x7f9384004d90 2024-07-17T07:12:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.357+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01f3140 con 0x7f9384004d90 2024-07-17T07:12:00.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.358+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a006fcf0 con 0x7f9384004d90 2024-07-17T07:12:00.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.359+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f9390019020 con 0x7f9384004d90 2024-07-17T07:12:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.360+0000 7f9397e00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 0x7f937c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.360+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5769+0+0 (secure 0 0 0) 0x7f93900987f0 con 0x7f9384004d90 2024-07-17T07:12:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.361+0000 7f939f400640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 0x7f937c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.362+0000 7f939f400640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 0x7f937c079190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93880058c0 tx=0x7f9388005eb0 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:00.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.364+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9390020020 con 0x7f9384004d90 2024-07-17T07:12:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.578+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f93a00b4a40 con 0x7f9384004d90 2024-07-17T07:12:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.579+0000 7f9397e00640 1 -- 172.21.15.44:0/1438671783 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) ==== 84+0+65 (secure 0 0 0) 0x7f9390061ad0 con 0x7f9384004d90 2024-07-17T07:12:00.579 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:00.579 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:12:00.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:00.580 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:12:00.580 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:12:00.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:00.580 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:12:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.582+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 msgr2=0x7f937c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.582+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 0x7f937c079190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93880058c0 tx=0x7f9388005eb0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.582+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 msgr2=0x7f93a01f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.582+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f939000d8f0 tx=0x7f939000ddb0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 shutdown_connections 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f937c076cd0 0x7f937c079190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93a0153830 0x7f93a0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93a014d290 0x7f93a01532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 --2- 172.21.15.44:0/1438671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9384004d90 0x7f93a01f3940 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 >> 172.21.15.44:0/1438671783 conn(0x7f93a00af940 msgr2=0x7f93a01596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 shutdown_connections 2024-07-17T07:12:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.583+0000 7f93a6a00640 1 -- 172.21.15.44:0/1438671783 wait complete. 2024-07-17T07:12:00.602 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:12:00.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:12:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 -- 172.21.15.44:0/609684373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 msgr2=0x7f260806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 --2- 172.21.15.44:0/609684373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f260806db60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2600005fd0 tx=0x7f260002a810 comp rx=0 tx=0).stop 2024-07-17T07:12:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 -- 172.21.15.44:0/609684373 shutdown_connections 2024-07-17T07:12:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 --2- 172.21.15.44:0/609684373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f260806db60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 --2- 172.21.15.44:0/609684373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f260815b060 0x7f260806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 -- 172.21.15.44:0/609684373 >> 172.21.15.44:0/609684373 conn(0x7f26080b0eb0 msgr2=0x7f26080b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:00.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 -- 172.21.15.44:0/609684373 shutdown_connections 2024-07-17T07:12:00.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.811+0000 7f260d600640 1 -- 172.21.15.44:0/609684373 wait complete. 2024-07-17T07:12:00.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.813+0000 7f260d600640 1 Processor -- start 2024-07-17T07:12:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.813+0000 7f260d600640 1 -- start start 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26080cb880 0x7f26081518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 0x7f2608159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f260815b5c0 con 0x7f260815c220 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f260815b440 con 0x7f260815b060 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f260d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f260815f4a0 con 0x7f26080cb880 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f2607400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.814+0000 7f2607400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34952/0 (socket says 172.21.15.44:34952) 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 -- 172.21.15.44:0/735162034 learned_addr learned my addr 172.21.15.44:0/735162034 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2606a00640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26080cb880 0x7f26081518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 -- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26080cb880 msgr2=0x7f26081518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2606000640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 0x7f2608159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26080cb880 0x7f26081518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 -- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 msgr2=0x7f2608159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 0x7f2608159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 -- 172.21.15.44:0/735162034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081548c0 con 0x7f260815c220 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2606000640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 0x7f2608159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.815+0000 7f2607400640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2600009b30 tx=0x7f26000315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.816+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600041020 con 0x7f260815c220 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.816+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2600031ac0 con 0x7f260815c220 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.816+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600037040 con 0x7f260815c220 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.816+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2608154ba0 con 0x7f260815c220 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.816+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2608155440 con 0x7f260815c220 2024-07-17T07:12:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.817+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26080b1570 con 0x7f260815c220 2024-07-17T07:12:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.818+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f2600044020 con 0x7f260815c220 2024-07-17T07:12:00.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.819+0000 7f25fea00640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 0x7f25e80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:00.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.819+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5769+0+0 (secure 0 0 0) 0x7f26000b9060 con 0x7f260815c220 2024-07-17T07:12:00.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.820+0000 7f2606a00640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 0x7f25e80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:00.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.821+0000 7f2606a00640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 0x7f25e80791c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005fd0 tx=0x7f25f8005850 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:00.824+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f260004b070 con 0x7f260815c220 2024-07-17T07:12:01.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.034+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f260815f940 con 0x7f260815c220 2024-07-17T07:12:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.703+0000 7f25fea00640 1 -- 172.21.15.44:0/735162034 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) ==== 138+0+0 (secure 0 0 0) 0x7f26000822c0 con 0x7f260815c220 2024-07-17T07:12:01.703 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:12:01.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.707+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 msgr2=0x7f25e80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:01.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.707+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 0x7f25e80791c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f25f8005fd0 tx=0x7f25f8005850 comp rx=0 tx=0).stop 2024-07-17T07:12:01.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.707+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 msgr2=0x7f2608156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:01.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.707+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2600009b30 tx=0x7f26000315d0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 shutdown_connections 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f25e8076d00 0x7f25e80791c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f260815c220 0x7f2608156190 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f260815b060 0x7f2608159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 --2- 172.21.15.44:0/735162034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26080cb880 0x7f26081518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 >> 172.21.15.44:0/735162034 conn(0x7f26080b0eb0 msgr2=0x7f2608157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:01.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 shutdown_connections 2024-07-17T07:12:01.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.708+0000 7f260d600640 1 -- 172.21.15.44:0/735162034 wait complete. 2024-07-17T07:12:01.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 -- 172.21.15.44:0/280326581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 msgr2=0x7f239415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 --2- 172.21.15.44:0/280326581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239415a450 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f239415ce40 tx=0x7f238802a560 comp rx=0 tx=0).stop 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 -- 172.21.15.44:0/280326581 shutdown_connections 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 --2- 172.21.15.44:0/280326581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 0x7f23940ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 --2- 172.21.15.44:0/280326581 >> v2:172.21.15.175:3300/0 conn(0x7f239406d420 0x7f239406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 --2- 172.21.15.44:0/280326581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239415a450 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 -- 172.21.15.44:0/280326581 >> 172.21.15.44:0/280326581 conn(0x7f23940cf0d0 msgr2=0x7f23940cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 -- 172.21.15.44:0/280326581 shutdown_connections 2024-07-17T07:12:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.938+0000 7f239ac00640 1 -- 172.21.15.44:0/280326581 wait complete. 2024-07-17T07:12:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.940+0000 7f239ac00640 1 Processor -- start 2024-07-17T07:12:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.940+0000 7f239ac00640 1 -- start start 2024-07-17T07:12:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f239406d420 0x7f239409c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:01.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 0x7f239406bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23940b3240 con 0x7f239406f4d0 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23940b30c0 con 0x7f239406d420 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23940b33c0 con 0x7f23941503c0 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34962/0 (socket says 172.21.15.44:34962) 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 -- 172.21.15.44:0/48758933 learned_addr learned my addr 172.21.15.44:0/48758933 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2399800640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f239406d420 0x7f239409c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f239a200640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 0x7f239406bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 -- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 msgr2=0x7f239406bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 0x7f239406bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 -- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f239406d420 msgr2=0x7f239409c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:01.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f239406d420 0x7f239409c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:01.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.941+0000 7f2398e00640 1 -- 172.21.15.44:0/48758933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239409d1b0 con 0x7f239406f4d0 2024-07-17T07:12:01.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f2398e00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f238000b4a0 tx=0x7f238000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:01.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380014070 con 0x7f239406f4d0 2024-07-17T07:12:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2380010700 con 0x7f239406f4d0 2024-07-17T07:12:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2394072560 con 0x7f239406f4d0 2024-07-17T07:12:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380010b50 con 0x7f239406f4d0 2024-07-17T07:12:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.942+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2394072b20 con 0x7f239406f4d0 2024-07-17T07:12:01.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.944+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239415a450 con 0x7f239406f4d0 2024-07-17T07:12:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.945+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f2380017070 con 0x7f239406f4d0 2024-07-17T07:12:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.946+0000 7f2391600640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 0x7f2374079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.946+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6139+0+0 (secure 0 0 0) 0x7f2380098820 con 0x7f239406f4d0 2024-07-17T07:12:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.947+0000 7f2399800640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 0x7f2374079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.948+0000 7f2399800640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 0x7f2374079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2388033000 tx=0x7f2388004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:01.950+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f239409d1b0 con 0x7f239406f4d0 2024-07-17T07:12:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.162+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2394074ee0 con 0x7f239406f4d0 2024-07-17T07:12:02.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.722+0000 7f2391600640 1 -- 172.21.15.44:0/48758933 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) ==== 170+0+0 (secure 0 0 0) 0x7f2380061aa0 con 0x7f239406f4d0 2024-07-17T07:12:02.723 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 msgr2=0x7f2374079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 0x7f2374079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2388033000 tx=0x7f2388004730 comp rx=0 tx=0).stop 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 msgr2=0x7f239414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f238000b4a0 tx=0x7f238000b960 comp rx=0 tx=0).stop 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 shutdown_connections 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f2374076cd0 0x7f2374079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23941503c0 0x7f239406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239406f4d0 0x7f239414ef70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.725+0000 7f239ac00640 1 --2- 172.21.15.44:0/48758933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f239406d420 0x7f239409c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.726+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 >> 172.21.15.44:0/48758933 conn(0x7f23940cf0d0 msgr2=0x7f23940afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:02.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.726+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 shutdown_connections 2024-07-17T07:12:02.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.726+0000 7f239ac00640 1 -- 172.21.15.44:0/48758933 wait complete. 2024-07-17T07:12:02.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:12:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 -- 172.21.15.44:0/2818505588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f716806f6d0 msgr2=0x7f716815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 --2- 172.21.15.44:0/2818505588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f716806f6d0 0x7f716815a5a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f71680b41a0 tx=0x7f715802a560 comp rx=0 tx=0).stop 2024-07-17T07:12:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 -- 172.21.15.44:0/2818505588 shutdown_connections 2024-07-17T07:12:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 --2- 172.21.15.44:0/2818505588 >> v2:172.21.15.175:3300/0 conn(0x7f71680b3280 0x7f716806eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 --2- 172.21.15.44:0/2818505588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f716806f6d0 0x7f716815a5a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.943+0000 7f716ce00640 1 -- 172.21.15.44:0/2818505588 >> 172.21.15.44:0/2818505588 conn(0x7f71680b0eb0 msgr2=0x7f71680b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.944+0000 7f716ce00640 1 -- 172.21.15.44:0/2818505588 shutdown_connections 2024-07-17T07:12:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.944+0000 7f716ce00640 1 -- 172.21.15.44:0/2818505588 wait complete. 2024-07-17T07:12:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.945+0000 7f716ce00640 1 Processor -- start 2024-07-17T07:12:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.945+0000 7f716ce00640 1 -- start start 2024-07-17T07:12:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 0x7f71680715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 0x7f71681f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71680aeb10 con 0x7f71680b3280 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71680ae990 con 0x7f716806f6d0 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.946+0000 7f716ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7168073e50 con 0x7f71680ae560 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34966/0 (socket says 172.21.15.44:34966) 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 -- 172.21.15.44:0/2155498907 learned_addr learned my addr 172.21.15.44:0/2155498907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7165e00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 0x7f71680715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7165400640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 0x7f71681f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 -- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 msgr2=0x7f71681f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 0x7f71681f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 -- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 msgr2=0x7f71680715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 0x7f71680715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7166800640 1 -- 172.21.15.44:0/2155498907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7168074360 con 0x7f71680b3280 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7165e00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 0x7f71680715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.947+0000 7f7165400640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 0x7f71681f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f7166800640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f716000c780 tx=0x7f716000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160017070 con 0x7f71680b3280 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7160005070 con 0x7f71680b3280 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160005500 con 0x7f71680b3280 2024-07-17T07:12:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7168074640 con 0x7f71680b3280 2024-07-17T07:12:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.948+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7168071b20 con 0x7f71680b3280 2024-07-17T07:12:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.949+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716815a5a0 con 0x7f71680b3280 2024-07-17T07:12:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.951+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f71600056a0 con 0x7f71680b3280 2024-07-17T07:12:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.952+0000 7f715e000640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 0x7f7148079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.952+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6521+0+0 (secure 0 0 0) 0x7f7160098010 con 0x7f71680b3280 2024-07-17T07:12:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.953+0000 7f7165e00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 0x7f7148079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:02.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.954+0000 7f7165e00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 0x7f7148079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7158033000 tx=0x7f7158004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:02.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:02.955+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7160020080 con 0x7f71680b3280 2024-07-17T07:12:03.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.212+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f71680b4200 con 0x7f71680b3280 2024-07-17T07:12:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.741+0000 7f715e000640 1 -- 172.21.15.44:0/2155498907 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v39) ==== 174+0+0 (secure 0 0 0) 0x7f7160061110 con 0x7f71680b3280 2024-07-17T07:12:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 11 and data pool 12 2024-07-17T07:12:03.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.754+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 msgr2=0x7f7148079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:03.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.754+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 0x7f7148079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7158033000 tx=0x7f7158004730 comp rx=0 tx=0).stop 2024-07-17T07:12:03.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.754+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 msgr2=0x7f71681f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:03.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.754+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f716000c780 tx=0x7f716000cc40 comp rx=0 tx=0).stop 2024-07-17T07:12:03.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 shutdown_connections 2024-07-17T07:12:03.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7148076cd0 0x7f7148079190 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7158033000 tx=0x7f7158004730 comp rx=0 tx=0).stop 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71680b3280 0x7f71681f5920 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71680ae560 0x7f71681f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 --2- 172.21.15.44:0/2155498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f716806f6d0 0x7f71680715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.755+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 >> 172.21.15.44:0/2155498907 conn(0x7f71680b0eb0 msgr2=0x7f71680b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.756+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 shutdown_connections 2024-07-17T07:12:03.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.756+0000 7f716ce00640 1 -- 172.21.15.44:0/2155498907 wait complete. 2024-07-17T07:12:03.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 -- 172.21.15.44:0/3099419943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 msgr2=0x7f43dc153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 --2- 172.21.15.44:0/3099419943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc153a50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005fd0 tx=0x7f43cc02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 -- 172.21.15.44:0/3099419943 shutdown_connections 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 --2- 172.21.15.44:0/3099419943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc0ac3e0 0x7f43dc0ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 --2- 172.21.15.44:0/3099419943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc153a50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.966+0000 7f43e1400640 1 -- 172.21.15.44:0/3099419943 >> 172.21.15.44:0/3099419943 conn(0x7f43dc0b0fb0 msgr2=0x7f43dc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.967+0000 7f43e1400640 1 -- 172.21.15.44:0/3099419943 shutdown_connections 2024-07-17T07:12:03.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.967+0000 7f43e1400640 1 -- 172.21.15.44:0/3099419943 wait complete. 2024-07-17T07:12:03.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.968+0000 7f43e1400640 1 Processor -- start 2024-07-17T07:12:03.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.968+0000 7f43e1400640 1 -- start start 2024-07-17T07:12:03.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:03.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43dc0ac3e0 0x7f43dc1f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:03.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc13ed60 0x7f43dc0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:03.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0ae800 con 0x7f43dc06f710 2024-07-17T07:12:03.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc06f0a0 con 0x7f43dc0ac3e0 2024-07-17T07:12:03.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.969+0000 7f43e1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc0ae980 con 0x7f43dc13ed60 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34984/0 (socket says 172.21.15.44:34984) 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 -- 172.21.15.44:0/2918407229 learned_addr learned my addr 172.21.15.44:0/2918407229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43db200640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc13ed60 0x7f43dc0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43d9e00640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43dc0ac3e0 0x7f43dc1f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 -- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc13ed60 msgr2=0x7f43dc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc13ed60 0x7f43dc0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 -- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43dc0ac3e0 msgr2=0x7f43dc1f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43dc0ac3e0 0x7f43dc1f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.970+0000 7f43da800640 1 -- 172.21.15.44:0/2918407229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc15d820 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43da800640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0099f0 tx=0x7f43cc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc03f070 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43cc005600 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc1f6320 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc035030 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.971+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc0997b0 con 0x7f43dc06f710 2024-07-17T07:12:03.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.972+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43dc14f910 con 0x7f43dc06f710 2024-07-17T07:12:03.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.974+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f43cc042070 con 0x7f43dc06f710 2024-07-17T07:12:03.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.975+0000 7f43d2a00640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 0x7f43bc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:03.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.976+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6621+0+0 (secure 0 0 0) 0x7f43cc0b9860 con 0x7f43dc06f710 2024-07-17T07:12:03.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.976+0000 7f43d9e00640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 0x7f43bc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:03.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.977+0000 7f43d9e00640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 0x7f43bc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005fd0 tx=0x7f43c8005850 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:03.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:03.979+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f43dc14f910 con 0x7f43dc06f710 2024-07-17T07:12:04.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.192+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43dc13fad0 con 0x7f43dc06f710 2024-07-17T07:12:04.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.196+0000 7f43d2a00640 1 -- 172.21.15.44:0/2918407229 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+17732 (secure 0 0 0) 0x7f43dc15d820 con 0x7f43dc06f710 2024-07-17T07:12:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:04.196 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":98,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:03.214444+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:01.050217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:02.180466+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 msgr2=0x7f43bc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 0x7f43bc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43c8005fd0 tx=0x7f43c8005850 comp rx=0 tx=0).stop 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 msgr2=0x7f43dc1f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0099f0 tx=0x7f43cc0038a0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 shutdown_connections 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f43bc076cd0 0x7f43bc079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43dc13ed60 0x7f43dc0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43dc0ac3e0 0x7f43dc1f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 --2- 172.21.15.44:0/2918407229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43dc06f710 0x7f43dc1f5760 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 >> 172.21.15.44:0/2918407229 conn(0x7f43dc0b0fb0 msgr2=0x7f43dc15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:04.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 shutdown_connections 2024-07-17T07:12:04.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.199+0000 7f43e1400640 1 -- 172.21.15.44:0/2918407229 wait complete. 2024-07-17T07:12:04.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.431+0000 7f38bf200640 1 -- 172.21.15.44:0/4225711893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 msgr2=0x7f38c015d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.431+0000 7f38bf200640 1 --2- 172.21.15.44:0/4225711893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015d2c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009860 tx=0x7f38ac02a540 comp rx=0 tx=0).stop 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.431+0000 7f38bf200640 1 -- 172.21.15.44:0/4225711893 shutdown_connections 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.432+0000 7f38bf200640 1 --2- 172.21.15.44:0/4225711893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015d2c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.432+0000 7f38bf200640 1 -- 172.21.15.44:0/4225711893 >> 172.21.15.44:0/4225711893 conn(0x7f38c00cfec0 msgr2=0x7f38c00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.432+0000 7f38bf200640 1 -- 172.21.15.44:0/4225711893 shutdown_connections 2024-07-17T07:12:04.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.432+0000 7f38bf200640 1 -- 172.21.15.44:0/4225711893 wait complete. 2024-07-17T07:12:04.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.434+0000 7f38bf200640 1 Processor -- start 2024-07-17T07:12:04.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.434+0000 7f38bf200640 1 -- start start 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 0x7f38c00c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 0x7f38c0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38c015e2a0 con 0x7f38c00ce5c0 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38c015e120 con 0x7f38c00b9060 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.435+0000 7f38bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38c015e420 con 0x7f38c00b8c80 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35006/0 (socket says 172.21.15.44:35006) 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 -- 172.21.15.44:0/2414655359 learned_addr learned my addr 172.21.15.44:0/2414655359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:04.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38bd400640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 0x7f38c0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 -- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 msgr2=0x7f38c00c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38bde00640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 0x7f38c00c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 0x7f38c00c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 -- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 msgr2=0x7f38c0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 0x7f38c0152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38be800640 1 -- 172.21.15.44:0/2414655359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c00c9590 con 0x7f38c00ce5c0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38bd400640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 0x7f38c0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.436+0000 7f38bde00640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 0x7f38c00c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38be800640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38b800cbc0 tx=0x7f38b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8016020 con 0x7f38c00ce5c0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b800ae30 con 0x7f38c00ce5c0 2024-07-17T07:12:04.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c00c9870 con 0x7f38c00ce5c0 2024-07-17T07:12:04.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012660 con 0x7f38c00ce5c0 2024-07-17T07:12:04.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.437+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c00c8320 con 0x7f38c00ce5c0 2024-07-17T07:12:04.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.438+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c00d2f90 con 0x7f38c00ce5c0 2024-07-17T07:12:04.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.440+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f38b8019020 con 0x7f38c00ce5c0 2024-07-17T07:12:04.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.441+0000 7f38b6000640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 0x7f38a00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.441+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6621+0+0 (secure 0 0 0) 0x7f38b8098f40 con 0x7f38c00ce5c0 2024-07-17T07:12:04.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.441+0000 7f38bde00640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 0x7f38a00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.443+0000 7f38bde00640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 0x7f38a00791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38b0003eb0 tx=0x7f38b0009040 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:04.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.445+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f38c00d2f90 con 0x7f38c00ce5c0 2024-07-17T07:12:04.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-07-17T07:12:04.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.658+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38c00b4ec0 con 0x7f38c00ce5c0 2024-07-17T07:12:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.661+0000 7f38b6000640 1 -- 172.21.15.44:0/2414655359 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+17732 (secure 0 0 0) 0x7f38b8020020 con 0x7f38c00ce5c0 2024-07-17T07:12:04.662 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:04.662 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":98,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:03.214444+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:01.050217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:02.180466+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 msgr2=0x7f38a00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 0x7f38a00791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38b0003eb0 tx=0x7f38b0009040 comp rx=0 tx=0).stop 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 msgr2=0x7f38c015fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38b800cbc0 tx=0x7f38b800a430 comp rx=0 tx=0).stop 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 shutdown_connections 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f38a0076d00 0x7f38a00791c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c00ce5c0 0x7f38c015fb30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c00b9060 0x7f38c0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 --2- 172.21.15.44:0/2414655359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c00b8c80 0x7f38c00c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.664+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 >> 172.21.15.44:0/2414655359 conn(0x7f38c00cfec0 msgr2=0x7f38c00b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.665+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 shutdown_connections 2024-07-17T07:12:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.665+0000 7f38bf200640 1 -- 172.21.15.44:0/2414655359 wait complete. 2024-07-17T07:12:04.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 -- 172.21.15.44:0/2487974303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe00040f0 msgr2=0x7fefe0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 --2- 172.21.15.44:0/2487974303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe00040f0 0x7fefe0004800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fefe400bb20 tx=0x7fefe0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffb400640 1 -- 172.21.15.44:0/2487974303 reap_dead start 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 -- 172.21.15.44:0/2487974303 shutdown_connections 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 -- 172.21.15.44:0/2487974303 >> 172.21.15.44:0/2487974303 conn(0x7feffc0cf0d0 msgr2=0x7feffc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 -- 172.21.15.44:0/2487974303 shutdown_connections 2024-07-17T07:12:04.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.878+0000 7feffbe00640 1 -- 172.21.15.44:0/2487974303 wait complete. 2024-07-17T07:12:04.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.880+0000 7feffbe00640 1 Processor -- start 2024-07-17T07:12:04.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.880+0000 7feffbe00640 1 -- start start 2024-07-17T07:12:04.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.880+0000 7feffbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 0x7feffc144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 0x7feffc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feffc0cb200 con 0x7feffc1f7970 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feffc0cb080 con 0x7fefe00040f0 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feffc0cb380 con 0x7feffc1484f0 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35018/0 (socket says 172.21.15.44:35018) 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 -- 172.21.15.44:0/1115874771 learned_addr learned my addr 172.21.15.44:0/1115874771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffaa00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 0x7feffc144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feff3e00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 0x7feffc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 -- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 msgr2=0x7feffc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 0x7feffc06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 -- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 msgr2=0x7feffc144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:04.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 0x7feffc144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffb400640 1 -- 172.21.15.44:0/1115874771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc1454a0 con 0x7feffc1f7970 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feff3e00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 0x7feffc06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.881+0000 7feffaa00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 0x7feffc144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feffb400640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feffc15c110 tx=0x7feff4009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4015070 con 0x7feffc1f7970 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feff4004e10 con 0x7feffc1f7970 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4010900 con 0x7feffc1f7970 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc154f90 con 0x7feffc1f7970 2024-07-17T07:12:04.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.882+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc155550 con 0x7feffc1f7970 2024-07-17T07:12:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.883+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc0cb910 con 0x7feffc1f7970 2024-07-17T07:12:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.885+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7feff4010aa0 con 0x7feffc1f7970 2024-07-17T07:12:04.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.886+0000 7feff2a00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 0x7fefd8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:04.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.886+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7feff4099130 con 0x7feffc1f7970 2024-07-17T07:12:04.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.886+0000 7feffaa00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 0x7fefd8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:04.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.887+0000 7feffaa00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 0x7fefd8079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec001860 tx=0x7fefec001c80 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:04.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:04.890+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feffc1454a0 con 0x7feffc1f7970 2024-07-17T07:12:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.129+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feffc15b120 con 0x7feffc1f7970 2024-07-17T07:12:05.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.752+0000 7feff2a00640 1 -- 172.21.15.44:0/1115874771 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7feff4062040 con 0x7feffc1f7970 2024-07-17T07:12:05.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 msgr2=0x7fefd8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:05.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 0x7fefd8079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefec001860 tx=0x7fefec001c80 comp rx=0 tx=0).stop 2024-07-17T07:12:05.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 msgr2=0x7feffc073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:05.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feffc15c110 tx=0x7feff4009a80 comp rx=0 tx=0).stop 2024-07-17T07:12:05.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 shutdown_connections 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fefd8076cd0 0x7fefd8079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feffc1f7970 0x7feffc073b20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feffc1484f0 0x7feffc06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 --2- 172.21.15.44:0/1115874771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefe00040f0 0x7feffc144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 >> 172.21.15.44:0/1115874771 conn(0x7feffc0cf0d0 msgr2=0x7feffc06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 shutdown_connections 2024-07-17T07:12:05.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.756+0000 7feffbe00640 1 -- 172.21.15.44:0/1115874771 wait complete. 2024-07-17T07:12:05.777 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:12:05.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:12:05.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 -- 172.21.15.44:0/951346025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 msgr2=0x7f108c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 --2- 172.21.15.44:0/951346025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f108c00b9c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c610 tx=0x7f108c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 -- 172.21.15.44:0/951346025 shutdown_connections 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 --2- 172.21.15.44:0/951346025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f108c00b9c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 -- 172.21.15.44:0/951346025 >> 172.21.15.44:0/951346025 conn(0x7f10a80cf0d0 msgr2=0x7f10a80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 -- 172.21.15.44:0/951346025 shutdown_connections 2024-07-17T07:12:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:05.999+0000 7f10a7e00640 1 -- 172.21.15.44:0/951346025 wait complete. 2024-07-17T07:12:06.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.001+0000 7f10a7e00640 1 Processor -- start 2024-07-17T07:12:06.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.001+0000 7f10a7e00640 1 -- start start 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a814e760 0x7f10a815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a81442e0 0x7f10a80afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10a806ec70 con 0x7f108c0052f0 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10a815c680 con 0x7f10a814e760 2024-07-17T07:12:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10a806edf0 con 0x7f10a81442e0 2024-07-17T07:12:06.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:06.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35032/0 (socket says 172.21.15.44:35032) 2024-07-17T07:12:06.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.002+0000 7f10a6a00640 1 -- 172.21.15.44:0/1967567355 learned_addr learned my addr 172.21.15.44:0/1967567355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 -- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a81442e0 msgr2=0x7f10a80afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a81442e0 0x7f10a80afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 -- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a814e760 msgr2=0x7f10a815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a814e760 0x7f10a815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 -- 172.21.15.44:0/1967567355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a814d6c0 con 0x7f108c0052f0 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f10a6a00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1098003ed0 tx=0x7f1098001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.003+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098015070 con 0x7f108c0052f0 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.004+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1098011840 con 0x7f108c0052f0 2024-07-17T07:12:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.004+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a814d9a0 con 0x7f108c0052f0 2024-07-17T07:12:06.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.004+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098011c70 con 0x7f108c0052f0 2024-07-17T07:12:06.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.004+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a814df60 con 0x7f108c0052f0 2024-07-17T07:12:06.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.005+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a806f2f0 con 0x7f108c0052f0 2024-07-17T07:12:06.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.006+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f1098018070 con 0x7f108c0052f0 2024-07-17T07:12:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.007+0000 7f109ea00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 0x7f1084079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.007+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f10980990e0 con 0x7f108c0052f0 2024-07-17T07:12:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.008+0000 7f109fe00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 0x7f1084079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.009+0000 7f109fe00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 0x7f1084079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10a814eb80 tx=0x7f1090005d50 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.012+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1098062070 con 0x7f108c0052f0 2024-07-17T07:12:06.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.265+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f10a80ae3d0 con 0x7f108c0052f0 2024-07-17T07:12:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.765+0000 7f109ea00640 1 -- 172.21.15.44:0/1967567355 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f1098061970 con 0x7f108c0052f0 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 msgr2=0x7f1084079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 0x7f1084079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10a814eb80 tx=0x7f1090005d50 comp rx=0 tx=0).stop 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 msgr2=0x7f10a814aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1098003ed0 tx=0x7f1098001630 comp rx=0 tx=0).stop 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 shutdown_connections 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f1084076cd0 0x7f1084079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a81442e0 0x7f10a80afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a814e760 0x7f10a815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 --2- 172.21.15.44:0/1967567355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f108c0052f0 0x7f10a814aa80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 >> 172.21.15.44:0/1967567355 conn(0x7f10a80cf0d0 msgr2=0x7f10a806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 shutdown_connections 2024-07-17T07:12:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:06.769+0000 7f10a7e00640 1 -- 172.21.15.44:0/1967567355 wait complete. 2024-07-17T07:12:06.792 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:12:06.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.000+0000 7f5197e00640 1 -- 172.21.15.44:0/2999347596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 msgr2=0x7f517c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.000+0000 7f5197e00640 1 --2- 172.21.15.44:0/2999347596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f517c0051c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f519000c640 tx=0x7f517c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.000+0000 7f5197400640 1 -- 172.21.15.44:0/2999347596 reap_dead start 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.001+0000 7f5197e00640 1 -- 172.21.15.44:0/2999347596 shutdown_connections 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.001+0000 7f5197e00640 1 -- 172.21.15.44:0/2999347596 >> 172.21.15.44:0/2999347596 conn(0x7f51980cfec0 msgr2=0x7f51980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.001+0000 7f5197e00640 1 -- 172.21.15.44:0/2999347596 shutdown_connections 2024-07-17T07:12:07.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.001+0000 7f5197e00640 1 -- 172.21.15.44:0/2999347596 wait complete. 2024-07-17T07:12:07.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.002+0000 7f5197e00640 1 Processor -- start 2024-07-17T07:12:07.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.002+0000 7f5197e00640 1 -- start start 2024-07-17T07:12:07.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 0x7f51980afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 0x7f51981f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e780 con 0x7f517c004d40 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e600 con 0x7f51980afa70 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.003+0000 7f5197e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f519806e900 con 0x7f51981f3c40 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35054/0 (socket says 172.21.15.44:35054) 2024-07-17T07:12:07.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 -- 172.21.15.44:0/2971392351 learned_addr learned my addr 172.21.15.44:0/2971392351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:07.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196000640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 0x7f51980afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5197400640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 0x7f51981f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 -- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 msgr2=0x7f51981f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 0x7f51981f40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 -- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 msgr2=0x7f51980afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 0x7f51980afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196a00640 1 -- 172.21.15.44:0/2971392351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51980b0430 con 0x7f517c004d40 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5196000640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 0x7f51980afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.004+0000 7f5197400640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 0x7f51981f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f5196a00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f518800e970 tx=0x7f518800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518800ccc0 con 0x7f517c004d40 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f518800ce60 con 0x7f517c004d40 2024-07-17T07:12:07.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51981f49a0 con 0x7f517c004d40 2024-07-17T07:12:07.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188005260 con 0x7f517c004d40 2024-07-17T07:12:07.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.005+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51980ab420 con 0x7f517c004d40 2024-07-17T07:12:07.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.006+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51980b51f0 con 0x7f517c004d40 2024-07-17T07:12:07.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.008+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f5188005400 con 0x7f517c004d40 2024-07-17T07:12:07.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.009+0000 7f518ea00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 0x7f5174079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.009+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f5188099370 con 0x7f517c004d40 2024-07-17T07:12:07.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.009+0000 7f5196000640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 0x7f5174079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.011+0000 7f5196000640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 0x7f5174079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5180009b10 tx=0x7f5180005f90 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.013+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f51880621f0 con 0x7f517c004d40 2024-07-17T07:12:07.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.265+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51980b5d70 con 0x7f517c004d40 2024-07-17T07:12:07.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.266+0000 7f518ea00640 1 -- 172.21.15.44:0/2971392351 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7f5188062e20 con 0x7f517c004d40 2024-07-17T07:12:07.267 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:07.267 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.269+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 msgr2=0x7f5174079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.269+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 0x7f5174079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5180009b10 tx=0x7f5180005f90 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.269+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 msgr2=0x7f519814f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.269+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f518800e970 tx=0x7f518800ee30 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 shutdown_connections 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f5174076cd0 0x7f5174079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51981f3c40 0x7f51981f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51980afa70 0x7f51980afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 --2- 172.21.15.44:0/2971392351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f517c004d40 0x7f519814f220 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 >> 172.21.15.44:0/2971392351 conn(0x7f51980cfec0 msgr2=0x7f51980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 shutdown_connections 2024-07-17T07:12:07.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.270+0000 7f5197e00640 1 -- 172.21.15.44:0/2971392351 wait complete. 2024-07-17T07:12:07.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 -- 172.21.15.44:0/31236086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 msgr2=0x7fd098003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/31236086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd098003f60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009b50 tx=0x7fd098005830 comp rx=0 tx=0).stop 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0b7e00640 1 -- 172.21.15.44:0/31236086 reap_dead start 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 -- 172.21.15.44:0/31236086 shutdown_connections 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 -- 172.21.15.44:0/31236086 >> 172.21.15.44:0/31236086 conn(0x7fd0b80b5890 msgr2=0x7fd0b80b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 -- 172.21.15.44:0/31236086 shutdown_connections 2024-07-17T07:12:07.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.501+0000 7fd0bcc00640 1 -- 172.21.15.44:0/31236086 wait complete. 2024-07-17T07:12:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.503+0000 7fd0bcc00640 1 Processor -- start 2024-07-17T07:12:07.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.503+0000 7fd0bcc00640 1 -- start start 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0b80cf3f0 0x7fd0b81f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0b8160390 0x7fd0b80ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b806ccb0 con 0x7fd098003b40 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b806cb30 con 0x7fd0b8160390 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0bcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b80afb00 con 0x7fd0b80cf3f0 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35072/0 (socket says 172.21.15.44:35072) 2024-07-17T07:12:07.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 -- 172.21.15.44:0/387093515 learned_addr learned my addr 172.21.15.44:0/387093515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 -- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0b80cf3f0 msgr2=0x7fd0b81f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0b80cf3f0 0x7fd0b81f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 -- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0b8160390 msgr2=0x7fd0b80ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0b8160390 0x7fd0b80ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.504+0000 7fd0b7400640 1 -- 172.21.15.44:0/387093515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b81f47c0 con 0x7fd098003b40 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0b7400640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009c70 tx=0x7fd0a8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8043070 con 0x7fd098003b40 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0a8004120 con 0x7fd098003b40 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8038040 con 0x7fd098003b40 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b81f4aa0 con 0x7fd098003b40 2024-07-17T07:12:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.505+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b815fb80 con 0x7fd098003b40 2024-07-17T07:12:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.507+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b80b0040 con 0x7fd098003b40 2024-07-17T07:12:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.508+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fd0a8046070 con 0x7fd098003b40 2024-07-17T07:12:07.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.509+0000 7fd0af400640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 0x7fd094079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.509+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fd0a80b8be0 con 0x7fd098003b40 2024-07-17T07:12:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.510+0000 7fd0b6a00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 0x7fd094079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.511+0000 7fd0b6a00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 0x7fd094079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0001c80 tx=0x7fd0a0001c10 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.513+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd0a80815b0 con 0x7fd098003b40 2024-07-17T07:12:07.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.724+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0b806fde0 con 0x7fd098003b40 2024-07-17T07:12:07.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.727+0000 7fd0af400640 1 -- 172.21.15.44:0/387093515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+17719 (secure 0 0 0) 0x7fd0a8085b00 con 0x7fd098003b40 2024-07-17T07:12:07.727 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:07.727 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":99,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:04.714050+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:01.050217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:02.180466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:07.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 msgr2=0x7fd094079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 0x7fd094079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0001c80 tx=0x7fd0a0001c10 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 msgr2=0x7fd0b815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009c70 tx=0x7fd0a8033670 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 shutdown_connections 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd094076cd0 0x7fd094079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0b8160390 0x7fd0b80ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0b80cf3f0 0x7fd0b81f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 --2- 172.21.15.44:0/387093515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd098003b40 0x7fd0b815e110 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.729+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 >> 172.21.15.44:0/387093515 conn(0x7fd0b80b5890 msgr2=0x7fd0b806f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.730+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 shutdown_connections 2024-07-17T07:12:07.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.730+0000 7fd0bcc00640 1 -- 172.21.15.44:0/387093515 wait complete. 2024-07-17T07:12:07.750 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:12:07.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:12:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 -- 172.21.15.44:0/3353914457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf920 msgr2=0x7fbf740ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 --2- 172.21.15.44:0/3353914457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf920 0x7fbf740ade20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbf7406f770 tx=0x7fbf6c02aa30 comp rx=0 tx=0).stop 2024-07-17T07:12:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 -- 172.21.15.44:0/3353914457 shutdown_connections 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 --2- 172.21.15.44:0/3353914457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf920 0x7fbf740ade20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 --2- 172.21.15.44:0/3353914457 >> v2:172.21.15.175:3300/0 conn(0x7fbf740b3db0 0x7fbf740ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 --2- 172.21.15.44:0/3353914457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf540 0x7fbf740b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 -- 172.21.15.44:0/3353914457 >> 172.21.15.44:0/3353914457 conn(0x7fbf740b0e80 msgr2=0x7fbf740b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 -- 172.21.15.44:0/3353914457 shutdown_connections 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.960+0000 7fbf79200640 1 -- 172.21.15.44:0/3353914457 wait complete. 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.962+0000 7fbf79200640 1 Processor -- start 2024-07-17T07:12:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.962+0000 7fbf79200640 1 -- start start 2024-07-17T07:12:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf740b3db0 0x7fbf741f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf920 0x7fbf74157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740ae6f0 con 0x7fbf740cf540 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740ae570 con 0x7fbf740b3db0 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf79200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740ae870 con 0x7fbf740cf920 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf6aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf6aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35084/0 (socket says 172.21.15.44:35084) 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.963+0000 7fbf6aa00640 1 -- 172.21.15.44:0/3927320348 learned_addr learned my addr 172.21.15.44:0/3927320348 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf73400640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf740b3db0 0x7fbf741f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf73e00640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf920 0x7fbf74157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 -- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf920 msgr2=0x7fbf74157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf920 0x7fbf74157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 -- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf740b3db0 msgr2=0x7fbf741f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf740b3db0 0x7fbf741f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 -- 172.21.15.44:0/3927320348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741f4c30 con 0x7fbf740cf540 2024-07-17T07:12:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6aa00640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c9a0 tx=0x7fbf5c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007bb0 con 0x7fbf740cf540 2024-07-17T07:12:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf5c007e40 con 0x7fbf740cf540 2024-07-17T07:12:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c005260 con 0x7fbf740cf540 2024-07-17T07:12:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.964+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf74155eb0 con 0x7fbf740cf540 2024-07-17T07:12:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.965+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf74156470 con 0x7fbf740cf540 2024-07-17T07:12:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.966+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf740b3870 con 0x7fbf740cf540 2024-07-17T07:12:07.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.967+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fbf5c005400 con 0x7fbf740cf540 2024-07-17T07:12:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.968+0000 7fbf6be00640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 0x7fbf50079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.968+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fbf5c098de0 con 0x7fbf740cf540 2024-07-17T07:12:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.968+0000 7fbf73400640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 0x7fbf50079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.969+0000 7fbf73400640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 0x7fbf50079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf741553c0 tx=0x7fbf64008040 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:07.972+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbf5c014030 con 0x7fbf740cf540 2024-07-17T07:12:08.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.171+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 --> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf741f16b0 con 0x7fbf50076cd0 2024-07-17T07:12:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.186+0000 7fbf6be00640 1 -- 172.21.15.44:0/3927320348 <== mgr.5426 v2:172.21.15.175:6800/2446491909 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7fbf741f16b0 con 0x7fbf50076cd0 2024-07-17T07:12:08.190 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:08.192 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:12:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 msgr2=0x7fbf50079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 0x7fbf50079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf741553c0 tx=0x7fbf64008040 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 msgr2=0x7fbf74157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c9a0 tx=0x7fbf5c00ce60 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 shutdown_connections 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fbf50076cd0 0x7fbf50079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf740cf920 0x7fbf74157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf740cf540 0x7fbf74157a50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 --2- 172.21.15.44:0/3927320348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf740b3db0 0x7fbf741f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 >> 172.21.15.44:0/3927320348 conn(0x7fbf740b0e80 msgr2=0x7fbf740aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 shutdown_connections 2024-07-17T07:12:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.193+0000 7fbf79200640 1 -- 172.21.15.44:0/3927320348 wait complete. 2024-07-17T07:12:08.210 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-07-17T07:12:06.358259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":217,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":343184,"kb_used_data":10448,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754631536,"statfs":{"total":773094113280,"available":772742692864,"internally_reserved":0,"allocated":10698752,"data_stored":6107736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.651320"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753722+0000","last_change":"2024-07-17T07:12:03.752325+0000","last_active":"2024-07-17T07:12:03.753722+0000","last_peered":"2024-07-17T07:12:03.753722+0000","last_clean":"2024-07-17T07:12:03.753722+0000","last_became_active":"2024-07-17T07:12:03.751976+0000","last_became_peered":"2024-07-17T07:12:03.751976+0000","last_unstale":"2024-07-17T07:12:03.753722+0000","last_undegraded":"2024-07-17T07:12:03.753722+0000","last_fullsized":"2024-07-17T07:12:03.753722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:14.921294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765957+0000","last_change":"2024-07-17T07:12:03.756771+0000","last_active":"2024-07-17T07:12:03.765957+0000","last_peered":"2024-07-17T07:12:03.765957+0000","last_clean":"2024-07-17T07:12:03.765957+0000","last_became_active":"2024-07-17T07:12:03.756256+0000","last_became_peered":"2024-07-17T07:12:03.756256+0000","last_unstale":"2024-07-17T07:12:03.765957+0000","last_undegraded":"2024-07-17T07:12:03.765957+0000","last_fullsized":"2024-07-17T07:12:03.765957+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:55.492395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.762200+0000","last_change":"2024-07-17T07:12:03.758185+0000","last_active":"2024-07-17T07:12:03.762200+0000","last_peered":"2024-07-17T07:12:03.762200+0000","last_clean":"2024-07-17T07:12:03.762200+0000","last_became_active":"2024-07-17T07:12:03.755490+0000","last_became_peered":"2024-07-17T07:12:03.755490+0000","last_unstale":"2024-07-17T07:12:03.762200+0000","last_undegraded":"2024-07-17T07:12:03.762200+0000","last_fullsized":"2024-07-17T07:12:03.762200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:30.344218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758797+0000","last_change":"2024-07-17T07:12:03.751187+0000","last_active":"2024-07-17T07:12:03.758797+0000","last_peered":"2024-07-17T07:12:03.758797+0000","last_clean":"2024-07-17T07:12:03.758797+0000","last_became_active":"2024-07-17T07:12:03.750582+0000","last_became_peered":"2024-07-17T07:12:03.750582+0000","last_unstale":"2024-07-17T07:12:03.758797+0000","last_undegraded":"2024-07-17T07:12:03.758797+0000","last_fullsized":"2024-07-17T07:12:03.758797+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:18.126146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754771+0000","last_change":"2024-07-17T07:12:03.747635+0000","last_active":"2024-07-17T07:12:03.754771+0000","last_peered":"2024-07-17T07:12:03.754771+0000","last_clean":"2024-07-17T07:12:03.754771+0000","last_became_active":"2024-07-17T07:12:03.746772+0000","last_became_peered":"2024-07-17T07:12:03.746772+0000","last_unstale":"2024-07-17T07:12:03.754771+0000","last_undegraded":"2024-07-17T07:12:03.754771+0000","last_fullsized":"2024-07-17T07:12:03.754771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:24:15.616673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767689+0000","last_change":"2024-07-17T07:12:03.756394+0000","last_active":"2024-07-17T07:12:03.767689+0000","last_peered":"2024-07-17T07:12:03.767689+0000","last_clean":"2024-07-17T07:12:03.767689+0000","last_became_active":"2024-07-17T07:12:03.756117+0000","last_became_peered":"2024-07-17T07:12:03.756117+0000","last_unstale":"2024-07-17T07:12:03.767689+0000","last_undegraded":"2024-07-17T07:12:03.767689+0000","last_fullsized":"2024-07-17T07:12:03.767689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:55.492395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.755259+0000","last_change":"2024-07-17T07:12:03.739416+0000","last_active":"2024-07-17T07:12:03.755259+0000","last_peered":"2024-07-17T07:12:03.755259+0000","last_clean":"2024-07-17T07:12:03.755259+0000","last_became_active":"2024-07-17T07:12:03.739078+0000","last_became_peered":"2024-07-17T07:12:03.739078+0000","last_unstale":"2024-07-17T07:12:03.755259+0000","last_undegraded":"2024-07-17T07:12:03.755259+0000","last_fullsized":"2024-07-17T07:12:03.755259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:05.825465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752953+0000","last_change":"2024-07-17T07:12:03.745184+0000","last_active":"2024-07-17T07:12:03.752953+0000","last_peered":"2024-07-17T07:12:03.752953+0000","last_clean":"2024-07-17T07:12:03.752953+0000","last_became_active":"2024-07-17T07:12:03.744366+0000","last_became_peered":"2024-07-17T07:12:03.744366+0000","last_unstale":"2024-07-17T07:12:03.752953+0000","last_undegraded":"2024-07-17T07:12:03.752953+0000","last_fullsized":"2024-07-17T07:12:03.752953+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.514527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759288+0000","last_change":"2024-07-17T07:12:03.756531+0000","last_active":"2024-07-17T07:12:03.759288+0000","last_peered":"2024-07-17T07:12:03.759288+0000","last_clean":"2024-07-17T07:12:03.759288+0000","last_became_active":"2024-07-17T07:12:03.754790+0000","last_became_peered":"2024-07-17T07:12:03.754790+0000","last_unstale":"2024-07-17T07:12:03.759288+0000","last_undegraded":"2024-07-17T07:12:03.759288+0000","last_fullsized":"2024-07-17T07:12:03.759288+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:25:26.406872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759939+0000","last_change":"2024-07-17T07:12:03.759106+0000","last_active":"2024-07-17T07:12:03.759939+0000","last_peered":"2024-07-17T07:12:03.759939+0000","last_clean":"2024-07-17T07:12:03.759939+0000","last_became_active":"2024-07-17T07:12:03.758020+0000","last_became_peered":"2024-07-17T07:12:03.758020+0000","last_unstale":"2024-07-17T07:12:03.759939+0000","last_undegraded":"2024-07-17T07:12:03.759939+0000","last_fullsized":"2024-07-17T07:12:03.759939+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:27.883450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764558+0000","last_change":"2024-07-17T07:12:03.754271+0000","last_active":"2024-07-17T07:12:03.764558+0000","last_peered":"2024-07-17T07:12:03.764558+0000","last_clean":"2024-07-17T07:12:03.764558+0000","last_became_active":"2024-07-17T07:12:03.754012+0000","last_became_peered":"2024-07-17T07:12:03.754012+0000","last_unstale":"2024-07-17T07:12:03.764558+0000","last_undegraded":"2024-07-17T07:12:03.764558+0000","last_fullsized":"2024-07-17T07:12:03.764558+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.541771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.768840+0000","last_change":"2024-07-17T07:12:03.767659+0000","last_active":"2024-07-17T07:12:03.768840+0000","last_peered":"2024-07-17T07:12:03.768840+0000","last_clean":"2024-07-17T07:12:03.768840+0000","last_became_active":"2024-07-17T07:12:03.767335+0000","last_became_peered":"2024-07-17T07:12:03.767335+0000","last_unstale":"2024-07-17T07:12:03.768840+0000","last_undegraded":"2024-07-17T07:12:03.768840+0000","last_fullsized":"2024-07-17T07:12:03.768840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:41.604245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757753+0000","last_change":"2024-07-17T07:12:03.751897+0000","last_active":"2024-07-17T07:12:03.757753+0000","last_peered":"2024-07-17T07:12:03.757753+0000","last_clean":"2024-07-17T07:12:03.757753+0000","last_became_active":"2024-07-17T07:12:03.751604+0000","last_became_peered":"2024-07-17T07:12:03.751604+0000","last_unstale":"2024-07-17T07:12:03.757753+0000","last_undegraded":"2024-07-17T07:12:03.757753+0000","last_fullsized":"2024-07-17T07:12:03.757753+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:59.499759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752394+0000","last_change":"2024-07-17T07:12:03.751143+0000","last_active":"2024-07-17T07:12:03.752394+0000","last_peered":"2024-07-17T07:12:03.752394+0000","last_clean":"2024-07-17T07:12:03.752394+0000","last_became_active":"2024-07-17T07:12:03.750802+0000","last_became_peered":"2024-07-17T07:12:03.750802+0000","last_unstale":"2024-07-17T07:12:03.752394+0000","last_undegraded":"2024-07-17T07:12:03.752394+0000","last_fullsized":"2024-07-17T07:12:03.752394+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:17.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764458+0000","last_change":"2024-07-17T07:12:03.754003+0000","last_active":"2024-07-17T07:12:03.764458+0000","last_peered":"2024-07-17T07:12:03.764458+0000","last_clean":"2024-07-17T07:12:03.764458+0000","last_became_active":"2024-07-17T07:12:03.753681+0000","last_became_peered":"2024-07-17T07:12:03.753681+0000","last_unstale":"2024-07-17T07:12:03.764458+0000","last_undegraded":"2024-07-17T07:12:03.764458+0000","last_fullsized":"2024-07-17T07:12:03.764458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:18.199015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.751402+0000","last_change":"2024-07-17T07:12:03.747788+0000","last_active":"2024-07-17T07:12:03.751402+0000","last_peered":"2024-07-17T07:12:03.751402+0000","last_clean":"2024-07-17T07:12:03.751402+0000","last_became_active":"2024-07-17T07:12:03.747459+0000","last_became_peered":"2024-07-17T07:12:03.747459+0000","last_unstale":"2024-07-17T07:12:03.751402+0000","last_undegraded":"2024-07-17T07:12:03.751402+0000","last_fullsized":"2024-07-17T07:12:03.751402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:27.795360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.750030+0000","last_change":"2024-07-17T07:12:03.736624+0000","last_active":"2024-07-17T07:12:03.750030+0000","last_peered":"2024-07-17T07:12:03.750030+0000","last_clean":"2024-07-17T07:12:03.750030+0000","last_became_active":"2024-07-17T07:12:03.735744+0000","last_became_peered":"2024-07-17T07:12:03.735744+0000","last_unstale":"2024-07-17T07:12:03.750030+0000","last_undegraded":"2024-07-17T07:12:03.750030+0000","last_fullsized":"2024-07-17T07:12:03.750030+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:40.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760792+0000","last_change":"2024-07-17T07:12:03.755113+0000","last_active":"2024-07-17T07:12:03.760792+0000","last_peered":"2024-07-17T07:12:03.760792+0000","last_clean":"2024-07-17T07:12:03.760792+0000","last_became_active":"2024-07-17T07:12:03.754809+0000","last_became_peered":"2024-07-17T07:12:03.754809+0000","last_unstale":"2024-07-17T07:12:03.760792+0000","last_undegraded":"2024-07-17T07:12:03.760792+0000","last_fullsized":"2024-07-17T07:12:03.760792+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:06.131252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767400+0000","last_change":"2024-07-17T07:12:03.759957+0000","last_active":"2024-07-17T07:12:03.767400+0000","last_peered":"2024-07-17T07:12:03.767400+0000","last_clean":"2024-07-17T07:12:03.767400+0000","last_became_active":"2024-07-17T07:12:03.759613+0000","last_became_peered":"2024-07-17T07:12:03.759613+0000","last_unstale":"2024-07-17T07:12:03.767400+0000","last_undegraded":"2024-07-17T07:12:03.767400+0000","last_fullsized":"2024-07-17T07:12:03.767400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:35.574322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754339+0000","last_change":"2024-07-17T07:12:03.747767+0000","last_active":"2024-07-17T07:12:03.754339+0000","last_peered":"2024-07-17T07:12:03.754339+0000","last_clean":"2024-07-17T07:12:03.754339+0000","last_became_active":"2024-07-17T07:12:03.747466+0000","last_became_peered":"2024-07-17T07:12:03.747466+0000","last_unstale":"2024-07-17T07:12:03.754339+0000","last_undegraded":"2024-07-17T07:12:03.754339+0000","last_fullsized":"2024-07-17T07:12:03.754339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:34.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759385+0000","last_change":"2024-07-17T07:12:03.756678+0000","last_active":"2024-07-17T07:12:03.759385+0000","last_peered":"2024-07-17T07:12:03.759385+0000","last_clean":"2024-07-17T07:12:03.759385+0000","last_became_active":"2024-07-17T07:12:03.753805+0000","last_became_peered":"2024-07-17T07:12:03.753805+0000","last_unstale":"2024-07-17T07:12:03.759385+0000","last_undegraded":"2024-07-17T07:12:03.759385+0000","last_fullsized":"2024-07-17T07:12:03.759385+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:01.197459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758103+0000","last_change":"2024-07-17T07:12:03.749590+0000","last_active":"2024-07-17T07:12:03.758103+0000","last_peered":"2024-07-17T07:12:03.758103+0000","last_clean":"2024-07-17T07:12:03.758103+0000","last_became_active":"2024-07-17T07:12:03.748652+0000","last_became_peered":"2024-07-17T07:12:03.748652+0000","last_unstale":"2024-07-17T07:12:03.758103+0000","last_undegraded":"2024-07-17T07:12:03.758103+0000","last_fullsized":"2024-07-17T07:12:03.758103+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:27.854224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754349+0000","last_change":"2024-07-17T07:12:03.748263+0000","last_active":"2024-07-17T07:12:03.754349+0000","last_peered":"2024-07-17T07:12:03.754349+0000","last_clean":"2024-07-17T07:12:03.754349+0000","last_became_active":"2024-07-17T07:12:03.747971+0000","last_became_peered":"2024-07-17T07:12:03.747971+0000","last_unstale":"2024-07-17T07:12:03.754349+0000","last_undegraded":"2024-07-17T07:12:03.754349+0000","last_fullsized":"2024-07-17T07:12:03.754349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:27.702914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752278+0000","last_change":"2024-07-17T07:12:03.741026+0000","last_active":"2024-07-17T07:12:03.752278+0000","last_peered":"2024-07-17T07:12:03.752278+0000","last_clean":"2024-07-17T07:12:03.752278+0000","last_became_active":"2024-07-17T07:12:03.740742+0000","last_became_peered":"2024-07-17T07:12:03.740742+0000","last_unstale":"2024-07-17T07:12:03.752278+0000","last_undegraded":"2024-07-17T07:12:03.752278+0000","last_fullsized":"2024-07-17T07:12:03.752278+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:04.812076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753606+0000","last_change":"2024-07-17T07:12:03.747269+0000","last_active":"2024-07-17T07:12:03.753606+0000","last_peered":"2024-07-17T07:12:03.753606+0000","last_clean":"2024-07-17T07:12:03.753606+0000","last_became_active":"2024-07-17T07:12:03.746159+0000","last_became_peered":"2024-07-17T07:12:03.746159+0000","last_unstale":"2024-07-17T07:12:03.753606+0000","last_undegraded":"2024-07-17T07:12:03.753606+0000","last_fullsized":"2024-07-17T07:12:03.753606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:22.127204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761898+0000","last_change":"2024-07-17T07:12:03.761146+0000","last_active":"2024-07-17T07:12:03.761898+0000","last_peered":"2024-07-17T07:12:03.761898+0000","last_clean":"2024-07-17T07:12:03.761898+0000","last_became_active":"2024-07-17T07:12:03.760868+0000","last_became_peered":"2024-07-17T07:12:03.760868+0000","last_unstale":"2024-07-17T07:12:03.761898+0000","last_undegraded":"2024-07-17T07:12:03.761898+0000","last_fullsized":"2024-07-17T07:12:03.761898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:13.026975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.751440+0000","last_change":"2024-07-17T07:12:03.748832+0000","last_active":"2024-07-17T07:12:03.751440+0000","last_peered":"2024-07-17T07:12:03.751440+0000","last_clean":"2024-07-17T07:12:03.751440+0000","last_became_active":"2024-07-17T07:12:03.748516+0000","last_became_peered":"2024-07-17T07:12:03.748516+0000","last_unstale":"2024-07-17T07:12:03.751440+0000","last_undegraded":"2024-07-17T07:12:03.751440+0000","last_fullsized":"2024-07-17T07:12:03.751440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:51.597102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757619+0000","last_change":"2024-07-17T07:12:03.751861+0000","last_active":"2024-07-17T07:12:03.757619+0000","last_peered":"2024-07-17T07:12:03.757619+0000","last_clean":"2024-07-17T07:12:03.757619+0000","last_became_active":"2024-07-17T07:12:03.751570+0000","last_became_peered":"2024-07-17T07:12:03.751570+0000","last_unstale":"2024-07-17T07:12:03.757619+0000","last_undegraded":"2024-07-17T07:12:03.757619+0000","last_fullsized":"2024-07-17T07:12:03.757619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:05:51.603804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.766540+0000","last_change":"2024-07-17T07:12:03.757515+0000","last_active":"2024-07-17T07:12:03.766540+0000","last_peered":"2024-07-17T07:12:03.766540+0000","last_clean":"2024-07-17T07:12:03.766540+0000","last_became_active":"2024-07-17T07:12:03.757216+0000","last_became_peered":"2024-07-17T07:12:03.757216+0000","last_unstale":"2024-07-17T07:12:03.766540+0000","last_undegraded":"2024-07-17T07:12:03.766540+0000","last_fullsized":"2024-07-17T07:12:03.766540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:05.861721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.766448+0000","last_change":"2024-07-17T07:12:03.765862+0000","last_active":"2024-07-17T07:12:03.766448+0000","last_peered":"2024-07-17T07:12:03.766448+0000","last_clean":"2024-07-17T07:12:03.766448+0000","last_became_active":"2024-07-17T07:12:03.765573+0000","last_became_peered":"2024-07-17T07:12:03.765573+0000","last_unstale":"2024-07-17T07:12:03.766448+0000","last_undegraded":"2024-07-17T07:12:03.766448+0000","last_fullsized":"2024-07-17T07:12:03.766448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:50.400417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761014+0000","last_change":"2024-07-17T07:12:03.757289+0000","last_active":"2024-07-17T07:12:03.761014+0000","last_peered":"2024-07-17T07:12:03.761014+0000","last_clean":"2024-07-17T07:12:03.761014+0000","last_became_active":"2024-07-17T07:12:03.757022+0000","last_became_peered":"2024-07-17T07:12:03.757022+0000","last_unstale":"2024-07-17T07:12:03.761014+0000","last_undegraded":"2024-07-17T07:12:03.761014+0000","last_fullsized":"2024-07-17T07:12:03.761014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:09:21.762609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759240+0000","last_change":"2024-07-17T07:12:03.755919+0000","last_active":"2024-07-17T07:12:03.759240+0000","last_peered":"2024-07-17T07:12:03.759240+0000","last_clean":"2024-07-17T07:12:03.759240+0000","last_became_active":"2024-07-17T07:12:03.755612+0000","last_became_peered":"2024-07-17T07:12:03.755612+0000","last_unstale":"2024-07-17T07:12:03.759240+0000","last_undegraded":"2024-07-17T07:12:03.759240+0000","last_fullsized":"2024-07-17T07:12:03.759240+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:07.976522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754734+0000","last_change":"2024-07-17T07:12:03.753865+0000","last_active":"2024-07-17T07:12:03.754734+0000","last_peered":"2024-07-17T07:12:03.754734+0000","last_clean":"2024-07-17T07:12:03.754734+0000","last_became_active":"2024-07-17T07:12:03.753581+0000","last_became_peered":"2024-07-17T07:12:03.753581+0000","last_unstale":"2024-07-17T07:12:03.754734+0000","last_undegraded":"2024-07-17T07:12:03.754734+0000","last_fullsized":"2024-07-17T07:12:03.754734+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:44.502547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776376+0000","last_change":"2024-07-17T07:12:02.747570+0000","last_active":"2024-07-17T07:12:03.776376+0000","last_peered":"2024-07-17T07:12:03.776376+0000","last_clean":"2024-07-17T07:12:03.776376+0000","last_became_active":"2024-07-17T07:12:02.746208+0000","last_became_peered":"2024-07-17T07:12:02.746208+0000","last_unstale":"2024-07-17T07:12:03.776376+0000","last_undegraded":"2024-07-17T07:12:03.776376+0000","last_fullsized":"2024-07-17T07:12:03.776376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:45.695793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764435+0000","last_change":"2024-07-17T07:12:03.753321+0000","last_active":"2024-07-17T07:12:03.764435+0000","last_peered":"2024-07-17T07:12:03.764435+0000","last_clean":"2024-07-17T07:12:03.764435+0000","last_became_active":"2024-07-17T07:12:03.750091+0000","last_became_peered":"2024-07-17T07:12:03.750091+0000","last_unstale":"2024-07-17T07:12:03.764435+0000","last_undegraded":"2024-07-17T07:12:03.764435+0000","last_fullsized":"2024-07-17T07:12:03.764435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:26.296929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775907+0000","last_change":"2024-07-17T07:12:02.756099+0000","last_active":"2024-07-17T07:12:03.775907+0000","last_peered":"2024-07-17T07:12:03.775907+0000","last_clean":"2024-07-17T07:12:03.775907+0000","last_became_active":"2024-07-17T07:12:02.755598+0000","last_became_peered":"2024-07-17T07:12:02.755598+0000","last_unstale":"2024-07-17T07:12:03.775907+0000","last_undegraded":"2024-07-17T07:12:03.775907+0000","last_fullsized":"2024-07-17T07:12:03.775907+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:39:56.012212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753623+0000","last_change":"2024-07-17T07:12:03.749872+0000","last_active":"2024-07-17T07:12:03.753623+0000","last_peered":"2024-07-17T07:12:03.753623+0000","last_clean":"2024-07-17T07:12:03.753623+0000","last_became_active":"2024-07-17T07:12:03.748947+0000","last_became_peered":"2024-07-17T07:12:03.748947+0000","last_unstale":"2024-07-17T07:12:03.753623+0000","last_undegraded":"2024-07-17T07:12:03.753623+0000","last_fullsized":"2024-07-17T07:12:03.753623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:03.438210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776872+0000","last_change":"2024-07-17T07:12:02.758649+0000","last_active":"2024-07-17T07:12:03.776872+0000","last_peered":"2024-07-17T07:12:03.776872+0000","last_clean":"2024-07-17T07:12:03.776872+0000","last_became_active":"2024-07-17T07:12:02.757979+0000","last_became_peered":"2024-07-17T07:12:02.757979+0000","last_unstale":"2024-07-17T07:12:03.776872+0000","last_undegraded":"2024-07-17T07:12:03.776872+0000","last_fullsized":"2024-07-17T07:12:03.776872+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:47:18.472480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759412+0000","last_change":"2024-07-17T07:12:03.756943+0000","last_active":"2024-07-17T07:12:03.759412+0000","last_peered":"2024-07-17T07:12:03.759412+0000","last_clean":"2024-07-17T07:12:03.759412+0000","last_became_active":"2024-07-17T07:12:03.754189+0000","last_became_peered":"2024-07-17T07:12:03.754189+0000","last_unstale":"2024-07-17T07:12:03.759412+0000","last_undegraded":"2024-07-17T07:12:03.759412+0000","last_fullsized":"2024-07-17T07:12:03.759412+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:43.529279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.771843+0000","last_change":"2024-07-17T07:12:02.754109+0000","last_active":"2024-07-17T07:12:03.771843+0000","last_peered":"2024-07-17T07:12:03.771843+0000","last_clean":"2024-07-17T07:12:03.771843+0000","last_became_active":"2024-07-17T07:12:02.753794+0000","last_became_peered":"2024-07-17T07:12:02.753794+0000","last_unstale":"2024-07-17T07:12:03.771843+0000","last_undegraded":"2024-07-17T07:12:03.771843+0000","last_fullsized":"2024-07-17T07:12:03.771843+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:31.365177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765704+0000","last_change":"2024-07-17T07:12:03.759887+0000","last_active":"2024-07-17T07:12:03.765704+0000","last_peered":"2024-07-17T07:12:03.765704+0000","last_clean":"2024-07-17T07:12:03.765704+0000","last_became_active":"2024-07-17T07:12:03.759613+0000","last_became_peered":"2024-07-17T07:12:03.759613+0000","last_unstale":"2024-07-17T07:12:03.765704+0000","last_undegraded":"2024-07-17T07:12:03.765704+0000","last_fullsized":"2024-07-17T07:12:03.765704+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:49.240230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.772294+0000","last_change":"2024-07-17T07:12:02.754014+0000","last_active":"2024-07-17T07:12:03.772294+0000","last_peered":"2024-07-17T07:12:03.772294+0000","last_clean":"2024-07-17T07:12:03.772294+0000","last_became_active":"2024-07-17T07:12:02.753724+0000","last_became_peered":"2024-07-17T07:12:02.753724+0000","last_unstale":"2024-07-17T07:12:03.772294+0000","last_undegraded":"2024-07-17T07:12:03.772294+0000","last_fullsized":"2024-07-17T07:12:03.772294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:09.863023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764707+0000","last_change":"2024-07-17T07:12:03.755277+0000","last_active":"2024-07-17T07:12:03.764707+0000","last_peered":"2024-07-17T07:12:03.764707+0000","last_clean":"2024-07-17T07:12:03.764707+0000","last_became_active":"2024-07-17T07:12:03.755003+0000","last_became_peered":"2024-07-17T07:12:03.755003+0000","last_unstale":"2024-07-17T07:12:03.764707+0000","last_undegraded":"2024-07-17T07:12:03.764707+0000","last_fullsized":"2024-07-17T07:12:03.764707+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:57:29.929560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.772724+0000","last_change":"2024-07-17T07:12:02.756904+0000","last_active":"2024-07-17T07:12:03.772724+0000","last_peered":"2024-07-17T07:12:03.772724+0000","last_clean":"2024-07-17T07:12:03.772724+0000","last_became_active":"2024-07-17T07:12:02.756607+0000","last_became_peered":"2024-07-17T07:12:02.756607+0000","last_unstale":"2024-07-17T07:12:03.772724+0000","last_undegraded":"2024-07-17T07:12:03.772724+0000","last_fullsized":"2024-07-17T07:12:03.772724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:50.470114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759445+0000","last_change":"2024-07-17T07:12:03.757296+0000","last_active":"2024-07-17T07:12:03.759445+0000","last_peered":"2024-07-17T07:12:03.759445+0000","last_clean":"2024-07-17T07:12:03.759445+0000","last_became_active":"2024-07-17T07:12:03.753152+0000","last_became_peered":"2024-07-17T07:12:03.753152+0000","last_unstale":"2024-07-17T07:12:03.759445+0000","last_undegraded":"2024-07-17T07:12:03.759445+0000","last_fullsized":"2024-07-17T07:12:03.759445+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:39:44.524648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.732640+0000","last_change":"2024-07-17T07:12:02.754825+0000","last_active":"2024-07-17T07:12:03.732640+0000","last_peered":"2024-07-17T07:12:03.732640+0000","last_clean":"2024-07-17T07:12:03.732640+0000","last_became_active":"2024-07-17T07:12:02.754564+0000","last_became_peered":"2024-07-17T07:12:02.754564+0000","last_unstale":"2024-07-17T07:12:03.732640+0000","last_undegraded":"2024-07-17T07:12:03.732640+0000","last_fullsized":"2024-07-17T07:12:03.732640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:40.586281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759572+0000","last_change":"2024-07-17T07:12:03.754368+0000","last_active":"2024-07-17T07:12:03.759572+0000","last_peered":"2024-07-17T07:12:03.759572+0000","last_clean":"2024-07-17T07:12:03.759572+0000","last_became_active":"2024-07-17T07:12:03.749198+0000","last_became_peered":"2024-07-17T07:12:03.749198+0000","last_unstale":"2024-07-17T07:12:03.759572+0000","last_undegraded":"2024-07-17T07:12:03.759572+0000","last_fullsized":"2024-07-17T07:12:03.759572+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:58.850482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.718722+0000","last_change":"2024-07-17T07:12:02.748339+0000","last_active":"2024-07-17T07:12:03.718722+0000","last_peered":"2024-07-17T07:12:03.718722+0000","last_clean":"2024-07-17T07:12:03.718722+0000","last_became_active":"2024-07-17T07:12:02.747964+0000","last_became_peered":"2024-07-17T07:12:02.747964+0000","last_unstale":"2024-07-17T07:12:03.718722+0000","last_undegraded":"2024-07-17T07:12:03.718722+0000","last_fullsized":"2024-07-17T07:12:03.718722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:39.358316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765312+0000","last_change":"2024-07-17T07:12:03.756583+0000","last_active":"2024-07-17T07:12:03.765312+0000","last_peered":"2024-07-17T07:12:03.765312+0000","last_clean":"2024-07-17T07:12:03.765312+0000","last_became_active":"2024-07-17T07:12:03.749888+0000","last_became_peered":"2024-07-17T07:12:03.749888+0000","last_unstale":"2024-07-17T07:12:03.765312+0000","last_undegraded":"2024-07-17T07:12:03.765312+0000","last_fullsized":"2024-07-17T07:12:03.765312+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:39.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.739518+0000","last_change":"2024-07-17T07:12:02.761855+0000","last_active":"2024-07-17T07:12:03.739518+0000","last_peered":"2024-07-17T07:12:03.739518+0000","last_clean":"2024-07-17T07:12:03.739518+0000","last_became_active":"2024-07-17T07:12:02.761512+0000","last_became_peered":"2024-07-17T07:12:02.761512+0000","last_unstale":"2024-07-17T07:12:03.739518+0000","last_undegraded":"2024-07-17T07:12:03.739518+0000","last_fullsized":"2024-07-17T07:12:03.739518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:30.040092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758922+0000","last_change":"2024-07-17T07:12:03.751127+0000","last_active":"2024-07-17T07:12:03.758922+0000","last_peered":"2024-07-17T07:12:03.758922+0000","last_clean":"2024-07-17T07:12:03.758922+0000","last_became_active":"2024-07-17T07:12:03.750794+0000","last_became_peered":"2024-07-17T07:12:03.750794+0000","last_unstale":"2024-07-17T07:12:03.758922+0000","last_undegraded":"2024-07-17T07:12:03.758922+0000","last_fullsized":"2024-07-17T07:12:03.758922+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:37.085417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.746804+0000","last_change":"2024-07-17T07:12:02.753713+0000","last_active":"2024-07-17T07:12:03.746804+0000","last_peered":"2024-07-17T07:12:03.746804+0000","last_clean":"2024-07-17T07:12:03.746804+0000","last_became_active":"2024-07-17T07:12:02.753414+0000","last_became_peered":"2024-07-17T07:12:02.753414+0000","last_unstale":"2024-07-17T07:12:03.746804+0000","last_undegraded":"2024-07-17T07:12:03.746804+0000","last_fullsized":"2024-07-17T07:12:03.746804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.634672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753092+0000","last_change":"2024-07-17T07:12:03.751842+0000","last_active":"2024-07-17T07:12:03.753092+0000","last_peered":"2024-07-17T07:12:03.753092+0000","last_clean":"2024-07-17T07:12:03.753092+0000","last_became_active":"2024-07-17T07:12:03.751569+0000","last_became_peered":"2024-07-17T07:12:03.751569+0000","last_unstale":"2024-07-17T07:12:03.753092+0000","last_undegraded":"2024-07-17T07:12:03.753092+0000","last_fullsized":"2024-07-17T07:12:03.753092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:39.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775559+0000","last_change":"2024-07-17T07:12:02.752955+0000","last_active":"2024-07-17T07:12:03.775559+0000","last_peered":"2024-07-17T07:12:03.775559+0000","last_clean":"2024-07-17T07:12:03.775559+0000","last_became_active":"2024-07-17T07:12:02.752633+0000","last_became_peered":"2024-07-17T07:12:02.752633+0000","last_unstale":"2024-07-17T07:12:03.775559+0000","last_undegraded":"2024-07-17T07:12:03.775559+0000","last_fullsized":"2024-07-17T07:12:03.775559+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:17:49.795454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.773241+0000","last_change":"2024-07-17T07:12:03.769253+0000","last_active":"2024-07-17T07:12:03.773241+0000","last_peered":"2024-07-17T07:12:03.773241+0000","last_clean":"2024-07-17T07:12:03.773241+0000","last_became_active":"2024-07-17T07:12:03.768014+0000","last_became_peered":"2024-07-17T07:12:03.768014+0000","last_unstale":"2024-07-17T07:12:03.773241+0000","last_undegraded":"2024-07-17T07:12:03.773241+0000","last_fullsized":"2024-07-17T07:12:03.773241+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:52.284911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776344+0000","last_change":"2024-07-17T07:12:02.756294+0000","last_active":"2024-07-17T07:12:03.776344+0000","last_peered":"2024-07-17T07:12:03.776344+0000","last_clean":"2024-07-17T07:12:03.776344+0000","last_became_active":"2024-07-17T07:12:02.755998+0000","last_became_peered":"2024-07-17T07:12:02.755998+0000","last_unstale":"2024-07-17T07:12:03.776344+0000","last_undegraded":"2024-07-17T07:12:03.776344+0000","last_fullsized":"2024-07-17T07:12:03.776344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:51.266946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761894+0000","last_change":"2024-07-17T07:12:03.761295+0000","last_active":"2024-07-17T07:12:03.761894+0000","last_peered":"2024-07-17T07:12:03.761894+0000","last_clean":"2024-07-17T07:12:03.761894+0000","last_became_active":"2024-07-17T07:12:03.761010+0000","last_became_peered":"2024-07-17T07:12:03.761010+0000","last_unstale":"2024-07-17T07:12:03.761894+0000","last_undegraded":"2024-07-17T07:12:03.761894+0000","last_fullsized":"2024-07-17T07:12:03.761894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:10.692235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.773473+0000","last_change":"2024-07-17T07:12:02.751897+0000","last_active":"2024-07-17T07:12:03.773473+0000","last_peered":"2024-07-17T07:12:03.773473+0000","last_clean":"2024-07-17T07:12:03.773473+0000","last_became_active":"2024-07-17T07:12:02.751577+0000","last_became_peered":"2024-07-17T07:12:02.751577+0000","last_unstale":"2024-07-17T07:12:03.773473+0000","last_undegraded":"2024-07-17T07:12:03.773473+0000","last_fullsized":"2024-07-17T07:12:03.773473+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:46.837660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.756509+0000","last_change":"2024-07-17T07:12:03.744689+0000","last_active":"2024-07-17T07:12:03.756509+0000","last_peered":"2024-07-17T07:12:03.756509+0000","last_clean":"2024-07-17T07:12:03.756509+0000","last_became_active":"2024-07-17T07:12:03.744385+0000","last_became_peered":"2024-07-17T07:12:03.744385+0000","last_unstale":"2024-07-17T07:12:03.756509+0000","last_undegraded":"2024-07-17T07:12:03.756509+0000","last_fullsized":"2024-07-17T07:12:03.756509+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:27.336360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.737259+0000","last_change":"2024-07-17T07:12:02.755014+0000","last_active":"2024-07-17T07:12:03.737259+0000","last_peered":"2024-07-17T07:12:03.737259+0000","last_clean":"2024-07-17T07:12:03.737259+0000","last_became_active":"2024-07-17T07:12:02.754542+0000","last_became_peered":"2024-07-17T07:12:02.754542+0000","last_unstale":"2024-07-17T07:12:03.737259+0000","last_undegraded":"2024-07-17T07:12:03.737259+0000","last_fullsized":"2024-07-17T07:12:03.737259+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:57.205689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758324+0000","last_change":"2024-07-17T07:12:03.751977+0000","last_active":"2024-07-17T07:12:03.758324+0000","last_peered":"2024-07-17T07:12:03.758324+0000","last_clean":"2024-07-17T07:12:03.758324+0000","last_became_active":"2024-07-17T07:12:03.750794+0000","last_became_peered":"2024-07-17T07:12:03.750794+0000","last_unstale":"2024-07-17T07:12:03.758324+0000","last_undegraded":"2024-07-17T07:12:03.758324+0000","last_fullsized":"2024-07-17T07:12:03.758324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:59:26.669466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.719283+0000","last_change":"2024-07-17T07:12:02.739956+0000","last_active":"2024-07-17T07:12:03.719283+0000","last_peered":"2024-07-17T07:12:03.719283+0000","last_clean":"2024-07-17T07:12:03.719283+0000","last_became_active":"2024-07-17T07:12:02.739666+0000","last_became_peered":"2024-07-17T07:12:02.739666+0000","last_unstale":"2024-07-17T07:12:03.719283+0000","last_undegraded":"2024-07-17T07:12:03.719283+0000","last_fullsized":"2024-07-17T07:12:03.719283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:26.113215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759466+0000","last_change":"2024-07-17T07:12:03.753038+0000","last_active":"2024-07-17T07:12:03.759466+0000","last_peered":"2024-07-17T07:12:03.759466+0000","last_clean":"2024-07-17T07:12:03.759466+0000","last_became_active":"2024-07-17T07:12:03.749591+0000","last_became_peered":"2024-07-17T07:12:03.749591+0000","last_unstale":"2024-07-17T07:12:03.759466+0000","last_undegraded":"2024-07-17T07:12:03.759466+0000","last_fullsized":"2024-07-17T07:12:03.759466+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:09.095176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.777398+0000","last_change":"2024-07-17T07:12:02.756834+0000","last_active":"2024-07-17T07:12:03.777398+0000","last_peered":"2024-07-17T07:12:03.777398+0000","last_clean":"2024-07-17T07:12:03.777398+0000","last_became_active":"2024-07-17T07:12:02.756239+0000","last_became_peered":"2024-07-17T07:12:02.756239+0000","last_unstale":"2024-07-17T07:12:03.777398+0000","last_undegraded":"2024-07-17T07:12:03.777398+0000","last_fullsized":"2024-07-17T07:12:03.777398+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:09.674270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757952+0000","last_change":"2024-07-17T07:12:03.757078+0000","last_active":"2024-07-17T07:12:03.757952+0000","last_peered":"2024-07-17T07:12:03.757952+0000","last_clean":"2024-07-17T07:12:03.757952+0000","last_became_active":"2024-07-17T07:12:03.755774+0000","last_became_peered":"2024-07-17T07:12:03.755774+0000","last_unstale":"2024-07-17T07:12:03.757952+0000","last_undegraded":"2024-07-17T07:12:03.757952+0000","last_fullsized":"2024-07-17T07:12:03.757952+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:05.861721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775653+0000","last_change":"2024-07-17T07:12:02.760505+0000","last_active":"2024-07-17T07:12:03.775653+0000","last_peered":"2024-07-17T07:12:03.775653+0000","last_clean":"2024-07-17T07:12:03.775653+0000","last_became_active":"2024-07-17T07:12:02.760211+0000","last_became_peered":"2024-07-17T07:12:02.760211+0000","last_unstale":"2024-07-17T07:12:03.775653+0000","last_undegraded":"2024-07-17T07:12:03.775653+0000","last_fullsized":"2024-07-17T07:12:03.775653+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:32.932164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738118+0000","last_change":"2024-07-17T07:12:03.732313+0000","last_active":"2024-07-17T07:12:03.738118+0000","last_peered":"2024-07-17T07:12:03.738118+0000","last_clean":"2024-07-17T07:12:03.738118+0000","last_became_active":"2024-07-17T07:12:03.731736+0000","last_became_peered":"2024-07-17T07:12:03.731736+0000","last_unstale":"2024-07-17T07:12:03.738118+0000","last_undegraded":"2024-07-17T07:12:03.738118+0000","last_fullsized":"2024-07-17T07:12:03.738118+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:36:30.357739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775474+0000","last_change":"2024-07-17T07:12:02.758542+0000","last_active":"2024-07-17T07:12:03.775474+0000","last_peered":"2024-07-17T07:12:03.775474+0000","last_clean":"2024-07-17T07:12:03.775474+0000","last_became_active":"2024-07-17T07:12:02.757350+0000","last_became_peered":"2024-07-17T07:12:02.757350+0000","last_unstale":"2024-07-17T07:12:03.775474+0000","last_undegraded":"2024-07-17T07:12:03.775474+0000","last_fullsized":"2024-07-17T07:12:03.775474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:09:32.511979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.762095+0000","last_change":"2024-07-17T07:12:03.760561+0000","last_active":"2024-07-17T07:12:03.762095+0000","last_peered":"2024-07-17T07:12:03.762095+0000","last_clean":"2024-07-17T07:12:03.762095+0000","last_became_active":"2024-07-17T07:12:03.760221+0000","last_became_peered":"2024-07-17T07:12:03.760221+0000","last_unstale":"2024-07-17T07:12:03.762095+0000","last_undegraded":"2024-07-17T07:12:03.762095+0000","last_fullsized":"2024-07-17T07:12:03.762095+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:42.932470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.774300+0000","last_change":"2024-07-17T07:12:02.745063+0000","last_active":"2024-07-17T07:12:03.774300+0000","last_peered":"2024-07-17T07:12:03.774300+0000","last_clean":"2024-07-17T07:12:03.774300+0000","last_became_active":"2024-07-17T07:12:02.744788+0000","last_became_peered":"2024-07-17T07:12:02.744788+0000","last_unstale":"2024-07-17T07:12:03.774300+0000","last_undegraded":"2024-07-17T07:12:03.774300+0000","last_fullsized":"2024-07-17T07:12:03.774300+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:57:27.807862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767558+0000","last_change":"2024-07-17T07:12:03.760850+0000","last_active":"2024-07-17T07:12:03.767558+0000","last_peered":"2024-07-17T07:12:03.767558+0000","last_clean":"2024-07-17T07:12:03.767558+0000","last_became_active":"2024-07-17T07:12:03.760513+0000","last_became_peered":"2024-07-17T07:12:03.760513+0000","last_unstale":"2024-07-17T07:12:03.767558+0000","last_undegraded":"2024-07-17T07:12:03.767558+0000","last_fullsized":"2024-07-17T07:12:03.767558+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:49.240230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738228+0000","last_change":"2024-07-17T07:12:02.756896+0000","last_active":"2024-07-17T07:12:03.738228+0000","last_peered":"2024-07-17T07:12:03.738228+0000","last_clean":"2024-07-17T07:12:03.738228+0000","last_became_active":"2024-07-17T07:12:02.756577+0000","last_became_peered":"2024-07-17T07:12:02.756577+0000","last_unstale":"2024-07-17T07:12:03.738228+0000","last_undegraded":"2024-07-17T07:12:03.738228+0000","last_fullsized":"2024-07-17T07:12:03.738228+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:12.287364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765682+0000","last_change":"2024-07-17T07:12:03.759165+0000","last_active":"2024-07-17T07:12:03.765682+0000","last_peered":"2024-07-17T07:12:03.765682+0000","last_clean":"2024-07-17T07:12:03.765682+0000","last_became_active":"2024-07-17T07:12:03.758899+0000","last_became_peered":"2024-07-17T07:12:03.758899+0000","last_unstale":"2024-07-17T07:12:03.765682+0000","last_undegraded":"2024-07-17T07:12:03.765682+0000","last_fullsized":"2024-07-17T07:12:03.765682+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:35.574322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775824+0000","last_change":"2024-07-17T07:12:02.754884+0000","last_active":"2024-07-17T07:12:03.775824+0000","last_peered":"2024-07-17T07:12:03.775824+0000","last_clean":"2024-07-17T07:12:03.775824+0000","last_became_active":"2024-07-17T07:12:02.753749+0000","last_became_peered":"2024-07-17T07:12:02.753749+0000","last_unstale":"2024-07-17T07:12:03.775824+0000","last_undegraded":"2024-07-17T07:12:03.775824+0000","last_fullsized":"2024-07-17T07:12:03.775824+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:57:28.911596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765084+0000","last_change":"2024-07-17T07:12:03.756521+0000","last_active":"2024-07-17T07:12:03.765084+0000","last_peered":"2024-07-17T07:12:03.765084+0000","last_clean":"2024-07-17T07:12:03.765084+0000","last_became_active":"2024-07-17T07:12:03.755845+0000","last_became_peered":"2024-07-17T07:12:03.755845+0000","last_unstale":"2024-07-17T07:12:03.765084+0000","last_undegraded":"2024-07-17T07:12:03.765084+0000","last_fullsized":"2024-07-17T07:12:03.765084+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:17.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.746768+0000","last_change":"2024-07-17T07:12:02.751155+0000","last_active":"2024-07-17T07:12:03.746768+0000","last_peered":"2024-07-17T07:12:03.746768+0000","last_clean":"2024-07-17T07:12:03.746768+0000","last_became_active":"2024-07-17T07:12:02.750856+0000","last_became_peered":"2024-07-17T07:12:02.750856+0000","last_unstale":"2024-07-17T07:12:03.746768+0000","last_undegraded":"2024-07-17T07:12:03.746768+0000","last_fullsized":"2024-07-17T07:12:03.746768+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:25.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759552+0000","last_change":"2024-07-17T07:12:03.753279+0000","last_active":"2024-07-17T07:12:03.759552+0000","last_peered":"2024-07-17T07:12:03.759552+0000","last_clean":"2024-07-17T07:12:03.759552+0000","last_became_active":"2024-07-17T07:12:03.752508+0000","last_became_peered":"2024-07-17T07:12:03.752508+0000","last_unstale":"2024-07-17T07:12:03.759552+0000","last_undegraded":"2024-07-17T07:12:03.759552+0000","last_fullsized":"2024-07-17T07:12:03.759552+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:52:23.219057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.747195+0000","last_change":"2024-07-17T07:12:02.752250+0000","last_active":"2024-07-17T07:12:03.747195+0000","last_peered":"2024-07-17T07:12:03.747195+0000","last_clean":"2024-07-17T07:12:03.747195+0000","last_became_active":"2024-07-17T07:12:02.751972+0000","last_became_peered":"2024-07-17T07:12:02.751972+0000","last_unstale":"2024-07-17T07:12:03.747195+0000","last_undegraded":"2024-07-17T07:12:03.747195+0000","last_fullsized":"2024-07-17T07:12:03.747195+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:32.523806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759270+0000","last_change":"2024-07-17T07:12:03.756402+0000","last_active":"2024-07-17T07:12:03.759270+0000","last_peered":"2024-07-17T07:12:03.759270+0000","last_clean":"2024-07-17T07:12:03.759270+0000","last_became_active":"2024-07-17T07:12:03.755855+0000","last_became_peered":"2024-07-17T07:12:03.755855+0000","last_unstale":"2024-07-17T07:12:03.759270+0000","last_undegraded":"2024-07-17T07:12:03.759270+0000","last_fullsized":"2024-07-17T07:12:03.759270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:42.316946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775144+0000","last_change":"2024-07-17T07:12:02.759835+0000","last_active":"2024-07-17T07:12:03.775144+0000","last_peered":"2024-07-17T07:12:03.775144+0000","last_clean":"2024-07-17T07:12:03.775144+0000","last_became_active":"2024-07-17T07:12:02.759456+0000","last_became_peered":"2024-07-17T07:12:02.759456+0000","last_unstale":"2024-07-17T07:12:03.775144+0000","last_undegraded":"2024-07-17T07:12:03.775144+0000","last_fullsized":"2024-07-17T07:12:03.775144+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:37.108104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757780+0000","last_change":"2024-07-17T07:12:03.751090+0000","last_active":"2024-07-17T07:12:03.757780+0000","last_peered":"2024-07-17T07:12:03.757780+0000","last_clean":"2024-07-17T07:12:03.757780+0000","last_became_active":"2024-07-17T07:12:03.750766+0000","last_became_peered":"2024-07-17T07:12:03.750766+0000","last_unstale":"2024-07-17T07:12:03.757780+0000","last_undegraded":"2024-07-17T07:12:03.757780+0000","last_fullsized":"2024-07-17T07:12:03.757780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:36.880110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.734122+0000","last_change":"2024-07-17T07:12:02.753693+0000","last_active":"2024-07-17T07:12:03.734122+0000","last_peered":"2024-07-17T07:12:03.734122+0000","last_clean":"2024-07-17T07:12:03.734122+0000","last_became_active":"2024-07-17T07:12:02.753309+0000","last_became_peered":"2024-07-17T07:12:02.753309+0000","last_unstale":"2024-07-17T07:12:03.734122+0000","last_undegraded":"2024-07-17T07:12:03.734122+0000","last_fullsized":"2024-07-17T07:12:03.734122+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:13.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753597+0000","last_change":"2024-07-17T07:12:03.749497+0000","last_active":"2024-07-17T07:12:03.753597+0000","last_peered":"2024-07-17T07:12:03.753597+0000","last_clean":"2024-07-17T07:12:03.753597+0000","last_became_active":"2024-07-17T07:12:03.748488+0000","last_became_peered":"2024-07-17T07:12:03.748488+0000","last_unstale":"2024-07-17T07:12:03.753597+0000","last_undegraded":"2024-07-17T07:12:03.753597+0000","last_fullsized":"2024-07-17T07:12:03.753597+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:41.167097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.741740+0000","last_change":"2024-07-17T07:12:02.758904+0000","last_active":"2024-07-17T07:12:03.741740+0000","last_peered":"2024-07-17T07:12:03.741740+0000","last_clean":"2024-07-17T07:12:03.741740+0000","last_became_active":"2024-07-17T07:12:02.758355+0000","last_became_peered":"2024-07-17T07:12:02.758355+0000","last_unstale":"2024-07-17T07:12:03.741740+0000","last_undegraded":"2024-07-17T07:12:03.741740+0000","last_fullsized":"2024-07-17T07:12:03.741740+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758361+0000","last_change":"2024-07-17T07:12:03.746973+0000","last_active":"2024-07-17T07:12:03.758361+0000","last_peered":"2024-07-17T07:12:03.758361+0000","last_clean":"2024-07-17T07:12:03.758361+0000","last_became_active":"2024-07-17T07:12:03.746686+0000","last_became_peered":"2024-07-17T07:12:03.746686+0000","last_unstale":"2024-07-17T07:12:03.758361+0000","last_undegraded":"2024-07-17T07:12:03.758361+0000","last_fullsized":"2024-07-17T07:12:03.758361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.469449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.740363+0000","last_change":"2024-07-17T07:12:02.758347+0000","last_active":"2024-07-17T07:12:03.740363+0000","last_peered":"2024-07-17T07:12:03.740363+0000","last_clean":"2024-07-17T07:12:03.740363+0000","last_became_active":"2024-07-17T07:12:02.758088+0000","last_became_peered":"2024-07-17T07:12:02.758088+0000","last_unstale":"2024-07-17T07:12:03.740363+0000","last_undegraded":"2024-07-17T07:12:03.740363+0000","last_fullsized":"2024-07-17T07:12:03.740363+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:16:22.872337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760243+0000","last_change":"2024-07-17T07:12:03.756407+0000","last_active":"2024-07-17T07:12:03.760243+0000","last_peered":"2024-07-17T07:12:03.760243+0000","last_clean":"2024-07-17T07:12:03.760243+0000","last_became_active":"2024-07-17T07:12:03.754759+0000","last_became_peered":"2024-07-17T07:12:03.754759+0000","last_unstale":"2024-07-17T07:12:03.760243+0000","last_undegraded":"2024-07-17T07:12:03.760243+0000","last_fullsized":"2024-07-17T07:12:03.760243+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:09.226399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.723996+0000","last_change":"2024-07-17T07:12:02.755379+0000","last_active":"2024-07-17T07:12:03.723996+0000","last_peered":"2024-07-17T07:12:03.723996+0000","last_clean":"2024-07-17T07:12:03.723996+0000","last_became_active":"2024-07-17T07:12:02.754903+0000","last_became_peered":"2024-07-17T07:12:02.754903+0000","last_unstale":"2024-07-17T07:12:03.723996+0000","last_undegraded":"2024-07-17T07:12:03.723996+0000","last_fullsized":"2024-07-17T07:12:03.723996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:09:20.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752829+0000","last_change":"2024-07-17T07:12:03.739130+0000","last_active":"2024-07-17T07:12:03.752829+0000","last_peered":"2024-07-17T07:12:03.752829+0000","last_clean":"2024-07-17T07:12:03.752829+0000","last_became_active":"2024-07-17T07:12:03.738360+0000","last_became_peered":"2024-07-17T07:12:03.738360+0000","last_unstale":"2024-07-17T07:12:03.752829+0000","last_undegraded":"2024-07-17T07:12:03.752829+0000","last_fullsized":"2024-07-17T07:12:03.752829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:06.159873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.720246+0000","last_change":"2024-07-17T07:12:02.741015+0000","last_active":"2024-07-17T07:12:03.720246+0000","last_peered":"2024-07-17T07:12:03.720246+0000","last_clean":"2024-07-17T07:12:03.720246+0000","last_became_active":"2024-07-17T07:12:02.740673+0000","last_became_peered":"2024-07-17T07:12:02.740673+0000","last_unstale":"2024-07-17T07:12:03.720246+0000","last_undegraded":"2024-07-17T07:12:03.720246+0000","last_fullsized":"2024-07-17T07:12:03.720246+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:51.650613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760815+0000","last_change":"2024-07-17T07:12:03.752140+0000","last_active":"2024-07-17T07:12:03.760815+0000","last_peered":"2024-07-17T07:12:03.760815+0000","last_clean":"2024-07-17T07:12:03.760815+0000","last_became_active":"2024-07-17T07:12:03.751866+0000","last_became_peered":"2024-07-17T07:12:03.751866+0000","last_unstale":"2024-07-17T07:12:03.760815+0000","last_undegraded":"2024-07-17T07:12:03.760815+0000","last_fullsized":"2024-07-17T07:12:03.760815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:34.971622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.725554+0000","last_change":"2024-07-17T07:12:02.755998+0000","last_active":"2024-07-17T07:12:03.725554+0000","last_peered":"2024-07-17T07:12:03.725554+0000","last_clean":"2024-07-17T07:12:03.725554+0000","last_became_active":"2024-07-17T07:12:02.755631+0000","last_became_peered":"2024-07-17T07:12:02.755631+0000","last_unstale":"2024-07-17T07:12:03.725554+0000","last_undegraded":"2024-07-17T07:12:03.725554+0000","last_fullsized":"2024-07-17T07:12:03.725554+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:29.326253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753140+0000","last_change":"2024-07-17T07:12:03.748012+0000","last_active":"2024-07-17T07:12:03.753140+0000","last_peered":"2024-07-17T07:12:03.753140+0000","last_clean":"2024-07-17T07:12:03.753140+0000","last_became_active":"2024-07-17T07:12:03.747718+0000","last_became_peered":"2024-07-17T07:12:03.747718+0000","last_unstale":"2024-07-17T07:12:03.753140+0000","last_undegraded":"2024-07-17T07:12:03.753140+0000","last_fullsized":"2024-07-17T07:12:03.753140+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:54.534226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.737985+0000","last_change":"2024-07-17T07:12:02.755560+0000","last_active":"2024-07-17T07:12:03.737985+0000","last_peered":"2024-07-17T07:12:03.737985+0000","last_clean":"2024-07-17T07:12:03.737985+0000","last_became_active":"2024-07-17T07:12:02.755217+0000","last_became_peered":"2024-07-17T07:12:02.755217+0000","last_unstale":"2024-07-17T07:12:03.737985+0000","last_undegraded":"2024-07-17T07:12:03.737985+0000","last_fullsized":"2024-07-17T07:12:03.737985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:42.774434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759121+0000","last_change":"2024-07-17T07:12:03.756954+0000","last_active":"2024-07-17T07:12:03.759121+0000","last_peered":"2024-07-17T07:12:03.759121+0000","last_clean":"2024-07-17T07:12:03.759121+0000","last_became_active":"2024-07-17T07:12:03.756655+0000","last_became_peered":"2024-07-17T07:12:03.756655+0000","last_unstale":"2024-07-17T07:12:03.759121+0000","last_undegraded":"2024-07-17T07:12:03.759121+0000","last_fullsized":"2024-07-17T07:12:03.759121+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:01:54.165878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.747061+0000","last_change":"2024-07-17T07:12:02.752641+0000","last_active":"2024-07-17T07:12:03.747061+0000","last_peered":"2024-07-17T07:12:03.747061+0000","last_clean":"2024-07-17T07:12:03.747061+0000","last_became_active":"2024-07-17T07:12:02.751933+0000","last_became_peered":"2024-07-17T07:12:02.751933+0000","last_unstale":"2024-07-17T07:12:03.747061+0000","last_undegraded":"2024-07-17T07:12:03.747061+0000","last_fullsized":"2024-07-17T07:12:03.747061+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:17.181050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"88'114","reported_seq":409,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738440+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:12:03.738440+0000","last_peered":"2024-07-17T07:12:03.738440+0000","last_clean":"2024-07-17T07:12:03.738440+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:12:03.738440+0000","last_undegraded":"2024-07-17T07:12:03.738440+0000","last_fullsized":"2024-07-17T07:12:03.738440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":173,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":173,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542352,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1789952,"data_stored":1209999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36496,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335344,"statfs":{"total":96636764160,"available":96599392256,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509648,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50384,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321456,"statfs":{"total":96636764160,"available":96585170944,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542351,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58724,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313116,"statfs":{"total":96636764160,"available":96576630784,"internally_reserved":0,"allocated":1204224,"data_stored":620351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1200128,"data_stored":619721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509647,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1785856,"data_stored":1209977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:12:08.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:12:08.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 -- 172.21.15.44:0/1165962517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680cf540 msgr2=0x7f69680b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 --2- 172.21.15.44:0/1165962517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680cf540 0x7f69680b3870 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6958005fd0 tx=0x7f695802a520 comp rx=0 tx=0).stop 2024-07-17T07:12:08.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 -- 172.21.15.44:0/1165962517 shutdown_connections 2024-07-17T07:12:08.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 --2- 172.21.15.44:0/1165962517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69680b3db0 0x7f69680ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 --2- 172.21.15.44:0/1165962517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680cf540 0x7f69680b3870 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 -- 172.21.15.44:0/1165962517 >> 172.21.15.44:0/1165962517 conn(0x7f69680b0e80 msgr2=0x7f69680b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:08.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 -- 172.21.15.44:0/1165962517 shutdown_connections 2024-07-17T07:12:08.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.465+0000 7f696d200640 1 -- 172.21.15.44:0/1165962517 wait complete. 2024-07-17T07:12:08.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.467+0000 7f696d200640 1 Processor -- start 2024-07-17T07:12:08.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.468+0000 7f696d200640 1 -- start start 2024-07-17T07:12:08.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69680cf540 0x7f6968158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69680cf920 0x7f69681563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69680ae940 con 0x7f69680b3db0 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69680ae7c0 con 0x7f69680cf920 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f696d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6968141040 con 0x7f69680cf540 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f6967400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f6967400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35110/0 (socket says 172.21.15.44:35110) 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.469+0000 7f6967400640 1 -- 172.21.15.44:0/3322351112 learned_addr learned my addr 172.21.15.44:0/3322351112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:08.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 -- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69680cf540 msgr2=0x7f6968158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69680cf540 0x7f6968158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 -- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69680cf920 msgr2=0x7f69681563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69680cf920 0x7f69681563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 -- 172.21.15.44:0/3322351112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696806de30 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f6967400640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6958009850 tx=0x7f69580039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695803f070 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6958004890 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.470+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958035030 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.471+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696806e110 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.471+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696806e6d0 con 0x7f69680b3db0 2024-07-17T07:12:08.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.472+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69681414c0 con 0x7f69680b3db0 2024-07-17T07:12:08.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.473+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f6958042070 con 0x7f69680b3db0 2024-07-17T07:12:08.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.474+0000 7f695f400640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 0x7f69440791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.474+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f69580b9820 con 0x7f69680b3db0 2024-07-17T07:12:08.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.475+0000 7f695fe00640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 0x7f69440791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.476+0000 7f695fe00640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 0x7f69440791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f696815eba0 tx=0x7f6950005c70 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:08.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.478+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f69580826a0 con 0x7f69680b3db0 2024-07-17T07:12:08.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.684+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 --> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f696815a0e0 con 0x7f6944076d00 2024-07-17T07:12:08.688 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-07-17T07:12:08.689+0000 7f063c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:12:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.696+0000 7f695f400640 1 -- 172.21.15.44:0/3322351112 <== mgr.5426 v2:172.21.15.175:6800/2446491909 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252019 (secure 0 0 0) 0x7f696815a0e0 con 0x7f6944076d00 2024-07-17T07:12:08.699 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:12:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 msgr2=0x7f69440791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 0x7f69440791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f696815eba0 tx=0x7f6950005c70 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 msgr2=0x7f69681f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6958009850 tx=0x7f69580039e0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 shutdown_connections 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f6944076d00 0x7f69440791c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69680cf920 0x7f69681563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69680cf540 0x7f6968158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 --2- 172.21.15.44:0/3322351112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69680b3db0 0x7f69681f0e00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.703+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 >> 172.21.15.44:0/3322351112 conn(0x7f69680b0e80 msgr2=0x7f69680b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.704+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 shutdown_connections 2024-07-17T07:12:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.704+0000 7f696d200640 1 -- 172.21.15.44:0/3322351112 wait complete. 2024-07-17T07:12:08.722 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-07-17T07:12:08.358893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":217,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":343184,"kb_used_data":10448,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754631536,"statfs":{"total":773094113280,"available":772742692864,"internally_reserved":0,"allocated":10698752,"data_stored":6107736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001896"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753722+0000","last_change":"2024-07-17T07:12:03.752325+0000","last_active":"2024-07-17T07:12:03.753722+0000","last_peered":"2024-07-17T07:12:03.753722+0000","last_clean":"2024-07-17T07:12:03.753722+0000","last_became_active":"2024-07-17T07:12:03.751976+0000","last_became_peered":"2024-07-17T07:12:03.751976+0000","last_unstale":"2024-07-17T07:12:03.753722+0000","last_undegraded":"2024-07-17T07:12:03.753722+0000","last_fullsized":"2024-07-17T07:12:03.753722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:14.921294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765957+0000","last_change":"2024-07-17T07:12:03.756771+0000","last_active":"2024-07-17T07:12:03.765957+0000","last_peered":"2024-07-17T07:12:03.765957+0000","last_clean":"2024-07-17T07:12:03.765957+0000","last_became_active":"2024-07-17T07:12:03.756256+0000","last_became_peered":"2024-07-17T07:12:03.756256+0000","last_unstale":"2024-07-17T07:12:03.765957+0000","last_undegraded":"2024-07-17T07:12:03.765957+0000","last_fullsized":"2024-07-17T07:12:03.765957+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:55.492395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.762200+0000","last_change":"2024-07-17T07:12:03.758185+0000","last_active":"2024-07-17T07:12:03.762200+0000","last_peered":"2024-07-17T07:12:03.762200+0000","last_clean":"2024-07-17T07:12:03.762200+0000","last_became_active":"2024-07-17T07:12:03.755490+0000","last_became_peered":"2024-07-17T07:12:03.755490+0000","last_unstale":"2024-07-17T07:12:03.762200+0000","last_undegraded":"2024-07-17T07:12:03.762200+0000","last_fullsized":"2024-07-17T07:12:03.762200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:30.344218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758797+0000","last_change":"2024-07-17T07:12:03.751187+0000","last_active":"2024-07-17T07:12:03.758797+0000","last_peered":"2024-07-17T07:12:03.758797+0000","last_clean":"2024-07-17T07:12:03.758797+0000","last_became_active":"2024-07-17T07:12:03.750582+0000","last_became_peered":"2024-07-17T07:12:03.750582+0000","last_unstale":"2024-07-17T07:12:03.758797+0000","last_undegraded":"2024-07-17T07:12:03.758797+0000","last_fullsized":"2024-07-17T07:12:03.758797+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:18.126146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754771+0000","last_change":"2024-07-17T07:12:03.747635+0000","last_active":"2024-07-17T07:12:03.754771+0000","last_peered":"2024-07-17T07:12:03.754771+0000","last_clean":"2024-07-17T07:12:03.754771+0000","last_became_active":"2024-07-17T07:12:03.746772+0000","last_became_peered":"2024-07-17T07:12:03.746772+0000","last_unstale":"2024-07-17T07:12:03.754771+0000","last_undegraded":"2024-07-17T07:12:03.754771+0000","last_fullsized":"2024-07-17T07:12:03.754771+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:24:15.616673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767689+0000","last_change":"2024-07-17T07:12:03.756394+0000","last_active":"2024-07-17T07:12:03.767689+0000","last_peered":"2024-07-17T07:12:03.767689+0000","last_clean":"2024-07-17T07:12:03.767689+0000","last_became_active":"2024-07-17T07:12:03.756117+0000","last_became_peered":"2024-07-17T07:12:03.756117+0000","last_unstale":"2024-07-17T07:12:03.767689+0000","last_undegraded":"2024-07-17T07:12:03.767689+0000","last_fullsized":"2024-07-17T07:12:03.767689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:55.492395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.755259+0000","last_change":"2024-07-17T07:12:03.739416+0000","last_active":"2024-07-17T07:12:03.755259+0000","last_peered":"2024-07-17T07:12:03.755259+0000","last_clean":"2024-07-17T07:12:03.755259+0000","last_became_active":"2024-07-17T07:12:03.739078+0000","last_became_peered":"2024-07-17T07:12:03.739078+0000","last_unstale":"2024-07-17T07:12:03.755259+0000","last_undegraded":"2024-07-17T07:12:03.755259+0000","last_fullsized":"2024-07-17T07:12:03.755259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:05.825465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752953+0000","last_change":"2024-07-17T07:12:03.745184+0000","last_active":"2024-07-17T07:12:03.752953+0000","last_peered":"2024-07-17T07:12:03.752953+0000","last_clean":"2024-07-17T07:12:03.752953+0000","last_became_active":"2024-07-17T07:12:03.744366+0000","last_became_peered":"2024-07-17T07:12:03.744366+0000","last_unstale":"2024-07-17T07:12:03.752953+0000","last_undegraded":"2024-07-17T07:12:03.752953+0000","last_fullsized":"2024-07-17T07:12:03.752953+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:59.514527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759288+0000","last_change":"2024-07-17T07:12:03.756531+0000","last_active":"2024-07-17T07:12:03.759288+0000","last_peered":"2024-07-17T07:12:03.759288+0000","last_clean":"2024-07-17T07:12:03.759288+0000","last_became_active":"2024-07-17T07:12:03.754790+0000","last_became_peered":"2024-07-17T07:12:03.754790+0000","last_unstale":"2024-07-17T07:12:03.759288+0000","last_undegraded":"2024-07-17T07:12:03.759288+0000","last_fullsized":"2024-07-17T07:12:03.759288+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:25:26.406872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759939+0000","last_change":"2024-07-17T07:12:03.759106+0000","last_active":"2024-07-17T07:12:03.759939+0000","last_peered":"2024-07-17T07:12:03.759939+0000","last_clean":"2024-07-17T07:12:03.759939+0000","last_became_active":"2024-07-17T07:12:03.758020+0000","last_became_peered":"2024-07-17T07:12:03.758020+0000","last_unstale":"2024-07-17T07:12:03.759939+0000","last_undegraded":"2024-07-17T07:12:03.759939+0000","last_fullsized":"2024-07-17T07:12:03.759939+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:27.883450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764558+0000","last_change":"2024-07-17T07:12:03.754271+0000","last_active":"2024-07-17T07:12:03.764558+0000","last_peered":"2024-07-17T07:12:03.764558+0000","last_clean":"2024-07-17T07:12:03.764558+0000","last_became_active":"2024-07-17T07:12:03.754012+0000","last_became_peered":"2024-07-17T07:12:03.754012+0000","last_unstale":"2024-07-17T07:12:03.764558+0000","last_undegraded":"2024-07-17T07:12:03.764558+0000","last_fullsized":"2024-07-17T07:12:03.764558+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.541771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.768840+0000","last_change":"2024-07-17T07:12:03.767659+0000","last_active":"2024-07-17T07:12:03.768840+0000","last_peered":"2024-07-17T07:12:03.768840+0000","last_clean":"2024-07-17T07:12:03.768840+0000","last_became_active":"2024-07-17T07:12:03.767335+0000","last_became_peered":"2024-07-17T07:12:03.767335+0000","last_unstale":"2024-07-17T07:12:03.768840+0000","last_undegraded":"2024-07-17T07:12:03.768840+0000","last_fullsized":"2024-07-17T07:12:03.768840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:41.604245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757753+0000","last_change":"2024-07-17T07:12:03.751897+0000","last_active":"2024-07-17T07:12:03.757753+0000","last_peered":"2024-07-17T07:12:03.757753+0000","last_clean":"2024-07-17T07:12:03.757753+0000","last_became_active":"2024-07-17T07:12:03.751604+0000","last_became_peered":"2024-07-17T07:12:03.751604+0000","last_unstale":"2024-07-17T07:12:03.757753+0000","last_undegraded":"2024-07-17T07:12:03.757753+0000","last_fullsized":"2024-07-17T07:12:03.757753+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:59.499759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752394+0000","last_change":"2024-07-17T07:12:03.751143+0000","last_active":"2024-07-17T07:12:03.752394+0000","last_peered":"2024-07-17T07:12:03.752394+0000","last_clean":"2024-07-17T07:12:03.752394+0000","last_became_active":"2024-07-17T07:12:03.750802+0000","last_became_peered":"2024-07-17T07:12:03.750802+0000","last_unstale":"2024-07-17T07:12:03.752394+0000","last_undegraded":"2024-07-17T07:12:03.752394+0000","last_fullsized":"2024-07-17T07:12:03.752394+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:17.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764458+0000","last_change":"2024-07-17T07:12:03.754003+0000","last_active":"2024-07-17T07:12:03.764458+0000","last_peered":"2024-07-17T07:12:03.764458+0000","last_clean":"2024-07-17T07:12:03.764458+0000","last_became_active":"2024-07-17T07:12:03.753681+0000","last_became_peered":"2024-07-17T07:12:03.753681+0000","last_unstale":"2024-07-17T07:12:03.764458+0000","last_undegraded":"2024-07-17T07:12:03.764458+0000","last_fullsized":"2024-07-17T07:12:03.764458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:18.199015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.751402+0000","last_change":"2024-07-17T07:12:03.747788+0000","last_active":"2024-07-17T07:12:03.751402+0000","last_peered":"2024-07-17T07:12:03.751402+0000","last_clean":"2024-07-17T07:12:03.751402+0000","last_became_active":"2024-07-17T07:12:03.747459+0000","last_became_peered":"2024-07-17T07:12:03.747459+0000","last_unstale":"2024-07-17T07:12:03.751402+0000","last_undegraded":"2024-07-17T07:12:03.751402+0000","last_fullsized":"2024-07-17T07:12:03.751402+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:27.795360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.750030+0000","last_change":"2024-07-17T07:12:03.736624+0000","last_active":"2024-07-17T07:12:03.750030+0000","last_peered":"2024-07-17T07:12:03.750030+0000","last_clean":"2024-07-17T07:12:03.750030+0000","last_became_active":"2024-07-17T07:12:03.735744+0000","last_became_peered":"2024-07-17T07:12:03.735744+0000","last_unstale":"2024-07-17T07:12:03.750030+0000","last_undegraded":"2024-07-17T07:12:03.750030+0000","last_fullsized":"2024-07-17T07:12:03.750030+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:40.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760792+0000","last_change":"2024-07-17T07:12:03.755113+0000","last_active":"2024-07-17T07:12:03.760792+0000","last_peered":"2024-07-17T07:12:03.760792+0000","last_clean":"2024-07-17T07:12:03.760792+0000","last_became_active":"2024-07-17T07:12:03.754809+0000","last_became_peered":"2024-07-17T07:12:03.754809+0000","last_unstale":"2024-07-17T07:12:03.760792+0000","last_undegraded":"2024-07-17T07:12:03.760792+0000","last_fullsized":"2024-07-17T07:12:03.760792+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:06.131252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767400+0000","last_change":"2024-07-17T07:12:03.759957+0000","last_active":"2024-07-17T07:12:03.767400+0000","last_peered":"2024-07-17T07:12:03.767400+0000","last_clean":"2024-07-17T07:12:03.767400+0000","last_became_active":"2024-07-17T07:12:03.759613+0000","last_became_peered":"2024-07-17T07:12:03.759613+0000","last_unstale":"2024-07-17T07:12:03.767400+0000","last_undegraded":"2024-07-17T07:12:03.767400+0000","last_fullsized":"2024-07-17T07:12:03.767400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:35.574322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754339+0000","last_change":"2024-07-17T07:12:03.747767+0000","last_active":"2024-07-17T07:12:03.754339+0000","last_peered":"2024-07-17T07:12:03.754339+0000","last_clean":"2024-07-17T07:12:03.754339+0000","last_became_active":"2024-07-17T07:12:03.747466+0000","last_became_peered":"2024-07-17T07:12:03.747466+0000","last_unstale":"2024-07-17T07:12:03.754339+0000","last_undegraded":"2024-07-17T07:12:03.754339+0000","last_fullsized":"2024-07-17T07:12:03.754339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:34.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759385+0000","last_change":"2024-07-17T07:12:03.756678+0000","last_active":"2024-07-17T07:12:03.759385+0000","last_peered":"2024-07-17T07:12:03.759385+0000","last_clean":"2024-07-17T07:12:03.759385+0000","last_became_active":"2024-07-17T07:12:03.753805+0000","last_became_peered":"2024-07-17T07:12:03.753805+0000","last_unstale":"2024-07-17T07:12:03.759385+0000","last_undegraded":"2024-07-17T07:12:03.759385+0000","last_fullsized":"2024-07-17T07:12:03.759385+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:01.197459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758103+0000","last_change":"2024-07-17T07:12:03.749590+0000","last_active":"2024-07-17T07:12:03.758103+0000","last_peered":"2024-07-17T07:12:03.758103+0000","last_clean":"2024-07-17T07:12:03.758103+0000","last_became_active":"2024-07-17T07:12:03.748652+0000","last_became_peered":"2024-07-17T07:12:03.748652+0000","last_unstale":"2024-07-17T07:12:03.758103+0000","last_undegraded":"2024-07-17T07:12:03.758103+0000","last_fullsized":"2024-07-17T07:12:03.758103+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:27.854224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754349+0000","last_change":"2024-07-17T07:12:03.748263+0000","last_active":"2024-07-17T07:12:03.754349+0000","last_peered":"2024-07-17T07:12:03.754349+0000","last_clean":"2024-07-17T07:12:03.754349+0000","last_became_active":"2024-07-17T07:12:03.747971+0000","last_became_peered":"2024-07-17T07:12:03.747971+0000","last_unstale":"2024-07-17T07:12:03.754349+0000","last_undegraded":"2024-07-17T07:12:03.754349+0000","last_fullsized":"2024-07-17T07:12:03.754349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:27.702914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752278+0000","last_change":"2024-07-17T07:12:03.741026+0000","last_active":"2024-07-17T07:12:03.752278+0000","last_peered":"2024-07-17T07:12:03.752278+0000","last_clean":"2024-07-17T07:12:03.752278+0000","last_became_active":"2024-07-17T07:12:03.740742+0000","last_became_peered":"2024-07-17T07:12:03.740742+0000","last_unstale":"2024-07-17T07:12:03.752278+0000","last_undegraded":"2024-07-17T07:12:03.752278+0000","last_fullsized":"2024-07-17T07:12:03.752278+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:47:04.812076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753606+0000","last_change":"2024-07-17T07:12:03.747269+0000","last_active":"2024-07-17T07:12:03.753606+0000","last_peered":"2024-07-17T07:12:03.753606+0000","last_clean":"2024-07-17T07:12:03.753606+0000","last_became_active":"2024-07-17T07:12:03.746159+0000","last_became_peered":"2024-07-17T07:12:03.746159+0000","last_unstale":"2024-07-17T07:12:03.753606+0000","last_undegraded":"2024-07-17T07:12:03.753606+0000","last_fullsized":"2024-07-17T07:12:03.753606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:22.127204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761898+0000","last_change":"2024-07-17T07:12:03.761146+0000","last_active":"2024-07-17T07:12:03.761898+0000","last_peered":"2024-07-17T07:12:03.761898+0000","last_clean":"2024-07-17T07:12:03.761898+0000","last_became_active":"2024-07-17T07:12:03.760868+0000","last_became_peered":"2024-07-17T07:12:03.760868+0000","last_unstale":"2024-07-17T07:12:03.761898+0000","last_undegraded":"2024-07-17T07:12:03.761898+0000","last_fullsized":"2024-07-17T07:12:03.761898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:13.026975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.751440+0000","last_change":"2024-07-17T07:12:03.748832+0000","last_active":"2024-07-17T07:12:03.751440+0000","last_peered":"2024-07-17T07:12:03.751440+0000","last_clean":"2024-07-17T07:12:03.751440+0000","last_became_active":"2024-07-17T07:12:03.748516+0000","last_became_peered":"2024-07-17T07:12:03.748516+0000","last_unstale":"2024-07-17T07:12:03.751440+0000","last_undegraded":"2024-07-17T07:12:03.751440+0000","last_fullsized":"2024-07-17T07:12:03.751440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:51.597102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757619+0000","last_change":"2024-07-17T07:12:03.751861+0000","last_active":"2024-07-17T07:12:03.757619+0000","last_peered":"2024-07-17T07:12:03.757619+0000","last_clean":"2024-07-17T07:12:03.757619+0000","last_became_active":"2024-07-17T07:12:03.751570+0000","last_became_peered":"2024-07-17T07:12:03.751570+0000","last_unstale":"2024-07-17T07:12:03.757619+0000","last_undegraded":"2024-07-17T07:12:03.757619+0000","last_fullsized":"2024-07-17T07:12:03.757619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:05:51.603804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.766540+0000","last_change":"2024-07-17T07:12:03.757515+0000","last_active":"2024-07-17T07:12:03.766540+0000","last_peered":"2024-07-17T07:12:03.766540+0000","last_clean":"2024-07-17T07:12:03.766540+0000","last_became_active":"2024-07-17T07:12:03.757216+0000","last_became_peered":"2024-07-17T07:12:03.757216+0000","last_unstale":"2024-07-17T07:12:03.766540+0000","last_undegraded":"2024-07-17T07:12:03.766540+0000","last_fullsized":"2024-07-17T07:12:03.766540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:05.861721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.766448+0000","last_change":"2024-07-17T07:12:03.765862+0000","last_active":"2024-07-17T07:12:03.766448+0000","last_peered":"2024-07-17T07:12:03.766448+0000","last_clean":"2024-07-17T07:12:03.766448+0000","last_became_active":"2024-07-17T07:12:03.765573+0000","last_became_peered":"2024-07-17T07:12:03.765573+0000","last_unstale":"2024-07-17T07:12:03.766448+0000","last_undegraded":"2024-07-17T07:12:03.766448+0000","last_fullsized":"2024-07-17T07:12:03.766448+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:50.400417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761014+0000","last_change":"2024-07-17T07:12:03.757289+0000","last_active":"2024-07-17T07:12:03.761014+0000","last_peered":"2024-07-17T07:12:03.761014+0000","last_clean":"2024-07-17T07:12:03.761014+0000","last_became_active":"2024-07-17T07:12:03.757022+0000","last_became_peered":"2024-07-17T07:12:03.757022+0000","last_unstale":"2024-07-17T07:12:03.761014+0000","last_undegraded":"2024-07-17T07:12:03.761014+0000","last_fullsized":"2024-07-17T07:12:03.761014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:09:21.762609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759240+0000","last_change":"2024-07-17T07:12:03.755919+0000","last_active":"2024-07-17T07:12:03.759240+0000","last_peered":"2024-07-17T07:12:03.759240+0000","last_clean":"2024-07-17T07:12:03.759240+0000","last_became_active":"2024-07-17T07:12:03.755612+0000","last_became_peered":"2024-07-17T07:12:03.755612+0000","last_unstale":"2024-07-17T07:12:03.759240+0000","last_undegraded":"2024-07-17T07:12:03.759240+0000","last_fullsized":"2024-07-17T07:12:03.759240+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:07.976522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.754734+0000","last_change":"2024-07-17T07:12:03.753865+0000","last_active":"2024-07-17T07:12:03.754734+0000","last_peered":"2024-07-17T07:12:03.754734+0000","last_clean":"2024-07-17T07:12:03.754734+0000","last_became_active":"2024-07-17T07:12:03.753581+0000","last_became_peered":"2024-07-17T07:12:03.753581+0000","last_unstale":"2024-07-17T07:12:03.754734+0000","last_undegraded":"2024-07-17T07:12:03.754734+0000","last_fullsized":"2024-07-17T07:12:03.754734+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:44.502547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"98'4","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776376+0000","last_change":"2024-07-17T07:12:02.747570+0000","last_active":"2024-07-17T07:12:03.776376+0000","last_peered":"2024-07-17T07:12:03.776376+0000","last_clean":"2024-07-17T07:12:03.776376+0000","last_became_active":"2024-07-17T07:12:02.746208+0000","last_became_peered":"2024-07-17T07:12:02.746208+0000","last_unstale":"2024-07-17T07:12:03.776376+0000","last_undegraded":"2024-07-17T07:12:03.776376+0000","last_fullsized":"2024-07-17T07:12:03.776376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:45.695793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764435+0000","last_change":"2024-07-17T07:12:03.753321+0000","last_active":"2024-07-17T07:12:03.764435+0000","last_peered":"2024-07-17T07:12:03.764435+0000","last_clean":"2024-07-17T07:12:03.764435+0000","last_became_active":"2024-07-17T07:12:03.750091+0000","last_became_peered":"2024-07-17T07:12:03.750091+0000","last_unstale":"2024-07-17T07:12:03.764435+0000","last_undegraded":"2024-07-17T07:12:03.764435+0000","last_fullsized":"2024-07-17T07:12:03.764435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:26.296929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775907+0000","last_change":"2024-07-17T07:12:02.756099+0000","last_active":"2024-07-17T07:12:03.775907+0000","last_peered":"2024-07-17T07:12:03.775907+0000","last_clean":"2024-07-17T07:12:03.775907+0000","last_became_active":"2024-07-17T07:12:02.755598+0000","last_became_peered":"2024-07-17T07:12:02.755598+0000","last_unstale":"2024-07-17T07:12:03.775907+0000","last_undegraded":"2024-07-17T07:12:03.775907+0000","last_fullsized":"2024-07-17T07:12:03.775907+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:39:56.012212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753623+0000","last_change":"2024-07-17T07:12:03.749872+0000","last_active":"2024-07-17T07:12:03.753623+0000","last_peered":"2024-07-17T07:12:03.753623+0000","last_clean":"2024-07-17T07:12:03.753623+0000","last_became_active":"2024-07-17T07:12:03.748947+0000","last_became_peered":"2024-07-17T07:12:03.748947+0000","last_unstale":"2024-07-17T07:12:03.753623+0000","last_undegraded":"2024-07-17T07:12:03.753623+0000","last_fullsized":"2024-07-17T07:12:03.753623+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:03.438210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776872+0000","last_change":"2024-07-17T07:12:02.758649+0000","last_active":"2024-07-17T07:12:03.776872+0000","last_peered":"2024-07-17T07:12:03.776872+0000","last_clean":"2024-07-17T07:12:03.776872+0000","last_became_active":"2024-07-17T07:12:02.757979+0000","last_became_peered":"2024-07-17T07:12:02.757979+0000","last_unstale":"2024-07-17T07:12:03.776872+0000","last_undegraded":"2024-07-17T07:12:03.776872+0000","last_fullsized":"2024-07-17T07:12:03.776872+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:47:18.472480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759412+0000","last_change":"2024-07-17T07:12:03.756943+0000","last_active":"2024-07-17T07:12:03.759412+0000","last_peered":"2024-07-17T07:12:03.759412+0000","last_clean":"2024-07-17T07:12:03.759412+0000","last_became_active":"2024-07-17T07:12:03.754189+0000","last_became_peered":"2024-07-17T07:12:03.754189+0000","last_unstale":"2024-07-17T07:12:03.759412+0000","last_undegraded":"2024-07-17T07:12:03.759412+0000","last_fullsized":"2024-07-17T07:12:03.759412+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:43.529279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.771843+0000","last_change":"2024-07-17T07:12:02.754109+0000","last_active":"2024-07-17T07:12:03.771843+0000","last_peered":"2024-07-17T07:12:03.771843+0000","last_clean":"2024-07-17T07:12:03.771843+0000","last_became_active":"2024-07-17T07:12:02.753794+0000","last_became_peered":"2024-07-17T07:12:02.753794+0000","last_unstale":"2024-07-17T07:12:03.771843+0000","last_undegraded":"2024-07-17T07:12:03.771843+0000","last_fullsized":"2024-07-17T07:12:03.771843+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:36:31.365177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765704+0000","last_change":"2024-07-17T07:12:03.759887+0000","last_active":"2024-07-17T07:12:03.765704+0000","last_peered":"2024-07-17T07:12:03.765704+0000","last_clean":"2024-07-17T07:12:03.765704+0000","last_became_active":"2024-07-17T07:12:03.759613+0000","last_became_peered":"2024-07-17T07:12:03.759613+0000","last_unstale":"2024-07-17T07:12:03.765704+0000","last_undegraded":"2024-07-17T07:12:03.765704+0000","last_fullsized":"2024-07-17T07:12:03.765704+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:49.240230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.772294+0000","last_change":"2024-07-17T07:12:02.754014+0000","last_active":"2024-07-17T07:12:03.772294+0000","last_peered":"2024-07-17T07:12:03.772294+0000","last_clean":"2024-07-17T07:12:03.772294+0000","last_became_active":"2024-07-17T07:12:02.753724+0000","last_became_peered":"2024-07-17T07:12:02.753724+0000","last_unstale":"2024-07-17T07:12:03.772294+0000","last_undegraded":"2024-07-17T07:12:03.772294+0000","last_fullsized":"2024-07-17T07:12:03.772294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:09.863023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.764707+0000","last_change":"2024-07-17T07:12:03.755277+0000","last_active":"2024-07-17T07:12:03.764707+0000","last_peered":"2024-07-17T07:12:03.764707+0000","last_clean":"2024-07-17T07:12:03.764707+0000","last_became_active":"2024-07-17T07:12:03.755003+0000","last_became_peered":"2024-07-17T07:12:03.755003+0000","last_unstale":"2024-07-17T07:12:03.764707+0000","last_undegraded":"2024-07-17T07:12:03.764707+0000","last_fullsized":"2024-07-17T07:12:03.764707+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:57:29.929560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.772724+0000","last_change":"2024-07-17T07:12:02.756904+0000","last_active":"2024-07-17T07:12:03.772724+0000","last_peered":"2024-07-17T07:12:03.772724+0000","last_clean":"2024-07-17T07:12:03.772724+0000","last_became_active":"2024-07-17T07:12:02.756607+0000","last_became_peered":"2024-07-17T07:12:02.756607+0000","last_unstale":"2024-07-17T07:12:03.772724+0000","last_undegraded":"2024-07-17T07:12:03.772724+0000","last_fullsized":"2024-07-17T07:12:03.772724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:50.470114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759445+0000","last_change":"2024-07-17T07:12:03.757296+0000","last_active":"2024-07-17T07:12:03.759445+0000","last_peered":"2024-07-17T07:12:03.759445+0000","last_clean":"2024-07-17T07:12:03.759445+0000","last_became_active":"2024-07-17T07:12:03.753152+0000","last_became_peered":"2024-07-17T07:12:03.753152+0000","last_unstale":"2024-07-17T07:12:03.759445+0000","last_undegraded":"2024-07-17T07:12:03.759445+0000","last_fullsized":"2024-07-17T07:12:03.759445+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:39:44.524648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.732640+0000","last_change":"2024-07-17T07:12:02.754825+0000","last_active":"2024-07-17T07:12:03.732640+0000","last_peered":"2024-07-17T07:12:03.732640+0000","last_clean":"2024-07-17T07:12:03.732640+0000","last_became_active":"2024-07-17T07:12:02.754564+0000","last_became_peered":"2024-07-17T07:12:02.754564+0000","last_unstale":"2024-07-17T07:12:03.732640+0000","last_undegraded":"2024-07-17T07:12:03.732640+0000","last_fullsized":"2024-07-17T07:12:03.732640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:40.586281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759572+0000","last_change":"2024-07-17T07:12:03.754368+0000","last_active":"2024-07-17T07:12:03.759572+0000","last_peered":"2024-07-17T07:12:03.759572+0000","last_clean":"2024-07-17T07:12:03.759572+0000","last_became_active":"2024-07-17T07:12:03.749198+0000","last_became_peered":"2024-07-17T07:12:03.749198+0000","last_unstale":"2024-07-17T07:12:03.759572+0000","last_undegraded":"2024-07-17T07:12:03.759572+0000","last_fullsized":"2024-07-17T07:12:03.759572+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:58.850482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.718722+0000","last_change":"2024-07-17T07:12:02.748339+0000","last_active":"2024-07-17T07:12:03.718722+0000","last_peered":"2024-07-17T07:12:03.718722+0000","last_clean":"2024-07-17T07:12:03.718722+0000","last_became_active":"2024-07-17T07:12:02.747964+0000","last_became_peered":"2024-07-17T07:12:02.747964+0000","last_unstale":"2024-07-17T07:12:03.718722+0000","last_undegraded":"2024-07-17T07:12:03.718722+0000","last_fullsized":"2024-07-17T07:12:03.718722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:39.358316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765312+0000","last_change":"2024-07-17T07:12:03.756583+0000","last_active":"2024-07-17T07:12:03.765312+0000","last_peered":"2024-07-17T07:12:03.765312+0000","last_clean":"2024-07-17T07:12:03.765312+0000","last_became_active":"2024-07-17T07:12:03.749888+0000","last_became_peered":"2024-07-17T07:12:03.749888+0000","last_unstale":"2024-07-17T07:12:03.765312+0000","last_undegraded":"2024-07-17T07:12:03.765312+0000","last_fullsized":"2024-07-17T07:12:03.765312+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:39.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.739518+0000","last_change":"2024-07-17T07:12:02.761855+0000","last_active":"2024-07-17T07:12:03.739518+0000","last_peered":"2024-07-17T07:12:03.739518+0000","last_clean":"2024-07-17T07:12:03.739518+0000","last_became_active":"2024-07-17T07:12:02.761512+0000","last_became_peered":"2024-07-17T07:12:02.761512+0000","last_unstale":"2024-07-17T07:12:03.739518+0000","last_undegraded":"2024-07-17T07:12:03.739518+0000","last_fullsized":"2024-07-17T07:12:03.739518+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:30.040092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758922+0000","last_change":"2024-07-17T07:12:03.751127+0000","last_active":"2024-07-17T07:12:03.758922+0000","last_peered":"2024-07-17T07:12:03.758922+0000","last_clean":"2024-07-17T07:12:03.758922+0000","last_became_active":"2024-07-17T07:12:03.750794+0000","last_became_peered":"2024-07-17T07:12:03.750794+0000","last_unstale":"2024-07-17T07:12:03.758922+0000","last_undegraded":"2024-07-17T07:12:03.758922+0000","last_fullsized":"2024-07-17T07:12:03.758922+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:37.085417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.746804+0000","last_change":"2024-07-17T07:12:02.753713+0000","last_active":"2024-07-17T07:12:03.746804+0000","last_peered":"2024-07-17T07:12:03.746804+0000","last_clean":"2024-07-17T07:12:03.746804+0000","last_became_active":"2024-07-17T07:12:02.753414+0000","last_became_peered":"2024-07-17T07:12:02.753414+0000","last_unstale":"2024-07-17T07:12:03.746804+0000","last_undegraded":"2024-07-17T07:12:03.746804+0000","last_fullsized":"2024-07-17T07:12:03.746804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.634672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753092+0000","last_change":"2024-07-17T07:12:03.751842+0000","last_active":"2024-07-17T07:12:03.753092+0000","last_peered":"2024-07-17T07:12:03.753092+0000","last_clean":"2024-07-17T07:12:03.753092+0000","last_became_active":"2024-07-17T07:12:03.751569+0000","last_became_peered":"2024-07-17T07:12:03.751569+0000","last_unstale":"2024-07-17T07:12:03.753092+0000","last_undegraded":"2024-07-17T07:12:03.753092+0000","last_fullsized":"2024-07-17T07:12:03.753092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:39.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"98'3","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775559+0000","last_change":"2024-07-17T07:12:02.752955+0000","last_active":"2024-07-17T07:12:03.775559+0000","last_peered":"2024-07-17T07:12:03.775559+0000","last_clean":"2024-07-17T07:12:03.775559+0000","last_became_active":"2024-07-17T07:12:02.752633+0000","last_became_peered":"2024-07-17T07:12:02.752633+0000","last_unstale":"2024-07-17T07:12:03.775559+0000","last_undegraded":"2024-07-17T07:12:03.775559+0000","last_fullsized":"2024-07-17T07:12:03.775559+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:17:49.795454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.773241+0000","last_change":"2024-07-17T07:12:03.769253+0000","last_active":"2024-07-17T07:12:03.773241+0000","last_peered":"2024-07-17T07:12:03.773241+0000","last_clean":"2024-07-17T07:12:03.773241+0000","last_became_active":"2024-07-17T07:12:03.768014+0000","last_became_peered":"2024-07-17T07:12:03.768014+0000","last_unstale":"2024-07-17T07:12:03.773241+0000","last_undegraded":"2024-07-17T07:12:03.773241+0000","last_fullsized":"2024-07-17T07:12:03.773241+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:52.284911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"98'6","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.776344+0000","last_change":"2024-07-17T07:12:02.756294+0000","last_active":"2024-07-17T07:12:03.776344+0000","last_peered":"2024-07-17T07:12:03.776344+0000","last_clean":"2024-07-17T07:12:03.776344+0000","last_became_active":"2024-07-17T07:12:02.755998+0000","last_became_peered":"2024-07-17T07:12:02.755998+0000","last_unstale":"2024-07-17T07:12:03.776344+0000","last_undegraded":"2024-07-17T07:12:03.776344+0000","last_fullsized":"2024-07-17T07:12:03.776344+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:51.266946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.761894+0000","last_change":"2024-07-17T07:12:03.761295+0000","last_active":"2024-07-17T07:12:03.761894+0000","last_peered":"2024-07-17T07:12:03.761894+0000","last_clean":"2024-07-17T07:12:03.761894+0000","last_became_active":"2024-07-17T07:12:03.761010+0000","last_became_peered":"2024-07-17T07:12:03.761010+0000","last_unstale":"2024-07-17T07:12:03.761894+0000","last_undegraded":"2024-07-17T07:12:03.761894+0000","last_fullsized":"2024-07-17T07:12:03.761894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:10.692235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.773473+0000","last_change":"2024-07-17T07:12:02.751897+0000","last_active":"2024-07-17T07:12:03.773473+0000","last_peered":"2024-07-17T07:12:03.773473+0000","last_clean":"2024-07-17T07:12:03.773473+0000","last_became_active":"2024-07-17T07:12:02.751577+0000","last_became_peered":"2024-07-17T07:12:02.751577+0000","last_unstale":"2024-07-17T07:12:03.773473+0000","last_undegraded":"2024-07-17T07:12:03.773473+0000","last_fullsized":"2024-07-17T07:12:03.773473+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:46.837660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.756509+0000","last_change":"2024-07-17T07:12:03.744689+0000","last_active":"2024-07-17T07:12:03.756509+0000","last_peered":"2024-07-17T07:12:03.756509+0000","last_clean":"2024-07-17T07:12:03.756509+0000","last_became_active":"2024-07-17T07:12:03.744385+0000","last_became_peered":"2024-07-17T07:12:03.744385+0000","last_unstale":"2024-07-17T07:12:03.756509+0000","last_undegraded":"2024-07-17T07:12:03.756509+0000","last_fullsized":"2024-07-17T07:12:03.756509+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:27.336360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.737259+0000","last_change":"2024-07-17T07:12:02.755014+0000","last_active":"2024-07-17T07:12:03.737259+0000","last_peered":"2024-07-17T07:12:03.737259+0000","last_clean":"2024-07-17T07:12:03.737259+0000","last_became_active":"2024-07-17T07:12:02.754542+0000","last_became_peered":"2024-07-17T07:12:02.754542+0000","last_unstale":"2024-07-17T07:12:03.737259+0000","last_undegraded":"2024-07-17T07:12:03.737259+0000","last_fullsized":"2024-07-17T07:12:03.737259+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:57.205689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758324+0000","last_change":"2024-07-17T07:12:03.751977+0000","last_active":"2024-07-17T07:12:03.758324+0000","last_peered":"2024-07-17T07:12:03.758324+0000","last_clean":"2024-07-17T07:12:03.758324+0000","last_became_active":"2024-07-17T07:12:03.750794+0000","last_became_peered":"2024-07-17T07:12:03.750794+0000","last_unstale":"2024-07-17T07:12:03.758324+0000","last_undegraded":"2024-07-17T07:12:03.758324+0000","last_fullsized":"2024-07-17T07:12:03.758324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:59:26.669466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.719283+0000","last_change":"2024-07-17T07:12:02.739956+0000","last_active":"2024-07-17T07:12:03.719283+0000","last_peered":"2024-07-17T07:12:03.719283+0000","last_clean":"2024-07-17T07:12:03.719283+0000","last_became_active":"2024-07-17T07:12:02.739666+0000","last_became_peered":"2024-07-17T07:12:02.739666+0000","last_unstale":"2024-07-17T07:12:03.719283+0000","last_undegraded":"2024-07-17T07:12:03.719283+0000","last_fullsized":"2024-07-17T07:12:03.719283+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:26.113215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759466+0000","last_change":"2024-07-17T07:12:03.753038+0000","last_active":"2024-07-17T07:12:03.759466+0000","last_peered":"2024-07-17T07:12:03.759466+0000","last_clean":"2024-07-17T07:12:03.759466+0000","last_became_active":"2024-07-17T07:12:03.749591+0000","last_became_peered":"2024-07-17T07:12:03.749591+0000","last_unstale":"2024-07-17T07:12:03.759466+0000","last_undegraded":"2024-07-17T07:12:03.759466+0000","last_fullsized":"2024-07-17T07:12:03.759466+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:09.095176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.777398+0000","last_change":"2024-07-17T07:12:02.756834+0000","last_active":"2024-07-17T07:12:03.777398+0000","last_peered":"2024-07-17T07:12:03.777398+0000","last_clean":"2024-07-17T07:12:03.777398+0000","last_became_active":"2024-07-17T07:12:02.756239+0000","last_became_peered":"2024-07-17T07:12:02.756239+0000","last_unstale":"2024-07-17T07:12:03.777398+0000","last_undegraded":"2024-07-17T07:12:03.777398+0000","last_fullsized":"2024-07-17T07:12:03.777398+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:09.674270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757952+0000","last_change":"2024-07-17T07:12:03.757078+0000","last_active":"2024-07-17T07:12:03.757952+0000","last_peered":"2024-07-17T07:12:03.757952+0000","last_clean":"2024-07-17T07:12:03.757952+0000","last_became_active":"2024-07-17T07:12:03.755774+0000","last_became_peered":"2024-07-17T07:12:03.755774+0000","last_unstale":"2024-07-17T07:12:03.757952+0000","last_undegraded":"2024-07-17T07:12:03.757952+0000","last_fullsized":"2024-07-17T07:12:03.757952+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:05.861721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775653+0000","last_change":"2024-07-17T07:12:02.760505+0000","last_active":"2024-07-17T07:12:03.775653+0000","last_peered":"2024-07-17T07:12:03.775653+0000","last_clean":"2024-07-17T07:12:03.775653+0000","last_became_active":"2024-07-17T07:12:02.760211+0000","last_became_peered":"2024-07-17T07:12:02.760211+0000","last_unstale":"2024-07-17T07:12:03.775653+0000","last_undegraded":"2024-07-17T07:12:03.775653+0000","last_fullsized":"2024-07-17T07:12:03.775653+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:32.932164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738118+0000","last_change":"2024-07-17T07:12:03.732313+0000","last_active":"2024-07-17T07:12:03.738118+0000","last_peered":"2024-07-17T07:12:03.738118+0000","last_clean":"2024-07-17T07:12:03.738118+0000","last_became_active":"2024-07-17T07:12:03.731736+0000","last_became_peered":"2024-07-17T07:12:03.731736+0000","last_unstale":"2024-07-17T07:12:03.738118+0000","last_undegraded":"2024-07-17T07:12:03.738118+0000","last_fullsized":"2024-07-17T07:12:03.738118+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:36:30.357739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775474+0000","last_change":"2024-07-17T07:12:02.758542+0000","last_active":"2024-07-17T07:12:03.775474+0000","last_peered":"2024-07-17T07:12:03.775474+0000","last_clean":"2024-07-17T07:12:03.775474+0000","last_became_active":"2024-07-17T07:12:02.757350+0000","last_became_peered":"2024-07-17T07:12:02.757350+0000","last_unstale":"2024-07-17T07:12:03.775474+0000","last_undegraded":"2024-07-17T07:12:03.775474+0000","last_fullsized":"2024-07-17T07:12:03.775474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:09:32.511979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.762095+0000","last_change":"2024-07-17T07:12:03.760561+0000","last_active":"2024-07-17T07:12:03.762095+0000","last_peered":"2024-07-17T07:12:03.762095+0000","last_clean":"2024-07-17T07:12:03.762095+0000","last_became_active":"2024-07-17T07:12:03.760221+0000","last_became_peered":"2024-07-17T07:12:03.760221+0000","last_unstale":"2024-07-17T07:12:03.762095+0000","last_undegraded":"2024-07-17T07:12:03.762095+0000","last_fullsized":"2024-07-17T07:12:03.762095+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:42.932470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.774300+0000","last_change":"2024-07-17T07:12:02.745063+0000","last_active":"2024-07-17T07:12:03.774300+0000","last_peered":"2024-07-17T07:12:03.774300+0000","last_clean":"2024-07-17T07:12:03.774300+0000","last_became_active":"2024-07-17T07:12:02.744788+0000","last_became_peered":"2024-07-17T07:12:02.744788+0000","last_unstale":"2024-07-17T07:12:03.774300+0000","last_undegraded":"2024-07-17T07:12:03.774300+0000","last_fullsized":"2024-07-17T07:12:03.774300+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:57:27.807862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.767558+0000","last_change":"2024-07-17T07:12:03.760850+0000","last_active":"2024-07-17T07:12:03.767558+0000","last_peered":"2024-07-17T07:12:03.767558+0000","last_clean":"2024-07-17T07:12:03.767558+0000","last_became_active":"2024-07-17T07:12:03.760513+0000","last_became_peered":"2024-07-17T07:12:03.760513+0000","last_unstale":"2024-07-17T07:12:03.767558+0000","last_undegraded":"2024-07-17T07:12:03.767558+0000","last_fullsized":"2024-07-17T07:12:03.767558+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:49.240230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738228+0000","last_change":"2024-07-17T07:12:02.756896+0000","last_active":"2024-07-17T07:12:03.738228+0000","last_peered":"2024-07-17T07:12:03.738228+0000","last_clean":"2024-07-17T07:12:03.738228+0000","last_became_active":"2024-07-17T07:12:02.756577+0000","last_became_peered":"2024-07-17T07:12:02.756577+0000","last_unstale":"2024-07-17T07:12:03.738228+0000","last_undegraded":"2024-07-17T07:12:03.738228+0000","last_fullsized":"2024-07-17T07:12:03.738228+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:12.287364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765682+0000","last_change":"2024-07-17T07:12:03.759165+0000","last_active":"2024-07-17T07:12:03.765682+0000","last_peered":"2024-07-17T07:12:03.765682+0000","last_clean":"2024-07-17T07:12:03.765682+0000","last_became_active":"2024-07-17T07:12:03.758899+0000","last_became_peered":"2024-07-17T07:12:03.758899+0000","last_unstale":"2024-07-17T07:12:03.765682+0000","last_undegraded":"2024-07-17T07:12:03.765682+0000","last_fullsized":"2024-07-17T07:12:03.765682+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:35.574322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775824+0000","last_change":"2024-07-17T07:12:02.754884+0000","last_active":"2024-07-17T07:12:03.775824+0000","last_peered":"2024-07-17T07:12:03.775824+0000","last_clean":"2024-07-17T07:12:03.775824+0000","last_became_active":"2024-07-17T07:12:02.753749+0000","last_became_peered":"2024-07-17T07:12:02.753749+0000","last_unstale":"2024-07-17T07:12:03.775824+0000","last_undegraded":"2024-07-17T07:12:03.775824+0000","last_fullsized":"2024-07-17T07:12:03.775824+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:57:28.911596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.765084+0000","last_change":"2024-07-17T07:12:03.756521+0000","last_active":"2024-07-17T07:12:03.765084+0000","last_peered":"2024-07-17T07:12:03.765084+0000","last_clean":"2024-07-17T07:12:03.765084+0000","last_became_active":"2024-07-17T07:12:03.755845+0000","last_became_peered":"2024-07-17T07:12:03.755845+0000","last_unstale":"2024-07-17T07:12:03.765084+0000","last_undegraded":"2024-07-17T07:12:03.765084+0000","last_fullsized":"2024-07-17T07:12:03.765084+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:17.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.746768+0000","last_change":"2024-07-17T07:12:02.751155+0000","last_active":"2024-07-17T07:12:03.746768+0000","last_peered":"2024-07-17T07:12:03.746768+0000","last_clean":"2024-07-17T07:12:03.746768+0000","last_became_active":"2024-07-17T07:12:02.750856+0000","last_became_peered":"2024-07-17T07:12:02.750856+0000","last_unstale":"2024-07-17T07:12:03.746768+0000","last_undegraded":"2024-07-17T07:12:03.746768+0000","last_fullsized":"2024-07-17T07:12:03.746768+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:25.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759552+0000","last_change":"2024-07-17T07:12:03.753279+0000","last_active":"2024-07-17T07:12:03.759552+0000","last_peered":"2024-07-17T07:12:03.759552+0000","last_clean":"2024-07-17T07:12:03.759552+0000","last_became_active":"2024-07-17T07:12:03.752508+0000","last_became_peered":"2024-07-17T07:12:03.752508+0000","last_unstale":"2024-07-17T07:12:03.759552+0000","last_undegraded":"2024-07-17T07:12:03.759552+0000","last_fullsized":"2024-07-17T07:12:03.759552+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:52:23.219057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.747195+0000","last_change":"2024-07-17T07:12:02.752250+0000","last_active":"2024-07-17T07:12:03.747195+0000","last_peered":"2024-07-17T07:12:03.747195+0000","last_clean":"2024-07-17T07:12:03.747195+0000","last_became_active":"2024-07-17T07:12:02.751972+0000","last_became_peered":"2024-07-17T07:12:02.751972+0000","last_unstale":"2024-07-17T07:12:03.747195+0000","last_undegraded":"2024-07-17T07:12:03.747195+0000","last_fullsized":"2024-07-17T07:12:03.747195+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:32.523806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759270+0000","last_change":"2024-07-17T07:12:03.756402+0000","last_active":"2024-07-17T07:12:03.759270+0000","last_peered":"2024-07-17T07:12:03.759270+0000","last_clean":"2024-07-17T07:12:03.759270+0000","last_became_active":"2024-07-17T07:12:03.755855+0000","last_became_peered":"2024-07-17T07:12:03.755855+0000","last_unstale":"2024-07-17T07:12:03.759270+0000","last_undegraded":"2024-07-17T07:12:03.759270+0000","last_fullsized":"2024-07-17T07:12:03.759270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:42.316946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"98'2","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.775144+0000","last_change":"2024-07-17T07:12:02.759835+0000","last_active":"2024-07-17T07:12:03.775144+0000","last_peered":"2024-07-17T07:12:03.775144+0000","last_clean":"2024-07-17T07:12:03.775144+0000","last_became_active":"2024-07-17T07:12:02.759456+0000","last_became_peered":"2024-07-17T07:12:02.759456+0000","last_unstale":"2024-07-17T07:12:03.775144+0000","last_undegraded":"2024-07-17T07:12:03.775144+0000","last_fullsized":"2024-07-17T07:12:03.775144+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:37.108104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.757780+0000","last_change":"2024-07-17T07:12:03.751090+0000","last_active":"2024-07-17T07:12:03.757780+0000","last_peered":"2024-07-17T07:12:03.757780+0000","last_clean":"2024-07-17T07:12:03.757780+0000","last_became_active":"2024-07-17T07:12:03.750766+0000","last_became_peered":"2024-07-17T07:12:03.750766+0000","last_unstale":"2024-07-17T07:12:03.757780+0000","last_undegraded":"2024-07-17T07:12:03.757780+0000","last_fullsized":"2024-07-17T07:12:03.757780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:36.880110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.734122+0000","last_change":"2024-07-17T07:12:02.753693+0000","last_active":"2024-07-17T07:12:03.734122+0000","last_peered":"2024-07-17T07:12:03.734122+0000","last_clean":"2024-07-17T07:12:03.734122+0000","last_became_active":"2024-07-17T07:12:02.753309+0000","last_became_peered":"2024-07-17T07:12:02.753309+0000","last_unstale":"2024-07-17T07:12:03.734122+0000","last_undegraded":"2024-07-17T07:12:03.734122+0000","last_fullsized":"2024-07-17T07:12:03.734122+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:13.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753597+0000","last_change":"2024-07-17T07:12:03.749497+0000","last_active":"2024-07-17T07:12:03.753597+0000","last_peered":"2024-07-17T07:12:03.753597+0000","last_clean":"2024-07-17T07:12:03.753597+0000","last_became_active":"2024-07-17T07:12:03.748488+0000","last_became_peered":"2024-07-17T07:12:03.748488+0000","last_unstale":"2024-07-17T07:12:03.753597+0000","last_undegraded":"2024-07-17T07:12:03.753597+0000","last_fullsized":"2024-07-17T07:12:03.753597+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:58:41.167097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.741740+0000","last_change":"2024-07-17T07:12:02.758904+0000","last_active":"2024-07-17T07:12:03.741740+0000","last_peered":"2024-07-17T07:12:03.741740+0000","last_clean":"2024-07-17T07:12:03.741740+0000","last_became_active":"2024-07-17T07:12:02.758355+0000","last_became_peered":"2024-07-17T07:12:02.758355+0000","last_unstale":"2024-07-17T07:12:03.741740+0000","last_undegraded":"2024-07-17T07:12:03.741740+0000","last_fullsized":"2024-07-17T07:12:03.741740+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.758361+0000","last_change":"2024-07-17T07:12:03.746973+0000","last_active":"2024-07-17T07:12:03.758361+0000","last_peered":"2024-07-17T07:12:03.758361+0000","last_clean":"2024-07-17T07:12:03.758361+0000","last_became_active":"2024-07-17T07:12:03.746686+0000","last_became_peered":"2024-07-17T07:12:03.746686+0000","last_unstale":"2024-07-17T07:12:03.758361+0000","last_undegraded":"2024-07-17T07:12:03.758361+0000","last_fullsized":"2024-07-17T07:12:03.758361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.469449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.740363+0000","last_change":"2024-07-17T07:12:02.758347+0000","last_active":"2024-07-17T07:12:03.740363+0000","last_peered":"2024-07-17T07:12:03.740363+0000","last_clean":"2024-07-17T07:12:03.740363+0000","last_became_active":"2024-07-17T07:12:02.758088+0000","last_became_peered":"2024-07-17T07:12:02.758088+0000","last_unstale":"2024-07-17T07:12:03.740363+0000","last_undegraded":"2024-07-17T07:12:03.740363+0000","last_fullsized":"2024-07-17T07:12:03.740363+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:16:22.872337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760243+0000","last_change":"2024-07-17T07:12:03.756407+0000","last_active":"2024-07-17T07:12:03.760243+0000","last_peered":"2024-07-17T07:12:03.760243+0000","last_clean":"2024-07-17T07:12:03.760243+0000","last_became_active":"2024-07-17T07:12:03.754759+0000","last_became_peered":"2024-07-17T07:12:03.754759+0000","last_unstale":"2024-07-17T07:12:03.760243+0000","last_undegraded":"2024-07-17T07:12:03.760243+0000","last_fullsized":"2024-07-17T07:12:03.760243+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:09.226399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.723996+0000","last_change":"2024-07-17T07:12:02.755379+0000","last_active":"2024-07-17T07:12:03.723996+0000","last_peered":"2024-07-17T07:12:03.723996+0000","last_clean":"2024-07-17T07:12:03.723996+0000","last_became_active":"2024-07-17T07:12:02.754903+0000","last_became_peered":"2024-07-17T07:12:02.754903+0000","last_unstale":"2024-07-17T07:12:03.723996+0000","last_undegraded":"2024-07-17T07:12:03.723996+0000","last_fullsized":"2024-07-17T07:12:03.723996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:09:20.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.752829+0000","last_change":"2024-07-17T07:12:03.739130+0000","last_active":"2024-07-17T07:12:03.752829+0000","last_peered":"2024-07-17T07:12:03.752829+0000","last_clean":"2024-07-17T07:12:03.752829+0000","last_became_active":"2024-07-17T07:12:03.738360+0000","last_became_peered":"2024-07-17T07:12:03.738360+0000","last_unstale":"2024-07-17T07:12:03.752829+0000","last_undegraded":"2024-07-17T07:12:03.752829+0000","last_fullsized":"2024-07-17T07:12:03.752829+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:06.159873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.720246+0000","last_change":"2024-07-17T07:12:02.741015+0000","last_active":"2024-07-17T07:12:03.720246+0000","last_peered":"2024-07-17T07:12:03.720246+0000","last_clean":"2024-07-17T07:12:03.720246+0000","last_became_active":"2024-07-17T07:12:02.740673+0000","last_became_peered":"2024-07-17T07:12:02.740673+0000","last_unstale":"2024-07-17T07:12:03.720246+0000","last_undegraded":"2024-07-17T07:12:03.720246+0000","last_fullsized":"2024-07-17T07:12:03.720246+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:55:51.650613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.760815+0000","last_change":"2024-07-17T07:12:03.752140+0000","last_active":"2024-07-17T07:12:03.760815+0000","last_peered":"2024-07-17T07:12:03.760815+0000","last_clean":"2024-07-17T07:12:03.760815+0000","last_became_active":"2024-07-17T07:12:03.751866+0000","last_became_peered":"2024-07-17T07:12:03.751866+0000","last_unstale":"2024-07-17T07:12:03.760815+0000","last_undegraded":"2024-07-17T07:12:03.760815+0000","last_fullsized":"2024-07-17T07:12:03.760815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:34.971622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.725554+0000","last_change":"2024-07-17T07:12:02.755998+0000","last_active":"2024-07-17T07:12:03.725554+0000","last_peered":"2024-07-17T07:12:03.725554+0000","last_clean":"2024-07-17T07:12:03.725554+0000","last_became_active":"2024-07-17T07:12:02.755631+0000","last_became_peered":"2024-07-17T07:12:02.755631+0000","last_unstale":"2024-07-17T07:12:03.725554+0000","last_undegraded":"2024-07-17T07:12:03.725554+0000","last_fullsized":"2024-07-17T07:12:03.725554+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:29.326253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.753140+0000","last_change":"2024-07-17T07:12:03.748012+0000","last_active":"2024-07-17T07:12:03.753140+0000","last_peered":"2024-07-17T07:12:03.753140+0000","last_clean":"2024-07-17T07:12:03.753140+0000","last_became_active":"2024-07-17T07:12:03.747718+0000","last_became_peered":"2024-07-17T07:12:03.747718+0000","last_unstale":"2024-07-17T07:12:03.753140+0000","last_undegraded":"2024-07-17T07:12:03.753140+0000","last_fullsized":"2024-07-17T07:12:03.753140+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:54.534226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.737985+0000","last_change":"2024-07-17T07:12:02.755560+0000","last_active":"2024-07-17T07:12:03.737985+0000","last_peered":"2024-07-17T07:12:03.737985+0000","last_clean":"2024-07-17T07:12:03.737985+0000","last_became_active":"2024-07-17T07:12:02.755217+0000","last_became_peered":"2024-07-17T07:12:02.755217+0000","last_unstale":"2024-07-17T07:12:03.737985+0000","last_undegraded":"2024-07-17T07:12:03.737985+0000","last_fullsized":"2024-07-17T07:12:03.737985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:42.774434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.759121+0000","last_change":"2024-07-17T07:12:03.756954+0000","last_active":"2024-07-17T07:12:03.759121+0000","last_peered":"2024-07-17T07:12:03.759121+0000","last_clean":"2024-07-17T07:12:03.759121+0000","last_became_active":"2024-07-17T07:12:03.756655+0000","last_became_peered":"2024-07-17T07:12:03.756655+0000","last_unstale":"2024-07-17T07:12:03.759121+0000","last_undegraded":"2024-07-17T07:12:03.759121+0000","last_fullsized":"2024-07-17T07:12:03.759121+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:02.706053+0000","last_clean_scrub_stamp":"2024-07-17T07:12:02.706053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:01:54.165878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.747061+0000","last_change":"2024-07-17T07:12:02.752641+0000","last_active":"2024-07-17T07:12:03.747061+0000","last_peered":"2024-07-17T07:12:03.747061+0000","last_clean":"2024-07-17T07:12:03.747061+0000","last_became_active":"2024-07-17T07:12:02.751933+0000","last_became_peered":"2024-07-17T07:12:02.751933+0000","last_unstale":"2024-07-17T07:12:03.747061+0000","last_undegraded":"2024-07-17T07:12:03.747061+0000","last_fullsized":"2024-07-17T07:12:03.747061+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:01.688088+0000","last_clean_scrub_stamp":"2024-07-17T07:12:01.688088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:17.181050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"88'114","reported_seq":409,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-07-17T07:12:03.738440+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:12:03.738440+0000","last_peered":"2024-07-17T07:12:03.738440+0000","last_clean":"2024-07-17T07:12:03.738440+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:12:03.738440+0000","last_undegraded":"2024-07-17T07:12:03.738440+0000","last_fullsized":"2024-07-17T07:12:03.738440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":173,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":426,"num_read_kb":1111,"num_write":173,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542352,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1789952,"data_stored":1209999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36496,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335344,"statfs":{"total":96636764160,"available":96599392256,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509648,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50384,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321456,"statfs":{"total":96636764160,"available":96585170944,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1179648,"data_stored":611922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542351,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58724,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313116,"statfs":{"total":96636764160,"available":96576630784,"internally_reserved":0,"allocated":1204224,"data_stored":620351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1200128,"data_stored":619721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509647,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1785856,"data_stored":1209977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:12:08.726 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:12:08.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:12:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 -- 172.21.15.44:0/3482281960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 msgr2=0x7f7d7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 --2- 172.21.15.44:0/3482281960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d7c004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7d84006f90 tx=0x7f7d7c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 -- 172.21.15.44:0/3482281960 shutdown_connections 2024-07-17T07:12:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 --2- 172.21.15.44:0/3482281960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d7c004c80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 -- 172.21.15.44:0/3482281960 >> 172.21.15.44:0/3482281960 conn(0x7f7d940cf0d0 msgr2=0x7f7d940cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 -- 172.21.15.44:0/3482281960 shutdown_connections 2024-07-17T07:12:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.988+0000 7f7d9a400640 1 -- 172.21.15.44:0/3482281960 wait complete. 2024-07-17T07:12:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.990+0000 7f7d9a400640 1 Processor -- start 2024-07-17T07:12:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.990+0000 7f7d9a400640 1 -- start start 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 0x7f7d940713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d940cace0 0x7f7d94071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9406f000 con 0x7f7d7c004800 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9406ee80 con 0x7f7d940cace0 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d9a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9406f180 con 0x7f7d940ca860 2024-07-17T07:12:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d99000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d99000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33564/0 (socket says 172.21.15.44:33564) 2024-07-17T07:12:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.991+0000 7f7d99000640 1 -- 172.21.15.44:0/2347061851 learned_addr learned my addr 172.21.15.44:0/2347061851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 -- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 msgr2=0x7f7d940713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d93e00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 0x7f7d940713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99a00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d940cace0 0x7f7d94071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 0x7f7d940713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 -- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d940cace0 msgr2=0x7f7d94071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d940cace0 0x7f7d94071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 -- 172.21.15.44:0/2347061851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d94071e60 con 0x7f7d7c004800 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d93e00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 0x7f7d940713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d99000640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800d5b0 tx=0x7f7d8800da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88016020 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d88012630 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d94072110 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.992+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88012a80 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.993+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d94099820 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.994+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d94050670 con 0x7f7d7c004800 2024-07-17T07:12:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.995+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f7d88019020 con 0x7f7d7c004800 2024-07-17T07:12:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.996+0000 7f7d90c00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 0x7f7d78079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.996+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f7d880986e0 con 0x7f7d7c004800 2024-07-17T07:12:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.997+0000 7f7d93e00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 0x7f7d78079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:08.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:08.998+0000 7f7d93e00640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 0x7f7d78079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d84033000 tx=0x7f7d940b4230 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.000+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7d94050670 con 0x7f7d7c004800 2024-07-17T07:12:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.278+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7d9409d1b0 con 0x7f7d7c004800 2024-07-17T07:12:09.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.282+0000 7f7d90c00640 1 -- 172.21.15.44:0/2347061851 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7f7d9409d1b0 con 0x7f7d7c004800 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:12:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:12:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 msgr2=0x7f7d78079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 0x7f7d78079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d84033000 tx=0x7f7d940b4230 comp rx=0 tx=0).stop 2024-07-17T07:12:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 msgr2=0x7f7d9409a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800d5b0 tx=0x7f7d8800da70 comp rx=0 tx=0).stop 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 shutdown_connections 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f7d78076cd0 0x7f7d78079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d940cace0 0x7f7d94071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d940ca860 0x7f7d940713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 --2- 172.21.15.44:0/2347061851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d7c004800 0x7f7d9409a020 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.285+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 >> 172.21.15.44:0/2347061851 conn(0x7f7d940cf0d0 msgr2=0x7f7d9415d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.286+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 shutdown_connections 2024-07-17T07:12:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.286+0000 7f7d9a400640 1 -- 172.21.15.44:0/2347061851 wait complete. 2024-07-17T07:12:09.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.511+0000 7fd3f9400640 1 -- 172.21.15.44:0/4205843819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f40cc890 msgr2=0x7fd3f40cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.511+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4205843819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f40cc890 0x7fd3f40cf6a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005fd0 tx=0x7fd3dc02a940 comp rx=0 tx=0).stop 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 -- 172.21.15.44:0/4205843819 shutdown_connections 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4205843819 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cfbe0 0x7fd3f40b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4205843819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f40cc890 0x7fd3f40cf6a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 -- 172.21.15.44:0/4205843819 >> 172.21.15.44:0/4205843819 conn(0x7fd3f40b3120 msgr2=0x7fd3f40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 -- 172.21.15.44:0/4205843819 shutdown_connections 2024-07-17T07:12:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.512+0000 7fd3f9400640 1 -- 172.21.15.44:0/4205843819 wait complete. 2024-07-17T07:12:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.513+0000 7fd3f9400640 1 Processor -- start 2024-07-17T07:12:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.514+0000 7fd3f9400640 1 -- start start 2024-07-17T07:12:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.514+0000 7fd3f9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cc890 0x7fd3f40afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.514+0000 7fd3f9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 0x7fd3f415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40b2540 con 0x7fd3f415b5e0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40b23c0 con 0x7fd3f40cfbe0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f40b26c0 con 0x7fd3f40cc890 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33592/0 (socket says 172.21.15.44:33592) 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 -- 172.21.15.44:0/4107366217 learned_addr learned my addr 172.21.15.44:0/4107366217 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f1e00640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 0x7fd3f415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f2800640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cc890 0x7fd3f40afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 -- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cc890 msgr2=0x7fd3f40afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cc890 0x7fd3f40afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 -- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 msgr2=0x7fd3f415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 0x7fd3f415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f3200640 1 -- 172.21.15.44:0/4107366217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41f4090 con 0x7fd3f415b5e0 2024-07-17T07:12:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.515+0000 7fd3f1e00640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 0x7fd3f415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3f3200640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e970 tx=0x7fd3ec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00ccc0 con 0x7fd3f415b5e0 2024-07-17T07:12:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3ec00ce60 con 0x7fd3f415b5e0 2024-07-17T07:12:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41f4370 con 0x7fd3f415b5e0 2024-07-17T07:12:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec005280 con 0x7fd3f415b5e0 2024-07-17T07:12:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.516+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41f4930 con 0x7fd3f415b5e0 2024-07-17T07:12:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.517+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f41f4e20 con 0x7fd3f415b5e0 2024-07-17T07:12:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.519+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fd3ec005420 con 0x7fd3f415b5e0 2024-07-17T07:12:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.519+0000 7fd3eaa00640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 0x7fd3d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.520+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fd3ec099360 con 0x7fd3f415b5e0 2024-07-17T07:12:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.520+0000 7fd3f2800640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 0x7fd3d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:09.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.521+0000 7fd3f2800640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 0x7fd3d0079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40013f0 tx=0x7fd3e4009040 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:09.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.524+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd3ec062270 con 0x7fd3f415b5e0 2024-07-17T07:12:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.775+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3f40cf6a0 con 0x7fd3f415b5e0 2024-07-17T07:12:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.777+0000 7fd3eaa00640 1 -- 172.21.15.44:0/4107366217 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7fd3ec061b70 con 0x7fd3f415b5e0 2024-07-17T07:12:09.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:09.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:09.777 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.779+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 msgr2=0x7fd3d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.779+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 0x7fd3d0079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40013f0 tx=0x7fd3e4009040 comp rx=0 tx=0).stop 2024-07-17T07:12:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 msgr2=0x7fd3f415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e970 tx=0x7fd3ec00ed70 comp rx=0 tx=0).stop 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 shutdown_connections 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fd3d0076cd0 0x7fd3d0079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f415b5e0 0x7fd3f415e740 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3f40cfbe0 0x7fd3f415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 --2- 172.21.15.44:0/4107366217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3f40cc890 0x7fd3f40afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 >> 172.21.15.44:0/4107366217 conn(0x7fd3f40b3120 msgr2=0x7fd3f40ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 shutdown_connections 2024-07-17T07:12:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:09.780+0000 7fd3f9400640 1 -- 172.21.15.44:0/4107366217 wait complete. 2024-07-17T07:12:10.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:12:03.214380+0000', 'modified': '2024-07-17T07:12:06.267491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6669}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6669': {'gid': 6669, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2060644552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 2060644552}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2060644552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6669, 'qdb_cluster': [6669]} 2024-07-17T07:12:10.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:12:10.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-17T07:12:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 -- 172.21.15.44:0/3285357494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 msgr2=0x7f080c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 --2- 172.21.15.44:0/3285357494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f080c004c80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e20 tx=0x7f080c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831000640 1 -- 172.21.15.44:0/3285357494 reap_dead start 2024-07-17T07:12:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 -- 172.21.15.44:0/3285357494 shutdown_connections 2024-07-17T07:12:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 -- 172.21.15.44:0/3285357494 >> 172.21.15.44:0/3285357494 conn(0x7f082c0b3090 msgr2=0x7f082c0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 -- 172.21.15.44:0/3285357494 shutdown_connections 2024-07-17T07:12:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.425+0000 7f0831a00640 1 -- 172.21.15.44:0/3285357494 wait complete. 2024-07-17T07:12:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.427+0000 7f0831a00640 1 Processor -- start 2024-07-17T07:12:10.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.427+0000 7f0831a00640 1 -- start start 2024-07-17T07:12:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f082c0b0be0 0x7f082c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f082c15ad90 0x7f082c140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f082c164f70 con 0x7f080c004800 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f082c164df0 con 0x7f082c0b0be0 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f0831a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f082c1650f0 con 0x7f082c15ad90 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33614/0 (socket says 172.21.15.44:33614) 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.428+0000 7f082be00640 1 -- 172.21.15.44:0/1053340059 learned_addr learned my addr 172.21.15.44:0/1053340059 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 -- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f082c15ad90 msgr2=0x7f082c140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f082c15ad90 0x7f082c140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 -- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f082c0b0be0 msgr2=0x7f082c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f082c0b0be0 0x7f082c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 -- 172.21.15.44:0/1053340059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c15b230 con 0x7f080c004800 2024-07-17T07:12:10.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f082be00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d7a0 tx=0x7f081c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f0823e00640 1 -- 172.21.15.44:0/1053340059 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c018070 con 0x7f080c004800 2024-07-17T07:12:10.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f0823e00640 1 -- 172.21.15.44:0/1053340059 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c00de50 con 0x7f080c004800 2024-07-17T07:12:10.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f0823e00640 1 -- 172.21.15.44:0/1053340059 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c00b6c0 con 0x7f080c004800 2024-07-17T07:12:10.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c15b510 con 0x7f080c004800 2024-07-17T07:12:10.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.429+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c15bad0 con 0x7f080c004800 2024-07-17T07:12:10.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.432+0000 7f0823e00640 1 -- 172.21.15.44:0/1053340059 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f081c01b070 con 0x7f080c004800 2024-07-17T07:12:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.433+0000 7f0823e00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 0x7f0808079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.433+0000 7f0823e00640 1 -- 172.21.15.44:0/1053340059 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f081c098010 con 0x7f080c004800 2024-07-17T07:12:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.433+0000 7f0835d63740 1 Processor -- start 2024-07-17T07:12:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.434+0000 7f0835d63740 1 -- start start 2024-07-17T07:12:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.434+0000 7f082b400640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 0x7f0808079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c5fe6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5f9c3b0 0x5629c5f9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5629c5f12d60 con 0x5629c5fc6f70 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629c5f02260 con 0x5629c5fe6bb0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5629c5f062a0 con 0x5629c5f9c3b0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33618/0 (socket says 172.21.15.44:33618) 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 -- 172.21.15.44:0/3818664749 learned_addr learned my addr 172.21.15.44:0/3818664749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082b400640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 0x7f0808079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0814035000 tx=0x7f0814004ab0 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082b400640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c5fe6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f0831000640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5f9c3b0 0x5629c5f9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 -- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5f9c3b0 msgr2=0x5629c5f9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5f9c3b0 0x5629c5f9c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 -- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 msgr2=0x5629c5fe6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c5fe6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082be00640 1 -- 172.21.15.44:0/3818664749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c5c66850 con 0x5629c5fc6f70 2024-07-17T07:12:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.435+0000 7f082b400640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c5fe6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.436+0000 7f082be00640 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f081c001cf0 tx=0x7f081c05c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa75f6aec3710b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.436+0000 7f0820c00640 1 -- 172.21.15.44:0/3818664749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0692a0 con 0x5629c5fc6f70 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.436+0000 7f0820c00640 1 -- 172.21.15.44:0/3818664749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c069950 con 0x5629c5fc6f70 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.436+0000 7f0820c00640 1 -- 172.21.15.44:0/3818664749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c067380 con 0x5629c5fc6f70 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 -- 172.21.15.44:0/3818664749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 msgr2=0x5629c5f9e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f081c001cf0 tx=0x7f081c05c9d0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 -- 172.21.15.44:0/3818664749 shutdown_connections 2024-07-17T07:12:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5f9c3b0 0x5629c5f9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c5fe6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 --2- 172.21.15.44:0/3818664749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5f9e8e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 -- 172.21.15.44:0/3818664749 >> 172.21.15.44:0/3818664749 conn(0x5629c5f9d460 msgr2=0x5629c5fa0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 -- 172.21.15.44:0/3818664749 shutdown_connections 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.437+0000 7f0835d63740 1 -- 172.21.15.44:0/3818664749 wait complete. 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.438+0000 7f0835d63740 1 -- start start 2024-07-17T07:12:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.438+0000 7f0835d63740 1 Processor -- start 2024-07-17T07:12:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 0x5629c5fc28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 0x5629c5fc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5629c5c75620 con 0x5629c5fe6bb0 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629c5ea9f90 con 0x5629c5fc6f70 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5629c5f9b650 con 0x5629c5fc8d70 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f0835d63740 10 client.0 _ll_register_callbacks cb 0x5629c5ecff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f082b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f082b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33628/0 (socket says 172.21.15.44:33628) 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.439+0000 7f082b400640 1 -- 172.21.15.44:0/3674993371 learned_addr learned my addr 172.21.15.44:0/3674993371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0831000640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 0x5629c5fc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082be00640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 0x5629c5fc28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 -- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 msgr2=0x5629c5fc91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 0x5629c5fc91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 -- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 msgr2=0x5629c5fc28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 0x5629c5fc28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 -- 172.21.15.44:0/3674993371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c5cdd6b0 con 0x5629c5fe6bb0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0831000640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 0x5629c5fc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082be00640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 0x5629c5fc28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f082b400640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f081402b390 tx=0x7f0814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c9ee23d73da692 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0819600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081404f060 con 0x5629c5fe6bb0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f0814033840 con 0x5629c5fe6bb0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.440+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081c069950 con 0x5629c5fe6bb0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0814055050 con 0x5629c5fe6bb0 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 10 client.6792 fetch_fsmap learned FSMap version 43 2024-07-17T07:12:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5629c5fc0630 con 0x5629c5fe6bb0 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c069950 con 0x5629c5fe6bb0 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 3502+0+0 (secure 0 0 0) 0x7f0814058020 con 0x5629c5fe6bb0 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 10 client.6792 fetch_fsmap finished waiting for FSMap version 43 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 10 client.6792 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 20 client.6792 populate_metadata read hostname 'smithi044' 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c5ff9480 0x5629c5fc11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 4 client.6792 mds_command: new command op to 6669 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.441+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 --> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c5f14cb0 con 0x5629c5ff9480 2024-07-17T07:12:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.442+0000 7f082be00640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c5ff9480 0x5629c5fc11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.442+0000 7f082be00640 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c5ff9480 0x5629c5fc11b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada784ed6b74b478 server_cookie=42b18ab1a14c36d1 in_seq=0 out_seq=0 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.442+0000 7f0819600640 10 client.6792 ms_handle_connect on v2:172.21.15.44:6834/2060644552 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.443+0000 7f0819600640 1 -- 172.21.15.44:0/3674993371 <== mds.0 v2:172.21.15.44:6834/2060644552 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629c5f14cb0 con 0x5629c5ff9480 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.443+0000 7f0819600640 10 client.6792 handle_command_reply: tid=0 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.443+0000 7f0835d63740 1 client.6792 shutdown 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.443+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 msgr2=0x5629c5fc2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.443+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f081402b390 tx=0x7f0814004a40 comp rx=0 tx=0).stop 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 shutdown_connections 2024-07-17T07:12:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c5ff9480 0x5629c5fc11b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc8d70 0x5629c5fc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 --2- 172.21.15.44:0/3674993371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc6f70 0x5629c5fc28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0819600640 0 client.6792 ms_handle_reset on v2:172.21.15.44:6834/2060644552 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 >> 172.21.15.44:0/3674993371 conn(0x5629c5f9c3b0 msgr2=0x5629c5fc7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 shutdown_connections 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 1 -- 172.21.15.44:0/3674993371 wait complete. 2024-07-17T07:12:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.444+0000 7f0835d63740 20 client.6792 trim_cache size 0 max 16384 2024-07-17T07:12:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.465+0000 7f0835d63740 1 Processor -- start 2024-07-17T07:12:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.465+0000 7f0835d63740 1 -- start start 2024-07-17T07:12:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc8d70 0x5629c5fc28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f082c200bd0 con 0x5629c5fc6f70 2024-07-17T07:12:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f082c1fff30 con 0x5629c5fc8d70 2024-07-17T07:12:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f082c200580 con 0x5629c5fe6bb0 2024-07-17T07:12:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33644/0 (socket says 172.21.15.44:33644) 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 -- 172.21.15.44:0/1916092965 learned_addr learned my addr 172.21.15.44:0/1916092965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 -- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fe6bb0 msgr2=0x5629c5fc2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 -- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc8d70 msgr2=0x5629c5fc28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc8d70 0x5629c5fc28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.466+0000 7f082be00640 1 -- 172.21.15.44:0/1916092965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c5fc9560 con 0x5629c5fc6f70 2024-07-17T07:12:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.467+0000 7f082be00640 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f081c05cfd0 tx=0x7f081c001c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be80e17c84a6b416 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.467+0000 7f081be00640 1 -- 172.21.15.44:0/1916092965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c069950 con 0x5629c5fc6f70 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.467+0000 7f081be00640 1 -- 172.21.15.44:0/1916092965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c05cc70 con 0x5629c5fc6f70 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.467+0000 7f081be00640 1 -- 172.21.15.44:0/1916092965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0673c0 con 0x5629c5fc6f70 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 -- 172.21.15.44:0/1916092965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 msgr2=0x5629c5fc11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f081c05cfd0 tx=0x7f081c001c80 comp rx=0 tx=0).stop 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 -- 172.21.15.44:0/1916092965 shutdown_connections 2024-07-17T07:12:10.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fe6bb0 0x5629c5fc2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fc8d70 0x5629c5fc28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 --2- 172.21.15.44:0/1916092965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc6f70 0x5629c5fc11b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 -- 172.21.15.44:0/1916092965 >> 172.21.15.44:0/1916092965 conn(0x5629c5f9c3b0 msgr2=0x5629c5fa1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 -- 172.21.15.44:0/1916092965 shutdown_connections 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.468+0000 7f0835d63740 1 -- 172.21.15.44:0/1916092965 wait complete. 2024-07-17T07:12:10.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.469+0000 7f0835d63740 1 -- start start 2024-07-17T07:12:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.469+0000 7f0835d63740 1 Processor -- start 2024-07-17T07:12:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc6f70 0x5629c60628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c6009b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5629c5ff9300 con 0x5629c5fc8d70 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5629c5fc1bd0 con 0x5629c5fe6bb0 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5629c5ff98f0 con 0x5629c5fc6f70 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0835d63740 10 client.0 _ll_register_callbacks cb 0x7f0814036070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f082b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f082b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33660/0 (socket says 172.21.15.44:33660) 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f082b400640 1 -- 172.21.15.44:0/3475505372 learned_addr learned my addr 172.21.15.44:0/3475505372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f0831000640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c6009b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.470+0000 7f082be00640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc6f70 0x5629c60628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 -- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc6f70 msgr2=0x5629c60628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc6f70 0x5629c60628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 -- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 msgr2=0x5629c6009b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c6009b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 -- 172.21.15.44:0/3475505372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c5fa1900 con 0x5629c5fc8d70 2024-07-17T07:12:10.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f0831000640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c6009b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f082b400640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d70 tx=0x7f0814050480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a10d5bae63d37ea server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f0819600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814038040 con 0x5629c5fc8d70 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f0814050980 con 0x5629c5fc8d70 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.471+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081c05cc70 con 0x5629c5fc8d70 2024-07-17T07:12:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0814066050 con 0x5629c5fc8d70 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 10 client.6798 fetch_fsmap learned FSMap version 43 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5629c5fc2e80 con 0x5629c5fc8d70 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c05cc70 con 0x5629c5fc8d70 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 43) ==== 3502+0+0 (secure 0 0 0) 0x7f0814069020 con 0x5629c5fc8d70 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 10 client.6798 fetch_fsmap finished waiting for FSMap version 43 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 10 client.6798 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 20 client.6798 populate_metadata read hostname 'smithi044' 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c6063a50 0x5629c5ffba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 4 client.6798 mds_command: new command op to 6669 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 --> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] -- command(tid 0: {"prefix": "status"}) -- 0x5629c5f10750 con 0x5629c6063a50 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.472+0000 7f082be00640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c6063a50 0x5629c5ffba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f082be00640 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c6063a50 0x5629c5ffba80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58f028a49a6b506 server_cookie=a63278e65e775c26 in_seq=0 out_seq=0 2024-07-17T07:12:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0819600640 10 client.6798 ms_handle_connect on v2:172.21.15.44:6834/2060644552 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0819600640 1 -- 172.21.15.44:0/3475505372 <== mds.0 v2:172.21.15.44:6834/2060644552 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5629c5f10750 con 0x5629c6063a50 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0819600640 10 client.6798 handle_command_reply: tid=0 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0835d63740 1 client.6798 shutdown 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 msgr2=0x5629c60095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.473+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d70 tx=0x7f0814050480 comp rx=0 tx=0).stop 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 shutdown_connections 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5629c5fe6bb0 0x5629c6009b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629c5fc8d70 0x5629c60095d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.44:6834/2060644552,v1:172.21.15.44:6835/2060644552] conn(0x5629c6063a50 0x5629c5ffba80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 --2- 172.21.15.44:0/3475505372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5629c5fc6f70 0x5629c60628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 >> 172.21.15.44:0/3475505372 conn(0x5629c5f9c3b0 msgr2=0x5629c5fe7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0819600640 0 client.6798 ms_handle_reset on v2:172.21.15.44:6834/2060644552 2024-07-17T07:12:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 shutdown_connections 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 1 -- 172.21.15.44:0/3475505372 wait complete. 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.474+0000 7f0835d63740 20 client.6798 trim_cache size 0 max 16384 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6669, 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7314824700000004, 2024-07-17T07:12:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 43, 2024-07-17T07:12:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 99, 2024-07-17T07:12:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 99, 2024-07-17T07:12:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 12.662769299000001 2024-07-17T07:12:10.478 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 msgr2=0x7f0808079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 0x7f0808079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0814035000 tx=0x7f0814004ab0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 msgr2=0x7f082c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d7a0 tx=0x7f081c00dc60 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 shutdown_connections 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0808076cd0 0x7f0808079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f082c15ad90 0x7f082c140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f082c0b0be0 0x7f082c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 --2- 172.21.15.44:0/1053340059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080c004800 0x7f082c0b06a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 >> 172.21.15.44:0/1053340059 conn(0x7f082c0b3090 msgr2=0x7f082c158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 shutdown_connections 2024-07-17T07:12:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.475+0000 7f0831a00640 1 -- 172.21.15.44:0/1053340059 wait complete. 2024-07-17T07:12:10.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:12:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 -- 172.21.15.44:0/2396069874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4003b10 msgr2=0x7f0af4003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 --2- 172.21.15.44:0/2396069874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4003b10 0x7f0af4003f30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0b04006f90 tx=0x7f0af4005830 comp rx=0 tx=0).stop 2024-07-17T07:12:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 -- 172.21.15.44:0/2396069874 shutdown_connections 2024-07-17T07:12:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 --2- 172.21.15.44:0/2396069874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4003b10 0x7f0af4003f30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 -- 172.21.15.44:0/2396069874 >> 172.21.15.44:0/2396069874 conn(0x7f0b140b0e80 msgr2=0x7f0b140b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 -- 172.21.15.44:0/2396069874 shutdown_connections 2024-07-17T07:12:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.698+0000 7f0b13200640 1 -- 172.21.15.44:0/2396069874 wait complete. 2024-07-17T07:12:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.700+0000 7f0b13200640 1 Processor -- start 2024-07-17T07:12:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.700+0000 7f0b13200640 1 -- start start 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 0x7f0b14148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b140cc810 0x7f0b1415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14159cb0 con 0x7f0b1414db30 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140cf800 con 0x7f0af4003b10 2024-07-17T07:12:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b13200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b14159e30 con 0x7f0b140cc810 2024-07-17T07:12:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33690/0 (socket says 172.21.15.44:33690) 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.701+0000 7f0b0be00640 1 -- 172.21.15.44:0/3062302731 learned_addr learned my addr 172.21.15.44:0/3062302731 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b11e00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 0x7f0b14148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 -- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b140cc810 msgr2=0x7f0b1415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b140cc810 0x7f0b1415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 -- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 msgr2=0x7f0b14148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 0x7f0b14148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 -- 172.21.15.44:0/3062302731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b140ccc90 con 0x7f0b1414db30 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b11e00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 0x7f0b14148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0be00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00eb30 tx=0x7f0afc00c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.702+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0183f0 con 0x7f0b1414db30 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.703+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0afc018680 con 0x7f0b1414db30 2024-07-17T07:12:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.703+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc018a90 con 0x7f0b1414db30 2024-07-17T07:12:10.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.703+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b140ce440 con 0x7f0b1414db30 2024-07-17T07:12:10.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.703+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b14072b40 con 0x7f0b1414db30 2024-07-17T07:12:10.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.704+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1409b300 con 0x7f0b1414db30 2024-07-17T07:12:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.705+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7f0afc018c30 con 0x7f0b1414db30 2024-07-17T07:12:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.706+0000 7f0b0a000640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 0x7f0af00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.706+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f0afc0991c0 con 0x7f0b1414db30 2024-07-17T07:12:10.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.707+0000 7f0b11e00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 0x7f0af00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.709+0000 7f0b11e00640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 0x7f0af00791e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b04033000 tx=0x7f0b1415a770 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.711+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0afc013070 con 0x7f0b1414db30 2024-07-17T07:12:10.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.971+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0b140b2140 con 0x7f0b1414db30 2024-07-17T07:12:10.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.972+0000 7f0b0a000640 1 -- 172.21.15.44:0/3062302731 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f0afc0620c0 con 0x7f0b1414db30 2024-07-17T07:12:10.972 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:12:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 msgr2=0x7f0af00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 0x7f0af00791e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b04033000 tx=0x7f0b1415a770 comp rx=0 tx=0).stop 2024-07-17T07:12:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 msgr2=0x7f0b141f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00eb30 tx=0x7f0afc00c370 comp rx=0 tx=0).stop 2024-07-17T07:12:10.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 shutdown_connections 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f0af0076d20 0x7f0af00791e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b140cc810 0x7f0b1415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1414db30 0x7f0b141f3ff0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 --2- 172.21.15.44:0/3062302731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af4003b10 0x7f0b14148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 >> 172.21.15.44:0/3062302731 conn(0x7f0b140b0e80 msgr2=0x7f0b14158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.975+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 shutdown_connections 2024-07-17T07:12:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:10.976+0000 7f0b13200640 1 -- 172.21.15.44:0/3062302731 wait complete. 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:12:10.997 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:10.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:10.998 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:12:11.047 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:11.048 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41527sec preferred_lft 41527sec 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:12:11.049 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:11.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.051 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:12:11.132 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:12:11.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.136 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:12:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:12:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:12:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:12:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:12:11.192 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:12:11.193 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:12:11.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:12:11.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:12:11.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.412 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:12:11.461 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:12:11.461 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:12:11.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:11.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:12:11.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:12:11.540 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:11.596 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:11.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:12:11.596 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:11.648 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:12:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:12:11.838 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:12:11.841 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:12:11.841 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:12:11.841 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:12:11.841 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:12:11.841 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:12:11.845 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:12:11.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:11.845 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:11.911 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:12:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:12:12.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:12:12.044 INFO:teuthology.orchestra.run.smithi044.stdout:client6807 2024-07-17T07:12:12.044 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:12.044 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:12.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.304+0000 7ff3eee00640 1 -- 172.21.15.44:0/300229626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 msgr2=0x7ff3e815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.304+0000 7ff3eee00640 1 --2- 172.21.15.44:0/300229626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e815a450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc005fd0 tx=0x7ff3dc02ae60 comp rx=0 tx=0).stop 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 -- 172.21.15.44:0/300229626 shutdown_connections 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 --2- 172.21.15.44:0/300229626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e806d420 0x7ff3e806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 --2- 172.21.15.44:0/300229626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e815a450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 -- 172.21.15.44:0/300229626 >> 172.21.15.44:0/300229626 conn(0x7ff3e80cf0d0 msgr2=0x7ff3e80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 -- 172.21.15.44:0/300229626 shutdown_connections 2024-07-17T07:12:12.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.305+0000 7ff3eee00640 1 -- 172.21.15.44:0/300229626 wait complete. 2024-07-17T07:12:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.306+0000 7ff3eee00640 1 Processor -- start 2024-07-17T07:12:12.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.307+0000 7ff3eee00640 1 -- start start 2024-07-17T07:12:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e806d420 0x7ff3e81443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80ae340 0x7ff3e8146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e806f650 con 0x7ff3e81503c0 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e806f4d0 con 0x7ff3e806d420 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e80ac010 con 0x7ff3e80ae340 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3ee400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3ee400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33716/0 (socket says 172.21.15.44:33716) 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3ee400640 1 -- 172.21.15.44:0/4081381871 learned_addr learned my addr 172.21.15.44:0/4081381871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3eda00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e806d420 0x7ff3e81443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.308+0000 7ff3ed000640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80ae340 0x7ff3e8146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 -- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80ae340 msgr2=0x7ff3e8146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80ae340 0x7ff3e8146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 -- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e806d420 msgr2=0x7ff3e81443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e806d420 0x7ff3e81443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 -- 172.21.15.44:0/4081381871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e8144900 con 0x7ff3e81503c0 2024-07-17T07:12:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3ee400640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000b9a0 tx=0x7ff3e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0019070 con 0x7ff3e81503c0 2024-07-17T07:12:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3e000ccb0 con 0x7ff3e81503c0 2024-07-17T07:12:12.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00146a0 con 0x7ff3e81503c0 2024-07-17T07:12:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e8073eb0 con 0x7ff3e81503c0 2024-07-17T07:12:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.309+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e814f760 con 0x7ff3e81503c0 2024-07-17T07:12:12.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.311+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e8073cc0 con 0x7ff3e81503c0 2024-07-17T07:12:12.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.312+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7ff3e001c070 con 0x7ff3e81503c0 2024-07-17T07:12:12.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.313+0000 7ff3e5600640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 0x7ff3c8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.313+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7ff3e0098670 con 0x7ff3e81503c0 2024-07-17T07:12:12.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.314+0000 7ff3eda00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 0x7ff3c8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.315+0000 7ff3eda00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 0x7ff3c8079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc004730 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:12.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.318+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff3e0061690 con 0x7ff3e81503c0 2024-07-17T07:12:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.554+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 --> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff3e806d8b0 con 0x7ff3c8076cd0 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.558+0000 7ff3e5600640 1 -- 172.21.15.44:0/4081381871 <== mgr.5426 v2:172.21.15.175:6800/2446491909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff3e806d8b0 con 0x7ff3c8076cd0 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:12.558 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:12:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 msgr2=0x7ff3c8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 0x7ff3c8079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc004730 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 msgr2=0x7ff3e81f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000b9a0 tx=0x7ff3e000be60 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 shutdown_connections 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7ff3c8076cd0 0x7ff3c8079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e81503c0 0x7ff3e81f3c90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80ae340 0x7ff3e8146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 --2- 172.21.15.44:0/4081381871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e806d420 0x7ff3e81443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.561+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 >> 172.21.15.44:0/4081381871 conn(0x7ff3e80cf0d0 msgr2=0x7ff3e80cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.562+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 shutdown_connections 2024-07-17T07:12:12.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.562+0000 7ff3eee00640 1 -- 172.21.15.44:0/4081381871 wait complete. 2024-07-17T07:12:12.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:12:12.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 -- 172.21.15.44:0/689885221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 msgr2=0x7fb420003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 --2- 172.21.15.44:0/689885221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb420003e00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00c2e0 tx=0x7fb420005830 comp rx=0 tx=0).stop 2024-07-17T07:12:12.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 -- 172.21.15.44:0/689885221 shutdown_connections 2024-07-17T07:12:12.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 --2- 172.21.15.44:0/689885221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb420003e00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 -- 172.21.15.44:0/689885221 >> 172.21.15.44:0/689885221 conn(0x7fb43c073920 msgr2=0x7fb43c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:12.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 -- 172.21.15.44:0/689885221 shutdown_connections 2024-07-17T07:12:12.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.799+0000 7fb441600640 1 -- 172.21.15.44:0/689885221 wait complete. 2024-07-17T07:12:12.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.801+0000 7fb441600640 1 Processor -- start 2024-07-17T07:12:12.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.801+0000 7fb441600640 1 -- start start 2024-07-17T07:12:12.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb43c0b0180 0x7fb43c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c0b0600 0x7fb43c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c06c550 con 0x7fb420003a00 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c06c3d0 con 0x7fb43c0b0600 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb441600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb43c06c6d0 con 0x7fb43c0b0180 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.802+0000 7fb43be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33730/0 (socket says 172.21.15.44:33730) 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 -- 172.21.15.44:0/242283827 learned_addr learned my addr 172.21.15.44:0/242283827 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb440c00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c0b0600 0x7fb43c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 -- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb43c0b0180 msgr2=0x7fb43c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb433e00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb43c0b0180 0x7fb43c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb43c0b0180 0x7fb43c1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 -- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c0b0600 msgr2=0x7fb43c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:12.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c0b0600 0x7fb43c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 -- 172.21.15.44:0/242283827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c0b0b60 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.803+0000 7fb43be00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00c400 tx=0x7fb42c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.804+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c031040 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.804+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb42c006440 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.804+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c006c70 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.804+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c1f76a0 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.804+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1f7c60 con 0x7fb420003a00 2024-07-17T07:12:12.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.805+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c0ce8b0 con 0x7fb420003a00 2024-07-17T07:12:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.806+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 40) ==== 98931+0+0 (secure 0 0 0) 0x7fb42c034070 con 0x7fb420003a00 2024-07-17T07:12:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.807+0000 7fb438c00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 0x7fb418079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:12.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.807+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fb42c0b8880 con 0x7fb420003a00 2024-07-17T07:12:12.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.808+0000 7fb433e00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 0x7fb418079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:12.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.809+0000 7fb433e00640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 0x7fb418079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb43c1547e0 tx=0x7fb424009040 comp rx=0 tx=0).ready entity=mgr.5426 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:12.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:12.812+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb42c081790 con 0x7fb420003a00 2024-07-17T07:12:13.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.074+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fb43c0ad2a0 con 0x7fb420003a00 2024-07-17T07:12:13.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.808+0000 7fb438c00640 1 -- 172.21.15.44:0/242283827 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v41) ==== 92+0+0 (secure 0 0 0) 0x7fb42c006630 con 0x7fb420003a00 2024-07-17T07:12:13.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 msgr2=0x7fb418079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:13.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 0x7fb418079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb43c1547e0 tx=0x7fb424009040 comp rx=0 tx=0).stop 2024-07-17T07:12:13.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 msgr2=0x7fb43c0b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:13.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00c400 tx=0x7fb42c004dd0 comp rx=0 tx=0).stop 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 shutdown_connections 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fb418076cd0 0x7fb418079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43c0b0600 0x7fb43c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb43c0b0180 0x7fb43c1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 --2- 172.21.15.44:0/242283827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb420003a00 0x7fb43c0b57a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.811+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 >> 172.21.15.44:0/242283827 conn(0x7fb43c073920 msgr2=0x7fb43c0ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.812+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 shutdown_connections 2024-07-17T07:12:13.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:13.812+0000 7fb441600640 1 -- 172.21.15.44:0/242283827 wait complete. 2024-07-17T07:12:13.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:12:14.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 -- 172.21.15.44:0/3132945447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e8003aa0 msgr2=0x7f09e8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3132945447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e8003aa0 0x7f09e8003ec0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009b20 tx=0x7f09e8005830 comp rx=0 tx=0).stop 2024-07-17T07:12:14.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06400640 1 -- 172.21.15.44:0/3132945447 reap_dead start 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 -- 172.21.15.44:0/3132945447 shutdown_connections 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 -- 172.21.15.44:0/3132945447 >> 172.21.15.44:0/3132945447 conn(0x7f0a000b4ff0 msgr2=0x7f0a000b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 -- 172.21.15.44:0/3132945447 shutdown_connections 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.038+0000 7f0a06e00640 1 -- 172.21.15.44:0/3132945447 wait complete. 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.040+0000 7f0a06e00640 1 Processor -- start 2024-07-17T07:12:14.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.040+0000 7f0a06e00640 1 -- start start 2024-07-17T07:12:14.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 0x7f0a001f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a000c9300 0x7f0a000b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001623e0 con 0x7f0a0014bd40 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00162260 con 0x7f09e8003aa0 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a00162560 con 0x7f0a000c9300 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a05000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a05000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33766/0 (socket says 172.21.15.44:33766) 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a05000640 1 -- 172.21.15.44:0/3155442554 learned_addr learned my addr 172.21.15.44:0/3155442554 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a05a00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 0x7f0a001f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.041+0000 7f0a06400640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a000c9300 0x7f0a000b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a000c9300 msgr2=0x7f0a000b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a000c9300 0x7f0a000b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 msgr2=0x7f0a001f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 0x7f0a001f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 -- 172.21.15.44:0/3155442554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a001f86c0 con 0x7f0a0014bd40 2024-07-17T07:12:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05a00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 0x7f0a001f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a05000640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d580 tx=0x7f09ec00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec016020 con 0x7f0a0014bd40 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09ec012630 con 0x7f0a0014bd40 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a00149740 con 0x7f0a0014bd40 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.042+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec012a80 con 0x7f0a0014bd40 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.043+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a000cbc00 con 0x7f0a0014bd40 2024-07-17T07:12:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.044+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a000cfdf0 con 0x7f0a0014bd40 2024-07-17T07:12:14.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.046+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 98948+0+0 (secure 0 0 0) 0x7f09ec019020 con 0x7f0a0014bd40 2024-07-17T07:12:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.047+0000 7f09fd600640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 0x7f09e00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.048+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7f09ec098fd0 con 0x7f0a0014bd40 2024-07-17T07:12:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.048+0000 7f0a05a00640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 msgr2=0x7f09e00791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.048+0000 7f0a05a00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 0x7f09e00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:14.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.051+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f09ec020020 con 0x7f0a0014bd40 2024-07-17T07:12:14.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.249+0000 7f0a05a00640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 msgr2=0x7f09e00791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:14.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.249+0000 7f0a05a00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 0x7f09e00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:14.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.260+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0a000d0e10 con 0x7f0a0014bd40 2024-07-17T07:12:14.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.265+0000 7f09fd600640 1 -- 172.21.15.44:0/3155442554 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) ==== 138+0+0 (secure 0 0 0) 0x7f0a001f86c0 con 0x7f0a0014bd40 2024-07-17T07:12:14.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 msgr2=0x7f09e00791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 0x7f09e00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 msgr2=0x7f0a00149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d580 tx=0x7f09ec00da40 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 shutdown_connections 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7f09e0076d20 0x7f09e00791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a000c9300 0x7f0a000b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0014bd40 0x7f0a00149200 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 --2- 172.21.15.44:0/3155442554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003aa0 0x7f0a001f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.267+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 >> 172.21.15.44:0/3155442554 conn(0x7f0a000b4ff0 msgr2=0x7f0a000b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.268+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 shutdown_connections 2024-07-17T07:12:14.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.268+0000 7f0a06e00640 1 -- 172.21.15.44:0/3155442554 wait complete. 2024-07-17T07:12:14.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:12:14.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 -- 172.21.15.44:0/3252490847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00ce1d0 msgr2=0x7fcba00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 --2- 172.21.15.44:0/3252490847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00ce1d0 0x7fcba00ce5b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009830 tx=0x7fcb9402a5a0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 -- 172.21.15.44:0/3252490847 shutdown_connections 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 --2- 172.21.15.44:0/3252490847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00ce1d0 0x7fcba00ce5b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 -- 172.21.15.44:0/3252490847 >> 172.21.15.44:0/3252490847 conn(0x7fcba00d3700 msgr2=0x7fcba00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 -- 172.21.15.44:0/3252490847 shutdown_connections 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.483+0000 7fcba8400640 1 -- 172.21.15.44:0/3252490847 wait complete. 2024-07-17T07:12:14.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.485+0000 7fcba8400640 1 Processor -- start 2024-07-17T07:12:14.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.485+0000 7fcba8400640 1 -- start start 2024-07-17T07:12:14.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcba00b29e0 0x7fcba00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba00ce1d0 0x7fcba01f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0168aa0 con 0x7fcba00b6790 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0168920 con 0x7fcba00ce1d0 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcba0168c20 con 0x7fcba00b29e0 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33782/0 (socket says 172.21.15.44:33782) 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba5000640 1 -- 172.21.15.44:0/203139638 learned_addr learned my addr 172.21.15.44:0/203139638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba5a00640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcba00b29e0 0x7fcba00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.486+0000 7fcba6400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba00ce1d0 0x7fcba01f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcba00b29e0 msgr2=0x7fcba00c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcba00b29e0 0x7fcba00c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba00ce1d0 msgr2=0x7fcba01f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba00ce1d0 0x7fcba01f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 -- 172.21.15.44:0/203139638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba015d820 con 0x7fcba00b6790 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.487+0000 7fcba5000640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c450 tx=0x7fcb8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.488+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c016020 con 0x7fcba00b6790 2024-07-17T07:12:14.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.488+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb8c012920 con 0x7fcba00b6790 2024-07-17T07:12:14.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.488+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c012d70 con 0x7fcba00b6790 2024-07-17T07:12:14.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.488+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba015db00 con 0x7fcba00b6790 2024-07-17T07:12:14.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.488+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba01f7a20 con 0x7fcba00b6790 2024-07-17T07:12:14.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.489+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba00ce5b0 con 0x7fcba00b6790 2024-07-17T07:12:14.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.491+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 98948+0+0 (secure 0 0 0) 0x7fcb8c019020 con 0x7fcba00b6790 2024-07-17T07:12:14.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.493+0000 7fcb9d600640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 0x7fcb800791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.493+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fcb8c0987f0 con 0x7fcba00b6790 2024-07-17T07:12:14.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.493+0000 7fcba5a00640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 msgr2=0x7fcb800791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:14.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.493+0000 7fcba5a00640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 0x7fcb800791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:14.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.497+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcb8c020020 con 0x7fcba00b6790 2024-07-17T07:12:14.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.694+0000 7fcba5a00640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 msgr2=0x7fcb800791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.695+0000 7fcba5a00640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 0x7fcb800791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:14.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.713+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fcba00d1fb0 con 0x7fcba00b6790 2024-07-17T07:12:14.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.718+0000 7fcb9d600640 1 -- 172.21.15.44:0/203139638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) ==== 133+0+0 (secure 0 0 0) 0x7fcb8c061810 con 0x7fcba00b6790 2024-07-17T07:12:14.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 msgr2=0x7fcb800791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:14.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 0x7fcb800791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 msgr2=0x7fcba01507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c450 tx=0x7fcb8c00c910 comp rx=0 tx=0).stop 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 shutdown_connections 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fcb80076d20 0x7fcb800791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcba00ce1d0 0x7fcba01f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcba00b6790 0x7fcba01507b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 --2- 172.21.15.44:0/203139638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcba00b29e0 0x7fcba00c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.721+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 >> 172.21.15.44:0/203139638 conn(0x7fcba00d3700 msgr2=0x7fcba00b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.722+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 shutdown_connections 2024-07-17T07:12:14.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.722+0000 7fcba8400640 1 -- 172.21.15.44:0/203139638 wait complete. 2024-07-17T07:12:14.741 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2024-07-17T07:12:14.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 -- 172.21.15.44:0/2318318302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c072150 msgr2=0x7fc07c068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 --2- 172.21.15.44:0/2318318302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c072150 0x7fc07c068860 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0cd860 tx=0x7fc06802a4a0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 -- 172.21.15.44:0/2318318302 shutdown_connections 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 --2- 172.21.15.44:0/2318318302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c072150 0x7fc07c068860 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 --2- 172.21.15.44:0/2318318302 >> v2:172.21.15.175:3300/0 conn(0x7fc07c159d50 0x7fc07c15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 -- 172.21.15.44:0/2318318302 >> 172.21.15.44:0/2318318302 conn(0x7fc07c0b1ce0 msgr2=0x7fc07c0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 -- 172.21.15.44:0/2318318302 shutdown_connections 2024-07-17T07:12:14.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.935+0000 7fc081800640 1 -- 172.21.15.44:0/2318318302 wait complete. 2024-07-17T07:12:14.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.937+0000 7fc081800640 1 Processor -- start 2024-07-17T07:12:14.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.937+0000 7fc081800640 1 -- start start 2024-07-17T07:12:14.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c06e2c0 0x7fc07c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc07c072150 0x7fc07c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc07c159d50 0x7fc07c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c06d6f0 con 0x7fc07c06e2c0 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c06d570 con 0x7fc07c072150 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc081800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc07c06e710 con 0x7fc07c159d50 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c06e2c0 0x7fc07c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c06e2c0 0x7fc07c1543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33790/0 (socket says 172.21.15.44:33790) 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 -- 172.21.15.44:0/1097216087 learned_addr learned my addr 172.21.15.44:0/1097216087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc07c072150 0x7fc07c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc080e00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc07c159d50 0x7fc07c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc07c159d50 msgr2=0x7fc07c14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc07c159d50 0x7fc07c14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc07c072150 msgr2=0x7fc07c1f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc07c072150 0x7fc07c1f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc07be00640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c1e9fe0 con 0x7fc07c06e2c0 2024-07-17T07:12:14.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.939+0000 7fc080e00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc07c159d50 0x7fc07c14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:14.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc07be00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc07c06e2c0 0x7fc07c1543f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00d740 tx=0x7fc06c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:14.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c017070 con 0x7fc07c06e2c0 2024-07-17T07:12:14.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06c009d20 con 0x7fc07c06e2c0 2024-07-17T07:12:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc081800640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c154930 con 0x7fc07c06e2c0 2024-07-17T07:12:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0106c0 con 0x7fc07c06e2c0 2024-07-17T07:12:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.940+0000 7fc081800640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c1e9a20 con 0x7fc07c06e2c0 2024-07-17T07:12:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.941+0000 7fc081800640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c06a1a0 con 0x7fc07c06e2c0 2024-07-17T07:12:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.942+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 41) ==== 98948+0+0 (secure 0 0 0) 0x7fc06c01a070 con 0x7fc07c06e2c0 2024-07-17T07:12:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.943+0000 7fc073e00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.943+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6621+0+0 (secure 0 0 0) 0x7fc06c098b00 con 0x7fc07c06e2c0 2024-07-17T07:12:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.944+0000 7fc07b400640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:14.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.944+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:14.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:14.948+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc07c06a1a0 con 0x7fc07c06e2c0 2024-07-17T07:12:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:15.145+0000 7fc07b400640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:15.145+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:15.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:15.180+0000 7fc081800640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7fc07c159ac0 con 0x7fc05c076d20 2024-07-17T07:12:15.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:15.548+0000 7fc07b400640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:15.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:15.548+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:12:15.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:12:15.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:12:16.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:16.349+0000 7fc07b400640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:16.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:16.349+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:12:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:17.951+0000 7fc07b400640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2446491909 2024-07-17T07:12:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:17.951+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:12:18.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:18.058+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 42) ==== 98557+0+0 (secure 0 0 0) 0x7fc06c05d5a0 con 0x7fc07c06e2c0 2024-07-17T07:12:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:18.060+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 msgr2=0x7fc05c0791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:18.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:18.060+0000 7fc073e00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/2446491909,v1:172.21.15.175:6801/2446491909] conn(0x7fc05c076d20 0x7fc05c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f5690e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f5690e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f5690e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f5690e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f5690e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.375 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f568d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.376 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f568d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.376 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f568d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.376 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f568d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:18.376 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:18.375+0000 7f568d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:19.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.072+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 43) ==== 98684+0+0 (secure 0 0 0) 0x7fc06c0822f0 con 0x7fc07c06e2c0 2024-07-17T07:12:19.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.073+0000 7fc073e00640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fc05c07f820 0x7fc05c081c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:19.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.073+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 --> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7fc05c00c880 con 0x7fc05c07f820 2024-07-17T07:12:19.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.080+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fc05c07f820 0x7fc05c081c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:19.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.091+0000 7fc07b400640 1 --2- 172.21.15.44:0/1097216087 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fc05c07f820 0x7fc05c081c10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc068034000 tx=0x7fc068004730 comp rx=0 tx=0).ready entity=mgr.5860 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:19.112 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:19.112+0000 7f569a400640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-07-17T07:12:19.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.113+0000 7fc073e00640 1 -- 172.21.15.44:0/1097216087 <== mgr.5860 v2:172.21.15.175:6800/1810999329 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) ==== 49+0+0 (secure 0 0 0) 0x7fc05c00c880 con 0x7fc05c07f820 2024-07-17T07:12:19.113 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-07-17T07:12:19.135 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:12:19.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.347+0000 7f02f3400640 1 -- 172.21.15.44:0/579950394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 msgr2=0x7f02d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f3400640 1 --2- 172.21.15.44:0/579950394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02d000b9c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0b3aa0 tx=0x7f02d0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f1400640 1 -- 172.21.15.44:0/579950394 reap_dead start 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f3400640 1 -- 172.21.15.44:0/579950394 shutdown_connections 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f3400640 1 -- 172.21.15.44:0/579950394 >> 172.21.15.44:0/579950394 conn(0x7f02ec0b0eb0 msgr2=0x7f02ec0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f3400640 1 -- 172.21.15.44:0/579950394 shutdown_connections 2024-07-17T07:12:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.348+0000 7f02f3400640 1 -- 172.21.15.44:0/579950394 wait complete. 2024-07-17T07:12:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.349+0000 7f02f3400640 1 Processor -- start 2024-07-17T07:12:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.350+0000 7f02f3400640 1 -- start start 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.350+0000 7f02f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 0x7f02ec154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec155100 0x7f02ec09c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0ac660 con 0x7f02d00051c0 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0ac4e0 con 0x7f02ec0992a0 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec0ac7e0 con 0x7f02ec155100 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46994/0 (socket says 172.21.15.44:46994) 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 -- 172.21.15.44:0/1679062781 learned_addr learned my addr 172.21.15.44:0/1679062781 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02e3e00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 0x7f02ec154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:19.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f1400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec155100 0x7f02ec09c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 -- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec155100 msgr2=0x7f02ec09c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec155100 0x7f02ec09c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 -- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 msgr2=0x7f02ec154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 0x7f02ec154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02f0a00640 1 -- 172.21.15.44:0/1679062781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec0997d0 con 0x7f02d00051c0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.351+0000 7f02e3e00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 0x7f02ec154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02f0a00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00da60 tx=0x7f02dc00df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00b9b0 con 0x7f02d00051c0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02dc00bc40 con 0x7f02d00051c0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0035f0 con 0x7f02d00051c0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec099ab0 con 0x7f02d00051c0 2024-07-17T07:12:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.352+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec09a070 con 0x7f02d00051c0 2024-07-17T07:12:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.353+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec15d730 con 0x7f02d00051c0 2024-07-17T07:12:19.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.354+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 43) ==== 98684+0+0 (secure 0 0 0) 0x7f02dc003790 con 0x7f02d00051c0 2024-07-17T07:12:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.355+0000 7f02e9600640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 0x7f02c8078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:19.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.355+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f02dc098cf0 con 0x7f02d00051c0 2024-07-17T07:12:19.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.356+0000 7f02e3e00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 0x7f02c8078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:19.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.357+0000 7f02e3e00640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 0x7f02c8078f10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cee30 tx=0x7f02d400b530 comp rx=0 tx=0).ready entity=mgr.5860 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:19.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.360+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f02ec15d730 con 0x7f02d00051c0 2024-07-17T07:12:19.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:19.626+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f02ec0b3ce0 con 0x7f02d00051c0 2024-07-17T07:12:20.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.117+0000 7f02e9600640 1 -- 172.21.15.44:0/1679062781 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v44) ==== 93+0+0 (secure 0 0 0) 0x7f02ec1549e0 con 0x7f02d00051c0 2024-07-17T07:12:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.121+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 msgr2=0x7f02c8078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.121+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 0x7f02c8078f10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cee30 tx=0x7f02d400b530 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 msgr2=0x7f02ec1f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00da60 tx=0x7f02dc00df20 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 shutdown_connections 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f02c8076a50 0x7f02c8078f10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec155100 0x7f02ec09c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec0992a0 0x7f02ec154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 --2- 172.21.15.44:0/1679062781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02d00051c0 0x7f02ec1f4730 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.122+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 >> 172.21.15.44:0/1679062781 conn(0x7f02ec0b0eb0 msgr2=0x7f02ec15cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.123+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 shutdown_connections 2024-07-17T07:12:20.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.123+0000 7f02f3400640 1 -- 172.21.15.44:0/1679062781 wait complete. 2024-07-17T07:12:20.144 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:12:20.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:12:20.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:12:20.337 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:12:20.337 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:12:20.338 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:20.338 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:20.390 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:20.390 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:20.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:12:20.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 -- 172.21.15.44:0/2284772150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 msgr2=0x7fe028070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 --2- 172.21.15.44:0/2284772150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe028070e50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005fd0 tx=0x7fe01c02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 -- 172.21.15.44:0/2284772150 shutdown_connections 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 --2- 172.21.15.44:0/2284772150 >> v2:172.21.15.175:3300/0 conn(0x7fe0280b4b10 0x7fe0280b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 --2- 172.21.15.44:0/2284772150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe028070e50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 -- 172.21.15.44:0/2284772150 >> 172.21.15.44:0/2284772150 conn(0x7fe0280d1350 msgr2=0x7fe0280d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 -- 172.21.15.44:0/2284772150 shutdown_connections 2024-07-17T07:12:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.678+0000 7fe02de00640 1 -- 172.21.15.44:0/2284772150 wait complete. 2024-07-17T07:12:20.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.680+0000 7fe02de00640 1 Processor -- start 2024-07-17T07:12:20.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.680+0000 7fe02de00640 1 -- start start 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe0281595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 0x7fe0280cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe028074520 con 0x7fe0280b4b10 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0280743a0 con 0x7fe028073f80 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0280b6ca0 con 0x7fe028071b60 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47034/0 (socket says 172.21.15.44:47034) 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02d400640 1 -- 172.21.15.44:0/593076623 learned_addr learned my addr 172.21.15.44:0/593076623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:20.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.681+0000 7fe02ca00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe0281595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe027e00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 0x7fe0280cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 msgr2=0x7fe0281595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe0281595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 msgr2=0x7fe0280cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 0x7fe0280cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 -- 172.21.15.44:0/593076623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0281f8110 con 0x7fe0280b4b10 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe027e00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 0x7fe0280cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:20.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02d400640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe02000ea80 tx=0x7fe02000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020004b20 con 0x7fe0280b4b10 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe020004ce0 con 0x7fe0280b4b10 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0281f3e60 con 0x7fe0280b4b10 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.682+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020005280 con 0x7fe0280b4b10 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.683+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281f4420 con 0x7fe0280b4b10 2024-07-17T07:12:20.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.684+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe028070e50 con 0x7fe0280b4b10 2024-07-17T07:12:20.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.685+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7fe020005420 con 0x7fe0280b4b10 2024-07-17T07:12:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.686+0000 7fe024c00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 0x7fe00c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.686+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7fe020099790 con 0x7fe0280b4b10 2024-07-17T07:12:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.686+0000 7fe02ca00640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 msgr2=0x7fe00c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:20.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.686+0000 7fe02ca00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 0x7fe00c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:20.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.694+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0200624d0 con 0x7fe0280b4b10 2024-07-17T07:12:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.888+0000 7fe02ca00640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 msgr2=0x7fe00c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.888+0000 7fe02ca00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 0x7fe00c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:20.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.916+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe028160bb0 con 0x7fe0280b4b10 2024-07-17T07:12:20.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.917+0000 7fe024c00640 1 -- 172.21.15.44:0/593076623 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v100) ==== 80+0+477 (secure 0 0 0) 0x7fe020061dd0 con 0x7fe0280b4b10 2024-07-17T07:12:20.917 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2672990180 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.917 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/305038534 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4259064219 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1740207257 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3153598340 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1282726969 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/2446491909 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/2446491909 2024-07-18T07:12:18.043292+0000 2024-07-17T07:12:20.918 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T07:12:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.920+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 msgr2=0x7fe00c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.920+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 0x7fe00c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.920+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 msgr2=0x7fe0280d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.920+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe02000ea80 tx=0x7fe02000ef40 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 shutdown_connections 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fe00c076cd0 0x7fe00c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0280b4b10 0x7fe0280d00d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe028073f80 0x7fe0280cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 --2- 172.21.15.44:0/593076623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe028071b60 0x7fe0281595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 >> 172.21.15.44:0/593076623 conn(0x7fe0280d1350 msgr2=0x7fe0280cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 shutdown_connections 2024-07-17T07:12:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:20.921+0000 7fe02de00640 1 -- 172.21.15.44:0/593076623 wait complete. 2024-07-17T07:12:20.938 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:12:20.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:20.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:21.025 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:12:21.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:21.026 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:12:21.069 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:12:21.069 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:12:21.069 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:21.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:12:21.112 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:12:21.112 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:21.112 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:21.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:12:21.157 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:12:21.157 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:12:21.157 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:21.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:12:21.205 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 -- 172.21.15.44:0/3081525470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20004d40 msgr2=0x7f1f200051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 --2- 172.21.15.44:0/3081525470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f20004d40 0x7f1f200051a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c590 tx=0x7f1f20005830 comp rx=0 tx=0).stop 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f3be00640 1 -- 172.21.15.44:0/3081525470 reap_dead start 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 -- 172.21.15.44:0/3081525470 shutdown_connections 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 -- 172.21.15.44:0/3081525470 >> 172.21.15.44:0/3081525470 conn(0x7f1f3c0d00a0 msgr2=0x7f1f3c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 -- 172.21.15.44:0/3081525470 shutdown_connections 2024-07-17T07:12:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.402+0000 7f1f40c00640 1 -- 172.21.15.44:0/3081525470 wait complete. 2024-07-17T07:12:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.404+0000 7f1f40c00640 1 Processor -- start 2024-07-17T07:12:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.404+0000 7f1f40c00640 1 -- start start 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f20004d40 0x7f1f3c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 0x7f1f3c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b5480 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b5300 con 0x7f1f3c0c7310 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.405+0000 7f1f40c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b5600 con 0x7f1f20004d40 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47046/0 (socket says 172.21.15.44:47046) 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 -- 172.21.15.44:0/2743348880 learned_addr learned my addr 172.21.15.44:0/2743348880 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f3b400640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f20004d40 0x7f1f3c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f3be00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 0x7f1f3c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f20004d40 msgr2=0x7f1f3c1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f20004d40 0x7f1f3c1f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 msgr2=0x7f1f3c0c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 0x7f1f3c0c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 -- 172.21.15.44:0/2743348880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c1f4100 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f3be00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 0x7f1f3c0c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.406+0000 7f1f32a00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011a80 tx=0x7f1f24011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.407+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400fd50 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.407+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f24016070 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.407+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f240126b0 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.407+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c1f43e0 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.407+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1f6ff0 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.408+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c1f4900 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.409+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7f1f24018030 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.410+0000 7f1f33e00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 0x7f1f180791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.410+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f1f240992e0 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.410+0000 7f1f3b400640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 msgr2=0x7f1f180791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.410+0000 7f1f3b400640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 0x7f1f180791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:21.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.414+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1f3c1f4900 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.611+0000 7f1f3b400640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 msgr2=0x7f1f180791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:21.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.611+0000 7f1f3b400640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 0x7f1f180791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:21.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.660+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f3c0cafe0 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.661+0000 7f1f33e00640 1 -- 172.21.15.44:0/2743348880 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7f1f24062110 con 0x7f1f3c0ccf80 2024-07-17T07:12:21.661 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:21.661 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:21.661 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 msgr2=0x7f1f180791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 0x7f1f180791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 msgr2=0x7f1f3c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011a80 tx=0x7f1f24011f40 comp rx=0 tx=0).stop 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 shutdown_connections 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f1f18076d00 0x7f1f180791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f3c0c7310 0x7f1f3c0c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f3c0ccf80 0x7f1f3c0c9d00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 --2- 172.21.15.44:0/2743348880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f20004d40 0x7f1f3c1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 >> 172.21.15.44:0/2743348880 conn(0x7f1f3c0d00a0 msgr2=0x7f1f3c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:21.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 shutdown_connections 2024-07-17T07:12:21.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.664+0000 7f1f40c00640 1 -- 172.21.15.44:0/2743348880 wait complete. 2024-07-17T07:12:21.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:21.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 -- 172.21.15.44:0/2965744532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 msgr2=0x7ff29006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 --2- 172.21.15.44:0/2965744532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff29006e480 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff284009830 tx=0x7ff28402a5a0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 -- 172.21.15.44:0/2965744532 shutdown_connections 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 --2- 172.21.15.44:0/2965744532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b2ad0 0x7ff2900b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 --2- 172.21.15.44:0/2965744532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff29006e480 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 -- 172.21.15.44:0/2965744532 >> 172.21.15.44:0/2965744532 conn(0x7ff2900d00a0 msgr2=0x7ff2900d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 -- 172.21.15.44:0/2965744532 shutdown_connections 2024-07-17T07:12:21.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.867+0000 7ff298800640 1 -- 172.21.15.44:0/2965744532 wait complete. 2024-07-17T07:12:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.869+0000 7ff298800640 1 Processor -- start 2024-07-17T07:12:21.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 -- start start 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 0x7ff29015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b5740 0x7ff2901f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff290161730 con 0x7ff29015d600 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2900cef10 con 0x7ff2900b2ad0 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.870+0000 7ff298800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2901618b0 con 0x7ff2900b5740 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47070/0 (socket says 172.21.15.44:47070) 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 -- 172.21.15.44:0/2825955771 learned_addr learned my addr 172.21.15.44:0/2825955771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff295400640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b5740 0x7ff2901f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff295e00640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 0x7ff29015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b5740 msgr2=0x7ff2901f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b5740 0x7ff2901f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 msgr2=0x7ff29015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 0x7ff29015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff296800640 1 -- 172.21.15.44:0/2825955771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2901f5000 con 0x7ff29015d600 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.871+0000 7ff295e00640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 0x7ff29015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff296800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00cbc0 tx=0x7ff28c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c016020 con 0x7ff29015d600 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff28c00ae30 con 0x7ff29015d600 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c012660 con 0x7ff29015d600 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff290159d10 con 0x7ff29015d600 2024-07-17T07:12:21.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.872+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29015a2d0 con 0x7ff29015d600 2024-07-17T07:12:21.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.873+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff290161d60 con 0x7ff29015d600 2024-07-17T07:12:21.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.874+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7ff28c019020 con 0x7ff29015d600 2024-07-17T07:12:21.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.875+0000 7ff28a000640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 0x7ff274079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:21.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.876+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7ff28c098ff0 con 0x7ff29015d600 2024-07-17T07:12:21.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.876+0000 7ff295e00640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 msgr2=0x7ff274079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:21.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.876+0000 7ff295e00640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 0x7ff274079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:21.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:21.880+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff28c020020 con 0x7ff29015d600 2024-07-17T07:12:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.077+0000 7ff295e00640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 msgr2=0x7ff274079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:22.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.077+0000 7ff295e00640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 0x7ff274079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:22.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.125+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff29015a4a0 con 0x7ff29015d600 2024-07-17T07:12:22.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.127+0000 7ff28a000640 1 -- 172.21.15.44:0/2825955771 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7ff28c061e60 con 0x7ff29015d600 2024-07-17T07:12:22.127 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:22.127 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:22.127 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 msgr2=0x7ff274079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 0x7ff274079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 msgr2=0x7ff2901f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00cbc0 tx=0x7ff28c00a430 comp rx=0 tx=0).stop 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 shutdown_connections 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7ff274076cd0 0x7ff274079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff29015d600 0x7ff2901f47a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2900b5740 0x7ff2901f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 --2- 172.21.15.44:0/2825955771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2900b2ad0 0x7ff29015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 >> 172.21.15.44:0/2825955771 conn(0x7ff2900d00a0 msgr2=0x7ff2900c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 shutdown_connections 2024-07-17T07:12:22.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.130+0000 7ff298800640 1 -- 172.21.15.44:0/2825955771 wait complete. 2024-07-17T07:12:22.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.331+0000 7f963d600640 1 -- 172.21.15.44:0/3333534582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ea50 msgr2=0x7f96380ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.331+0000 7f963d600640 1 --2- 172.21.15.44:0/3333534582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ea50 0x7f96380ac220 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9620009830 tx=0x7f962002a510 comp rx=0 tx=0).stop 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 -- 172.21.15.44:0/3333534582 shutdown_connections 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 --2- 172.21.15.44:0/3333534582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ea50 0x7f96380ac220 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 --2- 172.21.15.44:0/3333534582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 0x7f963815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 -- 172.21.15.44:0/3333534582 >> 172.21.15.44:0/3333534582 conn(0x7f96380b0fb0 msgr2=0x7f96380b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 -- 172.21.15.44:0/3333534582 shutdown_connections 2024-07-17T07:12:22.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.332+0000 7f963d600640 1 -- 172.21.15.44:0/3333534582 wait complete. 2024-07-17T07:12:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.333+0000 7f963d600640 1 Processor -- start 2024-07-17T07:12:22.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.333+0000 7f963d600640 1 -- start start 2024-07-17T07:12:22.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 0x7f9638150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 0x7f963814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96380b2e60 con 0x7f963806ee90 2024-07-17T07:12:22.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96380b2ce0 con 0x7f963806ea50 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.334+0000 7f963d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f96380b2fe0 con 0x7f96380ae810 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47092/0 (socket says 172.21.15.44:47092) 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 -- 172.21.15.44:0/1269502476 learned_addr learned my addr 172.21.15.44:0/1269502476 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f9636a00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 0x7f9638150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f9637400640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 0x7f963814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 msgr2=0x7f963814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 0x7f963814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 msgr2=0x7f9638150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 0x7f9638150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 -- 172.21.15.44:0/1269502476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963814e6e0 con 0x7f963806ee90 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f9636a00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 0x7f9638150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:22.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.335+0000 7f962fe00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9620009950 tx=0x7f9620032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.336+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620040070 con 0x7f963806ee90 2024-07-17T07:12:22.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.336+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9620005310 con 0x7f963806ee90 2024-07-17T07:12:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.336+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620005c90 con 0x7f963806ee90 2024-07-17T07:12:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.336+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963814e9c0 con 0x7f963806ee90 2024-07-17T07:12:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.336+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963814ef80 con 0x7f963806ee90 2024-07-17T07:12:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.337+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96380ac920 con 0x7f963806ee90 2024-07-17T07:12:22.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.338+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7f9620039040 con 0x7f963806ee90 2024-07-17T07:12:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.340+0000 7f962ea00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 0x7f9614079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.340+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f96200b9d60 con 0x7f963806ee90 2024-07-17T07:12:22.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.340+0000 7f9636a00640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 msgr2=0x7f9614079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:22.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.340+0000 7f9636a00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 0x7f9614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:22.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.344+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9620082a80 con 0x7f963806ee90 2024-07-17T07:12:22.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.542+0000 7f9636a00640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 msgr2=0x7f9614079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:22.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.542+0000 7f9636a00640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 0x7f9614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:22.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.580+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96380b23b0 con 0x7f963806ee90 2024-07-17T07:12:22.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.582+0000 7f962ea00640 1 -- 172.21.15.44:0/1269502476 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+18228 (secure 0 0 0) 0x7f9620082380 con 0x7f963806ee90 2024-07-17T07:12:22.582 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:22.583 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":100,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:18.043368+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:01.050217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:02.180466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2672990180":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/305038534":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/4259064219":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/1740207257":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/3153598340":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/1282726969":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6801/2446491909":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6800/2446491909":"2024-07-18T07:12:18.043292+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:22.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 msgr2=0x7f9614079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:22.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 0x7f9614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 msgr2=0x7f963814d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9620009950 tx=0x7f9620032c10 comp rx=0 tx=0).stop 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 shutdown_connections 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f9614076cd0 0x7f9614079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96380ae810 0x7f963814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f963806ee90 0x7f963814d320 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 --2- 172.21.15.44:0/1269502476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f963806ea50 0x7f9638150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 >> 172.21.15.44:0/1269502476 conn(0x7f96380b0fb0 msgr2=0x7f96380aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 shutdown_connections 2024-07-17T07:12:22.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.585+0000 7f963d600640 1 -- 172.21.15.44:0/1269502476 wait complete. 2024-07-17T07:12:22.602 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:12:22.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 -- 172.21.15.44:0/1218013871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c06bf40 msgr2=0x7f934c0cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 --2- 172.21.15.44:0/1218013871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c06bf40 0x7f934c0cfd00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f933c009830 tx=0x7f933c02a500 comp rx=0 tx=0).stop 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 -- 172.21.15.44:0/1218013871 shutdown_connections 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 --2- 172.21.15.44:0/1218013871 >> v2:172.21.15.175:3300/0 conn(0x7f934c15ef10 0x7f934c06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 --2- 172.21.15.44:0/1218013871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c06bf40 0x7f934c0cfd00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 -- 172.21.15.44:0/1218013871 >> 172.21.15.44:0/1218013871 conn(0x7f934c0b2240 msgr2=0x7f934c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 -- 172.21.15.44:0/1218013871 shutdown_connections 2024-07-17T07:12:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.798+0000 7f9352c00640 1 -- 172.21.15.44:0/1218013871 wait complete. 2024-07-17T07:12:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.800+0000 7f9352c00640 1 Processor -- start 2024-07-17T07:12:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.800+0000 7f9352c00640 1 -- start start 2024-07-17T07:12:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934c15b7f0 0x7f934c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c15ef10 0x7f934c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f934c15bfd0 con 0x7f934c06bf40 2024-07-17T07:12:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f934c15be50 con 0x7f934c15b7f0 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9352c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f934c06eea0 con 0x7f934c15ef10 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f934be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f934be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47104/0 (socket says 172.21.15.44:47104) 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f934be00640 1 -- 172.21.15.44:0/4131251720 learned_addr learned my addr 172.21.15.44:0/4131251720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f934b400640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934c15b7f0 0x7f934c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.801+0000 7f9350c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c15ef10 0x7f934c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c15ef10 msgr2=0x7f934c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c15ef10 0x7f934c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934c15b7f0 msgr2=0x7f934c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934c15b7f0 0x7f934c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 -- 172.21.15.44:0/4131251720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0af210 con 0x7f934c06bf40 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f934be00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f933c02aaf0 tx=0x7f933c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c040070 con 0x7f934c06bf40 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f933c005330 con 0x7f934c06bf40 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c0ada40 con 0x7f934c06bf40 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c0047a0 con 0x7f934c06bf40 2024-07-17T07:12:22.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.802+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c159130 con 0x7f934c06bf40 2024-07-17T07:12:22.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.804+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c0cfd00 con 0x7f934c06bf40 2024-07-17T07:12:22.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.805+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7f933c039040 con 0x7f934c06bf40 2024-07-17T07:12:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.806+0000 7f9343e00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 0x7f932c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.806+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f933c0b9a30 con 0x7f934c06bf40 2024-07-17T07:12:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.806+0000 7f934b400640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 msgr2=0x7f932c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:22.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.806+0000 7f934b400640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 0x7f932c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:22.811+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f933c0827e0 con 0x7f934c06bf40 2024-07-17T07:12:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.007+0000 7f934b400640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 msgr2=0x7f932c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:23.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.007+0000 7f934b400640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 0x7f932c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:23.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.103+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f934c0ccc70 con 0x7f934c06bf40 2024-07-17T07:12:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.104+0000 7f9343e00640 1 -- 172.21.15.44:0/4131251720 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7f933c0820e0 con 0x7f934c06bf40 2024-07-17T07:12:23.104 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:23.104 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 msgr2=0x7f932c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 0x7f932c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 msgr2=0x7f934c158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f933c02aaf0 tx=0x7f933c032c10 comp rx=0 tx=0).stop 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 shutdown_connections 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f932c076cd0 0x7f932c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934c15ef10 0x7f934c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934c15b7f0 0x7f934c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 --2- 172.21.15.44:0/4131251720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f934c06bf40 0x7f934c158bf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 >> 172.21.15.44:0/4131251720 conn(0x7f934c0b2240 msgr2=0x7f934c06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 shutdown_connections 2024-07-17T07:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.107+0000 7f9352c00640 1 -- 172.21.15.44:0/4131251720 wait complete. 2024-07-17T07:12:23.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:23.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 -- 172.21.15.44:0/1677351256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900b8c80 msgr2=0x7f61900b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 --2- 172.21.15.44:0/1677351256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900b8c80 0x7f61900b4ec0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6180009860 tx=0x7f618002a540 comp rx=0 tx=0).stop 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 -- 172.21.15.44:0/1677351256 shutdown_connections 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 --2- 172.21.15.44:0/1677351256 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 0x7f619015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 --2- 172.21.15.44:0/1677351256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900b8c80 0x7f61900b4ec0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 -- 172.21.15.44:0/1677351256 >> 172.21.15.44:0/1677351256 conn(0x7f61900cfec0 msgr2=0x7f61900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 -- 172.21.15.44:0/1677351256 shutdown_connections 2024-07-17T07:12:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.326+0000 7f6195400640 1 -- 172.21.15.44:0/1677351256 wait complete. 2024-07-17T07:12:23.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.328+0000 7f6195400640 1 Processor -- start 2024-07-17T07:12:23.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.328+0000 7f6195400640 1 -- start start 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 0x7f61901f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61900b8c80 0x7f619015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f619006e0c0 con 0x7f61900ce5c0 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61900b92f0 con 0x7f61900b8c80 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6195400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f619006e240 con 0x7f61900b5400 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6194a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6194a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47112/0 (socket says 172.21.15.44:47112) 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.329+0000 7f6194a00640 1 -- 172.21.15.44:0/1482819703 learned_addr learned my addr 172.21.15.44:0/1482819703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f618fe00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 0x7f61901f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f618f400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61900b8c80 0x7f619015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 msgr2=0x7f61901f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 0x7f61901f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61900b8c80 msgr2=0x7f619015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61900b8c80 0x7f619015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 -- 172.21.15.44:0/1482819703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61900aacd0 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.330+0000 7f6194a00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f618800d780 tx=0x7f618800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.331+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188017070 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.331+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f618800de90 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.331+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188005020 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.331+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61900aafb0 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.331+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901f4320 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.332+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619006e6f0 con 0x7f61900ce5c0 2024-07-17T07:12:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.333+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7f61880051c0 con 0x7f61900ce5c0 2024-07-17T07:12:23.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.334+0000 7f6187e00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 0x7f616c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.335+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f6188098ae0 con 0x7f61900ce5c0 2024-07-17T07:12:23.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.335+0000 7f618fe00640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 msgr2=0x7f616c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:23.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.335+0000 7f618fe00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 0x7f616c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:23.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.339+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6188061890 con 0x7f61900ce5c0 2024-07-17T07:12:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.536+0000 7f618fe00640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 msgr2=0x7f616c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:23.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.536+0000 7f618fe00640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 0x7f616c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:23.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.591+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61900b5d70 con 0x7f61900ce5c0 2024-07-17T07:12:23.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.592+0000 7f6187e00640 1 -- 172.21.15.44:0/1482819703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5228 (secure 0 0 0) 0x7f6188061190 con 0x7f61900ce5c0 2024-07-17T07:12:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:23.593 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":43,"btime":"2024-07-17T07:12:06:754604+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:03.214380+0000","modified":"2024-07-17T07:12:06.267491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6669},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6669":{"gid":6669,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2060644552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2060644552},{"type":"v1","addr":"172.21.15.44:6835","nonce":2060644552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6669,"qdb_cluster":[6669]},"id":5}]} 2024-07-17T07:12:23.593 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 43 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 msgr2=0x7f616c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 0x7f616c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 msgr2=0x7f61900aa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f618800d780 tx=0x7f618800dc40 comp rx=0 tx=0).stop 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 shutdown_connections 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f616c076cd0 0x7f616c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61900ce5c0 0x7f61900aa330 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61900b8c80 0x7f619015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.595+0000 7f6195400640 1 --2- 172.21.15.44:0/1482819703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61900b5400 0x7f61901f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.596+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 >> 172.21.15.44:0/1482819703 conn(0x7f61900cfec0 msgr2=0x7f61900b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:23.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.596+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 shutdown_connections 2024-07-17T07:12:23.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.596+0000 7f6195400640 1 -- 172.21.15.44:0/1482819703 wait complete. 2024-07-17T07:12:23.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.805+0000 7f77a6400640 1 -- 172.21.15.44:0/2677575617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0146cb0 msgr2=0x7f77a01610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.805+0000 7f77a6400640 1 --2- 172.21.15.44:0/2677575617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0146cb0 0x7f77a01610c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7790009890 tx=0x7f779002a5a0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.806+0000 7f77a6400640 1 -- 172.21.15.44:0/2677575617 shutdown_connections 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.806+0000 7f77a6400640 1 --2- 172.21.15.44:0/2677575617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0146cb0 0x7f77a01610c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.806+0000 7f77a6400640 1 -- 172.21.15.44:0/2677575617 >> 172.21.15.44:0/2677575617 conn(0x7f77a00b2240 msgr2=0x7f77a00b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.806+0000 7f77a6400640 1 -- 172.21.15.44:0/2677575617 shutdown_connections 2024-07-17T07:12:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.806+0000 7f77a6400640 1 -- 172.21.15.44:0/2677575617 wait complete. 2024-07-17T07:12:23.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.808+0000 7f77a6400640 1 Processor -- start 2024-07-17T07:12:23.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.808+0000 7f77a6400640 1 -- start start 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.809+0000 7f77a6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 0x7f77a014bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.809+0000 7f77a6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.809+0000 7f77a6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 0x7f77a013f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f77a6400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77a00b4170 con 0x7f77a0146cb0 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f77a6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77a00b3ff0 con 0x7f77a0147090 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f77a6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77a00b42f0 con 0x7f77a006c020 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f779ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f779ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47118/0 (socket says 172.21.15.44:47118) 2024-07-17T07:12:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f779ea00640 1 -- 172.21.15.44:0/2735119461 learned_addr learned my addr 172.21.15.44:0/2735119461 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:23.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f779f400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 0x7f77a014bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.810+0000 7f779fe00640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 0x7f77a013f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 msgr2=0x7f77a014bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 0x7f77a014bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 msgr2=0x7f77a013f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 0x7f77a013f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 -- 172.21.15.44:0/2735119461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a01f8970 con 0x7f77a0146cb0 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779f400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 0x7f77a014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:23.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779fe00640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 0x7f77a013f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f779ea00640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c480 tx=0x7f778c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0052d0 con 0x7f77a0146cb0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f778c005470 con 0x7f77a0146cb0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c012870 con 0x7f77a0146cb0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a0143cd0 con 0x7f77a0146cb0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.811+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a0141010 con 0x7f77a0146cb0 2024-07-17T07:12:23.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.813+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a00b47e0 con 0x7f77a0146cb0 2024-07-17T07:12:23.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.814+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7f778c0035f0 con 0x7f77a0146cb0 2024-07-17T07:12:23.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.815+0000 7f7797400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 0x7f7780079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:23.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.815+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7f778c098610 con 0x7f77a0146cb0 2024-07-17T07:12:23.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.815+0000 7f779f400640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 msgr2=0x7f7780079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:23.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.815+0000 7f779f400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 0x7f7780079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:23.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:23.819+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f77a01f8970 con 0x7f77a0146cb0 2024-07-17T07:12:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.017+0000 7f779f400640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 msgr2=0x7f7780079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:24.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.017+0000 7f779f400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 0x7f7780079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:24.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.021+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77a00b07b0 con 0x7f77a0146cb0 2024-07-17T07:12:24.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.024+0000 7f7797400640 1 -- 172.21.15.44:0/2735119461 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+18228 (secure 0 0 0) 0x7f778c0614d0 con 0x7f77a0146cb0 2024-07-17T07:12:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:24.025 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":100,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:18.043368+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:01.050217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:02.180466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2672990180":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/305038534":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/4259064219":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/1740207257":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/3153598340":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/1282726969":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6801/2446491909":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6800/2446491909":"2024-07-18T07:12:18.043292+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:24.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 msgr2=0x7f7780079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 0x7f7780079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 msgr2=0x7f77a01f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c480 tx=0x7f778c00c940 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 shutdown_connections 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7f7780076cd0 0x7f7780079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0147090 0x7f77a013f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a0146cb0 0x7f77a01f8430 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 --2- 172.21.15.44:0/2735119461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a006c020 0x7f77a014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.027+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 >> 172.21.15.44:0/2735119461 conn(0x7f77a00b2240 msgr2=0x7f77a00b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.028+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 shutdown_connections 2024-07-17T07:12:24.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.028+0000 7f77a6400640 1 -- 172.21.15.44:0/2735119461 wait complete. 2024-07-17T07:12:24.046 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:12:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 -- 172.21.15.44:0/536379879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa774004d40 msgr2=0x7fa7740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 --2- 172.21.15.44:0/536379879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa774004d40 0x7fa7740051c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa788005fd0 tx=0x7fa774005830 comp rx=0 tx=0).stop 2024-07-17T07:12:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa796600640 1 -- 172.21.15.44:0/536379879 reap_dead start 2024-07-17T07:12:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 -- 172.21.15.44:0/536379879 shutdown_connections 2024-07-17T07:12:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 -- 172.21.15.44:0/536379879 >> 172.21.15.44:0/536379879 conn(0x7fa790073d50 msgr2=0x7fa79009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:24.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 -- 172.21.15.44:0/536379879 shutdown_connections 2024-07-17T07:12:24.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.239+0000 7fa798600640 1 -- 172.21.15.44:0/536379879 wait complete. 2024-07-17T07:12:24.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.241+0000 7fa798600640 1 Processor -- start 2024-07-17T07:12:24.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.241+0000 7fa798600640 1 -- start start 2024-07-17T07:12:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 0x7fa7901f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa790153300 0x7fa79009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900b2080 con 0x7fa79014f670 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900b1f00 con 0x7fa774004d40 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa798600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7900b2200 con 0x7fa790153300 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa796600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa796600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47136/0 (socket says 172.21.15.44:47136) 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa796600640 1 -- 172.21.15.44:0/3773334937 learned_addr learned my addr 172.21.15.44:0/3773334937 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa795c00640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 0x7fa7901f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa795200640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa790153300 0x7fa79009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.242+0000 7fa796600640 1 -- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa790153300 msgr2=0x7fa79009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa796600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa790153300 0x7fa79009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa796600640 1 -- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 msgr2=0x7fa7901f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa796600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 0x7fa7901f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa796600640 1 -- 172.21.15.44:0/3773334937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7901f1e80 con 0x7fa79014f670 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa795c00640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 0x7fa7901f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa796600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa788005fd0 tx=0x7fa788033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788043070 con 0x7fa79014f670 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa788033d60 con 0x7fa79014f670 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788038040 con 0x7fa79014f670 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa790153790 con 0x7fa79014f670 2024-07-17T07:12:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.243+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa790153d50 con 0x7fa79014f670 2024-07-17T07:12:24.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.244+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7900b4f10 con 0x7fa79014f670 2024-07-17T07:12:24.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.246+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 44) ==== 98932+0+0 (secure 0 0 0) 0x7fa788003bd0 con 0x7fa79014f670 2024-07-17T07:12:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.247+0000 7fa78d600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 0x7fa7700791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.247+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6965+0+0 (secure 0 0 0) 0x7fa7880b86c0 con 0x7fa79014f670 2024-07-17T07:12:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.247+0000 7fa795c00640 1 -- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 msgr2=0x7fa7700791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1810999329 2024-07-17T07:12:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.247+0000 7fa795c00640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 0x7fa7700791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:24.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.255+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa788080e10 con 0x7fa79014f670 2024-07-17T07:12:24.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.384+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 45) ==== 98540+0+0 (secure 0 0 0) 0x7fa78807d0a0 con 0x7fa79014f670 2024-07-17T07:12:24.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.384+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 msgr2=0x7fa7700791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.384+0000 7fa78d600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 0x7fa7700791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.461 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f444dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f444dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f444dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f444dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f444dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f4450e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f4450e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f4450e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f4450e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.462 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:24.462+0000 7f4450e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.515+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa79015e000 con 0x7fa79014f670 2024-07-17T07:12:24.515 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:12:24.517+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:12:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.520+0000 7fa78d600640 1 -- 172.21.15.44:0/3773334937 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7fa7880a0000 con 0x7fa79014f670 2024-07-17T07:12:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 msgr2=0x7fa79006fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa788005fd0 tx=0x7fa788033bf0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 shutdown_connections 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:6800/1810999329,v1:172.21.15.175:6801/1810999329] conn(0x7fa770076d00 0x7fa7700791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79014f670 0x7fa79006fd30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa790153300 0x7fa79009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 --2- 172.21.15.44:0/3773334937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa774004d40 0x7fa7901f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.523+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 >> 172.21.15.44:0/3773334937 conn(0x7fa790073d50 msgr2=0x7fa79015e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.524+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 shutdown_connections 2024-07-17T07:12:24.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.525+0000 7fa798600640 1 -- 172.21.15.44:0/3773334937 wait complete. 2024-07-17T07:12:24.533 INFO:tasks.ceph.mds.c.smithi044.stderr: -2688> 2024-07-17T07:12:08.689+0000 7f063c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:12:24.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:12:24.593 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:12:24.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 -- 172.21.15.44:0/3322430066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 msgr2=0x7f0b1c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 --2- 172.21.15.44:0/3322430066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b1c003e50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0b28006f80 tx=0x7f0b1c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 -- 172.21.15.44:0/3322430066 shutdown_connections 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 --2- 172.21.15.44:0/3322430066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b1c003e50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 -- 172.21.15.44:0/3322430066 >> 172.21.15.44:0/3322430066 conn(0x7f0b380d3700 msgr2=0x7f0b380d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 -- 172.21.15.44:0/3322430066 shutdown_connections 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.723+0000 7f0b3da00640 1 -- 172.21.15.44:0/3322430066 wait complete. 2024-07-17T07:12:24.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.725+0000 7f0b3da00640 1 Processor -- start 2024-07-17T07:12:24.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.725+0000 7f0b3da00640 1 -- start start 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b38160560 0x7f0b380d2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38153750 0x7f0b380d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380cea00 con 0x7f0b1c003a50 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b380ce880 con 0x7f0b38153750 2024-07-17T07:12:24.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b3da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b380ceb80 con 0x7f0b38160560 2024-07-17T07:12:24.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b36a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b36a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47168/0 (socket says 172.21.15.44:47168) 2024-07-17T07:12:24.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b36a00640 1 -- 172.21.15.44:0/908668287 learned_addr learned my addr 172.21.15.44:0/908668287 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:24.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b37400640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38153750 0x7f0b380d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.727+0000 7f0b2e000640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b38160560 0x7f0b380d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 -- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b38160560 msgr2=0x7f0b380d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b38160560 0x7f0b380d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 -- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38153750 msgr2=0x7f0b380d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38153750 0x7f0b380d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 -- 172.21.15.44:0/908668287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38153f50 con 0x7f0b1c003a50 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b36a00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0b28006f80 tx=0x7f0b28003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28032030 con 0x7f0b1c003a50 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b280014f0 con 0x7f0b1c003a50 2024-07-17T07:12:24.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2802e680 con 0x7f0b1c003a50 2024-07-17T07:12:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381f7790 con 0x7f0b1c003a50 2024-07-17T07:12:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.728+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381f7d50 con 0x7f0b1c003a50 2024-07-17T07:12:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.730+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b38161940 con 0x7f0b1c003a50 2024-07-17T07:12:24.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.731+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 45) ==== 98540+0+0 (secure 0 0 0) 0x7f0b28035020 con 0x7f0b1c003a50 2024-07-17T07:12:24.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.732+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 7395+0+0 (secure 0 0 0) 0x7f0b280b88f0 con 0x7f0b1c003a50 2024-07-17T07:12:24.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.736+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0b28047030 con 0x7f0b1c003a50 2024-07-17T07:12:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:24.977+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0b380b89e0 con 0x7f0b1c003a50 2024-07-17T07:12:25.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.393+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 46) ==== 98667+0+0 (secure 0 0 0) 0x7f0b2807ccf0 con 0x7f0b1c003a50 2024-07-17T07:12:25.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.395+0000 7f0b2f400640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 0x7f0b1407f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:25.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.396+0000 7f0b2e000640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 0x7f0b1407f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.401+0000 7f0b2e000640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 0x7f0b1407f460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005fd0 tx=0x7f0b20005c50 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:25.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.523+0000 7f0b2f400640 1 -- 172.21.15.44:0/908668287 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f0b280808d0 con 0x7f0b1c003a50 2024-07-17T07:12:25.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 msgr2=0x7f0b1407f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:25.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 0x7f0b1407f460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005fd0 tx=0x7f0b20005c50 comp rx=0 tx=0).stop 2024-07-17T07:12:25.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 msgr2=0x7f0b38160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:25.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0b28006f80 tx=0x7f0b28003d50 comp rx=0 tx=0).stop 2024-07-17T07:12:25.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 shutdown_connections 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0b1407d070 0x7f0b1407f460 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38153750 0x7f0b380d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b38160560 0x7f0b380d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 --2- 172.21.15.44:0/908668287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b1c003a50 0x7f0b38160020 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.527+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 >> 172.21.15.44:0/908668287 conn(0x7f0b380d3700 msgr2=0x7f0b380b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.528+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 shutdown_connections 2024-07-17T07:12:25.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.528+0000 7f0b3da00640 1 -- 172.21.15.44:0/908668287 wait complete. 2024-07-17T07:12:25.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.756+0000 7f4180a00640 1 -- 172.21.15.44:0/3224682509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c070850 msgr2=0x7f417c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.756+0000 7f4180a00640 1 --2- 172.21.15.44:0/3224682509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c070850 0x7f417c06d970 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4168009900 tx=0x7f416802a580 comp rx=0 tx=0).stop 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.757+0000 7f4180a00640 1 -- 172.21.15.44:0/3224682509 shutdown_connections 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.757+0000 7f4180a00640 1 --2- 172.21.15.44:0/3224682509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c070850 0x7f417c06d970 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.757+0000 7f4180a00640 1 -- 172.21.15.44:0/3224682509 >> 172.21.15.44:0/3224682509 conn(0x7f417c0b3020 msgr2=0x7f417c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.757+0000 7f4180a00640 1 -- 172.21.15.44:0/3224682509 shutdown_connections 2024-07-17T07:12:25.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.757+0000 7f4180a00640 1 -- 172.21.15.44:0/3224682509 wait complete. 2024-07-17T07:12:25.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.758+0000 7f4180a00640 1 Processor -- start 2024-07-17T07:12:25.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.759+0000 7f4180a00640 1 -- start start 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f417c070850 0x7f417c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f417c0cfb20 0x7f417c0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f417c06e030 con 0x7f417c160c10 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f417c06deb0 con 0x7f417c070850 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4180a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f417c0d06a0 con 0x7f417c0cfb20 2024-07-17T07:12:25.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47194/0 (socket says 172.21.15.44:47194) 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 -- 172.21.15.44:0/4129218431 learned_addr learned my addr 172.21.15.44:0/4129218431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4179e00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f417c070850 0x7f417c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f4179400640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f417c0cfb20 0x7f417c0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 -- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f417c0cfb20 msgr2=0x7f417c0b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f417c0cfb20 0x7f417c0b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 -- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f417c070850 msgr2=0x7f417c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f417c070850 0x7f417c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.760+0000 7f417a800640 1 -- 172.21.15.44:0/4129218431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c0b4320 con 0x7f417c160c10 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f417a800640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f417400c780 tx=0x7f417400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004b60 con 0x7f417c160c10 2024-07-17T07:12:25.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4174004df0 con 0x7f417c160c10 2024-07-17T07:12:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417400aa70 con 0x7f417c160c10 2024-07-17T07:12:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c0b45c0 con 0x7f417c160c10 2024-07-17T07:12:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.761+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c0b4b80 con 0x7f417c160c10 2024-07-17T07:12:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.763+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c0d0b50 con 0x7f417c160c10 2024-07-17T07:12:25.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.764+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 46) ==== 98667+0+0 (secure 0 0 0) 0x7f417400ac10 con 0x7f417c160c10 2024-07-17T07:12:25.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.765+0000 7f4172000640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 0x7f415c078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:25.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.765+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 7395+0+0 (secure 0 0 0) 0x7f4174098940 con 0x7f417c160c10 2024-07-17T07:12:25.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.765+0000 7f4179e00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 0x7f415c078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:25.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.766+0000 7f4179e00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 0x7f415c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f416c0015e0 tx=0x7f416c009040 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:25.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.769+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4174020070 con 0x7f417c160c10 2024-07-17T07:12:25.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:25.981+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f417c150fe0 con 0x7f417c160c10 2024-07-17T07:12:26.444 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:26.444+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.528+0000 7f4172000640 1 -- 172.21.15.44:0/4129218431 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f41740619e0 con 0x7f417c160c10 2024-07-17T07:12:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:12:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 msgr2=0x7f415c078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 0x7f415c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f416c0015e0 tx=0x7f416c009040 comp rx=0 tx=0).stop 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 msgr2=0x7f417c1f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f417400c780 tx=0x7f417400cc40 comp rx=0 tx=0).stop 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 shutdown_connections 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f415c076a00 0x7f415c078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f417c160c10 0x7f417c1f7c30 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f417c0cfb20 0x7f417c0b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 --2- 172.21.15.44:0/4129218431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f417c070850 0x7f417c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.532+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 >> 172.21.15.44:0/4129218431 conn(0x7f417c0b3020 msgr2=0x7f417c0b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.533+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 shutdown_connections 2024-07-17T07:12:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.533+0000 7f4180a00640 1 -- 172.21.15.44:0/4129218431 wait complete. 2024-07-17T07:12:26.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:12:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 -- 172.21.15.44:0/3563600039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c0cb880 msgr2=0x7f6c7c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 --2- 172.21.15.44:0/3563600039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c0cb880 0x7f6c7c06db60 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005fd0 tx=0x7f6c6402a5a0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 -- 172.21.15.44:0/3563600039 shutdown_connections 2024-07-17T07:12:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 --2- 172.21.15.44:0/3563600039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c0cb880 0x7f6c7c06db60 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 --2- 172.21.15.44:0/3563600039 >> v2:172.21.15.175:3300/0 conn(0x7f6c7c15b060 0x7f6c7c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 -- 172.21.15.44:0/3563600039 >> 172.21.15.44:0/3563600039 conn(0x7f6c7c0b0eb0 msgr2=0x7f6c7c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 -- 172.21.15.44:0/3563600039 shutdown_connections 2024-07-17T07:12:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.768+0000 7f6c80a00640 1 -- 172.21.15.44:0/3563600039 wait complete. 2024-07-17T07:12:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.770+0000 7f6c80a00640 1 Processor -- start 2024-07-17T07:12:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.770+0000 7f6c80a00640 1 -- start start 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0cb880 0x7f6c7c15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15c220 0x7f6c7c1f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c15b5c0 con 0x7f6c7c15b060 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c15b440 con 0x7f6c7c15c220 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c80a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c15f4a0 con 0x7f6c7c0cb880 2024-07-17T07:12:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47216/0 (socket says 172.21.15.44:47216) 2024-07-17T07:12:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.771+0000 7f6c71400640 1 -- 172.21.15.44:0/61031354 learned_addr learned my addr 172.21.15.44:0/61031354 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 -- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0cb880 msgr2=0x7f6c7c15a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0cb880 0x7f6c7c15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 -- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15c220 msgr2=0x7f6c7c1f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15c220 0x7f6c7c1f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 -- 172.21.15.44:0/61031354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1f40a0 con 0x7f6c7c15b060 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c71400640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009850 tx=0x7f6c640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6403f070 con 0x7f6c7c15b060 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c64004890 con 0x7f6c7c15b060 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64035030 con 0x7f6c7c15b060 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.772+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1f4380 con 0x7f6c7c15b060 2024-07-17T07:12:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.773+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c1f4940 con 0x7f6c7c15b060 2024-07-17T07:12:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.774+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c158790 con 0x7f6c7c15b060 2024-07-17T07:12:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.775+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f6c64042070 con 0x7f6c7c15b060 2024-07-17T07:12:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.776+0000 7f6c72a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 0x7f6c58079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.776+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6957+0+0 (secure 0 0 0) 0x7f6c640b9790 con 0x7f6c7c15b060 2024-07-17T07:12:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.777+0000 7f6c79e00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 0x7f6c58079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.778+0000 7f6c79e00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 0x7f6c58079100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c158580 tx=0x7f6c6c009250 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:26.780+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6c64082680 con 0x7f6c7c15b060 2024-07-17T07:12:27.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.004+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c7c15f9c0 con 0x7f6c7c15b060 2024-07-17T07:12:27.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:27.349+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:27.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.546+0000 7f6c72a00640 1 -- 172.21.15.44:0/61031354 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f6c64081f80 con 0x7f6c7c15b060 2024-07-17T07:12:27.546 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.550+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 msgr2=0x7f6c58079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.550+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 0x7f6c58079100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c158580 tx=0x7f6c6c009250 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.550+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 msgr2=0x7f6c7c0c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.550+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009850 tx=0x7f6c640039e0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 shutdown_connections 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6c58076c40 0x7f6c58079100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15c220 0x7f6c7c1f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c7c15b060 0x7f6c7c0c97f0 secure :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009850 tx=0x7f6c640039e0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 --2- 172.21.15.44:0/61031354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0cb880 0x7f6c7c15a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.551+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 >> 172.21.15.44:0/61031354 conn(0x7f6c7c0b0eb0 msgr2=0x7f6c7c15d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.555+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 shutdown_connections 2024-07-17T07:12:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.555+0000 7f6c80a00640 1 -- 172.21.15.44:0/61031354 wait complete. 2024-07-17T07:12:27.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.774+0000 7f6272400640 1 -- 172.21.15.44:0/2651232768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 msgr2=0x7f626c0b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.774+0000 7f6272400640 1 --2- 172.21.15.44:0/2651232768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c0b21b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6254005fd0 tx=0x7f625402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.774+0000 7f6272400640 1 -- 172.21.15.44:0/2651232768 shutdown_connections 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.774+0000 7f6272400640 1 --2- 172.21.15.44:0/2651232768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.774+0000 7f6272400640 1 --2- 172.21.15.44:0/2651232768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c0b21b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.775+0000 7f6272400640 1 -- 172.21.15.44:0/2651232768 >> 172.21.15.44:0/2651232768 conn(0x7f626c0b0eb0 msgr2=0x7f626c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.775+0000 7f6272400640 1 -- 172.21.15.44:0/2651232768 shutdown_connections 2024-07-17T07:12:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.775+0000 7f6272400640 1 -- 172.21.15.44:0/2651232768 wait complete. 2024-07-17T07:12:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.776+0000 7f6272400640 1 Processor -- start 2024-07-17T07:12:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.776+0000 7f6272400640 1 -- start start 2024-07-17T07:12:27.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f626c15a280 0x7f626c1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f626c15b150 con 0x7f626c0ac8a0 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f626c06f1d0 con 0x7f626c15a280 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.777+0000 7f6272400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f626c15b2d0 con 0x7f626c0b1af0 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47238/0 (socket says 172.21.15.44:47238) 2024-07-17T07:12:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 -- 172.21.15.44:0/2936596144 learned_addr learned my addr 172.21.15.44:0/2936596144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:27.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f6262a00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626be00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f626c15a280 0x7f626c1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 -- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 msgr2=0x7f626c1ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c1ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 -- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f626c15a280 msgr2=0x7f626c1f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f626c15a280 0x7f626c1f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f626b400640 1 -- 172.21.15.44:0/2936596144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c155270 con 0x7f626c0ac8a0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.778+0000 7f6262a00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f626b400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c3a0 tx=0x7f625c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c017070 con 0x7f626c0ac8a0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f625c0053c0 con 0x7f626c0ac8a0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c003850 con 0x7f626c0ac8a0 2024-07-17T07:12:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c158240 con 0x7f626c0ac8a0 2024-07-17T07:12:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.779+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c158800 con 0x7f626c0ac8a0 2024-07-17T07:12:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.780+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c0ae040 con 0x7f626c0ac8a0 2024-07-17T07:12:27.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.782+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f625c01a070 con 0x7f626c0ac8a0 2024-07-17T07:12:27.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.783+0000 7f6263e00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 0x7f6248079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:27.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.783+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f625c0979f0 con 0x7f626c0ac8a0 2024-07-17T07:12:27.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.783+0000 7f6262a00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 0x7f6248079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:27.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.784+0000 7f6262a00640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 0x7f6248079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6254033000 tx=0x7f6254004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:27.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:27.787+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f626c0ae040 con 0x7f626c0ac8a0 2024-07-17T07:12:28.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.009+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f626c0b21b0 con 0x7f626c0ac8a0 2024-07-17T07:12:28.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.018+0000 7f6263e00640 1 -- 172.21.15.44:0/2936596144 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f625c060c20 con 0x7f626c0ac8a0 2024-07-17T07:12:28.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 msgr2=0x7f6248079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:28.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 0x7f6248079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6254033000 tx=0x7f6254004730 comp rx=0 tx=0).stop 2024-07-17T07:12:28.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 msgr2=0x7f626c158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c3a0 tx=0x7f625c00c860 comp rx=0 tx=0).stop 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 shutdown_connections 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f6248076c40 0x7f6248079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f626c15a280 0x7f626c1f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f626c0b1af0 0x7f626c1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 --2- 172.21.15.44:0/2936596144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f626c0ac8a0 0x7f626c158d20 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.021+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 >> 172.21.15.44:0/2936596144 conn(0x7f626c0b0eb0 msgr2=0x7f626c0cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.022+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 shutdown_connections 2024-07-17T07:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.022+0000 7f6272400640 1 -- 172.21.15.44:0/2936596144 wait complete. 2024-07-17T07:12:28.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2024-07-17T07:12:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 -- 172.21.15.44:0/3474415787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20600713c0 msgr2=0x7f2060071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 --2- 172.21.15.44:0/3474415787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20600713c0 0x7f2060071810 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2060075590 tx=0x7f205002aa10 comp rx=0 tx=0).stop 2024-07-17T07:12:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 -- 172.21.15.44:0/3474415787 shutdown_connections 2024-07-17T07:12:28.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 --2- 172.21.15.44:0/3474415787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f206006d150 0x7f206006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 --2- 172.21.15.44:0/3474415787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20600713c0 0x7f2060071810 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 --2- 172.21.15.44:0/3474415787 >> v2:172.21.15.175:3300/0 conn(0x7f2060070fe0 0x7f2060074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 -- 172.21.15.44:0/3474415787 >> 172.21.15.44:0/3474415787 conn(0x7f20600b3090 msgr2=0x7f20600b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 -- 172.21.15.44:0/3474415787 shutdown_connections 2024-07-17T07:12:28.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.248+0000 7f205f200640 1 -- 172.21.15.44:0/3474415787 wait complete. 2024-07-17T07:12:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.250+0000 7f205f200640 1 Processor -- start 2024-07-17T07:12:28.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.250+0000 7f205f200640 1 -- start start 2024-07-17T07:12:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2060070fe0 0x7f20601f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:28.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20600713c0 0x7f20600b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f206006db40 con 0x7f206006d150 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20600b5690 con 0x7f20600713c0 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f206006dcc0 con 0x7f2060070fe0 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.251+0000 7f205de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47244/0 (socket says 172.21.15.44:47244) 2024-07-17T07:12:28.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 -- 172.21.15.44:0/427520765 learned_addr learned my addr 172.21.15.44:0/427520765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205e800640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20600713c0 0x7f20600b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 -- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2060070fe0 msgr2=0x7f20601f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2060070fe0 0x7f20601f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 -- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20600713c0 msgr2=0x7f20600b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20600713c0 0x7f20600b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 -- 172.21.15.44:0/427520765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20601f7950 con 0x7f206006d150 2024-07-17T07:12:28.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.252+0000 7f205de00640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f204800c890 tx=0x7f204800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.253+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048004220 con 0x7f206006d150 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.253+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20480079b0 con 0x7f206006d150 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.253+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007de0 con 0x7f206006d150 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.253+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20601f7c30 con 0x7f206006d150 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.253+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20600b2800 con 0x7f206006d150 2024-07-17T07:12:28.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.254+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206006e250 con 0x7f206006d150 2024-07-17T07:12:28.254 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:28.254+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:28.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.255+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f2048005020 con 0x7f206006d150 2024-07-17T07:12:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.256+0000 7f2056000640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 0x7f203c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.256+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f2048098ac0 con 0x7f206006d150 2024-07-17T07:12:28.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.257+0000 7f205d400640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 0x7f203c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.258+0000 7f205d400640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 0x7f203c079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2050035000 tx=0x7f2050004a40 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.261+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2048014030 con 0x7f206006d150 2024-07-17T07:12:28.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:28.556+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) -- 0x7f20600cfd80 con 0x7f206006d150 2024-07-17T07:12:29.173 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:29.172+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:29.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.553+0000 7f2056000640 1 -- 172.21.15.44:0/427520765 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v659) ==== 166+0+0 (secure 0 0 0) 0x7f2048061be0 con 0x7f206006d150 2024-07-17T07:12:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 msgr2=0x7f203c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 0x7f203c079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2050035000 tx=0x7f2050004a40 comp rx=0 tx=0).stop 2024-07-17T07:12:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 msgr2=0x7f206014ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f204800c890 tx=0x7f204800cd50 comp rx=0 tx=0).stop 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 shutdown_connections 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f203c076c40 0x7f203c079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20600713c0 0x7f20600b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2060070fe0 0x7f20601f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 --2- 172.21.15.44:0/427520765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f206006d150 0x7f206014ae00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 >> 172.21.15.44:0/427520765 conn(0x7f20600b3090 msgr2=0x7f20600b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 shutdown_connections 2024-07-17T07:12:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.557+0000 7f205f200640 1 -- 172.21.15.44:0/427520765 wait complete. 2024-07-17T07:12:29.582 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2024-07-17T07:12:29.582 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2024-07-17T07:12:29.582 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:12:29.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 -- 172.21.15.44:0/99518615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 msgr2=0x7f1cd00ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/99518615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd00ac220 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005fd0 tx=0x7f1cbc02a530 comp rx=0 tx=0).stop 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 -- 172.21.15.44:0/99518615 shutdown_connections 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/99518615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd00ac220 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/99518615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 0x7f1cd015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 -- 172.21.15.44:0/99518615 >> 172.21.15.44:0/99518615 conn(0x7f1cd00b0fb0 msgr2=0x7f1cd00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.800+0000 7f1cd4a00640 1 -- 172.21.15.44:0/99518615 shutdown_connections 2024-07-17T07:12:29.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.801+0000 7f1cd4a00640 1 -- 172.21.15.44:0/99518615 wait complete. 2024-07-17T07:12:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.802+0000 7f1cd4a00640 1 Processor -- start 2024-07-17T07:12:29.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.802+0000 7f1cd4a00640 1 -- start start 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd006ee90 0x7f1cd014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 0x7f1cd0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00b2e60 con 0x7f1cd006ea50 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00b2ce0 con 0x7f1cd006ee90 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.803+0000 7f1cd4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd00b2fe0 con 0x7f1cd00ae810 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56030/0 (socket says 172.21.15.44:56030) 2024-07-17T07:12:29.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 -- 172.21.15.44:0/3474298985 learned_addr learned my addr 172.21.15.44:0/3474298985 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1cce800640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 0x7f1cd0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 -- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 msgr2=0x7f1cd0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 0x7f1cd0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 -- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd006ee90 msgr2=0x7f1cd014e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd006ee90 0x7f1cd014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 -- 172.21.15.44:0/3474298985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00735a0 con 0x7f1cd006ea50 2024-07-17T07:12:29.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.804+0000 7f1ccde00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ea50 tx=0x7f1cc000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.805+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000cdc0 con 0x7f1cd006ea50 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.805+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cc0011560 con 0x7f1cd006ea50 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.805+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd0072030 con 0x7f1cd006ea50 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.805+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0011990 con 0x7f1cd006ea50 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.805+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd0073b20 con 0x7f1cd006ea50 2024-07-17T07:12:29.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.806+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00b2b00 con 0x7f1cd006ea50 2024-07-17T07:12:29.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.807+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f1cc0011b30 con 0x7f1cd006ea50 2024-07-17T07:12:29.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.808+0000 7f1cc6000640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 0x7f1cb0079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:29.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.809+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f1cc0099420 con 0x7f1cd006ea50 2024-07-17T07:12:29.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.809+0000 7f1ccd400640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 0x7f1cb0079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:29.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.810+0000 7f1ccd400640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 0x7f1cb0079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc033000 tx=0x7f1cbc0046c0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:29.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:29.813+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1cd00b2b00 con 0x7f1cd006ea50 2024-07-17T07:12:30.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.032+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cd00ac920 con 0x7f1cd006ea50 2024-07-17T07:12:30.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.033+0000 7f1cc6000640 1 -- 172.21.15.44:0/3474298985 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+15086 (secure 0 0 0) 0x7f1cc0016030 con 0x7f1cd006ea50 2024-07-17T07:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":104,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:27.543175+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/2060644552":"2024-07-18T07:12:24.517124+0000","172.21.15.175:0/173171416":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/2081665747":"2024-07-18T07:12:24.367345+0000","172.21.15.44:6835/2060644552":"2024-07-18T07:12:24.517124+0000","172.21.15.175:6800/1810999329":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/1649965695":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/2672990180":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/305038534":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/532248064":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/4259064219":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/2249507766":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/1740207257":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/3153598340":"2024-07-18T07:12:18.043292+0000","172.21.15.175:0/3520109728":"2024-07-18T07:12:24.367345+0000","172.21.15.175:0/1282726969":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6801/2446491909":"2024-07-18T07:12:18.043292+0000","172.21.15.175:6801/1810999329":"2024-07-18T07:12:24.367345+0000","172.21.15.175:6800/2446491909":"2024-07-18T07:12:18.043292+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 msgr2=0x7f1cb0079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 0x7f1cb0079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc033000 tx=0x7f1cbc0046c0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 msgr2=0x7f1cd009cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ea50 tx=0x7f1cc000ef10 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 shutdown_connections 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1cb0076c40 0x7f1cb0079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cd00ae810 0x7f1cd0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cd006ee90 0x7f1cd014e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 --2- 172.21.15.44:0/3474298985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cd006ea50 0x7f1cd009cea0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.036+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 >> 172.21.15.44:0/3474298985 conn(0x7f1cd00b0fb0 msgr2=0x7f1cd00b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.037+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 shutdown_connections 2024-07-17T07:12:30.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.037+0000 7f1cd4a00640 1 -- 172.21.15.44:0/3474298985 wait complete. 2024-07-17T07:12:30.057 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:12:30.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:12:30.105 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:30.106+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.275+0000 7f2508e00640 1 -- 172.21.15.44:0/2865222055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504154600 msgr2=0x7f25040b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.275+0000 7f2508e00640 1 --2- 172.21.15.44:0/2865222055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504154600 0x7f25040b3ce0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f24f4005fd0 tx=0x7f24f402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 -- 172.21.15.44:0/2865222055 shutdown_connections 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 --2- 172.21.15.44:0/2865222055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25040ac190 0x7f25040ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 --2- 172.21.15.44:0/2865222055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504154600 0x7f25040b3ce0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 -- 172.21.15.44:0/2865222055 >> 172.21.15.44:0/2865222055 conn(0x7f25040b0eb0 msgr2=0x7f25040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 -- 172.21.15.44:0/2865222055 shutdown_connections 2024-07-17T07:12:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.276+0000 7f2508e00640 1 -- 172.21.15.44:0/2865222055 wait complete. 2024-07-17T07:12:30.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.277+0000 7f2508e00640 1 Processor -- start 2024-07-17T07:12:30.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.278+0000 7f2508e00640 1 -- start start 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25040ac190 0x7f25041f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2504154600 0x7f2504158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f250415e220 con 0x7f2504161120 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f250415e0a0 con 0x7f25040ac190 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2508e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f250415e3a0 con 0x7f2504154600 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56038/0 (socket says 172.21.15.44:56038) 2024-07-17T07:12:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 -- 172.21.15.44:0/3258547820 learned_addr learned my addr 172.21.15.44:0/3258547820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2501400640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2504154600 0x7f2504158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2501e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25040ac190 0x7f25041f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 -- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2504154600 msgr2=0x7f2504158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2504154600 0x7f2504158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 -- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25040ac190 msgr2=0x7f25041f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.279+0000 7f2502800640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25040ac190 0x7f25041f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f2502800640 1 -- 172.21.15.44:0/3258547820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25041f7d90 con 0x7f2504161120 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f2502800640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00eaa0 tx=0x7f24fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00cdb0 con 0x7f2504161120 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24fc004c10 con 0x7f2504161120 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc005020 con 0x7f2504161120 2024-07-17T07:12:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25041f8070 con 0x7f2504161120 2024-07-17T07:12:30.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.280+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25041f8630 con 0x7f2504161120 2024-07-17T07:12:30.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.282+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250415e930 con 0x7f2504161120 2024-07-17T07:12:30.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.283+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f24fc0051c0 con 0x7f2504161120 2024-07-17T07:12:30.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.284+0000 7f24fa000640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 0x7f24e0079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.284+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f24fc099190 con 0x7f2504161120 2024-07-17T07:12:30.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.284+0000 7f2501e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 0x7f24e0079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.286+0000 7f2501e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 0x7f24e0079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24f4033000 tx=0x7f24f4004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:30.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.288+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f24fc016030 con 0x7f2504161120 2024-07-17T07:12:30.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.509+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2504160e90 con 0x7f2504161120 2024-07-17T07:12:30.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.510+0000 7f24fa000640 1 -- 172.21.15.44:0/3258547820 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f24fc068030 con 0x7f2504161120 2024-07-17T07:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.512+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 msgr2=0x7f24e0079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 0x7f24e0079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24f4033000 tx=0x7f24f4004730 comp rx=0 tx=0).stop 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 msgr2=0x7f2504158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00eaa0 tx=0x7f24fc00ef60 comp rx=0 tx=0).stop 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 shutdown_connections 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f24e0076c40 0x7f24e0079100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2504161120 0x7f2504158860 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2504154600 0x7f2504158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 --2- 172.21.15.44:0/3258547820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25040ac190 0x7f25041f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 >> 172.21.15.44:0/3258547820 conn(0x7f25040b0eb0 msgr2=0x7f250406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:30.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 shutdown_connections 2024-07-17T07:12:30.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.513+0000 7f2508e00640 1 -- 172.21.15.44:0/3258547820 wait complete. 2024-07-17T07:12:30.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2024-07-17T07:12:30.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 -- 172.21.15.44:0/3951491194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c06f710 msgr2=0x7fd34c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 --2- 172.21.15.44:0/3951491194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c06f710 0x7fd34c153a50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1576d0 tx=0x7fd33c02a5f0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 -- 172.21.15.44:0/3951491194 shutdown_connections 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 --2- 172.21.15.44:0/3951491194 >> v2:172.21.15.175:3300/0 conn(0x7fd34c159b00 0x7fd34c159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 --2- 172.21.15.44:0/3951491194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c06f710 0x7fd34c153a50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 -- 172.21.15.44:0/3951491194 >> 172.21.15.44:0/3951491194 conn(0x7fd34c0b0fb0 msgr2=0x7fd34c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 -- 172.21.15.44:0/3951491194 shutdown_connections 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.751+0000 7fd351e00640 1 -- 172.21.15.44:0/3951491194 wait complete. 2024-07-17T07:12:30.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.753+0000 7fd351e00640 1 Processor -- start 2024-07-17T07:12:30.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.753+0000 7fd351e00640 1 -- start start 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd34c06f710 0x7fd34c1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 0x7fd34c0982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1643f0 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c164270 con 0x7fd34c159b00 2024-07-17T07:12:30.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd351e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c164570 con 0x7fd34c06f710 2024-07-17T07:12:30.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd34a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd34a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56054/0 (socket says 172.21.15.44:56054) 2024-07-17T07:12:30.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.754+0000 7fd34a000640 1 -- 172.21.15.44:0/3229749473 learned_addr learned my addr 172.21.15.44:0/3229749473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34b400640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 0x7fd34c0982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34aa00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd34c06f710 0x7fd34c1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 -- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd34c06f710 msgr2=0x7fd34c1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd34c06f710 0x7fd34c1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 -- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 msgr2=0x7fd34c0982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 0x7fd34c0982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 -- 172.21.15.44:0/3229749473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c098830 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34b400640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 0x7fd34c0982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:30.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.755+0000 7fd34a000640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c9a0 tx=0x7fd33800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.756+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338007bb0 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.756+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd338007e40 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.756+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33800f690 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.756+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c15dc80 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.756+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c0b22b0 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.757+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c153a50 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.758+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fd33800f830 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.759+0000 7fd343400640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 0x7fd32c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.759+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7fd338098f50 con 0x7fd34c0ac3e0 2024-07-17T07:12:30.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.760+0000 7fd34aa00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 0x7fd32c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:30.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.761+0000 7fd34aa00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 0x7fd32c079100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd33c033000 tx=0x7fd33c004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:30.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:30.764+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd338014030 con 0x7fd34c0ac3e0 2024-07-17T07:12:31.000 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:31.000+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:31.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.067+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) -- 0x7fd34c13fff0 con 0x7fd34c0ac3e0 2024-07-17T07:12:31.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.568+0000 7fd343400640 1 -- 172.21.15.44:0/3229749473 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v661) ==== 186+0+0 (secure 0 0 0) 0x7fd338062070 con 0x7fd34c0ac3e0 2024-07-17T07:12:31.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 msgr2=0x7fd32c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 0x7fd32c079100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd33c033000 tx=0x7fd33c004730 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 msgr2=0x7fd34c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c9a0 tx=0x7fd33800ce60 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 shutdown_connections 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd32c076c40 0x7fd32c079100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd34c159b00 0x7fd34c0982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd34c0ac3e0 0x7fd34c074260 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 --2- 172.21.15.44:0/3229749473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd34c06f710 0x7fd34c1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 >> 172.21.15.44:0/3229749473 conn(0x7fd34c0b0fb0 msgr2=0x7fd34c0af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.572+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 shutdown_connections 2024-07-17T07:12:31.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.573+0000 7fd351e00640 1 -- 172.21.15.44:0/3229749473 wait complete. 2024-07-17T07:12:31.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.813+0000 7f68f0600640 1 -- 172.21.15.44:0/1930792254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0004d40 msgr2=0x7f68d00051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.813+0000 7f68f0600640 1 --2- 172.21.15.44:0/1930792254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d0004d40 0x7f68d00051a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ed0 tx=0x7f68d0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.813+0000 7f68ee600640 1 -- 172.21.15.44:0/1930792254 reap_dead start 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.814+0000 7f68f0600640 1 -- 172.21.15.44:0/1930792254 shutdown_connections 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.814+0000 7f68f0600640 1 -- 172.21.15.44:0/1930792254 >> 172.21.15.44:0/1930792254 conn(0x7f68e806bf40 msgr2=0x7f68e806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.814+0000 7f68f0600640 1 -- 172.21.15.44:0/1930792254 shutdown_connections 2024-07-17T07:12:31.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.814+0000 7f68f0600640 1 -- 172.21.15.44:0/1930792254 wait complete. 2024-07-17T07:12:31.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.816+0000 7f68f0600640 1 Processor -- start 2024-07-17T07:12:31.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.816+0000 7f68f0600640 1 -- start start 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 0x7f68e8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68e80af940 0x7f68e80afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81663c0 con 0x7f68e813fb20 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8166240 con 0x7f68e80af940 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68f0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68e8166540 con 0x7f68d0004d40 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:31.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56074/0 (socket says 172.21.15.44:56074) 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 -- 172.21.15.44:0/968568124 learned_addr learned my addr 172.21.15.44:0/968568124 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68edc00640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 0x7f68e8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ed200640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68e80af940 0x7f68e80afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 -- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 msgr2=0x7f68e8141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 0x7f68e8141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 -- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68e80af940 msgr2=0x7f68e80afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68e80af940 0x7f68e80afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68ee600640 1 -- 172.21.15.44:0/968568124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e81404e0 con 0x7f68e813fb20 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.817+0000 7f68edc00640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 0x7f68e8141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68ee600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ed0 tx=0x7f68e0034c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:31.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0043070 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e0034cf0 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0038040 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e81439c0 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.818+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e8143f80 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.820+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e806eba0 con 0x7f68e813fb20 2024-07-17T07:12:31.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.821+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f68e0046070 con 0x7f68e813fb20 2024-07-17T07:12:31.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.822+0000 7f68e5600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 0x7f68c8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:31.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.822+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f68e00b83a0 con 0x7f68e813fb20 2024-07-17T07:12:31.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.822+0000 7f68edc00640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 0x7f68c8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:31.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.824+0000 7f68edc00640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 0x7f68c8079100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68e80aec70 tx=0x7f68dc0015e0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:31.826+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f68e0080df0 con 0x7f68e813fb20 2024-07-17T07:12:31.934 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:31.934+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:32.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.043+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f68e8073280 con 0x7f68e813fb20 2024-07-17T07:12:32.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.044+0000 7f68e5600640 1 -- 172.21.15.44:0/968568124 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+1998 (secure 0 0 0) 0x7f68e0085340 con 0x7f68e813fb20 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:12:32.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:12:32.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:12:32.046 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:12:32.046 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:12:32.046 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:12:32.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.047+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 msgr2=0x7f68c8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.047+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 0x7f68c8079100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68e80aec70 tx=0x7f68dc0015e0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.047+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 msgr2=0x7f68e813ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.047+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ed0 tx=0x7f68e0034c00 comp rx=0 tx=0).stop 2024-07-17T07:12:32.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 shutdown_connections 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f68c8076c40 0x7f68c8079100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68e813fb20 0x7f68e813ffa0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68e80af940 0x7f68e80afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 --2- 172.21.15.44:0/968568124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d0004d40 0x7f68e8141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 >> 172.21.15.44:0/968568124 conn(0x7f68e806bf40 msgr2=0x7f68e815f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 shutdown_connections 2024-07-17T07:12:32.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.048+0000 7f68f0600640 1 -- 172.21.15.44:0/968568124 wait complete. 2024-07-17T07:12:32.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:12:32.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 -- 172.21.15.44:0/3989301617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 msgr2=0x7fa0740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 --2- 172.21.15.44:0/3989301617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa0740051c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c2e0 tx=0x7fa074005830 comp rx=0 tx=0).stop 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa095a00640 1 -- 172.21.15.44:0/3989301617 reap_dead start 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 -- 172.21.15.44:0/3989301617 shutdown_connections 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 -- 172.21.15.44:0/3989301617 >> 172.21.15.44:0/3989301617 conn(0x7fa0900b4ff0 msgr2=0x7fa0900b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 -- 172.21.15.44:0/3989301617 shutdown_connections 2024-07-17T07:12:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.280+0000 7fa096400640 1 -- 172.21.15.44:0/3989301617 wait complete. 2024-07-17T07:12:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.282+0000 7fa096400640 1 Processor -- start 2024-07-17T07:12:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.282+0000 7fa096400640 1 -- start start 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090159e60 0x7fa0901f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 0x7fa0900c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901623e0 con 0x7fa074004d40 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa090162260 con 0x7fa0900c7010 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa096400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa090162560 con 0x7fa090159e60 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.283+0000 7fa095000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56088/0 (socket says 172.21.15.44:56088) 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 -- 172.21.15.44:0/2186359856 learned_addr learned my addr 172.21.15.44:0/2186359856 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095a00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 0x7fa0900c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa08fe00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090159e60 0x7fa0901f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 -- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090159e60 msgr2=0x7fa0901f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090159e60 0x7fa0901f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 -- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 msgr2=0x7fa0900c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 0x7fa0900c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 -- 172.21.15.44:0/2186359856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09015a2e0 con 0x7fa074004d40 2024-07-17T07:12:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095a00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 0x7fa0900c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.284+0000 7fa095000640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa080010cc0 tx=0x7fa08000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.285+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080003970 con 0x7fa074004d40 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.285+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa080003b30 con 0x7fa074004d40 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.285+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0901f7fa0 con 0x7fa074004d40 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.285+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080004050 con 0x7fa074004d40 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.285+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0900ca6e0 con 0x7fa074004d40 2024-07-17T07:12:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.286+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0900cfdf0 con 0x7fa074004d40 2024-07-17T07:12:32.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.287+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fa0800041f0 con 0x7fa074004d40 2024-07-17T07:12:32.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.288+0000 7fa08cc00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 0x7fa06c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:32.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.288+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7fa080099380 con 0x7fa074004d40 2024-07-17T07:12:32.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.289+0000 7fa08fe00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 0x7fa06c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:32.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.290+0000 7fa08fe00640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 0x7fa06c079130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa078000f60 tx=0x7fa078000ef0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:32.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.292+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa0800625b0 con 0x7fa074004d40 2024-07-17T07:12:32.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.519+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa0900cd680 con 0x7fa074004d40 2024-07-17T07:12:32.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.527+0000 7fa08cc00640 1 -- 172.21.15.44:0/2186359856 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fa080061eb0 con 0x7fa074004d40 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 msgr2=0x7fa06c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 0x7fa06c079130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa078000f60 tx=0x7fa078000ef0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 msgr2=0x7fa090154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa080010cc0 tx=0x7fa08000e720 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 shutdown_connections 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa06c076c70 0x7fa06c079130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0900c7010 0x7fa0900c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090159e60 0x7fa0901f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 --2- 172.21.15.44:0/2186359856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa074004d40 0x7fa090154e70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:32.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 >> 172.21.15.44:0/2186359856 conn(0x7fa0900b4ff0 msgr2=0x7fa0900d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:32.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 shutdown_connections 2024-07-17T07:12:32.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:32.530+0000 7fa096400640 1 -- 172.21.15.44:0/2186359856 wait complete. 2024-07-17T07:12:32.550 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:12:32.550 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:12:32.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:12:32.791 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:12:32.791 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:32.791 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:32.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:12:32.849 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:12:32.849 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:32.857 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:32.857+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:32.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:12:32.905 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:32.905 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:32.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:12:32.962 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:12:32.962 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:33.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:12:33.020 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:12:33.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.241+0000 7f1eabe00640 1 -- 172.21.15.44:0/3444759317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e90004d40 msgr2=0x7f1e900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.241+0000 7f1eabe00640 1 --2- 172.21.15.44:0/3444759317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e90004d40 0x7f1e900051c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009e20 tx=0x7f1e90005830 comp rx=0 tx=0).stop 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.241+0000 7f1eab400640 1 -- 172.21.15.44:0/3444759317 reap_dead start 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.241+0000 7f1eabe00640 1 -- 172.21.15.44:0/3444759317 shutdown_connections 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.241+0000 7f1eabe00640 1 -- 172.21.15.44:0/3444759317 >> 172.21.15.44:0/3444759317 conn(0x7f1eac0b0e80 msgr2=0x7f1eac0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.242+0000 7f1eabe00640 1 -- 172.21.15.44:0/3444759317 shutdown_connections 2024-07-17T07:12:33.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.242+0000 7f1eabe00640 1 -- 172.21.15.44:0/3444759317 wait complete. 2024-07-17T07:12:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.243+0000 7f1eabe00640 1 Processor -- start 2024-07-17T07:12:33.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.243+0000 7f1eabe00640 1 -- start start 2024-07-17T07:12:33.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e90004d40 0x7f1eac15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eac1f3e10 0x7f1eac1f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac159fa0 con 0x7f1eac073e10 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac159e20 con 0x7f1eac1f3e10 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.244+0000 7f1eabe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1eac15a120 con 0x7f1e90004d40 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56092/0 (socket says 172.21.15.44:56092) 2024-07-17T07:12:33.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 -- 172.21.15.44:0/632367486 learned_addr learned my addr 172.21.15.44:0/632367486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 -- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e90004d40 msgr2=0x7f1eac15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e90004d40 0x7f1eac15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 -- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eac1f3e10 msgr2=0x7f1eac1f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eac1f3e10 0x7f1eac1f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.245+0000 7f1ea3e00640 1 -- 172.21.15.44:0/632367486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1f73d0 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1ea3e00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400e600 tx=0x7f1e9400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94015070 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e9400ca10 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400ce60 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1f76b0 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.246+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac0cc0c0 con 0x7f1eac073e10 2024-07-17T07:12:33.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.247+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac06e060 con 0x7f1eac073e10 2024-07-17T07:12:33.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.248+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f1e94018070 con 0x7f1eac073e10 2024-07-17T07:12:33.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.249+0000 7f1ea2a00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 0x7f1e88079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.250+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7f1e94098bf0 con 0x7f1eac073e10 2024-07-17T07:12:33.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.250+0000 7f1eaaa00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 0x7f1e88079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.251+0000 7f1eaaa00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 0x7f1e88079130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c005d00 tx=0x7f1e9c005e70 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:33.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.254+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e94061b80 con 0x7f1eac073e10 2024-07-17T07:12:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.510+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eac1f7bd0 con 0x7f1eac073e10 2024-07-17T07:12:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.511+0000 7f1ea2a00640 1 -- 172.21.15.44:0/632367486 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+3911 (secure 0 0 0) 0x7f1e94061480 con 0x7f1eac073e10 2024-07-17T07:12:33.510 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:33.510 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":45,"btime":"2024-07-17T07:12:25:520403+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5537,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/822421491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":822421491},{"type":"v1","addr":"172.21.15.175:6837","nonce":822421491}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1715661885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1715661885},{"type":"v1","addr":"172.21.15.175:6836","nonce":1715661885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3035969080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3035969080},{"type":"v1","addr":"172.21.15.44:6833","nonce":3035969080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/568773694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":568773694},{"type":"v1","addr":"172.21.15.44:6835","nonce":568773694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[]} 2024-07-17T07:12:33.510 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 45 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 msgr2=0x7f1e88079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 0x7f1e88079130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c005d00 tx=0x7f1e9c005e70 comp rx=0 tx=0).stop 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 msgr2=0x7f1eac1f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400e600 tx=0x7f1e9400eac0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 shutdown_connections 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f1e88076c70 0x7f1e88079130 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1eac1f3e10 0x7f1eac1f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1eac073e10 0x7f1eac1f6d40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 --2- 172.21.15.44:0/632367486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e90004d40 0x7f1eac15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.514+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 >> 172.21.15.44:0/632367486 conn(0x7f1eac0b0e80 msgr2=0x7f1eac0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.515+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 shutdown_connections 2024-07-17T07:12:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.515+0000 7f1eabe00640 1 -- 172.21.15.44:0/632367486 wait complete. 2024-07-17T07:12:33.534 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:12:33.534 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:12:33.534 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:12:33.535 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:12:33.535 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:12:33.535 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:12:33.535 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:12:33.535 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:12:33.535 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:12:33.535 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:12:33.536 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:12:33.536 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:12:33.536 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:12:33.536 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:12:33.536 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:12:33.536 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:12:33.560 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:12:33.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:12:33.561 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:12:33.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:12:33.562 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:12:33.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:12:33.563 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:12:33.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:12:33.608 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:12:33.609 DEBUG:teuthology.parallel:result is None 2024-07-17T07:12:33.610 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:12:33.610 DEBUG:teuthology.parallel:result is None 2024-07-17T07:12:33.614 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:12:33.614 DEBUG:teuthology.parallel:result is None 2024-07-17T07:12:33.616 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:12:33.616 DEBUG:teuthology.parallel:result is None 2024-07-17T07:12:33.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:12:33.760 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:33.761+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:33.761 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:12:33.762 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:12:33.763 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:12:33.765 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 -- 172.21.15.44:0/700833331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0008154490 msgr2=0x7f000815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 --2- 172.21.15.44:0/700833331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0008154490 0x7f000815d830 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f000809d310 tx=0x7efff802a5f0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 -- 172.21.15.44:0/700833331 shutdown_connections 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 --2- 172.21.15.44:0/700833331 >> v2:172.21.15.175:3300/0 conn(0x7f000806f4f0 0x7f00080ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 --2- 172.21.15.44:0/700833331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0008154490 0x7f000815d830 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.812+0000 7f000da00640 1 -- 172.21.15.44:0/700833331 >> 172.21.15.44:0/700833331 conn(0x7f00080cf0d0 msgr2=0x7f00080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.813+0000 7f000da00640 1 -- 172.21.15.44:0/700833331 shutdown_connections 2024-07-17T07:12:33.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.813+0000 7f000da00640 1 -- 172.21.15.44:0/700833331 wait complete. 2024-07-17T07:12:33.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.814+0000 7f000da00640 1 Processor -- start 2024-07-17T07:12:33.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.815+0000 7f000da00640 1 -- start start 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.815+0000 7f000da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.815+0000 7f000da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 0x7f000815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f000da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f000815dd70 0x7f000809b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f000da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00080b2220 con 0x7f000806f4f0 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f000da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00080b20a0 con 0x7f0008154490 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f000da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00080b23a0 con 0x7f000815dd70 2024-07-17T07:12:33.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56150/0 (socket says 172.21.15.44:56150) 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 -- 172.21.15.44:0/3405986622 learned_addr learned my addr 172.21.15.44:0/3405986622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f000d000640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f000815dd70 0x7f000809b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007400640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 0x7f000815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 -- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f000815dd70 msgr2=0x7f000809b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f000815dd70 0x7f000809b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 -- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 msgr2=0x7f000815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 0x7f000815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007e00640 1 -- 172.21.15.44:0/3405986622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00081f4920 con 0x7f000806f4f0 2024-07-17T07:12:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.816+0000 7f0007400640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 0x7f000815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7f0007e00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff803f070 con 0x7f000806f4f0 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff80047a0 con 0x7f000806f4f0 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8035030 con 0x7f000806f4f0 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00080996a0 con 0x7f000806f4f0 2024-07-17T07:12:33.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.817+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0008099c60 con 0x7f000806f4f0 2024-07-17T07:12:33.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.818+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00080b2930 con 0x7f000806f4f0 2024-07-17T07:12:33.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.820+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7efff8042070 con 0x7f000806f4f0 2024-07-17T07:12:33.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.821+0000 7effffe00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 0x7effe8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:33.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.821+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6543+0+0 (secure 0 0 0) 0x7efff80b96a0 con 0x7f000806f4f0 2024-07-17T07:12:33.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.821+0000 7f0007400640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 0x7effe8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:33.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.822+0000 7f0007400640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 0x7effe8079100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efff4007fd0 tx=0x7efff4009040 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:33.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:33.825+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efff8082630 con 0x7f000806f4f0 2024-07-17T07:12:34.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.059+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f00081606c0 con 0x7f000806f4f0 2024-07-17T07:12:34.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.607+0000 7effffe00640 1 -- 172.21.15.44:0/3405986622 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) ==== 97+0+0 (secure 0 0 0) 0x7efff8081f30 con 0x7f000806f4f0 2024-07-17T07:12:34.607 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 msgr2=0x7effe8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 0x7effe8079100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efff4007fd0 tx=0x7efff4009040 comp rx=0 tx=0).stop 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 msgr2=0x7f00081f43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff8003820 comp rx=0 tx=0).stop 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 shutdown_connections 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7effe8076c40 0x7effe8079100 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f000815dd70 0x7f000809b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0008154490 0x7f000815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 --2- 172.21.15.44:0/3405986622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f000806f4f0 0x7f00081f43e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 >> 172.21.15.44:0/3405986622 conn(0x7f00080cf0d0 msgr2=0x7f000806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 shutdown_connections 2024-07-17T07:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.612+0000 7f000da00640 1 -- 172.21.15.44:0/3405986622 wait complete. 2024-07-17T07:12:34.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:12:34.733 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:34.732+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 -- 172.21.15.44:0/2836802863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 msgr2=0x7f2fdc0071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 --2- 172.21.15.44:0/2836802863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2fdc0071d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800bb20 tx=0x7f2fdc003810 comp rx=0 tx=0).stop 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 -- 172.21.15.44:0/2836802863 shutdown_connections 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 --2- 172.21.15.44:0/2836802863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2fdc0071d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 -- 172.21.15.44:0/2836802863 >> 172.21.15.44:0/2836802863 conn(0x7f2ff40b0fb0 msgr2=0x7f2ff40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 -- 172.21.15.44:0/2836802863 shutdown_connections 2024-07-17T07:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.835+0000 7f2ffc400640 1 -- 172.21.15.44:0/2836802863 wait complete. 2024-07-17T07:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.837+0000 7f2ffc400640 1 Processor -- start 2024-07-17T07:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.837+0000 7f2ffc400640 1 -- start start 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 0x7f2ff4151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff4151a40 0x7f2ff4151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff406f710 con 0x7f2fdc004de0 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4154040 con 0x7f2ff4151a40 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ffc400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff406f890 con 0x7f2ff4156e80 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.838+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56164/0 (socket says 172.21.15.44:56164) 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 -- 172.21.15.44:0/4271880046 learned_addr learned my addr 172.21.15.44:0/4271880046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 -- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 msgr2=0x7f2ff4151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9000640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 0x7f2ff4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 0x7f2ff4151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 -- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff4151a40 msgr2=0x7f2ff4151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff4151a40 0x7f2ff4151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 -- 172.21.15.44:0/4271880046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff4071da0 con 0x7f2fdc004de0 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9000640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 0x7f2ff4151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.839+0000 7f2ff9a00640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2fe802b000 tx=0x7f2fe800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.840+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8035040 con 0x7f2fdc004de0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.840+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe8004340 con 0x7f2fdc004de0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.840+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe803a890 con 0x7f2fdc004de0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.840+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff40717e0 con 0x7f2fdc004de0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.840+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff4158d90 con 0x7f2fdc004de0 2024-07-17T07:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.841+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4150920 con 0x7f2fdc004de0 2024-07-17T07:12:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.842+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f2fe8038020 con 0x7f2fdc004de0 2024-07-17T07:12:34.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.844+0000 7f2ff1600640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 0x7f2fd8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:34.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.844+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5769+0+0 (secure 0 0 0) 0x7f2fe80b8b10 con 0x7f2fdc004de0 2024-07-17T07:12:34.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.844+0000 7f2ff9000640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 0x7f2fd8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:34.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.846+0000 7f2ff9000640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 0x7f2fd8079100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4001a30 tx=0x7f2fe4001c00 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:34.847+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2fe8081eb0 con 0x7f2fdc004de0 2024-07-17T07:12:35.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.143+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2ff4159df0 con 0x7f2fdc004de0 2024-07-17T07:12:35.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.143+0000 7f2ff1600640 1 -- 172.21.15.44:0/4271880046 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f2fe80817b0 con 0x7f2fdc004de0 2024-07-17T07:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 msgr2=0x7f2fd8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 0x7f2fd8079100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4001a30 tx=0x7f2fe4001c00 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 msgr2=0x7f2ff4150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2fe802b000 tx=0x7f2fe800b770 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 shutdown_connections 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f2fd8076c40 0x7f2fd8079100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ff4151a40 0x7f2ff4151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ff4156e80 0x7f2ff4151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 --2- 172.21.15.44:0/4271880046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fdc004de0 0x7f2ff4150fc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 >> 172.21.15.44:0/4271880046 conn(0x7f2ff40b0fb0 msgr2=0x7f2ff406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 shutdown_connections 2024-07-17T07:12:35.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.147+0000 7f2ffc400640 1 -- 172.21.15.44:0/4271880046 wait complete. 2024-07-17T07:12:35.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.387+0000 7fdadf200640 1 -- 172.21.15.44:0/3122058708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 msgr2=0x7fdac4003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.387+0000 7fdadf200640 1 --2- 172.21.15.44:0/3122058708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdac4003ee0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fdac8001690 tx=0x7fdac4005830 comp rx=0 tx=0).stop 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.387+0000 7fdadf200640 1 -- 172.21.15.44:0/3122058708 shutdown_connections 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.387+0000 7fdadf200640 1 --2- 172.21.15.44:0/3122058708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdac4003ee0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.387+0000 7fdadf200640 1 -- 172.21.15.44:0/3122058708 >> 172.21.15.44:0/3122058708 conn(0x7fdae00cfec0 msgr2=0x7fdae00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.388+0000 7fdadf200640 1 -- 172.21.15.44:0/3122058708 shutdown_connections 2024-07-17T07:12:35.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.388+0000 7fdadf200640 1 -- 172.21.15.44:0/3122058708 wait complete. 2024-07-17T07:12:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.389+0000 7fdadf200640 1 Processor -- start 2024-07-17T07:12:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 -- start start 2024-07-17T07:12:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 0x7fdae0151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae00c9950 0x7fdae00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdae00b54c0 con 0x7fdac4003ac0 2024-07-17T07:12:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdae00b5340 con 0x7fdae00c9950 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.390+0000 7fdadf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdae00b5640 con 0x7fdae015b190 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56184/0 (socket says 172.21.15.44:56184) 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 -- 172.21.15.44:0/2794873934 learned_addr learned my addr 172.21.15.44:0/2794873934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdade800640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae00c9950 0x7fdae00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadd400640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 0x7fdae0151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 -- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 msgr2=0x7fdae0151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 0x7fdae0151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 -- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae00c9950 msgr2=0x7fdae00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae00c9950 0x7fdae00c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadde00640 1 -- 172.21.15.44:0/2794873934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae015b610 con 0x7fdac4003ac0 2024-07-17T07:12:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.391+0000 7fdadd400640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 0x7fdae0151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdadde00640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdac802a8f0 tx=0x7fdac80063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003a00 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac8039070 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003e40 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae015b890 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.392+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae015be50 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.394+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae006dca0 con 0x7fdac4003ac0 2024-07-17T07:12:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.395+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fdac8044020 con 0x7fdac4003ac0 2024-07-17T07:12:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.396+0000 7fdad6000640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 0x7fdabc079180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.396+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5769+0+0 (secure 0 0 0) 0x7fdac80b7930 con 0x7fdac4003ac0 2024-07-17T07:12:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.396+0000 7fdadd400640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 0x7fdabc079180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.398+0000 7fdadd400640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 0x7fdabc079180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdad0001860 tx=0x7fdad0001190 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.400+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdae015b610 con 0x7fdac4003ac0 2024-07-17T07:12:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.619+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdae00c6f50 con 0x7fdac4003ac0 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.620+0000 7fdad6000640 1 -- 172.21.15.44:0/2794873934 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) ==== 84+0+65 (secure 0 0 0) 0x7fdac8080e50 con 0x7fdac4003ac0 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:35.619 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:12:35.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 msgr2=0x7fdabc079180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 0x7fdabc079180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdad0001860 tx=0x7fdad0001190 comp rx=0 tx=0).stop 2024-07-17T07:12:35.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 msgr2=0x7fdae015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdac802a8f0 tx=0x7fdac80063a0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 shutdown_connections 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fdabc076cc0 0x7fdabc079180 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae00c9950 0x7fdae00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae015b190 0x7fdae0151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 --2- 172.21.15.44:0/2794873934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4003ac0 0x7fdae015d570 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.622+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 >> 172.21.15.44:0/2794873934 conn(0x7fdae00cfec0 msgr2=0x7fdae00d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:35.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.623+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 shutdown_connections 2024-07-17T07:12:35.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.623+0000 7fdadf200640 1 -- 172.21.15.44:0/2794873934 wait complete. 2024-07-17T07:12:35.640 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:12:35.640 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:12:35.686 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:35.686+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 -- 172.21.15.44:0/3171485617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb815e1e0 msgr2=0x7f3bb80ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 --2- 172.21.15.44:0/3171485617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb815e1e0 0x7f3bb80ae6e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80b5310 tx=0x7f3bb002a980 comp rx=0 tx=0).stop 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 -- 172.21.15.44:0/3171485617 shutdown_connections 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 --2- 172.21.15.44:0/3171485617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb815e1e0 0x7f3bb80ae6e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 --2- 172.21.15.44:0/3171485617 >> v2:172.21.15.175:3300/0 conn(0x7f3bb8154560 0x7f3bb80ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 -- 172.21.15.44:0/3171485617 >> 172.21.15.44:0/3171485617 conn(0x7f3bb80b0eb0 msgr2=0x7f3bb80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 -- 172.21.15.44:0/3171485617 shutdown_connections 2024-07-17T07:12:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.846+0000 7f3bbea00640 1 -- 172.21.15.44:0/3171485617 wait complete. 2024-07-17T07:12:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.848+0000 7f3bbea00640 1 Processor -- start 2024-07-17T07:12:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.848+0000 7f3bbea00640 1 -- start start 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb806e690 0x7f3bb8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 0x7f3bb81e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80b2650 con 0x7f3bb8154560 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80b24d0 con 0x7f3bb815e1e0 2024-07-17T07:12:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.849+0000 7f3bbea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb80b27d0 con 0x7f3bb806e690 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56212/0 (socket says 172.21.15.44:56212) 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 -- 172.21.15.44:0/2487946147 learned_addr learned my addr 172.21.15.44:0/2487946147 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7e00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb806e690 0x7f3bb8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bbca00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 0x7f3bb81e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 -- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb806e690 msgr2=0x7f3bb8151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb806e690 0x7f3bb8151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 -- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 msgr2=0x7f3bb81e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 0x7f3bb81e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 -- 172.21.15.44:0/2487946147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb81560a0 con 0x7f3bb8154560 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bbca00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 0x7f3bb81e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.850+0000 7f3bb7400640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c460 tx=0x7f3ba400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.851+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4016020 con 0x7f3bb8154560 2024-07-17T07:12:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.851+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba4012960 con 0x7f3bb8154560 2024-07-17T07:12:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.851+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb8156320 con 0x7f3bb8154560 2024-07-17T07:12:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.851+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012db0 con 0x7f3bb8154560 2024-07-17T07:12:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.851+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8071680 con 0x7f3bb8154560 2024-07-17T07:12:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.852+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb806a1a0 con 0x7f3bb8154560 2024-07-17T07:12:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.853+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f3ba4019020 con 0x7f3bb8154560 2024-07-17T07:12:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.855+0000 7f3bafe00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 0x7f3b98079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.855+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5769+0+0 (secure 0 0 0) 0x7f3ba4097f20 con 0x7f3bb8154560 2024-07-17T07:12:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.856+0000 7f3bb7e00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 0x7f3b98079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:35.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.857+0000 7f3bb7e00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 0x7f3b98079100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80058c0 tx=0x7f3ba800a4c0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:35.859+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bb806a1a0 con 0x7f3bb8154560 2024-07-17T07:12:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.081+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3bb80b34b0 con 0x7f3bb8154560 2024-07-17T07:12:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.621+0000 7f3bafe00640 1 -- 172.21.15.44:0/2487946147 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) ==== 138+0+0 (secure 0 0 0) 0x7f3ba4020070 con 0x7f3bb8154560 2024-07-17T07:12:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:12:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 msgr2=0x7f3b98079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 0x7f3b98079100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80058c0 tx=0x7f3ba800a4c0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 msgr2=0x7f3bb8155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c460 tx=0x7f3ba400c920 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 shutdown_connections 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f3b98076c40 0x7f3b98079100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bb815e1e0 0x7f3bb81e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb8154560 0x7f3bb8155b60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 --2- 172.21.15.44:0/2487946147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bb806e690 0x7f3bb8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.625+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 >> 172.21.15.44:0/2487946147 conn(0x7f3bb80b0eb0 msgr2=0x7f3bb81522d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.626+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 shutdown_connections 2024-07-17T07:12:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.626+0000 7f3bbea00640 1 -- 172.21.15.44:0/2487946147 wait complete. 2024-07-17T07:12:36.651 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:12:36.669 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:36.669+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 -- 172.21.15.44:0/2931973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff064003ac0 msgr2=0x7ff064003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 --2- 172.21.15.44:0/2931973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff064003ac0 0x7ff064003ee0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff070001690 tx=0x7ff064005830 comp rx=0 tx=0).stop 2024-07-17T07:12:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 -- 172.21.15.44:0/2931973244 shutdown_connections 2024-07-17T07:12:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 --2- 172.21.15.44:0/2931973244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff064003ac0 0x7ff064003ee0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 -- 172.21.15.44:0/2931973244 >> 172.21.15.44:0/2931973244 conn(0x7ff0800b3090 msgr2=0x7ff0800b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 -- 172.21.15.44:0/2931973244 shutdown_connections 2024-07-17T07:12:36.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.872+0000 7ff07fe00640 1 -- 172.21.15.44:0/2931973244 wait complete. 2024-07-17T07:12:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.874+0000 7ff07fe00640 1 Processor -- start 2024-07-17T07:12:36.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.874+0000 7ff07fe00640 1 -- start start 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 0x7ff080149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0800afbb0 0x7ff0800b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff080071540 con 0x7ff080157090 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800713c0 con 0x7ff064003ac0 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff07fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0800716c0 con 0x7ff0800afbb0 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff076000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff076000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56240/0 (socket says 172.21.15.44:56240) 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.875+0000 7ff076000640 1 -- 172.21.15.44:0/929905100 learned_addr learned my addr 172.21.15.44:0/929905100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff07ea00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 0x7ff080149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:36.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff07f400640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0800afbb0 0x7ff0800b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 -- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0800afbb0 msgr2=0x7ff0800b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0800afbb0 0x7ff0800b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 -- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 msgr2=0x7ff080149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 0x7ff080149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 -- 172.21.15.44:0/929905100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0800c6c00 con 0x7ff080157090 2024-07-17T07:12:36.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff07ea00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 0x7ff080149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.876+0000 7ff076000640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff06800f7a0 tx=0x7ff06800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.877+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800bc50 con 0x7ff080157090 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.877+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff06800fe50 con 0x7ff080157090 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.877+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680116a0 con 0x7ff080157090 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.877+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0800c6ee0 con 0x7ff080157090 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.877+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0800c74a0 con 0x7ff080157090 2024-07-17T07:12:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.878+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0800694a0 con 0x7ff080157090 2024-07-17T07:12:36.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.880+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7ff068019020 con 0x7ff080157090 2024-07-17T07:12:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.882+0000 7ff077400640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 0x7ff05c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:36.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.882+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6139+0+0 (secure 0 0 0) 0x7ff0680992f0 con 0x7ff080157090 2024-07-17T07:12:36.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.882+0000 7ff07ea00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 0x7ff05c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:36.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.884+0000 7ff07ea00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 0x7ff05c079130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff070032000 tx=0x7ff070001190 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:36.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:36.886+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff0800694a0 con 0x7ff080157090 2024-07-17T07:12:37.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.104+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff080074360 con 0x7ff080157090 2024-07-17T07:12:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.630+0000 7ff077400640 1 -- 172.21.15.44:0/929905100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) ==== 170+0+0 (secure 0 0 0) 0x7ff068029020 con 0x7ff080157090 2024-07-17T07:12:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.634+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 msgr2=0x7ff05c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.634+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 0x7ff05c079130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff070032000 tx=0x7ff070001190 comp rx=0 tx=0).stop 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 msgr2=0x7ff0800c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff06800f7a0 tx=0x7ff06800fc60 comp rx=0 tx=0).stop 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 shutdown_connections 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff05c076c70 0x7ff05c079130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0800afbb0 0x7ff0800b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080157090 0x7ff0800c6300 secure :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff06800f7a0 tx=0x7ff06800fc60 comp rx=0 tx=0).stop 2024-07-17T07:12:37.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 --2- 172.21.15.44:0/929905100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff064003ac0 0x7ff080149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.635+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 >> 172.21.15.44:0/929905100 conn(0x7ff0800b3090 msgr2=0x7ff08015bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:37.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.639+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 shutdown_connections 2024-07-17T07:12:37.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.640+0000 7ff07fe00640 1 -- 172.21.15.44:0/929905100 wait complete. 2024-07-17T07:12:37.675 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:12:37.688 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:37.689+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:37.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 -- 172.21.15.44:0/4123245303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc540052e0 msgr2=0x7efc5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:37.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 --2- 172.21.15.44:0/4123245303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc540052e0 0x7efc5400b9c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c2e0 tx=0x7efc54005830 comp rx=0 tx=0).stop 2024-07-17T07:12:37.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 -- 172.21.15.44:0/4123245303 shutdown_connections 2024-07-17T07:12:37.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 --2- 172.21.15.44:0/4123245303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc540052e0 0x7efc5400b9c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 -- 172.21.15.44:0/4123245303 >> 172.21.15.44:0/4123245303 conn(0x7efc7009aa30 msgr2=0x7efc700b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:37.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 -- 172.21.15.44:0/4123245303 shutdown_connections 2024-07-17T07:12:37.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.861+0000 7efc76600640 1 -- 172.21.15.44:0/4123245303 wait complete. 2024-07-17T07:12:37.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.863+0000 7efc76600640 1 Processor -- start 2024-07-17T07:12:37.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.863+0000 7efc76600640 1 -- start start 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc540052e0 0x7efc701f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc7015ddb0 0x7efc7014f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc700ce4b0 con 0x7efc7015ccd0 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc700cd7e0 con 0x7efc540052e0 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc76600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc700ce630 con 0x7efc7015ddb0 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:37.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc6ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56260/0 (socket says 172.21.15.44:56260) 2024-07-17T07:12:37.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.864+0000 7efc6ea00640 1 -- 172.21.15.44:0/2900850676 learned_addr learned my addr 172.21.15.44:0/2900850676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6fe00640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc7015ddb0 0x7efc7014f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6f400640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc540052e0 0x7efc701f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 -- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc7015ddb0 msgr2=0x7efc7014f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc7015ddb0 0x7efc7014f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 -- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc540052e0 msgr2=0x7efc701f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc540052e0 0x7efc701f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 -- 172.21.15.44:0/2900850676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7015bc30 con 0x7efc7015ccd0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc6ea00640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d7b0 tx=0x7efc5800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58016020 con 0x7efc7015ccd0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc5800b6c0 con 0x7efc7015ccd0 2024-07-17T07:12:37.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800bad0 con 0x7efc7015ccd0 2024-07-17T07:12:37.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.865+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7015bf10 con 0x7efc7015ccd0 2024-07-17T07:12:37.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.866+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7015c4d0 con 0x7efc7015ccd0 2024-07-17T07:12:37.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.867+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7006a1a0 con 0x7efc7015ccd0 2024-07-17T07:12:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.870+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7efc58019020 con 0x7efc7015ccd0 2024-07-17T07:12:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.871+0000 7efc67400640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 0x7efc4c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.871+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6521+0+0 (secure 0 0 0) 0x7efc58098a20 con 0x7efc7015ccd0 2024-07-17T07:12:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.872+0000 7efc6f400640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 0x7efc4c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.873+0000 7efc6f400640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 0x7efc4c079130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc600092f0 tx=0x7efc60009200 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:37.875+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efc58020020 con 0x7efc7015ccd0 2024-07-17T07:12:38.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.133+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efc700b06e0 con 0x7efc7015ccd0 2024-07-17T07:12:38.692 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 13 and data pool 14 2024-07-17T07:12:38.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.693+0000 7efc67400640 1 -- 172.21.15.44:0/2900850676 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) ==== 174+0+0 (secure 0 0 0) 0x7efc58061be0 con 0x7efc7015ccd0 2024-07-17T07:12:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.700+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 msgr2=0x7efc4c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.700+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 0x7efc4c079130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc600092f0 tx=0x7efc60009200 comp rx=0 tx=0).stop 2024-07-17T07:12:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.700+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 msgr2=0x7efc700b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:38.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.700+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d7b0 tx=0x7efc5800dc70 comp rx=0 tx=0).stop 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 shutdown_connections 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7efc4c076c70 0x7efc4c079130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc7015ddb0 0x7efc7014f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc7015ccd0 0x7efc700b6d60 secure :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d7b0 tx=0x7efc5800dc70 comp rx=0 tx=0).stop 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 --2- 172.21.15.44:0/2900850676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc540052e0 0x7efc701f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.701+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 >> 172.21.15.44:0/2900850676 conn(0x7efc7009aa30 msgr2=0x7efc7006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:38.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.706+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 shutdown_connections 2024-07-17T07:12:38.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.708+0000 7efc76600640 1 -- 172.21.15.44:0/2900850676 wait complete. 2024-07-17T07:12:38.725 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:38.726+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:38.741 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 -- 172.21.15.44:0/362478270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 msgr2=0x7fa3d4159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 --2- 172.21.15.44:0/362478270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d4159b00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c402adc0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 -- 172.21.15.44:0/362478270 shutdown_connections 2024-07-17T07:12:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 --2- 172.21.15.44:0/362478270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d40ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 --2- 172.21.15.44:0/362478270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d4159b00 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.921+0000 7fa3da600640 1 -- 172.21.15.44:0/362478270 >> 172.21.15.44:0/362478270 conn(0x7fa3d40b0fb0 msgr2=0x7fa3d40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:38.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.922+0000 7fa3da600640 1 -- 172.21.15.44:0/362478270 shutdown_connections 2024-07-17T07:12:38.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.922+0000 7fa3da600640 1 -- 172.21.15.44:0/362478270 wait complete. 2024-07-17T07:12:38.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.923+0000 7fa3da600640 1 Processor -- start 2024-07-17T07:12:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.924+0000 7fa3da600640 1 -- start start 2024-07-17T07:12:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40ac920 0x7fa3d4142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d4071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:38.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4153fb0 con 0x7fa3d4153a50 2024-07-17T07:12:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4153e30 con 0x7fa3d40ac920 2024-07-17T07:12:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3da600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d406f710 con 0x7fa3d415a040 2024-07-17T07:12:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38720/0 (socket says 172.21.15.44:38720) 2024-07-17T07:12:38.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.925+0000 7fa3d2a00640 1 -- 172.21.15.44:0/3180074538 learned_addr learned my addr 172.21.15.44:0/3180074538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 -- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 msgr2=0x7fa3d4071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d3e00640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d4071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d3400640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40ac920 0x7fa3d4142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d4071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 -- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40ac920 msgr2=0x7fa3d4142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:38.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40ac920 0x7fa3d4142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 -- 172.21.15.44:0/3180074538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d41572e0 con 0x7fa3d4153a50 2024-07-17T07:12:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d3e00640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d4071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3d2a00640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000d770 tx=0x7fa3c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0017070 con 0x7fa3d4153a50 2024-07-17T07:12:38.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3c0009d20 con 0x7fa3d4153a50 2024-07-17T07:12:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d41f4230 con 0x7fa3d4153a50 2024-07-17T07:12:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0004280 con 0x7fa3d4153a50 2024-07-17T07:12:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.926+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d41f47f0 con 0x7fa3d4153a50 2024-07-17T07:12:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.928+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d4159b00 con 0x7fa3d4153a50 2024-07-17T07:12:38.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.929+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fa3c00035f0 con 0x7fa3d4153a50 2024-07-17T07:12:38.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.930+0000 7fa3cb400640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 0x7fa3b4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:38.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.930+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6621+0+0 (secure 0 0 0) 0x7fa3c0098980 con 0x7fa3d4153a50 2024-07-17T07:12:38.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.930+0000 7fa3d3400640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 0x7fa3b4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:38.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.932+0000 7fa3d3400640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 0x7fa3b4079100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4034000 tx=0x7fa3c4004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:38.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:38.934+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa3c00618c0 con 0x7fa3d4153a50 2024-07-17T07:12:39.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.150+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3d415c2a0 con 0x7fa3d4153a50 2024-07-17T07:12:39.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.152+0000 7fa3cb400640 1 -- 172.21.15.44:0/3180074538 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17683 (secure 0 0 0) 0x7fa3c00611c0 con 0x7fa3d4153a50 2024-07-17T07:12:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:39.152 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":111,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:38.135190+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:36.099092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:37.121054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 msgr2=0x7fa3b4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 0x7fa3b4079100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4034000 tx=0x7fa3c4004730 comp rx=0 tx=0).stop 2024-07-17T07:12:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 msgr2=0x7fa3d41f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000d770 tx=0x7fa3c000dc30 comp rx=0 tx=0).stop 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 shutdown_connections 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa3b4076c40 0x7fa3b4079100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3d415a040 0x7fa3d4071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d4153a50 0x7fa3d41f14b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 --2- 172.21.15.44:0/3180074538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3d40ac920 0x7fa3d4142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.155+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 >> 172.21.15.44:0/3180074538 conn(0x7fa3d40b0fb0 msgr2=0x7fa3d415aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.156+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 shutdown_connections 2024-07-17T07:12:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.156+0000 7fa3da600640 1 -- 172.21.15.44:0/3180074538 wait complete. 2024-07-17T07:12:39.174 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 -- 172.21.15.44:0/505396120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0003a50 msgr2=0x7facc0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 --2- 172.21.15.44:0/505396120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0003a50 0x7facc0003e50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7faccc00baf0 tx=0x7facc0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facda800640 1 -- 172.21.15.44:0/505396120 reap_dead start 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 -- 172.21.15.44:0/505396120 shutdown_connections 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 -- 172.21.15.44:0/505396120 >> 172.21.15.44:0/505396120 conn(0x7facdc0cfec0 msgr2=0x7facdc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 -- 172.21.15.44:0/505396120 shutdown_connections 2024-07-17T07:12:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.378+0000 7facdb200640 1 -- 172.21.15.44:0/505396120 wait complete. 2024-07-17T07:12:39.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.380+0000 7facdb200640 1 Processor -- start 2024-07-17T07:12:39.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.380+0000 7facdb200640 1 -- start start 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc0003a50 0x7facdc1f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc0c9d90 0x7facdc1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b4ec0 con 0x7facdc14e5a0 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b9270 con 0x7facc0003a50 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.381+0000 7facdb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0b5040 con 0x7facdc0c9d90 2024-07-17T07:12:39.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38744/0 (socket says 172.21.15.44:38744) 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 -- 172.21.15.44:0/1143892505 learned_addr learned my addr 172.21.15.44:0/1143892505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9e00640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc0003a50 0x7facdc1f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 -- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc0c9d90 msgr2=0x7facdc1f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facda800640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc0c9d90 0x7facdc1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc0c9d90 0x7facdc1f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 -- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc0003a50 msgr2=0x7facdc1f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc0003a50 0x7facdc1f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.382+0000 7facd9400640 1 -- 172.21.15.44:0/1143892505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0aa010 con 0x7facdc14e5a0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facd9400640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7facc800c4b0 tx=0x7facc800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8016020 con 0x7facdc14e5a0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc8012920 con 0x7facdc14e5a0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8012d70 con 0x7facdc14e5a0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc14ea20 con 0x7facdc14e5a0 2024-07-17T07:12:39.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.383+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc0aa560 con 0x7facdc14e5a0 2024-07-17T07:12:39.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.384+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc15e3b0 con 0x7facdc14e5a0 2024-07-17T07:12:39.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.386+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7facc8019020 con 0x7facdc14e5a0 2024-07-17T07:12:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.387+0000 7facd2000640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 0x7facbc079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.387+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6621+0+0 (secure 0 0 0) 0x7facc80984c0 con 0x7facdc14e5a0 2024-07-17T07:12:39.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.388+0000 7facd9e00640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 0x7facbc079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.389+0000 7facd9e00640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 0x7facbc079100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faccc036000 tx=0x7faccc0046c0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:39.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.391+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7facc8020030 con 0x7facdc14e5a0 2024-07-17T07:12:39.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.609+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facdc0b5d70 con 0x7facdc14e5a0 2024-07-17T07:12:39.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.612+0000 7facd2000640 1 -- 172.21.15.44:0/1143892505 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17683 (secure 0 0 0) 0x7facdc14ea20 con 0x7facdc14e5a0 2024-07-17T07:12:39.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:39.614 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":111,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:38.135190+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:36.099092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:37.121054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.614+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 msgr2=0x7facbc079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.614+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 0x7facbc079100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faccc036000 tx=0x7faccc0046c0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.614+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 msgr2=0x7facdc0b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.614+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7facc800c4b0 tx=0x7facc800c970 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 shutdown_connections 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7facbc076c40 0x7facbc079100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc0c9d90 0x7facdc1f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facdc14e5a0 0x7facdc0b17c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 --2- 172.21.15.44:0/1143892505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc0003a50 0x7facdc1f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 >> 172.21.15.44:0/1143892505 conn(0x7facdc0cfec0 msgr2=0x7facdc15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:39.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 shutdown_connections 2024-07-17T07:12:39.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.615+0000 7facdb200640 1 -- 172.21.15.44:0/1143892505 wait complete. 2024-07-17T07:12:39.634 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:12:39.688 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:39.688+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 -- 172.21.15.44:0/187765247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f786c0052e0 msgr2=0x7f786c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 --2- 172.21.15.44:0/187765247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f786c0052e0 0x7f786c00b9c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7880009f20 tx=0x7f786c005830 comp rx=0 tx=0).stop 2024-07-17T07:12:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 -- 172.21.15.44:0/187765247 shutdown_connections 2024-07-17T07:12:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 --2- 172.21.15.44:0/187765247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f786c0052e0 0x7f786c00b9c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 -- 172.21.15.44:0/187765247 >> 172.21.15.44:0/187765247 conn(0x7f78880b0eb0 msgr2=0x7f78880b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 -- 172.21.15.44:0/187765247 shutdown_connections 2024-07-17T07:12:39.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.847+0000 7f788e600640 1 -- 172.21.15.44:0/187765247 wait complete. 2024-07-17T07:12:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.849+0000 7f788e600640 1 Processor -- start 2024-07-17T07:12:39.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.849+0000 7f788e600640 1 -- start start 2024-07-17T07:12:39.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f786c0052e0 0x7f7888073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7888151fa0 0x7f7888158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78880ac780 con 0x7f788815b1e0 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78880ac600 con 0x7f7888151fa0 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.850+0000 7f788e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78880ac900 con 0x7f786c0052e0 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38756/0 (socket says 172.21.15.44:38756) 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 -- 172.21.15.44:0/431342879 learned_addr learned my addr 172.21.15.44:0/431342879 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7887400640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f786c0052e0 0x7f7888073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7887e00640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7888151fa0 0x7f7888158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 -- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f786c0052e0 msgr2=0x7f7888073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f786c0052e0 0x7f7888073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 -- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7888151fa0 msgr2=0x7f7888158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:39.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7888151fa0 0x7f7888158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 -- 172.21.15.44:0/431342879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7888074490 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.851+0000 7f7886a00640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7870000fb0 tx=0x7f787000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.852+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870015070 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.852+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f787000c9e0 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.852+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000ce30 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.852+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7888072eb0 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.852+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7888073470 con 0x7f788815b1e0 2024-07-17T07:12:39.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.853+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78880b4060 con 0x7f788815b1e0 2024-07-17T07:12:39.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.854+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f7870018070 con 0x7f788815b1e0 2024-07-17T07:12:39.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.855+0000 7f787f400640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 0x7f7864079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:39.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.856+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f7870098ba0 con 0x7f788815b1e0 2024-07-17T07:12:39.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.856+0000 7f7887400640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 0x7f7864079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:39.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.857+0000 7f7887400640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 0x7f7864079130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878006a00 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:39.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:39.860+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f78881549e0 con 0x7f788815b1e0 2024-07-17T07:12:40.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.120+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7888160e90 con 0x7f788815b1e0 2024-07-17T07:12:40.704 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:40.704+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:40.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.707+0000 7f787f400640 1 -- 172.21.15.44:0/431342879 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) ==== 118+0+0 (secure 0 0 0) 0x7f7870061ae0 con 0x7f788815b1e0 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 msgr2=0x7f7864079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 0x7f7864079130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f7878006a00 comp rx=0 tx=0).stop 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 msgr2=0x7f788814e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7870000fb0 tx=0x7f787000ea20 comp rx=0 tx=0).stop 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 shutdown_connections 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f7864076c70 0x7f7864079130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7888151fa0 0x7f7888158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f788815b1e0 0x7f788814e950 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 --2- 172.21.15.44:0/431342879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f786c0052e0 0x7f7888073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 >> 172.21.15.44:0/431342879 conn(0x7f78880b0eb0 msgr2=0x7f78880b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 shutdown_connections 2024-07-17T07:12:40.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.712+0000 7f788e600640 1 -- 172.21.15.44:0/431342879 wait complete. 2024-07-17T07:12:40.736 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:12:40.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:12:40.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 -- 172.21.15.44:0/3667523741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 msgr2=0x7fd2e80b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:40.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 --2- 172.21.15.44:0/3667523741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e80b5fb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009860 tx=0x7fd2d802a540 comp rx=0 tx=0).stop 2024-07-17T07:12:40.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 -- 172.21.15.44:0/3667523741 shutdown_connections 2024-07-17T07:12:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 --2- 172.21.15.44:0/3667523741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e806dda0 0x7fd2e80ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 --2- 172.21.15.44:0/3667523741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e80b5fb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 -- 172.21.15.44:0/3667523741 >> 172.21.15.44:0/3667523741 conn(0x7fd2e8073d50 msgr2=0x7fd2e809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 -- 172.21.15.44:0/3667523741 shutdown_connections 2024-07-17T07:12:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.951+0000 7fd2ef000640 1 -- 172.21.15.44:0/3667523741 wait complete. 2024-07-17T07:12:40.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.953+0000 7fd2ef000640 1 Processor -- start 2024-07-17T07:12:40.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.953+0000 7fd2ef000640 1 -- start start 2024-07-17T07:12:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e806dda0 0x7fd2e8160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 0x7fd2e815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80aed40 con 0x7fd2e815d810 2024-07-17T07:12:40.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e80aebc0 con 0x7fd2e806dda0 2024-07-17T07:12:40.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ef000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e80aeec0 con 0x7fd2e80b1a10 2024-07-17T07:12:40.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ed000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:40.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ed000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38786/0 (socket says 172.21.15.44:38786) 2024-07-17T07:12:40.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.954+0000 7fd2ed000640 1 -- 172.21.15.44:0/1475324141 learned_addr learned my addr 172.21.15.44:0/1475324141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:40.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 -- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 msgr2=0x7fd2e815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:40.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2e7400640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 0x7fd2e815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:40.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 0x7fd2e815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 -- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e806dda0 msgr2=0x7fd2e8160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:40.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e806dda0 0x7fd2e8160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 -- 172.21.15.44:0/1475324141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e814bf80 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2e7400640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 0x7fd2e815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.955+0000 7fd2ed000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000cab0 tx=0x7fd2e000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.956+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000ae20 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.956+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2e000f450 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.956+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000f8c0 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.956+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e81585a0 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.956+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e815f970 con 0x7fd2e815d810 2024-07-17T07:12:40.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.957+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e80af370 con 0x7fd2e815d810 2024-07-17T07:12:40.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.958+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fd2e000fa60 con 0x7fd2e815d810 2024-07-17T07:12:40.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.959+0000 7fd2dfe00640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 0x7fd2c8079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:40.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.959+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fd2e0099270 con 0x7fd2e815d810 2024-07-17T07:12:40.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.960+0000 7fd2e7e00640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 0x7fd2c8079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:40.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.961+0000 7fd2e7e00640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 0x7fd2c8079130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8034000 tx=0x7fd2d80046c0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:40.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:40.963+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd2e814bf80 con 0x7fd2e815d810 2024-07-17T07:12:41.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.224+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd2e80ae610 con 0x7fd2e815d810 2024-07-17T07:12:41.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.718+0000 7fd2dfe00640 1 -- 172.21.15.44:0/1475324141 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7fd2e0014030 con 0x7fd2e815d810 2024-07-17T07:12:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.722+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 msgr2=0x7fd2c8079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.722+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 0x7fd2c8079130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8034000 tx=0x7fd2d80046c0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.722+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 msgr2=0x7fd2e815f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.722+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000cab0 tx=0x7fd2e000cf70 comp rx=0 tx=0).stop 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 shutdown_connections 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd2c8076c70 0x7fd2c8079130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2e815d810 0x7fd2e815f300 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2e80b1a10 0x7fd2e815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 --2- 172.21.15.44:0/1475324141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2e806dda0 0x7fd2e8160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 >> 172.21.15.44:0/1475324141 conn(0x7fd2e8073d50 msgr2=0x7fd2e8070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 shutdown_connections 2024-07-17T07:12:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.723+0000 7fd2ef000640 1 -- 172.21.15.44:0/1475324141 wait complete. 2024-07-17T07:12:41.736 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:41.736+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:41.744 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:12:41.744 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 -- 172.21.15.44:0/728744292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 msgr2=0x7fd83c06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 --2- 172.21.15.44:0/728744292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c06d9b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd828009900 tx=0x7fd82802a600 comp rx=0 tx=0).stop 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 -- 172.21.15.44:0/728744292 shutdown_connections 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 --2- 172.21.15.44:0/728744292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c06d9b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 --2- 172.21.15.44:0/728744292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b8c80 0x7fd83c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 -- 172.21.15.44:0/728744292 >> 172.21.15.44:0/728744292 conn(0x7fd83c0cfec0 msgr2=0x7fd83c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 -- 172.21.15.44:0/728744292 shutdown_connections 2024-07-17T07:12:41.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.962+0000 7fd840a00640 1 -- 172.21.15.44:0/728744292 wait complete. 2024-07-17T07:12:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.964+0000 7fd840a00640 1 Processor -- start 2024-07-17T07:12:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.964+0000 7fd840a00640 1 -- start start 2024-07-17T07:12:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 0x7fd83c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 0x7fd83c0b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c06e780 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c06e600 con 0x7fd83c0b8c80 2024-07-17T07:12:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd840a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd83c06e900 con 0x7fd83c0b5400 2024-07-17T07:12:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.965+0000 7fd83b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38808/0 (socket says 172.21.15.44:38808) 2024-07-17T07:12:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 -- 172.21.15.44:0/1619210142 learned_addr learned my addr 172.21.15.44:0/1619210142 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd839e00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 0x7fd83c0b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83a800640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 0x7fd83c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 -- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 msgr2=0x7fd83c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 0x7fd83c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 -- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 msgr2=0x7fd83c0b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 0x7fd83c0b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 -- 172.21.15.44:0/1619210142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c0ca820 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83a800640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 0x7fd83c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd839e00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 0x7fd83c0b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.966+0000 7fd83b200640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c7a0 tx=0x7fd83400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.967+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834003b70 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.967+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd834003d10 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.967+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c0cab00 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.967+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83400aab0 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.967+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c0cb0c0 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.968+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c0d43b0 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.969+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fd83400ac50 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.970+0000 7fd832a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 0x7fd81c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:41.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.970+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fd834098650 con 0x7fd83c0ce5c0 2024-07-17T07:12:41.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.971+0000 7fd83a800640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 0x7fd81c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:41.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.972+0000 7fd83a800640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 0x7fd81c079100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00baa0 tx=0x7fd82c005d90 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:41.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:41.975+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd83c0d43b0 con 0x7fd83c0ce5c0 2024-07-17T07:12:42.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.239+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83c0b5d70 con 0x7fd83c0ce5c0 2024-07-17T07:12:42.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.241+0000 7fd832a00640 1 -- 172.21.15.44:0/1619210142 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7fd834061730 con 0x7fd83c0ce5c0 2024-07-17T07:12:42.240 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:42.240 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:12:42.240 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:12:42.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 msgr2=0x7fd81c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 0x7fd81c079100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00baa0 tx=0x7fd82c005d90 comp rx=0 tx=0).stop 2024-07-17T07:12:42.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 msgr2=0x7fd83c0b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c7a0 tx=0x7fd83400cc60 comp rx=0 tx=0).stop 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 shutdown_connections 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd81c076c40 0x7fd81c079100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd83c0ce5c0 0x7fd83c0b7800 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd83c0b8c80 0x7fd83c0b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 --2- 172.21.15.44:0/1619210142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd83c0b5400 0x7fd83c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 >> 172.21.15.44:0/1619210142 conn(0x7fd83c0cfec0 msgr2=0x7fd83c15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 shutdown_connections 2024-07-17T07:12:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.244+0000 7fd840a00640 1 -- 172.21.15.44:0/1619210142 wait complete. 2024-07-17T07:12:42.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:12:42.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 -- 172.21.15.44:0/1479795905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 msgr2=0x7f42300ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 --2- 172.21.15.44:0/1479795905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300ac650 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f423015dcd0 tx=0x7f422002a560 comp rx=0 tx=0).stop 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 -- 172.21.15.44:0/1479795905 shutdown_connections 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 --2- 172.21.15.44:0/1479795905 >> v2:172.21.15.175:3300/0 conn(0x7f423006f730 0x7f4230153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 --2- 172.21.15.44:0/1479795905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300ac650 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 -- 172.21.15.44:0/1479795905 >> 172.21.15.44:0/1479795905 conn(0x7f42300b0fb0 msgr2=0x7f42300b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 -- 172.21.15.44:0/1479795905 shutdown_connections 2024-07-17T07:12:42.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.477+0000 7f4235400640 1 -- 172.21.15.44:0/1479795905 wait complete. 2024-07-17T07:12:42.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.479+0000 7f4235400640 1 Processor -- start 2024-07-17T07:12:42.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.479+0000 7f4235400640 1 -- start start 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f423006f730 0x7f42300ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42300acb90 0x7f42300cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4230154710 con 0x7f423015b2e0 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4230154590 con 0x7f423006f730 2024-07-17T07:12:42.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.480+0000 7f4235400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4230154890 con 0x7f42300acb90 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38824/0 (socket says 172.21.15.44:38824) 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 -- 172.21.15.44:0/2676486075 learned_addr learned my addr 172.21.15.44:0/2676486075 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422e800640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f423006f730 0x7f42300ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422de00640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42300acb90 0x7f42300cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 -- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42300acb90 msgr2=0x7f42300cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42300acb90 0x7f42300cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 -- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f423006f730 msgr2=0x7f42300ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f423006f730 0x7f42300ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.481+0000 7f422f200640 1 -- 172.21.15.44:0/2676486075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42300cc1c0 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f422f200640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f422800b9e0 tx=0x7f422800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228019070 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f422800cbb0 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228014440 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42300cc4a0 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.482+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42300cca60 con 0x7f423015b2e0 2024-07-17T07:12:42.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.483+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4230153e40 con 0x7f423015b2e0 2024-07-17T07:12:42.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.485+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f422801c070 con 0x7f423015b2e0 2024-07-17T07:12:42.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.486+0000 7f4226a00640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 0x7f4210079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.486+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f4228098520 con 0x7f423015b2e0 2024-07-17T07:12:42.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.487+0000 7f422e800640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 0x7f4210079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.488+0000 7f422e800640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 0x7f4210079100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4220033000 tx=0x7f4220004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:42.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.490+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4228061570 con 0x7f423015b2e0 2024-07-17T07:12:42.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.713+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f423006dbf0 con 0x7f423015b2e0 2024-07-17T07:12:42.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.716+0000 7f4226a00640 1 -- 172.21.15.44:0/2676486075 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17670 (secure 0 0 0) 0x7f4228060e70 con 0x7f423015b2e0 2024-07-17T07:12:42.716 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:42.716 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":112,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:12:39.664772+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:36.099092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:37.121054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:12:42.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 msgr2=0x7f4210079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 0x7f4210079100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4220033000 tx=0x7f4220004730 comp rx=0 tx=0).stop 2024-07-17T07:12:42.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 msgr2=0x7f42300b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f422800b9e0 tx=0x7f422800bea0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 shutdown_connections 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4210076c40 0x7f4210079100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423015b2e0 0x7f42300b0010 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42300acb90 0x7f42300cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 --2- 172.21.15.44:0/2676486075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f423006f730 0x7f42300ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 >> 172.21.15.44:0/2676486075 conn(0x7f42300b0fb0 msgr2=0x7f42300b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 shutdown_connections 2024-07-17T07:12:42.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.720+0000 7f4235400640 1 -- 172.21.15.44:0/2676486075 wait complete. 2024-07-17T07:12:42.740 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:12:42.740 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:12:42.745 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:42.745+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:42.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.945+0000 7fb76ac00640 1 -- 172.21.15.44:0/897517058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 msgr2=0x7fb748004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.945+0000 7fb76ac00640 1 --2- 172.21.15.44:0/897517058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb748004c80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb7640b0000 tx=0x7fb748005830 comp rx=0 tx=0).stop 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.945+0000 7fb768c00640 1 -- 172.21.15.44:0/897517058 reap_dead start 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.945+0000 7fb76ac00640 1 -- 172.21.15.44:0/897517058 shutdown_connections 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.945+0000 7fb76ac00640 1 -- 172.21.15.44:0/897517058 >> 172.21.15.44:0/897517058 conn(0x7fb7640b2240 msgr2=0x7fb7640b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.946+0000 7fb76ac00640 1 -- 172.21.15.44:0/897517058 shutdown_connections 2024-07-17T07:12:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.946+0000 7fb76ac00640 1 -- 172.21.15.44:0/897517058 wait complete. 2024-07-17T07:12:42.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.947+0000 7fb76ac00640 1 Processor -- start 2024-07-17T07:12:42.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.947+0000 7fb76ac00640 1 -- start start 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.948+0000 7fb76ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.948+0000 7fb76ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb764152010 0x7fb764072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb76ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb76414e9a0 0x7fb76414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb76ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640d0160 con 0x7fb748004800 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb76ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7640cffe0 con 0x7fb76414e9a0 2024-07-17T07:12:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb76ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7640b5cb0 con 0x7fb764152010 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38842/0 (socket says 172.21.15.44:38842) 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 -- 172.21.15.44:0/2163808209 learned_addr learned my addr 172.21.15.44:0/2163808209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 -- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb764152010 msgr2=0x7fb764072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb764152010 0x7fb764072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 -- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb76414e9a0 msgr2=0x7fb76414ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb76414e9a0 0x7fb76414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 -- 172.21.15.44:0/2163808209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb764152860 con 0x7fb748004800 2024-07-17T07:12:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.949+0000 7fb763e00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb75400f9b0 tx=0x7fb75400fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.950+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75400bc50 con 0x7fb748004800 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.950+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb754011440 con 0x7fb748004800 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.950+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754011870 con 0x7fb748004800 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.950+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb764152b40 con 0x7fb748004800 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.950+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb764153100 con 0x7fb748004800 2024-07-17T07:12:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.951+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76415ef50 con 0x7fb748004800 2024-07-17T07:12:42.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.952+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fb754019020 con 0x7fb748004800 2024-07-17T07:12:42.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.953+0000 7fb75be00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 0x7fb744079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:42.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.953+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fb7540991d0 con 0x7fb748004800 2024-07-17T07:12:42.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.954+0000 7fb763400640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 0x7fb744079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:42.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.955+0000 7fb763400640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 0x7fb744079100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb750033000 tx=0x7fb7640cf4c0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:42.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:42.958+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb754062110 con 0x7fb748004800 2024-07-17T07:12:43.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.167+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 --> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7640b0c60 con 0x7fb744076c40 2024-07-17T07:12:43.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.184+0000 7fb75be00640 1 -- 172.21.15.44:0/2163808209 <== mgr.5923 v2:172.21.15.175:6800/3550495011 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7fb7640b0c60 con 0x7fb744076c40 2024-07-17T07:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:43.189 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:12:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 msgr2=0x7fb744079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 0x7fb744079100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb750033000 tx=0x7fb7640cf4c0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 msgr2=0x7fb76415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb75400f9b0 tx=0x7fb75400fe70 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 shutdown_connections 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fb744076c40 0x7fb744079100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb76414e9a0 0x7fb76414ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb764152010 0x7fb764072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 --2- 172.21.15.44:0/2163808209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb748004800 0x7fb76415e820 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 >> 172.21.15.44:0/2163808209 conn(0x7fb7640b2240 msgr2=0x7fb7640af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.191+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 shutdown_connections 2024-07-17T07:12:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.192+0000 7fb76ac00640 1 -- 172.21.15.44:0/2163808209 wait complete. 2024-07-17T07:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:12:42.404578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":224,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1631,"ondisk_log_size":1631,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":336316,"kb_used_data":11908,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754638404,"statfs":{"total":773094113280,"available":772749725696,"internally_reserved":0,"allocated":12193792,"data_stored":6913736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":332058160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.781324"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671483+0000","last_change":"2024-07-17T07:12:38.713310+0000","last_active":"2024-07-17T07:12:39.671483+0000","last_peered":"2024-07-17T07:12:39.671483+0000","last_clean":"2024-07-17T07:12:39.671483+0000","last_became_active":"2024-07-17T07:12:38.713036+0000","last_became_peered":"2024-07-17T07:12:38.713036+0000","last_unstale":"2024-07-17T07:12:39.671483+0000","last_undegraded":"2024-07-17T07:12:39.671483+0000","last_fullsized":"2024-07-17T07:12:39.671483+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:04.899929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678167+0000","last_change":"2024-07-17T07:12:38.707638+0000","last_active":"2024-07-17T07:12:39.678167+0000","last_peered":"2024-07-17T07:12:39.678167+0000","last_clean":"2024-07-17T07:12:39.678167+0000","last_became_active":"2024-07-17T07:12:38.706605+0000","last_became_peered":"2024-07-17T07:12:38.706605+0000","last_unstale":"2024-07-17T07:12:39.678167+0000","last_undegraded":"2024-07-17T07:12:39.678167+0000","last_fullsized":"2024-07-17T07:12:39.678167+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:47:37.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671080+0000","last_change":"2024-07-17T07:12:38.713390+0000","last_active":"2024-07-17T07:12:39.671080+0000","last_peered":"2024-07-17T07:12:39.671080+0000","last_clean":"2024-07-17T07:12:39.671080+0000","last_became_active":"2024-07-17T07:12:38.709084+0000","last_became_peered":"2024-07-17T07:12:38.709084+0000","last_unstale":"2024-07-17T07:12:39.671080+0000","last_undegraded":"2024-07-17T07:12:39.671080+0000","last_fullsized":"2024-07-17T07:12:39.671080+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:44.007112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741306+0000","last_change":"2024-07-17T07:12:38.711376+0000","last_active":"2024-07-17T07:12:39.741306+0000","last_peered":"2024-07-17T07:12:39.741306+0000","last_clean":"2024-07-17T07:12:39.741306+0000","last_became_active":"2024-07-17T07:12:38.710869+0000","last_became_peered":"2024-07-17T07:12:38.710869+0000","last_unstale":"2024-07-17T07:12:39.741306+0000","last_undegraded":"2024-07-17T07:12:39.741306+0000","last_fullsized":"2024-07-17T07:12:39.741306+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:11.228490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715106+0000","last_change":"2024-07-17T07:12:38.701213+0000","last_active":"2024-07-17T07:12:39.715106+0000","last_peered":"2024-07-17T07:12:39.715106+0000","last_clean":"2024-07-17T07:12:39.715106+0000","last_became_active":"2024-07-17T07:12:38.700909+0000","last_became_peered":"2024-07-17T07:12:38.700909+0000","last_unstale":"2024-07-17T07:12:39.715106+0000","last_undegraded":"2024-07-17T07:12:39.715106+0000","last_fullsized":"2024-07-17T07:12:39.715106+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:12:09.728310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741277+0000","last_change":"2024-07-17T07:12:38.722593+0000","last_active":"2024-07-17T07:12:39.741277+0000","last_peered":"2024-07-17T07:12:39.741277+0000","last_clean":"2024-07-17T07:12:39.741277+0000","last_became_active":"2024-07-17T07:12:38.722271+0000","last_became_peered":"2024-07-17T07:12:38.722271+0000","last_unstale":"2024-07-17T07:12:39.741277+0000","last_undegraded":"2024-07-17T07:12:39.741277+0000","last_fullsized":"2024-07-17T07:12:39.741277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:43.218920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741845+0000","last_change":"2024-07-17T07:12:38.703404+0000","last_active":"2024-07-17T07:12:39.741845+0000","last_peered":"2024-07-17T07:12:39.741845+0000","last_clean":"2024-07-17T07:12:39.741845+0000","last_became_active":"2024-07-17T07:12:38.703070+0000","last_became_peered":"2024-07-17T07:12:38.703070+0000","last_unstale":"2024-07-17T07:12:39.741845+0000","last_undegraded":"2024-07-17T07:12:39.741845+0000","last_fullsized":"2024-07-17T07:12:39.741845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:54:18.887110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741486+0000","last_change":"2024-07-17T07:12:38.726893+0000","last_active":"2024-07-17T07:12:39.741486+0000","last_peered":"2024-07-17T07:12:39.741486+0000","last_clean":"2024-07-17T07:12:39.741486+0000","last_became_active":"2024-07-17T07:12:38.726357+0000","last_became_peered":"2024-07-17T07:12:38.726357+0000","last_unstale":"2024-07-17T07:12:39.741486+0000","last_undegraded":"2024-07-17T07:12:39.741486+0000","last_fullsized":"2024-07-17T07:12:39.741486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:47:51.342467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716581+0000","last_change":"2024-07-17T07:12:38.704233+0000","last_active":"2024-07-17T07:12:39.716581+0000","last_peered":"2024-07-17T07:12:39.716581+0000","last_clean":"2024-07-17T07:12:39.716581+0000","last_became_active":"2024-07-17T07:12:38.703831+0000","last_became_peered":"2024-07-17T07:12:38.703831+0000","last_unstale":"2024-07-17T07:12:39.716581+0000","last_undegraded":"2024-07-17T07:12:39.716581+0000","last_fullsized":"2024-07-17T07:12:39.716581+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:56.733128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740873+0000","last_change":"2024-07-17T07:12:38.716131+0000","last_active":"2024-07-17T07:12:39.740873+0000","last_peered":"2024-07-17T07:12:39.740873+0000","last_clean":"2024-07-17T07:12:39.740873+0000","last_became_active":"2024-07-17T07:12:38.710773+0000","last_became_peered":"2024-07-17T07:12:38.710773+0000","last_unstale":"2024-07-17T07:12:39.740873+0000","last_undegraded":"2024-07-17T07:12:39.740873+0000","last_fullsized":"2024-07-17T07:12:39.740873+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:36.113058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716125+0000","last_change":"2024-07-17T07:12:38.700853+0000","last_active":"2024-07-17T07:12:39.716125+0000","last_peered":"2024-07-17T07:12:39.716125+0000","last_clean":"2024-07-17T07:12:39.716125+0000","last_became_active":"2024-07-17T07:12:38.700511+0000","last_became_peered":"2024-07-17T07:12:38.700511+0000","last_unstale":"2024-07-17T07:12:39.716125+0000","last_undegraded":"2024-07-17T07:12:39.716125+0000","last_fullsized":"2024-07-17T07:12:39.716125+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:13.100796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740428+0000","last_change":"2024-07-17T07:12:38.704874+0000","last_active":"2024-07-17T07:12:39.740428+0000","last_peered":"2024-07-17T07:12:39.740428+0000","last_clean":"2024-07-17T07:12:39.740428+0000","last_became_active":"2024-07-17T07:12:38.704539+0000","last_became_peered":"2024-07-17T07:12:38.704539+0000","last_unstale":"2024-07-17T07:12:39.740428+0000","last_undegraded":"2024-07-17T07:12:39.740428+0000","last_fullsized":"2024-07-17T07:12:39.740428+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:26:01.322472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742646+0000","last_change":"2024-07-17T07:12:38.708402+0000","last_active":"2024-07-17T07:12:39.742646+0000","last_peered":"2024-07-17T07:12:39.742646+0000","last_clean":"2024-07-17T07:12:39.742646+0000","last_became_active":"2024-07-17T07:12:38.708092+0000","last_became_peered":"2024-07-17T07:12:38.708092+0000","last_unstale":"2024-07-17T07:12:39.742646+0000","last_undegraded":"2024-07-17T07:12:39.742646+0000","last_fullsized":"2024-07-17T07:12:39.742646+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:49.036240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714761+0000","last_change":"2024-07-17T07:12:38.722603+0000","last_active":"2024-07-17T07:12:39.714761+0000","last_peered":"2024-07-17T07:12:39.714761+0000","last_clean":"2024-07-17T07:12:39.714761+0000","last_became_active":"2024-07-17T07:12:38.722091+0000","last_became_peered":"2024-07-17T07:12:38.722091+0000","last_unstale":"2024-07-17T07:12:39.714761+0000","last_undegraded":"2024-07-17T07:12:39.714761+0000","last_fullsized":"2024-07-17T07:12:39.714761+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:47:47.681480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741790+0000","last_change":"2024-07-17T07:12:38.715969+0000","last_active":"2024-07-17T07:12:39.741790+0000","last_peered":"2024-07-17T07:12:39.741790+0000","last_clean":"2024-07-17T07:12:39.741790+0000","last_became_active":"2024-07-17T07:12:38.715630+0000","last_became_peered":"2024-07-17T07:12:38.715630+0000","last_unstale":"2024-07-17T07:12:39.741790+0000","last_undegraded":"2024-07-17T07:12:39.741790+0000","last_fullsized":"2024-07-17T07:12:39.741790+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:29:45.431930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740577+0000","last_change":"2024-07-17T07:12:38.703871+0000","last_active":"2024-07-17T07:12:39.740577+0000","last_peered":"2024-07-17T07:12:39.740577+0000","last_clean":"2024-07-17T07:12:39.740577+0000","last_became_active":"2024-07-17T07:12:38.703270+0000","last_became_peered":"2024-07-17T07:12:38.703270+0000","last_unstale":"2024-07-17T07:12:39.740577+0000","last_undegraded":"2024-07-17T07:12:39.740577+0000","last_fullsized":"2024-07-17T07:12:39.740577+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:37.983657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743685+0000","last_change":"2024-07-17T07:12:38.708354+0000","last_active":"2024-07-17T07:12:39.743685+0000","last_peered":"2024-07-17T07:12:39.743685+0000","last_clean":"2024-07-17T07:12:39.743685+0000","last_became_active":"2024-07-17T07:12:38.703621+0000","last_became_peered":"2024-07-17T07:12:38.703621+0000","last_unstale":"2024-07-17T07:12:39.743685+0000","last_undegraded":"2024-07-17T07:12:39.743685+0000","last_fullsized":"2024-07-17T07:12:39.743685+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:24:09.571539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671335+0000","last_change":"2024-07-17T07:12:38.704075+0000","last_active":"2024-07-17T07:12:39.671335+0000","last_peered":"2024-07-17T07:12:39.671335+0000","last_clean":"2024-07-17T07:12:39.671335+0000","last_became_active":"2024-07-17T07:12:38.702156+0000","last_became_peered":"2024-07-17T07:12:38.702156+0000","last_unstale":"2024-07-17T07:12:39.671335+0000","last_undegraded":"2024-07-17T07:12:39.671335+0000","last_fullsized":"2024-07-17T07:12:39.671335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:18.247691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716005+0000","last_change":"2024-07-17T07:12:38.712498+0000","last_active":"2024-07-17T07:12:39.716005+0000","last_peered":"2024-07-17T07:12:39.716005+0000","last_clean":"2024-07-17T07:12:39.716005+0000","last_became_active":"2024-07-17T07:12:38.711367+0000","last_became_peered":"2024-07-17T07:12:38.711367+0000","last_unstale":"2024-07-17T07:12:39.716005+0000","last_undegraded":"2024-07-17T07:12:39.716005+0000","last_fullsized":"2024-07-17T07:12:39.716005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:40.190448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741514+0000","last_change":"2024-07-17T07:12:38.709329+0000","last_active":"2024-07-17T07:12:39.741514+0000","last_peered":"2024-07-17T07:12:39.741514+0000","last_clean":"2024-07-17T07:12:39.741514+0000","last_became_active":"2024-07-17T07:12:38.708992+0000","last_became_peered":"2024-07-17T07:12:38.708992+0000","last_unstale":"2024-07-17T07:12:39.741514+0000","last_undegraded":"2024-07-17T07:12:39.741514+0000","last_fullsized":"2024-07-17T07:12:39.741514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:38.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714909+0000","last_change":"2024-07-17T07:12:38.710151+0000","last_active":"2024-07-17T07:12:39.714909+0000","last_peered":"2024-07-17T07:12:39.714909+0000","last_clean":"2024-07-17T07:12:39.714909+0000","last_became_active":"2024-07-17T07:12:38.709291+0000","last_became_peered":"2024-07-17T07:12:38.709291+0000","last_unstale":"2024-07-17T07:12:39.714909+0000","last_undegraded":"2024-07-17T07:12:39.714909+0000","last_fullsized":"2024-07-17T07:12:39.714909+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.463184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714892+0000","last_change":"2024-07-17T07:12:38.715606+0000","last_active":"2024-07-17T07:12:39.714892+0000","last_peered":"2024-07-17T07:12:39.714892+0000","last_clean":"2024-07-17T07:12:39.714892+0000","last_became_active":"2024-07-17T07:12:38.714895+0000","last_became_peered":"2024-07-17T07:12:38.714895+0000","last_unstale":"2024-07-17T07:12:39.714892+0000","last_undegraded":"2024-07-17T07:12:39.714892+0000","last_fullsized":"2024-07-17T07:12:39.714892+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:31.466863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677307+0000","last_change":"2024-07-17T07:12:38.715282+0000","last_active":"2024-07-17T07:12:39.677307+0000","last_peered":"2024-07-17T07:12:39.677307+0000","last_clean":"2024-07-17T07:12:39.677307+0000","last_became_active":"2024-07-17T07:12:38.715004+0000","last_became_peered":"2024-07-17T07:12:38.715004+0000","last_unstale":"2024-07-17T07:12:39.677307+0000","last_undegraded":"2024-07-17T07:12:39.677307+0000","last_fullsized":"2024-07-17T07:12:39.677307+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:40.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.670879+0000","last_change":"2024-07-17T07:12:38.704937+0000","last_active":"2024-07-17T07:12:39.670879+0000","last_peered":"2024-07-17T07:12:39.670879+0000","last_clean":"2024-07-17T07:12:39.670879+0000","last_became_active":"2024-07-17T07:12:38.703518+0000","last_became_peered":"2024-07-17T07:12:38.703518+0000","last_unstale":"2024-07-17T07:12:39.670879+0000","last_undegraded":"2024-07-17T07:12:39.670879+0000","last_fullsized":"2024-07-17T07:12:39.670879+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:51.009618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678371+0000","last_change":"2024-07-17T07:12:38.705084+0000","last_active":"2024-07-17T07:12:39.678371+0000","last_peered":"2024-07-17T07:12:39.678371+0000","last_clean":"2024-07-17T07:12:39.678371+0000","last_became_active":"2024-07-17T07:12:38.704253+0000","last_became_peered":"2024-07-17T07:12:38.704253+0000","last_unstale":"2024-07-17T07:12:39.678371+0000","last_undegraded":"2024-07-17T07:12:39.678371+0000","last_fullsized":"2024-07-17T07:12:39.678371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:32.835642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716206+0000","last_change":"2024-07-17T07:12:38.709905+0000","last_active":"2024-07-17T07:12:39.716206+0000","last_peered":"2024-07-17T07:12:39.716206+0000","last_clean":"2024-07-17T07:12:39.716206+0000","last_became_active":"2024-07-17T07:12:38.709620+0000","last_became_peered":"2024-07-17T07:12:38.709620+0000","last_unstale":"2024-07-17T07:12:39.716206+0000","last_undegraded":"2024-07-17T07:12:39.716206+0000","last_fullsized":"2024-07-17T07:12:39.716206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:38.168971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741506+0000","last_change":"2024-07-17T07:12:38.689483+0000","last_active":"2024-07-17T07:12:39.741506+0000","last_peered":"2024-07-17T07:12:39.741506+0000","last_clean":"2024-07-17T07:12:39.741506+0000","last_became_active":"2024-07-17T07:12:38.689014+0000","last_became_peered":"2024-07-17T07:12:38.689014+0000","last_unstale":"2024-07-17T07:12:39.741506+0000","last_undegraded":"2024-07-17T07:12:39.741506+0000","last_fullsized":"2024-07-17T07:12:39.741506+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:18.444879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741757+0000","last_change":"2024-07-17T07:12:38.723363+0000","last_active":"2024-07-17T07:12:39.741757+0000","last_peered":"2024-07-17T07:12:39.741757+0000","last_clean":"2024-07-17T07:12:39.741757+0000","last_became_active":"2024-07-17T07:12:38.719622+0000","last_became_peered":"2024-07-17T07:12:38.719622+0000","last_unstale":"2024-07-17T07:12:39.741757+0000","last_undegraded":"2024-07-17T07:12:39.741757+0000","last_fullsized":"2024-07-17T07:12:39.741757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:47.486876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742006+0000","last_change":"2024-07-17T07:12:38.702081+0000","last_active":"2024-07-17T07:12:39.742006+0000","last_peered":"2024-07-17T07:12:39.742006+0000","last_clean":"2024-07-17T07:12:39.742006+0000","last_became_active":"2024-07-17T07:12:38.700536+0000","last_became_peered":"2024-07-17T07:12:38.700536+0000","last_unstale":"2024-07-17T07:12:39.742006+0000","last_undegraded":"2024-07-17T07:12:39.742006+0000","last_fullsized":"2024-07-17T07:12:39.742006+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:42.857145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715180+0000","last_change":"2024-07-17T07:12:38.707375+0000","last_active":"2024-07-17T07:12:39.715180+0000","last_peered":"2024-07-17T07:12:39.715180+0000","last_clean":"2024-07-17T07:12:39.715180+0000","last_became_active":"2024-07-17T07:12:38.702549+0000","last_became_peered":"2024-07-17T07:12:38.702549+0000","last_unstale":"2024-07-17T07:12:39.715180+0000","last_undegraded":"2024-07-17T07:12:39.715180+0000","last_fullsized":"2024-07-17T07:12:39.715180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:37.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.673850+0000","last_change":"2024-07-17T07:12:38.715626+0000","last_active":"2024-07-17T07:12:39.673850+0000","last_peered":"2024-07-17T07:12:39.673850+0000","last_clean":"2024-07-17T07:12:39.673850+0000","last_became_active":"2024-07-17T07:12:38.715338+0000","last_became_peered":"2024-07-17T07:12:38.715338+0000","last_unstale":"2024-07-17T07:12:39.673850+0000","last_undegraded":"2024-07-17T07:12:39.673850+0000","last_fullsized":"2024-07-17T07:12:39.673850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:13.872210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741096+0000","last_change":"2024-07-17T07:12:38.703602+0000","last_active":"2024-07-17T07:12:39.741096+0000","last_peered":"2024-07-17T07:12:39.741096+0000","last_clean":"2024-07-17T07:12:39.741096+0000","last_became_active":"2024-07-17T07:12:38.702798+0000","last_became_peered":"2024-07-17T07:12:38.702798+0000","last_unstale":"2024-07-17T07:12:39.741096+0000","last_undegraded":"2024-07-17T07:12:39.741096+0000","last_fullsized":"2024-07-17T07:12:39.741096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:05.026869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742694+0000","last_change":"2024-07-17T07:12:38.703945+0000","last_active":"2024-07-17T07:12:39.742694+0000","last_peered":"2024-07-17T07:12:39.742694+0000","last_clean":"2024-07-17T07:12:39.742694+0000","last_became_active":"2024-07-17T07:12:38.703424+0000","last_became_peered":"2024-07-17T07:12:38.703424+0000","last_unstale":"2024-07-17T07:12:39.742694+0000","last_undegraded":"2024-07-17T07:12:39.742694+0000","last_fullsized":"2024-07-17T07:12:39.742694+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:59.250951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677640+0000","last_change":"2024-07-17T07:12:37.658029+0000","last_active":"2024-07-17T07:12:39.677640+0000","last_peered":"2024-07-17T07:12:39.677640+0000","last_clean":"2024-07-17T07:12:39.677640+0000","last_became_active":"2024-07-17T07:12:37.657698+0000","last_became_peered":"2024-07-17T07:12:37.657698+0000","last_unstale":"2024-07-17T07:12:39.677640+0000","last_undegraded":"2024-07-17T07:12:39.677640+0000","last_fullsized":"2024-07-17T07:12:39.677640+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:29:44.414198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743005+0000","last_change":"2024-07-17T07:12:38.695671+0000","last_active":"2024-07-17T07:12:39.743005+0000","last_peered":"2024-07-17T07:12:39.743005+0000","last_clean":"2024-07-17T07:12:39.743005+0000","last_became_active":"2024-07-17T07:12:38.693759+0000","last_became_peered":"2024-07-17T07:12:38.693759+0000","last_unstale":"2024-07-17T07:12:39.743005+0000","last_undegraded":"2024-07-17T07:12:39.743005+0000","last_fullsized":"2024-07-17T07:12:39.743005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:03.876326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716573+0000","last_change":"2024-07-17T07:12:37.669918+0000","last_active":"2024-07-17T07:12:39.716573+0000","last_peered":"2024-07-17T07:12:39.716573+0000","last_clean":"2024-07-17T07:12:39.716573+0000","last_became_active":"2024-07-17T07:12:37.669489+0000","last_became_peered":"2024-07-17T07:12:37.669489+0000","last_unstale":"2024-07-17T07:12:39.716573+0000","last_undegraded":"2024-07-17T07:12:39.716573+0000","last_fullsized":"2024-07-17T07:12:39.716573+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:22:22.398136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741989+0000","last_change":"2024-07-17T07:12:38.709936+0000","last_active":"2024-07-17T07:12:39.741989+0000","last_peered":"2024-07-17T07:12:39.741989+0000","last_clean":"2024-07-17T07:12:39.741989+0000","last_became_active":"2024-07-17T07:12:38.709554+0000","last_became_peered":"2024-07-17T07:12:38.709554+0000","last_unstale":"2024-07-17T07:12:39.741989+0000","last_undegraded":"2024-07-17T07:12:39.741989+0000","last_fullsized":"2024-07-17T07:12:39.741989+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:16.982429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741889+0000","last_change":"2024-07-17T07:12:37.666229+0000","last_active":"2024-07-17T07:12:39.741889+0000","last_peered":"2024-07-17T07:12:39.741889+0000","last_clean":"2024-07-17T07:12:39.741889+0000","last_became_active":"2024-07-17T07:12:37.665881+0000","last_became_peered":"2024-07-17T07:12:37.665881+0000","last_unstale":"2024-07-17T07:12:39.741889+0000","last_undegraded":"2024-07-17T07:12:39.741889+0000","last_fullsized":"2024-07-17T07:12:39.741889+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:09.226047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743598+0000","last_change":"2024-07-17T07:12:38.685526+0000","last_active":"2024-07-17T07:12:39.743598+0000","last_peered":"2024-07-17T07:12:39.743598+0000","last_clean":"2024-07-17T07:12:39.743598+0000","last_became_active":"2024-07-17T07:12:38.685076+0000","last_became_peered":"2024-07-17T07:12:38.685076+0000","last_unstale":"2024-07-17T07:12:39.743598+0000","last_undegraded":"2024-07-17T07:12:39.743598+0000","last_fullsized":"2024-07-17T07:12:39.743598+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:21:14.108644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676957+0000","last_change":"2024-07-17T07:12:37.656657+0000","last_active":"2024-07-17T07:12:39.676957+0000","last_peered":"2024-07-17T07:12:39.676957+0000","last_clean":"2024-07-17T07:12:39.676957+0000","last_became_active":"2024-07-17T07:12:37.656050+0000","last_became_peered":"2024-07-17T07:12:37.656050+0000","last_unstale":"2024-07-17T07:12:39.676957+0000","last_undegraded":"2024-07-17T07:12:39.676957+0000","last_fullsized":"2024-07-17T07:12:39.676957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:58.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741704+0000","last_change":"2024-07-17T07:12:38.727829+0000","last_active":"2024-07-17T07:12:39.741704+0000","last_peered":"2024-07-17T07:12:39.741704+0000","last_clean":"2024-07-17T07:12:39.741704+0000","last_became_active":"2024-07-17T07:12:38.727495+0000","last_became_peered":"2024-07-17T07:12:38.727495+0000","last_unstale":"2024-07-17T07:12:39.741704+0000","last_undegraded":"2024-07-17T07:12:39.741704+0000","last_fullsized":"2024-07-17T07:12:39.741704+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:31.741494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715350+0000","last_change":"2024-07-17T07:12:37.667487+0000","last_active":"2024-07-17T07:12:39.715350+0000","last_peered":"2024-07-17T07:12:39.715350+0000","last_clean":"2024-07-17T07:12:39.715350+0000","last_became_active":"2024-07-17T07:12:37.667208+0000","last_became_peered":"2024-07-17T07:12:37.667208+0000","last_unstale":"2024-07-17T07:12:39.715350+0000","last_undegraded":"2024-07-17T07:12:39.715350+0000","last_fullsized":"2024-07-17T07:12:39.715350+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:56.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675561+0000","last_change":"2024-07-17T07:12:38.694819+0000","last_active":"2024-07-17T07:12:39.675561+0000","last_peered":"2024-07-17T07:12:39.675561+0000","last_clean":"2024-07-17T07:12:39.675561+0000","last_became_active":"2024-07-17T07:12:38.693717+0000","last_became_peered":"2024-07-17T07:12:38.693717+0000","last_unstale":"2024-07-17T07:12:39.675561+0000","last_undegraded":"2024-07-17T07:12:39.675561+0000","last_fullsized":"2024-07-17T07:12:39.675561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:33.904831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742561+0000","last_change":"2024-07-17T07:12:37.654437+0000","last_active":"2024-07-17T07:12:39.742561+0000","last_peered":"2024-07-17T07:12:39.742561+0000","last_clean":"2024-07-17T07:12:39.742561+0000","last_became_active":"2024-07-17T07:12:37.654101+0000","last_became_peered":"2024-07-17T07:12:37.654101+0000","last_unstale":"2024-07-17T07:12:39.742561+0000","last_undegraded":"2024-07-17T07:12:39.742561+0000","last_fullsized":"2024-07-17T07:12:39.742561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:33.663053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741183+0000","last_change":"2024-07-17T07:12:38.711793+0000","last_active":"2024-07-17T07:12:39.741183+0000","last_peered":"2024-07-17T07:12:39.741183+0000","last_clean":"2024-07-17T07:12:39.741183+0000","last_became_active":"2024-07-17T07:12:38.711388+0000","last_became_peered":"2024-07-17T07:12:38.711388+0000","last_unstale":"2024-07-17T07:12:39.741183+0000","last_undegraded":"2024-07-17T07:12:39.741183+0000","last_fullsized":"2024-07-17T07:12:39.741183+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:28.941309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671124+0000","last_change":"2024-07-17T07:12:37.666177+0000","last_active":"2024-07-17T07:12:39.671124+0000","last_peered":"2024-07-17T07:12:39.671124+0000","last_clean":"2024-07-17T07:12:39.671124+0000","last_became_active":"2024-07-17T07:12:37.665910+0000","last_became_peered":"2024-07-17T07:12:37.665910+0000","last_unstale":"2024-07-17T07:12:39.671124+0000","last_undegraded":"2024-07-17T07:12:39.671124+0000","last_fullsized":"2024-07-17T07:12:39.671124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:52.635951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742450+0000","last_change":"2024-07-17T07:12:38.712400+0000","last_active":"2024-07-17T07:12:39.742450+0000","last_peered":"2024-07-17T07:12:39.742450+0000","last_clean":"2024-07-17T07:12:39.742450+0000","last_became_active":"2024-07-17T07:12:38.712049+0000","last_became_peered":"2024-07-17T07:12:38.712049+0000","last_unstale":"2024-07-17T07:12:39.742450+0000","last_undegraded":"2024-07-17T07:12:39.742450+0000","last_fullsized":"2024-07-17T07:12:39.742450+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:04.351239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741550+0000","last_change":"2024-07-17T07:12:37.669060+0000","last_active":"2024-07-17T07:12:39.741550+0000","last_peered":"2024-07-17T07:12:39.741550+0000","last_clean":"2024-07-17T07:12:39.741550+0000","last_became_active":"2024-07-17T07:12:37.668810+0000","last_became_peered":"2024-07-17T07:12:37.668810+0000","last_unstale":"2024-07-17T07:12:39.741550+0000","last_undegraded":"2024-07-17T07:12:39.741550+0000","last_fullsized":"2024-07-17T07:12:39.741550+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:44.136760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715540+0000","last_change":"2024-07-17T07:12:38.709074+0000","last_active":"2024-07-17T07:12:39.715540+0000","last_peered":"2024-07-17T07:12:39.715540+0000","last_clean":"2024-07-17T07:12:39.715540+0000","last_became_active":"2024-07-17T07:12:38.708792+0000","last_became_peered":"2024-07-17T07:12:38.708792+0000","last_unstale":"2024-07-17T07:12:39.715540+0000","last_undegraded":"2024-07-17T07:12:39.715540+0000","last_fullsized":"2024-07-17T07:12:39.715540+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:17.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678603+0000","last_change":"2024-07-17T07:12:37.659519+0000","last_active":"2024-07-17T07:12:39.678603+0000","last_peered":"2024-07-17T07:12:39.678603+0000","last_clean":"2024-07-17T07:12:39.678603+0000","last_became_active":"2024-07-17T07:12:37.659211+0000","last_became_peered":"2024-07-17T07:12:37.659211+0000","last_unstale":"2024-07-17T07:12:39.678603+0000","last_undegraded":"2024-07-17T07:12:39.678603+0000","last_fullsized":"2024-07-17T07:12:39.678603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:11.283026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742051+0000","last_change":"2024-07-17T07:12:38.712142+0000","last_active":"2024-07-17T07:12:39.742051+0000","last_peered":"2024-07-17T07:12:39.742051+0000","last_clean":"2024-07-17T07:12:39.742051+0000","last_became_active":"2024-07-17T07:12:38.711020+0000","last_became_peered":"2024-07-17T07:12:38.711020+0000","last_unstale":"2024-07-17T07:12:39.742051+0000","last_undegraded":"2024-07-17T07:12:39.742051+0000","last_fullsized":"2024-07-17T07:12:39.742051+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:00.101735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671413+0000","last_change":"2024-07-17T07:12:37.665540+0000","last_active":"2024-07-17T07:12:39.671413+0000","last_peered":"2024-07-17T07:12:39.671413+0000","last_clean":"2024-07-17T07:12:39.671413+0000","last_became_active":"2024-07-17T07:12:37.665247+0000","last_became_peered":"2024-07-17T07:12:37.665247+0000","last_unstale":"2024-07-17T07:12:39.671413+0000","last_undegraded":"2024-07-17T07:12:39.671413+0000","last_fullsized":"2024-07-17T07:12:39.671413+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:16.324968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671493+0000","last_change":"2024-07-17T07:12:38.712217+0000","last_active":"2024-07-17T07:12:39.671493+0000","last_peered":"2024-07-17T07:12:39.671493+0000","last_clean":"2024-07-17T07:12:39.671493+0000","last_became_active":"2024-07-17T07:12:38.711920+0000","last_became_peered":"2024-07-17T07:12:38.711920+0000","last_unstale":"2024-07-17T07:12:39.671493+0000","last_undegraded":"2024-07-17T07:12:39.671493+0000","last_fullsized":"2024-07-17T07:12:39.671493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:33.560453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716436+0000","last_change":"2024-07-17T07:12:37.669728+0000","last_active":"2024-07-17T07:12:39.716436+0000","last_peered":"2024-07-17T07:12:39.716436+0000","last_clean":"2024-07-17T07:12:39.716436+0000","last_became_active":"2024-07-17T07:12:37.668971+0000","last_became_peered":"2024-07-17T07:12:37.668971+0000","last_unstale":"2024-07-17T07:12:39.716436+0000","last_undegraded":"2024-07-17T07:12:39.716436+0000","last_fullsized":"2024-07-17T07:12:39.716436+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:08:16.833343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743396+0000","last_change":"2024-07-17T07:12:38.717267+0000","last_active":"2024-07-17T07:12:39.743396+0000","last_peered":"2024-07-17T07:12:39.743396+0000","last_clean":"2024-07-17T07:12:39.743396+0000","last_became_active":"2024-07-17T07:12:38.716937+0000","last_became_peered":"2024-07-17T07:12:38.716937+0000","last_unstale":"2024-07-17T07:12:39.743396+0000","last_undegraded":"2024-07-17T07:12:39.743396+0000","last_fullsized":"2024-07-17T07:12:39.743396+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:44:41.254304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743022+0000","last_change":"2024-07-17T07:12:37.656357+0000","last_active":"2024-07-17T07:12:39.743022+0000","last_peered":"2024-07-17T07:12:39.743022+0000","last_clean":"2024-07-17T07:12:39.743022+0000","last_became_active":"2024-07-17T07:12:37.656055+0000","last_became_peered":"2024-07-17T07:12:37.656055+0000","last_unstale":"2024-07-17T07:12:39.743022+0000","last_undegraded":"2024-07-17T07:12:39.743022+0000","last_fullsized":"2024-07-17T07:12:39.743022+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:56.455202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741906+0000","last_change":"2024-07-17T07:12:38.726308+0000","last_active":"2024-07-17T07:12:39.741906+0000","last_peered":"2024-07-17T07:12:39.741906+0000","last_clean":"2024-07-17T07:12:39.741906+0000","last_became_active":"2024-07-17T07:12:38.723197+0000","last_became_peered":"2024-07-17T07:12:38.723197+0000","last_unstale":"2024-07-17T07:12:39.741906+0000","last_undegraded":"2024-07-17T07:12:39.741906+0000","last_fullsized":"2024-07-17T07:12:39.741906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:59.106505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742077+0000","last_change":"2024-07-17T07:12:37.670827+0000","last_active":"2024-07-17T07:12:39.742077+0000","last_peered":"2024-07-17T07:12:39.742077+0000","last_clean":"2024-07-17T07:12:39.742077+0000","last_became_active":"2024-07-17T07:12:37.670521+0000","last_became_peered":"2024-07-17T07:12:37.670521+0000","last_unstale":"2024-07-17T07:12:39.742077+0000","last_undegraded":"2024-07-17T07:12:39.742077+0000","last_fullsized":"2024-07-17T07:12:39.742077+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:47.113275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714820+0000","last_change":"2024-07-17T07:12:38.700266+0000","last_active":"2024-07-17T07:12:39.714820+0000","last_peered":"2024-07-17T07:12:39.714820+0000","last_clean":"2024-07-17T07:12:39.714820+0000","last_became_active":"2024-07-17T07:12:38.699932+0000","last_became_peered":"2024-07-17T07:12:38.699932+0000","last_unstale":"2024-07-17T07:12:39.714820+0000","last_undegraded":"2024-07-17T07:12:39.714820+0000","last_fullsized":"2024-07-17T07:12:39.714820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:58.888424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742003+0000","last_change":"2024-07-17T07:12:37.667846+0000","last_active":"2024-07-17T07:12:39.742003+0000","last_peered":"2024-07-17T07:12:39.742003+0000","last_clean":"2024-07-17T07:12:39.742003+0000","last_became_active":"2024-07-17T07:12:37.666937+0000","last_became_peered":"2024-07-17T07:12:37.666937+0000","last_unstale":"2024-07-17T07:12:39.742003+0000","last_undegraded":"2024-07-17T07:12:39.742003+0000","last_fullsized":"2024-07-17T07:12:39.742003+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:49.991886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740103+0000","last_change":"2024-07-17T07:12:38.720716+0000","last_active":"2024-07-17T07:12:39.740103+0000","last_peered":"2024-07-17T07:12:39.740103+0000","last_clean":"2024-07-17T07:12:39.740103+0000","last_became_active":"2024-07-17T07:12:38.720409+0000","last_became_peered":"2024-07-17T07:12:38.720409+0000","last_unstale":"2024-07-17T07:12:39.740103+0000","last_undegraded":"2024-07-17T07:12:39.740103+0000","last_fullsized":"2024-07-17T07:12:39.740103+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:38.353810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741830+0000","last_change":"2024-07-17T07:12:37.659483+0000","last_active":"2024-07-17T07:12:39.741830+0000","last_peered":"2024-07-17T07:12:39.741830+0000","last_clean":"2024-07-17T07:12:39.741830+0000","last_became_active":"2024-07-17T07:12:37.659153+0000","last_became_peered":"2024-07-17T07:12:37.659153+0000","last_unstale":"2024-07-17T07:12:39.741830+0000","last_undegraded":"2024-07-17T07:12:39.741830+0000","last_fullsized":"2024-07-17T07:12:39.741830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:26:32.582900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740402+0000","last_change":"2024-07-17T07:12:38.702732+0000","last_active":"2024-07-17T07:12:39.740402+0000","last_peered":"2024-07-17T07:12:39.740402+0000","last_clean":"2024-07-17T07:12:39.740402+0000","last_became_active":"2024-07-17T07:12:38.702357+0000","last_became_peered":"2024-07-17T07:12:38.702357+0000","last_unstale":"2024-07-17T07:12:39.740402+0000","last_undegraded":"2024-07-17T07:12:39.740402+0000","last_fullsized":"2024-07-17T07:12:39.740402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:17.232545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742041+0000","last_change":"2024-07-17T07:12:37.662612+0000","last_active":"2024-07-17T07:12:39.742041+0000","last_peered":"2024-07-17T07:12:39.742041+0000","last_clean":"2024-07-17T07:12:39.742041+0000","last_became_active":"2024-07-17T07:12:37.661583+0000","last_became_peered":"2024-07-17T07:12:37.661583+0000","last_unstale":"2024-07-17T07:12:39.742041+0000","last_undegraded":"2024-07-17T07:12:39.742041+0000","last_fullsized":"2024-07-17T07:12:39.742041+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:38:37.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742884+0000","last_change":"2024-07-17T07:12:38.712965+0000","last_active":"2024-07-17T07:12:39.742884+0000","last_peered":"2024-07-17T07:12:39.742884+0000","last_clean":"2024-07-17T07:12:39.742884+0000","last_became_active":"2024-07-17T07:12:38.711424+0000","last_became_peered":"2024-07-17T07:12:38.711424+0000","last_unstale":"2024-07-17T07:12:39.742884+0000","last_undegraded":"2024-07-17T07:12:39.742884+0000","last_fullsized":"2024-07-17T07:12:39.742884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:14:41.118839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677953+0000","last_change":"2024-07-17T07:12:37.660151+0000","last_active":"2024-07-17T07:12:39.677953+0000","last_peered":"2024-07-17T07:12:39.677953+0000","last_clean":"2024-07-17T07:12:39.677953+0000","last_became_active":"2024-07-17T07:12:37.659851+0000","last_became_peered":"2024-07-17T07:12:37.659851+0000","last_unstale":"2024-07-17T07:12:39.677953+0000","last_undegraded":"2024-07-17T07:12:39.677953+0000","last_fullsized":"2024-07-17T07:12:39.677953+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:03.333506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676944+0000","last_change":"2024-07-17T07:12:38.701062+0000","last_active":"2024-07-17T07:12:39.676944+0000","last_peered":"2024-07-17T07:12:39.676944+0000","last_clean":"2024-07-17T07:12:39.676944+0000","last_became_active":"2024-07-17T07:12:38.700772+0000","last_became_peered":"2024-07-17T07:12:38.700772+0000","last_unstale":"2024-07-17T07:12:39.676944+0000","last_undegraded":"2024-07-17T07:12:39.676944+0000","last_fullsized":"2024-07-17T07:12:39.676944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:01.704078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741928+0000","last_change":"2024-07-17T07:12:37.667835+0000","last_active":"2024-07-17T07:12:39.741928+0000","last_peered":"2024-07-17T07:12:39.741928+0000","last_clean":"2024-07-17T07:12:39.741928+0000","last_became_active":"2024-07-17T07:12:37.667542+0000","last_became_peered":"2024-07-17T07:12:37.667542+0000","last_unstale":"2024-07-17T07:12:39.741928+0000","last_undegraded":"2024-07-17T07:12:39.741928+0000","last_fullsized":"2024-07-17T07:12:39.741928+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:35:59.328496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716330+0000","last_change":"2024-07-17T07:12:38.708283+0000","last_active":"2024-07-17T07:12:39.716330+0000","last_peered":"2024-07-17T07:12:39.716330+0000","last_clean":"2024-07-17T07:12:39.716330+0000","last_became_active":"2024-07-17T07:12:38.699913+0000","last_became_peered":"2024-07-17T07:12:38.699913+0000","last_unstale":"2024-07-17T07:12:39.716330+0000","last_undegraded":"2024-07-17T07:12:39.716330+0000","last_fullsized":"2024-07-17T07:12:39.716330+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.423010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716470+0000","last_change":"2024-07-17T07:12:37.673523+0000","last_active":"2024-07-17T07:12:39.716470+0000","last_peered":"2024-07-17T07:12:39.716470+0000","last_clean":"2024-07-17T07:12:39.716470+0000","last_became_active":"2024-07-17T07:12:37.669347+0000","last_became_peered":"2024-07-17T07:12:37.669347+0000","last_unstale":"2024-07-17T07:12:39.716470+0000","last_undegraded":"2024-07-17T07:12:39.716470+0000","last_fullsized":"2024-07-17T07:12:39.716470+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:28.526105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740782+0000","last_change":"2024-07-17T07:12:38.722400+0000","last_active":"2024-07-17T07:12:39.740782+0000","last_peered":"2024-07-17T07:12:39.740782+0000","last_clean":"2024-07-17T07:12:39.740782+0000","last_became_active":"2024-07-17T07:12:38.722095+0000","last_became_peered":"2024-07-17T07:12:38.722095+0000","last_unstale":"2024-07-17T07:12:39.740782+0000","last_undegraded":"2024-07-17T07:12:39.740782+0000","last_fullsized":"2024-07-17T07:12:39.740782+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:46.608059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676115+0000","last_change":"2024-07-17T07:12:37.669987+0000","last_active":"2024-07-17T07:12:39.676115+0000","last_peered":"2024-07-17T07:12:39.676115+0000","last_clean":"2024-07-17T07:12:39.676115+0000","last_became_active":"2024-07-17T07:12:37.668534+0000","last_became_peered":"2024-07-17T07:12:37.668534+0000","last_unstale":"2024-07-17T07:12:39.676115+0000","last_undegraded":"2024-07-17T07:12:39.676115+0000","last_fullsized":"2024-07-17T07:12:39.676115+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:21:23.486022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.674189+0000","last_change":"2024-07-17T07:12:38.703281+0000","last_active":"2024-07-17T07:12:39.674189+0000","last_peered":"2024-07-17T07:12:39.674189+0000","last_clean":"2024-07-17T07:12:39.674189+0000","last_became_active":"2024-07-17T07:12:38.702955+0000","last_became_peered":"2024-07-17T07:12:38.702955+0000","last_unstale":"2024-07-17T07:12:39.674189+0000","last_undegraded":"2024-07-17T07:12:39.674189+0000","last_fullsized":"2024-07-17T07:12:39.674189+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:17.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740203+0000","last_change":"2024-07-17T07:12:37.659882+0000","last_active":"2024-07-17T07:12:39.740203+0000","last_peered":"2024-07-17T07:12:39.740203+0000","last_clean":"2024-07-17T07:12:39.740203+0000","last_became_active":"2024-07-17T07:12:37.659585+0000","last_became_peered":"2024-07-17T07:12:37.659585+0000","last_unstale":"2024-07-17T07:12:39.740203+0000","last_undegraded":"2024-07-17T07:12:39.740203+0000","last_fullsized":"2024-07-17T07:12:39.740203+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:56.226193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675041+0000","last_change":"2024-07-17T07:12:38.728528+0000","last_active":"2024-07-17T07:12:39.675041+0000","last_peered":"2024-07-17T07:12:39.675041+0000","last_clean":"2024-07-17T07:12:39.675041+0000","last_became_active":"2024-07-17T07:12:38.728175+0000","last_became_peered":"2024-07-17T07:12:38.728175+0000","last_unstale":"2024-07-17T07:12:39.675041+0000","last_undegraded":"2024-07-17T07:12:39.675041+0000","last_fullsized":"2024-07-17T07:12:39.675041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:53.653684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.674425+0000","last_change":"2024-07-17T07:12:37.664839+0000","last_active":"2024-07-17T07:12:39.674425+0000","last_peered":"2024-07-17T07:12:39.674425+0000","last_clean":"2024-07-17T07:12:39.674425+0000","last_became_active":"2024-07-17T07:12:37.664369+0000","last_became_peered":"2024-07-17T07:12:37.664369+0000","last_unstale":"2024-07-17T07:12:39.674425+0000","last_undegraded":"2024-07-17T07:12:39.674425+0000","last_fullsized":"2024-07-17T07:12:39.674425+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:16:55.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742266+0000","last_change":"2024-07-17T07:12:38.695034+0000","last_active":"2024-07-17T07:12:39.742266+0000","last_peered":"2024-07-17T07:12:39.742266+0000","last_clean":"2024-07-17T07:12:39.742266+0000","last_became_active":"2024-07-17T07:12:38.694696+0000","last_became_peered":"2024-07-17T07:12:38.694696+0000","last_unstale":"2024-07-17T07:12:39.742266+0000","last_undegraded":"2024-07-17T07:12:39.742266+0000","last_fullsized":"2024-07-17T07:12:39.742266+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:39:41.277426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.739950+0000","last_change":"2024-07-17T07:12:37.668013+0000","last_active":"2024-07-17T07:12:39.739950+0000","last_peered":"2024-07-17T07:12:39.739950+0000","last_clean":"2024-07-17T07:12:39.739950+0000","last_became_active":"2024-07-17T07:12:37.667594+0000","last_became_peered":"2024-07-17T07:12:37.667594+0000","last_unstale":"2024-07-17T07:12:39.739950+0000","last_undegraded":"2024-07-17T07:12:39.739950+0000","last_fullsized":"2024-07-17T07:12:39.739950+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:21:39.154468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677048+0000","last_change":"2024-07-17T07:12:38.695739+0000","last_active":"2024-07-17T07:12:39.677048+0000","last_peered":"2024-07-17T07:12:39.677048+0000","last_clean":"2024-07-17T07:12:39.677048+0000","last_became_active":"2024-07-17T07:12:38.695299+0000","last_became_peered":"2024-07-17T07:12:38.695299+0000","last_unstale":"2024-07-17T07:12:39.677048+0000","last_undegraded":"2024-07-17T07:12:39.677048+0000","last_fullsized":"2024-07-17T07:12:39.677048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:48:35.710592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742289+0000","last_change":"2024-07-17T07:12:37.666148+0000","last_active":"2024-07-17T07:12:39.742289+0000","last_peered":"2024-07-17T07:12:39.742289+0000","last_clean":"2024-07-17T07:12:39.742289+0000","last_became_active":"2024-07-17T07:12:37.665776+0000","last_became_peered":"2024-07-17T07:12:37.665776+0000","last_unstale":"2024-07-17T07:12:39.742289+0000","last_undegraded":"2024-07-17T07:12:39.742289+0000","last_fullsized":"2024-07-17T07:12:39.742289+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:07.951745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743131+0000","last_change":"2024-07-17T07:12:38.702668+0000","last_active":"2024-07-17T07:12:39.743131+0000","last_peered":"2024-07-17T07:12:39.743131+0000","last_clean":"2024-07-17T07:12:39.743131+0000","last_became_active":"2024-07-17T07:12:38.702361+0000","last_became_peered":"2024-07-17T07:12:38.702361+0000","last_unstale":"2024-07-17T07:12:39.743131+0000","last_undegraded":"2024-07-17T07:12:39.743131+0000","last_fullsized":"2024-07-17T07:12:39.743131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:46.118720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"112'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.720380+0000","last_change":"2024-07-17T07:12:37.673507+0000","last_active":"2024-07-17T07:12:39.720380+0000","last_peered":"2024-07-17T07:12:39.720380+0000","last_clean":"2024-07-17T07:12:39.720380+0000","last_became_active":"2024-07-17T07:12:37.672594+0000","last_became_peered":"2024-07-17T07:12:37.672594+0000","last_unstale":"2024-07-17T07:12:39.720380+0000","last_undegraded":"2024-07-17T07:12:39.720380+0000","last_fullsized":"2024-07-17T07:12:39.720380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:27.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743385+0000","last_change":"2024-07-17T07:12:38.689323+0000","last_active":"2024-07-17T07:12:39.743385+0000","last_peered":"2024-07-17T07:12:39.743385+0000","last_clean":"2024-07-17T07:12:39.743385+0000","last_became_active":"2024-07-17T07:12:38.689021+0000","last_became_peered":"2024-07-17T07:12:38.689021+0000","last_unstale":"2024-07-17T07:12:39.743385+0000","last_undegraded":"2024-07-17T07:12:39.743385+0000","last_fullsized":"2024-07-17T07:12:39.743385+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:35:08.183779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716750+0000","last_change":"2024-07-17T07:12:37.667832+0000","last_active":"2024-07-17T07:12:39.716750+0000","last_peered":"2024-07-17T07:12:39.716750+0000","last_clean":"2024-07-17T07:12:39.716750+0000","last_became_active":"2024-07-17T07:12:37.666502+0000","last_became_peered":"2024-07-17T07:12:37.666502+0000","last_unstale":"2024-07-17T07:12:39.716750+0000","last_undegraded":"2024-07-17T07:12:39.716750+0000","last_fullsized":"2024-07-17T07:12:39.716750+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:08:58.697317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678546+0000","last_change":"2024-07-17T07:12:38.713683+0000","last_active":"2024-07-17T07:12:39.678546+0000","last_peered":"2024-07-17T07:12:39.678546+0000","last_clean":"2024-07-17T07:12:39.678546+0000","last_became_active":"2024-07-17T07:12:38.713380+0000","last_became_peered":"2024-07-17T07:12:38.713380+0000","last_unstale":"2024-07-17T07:12:39.678546+0000","last_undegraded":"2024-07-17T07:12:39.678546+0000","last_fullsized":"2024-07-17T07:12:39.678546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:24.489401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677205+0000","last_change":"2024-07-17T07:12:37.648851+0000","last_active":"2024-07-17T07:12:39.677205+0000","last_peered":"2024-07-17T07:12:39.677205+0000","last_clean":"2024-07-17T07:12:39.677205+0000","last_became_active":"2024-07-17T07:12:37.646362+0000","last_became_peered":"2024-07-17T07:12:37.646362+0000","last_unstale":"2024-07-17T07:12:39.677205+0000","last_undegraded":"2024-07-17T07:12:39.677205+0000","last_fullsized":"2024-07-17T07:12:39.677205+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:48.018507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.739847+0000","last_change":"2024-07-17T07:12:38.686630+0000","last_active":"2024-07-17T07:12:39.739847+0000","last_peered":"2024-07-17T07:12:39.739847+0000","last_clean":"2024-07-17T07:12:39.739847+0000","last_became_active":"2024-07-17T07:12:38.686172+0000","last_became_peered":"2024-07-17T07:12:38.686172+0000","last_unstale":"2024-07-17T07:12:39.739847+0000","last_undegraded":"2024-07-17T07:12:39.739847+0000","last_fullsized":"2024-07-17T07:12:39.739847+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:59:16.082697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742435+0000","last_change":"2024-07-17T07:12:37.668131+0000","last_active":"2024-07-17T07:12:39.742435+0000","last_peered":"2024-07-17T07:12:39.742435+0000","last_clean":"2024-07-17T07:12:39.742435+0000","last_became_active":"2024-07-17T07:12:37.666751+0000","last_became_peered":"2024-07-17T07:12:37.666751+0000","last_unstale":"2024-07-17T07:12:39.742435+0000","last_undegraded":"2024-07-17T07:12:39.742435+0000","last_fullsized":"2024-07-17T07:12:39.742435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:59:00.056759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.670991+0000","last_change":"2024-07-17T07:12:38.720291+0000","last_active":"2024-07-17T07:12:39.670991+0000","last_peered":"2024-07-17T07:12:39.670991+0000","last_clean":"2024-07-17T07:12:39.670991+0000","last_became_active":"2024-07-17T07:12:38.719595+0000","last_became_peered":"2024-07-17T07:12:38.719595+0000","last_unstale":"2024-07-17T07:12:39.670991+0000","last_undegraded":"2024-07-17T07:12:39.670991+0000","last_fullsized":"2024-07-17T07:12:39.670991+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:33.942741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676271+0000","last_change":"2024-07-17T07:12:37.669522+0000","last_active":"2024-07-17T07:12:39.676271+0000","last_peered":"2024-07-17T07:12:39.676271+0000","last_clean":"2024-07-17T07:12:39.676271+0000","last_became_active":"2024-07-17T07:12:37.669285+0000","last_became_peered":"2024-07-17T07:12:37.669285+0000","last_unstale":"2024-07-17T07:12:39.676271+0000","last_undegraded":"2024-07-17T07:12:39.676271+0000","last_fullsized":"2024-07-17T07:12:39.676271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:47:28.382928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677104+0000","last_change":"2024-07-17T07:12:38.721026+0000","last_active":"2024-07-17T07:12:39.677104+0000","last_peered":"2024-07-17T07:12:39.677104+0000","last_clean":"2024-07-17T07:12:39.677104+0000","last_became_active":"2024-07-17T07:12:38.720694+0000","last_became_peered":"2024-07-17T07:12:38.720694+0000","last_unstale":"2024-07-17T07:12:39.677104+0000","last_undegraded":"2024-07-17T07:12:39.677104+0000","last_fullsized":"2024-07-17T07:12:39.677104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:39:31.369488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675862+0000","last_change":"2024-07-17T07:12:37.654867+0000","last_active":"2024-07-17T07:12:39.675862+0000","last_peered":"2024-07-17T07:12:39.675862+0000","last_clean":"2024-07-17T07:12:39.675862+0000","last_became_active":"2024-07-17T07:12:37.654399+0000","last_became_peered":"2024-07-17T07:12:37.654399+0000","last_unstale":"2024-07-17T07:12:39.675862+0000","last_undegraded":"2024-07-17T07:12:39.675862+0000","last_fullsized":"2024-07-17T07:12:39.675862+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:46.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742501+0000","last_change":"2024-07-17T07:12:38.700261+0000","last_active":"2024-07-17T07:12:39.742501+0000","last_peered":"2024-07-17T07:12:39.742501+0000","last_clean":"2024-07-17T07:12:39.742501+0000","last_became_active":"2024-07-17T07:12:38.699974+0000","last_became_peered":"2024-07-17T07:12:38.699974+0000","last_unstale":"2024-07-17T07:12:39.742501+0000","last_undegraded":"2024-07-17T07:12:39.742501+0000","last_fullsized":"2024-07-17T07:12:39.742501+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:37:08.662336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741541+0000","last_change":"2024-07-17T07:12:37.667021+0000","last_active":"2024-07-17T07:12:39.741541+0000","last_peered":"2024-07-17T07:12:39.741541+0000","last_clean":"2024-07-17T07:12:39.741541+0000","last_became_active":"2024-07-17T07:12:37.665511+0000","last_became_peered":"2024-07-17T07:12:37.665511+0000","last_unstale":"2024-07-17T07:12:39.741541+0000","last_undegraded":"2024-07-17T07:12:39.741541+0000","last_fullsized":"2024-07-17T07:12:39.741541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:19.060716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742119+0000","last_change":"2024-07-17T07:12:38.707274+0000","last_active":"2024-07-17T07:12:39.742119+0000","last_peered":"2024-07-17T07:12:39.742119+0000","last_clean":"2024-07-17T07:12:39.742119+0000","last_became_active":"2024-07-17T07:12:38.706523+0000","last_became_peered":"2024-07-17T07:12:38.706523+0000","last_unstale":"2024-07-17T07:12:39.742119+0000","last_undegraded":"2024-07-17T07:12:39.742119+0000","last_fullsized":"2024-07-17T07:12:39.742119+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:52:42.145408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741985+0000","last_change":"2024-07-17T07:12:37.662522+0000","last_active":"2024-07-17T07:12:39.741985+0000","last_peered":"2024-07-17T07:12:39.741985+0000","last_clean":"2024-07-17T07:12:39.741985+0000","last_became_active":"2024-07-17T07:12:37.662041+0000","last_became_peered":"2024-07-17T07:12:37.662041+0000","last_unstale":"2024-07-17T07:12:39.741985+0000","last_undegraded":"2024-07-17T07:12:39.741985+0000","last_fullsized":"2024-07-17T07:12:39.741985+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:15:23.241910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"112'1592","reported_seq":1939,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:40.401604+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:12:40.401604+0000","last_peered":"2024-07-17T07:12:40.401604+0000","last_clean":"2024-07-17T07:12:40.401604+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:12:40.401604+0000","last_undegraded":"2024-07-17T07:12:40.401604+0000","last_fullsized":"2024-07-17T07:12:40.401604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":1592,"log_dups_size":0,"ondisk_log_size":1592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":179,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":179,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1592,"ondisk_log_size":1592,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542358,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46988,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324852,"statfs":{"total":96636764160,"available":96588648448,"internally_reserved":0,"allocated":1961984,"data_stored":1301939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1392640,"data_stored":720943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509654,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50572,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321268,"statfs":{"total":96636764160,"available":96584978432,"internally_reserved":0,"allocated":1372160,"data_stored":718727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542359,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":711675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1359872,"data_stored":711515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58888,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312952,"statfs":{"total":96636764160,"available":96576462848,"internally_reserved":0,"allocated":1372160,"data_stored":718727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509654,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":718797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509653,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":1990656,"data_stored":1311413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:12:43.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:12:43.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 -- 172.21.15.44:0/799633197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200ac220 msgr2=0x7f75200ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 --2- 172.21.15.44:0/799633197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200ac220 0x7f75200ac690 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f752015edd0 tx=0x7f750802a560 comp rx=0 tx=0).stop 2024-07-17T07:12:43.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 -- 172.21.15.44:0/799633197 shutdown_connections 2024-07-17T07:12:43.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 --2- 172.21.15.44:0/799633197 >> v2:172.21.15.175:3300/0 conn(0x7f75200cd690 0x7f75200cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 --2- 172.21.15.44:0/799633197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200ac220 0x7f75200ac690 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 -- 172.21.15.44:0/799633197 >> 172.21.15.44:0/799633197 conn(0x7f75200b0fb0 msgr2=0x7f75200b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:43.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 -- 172.21.15.44:0/799633197 shutdown_connections 2024-07-17T07:12:43.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.471+0000 7f7525200640 1 -- 172.21.15.44:0/799633197 wait complete. 2024-07-17T07:12:43.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.473+0000 7f7525200640 1 Processor -- start 2024-07-17T07:12:43.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.473+0000 7f7525200640 1 -- start start 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75200ac220 0x7f75200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 0x7f7520154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75200ce020 con 0x7f75200cd690 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75200adf80 con 0x7f75200ac220 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f7525200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75200ce1a0 con 0x7f75201522c0 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.474+0000 7f751d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38856/0 (socket says 172.21.15.44:38856) 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 -- 172.21.15.44:0/4130464783 learned_addr learned my addr 172.21.15.44:0/4130464783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:43.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751de00640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75200ac220 0x7f75200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751e800640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 0x7f7520154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 -- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 msgr2=0x7f7520154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 0x7f7520154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 -- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75200ac220 msgr2=0x7f75200740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75200ac220 0x7f75200740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 -- 172.21.15.44:0/4130464783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7520152b80 con 0x7f75200cd690 2024-07-17T07:12:43.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751e800640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 0x7f7520154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.475+0000 7f751d400640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f75080099c0 tx=0x7f75080039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.476+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750803f070 con 0x7f75200cd690 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.476+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7508004890 con 0x7f75200cd690 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.476+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508035030 con 0x7f75200cd690 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.476+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7520152e60 con 0x7f75200cd690 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.476+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7520153420 con 0x7f75200cd690 2024-07-17T07:12:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.477+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75200ce6c0 con 0x7f75200cd690 2024-07-17T07:12:43.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.478+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f7508042070 con 0x7f75200cd690 2024-07-17T07:12:43.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.479+0000 7f7516000640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 0x7f74fc079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:43.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.479+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f75080b96c0 con 0x7f75200cd690 2024-07-17T07:12:43.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.480+0000 7f751de00640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 0x7f74fc079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:43.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.481+0000 7f751de00640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 0x7f74fc079100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7510006fd0 tx=0x7f7510008040 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:43.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.484+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7508082600 con 0x7f75200cd690 2024-07-17T07:12:43.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.702+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 --> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7520152930 con 0x7f74fc076c40 2024-07-17T07:12:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.716+0000 7f7516000640 1 -- 172.21.15.44:0/4130464783 <== mgr.5923 v2:172.21.15.175:6800/3550495011 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7f7520152930 con 0x7f74fc076c40 2024-07-17T07:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:43.721 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:12:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 msgr2=0x7f74fc079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 0x7f74fc079100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7510006fd0 tx=0x7f7510008040 comp rx=0 tx=0).stop 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 msgr2=0x7f752014a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f75080099c0 tx=0x7f75080039e0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 shutdown_connections 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f74fc076c40 0x7f74fc079100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75201522c0 0x7f7520154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75200cd690 0x7f752014a560 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 --2- 172.21.15.44:0/4130464783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75200ac220 0x7f75200740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 >> 172.21.15.44:0/4130464783 conn(0x7f75200b0fb0 msgr2=0x7f75200afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 shutdown_connections 2024-07-17T07:12:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:43.724+0000 7f7525200640 1 -- 172.21.15.44:0/4130464783 wait complete. 2024-07-17T07:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:12:42.404578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":224,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1631,"ondisk_log_size":1631,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":336316,"kb_used_data":11908,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754638404,"statfs":{"total":773094113280,"available":772749725696,"internally_reserved":0,"allocated":12193792,"data_stored":6913736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":332058160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.781324"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671483+0000","last_change":"2024-07-17T07:12:38.713310+0000","last_active":"2024-07-17T07:12:39.671483+0000","last_peered":"2024-07-17T07:12:39.671483+0000","last_clean":"2024-07-17T07:12:39.671483+0000","last_became_active":"2024-07-17T07:12:38.713036+0000","last_became_peered":"2024-07-17T07:12:38.713036+0000","last_unstale":"2024-07-17T07:12:39.671483+0000","last_undegraded":"2024-07-17T07:12:39.671483+0000","last_fullsized":"2024-07-17T07:12:39.671483+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:04.899929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678167+0000","last_change":"2024-07-17T07:12:38.707638+0000","last_active":"2024-07-17T07:12:39.678167+0000","last_peered":"2024-07-17T07:12:39.678167+0000","last_clean":"2024-07-17T07:12:39.678167+0000","last_became_active":"2024-07-17T07:12:38.706605+0000","last_became_peered":"2024-07-17T07:12:38.706605+0000","last_unstale":"2024-07-17T07:12:39.678167+0000","last_undegraded":"2024-07-17T07:12:39.678167+0000","last_fullsized":"2024-07-17T07:12:39.678167+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:47:37.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671080+0000","last_change":"2024-07-17T07:12:38.713390+0000","last_active":"2024-07-17T07:12:39.671080+0000","last_peered":"2024-07-17T07:12:39.671080+0000","last_clean":"2024-07-17T07:12:39.671080+0000","last_became_active":"2024-07-17T07:12:38.709084+0000","last_became_peered":"2024-07-17T07:12:38.709084+0000","last_unstale":"2024-07-17T07:12:39.671080+0000","last_undegraded":"2024-07-17T07:12:39.671080+0000","last_fullsized":"2024-07-17T07:12:39.671080+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:44.007112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741306+0000","last_change":"2024-07-17T07:12:38.711376+0000","last_active":"2024-07-17T07:12:39.741306+0000","last_peered":"2024-07-17T07:12:39.741306+0000","last_clean":"2024-07-17T07:12:39.741306+0000","last_became_active":"2024-07-17T07:12:38.710869+0000","last_became_peered":"2024-07-17T07:12:38.710869+0000","last_unstale":"2024-07-17T07:12:39.741306+0000","last_undegraded":"2024-07-17T07:12:39.741306+0000","last_fullsized":"2024-07-17T07:12:39.741306+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:11.228490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715106+0000","last_change":"2024-07-17T07:12:38.701213+0000","last_active":"2024-07-17T07:12:39.715106+0000","last_peered":"2024-07-17T07:12:39.715106+0000","last_clean":"2024-07-17T07:12:39.715106+0000","last_became_active":"2024-07-17T07:12:38.700909+0000","last_became_peered":"2024-07-17T07:12:38.700909+0000","last_unstale":"2024-07-17T07:12:39.715106+0000","last_undegraded":"2024-07-17T07:12:39.715106+0000","last_fullsized":"2024-07-17T07:12:39.715106+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:12:09.728310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741277+0000","last_change":"2024-07-17T07:12:38.722593+0000","last_active":"2024-07-17T07:12:39.741277+0000","last_peered":"2024-07-17T07:12:39.741277+0000","last_clean":"2024-07-17T07:12:39.741277+0000","last_became_active":"2024-07-17T07:12:38.722271+0000","last_became_peered":"2024-07-17T07:12:38.722271+0000","last_unstale":"2024-07-17T07:12:39.741277+0000","last_undegraded":"2024-07-17T07:12:39.741277+0000","last_fullsized":"2024-07-17T07:12:39.741277+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:43.218920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741845+0000","last_change":"2024-07-17T07:12:38.703404+0000","last_active":"2024-07-17T07:12:39.741845+0000","last_peered":"2024-07-17T07:12:39.741845+0000","last_clean":"2024-07-17T07:12:39.741845+0000","last_became_active":"2024-07-17T07:12:38.703070+0000","last_became_peered":"2024-07-17T07:12:38.703070+0000","last_unstale":"2024-07-17T07:12:39.741845+0000","last_undegraded":"2024-07-17T07:12:39.741845+0000","last_fullsized":"2024-07-17T07:12:39.741845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:54:18.887110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741486+0000","last_change":"2024-07-17T07:12:38.726893+0000","last_active":"2024-07-17T07:12:39.741486+0000","last_peered":"2024-07-17T07:12:39.741486+0000","last_clean":"2024-07-17T07:12:39.741486+0000","last_became_active":"2024-07-17T07:12:38.726357+0000","last_became_peered":"2024-07-17T07:12:38.726357+0000","last_unstale":"2024-07-17T07:12:39.741486+0000","last_undegraded":"2024-07-17T07:12:39.741486+0000","last_fullsized":"2024-07-17T07:12:39.741486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:47:51.342467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716581+0000","last_change":"2024-07-17T07:12:38.704233+0000","last_active":"2024-07-17T07:12:39.716581+0000","last_peered":"2024-07-17T07:12:39.716581+0000","last_clean":"2024-07-17T07:12:39.716581+0000","last_became_active":"2024-07-17T07:12:38.703831+0000","last_became_peered":"2024-07-17T07:12:38.703831+0000","last_unstale":"2024-07-17T07:12:39.716581+0000","last_undegraded":"2024-07-17T07:12:39.716581+0000","last_fullsized":"2024-07-17T07:12:39.716581+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:56.733128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740873+0000","last_change":"2024-07-17T07:12:38.716131+0000","last_active":"2024-07-17T07:12:39.740873+0000","last_peered":"2024-07-17T07:12:39.740873+0000","last_clean":"2024-07-17T07:12:39.740873+0000","last_became_active":"2024-07-17T07:12:38.710773+0000","last_became_peered":"2024-07-17T07:12:38.710773+0000","last_unstale":"2024-07-17T07:12:39.740873+0000","last_undegraded":"2024-07-17T07:12:39.740873+0000","last_fullsized":"2024-07-17T07:12:39.740873+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:36.113058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716125+0000","last_change":"2024-07-17T07:12:38.700853+0000","last_active":"2024-07-17T07:12:39.716125+0000","last_peered":"2024-07-17T07:12:39.716125+0000","last_clean":"2024-07-17T07:12:39.716125+0000","last_became_active":"2024-07-17T07:12:38.700511+0000","last_became_peered":"2024-07-17T07:12:38.700511+0000","last_unstale":"2024-07-17T07:12:39.716125+0000","last_undegraded":"2024-07-17T07:12:39.716125+0000","last_fullsized":"2024-07-17T07:12:39.716125+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:13.100796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740428+0000","last_change":"2024-07-17T07:12:38.704874+0000","last_active":"2024-07-17T07:12:39.740428+0000","last_peered":"2024-07-17T07:12:39.740428+0000","last_clean":"2024-07-17T07:12:39.740428+0000","last_became_active":"2024-07-17T07:12:38.704539+0000","last_became_peered":"2024-07-17T07:12:38.704539+0000","last_unstale":"2024-07-17T07:12:39.740428+0000","last_undegraded":"2024-07-17T07:12:39.740428+0000","last_fullsized":"2024-07-17T07:12:39.740428+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:26:01.322472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742646+0000","last_change":"2024-07-17T07:12:38.708402+0000","last_active":"2024-07-17T07:12:39.742646+0000","last_peered":"2024-07-17T07:12:39.742646+0000","last_clean":"2024-07-17T07:12:39.742646+0000","last_became_active":"2024-07-17T07:12:38.708092+0000","last_became_peered":"2024-07-17T07:12:38.708092+0000","last_unstale":"2024-07-17T07:12:39.742646+0000","last_undegraded":"2024-07-17T07:12:39.742646+0000","last_fullsized":"2024-07-17T07:12:39.742646+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:49.036240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714761+0000","last_change":"2024-07-17T07:12:38.722603+0000","last_active":"2024-07-17T07:12:39.714761+0000","last_peered":"2024-07-17T07:12:39.714761+0000","last_clean":"2024-07-17T07:12:39.714761+0000","last_became_active":"2024-07-17T07:12:38.722091+0000","last_became_peered":"2024-07-17T07:12:38.722091+0000","last_unstale":"2024-07-17T07:12:39.714761+0000","last_undegraded":"2024-07-17T07:12:39.714761+0000","last_fullsized":"2024-07-17T07:12:39.714761+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:47:47.681480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741790+0000","last_change":"2024-07-17T07:12:38.715969+0000","last_active":"2024-07-17T07:12:39.741790+0000","last_peered":"2024-07-17T07:12:39.741790+0000","last_clean":"2024-07-17T07:12:39.741790+0000","last_became_active":"2024-07-17T07:12:38.715630+0000","last_became_peered":"2024-07-17T07:12:38.715630+0000","last_unstale":"2024-07-17T07:12:39.741790+0000","last_undegraded":"2024-07-17T07:12:39.741790+0000","last_fullsized":"2024-07-17T07:12:39.741790+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:29:45.431930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740577+0000","last_change":"2024-07-17T07:12:38.703871+0000","last_active":"2024-07-17T07:12:39.740577+0000","last_peered":"2024-07-17T07:12:39.740577+0000","last_clean":"2024-07-17T07:12:39.740577+0000","last_became_active":"2024-07-17T07:12:38.703270+0000","last_became_peered":"2024-07-17T07:12:38.703270+0000","last_unstale":"2024-07-17T07:12:39.740577+0000","last_undegraded":"2024-07-17T07:12:39.740577+0000","last_fullsized":"2024-07-17T07:12:39.740577+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:37.983657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743685+0000","last_change":"2024-07-17T07:12:38.708354+0000","last_active":"2024-07-17T07:12:39.743685+0000","last_peered":"2024-07-17T07:12:39.743685+0000","last_clean":"2024-07-17T07:12:39.743685+0000","last_became_active":"2024-07-17T07:12:38.703621+0000","last_became_peered":"2024-07-17T07:12:38.703621+0000","last_unstale":"2024-07-17T07:12:39.743685+0000","last_undegraded":"2024-07-17T07:12:39.743685+0000","last_fullsized":"2024-07-17T07:12:39.743685+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:24:09.571539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671335+0000","last_change":"2024-07-17T07:12:38.704075+0000","last_active":"2024-07-17T07:12:39.671335+0000","last_peered":"2024-07-17T07:12:39.671335+0000","last_clean":"2024-07-17T07:12:39.671335+0000","last_became_active":"2024-07-17T07:12:38.702156+0000","last_became_peered":"2024-07-17T07:12:38.702156+0000","last_unstale":"2024-07-17T07:12:39.671335+0000","last_undegraded":"2024-07-17T07:12:39.671335+0000","last_fullsized":"2024-07-17T07:12:39.671335+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:18.247691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716005+0000","last_change":"2024-07-17T07:12:38.712498+0000","last_active":"2024-07-17T07:12:39.716005+0000","last_peered":"2024-07-17T07:12:39.716005+0000","last_clean":"2024-07-17T07:12:39.716005+0000","last_became_active":"2024-07-17T07:12:38.711367+0000","last_became_peered":"2024-07-17T07:12:38.711367+0000","last_unstale":"2024-07-17T07:12:39.716005+0000","last_undegraded":"2024-07-17T07:12:39.716005+0000","last_fullsized":"2024-07-17T07:12:39.716005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:40.190448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741514+0000","last_change":"2024-07-17T07:12:38.709329+0000","last_active":"2024-07-17T07:12:39.741514+0000","last_peered":"2024-07-17T07:12:39.741514+0000","last_clean":"2024-07-17T07:12:39.741514+0000","last_became_active":"2024-07-17T07:12:38.708992+0000","last_became_peered":"2024-07-17T07:12:38.708992+0000","last_unstale":"2024-07-17T07:12:39.741514+0000","last_undegraded":"2024-07-17T07:12:39.741514+0000","last_fullsized":"2024-07-17T07:12:39.741514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:38.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714909+0000","last_change":"2024-07-17T07:12:38.710151+0000","last_active":"2024-07-17T07:12:39.714909+0000","last_peered":"2024-07-17T07:12:39.714909+0000","last_clean":"2024-07-17T07:12:39.714909+0000","last_became_active":"2024-07-17T07:12:38.709291+0000","last_became_peered":"2024-07-17T07:12:38.709291+0000","last_unstale":"2024-07-17T07:12:39.714909+0000","last_undegraded":"2024-07-17T07:12:39.714909+0000","last_fullsized":"2024-07-17T07:12:39.714909+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.463184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714892+0000","last_change":"2024-07-17T07:12:38.715606+0000","last_active":"2024-07-17T07:12:39.714892+0000","last_peered":"2024-07-17T07:12:39.714892+0000","last_clean":"2024-07-17T07:12:39.714892+0000","last_became_active":"2024-07-17T07:12:38.714895+0000","last_became_peered":"2024-07-17T07:12:38.714895+0000","last_unstale":"2024-07-17T07:12:39.714892+0000","last_undegraded":"2024-07-17T07:12:39.714892+0000","last_fullsized":"2024-07-17T07:12:39.714892+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:31.466863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677307+0000","last_change":"2024-07-17T07:12:38.715282+0000","last_active":"2024-07-17T07:12:39.677307+0000","last_peered":"2024-07-17T07:12:39.677307+0000","last_clean":"2024-07-17T07:12:39.677307+0000","last_became_active":"2024-07-17T07:12:38.715004+0000","last_became_peered":"2024-07-17T07:12:38.715004+0000","last_unstale":"2024-07-17T07:12:39.677307+0000","last_undegraded":"2024-07-17T07:12:39.677307+0000","last_fullsized":"2024-07-17T07:12:39.677307+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:40.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.670879+0000","last_change":"2024-07-17T07:12:38.704937+0000","last_active":"2024-07-17T07:12:39.670879+0000","last_peered":"2024-07-17T07:12:39.670879+0000","last_clean":"2024-07-17T07:12:39.670879+0000","last_became_active":"2024-07-17T07:12:38.703518+0000","last_became_peered":"2024-07-17T07:12:38.703518+0000","last_unstale":"2024-07-17T07:12:39.670879+0000","last_undegraded":"2024-07-17T07:12:39.670879+0000","last_fullsized":"2024-07-17T07:12:39.670879+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:51.009618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678371+0000","last_change":"2024-07-17T07:12:38.705084+0000","last_active":"2024-07-17T07:12:39.678371+0000","last_peered":"2024-07-17T07:12:39.678371+0000","last_clean":"2024-07-17T07:12:39.678371+0000","last_became_active":"2024-07-17T07:12:38.704253+0000","last_became_peered":"2024-07-17T07:12:38.704253+0000","last_unstale":"2024-07-17T07:12:39.678371+0000","last_undegraded":"2024-07-17T07:12:39.678371+0000","last_fullsized":"2024-07-17T07:12:39.678371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:32.835642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716206+0000","last_change":"2024-07-17T07:12:38.709905+0000","last_active":"2024-07-17T07:12:39.716206+0000","last_peered":"2024-07-17T07:12:39.716206+0000","last_clean":"2024-07-17T07:12:39.716206+0000","last_became_active":"2024-07-17T07:12:38.709620+0000","last_became_peered":"2024-07-17T07:12:38.709620+0000","last_unstale":"2024-07-17T07:12:39.716206+0000","last_undegraded":"2024-07-17T07:12:39.716206+0000","last_fullsized":"2024-07-17T07:12:39.716206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:38.168971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741506+0000","last_change":"2024-07-17T07:12:38.689483+0000","last_active":"2024-07-17T07:12:39.741506+0000","last_peered":"2024-07-17T07:12:39.741506+0000","last_clean":"2024-07-17T07:12:39.741506+0000","last_became_active":"2024-07-17T07:12:38.689014+0000","last_became_peered":"2024-07-17T07:12:38.689014+0000","last_unstale":"2024-07-17T07:12:39.741506+0000","last_undegraded":"2024-07-17T07:12:39.741506+0000","last_fullsized":"2024-07-17T07:12:39.741506+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:18.444879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741757+0000","last_change":"2024-07-17T07:12:38.723363+0000","last_active":"2024-07-17T07:12:39.741757+0000","last_peered":"2024-07-17T07:12:39.741757+0000","last_clean":"2024-07-17T07:12:39.741757+0000","last_became_active":"2024-07-17T07:12:38.719622+0000","last_became_peered":"2024-07-17T07:12:38.719622+0000","last_unstale":"2024-07-17T07:12:39.741757+0000","last_undegraded":"2024-07-17T07:12:39.741757+0000","last_fullsized":"2024-07-17T07:12:39.741757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:47.486876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742006+0000","last_change":"2024-07-17T07:12:38.702081+0000","last_active":"2024-07-17T07:12:39.742006+0000","last_peered":"2024-07-17T07:12:39.742006+0000","last_clean":"2024-07-17T07:12:39.742006+0000","last_became_active":"2024-07-17T07:12:38.700536+0000","last_became_peered":"2024-07-17T07:12:38.700536+0000","last_unstale":"2024-07-17T07:12:39.742006+0000","last_undegraded":"2024-07-17T07:12:39.742006+0000","last_fullsized":"2024-07-17T07:12:39.742006+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:42.857145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715180+0000","last_change":"2024-07-17T07:12:38.707375+0000","last_active":"2024-07-17T07:12:39.715180+0000","last_peered":"2024-07-17T07:12:39.715180+0000","last_clean":"2024-07-17T07:12:39.715180+0000","last_became_active":"2024-07-17T07:12:38.702549+0000","last_became_peered":"2024-07-17T07:12:38.702549+0000","last_unstale":"2024-07-17T07:12:39.715180+0000","last_undegraded":"2024-07-17T07:12:39.715180+0000","last_fullsized":"2024-07-17T07:12:39.715180+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:37.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.673850+0000","last_change":"2024-07-17T07:12:38.715626+0000","last_active":"2024-07-17T07:12:39.673850+0000","last_peered":"2024-07-17T07:12:39.673850+0000","last_clean":"2024-07-17T07:12:39.673850+0000","last_became_active":"2024-07-17T07:12:38.715338+0000","last_became_peered":"2024-07-17T07:12:38.715338+0000","last_unstale":"2024-07-17T07:12:39.673850+0000","last_undegraded":"2024-07-17T07:12:39.673850+0000","last_fullsized":"2024-07-17T07:12:39.673850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:13.872210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741096+0000","last_change":"2024-07-17T07:12:38.703602+0000","last_active":"2024-07-17T07:12:39.741096+0000","last_peered":"2024-07-17T07:12:39.741096+0000","last_clean":"2024-07-17T07:12:39.741096+0000","last_became_active":"2024-07-17T07:12:38.702798+0000","last_became_peered":"2024-07-17T07:12:38.702798+0000","last_unstale":"2024-07-17T07:12:39.741096+0000","last_undegraded":"2024-07-17T07:12:39.741096+0000","last_fullsized":"2024-07-17T07:12:39.741096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:05.026869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742694+0000","last_change":"2024-07-17T07:12:38.703945+0000","last_active":"2024-07-17T07:12:39.742694+0000","last_peered":"2024-07-17T07:12:39.742694+0000","last_clean":"2024-07-17T07:12:39.742694+0000","last_became_active":"2024-07-17T07:12:38.703424+0000","last_became_peered":"2024-07-17T07:12:38.703424+0000","last_unstale":"2024-07-17T07:12:39.742694+0000","last_undegraded":"2024-07-17T07:12:39.742694+0000","last_fullsized":"2024-07-17T07:12:39.742694+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:59.250951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677640+0000","last_change":"2024-07-17T07:12:37.658029+0000","last_active":"2024-07-17T07:12:39.677640+0000","last_peered":"2024-07-17T07:12:39.677640+0000","last_clean":"2024-07-17T07:12:39.677640+0000","last_became_active":"2024-07-17T07:12:37.657698+0000","last_became_peered":"2024-07-17T07:12:37.657698+0000","last_unstale":"2024-07-17T07:12:39.677640+0000","last_undegraded":"2024-07-17T07:12:39.677640+0000","last_fullsized":"2024-07-17T07:12:39.677640+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:29:44.414198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743005+0000","last_change":"2024-07-17T07:12:38.695671+0000","last_active":"2024-07-17T07:12:39.743005+0000","last_peered":"2024-07-17T07:12:39.743005+0000","last_clean":"2024-07-17T07:12:39.743005+0000","last_became_active":"2024-07-17T07:12:38.693759+0000","last_became_peered":"2024-07-17T07:12:38.693759+0000","last_unstale":"2024-07-17T07:12:39.743005+0000","last_undegraded":"2024-07-17T07:12:39.743005+0000","last_fullsized":"2024-07-17T07:12:39.743005+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:03.876326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716573+0000","last_change":"2024-07-17T07:12:37.669918+0000","last_active":"2024-07-17T07:12:39.716573+0000","last_peered":"2024-07-17T07:12:39.716573+0000","last_clean":"2024-07-17T07:12:39.716573+0000","last_became_active":"2024-07-17T07:12:37.669489+0000","last_became_peered":"2024-07-17T07:12:37.669489+0000","last_unstale":"2024-07-17T07:12:39.716573+0000","last_undegraded":"2024-07-17T07:12:39.716573+0000","last_fullsized":"2024-07-17T07:12:39.716573+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:22:22.398136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741989+0000","last_change":"2024-07-17T07:12:38.709936+0000","last_active":"2024-07-17T07:12:39.741989+0000","last_peered":"2024-07-17T07:12:39.741989+0000","last_clean":"2024-07-17T07:12:39.741989+0000","last_became_active":"2024-07-17T07:12:38.709554+0000","last_became_peered":"2024-07-17T07:12:38.709554+0000","last_unstale":"2024-07-17T07:12:39.741989+0000","last_undegraded":"2024-07-17T07:12:39.741989+0000","last_fullsized":"2024-07-17T07:12:39.741989+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:16.982429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741889+0000","last_change":"2024-07-17T07:12:37.666229+0000","last_active":"2024-07-17T07:12:39.741889+0000","last_peered":"2024-07-17T07:12:39.741889+0000","last_clean":"2024-07-17T07:12:39.741889+0000","last_became_active":"2024-07-17T07:12:37.665881+0000","last_became_peered":"2024-07-17T07:12:37.665881+0000","last_unstale":"2024-07-17T07:12:39.741889+0000","last_undegraded":"2024-07-17T07:12:39.741889+0000","last_fullsized":"2024-07-17T07:12:39.741889+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:09.226047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743598+0000","last_change":"2024-07-17T07:12:38.685526+0000","last_active":"2024-07-17T07:12:39.743598+0000","last_peered":"2024-07-17T07:12:39.743598+0000","last_clean":"2024-07-17T07:12:39.743598+0000","last_became_active":"2024-07-17T07:12:38.685076+0000","last_became_peered":"2024-07-17T07:12:38.685076+0000","last_unstale":"2024-07-17T07:12:39.743598+0000","last_undegraded":"2024-07-17T07:12:39.743598+0000","last_fullsized":"2024-07-17T07:12:39.743598+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:21:14.108644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676957+0000","last_change":"2024-07-17T07:12:37.656657+0000","last_active":"2024-07-17T07:12:39.676957+0000","last_peered":"2024-07-17T07:12:39.676957+0000","last_clean":"2024-07-17T07:12:39.676957+0000","last_became_active":"2024-07-17T07:12:37.656050+0000","last_became_peered":"2024-07-17T07:12:37.656050+0000","last_unstale":"2024-07-17T07:12:39.676957+0000","last_undegraded":"2024-07-17T07:12:39.676957+0000","last_fullsized":"2024-07-17T07:12:39.676957+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:58.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741704+0000","last_change":"2024-07-17T07:12:38.727829+0000","last_active":"2024-07-17T07:12:39.741704+0000","last_peered":"2024-07-17T07:12:39.741704+0000","last_clean":"2024-07-17T07:12:39.741704+0000","last_became_active":"2024-07-17T07:12:38.727495+0000","last_became_peered":"2024-07-17T07:12:38.727495+0000","last_unstale":"2024-07-17T07:12:39.741704+0000","last_undegraded":"2024-07-17T07:12:39.741704+0000","last_fullsized":"2024-07-17T07:12:39.741704+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:31.741494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715350+0000","last_change":"2024-07-17T07:12:37.667487+0000","last_active":"2024-07-17T07:12:39.715350+0000","last_peered":"2024-07-17T07:12:39.715350+0000","last_clean":"2024-07-17T07:12:39.715350+0000","last_became_active":"2024-07-17T07:12:37.667208+0000","last_became_peered":"2024-07-17T07:12:37.667208+0000","last_unstale":"2024-07-17T07:12:39.715350+0000","last_undegraded":"2024-07-17T07:12:39.715350+0000","last_fullsized":"2024-07-17T07:12:39.715350+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:56.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675561+0000","last_change":"2024-07-17T07:12:38.694819+0000","last_active":"2024-07-17T07:12:39.675561+0000","last_peered":"2024-07-17T07:12:39.675561+0000","last_clean":"2024-07-17T07:12:39.675561+0000","last_became_active":"2024-07-17T07:12:38.693717+0000","last_became_peered":"2024-07-17T07:12:38.693717+0000","last_unstale":"2024-07-17T07:12:39.675561+0000","last_undegraded":"2024-07-17T07:12:39.675561+0000","last_fullsized":"2024-07-17T07:12:39.675561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:33.904831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742561+0000","last_change":"2024-07-17T07:12:37.654437+0000","last_active":"2024-07-17T07:12:39.742561+0000","last_peered":"2024-07-17T07:12:39.742561+0000","last_clean":"2024-07-17T07:12:39.742561+0000","last_became_active":"2024-07-17T07:12:37.654101+0000","last_became_peered":"2024-07-17T07:12:37.654101+0000","last_unstale":"2024-07-17T07:12:39.742561+0000","last_undegraded":"2024-07-17T07:12:39.742561+0000","last_fullsized":"2024-07-17T07:12:39.742561+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:33.663053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741183+0000","last_change":"2024-07-17T07:12:38.711793+0000","last_active":"2024-07-17T07:12:39.741183+0000","last_peered":"2024-07-17T07:12:39.741183+0000","last_clean":"2024-07-17T07:12:39.741183+0000","last_became_active":"2024-07-17T07:12:38.711388+0000","last_became_peered":"2024-07-17T07:12:38.711388+0000","last_unstale":"2024-07-17T07:12:39.741183+0000","last_undegraded":"2024-07-17T07:12:39.741183+0000","last_fullsized":"2024-07-17T07:12:39.741183+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:28.941309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671124+0000","last_change":"2024-07-17T07:12:37.666177+0000","last_active":"2024-07-17T07:12:39.671124+0000","last_peered":"2024-07-17T07:12:39.671124+0000","last_clean":"2024-07-17T07:12:39.671124+0000","last_became_active":"2024-07-17T07:12:37.665910+0000","last_became_peered":"2024-07-17T07:12:37.665910+0000","last_unstale":"2024-07-17T07:12:39.671124+0000","last_undegraded":"2024-07-17T07:12:39.671124+0000","last_fullsized":"2024-07-17T07:12:39.671124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:52.635951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742450+0000","last_change":"2024-07-17T07:12:38.712400+0000","last_active":"2024-07-17T07:12:39.742450+0000","last_peered":"2024-07-17T07:12:39.742450+0000","last_clean":"2024-07-17T07:12:39.742450+0000","last_became_active":"2024-07-17T07:12:38.712049+0000","last_became_peered":"2024-07-17T07:12:38.712049+0000","last_unstale":"2024-07-17T07:12:39.742450+0000","last_undegraded":"2024-07-17T07:12:39.742450+0000","last_fullsized":"2024-07-17T07:12:39.742450+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:04.351239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741550+0000","last_change":"2024-07-17T07:12:37.669060+0000","last_active":"2024-07-17T07:12:39.741550+0000","last_peered":"2024-07-17T07:12:39.741550+0000","last_clean":"2024-07-17T07:12:39.741550+0000","last_became_active":"2024-07-17T07:12:37.668810+0000","last_became_peered":"2024-07-17T07:12:37.668810+0000","last_unstale":"2024-07-17T07:12:39.741550+0000","last_undegraded":"2024-07-17T07:12:39.741550+0000","last_fullsized":"2024-07-17T07:12:39.741550+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:44.136760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.715540+0000","last_change":"2024-07-17T07:12:38.709074+0000","last_active":"2024-07-17T07:12:39.715540+0000","last_peered":"2024-07-17T07:12:39.715540+0000","last_clean":"2024-07-17T07:12:39.715540+0000","last_became_active":"2024-07-17T07:12:38.708792+0000","last_became_peered":"2024-07-17T07:12:38.708792+0000","last_unstale":"2024-07-17T07:12:39.715540+0000","last_undegraded":"2024-07-17T07:12:39.715540+0000","last_fullsized":"2024-07-17T07:12:39.715540+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:17.802835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678603+0000","last_change":"2024-07-17T07:12:37.659519+0000","last_active":"2024-07-17T07:12:39.678603+0000","last_peered":"2024-07-17T07:12:39.678603+0000","last_clean":"2024-07-17T07:12:39.678603+0000","last_became_active":"2024-07-17T07:12:37.659211+0000","last_became_peered":"2024-07-17T07:12:37.659211+0000","last_unstale":"2024-07-17T07:12:39.678603+0000","last_undegraded":"2024-07-17T07:12:39.678603+0000","last_fullsized":"2024-07-17T07:12:39.678603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:11.283026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742051+0000","last_change":"2024-07-17T07:12:38.712142+0000","last_active":"2024-07-17T07:12:39.742051+0000","last_peered":"2024-07-17T07:12:39.742051+0000","last_clean":"2024-07-17T07:12:39.742051+0000","last_became_active":"2024-07-17T07:12:38.711020+0000","last_became_peered":"2024-07-17T07:12:38.711020+0000","last_unstale":"2024-07-17T07:12:39.742051+0000","last_undegraded":"2024-07-17T07:12:39.742051+0000","last_fullsized":"2024-07-17T07:12:39.742051+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:00.101735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671413+0000","last_change":"2024-07-17T07:12:37.665540+0000","last_active":"2024-07-17T07:12:39.671413+0000","last_peered":"2024-07-17T07:12:39.671413+0000","last_clean":"2024-07-17T07:12:39.671413+0000","last_became_active":"2024-07-17T07:12:37.665247+0000","last_became_peered":"2024-07-17T07:12:37.665247+0000","last_unstale":"2024-07-17T07:12:39.671413+0000","last_undegraded":"2024-07-17T07:12:39.671413+0000","last_fullsized":"2024-07-17T07:12:39.671413+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:16.324968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.671493+0000","last_change":"2024-07-17T07:12:38.712217+0000","last_active":"2024-07-17T07:12:39.671493+0000","last_peered":"2024-07-17T07:12:39.671493+0000","last_clean":"2024-07-17T07:12:39.671493+0000","last_became_active":"2024-07-17T07:12:38.711920+0000","last_became_peered":"2024-07-17T07:12:38.711920+0000","last_unstale":"2024-07-17T07:12:39.671493+0000","last_undegraded":"2024-07-17T07:12:39.671493+0000","last_fullsized":"2024-07-17T07:12:39.671493+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:33.560453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716436+0000","last_change":"2024-07-17T07:12:37.669728+0000","last_active":"2024-07-17T07:12:39.716436+0000","last_peered":"2024-07-17T07:12:39.716436+0000","last_clean":"2024-07-17T07:12:39.716436+0000","last_became_active":"2024-07-17T07:12:37.668971+0000","last_became_peered":"2024-07-17T07:12:37.668971+0000","last_unstale":"2024-07-17T07:12:39.716436+0000","last_undegraded":"2024-07-17T07:12:39.716436+0000","last_fullsized":"2024-07-17T07:12:39.716436+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:08:16.833343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743396+0000","last_change":"2024-07-17T07:12:38.717267+0000","last_active":"2024-07-17T07:12:39.743396+0000","last_peered":"2024-07-17T07:12:39.743396+0000","last_clean":"2024-07-17T07:12:39.743396+0000","last_became_active":"2024-07-17T07:12:38.716937+0000","last_became_peered":"2024-07-17T07:12:38.716937+0000","last_unstale":"2024-07-17T07:12:39.743396+0000","last_undegraded":"2024-07-17T07:12:39.743396+0000","last_fullsized":"2024-07-17T07:12:39.743396+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:44:41.254304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743022+0000","last_change":"2024-07-17T07:12:37.656357+0000","last_active":"2024-07-17T07:12:39.743022+0000","last_peered":"2024-07-17T07:12:39.743022+0000","last_clean":"2024-07-17T07:12:39.743022+0000","last_became_active":"2024-07-17T07:12:37.656055+0000","last_became_peered":"2024-07-17T07:12:37.656055+0000","last_unstale":"2024-07-17T07:12:39.743022+0000","last_undegraded":"2024-07-17T07:12:39.743022+0000","last_fullsized":"2024-07-17T07:12:39.743022+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:56.455202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741906+0000","last_change":"2024-07-17T07:12:38.726308+0000","last_active":"2024-07-17T07:12:39.741906+0000","last_peered":"2024-07-17T07:12:39.741906+0000","last_clean":"2024-07-17T07:12:39.741906+0000","last_became_active":"2024-07-17T07:12:38.723197+0000","last_became_peered":"2024-07-17T07:12:38.723197+0000","last_unstale":"2024-07-17T07:12:39.741906+0000","last_undegraded":"2024-07-17T07:12:39.741906+0000","last_fullsized":"2024-07-17T07:12:39.741906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:59.106505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742077+0000","last_change":"2024-07-17T07:12:37.670827+0000","last_active":"2024-07-17T07:12:39.742077+0000","last_peered":"2024-07-17T07:12:39.742077+0000","last_clean":"2024-07-17T07:12:39.742077+0000","last_became_active":"2024-07-17T07:12:37.670521+0000","last_became_peered":"2024-07-17T07:12:37.670521+0000","last_unstale":"2024-07-17T07:12:39.742077+0000","last_undegraded":"2024-07-17T07:12:39.742077+0000","last_fullsized":"2024-07-17T07:12:39.742077+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:47.113275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.714820+0000","last_change":"2024-07-17T07:12:38.700266+0000","last_active":"2024-07-17T07:12:39.714820+0000","last_peered":"2024-07-17T07:12:39.714820+0000","last_clean":"2024-07-17T07:12:39.714820+0000","last_became_active":"2024-07-17T07:12:38.699932+0000","last_became_peered":"2024-07-17T07:12:38.699932+0000","last_unstale":"2024-07-17T07:12:39.714820+0000","last_undegraded":"2024-07-17T07:12:39.714820+0000","last_fullsized":"2024-07-17T07:12:39.714820+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:58.888424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742003+0000","last_change":"2024-07-17T07:12:37.667846+0000","last_active":"2024-07-17T07:12:39.742003+0000","last_peered":"2024-07-17T07:12:39.742003+0000","last_clean":"2024-07-17T07:12:39.742003+0000","last_became_active":"2024-07-17T07:12:37.666937+0000","last_became_peered":"2024-07-17T07:12:37.666937+0000","last_unstale":"2024-07-17T07:12:39.742003+0000","last_undegraded":"2024-07-17T07:12:39.742003+0000","last_fullsized":"2024-07-17T07:12:39.742003+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:49.991886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740103+0000","last_change":"2024-07-17T07:12:38.720716+0000","last_active":"2024-07-17T07:12:39.740103+0000","last_peered":"2024-07-17T07:12:39.740103+0000","last_clean":"2024-07-17T07:12:39.740103+0000","last_became_active":"2024-07-17T07:12:38.720409+0000","last_became_peered":"2024-07-17T07:12:38.720409+0000","last_unstale":"2024-07-17T07:12:39.740103+0000","last_undegraded":"2024-07-17T07:12:39.740103+0000","last_fullsized":"2024-07-17T07:12:39.740103+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:38.353810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741830+0000","last_change":"2024-07-17T07:12:37.659483+0000","last_active":"2024-07-17T07:12:39.741830+0000","last_peered":"2024-07-17T07:12:39.741830+0000","last_clean":"2024-07-17T07:12:39.741830+0000","last_became_active":"2024-07-17T07:12:37.659153+0000","last_became_peered":"2024-07-17T07:12:37.659153+0000","last_unstale":"2024-07-17T07:12:39.741830+0000","last_undegraded":"2024-07-17T07:12:39.741830+0000","last_fullsized":"2024-07-17T07:12:39.741830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:26:32.582900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740402+0000","last_change":"2024-07-17T07:12:38.702732+0000","last_active":"2024-07-17T07:12:39.740402+0000","last_peered":"2024-07-17T07:12:39.740402+0000","last_clean":"2024-07-17T07:12:39.740402+0000","last_became_active":"2024-07-17T07:12:38.702357+0000","last_became_peered":"2024-07-17T07:12:38.702357+0000","last_unstale":"2024-07-17T07:12:39.740402+0000","last_undegraded":"2024-07-17T07:12:39.740402+0000","last_fullsized":"2024-07-17T07:12:39.740402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:17.232545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742041+0000","last_change":"2024-07-17T07:12:37.662612+0000","last_active":"2024-07-17T07:12:39.742041+0000","last_peered":"2024-07-17T07:12:39.742041+0000","last_clean":"2024-07-17T07:12:39.742041+0000","last_became_active":"2024-07-17T07:12:37.661583+0000","last_became_peered":"2024-07-17T07:12:37.661583+0000","last_unstale":"2024-07-17T07:12:39.742041+0000","last_undegraded":"2024-07-17T07:12:39.742041+0000","last_fullsized":"2024-07-17T07:12:39.742041+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:38:37.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742884+0000","last_change":"2024-07-17T07:12:38.712965+0000","last_active":"2024-07-17T07:12:39.742884+0000","last_peered":"2024-07-17T07:12:39.742884+0000","last_clean":"2024-07-17T07:12:39.742884+0000","last_became_active":"2024-07-17T07:12:38.711424+0000","last_became_peered":"2024-07-17T07:12:38.711424+0000","last_unstale":"2024-07-17T07:12:39.742884+0000","last_undegraded":"2024-07-17T07:12:39.742884+0000","last_fullsized":"2024-07-17T07:12:39.742884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:14:41.118839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677953+0000","last_change":"2024-07-17T07:12:37.660151+0000","last_active":"2024-07-17T07:12:39.677953+0000","last_peered":"2024-07-17T07:12:39.677953+0000","last_clean":"2024-07-17T07:12:39.677953+0000","last_became_active":"2024-07-17T07:12:37.659851+0000","last_became_peered":"2024-07-17T07:12:37.659851+0000","last_unstale":"2024-07-17T07:12:39.677953+0000","last_undegraded":"2024-07-17T07:12:39.677953+0000","last_fullsized":"2024-07-17T07:12:39.677953+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:03.333506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676944+0000","last_change":"2024-07-17T07:12:38.701062+0000","last_active":"2024-07-17T07:12:39.676944+0000","last_peered":"2024-07-17T07:12:39.676944+0000","last_clean":"2024-07-17T07:12:39.676944+0000","last_became_active":"2024-07-17T07:12:38.700772+0000","last_became_peered":"2024-07-17T07:12:38.700772+0000","last_unstale":"2024-07-17T07:12:39.676944+0000","last_undegraded":"2024-07-17T07:12:39.676944+0000","last_fullsized":"2024-07-17T07:12:39.676944+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:01.704078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741928+0000","last_change":"2024-07-17T07:12:37.667835+0000","last_active":"2024-07-17T07:12:39.741928+0000","last_peered":"2024-07-17T07:12:39.741928+0000","last_clean":"2024-07-17T07:12:39.741928+0000","last_became_active":"2024-07-17T07:12:37.667542+0000","last_became_peered":"2024-07-17T07:12:37.667542+0000","last_unstale":"2024-07-17T07:12:39.741928+0000","last_undegraded":"2024-07-17T07:12:39.741928+0000","last_fullsized":"2024-07-17T07:12:39.741928+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:35:59.328496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716330+0000","last_change":"2024-07-17T07:12:38.708283+0000","last_active":"2024-07-17T07:12:39.716330+0000","last_peered":"2024-07-17T07:12:39.716330+0000","last_clean":"2024-07-17T07:12:39.716330+0000","last_became_active":"2024-07-17T07:12:38.699913+0000","last_became_peered":"2024-07-17T07:12:38.699913+0000","last_unstale":"2024-07-17T07:12:39.716330+0000","last_undegraded":"2024-07-17T07:12:39.716330+0000","last_fullsized":"2024-07-17T07:12:39.716330+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.423010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716470+0000","last_change":"2024-07-17T07:12:37.673523+0000","last_active":"2024-07-17T07:12:39.716470+0000","last_peered":"2024-07-17T07:12:39.716470+0000","last_clean":"2024-07-17T07:12:39.716470+0000","last_became_active":"2024-07-17T07:12:37.669347+0000","last_became_peered":"2024-07-17T07:12:37.669347+0000","last_unstale":"2024-07-17T07:12:39.716470+0000","last_undegraded":"2024-07-17T07:12:39.716470+0000","last_fullsized":"2024-07-17T07:12:39.716470+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:28.526105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740782+0000","last_change":"2024-07-17T07:12:38.722400+0000","last_active":"2024-07-17T07:12:39.740782+0000","last_peered":"2024-07-17T07:12:39.740782+0000","last_clean":"2024-07-17T07:12:39.740782+0000","last_became_active":"2024-07-17T07:12:38.722095+0000","last_became_peered":"2024-07-17T07:12:38.722095+0000","last_unstale":"2024-07-17T07:12:39.740782+0000","last_undegraded":"2024-07-17T07:12:39.740782+0000","last_fullsized":"2024-07-17T07:12:39.740782+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:46.608059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676115+0000","last_change":"2024-07-17T07:12:37.669987+0000","last_active":"2024-07-17T07:12:39.676115+0000","last_peered":"2024-07-17T07:12:39.676115+0000","last_clean":"2024-07-17T07:12:39.676115+0000","last_became_active":"2024-07-17T07:12:37.668534+0000","last_became_peered":"2024-07-17T07:12:37.668534+0000","last_unstale":"2024-07-17T07:12:39.676115+0000","last_undegraded":"2024-07-17T07:12:39.676115+0000","last_fullsized":"2024-07-17T07:12:39.676115+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:21:23.486022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.674189+0000","last_change":"2024-07-17T07:12:38.703281+0000","last_active":"2024-07-17T07:12:39.674189+0000","last_peered":"2024-07-17T07:12:39.674189+0000","last_clean":"2024-07-17T07:12:39.674189+0000","last_became_active":"2024-07-17T07:12:38.702955+0000","last_became_peered":"2024-07-17T07:12:38.702955+0000","last_unstale":"2024-07-17T07:12:39.674189+0000","last_undegraded":"2024-07-17T07:12:39.674189+0000","last_fullsized":"2024-07-17T07:12:39.674189+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:17.342700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.740203+0000","last_change":"2024-07-17T07:12:37.659882+0000","last_active":"2024-07-17T07:12:39.740203+0000","last_peered":"2024-07-17T07:12:39.740203+0000","last_clean":"2024-07-17T07:12:39.740203+0000","last_became_active":"2024-07-17T07:12:37.659585+0000","last_became_peered":"2024-07-17T07:12:37.659585+0000","last_unstale":"2024-07-17T07:12:39.740203+0000","last_undegraded":"2024-07-17T07:12:39.740203+0000","last_fullsized":"2024-07-17T07:12:39.740203+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:56.226193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675041+0000","last_change":"2024-07-17T07:12:38.728528+0000","last_active":"2024-07-17T07:12:39.675041+0000","last_peered":"2024-07-17T07:12:39.675041+0000","last_clean":"2024-07-17T07:12:39.675041+0000","last_became_active":"2024-07-17T07:12:38.728175+0000","last_became_peered":"2024-07-17T07:12:38.728175+0000","last_unstale":"2024-07-17T07:12:39.675041+0000","last_undegraded":"2024-07-17T07:12:39.675041+0000","last_fullsized":"2024-07-17T07:12:39.675041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:53.653684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.674425+0000","last_change":"2024-07-17T07:12:37.664839+0000","last_active":"2024-07-17T07:12:39.674425+0000","last_peered":"2024-07-17T07:12:39.674425+0000","last_clean":"2024-07-17T07:12:39.674425+0000","last_became_active":"2024-07-17T07:12:37.664369+0000","last_became_peered":"2024-07-17T07:12:37.664369+0000","last_unstale":"2024-07-17T07:12:39.674425+0000","last_undegraded":"2024-07-17T07:12:39.674425+0000","last_fullsized":"2024-07-17T07:12:39.674425+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:16:55.862933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742266+0000","last_change":"2024-07-17T07:12:38.695034+0000","last_active":"2024-07-17T07:12:39.742266+0000","last_peered":"2024-07-17T07:12:39.742266+0000","last_clean":"2024-07-17T07:12:39.742266+0000","last_became_active":"2024-07-17T07:12:38.694696+0000","last_became_peered":"2024-07-17T07:12:38.694696+0000","last_unstale":"2024-07-17T07:12:39.742266+0000","last_undegraded":"2024-07-17T07:12:39.742266+0000","last_fullsized":"2024-07-17T07:12:39.742266+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:39:41.277426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.739950+0000","last_change":"2024-07-17T07:12:37.668013+0000","last_active":"2024-07-17T07:12:39.739950+0000","last_peered":"2024-07-17T07:12:39.739950+0000","last_clean":"2024-07-17T07:12:39.739950+0000","last_became_active":"2024-07-17T07:12:37.667594+0000","last_became_peered":"2024-07-17T07:12:37.667594+0000","last_unstale":"2024-07-17T07:12:39.739950+0000","last_undegraded":"2024-07-17T07:12:39.739950+0000","last_fullsized":"2024-07-17T07:12:39.739950+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:21:39.154468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677048+0000","last_change":"2024-07-17T07:12:38.695739+0000","last_active":"2024-07-17T07:12:39.677048+0000","last_peered":"2024-07-17T07:12:39.677048+0000","last_clean":"2024-07-17T07:12:39.677048+0000","last_became_active":"2024-07-17T07:12:38.695299+0000","last_became_peered":"2024-07-17T07:12:38.695299+0000","last_unstale":"2024-07-17T07:12:39.677048+0000","last_undegraded":"2024-07-17T07:12:39.677048+0000","last_fullsized":"2024-07-17T07:12:39.677048+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:48:35.710592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742289+0000","last_change":"2024-07-17T07:12:37.666148+0000","last_active":"2024-07-17T07:12:39.742289+0000","last_peered":"2024-07-17T07:12:39.742289+0000","last_clean":"2024-07-17T07:12:39.742289+0000","last_became_active":"2024-07-17T07:12:37.665776+0000","last_became_peered":"2024-07-17T07:12:37.665776+0000","last_unstale":"2024-07-17T07:12:39.742289+0000","last_undegraded":"2024-07-17T07:12:39.742289+0000","last_fullsized":"2024-07-17T07:12:39.742289+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:07.951745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743131+0000","last_change":"2024-07-17T07:12:38.702668+0000","last_active":"2024-07-17T07:12:39.743131+0000","last_peered":"2024-07-17T07:12:39.743131+0000","last_clean":"2024-07-17T07:12:39.743131+0000","last_became_active":"2024-07-17T07:12:38.702361+0000","last_became_peered":"2024-07-17T07:12:38.702361+0000","last_unstale":"2024-07-17T07:12:39.743131+0000","last_undegraded":"2024-07-17T07:12:39.743131+0000","last_fullsized":"2024-07-17T07:12:39.743131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:46.118720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"112'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.720380+0000","last_change":"2024-07-17T07:12:37.673507+0000","last_active":"2024-07-17T07:12:39.720380+0000","last_peered":"2024-07-17T07:12:39.720380+0000","last_clean":"2024-07-17T07:12:39.720380+0000","last_became_active":"2024-07-17T07:12:37.672594+0000","last_became_peered":"2024-07-17T07:12:37.672594+0000","last_unstale":"2024-07-17T07:12:39.720380+0000","last_undegraded":"2024-07-17T07:12:39.720380+0000","last_fullsized":"2024-07-17T07:12:39.720380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:27.017511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.743385+0000","last_change":"2024-07-17T07:12:38.689323+0000","last_active":"2024-07-17T07:12:39.743385+0000","last_peered":"2024-07-17T07:12:39.743385+0000","last_clean":"2024-07-17T07:12:39.743385+0000","last_became_active":"2024-07-17T07:12:38.689021+0000","last_became_peered":"2024-07-17T07:12:38.689021+0000","last_unstale":"2024-07-17T07:12:39.743385+0000","last_undegraded":"2024-07-17T07:12:39.743385+0000","last_fullsized":"2024-07-17T07:12:39.743385+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:35:08.183779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.716750+0000","last_change":"2024-07-17T07:12:37.667832+0000","last_active":"2024-07-17T07:12:39.716750+0000","last_peered":"2024-07-17T07:12:39.716750+0000","last_clean":"2024-07-17T07:12:39.716750+0000","last_became_active":"2024-07-17T07:12:37.666502+0000","last_became_peered":"2024-07-17T07:12:37.666502+0000","last_unstale":"2024-07-17T07:12:39.716750+0000","last_undegraded":"2024-07-17T07:12:39.716750+0000","last_fullsized":"2024-07-17T07:12:39.716750+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:08:58.697317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.678546+0000","last_change":"2024-07-17T07:12:38.713683+0000","last_active":"2024-07-17T07:12:39.678546+0000","last_peered":"2024-07-17T07:12:39.678546+0000","last_clean":"2024-07-17T07:12:39.678546+0000","last_became_active":"2024-07-17T07:12:38.713380+0000","last_became_peered":"2024-07-17T07:12:38.713380+0000","last_unstale":"2024-07-17T07:12:39.678546+0000","last_undegraded":"2024-07-17T07:12:39.678546+0000","last_fullsized":"2024-07-17T07:12:39.678546+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:24.489401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677205+0000","last_change":"2024-07-17T07:12:37.648851+0000","last_active":"2024-07-17T07:12:39.677205+0000","last_peered":"2024-07-17T07:12:39.677205+0000","last_clean":"2024-07-17T07:12:39.677205+0000","last_became_active":"2024-07-17T07:12:37.646362+0000","last_became_peered":"2024-07-17T07:12:37.646362+0000","last_unstale":"2024-07-17T07:12:39.677205+0000","last_undegraded":"2024-07-17T07:12:39.677205+0000","last_fullsized":"2024-07-17T07:12:39.677205+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:48.018507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.739847+0000","last_change":"2024-07-17T07:12:38.686630+0000","last_active":"2024-07-17T07:12:39.739847+0000","last_peered":"2024-07-17T07:12:39.739847+0000","last_clean":"2024-07-17T07:12:39.739847+0000","last_became_active":"2024-07-17T07:12:38.686172+0000","last_became_peered":"2024-07-17T07:12:38.686172+0000","last_unstale":"2024-07-17T07:12:39.739847+0000","last_undegraded":"2024-07-17T07:12:39.739847+0000","last_fullsized":"2024-07-17T07:12:39.739847+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:59:16.082697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742435+0000","last_change":"2024-07-17T07:12:37.668131+0000","last_active":"2024-07-17T07:12:39.742435+0000","last_peered":"2024-07-17T07:12:39.742435+0000","last_clean":"2024-07-17T07:12:39.742435+0000","last_became_active":"2024-07-17T07:12:37.666751+0000","last_became_peered":"2024-07-17T07:12:37.666751+0000","last_unstale":"2024-07-17T07:12:39.742435+0000","last_undegraded":"2024-07-17T07:12:39.742435+0000","last_fullsized":"2024-07-17T07:12:39.742435+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:59:00.056759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.670991+0000","last_change":"2024-07-17T07:12:38.720291+0000","last_active":"2024-07-17T07:12:39.670991+0000","last_peered":"2024-07-17T07:12:39.670991+0000","last_clean":"2024-07-17T07:12:39.670991+0000","last_became_active":"2024-07-17T07:12:38.719595+0000","last_became_peered":"2024-07-17T07:12:38.719595+0000","last_unstale":"2024-07-17T07:12:39.670991+0000","last_undegraded":"2024-07-17T07:12:39.670991+0000","last_fullsized":"2024-07-17T07:12:39.670991+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:33.942741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.676271+0000","last_change":"2024-07-17T07:12:37.669522+0000","last_active":"2024-07-17T07:12:39.676271+0000","last_peered":"2024-07-17T07:12:39.676271+0000","last_clean":"2024-07-17T07:12:39.676271+0000","last_became_active":"2024-07-17T07:12:37.669285+0000","last_became_peered":"2024-07-17T07:12:37.669285+0000","last_unstale":"2024-07-17T07:12:39.676271+0000","last_undegraded":"2024-07-17T07:12:39.676271+0000","last_fullsized":"2024-07-17T07:12:39.676271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:47:28.382928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.677104+0000","last_change":"2024-07-17T07:12:38.721026+0000","last_active":"2024-07-17T07:12:39.677104+0000","last_peered":"2024-07-17T07:12:39.677104+0000","last_clean":"2024-07-17T07:12:39.677104+0000","last_became_active":"2024-07-17T07:12:38.720694+0000","last_became_peered":"2024-07-17T07:12:38.720694+0000","last_unstale":"2024-07-17T07:12:39.677104+0000","last_undegraded":"2024-07-17T07:12:39.677104+0000","last_fullsized":"2024-07-17T07:12:39.677104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:39:31.369488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.675862+0000","last_change":"2024-07-17T07:12:37.654867+0000","last_active":"2024-07-17T07:12:39.675862+0000","last_peered":"2024-07-17T07:12:39.675862+0000","last_clean":"2024-07-17T07:12:39.675862+0000","last_became_active":"2024-07-17T07:12:37.654399+0000","last_became_peered":"2024-07-17T07:12:37.654399+0000","last_unstale":"2024-07-17T07:12:39.675862+0000","last_undegraded":"2024-07-17T07:12:39.675862+0000","last_fullsized":"2024-07-17T07:12:39.675862+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:46.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742501+0000","last_change":"2024-07-17T07:12:38.700261+0000","last_active":"2024-07-17T07:12:39.742501+0000","last_peered":"2024-07-17T07:12:39.742501+0000","last_clean":"2024-07-17T07:12:39.742501+0000","last_became_active":"2024-07-17T07:12:38.699974+0000","last_became_peered":"2024-07-17T07:12:38.699974+0000","last_unstale":"2024-07-17T07:12:39.742501+0000","last_undegraded":"2024-07-17T07:12:39.742501+0000","last_fullsized":"2024-07-17T07:12:39.742501+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:37:08.662336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741541+0000","last_change":"2024-07-17T07:12:37.667021+0000","last_active":"2024-07-17T07:12:39.741541+0000","last_peered":"2024-07-17T07:12:39.741541+0000","last_clean":"2024-07-17T07:12:39.741541+0000","last_became_active":"2024-07-17T07:12:37.665511+0000","last_became_peered":"2024-07-17T07:12:37.665511+0000","last_unstale":"2024-07-17T07:12:39.741541+0000","last_undegraded":"2024-07-17T07:12:39.741541+0000","last_fullsized":"2024-07-17T07:12:39.741541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:19.060716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.742119+0000","last_change":"2024-07-17T07:12:38.707274+0000","last_active":"2024-07-17T07:12:39.742119+0000","last_peered":"2024-07-17T07:12:39.742119+0000","last_clean":"2024-07-17T07:12:39.742119+0000","last_became_active":"2024-07-17T07:12:38.706523+0000","last_became_peered":"2024-07-17T07:12:38.706523+0000","last_unstale":"2024-07-17T07:12:39.742119+0000","last_undegraded":"2024-07-17T07:12:39.742119+0000","last_fullsized":"2024-07-17T07:12:39.742119+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:37.621653+0000","last_clean_scrub_stamp":"2024-07-17T07:12:37.621653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:52:42.145408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:39.741985+0000","last_change":"2024-07-17T07:12:37.662522+0000","last_active":"2024-07-17T07:12:39.741985+0000","last_peered":"2024-07-17T07:12:39.741985+0000","last_clean":"2024-07-17T07:12:39.741985+0000","last_became_active":"2024-07-17T07:12:37.662041+0000","last_became_peered":"2024-07-17T07:12:37.662041+0000","last_unstale":"2024-07-17T07:12:39.741985+0000","last_undegraded":"2024-07-17T07:12:39.741985+0000","last_fullsized":"2024-07-17T07:12:39.741985+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:12:36.603920+0000","last_clean_scrub_stamp":"2024-07-17T07:12:36.603920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:15:23.241910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"112'1592","reported_seq":1939,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-07-17T07:12:40.401604+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:12:40.401604+0000","last_peered":"2024-07-17T07:12:40.401604+0000","last_clean":"2024-07-17T07:12:40.401604+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:12:40.401604+0000","last_undegraded":"2024-07-17T07:12:40.401604+0000","last_fullsized":"2024-07-17T07:12:40.401604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":1592,"log_dups_size":0,"ondisk_log_size":1592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":179,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":479,"num_read_kb":1221,"num_write":179,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1592,"ondisk_log_size":1592,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542358,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46988,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324852,"statfs":{"total":96636764160,"available":96588648448,"internally_reserved":0,"allocated":1961984,"data_stored":1301939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1392640,"data_stored":720943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509654,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50572,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321268,"statfs":{"total":96636764160,"available":96584978432,"internally_reserved":0,"allocated":1372160,"data_stored":718727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542359,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":711675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1359872,"data_stored":711515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58888,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312952,"statfs":{"total":96636764160,"available":96576462848,"internally_reserved":0,"allocated":1372160,"data_stored":718727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509654,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":718797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509653,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":1990656,"data_stored":1311413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:12:43.749 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:12:43.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:12:43.772 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:43.773+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 -- 172.21.15.44:0/2274835709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 msgr2=0x7f99c015a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 --2- 172.21.15.44:0/2274835709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c015a5a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00a940 tx=0x7f99ac02b850 comp rx=0 tx=0).stop 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 -- 172.21.15.44:0/2274835709 shutdown_connections 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 --2- 172.21.15.44:0/2274835709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c00b3280 0x7f99c006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 --2- 172.21.15.44:0/2274835709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c015a5a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.013+0000 7f99c5800640 1 -- 172.21.15.44:0/2274835709 >> 172.21.15.44:0/2274835709 conn(0x7f99c00b0eb0 msgr2=0x7f99c00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.014+0000 7f99c5800640 1 -- 172.21.15.44:0/2274835709 shutdown_connections 2024-07-17T07:12:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.014+0000 7f99c5800640 1 -- 172.21.15.44:0/2274835709 wait complete. 2024-07-17T07:12:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.015+0000 7f99c5800640 1 Processor -- start 2024-07-17T07:12:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.015+0000 7f99c5800640 1 -- start start 2024-07-17T07:12:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c006f6d0 0x7f99c01f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c00b3280 0x7f99c0073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00aea80 con 0x7f99c00acab0 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00ae900 con 0x7f99c006f6d0 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.016+0000 7f99c5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00aec00 con 0x7f99c00b3280 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38868/0 (socket says 172.21.15.44:38868) 2024-07-17T07:12:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 -- 172.21.15.44:0/556144018 learned_addr learned my addr 172.21.15.44:0/556144018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:44.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 -- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c00b3280 msgr2=0x7f99c0073c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c00b3280 0x7f99c0073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 -- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c006f6d0 msgr2=0x7f99c01f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c006f6d0 0x7f99c01f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 -- 172.21.15.44:0/556144018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c0150bf0 con 0x7f99c00acab0 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.017+0000 7f99be000640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00aa30 tx=0x7f99ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.018+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac005c90 con 0x7f99c00acab0 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.018+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99ac005310 con 0x7f99c00acab0 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.018+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac003e30 con 0x7f99c00acab0 2024-07-17T07:12:44.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.018+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c0150ed0 con 0x7f99c00acab0 2024-07-17T07:12:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.018+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c01f27e0 con 0x7f99c00acab0 2024-07-17T07:12:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.019+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c015a5a0 con 0x7f99c00acab0 2024-07-17T07:12:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.020+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f99ac0047a0 con 0x7f99c00acab0 2024-07-17T07:12:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.021+0000 7f99b6a00640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 0x7f99a0079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.021+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f99ac0b9bd0 con 0x7f99c00acab0 2024-07-17T07:12:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.022+0000 7f99bea00640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 0x7f99a0079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.023+0000 7f99bea00640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 0x7f99a0079130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99b0007fd0 tx=0x7f99b0007900 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.026+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f99ac041030 con 0x7f99c00acab0 2024-07-17T07:12:44.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.331+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f99c00b4200 con 0x7f99c00acab0 2024-07-17T07:12:44.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.336+0000 7f99b6a00640 1 -- 172.21.15.44:0/556144018 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7f99ac082c20 con 0x7f99c00acab0 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:12:44.335 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.338+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 msgr2=0x7f99a0079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.338+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 0x7f99a0079130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99b0007fd0 tx=0x7f99b0007900 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 msgr2=0x7f99c00715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00aa30 tx=0x7f99ac003c10 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 shutdown_connections 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f99a0076c70 0x7f99a0079130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c00b3280 0x7f99c0073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99c00acab0 0x7f99c00715e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 --2- 172.21.15.44:0/556144018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c006f6d0 0x7f99c01f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 >> 172.21.15.44:0/556144018 conn(0x7f99c00b0eb0 msgr2=0x7f99c015ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 shutdown_connections 2024-07-17T07:12:44.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.339+0000 7f99c5800640 1 -- 172.21.15.44:0/556144018 wait complete. 2024-07-17T07:12:44.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:12:44.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 -- 172.21.15.44:0/16361182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e8072600 msgr2=0x7ff3e815d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/16361182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e8072600 0x7ff3e815d910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8005fd0 tx=0x7ff3d802a540 comp rx=0 tx=0).stop 2024-07-17T07:12:44.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 -- 172.21.15.44:0/16361182 shutdown_connections 2024-07-17T07:12:44.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/16361182 >> v2:172.21.15.175:3300/0 conn(0x7ff3e80adfb0 0x7ff3e8071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/16361182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e8072600 0x7ff3e815d910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 -- 172.21.15.44:0/16361182 >> 172.21.15.44:0/16361182 conn(0x7ff3e80cf4d0 msgr2=0x7ff3e80cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:44.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 -- 172.21.15.44:0/16361182 shutdown_connections 2024-07-17T07:12:44.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.569+0000 7ff3e7e00640 1 -- 172.21.15.44:0/16361182 wait complete. 2024-07-17T07:12:44.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.571+0000 7ff3e7e00640 1 Processor -- start 2024-07-17T07:12:44.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.571+0000 7ff3e7e00640 1 -- start start 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80729e0 0x7ff3e8154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8144200 0x7ff3e8144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e815f810 con 0x7ff3e80adfb0 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e815f690 con 0x7ff3e8144200 2024-07-17T07:12:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e815f990 con 0x7ff3e80729e0 2024-07-17T07:12:44.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38882/0 (socket says 172.21.15.44:38882) 2024-07-17T07:12:44.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.572+0000 7ff3e6000640 1 -- 172.21.15.44:0/1156064077 learned_addr learned my addr 172.21.15.44:0/1156064077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e7400640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8144200 0x7ff3e8144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6a00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80729e0 0x7ff3e8154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 -- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80729e0 msgr2=0x7ff3e8154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80729e0 0x7ff3e8154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 -- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8144200 msgr2=0x7ff3e8144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8144200 0x7ff3e8144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 -- 172.21.15.44:0/1156064077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e815b610 con 0x7ff3e80adfb0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.573+0000 7ff3e6000640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c450 tx=0x7ff3d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.574+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40052d0 con 0x7ff3e80adfb0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.574+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3d4005470 con 0x7ff3e80adfb0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.574+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4012870 con 0x7ff3e80adfb0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.574+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e815b890 con 0x7ff3e80adfb0 2024-07-17T07:12:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.574+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e815be50 con 0x7ff3e80adfb0 2024-07-17T07:12:44.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.575+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e815d910 con 0x7ff3e80adfb0 2024-07-17T07:12:44.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.576+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7ff3d40035f0 con 0x7ff3e80adfb0 2024-07-17T07:12:44.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.577+0000 7ff3dea00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 0x7ff3c8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:44.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.577+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7ff3d4098320 con 0x7ff3e80adfb0 2024-07-17T07:12:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.578+0000 7ff3e6a00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 0x7ff3c8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.579+0000 7ff3e6a00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 0x7ff3c8079100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8033000 tx=0x7ff3d8004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:44.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.582+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff3e815b610 con 0x7ff3e80adfb0 2024-07-17T07:12:44.618 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T07:12:44.620+0000 7f7de6000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:12:44.810 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:44.810+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:44.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.844+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e8071ed0 con 0x7ff3e80adfb0 2024-07-17T07:12:44.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.845+0000 7ff3dea00640 1 -- 172.21.15.44:0/1156064077 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7ff3d4061370 con 0x7ff3e80adfb0 2024-07-17T07:12:44.844 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:44.845 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:12:44.845 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:12:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 msgr2=0x7ff3c8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 0x7ff3c8079100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8033000 tx=0x7ff3d8004730 comp rx=0 tx=0).stop 2024-07-17T07:12:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 msgr2=0x7ff3e81f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c450 tx=0x7ff3d400c910 comp rx=0 tx=0).stop 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 shutdown_connections 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7ff3c8076c40 0x7ff3c8079100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8144200 0x7ff3e8144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e80adfb0 0x7ff3e81f4760 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 --2- 172.21.15.44:0/1156064077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3e80729e0 0x7ff3e8154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.848+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 >> 172.21.15.44:0/1156064077 conn(0x7ff3e80cf4d0 msgr2=0x7ff3e80cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.849+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 shutdown_connections 2024-07-17T07:12:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:44.849+0000 7ff3e7e00640 1 -- 172.21.15.44:0/1156064077 wait complete. 2024-07-17T07:12:45.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:12:38.135104+0000', 'modified': '2024-07-17T07:12:41.225421+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7086}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7086': {'gid': 7086, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6833/1121698347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1121698347}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 1121698347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7086, 'qdb_cluster': [7086]} 2024-07-17T07:12:45.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:12:45.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.498+0000 7f0ce3a00640 1 -- 172.21.15.44:0/742853229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 msgr2=0x7f0cc0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.498+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/742853229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cc0003ec0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009b20 tx=0x7f0cc0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.499+0000 7f0ce3a00640 1 -- 172.21.15.44:0/742853229 shutdown_connections 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.499+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/742853229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cc0003ec0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.499+0000 7f0ce3a00640 1 -- 172.21.15.44:0/742853229 >> 172.21.15.44:0/742853229 conn(0x7f0cdc0b0eb0 msgr2=0x7f0cdc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.499+0000 7f0ce3a00640 1 -- 172.21.15.44:0/742853229 shutdown_connections 2024-07-17T07:12:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.499+0000 7f0ce3a00640 1 -- 172.21.15.44:0/742853229 wait complete. 2024-07-17T07:12:45.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.500+0000 7f0ce3a00640 1 Processor -- start 2024-07-17T07:12:45.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.501+0000 7f0ce3a00640 1 -- start start 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 0x7f0cdc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cdc15bc00 0x7f0cdc0992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b3e60 con 0x7f0cc0003aa0 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b3ce0 con 0x7f0cdc1f3ef0 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce3a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc0b3fe0 con 0x7f0cdc15bc00 2024-07-17T07:12:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38906/0 (socket says 172.21.15.44:38906) 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 -- 172.21.15.44:0/1999210984 learned_addr learned my addr 172.21.15.44:0/1999210984 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 -- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cdc15bc00 msgr2=0x7f0cdc0992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cdc15bc00 0x7f0cdc0992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 0x7f0cdc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.502+0000 7f0ce1000640 1 -- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 msgr2=0x7f0cdc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 0x7f0cdc15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0ce1000640 1 -- 172.21.15.44:0/1999210984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc143620 con 0x7f0cc0003aa0 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 0x7f0cdc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:45.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc036040 tx=0x7f0ccc033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0cd9600640 1 -- 172.21.15.44:0/1999210984 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc038040 con 0x7f0cc0003aa0 2024-07-17T07:12:45.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0cd9600640 1 -- 172.21.15.44:0/1999210984 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ccc004020 con 0x7f0cc0003aa0 2024-07-17T07:12:45.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0cd9600640 1 -- 172.21.15.44:0/1999210984 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc004450 con 0x7f0cc0003aa0 2024-07-17T07:12:45.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc1e7290 con 0x7f0cc0003aa0 2024-07-17T07:12:45.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.503+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc143bd0 con 0x7f0cc0003aa0 2024-07-17T07:12:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.506+0000 7f0cd9600640 1 -- 172.21.15.44:0/1999210984 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7f0ccc045020 con 0x7f0cc0003aa0 2024-07-17T07:12:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.507+0000 7f0cd9600640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 0x7f0cb8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.507+0000 7f0cd9600640 1 -- 172.21.15.44:0/1999210984 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f0ccc0b8f00 con 0x7f0cc0003aa0 2024-07-17T07:12:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.507+0000 7f0ce6a03740 1 Processor -- start 2024-07-17T07:12:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.508+0000 7f0ce6a03740 1 -- start start 2024-07-17T07:12:45.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.508+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 0x7f0cb8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321d7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562632103d60 con 0x56263218d1e0 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5626320f3260 con 0x5626321d79e0 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5626320f72a0 con 0x5626321b7da0 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38914/0 (socket says 172.21.15.44:38914) 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce1a00640 1 -- 172.21.15.44:0/3045671304 learned_addr learned my addr 172.21.15.44:0/3045671304 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.509+0000 7f0ce1000640 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 -- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 msgr2=0x56263218f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 -- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 msgr2=0x5626321d7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321d7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321d7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 -- 172.21.15.44:0/3045671304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562631e57850 con 0x56263218d1e0 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 0x7f0cb8079100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0aefe0 tx=0x7f0cc4006ad0 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0ce1a00640 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc071570 tx=0x7f0cd400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaf6f6b0ebdbdec server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0cd1600640 1 -- 172.21.15.44:0/3045671304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017070 con 0x56263218d1e0 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0cd1600640 1 -- 172.21.15.44:0/3045671304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd4003d80 con 0x56263218d1e0 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.510+0000 7f0cd1600640 1 -- 172.21.15.44:0/3045671304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400cb40 con 0x56263218d1e0 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3045671304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 msgr2=0x56263218d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc071570 tx=0x7f0cd400c0e0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3045671304 shutdown_connections 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56263218d1e0 0x56263218d630 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321d7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3045671304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3045671304 >> 172.21.15.44:0/3045671304 conn(0x56263218e290 msgr2=0x562632191cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3045671304 shutdown_connections 2024-07-17T07:12:45.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.511+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3045671304 wait complete. 2024-07-17T07:12:45.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.512+0000 7f0ce6a03740 1 -- start start 2024-07-17T07:12:45.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.512+0000 7f0ce6a03740 1 Processor -- start 2024-07-17T07:12:45.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b3870 0x5626321b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562631e66620 con 0x5626321b7da0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56263209af90 con 0x5626321d79e0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56263218c720 con 0x5626321b3870 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.513+0000 7f0ce6a03740 10 client.0 _ll_register_callbacks cb 0x5626320c0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38916/0 (socket says 172.21.15.44:38916) 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 -- 172.21.15.44:0/1390342018 learned_addr learned my addr 172.21.15.44:0/1390342018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 -- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b3870 msgr2=0x5626321b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1a00640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b3870 0x5626321b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b3870 0x5626321b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 -- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 msgr2=0x5626321a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 -- 172.21.15.44:0/1390342018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562631ece6b0 con 0x5626321b7da0 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0ce1000640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009b20 tx=0x7f0ccc081080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc660395499751 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.514+0000 7f0cca000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc096e60 con 0x5626321b7da0 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f0ccc08a3e0 con 0x5626321b7da0 2024-07-17T07:12:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd4003d80 con 0x5626321b7da0 2024-07-17T07:12:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc081f50 con 0x5626321b7da0 2024-07-17T07:12:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 10 client.7206 fetch_fsmap learned FSMap version 52 2024-07-17T07:12:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5626321b4210 con 0x5626321b7da0 2024-07-17T07:12:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd4003d80 con 0x5626321b7da0 2024-07-17T07:12:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f0ccc03b030 con 0x5626321b7da0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 10 client.7206 fetch_fsmap finished waiting for FSMap version 52 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 10 client.7206 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 20 client.7206 populate_metadata read hostname 'smithi044' 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321cf6f0 0x5626321cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 4 client.7206 mds_command: new command op to 7086 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.515+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 --> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562632105cb0 con 0x5626321cf6f0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.516+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321cf6f0 0x5626321cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.516+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321cf6f0 0x5626321cfad0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c78bc94e368171c server_cookie=4491e3d479c2e5dc in_seq=0 out_seq=0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.516+0000 7f0cca000640 10 client.7206 ms_handle_connect on v2:172.21.15.44:6832/1121698347 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.517+0000 7f0cca000640 1 -- 172.21.15.44:0/1390342018 <== mds.0 v2:172.21.15.44:6832/1121698347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562632105cb0 con 0x5626321cf6f0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.517+0000 7f0cca000640 10 client.7206 handle_command_reply: tid=0 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.517+0000 7f0ce6a03740 1 client.7206 shutdown 2024-07-17T07:12:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.517+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 msgr2=0x5626321a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009b20 tx=0x7f0ccc081080 comp rx=0 tx=0).stop 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 shutdown_connections 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321cf6f0 0x5626321cfad0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b3870 0x5626321b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/1390342018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b7da0 0x5626321a46b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0cca000640 0 client.7206 ms_handle_reset on v2:172.21.15.44:6832/1121698347 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 >> 172.21.15.44:0/1390342018 conn(0x56263218d1e0 msgr2=0x5626321d88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 shutdown_connections 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 1 -- 172.21.15.44:0/1390342018 wait complete. 2024-07-17T07:12:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.518+0000 7f0ce6a03740 20 client.7206 trim_cache size 0 max 16384 2024-07-17T07:12:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.540+0000 7f0ce6a03740 1 Processor -- start 2024-07-17T07:12:45.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.540+0000 7f0ce6a03740 1 -- start start 2024-07-17T07:12:45.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc200dd0 con 0x5626321b3870 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc200130 con 0x5626321d79e0 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cdc200780 con 0x5626321b7da0 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38922/0 (socket says 172.21.15.44:38922) 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 -- 172.21.15.44:0/2444057278 learned_addr learned my addr 172.21.15.44:0/2444057278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 -- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 msgr2=0x56263218b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 -- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 msgr2=0x5626321fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0ce1000640 1 -- 172.21.15.44:0/2444057278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626321b4040 con 0x5626321b3870 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.541+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.542+0000 7f0ce1000640 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc02bfd0 tx=0x7f0ccc005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83556140a7747067 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.542+0000 7f0cd0c00640 1 -- 172.21.15.44:0/2444057278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc096e60 con 0x5626321b3870 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.542+0000 7f0cd0c00640 1 -- 172.21.15.44:0/2444057278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ccc0a31c0 con 0x5626321b3870 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.542+0000 7f0cd0c00640 1 -- 172.21.15.44:0/2444057278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc08a5e0 con 0x5626321b3870 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 -- 172.21.15.44:0/2444057278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 msgr2=0x56263218b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc02bfd0 tx=0x7f0ccc005ab0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 -- 172.21.15.44:0/2444057278 shutdown_connections 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321d79e0 0x5626321fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321b7da0 0x56263218b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/2444057278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x56263218b240 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 -- 172.21.15.44:0/2444057278 >> 172.21.15.44:0/2444057278 conn(0x56263218d1e0 msgr2=0x5626321d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 -- 172.21.15.44:0/2444057278 shutdown_connections 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.543+0000 7f0ce6a03740 1 -- 172.21.15.44:0/2444057278 wait complete. 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.544+0000 7f0ce6a03740 1 -- start start 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.544+0000 7f0ce6a03740 1 Processor -- start 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321b7da0 0x56263224e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321d79e0 0x5626321b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5626321bb5f0 con 0x5626321b3870 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562631e66620 con 0x5626321b7da0 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5626321cebb0 con 0x5626321d79e0 2024-07-17T07:12:45.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce6a03740 10 client.0 _ll_register_callbacks cb 0x7f0ccc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38936/0 (socket says 172.21.15.44:38936) 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.545+0000 7f0ce1000640 1 -- 172.21.15.44:0/3639227487 learned_addr learned my addr 172.21.15.44:0/3639227487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 -- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321d79e0 msgr2=0x5626321b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321d79e0 0x5626321b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 -- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321b7da0 msgr2=0x56263224e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321b7da0 0x56263224e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 -- 172.21.15.44:0/3639227487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562632191470 con 0x5626321b3870 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce1000640 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc07de40 tx=0x7f0ccc07d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e19adb51789030 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0cca000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:12:45.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc096e60 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f0ccc0b3c80 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ccc0a31c0 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc08a870 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce6a03740 10 client.7212 fetch_fsmap learned FSMap version 52 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.546+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x562632191470 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ccc0a31c0 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f0ccc0880b0 con 0x5626321b3870 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 10 client.7212 fetch_fsmap finished waiting for FSMap version 52 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 10 client.7212 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 20 client.7212 populate_metadata read hostname 'smithi044' 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321d8e70 0x5626321b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 4 client.7212 mds_command: new command op to 7086 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 --> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] -- command(tid 0: {"prefix": "status"}) -- 0x562632101750 con 0x5626321d8e70 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.547+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321d8e70 0x5626321b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0cca000640 10 client.7212 ms_handle_connect on v2:172.21.15.44:6832/1121698347 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0cd3e00640 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321d8e70 0x5626321b86c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c48db49756a454 server_cookie=26e38edeafe3d301 in_seq=0 out_seq=0 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0cca000640 1 -- 172.21.15.44:0/3639227487 <== mds.0 v2:172.21.15.44:6832/1121698347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x562632101750 con 0x5626321d8e70 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0cca000640 10 client.7212 handle_command_reply: tid=0 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 client.7212 shutdown 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 msgr2=0x5626321d71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc07de40 tx=0x7f0ccc07d710 comp rx=0 tx=0).stop 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 shutdown_connections 2024-07-17T07:12:45.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:6832/1121698347,v1:172.21.15.44:6833/1121698347] conn(0x5626321d8e70 0x5626321b86c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5626321d79e0 0x5626321b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626321b7da0 0x56263224e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 --2- 172.21.15.44:0/3639227487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626321b3870 0x5626321d71e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0cca000640 0 client.7212 ms_handle_reset on v2:172.21.15.44:6832/1121698347 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 >> 172.21.15.44:0/3639227487 conn(0x56263218d1e0 msgr2=0x5626321d7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.548+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 shutdown_connections 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce6a03740 1 -- 172.21.15.44:0/3639227487 wait complete. 2024-07-17T07:12:45.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce6a03740 20 client.7212 trim_cache size 0 max 16384 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 7086, 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8541854070000001, 2024-07-17T07:12:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 52, 2024-07-17T07:12:45.557 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 112, 2024-07-17T07:12:45.557 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 112, 2024-07-17T07:12:45.557 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.781581211000001 2024-07-17T07:12:45.557 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 msgr2=0x7f0cb8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 0x7f0cb8079100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0aefe0 tx=0x7f0cc4006ad0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 msgr2=0x7f0cdc1f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.549+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc036040 tx=0x7f0ccc033650 comp rx=0 tx=0).stop 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 shutdown_connections 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f0cb8076c40 0x7f0cb8079100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cdc15bc00 0x7f0cdc0992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cdc1f3ef0 0x7f0cdc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 --2- 172.21.15.44:0/1999210984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cc0003aa0 0x7f0cdc1f8520 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 >> 172.21.15.44:0/1999210984 conn(0x7f0cdc0b0eb0 msgr2=0x7f0cdc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 shutdown_connections 2024-07-17T07:12:45.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.550+0000 7f0ce3a00640 1 -- 172.21.15.44:0/1999210984 wait complete. 2024-07-17T07:12:45.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:12:45.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:12:45.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:12:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 -- 172.21.15.44:0/3301772395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc0ac3e0 msgr2=0x7fcafc0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 --2- 172.21.15.44:0/3301772395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc0ac3e0 0x7fcafc0ac870 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcafc070c50 tx=0x7fcae802e970 comp rx=0 tx=0).stop 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 -- 172.21.15.44:0/3301772395 shutdown_connections 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 --2- 172.21.15.44:0/3301772395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc0ac3e0 0x7fcafc0ac870 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 --2- 172.21.15.44:0/3301772395 >> v2:172.21.15.175:3300/0 conn(0x7fcafc06f710 0x7fcafc153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 -- 172.21.15.44:0/3301772395 >> 172.21.15.44:0/3301772395 conn(0x7fcafc0b0fb0 msgr2=0x7fcafc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.778+0000 7fcb04000640 1 -- 172.21.15.44:0/3301772395 shutdown_connections 2024-07-17T07:12:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.779+0000 7fcb04000640 1 -- 172.21.15.44:0/3301772395 wait complete. 2024-07-17T07:12:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.781+0000 7fcb04000640 1 Processor -- start 2024-07-17T07:12:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.781+0000 7fcb04000640 1 -- start start 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06f710 0x7fcafc0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcafc0ac3e0 0x7fcafc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0ae800 con 0x7fcafc13ed60 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc06f0a0 con 0x7fcafc06f710 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb04000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc0ae980 con 0x7fcafc0ac3e0 2024-07-17T07:12:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.782+0000 7fcb02000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38956/0 (socket says 172.21.15.44:38956) 2024-07-17T07:12:45.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 -- 172.21.15.44:0/2682113371 learned_addr learned my addr 172.21.15.44:0/2682113371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb00c00640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcafc0ac3e0 0x7fcafc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb01600640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06f710 0x7fcafc0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 -- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcafc0ac3e0 msgr2=0x7fcafc071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcafc0ac3e0 0x7fcafc071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 -- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06f710 msgr2=0x7fcafc0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06f710 0x7fcafc0b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.783+0000 7fcb02000640 1 -- 172.21.15.44:0/2682113371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1f08e0 con 0x7fcafc13ed60 2024-07-17T07:12:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcb02000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400ca50 tx=0x7fcaf400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4016020 con 0x7fcafc13ed60 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf400aca0 con 0x7fcafc13ed60 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4012680 con 0x7fcafc13ed60 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1f0bc0 con 0x7fcafc13ed60 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.784+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc0b2730 con 0x7fcafc13ed60 2024-07-17T07:12:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.786+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc1f0e00 con 0x7fcafc13ed60 2024-07-17T07:12:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.787+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fcaf4019020 con 0x7fcafc13ed60 2024-07-17T07:12:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.788+0000 7fcaf9600640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 0x7fcadc079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.788+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fcaf4098c60 con 0x7fcafc13ed60 2024-07-17T07:12:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.788+0000 7fcb01600640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 0x7fcadc079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:45.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.789+0000 7fcb01600640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 0x7fcadc079100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fd0 tx=0x7fcaf000d040 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:45.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:45.792+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcafc1f0e00 con 0x7fcafc13ed60 2024-07-17T07:12:45.836 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:45.837+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:46.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.058+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcafc13fa40 con 0x7fcafc13ed60 2024-07-17T07:12:46.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.059+0000 7fcaf9600640 1 -- 172.21.15.44:0/2682113371 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7fcaf4020020 con 0x7fcafc13ed60 2024-07-17T07:12:46.057 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 msgr2=0x7fcadc079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 0x7fcadc079100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fd0 tx=0x7fcaf000d040 comp rx=0 tx=0).stop 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 msgr2=0x7fcafc1eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400ca50 tx=0x7fcaf400cf10 comp rx=0 tx=0).stop 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 shutdown_connections 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fcadc076c40 0x7fcadc079100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:46.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcafc13ed60 0x7fcafc1eff20 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcafc0ac3e0 0x7fcafc071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 --2- 172.21.15.44:0/2682113371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcafc06f710 0x7fcafc0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 >> 172.21.15.44:0/2682113371 conn(0x7fcafc0b0fb0 msgr2=0x7fcafc0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 shutdown_connections 2024-07-17T07:12:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46.062+0000 7fcb04000640 1 -- 172.21.15.44:0/2682113371 wait complete. 2024-07-17T07:12:46.080 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:12:46.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:12:46.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:12:46.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:46.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.081 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:12:46.129 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:46.130 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41492sec preferred_lft 41492sec 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:46.131 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:12:46.132 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:12:46.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.134 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:12:46.202 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:12:46.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.206 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:12:46.252 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:12:46.253 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:12:46.253 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:12:46.253 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:12:46.253 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:12:46.253 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:12:46.253 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:12:46.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:12:46.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:12:46.254 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:12:46.254 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:12:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:12:46.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:12:46.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.465 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:12:46.514 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:12:46.514 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:12:46.514 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:46.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.515 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:12:46.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:12:46.645 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:46.700 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:12:46.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:12:46.700 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:46.750 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:12:46.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.750 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:46.845 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:46.845+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:46.871 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:46.871 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:46.871 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:46.892 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:46.892 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:46.892 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:46.892 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:12:46.892 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:12:46.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:12:46.918 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:12:46.920 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:12:46.920 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:12:46.920 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:12:46.920 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:12:46.920 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:12:46.924 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:12:46.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:12:46.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:12:46.997 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:12:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:12:47.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout:client7224 2024-07-17T07:12:47.132 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:12:47.132 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:12:47.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:12:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 -- 172.21.15.44:0/2004196794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 msgr2=0x7fa91415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 --2- 172.21.15.44:0/2004196794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa91415a450 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa904007350 tx=0x7fa90402b1a0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 -- 172.21.15.44:0/2004196794 shutdown_connections 2024-07-17T07:12:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 --2- 172.21.15.44:0/2004196794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa91415a450 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 -- 172.21.15.44:0/2004196794 >> 172.21.15.44:0/2004196794 conn(0x7fa9140cf0d0 msgr2=0x7fa9140cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 -- 172.21.15.44:0/2004196794 shutdown_connections 2024-07-17T07:12:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.438+0000 7fa913e00640 1 -- 172.21.15.44:0/2004196794 wait complete. 2024-07-17T07:12:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.440+0000 7fa913e00640 1 Processor -- start 2024-07-17T07:12:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.440+0000 7fa913e00640 1 -- start start 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa91406d420 0x7fa9141f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140ae340 0x7fa9140cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140ae900 con 0x7fa9141503c0 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140ac930 con 0x7fa9140ae340 2024-07-17T07:12:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa91406f4d0 con 0x7fa91406d420 2024-07-17T07:12:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38978/0 (socket says 172.21.15.44:38978) 2024-07-17T07:12:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.441+0000 7fa913400640 1 -- 172.21.15.44:0/2015505338 learned_addr learned my addr 172.21.15.44:0/2015505338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa912a00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa91406d420 0x7fa9141f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa90be00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140ae340 0x7fa9140cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 -- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa91406d420 msgr2=0x7fa9141f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa91406d420 0x7fa9141f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 -- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140ae340 msgr2=0x7fa9140cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140ae340 0x7fa9140cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 -- 172.21.15.44:0/2015505338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9141f4d90 con 0x7fa9141503c0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.442+0000 7fa913400640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cbc0 tx=0x7fa90c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.443+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c016020 con 0x7fa9141503c0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.443+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa90c00ae30 con 0x7fa9141503c0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.443+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c012660 con 0x7fa9141503c0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.443+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9140cb990 con 0x7fa9141503c0 2024-07-17T07:12:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.443+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9140cbf50 con 0x7fa9141503c0 2024-07-17T07:12:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.444+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91415a450 con 0x7fa9141503c0 2024-07-17T07:12:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.446+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fa90c019020 con 0x7fa9141503c0 2024-07-17T07:12:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.447+0000 7fa90aa00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 0x7fa8f0079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.447+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fa90c098b30 con 0x7fa9141503c0 2024-07-17T07:12:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.448+0000 7fa912a00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 0x7fa8f0079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.449+0000 7fa912a00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 0x7fa8f0079130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa904034000 tx=0x7fa904004730 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:47.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.452+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa90c020030 con 0x7fa9141503c0 2024-07-17T07:12:47.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.702+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 --> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa9140cffe0 con 0x7fa8f0076c70 2024-07-17T07:12:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.706+0000 7fa90aa00640 1 -- 172.21.15.44:0/2015505338 <== mgr.5923 v2:172.21.15.175:6800/3550495011 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa9140cffe0 con 0x7fa8f0076c70 2024-07-17T07:12:47.705 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:12:47.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:12:47.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:12:47.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:12:47.706 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:12:47.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 msgr2=0x7fa8f0079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 0x7fa8f0079130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa904034000 tx=0x7fa904004730 comp rx=0 tx=0).stop 2024-07-17T07:12:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 msgr2=0x7fa914072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00cbc0 tx=0x7fa90c00a430 comp rx=0 tx=0).stop 2024-07-17T07:12:47.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 shutdown_connections 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fa8f0076c70 0x7fa8f0079130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9141503c0 0x7fa914072ad0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140ae340 0x7fa9140cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 --2- 172.21.15.44:0/2015505338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa91406d420 0x7fa9141f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 >> 172.21.15.44:0/2015505338 conn(0x7fa9140cf0d0 msgr2=0x7fa91415ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.709+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 shutdown_connections 2024-07-17T07:12:47.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.710+0000 7fa913e00640 1 -- 172.21.15.44:0/2015505338 wait complete. 2024-07-17T07:12:47.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:12:47.867 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:47.866+0000 7f446b200640 -1 client.5693: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5881:e822f4da-e731-4332-ae91-f91fc9a99581:v1:172.21.15.175:0/173171416] 2024-07-17T07:12:47.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 -- 172.21.15.44:0/1864641385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3980b2140 msgr2=0x7fe3980cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 --2- 172.21.15.44:0/1864641385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3980b2140 0x7fe3980cefe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe388005fd0 tx=0x7fe38802a5d0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 -- 172.21.15.44:0/1864641385 shutdown_connections 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 --2- 172.21.15.44:0/1864641385 >> v2:172.21.15.175:3300/0 conn(0x7fe3980cf520 0x7fe39806d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 --2- 172.21.15.44:0/1864641385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3980b2140 0x7fe3980cefe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 -- 172.21.15.44:0/1864641385 >> 172.21.15.44:0/1864641385 conn(0x7fe39806c120 msgr2=0x7fe39806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 -- 172.21.15.44:0/1864641385 shutdown_connections 2024-07-17T07:12:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.944+0000 7fe39ee00640 1 -- 172.21.15.44:0/1864641385 wait complete. 2024-07-17T07:12:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.947+0000 7fe39ee00640 1 Processor -- start 2024-07-17T07:12:47.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.947+0000 7fe39ee00640 1 -- start start 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.948+0000 7fe39ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3980b2140 0x7fe3980b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.948+0000 7fe39ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.948+0000 7fe39ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe398156d40 0x7fe3981559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.948+0000 7fe39ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe39806f210 con 0x7fe3980cf520 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe39806f090 con 0x7fe3980b2140 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe39806f5a0 con 0x7fe398156d40 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38990/0 (socket says 172.21.15.44:38990) 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 -- 172.21.15.44:0/4251818670 learned_addr learned my addr 172.21.15.44:0/4251818670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39da00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3980b2140 0x7fe3980b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39e400640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe398156d40 0x7fe3981559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 -- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe398156d40 msgr2=0x7fe3981559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe398156d40 0x7fe3981559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 -- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3980b2140 msgr2=0x7fe3980b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3980b2140 0x7fe3980b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.949+0000 7fe39d000640 1 -- 172.21.15.44:0/4251818670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3980aefe0 con 0x7fe3980cf520 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe39d000640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe388009850 tx=0x7fe3880035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38803f070 con 0x7fe3980cf520 2024-07-17T07:12:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe388003ce0 con 0x7fe3980cf520 2024-07-17T07:12:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388035030 con 0x7fe3980cf520 2024-07-17T07:12:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3980af2c0 con 0x7fe3980cf520 2024-07-17T07:12:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.950+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3980af880 con 0x7fe3980cf520 2024-07-17T07:12:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.952+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3980cefe0 con 0x7fe3980cf520 2024-07-17T07:12:47.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.953+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 47) ==== 98875+0+0 (secure 0 0 0) 0x7fe388042070 con 0x7fe3980cf520 2024-07-17T07:12:47.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.955+0000 7fe395600640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 0x7fe37c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:47.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.955+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fe3880b93d0 con 0x7fe3980cf520 2024-07-17T07:12:47.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.955+0000 7fe39da00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 0x7fe37c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:47.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.957+0000 7fe39da00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 0x7fe37c079130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe38c001660 tx=0x7fe38c009040 comp rx=0 tx=0).ready entity=mgr.5923 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:47.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:47.960+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe388082310 con 0x7fe3980cf520 2024-07-17T07:12:48.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.229+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe3980cb8a0 con 0x7fe3980cf520 2024-07-17T07:12:48.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.767+0000 7fe395600640 1 -- 172.21.15.44:0/4251818670 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) ==== 92+0+0 (secure 0 0 0) 0x7fe388081c10 con 0x7fe3980cf520 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 msgr2=0x7fe37c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 0x7fe37c079130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe38c001660 tx=0x7fe38c009040 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 msgr2=0x7fe3981554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe388009850 tx=0x7fe3880035f0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 shutdown_connections 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fe37c076c70 0x7fe37c079130 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe398156d40 0x7fe3981559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3980cf520 0x7fe3981554b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 --2- 172.21.15.44:0/4251818670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3980b2140 0x7fe3980b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 >> 172.21.15.44:0/4251818670 conn(0x7fe39806c120 msgr2=0x7fe398158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:48.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 shutdown_connections 2024-07-17T07:12:48.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.771+0000 7fe39ee00640 1 -- 172.21.15.44:0/4251818670 wait complete. 2024-07-17T07:12:48.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 -- 172.21.15.44:0/461398112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40ce1d0 msgr2=0x7fd1f40ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 --2- 172.21.15.44:0/461398112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40ce1d0 0x7fd1f40ce5b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4005fd0 tx=0x7fd1e402a5c0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 -- 172.21.15.44:0/461398112 shutdown_connections 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 --2- 172.21.15.44:0/461398112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40b6790 0x7fd1f4161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 --2- 172.21.15.44:0/461398112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40ce1d0 0x7fd1f40ce5b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 -- 172.21.15.44:0/461398112 >> 172.21.15.44:0/461398112 conn(0x7fd1f40d3700 msgr2=0x7fd1f40d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:48.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 -- 172.21.15.44:0/461398112 shutdown_connections 2024-07-17T07:12:48.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.994+0000 7fd1fb800640 1 -- 172.21.15.44:0/461398112 wait complete. 2024-07-17T07:12:48.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.996+0000 7fd1fb800640 1 Processor -- start 2024-07-17T07:12:48.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.996+0000 7fd1fb800640 1 -- start start 2024-07-17T07:12:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f40b29e0 0x7fd1f414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:48.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 0x7fd1f41559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4162420 con 0x7fd1f40b6790 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41622a0 con 0x7fd1f40b29e0 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1fb800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f4167910 con 0x7fd1f40ce1d0 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:48.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47946/0 (socket says 172.21.15.44:47946) 2024-07-17T07:12:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1f3e00640 1 -- 172.21.15.44:0/884540297 learned_addr learned my addr 172.21.15.44:0/884540297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1f9800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 0x7fd1f41559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.997+0000 7fd1f8e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f40b29e0 0x7fd1f414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 msgr2=0x7fd1f41559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:48.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 0x7fd1f41559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f40b29e0 msgr2=0x7fd1f414c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f40b29e0 0x7fd1f414c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 -- 172.21.15.44:0/884540297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41f8430 con 0x7fd1f40b6790 2024-07-17T07:12:48.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f9800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 0x7fd1f41559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f3e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000ea80 tx=0x7fd1e000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000ce40 con 0x7fd1f40b6790 2024-07-17T07:12:49.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1e0011560 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0011970 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f414cdf0 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:48.998+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f414d3b0 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.000+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f40ce5b0 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.001+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 48) ==== 98892+0+0 (secure 0 0 0) 0x7fd1e0011b10 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.002+0000 7fd1f0c00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 0x7fd1d4079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.002+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7fd1e0099560 con 0x7fd1f40b6790 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.002+0000 7fd1f8e00640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 msgr2=0x7fd1d4079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:49.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.002+0000 7fd1f8e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 0x7fd1d4079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:49.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.007+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd1e0016030 con 0x7fd1f40b6790 2024-07-17T07:12:49.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.203+0000 7fd1f8e00640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 msgr2=0x7fd1d4079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:49.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.203+0000 7fd1f8e00640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 0x7fd1d4079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:49.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.221+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd1f414c9f0 con 0x7fd1f40b6790 2024-07-17T07:12:49.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.227+0000 7fd1f0c00640 1 -- 172.21.15.44:0/884540297 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v45)=0 v45) ==== 138+0+0 (secure 0 0 0) 0x7fd1e0061e50 con 0x7fd1f40b6790 2024-07-17T07:12:49.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 msgr2=0x7fd1d4079150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:49.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 0x7fd1d4079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 msgr2=0x7fd1f414d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000ea80 tx=0x7fd1e000ef40 comp rx=0 tx=0).stop 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 shutdown_connections 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7fd1d4076c90 0x7fd1d4079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40ce1d0 0x7fd1f41559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1f40b6790 0x7fd1f414d8d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.230+0000 7fd1fb800640 1 --2- 172.21.15.44:0/884540297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f40b29e0 0x7fd1f414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.231+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 >> 172.21.15.44:0/884540297 conn(0x7fd1f40d3700 msgr2=0x7fd1f40b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:49.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.231+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 shutdown_connections 2024-07-17T07:12:49.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.231+0000 7fd1fb800640 1 -- 172.21.15.44:0/884540297 wait complete. 2024-07-17T07:12:49.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:12:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 -- 172.21.15.44:0/746054547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 msgr2=0x7f4e640cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 --2- 172.21.15.44:0/746054547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e640cfd00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e58005fd0 tx=0x7f4e5802a540 comp rx=0 tx=0).stop 2024-07-17T07:12:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 -- 172.21.15.44:0/746054547 shutdown_connections 2024-07-17T07:12:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 --2- 172.21.15.44:0/746054547 >> v2:172.21.15.175:3300/0 conn(0x7f4e6415ef10 0x7f4e6406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 --2- 172.21.15.44:0/746054547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e640cfd00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 -- 172.21.15.44:0/746054547 >> 172.21.15.44:0/746054547 conn(0x7f4e640b2240 msgr2=0x7f4e640b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:49.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 -- 172.21.15.44:0/746054547 shutdown_connections 2024-07-17T07:12:49.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.451+0000 7f4e6b800640 1 -- 172.21.15.44:0/746054547 wait complete. 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.454+0000 7f4e6b800640 1 Processor -- start 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.454+0000 7f4e6b800640 1 -- start start 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e64148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e6415b840 0x7f4e640b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 0x7f4e641500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6406eea0 con 0x7f4e6415ef10 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6415bec0 con 0x7f4e6415b840 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.455+0000 7f4e6b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6406f020 con 0x7f4e6406bf40 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 0x7f4e641500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e68e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e64148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e68e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e64148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:55386/0 (socket says 172.21.15.44:55386) 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e68e00640 1 -- 172.21.15.44:0/689764290 learned_addr learned my addr 172.21.15.44:0/689764290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e63e00640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e6415b840 0x7f4e640b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 msgr2=0x7f4e64148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e64148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e6415b840 msgr2=0x7f4e640b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e6415b840 0x7f4e640b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.456+0000 7f4e69800640 1 -- 172.21.15.44:0/689764290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e640b4050 con 0x7f4e6415ef10 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e69800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 0x7f4e641500d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c760 tx=0x7f4e5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:49.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003b70 con 0x7f4e6415ef10 2024-07-17T07:12:49.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c003d10 con 0x7f4e6415ef10 2024-07-17T07:12:49.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00aab0 con 0x7f4e6415ef10 2024-07-17T07:12:49.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e640b4330 con 0x7f4e6415ef10 2024-07-17T07:12:49.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.457+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e640b48f0 con 0x7f4e6415ef10 2024-07-17T07:12:49.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.459+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640cfd00 con 0x7f4e6415ef10 2024-07-17T07:12:49.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.461+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 48) ==== 98892+0+0 (secure 0 0 0) 0x7f4e5c00ac50 con 0x7f4e6415ef10 2024-07-17T07:12:49.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.462+0000 7f4e60c00640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 0x7f4e44079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.462+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f4e5c0984c0 con 0x7f4e6415ef10 2024-07-17T07:12:49.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.462+0000 7f4e68e00640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 msgr2=0x7f4e44079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:49.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.462+0000 7f4e68e00640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 0x7f4e44079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:49.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.467+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4e640cf280 con 0x7f4e6415ef10 2024-07-17T07:12:49.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.663+0000 7f4e68e00640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 msgr2=0x7f4e44079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:49.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.663+0000 7f4e68e00640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 0x7f4e44079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.675+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4e6406cd20 con 0x7f4e6415ef10 2024-07-17T07:12:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.680+0000 7f4e60c00640 1 -- 172.21.15.44:0/689764290 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v46)=0 v46) ==== 133+0+0 (secure 0 0 0) 0x7f4e5c060ec0 con 0x7f4e6415ef10 2024-07-17T07:12:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 msgr2=0x7f4e44079150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 0x7f4e44079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 msgr2=0x7f4e641500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 0x7f4e641500d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c760 tx=0x7f4e5c00cc20 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 shutdown_connections 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f4e44076c90 0x7f4e44079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6415ef10 0x7f4e641500d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e6415b840 0x7f4e640b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 --2- 172.21.15.44:0/689764290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e6406bf40 0x7f4e64148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 >> 172.21.15.44:0/689764290 conn(0x7f4e640b2240 msgr2=0x7f4e641e4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 shutdown_connections 2024-07-17T07:12:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.683+0000 7f4e6b800640 1 -- 172.21.15.44:0/689764290 wait complete. 2024-07-17T07:12:49.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2024-07-17T07:12:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 -- 172.21.15.44:0/712574209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8958161be0 msgr2=0x7f895815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 --2- 172.21.15.44:0/712574209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8958161be0 0x7f895815d510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f894002a540 comp rx=0 tx=0).stop 2024-07-17T07:12:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 -- 172.21.15.44:0/712574209 shutdown_connections 2024-07-17T07:12:49.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 --2- 172.21.15.44:0/712574209 >> v2:172.21.15.175:3300/0 conn(0x7f89580ce690 0x7f89580cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 --2- 172.21.15.44:0/712574209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8958161be0 0x7f895815d510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 -- 172.21.15.44:0/712574209 >> 172.21.15.44:0/712574209 conn(0x7f89580b4ff0 msgr2=0x7f89580b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:49.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 -- 172.21.15.44:0/712574209 shutdown_connections 2024-07-17T07:12:49.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.898+0000 7f895ca00640 1 -- 172.21.15.44:0/712574209 wait complete. 2024-07-17T07:12:49.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.900+0000 7f895ca00640 1 Processor -- start 2024-07-17T07:12:49.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.900+0000 7f895ca00640 1 -- start start 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89580ce690 0x7f8958160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 0x7f89581e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8958161be0 0x7f89580c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89580d1a60 con 0x7f895815d130 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89580d18e0 con 0x7f8958161be0 2024-07-17T07:12:49.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f895ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89580d1be0 con 0x7f89580ce690 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 0x7f89581e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 0x7f89581e91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47960/0 (socket says 172.21.15.44:47960) 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 -- 172.21.15.44:0/3400515162 learned_addr learned my addr 172.21.15.44:0/3400515162 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89580ce690 0x7f8958160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8957200640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8958161be0 0x7f89580c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89580ce690 msgr2=0x7f8958160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89580ce690 0x7f8958160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8958161be0 msgr2=0x7f89580c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:49.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8958161be0 0x7f89580c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:49.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8955e00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895815e500 con 0x7f895815d130 2024-07-17T07:12:49.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89580ce690 0x7f8958160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:12:49.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.901+0000 7f8957200640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8958161be0 0x7f89580c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f8955e00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 0x7f89581e91d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8940000c00 tx=0x7f89400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894003f070 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89400047a0 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940035030 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895815e7e0 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.902+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895815eda0 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.904+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89580d2170 con 0x7f895815d130 2024-07-17T07:12:49.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.905+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 48) ==== 98892+0+0 (secure 0 0 0) 0x7f8940042070 con 0x7f895815d130 2024-07-17T07:12:49.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.906+0000 7f894ea00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:49.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.906+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6621+0+0 (secure 0 0 0) 0x7f89400b9990 con 0x7f895815d130 2024-07-17T07:12:49.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.906+0000 7f8956800640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:49.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.906+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:12:49.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:49.914+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8940082960 con 0x7f895815d130 2024-07-17T07:12:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:50.108+0000 7f8956800640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:50.108+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:12:50.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:50.148+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f89580b4680 con 0x7f8934076c90 2024-07-17T07:12:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:50.510+0000 7f8956800640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:50.510+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:12:51.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:51.311+0000 7f8956800640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:51.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:51.311+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:12:52.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:52.914+0000 7f8956800640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/3550495011 2024-07-17T07:12:52.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:52.914+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:12:53.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:53.016+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 49) ==== 98557+0+0 (secure 0 0 0) 0x7f894007e4f0 con 0x7f895815d130 2024-07-17T07:12:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:53.017+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 msgr2=0x7f8934079150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:12:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:53.017+0000 7f894ea00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/3550495011,v1:172.21.15.175:6801/3550495011] conn(0x7f8934076c90 0x7f8934079150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff88bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff88bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff88bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff88bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff88bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff888000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.251 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff888000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.252 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff888000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.252 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff888000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:53.252 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:12:53.252+0000 7ff888000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:12:54.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.028+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 50) ==== 98684+0+0 (secure 0 0 0) 0x7f89400b6cd0 con 0x7f895815d130 2024-07-17T07:12:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.029+0000 7f894ea00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f893407f790 0x7f8934081b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.029+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 --> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f893400c7f0 con 0x7f893407f790 2024-07-17T07:12:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.044+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f893407f790 0x7f8934081b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.052+0000 7f8956800640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f893407f790 0x7f8934081b80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8948007fd0 tx=0x7f8948009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:54.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.222+0000 7f894ea00640 1 -- 172.21.15.44:0/3400515162 <== mgr.5819 v2:172.21.15.175:6800/372663266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f893400c7f0 con 0x7f893407f790 2024-07-17T07:12:54.221 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /bad-path 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f893407f790 msgr2=0x7f8934081b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f893407f790 0x7f8934081b80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8948007fd0 tx=0x7f8948009040 comp rx=0 tx=0).stop 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 msgr2=0x7f89581e91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 --2- 172.21.15.44:0/3400515162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f895815d130 0x7f89581e91d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8940000c00 tx=0x7f89400038a0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f8957200640 1 -- 172.21.15.44:0/3400515162 reap_dead start 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 shutdown_connections 2024-07-17T07:12:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 >> 172.21.15.44:0/3400515162 conn(0x7f89580b4ff0 msgr2=0x7f89580b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 shutdown_connections 2024-07-17T07:12:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.226+0000 7f895ca00640 1 -- 172.21.15.44:0/3400515162 wait complete. 2024-07-17T07:12:54.247 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2024-07-17T07:12:54.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2024-07-17T07:12:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 -- 172.21.15.44:0/446188499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 msgr2=0x7f88c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 --2- 172.21.15.44:0/446188499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88c0004800 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009e20 tx=0x7f88c0005830 comp rx=0 tx=0).stop 2024-07-17T07:12:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e1600640 1 -- 172.21.15.44:0/446188499 reap_dead start 2024-07-17T07:12:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 -- 172.21.15.44:0/446188499 shutdown_connections 2024-07-17T07:12:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 -- 172.21.15.44:0/446188499 >> 172.21.15.44:0/446188499 conn(0x7f88dc0d04c0 msgr2=0x7f88dc0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 -- 172.21.15.44:0/446188499 shutdown_connections 2024-07-17T07:12:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.459+0000 7f88e2000640 1 -- 172.21.15.44:0/446188499 wait complete. 2024-07-17T07:12:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.461+0000 7f88e2000640 1 Processor -- start 2024-07-17T07:12:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.461+0000 7f88e2000640 1 -- start start 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88dc152be0 0x7f88dc0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88dc160220 0x7f88dc152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0ce2d0 con 0x7f88c0004100 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0ce150 con 0x7f88dc160220 2024-07-17T07:12:54.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc0ce450 con 0x7f88dc152be0 2024-07-17T07:12:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47968/0 (socket says 172.21.15.44:47968) 2024-07-17T07:12:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.462+0000 7f88e0c00640 1 -- 172.21.15.44:0/3701712221 learned_addr learned my addr 172.21.15.44:0/3701712221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 -- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88dc152be0 msgr2=0x7f88dc0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88dc152be0 0x7f88dc0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 -- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88dc160220 msgr2=0x7f88dc152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88dc160220 0x7f88dc152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 -- 172.21.15.44:0/3701712221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc0cd630 con 0x7f88c0004100 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e0c00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00c730 tx=0x7f88cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc017070 con 0x7f88c0004100 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88cc005380 con 0x7f88c0004100 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc0cd8f0 con 0x7f88c0004100 2024-07-17T07:12:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00a8a0 con 0x7f88c0004100 2024-07-17T07:12:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.463+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc159af0 con 0x7f88c0004100 2024-07-17T07:12:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.465+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc0d1820 con 0x7f88c0004100 2024-07-17T07:12:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.466+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 50) ==== 98684+0+0 (secure 0 0 0) 0x7f88cc01a070 con 0x7f88c0004100 2024-07-17T07:12:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.467+0000 7f88d8c00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 0x7f88bc078f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:12:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.467+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6922+0+0 (secure 0 0 0) 0x7f88cc097d20 con 0x7f88c0004100 2024-07-17T07:12:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.468+0000 7f88dbe00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 0x7f88bc078f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:12:54.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.469+0000 7f88dbe00640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 0x7f88bc078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f88c8035000 tx=0x7f88c8004ab0 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:12:54.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.471+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f88cc060fa0 con 0x7f88c0004100 2024-07-17T07:12:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.709+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 --> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88dc0b29f0 con 0x7f88bc076a80 2024-07-17T07:12:54.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.727+0000 7f88d8c00640 1 -- 172.21.15.44:0/3701712221 <== mgr.5819 v2:172.21.15.175:6800/372663266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+302 (secure 0 0 0) 0x7f88dc0b29f0 con 0x7f88bc076a80 2024-07-17T07:12:54.726 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:12:54.727 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:12:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-07-17T07:12:54.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 msgr2=0x7f88bc078f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 0x7f88bc078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f88c8035000 tx=0x7f88c8004ab0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 msgr2=0x7f88dc15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00c730 tx=0x7f88cc00cbf0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 shutdown_connections 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f88bc076a80 0x7f88bc078f40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88dc160220 0x7f88dc152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88dc152be0 0x7f88dc0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 --2- 172.21.15.44:0/3701712221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88c0004100 0x7f88dc15a2f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.730+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 >> 172.21.15.44:0/3701712221 conn(0x7f88dc0d04c0 msgr2=0x7f88dc0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.731+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 shutdown_connections 2024-07-17T07:12:54.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:12:54.731+0000 7f88e2000640 1 -- 172.21.15.44:0/3701712221 wait complete. 2024-07-17T07:12:54.752 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:12:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-07-17T07:13:00.244 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:13:00.245+0000 7ff877200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:13:00.244 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:13:00.245+0000 7ff877200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:13:00.244 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:13:00.245+0000 7ff877200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:13:00.244 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:13:00.245+0000 7ff877200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:13:00.245 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:13:00.245+0000 7ff877200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:13:15.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:13:15.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:13:46.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:13:46.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:13:54.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2024-07-17T07:13:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.979+0000 7f4e30c00640 1 -- 172.21.15.44:0/2743378624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c152220 msgr2=0x7f4e2c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.979+0000 7f4e30c00640 1 --2- 172.21.15.44:0/2743378624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c152220 0x7f4e2c06d620 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0b43d0 tx=0x7f4e1c02a4d0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 -- 172.21.15.44:0/2743378624 shutdown_connections 2024-07-17T07:13:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 --2- 172.21.15.44:0/2743378624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c06ed30 0x7f4e2c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 --2- 172.21.15.44:0/2743378624 >> v2:172.21.15.175:3300/0 conn(0x7f4e2c0cb3a0 0x7f4e2c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 --2- 172.21.15.44:0/2743378624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c152220 0x7f4e2c06d620 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 -- 172.21.15.44:0/2743378624 >> 172.21.15.44:0/2743378624 conn(0x7f4e2c0b0eb0 msgr2=0x7f4e2c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 -- 172.21.15.44:0/2743378624 shutdown_connections 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.980+0000 7f4e30c00640 1 -- 172.21.15.44:0/2743378624 wait complete. 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.981+0000 7f4e30c00640 1 Processor -- start 2024-07-17T07:13:54.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.982+0000 7f4e30c00640 1 -- start start 2024-07-17T07:13:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.982+0000 7f4e30c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c06ed30 0x7f4e2c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:54.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e30c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e30c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c152220 0x7f4e2c1507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e30c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c15f1f0 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e30c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c15f070 con 0x7f4e2c06ed30 2024-07-17T07:13:54.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e30c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c15f370 con 0x7f4e2c152220 2024-07-17T07:13:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55976/0 (socket says 172.21.15.44:55976) 2024-07-17T07:13:54.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e23e00640 1 -- 172.21.15.44:0/3391469976 learned_addr learned my addr 172.21.15.44:0/3391469976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e29e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c06ed30 0x7f4e2c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.983+0000 7f4e2a800640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c152220 0x7f4e2c1507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 -- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c152220 msgr2=0x7f4e2c1507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c152220 0x7f4e2c1507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 -- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c06ed30 msgr2=0x7f4e2c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c06ed30 0x7f4e2c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 -- 172.21.15.44:0/3391469976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c0ac230 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e23e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b4a0 tx=0x7f4e1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14014070 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e14010700 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14010b50 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c0adc40 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.984+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c0ac780 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.986+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c0cb810 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.987+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 53) ==== 99062+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.988+0000 7f4e22000640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 0x7f4e08079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:54.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.988+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7f4e14098880 con 0x7f4e2c0cb3a0 2024-07-17T07:13:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.989+0000 7f4e29e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 0x7f4e08079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:54.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.990+0000 7f4e29e00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 0x7f4e08079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c033000 tx=0x7f4e1c004730 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:54.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:54.992+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4e140613f0 con 0x7f4e2c0cb3a0 2024-07-17T07:13:55.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.227+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 --> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e2c0ade70 con 0x7f4e08076e40 2024-07-17T07:13:55.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.244+0000 7f4e22000640 1 -- 172.21.15.44:0/3391469976 <== mgr.5819 v2:172.21.15.175:6800/372663266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f4e2c0ade70 con 0x7f4e08076e40 2024-07-17T07:13:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:13:55.244 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:12:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2024-07-17T07:13:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 msgr2=0x7f4e08079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 0x7f4e08079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c033000 tx=0x7f4e1c004730 comp rx=0 tx=0).stop 2024-07-17T07:13:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 msgr2=0x7f4e2c0ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b4a0 tx=0x7f4e1400b960 comp rx=0 tx=0).stop 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 shutdown_connections 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f4e08076e40 0x7f4e08079300 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c152220 0x7f4e2c1507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e2c0cb3a0 0x7f4e2c0ad700 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 --2- 172.21.15.44:0/3391469976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c06ed30 0x7f4e2c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 >> 172.21.15.44:0/3391469976 conn(0x7f4e2c0b0eb0 msgr2=0x7f4e2c14b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 shutdown_connections 2024-07-17T07:13:55.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.247+0000 7f4e30c00640 1 -- 172.21.15.44:0/3391469976 wait complete. 2024-07-17T07:13:55.265 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:12:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2024-07-17T07:13:55.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2024-07-17T07:13:55.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 -- 172.21.15.44:0/1276761178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac390 msgr2=0x7ff38415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 --2- 172.21.15.44:0/1276761178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac390 0x7ff38415d7c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff374005fd0 tx=0x7ff37402a4a0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 -- 172.21.15.44:0/1276761178 shutdown_connections 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 --2- 172.21.15.44:0/1276761178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840ac770 0x7ff38415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 --2- 172.21.15.44:0/1276761178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac390 0x7ff38415d7c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 -- 172.21.15.44:0/1276761178 >> 172.21.15.44:0/1276761178 conn(0x7ff38406c120 msgr2=0x7ff38406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 -- 172.21.15.44:0/1276761178 shutdown_connections 2024-07-17T07:13:55.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.466+0000 7ff389400640 1 -- 172.21.15.44:0/1276761178 wait complete. 2024-07-17T07:13:55.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.469+0000 7ff389400640 1 Processor -- start 2024-07-17T07:13:55.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.469+0000 7ff389400640 1 -- start start 2024-07-17T07:13:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 0x7ff3840b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840b3cb0 0x7ff38415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff38415b1d0 con 0x7ff3840ac770 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff38415b050 con 0x7ff3840ac390 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.470+0000 7ff389400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff38406d690 con 0x7ff3840b3cb0 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55994/0 (socket says 172.21.15.44:55994) 2024-07-17T07:13:55.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 -- 172.21.15.44:0/2814265455 learned_addr learned my addr 172.21.15.44:0/2814265455 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:55.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 -- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840b3cb0 msgr2=0x7ff38415bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff388a00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840b3cb0 0x7ff38415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:55.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff383e00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 0x7ff3840b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:55.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840b3cb0 0x7ff38415bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 -- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 msgr2=0x7ff3840b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 0x7ff3840b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 -- 172.21.15.44:0/2814265455 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3841f6a80 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff383e00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 0x7ff3840b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.471+0000 7ff37be00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00b420 tx=0x7ff36c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.472+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c011830 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.472+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff36c016070 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.472+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c011d60 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.472+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3841f36d0 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.472+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841f3c30 con 0x7ff3840ac770 2024-07-17T07:13:55.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.473+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3840008d0 con 0x7ff3840ac770 2024-07-17T07:13:55.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.474+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 53) ==== 99062+0+0 (secure 0 0 0) 0x7ff36c01b460 con 0x7ff3840ac770 2024-07-17T07:13:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.475+0000 7ff380c00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 0x7ff360079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.476+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7ff36c098970 con 0x7ff3840ac770 2024-07-17T07:13:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.476+0000 7ff383e00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 0x7ff360079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.477+0000 7ff383e00640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 0x7ff360079330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff374033000 tx=0x7ff374004730 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:55.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.480+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff36c020020 con 0x7ff3840ac770 2024-07-17T07:13:55.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.714+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 --> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) -- 0x7ff38409ce40 con 0x7ff360076e70 2024-07-17T07:13:55.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.897+0000 7ff380c00640 1 -- 172.21.15.44:0/2814265455 <== mgr.5819 v2:172.21.15.175:6800/372663266 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7ff38409ce40 con 0x7ff360076e70 2024-07-17T07:13:55.896 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /bad-path 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 msgr2=0x7ff360079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 0x7ff360079330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff374033000 tx=0x7ff374004730 comp rx=0 tx=0).stop 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 msgr2=0x7ff3841f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00b420 tx=0x7ff36c00b8e0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 shutdown_connections 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7ff360076e70 0x7ff360079330 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3840b3cb0 0x7ff38415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3840ac770 0x7ff3841f6540 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 --2- 172.21.15.44:0/2814265455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3840ac390 0x7ff3840b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:55.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 >> 172.21.15.44:0/2814265455 conn(0x7ff38406c120 msgr2=0x7ff38406f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:55.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 shutdown_connections 2024-07-17T07:13:55.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:55.901+0000 7ff389400640 1 -- 172.21.15.44:0/2814265455 wait complete. 2024-07-17T07:13:55.920 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2024-07-17T07:13:55.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:13:56.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.137+0000 7fd332200640 1 -- 172.21.15.44:0/2317238468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd314004020 msgr2=0x7fd314004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.137+0000 7fd332200640 1 --2- 172.21.15.44:0/2317238468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd314004020 0x7fd314004800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd32c06e790 tx=0x7fd314005830 comp rx=0 tx=0).stop 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.137+0000 7fd331800640 1 -- 172.21.15.44:0/2317238468 reap_dead start 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.137+0000 7fd332200640 1 -- 172.21.15.44:0/2317238468 shutdown_connections 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.137+0000 7fd332200640 1 -- 172.21.15.44:0/2317238468 >> 172.21.15.44:0/2317238468 conn(0x7fd32c06c120 msgr2=0x7fd32c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.138+0000 7fd332200640 1 -- 172.21.15.44:0/2317238468 shutdown_connections 2024-07-17T07:13:56.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.138+0000 7fd332200640 1 -- 172.21.15.44:0/2317238468 wait complete. 2024-07-17T07:13:56.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.139+0000 7fd332200640 1 Processor -- start 2024-07-17T07:13:56.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.140+0000 7fd332200640 1 -- start start 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.140+0000 7fd332200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd314004020 0x7fd32c153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.140+0000 7fd332200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd332200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 0x7fd32c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd332200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0b24d0 con 0x7fd32c14e600 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd332200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0b2350 con 0x7fd32c1f35f0 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd332200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c0ae480 con 0x7fd314004020 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56018/0 (socket says 172.21.15.44:56018) 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 -- 172.21.15.44:0/415417629 learned_addr learned my addr 172.21.15.44:0/415417629 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd331800640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 0x7fd32c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd330e00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd314004020 0x7fd32c153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 -- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd314004020 msgr2=0x7fd32c153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd314004020 0x7fd32c153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 -- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 msgr2=0x7fd32c0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 0x7fd32c0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd32be00640 1 -- 172.21.15.44:0/415417629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c1541b0 con 0x7fd32c14e600 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.141+0000 7fd331800640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 0x7fd32c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd32be00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00aca0 tx=0x7fd31c033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c038040 con 0x7fd32c14e600 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd31c041070 con 0x7fd32c14e600 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c154490 con 0x7fd32c14e600 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c033bf0 con 0x7fd32c14e600 2024-07-17T07:13:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.142+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1f3a80 con 0x7fd32c14e600 2024-07-17T07:13:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.143+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c0aea10 con 0x7fd32c14e600 2024-07-17T07:13:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.144+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 53) ==== 99062+0+0 (secure 0 0 0) 0x7fd31c046460 con 0x7fd32c14e600 2024-07-17T07:13:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.146+0000 7fd328c00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 0x7fd310079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.146+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7fd31c0b93d0 con 0x7fd32c14e600 2024-07-17T07:13:56.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.146+0000 7fd330e00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 0x7fd310079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.148+0000 7fd330e00640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 0x7fd310079330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0ac4c0 tx=0x7fd320001da0 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:56.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.150+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd31c081f40 con 0x7fd32c14e600 2024-07-17T07:13:56.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:56.416+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fd32c154660 con 0x7fd32c14e600 2024-07-17T07:13:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.368+0000 7fd328c00640 1 -- 172.21.15.44:0/415417629 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v54) ==== 93+0+0 (secure 0 0 0) 0x7fd31c08a180 con 0x7fd32c14e600 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 msgr2=0x7fd310079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 0x7fd310079330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0ac4c0 tx=0x7fd320001da0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 msgr2=0x7fd32c074400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00aca0 tx=0x7fd31c033650 comp rx=0 tx=0).stop 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 shutdown_connections 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fd310076e70 0x7fd310079330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd32c1f35f0 0x7fd32c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd32c14e600 0x7fd32c074400 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 --2- 172.21.15.44:0/415417629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd314004020 0x7fd32c153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.373+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 >> 172.21.15.44:0/415417629 conn(0x7fd32c06c120 msgr2=0x7fd32c15a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.374+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 shutdown_connections 2024-07-17T07:13:57.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.374+0000 7fd332200640 1 -- 172.21.15.44:0/415417629 wait complete. 2024-07-17T07:13:57.397 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:13:57.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:13:57.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:13:57.608 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:13:57.608 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:13:57.608 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:13:57.608 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:13:57.662 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:13:57.662 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:13:57.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.958+0000 7fb1a5a00640 1 -- 172.21.15.44:0/432945006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a00ce690 msgr2=0x7fb1a00cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.958+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/432945006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a00ce690 0x7fb1a00cfdf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb198009830 tx=0x7fb19802a480 comp rx=0 tx=0).stop 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 -- 172.21.15.44:0/432945006 shutdown_connections 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/432945006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a00ce690 0x7fb1a00cfdf0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/432945006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a0161be0 0x7fb1a015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 -- 172.21.15.44:0/432945006 >> 172.21.15.44:0/432945006 conn(0x7fb1a00b4ff0 msgr2=0x7fb1a00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:57.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 -- 172.21.15.44:0/432945006 shutdown_connections 2024-07-17T07:13:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.959+0000 7fb1a5a00640 1 -- 172.21.15.44:0/432945006 wait complete. 2024-07-17T07:13:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.961+0000 7fb1a5a00640 1 Processor -- start 2024-07-17T07:13:57.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.961+0000 7fb1a5a00640 1 -- start start 2024-07-17T07:13:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00ce690 0x7fb1a00cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 0x7fb1a01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:57.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00d1a60 con 0x7fb1a0161be0 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00d18e0 con 0x7fb1a015d130 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a00d1be0 con 0x7fb1a00ce690 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56064/0 (socket says 172.21.15.44:56064) 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb1a5000640 1 -- 172.21.15.44:0/2435630515 learned_addr learned my addr 172.21.15.44:0/2435630515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb197400640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 0x7fb1a01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.962+0000 7fb19fe00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00ce690 0x7fb1a00cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00ce690 msgr2=0x7fb1a00cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00ce690 0x7fb1a00cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 msgr2=0x7fb1a01f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 0x7fb1a01f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 -- 172.21.15.44:0/2435630515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a01f48b0 con 0x7fb1a0161be0 2024-07-17T07:13:57.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb197400640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 0x7fb1a01f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:13:57.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5000640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb19802aa70 tx=0x7fb198033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:57.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198005c90 con 0x7fb1a0161be0 2024-07-17T07:13:57.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb198031e50 con 0x7fb1a0161be0 2024-07-17T07:13:57.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198005220 con 0x7fb1a0161be0 2024-07-17T07:13:57.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a01f4b90 con 0x7fb1a0161be0 2024-07-17T07:13:57.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.963+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a015d510 con 0x7fb1a0161be0 2024-07-17T07:13:57.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.965+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a00d2170 con 0x7fb1a0161be0 2024-07-17T07:13:57.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.967+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7fb1980053c0 con 0x7fb1a0161be0 2024-07-17T07:13:57.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.968+0000 7fb19cc00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 0x7fb17c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:57.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.968+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7fb1980b9be0 con 0x7fb1a0161be0 2024-07-17T07:13:57.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.968+0000 7fb19fe00640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 msgr2=0x7fb17c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:57.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.968+0000 7fb19fe00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 0x7fb17c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:13:57.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:57.972+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb19803f030 con 0x7fb1a0161be0 2024-07-17T07:13:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.170+0000 7fb19fe00640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 msgr2=0x7fb17c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.170+0000 7fb19fe00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 0x7fb17c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:13:58.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.196+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1a00b9010 con 0x7fb1a0161be0 2024-07-17T07:13:58.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.197+0000 7fb19cc00640 1 -- 172.21.15.44:0/2435630515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v116) ==== 80+0+477 (secure 0 0 0) 0x7fb1980827b0 con 0x7fb1a0161be0 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/173171416 2024-07-17T08:12:55.098567+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2377117402 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2895451842 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/3550495011 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/3550495011 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1860925725 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1811995408 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1044580353 2024-07-18T07:12:52.999080+0000 2024-07-17T07:13:58.196 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T07:13:58.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 msgr2=0x7fb17c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 0x7fb17c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 msgr2=0x7fb1a01598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb19802aa70 tx=0x7fb198033c20 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 shutdown_connections 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fb17c076df0 0x7fb17c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a0161be0 0x7fb1a01598b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a015d130 0x7fb1a01f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 --2- 172.21.15.44:0/2435630515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a00ce690 0x7fb1a00cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.199+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 >> 172.21.15.44:0/2435630515 conn(0x7fb1a00b4ff0 msgr2=0x7fb1a00b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:58.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.200+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 shutdown_connections 2024-07-17T07:13:58.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.200+0000 7fb1a5a00640 1 -- 172.21.15.44:0/2435630515 wait complete. 2024-07-17T07:13:58.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:13:58.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:13:58.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:13:58.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:13:58.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:13:58.292 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:13:58.330 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:13:58.331 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:13:58.331 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:13:58.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:13:58.375 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:13:58.376 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:13:58.376 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:13:58.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:13:58.418 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:13:58.419 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:13:58.419 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:13:58.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:13:58.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:13:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9461400640 1 -- 172.21.15.44:0/1491474229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 msgr2=0x7f9440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9461400640 1 --2- 172.21.15.44:0/1491474229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f9440004c80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9444009b50 tx=0x7f9440005830 comp rx=0 tx=0).stop 2024-07-17T07:13:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9460a00640 1 -- 172.21.15.44:0/1491474229 reap_dead start 2024-07-17T07:13:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9461400640 1 -- 172.21.15.44:0/1491474229 shutdown_connections 2024-07-17T07:13:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9461400640 1 -- 172.21.15.44:0/1491474229 >> 172.21.15.44:0/1491474229 conn(0x7f945c0b0e80 msgr2=0x7f945c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.671+0000 7f9461400640 1 -- 172.21.15.44:0/1491474229 shutdown_connections 2024-07-17T07:13:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.672+0000 7f9461400640 1 -- 172.21.15.44:0/1491474229 wait complete. 2024-07-17T07:13:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.673+0000 7f9461400640 1 Processor -- start 2024-07-17T07:13:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.673+0000 7f9461400640 1 -- start start 2024-07-17T07:13:58.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c139c00 0x7f945c13c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c13cca0 0x7f945c13d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0ac1e0 con 0x7f9440004800 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0ac060 con 0x7f945c13cca0 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.674+0000 7f9461400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f945c140a00 con 0x7f945c139c00 2024-07-17T07:13:58.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56080/0 (socket says 172.21.15.44:56080) 2024-07-17T07:13:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 -- 172.21.15.44:0/1751276866 learned_addr learned my addr 172.21.15.44:0/1751276866 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945b400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c139c00 0x7f945c13c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f9460a00640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c13cca0 0x7f945c13d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c139c00 msgr2=0x7f945c13c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c139c00 0x7f945c13c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c13cca0 msgr2=0x7f945c13d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c13cca0 0x7f945c13d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.675+0000 7f945be00640 1 -- 172.21.15.44:0/1751276866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c072260 con 0x7f9440004800 2024-07-17T07:13:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f945be00640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f944c00dc10 tx=0x7f944c00b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c003970 con 0x7f9440004800 2024-07-17T07:13:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f944c003c00 con 0x7f9440004800 2024-07-17T07:13:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c072540 con 0x7f9440004800 2024-07-17T07:13:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c006110 con 0x7f9440004800 2024-07-17T07:13:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.676+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c072b00 con 0x7f9440004800 2024-07-17T07:13:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.677+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c140030 con 0x7f9440004800 2024-07-17T07:13:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.678+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7f944c0062b0 con 0x7f9440004800 2024-07-17T07:13:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.679+0000 7f9453e00640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 0x7f94380792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.679+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7f944c099700 con 0x7f9440004800 2024-07-17T07:13:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.679+0000 7f945b400640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 msgr2=0x7f94380792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.679+0000 7f945b400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 0x7f94380792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:13:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.683+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f944c062300 con 0x7f9440004800 2024-07-17T07:13:58.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.880+0000 7f945b400640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 msgr2=0x7f94380792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:58.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.880+0000 7f945b400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 0x7f94380792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:13:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.940+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f945c06c280 con 0x7f9440004800 2024-07-17T07:13:58.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.941+0000 7f9453e00640 1 -- 172.21.15.44:0/1751276866 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7f944c061c00 con 0x7f9440004800 2024-07-17T07:13:58.941 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:13:58.941 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:13:58.941 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:13:58.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 msgr2=0x7f94380792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:13:58.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 0x7f94380792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 msgr2=0x7f945c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f944c00dc10 tx=0x7f944c00b620 comp rx=0 tx=0).stop 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 shutdown_connections 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9438076df0 0x7f94380792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f945c13cca0 0x7f945c13d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f945c139c00 0x7f945c13c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 --2- 172.21.15.44:0/1751276866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9440004800 0x7f945c154f60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.944+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 >> 172.21.15.44:0/1751276866 conn(0x7f945c0b0e80 msgr2=0x7f945c15a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.945+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 shutdown_connections 2024-07-17T07:13:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:58.945+0000 7f9461400640 1 -- 172.21.15.44:0/1751276866 wait complete. 2024-07-17T07:13:58.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 -- 172.21.15.44:0/2892233408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac0acaf0 msgr2=0x7f42ac06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 --2- 172.21.15.44:0/2892233408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac0acaf0 0x7f42ac06e090 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005fd0 tx=0x7f42a402af50 comp rx=0 tx=0).stop 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 -- 172.21.15.44:0/2892233408 shutdown_connections 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 --2- 172.21.15.44:0/2892233408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac0acaf0 0x7f42ac06e090 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 --2- 172.21.15.44:0/2892233408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac153a50 0x7f42ac159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 -- 172.21.15.44:0/2892233408 >> 172.21.15.44:0/2892233408 conn(0x7f42ac0b0fb0 msgr2=0x7f42ac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 -- 172.21.15.44:0/2892233408 shutdown_connections 2024-07-17T07:13:59.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.154+0000 7f42b4200640 1 -- 172.21.15.44:0/2892233408 wait complete. 2024-07-17T07:13:59.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.156+0000 7f42b4200640 1 Processor -- start 2024-07-17T07:13:59.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.156+0000 7f42b4200640 1 -- start start 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac0acaf0 0x7f42ac1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42ac15a040 0x7f42ac158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac06f890 con 0x7f42ac153a50 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac06f710 con 0x7f42ac15a040 2024-07-17T07:13:59.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b4200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42ac06e5d0 con 0x7f42ac0acaf0 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35152/0 (socket says 172.21.15.44:35152) 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 -- 172.21.15.44:0/4244299620 learned_addr learned my addr 172.21.15.44:0/4244299620 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b1800640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac0acaf0 0x7f42ac1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac0acaf0 msgr2=0x7f42ac1f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac0acaf0 0x7f42ac1f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42ac15a040 msgr2=0x7f42ac158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42ac15a040 0x7f42ac158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.157+0000 7f42b0e00640 1 -- 172.21.15.44:0/4244299620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac1f5300 con 0x7f42ac153a50 2024-07-17T07:13:59.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42b0e00640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f429800b3f0 tx=0x7f429800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:59.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298015020 con 0x7f42ac153a50 2024-07-17T07:13:59.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4298011920 con 0x7f42ac153a50 2024-07-17T07:13:59.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298011d70 con 0x7f42ac153a50 2024-07-17T07:13:59.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac15c290 con 0x7f42ac153a50 2024-07-17T07:13:59.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.158+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac15c850 con 0x7f42ac153a50 2024-07-17T07:13:59.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.160+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac0adb70 con 0x7f42ac153a50 2024-07-17T07:13:59.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.160+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7f4298018020 con 0x7f42ac153a50 2024-07-17T07:13:59.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.161+0000 7f42a9600640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 0x7f428c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.161+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7f42980989c0 con 0x7f42ac153a50 2024-07-17T07:13:59.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.161+0000 7f42b1800640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 msgr2=0x7f428c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:59.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.162+0000 7f42b1800640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 0x7f428c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:13:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.165+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f42ac0adb70 con 0x7f42ac153a50 2024-07-17T07:13:59.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.363+0000 7f42b1800640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 msgr2=0x7f428c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.363+0000 7f42b1800640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 0x7f428c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:13:59.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.434+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42ac06e090 con 0x7f42ac153a50 2024-07-17T07:13:59.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.434+0000 7f42a9600640 1 -- 172.21.15.44:0/4244299620 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7f429801f030 con 0x7f42ac153a50 2024-07-17T07:13:59.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:13:59.434 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:13:59.434 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:13:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 msgr2=0x7f428c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:13:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 0x7f428c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 msgr2=0x7f42ac1f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f429800b3f0 tx=0x7f429800b8b0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 shutdown_connections 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f428c076df0 0x7f428c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42ac15a040 0x7f42ac158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42ac153a50 0x7f42ac1f4dc0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 --2- 172.21.15.44:0/4244299620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42ac0acaf0 0x7f42ac1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 >> 172.21.15.44:0/4244299620 conn(0x7f42ac0b0fb0 msgr2=0x7f42ac152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 shutdown_connections 2024-07-17T07:13:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.438+0000 7f42b4200640 1 -- 172.21.15.44:0/4244299620 wait complete. 2024-07-17T07:13:59.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:13:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 -- 172.21.15.44:0/3474498819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda301448b0 msgr2=0x7fda300cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 --2- 172.21.15.44:0/3474498819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda301448b0 0x7fda300cd8e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fda300cea70 tx=0x7fda1c02a4d0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 -- 172.21.15.44:0/3474498819 shutdown_connections 2024-07-17T07:13:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 --2- 172.21.15.44:0/3474498819 >> v2:172.21.15.175:3300/0 conn(0x7fda3015eff0 0x7fda3006f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 --2- 172.21.15.44:0/3474498819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda301448b0 0x7fda300cd8e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 -- 172.21.15.44:0/3474498819 >> 172.21.15.44:0/3474498819 conn(0x7fda30072730 msgr2=0x7fda30072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 -- 172.21.15.44:0/3474498819 shutdown_connections 2024-07-17T07:13:59.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.652+0000 7fda37a00640 1 -- 172.21.15.44:0/3474498819 wait complete. 2024-07-17T07:13:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.655+0000 7fda37a00640 1 Processor -- start 2024-07-17T07:13:59.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.655+0000 7fda37a00640 1 -- start start 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.656+0000 7fda37a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 0x7fda3015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda37a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 0x7fda30141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda37a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda37a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda300700a0 con 0x7fda3015eff0 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda37a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda3006ff20 con 0x7fda301444d0 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda37a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda300b5850 con 0x7fda301448b0 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35170/0 (socket says 172.21.15.44:35170) 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 -- 172.21.15.44:0/2807878219 learned_addr learned my addr 172.21.15.44:0/2807878219 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:13:59.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda2fe00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 0x7fda30141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 msgr2=0x7fda30141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 0x7fda30141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 msgr2=0x7fda3015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35000640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 0x7fda3015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 0x7fda3015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35a00640 1 -- 172.21.15.44:0/2807878219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda30153450 con 0x7fda3015eff0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda2fe00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 0x7fda30141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.657+0000 7fda35000640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 0x7fda3015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda35a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b970 tx=0x7fda2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28019070 con 0x7fda3015eff0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda2800cc50 con 0x7fda3015eff0 2024-07-17T07:13:59.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280146a0 con 0x7fda3015eff0 2024-07-17T07:13:59.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda30153730 con 0x7fda3015eff0 2024-07-17T07:13:59.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.658+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda300cad20 con 0x7fda3015eff0 2024-07-17T07:13:59.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.659+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda300cd8e0 con 0x7fda3015eff0 2024-07-17T07:13:59.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.661+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7fda2801c070 con 0x7fda3015eff0 2024-07-17T07:13:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.662+0000 7fda2cc00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 0x7fda100792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:13:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.662+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7fda28098b80 con 0x7fda3015eff0 2024-07-17T07:13:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.662+0000 7fda35000640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 msgr2=0x7fda100792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.662+0000 7fda35000640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 0x7fda100792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:13:59.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.666+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fda280616f0 con 0x7fda3015eff0 2024-07-17T07:13:59.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.863+0000 7fda35000640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 msgr2=0x7fda100792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:13:59.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.863+0000 7fda35000640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 0x7fda100792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:13:59.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.873+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda300cd1e0 con 0x7fda3015eff0 2024-07-17T07:13:59.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.876+0000 7fda2cc00640 1 -- 172.21.15.44:0/2807878219 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+18371 (secure 0 0 0) 0x7fda28060ff0 con 0x7fda3015eff0 2024-07-17T07:13:59.876 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:13:59.877 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":116,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:13:54.323258+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:36.099092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:37.121054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/173171416":"2024-07-17T08:12:55.098567+0000","172.21.15.175:0/2377117402":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/2895451842":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6801/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6800/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1860925725":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1811995408":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1044580353":"2024-07-18T07:12:52.999080+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:13:59.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.878+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 msgr2=0x7fda100792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:13:59.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.878+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 0x7fda100792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 msgr2=0x7fda301f7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:13:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b970 tx=0x7fda2800be30 comp rx=0 tx=0).stop 2024-07-17T07:13:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 shutdown_connections 2024-07-17T07:13:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fda10076df0 0x7fda100792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda3015eff0 0x7fda301f7850 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda301448b0 0x7fda30141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 --2- 172.21.15.44:0/2807878219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda301444d0 0x7fda3015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:13:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 >> 172.21.15.44:0/2807878219 conn(0x7fda30072730 msgr2=0x7fda300aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:13:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 shutdown_connections 2024-07-17T07:13:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:13:59.879+0000 7fda37a00640 1 -- 172.21.15.44:0/2807878219 wait complete. 2024-07-17T07:13:59.898 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:13:59.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 -- 172.21.15.44:0/3410161090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b8c80 msgr2=0x7f36980b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 --2- 172.21.15.44:0/3410161090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b8c80 0x7f36980b4ec0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3688009900 tx=0x7f368802a610 comp rx=0 tx=0).stop 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 -- 172.21.15.44:0/3410161090 shutdown_connections 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 --2- 172.21.15.44:0/3410161090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f369806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 --2- 172.21.15.44:0/3410161090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b8c80 0x7f36980b4ec0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.092+0000 7f3697e00640 1 -- 172.21.15.44:0/3410161090 >> 172.21.15.44:0/3410161090 conn(0x7f36980cfec0 msgr2=0x7f36980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.093+0000 7f3697e00640 1 -- 172.21.15.44:0/3410161090 shutdown_connections 2024-07-17T07:14:00.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.093+0000 7f3697e00640 1 -- 172.21.15.44:0/3410161090 wait complete. 2024-07-17T07:14:00.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.094+0000 7f3697e00640 1 Processor -- start 2024-07-17T07:14:00.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.094+0000 7f3697e00640 1 -- start start 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36980b8c80 0x7f369814c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f3698148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f369806e780 con 0x7f36980b5400 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f369806e600 con 0x7f36980b8c80 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.095+0000 7f3697e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f369806e900 con 0x7f36980ce5c0 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35182/0 (socket says 172.21.15.44:35182) 2024-07-17T07:14:00.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 -- 172.21.15.44:0/3940111002 learned_addr learned my addr 172.21.15.44:0/3940111002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3697400640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f3698148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696000640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36980b8c80 0x7f369814c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 msgr2=0x7f3698148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f3698148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36980b8c80 msgr2=0x7f369814c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36980b8c80 0x7f369814c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3696a00640 1 -- 172.21.15.44:0/3940111002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3698159610 con 0x7f36980b5400 2024-07-17T07:14:00.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.096+0000 7f3697400640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f3698148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f3696a00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3688000c00 tx=0x7f3688003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368803f070 con 0x7f36980b5400 2024-07-17T07:14:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3688003d80 con 0x7f36980b5400 2024-07-17T07:14:00.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688035030 con 0x7f36980b5400 2024-07-17T07:14:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3698158850 con 0x7f36980b5400 2024-07-17T07:14:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.097+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3698158e10 con 0x7f36980b5400 2024-07-17T07:14:00.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.098+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369815e520 con 0x7f36980b5400 2024-07-17T07:14:00.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.100+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7f3688042070 con 0x7f36980b5400 2024-07-17T07:14:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.101+0000 7f368ea00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 0x7f36780792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.102+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7f36880b98c0 con 0x7f36980b5400 2024-07-17T07:14:00.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.102+0000 7f3696000640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 msgr2=0x7f36780792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:00.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.102+0000 7f3696000640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 0x7f36780792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:00.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.106+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3688082480 con 0x7f36980b5400 2024-07-17T07:14:00.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.303+0000 7f3696000640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 msgr2=0x7f36780792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:00.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.303+0000 7f3696000640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 0x7f36780792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.351+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36980b5d70 con 0x7f36980b5400 2024-07-17T07:14:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.352+0000 7f368ea00640 1 -- 172.21.15.44:0/3940111002 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7f3688081d80 con 0x7f36980b5400 2024-07-17T07:14:00.352 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:00.352 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:14:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 msgr2=0x7f36780792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 0x7f36780792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 msgr2=0x7f36980c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3688000c00 tx=0x7f3688003690 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 shutdown_connections 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f3678076e20 0x7f36780792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36980ce5c0 0x7f3698148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36980b8c80 0x7f369814c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 --2- 172.21.15.44:0/3940111002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36980b5400 0x7f36980c9560 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.355+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 >> 172.21.15.44:0/3940111002 conn(0x7f36980cfec0 msgr2=0x7f3698160610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:00.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.356+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 shutdown_connections 2024-07-17T07:14:00.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.356+0000 7f3697e00640 1 -- 172.21.15.44:0/3940111002 wait complete. 2024-07-17T07:14:00.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:14:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 -- 172.21.15.44:0/1087850282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0ce5c0 msgr2=0x7fba3c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 --2- 172.21.15.44:0/1087850282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0ce5c0 0x7fba3c15d2c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba24009ca0 tx=0x7fba2402aac0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 -- 172.21.15.44:0/1087850282 shutdown_connections 2024-07-17T07:14:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 --2- 172.21.15.44:0/1087850282 >> v2:172.21.15.175:3300/0 conn(0x7fba3c0b9060 0x7fba3c15d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 --2- 172.21.15.44:0/1087850282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0ce5c0 0x7fba3c15d2c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 -- 172.21.15.44:0/1087850282 >> 172.21.15.44:0/1087850282 conn(0x7fba3c0cfec0 msgr2=0x7fba3c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 -- 172.21.15.44:0/1087850282 shutdown_connections 2024-07-17T07:14:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.569+0000 7fba41400640 1 -- 172.21.15.44:0/1087850282 wait complete. 2024-07-17T07:14:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.571+0000 7fba41400640 1 Processor -- start 2024-07-17T07:14:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.571+0000 7fba41400640 1 -- start start 2024-07-17T07:14:00.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 0x7fba3c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0b9060 0x7fba3c0cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15dec0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15dd40 con 0x7fba3c0b8c80 2024-07-17T07:14:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba41400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c15e040 con 0x7fba3c0b9060 2024-07-17T07:14:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35194/0 (socket says 172.21.15.44:35194) 2024-07-17T07:14:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 -- 172.21.15.44:0/1066199705 learned_addr learned my addr 172.21.15.44:0/1066199705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba3be00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 0x7fba3c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0b9060 msgr2=0x7fba3c0cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0b9060 0x7fba3c0cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 msgr2=0x7fba3c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 0x7fba3c0b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba40a00640 1 -- 172.21.15.44:0/1066199705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0b24c0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.572+0000 7fba3be00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 0x7fba3c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba40a00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9a0 tx=0x7fba3400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400ccc0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba3400ce60 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005280 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c14f4f0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.573+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c14fab0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.574+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c15e4a0 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.577+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7fba34005420 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.578+0000 7fba33e00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 0x7fba180792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.578+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7fba34099860 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.578+0000 7fba3be00640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 msgr2=0x7fba180792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.578+0000 7fba3be00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 0x7fba180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:00.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.582+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fba34062390 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.780+0000 7fba3be00640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 msgr2=0x7fba180792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:00.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.780+0000 7fba3be00640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 0x7fba180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.827+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba3c0b5d70 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.829+0000 7fba33e00640 1 -- 172.21.15.44:0/1066199705 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5231 (secure 0 0 0) 0x7fba34061c90 con 0x7fba3c0ce5c0 2024-07-17T07:14:00.828 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:00.828 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":52,"btime":"2024-07-17T07:12:41:715788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:12:38.135104+0000","modified":"2024-07-17T07:12:41.225421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7086":{"gid":7086,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/1121698347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1121698347},{"type":"v1","addr":"172.21.15.44:6833","nonce":1121698347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7086,"qdb_cluster":[7086]},"id":6}]} 2024-07-17T07:14:00.828 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 52 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 msgr2=0x7fba180792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 0x7fba180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 msgr2=0x7fba3c0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fba3400e9a0 tx=0x7fba3400eda0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 shutdown_connections 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fba18076df0 0x7fba180792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba3c0ce5c0 0x7fba3c0cc040 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba3c0b9060 0x7fba3c0cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 --2- 172.21.15.44:0/1066199705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba3c0b8c80 0x7fba3c0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.831+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 >> 172.21.15.44:0/1066199705 conn(0x7fba3c0cfec0 msgr2=0x7fba3c0b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.832+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 shutdown_connections 2024-07-17T07:14:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:00.832+0000 7fba41400640 1 -- 172.21.15.44:0/1066199705 wait complete. 2024-07-17T07:14:00.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:14:01.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 -- 172.21.15.44:0/3750665094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 msgr2=0x7fc0e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 --2- 172.21.15.44:0/3750665094 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc0e8004800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000c710 tx=0x7fc0e8005830 comp rx=0 tx=0).stop 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc103e00640 1 -- 172.21.15.44:0/3750665094 reap_dead start 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 -- 172.21.15.44:0/3750665094 shutdown_connections 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 -- 172.21.15.44:0/3750665094 >> 172.21.15.44:0/3750665094 conn(0x7fc1040d00a0 msgr2=0x7fc1040d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 -- 172.21.15.44:0/3750665094 shutdown_connections 2024-07-17T07:14:01.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.047+0000 7fc10a200640 1 -- 172.21.15.44:0/3750665094 wait complete. 2024-07-17T07:14:01.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.049+0000 7fc10a200640 1 Processor -- start 2024-07-17T07:14:01.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.049+0000 7fc10a200640 1 -- start start 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1041f3f90 0x7fc10414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1040b8bf0 0x7fc1040b9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1041662d0 con 0x7fc0e8004040 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc104166150 con 0x7fc1041f3f90 2024-07-17T07:14:01.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc10a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc104166450 con 0x7fc1040b8bf0 2024-07-17T07:14:01.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc103400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:01.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc103400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35206/0 (socket says 172.21.15.44:35206) 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.050+0000 7fc103400640 1 -- 172.21.15.44:0/3239043855 learned_addr learned my addr 172.21.15.44:0/3239043855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc102a00640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1041f3f90 0x7fc10414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1040b8bf0 msgr2=0x7fc1040b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1040b8bf0 0x7fc1040b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1041f3f90 msgr2=0x7fc10414a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1041f3f90 0x7fc10414a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 -- 172.21.15.44:0/3239043855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10415acd0 con 0x7fc0e8004040 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc103400640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400e5f0 tx=0x7fc0f400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4015020 con 0x7fc0e8004040 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f400ca90 con 0x7fc0e8004040 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011460 con 0x7fc0e8004040 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.051+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10414b050 con 0x7fc0e8004040 2024-07-17T07:14:01.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.052+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1041f44e0 con 0x7fc0e8004040 2024-07-17T07:14:01.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.053+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1040b5200 con 0x7fc0e8004040 2024-07-17T07:14:01.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.054+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7fc0f4018020 con 0x7fc0e8004040 2024-07-17T07:14:01.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.055+0000 7fc0fb400640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 0x7fc0e40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.055+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7fc0f4099380 con 0x7fc0e8004040 2024-07-17T07:14:01.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.055+0000 7fc102a00640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 msgr2=0x7fc0e40792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:01.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.055+0000 7fc102a00640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 0x7fc0e40792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:01.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.059+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc0f401f020 con 0x7fc0e8004040 2024-07-17T07:14:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.257+0000 7fc102a00640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 msgr2=0x7fc0e40792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.257+0000 7fc102a00640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 0x7fc0e40792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.323+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc104154d90 con 0x7fc0e8004040 2024-07-17T07:14:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.326+0000 7fc0fb400640 1 -- 172.21.15.44:0/3239043855 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+18371 (secure 0 0 0) 0x7fc10414b050 con 0x7fc0e8004040 2024-07-17T07:14:01.325 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:01.325 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":116,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:13:54.323258+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:12:36.099092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-07-17T07:12:37.121054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/173171416":"2024-07-17T08:12:55.098567+0000","172.21.15.175:0/2377117402":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/2895451842":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6801/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6800/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1860925725":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1811995408":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1044580353":"2024-07-18T07:12:52.999080+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 msgr2=0x7fc0e40792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 0x7fc0e40792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 msgr2=0x7fc1041f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400e5f0 tx=0x7fc0f400eab0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 shutdown_connections 2024-07-17T07:14:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7fc0e4076df0 0x7fc0e40792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1040b8bf0 0x7fc1040b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1041f3f90 0x7fc10414a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 --2- 172.21.15.44:0/3239043855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0e8004040 0x7fc1041f7f20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 >> 172.21.15.44:0/3239043855 conn(0x7fc1040d00a0 msgr2=0x7fc10406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 shutdown_connections 2024-07-17T07:14:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.329+0000 7fc10a200640 1 -- 172.21.15.44:0/3239043855 wait complete. 2024-07-17T07:14:01.350 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 -- 172.21.15.44:0/3637770634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c500051c0 msgr2=0x7f9c5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 --2- 172.21.15.44:0/3637770634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c500051c0 0x7f9c5000b9c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c730 tx=0x7f9c50005830 comp rx=0 tx=0).stop 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c6b400640 1 -- 172.21.15.44:0/3637770634 reap_dead start 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 -- 172.21.15.44:0/3637770634 shutdown_connections 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 -- 172.21.15.44:0/3637770634 >> 172.21.15.44:0/3637770634 conn(0x7f9c6c06d050 msgr2=0x7f9c6c06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 -- 172.21.15.44:0/3637770634 shutdown_connections 2024-07-17T07:14:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.548+0000 7f9c71a00640 1 -- 172.21.15.44:0/3637770634 wait complete. 2024-07-17T07:14:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.550+0000 7f9c71a00640 1 Processor -- start 2024-07-17T07:14:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.550+0000 7f9c71a00640 1 -- start start 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c500051c0 0x7f9c6c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 0x7f9c6c1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c15f840 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c15f6c0 con 0x7f9c500051c0 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c71a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c06c480 con 0x7f9c6c0aefd0 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35212/0 (socket says 172.21.15.44:35212) 2024-07-17T07:14:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c6a000640 1 -- 172.21.15.44:0/1949778642 learned_addr learned my addr 172.21.15.44:0/1949778642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c6aa00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c500051c0 0x7f9c6c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.551+0000 7f9c6b400640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 0x7f9c6c1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 -- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 msgr2=0x7f9c6c1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 0x7f9c6c1f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 -- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c500051c0 msgr2=0x7f9c6c13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c500051c0 0x7f9c6c13d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 -- 172.21.15.44:0/1949778642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c0b4080 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6b400640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 0x7f9c6c1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c6a000640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800e960 tx=0x7f9c5800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800cc60 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c58004020 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c0b4360 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58004470 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.552+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c0ae350 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.554+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c06bf40 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.555+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 54) ==== 99045+0+0 (secure 0 0 0) 0x7f9c580035f0 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.556+0000 7f9c62a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 0x7f9c4c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.556+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7081+0+0 (secure 0 0 0) 0x7f9c58099680 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.556+0000 7f9c6aa00640 1 -- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 msgr2=0x7f9c4c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/372663266 2024-07-17T07:14:01.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.556+0000 7f9c6aa00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 0x7f9c4c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:01.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.560+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9c58016030 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.606+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 55) ==== 98540+0+0 (secure 0 0 0) 0x7f9c5805ddc0 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.606+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 msgr2=0x7f9c4c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:01.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.606+0000 7f9c62a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 0x7f9c4c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.690+0000 7f75f7c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.690+0000 7f75f7c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.690+0000 7f75f7c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.690+0000 7f75f7c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.690+0000 7f75f7c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.691+0000 7f75fb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.691+0000 7f75fb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.690 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.691+0000 7f75fb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.691 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.691+0000 7f75fb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.691 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:01.691+0000 7f75fb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:01.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.821+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c6c06ca70 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.821 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:14:01.822+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:14:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.825+0000 7f9c62a00640 1 -- 172.21.15.44:0/1949778642 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7f9c58062230 con 0x7f9c6c0aeb50 2024-07-17T07:14:01.824 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:14:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.829+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 msgr2=0x7f9c6c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.829+0000 7f9c71a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800e960 tx=0x7f9c5800ee20 comp rx=0 tx=0).stop 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 shutdown_connections 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:6800/372663266,v1:172.21.15.175:6801/372663266] conn(0x7f9c4c076df0 0x7f9c4c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c6c0aefd0 0x7f9c6c1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c6c0aeb50 0x7f9c6c0b4de0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 --2- 172.21.15.44:0/1949778642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c500051c0 0x7f9c6c13d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 >> 172.21.15.44:0/1949778642 conn(0x7f9c6c06d050 msgr2=0x7f9c6c15a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 shutdown_connections 2024-07-17T07:14:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:01.830+0000 7f9c71a00640 1 -- 172.21.15.44:0/1949778642 wait complete. 2024-07-17T07:14:01.847 INFO:tasks.ceph.mds.a.smithi044.stderr: -8072> 2024-07-17T07:12:44.620+0000 7f7de6000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:14:01.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:14:01.923 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 -- 172.21.15.44:0/3341298022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080ce1d0 msgr2=0x7f82080ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 --2- 172.21.15.44:0/3341298022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080ce1d0 0x7f82080ce5b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f820808c1e0 tx=0x7f81f802a680 comp rx=0 tx=0).stop 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 -- 172.21.15.44:0/3341298022 shutdown_connections 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 --2- 172.21.15.44:0/3341298022 >> v2:172.21.15.175:3300/0 conn(0x7f82080b29e0 0x7f8208161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 --2- 172.21.15.44:0/3341298022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080ce1d0 0x7f82080ce5b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 -- 172.21.15.44:0/3341298022 >> 172.21.15.44:0/3341298022 conn(0x7f82080d3700 msgr2=0x7f82080d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.040+0000 7f820d600640 1 -- 172.21.15.44:0/3341298022 shutdown_connections 2024-07-17T07:14:02.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.041+0000 7f820d600640 1 -- 172.21.15.44:0/3341298022 wait complete. 2024-07-17T07:14:02.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.043+0000 7f820d600640 1 Processor -- start 2024-07-17T07:14:02.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.043+0000 7f820d600640 1 -- start start 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080b29e0 0x7f82080c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82080ce1d0 0x7f82081f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8208162420 con 0x7f82080b6790 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82081622a0 con 0x7f82080b29e0 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f820d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8208167910 con 0x7f82080ce1d0 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f81ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f81ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35256/0 (socket says 172.21.15.44:35256) 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f81ffe00640 1 -- 172.21.15.44:0/340330144 learned_addr learned my addr 172.21.15.44:0/340330144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.044+0000 7f8206a00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080b29e0 0x7f82080c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 -- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82080ce1d0 msgr2=0x7f82081f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:02.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f8207400640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82080ce1d0 0x7f82081f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:02.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82080ce1d0 0x7f82081f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 -- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080b29e0 msgr2=0x7f82080c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:02.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080b29e0 0x7f82080c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 -- 172.21.15.44:0/340330144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081f7320 con 0x7f82080b6790 2024-07-17T07:14:02.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81ffe00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c530 tx=0x7f81f000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0017070 con 0x7f82080b6790 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81f00035f0 con 0x7f82080b6790 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82080c96d0 con 0x7f82080b6790 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.045+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0003a40 con 0x7f82080b6790 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.046+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82080c9c50 con 0x7f82080b6790 2024-07-17T07:14:02.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.047+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080ce5b0 con 0x7f82080b6790 2024-07-17T07:14:02.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.048+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 55) ==== 98540+0+0 (secure 0 0 0) 0x7f81f001a070 con 0x7f82080b6790 2024-07-17T07:14:02.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.050+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7597+0+0 (secure 0 0 0) 0x7f81f0097bd0 con 0x7f82080b6790 2024-07-17T07:14:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.054+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f81f0060c20 con 0x7f82080b6790 2024-07-17T07:14:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.305+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f82080ca540 con 0x7f82080b6790 2024-07-17T07:14:02.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.618+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 56) ==== 98667+0+0 (secure 0 0 0) 0x7f81f005be90 con 0x7f82080b6790 2024-07-17T07:14:02.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.619+0000 7f81fea00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 0x7f81e407f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:02.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.629+0000 7f8206a00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 0x7f81e407f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.639+0000 7f8206a00640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 0x7f81e407f650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f8004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:02.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.828+0000 7f81fea00640 1 -- 172.21.15.44:0/340330144 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f81f00192c0 con 0x7f82080b6790 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 msgr2=0x7f81e407f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 0x7f81e407f650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f8004730 comp rx=0 tx=0).stop 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 msgr2=0x7f82081f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c530 tx=0x7f81f000c9f0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 shutdown_connections 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f81e407d260 0x7f81e407f650 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82080ce1d0 0x7f82081f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82080b6790 0x7f82081f6de0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 --2- 172.21.15.44:0/340330144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080b29e0 0x7f82080c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 >> 172.21.15.44:0/340330144 conn(0x7f82080d3700 msgr2=0x7f82080b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 shutdown_connections 2024-07-17T07:14:02.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:02.832+0000 7f820d600640 1 -- 172.21.15.44:0/340330144 wait complete. 2024-07-17T07:14:02.853 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.068+0000 7fe92ee00640 1 -- 172.21.15.44:0/1642970974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928155860 msgr2=0x7fe9280ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.068+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1642970974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928155860 0x7fe9280ad250 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe9280b2050 tx=0x7fe91c02a620 comp rx=0 tx=0).stop 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 -- 172.21.15.44:0/1642970974 shutdown_connections 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1642970974 >> v2:172.21.15.175:3300/0 conn(0x7fe9280ae490 0x7fe9280729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1642970974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928155860 0x7fe9280ad250 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 -- 172.21.15.44:0/1642970974 >> 172.21.15.44:0/1642970974 conn(0x7fe9280cf050 msgr2=0x7fe9280cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 -- 172.21.15.44:0/1642970974 shutdown_connections 2024-07-17T07:14:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.069+0000 7fe92ee00640 1 -- 172.21.15.44:0/1642970974 wait complete. 2024-07-17T07:14:03.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.070+0000 7fe92ee00640 1 Processor -- start 2024-07-17T07:14:03.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.071+0000 7fe92ee00640 1 -- start start 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9280ae490 0x7fe9280af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe928155860 0x7fe9280af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe928073280 con 0x7fe9280ad790 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe928073100 con 0x7fe928155860 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe928073400 con 0x7fe9280ae490 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35266/0 (socket says 172.21.15.44:35266) 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 -- 172.21.15.44:0/1358767945 learned_addr learned my addr 172.21.15.44:0/1358767945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:03.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92e400640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe928155860 0x7fe9280af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 -- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9280ae490 msgr2=0x7fe9280af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92d000640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9280ae490 0x7fe9280af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9280ae490 0x7fe9280af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 -- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe928155860 msgr2=0x7fe9280af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe928155860 0x7fe9280af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.072+0000 7fe92da00640 1 -- 172.21.15.44:0/1358767945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92809c7d0 con 0x7fe9280ad790 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe92da00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099c0 tx=0x7fe91c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c03e030 con 0x7fe9280ad790 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe91c0047a0 con 0x7fe9280ad790 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c035040 con 0x7fe9280ad790 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92809cab0 con 0x7fe9280ad790 2024-07-17T07:14:03.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.073+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92809d070 con 0x7fe9280ad790 2024-07-17T07:14:03.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.074+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9280729b0 con 0x7fe9280ad790 2024-07-17T07:14:03.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.075+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 56) ==== 98667+0+0 (secure 0 0 0) 0x7fe91c041030 con 0x7fe9280ad790 2024-07-17T07:14:03.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.077+0000 7fe925600640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 0x7fe908078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:03.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.077+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7597+0+0 (secure 0 0 0) 0x7fe91c0b8e00 con 0x7fe9280ad790 2024-07-17T07:14:03.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.078+0000 7fe92d000640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 0x7fe908078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:03.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.079+0000 7fe92d000640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 0x7fe908078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe92809d310 tx=0x7fe914009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:03.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.082+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe91c081de0 con 0x7fe9280ad790 2024-07-17T07:14:03.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.292+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe928154ec0 con 0x7fe9280ad790 2024-07-17T07:14:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.832+0000 7fe925600640 1 -- 172.21.15.44:0/1358767945 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) ==== 181+0+0 (secure 0 0 0) 0x7fe91c0816e0 con 0x7fe9280ad790 2024-07-17T07:14:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:14:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.835+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 msgr2=0x7fe908078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:03.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.835+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 0x7fe908078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe92809d310 tx=0x7fe914009040 comp rx=0 tx=0).stop 2024-07-17T07:14:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 msgr2=0x7fe928099fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:03.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099c0 tx=0x7fe91c0039e0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 shutdown_connections 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe908076a00 0x7fe908078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe928155860 0x7fe9280af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9280ae490 0x7fe9280af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 --2- 172.21.15.44:0/1358767945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9280ad790 0x7fe928099fd0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.836+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 >> 172.21.15.44:0/1358767945 conn(0x7fe9280cf050 msgr2=0x7fe92806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.837+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 shutdown_connections 2024-07-17T07:14:03.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:03.837+0000 7fe92ee00640 1 -- 172.21.15.44:0/1358767945 wait complete. 2024-07-17T07:14:03.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:14:04.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 -- 172.21.15.44:0/1699873856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f415a5d0 msgr2=0x7f85f41539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 --2- 172.21.15.44:0/1699873856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f415a5d0 0x7f85f41539f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85dc005fd0 tx=0x7f85dc02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 -- 172.21.15.44:0/1699873856 shutdown_connections 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 --2- 172.21.15.44:0/1699873856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f40b21b0 0x7f85f40b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 --2- 172.21.15.44:0/1699873856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f415a5d0 0x7f85f41539f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 -- 172.21.15.44:0/1699873856 >> 172.21.15.44:0/1699873856 conn(0x7f85f40b0eb0 msgr2=0x7f85f40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.067+0000 7f85fa600640 1 -- 172.21.15.44:0/1699873856 shutdown_connections 2024-07-17T07:14:04.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.068+0000 7f85fa600640 1 -- 172.21.15.44:0/1699873856 wait complete. 2024-07-17T07:14:04.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.069+0000 7f85fa600640 1 Processor -- start 2024-07-17T07:14:04.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.070+0000 7f85fa600640 1 -- start start 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.070+0000 7f85fa600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.070+0000 7f85fa600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f40b21b0 0x7f85f40715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85fa600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 0x7f85f41f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85fa600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40b30e0 con 0x7f85f406f6d0 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85fa600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40b2f60 con 0x7f85f40b21b0 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85fa600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40b3260 con 0x7f85f415a5d0 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35272/0 (socket says 172.21.15.44:35272) 2024-07-17T07:14:04.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 -- 172.21.15.44:0/4042893915 learned_addr learned my addr 172.21.15.44:0/4042893915 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:04.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85ebe00640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f40b21b0 0x7f85f40715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3e00640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 0x7f85f41f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 -- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 msgr2=0x7f85f41f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 0x7f85f41f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 -- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f40b21b0 msgr2=0x7f85f40715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f40b21b0 0x7f85f40715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3400640 1 -- 172.21.15.44:0/4042893915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f41f4c50 con 0x7f85f406f6d0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.071+0000 7f85f3e00640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 0x7f85f41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85f3400640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c420 tx=0x7f85e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40052d0 con 0x7f85f406f6d0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f41f4f30 con 0x7f85f406f6d0 2024-07-17T07:14:04.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85e4007020 con 0x7f85f406f6d0 2024-07-17T07:14:04.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4007470 con 0x7f85f406f6d0 2024-07-17T07:14:04.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.072+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f40ccc10 con 0x7f85f406f6d0 2024-07-17T07:14:04.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.074+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f40b2620 con 0x7f85f406f6d0 2024-07-17T07:14:04.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.075+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 98875+0+0 (secure 0 0 0) 0x7f85e40067d0 con 0x7f85f406f6d0 2024-07-17T07:14:04.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.075+0000 7f85eb400640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 0x7f85d0079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:04.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.076+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 7159+0+0 (secure 0 0 0) 0x7f85e4097f30 con 0x7f85f406f6d0 2024-07-17T07:14:04.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.076+0000 7f85ebe00640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 0x7f85d0079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:04.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.077+0000 7f85ebe00640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 0x7f85d0079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f85dc033000 tx=0x7f85dc004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:04.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.081+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f85e4060d70 con 0x7f85f406f6d0 2024-07-17T07:14:04.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.293+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85f4159000 con 0x7f85f406f6d0 2024-07-17T07:14:04.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.852+0000 7f85eb400640 1 -- 172.21.15.44:0/4042893915 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) ==== 169+0+0 (secure 0 0 0) 0x7f85e4060670 con 0x7f85f406f6d0 2024-07-17T07:14:04.852 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:14:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 msgr2=0x7f85d0079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 0x7f85d0079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f85dc033000 tx=0x7f85dc004730 comp rx=0 tx=0).stop 2024-07-17T07:14:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 msgr2=0x7f85f41527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:04.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c420 tx=0x7f85e400c8e0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 shutdown_connections 2024-07-17T07:14:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f85d0076c40 0x7f85d0079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f415a5d0 0x7f85f41f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f40b21b0 0x7f85f40715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 --2- 172.21.15.44:0/4042893915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f406f6d0 0x7f85f41527d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.856+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 >> 172.21.15.44:0/4042893915 conn(0x7f85f40b0eb0 msgr2=0x7f85f415b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.858+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 shutdown_connections 2024-07-17T07:14:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:04.858+0000 7f85fa600640 1 -- 172.21.15.44:0/4042893915 wait complete. 2024-07-17T07:14:04.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:14:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 -- 172.21.15.44:0/2972193333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4540b2eb0 msgr2=0x7fd4540b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 --2- 172.21.15.44:0/2972193333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4540b2eb0 0x7fd4540b3340 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005fd0 tx=0x7fd44c02a9f0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 -- 172.21.15.44:0/2972193333 shutdown_connections 2024-07-17T07:14:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 --2- 172.21.15.44:0/2972193333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4540b2eb0 0x7fd4540b3340 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 --2- 172.21.15.44:0/2972193333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45406f4d0 0x7fd45415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 -- 172.21.15.44:0/2972193333 >> 172.21.15.44:0/2972193333 conn(0x7fd4540cf0d0 msgr2=0x7fd4540cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 -- 172.21.15.44:0/2972193333 shutdown_connections 2024-07-17T07:14:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.084+0000 7fd459e00640 1 -- 172.21.15.44:0/2972193333 wait complete. 2024-07-17T07:14:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.086+0000 7fd459e00640 1 Processor -- start 2024-07-17T07:14:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.086+0000 7fd459e00640 1 -- start start 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 0x7fd454073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4541503c0 0x7fd45414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd45406da80 con 0x7fd45406f4d0 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd45406d900 con 0x7fd4540b2eb0 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd459e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd45406dc00 con 0x7fd4541503c0 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35286/0 (socket says 172.21.15.44:35286) 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 -- 172.21.15.44:0/3213030760 learned_addr learned my addr 172.21.15.44:0/3213030760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 -- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4541503c0 msgr2=0x7fd45414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4541503c0 0x7fd45414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 -- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 msgr2=0x7fd454073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd453e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 0x7fd454073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 0x7fd454073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd458a00640 1 -- 172.21.15.44:0/3213030760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4541f5c50 con 0x7fd45406f4d0 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.087+0000 7fd453e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 0x7fd454073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd458a00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d770 tx=0x7fd44800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448017070 con 0x7fd45406f4d0 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd448009d20 con 0x7fd45406f4d0 2024-07-17T07:14:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448005280 con 0x7fd45406f4d0 2024-07-17T07:14:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd454073460 con 0x7fd45406f4d0 2024-07-17T07:14:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.088+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd454073a20 con 0x7fd45406f4d0 2024-07-17T07:14:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.090+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd454073290 con 0x7fd45406f4d0 2024-07-17T07:14:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.091+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 98875+0+0 (secure 0 0 0) 0x7fd448005420 con 0x7fd45406f4d0 2024-07-17T07:14:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.092+0000 7fd450c00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 0x7fd434079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.092+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7fd448098850 con 0x7fd45406f4d0 2024-07-17T07:14:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.093+0000 7fd453e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 0x7fd434079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.094+0000 7fd453e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 0x7fd434079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd454150ad0 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:05.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.097+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd448061910 con 0x7fd45406f4d0 2024-07-17T07:14:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.324+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fd45415a450 con 0x7fd45406f4d0 2024-07-17T07:14:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.336+0000 7fd450c00640 1 -- 172.21.15.44:0/3213030760 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7fd448061210 con 0x7fd45406f4d0 2024-07-17T07:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 msgr2=0x7fd434079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 0x7fd434079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd454150ad0 tx=0x7fd440008040 comp rx=0 tx=0).stop 2024-07-17T07:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 msgr2=0x7fd4540987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d770 tx=0x7fd44800dc30 comp rx=0 tx=0).stop 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 shutdown_connections 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.339+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd434076c40 0x7fd434079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4541503c0 0x7fd45414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4540b2eb0 0x7fd454073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 --2- 172.21.15.44:0/3213030760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45406f4d0 0x7fd4540987b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 >> 172.21.15.44:0/3213030760 conn(0x7fd4540cf0d0 msgr2=0x7fd4540ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 shutdown_connections 2024-07-17T07:14:05.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.340+0000 7fd459e00640 1 -- 172.21.15.44:0/3213030760 wait complete. 2024-07-17T07:14:05.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2024-07-17T07:14:05.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 -- 172.21.15.44:0/1537222816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c8153a50 msgr2=0x7f49c8159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 --2- 172.21.15.44:0/1537222816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c8153a50 0x7f49c8159b00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b802b980 comp rx=0 tx=0).stop 2024-07-17T07:14:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 -- 172.21.15.44:0/1537222816 shutdown_connections 2024-07-17T07:14:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 --2- 172.21.15.44:0/1537222816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815a040 0x7f49c80ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 --2- 172.21.15.44:0/1537222816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c8153a50 0x7f49c8159b00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.560+0000 7f49cfc00640 1 -- 172.21.15.44:0/1537222816 >> 172.21.15.44:0/1537222816 conn(0x7f49c80b0fb0 msgr2=0x7f49c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.561+0000 7f49cfc00640 1 -- 172.21.15.44:0/1537222816 shutdown_connections 2024-07-17T07:14:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.561+0000 7f49cfc00640 1 -- 172.21.15.44:0/1537222816 wait complete. 2024-07-17T07:14:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.562+0000 7f49cfc00640 1 Processor -- start 2024-07-17T07:14:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.563+0000 7f49cfc00640 1 -- start start 2024-07-17T07:14:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c8153a50 0x7f49c81f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815a040 0x7f49c8150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8153fb0 con 0x7f49c80ac920 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8153e30 con 0x7f49c8153a50 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cfc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49c806f710 con 0x7f49c815a040 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35312/0 (socket says 172.21.15.44:35312) 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cd200640 1 -- 172.21.15.44:0/3464284246 learned_addr learned my addr 172.21.15.44:0/3464284246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cd200640 1 -- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815a040 msgr2=0x7f49c8150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.564+0000 7f49cd200640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815a040 0x7f49c8150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cd200640 1 -- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c8153a50 msgr2=0x7f49c81f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c8153a50 0x7f49c81f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cd200640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c8153a50 0x7f49c81f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cd200640 1 -- 172.21.15.44:0/3464284246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c81f4e60 con 0x7f49c80ac920 2024-07-17T07:14:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cd200640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f80 tx=0x7f49b8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8035040 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49b80043c0 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80047f0 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c8152d30 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.565+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c81f4120 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.567+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c806a1a0 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.568+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 57) ==== 98875+0+0 (secure 0 0 0) 0x7f49b8038020 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.569+0000 7f49c4c00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 0x7f49a4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.570+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f49b80ba050 con 0x7f49c80ac920 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.570+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 0x7f49a4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.571+0000 7f49c7e00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 0x7f49a4079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49c814fd40 tx=0x7f49b0005c60 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.574+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f49c806a1a0 con 0x7f49c80ac920 2024-07-17T07:14:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:05.863+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) -- 0x7f49c8159b00 con 0x7f49c80ac920 2024-07-17T07:14:06.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.355+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f49b807dfa0 con 0x7f49c80ac920 2024-07-17T07:14:06.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.858+0000 7f49c4c00640 1 -- 172.21.15.44:0/3464284246 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v726) ==== 183+0+0 (secure 0 0 0) 0x7f49c81f4e60 con 0x7f49c80ac920 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 msgr2=0x7f49a4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 0x7f49a4079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49c814fd40 tx=0x7f49b0005c60 comp rx=0 tx=0).stop 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 msgr2=0x7f49c81f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f80 tx=0x7f49b8003f70 comp rx=0 tx=0).stop 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 shutdown_connections 2024-07-17T07:14:06.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f49a4076c40 0x7f49a4079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815a040 0x7f49c8150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c8153a50 0x7f49c81f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 --2- 172.21.15.44:0/3464284246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49c80ac920 0x7f49c81f4920 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 >> 172.21.15.44:0/3464284246 conn(0x7f49c80b0fb0 msgr2=0x7f49c815c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.862+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 shutdown_connections 2024-07-17T07:14:06.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:06.863+0000 7f49cfc00640 1 -- 172.21.15.44:0/3464284246 wait complete. 2024-07-17T07:14:06.884 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2024-07-17T07:14:06.884 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2024-07-17T07:14:06.885 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:14:06.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:14:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 -- 172.21.15.44:0/3655548495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b6e0 msgr2=0x7fb914159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 --2- 172.21.15.44:0/3655548495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b6e0 0x7fb914159800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb90c005fd0 tx=0x7fb90c02aa90 comp rx=0 tx=0).stop 2024-07-17T07:14:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 -- 172.21.15.44:0/3655548495 shutdown_connections 2024-07-17T07:14:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 --2- 172.21.15.44:0/3655548495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b6e0 0x7fb914159800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 --2- 172.21.15.44:0/3655548495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9140cfa90 0x7fb91406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 -- 172.21.15.44:0/3655548495 >> 172.21.15.44:0/3655548495 conn(0x7fb9140b0e80 msgr2=0x7fb9140b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 -- 172.21.15.44:0/3655548495 shutdown_connections 2024-07-17T07:14:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.105+0000 7fb918c00640 1 -- 172.21.15.44:0/3655548495 wait complete. 2024-07-17T07:14:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.108+0000 7fb918c00640 1 Processor -- start 2024-07-17T07:14:07.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.108+0000 7fb918c00640 1 -- start start 2024-07-17T07:14:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b300 0x7fb914141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91409b6e0 0x7fb91415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140b2120 con 0x7fb9140cfa90 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140b1fa0 con 0x7fb91409b300 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.109+0000 7fb918c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9140b22a0 con 0x7fb91409b6e0 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35316/0 (socket says 172.21.15.44:35316) 2024-07-17T07:14:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 -- 172.21.15.44:0/2259980164 learned_addr learned my addr 172.21.15.44:0/2259980164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb90be00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91409b6e0 0x7fb91415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913400640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b300 0x7fb914141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 -- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91409b6e0 msgr2=0x7fb91415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91409b6e0 0x7fb91415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 -- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b300 msgr2=0x7fb914141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b300 0x7fb914141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.110+0000 7fb913e00640 1 -- 172.21.15.44:0/2259980164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb914073120 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb913e00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c046070 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb90c004040 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c004430 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9141f0b40 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.111+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb914141f80 con 0x7fb9140cfa90 2024-07-17T07:14:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.112+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb914159800 con 0x7fb9140cfa90 2024-07-17T07:14:07.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.113+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7fb90c049070 con 0x7fb9140cfa90 2024-07-17T07:14:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.114+0000 7fb90b400640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 0x7fb8f0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.115+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7fb90c0b9640 con 0x7fb9140cfa90 2024-07-17T07:14:07.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.115+0000 7fb913400640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 0x7fb8f0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.116+0000 7fb913400640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 0x7fb8f0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb904001680 tx=0x7fb904009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:07.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.119+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb90c032030 con 0x7fb9140cfa90 2024-07-17T07:14:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.330+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9140b3f30 con 0x7fb9140cfa90 2024-07-17T07:14:07.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.332+0000 7fb90b400640 1 -- 172.21.15.44:0/2259980164 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+15218 (secure 0 0 0) 0x7fb90c082540 con 0x7fb9140cfa90 2024-07-17T07:14:07.332 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:07.332 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":120,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:14:04.849424+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6833/1121698347":"2024-07-18T07:14:01.822245+0000","172.21.15.175:6801/372663266":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/406108101":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/2268482495":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/4063149711":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/4127919743":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/2651704140":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/173171416":"2024-07-17T08:12:55.098567+0000","172.21.15.175:0/3247461052":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/2377117402":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/2895451842":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6801/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.175:6800/372663266":"2024-07-18T07:14:01.594474+0000","172.21.15.175:6800/3550495011":"2024-07-18T07:12:52.999080+0000","172.21.15.44:6832/1121698347":"2024-07-18T07:14:01.822245+0000","172.21.15.175:0/1860925725":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/1811995408":"2024-07-18T07:12:52.999080+0000","172.21.15.175:0/747846274":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/2542563905":"2024-07-18T07:14:01.594474+0000","172.21.15.175:0/1044580353":"2024-07-18T07:12:52.999080+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 msgr2=0x7fb8f0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 0x7fb8f0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb904001680 tx=0x7fb904009040 comp rx=0 tx=0).stop 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 msgr2=0x7fb91415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c003ce0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 shutdown_connections 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb8f0076cd0 0x7fb8f0079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9140cfa90 0x7fb91415d2c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb91409b6e0 0x7fb91415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 --2- 172.21.15.44:0/2259980164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb91409b300 0x7fb914141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 >> 172.21.15.44:0/2259980164 conn(0x7fb9140b0e80 msgr2=0x7fb9140cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:07.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 shutdown_connections 2024-07-17T07:14:07.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.335+0000 7fb918c00640 1 -- 172.21.15.44:0/2259980164 wait complete. 2024-07-17T07:14:07.355 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:14:07.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:14:07.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 -- 172.21.15.44:0/3047366350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 msgr2=0x7f80ec15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 --2- 172.21.15.44:0/3047366350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec15c140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f80ec155d10 tx=0x7f80e002a560 comp rx=0 tx=0).stop 2024-07-17T07:14:07.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 -- 172.21.15.44:0/3047366350 shutdown_connections 2024-07-17T07:14:07.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 --2- 172.21.15.44:0/3047366350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec152300 0x7f80ec06f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 --2- 172.21.15.44:0/3047366350 >> v2:172.21.15.175:3300/0 conn(0x7f80ec0cd0b0 0x7f80ec06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 --2- 172.21.15.44:0/3047366350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec15c140 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 -- 172.21.15.44:0/3047366350 >> 172.21.15.44:0/3047366350 conn(0x7f80ec0cf0d0 msgr2=0x7f80ec0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:07.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 -- 172.21.15.44:0/3047366350 shutdown_connections 2024-07-17T07:14:07.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.572+0000 7f80f3000640 1 -- 172.21.15.44:0/3047366350 wait complete. 2024-07-17T07:14:07.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.574+0000 7f80f3000640 1 Processor -- start 2024-07-17T07:14:07.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.575+0000 7f80f3000640 1 -- start start 2024-07-17T07:14:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec0cd0b0 0x7f80ec14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec152300 0x7f80ec15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0cb490 con 0x7f80ec151f20 2024-07-17T07:14:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec0cb310 con 0x7f80ec152300 2024-07-17T07:14:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec0cb610 con 0x7f80ec0cd0b0 2024-07-17T07:14:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.576+0000 7f80f1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35328/0 (socket says 172.21.15.44:35328) 2024-07-17T07:14:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 -- 172.21.15.44:0/2746843586 learned_addr learned my addr 172.21.15.44:0/2746843586 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:07.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f2600640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec152300 0x7f80ec15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1c00640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec0cd0b0 0x7f80ec14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 -- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec0cd0b0 msgr2=0x7f80ec14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec0cd0b0 0x7f80ec14e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 -- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec152300 msgr2=0x7f80ec15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec152300 0x7f80ec15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 -- 172.21.15.44:0/2746843586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec099b90 con 0x7f80ec151f20 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.577+0000 7f80f1200640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f80d800d7a0 tx=0x7f80d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.578+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8017070 con 0x7f80ec151f20 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.578+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d8009cc0 con 0x7f80ec151f20 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.578+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec099e70 con 0x7f80ec151f20 2024-07-17T07:14:07.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.578+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80106c0 con 0x7f80ec151f20 2024-07-17T07:14:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.578+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec09a430 con 0x7f80ec151f20 2024-07-17T07:14:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.579+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec0cbac0 con 0x7f80ec151f20 2024-07-17T07:14:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.581+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f80d801a070 con 0x7f80ec151f20 2024-07-17T07:14:07.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.582+0000 7f80e9600640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 0x7f80cc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:07.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.582+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f80d8098a60 con 0x7f80ec151f20 2024-07-17T07:14:07.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.582+0000 7f80f1c00640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 0x7f80cc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:07.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.584+0000 7f80f1c00640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 0x7f80cc079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80e0033000 tx=0x7f80e0004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:07.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.587+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f80d80618d0 con 0x7f80ec151f20 2024-07-17T07:14:07.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.796+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80ec06f4d0 con 0x7f80ec151f20 2024-07-17T07:14:07.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.797+0000 7f80e9600640 1 -- 172.21.15.44:0/2746843586 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7f80d80611d0 con 0x7f80ec151f20 2024-07-17T07:14:07.797 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:14:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 msgr2=0x7f80cc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 0x7f80cc079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80e0033000 tx=0x7f80e0004730 comp rx=0 tx=0).stop 2024-07-17T07:14:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 msgr2=0x7f80ec099220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f80d800d7a0 tx=0x7f80d800dc60 comp rx=0 tx=0).stop 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 shutdown_connections 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f80cc076cd0 0x7f80cc079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec152300 0x7f80ec15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80ec151f20 0x7f80ec099220 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 --2- 172.21.15.44:0/2746843586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80ec0cd0b0 0x7f80ec14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 >> 172.21.15.44:0/2746843586 conn(0x7f80ec0cf0d0 msgr2=0x7f80ec0af030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 shutdown_connections 2024-07-17T07:14:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:07.800+0000 7f80f3000640 1 -- 172.21.15.44:0/2746843586 wait complete. 2024-07-17T07:14:07.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2024-07-17T07:14:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 -- 172.21.15.44:0/3742990801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76000cfa90 msgr2=0x7f760006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 --2- 172.21.15.44:0/3742990801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76000cfa90 0x7f760006e060 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f75ec005fd0 tx=0x7f75ec02ae70 comp rx=0 tx=0).stop 2024-07-17T07:14:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 -- 172.21.15.44:0/3742990801 shutdown_connections 2024-07-17T07:14:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 --2- 172.21.15.44:0/3742990801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76000cfa90 0x7f760006e060 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 -- 172.21.15.44:0/3742990801 >> 172.21.15.44:0/3742990801 conn(0x7f76000b0e80 msgr2=0x7f76000b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 -- 172.21.15.44:0/3742990801 shutdown_connections 2024-07-17T07:14:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.032+0000 7f7606000640 1 -- 172.21.15.44:0/3742990801 wait complete. 2024-07-17T07:14:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.034+0000 7f7606000640 1 Processor -- start 2024-07-17T07:14:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.034+0000 7f7606000640 1 -- start start 2024-07-17T07:14:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760009b300 0x7f76001502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76000cfa90 0x7f76000b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76000b2330 con 0x7f760009b6e0 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76000b21b0 con 0x7f760009b300 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f7606000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76000b24b0 con 0x7f76000cfa90 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35342/0 (socket says 172.21.15.44:35342) 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 -- 172.21.15.44:0/1875640669 learned_addr learned my addr 172.21.15.44:0/1875640669 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 -- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76000cfa90 msgr2=0x7f76000b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76000cfa90 0x7f76000b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 -- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760009b300 msgr2=0x7f76001502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760009b300 0x7f76001502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.035+0000 7f75ffe00640 1 -- 172.21.15.44:0/1875640669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7600150ff0 con 0x7f760009b6e0 2024-07-17T07:14:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f75ffe00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b3c0 tx=0x7f75ec02cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec041020 con 0x7f760009b6e0 2024-07-17T07:14:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75ec0094b0 con 0x7f760009b6e0 2024-07-17T07:14:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec009860 con 0x7f760009b6e0 2024-07-17T07:14:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760013e2d0 con 0x7f760009b6e0 2024-07-17T07:14:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.036+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760014f810 con 0x7f760009b6e0 2024-07-17T07:14:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.038+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760013b380 con 0x7f760009b6e0 2024-07-17T07:14:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.039+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f75ec044020 con 0x7f760009b6e0 2024-07-17T07:14:08.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.040+0000 7f75fcc00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 0x7f75e0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.040+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f75ec0b9560 con 0x7f760009b6e0 2024-07-17T07:14:08.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.041+0000 7f7604c00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 0x7f75e0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:08.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.042+0000 7f7604c00640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 0x7f75e0079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000f60 tx=0x7f75f40079b0 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.045+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f760013b380 con 0x7f760009b6e0 2024-07-17T07:14:08.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.333+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) -- 0x7f76000b2a40 con 0x7f760009b6e0 2024-07-17T07:14:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.873+0000 7f75fcc00640 1 -- 172.21.15.44:0/1875640669 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v728) ==== 151+0+0 (secure 0 0 0) 0x7f75ec04b030 con 0x7f760009b6e0 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.876+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 msgr2=0x7f75e0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.876+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 0x7f75e0079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000f60 tx=0x7f75f40079b0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.876+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 msgr2=0x7f76000b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.876+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00b3c0 tx=0x7f75ec02cdc0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 shutdown_connections 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f75e0076cd0 0x7f75e0079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76000cfa90 0x7f76000b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f760009b6e0 0x7f76000b31f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 --2- 172.21.15.44:0/1875640669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760009b300 0x7f76001502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 >> 172.21.15.44:0/1875640669 conn(0x7f76000b0e80 msgr2=0x7f76000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 shutdown_connections 2024-07-17T07:14:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:08.877+0000 7f7606000640 1 -- 172.21.15.44:0/1875640669 wait complete. 2024-07-17T07:14:08.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:14:09.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 -- 172.21.15.44:0/2841199975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc0ce410 msgr2=0x7f08fc160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 --2- 172.21.15.44:0/2841199975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc0ce410 0x7f08fc160d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005fd0 tx=0x7f08e402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 -- 172.21.15.44:0/2841199975 shutdown_connections 2024-07-17T07:14:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 --2- 172.21.15.44:0/2841199975 >> v2:172.21.15.175:3300/0 conn(0x7f08fc15e3d0 0x7f08fc161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 --2- 172.21.15.44:0/2841199975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc0ce410 0x7f08fc160d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 -- 172.21.15.44:0/2841199975 >> 172.21.15.44:0/2841199975 conn(0x7f08fc06c390 msgr2=0x7f08fc06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 -- 172.21.15.44:0/2841199975 shutdown_connections 2024-07-17T07:14:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.115+0000 7f08fb200640 1 -- 172.21.15.44:0/2841199975 wait complete. 2024-07-17T07:14:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.117+0000 7f08fb200640 1 Processor -- start 2024-07-17T07:14:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.118+0000 7f08fb200640 1 -- start start 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 0x7f08fc157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 0x7f08fc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0d1810 con 0x7f08fc0ce410 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0d1690 con 0x7f08fc15e3d0 2024-07-17T07:14:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0d1990 con 0x7f08fc1579d0 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53878/0 (socket says 172.21.15.44:53878) 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.119+0000 7f08f9e00640 1 -- 172.21.15.44:0/2713516078 learned_addr learned my addr 172.21.15.44:0/2713516078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08fa800640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 0x7f08fc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 -- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 msgr2=0x7f08fc157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f1400640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 0x7f08fc157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 0x7f08fc157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 -- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 msgr2=0x7f08fc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 0x7f08fc0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 -- 172.21.15.44:0/2713516078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc152180 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08fa800640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 0x7f08fc0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f1400640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 0x7f08fc157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.120+0000 7f08f9e00640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c450 tx=0x7f08ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.121+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0052d0 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.121+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08ec005470 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.121+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc158840 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.121+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec012870 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.121+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc158e00 con 0x7f08fc0ce410 2024-07-17T07:14:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.122+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc0d1eb0 con 0x7f08fc0ce410 2024-07-17T07:14:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.123+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f08ec0035f0 con 0x7f08fc0ce410 2024-07-17T07:14:09.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.124+0000 7f08f2a00640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 0x7f08d8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.124+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f08ec098470 con 0x7f08fc0ce410 2024-07-17T07:14:09.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.125+0000 7f08f1400640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 0x7f08d8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.126+0000 7f08f1400640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 0x7f08d8079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08e4033000 tx=0x7f08e4004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.131+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f08ec061400 con 0x7f08fc0ce410 2024-07-17T07:14:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.342+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f08fc0d0e80 con 0x7f08fc0ce410 2024-07-17T07:14:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.343+0000 7f08f2a00640 1 -- 172.21.15.44:0/2713516078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7f08ec060d00 con 0x7f08fc0ce410 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:14:09.344 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:14:09.345 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:14:09.346 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:14:09.347 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:14:09.347 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:14:09.347 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 msgr2=0x7f08d8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 0x7f08d8079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08e4033000 tx=0x7f08e4004730 comp rx=0 tx=0).stop 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 msgr2=0x7f08fc151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c450 tx=0x7f08ec00c910 comp rx=0 tx=0).stop 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 shutdown_connections 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f08d8076cd0 0x7f08d8079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08fc15e3d0 0x7f08fc0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08fc1579d0 0x7f08fc157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 --2- 172.21.15.44:0/2713516078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08fc0ce410 0x7f08fc151c40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.347+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 >> 172.21.15.44:0/2713516078 conn(0x7f08fc06c390 msgr2=0x7f08fc15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:09.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.348+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 shutdown_connections 2024-07-17T07:14:09.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.348+0000 7f08fb200640 1 -- 172.21.15.44:0/2713516078 wait complete. 2024-07-17T07:14:09.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 -- 172.21.15.44:0/491267383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2264151f20 msgr2=0x7f226415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 --2- 172.21.15.44:0/491267383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2264151f20 0x7f226415c140 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2258005fd0 tx=0x7f225802aa10 comp rx=0 tx=0).stop 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 -- 172.21.15.44:0/491267383 shutdown_connections 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 --2- 172.21.15.44:0/491267383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f226406f380 0x7f22640cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 --2- 172.21.15.44:0/491267383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2264151f20 0x7f226415c140 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 -- 172.21.15.44:0/491267383 >> 172.21.15.44:0/491267383 conn(0x7f22640cf0d0 msgr2=0x7f22640cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 -- 172.21.15.44:0/491267383 shutdown_connections 2024-07-17T07:14:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.582+0000 7f226b000640 1 -- 172.21.15.44:0/491267383 wait complete. 2024-07-17T07:14:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.584+0000 7f226b000640 1 Processor -- start 2024-07-17T07:14:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.584+0000 7f226b000640 1 -- start start 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22640cd0b0 0x7f2264150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2264151f20 0x7f22641506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22640cb740 con 0x7f226406f380 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22640cb5c0 con 0x7f22640cd0b0 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f226b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22640cb8c0 con 0x7f2264151f20 2024-07-17T07:14:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f2269c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f2269c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53894/0 (socket says 172.21.15.44:53894) 2024-07-17T07:14:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.586+0000 7f2269c00640 1 -- 172.21.15.44:0/4185037229 learned_addr learned my addr 172.21.15.44:0/4185037229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 -- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2264151f20 msgr2=0x7f22641506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2264151f20 0x7f22641506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 -- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22640cd0b0 msgr2=0x7f2264150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22640cd0b0 0x7f2264150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 -- 172.21.15.44:0/4185037229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641f6f70 con 0x7f226406f380 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2269c00640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258033050 con 0x7f226406f380 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2258004450 con 0x7f226406f380 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258004840 con 0x7f226406f380 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.587+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641f7250 con 0x7f226406f380 2024-07-17T07:14:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.588+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641f7810 con 0x7f226406f380 2024-07-17T07:14:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.589+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22640cb080 con 0x7f226406f380 2024-07-17T07:14:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.590+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f2258036070 con 0x7f226406f380 2024-07-17T07:14:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.591+0000 7f2261600640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 0x7f2244079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.591+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f22580b9cc0 con 0x7f226406f380 2024-07-17T07:14:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.592+0000 7f2269200640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 0x7f2244079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.593+0000 7f2269200640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 0x7f2244079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2250000f60 tx=0x7f22500079b0 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:09.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.595+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f22580c0080 con 0x7f226406f380 2024-07-17T07:14:09.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.805+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f22641f7bf0 con 0x7f226406f380 2024-07-17T07:14:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.811+0000 7f2261600640 1 -- 172.21.15.44:0/4185037229 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f2258082b30 con 0x7f226406f380 2024-07-17T07:14:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.814+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 msgr2=0x7f2244079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.814+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 0x7f2244079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2250000f60 tx=0x7f22500079b0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.814+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 msgr2=0x7f22640affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.814+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258003f90 comp rx=0 tx=0).stop 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 shutdown_connections 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f2244076cd0 0x7f2244079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2264151f20 0x7f22641506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22640cd0b0 0x7f2264150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 --2- 172.21.15.44:0/4185037229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f226406f380 0x7f22640affb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 >> 172.21.15.44:0/4185037229 conn(0x7f22640cf0d0 msgr2=0x7f22640cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 shutdown_connections 2024-07-17T07:14:09.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:09.815+0000 7f226b000640 1 -- 172.21.15.44:0/4185037229 wait complete. 2024-07-17T07:14:09.834 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:14:09.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:14:09.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:14:10.079 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:14:10.079 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:14:10.079 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:14:10.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:14:10.139 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:14:10.139 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:14:10.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:14:10.195 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:14:10.195 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:14:10.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:14:10.254 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:14:10.254 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:14:10.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:14:10.310 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:14:10.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:14:10.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 -- 172.21.15.44:0/3455934619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36240ac3e0 msgr2=0x7f36240ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 --2- 172.21.15.44:0/3455934619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36240ac3e0 0x7f36240ac870 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f36240aeda0 tx=0x7f361002ae80 comp rx=0 tx=0).stop 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 -- 172.21.15.44:0/3455934619 shutdown_connections 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 --2- 172.21.15.44:0/3455934619 >> v2:172.21.15.175:3300/0 conn(0x7f362406e090 0x7f362406e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 --2- 172.21.15.44:0/3455934619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36240ac3e0 0x7f36240ac870 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 -- 172.21.15.44:0/3455934619 >> 172.21.15.44:0/3455934619 conn(0x7f36240b0fb0 msgr2=0x7f36240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 -- 172.21.15.44:0/3455934619 shutdown_connections 2024-07-17T07:14:10.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.530+0000 7f3628a00640 1 -- 172.21.15.44:0/3455934619 wait complete. 2024-07-17T07:14:10.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.533+0000 7f3628a00640 1 Processor -- start 2024-07-17T07:14:10.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.533+0000 7f3628a00640 1 -- start start 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362406e090 0x7f36240b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240ac3e0 0x7f36241f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3624153fb0 con 0x7f3624153a50 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3624153e30 con 0x7f36240ac3e0 2024-07-17T07:14:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3628a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f362406f710 con 0x7f362406e090 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3622800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3622800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53908/0 (socket says 172.21.15.44:53908) 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.534+0000 7f3622800640 1 -- 172.21.15.44:0/1959228621 learned_addr learned my addr 172.21.15.44:0/1959228621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3621400640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240ac3e0 0x7f36241f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3621e00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362406e090 0x7f36240b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 -- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362406e090 msgr2=0x7f36240b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362406e090 0x7f36240b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 -- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240ac3e0 msgr2=0x7f36241f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240ac3e0 0x7f36241f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 -- 172.21.15.44:0/1959228621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241f8da0 con 0x7f3624153a50 2024-07-17T07:14:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.535+0000 7f3622800640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b9e0 tx=0x7f361c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.536+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c019070 con 0x7f3624153a50 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.536+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f361c00cbb0 con 0x7f3624153a50 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.536+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3624140870 con 0x7f3624153a50 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.536+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c014440 con 0x7f3624153a50 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.536+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36241f8ac0 con 0x7f3624153a50 2024-07-17T07:14:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.537+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36240ac870 con 0x7f3624153a50 2024-07-17T07:14:10.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.538+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f361c01c070 con 0x7f3624153a50 2024-07-17T07:14:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.539+0000 7f361a000640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 0x7f36040791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.539+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f361c098520 con 0x7f3624153a50 2024-07-17T07:14:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.540+0000 7f3621e00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 0x7f36040791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.541+0000 7f3621e00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 0x7f36040791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36140015a0 tx=0x7f3614009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.544+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f361c061530 con 0x7f3624153a50 2024-07-17T07:14:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.792+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3624158cd0 con 0x7f3624153a50 2024-07-17T07:14:10.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.793+0000 7f361a000640 1 -- 172.21.15.44:0/1959228621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+3914 (secure 0 0 0) 0x7f361c060e30 con 0x7f3624153a50 2024-07-17T07:14:10.793 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:10.793 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":54,"btime":"2024-07-17T07:14:02:826201+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3717190701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3717190701},{"type":"v1","addr":"172.21.15.175:6835","nonce":3717190701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1392522397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1392522397},{"type":"v1","addr":"172.21.15.175:6837","nonce":1392522397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2326109078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2326109078},{"type":"v1","addr":"172.21.15.44:6835","nonce":2326109078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":7410,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/626912056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":626912056},{"type":"v1","addr":"172.21.15.44:6833","nonce":626912056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-07-17T07:14:10.794 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 54 2024-07-17T07:14:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 msgr2=0x7f36040791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 0x7f36040791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36140015a0 tx=0x7f3614009040 comp rx=0 tx=0).stop 2024-07-17T07:14:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 msgr2=0x7f3624156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b9e0 tx=0x7f361c00bea0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 shutdown_connections 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3604076d00 0x7f36040791c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3624153a50 0x7f3624156930 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36240ac3e0 0x7f36241f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 --2- 172.21.15.44:0/1959228621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f362406e090 0x7f36240b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 >> 172.21.15.44:0/1959228621 conn(0x7f36240b0fb0 msgr2=0x7f362413ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 shutdown_connections 2024-07-17T07:14:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:10.796+0000 7f3628a00640 1 -- 172.21.15.44:0/1959228621 wait complete. 2024-07-17T07:14:10.816 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:14:10.816 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:14:10.816 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:14:10.816 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:14:10.816 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:14:10.817 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:14:10.817 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:14:10.817 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:14:10.817 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:14:10.817 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:14:10.817 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:14:10.817 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:14:10.818 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:14:10.818 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:14:10.818 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:14:10.818 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:14:10.838 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:14:10.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:14:10.839 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:14:10.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:14:10.840 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:14:10.840 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:14:10.843 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:14:10.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:14:10.888 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:14:10.888 DEBUG:teuthology.parallel:result is None 2024-07-17T07:14:10.891 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:14:10.891 DEBUG:teuthology.parallel:result is None 2024-07-17T07:14:10.895 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:14:10.895 DEBUG:teuthology.parallel:result is None 2024-07-17T07:14:10.899 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:14:10.899 DEBUG:teuthology.parallel:result is None 2024-07-17T07:14:10.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:14:11.037 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:14:11.038 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:14:11.046 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:14:11.047 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:14:11.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.083+0000 7f1b40e00640 1 -- 172.21.15.44:0/520518059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c06dda0 msgr2=0x7f1b3c1682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.083+0000 7f1b40e00640 1 --2- 172.21.15.44:0/520518059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c06dda0 0x7f1b3c1682d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1b34005fd0 tx=0x7f1b3402a940 comp rx=0 tx=0).stop 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.083+0000 7f1b40e00640 1 -- 172.21.15.44:0/520518059 shutdown_connections 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.083+0000 7f1b40e00640 1 --2- 172.21.15.44:0/520518059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c06dda0 0x7f1b3c1682d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.083+0000 7f1b40e00640 1 -- 172.21.15.44:0/520518059 >> 172.21.15.44:0/520518059 conn(0x7f1b3c073d50 msgr2=0x7f1b3c09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.084+0000 7f1b40e00640 1 -- 172.21.15.44:0/520518059 shutdown_connections 2024-07-17T07:14:11.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.084+0000 7f1b40e00640 1 -- 172.21.15.44:0/520518059 wait complete. 2024-07-17T07:14:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.085+0000 7f1b40e00640 1 Processor -- start 2024-07-17T07:14:11.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.085+0000 7f1b40e00640 1 -- start start 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b3c06dda0 0x7f1b3c147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b3c15d810 0x7f1b3c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c168990 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c168810 con 0x7f1b3c15d810 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.086+0000 7f1b40e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c168b10 con 0x7f1b3c06dda0 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:11.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53950/0 (socket says 172.21.15.44:53950) 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 -- 172.21.15.44:0/360385704 learned_addr learned my addr 172.21.15.44:0/360385704 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b3a800640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b3c15d810 0x7f1b3c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b3c06dda0 0x7f1b3c147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 -- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b3c06dda0 msgr2=0x7f1b3c147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b3c06dda0 0x7f1b3c147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 -- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b3c15d810 msgr2=0x7f1b3c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b3c15d810 0x7f1b3c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.087+0000 7f1b39400640 1 -- 172.21.15.44:0/360385704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c158320 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b39400640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c440 tx=0x7f1b2800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005480 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b280035f0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28003a20 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c158600 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.088+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c155b20 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.089+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c06ede0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.090+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f1b28003bc0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.091+0000 7f1b32000640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 0x7f1b1c079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:11.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.092+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6629+0+0 (secure 0 0 0) 0x7f1b28098370 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.092+0000 7f1b39e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 0x7f1b1c079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:11.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.093+0000 7f1b39e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 0x7f1b1c079330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000f60 tx=0x7f1b2c009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:11.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.096+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1b3c06ede0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.325+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1b3c09f7d0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.913+0000 7f1b32000640 1 -- 172.21.15.44:0/360385704 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v122) ==== 97+0+0 (secure 0 0 0) 0x7f1b3c09f7d0 con 0x7f1b3c0b1a10 2024-07-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.917+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 msgr2=0x7f1b1c079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.917+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 0x7f1b1c079330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000f60 tx=0x7f1b2c009040 comp rx=0 tx=0).stop 2024-07-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.917+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 msgr2=0x7f1b3c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c440 tx=0x7f1b2800c900 comp rx=0 tx=0).stop 2024-07-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 shutdown_connections 2024-07-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1b1c076e70 0x7f1b1c079330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b3c15d810 0x7f1b3c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b3c0b1a10 0x7f1b3c157940 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 --2- 172.21.15.44:0/360385704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b3c06dda0 0x7f1b3c147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 >> 172.21.15.44:0/360385704 conn(0x7f1b3c073d50 msgr2=0x7f1b3c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 shutdown_connections 2024-07-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:11.918+0000 7f1b40e00640 1 -- 172.21.15.44:0/360385704 wait complete. 2024-07-17T07:14:11.942 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.152+0000 7f05fca00640 1 -- 172.21.15.44:0/1324863819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80b3db0 msgr2=0x7f05f80ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.152+0000 7f05fca00640 1 --2- 172.21.15.44:0/1324863819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80b3db0 0x7f05f80ad8e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f05f80d00a0 tx=0x7f05e002a4d0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 -- 172.21.15.44:0/1324863819 shutdown_connections 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 --2- 172.21.15.44:0/1324863819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80b3db0 0x7f05f80ad8e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 --2- 172.21.15.44:0/1324863819 >> v2:172.21.15.175:3300/0 conn(0x7f05f80cf540 0x7f05f80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 -- 172.21.15.44:0/1324863819 >> 172.21.15.44:0/1324863819 conn(0x7f05f80b0e80 msgr2=0x7f05f80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 -- 172.21.15.44:0/1324863819 shutdown_connections 2024-07-17T07:14:12.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.153+0000 7f05fca00640 1 -- 172.21.15.44:0/1324863819 wait complete. 2024-07-17T07:14:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.154+0000 7f05fca00640 1 Processor -- start 2024-07-17T07:14:12.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.154+0000 7f05fca00640 1 -- start start 2024-07-17T07:14:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 0x7f05f81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 0x7f05f80736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80ae8c0 con 0x7f05f80cf920 2024-07-17T07:14:12.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05f80ae740 con 0x7f05f80cf540 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.155+0000 7f05fca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f05f80aea40 con 0x7f05f80b3db0 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53966/0 (socket says 172.21.15.44:53966) 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 -- 172.21.15.44:0/2762534254 learned_addr learned my addr 172.21.15.44:0/2762534254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f6a00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 0x7f05f80736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 -- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 msgr2=0x7f05f81f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7400640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 0x7f05f81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 0x7f05f81f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 -- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 msgr2=0x7f05f80736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 0x7f05f80736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7e00640 1 -- 172.21.15.44:0/2762534254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f81f4140 con 0x7f05f80cf920 2024-07-17T07:14:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f6a00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 0x7f05f80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.156+0000 7f05f7400640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 0x7f05f81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05f7e00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05f000b990 tx=0x7f05f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:12.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0019070 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05f000cc30 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f000ce60 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f8071fa0 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.157+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f8072560 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.158+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f81412e0 con 0x7f05f80cf920 2024-07-17T07:14:12.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.160+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f05f001c070 con 0x7f05f80cf920 2024-07-17T07:14:12.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.161+0000 7f05ef400640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 0x7f05d4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.161+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5769+0+0 (secure 0 0 0) 0x7f05f0098190 con 0x7f05f80cf920 2024-07-17T07:14:12.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.161+0000 7f05f7400640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 0x7f05d4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.162+0000 7f05f7400640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 0x7f05d4079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006fd0 tx=0x7f05e8008040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:12.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.166+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f05f00615a0 con 0x7f05f80cf920 2024-07-17T07:14:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.455+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f05f8140630 con 0x7f05f80cf920 2024-07-17T07:14:12.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.457+0000 7f05ef400640 1 -- 172.21.15.44:0/2762534254 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) ==== 80+0+5304 (secure 0 0 0) 0x7f05f0060ea0 con 0x7f05f80cf920 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:14:12.457 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:14:12.459 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:14:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.463 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.465 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:14:12.466 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:14:12.467 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:14:12.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:14:12.470 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:14:12.471 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:14:12.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.459+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 msgr2=0x7f05d4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.459+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 0x7f05d4079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006fd0 tx=0x7f05e8008040 comp rx=0 tx=0).stop 2024-07-17T07:14:12.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.459+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 msgr2=0x7f05f8073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.459+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05f000b990 tx=0x7f05f000be50 comp rx=0 tx=0).stop 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 shutdown_connections 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f05d4076cd0 0x7f05d4079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05f80cf920 0x7f05f8073c30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05f80cf540 0x7f05f80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 --2- 172.21.15.44:0/2762534254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05f80b3db0 0x7f05f81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 >> 172.21.15.44:0/2762534254 conn(0x7f05f80b0e80 msgr2=0x7f05f80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 shutdown_connections 2024-07-17T07:14:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.460+0000 7f05fca00640 1 -- 172.21.15.44:0/2762534254 wait complete. 2024-07-17T07:14:12.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:14:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 -- 172.21.15.44:0/3877694088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0974004090 msgr2=0x7f0974004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 --2- 172.21.15.44:0/3877694088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0974004090 0x7f0974004800 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f097c00baf0 tx=0x7f0974005830 comp rx=0 tx=0).stop 2024-07-17T07:14:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0992800640 1 -- 172.21.15.44:0/3877694088 reap_dead start 2024-07-17T07:14:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 -- 172.21.15.44:0/3877694088 shutdown_connections 2024-07-17T07:14:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 -- 172.21.15.44:0/3877694088 >> 172.21.15.44:0/3877694088 conn(0x7f09940b0fb0 msgr2=0x7f09940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 -- 172.21.15.44:0/3877694088 shutdown_connections 2024-07-17T07:14:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.680+0000 7f0998c00640 1 -- 172.21.15.44:0/3877694088 wait complete. 2024-07-17T07:14:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.682+0000 7f0998c00640 1 Processor -- start 2024-07-17T07:14:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.682+0000 7f0998c00640 1 -- start start 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0974004090 0x7f0994152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0994152590 0x7f0994151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0994154a00 con 0x7f0994073eb0 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f099415b8d0 con 0x7f0974004090 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0998c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0994154b80 con 0x7f0994152590 2024-07-17T07:14:12.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0992800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0992800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53984/0 (socket says 172.21.15.44:53984) 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.683+0000 7f0992800640 1 -- 172.21.15.44:0/3537615040 learned_addr learned my addr 172.21.15.44:0/3537615040 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0991e00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0974004090 0x7f0994152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0991400640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0994152590 0x7f0994151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 -- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0994152590 msgr2=0x7f0994151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0994152590 0x7f0994151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 -- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0974004090 msgr2=0x7f0994152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0974004090 0x7f0994152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 -- 172.21.15.44:0/3537615040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09940cd1e0 con 0x7f0994073eb0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0992800640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f098c001350 tx=0x7f098c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00b840 con 0x7f0994073eb0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f098c017070 con 0x7f0994073eb0 2024-07-17T07:14:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09940cd4c0 con 0x7f0994073eb0 2024-07-17T07:14:12.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00bc80 con 0x7f0994073eb0 2024-07-17T07:14:12.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.684+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09940cda80 con 0x7f0994073eb0 2024-07-17T07:14:12.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.686+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099406f480 con 0x7f0994073eb0 2024-07-17T07:14:12.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.687+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f098c01c460 con 0x7f0994073eb0 2024-07-17T07:14:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.688+0000 7f098a000640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 0x7f09700791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.688+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5769+0+0 (secure 0 0 0) 0x7f098c098320 con 0x7f0994073eb0 2024-07-17T07:14:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.689+0000 7f0991e00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 0x7f09700791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:12.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.690+0000 7f0991e00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 0x7f09700791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0994158d80 tx=0x7f0984006bc0 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:12.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.693+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f098c061600 con 0x7f0994073eb0 2024-07-17T07:14:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.907+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f09940737f0 con 0x7f0994073eb0 2024-07-17T07:14:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.914+0000 7f098a000640 1 -- 172.21.15.44:0/3537615040 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v124) ==== 84+0+65 (secure 0 0 0) 0x7f098c060f00 con 0x7f0994073eb0 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.917+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 msgr2=0x7f09700791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.917+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 0x7f09700791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0994158d80 tx=0x7f0984006bc0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.917+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 msgr2=0x7f099415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.917+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f098c001350 tx=0x7f098c00dd50 comp rx=0 tx=0).stop 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 shutdown_connections 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f0970076d00 0x7f09700791c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0994073eb0 0x7f099415bcd0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0994152590 0x7f0994151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 --2- 172.21.15.44:0/3537615040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0974004090 0x7f0994152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:12.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 >> 172.21.15.44:0/3537615040 conn(0x7f09940b0fb0 msgr2=0x7f09941542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:12.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 shutdown_connections 2024-07-17T07:14:12.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:12.918+0000 7f0998c00640 1 -- 172.21.15.44:0/3537615040 wait complete. 2024-07-17T07:14:12.938 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:14:12.938 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.150+0000 7ff32b200640 1 -- 172.21.15.44:0/4186909182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff310004d90 msgr2=0x7ff310005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.150+0000 7ff32b200640 1 --2- 172.21.15.44:0/4186909182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff310004d90 0x7ff310005210 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff324001d90 tx=0x7ff310005830 comp rx=0 tx=0).stop 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.151+0000 7ff32a800640 1 -- 172.21.15.44:0/4186909182 reap_dead start 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.151+0000 7ff32b200640 1 -- 172.21.15.44:0/4186909182 shutdown_connections 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.151+0000 7ff32b200640 1 -- 172.21.15.44:0/4186909182 >> 172.21.15.44:0/4186909182 conn(0x7ff32c0cf050 msgr2=0x7ff32c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.151+0000 7ff32b200640 1 -- 172.21.15.44:0/4186909182 shutdown_connections 2024-07-17T07:14:13.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.151+0000 7ff32b200640 1 -- 172.21.15.44:0/4186909182 wait complete. 2024-07-17T07:14:13.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.153+0000 7ff32b200640 1 Processor -- start 2024-07-17T07:14:13.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.153+0000 7ff32b200640 1 -- start start 2024-07-17T07:14:13.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 0x7ff32c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:13.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:13.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff32c1584c0 0x7ff32c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0ae870 con 0x7ff32c1f3f00 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0ae6f0 con 0x7ff32c1584c0 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c153e60 con 0x7ff32c15bea0 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54002/0 (socket says 172.21.15.44:54002) 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 -- 172.21.15.44:0/3858122700 learned_addr learned my addr 172.21.15.44:0/3858122700 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff329e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 0x7ff32c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff32a800640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff32c1584c0 0x7ff32c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 -- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 msgr2=0x7ff32c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 0x7ff32c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 -- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff32c1584c0 msgr2=0x7ff32c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff32c1584c0 0x7ff32c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff323e00640 1 -- 172.21.15.44:0/3858122700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c1556f0 con 0x7ff32c1f3f00 2024-07-17T07:14:13.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.155+0000 7ff329e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 0x7ff32c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:13.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff323e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff31400fa30 tx=0x7ff31400fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:13.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31400bcb0 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff314011440 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c1f4450 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314011890 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.156+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c1f4cf0 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.158+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c1543f0 con 0x7ff32c1f3f00 2024-07-17T07:14:13.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.159+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7ff314019020 con 0x7ff32c1f3f00 2024-07-17T07:14:13.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.160+0000 7ff322000640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 0x7ff308079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:13.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.161+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5769+0+0 (secure 0 0 0) 0x7ff314098f90 con 0x7ff32c1f3f00 2024-07-17T07:14:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.161+0000 7ff329e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 0x7ff308079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.162+0000 7ff329e00640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 0x7ff308079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff31c001880 tx=0x7ff31c001d20 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.165+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff314062160 con 0x7ff32c1f3f00 2024-07-17T07:14:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.371+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff32c06f720 con 0x7ff32c1f3f00 2024-07-17T07:14:13.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.930+0000 7ff322000640 1 -- 172.21.15.44:0/3858122700 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v125) ==== 138+0+0 (secure 0 0 0) 0x7ff314061a60 con 0x7ff32c1f3f00 2024-07-17T07:14:13.931 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:14:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 msgr2=0x7ff308079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 0x7ff308079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff31c001880 tx=0x7ff31c001d20 comp rx=0 tx=0).stop 2024-07-17T07:14:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 msgr2=0x7ff32c157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff31400fa30 tx=0x7ff31400fef0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 shutdown_connections 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff308076cd0 0x7ff308079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff32c1584c0 0x7ff32c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.936+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff32c1f3f00 0x7ff32c157f80 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.937+0000 7ff32b200640 1 --2- 172.21.15.44:0/3858122700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff32c15bea0 0x7ff32c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.937+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 >> 172.21.15.44:0/3858122700 conn(0x7ff32c0cf050 msgr2=0x7ff32c073410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.937+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 shutdown_connections 2024-07-17T07:14:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:13.937+0000 7ff32b200640 1 -- 172.21.15.44:0/3858122700 wait complete. 2024-07-17T07:14:13.965 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 -- 172.21.15.44:0/1216656109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 msgr2=0x7f8968004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 --2- 172.21.15.44:0/1216656109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f8968004c80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8980161550 tx=0x7f8968005830 comp rx=0 tx=0).stop 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 -- 172.21.15.44:0/1216656109 shutdown_connections 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 --2- 172.21.15.44:0/1216656109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f8968004c80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 -- 172.21.15.44:0/1216656109 >> 172.21.15.44:0/1216656109 conn(0x7f89800b0eb0 msgr2=0x7f89800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 -- 172.21.15.44:0/1216656109 shutdown_connections 2024-07-17T07:14:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.180+0000 7f8988400640 1 -- 172.21.15.44:0/1216656109 wait complete. 2024-07-17T07:14:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.182+0000 7f8988400640 1 Processor -- start 2024-07-17T07:14:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.182+0000 7f8988400640 1 -- start start 2024-07-17T07:14:14.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:14.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 0x7f89801f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:14.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89801f47c0 0x7f89801f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:14.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89800b3e60 con 0x7f8968004800 2024-07-17T07:14:14.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89800b3ce0 con 0x7f89801f47c0 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8988400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89800b3fe0 con 0x7f898009bf20 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8985a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8985a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54024/0 (socket says 172.21.15.44:54024) 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.183+0000 7f8985a00640 1 -- 172.21.15.44:0/1996820335 learned_addr learned my addr 172.21.15.44:0/1996820335 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985000640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 0x7f89801f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8986400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89801f47c0 0x7f89801f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 -- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 msgr2=0x7f89801f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 0x7f89801f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 -- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89801f47c0 msgr2=0x7f89801f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89801f47c0 0x7f89801f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 -- 172.21.15.44:0/1996820335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89801f4020 con 0x7f8968004800 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985000640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 0x7f89801f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8985a00640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f897400fa60 tx=0x7f897400ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:14.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400bd10 con 0x7f8968004800 2024-07-17T07:14:14.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8974004030 con 0x7f8968004800 2024-07-17T07:14:14.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974004460 con 0x7f8968004800 2024-07-17T07:14:14.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.184+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801f7b50 con 0x7f8968004800 2024-07-17T07:14:14.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.185+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801f80d0 con 0x7f8968004800 2024-07-17T07:14:14.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.186+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89800b41c0 con 0x7f8968004800 2024-07-17T07:14:14.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.187+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f89740035f0 con 0x7f8968004800 2024-07-17T07:14:14.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.188+0000 7f897d600640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 0x7f8960079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:14.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.188+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6139+0+0 (secure 0 0 0) 0x7f8974099590 con 0x7f8968004800 2024-07-17T07:14:14.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.189+0000 7f8985000640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 0x7f8960079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:14.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.190+0000 7f8985000640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 0x7f8960079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f896c033000 tx=0x7f896c001c70 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:14.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.192+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f89801f4020 con 0x7f8968004800 2024-07-17T07:14:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.412+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8980160e90 con 0x7f8968004800 2024-07-17T07:14:14.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.937+0000 7f897d600640 1 -- 172.21.15.44:0/1996820335 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v126) ==== 170+0+0 (secure 0 0 0) 0x7f89740620b0 con 0x7f8968004800 2024-07-17T07:14:14.938 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:14:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 msgr2=0x7f8960079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 0x7f8960079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f896c033000 tx=0x7f896c001c70 comp rx=0 tx=0).stop 2024-07-17T07:14:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 msgr2=0x7f898015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:14.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f897400fa60 tx=0x7f897400ff20 comp rx=0 tx=0).stop 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 shutdown_connections 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f8960076cd0 0x7f8960079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89801f47c0 0x7f89801f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f898009bf20 0x7f89801f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 --2- 172.21.15.44:0/1996820335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8968004800 0x7f898015a730 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.942+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 >> 172.21.15.44:0/1996820335 conn(0x7f89800b0eb0 msgr2=0x7f89800aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.943+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 shutdown_connections 2024-07-17T07:14:14.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:14.943+0000 7f8988400640 1 -- 172.21.15.44:0/1996820335 wait complete. 2024-07-17T07:14:14.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:14:15.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 -- 172.21.15.44:0/3843908471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec06f6d0 msgr2=0x7f1bec15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/3843908471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec06f6d0 0x7f1bec15a5a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0aedf0 tx=0x7f1bdc02a560 comp rx=0 tx=0).stop 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 -- 172.21.15.44:0/3843908471 shutdown_connections 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/3843908471 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 0x7f1bec0ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/3843908471 >> v2:172.21.15.175:3300/0 conn(0x7f1bec0b3280 0x7f1bec06eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/3843908471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec06f6d0 0x7f1bec15a5a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 -- 172.21.15.44:0/3843908471 >> 172.21.15.44:0/3843908471 conn(0x7f1bec0b0eb0 msgr2=0x7f1bec0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:15.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 -- 172.21.15.44:0/3843908471 shutdown_connections 2024-07-17T07:14:15.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.163+0000 7f1bf3c00640 1 -- 172.21.15.44:0/3843908471 wait complete. 2024-07-17T07:14:15.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.165+0000 7f1bf3c00640 1 Processor -- start 2024-07-17T07:14:15.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.166+0000 7f1bf3c00640 1 -- start start 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bec06f6d0 0x7f1bec1f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 0x7f1bec15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0ae8f0 con 0x7f1bec0b3280 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0ae770 con 0x7f1bec06f6d0 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bec0aea70 con 0x7f1bec0ac3a0 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54036/0 (socket says 172.21.15.44:54036) 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf1c00640 1 -- 172.21.15.44:0/969280862 learned_addr learned my addr 172.21.15.44:0/969280862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:15.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.167+0000 7f1bf1200640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bec06f6d0 0x7f1bec1f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1be3e00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 0x7f1bec15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 -- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 msgr2=0x7f1bec15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 0x7f1bec15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 -- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bec06f6d0 msgr2=0x7f1bec1f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bec06f6d0 0x7f1bec1f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 -- 172.21.15.44:0/969280862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec1e3a90 con 0x7f1bec0b3280 2024-07-17T07:14:15.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.168+0000 7f1bf1c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1be400eaa0 tx=0x7f1be400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:15.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.169+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be400cdb0 con 0x7f1bec0b3280 2024-07-17T07:14:15.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.169+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be4004c10 con 0x7f1bec0b3280 2024-07-17T07:14:15.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.169+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4005020 con 0x7f1bec0b3280 2024-07-17T07:14:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.169+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec1e3dd0 con 0x7f1bec0b3280 2024-07-17T07:14:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.169+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec14ed10 con 0x7f1bec0b3280 2024-07-17T07:14:15.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.170+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec0ac7f0 con 0x7f1bec0b3280 2024-07-17T07:14:15.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.173+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f1be40051c0 con 0x7f1bec0b3280 2024-07-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.174+0000 7f1be9600640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 0x7f1bc8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.174+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6521+0+0 (secure 0 0 0) 0x7f1be4099640 con 0x7f1bec0b3280 2024-07-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.175+0000 7f1bf1200640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 0x7f1bc8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:15.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.176+0000 7f1bf1200640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 0x7f1bc8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc033000 tx=0x7f1bdc004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.178+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1bec1e3dd0 con 0x7f1bec0b3280 2024-07-17T07:14:15.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.433+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1bec14eee0 con 0x7f1bec0b3280 2024-07-17T07:14:15.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.982+0000 7f1be9600640 1 -- 172.21.15.44:0/969280862 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v57) ==== 174+0+0 (secure 0 0 0) 0x7f1be4016030 con 0x7f1bec0b3280 2024-07-17T07:14:15.983 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 15 and data pool 16 2024-07-17T07:14:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.991+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 msgr2=0x7f1bc8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.991+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 0x7f1bc8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc033000 tx=0x7f1bdc004730 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 msgr2=0x7f1bec14f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1be400eaa0 tx=0x7f1be400ef60 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 shutdown_connections 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1bc8076cd0 0x7f1bc8079190 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc033000 tx=0x7f1bdc004730 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bec0b3280 0x7f1bec14f230 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bec0ac3a0 0x7f1bec15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 --2- 172.21.15.44:0/969280862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bec06f6d0 0x7f1bec1f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.992+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 >> 172.21.15.44:0/969280862 conn(0x7f1bec0b0eb0 msgr2=0x7f1bec0cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:15.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.995+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 shutdown_connections 2024-07-17T07:14:15.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:15.996+0000 7f1bf3c00640 1 -- 172.21.15.44:0/969280862 wait complete. 2024-07-17T07:14:16.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:14:16.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:14:16.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:14:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.208+0000 7f51be600640 1 -- 172.21.15.44:0/3890373778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b806f710 msgr2=0x7f51b8153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.208+0000 7f51be600640 1 --2- 172.21.15.44:0/3890373778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b806f710 0x7f51b8153a50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f51b815d030 tx=0x7f51a802a560 comp rx=0 tx=0).stop 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 -- 172.21.15.44:0/3890373778 shutdown_connections 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 --2- 172.21.15.44:0/3890373778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b80ac3e0 0x7f51b80ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 --2- 172.21.15.44:0/3890373778 >> v2:172.21.15.175:3300/0 conn(0x7f51b813ed60 0x7f51b813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 --2- 172.21.15.44:0/3890373778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b806f710 0x7f51b8153a50 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 -- 172.21.15.44:0/3890373778 >> 172.21.15.44:0/3890373778 conn(0x7f51b80b0fb0 msgr2=0x7f51b80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 -- 172.21.15.44:0/3890373778 shutdown_connections 2024-07-17T07:14:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.209+0000 7f51be600640 1 -- 172.21.15.44:0/3890373778 wait complete. 2024-07-17T07:14:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.211+0000 7f51be600640 1 Processor -- start 2024-07-17T07:14:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.211+0000 7f51be600640 1 -- start start 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b806f710 0x7f51b815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51b813ed60 0x7f51b814ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51b80ae800 con 0x7f51b80ac3e0 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51b806db70 con 0x7f51b813ed60 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51be600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51b80ae980 con 0x7f51b806f710 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54046/0 (socket says 172.21.15.44:54046) 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51b6a00640 1 -- 172.21.15.44:0/1970390999 learned_addr learned my addr 172.21.15.44:0/1970390999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.212+0000 7f51b7400640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b806f710 0x7f51b815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b7e00640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51b813ed60 0x7f51b814ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 -- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b806f710 msgr2=0x7f51b815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b806f710 0x7f51b815b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 -- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51b813ed60 msgr2=0x7f51b814ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51b813ed60 0x7f51b814ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 -- 172.21.15.44:0/1970390999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b81f7120 con 0x7f51b80ac3e0 2024-07-17T07:14:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51b6a00640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f51a400d950 tx=0x7f51a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4017070 con 0x7f51b80ac3e0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51a4009d20 con 0x7f51b80ac3e0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.213+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b815bec0 con 0x7f51b80ac3e0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.214+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a40106c0 con 0x7f51b80ac3e0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.214+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b815c480 con 0x7f51b80ac3e0 2024-07-17T07:14:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.215+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b813f950 con 0x7f51b80ac3e0 2024-07-17T07:14:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.216+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f51a4010860 con 0x7f51b80ac3e0 2024-07-17T07:14:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.218+0000 7f51af400640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 0x7f51980791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.218+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6621+0+0 (secure 0 0 0) 0x7f51a4098cb0 con 0x7f51b80ac3e0 2024-07-17T07:14:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.218+0000 7f51b7400640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 0x7f51980791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.219+0000 7f51b7400640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 0x7f51980791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51a8033000 tx=0x7f51a8004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:16.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.222+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f51a4061bc0 con 0x7f51b80ac3e0 2024-07-17T07:14:16.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.425+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b806ed80 con 0x7f51b80ac3e0 2024-07-17T07:14:16.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.428+0000 7f51af400640 1 -- 172.21.15.44:0/1970390999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+17671 (secure 0 0 0) 0x7f51a40614c0 con 0x7f51b80ac3e0 2024-07-17T07:14:16.429 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:16.429 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":127,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:14:15.436195+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:14:13.391372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-07-17T07:14:14.431910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:14:16.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 msgr2=0x7f51980791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 0x7f51980791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51a8033000 tx=0x7f51a8004730 comp rx=0 tx=0).stop 2024-07-17T07:14:16.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 msgr2=0x7f51b81f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f51a400d950 tx=0x7f51a400de10 comp rx=0 tx=0).stop 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 shutdown_connections 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5198076d00 0x7f51980791c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51b813ed60 0x7f51b814ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51b80ac3e0 0x7f51b81f6be0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 --2- 172.21.15.44:0/1970390999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51b806f710 0x7f51b815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 >> 172.21.15.44:0/1970390999 conn(0x7f51b80b0fb0 msgr2=0x7f51b81596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 shutdown_connections 2024-07-17T07:14:16.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.431+0000 7f51be600640 1 -- 172.21.15.44:0/1970390999 wait complete. 2024-07-17T07:14:16.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:14:16.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 -- 172.21.15.44:0/2722571040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac003aa0 msgr2=0x7fb3ac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/2722571040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3ac003aa0 0x7fb3ac003ec0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009b50 tx=0x7fb3ac005830 comp rx=0 tx=0).stop 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7400640 1 -- 172.21.15.44:0/2722571040 reap_dead start 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 -- 172.21.15.44:0/2722571040 shutdown_connections 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 -- 172.21.15.44:0/2722571040 >> 172.21.15.44:0/2722571040 conn(0x7fb3c80cfec0 msgr2=0x7fb3c80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 -- 172.21.15.44:0/2722571040 shutdown_connections 2024-07-17T07:14:16.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.657+0000 7fb3c7e00640 1 -- 172.21.15.44:0/2722571040 wait complete. 2024-07-17T07:14:16.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.659+0000 7fb3c7e00640 1 Processor -- start 2024-07-17T07:14:16.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.659+0000 7fb3c7e00640 1 -- start start 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3ac003aa0 0x7fb3c80aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3c815a8c0 0x7fb3c80ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80b4ec0 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c80b9270 con 0x7fb3c815a8c0 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c80b5040 con 0x7fb3ac003aa0 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54056/0 (socket says 172.21.15.44:54056) 2024-07-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 -- 172.21.15.44:0/3411985249 learned_addr learned my addr 172.21.15.44:0/3411985249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:16.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c6a00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3ac003aa0 0x7fb3c80aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 -- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3ac003aa0 msgr2=0x7fb3c80aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3c7400640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3c815a8c0 0x7fb3c80ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3ac003aa0 0x7fb3c80aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.660+0000 7fb3be000640 1 -- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3c815a8c0 msgr2=0x7fb3c80ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3be000640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3c815a8c0 0x7fb3c80ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3be000640 1 -- 172.21.15.44:0/3411985249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80cc830 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3be000640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000d740 tx=0x7fb3b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0018070 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3b000de50 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000b6c0 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c80ca5b0 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.661+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80cab70 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.663+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c815e320 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.664+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7fb3b001b070 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.664+0000 7fb3bf400640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 0x7fb3a4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:16.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.665+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 6621+0+0 (secure 0 0 0) 0x7fb3b0097f20 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.665+0000 7fb3c6a00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 0x7fb3a4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:16.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.666+0000 7fb3c6a00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 0x7fb3a4079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8035000 tx=0x7fb3b802d040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.669+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb3b0060860 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.872+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3c80b5d70 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.875+0000 7fb3bf400640 1 -- 172.21.15.44:0/3411985249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+17671 (secure 0 0 0) 0x7fb3b00690f0 con 0x7fb3c80cc3b0 2024-07-17T07:14:16.875 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:16.875 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":127,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:14:15.436195+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:14:13.391372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-07-17T07:14:14.431910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 msgr2=0x7fb3a4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 0x7fb3a4079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8035000 tx=0x7fb3b802d040 comp rx=0 tx=0).stop 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 msgr2=0x7fb3c81f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000d740 tx=0x7fb3b000dc00 comp rx=0 tx=0).stop 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 shutdown_connections 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fb3a4076cd0 0x7fb3a4079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3c815a8c0 0x7fb3c80ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c80cc3b0 0x7fb3c81f3fa0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 --2- 172.21.15.44:0/3411985249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3ac003aa0 0x7fb3c80aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 >> 172.21.15.44:0/3411985249 conn(0x7fb3c80cfec0 msgr2=0x7fb3c815f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 shutdown_connections 2024-07-17T07:14:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:16.878+0000 7fb3c7e00640 1 -- 172.21.15.44:0/3411985249 wait complete. 2024-07-17T07:14:16.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 -- 172.21.15.44:0/425061485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 msgr2=0x7f5a9c072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 --2- 172.21.15.44:0/425061485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c072e90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0b42b0 tx=0x7f5a8402a530 comp rx=0 tx=0).stop 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 -- 172.21.15.44:0/425061485 shutdown_connections 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 --2- 172.21.15.44:0/425061485 >> v2:172.21.15.175:3300/0 conn(0x7f5a9c0733d0 0x7f5a9c163520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 --2- 172.21.15.44:0/425061485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c072e90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 -- 172.21.15.44:0/425061485 >> 172.21.15.44:0/425061485 conn(0x7f5a9c06ce80 msgr2=0x7f5a9c06d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 -- 172.21.15.44:0/425061485 shutdown_connections 2024-07-17T07:14:17.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.114+0000 7f5aa3200640 1 -- 172.21.15.44:0/425061485 wait complete. 2024-07-17T07:14:17.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.116+0000 7f5aa3200640 1 Processor -- start 2024-07-17T07:14:17.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.116+0000 7f5aa3200640 1 -- start start 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a9c0733d0 0x7f5a9c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a9c159990 0x7f5a9c1f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c163be0 con 0x7f5a9c15c280 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c163a60 con 0x7f5a9c159990 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c163d60 con 0x7f5a9c0733d0 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.117+0000 7f5aa1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54078/0 (socket says 172.21.15.44:54078) 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 -- 172.21.15.44:0/1080732092 learned_addr learned my addr 172.21.15.44:0/1080732092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:17.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5a93e00640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a9c159990 0x7f5a9c1f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5a9be00640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a9c0733d0 0x7f5a9c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 -- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a9c0733d0 msgr2=0x7f5a9c153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a9c0733d0 0x7f5a9c153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 -- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a9c159990 msgr2=0x7f5a9c1f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a9c159990 0x7f5a9c1f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 -- 172.21.15.44:0/1080732092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c0cc190 con 0x7f5a9c15c280 2024-07-17T07:14:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.118+0000 7f5aa1200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400cb00 tx=0x7f5a9400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.119+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94004b20 con 0x7f5a9c15c280 2024-07-17T07:14:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.119+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a94004db0 con 0x7f5a9c15c280 2024-07-17T07:14:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.119+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94005280 con 0x7f5a9c15c280 2024-07-17T07:14:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.119+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c13a330 con 0x7f5a9c15c280 2024-07-17T07:14:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.119+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c0cc710 con 0x7f5a9c15c280 2024-07-17T07:14:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.120+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9c158100 con 0x7f5a9c15c280 2024-07-17T07:14:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.121+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f5a94005420 con 0x7f5a9c15c280 2024-07-17T07:14:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.122+0000 7f5a98c00640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 0x7f5a78079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:17.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.123+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f5a94099160 con 0x7f5a9c15c280 2024-07-17T07:14:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.123+0000 7f5a9be00640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 0x7f5a78079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.124+0000 7f5a9be00640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 0x7f5a78079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c000f60 tx=0x7f5a8c009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:17.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.127+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5a94014030 con 0x7f5a9c15c280 2024-07-17T07:14:17.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:17.378+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5a9c157e20 con 0x7f5a9c15c280 2024-07-17T07:14:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.012+0000 7f5a98c00640 1 -- 172.21.15.44:0/1080732092 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f5a94061fe0 con 0x7f5a9c15c280 2024-07-17T07:14:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 msgr2=0x7f5a78079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 0x7f5a78079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c000f60 tx=0x7f5a8c009040 comp rx=0 tx=0).stop 2024-07-17T07:14:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 msgr2=0x7f5a9c1f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400cb00 tx=0x7f5a9400a250 comp rx=0 tx=0).stop 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 shutdown_connections 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f5a78076cd0 0x7f5a78079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a9c15c280 0x7f5a9c1f2860 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a9c159990 0x7f5a9c1f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 --2- 172.21.15.44:0/1080732092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a9c0733d0 0x7f5a9c153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.017+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 >> 172.21.15.44:0/1080732092 conn(0x7f5a9c06ce80 msgr2=0x7f5a9c0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.018+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 shutdown_connections 2024-07-17T07:14:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.018+0000 7f5aa3200640 1 -- 172.21.15.44:0/1080732092 wait complete. 2024-07-17T07:14:18.041 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:14:18.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.256+0000 7f31f0a00640 1 -- 172.21.15.44:0/2706172601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec1503c0 msgr2=0x7f31ec0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.256+0000 7f31f0a00640 1 --2- 172.21.15.44:0/2706172601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec1503c0 0x7f31ec0ae340 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f31ec070830 tx=0x7f31d802a560 comp rx=0 tx=0).stop 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 -- 172.21.15.44:0/2706172601 shutdown_connections 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 --2- 172.21.15.44:0/2706172601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec06d420 0x7f31ec06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 --2- 172.21.15.44:0/2706172601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec1503c0 0x7f31ec0ae340 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 --2- 172.21.15.44:0/2706172601 >> v2:172.21.15.175:3300/0 conn(0x7f31ec06f4d0 0x7f31ec15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 -- 172.21.15.44:0/2706172601 >> 172.21.15.44:0/2706172601 conn(0x7f31ec0cf0d0 msgr2=0x7f31ec0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 -- 172.21.15.44:0/2706172601 shutdown_connections 2024-07-17T07:14:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.257+0000 7f31f0a00640 1 -- 172.21.15.44:0/2706172601 wait complete. 2024-07-17T07:14:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.258+0000 7f31f0a00640 1 Processor -- start 2024-07-17T07:14:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.259+0000 7f31f0a00640 1 -- start start 2024-07-17T07:14:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.259+0000 7f31f0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.259+0000 7f31f0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06f4d0 0x7f31ec15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31f0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 0x7f31ec14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31f0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec0ac3a0 con 0x7f31ec06d420 2024-07-17T07:14:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31f0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec0ac220 con 0x7f31ec06f4d0 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31f0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec0ac520 con 0x7f31ec1503c0 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54094/0 (socket says 172.21.15.44:54094) 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 -- 172.21.15.44:0/3350813588 learned_addr learned my addr 172.21.15.44:0/3350813588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31ebe00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 0x7f31ec14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eaa00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06f4d0 0x7f31ec15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 -- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 msgr2=0x7f31ec14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 0x7f31ec14efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 -- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06f4d0 msgr2=0x7f31ec15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06f4d0 0x7f31ec15bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31eb400640 1 -- 172.21.15.44:0/3350813588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec0ceb30 con 0x7f31ec06d420 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.260+0000 7f31ebe00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 0x7f31ec14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31eb400640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b4a0 tx=0x7f31dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc014070 con 0x7f31ec06d420 2024-07-17T07:14:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31dc010700 con 0x7f31ec06d420 2024-07-17T07:14:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec073230 con 0x7f31ec06d420 2024-07-17T07:14:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc010b50 con 0x7f31ec06d420 2024-07-17T07:14:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.261+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec0ce0d0 con 0x7f31ec06d420 2024-07-17T07:14:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.263+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec15a450 con 0x7f31ec06d420 2024-07-17T07:14:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.263+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f31dc017070 con 0x7f31ec06d420 2024-07-17T07:14:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.265+0000 7f31e3400640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 0x7f31cc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.265+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f31dc098340 con 0x7f31ec06d420 2024-07-17T07:14:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.265+0000 7f31eaa00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 0x7f31cc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:18.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.267+0000 7f31eaa00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 0x7f31cc079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31d8033000 tx=0x7f31d8004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:18.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.269+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f31dc061360 con 0x7f31ec06d420 2024-07-17T07:14:18.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:18.519+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f31ec0cffe0 con 0x7f31ec06d420 2024-07-17T07:14:19.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.016+0000 7f31e3400640 1 -- 172.21.15.44:0/3350813588 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f31dc060c60 con 0x7f31ec06d420 2024-07-17T07:14:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.019+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 msgr2=0x7f31cc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.019+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 0x7f31cc079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31d8033000 tx=0x7f31d8004730 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.019+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 msgr2=0x7f31ec0ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.019+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b4a0 tx=0x7f31dc00b960 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 shutdown_connections 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f31cc076cd0 0x7f31cc079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31ec1503c0 0x7f31ec14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31ec06f4d0 0x7f31ec15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 --2- 172.21.15.44:0/3350813588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31ec06d420 0x7f31ec0ce5f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 >> 172.21.15.44:0/3350813588 conn(0x7f31ec0cf0d0 msgr2=0x7f31ec0aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 shutdown_connections 2024-07-17T07:14:19.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.020+0000 7f31f0a00640 1 -- 172.21.15.44:0/3350813588 wait complete. 2024-07-17T07:14:19.041 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:14:19.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:14:19.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 -- 172.21.15.44:0/437027065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00b9060 msgr2=0x7f64e015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 --2- 172.21.15.44:0/437027065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00b9060 0x7f64e015d800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802a610 comp rx=0 tx=0).stop 2024-07-17T07:14:19.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 -- 172.21.15.44:0/437027065 shutdown_connections 2024-07-17T07:14:19.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 --2- 172.21.15.44:0/437027065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00b9060 0x7f64e015d800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 --2- 172.21.15.44:0/437027065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00ce5c0 0x7f64e015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 -- 172.21.15.44:0/437027065 >> 172.21.15.44:0/437027065 conn(0x7f64e00cfec0 msgr2=0x7f64e00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:19.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 -- 172.21.15.44:0/437027065 shutdown_connections 2024-07-17T07:14:19.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.250+0000 7f64df200640 1 -- 172.21.15.44:0/437027065 wait complete. 2024-07-17T07:14:19.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.252+0000 7f64df200640 1 Processor -- start 2024-07-17T07:14:19.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.252+0000 7f64df200640 1 -- start start 2024-07-17T07:14:19.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00b8c80 0x7f64e014cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 0x7f64e00b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64e015e0d0 con 0x7f64e00b9060 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64e015df50 con 0x7f64e00ce5c0 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64df200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64e015e250 con 0x7f64e00b8c80 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.253+0000 7f64dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35684/0 (socket says 172.21.15.44:35684) 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 -- 172.21.15.44:0/1602160423 learned_addr learned my addr 172.21.15.44:0/1602160423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dde00640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00b8c80 0x7f64e014cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64de800640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 0x7f64e00b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 -- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00b8c80 msgr2=0x7f64e014cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00b8c80 0x7f64e014cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 -- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 msgr2=0x7f64e00b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 0x7f64e00b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 -- 172.21.15.44:0/1602160423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e014e6f0 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64de800640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 0x7f64e00b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.254+0000 7f64dd400640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f64c800eb30 tx=0x7f64c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.255+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8010430 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.255+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64c80105d0 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.255+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8010a20 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.255+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e014e9d0 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.255+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e014ef90 con 0x7f64e00b9060 2024-07-17T07:14:19.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.256+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e015e700 con 0x7f64e00b9060 2024-07-17T07:14:19.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.257+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f64c8010bc0 con 0x7f64e00b9060 2024-07-17T07:14:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.258+0000 7f64d6000640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 0x7f64bc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.258+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f64c8099640 con 0x7f64e00b9060 2024-07-17T07:14:19.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.259+0000 7f64dde00640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 0x7f64bc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.260+0000 7f64dde00640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 0x7f64bc079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64e00adac0 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:19.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.262+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f64c8062550 con 0x7f64e00b9060 2024-07-17T07:14:19.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.514+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e00b5d70 con 0x7f64e00b9060 2024-07-17T07:14:19.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.516+0000 7f64d6000640 1 -- 172.21.15.44:0/1602160423 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f64c8061e50 con 0x7f64e00b9060 2024-07-17T07:14:19.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:19.516 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:14:19.516 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.518+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 msgr2=0x7f64bc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.518+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 0x7f64bc079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64e00adac0 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.518+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 msgr2=0x7f64e00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.518+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f64c800eb30 tx=0x7f64c800c2d0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 shutdown_connections 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f64bc076cd0 0x7f64bc079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64e00ce5c0 0x7f64e00b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64e00b9060 0x7f64e00b6310 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 --2- 172.21.15.44:0/1602160423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64e00b8c80 0x7f64e014cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 >> 172.21.15.44:0/1602160423 conn(0x7f64e00cfec0 msgr2=0x7f64e00b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 shutdown_connections 2024-07-17T07:14:19.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.519+0000 7f64df200640 1 -- 172.21.15.44:0/1602160423 wait complete. 2024-07-17T07:14:19.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:14:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 -- 172.21.15.44:0/3321462653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf540 msgr2=0x7f3adc0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3321462653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf540 0x7f3adc0b3870 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0cf040 tx=0x7f3ad002a580 comp rx=0 tx=0).stop 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 -- 172.21.15.44:0/3321462653 shutdown_connections 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3321462653 >> v2:172.21.15.175:3300/0 conn(0x7f3adc0cf920 0x7f3adc141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3321462653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf540 0x7f3adc0b3870 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 -- 172.21.15.44:0/3321462653 >> 172.21.15.44:0/3321462653 conn(0x7f3adc0b0e80 msgr2=0x7f3adc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 -- 172.21.15.44:0/3321462653 shutdown_connections 2024-07-17T07:14:19.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.745+0000 7f3ae3200640 1 -- 172.21.15.44:0/3321462653 wait complete. 2024-07-17T07:14:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.747+0000 7f3ae3200640 1 Processor -- start 2024-07-17T07:14:19.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.747+0000 7f3ae3200640 1 -- start start 2024-07-17T07:14:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3adc0b3db0 0x7f3adc1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3adc0cf540 0x7f3adc1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc0ae170 con 0x7f3adc0cf920 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc0adff0 con 0x7f3adc0cf540 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc0ae2f0 con 0x7f3adc0b3db0 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35704/0 (socket says 172.21.15.44:35704) 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae2800640 1 -- 172.21.15.44:0/3746832866 learned_addr learned my addr 172.21.15.44:0/3746832866 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.748+0000 7f3ae1400640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3adc0cf540 0x7f3adc1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae1e00640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3adc0b3db0 0x7f3adc1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 -- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3adc0b3db0 msgr2=0x7f3adc1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3adc0b3db0 0x7f3adc1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 -- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3adc0cf540 msgr2=0x7f3adc1f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3adc0cf540 0x7f3adc1f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 -- 172.21.15.44:0/3746832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc073e30 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae2800640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800cab0 tx=0x7f3ad800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8007d90 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad800f540 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800f990 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.749+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc073870 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.750+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc15b7d0 con 0x7f3adc0cf920 2024-07-17T07:14:19.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.751+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc0aefd0 con 0x7f3adc0cf920 2024-07-17T07:14:19.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.752+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f3ad800fb30 con 0x7f3adc0cf920 2024-07-17T07:14:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.753+0000 7f3ad6000640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 0x7f3ac00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.753+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f3ad8099270 con 0x7f3adc0cf920 2024-07-17T07:14:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.754+0000 7f3ae1e00640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 0x7f3ac00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:19.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.755+0000 7f3ae1e00640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 0x7f3ac00791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0033000 tx=0x7f3ad0004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:19.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.757+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3ad8014070 con 0x7f3adc0cf920 2024-07-17T07:14:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.968+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3adc141170 con 0x7f3adc0cf920 2024-07-17T07:14:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.971+0000 7f3ad6000640 1 -- 172.21.15.44:0/3746832866 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+17658 (secure 0 0 0) 0x7f3ad800fe20 con 0x7f3adc0cf920 2024-07-17T07:14:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:19.971 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":128,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:14:16.963950+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:14:13.391372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-07-17T07:14:14.431910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:14:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.973+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 msgr2=0x7f3ac00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.973+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 0x7f3ac00791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0033000 tx=0x7f3ad0004730 comp rx=0 tx=0).stop 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.973+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 msgr2=0x7f3adc1f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.973+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800cab0 tx=0x7f3ad800cf70 comp rx=0 tx=0).stop 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 shutdown_connections 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3ac0076d00 0x7f3ac00791c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3adc0cf920 0x7f3adc1f78c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3adc0cf540 0x7f3adc1f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 --2- 172.21.15.44:0/3746832866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3adc0b3db0 0x7f3adc1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 >> 172.21.15.44:0/3746832866 conn(0x7f3adc0b0e80 msgr2=0x7f3adc09ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 shutdown_connections 2024-07-17T07:14:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:19.974+0000 7f3ae3200640 1 -- 172.21.15.44:0/3746832866 wait complete. 2024-07-17T07:14:19.991 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:14:19.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.204+0000 7ffa0d800640 1 -- 172.21.15.44:0/1868993600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9ec004d40 msgr2=0x7ff9ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.204+0000 7ffa0d800640 1 --2- 172.21.15.44:0/1868993600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9ec004d40 0x7ff9ec0051c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009e20 tx=0x7ff9ec005830 comp rx=0 tx=0).stop 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.205+0000 7ffa07400640 1 -- 172.21.15.44:0/1868993600 reap_dead start 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.205+0000 7ffa0d800640 1 -- 172.21.15.44:0/1868993600 shutdown_connections 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.205+0000 7ffa0d800640 1 -- 172.21.15.44:0/1868993600 >> 172.21.15.44:0/1868993600 conn(0x7ffa080b3020 msgr2=0x7ffa080b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.205+0000 7ffa0d800640 1 -- 172.21.15.44:0/1868993600 shutdown_connections 2024-07-17T07:14:20.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.205+0000 7ffa0d800640 1 -- 172.21.15.44:0/1868993600 wait complete. 2024-07-17T07:14:20.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.206+0000 7ffa0d800640 1 Processor -- start 2024-07-17T07:14:20.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.207+0000 7ffa0d800640 1 -- start start 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ec004d40 0x7ffa080c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0809ead0 0x7ffa08152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0806df10 con 0x7ffa0809ef50 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0806dd90 con 0x7ff9ec004d40 2024-07-17T07:14:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa0d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0806e090 con 0x7ffa0809ead0 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35720/0 (socket says 172.21.15.44:35720) 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 -- 172.21.15.44:0/3038094774 learned_addr learned my addr 172.21.15.44:0/3038094774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa06a00640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ec004d40 0x7ffa080c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa06000640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0809ead0 0x7ffa08152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 -- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0809ead0 msgr2=0x7ffa08152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0809ead0 0x7ffa08152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 -- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ec004d40 msgr2=0x7ffa080c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ec004d40 0x7ffa080c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.208+0000 7ffa07400640 1 -- 172.21.15.44:0/3038094774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080c94c0 con 0x7ffa0809ef50 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ffa07400640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffa00002230 tx=0x7ffa00034680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00043070 con 0x7ffa0809ef50 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa00003970 con 0x7ffa0809ef50 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00038040 con 0x7ffa0809ef50 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081f3d60 con 0x7ffa0809ef50 2024-07-17T07:14:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.209+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa08153080 con 0x7ffa0809ef50 2024-07-17T07:14:20.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.210+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa08070de0 con 0x7ffa0809ef50 2024-07-17T07:14:20.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.212+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7ffa00046070 con 0x7ffa0809ef50 2024-07-17T07:14:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.213+0000 7ff9fea00640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 0x7ff9e4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.213+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7ffa000b8480 con 0x7ffa0809ef50 2024-07-17T07:14:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.214+0000 7ffa06a00640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 0x7ff9e4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.215+0000 7ffa06a00640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 0x7ff9e4079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009b20 tx=0x7ff9f0005f90 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:20.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.218+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffa00080dc0 con 0x7ffa0809ef50 2024-07-17T07:14:20.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.425+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 --> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa08071690 con 0x7ff9e4076cd0 2024-07-17T07:14:20.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.441+0000 7ff9fea00640 1 -- 172.21.15.44:0/3038094774 <== mgr.6202 v2:172.21.15.175:6800/1224757683 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7ffa08071690 con 0x7ff9e4076cd0 2024-07-17T07:14:20.444 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:14:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.447+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 msgr2=0x7ff9e4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.447+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 0x7ff9e4079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009b20 tx=0x7ff9f0005f90 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.447+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 msgr2=0x7ffa08152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.447+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffa00002230 tx=0x7ffa00034680 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 shutdown_connections 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff9e4076cd0 0x7ff9e4079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0809ef50 0x7ffa08152770 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa0809ead0 0x7ffa08152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 --2- 172.21.15.44:0/3038094774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ec004d40 0x7ffa080c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 >> 172.21.15.44:0/3038094774 conn(0x7ffa080b3020 msgr2=0x7ffa080d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 shutdown_connections 2024-07-17T07:14:20.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.448+0000 7ffa0d800640 1 -- 172.21.15.44:0/3038094774 wait complete. 2024-07-17T07:14:20.469 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:14:19.631220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":248,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2459,"ondisk_log_size":2459,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333752,"kb_used_data":13508,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754640968,"statfs":{"total":773094113280,"available":772752351232,"internally_reserved":0,"allocated":13832192,"data_stored":7831592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":327798320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.693212"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004675+0000","last_change":"2024-07-17T07:14:16.012995+0000","last_active":"2024-07-17T07:14:17.004675+0000","last_peered":"2024-07-17T07:14:17.004675+0000","last_clean":"2024-07-17T07:14:17.004675+0000","last_became_active":"2024-07-17T07:14:16.010891+0000","last_became_peered":"2024-07-17T07:14:16.010891+0000","last_unstale":"2024-07-17T07:14:17.004675+0000","last_undegraded":"2024-07-17T07:14:17.004675+0000","last_fullsized":"2024-07-17T07:14:17.004675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:53.227079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989707+0000","last_change":"2024-07-17T07:14:15.996008+0000","last_active":"2024-07-17T07:14:16.989707+0000","last_peered":"2024-07-17T07:14:16.989707+0000","last_clean":"2024-07-17T07:14:16.989707+0000","last_became_active":"2024-07-17T07:14:15.995651+0000","last_became_peered":"2024-07-17T07:14:15.995651+0000","last_unstale":"2024-07-17T07:14:16.989707+0000","last_undegraded":"2024-07-17T07:14:16.989707+0000","last_fullsized":"2024-07-17T07:14:16.989707+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:28.319554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004657+0000","last_change":"2024-07-17T07:14:16.012316+0000","last_active":"2024-07-17T07:14:17.004657+0000","last_peered":"2024-07-17T07:14:17.004657+0000","last_clean":"2024-07-17T07:14:17.004657+0000","last_became_active":"2024-07-17T07:14:16.011029+0000","last_became_peered":"2024-07-17T07:14:16.011029+0000","last_unstale":"2024-07-17T07:14:17.004657+0000","last_undegraded":"2024-07-17T07:14:17.004657+0000","last_fullsized":"2024-07-17T07:14:17.004657+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:33:31.367841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529889+0000","last_change":"2024-07-17T07:14:16.006107+0000","last_active":"2024-07-17T07:14:17.529889+0000","last_peered":"2024-07-17T07:14:17.529889+0000","last_clean":"2024-07-17T07:14:17.529889+0000","last_became_active":"2024-07-17T07:14:16.005816+0000","last_became_peered":"2024-07-17T07:14:16.005816+0000","last_unstale":"2024-07-17T07:14:17.529889+0000","last_undegraded":"2024-07-17T07:14:17.529889+0000","last_fullsized":"2024-07-17T07:14:17.529889+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:57:59.997536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984992+0000","last_change":"2024-07-17T07:14:16.018765+0000","last_active":"2024-07-17T07:14:16.984992+0000","last_peered":"2024-07-17T07:14:16.984992+0000","last_clean":"2024-07-17T07:14:16.984992+0000","last_became_active":"2024-07-17T07:14:16.018423+0000","last_became_peered":"2024-07-17T07:14:16.018423+0000","last_unstale":"2024-07-17T07:14:16.984992+0000","last_undegraded":"2024-07-17T07:14:16.984992+0000","last_fullsized":"2024-07-17T07:14:16.984992+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:26.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988579+0000","last_change":"2024-07-17T07:14:16.006160+0000","last_active":"2024-07-17T07:14:16.988579+0000","last_peered":"2024-07-17T07:14:16.988579+0000","last_clean":"2024-07-17T07:14:16.988579+0000","last_became_active":"2024-07-17T07:14:16.005848+0000","last_became_peered":"2024-07-17T07:14:16.005848+0000","last_unstale":"2024-07-17T07:14:16.988579+0000","last_undegraded":"2024-07-17T07:14:16.988579+0000","last_fullsized":"2024-07-17T07:14:16.988579+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:34:41.996529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988426+0000","last_change":"2024-07-17T07:14:16.013168+0000","last_active":"2024-07-17T07:14:16.988426+0000","last_peered":"2024-07-17T07:14:16.988426+0000","last_clean":"2024-07-17T07:14:16.988426+0000","last_became_active":"2024-07-17T07:14:16.012754+0000","last_became_peered":"2024-07-17T07:14:16.012754+0000","last_unstale":"2024-07-17T07:14:16.988426+0000","last_undegraded":"2024-07-17T07:14:16.988426+0000","last_fullsized":"2024-07-17T07:14:16.988426+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:01.799337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.982821+0000","last_change":"2024-07-17T07:14:16.004033+0000","last_active":"2024-07-17T07:14:16.982821+0000","last_peered":"2024-07-17T07:14:16.982821+0000","last_clean":"2024-07-17T07:14:16.982821+0000","last_became_active":"2024-07-17T07:14:16.003645+0000","last_became_peered":"2024-07-17T07:14:16.003645+0000","last_unstale":"2024-07-17T07:14:16.982821+0000","last_undegraded":"2024-07-17T07:14:16.982821+0000","last_fullsized":"2024-07-17T07:14:16.982821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:56.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989791+0000","last_change":"2024-07-17T07:14:16.010867+0000","last_active":"2024-07-17T07:14:16.989791+0000","last_peered":"2024-07-17T07:14:16.989791+0000","last_clean":"2024-07-17T07:14:16.989791+0000","last_became_active":"2024-07-17T07:14:16.010406+0000","last_became_peered":"2024-07-17T07:14:16.010406+0000","last_unstale":"2024-07-17T07:14:16.989791+0000","last_undegraded":"2024-07-17T07:14:16.989791+0000","last_fullsized":"2024-07-17T07:14:16.989791+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:11.252677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985429+0000","last_change":"2024-07-17T07:14:16.007400+0000","last_active":"2024-07-17T07:14:16.985429+0000","last_peered":"2024-07-17T07:14:16.985429+0000","last_clean":"2024-07-17T07:14:16.985429+0000","last_became_active":"2024-07-17T07:14:16.007102+0000","last_became_peered":"2024-07-17T07:14:16.007102+0000","last_unstale":"2024-07-17T07:14:16.985429+0000","last_undegraded":"2024-07-17T07:14:16.985429+0000","last_fullsized":"2024-07-17T07:14:16.985429+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:05.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010229+0000","last_change":"2024-07-17T07:14:16.008251+0000","last_active":"2024-07-17T07:14:17.010229+0000","last_peered":"2024-07-17T07:14:17.010229+0000","last_clean":"2024-07-17T07:14:17.010229+0000","last_became_active":"2024-07-17T07:14:16.007946+0000","last_became_peered":"2024-07-17T07:14:16.007946+0000","last_unstale":"2024-07-17T07:14:17.010229+0000","last_undegraded":"2024-07-17T07:14:17.010229+0000","last_fullsized":"2024-07-17T07:14:17.010229+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:03.923444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.999140+0000","last_change":"2024-07-17T07:14:15.992342+0000","last_active":"2024-07-17T07:14:15.999140+0000","last_peered":"2024-07-17T07:14:15.999140+0000","last_clean":"2024-07-17T07:14:15.999140+0000","last_became_active":"2024-07-17T07:14:15.991920+0000","last_became_peered":"2024-07-17T07:14:15.991920+0000","last_unstale":"2024-07-17T07:14:15.999140+0000","last_undegraded":"2024-07-17T07:14:15.999140+0000","last_fullsized":"2024-07-17T07:14:15.999140+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:30.036103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529391+0000","last_change":"2024-07-17T07:14:16.007633+0000","last_active":"2024-07-17T07:14:17.529391+0000","last_peered":"2024-07-17T07:14:17.529391+0000","last_clean":"2024-07-17T07:14:17.529391+0000","last_became_active":"2024-07-17T07:14:16.007305+0000","last_became_peered":"2024-07-17T07:14:16.007305+0000","last_unstale":"2024-07-17T07:14:17.529391+0000","last_undegraded":"2024-07-17T07:14:17.529391+0000","last_fullsized":"2024-07-17T07:14:17.529391+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:50.162207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238463+0000","last_change":"2024-07-17T07:14:16.012002+0000","last_active":"2024-07-17T07:14:17.238463+0000","last_peered":"2024-07-17T07:14:17.238463+0000","last_clean":"2024-07-17T07:14:17.238463+0000","last_became_active":"2024-07-17T07:14:16.011428+0000","last_became_peered":"2024-07-17T07:14:16.011428+0000","last_unstale":"2024-07-17T07:14:17.238463+0000","last_undegraded":"2024-07-17T07:14:17.238463+0000","last_fullsized":"2024-07-17T07:14:17.238463+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:50.385822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238381+0000","last_change":"2024-07-17T07:14:16.006815+0000","last_active":"2024-07-17T07:14:17.238381+0000","last_peered":"2024-07-17T07:14:17.238381+0000","last_clean":"2024-07-17T07:14:17.238381+0000","last_became_active":"2024-07-17T07:14:16.006511+0000","last_became_peered":"2024-07-17T07:14:16.006511+0000","last_unstale":"2024-07-17T07:14:17.238381+0000","last_undegraded":"2024-07-17T07:14:17.238381+0000","last_fullsized":"2024-07-17T07:14:17.238381+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:12.334189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238410+0000","last_change":"2024-07-17T07:14:15.994936+0000","last_active":"2024-07-17T07:14:17.238410+0000","last_peered":"2024-07-17T07:14:17.238410+0000","last_clean":"2024-07-17T07:14:17.238410+0000","last_became_active":"2024-07-17T07:14:15.994399+0000","last_became_peered":"2024-07-17T07:14:15.994399+0000","last_unstale":"2024-07-17T07:14:17.238410+0000","last_undegraded":"2024-07-17T07:14:17.238410+0000","last_fullsized":"2024-07-17T07:14:17.238410+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:12.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004825+0000","last_change":"2024-07-17T07:14:16.012515+0000","last_active":"2024-07-17T07:14:17.004825+0000","last_peered":"2024-07-17T07:14:17.004825+0000","last_clean":"2024-07-17T07:14:17.004825+0000","last_became_active":"2024-07-17T07:14:16.011828+0000","last_became_peered":"2024-07-17T07:14:16.011828+0000","last_unstale":"2024-07-17T07:14:17.004825+0000","last_undegraded":"2024-07-17T07:14:17.004825+0000","last_fullsized":"2024-07-17T07:14:17.004825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:39.615469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.990090+0000","last_change":"2024-07-17T07:14:16.004510+0000","last_active":"2024-07-17T07:14:16.990090+0000","last_peered":"2024-07-17T07:14:16.990090+0000","last_clean":"2024-07-17T07:14:16.990090+0000","last_became_active":"2024-07-17T07:14:16.003956+0000","last_became_peered":"2024-07-17T07:14:16.003956+0000","last_unstale":"2024-07-17T07:14:16.990090+0000","last_undegraded":"2024-07-17T07:14:16.990090+0000","last_fullsized":"2024-07-17T07:14:16.990090+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:55.557627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004841+0000","last_change":"2024-07-17T07:14:16.002351+0000","last_active":"2024-07-17T07:14:17.004841+0000","last_peered":"2024-07-17T07:14:17.004841+0000","last_clean":"2024-07-17T07:14:17.004841+0000","last_became_active":"2024-07-17T07:14:16.001990+0000","last_became_peered":"2024-07-17T07:14:16.001990+0000","last_unstale":"2024-07-17T07:14:17.004841+0000","last_undegraded":"2024-07-17T07:14:17.004841+0000","last_fullsized":"2024-07-17T07:14:17.004841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:43.660355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.000376+0000","last_change":"2024-07-17T07:14:15.995908+0000","last_active":"2024-07-17T07:14:16.000376+0000","last_peered":"2024-07-17T07:14:16.000376+0000","last_clean":"2024-07-17T07:14:16.000376+0000","last_became_active":"2024-07-17T07:14:15.995612+0000","last_became_peered":"2024-07-17T07:14:15.995612+0000","last_unstale":"2024-07-17T07:14:16.000376+0000","last_undegraded":"2024-07-17T07:14:16.000376+0000","last_fullsized":"2024-07-17T07:14:16.000376+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:45.086371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984563+0000","last_change":"2024-07-17T07:14:16.001364+0000","last_active":"2024-07-17T07:14:16.984563+0000","last_peered":"2024-07-17T07:14:16.984563+0000","last_clean":"2024-07-17T07:14:16.984563+0000","last_became_active":"2024-07-17T07:14:16.001057+0000","last_became_peered":"2024-07-17T07:14:16.001057+0000","last_unstale":"2024-07-17T07:14:16.984563+0000","last_undegraded":"2024-07-17T07:14:16.984563+0000","last_fullsized":"2024-07-17T07:14:16.984563+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:13.187794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984881+0000","last_change":"2024-07-17T07:14:16.017614+0000","last_active":"2024-07-17T07:14:16.984881+0000","last_peered":"2024-07-17T07:14:16.984881+0000","last_clean":"2024-07-17T07:14:16.984881+0000","last_became_active":"2024-07-17T07:14:16.012502+0000","last_became_peered":"2024-07-17T07:14:16.012502+0000","last_unstale":"2024-07-17T07:14:16.984881+0000","last_undegraded":"2024-07-17T07:14:16.984881+0000","last_fullsized":"2024-07-17T07:14:16.984881+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:41.089940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004241+0000","last_change":"2024-07-17T07:14:16.010144+0000","last_active":"2024-07-17T07:14:17.004241+0000","last_peered":"2024-07-17T07:14:17.004241+0000","last_clean":"2024-07-17T07:14:17.004241+0000","last_became_active":"2024-07-17T07:14:16.009857+0000","last_became_peered":"2024-07-17T07:14:16.009857+0000","last_unstale":"2024-07-17T07:14:17.004241+0000","last_undegraded":"2024-07-17T07:14:17.004241+0000","last_fullsized":"2024-07-17T07:14:17.004241+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:14.750206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985657+0000","last_change":"2024-07-17T07:14:16.005332+0000","last_active":"2024-07-17T07:14:16.985657+0000","last_peered":"2024-07-17T07:14:16.985657+0000","last_clean":"2024-07-17T07:14:16.985657+0000","last_became_active":"2024-07-17T07:14:16.001688+0000","last_became_peered":"2024-07-17T07:14:16.001688+0000","last_unstale":"2024-07-17T07:14:16.985657+0000","last_undegraded":"2024-07-17T07:14:16.985657+0000","last_fullsized":"2024-07-17T07:14:16.985657+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:35:51.182145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.001061+0000","last_change":"2024-07-17T07:14:15.996810+0000","last_active":"2024-07-17T07:14:16.001061+0000","last_peered":"2024-07-17T07:14:16.001061+0000","last_clean":"2024-07-17T07:14:16.001061+0000","last_became_active":"2024-07-17T07:14:15.996526+0000","last_became_peered":"2024-07-17T07:14:15.996526+0000","last_unstale":"2024-07-17T07:14:16.001061+0000","last_undegraded":"2024-07-17T07:14:16.001061+0000","last_fullsized":"2024-07-17T07:14:16.001061+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:44.895464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.999119+0000","last_change":"2024-07-17T07:14:15.990517+0000","last_active":"2024-07-17T07:14:15.999119+0000","last_peered":"2024-07-17T07:14:15.999119+0000","last_clean":"2024-07-17T07:14:15.999119+0000","last_became_active":"2024-07-17T07:14:15.990226+0000","last_became_peered":"2024-07-17T07:14:15.990226+0000","last_unstale":"2024-07-17T07:14:15.999119+0000","last_undegraded":"2024-07-17T07:14:15.999119+0000","last_fullsized":"2024-07-17T07:14:15.999119+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:28:50.571361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010850+0000","last_change":"2024-07-17T07:14:16.006259+0000","last_active":"2024-07-17T07:14:17.010850+0000","last_peered":"2024-07-17T07:14:17.010850+0000","last_clean":"2024-07-17T07:14:17.010850+0000","last_became_active":"2024-07-17T07:14:16.005785+0000","last_became_peered":"2024-07-17T07:14:16.005785+0000","last_unstale":"2024-07-17T07:14:17.010850+0000","last_undegraded":"2024-07-17T07:14:17.010850+0000","last_fullsized":"2024-07-17T07:14:17.010850+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:37.717537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989590+0000","last_change":"2024-07-17T07:14:16.003709+0000","last_active":"2024-07-17T07:14:16.989590+0000","last_peered":"2024-07-17T07:14:16.989590+0000","last_clean":"2024-07-17T07:14:16.989590+0000","last_became_active":"2024-07-17T07:14:16.003408+0000","last_became_peered":"2024-07-17T07:14:16.003408+0000","last_unstale":"2024-07-17T07:14:16.989590+0000","last_undegraded":"2024-07-17T07:14:16.989590+0000","last_fullsized":"2024-07-17T07:14:16.989590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:34.782870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984741+0000","last_change":"2024-07-17T07:14:15.998776+0000","last_active":"2024-07-17T07:14:16.984741+0000","last_peered":"2024-07-17T07:14:16.984741+0000","last_clean":"2024-07-17T07:14:16.984741+0000","last_became_active":"2024-07-17T07:14:15.998319+0000","last_became_peered":"2024-07-17T07:14:15.998319+0000","last_unstale":"2024-07-17T07:14:16.984741+0000","last_undegraded":"2024-07-17T07:14:16.984741+0000","last_fullsized":"2024-07-17T07:14:16.984741+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:06.326565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.002562+0000","last_change":"2024-07-17T07:14:16.000777+0000","last_active":"2024-07-17T07:14:16.002562+0000","last_peered":"2024-07-17T07:14:16.002562+0000","last_clean":"2024-07-17T07:14:16.002562+0000","last_became_active":"2024-07-17T07:14:16.000489+0000","last_became_peered":"2024-07-17T07:14:16.000489+0000","last_unstale":"2024-07-17T07:14:16.002562+0000","last_undegraded":"2024-07-17T07:14:16.002562+0000","last_fullsized":"2024-07-17T07:14:16.002562+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:51.220352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988910+0000","last_change":"2024-07-17T07:14:16.002290+0000","last_active":"2024-07-17T07:14:16.988910+0000","last_peered":"2024-07-17T07:14:16.988910+0000","last_clean":"2024-07-17T07:14:16.988910+0000","last_became_active":"2024-07-17T07:14:15.999571+0000","last_became_peered":"2024-07-17T07:14:15.999571+0000","last_unstale":"2024-07-17T07:14:16.988910+0000","last_undegraded":"2024-07-17T07:14:16.988910+0000","last_fullsized":"2024-07-17T07:14:16.988910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983097+0000","last_change":"2024-07-17T07:14:15.982731+0000","last_active":"2024-07-17T07:14:16.983097+0000","last_peered":"2024-07-17T07:14:16.983097+0000","last_clean":"2024-07-17T07:14:16.983097+0000","last_became_active":"2024-07-17T07:14:15.981688+0000","last_became_peered":"2024-07-17T07:14:15.981688+0000","last_unstale":"2024-07-17T07:14:16.983097+0000","last_undegraded":"2024-07-17T07:14:16.983097+0000","last_fullsized":"2024-07-17T07:14:16.983097+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:32.802645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004401+0000","last_change":"2024-07-17T07:14:16.001377+0000","last_active":"2024-07-17T07:14:17.004401+0000","last_peered":"2024-07-17T07:14:17.004401+0000","last_clean":"2024-07-17T07:14:17.004401+0000","last_became_active":"2024-07-17T07:14:16.000340+0000","last_became_peered":"2024-07-17T07:14:16.000340+0000","last_unstale":"2024-07-17T07:14:17.004401+0000","last_undegraded":"2024-07-17T07:14:17.004401+0000","last_fullsized":"2024-07-17T07:14:17.004401+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.662506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.530320+0000","last_change":"2024-07-17T07:14:14.974979+0000","last_active":"2024-07-17T07:14:17.530320+0000","last_peered":"2024-07-17T07:14:17.530320+0000","last_clean":"2024-07-17T07:14:17.530320+0000","last_became_active":"2024-07-17T07:14:14.973817+0000","last_became_peered":"2024-07-17T07:14:14.973817+0000","last_unstale":"2024-07-17T07:14:17.530320+0000","last_undegraded":"2024-07-17T07:14:17.530320+0000","last_fullsized":"2024-07-17T07:14:17.530320+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:06.660284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983453+0000","last_change":"2024-07-17T07:14:16.015468+0000","last_active":"2024-07-17T07:14:16.983453+0000","last_peered":"2024-07-17T07:14:16.983453+0000","last_clean":"2024-07-17T07:14:16.983453+0000","last_became_active":"2024-07-17T07:14:16.013670+0000","last_became_peered":"2024-07-17T07:14:16.013670+0000","last_unstale":"2024-07-17T07:14:16.983453+0000","last_undegraded":"2024-07-17T07:14:16.983453+0000","last_fullsized":"2024-07-17T07:14:16.983453+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:41.930677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984678+0000","last_change":"2024-07-17T07:14:14.970196+0000","last_active":"2024-07-17T07:14:16.984678+0000","last_peered":"2024-07-17T07:14:16.984678+0000","last_clean":"2024-07-17T07:14:16.984678+0000","last_became_active":"2024-07-17T07:14:14.969926+0000","last_became_peered":"2024-07-17T07:14:14.969926+0000","last_unstale":"2024-07-17T07:14:16.984678+0000","last_undegraded":"2024-07-17T07:14:16.984678+0000","last_fullsized":"2024-07-17T07:14:16.984678+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:36:35.189388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.019420+0000","last_change":"2024-07-17T07:14:16.017827+0000","last_active":"2024-07-17T07:14:16.019420+0000","last_peered":"2024-07-17T07:14:16.019420+0000","last_clean":"2024-07-17T07:14:16.019420+0000","last_became_active":"2024-07-17T07:14:16.017550+0000","last_became_peered":"2024-07-17T07:14:16.017550+0000","last_unstale":"2024-07-17T07:14:16.019420+0000","last_undegraded":"2024-07-17T07:14:16.019420+0000","last_fullsized":"2024-07-17T07:14:16.019420+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:39:13.247448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"127'2","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.020047+0000","last_change":"2024-07-17T07:14:14.979346+0000","last_active":"2024-07-17T07:14:16.020047+0000","last_peered":"2024-07-17T07:14:16.020047+0000","last_clean":"2024-07-17T07:14:16.020047+0000","last_became_active":"2024-07-17T07:14:14.979072+0000","last_became_peered":"2024-07-17T07:14:14.979072+0000","last_unstale":"2024-07-17T07:14:16.020047+0000","last_undegraded":"2024-07-17T07:14:16.020047+0000","last_fullsized":"2024-07-17T07:14:16.020047+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:58.167433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010166+0000","last_change":"2024-07-17T07:14:16.004647+0000","last_active":"2024-07-17T07:14:17.010166+0000","last_peered":"2024-07-17T07:14:17.010166+0000","last_clean":"2024-07-17T07:14:17.010166+0000","last_became_active":"2024-07-17T07:14:16.002100+0000","last_became_peered":"2024-07-17T07:14:16.002100+0000","last_unstale":"2024-07-17T07:14:17.010166+0000","last_undegraded":"2024-07-17T07:14:17.010166+0000","last_fullsized":"2024-07-17T07:14:17.010166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:07.669256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985415+0000","last_change":"2024-07-17T07:14:14.976145+0000","last_active":"2024-07-17T07:14:16.985415+0000","last_peered":"2024-07-17T07:14:16.985415+0000","last_clean":"2024-07-17T07:14:16.985415+0000","last_became_active":"2024-07-17T07:14:14.975865+0000","last_became_peered":"2024-07-17T07:14:14.975865+0000","last_unstale":"2024-07-17T07:14:16.985415+0000","last_undegraded":"2024-07-17T07:14:16.985415+0000","last_fullsized":"2024-07-17T07:14:16.985415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:03.475977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529897+0000","last_change":"2024-07-17T07:14:16.006761+0000","last_active":"2024-07-17T07:14:17.529897+0000","last_peered":"2024-07-17T07:14:17.529897+0000","last_clean":"2024-07-17T07:14:17.529897+0000","last_became_active":"2024-07-17T07:14:16.006479+0000","last_became_peered":"2024-07-17T07:14:16.006479+0000","last_unstale":"2024-07-17T07:14:17.529897+0000","last_undegraded":"2024-07-17T07:14:17.529897+0000","last_fullsized":"2024-07-17T07:14:17.529897+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:50.524212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238896+0000","last_change":"2024-07-17T07:14:14.973184+0000","last_active":"2024-07-17T07:14:17.238896+0000","last_peered":"2024-07-17T07:14:17.238896+0000","last_clean":"2024-07-17T07:14:17.238896+0000","last_became_active":"2024-07-17T07:14:14.972752+0000","last_became_peered":"2024-07-17T07:14:14.972752+0000","last_unstale":"2024-07-17T07:14:17.238896+0000","last_undegraded":"2024-07-17T07:14:17.238896+0000","last_fullsized":"2024-07-17T07:14:17.238896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:43:21.724486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005209+0000","last_change":"2024-07-17T07:14:16.000179+0000","last_active":"2024-07-17T07:14:17.005209+0000","last_peered":"2024-07-17T07:14:17.005209+0000","last_clean":"2024-07-17T07:14:17.005209+0000","last_became_active":"2024-07-17T07:14:15.999408+0000","last_became_peered":"2024-07-17T07:14:15.999408+0000","last_unstale":"2024-07-17T07:14:17.005209+0000","last_undegraded":"2024-07-17T07:14:17.005209+0000","last_fullsized":"2024-07-17T07:14:17.005209+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:04.449955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009715+0000","last_change":"2024-07-17T07:14:14.976881+0000","last_active":"2024-07-17T07:14:17.009715+0000","last_peered":"2024-07-17T07:14:17.009715+0000","last_clean":"2024-07-17T07:14:17.009715+0000","last_became_active":"2024-07-17T07:14:14.976552+0000","last_became_peered":"2024-07-17T07:14:14.976552+0000","last_unstale":"2024-07-17T07:14:17.009715+0000","last_undegraded":"2024-07-17T07:14:17.009715+0000","last_fullsized":"2024-07-17T07:14:17.009715+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:40.177290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.990249+0000","last_change":"2024-07-17T07:14:16.000551+0000","last_active":"2024-07-17T07:14:16.990249+0000","last_peered":"2024-07-17T07:14:16.990249+0000","last_clean":"2024-07-17T07:14:16.990249+0000","last_became_active":"2024-07-17T07:14:16.000269+0000","last_became_peered":"2024-07-17T07:14:16.000269+0000","last_unstale":"2024-07-17T07:14:16.990249+0000","last_undegraded":"2024-07-17T07:14:16.990249+0000","last_fullsized":"2024-07-17T07:14:16.990249+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:42.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.984361+0000","last_change":"2024-07-17T07:14:14.976043+0000","last_active":"2024-07-17T07:14:15.984361+0000","last_peered":"2024-07-17T07:14:15.984361+0000","last_clean":"2024-07-17T07:14:15.984361+0000","last_became_active":"2024-07-17T07:14:14.975739+0000","last_became_peered":"2024-07-17T07:14:14.975739+0000","last_unstale":"2024-07-17T07:14:15.984361+0000","last_undegraded":"2024-07-17T07:14:15.984361+0000","last_fullsized":"2024-07-17T07:14:15.984361+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:21.800503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.530262+0000","last_change":"2024-07-17T07:14:16.011247+0000","last_active":"2024-07-17T07:14:17.530262+0000","last_peered":"2024-07-17T07:14:17.530262+0000","last_clean":"2024-07-17T07:14:17.530262+0000","last_became_active":"2024-07-17T07:14:16.010107+0000","last_became_peered":"2024-07-17T07:14:16.010107+0000","last_unstale":"2024-07-17T07:14:17.530262+0000","last_undegraded":"2024-07-17T07:14:17.530262+0000","last_fullsized":"2024-07-17T07:14:17.530262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:27.014918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529400+0000","last_change":"2024-07-17T07:14:14.975428+0000","last_active":"2024-07-17T07:14:17.529400+0000","last_peered":"2024-07-17T07:14:17.529400+0000","last_clean":"2024-07-17T07:14:17.529400+0000","last_became_active":"2024-07-17T07:14:14.975074+0000","last_became_peered":"2024-07-17T07:14:14.975074+0000","last_unstale":"2024-07-17T07:14:17.529400+0000","last_undegraded":"2024-07-17T07:14:17.529400+0000","last_fullsized":"2024-07-17T07:14:17.529400+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:44.423592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004406+0000","last_change":"2024-07-17T07:14:16.005726+0000","last_active":"2024-07-17T07:14:17.004406+0000","last_peered":"2024-07-17T07:14:17.004406+0000","last_clean":"2024-07-17T07:14:17.004406+0000","last_became_active":"2024-07-17T07:14:16.002141+0000","last_became_peered":"2024-07-17T07:14:16.002141+0000","last_unstale":"2024-07-17T07:14:17.004406+0000","last_undegraded":"2024-07-17T07:14:17.004406+0000","last_fullsized":"2024-07-17T07:14:17.004406+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:37.487996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.983724+0000","last_change":"2024-07-17T07:14:14.982510+0000","last_active":"2024-07-17T07:14:15.983724+0000","last_peered":"2024-07-17T07:14:15.983724+0000","last_clean":"2024-07-17T07:14:15.983724+0000","last_became_active":"2024-07-17T07:14:14.981063+0000","last_became_peered":"2024-07-17T07:14:14.981063+0000","last_unstale":"2024-07-17T07:14:15.983724+0000","last_undegraded":"2024-07-17T07:14:15.983724+0000","last_fullsized":"2024-07-17T07:14:15.983724+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:58.525999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985408+0000","last_change":"2024-07-17T07:14:16.009291+0000","last_active":"2024-07-17T07:14:16.985408+0000","last_peered":"2024-07-17T07:14:16.985408+0000","last_clean":"2024-07-17T07:14:16.985408+0000","last_became_active":"2024-07-17T07:14:16.002499+0000","last_became_peered":"2024-07-17T07:14:16.002499+0000","last_unstale":"2024-07-17T07:14:16.985408+0000","last_undegraded":"2024-07-17T07:14:16.985408+0000","last_fullsized":"2024-07-17T07:14:16.985408+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:49.076142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004363+0000","last_change":"2024-07-17T07:14:14.978277+0000","last_active":"2024-07-17T07:14:17.004363+0000","last_peered":"2024-07-17T07:14:17.004363+0000","last_clean":"2024-07-17T07:14:17.004363+0000","last_became_active":"2024-07-17T07:14:14.977993+0000","last_became_peered":"2024-07-17T07:14:14.977993+0000","last_unstale":"2024-07-17T07:14:17.004363+0000","last_undegraded":"2024-07-17T07:14:17.004363+0000","last_fullsized":"2024-07-17T07:14:17.004363+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:04.492201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983834+0000","last_change":"2024-07-17T07:14:16.006843+0000","last_active":"2024-07-17T07:14:16.983834+0000","last_peered":"2024-07-17T07:14:16.983834+0000","last_clean":"2024-07-17T07:14:16.983834+0000","last_became_active":"2024-07-17T07:14:16.006491+0000","last_became_peered":"2024-07-17T07:14:16.006491+0000","last_unstale":"2024-07-17T07:14:16.983834+0000","last_undegraded":"2024-07-17T07:14:16.983834+0000","last_fullsized":"2024-07-17T07:14:16.983834+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:49.004690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238589+0000","last_change":"2024-07-17T07:14:14.975659+0000","last_active":"2024-07-17T07:14:17.238589+0000","last_peered":"2024-07-17T07:14:17.238589+0000","last_clean":"2024-07-17T07:14:17.238589+0000","last_became_active":"2024-07-17T07:14:14.975373+0000","last_became_peered":"2024-07-17T07:14:14.975373+0000","last_unstale":"2024-07-17T07:14:17.238589+0000","last_undegraded":"2024-07-17T07:14:17.238589+0000","last_fullsized":"2024-07-17T07:14:17.238589+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:47:41.076767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985312+0000","last_change":"2024-07-17T07:14:16.006366+0000","last_active":"2024-07-17T07:14:16.985312+0000","last_peered":"2024-07-17T07:14:16.985312+0000","last_clean":"2024-07-17T07:14:16.985312+0000","last_became_active":"2024-07-17T07:14:16.006064+0000","last_became_peered":"2024-07-17T07:14:16.006064+0000","last_unstale":"2024-07-17T07:14:16.985312+0000","last_undegraded":"2024-07-17T07:14:16.985312+0000","last_fullsized":"2024-07-17T07:14:16.985312+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:23:59.931616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":20,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989399+0000","last_change":"2024-07-17T07:14:14.979165+0000","last_active":"2024-07-17T07:14:16.989399+0000","last_peered":"2024-07-17T07:14:16.989399+0000","last_clean":"2024-07-17T07:14:16.989399+0000","last_became_active":"2024-07-17T07:14:14.977784+0000","last_became_peered":"2024-07-17T07:14:14.977784+0000","last_unstale":"2024-07-17T07:14:16.989399+0000","last_undegraded":"2024-07-17T07:14:16.989399+0000","last_fullsized":"2024-07-17T07:14:16.989399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:40.080968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010343+0000","last_change":"2024-07-17T07:14:16.011941+0000","last_active":"2024-07-17T07:14:17.010343+0000","last_peered":"2024-07-17T07:14:17.010343+0000","last_clean":"2024-07-17T07:14:17.010343+0000","last_became_active":"2024-07-17T07:14:16.011634+0000","last_became_peered":"2024-07-17T07:14:16.011634+0000","last_unstale":"2024-07-17T07:14:17.010343+0000","last_undegraded":"2024-07-17T07:14:17.010343+0000","last_fullsized":"2024-07-17T07:14:17.010343+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:00.439218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985517+0000","last_change":"2024-07-17T07:14:14.977396+0000","last_active":"2024-07-17T07:14:16.985517+0000","last_peered":"2024-07-17T07:14:16.985517+0000","last_clean":"2024-07-17T07:14:16.985517+0000","last_became_active":"2024-07-17T07:14:14.977126+0000","last_became_peered":"2024-07-17T07:14:14.977126+0000","last_unstale":"2024-07-17T07:14:16.985517+0000","last_undegraded":"2024-07-17T07:14:16.985517+0000","last_fullsized":"2024-07-17T07:14:16.985517+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:31:44.879824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989975+0000","last_change":"2024-07-17T07:14:15.995870+0000","last_active":"2024-07-17T07:14:16.989975+0000","last_peered":"2024-07-17T07:14:16.989975+0000","last_clean":"2024-07-17T07:14:16.989975+0000","last_became_active":"2024-07-17T07:14:15.995551+0000","last_became_peered":"2024-07-17T07:14:15.995551+0000","last_unstale":"2024-07-17T07:14:16.989975+0000","last_undegraded":"2024-07-17T07:14:16.989975+0000","last_fullsized":"2024-07-17T07:14:16.989975+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:21.317048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529664+0000","last_change":"2024-07-17T07:14:14.973195+0000","last_active":"2024-07-17T07:14:17.529664+0000","last_peered":"2024-07-17T07:14:17.529664+0000","last_clean":"2024-07-17T07:14:17.529664+0000","last_became_active":"2024-07-17T07:14:14.972804+0000","last_became_peered":"2024-07-17T07:14:14.972804+0000","last_unstale":"2024-07-17T07:14:17.529664+0000","last_undegraded":"2024-07-17T07:14:17.529664+0000","last_fullsized":"2024-07-17T07:14:17.529664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:09.471868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984337+0000","last_change":"2024-07-17T07:14:15.998253+0000","last_active":"2024-07-17T07:14:16.984337+0000","last_peered":"2024-07-17T07:14:16.984337+0000","last_clean":"2024-07-17T07:14:16.984337+0000","last_became_active":"2024-07-17T07:14:15.997492+0000","last_became_peered":"2024-07-17T07:14:15.997492+0000","last_unstale":"2024-07-17T07:14:16.984337+0000","last_undegraded":"2024-07-17T07:14:16.984337+0000","last_fullsized":"2024-07-17T07:14:16.984337+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:40:15.837434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983473+0000","last_change":"2024-07-17T07:14:14.976413+0000","last_active":"2024-07-17T07:14:16.983473+0000","last_peered":"2024-07-17T07:14:16.983473+0000","last_clean":"2024-07-17T07:14:16.983473+0000","last_became_active":"2024-07-17T07:14:14.976130+0000","last_became_peered":"2024-07-17T07:14:14.976130+0000","last_unstale":"2024-07-17T07:14:16.983473+0000","last_undegraded":"2024-07-17T07:14:16.983473+0000","last_fullsized":"2024-07-17T07:14:16.983473+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:15:04.605713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005187+0000","last_change":"2024-07-17T07:14:16.012137+0000","last_active":"2024-07-17T07:14:17.005187+0000","last_peered":"2024-07-17T07:14:17.005187+0000","last_clean":"2024-07-17T07:14:17.005187+0000","last_became_active":"2024-07-17T07:14:16.011379+0000","last_became_peered":"2024-07-17T07:14:16.011379+0000","last_unstale":"2024-07-17T07:14:17.005187+0000","last_undegraded":"2024-07-17T07:14:17.005187+0000","last_fullsized":"2024-07-17T07:14:17.005187+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:03.823127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"127'8","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.025887+0000","last_change":"2024-07-17T07:14:14.978899+0000","last_active":"2024-07-17T07:14:16.025887+0000","last_peered":"2024-07-17T07:14:16.025887+0000","last_clean":"2024-07-17T07:14:16.025887+0000","last_became_active":"2024-07-17T07:14:14.978596+0000","last_became_peered":"2024-07-17T07:14:14.978596+0000","last_unstale":"2024-07-17T07:14:16.025887+0000","last_undegraded":"2024-07-17T07:14:16.025887+0000","last_fullsized":"2024-07-17T07:14:16.025887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:57.620154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983028+0000","last_change":"2024-07-17T07:14:16.002003+0000","last_active":"2024-07-17T07:14:16.983028+0000","last_peered":"2024-07-17T07:14:16.983028+0000","last_clean":"2024-07-17T07:14:16.983028+0000","last_became_active":"2024-07-17T07:14:16.001707+0000","last_became_peered":"2024-07-17T07:14:16.001707+0000","last_unstale":"2024-07-17T07:14:16.983028+0000","last_undegraded":"2024-07-17T07:14:16.983028+0000","last_fullsized":"2024-07-17T07:14:16.983028+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:58.591119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989787+0000","last_change":"2024-07-17T07:14:14.976840+0000","last_active":"2024-07-17T07:14:16.989787+0000","last_peered":"2024-07-17T07:14:16.989787+0000","last_clean":"2024-07-17T07:14:16.989787+0000","last_became_active":"2024-07-17T07:14:14.976396+0000","last_became_peered":"2024-07-17T07:14:14.976396+0000","last_unstale":"2024-07-17T07:14:16.989787+0000","last_undegraded":"2024-07-17T07:14:16.989787+0000","last_fullsized":"2024-07-17T07:14:16.989787+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:48.601722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988653+0000","last_change":"2024-07-17T07:14:15.997739+0000","last_active":"2024-07-17T07:14:16.988653+0000","last_peered":"2024-07-17T07:14:16.988653+0000","last_clean":"2024-07-17T07:14:16.988653+0000","last_became_active":"2024-07-17T07:14:15.997417+0000","last_became_peered":"2024-07-17T07:14:15.997417+0000","last_unstale":"2024-07-17T07:14:16.988653+0000","last_undegraded":"2024-07-17T07:14:16.988653+0000","last_fullsized":"2024-07-17T07:14:16.988653+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:25.163906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238741+0000","last_change":"2024-07-17T07:14:14.978093+0000","last_active":"2024-07-17T07:14:17.238741+0000","last_peered":"2024-07-17T07:14:17.238741+0000","last_clean":"2024-07-17T07:14:17.238741+0000","last_became_active":"2024-07-17T07:14:14.977688+0000","last_became_peered":"2024-07-17T07:14:14.977688+0000","last_unstale":"2024-07-17T07:14:17.238741+0000","last_undegraded":"2024-07-17T07:14:17.238741+0000","last_fullsized":"2024-07-17T07:14:17.238741+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:49.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985629+0000","last_change":"2024-07-17T07:14:16.019118+0000","last_active":"2024-07-17T07:14:16.985629+0000","last_peered":"2024-07-17T07:14:16.985629+0000","last_clean":"2024-07-17T07:14:16.985629+0000","last_became_active":"2024-07-17T07:14:16.012518+0000","last_became_peered":"2024-07-17T07:14:16.012518+0000","last_unstale":"2024-07-17T07:14:16.985629+0000","last_undegraded":"2024-07-17T07:14:16.985629+0000","last_fullsized":"2024-07-17T07:14:16.985629+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:11.542469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010509+0000","last_change":"2024-07-17T07:14:14.980435+0000","last_active":"2024-07-17T07:14:17.010509+0000","last_peered":"2024-07-17T07:14:17.010509+0000","last_clean":"2024-07-17T07:14:17.010509+0000","last_became_active":"2024-07-17T07:14:14.979864+0000","last_became_peered":"2024-07-17T07:14:14.979864+0000","last_unstale":"2024-07-17T07:14:17.010509+0000","last_undegraded":"2024-07-17T07:14:17.010509+0000","last_fullsized":"2024-07-17T07:14:17.010509+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:44.484742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004361+0000","last_change":"2024-07-17T07:14:16.018290+0000","last_active":"2024-07-17T07:14:17.004361+0000","last_peered":"2024-07-17T07:14:17.004361+0000","last_clean":"2024-07-17T07:14:17.004361+0000","last_became_active":"2024-07-17T07:14:16.017965+0000","last_became_peered":"2024-07-17T07:14:16.017965+0000","last_unstale":"2024-07-17T07:14:17.004361+0000","last_undegraded":"2024-07-17T07:14:17.004361+0000","last_fullsized":"2024-07-17T07:14:17.004361+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:40.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984708+0000","last_change":"2024-07-17T07:14:14.970797+0000","last_active":"2024-07-17T07:14:16.984708+0000","last_peered":"2024-07-17T07:14:16.984708+0000","last_clean":"2024-07-17T07:14:16.984708+0000","last_became_active":"2024-07-17T07:14:14.969013+0000","last_became_peered":"2024-07-17T07:14:14.969013+0000","last_unstale":"2024-07-17T07:14:16.984708+0000","last_undegraded":"2024-07-17T07:14:16.984708+0000","last_fullsized":"2024-07-17T07:14:16.984708+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:23.982444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005032+0000","last_change":"2024-07-17T07:14:16.006338+0000","last_active":"2024-07-17T07:14:17.005032+0000","last_peered":"2024-07-17T07:14:17.005032+0000","last_clean":"2024-07-17T07:14:17.005032+0000","last_became_active":"2024-07-17T07:14:16.006015+0000","last_became_peered":"2024-07-17T07:14:16.006015+0000","last_unstale":"2024-07-17T07:14:17.005032+0000","last_undegraded":"2024-07-17T07:14:17.005032+0000","last_fullsized":"2024-07-17T07:14:17.005032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:12.463368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983498+0000","last_change":"2024-07-17T07:14:14.967510+0000","last_active":"2024-07-17T07:14:16.983498+0000","last_peered":"2024-07-17T07:14:16.983498+0000","last_clean":"2024-07-17T07:14:16.983498+0000","last_became_active":"2024-07-17T07:14:14.967199+0000","last_became_peered":"2024-07-17T07:14:14.967199+0000","last_unstale":"2024-07-17T07:14:16.983498+0000","last_undegraded":"2024-07-17T07:14:16.983498+0000","last_fullsized":"2024-07-17T07:14:16.983498+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:16.851764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529510+0000","last_change":"2024-07-17T07:14:16.010062+0000","last_active":"2024-07-17T07:14:17.529510+0000","last_peered":"2024-07-17T07:14:17.529510+0000","last_clean":"2024-07-17T07:14:17.529510+0000","last_became_active":"2024-07-17T07:14:16.009574+0000","last_became_peered":"2024-07-17T07:14:16.009574+0000","last_unstale":"2024-07-17T07:14:17.529510+0000","last_undegraded":"2024-07-17T07:14:17.529510+0000","last_fullsized":"2024-07-17T07:14:17.529510+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:59.609702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988737+0000","last_change":"2024-07-17T07:14:14.976419+0000","last_active":"2024-07-17T07:14:16.988737+0000","last_peered":"2024-07-17T07:14:16.988737+0000","last_clean":"2024-07-17T07:14:16.988737+0000","last_became_active":"2024-07-17T07:14:14.976098+0000","last_became_peered":"2024-07-17T07:14:14.976098+0000","last_unstale":"2024-07-17T07:14:16.988737+0000","last_undegraded":"2024-07-17T07:14:16.988737+0000","last_fullsized":"2024-07-17T07:14:16.988737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:05.701624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984178+0000","last_change":"2024-07-17T07:14:16.014314+0000","last_active":"2024-07-17T07:14:16.984178+0000","last_peered":"2024-07-17T07:14:16.984178+0000","last_clean":"2024-07-17T07:14:16.984178+0000","last_became_active":"2024-07-17T07:14:16.013974+0000","last_became_peered":"2024-07-17T07:14:16.013974+0000","last_unstale":"2024-07-17T07:14:16.984178+0000","last_undegraded":"2024-07-17T07:14:16.984178+0000","last_fullsized":"2024-07-17T07:14:16.984178+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:43.888659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983169+0000","last_change":"2024-07-17T07:14:14.966796+0000","last_active":"2024-07-17T07:14:16.983169+0000","last_peered":"2024-07-17T07:14:16.983169+0000","last_clean":"2024-07-17T07:14:16.983169+0000","last_became_active":"2024-07-17T07:14:14.966484+0000","last_became_peered":"2024-07-17T07:14:14.966484+0000","last_unstale":"2024-07-17T07:14:16.983169+0000","last_undegraded":"2024-07-17T07:14:16.983169+0000","last_fullsized":"2024-07-17T07:14:16.983169+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:37.943347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004012+0000","last_change":"2024-07-17T07:14:16.022653+0000","last_active":"2024-07-17T07:14:17.004012+0000","last_peered":"2024-07-17T07:14:17.004012+0000","last_clean":"2024-07-17T07:14:17.004012+0000","last_became_active":"2024-07-17T07:14:16.022273+0000","last_became_peered":"2024-07-17T07:14:16.022273+0000","last_unstale":"2024-07-17T07:14:17.004012+0000","last_undegraded":"2024-07-17T07:14:17.004012+0000","last_fullsized":"2024-07-17T07:14:17.004012+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:25.404667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985709+0000","last_change":"2024-07-17T07:14:14.980947+0000","last_active":"2024-07-17T07:14:16.985709+0000","last_peered":"2024-07-17T07:14:16.985709+0000","last_clean":"2024-07-17T07:14:16.985709+0000","last_became_active":"2024-07-17T07:14:14.980606+0000","last_became_peered":"2024-07-17T07:14:14.980606+0000","last_unstale":"2024-07-17T07:14:16.985709+0000","last_undegraded":"2024-07-17T07:14:16.985709+0000","last_fullsized":"2024-07-17T07:14:16.985709+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:51:41.160690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.007519+0000","last_change":"2024-07-17T07:14:16.005475+0000","last_active":"2024-07-17T07:14:16.007519+0000","last_peered":"2024-07-17T07:14:16.007519+0000","last_clean":"2024-07-17T07:14:16.007519+0000","last_became_active":"2024-07-17T07:14:16.005130+0000","last_became_peered":"2024-07-17T07:14:16.005130+0000","last_unstale":"2024-07-17T07:14:16.007519+0000","last_undegraded":"2024-07-17T07:14:16.007519+0000","last_fullsized":"2024-07-17T07:14:16.007519+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:40.603992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"128'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.007972+0000","last_change":"2024-07-17T07:14:14.976115+0000","last_active":"2024-07-17T07:14:17.007972+0000","last_peered":"2024-07-17T07:14:17.007972+0000","last_clean":"2024-07-17T07:14:17.007972+0000","last_became_active":"2024-07-17T07:14:14.975487+0000","last_became_peered":"2024-07-17T07:14:14.975487+0000","last_unstale":"2024-07-17T07:14:17.007972+0000","last_undegraded":"2024-07-17T07:14:17.007972+0000","last_fullsized":"2024-07-17T07:14:17.007972+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:15:34.933058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529417+0000","last_change":"2024-07-17T07:14:16.007519+0000","last_active":"2024-07-17T07:14:17.529417+0000","last_peered":"2024-07-17T07:14:17.529417+0000","last_clean":"2024-07-17T07:14:17.529417+0000","last_became_active":"2024-07-17T07:14:16.007124+0000","last_became_peered":"2024-07-17T07:14:16.007124+0000","last_unstale":"2024-07-17T07:14:17.529417+0000","last_undegraded":"2024-07-17T07:14:17.529417+0000","last_fullsized":"2024-07-17T07:14:17.529417+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:51.011150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989504+0000","last_change":"2024-07-17T07:14:14.975415+0000","last_active":"2024-07-17T07:14:16.989504+0000","last_peered":"2024-07-17T07:14:16.989504+0000","last_clean":"2024-07-17T07:14:16.989504+0000","last_became_active":"2024-07-17T07:14:14.975117+0000","last_became_peered":"2024-07-17T07:14:14.975117+0000","last_unstale":"2024-07-17T07:14:16.989504+0000","last_undegraded":"2024-07-17T07:14:16.989504+0000","last_fullsized":"2024-07-17T07:14:16.989504+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:33.181629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009944+0000","last_change":"2024-07-17T07:14:16.010906+0000","last_active":"2024-07-17T07:14:17.009944+0000","last_peered":"2024-07-17T07:14:17.009944+0000","last_clean":"2024-07-17T07:14:17.009944+0000","last_became_active":"2024-07-17T07:14:16.010118+0000","last_became_peered":"2024-07-17T07:14:16.010118+0000","last_unstale":"2024-07-17T07:14:17.009944+0000","last_undegraded":"2024-07-17T07:14:17.009944+0000","last_fullsized":"2024-07-17T07:14:17.009944+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:10.480840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985575+0000","last_change":"2024-07-17T07:14:14.979169+0000","last_active":"2024-07-17T07:14:16.985575+0000","last_peered":"2024-07-17T07:14:16.985575+0000","last_clean":"2024-07-17T07:14:16.985575+0000","last_became_active":"2024-07-17T07:14:14.978882+0000","last_became_peered":"2024-07-17T07:14:14.978882+0000","last_unstale":"2024-07-17T07:14:16.985575+0000","last_undegraded":"2024-07-17T07:14:16.985575+0000","last_fullsized":"2024-07-17T07:14:16.985575+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:51.787457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984018+0000","last_change":"2024-07-17T07:14:16.006420+0000","last_active":"2024-07-17T07:14:16.984018+0000","last_peered":"2024-07-17T07:14:16.984018+0000","last_clean":"2024-07-17T07:14:16.984018+0000","last_became_active":"2024-07-17T07:14:16.004018+0000","last_became_peered":"2024-07-17T07:14:16.004018+0000","last_unstale":"2024-07-17T07:14:16.984018+0000","last_undegraded":"2024-07-17T07:14:16.984018+0000","last_fullsized":"2024-07-17T07:14:16.984018+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:09.543489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.982907+0000","last_change":"2024-07-17T07:14:14.968717+0000","last_active":"2024-07-17T07:14:15.982907+0000","last_peered":"2024-07-17T07:14:15.982907+0000","last_clean":"2024-07-17T07:14:15.982907+0000","last_became_active":"2024-07-17T07:14:14.968383+0000","last_became_peered":"2024-07-17T07:14:14.968383+0000","last_unstale":"2024-07-17T07:14:15.982907+0000","last_undegraded":"2024-07-17T07:14:15.982907+0000","last_fullsized":"2024-07-17T07:14:15.982907+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:37:23.544277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983944+0000","last_change":"2024-07-17T07:14:16.005048+0000","last_active":"2024-07-17T07:14:16.983944+0000","last_peered":"2024-07-17T07:14:16.983944+0000","last_clean":"2024-07-17T07:14:16.983944+0000","last_became_active":"2024-07-17T07:14:16.004757+0000","last_became_peered":"2024-07-17T07:14:16.004757+0000","last_unstale":"2024-07-17T07:14:16.983944+0000","last_undegraded":"2024-07-17T07:14:16.983944+0000","last_fullsized":"2024-07-17T07:14:16.983944+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:52:33.466147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983644+0000","last_change":"2024-07-17T07:14:14.978987+0000","last_active":"2024-07-17T07:14:16.983644+0000","last_peered":"2024-07-17T07:14:16.983644+0000","last_clean":"2024-07-17T07:14:16.983644+0000","last_became_active":"2024-07-17T07:14:14.977039+0000","last_became_peered":"2024-07-17T07:14:14.977039+0000","last_unstale":"2024-07-17T07:14:16.983644+0000","last_undegraded":"2024-07-17T07:14:16.983644+0000","last_fullsized":"2024-07-17T07:14:16.983644+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:47.950517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238916+0000","last_change":"2024-07-17T07:14:16.004077+0000","last_active":"2024-07-17T07:14:17.238916+0000","last_peered":"2024-07-17T07:14:17.238916+0000","last_clean":"2024-07-17T07:14:17.238916+0000","last_became_active":"2024-07-17T07:14:16.003805+0000","last_became_peered":"2024-07-17T07:14:16.003805+0000","last_unstale":"2024-07-17T07:14:17.238916+0000","last_undegraded":"2024-07-17T07:14:17.238916+0000","last_fullsized":"2024-07-17T07:14:17.238916+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:59.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988991+0000","last_change":"2024-07-17T07:14:14.974611+0000","last_active":"2024-07-17T07:14:16.988991+0000","last_peered":"2024-07-17T07:14:16.988991+0000","last_clean":"2024-07-17T07:14:16.988991+0000","last_became_active":"2024-07-17T07:14:14.974325+0000","last_became_peered":"2024-07-17T07:14:14.974325+0000","last_unstale":"2024-07-17T07:14:16.988991+0000","last_undegraded":"2024-07-17T07:14:16.988991+0000","last_fullsized":"2024-07-17T07:14:16.988991+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:23:00.804718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.011232+0000","last_change":"2024-07-17T07:14:16.006149+0000","last_active":"2024-07-17T07:14:17.011232+0000","last_peered":"2024-07-17T07:14:17.011232+0000","last_clean":"2024-07-17T07:14:17.011232+0000","last_became_active":"2024-07-17T07:14:16.005884+0000","last_became_peered":"2024-07-17T07:14:16.005884+0000","last_unstale":"2024-07-17T07:14:17.011232+0000","last_undegraded":"2024-07-17T07:14:17.011232+0000","last_fullsized":"2024-07-17T07:14:17.011232+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:39.246748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983368+0000","last_change":"2024-07-17T07:14:14.962790+0000","last_active":"2024-07-17T07:14:16.983368+0000","last_peered":"2024-07-17T07:14:16.983368+0000","last_clean":"2024-07-17T07:14:16.983368+0000","last_became_active":"2024-07-17T07:14:14.962497+0000","last_became_peered":"2024-07-17T07:14:14.962497+0000","last_unstale":"2024-07-17T07:14:16.983368+0000","last_undegraded":"2024-07-17T07:14:16.983368+0000","last_fullsized":"2024-07-17T07:14:16.983368+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:53:47.163320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009335+0000","last_change":"2024-07-17T07:14:16.012570+0000","last_active":"2024-07-17T07:14:17.009335+0000","last_peered":"2024-07-17T07:14:17.009335+0000","last_clean":"2024-07-17T07:14:17.009335+0000","last_became_active":"2024-07-17T07:14:16.012298+0000","last_became_peered":"2024-07-17T07:14:16.012298+0000","last_unstale":"2024-07-17T07:14:17.009335+0000","last_undegraded":"2024-07-17T07:14:17.009335+0000","last_fullsized":"2024-07-17T07:14:17.009335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:45.432564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984976+0000","last_change":"2024-07-17T07:14:14.967431+0000","last_active":"2024-07-17T07:14:16.984976+0000","last_peered":"2024-07-17T07:14:16.984976+0000","last_clean":"2024-07-17T07:14:16.984976+0000","last_became_active":"2024-07-17T07:14:14.966501+0000","last_became_peered":"2024-07-17T07:14:14.966501+0000","last_unstale":"2024-07-17T07:14:16.984976+0000","last_undegraded":"2024-07-17T07:14:16.984976+0000","last_fullsized":"2024-07-17T07:14:16.984976+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:57.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"118'2420","reported_seq":2841,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238001+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:14:17.238001+0000","last_peered":"2024-07-17T07:14:17.238001+0000","last_clean":"2024-07-17T07:14:17.238001+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:14:17.238001+0000","last_undegraded":"2024-07-17T07:14:17.238001+0000","last_fullsized":"2024-07-17T07:14:17.238001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":203,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":203,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2420,"ondisk_log_size":2420,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542376,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51356,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320484,"statfs":{"total":96636764160,"available":96584175616,"internally_reserved":0,"allocated":2179072,"data_stored":1423987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":835675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509672,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50760,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321080,"statfs":{"total":96636764160,"available":96584785920,"internally_reserved":0,"allocated":1564672,"data_stored":826247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542377,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":826339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1568768,"data_stored":826829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59096,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312744,"statfs":{"total":96636764160,"available":96576249856,"internally_reserved":0,"allocated":1581056,"data_stored":833493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1581056,"data_stored":833493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509671,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2187264,"data_stored":1425529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:14:20.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 -- 172.21.15.44:0/996275780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 msgr2=0x7fd95415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 --2- 172.21.15.44:0/996275780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd95415e510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd944005fd0 tx=0x7fd94402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 -- 172.21.15.44:0/996275780 shutdown_connections 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 --2- 172.21.15.44:0/996275780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 0x7fd9540b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 --2- 172.21.15.44:0/996275780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd95415e510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.733+0000 7fd95a800640 1 -- 172.21.15.44:0/996275780 >> 172.21.15.44:0/996275780 conn(0x7fd9540b0fb0 msgr2=0x7fd9540b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.734+0000 7fd95a800640 1 -- 172.21.15.44:0/996275780 shutdown_connections 2024-07-17T07:14:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.734+0000 7fd95a800640 1 -- 172.21.15.44:0/996275780 wait complete. 2024-07-17T07:14:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.736+0000 7fd95a800640 1 Processor -- start 2024-07-17T07:14:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.736+0000 7fd95a800640 1 -- start start 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 0x7fd954157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd9541577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd95406f040 con 0x7fd95406ea50 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd95406eec0 con 0x7fd9540ae810 2024-07-17T07:14:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd95a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9540b2b10 con 0x7fd9540ac930 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.737+0000 7fd953400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35728/0 (socket says 172.21.15.44:35728) 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 -- 172.21.15.44:0/495633643 learned_addr learned my addr 172.21.15.44:0/495633643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 -- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 msgr2=0x7fd954157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd952a00640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 0x7fd954157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953e00640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd9541577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 0x7fd954157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 -- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 msgr2=0x7fd9541577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd9541577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 -- 172.21.15.44:0/495633643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9541f4350 con 0x7fd95406ea50 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.738+0000 7fd953400640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd944034040 tx=0x7fd9440035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.739+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94403e030 con 0x7fd95406ea50 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.739+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd944003ae0 con 0x7fd95406ea50 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.739+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944035040 con 0x7fd95406ea50 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.739+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9541f4610 con 0x7fd95406ea50 2024-07-17T07:14:20.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.739+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541f4bd0 con 0x7fd95406ea50 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.740+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95406a1a0 con 0x7fd95406ea50 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.741+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7fd944041030 con 0x7fd95406ea50 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.742+0000 7fd94b400640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 0x7fd934079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.742+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7fd9440b9110 con 0x7fd95406ea50 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.743+0000 7fd952a00640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 0x7fd934079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.744+0000 7fd952a00640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 0x7fd934079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd940001640 tx=0x7fd940009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.747+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd944081f90 con 0x7fd95406ea50 2024-07-17T07:14:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.954+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 --> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd954157ce0 con 0x7fd934076cd0 2024-07-17T07:14:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.967+0000 7fd94b400640 1 -- 172.21.15.44:0/495633643 <== mgr.6202 v2:172.21.15.175:6800/1224757683 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7fd954157ce0 con 0x7fd934076cd0 2024-07-17T07:14:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:20.973 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 msgr2=0x7fd934079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 0x7fd934079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd940001640 tx=0x7fd940009040 comp rx=0 tx=0).stop 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 msgr2=0x7fd9541422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd944034040 tx=0x7fd9440035f0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 shutdown_connections 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fd934076cd0 0x7fd934079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540ae810 0x7fd9541577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540ac930 0x7fd954157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 --2- 172.21.15.44:0/495633643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd95406ea50 0x7fd9541422c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 >> 172.21.15.44:0/495633643 conn(0x7fd9540b0fb0 msgr2=0x7fd9540b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 shutdown_connections 2024-07-17T07:14:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:20.974+0000 7fd95a800640 1 -- 172.21.15.44:0/495633643 wait complete. 2024-07-17T07:14:20.995 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:14:19.631220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":248,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2459,"ondisk_log_size":2459,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333752,"kb_used_data":13508,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754640968,"statfs":{"total":773094113280,"available":772752351232,"internally_reserved":0,"allocated":13832192,"data_stored":7831592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":327798320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.693212"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004675+0000","last_change":"2024-07-17T07:14:16.012995+0000","last_active":"2024-07-17T07:14:17.004675+0000","last_peered":"2024-07-17T07:14:17.004675+0000","last_clean":"2024-07-17T07:14:17.004675+0000","last_became_active":"2024-07-17T07:14:16.010891+0000","last_became_peered":"2024-07-17T07:14:16.010891+0000","last_unstale":"2024-07-17T07:14:17.004675+0000","last_undegraded":"2024-07-17T07:14:17.004675+0000","last_fullsized":"2024-07-17T07:14:17.004675+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:53.227079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989707+0000","last_change":"2024-07-17T07:14:15.996008+0000","last_active":"2024-07-17T07:14:16.989707+0000","last_peered":"2024-07-17T07:14:16.989707+0000","last_clean":"2024-07-17T07:14:16.989707+0000","last_became_active":"2024-07-17T07:14:15.995651+0000","last_became_peered":"2024-07-17T07:14:15.995651+0000","last_unstale":"2024-07-17T07:14:16.989707+0000","last_undegraded":"2024-07-17T07:14:16.989707+0000","last_fullsized":"2024-07-17T07:14:16.989707+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:28.319554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004657+0000","last_change":"2024-07-17T07:14:16.012316+0000","last_active":"2024-07-17T07:14:17.004657+0000","last_peered":"2024-07-17T07:14:17.004657+0000","last_clean":"2024-07-17T07:14:17.004657+0000","last_became_active":"2024-07-17T07:14:16.011029+0000","last_became_peered":"2024-07-17T07:14:16.011029+0000","last_unstale":"2024-07-17T07:14:17.004657+0000","last_undegraded":"2024-07-17T07:14:17.004657+0000","last_fullsized":"2024-07-17T07:14:17.004657+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:33:31.367841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529889+0000","last_change":"2024-07-17T07:14:16.006107+0000","last_active":"2024-07-17T07:14:17.529889+0000","last_peered":"2024-07-17T07:14:17.529889+0000","last_clean":"2024-07-17T07:14:17.529889+0000","last_became_active":"2024-07-17T07:14:16.005816+0000","last_became_peered":"2024-07-17T07:14:16.005816+0000","last_unstale":"2024-07-17T07:14:17.529889+0000","last_undegraded":"2024-07-17T07:14:17.529889+0000","last_fullsized":"2024-07-17T07:14:17.529889+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:57:59.997536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984992+0000","last_change":"2024-07-17T07:14:16.018765+0000","last_active":"2024-07-17T07:14:16.984992+0000","last_peered":"2024-07-17T07:14:16.984992+0000","last_clean":"2024-07-17T07:14:16.984992+0000","last_became_active":"2024-07-17T07:14:16.018423+0000","last_became_peered":"2024-07-17T07:14:16.018423+0000","last_unstale":"2024-07-17T07:14:16.984992+0000","last_undegraded":"2024-07-17T07:14:16.984992+0000","last_fullsized":"2024-07-17T07:14:16.984992+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:26.991068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988579+0000","last_change":"2024-07-17T07:14:16.006160+0000","last_active":"2024-07-17T07:14:16.988579+0000","last_peered":"2024-07-17T07:14:16.988579+0000","last_clean":"2024-07-17T07:14:16.988579+0000","last_became_active":"2024-07-17T07:14:16.005848+0000","last_became_peered":"2024-07-17T07:14:16.005848+0000","last_unstale":"2024-07-17T07:14:16.988579+0000","last_undegraded":"2024-07-17T07:14:16.988579+0000","last_fullsized":"2024-07-17T07:14:16.988579+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:34:41.996529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988426+0000","last_change":"2024-07-17T07:14:16.013168+0000","last_active":"2024-07-17T07:14:16.988426+0000","last_peered":"2024-07-17T07:14:16.988426+0000","last_clean":"2024-07-17T07:14:16.988426+0000","last_became_active":"2024-07-17T07:14:16.012754+0000","last_became_peered":"2024-07-17T07:14:16.012754+0000","last_unstale":"2024-07-17T07:14:16.988426+0000","last_undegraded":"2024-07-17T07:14:16.988426+0000","last_fullsized":"2024-07-17T07:14:16.988426+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:01.799337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.982821+0000","last_change":"2024-07-17T07:14:16.004033+0000","last_active":"2024-07-17T07:14:16.982821+0000","last_peered":"2024-07-17T07:14:16.982821+0000","last_clean":"2024-07-17T07:14:16.982821+0000","last_became_active":"2024-07-17T07:14:16.003645+0000","last_became_peered":"2024-07-17T07:14:16.003645+0000","last_unstale":"2024-07-17T07:14:16.982821+0000","last_undegraded":"2024-07-17T07:14:16.982821+0000","last_fullsized":"2024-07-17T07:14:16.982821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:56.136461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989791+0000","last_change":"2024-07-17T07:14:16.010867+0000","last_active":"2024-07-17T07:14:16.989791+0000","last_peered":"2024-07-17T07:14:16.989791+0000","last_clean":"2024-07-17T07:14:16.989791+0000","last_became_active":"2024-07-17T07:14:16.010406+0000","last_became_peered":"2024-07-17T07:14:16.010406+0000","last_unstale":"2024-07-17T07:14:16.989791+0000","last_undegraded":"2024-07-17T07:14:16.989791+0000","last_fullsized":"2024-07-17T07:14:16.989791+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:11.252677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985429+0000","last_change":"2024-07-17T07:14:16.007400+0000","last_active":"2024-07-17T07:14:16.985429+0000","last_peered":"2024-07-17T07:14:16.985429+0000","last_clean":"2024-07-17T07:14:16.985429+0000","last_became_active":"2024-07-17T07:14:16.007102+0000","last_became_peered":"2024-07-17T07:14:16.007102+0000","last_unstale":"2024-07-17T07:14:16.985429+0000","last_undegraded":"2024-07-17T07:14:16.985429+0000","last_fullsized":"2024-07-17T07:14:16.985429+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:39:05.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010229+0000","last_change":"2024-07-17T07:14:16.008251+0000","last_active":"2024-07-17T07:14:17.010229+0000","last_peered":"2024-07-17T07:14:17.010229+0000","last_clean":"2024-07-17T07:14:17.010229+0000","last_became_active":"2024-07-17T07:14:16.007946+0000","last_became_peered":"2024-07-17T07:14:16.007946+0000","last_unstale":"2024-07-17T07:14:17.010229+0000","last_undegraded":"2024-07-17T07:14:17.010229+0000","last_fullsized":"2024-07-17T07:14:17.010229+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:03.923444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.999140+0000","last_change":"2024-07-17T07:14:15.992342+0000","last_active":"2024-07-17T07:14:15.999140+0000","last_peered":"2024-07-17T07:14:15.999140+0000","last_clean":"2024-07-17T07:14:15.999140+0000","last_became_active":"2024-07-17T07:14:15.991920+0000","last_became_peered":"2024-07-17T07:14:15.991920+0000","last_unstale":"2024-07-17T07:14:15.999140+0000","last_undegraded":"2024-07-17T07:14:15.999140+0000","last_fullsized":"2024-07-17T07:14:15.999140+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:30.036103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529391+0000","last_change":"2024-07-17T07:14:16.007633+0000","last_active":"2024-07-17T07:14:17.529391+0000","last_peered":"2024-07-17T07:14:17.529391+0000","last_clean":"2024-07-17T07:14:17.529391+0000","last_became_active":"2024-07-17T07:14:16.007305+0000","last_became_peered":"2024-07-17T07:14:16.007305+0000","last_unstale":"2024-07-17T07:14:17.529391+0000","last_undegraded":"2024-07-17T07:14:17.529391+0000","last_fullsized":"2024-07-17T07:14:17.529391+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:50.162207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238463+0000","last_change":"2024-07-17T07:14:16.012002+0000","last_active":"2024-07-17T07:14:17.238463+0000","last_peered":"2024-07-17T07:14:17.238463+0000","last_clean":"2024-07-17T07:14:17.238463+0000","last_became_active":"2024-07-17T07:14:16.011428+0000","last_became_peered":"2024-07-17T07:14:16.011428+0000","last_unstale":"2024-07-17T07:14:17.238463+0000","last_undegraded":"2024-07-17T07:14:17.238463+0000","last_fullsized":"2024-07-17T07:14:17.238463+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:50.385822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238381+0000","last_change":"2024-07-17T07:14:16.006815+0000","last_active":"2024-07-17T07:14:17.238381+0000","last_peered":"2024-07-17T07:14:17.238381+0000","last_clean":"2024-07-17T07:14:17.238381+0000","last_became_active":"2024-07-17T07:14:16.006511+0000","last_became_peered":"2024-07-17T07:14:16.006511+0000","last_unstale":"2024-07-17T07:14:17.238381+0000","last_undegraded":"2024-07-17T07:14:17.238381+0000","last_fullsized":"2024-07-17T07:14:17.238381+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:12.334189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238410+0000","last_change":"2024-07-17T07:14:15.994936+0000","last_active":"2024-07-17T07:14:17.238410+0000","last_peered":"2024-07-17T07:14:17.238410+0000","last_clean":"2024-07-17T07:14:17.238410+0000","last_became_active":"2024-07-17T07:14:15.994399+0000","last_became_peered":"2024-07-17T07:14:15.994399+0000","last_unstale":"2024-07-17T07:14:17.238410+0000","last_undegraded":"2024-07-17T07:14:17.238410+0000","last_fullsized":"2024-07-17T07:14:17.238410+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:12.075572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004825+0000","last_change":"2024-07-17T07:14:16.012515+0000","last_active":"2024-07-17T07:14:17.004825+0000","last_peered":"2024-07-17T07:14:17.004825+0000","last_clean":"2024-07-17T07:14:17.004825+0000","last_became_active":"2024-07-17T07:14:16.011828+0000","last_became_peered":"2024-07-17T07:14:16.011828+0000","last_unstale":"2024-07-17T07:14:17.004825+0000","last_undegraded":"2024-07-17T07:14:17.004825+0000","last_fullsized":"2024-07-17T07:14:17.004825+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:39.615469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.990090+0000","last_change":"2024-07-17T07:14:16.004510+0000","last_active":"2024-07-17T07:14:16.990090+0000","last_peered":"2024-07-17T07:14:16.990090+0000","last_clean":"2024-07-17T07:14:16.990090+0000","last_became_active":"2024-07-17T07:14:16.003956+0000","last_became_peered":"2024-07-17T07:14:16.003956+0000","last_unstale":"2024-07-17T07:14:16.990090+0000","last_undegraded":"2024-07-17T07:14:16.990090+0000","last_fullsized":"2024-07-17T07:14:16.990090+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:55.557627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004841+0000","last_change":"2024-07-17T07:14:16.002351+0000","last_active":"2024-07-17T07:14:17.004841+0000","last_peered":"2024-07-17T07:14:17.004841+0000","last_clean":"2024-07-17T07:14:17.004841+0000","last_became_active":"2024-07-17T07:14:16.001990+0000","last_became_peered":"2024-07-17T07:14:16.001990+0000","last_unstale":"2024-07-17T07:14:17.004841+0000","last_undegraded":"2024-07-17T07:14:17.004841+0000","last_fullsized":"2024-07-17T07:14:17.004841+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:43.660355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.000376+0000","last_change":"2024-07-17T07:14:15.995908+0000","last_active":"2024-07-17T07:14:16.000376+0000","last_peered":"2024-07-17T07:14:16.000376+0000","last_clean":"2024-07-17T07:14:16.000376+0000","last_became_active":"2024-07-17T07:14:15.995612+0000","last_became_peered":"2024-07-17T07:14:15.995612+0000","last_unstale":"2024-07-17T07:14:16.000376+0000","last_undegraded":"2024-07-17T07:14:16.000376+0000","last_fullsized":"2024-07-17T07:14:16.000376+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:45.086371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984563+0000","last_change":"2024-07-17T07:14:16.001364+0000","last_active":"2024-07-17T07:14:16.984563+0000","last_peered":"2024-07-17T07:14:16.984563+0000","last_clean":"2024-07-17T07:14:16.984563+0000","last_became_active":"2024-07-17T07:14:16.001057+0000","last_became_peered":"2024-07-17T07:14:16.001057+0000","last_unstale":"2024-07-17T07:14:16.984563+0000","last_undegraded":"2024-07-17T07:14:16.984563+0000","last_fullsized":"2024-07-17T07:14:16.984563+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:13.187794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984881+0000","last_change":"2024-07-17T07:14:16.017614+0000","last_active":"2024-07-17T07:14:16.984881+0000","last_peered":"2024-07-17T07:14:16.984881+0000","last_clean":"2024-07-17T07:14:16.984881+0000","last_became_active":"2024-07-17T07:14:16.012502+0000","last_became_peered":"2024-07-17T07:14:16.012502+0000","last_unstale":"2024-07-17T07:14:16.984881+0000","last_undegraded":"2024-07-17T07:14:16.984881+0000","last_fullsized":"2024-07-17T07:14:16.984881+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:41.089940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004241+0000","last_change":"2024-07-17T07:14:16.010144+0000","last_active":"2024-07-17T07:14:17.004241+0000","last_peered":"2024-07-17T07:14:17.004241+0000","last_clean":"2024-07-17T07:14:17.004241+0000","last_became_active":"2024-07-17T07:14:16.009857+0000","last_became_peered":"2024-07-17T07:14:16.009857+0000","last_unstale":"2024-07-17T07:14:17.004241+0000","last_undegraded":"2024-07-17T07:14:17.004241+0000","last_fullsized":"2024-07-17T07:14:17.004241+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:14.750206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985657+0000","last_change":"2024-07-17T07:14:16.005332+0000","last_active":"2024-07-17T07:14:16.985657+0000","last_peered":"2024-07-17T07:14:16.985657+0000","last_clean":"2024-07-17T07:14:16.985657+0000","last_became_active":"2024-07-17T07:14:16.001688+0000","last_became_peered":"2024-07-17T07:14:16.001688+0000","last_unstale":"2024-07-17T07:14:16.985657+0000","last_undegraded":"2024-07-17T07:14:16.985657+0000","last_fullsized":"2024-07-17T07:14:16.985657+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:35:51.182145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.001061+0000","last_change":"2024-07-17T07:14:15.996810+0000","last_active":"2024-07-17T07:14:16.001061+0000","last_peered":"2024-07-17T07:14:16.001061+0000","last_clean":"2024-07-17T07:14:16.001061+0000","last_became_active":"2024-07-17T07:14:15.996526+0000","last_became_peered":"2024-07-17T07:14:15.996526+0000","last_unstale":"2024-07-17T07:14:16.001061+0000","last_undegraded":"2024-07-17T07:14:16.001061+0000","last_fullsized":"2024-07-17T07:14:16.001061+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:44.895464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.999119+0000","last_change":"2024-07-17T07:14:15.990517+0000","last_active":"2024-07-17T07:14:15.999119+0000","last_peered":"2024-07-17T07:14:15.999119+0000","last_clean":"2024-07-17T07:14:15.999119+0000","last_became_active":"2024-07-17T07:14:15.990226+0000","last_became_peered":"2024-07-17T07:14:15.990226+0000","last_unstale":"2024-07-17T07:14:15.999119+0000","last_undegraded":"2024-07-17T07:14:15.999119+0000","last_fullsized":"2024-07-17T07:14:15.999119+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:28:50.571361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010850+0000","last_change":"2024-07-17T07:14:16.006259+0000","last_active":"2024-07-17T07:14:17.010850+0000","last_peered":"2024-07-17T07:14:17.010850+0000","last_clean":"2024-07-17T07:14:17.010850+0000","last_became_active":"2024-07-17T07:14:16.005785+0000","last_became_peered":"2024-07-17T07:14:16.005785+0000","last_unstale":"2024-07-17T07:14:17.010850+0000","last_undegraded":"2024-07-17T07:14:17.010850+0000","last_fullsized":"2024-07-17T07:14:17.010850+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:37.717537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989590+0000","last_change":"2024-07-17T07:14:16.003709+0000","last_active":"2024-07-17T07:14:16.989590+0000","last_peered":"2024-07-17T07:14:16.989590+0000","last_clean":"2024-07-17T07:14:16.989590+0000","last_became_active":"2024-07-17T07:14:16.003408+0000","last_became_peered":"2024-07-17T07:14:16.003408+0000","last_unstale":"2024-07-17T07:14:16.989590+0000","last_undegraded":"2024-07-17T07:14:16.989590+0000","last_fullsized":"2024-07-17T07:14:16.989590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:34.782870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984741+0000","last_change":"2024-07-17T07:14:15.998776+0000","last_active":"2024-07-17T07:14:16.984741+0000","last_peered":"2024-07-17T07:14:16.984741+0000","last_clean":"2024-07-17T07:14:16.984741+0000","last_became_active":"2024-07-17T07:14:15.998319+0000","last_became_peered":"2024-07-17T07:14:15.998319+0000","last_unstale":"2024-07-17T07:14:16.984741+0000","last_undegraded":"2024-07-17T07:14:16.984741+0000","last_fullsized":"2024-07-17T07:14:16.984741+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:06.326565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.002562+0000","last_change":"2024-07-17T07:14:16.000777+0000","last_active":"2024-07-17T07:14:16.002562+0000","last_peered":"2024-07-17T07:14:16.002562+0000","last_clean":"2024-07-17T07:14:16.002562+0000","last_became_active":"2024-07-17T07:14:16.000489+0000","last_became_peered":"2024-07-17T07:14:16.000489+0000","last_unstale":"2024-07-17T07:14:16.002562+0000","last_undegraded":"2024-07-17T07:14:16.002562+0000","last_fullsized":"2024-07-17T07:14:16.002562+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:51.220352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988910+0000","last_change":"2024-07-17T07:14:16.002290+0000","last_active":"2024-07-17T07:14:16.988910+0000","last_peered":"2024-07-17T07:14:16.988910+0000","last_clean":"2024-07-17T07:14:16.988910+0000","last_became_active":"2024-07-17T07:14:15.999571+0000","last_became_peered":"2024-07-17T07:14:15.999571+0000","last_unstale":"2024-07-17T07:14:16.988910+0000","last_undegraded":"2024-07-17T07:14:16.988910+0000","last_fullsized":"2024-07-17T07:14:16.988910+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:38.384428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983097+0000","last_change":"2024-07-17T07:14:15.982731+0000","last_active":"2024-07-17T07:14:16.983097+0000","last_peered":"2024-07-17T07:14:16.983097+0000","last_clean":"2024-07-17T07:14:16.983097+0000","last_became_active":"2024-07-17T07:14:15.981688+0000","last_became_peered":"2024-07-17T07:14:15.981688+0000","last_unstale":"2024-07-17T07:14:16.983097+0000","last_undegraded":"2024-07-17T07:14:16.983097+0000","last_fullsized":"2024-07-17T07:14:16.983097+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:32.802645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004401+0000","last_change":"2024-07-17T07:14:16.001377+0000","last_active":"2024-07-17T07:14:17.004401+0000","last_peered":"2024-07-17T07:14:17.004401+0000","last_clean":"2024-07-17T07:14:17.004401+0000","last_became_active":"2024-07-17T07:14:16.000340+0000","last_became_peered":"2024-07-17T07:14:16.000340+0000","last_unstale":"2024-07-17T07:14:17.004401+0000","last_undegraded":"2024-07-17T07:14:17.004401+0000","last_fullsized":"2024-07-17T07:14:17.004401+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.662506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"127'4","reported_seq":18,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.530320+0000","last_change":"2024-07-17T07:14:14.974979+0000","last_active":"2024-07-17T07:14:17.530320+0000","last_peered":"2024-07-17T07:14:17.530320+0000","last_clean":"2024-07-17T07:14:17.530320+0000","last_became_active":"2024-07-17T07:14:14.973817+0000","last_became_peered":"2024-07-17T07:14:14.973817+0000","last_unstale":"2024-07-17T07:14:17.530320+0000","last_undegraded":"2024-07-17T07:14:17.530320+0000","last_fullsized":"2024-07-17T07:14:17.530320+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:06.660284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983453+0000","last_change":"2024-07-17T07:14:16.015468+0000","last_active":"2024-07-17T07:14:16.983453+0000","last_peered":"2024-07-17T07:14:16.983453+0000","last_clean":"2024-07-17T07:14:16.983453+0000","last_became_active":"2024-07-17T07:14:16.013670+0000","last_became_peered":"2024-07-17T07:14:16.013670+0000","last_unstale":"2024-07-17T07:14:16.983453+0000","last_undegraded":"2024-07-17T07:14:16.983453+0000","last_fullsized":"2024-07-17T07:14:16.983453+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:41.930677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984678+0000","last_change":"2024-07-17T07:14:14.970196+0000","last_active":"2024-07-17T07:14:16.984678+0000","last_peered":"2024-07-17T07:14:16.984678+0000","last_clean":"2024-07-17T07:14:16.984678+0000","last_became_active":"2024-07-17T07:14:14.969926+0000","last_became_peered":"2024-07-17T07:14:14.969926+0000","last_unstale":"2024-07-17T07:14:16.984678+0000","last_undegraded":"2024-07-17T07:14:16.984678+0000","last_fullsized":"2024-07-17T07:14:16.984678+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:36:35.189388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.019420+0000","last_change":"2024-07-17T07:14:16.017827+0000","last_active":"2024-07-17T07:14:16.019420+0000","last_peered":"2024-07-17T07:14:16.019420+0000","last_clean":"2024-07-17T07:14:16.019420+0000","last_became_active":"2024-07-17T07:14:16.017550+0000","last_became_peered":"2024-07-17T07:14:16.017550+0000","last_unstale":"2024-07-17T07:14:16.019420+0000","last_undegraded":"2024-07-17T07:14:16.019420+0000","last_fullsized":"2024-07-17T07:14:16.019420+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:39:13.247448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"127'2","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.020047+0000","last_change":"2024-07-17T07:14:14.979346+0000","last_active":"2024-07-17T07:14:16.020047+0000","last_peered":"2024-07-17T07:14:16.020047+0000","last_clean":"2024-07-17T07:14:16.020047+0000","last_became_active":"2024-07-17T07:14:14.979072+0000","last_became_peered":"2024-07-17T07:14:14.979072+0000","last_unstale":"2024-07-17T07:14:16.020047+0000","last_undegraded":"2024-07-17T07:14:16.020047+0000","last_fullsized":"2024-07-17T07:14:16.020047+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:58.167433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010166+0000","last_change":"2024-07-17T07:14:16.004647+0000","last_active":"2024-07-17T07:14:17.010166+0000","last_peered":"2024-07-17T07:14:17.010166+0000","last_clean":"2024-07-17T07:14:17.010166+0000","last_became_active":"2024-07-17T07:14:16.002100+0000","last_became_peered":"2024-07-17T07:14:16.002100+0000","last_unstale":"2024-07-17T07:14:17.010166+0000","last_undegraded":"2024-07-17T07:14:17.010166+0000","last_fullsized":"2024-07-17T07:14:17.010166+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:07.669256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985415+0000","last_change":"2024-07-17T07:14:14.976145+0000","last_active":"2024-07-17T07:14:16.985415+0000","last_peered":"2024-07-17T07:14:16.985415+0000","last_clean":"2024-07-17T07:14:16.985415+0000","last_became_active":"2024-07-17T07:14:14.975865+0000","last_became_peered":"2024-07-17T07:14:14.975865+0000","last_unstale":"2024-07-17T07:14:16.985415+0000","last_undegraded":"2024-07-17T07:14:16.985415+0000","last_fullsized":"2024-07-17T07:14:16.985415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:03.475977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529897+0000","last_change":"2024-07-17T07:14:16.006761+0000","last_active":"2024-07-17T07:14:17.529897+0000","last_peered":"2024-07-17T07:14:17.529897+0000","last_clean":"2024-07-17T07:14:17.529897+0000","last_became_active":"2024-07-17T07:14:16.006479+0000","last_became_peered":"2024-07-17T07:14:16.006479+0000","last_unstale":"2024-07-17T07:14:17.529897+0000","last_undegraded":"2024-07-17T07:14:17.529897+0000","last_fullsized":"2024-07-17T07:14:17.529897+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:50.524212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238896+0000","last_change":"2024-07-17T07:14:14.973184+0000","last_active":"2024-07-17T07:14:17.238896+0000","last_peered":"2024-07-17T07:14:17.238896+0000","last_clean":"2024-07-17T07:14:17.238896+0000","last_became_active":"2024-07-17T07:14:14.972752+0000","last_became_peered":"2024-07-17T07:14:14.972752+0000","last_unstale":"2024-07-17T07:14:17.238896+0000","last_undegraded":"2024-07-17T07:14:17.238896+0000","last_fullsized":"2024-07-17T07:14:17.238896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:43:21.724486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005209+0000","last_change":"2024-07-17T07:14:16.000179+0000","last_active":"2024-07-17T07:14:17.005209+0000","last_peered":"2024-07-17T07:14:17.005209+0000","last_clean":"2024-07-17T07:14:17.005209+0000","last_became_active":"2024-07-17T07:14:15.999408+0000","last_became_peered":"2024-07-17T07:14:15.999408+0000","last_unstale":"2024-07-17T07:14:17.005209+0000","last_undegraded":"2024-07-17T07:14:17.005209+0000","last_fullsized":"2024-07-17T07:14:17.005209+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:04.449955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009715+0000","last_change":"2024-07-17T07:14:14.976881+0000","last_active":"2024-07-17T07:14:17.009715+0000","last_peered":"2024-07-17T07:14:17.009715+0000","last_clean":"2024-07-17T07:14:17.009715+0000","last_became_active":"2024-07-17T07:14:14.976552+0000","last_became_peered":"2024-07-17T07:14:14.976552+0000","last_unstale":"2024-07-17T07:14:17.009715+0000","last_undegraded":"2024-07-17T07:14:17.009715+0000","last_fullsized":"2024-07-17T07:14:17.009715+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:40.177290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.990249+0000","last_change":"2024-07-17T07:14:16.000551+0000","last_active":"2024-07-17T07:14:16.990249+0000","last_peered":"2024-07-17T07:14:16.990249+0000","last_clean":"2024-07-17T07:14:16.990249+0000","last_became_active":"2024-07-17T07:14:16.000269+0000","last_became_peered":"2024-07-17T07:14:16.000269+0000","last_unstale":"2024-07-17T07:14:16.990249+0000","last_undegraded":"2024-07-17T07:14:16.990249+0000","last_fullsized":"2024-07-17T07:14:16.990249+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:59:42.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.984361+0000","last_change":"2024-07-17T07:14:14.976043+0000","last_active":"2024-07-17T07:14:15.984361+0000","last_peered":"2024-07-17T07:14:15.984361+0000","last_clean":"2024-07-17T07:14:15.984361+0000","last_became_active":"2024-07-17T07:14:14.975739+0000","last_became_peered":"2024-07-17T07:14:14.975739+0000","last_unstale":"2024-07-17T07:14:15.984361+0000","last_undegraded":"2024-07-17T07:14:15.984361+0000","last_fullsized":"2024-07-17T07:14:15.984361+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:21.800503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.530262+0000","last_change":"2024-07-17T07:14:16.011247+0000","last_active":"2024-07-17T07:14:17.530262+0000","last_peered":"2024-07-17T07:14:17.530262+0000","last_clean":"2024-07-17T07:14:17.530262+0000","last_became_active":"2024-07-17T07:14:16.010107+0000","last_became_peered":"2024-07-17T07:14:16.010107+0000","last_unstale":"2024-07-17T07:14:17.530262+0000","last_undegraded":"2024-07-17T07:14:17.530262+0000","last_fullsized":"2024-07-17T07:14:17.530262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:27.014918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529400+0000","last_change":"2024-07-17T07:14:14.975428+0000","last_active":"2024-07-17T07:14:17.529400+0000","last_peered":"2024-07-17T07:14:17.529400+0000","last_clean":"2024-07-17T07:14:17.529400+0000","last_became_active":"2024-07-17T07:14:14.975074+0000","last_became_peered":"2024-07-17T07:14:14.975074+0000","last_unstale":"2024-07-17T07:14:17.529400+0000","last_undegraded":"2024-07-17T07:14:17.529400+0000","last_fullsized":"2024-07-17T07:14:17.529400+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:44.423592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004406+0000","last_change":"2024-07-17T07:14:16.005726+0000","last_active":"2024-07-17T07:14:17.004406+0000","last_peered":"2024-07-17T07:14:17.004406+0000","last_clean":"2024-07-17T07:14:17.004406+0000","last_became_active":"2024-07-17T07:14:16.002141+0000","last_became_peered":"2024-07-17T07:14:16.002141+0000","last_unstale":"2024-07-17T07:14:17.004406+0000","last_undegraded":"2024-07-17T07:14:17.004406+0000","last_fullsized":"2024-07-17T07:14:17.004406+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:37.487996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.983724+0000","last_change":"2024-07-17T07:14:14.982510+0000","last_active":"2024-07-17T07:14:15.983724+0000","last_peered":"2024-07-17T07:14:15.983724+0000","last_clean":"2024-07-17T07:14:15.983724+0000","last_became_active":"2024-07-17T07:14:14.981063+0000","last_became_peered":"2024-07-17T07:14:14.981063+0000","last_unstale":"2024-07-17T07:14:15.983724+0000","last_undegraded":"2024-07-17T07:14:15.983724+0000","last_fullsized":"2024-07-17T07:14:15.983724+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:58.525999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985408+0000","last_change":"2024-07-17T07:14:16.009291+0000","last_active":"2024-07-17T07:14:16.985408+0000","last_peered":"2024-07-17T07:14:16.985408+0000","last_clean":"2024-07-17T07:14:16.985408+0000","last_became_active":"2024-07-17T07:14:16.002499+0000","last_became_peered":"2024-07-17T07:14:16.002499+0000","last_unstale":"2024-07-17T07:14:16.985408+0000","last_undegraded":"2024-07-17T07:14:16.985408+0000","last_fullsized":"2024-07-17T07:14:16.985408+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:49.076142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004363+0000","last_change":"2024-07-17T07:14:14.978277+0000","last_active":"2024-07-17T07:14:17.004363+0000","last_peered":"2024-07-17T07:14:17.004363+0000","last_clean":"2024-07-17T07:14:17.004363+0000","last_became_active":"2024-07-17T07:14:14.977993+0000","last_became_peered":"2024-07-17T07:14:14.977993+0000","last_unstale":"2024-07-17T07:14:17.004363+0000","last_undegraded":"2024-07-17T07:14:17.004363+0000","last_fullsized":"2024-07-17T07:14:17.004363+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:04.492201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983834+0000","last_change":"2024-07-17T07:14:16.006843+0000","last_active":"2024-07-17T07:14:16.983834+0000","last_peered":"2024-07-17T07:14:16.983834+0000","last_clean":"2024-07-17T07:14:16.983834+0000","last_became_active":"2024-07-17T07:14:16.006491+0000","last_became_peered":"2024-07-17T07:14:16.006491+0000","last_unstale":"2024-07-17T07:14:16.983834+0000","last_undegraded":"2024-07-17T07:14:16.983834+0000","last_fullsized":"2024-07-17T07:14:16.983834+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:49.004690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"127'3","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238589+0000","last_change":"2024-07-17T07:14:14.975659+0000","last_active":"2024-07-17T07:14:17.238589+0000","last_peered":"2024-07-17T07:14:17.238589+0000","last_clean":"2024-07-17T07:14:17.238589+0000","last_became_active":"2024-07-17T07:14:14.975373+0000","last_became_peered":"2024-07-17T07:14:14.975373+0000","last_unstale":"2024-07-17T07:14:17.238589+0000","last_undegraded":"2024-07-17T07:14:17.238589+0000","last_fullsized":"2024-07-17T07:14:17.238589+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:47:41.076767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985312+0000","last_change":"2024-07-17T07:14:16.006366+0000","last_active":"2024-07-17T07:14:16.985312+0000","last_peered":"2024-07-17T07:14:16.985312+0000","last_clean":"2024-07-17T07:14:16.985312+0000","last_became_active":"2024-07-17T07:14:16.006064+0000","last_became_peered":"2024-07-17T07:14:16.006064+0000","last_unstale":"2024-07-17T07:14:16.985312+0000","last_undegraded":"2024-07-17T07:14:16.985312+0000","last_fullsized":"2024-07-17T07:14:16.985312+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:23:59.931616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"127'6","reported_seq":20,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989399+0000","last_change":"2024-07-17T07:14:14.979165+0000","last_active":"2024-07-17T07:14:16.989399+0000","last_peered":"2024-07-17T07:14:16.989399+0000","last_clean":"2024-07-17T07:14:16.989399+0000","last_became_active":"2024-07-17T07:14:14.977784+0000","last_became_peered":"2024-07-17T07:14:14.977784+0000","last_unstale":"2024-07-17T07:14:16.989399+0000","last_undegraded":"2024-07-17T07:14:16.989399+0000","last_fullsized":"2024-07-17T07:14:16.989399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:40.080968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010343+0000","last_change":"2024-07-17T07:14:16.011941+0000","last_active":"2024-07-17T07:14:17.010343+0000","last_peered":"2024-07-17T07:14:17.010343+0000","last_clean":"2024-07-17T07:14:17.010343+0000","last_became_active":"2024-07-17T07:14:16.011634+0000","last_became_peered":"2024-07-17T07:14:16.011634+0000","last_unstale":"2024-07-17T07:14:17.010343+0000","last_undegraded":"2024-07-17T07:14:17.010343+0000","last_fullsized":"2024-07-17T07:14:17.010343+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:00.439218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985517+0000","last_change":"2024-07-17T07:14:14.977396+0000","last_active":"2024-07-17T07:14:16.985517+0000","last_peered":"2024-07-17T07:14:16.985517+0000","last_clean":"2024-07-17T07:14:16.985517+0000","last_became_active":"2024-07-17T07:14:14.977126+0000","last_became_peered":"2024-07-17T07:14:14.977126+0000","last_unstale":"2024-07-17T07:14:16.985517+0000","last_undegraded":"2024-07-17T07:14:16.985517+0000","last_fullsized":"2024-07-17T07:14:16.985517+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:31:44.879824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989975+0000","last_change":"2024-07-17T07:14:15.995870+0000","last_active":"2024-07-17T07:14:16.989975+0000","last_peered":"2024-07-17T07:14:16.989975+0000","last_clean":"2024-07-17T07:14:16.989975+0000","last_became_active":"2024-07-17T07:14:15.995551+0000","last_became_peered":"2024-07-17T07:14:15.995551+0000","last_unstale":"2024-07-17T07:14:16.989975+0000","last_undegraded":"2024-07-17T07:14:16.989975+0000","last_fullsized":"2024-07-17T07:14:16.989975+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:21.317048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529664+0000","last_change":"2024-07-17T07:14:14.973195+0000","last_active":"2024-07-17T07:14:17.529664+0000","last_peered":"2024-07-17T07:14:17.529664+0000","last_clean":"2024-07-17T07:14:17.529664+0000","last_became_active":"2024-07-17T07:14:14.972804+0000","last_became_peered":"2024-07-17T07:14:14.972804+0000","last_unstale":"2024-07-17T07:14:17.529664+0000","last_undegraded":"2024-07-17T07:14:17.529664+0000","last_fullsized":"2024-07-17T07:14:17.529664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:09.471868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984337+0000","last_change":"2024-07-17T07:14:15.998253+0000","last_active":"2024-07-17T07:14:16.984337+0000","last_peered":"2024-07-17T07:14:16.984337+0000","last_clean":"2024-07-17T07:14:16.984337+0000","last_became_active":"2024-07-17T07:14:15.997492+0000","last_became_peered":"2024-07-17T07:14:15.997492+0000","last_unstale":"2024-07-17T07:14:16.984337+0000","last_undegraded":"2024-07-17T07:14:16.984337+0000","last_fullsized":"2024-07-17T07:14:16.984337+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:40:15.837434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983473+0000","last_change":"2024-07-17T07:14:14.976413+0000","last_active":"2024-07-17T07:14:16.983473+0000","last_peered":"2024-07-17T07:14:16.983473+0000","last_clean":"2024-07-17T07:14:16.983473+0000","last_became_active":"2024-07-17T07:14:14.976130+0000","last_became_peered":"2024-07-17T07:14:14.976130+0000","last_unstale":"2024-07-17T07:14:16.983473+0000","last_undegraded":"2024-07-17T07:14:16.983473+0000","last_fullsized":"2024-07-17T07:14:16.983473+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:15:04.605713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005187+0000","last_change":"2024-07-17T07:14:16.012137+0000","last_active":"2024-07-17T07:14:17.005187+0000","last_peered":"2024-07-17T07:14:17.005187+0000","last_clean":"2024-07-17T07:14:17.005187+0000","last_became_active":"2024-07-17T07:14:16.011379+0000","last_became_peered":"2024-07-17T07:14:16.011379+0000","last_unstale":"2024-07-17T07:14:17.005187+0000","last_undegraded":"2024-07-17T07:14:17.005187+0000","last_fullsized":"2024-07-17T07:14:17.005187+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:03.823127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"127'8","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.025887+0000","last_change":"2024-07-17T07:14:14.978899+0000","last_active":"2024-07-17T07:14:16.025887+0000","last_peered":"2024-07-17T07:14:16.025887+0000","last_clean":"2024-07-17T07:14:16.025887+0000","last_became_active":"2024-07-17T07:14:14.978596+0000","last_became_peered":"2024-07-17T07:14:14.978596+0000","last_unstale":"2024-07-17T07:14:16.025887+0000","last_undegraded":"2024-07-17T07:14:16.025887+0000","last_fullsized":"2024-07-17T07:14:16.025887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:57.620154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983028+0000","last_change":"2024-07-17T07:14:16.002003+0000","last_active":"2024-07-17T07:14:16.983028+0000","last_peered":"2024-07-17T07:14:16.983028+0000","last_clean":"2024-07-17T07:14:16.983028+0000","last_became_active":"2024-07-17T07:14:16.001707+0000","last_became_peered":"2024-07-17T07:14:16.001707+0000","last_unstale":"2024-07-17T07:14:16.983028+0000","last_undegraded":"2024-07-17T07:14:16.983028+0000","last_fullsized":"2024-07-17T07:14:16.983028+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:58.591119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989787+0000","last_change":"2024-07-17T07:14:14.976840+0000","last_active":"2024-07-17T07:14:16.989787+0000","last_peered":"2024-07-17T07:14:16.989787+0000","last_clean":"2024-07-17T07:14:16.989787+0000","last_became_active":"2024-07-17T07:14:14.976396+0000","last_became_peered":"2024-07-17T07:14:14.976396+0000","last_unstale":"2024-07-17T07:14:16.989787+0000","last_undegraded":"2024-07-17T07:14:16.989787+0000","last_fullsized":"2024-07-17T07:14:16.989787+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:48.601722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988653+0000","last_change":"2024-07-17T07:14:15.997739+0000","last_active":"2024-07-17T07:14:16.988653+0000","last_peered":"2024-07-17T07:14:16.988653+0000","last_clean":"2024-07-17T07:14:16.988653+0000","last_became_active":"2024-07-17T07:14:15.997417+0000","last_became_peered":"2024-07-17T07:14:15.997417+0000","last_unstale":"2024-07-17T07:14:16.988653+0000","last_undegraded":"2024-07-17T07:14:16.988653+0000","last_fullsized":"2024-07-17T07:14:16.988653+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:25.163906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"127'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238741+0000","last_change":"2024-07-17T07:14:14.978093+0000","last_active":"2024-07-17T07:14:17.238741+0000","last_peered":"2024-07-17T07:14:17.238741+0000","last_clean":"2024-07-17T07:14:17.238741+0000","last_became_active":"2024-07-17T07:14:14.977688+0000","last_became_peered":"2024-07-17T07:14:14.977688+0000","last_unstale":"2024-07-17T07:14:17.238741+0000","last_undegraded":"2024-07-17T07:14:17.238741+0000","last_fullsized":"2024-07-17T07:14:17.238741+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:49.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985629+0000","last_change":"2024-07-17T07:14:16.019118+0000","last_active":"2024-07-17T07:14:16.985629+0000","last_peered":"2024-07-17T07:14:16.985629+0000","last_clean":"2024-07-17T07:14:16.985629+0000","last_became_active":"2024-07-17T07:14:16.012518+0000","last_became_peered":"2024-07-17T07:14:16.012518+0000","last_unstale":"2024-07-17T07:14:16.985629+0000","last_undegraded":"2024-07-17T07:14:16.985629+0000","last_fullsized":"2024-07-17T07:14:16.985629+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:11.542469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.010509+0000","last_change":"2024-07-17T07:14:14.980435+0000","last_active":"2024-07-17T07:14:17.010509+0000","last_peered":"2024-07-17T07:14:17.010509+0000","last_clean":"2024-07-17T07:14:17.010509+0000","last_became_active":"2024-07-17T07:14:14.979864+0000","last_became_peered":"2024-07-17T07:14:14.979864+0000","last_unstale":"2024-07-17T07:14:17.010509+0000","last_undegraded":"2024-07-17T07:14:17.010509+0000","last_fullsized":"2024-07-17T07:14:17.010509+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:44.484742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004361+0000","last_change":"2024-07-17T07:14:16.018290+0000","last_active":"2024-07-17T07:14:17.004361+0000","last_peered":"2024-07-17T07:14:17.004361+0000","last_clean":"2024-07-17T07:14:17.004361+0000","last_became_active":"2024-07-17T07:14:16.017965+0000","last_became_peered":"2024-07-17T07:14:16.017965+0000","last_unstale":"2024-07-17T07:14:17.004361+0000","last_undegraded":"2024-07-17T07:14:17.004361+0000","last_fullsized":"2024-07-17T07:14:17.004361+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:40.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984708+0000","last_change":"2024-07-17T07:14:14.970797+0000","last_active":"2024-07-17T07:14:16.984708+0000","last_peered":"2024-07-17T07:14:16.984708+0000","last_clean":"2024-07-17T07:14:16.984708+0000","last_became_active":"2024-07-17T07:14:14.969013+0000","last_became_peered":"2024-07-17T07:14:14.969013+0000","last_unstale":"2024-07-17T07:14:16.984708+0000","last_undegraded":"2024-07-17T07:14:16.984708+0000","last_fullsized":"2024-07-17T07:14:16.984708+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:23.982444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.005032+0000","last_change":"2024-07-17T07:14:16.006338+0000","last_active":"2024-07-17T07:14:17.005032+0000","last_peered":"2024-07-17T07:14:17.005032+0000","last_clean":"2024-07-17T07:14:17.005032+0000","last_became_active":"2024-07-17T07:14:16.006015+0000","last_became_peered":"2024-07-17T07:14:16.006015+0000","last_unstale":"2024-07-17T07:14:17.005032+0000","last_undegraded":"2024-07-17T07:14:17.005032+0000","last_fullsized":"2024-07-17T07:14:17.005032+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:12.463368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983498+0000","last_change":"2024-07-17T07:14:14.967510+0000","last_active":"2024-07-17T07:14:16.983498+0000","last_peered":"2024-07-17T07:14:16.983498+0000","last_clean":"2024-07-17T07:14:16.983498+0000","last_became_active":"2024-07-17T07:14:14.967199+0000","last_became_peered":"2024-07-17T07:14:14.967199+0000","last_unstale":"2024-07-17T07:14:16.983498+0000","last_undegraded":"2024-07-17T07:14:16.983498+0000","last_fullsized":"2024-07-17T07:14:16.983498+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:16.851764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529510+0000","last_change":"2024-07-17T07:14:16.010062+0000","last_active":"2024-07-17T07:14:17.529510+0000","last_peered":"2024-07-17T07:14:17.529510+0000","last_clean":"2024-07-17T07:14:17.529510+0000","last_became_active":"2024-07-17T07:14:16.009574+0000","last_became_peered":"2024-07-17T07:14:16.009574+0000","last_unstale":"2024-07-17T07:14:17.529510+0000","last_undegraded":"2024-07-17T07:14:17.529510+0000","last_fullsized":"2024-07-17T07:14:17.529510+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:59.609702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988737+0000","last_change":"2024-07-17T07:14:14.976419+0000","last_active":"2024-07-17T07:14:16.988737+0000","last_peered":"2024-07-17T07:14:16.988737+0000","last_clean":"2024-07-17T07:14:16.988737+0000","last_became_active":"2024-07-17T07:14:14.976098+0000","last_became_peered":"2024-07-17T07:14:14.976098+0000","last_unstale":"2024-07-17T07:14:16.988737+0000","last_undegraded":"2024-07-17T07:14:16.988737+0000","last_fullsized":"2024-07-17T07:14:16.988737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:05.701624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984178+0000","last_change":"2024-07-17T07:14:16.014314+0000","last_active":"2024-07-17T07:14:16.984178+0000","last_peered":"2024-07-17T07:14:16.984178+0000","last_clean":"2024-07-17T07:14:16.984178+0000","last_became_active":"2024-07-17T07:14:16.013974+0000","last_became_peered":"2024-07-17T07:14:16.013974+0000","last_unstale":"2024-07-17T07:14:16.984178+0000","last_undegraded":"2024-07-17T07:14:16.984178+0000","last_fullsized":"2024-07-17T07:14:16.984178+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:43.888659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983169+0000","last_change":"2024-07-17T07:14:14.966796+0000","last_active":"2024-07-17T07:14:16.983169+0000","last_peered":"2024-07-17T07:14:16.983169+0000","last_clean":"2024-07-17T07:14:16.983169+0000","last_became_active":"2024-07-17T07:14:14.966484+0000","last_became_peered":"2024-07-17T07:14:14.966484+0000","last_unstale":"2024-07-17T07:14:16.983169+0000","last_undegraded":"2024-07-17T07:14:16.983169+0000","last_fullsized":"2024-07-17T07:14:16.983169+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:37.943347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.004012+0000","last_change":"2024-07-17T07:14:16.022653+0000","last_active":"2024-07-17T07:14:17.004012+0000","last_peered":"2024-07-17T07:14:17.004012+0000","last_clean":"2024-07-17T07:14:17.004012+0000","last_became_active":"2024-07-17T07:14:16.022273+0000","last_became_peered":"2024-07-17T07:14:16.022273+0000","last_unstale":"2024-07-17T07:14:17.004012+0000","last_undegraded":"2024-07-17T07:14:17.004012+0000","last_fullsized":"2024-07-17T07:14:17.004012+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:25.404667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985709+0000","last_change":"2024-07-17T07:14:14.980947+0000","last_active":"2024-07-17T07:14:16.985709+0000","last_peered":"2024-07-17T07:14:16.985709+0000","last_clean":"2024-07-17T07:14:16.985709+0000","last_became_active":"2024-07-17T07:14:14.980606+0000","last_became_peered":"2024-07-17T07:14:14.980606+0000","last_unstale":"2024-07-17T07:14:16.985709+0000","last_undegraded":"2024-07-17T07:14:16.985709+0000","last_fullsized":"2024-07-17T07:14:16.985709+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:51:41.160690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.007519+0000","last_change":"2024-07-17T07:14:16.005475+0000","last_active":"2024-07-17T07:14:16.007519+0000","last_peered":"2024-07-17T07:14:16.007519+0000","last_clean":"2024-07-17T07:14:16.007519+0000","last_became_active":"2024-07-17T07:14:16.005130+0000","last_became_peered":"2024-07-17T07:14:16.005130+0000","last_unstale":"2024-07-17T07:14:16.007519+0000","last_undegraded":"2024-07-17T07:14:16.007519+0000","last_fullsized":"2024-07-17T07:14:16.007519+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:40.603992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"128'1","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.007972+0000","last_change":"2024-07-17T07:14:14.976115+0000","last_active":"2024-07-17T07:14:17.007972+0000","last_peered":"2024-07-17T07:14:17.007972+0000","last_clean":"2024-07-17T07:14:17.007972+0000","last_became_active":"2024-07-17T07:14:14.975487+0000","last_became_peered":"2024-07-17T07:14:14.975487+0000","last_unstale":"2024-07-17T07:14:17.007972+0000","last_undegraded":"2024-07-17T07:14:17.007972+0000","last_fullsized":"2024-07-17T07:14:17.007972+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:15:34.933058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.529417+0000","last_change":"2024-07-17T07:14:16.007519+0000","last_active":"2024-07-17T07:14:17.529417+0000","last_peered":"2024-07-17T07:14:17.529417+0000","last_clean":"2024-07-17T07:14:17.529417+0000","last_became_active":"2024-07-17T07:14:16.007124+0000","last_became_peered":"2024-07-17T07:14:16.007124+0000","last_unstale":"2024-07-17T07:14:17.529417+0000","last_undegraded":"2024-07-17T07:14:17.529417+0000","last_fullsized":"2024-07-17T07:14:17.529417+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:51.011150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.989504+0000","last_change":"2024-07-17T07:14:14.975415+0000","last_active":"2024-07-17T07:14:16.989504+0000","last_peered":"2024-07-17T07:14:16.989504+0000","last_clean":"2024-07-17T07:14:16.989504+0000","last_became_active":"2024-07-17T07:14:14.975117+0000","last_became_peered":"2024-07-17T07:14:14.975117+0000","last_unstale":"2024-07-17T07:14:16.989504+0000","last_undegraded":"2024-07-17T07:14:16.989504+0000","last_fullsized":"2024-07-17T07:14:16.989504+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:18:33.181629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009944+0000","last_change":"2024-07-17T07:14:16.010906+0000","last_active":"2024-07-17T07:14:17.009944+0000","last_peered":"2024-07-17T07:14:17.009944+0000","last_clean":"2024-07-17T07:14:17.009944+0000","last_became_active":"2024-07-17T07:14:16.010118+0000","last_became_peered":"2024-07-17T07:14:16.010118+0000","last_unstale":"2024-07-17T07:14:17.009944+0000","last_undegraded":"2024-07-17T07:14:17.009944+0000","last_fullsized":"2024-07-17T07:14:17.009944+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:10.480840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.985575+0000","last_change":"2024-07-17T07:14:14.979169+0000","last_active":"2024-07-17T07:14:16.985575+0000","last_peered":"2024-07-17T07:14:16.985575+0000","last_clean":"2024-07-17T07:14:16.985575+0000","last_became_active":"2024-07-17T07:14:14.978882+0000","last_became_peered":"2024-07-17T07:14:14.978882+0000","last_unstale":"2024-07-17T07:14:16.985575+0000","last_undegraded":"2024-07-17T07:14:16.985575+0000","last_fullsized":"2024-07-17T07:14:16.985575+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:51.787457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984018+0000","last_change":"2024-07-17T07:14:16.006420+0000","last_active":"2024-07-17T07:14:16.984018+0000","last_peered":"2024-07-17T07:14:16.984018+0000","last_clean":"2024-07-17T07:14:16.984018+0000","last_became_active":"2024-07-17T07:14:16.004018+0000","last_became_peered":"2024-07-17T07:14:16.004018+0000","last_unstale":"2024-07-17T07:14:16.984018+0000","last_undegraded":"2024-07-17T07:14:16.984018+0000","last_fullsized":"2024-07-17T07:14:16.984018+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:09.543489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-07-17T07:14:15.982907+0000","last_change":"2024-07-17T07:14:14.968717+0000","last_active":"2024-07-17T07:14:15.982907+0000","last_peered":"2024-07-17T07:14:15.982907+0000","last_clean":"2024-07-17T07:14:15.982907+0000","last_became_active":"2024-07-17T07:14:14.968383+0000","last_became_peered":"2024-07-17T07:14:14.968383+0000","last_unstale":"2024-07-17T07:14:15.982907+0000","last_undegraded":"2024-07-17T07:14:15.982907+0000","last_fullsized":"2024-07-17T07:14:15.982907+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:37:23.544277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983944+0000","last_change":"2024-07-17T07:14:16.005048+0000","last_active":"2024-07-17T07:14:16.983944+0000","last_peered":"2024-07-17T07:14:16.983944+0000","last_clean":"2024-07-17T07:14:16.983944+0000","last_became_active":"2024-07-17T07:14:16.004757+0000","last_became_peered":"2024-07-17T07:14:16.004757+0000","last_unstale":"2024-07-17T07:14:16.983944+0000","last_undegraded":"2024-07-17T07:14:16.983944+0000","last_fullsized":"2024-07-17T07:14:16.983944+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:52:33.466147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983644+0000","last_change":"2024-07-17T07:14:14.978987+0000","last_active":"2024-07-17T07:14:16.983644+0000","last_peered":"2024-07-17T07:14:16.983644+0000","last_clean":"2024-07-17T07:14:16.983644+0000","last_became_active":"2024-07-17T07:14:14.977039+0000","last_became_peered":"2024-07-17T07:14:14.977039+0000","last_unstale":"2024-07-17T07:14:16.983644+0000","last_undegraded":"2024-07-17T07:14:16.983644+0000","last_fullsized":"2024-07-17T07:14:16.983644+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:47.950517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238916+0000","last_change":"2024-07-17T07:14:16.004077+0000","last_active":"2024-07-17T07:14:17.238916+0000","last_peered":"2024-07-17T07:14:17.238916+0000","last_clean":"2024-07-17T07:14:17.238916+0000","last_became_active":"2024-07-17T07:14:16.003805+0000","last_became_peered":"2024-07-17T07:14:16.003805+0000","last_unstale":"2024-07-17T07:14:17.238916+0000","last_undegraded":"2024-07-17T07:14:17.238916+0000","last_fullsized":"2024-07-17T07:14:17.238916+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:59.710757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.988991+0000","last_change":"2024-07-17T07:14:14.974611+0000","last_active":"2024-07-17T07:14:16.988991+0000","last_peered":"2024-07-17T07:14:16.988991+0000","last_clean":"2024-07-17T07:14:16.988991+0000","last_became_active":"2024-07-17T07:14:14.974325+0000","last_became_peered":"2024-07-17T07:14:14.974325+0000","last_unstale":"2024-07-17T07:14:16.988991+0000","last_undegraded":"2024-07-17T07:14:16.988991+0000","last_fullsized":"2024-07-17T07:14:16.988991+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:23:00.804718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.011232+0000","last_change":"2024-07-17T07:14:16.006149+0000","last_active":"2024-07-17T07:14:17.011232+0000","last_peered":"2024-07-17T07:14:17.011232+0000","last_clean":"2024-07-17T07:14:17.011232+0000","last_became_active":"2024-07-17T07:14:16.005884+0000","last_became_peered":"2024-07-17T07:14:16.005884+0000","last_unstale":"2024-07-17T07:14:17.011232+0000","last_undegraded":"2024-07-17T07:14:17.011232+0000","last_fullsized":"2024-07-17T07:14:17.011232+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:39.246748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.983368+0000","last_change":"2024-07-17T07:14:14.962790+0000","last_active":"2024-07-17T07:14:16.983368+0000","last_peered":"2024-07-17T07:14:16.983368+0000","last_clean":"2024-07-17T07:14:16.983368+0000","last_became_active":"2024-07-17T07:14:14.962497+0000","last_became_peered":"2024-07-17T07:14:14.962497+0000","last_unstale":"2024-07-17T07:14:16.983368+0000","last_undegraded":"2024-07-17T07:14:16.983368+0000","last_fullsized":"2024-07-17T07:14:16.983368+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:53:47.163320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.009335+0000","last_change":"2024-07-17T07:14:16.012570+0000","last_active":"2024-07-17T07:14:17.009335+0000","last_peered":"2024-07-17T07:14:17.009335+0000","last_clean":"2024-07-17T07:14:17.009335+0000","last_became_active":"2024-07-17T07:14:16.012298+0000","last_became_peered":"2024-07-17T07:14:16.012298+0000","last_unstale":"2024-07-17T07:14:17.009335+0000","last_undegraded":"2024-07-17T07:14:17.009335+0000","last_fullsized":"2024-07-17T07:14:17.009335+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:14.931588+0000","last_clean_scrub_stamp":"2024-07-17T07:14:14.931588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:45.432564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"127'2","reported_seq":16,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:16.984976+0000","last_change":"2024-07-17T07:14:14.967431+0000","last_active":"2024-07-17T07:14:16.984976+0000","last_peered":"2024-07-17T07:14:16.984976+0000","last_clean":"2024-07-17T07:14:16.984976+0000","last_became_active":"2024-07-17T07:14:14.966501+0000","last_became_peered":"2024-07-17T07:14:14.966501+0000","last_unstale":"2024-07-17T07:14:16.984976+0000","last_undegraded":"2024-07-17T07:14:16.984976+0000","last_fullsized":"2024-07-17T07:14:16.984976+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:14:13.922616+0000","last_clean_scrub_stamp":"2024-07-17T07:14:13.922616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:57.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"118'2420","reported_seq":2841,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-07-17T07:14:17.238001+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:14:17.238001+0000","last_peered":"2024-07-17T07:14:17.238001+0000","last_clean":"2024-07-17T07:14:17.238001+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:14:17.238001+0000","last_undegraded":"2024-07-17T07:14:17.238001+0000","last_fullsized":"2024-07-17T07:14:17.238001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":203,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":607,"num_read_kb":1583,"num_write":203,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2420,"ondisk_log_size":2420,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542376,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51356,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320484,"statfs":{"total":96636764160,"available":96584175616,"internally_reserved":0,"allocated":2179072,"data_stored":1423987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509671,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":835675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509672,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50760,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321080,"statfs":{"total":96636764160,"available":96584785920,"internally_reserved":0,"allocated":1564672,"data_stored":826247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542377,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":826339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1568768,"data_stored":826829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59096,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312744,"statfs":{"total":96636764160,"available":96576249856,"internally_reserved":0,"allocated":1581056,"data_stored":833493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1581056,"data_stored":833493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509671,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29156,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342684,"statfs":{"total":96636764160,"available":96606908416,"internally_reserved":0,"allocated":2187264,"data_stored":1425529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:14:20.999 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:14:20.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:14:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 -- 172.21.15.44:0/133233801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f815a5a0 msgr2=0x7f23f80b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 --2- 172.21.15.44:0/133233801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f815a5a0 0x7f23f80b3280 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f23f80b1f20 tx=0x7f23e402a480 comp rx=0 tx=0).stop 2024-07-17T07:14:21.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 -- 172.21.15.44:0/133233801 shutdown_connections 2024-07-17T07:14:21.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 --2- 172.21.15.44:0/133233801 >> v2:172.21.15.175:3300/0 conn(0x7f23f80ac3a0 0x7f23f80ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 --2- 172.21.15.44:0/133233801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f815a5a0 0x7f23f80b3280 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 -- 172.21.15.44:0/133233801 >> 172.21.15.44:0/133233801 conn(0x7f23f80b0eb0 msgr2=0x7f23f80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:21.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 -- 172.21.15.44:0/133233801 shutdown_connections 2024-07-17T07:14:21.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.269+0000 7f23fd400640 1 -- 172.21.15.44:0/133233801 wait complete. 2024-07-17T07:14:21.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.271+0000 7f23fd400640 1 Processor -- start 2024-07-17T07:14:21.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.271+0000 7f23fd400640 1 -- start start 2024-07-17T07:14:21.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 0x7f23f80cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23f815a5a0 0x7f23f81f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23f806f850 con 0x7f23f806eb90 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23f806f6d0 con 0x7f23f815a5a0 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.272+0000 7f23fd400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23f814fd60 con 0x7f23f80ac3a0 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35750/0 (socket says 172.21.15.44:35750) 2024-07-17T07:14:21.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 -- 172.21.15.44:0/3480347936 learned_addr learned my addr 172.21.15.44:0/3480347936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:21.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f7200640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23f815a5a0 0x7f23f81f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f5e00640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 0x7f23f80cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 -- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 msgr2=0x7f23f80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 0x7f23f80cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 -- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23f815a5a0 msgr2=0x7f23f81f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23f815a5a0 0x7f23f81f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f6800640 1 -- 172.21.15.44:0/3480347936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f80cbd00 con 0x7f23f806eb90 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.273+0000 7f23f5e00640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 0x7f23f80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23f6800640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c490 tx=0x7f23e800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8016020 con 0x7f23f806eb90 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23e8012940 con 0x7f23f806eb90 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8012d90 con 0x7f23f806eb90 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f80cdeb0 con 0x7f23f806eb90 2024-07-17T07:14:21.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.274+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f80ce470 con 0x7f23f806eb90 2024-07-17T07:14:21.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.275+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f80ac7b0 con 0x7f23f806eb90 2024-07-17T07:14:21.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.276+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f23e8019020 con 0x7f23f806eb90 2024-07-17T07:14:21.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.277+0000 7f23eea00640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 0x7f23d8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.278+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f23e8098280 con 0x7f23f806eb90 2024-07-17T07:14:21.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.278+0000 7f23f5e00640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 0x7f23d8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.279+0000 7f23f5e00640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 0x7f23d8079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:21.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.282+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f23e8020070 con 0x7f23f806eb90 2024-07-17T07:14:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.580+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f23f80b4200 con 0x7f23f806eb90 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.585+0000 7f23eea00640 1 -- 172.21.15.44:0/3480347936 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v37) ==== 190+0+147 (secure 0 0 0) 0x7f23e80612a0 con 0x7f23f806eb90 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:14:21.585 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 msgr2=0x7f23d8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 0x7f23d8079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4004730 comp rx=0 tx=0).stop 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 msgr2=0x7f23f80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c490 tx=0x7f23e800c950 comp rx=0 tx=0).stop 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 shutdown_connections 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f23d8076cd0 0x7f23d8079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23f815a5a0 0x7f23f81f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23f80ac3a0 0x7f23f80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 --2- 172.21.15.44:0/3480347936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f806eb90 0x7f23f80cd730 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 >> 172.21.15.44:0/3480347936 conn(0x7f23f80b0eb0 msgr2=0x7f23f815abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 shutdown_connections 2024-07-17T07:14:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.588+0000 7f23fd400640 1 -- 172.21.15.44:0/3480347936 wait complete. 2024-07-17T07:14:21.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.811+0000 7ff4a4800640 1 -- 172.21.15.44:0/1026124149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d6d0 msgr2=0x7ff49c06db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.811+0000 7ff4a4800640 1 --2- 172.21.15.44:0/1026124149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d6d0 0x7ff49c06db80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff498005fd0 tx=0x7ff49802a9f0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.811+0000 7ff4a4800640 1 -- 172.21.15.44:0/1026124149 shutdown_connections 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.811+0000 7ff4a4800640 1 --2- 172.21.15.44:0/1026124149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d6d0 0x7ff49c06db80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.811+0000 7ff4a4800640 1 --2- 172.21.15.44:0/1026124149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff49c06d2f0 0x7ff49c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.812+0000 7ff4a4800640 1 -- 172.21.15.44:0/1026124149 >> 172.21.15.44:0/1026124149 conn(0x7ff49c0b3020 msgr2=0x7ff49c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.812+0000 7ff4a4800640 1 -- 172.21.15.44:0/1026124149 shutdown_connections 2024-07-17T07:14:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.812+0000 7ff4a4800640 1 -- 172.21.15.44:0/1026124149 wait complete. 2024-07-17T07:14:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.813+0000 7ff4a4800640 1 Processor -- start 2024-07-17T07:14:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.814+0000 7ff4a4800640 1 -- start start 2024-07-17T07:14:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff49c06d6d0 0x7ff49c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49c0c6290 0x7ff49c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c0c7ba0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c0c7a20 con 0x7ff49c0c6290 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a4800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c0c7d20 con 0x7ff49c06d6d0 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35778/0 (socket says 172.21.15.44:35778) 2024-07-17T07:14:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 -- 172.21.15.44:0/4140374314 learned_addr learned my addr 172.21.15.44:0/4140374314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 -- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff49c06d6d0 msgr2=0x7ff49c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff49c06d6d0 0x7ff49c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 -- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49c0c6290 msgr2=0x7ff49c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49c0c6290 0x7ff49c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.815+0000 7ff4a1e00640 1 -- 172.21.15.44:0/4140374314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c14e3a0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff4a1e00640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c970 tx=0x7ff49000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490007bb0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff490005020 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490007e60 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c14e680 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.816+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c15cb60 con 0x7ff49c06d2f0 2024-07-17T07:14:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.817+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c06eba0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.819+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7ff490005470 con 0x7ff49c06d2f0 2024-07-17T07:14:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.819+0000 7ff496000640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 0x7ff4800791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.820+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7ff490098dc0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.820+0000 7ff4a1400640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 0x7ff4800791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:21.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.821+0000 7ff4a1400640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 0x7ff4800791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff49c151f60 tx=0x7ff48c0079b0 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:21.824+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff49c06eba0 con 0x7ff49c06d2f0 2024-07-17T07:14:21.926 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T07:14:21.926+0000 7f0181400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:14:22.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.079+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff49c1f7890 con 0x7ff49c06d2f0 2024-07-17T07:14:22.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.080+0000 7ff496000640 1 -- 172.21.15.44:0/4140374314 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7ff490014030 con 0x7ff49c06d2f0 2024-07-17T07:14:22.080 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:22.080 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:14:22.080 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:14:22.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 msgr2=0x7ff4800791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 0x7ff4800791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff49c151f60 tx=0x7ff48c0079b0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 msgr2=0x7ff49c1f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c970 tx=0x7ff49000ce30 comp rx=0 tx=0).stop 2024-07-17T07:14:22.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 shutdown_connections 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7ff480076d00 0x7ff4800791c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff49c0c6290 0x7ff49c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff49c06d6d0 0x7ff49c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 --2- 172.21.15.44:0/4140374314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff49c06d2f0 0x7ff49c1f82a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 >> 172.21.15.44:0/4140374314 conn(0x7ff49c0b3020 msgr2=0x7ff49c0ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 shutdown_connections 2024-07-17T07:14:22.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.083+0000 7ff4a4800640 1 -- 172.21.15.44:0/4140374314 wait complete. 2024-07-17T07:14:22.506 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:14:15.436136+0000', 'modified': '2024-07-17T07:14:18.521975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7500': {'gid': 7500, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2442230112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2442230112}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2442230112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7500, 'qdb_cluster': [7500]} 2024-07-17T07:14:22.507 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:14:22.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 -- 172.21.15.44:0/1243815621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de406f6d0 msgr2=0x7f1de415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 --2- 172.21.15.44:0/1243815621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de406f6d0 0x7f1de415a5a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8005fd0 tx=0x7f1dd802b750 comp rx=0 tx=0).stop 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 -- 172.21.15.44:0/1243815621 shutdown_connections 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 --2- 172.21.15.44:0/1243815621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de40acab0 0x7f1de406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 --2- 172.21.15.44:0/1243815621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de406f6d0 0x7f1de415a5a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.733+0000 7f1debc00640 1 -- 172.21.15.44:0/1243815621 >> 172.21.15.44:0/1243815621 conn(0x7f1de40b0eb0 msgr2=0x7f1de40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.734+0000 7f1debc00640 1 -- 172.21.15.44:0/1243815621 shutdown_connections 2024-07-17T07:14:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.734+0000 7f1debc00640 1 -- 172.21.15.44:0/1243815621 wait complete. 2024-07-17T07:14:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.735+0000 7f1debc00640 1 Processor -- start 2024-07-17T07:14:22.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.735+0000 7f1debc00640 1 -- start start 2024-07-17T07:14:22.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de406f6d0 0x7f1de4072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de40b3280 0x7f1de41e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1de40aea90 con 0x7f1de40acab0 2024-07-17T07:14:22.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1de40ae910 con 0x7f1de406f6d0 2024-07-17T07:14:22.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.736+0000 7f1debc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1de40aec10 con 0x7f1de40b3280 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35810/0 (socket says 172.21.15.44:35810) 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 -- 172.21.15.44:0/3174938245 learned_addr learned my addr 172.21.15.44:0/3174938245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 -- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de40b3280 msgr2=0x7f1de41e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de40b3280 0x7f1de41e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 -- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de406f6d0 msgr2=0x7f1de4072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de9200640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de406f6d0 0x7f1de4072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de406f6d0 0x7f1de4072630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.737+0000 7f1de3e00640 1 -- 172.21.15.44:0/3174938245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de4072cc0 con 0x7f1de40acab0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b3f0 tx=0x7f1dd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1de0c00640 1 -- 172.21.15.44:0/3174938245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0015020 con 0x7f1de40acab0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1de0c00640 1 -- 172.21.15.44:0/3174938245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd0011920 con 0x7f1de40acab0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1de0c00640 1 -- 172.21.15.44:0/3174938245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0011d70 con 0x7f1de40acab0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de4072fa0 con 0x7f1de40acab0 2024-07-17T07:14:22.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.738+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de4073560 con 0x7f1de40acab0 2024-07-17T07:14:22.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.740+0000 7f1de0c00640 1 -- 172.21.15.44:0/3174938245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f1dd0018020 con 0x7f1de40acab0 2024-07-17T07:14:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.741+0000 7f1de0c00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 0x7f1dc4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.742+0000 7f1de0c00640 1 -- 172.21.15.44:0/3174938245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f1dd0098670 con 0x7f1de40acab0 2024-07-17T07:14:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.742+0000 7f1deebc2740 1 Processor -- start 2024-07-17T07:14:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.742+0000 7f1deebc2740 1 -- start start 2024-07-17T07:14:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.742+0000 7f1de9200640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 0x7f1dc4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de6781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de674a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5617de5bfd60 con 0x5617de64f0a0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617de5af260 con 0x5617de6745b0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5617de5b32a0 con 0x5617de677de0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1de9200640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 0x7f1dc4079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd8004730 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.743+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35812/0 (socket says 172.21.15.44:35812) 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 -- 172.21.15.44:0/2387449221 learned_addr learned my addr 172.21.15.44:0/2387449221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de9200640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de6781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de9c00640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de674a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 -- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 msgr2=0x5617de6781c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de6781c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 -- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 msgr2=0x5617de674a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de674a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 -- 172.21.15.44:0/2387449221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617de313850 con 0x5617de64f0a0 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de9200640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de6781c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de9c00640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.744+0000 7f1de3e00640 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0013000 tx=0x7f1dd00616b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c88c426bbeb1462 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1dd5600640 1 -- 172.21.15.44:0/2387449221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd007a020 con 0x5617de64f0a0 2024-07-17T07:14:22.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1dd5600640 1 -- 172.21.15.44:0/2387449221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd0069f20 con 0x5617de64f0a0 2024-07-17T07:14:22.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1dd5600640 1 -- 172.21.15.44:0/2387449221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0069cb0 con 0x5617de64f0a0 2024-07-17T07:14:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 -- 172.21.15.44:0/2387449221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 msgr2=0x5617de674070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0013000 tx=0x7f1dd00616b0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 -- 172.21.15.44:0/2387449221 shutdown_connections 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de674a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de64f0a0 0x5617de674070 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2387449221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de6781c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.745+0000 7f1deebc2740 1 -- 172.21.15.44:0/2387449221 >> 172.21.15.44:0/2387449221 conn(0x5617de64a280 msgr2=0x5617de64dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.746+0000 7f1deebc2740 1 -- 172.21.15.44:0/2387449221 shutdown_connections 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.746+0000 7f1deebc2740 1 -- 172.21.15.44:0/2387449221 wait complete. 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.746+0000 7f1deebc2740 1 -- start start 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.746+0000 7f1deebc2740 1 Processor -- start 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.747+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de6894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.747+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de66def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.747+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1deebc2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5617de322620 con 0x5617de6726a0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617de556f90 con 0x5617de6745b0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5617de648980 con 0x5617de677de0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1deebc2740 10 client.0 _ll_register_callbacks cb 0x5617de57cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35828/0 (socket says 172.21.15.44:35828) 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 -- 172.21.15.44:0/2795879556 learned_addr learned my addr 172.21.15.44:0/2795879556 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9200640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de6894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de3e00640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de66def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 -- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 msgr2=0x5617de66def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de66def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 -- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 msgr2=0x5617de6894e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de6894e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de9c00640 1 -- 172.21.15.44:0/2795879556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617de38a6b0 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.748+0000 7f1de3e00640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de66def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1de9c00640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00f3c0 tx=0x7f1ddc00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd184fea99ef2a16 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1dca000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc003b70 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f1ddc003d10 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617de586900 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1ddc011710 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1deebc2740 10 client.7617 fetch_fsmap learned FSMap version 61 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.749+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x5617de673060 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5617de586900 con 0x5617de6726a0 2024-07-17T07:14:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f1ddc0118b0 con 0x5617de6726a0 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 10 client.7617 fetch_fsmap finished waiting for FSMap version 61 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 10 client.7617 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 20 client.7617 populate_metadata read hostname 'smithi044' 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de65bbd0 0x5617de65bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 4 client.7617 mds_command: new command op to 7500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 --> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617de5c1cb0 con 0x5617de65bbd0 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.750+0000 7f1de9200640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de65bbd0 0x5617de65bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.751+0000 7f1de9200640 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de65bbd0 0x5617de65bfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1c71cae4742324 server_cookie=e76e0289311c9620 in_seq=0 out_seq=0 2024-07-17T07:14:22.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.751+0000 7f1dca000640 10 client.7617 ms_handle_connect on v2:172.21.15.44:6833/2442230112 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1dca000640 1 -- 172.21.15.44:0/2795879556 <== mds.0 v2:172.21.15.44:6833/2442230112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617de5c1cb0 con 0x5617de65bbd0 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1dca000640 10 client.7617 handle_command_reply: tid=0 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 client.7617 shutdown 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 msgr2=0x5617de672b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00f3c0 tx=0x7f1ddc00f880 comp rx=0 tx=0).stop 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 shutdown_connections 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de672b20 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de66def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de65bbd0 0x5617de65bfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 --2- 172.21.15.44:0/2795879556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de6894e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1dca000640 0 client.7617 ms_handle_reset on v2:172.21.15.44:6833/2442230112 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 >> 172.21.15.44:0/2795879556 conn(0x5617de64f0a0 msgr2=0x5617de66c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 shutdown_connections 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.752+0000 7f1deebc2740 1 -- 172.21.15.44:0/2795879556 wait complete. 2024-07-17T07:14:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.753+0000 7f1deebc2740 20 client.7617 trim_cache size 0 max 16384 2024-07-17T07:14:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.773+0000 7f1deebc2740 1 Processor -- start 2024-07-17T07:14:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.774+0000 7f1deebc2740 1 -- start start 2024-07-17T07:14:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.774+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6726a0 0x5617de671b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.774+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de6745b0 0x5617de672070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.774+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1deebc2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4201600 con 0x5617de677de0 2024-07-17T07:14:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1de42005e0 con 0x5617de6726a0 2024-07-17T07:14:22.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1de41ff960 con 0x5617de6745b0 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1de9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1de9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35830/0 (socket says 172.21.15.44:35830) 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1de9c00640 1 -- 172.21.15.44:0/3571279216 learned_addr learned my addr 172.21.15.44:0/3571279216 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1de9200640 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6726a0 0x5617de671b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.775+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de6745b0 0x5617de672070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 -- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de6745b0 msgr2=0x5617de672070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de6745b0 0x5617de672070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 -- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6726a0 msgr2=0x5617de671b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6726a0 0x5617de671b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 -- 172.21.15.44:0/3571279216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617de5af3e0 con 0x5617de677de0 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1de9c00640 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc002230 tx=0x7f1ddc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c116a8a868f1092 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1dd4c00640 1 -- 172.21.15.44:0/3571279216 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc030070 con 0x5617de677de0 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1dd4c00640 1 -- 172.21.15.44:0/3571279216 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ddc0118b0 con 0x5617de677de0 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.776+0000 7f1dd4c00640 1 -- 172.21.15.44:0/3571279216 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc003970 con 0x5617de677de0 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 -- 172.21.15.44:0/3571279216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 msgr2=0x5617de6964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc002230 tx=0x7f1ddc009bf0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 -- 172.21.15.44:0/3571279216 shutdown_connections 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de677de0 0x5617de6964f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de6745b0 0x5617de672070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 --2- 172.21.15.44:0/3571279216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6726a0 0x5617de671b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 -- 172.21.15.44:0/3571279216 >> 172.21.15.44:0/3571279216 conn(0x5617de64f0a0 msgr2=0x5617de6781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 -- 172.21.15.44:0/3571279216 shutdown_connections 2024-07-17T07:14:22.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.777+0000 7f1deebc2740 1 -- 172.21.15.44:0/3571279216 wait complete. 2024-07-17T07:14:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.778+0000 7f1deebc2740 1 -- start start 2024-07-17T07:14:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.778+0000 7f1deebc2740 1 Processor -- start 2024-07-17T07:14:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de70d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de671b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5617de65ae10 con 0x5617de6726a0 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41fff30 con 0x5617de6745b0 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5617de66bf60 con 0x5617de677de0 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1deebc2740 10 client.0 _ll_register_callbacks cb 0x7f1ddc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.779+0000 7f1de9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35844/0 (socket says 172.21.15.44:35844) 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 -- 172.21.15.44:0/687250242 learned_addr learned my addr 172.21.15.44:0/687250242 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 -- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 msgr2=0x5617de671b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de671b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 -- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 msgr2=0x5617de70d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:14:22.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de70d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 -- 172.21.15.44:0/687250242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617de66d360 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1de9200640 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aa2eda072dcb02 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1dca000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8044040 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f1dd8055630 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.780+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ddc0118b0 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1dd805d050 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 10 client.7623 fetch_fsmap learned FSMap version 61 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x5617de66d970 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ddc0118b0 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f1dd8055830 con 0x5617de6726a0 2024-07-17T07:14:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 10 client.7623 fetch_fsmap finished waiting for FSMap version 61 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 10 client.7623 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 20 client.7623 populate_metadata read hostname 'smithi044' 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de714410 0x5617de6b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 4 client.7623 mds_command: new command op to 7500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.781+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 --> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] -- command(tid 0: {"prefix": "status"}) -- 0x5617de5bd750 con 0x5617de714410 2024-07-17T07:14:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1de3e00640 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de714410 0x5617de6b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1de3e00640 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de714410 0x5617de6b5ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7463880d9365fc08 server_cookie=bc43ff1b37359ec1 in_seq=0 out_seq=0 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1dca000640 10 client.7623 ms_handle_connect on v2:172.21.15.44:6833/2442230112 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1dca000640 1 -- 172.21.15.44:0/687250242 <== mds.0 v2:172.21.15.44:6833/2442230112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5617de5bd750 con 0x5617de714410 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1dca000640 10 client.7623 handle_command_reply: tid=0 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.782+0000 7f1deebc2740 1 client.7623 shutdown 2024-07-17T07:14:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 msgr2=0x5617de7101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd8009260 comp rx=0 tx=0).stop 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 shutdown_connections 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5617de677de0 0x5617de671b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:6833/2442230112,v1:172.21.15.44:6835/2442230112] conn(0x5617de714410 0x5617de6b5ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617de6745b0 0x5617de70d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 --2- 172.21.15.44:0/687250242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5617de6726a0 0x5617de7101d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1dca000640 0 client.7623 ms_handle_reset on v2:172.21.15.44:6833/2442230112 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 >> 172.21.15.44:0/687250242 conn(0x5617de64f0a0 msgr2=0x5617de643560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 shutdown_connections 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 1 -- 172.21.15.44:0/687250242 wait complete. 2024-07-17T07:14:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.783+0000 7f1deebc2740 20 client.7623 trim_cache size 0 max 16384 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 7500, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7974236799999996, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 61, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 128, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 128, 2024-07-17T07:14:22.787 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.740026666 2024-07-17T07:14:22.788 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:14:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 msgr2=0x7f1dc4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 0x7f1dc4079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8034000 tx=0x7f1dd8004730 comp rx=0 tx=0).stop 2024-07-17T07:14:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 msgr2=0x7f1de4098a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b3f0 tx=0x7f1dd000b8b0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 shutdown_connections 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f1dc4076cd0 0x7f1dc4079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de40b3280 0x7f1de41e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de40acab0 0x7f1de4098a70 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 --2- 172.21.15.44:0/3174938245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de406f6d0 0x7f1de4072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 >> 172.21.15.44:0/3174938245 conn(0x7f1de40b0eb0 msgr2=0x7f1de4153820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 shutdown_connections 2024-07-17T07:14:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.784+0000 7f1debc00640 1 -- 172.21.15.44:0/3174938245 wait complete. 2024-07-17T07:14:22.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:14:22.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 -- 172.21.15.44:0/2824596047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 msgr2=0x7f6b7c153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 --2- 172.21.15.44:0/2824596047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c153e40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6b74005fd0 tx=0x7f6b7402a8b0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 -- 172.21.15.44:0/2824596047 shutdown_connections 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 --2- 172.21.15.44:0/2824596047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c153e40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 --2- 172.21.15.44:0/2824596047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 0x7f6b7c0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 -- 172.21.15.44:0/2824596047 >> 172.21.15.44:0/2824596047 conn(0x7f6b7c0b0fb0 msgr2=0x7f6b7c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:22.999+0000 7f6b82600640 1 -- 172.21.15.44:0/2824596047 shutdown_connections 2024-07-17T07:14:23.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.000+0000 7f6b82600640 1 -- 172.21.15.44:0/2824596047 wait complete. 2024-07-17T07:14:23.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.001+0000 7f6b82600640 1 Processor -- start 2024-07-17T07:14:23.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.002+0000 7f6b82600640 1 -- start start 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b7c0acb90 0x7f6b7c157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 0x7f6b7c1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c154760 con 0x7f6b7c06f730 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1545e0 con 0x7f6b7c0acb90 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b82600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c1548e0 con 0x7f6b7c15b2e0 2024-07-17T07:14:23.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35864/0 (socket says 172.21.15.44:35864) 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 -- 172.21.15.44:0/1879080330 learned_addr learned my addr 172.21.15.44:0/1879080330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 -- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 msgr2=0x7f6b7c1f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7be00640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 0x7f6b7c1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b72a00640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b7c0acb90 0x7f6b7c157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 0x7f6b7c1f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 -- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b7c0acb90 msgr2=0x7f6b7c157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b7c0acb90 0x7f6b7c157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.003+0000 7f6b7b400640 1 -- 172.21.15.44:0/1879080330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c156340 con 0x7f6b7c06f730 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b7b400640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d740 tx=0x7f6b6c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c017070 con 0x7f6b7c06f730 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b6c009d20 con 0x7f6b7c06f730 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0106c0 con 0x7f6b7c06f730 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c156620 con 0x7f6b7c06f730 2024-07-17T07:14:23.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.004+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c156be0 con 0x7f6b7c06f730 2024-07-17T07:14:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.006+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c15daa0 con 0x7f6b7c06f730 2024-07-17T07:14:23.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.007+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f6b6c01a070 con 0x7f6b7c06f730 2024-07-17T07:14:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.008+0000 7f6b73e00640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 0x7f6b580791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.008+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f6b6c0989b0 con 0x7f6b7c06f730 2024-07-17T07:14:23.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.009+0000 7f6b72a00640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 0x7f6b580791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:23.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.010+0000 7f6b72a00640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 0x7f6b580791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b64001220 tx=0x7f6b64009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:23.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.012+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6b7c15daa0 con 0x7f6b7c06f730 2024-07-17T07:14:23.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.267+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6b7c153e40 con 0x7f6b7c06f730 2024-07-17T07:14:23.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.268+0000 7f6b73e00640 1 -- 172.21.15.44:0/1879080330 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f6b6c0618c0 con 0x7f6b7c06f730 2024-07-17T07:14:23.271 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:14:23.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 msgr2=0x7f6b580791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:23.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 0x7f6b580791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b64001220 tx=0x7f6b64009040 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 msgr2=0x7f6b7c073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d740 tx=0x7f6b6c00dc00 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 shutdown_connections 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f6b58076d00 0x7f6b580791c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b7c15b2e0 0x7f6b7c1f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b7c0acb90 0x7f6b7c157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 --2- 172.21.15.44:0/1879080330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b7c06f730 0x7f6b7c073e10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.271+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 >> 172.21.15.44:0/1879080330 conn(0x7f6b7c0b0fb0 msgr2=0x7f6b7c159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.272+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 shutdown_connections 2024-07-17T07:14:23.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23.272+0000 7f6b82600640 1 -- 172.21.15.44:0/1879080330 wait complete. 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:14:23.290 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:14:23.291 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:14:23.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.291 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:14:23.340 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:14:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41394sec preferred_lft 41394sec 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:14:23.342 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:14:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:14:23.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.345 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:14:23.416 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:14:23.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.420 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:14:23.470 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:14:23.470 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:14:23.470 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:14:23.474 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:14:23.474 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:14:23.474 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:14:23.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:14:23.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:14:23.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.679 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:14:23.724 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:14:23.724 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:14:23.725 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:14:23.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:14:23.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:14:23.860 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:14:23.913 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:14:23.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:14:23.913 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:14:23.964 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:14:23.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:23.965 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:14:24.094 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:14:24.094 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:14:24.094 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:14:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:14:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:14:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:14:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:14:24.116 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:14:24.145 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:14:24.148 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:14:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:14:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:14:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:14:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:14:24.152 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:14:24.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:24.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:14:24.231 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:14:24.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:14:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:14:24.363 INFO:teuthology.orchestra.run.smithi044.stdout:client6385 2024-07-17T07:14:24.363 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:14:24.363 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:14:24.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:14:24.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 -- 172.21.15.44:0/3816086027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c152300 msgr2=0x7fe63c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:24.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 --2- 172.21.15.44:0/3816086027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c152300 0x7fe63c06f1b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe63c070a70 tx=0x7fe63402a980 comp rx=0 tx=0).stop 2024-07-17T07:14:24.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 -- 172.21.15.44:0/3816086027 shutdown_connections 2024-07-17T07:14:24.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 --2- 172.21.15.44:0/3816086027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c152300 0x7fe63c06f1b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 --2- 172.21.15.44:0/3816086027 >> v2:172.21.15.175:3300/0 conn(0x7fe63c0cd0b0 0x7fe63c06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 --2- 172.21.15.44:0/3816086027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63c151f20 0x7fe63c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 -- 172.21.15.44:0/3816086027 >> 172.21.15.44:0/3816086027 conn(0x7fe63c0cf0d0 msgr2=0x7fe63c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 -- 172.21.15.44:0/3816086027 shutdown_connections 2024-07-17T07:14:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.674+0000 7fe63b200640 1 -- 172.21.15.44:0/3816086027 wait complete. 2024-07-17T07:14:24.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.676+0000 7fe63b200640 1 Processor -- start 2024-07-17T07:14:24.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.676+0000 7fe63b200640 1 -- start start 2024-07-17T07:14:24.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe63c0cd0b0 0x7fe63c1f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63c152300 0x7fe63c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0cb450 con 0x7fe63c151f20 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c0cb2d0 con 0x7fe63c0cd0b0 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe63b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe63c0cb5d0 con 0x7fe63c152300 2024-07-17T07:14:24.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:24.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35878/0 (socket says 172.21.15.44:35878) 2024-07-17T07:14:24.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 -- 172.21.15.44:0/1907041515 learned_addr learned my addr 172.21.15.44:0/1907041515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:24.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639e00640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe63c0cd0b0 0x7fe63c1f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:24.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 -- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63c152300 msgr2=0x7fe63c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:24.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63c152300 0x7fe63c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 -- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe63c0cd0b0 msgr2=0x7fe63c1f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:24.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe63c0cd0b0 0x7fe63c1f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.677+0000 7fe639400640 1 -- 172.21.15.44:0/1907041515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c14bc70 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe639400640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d770 tx=0x7fe62800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628017070 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c14bf50 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe628009d20 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6280106c0 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.678+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c153bb0 con 0x7fe63c151f20 2024-07-17T07:14:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.679+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c0cba30 con 0x7fe63c151f20 2024-07-17T07:14:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.681+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7fe62801a070 con 0x7fe63c151f20 2024-07-17T07:14:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.682+0000 7fe632000640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 0x7fe61c07d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.682+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7fe628098b20 con 0x7fe63c151f20 2024-07-17T07:14:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.682+0000 7fe639e00640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 0x7fe61c07d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.684+0000 7fe639e00640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 0x7fe61c07d5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00ba40 tx=0x7fe62c005cc0 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:24.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.686+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe628061920 con 0x7fe63c151f20 2024-07-17T07:14:24.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.924+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 --> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe63c074ee0 con 0x7fe61c07b0e0 2024-07-17T07:14:24.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.928+0000 7fe632000640 1 -- 172.21.15.44:0/1907041515 <== mgr.6202 v2:172.21.15.175:6800/1224757683 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe63c074ee0 con 0x7fe61c07b0e0 2024-07-17T07:14:24.928 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:14:24.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:14:24.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:14:24.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:14:24.928 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:14:24.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 msgr2=0x7fe61c07d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:24.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 0x7fe61c07d5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00ba40 tx=0x7fe62c005cc0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 msgr2=0x7fe63c1f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d770 tx=0x7fe62800dc30 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 shutdown_connections 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.930+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fe61c07b0e0 0x7fe61c07d5a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63c152300 0x7fe63c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe63c151f20 0x7fe63c1f0f40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 --2- 172.21.15.44:0/1907041515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe63c0cd0b0 0x7fe63c1f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 >> 172.21.15.44:0/1907041515 conn(0x7fe63c0cf0d0 msgr2=0x7fe63c0cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 shutdown_connections 2024-07-17T07:14:24.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:24.931+0000 7fe63b200640 1 -- 172.21.15.44:0/1907041515 wait complete. 2024-07-17T07:14:24.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 -- 172.21.15.44:0/3999338654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ad790 msgr2=0x7f308c0adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 --2- 172.21.15.44:0/3999338654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ad790 0x7f308c0adcb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3074009830 tx=0x7f307402a480 comp rx=0 tx=0).stop 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 -- 172.21.15.44:0/3999338654 shutdown_connections 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 --2- 172.21.15.44:0/3999338654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ae1f0 0x7f308c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 --2- 172.21.15.44:0/3999338654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ad790 0x7f308c0adcb0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 -- 172.21.15.44:0/3999338654 >> 172.21.15.44:0/3999338654 conn(0x7f308c06c120 msgr2=0x7f308c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 -- 172.21.15.44:0/3999338654 shutdown_connections 2024-07-17T07:14:25.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.174+0000 7f3090a00640 1 -- 172.21.15.44:0/3999338654 wait complete. 2024-07-17T07:14:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.176+0000 7f3090a00640 1 Processor -- start 2024-07-17T07:14:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.176+0000 7f3090a00640 1 -- start start 2024-07-17T07:14:25.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.177+0000 7f3090a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ad790 0x7f308c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:25.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3090a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:25.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3090a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f308c15afd0 0x7f308c0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3090a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0ceb40 con 0x7f308c0ae1f0 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3090a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0ce9c0 con 0x7f308c15afd0 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3090a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f308c0cecc0 con 0x7f308c0ad790 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35894/0 (socket says 172.21.15.44:35894) 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 -- 172.21.15.44:0/3528331743 learned_addr learned my addr 172.21.15.44:0/3528331743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f308a800640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ad790 0x7f308c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f308b200640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f308c15afd0 0x7f308c0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 -- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ad790 msgr2=0x7f308c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ad790 0x7f308c14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 -- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f308c15afd0 msgr2=0x7f308c0b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f308c15afd0 0x7f308c0b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.178+0000 7f3089e00640 1 -- 172.21.15.44:0/3528331743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c1f7b00 con 0x7f308c0ae1f0 2024-07-17T07:14:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3089e00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3074009950 tx=0x7f3074032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074040070 con 0x7f308c0ae1f0 2024-07-17T07:14:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3074032e70 con 0x7f308c0ae1f0 2024-07-17T07:14:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30740055b0 con 0x7f308c0ae1f0 2024-07-17T07:14:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c150d90 con 0x7f308c0ae1f0 2024-07-17T07:14:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.179+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1f80a0 con 0x7f308c0ae1f0 2024-07-17T07:14:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.181+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c0cf1e0 con 0x7f308c0ae1f0 2024-07-17T07:14:25.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.182+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 58) ==== 98931+0+0 (secure 0 0 0) 0x7f3074039040 con 0x7f308c0ae1f0 2024-07-17T07:14:25.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.183+0000 7f3082a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 0x7f3068079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.183+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f30740b9aa0 con 0x7f308c0ae1f0 2024-07-17T07:14:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.183+0000 7f308a800640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 0x7f3068079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:25.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.185+0000 7f308a800640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 0x7f3068079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307c002050 tx=0x7f307c009040 comp rx=0 tx=0).ready entity=mgr.6202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.187+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3074082920 con 0x7f308c0ae1f0 2024-07-17T07:14:25.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:25.456+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f308c0cb440 con 0x7f308c0ae1f0 2024-07-17T07:14:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.065+0000 7f3082a00640 1 -- 172.21.15.44:0/3528331743 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v59) ==== 92+0+0 (secure 0 0 0) 0x7f3074082220 con 0x7f308c0ae1f0 2024-07-17T07:14:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 msgr2=0x7f3068079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 0x7f3068079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307c002050 tx=0x7f307c009040 comp rx=0 tx=0).stop 2024-07-17T07:14:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 msgr2=0x7f308c14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3074009950 tx=0x7f3074032c80 comp rx=0 tx=0).stop 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 shutdown_connections 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f3068076cd0 0x7f3068079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f308c15afd0 0x7f308c0b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0ae1f0 0x7f308c14fea0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 --2- 172.21.15.44:0/3528331743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f308c0ad790 0x7f308c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 >> 172.21.15.44:0/3528331743 conn(0x7f308c06c120 msgr2=0x7f308c15c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 shutdown_connections 2024-07-17T07:14:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.069+0000 7f3090a00640 1 -- 172.21.15.44:0/3528331743 wait complete. 2024-07-17T07:14:26.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 -- 172.21.15.44:0/1530983570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380b29e0 msgr2=0x7fad38161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 --2- 172.21.15.44:0/1530983570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380b29e0 0x7fad38161b90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fad380694e0 tx=0x7fad3002aac0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 -- 172.21.15.44:0/1530983570 shutdown_connections 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 --2- 172.21.15.44:0/1530983570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380b29e0 0x7fad38161b90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 --2- 172.21.15.44:0/1530983570 >> v2:172.21.15.175:3300/0 conn(0x7fad380b6790 0x7fad38161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 --2- 172.21.15.44:0/1530983570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380ce1d0 0x7fad380ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 -- 172.21.15.44:0/1530983570 >> 172.21.15.44:0/1530983570 conn(0x7fad380d3700 msgr2=0x7fad380d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:26.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 -- 172.21.15.44:0/1530983570 shutdown_connections 2024-07-17T07:14:26.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.313+0000 7fad3cc00640 1 -- 172.21.15.44:0/1530983570 wait complete. 2024-07-17T07:14:26.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.315+0000 7fad3cc00640 1 Processor -- start 2024-07-17T07:14:26.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.315+0000 7fad3cc00640 1 -- start start 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 0x7fad380cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad380b6790 0x7fad381555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad38162250 con 0x7fad380ce1d0 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad381620d0 con 0x7fad380b6790 2024-07-17T07:14:26.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad3cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad381623d0 con 0x7fad380b29e0 2024-07-17T07:14:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad36800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad36800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35944/0 (socket says 172.21.15.44:35944) 2024-07-17T07:14:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad36800640 1 -- 172.21.15.44:0/373536662 learned_addr learned my addr 172.21.15.44:0/373536662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad35e00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 0x7fad380cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.316+0000 7fad2fe00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad380b6790 0x7fad381555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 msgr2=0x7fad380cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 0x7fad380cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad380b6790 msgr2=0x7fad381555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad380b6790 0x7fad381555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 -- 172.21.15.44:0/373536662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381f4950 con 0x7fad380ce1d0 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad35e00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 0x7fad380cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad36800640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fad30009d60 tx=0x7fad30003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30044070 con 0x7fad380ce1d0 2024-07-17T07:14:26.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.317+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad30004250 con 0x7fad380ce1d0 2024-07-17T07:14:26.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.318+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad300046a0 con 0x7fad380ce1d0 2024-07-17T07:14:26.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.318+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381f4c30 con 0x7fad380ce1d0 2024-07-17T07:14:26.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.318+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad38155ae0 con 0x7fad380ce1d0 2024-07-17T07:14:26.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.319+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad380ce5b0 con 0x7fad380ce1d0 2024-07-17T07:14:26.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.320+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 59) ==== 98948+0+0 (secure 0 0 0) 0x7fad3003f410 con 0x7fad380ce1d0 2024-07-17T07:14:26.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.321+0000 7fad2e000640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 0x7fad140791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.321+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7fad300b8fa0 con 0x7fad380ce1d0 2024-07-17T07:14:26.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.322+0000 7fad35e00640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 msgr2=0x7fad140791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:26.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.322+0000 7fad35e00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 0x7fad140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:26.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.325+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fad30032030 con 0x7fad380ce1d0 2024-07-17T07:14:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.523+0000 7fad35e00640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 msgr2=0x7fad140791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.523+0000 7fad35e00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 0x7fad140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:26.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.536+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fad380b8400 con 0x7fad380ce1d0 2024-07-17T07:14:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.542+0000 7fad2e000640 1 -- 172.21.15.44:0/373536662 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v49)=0 v49) ==== 138+0+0 (secure 0 0 0) 0x7fad300817d0 con 0x7fad380ce1d0 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 msgr2=0x7fad140791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 0x7fad140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 msgr2=0x7fad381f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fad30009d60 tx=0x7fad30003ce0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 shutdown_connections 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7fad14076d20 0x7fad140791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad380ce1d0 0x7fad381f40d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad380b6790 0x7fad381555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 --2- 172.21.15.44:0/373536662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad380b29e0 0x7fad380cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 >> 172.21.15.44:0/373536662 conn(0x7fad380d3700 msgr2=0x7fad380d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 shutdown_connections 2024-07-17T07:14:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.545+0000 7fad3cc00640 1 -- 172.21.15.44:0/373536662 wait complete. 2024-07-17T07:14:26.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:14:26.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 -- 172.21.15.44:0/2323111342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f0004800 msgr2=0x7f26f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 --2- 172.21.15.44:0/2323111342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f0004800 0x7f26f0004c80 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009b50 tx=0x7f26f0005830 comp rx=0 tx=0).stop 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 -- 172.21.15.44:0/2323111342 shutdown_connections 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 --2- 172.21.15.44:0/2323111342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f0004800 0x7f26f0004c80 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 -- 172.21.15.44:0/2323111342 >> 172.21.15.44:0/2323111342 conn(0x7f27080d3700 msgr2=0x7f27080d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 -- 172.21.15.44:0/2323111342 shutdown_connections 2024-07-17T07:14:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.761+0000 7f270fa00640 1 -- 172.21.15.44:0/2323111342 wait complete. 2024-07-17T07:14:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.764+0000 7f270fa00640 1 Processor -- start 2024-07-17T07:14:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.764+0000 7f270fa00640 1 -- start start 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f0004800 0x7f270815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27081f83c0 0x7f27081f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27080cea00 con 0x7f27081f41e0 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27080ce880 con 0x7f27081f83c0 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27080ceb80 con 0x7f26f0004800 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35972/0 (socket says 172.21.15.44:35972) 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270da00640 1 -- 172.21.15.44:0/1621752775 learned_addr learned my addr 172.21.15.44:0/1621752775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f270d000640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f0004800 0x7f270815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.766+0000 7f2707e00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27081f83c0 0x7f27081f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f0004800 msgr2=0x7f270815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f0004800 0x7f270815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27081f83c0 msgr2=0x7f27081f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27081f83c0 0x7f27081f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 -- 172.21.15.44:0/1621752775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27080cbbb0 con 0x7f27081f41e0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270da00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2700002230 tx=0x7f270000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000cba0 con 0x7f27081f41e0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2700018070 con 0x7f27081f41e0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000ee60 con 0x7f27081f41e0 2024-07-17T07:14:26.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27081f4b60 con 0x7f27081f41e0 2024-07-17T07:14:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.767+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27081f5120 con 0x7f27081f41e0 2024-07-17T07:14:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.769+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2708161940 con 0x7f27081f41e0 2024-07-17T07:14:26.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.770+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 59) ==== 98948+0+0 (secure 0 0 0) 0x7f270001d460 con 0x7f27081f41e0 2024-07-17T07:14:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.770+0000 7f2704c00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 0x7f26e80791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.771+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f2700098de0 con 0x7f27081f41e0 2024-07-17T07:14:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.771+0000 7f270d000640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 msgr2=0x7f26e80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.771+0000 7f270d000640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 0x7f26e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.775+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2700061c60 con 0x7f27081f41e0 2024-07-17T07:14:26.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.972+0000 7f270d000640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 msgr2=0x7f26e80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:26.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.973+0000 7f270d000640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 0x7f26e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:26.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.984+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f27080d2c90 con 0x7f27081f41e0 2024-07-17T07:14:26.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.992+0000 7f2704c00640 1 -- 172.21.15.44:0/1621752775 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v50)=0 v50) ==== 133+0+0 (secure 0 0 0) 0x7f2700061560 con 0x7f27081f41e0 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.994+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 msgr2=0x7f26e80791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.994+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 0x7f26e80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.994+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 msgr2=0x7f27081534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.994+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2700002230 tx=0x7f270000ed30 comp rx=0 tx=0).stop 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 shutdown_connections 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f26e8076d20 0x7f26e80791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27081f41e0 0x7f27081534e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27081f83c0 0x7f27081f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 --2- 172.21.15.44:0/1621752775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26f0004800 0x7f270815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 >> 172.21.15.44:0/1621752775 conn(0x7f27080d3700 msgr2=0x7f27080b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 shutdown_connections 2024-07-17T07:14:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:26.995+0000 7f270fa00640 1 -- 172.21.15.44:0/1621752775 wait complete. 2024-07-17T07:14:27.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:27.014 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T07:14:27.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-07-17T07:14:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 -- 172.21.15.44:0/2634526045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db8004800 msgr2=0x7f7db8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 --2- 172.21.15.44:0/2634526045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db8004800 0x7f7db8004c80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000bb20 tx=0x7f7db8005830 comp rx=0 tx=0).stop 2024-07-17T07:14:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd6800640 1 -- 172.21.15.44:0/2634526045 reap_dead start 2024-07-17T07:14:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 -- 172.21.15.44:0/2634526045 shutdown_connections 2024-07-17T07:14:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 -- 172.21.15.44:0/2634526045 >> 172.21.15.44:0/2634526045 conn(0x7f7dd00b4ff0 msgr2=0x7f7dd00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 -- 172.21.15.44:0/2634526045 shutdown_connections 2024-07-17T07:14:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.254+0000 7f7dd7200640 1 -- 172.21.15.44:0/2634526045 wait complete. 2024-07-17T07:14:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.256+0000 7f7dd7200640 1 Processor -- start 2024-07-17T07:14:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.256+0000 7f7dd7200640 1 -- start start 2024-07-17T07:14:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8004800 0x7f7dd00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 0x7f7dd015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dd01f3b70 0x7f7dd00ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d15c0 con 0x7f7dd00cd600 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d0760 con 0x7f7db8004800 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd00d1740 con 0x7f7dd01f3b70 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 0x7f7dd015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 0x7f7dd015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35988/0 (socket says 172.21.15.44:35988) 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd5400640 1 -- 172.21.15.44:0/3858202711 learned_addr learned my addr 172.21.15.44:0/3858202711 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.257+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8004800 0x7f7dd00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd6800640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dd01f3b70 0x7f7dd00ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dd01f3b70 msgr2=0x7f7dd00ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dd01f3b70 0x7f7dd00ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8004800 msgr2=0x7f7dd00cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8004800 0x7f7dd00cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd01f43c0 con 0x7f7dd00cd600 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd6800640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dd01f3b70 0x7f7dd00ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7db8004800 0x7f7dd00cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd5400640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 0x7f7dd015af40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000bc10 tx=0x7f7dc00058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0040070 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dc003a920 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0035040 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.258+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd01f46a0 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.259+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd00aa180 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.260+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd00a9f70 con 0x7f7dd00cd600 2024-07-17T07:14:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.261+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 59) ==== 98948+0+0 (secure 0 0 0) 0x7f7dc0038020 con 0x7f7dd00cd600 2024-07-17T07:14:27.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.262+0000 7f7dca000640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:27.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.262+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6621+0+0 (secure 0 0 0) 0x7f7dc00b9520 con 0x7f7dd00cd600 2024-07-17T07:14:27.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.262+0000 7f7dd5e00640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:27.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.262+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:14:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.270+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7dc00824b0 con 0x7f7dd00cd600 2024-07-17T07:14:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.463+0000 7f7dd5e00640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.464+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:14:27.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.497+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7dd00cfdf0 con 0x7f7db4076d20 2024-07-17T07:14:27.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.866+0000 7f7dd5e00640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:27.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:27.866+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:14:28.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:28.667+0000 7f7dd5e00640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:28.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:28.667+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:14:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:30.269+0000 7f7dd5e00640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1224757683 2024-07-17T07:14:30.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:30.269+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:14:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:30.325+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 60) ==== 98557+0+0 (secure 0 0 0) 0x7f7dc00314f0 con 0x7f7dd00cd600 2024-07-17T07:14:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:30.326+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 msgr2=0x7f7db40791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:14:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:30.326+0000 7f7dca000640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1224757683,v1:172.21.15.175:6801/1224757683] conn(0x7f7db4076d20 0x7f7db40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:30.612 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff086a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff089c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff086a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff089c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff089c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff086a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff086a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff086a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.613 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff089c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:30.614 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:14:30.613+0000 7ff089c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:14:31.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.338+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 61) ==== 98684+0+0 (secure 0 0 0) 0x7f7dc007e040 con 0x7f7dd00cd600 2024-07-17T07:14:31.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.340+0000 7f7dca000640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f7db407f820 0x7f7db4081c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:31.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.340+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 --> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7db400c880 con 0x7f7db407f820 2024-07-17T07:14:31.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.344+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f7db407f820 0x7f7db4081c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:31.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.348+0000 7f7dd5e00640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f7db407f820 0x7f7db4081c10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00d2920 tx=0x7f7dc4001d70 comp rx=0 tx=0).ready entity=mgr.6397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:31.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.728+0000 7f7dca000640 1 -- 172.21.15.44:0/3858202711 <== mgr.6397 v2:172.21.15.175:6800/1035934967 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7db400c880 con 0x7f7db407f820 2024-07-17T07:14:31.728 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T07:14:31.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f7db407f820 msgr2=0x7f7db4081c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:31.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f7db407f820 0x7f7db4081c10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00d2920 tx=0x7f7dc4001d70 comp rx=0 tx=0).stop 2024-07-17T07:14:31.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 msgr2=0x7f7dd015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:31.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 --2- 172.21.15.44:0/3858202711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd00cd600 0x7f7dd015af40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000bc10 tx=0x7f7dc00058f0 comp rx=0 tx=0).stop 2024-07-17T07:14:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd6800640 1 -- 172.21.15.44:0/3858202711 reap_dead start 2024-07-17T07:14:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 shutdown_connections 2024-07-17T07:14:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 >> 172.21.15.44:0/3858202711 conn(0x7f7dd00b4ff0 msgr2=0x7f7dd00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.732+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 shutdown_connections 2024-07-17T07:14:31.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.733+0000 7f7dd7200640 1 -- 172.21.15.44:0/3858202711 wait complete. 2024-07-17T07:14:31.752 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T07:14:31.753 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-07-17-07_15_00 in ~29s... 2024-07-17T07:14:31.753 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2024-07-17T07:14:31.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-07-17T07:14:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 -- 172.21.15.44:0/2583781556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 msgr2=0x7f92b40ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 --2- 172.21.15.44:0/2583781556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b40ac190 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f929c009830 tx=0x7f929c02a510 comp rx=0 tx=0).stop 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 -- 172.21.15.44:0/2583781556 shutdown_connections 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 --2- 172.21.15.44:0/2583781556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b40ac190 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 --2- 172.21.15.44:0/2583781556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b415a280 0x7f92b40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.967+0000 7f92ba800640 1 -- 172.21.15.44:0/2583781556 >> 172.21.15.44:0/2583781556 conn(0x7f92b40b0eb0 msgr2=0x7f92b40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.968+0000 7f92ba800640 1 -- 172.21.15.44:0/2583781556 shutdown_connections 2024-07-17T07:14:31.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.968+0000 7f92ba800640 1 -- 172.21.15.44:0/2583781556 wait complete. 2024-07-17T07:14:31.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.969+0000 7f92ba800640 1 Processor -- start 2024-07-17T07:14:31.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.969+0000 7f92ba800640 1 -- start start 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b406efc0 0x7f92b4152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 0x7f92b41594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92b40aca60 con 0x7f92b40b1af0 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92b40ac8e0 con 0x7f92b415a280 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.970+0000 7f92ba800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92b406d7c0 con 0x7f92b406efc0 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35598/0 (socket says 172.21.15.44:35598) 2024-07-17T07:14:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 -- 172.21.15.44:0/3894949144 learned_addr learned my addr 172.21.15.44:0/3894949144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92b3e00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 0x7f92b41594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92b3400640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b406efc0 0x7f92b4152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 -- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b406efc0 msgr2=0x7f92b4152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b406efc0 0x7f92b4152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 -- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 msgr2=0x7f92b41594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 0x7f92b41594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92aaa00640 1 -- 172.21.15.44:0/3894949144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4152e70 con 0x7f92b40b1af0 2024-07-17T07:14:31.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.971+0000 7f92b3e00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 0x7f92b41594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92aaa00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f929c009950 tx=0x7f929c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c040070 con 0x7f92b40b1af0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f929c005310 con 0x7f92b40b1af0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b4153150 con 0x7f92b40b1af0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0047a0 con 0x7f92b40b1af0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.972+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b4153710 con 0x7f92b40b1af0 2024-07-17T07:14:31.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.973+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b415b6d0 con 0x7f92b40b1af0 2024-07-17T07:14:31.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.974+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 61) ==== 98684+0+0 (secure 0 0 0) 0x7f929c039040 con 0x7f92b40b1af0 2024-07-17T07:14:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.975+0000 7f92abe00640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 0x7f9290078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:14:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.975+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7f929c0b98e0 con 0x7f92b40b1af0 2024-07-17T07:14:31.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.976+0000 7f92b3400640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 0x7f9290078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:14:31.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.977+0000 7f92b3400640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 0x7f9290078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a4005850 comp rx=0 tx=0).ready entity=mgr.6397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:14:31.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:31.981+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f929c082b30 con 0x7f92b40b1af0 2024-07-17T07:14:32.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.225+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 --> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92b406e050 con 0x7f9290076a50 2024-07-17T07:14:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.243+0000 7f92abe00640 1 -- 172.21.15.44:0/3894949144 <== mgr.6397 v2:172.21.15.175:6800/1035934967 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f92b406e050 con 0x7f9290076a50 2024-07-17T07:14:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:14:32.243 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2024-07-17T07:14:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 msgr2=0x7f9290078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 0x7f9290078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a4005850 comp rx=0 tx=0).stop 2024-07-17T07:14:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 msgr2=0x7f92b41508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:14:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f929c009950 tx=0x7f929c032c10 comp rx=0 tx=0).stop 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 shutdown_connections 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f9290076a50 0x7f9290078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92b415a280 0x7f92b41594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92b40b1af0 0x7f92b41508c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 --2- 172.21.15.44:0/3894949144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92b406efc0 0x7f92b4152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 >> 172.21.15.44:0/3894949144 conn(0x7f92b40b0eb0 msgr2=0x7f92b40cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.246+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 shutdown_connections 2024-07-17T07:14:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:14:32.247+0000 7f92ba800640 1 -- 172.21.15.44:0/3894949144 wait complete. 2024-07-17T07:14:32.264 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2024-07-17T07:14:32.264 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2024-07-17T07:14:32.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:32.265 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:32.318 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:32.318 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:32.318 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:33.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:33.320 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:33.381 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:33.381 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:33.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:34.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:34.383 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:34.445 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:34.445 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:34.445 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:35.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:35.446 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:35.508 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:35.508 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:35.508 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:36.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:36.509 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:36.572 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:36.573 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:36.573 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:37.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:37.574 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:37.636 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:37.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:37.637 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:38.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:38.637 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:38.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:38.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:38.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:39.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:39.695 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:39.757 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:39.757 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:39.757 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:40.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:40.758 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:40.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:40.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:40.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:41.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:41.822 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:41.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:41.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:41.885 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:42.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:42.885 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:42.948 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:42.948 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:42.948 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:43.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:43.949 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:44.013 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:44.013 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:44.013 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:45.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:45.014 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:45.075 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:45.075 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:45.076 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:46.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:46.076 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:46.137 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:46.137 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:46.137 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:46.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:14:46.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:14:47.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:47.139 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:47.199 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:47.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:47.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:48.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:48.200 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:48.262 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:48.263 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:48.263 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:49.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:49.264 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:49.326 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:49.327 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:49.327 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:50.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:50.328 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:50.391 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:50.391 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:50.391 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:51.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:51.392 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:51.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:51.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:51.453 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:52.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:52.454 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:52.516 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:52.516 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:52.517 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:53.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:53.517 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:53.578 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:53.578 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:53.578 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:54.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:54.580 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:54.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:54.641 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:54.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:55.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:55.642 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:55.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:55.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:55.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:56.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:56.709 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:56.771 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:56.772 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:56.772 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:57.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:57.773 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:57.836 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:57.836 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:57.837 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:58.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:58.838 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:58.899 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:58.899 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:58.899 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:14:59.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:14:59.900 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:14:59.961 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T07:14:59.962 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T07:14:59.962 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:15:00.750 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:00.752+0000 7ff075c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:00.751 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:00.752+0000 7ff075c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:00.751 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:00.752+0000 7ff075c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:00.751 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:00.752+0000 7ff075c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:00.751 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:00.752+0000 7ff075c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:00.963 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:15:01.022 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_15_00_UTC 2024-07-17T07:15:01.023 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-07_15_00_UTC'} 2024-07-17T07:15:01.023 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2024-07-17-07_15_00_UTC'} 2024-07-17T07:15:01.023 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T07:15:01.024 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-07-17-07_15_00_UTC'] 2024-07-17T07:15:01.024 DEBUG:tasks.cephfs.test_snap_schedules:exec=1721200471.7530193, now = 1721200501.024193, timo = 29 2024-07-17T07:15:02.025 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-07-17T07:15:02.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 -- 172.21.15.44:0/522267473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40b1af0 msgr2=0x7f81a415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 --2- 172.21.15.44:0/522267473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40b1af0 0x7f81a415ac10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f819c005fd0 tx=0x7f819c02a8b0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 -- 172.21.15.44:0/522267473 shutdown_connections 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 --2- 172.21.15.44:0/522267473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40b1af0 0x7f81a415ac10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 --2- 172.21.15.44:0/522267473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a40ac8a0 0x7f81a40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 -- 172.21.15.44:0/522267473 >> 172.21.15.44:0/522267473 conn(0x7f81a40b0eb0 msgr2=0x7f81a40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.255+0000 7f81aa600640 1 -- 172.21.15.44:0/522267473 shutdown_connections 2024-07-17T07:15:02.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.256+0000 7f81aa600640 1 -- 172.21.15.44:0/522267473 wait complete. 2024-07-17T07:15:02.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.258+0000 7f81aa600640 1 Processor -- start 2024-07-17T07:15:02.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.258+0000 7f81aa600640 1 -- start start 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a40b1af0 0x7f81a4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a415a280 0x7f81a40ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81a415b150 con 0x7f81a40ac8a0 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81a406f1d0 con 0x7f81a40b1af0 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81aa600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81a415b2d0 con 0x7f81a415a280 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58404/0 (socket says 172.21.15.44:58404) 2024-07-17T07:15:02.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81a3400640 1 -- 172.21.15.44:0/972321120 learned_addr learned my addr 172.21.15.44:0/972321120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.259+0000 7f81a2a00640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a40b1af0 0x7f81a4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3e00640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a415a280 0x7f81a40ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 -- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a415a280 msgr2=0x7f81a40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a415a280 0x7f81a40ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 -- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a40b1af0 msgr2=0x7f81a4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:02.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a40b1af0 0x7f81a4154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 -- 172.21.15.44:0/972321120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41f8000 con 0x7f81a40ac8a0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81a3400640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f819400c3a0 tx=0x7f819400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194011790 con 0x7f81a40ac8a0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8194017070 con 0x7f81a40ac8a0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194011cc0 con 0x7f81a40ac8a0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.260+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41f82a0 con 0x7f81a40ac8a0 2024-07-17T07:15:02.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.261+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a40cac00 con 0x7f81a40ac8a0 2024-07-17T07:15:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.262+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a40b21b0 con 0x7f81a40ac8a0 2024-07-17T07:15:02.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.263+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 63) ==== 99005+0+0 (secure 0 0 0) 0x7f819401c460 con 0x7f81a40ac8a0 2024-07-17T07:15:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.264+0000 7f819b400640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 0x7f8180079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.264+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7f8194098930 con 0x7f81a40ac8a0 2024-07-17T07:15:02.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.265+0000 7f81a2a00640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 0x7f8180079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:02.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.266+0000 7f81a2a00640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 0x7f8180079270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f818c005fd0 tx=0x7f818c005850 comp rx=0 tx=0).ready entity=mgr.6397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:02.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.268+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8194061680 con 0x7f81a40ac8a0 2024-07-17T07:15:02.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.504+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 --> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f81a40ac310 con 0x7f8180076db0 2024-07-17T07:15:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.706+0000 7f819b400640 1 -- 172.21.15.44:0/972321120 <== mgr.6397 v2:172.21.15.175:6800/1035934967 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f81a40ac310 con 0x7f8180076db0 2024-07-17T07:15:02.706 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path snap_test_dir1 2024-07-17T07:15:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 msgr2=0x7f8180079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 0x7f8180079270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f818c005fd0 tx=0x7f818c005850 comp rx=0 tx=0).stop 2024-07-17T07:15:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 msgr2=0x7f81a414f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f819400c3a0 tx=0x7f819400c860 comp rx=0 tx=0).stop 2024-07-17T07:15:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 shutdown_connections 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f8180076db0 0x7f8180079270 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81a415a280 0x7f81a40ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a40b1af0 0x7f81a4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 --2- 172.21.15.44:0/972321120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81a40ac8a0 0x7f81a414f740 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 >> 172.21.15.44:0/972321120 conn(0x7f81a40b0eb0 msgr2=0x7f81a4155b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.710+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 shutdown_connections 2024-07-17T07:15:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:02.711+0000 7f81aa600640 1 -- 172.21.15.44:0/972321120 wait complete. 2024-07-17T07:15:02.734 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-07-17T07:15:02.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:02.734 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T07:15:02.788 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_15_00_UTC 2024-07-17T07:15:02.790 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_15_00_UTC 2024-07-17T07:15:02.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:02.790 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_15_00_UTC) 2024-07-17T07:15:02.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:02.850 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T07:15:02.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:15:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 -- 172.21.15.44:0/1220909908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 msgr2=0x7fbb74165c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 --2- 172.21.15.44:0/1220909908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb74165c10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005fd0 tx=0x7fbb6c02a820 comp rx=0 tx=0).stop 2024-07-17T07:15:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 -- 172.21.15.44:0/1220909908 shutdown_connections 2024-07-17T07:15:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 --2- 172.21.15.44:0/1220909908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb74165c10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 -- 172.21.15.44:0/1220909908 >> 172.21.15.44:0/1220909908 conn(0x7fbb7406c120 msgr2=0x7fbb7406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 -- 172.21.15.44:0/1220909908 shutdown_connections 2024-07-17T07:15:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.121+0000 7fbb73200640 1 -- 172.21.15.44:0/1220909908 wait complete. 2024-07-17T07:15:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.123+0000 7fbb73200640 1 Processor -- start 2024-07-17T07:15:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.123+0000 7fbb73200640 1 -- start start 2024-07-17T07:15:03.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.124+0000 7fbb73200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb740ad790 0x7fbb74154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:03.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.124+0000 7fbb73200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.124+0000 7fbb73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb7415afd0 0x7fbb741574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb73200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb741664e0 con 0x7fbb740ae3c0 2024-07-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb73200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb74166360 con 0x7fbb740ad790 2024-07-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbb74166660 con 0x7fbb7415afd0 2024-07-17T07:15:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb6be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb6be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58418/0 (socket says 172.21.15.44:58418) 2024-07-17T07:15:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb6be00640 1 -- 172.21.15.44:0/1775912865 learned_addr learned my addr 172.21.15.44:0/1775912865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb72800640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb7415afd0 0x7fbb741574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.125+0000 7fbb71e00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb740ad790 0x7fbb74154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 -- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb7415afd0 msgr2=0x7fbb741574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb7415afd0 0x7fbb741574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 -- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb740ad790 msgr2=0x7fbb74154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb740ad790 0x7fbb74154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 -- 172.21.15.44:0/1775912865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb74154c80 con 0x7fbb740ae3c0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6be00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c430 tx=0x7fbb5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011790 con 0x7fbb740ae3c0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb5c017070 con 0x7fbb740ae3c0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011cc0 con 0x7fbb740ae3c0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb74154ef0 con 0x7fbb740ae3c0 2024-07-17T07:15:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.126+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb741f8530 con 0x7fbb740ae3c0 2024-07-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.127+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb740adcb0 con 0x7fbb740ae3c0 2024-07-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.129+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 64) ==== 99062+0+0 (secure 0 0 0) 0x7fbb5c01c460 con 0x7fbb740ae3c0 2024-07-17T07:15:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.130+0000 7fbb6a000640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 0x7fbb50079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:03.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.130+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7fbb5c098b10 con 0x7fbb740ae3c0 2024-07-17T07:15:03.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.130+0000 7fbb71e00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 0x7fbb50079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:03.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.132+0000 7fbb71e00640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 0x7fbb50079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb740cd2a0 tx=0x7fbb64009040 comp rx=0 tx=0).ready entity=mgr.6397 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:03.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.135+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbb5c061850 con 0x7fbb740ae3c0 2024-07-17T07:15:03.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.405+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fbb74156b70 con 0x7fbb740ae3c0 2024-07-17T07:15:03.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.743+0000 7fbb6a000640 1 -- 172.21.15.44:0/1775912865 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v65) ==== 93+0+0 (secure 0 0 0) 0x7fbb5c061150 con 0x7fbb740ae3c0 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.747+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 msgr2=0x7fbb50079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.747+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 0x7fbb50079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb740cd2a0 tx=0x7fbb64009040 comp rx=0 tx=0).stop 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.747+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 msgr2=0x7fbb7414e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.747+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c430 tx=0x7fbb5c00c8f0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 shutdown_connections 2024-07-17T07:15:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fbb50076e40 0x7fbb50079300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb7415afd0 0x7fbb741574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb740ae3c0 0x7fbb7414e220 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 --2- 172.21.15.44:0/1775912865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb740ad790 0x7fbb74154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 >> 172.21.15.44:0/1775912865 conn(0x7fbb7406c120 msgr2=0x7fbb74159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 shutdown_connections 2024-07-17T07:15:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:03.748+0000 7fbb73200640 1 -- 172.21.15.44:0/1775912865 wait complete. 2024-07-17T07:15:03.772 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:15:03.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:03.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:03.964 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:15:03.965 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:03.965 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:03.965 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:04.019 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:04.019 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:04.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 -- 172.21.15.44:0/1259622712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc00003a00 msgr2=0x7fcc00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 --2- 172.21.15.44:0/1259622712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc00003a00 0x7fcc00003e20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e20 tx=0x7fcc00005830 comp rx=0 tx=0).stop 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc1b400640 1 -- 172.21.15.44:0/1259622712 reap_dead start 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 -- 172.21.15.44:0/1259622712 shutdown_connections 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 -- 172.21.15.44:0/1259622712 >> 172.21.15.44:0/1259622712 conn(0x7fcc1c0b0eb0 msgr2=0x7fcc1c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 -- 172.21.15.44:0/1259622712 shutdown_connections 2024-07-17T07:15:04.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.355+0000 7fcc21e00640 1 -- 172.21.15.44:0/1259622712 wait complete. 2024-07-17T07:15:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.357+0000 7fcc21e00640 1 Processor -- start 2024-07-17T07:15:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.357+0000 7fcc21e00640 1 -- start start 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc00003a00 0x7fcc1c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 0x7fcc1c1f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0cb810 con 0x7fcc1c158860 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c15c6c0 con 0x7fcc1c158ce0 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1c0cb990 con 0x7fcc00003a00 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc1a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.358+0000 7fcc1a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58444/0 (socket says 172.21.15.44:58444) 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 -- 172.21.15.44:0/3508410399 learned_addr learned my addr 172.21.15.44:0/3508410399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1b400640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 0x7fcc1c1f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc00003a00 0x7fcc1c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc00003a00 msgr2=0x7fcc1c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc00003a00 0x7fcc1c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 msgr2=0x7fcc1c1f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 0x7fcc1c1f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 -- 172.21.15.44:0/3508410399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c1563b0 con 0x7fcc1c158860 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1b400640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 0x7fcc1c1f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.359+0000 7fcc1a000640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d750 tx=0x7fcc0400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.360+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04017070 con 0x7fcc1c158860 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.360+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc04009cc0 con 0x7fcc1c158860 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.360+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400de60 con 0x7fcc1c158860 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.360+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c157aa0 con 0x7fcc1c158860 2024-07-17T07:15:04.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.360+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c158060 con 0x7fcc1c158860 2024-07-17T07:15:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.361+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c06d620 con 0x7fcc1c158860 2024-07-17T07:15:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.362+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x7fcc1c158860 2024-07-17T07:15:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.363+0000 7fcc13400640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 0x7fcbf80792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.363+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7fcc04098e60 con 0x7fcc1c158860 2024-07-17T07:15:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.363+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 msgr2=0x7fcbf80792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.363+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 0x7fcbf80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:04.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.371+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcc04061930 con 0x7fcc1c158860 2024-07-17T07:15:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.564+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 msgr2=0x7fcbf80792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.564+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 0x7fcbf80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:04.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.595+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc1c075550 con 0x7fcc1c158860 2024-07-17T07:15:04.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.596+0000 7fcc13400640 1 -- 172.21.15.44:0/3508410399 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v129) ==== 80+0+476 (secure 0 0 0) 0x7fcc04061230 con 0x7fcc1c158860 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/232965093 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2886999503 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/1224757683 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/1224757683 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4116488581 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1212729465 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/347512883 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1010255644 2024-07-18T07:14:30.308488+0000 2024-07-17T07:15:04.595 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.598+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 msgr2=0x7fcbf80792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.598+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 0x7fcbf80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.598+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 msgr2=0x7fcc1c1f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.598+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d750 tx=0x7fcc0400dc10 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 shutdown_connections 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcbf8076e20 0x7fcbf80792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c158ce0 0x7fcc1c1f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c158860 0x7fcc1c1f6f00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 --2- 172.21.15.44:0/3508410399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc00003a00 0x7fcc1c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 >> 172.21.15.44:0/3508410399 conn(0x7fcc1c0b0eb0 msgr2=0x7fcc1c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 shutdown_connections 2024-07-17T07:15:04.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:04.599+0000 7fcc21e00640 1 -- 172.21.15.44:0/3508410399 wait complete. 2024-07-17T07:15:04.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:15:04.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:04.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:04.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:15:04.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:04.698 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:15:04.738 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:04.738 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:04.738 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:04.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:04.780 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:04.780 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:04.780 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:04.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:04.832 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:04.833 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:04.833 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:04.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:04.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 -- 172.21.15.44:0/449842999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 msgr2=0x7fd4a00cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/449842999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a00cf340 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd490009830 tx=0x7fd49002a510 comp rx=0 tx=0).stop 2024-07-17T07:15:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 -- 172.21.15.44:0/449842999 shutdown_connections 2024-07-17T07:15:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/449842999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a00cf340 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 -- 172.21.15.44:0/449842999 >> 172.21.15.44:0/449842999 conn(0x7fd4a00af940 msgr2=0x7fd4a00afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 -- 172.21.15.44:0/449842999 shutdown_connections 2024-07-17T07:15:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.089+0000 7fd4a5e00640 1 -- 172.21.15.44:0/449842999 wait complete. 2024-07-17T07:15:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.091+0000 7fd4a5e00640 1 Processor -- start 2024-07-17T07:15:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.091+0000 7fd4a5e00640 1 -- start start 2024-07-17T07:15:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 0x7fd4a01f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00b4f70 0x7fd4a0073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00ce0f0 con 0x7fd4a0154650 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00cdf70 con 0x7fd4a006efd0 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a00b53e0 con 0x7fd4a00b4f70 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd49f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd49f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58450/0 (socket says 172.21.15.44:58450) 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.092+0000 7fd49f400640 1 -- 172.21.15.44:0/3619292429 learned_addr learned my addr 172.21.15.44:0/3619292429 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49ea00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 0x7fd4a01f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49e000640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00b4f70 0x7fd4a0073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00b4f70 msgr2=0x7fd4a0073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00b4f70 0x7fd4a0073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 msgr2=0x7fd4a01f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 0x7fd4a01f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 -- 172.21.15.44:0/3619292429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a014ad00 con 0x7fd4a0154650 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49ea00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 0x7fd4a01f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.093+0000 7fd49f400640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c770 tx=0x7fd49800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.094+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498003da0 con 0x7fd4a0154650 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.094+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd498005070 con 0x7fd4a0154650 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.094+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498005500 con 0x7fd4a0154650 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.094+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a0146e90 con 0x7fd4a0154650 2024-07-17T07:15:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.094+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a0147450 con 0x7fd4a0154650 2024-07-17T07:15:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.095+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a00cf340 con 0x7fd4a0154650 2024-07-17T07:15:05.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.096+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fd4980056a0 con 0x7fd4a0154650 2024-07-17T07:15:05.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.097+0000 7fd497400640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 0x7fd47c079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.097+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7fd498098a10 con 0x7fd4a0154650 2024-07-17T07:15:05.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.097+0000 7fd49ea00640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 msgr2=0x7fd47c079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:05.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.098+0000 7fd49ea00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 0x7fd47c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:05.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.101+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd498061700 con 0x7fd4a0154650 2024-07-17T07:15:05.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.299+0000 7fd49ea00640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 msgr2=0x7fd47c079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:05.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.299+0000 7fd49ea00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 0x7fd47c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.347+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a00a9980 con 0x7fd4a0154650 2024-07-17T07:15:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.348+0000 7fd497400640 1 -- 172.21.15.44:0/3619292429 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7fd498061000 con 0x7fd4a0154650 2024-07-17T07:15:05.347 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:05.347 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:15:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:15:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.351+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 msgr2=0x7fd47c079330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.351+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 0x7fd47c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.351+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 msgr2=0x7fd4a0143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.351+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c770 tx=0x7fd49800cc30 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 shutdown_connections 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fd47c076e70 0x7fd47c079330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4a0154650 0x7fd4a0143c60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00b4f70 0x7fd4a0073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 --2- 172.21.15.44:0/3619292429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a006efd0 0x7fd4a01f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 >> 172.21.15.44:0/3619292429 conn(0x7fd4a00af940 msgr2=0x7fd4a0072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 shutdown_connections 2024-07-17T07:15:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.352+0000 7fd4a5e00640 1 -- 172.21.15.44:0/3619292429 wait complete. 2024-07-17T07:15:05.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.568+0000 7fdc64a00640 1 -- 172.21.15.44:0/992796473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 msgr2=0x7fdc600ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.568+0000 7fdc64a00640 1 --2- 172.21.15.44:0/992796473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc600ca7a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005fd0 tx=0x7fdc4802a530 comp rx=0 tx=0).stop 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 -- 172.21.15.44:0/992796473 shutdown_connections 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 --2- 172.21.15.44:0/992796473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 0x7fdc600cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 --2- 172.21.15.44:0/992796473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc600ca7a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 -- 172.21.15.44:0/992796473 >> 172.21.15.44:0/992796473 conn(0x7fdc600b3120 msgr2=0x7fdc600b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 -- 172.21.15.44:0/992796473 shutdown_connections 2024-07-17T07:15:05.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.569+0000 7fdc64a00640 1 -- 172.21.15.44:0/992796473 wait complete. 2024-07-17T07:15:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.570+0000 7fdc64a00640 1 Processor -- start 2024-07-17T07:15:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.571+0000 7fdc64a00640 1 -- start start 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.571+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 0x7fdc60153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 0x7fdc6015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc64a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc600b2c70 con 0x7fdc600b1c50 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc64a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006ed90 con 0x7fdc60160a80 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc64a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc600b2df0 con 0x7fdc600b3c40 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58482/0 (socket says 172.21.15.44:58482) 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 -- 172.21.15.44:0/2886318882 learned_addr learned my addr 172.21.15.44:0/2886318882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 msgr2=0x7fdc60153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc55400640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 0x7fdc60153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5e800640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 0x7fdc6015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 0x7fdc60153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 msgr2=0x7fdc6015e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 0x7fdc6015e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5de00640 1 -- 172.21.15.44:0/2886318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc60158200 con 0x7fdc600b1c50 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.572+0000 7fdc5e800640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 0x7fdc6015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc5de00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7d0 tx=0x7fdc5000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50017070 con 0x7fdc600b1c50 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc50005070 con 0x7fdc600b1c50 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500054c0 con 0x7fdc600b1c50 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601f7a40 con 0x7fdc600b1c50 2024-07-17T07:15:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.573+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601f8000 con 0x7fdc600b1c50 2024-07-17T07:15:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.574+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc600cb290 con 0x7fdc600b1c50 2024-07-17T07:15:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.576+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fdc50005660 con 0x7fdc600b1c50 2024-07-17T07:15:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.577+0000 7fdc56a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 0x7fdc3c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.577+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7fdc500981d0 con 0x7fdc600b1c50 2024-07-17T07:15:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.577+0000 7fdc55400640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 msgr2=0x7fdc3c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.577+0000 7fdc55400640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 0x7fdc3c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:05.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.582+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdc50060e40 con 0x7fdc600b1c50 2024-07-17T07:15:05.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.778+0000 7fdc55400640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 msgr2=0x7fdc3c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:05.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.778+0000 7fdc55400640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 0x7fdc3c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:05.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.837+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc6015c180 con 0x7fdc600b1c50 2024-07-17T07:15:05.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.838+0000 7fdc56a00640 1 -- 172.21.15.44:0/2886318882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7fdc50069080 con 0x7fdc600b1c50 2024-07-17T07:15:05.837 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:05.837 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:15:05.838 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 msgr2=0x7fdc3c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 0x7fdc3c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 msgr2=0x7fdc60157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7d0 tx=0x7fdc5000cc90 comp rx=0 tx=0).stop 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 shutdown_connections 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fdc3c076df0 0x7fdc3c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc60160a80 0x7fdc6015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc600b3c40 0x7fdc60153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 --2- 172.21.15.44:0/2886318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc600b1c50 0x7fdc60157cc0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.841+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 >> 172.21.15.44:0/2886318882 conn(0x7fdc600b3120 msgr2=0x7fdc600b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.842+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 shutdown_connections 2024-07-17T07:15:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:05.842+0000 7fdc64a00640 1 -- 172.21.15.44:0/2886318882 wait complete. 2024-07-17T07:15:05.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 -- 172.21.15.44:0/3431831639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c15b1d0 msgr2=0x7fa39c070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 --2- 172.21.15.44:0/3431831639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c15b1d0 0x7fa39c070810 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0d02c0 tx=0x7fa39402a8b0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 -- 172.21.15.44:0/3431831639 shutdown_connections 2024-07-17T07:15:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 --2- 172.21.15.44:0/3431831639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c15b1d0 0x7fa39c070810 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 --2- 172.21.15.44:0/3431831639 >> v2:172.21.15.175:3300/0 conn(0x7fa39c0afb30 0x7fa39c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 -- 172.21.15.44:0/3431831639 >> 172.21.15.44:0/3431831639 conn(0x7fa39c0b2240 msgr2=0x7fa39c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 -- 172.21.15.44:0/3431831639 shutdown_connections 2024-07-17T07:15:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.049+0000 7fa3a3200640 1 -- 172.21.15.44:0/3431831639 wait complete. 2024-07-17T07:15:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.052+0000 7fa3a3200640 1 Processor -- start 2024-07-17T07:15:06.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.052+0000 7fa3a3200640 1 -- start start 2024-07-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa39c157da0 0x7fa39c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa39c15b1d0 0x7fa39c146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0ce4a0 con 0x7fa39c0afb30 2024-07-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0ce320 con 0x7fa39c15b1d0 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.053+0000 7fa3a3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c0ce620 con 0x7fa39c157da0 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58512/0 (socket says 172.21.15.44:58512) 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 -- 172.21.15.44:0/1658012054 learned_addr learned my addr 172.21.15.44:0/1658012054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa3a1200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa39c15b1d0 0x7fa39c146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39b400640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa39c157da0 0x7fa39c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa39c157da0 msgr2=0x7fa39c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa39c157da0 0x7fa39c0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa39c15b1d0 msgr2=0x7fa39c146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa39c15b1d0 0x7fa39c146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.054+0000 7fa39be00640 1 -- 172.21.15.44:0/1658012054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c0b3d90 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa39be00640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c450 tx=0x7fa38c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011790 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa38c017070 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011cc0 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c0b4070 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.055+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c0b4630 con 0x7fa39c0afb30 2024-07-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.056+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c0ceb40 con 0x7fa39c0afb30 2024-07-17T07:15:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.058+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fa38c01c460 con 0x7fa39c0afb30 2024-07-17T07:15:06.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.059+0000 7fa393e00640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 0x7fa37c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.059+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7fa38c0994d0 con 0x7fa39c0afb30 2024-07-17T07:15:06.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.059+0000 7fa39b400640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 msgr2=0x7fa37c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:06.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.059+0000 7fa39b400640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 0x7fa37c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:06.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.063+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa38c062210 con 0x7fa39c0afb30 2024-07-17T07:15:06.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.260+0000 7fa39b400640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 msgr2=0x7fa37c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:06.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.260+0000 7fa39b400640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 0x7fa37c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:06.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.271+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa39c15dde0 con 0x7fa39c0afb30 2024-07-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.273+0000 7fa393e00640 1 -- 172.21.15.44:0/1658012054 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+18165 (secure 0 0 0) 0x7fa38c061b10 con 0x7fa39c0afb30 2024-07-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":129,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:14:30.308538+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:14:13.391372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-07-17T07:14:14.431910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/232965093":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/2886999503":"2024-07-18T07:14:30.308488+0000","172.21.15.175:6801/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:6800/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/4116488581":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1212729465":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/347512883":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1010255644":"2024-07-18T07:14:30.308488+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 msgr2=0x7fa37c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 0x7fa37c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 msgr2=0x7fa39c14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c450 tx=0x7fa38c00c910 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 shutdown_connections 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa37c076df0 0x7fa37c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa39c15b1d0 0x7fa39c146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa39c157da0 0x7fa39c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 --2- 172.21.15.44:0/1658012054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa39c0afb30 0x7fa39c14d3a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.276+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 >> 172.21.15.44:0/1658012054 conn(0x7fa39c0b2240 msgr2=0x7fa39c15e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.277+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 shutdown_connections 2024-07-17T07:15:06.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.277+0000 7fa3a3200640 1 -- 172.21.15.44:0/1658012054 wait complete. 2024-07-17T07:15:06.297 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:15:06.297 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.499+0000 7f14f7200640 1 -- 172.21.15.44:0/268597148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc004800 msgr2=0x7f14dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.499+0000 7f14f7200640 1 --2- 172.21.15.44:0/268597148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc004800 0x7f14dc004c80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c260 tx=0x7f14dc005830 comp rx=0 tx=0).stop 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.500+0000 7f14f7200640 1 -- 172.21.15.44:0/268597148 shutdown_connections 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.500+0000 7f14f7200640 1 --2- 172.21.15.44:0/268597148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14dc004800 0x7f14dc004c80 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.500+0000 7f14f7200640 1 -- 172.21.15.44:0/268597148 >> 172.21.15.44:0/268597148 conn(0x7f14f80cfec0 msgr2=0x7f14f80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.500+0000 7f14f7200640 1 -- 172.21.15.44:0/268597148 shutdown_connections 2024-07-17T07:15:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.500+0000 7f14f7200640 1 -- 172.21.15.44:0/268597148 wait complete. 2024-07-17T07:15:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.501+0000 7f14f7200640 1 Processor -- start 2024-07-17T07:15:06.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 -- start start 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 0x7f14f815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 0x7f14f80ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80b5040 con 0x7f14f80ad0f0 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80b4ec0 con 0x7f14dc004800 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.502+0000 7f14f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80b51c0 con 0x7f14f81f7930 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58532/0 (socket says 172.21.15.44:58532) 2024-07-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 -- 172.21.15.44:0/2193815818 learned_addr learned my addr 172.21.15.44:0/2193815818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14f5e00640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 0x7f14f815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14f6800640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 0x7f14f80ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 msgr2=0x7f14f80ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 0x7f14f80ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 msgr2=0x7f14f815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 0x7f14f815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14ed400640 1 -- 172.21.15.44:0/2193815818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f80ac330 con 0x7f14f80ad0f0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14f6800640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 0x7f14f80ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.503+0000 7f14f5e00640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 0x7f14f815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14ed400640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c380 tx=0x7f14e0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0031040 con 0x7f14f80ad0f0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14e0006440 con 0x7f14f80ad0f0 2024-07-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0001cb0 con 0x7f14f80ad0f0 2024-07-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f80ac610 con 0x7f14f80ad0f0 2024-07-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.504+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f80acbd0 con 0x7f14f80ad0f0 2024-07-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.505+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f815f7d0 con 0x7f14f80ad0f0 2024-07-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.506+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7f14e0034070 con 0x7f14f80ad0f0 2024-07-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.507+0000 7f14eea00640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 0x7f14d40792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.507+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6965+0+0 (secure 0 0 0) 0x7f14e00b8b80 con 0x7f14f80ad0f0 2024-07-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.507+0000 7f14f5e00640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 msgr2=0x7f14d40792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.507+0000 7f14f5e00640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 0x7f14d40792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:06.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.511+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f14e00816e0 con 0x7f14f80ad0f0 2024-07-17T07:15:06.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.708+0000 7f14f5e00640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 msgr2=0x7f14d40792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:06.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.708+0000 7f14f5e00640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 0x7f14d40792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:06.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.766+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14f806dca0 con 0x7f14f80ad0f0 2024-07-17T07:15:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.770+0000 7f14eea00640 1 -- 172.21.15.44:0/2193815818 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f14e0001710 con 0x7f14f80ad0f0 2024-07-17T07:15:06.770 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:06.770 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:15:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 msgr2=0x7f14d40792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 0x7f14d40792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 msgr2=0x7f14f81f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c380 tx=0x7f14e0004dd0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 shutdown_connections 2024-07-17T07:15:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7f14d4076e20 0x7f14d40792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14f81f7930 0x7f14f80ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80ad0f0 0x7f14f81f73f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 --2- 172.21.15.44:0/2193815818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc004800 0x7f14f815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 >> 172.21.15.44:0/2193815818 conn(0x7f14f80cfec0 msgr2=0x7f14f80d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.773+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 shutdown_connections 2024-07-17T07:15:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.774+0000 7f14f7200640 1 -- 172.21.15.44:0/2193815818 wait complete. 2024-07-17T07:15:06.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 -- 172.21.15.44:0/1896571146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0ac760 msgr2=0x7ff73c15ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 --2- 172.21.15.44:0/1896571146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0ac760 0x7ff73c15ee90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff73c070bf0 tx=0x7ff73402aa30 comp rx=0 tx=0).stop 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 -- 172.21.15.44:0/1896571146 shutdown_connections 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 --2- 172.21.15.44:0/1896571146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0adcd0 0x7ff73c0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 --2- 172.21.15.44:0/1896571146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0ac760 0x7ff73c15ee90 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 --2- 172.21.15.44:0/1896571146 >> v2:172.21.15.175:3300/0 conn(0x7ff73c1522c0 0x7ff73c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 -- 172.21.15.44:0/1896571146 >> 172.21.15.44:0/1896571146 conn(0x7ff73c0b0fb0 msgr2=0x7ff73c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 -- 172.21.15.44:0/1896571146 shutdown_connections 2024-07-17T07:15:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.982+0000 7ff744000640 1 -- 172.21.15.44:0/1896571146 wait complete. 2024-07-17T07:15:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.984+0000 7ff744000640 1 Processor -- start 2024-07-17T07:15:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.984+0000 7ff744000640 1 -- start start 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0ac760 0x7ff73c147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73c1522c0 0x7ff73c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c15f760 con 0x7ff73c0adcd0 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c15f5e0 con 0x7ff73c1522c0 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.985+0000 7ff744000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c15f8e0 con 0x7ff73c0ac760 2024-07-17T07:15:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58548/0 (socket says 172.21.15.44:58548) 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 -- 172.21.15.44:0/3232077944 learned_addr learned my addr 172.21.15.44:0/3232077944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff742000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73c1522c0 0x7ff73c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff741600640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0ac760 0x7ff73c147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0ac760 msgr2=0x7ff73c147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0ac760 0x7ff73c147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73c1522c0 msgr2=0x7ff73c150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73c1522c0 0x7ff73c150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.986+0000 7ff740c00640 1 -- 172.21.15.44:0/3232077944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c072e30 con 0x7ff73c0adcd0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff740c00640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c970 tx=0x7ff72800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728004220 con 0x7ff73c0adcd0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7280044b0 con 0x7ff73c0adcd0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728007bb0 con 0x7ff73c0adcd0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c073110 con 0x7ff73c0adcd0 2024-07-17T07:15:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.987+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c0736d0 con 0x7ff73c0adcd0 2024-07-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.988+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c0ce840 con 0x7ff73c0adcd0 2024-07-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.990+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7ff728003610 con 0x7ff73c0adcd0 2024-07-17T07:15:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.990+0000 7ff739600640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 0x7ff71c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.991+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 7021+0+0 (secure 0 0 0) 0x7ff728098fe0 con 0x7ff73c0adcd0 2024-07-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.991+0000 7ff741600640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 msgr2=0x7ff71c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.991+0000 7ff741600640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 0x7ff71c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:06.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:06.995+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff73c0ce840 con 0x7ff73c0adcd0 2024-07-17T07:15:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.193+0000 7ff741600640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 msgr2=0x7ff71c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.193+0000 7ff741600640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 0x7ff71c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:07.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.241+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff73c0d0140 con 0x7ff73c0adcd0 2024-07-17T07:15:07.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.242+0000 7ff739600640 1 -- 172.21.15.44:0/3232077944 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7ff728014030 con 0x7ff73c0adcd0 2024-07-17T07:15:07.242 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:07.242 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":61,"btime":"2024-07-17T07:14:19:013947+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:14:15.436136+0000","modified":"2024-07-17T07:14:18.521975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7500":{"gid":7500,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2442230112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2442230112},{"type":"v1","addr":"172.21.15.44:6835","nonce":2442230112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7500,"qdb_cluster":[7500]},"id":7}]} 2024-07-17T07:15:07.243 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 61 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 msgr2=0x7ff71c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 0x7ff71c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 msgr2=0x7ff73c0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c970 tx=0x7ff72800ce30 comp rx=0 tx=0).stop 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 shutdown_connections 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7ff71c076df0 0x7ff71c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73c1522c0 0x7ff73c150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff73c0adcd0 0x7ff73c0c9270 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 --2- 172.21.15.44:0/3232077944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73c0ac760 0x7ff73c147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 >> 172.21.15.44:0/3232077944 conn(0x7ff73c0b0fb0 msgr2=0x7ff73c06e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 shutdown_connections 2024-07-17T07:15:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.245+0000 7ff744000640 1 -- 172.21.15.44:0/3232077944 wait complete. 2024-07-17T07:15:07.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 -- 172.21.15.44:0/2060883807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 msgr2=0x7fcaf4009cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 --2- 172.21.15.44:0/2060883807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcaf4009cb0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800c310 tx=0x7fcaf4003810 comp rx=0 tx=0).stop 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0e800640 1 -- 172.21.15.44:0/2060883807 reap_dead start 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 -- 172.21.15.44:0/2060883807 shutdown_connections 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 -- 172.21.15.44:0/2060883807 >> 172.21.15.44:0/2060883807 conn(0x7fcb100b0e80 msgr2=0x7fcb100b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 -- 172.21.15.44:0/2060883807 shutdown_connections 2024-07-17T07:15:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.459+0000 7fcb0f200640 1 -- 172.21.15.44:0/2060883807 wait complete. 2024-07-17T07:15:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.461+0000 7fcb0f200640 1 Processor -- start 2024-07-17T07:15:07.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.461+0000 7fcb0f200640 1 -- start start 2024-07-17T07:15:07.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb101f4190 0x7fcb100cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb10158500 0x7fcb10149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015a2a0 con 0x7fcaf4007820 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015a120 con 0x7fcb10158500 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb100b1fa0 con 0x7fcb101f4190 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58562/0 (socket says 172.21.15.44:58562) 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0de00640 1 -- 172.21.15.44:0/3662585805 learned_addr learned my addr 172.21.15.44:0/3662585805 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb05400640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb101f4190 0x7fcb100cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.462+0000 7fcb0e800640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb10158500 0x7fcb10149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb101f4190 msgr2=0x7fcb100cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb101f4190 0x7fcb100cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb10158500 msgr2=0x7fcb10149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb10158500 0x7fcb10149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 -- 172.21.15.44:0/3662585805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101f4710 con 0x7fcaf4007820 2024-07-17T07:15:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0de00640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcb00004e20 tx=0x7fcb00001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00015020 con 0x7fcaf4007820 2024-07-17T07:15:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb00011960 con 0x7fcaf4007820 2024-07-17T07:15:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101f49f0 con 0x7fcaf4007820 2024-07-17T07:15:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00011d70 con 0x7fcaf4007820 2024-07-17T07:15:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.463+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb10072410 con 0x7fcaf4007820 2024-07-17T07:15:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.464+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb100cf380 con 0x7fcaf4007820 2024-07-17T07:15:07.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.467+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fcb00018020 con 0x7fcaf4007820 2024-07-17T07:15:07.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.468+0000 7fcb06a00640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 0x7fcaec0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.468+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 7021+0+0 (secure 0 0 0) 0x7fcb00099860 con 0x7fcaf4007820 2024-07-17T07:15:07.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.468+0000 7fcb05400640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 msgr2=0x7fcaec0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:07.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.468+0000 7fcb05400640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 0x7fcaec0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:07.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.472+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcb0001f020 con 0x7fcaf4007820 2024-07-17T07:15:07.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.669+0000 7fcb05400640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 msgr2=0x7fcaec0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:07.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.669+0000 7fcb05400640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 0x7fcaec0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:07.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.682+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb101f4bc0 con 0x7fcaf4007820 2024-07-17T07:15:07.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.684+0000 7fcb06a00640 1 -- 172.21.15.44:0/3662585805 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+18277 (secure 0 0 0) 0x7fcb00062300 con 0x7fcaf4007820 2024-07-17T07:15:07.684 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:07.685 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":130,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:06.747103+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:14:13.391372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-07-17T07:14:14.431910+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/232965093":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/2886999503":"2024-07-18T07:14:30.308488+0000","172.21.15.175:6801/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:6800/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/4116488581":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1212729465":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/347512883":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1010255644":"2024-07-18T07:14:30.308488+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 msgr2=0x7fcaec0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 0x7fcaec0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 msgr2=0x7fcb10073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcb00004e20 tx=0x7fcb00001630 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 shutdown_connections 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fcaec076df0 0x7fcaec0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb10158500 0x7fcb10149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb101f4190 0x7fcb100cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 --2- 172.21.15.44:0/3662585805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4007820 0x7fcb10073e50 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.687+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 >> 172.21.15.44:0/3662585805 conn(0x7fcb100b0e80 msgr2=0x7fcb100adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.688+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 shutdown_connections 2024-07-17T07:15:07.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.688+0000 7fcb0f200640 1 -- 172.21.15.44:0/3662585805 wait complete. 2024-07-17T07:15:07.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:15:07.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.896+0000 7fa083e00640 1 -- 172.21.15.44:0/2197158713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 msgr2=0x7fa064009c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.896+0000 7fa083e00640 1 --2- 172.21.15.44:0/2197158713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa064009c60 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa068006f10 tx=0x7fa064003810 comp rx=0 tx=0).stop 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.896+0000 7fa081e00640 1 -- 172.21.15.44:0/2197158713 reap_dead start 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.896+0000 7fa083e00640 1 -- 172.21.15.44:0/2197158713 shutdown_connections 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.896+0000 7fa083e00640 1 -- 172.21.15.44:0/2197158713 >> 172.21.15.44:0/2197158713 conn(0x7fa07c0b0eb0 msgr2=0x7fa07c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.897+0000 7fa083e00640 1 -- 172.21.15.44:0/2197158713 shutdown_connections 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.897+0000 7fa083e00640 1 -- 172.21.15.44:0/2197158713 wait complete. 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.898+0000 7fa083e00640 1 Processor -- start 2024-07-17T07:15:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.898+0000 7fa083e00640 1 -- start start 2024-07-17T07:15:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 0x7fa07c1f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 0x7fa07c1f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c06f850 con 0x7fa0640077d0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c06f6d0 con 0x7fa07c157fa0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.899+0000 7fa083e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c15a5a0 con 0x7fa07c1f80f0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58576/0 (socket says 172.21.15.44:58576) 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 -- 172.21.15.44:0/1490216897 learned_addr learned my addr 172.21.15.44:0/1490216897 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa080a00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 0x7fa07c1f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 0x7fa07c1f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 -- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 msgr2=0x7fa07c1f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 0x7fa07c1f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 -- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 msgr2=0x7fa07c1f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 0x7fa07c1f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081400640 1 -- 172.21.15.44:0/1490216897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c1584f0 con 0x7fa0640077d0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa080a00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 0x7fa07c1f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.900+0000 7fa081e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 0x7fa07c1f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa081400640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa0700117d0 tx=0x7fa070011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070004460 con 0x7fa0640077d0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa070016070 con 0x7fa0640077d0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c1587d0 con 0x7fa0640077d0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000fb60 con 0x7fa0640077d0 2024-07-17T07:15:07.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.901+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c158d90 con 0x7fa0640077d0 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.902+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07c154030 con 0x7fa0640077d0 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.903+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 65) ==== 99045+0+0 (secure 0 0 0) 0x7fa070018030 con 0x7fa0640077d0 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.904+0000 7fa079600640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 0x7fa05c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.904+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 7021+0+0 (secure 0 0 0) 0x7fa070099420 con 0x7fa0640077d0 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.904+0000 7fa080a00640 1 -- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 msgr2=0x7fa05c0792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1035934967 2024-07-17T07:15:07.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.904+0000 7fa080a00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 0x7fa05c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:07.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:07.908+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa070062060 con 0x7fa0640077d0 2024-07-17T07:15:08.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.058+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 66) ==== 98540+0+0 (secure 0 0 0) 0x7fa07005dbf0 con 0x7fa0640077d0 2024-07-17T07:15:08.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.059+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 msgr2=0x7fa05c0792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:08.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.059+0000 7fa079600640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 0x7fa05c0792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.130 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.127+0000 7f7519000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.127+0000 7f7519000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.127+0000 7f7519000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.127+0000 7f7519000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.127+0000 7f7519000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.128+0000 7f751b800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.128+0000 7f751b800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.128+0000 7f751b800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.128+0000 7f751b800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.131 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:08.128+0000 7f751b800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.163+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa07c0ac830 con 0x7fa0640077d0 2024-07-17T07:15:08.164 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:15:08.165+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:15:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.167+0000 7fa079600640 1 -- 172.21.15.44:0/1490216897 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7fa0700810a0 con 0x7fa0640077d0 2024-07-17T07:15:08.168 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 msgr2=0x7fa07c149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa0700117d0 tx=0x7fa070011c90 comp rx=0 tx=0).stop 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 shutdown_connections 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:6800/1035934967,v1:172.21.15.175:6801/1035934967] conn(0x7fa05c076df0 0x7fa05c0792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa07c1f80f0 0x7fa07c1f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa07c157fa0 0x7fa07c1f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 --2- 172.21.15.44:0/1490216897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0640077d0 0x7fa07c149670 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 >> 172.21.15.44:0/1490216897 conn(0x7fa07c0b0eb0 msgr2=0x7fa07c0aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 shutdown_connections 2024-07-17T07:15:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.172+0000 7fa083e00640 1 -- 172.21.15.44:0/1490216897 wait complete. 2024-07-17T07:15:08.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:15:08.198 INFO:tasks.ceph.mds.a.smithi044.stderr: -8171> 2024-07-17T07:14:21.926+0000 7f0181400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:15:08.268 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:15:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 -- 172.21.15.44:0/3681780998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef0003a50 msgr2=0x7f5ef0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 --2- 172.21.15.44:0/3681780998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef0003a50 0x7f5ef0003e70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5efc006fa0 tx=0x7f5ef0005830 comp rx=0 tx=0).stop 2024-07-17T07:15:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 -- 172.21.15.44:0/3681780998 shutdown_connections 2024-07-17T07:15:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 --2- 172.21.15.44:0/3681780998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ef0003a50 0x7f5ef0003e70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 -- 172.21.15.44:0/3681780998 >> 172.21.15.44:0/3681780998 conn(0x7f5f0c06c120 msgr2=0x7f5f0c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 -- 172.21.15.44:0/3681780998 shutdown_connections 2024-07-17T07:15:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.379+0000 7f5f11800640 1 -- 172.21.15.44:0/3681780998 wait complete. 2024-07-17T07:15:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.381+0000 7f5f11800640 1 Processor -- start 2024-07-17T07:15:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.381+0000 7f5f11800640 1 -- start start 2024-07-17T07:15:08.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef0003a50 0x7f5f0c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 0x7f5f0c1f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c156ec0 con 0x7f5f0c149bb0 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c156d40 con 0x7f5f0c151a20 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f11800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c157040 con 0x7f5ef0003a50 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58618/0 (socket says 172.21.15.44:58618) 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.382+0000 7f5f0b400640 1 -- 172.21.15.44:0/382417703 learned_addr learned my addr 172.21.15.44:0/382417703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0be00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef0003a50 0x7f5f0c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 -- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef0003a50 msgr2=0x7f5f0c14fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef0003a50 0x7f5f0c14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 -- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 msgr2=0x7f5f0c1f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:08.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f10e00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 0x7f5f0c1f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 0x7f5f0c1f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 -- 172.21.15.44:0/382417703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0b1d00 con 0x7f5f0c149bb0 2024-07-17T07:15:08.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f10e00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 0x7f5f0c1f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.383+0000 7f5f0b400640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e970 tx=0x7f5ef400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.384+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400ccc0 con 0x7f5f0c149bb0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.384+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef400ce60 con 0x7f5f0c149bb0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.384+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4005260 con 0x7f5f0c149bb0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.384+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c1fb590 con 0x7f5f0c149bb0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.384+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c150720 con 0x7f5f0c149bb0 2024-07-17T07:15:08.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.385+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c0cf610 con 0x7f5f0c149bb0 2024-07-17T07:15:08.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.386+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 66) ==== 98540+0+0 (secure 0 0 0) 0x7f5ef4005400 con 0x7f5f0c149bb0 2024-07-17T07:15:08.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.387+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 7509+0+0 (secure 0 0 0) 0x7f5ef4098e60 con 0x7f5f0c149bb0 2024-07-17T07:15:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.391+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5ef4062000 con 0x7f5f0c149bb0 2024-07-17T07:15:08.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:08.649+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5f0c158b50 con 0x7f5f0c149bb0 2024-07-17T07:15:09.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.063+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 67) ==== 98667+0+0 (secure 0 0 0) 0x7f5ef405d270 con 0x7f5f0c149bb0 2024-07-17T07:15:09.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.064+0000 7f5f03e00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 0x7f5ee807f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.066+0000 7f5f0be00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 0x7f5ee807f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.076+0000 7f5f0be00640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 0x7f5ee807f610 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5efc033000 tx=0x7f5f0c0ad470 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.171+0000 7f5f03e00640 1 -- 172.21.15.44:0/382417703 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7f5ef4059640 con 0x7f5f0c149bb0 2024-07-17T07:15:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 msgr2=0x7f5ee807f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 0x7f5ee807f610 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5efc033000 tx=0x7f5f0c0ad470 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 msgr2=0x7f5f0c1514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e970 tx=0x7f5ef400ee30 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 shutdown_connections 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5ee807d220 0x7f5ee807f610 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f0c151a20 0x7f5f0c1f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f0c149bb0 0x7f5f0c1514e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 --2- 172.21.15.44:0/382417703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef0003a50 0x7f5f0c14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.175+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 >> 172.21.15.44:0/382417703 conn(0x7f5f0c06c120 msgr2=0x7f5f0c1539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.176+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 shutdown_connections 2024-07-17T07:15:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.176+0000 7f5f11800640 1 -- 172.21.15.44:0/382417703 wait complete. 2024-07-17T07:15:09.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:15:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 -- 172.21.15.44:0/1390527574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b2ad0 msgr2=0x7f55f00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 --2- 172.21.15.44:0/1390527574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b2ad0 0x7f55f00b5200 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a6a0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 -- 172.21.15.44:0/1390527574 shutdown_connections 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 --2- 172.21.15.44:0/1390527574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b5740 0x7f55f00cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 --2- 172.21.15.44:0/1390527574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b2ad0 0x7f55f00b5200 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 -- 172.21.15.44:0/1390527574 >> 172.21.15.44:0/1390527574 conn(0x7f55f00d00a0 msgr2=0x7f55f00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 -- 172.21.15.44:0/1390527574 shutdown_connections 2024-07-17T07:15:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.459+0000 7f55f7a00640 1 -- 172.21.15.44:0/1390527574 wait complete. 2024-07-17T07:15:09.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.461+0000 7f55f7a00640 1 Processor -- start 2024-07-17T07:15:09.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.462+0000 7f55f7a00640 1 -- start start 2024-07-17T07:15:09.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f00b2ad0 0x7f55f006df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 0x7f55f00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00ce760 con 0x7f55f00b5740 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00ce5e0 con 0x7f55f00b2ad0 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55f7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00ce8e0 con 0x7f55f015d600 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37900/0 (socket says 172.21.15.44:37900) 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.463+0000 7f55efe00640 1 -- 172.21.15.44:0/2136474355 learned_addr learned my addr 172.21.15.44:0/2136474355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55f5000640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f00b2ad0 0x7f55f006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55f5a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 0x7f55f00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 -- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 msgr2=0x7f55f00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 0x7f55f00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 -- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f00b2ad0 msgr2=0x7f55f006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f00b2ad0 0x7f55f006df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 -- 172.21.15.44:0/2136474355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f01f4940 con 0x7f55f00b5740 2024-07-17T07:15:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55f5a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 0x7f55f00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:09.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55efe00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009230 tx=0x7f55dc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc03f070 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55dc0047a0 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.464+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035030 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.465+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f01f4c20 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.465+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f01f51e0 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.466+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f01f5410 con 0x7f55f00b5740 2024-07-17T07:15:09.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.467+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 67) ==== 98667+0+0 (secure 0 0 0) 0x7f55dc042070 con 0x7f55f00b5740 2024-07-17T07:15:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.468+0000 7f55ecc00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 0x7f55d0078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.468+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 7509+0+0 (secure 0 0 0) 0x7f55dc0b9400 con 0x7f55f00b5740 2024-07-17T07:15:09.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.469+0000 7f55f5000640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 0x7f55d0078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:09.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.470+0000 7f55f5000640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 0x7f55d0078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55e40015e0 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.472+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55dc082430 con 0x7f55f00b5740 2024-07-17T07:15:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:09.684+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f55f00ced90 con 0x7f55f00b5740 2024-07-17T07:15:10.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.187+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 68) ==== 98875+0+0 (secure 0 0 0) 0x7f55dc0b90c0 con 0x7f55f00b5740 2024-07-17T07:15:10.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.191+0000 7f55ecc00640 1 -- 172.21.15.44:0/2136474355 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7f55dc0b3420 con 0x7f55f00b5740 2024-07-17T07:15:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:15:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.196+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 msgr2=0x7f55d0078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.196+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 0x7f55d0078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55e40015e0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2024-07-17T07:15:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.196+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 msgr2=0x7f55f00cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.196+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009230 tx=0x7f55dc0038a0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 shutdown_connections 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f55d0076a00 0x7f55d0078ec0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55e40015e0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f015d600 0x7f55f00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55f00b5740 0x7f55f00cd1c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 --2- 172.21.15.44:0/2136474355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f00b2ad0 0x7f55f006df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.197+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 >> 172.21.15.44:0/2136474355 conn(0x7f55f00d00a0 msgr2=0x7f55f015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.198+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 shutdown_connections 2024-07-17T07:15:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.198+0000 7f55f7a00640 1 -- 172.21.15.44:0/2136474355 wait complete. 2024-07-17T07:15:10.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:15:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 -- 172.21.15.44:0/1327541086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 msgr2=0x7f508c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 --2- 172.21.15.44:0/1327541086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f508c004800 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f50940016c0 tx=0x7f508c005830 comp rx=0 tx=0).stop 2024-07-17T07:15:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7400640 1 -- 172.21.15.44:0/1327541086 reap_dead start 2024-07-17T07:15:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 -- 172.21.15.44:0/1327541086 shutdown_connections 2024-07-17T07:15:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 -- 172.21.15.44:0/1327541086 >> 172.21.15.44:0/1327541086 conn(0x7f50a80b1960 msgr2=0x7f50a80b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 -- 172.21.15.44:0/1327541086 shutdown_connections 2024-07-17T07:15:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.435+0000 7f50a7e00640 1 -- 172.21.15.44:0/1327541086 wait complete. 2024-07-17T07:15:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.437+0000 7f50a7e00640 1 Processor -- start 2024-07-17T07:15:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.437+0000 7f50a7e00640 1 -- start start 2024-07-17T07:15:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 0x7f50a815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80b0a10 0x7f50a813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50a815e700 con 0x7f508c004000 2024-07-17T07:15:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50a815e580 con 0x7f50a81f7b10 2024-07-17T07:15:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50a815e880 con 0x7f50a80b0a10 2024-07-17T07:15:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.438+0000 7f50a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37916/0 (socket says 172.21.15.44:37916) 2024-07-17T07:15:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 -- 172.21.15.44:0/3131291208 learned_addr learned my addr 172.21.15.44:0/3131291208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a7400640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80b0a10 0x7f50a813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6000640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 0x7f50a815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 -- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80b0a10 msgr2=0x7f50a813d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80b0a10 0x7f50a813d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 -- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 msgr2=0x7f50a815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 0x7f50a815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 -- 172.21.15.44:0/3131291208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a815bc40 con 0x7f508c004000 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6000640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 0x7f50a815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.439+0000 7f50a6a00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f509800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.440+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980064e0 con 0x7f508c004000 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.440+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f509800b920 con 0x7f508c004000 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.440+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509800bd10 con 0x7f508c004000 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.440+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a80adb80 con 0x7f508c004000 2024-07-17T07:15:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.440+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a80ae140 con 0x7f508c004000 2024-07-17T07:15:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.441+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a80b2c60 con 0x7f508c004000 2024-07-17T07:15:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.442+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 68) ==== 98875+0+0 (secure 0 0 0) 0x7f50980035f0 con 0x7f508c004000 2024-07-17T07:15:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.443+0000 7f509ea00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 0x7f5088079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.443+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 7071+0+0 (secure 0 0 0) 0x7f5098099080 con 0x7f508c004000 2024-07-17T07:15:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.444+0000 7f50a6000640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 0x7f5088079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:10.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.445+0000 7f50a6000640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 0x7f5088079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5094032000 tx=0x7f5094001190 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.447+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5098061f10 con 0x7f508c004000 2024-07-17T07:15:10.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:10.667+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50a80728a0 con 0x7f508c004000 2024-07-17T07:15:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.195+0000 7f509ea00640 1 -- 172.21.15.44:0/3131291208 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7f5098061810 con 0x7f508c004000 2024-07-17T07:15:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 msgr2=0x7f5088079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 0x7f5088079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5094032000 tx=0x7f5094001190 comp rx=0 tx=0).stop 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 msgr2=0x7f50a8157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f509800dd80 comp rx=0 tx=0).stop 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 shutdown_connections 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5088076c40 0x7f5088079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80b0a10 0x7f50a813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a81f7b10 0x7f50a815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 --2- 172.21.15.44:0/3131291208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f508c004000 0x7f50a8157c20 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 >> 172.21.15.44:0/3131291208 conn(0x7f50a80b1960 msgr2=0x7f50a8141e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.199+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 shutdown_connections 2024-07-17T07:15:11.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.201+0000 7f50a7e00640 1 -- 172.21.15.44:0/3131291208 wait complete. 2024-07-17T07:15:11.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:15:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 -- 172.21.15.44:0/2361659548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c152220 msgr2=0x7f1c5c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 --2- 172.21.15.44:0/2361659548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c152220 0x7f1c5c06d620 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009830 tx=0x7f1c4c02a480 comp rx=0 tx=0).stop 2024-07-17T07:15:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 -- 172.21.15.44:0/2361659548 shutdown_connections 2024-07-17T07:15:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 --2- 172.21.15.44:0/2361659548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c152220 0x7f1c5c06d620 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 -- 172.21.15.44:0/2361659548 >> 172.21.15.44:0/2361659548 conn(0x7f1c5c0b0eb0 msgr2=0x7f1c5c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 -- 172.21.15.44:0/2361659548 shutdown_connections 2024-07-17T07:15:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.431+0000 7f1c63800640 1 -- 172.21.15.44:0/2361659548 wait complete. 2024-07-17T07:15:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.433+0000 7f1c63800640 1 Processor -- start 2024-07-17T07:15:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.433+0000 7f1c63800640 1 -- start start 2024-07-17T07:15:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 0x7f1c5c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 0x7f1c5c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c0cd9f0 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c15c7e0 con 0x7f1c5c152220 2024-07-17T07:15:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.434+0000 7f1c63800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5c0cdb70 con 0x7f1c5c06ed30 2024-07-17T07:15:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37942/0 (socket says 172.21.15.44:37942) 2024-07-17T07:15:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 -- 172.21.15.44:0/269430432 learned_addr learned my addr 172.21.15.44:0/269430432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c60e00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 0x7f1c5c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c61800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 0x7f1c5c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 -- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 msgr2=0x7f1c5c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 0x7f1c5c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 -- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 msgr2=0x7f1c5c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 0x7f1c5c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 -- 172.21.15.44:0/269430432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c1f25b0 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c61800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 0x7f1c5c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c60e00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 0x7f1c5c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.435+0000 7f1c5be00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c500 tx=0x7f1c4400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.436+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44017070 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.436+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c440035f0 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.436+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c1f2890 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.436+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44003a40 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.436+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c15e820 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.437+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c0ce0d0 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.438+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 68) ==== 98875+0+0 (secure 0 0 0) 0x7f1c4401a070 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.439+0000 7f1c58c00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 0x7f1c38079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.439+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7f1c44097ba0 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.440+0000 7f1c60e00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 0x7f1c38079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.441+0000 7f1c60e00640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 0x7f1c38079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c034000 tx=0x7f1c4c004730 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:11.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.443+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1c44060d50 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.653+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f1c5c0af050 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.659+0000 7f1c58c00640 1 -- 172.21.15.44:0/269430432 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7f1c44060650 con 0x7f1c5c0cbab0 2024-07-17T07:15:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.662+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 msgr2=0x7f1c38079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.662+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 0x7f1c38079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c034000 tx=0x7f1c4c004730 comp rx=0 tx=0).stop 2024-07-17T07:15:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 msgr2=0x7f1c5c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c500 tx=0x7f1c4400c9c0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 shutdown_connections 2024-07-17T07:15:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f1c38076c40 0x7f1c38079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c5c152220 0x7f1c5c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c0cbab0 0x7f1c5c0b2210 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 --2- 172.21.15.44:0/269430432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c5c06ed30 0x7f1c5c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 >> 172.21.15.44:0/269430432 conn(0x7f1c5c0b0eb0 msgr2=0x7f1c5c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 shutdown_connections 2024-07-17T07:15:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.663+0000 7f1c63800640 1 -- 172.21.15.44:0/269430432 wait complete. 2024-07-17T07:15:11.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2024-07-17T07:15:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 -- 172.21.15.44:0/36834143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c940ac3e0 msgr2=0x7f2c940ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 --2- 172.21.15.44:0/36834143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c940ac3e0 0x7f2c940ac870 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f2c94070c50 tx=0x7f2c7c02a560 comp rx=0 tx=0).stop 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 -- 172.21.15.44:0/36834143 shutdown_connections 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 --2- 172.21.15.44:0/36834143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9413ed60 0x7f2c9413f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 --2- 172.21.15.44:0/36834143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c940ac3e0 0x7f2c940ac870 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 --2- 172.21.15.44:0/36834143 >> v2:172.21.15.175:3300/0 conn(0x7f2c9406f710 0x7f2c94153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 -- 172.21.15.44:0/36834143 >> 172.21.15.44:0/36834143 conn(0x7f2c940b0fb0 msgr2=0x7f2c940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 -- 172.21.15.44:0/36834143 shutdown_connections 2024-07-17T07:15:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.890+0000 7f2c9a600640 1 -- 172.21.15.44:0/36834143 wait complete. 2024-07-17T07:15:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.892+0000 7f2c9a600640 1 Processor -- start 2024-07-17T07:15:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.892+0000 7f2c9a600640 1 -- start start 2024-07-17T07:15:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9406f710 0x7f2c941f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c940ac3e0 0x7f2c94152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940ae980 con 0x7f2c9413ed60 2024-07-17T07:15:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940ae800 con 0x7f2c940ac3e0 2024-07-17T07:15:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.893+0000 7f2c9a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9413f6b0 con 0x7f2c9406f710 2024-07-17T07:15:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37962/0 (socket says 172.21.15.44:37962) 2024-07-17T07:15:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 -- 172.21.15.44:0/1527432798 learned_addr learned my addr 172.21.15.44:0/1527432798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c8aa00640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c940ac3e0 0x7f2c94152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93400640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9406f710 0x7f2c941f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 -- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9406f710 msgr2=0x7f2c941f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9406f710 0x7f2c941f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 -- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c940ac3e0 msgr2=0x7f2c94152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c940ac3e0 0x7f2c94152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.894+0000 7f2c93e00640 1 -- 172.21.15.44:0/1527432798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c941f1c90 con 0x7f2c9413ed60 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c93e00640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b410 tx=0x7f2c8c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c015020 con 0x7f2c9413ed60 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8c011940 con 0x7f2c9413ed60 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c011d90 con 0x7f2c9413ed60 2024-07-17T07:15:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c94154260 con 0x7f2c9413ed60 2024-07-17T07:15:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.895+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c94154820 con 0x7f2c9413ed60 2024-07-17T07:15:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.896+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c940ac870 con 0x7f2c9413ed60 2024-07-17T07:15:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.897+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 68) ==== 98875+0+0 (secure 0 0 0) 0x7f2c8c018020 con 0x7f2c9413ed60 2024-07-17T07:15:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.898+0000 7f2c8be00640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.898+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7f2c8c098640 con 0x7f2c9413ed60 2024-07-17T07:15:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.899+0000 7f2c93400640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.900+0000 7f2c93400640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 0x7f2c70079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005c40 tx=0x7f2c8400a780 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:11.903+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2c8c01f020 con 0x7f2c9413ed60 2024-07-17T07:15:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.192+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) -- 0x7f2c9406e380 con 0x7f2c9413ed60 2024-07-17T07:15:12.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.198+0000 7f2c8be00640 1 -- 172.21.15.44:0/1527432798 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v773) ==== 148+0+0 (secure 0 0 0) 0x7f2c8c061660 con 0x7f2c9413ed60 2024-07-17T07:15:12.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 msgr2=0x7f2c70079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 0x7f2c70079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005c40 tx=0x7f2c8400a780 comp rx=0 tx=0).stop 2024-07-17T07:15:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 msgr2=0x7f2c9414f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b410 tx=0x7f2c8c00b8d0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 shutdown_connections 2024-07-17T07:15:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c9413ed60 0x7f2c9414f4d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c940ac3e0 0x7f2c94152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 --2- 172.21.15.44:0/1527432798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c9406f710 0x7f2c941f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 >> 172.21.15.44:0/1527432798 conn(0x7f2c940b0fb0 msgr2=0x7f2c940aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 shutdown_connections 2024-07-17T07:15:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.202+0000 7f2c9a600640 1 -- 172.21.15.44:0/1527432798 wait complete. 2024-07-17T07:15:12.222 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2024-07-17T07:15:12.222 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:15:12.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.433+0000 7f5373200640 1 -- 172.21.15.44:0/2480623845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740cf540 msgr2=0x7f53740b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.433+0000 7f5373200640 1 --2- 172.21.15.44:0/2480623845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740cf540 0x7f53740b3870 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5364009830 tx=0x7f536402a480 comp rx=0 tx=0).stop 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.433+0000 7f5373200640 1 -- 172.21.15.44:0/2480623845 shutdown_connections 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.433+0000 7f5373200640 1 --2- 172.21.15.44:0/2480623845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740cf540 0x7f53740b3870 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.433+0000 7f5373200640 1 -- 172.21.15.44:0/2480623845 >> 172.21.15.44:0/2480623845 conn(0x7f53740b0e80 msgr2=0x7f53740b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.434+0000 7f5373200640 1 -- 172.21.15.44:0/2480623845 shutdown_connections 2024-07-17T07:15:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.434+0000 7f5373200640 1 -- 172.21.15.44:0/2480623845 wait complete. 2024-07-17T07:15:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.435+0000 7f5373200640 1 Processor -- start 2024-07-17T07:15:12.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.436+0000 7f5373200640 1 -- start start 2024-07-17T07:15:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740cf540 0x7f5374156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740cf920 0x7f5374155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53740ae380 con 0x7f53740b3db0 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53740ae200 con 0x7f53740cf920 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5373200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53740ae500 con 0x7f53740cf540 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37992/0 (socket says 172.21.15.44:37992) 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 -- 172.21.15.44:0/717793454 learned_addr learned my addr 172.21.15.44:0/717793454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5369400640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740cf540 0x7f5374156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 -- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740cf540 msgr2=0x7f5374156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740cf540 0x7f5374156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5371e00640 1 -- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740cf920 msgr2=0x7f5374155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.437+0000 7f5372800640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740cf920 0x7f5374155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f5371e00640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740cf920 0x7f5374155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f5371e00640 1 -- 172.21.15.44:0/717793454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537415bf60 con 0x7f53740b3db0 2024-07-17T07:15:12.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f5371e00640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5364009950 tx=0x7f5364032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364040070 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5364005310 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364003800 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5374073c20 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.438+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53740714f0 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.440+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53740ae990 con 0x7f53740b3db0 2024-07-17T07:15:12.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.441+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 68) ==== 98875+0+0 (secure 0 0 0) 0x7f5364039040 con 0x7f53740b3db0 2024-07-17T07:15:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.442+0000 7f536aa00640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 0x7f5350079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.442+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7f53640b99a0 con 0x7f53740b3db0 2024-07-17T07:15:12.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.442+0000 7f5369400640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 0x7f5350079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.444+0000 7f5369400640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 0x7f5350079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f537413ce40 tx=0x7f535c009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.446+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f537415bf60 con 0x7f53740b3db0 2024-07-17T07:15:12.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.655+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f537413fe30 con 0x7f53740b3db0 2024-07-17T07:15:12.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.658+0000 7f536aa00640 1 -- 172.21.15.44:0/717793454 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+15266 (secure 0 0 0) 0x7f5364082940 con 0x7f53740b3db0 2024-07-17T07:15:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:12.658 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":134,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:11.193012+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6833/2442230112":"2024-07-18T07:15:08.165931+0000","172.21.15.175:6801/1035934967":"2024-07-18T07:15:08.035610+0000","172.21.15.175:6800/1035934967":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/2271459032":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/2307835811":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/1731429247":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/1970252109":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/1398139416":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/3696958724":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/232965093":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1434479146":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/1167054605":"2024-07-18T07:15:08.035610+0000","172.21.15.175:0/2886999503":"2024-07-18T07:14:30.308488+0000","172.21.15.44:6835/2442230112":"2024-07-18T07:15:08.165931+0000","172.21.15.175:6801/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:6800/1224757683":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/4116488581":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1212729465":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/347512883":"2024-07-18T07:14:30.308488+0000","172.21.15.175:0/1010255644":"2024-07-18T07:14:30.308488+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.660+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 msgr2=0x7f5350079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.660+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 0x7f5350079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f537413ce40 tx=0x7f535c009040 comp rx=0 tx=0).stop 2024-07-17T07:15:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 msgr2=0x7f5374073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5364009950 tx=0x7f5364032cd0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 shutdown_connections 2024-07-17T07:15:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f5350076c40 0x7f5350079100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53740cf920 0x7f5374155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53740cf540 0x7f5374156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 --2- 172.21.15.44:0/717793454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53740b3db0 0x7f5374073560 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 >> 172.21.15.44:0/717793454 conn(0x7f53740b0e80 msgr2=0x7f537409bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:12.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 shutdown_connections 2024-07-17T07:15:12.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.661+0000 7f5373200640 1 -- 172.21.15.44:0/717793454 wait complete. 2024-07-17T07:15:12.682 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:15:12.682 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 -- 172.21.15.44:0/3492747777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 msgr2=0x7ff688004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/3492747777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff688004c80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff690001690 tx=0x7ff688005830 comp rx=0 tx=0).stop 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 -- 172.21.15.44:0/3492747777 shutdown_connections 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/3492747777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff688004c80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 -- 172.21.15.44:0/3492747777 >> 172.21.15.44:0/3492747777 conn(0x7ff6a40b0eb0 msgr2=0x7ff6a40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 -- 172.21.15.44:0/3492747777 shutdown_connections 2024-07-17T07:15:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.890+0000 7ff6a8e00640 1 -- 172.21.15.44:0/3492747777 wait complete. 2024-07-17T07:15:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.892+0000 7ff6a8e00640 1 Processor -- start 2024-07-17T07:15:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.892+0000 7ff6a8e00640 1 -- start start 2024-07-17T07:15:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a415c180 0x7ff6a4146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 0x7ff6a4146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40b2540 con 0x7ff688004800 2024-07-17T07:15:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40b23c0 con 0x7ff6a415c600 2024-07-17T07:15:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a40ac190 con 0x7ff6a415c180 2024-07-17T07:15:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.893+0000 7ff6a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37998/0 (socket says 172.21.15.44:37998) 2024-07-17T07:15:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 -- 172.21.15.44:0/2307283324 learned_addr learned my addr 172.21.15.44:0/2307283324 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a2800640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 0x7ff6a4146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1400640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a415c180 0x7ff6a4146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 -- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a415c180 msgr2=0x7ff6a4146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a415c180 0x7ff6a4146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 -- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 msgr2=0x7ff6a4146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 0x7ff6a4146ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 -- 172.21.15.44:0/2307283324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a41472d0 con 0x7ff688004800 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a2800640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 0x7ff6a4146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.894+0000 7ff6a1e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff69400e600 tx=0x7ff69400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.895+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694015020 con 0x7ff688004800 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.895+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff69400ca90 con 0x7ff688004800 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.895+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694011460 con 0x7ff688004800 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.895+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a41475b0 con 0x7ff688004800 2024-07-17T07:15:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.895+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a41f78c0 con 0x7ff688004800 2024-07-17T07:15:12.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.896+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a415ac10 con 0x7ff688004800 2024-07-17T07:15:12.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.897+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7ff694018020 con 0x7ff688004800 2024-07-17T07:15:12.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.898+0000 7ff69a000640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 0x7ff6840791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:12.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.898+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7ff694098c40 con 0x7ff688004800 2024-07-17T07:15:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.899+0000 7ff6a1400640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 0x7ff6840791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.900+0000 7ff6a1400640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 0x7ff6840791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff690032000 tx=0x7ff690001190 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:12.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:12.903+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff69401f030 con 0x7ff688004800 2024-07-17T07:15:13.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.116+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6a406ed40 con 0x7ff688004800 2024-07-17T07:15:13.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.117+0000 7ff69a000640 1 -- 172.21.15.44:0/2307283324 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7ff694061a90 con 0x7ff688004800 2024-07-17T07:15:13.117 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 msgr2=0x7ff6840791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 0x7ff6840791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff690032000 tx=0x7ff690001190 comp rx=0 tx=0).stop 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 msgr2=0x7ff6a41f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff69400e600 tx=0x7ff69400eac0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 shutdown_connections 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff684076d00 0x7ff6840791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6a415c600 0x7ff6a4146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6a415c180 0x7ff6a4146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 --2- 172.21.15.44:0/2307283324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff688004800 0x7ff6a41f80c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 >> 172.21.15.44:0/2307283324 conn(0x7ff6a40b0eb0 msgr2=0x7ff6a406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 shutdown_connections 2024-07-17T07:15:13.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.120+0000 7ff6a8e00640 1 -- 172.21.15.44:0/2307283324 wait complete. 2024-07-17T07:15:13.139 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2024-07-17T07:15:13.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 -- 172.21.15.44:0/2211048642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 msgr2=0x7fa6e406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/2211048642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e406e060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc02b910 comp rx=0 tx=0).stop 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 -- 172.21.15.44:0/2211048642 shutdown_connections 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/2211048642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b6e0 0x7fa6e4159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/2211048642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e406e060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.351+0000 7fa6e8a00640 1 -- 172.21.15.44:0/2211048642 >> 172.21.15.44:0/2211048642 conn(0x7fa6e40b0e80 msgr2=0x7fa6e40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.352+0000 7fa6e8a00640 1 -- 172.21.15.44:0/2211048642 shutdown_connections 2024-07-17T07:15:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.352+0000 7fa6e8a00640 1 -- 172.21.15.44:0/2211048642 wait complete. 2024-07-17T07:15:13.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.353+0000 7fa6e8a00640 1 Processor -- start 2024-07-17T07:15:13.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.354+0000 7fa6e8a00640 1 -- start start 2024-07-17T07:15:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b300 0x7fa6e40713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b2120 con 0x7fa6e409b6e0 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b1fa0 con 0x7fa6e40cfa90 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e40b22a0 con 0x7fa6e409b300 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6d9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38000/0 (socket says 172.21.15.44:38000) 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6d9e00640 1 -- 172.21.15.44:0/4233221680 learned_addr learned my addr 172.21.15.44:0/4233221680 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e2800640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b300 0x7fa6e40713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.355+0000 7fa6e3200640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 -- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b300 msgr2=0x7fa6e40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b300 0x7fa6e40713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 -- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 msgr2=0x7fa6e40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 -- 172.21.15.44:0/4233221680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e41f5370 con 0x7fa6e409b6e0 2024-07-17T07:15:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6d9e00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc03e070 con 0x7fa6e409b6e0 2024-07-17T07:15:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e41f5650 con 0x7fa6e409b6e0 2024-07-17T07:15:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6cc031610 con 0x7fa6e409b6e0 2024-07-17T07:15:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc031a40 con 0x7fa6e409b6e0 2024-07-17T07:15:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.356+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e41f5c10 con 0x7fa6e409b6e0 2024-07-17T07:15:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.358+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e4159800 con 0x7fa6e409b6e0 2024-07-17T07:15:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.359+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fa6cc038070 con 0x7fa6e409b6e0 2024-07-17T07:15:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.360+0000 7fa6db400640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 0x7fa6c00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.360+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7fa6cc0b9930 con 0x7fa6e409b6e0 2024-07-17T07:15:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.361+0000 7fa6e2800640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 0x7fa6c00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.362+0000 7fa6e2800640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 0x7fa6c00791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001680 tx=0x7fa6d4009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.364+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa6cc082700 con 0x7fa6e409b6e0 2024-07-17T07:15:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:13.661+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) -- 0x7fa6e40b0680 con 0x7fa6e409b6e0 2024-07-17T07:15:14.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.204+0000 7fa6db400640 1 -- 172.21.15.44:0/4233221680 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v775) ==== 163+0+0 (secure 0 0 0) 0x7fa6cc0b7e20 con 0x7fa6e409b6e0 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 msgr2=0x7fa6c00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 0x7fa6c00791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001680 tx=0x7fa6d4009040 comp rx=0 tx=0).stop 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 msgr2=0x7fa6e40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc000c00 tx=0x7fa6cc0047a0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 shutdown_connections 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa6c0076d00 0x7fa6c00791c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6e40cfa90 0x7fa6e40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e409b6e0 0x7fa6e40cc300 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 --2- 172.21.15.44:0/4233221680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6e409b300 0x7fa6e40713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 >> 172.21.15.44:0/4233221680 conn(0x7fa6e40b0e80 msgr2=0x7fa6e40aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 shutdown_connections 2024-07-17T07:15:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.208+0000 7fa6e8a00640 1 -- 172.21.15.44:0/4233221680 wait complete. 2024-07-17T07:15:14.230 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:15:14.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9412400640 1 -- 172.21.15.44:0/2505171309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f0004b60 msgr2=0x7f93f0004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9412400640 1 --2- 172.21.15.44:0/2505171309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93f0004b60 0x7f93f0004f80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0096e0 tx=0x7f93f0006940 comp rx=0 tx=0).stop 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9411a00640 1 -- 172.21.15.44:0/2505171309 reap_dead start 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9412400640 1 -- 172.21.15.44:0/2505171309 shutdown_connections 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9412400640 1 -- 172.21.15.44:0/2505171309 >> 172.21.15.44:0/2505171309 conn(0x7f940c06c120 msgr2=0x7f940c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.441+0000 7f9412400640 1 -- 172.21.15.44:0/2505171309 shutdown_connections 2024-07-17T07:15:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.442+0000 7f9412400640 1 -- 172.21.15.44:0/2505171309 wait complete. 2024-07-17T07:15:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.443+0000 7f9412400640 1 Processor -- start 2024-07-17T07:15:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.443+0000 7f9412400640 1 -- start start 2024-07-17T07:15:14.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93f0004b60 0x7f940c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940c149d90 0x7f940c1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f940c0b3f10 con 0x7f940c1f7770 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f940c0b3d90 con 0x7f93f0004b60 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.444+0000 7f9412400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f940c06f820 con 0x7f940c149d90 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38020/0 (socket says 172.21.15.44:38020) 2024-07-17T07:15:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 -- 172.21.15.44:0/1244439930 learned_addr learned my addr 172.21.15.44:0/1244439930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f940be00640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940c149d90 0x7f940c1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411000640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93f0004b60 0x7f940c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 -- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940c149d90 msgr2=0x7f940c1f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940c149d90 0x7f940c1f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 -- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93f0004b60 msgr2=0x7f940c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93f0004b60 0x7f940c1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.445+0000 7f9411a00640 1 -- 172.21.15.44:0/1244439930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c1e2c70 con 0x7f940c1f7770 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9411a00640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f940c0b25a0 tx=0x7f940400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003970 con 0x7f940c1f7770 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9404003b10 con 0x7f940c1f7770 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c1529d0 con 0x7f940c1f7770 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003d60 con 0x7f940c1f7770 2024-07-17T07:15:14.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.446+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c14a1d0 con 0x7f940c1f7770 2024-07-17T07:15:14.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.447+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940c0add80 con 0x7f940c1f7770 2024-07-17T07:15:14.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.449+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f940401a020 con 0x7f940c1f7770 2024-07-17T07:15:14.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.450+0000 7f9408c00640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 0x7f93ec079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.450+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7f9404097f40 con 0x7f940c1f7770 2024-07-17T07:15:14.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.450+0000 7f9411000640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 0x7f93ec079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.452+0000 7f9411000640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 0x7f93ec079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93fc037000 tx=0x7f93fc0046c0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:14.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.454+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9404060f40 con 0x7f940c1f7770 2024-07-17T07:15:14.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.669+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f940c0b1d50 con 0x7f940c1f7770 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.670+0000 7f9408c00640 1 -- 172.21.15.44:0/1244439930 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+1998 (secure 0 0 0) 0x7f9404069180 con 0x7f940c1f7770 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:15:14.670 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:15:14.671 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:15:14.672 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:15:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 msgr2=0x7f93ec079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 0x7f93ec079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93fc037000 tx=0x7f93fc0046c0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 msgr2=0x7f940c1f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f940c0b25a0 tx=0x7f940400dc60 comp rx=0 tx=0).stop 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 shutdown_connections 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f93ec076cd0 0x7f93ec079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f940c1f7770 0x7f940c1f7bb0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f940c149d90 0x7f940c1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 --2- 172.21.15.44:0/1244439930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93f0004b60 0x7f940c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.673+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 >> 172.21.15.44:0/1244439930 conn(0x7f940c06c120 msgr2=0x7f940c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.674+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 shutdown_connections 2024-07-17T07:15:14.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.674+0000 7f9412400640 1 -- 172.21.15.44:0/1244439930 wait complete. 2024-07-17T07:15:14.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.911+0000 7fec11c00640 1 -- 172.21.15.44:0/3293287963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0005230 msgr2=0x7febf000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.911+0000 7fec11c00640 1 --2- 172.21.15.44:0/3293287963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0005230 0x7febf000b9c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fec04001690 tx=0x7febf0005830 comp rx=0 tx=0).stop 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.912+0000 7fec11c00640 1 -- 172.21.15.44:0/3293287963 shutdown_connections 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.912+0000 7fec11c00640 1 --2- 172.21.15.44:0/3293287963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febf0005230 0x7febf000b9c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.912+0000 7fec11c00640 1 -- 172.21.15.44:0/3293287963 >> 172.21.15.44:0/3293287963 conn(0x7fec0c0cd490 msgr2=0x7fec0c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.912+0000 7fec11c00640 1 -- 172.21.15.44:0/3293287963 shutdown_connections 2024-07-17T07:15:14.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.912+0000 7fec11c00640 1 -- 172.21.15.44:0/3293287963 wait complete. 2024-07-17T07:15:14.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.914+0000 7fec11c00640 1 Processor -- start 2024-07-17T07:15:14.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.914+0000 7fec11c00640 1 -- start start 2024-07-17T07:15:14.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febf0005230 0x7fec0c1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0c0c4ac0 0x7fec0c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c15e680 con 0x7fec0c0c6320 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c0aedd0 con 0x7fec0c0c4ac0 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c0c5e90 con 0x7febf0005230 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec0b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38046/0 (socket says 172.21.15.44:38046) 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec0b400640 1 -- 172.21.15.44:0/207927323 learned_addr learned my addr 172.21.15.44:0/207927323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec0be00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febf0005230 0x7fec0c1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.915+0000 7fec11200640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0c0c4ac0 0x7fec0c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 -- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febf0005230 msgr2=0x7fec0c1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febf0005230 0x7fec0c1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 -- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0c0c4ac0 msgr2=0x7fec0c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0c0c4ac0 0x7fec0c0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 -- 172.21.15.44:0/207927323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c0c67a0 con 0x7fec0c0c6320 2024-07-17T07:15:14.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec0b400640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7febf800dc90 tx=0x7febf800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8003970 con 0x7fec0c0c6320 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf8003c00 con 0x7fec0c0c6320 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c0c5280 con 0x7fec0c0c6320 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8006110 con 0x7fec0c0c6320 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.916+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c0c5840 con 0x7fec0c0c6320 2024-07-17T07:15:14.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.918+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c071bb0 con 0x7fec0c0c6320 2024-07-17T07:15:14.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.919+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7febf80062b0 con 0x7fec0c0c6320 2024-07-17T07:15:14.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.920+0000 7fec03e00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 0x7febec0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:14.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.920+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7febf8099180 con 0x7fec0c0c6320 2024-07-17T07:15:14.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.920+0000 7fec0be00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 0x7febec0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:14.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.921+0000 7fec0be00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 0x7febec0791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7febfc009aa0 tx=0x7febfc005f90 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:14.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:14.924+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7febf8062180 con 0x7fec0c0c6320 2024-07-17T07:15:15.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.137+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fec0c1438c0 con 0x7fec0c0c6320 2024-07-17T07:15:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.144+0000 7fec03e00640 1 -- 172.21.15.44:0/207927323 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7febf8061a80 con 0x7fec0c0c6320 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 msgr2=0x7febec0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 0x7febec0791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7febfc009aa0 tx=0x7febfc005f90 comp rx=0 tx=0).stop 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 msgr2=0x7fec0c0cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7febf800dc90 tx=0x7febf800b620 comp rx=0 tx=0).stop 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 shutdown_connections 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7febec076d00 0x7febec0791c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec0c0c4ac0 0x7fec0c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec0c0c6320 0x7fec0c0cb400 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 --2- 172.21.15.44:0/207927323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febf0005230 0x7fec0c1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 >> 172.21.15.44:0/207927323 conn(0x7fec0c0cd490 msgr2=0x7fec0c0b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 shutdown_connections 2024-07-17T07:15:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.147+0000 7fec11c00640 1 -- 172.21.15.44:0/207927323 wait complete. 2024-07-17T07:15:15.167 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:15:15.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:15.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:15.404 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:15:15.405 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:15.405 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:15.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:15.463 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:15.463 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:15.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:15.518 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:15.518 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:15.574 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:15.574 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:15.574 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:15.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:15.629 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:15:15.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.849+0000 7fbdaa400640 1 -- 172.21.15.44:0/2647391125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd88004d40 msgr2=0x7fbd880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.849+0000 7fbdaa400640 1 --2- 172.21.15.44:0/2647391125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd88004d40 0x7fbd880051c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c001dc0 tx=0x7fbd88005830 comp rx=0 tx=0).stop 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.849+0000 7fbda3e00640 1 -- 172.21.15.44:0/2647391125 reap_dead start 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.849+0000 7fbdaa400640 1 -- 172.21.15.44:0/2647391125 shutdown_connections 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.849+0000 7fbdaa400640 1 -- 172.21.15.44:0/2647391125 >> 172.21.15.44:0/2647391125 conn(0x7fbda40d00a0 msgr2=0x7fbda40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.850+0000 7fbdaa400640 1 -- 172.21.15.44:0/2647391125 shutdown_connections 2024-07-17T07:15:15.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.850+0000 7fbdaa400640 1 -- 172.21.15.44:0/2647391125 wait complete. 2024-07-17T07:15:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.851+0000 7fbdaa400640 1 Processor -- start 2024-07-17T07:15:15.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.851+0000 7fbdaa400640 1 -- start start 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd88004d40 0x7fbda41f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda4157210 0x7fbda41f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40ce400 con 0x7fbda41f7a20 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40ce580 con 0x7fbda4157210 2024-07-17T07:15:15.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.852+0000 7fbdaa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbda40ce700 con 0x7fbd88004d40 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38054/0 (socket says 172.21.15.44:38054) 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 -- 172.21.15.44:0/1916009249 learned_addr learned my addr 172.21.15.44:0/1916009249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd88004d40 0x7fbda41f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 -- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd88004d40 msgr2=0x7fbda41f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd88004d40 0x7fbda41f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 -- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda4157210 msgr2=0x7fbda41f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda4157210 0x7fbda41f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:15.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 -- 172.21.15.44:0/1916009249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda41f3ef0 con 0x7fbda41f7a20 2024-07-17T07:15:15.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.853+0000 7fbda3e00640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c02acc0 tx=0x7fbd9c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:15.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.854+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0062b0 con 0x7fbda41f7a20 2024-07-17T07:15:15.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.854+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd9c006450 con 0x7fbda41f7a20 2024-07-17T07:15:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.854+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0068a0 con 0x7fbda41f7a20 2024-07-17T07:15:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.854+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda41f7f30 con 0x7fbda41f7a20 2024-07-17T07:15:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.854+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda41f41b0 con 0x7fbda41f7a20 2024-07-17T07:15:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.855+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda40cf320 con 0x7fbda41f7a20 2024-07-17T07:15:15.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.856+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fbd9c003a30 con 0x7fbda41f7a20 2024-07-17T07:15:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.857+0000 7fbd9b400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 0x7fbd80079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:15.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.858+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7fbd9c0b8410 con 0x7fbda41f7a20 2024-07-17T07:15:15.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.858+0000 7fbda3400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 0x7fbd80079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:15.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.859+0000 7fbda3400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 0x7fbd80079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000fb0 tx=0x7fbd940015a0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:15.862+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbda40cf320 con 0x7fbda41f7a20 2024-07-17T07:15:16.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.118+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbda40cec20 con 0x7fbda41f7a20 2024-07-17T07:15:16.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.119+0000 7fbd9b400640 1 -- 172.21.15.44:0/1916009249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+3914 (secure 0 0 0) 0x7fbd9c081400 con 0x7fbda41f7a20 2024-07-17T07:15:16.119 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:16.119 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":63,"btime":"2024-07-17T07:15:09:169246+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1203907048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1203907048},{"type":"v1","addr":"172.21.15.175:6836","nonce":1203907048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2853776743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2853776743},{"type":"v1","addr":"172.21.15.44:6834","nonce":2853776743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/899832647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":899832647},{"type":"v1","addr":"172.21.15.175:6837","nonce":899832647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3326601782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3326601782},{"type":"v1","addr":"172.21.15.44:6835","nonce":3326601782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-07-17T07:15:16.120 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 63 2024-07-17T07:15:16.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 msgr2=0x7fbd80079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:16.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 0x7fbd80079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd94000fb0 tx=0x7fbd940015a0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 msgr2=0x7fbda415abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:16.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c02acc0 tx=0x7fbd9c001b10 comp rx=0 tx=0).stop 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 shutdown_connections 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbd80076cd0 0x7fbd80079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbda41f7a20 0x7fbda415abf0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda4157210 0x7fbda41f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 --2- 172.21.15.44:0/1916009249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd88004d40 0x7fbda41f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.122+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 >> 172.21.15.44:0/1916009249 conn(0x7fbda40d00a0 msgr2=0x7fbda415bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:16.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.123+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 shutdown_connections 2024-07-17T07:15:16.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.123+0000 7fbdaa400640 1 -- 172.21.15.44:0/1916009249 wait complete. 2024-07-17T07:15:16.143 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:15:16.143 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:15:16.144 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:15:16.144 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:16.144 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:15:16.144 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:15:16.144 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:15:16.145 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:16.145 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:15:16.145 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:15:16.146 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:15:16.146 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:16.146 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:15:16.146 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:15:16.147 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:15:16.147 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:16.164 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:15:16.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:15:16.168 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:15:16.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:15:16.169 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:15:16.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:15:16.172 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:15:16.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:15:16.215 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:15:16.216 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:16.220 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:15:16.221 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:16.221 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:15:16.221 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:16.227 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:15:16.227 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:16.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:15:16.365 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:15:16.365 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:15:16.374 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:15:16.375 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:15:16.376 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T07:15:16.376+0000 7f9d27000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 -- 172.21.15.44:0/2381242847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494006ed30 msgr2=0x7f494015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 --2- 172.21.15.44:0/2381242847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494006ed30 0x7f494015c2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4938005fd0 tx=0x7f493802a7a0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 -- 172.21.15.44:0/2381242847 shutdown_connections 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 --2- 172.21.15.44:0/2381242847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f494006ed30 0x7f494015c2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 --2- 172.21.15.44:0/2381242847 >> v2:172.21.15.175:3300/0 conn(0x7f49400cbab0 0x7f494006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 -- 172.21.15.44:0/2381242847 >> 172.21.15.44:0/2381242847 conn(0x7f49400b0eb0 msgr2=0x7f49400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:16.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 -- 172.21.15.44:0/2381242847 shutdown_connections 2024-07-17T07:15:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.413+0000 7f4947600640 1 -- 172.21.15.44:0/2381242847 wait complete. 2024-07-17T07:15:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.415+0000 7f4947600640 1 Processor -- start 2024-07-17T07:15:16.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.415+0000 7f4947600640 1 -- start start 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49400cbab0 0x7f49401505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940152220 0x7f494015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49400cd7b0 con 0x7f494006ed30 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49400cd630 con 0x7f4940152220 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4947600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49400cd930 con 0x7f49400cbab0 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38104/0 (socket says 172.21.15.44:38104) 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 -- 172.21.15.44:0/211358099 learned_addr learned my addr 172.21.15.44:0/211358099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:16.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4945600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940152220 0x7f494015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:16.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f493fe00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49400cbab0 0x7f49401505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:16.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 -- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49400cbab0 msgr2=0x7f49401505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49400cbab0 0x7f49401505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.416+0000 7f4944c00640 1 -- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940152220 msgr2=0x7f494015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f4944c00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940152220 0x7f494015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f4944c00640 1 -- 172.21.15.44:0/211358099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49401f4c10 con 0x7f494006ed30 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f4944c00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f493400c530 tx=0x7f493400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934016020 con 0x7f494006ed30 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4934012940 con 0x7f494006ed30 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49401f3f90 con 0x7f494006ed30 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934012d90 con 0x7f494006ed30 2024-07-17T07:15:16.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.417+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401f4550 con 0x7f494006ed30 2024-07-17T07:15:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.418+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49400cde50 con 0x7f494006ed30 2024-07-17T07:15:16.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.420+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f4934019020 con 0x7f494006ed30 2024-07-17T07:15:16.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.421+0000 7f493cc00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 0x7f4920079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:16.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.421+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6657+0+0 (secure 0 0 0) 0x7f4934098790 con 0x7f494006ed30 2024-07-17T07:15:16.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.422+0000 7f493fe00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 0x7f4920079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:16.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.423+0000 7f493fe00640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 0x7f4920079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f492c0095f0 tx=0x7f492c005f90 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:16.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.426+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4934020020 con 0x7f494006ed30 2024-07-17T07:15:16.473 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:15:16.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:15:16.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:16.649+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f49401f48f0 con 0x7f494006ed30 2024-07-17T07:15:17.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.245+0000 7f493cc00640 1 -- 172.21.15.44:0/211358099 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7f4934061130 con 0x7f494006ed30 2024-07-17T07:15:17.245 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 msgr2=0x7f4920079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 0x7f4920079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f492c0095f0 tx=0x7f492c005f90 comp rx=0 tx=0).stop 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 msgr2=0x7f49400ac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f493400c530 tx=0x7f493400c9f0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 shutdown_connections 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f4920076cd0 0x7f4920079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4940152220 0x7f494015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49400cbab0 0x7f49401505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 --2- 172.21.15.44:0/211358099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f494006ed30 0x7f49400ac7a0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 >> 172.21.15.44:0/211358099 conn(0x7f49400b0eb0 msgr2=0x7f49400b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.249+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 shutdown_connections 2024-07-17T07:15:17.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.250+0000 7f4947600640 1 -- 172.21.15.44:0/211358099 wait complete. 2024-07-17T07:15:17.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.464+0000 7fcecac00640 1 -- 172.21.15.44:0/1979953148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec41594b0 msgr2=0x7fcec40abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.464+0000 7fcecac00640 1 --2- 172.21.15.44:0/1979953148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec41594b0 0x7fcec40abf50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fcec409cdf0 tx=0x7fceb402a5f0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 -- 172.21.15.44:0/1979953148 shutdown_connections 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 --2- 172.21.15.44:0/1979953148 >> v2:172.21.15.175:3300/0 conn(0x7fcec4159890 0x7fcec409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 --2- 172.21.15.44:0/1979953148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec41594b0 0x7fcec40abf50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 -- 172.21.15.44:0/1979953148 >> 172.21.15.44:0/1979953148 conn(0x7fcec40b0e80 msgr2=0x7fcec40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 -- 172.21.15.44:0/1979953148 shutdown_connections 2024-07-17T07:15:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.465+0000 7fcecac00640 1 -- 172.21.15.44:0/1979953148 wait complete. 2024-07-17T07:15:17.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.466+0000 7fcecac00640 1 Processor -- start 2024-07-17T07:15:17.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.466+0000 7fcecac00640 1 -- start start 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec409b5f0 0x7fcec4140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec41594b0 0x7fcec41f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcec406ec10 con 0x7fcec4159890 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcec406ea90 con 0x7fcec41594b0 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.467+0000 7fcecac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcec406ed90 con 0x7fcec409b5f0 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38120/0 (socket says 172.21.15.44:38120) 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 -- 172.21.15.44:0/126259892 learned_addr learned my addr 172.21.15.44:0/126259892 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fcec9800640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec409b5f0 0x7fcec4140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fcec8e00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec41594b0 0x7fcec41f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 -- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec409b5f0 msgr2=0x7fcec4140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec409b5f0 0x7fcec4140670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 -- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec41594b0 msgr2=0x7fcec41f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec41594b0 0x7fcec41f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 -- 172.21.15.44:0/126259892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec40c8450 con 0x7fcec4159890 2024-07-17T07:15:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.468+0000 7fceca200640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c7a0 tx=0x7fcebc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.469+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc003b70 con 0x7fcec4159890 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.469+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcebc003d10 con 0x7fcec4159890 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.469+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00aab0 con 0x7fcec4159890 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.469+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec40c8730 con 0x7fcec4159890 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.469+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec40726f0 con 0x7fcec4159890 2024-07-17T07:15:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.470+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec406f240 con 0x7fcec4159890 2024-07-17T07:15:17.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.471+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fcebc00ac50 con 0x7fcec4159890 2024-07-17T07:15:17.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.473+0000 7fcec1600640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 0x7fcea8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.473+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5797+0+0 (secure 0 0 0) 0x7fcebc098380 con 0x7fcec4159890 2024-07-17T07:15:17.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.473+0000 7fcec9800640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 0x7fcea8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.474+0000 7fcec9800640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 0x7fcea8079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fceb4033000 tx=0x7fceb4004730 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:17.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.476+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcebc061770 con 0x7fcec4159890 2024-07-17T07:15:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.752+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcec40b2f60 con 0x7fcec4159890 2024-07-17T07:15:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.752+0000 7fcec1600640 1 -- 172.21.15.44:0/126259892 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) ==== 80+0+5304 (secure 0 0 0) 0x7fcebc061070 con 0x7fcec4159890 2024-07-17T07:15:17.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:17.752 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.754 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:15:17.755 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.758 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:15:17.758 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:15:17.761 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:15:17.763 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:15:17.765 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:15:17.766 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.767 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:15:17.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:17.769 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:15:17.770 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:15:17.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 msgr2=0x7fcea8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 0x7fcea8079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fceb4033000 tx=0x7fceb4004730 comp rx=0 tx=0).stop 2024-07-17T07:15:17.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 msgr2=0x7fcec41f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c7a0 tx=0x7fcebc00cc60 comp rx=0 tx=0).stop 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 shutdown_connections 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcea8076cd0 0x7fcea8079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec4159890 0x7fcec41f4530 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec41594b0 0x7fcec41f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 --2- 172.21.15.44:0/126259892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec409b5f0 0x7fcec4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 >> 172.21.15.44:0/126259892 conn(0x7fcec40b0e80 msgr2=0x7fcec415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 shutdown_connections 2024-07-17T07:15:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.755+0000 7fcecac00640 1 -- 172.21.15.44:0/126259892 wait complete. 2024-07-17T07:15:17.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:15:17.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 -- 172.21.15.44:0/3996148587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 msgr2=0x7f2b3c0ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 --2- 172.21.15.44:0/3996148587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0ade20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c06f770 tx=0x7f2b3402aa10 comp rx=0 tx=0).stop 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 -- 172.21.15.44:0/3996148587 shutdown_connections 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 --2- 172.21.15.44:0/3996148587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0ade20 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 --2- 172.21.15.44:0/3996148587 >> v2:172.21.15.175:3300/0 conn(0x7f2b3c0b3db0 0x7f2b3c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 --2- 172.21.15.44:0/3996148587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 0x7f2b3c0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 -- 172.21.15.44:0/3996148587 >> 172.21.15.44:0/3996148587 conn(0x7f2b3c0b0e80 msgr2=0x7f2b3c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 -- 172.21.15.44:0/3996148587 shutdown_connections 2024-07-17T07:15:17.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.982+0000 7f2b3be00640 1 -- 172.21.15.44:0/3996148587 wait complete. 2024-07-17T07:15:17.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.984+0000 7f2b3be00640 1 Processor -- start 2024-07-17T07:15:17.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.984+0000 7f2b3be00640 1 -- start start 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b3c0b3db0 0x7f2b3c1f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 0x7f2b3c072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0ae6f0 con 0x7f2b3c0cf920 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0ae570 con 0x7f2b3c0b3db0 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c0ae870 con 0x7f2b3c0cf540 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38142/0 (socket says 172.21.15.44:38142) 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.985+0000 7f2b3b400640 1 -- 172.21.15.44:0/2856420645 learned_addr learned my addr 172.21.15.44:0/2856420645 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3a000640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 0x7f2b3c072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3aa00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b3c0b3db0 0x7f2b3c1f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 -- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 msgr2=0x7f2b3c072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 0x7f2b3c072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 -- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b3c0b3db0 msgr2=0x7f2b3c1f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b3c0b3db0 0x7f2b3c1f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 -- 172.21.15.44:0/2856420645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1f7bf0 con 0x7f2b3c0cf920 2024-07-17T07:15:17.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b3b400640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009d60 tx=0x7f2b34003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34033050 con 0x7f2b3c0cf920 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.986+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b34004450 con 0x7f2b3c0cf920 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.987+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34004860 con 0x7f2b3c0cf920 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.987+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c071b50 con 0x7f2b3c0cf920 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.987+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c0723f0 con 0x7f2b3c0cf920 2024-07-17T07:15:17.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.988+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c0b3870 con 0x7f2b3c0cf920 2024-07-17T07:15:17.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.989+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f2b34036070 con 0x7f2b3c0cf920 2024-07-17T07:15:17.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.990+0000 7f2b32a00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 0x7f2b1c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:17.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.990+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5797+0+0 (secure 0 0 0) 0x7f2b340b9870 con 0x7f2b3c0cf920 2024-07-17T07:15:17.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.991+0000 7f2b3aa00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 0x7f2b1c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:17.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.992+0000 7f2b3aa00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 0x7f2b1c0791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005fd0 tx=0x7f2b2c00b510 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:17.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:17.994+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2b3c1f7bf0 con 0x7f2b3c0cf920 2024-07-17T07:15:18.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.198+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2b3c13f5a0 con 0x7f2b3c0cf920 2024-07-17T07:15:18.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.199+0000 7f2b32a00640 1 -- 172.21.15.44:0/2856420645 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7f2b34082ab0 con 0x7f2b3c0cf920 2024-07-17T07:15:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:18.199 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:15:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:15:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:15:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:18.200 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.201+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 msgr2=0x7f2b1c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.201+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 0x7f2b1c0791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005fd0 tx=0x7f2b2c00b510 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.201+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 msgr2=0x7f2b3c0cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.201+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009d60 tx=0x7f2b34003f10 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 shutdown_connections 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2b1c076d00 0x7f2b1c0791c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b3c0cf920 0x7f2b3c0cdd60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b3c0cf540 0x7f2b3c072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 --2- 172.21.15.44:0/2856420645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b3c0b3db0 0x7f2b3c1f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 >> 172.21.15.44:0/2856420645 conn(0x7f2b3c0b0e80 msgr2=0x7f2b3c06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 shutdown_connections 2024-07-17T07:15:18.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.202+0000 7f2b3be00640 1 -- 172.21.15.44:0/2856420645 wait complete. 2024-07-17T07:15:18.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:15:18.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:15:18.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.429+0000 7f82b7e00640 1 -- 172.21.15.44:0/429861397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8298005210 msgr2=0x7f829800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.429+0000 7f82b7e00640 1 --2- 172.21.15.44:0/429861397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8298005210 0x7f829800b9c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f82b80ad9e0 tx=0x7f8298005830 comp rx=0 tx=0).stop 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.430+0000 7f82b7e00640 1 -- 172.21.15.44:0/429861397 shutdown_connections 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.430+0000 7f82b7e00640 1 --2- 172.21.15.44:0/429861397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8298005210 0x7f829800b9c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.430+0000 7f82b7e00640 1 -- 172.21.15.44:0/429861397 >> 172.21.15.44:0/429861397 conn(0x7f82b80cf050 msgr2=0x7f82b80cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.430+0000 7f82b7e00640 1 -- 172.21.15.44:0/429861397 shutdown_connections 2024-07-17T07:15:18.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.430+0000 7f82b7e00640 1 -- 172.21.15.44:0/429861397 wait complete. 2024-07-17T07:15:18.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.431+0000 7f82b7e00640 1 Processor -- start 2024-07-17T07:15:18.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.432+0000 7f82b7e00640 1 -- start start 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.432+0000 7f82b7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 0x7f82b814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.432+0000 7f82b7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b814f180 0x7f82b81f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80ae4a0 con 0x7f82b81f4f50 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80ae320 con 0x7f8298005210 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82b80ae620 con 0x7f82b814f180 2024-07-17T07:15:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38166/0 (socket says 172.21.15.44:38166) 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 -- 172.21.15.44:0/1243273186 learned_addr learned my addr 172.21.15.44:0/1243273186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b6a00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 0x7f82b814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b6000640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b814f180 0x7f82b81f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 -- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b814f180 msgr2=0x7f82b81f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b814f180 0x7f82b81f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 -- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 msgr2=0x7f82b814e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 0x7f82b814e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b7400640 1 -- 172.21.15.44:0/1243273186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b81f3e90 con 0x7f82b81f4f50 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.433+0000 7f82b6a00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 0x7f82b814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82b7400640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f82a802afa0 tx=0x7f82a8032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a800b6f0 con 0x7f82b81f4f50 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a803d550 con 0x7f82b81f4f50 2024-07-17T07:15:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b81f4170 con 0x7f82b81f4f50 2024-07-17T07:15:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a803d9a0 con 0x7f82b81f4f50 2024-07-17T07:15:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.434+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b81f4730 con 0x7f82b81f4f50 2024-07-17T07:15:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.435+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b8153a50 con 0x7f82b81f4f50 2024-07-17T07:15:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.436+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f82a8045020 con 0x7f82b81f4f50 2024-07-17T07:15:18.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.437+0000 7f82aea00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 0x7f8294079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:18.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.437+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5797+0+0 (secure 0 0 0) 0x7f82a80b8d90 con 0x7f82b81f4f50 2024-07-17T07:15:18.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.438+0000 7f82b6a00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 0x7f8294079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:18.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.439+0000 7f82b6a00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 0x7f8294079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82b0003e30 tx=0x7f82b0009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.441+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f82a804c020 con 0x7f82b81f4f50 2024-07-17T07:15:18.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:18.648+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f82b81e9960 con 0x7f82b81f4f50 2024-07-17T07:15:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.279+0000 7f82aea00640 1 -- 172.21.15.44:0/1243273186 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7f82a8081fd0 con 0x7f82b81f4f50 2024-07-17T07:15:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:15:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 msgr2=0x7f8294079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 0x7f8294079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82b0003e30 tx=0x7f82b0009040 comp rx=0 tx=0).stop 2024-07-17T07:15:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 msgr2=0x7f82b80b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f82a802afa0 tx=0x7f82a8032cc0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 shutdown_connections 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f8294076cd0 0x7f8294079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82b81f4f50 0x7f82b80b2ab0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82b814f180 0x7f82b81f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 --2- 172.21.15.44:0/1243273186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8298005210 0x7f82b814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.283+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 >> 172.21.15.44:0/1243273186 conn(0x7f82b80cf050 msgr2=0x7f82b80abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.284+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 shutdown_connections 2024-07-17T07:15:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.284+0000 7f82b7e00640 1 -- 172.21.15.44:0/1243273186 wait complete. 2024-07-17T07:15:19.313 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 -- 172.21.15.44:0/771368949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080ceae0 msgr2=0x7fc9080b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 --2- 172.21.15.44:0/771368949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080ceae0 0x7fc9080b6c60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc90808c4c0 tx=0x7fc90002a4d0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 -- 172.21.15.44:0/771368949 shutdown_connections 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 --2- 172.21.15.44:0/771368949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080ceae0 0x7fc9080b6c60 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 --2- 172.21.15.44:0/771368949 >> v2:172.21.15.175:3300/0 conn(0x7fc9080d0750 0x7fc9080b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 -- 172.21.15.44:0/771368949 >> 172.21.15.44:0/771368949 conn(0x7fc90806c830 msgr2=0x7fc90806cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.525+0000 7fc90ea00640 1 -- 172.21.15.44:0/771368949 shutdown_connections 2024-07-17T07:15:19.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.526+0000 7fc90ea00640 1 -- 172.21.15.44:0/771368949 wait complete. 2024-07-17T07:15:19.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.527+0000 7fc90ea00640 1 Processor -- start 2024-07-17T07:15:19.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.527+0000 7fc90ea00640 1 -- start start 2024-07-17T07:15:19.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 0x7fc9080cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:19.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 0x7fc9080b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b2510 con 0x7fc9080d0750 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b2390 con 0x7fc9081611d0 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.528+0000 7fc90ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080b2690 con 0x7fc9080ceae0 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39048/0 (socket says 172.21.15.44:39048) 2024-07-17T07:15:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 -- 172.21.15.44:0/3092250303 learned_addr learned my addr 172.21.15.44:0/3092250303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:19.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 -- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 msgr2=0x7fc9080cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:19.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc90ca00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 0x7fc9080b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907e00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 0x7fc9080cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 0x7fc9080cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 -- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 msgr2=0x7fc9080b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 0x7fc9080b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907400640 1 -- 172.21.15.44:0/3092250303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080aaa60 con 0x7fc9080d0750 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc907e00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 0x7fc9080cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.529+0000 7fc90ca00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 0x7fc9080b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc907400640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d980 tx=0x7fc8f400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4017070 con 0x7fc9080d0750 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8f4009d20 con 0x7fc9080d0750 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40106c0 con 0x7fc9080d0750 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9080b1f10 con 0x7fc9080d0750 2024-07-17T07:15:19.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.530+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9080aafe0 con 0x7fc9080d0750 2024-07-17T07:15:19.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.531+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9080b7440 con 0x7fc9080d0750 2024-07-17T07:15:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.533+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fc8f4010860 con 0x7fc9080d0750 2024-07-17T07:15:19.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.535+0000 7fc8ffe00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 0x7fc8e8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:19.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.535+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6167+0+0 (secure 0 0 0) 0x7fc8f4098f80 con 0x7fc9080d0750 2024-07-17T07:15:19.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.536+0000 7fc907e00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 0x7fc8e8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:19.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.537+0000 7fc907e00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 0x7fc8e8079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007e90 tx=0x7fc8f80073a0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:19.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.539+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc8f40620d0 con 0x7fc9080d0750 2024-07-17T07:15:19.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:19.748+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9080b95b0 con 0x7fc9080d0750 2024-07-17T07:15:20.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.286+0000 7fc8ffe00640 1 -- 172.21.15.44:0/3092250303 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7fc8f40619d0 con 0x7fc9080d0750 2024-07-17T07:15:20.286 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:15:20.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.289+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 msgr2=0x7fc8e8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.289+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 0x7fc8e8079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007e90 tx=0x7fc8f80073a0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 msgr2=0x7fc9080cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d980 tx=0x7fc8f400de40 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 shutdown_connections 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fc8e8076cd0 0x7fc8e8079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9081611d0 0x7fc9080b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9080d0750 0x7fc9080cdcd0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 --2- 172.21.15.44:0/3092250303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9080ceae0 0x7fc9080cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 >> 172.21.15.44:0/3092250303 conn(0x7fc90806c830 msgr2=0x7fc9080b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:20.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 shutdown_connections 2024-07-17T07:15:20.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.290+0000 7fc90ea00640 1 -- 172.21.15.44:0/3092250303 wait complete. 2024-07-17T07:15:20.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:15:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.509+0000 7fa2c3400640 1 -- 172.21.15.44:0/3737981550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 msgr2=0x7fa2bc0b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.509+0000 7fa2c3400640 1 --2- 172.21.15.44:0/3737981550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc0b3ce0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0b43f0 tx=0x7fa2b002a540 comp rx=0 tx=0).stop 2024-07-17T07:15:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 -- 172.21.15.44:0/3737981550 shutdown_connections 2024-07-17T07:15:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 --2- 172.21.15.44:0/3737981550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06db60 0x7fa2bc0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 --2- 172.21.15.44:0/3737981550 >> v2:172.21.15.175:3300/0 conn(0x7fa2bc161120 0x7fa2bc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 --2- 172.21.15.44:0/3737981550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc0b3ce0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 -- 172.21.15.44:0/3737981550 >> 172.21.15.44:0/3737981550 conn(0x7fa2bc0b0eb0 msgr2=0x7fa2bc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 -- 172.21.15.44:0/3737981550 shutdown_connections 2024-07-17T07:15:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.510+0000 7fa2c3400640 1 -- 172.21.15.44:0/3737981550 wait complete. 2024-07-17T07:15:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.512+0000 7fa2c3400640 1 Processor -- start 2024-07-17T07:15:20.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.512+0000 7fa2c3400640 1 -- start start 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 0x7fa2bc1f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc0736c0 0x7fa2bc159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc0aca60 con 0x7fa2bc154600 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc0ac8e0 con 0x7fa2bc161120 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc15e0a0 con 0x7fa2bc0736c0 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.513+0000 7fa2c0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39066/0 (socket says 172.21.15.44:39066) 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 -- 172.21.15.44:0/2437782993 learned_addr learned my addr 172.21.15.44:0/2437782993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 0x7fa2bc1f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c1400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc0736c0 0x7fa2bc159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 -- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc0736c0 msgr2=0x7fa2bc159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc0736c0 0x7fa2bc159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 -- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 msgr2=0x7fa2bc1f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 0x7fa2bc1f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 -- 172.21.15.44:0/2437782993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc1f4f80 con 0x7fa2bc154600 2024-07-17T07:15:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 0x7fa2bc1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.514+0000 7fa2c0a00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005fd0 tx=0x7fa2b00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.515+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b003f070 con 0x7fa2bc154600 2024-07-17T07:15:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.515+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2b0004890 con 0x7fa2bc154600 2024-07-17T07:15:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.515+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc073ba0 con 0x7fa2bc154600 2024-07-17T07:15:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.515+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0035030 con 0x7fa2bc154600 2024-07-17T07:15:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.515+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc159250 con 0x7fa2bc154600 2024-07-17T07:15:20.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.516+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc0b3ce0 con 0x7fa2bc154600 2024-07-17T07:15:20.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.519+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fa2b0042070 con 0x7fa2bc154600 2024-07-17T07:15:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.519+0000 7fa2b8c00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 0x7fa29c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.520+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6549+0+0 (secure 0 0 0) 0x7fa2b00b9ab0 con 0x7fa2bc154600 2024-07-17T07:15:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.520+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 0x7fa29c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:20.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.521+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 0x7fa29c079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc158110 tx=0x7fa2a8005cf0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:20.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.523+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa2bc1f4f80 con 0x7fa2bc154600 2024-07-17T07:15:20.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:20.774+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2bc0734a0 con 0x7fa2bc154600 2024-07-17T07:15:21.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.334+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2437782993 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v66) ==== 174+0+0 (secure 0 0 0) 0x7fa2b0082b80 con 0x7fa2bc154600 2024-07-17T07:15:21.338 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 17 and data pool 18 2024-07-17T07:15:21.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 msgr2=0x7fa29c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 0x7fa29c079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc158110 tx=0x7fa2a8005cf0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 msgr2=0x7fa2bc1f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005fd0 tx=0x7fa2b00039e0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 shutdown_connections 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa29c076cd0 0x7fa29c079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc0736c0 0x7fa2bc159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc161120 0x7fa2bc1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 --2- 172.21.15.44:0/2437782993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc154600 0x7fa2bc1f1580 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.346+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 >> 172.21.15.44:0/2437782993 conn(0x7fa2bc0b0eb0 msgr2=0x7fa2bc0cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.347+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 shutdown_connections 2024-07-17T07:15:21.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.347+0000 7fa2c3400640 1 -- 172.21.15.44:0/2437782993 wait complete. 2024-07-17T07:15:21.379 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 -- 172.21.15.44:0/3284784955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 msgr2=0x7fd8e806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 --2- 172.21.15.44:0/3284784955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e806e7a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0005fd0 tx=0x7fd8e002b980 comp rx=0 tx=0).stop 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 -- 172.21.15.44:0/3284784955 shutdown_connections 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 --2- 172.21.15.44:0/3284784955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e806e7a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 -- 172.21.15.44:0/3284784955 >> 172.21.15.44:0/3284784955 conn(0x7fd8e80b0fb0 msgr2=0x7fd8e80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 -- 172.21.15.44:0/3284784955 shutdown_connections 2024-07-17T07:15:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.550+0000 7fd8ed400640 1 -- 172.21.15.44:0/3284784955 wait complete. 2024-07-17T07:15:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.551+0000 7fd8ed400640 1 Processor -- start 2024-07-17T07:15:21.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.552+0000 7fd8ed400640 1 -- start start 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.552+0000 7fd8ed400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e80ac3e0 0x7fd8e81f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.552+0000 7fd8ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8ed400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e8153a50 0x7fd8e81e3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8ed400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8153fb0 con 0x7fd8e80ac870 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8ed400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8153e30 con 0x7fd8e80ac3e0 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8ed400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e806f710 con 0x7fd8e8153a50 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39094/0 (socket says 172.21.15.44:39094) 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 -- 172.21.15.44:0/2093657983 learned_addr learned my addr 172.21.15.44:0/2093657983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8e6800640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e80ac3e0 0x7fd8e81f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 -- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e8153a50 msgr2=0x7fd8e81e3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e8153a50 0x7fd8e81e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 -- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e80ac3e0 msgr2=0x7fd8e81f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e80ac3e0 0x7fd8e81f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.553+0000 7fd8dfe00640 1 -- 172.21.15.44:0/2093657983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e8095b50 con 0x7fd8e80ac870 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8dfe00640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000b3f0 tx=0x7fd8d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0015020 con 0x7fd8e80ac870 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8d0011920 con 0x7fd8e80ac870 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e8094d60 con 0x7fd8e80ac870 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0011d70 con 0x7fd8e80ac870 2024-07-17T07:15:21.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.554+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e8095320 con 0x7fd8e80ac870 2024-07-17T07:15:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.555+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e8159b00 con 0x7fd8e80ac870 2024-07-17T07:15:21.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.556+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fd8d0018020 con 0x7fd8e80ac870 2024-07-17T07:15:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.557+0000 7fd8dea00640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 0x7fd8c40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:21.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.557+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6649+0+0 (secure 0 0 0) 0x7fd8d00984f0 con 0x7fd8e80ac870 2024-07-17T07:15:21.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.558+0000 7fd8e6800640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 0x7fd8c40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:21.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.559+0000 7fd8e6800640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 0x7fd8c40791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8001640 tx=0x7fd8d8009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:21.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.561+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd8d001f030 con 0x7fd8e80ac870 2024-07-17T07:15:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.769+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8e8094a80 con 0x7fd8e80ac870 2024-07-17T07:15:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.772+0000 7fd8dea00640 1 -- 172.21.15.44:0/2093657983 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+17784 (secure 0 0 0) 0x7fd8e8094a80 con 0x7fd8e80ac870 2024-07-17T07:15:21.773 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:21.773 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":141,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:20.777262+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:15:18.669721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-07-17T07:15:19.770046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:21.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 msgr2=0x7fd8c40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 0x7fd8c40791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8001640 tx=0x7fd8d8009040 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 msgr2=0x7fd8e80943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000b3f0 tx=0x7fd8d000b8b0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 shutdown_connections 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fd8c4076d00 0x7fd8c40791c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8e8153a50 0x7fd8e81e3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e80ac870 0x7fd8e80943f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 --2- 172.21.15.44:0/2093657983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e80ac3e0 0x7fd8e81f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.775+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 >> 172.21.15.44:0/2093657983 conn(0x7fd8e80b0fb0 msgr2=0x7fd8e806e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.776+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 shutdown_connections 2024-07-17T07:15:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.776+0000 7fd8ed400640 1 -- 172.21.15.44:0/2093657983 wait complete. 2024-07-17T07:15:21.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.996+0000 7f09e9e00640 1 -- 172.21.15.44:0/3858053626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 msgr2=0x7f09e4166050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.996+0000 7f09e9e00640 1 --2- 172.21.15.44:0/3858053626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4166050 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f09dc005fd0 tx=0x7f09dc02ab20 comp rx=0 tx=0).stop 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.996+0000 7f09e9e00640 1 -- 172.21.15.44:0/3858053626 shutdown_connections 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.997+0000 7f09e9e00640 1 --2- 172.21.15.44:0/3858053626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4166050 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.997+0000 7f09e9e00640 1 -- 172.21.15.44:0/3858053626 >> 172.21.15.44:0/3858053626 conn(0x7f09e40b3120 msgr2=0x7f09e40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.997+0000 7f09e9e00640 1 -- 172.21.15.44:0/3858053626 shutdown_connections 2024-07-17T07:15:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.997+0000 7f09e9e00640 1 -- 172.21.15.44:0/3858053626 wait complete. 2024-07-17T07:15:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.998+0000 7f09e9e00640 1 Processor -- start 2024-07-17T07:15:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.998+0000 7f09e9e00640 1 -- start start 2024-07-17T07:15:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 0x7f09e41f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 0x7f09e4152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4166710 con 0x7f09e40cfdb0 2024-07-17T07:15:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4166590 con 0x7f09e415b5e0 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:21.999+0000 7f09e9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09e4166890 con 0x7f09e40cc890 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39118/0 (socket says 172.21.15.44:39118) 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 -- 172.21.15.44:0/1734710829 learned_addr learned my addr 172.21.15.44:0/1734710829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09e3400640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 0x7f09e4152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09e2a00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 0x7f09e41f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 -- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 msgr2=0x7f09e41f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 0x7f09e41f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 -- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 msgr2=0x7f09e4152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 0x7f09e4152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 -- 172.21.15.44:0/1734710829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e41f7cf0 con 0x7f09e40cfdb0 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09e2a00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 0x7f09e41f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09e3400640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 0x7f09e4152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.000+0000 7f09da000640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c460 tx=0x7f09cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.001+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc011790 con 0x7f09e40cfdb0 2024-07-17T07:15:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.001+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09cc017070 con 0x7f09e40cfdb0 2024-07-17T07:15:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.001+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc011d00 con 0x7f09e40cfdb0 2024-07-17T07:15:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.001+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e41f7fd0 con 0x7f09e40cfdb0 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.001+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41f8590 con 0x7f09e40cfdb0 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.002+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e40b0d50 con 0x7f09e40cfdb0 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.003+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f09cc01c460 con 0x7f09e40cfdb0 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.004+0000 7f09dbe00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 0x7f09c0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.004+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6649+0+0 (secure 0 0 0) 0x7f09cc098790 con 0x7f09e40cfdb0 2024-07-17T07:15:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.005+0000 7f09e2a00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 0x7f09c0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.006+0000 7f09e2a00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 0x7f09c0079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09d40016a0 tx=0x7f09d4009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:22.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.008+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f09cc061700 con 0x7f09e40cfdb0 2024-07-17T07:15:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.218+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e4154930 con 0x7f09e40cfdb0 2024-07-17T07:15:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.221+0000 7f09dbe00640 1 -- 172.21.15.44:0/1734710829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+17784 (secure 0 0 0) 0x7f09cc061000 con 0x7f09e40cfdb0 2024-07-17T07:15:22.221 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:22.221 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":141,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:20.777262+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:15:18.669721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-07-17T07:15:19.770046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 msgr2=0x7f09c0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 0x7f09c0079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09d40016a0 tx=0x7f09d4009040 comp rx=0 tx=0).stop 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 msgr2=0x7f09e4157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c460 tx=0x7f09cc00c920 comp rx=0 tx=0).stop 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 shutdown_connections 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f09c0076cd0 0x7f09c0079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e415b5e0 0x7f09e4152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e40cfdb0 0x7f09e4157e80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 --2- 172.21.15.44:0/1734710829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e40cc890 0x7f09e41f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 >> 172.21.15.44:0/1734710829 conn(0x7f09e40b3120 msgr2=0x7f09e40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 shutdown_connections 2024-07-17T07:15:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.224+0000 7f09e9e00640 1 -- 172.21.15.44:0/1734710829 wait complete. 2024-07-17T07:15:22.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:15:22.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 -- 172.21.15.44:0/2815899574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b8161be0 msgr2=0x7f55b80b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 --2- 172.21.15.44:0/2815899574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b8161be0 0x7f55b80b2200 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f55a8005fd0 tx=0x7f55a802a540 comp rx=0 tx=0).stop 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 -- 172.21.15.44:0/2815899574 shutdown_connections 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 --2- 172.21.15.44:0/2815899574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b8161be0 0x7f55b80b2200 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 --2- 172.21.15.44:0/2815899574 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55b815d130 0x7f55b815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 -- 172.21.15.44:0/2815899574 >> 172.21.15.44:0/2815899574 conn(0x7f55b80b4ff0 msgr2=0x7f55b80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 -- 172.21.15.44:0/2815899574 shutdown_connections 2024-07-17T07:15:22.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.461+0000 7f55be200640 1 -- 172.21.15.44:0/2815899574 wait complete. 2024-07-17T07:15:22.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.463+0000 7f55be200640 1 Processor -- start 2024-07-17T07:15:22.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.463+0000 7f55be200640 1 -- start start 2024-07-17T07:15:22.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 0x7f55b80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55b8161be0 0x7f55b815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55b80d1a10 con 0x7f55b80ce690 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55b80d1890 con 0x7f55b815d130 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55be200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55b80d1b90 con 0x7f55b8161be0 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39132/0 (socket says 172.21.15.44:39132) 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.464+0000 7f55bce00640 1 -- 172.21.15.44:0/3466706144 learned_addr learned my addr 172.21.15.44:0/3466706144 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55b7e00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 0x7f55b80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 -- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55b8161be0 msgr2=0x7f55b815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55b8161be0 0x7f55b815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 -- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 msgr2=0x7f55b80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 0x7f55b80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 -- 172.21.15.44:0/3466706144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b8157490 con 0x7f55b80ce690 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55b7e00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 0x7f55b80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55bce00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00e9f0 tx=0x7f55ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00cd80 con 0x7f55b80ce690 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55ac010430 con 0x7f55b80ce690 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.465+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac010840 con 0x7f55b80ce690 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.466+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b8157940 con 0x7f55b80ce690 2024-07-17T07:15:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.466+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b8157f00 con 0x7f55b80ce690 2024-07-17T07:15:22.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.467+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8160900 con 0x7f55b80ce690 2024-07-17T07:15:22.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.468+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f55ac0109e0 con 0x7f55b80ce690 2024-07-17T07:15:22.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.469+0000 7f55b4c00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 0x7f559c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:22.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.470+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f55ac099500 con 0x7f55b80ce690 2024-07-17T07:15:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.470+0000 7f55b7e00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 0x7f559c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:22.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.471+0000 7f55b7e00640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 0x7f559c079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55a8033000 tx=0x7f55a8004730 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:22.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.474+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55ac062420 con 0x7f55b80ce690 2024-07-17T07:15:22.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:22.716+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f55b8160530 con 0x7f55b80ce690 2024-07-17T07:15:23.250 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T07:15:23.250+0000 7f5ba2400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:15:23.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.359+0000 7f55b4c00640 1 -- 172.21.15.44:0/3466706144 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) ==== 118+0+0 (secure 0 0 0) 0x7f55ac061d20 con 0x7f55b80ce690 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.362+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 msgr2=0x7f559c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.362+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 0x7f559c079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55a8033000 tx=0x7f55a8004730 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.362+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 msgr2=0x7f55b8153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.362+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00e9f0 tx=0x7f55ac00eeb0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 shutdown_connections 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f559c076cd0 0x7f559c079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55b8161be0 0x7f55b815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55b815d130 0x7f55b80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 --2- 172.21.15.44:0/3466706144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55b80ce690 0x7f55b8153aa0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 >> 172.21.15.44:0/3466706144 conn(0x7f55b80b4ff0 msgr2=0x7f55b80b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:23.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 shutdown_connections 2024-07-17T07:15:23.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.363+0000 7f55be200640 1 -- 172.21.15.44:0/3466706144 wait complete. 2024-07-17T07:15:23.384 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:15:23.384 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:15:23.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.599+0000 7f2d81e00640 1 -- 172.21.15.44:0/2905329235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d60003af0 msgr2=0x7f2d60003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.599+0000 7f2d81e00640 1 --2- 172.21.15.44:0/2905329235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d60003af0 0x7f2d60003f10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006f80 tx=0x7f2d60005830 comp rx=0 tx=0).stop 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.599+0000 7f2d81400640 1 -- 172.21.15.44:0/2905329235 reap_dead start 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.599+0000 7f2d81e00640 1 -- 172.21.15.44:0/2905329235 shutdown_connections 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.599+0000 7f2d81e00640 1 -- 172.21.15.44:0/2905329235 >> 172.21.15.44:0/2905329235 conn(0x7f2d7c0cf0d0 msgr2=0x7f2d7c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.600+0000 7f2d81e00640 1 -- 172.21.15.44:0/2905329235 shutdown_connections 2024-07-17T07:15:23.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.600+0000 7f2d81e00640 1 -- 172.21.15.44:0/2905329235 wait complete. 2024-07-17T07:15:23.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.601+0000 7f2d81e00640 1 Processor -- start 2024-07-17T07:15:23.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.601+0000 7f2d81e00640 1 -- start start 2024-07-17T07:15:23.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60003af0 0x7f2d7c154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:23.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:23.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 0x7f2d7c073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:23.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0cbb20 con 0x7f2d7c14f120 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0cb9a0 con 0x7f2d7c073010 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.602+0000 7f2d81e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c14f5a0 con 0x7f2d60003af0 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39160/0 (socket says 172.21.15.44:39160) 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 -- 172.21.15.44:0/870991780 learned_addr learned my addr 172.21.15.44:0/870991780 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d81400640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 0x7f2d7c073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d80a00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60003af0 0x7f2d7c154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 -- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60003af0 msgr2=0x7f2d7c154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60003af0 0x7f2d7c154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 -- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 msgr2=0x7f2d7c073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 0x7f2d7c073490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d7be00640 1 -- 172.21.15.44:0/870991780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1f7ac0 con 0x7f2d7c14f120 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.603+0000 7f2d81400640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 0x7f2d7c073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:23.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d7be00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e770 tx=0x7f2d6400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:23.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cc60 con 0x7f2d7c14f120 2024-07-17T07:15:23.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6400ee80 con 0x7f2d7c14f120 2024-07-17T07:15:23.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64005280 con 0x7f2d7c14f120 2024-07-17T07:15:23.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c1f7d60 con 0x7f2d7c14f120 2024-07-17T07:15:23.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.604+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c1f8320 con 0x7f2d7c14f120 2024-07-17T07:15:23.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.605+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c0728e0 con 0x7f2d7c14f120 2024-07-17T07:15:23.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.607+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f2d64005420 con 0x7f2d7c14f120 2024-07-17T07:15:23.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.608+0000 7f2d78c00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 0x7f2d580791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:23.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.608+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f2d64099060 con 0x7f2d7c14f120 2024-07-17T07:15:23.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.609+0000 7f2d80a00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 0x7f2d580791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:23.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.610+0000 7f2d80a00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 0x7f2d580791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c033000 tx=0x7f2d6c001930 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:23.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.612+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2d64016030 con 0x7f2d7c14f120 2024-07-17T07:15:23.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:23.865+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2d7c0cb060 con 0x7f2d7c14f120 2024-07-17T07:15:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.371+0000 7f2d78c00640 1 -- 172.21.15.44:0/870991780 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) ==== 122+0+0 (secure 0 0 0) 0x7f2d64061ec0 con 0x7f2d7c14f120 2024-07-17T07:15:24.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 msgr2=0x7f2d580791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 0x7f2d580791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c033000 tx=0x7f2d6c001930 comp rx=0 tx=0).stop 2024-07-17T07:15:24.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 msgr2=0x7f2d7c1f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e770 tx=0x7f2d6400ec30 comp rx=0 tx=0).stop 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 shutdown_connections 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f2d58076d00 0x7f2d580791c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d7c073010 0x7f2d7c073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d7c14f120 0x7f2d7c1f7580 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 --2- 172.21.15.44:0/870991780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60003af0 0x7f2d7c154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.375+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 >> 172.21.15.44:0/870991780 conn(0x7f2d7c0cf0d0 msgr2=0x7f2d7c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.376+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 shutdown_connections 2024-07-17T07:15:24.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.376+0000 7f2d81e00640 1 -- 172.21.15.44:0/870991780 wait complete. 2024-07-17T07:15:24.398 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:15:24.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 -- 172.21.15.44:0/277366948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 msgr2=0x7ff3c4159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 --2- 172.21.15.44:0/277366948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4159f90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40d0340 tx=0x7ff3b402a530 comp rx=0 tx=0).stop 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 -- 172.21.15.44:0/277366948 shutdown_connections 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 --2- 172.21.15.44:0/277366948 >> v2:172.21.15.175:3300/0 conn(0x7ff3c40ac3e0 0x7ff3c40ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 --2- 172.21.15.44:0/277366948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4159f90 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 -- 172.21.15.44:0/277366948 >> 172.21.15.44:0/277366948 conn(0x7ff3c40b0fb0 msgr2=0x7ff3c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:24.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 -- 172.21.15.44:0/277366948 shutdown_connections 2024-07-17T07:15:24.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.613+0000 7ff3cc800640 1 -- 172.21.15.44:0/277366948 wait complete. 2024-07-17T07:15:24.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.615+0000 7ff3cc800640 1 Processor -- start 2024-07-17T07:15:24.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.615+0000 7ff3cc800640 1 -- start start 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406f710 0x7ff3c414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c40ac3e0 0x7ff3c40989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41643f0 con 0x7ff3c4159b00 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4164270 con 0x7ff3c406f710 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3cc800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c4164570 con 0x7ff3c40ac3e0 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39172/0 (socket says 172.21.15.44:39172) 2024-07-17T07:15:24.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3ca800640 1 -- 172.21.15.44:0/1804959281 learned_addr learned my addr 172.21.15.44:0/1804959281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3c9e00640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406f710 0x7ff3c414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.616+0000 7ff3c9400640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c40ac3e0 0x7ff3c40989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 -- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c40ac3e0 msgr2=0x7ff3c40989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c40ac3e0 0x7ff3c40989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 -- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406f710 msgr2=0x7ff3c414f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406f710 0x7ff3c414f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 -- 172.21.15.44:0/1804959281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40718c0 con 0x7ff3c4159b00 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ca800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e960 tx=0x7ff3c000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:24.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0004d20 con 0x7ff3c4159b00 2024-07-17T07:15:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3c000cb20 con 0x7ff3c4159b00 2024-07-17T07:15:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c4142ad0 con 0x7ff3c4159b00 2024-07-17T07:15:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0005020 con 0x7ff3c4159b00 2024-07-17T07:15:24.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.617+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c4073960 con 0x7ff3c4159b00 2024-07-17T07:15:24.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.618+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c4158f00 con 0x7ff3c4159b00 2024-07-17T07:15:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.620+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7ff3c00051c0 con 0x7ff3c4159b00 2024-07-17T07:15:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.621+0000 7ff3ba000640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 0x7ff3a8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.621+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7ff3c0099240 con 0x7ff3c4159b00 2024-07-17T07:15:24.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.622+0000 7ff3c9e00640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 0x7ff3a8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:24.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.623+0000 7ff3c9e00640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 0x7ff3a8079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc001700 tx=0x7ff3bc009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:24.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.625+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff3c4158f00 con 0x7ff3c4159b00 2024-07-17T07:15:24.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.881+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3c40af2a0 con 0x7ff3c4159b00 2024-07-17T07:15:24.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.882+0000 7ff3ba000640 1 -- 172.21.15.44:0/1804959281 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7ff3c40af2a0 con 0x7ff3c4159b00 2024-07-17T07:15:24.883 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:24.883 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:24.883 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:24.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 msgr2=0x7ff3a8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 0x7ff3a8079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc001700 tx=0x7ff3bc009040 comp rx=0 tx=0).stop 2024-07-17T07:15:24.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 msgr2=0x7ff3c4154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e960 tx=0x7ff3c000ee20 comp rx=0 tx=0).stop 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 shutdown_connections 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff3a8076cd0 0x7ff3a8079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c4159b00 0x7ff3c4154fb0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c40ac3e0 0x7ff3c40989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 --2- 172.21.15.44:0/1804959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406f710 0x7ff3c414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.886+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 >> 172.21.15.44:0/1804959281 conn(0x7ff3c40b0fb0 msgr2=0x7ff3c4150820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.887+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 shutdown_connections 2024-07-17T07:15:24.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:24.887+0000 7ff3cc800640 1 -- 172.21.15.44:0/1804959281 wait complete. 2024-07-17T07:15:24.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:25.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 -- 172.21.15.44:0/1048041567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40ac3e0 msgr2=0x7fb5f40ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/1048041567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f40ac870 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005fd0 tx=0x7fb5e002ae60 comp rx=0 tx=0).stop 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 -- 172.21.15.44:0/1048041567 shutdown_connections 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/1048041567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f40ac870 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 -- 172.21.15.44:0/1048041567 >> 172.21.15.44:0/1048041567 conn(0x7fb5f40b0fb0 msgr2=0x7fb5f40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 -- 172.21.15.44:0/1048041567 shutdown_connections 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.122+0000 7fb5fbe00640 1 -- 172.21.15.44:0/1048041567 wait complete. 2024-07-17T07:15:25.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.124+0000 7fb5fbe00640 1 Processor -- start 2024-07-17T07:15:25.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.124+0000 7fb5fbe00640 1 -- start start 2024-07-17T07:15:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406e090 0x7fb5f4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f41f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4153fb0 con 0x7fb5f4153a50 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4153e30 con 0x7fb5f40ac3e0 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5fbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f406f710 con 0x7fb5f406e090 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39184/0 (socket says 172.21.15.44:39184) 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.125+0000 7fb5f9e00640 1 -- 172.21.15.44:0/2327772029 learned_addr learned my addr 172.21.15.44:0/2327772029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f8a00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f41f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9400640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406e090 0x7fb5f4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 -- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406e090 msgr2=0x7fb5f4152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406e090 0x7fb5f4152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 -- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40ac3e0 msgr2=0x7fb5f41f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f41f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 -- 172.21.15.44:0/2327772029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f4158a90 con 0x7fb5f4153a50 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.126+0000 7fb5f9e00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c780 tx=0x7fb5ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.127+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec017070 con 0x7fb5f4153a50 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.127+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ec00ce30 con 0x7fb5f4153a50 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.127+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec005290 con 0x7fb5f4153a50 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.127+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f4158d70 con 0x7fb5f4153a50 2024-07-17T07:15:25.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.127+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f4159330 con 0x7fb5f4153a50 2024-07-17T07:15:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.128+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f40ac870 con 0x7fb5f4153a50 2024-07-17T07:15:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.129+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fb5ec005430 con 0x7fb5f4153a50 2024-07-17T07:15:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.130+0000 7fb5f1600640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.130+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fb5ec097ea0 con 0x7fb5f4153a50 2024-07-17T07:15:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.131+0000 7fb5f9400640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.132+0000 7fb5f9400640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 0x7fb5d4079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80015e0 tx=0x7fb5e8009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.135+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb5ec060e10 con 0x7fb5f4153a50 2024-07-17T07:15:25.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.344+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f40af010 con 0x7fb5f4153a50 2024-07-17T07:15:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.347+0000 7fb5f1600640 1 -- 172.21.15.44:0/2327772029 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+17771 (secure 0 0 0) 0x7fb5ec060710 con 0x7fb5f4153a50 2024-07-17T07:15:25.347 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:25.348 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":142,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:22.333343+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:15:18.669721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-07-17T07:15:19.770046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:25.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 msgr2=0x7fb5d4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 0x7fb5d4079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80015e0 tx=0x7fb5e8009040 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 msgr2=0x7fb5f41e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c780 tx=0x7fb5ec00cc40 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 shutdown_connections 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4153a50 0x7fb5f41e5320 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40ac3e0 0x7fb5f41f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 --2- 172.21.15.44:0/2327772029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406e090 0x7fb5f4152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.350+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 >> 172.21.15.44:0/2327772029 conn(0x7fb5f40b0fb0 msgr2=0x7fb5f40b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.351+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 shutdown_connections 2024-07-17T07:15:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.351+0000 7fb5fbe00640 1 -- 172.21.15.44:0/2327772029 wait complete. 2024-07-17T07:15:25.371 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:15:25.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:15:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 -- 172.21.15.44:0/2908556130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee00cfa90 msgr2=0x7fbee006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 --2- 172.21.15.44:0/2908556130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee00cfa90 0x7fbee006e060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009830 tx=0x7fbecc02a520 comp rx=0 tx=0).stop 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 -- 172.21.15.44:0/2908556130 shutdown_connections 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 --2- 172.21.15.44:0/2908556130 >> v2:172.21.15.175:3300/0 conn(0x7fbee0159800 0x7fbee0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 --2- 172.21.15.44:0/2908556130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee00cfa90 0x7fbee006e060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 -- 172.21.15.44:0/2908556130 >> 172.21.15.44:0/2908556130 conn(0x7fbee00b0e80 msgr2=0x7fbee00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.578+0000 7fbee5200640 1 -- 172.21.15.44:0/2908556130 shutdown_connections 2024-07-17T07:15:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.579+0000 7fbee5200640 1 -- 172.21.15.44:0/2908556130 wait complete. 2024-07-17T07:15:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.581+0000 7fbee5200640 1 Processor -- start 2024-07-17T07:15:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.581+0000 7fbee5200640 1 -- start start 2024-07-17T07:15:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbee006f4b0 0x7fbee00cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee0159800 0x7fbee00733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbee00b2120 con 0x7fbee00cfa90 2024-07-17T07:15:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbee00b1fa0 con 0x7fbee006f4b0 2024-07-17T07:15:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.582+0000 7fbee5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbee00b22a0 con 0x7fbee0159800 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39194/0 (socket says 172.21.15.44:39194) 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 -- 172.21.15.44:0/2571304620 learned_addr learned my addr 172.21.15.44:0/2571304620 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedf400640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbee006f4b0 0x7fbee00cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedfe00640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee0159800 0x7fbee00733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 -- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee0159800 msgr2=0x7fbee00733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee0159800 0x7fbee00733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 -- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbee006f4b0 msgr2=0x7fbee00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbee006f4b0 0x7fbee00cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.583+0000 7fbedea00640 1 -- 172.21.15.44:0/2571304620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee0155ad0 con 0x7fbee00cfa90 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbedea00640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbecc000c00 tx=0x7fbecc032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc040070 con 0x7fbee00cfa90 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc003af0 con 0x7fbee00cfa90 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee00cc4d0 con 0x7fbee00cfa90 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc032e40 con 0x7fbee00cfa90 2024-07-17T07:15:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.584+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee00ccd70 con 0x7fbee00cfa90 2024-07-17T07:15:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.585+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee006e060 con 0x7fbee00cfa90 2024-07-17T07:15:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.587+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fbecc039040 con 0x7fbee00cfa90 2024-07-17T07:15:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.588+0000 7fbed7400640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 0x7fbec0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.588+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fbecc0b9da0 con 0x7fbee00cfa90 2024-07-17T07:15:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.588+0000 7fbedf400640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 0x7fbec0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.589+0000 7fbedf400640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 0x7fbec0079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001680 tx=0x7fbed0009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.592+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbecc082c00 con 0x7fbee00cfa90 2024-07-17T07:15:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.807+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 --> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbee00cc6a0 con 0x7fbec0076cd0 2024-07-17T07:15:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.822+0000 7fbed7400640 1 -- 172.21.15.44:0/2571304620 <== mgr.6161 v2:172.21.15.175:6800/2606536318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238496 (secure 0 0 0) 0x7fbee00cc6a0 con 0x7fbec0076cd0 2024-07-17T07:15:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:15:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 msgr2=0x7fbec0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 0x7fbec0079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbed0001680 tx=0x7fbed0009040 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 msgr2=0x7fbee0155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbecc000c00 tx=0x7fbecc032b90 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 shutdown_connections 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fbec0076cd0 0x7fbec0079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbee0159800 0x7fbee00733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbee00cfa90 0x7fbee0155590 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 --2- 172.21.15.44:0/2571304620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbee006f4b0 0x7fbee00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.829+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 >> 172.21.15.44:0/2571304620 conn(0x7fbee00b0e80 msgr2=0x7fbee00addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.830+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 shutdown_connections 2024-07-17T07:15:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:25.830+0000 7fbee5200640 1 -- 172.21.15.44:0/2571304620 wait complete. 2024-07-17T07:15:25.850 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-07-17T07:15:24.073931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":230,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2436,"ondisk_log_size":2436,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":338940,"kb_used_data":14528,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754635780,"statfs":{"total":773094113280,"available":772747038720,"internally_reserved":0,"allocated":14876672,"data_stored":8437669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":332058160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.791394"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839270+0000","last_change":"2024-07-17T07:15:21.365871+0000","last_active":"2024-07-17T07:15:22.839270+0000","last_peered":"2024-07-17T07:15:22.839270+0000","last_clean":"2024-07-17T07:15:22.839270+0000","last_became_active":"2024-07-17T07:15:21.365565+0000","last_became_peered":"2024-07-17T07:15:21.365565+0000","last_unstale":"2024-07-17T07:15:22.839270+0000","last_undegraded":"2024-07-17T07:15:22.839270+0000","last_fullsized":"2024-07-17T07:15:22.839270+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:43.087964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839710+0000","last_change":"2024-07-17T07:15:21.345429+0000","last_active":"2024-07-17T07:15:22.839710+0000","last_peered":"2024-07-17T07:15:22.839710+0000","last_clean":"2024-07-17T07:15:22.839710+0000","last_became_active":"2024-07-17T07:15:21.345141+0000","last_became_peered":"2024-07-17T07:15:21.345141+0000","last_unstale":"2024-07-17T07:15:22.839710+0000","last_undegraded":"2024-07-17T07:15:22.839710+0000","last_fullsized":"2024-07-17T07:15:22.839710+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:03.990658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839592+0000","last_change":"2024-07-17T07:15:21.336163+0000","last_active":"2024-07-17T07:15:22.839592+0000","last_peered":"2024-07-17T07:15:22.839592+0000","last_clean":"2024-07-17T07:15:22.839592+0000","last_became_active":"2024-07-17T07:15:21.333722+0000","last_became_peered":"2024-07-17T07:15:21.333722+0000","last_unstale":"2024-07-17T07:15:22.839592+0000","last_undegraded":"2024-07-17T07:15:22.839592+0000","last_fullsized":"2024-07-17T07:15:22.839592+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:30.699254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839051+0000","last_change":"2024-07-17T07:15:21.333737+0000","last_active":"2024-07-17T07:15:22.839051+0000","last_peered":"2024-07-17T07:15:22.839051+0000","last_clean":"2024-07-17T07:15:22.839051+0000","last_became_active":"2024-07-17T07:15:21.332790+0000","last_became_peered":"2024-07-17T07:15:21.332790+0000","last_unstale":"2024-07-17T07:15:22.839051+0000","last_undegraded":"2024-07-17T07:15:22.839051+0000","last_fullsized":"2024-07-17T07:15:22.839051+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:58.172910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839616+0000","last_change":"2024-07-17T07:15:20.318202+0000","last_active":"2024-07-17T07:15:22.839616+0000","last_peered":"2024-07-17T07:15:22.839616+0000","last_clean":"2024-07-17T07:15:22.839616+0000","last_became_active":"2024-07-17T07:15:20.315104+0000","last_became_peered":"2024-07-17T07:15:20.315104+0000","last_unstale":"2024-07-17T07:15:22.839616+0000","last_undegraded":"2024-07-17T07:15:22.839616+0000","last_fullsized":"2024-07-17T07:15:22.839616+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:55.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839877+0000","last_change":"2024-07-17T07:15:21.355713+0000","last_active":"2024-07-17T07:15:22.839877+0000","last_peered":"2024-07-17T07:15:22.839877+0000","last_clean":"2024-07-17T07:15:22.839877+0000","last_became_active":"2024-07-17T07:15:21.353960+0000","last_became_peered":"2024-07-17T07:15:21.353960+0000","last_unstale":"2024-07-17T07:15:22.839877+0000","last_undegraded":"2024-07-17T07:15:22.839877+0000","last_fullsized":"2024-07-17T07:15:22.839877+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:52.349147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:20.280314+0000","last_change":"2024-07-17T07:15:20.280314+0000","last_active":"2024-07-17T07:15:20.280314+0000","last_peered":"2024-07-17T07:15:20.280314+0000","last_clean":"2024-07-17T07:15:20.280314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:20.280314+0000","last_undegraded":"2024-07-17T07:15:20.280314+0000","last_fullsized":"2024-07-17T07:15:20.280314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-07-17T07:15:19.275299+0000","last_change":"2024-07-17T07:15:19.275299+0000","last_active":"2024-07-17T07:15:19.275299+0000","last_peered":"2024-07-17T07:15:19.275299+0000","last_clean":"2024-07-17T07:15:19.275299+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T07:15:19.275299+0000","last_undegraded":"2024-07-17T07:15:19.275299+0000","last_fullsized":"2024-07-17T07:15:19.275299+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"132'2434","reported_seq":2911,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-07-17T07:15:17.655014+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:15:17.655014+0000","last_peered":"2024-07-17T07:15:17.655014+0000","last_clean":"2024-07-17T07:15:17.655014+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:15:17.655014+0000","last_undegraded":"2024-07-17T07:15:17.655014+0000","last_fullsized":"2024-07-17T07:15:17.655014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":227,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":227,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2434,"ondisk_log_size":2434,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542387,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51484,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320356,"statfs":{"total":96636764160,"available":96584044544,"internally_reserved":0,"allocated":2301952,"data_stored":1495368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509682,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37016,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":1708032,"data_stored":905000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55056,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1708032,"data_stored":905000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542388,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1708032,"data_stored":905000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509682,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37012,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334828,"statfs":{"total":96636764160,"available":96598863872,"internally_reserved":0,"allocated":1708032,"data_stored":905000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542387,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59264,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312576,"statfs":{"total":96636764160,"available":96576077824,"internally_reserved":0,"allocated":1757184,"data_stored":933927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36996,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1695744,"data_stored":899003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509682,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2289664,"data_stored":1489371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:15:25.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:15:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 -- 172.21.15.44:0/3199384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e1c004e40 msgr2=0x7f3e1c0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/3199384669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e1c004e40 0x7f3e1c0052c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f3e30001d90 tx=0x7f3e1c005830 comp rx=0 tx=0).stop 2024-07-17T07:15:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e37e00640 1 -- 172.21.15.44:0/3199384669 reap_dead start 2024-07-17T07:15:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 -- 172.21.15.44:0/3199384669 shutdown_connections 2024-07-17T07:15:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 -- 172.21.15.44:0/3199384669 >> 172.21.15.44:0/3199384669 conn(0x7f3e380b0e80 msgr2=0x7f3e380b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 -- 172.21.15.44:0/3199384669 shutdown_connections 2024-07-17T07:15:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.111+0000 7f3e3ca00640 1 -- 172.21.15.44:0/3199384669 wait complete. 2024-07-17T07:15:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.113+0000 7f3e3ca00640 1 Processor -- start 2024-07-17T07:15:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.113+0000 7f3e3ca00640 1 -- start start 2024-07-17T07:15:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e38144620 0x7f3e380729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e38153f70 0x7f3e38072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e380acc70 con 0x7f3e38073430 2024-07-17T07:15:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3809cdf0 con 0x7f3e38153f70 2024-07-17T07:15:26.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e380acdf0 con 0x7f3e38144620 2024-07-17T07:15:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.114+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39208/0 (socket says 172.21.15.44:39208) 2024-07-17T07:15:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 -- 172.21.15.44:0/2864606166 learned_addr learned my addr 172.21.15.44:0/2864606166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e36a00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e38153f70 0x7f3e38072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37400640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e38144620 0x7f3e380729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 -- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e38144620 msgr2=0x7f3e380729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e38144620 0x7f3e380729b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 -- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e38153f70 msgr2=0x7f3e38072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e38153f70 0x7f3e38072ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 -- 172.21.15.44:0/2864606166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3815c9a0 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.115+0000 7f3e37e00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3e3002ac80 tx=0x7f3e300019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.116+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30006ca0 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.116+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e300060b0 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.116+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30006500 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.116+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e38159d40 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.116+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e38144a00 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.117+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e380cf2d0 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.118+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f3e30033480 con 0x7f3e38073430 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.119+0000 7f3e2f400640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 0x7f3e180791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.120+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f3e300b8380 con 0x7f3e38073430 2024-07-17T07:15:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.120+0000 7f3e37400640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 0x7f3e180791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.121+0000 7f3e37400640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 0x7f3e180791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e28005fd0 tx=0x7f3e28005cf0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.124+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3e30081280 con 0x7f3e38073430 2024-07-17T07:15:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.337+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 --> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e3815cc80 con 0x7f3e18076d00 2024-07-17T07:15:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.352+0000 7f3e2f400640 1 -- 172.21.15.44:0/2864606166 <== mgr.6161 v2:172.21.15.175:6800/2606536318 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252497 (secure 0 0 0) 0x7f3e3815cc80 con 0x7f3e18076d00 2024-07-17T07:15:26.353 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:26.357 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:15:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 msgr2=0x7f3e180791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 0x7f3e180791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e28005fd0 tx=0x7f3e28005cf0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 msgr2=0x7f3e38146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3e3002ac80 tx=0x7f3e300019e0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 shutdown_connections 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f3e18076d00 0x7f3e180791c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e38073430 0x7f3e38146df0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e38153f70 0x7f3e38072ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 --2- 172.21.15.44:0/2864606166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e38144620 0x7f3e380729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.359+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 >> 172.21.15.44:0/2864606166 conn(0x7f3e380b0e80 msgr2=0x7f3e380afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.360+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 shutdown_connections 2024-07-17T07:15:26.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.360+0000 7f3e3ca00640 1 -- 172.21.15.44:0/2864606166 wait complete. 2024-07-17T07:15:26.382 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:15:26.074607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":272,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2473,"ondisk_log_size":2473,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":343520,"kb_used_data":14952,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754631200,"statfs":{"total":773094113280,"available":772742348800,"internally_reserved":0,"allocated":15310848,"data_stored":8656972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002047"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858251+0000","last_change":"2024-07-17T07:15:21.347517+0000","last_active":"2024-07-17T07:15:22.858251+0000","last_peered":"2024-07-17T07:15:22.858251+0000","last_clean":"2024-07-17T07:15:22.858251+0000","last_became_active":"2024-07-17T07:15:21.347201+0000","last_became_peered":"2024-07-17T07:15:21.347201+0000","last_unstale":"2024-07-17T07:15:22.858251+0000","last_undegraded":"2024-07-17T07:15:22.858251+0000","last_fullsized":"2024-07-17T07:15:22.858251+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:15.050719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353178+0000","last_change":"2024-07-17T07:15:21.346818+0000","last_active":"2024-07-17T07:15:22.353178+0000","last_peered":"2024-07-17T07:15:22.353178+0000","last_clean":"2024-07-17T07:15:22.353178+0000","last_became_active":"2024-07-17T07:15:21.346310+0000","last_became_peered":"2024-07-17T07:15:21.346310+0000","last_unstale":"2024-07-17T07:15:22.353178+0000","last_undegraded":"2024-07-17T07:15:22.353178+0000","last_fullsized":"2024-07-17T07:15:22.353178+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:31:31.741563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353314+0000","last_change":"2024-07-17T07:15:21.341735+0000","last_active":"2024-07-17T07:15:22.353314+0000","last_peered":"2024-07-17T07:15:22.353314+0000","last_clean":"2024-07-17T07:15:22.353314+0000","last_became_active":"2024-07-17T07:15:21.341440+0000","last_became_peered":"2024-07-17T07:15:21.341440+0000","last_unstale":"2024-07-17T07:15:22.353314+0000","last_undegraded":"2024-07-17T07:15:22.353314+0000","last_fullsized":"2024-07-17T07:15:22.353314+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:32.450188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354910+0000","last_change":"2024-07-17T07:15:21.357594+0000","last_active":"2024-07-17T07:15:22.354910+0000","last_peered":"2024-07-17T07:15:22.354910+0000","last_clean":"2024-07-17T07:15:22.354910+0000","last_became_active":"2024-07-17T07:15:21.357062+0000","last_became_peered":"2024-07-17T07:15:21.357062+0000","last_unstale":"2024-07-17T07:15:22.354910+0000","last_undegraded":"2024-07-17T07:15:22.354910+0000","last_fullsized":"2024-07-17T07:15:22.354910+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:23:12.552934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858432+0000","last_change":"2024-07-17T07:15:21.332308+0000","last_active":"2024-07-17T07:15:22.858432+0000","last_peered":"2024-07-17T07:15:22.858432+0000","last_clean":"2024-07-17T07:15:22.858432+0000","last_became_active":"2024-07-17T07:15:21.331609+0000","last_became_peered":"2024-07-17T07:15:21.331609+0000","last_unstale":"2024-07-17T07:15:22.858432+0000","last_undegraded":"2024-07-17T07:15:22.858432+0000","last_fullsized":"2024-07-17T07:15:22.858432+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:47:01.654152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366473+0000","last_change":"2024-07-17T07:15:21.368054+0000","last_active":"2024-07-17T07:15:22.366473+0000","last_peered":"2024-07-17T07:15:22.366473+0000","last_clean":"2024-07-17T07:15:22.366473+0000","last_became_active":"2024-07-17T07:15:21.366224+0000","last_became_peered":"2024-07-17T07:15:21.366224+0000","last_unstale":"2024-07-17T07:15:22.366473+0000","last_undegraded":"2024-07-17T07:15:22.366473+0000","last_fullsized":"2024-07-17T07:15:22.366473+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:17.114178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366268+0000","last_change":"2024-07-17T07:15:21.362838+0000","last_active":"2024-07-17T07:15:22.366268+0000","last_peered":"2024-07-17T07:15:22.366268+0000","last_clean":"2024-07-17T07:15:22.366268+0000","last_became_active":"2024-07-17T07:15:21.357005+0000","last_became_peered":"2024-07-17T07:15:21.357005+0000","last_unstale":"2024-07-17T07:15:22.366268+0000","last_undegraded":"2024-07-17T07:15:22.366268+0000","last_fullsized":"2024-07-17T07:15:22.366268+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:07:22.764626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352270+0000","last_change":"2024-07-17T07:15:21.343226+0000","last_active":"2024-07-17T07:15:22.352270+0000","last_peered":"2024-07-17T07:15:22.352270+0000","last_clean":"2024-07-17T07:15:22.352270+0000","last_became_active":"2024-07-17T07:15:21.342462+0000","last_became_peered":"2024-07-17T07:15:21.342462+0000","last_unstale":"2024-07-17T07:15:22.352270+0000","last_undegraded":"2024-07-17T07:15:22.352270+0000","last_fullsized":"2024-07-17T07:15:22.352270+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:51.743138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355634+0000","last_change":"2024-07-17T07:15:21.366313+0000","last_active":"2024-07-17T07:15:22.355634+0000","last_peered":"2024-07-17T07:15:22.355634+0000","last_clean":"2024-07-17T07:15:22.355634+0000","last_became_active":"2024-07-17T07:15:21.365933+0000","last_became_peered":"2024-07-17T07:15:21.365933+0000","last_unstale":"2024-07-17T07:15:22.355634+0000","last_undegraded":"2024-07-17T07:15:22.355634+0000","last_fullsized":"2024-07-17T07:15:22.355634+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:08:31.378520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352581+0000","last_change":"2024-07-17T07:15:21.365704+0000","last_active":"2024-07-17T07:15:22.352581+0000","last_peered":"2024-07-17T07:15:22.352581+0000","last_clean":"2024-07-17T07:15:22.352581+0000","last_became_active":"2024-07-17T07:15:21.365377+0000","last_became_peered":"2024-07-17T07:15:21.365377+0000","last_unstale":"2024-07-17T07:15:22.352581+0000","last_undegraded":"2024-07-17T07:15:22.352581+0000","last_fullsized":"2024-07-17T07:15:22.352581+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:05.787943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858473+0000","last_change":"2024-07-17T07:15:21.351065+0000","last_active":"2024-07-17T07:15:22.858473+0000","last_peered":"2024-07-17T07:15:22.858473+0000","last_clean":"2024-07-17T07:15:22.858473+0000","last_became_active":"2024-07-17T07:15:21.350541+0000","last_became_peered":"2024-07-17T07:15:21.350541+0000","last_unstale":"2024-07-17T07:15:22.858473+0000","last_undegraded":"2024-07-17T07:15:22.858473+0000","last_fullsized":"2024-07-17T07:15:22.858473+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:41:51.361647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.367510+0000","last_change":"2024-07-17T07:15:21.361687+0000","last_active":"2024-07-17T07:15:22.367510+0000","last_peered":"2024-07-17T07:15:22.367510+0000","last_clean":"2024-07-17T07:15:22.367510+0000","last_became_active":"2024-07-17T07:15:21.361075+0000","last_became_peered":"2024-07-17T07:15:21.361075+0000","last_unstale":"2024-07-17T07:15:22.367510+0000","last_undegraded":"2024-07-17T07:15:22.367510+0000","last_fullsized":"2024-07-17T07:15:22.367510+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:04.356239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350640+0000","last_change":"2024-07-17T07:15:21.352004+0000","last_active":"2024-07-17T07:15:22.350640+0000","last_peered":"2024-07-17T07:15:22.350640+0000","last_clean":"2024-07-17T07:15:22.350640+0000","last_became_active":"2024-07-17T07:15:21.351735+0000","last_became_peered":"2024-07-17T07:15:21.351735+0000","last_unstale":"2024-07-17T07:15:22.350640+0000","last_undegraded":"2024-07-17T07:15:22.350640+0000","last_fullsized":"2024-07-17T07:15:22.350640+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:15:17.079182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350992+0000","last_change":"2024-07-17T07:15:21.359591+0000","last_active":"2024-07-17T07:15:22.350992+0000","last_peered":"2024-07-17T07:15:22.350992+0000","last_clean":"2024-07-17T07:15:22.350992+0000","last_became_active":"2024-07-17T07:15:21.359075+0000","last_became_peered":"2024-07-17T07:15:21.359075+0000","last_unstale":"2024-07-17T07:15:22.350992+0000","last_undegraded":"2024-07-17T07:15:22.350992+0000","last_fullsized":"2024-07-17T07:15:22.350992+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:54.581362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857411+0000","last_change":"2024-07-17T07:15:21.352367+0000","last_active":"2024-07-17T07:15:22.857411+0000","last_peered":"2024-07-17T07:15:22.857411+0000","last_clean":"2024-07-17T07:15:22.857411+0000","last_became_active":"2024-07-17T07:15:21.352047+0000","last_became_peered":"2024-07-17T07:15:21.352047+0000","last_unstale":"2024-07-17T07:15:22.857411+0000","last_undegraded":"2024-07-17T07:15:22.857411+0000","last_fullsized":"2024-07-17T07:15:22.857411+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:53:55.392978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.367757+0000","last_change":"2024-07-17T07:15:21.362180+0000","last_active":"2024-07-17T07:15:22.367757+0000","last_peered":"2024-07-17T07:15:22.367757+0000","last_clean":"2024-07-17T07:15:22.367757+0000","last_became_active":"2024-07-17T07:15:21.355379+0000","last_became_peered":"2024-07-17T07:15:21.355379+0000","last_unstale":"2024-07-17T07:15:22.367757+0000","last_undegraded":"2024-07-17T07:15:22.367757+0000","last_fullsized":"2024-07-17T07:15:22.367757+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:44:52.848326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352526+0000","last_change":"2024-07-17T07:15:21.352630+0000","last_active":"2024-07-17T07:15:22.352526+0000","last_peered":"2024-07-17T07:15:22.352526+0000","last_clean":"2024-07-17T07:15:22.352526+0000","last_became_active":"2024-07-17T07:15:21.352321+0000","last_became_peered":"2024-07-17T07:15:21.352321+0000","last_unstale":"2024-07-17T07:15:22.352526+0000","last_undegraded":"2024-07-17T07:15:22.352526+0000","last_fullsized":"2024-07-17T07:15:22.352526+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:24:09.272170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353220+0000","last_change":"2024-07-17T07:15:21.346587+0000","last_active":"2024-07-17T07:15:22.353220+0000","last_peered":"2024-07-17T07:15:22.353220+0000","last_clean":"2024-07-17T07:15:22.353220+0000","last_became_active":"2024-07-17T07:15:21.345933+0000","last_became_peered":"2024-07-17T07:15:21.345933+0000","last_unstale":"2024-07-17T07:15:22.353220+0000","last_undegraded":"2024-07-17T07:15:22.353220+0000","last_fullsized":"2024-07-17T07:15:22.353220+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:55.396956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363090+0000","last_change":"2024-07-17T07:15:21.351704+0000","last_active":"2024-07-17T07:15:22.363090+0000","last_peered":"2024-07-17T07:15:22.363090+0000","last_clean":"2024-07-17T07:15:22.363090+0000","last_became_active":"2024-07-17T07:15:21.346563+0000","last_became_peered":"2024-07-17T07:15:21.346563+0000","last_unstale":"2024-07-17T07:15:22.363090+0000","last_undegraded":"2024-07-17T07:15:22.363090+0000","last_fullsized":"2024-07-17T07:15:22.363090+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:56.359876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839270+0000","last_change":"2024-07-17T07:15:21.365871+0000","last_active":"2024-07-17T07:15:22.839270+0000","last_peered":"2024-07-17T07:15:22.839270+0000","last_clean":"2024-07-17T07:15:22.839270+0000","last_became_active":"2024-07-17T07:15:21.365565+0000","last_became_peered":"2024-07-17T07:15:21.365565+0000","last_unstale":"2024-07-17T07:15:22.839270+0000","last_undegraded":"2024-07-17T07:15:22.839270+0000","last_fullsized":"2024-07-17T07:15:22.839270+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:43.087964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355535+0000","last_change":"2024-07-17T07:15:21.347752+0000","last_active":"2024-07-17T07:15:22.355535+0000","last_peered":"2024-07-17T07:15:22.355535+0000","last_clean":"2024-07-17T07:15:22.355535+0000","last_became_active":"2024-07-17T07:15:21.347126+0000","last_became_peered":"2024-07-17T07:15:21.347126+0000","last_unstale":"2024-07-17T07:15:22.355535+0000","last_undegraded":"2024-07-17T07:15:22.355535+0000","last_fullsized":"2024-07-17T07:15:22.355535+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:26:12.699005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858336+0000","last_change":"2024-07-17T07:15:21.350747+0000","last_active":"2024-07-17T07:15:22.858336+0000","last_peered":"2024-07-17T07:15:22.858336+0000","last_clean":"2024-07-17T07:15:22.858336+0000","last_became_active":"2024-07-17T07:15:21.350435+0000","last_became_peered":"2024-07-17T07:15:21.350435+0000","last_unstale":"2024-07-17T07:15:22.858336+0000","last_undegraded":"2024-07-17T07:15:22.858336+0000","last_fullsized":"2024-07-17T07:15:22.858336+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:19.610912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354474+0000","last_change":"2024-07-17T07:15:21.359065+0000","last_active":"2024-07-17T07:15:22.354474+0000","last_peered":"2024-07-17T07:15:22.354474+0000","last_clean":"2024-07-17T07:15:22.354474+0000","last_became_active":"2024-07-17T07:15:21.358766+0000","last_became_peered":"2024-07-17T07:15:21.358766+0000","last_unstale":"2024-07-17T07:15:22.354474+0000","last_undegraded":"2024-07-17T07:15:22.354474+0000","last_fullsized":"2024-07-17T07:15:22.354474+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:53.598738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856322+0000","last_change":"2024-07-17T07:15:21.352608+0000","last_active":"2024-07-17T07:15:22.856322+0000","last_peered":"2024-07-17T07:15:22.856322+0000","last_clean":"2024-07-17T07:15:22.856322+0000","last_became_active":"2024-07-17T07:15:21.351419+0000","last_became_peered":"2024-07-17T07:15:21.351419+0000","last_unstale":"2024-07-17T07:15:22.856322+0000","last_undegraded":"2024-07-17T07:15:22.856322+0000","last_fullsized":"2024-07-17T07:15:22.856322+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:49.009080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857381+0000","last_change":"2024-07-17T07:15:21.352093+0000","last_active":"2024-07-17T07:15:22.857381+0000","last_peered":"2024-07-17T07:15:22.857381+0000","last_clean":"2024-07-17T07:15:22.857381+0000","last_became_active":"2024-07-17T07:15:21.351807+0000","last_became_peered":"2024-07-17T07:15:21.351807+0000","last_unstale":"2024-07-17T07:15:22.857381+0000","last_undegraded":"2024-07-17T07:15:22.857381+0000","last_fullsized":"2024-07-17T07:15:22.857381+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:02.538148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839710+0000","last_change":"2024-07-17T07:15:21.345429+0000","last_active":"2024-07-17T07:15:22.839710+0000","last_peered":"2024-07-17T07:15:22.839710+0000","last_clean":"2024-07-17T07:15:22.839710+0000","last_became_active":"2024-07-17T07:15:21.345141+0000","last_became_peered":"2024-07-17T07:15:21.345141+0000","last_unstale":"2024-07-17T07:15:22.839710+0000","last_undegraded":"2024-07-17T07:15:22.839710+0000","last_fullsized":"2024-07-17T07:15:22.839710+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:03.990658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363482+0000","last_change":"2024-07-17T07:15:21.341008+0000","last_active":"2024-07-17T07:15:22.363482+0000","last_peered":"2024-07-17T07:15:22.363482+0000","last_clean":"2024-07-17T07:15:22.363482+0000","last_became_active":"2024-07-17T07:15:21.340544+0000","last_became_peered":"2024-07-17T07:15:21.340544+0000","last_unstale":"2024-07-17T07:15:22.363482+0000","last_undegraded":"2024-07-17T07:15:22.363482+0000","last_fullsized":"2024-07-17T07:15:22.363482+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:55.872938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.367918+0000","last_change":"2024-07-17T07:15:21.361732+0000","last_active":"2024-07-17T07:15:22.367918+0000","last_peered":"2024-07-17T07:15:22.367918+0000","last_clean":"2024-07-17T07:15:22.367918+0000","last_became_active":"2024-07-17T07:15:21.361074+0000","last_became_peered":"2024-07-17T07:15:21.361074+0000","last_unstale":"2024-07-17T07:15:22.367918+0000","last_undegraded":"2024-07-17T07:15:22.367918+0000","last_fullsized":"2024-07-17T07:15:22.367918+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:12.083712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839592+0000","last_change":"2024-07-17T07:15:21.336163+0000","last_active":"2024-07-17T07:15:22.839592+0000","last_peered":"2024-07-17T07:15:22.839592+0000","last_clean":"2024-07-17T07:15:22.839592+0000","last_became_active":"2024-07-17T07:15:21.333722+0000","last_became_peered":"2024-07-17T07:15:21.333722+0000","last_unstale":"2024-07-17T07:15:22.839592+0000","last_undegraded":"2024-07-17T07:15:22.839592+0000","last_fullsized":"2024-07-17T07:15:22.839592+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:30.699254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351142+0000","last_change":"2024-07-17T07:15:21.344572+0000","last_active":"2024-07-17T07:15:22.351142+0000","last_peered":"2024-07-17T07:15:22.351142+0000","last_clean":"2024-07-17T07:15:22.351142+0000","last_became_active":"2024-07-17T07:15:21.344215+0000","last_became_peered":"2024-07-17T07:15:21.344215+0000","last_unstale":"2024-07-17T07:15:22.351142+0000","last_undegraded":"2024-07-17T07:15:22.351142+0000","last_fullsized":"2024-07-17T07:15:22.351142+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:31.569043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856271+0000","last_change":"2024-07-17T07:15:21.354077+0000","last_active":"2024-07-17T07:15:22.856271+0000","last_peered":"2024-07-17T07:15:22.856271+0000","last_clean":"2024-07-17T07:15:22.856271+0000","last_became_active":"2024-07-17T07:15:21.353799+0000","last_became_peered":"2024-07-17T07:15:21.353799+0000","last_unstale":"2024-07-17T07:15:22.856271+0000","last_undegraded":"2024-07-17T07:15:22.856271+0000","last_fullsized":"2024-07-17T07:15:22.856271+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:44.964195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856016+0000","last_change":"2024-07-17T07:15:21.352336+0000","last_active":"2024-07-17T07:15:22.856016+0000","last_peered":"2024-07-17T07:15:22.856016+0000","last_clean":"2024-07-17T07:15:22.856016+0000","last_became_active":"2024-07-17T07:15:21.351357+0000","last_became_peered":"2024-07-17T07:15:21.351357+0000","last_unstale":"2024-07-17T07:15:22.856016+0000","last_undegraded":"2024-07-17T07:15:22.856016+0000","last_fullsized":"2024-07-17T07:15:22.856016+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:58.575805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351015+0000","last_change":"2024-07-17T07:15:21.366551+0000","last_active":"2024-07-17T07:15:22.351015+0000","last_peered":"2024-07-17T07:15:22.351015+0000","last_clean":"2024-07-17T07:15:22.351015+0000","last_became_active":"2024-07-17T07:15:21.365335+0000","last_became_peered":"2024-07-17T07:15:21.365335+0000","last_unstale":"2024-07-17T07:15:22.351015+0000","last_undegraded":"2024-07-17T07:15:22.351015+0000","last_fullsized":"2024-07-17T07:15:22.351015+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:15.829566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363264+0000","last_change":"2024-07-17T07:15:20.324380+0000","last_active":"2024-07-17T07:15:22.363264+0000","last_peered":"2024-07-17T07:15:22.363264+0000","last_clean":"2024-07-17T07:15:22.363264+0000","last_became_active":"2024-07-17T07:15:20.324073+0000","last_became_peered":"2024-07-17T07:15:20.324073+0000","last_unstale":"2024-07-17T07:15:22.363264+0000","last_undegraded":"2024-07-17T07:15:22.363264+0000","last_fullsized":"2024-07-17T07:15:22.363264+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:24:08.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366698+0000","last_change":"2024-07-17T07:15:21.361754+0000","last_active":"2024-07-17T07:15:22.366698+0000","last_peered":"2024-07-17T07:15:22.366698+0000","last_clean":"2024-07-17T07:15:22.366698+0000","last_became_active":"2024-07-17T07:15:21.361291+0000","last_became_peered":"2024-07-17T07:15:21.361291+0000","last_unstale":"2024-07-17T07:15:22.366698+0000","last_undegraded":"2024-07-17T07:15:22.366698+0000","last_fullsized":"2024-07-17T07:15:22.366698+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:49.114663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352763+0000","last_change":"2024-07-17T07:15:20.329049+0000","last_active":"2024-07-17T07:15:22.352763+0000","last_peered":"2024-07-17T07:15:22.352763+0000","last_clean":"2024-07-17T07:15:22.352763+0000","last_became_active":"2024-07-17T07:15:20.328511+0000","last_became_peered":"2024-07-17T07:15:20.328511+0000","last_unstale":"2024-07-17T07:15:22.352763+0000","last_undegraded":"2024-07-17T07:15:22.352763+0000","last_fullsized":"2024-07-17T07:15:22.352763+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:10.594955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856729+0000","last_change":"2024-07-17T07:15:21.365508+0000","last_active":"2024-07-17T07:15:22.856729+0000","last_peered":"2024-07-17T07:15:22.856729+0000","last_clean":"2024-07-17T07:15:22.856729+0000","last_became_active":"2024-07-17T07:15:21.364990+0000","last_became_peered":"2024-07-17T07:15:21.364990+0000","last_unstale":"2024-07-17T07:15:22.856729+0000","last_undegraded":"2024-07-17T07:15:22.856729+0000","last_fullsized":"2024-07-17T07:15:22.856729+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:28:09.171853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355278+0000","last_change":"2024-07-17T07:15:20.323983+0000","last_active":"2024-07-17T07:15:22.355278+0000","last_peered":"2024-07-17T07:15:22.355278+0000","last_clean":"2024-07-17T07:15:22.355278+0000","last_became_active":"2024-07-17T07:15:20.323713+0000","last_became_peered":"2024-07-17T07:15:20.323713+0000","last_unstale":"2024-07-17T07:15:22.355278+0000","last_undegraded":"2024-07-17T07:15:22.355278+0000","last_fullsized":"2024-07-17T07:15:22.355278+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:44.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355795+0000","last_change":"2024-07-17T07:15:21.346326+0000","last_active":"2024-07-17T07:15:22.355795+0000","last_peered":"2024-07-17T07:15:22.355795+0000","last_clean":"2024-07-17T07:15:22.355795+0000","last_became_active":"2024-07-17T07:15:21.346051+0000","last_became_peered":"2024-07-17T07:15:21.346051+0000","last_unstale":"2024-07-17T07:15:22.355795+0000","last_undegraded":"2024-07-17T07:15:22.355795+0000","last_fullsized":"2024-07-17T07:15:22.355795+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:58:14.175833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351491+0000","last_change":"2024-07-17T07:15:20.327351+0000","last_active":"2024-07-17T07:15:22.351491+0000","last_peered":"2024-07-17T07:15:22.351491+0000","last_clean":"2024-07-17T07:15:22.351491+0000","last_became_active":"2024-07-17T07:15:20.325850+0000","last_became_peered":"2024-07-17T07:15:20.325850+0000","last_unstale":"2024-07-17T07:15:22.351491+0000","last_undegraded":"2024-07-17T07:15:22.351491+0000","last_fullsized":"2024-07-17T07:15:22.351491+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:46.680516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350097+0000","last_change":"2024-07-17T07:15:21.363308+0000","last_active":"2024-07-17T07:15:22.350097+0000","last_peered":"2024-07-17T07:15:22.350097+0000","last_clean":"2024-07-17T07:15:22.350097+0000","last_became_active":"2024-07-17T07:15:21.359737+0000","last_became_peered":"2024-07-17T07:15:21.359737+0000","last_unstale":"2024-07-17T07:15:22.350097+0000","last_undegraded":"2024-07-17T07:15:22.350097+0000","last_fullsized":"2024-07-17T07:15:22.350097+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:10.155251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857610+0000","last_change":"2024-07-17T07:15:20.328422+0000","last_active":"2024-07-17T07:15:22.857610+0000","last_peered":"2024-07-17T07:15:22.857610+0000","last_clean":"2024-07-17T07:15:22.857610+0000","last_became_active":"2024-07-17T07:15:20.328046+0000","last_became_peered":"2024-07-17T07:15:20.328046+0000","last_unstale":"2024-07-17T07:15:22.857610+0000","last_undegraded":"2024-07-17T07:15:22.857610+0000","last_fullsized":"2024-07-17T07:15:22.857610+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:21:45.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857822+0000","last_change":"2024-07-17T07:15:21.347429+0000","last_active":"2024-07-17T07:15:22.857822+0000","last_peered":"2024-07-17T07:15:22.857822+0000","last_clean":"2024-07-17T07:15:22.857822+0000","last_became_active":"2024-07-17T07:15:21.347122+0000","last_became_peered":"2024-07-17T07:15:21.347122+0000","last_unstale":"2024-07-17T07:15:22.857822+0000","last_undegraded":"2024-07-17T07:15:22.857822+0000","last_fullsized":"2024-07-17T07:15:22.857822+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:28.082184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366471+0000","last_change":"2024-07-17T07:15:20.325328+0000","last_active":"2024-07-17T07:15:22.366471+0000","last_peered":"2024-07-17T07:15:22.366471+0000","last_clean":"2024-07-17T07:15:22.366471+0000","last_became_active":"2024-07-17T07:15:20.325027+0000","last_became_peered":"2024-07-17T07:15:20.325027+0000","last_unstale":"2024-07-17T07:15:22.366471+0000","last_undegraded":"2024-07-17T07:15:22.366471+0000","last_fullsized":"2024-07-17T07:15:22.366471+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:59.150908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351289+0000","last_change":"2024-07-17T07:15:21.353659+0000","last_active":"2024-07-17T07:15:22.351289+0000","last_peered":"2024-07-17T07:15:22.351289+0000","last_clean":"2024-07-17T07:15:22.351289+0000","last_became_active":"2024-07-17T07:15:21.352428+0000","last_became_peered":"2024-07-17T07:15:21.352428+0000","last_unstale":"2024-07-17T07:15:22.351289+0000","last_undegraded":"2024-07-17T07:15:22.351289+0000","last_fullsized":"2024-07-17T07:15:22.351289+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:02:42.175606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856143+0000","last_change":"2024-07-17T07:15:20.311558+0000","last_active":"2024-07-17T07:15:22.856143+0000","last_peered":"2024-07-17T07:15:22.856143+0000","last_clean":"2024-07-17T07:15:22.856143+0000","last_became_active":"2024-07-17T07:15:20.311261+0000","last_became_peered":"2024-07-17T07:15:20.311261+0000","last_unstale":"2024-07-17T07:15:22.856143+0000","last_undegraded":"2024-07-17T07:15:22.856143+0000","last_fullsized":"2024-07-17T07:15:22.856143+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:29:53.652834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350493+0000","last_change":"2024-07-17T07:15:21.360592+0000","last_active":"2024-07-17T07:15:22.350493+0000","last_peered":"2024-07-17T07:15:22.350493+0000","last_clean":"2024-07-17T07:15:22.350493+0000","last_became_active":"2024-07-17T07:15:21.360293+0000","last_became_peered":"2024-07-17T07:15:21.360293+0000","last_unstale":"2024-07-17T07:15:22.350493+0000","last_undegraded":"2024-07-17T07:15:22.350493+0000","last_fullsized":"2024-07-17T07:15:22.350493+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:48.551289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.367255+0000","last_change":"2024-07-17T07:15:20.328921+0000","last_active":"2024-07-17T07:15:22.367255+0000","last_peered":"2024-07-17T07:15:22.367255+0000","last_clean":"2024-07-17T07:15:22.367255+0000","last_became_active":"2024-07-17T07:15:20.328504+0000","last_became_peered":"2024-07-17T07:15:20.328504+0000","last_unstale":"2024-07-17T07:15:22.367255+0000","last_undegraded":"2024-07-17T07:15:22.367255+0000","last_fullsized":"2024-07-17T07:15:22.367255+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:37.255636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.362511+0000","last_change":"2024-07-17T07:15:21.345791+0000","last_active":"2024-07-17T07:15:22.362511+0000","last_peered":"2024-07-17T07:15:22.362511+0000","last_clean":"2024-07-17T07:15:22.362511+0000","last_became_active":"2024-07-17T07:15:21.345445+0000","last_became_peered":"2024-07-17T07:15:21.345445+0000","last_unstale":"2024-07-17T07:15:22.362511+0000","last_undegraded":"2024-07-17T07:15:22.362511+0000","last_fullsized":"2024-07-17T07:15:22.362511+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:55.510933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363511+0000","last_change":"2024-07-17T07:15:20.316641+0000","last_active":"2024-07-17T07:15:22.363511+0000","last_peered":"2024-07-17T07:15:22.363511+0000","last_clean":"2024-07-17T07:15:22.363511+0000","last_became_active":"2024-07-17T07:15:20.316349+0000","last_became_peered":"2024-07-17T07:15:20.316349+0000","last_unstale":"2024-07-17T07:15:22.363511+0000","last_undegraded":"2024-07-17T07:15:22.363511+0000","last_fullsized":"2024-07-17T07:15:22.363511+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:04.782928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856947+0000","last_change":"2024-07-17T07:15:21.346284+0000","last_active":"2024-07-17T07:15:22.856947+0000","last_peered":"2024-07-17T07:15:22.856947+0000","last_clean":"2024-07-17T07:15:22.856947+0000","last_became_active":"2024-07-17T07:15:21.346008+0000","last_became_peered":"2024-07-17T07:15:21.346008+0000","last_unstale":"2024-07-17T07:15:22.856947+0000","last_undegraded":"2024-07-17T07:15:22.856947+0000","last_fullsized":"2024-07-17T07:15:22.856947+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:30:09.798681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856821+0000","last_change":"2024-07-17T07:15:20.308133+0000","last_active":"2024-07-17T07:15:22.856821+0000","last_peered":"2024-07-17T07:15:22.856821+0000","last_clean":"2024-07-17T07:15:22.856821+0000","last_became_active":"2024-07-17T07:15:20.307828+0000","last_became_peered":"2024-07-17T07:15:20.307828+0000","last_unstale":"2024-07-17T07:15:22.856821+0000","last_undegraded":"2024-07-17T07:15:22.856821+0000","last_fullsized":"2024-07-17T07:15:22.856821+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:08.270674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353370+0000","last_change":"2024-07-17T07:15:21.339380+0000","last_active":"2024-07-17T07:15:22.353370+0000","last_peered":"2024-07-17T07:15:22.353370+0000","last_clean":"2024-07-17T07:15:22.353370+0000","last_became_active":"2024-07-17T07:15:21.338902+0000","last_became_peered":"2024-07-17T07:15:21.338902+0000","last_unstale":"2024-07-17T07:15:22.353370+0000","last_undegraded":"2024-07-17T07:15:22.353370+0000","last_fullsized":"2024-07-17T07:15:22.353370+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:10:42.214621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"141'3","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856828+0000","last_change":"2024-07-17T07:15:20.321376+0000","last_active":"2024-07-17T07:15:22.856828+0000","last_peered":"2024-07-17T07:15:22.856828+0000","last_clean":"2024-07-17T07:15:22.856828+0000","last_became_active":"2024-07-17T07:15:20.319525+0000","last_became_peered":"2024-07-17T07:15:20.319525+0000","last_unstale":"2024-07-17T07:15:22.856828+0000","last_undegraded":"2024-07-17T07:15:22.856828+0000","last_fullsized":"2024-07-17T07:15:22.856828+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:11.809586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857135+0000","last_change":"2024-07-17T07:15:21.348224+0000","last_active":"2024-07-17T07:15:22.857135+0000","last_peered":"2024-07-17T07:15:22.857135+0000","last_clean":"2024-07-17T07:15:22.857135+0000","last_became_active":"2024-07-17T07:15:21.347942+0000","last_became_peered":"2024-07-17T07:15:21.347942+0000","last_unstale":"2024-07-17T07:15:22.857135+0000","last_undegraded":"2024-07-17T07:15:22.857135+0000","last_fullsized":"2024-07-17T07:15:22.857135+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:11:09.866501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352059+0000","last_change":"2024-07-17T07:15:20.325459+0000","last_active":"2024-07-17T07:15:22.352059+0000","last_peered":"2024-07-17T07:15:22.352059+0000","last_clean":"2024-07-17T07:15:22.352059+0000","last_became_active":"2024-07-17T07:15:20.323729+0000","last_became_peered":"2024-07-17T07:15:20.323729+0000","last_unstale":"2024-07-17T07:15:22.352059+0000","last_undegraded":"2024-07-17T07:15:22.352059+0000","last_fullsized":"2024-07-17T07:15:22.352059+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:43.959180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363177+0000","last_change":"2024-07-17T07:15:21.343142+0000","last_active":"2024-07-17T07:15:22.363177+0000","last_peered":"2024-07-17T07:15:22.363177+0000","last_clean":"2024-07-17T07:15:22.363177+0000","last_became_active":"2024-07-17T07:15:21.340974+0000","last_became_peered":"2024-07-17T07:15:21.340974+0000","last_unstale":"2024-07-17T07:15:22.363177+0000","last_undegraded":"2024-07-17T07:15:22.363177+0000","last_fullsized":"2024-07-17T07:15:22.363177+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:04.958427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351867+0000","last_change":"2024-07-17T07:15:20.318539+0000","last_active":"2024-07-17T07:15:22.351867+0000","last_peered":"2024-07-17T07:15:22.351867+0000","last_clean":"2024-07-17T07:15:22.351867+0000","last_became_active":"2024-07-17T07:15:20.318258+0000","last_became_peered":"2024-07-17T07:15:20.318258+0000","last_unstale":"2024-07-17T07:15:22.351867+0000","last_undegraded":"2024-07-17T07:15:22.351867+0000","last_fullsized":"2024-07-17T07:15:22.351867+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:48.004065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350081+0000","last_change":"2024-07-17T07:15:21.359942+0000","last_active":"2024-07-17T07:15:22.350081+0000","last_peered":"2024-07-17T07:15:22.350081+0000","last_clean":"2024-07-17T07:15:22.350081+0000","last_became_active":"2024-07-17T07:15:21.355656+0000","last_became_peered":"2024-07-17T07:15:21.355656+0000","last_unstale":"2024-07-17T07:15:22.350081+0000","last_undegraded":"2024-07-17T07:15:22.350081+0000","last_fullsized":"2024-07-17T07:15:22.350081+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:10.693742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351648+0000","last_change":"2024-07-17T07:15:20.305414+0000","last_active":"2024-07-17T07:15:22.351648+0000","last_peered":"2024-07-17T07:15:22.351648+0000","last_clean":"2024-07-17T07:15:22.351648+0000","last_became_active":"2024-07-17T07:15:20.304966+0000","last_became_peered":"2024-07-17T07:15:20.304966+0000","last_unstale":"2024-07-17T07:15:22.351648+0000","last_undegraded":"2024-07-17T07:15:22.351648+0000","last_fullsized":"2024-07-17T07:15:22.351648+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:12:16.807079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839051+0000","last_change":"2024-07-17T07:15:21.333737+0000","last_active":"2024-07-17T07:15:22.839051+0000","last_peered":"2024-07-17T07:15:22.839051+0000","last_clean":"2024-07-17T07:15:22.839051+0000","last_became_active":"2024-07-17T07:15:21.332790+0000","last_became_peered":"2024-07-17T07:15:21.332790+0000","last_unstale":"2024-07-17T07:15:22.839051+0000","last_undegraded":"2024-07-17T07:15:22.839051+0000","last_fullsized":"2024-07-17T07:15:22.839051+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:58.172910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351678+0000","last_change":"2024-07-17T07:15:20.325281+0000","last_active":"2024-07-17T07:15:22.351678+0000","last_peered":"2024-07-17T07:15:22.351678+0000","last_clean":"2024-07-17T07:15:22.351678+0000","last_became_active":"2024-07-17T07:15:20.324064+0000","last_became_peered":"2024-07-17T07:15:20.324064+0000","last_unstale":"2024-07-17T07:15:22.351678+0000","last_undegraded":"2024-07-17T07:15:22.351678+0000","last_fullsized":"2024-07-17T07:15:22.351678+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:35.711552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857108+0000","last_change":"2024-07-17T07:15:21.362319+0000","last_active":"2024-07-17T07:15:22.857108+0000","last_peered":"2024-07-17T07:15:22.857108+0000","last_clean":"2024-07-17T07:15:22.857108+0000","last_became_active":"2024-07-17T07:15:21.362078+0000","last_became_peered":"2024-07-17T07:15:21.362078+0000","last_unstale":"2024-07-17T07:15:22.857108+0000","last_undegraded":"2024-07-17T07:15:22.857108+0000","last_fullsized":"2024-07-17T07:15:22.857108+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:07.610490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.352478+0000","last_change":"2024-07-17T07:15:20.324458+0000","last_active":"2024-07-17T07:15:22.352478+0000","last_peered":"2024-07-17T07:15:22.352478+0000","last_clean":"2024-07-17T07:15:22.352478+0000","last_became_active":"2024-07-17T07:15:20.323664+0000","last_became_peered":"2024-07-17T07:15:20.323664+0000","last_unstale":"2024-07-17T07:15:22.352478+0000","last_undegraded":"2024-07-17T07:15:22.352478+0000","last_fullsized":"2024-07-17T07:15:22.352478+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:24.872566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858438+0000","last_change":"2024-07-17T07:15:21.343465+0000","last_active":"2024-07-17T07:15:22.858438+0000","last_peered":"2024-07-17T07:15:22.858438+0000","last_clean":"2024-07-17T07:15:22.858438+0000","last_became_active":"2024-07-17T07:15:21.343098+0000","last_became_peered":"2024-07-17T07:15:21.343098+0000","last_unstale":"2024-07-17T07:15:22.858438+0000","last_undegraded":"2024-07-17T07:15:22.858438+0000","last_fullsized":"2024-07-17T07:15:22.858438+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:40.982831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839616+0000","last_change":"2024-07-17T07:15:20.318202+0000","last_active":"2024-07-17T07:15:22.839616+0000","last_peered":"2024-07-17T07:15:22.839616+0000","last_clean":"2024-07-17T07:15:22.839616+0000","last_became_active":"2024-07-17T07:15:20.315104+0000","last_became_peered":"2024-07-17T07:15:20.315104+0000","last_unstale":"2024-07-17T07:15:22.839616+0000","last_undegraded":"2024-07-17T07:15:22.839616+0000","last_fullsized":"2024-07-17T07:15:22.839616+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:55.564062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366886+0000","last_change":"2024-07-17T07:15:21.364268+0000","last_active":"2024-07-17T07:15:22.366886+0000","last_peered":"2024-07-17T07:15:22.366886+0000","last_clean":"2024-07-17T07:15:22.366886+0000","last_became_active":"2024-07-17T07:15:21.363448+0000","last_became_peered":"2024-07-17T07:15:21.363448+0000","last_unstale":"2024-07-17T07:15:22.366886+0000","last_undegraded":"2024-07-17T07:15:22.366886+0000","last_fullsized":"2024-07-17T07:15:22.366886+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:17:40.222069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355728+0000","last_change":"2024-07-17T07:15:20.324747+0000","last_active":"2024-07-17T07:15:22.355728+0000","last_peered":"2024-07-17T07:15:22.355728+0000","last_clean":"2024-07-17T07:15:22.355728+0000","last_became_active":"2024-07-17T07:15:20.324025+0000","last_became_peered":"2024-07-17T07:15:20.324025+0000","last_unstale":"2024-07-17T07:15:22.355728+0000","last_undegraded":"2024-07-17T07:15:22.355728+0000","last_fullsized":"2024-07-17T07:15:22.355728+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:36:04.696409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858052+0000","last_change":"2024-07-17T07:15:21.347549+0000","last_active":"2024-07-17T07:15:22.858052+0000","last_peered":"2024-07-17T07:15:22.858052+0000","last_clean":"2024-07-17T07:15:22.858052+0000","last_became_active":"2024-07-17T07:15:21.346019+0000","last_became_peered":"2024-07-17T07:15:21.346019+0000","last_unstale":"2024-07-17T07:15:22.858052+0000","last_undegraded":"2024-07-17T07:15:22.858052+0000","last_fullsized":"2024-07-17T07:15:22.858052+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:22:47.822313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355106+0000","last_change":"2024-07-17T07:15:20.323122+0000","last_active":"2024-07-17T07:15:22.355106+0000","last_peered":"2024-07-17T07:15:22.355106+0000","last_clean":"2024-07-17T07:15:22.355106+0000","last_became_active":"2024-07-17T07:15:20.322819+0000","last_became_peered":"2024-07-17T07:15:20.322819+0000","last_unstale":"2024-07-17T07:15:22.355106+0000","last_undegraded":"2024-07-17T07:15:22.355106+0000","last_fullsized":"2024-07-17T07:15:22.355106+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:28.179465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366497+0000","last_change":"2024-07-17T07:15:21.363482+0000","last_active":"2024-07-17T07:15:22.366497+0000","last_peered":"2024-07-17T07:15:22.366497+0000","last_clean":"2024-07-17T07:15:22.366497+0000","last_became_active":"2024-07-17T07:15:21.363203+0000","last_became_peered":"2024-07-17T07:15:21.363203+0000","last_unstale":"2024-07-17T07:15:22.366497+0000","last_undegraded":"2024-07-17T07:15:22.366497+0000","last_fullsized":"2024-07-17T07:15:22.366497+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:56:44.117607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366424+0000","last_change":"2024-07-17T07:15:20.325220+0000","last_active":"2024-07-17T07:15:22.366424+0000","last_peered":"2024-07-17T07:15:22.366424+0000","last_clean":"2024-07-17T07:15:22.366424+0000","last_became_active":"2024-07-17T07:15:20.324062+0000","last_became_peered":"2024-07-17T07:15:20.324062+0000","last_unstale":"2024-07-17T07:15:22.366424+0000","last_undegraded":"2024-07-17T07:15:22.366424+0000","last_fullsized":"2024-07-17T07:15:22.366424+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:05.485166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351309+0000","last_change":"2024-07-17T07:15:21.356502+0000","last_active":"2024-07-17T07:15:22.351309+0000","last_peered":"2024-07-17T07:15:22.351309+0000","last_clean":"2024-07-17T07:15:22.351309+0000","last_became_active":"2024-07-17T07:15:21.356202+0000","last_became_peered":"2024-07-17T07:15:21.356202+0000","last_unstale":"2024-07-17T07:15:22.351309+0000","last_undegraded":"2024-07-17T07:15:22.351309+0000","last_fullsized":"2024-07-17T07:15:22.351309+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:47.115649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354751+0000","last_change":"2024-07-17T07:15:20.326232+0000","last_active":"2024-07-17T07:15:22.354751+0000","last_peered":"2024-07-17T07:15:22.354751+0000","last_clean":"2024-07-17T07:15:22.354751+0000","last_became_active":"2024-07-17T07:15:20.325813+0000","last_became_peered":"2024-07-17T07:15:20.325813+0000","last_unstale":"2024-07-17T07:15:22.354751+0000","last_undegraded":"2024-07-17T07:15:22.354751+0000","last_fullsized":"2024-07-17T07:15:22.354751+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:58.900325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351514+0000","last_change":"2024-07-17T07:15:21.343431+0000","last_active":"2024-07-17T07:15:22.351514+0000","last_peered":"2024-07-17T07:15:22.351514+0000","last_clean":"2024-07-17T07:15:22.351514+0000","last_became_active":"2024-07-17T07:15:21.343134+0000","last_became_peered":"2024-07-17T07:15:21.343134+0000","last_unstale":"2024-07-17T07:15:22.351514+0000","last_undegraded":"2024-07-17T07:15:22.351514+0000","last_fullsized":"2024-07-17T07:15:22.351514+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:30:31.170962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351825+0000","last_change":"2024-07-17T07:15:20.321055+0000","last_active":"2024-07-17T07:15:22.351825+0000","last_peered":"2024-07-17T07:15:22.351825+0000","last_clean":"2024-07-17T07:15:22.351825+0000","last_became_active":"2024-07-17T07:15:20.320797+0000","last_became_peered":"2024-07-17T07:15:20.320797+0000","last_unstale":"2024-07-17T07:15:22.351825+0000","last_undegraded":"2024-07-17T07:15:22.351825+0000","last_fullsized":"2024-07-17T07:15:22.351825+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:57.570790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.856728+0000","last_change":"2024-07-17T07:15:21.349234+0000","last_active":"2024-07-17T07:15:22.856728+0000","last_peered":"2024-07-17T07:15:22.856728+0000","last_clean":"2024-07-17T07:15:22.856728+0000","last_became_active":"2024-07-17T07:15:21.348948+0000","last_became_peered":"2024-07-17T07:15:21.348948+0000","last_unstale":"2024-07-17T07:15:22.856728+0000","last_undegraded":"2024-07-17T07:15:22.856728+0000","last_fullsized":"2024-07-17T07:15:22.856728+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:12:17.812094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353618+0000","last_change":"2024-07-17T07:15:20.324274+0000","last_active":"2024-07-17T07:15:22.353618+0000","last_peered":"2024-07-17T07:15:22.353618+0000","last_clean":"2024-07-17T07:15:22.353618+0000","last_became_active":"2024-07-17T07:15:20.323610+0000","last_became_peered":"2024-07-17T07:15:20.323610+0000","last_unstale":"2024-07-17T07:15:22.353618+0000","last_undegraded":"2024-07-17T07:15:22.353618+0000","last_fullsized":"2024-07-17T07:15:22.353618+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:09.733002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.368239+0000","last_change":"2024-07-17T07:15:21.366652+0000","last_active":"2024-07-17T07:15:22.368239+0000","last_peered":"2024-07-17T07:15:22.368239+0000","last_clean":"2024-07-17T07:15:22.368239+0000","last_became_active":"2024-07-17T07:15:21.366321+0000","last_became_peered":"2024-07-17T07:15:21.366321+0000","last_unstale":"2024-07-17T07:15:22.368239+0000","last_undegraded":"2024-07-17T07:15:22.368239+0000","last_fullsized":"2024-07-17T07:15:22.368239+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:09.184001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366880+0000","last_change":"2024-07-17T07:15:20.328989+0000","last_active":"2024-07-17T07:15:22.366880+0000","last_peered":"2024-07-17T07:15:22.366880+0000","last_clean":"2024-07-17T07:15:22.366880+0000","last_became_active":"2024-07-17T07:15:20.328716+0000","last_became_peered":"2024-07-17T07:15:20.328716+0000","last_unstale":"2024-07-17T07:15:22.366880+0000","last_undegraded":"2024-07-17T07:15:22.366880+0000","last_fullsized":"2024-07-17T07:15:22.366880+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:33.319175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351334+0000","last_change":"2024-07-17T07:15:21.354338+0000","last_active":"2024-07-17T07:15:22.351334+0000","last_peered":"2024-07-17T07:15:22.351334+0000","last_clean":"2024-07-17T07:15:22.351334+0000","last_became_active":"2024-07-17T07:15:21.354059+0000","last_became_peered":"2024-07-17T07:15:21.354059+0000","last_unstale":"2024-07-17T07:15:22.351334+0000","last_undegraded":"2024-07-17T07:15:22.351334+0000","last_fullsized":"2024-07-17T07:15:22.351334+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:13.017982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.365852+0000","last_change":"2024-07-17T07:15:20.320142+0000","last_active":"2024-07-17T07:15:22.365852+0000","last_peered":"2024-07-17T07:15:22.365852+0000","last_clean":"2024-07-17T07:15:22.365852+0000","last_became_active":"2024-07-17T07:15:20.319024+0000","last_became_peered":"2024-07-17T07:15:20.319024+0000","last_unstale":"2024-07-17T07:15:22.365852+0000","last_undegraded":"2024-07-17T07:15:22.365852+0000","last_fullsized":"2024-07-17T07:15:22.365852+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:12.012967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.355975+0000","last_change":"2024-07-17T07:15:21.357390+0000","last_active":"2024-07-17T07:15:22.355975+0000","last_peered":"2024-07-17T07:15:22.355975+0000","last_clean":"2024-07-17T07:15:22.355975+0000","last_became_active":"2024-07-17T07:15:21.357081+0000","last_became_peered":"2024-07-17T07:15:21.357081+0000","last_unstale":"2024-07-17T07:15:22.355975+0000","last_undegraded":"2024-07-17T07:15:22.355975+0000","last_fullsized":"2024-07-17T07:15:22.355975+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:39.601161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.363730+0000","last_change":"2024-07-17T07:15:20.317212+0000","last_active":"2024-07-17T07:15:22.363730+0000","last_peered":"2024-07-17T07:15:22.363730+0000","last_clean":"2024-07-17T07:15:22.363730+0000","last_became_active":"2024-07-17T07:15:20.316798+0000","last_became_peered":"2024-07-17T07:15:20.316798+0000","last_unstale":"2024-07-17T07:15:22.363730+0000","last_undegraded":"2024-07-17T07:15:22.363730+0000","last_fullsized":"2024-07-17T07:15:22.363730+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:42.061248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.839877+0000","last_change":"2024-07-17T07:15:21.355713+0000","last_active":"2024-07-17T07:15:22.839877+0000","last_peered":"2024-07-17T07:15:22.839877+0000","last_clean":"2024-07-17T07:15:22.839877+0000","last_became_active":"2024-07-17T07:15:21.353960+0000","last_became_peered":"2024-07-17T07:15:21.353960+0000","last_unstale":"2024-07-17T07:15:22.839877+0000","last_undegraded":"2024-07-17T07:15:22.839877+0000","last_fullsized":"2024-07-17T07:15:22.839877+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:52.349147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.368159+0000","last_change":"2024-07-17T07:15:20.330675+0000","last_active":"2024-07-17T07:15:22.368159+0000","last_peered":"2024-07-17T07:15:22.368159+0000","last_clean":"2024-07-17T07:15:22.368159+0000","last_became_active":"2024-07-17T07:15:20.330121+0000","last_became_peered":"2024-07-17T07:15:20.330121+0000","last_unstale":"2024-07-17T07:15:22.368159+0000","last_undegraded":"2024-07-17T07:15:22.368159+0000","last_fullsized":"2024-07-17T07:15:22.368159+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:16.773874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.367004+0000","last_change":"2024-07-17T07:15:21.361747+0000","last_active":"2024-07-17T07:15:22.367004+0000","last_peered":"2024-07-17T07:15:22.367004+0000","last_clean":"2024-07-17T07:15:22.367004+0000","last_became_active":"2024-07-17T07:15:21.361428+0000","last_became_peered":"2024-07-17T07:15:21.361428+0000","last_unstale":"2024-07-17T07:15:22.367004+0000","last_undegraded":"2024-07-17T07:15:22.367004+0000","last_fullsized":"2024-07-17T07:15:22.367004+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:56:04.435552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.857915+0000","last_change":"2024-07-17T07:15:20.321279+0000","last_active":"2024-07-17T07:15:22.857915+0000","last_peered":"2024-07-17T07:15:22.857915+0000","last_clean":"2024-07-17T07:15:22.857915+0000","last_became_active":"2024-07-17T07:15:20.321000+0000","last_became_peered":"2024-07-17T07:15:20.321000+0000","last_unstale":"2024-07-17T07:15:22.857915+0000","last_undegraded":"2024-07-17T07:15:22.857915+0000","last_fullsized":"2024-07-17T07:15:22.857915+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:48.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.351509+0000","last_change":"2024-07-17T07:15:21.340731+0000","last_active":"2024-07-17T07:15:22.351509+0000","last_peered":"2024-07-17T07:15:22.351509+0000","last_clean":"2024-07-17T07:15:22.351509+0000","last_became_active":"2024-07-17T07:15:21.340445+0000","last_became_peered":"2024-07-17T07:15:21.340445+0000","last_unstale":"2024-07-17T07:15:22.351509+0000","last_undegraded":"2024-07-17T07:15:22.351509+0000","last_fullsized":"2024-07-17T07:15:22.351509+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:14:05.946578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.858584+0000","last_change":"2024-07-17T07:15:20.317468+0000","last_active":"2024-07-17T07:15:22.858584+0000","last_peered":"2024-07-17T07:15:22.858584+0000","last_clean":"2024-07-17T07:15:22.858584+0000","last_became_active":"2024-07-17T07:15:20.317162+0000","last_became_peered":"2024-07-17T07:15:20.317162+0000","last_unstale":"2024-07-17T07:15:22.858584+0000","last_undegraded":"2024-07-17T07:15:22.858584+0000","last_fullsized":"2024-07-17T07:15:22.858584+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:19:51.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354857+0000","last_change":"2024-07-17T07:15:21.365870+0000","last_active":"2024-07-17T07:15:22.354857+0000","last_peered":"2024-07-17T07:15:22.354857+0000","last_clean":"2024-07-17T07:15:22.354857+0000","last_became_active":"2024-07-17T07:15:21.365535+0000","last_became_peered":"2024-07-17T07:15:21.365535+0000","last_unstale":"2024-07-17T07:15:22.354857+0000","last_undegraded":"2024-07-17T07:15:22.354857+0000","last_fullsized":"2024-07-17T07:15:22.354857+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:26:46.042719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.366403+0000","last_change":"2024-07-17T07:15:20.329247+0000","last_active":"2024-07-17T07:15:22.366403+0000","last_peered":"2024-07-17T07:15:22.366403+0000","last_clean":"2024-07-17T07:15:22.366403+0000","last_became_active":"2024-07-17T07:15:20.328862+0000","last_became_peered":"2024-07-17T07:15:20.328862+0000","last_unstale":"2024-07-17T07:15:22.366403+0000","last_undegraded":"2024-07-17T07:15:22.366403+0000","last_fullsized":"2024-07-17T07:15:22.366403+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:00.154490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350673+0000","last_change":"2024-07-17T07:15:21.355429+0000","last_active":"2024-07-17T07:15:22.350673+0000","last_peered":"2024-07-17T07:15:22.350673+0000","last_clean":"2024-07-17T07:15:22.350673+0000","last_became_active":"2024-07-17T07:15:21.355166+0000","last_became_peered":"2024-07-17T07:15:21.355166+0000","last_unstale":"2024-07-17T07:15:22.350673+0000","last_undegraded":"2024-07-17T07:15:22.350673+0000","last_fullsized":"2024-07-17T07:15:22.350673+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:50.781290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.353640+0000","last_change":"2024-07-17T07:15:20.326694+0000","last_active":"2024-07-17T07:15:22.353640+0000","last_peered":"2024-07-17T07:15:22.353640+0000","last_clean":"2024-07-17T07:15:22.353640+0000","last_became_active":"2024-07-17T07:15:20.325864+0000","last_became_peered":"2024-07-17T07:15:20.325864+0000","last_unstale":"2024-07-17T07:15:22.353640+0000","last_undegraded":"2024-07-17T07:15:22.353640+0000","last_fullsized":"2024-07-17T07:15:22.353640+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:53.005336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354597+0000","last_change":"2024-07-17T07:15:21.348056+0000","last_active":"2024-07-17T07:15:22.354597+0000","last_peered":"2024-07-17T07:15:22.354597+0000","last_clean":"2024-07-17T07:15:22.354597+0000","last_became_active":"2024-07-17T07:15:21.347478+0000","last_became_peered":"2024-07-17T07:15:21.347478+0000","last_unstale":"2024-07-17T07:15:22.354597+0000","last_undegraded":"2024-07-17T07:15:22.354597+0000","last_fullsized":"2024-07-17T07:15:22.354597+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:20.280314+0000","last_clean_scrub_stamp":"2024-07-17T07:15:20.280314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:38:06.575395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.350814+0000","last_change":"2024-07-17T07:15:20.313698+0000","last_active":"2024-07-17T07:15:22.350814+0000","last_peered":"2024-07-17T07:15:22.350814+0000","last_clean":"2024-07-17T07:15:22.350814+0000","last_became_active":"2024-07-17T07:15:20.313293+0000","last_became_peered":"2024-07-17T07:15:20.313293+0000","last_unstale":"2024-07-17T07:15:22.350814+0000","last_undegraded":"2024-07-17T07:15:22.350814+0000","last_fullsized":"2024-07-17T07:15:22.350814+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:15:19.275299+0000","last_clean_scrub_stamp":"2024-07-17T07:15:19.275299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:46.006216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"132'2434","reported_seq":2919,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-07-17T07:15:22.354011+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:15:22.354011+0000","last_peered":"2024-07-17T07:15:22.354011+0000","last_clean":"2024-07-17T07:15:22.354011+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:15:22.354011+0000","last_undegraded":"2024-07-17T07:15:22.354011+0000","last_fullsized":"2024-07-17T07:15:22.354011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":227,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":727,"num_read_kb":1935,"num_write":227,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2434,"ondisk_log_size":2434,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542388,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55704,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316136,"statfs":{"total":96636764160,"available":96579723264,"internally_reserved":0,"allocated":2367488,"data_stored":1525071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509683,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":1761280,"data_stored":934509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509684,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55120,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316720,"statfs":{"total":96636764160,"available":96580321280,"internally_reserved":0,"allocated":1765376,"data_stored":934087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542389,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1769472,"data_stored":935539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509683,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1773568,"data_stored":935573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542387,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59264,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312576,"statfs":{"total":96636764160,"available":96576077824,"internally_reserved":0,"allocated":1757184,"data_stored":933927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":1761280,"data_stored":933949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509683,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29320,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342520,"statfs":{"total":96636764160,"available":96606740480,"internally_reserved":0,"allocated":2355200,"data_stored":1524317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:15:26.386 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:15:26.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.645+0000 7fcfc1200640 1 -- 172.21.15.44:0/3046222476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa00052e0 msgr2=0x7fcfa000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.645+0000 7fcfc1200640 1 --2- 172.21.15.44:0/3046222476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa00052e0 0x7fcfa000b9c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4001d90 tx=0x7fcfa0005830 comp rx=0 tx=0).stop 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.645+0000 7fcfc1200640 1 -- 172.21.15.44:0/3046222476 shutdown_connections 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.646+0000 7fcfc1200640 1 --2- 172.21.15.44:0/3046222476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa00052e0 0x7fcfa000b9c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.646+0000 7fcfc1200640 1 -- 172.21.15.44:0/3046222476 >> 172.21.15.44:0/3046222476 conn(0x7fcfbc0b3090 msgr2=0x7fcfbc0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.646+0000 7fcfc1200640 1 -- 172.21.15.44:0/3046222476 shutdown_connections 2024-07-17T07:15:26.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.646+0000 7fcfc1200640 1 -- 172.21.15.44:0/3046222476 wait complete. 2024-07-17T07:15:26.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.647+0000 7fcfc1200640 1 Processor -- start 2024-07-17T07:15:26.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.647+0000 7fcfc1200640 1 -- start start 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 0x7fcfbc1f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 0x7fcfbc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc0b3f10 con 0x7fcfbc06d050 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc0b3d90 con 0x7fcfbc0cc410 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.648+0000 7fcfc1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcfbc0b5530 con 0x7fcfa00052e0 2024-07-17T07:15:26.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39234/0 (socket says 172.21.15.44:39234) 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 -- 172.21.15.44:0/1900038840 learned_addr learned my addr 172.21.15.44:0/1900038840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbbe00640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 0x7fcfbc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbb400640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 0x7fcfbc1f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 -- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 msgr2=0x7fcfbc1f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 0x7fcfbc1f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 -- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 msgr2=0x7fcfbc0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 0x7fcfbc0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbaa00640 1 -- 172.21.15.44:0/1900038840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc06d8a0 con 0x7fcfbc06d050 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbb400640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 0x7fcfbc1f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:26.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.649+0000 7fcfbbe00640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 0x7fcfbc0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfbaa00640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40117d0 tx=0x7fcfa4011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004460 con 0x7fcfbc06d050 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa4016070 con 0x7fcfbc06d050 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400fb60 con 0x7fcfbc06d050 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc06db80 con 0x7fcfbc06d050 2024-07-17T07:15:26.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.650+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc14e500 con 0x7fcfbc06d050 2024-07-17T07:15:26.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.651+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc0d23d0 con 0x7fcfbc06d050 2024-07-17T07:15:26.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.653+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fcfa4018030 con 0x7fcfbc06d050 2024-07-17T07:15:26.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.654+0000 7fcfb3400640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 0x7fcf980791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.654+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fcfa40990b0 con 0x7fcfbc06d050 2024-07-17T07:15:26.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.654+0000 7fcfbb400640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 0x7fcf980791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:26.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.655+0000 7fcfbb400640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 0x7fcf980791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001860 tx=0x7fcfac001bb0 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:26.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.658+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcfa4062020 con 0x7fcfbc06d050 2024-07-17T07:15:26.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.952+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcfbc0b5a20 con 0x7fcfbc06d050 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.957+0000 7fcfb3400640 1 -- 172.21.15.44:0/1900038840 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) ==== 190+0+147 (secure 0 0 0) 0x7fcfa4061920 con 0x7fcfbc06d050 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:15:26.957 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 msgr2=0x7fcf980791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 0x7fcf980791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001860 tx=0x7fcfac001bb0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 msgr2=0x7fcfbc1561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40117d0 tx=0x7fcfa4011c90 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 shutdown_connections 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fcf98076d00 0x7fcf980791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfbc0cc410 0x7fcfbc0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfbc06d050 0x7fcfbc1561a0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 --2- 172.21.15.44:0/1900038840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfa00052e0 0x7fcfbc1f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.960+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 >> 172.21.15.44:0/1900038840 conn(0x7fcfbc0b3090 msgr2=0x7fcfbc0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.961+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 shutdown_connections 2024-07-17T07:15:26.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:26.961+0000 7fcfc1200640 1 -- 172.21.15.44:0/1900038840 wait complete. 2024-07-17T07:15:26.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.190+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2026913182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994003aa0 msgr2=0x7fa994003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.190+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2026913182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994003aa0 0x7fa994003ec0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009b20 tx=0x7fa994005830 comp rx=0 tx=0).stop 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.191+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2026913182 shutdown_connections 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.191+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2026913182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa994003aa0 0x7fa994003ec0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.191+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2026913182 >> 172.21.15.44:0/2026913182 conn(0x7fa9b40b0fb0 msgr2=0x7fa9b40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.191+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2026913182 shutdown_connections 2024-07-17T07:15:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.191+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2026913182 wait complete. 2024-07-17T07:15:27.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.192+0000 7fa9b9e00640 1 Processor -- start 2024-07-17T07:15:27.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.193+0000 7fa9b9e00640 1 -- start start 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa994003aa0 0x7fa9b40b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41e93a0 0x7fa9b4152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b40ac9e0 con 0x7fa9b40737f0 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b40ac860 con 0x7fa9b41e93a0 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b406f020 con 0x7fa994003aa0 2024-07-17T07:15:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39262/0 (socket says 172.21.15.44:39262) 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 -- 172.21.15.44:0/2509213127 learned_addr learned my addr 172.21.15.44:0/2509213127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2a00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa994003aa0 0x7fa9b40b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b3400640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41e93a0 0x7fa9b4152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 -- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa994003aa0 msgr2=0x7fa9b40b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa994003aa0 0x7fa9b40b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 -- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41e93a0 msgr2=0x7fa9b4152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41e93a0 0x7fa9b4152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.194+0000 7fa9b2000640 1 -- 172.21.15.44:0/2509213127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41e8be0 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9b2000640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00db70 tx=0x7fa99c00b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00de60 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa99c003970 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c003da0 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b4073d60 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.195+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b4074320 con 0x7fa9b40737f0 2024-07-17T07:15:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.196+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b4153e40 con 0x7fa9b40737f0 2024-07-17T07:15:27.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.198+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7fa99c006110 con 0x7fa9b40737f0 2024-07-17T07:15:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.199+0000 7fa9ab400640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 0x7fa990079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:27.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.199+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fa99c0991c0 con 0x7fa9b40737f0 2024-07-17T07:15:27.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.199+0000 7fa9b2a00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 0x7fa990079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:27.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.200+0000 7fa9b2a00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 0x7fa990079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4035000 tx=0x7fa9a402d040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:27.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.203+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa99c062130 con 0x7fa9b40737f0 2024-07-17T07:15:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.462+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9b40b46e0 con 0x7fa9b40737f0 2024-07-17T07:15:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.463+0000 7fa9ab400640 1 -- 172.21.15.44:0/2509213127 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7fa99c061a30 con 0x7fa9b40737f0 2024-07-17T07:15:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:27.464 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.466+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 msgr2=0x7fa990079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.466+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 0x7fa990079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4035000 tx=0x7fa9a402d040 comp rx=0 tx=0).stop 2024-07-17T07:15:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.466+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 msgr2=0x7fa9b406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.466+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00db70 tx=0x7fa99c00b620 comp rx=0 tx=0).stop 2024-07-17T07:15:27.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.466+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 shutdown_connections 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fa990076cd0 0x7fa990079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9b41e93a0 0x7fa9b4152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9b40737f0 0x7fa9b406e180 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 --2- 172.21.15.44:0/2509213127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa994003aa0 0x7fa9b40b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 >> 172.21.15.44:0/2509213127 conn(0x7fa9b40b0fb0 msgr2=0x7fa9b40af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 shutdown_connections 2024-07-17T07:15:27.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:27.467+0000 7fa9b9e00640 1 -- 172.21.15.44:0/2509213127 wait complete. 2024-07-17T07:15:27.888 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:15:20.777186+0000', 'modified': '2024-07-17T07:15:23.867664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6619': {'gid': 6619, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/3810295277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 3810295277}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 3810295277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6619, 'qdb_cluster': [6619]} 2024-07-17T07:15:27.888 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:15:27.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 -- 172.21.15.44:0/1590089278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 msgr2=0x7f19c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 --2- 172.21.15.44:0/1590089278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19c4004c80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f19e0161550 tx=0x7f19c4005830 comp rx=0 tx=0).stop 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 -- 172.21.15.44:0/1590089278 shutdown_connections 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 --2- 172.21.15.44:0/1590089278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19c4004c80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 -- 172.21.15.44:0/1590089278 >> 172.21.15.44:0/1590089278 conn(0x7f19e00b0eb0 msgr2=0x7f19e00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 -- 172.21.15.44:0/1590089278 shutdown_connections 2024-07-17T07:15:28.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.116+0000 7f19e6e00640 1 -- 172.21.15.44:0/1590089278 wait complete. 2024-07-17T07:15:28.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.118+0000 7f19e6e00640 1 Processor -- start 2024-07-17T07:15:28.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.118+0000 7f19e6e00640 1 -- start start 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19e01f8000 0x7f19e0073360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19e0072630 0x7f19e0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19e00ac780 con 0x7f19c4004800 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19e00ac600 con 0x7f19e0072630 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f19e00ac900 con 0x7f19e01f8000 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39278/0 (socket says 172.21.15.44:39278) 2024-07-17T07:15:28.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 -- 172.21.15.44:0/3101033284 learned_addr learned my addr 172.21.15.44:0/3101033284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 -- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19e01f8000 msgr2=0x7f19e0073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19e01f8000 0x7f19e0073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19e0072630 0x7f19e0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 -- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19e0072630 msgr2=0x7f19e0073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.119+0000 7f19dfe00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19e0072630 0x7f19e0073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19dfe00640 1 -- 172.21.15.44:0/3101033284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e00738a0 con 0x7f19c4004800 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19dfe00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f19d000d920 tx=0x7f19d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19dcc00640 1 -- 172.21.15.44:0/3101033284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0001870 con 0x7f19c4004800 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19dcc00640 1 -- 172.21.15.44:0/3101033284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d000b830 con 0x7f19c4004800 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19dcc00640 1 -- 172.21.15.44:0/3101033284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d000bc60 con 0x7f19c4004800 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e01f47e0 con 0x7f19c4004800 2024-07-17T07:15:28.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.120+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01f4da0 con 0x7f19c4004800 2024-07-17T07:15:28.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.123+0000 7f19dcc00640 1 -- 172.21.15.44:0/3101033284 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f19d00035f0 con 0x7f19c4004800 2024-07-17T07:15:28.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.124+0000 7f19e9d90740 1 Processor -- start 2024-07-17T07:15:28.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.124+0000 7f19e9d90740 1 -- start start 2024-07-17T07:15:28.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.124+0000 7f19dcc00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 0x7f19bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.124+0000 7f19dcc00640 1 -- 172.21.15.44:0/3101033284 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f19d0098de0 con 0x7f19c4004800 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79f940a0 0x558f79fb9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fb95b0 0x558f79fb9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f79f04d60 con 0x558f79fbcde0 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f79ef4260 con 0x558f79fb95b0 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558f79ef82a0 con 0x558f79f940a0 2024-07-17T07:15:28.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 0x7f19bc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39284/0 (socket says 172.21.15.44:39284) 2024-07-17T07:15:28.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.125+0000 7f19dfe00640 1 -- 172.21.15.44:0/114150288 learned_addr learned my addr 172.21.15.44:0/114150288 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19d7400640 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79f940a0 0x558f79fb9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 -- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79f940a0 msgr2=0x558f79fb9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79f940a0 0x558f79fb9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 -- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fb95b0 msgr2=0x558f79fb9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fb95b0 0x558f79fb9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 -- 172.21.15.44:0/114150288 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f79c58850 con 0x558f79fbcde0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19dfe00640 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f19d005d6d0 tx=0x7f19d0061550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584a65451754f61b server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19d4c00640 1 -- 172.21.15.44:0/114150288 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0061650 con 0x558f79fbcde0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19d4c00640 1 -- 172.21.15.44:0/114150288 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d0065c50 con 0x558f79fbcde0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.126+0000 7f19d4c00640 1 -- 172.21.15.44:0/114150288 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d006a670 con 0x558f79fbcde0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 -- 172.21.15.44:0/114150288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 msgr2=0x558f79fbd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f19d005d6d0 tx=0x7f19d0061550 comp rx=0 tx=0).stop 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 0x7f19bc0791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19e0070b60 tx=0x7f19d8009f40 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 -- 172.21.15.44:0/114150288 shutdown_connections 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fb95b0 0x558f79fb9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79f940a0 0x558f79fb9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 --2- 172.21.15.44:0/114150288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fbcde0 0x558f79fbd1c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 -- 172.21.15.44:0/114150288 >> 172.21.15.44:0/114150288 conn(0x558f79f8f280 msgr2=0x558f79f92ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 -- 172.21.15.44:0/114150288 shutdown_connections 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.127+0000 7f19e9d90740 1 -- 172.21.15.44:0/114150288 wait complete. 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.128+0000 7f19e9d90740 1 -- start start 2024-07-17T07:15:28.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.128+0000 7f19e9d90740 1 Processor -- start 2024-07-17T07:15:28.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f79c67620 con 0x558f79fe7df0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f79e9bf90 con 0x558f79fbcde0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558f79f8d980 con 0x558f79fb95b0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e9d90740 10 client.0 _ll_register_callbacks cb 0x558f79ec1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39298/0 (socket says 172.21.15.44:39298) 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.129+0000 7f19e4e00640 1 -- 172.21.15.44:0/2931034346 learned_addr learned my addr 172.21.15.44:0/2931034346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19d7400640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19dfe00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 -- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 msgr2=0x558f79fce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 -- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 msgr2=0x558f79f8c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f8c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 -- 172.21.15.44:0/2931034346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f79ccf6b0 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19dfe00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19e4e00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f19e015b670 tx=0x7f19d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c00a0b54a6d6e78 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19cd600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.130+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8022070 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f19d801e650 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f79ecb900 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f19d802d050 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 10 client.8043 fetch_fsmap learned FSMap version 70 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x558f79fb1410 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558f79ecb900 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f19d8030020 con 0x558f79fe7df0 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 10 client.8043 fetch_fsmap finished waiting for FSMap version 70 2024-07-17T07:15:28.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 10 client.8043 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 20 client.8043 populate_metadata read hostname 'smithi044' 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79feade0 0x558f79fb2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 4 client.8043 mds_command: new command op to 6619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.131+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 --> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f79f06cb0 con 0x558f79feade0 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.132+0000 7f19dfe00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79feade0 0x558f79fb2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.132+0000 7f19dfe00640 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79feade0 0x558f79fb2820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b930ab636fee761 server_cookie=a71791f0dc80f99a in_seq=0 out_seq=0 2024-07-17T07:15:28.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.132+0000 7f19cd600640 10 client.8043 ms_handle_connect on v2:172.21.15.44:6833/3810295277 2024-07-17T07:15:28.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.133+0000 7f19cd600640 1 -- 172.21.15.44:0/2931034346 <== mds.0 v2:172.21.15.44:6833/3810295277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f79f06cb0 con 0x558f79feade0 2024-07-17T07:15:28.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.133+0000 7f19cd600640 10 client.8043 handle_command_reply: tid=0 2024-07-17T07:15:28.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.133+0000 7f19e9d90740 1 client.8043 shutdown 2024-07-17T07:15:28.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 msgr2=0x558f79f8c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f19e015b670 tx=0x7f19d8004a40 comp rx=0 tx=0).stop 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 shutdown_connections 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79feade0 0x558f79fb2820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79f8c6d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 --2- 172.21.15.44:0/2931034346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 >> 172.21.15.44:0/2931034346 conn(0x558f79f940a0 msgr2=0x558f79f92b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19cd600640 0 client.8043 ms_handle_reset on v2:172.21.15.44:6833/3810295277 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 shutdown_connections 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.134+0000 7f19e9d90740 1 -- 172.21.15.44:0/2931034346 wait complete. 2024-07-17T07:15:28.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.135+0000 7f19e9d90740 20 client.8043 trim_cache size 0 max 16384 2024-07-17T07:15:28.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.155+0000 7f19e9d90740 1 Processor -- start 2024-07-17T07:15:28.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.155+0000 7f19e9d90740 1 -- start start 2024-07-17T07:15:28.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79f92b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0201d40 con 0x558f79fe7df0 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0200d20 con 0x558f79fbcde0 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.156+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f19e0151600 con 0x558f79fb95b0 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39304/0 (socket says 172.21.15.44:39304) 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 -- 172.21.15.44:0/3326953579 learned_addr learned my addr 172.21.15.44:0/3326953579 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19d7400640 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19dfe00640 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79f92b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 -- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 msgr2=0x558f79f92b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79f92b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 -- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 msgr2=0x558f79f930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.157+0000 7f19e4e00640 1 -- 172.21.15.44:0/3326953579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f79fb7850 con 0x558f79fe7df0 2024-07-17T07:15:28.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.158+0000 7f19e4e00640 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009950 tx=0x7f19d8009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba554f9a8467de7b server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.158+0000 7f19cfe00640 1 -- 172.21.15.44:0/3326953579 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8022050 con 0x558f79fe7df0 2024-07-17T07:15:28.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.158+0000 7f19cfe00640 1 -- 172.21.15.44:0/3326953579 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d801e540 con 0x558f79fe7df0 2024-07-17T07:15:28.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.158+0000 7f19cfe00640 1 -- 172.21.15.44:0/3326953579 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80276a0 con 0x558f79fe7df0 2024-07-17T07:15:28.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- 172.21.15.44:0/3326953579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 msgr2=0x558f79ff0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009950 tx=0x7f19d8009980 comp rx=0 tx=0).stop 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- 172.21.15.44:0/3326953579 shutdown_connections 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0b00 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f79f930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 --2- 172.21.15.44:0/3326953579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79f92b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- 172.21.15.44:0/3326953579 >> 172.21.15.44:0/3326953579 conn(0x558f79f940a0 msgr2=0x558f79fb7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- 172.21.15.44:0/3326953579 shutdown_connections 2024-07-17T07:15:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- 172.21.15.44:0/3326953579 wait complete. 2024-07-17T07:15:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.159+0000 7f19e9d90740 1 -- start start 2024-07-17T07:15:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.160+0000 7f19e9d90740 1 Processor -- start 2024-07-17T07:15:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f7a03f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f79feac60 con 0x558f79fe7df0 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0200690 con 0x558f79fbcde0 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558f79feb280 con 0x558f79fb95b0 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e9d90740 10 client.0 _ll_register_callbacks cb 0x7f19d8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:15:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39310/0 (socket says 172.21.15.44:39310) 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 -- 172.21.15.44:0/1452891640 learned_addr learned my addr 172.21.15.44:0/1452891640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19d7400640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f7a03f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19dfe00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 -- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 msgr2=0x558f79fb7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fb7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 -- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 msgr2=0x558f7a03f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f7a03f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.161+0000 7f19e4e00640 1 -- 172.21.15.44:0/1452891640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f79f9e570 con 0x558f79fe7df0 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19d7400640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f7a03f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19e4e00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099f0 tx=0x7f19d8028760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f6d0b85c281c95 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19cd600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:15:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8022040 con 0x558f79fe7df0 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d801e540 con 0x558f79fe7df0 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f19d8004ab0 con 0x558f79fe7df0 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f19d8048050 con 0x558f79fe7df0 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19e9d90740 10 client.8049 fetch_fsmap learned FSMap version 70 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.162+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x558f79fed840 con 0x558f79fe7df0 2024-07-17T07:15:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d801e540 con 0x558f79fe7df0 2024-07-17T07:15:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f19d804b020 con 0x558f79fe7df0 2024-07-17T07:15:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 10 client.8049 fetch_fsmap finished waiting for FSMap version 70 2024-07-17T07:15:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 10 client.8049 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T07:15:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 20 client.8049 populate_metadata read hostname 'smithi044' 2024-07-17T07:15:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79ffa870 0x558f79f90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 4 client.8049 mds_command: new command op to 6619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:15:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 --> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] -- command(tid 0: {"prefix": "status"}) -- 0x558f79f02750 con 0x558f79ffa870 2024-07-17T07:15:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.163+0000 7f19dfe00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79ffa870 0x558f79f90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19dfe00640 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79ffa870 0x558f79f90700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5c95fea987f827 server_cookie=3d13f10cdc1e16b5 in_seq=0 out_seq=0 2024-07-17T07:15:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19cd600640 10 client.8049 ms_handle_connect on v2:172.21.15.44:6833/3810295277 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19cd600640 1 -- 172.21.15.44:0/1452891640 <== mds.0 v2:172.21.15.44:6833/3810295277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x558f79f02750 con 0x558f79ffa870 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19cd600640 10 client.8049 handle_command_reply: tid=0 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 client.8049 shutdown 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 msgr2=0x558f79ff0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f19d80099f0 tx=0x7f19d8028760 comp rx=0 tx=0).stop 2024-07-17T07:15:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 shutdown_connections 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:6833/3810295277,v1:172.21.15.44:6835/3810295277] conn(0x558f79ffa870 0x558f79f90700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f79fe7df0 0x558f79ff0aa0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f79fbcde0 0x558f7a03f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 --2- 172.21.15.44:0/1452891640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x558f79fb95b0 0x558f79fb7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19cd600640 0 client.8049 ms_handle_reset on v2:172.21.15.44:6833/3810295277 2024-07-17T07:15:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 >> 172.21.15.44:0/1452891640 conn(0x558f79f940a0 msgr2=0x558f79fcdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 shutdown_connections 2024-07-17T07:15:28.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.164+0000 7f19e9d90740 1 -- 172.21.15.44:0/1452891640 wait complete. 2024-07-17T07:15:28.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e9d90740 20 client.8049 trim_cache size 0 max 16384 2024-07-17T07:15:28.179 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6619, 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:15:28.180 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:15:28.181 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8262318410000002, 2024-07-17T07:15:28.181 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 70, 2024-07-17T07:15:28.181 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 142, 2024-07-17T07:15:28.181 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 142, 2024-07-17T07:15:28.181 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.793319415999999 2024-07-17T07:15:28.182 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:15:28.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 msgr2=0x7f19bc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 0x7f19bc0791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19e0070b60 tx=0x7f19d8009f40 comp rx=0 tx=0).stop 2024-07-17T07:15:28.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 msgr2=0x7f19e01f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f19d000d920 tx=0x7f19d000dde0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 shutdown_connections 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f19bc076d00 0x7f19bc0791c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19e0072630 0x7f19e0073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19e01f8000 0x7f19e0073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 --2- 172.21.15.44:0/3101033284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19c4004800 0x7f19e01f4070 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 >> 172.21.15.44:0/3101033284 conn(0x7f19e00b0eb0 msgr2=0x7f19e00b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 shutdown_connections 2024-07-17T07:15:28.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.165+0000 7f19e6e00640 1 -- 172.21.15.44:0/3101033284 wait complete. 2024-07-17T07:15:28.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 -- 172.21.15.44:0/4099768318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 msgr2=0x7ff8180ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 --2- 172.21.15.44:0/4099768318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8180ac3e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff818070a70 tx=0x7ff80402a560 comp rx=0 tx=0).stop 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 -- 172.21.15.44:0/4099768318 shutdown_connections 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 --2- 172.21.15.44:0/4099768318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8180ac920 0x7ff81813ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 --2- 172.21.15.44:0/4099768318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8180ac3e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 --2- 172.21.15.44:0/4099768318 >> v2:172.21.15.175:3300/0 conn(0x7ff818153a50 0x7ff818159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 -- 172.21.15.44:0/4099768318 >> 172.21.15.44:0/4099768318 conn(0x7ff8180b0fb0 msgr2=0x7ff8180b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.388+0000 7ff81f200640 1 -- 172.21.15.44:0/4099768318 shutdown_connections 2024-07-17T07:15:28.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.389+0000 7ff81f200640 1 -- 172.21.15.44:0/4099768318 wait complete. 2024-07-17T07:15:28.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.390+0000 7ff81f200640 1 Processor -- start 2024-07-17T07:15:28.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.390+0000 7ff81f200640 1 -- start start 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.391+0000 7ff81f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 0x7ff8181f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.391+0000 7ff81f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 0x7ff8181f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.391+0000 7ff81f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff818153fb0 con 0x7ff81815a040 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff818153e30 con 0x7ff8180ac920 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff81806f710 con 0x7ff818153a50 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39320/0 (socket says 172.21.15.44:39320) 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 -- 172.21.15.44:0/2724952593 learned_addr learned my addr 172.21.15.44:0/2724952593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:28.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff817400640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 0x7ff8181f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff817e00640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 0x7ff8181f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 -- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 msgr2=0x7ff8181f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 0x7ff8181f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 -- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 msgr2=0x7ff8181f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 0x7ff8181f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff81d200640 1 -- 172.21.15.44:0/2724952593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8181f5a30 con 0x7ff81815a040 2024-07-17T07:15:28.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff817e00640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 0x7ff8181f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:28.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.392+0000 7ff817400640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 0x7ff8181f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:28.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.393+0000 7ff81d200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff81000cae0 tx=0x7ff810007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.394+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81000ad90 con 0x7ff81815a040 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.394+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff810004b20 con 0x7ff81815a040 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.394+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810005020 con 0x7ff81815a040 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.394+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8181f5d10 con 0x7ff81815a040 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.394+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181f62d0 con 0x7ff81815a040 2024-07-17T07:15:28.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.396+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818159b00 con 0x7ff81815a040 2024-07-17T07:15:28.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.397+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7ff8100051c0 con 0x7ff81815a040 2024-07-17T07:15:28.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.398+0000 7ff80fe00640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 0x7ff7f80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:28.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.398+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7ff810098fe0 con 0x7ff81815a040 2024-07-17T07:15:28.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.399+0000 7ff817e00640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 0x7ff7f80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.400+0000 7ff817e00640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 0x7ff7f80791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff80800baa0 tx=0x7ff808005d20 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.403+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff810014030 con 0x7ff81815a040 2024-07-17T07:15:28.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.657+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff81806ee20 con 0x7ff81815a040 2024-07-17T07:15:28.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.658+0000 7ff80fe00640 1 -- 172.21.15.44:0/2724952593 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7ff810061ed0 con 0x7ff81815a040 2024-07-17T07:15:28.658 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:15:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.660+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 msgr2=0x7ff7f80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.660+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 0x7ff7f80791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff80800baa0 tx=0x7ff808005d20 comp rx=0 tx=0).stop 2024-07-17T07:15:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.660+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 msgr2=0x7ff8181f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.660+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff81000cae0 tx=0x7ff810007f60 comp rx=0 tx=0).stop 2024-07-17T07:15:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 shutdown_connections 2024-07-17T07:15:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ff7f8076d00 0x7ff7f80791c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff81815a040 0x7ff8181f7200 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff818153a50 0x7ff8181f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 --2- 172.21.15.44:0/2724952593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8180ac920 0x7ff8181f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 >> 172.21.15.44:0/2724952593 conn(0x7ff8180b0fb0 msgr2=0x7ff818159050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 shutdown_connections 2024-07-17T07:15:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28.661+0000 7ff81f200640 1 -- 172.21.15.44:0/2724952593 wait complete. 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:15:28.681 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:15:28.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:28.682 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:15:28.733 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:15:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:15:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:15:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41329sec preferred_lft 41329sec 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.735 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:15:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:15:28.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:28.737 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:15:28.822 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:15:28.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:28.827 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:15:28.880 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:15:28.880 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:15:28.881 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:15:28.881 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:15:28.881 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:15:28.882 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:15:28.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:15:29.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:15:29.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:29.108 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:15:29.153 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:15:29.153 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:15:29.153 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:15:29.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:29.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:15:29.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:15:29.283 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:29.338 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:15:29.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:15:29.338 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:29.391 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:15:29.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:29.392 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:15:29.516 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:15:29.516 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:15:29.516 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:15:29.536 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:15:29.536 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:15:29.537 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:15:29.537 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:15:29.537 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:15:29.565 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:15:29.565 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:15:29.565 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:15:29.565 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:15:29.566 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:15:29.566 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:15:29.566 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:15:29.566 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:15:29.568 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:15:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:15:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:15:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:15:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:15:29.573 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:15:29.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:29.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:29.649 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:15:29.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:15:29.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:15:29.781 INFO:teuthology.orchestra.run.smithi044.stdout:client6697 2024-07-17T07:15:29.782 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:29.782 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:29.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.090+0000 7f7c5e400640 1 -- 172.21.15.44:0/2707446455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580b9060 msgr2=0x7f7c5815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.090+0000 7f7c5e400640 1 --2- 172.21.15.44:0/2707446455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580b9060 0x7f7c5815d800 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009ca0 tx=0x7f7c5002aac0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 -- 172.21.15.44:0/2707446455 shutdown_connections 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 --2- 172.21.15.44:0/2707446455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580b9060 0x7f7c5815d800 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 --2- 172.21.15.44:0/2707446455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580ce5c0 0x7f7c5815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 -- 172.21.15.44:0/2707446455 >> 172.21.15.44:0/2707446455 conn(0x7f7c580cfec0 msgr2=0x7f7c580d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 -- 172.21.15.44:0/2707446455 shutdown_connections 2024-07-17T07:15:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.091+0000 7f7c5e400640 1 -- 172.21.15.44:0/2707446455 wait complete. 2024-07-17T07:15:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.093+0000 7f7c5e400640 1 Processor -- start 2024-07-17T07:15:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.093+0000 7f7c5e400640 1 -- start start 2024-07-17T07:15:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580b8c80 0x7f7c5814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c580b9060 0x7f7c581495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5815e0d0 con 0x7f7c580ce5c0 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5815df50 con 0x7f7c580b9060 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5815e250 con 0x7f7c580b8c80 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39824/0 (socket says 172.21.15.44:39824) 2024-07-17T07:15:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.094+0000 7f7c5da00640 1 -- 172.21.15.44:0/1335725589 learned_addr learned my addr 172.21.15.44:0/1335725589 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c57e00640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c580b9060 0x7f7c581495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5d000640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580b8c80 0x7f7c5814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 -- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580b8c80 msgr2=0x7f7c5814af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580b8c80 0x7f7c5814af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 -- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c580b9060 msgr2=0x7f7c581495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c580b9060 0x7f7c581495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 -- 172.21.15.44:0/1335725589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c58149ec0 con 0x7f7c580ce5c0 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5da00640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006010 tx=0x7f7c50003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50046070 con 0x7f7c580ce5c0 2024-07-17T07:15:30.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c50004040 con 0x7f7c580ce5c0 2024-07-17T07:15:30.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.095+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5814a1a0 con 0x7f7c580ce5c0 2024-07-17T07:15:30.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.096+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004430 con 0x7f7c580ce5c0 2024-07-17T07:15:30.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.096+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5814a760 con 0x7f7c580ce5c0 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.097+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5815e700 con 0x7f7c580ce5c0 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.098+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7f7c50049070 con 0x7f7c580ce5c0 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.099+0000 7f7c54c00640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 0x7f7c340791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.099+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f7c500b9420 con 0x7f7c580ce5c0 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.100+0000 7f7c5d000640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 0x7f7c340791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.101+0000 7f7c5d000640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 0x7f7c340791c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c580aa220 tx=0x7f7c48009040 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:30.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.104+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7c50032030 con 0x7f7c580ce5c0 2024-07-17T07:15:30.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.339+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 --> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7c58160110 con 0x7f7c34076d00 2024-07-17T07:15:30.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.343+0000 7f7c54c00640 1 -- 172.21.15.44:0/1335725589 <== mgr.6161 v2:172.21.15.175:6800/2606536318 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7c58160110 con 0x7f7c34076d00 2024-07-17T07:15:30.343 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:15:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:15:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:15:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:15:30.343 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:15:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 msgr2=0x7f7c340791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 0x7f7c340791c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c580aa220 tx=0x7f7c48009040 comp rx=0 tx=0).stop 2024-07-17T07:15:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 msgr2=0x7f7c581f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006010 tx=0x7f7c50003ce0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 shutdown_connections 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7c34076d00 0x7f7c340791c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c580ce5c0 0x7f7c581f3a50 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c580b9060 0x7f7c581495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 --2- 172.21.15.44:0/1335725589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c580b8c80 0x7f7c5814af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 >> 172.21.15.44:0/1335725589 conn(0x7f7c580cfec0 msgr2=0x7f7c58160330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 shutdown_connections 2024-07-17T07:15:30.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.346+0000 7f7c5e400640 1 -- 172.21.15.44:0/1335725589 wait complete. 2024-07-17T07:15:30.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:15:30.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 -- 172.21.15.44:0/307694435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00b3f20 msgr2=0x7ffbf006e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 --2- 172.21.15.44:0/307694435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00b3f20 0x7ffbf006e8e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ffbf00d0330 tx=0x7ffbd802a5f0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 -- 172.21.15.44:0/307694435 shutdown_connections 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 --2- 172.21.15.44:0/307694435 >> v2:172.21.15.175:3300/0 conn(0x7ffbf00b4360 0x7ffbf006ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 --2- 172.21.15.44:0/307694435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00b3f20 0x7ffbf006e8e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 -- 172.21.15.44:0/307694435 >> 172.21.15.44:0/307694435 conn(0x7ffbf006d050 msgr2=0x7ffbf006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.572+0000 7ffbf6000640 1 -- 172.21.15.44:0/307694435 shutdown_connections 2024-07-17T07:15:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.573+0000 7ffbf6000640 1 -- 172.21.15.44:0/307694435 wait complete. 2024-07-17T07:15:30.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.574+0000 7ffbf6000640 1 Processor -- start 2024-07-17T07:15:30.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.574+0000 7ffbf6000640 1 -- start start 2024-07-17T07:15:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbf00b3f20 0x7ffbf01389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf00b4360 0x7ffbf015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf006f360 con 0x7ffbf006bf40 2024-07-17T07:15:30.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf006f4e0 con 0x7ffbf00b4360 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.575+0000 7ffbf6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf006f660 con 0x7ffbf00b3f20 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39838/0 (socket says 172.21.15.44:39838) 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 -- 172.21.15.44:0/2399424721 learned_addr learned my addr 172.21.15.44:0/2399424721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbeea00640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbf00b3f20 0x7ffbf01389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 -- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbf00b3f20 msgr2=0x7ffbf01389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbf00b3f20 0x7ffbf01389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 -- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf00b4360 msgr2=0x7ffbf015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf00b4360 0x7ffbf015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.576+0000 7ffbef400640 1 -- 172.21.15.44:0/2399424721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf0146840 con 0x7ffbf006bf40 2024-07-17T07:15:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbef400640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000c8a0 tx=0x7ffbe000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0003b70 con 0x7ffbf006bf40 2024-07-17T07:15:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbe0003d30 con 0x7ffbf006bf40 2024-07-17T07:15:30.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf015cbe0 con 0x7ffbf006bf40 2024-07-17T07:15:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000aab0 con 0x7ffbf006bf40 2024-07-17T07:15:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.577+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf015c900 con 0x7ffbf006bf40 2024-07-17T07:15:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.578+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf006fac0 con 0x7ffbf006bf40 2024-07-17T07:15:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.579+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 69) ==== 98931+0+0 (secure 0 0 0) 0x7ffbe000ac50 con 0x7ffbf006bf40 2024-07-17T07:15:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.580+0000 7ffbe7400640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 0x7ffbcc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.581+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7ffbe00987f0 con 0x7ffbf006bf40 2024-07-17T07:15:30.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.581+0000 7ffbeea00640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 0x7ffbcc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.582+0000 7ffbeea00640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 0x7ffbcc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8033000 tx=0x7ffbd8004730 comp rx=0 tx=0).ready entity=mgr.6161 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:30.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.585+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffbe00617f0 con 0x7ffbf006bf40 2024-07-17T07:15:30.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:30.848+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ffbf006ce40 con 0x7ffbf006bf40 2024-07-17T07:15:31.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.426+0000 7ffbe7400640 1 -- 172.21.15.44:0/2399424721 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v70) ==== 92+0+0 (secure 0 0 0) 0x7ffbe00610f0 con 0x7ffbf006bf40 2024-07-17T07:15:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 msgr2=0x7ffbcc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 0x7ffbcc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8033000 tx=0x7ffbd8004730 comp rx=0 tx=0).stop 2024-07-17T07:15:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 msgr2=0x7ffbf0146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000c8a0 tx=0x7ffbe000cd60 comp rx=0 tx=0).stop 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 shutdown_connections 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7ffbcc076cd0 0x7ffbcc079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbf00b4360 0x7ffbf015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbf00b3f20 0x7ffbf01389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 --2- 172.21.15.44:0/2399424721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf006bf40 0x7ffbf0146300 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.430+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 >> 172.21.15.44:0/2399424721 conn(0x7ffbf006d050 msgr2=0x7ffbf00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.431+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 shutdown_connections 2024-07-17T07:15:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.431+0000 7ffbf6000640 1 -- 172.21.15.44:0/2399424721 wait complete. 2024-07-17T07:15:31.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:15:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.678+0000 7f754f400640 1 -- 172.21.15.44:0/2427820746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480b6790 msgr2=0x7f7548161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 --2- 172.21.15.44:0/2427820746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480b6790 0x7f7548161650 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7548075790 tx=0x7f753402a680 comp rx=0 tx=0).stop 2024-07-17T07:15:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 -- 172.21.15.44:0/2427820746 shutdown_connections 2024-07-17T07:15:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 --2- 172.21.15.44:0/2427820746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480b6790 0x7f7548161650 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 --2- 172.21.15.44:0/2427820746 >> v2:172.21.15.175:3300/0 conn(0x7f75480ce1d0 0x7f75480ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 -- 172.21.15.44:0/2427820746 >> 172.21.15.44:0/2427820746 conn(0x7f75480d3700 msgr2=0x7f75480d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 -- 172.21.15.44:0/2427820746 shutdown_connections 2024-07-17T07:15:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.679+0000 7f754f400640 1 -- 172.21.15.44:0/2427820746 wait complete. 2024-07-17T07:15:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.680+0000 7f754f400640 1 Processor -- start 2024-07-17T07:15:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.681+0000 7f754f400640 1 -- start start 2024-07-17T07:15:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.681+0000 7f754f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75480b29e0 0x7f754814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.681+0000 7f754f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75480b6790 0x7f75480c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:31.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7548162420 con 0x7f75480ce1d0 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75481622a0 con 0x7f75480b29e0 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7548167910 con 0x7f75480b6790 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39866/0 (socket says 172.21.15.44:39866) 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754d400640 1 -- 172.21.15.44:0/1689775914 learned_addr learned my addr 172.21.15.44:0/1689775914 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f754ca00640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75480b29e0 0x7f754814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.682+0000 7f7547e00640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75480b6790 0x7f75480c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75480b6790 msgr2=0x7f75480c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75480b6790 0x7f75480c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75480b29e0 msgr2=0x7f754814d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75480b29e0 0x7f754814d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 -- 172.21.15.44:0/1689775914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75480c8780 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754d400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f754000c7a0 tx=0x7f754000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540003b70 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7540003d10 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000aab0 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75480c8a20 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.683+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754814dc30 con 0x7f75480ce1d0 2024-07-17T07:15:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.685+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75480ce5b0 con 0x7f75480ce1d0 2024-07-17T07:15:31.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.686+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 70) ==== 98948+0+0 (secure 0 0 0) 0x7f754000ac50 con 0x7f75480ce1d0 2024-07-17T07:15:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.688+0000 7f7544c00640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 0x7f75280791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.688+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7f7540098660 con 0x7f75480ce1d0 2024-07-17T07:15:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.688+0000 7f754ca00640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 msgr2=0x7f75280791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.688+0000 7f754ca00640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 0x7f75280791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.692+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7540061010 con 0x7f75480ce1d0 2024-07-17T07:15:31.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.890+0000 7f754ca00640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 msgr2=0x7f75280791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:31.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.890+0000 7f754ca00640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 0x7f75280791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.905+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f75480b6b70 con 0x7f75480ce1d0 2024-07-17T07:15:31.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.911+0000 7f7544c00640 1 -- 172.21.15.44:0/1689775914 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) ==== 138+0+0 (secure 0 0 0) 0x7f7540065560 con 0x7f75480ce1d0 2024-07-17T07:15:31.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 msgr2=0x7f75280791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 0x7f75280791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 msgr2=0x7f754815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f754000c7a0 tx=0x7f754000cc60 comp rx=0 tx=0).stop 2024-07-17T07:15:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 shutdown_connections 2024-07-17T07:15:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7f7528076d20 0x7f75280791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75480ce1d0 0x7f754815d400 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75480b6790 0x7f75480c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 --2- 172.21.15.44:0/1689775914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75480b29e0 0x7f754814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 >> 172.21.15.44:0/1689775914 conn(0x7f75480d3700 msgr2=0x7f75480d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 shutdown_connections 2024-07-17T07:15:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:31.915+0000 7f754f400640 1 -- 172.21.15.44:0/1689775914 wait complete. 2024-07-17T07:15:31.937 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.130+0000 7fdb3a800640 1 -- 172.21.15.44:0/1293075395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb340b5a70 msgr2=0x7fdb3415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.130+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1293075395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb340b5a70 0x7fdb3415f270 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005fd0 tx=0x7fdb2c02ab20 comp rx=0 tx=0).stop 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 -- 172.21.15.44:0/1293075395 shutdown_connections 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1293075395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb340b5a70 0x7fdb3415f270 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1293075395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb3415b030 0x7fdb340b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 -- 172.21.15.44:0/1293075395 >> 172.21.15.44:0/1293075395 conn(0x7fdb340b3090 msgr2=0x7fdb340b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 -- 172.21.15.44:0/1293075395 shutdown_connections 2024-07-17T07:15:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.131+0000 7fdb3a800640 1 -- 172.21.15.44:0/1293075395 wait complete. 2024-07-17T07:15:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.133+0000 7fdb3a800640 1 Processor -- start 2024-07-17T07:15:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.133+0000 7fdb3a800640 1 -- start start 2024-07-17T07:15:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb340b09a0 0x7fdb3414fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb340b5a70 0x7fdb340d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34070ef0 con 0x7fdb3415b030 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34070d70 con 0x7fdb340b09a0 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb3a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb340b3d90 con 0x7fdb340b5a70 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39886/0 (socket says 172.21.15.44:39886) 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 -- 172.21.15.44:0/1371330816 learned_addr learned my addr 172.21.15.44:0/1371330816 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb38a00640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb340b5a70 0x7fdb340d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39400640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb340b09a0 0x7fdb3414fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb340b5a70 msgr2=0x7fdb340d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb340b5a70 0x7fdb340d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb340b09a0 msgr2=0x7fdb3414fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb340b09a0 0x7fdb3414fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.134+0000 7fdb39e00640 1 -- 172.21.15.44:0/1371330816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3414ee20 con 0x7fdb3415b030 2024-07-17T07:15:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb39e00640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009bf0 tx=0x7fdb2c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:32.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c044070 con 0x7fdb3415b030 2024-07-17T07:15:32.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb2c004250 con 0x7fdb3415b030 2024-07-17T07:15:32.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c004660 con 0x7fdb3415b030 2024-07-17T07:15:32.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3414f0c0 con 0x7fdb3415b030 2024-07-17T07:15:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.135+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3414e9c0 con 0x7fdb3415b030 2024-07-17T07:15:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.136+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3414f6d0 con 0x7fdb3415b030 2024-07-17T07:15:32.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.138+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 70) ==== 98948+0+0 (secure 0 0 0) 0x7fdb2c02c980 con 0x7fdb3415b030 2024-07-17T07:15:32.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.139+0000 7fdb31600640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 0x7fdb14079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.139+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fdb2c0b8fe0 con 0x7fdb3415b030 2024-07-17T07:15:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.139+0000 7fdb39400640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 msgr2=0x7fdb14079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.139+0000 7fdb39400640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 0x7fdb14079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:32.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.147+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdb2c032030 con 0x7fdb3415b030 2024-07-17T07:15:32.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.341+0000 7fdb39400640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 msgr2=0x7fdb14079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:32.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.341+0000 7fdb39400640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 0x7fdb14079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:32.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.359+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fdb340694a0 con 0x7fdb3415b030 2024-07-17T07:15:32.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.365+0000 7fdb31600640 1 -- 172.21.15.44:0/1371330816 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) ==== 133+0+0 (secure 0 0 0) 0x7fdb340694a0 con 0x7fdb3415b030 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.367+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 msgr2=0x7fdb14079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 0x7fdb14079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 msgr2=0x7fdb340cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009bf0 tx=0x7fdb2c003ce0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 shutdown_connections 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fdb14076d50 0x7fdb14079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb3415b030 0x7fdb340cee80 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb340b5a70 0x7fdb340d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 --2- 172.21.15.44:0/1371330816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb340b09a0 0x7fdb3414fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 >> 172.21.15.44:0/1371330816 conn(0x7fdb340b3090 msgr2=0x7fdb340b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 shutdown_connections 2024-07-17T07:15:32.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.368+0000 7fdb3a800640 1 -- 172.21.15.44:0/1371330816 wait complete. 2024-07-17T07:15:32.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:32.386 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2024-07-17T07:15:32.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2024-07-17T07:15:32.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 -- 172.21.15.44:0/1646971852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 msgr2=0x7fae8c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 --2- 172.21.15.44:0/1646971852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c0b2200 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0d0de0 tx=0x7fae7802a4d0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 -- 172.21.15.44:0/1646971852 shutdown_connections 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 --2- 172.21.15.44:0/1646971852 >> v2:172.21.15.175:3300/0 conn(0x7fae8c0ce690 0x7fae8c0cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 --2- 172.21.15.44:0/1646971852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c0b2200 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 --2- 172.21.15.44:0/1646971852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae8c15d130 0x7fae8c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 -- 172.21.15.44:0/1646971852 >> 172.21.15.44:0/1646971852 conn(0x7fae8c0b4ff0 msgr2=0x7fae8c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 -- 172.21.15.44:0/1646971852 shutdown_connections 2024-07-17T07:15:32.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.618+0000 7fae8b200640 1 -- 172.21.15.44:0/1646971852 wait complete. 2024-07-17T07:15:32.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.620+0000 7fae8b200640 1 Processor -- start 2024-07-17T07:15:32.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.620+0000 7fae8b200640 1 -- start start 2024-07-17T07:15:32.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c0ce690 0x7fae8c15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae8c15d130 0x7fae8c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c06c7a0 con 0x7fae8c161be0 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c06c620 con 0x7fae8c0ce690 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.621+0000 7fae8b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae8c06c920 con 0x7fae8c15d130 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c15fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39906/0 (socket says 172.21.15.44:39906) 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 -- 172.21.15.44:0/3360708919 learned_addr learned my addr 172.21.15.44:0/3360708919 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae89400640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae8c15d130 0x7fae8c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c0ce690 0x7fae8c15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:32.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae8c15d130 msgr2=0x7fae8c15f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae8c15d130 0x7fae8c15f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c0ce690 msgr2=0x7fae8c15ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:32.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c0ce690 0x7fae8c15ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.622+0000 7fae8a800640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c15d720 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae8a800640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c15fa30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ca80 tx=0x7fae8400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84007df0 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae84003aa0 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c15da00 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84004030 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.623+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c15dfc0 con 0x7fae8c161be0 2024-07-17T07:15:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.624+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c0cfdf0 con 0x7fae8c161be0 2024-07-17T07:15:32.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.625+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 70) ==== 98948+0+0 (secure 0 0 0) 0x7fae840041d0 con 0x7fae8c161be0 2024-07-17T07:15:32.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.627+0000 7fae82000640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:32.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.627+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6649+0+0 (secure 0 0 0) 0x7fae84099400 con 0x7fae8c161be0 2024-07-17T07:15:32.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.627+0000 7fae89e00640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:32.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.627+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:32.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.631+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fae84014070 con 0x7fae8c161be0 2024-07-17T07:15:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.829+0000 7fae89e00640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.829+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:32.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:32.861+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fae8c0d0e10 con 0x7fae6c076d20 2024-07-17T07:15:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:33.231+0000 7fae89e00640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:33.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:33.231+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:15:34.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:34.032+0000 7fae89e00640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:34.032+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:15:35.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:35.636+0000 7fae89e00640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2606536318 2024-07-17T07:15:35.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:35.636+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:15:35.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:35.653+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 71) ==== 98557+0+0 (secure 0 0 0) 0x7fae8405dea0 con 0x7fae8c161be0 2024-07-17T07:15:35.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:35.654+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 msgr2=0x7fae6c0791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:35.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:35.654+0000 7fae82000640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/2606536318,v1:172.21.15.175:6801/2606536318] conn(0x7fae6c076d20 0x7fae6c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.919+0000 7f365d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.919+0000 7f365d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.919+0000 7f365d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.919+0000 7f365d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.919+0000 7f365d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.919 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.920+0000 7f3659600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.920 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.920+0000 7f3659600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.920 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.920+0000 7f3659600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.920 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.920+0000 7f3659600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:35.920 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:35.920+0000 7f3659600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.663+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 72) ==== 98684+0+0 (secure 0 0 0) 0x7fae84039570 con 0x7fae8c161be0 2024-07-17T07:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.664+0000 7fae82000640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fae6c07f880 0x7fae6c081c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.664+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fae6c00c880 con 0x7fae6c07f880 2024-07-17T07:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.665+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fae6c07f880 0x7fae6c081c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:36.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.674+0000 7fae89e00640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fae6c07f880 0x7fae6c081c70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae7c006010 tx=0x7fae7c007360 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.847+0000 7fae82000640 1 -- 172.21.15.44:0/3360708919 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+45 (secure 0 0 0) 0x7fae6c00c880 con 0x7fae6c07f880 2024-07-17T07:15:36.846 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/minutes 2024-07-17T07:15:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.850+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fae6c07f880 msgr2=0x7fae6c081c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:36.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.850+0000 7fae8b200640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fae6c07f880 0x7fae6c081c70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fae7c006010 tx=0x7fae7c007360 comp rx=0 tx=0).stop 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.850+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 msgr2=0x7fae8c15fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.850+0000 7fae8b200640 1 --2- 172.21.15.44:0/3360708919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae8c161be0 0x7fae8c15fa30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ca80 tx=0x7fae8400cf40 comp rx=0 tx=0).stop 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.850+0000 7fae8a800640 1 -- 172.21.15.44:0/3360708919 reap_dead start 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.851+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 shutdown_connections 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.851+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 >> 172.21.15.44:0/3360708919 conn(0x7fae8c0b4ff0 msgr2=0x7fae8c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.851+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 shutdown_connections 2024-07-17T07:15:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:36.851+0000 7fae8b200640 1 -- 172.21.15.44:0/3360708919 wait complete. 2024-07-17T07:15:36.872 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2024-07-17T07:15:36.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:36.873 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2024-07-17T07:15:36.928 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 -- 172.21.15.44:0/111229941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 msgr2=0x7f4ce006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 --2- 172.21.15.44:0/111229941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce006d690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8005fd0 tx=0x7f4cc802a4d0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 -- 172.21.15.44:0/111229941 shutdown_connections 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 --2- 172.21.15.44:0/111229941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce006d690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 --2- 172.21.15.44:0/111229941 >> v2:172.21.15.175:3300/0 conn(0x7f4ce015d810 0x7f4ce00b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 -- 172.21.15.44:0/111229941 >> 172.21.15.44:0/111229941 conn(0x7f4ce0073d50 msgr2=0x7f4ce009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.137+0000 7f4ce5200640 1 -- 172.21.15.44:0/111229941 shutdown_connections 2024-07-17T07:15:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.138+0000 7f4ce5200640 1 -- 172.21.15.44:0/111229941 wait complete. 2024-07-17T07:15:37.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.139+0000 7f4ce5200640 1 Processor -- start 2024-07-17T07:15:37.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.139+0000 7f4ce5200640 1 -- start start 2024-07-17T07:15:37.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce0160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 0x7f4ce009d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00b56c0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00b5540 con 0x7f4ce015d810 2024-07-17T07:15:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4ce5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce00b5840 con 0x7f4ce00b1a10 2024-07-17T07:15:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.140+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39912/0 (socket says 172.21.15.44:39912) 2024-07-17T07:15:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 -- 172.21.15.44:0/3401979927 learned_addr learned my addr 172.21.15.44:0/3401979927 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cde800640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 0x7f4ce009d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdd400640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce0160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 -- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 msgr2=0x7f4ce0160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce0160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 -- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 msgr2=0x7f4ce009d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 0x7f4ce009d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 -- 172.21.15.44:0/3401979927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce0160ba0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cde800640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 0x7f4ce009d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.141+0000 7f4cdde00640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e790 tx=0x7f4cd000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.142+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000eea0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.142+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd000cdb0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.142+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce015e330 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.142+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0010690 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.142+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce015ebd0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.143+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce009f7d0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.144+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 72) ==== 98684+0+0 (secure 0 0 0) 0x7f4cd0010830 con 0x7f4ce006dbd0 2024-07-17T07:15:37.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.145+0000 7f4cd6000640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 0x7f4cbc078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.145+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f4cd0098ec0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.146+0000 7f4cdd400640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 0x7f4cbc078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.147+0000 7f4cdd400640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 0x7f4cbc078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8033000 tx=0x7f4cc8004730 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.149+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4ce009f7d0 con 0x7f4ce006dbd0 2024-07-17T07:15:37.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.383+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) -- 0x7f4ce0070b30 con 0x7f4cbc076a50 2024-07-17T07:15:37.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.563+0000 7f4cd6000640 1 -- 172.21.15.44:0/3401979927 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f4ce0070b30 con 0x7f4cbc076a50 2024-07-17T07:15:37.562 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/hourly 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 msgr2=0x7f4cbc078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 0x7f4cbc078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8033000 tx=0x7f4cc8004730 comp rx=0 tx=0).stop 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 msgr2=0x7f4ce015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e790 tx=0x7f4cd000ec50 comp rx=0 tx=0).stop 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 shutdown_connections 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f4cbc076a50 0x7f4cbc078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ce015d810 0x7f4ce009d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ce00b1a10 0x7f4ce0160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 --2- 172.21.15.44:0/3401979927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ce006dbd0 0x7f4ce015f3d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 >> 172.21.15.44:0/3401979927 conn(0x7f4ce0073d50 msgr2=0x7f4ce00b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:37.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 shutdown_connections 2024-07-17T07:15:37.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.567+0000 7f4ce5200640 1 -- 172.21.15.44:0/3401979927 wait complete. 2024-07-17T07:15:37.588 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2024-07-17T07:15:37.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:37.588 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2024-07-17T07:15:37.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2024-07-17T07:15:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 -- 172.21.15.44:0/4030885347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 msgr2=0x7fad0c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 --2- 172.21.15.44:0/4030885347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 0x7fad0c06d620 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fad04009be0 tx=0x7fad0402a890 comp rx=0 tx=0).stop 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 -- 172.21.15.44:0/4030885347 shutdown_connections 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 --2- 172.21.15.44:0/4030885347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 0x7fad0c06d620 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 -- 172.21.15.44:0/4030885347 >> 172.21.15.44:0/4030885347 conn(0x7fad0c0b0eb0 msgr2=0x7fad0c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 -- 172.21.15.44:0/4030885347 shutdown_connections 2024-07-17T07:15:37.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.869+0000 7fad13a00640 1 -- 172.21.15.44:0/4030885347 wait complete. 2024-07-17T07:15:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.871+0000 7fad13a00640 1 Processor -- start 2024-07-17T07:15:37.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.871+0000 7fad13a00640 1 -- start start 2024-07-17T07:15:37.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 0x7fad0c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad0c0cbab0 0x7fad0c147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c0cd9f0 con 0x7fad0c152220 2024-07-17T07:15:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c15c7d0 con 0x7fad0c0cbab0 2024-07-17T07:15:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad13a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c0cdb70 con 0x7fad0c06ed30 2024-07-17T07:15:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad11a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad11a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39920/0 (socket says 172.21.15.44:39920) 2024-07-17T07:15:37.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.872+0000 7fad11a00640 1 -- 172.21.15.44:0/1106628749 learned_addr learned my addr 172.21.15.44:0/1106628749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 -- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 msgr2=0x7fad0c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 0x7fad0c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 -- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad0c0cbab0 msgr2=0x7fad0c147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad0c0cbab0 0x7fad0c147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 -- 172.21.15.44:0/1106628749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c14a400 con 0x7fad0c152220 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.873+0000 7fad11a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fad04009be0 tx=0x7fad04003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.874+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04031d80 con 0x7fad0c152220 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.874+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad04004040 con 0x7fad0c152220 2024-07-17T07:15:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.874+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004450 con 0x7fad0c152220 2024-07-17T07:15:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.874+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c14a6e0 con 0x7fad0c152220 2024-07-17T07:15:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.874+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c15dee0 con 0x7fad0c152220 2024-07-17T07:15:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.875+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c15d9f0 con 0x7fad0c152220 2024-07-17T07:15:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.876+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 73) ==== 98949+0+0 (secure 0 0 0) 0x7fad04039040 con 0x7fad0c152220 2024-07-17T07:15:37.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.877+0000 7fad08c00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 0x7facec079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:37.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.877+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7fad040b9ab0 con 0x7fad0c152220 2024-07-17T07:15:37.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.878+0000 7fad11000640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 0x7facec079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:37.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.879+0000 7fad11000640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 0x7facec079210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad00007fd0 tx=0x7fad00007900 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:37.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:37.882+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fad04040030 con 0x7fad0c152220 2024-07-17T07:15:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.117+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) -- 0x7fad0c147b70 con 0x7facec076d50 2024-07-17T07:15:38.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.304+0000 7fad08c00640 1 -- 172.21.15.44:0/1106628749 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fad00014400 con 0x7facec076d50 2024-07-17T07:15:38.303 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/daily 2024-07-17T07:15:38.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.307+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 msgr2=0x7facec079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.307+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 0x7facec079210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad00007fd0 tx=0x7fad00007900 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.307+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 msgr2=0x7fad0c1f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.307+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fad04009be0 tx=0x7fad04003ce0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 shutdown_connections 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7facec076d50 0x7facec079210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad0c152220 0x7fad0c1f5060 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad0c0cbab0 0x7fad0c147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 --2- 172.21.15.44:0/1106628749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad0c06ed30 0x7fad0c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 >> 172.21.15.44:0/1106628749 conn(0x7fad0c0b0eb0 msgr2=0x7fad0c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 shutdown_connections 2024-07-17T07:15:38.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.308+0000 7fad13a00640 1 -- 172.21.15.44:0/1106628749 wait complete. 2024-07-17T07:15:38.328 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2024-07-17T07:15:38.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:38.329 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2024-07-17T07:15:38.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.605+0000 7f86b2000640 1 -- 172.21.15.44:0/1087194114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 msgr2=0x7f8690004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.605+0000 7f86b2000640 1 --2- 172.21.15.44:0/1087194114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f8690004c80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8694009b50 tx=0x7f8690005830 comp rx=0 tx=0).stop 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.606+0000 7f86abe00640 1 -- 172.21.15.44:0/1087194114 reap_dead start 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.606+0000 7f86b2000640 1 -- 172.21.15.44:0/1087194114 shutdown_connections 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.606+0000 7f86b2000640 1 -- 172.21.15.44:0/1087194114 >> 172.21.15.44:0/1087194114 conn(0x7f86ac0b0eb0 msgr2=0x7f86ac0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.606+0000 7f86b2000640 1 -- 172.21.15.44:0/1087194114 shutdown_connections 2024-07-17T07:15:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.606+0000 7f86b2000640 1 -- 172.21.15.44:0/1087194114 wait complete. 2024-07-17T07:15:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.608+0000 7f86b2000640 1 Processor -- start 2024-07-17T07:15:38.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.608+0000 7f86b2000640 1 -- start start 2024-07-17T07:15:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86ac14c840 0x7f86ac1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 0x7f86ac1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:38.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac15c470 con 0x7f8690004800 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac15c2f0 con 0x7f86ac14c840 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86b2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f86ac15c5f0 con 0x7f86ac1f5da0 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86ab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86ab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39938/0 (socket says 172.21.15.44:39938) 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86ab400640 1 -- 172.21.15.44:0/2237481176 learned_addr learned my addr 172.21.15.44:0/2237481176 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86aaa00640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86ac14c840 0x7f86ac1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.609+0000 7f86abe00640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 0x7f86ac1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 -- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 msgr2=0x7f86ac1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 0x7f86ac1f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 -- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86ac14c840 msgr2=0x7f86ac1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86ac14c840 0x7f86ac1f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 -- 172.21.15.44:0/2237481176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac144640 con 0x7f8690004800 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86abe00640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 0x7f86ac1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86ab400640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f869c00db50 tx=0x7f869c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c016020 con 0x7f8690004800 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f869c00de50 con 0x7f8690004800 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00be00 con 0x7f8690004800 2024-07-17T07:15:38.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac074050 con 0x7f8690004800 2024-07-17T07:15:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.610+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac073d50 con 0x7f8690004800 2024-07-17T07:15:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.612+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ac0cd2e0 con 0x7f8690004800 2024-07-17T07:15:38.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.613+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 73) ==== 98949+0+0 (secure 0 0 0) 0x7f869c019020 con 0x7f8690004800 2024-07-17T07:15:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.614+0000 7f86a3400640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 0x7f86880791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:38.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.615+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f869c098c40 con 0x7f8690004800 2024-07-17T07:15:38.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.615+0000 7f86aaa00640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 0x7f86880791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:38.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.616+0000 7f86aaa00640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 0x7f86880791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8694035000 tx=0x7f869402d040 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:38.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.619+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f869c020020 con 0x7f8690004800 2024-07-17T07:15:38.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:38.855+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) -- 0x7f86ac0b0440 con 0x7f8688076d20 2024-07-17T07:15:39.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.046+0000 7f86a3400640 1 -- 172.21.15.44:0/2237481176 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f86ac0b0440 con 0x7f8688076d20 2024-07-17T07:15:39.045 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/weekly 2024-07-17T07:15:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 msgr2=0x7f86880791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 0x7f86880791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8694035000 tx=0x7f869402d040 comp rx=0 tx=0).stop 2024-07-17T07:15:39.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 msgr2=0x7f86ac1f1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f869c00db50 tx=0x7f869c00b4b0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 shutdown_connections 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8688076d20 0x7f86880791e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86ac1f5da0 0x7f86ac1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86ac14c840 0x7f86ac1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 --2- 172.21.15.44:0/2237481176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8690004800 0x7f86ac1f1c30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 >> 172.21.15.44:0/2237481176 conn(0x7f86ac0b0eb0 msgr2=0x7f86ac0b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 shutdown_connections 2024-07-17T07:15:39.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.050+0000 7f86b2000640 1 -- 172.21.15.44:0/2237481176 wait complete. 2024-07-17T07:15:39.072 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2024-07-17T07:15:39.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:39.072 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2024-07-17T07:15:39.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2024-07-17T07:15:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 -- 172.21.15.44:0/2313011684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc15f9f0 msgr2=0x7faffc06bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 --2- 172.21.15.44:0/2313011684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc15f9f0 0x7faffc06bf40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7faffc0d0140 tx=0x7fafec02a540 comp rx=0 tx=0).stop 2024-07-17T07:15:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 -- 172.21.15.44:0/2313011684 shutdown_connections 2024-07-17T07:15:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 --2- 172.21.15.44:0/2313011684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc15f9f0 0x7faffc06bf40 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 --2- 172.21.15.44:0/2313011684 >> v2:172.21.15.175:3300/0 conn(0x7faffc0afa90 0x7faffc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 -- 172.21.15.44:0/2313011684 >> 172.21.15.44:0/2313011684 conn(0x7faffc06d050 msgr2=0x7faffc06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 -- 172.21.15.44:0/2313011684 shutdown_connections 2024-07-17T07:15:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.350+0000 7fb004800640 1 -- 172.21.15.44:0/2313011684 wait complete. 2024-07-17T07:15:39.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.352+0000 7fb004800640 1 Processor -- start 2024-07-17T07:15:39.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.352+0000 7fb004800640 1 -- start start 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faffc13e860 0x7faffc1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faffc15f9f0 0x7faffc1f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06c810 con 0x7faffc0afa90 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faffc06c690 con 0x7faffc13e860 2024-07-17T07:15:39.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb004800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faffc15f0b0 con 0x7faffc15f9f0 2024-07-17T07:15:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb001e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb001e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45532/0 (socket says 172.21.15.44:45532) 2024-07-17T07:15:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.353+0000 7fb001e00640 1 -- 172.21.15.44:0/469194552 learned_addr learned my addr 172.21.15.44:0/469194552 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001400640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faffc13e860 0x7faffc1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 -- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faffc15f9f0 msgr2=0x7faffc1f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faffc15f9f0 0x7faffc1f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 -- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faffc13e860 msgr2=0x7faffc1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faffc13e860 0x7faffc1f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 -- 172.21.15.44:0/469194552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc1315b0 con 0x7faffc0afa90 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb001e00640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faff400ea80 tx=0x7faff400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:39.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff400ce40 con 0x7faffc0afa90 2024-07-17T07:15:39.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faff4004c10 con 0x7faffc0afa90 2024-07-17T07:15:39.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4005020 con 0x7faffc0afa90 2024-07-17T07:15:39.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.354+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc1424b0 con 0x7faffc0afa90 2024-07-17T07:15:39.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.355+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc1312d0 con 0x7faffc0afa90 2024-07-17T07:15:39.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.356+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc15f4b0 con 0x7faffc0afa90 2024-07-17T07:15:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.357+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 73) ==== 98949+0+0 (secure 0 0 0) 0x7faff40051e0 con 0x7faffc0afa90 2024-07-17T07:15:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.358+0000 7faff2000640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 0x7fafe00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.358+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7faff4099530 con 0x7faffc0afa90 2024-07-17T07:15:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.359+0000 7fb001400640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 0x7fafe00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:39.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.360+0000 7fb001400640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 0x7fafe00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafec033000 tx=0x7fafec004730 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:39.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.363+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faff4016030 con 0x7faffc0afa90 2024-07-17T07:15:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.597+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) -- 0x7faffc06ca70 con 0x7fafe0076d20 2024-07-17T07:15:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.732+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7faff405de50 con 0x7faffc0afa90 2024-07-17T07:15:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.792+0000 7faff2000640 1 -- 172.21.15.44:0/469194552 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+45 (secure 0 0 0) 0x7faffc06ca70 con 0x7fafe0076d20 2024-07-17T07:15:39.792 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/monthly 2024-07-17T07:15:39.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 msgr2=0x7fafe00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 0x7fafe00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafec033000 tx=0x7fafec004730 comp rx=0 tx=0).stop 2024-07-17T07:15:39.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 msgr2=0x7faffc158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:39.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faff400ea80 tx=0x7faff400ef40 comp rx=0 tx=0).stop 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 shutdown_connections 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fafe0076d20 0x7fafe00791e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faffc15f9f0 0x7faffc1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faffc13e860 0x7faffc1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 --2- 172.21.15.44:0/469194552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faffc0afa90 0x7faffc158dd0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 >> 172.21.15.44:0/469194552 conn(0x7faffc06d050 msgr2=0x7faffc15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 shutdown_connections 2024-07-17T07:15:39.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:39.796+0000 7fb004800640 1 -- 172.21.15.44:0/469194552 wait complete. 2024-07-17T07:15:39.816 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2024-07-17T07:15:39.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:39.817 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2024-07-17T07:15:39.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1y 2024-07-17T07:15:40.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 -- 172.21.15.44:0/4179690925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8161be0 msgr2=0x7f8fd815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 --2- 172.21.15.44:0/4179690925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8161be0 0x7f8fd815d510 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009830 tx=0x7f8fc002a480 comp rx=0 tx=0).stop 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 -- 172.21.15.44:0/4179690925 shutdown_connections 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 --2- 172.21.15.44:0/4179690925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd80ce690 0x7f8fd80cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 --2- 172.21.15.44:0/4179690925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8161be0 0x7f8fd815d510 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.097+0000 7f8fdd400640 1 -- 172.21.15.44:0/4179690925 >> 172.21.15.44:0/4179690925 conn(0x7f8fd80b4ff0 msgr2=0x7f8fd80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.098+0000 7f8fdd400640 1 -- 172.21.15.44:0/4179690925 shutdown_connections 2024-07-17T07:15:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.098+0000 7f8fdd400640 1 -- 172.21.15.44:0/4179690925 wait complete. 2024-07-17T07:15:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.100+0000 7f8fdd400640 1 Processor -- start 2024-07-17T07:15:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.100+0000 7f8fdd400640 1 -- start start 2024-07-17T07:15:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.101+0000 7f8fdd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd80ce690 0x7f8fd8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.101+0000 7f8fdd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd815d130 0x7f8fd8157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdd400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80d1a60 con 0x7f8fd8161be0 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdd400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80d18e0 con 0x7f8fd80ce690 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdd400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd80d1be0 con 0x7f8fd815d130 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45534/0 (socket says 172.21.15.44:45534) 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdca00640 1 -- 172.21.15.44:0/1655804825 learned_addr learned my addr 172.21.15.44:0/1655804825 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fd7e00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd80ce690 0x7f8fd8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fcfe00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd815d130 0x7f8fd8157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdca00640 1 -- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd815d130 msgr2=0x7f8fd8157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.102+0000 7f8fdca00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd815d130 0x7f8fd8157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdca00640 1 -- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd80ce690 msgr2=0x7f8fd8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdca00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd80ce690 0x7f8fd8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdca00640 1 -- 172.21.15.44:0/1655804825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81f3dd0 con 0x7f8fd8161be0 2024-07-17T07:15:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdca00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000ec20 tx=0x7f8fd000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0003b70 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fd0003e00 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004280 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81f40b0 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.103+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81f4670 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.105+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd80ca670 con 0x7f8fd8161be0 2024-07-17T07:15:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.106+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f8fd0004420 con 0x7f8fd8161be0 2024-07-17T07:15:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.107+0000 7f8fd4c00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 0x7f8fb4079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.107+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f8fd0099c00 con 0x7f8fd8161be0 2024-07-17T07:15:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.107+0000 7f8fd7e00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 0x7f8fb4079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.108+0000 7f8fd7e00640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 0x7f8fb4079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80b1670 tx=0x7f8fc8009040 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:40.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.111+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8fd80ca670 con 0x7f8fd8161be0 2024-07-17T07:15:40.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.358+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1y", "target": ["mon-mgr", ""]}) -- 0x7f8fd80cfdf0 con 0x7f8fb4076db0 2024-07-17T07:15:40.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.538+0000 7f8fd4c00640 1 -- 172.21.15.44:0/1655804825 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f8fd80cfdf0 con 0x7f8fb4076db0 2024-07-17T07:15:40.538 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1/yearly 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 msgr2=0x7f8fb4079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 0x7f8fb4079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80b1670 tx=0x7f8fc8009040 comp rx=0 tx=0).stop 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 msgr2=0x7f8fd80b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000ec20 tx=0x7f8fd000c6a0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 shutdown_connections 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f8fb4076db0 0x7f8fb4079270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8fd8161be0 0x7f8fd80b1c00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fd815d130 0x7f8fd8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 --2- 172.21.15.44:0/1655804825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd80ce690 0x7f8fd8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 >> 172.21.15.44:0/1655804825 conn(0x7f8fd80b4ff0 msgr2=0x7f8fd80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:40.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 shutdown_connections 2024-07-17T07:15:40.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.542+0000 7f8fdd400640 1 -- 172.21.15.44:0/1655804825 wait complete. 2024-07-17T07:15:40.563 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2024-07-17T07:15:40.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:40.563 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2024-07-17T07:15:40.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 -- 172.21.15.44:0/1378967748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 msgr2=0x7f735806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 --2- 172.21.15.44:0/1378967748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f735806d850 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7358070830 tx=0x7f735002aa10 comp rx=0 tx=0).stop 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 -- 172.21.15.44:0/1378967748 shutdown_connections 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 --2- 172.21.15.44:0/1378967748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f735806d850 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 --2- 172.21.15.44:0/1378967748 >> v2:172.21.15.175:3300/0 conn(0x7f735806f4d0 0x7f735815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 -- 172.21.15.44:0/1378967748 >> 172.21.15.44:0/1378967748 conn(0x7f73580cf0d0 msgr2=0x7f73580cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:40.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 -- 172.21.15.44:0/1378967748 shutdown_connections 2024-07-17T07:15:40.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.842+0000 7f735d000640 1 -- 172.21.15.44:0/1378967748 wait complete. 2024-07-17T07:15:40.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.845+0000 7f735d000640 1 Processor -- start 2024-07-17T07:15:40.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.845+0000 7f735d000640 1 -- start start 2024-07-17T07:15:40.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f73581fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f735806f4d0 0x7f735815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73581503c0 0x7f7358148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73580ac4e0 con 0x7f735806d420 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73580ac360 con 0x7f73581503c0 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.846+0000 7f735d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73580ac660 con 0x7f735806f4d0 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f73581fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f73581fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45550/0 (socket says 172.21.15.44:45550) 2024-07-17T07:15:40.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 -- 172.21.15.44:0/2555547541 learned_addr learned my addr 172.21.15.44:0/2555547541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 -- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f735806f4d0 msgr2=0x7f735815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f735806f4d0 0x7f735815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 -- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73581503c0 msgr2=0x7f7358148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73581503c0 0x7f7358148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.847+0000 7f7357400640 1 -- 172.21.15.44:0/2555547541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73581fc290 con 0x7f735806d420 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f7357400640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735806d420 0x7f73581fbc00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f734800c920 tx=0x7f734800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348007c70 con 0x7f735806d420 2024-07-17T07:15:40.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f734800f450 con 0x7f735806d420 2024-07-17T07:15:40.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734800f8a0 con 0x7f735806d420 2024-07-17T07:15:40.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f735d000640 1 -- 172.21.15.44:0/2555547541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73581fc570 con 0x7f735806d420 2024-07-17T07:15:40.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.848+0000 7f735d000640 1 -- 172.21.15.44:0/2555547541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735815abb0 con 0x7f735806d420 2024-07-17T07:15:40.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.850+0000 7f735d000640 1 -- 172.21.15.44:0/2555547541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73580cffe0 con 0x7f735806d420 2024-07-17T07:15:40.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.850+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f734800fa40 con 0x7f735806d420 2024-07-17T07:15:40.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.851+0000 7f734f400640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f7338076db0 0x7f7338079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:40.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.852+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f73480994d0 con 0x7f735806d420 2024-07-17T07:15:40.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.852+0000 7f7356a00640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f7338076db0 0x7f7338079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:40.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.853+0000 7f7356a00640 1 --2- 172.21.15.44:0/2555547541 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f7338076db0 0x7f7338079270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7344006fd0 tx=0x7f7344006bc0 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:40.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:40.856+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7348014030 con 0x7f735806d420 2024-07-17T07:15:41.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.091+0000 7f735d000640 1 -- 172.21.15.44:0/2555547541 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) -- 0x7f73580cee90 con 0x7f7338076db0 2024-07-17T07:15:41.093 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:41.094+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,y 2024-07-17T07:15:41.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.094+0000 7f734f400640 1 -- 172.21.15.44:0/2555547541 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,y) ==== 74+0+0 (secure 0 0 0) 0x7f73580cee90 con 0x7f7338076db0 2024-07-17T07:15:41.094 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,y 2024-07-17T07:15:41.108 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:41.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2024-07-17T07:15:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 -- 172.21.15.44:0/1383928465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43180cd0b0 msgr2=0x7f431806ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 --2- 172.21.15.44:0/1383928465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43180cd0b0 0x7f431806ec70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4304005fd0 tx=0x7f430402a5a0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 -- 172.21.15.44:0/1383928465 shutdown_connections 2024-07-17T07:15:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 --2- 172.21.15.44:0/1383928465 >> v2:172.21.15.175:3300/0 conn(0x7f4318152300 0x7f431806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 --2- 172.21.15.44:0/1383928465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43180cd0b0 0x7f431806ec70 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 -- 172.21.15.44:0/1383928465 >> 172.21.15.44:0/1383928465 conn(0x7f43180cf0d0 msgr2=0x7f43180cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 -- 172.21.15.44:0/1383928465 shutdown_connections 2024-07-17T07:15:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.329+0000 7f431ec00640 1 -- 172.21.15.44:0/1383928465 wait complete. 2024-07-17T07:15:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.331+0000 7f431ec00640 1 Processor -- start 2024-07-17T07:15:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.331+0000 7f431ec00640 1 -- start start 2024-07-17T07:15:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43180cd0b0 0x7f431809cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4318151f20 0x7f43181f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4318152300 0x7f4318144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43180cb720 con 0x7f4318152300 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43180cb5a0 con 0x7f4318151f20 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43180cb8a0 con 0x7f43180cd0b0 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.332+0000 7f431e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4318152300 0x7f4318144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4318152300 0x7f4318144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45568/0 (socket says 172.21.15.44:45568) 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 -- 172.21.15.44:0/2485864434 learned_addr learned my addr 172.21.15.44:0/2485864434 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:41.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431ce00640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4318151f20 0x7f43181f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 -- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43180cd0b0 msgr2=0x7f431809cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43180cd0b0 0x7f431809cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 -- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4318151f20 msgr2=0x7f43181f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4318151f20 0x7f43181f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 -- 172.21.15.44:0/2485864434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181f8040 con 0x7f4318152300 2024-07-17T07:15:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431ce00640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4318151f20 0x7f43181f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.333+0000 7f431e200640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4318152300 0x7f4318144190 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f431000ea80 tx=0x7f431000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.334+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310004b20 con 0x7f4318152300 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.334+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4310004ce0 con 0x7f4318152300 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.334+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310005280 con 0x7f4318152300 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.334+0000 7f431ec00640 1 -- 172.21.15.44:0/2485864434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431815c680 con 0x7f4318152300 2024-07-17T07:15:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.334+0000 7f431ec00640 1 -- 172.21.15.44:0/2485864434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431815cc40 con 0x7f4318152300 2024-07-17T07:15:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.335+0000 7f431ec00640 1 -- 172.21.15.44:0/2485864434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43180caa10 con 0x7f4318152300 2024-07-17T07:15:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.336+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f4310005420 con 0x7f4318152300 2024-07-17T07:15:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.337+0000 7f4315600640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f42f8076db0 0x7f42f8079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.337+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f4310099980 con 0x7f4318152300 2024-07-17T07:15:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.338+0000 7f431d800640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f42f8076db0 0x7f42f8079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.339+0000 7f431d800640 1 --2- 172.21.15.44:0/2485864434 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f42f8076db0 0x7f42f8079270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f430c000f60 tx=0x7f430c0079b0 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.341+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f43181f8040 con 0x7f4318152300 2024-07-17T07:15:41.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.578+0000 7f431ec00640 1 -- 172.21.15.44:0/2485864434 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) -- 0x7f4318074ee0 con 0x7f42f8076db0 2024-07-17T07:15:41.580 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:41.581+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.582+0000 7f4315600640 1 -- 172.21.15.44:0/2485864434 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc.) ==== 138+0+0 (secure 0 0 0) 0x7f4318074ee0 con 0x7f42f8076db0 2024-07-17T07:15:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:41.598 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:41.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.818+0000 7f44d9a00640 1 -- 172.21.15.44:0/2757851989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 msgr2=0x7f44d40cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.818+0000 7f44d9a00640 1 --2- 172.21.15.44:0/2757851989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d40cb810 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f44d40d0260 tx=0x7f44cc02a980 comp rx=0 tx=0).stop 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 -- 172.21.15.44:0/2757851989 shutdown_connections 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 --2- 172.21.15.44:0/2757851989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44d406ed30 0x7f44d415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 --2- 172.21.15.44:0/2757851989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d40cb810 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 --2- 172.21.15.44:0/2757851989 >> v2:172.21.15.175:3300/0 conn(0x7f44d4152220 0x7f44d406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 -- 172.21.15.44:0/2757851989 >> 172.21.15.44:0/2757851989 conn(0x7f44d40b0eb0 msgr2=0x7f44d40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:41.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 -- 172.21.15.44:0/2757851989 shutdown_connections 2024-07-17T07:15:41.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.819+0000 7f44d9a00640 1 -- 172.21.15.44:0/2757851989 wait complete. 2024-07-17T07:15:41.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.821+0000 7f44d9a00640 1 Processor -- start 2024-07-17T07:15:41.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.821+0000 7f44d9a00640 1 -- start start 2024-07-17T07:15:41.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44d406ed30 0x7f44d41527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d41f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44d4152220 0x7f44d41f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44d415f1f0 con 0x7f44d40cb3a0 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44d415f070 con 0x7f44d406ed30 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44d9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f44d415f370 con 0x7f44d4152220 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d41f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44ca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d41f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45576/0 (socket says 172.21.15.44:45576) 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.822+0000 7f44ca000640 1 -- 172.21.15.44:0/865220497 learned_addr learned my addr 172.21.15.44:0/865220497 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44d2a00640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44d406ed30 0x7f44d41527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 -- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44d4152220 msgr2=0x7f44d41f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44d4152220 0x7f44d41f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 -- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44d406ed30 msgr2=0x7f44d41527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:41.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44d406ed30 0x7f44d41527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:41.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 -- 172.21.15.44:0/865220497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d415b520 con 0x7f44d40cb3a0 2024-07-17T07:15:41.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44ca000640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d40cb3a0 0x7f44d41f7470 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00d770 tx=0x7f44bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:41.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44bc009d20 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc005280 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44d9a00640 1 -- 172.21.15.44:0/865220497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d40ac3e0 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.823+0000 7f44d9a00640 1 -- 172.21.15.44:0/865220497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d40ac9a0 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.825+0000 7f44d9a00640 1 -- 172.21.15.44:0/865220497 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d40cb810 con 0x7f44d40cb3a0 2024-07-17T07:15:41.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.826+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f44bc005420 con 0x7f44d40cb3a0 2024-07-17T07:15:41.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.827+0000 7f44cb400640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f44b0076de0 0x7f44b00792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:41.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.827+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f44bc098dc0 con 0x7f44d40cb3a0 2024-07-17T07:15:41.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.828+0000 7f44d2a00640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f44b0076de0 0x7f44b00792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:41.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.829+0000 7f44d2a00640 1 --2- 172.21.15.44:0/865220497 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f44b0076de0 0x7f44b00792a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f44d41eaf70 tx=0x7f44c4005c90 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:41.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:41.831+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f44bc0619e0 con 0x7f44d40cb3a0 2024-07-17T07:15:42.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.070+0000 7f44d9a00640 1 -- 172.21.15.44:0/865220497 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) -- 0x7f44d415d370 con 0x7f44b0076de0 2024-07-17T07:15:42.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:42.073+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:42.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.073+0000 7f44cb400640 1 -- 172.21.15.44:0/865220497 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc.) ==== 138+0+0 (secure 0 0 0) 0x7f44d415d370 con 0x7f44b0076de0 2024-07-17T07:15:42.072 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:42.086 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:42.087 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2024-07-17T07:15:42.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 -- 172.21.15.44:0/1631426440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b6790 msgr2=0x7f77f81648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 --2- 172.21.15.44:0/1631426440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b6790 0x7f77f81648f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009830 tx=0x7f77e002a5a0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 -- 172.21.15.44:0/1631426440 shutdown_connections 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 --2- 172.21.15.44:0/1631426440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f80b29e0 0x7f77f8166e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 --2- 172.21.15.44:0/1631426440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b6790 0x7f77f81648f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.309+0000 7f77fce00640 1 -- 172.21.15.44:0/1631426440 >> 172.21.15.44:0/1631426440 conn(0x7f77f80d3700 msgr2=0x7f77f80d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.310+0000 7f77fce00640 1 -- 172.21.15.44:0/1631426440 shutdown_connections 2024-07-17T07:15:42.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.310+0000 7f77fce00640 1 -- 172.21.15.44:0/1631426440 wait complete. 2024-07-17T07:15:42.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.311+0000 7f77fce00640 1 Processor -- start 2024-07-17T07:15:42.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.311+0000 7f77fce00640 1 -- start start 2024-07-17T07:15:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b29e0 0x7f77f80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f80b6790 0x7f77f8155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f80ce1d0 0x7f77f81f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8167710 con 0x7f77f80b29e0 2024-07-17T07:15:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8167590 con 0x7f77f80ce1d0 2024-07-17T07:15:42.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.312+0000 7f77fce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77f8167890 con 0x7f77f80b6790 2024-07-17T07:15:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b29e0 0x7f77f80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b29e0 0x7f77f80c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45604/0 (socket says 172.21.15.44:45604) 2024-07-17T07:15:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 -- 172.21.15.44:0/1217365697 learned_addr learned my addr 172.21.15.44:0/1217365697 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5400640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f80b6790 0x7f77f8155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f6800640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f80ce1d0 0x7f77f81f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 -- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f80b6790 msgr2=0x7f77f8155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f80b6790 0x7f77f8155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 -- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f80ce1d0 msgr2=0x7f77f81f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f80ce1d0 0x7f77f81f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f5e00640 1 -- 172.21.15.44:0/1217365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f815bc40 con 0x7f77f80b29e0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.313+0000 7f77f6800640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f80ce1d0 0x7f77f81f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77f5e00640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f80b29e0 0x7f77f80c9230 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c3c0 tx=0x7f77e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8017070 con 0x7f77f80b29e0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e80035f0 con 0x7f77f80b29e0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8003a40 con 0x7f77f80b29e0 2024-07-17T07:15:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77fce00640 1 -- 172.21.15.44:0/1217365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f81f4b30 con 0x7f77f80b29e0 2024-07-17T07:15:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.314+0000 7f77fce00640 1 -- 172.21.15.44:0/1217365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f81f50f0 con 0x7f77f80b29e0 2024-07-17T07:15:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.315+0000 7f77fce00640 1 -- 172.21.15.44:0/1217365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f80cf200 con 0x7f77f80b29e0 2024-07-17T07:15:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.317+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f77e801a070 con 0x7f77f80b29e0 2024-07-17T07:15:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.318+0000 7f77ee000640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f77d4076db0 0x7f77d4079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.318+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f77e8097e90 con 0x7f77f80b29e0 2024-07-17T07:15:42.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.318+0000 7f77f5400640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f77d4076db0 0x7f77d4079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.320+0000 7f77f5400640 1 --2- 172.21.15.44:0/1217365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f77d4076db0 0x7f77d4079270 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77e0034000 tx=0x7f77e0004730 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:42.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.323+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f77f80cf200 con 0x7f77f80b29e0 2024-07-17T07:15:42.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.557+0000 7f77fce00640 1 -- 172.21.15.44:0/1217365697 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) -- 0x7f77f80cfaa0 con 0x7f77d4076db0 2024-07-17T07:15:42.559 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:42.560+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.561+0000 7f77ee000640 1 -- 172.21.15.44:0/1217365697 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc.) ==== 138+0+0 (secure 0 0 0) 0x7f77f80cfaa0 con 0x7f77d4076db0 2024-07-17T07:15:42.559 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:42.581 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:42.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2024-07-17T07:15:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 -- 172.21.15.44:0/554249496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 msgr2=0x7feb9c06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 --2- 172.21.15.44:0/554249496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 0x7feb9c06dfd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feb88009830 tx=0x7feb8802a510 comp rx=0 tx=0).stop 2024-07-17T07:15:42.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 -- 172.21.15.44:0/554249496 shutdown_connections 2024-07-17T07:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 --2- 172.21.15.44:0/554249496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb9c06e510 0x7feb9c0ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 --2- 172.21.15.44:0/554249496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 0x7feb9c06dfd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 -- 172.21.15.44:0/554249496 >> 172.21.15.44:0/554249496 conn(0x7feb9c0cf0d0 msgr2=0x7feb9c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 -- 172.21.15.44:0/554249496 shutdown_connections 2024-07-17T07:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.801+0000 7feba1000640 1 -- 172.21.15.44:0/554249496 wait complete. 2024-07-17T07:15:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.803+0000 7feba1000640 1 Processor -- start 2024-07-17T07:15:42.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.803+0000 7feba1000640 1 -- start start 2024-07-17T07:15:42.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb9c06e510 0x7feb9c072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 0x7feb9c072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb9c154490 0x7feb9c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0b2220 con 0x7feb9c154490 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0b20a0 con 0x7feb9c06f4f0 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feba1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0b23a0 con 0x7feb9c06e510 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feb9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb9c154490 0x7feb9c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feb9be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb9c154490 0x7feb9c1f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45614/0 (socket says 172.21.15.44:45614) 2024-07-17T07:15:42.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feb9be00640 1 -- 172.21.15.44:0/3725237337 learned_addr learned my addr 172.21.15.44:0/3725237337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feb9aa00640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 0x7feb9c072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.805+0000 7feb9b400640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb9c06e510 0x7feb9c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 -- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb9c06e510 msgr2=0x7feb9c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb9c06e510 0x7feb9c072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 -- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 msgr2=0x7feb9c072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb9c06f4f0 0x7feb9c072d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 -- 172.21.15.44:0/3725237337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0713e0 con 0x7feb9c154490 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb9be00640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb9c154490 0x7feb9c1f7240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7feb9400eb40 tx=0x7feb9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94004b20 con 0x7feb9c154490 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feba1000640 1 -- 172.21.15.44:0/3725237337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c157b30 con 0x7feb9c154490 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb94004db0 con 0x7feb9c154490 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94005280 con 0x7feb9c154490 2024-07-17T07:15:42.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.806+0000 7feba1000640 1 -- 172.21.15.44:0/3725237337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c157850 con 0x7feb9c154490 2024-07-17T07:15:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.808+0000 7feba1000640 1 -- 172.21.15.44:0/3725237337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c0b2930 con 0x7feb9c154490 2024-07-17T07:15:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.809+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7feb94005420 con 0x7feb9c154490 2024-07-17T07:15:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.810+0000 7feb93400640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feb7c076db0 0x7feb7c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:42.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.810+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7feb94099ba0 con 0x7feb9c154490 2024-07-17T07:15:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.811+0000 7feb9b400640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feb7c076db0 0x7feb7c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:42.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.812+0000 7feb9b400640 1 --2- 172.21.15.44:0/3725237337 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feb7c076db0 0x7feb7c079270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0b3ce0 tx=0x7feb8c009040 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:42.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:42.815+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feb94062740 con 0x7feb9c154490 2024-07-17T07:15:43.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.057+0000 7feba1000640 1 -- 172.21.15.44:0/3725237337 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) -- 0x7feb9c0cf9e0 con 0x7feb7c076db0 2024-07-17T07:15:43.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:43.060+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2024-07-17T07:15:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.061+0000 7feb93400640 1 -- 172.21.15.44:0/3725237337 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) ==== 96+0+0 (secure 0 0 0) 0x7feb9c0cf9e0 con 0x7feb7c076db0 2024-07-17T07:15:43.060 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2024-07-17T07:15:43.079 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:43.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2024-07-17T07:15:43.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 -- 172.21.15.44:0/858264468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd00acab0 msgr2=0x7f3dd006eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:43.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 --2- 172.21.15.44:0/858264468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd00acab0 0x7f3dd006eb90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3dd006f250 tx=0x7f3dc802bbe0 comp rx=0 tx=0).stop 2024-07-17T07:15:43.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 -- 172.21.15.44:0/858264468 shutdown_connections 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 --2- 172.21.15.44:0/858264468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd00acab0 0x7f3dd006eb90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 --2- 172.21.15.44:0/858264468 >> v2:172.21.15.175:3300/0 conn(0x7f3dd006f6d0 0x7f3dd015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 -- 172.21.15.44:0/858264468 >> 172.21.15.44:0/858264468 conn(0x7f3dd00b0eb0 msgr2=0x7f3dd00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 -- 172.21.15.44:0/858264468 shutdown_connections 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.298+0000 7f3dd7600640 1 -- 172.21.15.44:0/858264468 wait complete. 2024-07-17T07:15:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.300+0000 7f3dd7600640 1 Processor -- start 2024-07-17T07:15:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.300+0000 7f3dd7600640 1 -- start start 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd006f6d0 0x7f3dd0143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3dd00acab0 0x7f3dd01f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3dd00b3280 0x7f3dd0154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00aea90 con 0x7f3dd006f6d0 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00ae910 con 0x7f3dd00b3280 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd00aec10 con 0x7f3dd00acab0 2024-07-17T07:15:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd006f6d0 0x7f3dd0143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd006f6d0 0x7f3dd0143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45638/0 (socket says 172.21.15.44:45638) 2024-07-17T07:15:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.301+0000 7f3dd4c00640 1 -- 172.21.15.44:0/2348621649 learned_addr learned my addr 172.21.15.44:0/2348621649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd5600640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3dd00b3280 0x7f3dd0154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dcfe00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3dd00acab0 0x7f3dd01f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 -- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3dd00acab0 msgr2=0x7f3dd01f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3dd00acab0 0x7f3dd01f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 -- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3dd00b3280 msgr2=0x7f3dd0154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3dd00b3280 0x7f3dd0154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 -- 172.21.15.44:0/2348621649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd0144250 con 0x7f3dd006f6d0 2024-07-17T07:15:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd5600640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3dd00b3280 0x7f3dd0154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dcfe00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3dd00acab0 0x7f3dd01f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.302+0000 7f3dd4c00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dd006f6d0 0x7f3dd0143d10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400d950 tx=0x7f3dc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.303+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4017070 con 0x7f3dd006f6d0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.303+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc4009d20 con 0x7f3dd006f6d0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.303+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40106c0 con 0x7f3dd006f6d0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.303+0000 7f3dd7600640 1 -- 172.21.15.44:0/2348621649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd01415a0 con 0x7f3dd006f6d0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.303+0000 7f3dd7600640 1 -- 172.21.15.44:0/2348621649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd0141b60 con 0x7f3dd006f6d0 2024-07-17T07:15:43.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.304+0000 7f3dd7600640 1 -- 172.21.15.44:0/2348621649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd015a5a0 con 0x7f3dd006f6d0 2024-07-17T07:15:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.305+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f3dc4010860 con 0x7f3dd006f6d0 2024-07-17T07:15:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.306+0000 7f3dccc00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3db0076db0 0x7f3db0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.306+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f3dc4098f90 con 0x7f3dd006f6d0 2024-07-17T07:15:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.307+0000 7f3dcfe00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3db0076db0 0x7f3db0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.308+0000 7f3dcfe00640 1 --2- 172.21.15.44:0/2348621649 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3db0076db0 0x7f3db0079270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc001220 tx=0x7f3dbc009040 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.310+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3dd0144250 con 0x7f3dd006f6d0 2024-07-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.538+0000 7f3dd7600640 1 -- 172.21.15.44:0/2348621649 --> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) -- 0x7f3dd00b4200 con 0x7f3db0076db0 2024-07-17T07:15:43.540 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:43.541+0000 7f3665e00640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:43.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:43.542+0000 7f3dccc00640 1 -- 172.21.15.44:0/2348621649 <== mgr.6332 v2:172.21.15.175:6800/1241312127 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc.) ==== 138+0+0 (secure 0 0 0) 0x7f3dd00b4200 con 0x7f3db0076db0 2024-07-17T07:15:43.541 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,y e.g. 1h or 4d etc. 2024-07-17T07:15:43.561 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T07:15:43.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.561 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/minutes/.snap) 2024-07-17T07:15:43.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.615 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2024-07-17T07:15:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.669 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/hourly/.snap) 2024-07-17T07:15:43.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.725 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2024-07-17T07:15:43.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.782 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/daily/.snap) 2024-07-17T07:15:43.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.840 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2024-07-17T07:15:43.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.898 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/weekly/.snap) 2024-07-17T07:15:43.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:43.951 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2024-07-17T07:15:44.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.006 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/monthly/.snap) 2024-07-17T07:15:44.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.062 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2024-07-17T07:15:44.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.118 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/yearly/.snap) 2024-07-17T07:15:44.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.173 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2024-07-17T07:15:44.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.229 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/bad_period_spec/.snap) 2024-07-17T07:15:44.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:44.288 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2024-07-17T07:15:44.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T07:15:44.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.557+0000 7f9dda600640 1 -- 172.21.15.44:0/386955450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dbc004d40 msgr2=0x7f9dbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.557+0000 7f9dda600640 1 --2- 172.21.15.44:0/386955450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009e20 tx=0x7f9dbc005830 comp rx=0 tx=0).stop 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.557+0000 7f9dd9c00640 1 -- 172.21.15.44:0/386955450 reap_dead start 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.557+0000 7f9dda600640 1 -- 172.21.15.44:0/386955450 shutdown_connections 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.557+0000 7f9dda600640 1 -- 172.21.15.44:0/386955450 >> 172.21.15.44:0/386955450 conn(0x7f9dd406c120 msgr2=0x7f9dd406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.558+0000 7f9dda600640 1 -- 172.21.15.44:0/386955450 shutdown_connections 2024-07-17T07:15:44.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.558+0000 7f9dda600640 1 -- 172.21.15.44:0/386955450 wait complete. 2024-07-17T07:15:44.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.559+0000 7f9dda600640 1 Processor -- start 2024-07-17T07:15:44.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.559+0000 7f9dda600640 1 -- start start 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc004d40 0x7f9dd41f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 0x7f9dd4159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40ce670 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd40adfd0 con 0x7f9dbc004d40 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.560+0000 7f9dda600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd40ce7f0 con 0x7f9dd41f5230 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45658/0 (socket says 172.21.15.44:45658) 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 -- 172.21.15.44:0/3542284762 learned_addr learned my addr 172.21.15.44:0/3542284762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd9200640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc004d40 0x7f9dd41f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd9c00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 0x7f9dd4159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 -- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 msgr2=0x7f9dd4159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 0x7f9dd4159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 -- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc004d40 msgr2=0x7f9dd41f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc004d40 0x7f9dd41f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd3e00640 1 -- 172.21.15.44:0/3542284762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd41f5720 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.561+0000 7f9dd9c00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 0x7f9dd4159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dd3e00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400fa00 tx=0x7f9dc400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400bcb0 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dc4011440 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4011870 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd41f5a00 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.562+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd4074030 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.563+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd41f5c40 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.565+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 74) ==== 99005+0+0 (secure 0 0 0) 0x7f9dc4019020 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.566+0000 7f9dd0c00640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 0x7f9db80792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:44.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.566+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f9dc4099670 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.567+0000 7f9dd9200640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 0x7f9db80792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:44.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.568+0000 7f9dd9200640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 0x7f9db80792a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8001cb0 tx=0x7f9dc8001c40 comp rx=0 tx=0).ready entity=mgr.6332 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:44.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.571+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9dd41f5c40 con 0x7f9dd40ca8d0 2024-07-17T07:15:44.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:44.836+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f9dd40ce130 con 0x7f9dd40ca8d0 2024-07-17T07:15:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.768+0000 7f9dd0c00640 1 -- 172.21.15.44:0/3542284762 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v75) ==== 93+0+0 (secure 0 0 0) 0x7f9dc4062200 con 0x7f9dd40ca8d0 2024-07-17T07:15:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.772+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 msgr2=0x7f9db80792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.772+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 0x7f9db80792a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8001cb0 tx=0x7f9dc8001c40 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.772+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 msgr2=0x7f9dd41f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.772+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400fa00 tx=0x7f9dc400fec0 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 shutdown_connections 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f9db8076de0 0x7f9db80792a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dd41f5230 0x7f9dd4159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9dd40ca8d0 0x7f9dd41f4cf0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 --2- 172.21.15.44:0/3542284762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc004d40 0x7f9dd41f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 >> 172.21.15.44:0/3542284762 conn(0x7f9dd406c120 msgr2=0x7f9dd41584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 shutdown_connections 2024-07-17T07:15:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:45.773+0000 7f9dda600640 1 -- 172.21.15.44:0/3542284762 wait complete. 2024-07-17T07:15:45.793 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:15:45.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:45.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:45.981 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:15:45.981 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:45.981 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:45.981 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:46.027 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:46.028 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:46.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.320+0000 7fbb6ae00640 1 -- 172.21.15.44:0/3936976109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb4c004800 msgr2=0x7fbb4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.320+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/3936976109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb4c004800 0x7fbb4c004c80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c2e0 tx=0x7fbb4c005830 comp rx=0 tx=0).stop 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.321+0000 7fbb6ae00640 1 -- 172.21.15.44:0/3936976109 shutdown_connections 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.321+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/3936976109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb4c004800 0x7fbb4c004c80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.321+0000 7fbb6ae00640 1 -- 172.21.15.44:0/3936976109 >> 172.21.15.44:0/3936976109 conn(0x7fbb640cf0d0 msgr2=0x7fbb640cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.321+0000 7fbb6ae00640 1 -- 172.21.15.44:0/3936976109 shutdown_connections 2024-07-17T07:15:46.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.321+0000 7fbb6ae00640 1 -- 172.21.15.44:0/3936976109 wait complete. 2024-07-17T07:15:46.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.323+0000 7fbb6ae00640 1 Processor -- start 2024-07-17T07:15:46.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.323+0000 7fbb6ae00640 1 -- start start 2024-07-17T07:15:46.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb4c004800 0x7fbb64158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:46.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64073250 0x7fbb640736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6406dc20 con 0x7fbb64072220 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6406daa0 con 0x7fbb64073250 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406dda0 con 0x7fbb4c004800 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45696/0 (socket says 172.21.15.44:45696) 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6a400640 1 -- 172.21.15.44:0/7375993 learned_addr learned my addr 172.21.15.44:0/7375993 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb69a00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb4c004800 0x7fbb64158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb69000640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64073250 0x7fbb640736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.324+0000 7fbb6a400640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb4c004800 msgr2=0x7fbb64158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6a400640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb4c004800 0x7fbb64158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6a400640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64073250 msgr2=0x7fbb640736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6a400640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64073250 0x7fbb640736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6a400640 1 -- 172.21.15.44:0/7375993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb64071480 con 0x7fbb64072220 2024-07-17T07:15:46.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6a400640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbb6415ce40 tx=0x7fbb5c00aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:46.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c019070 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb5c007be0 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c003970 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb64071760 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.325+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6414c950 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.327+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb640ae550 con 0x7fbb64072220 2024-07-17T07:15:46.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.328+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7fbb5c01c070 con 0x7fbb64072220 2024-07-17T07:15:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.329+0000 7fbb61600640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 0x7fbb44079240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.329+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7fbb5c0984d0 con 0x7fbb64072220 2024-07-17T07:15:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.329+0000 7fbb69a00640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 msgr2=0x7fbb44079240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.329+0000 7fbb69a00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 0x7fbb44079240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.334+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbb5c011070 con 0x7fbb64072220 2024-07-17T07:15:46.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.531+0000 7fbb69a00640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 msgr2=0x7fbb44079240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:46.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.531+0000 7fbb69a00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 0x7fbb44079240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:46.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.557+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb64071930 con 0x7fbb64072220 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.558+0000 7fbb61600640 1 -- 172.21.15.44:0/7375993 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v143) ==== 80+0+477 (secure 0 0 0) 0x7fbb5c061100 con 0x7fbb64072220 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4151616996 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3602435312 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3869263054 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/526872756 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.557 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4113173171 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.558 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/2606536318 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.558 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/2606536318 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.558 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2312626755 2024-07-18T07:15:35.635020+0000 2024-07-17T07:15:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T07:15:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 msgr2=0x7fbb44079240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 0x7fbb44079240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 msgr2=0x7fbb640726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbb6415ce40 tx=0x7fbb5c00aa70 comp rx=0 tx=0).stop 2024-07-17T07:15:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 shutdown_connections 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fbb44076d80 0x7fbb44079240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb64072220 0x7fbb640726a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb64073250 0x7fbb640736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 --2- 172.21.15.44:0/7375993 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb4c004800 0x7fbb64158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 >> 172.21.15.44:0/7375993 conn(0x7fbb640cf0d0 msgr2=0x7fbb640b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 shutdown_connections 2024-07-17T07:15:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:46.561+0000 7fbb6ae00640 1 -- 172.21.15.44:0/7375993 wait complete. 2024-07-17T07:15:46.579 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T07:15:46.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:46.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T07:15:46.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:15:46.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:15:46.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T07:15:46.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:15:46.658 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T07:15:46.738 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:46.738 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:46.738 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:46.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:46.783 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:46.783 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:46.783 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:46.825 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:46.825 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T07:15:46.825 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:46.825 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:46.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:46.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 -- 172.21.15.44:0/4052266188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 msgr2=0x7fe3e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 --2- 172.21.15.44:0/4052266188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c6e0 tx=0x7fe3e0005830 comp rx=0 tx=0).stop 2024-07-17T07:15:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 -- 172.21.15.44:0/4052266188 shutdown_connections 2024-07-17T07:15:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 --2- 172.21.15.44:0/4052266188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 -- 172.21.15.44:0/4052266188 >> 172.21.15.44:0/4052266188 conn(0x7fe3fc0b2240 msgr2=0x7fe3fc0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:47.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 -- 172.21.15.44:0/4052266188 shutdown_connections 2024-07-17T07:15:47.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.075+0000 7fe400e00640 1 -- 172.21.15.44:0/4052266188 wait complete. 2024-07-17T07:15:47.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.077+0000 7fe400e00640 1 Processor -- start 2024-07-17T07:15:47.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.077+0000 7fe400e00640 1 -- start start 2024-07-17T07:15:47.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3fc14e3f0 0x7fe3fc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3fc0aea20 0x7fe3fc0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b5e80 con 0x7fe3e0004800 2024-07-17T07:15:47.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b5d00 con 0x7fe3fc14e3f0 2024-07-17T07:15:47.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe400e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b6000 con 0x7fe3fc0aea20 2024-07-17T07:15:47.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe3f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe3f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45708/0 (socket says 172.21.15.44:45708) 2024-07-17T07:15:47.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.078+0000 7fe3f9e00640 1 -- 172.21.15.44:0/3524450688 learned_addr learned my addr 172.21.15.44:0/3524450688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:47.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f3e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3fc14e3f0 0x7fe3fc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3fc0aea20 msgr2=0x7fe3fc0aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3fa800640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3fc0aea20 0x7fe3fc0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3fc0aea20 0x7fe3fc0aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3fc14e3f0 msgr2=0x7fe3fc14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3fc14e3f0 0x7fe3fc14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 -- 172.21.15.44:0/3524450688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc14e940 con 0x7fe3e0004800 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f9e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00f770 tx=0x7fe3ec00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.079+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec00bcb0 con 0x7fe3e0004800 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.080+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3ec00fe80 con 0x7fe3e0004800 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.080+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0116a0 con 0x7fe3e0004800 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.080+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc14ec20 con 0x7fe3e0004800 2024-07-17T07:15:47.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.080+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc14f1e0 con 0x7fe3e0004800 2024-07-17T07:15:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.081+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc15f0c0 con 0x7fe3e0004800 2024-07-17T07:15:47.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.082+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7fe3ec019020 con 0x7fe3e0004800 2024-07-17T07:15:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.083+0000 7fe3f2000640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 0x7fe3d8079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.083+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7fe3ec099560 con 0x7fe3e0004800 2024-07-17T07:15:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.083+0000 7fe3f3e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 msgr2=0x7fe3d8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:47.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.083+0000 7fe3f3e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 0x7fe3d8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:47.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.087+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe3ec0620f0 con 0x7fe3e0004800 2024-07-17T07:15:47.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.285+0000 7fe3f3e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 msgr2=0x7fe3d8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:47.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.285+0000 7fe3f3e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 0x7fe3d8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:47.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.329+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3fc0b2750 con 0x7fe3e0004800 2024-07-17T07:15:47.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.331+0000 7fe3f2000640 1 -- 172.21.15.44:0/3524450688 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7fe3ec0619f0 con 0x7fe3e0004800 2024-07-17T07:15:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:47.330 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:47.330 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.333+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 msgr2=0x7fe3d8079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.333+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 0x7fe3d8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 msgr2=0x7fe3fc1f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00f770 tx=0x7fe3ec00fc30 comp rx=0 tx=0).stop 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 shutdown_connections 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fe3d8076d90 0x7fe3d8079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3fc0aea20 0x7fe3fc0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3fc14e3f0 0x7fe3fc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 --2- 172.21.15.44:0/3524450688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e0004800 0x7fe3fc1f7a10 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 >> 172.21.15.44:0/3524450688 conn(0x7fe3fc0b2240 msgr2=0x7fe3fc072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:47.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 shutdown_connections 2024-07-17T07:15:47.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.334+0000 7fe400e00640 1 -- 172.21.15.44:0/3524450688 wait complete. 2024-07-17T07:15:47.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 -- 172.21.15.44:0/794537376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b5740 msgr2=0x7fc530161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 --2- 172.21.15.44:0/794537376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b5740 0x7fc530161020 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc530069c00 tx=0x7fc52802aaa0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 -- 172.21.15.44:0/794537376 shutdown_connections 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 --2- 172.21.15.44:0/794537376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b2ad0 0x7fc5300b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 --2- 172.21.15.44:0/794537376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b5740 0x7fc530161020 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 --2- 172.21.15.44:0/794537376 >> v2:172.21.15.175:3300/0 conn(0x7fc53015d600 0x7fc53006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 -- 172.21.15.44:0/794537376 >> 172.21.15.44:0/794537376 conn(0x7fc5300d00a0 msgr2=0x7fc5300d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 -- 172.21.15.44:0/794537376 shutdown_connections 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.536+0000 7fc537c00640 1 -- 172.21.15.44:0/794537376 wait complete. 2024-07-17T07:15:47.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.538+0000 7fc537c00640 1 Processor -- start 2024-07-17T07:15:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.538+0000 7fc537c00640 1 -- start start 2024-07-17T07:15:47.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 0x7fc5300b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53015d600 0x7fc53015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301618f0 con 0x7fc5300b2ad0 2024-07-17T07:15:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc530161770 con 0x7fc53015d600 2024-07-17T07:15:47.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc537c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc530161a70 con 0x7fc5300b5740 2024-07-17T07:15:47.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc535200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc535200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45718/0 (socket says 172.21.15.44:45718) 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc535200640 1 -- 172.21.15.44:0/502365697 learned_addr learned my addr 172.21.15.44:0/502365697 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.539+0000 7fc52fe00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 0x7fc5300b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53015d600 0x7fc53015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 msgr2=0x7fc5300b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 0x7fc5300b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53015d600 msgr2=0x7fc53015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53015d600 0x7fc53015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 -- 172.21.15.44:0/502365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53015edd0 con 0x7fc5300b2ad0 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc52fe00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 0x7fc5300b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.540+0000 7fc535200640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b440 tx=0x7fc52400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:47.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.541+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524015020 con 0x7fc5300b2ad0 2024-07-17T07:15:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.541+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc524011980 con 0x7fc5300b2ad0 2024-07-17T07:15:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.541+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524011dd0 con 0x7fc5300b2ad0 2024-07-17T07:15:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.541+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53015dd30 con 0x7fc5300b2ad0 2024-07-17T07:15:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.541+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53015e2f0 con 0x7fc5300b2ad0 2024-07-17T07:15:47.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.542+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc530161020 con 0x7fc5300b2ad0 2024-07-17T07:15:47.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.543+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7fc524018020 con 0x7fc5300b2ad0 2024-07-17T07:15:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.544+0000 7fc52cc00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 0x7fc510079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.544+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7fc524098cb0 con 0x7fc5300b2ad0 2024-07-17T07:15:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.545+0000 7fc52fe00640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 msgr2=0x7fc510079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.545+0000 7fc52fe00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 0x7fc510079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:47.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.552+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc52401f020 con 0x7fc5300b2ad0 2024-07-17T07:15:47.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.746+0000 7fc52fe00640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 msgr2=0x7fc510079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:47.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.746+0000 7fc52fe00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 0x7fc510079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:47.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.801+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5300c9a90 con 0x7fc5300b2ad0 2024-07-17T07:15:47.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.802+0000 7fc52cc00640 1 -- 172.21.15.44:0/502365697 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7fc52400d040 con 0x7fc5300b2ad0 2024-07-17T07:15:47.801 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:47.801 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:47.801 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 msgr2=0x7fc510079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 0x7fc510079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 msgr2=0x7fc53014f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b440 tx=0x7fc52400b900 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 shutdown_connections 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc510076d60 0x7fc510079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc53015d600 0x7fc53015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5300b5740 0x7fc5300b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 --2- 172.21.15.44:0/502365697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5300b2ad0 0x7fc53014f650 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 >> 172.21.15.44:0/502365697 conn(0x7fc5300d00a0 msgr2=0x7fc5300c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:47.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 shutdown_connections 2024-07-17T07:15:47.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:47.805+0000 7fc537c00640 1 -- 172.21.15.44:0/502365697 wait complete. 2024-07-17T07:15:47.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 -- 172.21.15.44:0/567065788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cded0 msgr2=0x7f70800ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 --2- 172.21.15.44:0/567065788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cded0 0x7f70800ce340 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f708008c9f0 tx=0x7f707802a560 comp rx=0 tx=0).stop 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 -- 172.21.15.44:0/567065788 shutdown_connections 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 --2- 172.21.15.44:0/567065788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cded0 0x7f70800ce340 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 --2- 172.21.15.44:0/567065788 >> v2:172.21.15.175:3300/0 conn(0x7f70800d3420 0x7f70800b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 -- 172.21.15.44:0/567065788 >> 172.21.15.44:0/567065788 conn(0x7f708008ca20 msgr2=0x7f708008ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 -- 172.21.15.44:0/567065788 shutdown_connections 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.014+0000 7f707fe00640 1 -- 172.21.15.44:0/567065788 wait complete. 2024-07-17T07:15:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.016+0000 7f707fe00640 1 Processor -- start 2024-07-17T07:15:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.016+0000 7f707fe00640 1 -- start start 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70800cded0 0x7f70800b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70800d3800 0x7f7080156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70800ce880 con 0x7f70800d3420 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70800cea00 con 0x7f70800cded0 2024-07-17T07:15:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70800ceb80 con 0x7f70800d3800 2024-07-17T07:15:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45734/0 (socket says 172.21.15.44:45734) 2024-07-17T07:15:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.017+0000 7f707e000640 1 -- 172.21.15.44:0/4110044206 learned_addr learned my addr 172.21.15.44:0/4110044206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707ea00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70800cded0 0x7f70800b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70800d3800 msgr2=0x7f7080156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70800d3800 0x7f7080156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70800cded0 msgr2=0x7f70800b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70800cded0 0x7f70800b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 -- 172.21.15.44:0/4110044206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70801f3e90 con 0x7f70800d3420 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707e000640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f706800c730 tx=0x7f706800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068003b70 con 0x7f70800d3420 2024-07-17T07:15:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7068003d30 con 0x7f70800d3420 2024-07-17T07:15:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70801f4170 con 0x7f70800d3420 2024-07-17T07:15:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.018+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706800aab0 con 0x7f70800d3420 2024-07-17T07:15:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.019+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801f4730 con 0x7f70800d3420 2024-07-17T07:15:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.020+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70801f4c20 con 0x7f70800d3420 2024-07-17T07:15:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.021+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7f706800ac50 con 0x7f70800d3420 2024-07-17T07:15:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.022+0000 7f7076a00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 0x7f705c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.022+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f7068098a70 con 0x7f70800d3420 2024-07-17T07:15:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.022+0000 7f707ea00640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 msgr2=0x7f705c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.022+0000 7f707ea00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 0x7f705c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:48.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.030+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7068061720 con 0x7f70800d3420 2024-07-17T07:15:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.224+0000 7f707ea00640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 msgr2=0x7f705c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.224+0000 7f707ea00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 0x7f705c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:48.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.238+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70800ce340 con 0x7f70800d3420 2024-07-17T07:15:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.240+0000 7f7076a00640 1 -- 172.21.15.44:0/4110044206 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+18279 (secure 0 0 0) 0x7f7068061020 con 0x7f70800d3420 2024-07-17T07:15:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:48.239 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":143,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:35.635135+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:15:18.669721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-07-17T07:15:19.770046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4151616996":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3602435312":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3869263054":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/526872756":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/4113173171":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6801/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6800/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/2312626755":"2024-07-18T07:15:35.635020+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 msgr2=0x7f705c079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 0x7f705c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 msgr2=0x7f7080156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f706800c730 tx=0x7f706800cbf0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 shutdown_connections 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f705c076d60 0x7f705c079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70800d3800 0x7f7080156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800d3420 0x7f7080156850 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 --2- 172.21.15.44:0/4110044206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70800cded0 0x7f70800b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 >> 172.21.15.44:0/4110044206 conn(0x7f708008ca20 msgr2=0x7f70800cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 shutdown_connections 2024-07-17T07:15:48.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.243+0000 7f707fe00640 1 -- 172.21.15.44:0/4110044206 wait complete. 2024-07-17T07:15:48.262 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T07:15:48.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.457+0000 7feed3200640 1 -- 172.21.15.44:0/3538996641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 msgr2=0x7feed40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.457+0000 7feed3200640 1 --2- 172.21.15.44:0/3538996641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40b4ec0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feec4009900 tx=0x7feec402a610 comp rx=0 tx=0).stop 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 -- 172.21.15.44:0/3538996641 shutdown_connections 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 --2- 172.21.15.44:0/3538996641 >> v2:172.21.15.175:3300/0 conn(0x7feed40b5400 0x7feed415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 --2- 172.21.15.44:0/3538996641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40b4ec0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 -- 172.21.15.44:0/3538996641 >> 172.21.15.44:0/3538996641 conn(0x7feed40cfec0 msgr2=0x7feed40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 -- 172.21.15.44:0/3538996641 shutdown_connections 2024-07-17T07:15:48.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.458+0000 7feed3200640 1 -- 172.21.15.44:0/3538996641 wait complete. 2024-07-17T07:15:48.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.459+0000 7feed3200640 1 Processor -- start 2024-07-17T07:15:48.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.460+0000 7feed3200640 1 -- start start 2024-07-17T07:15:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.460+0000 7feed3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.460+0000 7feed3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.460+0000 7feed3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 0x7feed40aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feed406e0c0 con 0x7feed40b5400 2024-07-17T07:15:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feed40b92f0 con 0x7feed40ce5c0 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feed406e240 con 0x7feed40b8c80 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45750/0 (socket says 172.21.15.44:45750) 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 -- 172.21.15.44:0/2669986945 learned_addr learned my addr 172.21.15.44:0/2669986945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed2800640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 0x7feed40aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1400640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 msgr2=0x7feed40aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 msgr2=0x7feed40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 0x7feed40aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed1e00640 1 -- 172.21.15.44:0/2669986945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41f3c30 con 0x7feed40b5400 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.461+0000 7feed2800640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 0x7feed40aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:48.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feed1e00640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:48.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec403f070 con 0x7feed40b5400 2024-07-17T07:15:48.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec4005600 con 0x7feed40b5400 2024-07-17T07:15:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41f8000 con 0x7feed40b5400 2024-07-17T07:15:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4035030 con 0x7feed40b5400 2024-07-17T07:15:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.462+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41f41d0 con 0x7feed40b5400 2024-07-17T07:15:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.463+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed406e6f0 con 0x7feed40b5400 2024-07-17T07:15:48.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.464+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7feec4042070 con 0x7feed40b5400 2024-07-17T07:15:48.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.466+0000 7feeca000640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 0x7feeb4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.466+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7feec40b9790 con 0x7feed40b5400 2024-07-17T07:15:48.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.466+0000 7feed1400640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 msgr2=0x7feeb4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:48.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.466+0000 7feed1400640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 0x7feeb4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:48.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.470+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feec40823c0 con 0x7feed40b5400 2024-07-17T07:15:48.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.668+0000 7feed1400640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 msgr2=0x7feeb4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:48.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.668+0000 7feed1400640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 0x7feeb4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:48.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.718+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feed40b5d70 con 0x7feed40b5400 2024-07-17T07:15:48.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.720+0000 7feeca000640 1 -- 172.21.15.44:0/2669986945 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7feec4081cc0 con 0x7feed40b5400 2024-07-17T07:15:48.719 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:48.719 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:48.719 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:48.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.722+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 msgr2=0x7feeb4079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:48.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.722+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 0x7feeb4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.722+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 msgr2=0x7feed40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.722+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4003820 comp rx=0 tx=0).stop 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 shutdown_connections 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7feeb4076d60 0x7feeb4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feed40ce5c0 0x7feed40aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feed40b8c80 0x7feed40aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 --2- 172.21.15.44:0/2669986945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feed40b5400 0x7feed40cc4d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 >> 172.21.15.44:0/2669986945 conn(0x7feed40cfec0 msgr2=0x7feed40d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 shutdown_connections 2024-07-17T07:15:48.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.723+0000 7feed3200640 1 -- 172.21.15.44:0/2669986945 wait complete. 2024-07-17T07:15:48.740 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 -- 172.21.15.44:0/1431323674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc960003aa0 msgr2=0x7fc960003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 --2- 172.21.15.44:0/1431323674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc960003aa0 0x7fc960003ea0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c290 tx=0x7fc960005830 comp rx=0 tx=0).stop 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 -- 172.21.15.44:0/1431323674 shutdown_connections 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 --2- 172.21.15.44:0/1431323674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc960003aa0 0x7fc960003ea0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 -- 172.21.15.44:0/1431323674 >> 172.21.15.44:0/1431323674 conn(0x7fc97c0cfec0 msgr2=0x7fc97c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 -- 172.21.15.44:0/1431323674 shutdown_connections 2024-07-17T07:15:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.935+0000 7fc97b200640 1 -- 172.21.15.44:0/1431323674 wait complete. 2024-07-17T07:15:48.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.936+0000 7fc97b200640 1 Processor -- start 2024-07-17T07:15:48.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.937+0000 7fc97b200640 1 -- start start 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.937+0000 7fc97b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc960003aa0 0x7fc97c0c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc97c0b1a20 0x7fc97c1e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0b54c0 con 0x7fc97c0c8100 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0b5340 con 0x7fc97c0b1a20 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c0b5640 con 0x7fc960003aa0 2024-07-17T07:15:48.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41472/0 (socket says 172.21.15.44:41472) 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 -- 172.21.15.44:0/951362909 learned_addr learned my addr 172.21.15.44:0/951362909 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc979e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc960003aa0 0x7fc97c0c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc97a800640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc97c0b1a20 0x7fc97c1e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc960003aa0 msgr2=0x7fc97c0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc960003aa0 0x7fc97c0c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc97c0b1a20 msgr2=0x7fc97c1e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc97c0b1a20 0x7fc97c1e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:48.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.938+0000 7fc973e00640 1 -- 172.21.15.44:0/951362909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c14f320 con 0x7fc97c0c8100 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc973e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e910 tx=0x7fc96400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96400cc60 con 0x7fc97c0c8100 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc964011470 con 0x7fc97c0c8100 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964011860 con 0x7fc97c0c8100 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c0c8630 con 0x7fc97c0c8100 2024-07-17T07:15:48.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.939+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c0c8bf0 con 0x7fc97c0c8100 2024-07-17T07:15:48.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.941+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c06dca0 con 0x7fc97c0c8100 2024-07-17T07:15:48.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.941+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7fc964011a00 con 0x7fc97c0c8100 2024-07-17T07:15:48.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.942+0000 7fc972000640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 0x7fc958079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:48.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.942+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7fc964099560 con 0x7fc97c0c8100 2024-07-17T07:15:48.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.943+0000 7fc979e00640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 msgr2=0x7fc958079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:48.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.943+0000 7fc979e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 0x7fc958079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:48.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:48.947+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc97c14f320 con 0x7fc97c0c8100 2024-07-17T07:15:49.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.144+0000 7fc979e00640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 msgr2=0x7fc958079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:49.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.144+0000 7fc979e00640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 0x7fc958079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:49.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.213+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc97c0b5d70 con 0x7fc97c0c8100 2024-07-17T07:15:49.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.214+0000 7fc972000640 1 -- 172.21.15.44:0/951362909 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5228 (secure 0 0 0) 0x7fc964016030 con 0x7fc97c0c8100 2024-07-17T07:15:49.213 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:49.213 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"btime":"2024-07-17T07:15:24:361320+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:15:20.777186+0000","modified":"2024-07-17T07:15:23.867664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/3810295277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3810295277},{"type":"v1","addr":"172.21.15.44:6835","nonce":3810295277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619]},"id":8}]} 2024-07-17T07:15:49.213 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 70 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 msgr2=0x7fc958079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 0x7fc958079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 msgr2=0x7fc97c14ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e910 tx=0x7fc96400edd0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 shutdown_connections 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7fc958076d60 0x7fc958079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc97c0b1a20 0x7fc97c1e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc97c0c8100 0x7fc97c14ede0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 --2- 172.21.15.44:0/951362909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc960003aa0 0x7fc97c0c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 >> 172.21.15.44:0/951362909 conn(0x7fc97c0cfec0 msgr2=0x7fc97c0cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:49.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 shutdown_connections 2024-07-17T07:15:49.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.217+0000 7fc97b200640 1 -- 172.21.15.44:0/951362909 wait complete. 2024-07-17T07:15:49.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:49.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3721773008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 msgr2=0x7f3bbc06e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3721773008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc06e480 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0cf2f0 tx=0x7f3bac02aaa0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3721773008 shutdown_connections 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3721773008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 0x7f3bbc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3721773008 >> v2:172.21.15.175:3300/0 conn(0x7f3bbc0b5740 0x7f3bbc161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3721773008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc06e480 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3721773008 >> 172.21.15.44:0/3721773008 conn(0x7f3bbc0d00a0 msgr2=0x7f3bbc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3721773008 shutdown_connections 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.433+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3721773008 wait complete. 2024-07-17T07:15:49.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.435+0000 7f3bc2c00640 1 Processor -- start 2024-07-17T07:15:49.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.435+0000 7f3bc2c00640 1 -- start start 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 0x7f3bbc158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bbc0b5740 0x7f3bbc15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc1618f0 con 0x7f3bbc15d600 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc161770 con 0x7f3bbc0b5740 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc161a70 con 0x7f3bbc0b2ad0 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41490/0 (socket says 172.21.15.44:41490) 2024-07-17T07:15:49.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bc0c00640 1 -- 172.21.15.44:0/3392598997 learned_addr learned my addr 172.21.15.44:0/3392598997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bbbe00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 0x7f3bbc158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.436+0000 7f3bbb400640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bbc0b5740 0x7f3bbc15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 msgr2=0x7f3bbc158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 0x7f3bbc158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bbc0b5740 msgr2=0x7f3bbc15a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bbc0b5740 0x7f3bbc15a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 -- 172.21.15.44:0/3392598997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc15aa00 con 0x7f3bbc15d600 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bc0c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c970 tx=0x7f3bb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4007bb0 con 0x7f3bbc15d600 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bb4007e40 con 0x7f3bbc15d600 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.437+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4005280 con 0x7f3bbc15d600 2024-07-17T07:15:49.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.438+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc15ace0 con 0x7f3bbc15d600 2024-07-17T07:15:49.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.438+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc1f4520 con 0x7f3bbc15d600 2024-07-17T07:15:49.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.439+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bbc1f4030 con 0x7f3bbc15d600 2024-07-17T07:15:49.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.440+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7f3bb4005420 con 0x7f3bbc15d600 2024-07-17T07:15:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.441+0000 7f3bb3e00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 0x7f3b98079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.441+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f3bb4099050 con 0x7f3bbc15d600 2024-07-17T07:15:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.441+0000 7f3bbbe00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 msgr2=0x7f3b98079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.441+0000 7f3bbbe00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 0x7f3b98079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:49.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.447+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bb4014030 con 0x7f3bbc15d600 2024-07-17T07:15:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.642+0000 7f3bbbe00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 msgr2=0x7f3b98079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.642+0000 7f3bbbe00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 0x7f3b98079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:15:49.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.750+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bbc161020 con 0x7f3bbc15d600 2024-07-17T07:15:49.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.753+0000 7f3bb3e00640 1 -- 172.21.15.44:0/3392598997 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+18279 (secure 0 0 0) 0x7f3bb4061c80 con 0x7f3bbc15d600 2024-07-17T07:15:49.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:49.752 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":143,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:35.635135+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:15:18.669721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-07-17T07:15:19.770046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4151616996":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3602435312":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3869263054":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/526872756":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/4113173171":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6801/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6800/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/2312626755":"2024-07-18T07:15:35.635020+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:49.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 msgr2=0x7f3b98079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:49.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 0x7f3b98079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 msgr2=0x7f3bbc14efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c970 tx=0x7f3bb400ce30 comp rx=0 tx=0).stop 2024-07-17T07:15:49.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 shutdown_connections 2024-07-17T07:15:49.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f3b98076d60 0x7f3b98079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc15d600 0x7f3bbc14efb0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bbc0b5740 0x7f3bbc15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 --2- 172.21.15.44:0/3392598997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc0b2ad0 0x7f3bbc158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 >> 172.21.15.44:0/3392598997 conn(0x7f3bbc0d00a0 msgr2=0x7f3bbc0cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:49.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 shutdown_connections 2024-07-17T07:15:49.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.756+0000 7f3bc2c00640 1 -- 172.21.15.44:0/3392598997 wait complete. 2024-07-17T07:15:49.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T07:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 -- 172.21.15.44:0/71426473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80ce1d0 msgr2=0x7f0ff80ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 --2- 172.21.15.44:0/71426473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff80ce5b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80cdce0 tx=0x7f0fe802a6c0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 -- 172.21.15.44:0/71426473 shutdown_connections 2024-07-17T07:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 --2- 172.21.15.44:0/71426473 >> v2:172.21.15.175:3300/0 conn(0x7f0ff80b6790 0x7f0ff8161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 --2- 172.21.15.44:0/71426473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff80ce5b0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 -- 172.21.15.44:0/71426473 >> 172.21.15.44:0/71426473 conn(0x7f0ff80d3700 msgr2=0x7f0ff80d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 -- 172.21.15.44:0/71426473 shutdown_connections 2024-07-17T07:15:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.972+0000 7f0ffe600640 1 -- 172.21.15.44:0/71426473 wait complete. 2024-07-17T07:15:49.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.974+0000 7f0ffe600640 1 Processor -- start 2024-07-17T07:15:49.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.974+0000 7f0ffe600640 1 -- start start 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ff80b29e0 0x7f0ff815d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff81f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8162420 con 0x7f0ff80b6790 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81622a0 con 0x7f0ff80ce1d0 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ffe600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff8167910 con 0x7f0ff80b29e0 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41510/0 (socket says 172.21.15.44:41510) 2024-07-17T07:15:49.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 -- 172.21.15.44:0/3983974269 learned_addr learned my addr 172.21.15.44:0/3983974269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ff7400640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ff80b29e0 0x7f0ff815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0ff7e00640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff81f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 -- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ff80b29e0 msgr2=0x7f0ff815d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ff80b29e0 0x7f0ff815d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 -- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ff80ce1d0 msgr2=0x7f0ff81f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff81f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.976+0000 7f0fefe00640 1 -- 172.21.15.44:0/3983974269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff80c9ea0 con 0x7f0ff80b6790 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0fefe00640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c890 tx=0x7f0fe000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0004220 con 0x7f0ff80b6790 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fe00079b0 con 0x7f0ff80b6790 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff80c7d70 con 0x7f0ff80b6790 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0007de0 con 0x7f0ff80b6790 2024-07-17T07:15:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.977+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff814df40 con 0x7f0ff80b6790 2024-07-17T07:15:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.979+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff80ce5b0 con 0x7f0ff80b6790 2024-07-17T07:15:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.979+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 75) ==== 98988+0+0 (secure 0 0 0) 0x7f0fe0005020 con 0x7f0ff80b6790 2024-07-17T07:15:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.980+0000 7f0fef400640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 0x7f0fd4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.981+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6993+0+0 (secure 0 0 0) 0x7f0fe0099060 con 0x7f0ff80b6790 2024-07-17T07:15:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.981+0000 7f0ff7400640 1 -- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 msgr2=0x7f0fd4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1241312127 2024-07-17T07:15:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.981+0000 7f0ff7400640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 0x7f0fd4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:15:49.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:49.985+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0fe0014030 con 0x7f0ff80b6790 2024-07-17T07:15:50.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.051+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 76) ==== 98540+0+0 (secure 0 0 0) 0x7f0fe005d810 con 0x7f0ff80b6790 2024-07-17T07:15:50.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.052+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 msgr2=0x7f0fd4079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:15:50.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.052+0000 7f0fef400640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 0x7f0fd4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fd0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fd0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fd0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fcc800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fcc800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fcc800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fcc800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.119+0000 7f3fcc800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.120+0000 7f3fd0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.119 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:15:50.120+0000 7f3fd0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:15:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.244+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0ff8157a70 con 0x7f0ff80b6790 2024-07-17T07:15:50.244 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T07:15:50.245+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T07:15:50.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.249+0000 7f0fef400640 1 -- 172.21.15.44:0/3983974269 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) ==== 161+0+0 (secure 0 0 0) 0x7f0fe0061c80 con 0x7f0ff80b6790 2024-07-17T07:15:50.248 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T07:15:50.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 msgr2=0x7f0ff815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c890 tx=0x7f0fe000cd50 comp rx=0 tx=0).stop 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 shutdown_connections 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:6800/1241312127,v1:172.21.15.175:6801/1241312127] conn(0x7f0fd4076d60 0x7f0fd4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ff80ce1d0 0x7f0ff81f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ff80b6790 0x7f0ff815b0e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 --2- 172.21.15.44:0/3983974269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ff80b29e0 0x7f0ff815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.252+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 >> 172.21.15.44:0/3983974269 conn(0x7f0ff80d3700 msgr2=0x7f0ff80d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.253+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 shutdown_connections 2024-07-17T07:15:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.253+0000 7f0ffe600640 1 -- 172.21.15.44:0/3983974269 wait complete. 2024-07-17T07:15:50.265 INFO:tasks.ceph.mds.a.smithi044.stderr: -9571> 2024-07-17T07:15:23.250+0000 7f5ba2400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:15:50.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T07:15:50.319 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:15:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 -- 172.21.15.44:0/103005894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc0acab0 msgr2=0x7fcedc0b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 --2- 172.21.15.44:0/103005894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc0acab0 0x7fcedc0b3280 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fced400afd0 tx=0x7fced402b5d0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 -- 172.21.15.44:0/103005894 shutdown_connections 2024-07-17T07:15:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 --2- 172.21.15.44:0/103005894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc0acab0 0x7fcedc0b3280 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 -- 172.21.15.44:0/103005894 >> 172.21.15.44:0/103005894 conn(0x7fcedc0b0eb0 msgr2=0x7fcedc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 -- 172.21.15.44:0/103005894 shutdown_connections 2024-07-17T07:15:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.451+0000 7fcee3400640 1 -- 172.21.15.44:0/103005894 wait complete. 2024-07-17T07:15:50.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.453+0000 7fcee3400640 1 Processor -- start 2024-07-17T07:15:50.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.453+0000 7fcee3400640 1 -- start start 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 0x7fcedc09caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0acab0 0x7fcedc074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc14fd60 con 0x7fcedc15a5a0 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc06f8e0 con 0x7fcedc06eb90 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc14fee0 con 0x7fcedc0acab0 2024-07-17T07:15:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.454+0000 7fcee1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41560/0 (socket says 172.21.15.44:41560) 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 -- 172.21.15.44:0/2555989823 learned_addr learned my addr 172.21.15.44:0/2555989823 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0acab0 0x7fcedc074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee0a00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 0x7fcedc09caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 -- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0acab0 msgr2=0x7fcedc074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0acab0 0x7fcedc074040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 -- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 msgr2=0x7fcedc09caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 0x7fcedc09caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 -- 172.21.15.44:0/2555989823 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc1f4110 con 0x7fcedc15a5a0 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee0a00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 0x7fcedc09caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.455+0000 7fcee1400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fced400c020 tx=0x7fced40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.456+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4037040 con 0x7fcedc15a5a0 2024-07-17T07:15:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.456+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced4004720 con 0x7fcedc15a5a0 2024-07-17T07:15:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.456+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc1f4c70 con 0x7fcedc15a5a0 2024-07-17T07:15:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.456+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced400a330 con 0x7fcedc15a5a0 2024-07-17T07:15:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.456+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc1f46b0 con 0x7fcedc15a5a0 2024-07-17T07:15:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.457+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc0b3280 con 0x7fcedc15a5a0 2024-07-17T07:15:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.458+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 76) ==== 98540+0+0 (secure 0 0 0) 0x7fced4008c90 con 0x7fcedc15a5a0 2024-07-17T07:15:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.459+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 7466+0+0 (secure 0 0 0) 0x7fced40b8ba0 con 0x7fcedc15a5a0 2024-07-17T07:15:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.463+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fced4081cf0 con 0x7fcedc15a5a0 2024-07-17T07:15:50.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:50.717+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcedc0b4200 con 0x7fcedc15a5a0 2024-07-17T07:15:51.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.058+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 77) ==== 98667+0+0 (secure 0 0 0) 0x7fced407cf60 con 0x7fcedc15a5a0 2024-07-17T07:15:51.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.058+0000 7fced8c00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 0x7fcebc07f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:51.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.060+0000 7fcee0a00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 0x7fcebc07f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:51.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.066+0000 7fcee0a00640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 0x7fcebc07f530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fced0003eb0 tx=0x7fced0009040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.254+0000 7fced8c00640 1 -- 172.21.15.44:0/2555989823 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) ==== 104+0+0 (secure 0 0 0) 0x7fced40b3700 con 0x7fcedc15a5a0 2024-07-17T07:15:51.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 msgr2=0x7fcebc07f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 0x7fcebc07f530 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fced0003eb0 tx=0x7fced0009040 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 msgr2=0x7fcedc0ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fced400c020 tx=0x7fced40042f0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 shutdown_connections 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fcebc07d140 0x7fcebc07f530 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc15a5a0 0x7fcedc0ce2e0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.257+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0acab0 0x7fcedc074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.258+0000 7fcee3400640 1 --2- 172.21.15.44:0/2555989823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc06eb90 0x7fcedc09caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.258+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 >> 172.21.15.44:0/2555989823 conn(0x7fcedc0b0eb0 msgr2=0x7fcedc0ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:51.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.258+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 shutdown_connections 2024-07-17T07:15:51.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.258+0000 7fcee3400640 1 -- 172.21.15.44:0/2555989823 wait complete. 2024-07-17T07:15:51.278 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T07:15:51.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 -- 172.21.15.44:0/2692538613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 msgr2=0x7f86c40ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 --2- 172.21.15.44:0/2692538613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c40ae1a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005fd0 tx=0x7f86b802a4a0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 -- 172.21.15.44:0/2692538613 shutdown_connections 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 --2- 172.21.15.44:0/2692538613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c40ae1a0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 -- 172.21.15.44:0/2692538613 >> 172.21.15.44:0/2692538613 conn(0x7f86c40b0eb0 msgr2=0x7f86c40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 -- 172.21.15.44:0/2692538613 shutdown_connections 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.490+0000 7f86cb800640 1 -- 172.21.15.44:0/2692538613 wait complete. 2024-07-17T07:15:51.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.492+0000 7f86cb800640 1 Processor -- start 2024-07-17T07:15:51.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.492+0000 7f86cb800640 1 -- start start 2024-07-17T07:15:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86c406e690 0x7f86c40728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c415e1e0 0x7f86c41f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:51.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b2a40 con 0x7f86c4154560 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b28c0 con 0x7f86c415e1e0 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.493+0000 7f86cb800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f86c40b2bc0 con 0x7f86c406e690 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41568/0 (socket says 172.21.15.44:41568) 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 -- 172.21.15.44:0/1475798517 learned_addr learned my addr 172.21.15.44:0/1475798517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c8e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86c406e690 0x7f86c40728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c9800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c415e1e0 0x7f86c41f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 -- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86c406e690 msgr2=0x7f86c40728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86c406e690 0x7f86c40728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 -- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c415e1e0 msgr2=0x7f86c41f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c415e1e0 0x7f86c41f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:51.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.494+0000 7f86c3e00640 1 -- 172.21.15.44:0/1475798517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c4073100 con 0x7f86c4154560 2024-07-17T07:15:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86c3e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c410 tx=0x7f86b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017070 con 0x7f86c4154560 2024-07-17T07:15:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86b00035f0 con 0x7f86c4154560 2024-07-17T07:15:51.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0003a80 con 0x7f86c4154560 2024-07-17T07:15:51.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c40733e0 con 0x7f86c4154560 2024-07-17T07:15:51.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.495+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c40739a0 con 0x7f86c4154560 2024-07-17T07:15:51.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.496+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c40ad4e0 con 0x7f86c4154560 2024-07-17T07:15:51.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.498+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 77) ==== 98667+0+0 (secure 0 0 0) 0x7f86b001a070 con 0x7f86c4154560 2024-07-17T07:15:51.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.499+0000 7f86c0c00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 0x7f86a4078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:51.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.499+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 7466+0+0 (secure 0 0 0) 0x7f86b0097ad0 con 0x7f86c4154560 2024-07-17T07:15:51.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.500+0000 7f86c8e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 0x7f86a4078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:51.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.501+0000 7f86c8e00640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 0x7f86a4078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f86b8033000 tx=0x7f86b8004730 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:51.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.503+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f86c40ad4e0 con 0x7f86c4154560 2024-07-17T07:15:51.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:51.708+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86c40ae1a0 con 0x7f86c4154560 2024-07-17T07:15:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.269+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 78) ==== 98875+0+0 (secure 0 0 0) 0x7f86b005c790 con 0x7f86c4154560 2024-07-17T07:15:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.279+0000 7f86c0c00640 1 -- 172.21.15.44:0/1475798517 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v146) ==== 181+0+0 (secure 0 0 0) 0x7f86b0060c00 con 0x7f86c4154560 2024-07-17T07:15:52.278 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 msgr2=0x7f86a4078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 0x7f86a4078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f86b8033000 tx=0x7f86b8004730 comp rx=0 tx=0).stop 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 msgr2=0x7f86c41f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c410 tx=0x7f86b000c8d0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 shutdown_connections 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.283+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f86a4076a00 0x7f86a4078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86c415e1e0 0x7f86c41f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c4154560 0x7f86c41f7a30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 --2- 172.21.15.44:0/1475798517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86c406e690 0x7f86c40728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 >> 172.21.15.44:0/1475798517 conn(0x7f86c40b0eb0 msgr2=0x7f86c415ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:52.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 shutdown_connections 2024-07-17T07:15:52.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.284+0000 7f86cb800640 1 -- 172.21.15.44:0/1475798517 wait complete. 2024-07-17T07:15:52.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T07:15:52.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 -- 172.21.15.44:0/3023533338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8003a00 msgr2=0x7f20b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:52.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 --2- 172.21.15.44:0/3023533338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20b8003a00 0x7f20b8003e20 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f20d40d0400 tx=0x7f20b8005830 comp rx=0 tx=0).stop 2024-07-17T07:15:52.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20d9e00640 1 -- 172.21.15.44:0/3023533338 reap_dead start 2024-07-17T07:15:52.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 -- 172.21.15.44:0/3023533338 shutdown_connections 2024-07-17T07:15:52.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 -- 172.21.15.44:0/3023533338 >> 172.21.15.44:0/3023533338 conn(0x7f20d4073a60 msgr2=0x7f20d4073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:52.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 -- 172.21.15.44:0/3023533338 shutdown_connections 2024-07-17T07:15:52.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.520+0000 7f20da800640 1 -- 172.21.15.44:0/3023533338 wait complete. 2024-07-17T07:15:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.522+0000 7f20da800640 1 Processor -- start 2024-07-17T07:15:52.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.522+0000 7f20da800640 1 -- start start 2024-07-17T07:15:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b8003a00 0x7f20d41f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d40c9560 0x7f20d40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40ce890 con 0x7f20d4157a30 2024-07-17T07:15:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40ce710 con 0x7f20b8003a00 2024-07-17T07:15:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20da800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40cea10 con 0x7f20d40c9560 2024-07-17T07:15:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20d8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20d8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41600/0 (socket says 172.21.15.44:41600) 2024-07-17T07:15:52.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.523+0000 7f20d8a00640 1 -- 172.21.15.44:0/1536000054 learned_addr learned my addr 172.21.15.44:0/1536000054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 -- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d40c9560 msgr2=0x7f20d40b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d40c9560 0x7f20d40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 -- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b8003a00 msgr2=0x7f20d41f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d9400640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b8003a00 0x7f20d41f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b8003a00 0x7f20d41f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 -- 172.21.15.44:0/1536000054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d41f4fc0 con 0x7f20d4157a30 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d8a00640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d700 tx=0x7f20c000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:52.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0018070 con 0x7f20d4157a30 2024-07-17T07:15:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c00050d0 con 0x7f20d4157a30 2024-07-17T07:15:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.524+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c000b6c0 con 0x7f20d4157a30 2024-07-17T07:15:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.525+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d40c9b80 con 0x7f20d4157a30 2024-07-17T07:15:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.525+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d40c7500 con 0x7f20d4157a30 2024-07-17T07:15:52.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.526+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d40b8670 con 0x7f20d4157a30 2024-07-17T07:15:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.527+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 78) ==== 98875+0+0 (secure 0 0 0) 0x7f20c001b070 con 0x7f20d4157a30 2024-07-17T07:15:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.528+0000 7f20d1600640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 0x7f20b4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.528+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 7028+0+0 (secure 0 0 0) 0x7f20c0097e70 con 0x7f20d4157a30 2024-07-17T07:15:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.529+0000 7f20d9400640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 0x7f20b4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:52.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.530+0000 7f20d9400640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 0x7f20b4079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d40d0400 tx=0x7f20c4004a40 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:52.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.532+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f20c00606d0 con 0x7f20d4157a30 2024-07-17T07:15:52.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:52.755+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20d4069460 con 0x7f20d4157a30 2024-07-17T07:15:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.282+0000 7f20d1600640 1 -- 172.21.15.44:0/1536000054 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v147) ==== 169+0+0 (secure 0 0 0) 0x7f20c0064c20 con 0x7f20d4157a30 2024-07-17T07:15:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T07:15:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.287+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 msgr2=0x7f20b4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.287+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 0x7f20b4079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d40d0400 tx=0x7f20c4004a40 comp rx=0 tx=0).stop 2024-07-17T07:15:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.287+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 msgr2=0x7f20d40c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.287+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d700 tx=0x7f20c000dbc0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 shutdown_connections 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f20b4076c40 0x7f20b4079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d40c9560 0x7f20d40b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d4157a30 0x7f20d40c9020 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 --2- 172.21.15.44:0/1536000054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20b8003a00 0x7f20d41f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.288+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 >> 172.21.15.44:0/1536000054 conn(0x7f20d4073a60 msgr2=0x7f20d40b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.289+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 shutdown_connections 2024-07-17T07:15:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.289+0000 7f20da800640 1 -- 172.21.15.44:0/1536000054 wait complete. 2024-07-17T07:15:53.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T07:15:53.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 -- 172.21.15.44:0/1969789245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0b1af0 msgr2=0x7fe6cc15ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1969789245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0b1af0 0x7fe6cc15ac10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005fd0 tx=0x7fe6c402aa00 comp rx=0 tx=0).stop 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 -- 172.21.15.44:0/1969789245 shutdown_connections 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1969789245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0b1af0 0x7fe6cc15ac10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1969789245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6cc0ac8a0 0x7fe6cc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 -- 172.21.15.44:0/1969789245 >> 172.21.15.44:0/1969789245 conn(0x7fe6cc0b0eb0 msgr2=0x7fe6cc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 -- 172.21.15.44:0/1969789245 shutdown_connections 2024-07-17T07:15:53.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.516+0000 7fe6d2400640 1 -- 172.21.15.44:0/1969789245 wait complete. 2024-07-17T07:15:53.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.518+0000 7fe6d2400640 1 Processor -- start 2024-07-17T07:15:53.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.518+0000 7fe6d2400640 1 -- start start 2024-07-17T07:15:53.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6cc0b1af0 0x7fe6cc158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6cc15a280 0x7fe6cc158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc15b150 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc06f1d0 con 0x7fe6cc0b1af0 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6d2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc15b2d0 con 0x7fe6cc15a280 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6cb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41618/0 (socket says 172.21.15.44:41618) 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.519+0000 7fe6cb400640 1 -- 172.21.15.44:0/1793790920 learned_addr learned my addr 172.21.15.44:0/1793790920 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 -- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6cc15a280 msgr2=0x7fe6cc158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6cc15a280 0x7fe6cc158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 -- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6cc0b1af0 msgr2=0x7fe6cc158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6cc0b1af0 0x7fe6cc158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 -- 172.21.15.44:0/1793790920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc1574b0 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6cb400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00b410 tx=0x7fe6bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc015020 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6bc011920 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1f1e90 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.520+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc011d70 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.521+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1f1b80 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.522+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc0b21b0 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.523+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 78) ==== 98875+0+0 (secure 0 0 0) 0x7fe6bc018020 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.524+0000 7fe6c3400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 0x7fe6a8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.524+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7fe6bc098650 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.525+0000 7fe6c3e00640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 0x7fe6a8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:53.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.526+0000 7fe6c3e00640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 0x7fe6a8079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b05d0 tx=0x7fe6b4005bf0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:53.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.529+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe6bc01f020 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.741+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fe6cc0ac3e0 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.750+0000 7fe6c3400640 1 -- 172.21.15.44:0/1793790920 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v55) ==== 71+0+0 (secure 0 0 0) 0x7fe6bc061830 con 0x7fe6cc0ac8a0 2024-07-17T07:15:53.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 msgr2=0x7fe6a8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 0x7fe6a8079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b05d0 tx=0x7fe6b4005bf0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 msgr2=0x7fe6cc151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00b410 tx=0x7fe6bc00b8d0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 shutdown_connections 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe6a8076c40 0x7fe6a8079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6cc15a280 0x7fe6cc158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6cc0b1af0 0x7fe6cc158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 --2- 172.21.15.44:0/1793790920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6cc0ac8a0 0x7fe6cc151980 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.753+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 >> 172.21.15.44:0/1793790920 conn(0x7fe6cc0b0eb0 msgr2=0x7fe6cc0cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.754+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 shutdown_connections 2024-07-17T07:15:53.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.754+0000 7fe6d2400640 1 -- 172.21.15.44:0/1793790920 wait complete. 2024-07-17T07:15:53.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2024-07-17T07:15:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 -- 172.21.15.44:0/272877687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 msgr2=0x7f19f00ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 --2- 172.21.15.44:0/272877687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00ad790 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f19f00b1f40 tx=0x7f19e002a560 comp rx=0 tx=0).stop 2024-07-17T07:15:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 -- 172.21.15.44:0/272877687 shutdown_connections 2024-07-17T07:15:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 --2- 172.21.15.44:0/272877687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f00adcd0 0x7f19f00ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 --2- 172.21.15.44:0/272877687 >> v2:172.21.15.175:3300/0 conn(0x7f19f00ac760 0x7f19f015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 --2- 172.21.15.44:0/272877687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00ad790 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 -- 172.21.15.44:0/272877687 >> 172.21.15.44:0/272877687 conn(0x7f19f00b0fb0 msgr2=0x7f19f00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:53.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 -- 172.21.15.44:0/272877687 shutdown_connections 2024-07-17T07:15:53.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.983+0000 7f19f7000640 1 -- 172.21.15.44:0/272877687 wait complete. 2024-07-17T07:15:53.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.985+0000 7f19f7000640 1 Processor -- start 2024-07-17T07:15:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.985+0000 7f19f7000640 1 -- start start 2024-07-17T07:15:53.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f00ac760 0x7f19f009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f00adcd0 0x7f19f0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015f760 con 0x7f19f01522c0 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015f5e0 con 0x7f19f00ac760 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f19f015f8e0 con 0x7f19f00adcd0 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41648/0 (socket says 172.21.15.44:41648) 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.986+0000 7f19f5000640 1 -- 172.21.15.44:0/2253606300 learned_addr learned my addr 172.21.15.44:0/2253606300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19efe00640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f00ac760 0x7f19f009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:53.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 -- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f00adcd0 msgr2=0x7f19f0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f00adcd0 0x7f19f0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 -- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f00ac760 msgr2=0x7f19f009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f00ac760 0x7f19f009ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 -- 172.21.15.44:0/2253606300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f009cfd0 con 0x7f19f01522c0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f5000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f19e800ca30 tx=0x7f19e800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800adc0 con 0x7f19f01522c0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19e800f450 con 0x7f19f01522c0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f0153c20 con 0x7f19f01522c0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.987+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800f8c0 con 0x7f19f01522c0 2024-07-17T07:15:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.988+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01f7770 con 0x7f19f01522c0 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.989+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00ac220 con 0x7f19f01522c0 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.990+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 78) ==== 98875+0+0 (secure 0 0 0) 0x7f19e800fa60 con 0x7f19f01522c0 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.992+0000 7f19e7e00640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 0x7f19d0079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.992+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7f19e8099280 con 0x7f19f01522c0 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.992+0000 7f19efe00640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 0x7f19d0079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.994+0000 7f19efe00640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 0x7f19d0079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19e0033000 tx=0x7f19e0004730 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:53.996+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f19f009cfd0 con 0x7f19f01522c0 2024-07-17T07:15:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:54.286+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) -- 0x7f19f00d0070 con 0x7f19f01522c0 2024-07-17T07:15:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:54.765+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f19e805dd50 con 0x7f19f01522c0 2024-07-17T07:15:55.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.289+0000 7f19e7e00640 1 -- 172.21.15.44:0/2253606300 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v812) ==== 160+0+0 (secure 0 0 0) 0x7f19e8014030 con 0x7f19f01522c0 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.292+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 msgr2=0x7f19d0079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.292+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 0x7f19d0079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19e0033000 tx=0x7f19e0004730 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 msgr2=0x7f19f00acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f19e800ca30 tx=0x7f19e800cef0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 shutdown_connections 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f19d0076c40 0x7f19d0079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19f01522c0 0x7f19f00acd50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f00adcd0 0x7f19f0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 --2- 172.21.15.44:0/2253606300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f00ac760 0x7f19f009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 >> 172.21.15.44:0/2253606300 conn(0x7f19f00b0fb0 msgr2=0x7f19f006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 shutdown_connections 2024-07-17T07:15:55.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.293+0000 7f19f7000640 1 -- 172.21.15.44:0/2253606300 wait complete. 2024-07-17T07:15:55.314 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2024-07-17T07:15:55.315 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2024-07-17T07:15:55.315 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T07:15:55.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 -- 172.21.15.44:0/3170570374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc0b6390 msgr2=0x7fa7bc0b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/3170570374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc0b6390 0x7fa7bc0b67c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0cfe90 tx=0x7fa7b402a8b0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 -- 172.21.15.44:0/3170570374 shutdown_connections 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/3170570374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc0b6390 0x7fa7bc0b67c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/3170570374 >> v2:172.21.15.175:3300/0 conn(0x7fa7bc15d530 0x7fa7bc0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.532+0000 7fa7c2a00640 1 -- 172.21.15.44:0/3170570374 >> 172.21.15.44:0/3170570374 conn(0x7fa7bc0cfec0 msgr2=0x7fa7bc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.533+0000 7fa7c2a00640 1 -- 172.21.15.44:0/3170570374 shutdown_connections 2024-07-17T07:15:55.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.533+0000 7fa7c2a00640 1 -- 172.21.15.44:0/3170570374 wait complete. 2024-07-17T07:15:55.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.534+0000 7fa7c2a00640 1 Processor -- start 2024-07-17T07:15:55.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.534+0000 7fa7c2a00640 1 -- start start 2024-07-17T07:15:55.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.535+0000 7fa7c2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7bc0b6390 0x7fa7bc151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:55.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.535+0000 7fa7c2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7bc0cdca0 0x7fa7bc0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:55.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.535+0000 7fa7c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:55.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.535+0000 7fa7c2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc15da90 con 0x7fa7bc15d530 2024-07-17T07:15:55.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.535+0000 7fa7c2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc15d910 con 0x7fa7bc0cdca0 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7bc0b4ec0 con 0x7fa7bc0b6390 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41668/0 (socket says 172.21.15.44:41668) 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 -- 172.21.15.44:0/1453070025 learned_addr learned my addr 172.21.15.44:0/1453070025 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c0c00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7bc0cdca0 0x7fa7bc0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c1600640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7bc0b6390 0x7fa7bc151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 -- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7bc0b6390 msgr2=0x7fa7bc151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7bc0b6390 0x7fa7bc151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 -- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7bc0cdca0 msgr2=0x7fa7bc0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7bc0cdca0 0x7fa7bc0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.536+0000 7fa7c2000640 1 -- 172.21.15.44:0/1453070025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc1f82c0 con 0x7fa7bc15d530 2024-07-17T07:15:55.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7c2000640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009ca0 tx=0x7fa7b4031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4040070 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7b4004040 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004470 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc1f7740 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.537+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1f7d00 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.538+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7bc0cf760 con 0x7fa7bc15d530 2024-07-17T07:15:55.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.540+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fa7b4039040 con 0x7fa7bc15d530 2024-07-17T07:15:55.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.541+0000 7fa7b9600640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 0x7fa79c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:55.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.541+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7fa7b40b9ec0 con 0x7fa7bc15d530 2024-07-17T07:15:55.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.541+0000 7fa7c1600640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 0x7fa79c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:55.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.542+0000 7fa7c1600640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 0x7fa79c079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005fd0 tx=0x7fa7b0005850 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:55.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.545+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa7bc0cf760 con 0x7fa7bc15d530 2024-07-17T07:15:55.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.755+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7bc0b67c0 con 0x7fa7bc15d530 2024-07-17T07:15:55.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.757+0000 7fa7b9600640 1 -- 172.21.15.44:0/1453070025 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+15200 (secure 0 0 0) 0x7fa7b4082dd0 con 0x7fa7bc15d530 2024-07-17T07:15:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:55.756 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":147,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:15:53.279393+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/3810295277":"2024-07-18T07:15:50.245727+0000","172.21.15.44:6833/3810295277":"2024-07-18T07:15:50.245727+0000","172.21.15.175:0/3264814062":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/42565295":"2024-07-18T07:15:50.034062+0000","172.21.15.175:6801/1241312127":"2024-07-18T07:15:50.034062+0000","172.21.15.175:6800/1241312127":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/657120562":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/647411275":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/1700864794":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/3578562168":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/4151616996":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3602435312":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/3869263054":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/526872756":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/2504881623":"2024-07-18T07:15:50.034062+0000","172.21.15.175:0/4113173171":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6801/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:6800/2606536318":"2024-07-18T07:15:35.635020+0000","172.21.15.175:0/2312626755":"2024-07-18T07:15:35.635020+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:15:55.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 msgr2=0x7fa79c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 0x7fa79c079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005fd0 tx=0x7fa7b0005850 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 msgr2=0x7fa7bc0b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009ca0 tx=0x7fa7b4031720 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 shutdown_connections 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa79c076cd0 0x7fa79c079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc15d530 0x7fa7bc0b7420 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7bc0cdca0 0x7fa7bc0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 --2- 172.21.15.44:0/1453070025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7bc0b6390 0x7fa7bc151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 >> 172.21.15.44:0/1453070025 conn(0x7fa7bc0cfec0 msgr2=0x7fa7bc0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 shutdown_connections 2024-07-17T07:15:55.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.760+0000 7fa7c2a00640 1 -- 172.21.15.44:0/1453070025 wait complete. 2024-07-17T07:15:55.780 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T07:15:55.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 -- 172.21.15.44:0/4024357236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00ce1d0 msgr2=0x7f01c00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 --2- 172.21.15.44:0/4024357236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00ce1d0 0x7f01c00ce5b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f01c00cdce0 tx=0x7f01b002a5f0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 -- 172.21.15.44:0/4024357236 shutdown_connections 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 --2- 172.21.15.44:0/4024357236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00b29e0 0x7f01c0161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 --2- 172.21.15.44:0/4024357236 >> v2:172.21.15.175:3300/0 conn(0x7f01c00b6790 0x7f01c0161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 --2- 172.21.15.44:0/4024357236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00ce1d0 0x7f01c00ce5b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:55.999+0000 7f01c6c00640 1 -- 172.21.15.44:0/4024357236 >> 172.21.15.44:0/4024357236 conn(0x7f01c00d3700 msgr2=0x7f01c00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:55.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.000+0000 7f01c6c00640 1 -- 172.21.15.44:0/4024357236 shutdown_connections 2024-07-17T07:15:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.000+0000 7f01c6c00640 1 -- 172.21.15.44:0/4024357236 wait complete. 2024-07-17T07:15:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.002+0000 7f01c6c00640 1 Processor -- start 2024-07-17T07:15:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.002+0000 7f01c6c00640 1 -- start start 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 0x7f01c0159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00ce1d0 0x7f01c015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0162250 con 0x7f01c00b29e0 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01620d0 con 0x7f01c00b6790 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01c6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01c01623d0 con 0x7f01c00ce1d0 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41698/0 (socket says 172.21.15.44:41698) 2024-07-17T07:15:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.003+0000 7f01bfe00640 1 -- 172.21.15.44:0/985946944 learned_addr learned my addr 172.21.15.44:0/985946944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 -- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00ce1d0 msgr2=0x7f01c015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01c4c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00ce1d0 0x7f01c015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00ce1d0 0x7f01c015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01b7400640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 0x7f01c0159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 -- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 msgr2=0x7f01c0159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 0x7f01c0159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 -- 172.21.15.44:0/985946944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00ca900 con 0x7f01c00b29e0 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01b7400640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 0x7f01c0159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bfe00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f01b0034040 tx=0x7f01b0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.004+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b003e030 con 0x7f01c00b29e0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.005+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01b0003e00 con 0x7f01c00b29e0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.005+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c00cabe0 con 0x7f01c00b29e0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.005+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0035040 con 0x7f01c00b29e0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.005+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0152bd0 con 0x7f01c00b29e0 2024-07-17T07:15:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.006+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c00d0fa0 con 0x7f01c00b29e0 2024-07-17T07:15:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.007+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f01b0041030 con 0x7f01c00b29e0 2024-07-17T07:15:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.008+0000 7f01bcc00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 0x7f019c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.008+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7f01b00b8fe0 con 0x7f01c00b29e0 2024-07-17T07:15:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.009+0000 7f01b7400640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 0x7f019c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.010+0000 7f01b7400640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 0x7f019c079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01c0152410 tx=0x7f01a8005d90 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.012+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01c00d0fa0 con 0x7f01c00b29e0 2024-07-17T07:15:56.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.224+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01c00cfaa0 con 0x7f01c00b29e0 2024-07-17T07:15:56.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.225+0000 7f01bcc00640 1 -- 172.21.15.44:0/985946944 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) ==== 93+0+10 (secure 0 0 0) 0x7f01c00cfaa0 con 0x7f01c00b29e0 2024-07-17T07:15:56.224 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 msgr2=0x7f019c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 0x7f019c079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01c0152410 tx=0x7f01a8005d90 comp rx=0 tx=0).stop 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 msgr2=0x7f01c0158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f01b0034040 tx=0x7f01b0003910 comp rx=0 tx=0).stop 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 shutdown_connections 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.228+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f019c076cd0 0x7f019c079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c00ce1d0 0x7f01c015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c00b6790 0x7f01c0159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 --2- 172.21.15.44:0/985946944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c00b29e0 0x7f01c0158dd0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 >> 172.21.15.44:0/985946944 conn(0x7f01c00d3700 msgr2=0x7f01c00b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:56.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 shutdown_connections 2024-07-17T07:15:56.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.229+0000 7f01c6c00640 1 -- 172.21.15.44:0/985946944 wait complete. 2024-07-17T07:15:56.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2024-07-17T07:15:56.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 -- 172.21.15.44:0/1697095357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 msgr2=0x7facd40cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 --2- 172.21.15.44:0/1697095357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40cd690 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7facd4070bf0 tx=0x7faccc02aa30 comp rx=0 tx=0).stop 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 -- 172.21.15.44:0/1697095357 shutdown_connections 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 --2- 172.21.15.44:0/1697095357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40cd690 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 --2- 172.21.15.44:0/1697095357 >> v2:172.21.15.175:3300/0 conn(0x7facd41522c0 0x7facd40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 -- 172.21.15.44:0/1697095357 >> 172.21.15.44:0/1697095357 conn(0x7facd40b0fb0 msgr2=0x7facd40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 -- 172.21.15.44:0/1697095357 shutdown_connections 2024-07-17T07:15:56.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.466+0000 7facdc200640 1 -- 172.21.15.44:0/1697095357 wait complete. 2024-07-17T07:15:56.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.468+0000 7facdc200640 1 Processor -- start 2024-07-17T07:15:56.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.469+0000 7facdc200640 1 -- start start 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd40adcd0 0x7facd40ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 0x7facd41f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facd40cdf60 con 0x7facd40ac930 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facd40cdde0 con 0x7facd41522c0 2024-07-17T07:15:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facdc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facd40ce0e0 con 0x7facd40adcd0 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facd9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facd9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41716/0 (socket says 172.21.15.44:41716) 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.470+0000 7facd9800640 1 -- 172.21.15.44:0/3720392338 learned_addr learned my addr 172.21.15.44:0/3720392338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facda200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 0x7facd41f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 -- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd40adcd0 msgr2=0x7facd40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd8e00640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd40adcd0 0x7facd40ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd40adcd0 0x7facd40ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 -- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 msgr2=0x7facd41f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 0x7facd41f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 -- 172.21.15.44:0/3720392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd40b33d0 con 0x7facd40ac930 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facda200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 0x7facd41f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd9800640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7facc800d6d0 tx=0x7facc800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8017070 con 0x7facd40ac930 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc8005330 con 0x7facd40ac930 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8009820 con 0x7facd40ac930 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.471+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd40cb1b0 con 0x7facd40ac930 2024-07-17T07:15:56.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.472+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd40b2690 con 0x7facd40ac930 2024-07-17T07:15:56.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.473+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd40cd690 con 0x7facd40ac930 2024-07-17T07:15:56.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.474+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7facc8009bd0 con 0x7facd40ac930 2024-07-17T07:15:56.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.475+0000 7facd1600640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 0x7facb40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:56.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.475+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7facc8098b70 con 0x7facd40ac930 2024-07-17T07:15:56.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.476+0000 7facd8e00640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 0x7facb40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:56.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.477+0000 7facd8e00640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 0x7facb40791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7facd40ca000 tx=0x7facc0009040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:56.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.480+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7facc8061a80 con 0x7facd40ac930 2024-07-17T07:15:56.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:56.770+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) -- 0x7facd40caf90 con 0x7facd40ac930 2024-07-17T07:15:57.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.304+0000 7facd1600640 1 -- 172.21.15.44:0/3720392338 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v814) ==== 181+0+0 (secure 0 0 0) 0x7facc8061380 con 0x7facd40ac930 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.307+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 msgr2=0x7facb40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.307+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 0x7facb40791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7facd40ca000 tx=0x7facc0009040 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.307+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 msgr2=0x7facd40b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.307+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7facc800d6d0 tx=0x7facc800db90 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 shutdown_connections 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7facb4076d00 0x7facb40791c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facd41522c0 0x7facd41f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facd40adcd0 0x7facd40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 --2- 172.21.15.44:0/3720392338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd40ac930 0x7facd40b2e90 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 >> 172.21.15.44:0/3720392338 conn(0x7facd40b0fb0 msgr2=0x7facd40acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:57.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 shutdown_connections 2024-07-17T07:15:57.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.308+0000 7facdc200640 1 -- 172.21.15.44:0/3720392338 wait complete. 2024-07-17T07:15:57.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T07:15:57.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 -- 172.21.15.44:0/1305602024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe600b31d0 msgr2=0x7fbe60071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 --2- 172.21.15.44:0/1305602024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe600b31d0 0x7fbe60071a30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fbe600b4890 tx=0x7fbe5002a900 comp rx=0 tx=0).stop 2024-07-17T07:15:57.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 -- 172.21.15.44:0/1305602024 shutdown_connections 2024-07-17T07:15:57.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 --2- 172.21.15.44:0/1305602024 >> v2:172.21.15.175:3300/0 conn(0x7fbe6015d000 0x7fbe6015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 --2- 172.21.15.44:0/1305602024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe600b31d0 0x7fbe60071a30 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 -- 172.21.15.44:0/1305602024 >> 172.21.15.44:0/1305602024 conn(0x7fbe6006bf40 msgr2=0x7fbe6006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 -- 172.21.15.44:0/1305602024 shutdown_connections 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.551+0000 7fbe65a00640 1 -- 172.21.15.44:0/1305602024 wait complete. 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.553+0000 7fbe65a00640 1 Processor -- start 2024-07-17T07:15:57.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.553+0000 7fbe65a00640 1 -- start start 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe6006f750 0x7fbe601f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 0x7fbe60156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600ce5e0 con 0x7fbe6015d000 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600ce460 con 0x7fbe6006f750 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe65a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600ce760 con 0x7fbe600b31d0 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe5f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41736/0 (socket says 172.21.15.44:41736) 2024-07-17T07:15:57.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.554+0000 7fbe5f400640 1 -- 172.21.15.44:0/434993440 learned_addr learned my addr 172.21.15.44:0/434993440 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5ea00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe6006f750 0x7fbe601f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5e000640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 0x7fbe60156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 -- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 msgr2=0x7fbe60156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 0x7fbe60156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 -- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe6006f750 msgr2=0x7fbe601f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe6006f750 0x7fbe601f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 -- 172.21.15.44:0/434993440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe601570b0 con 0x7fbe6015d000 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5e000640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 0x7fbe60156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:57.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe5f400640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c920 tx=0x7fbe5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007bb0 con 0x7fbe6015d000 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe58007e40 con 0x7fbe6015d000 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58005280 con 0x7fbe6015d000 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.555+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe601f7f90 con 0x7fbe6015d000 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.556+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601f41e0 con 0x7fbe6015d000 2024-07-17T07:15:57.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.557+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe60071a30 con 0x7fbe6015d000 2024-07-17T07:15:57.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.558+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fbe58005420 con 0x7fbe6015d000 2024-07-17T07:15:57.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.559+0000 7fbe56a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 0x7fbe40079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.559+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7fbe58098c70 con 0x7fbe6015d000 2024-07-17T07:15:57.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.560+0000 7fbe5ea00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 0x7fbe40079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.561+0000 7fbe5ea00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 0x7fbe40079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe50034000 tx=0x7fbe50004ab0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:57.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.564+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbe58014030 con 0x7fbe6015d000 2024-07-17T07:15:57.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.768+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbe600b3850 con 0x7fbe6015d000 2024-07-17T07:15:57.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.769+0000 7fbe56a00640 1 -- 172.21.15.44:0/434993440 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v55) ==== 59+0+1998 (secure 0 0 0) 0x7fbe58061b80 con 0x7fbe6015d000 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T07:15:57.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T07:15:57.769 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T07:15:57.770 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T07:15:57.770 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T07:15:57.770 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T07:15:57.770 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T07:15:57.770 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.771+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 msgr2=0x7fbe40079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.771+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 0x7fbe40079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe50034000 tx=0x7fbe50004ab0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.771+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 msgr2=0x7fbe601f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c920 tx=0x7fbe5800cde0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 shutdown_connections 2024-07-17T07:15:57.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fbe40076cd0 0x7fbe40079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe6015d000 0x7fbe601f7510 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe600b31d0 0x7fbe60156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 --2- 172.21.15.44:0/434993440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe6006f750 0x7fbe601f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 >> 172.21.15.44:0/434993440 conn(0x7fbe6006bf40 msgr2=0x7fbe6014fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 shutdown_connections 2024-07-17T07:15:57.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.772+0000 7fbe65a00640 1 -- 172.21.15.44:0/434993440 wait complete. 2024-07-17T07:15:57.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 -- 172.21.15.44:0/581004352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 msgr2=0x7f0a6006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 --2- 172.21.15.44:0/581004352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a6006f1b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0a58005fd0 tx=0x7f0a5802a9f0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 -- 172.21.15.44:0/581004352 shutdown_connections 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 --2- 172.21.15.44:0/581004352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a6006f1b0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 --2- 172.21.15.44:0/581004352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 0x7f0a6006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.991+0000 7f0a66600640 1 -- 172.21.15.44:0/581004352 >> 172.21.15.44:0/581004352 conn(0x7f0a600cf0d0 msgr2=0x7f0a600cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.992+0000 7f0a66600640 1 -- 172.21.15.44:0/581004352 shutdown_connections 2024-07-17T07:15:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.992+0000 7f0a66600640 1 -- 172.21.15.44:0/581004352 wait complete. 2024-07-17T07:15:57.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.993+0000 7f0a66600640 1 Processor -- start 2024-07-17T07:15:57.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.993+0000 7f0a66600640 1 -- start start 2024-07-17T07:15:57.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 0x7f0a60154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a60151f20 0x7f0a60149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600cb6b0 con 0x7f0a60152300 2024-07-17T07:15:57.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600cb530 con 0x7f0a60151f20 2024-07-17T07:15:57.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a66600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a600cb830 con 0x7f0a600cd0b0 2024-07-17T07:15:57.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.994+0000 7f0a65c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41762/0 (socket says 172.21.15.44:41762) 2024-07-17T07:15:57.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 -- 172.21.15.44:0/3794189808 learned_addr learned my addr 172.21.15.44:0/3794189808 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a57e00640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a60151f20 0x7f0a60149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65200640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 0x7f0a60154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 -- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 msgr2=0x7f0a60154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 0x7f0a60154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 -- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a60151f20 msgr2=0x7f0a60149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a60151f20 0x7f0a60149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:57.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 -- 172.21.15.44:0/3794189808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a601f1df0 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.995+0000 7f0a65c00640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006010 tx=0x7f0a58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.996+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58046070 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.996+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a58004640 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.996+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58033070 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.996+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a601f2130 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.996+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a601f2970 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.997+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a601f2310 con 0x7f0a60152300 2024-07-17T07:15:57.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.998+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f0a58049070 con 0x7f0a60152300 2024-07-17T07:15:57.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.999+0000 7f0a5d600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 0x7f0a3c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:57.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:57.999+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7f0a580b9aa0 con 0x7f0a60152300 2024-07-17T07:15:57.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.000+0000 7f0a65200640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 0x7f0a3c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:57.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.001+0000 7f0a65200640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 0x7f0a3c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a50003d90 tx=0x7f0a50009040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.003+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0a601f2130 con 0x7f0a60152300 2024-07-17T07:15:58.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.213+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0a6015c140 con 0x7f0a60152300 2024-07-17T07:15:58.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.218+0000 7f0a5d600640 1 -- 172.21.15.44:0/3794189808 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v56)=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7f0a5803e070 con 0x7f0a60152300 2024-07-17T07:15:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 msgr2=0x7f0a3c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 0x7f0a3c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a50003d90 tx=0x7f0a50009040 comp rx=0 tx=0).stop 2024-07-17T07:15:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 msgr2=0x7f0a60149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006010 tx=0x7f0a58003f10 comp rx=0 tx=0).stop 2024-07-17T07:15:58.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 shutdown_connections 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0a3c076cd0 0x7f0a3c079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a60152300 0x7f0a60149f50 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a60151f20 0x7f0a60149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 --2- 172.21.15.44:0/3794189808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a600cd0b0 0x7f0a60154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 >> 172.21.15.44:0/3794189808 conn(0x7f0a600cf0d0 msgr2=0x7f0a6015c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 shutdown_connections 2024-07-17T07:15:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.221+0000 7f0a66600640 1 -- 172.21.15.44:0/3794189808 wait complete. 2024-07-17T07:15:58.239 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T07:15:58.239 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:58.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T07:15:58.473 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T07:15:58.473 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:58.473 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:58.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:58.535 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:58.535 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:58.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:58.588 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:15:58.588 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:58.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T07:15:58.648 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T07:15:58.648 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:15:58.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T07:15:58.701 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T07:15:58.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:15:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 -- 172.21.15.44:0/4094233602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f1c004e60 msgr2=0x7f0f1c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/4094233602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f1c004e60 0x7f0f1c00b9c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0f300016c0 tx=0x7f0f1c005830 comp rx=0 tx=0).stop 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f37200640 1 -- 172.21.15.44:0/4094233602 reap_dead start 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 -- 172.21.15.44:0/4094233602 shutdown_connections 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 -- 172.21.15.44:0/4094233602 >> 172.21.15.44:0/4094233602 conn(0x7f0f380b0e80 msgr2=0x7f0f380b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 -- 172.21.15.44:0/4094233602 shutdown_connections 2024-07-17T07:15:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.924+0000 7f0f3ca00640 1 -- 172.21.15.44:0/4094233602 wait complete. 2024-07-17T07:15:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.926+0000 7f0f3ca00640 1 Processor -- start 2024-07-17T07:15:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.926+0000 7f0f3ca00640 1 -- start start 2024-07-17T07:15:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 0x7f0f38073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f3815b4d0 0x7f0f3815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380acc70 con 0x7f0f3814e980 2024-07-17T07:15:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3809ce20 con 0x7f0f1c004e60 2024-07-17T07:15:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380acdf0 con 0x7f0f3815b4d0 2024-07-17T07:15:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.927+0000 7f0f37200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57340/0 (socket says 172.21.15.44:57340) 2024-07-17T07:15:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 -- 172.21.15.44:0/2010257005 learned_addr learned my addr 172.21.15.44:0/2010257005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f36800640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 0x7f0f38073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 -- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f3815b4d0 msgr2=0x7f0f3815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f35e00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f3815b4d0 0x7f0f3815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f3815b4d0 0x7f0f3815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 -- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 msgr2=0x7f0f38073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 0x7f0f38073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 -- 172.21.15.44:0/2010257005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f381f3fe0 con 0x7f0f3814e980 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f36800640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 0x7f0f38073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.928+0000 7f0f37200640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0f300017b0 tx=0x7f0f300062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.929+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30006920 con 0x7f0f3814e980 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.929+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f30003af0 con 0x7f0f3814e980 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.929+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30006500 con 0x7f0f3814e980 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.929+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381f4d60 con 0x7f0f3814e980 2024-07-17T07:15:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.929+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f381f4560 con 0x7f0f3814e980 2024-07-17T07:15:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.930+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f380ac560 con 0x7f0f3814e980 2024-07-17T07:15:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.931+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f0f30030050 con 0x7f0f3814e980 2024-07-17T07:15:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.932+0000 7f0f2ea00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 0x7f0f14079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.933+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7f0f300b79d0 con 0x7f0f3814e980 2024-07-17T07:15:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.933+0000 7f0f36800640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 0x7f0f14079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:58.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.934+0000 7f0f36800640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 0x7f0f14079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f280092f0 tx=0x7f0f28009420 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:58.937+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0f30080960 con 0x7f0f3814e980 2024-07-17T07:15:59.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.197+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f380afc90 con 0x7f0f3814e980 2024-07-17T07:15:59.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.198+0000 7f0f2ea00640 1 -- 172.21.15.44:0/2010257005 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+3914 (secure 0 0 0) 0x7f0f30080260 con 0x7f0f3814e980 2024-07-17T07:15:59.198 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:15:59.198 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":72,"btime":"2024-07-17T07:15:51:249588+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/310770571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":310770571},{"type":"v1","addr":"172.21.15.175:6836","nonce":310770571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6275,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1862461576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1862461576},{"type":"v1","addr":"172.21.15.175:6837","nonce":1862461576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2992550804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2992550804},{"type":"v1","addr":"172.21.15.44:6834","nonce":2992550804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2585363735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2585363735},{"type":"v1","addr":"172.21.15.44:6835","nonce":2585363735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[]} 2024-07-17T07:15:59.198 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 72 2024-07-17T07:15:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.201+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 msgr2=0x7f0f14079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.201+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 0x7f0f14079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f280092f0 tx=0x7f0f28009420 comp rx=0 tx=0).stop 2024-07-17T07:15:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.201+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 msgr2=0x7f0f3814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.201+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0f300017b0 tx=0x7f0f300062e0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 shutdown_connections 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0f14076cd0 0x7f0f14079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f3814e980 0x7f0f3814ede0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f3815b4d0 0x7f0f3815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 --2- 172.21.15.44:0/2010257005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f1c004e60 0x7f0f38073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 >> 172.21.15.44:0/2010257005 conn(0x7f0f380b0e80 msgr2=0x7f0f3806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 shutdown_connections 2024-07-17T07:15:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.202+0000 7f0f3ca00640 1 -- 172.21.15.44:0/2010257005 wait complete. 2024-07-17T07:15:59.220 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T07:15:59.220 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T07:15:59.220 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T07:15:59.221 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:59.221 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T07:15:59.221 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T07:15:59.221 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T07:15:59.221 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:59.221 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T07:15:59.221 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T07:15:59.222 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T07:15:59.222 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:59.222 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T07:15:59.222 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T07:15:59.222 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T07:15:59.222 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T07:15:59.238 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T07:15:59.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T07:15:59.241 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T07:15:59.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T07:15:59.244 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T07:15:59.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T07:15:59.248 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T07:15:59.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T07:15:59.288 INFO:tasks.ceph.mds.a:Started 2024-07-17T07:15:59.288 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:59.294 INFO:tasks.ceph.mds.c:Started 2024-07-17T07:15:59.294 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:59.295 INFO:tasks.ceph.mds.d:Started 2024-07-17T07:15:59.296 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:59.301 INFO:tasks.ceph.mds.b:Started 2024-07-17T07:15:59.301 DEBUG:teuthology.parallel:result is None 2024-07-17T07:15:59.302 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T07:15:59.442 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T07:15:59.444 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T07:15:59.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-07-17T07:15:59.448 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T07:15:59.448 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T07:15:59.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 -- 172.21.15.44:0/2017237891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 msgr2=0x7f5d4415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:59.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 --2- 172.21.15.44:0/2017237891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d4415d510 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5d440cf340 tx=0x7f5d2c02a560 comp rx=0 tx=0).stop 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 -- 172.21.15.44:0/2017237891 shutdown_connections 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 --2- 172.21.15.44:0/2017237891 >> v2:172.21.15.175:3300/0 conn(0x7f5d440ce690 0x7f5d440cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 --2- 172.21.15.44:0/2017237891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d4415d510 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 -- 172.21.15.44:0/2017237891 >> 172.21.15.44:0/2017237891 conn(0x7f5d440b4ff0 msgr2=0x7f5d440b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 -- 172.21.15.44:0/2017237891 shutdown_connections 2024-07-17T07:15:59.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.488+0000 7f5d48c00640 1 -- 172.21.15.44:0/2017237891 wait complete. 2024-07-17T07:15:59.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.490+0000 7f5d48c00640 1 Processor -- start 2024-07-17T07:15:59.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.490+0000 7f5d48c00640 1 -- start start 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 0x7f5d4415e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 0x7f5d44160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440d19d0 con 0x7f5d44161be0 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440d1850 con 0x7f5d440ce690 2024-07-17T07:15:59.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.491+0000 7f5d48c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440d1b50 con 0x7f5d4415d130 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57384/0 (socket says 172.21.15.44:57384) 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 -- 172.21.15.44:0/36574013 learned_addr learned my addr 172.21.15.44:0/36574013 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d3be00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 0x7f5d44160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43400640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 0x7f5d4415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 -- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 msgr2=0x7f5d44160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 0x7f5d44160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 -- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 msgr2=0x7f5d4415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 0x7f5d4415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 -- 172.21.15.44:0/36574013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4415d720 con 0x7f5d44161be0 2024-07-17T07:15:59.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d3be00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 0x7f5d44160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43400640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 0x7f5d4415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.492+0000 7f5d43e00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00eab0 tx=0x7f5d3c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.493+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00cea0 con 0x7f5d44161be0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.493+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d3c004b20 con 0x7f5d44161be0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.493+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4415da00 con 0x7f5d44161be0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.493+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c005020 con 0x7f5d44161be0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.493+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4415eb80 con 0x7f5d44161be0 2024-07-17T07:15:59.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.494+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d441f3c60 con 0x7f5d44161be0 2024-07-17T07:15:59.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.495+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f5d3c0051c0 con 0x7f5d44161be0 2024-07-17T07:15:59.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.496+0000 7f5d3b400640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 0x7f5d20079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:15:59.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.496+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6614+0+0 (secure 0 0 0) 0x7f5d3c099520 con 0x7f5d44161be0 2024-07-17T07:15:59.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.497+0000 7f5d43400640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 0x7f5d20079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:15:59.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.497+0000 7f5d43400640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 0x7f5d20079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007fd0 tx=0x7f5d34009040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:15:59.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.500+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5d3c062440 con 0x7f5d44161be0 2024-07-17T07:15:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:15:59.724+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5d440cfdf0 con 0x7f5d44161be0 2024-07-17T07:16:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.342+0000 7f5d3b400640 1 -- 172.21.15.44:0/36574013 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) ==== 97+0+0 (secure 0 0 0) 0x7f5d3c061d40 con 0x7f5d44161be0 2024-07-17T07:16:00.341 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T07:16:00.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.346+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 msgr2=0x7f5d20079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.346+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 0x7f5d20079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007fd0 tx=0x7f5d34009040 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.346+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 msgr2=0x7f5d441f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.346+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00eab0 tx=0x7f5d3c00ef70 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 shutdown_connections 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f5d20076cd0 0x7f5d20079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d44161be0 0x7f5d441f4390 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d4415d130 0x7f5d44160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 --2- 172.21.15.44:0/36574013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d440ce690 0x7f5d4415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 >> 172.21.15.44:0/36574013 conn(0x7f5d440b4ff0 msgr2=0x7f5d440b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 shutdown_connections 2024-07-17T07:16:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.347+0000 7f5d48c00640 1 -- 172.21.15.44:0/36574013 wait complete. 2024-07-17T07:16:00.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T07:16:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.571+0000 7f3ebdc00640 1 -- 172.21.15.44:0/255235329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806e090 msgr2=0x7f3eb806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.571+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/255235329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806e090 0x7f3eb806e4c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005fd0 tx=0x7f3eb002a8a0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 -- 172.21.15.44:0/255235329 shutdown_connections 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/255235329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806e090 0x7f3eb806e4c0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/255235329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb80ac3e0 0x7f3eb80ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 -- 172.21.15.44:0/255235329 >> 172.21.15.44:0/255235329 conn(0x7f3eb80b0fb0 msgr2=0x7f3eb80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 -- 172.21.15.44:0/255235329 shutdown_connections 2024-07-17T07:16:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.572+0000 7f3ebdc00640 1 -- 172.21.15.44:0/255235329 wait complete. 2024-07-17T07:16:00.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.574+0000 7f3ebdc00640 1 Processor -- start 2024-07-17T07:16:00.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.574+0000 7f3ebdc00640 1 -- start start 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb806e090 0x7f3eb8098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb80ac3e0 0x7f3eb8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb806eb80 con 0x7f3eb806f710 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb806ea00 con 0x7f3eb80ac3e0 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3ebdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb806ed00 con 0x7f3eb806e090 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57394/0 (socket says 172.21.15.44:57394) 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.575+0000 7f3eafe00640 1 -- 172.21.15.44:0/3062581840 learned_addr learned my addr 172.21.15.44:0/3062581840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eb6a00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb806e090 0x7f3eb8098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 -- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb806e090 msgr2=0x7f3eb8098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb806e090 0x7f3eb8098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 -- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb80ac3e0 msgr2=0x7f3eb8156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eb7400640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb80ac3e0 0x7f3eb8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb80ac3e0 0x7f3eb8156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 -- 172.21.15.44:0/3062581840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb8073200 con 0x7f3eb806f710 2024-07-17T07:16:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eafe00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b440 tx=0x7f3ea000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0015020 con 0x7f3eb806f710 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea0011980 con 0x7f3eb806f710 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011dd0 con 0x7f3eb806f710 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.576+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb8154650 con 0x7f3eb806f710 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.577+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb8073780 con 0x7f3eb806f710 2024-07-17T07:16:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.578+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb806ff30 con 0x7f3eb806f710 2024-07-17T07:16:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.579+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f3ea0018020 con 0x7f3eb806f710 2024-07-17T07:16:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.580+0000 7f3eaea00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.580+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5797+0+0 (secure 0 0 0) 0x7f3ea0098630 con 0x7f3eb806f710 2024-07-17T07:16:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.581+0000 7f3eb6a00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.582+0000 7f3eb6a00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 0x7f3e94079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80015e0 tx=0x7f3ea8009040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.584+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3eb806ff30 con 0x7f3eb806f710 2024-07-17T07:16:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.876+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3eb806e4c0 con 0x7f3eb806f710 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.878+0000 7f3eaea00640 1 -- 172.21.15.44:0/3062581840 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) ==== 80+0+5304 (secure 0 0 0) 0x7f3ea001f020 con 0x7f3eb806f710 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T07:16:00.877 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.878 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T07:16:00.879 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T07:16:00.880 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.881 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.883 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.885 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.887 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T07:16:00.889 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T07:16:00.891 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T07:16:00.892 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T07:16:00.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:00.892 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T07:16:00.892 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.880+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 msgr2=0x7f3e94079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.880+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 0x7f3e94079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80015e0 tx=0x7f3ea8009040 comp rx=0 tx=0).stop 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.880+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 msgr2=0x7f3eb8073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.880+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b440 tx=0x7f3ea000b900 comp rx=0 tx=0).stop 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.880+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 shutdown_connections 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb80ac3e0 0x7f3eb8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806f710 0x7f3eb8073ca0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 --2- 172.21.15.44:0/3062581840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb806e090 0x7f3eb8098a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 >> 172.21.15.44:0/3062581840 conn(0x7f3eb80b0fb0 msgr2=0x7f3eb80b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 shutdown_connections 2024-07-17T07:16:00.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:00.881+0000 7f3ebdc00640 1 -- 172.21.15.44:0/3062581840 wait complete. 2024-07-17T07:16:00.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T07:16:01.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 -- 172.21.15.44:0/3132102027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa340004800 msgr2=0x7fa340004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 --2- 172.21.15.44:0/3132102027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa340004800 0x7fa340004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c4a0 tx=0x7fa340005830 comp rx=0 tx=0).stop 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 -- 172.21.15.44:0/3132102027 shutdown_connections 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 --2- 172.21.15.44:0/3132102027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa340004800 0x7fa340004c80 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 -- 172.21.15.44:0/3132102027 >> 172.21.15.44:0/3132102027 conn(0x7fa3580b0fb0 msgr2=0x7fa3580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 -- 172.21.15.44:0/3132102027 shutdown_connections 2024-07-17T07:16:01.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.117+0000 7fa360200640 1 -- 172.21.15.44:0/3132102027 wait complete. 2024-07-17T07:16:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.119+0000 7fa360200640 1 Processor -- start 2024-07-17T07:16:01.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.119+0000 7fa360200640 1 -- start start 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa340004800 0x7fa3580745c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 0x7fa358152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa358153bd0 con 0x7fa3581f3440 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa358153a50 con 0x7fa340004800 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa360200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa358153d50 con 0x7fa3581411c0 2024-07-17T07:16:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa35e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa35e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57418/0 (socket says 172.21.15.44:57418) 2024-07-17T07:16:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.120+0000 7fa35e200640 1 -- 172.21.15.44:0/3657161308 learned_addr learned my addr 172.21.15.44:0/3657161308 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35ce00640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 0x7fa358152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 -- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 msgr2=0x7fa358152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35d800640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa340004800 0x7fa3580745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 0x7fa358152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 -- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa340004800 msgr2=0x7fa3580745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa340004800 0x7fa3580745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 -- 172.21.15.44:0/3657161308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3581426b0 con 0x7fa3581f3440 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35ce00640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 0x7fa358152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa35e200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa35806d4c0 tx=0x7fa35000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350018070 con 0x7fa3581f3440 2024-07-17T07:16:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa350003b80 con 0x7fa3581f3440 2024-07-17T07:16:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.121+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500050d0 con 0x7fa3581f3440 2024-07-17T07:16:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.122+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa358071980 con 0x7fa3581f3440 2024-07-17T07:16:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.122+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa358072080 con 0x7fa3581f3440 2024-07-17T07:16:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.123+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35815a150 con 0x7fa3581f3440 2024-07-17T07:16:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.124+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fa35001b070 con 0x7fa3581f3440 2024-07-17T07:16:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.125+0000 7fa355600640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 0x7fa338079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.125+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5797+0+0 (secure 0 0 0) 0x7fa350097ac0 con 0x7fa3581f3440 2024-07-17T07:16:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.126+0000 7fa35d800640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 0x7fa338079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.127+0000 7fa35d800640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 0x7fa338079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0058c0 tx=0x7fa34c005850 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:01.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.130+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa35806f1a0 con 0x7fa3581f3440 2024-07-17T07:16:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.341+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa358071e30 con 0x7fa3581f3440 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.342+0000 7fa355600640 1 -- 172.21.15.44:0/3657161308 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) ==== 84+0+65 (secure 0 0 0) 0x7fa350060d80 con 0x7fa3581f3440 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T07:16:01.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:01.342 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 msgr2=0x7fa338079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 0x7fa338079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0058c0 tx=0x7fa34c005850 comp rx=0 tx=0).stop 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 msgr2=0x7fa358072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa35806d4c0 tx=0x7fa35000da00 comp rx=0 tx=0).stop 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 shutdown_connections 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa338076cd0 0x7fa338079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3581f3440 0x7fa358072380 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3581411c0 0x7fa358152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 --2- 172.21.15.44:0/3657161308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa340004800 0x7fa3580745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 >> 172.21.15.44:0/3657161308 conn(0x7fa3580b0fb0 msgr2=0x7fa35815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 shutdown_connections 2024-07-17T07:16:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.345+0000 7fa360200640 1 -- 172.21.15.44:0/3657161308 wait complete. 2024-07-17T07:16:01.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T07:16:01.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T07:16:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 -- 172.21.15.44:0/161855157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe56815e370 msgr2=0x7fe5680ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 --2- 172.21.15.44:0/161855157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe56815e370 0x7fe5680ce500 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe568160e60 tx=0x7fe55802a4d0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 -- 172.21.15.44:0/161855157 shutdown_connections 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 --2- 172.21.15.44:0/161855157 >> v2:172.21.15.175:3300/0 conn(0x7fe5680d0b40 0x7fe5680b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 --2- 172.21.15.44:0/161855157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe56815e370 0x7fe5680ce500 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 -- 172.21.15.44:0/161855157 >> 172.21.15.44:0/161855157 conn(0x7fe568073a60 msgr2=0x7fe568073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 -- 172.21.15.44:0/161855157 shutdown_connections 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.579+0000 7fe56d000640 1 -- 172.21.15.44:0/161855157 wait complete. 2024-07-17T07:16:01.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.581+0000 7fe56d000640 1 Processor -- start 2024-07-17T07:16:01.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.581+0000 7fe56d000640 1 -- start start 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5680d0b40 0x7fe5680c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe56815e370 0x7fe5680cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680b8d30 con 0x7fe568161b50 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680b8bb0 con 0x7fe5680d0b40 2024-07-17T07:16:01.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe56d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5680b8eb0 con 0x7fe56815e370 2024-07-17T07:16:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe567e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe567e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57446/0 (socket says 172.21.15.44:57446) 2024-07-17T07:16:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.582+0000 7fe567e00640 1 -- 172.21.15.44:0/990274935 learned_addr learned my addr 172.21.15.44:0/990274935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:01.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567400640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5680d0b40 0x7fe5680c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 -- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe56815e370 msgr2=0x7fe5680cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe566a00640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe56815e370 0x7fe5680cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe56815e370 0x7fe5680cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 -- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5680d0b40 msgr2=0x7fe5680c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5680d0b40 0x7fe5680c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 -- 172.21.15.44:0/990274935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5680b57b0 con 0x7fe568161b50 2024-07-17T07:16:01.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe567e00640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe560009fd0 tx=0x7fe56000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.583+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560011830 con 0x7fe568161b50 2024-07-17T07:16:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.584+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe560016070 con 0x7fe568161b50 2024-07-17T07:16:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.584+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5680b5a70 con 0x7fe568161b50 2024-07-17T07:16:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.584+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560011d40 con 0x7fe568161b50 2024-07-17T07:16:01.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.584+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568157b20 con 0x7fe568161b50 2024-07-17T07:16:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.585+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5680b9360 con 0x7fe568161b50 2024-07-17T07:16:01.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.586+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fe56001b460 con 0x7fe568161b50 2024-07-17T07:16:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.587+0000 7fe55f400640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 0x7fe544079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.587+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5797+0+0 (secure 0 0 0) 0x7fe5600982b0 con 0x7fe568161b50 2024-07-17T07:16:01.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.588+0000 7fe567400640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 0x7fe544079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:01.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.589+0000 7fe567400640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 0x7fe544079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe558033000 tx=0x7fe558004730 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:01.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.591+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe560061600 con 0x7fe568161b50 2024-07-17T07:16:01.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:01.804+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe5681582a0 con 0x7fe568161b50 2024-07-17T07:16:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.355+0000 7fe55f400640 1 -- 172.21.15.44:0/990274935 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v152) ==== 138+0+0 (secure 0 0 0) 0x7fe560060f00 con 0x7fe568161b50 2024-07-17T07:16:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T07:16:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 msgr2=0x7fe544079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 0x7fe544079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe558033000 tx=0x7fe558004730 comp rx=0 tx=0).stop 2024-07-17T07:16:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 msgr2=0x7fe5680cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe560009fd0 tx=0x7fe56000b8e0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 shutdown_connections 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe544076cd0 0x7fe544079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568161b50 0x7fe5680cbb20 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe56815e370 0x7fe5680cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 --2- 172.21.15.44:0/990274935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5680d0b40 0x7fe5680c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.359+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 >> 172.21.15.44:0/990274935 conn(0x7fe568073a60 msgr2=0x7fe56815baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.360+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 shutdown_connections 2024-07-17T07:16:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.360+0000 7fe56d000640 1 -- 172.21.15.44:0/990274935 wait complete. 2024-07-17T07:16:02.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T07:16:02.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 -- 172.21.15.44:0/3496077405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 msgr2=0x7f729c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 --2- 172.21.15.44:0/3496077405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f729c004c80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009b50 tx=0x7f729c005830 comp rx=0 tx=0).stop 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72b6800640 1 -- 172.21.15.44:0/3496077405 reap_dead start 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 -- 172.21.15.44:0/3496077405 shutdown_connections 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 -- 172.21.15.44:0/3496077405 >> 172.21.15.44:0/3496077405 conn(0x7f72b80b0eb0 msgr2=0x7f72b80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 -- 172.21.15.44:0/3496077405 shutdown_connections 2024-07-17T07:16:02.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.600+0000 7f72bcc00640 1 -- 172.21.15.44:0/3496077405 wait complete. 2024-07-17T07:16:02.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.602+0000 7f72bcc00640 1 Processor -- start 2024-07-17T07:16:02.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.602+0000 7f72bcc00640 1 -- start start 2024-07-17T07:16:02.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:02.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8073dd0 0x7f72b8147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:02.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 0x7f72b814b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72b815c470 con 0x7f729c004800 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72b815c2f0 con 0x7f72b8073dd0 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.603+0000 7f72bcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72b815c5f0 con 0x7f72b81476d0 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57458/0 (socket says 172.21.15.44:57458) 2024-07-17T07:16:02.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 -- 172.21.15.44:0/1100778701 learned_addr learned my addr 172.21.15.44:0/1100778701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72ad400640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8073dd0 0x7f72b8147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b6800640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 0x7f72b814b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 -- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 msgr2=0x7f72b814b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 0x7f72b814b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 -- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8073dd0 msgr2=0x7f72b8147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8073dd0 0x7f72b8147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b5e00640 1 -- 172.21.15.44:0/1100778701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b81489d0 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.604+0000 7f72b6800640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 0x7f72b814b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72b5e00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f72a800d580 tx=0x7f72a800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8016020 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a8012630 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012a40 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b8148cb0 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.605+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b8074230 con 0x7f729c004800 2024-07-17T07:16:02.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.606+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b80cd2e0 con 0x7f729c004800 2024-07-17T07:16:02.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.608+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f72a8019020 con 0x7f729c004800 2024-07-17T07:16:02.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.610+0000 7f72aea00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 0x7f72940791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:02.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.610+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6167+0+0 (secure 0 0 0) 0x7f72a8098610 con 0x7f729c004800 2024-07-17T07:16:02.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.610+0000 7f72ad400640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 0x7f72940791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:02.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.612+0000 7f72ad400640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 0x7f72940791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72a0035000 tx=0x7f72a002d040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:02.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.614+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f72a8020030 con 0x7f729c004800 2024-07-17T07:16:02.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:02.824+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f72b81504f0 con 0x7f729c004800 2024-07-17T07:16:03.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.363+0000 7f72aea00640 1 -- 172.21.15.44:0/1100778701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v153) ==== 170+0+0 (secure 0 0 0) 0x7f72a80617f0 con 0x7f729c004800 2024-07-17T07:16:03.362 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T07:16:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.366+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 msgr2=0x7f72940791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.366+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 0x7f72940791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72a0035000 tx=0x7f72a002d040 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.366+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 msgr2=0x7f72b80715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.366+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f72a800d580 tx=0x7f72a800da40 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 shutdown_connections 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7294076d00 0x7f72940791c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72b81476d0 0x7f72b814b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72b8073dd0 0x7f72b8147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 --2- 172.21.15.44:0/1100778701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f729c004800 0x7f72b80715e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 >> 172.21.15.44:0/1100778701 conn(0x7f72b80b0eb0 msgr2=0x7f72b80aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:03.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 shutdown_connections 2024-07-17T07:16:03.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.367+0000 7f72bcc00640 1 -- 172.21.15.44:0/1100778701 wait complete. 2024-07-17T07:16:03.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T07:16:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 -- 172.21.15.44:0/4218544178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c0b5ae0 msgr2=0x7fb60c072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 --2- 172.21.15.44:0/4218544178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c0b5ae0 0x7fb60c072e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4005fd0 tx=0x7fb5f402a5d0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 -- 172.21.15.44:0/4218544178 shutdown_connections 2024-07-17T07:16:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 --2- 172.21.15.44:0/4218544178 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c0b5ae0 0x7fb60c072e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 --2- 172.21.15.44:0/4218544178 >> v2:172.21.15.175:3300/0 conn(0x7fb60c159990 0x7fb60c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 -- 172.21.15.44:0/4218544178 >> 172.21.15.44:0/4218544178 conn(0x7fb60c06ce80 msgr2=0x7fb60c06d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 -- 172.21.15.44:0/4218544178 shutdown_connections 2024-07-17T07:16:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.588+0000 7fb612000640 1 -- 172.21.15.44:0/4218544178 wait complete. 2024-07-17T07:16:03.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.590+0000 7fb612000640 1 Processor -- start 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.590+0000 7fb612000640 1 -- start start 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 0x7fb60c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 0x7fb60c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c068ca0 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c068b20 con 0x7fb60c0733d0 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb612000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c068e20 con 0x7fb60c159990 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb602a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb602a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57466/0 (socket says 172.21.15.44:57466) 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.591+0000 7fb602a00640 1 -- 172.21.15.44:0/3221253512 learned_addr learned my addr 172.21.15.44:0/3221253512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb60be00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 0x7fb60c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb60b400640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 0x7fb60c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 -- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 msgr2=0x7fb60c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 0x7fb60c06ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 -- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 msgr2=0x7fb60c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 0x7fb60c1517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 -- 172.21.15.44:0/3221253512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c1f4e90 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb60b400640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 0x7fb60c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb60be00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 0x7fb60c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.592+0000 7fb602a00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40099f0 tx=0x7fb5f40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.593+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f403f070 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.593+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f4005600 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.593+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4035030 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.593+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c1f5170 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.593+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c152a40 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.594+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c0b06f0 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.596+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fb5f4042070 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.597+0000 7fb603e00640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 0x7fb5e8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.597+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 6549+0+0 (secure 0 0 0) 0x7fb5f40b9720 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.597+0000 7fb60b400640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 0x7fb5e8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.599+0000 7fb60b400640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 0x7fb5e8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00cb50 tx=0x7fb5fc005db0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.600+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb5f40827f0 con 0x7fb60c0b5ae0 2024-07-17T07:16:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:03.847+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb60c157ed0 con 0x7fb60c0b5ae0 2024-07-17T07:16:04.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.408+0000 7fb603e00640 1 -- 172.21.15.44:0/3221253512 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v75) ==== 174+0+0 (secure 0 0 0) 0x7fb5f40820f0 con 0x7fb60c0b5ae0 2024-07-17T07:16:04.408 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 19 and data pool 20 2024-07-17T07:16:04.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.415+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 msgr2=0x7fb5e8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.415+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 0x7fb5e8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00cb50 tx=0x7fb5fc005db0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.415+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 msgr2=0x7fb60c1f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.415+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40099f0 tx=0x7fb5f40038a0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 shutdown_connections 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb5e8076cd0 0x7fb5e8079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c159990 0x7fb60c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb60c0b5ae0 0x7fb60c1f4520 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 --2- 172.21.15.44:0/3221253512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c0733d0 0x7fb60c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 >> 172.21.15.44:0/3221253512 conn(0x7fb60c06ce80 msgr2=0x7fb60c1583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:04.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.416+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 shutdown_connections 2024-07-17T07:16:04.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.421+0000 7fb612000640 1 -- 172.21.15.44:0/3221253512 wait complete. 2024-07-17T07:16:04.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.634+0000 7efd07400640 1 -- 172.21.15.44:0/1200138014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b2ad0 msgr2=0x7efd000b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.634+0000 7efd07400640 1 --2- 172.21.15.44:0/1200138014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b2ad0 0x7efd000b5200 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efd00069c00 tx=0x7efce802a5d0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 -- 172.21.15.44:0/1200138014 shutdown_connections 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 --2- 172.21.15.44:0/1200138014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b2ad0 0x7efd000b5200 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 --2- 172.21.15.44:0/1200138014 >> v2:172.21.15.175:3300/0 conn(0x7efd0015d600 0x7efd0006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 -- 172.21.15.44:0/1200138014 >> 172.21.15.44:0/1200138014 conn(0x7efd000d00a0 msgr2=0x7efd000d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 -- 172.21.15.44:0/1200138014 shutdown_connections 2024-07-17T07:16:04.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.635+0000 7efd07400640 1 -- 172.21.15.44:0/1200138014 wait complete. 2024-07-17T07:16:04.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.636+0000 7efd07400640 1 Processor -- start 2024-07-17T07:16:04.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.637+0000 7efd07400640 1 -- start start 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd000b2ad0 0x7efd001f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0015d600 0x7efd000b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd00161730 con 0x7efd000b5740 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd000cef10 con 0x7efd000b2ad0 2024-07-17T07:16:04.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd07400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd001618b0 con 0x7efd0015d600 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57496/0 (socket says 172.21.15.44:57496) 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 -- 172.21.15.44:0/3508570716 learned_addr learned my addr 172.21.15.44:0/3508570716 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efd04a00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd000b2ad0 0x7efd001f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 -- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0015d600 msgr2=0x7efd000b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0015d600 0x7efd000b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 -- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd000b2ad0 msgr2=0x7efd001f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd000b2ad0 0x7efd001f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.638+0000 7efcf7e00640 1 -- 172.21.15.44:0/3508570716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd001f3e70 con 0x7efd000b5740 2024-07-17T07:16:04.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efcf7e00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efce8009780 tx=0x7efce80329e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:04.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8040070 con 0x7efd000b5740 2024-07-17T07:16:04.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efce8005110 con 0x7efd000b5740 2024-07-17T07:16:04.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8005560 con 0x7efd000b5740 2024-07-17T07:16:04.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd00154a00 con 0x7efd000b5740 2024-07-17T07:16:04.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.639+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd001f4130 con 0x7efd000b5740 2024-07-17T07:16:04.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.641+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd00161dd0 con 0x7efd000b5740 2024-07-17T07:16:04.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.642+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7efce8039040 con 0x7efd000b5740 2024-07-17T07:16:04.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.643+0000 7efcfd600640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 0x7efcdc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:04.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.643+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6649+0+0 (secure 0 0 0) 0x7efce80b9860 con 0x7efd000b5740 2024-07-17T07:16:04.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.643+0000 7efd04a00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 0x7efcdc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:04.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.645+0000 7efd04a00640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 0x7efcdc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005fd0 tx=0x7efcf0005850 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:04.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.647+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efce8082750 con 0x7efd000b5740 2024-07-17T07:16:04.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.858+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd0015a340 con 0x7efd000b5740 2024-07-17T07:16:04.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.860+0000 7efcfd600640 1 -- 172.21.15.44:0/3508570716 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+17732 (secure 0 0 0) 0x7efce8082050 con 0x7efd000b5740 2024-07-17T07:16:04.859 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:04.859 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":154,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:16:03.849524+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:16:01.823652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-07-17T07:16:02.840012+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:16:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.862+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 msgr2=0x7efcdc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.862+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 0x7efcdc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005fd0 tx=0x7efcf0005850 comp rx=0 tx=0).stop 2024-07-17T07:16:04.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.862+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 msgr2=0x7efd001541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.862+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efce8009780 tx=0x7efce80329e0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 shutdown_connections 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efcdc076cd0 0x7efcdc079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0015d600 0x7efd000b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd000b5740 0x7efd001541b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 --2- 172.21.15.44:0/3508570716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd000b2ad0 0x7efd001f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 >> 172.21.15.44:0/3508570716 conn(0x7efd000d00a0 msgr2=0x7efd0006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:04.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 shutdown_connections 2024-07-17T07:16:04.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:04.863+0000 7efd07400640 1 -- 172.21.15.44:0/3508570716 wait complete. 2024-07-17T07:16:04.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:16:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 -- 172.21.15.44:0/1626304660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 msgr2=0x7f7be40b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 --2- 172.21.15.44:0/1626304660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be40b3870 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4005fd0 tx=0x7f7bd402aa10 comp rx=0 tx=0).stop 2024-07-17T07:16:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 -- 172.21.15.44:0/1626304660 shutdown_connections 2024-07-17T07:16:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 --2- 172.21.15.44:0/1626304660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be40b3870 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 -- 172.21.15.44:0/1626304660 >> 172.21.15.44:0/1626304660 conn(0x7f7be40b0e80 msgr2=0x7f7be40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 -- 172.21.15.44:0/1626304660 shutdown_connections 2024-07-17T07:16:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.085+0000 7f7bea400640 1 -- 172.21.15.44:0/1626304660 wait complete. 2024-07-17T07:16:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.087+0000 7f7bea400640 1 Processor -- start 2024-07-17T07:16:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.087+0000 7f7bea400640 1 -- start start 2024-07-17T07:16:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7be40b3db0 0x7f7be413b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 0x7f7be415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7be40ae380 con 0x7f7be40cf540 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7be40ae200 con 0x7f7be40cf920 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7bea400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7be40ae500 con 0x7f7be40b3db0 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57516/0 (socket says 172.21.15.44:57516) 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 -- 172.21.15.44:0/204812659 learned_addr learned my addr 172.21.15.44:0/204812659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be9000640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7be40b3db0 0x7f7be413b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be9a00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 0x7f7be415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 -- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7be40b3db0 msgr2=0x7f7be413b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7be40b3db0 0x7f7be413b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 -- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 msgr2=0x7f7be415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 0x7f7be415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.089+0000 7f7be3e00640 1 -- 172.21.15.44:0/204812659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be4157a30 con 0x7f7be40cf540 2024-07-17T07:16:05.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7be9a00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 0x7f7be415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7be3e00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c450 tx=0x7f7bcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:05.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0052d0 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bcc005470 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be4157d10 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc012870 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.090+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be41582d0 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.091+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be40ae990 con 0x7f7be40cf540 2024-07-17T07:16:05.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.093+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f7bcc0035f0 con 0x7f7be40cf540 2024-07-17T07:16:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.093+0000 7f7be0c00640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 0x7f7bc0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.094+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6649+0+0 (secure 0 0 0) 0x7f7bcc098440 con 0x7f7be40cf540 2024-07-17T07:16:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.094+0000 7f7be9000640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 0x7f7bc0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.096+0000 7f7be9000640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 0x7f7bc0079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4035000 tx=0x7f7bd403a040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:05.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.098+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7bcc0613b0 con 0x7f7be40cf540 2024-07-17T07:16:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.311+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7be40adaf0 con 0x7f7be40cf540 2024-07-17T07:16:05.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.313+0000 7f7be0c00640 1 -- 172.21.15.44:0/204812659 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+17732 (secure 0 0 0) 0x7f7bcc060cb0 con 0x7f7be40cf540 2024-07-17T07:16:05.313 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:05.313 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":154,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:16:03.849524+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:16:01.823652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-07-17T07:16:02.840012+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.316+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 msgr2=0x7f7bc0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.316+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 0x7f7bc0079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4035000 tx=0x7f7bd403a040 comp rx=0 tx=0).stop 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 msgr2=0x7f7be4073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c450 tx=0x7f7bcc00c910 comp rx=0 tx=0).stop 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 shutdown_connections 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f7bc0076cd0 0x7f7bc0079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7be40cf920 0x7f7be415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be40cf540 0x7f7be4073bf0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 --2- 172.21.15.44:0/204812659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7be40b3db0 0x7f7be413b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 >> 172.21.15.44:0/204812659 conn(0x7f7be40b0e80 msgr2=0x7f7be409ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:05.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 shutdown_connections 2024-07-17T07:16:05.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.317+0000 7f7bea400640 1 -- 172.21.15.44:0/204812659 wait complete. 2024-07-17T07:16:05.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T07:16:05.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.553+0000 7f1343600640 1 -- 172.21.15.44:0/3103456915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1320003a50 msgr2=0x7f1320003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.553+0000 7f1343600640 1 --2- 172.21.15.44:0/3103456915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1320003a50 0x7f1320003e70 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f132c009e20 tx=0x7f1320005830 comp rx=0 tx=0).stop 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.554+0000 7f1343600640 1 -- 172.21.15.44:0/3103456915 shutdown_connections 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.554+0000 7f1343600640 1 --2- 172.21.15.44:0/3103456915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1320003a50 0x7f1320003e70 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.554+0000 7f1343600640 1 -- 172.21.15.44:0/3103456915 >> 172.21.15.44:0/3103456915 conn(0x7f133c0b2240 msgr2=0x7f133c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.554+0000 7f1343600640 1 -- 172.21.15.44:0/3103456915 shutdown_connections 2024-07-17T07:16:05.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.554+0000 7f1343600640 1 -- 172.21.15.44:0/3103456915 wait complete. 2024-07-17T07:16:05.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.555+0000 7f1343600640 1 Processor -- start 2024-07-17T07:16:05.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.555+0000 7f1343600640 1 -- start start 2024-07-17T07:16:05.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 0x7f133c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 0x7f133c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0cfd00 con 0x7f133c14cf00 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f133c15bef0 con 0x7f133c150b00 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.556+0000 7f1343600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f133c0cfe80 con 0x7f1320003a50 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57532/0 (socket says 172.21.15.44:57532) 2024-07-17T07:16:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 -- 172.21.15.44:0/4223474986 learned_addr learned my addr 172.21.15.44:0/4223474986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f1340c00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 0x7f133c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f1341600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 0x7f133c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 -- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 msgr2=0x7f133c14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 0x7f133c14f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 -- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 msgr2=0x7f133c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 0x7f133c150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 -- 172.21.15.44:0/4223474986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1f4790 con 0x7f133c14cf00 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f1340c00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 0x7f133c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f1341600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 0x7f133c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.557+0000 7f133be00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f132400c730 tx=0x7f132400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.558+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324017070 con 0x7f133c14cf00 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.558+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1324005380 con 0x7f133c14cf00 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.558+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324003850 con 0x7f133c14cf00 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.558+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1f4a70 con 0x7f133c14cf00 2024-07-17T07:16:05.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.558+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1f5030 con 0x7f133c14cf00 2024-07-17T07:16:05.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.559+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c06c400 con 0x7f133c14cf00 2024-07-17T07:16:05.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.560+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f132401a070 con 0x7f133c14cf00 2024-07-17T07:16:05.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.561+0000 7f1338c00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 0x7f13180791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:05.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.561+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f1324097f90 con 0x7f133c14cf00 2024-07-17T07:16:05.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.562+0000 7f1340c00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 0x7f13180791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:05.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.563+0000 7f1340c00640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 0x7f13180791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f132c035000 tx=0x7f132c02d040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.565+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f133c0cf280 con 0x7f133c14cf00 2024-07-17T07:16:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:05.812+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f133c0b2df0 con 0x7f133c14cf00 2024-07-17T07:16:06.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-07-17T07:16:06.349 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-07-17T07:16:06.349+0000 7f10cac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T07:16:06.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.433+0000 7f1338c00640 1 -- 172.21.15.44:0/4223474986 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) ==== 118+0+0 (secure 0 0 0) 0x7f1324060f90 con 0x7f133c14cf00 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.437+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 msgr2=0x7f13180791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.437+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 0x7f13180791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f132c035000 tx=0x7f132c02d040 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 msgr2=0x7f133c13dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f132400c730 tx=0x7f132400cbf0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 shutdown_connections 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1318076d00 0x7f13180791c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133c150b00 0x7f133c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f133c14cf00 0x7f133c13dee0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 --2- 172.21.15.44:0/4223474986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1320003a50 0x7f133c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 >> 172.21.15.44:0/4223474986 conn(0x7f133c0b2240 msgr2=0x7f133c15ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:06.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 shutdown_connections 2024-07-17T07:16:06.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.438+0000 7f1343600640 1 -- 172.21.15.44:0/4223474986 wait complete. 2024-07-17T07:16:06.459 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T07:16:06.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T07:16:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 -- 172.21.15.44:0/1498834908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd4003a30 msgr2=0x7effd4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 --2- 172.21.15.44:0/1498834908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd4003a30 0x7effd4003e50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7effe0007540 tx=0x7effd4005830 comp rx=0 tx=0).stop 2024-07-17T07:16:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff2200640 1 -- 172.21.15.44:0/1498834908 reap_dead start 2024-07-17T07:16:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 -- 172.21.15.44:0/1498834908 shutdown_connections 2024-07-17T07:16:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 -- 172.21.15.44:0/1498834908 >> 172.21.15.44:0/1498834908 conn(0x7effec0b0eb0 msgr2=0x7effec0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 -- 172.21.15.44:0/1498834908 shutdown_connections 2024-07-17T07:16:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.681+0000 7efff4200640 1 -- 172.21.15.44:0/1498834908 wait complete. 2024-07-17T07:16:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.683+0000 7efff4200640 1 Processor -- start 2024-07-17T07:16:06.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.683+0000 7efff4200640 1 -- start start 2024-07-17T07:16:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd4003a30 0x7effec151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 0x7effec073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effec15a5a0 con 0x7effec151e80 2024-07-17T07:16:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effec06f8e0 con 0x7effd4003a30 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff4200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effec15a720 con 0x7effec0737b0 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57548/0 (socket says 172.21.15.44:57548) 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.684+0000 7efff0e00640 1 -- 172.21.15.44:0/2812531462 learned_addr learned my addr 172.21.15.44:0/2812531462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff2200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 0x7effec073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:06.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 -- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 msgr2=0x7effec073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff1800640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd4003a30 0x7effec151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 0x7effec073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 -- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd4003a30 msgr2=0x7effec151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd4003a30 0x7effec151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:06.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 -- 172.21.15.44:0/2812531462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec152420 con 0x7effec151e80 2024-07-17T07:16:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff2200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 0x7effec073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.685+0000 7efff0e00640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7effd800c500 tx=0x7effd800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.686+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8017070 con 0x7effec151e80 2024-07-17T07:16:06.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.686+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effd80053e0 con 0x7effec151e80 2024-07-17T07:16:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.686+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8003850 con 0x7effec151e80 2024-07-17T07:16:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.686+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec152700 con 0x7effec151e80 2024-07-17T07:16:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.686+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec152cc0 con 0x7effec151e80 2024-07-17T07:16:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.687+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effec074170 con 0x7effec151e80 2024-07-17T07:16:06.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.688+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7effd80039f0 con 0x7effec151e80 2024-07-17T07:16:06.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.689+0000 7effe9600640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 0x7effcc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:06.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.689+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7effd8097c20 con 0x7effec151e80 2024-07-17T07:16:06.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.690+0000 7efff1800640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 0x7effcc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:06.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.691+0000 7efff1800640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 0x7effcc0791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe0034000 tx=0x7effe000f040 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:06.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.693+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7effd8060c60 con 0x7effec151e80 2024-07-17T07:16:06.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:06.946+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7effec150550 con 0x7effec151e80 2024-07-17T07:16:07.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.447+0000 7effe9600640 1 -- 172.21.15.44:0/2812531462 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) ==== 122+0+0 (secure 0 0 0) 0x7effd8060560 con 0x7effec151e80 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 msgr2=0x7effcc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 0x7effcc0791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effe0034000 tx=0x7effe000f040 comp rx=0 tx=0).stop 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 msgr2=0x7effec1f1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7effd800c500 tx=0x7effd800c9c0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 shutdown_connections 2024-07-17T07:16:07.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7effcc076d00 0x7effcc0791c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effec0737b0 0x7effec073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effec151e80 0x7effec1f1a50 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 --2- 172.21.15.44:0/2812531462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effd4003a30 0x7effec151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 >> 172.21.15.44:0/2812531462 conn(0x7effec0b0eb0 msgr2=0x7effec0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 shutdown_connections 2024-07-17T07:16:07.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.453+0000 7efff4200640 1 -- 172.21.15.44:0/2812531462 wait complete. 2024-07-17T07:16:07.475 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T07:16:07.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:16:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 -- 172.21.15.44:0/500485116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e000ac220 msgr2=0x7f8e000ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 --2- 172.21.15.44:0/500485116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e000ac220 0x7f8e000ac690 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005fd0 tx=0x7f8dec02a5d0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 -- 172.21.15.44:0/500485116 shutdown_connections 2024-07-17T07:16:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 --2- 172.21.15.44:0/500485116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e000cd690 0x7f8e000cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 --2- 172.21.15.44:0/500485116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e000ac220 0x7f8e000ac690 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.674+0000 7f8e05800640 1 -- 172.21.15.44:0/500485116 >> 172.21.15.44:0/500485116 conn(0x7f8e000b0fb0 msgr2=0x7f8e000b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:07.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.675+0000 7f8e05800640 1 -- 172.21.15.44:0/500485116 shutdown_connections 2024-07-17T07:16:07.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.675+0000 7f8e05800640 1 -- 172.21.15.44:0/500485116 wait complete. 2024-07-17T07:16:07.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.676+0000 7f8e05800640 1 Processor -- start 2024-07-17T07:16:07.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.677+0000 7f8e05800640 1 -- start start 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e000ac220 0x7f8e0015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e000cd690 0x7f8e00073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000ce1e0 con 0x7f8e001522c0 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000ce060 con 0x7f8e000ac220 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8e05800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e000ce360 con 0x7f8e000cd690 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57556/0 (socket says 172.21.15.44:57556) 2024-07-17T07:16:07.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 -- 172.21.15.44:0/3330686945 learned_addr learned my addr 172.21.15.44:0/3330686945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:07.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 -- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e000cd690 msgr2=0x7f8e00073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e000cd690 0x7f8e00073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 -- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e000ac220 msgr2=0x7f8e0015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e000ac220 0x7f8e0015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.678+0000 7f8dff400640 1 -- 172.21.15.44:0/3330686945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e000729e0 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8dff400640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8df800b9e0 tx=0x7f8df800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8019070 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8df800cbb0 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8014440 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e00072cc0 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.679+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e00073280 con 0x7f8e001522c0 2024-07-17T07:16:07.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.681+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e001547a0 con 0x7f8e001522c0 2024-07-17T07:16:07.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.682+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f8df801c070 con 0x7f8e001522c0 2024-07-17T07:16:07.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.683+0000 7f8df6a00640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 0x7f8de0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.683+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f8df8098640 con 0x7f8e001522c0 2024-07-17T07:16:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.684+0000 7f8dfea00640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 0x7f8de0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.685+0000 7f8dfea00640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 0x7f8de0079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8df0001510 tx=0x7f8df0007930 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:07.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.687+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8df80615b0 con 0x7f8e001522c0 2024-07-17T07:16:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.940+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e000ce880 con 0x7f8e001522c0 2024-07-17T07:16:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.941+0000 7f8df6a00640 1 -- 172.21.15.44:0/3330686945 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7f8df8060eb0 con 0x7f8e001522c0 2024-07-17T07:16:07.941 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:07.941 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T07:16:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T07:16:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 msgr2=0x7f8de0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 0x7f8de0079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8df0001510 tx=0x7f8df0007930 comp rx=0 tx=0).stop 2024-07-17T07:16:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 msgr2=0x7f8e00154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8df800b9e0 tx=0x7f8df800bea0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 shutdown_connections 2024-07-17T07:16:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8de0076cd0 0x7f8de0079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e001522c0 0x7f8e00154ae0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e000cd690 0x7f8e00073ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 --2- 172.21.15.44:0/3330686945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e000ac220 0x7f8e0015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 >> 172.21.15.44:0/3330686945 conn(0x7f8e000b0fb0 msgr2=0x7f8e000afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 shutdown_connections 2024-07-17T07:16:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:07.944+0000 7f8e05800640 1 -- 172.21.15.44:0/3330686945 wait complete. 2024-07-17T07:16:07.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T07:16:08.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 -- 172.21.15.44:0/329955531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b6e0 msgr2=0x7f56500cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 --2- 172.21.15.44:0/329955531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b6e0 0x7f56500cf380 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5638005fd0 tx=0x7f563802b1a0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 -- 172.21.15.44:0/329955531 shutdown_connections 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 --2- 172.21.15.44:0/329955531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b6e0 0x7f56500cf380 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 --2- 172.21.15.44:0/329955531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f565009b300 0x7f565006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 -- 172.21.15.44:0/329955531 >> 172.21.15.44:0/329955531 conn(0x7f56500b0e80 msgr2=0x7f56500b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 -- 172.21.15.44:0/329955531 shutdown_connections 2024-07-17T07:16:08.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.172+0000 7f564fe00640 1 -- 172.21.15.44:0/329955531 wait complete. 2024-07-17T07:16:08.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.174+0000 7f564fe00640 1 Processor -- start 2024-07-17T07:16:08.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.174+0000 7f564fe00640 1 -- start start 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f565009b6e0 0x7f5650073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56500cf8c0 0x7f565015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56500b2120 con 0x7f565009b300 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56500b1fa0 con 0x7f565009b6e0 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56500b22a0 con 0x7f56500cf8c0 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.175+0000 7f564ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:08.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57584/0 (socket says 172.21.15.44:57584) 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 -- 172.21.15.44:0/2146688504 learned_addr learned my addr 172.21.15.44:0/2146688504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 -- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56500cf8c0 msgr2=0x7f565015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56500cf8c0 0x7f565015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 -- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f565009b6e0 msgr2=0x7f5650073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f565009b6e0 0x7f5650073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 -- 172.21.15.44:0/2146688504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56501f76d0 con 0x7f565009b300 2024-07-17T07:16:08.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.176+0000 7f564ea00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f564000c970 tx=0x7f564000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.177+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640007bb0 con 0x7f565009b300 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.177+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5640005020 con 0x7f565009b300 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.177+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640007e60 con 0x7f565009b300 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.177+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56501f49e0 con 0x7f565009b300 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.177+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5650073010 con 0x7f565009b300 2024-07-17T07:16:08.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.178+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f565006e980 con 0x7f565009b300 2024-07-17T07:16:08.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.179+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f5640005470 con 0x7f565009b300 2024-07-17T07:16:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.180+0000 7f5646a00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 0x7f562c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.180+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f5640098d90 con 0x7f565009b300 2024-07-17T07:16:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.181+0000 7f5647e00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 0x7f562c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:08.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.182+0000 7f5647e00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 0x7f562c0791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5638033000 tx=0x7f56380046c0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.185+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f565006e980 con 0x7f565009b300 2024-07-17T07:16:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.400+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f565006e060 con 0x7f565009b300 2024-07-17T07:16:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.403+0000 7f5646a00640 1 -- 172.21.15.44:0/2146688504 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+17719 (secure 0 0 0) 0x7f5640014030 con 0x7f565009b300 2024-07-17T07:16:08.402 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:08.402 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T07:16:05.407453+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:16:01.823652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-07-17T07:16:02.840012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 msgr2=0x7f562c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 0x7f562c0791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5638033000 tx=0x7f56380046c0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 msgr2=0x7f565014bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f564000c970 tx=0x7f564000ce30 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 shutdown_connections 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f562c076d00 0x7f562c0791c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56500cf8c0 0x7f565015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f565009b6e0 0x7f5650073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 --2- 172.21.15.44:0/2146688504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f565009b300 0x7f565014bd70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.406+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 >> 172.21.15.44:0/2146688504 conn(0x7f56500b0e80 msgr2=0x7f5650153090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.407+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 shutdown_connections 2024-07-17T07:16:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.407+0000 7f564fe00640 1 -- 172.21.15.44:0/2146688504 wait complete. 2024-07-17T07:16:08.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T07:16:08.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:16:08.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 -- 172.21.15.44:0/737572548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 msgr2=0x7fa74c153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 --2- 172.21.15.44:0/737572548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c153e40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa74c070e20 tx=0x7fa74402aa30 comp rx=0 tx=0).stop 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 -- 172.21.15.44:0/737572548 shutdown_connections 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 --2- 172.21.15.44:0/737572548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c153e40 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 --2- 172.21.15.44:0/737572548 >> v2:172.21.15.175:3300/0 conn(0x7fa74c0acb90 0x7fa74c06f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.632+0000 7fa753000640 1 --2- 172.21.15.44:0/737572548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74c15b2e0 0x7fa74c0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.633+0000 7fa753000640 1 -- 172.21.15.44:0/737572548 >> 172.21.15.44:0/737572548 conn(0x7fa74c0b0fb0 msgr2=0x7fa74c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.633+0000 7fa753000640 1 -- 172.21.15.44:0/737572548 shutdown_connections 2024-07-17T07:16:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.633+0000 7fa753000640 1 -- 172.21.15.44:0/737572548 wait complete. 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.634+0000 7fa753000640 1 Processor -- start 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.634+0000 7fa753000640 1 -- start start 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74c0acb90 0x7fa74c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74c15b2e0 0x7fa74c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c154590 con 0x7fa74c06f560 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15b900 con 0x7fa74c0acb90 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.635+0000 7fa753000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c154710 con 0x7fa74c15b2e0 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57598/0 (socket says 172.21.15.44:57598) 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 -- 172.21.15.44:0/467503738 learned_addr learned my addr 172.21.15.44:0/467503738 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 -- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74c15b2e0 msgr2=0x7fa74c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74c15b2e0 0x7fa74c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 -- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74c0acb90 msgr2=0x7fa74c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74c0acb90 0x7fa74c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 -- 172.21.15.44:0/467503738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c071a00 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.636+0000 7fa74be00640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d770 tx=0x7fa73c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.637+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c017070 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.637+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa73c009d20 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.637+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c0106c0 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.637+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c157240 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.637+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c146470 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.638+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c153e40 con 0x7fa74c06f560 2024-07-17T07:16:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.639+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fa73c01a070 con 0x7fa74c06f560 2024-07-17T07:16:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.640+0000 7fa748c00640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 0x7fa728079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.640+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7fa73c0988f0 con 0x7fa74c06f560 2024-07-17T07:16:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.641+0000 7fa743400640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 0x7fa728079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.642+0000 7fa743400640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 0x7fa728079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0b4920 tx=0x7fa734005c70 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.645+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa73c061880 con 0x7fa74c06f560 2024-07-17T07:16:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.853+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 --> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa74c06dbf0 con 0x7fa728076cd0 2024-07-17T07:16:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.869+0000 7fa748c00640 1 -- 172.21.15.44:0/467503738 <== mgr.6434 v2:172.21.15.175:6800/729654308 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7fa74c06dbf0 con 0x7fa728076cd0 2024-07-17T07:16:08.871 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 msgr2=0x7fa728079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 0x7fa728079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0b4920 tx=0x7fa734005c70 comp rx=0 tx=0).stop 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 msgr2=0x7fa74c145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d770 tx=0x7fa73c00dc30 comp rx=0 tx=0).stop 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 shutdown_connections 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fa728076cd0 0x7fa728079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74c15b2e0 0x7fa74c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74c0acb90 0x7fa74c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 --2- 172.21.15.44:0/467503738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa74c06f560 0x7fa74c145f30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 >> 172.21.15.44:0/467503738 conn(0x7fa74c0b0fb0 msgr2=0x7fa74c159290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 shutdown_connections 2024-07-17T07:16:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:08.874+0000 7fa753000640 1 -- 172.21.15.44:0/467503738 wait complete. 2024-07-17T07:16:08.892 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:16:08.070065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":296,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2487,"ondisk_log_size":2487,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":348956,"kb_used_data":16228,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754625764,"statfs":{"total":773094113280,"available":772736782336,"internally_reserved":0,"allocated":16617472,"data_stored":9375692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.711541"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414642+0000","last_change":"2024-07-17T07:16:04.426483+0000","last_active":"2024-07-17T07:16:05.414642+0000","last_peered":"2024-07-17T07:16:05.414642+0000","last_clean":"2024-07-17T07:16:05.414642+0000","last_became_active":"2024-07-17T07:16:04.426158+0000","last_became_peered":"2024-07-17T07:16:04.426158+0000","last_unstale":"2024-07-17T07:16:05.414642+0000","last_undegraded":"2024-07-17T07:16:05.414642+0000","last_fullsized":"2024-07-17T07:16:05.414642+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:13:50.539011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431942+0000","last_change":"2024-07-17T07:16:04.419936+0000","last_active":"2024-07-17T07:16:05.431942+0000","last_peered":"2024-07-17T07:16:05.431942+0000","last_clean":"2024-07-17T07:16:05.431942+0000","last_became_active":"2024-07-17T07:16:04.419551+0000","last_became_peered":"2024-07-17T07:16:04.419551+0000","last_unstale":"2024-07-17T07:16:05.431942+0000","last_undegraded":"2024-07-17T07:16:05.431942+0000","last_fullsized":"2024-07-17T07:16:05.431942+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:01.507866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419378+0000","last_change":"2024-07-17T07:16:04.432493+0000","last_active":"2024-07-17T07:16:05.419378+0000","last_peered":"2024-07-17T07:16:05.419378+0000","last_clean":"2024-07-17T07:16:05.419378+0000","last_became_active":"2024-07-17T07:16:04.430398+0000","last_became_peered":"2024-07-17T07:16:04.430398+0000","last_unstale":"2024-07-17T07:16:05.419378+0000","last_undegraded":"2024-07-17T07:16:05.419378+0000","last_fullsized":"2024-07-17T07:16:05.419378+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:30.086416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.430828+0000","last_change":"2024-07-17T07:16:04.434360+0000","last_active":"2024-07-17T07:16:05.430828+0000","last_peered":"2024-07-17T07:16:05.430828+0000","last_clean":"2024-07-17T07:16:05.430828+0000","last_became_active":"2024-07-17T07:16:04.434082+0000","last_became_peered":"2024-07-17T07:16:04.434082+0000","last_unstale":"2024-07-17T07:16:05.430828+0000","last_undegraded":"2024-07-17T07:16:05.430828+0000","last_fullsized":"2024-07-17T07:16:05.430828+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.422131+0000","last_change":"2024-07-17T07:16:04.418222+0000","last_active":"2024-07-17T07:16:04.422131+0000","last_peered":"2024-07-17T07:16:04.422131+0000","last_clean":"2024-07-17T07:16:04.422131+0000","last_became_active":"2024-07-17T07:16:04.417800+0000","last_became_peered":"2024-07-17T07:16:04.417800+0000","last_unstale":"2024-07-17T07:16:04.422131+0000","last_undegraded":"2024-07-17T07:16:04.422131+0000","last_fullsized":"2024-07-17T07:16:04.422131+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:54.121775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772130+0000","last_change":"2024-07-17T07:16:04.431021+0000","last_active":"2024-07-17T07:16:05.772130+0000","last_peered":"2024-07-17T07:16:05.772130+0000","last_clean":"2024-07-17T07:16:05.772130+0000","last_became_active":"2024-07-17T07:16:04.430712+0000","last_became_peered":"2024-07-17T07:16:04.430712+0000","last_unstale":"2024-07-17T07:16:05.772130+0000","last_undegraded":"2024-07-17T07:16:05.772130+0000","last_fullsized":"2024-07-17T07:16:05.772130+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:44.062549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447131+0000","last_change":"2024-07-17T07:16:04.427861+0000","last_active":"2024-07-17T07:16:04.447131+0000","last_peered":"2024-07-17T07:16:04.447131+0000","last_clean":"2024-07-17T07:16:04.447131+0000","last_became_active":"2024-07-17T07:16:04.427574+0000","last_became_peered":"2024-07-17T07:16:04.427574+0000","last_unstale":"2024-07-17T07:16:04.447131+0000","last_undegraded":"2024-07-17T07:16:04.447131+0000","last_fullsized":"2024-07-17T07:16:04.447131+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:18.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417866+0000","last_change":"2024-07-17T07:16:04.437314+0000","last_active":"2024-07-17T07:16:05.417866+0000","last_peered":"2024-07-17T07:16:05.417866+0000","last_clean":"2024-07-17T07:16:05.417866+0000","last_became_active":"2024-07-17T07:16:04.436994+0000","last_became_peered":"2024-07-17T07:16:04.436994+0000","last_unstale":"2024-07-17T07:16:05.417866+0000","last_undegraded":"2024-07-17T07:16:05.417866+0000","last_fullsized":"2024-07-17T07:16:05.417866+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:30.257635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772135+0000","last_change":"2024-07-17T07:16:04.429212+0000","last_active":"2024-07-17T07:16:05.772135+0000","last_peered":"2024-07-17T07:16:05.772135+0000","last_clean":"2024-07-17T07:16:05.772135+0000","last_became_active":"2024-07-17T07:16:04.428916+0000","last_became_peered":"2024-07-17T07:16:04.428916+0000","last_unstale":"2024-07-17T07:16:05.772135+0000","last_undegraded":"2024-07-17T07:16:05.772135+0000","last_fullsized":"2024-07-17T07:16:05.772135+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:13.828577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678915+0000","last_change":"2024-07-17T07:16:04.443400+0000","last_active":"2024-07-17T07:16:05.678915+0000","last_peered":"2024-07-17T07:16:05.678915+0000","last_clean":"2024-07-17T07:16:05.678915+0000","last_became_active":"2024-07-17T07:16:04.441734+0000","last_became_peered":"2024-07-17T07:16:04.441734+0000","last_unstale":"2024-07-17T07:16:05.678915+0000","last_undegraded":"2024-07-17T07:16:05.678915+0000","last_fullsized":"2024-07-17T07:16:05.678915+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:46.767537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431073+0000","last_change":"2024-07-17T07:16:04.431402+0000","last_active":"2024-07-17T07:16:05.431073+0000","last_peered":"2024-07-17T07:16:05.431073+0000","last_clean":"2024-07-17T07:16:05.431073+0000","last_became_active":"2024-07-17T07:16:04.431120+0000","last_became_peered":"2024-07-17T07:16:04.431120+0000","last_unstale":"2024-07-17T07:16:05.431073+0000","last_undegraded":"2024-07-17T07:16:05.431073+0000","last_fullsized":"2024-07-17T07:16:05.431073+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:40:08.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414692+0000","last_change":"2024-07-17T07:16:04.409086+0000","last_active":"2024-07-17T07:16:05.414692+0000","last_peered":"2024-07-17T07:16:05.414692+0000","last_clean":"2024-07-17T07:16:05.414692+0000","last_became_active":"2024-07-17T07:16:04.408623+0000","last_became_peered":"2024-07-17T07:16:04.408623+0000","last_unstale":"2024-07-17T07:16:05.414692+0000","last_undegraded":"2024-07-17T07:16:05.414692+0000","last_fullsized":"2024-07-17T07:16:05.414692+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:33:03.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772683+0000","last_change":"2024-07-17T07:16:04.432591+0000","last_active":"2024-07-17T07:16:05.772683+0000","last_peered":"2024-07-17T07:16:05.772683+0000","last_clean":"2024-07-17T07:16:05.772683+0000","last_became_active":"2024-07-17T07:16:04.430977+0000","last_became_peered":"2024-07-17T07:16:04.430977+0000","last_unstale":"2024-07-17T07:16:05.772683+0000","last_undegraded":"2024-07-17T07:16:05.772683+0000","last_fullsized":"2024-07-17T07:16:05.772683+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:35:19.791752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414019+0000","last_change":"2024-07-17T07:16:04.418037+0000","last_active":"2024-07-17T07:16:05.414019+0000","last_peered":"2024-07-17T07:16:05.414019+0000","last_clean":"2024-07-17T07:16:05.414019+0000","last_became_active":"2024-07-17T07:16:04.414615+0000","last_became_peered":"2024-07-17T07:16:04.414615+0000","last_unstale":"2024-07-17T07:16:05.414019+0000","last_undegraded":"2024-07-17T07:16:05.414019+0000","last_fullsized":"2024-07-17T07:16:05.414019+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:10.844648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413745+0000","last_change":"2024-07-17T07:16:04.414313+0000","last_active":"2024-07-17T07:16:05.413745+0000","last_peered":"2024-07-17T07:16:05.413745+0000","last_clean":"2024-07-17T07:16:05.413745+0000","last_became_active":"2024-07-17T07:16:04.413869+0000","last_became_peered":"2024-07-17T07:16:04.413869+0000","last_unstale":"2024-07-17T07:16:05.413745+0000","last_undegraded":"2024-07-17T07:16:05.413745+0000","last_fullsized":"2024-07-17T07:16:05.413745+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:29.493082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772852+0000","last_change":"2024-07-17T07:16:04.427278+0000","last_active":"2024-07-17T07:16:05.772852+0000","last_peered":"2024-07-17T07:16:05.772852+0000","last_clean":"2024-07-17T07:16:05.772852+0000","last_became_active":"2024-07-17T07:16:04.426379+0000","last_became_peered":"2024-07-17T07:16:04.426379+0000","last_unstale":"2024-07-17T07:16:05.772852+0000","last_undegraded":"2024-07-17T07:16:05.772852+0000","last_fullsized":"2024-07-17T07:16:05.772852+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:41.650990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772349+0000","last_change":"2024-07-17T07:16:04.431227+0000","last_active":"2024-07-17T07:16:05.772349+0000","last_peered":"2024-07-17T07:16:05.772349+0000","last_clean":"2024-07-17T07:16:05.772349+0000","last_became_active":"2024-07-17T07:16:04.430575+0000","last_became_peered":"2024-07-17T07:16:04.430575+0000","last_unstale":"2024-07-17T07:16:05.772349+0000","last_undegraded":"2024-07-17T07:16:05.772349+0000","last_fullsized":"2024-07-17T07:16:05.772349+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:25.911907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.427229+0000","last_change":"2024-07-17T07:16:04.420003+0000","last_active":"2024-07-17T07:16:04.427229+0000","last_peered":"2024-07-17T07:16:04.427229+0000","last_clean":"2024-07-17T07:16:04.427229+0000","last_became_active":"2024-07-17T07:16:04.419348+0000","last_became_peered":"2024-07-17T07:16:04.419348+0000","last_unstale":"2024-07-17T07:16:04.427229+0000","last_undegraded":"2024-07-17T07:16:04.427229+0000","last_fullsized":"2024-07-17T07:16:04.427229+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:15.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439714+0000","last_change":"2024-07-17T07:16:04.422546+0000","last_active":"2024-07-17T07:16:04.439714+0000","last_peered":"2024-07-17T07:16:04.439714+0000","last_clean":"2024-07-17T07:16:04.439714+0000","last_became_active":"2024-07-17T07:16:04.422234+0000","last_became_peered":"2024-07-17T07:16:04.422234+0000","last_unstale":"2024-07-17T07:16:04.439714+0000","last_undegraded":"2024-07-17T07:16:04.439714+0000","last_fullsized":"2024-07-17T07:16:04.439714+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.085733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.417711+0000","last_change":"2024-07-17T07:16:04.413812+0000","last_active":"2024-07-17T07:16:04.417711+0000","last_peered":"2024-07-17T07:16:04.417711+0000","last_clean":"2024-07-17T07:16:04.417711+0000","last_became_active":"2024-07-17T07:16:04.413263+0000","last_became_peered":"2024-07-17T07:16:04.413263+0000","last_unstale":"2024-07-17T07:16:04.417711+0000","last_undegraded":"2024-07-17T07:16:04.417711+0000","last_fullsized":"2024-07-17T07:16:04.417711+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:44.234690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431170+0000","last_change":"2024-07-17T07:16:04.435133+0000","last_active":"2024-07-17T07:16:05.431170+0000","last_peered":"2024-07-17T07:16:05.431170+0000","last_clean":"2024-07-17T07:16:05.431170+0000","last_became_active":"2024-07-17T07:16:04.434789+0000","last_became_peered":"2024-07-17T07:16:04.434789+0000","last_unstale":"2024-07-17T07:16:05.431170+0000","last_undegraded":"2024-07-17T07:16:05.431170+0000","last_fullsized":"2024-07-17T07:16:05.431170+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:58.239590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414346+0000","last_change":"2024-07-17T07:16:04.420589+0000","last_active":"2024-07-17T07:16:05.414346+0000","last_peered":"2024-07-17T07:16:05.414346+0000","last_clean":"2024-07-17T07:16:05.414346+0000","last_became_active":"2024-07-17T07:16:04.420031+0000","last_became_peered":"2024-07-17T07:16:04.420031+0000","last_unstale":"2024-07-17T07:16:05.414346+0000","last_undegraded":"2024-07-17T07:16:05.414346+0000","last_fullsized":"2024-07-17T07:16:05.414346+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:25.512963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.430542+0000","last_change":"2024-07-17T07:16:04.441190+0000","last_active":"2024-07-17T07:16:05.430542+0000","last_peered":"2024-07-17T07:16:05.430542+0000","last_clean":"2024-07-17T07:16:05.430542+0000","last_became_active":"2024-07-17T07:16:04.440906+0000","last_became_peered":"2024-07-17T07:16:04.440906+0000","last_unstale":"2024-07-17T07:16:05.430542+0000","last_undegraded":"2024-07-17T07:16:05.430542+0000","last_fullsized":"2024-07-17T07:16:05.430542+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:15.981756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772982+0000","last_change":"2024-07-17T07:16:04.429379+0000","last_active":"2024-07-17T07:16:05.772982+0000","last_peered":"2024-07-17T07:16:05.772982+0000","last_clean":"2024-07-17T07:16:05.772982+0000","last_became_active":"2024-07-17T07:16:04.429090+0000","last_became_peered":"2024-07-17T07:16:04.429090+0000","last_unstale":"2024-07-17T07:16:05.772982+0000","last_undegraded":"2024-07-17T07:16:05.772982+0000","last_fullsized":"2024-07-17T07:16:05.772982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:32.084265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420661+0000","last_change":"2024-07-17T07:16:04.441263+0000","last_active":"2024-07-17T07:16:05.420661+0000","last_peered":"2024-07-17T07:16:05.420661+0000","last_clean":"2024-07-17T07:16:05.420661+0000","last_became_active":"2024-07-17T07:16:04.440996+0000","last_became_peered":"2024-07-17T07:16:04.440996+0000","last_unstale":"2024-07-17T07:16:05.420661+0000","last_undegraded":"2024-07-17T07:16:05.420661+0000","last_fullsized":"2024-07-17T07:16:05.420661+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:13:00.887279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418864+0000","last_change":"2024-07-17T07:16:04.443050+0000","last_active":"2024-07-17T07:16:05.418864+0000","last_peered":"2024-07-17T07:16:05.418864+0000","last_clean":"2024-07-17T07:16:05.418864+0000","last_became_active":"2024-07-17T07:16:04.442505+0000","last_became_peered":"2024-07-17T07:16:04.442505+0000","last_unstale":"2024-07-17T07:16:05.418864+0000","last_undegraded":"2024-07-17T07:16:05.418864+0000","last_fullsized":"2024-07-17T07:16:05.418864+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:29.128517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447266+0000","last_change":"2024-07-17T07:16:04.431944+0000","last_active":"2024-07-17T07:16:04.447266+0000","last_peered":"2024-07-17T07:16:04.447266+0000","last_clean":"2024-07-17T07:16:04.447266+0000","last_became_active":"2024-07-17T07:16:04.430781+0000","last_became_peered":"2024-07-17T07:16:04.430781+0000","last_unstale":"2024-07-17T07:16:04.447266+0000","last_undegraded":"2024-07-17T07:16:04.447266+0000","last_fullsized":"2024-07-17T07:16:04.447266+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:33.666282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.679086+0000","last_change":"2024-07-17T07:16:04.433922+0000","last_active":"2024-07-17T07:16:05.679086+0000","last_peered":"2024-07-17T07:16:05.679086+0000","last_clean":"2024-07-17T07:16:05.679086+0000","last_became_active":"2024-07-17T07:16:04.433640+0000","last_became_peered":"2024-07-17T07:16:04.433640+0000","last_unstale":"2024-07-17T07:16:05.679086+0000","last_undegraded":"2024-07-17T07:16:05.679086+0000","last_fullsized":"2024-07-17T07:16:05.679086+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:41.157337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420010+0000","last_change":"2024-07-17T07:16:04.433532+0000","last_active":"2024-07-17T07:16:05.420010+0000","last_peered":"2024-07-17T07:16:05.420010+0000","last_clean":"2024-07-17T07:16:05.420010+0000","last_became_active":"2024-07-17T07:16:04.433108+0000","last_became_peered":"2024-07-17T07:16:04.433108+0000","last_unstale":"2024-07-17T07:16:05.420010+0000","last_undegraded":"2024-07-17T07:16:05.420010+0000","last_fullsized":"2024-07-17T07:16:05.420010+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.039380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419273+0000","last_change":"2024-07-17T07:16:04.441337+0000","last_active":"2024-07-17T07:16:05.419273+0000","last_peered":"2024-07-17T07:16:05.419273+0000","last_clean":"2024-07-17T07:16:05.419273+0000","last_became_active":"2024-07-17T07:16:04.440942+0000","last_became_peered":"2024-07-17T07:16:04.440942+0000","last_unstale":"2024-07-17T07:16:05.419273+0000","last_undegraded":"2024-07-17T07:16:05.419273+0000","last_fullsized":"2024-07-17T07:16:05.419273+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:25.911907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420052+0000","last_change":"2024-07-17T07:16:04.437678+0000","last_active":"2024-07-17T07:16:05.420052+0000","last_peered":"2024-07-17T07:16:05.420052+0000","last_clean":"2024-07-17T07:16:05.420052+0000","last_became_active":"2024-07-17T07:16:04.431166+0000","last_became_peered":"2024-07-17T07:16:04.431166+0000","last_unstale":"2024-07-17T07:16:05.420052+0000","last_undegraded":"2024-07-17T07:16:05.420052+0000","last_fullsized":"2024-07-17T07:16:05.420052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:04.308976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678537+0000","last_change":"2024-07-17T07:16:04.435037+0000","last_active":"2024-07-17T07:16:05.678537+0000","last_peered":"2024-07-17T07:16:05.678537+0000","last_clean":"2024-07-17T07:16:05.678537+0000","last_became_active":"2024-07-17T07:16:04.433246+0000","last_became_peered":"2024-07-17T07:16:04.433246+0000","last_unstale":"2024-07-17T07:16:05.678537+0000","last_undegraded":"2024-07-17T07:16:05.678537+0000","last_fullsized":"2024-07-17T07:16:05.678537+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:44.323473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418930+0000","last_change":"2024-07-17T07:16:04.429320+0000","last_active":"2024-07-17T07:16:05.418930+0000","last_peered":"2024-07-17T07:16:05.418930+0000","last_clean":"2024-07-17T07:16:05.418930+0000","last_became_active":"2024-07-17T07:16:04.428522+0000","last_became_peered":"2024-07-17T07:16:04.428522+0000","last_unstale":"2024-07-17T07:16:05.418930+0000","last_undegraded":"2024-07-17T07:16:05.418930+0000","last_fullsized":"2024-07-17T07:16:05.418930+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:22.929008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"154'4","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420552+0000","last_change":"2024-07-17T07:16:03.395736+0000","last_active":"2024-07-17T07:16:05.420552+0000","last_peered":"2024-07-17T07:16:05.420552+0000","last_clean":"2024-07-17T07:16:05.420552+0000","last_became_active":"2024-07-17T07:16:03.395400+0000","last_became_peered":"2024-07-17T07:16:03.395400+0000","last_unstale":"2024-07-17T07:16:05.420552+0000","last_undegraded":"2024-07-17T07:16:05.420552+0000","last_fullsized":"2024-07-17T07:16:05.420552+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:23.408953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414016+0000","last_change":"2024-07-17T07:16:04.421804+0000","last_active":"2024-07-17T07:16:05.414016+0000","last_peered":"2024-07-17T07:16:05.414016+0000","last_clean":"2024-07-17T07:16:05.414016+0000","last_became_active":"2024-07-17T07:16:04.421339+0000","last_became_peered":"2024-07-17T07:16:04.421339+0000","last_unstale":"2024-07-17T07:16:05.414016+0000","last_undegraded":"2024-07-17T07:16:05.414016+0000","last_fullsized":"2024-07-17T07:16:05.414016+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:55.646028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.443414+0000","last_change":"2024-07-17T07:16:03.404827+0000","last_active":"2024-07-17T07:16:04.443414+0000","last_peered":"2024-07-17T07:16:04.443414+0000","last_clean":"2024-07-17T07:16:04.443414+0000","last_became_active":"2024-07-17T07:16:03.404036+0000","last_became_peered":"2024-07-17T07:16:03.404036+0000","last_unstale":"2024-07-17T07:16:04.443414+0000","last_undegraded":"2024-07-17T07:16:04.443414+0000","last_fullsized":"2024-07-17T07:16:04.443414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:59.836967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431198+0000","last_change":"2024-07-17T07:16:04.437543+0000","last_active":"2024-07-17T07:16:05.431198+0000","last_peered":"2024-07-17T07:16:05.431198+0000","last_clean":"2024-07-17T07:16:05.431198+0000","last_became_active":"2024-07-17T07:16:04.437270+0000","last_became_peered":"2024-07-17T07:16:04.437270+0000","last_unstale":"2024-07-17T07:16:05.431198+0000","last_undegraded":"2024-07-17T07:16:05.431198+0000","last_fullsized":"2024-07-17T07:16:05.431198+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:38.545638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.449973+0000","last_change":"2024-07-17T07:16:03.404705+0000","last_active":"2024-07-17T07:16:04.449973+0000","last_peered":"2024-07-17T07:16:04.449973+0000","last_clean":"2024-07-17T07:16:04.449973+0000","last_became_active":"2024-07-17T07:16:03.403211+0000","last_became_peered":"2024-07-17T07:16:03.403211+0000","last_unstale":"2024-07-17T07:16:04.449973+0000","last_undegraded":"2024-07-17T07:16:04.449973+0000","last_fullsized":"2024-07-17T07:16:04.449973+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:07.796007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414367+0000","last_change":"2024-07-17T07:16:04.443170+0000","last_active":"2024-07-17T07:16:05.414367+0000","last_peered":"2024-07-17T07:16:05.414367+0000","last_clean":"2024-07-17T07:16:05.414367+0000","last_became_active":"2024-07-17T07:16:04.442375+0000","last_became_peered":"2024-07-17T07:16:04.442375+0000","last_unstale":"2024-07-17T07:16:05.414367+0000","last_undegraded":"2024-07-17T07:16:05.414367+0000","last_fullsized":"2024-07-17T07:16:05.414367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.450887+0000","last_change":"2024-07-17T07:16:03.402762+0000","last_active":"2024-07-17T07:16:04.450887+0000","last_peered":"2024-07-17T07:16:04.450887+0000","last_clean":"2024-07-17T07:16:04.450887+0000","last_became_active":"2024-07-17T07:16:03.402442+0000","last_became_peered":"2024-07-17T07:16:03.402442+0000","last_unstale":"2024-07-17T07:16:04.450887+0000","last_undegraded":"2024-07-17T07:16:04.450887+0000","last_fullsized":"2024-07-17T07:16:04.450887+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:31.163924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421317+0000","last_change":"2024-07-17T07:16:04.430860+0000","last_active":"2024-07-17T07:16:05.421317+0000","last_peered":"2024-07-17T07:16:05.421317+0000","last_clean":"2024-07-17T07:16:05.421317+0000","last_became_active":"2024-07-17T07:16:04.430510+0000","last_became_peered":"2024-07-17T07:16:04.430510+0000","last_unstale":"2024-07-17T07:16:05.421317+0000","last_undegraded":"2024-07-17T07:16:05.421317+0000","last_fullsized":"2024-07-17T07:16:05.421317+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:43.631545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420530+0000","last_change":"2024-07-17T07:16:03.401409+0000","last_active":"2024-07-17T07:16:05.420530+0000","last_peered":"2024-07-17T07:16:05.420530+0000","last_clean":"2024-07-17T07:16:05.420530+0000","last_became_active":"2024-07-17T07:16:03.401112+0000","last_became_peered":"2024-07-17T07:16:03.401112+0000","last_unstale":"2024-07-17T07:16:05.420530+0000","last_undegraded":"2024-07-17T07:16:05.420530+0000","last_fullsized":"2024-07-17T07:16:05.420530+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:30.485592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421198+0000","last_change":"2024-07-17T07:16:04.430399+0000","last_active":"2024-07-17T07:16:05.421198+0000","last_peered":"2024-07-17T07:16:05.421198+0000","last_clean":"2024-07-17T07:16:05.421198+0000","last_became_active":"2024-07-17T07:16:04.430131+0000","last_became_peered":"2024-07-17T07:16:04.430131+0000","last_unstale":"2024-07-17T07:16:05.421198+0000","last_undegraded":"2024-07-17T07:16:05.421198+0000","last_fullsized":"2024-07-17T07:16:05.421198+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:11.631473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.442403+0000","last_change":"2024-07-17T07:16:03.401474+0000","last_active":"2024-07-17T07:16:04.442403+0000","last_peered":"2024-07-17T07:16:04.442403+0000","last_clean":"2024-07-17T07:16:04.442403+0000","last_became_active":"2024-07-17T07:16:03.401157+0000","last_became_peered":"2024-07-17T07:16:03.401157+0000","last_unstale":"2024-07-17T07:16:04.442403+0000","last_undegraded":"2024-07-17T07:16:04.442403+0000","last_fullsized":"2024-07-17T07:16:04.442403+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:48.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.428671+0000","last_change":"2024-07-17T07:16:04.426091+0000","last_active":"2024-07-17T07:16:04.428671+0000","last_peered":"2024-07-17T07:16:04.428671+0000","last_clean":"2024-07-17T07:16:04.428671+0000","last_became_active":"2024-07-17T07:16:04.425749+0000","last_became_peered":"2024-07-17T07:16:04.425749+0000","last_unstale":"2024-07-17T07:16:04.428671+0000","last_undegraded":"2024-07-17T07:16:04.428671+0000","last_fullsized":"2024-07-17T07:16:04.428671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:57.981566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.771929+0000","last_change":"2024-07-17T07:16:03.407154+0000","last_active":"2024-07-17T07:16:05.771929+0000","last_peered":"2024-07-17T07:16:05.771929+0000","last_clean":"2024-07-17T07:16:05.771929+0000","last_became_active":"2024-07-17T07:16:03.406825+0000","last_became_peered":"2024-07-17T07:16:03.406825+0000","last_unstale":"2024-07-17T07:16:05.771929+0000","last_undegraded":"2024-07-17T07:16:05.771929+0000","last_fullsized":"2024-07-17T07:16:05.771929+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:42.550236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447663+0000","last_change":"2024-07-17T07:16:04.437105+0000","last_active":"2024-07-17T07:16:04.447663+0000","last_peered":"2024-07-17T07:16:04.447663+0000","last_clean":"2024-07-17T07:16:04.447663+0000","last_became_active":"2024-07-17T07:16:04.436759+0000","last_became_peered":"2024-07-17T07:16:04.436759+0000","last_unstale":"2024-07-17T07:16:04.447663+0000","last_undegraded":"2024-07-17T07:16:04.447663+0000","last_fullsized":"2024-07-17T07:16:04.447663+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:01.263102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418233+0000","last_change":"2024-07-17T07:16:03.403575+0000","last_active":"2024-07-17T07:16:05.418233+0000","last_peered":"2024-07-17T07:16:05.418233+0000","last_clean":"2024-07-17T07:16:05.418233+0000","last_became_active":"2024-07-17T07:16:03.403259+0000","last_became_peered":"2024-07-17T07:16:03.403259+0000","last_unstale":"2024-07-17T07:16:05.418233+0000","last_undegraded":"2024-07-17T07:16:05.418233+0000","last_fullsized":"2024-07-17T07:16:05.418233+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:52.907978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413744+0000","last_change":"2024-07-17T07:16:04.428753+0000","last_active":"2024-07-17T07:16:05.413744+0000","last_peered":"2024-07-17T07:16:05.413744+0000","last_clean":"2024-07-17T07:16:05.413744+0000","last_became_active":"2024-07-17T07:16:04.428441+0000","last_became_peered":"2024-07-17T07:16:04.428441+0000","last_unstale":"2024-07-17T07:16:05.413744+0000","last_undegraded":"2024-07-17T07:16:05.413744+0000","last_fullsized":"2024-07-17T07:16:05.413744+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:33.076840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431477+0000","last_change":"2024-07-17T07:16:03.407565+0000","last_active":"2024-07-17T07:16:05.431477+0000","last_peered":"2024-07-17T07:16:05.431477+0000","last_clean":"2024-07-17T07:16:05.431477+0000","last_became_active":"2024-07-17T07:16:03.407247+0000","last_became_peered":"2024-07-17T07:16:03.407247+0000","last_unstale":"2024-07-17T07:16:05.431477+0000","last_undegraded":"2024-07-17T07:16:05.431477+0000","last_fullsized":"2024-07-17T07:16:05.431477+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:37.273651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772279+0000","last_change":"2024-07-17T07:16:04.441854+0000","last_active":"2024-07-17T07:16:05.772279+0000","last_peered":"2024-07-17T07:16:05.772279+0000","last_clean":"2024-07-17T07:16:05.772279+0000","last_became_active":"2024-07-17T07:16:04.441559+0000","last_became_peered":"2024-07-17T07:16:04.441559+0000","last_unstale":"2024-07-17T07:16:05.772279+0000","last_undegraded":"2024-07-17T07:16:05.772279+0000","last_fullsized":"2024-07-17T07:16:05.772279+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:03.174116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414215+0000","last_change":"2024-07-17T07:16:03.401502+0000","last_active":"2024-07-17T07:16:05.414215+0000","last_peered":"2024-07-17T07:16:05.414215+0000","last_clean":"2024-07-17T07:16:05.414215+0000","last_became_active":"2024-07-17T07:16:03.401219+0000","last_became_peered":"2024-07-17T07:16:03.401219+0000","last_unstale":"2024-07-17T07:16:05.414215+0000","last_undegraded":"2024-07-17T07:16:05.414215+0000","last_fullsized":"2024-07-17T07:16:05.414215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:06.711274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419593+0000","last_change":"2024-07-17T07:16:04.426331+0000","last_active":"2024-07-17T07:16:05.419593+0000","last_peered":"2024-07-17T07:16:05.419593+0000","last_clean":"2024-07-17T07:16:05.419593+0000","last_became_active":"2024-07-17T07:16:04.425812+0000","last_became_peered":"2024-07-17T07:16:04.425812+0000","last_unstale":"2024-07-17T07:16:05.419593+0000","last_undegraded":"2024-07-17T07:16:05.419593+0000","last_fullsized":"2024-07-17T07:16:05.419593+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:35:19.791752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.450401+0000","last_change":"2024-07-17T07:16:03.406006+0000","last_active":"2024-07-17T07:16:04.450401+0000","last_peered":"2024-07-17T07:16:04.450401+0000","last_clean":"2024-07-17T07:16:04.450401+0000","last_became_active":"2024-07-17T07:16:03.404727+0000","last_became_peered":"2024-07-17T07:16:03.404727+0000","last_unstale":"2024-07-17T07:16:04.450401+0000","last_undegraded":"2024-07-17T07:16:04.450401+0000","last_fullsized":"2024-07-17T07:16:04.450401+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:29:20.909658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420010+0000","last_change":"2024-07-17T07:16:04.431989+0000","last_active":"2024-07-17T07:16:05.420010+0000","last_peered":"2024-07-17T07:16:05.420010+0000","last_clean":"2024-07-17T07:16:05.420010+0000","last_became_active":"2024-07-17T07:16:04.431596+0000","last_became_peered":"2024-07-17T07:16:04.431596+0000","last_unstale":"2024-07-17T07:16:05.420010+0000","last_undegraded":"2024-07-17T07:16:05.420010+0000","last_fullsized":"2024-07-17T07:16:05.420010+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:32.084265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"154'6","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.453193+0000","last_change":"2024-07-17T07:16:03.403387+0000","last_active":"2024-07-17T07:16:04.453193+0000","last_peered":"2024-07-17T07:16:04.453193+0000","last_clean":"2024-07-17T07:16:04.453193+0000","last_became_active":"2024-07-17T07:16:03.403122+0000","last_became_peered":"2024-07-17T07:16:03.403122+0000","last_unstale":"2024-07-17T07:16:04.453193+0000","last_undegraded":"2024-07-17T07:16:04.453193+0000","last_fullsized":"2024-07-17T07:16:04.453193+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:47.591234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.679307+0000","last_change":"2024-07-17T07:16:04.427357+0000","last_active":"2024-07-17T07:16:05.679307+0000","last_peered":"2024-07-17T07:16:05.679307+0000","last_clean":"2024-07-17T07:16:05.679307+0000","last_became_active":"2024-07-17T07:16:04.425814+0000","last_became_peered":"2024-07-17T07:16:04.425814+0000","last_unstale":"2024-07-17T07:16:05.679307+0000","last_undegraded":"2024-07-17T07:16:05.679307+0000","last_fullsized":"2024-07-17T07:16:05.679307+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:40.190741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413986+0000","last_change":"2024-07-17T07:16:03.395142+0000","last_active":"2024-07-17T07:16:05.413986+0000","last_peered":"2024-07-17T07:16:05.413986+0000","last_clean":"2024-07-17T07:16:05.413986+0000","last_became_active":"2024-07-17T07:16:03.394374+0000","last_became_peered":"2024-07-17T07:16:03.394374+0000","last_unstale":"2024-07-17T07:16:05.413986+0000","last_undegraded":"2024-07-17T07:16:05.413986+0000","last_fullsized":"2024-07-17T07:16:05.413986+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:34:38.779860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439570+0000","last_change":"2024-07-17T07:16:04.424235+0000","last_active":"2024-07-17T07:16:04.439570+0000","last_peered":"2024-07-17T07:16:04.439570+0000","last_clean":"2024-07-17T07:16:04.439570+0000","last_became_active":"2024-07-17T07:16:04.423885+0000","last_became_peered":"2024-07-17T07:16:04.423885+0000","last_unstale":"2024-07-17T07:16:04.439570+0000","last_undegraded":"2024-07-17T07:16:04.439570+0000","last_fullsized":"2024-07-17T07:16:04.439570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:14.345455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417898+0000","last_change":"2024-07-17T07:16:03.397100+0000","last_active":"2024-07-17T07:16:05.417898+0000","last_peered":"2024-07-17T07:16:05.417898+0000","last_clean":"2024-07-17T07:16:05.417898+0000","last_became_active":"2024-07-17T07:16:03.396805+0000","last_became_peered":"2024-07-17T07:16:03.396805+0000","last_unstale":"2024-07-17T07:16:05.417898+0000","last_undegraded":"2024-07-17T07:16:05.417898+0000","last_fullsized":"2024-07-17T07:16:05.417898+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:23.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.442566+0000","last_change":"2024-07-17T07:16:04.435956+0000","last_active":"2024-07-17T07:16:04.442566+0000","last_peered":"2024-07-17T07:16:04.442566+0000","last_clean":"2024-07-17T07:16:04.442566+0000","last_became_active":"2024-07-17T07:16:04.435646+0000","last_became_peered":"2024-07-17T07:16:04.435646+0000","last_unstale":"2024-07-17T07:16:04.442566+0000","last_undegraded":"2024-07-17T07:16:04.442566+0000","last_fullsized":"2024-07-17T07:16:04.442566+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:27.867367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"154'8","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.773010+0000","last_change":"2024-07-17T07:16:03.395816+0000","last_active":"2024-07-17T07:16:05.773010+0000","last_peered":"2024-07-17T07:16:05.773010+0000","last_clean":"2024-07-17T07:16:05.773010+0000","last_became_active":"2024-07-17T07:16:03.395526+0000","last_became_peered":"2024-07-17T07:16:03.395526+0000","last_unstale":"2024-07-17T07:16:05.773010+0000","last_undegraded":"2024-07-17T07:16:05.773010+0000","last_fullsized":"2024-07-17T07:16:05.773010+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:03.033041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418127+0000","last_change":"2024-07-17T07:16:04.430043+0000","last_active":"2024-07-17T07:16:05.418127+0000","last_peered":"2024-07-17T07:16:05.418127+0000","last_clean":"2024-07-17T07:16:05.418127+0000","last_became_active":"2024-07-17T07:16:04.429499+0000","last_became_peered":"2024-07-17T07:16:04.429499+0000","last_unstale":"2024-07-17T07:16:05.418127+0000","last_undegraded":"2024-07-17T07:16:05.418127+0000","last_fullsized":"2024-07-17T07:16:05.418127+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:03.174116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431467+0000","last_change":"2024-07-17T07:16:03.401423+0000","last_active":"2024-07-17T07:16:05.431467+0000","last_peered":"2024-07-17T07:16:05.431467+0000","last_clean":"2024-07-17T07:16:05.431467+0000","last_became_active":"2024-07-17T07:16:03.401099+0000","last_became_peered":"2024-07-17T07:16:03.401099+0000","last_unstale":"2024-07-17T07:16:05.431467+0000","last_undegraded":"2024-07-17T07:16:05.431467+0000","last_fullsized":"2024-07-17T07:16:05.431467+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:40.281264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439437+0000","last_change":"2024-07-17T07:16:04.417937+0000","last_active":"2024-07-17T07:16:04.439437+0000","last_peered":"2024-07-17T07:16:04.439437+0000","last_clean":"2024-07-17T07:16:04.439437+0000","last_became_active":"2024-07-17T07:16:04.412375+0000","last_became_peered":"2024-07-17T07:16:04.412375+0000","last_unstale":"2024-07-17T07:16:04.439437+0000","last_undegraded":"2024-07-17T07:16:04.439437+0000","last_fullsized":"2024-07-17T07:16:04.439437+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:07.122238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419148+0000","last_change":"2024-07-17T07:16:03.397323+0000","last_active":"2024-07-17T07:16:05.419148+0000","last_peered":"2024-07-17T07:16:05.419148+0000","last_clean":"2024-07-17T07:16:05.419148+0000","last_became_active":"2024-07-17T07:16:03.397027+0000","last_became_peered":"2024-07-17T07:16:03.397027+0000","last_unstale":"2024-07-17T07:16:05.419148+0000","last_undegraded":"2024-07-17T07:16:05.419148+0000","last_fullsized":"2024-07-17T07:16:05.419148+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:39.349083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447283+0000","last_change":"2024-07-17T07:16:04.432346+0000","last_active":"2024-07-17T07:16:04.447283+0000","last_peered":"2024-07-17T07:16:04.447283+0000","last_clean":"2024-07-17T07:16:04.447283+0000","last_became_active":"2024-07-17T07:16:04.430603+0000","last_became_peered":"2024-07-17T07:16:04.430603+0000","last_unstale":"2024-07-17T07:16:04.447283+0000","last_undegraded":"2024-07-17T07:16:04.447283+0000","last_fullsized":"2024-07-17T07:16:04.447283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:48.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772818+0000","last_change":"2024-07-17T07:16:03.407915+0000","last_active":"2024-07-17T07:16:05.772818+0000","last_peered":"2024-07-17T07:16:05.772818+0000","last_clean":"2024-07-17T07:16:05.772818+0000","last_became_active":"2024-07-17T07:16:03.407350+0000","last_became_peered":"2024-07-17T07:16:03.407350+0000","last_unstale":"2024-07-17T07:16:05.772818+0000","last_undegraded":"2024-07-17T07:16:05.772818+0000","last_fullsized":"2024-07-17T07:16:05.772818+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:52.907978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678295+0000","last_change":"2024-07-17T07:16:04.428996+0000","last_active":"2024-07-17T07:16:05.678295+0000","last_peered":"2024-07-17T07:16:05.678295+0000","last_clean":"2024-07-17T07:16:05.678295+0000","last_became_active":"2024-07-17T07:16:04.428466+0000","last_became_peered":"2024-07-17T07:16:04.428466+0000","last_unstale":"2024-07-17T07:16:05.678295+0000","last_undegraded":"2024-07-17T07:16:05.678295+0000","last_fullsized":"2024-07-17T07:16:05.678295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:44.283225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.395111+0000","last_change":"2024-07-17T07:16:03.405437+0000","last_active":"2024-07-17T07:16:04.395111+0000","last_peered":"2024-07-17T07:16:04.395111+0000","last_clean":"2024-07-17T07:16:04.395111+0000","last_became_active":"2024-07-17T07:16:03.405155+0000","last_became_peered":"2024-07-17T07:16:03.405155+0000","last_unstale":"2024-07-17T07:16:04.395111+0000","last_undegraded":"2024-07-17T07:16:04.395111+0000","last_fullsized":"2024-07-17T07:16:04.395111+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:16.382269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.414282+0000","last_change":"2024-07-17T07:16:04.412413+0000","last_active":"2024-07-17T07:16:04.414282+0000","last_peered":"2024-07-17T07:16:04.414282+0000","last_clean":"2024-07-17T07:16:04.414282+0000","last_became_active":"2024-07-17T07:16:04.411797+0000","last_became_peered":"2024-07-17T07:16:04.411797+0000","last_unstale":"2024-07-17T07:16:04.414282+0000","last_undegraded":"2024-07-17T07:16:04.414282+0000","last_fullsized":"2024-07-17T07:16:04.414282+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:43.231108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420714+0000","last_change":"2024-07-17T07:16:03.396112+0000","last_active":"2024-07-17T07:16:05.420714+0000","last_peered":"2024-07-17T07:16:05.420714+0000","last_clean":"2024-07-17T07:16:05.420714+0000","last_became_active":"2024-07-17T07:16:03.395809+0000","last_became_peered":"2024-07-17T07:16:03.395809+0000","last_unstale":"2024-07-17T07:16:05.420714+0000","last_undegraded":"2024-07-17T07:16:05.420714+0000","last_fullsized":"2024-07-17T07:16:05.420714+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:19:59.378950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414295+0000","last_change":"2024-07-17T07:16:04.443194+0000","last_active":"2024-07-17T07:16:05.414295+0000","last_peered":"2024-07-17T07:16:05.414295+0000","last_clean":"2024-07-17T07:16:05.414295+0000","last_became_active":"2024-07-17T07:16:04.442319+0000","last_became_peered":"2024-07-17T07:16:04.442319+0000","last_unstale":"2024-07-17T07:16:05.414295+0000","last_undegraded":"2024-07-17T07:16:05.414295+0000","last_fullsized":"2024-07-17T07:16:05.414295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:17.714355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414170+0000","last_change":"2024-07-17T07:16:03.387769+0000","last_active":"2024-07-17T07:16:05.414170+0000","last_peered":"2024-07-17T07:16:05.414170+0000","last_clean":"2024-07-17T07:16:05.414170+0000","last_became_active":"2024-07-17T07:16:03.386309+0000","last_became_peered":"2024-07-17T07:16:03.386309+0000","last_unstale":"2024-07-17T07:16:05.414170+0000","last_undegraded":"2024-07-17T07:16:05.414170+0000","last_fullsized":"2024-07-17T07:16:05.414170+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:18.367156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.440121+0000","last_change":"2024-07-17T07:16:04.429704+0000","last_active":"2024-07-17T07:16:04.440121+0000","last_peered":"2024-07-17T07:16:04.440121+0000","last_clean":"2024-07-17T07:16:04.440121+0000","last_became_active":"2024-07-17T07:16:04.429395+0000","last_became_peered":"2024-07-17T07:16:04.429395+0000","last_unstale":"2024-07-17T07:16:04.440121+0000","last_undegraded":"2024-07-17T07:16:04.440121+0000","last_fullsized":"2024-07-17T07:16:04.440121+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:09:29.238417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.399521+0000","last_change":"2024-07-17T07:16:03.401590+0000","last_active":"2024-07-17T07:16:04.399521+0000","last_peered":"2024-07-17T07:16:04.399521+0000","last_clean":"2024-07-17T07:16:04.399521+0000","last_became_active":"2024-07-17T07:16:03.400752+0000","last_became_peered":"2024-07-17T07:16:03.400752+0000","last_unstale":"2024-07-17T07:16:04.399521+0000","last_undegraded":"2024-07-17T07:16:04.399521+0000","last_fullsized":"2024-07-17T07:16:04.399521+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:33.470831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431350+0000","last_change":"2024-07-17T07:16:04.429287+0000","last_active":"2024-07-17T07:16:05.431350+0000","last_peered":"2024-07-17T07:16:05.431350+0000","last_clean":"2024-07-17T07:16:05.431350+0000","last_became_active":"2024-07-17T07:16:04.428975+0000","last_became_peered":"2024-07-17T07:16:04.428975+0000","last_unstale":"2024-07-17T07:16:05.431350+0000","last_undegraded":"2024-07-17T07:16:05.431350+0000","last_fullsized":"2024-07-17T07:16:05.431350+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:48.339033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.773126+0000","last_change":"2024-07-17T07:16:03.396736+0000","last_active":"2024-07-17T07:16:05.773126+0000","last_peered":"2024-07-17T07:16:05.773126+0000","last_clean":"2024-07-17T07:16:05.773126+0000","last_became_active":"2024-07-17T07:16:03.396341+0000","last_became_peered":"2024-07-17T07:16:03.396341+0000","last_unstale":"2024-07-17T07:16:05.773126+0000","last_undegraded":"2024-07-17T07:16:05.773126+0000","last_fullsized":"2024-07-17T07:16:05.773126+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:55.077860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.417802+0000","last_change":"2024-07-17T07:16:04.414006+0000","last_active":"2024-07-17T07:16:04.417802+0000","last_peered":"2024-07-17T07:16:04.417802+0000","last_clean":"2024-07-17T07:16:04.417802+0000","last_became_active":"2024-07-17T07:16:04.413590+0000","last_became_peered":"2024-07-17T07:16:04.413590+0000","last_unstale":"2024-07-17T07:16:04.417802+0000","last_undegraded":"2024-07-17T07:16:04.417802+0000","last_fullsized":"2024-07-17T07:16:04.417802+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:46.520338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414541+0000","last_change":"2024-07-17T07:16:03.395402+0000","last_active":"2024-07-17T07:16:05.414541+0000","last_peered":"2024-07-17T07:16:05.414541+0000","last_clean":"2024-07-17T07:16:05.414541+0000","last_became_active":"2024-07-17T07:16:03.394864+0000","last_became_peered":"2024-07-17T07:16:03.394864+0000","last_unstale":"2024-07-17T07:16:05.414541+0000","last_undegraded":"2024-07-17T07:16:05.414541+0000","last_fullsized":"2024-07-17T07:16:05.414541+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:49.714981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417850+0000","last_change":"2024-07-17T07:16:04.431964+0000","last_active":"2024-07-17T07:16:05.417850+0000","last_peered":"2024-07-17T07:16:05.417850+0000","last_clean":"2024-07-17T07:16:05.417850+0000","last_became_active":"2024-07-17T07:16:04.431665+0000","last_became_peered":"2024-07-17T07:16:04.431665+0000","last_unstale":"2024-07-17T07:16:05.417850+0000","last_undegraded":"2024-07-17T07:16:05.417850+0000","last_fullsized":"2024-07-17T07:16:05.417850+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:13.828577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.429554+0000","last_change":"2024-07-17T07:16:03.398335+0000","last_active":"2024-07-17T07:16:05.429554+0000","last_peered":"2024-07-17T07:16:05.429554+0000","last_clean":"2024-07-17T07:16:05.429554+0000","last_became_active":"2024-07-17T07:16:03.398040+0000","last_became_peered":"2024-07-17T07:16:03.398040+0000","last_unstale":"2024-07-17T07:16:05.429554+0000","last_undegraded":"2024-07-17T07:16:05.429554+0000","last_fullsized":"2024-07-17T07:16:05.429554+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:41.665116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447253+0000","last_change":"2024-07-17T07:16:04.431016+0000","last_active":"2024-07-17T07:16:04.447253+0000","last_peered":"2024-07-17T07:16:04.447253+0000","last_clean":"2024-07-17T07:16:04.447253+0000","last_became_active":"2024-07-17T07:16:04.430201+0000","last_became_peered":"2024-07-17T07:16:04.430201+0000","last_unstale":"2024-07-17T07:16:04.447253+0000","last_undegraded":"2024-07-17T07:16:04.447253+0000","last_fullsized":"2024-07-17T07:16:04.447253+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:54.981252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421441+0000","last_change":"2024-07-17T07:16:03.386171+0000","last_active":"2024-07-17T07:16:05.421441+0000","last_peered":"2024-07-17T07:16:05.421441+0000","last_clean":"2024-07-17T07:16:05.421441+0000","last_became_active":"2024-07-17T07:16:03.385707+0000","last_became_peered":"2024-07-17T07:16:03.385707+0000","last_unstale":"2024-07-17T07:16:05.421441+0000","last_undegraded":"2024-07-17T07:16:05.421441+0000","last_fullsized":"2024-07-17T07:16:05.421441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:27.306260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678586+0000","last_change":"2024-07-17T07:16:04.441192+0000","last_active":"2024-07-17T07:16:05.678586+0000","last_peered":"2024-07-17T07:16:05.678586+0000","last_clean":"2024-07-17T07:16:05.678586+0000","last_became_active":"2024-07-17T07:16:04.440884+0000","last_became_peered":"2024-07-17T07:16:04.440884+0000","last_unstale":"2024-07-17T07:16:05.678586+0000","last_undegraded":"2024-07-17T07:16:05.678586+0000","last_fullsized":"2024-07-17T07:16:05.678586+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:16.353468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772605+0000","last_change":"2024-07-17T07:16:03.399109+0000","last_active":"2024-07-17T07:16:05.772605+0000","last_peered":"2024-07-17T07:16:05.772605+0000","last_clean":"2024-07-17T07:16:05.772605+0000","last_became_active":"2024-07-17T07:16:03.398828+0000","last_became_peered":"2024-07-17T07:16:03.398828+0000","last_unstale":"2024-07-17T07:16:05.772605+0000","last_undegraded":"2024-07-17T07:16:05.772605+0000","last_fullsized":"2024-07-17T07:16:05.772605+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:39.349083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772280+0000","last_change":"2024-07-17T07:16:04.434311+0000","last_active":"2024-07-17T07:16:05.772280+0000","last_peered":"2024-07-17T07:16:05.772280+0000","last_clean":"2024-07-17T07:16:05.772280+0000","last_became_active":"2024-07-17T07:16:04.433407+0000","last_became_peered":"2024-07-17T07:16:04.433407+0000","last_unstale":"2024-07-17T07:16:05.772280+0000","last_undegraded":"2024-07-17T07:16:05.772280+0000","last_fullsized":"2024-07-17T07:16:05.772280+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:29.128517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431195+0000","last_change":"2024-07-17T07:16:03.398607+0000","last_active":"2024-07-17T07:16:05.431195+0000","last_peered":"2024-07-17T07:16:05.431195+0000","last_clean":"2024-07-17T07:16:05.431195+0000","last_became_active":"2024-07-17T07:16:03.397930+0000","last_became_peered":"2024-07-17T07:16:03.397930+0000","last_unstale":"2024-07-17T07:16:05.431195+0000","last_undegraded":"2024-07-17T07:16:05.431195+0000","last_fullsized":"2024-07-17T07:16:05.431195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:24.272699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772488+0000","last_change":"2024-07-17T07:16:04.431162+0000","last_active":"2024-07-17T07:16:05.772488+0000","last_peered":"2024-07-17T07:16:05.772488+0000","last_clean":"2024-07-17T07:16:05.772488+0000","last_became_active":"2024-07-17T07:16:04.430847+0000","last_became_peered":"2024-07-17T07:16:04.430847+0000","last_unstale":"2024-07-17T07:16:05.772488+0000","last_undegraded":"2024-07-17T07:16:05.772488+0000","last_fullsized":"2024-07-17T07:16:05.772488+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:30.086416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.401320+0000","last_change":"2024-07-17T07:16:03.405465+0000","last_active":"2024-07-17T07:16:04.401320+0000","last_peered":"2024-07-17T07:16:04.401320+0000","last_clean":"2024-07-17T07:16:04.401320+0000","last_became_active":"2024-07-17T07:16:03.405184+0000","last_became_peered":"2024-07-17T07:16:03.405184+0000","last_unstale":"2024-07-17T07:16:04.401320+0000","last_undegraded":"2024-07-17T07:16:04.401320+0000","last_fullsized":"2024-07-17T07:16:04.401320+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:19.810051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772991+0000","last_change":"2024-07-17T07:16:04.442153+0000","last_active":"2024-07-17T07:16:05.772991+0000","last_peered":"2024-07-17T07:16:05.772991+0000","last_clean":"2024-07-17T07:16:05.772991+0000","last_became_active":"2024-07-17T07:16:04.441875+0000","last_became_peered":"2024-07-17T07:16:04.441875+0000","last_unstale":"2024-07-17T07:16:05.772991+0000","last_undegraded":"2024-07-17T07:16:05.772991+0000","last_fullsized":"2024-07-17T07:16:05.772991+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.039380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772209+0000","last_change":"2024-07-17T07:16:03.405492+0000","last_active":"2024-07-17T07:16:05.772209+0000","last_peered":"2024-07-17T07:16:05.772209+0000","last_clean":"2024-07-17T07:16:05.772209+0000","last_became_active":"2024-07-17T07:16:03.405201+0000","last_became_peered":"2024-07-17T07:16:03.405201+0000","last_unstale":"2024-07-17T07:16:05.772209+0000","last_undegraded":"2024-07-17T07:16:05.772209+0000","last_fullsized":"2024-07-17T07:16:05.772209+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:23.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447421+0000","last_change":"2024-07-17T07:16:04.436149+0000","last_active":"2024-07-17T07:16:04.447421+0000","last_peered":"2024-07-17T07:16:04.447421+0000","last_clean":"2024-07-17T07:16:04.447421+0000","last_became_active":"2024-07-17T07:16:04.429030+0000","last_became_peered":"2024-07-17T07:16:04.429030+0000","last_unstale":"2024-07-17T07:16:04.447421+0000","last_undegraded":"2024-07-17T07:16:04.447421+0000","last_fullsized":"2024-07-17T07:16:04.447421+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:49.830227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.393841+0000","last_change":"2024-07-17T07:16:03.405566+0000","last_active":"2024-07-17T07:16:04.393841+0000","last_peered":"2024-07-17T07:16:04.393841+0000","last_clean":"2024-07-17T07:16:04.393841+0000","last_became_active":"2024-07-17T07:16:03.404684+0000","last_became_peered":"2024-07-17T07:16:03.404684+0000","last_unstale":"2024-07-17T07:16:04.393841+0000","last_undegraded":"2024-07-17T07:16:04.393841+0000","last_fullsized":"2024-07-17T07:16:04.393841+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:20.318729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419974+0000","last_change":"2024-07-17T07:16:04.430647+0000","last_active":"2024-07-17T07:16:05.419974+0000","last_peered":"2024-07-17T07:16:05.419974+0000","last_clean":"2024-07-17T07:16:05.419974+0000","last_became_active":"2024-07-17T07:16:04.430072+0000","last_became_peered":"2024-07-17T07:16:04.430072+0000","last_unstale":"2024-07-17T07:16:05.419974+0000","last_undegraded":"2024-07-17T07:16:05.419974+0000","last_fullsized":"2024-07-17T07:16:05.419974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:41.650990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.400003+0000","last_change":"2024-07-17T07:16:03.401772+0000","last_active":"2024-07-17T07:16:04.400003+0000","last_peered":"2024-07-17T07:16:04.400003+0000","last_clean":"2024-07-17T07:16:04.400003+0000","last_became_active":"2024-07-17T07:16:03.401464+0000","last_became_peered":"2024-07-17T07:16:03.401464+0000","last_unstale":"2024-07-17T07:16:04.400003+0000","last_undegraded":"2024-07-17T07:16:04.400003+0000","last_fullsized":"2024-07-17T07:16:04.400003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:42.033603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"145'2448","reported_seq":2991,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.397649+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:16:04.397649+0000","last_peered":"2024-07-17T07:16:04.397649+0000","last_clean":"2024-07-17T07:16:04.397649+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:16:04.397649+0000","last_undegraded":"2024-07-17T07:16:04.397649+0000","last_fullsized":"2024-07-17T07:16:04.397649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":251,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":251,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2448,"ondisk_log_size":2448,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542395,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55860,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315980,"statfs":{"total":96636764160,"available":96579563520,"internally_reserved":0,"allocated":2523136,"data_stored":1611312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509690,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":1027154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509691,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55280,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316560,"statfs":{"total":96636764160,"available":96580157440,"internally_reserved":0,"allocated":1929216,"data_stored":1026504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542396,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1026482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509690,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1929216,"data_stored":1020924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542394,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1933312,"data_stored":1027098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509691,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1929216,"data_stored":1026552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509690,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2514944,"data_stored":1609666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:16:08.897 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.153+0000 7fb7eac00640 1 -- 172.21.15.44:0/306541651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e409b6e0 msgr2=0x7fb7e40cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.153+0000 7fb7eac00640 1 --2- 172.21.15.44:0/306541651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e409b6e0 0x7fb7e40cf380 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb7e409bb70 tx=0x7fb7d002a560 comp rx=0 tx=0).stop 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 -- 172.21.15.44:0/306541651 shutdown_connections 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 --2- 172.21.15.44:0/306541651 >> v2:172.21.15.175:3300/0 conn(0x7fb7e40cf8c0 0x7fb7e4159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 --2- 172.21.15.44:0/306541651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e409b6e0 0x7fb7e40cf380 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 --2- 172.21.15.44:0/306541651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b300 0x7fb7e406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 -- 172.21.15.44:0/306541651 >> 172.21.15.44:0/306541651 conn(0x7fb7e40b0e80 msgr2=0x7fb7e40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 -- 172.21.15.44:0/306541651 shutdown_connections 2024-07-17T07:16:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.154+0000 7fb7eac00640 1 -- 172.21.15.44:0/306541651 wait complete. 2024-07-17T07:16:09.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.156+0000 7fb7eac00640 1 Processor -- start 2024-07-17T07:16:09.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.156+0000 7fb7eac00640 1 -- start start 2024-07-17T07:16:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e409b300 0x7fb7e41f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 0x7fb7e41582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e415a2d0 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e415a150 con 0x7fb7e409b300 2024-07-17T07:16:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7eac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e40b1fa0 con 0x7fb7e409b6e0 2024-07-17T07:16:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.157+0000 7fb7ea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41290/0 (socket says 172.21.15.44:41290) 2024-07-17T07:16:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 -- 172.21.15.44:0/2463557264 learned_addr learned my addr 172.21.15.44:0/2463557264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7e9800640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e409b300 0x7fb7e41f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7e8e00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 0x7fb7e41582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 -- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 msgr2=0x7fb7e41582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 0x7fb7e41582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 -- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e409b300 msgr2=0x7fb7e41f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e409b300 0x7fb7e41f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 -- 172.21.15.44:0/2463557264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e4158cb0 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7e8e00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 0x7fb7e41582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.158+0000 7fb7ea200640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00ddc0 tx=0x7fb7dc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:09.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.159+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc004d20 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.159+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7dc00be40 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.159+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00e8a0 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.159+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41589d0 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.159+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41f7760 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.160+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e40cf380 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.161+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fb7dc00ea40 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.162+0000 7fb7e1600640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 0x7fb7c4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.162+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7fb7dc099410 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.163+0000 7fb7e9800640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 0x7fb7c4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.164+0000 7fb7e9800640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 0x7fb7c4079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005940 tx=0x7fb7d800c460 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.167+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb7dc062410 con 0x7fb7e40cf8c0 2024-07-17T07:16:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.375+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 --> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7e40ac1c0 con 0x7fb7c4076cd0 2024-07-17T07:16:09.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.388+0000 7fb7e1600640 1 -- 172.21.15.44:0/2463557264 <== mgr.6434 v2:172.21.15.175:6800/729654308 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7fb7e40ac1c0 con 0x7fb7c4076cd0 2024-07-17T07:16:09.391 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:09.392 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T07:16:09.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.393+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 msgr2=0x7fb7c4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.393+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 0x7fb7c4079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8005940 tx=0x7fb7d800c460 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 msgr2=0x7fb7e4071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00ddc0 tx=0x7fb7dc00b730 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 shutdown_connections 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fb7c4076cd0 0x7fb7c4079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7e40cf8c0 0x7fb7e4071dd0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7e409b6e0 0x7fb7e41582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 --2- 172.21.15.44:0/2463557264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e409b300 0x7fb7e41f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 >> 172.21.15.44:0/2463557264 conn(0x7fb7e40b0e80 msgr2=0x7fb7e415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 shutdown_connections 2024-07-17T07:16:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.394+0000 7fb7eac00640 1 -- 172.21.15.44:0/2463557264 wait complete. 2024-07-17T07:16:09.414 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T07:16:08.070065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":296,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2487,"ondisk_log_size":2487,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":348956,"kb_used_data":16228,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754625764,"statfs":{"total":773094113280,"available":772736782336,"internally_reserved":0,"allocated":16617472,"data_stored":9375692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":340577840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.711541"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414642+0000","last_change":"2024-07-17T07:16:04.426483+0000","last_active":"2024-07-17T07:16:05.414642+0000","last_peered":"2024-07-17T07:16:05.414642+0000","last_clean":"2024-07-17T07:16:05.414642+0000","last_became_active":"2024-07-17T07:16:04.426158+0000","last_became_peered":"2024-07-17T07:16:04.426158+0000","last_unstale":"2024-07-17T07:16:05.414642+0000","last_undegraded":"2024-07-17T07:16:05.414642+0000","last_fullsized":"2024-07-17T07:16:05.414642+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:13:50.539011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431942+0000","last_change":"2024-07-17T07:16:04.419936+0000","last_active":"2024-07-17T07:16:05.431942+0000","last_peered":"2024-07-17T07:16:05.431942+0000","last_clean":"2024-07-17T07:16:05.431942+0000","last_became_active":"2024-07-17T07:16:04.419551+0000","last_became_peered":"2024-07-17T07:16:04.419551+0000","last_unstale":"2024-07-17T07:16:05.431942+0000","last_undegraded":"2024-07-17T07:16:05.431942+0000","last_fullsized":"2024-07-17T07:16:05.431942+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:01.507866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419378+0000","last_change":"2024-07-17T07:16:04.432493+0000","last_active":"2024-07-17T07:16:05.419378+0000","last_peered":"2024-07-17T07:16:05.419378+0000","last_clean":"2024-07-17T07:16:05.419378+0000","last_became_active":"2024-07-17T07:16:04.430398+0000","last_became_peered":"2024-07-17T07:16:04.430398+0000","last_unstale":"2024-07-17T07:16:05.419378+0000","last_undegraded":"2024-07-17T07:16:05.419378+0000","last_fullsized":"2024-07-17T07:16:05.419378+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:30.086416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.430828+0000","last_change":"2024-07-17T07:16:04.434360+0000","last_active":"2024-07-17T07:16:05.430828+0000","last_peered":"2024-07-17T07:16:05.430828+0000","last_clean":"2024-07-17T07:16:05.430828+0000","last_became_active":"2024-07-17T07:16:04.434082+0000","last_became_peered":"2024-07-17T07:16:04.434082+0000","last_unstale":"2024-07-17T07:16:05.430828+0000","last_undegraded":"2024-07-17T07:16:05.430828+0000","last_fullsized":"2024-07-17T07:16:05.430828+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.422131+0000","last_change":"2024-07-17T07:16:04.418222+0000","last_active":"2024-07-17T07:16:04.422131+0000","last_peered":"2024-07-17T07:16:04.422131+0000","last_clean":"2024-07-17T07:16:04.422131+0000","last_became_active":"2024-07-17T07:16:04.417800+0000","last_became_peered":"2024-07-17T07:16:04.417800+0000","last_unstale":"2024-07-17T07:16:04.422131+0000","last_undegraded":"2024-07-17T07:16:04.422131+0000","last_fullsized":"2024-07-17T07:16:04.422131+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:54.121775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772130+0000","last_change":"2024-07-17T07:16:04.431021+0000","last_active":"2024-07-17T07:16:05.772130+0000","last_peered":"2024-07-17T07:16:05.772130+0000","last_clean":"2024-07-17T07:16:05.772130+0000","last_became_active":"2024-07-17T07:16:04.430712+0000","last_became_peered":"2024-07-17T07:16:04.430712+0000","last_unstale":"2024-07-17T07:16:05.772130+0000","last_undegraded":"2024-07-17T07:16:05.772130+0000","last_fullsized":"2024-07-17T07:16:05.772130+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:44.062549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447131+0000","last_change":"2024-07-17T07:16:04.427861+0000","last_active":"2024-07-17T07:16:04.447131+0000","last_peered":"2024-07-17T07:16:04.447131+0000","last_clean":"2024-07-17T07:16:04.447131+0000","last_became_active":"2024-07-17T07:16:04.427574+0000","last_became_peered":"2024-07-17T07:16:04.427574+0000","last_unstale":"2024-07-17T07:16:04.447131+0000","last_undegraded":"2024-07-17T07:16:04.447131+0000","last_fullsized":"2024-07-17T07:16:04.447131+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:18.220444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417866+0000","last_change":"2024-07-17T07:16:04.437314+0000","last_active":"2024-07-17T07:16:05.417866+0000","last_peered":"2024-07-17T07:16:05.417866+0000","last_clean":"2024-07-17T07:16:05.417866+0000","last_became_active":"2024-07-17T07:16:04.436994+0000","last_became_peered":"2024-07-17T07:16:04.436994+0000","last_unstale":"2024-07-17T07:16:05.417866+0000","last_undegraded":"2024-07-17T07:16:05.417866+0000","last_fullsized":"2024-07-17T07:16:05.417866+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:30.257635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772135+0000","last_change":"2024-07-17T07:16:04.429212+0000","last_active":"2024-07-17T07:16:05.772135+0000","last_peered":"2024-07-17T07:16:05.772135+0000","last_clean":"2024-07-17T07:16:05.772135+0000","last_became_active":"2024-07-17T07:16:04.428916+0000","last_became_peered":"2024-07-17T07:16:04.428916+0000","last_unstale":"2024-07-17T07:16:05.772135+0000","last_undegraded":"2024-07-17T07:16:05.772135+0000","last_fullsized":"2024-07-17T07:16:05.772135+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:13.828577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678915+0000","last_change":"2024-07-17T07:16:04.443400+0000","last_active":"2024-07-17T07:16:05.678915+0000","last_peered":"2024-07-17T07:16:05.678915+0000","last_clean":"2024-07-17T07:16:05.678915+0000","last_became_active":"2024-07-17T07:16:04.441734+0000","last_became_peered":"2024-07-17T07:16:04.441734+0000","last_unstale":"2024-07-17T07:16:05.678915+0000","last_undegraded":"2024-07-17T07:16:05.678915+0000","last_fullsized":"2024-07-17T07:16:05.678915+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:46.767537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431073+0000","last_change":"2024-07-17T07:16:04.431402+0000","last_active":"2024-07-17T07:16:05.431073+0000","last_peered":"2024-07-17T07:16:05.431073+0000","last_clean":"2024-07-17T07:16:05.431073+0000","last_became_active":"2024-07-17T07:16:04.431120+0000","last_became_peered":"2024-07-17T07:16:04.431120+0000","last_unstale":"2024-07-17T07:16:05.431073+0000","last_undegraded":"2024-07-17T07:16:05.431073+0000","last_fullsized":"2024-07-17T07:16:05.431073+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:40:08.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414692+0000","last_change":"2024-07-17T07:16:04.409086+0000","last_active":"2024-07-17T07:16:05.414692+0000","last_peered":"2024-07-17T07:16:05.414692+0000","last_clean":"2024-07-17T07:16:05.414692+0000","last_became_active":"2024-07-17T07:16:04.408623+0000","last_became_peered":"2024-07-17T07:16:04.408623+0000","last_unstale":"2024-07-17T07:16:05.414692+0000","last_undegraded":"2024-07-17T07:16:05.414692+0000","last_fullsized":"2024-07-17T07:16:05.414692+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:33:03.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772683+0000","last_change":"2024-07-17T07:16:04.432591+0000","last_active":"2024-07-17T07:16:05.772683+0000","last_peered":"2024-07-17T07:16:05.772683+0000","last_clean":"2024-07-17T07:16:05.772683+0000","last_became_active":"2024-07-17T07:16:04.430977+0000","last_became_peered":"2024-07-17T07:16:04.430977+0000","last_unstale":"2024-07-17T07:16:05.772683+0000","last_undegraded":"2024-07-17T07:16:05.772683+0000","last_fullsized":"2024-07-17T07:16:05.772683+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:35:19.791752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414019+0000","last_change":"2024-07-17T07:16:04.418037+0000","last_active":"2024-07-17T07:16:05.414019+0000","last_peered":"2024-07-17T07:16:05.414019+0000","last_clean":"2024-07-17T07:16:05.414019+0000","last_became_active":"2024-07-17T07:16:04.414615+0000","last_became_peered":"2024-07-17T07:16:04.414615+0000","last_unstale":"2024-07-17T07:16:05.414019+0000","last_undegraded":"2024-07-17T07:16:05.414019+0000","last_fullsized":"2024-07-17T07:16:05.414019+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:10.844648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413745+0000","last_change":"2024-07-17T07:16:04.414313+0000","last_active":"2024-07-17T07:16:05.413745+0000","last_peered":"2024-07-17T07:16:05.413745+0000","last_clean":"2024-07-17T07:16:05.413745+0000","last_became_active":"2024-07-17T07:16:04.413869+0000","last_became_peered":"2024-07-17T07:16:04.413869+0000","last_unstale":"2024-07-17T07:16:05.413745+0000","last_undegraded":"2024-07-17T07:16:05.413745+0000","last_fullsized":"2024-07-17T07:16:05.413745+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:29.493082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772852+0000","last_change":"2024-07-17T07:16:04.427278+0000","last_active":"2024-07-17T07:16:05.772852+0000","last_peered":"2024-07-17T07:16:05.772852+0000","last_clean":"2024-07-17T07:16:05.772852+0000","last_became_active":"2024-07-17T07:16:04.426379+0000","last_became_peered":"2024-07-17T07:16:04.426379+0000","last_unstale":"2024-07-17T07:16:05.772852+0000","last_undegraded":"2024-07-17T07:16:05.772852+0000","last_fullsized":"2024-07-17T07:16:05.772852+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:41.650990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772349+0000","last_change":"2024-07-17T07:16:04.431227+0000","last_active":"2024-07-17T07:16:05.772349+0000","last_peered":"2024-07-17T07:16:05.772349+0000","last_clean":"2024-07-17T07:16:05.772349+0000","last_became_active":"2024-07-17T07:16:04.430575+0000","last_became_peered":"2024-07-17T07:16:04.430575+0000","last_unstale":"2024-07-17T07:16:05.772349+0000","last_undegraded":"2024-07-17T07:16:05.772349+0000","last_fullsized":"2024-07-17T07:16:05.772349+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:25.911907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.427229+0000","last_change":"2024-07-17T07:16:04.420003+0000","last_active":"2024-07-17T07:16:04.427229+0000","last_peered":"2024-07-17T07:16:04.427229+0000","last_clean":"2024-07-17T07:16:04.427229+0000","last_became_active":"2024-07-17T07:16:04.419348+0000","last_became_peered":"2024-07-17T07:16:04.419348+0000","last_unstale":"2024-07-17T07:16:04.427229+0000","last_undegraded":"2024-07-17T07:16:04.427229+0000","last_fullsized":"2024-07-17T07:16:04.427229+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:15.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439714+0000","last_change":"2024-07-17T07:16:04.422546+0000","last_active":"2024-07-17T07:16:04.439714+0000","last_peered":"2024-07-17T07:16:04.439714+0000","last_clean":"2024-07-17T07:16:04.439714+0000","last_became_active":"2024-07-17T07:16:04.422234+0000","last_became_peered":"2024-07-17T07:16:04.422234+0000","last_unstale":"2024-07-17T07:16:04.439714+0000","last_undegraded":"2024-07-17T07:16:04.439714+0000","last_fullsized":"2024-07-17T07:16:04.439714+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.085733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.417711+0000","last_change":"2024-07-17T07:16:04.413812+0000","last_active":"2024-07-17T07:16:04.417711+0000","last_peered":"2024-07-17T07:16:04.417711+0000","last_clean":"2024-07-17T07:16:04.417711+0000","last_became_active":"2024-07-17T07:16:04.413263+0000","last_became_peered":"2024-07-17T07:16:04.413263+0000","last_unstale":"2024-07-17T07:16:04.417711+0000","last_undegraded":"2024-07-17T07:16:04.417711+0000","last_fullsized":"2024-07-17T07:16:04.417711+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:44.234690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431170+0000","last_change":"2024-07-17T07:16:04.435133+0000","last_active":"2024-07-17T07:16:05.431170+0000","last_peered":"2024-07-17T07:16:05.431170+0000","last_clean":"2024-07-17T07:16:05.431170+0000","last_became_active":"2024-07-17T07:16:04.434789+0000","last_became_peered":"2024-07-17T07:16:04.434789+0000","last_unstale":"2024-07-17T07:16:05.431170+0000","last_undegraded":"2024-07-17T07:16:05.431170+0000","last_fullsized":"2024-07-17T07:16:05.431170+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:58.239590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414346+0000","last_change":"2024-07-17T07:16:04.420589+0000","last_active":"2024-07-17T07:16:05.414346+0000","last_peered":"2024-07-17T07:16:05.414346+0000","last_clean":"2024-07-17T07:16:05.414346+0000","last_became_active":"2024-07-17T07:16:04.420031+0000","last_became_peered":"2024-07-17T07:16:04.420031+0000","last_unstale":"2024-07-17T07:16:05.414346+0000","last_undegraded":"2024-07-17T07:16:05.414346+0000","last_fullsized":"2024-07-17T07:16:05.414346+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:25.512963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.430542+0000","last_change":"2024-07-17T07:16:04.441190+0000","last_active":"2024-07-17T07:16:05.430542+0000","last_peered":"2024-07-17T07:16:05.430542+0000","last_clean":"2024-07-17T07:16:05.430542+0000","last_became_active":"2024-07-17T07:16:04.440906+0000","last_became_peered":"2024-07-17T07:16:04.440906+0000","last_unstale":"2024-07-17T07:16:05.430542+0000","last_undegraded":"2024-07-17T07:16:05.430542+0000","last_fullsized":"2024-07-17T07:16:05.430542+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:15.981756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772982+0000","last_change":"2024-07-17T07:16:04.429379+0000","last_active":"2024-07-17T07:16:05.772982+0000","last_peered":"2024-07-17T07:16:05.772982+0000","last_clean":"2024-07-17T07:16:05.772982+0000","last_became_active":"2024-07-17T07:16:04.429090+0000","last_became_peered":"2024-07-17T07:16:04.429090+0000","last_unstale":"2024-07-17T07:16:05.772982+0000","last_undegraded":"2024-07-17T07:16:05.772982+0000","last_fullsized":"2024-07-17T07:16:05.772982+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:32.084265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420661+0000","last_change":"2024-07-17T07:16:04.441263+0000","last_active":"2024-07-17T07:16:05.420661+0000","last_peered":"2024-07-17T07:16:05.420661+0000","last_clean":"2024-07-17T07:16:05.420661+0000","last_became_active":"2024-07-17T07:16:04.440996+0000","last_became_peered":"2024-07-17T07:16:04.440996+0000","last_unstale":"2024-07-17T07:16:05.420661+0000","last_undegraded":"2024-07-17T07:16:05.420661+0000","last_fullsized":"2024-07-17T07:16:05.420661+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:13:00.887279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418864+0000","last_change":"2024-07-17T07:16:04.443050+0000","last_active":"2024-07-17T07:16:05.418864+0000","last_peered":"2024-07-17T07:16:05.418864+0000","last_clean":"2024-07-17T07:16:05.418864+0000","last_became_active":"2024-07-17T07:16:04.442505+0000","last_became_peered":"2024-07-17T07:16:04.442505+0000","last_unstale":"2024-07-17T07:16:05.418864+0000","last_undegraded":"2024-07-17T07:16:05.418864+0000","last_fullsized":"2024-07-17T07:16:05.418864+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:29.128517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447266+0000","last_change":"2024-07-17T07:16:04.431944+0000","last_active":"2024-07-17T07:16:04.447266+0000","last_peered":"2024-07-17T07:16:04.447266+0000","last_clean":"2024-07-17T07:16:04.447266+0000","last_became_active":"2024-07-17T07:16:04.430781+0000","last_became_peered":"2024-07-17T07:16:04.430781+0000","last_unstale":"2024-07-17T07:16:04.447266+0000","last_undegraded":"2024-07-17T07:16:04.447266+0000","last_fullsized":"2024-07-17T07:16:04.447266+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:33.666282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.679086+0000","last_change":"2024-07-17T07:16:04.433922+0000","last_active":"2024-07-17T07:16:05.679086+0000","last_peered":"2024-07-17T07:16:05.679086+0000","last_clean":"2024-07-17T07:16:05.679086+0000","last_became_active":"2024-07-17T07:16:04.433640+0000","last_became_peered":"2024-07-17T07:16:04.433640+0000","last_unstale":"2024-07-17T07:16:05.679086+0000","last_undegraded":"2024-07-17T07:16:05.679086+0000","last_fullsized":"2024-07-17T07:16:05.679086+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:41.157337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420010+0000","last_change":"2024-07-17T07:16:04.433532+0000","last_active":"2024-07-17T07:16:05.420010+0000","last_peered":"2024-07-17T07:16:05.420010+0000","last_clean":"2024-07-17T07:16:05.420010+0000","last_became_active":"2024-07-17T07:16:04.433108+0000","last_became_peered":"2024-07-17T07:16:04.433108+0000","last_unstale":"2024-07-17T07:16:05.420010+0000","last_undegraded":"2024-07-17T07:16:05.420010+0000","last_fullsized":"2024-07-17T07:16:05.420010+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.039380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419273+0000","last_change":"2024-07-17T07:16:04.441337+0000","last_active":"2024-07-17T07:16:05.419273+0000","last_peered":"2024-07-17T07:16:05.419273+0000","last_clean":"2024-07-17T07:16:05.419273+0000","last_became_active":"2024-07-17T07:16:04.440942+0000","last_became_peered":"2024-07-17T07:16:04.440942+0000","last_unstale":"2024-07-17T07:16:05.419273+0000","last_undegraded":"2024-07-17T07:16:05.419273+0000","last_fullsized":"2024-07-17T07:16:05.419273+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:25.911907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420052+0000","last_change":"2024-07-17T07:16:04.437678+0000","last_active":"2024-07-17T07:16:05.420052+0000","last_peered":"2024-07-17T07:16:05.420052+0000","last_clean":"2024-07-17T07:16:05.420052+0000","last_became_active":"2024-07-17T07:16:04.431166+0000","last_became_peered":"2024-07-17T07:16:04.431166+0000","last_unstale":"2024-07-17T07:16:05.420052+0000","last_undegraded":"2024-07-17T07:16:05.420052+0000","last_fullsized":"2024-07-17T07:16:05.420052+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:04.308976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678537+0000","last_change":"2024-07-17T07:16:04.435037+0000","last_active":"2024-07-17T07:16:05.678537+0000","last_peered":"2024-07-17T07:16:05.678537+0000","last_clean":"2024-07-17T07:16:05.678537+0000","last_became_active":"2024-07-17T07:16:04.433246+0000","last_became_peered":"2024-07-17T07:16:04.433246+0000","last_unstale":"2024-07-17T07:16:05.678537+0000","last_undegraded":"2024-07-17T07:16:05.678537+0000","last_fullsized":"2024-07-17T07:16:05.678537+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:44.323473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418930+0000","last_change":"2024-07-17T07:16:04.429320+0000","last_active":"2024-07-17T07:16:05.418930+0000","last_peered":"2024-07-17T07:16:05.418930+0000","last_clean":"2024-07-17T07:16:05.418930+0000","last_became_active":"2024-07-17T07:16:04.428522+0000","last_became_peered":"2024-07-17T07:16:04.428522+0000","last_unstale":"2024-07-17T07:16:05.418930+0000","last_undegraded":"2024-07-17T07:16:05.418930+0000","last_fullsized":"2024-07-17T07:16:05.418930+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:22.929008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"154'4","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420552+0000","last_change":"2024-07-17T07:16:03.395736+0000","last_active":"2024-07-17T07:16:05.420552+0000","last_peered":"2024-07-17T07:16:05.420552+0000","last_clean":"2024-07-17T07:16:05.420552+0000","last_became_active":"2024-07-17T07:16:03.395400+0000","last_became_peered":"2024-07-17T07:16:03.395400+0000","last_unstale":"2024-07-17T07:16:05.420552+0000","last_undegraded":"2024-07-17T07:16:05.420552+0000","last_fullsized":"2024-07-17T07:16:05.420552+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:23.408953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414016+0000","last_change":"2024-07-17T07:16:04.421804+0000","last_active":"2024-07-17T07:16:05.414016+0000","last_peered":"2024-07-17T07:16:05.414016+0000","last_clean":"2024-07-17T07:16:05.414016+0000","last_became_active":"2024-07-17T07:16:04.421339+0000","last_became_peered":"2024-07-17T07:16:04.421339+0000","last_unstale":"2024-07-17T07:16:05.414016+0000","last_undegraded":"2024-07-17T07:16:05.414016+0000","last_fullsized":"2024-07-17T07:16:05.414016+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:55.646028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.443414+0000","last_change":"2024-07-17T07:16:03.404827+0000","last_active":"2024-07-17T07:16:04.443414+0000","last_peered":"2024-07-17T07:16:04.443414+0000","last_clean":"2024-07-17T07:16:04.443414+0000","last_became_active":"2024-07-17T07:16:03.404036+0000","last_became_peered":"2024-07-17T07:16:03.404036+0000","last_unstale":"2024-07-17T07:16:04.443414+0000","last_undegraded":"2024-07-17T07:16:04.443414+0000","last_fullsized":"2024-07-17T07:16:04.443414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:59.836967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431198+0000","last_change":"2024-07-17T07:16:04.437543+0000","last_active":"2024-07-17T07:16:05.431198+0000","last_peered":"2024-07-17T07:16:05.431198+0000","last_clean":"2024-07-17T07:16:05.431198+0000","last_became_active":"2024-07-17T07:16:04.437270+0000","last_became_peered":"2024-07-17T07:16:04.437270+0000","last_unstale":"2024-07-17T07:16:05.431198+0000","last_undegraded":"2024-07-17T07:16:05.431198+0000","last_fullsized":"2024-07-17T07:16:05.431198+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:38.545638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.449973+0000","last_change":"2024-07-17T07:16:03.404705+0000","last_active":"2024-07-17T07:16:04.449973+0000","last_peered":"2024-07-17T07:16:04.449973+0000","last_clean":"2024-07-17T07:16:04.449973+0000","last_became_active":"2024-07-17T07:16:03.403211+0000","last_became_peered":"2024-07-17T07:16:03.403211+0000","last_unstale":"2024-07-17T07:16:04.449973+0000","last_undegraded":"2024-07-17T07:16:04.449973+0000","last_fullsized":"2024-07-17T07:16:04.449973+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:07.796007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414367+0000","last_change":"2024-07-17T07:16:04.443170+0000","last_active":"2024-07-17T07:16:05.414367+0000","last_peered":"2024-07-17T07:16:05.414367+0000","last_clean":"2024-07-17T07:16:05.414367+0000","last_became_active":"2024-07-17T07:16:04.442375+0000","last_became_peered":"2024-07-17T07:16:04.442375+0000","last_unstale":"2024-07-17T07:16:05.414367+0000","last_undegraded":"2024-07-17T07:16:05.414367+0000","last_fullsized":"2024-07-17T07:16:05.414367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:32.857273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.450887+0000","last_change":"2024-07-17T07:16:03.402762+0000","last_active":"2024-07-17T07:16:04.450887+0000","last_peered":"2024-07-17T07:16:04.450887+0000","last_clean":"2024-07-17T07:16:04.450887+0000","last_became_active":"2024-07-17T07:16:03.402442+0000","last_became_peered":"2024-07-17T07:16:03.402442+0000","last_unstale":"2024-07-17T07:16:04.450887+0000","last_undegraded":"2024-07-17T07:16:04.450887+0000","last_fullsized":"2024-07-17T07:16:04.450887+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:31.163924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421317+0000","last_change":"2024-07-17T07:16:04.430860+0000","last_active":"2024-07-17T07:16:05.421317+0000","last_peered":"2024-07-17T07:16:05.421317+0000","last_clean":"2024-07-17T07:16:05.421317+0000","last_became_active":"2024-07-17T07:16:04.430510+0000","last_became_peered":"2024-07-17T07:16:04.430510+0000","last_unstale":"2024-07-17T07:16:05.421317+0000","last_undegraded":"2024-07-17T07:16:05.421317+0000","last_fullsized":"2024-07-17T07:16:05.421317+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:43.631545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420530+0000","last_change":"2024-07-17T07:16:03.401409+0000","last_active":"2024-07-17T07:16:05.420530+0000","last_peered":"2024-07-17T07:16:05.420530+0000","last_clean":"2024-07-17T07:16:05.420530+0000","last_became_active":"2024-07-17T07:16:03.401112+0000","last_became_peered":"2024-07-17T07:16:03.401112+0000","last_unstale":"2024-07-17T07:16:05.420530+0000","last_undegraded":"2024-07-17T07:16:05.420530+0000","last_fullsized":"2024-07-17T07:16:05.420530+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:30.485592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421198+0000","last_change":"2024-07-17T07:16:04.430399+0000","last_active":"2024-07-17T07:16:05.421198+0000","last_peered":"2024-07-17T07:16:05.421198+0000","last_clean":"2024-07-17T07:16:05.421198+0000","last_became_active":"2024-07-17T07:16:04.430131+0000","last_became_peered":"2024-07-17T07:16:04.430131+0000","last_unstale":"2024-07-17T07:16:05.421198+0000","last_undegraded":"2024-07-17T07:16:05.421198+0000","last_fullsized":"2024-07-17T07:16:05.421198+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:11.631473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.442403+0000","last_change":"2024-07-17T07:16:03.401474+0000","last_active":"2024-07-17T07:16:04.442403+0000","last_peered":"2024-07-17T07:16:04.442403+0000","last_clean":"2024-07-17T07:16:04.442403+0000","last_became_active":"2024-07-17T07:16:03.401157+0000","last_became_peered":"2024-07-17T07:16:03.401157+0000","last_unstale":"2024-07-17T07:16:04.442403+0000","last_undegraded":"2024-07-17T07:16:04.442403+0000","last_fullsized":"2024-07-17T07:16:04.442403+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:48.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.428671+0000","last_change":"2024-07-17T07:16:04.426091+0000","last_active":"2024-07-17T07:16:04.428671+0000","last_peered":"2024-07-17T07:16:04.428671+0000","last_clean":"2024-07-17T07:16:04.428671+0000","last_became_active":"2024-07-17T07:16:04.425749+0000","last_became_peered":"2024-07-17T07:16:04.425749+0000","last_unstale":"2024-07-17T07:16:04.428671+0000","last_undegraded":"2024-07-17T07:16:04.428671+0000","last_fullsized":"2024-07-17T07:16:04.428671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:57.981566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.771929+0000","last_change":"2024-07-17T07:16:03.407154+0000","last_active":"2024-07-17T07:16:05.771929+0000","last_peered":"2024-07-17T07:16:05.771929+0000","last_clean":"2024-07-17T07:16:05.771929+0000","last_became_active":"2024-07-17T07:16:03.406825+0000","last_became_peered":"2024-07-17T07:16:03.406825+0000","last_unstale":"2024-07-17T07:16:05.771929+0000","last_undegraded":"2024-07-17T07:16:05.771929+0000","last_fullsized":"2024-07-17T07:16:05.771929+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:42.550236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447663+0000","last_change":"2024-07-17T07:16:04.437105+0000","last_active":"2024-07-17T07:16:04.447663+0000","last_peered":"2024-07-17T07:16:04.447663+0000","last_clean":"2024-07-17T07:16:04.447663+0000","last_became_active":"2024-07-17T07:16:04.436759+0000","last_became_peered":"2024-07-17T07:16:04.436759+0000","last_unstale":"2024-07-17T07:16:04.447663+0000","last_undegraded":"2024-07-17T07:16:04.447663+0000","last_fullsized":"2024-07-17T07:16:04.447663+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:01.263102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418233+0000","last_change":"2024-07-17T07:16:03.403575+0000","last_active":"2024-07-17T07:16:05.418233+0000","last_peered":"2024-07-17T07:16:05.418233+0000","last_clean":"2024-07-17T07:16:05.418233+0000","last_became_active":"2024-07-17T07:16:03.403259+0000","last_became_peered":"2024-07-17T07:16:03.403259+0000","last_unstale":"2024-07-17T07:16:05.418233+0000","last_undegraded":"2024-07-17T07:16:05.418233+0000","last_fullsized":"2024-07-17T07:16:05.418233+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:52.907978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413744+0000","last_change":"2024-07-17T07:16:04.428753+0000","last_active":"2024-07-17T07:16:05.413744+0000","last_peered":"2024-07-17T07:16:05.413744+0000","last_clean":"2024-07-17T07:16:05.413744+0000","last_became_active":"2024-07-17T07:16:04.428441+0000","last_became_peered":"2024-07-17T07:16:04.428441+0000","last_unstale":"2024-07-17T07:16:05.413744+0000","last_undegraded":"2024-07-17T07:16:05.413744+0000","last_fullsized":"2024-07-17T07:16:05.413744+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:33.076840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431477+0000","last_change":"2024-07-17T07:16:03.407565+0000","last_active":"2024-07-17T07:16:05.431477+0000","last_peered":"2024-07-17T07:16:05.431477+0000","last_clean":"2024-07-17T07:16:05.431477+0000","last_became_active":"2024-07-17T07:16:03.407247+0000","last_became_peered":"2024-07-17T07:16:03.407247+0000","last_unstale":"2024-07-17T07:16:05.431477+0000","last_undegraded":"2024-07-17T07:16:05.431477+0000","last_fullsized":"2024-07-17T07:16:05.431477+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:37.273651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772279+0000","last_change":"2024-07-17T07:16:04.441854+0000","last_active":"2024-07-17T07:16:05.772279+0000","last_peered":"2024-07-17T07:16:05.772279+0000","last_clean":"2024-07-17T07:16:05.772279+0000","last_became_active":"2024-07-17T07:16:04.441559+0000","last_became_peered":"2024-07-17T07:16:04.441559+0000","last_unstale":"2024-07-17T07:16:05.772279+0000","last_undegraded":"2024-07-17T07:16:05.772279+0000","last_fullsized":"2024-07-17T07:16:05.772279+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:03.174116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414215+0000","last_change":"2024-07-17T07:16:03.401502+0000","last_active":"2024-07-17T07:16:05.414215+0000","last_peered":"2024-07-17T07:16:05.414215+0000","last_clean":"2024-07-17T07:16:05.414215+0000","last_became_active":"2024-07-17T07:16:03.401219+0000","last_became_peered":"2024-07-17T07:16:03.401219+0000","last_unstale":"2024-07-17T07:16:05.414215+0000","last_undegraded":"2024-07-17T07:16:05.414215+0000","last_fullsized":"2024-07-17T07:16:05.414215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:06.711274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419593+0000","last_change":"2024-07-17T07:16:04.426331+0000","last_active":"2024-07-17T07:16:05.419593+0000","last_peered":"2024-07-17T07:16:05.419593+0000","last_clean":"2024-07-17T07:16:05.419593+0000","last_became_active":"2024-07-17T07:16:04.425812+0000","last_became_peered":"2024-07-17T07:16:04.425812+0000","last_unstale":"2024-07-17T07:16:05.419593+0000","last_undegraded":"2024-07-17T07:16:05.419593+0000","last_fullsized":"2024-07-17T07:16:05.419593+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:35:19.791752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.450401+0000","last_change":"2024-07-17T07:16:03.406006+0000","last_active":"2024-07-17T07:16:04.450401+0000","last_peered":"2024-07-17T07:16:04.450401+0000","last_clean":"2024-07-17T07:16:04.450401+0000","last_became_active":"2024-07-17T07:16:03.404727+0000","last_became_peered":"2024-07-17T07:16:03.404727+0000","last_unstale":"2024-07-17T07:16:04.450401+0000","last_undegraded":"2024-07-17T07:16:04.450401+0000","last_fullsized":"2024-07-17T07:16:04.450401+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:29:20.909658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420010+0000","last_change":"2024-07-17T07:16:04.431989+0000","last_active":"2024-07-17T07:16:05.420010+0000","last_peered":"2024-07-17T07:16:05.420010+0000","last_clean":"2024-07-17T07:16:05.420010+0000","last_became_active":"2024-07-17T07:16:04.431596+0000","last_became_peered":"2024-07-17T07:16:04.431596+0000","last_unstale":"2024-07-17T07:16:05.420010+0000","last_undegraded":"2024-07-17T07:16:05.420010+0000","last_fullsized":"2024-07-17T07:16:05.420010+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:32.084265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"154'6","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.453193+0000","last_change":"2024-07-17T07:16:03.403387+0000","last_active":"2024-07-17T07:16:04.453193+0000","last_peered":"2024-07-17T07:16:04.453193+0000","last_clean":"2024-07-17T07:16:04.453193+0000","last_became_active":"2024-07-17T07:16:03.403122+0000","last_became_peered":"2024-07-17T07:16:03.403122+0000","last_unstale":"2024-07-17T07:16:04.453193+0000","last_undegraded":"2024-07-17T07:16:04.453193+0000","last_fullsized":"2024-07-17T07:16:04.453193+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:47.591234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.679307+0000","last_change":"2024-07-17T07:16:04.427357+0000","last_active":"2024-07-17T07:16:05.679307+0000","last_peered":"2024-07-17T07:16:05.679307+0000","last_clean":"2024-07-17T07:16:05.679307+0000","last_became_active":"2024-07-17T07:16:04.425814+0000","last_became_peered":"2024-07-17T07:16:04.425814+0000","last_unstale":"2024-07-17T07:16:05.679307+0000","last_undegraded":"2024-07-17T07:16:05.679307+0000","last_fullsized":"2024-07-17T07:16:05.679307+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:40.190741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.413986+0000","last_change":"2024-07-17T07:16:03.395142+0000","last_active":"2024-07-17T07:16:05.413986+0000","last_peered":"2024-07-17T07:16:05.413986+0000","last_clean":"2024-07-17T07:16:05.413986+0000","last_became_active":"2024-07-17T07:16:03.394374+0000","last_became_peered":"2024-07-17T07:16:03.394374+0000","last_unstale":"2024-07-17T07:16:05.413986+0000","last_undegraded":"2024-07-17T07:16:05.413986+0000","last_fullsized":"2024-07-17T07:16:05.413986+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:34:38.779860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439570+0000","last_change":"2024-07-17T07:16:04.424235+0000","last_active":"2024-07-17T07:16:04.439570+0000","last_peered":"2024-07-17T07:16:04.439570+0000","last_clean":"2024-07-17T07:16:04.439570+0000","last_became_active":"2024-07-17T07:16:04.423885+0000","last_became_peered":"2024-07-17T07:16:04.423885+0000","last_unstale":"2024-07-17T07:16:04.439570+0000","last_undegraded":"2024-07-17T07:16:04.439570+0000","last_fullsized":"2024-07-17T07:16:04.439570+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:14.345455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417898+0000","last_change":"2024-07-17T07:16:03.397100+0000","last_active":"2024-07-17T07:16:05.417898+0000","last_peered":"2024-07-17T07:16:05.417898+0000","last_clean":"2024-07-17T07:16:05.417898+0000","last_became_active":"2024-07-17T07:16:03.396805+0000","last_became_peered":"2024-07-17T07:16:03.396805+0000","last_unstale":"2024-07-17T07:16:05.417898+0000","last_undegraded":"2024-07-17T07:16:05.417898+0000","last_fullsized":"2024-07-17T07:16:05.417898+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:23.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.442566+0000","last_change":"2024-07-17T07:16:04.435956+0000","last_active":"2024-07-17T07:16:04.442566+0000","last_peered":"2024-07-17T07:16:04.442566+0000","last_clean":"2024-07-17T07:16:04.442566+0000","last_became_active":"2024-07-17T07:16:04.435646+0000","last_became_peered":"2024-07-17T07:16:04.435646+0000","last_unstale":"2024-07-17T07:16:04.442566+0000","last_undegraded":"2024-07-17T07:16:04.442566+0000","last_fullsized":"2024-07-17T07:16:04.442566+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:27.867367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"154'8","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.773010+0000","last_change":"2024-07-17T07:16:03.395816+0000","last_active":"2024-07-17T07:16:05.773010+0000","last_peered":"2024-07-17T07:16:05.773010+0000","last_clean":"2024-07-17T07:16:05.773010+0000","last_became_active":"2024-07-17T07:16:03.395526+0000","last_became_peered":"2024-07-17T07:16:03.395526+0000","last_unstale":"2024-07-17T07:16:05.773010+0000","last_undegraded":"2024-07-17T07:16:05.773010+0000","last_fullsized":"2024-07-17T07:16:05.773010+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:03.033041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.418127+0000","last_change":"2024-07-17T07:16:04.430043+0000","last_active":"2024-07-17T07:16:05.418127+0000","last_peered":"2024-07-17T07:16:05.418127+0000","last_clean":"2024-07-17T07:16:05.418127+0000","last_became_active":"2024-07-17T07:16:04.429499+0000","last_became_peered":"2024-07-17T07:16:04.429499+0000","last_unstale":"2024-07-17T07:16:05.418127+0000","last_undegraded":"2024-07-17T07:16:05.418127+0000","last_fullsized":"2024-07-17T07:16:05.418127+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:03.174116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431467+0000","last_change":"2024-07-17T07:16:03.401423+0000","last_active":"2024-07-17T07:16:05.431467+0000","last_peered":"2024-07-17T07:16:05.431467+0000","last_clean":"2024-07-17T07:16:05.431467+0000","last_became_active":"2024-07-17T07:16:03.401099+0000","last_became_peered":"2024-07-17T07:16:03.401099+0000","last_unstale":"2024-07-17T07:16:05.431467+0000","last_undegraded":"2024-07-17T07:16:05.431467+0000","last_fullsized":"2024-07-17T07:16:05.431467+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:40.281264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.439437+0000","last_change":"2024-07-17T07:16:04.417937+0000","last_active":"2024-07-17T07:16:04.439437+0000","last_peered":"2024-07-17T07:16:04.439437+0000","last_clean":"2024-07-17T07:16:04.439437+0000","last_became_active":"2024-07-17T07:16:04.412375+0000","last_became_peered":"2024-07-17T07:16:04.412375+0000","last_unstale":"2024-07-17T07:16:04.439437+0000","last_undegraded":"2024-07-17T07:16:04.439437+0000","last_fullsized":"2024-07-17T07:16:04.439437+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:07.122238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419148+0000","last_change":"2024-07-17T07:16:03.397323+0000","last_active":"2024-07-17T07:16:05.419148+0000","last_peered":"2024-07-17T07:16:05.419148+0000","last_clean":"2024-07-17T07:16:05.419148+0000","last_became_active":"2024-07-17T07:16:03.397027+0000","last_became_peered":"2024-07-17T07:16:03.397027+0000","last_unstale":"2024-07-17T07:16:05.419148+0000","last_undegraded":"2024-07-17T07:16:05.419148+0000","last_fullsized":"2024-07-17T07:16:05.419148+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:39.349083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447283+0000","last_change":"2024-07-17T07:16:04.432346+0000","last_active":"2024-07-17T07:16:04.447283+0000","last_peered":"2024-07-17T07:16:04.447283+0000","last_clean":"2024-07-17T07:16:04.447283+0000","last_became_active":"2024-07-17T07:16:04.430603+0000","last_became_peered":"2024-07-17T07:16:04.430603+0000","last_unstale":"2024-07-17T07:16:04.447283+0000","last_undegraded":"2024-07-17T07:16:04.447283+0000","last_fullsized":"2024-07-17T07:16:04.447283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:48.892160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772818+0000","last_change":"2024-07-17T07:16:03.407915+0000","last_active":"2024-07-17T07:16:05.772818+0000","last_peered":"2024-07-17T07:16:05.772818+0000","last_clean":"2024-07-17T07:16:05.772818+0000","last_became_active":"2024-07-17T07:16:03.407350+0000","last_became_peered":"2024-07-17T07:16:03.407350+0000","last_unstale":"2024-07-17T07:16:05.772818+0000","last_undegraded":"2024-07-17T07:16:05.772818+0000","last_fullsized":"2024-07-17T07:16:05.772818+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:20:52.907978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678295+0000","last_change":"2024-07-17T07:16:04.428996+0000","last_active":"2024-07-17T07:16:05.678295+0000","last_peered":"2024-07-17T07:16:05.678295+0000","last_clean":"2024-07-17T07:16:05.678295+0000","last_became_active":"2024-07-17T07:16:04.428466+0000","last_became_peered":"2024-07-17T07:16:04.428466+0000","last_unstale":"2024-07-17T07:16:05.678295+0000","last_undegraded":"2024-07-17T07:16:05.678295+0000","last_fullsized":"2024-07-17T07:16:05.678295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:44.283225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.395111+0000","last_change":"2024-07-17T07:16:03.405437+0000","last_active":"2024-07-17T07:16:04.395111+0000","last_peered":"2024-07-17T07:16:04.395111+0000","last_clean":"2024-07-17T07:16:04.395111+0000","last_became_active":"2024-07-17T07:16:03.405155+0000","last_became_peered":"2024-07-17T07:16:03.405155+0000","last_unstale":"2024-07-17T07:16:04.395111+0000","last_undegraded":"2024-07-17T07:16:04.395111+0000","last_fullsized":"2024-07-17T07:16:04.395111+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:16.382269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.414282+0000","last_change":"2024-07-17T07:16:04.412413+0000","last_active":"2024-07-17T07:16:04.414282+0000","last_peered":"2024-07-17T07:16:04.414282+0000","last_clean":"2024-07-17T07:16:04.414282+0000","last_became_active":"2024-07-17T07:16:04.411797+0000","last_became_peered":"2024-07-17T07:16:04.411797+0000","last_unstale":"2024-07-17T07:16:04.414282+0000","last_undegraded":"2024-07-17T07:16:04.414282+0000","last_fullsized":"2024-07-17T07:16:04.414282+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:43.231108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.420714+0000","last_change":"2024-07-17T07:16:03.396112+0000","last_active":"2024-07-17T07:16:05.420714+0000","last_peered":"2024-07-17T07:16:05.420714+0000","last_clean":"2024-07-17T07:16:05.420714+0000","last_became_active":"2024-07-17T07:16:03.395809+0000","last_became_peered":"2024-07-17T07:16:03.395809+0000","last_unstale":"2024-07-17T07:16:05.420714+0000","last_undegraded":"2024-07-17T07:16:05.420714+0000","last_fullsized":"2024-07-17T07:16:05.420714+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:19:59.378950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414295+0000","last_change":"2024-07-17T07:16:04.443194+0000","last_active":"2024-07-17T07:16:05.414295+0000","last_peered":"2024-07-17T07:16:05.414295+0000","last_clean":"2024-07-17T07:16:05.414295+0000","last_became_active":"2024-07-17T07:16:04.442319+0000","last_became_peered":"2024-07-17T07:16:04.442319+0000","last_unstale":"2024-07-17T07:16:05.414295+0000","last_undegraded":"2024-07-17T07:16:05.414295+0000","last_fullsized":"2024-07-17T07:16:05.414295+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T07:24:17.714355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414170+0000","last_change":"2024-07-17T07:16:03.387769+0000","last_active":"2024-07-17T07:16:05.414170+0000","last_peered":"2024-07-17T07:16:05.414170+0000","last_clean":"2024-07-17T07:16:05.414170+0000","last_became_active":"2024-07-17T07:16:03.386309+0000","last_became_peered":"2024-07-17T07:16:03.386309+0000","last_unstale":"2024-07-17T07:16:05.414170+0000","last_undegraded":"2024-07-17T07:16:05.414170+0000","last_fullsized":"2024-07-17T07:16:05.414170+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:18.367156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.440121+0000","last_change":"2024-07-17T07:16:04.429704+0000","last_active":"2024-07-17T07:16:04.440121+0000","last_peered":"2024-07-17T07:16:04.440121+0000","last_clean":"2024-07-17T07:16:04.440121+0000","last_became_active":"2024-07-17T07:16:04.429395+0000","last_became_peered":"2024-07-17T07:16:04.429395+0000","last_unstale":"2024-07-17T07:16:04.440121+0000","last_undegraded":"2024-07-17T07:16:04.440121+0000","last_fullsized":"2024-07-17T07:16:04.440121+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:09:29.238417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.399521+0000","last_change":"2024-07-17T07:16:03.401590+0000","last_active":"2024-07-17T07:16:04.399521+0000","last_peered":"2024-07-17T07:16:04.399521+0000","last_clean":"2024-07-17T07:16:04.399521+0000","last_became_active":"2024-07-17T07:16:03.400752+0000","last_became_peered":"2024-07-17T07:16:03.400752+0000","last_unstale":"2024-07-17T07:16:04.399521+0000","last_undegraded":"2024-07-17T07:16:04.399521+0000","last_fullsized":"2024-07-17T07:16:04.399521+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:33.470831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431350+0000","last_change":"2024-07-17T07:16:04.429287+0000","last_active":"2024-07-17T07:16:05.431350+0000","last_peered":"2024-07-17T07:16:05.431350+0000","last_clean":"2024-07-17T07:16:05.431350+0000","last_became_active":"2024-07-17T07:16:04.428975+0000","last_became_peered":"2024-07-17T07:16:04.428975+0000","last_unstale":"2024-07-17T07:16:05.431350+0000","last_undegraded":"2024-07-17T07:16:05.431350+0000","last_fullsized":"2024-07-17T07:16:05.431350+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:48.339033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.773126+0000","last_change":"2024-07-17T07:16:03.396736+0000","last_active":"2024-07-17T07:16:05.773126+0000","last_peered":"2024-07-17T07:16:05.773126+0000","last_clean":"2024-07-17T07:16:05.773126+0000","last_became_active":"2024-07-17T07:16:03.396341+0000","last_became_peered":"2024-07-17T07:16:03.396341+0000","last_unstale":"2024-07-17T07:16:05.773126+0000","last_undegraded":"2024-07-17T07:16:05.773126+0000","last_fullsized":"2024-07-17T07:16:05.773126+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:55.077860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.417802+0000","last_change":"2024-07-17T07:16:04.414006+0000","last_active":"2024-07-17T07:16:04.417802+0000","last_peered":"2024-07-17T07:16:04.417802+0000","last_clean":"2024-07-17T07:16:04.417802+0000","last_became_active":"2024-07-17T07:16:04.413590+0000","last_became_peered":"2024-07-17T07:16:04.413590+0000","last_unstale":"2024-07-17T07:16:04.417802+0000","last_undegraded":"2024-07-17T07:16:04.417802+0000","last_fullsized":"2024-07-17T07:16:04.417802+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:46.520338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.414541+0000","last_change":"2024-07-17T07:16:03.395402+0000","last_active":"2024-07-17T07:16:05.414541+0000","last_peered":"2024-07-17T07:16:05.414541+0000","last_clean":"2024-07-17T07:16:05.414541+0000","last_became_active":"2024-07-17T07:16:03.394864+0000","last_became_peered":"2024-07-17T07:16:03.394864+0000","last_unstale":"2024-07-17T07:16:05.414541+0000","last_undegraded":"2024-07-17T07:16:05.414541+0000","last_fullsized":"2024-07-17T07:16:05.414541+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:49.714981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.417850+0000","last_change":"2024-07-17T07:16:04.431964+0000","last_active":"2024-07-17T07:16:05.417850+0000","last_peered":"2024-07-17T07:16:05.417850+0000","last_clean":"2024-07-17T07:16:05.417850+0000","last_became_active":"2024-07-17T07:16:04.431665+0000","last_became_peered":"2024-07-17T07:16:04.431665+0000","last_unstale":"2024-07-17T07:16:05.417850+0000","last_undegraded":"2024-07-17T07:16:05.417850+0000","last_fullsized":"2024-07-17T07:16:05.417850+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:13.828577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.429554+0000","last_change":"2024-07-17T07:16:03.398335+0000","last_active":"2024-07-17T07:16:05.429554+0000","last_peered":"2024-07-17T07:16:05.429554+0000","last_clean":"2024-07-17T07:16:05.429554+0000","last_became_active":"2024-07-17T07:16:03.398040+0000","last_became_peered":"2024-07-17T07:16:03.398040+0000","last_unstale":"2024-07-17T07:16:05.429554+0000","last_undegraded":"2024-07-17T07:16:05.429554+0000","last_fullsized":"2024-07-17T07:16:05.429554+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:41.665116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447253+0000","last_change":"2024-07-17T07:16:04.431016+0000","last_active":"2024-07-17T07:16:04.447253+0000","last_peered":"2024-07-17T07:16:04.447253+0000","last_clean":"2024-07-17T07:16:04.447253+0000","last_became_active":"2024-07-17T07:16:04.430201+0000","last_became_peered":"2024-07-17T07:16:04.430201+0000","last_unstale":"2024-07-17T07:16:04.447253+0000","last_undegraded":"2024-07-17T07:16:04.447253+0000","last_fullsized":"2024-07-17T07:16:04.447253+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:54.981252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.421441+0000","last_change":"2024-07-17T07:16:03.386171+0000","last_active":"2024-07-17T07:16:05.421441+0000","last_peered":"2024-07-17T07:16:05.421441+0000","last_clean":"2024-07-17T07:16:05.421441+0000","last_became_active":"2024-07-17T07:16:03.385707+0000","last_became_peered":"2024-07-17T07:16:03.385707+0000","last_unstale":"2024-07-17T07:16:05.421441+0000","last_undegraded":"2024-07-17T07:16:05.421441+0000","last_fullsized":"2024-07-17T07:16:05.421441+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:27.306260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.678586+0000","last_change":"2024-07-17T07:16:04.441192+0000","last_active":"2024-07-17T07:16:05.678586+0000","last_peered":"2024-07-17T07:16:05.678586+0000","last_clean":"2024-07-17T07:16:05.678586+0000","last_became_active":"2024-07-17T07:16:04.440884+0000","last_became_peered":"2024-07-17T07:16:04.440884+0000","last_unstale":"2024-07-17T07:16:05.678586+0000","last_undegraded":"2024-07-17T07:16:05.678586+0000","last_fullsized":"2024-07-17T07:16:05.678586+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:16.353468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772605+0000","last_change":"2024-07-17T07:16:03.399109+0000","last_active":"2024-07-17T07:16:05.772605+0000","last_peered":"2024-07-17T07:16:05.772605+0000","last_clean":"2024-07-17T07:16:05.772605+0000","last_became_active":"2024-07-17T07:16:03.398828+0000","last_became_peered":"2024-07-17T07:16:03.398828+0000","last_unstale":"2024-07-17T07:16:05.772605+0000","last_undegraded":"2024-07-17T07:16:05.772605+0000","last_fullsized":"2024-07-17T07:16:05.772605+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:39.349083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772280+0000","last_change":"2024-07-17T07:16:04.434311+0000","last_active":"2024-07-17T07:16:05.772280+0000","last_peered":"2024-07-17T07:16:05.772280+0000","last_clean":"2024-07-17T07:16:05.772280+0000","last_became_active":"2024-07-17T07:16:04.433407+0000","last_became_peered":"2024-07-17T07:16:04.433407+0000","last_unstale":"2024-07-17T07:16:05.772280+0000","last_undegraded":"2024-07-17T07:16:05.772280+0000","last_fullsized":"2024-07-17T07:16:05.772280+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:29.128517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.431195+0000","last_change":"2024-07-17T07:16:03.398607+0000","last_active":"2024-07-17T07:16:05.431195+0000","last_peered":"2024-07-17T07:16:05.431195+0000","last_clean":"2024-07-17T07:16:05.431195+0000","last_became_active":"2024-07-17T07:16:03.397930+0000","last_became_peered":"2024-07-17T07:16:03.397930+0000","last_unstale":"2024-07-17T07:16:05.431195+0000","last_undegraded":"2024-07-17T07:16:05.431195+0000","last_fullsized":"2024-07-17T07:16:05.431195+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:11:24.272699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772488+0000","last_change":"2024-07-17T07:16:04.431162+0000","last_active":"2024-07-17T07:16:05.772488+0000","last_peered":"2024-07-17T07:16:05.772488+0000","last_clean":"2024-07-17T07:16:05.772488+0000","last_became_active":"2024-07-17T07:16:04.430847+0000","last_became_peered":"2024-07-17T07:16:04.430847+0000","last_unstale":"2024-07-17T07:16:05.772488+0000","last_undegraded":"2024-07-17T07:16:05.772488+0000","last_fullsized":"2024-07-17T07:16:05.772488+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:30.086416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.401320+0000","last_change":"2024-07-17T07:16:03.405465+0000","last_active":"2024-07-17T07:16:04.401320+0000","last_peered":"2024-07-17T07:16:04.401320+0000","last_clean":"2024-07-17T07:16:04.401320+0000","last_became_active":"2024-07-17T07:16:03.405184+0000","last_became_peered":"2024-07-17T07:16:03.405184+0000","last_unstale":"2024-07-17T07:16:04.401320+0000","last_undegraded":"2024-07-17T07:16:04.401320+0000","last_fullsized":"2024-07-17T07:16:04.401320+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:19.810051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772991+0000","last_change":"2024-07-17T07:16:04.442153+0000","last_active":"2024-07-17T07:16:05.772991+0000","last_peered":"2024-07-17T07:16:05.772991+0000","last_clean":"2024-07-17T07:16:05.772991+0000","last_became_active":"2024-07-17T07:16:04.441875+0000","last_became_peered":"2024-07-17T07:16:04.441875+0000","last_unstale":"2024-07-17T07:16:05.772991+0000","last_undegraded":"2024-07-17T07:16:05.772991+0000","last_fullsized":"2024-07-17T07:16:05.772991+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.039380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.772209+0000","last_change":"2024-07-17T07:16:03.405492+0000","last_active":"2024-07-17T07:16:05.772209+0000","last_peered":"2024-07-17T07:16:05.772209+0000","last_clean":"2024-07-17T07:16:05.772209+0000","last_became_active":"2024-07-17T07:16:03.405201+0000","last_became_peered":"2024-07-17T07:16:03.405201+0000","last_unstale":"2024-07-17T07:16:05.772209+0000","last_undegraded":"2024-07-17T07:16:05.772209+0000","last_fullsized":"2024-07-17T07:16:05.772209+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:23.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.447421+0000","last_change":"2024-07-17T07:16:04.436149+0000","last_active":"2024-07-17T07:16:04.447421+0000","last_peered":"2024-07-17T07:16:04.447421+0000","last_clean":"2024-07-17T07:16:04.447421+0000","last_became_active":"2024-07-17T07:16:04.429030+0000","last_became_peered":"2024-07-17T07:16:04.429030+0000","last_unstale":"2024-07-17T07:16:04.447421+0000","last_undegraded":"2024-07-17T07:16:04.447421+0000","last_fullsized":"2024-07-17T07:16:04.447421+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:49.830227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.393841+0000","last_change":"2024-07-17T07:16:03.405566+0000","last_active":"2024-07-17T07:16:04.393841+0000","last_peered":"2024-07-17T07:16:04.393841+0000","last_clean":"2024-07-17T07:16:04.393841+0000","last_became_active":"2024-07-17T07:16:03.404684+0000","last_became_peered":"2024-07-17T07:16:03.404684+0000","last_unstale":"2024-07-17T07:16:04.393841+0000","last_undegraded":"2024-07-17T07:16:04.393841+0000","last_fullsized":"2024-07-17T07:16:04.393841+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:20.318729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-07-17T07:16:05.419974+0000","last_change":"2024-07-17T07:16:04.430647+0000","last_active":"2024-07-17T07:16:05.419974+0000","last_peered":"2024-07-17T07:16:05.419974+0000","last_clean":"2024-07-17T07:16:05.419974+0000","last_became_active":"2024-07-17T07:16:04.430072+0000","last_became_peered":"2024-07-17T07:16:04.430072+0000","last_unstale":"2024-07-17T07:16:05.419974+0000","last_undegraded":"2024-07-17T07:16:05.419974+0000","last_fullsized":"2024-07-17T07:16:05.419974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:03.355499+0000","last_clean_scrub_stamp":"2024-07-17T07:16:03.355499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:41.650990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.400003+0000","last_change":"2024-07-17T07:16:03.401772+0000","last_active":"2024-07-17T07:16:04.400003+0000","last_peered":"2024-07-17T07:16:04.400003+0000","last_clean":"2024-07-17T07:16:04.400003+0000","last_became_active":"2024-07-17T07:16:03.401464+0000","last_became_peered":"2024-07-17T07:16:03.401464+0000","last_unstale":"2024-07-17T07:16:04.400003+0000","last_undegraded":"2024-07-17T07:16:04.400003+0000","last_fullsized":"2024-07-17T07:16:04.400003+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T07:16:02.338392+0000","last_clean_scrub_stamp":"2024-07-17T07:16:02.338392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:42.033603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"145'2448","reported_seq":2991,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-07-17T07:16:04.397649+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T07:16:04.397649+0000","last_peered":"2024-07-17T07:16:04.397649+0000","last_clean":"2024-07-17T07:16:04.397649+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T07:16:04.397649+0000","last_undegraded":"2024-07-17T07:16:04.397649+0000","last_fullsized":"2024-07-17T07:16:04.397649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":251,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":847,"num_read_kb":2287,"num_write":251,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2448,"ondisk_log_size":2448,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542395,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55860,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315980,"statfs":{"total":96636764160,"available":96579563520,"internally_reserved":0,"allocated":2523136,"data_stored":1611312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509690,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1933312,"data_stored":1027154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509691,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55280,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316560,"statfs":{"total":96636764160,"available":96580157440,"internally_reserved":0,"allocated":1929216,"data_stored":1026504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542396,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1026482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509690,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1929216,"data_stored":1020924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542394,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1933312,"data_stored":1027098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509691,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1929216,"data_stored":1026552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509690,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2514944,"data_stored":1609666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T07:16:09.418 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T07:16:09.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 -- 172.21.15.44:0/902712376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11fc004e60 msgr2=0x7f11fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 --2- 172.21.15.44:0/902712376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11fc004e60 0x7f11fc00b9c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f121000ad20 tx=0x7f11fc005830 comp rx=0 tx=0).stop 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f1216800640 1 -- 172.21.15.44:0/902712376 reap_dead start 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 -- 172.21.15.44:0/902712376 shutdown_connections 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 -- 172.21.15.44:0/902712376 >> 172.21.15.44:0/902712376 conn(0x7f12180b0eb0 msgr2=0x7f12180b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 -- 172.21.15.44:0/902712376 shutdown_connections 2024-07-17T07:16:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.683+0000 7f121d000640 1 -- 172.21.15.44:0/902712376 wait complete. 2024-07-17T07:16:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.685+0000 7f121d000640 1 Processor -- start 2024-07-17T07:16:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.685+0000 7f121d000640 1 -- start start 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11fc004e60 0x7f1218073900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12181fbd50 0x7f1218150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f121815c470 con 0x7f12181fb8f0 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f121815c2f0 con 0x7f12181fbd50 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f121d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f121815c5f0 con 0x7f11fc004e60 2024-07-17T07:16:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f1215400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f1215400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41312/0 (socket says 172.21.15.44:41312) 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.686+0000 7f1215400640 1 -- 172.21.15.44:0/2492950975 learned_addr learned my addr 172.21.15.44:0/2492950975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 -- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11fc004e60 msgr2=0x7f1218073900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11fc004e60 0x7f1218073900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 -- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12181fbd50 msgr2=0x7f1218150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12181fbd50 0x7f1218150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 -- 172.21.15.44:0/2492950975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12181fc850 con 0x7f12181fb8f0 2024-07-17T07:16:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f1215400640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f120400c840 tx=0x7f120400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204017070 con 0x7f12181fb8f0 2024-07-17T07:16:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12040035f0 con 0x7f12181fb8f0 2024-07-17T07:16:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204003a20 con 0x7f12181fb8f0 2024-07-17T07:16:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.687+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1218150cc0 con 0x7f12181fb8f0 2024-07-17T07:16:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.688+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181529d0 con 0x7f12181fb8f0 2024-07-17T07:16:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.689+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12180cd2e0 con 0x7f12181fb8f0 2024-07-17T07:16:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.690+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f120401a070 con 0x7f12181fb8f0 2024-07-17T07:16:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.691+0000 7f120e000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 0x7f11f80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.691+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f1204097e30 con 0x7f12181fb8f0 2024-07-17T07:16:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.692+0000 7f1215e00640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 0x7f11f80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.693+0000 7f1215e00640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 0x7f11f80791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1208003ce0 tx=0x7f12080069c0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.695+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1204060e30 con 0x7f12181fb8f0 2024-07-17T07:16:09.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.985+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f12180ce3d0 con 0x7f12181fb8f0 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.990+0000 7f120e000640 1 -- 172.21.15.44:0/2492950975 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) ==== 190+0+147 (secure 0 0 0) 0x7f1204069070 con 0x7f12181fb8f0 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T07:16:09.989 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.992+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 msgr2=0x7f11f80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.992+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 0x7f11f80791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1208003ce0 tx=0x7f12080069c0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.992+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 msgr2=0x7f1218150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.992+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f120400c840 tx=0x7f120400cd00 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 shutdown_connections 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f11f8076d00 0x7f11f80791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12181fbd50 0x7f1218150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12181fb8f0 0x7f1218150240 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 --2- 172.21.15.44:0/2492950975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11fc004e60 0x7f1218073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 >> 172.21.15.44:0/2492950975 conn(0x7f12180b0eb0 msgr2=0x7f12180b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 shutdown_connections 2024-07-17T07:16:09.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:09.993+0000 7f121d000640 1 -- 172.21.15.44:0/2492950975 wait complete. 2024-07-17T07:16:10.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 -- 172.21.15.44:0/3707249566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b6e0 msgr2=0x7f88940cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 --2- 172.21.15.44:0/3707249566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b6e0 0x7f88940cf380 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f887c00a940 tx=0x7f887c02b7d0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 -- 172.21.15.44:0/3707249566 shutdown_connections 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 --2- 172.21.15.44:0/3707249566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b6e0 0x7f88940cf380 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 --2- 172.21.15.44:0/3707249566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b300 0x7f889406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.211+0000 7f8893e00640 1 -- 172.21.15.44:0/3707249566 >> 172.21.15.44:0/3707249566 conn(0x7f88940b0e80 msgr2=0x7f88940b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.212+0000 7f8893e00640 1 -- 172.21.15.44:0/3707249566 shutdown_connections 2024-07-17T07:16:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.212+0000 7f8893e00640 1 -- 172.21.15.44:0/3707249566 wait complete. 2024-07-17T07:16:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.213+0000 7f8893e00640 1 Processor -- start 2024-07-17T07:16:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.214+0000 7f8893e00640 1 -- start start 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b6e0 0x7f88941f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88940cf8c0 0x7f88940afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88940b2120 con 0x7f889409b300 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88940b1fa0 con 0x7f88940cf8c0 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88940b22a0 con 0x7f889409b6e0 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8892a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8892a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41340/0 (socket says 172.21.15.44:41340) 2024-07-17T07:16:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8892a00640 1 -- 172.21.15.44:0/566112038 learned_addr learned my addr 172.21.15.44:0/566112038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:10.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f888a000640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b6e0 0x7f88941f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.215+0000 7f8893400640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88940cf8c0 0x7f88940afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 -- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b6e0 msgr2=0x7f88941f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b6e0 0x7f88941f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 -- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88940cf8c0 msgr2=0x7f88940afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88940cf8c0 0x7f88940afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 -- 172.21.15.44:0/566112038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8894072ca0 con 0x7f889409b300 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8892a00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f888400eab0 tx=0x7f888400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888400ce40 con 0x7f889409b300 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8884011560 con 0x7f889409b300 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884011930 con 0x7f889409b300 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.216+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8894072f40 con 0x7f889409b300 2024-07-17T07:16:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.217+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8894073500 con 0x7f889409b300 2024-07-17T07:16:10.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.218+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8894073930 con 0x7f889409b300 2024-07-17T07:16:10.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.219+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f8884011ad0 con 0x7f889409b300 2024-07-17T07:16:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.220+0000 7f888b400640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 0x7f8870079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.220+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f8884099620 con 0x7f889409b300 2024-07-17T07:16:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.221+0000 7f888a000640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 0x7f8870079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:10.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.222+0000 7f888a000640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 0x7f8870079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f887c035000 tx=0x7f887c004730 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:10.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.225+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8884016030 con 0x7f889409b300 2024-07-17T07:16:10.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.474+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889406e060 con 0x7f889409b300 2024-07-17T07:16:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.475+0000 7f888b400640 1 -- 172.21.15.44:0/566112038 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7f8884061ec0 con 0x7f889409b300 2024-07-17T07:16:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:10.475 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T07:16:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T07:16:10.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 msgr2=0x7f8870079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:10.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 0x7f8870079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f887c035000 tx=0x7f887c004730 comp rx=0 tx=0).stop 2024-07-17T07:16:10.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 msgr2=0x7f88941e4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f888400eab0 tx=0x7f888400ef70 comp rx=0 tx=0).stop 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 shutdown_connections 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f8870076cd0 0x7f8870079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88940cf8c0 0x7f88940afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.478+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f889409b6e0 0x7f88941f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.479+0000 7f8893e00640 1 --2- 172.21.15.44:0/566112038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f889409b300 0x7f88941e4540 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:10.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.479+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 >> 172.21.15.44:0/566112038 conn(0x7f88940b0e80 msgr2=0x7f88940b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:10.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.479+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 shutdown_connections 2024-07-17T07:16:10.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:10.479+0000 7f8893e00640 1 -- 172.21.15.44:0/566112038 wait complete. 2024-07-17T07:16:10.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T07:16:03.849433+0000', 'modified': '2024-07-17T07:16:06.948093+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6949}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6949': {'gid': 6949, 'name': 'd', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/569905425', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 569905425}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 569905425}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6949, 'qdb_cluster': [6949]} 2024-07-17T07:16:10.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T07:16:10.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-17T07:16:11.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 -- 172.21.15.44:0/326443538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 msgr2=0x7f4320003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 --2- 172.21.15.44:0/326443538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f4320003e50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f432c009b50 tx=0x7f4320005830 comp rx=0 tx=0).stop 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f433be00640 1 -- 172.21.15.44:0/326443538 reap_dead start 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 -- 172.21.15.44:0/326443538 shutdown_connections 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 -- 172.21.15.44:0/326443538 >> 172.21.15.44:0/326443538 conn(0x7f433c0b0eb0 msgr2=0x7f433c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 -- 172.21.15.44:0/326443538 shutdown_connections 2024-07-17T07:16:11.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.125+0000 7f4342400640 1 -- 172.21.15.44:0/326443538 wait complete. 2024-07-17T07:16:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.127+0000 7f4342400640 1 Processor -- start 2024-07-17T07:16:11.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.127+0000 7f4342400640 1 -- start start 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f433c1f27b0 0x7f433c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f433c155520 0x7f433c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f433c15c470 con 0x7f4320003a30 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f433c15c2f0 con 0x7f433c1f27b0 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f4342400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f433c15c5f0 con 0x7f433c155520 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41372/0 (socket says 172.21.15.44:41372) 2024-07-17T07:16:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.128+0000 7f433b400640 1 -- 172.21.15.44:0/328400405 learned_addr learned my addr 172.21.15.44:0/328400405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 -- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f433c155520 msgr2=0x7f433c0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f433c155520 0x7f433c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 -- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f433c1f27b0 msgr2=0x7f433c154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f433c1f27b0 0x7f433c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 -- 172.21.15.44:0/328400405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c141130 con 0x7f4320003a30 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f433b400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f432c009a70 tx=0x7f432c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f4333e00640 1 -- 172.21.15.44:0/328400405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c043070 con 0x7f4320003a30 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f4333e00640 1 -- 172.21.15.44:0/328400405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f432c004320 con 0x7f4320003a30 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f4333e00640 1 -- 172.21.15.44:0/328400405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c038040 con 0x7f4320003a30 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c141410 con 0x7f4320003a30 2024-07-17T07:16:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.129+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c1419d0 con 0x7f4320003a30 2024-07-17T07:16:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.132+0000 7f4333e00640 1 -- 172.21.15.44:0/328400405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7f432c02b8c0 con 0x7f4320003a30 2024-07-17T07:16:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.133+0000 7f4333e00640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 0x7f4318079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.133+0000 7f4333e00640 1 -- 172.21.15.44:0/328400405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f432c0b8cf0 con 0x7f4320003a30 2024-07-17T07:16:11.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.133+0000 7f43451e1740 1 Processor -- start 2024-07-17T07:16:11.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.134+0000 7f43451e1740 1 -- start start 2024-07-17T07:16:11.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.134+0000 7f4332a00640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 0x7f4318079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df2731b0 0x5616df24ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df2931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5616df1bad60 con 0x5616df2485f0 2024-07-17T07:16:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616df1aa260 con 0x5616df2731b0 2024-07-17T07:16:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5616df1ae2a0 con 0x5616df292df0 2024-07-17T07:16:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f433be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f4332a00640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 0x7f4318079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4324001860 tx=0x7f43240014f0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f433be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41376/0 (socket says 172.21.15.44:41376) 2024-07-17T07:16:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f433be00640 1 -- 172.21.15.44:0/430729149 learned_addr learned my addr 172.21.15.44:0/430729149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f433b400640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df2731b0 0x5616df24ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.135+0000 7f4332a00640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df2931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 -- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 msgr2=0x5616df2931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df2931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 -- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df2731b0 msgr2=0x5616df24ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df2731b0 0x5616df24ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 -- 172.21.15.44:0/430729149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616def0e850 con 0x5616df2485f0 2024-07-17T07:16:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f4332a00640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df2931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f433be00640 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f433c0d0260 tx=0x7f4334002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c749d45c6326234 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f432be00640 1 -- 172.21.15.44:0/430729149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334017070 con 0x5616df2485f0 2024-07-17T07:16:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f432be00640 1 -- 172.21.15.44:0/430729149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f433400c750 con 0x5616df2485f0 2024-07-17T07:16:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.136+0000 7f432be00640 1 -- 172.21.15.44:0/430729149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400cbe0 con 0x5616df2485f0 2024-07-17T07:16:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 -- 172.21.15.44:0/430729149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 msgr2=0x5616df248a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f433c0d0260 tx=0x7f4334002020 comp rx=0 tx=0).stop 2024-07-17T07:16:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 -- 172.21.15.44:0/430729149 shutdown_connections 2024-07-17T07:16:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2485f0 0x5616df248a40 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df2931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 --2- 172.21.15.44:0/430729149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df2731b0 0x5616df24ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 -- 172.21.15.44:0/430729149 >> 172.21.15.44:0/430729149 conn(0x5616df2496a0 msgr2=0x5616df24d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 -- 172.21.15.44:0/430729149 shutdown_connections 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.137+0000 7f43451e1740 1 -- 172.21.15.44:0/430729149 wait complete. 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.138+0000 7f43451e1740 1 -- start start 2024-07-17T07:16:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.138+0000 7f43451e1740 1 Processor -- start 2024-07-17T07:16:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df276740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df276c80 0x5616df26d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5616def1d620 con 0x5616df2731b0 2024-07-17T07:16:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616df151f90 con 0x5616df276c80 2024-07-17T07:16:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5616df247b30 con 0x5616df292df0 2024-07-17T07:16:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.139+0000 7f43451e1740 10 client.0 _ll_register_callbacks cb 0x5616df177f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:16:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41390/0 (socket says 172.21.15.44:41390) 2024-07-17T07:16:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 -- 172.21.15.44:0/1354870772 learned_addr learned my addr 172.21.15.44:0/1354870772 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 -- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 msgr2=0x5616df276740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df276740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 -- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df276c80 msgr2=0x5616df26d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df276c80 0x5616df26d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 -- 172.21.15.44:0/1354870772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616def856b0 con 0x5616df2731b0 2024-07-17T07:16:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f433b400640 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f432c0b6cb0 tx=0x7f432c081510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c1cd51cc16b70f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f4328c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:16:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.140+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0afe60 con 0x5616df2731b0 2024-07-17T07:16:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f432c081870 con 0x5616df2731b0 2024-07-17T07:16:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433400c750 con 0x5616df2731b0 2024-07-17T07:16:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f432c081120 con 0x5616df2731b0 2024-07-17T07:16:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 10 client.8508 fetch_fsmap learned FSMap version 79 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x5616df2a1870 con 0x5616df2731b0 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f433400c750 con 0x5616df2731b0 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 79) ==== 3502+0+0 (secure 0 0 0) 0x7f432c03b030 con 0x5616df2731b0 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 10 client.8508 fetch_fsmap finished waiting for FSMap version 79 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 10 client.8508 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 20 client.8508 populate_metadata read hostname 'smithi044' 2024-07-17T07:16:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df2ad6a0 0x5616df2afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 4 client.8508 mds_command: new command op to 6949 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.141+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 --> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616df1bccb0 con 0x5616df2ad6a0 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.142+0000 7f4332a00640 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df2ad6a0 0x5616df2afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.143+0000 7f4332a00640 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df2ad6a0 0x5616df2afa90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b774b881c231d32 server_cookie=e5dbfac502ed09dc in_seq=0 out_seq=0 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.143+0000 7f4328c00640 10 client.8508 ms_handle_connect on v2:172.21.15.175:6834/569905425 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f4328c00640 1 -- 172.21.15.44:0/1354870772 <== mds.0 v2:172.21.15.175:6834/569905425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5616df1bccb0 con 0x5616df2ad6a0 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f4328c00640 10 client.8508 handle_command_reply: tid=0 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 client.8508 shutdown 2024-07-17T07:16:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 msgr2=0x5616df28b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f432c0b6cb0 tx=0x7f432c081510 comp rx=0 tx=0).stop 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 shutdown_connections 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df2ad6a0 0x5616df2afa90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df276c80 0x5616df26d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df292df0 0x5616df276740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f4328c00640 0 client.8508 ms_handle_reset on v2:172.21.15.175:6834/569905425 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 --2- 172.21.15.44:0/1354870772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b4a0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.145+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 >> 172.21.15.44:0/1354870772 conn(0x5616df2485f0 msgr2=0x5616df242240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.146+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 shutdown_connections 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.146+0000 7f43451e1740 1 -- 172.21.15.44:0/1354870772 wait complete. 2024-07-17T07:16:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.146+0000 7f43451e1740 20 client.8508 trim_cache size 0 max 16384 2024-07-17T07:16:11.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.166+0000 7f43451e1740 1 Processor -- start 2024-07-17T07:16:11.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.166+0000 7f43451e1740 1 -- start start 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df273950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df273e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f433c1fca80 con 0x5616df2731b0 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0adcb0 con 0x5616df292df0 2024-07-17T07:16:11.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.167+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f433c1fc630 con 0x5616df276c80 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41392/0 (socket says 172.21.15.44:41392) 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 -- 172.21.15.44:0/2981766843 learned_addr learned my addr 172.21.15.44:0/2981766843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f4332a00640 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df273950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433be00640 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df273e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 -- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 msgr2=0x5616df273950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df273950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 -- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 msgr2=0x5616df273e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df273e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.168+0000 7f433b400640 1 -- 172.21.15.44:0/2981766843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616df23dee0 con 0x5616df2731b0 2024-07-17T07:16:11.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f433b400640 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f432c07e880 tx=0x7f432c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da355237dc7fdc81 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f432b400640 1 -- 172.21.15.44:0/2981766843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0afe60 con 0x5616df2731b0 2024-07-17T07:16:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f432b400640 1 -- 172.21.15.44:0/2981766843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f432c09f070 con 0x5616df2731b0 2024-07-17T07:16:11.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f432b400640 1 -- 172.21.15.44:0/2981766843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c085bb0 con 0x5616df2731b0 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 -- 172.21.15.44:0/2981766843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 msgr2=0x5616df28b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f432c07e880 tx=0x7f432c004510 comp rx=0 tx=0).stop 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 -- 172.21.15.44:0/2981766843 shutdown_connections 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df273e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df273950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.169+0000 7f43451e1740 1 --2- 172.21.15.44:0/2981766843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df28b040 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.170+0000 7f43451e1740 1 -- 172.21.15.44:0/2981766843 >> 172.21.15.44:0/2981766843 conn(0x5616df2485f0 msgr2=0x5616df2931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.170+0000 7f43451e1740 1 -- 172.21.15.44:0/2981766843 shutdown_connections 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.170+0000 7f43451e1740 1 -- 172.21.15.44:0/2981766843 wait complete. 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.170+0000 7f43451e1740 1 -- start start 2024-07-17T07:16:11.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.170+0000 7f43451e1740 1 Processor -- start 2024-07-17T07:16:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.171+0000 7f43451e1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.171+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df2770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f43451e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df277600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f43451e1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5616df275d40 con 0x5616df2731b0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616def1d620 con 0x5616df292df0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f43451e1740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5616df26cef0 con 0x5616df276c80 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f43451e1740 10 client.0 _ll_register_callbacks cb 0x7f432c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41406/0 (socket says 172.21.15.44:41406) 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 -- 172.21.15.44:0/482417038 learned_addr learned my addr 172.21.15.44:0/482417038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f4332a00640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df2770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433be00640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df277600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 -- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 msgr2=0x5616df2770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df2770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 -- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 msgr2=0x5616df277600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df277600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.172+0000 7f433b400640 1 -- 172.21.15.44:0/482417038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616df28e670 con 0x5616df2731b0 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f433b400640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f432c0812c0 tx=0x7f432c0813e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae58b19dc8a3969 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0afe60 con 0x5616df2731b0 2024-07-17T07:16:11.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f432c096d80 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432c09f070 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f432c080d80 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f43451e1740 10 client.8514 fetch_fsmap learned FSMap version 79 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) -- 0x5616df30acf0 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f432c09f070 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.173+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 79) ==== 3502+0+0 (secure 0 0 0) 0x7f432c080870 con 0x5616df2731b0 2024-07-17T07:16:11.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 10 client.8514 fetch_fsmap finished waiting for FSMap version 79 2024-07-17T07:16:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 10 client.8514 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T07:16:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 20 client.8514 populate_metadata read hostname 'smithi044' 2024-07-17T07:16:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df310d40 0x5616df311120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 4 client.8514 mds_command: new command op to 6949 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T07:16:11.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 --> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] -- command(tid 0: {"prefix": "status"}) -- 0x5616df1b8750 con 0x5616df310d40 2024-07-17T07:16:11.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.174+0000 7f4332a00640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df310d40 0x5616df311120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.175+0000 7f4332a00640 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df310d40 0x5616df311120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea5eb00833b06fc server_cookie=c867f69f21440504 in_seq=0 out_seq=0 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.175+0000 7f4328c00640 10 client.8514 ms_handle_connect on v2:172.21.15.175:6834/569905425 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f4328c00640 1 -- 172.21.15.44:0/482417038 <== mds.0 v2:172.21.15.175:6834/569905425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5616df1b8750 con 0x5616df310d40 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f4328c00640 10 client.8514 handle_command_reply: tid=0 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 client.8514 shutdown 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 msgr2=0x5616df30a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f432c0812c0 tx=0x7f432c0813e0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 shutdown_connections 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:6834/569905425,v1:172.21.15.175:6837/569905425] conn(0x5616df310d40 0x5616df311120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616df292df0 0x5616df277600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5616df276c80 0x5616df2770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 --2- 172.21.15.44:0/482417038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616df2731b0 0x5616df30a750 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f4328c00640 0 client.8514 ms_handle_reset on v2:172.21.15.175:6834/569905425 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 >> 172.21.15.44:0/482417038 conn(0x5616df2485f0 msgr2=0x5616df26b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 shutdown_connections 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.176+0000 7f43451e1740 1 -- 172.21.15.44:0/482417038 wait complete. 2024-07-17T07:16:11.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f43451e1740 20 client.8514 trim_cache size 0 max 16384 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 6949, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7628694400000002, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 79, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 155, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 155, 2024-07-17T07:16:11.186 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.723111367 2024-07-17T07:16:11.187 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T07:16:11.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 msgr2=0x7f4318079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 0x7f4318079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4324001860 tx=0x7f43240014f0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 msgr2=0x7f433c072ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f432c009a70 tx=0x7f432c033670 comp rx=0 tx=0).stop 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 shutdown_connections 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f4318076cd0 0x7f4318079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f433c155520 0x7f433c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f433c1f27b0 0x7f433c154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 --2- 172.21.15.44:0/328400405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4320003a30 0x7f433c072ce0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 >> 172.21.15.44:0/328400405 conn(0x7f433c0b0eb0 msgr2=0x7f433c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 shutdown_connections 2024-07-17T07:16:11.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.177+0000 7f4342400640 1 -- 172.21.15.44:0/328400405 wait complete. 2024-07-17T07:16:11.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T07:16:11.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 -- 172.21.15.44:0/2433704175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe23015d620 msgr2=0x7fe23006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 --2- 172.21.15.44:0/2433704175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe23015d620 0x7fe23006c920 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe23006d020 tx=0x7fe22002a560 comp rx=0 tx=0).stop 2024-07-17T07:16:11.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 -- 172.21.15.44:0/2433704175 shutdown_connections 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 --2- 172.21.15.44:0/2433704175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2300b1160 0x7fe2300738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 --2- 172.21.15.44:0/2433704175 >> v2:172.21.15.175:3300/0 conn(0x7fe230072a10 0x7fe2300b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 --2- 172.21.15.44:0/2433704175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe23015d620 0x7fe23006c920 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 -- 172.21.15.44:0/2433704175 >> 172.21.15.44:0/2433704175 conn(0x7fe2300b5890 msgr2=0x7fe2300b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 -- 172.21.15.44:0/2433704175 shutdown_connections 2024-07-17T07:16:11.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.392+0000 7fe234a00640 1 -- 172.21.15.44:0/2433704175 wait complete. 2024-07-17T07:16:11.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.394+0000 7fe234a00640 1 Processor -- start 2024-07-17T07:16:11.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.394+0000 7fe234a00640 1 -- start start 2024-07-17T07:16:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2300b1160 0x7fe230143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe23015d620 0x7fe2301437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe230073f80 con 0x7fe230072a10 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe230073e00 con 0x7fe23015d620 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.395+0000 7fe234a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe230074100 con 0x7fe2300b1160 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41424/0 (socket says 172.21.15.44:41424) 2024-07-17T07:16:11.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 -- 172.21.15.44:0/4175122030 learned_addr learned my addr 172.21.15.44:0/4175122030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 -- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2300b1160 msgr2=0x7fe230143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2300b1160 0x7fe230143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 -- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe23015d620 msgr2=0x7fe2301437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22f200640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe23015d620 0x7fe2301437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe23015d620 0x7fe2301437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.396+0000 7fe22e800640 1 -- 172.21.15.44:0/4175122030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2300aebb0 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe22e800640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe220034040 tx=0x7fe2200039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22003e030 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2200047a0 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220035040 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2301f8010 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.397+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2300ad1d0 con 0x7fe230072a10 2024-07-17T07:16:11.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.398+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2300738c0 con 0x7fe230072a10 2024-07-17T07:16:11.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.400+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fe220041030 con 0x7fe230072a10 2024-07-17T07:16:11.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.401+0000 7fe227400640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 0x7fe20c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:11.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.401+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7fe2200b9180 con 0x7fe230072a10 2024-07-17T07:16:11.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.402+0000 7fe225e00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 0x7fe20c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:11.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.403+0000 7fe225e00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 0x7fe20c079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe23015df00 tx=0x7fe218006c90 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:11.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.405+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe2300aebb0 con 0x7fe230072a10 2024-07-17T07:16:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.658+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe2301540b0 con 0x7fe230072a10 2024-07-17T07:16:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.659+0000 7fe227400640 1 -- 172.21.15.44:0/4175122030 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) ==== 53+0+73 (secure 0 0 0) 0x7fe220081fe0 con 0x7fe230072a10 2024-07-17T07:16:11.659 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T07:16:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 msgr2=0x7fe20c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 0x7fe20c079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe23015df00 tx=0x7fe218006c90 comp rx=0 tx=0).stop 2024-07-17T07:16:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 msgr2=0x7fe2300cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe220034040 tx=0x7fe2200039e0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 shutdown_connections 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe20c076cd0 0x7fe20c079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe23015d620 0x7fe2301437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2300b1160 0x7fe230143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 --2- 172.21.15.44:0/4175122030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe230072a10 0x7fe2300cc090 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 >> 172.21.15.44:0/4175122030 conn(0x7fe2300b5890 msgr2=0x7fe230160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 shutdown_connections 2024-07-17T07:16:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11.662+0000 7fe234a00640 1 -- 172.21.15.44:0/4175122030 wait complete. 2024-07-17T07:16:11.682 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T07:16:11.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T07:16:11.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T07:16:11.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T07:16:11.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T07:16:11.683 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T07:16:11.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T07:16:11.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T07:16:11.683 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:16:11.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:11.684 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.735 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 41286sec preferred_lft 41286sec 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.736 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.737 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T07:16:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T07:16:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T07:16:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T07:16:11.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:11.739 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T07:16:11.820 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T07:16:11.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:11.824 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T07:16:11.876 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T07:16:11.876 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T07:16:11.876 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T07:16:11.876 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T07:16:11.876 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T07:16:11.876 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T07:16:11.876 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T07:16:11.876 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T07:16:11.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T07:16:11.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T07:16:11.877 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T07:16:11.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T07:16:12.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:16:12.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:12.097 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T07:16:12.147 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T07:16:12.148 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T07:16:12.148 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:16:12.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:12.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T07:16:12.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:16:12.233 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T07:16:12.287 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T07:16:12.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T07:16:12.287 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:16:12.340 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T07:16:12.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:12.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:16:12.472 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:16:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:16:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:16:12.494 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:16:12.494 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:16:12.494 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:16:12.494 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:16:12.495 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T07:16:12.523 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T07:16:12.524 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T07:16:12.524 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T07:16:12.525 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T07:16:12.525 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T07:16:12.525 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T07:16:12.525 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T07:16:12.526 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T07:16:12.530 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T07:16:12.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:12.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T07:16:12.607 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T07:16:12.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T07:16:12.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T07:16:12.758 INFO:teuthology.orchestra.run.smithi044.stdout:client6563 2024-07-17T07:16:12.758 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T07:16:12.758 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T07:16:12.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T07:16:13.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 -- 172.21.15.44:0/735703055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406f4d0 msgr2=0x7fe88415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 --2- 172.21.15.44:0/735703055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406f4d0 0x7fe88415a450 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fe8840cb730 tx=0x7fe87802b880 comp rx=0 tx=0).stop 2024-07-17T07:16:13.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 -- 172.21.15.44:0/735703055 shutdown_connections 2024-07-17T07:16:13.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 --2- 172.21.15.44:0/735703055 >> v2:172.21.15.175:3300/0 conn(0x7fe8841503c0 0x7fe8840ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 --2- 172.21.15.44:0/735703055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406f4d0 0x7fe88415a450 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 -- 172.21.15.44:0/735703055 >> 172.21.15.44:0/735703055 conn(0x7fe8840cf0d0 msgr2=0x7fe8840cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 -- 172.21.15.44:0/735703055 shutdown_connections 2024-07-17T07:16:13.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.064+0000 7fe88a800640 1 -- 172.21.15.44:0/735703055 wait complete. 2024-07-17T07:16:13.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.066+0000 7fe88a800640 1 Processor -- start 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.066+0000 7fe88a800640 1 -- start start 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe88406f4d0 0x7fe884158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8841503c0 0x7fe884146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe88406d850 con 0x7fe88406d420 2024-07-17T07:16:13.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840ac930 con 0x7fe88406f4d0 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe88a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe88406d9d0 con 0x7fe8841503c0 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.067+0000 7fe889400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41440/0 (socket says 172.21.15.44:41440) 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 -- 172.21.15.44:0/3535465227 learned_addr learned my addr 172.21.15.44:0/3535465227 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 -- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8841503c0 msgr2=0x7fe884146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8841503c0 0x7fe884146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 -- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe88406f4d0 msgr2=0x7fe884158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe88406f4d0 0x7fe884158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 -- 172.21.15.44:0/3535465227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8841442e0 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe889400640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe8780039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.068+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878040030 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.069+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8780047a0 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.069+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878004970 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.069+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8841f4480 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.069+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841f4a40 con 0x7fe88406d420 2024-07-17T07:16:13.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.070+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8840cffe0 con 0x7fe88406d420 2024-07-17T07:16:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.071+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fe878043030 con 0x7fe88406d420 2024-07-17T07:16:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.072+0000 7fe881600640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 0x7fe864079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.072+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7fe8780b97d0 con 0x7fe88406d420 2024-07-17T07:16:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.073+0000 7fe888a00640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 0x7fe864079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:13.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.074+0000 7fe888a00640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 0x7fe864079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe87000a8f0 tx=0x7fe870005bd0 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:13.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.076+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe87802f050 con 0x7fe88406d420 2024-07-17T07:16:13.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.313+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 --> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe8840af9f0 con 0x7fe864076cd0 2024-07-17T07:16:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.318+0000 7fe881600640 1 -- 172.21.15.44:0/3535465227 <== mgr.6434 v2:172.21.15.175:6800/729654308 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe8840af9f0 con 0x7fe864076cd0 2024-07-17T07:16:13.317 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T07:16:13.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T07:16:13.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T07:16:13.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T07:16:13.318 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T07:16:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.320+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 msgr2=0x7fe864079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.320+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 0x7fe864079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe87000a8f0 tx=0x7fe870005bd0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.320+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 msgr2=0x7fe8841f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.320+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe878000c00 tx=0x7fe8780039e0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 shutdown_connections 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fe864076cd0 0x7fe864079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8841503c0 0x7fe884146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe88406f4d0 0x7fe884158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 --2- 172.21.15.44:0/3535465227 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe88406d420 0x7fe8841f3dd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 >> 172.21.15.44:0/3535465227 conn(0x7fe8840cf0d0 msgr2=0x7fe88415aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:13.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 shutdown_connections 2024-07-17T07:16:13.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.321+0000 7fe88a800640 1 -- 172.21.15.44:0/3535465227 wait complete. 2024-07-17T07:16:13.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.557+0000 7fc8d7200640 1 -- 172.21.15.44:0/1826097575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 msgr2=0x7fc8d00cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.557+0000 7fc8d7200640 1 --2- 172.21.15.44:0/1826097575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00cb830 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0070b60 tx=0x7fc8c802a980 comp rx=0 tx=0).stop 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 -- 172.21.15.44:0/1826097575 shutdown_connections 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 --2- 172.21.15.44:0/1826097575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00cb830 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 --2- 172.21.15.44:0/1826097575 >> v2:172.21.15.175:3300/0 conn(0x7fc8d006ed30 0x7fc8d015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 --2- 172.21.15.44:0/1826097575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8d0152220 0x7fc8d006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 -- 172.21.15.44:0/1826097575 >> 172.21.15.44:0/1826097575 conn(0x7fc8d00b0eb0 msgr2=0x7fc8d00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:13.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 -- 172.21.15.44:0/1826097575 shutdown_connections 2024-07-17T07:16:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.558+0000 7fc8d7200640 1 -- 172.21.15.44:0/1826097575 wait complete. 2024-07-17T07:16:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.559+0000 7fc8d7200640 1 Processor -- start 2024-07-17T07:16:13.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.560+0000 7fc8d7200640 1 -- start start 2024-07-17T07:16:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.560+0000 7fc8d7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8d006ed30 0x7fc8d00acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.560+0000 7fc8d7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.560+0000 7fc8d7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8d0152220 0x7fc8d00ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8d7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d015f1f0 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8d7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d015f070 con 0x7fc8d006ed30 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8d7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d015f370 con 0x7fc8d0152220 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41462/0 (socket says 172.21.15.44:41462) 2024-07-17T07:16:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 -- 172.21.15.44:0/441920591 learned_addr learned my addr 172.21.15.44:0/441920591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cfe00640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8d006ed30 0x7fc8d00acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:13.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 -- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8d0152220 msgr2=0x7fc8d00ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8d0152220 0x7fc8d00ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 -- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8d006ed30 msgr2=0x7fc8d00acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8d006ed30 0x7fc8d00acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 -- 172.21.15.44:0/441920591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0150c90 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.561+0000 7fc8cf400640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00d7a0 tx=0x7fc8bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.562+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc017070 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.562+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8bc009d20 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.562+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc004280 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.562+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d0074490 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.562+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d0073ed0 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.563+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d00cb830 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.564+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 79) ==== 98931+0+0 (secure 0 0 0) 0x7fc8bc0035f0 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.565+0000 7fc8c7e00640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 0x7fc8b0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:13.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.565+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7fc8bc098b20 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.566+0000 7fc8cfe00640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 0x7fc8b0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:13.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.567+0000 7fc8cfe00640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 0x7fc8b0079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009960 tx=0x7fc8c0005c90 comp rx=0 tx=0).ready entity=mgr.6434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:13.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.570+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc8d0150c90 con 0x7fc8d00cb3a0 2024-07-17T07:16:13.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:13.837+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc8d00b3010 con 0x7fc8d00cb3a0 2024-07-17T07:16:14.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.497+0000 7fc8c7e00640 1 -- 172.21.15.44:0/441920591 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v80) ==== 92+0+0 (secure 0 0 0) 0x7fc8bc061980 con 0x7fc8d00cb3a0 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 msgr2=0x7fc8b0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 0x7fc8b0079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009960 tx=0x7fc8c0005c90 comp rx=0 tx=0).stop 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 msgr2=0x7fc8d00ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00d7a0 tx=0x7fc8bc00dc60 comp rx=0 tx=0).stop 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 shutdown_connections 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7fc8b0076cd0 0x7fc8b0079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8d0152220 0x7fc8d00ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8d00cb3a0 0x7fc8d00ad0e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 --2- 172.21.15.44:0/441920591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8d006ed30 0x7fc8d00acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 >> 172.21.15.44:0/441920591 conn(0x7fc8d00b0eb0 msgr2=0x7fc8d00b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 shutdown_connections 2024-07-17T07:16:14.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.501+0000 7fc8d7200640 1 -- 172.21.15.44:0/441920591 wait complete. 2024-07-17T07:16:14.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T07:16:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 -- 172.21.15.44:0/2260951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 msgr2=0x7f0748003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 --2- 172.21.15.44:0/2260951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f0748003ec0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0758009b50 tx=0x7f0748005830 comp rx=0 tx=0).stop 2024-07-17T07:16:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ce00640 1 -- 172.21.15.44:0/2260951512 reap_dead start 2024-07-17T07:16:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 -- 172.21.15.44:0/2260951512 shutdown_connections 2024-07-17T07:16:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 -- 172.21.15.44:0/2260951512 >> 172.21.15.44:0/2260951512 conn(0x7f07680d00f0 msgr2=0x7f07680d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 -- 172.21.15.44:0/2260951512 shutdown_connections 2024-07-17T07:16:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.737+0000 7f076ee00640 1 -- 172.21.15.44:0/2260951512 wait complete. 2024-07-17T07:16:14.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.739+0000 7f076ee00640 1 Processor -- start 2024-07-17T07:16:14.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.739+0000 7f076ee00640 1 -- start start 2024-07-17T07:16:14.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 0x7f07681f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 0x7f07680cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07680ca370 con 0x7f0748003aa0 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07680d34d0 con 0x7f0768157540 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f076ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07680ca4f0 con 0x7f07680cc360 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f0767e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f0767e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41516/0 (socket says 172.21.15.44:41516) 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.740+0000 7f0767e00640 1 -- 172.21.15.44:0/2553003214 learned_addr learned my addr 172.21.15.44:0/2553003214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f075fe00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 0x7f07681f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f076ce00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 0x7f07680cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:14.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 msgr2=0x7f07680cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 0x7f07680cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 msgr2=0x7f07681f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 0x7f07681f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 -- 172.21.15.44:0/2553003214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0768157a20 con 0x7f0748003aa0 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f075fe00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 0x7f07681f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:14.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f076ce00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 0x7f07680cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0767e00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f075802b4f0 tx=0x7f0758033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758038040 con 0x7f0748003aa0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.741+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0758004020 con 0x7f0748003aa0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.742+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758004450 con 0x7f0748003aa0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.742+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0768157d00 con 0x7f0748003aa0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.742+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07680cbb60 con 0x7f0748003aa0 2024-07-17T07:16:14.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.743+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0768050670 con 0x7f0748003aa0 2024-07-17T07:16:14.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.744+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 80) ==== 98948+0+0 (secure 0 0 0) 0x7f0758045020 con 0x7f0748003aa0 2024-07-17T07:16:14.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.745+0000 7f0764c00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 0x7f07440791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:14.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.745+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f07580b9170 con 0x7f0748003aa0 2024-07-17T07:16:14.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.745+0000 7f075fe00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 msgr2=0x7f07440791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:14.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.745+0000 7f075fe00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 0x7f07440791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:16:14.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.750+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0768050670 con 0x7f0748003aa0 2024-07-17T07:16:14.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.946+0000 7f075fe00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 msgr2=0x7f07440791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:14.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.946+0000 7f075fe00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 0x7f07440791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:16:14.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.962+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f07680c98a0 con 0x7f0748003aa0 2024-07-17T07:16:14.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.967+0000 7f0764c00640 1 -- 172.21.15.44:0/2553003214 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v57)=0 v57) ==== 138+0+0 (secure 0 0 0) 0x7f07580820e0 con 0x7f0748003aa0 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 msgr2=0x7f07440791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 0x7f07440791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 msgr2=0x7f07681458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f075802b4f0 tx=0x7f0758033670 comp rx=0 tx=0).stop 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 shutdown_connections 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f0744076d20 0x7f07440791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07680cc360 0x7f07680cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0768157540 0x7f07681f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 --2- 172.21.15.44:0/2553003214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0748003aa0 0x7f07681458e0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:14.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 >> 172.21.15.44:0/2553003214 conn(0x7f07680d00f0 msgr2=0x7f0768160b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:14.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 shutdown_connections 2024-07-17T07:16:14.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:14.970+0000 7f076ee00640 1 -- 172.21.15.44:0/2553003214 wait complete. 2024-07-17T07:16:14.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.182+0000 7f123a000640 1 -- 172.21.15.44:0/3362616392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 msgr2=0x7f1218003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.182+0000 7f123a000640 1 --2- 172.21.15.44:0/3362616392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f1218003e70 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f12240016c0 tx=0x7f1218005830 comp rx=0 tx=0).stop 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.183+0000 7f123a000640 1 -- 172.21.15.44:0/3362616392 shutdown_connections 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.183+0000 7f123a000640 1 --2- 172.21.15.44:0/3362616392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f1218003e70 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.183+0000 7f123a000640 1 -- 172.21.15.44:0/3362616392 >> 172.21.15.44:0/3362616392 conn(0x7f12340d3700 msgr2=0x7f12340d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.183+0000 7f123a000640 1 -- 172.21.15.44:0/3362616392 shutdown_connections 2024-07-17T07:16:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.183+0000 7f123a000640 1 -- 172.21.15.44:0/3362616392 wait complete. 2024-07-17T07:16:15.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.184+0000 7f123a000640 1 Processor -- start 2024-07-17T07:16:15.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.185+0000 7f123a000640 1 -- start start 2024-07-17T07:16:15.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.185+0000 7f123a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f123a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12341f4220 0x7f12340cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f123a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12340c7230 0x7f12340c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f123a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12340cea00 con 0x7f1218003a50 2024-07-17T07:16:15.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f123a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12340ce880 con 0x7f12340c7230 2024-07-17T07:16:15.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f123a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12340ceb80 con 0x7f12341f4220 2024-07-17T07:16:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f1233400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f1233400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41538/0 (socket says 172.21.15.44:41538) 2024-07-17T07:16:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f1233400640 1 -- 172.21.15.44:0/441898440 learned_addr learned my addr 172.21.15.44:0/441898440 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f122be00640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12341f4220 0x7f12340cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.186+0000 7f1233e00640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12340c7230 0x7f12340c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12341f4220 msgr2=0x7f12340cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12341f4220 0x7f12340cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12340c7230 msgr2=0x7f12340c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12340c7230 0x7f12340c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 -- 172.21.15.44:0/441898440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12341f4a40 con 0x7f1218003a50 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f1233400640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f12240017e0 tx=0x7f12240063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224042070 con 0x7f1218003a50 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1224039070 con 0x7f1218003a50 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12341f4d20 con 0x7f1218003a50 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224003d90 con 0x7f1218003a50 2024-07-17T07:16:15.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.187+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341f46a0 con 0x7f1218003a50 2024-07-17T07:16:15.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.189+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1234161940 con 0x7f1218003a50 2024-07-17T07:16:15.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.190+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 80) ==== 98948+0+0 (secure 0 0 0) 0x7f122403b070 con 0x7f1218003a50 2024-07-17T07:16:15.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.191+0000 7f122b400640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 0x7f12100791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.192+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7f12240b7d60 con 0x7f1218003a50 2024-07-17T07:16:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.192+0000 7f122be00640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 msgr2=0x7f12100791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:15.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.192+0000 7f122be00640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 0x7f12100791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:16:15.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.196+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1224080d70 con 0x7f1218003a50 2024-07-17T07:16:15.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.393+0000 7f122be00640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 msgr2=0x7f12100791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:15.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.393+0000 7f122be00640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 0x7f12100791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:16:15.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.398+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f12340c6db0 con 0x7f1218003a50 2024-07-17T07:16:15.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.403+0000 7f122b400640 1 -- 172.21.15.44:0/441898440 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v58)=0 v58) ==== 133+0+0 (secure 0 0 0) 0x7f1224080670 con 0x7f1218003a50 2024-07-17T07:16:15.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 msgr2=0x7f12100791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 0x7f12100791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 msgr2=0x7f12341f8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f12240017e0 tx=0x7f12240063a0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 shutdown_connections 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7f1210076d20 0x7f12100791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12340c7230 0x7f12340c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12341f4220 0x7f12340cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 --2- 172.21.15.44:0/441898440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1218003a50 0x7f12341f8280 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.406+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 >> 172.21.15.44:0/441898440 conn(0x7f12340d3700 msgr2=0x7f12340b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.407+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 shutdown_connections 2024-07-17T07:16:15.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.407+0000 7f123a000640 1 -- 172.21.15.44:0/441898440 wait complete. 2024-07-17T07:16:15.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T07:16:15.425 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T07:16:15.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2024-07-17T07:16:15.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 -- 172.21.15.44:0/3329622977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0b3fd0 msgr2=0x7efddc0d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 --2- 172.21.15.44:0/3329622977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0b3fd0 0x7efddc0d0860 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009830 tx=0x7efdc402a5c0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 -- 172.21.15.44:0/3329622977 shutdown_connections 2024-07-17T07:16:15.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 --2- 172.21.15.44:0/3329622977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0b3fd0 0x7efddc0d0860 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 -- 172.21.15.44:0/3329622977 >> 172.21.15.44:0/3329622977 conn(0x7efddc0b55d0 msgr2=0x7efddc0b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 -- 172.21.15.44:0/3329622977 shutdown_connections 2024-07-17T07:16:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.661+0000 7efddb200640 1 -- 172.21.15.44:0/3329622977 wait complete. 2024-07-17T07:16:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.663+0000 7efddb200640 1 Processor -- start 2024-07-17T07:16:15.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.663+0000 7efddb200640 1 -- start start 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efddc06e570 0x7efddc159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efddc0b3fd0 0x7efddc1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 0x7efddc0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b2780 con 0x7efddc0d0da0 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b2600 con 0x7efddc06e570 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efddb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efddc0b2900 con 0x7efddc0b3fd0 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efdda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 0x7efddc0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efdda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 0x7efddc0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41558/0 (socket says 172.21.15.44:41558) 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.664+0000 7efdda800640 1 -- 172.21.15.44:0/1649304999 learned_addr learned my addr 172.21.15.44:0/1649304999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efddc06e570 0x7efddc159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd1400640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efddc0b3fd0 0x7efddc1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:15.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efddc0b3fd0 msgr2=0x7efddc1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efddc0b3fd0 0x7efddc1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efddc06e570 msgr2=0x7efddc159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efddc06e570 0x7efddc159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc1f42d0 con 0x7efddc0d0da0 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efddc06e570 0x7efddc159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd1400640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efddc0b3fd0 0x7efddc1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T07:16:15.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdda800640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 0x7efddc0b0ee0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c730 tx=0x7efdd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4003b70 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd4003d30 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd40052d0 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.665+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc1f4880 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.666+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc0c9430 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.667+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc0b4440 con 0x7efddc0d0da0 2024-07-17T07:16:15.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.668+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 80) ==== 98948+0+0 (secure 0 0 0) 0x7efdd4005680 con 0x7efddc0d0da0 2024-07-17T07:16:15.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.669+0000 7efdd2a00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:15.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.669+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6649+0+0 (secure 0 0 0) 0x7efdd4098850 con 0x7efddc0d0da0 2024-07-17T07:16:15.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.669+0000 7efdd9e00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:15.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.669+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T07:16:15.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.673+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efdd4061850 con 0x7efddc0d0da0 2024-07-17T07:16:15.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.871+0000 7efdd9e00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:15.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.871+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T07:16:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:15.917+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7efddc0cc820 con 0x7efdb8076d20 2024-07-17T07:16:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:16.273+0000 7efdd9e00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:16.273+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T07:16:16.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:16:16.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:16:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:17.074+0000 7efdd9e00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:17.074+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T07:16:18.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:18.677+0000 7efdd9e00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/729654308 2024-07-17T07:16:18.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:18.677+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T07:16:18.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:18.761+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 81) ==== 98557+0+0 (secure 0 0 0) 0x7efdd405d3e0 con 0x7efddc0d0da0 2024-07-17T07:16:18.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:18.762+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 msgr2=0x7efdb80791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T07:16:18.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:18.762+0000 7efdd2a00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/729654308,v1:172.21.15.175:6801/729654308] conn(0x7efdb8076d20 0x7efdb80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feeb8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feeb8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feeb8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feeb8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feeb8e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feebca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feebca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feebca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feebca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.059 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:16:19.059+0000 7feebca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:16:19.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.773+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 82) ==== 98684+0+0 (secure 0 0 0) 0x7efdd406ce40 con 0x7efddc0d0da0 2024-07-17T07:16:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.774+0000 7efdd2a00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7efdb807f880 0x7efdb8081c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:19.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.774+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 --> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7efdb800c880 con 0x7efdb807f880 2024-07-17T07:16:19.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.776+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7efdb807f880 0x7efdb8081c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:19.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.779+0000 7efdd9e00640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7efdb807f880 0x7efdb8081c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efdcc007fd0 tx=0x7efdcc007da0 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.970+0000 7efdd2a00640 1 -- 172.21.15.44:0/1649304999 <== mgr.6572 v2:172.21.15.175:6800/2098888629 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efdb800c880 con 0x7efdb807f880 2024-07-17T07:16:19.969 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7efdb807f880 msgr2=0x7efdb8081c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7efdb807f880 0x7efdb8081c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efdcc007fd0 tx=0x7efdcc007da0 comp rx=0 tx=0).stop 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 msgr2=0x7efddc0b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 --2- 172.21.15.44:0/1649304999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efddc0d0da0 0x7efddc0b0ee0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c730 tx=0x7efdd400cbf0 comp rx=0 tx=0).stop 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efdda800640 1 -- 172.21.15.44:0/1649304999 reap_dead start 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 shutdown_connections 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 >> 172.21.15.44:0/1649304999 conn(0x7efddc0b55d0 msgr2=0x7efddc15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 shutdown_connections 2024-07-17T07:16:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:19.973+0000 7efddb200640 1 -- 172.21.15.44:0/1649304999 wait complete. 2024-07-17T07:16:19.995 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2024-07-17T07:16:19.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 -- 172.21.15.44:0/1006196880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 msgr2=0x7f8834004fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 --2- 172.21.15.44:0/1006196880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f8834004fb0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8840009b30 tx=0x7f8834006940 comp rx=0 tx=0).stop 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f884fe00640 1 -- 172.21.15.44:0/1006196880 reap_dead start 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 -- 172.21.15.44:0/1006196880 shutdown_connections 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 -- 172.21.15.44:0/1006196880 >> 172.21.15.44:0/1006196880 conn(0x7f88500cf050 msgr2=0x7f88500cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:20.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 -- 172.21.15.44:0/1006196880 shutdown_connections 2024-07-17T07:16:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.208+0000 7f8854e00640 1 -- 172.21.15.44:0/1006196880 wait complete. 2024-07-17T07:16:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.210+0000 7f8854e00640 1 Processor -- start 2024-07-17T07:16:20.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.210+0000 7f8854e00640 1 -- start start 2024-07-17T07:16:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f885009cce0 0x7f8850098970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850099190 0x7f8850099620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88500ae9c0 con 0x7f8834004b90 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88500ae840 con 0x7f8850099190 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f8854e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88501537d0 con 0x7f885009cce0 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f884f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f884f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60532/0 (socket says 172.21.15.44:60532) 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.211+0000 7f884f400640 1 -- 172.21.15.44:0/4262345884 learned_addr learned my addr 172.21.15.44:0/4262345884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884fe00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850099190 0x7f8850099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 -- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f885009cce0 msgr2=0x7f8850098970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f885009cce0 0x7f8850098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 -- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850099190 msgr2=0x7f8850099620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850099190 0x7f8850099620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 -- 172.21.15.44:0/4262345884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885009a910 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f884f400640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8840009c20 tx=0x7f884002df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840041020 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f884002d360 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885009abf0 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.212+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400047a0 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.213+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885009b1b0 con 0x7f8834004b90 2024-07-17T07:16:20.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.214+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8850153d30 con 0x7f8834004b90 2024-07-17T07:16:20.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.215+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 82) ==== 98684+0+0 (secure 0 0 0) 0x7f8840044020 con 0x7f8834004b90 2024-07-17T07:16:20.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.216+0000 7f8847400640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 0x7f8830078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.216+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6993+0+0 (secure 0 0 0) 0x7f88400b9500 con 0x7f8834004b90 2024-07-17T07:16:20.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.216+0000 7f884ea00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 0x7f8830078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.218+0000 7f884ea00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 0x7f8830078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c006890 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:20.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.221+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f884004b030 con 0x7f8834004b90 2024-07-17T07:16:20.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.465+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 --> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) -- 0x7f8850098eb0 con 0x7f8830076a50 2024-07-17T07:16:20.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.633+0000 7f8847400640 1 -- 172.21.15.44:0/4262345884 <== mgr.6572 v2:172.21.15.175:6800/2098888629 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+40 (secure 0 0 0) 0x7f883c0060d0 con 0x7f8830076a50 2024-07-17T07:16:20.634 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path /snap_test_dir1 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 msgr2=0x7f8830078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 0x7f8830078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f883c006fd0 tx=0x7f883c006890 comp rx=0 tx=0).stop 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 msgr2=0x7f88501f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8840009c20 tx=0x7f884002df70 comp rx=0 tx=0).stop 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 shutdown_connections 2024-07-17T07:16:20.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f8830076a50 0x7f8830078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8850099190 0x7f8850099620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f885009cce0 0x7f8850098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 --2- 172.21.15.44:0/4262345884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8834004b90 0x7f88501f2680 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.637+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 >> 172.21.15.44:0/4262345884 conn(0x7f88500cf050 msgr2=0x7f88500af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.638+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 shutdown_connections 2024-07-17T07:16:20.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.638+0000 7f8854e00640 1 -- 172.21.15.44:0/4262345884 wait complete. 2024-07-17T07:16:20.660 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2024-07-17T07:16:20.660 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2024-07-17T07:16:20.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2024-07-17T07:16:20.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 -- 172.21.15.44:0/1882894531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc815b840 msgr2=0x7fecc806e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 --2- 172.21.15.44:0/1882894531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc815b840 0x7fecc806e790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009be0 tx=0x7fecc002a890 comp rx=0 tx=0).stop 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 -- 172.21.15.44:0/1882894531 shutdown_connections 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 --2- 172.21.15.44:0/1882894531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc815b840 0x7fecc806e790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 -- 172.21.15.44:0/1882894531 >> 172.21.15.44:0/1882894531 conn(0x7fecc80b2240 msgr2=0x7fecc80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.884+0000 7fecce600640 1 -- 172.21.15.44:0/1882894531 shutdown_connections 2024-07-17T07:16:20.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.885+0000 7fecce600640 1 -- 172.21.15.44:0/1882894531 wait complete. 2024-07-17T07:16:20.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.886+0000 7fecce600640 1 Processor -- start 2024-07-17T07:16:20.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.886+0000 7fecce600640 1 -- start start 2024-07-17T07:16:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc806bf40 0x7fecc814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc815ef10 0x7fecc8149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fecc806f060 con 0x7fecc815b840 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecc806eee0 con 0x7fecc815ef10 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.887+0000 7fecce600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecc806f1e0 con 0x7fecc806bf40 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60540/0 (socket says 172.21.15.44:60540) 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 -- 172.21.15.44:0/3196969672 learned_addr learned my addr 172.21.15.44:0/3196969672 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc7400640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc806bf40 0x7fecc814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc7e00640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc815ef10 0x7fecc8149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 -- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc806bf40 msgr2=0x7fecc814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc806bf40 0x7fecc814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 -- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc815ef10 msgr2=0x7fecc8149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc815ef10 0x7fecc8149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:20.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.888+0000 7fecc6a00640 1 -- 172.21.15.44:0/3196969672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc80af330 con 0x7fecc815b840 2024-07-17T07:16:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecc6a00640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fecb400c730 tx=0x7fecb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4016020 con 0x7fecc815b840 2024-07-17T07:16:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc81497e0 con 0x7fecc815b840 2024-07-17T07:16:20.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb400a950 con 0x7fecc815b840 2024-07-17T07:16:20.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb400ade0 con 0x7fecc815b840 2024-07-17T07:16:20.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.889+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc81f4170 con 0x7fecc815b840 2024-07-17T07:16:20.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.890+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc81f4340 con 0x7fecc815b840 2024-07-17T07:16:20.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.891+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 83) ==== 98949+0+0 (secure 0 0 0) 0x7fecb4019020 con 0x7fecc815b840 2024-07-17T07:16:20.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.892+0000 7fecbf400640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 0x7feca80791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T07:16:20.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.893+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6993+0+0 (secure 0 0 0) 0x7fecb40989e0 con 0x7fecc815b840 2024-07-17T07:16:20.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.893+0000 7fecc7400640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 0x7feca80791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T07:16:20.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.894+0000 7fecc7400640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 0x7feca80791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007fd0 tx=0x7fecb8009040 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T07:16:20.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:20.897+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fecb4020030 con 0x7fecc815b840 2024-07-17T07:16:21.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.135+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 --> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecc80b2df0 con 0x7feca8076d20 2024-07-17T07:16:21.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.154+0000 7fecbf400640 1 -- 172.21.15.44:0/3196969672 <== mgr.6572 v2:172.21.15.175:6800/2098888629 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fecc80b2df0 con 0x7feca8076d20 2024-07-17T07:16:21.154 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T07:16:21.154 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 msgr2=0x7feca80791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 0x7feca80791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007fd0 tx=0x7fecb8009040 comp rx=0 tx=0).stop 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 msgr2=0x7fecc81490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fecb400c730 tx=0x7fecb400cbf0 comp rx=0 tx=0).stop 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 shutdown_connections 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7feca8076d20 0x7feca80791e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc815ef10 0x7fecc8149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc815b840 0x7fecc81490b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 --2- 172.21.15.44:0/3196969672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc806bf40 0x7fecc814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T07:16:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 >> 172.21.15.44:0/3196969672 conn(0x7fecc80b2240 msgr2=0x7fecc80cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T07:16:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 shutdown_connections 2024-07-17T07:16:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T07:16:21.157+0000 7fecce600640 1 -- 172.21.15.44:0/3196969672 wait complete. 2024-07-17T07:16:21.176 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2024-07-17T07:16:21.176 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2024-07-17T07:16:46.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:16:46.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:17:00.655 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:17:00.656+0000 7feea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:17:00.655 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:17:00.656+0000 7feea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:17:00.655 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:17:00.656+0000 7feea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:17:00.656 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:17:00.656+0000 7feea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:17:00.656 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T07:17:00.656+0000 7feea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T07:17:16.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:17:16.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:17:47.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:17:47.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:18:17.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:18:17.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:18:47.663 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:18:47.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:19:17.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:19:17.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:19:47.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:19:47.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:20:18.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:20:18.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:20:48.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:20:48.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:21:18.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:21:18.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:21:48.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:21:48.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:22:18.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:22:18.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:22:48.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:22:48.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:23:18.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:23:18.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:23:49.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:23:49.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:24:19.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:24:19.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:24:49.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:24:49.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:25:19.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:25:19.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:25:49.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:25:49.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:26:19.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:26:19.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:26:49.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:26:49.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:27:20.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:27:20.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:27:50.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:27:50.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:28:20.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:28:20.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:28:50.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:28:50.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:29:20.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:29:20.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:29:50.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:29:50.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:30:20.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:30:20.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:30:51.026 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:30:51.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:31:21.163 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:31:21.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:31:51.314 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:31:51.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:32:21.465 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:32:21.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:32:51.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:32:51.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:33:21.740 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:33:21.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:33:51.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:33:51.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:34:22.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:34:22.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:34:52.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:34:52.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:35:22.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:35:22.300 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:35:52.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:35:52.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:36:22.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:36:22.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:36:52.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:36:52.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:37:22.842 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:37:22.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:37:52.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:37:52.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:38:23.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:38:23.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:38:53.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:38:53.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:39:23.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:39:23.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:39:53.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:39:53.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:40:23.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:40:23.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:40:53.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:40:53.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:41:23.965 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:41:23.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:41:54.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:41:54.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:42:24.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:42:24.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:42:54.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:42:54.658 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:43:24.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:43:24.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:43:54.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:43:54.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:44:25.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:44:25.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:44:55.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:44:55.198 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:45:25.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:45:25.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:45:55.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:45:55.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:46:25.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:46:25.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:46:55.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:46:55.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:47:25.854 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:47:25.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:47:55.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:47:55.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:48:26.120 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:48:26.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:48:56.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:48:56.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:49:26.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:49:26.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:49:56.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:49:56.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:50:26.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:50:26.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:50:56.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:50:56.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:51:26.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:51:26.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:51:57.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:51:57.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:52:27.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:52:27.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:52:57.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:52:57.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:53:27.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:53:27.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:53:57.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:53:57.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:54:27.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:54:27.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:54:57.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:54:57.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:55:28.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:55:28.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:55:58.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:55:58.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:56:28.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:56:28.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:56:58.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:56:58.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:57:28.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:57:28.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:57:58.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:57:58.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:58:28.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:58:28.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:58:58.983 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:58:58.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:59:29.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:59:29.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:59:59.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T07:59:59.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:00:29.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:00:29.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:00:59.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:00:59.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:01:29.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:01:29.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:01:59.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:01:59.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:02:29.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:02:29.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:03:00.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:03:00.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:03:30.184 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:03:30.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:04:00.313 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:04:00.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:04:30.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:04:30.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:05:00.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:05:00.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:05:30.743 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:05:30.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:06:00.876 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:06:00.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:06:31.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:06:31.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:07:01.245 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:07:01.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:07:31.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:07:31.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:08:01.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:08:01.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:08:31.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:08:31.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:09:01.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:09:01.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:09:31.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:09:31.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:10:02.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:10:02.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:10:32.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:10:32.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:11:02.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:11:02.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:11:32.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:11:32.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:12:02.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:12:02.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:12:16.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:16.178 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:12:16.248 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_18_00_UTC 2024-07-17T08:12:16.248 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_19_00_UTC 2024-07-17T08:12:16.248 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_20_00_UTC 2024-07-17T08:12:16.248 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_21_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_22_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_23_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_24_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_25_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_26_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_27_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_28_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_29_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_30_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_31_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_32_00_UTC 2024-07-17T08:12:16.249 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_33_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_34_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_35_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_36_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_37_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_38_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_39_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_40_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_41_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_42_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_43_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_44_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_45_00_UTC 2024-07-17T08:12:16.250 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_46_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_47_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_48_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_49_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_50_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_51_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_52_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_53_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_54_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_55_01_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_56_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_57_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_58_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_59_00_UTC 2024-07-17T08:12:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_00_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_01_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_02_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_03_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_04_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_05_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_06_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_07_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_08_00_UTC 2024-07-17T08:12:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_09_00_UTC 2024-07-17T08:12:16.253 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_10_00_UTC 2024-07-17T08:12:16.253 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_11_00_UTC 2024-07-17T08:12:16.253 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_12_00_UTC 2024-07-17T08:12:16.254 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-07_18_00_UTC', 'scheduled-2024-07-17-07_19_00_UTC', 'scheduled-2024-07-17-07_20_00_UTC', 'scheduled-2024-07-17-07_21_00_UTC', 'scheduled-2024-07-17-07_22_00_UTC', 'scheduled-2024-07-17-07_23_00_UTC', 'scheduled-2024-07-17-07_24_00_UTC', 'scheduled-2024-07-17-07_25_00_UTC', 'scheduled-2024-07-17-07_26_00_UTC', 'scheduled-2024-07-17-07_27_00_UTC', 'scheduled-2024-07-17-07_28_00_UTC', 'scheduled-2024-07-17-07_29_00_UTC', 'scheduled-2024-07-17-07_30_00_UTC', 'scheduled-2024-07-17-07_31_00_UTC', 'scheduled-2024-07-17-07_32_00_UTC', 'scheduled-2024-07-17-07_33_00_UTC', 'scheduled-2024-07-17-07_34_00_UTC', 'scheduled-2024-07-17-07_35_00_UTC', 'scheduled-2024-07-17-07_36_00_UTC', 'scheduled-2024-07-17-07_37_00_UTC', 'scheduled-2024-07-17-07_38_00_UTC', 'scheduled-2024-07-17-07_39_00_UTC', 'scheduled-2024-07-17-07_40_00_UTC', 'scheduled-2024-07-17-07_41_00_UTC', 'scheduled-2024-07-17-07_42_00_UTC', 'scheduled-2024-07-17-07_43_00_UTC', 'scheduled-2024-07-17-07_44_00_UTC', 'scheduled-2024-07-17-07_45_00_UTC', 'scheduled-2024-07-17-07_46_00_UTC', 'scheduled-2024-07-17-07_47_00_UTC', 'scheduled-2024-07-17-07_48_00_UTC', 'scheduled-2024-07-17-07_49_00_UTC', 'scheduled-2024-07-17-07_50_00_UTC', 'scheduled-2024-07-17-07_51_00_UTC', 'scheduled-2024-07-17-07_52_00_UTC', 'scheduled-2024-07-17-07_53_00_UTC', 'scheduled-2024-07-17-07_54_00_UTC', 'scheduled-2024-07-17-07_55_01_UTC', 'scheduled-2024-07-17-07_56_00_UTC', 'scheduled-2024-07-17-07_57_00_UTC', 'scheduled-2024-07-17-07_58_00_UTC', 'scheduled-2024-07-17-07_59_00_UTC', 'scheduled-2024-07-17-08_00_00_UTC', 'scheduled-2024-07-17-08_01_00_UTC', 'scheduled-2024-07-17-08_02_00_UTC', 'scheduled-2024-07-17-08_03_00_UTC', 'scheduled-2024-07-17-08_04_00_UTC', 'scheduled-2024-07-17-08_05_00_UTC', 'scheduled-2024-07-17-08_06_00_UTC', 'scheduled-2024-07-17-08_07_00_UTC', 'scheduled-2024-07-17-08_08_00_UTC', 'scheduled-2024-07-17-08_09_00_UTC', 'scheduled-2024-07-17-08_10_00_UTC', 'scheduled-2024-07-17-08_11_00_UTC', 'scheduled-2024-07-17-08_12_00_UTC'] 2024-07-17T08:12:16.254 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2024-07-17T08:12:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 -- 172.21.15.44:0/150840743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 msgr2=0x7fb35006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 --2- 172.21.15.44:0/150840743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35006d620 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb3500b3a80 tx=0x7fb33c02a530 comp rx=0 tx=0).stop 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 -- 172.21.15.44:0/150840743 shutdown_connections 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 --2- 172.21.15.44:0/150840743 >> v2:172.21.15.175:3300/0 conn(0x7fb35006db60 0x7fb3500ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 --2- 172.21.15.44:0/150840743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35006d620 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.473+0000 7fb358000640 1 -- 172.21.15.44:0/150840743 >> 172.21.15.44:0/150840743 conn(0x7fb3500b0eb0 msgr2=0x7fb3500b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.474+0000 7fb358000640 1 -- 172.21.15.44:0/150840743 shutdown_connections 2024-07-17T08:12:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.474+0000 7fb358000640 1 -- 172.21.15.44:0/150840743 wait complete. 2024-07-17T08:12:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.476+0000 7fb358000640 1 Processor -- start 2024-07-17T08:12:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.476+0000 7fb358000640 1 -- start start 2024-07-17T08:12:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 0x7fb3501f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3501f84e0 0x7fb3501f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb35015e0a0 con 0x7fb350161120 2024-07-17T08:12:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb35015e220 con 0x7fb3501f84e0 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb358000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb35015e3a0 con 0x7fb350154600 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb354c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb354c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48676/0 (socket says 172.21.15.44:48676) 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb354c00640 1 -- 172.21.15.44:0/305809978 learned_addr learned my addr 172.21.15.44:0/305809978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb354c00640 1 -- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 msgr2=0x7fb3501f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb355600640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 0x7fb3501f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.477+0000 7fb356000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3501f84e0 0x7fb3501f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb354c00640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 0x7fb3501f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb354c00640 1 -- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3501f84e0 msgr2=0x7fb3501f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb354c00640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3501f84e0 0x7fb3501f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb354c00640 1 -- 172.21.15.44:0/305809978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3501501d0 con 0x7fb350161120 2024-07-17T08:12:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb355600640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 0x7fb3501f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb354c00640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb33c02ab20 tx=0x7fb33c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c031d60 con 0x7fb350161120 2024-07-17T08:12:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c033e10 con 0x7fb350161120 2024-07-17T08:12:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c005260 con 0x7fb350161120 2024-07-17T08:12:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500715e0 con 0x7fb350161120 2024-07-17T08:12:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.478+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb350071ba0 con 0x7fb350161120 2024-07-17T08:12:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.480+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35015f0f0 con 0x7fb350161120 2024-07-17T08:12:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.481+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 86) ==== 98960+0+0 (secure 0 0 0) 0x7fb33c005400 con 0x7fb350161120 2024-07-17T08:12:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.482+0000 7fb34d600640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 0x7fb3300791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.482+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 7081+0+0 (secure 0 0 0) 0x7fb33c0b9c40 con 0x7fb350161120 2024-07-17T08:12:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.483+0000 7fb355600640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 0x7fb3300791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.484+0000 7fb355600640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 0x7fb3300791e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb344000f60 tx=0x7fb344009040 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:16.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.488+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb33c03f030 con 0x7fb350161120 2024-07-17T08:12:16.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.730+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 --> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb350160e90 con 0x7fb330076d20 2024-07-17T08:12:16.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.752+0000 7fb34d600640 1 -- 172.21.15.44:0/305809978 <== mgr.6572 v2:172.21.15.175:6800/2098888629 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+373 (secure 0 0 0) 0x7fb350160e90 con 0x7fb330076d20 2024-07-17T08:12:16.753 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:16.753 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:16:19", "first": "2024-07-17T07:17:00", "last": "2024-07-17T08:12:00", "last_pruned": "2024-07-17T08:12:00", "created_count": 56, "pruned_count": 1, "active": true}] 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 msgr2=0x7fb3300791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 0x7fb3300791e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb344000f60 tx=0x7fb344009040 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 msgr2=0x7fb35014f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb33c02ab20 tx=0x7fb33c033c20 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 shutdown_connections 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fb330076d20 0x7fb3300791e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3501f84e0 0x7fb3501f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb350161120 0x7fb35014f8c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 --2- 172.21.15.44:0/305809978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb350154600 0x7fb3501f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 >> 172.21.15.44:0/305809978 conn(0x7fb3500b0eb0 msgr2=0x7fb35015fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:16.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 shutdown_connections 2024-07-17T08:12:16.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.754+0000 7fb358000640 1 -- 172.21.15.44:0/305809978 wait complete. 2024-07-17T08:12:16.777 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T07:16:19", "first": "2024-07-17T07:17:00", "last": "2024-07-17T08:12:00", "last_pruned": "2024-07-17T08:12:00", "created_count": 56, "pruned_count": 1, "active": true}] 2024-07-17T08:12:16.777 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T07:16:19', 'first': '2024-07-17T07:17:00', 'last': '2024-07-17T08:12:00', 'last_pruned': '2024-07-17T08:12:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2024-07-17T08:12:16.778 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2024-07-17T08:12:16.778 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2024-07-17T08:12:16.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2024-07-17T08:12:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 -- 172.21.15.44:0/2921210287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8004b60 msgr2=0x7fa1b8004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 --2- 172.21.15.44:0/2921210287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8004b60 0x7fa1b8004f60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4005fd0 tx=0x7fa1b8006940 comp rx=0 tx=0).stop 2024-07-17T08:12:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 -- 172.21.15.44:0/2921210287 shutdown_connections 2024-07-17T08:12:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 --2- 172.21.15.44:0/2921210287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1b8004b60 0x7fa1b8004f60 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 -- 172.21.15.44:0/2921210287 >> 172.21.15.44:0/2921210287 conn(0x7fa1d40b0eb0 msgr2=0x7fa1d40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:16.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 -- 172.21.15.44:0/2921210287 shutdown_connections 2024-07-17T08:12:16.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.989+0000 7fa1d9200640 1 -- 172.21.15.44:0/2921210287 wait complete. 2024-07-17T08:12:16.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.991+0000 7fa1d9200640 1 Processor -- start 2024-07-17T08:12:16.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.991+0000 7fa1d9200640 1 -- start start 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1b8004b60 0x7fa1d40989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 0x7fa1d41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40acec0 con 0x7fa1d414faa0 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40acd40 con 0x7fa1d413fe70 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.992+0000 7fa1d9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d40ad040 con 0x7fa1b8004b60 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48696/0 (socket says 172.21.15.44:48696) 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 -- 172.21.15.44:0/1137320180 learned_addr learned my addr 172.21.15.44:0/1137320180 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1d1e00640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1b8004b60 0x7fa1d40989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1d2800640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 0x7fa1d41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 -- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1b8004b60 msgr2=0x7fa1d40989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1b8004b60 0x7fa1d40989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 -- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 msgr2=0x7fa1d41502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 0x7fa1d41502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 -- 172.21.15.44:0/1137320180 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41e7a00 con 0x7fa1d414faa0 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1d2800640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 0x7fa1d41502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:16.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.993+0000 7fa1c9400640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c730 tx=0x7fa1bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.994+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc017070 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.994+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1bc005380 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.994+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00a8a0 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.994+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d41e7ce0 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.994+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41e82a0 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.995+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d406d910 con 0x7fa1d414faa0 2024-07-17T08:12:16.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.996+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 86) ==== 98960+0+0 (secure 0 0 0) 0x7fa1bc01a070 con 0x7fa1d414faa0 2024-07-17T08:12:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.997+0000 7fa1caa00640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 0x7fa1b0079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.997+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 7081+0+0 (secure 0 0 0) 0x7fa1bc098180 con 0x7fa1d414faa0 2024-07-17T08:12:16.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.998+0000 7fa1d1e00640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 0x7fa1b0079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:16.999+0000 7fa1d1e00640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 0x7fa1b0079210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4035000 tx=0x7fa1c4004a40 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:17.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.002+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa1d406e800 con 0x7fa1d414faa0 2024-07-17T08:12:17.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.233+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 --> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7fa1d406e9e0 con 0x7fa1b0076d50 2024-07-17T08:12:17.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.447+0000 7fa1caa00640 1 -- 172.21.15.44:0/1137320180 <== mgr.6572 v2:172.21.15.175:6800/2098888629 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+42 (secure 0 0 0) 0x7fa1d406e9e0 con 0x7fa1b0076d50 2024-07-17T08:12:17.449 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1 2024-07-17T08:12:17.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 msgr2=0x7fa1b0079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:17.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 0x7fa1b0079210 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4035000 tx=0x7fa1c4004a40 comp rx=0 tx=0).stop 2024-07-17T08:12:17.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 msgr2=0x7fa1d41e3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:17.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c730 tx=0x7fa1bc00cbf0 comp rx=0 tx=0).stop 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 shutdown_connections 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fa1b0076d50 0x7fa1b0079210 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1d413fe70 0x7fa1d41502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.450+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1d414faa0 0x7fa1d41e3410 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.451+0000 7fa1d9200640 1 --2- 172.21.15.44:0/1137320180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1b8004b60 0x7fa1d40989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.451+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 >> 172.21.15.44:0/1137320180 conn(0x7fa1d40b0eb0 msgr2=0x7fa1d40b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.451+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 shutdown_connections 2024-07-17T08:12:17.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:17.451+0000 7fa1d9200640 1 -- 172.21.15.44:0/1137320180 wait complete. 2024-07-17T08:12:17.472 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2024-07-17T08:12:17.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.472 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_18_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_19_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_20_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_21_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_22_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_23_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_24_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_25_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_26_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_27_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_28_00_UTC 2024-07-17T08:12:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_29_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_30_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_31_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_32_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_33_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_34_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_35_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_36_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_37_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_38_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_39_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_40_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_41_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_42_00_UTC 2024-07-17T08:12:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_43_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_44_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_45_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_46_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_47_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_48_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_49_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_50_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_51_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_52_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_53_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_54_00_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_55_01_UTC 2024-07-17T08:12:17.530 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_56_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_57_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_58_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-07_59_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_00_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_01_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_02_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_03_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_04_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_05_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_06_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_07_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_08_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_09_00_UTC 2024-07-17T08:12:17.531 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_10_00_UTC 2024-07-17T08:12:17.532 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_11_00_UTC 2024-07-17T08:12:17.532 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_12_00_UTC 2024-07-17T08:12:17.533 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_18_00_UTC 2024-07-17T08:12:17.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.533 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_18_00_UTC) 2024-07-17T08:12:17.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_19_00_UTC 2024-07-17T08:12:17.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.604 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_19_00_UTC) 2024-07-17T08:12:17.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_20_00_UTC 2024-07-17T08:12:17.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.673 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_20_00_UTC) 2024-07-17T08:12:17.744 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_21_00_UTC 2024-07-17T08:12:17.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.744 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_21_00_UTC) 2024-07-17T08:12:17.815 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_22_00_UTC 2024-07-17T08:12:17.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.815 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_22_00_UTC) 2024-07-17T08:12:17.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_23_00_UTC 2024-07-17T08:12:17.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.886 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_23_00_UTC) 2024-07-17T08:12:17.955 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_24_00_UTC 2024-07-17T08:12:17.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:17.955 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_24_00_UTC) 2024-07-17T08:12:18.023 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_25_00_UTC 2024-07-17T08:12:18.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.023 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_25_00_UTC) 2024-07-17T08:12:18.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_26_00_UTC 2024-07-17T08:12:18.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.093 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_26_00_UTC) 2024-07-17T08:12:18.166 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_27_00_UTC 2024-07-17T08:12:18.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.167 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_27_00_UTC) 2024-07-17T08:12:18.232 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_28_00_UTC 2024-07-17T08:12:18.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.232 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_28_00_UTC) 2024-07-17T08:12:18.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_29_00_UTC 2024-07-17T08:12:18.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.299 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_29_00_UTC) 2024-07-17T08:12:18.368 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_30_00_UTC 2024-07-17T08:12:18.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.368 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_30_00_UTC) 2024-07-17T08:12:18.441 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_31_00_UTC 2024-07-17T08:12:18.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.442 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_31_00_UTC) 2024-07-17T08:12:18.510 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_32_00_UTC 2024-07-17T08:12:18.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.510 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_32_00_UTC) 2024-07-17T08:12:18.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_33_00_UTC 2024-07-17T08:12:18.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.581 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_33_00_UTC) 2024-07-17T08:12:18.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_34_00_UTC 2024-07-17T08:12:18.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.653 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_34_00_UTC) 2024-07-17T08:12:18.724 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_35_00_UTC 2024-07-17T08:12:18.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.725 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_35_00_UTC) 2024-07-17T08:12:18.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_36_00_UTC 2024-07-17T08:12:18.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.795 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_36_00_UTC) 2024-07-17T08:12:18.866 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_37_00_UTC 2024-07-17T08:12:18.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.866 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_37_00_UTC) 2024-07-17T08:12:18.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_38_00_UTC 2024-07-17T08:12:18.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:18.937 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_38_00_UTC) 2024-07-17T08:12:19.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_39_00_UTC 2024-07-17T08:12:19.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.002 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_39_00_UTC) 2024-07-17T08:12:19.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_40_00_UTC 2024-07-17T08:12:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.074 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_40_00_UTC) 2024-07-17T08:12:19.145 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_41_00_UTC 2024-07-17T08:12:19.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.145 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_41_00_UTC) 2024-07-17T08:12:19.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_42_00_UTC 2024-07-17T08:12:19.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.217 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_42_00_UTC) 2024-07-17T08:12:19.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_43_00_UTC 2024-07-17T08:12:19.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.284 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_43_00_UTC) 2024-07-17T08:12:19.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_44_00_UTC 2024-07-17T08:12:19.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.348 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_44_00_UTC) 2024-07-17T08:12:19.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_45_00_UTC 2024-07-17T08:12:19.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.421 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_45_00_UTC) 2024-07-17T08:12:19.491 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_46_00_UTC 2024-07-17T08:12:19.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.491 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_46_00_UTC) 2024-07-17T08:12:19.564 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_47_00_UTC 2024-07-17T08:12:19.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.564 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_47_00_UTC) 2024-07-17T08:12:19.638 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_48_00_UTC 2024-07-17T08:12:19.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.639 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_48_00_UTC) 2024-07-17T08:12:19.709 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_49_00_UTC 2024-07-17T08:12:19.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.709 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_49_00_UTC) 2024-07-17T08:12:19.827 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_50_00_UTC 2024-07-17T08:12:19.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.828 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_50_00_UTC) 2024-07-17T08:12:19.915 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_51_00_UTC 2024-07-17T08:12:19.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.916 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_51_00_UTC) 2024-07-17T08:12:19.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_52_00_UTC 2024-07-17T08:12:19.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:19.994 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_52_00_UTC) 2024-07-17T08:12:20.039 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_53_00_UTC 2024-07-17T08:12:20.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.039 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_53_00_UTC) 2024-07-17T08:12:20.134 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_54_00_UTC 2024-07-17T08:12:20.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.135 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_54_00_UTC) 2024-07-17T08:12:20.191 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_55_01_UTC 2024-07-17T08:12:20.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.191 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_55_01_UTC) 2024-07-17T08:12:20.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_56_00_UTC 2024-07-17T08:12:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.259 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_56_00_UTC) 2024-07-17T08:12:20.403 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_57_00_UTC 2024-07-17T08:12:20.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.403 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_57_00_UTC) 2024-07-17T08:12:20.480 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_58_00_UTC 2024-07-17T08:12:20.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.480 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_58_00_UTC) 2024-07-17T08:12:20.548 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-07_59_00_UTC 2024-07-17T08:12:20.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.548 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-07_59_00_UTC) 2024-07-17T08:12:20.599 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_00_00_UTC 2024-07-17T08:12:20.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.599 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_00_00_UTC) 2024-07-17T08:12:20.655 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_01_00_UTC 2024-07-17T08:12:20.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.655 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_01_00_UTC) 2024-07-17T08:12:20.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_02_00_UTC 2024-07-17T08:12:20.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.714 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_02_00_UTC) 2024-07-17T08:12:20.801 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_03_00_UTC 2024-07-17T08:12:20.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.802 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_03_00_UTC) 2024-07-17T08:12:20.867 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_04_00_UTC 2024-07-17T08:12:20.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.868 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_04_00_UTC) 2024-07-17T08:12:20.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_05_00_UTC 2024-07-17T08:12:20.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:20.936 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_05_00_UTC) 2024-07-17T08:12:21.000 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_06_00_UTC 2024-07-17T08:12:21.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.000 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_06_00_UTC) 2024-07-17T08:12:21.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_07_00_UTC 2024-07-17T08:12:21.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.060 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_07_00_UTC) 2024-07-17T08:12:21.130 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_08_00_UTC 2024-07-17T08:12:21.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.131 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_08_00_UTC) 2024-07-17T08:12:21.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_09_00_UTC 2024-07-17T08:12:21.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.194 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_09_00_UTC) 2024-07-17T08:12:21.264 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_10_00_UTC 2024-07-17T08:12:21.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.264 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_10_00_UTC) 2024-07-17T08:12:21.332 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_11_00_UTC 2024-07-17T08:12:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.332 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_11_00_UTC) 2024-07-17T08:12:21.401 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_12_00_UTC 2024-07-17T08:12:21.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.401 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_12_00_UTC) 2024-07-17T08:12:21.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:21.472 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T08:12:21.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 -- 172.21.15.44:0/1359525367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 msgr2=0x7fba58037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 --2- 172.21.15.44:0/1359525367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba58037940 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fba60065cb0 tx=0x7fba600a54c0 comp rx=0 tx=0).stop 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 -- 172.21.15.44:0/1359525367 shutdown_connections 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 --2- 172.21.15.44:0/1359525367 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba58052ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 --2- 172.21.15.44:0/1359525367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba58037940 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.778+0000 7fba67000640 1 -- 172.21.15.44:0/1359525367 >> 172.21.15.44:0/1359525367 conn(0x7fba58055540 msgr2=0x7fba58055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.779+0000 7fba67000640 1 -- 172.21.15.44:0/1359525367 shutdown_connections 2024-07-17T08:12:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.780+0000 7fba67000640 1 -- 172.21.15.44:0/1359525367 wait complete. 2024-07-17T08:12:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.783+0000 7fba67000640 1 Processor -- start 2024-07-17T08:12:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.784+0000 7fba67000640 1 -- start start 2024-07-17T08:12:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba5800bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba580dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba580e39e0 0x7fba58181a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba58053820 con 0x7fba580e2a60 2024-07-17T08:12:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba580536a0 con 0x7fba580e39e0 2024-07-17T08:12:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.787+0000 7fba67000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba580539a0 con 0x7fba5803ab10 2024-07-17T08:12:21.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.789+0000 7fba65c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba5800bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.789+0000 7fba65c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba5800bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:55648/0 (socket says 172.21.15.44:55648) 2024-07-17T08:12:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.789+0000 7fba65c00640 1 -- 172.21.15.44:0/303840184 learned_addr learned my addr 172.21.15.44:0/303840184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.789+0000 7fba65200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba580dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.789+0000 7fba66600640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba580e39e0 0x7fba58181a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.790+0000 7fba65200640 1 -- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 msgr2=0x7fba5800bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.790+0000 7fba65200640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba5800bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.790+0000 7fba65200640 1 -- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba580e39e0 msgr2=0x7fba58181a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.790+0000 7fba65200640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba580e39e0 0x7fba58181a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.790+0000 7fba65200640 1 -- 172.21.15.44:0/303840184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5800c2e0 con 0x7fba580e2a60 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.791+0000 7fba65200640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba580dbe20 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c3c0 tx=0x7fba5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.791+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50017070 con 0x7fba580e2a60 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.792+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba500035f0 con 0x7fba580e2a60 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.792+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50003a40 con 0x7fba580e2a60 2024-07-17T08:12:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.792+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5800c5c0 con 0x7fba580e2a60 2024-07-17T08:12:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.792+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5800c8a0 con 0x7fba580e2a60 2024-07-17T08:12:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.795+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba58053f30 con 0x7fba580e2a60 2024-07-17T08:12:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.797+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 86) ==== 98960+0+0 (secure 0 0 0) 0x7fba5001a070 con 0x7fba580e2a60 2024-07-17T08:12:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.800+0000 7fba5d600640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 0x7fba44079230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.800+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 7081+0+0 (secure 0 0 0) 0x7fba50097c30 con 0x7fba580e2a60 2024-07-17T08:12:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.801+0000 7fba65c00640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 0x7fba44079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.804+0000 7fba65c00640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 0x7fba44079230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba6006b060 tx=0x7fba60069df0 comp rx=0 tx=0).ready entity=mgr.6572 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:21.809+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fba50060420 con 0x7fba580e2a60 2024-07-17T08:12:22.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.127+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fba5803bd70 con 0x7fba580e2a60 2024-07-17T08:12:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.749+0000 7fba5d600640 1 -- 172.21.15.44:0/303840184 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v87) ==== 93+0+0 (secure 0 0 0) 0x7fba50064970 con 0x7fba580e2a60 2024-07-17T08:12:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.757+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 msgr2=0x7fba44079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.757+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 0x7fba44079230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba6006b060 tx=0x7fba60069df0 comp rx=0 tx=0).stop 2024-07-17T08:12:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.757+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 msgr2=0x7fba580dbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.759+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba580dbe20 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c3c0 tx=0x7fba5000c880 comp rx=0 tx=0).stop 2024-07-17T08:12:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.760+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 shutdown_connections 2024-07-17T08:12:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.760+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fba44076d70 0x7fba44079230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.760+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba580e39e0 0x7fba58181a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:22.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.760+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba580e2a60 0x7fba580dbe20 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.761+0000 7fba67000640 1 --2- 172.21.15.44:0/303840184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba5803ab10 0x7fba5800bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.761+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 >> 172.21.15.44:0/303840184 conn(0x7fba58055540 msgr2=0x7fba580e4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.761+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 shutdown_connections 2024-07-17T08:12:22.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:22.761+0000 7fba67000640 1 -- 172.21.15.44:0/303840184 wait complete. 2024-07-17T08:12:22.819 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:12:22.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:12:22.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:12:23.058 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:12:23.059 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:12:23.059 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:23.059 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:23.108 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:23.108 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:23.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 -- 172.21.15.44:0/191879352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc478068c40 msgr2=0x7fc47815a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 --2- 172.21.15.44:0/191879352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc478068c40 0x7fc47815a7b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc470005fd0 tx=0x7fc47002ab20 comp rx=0 tx=0).stop 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 -- 172.21.15.44:0/191879352 shutdown_connections 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 --2- 172.21.15.44:0/191879352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc478068c40 0x7fc47815a7b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 --2- 172.21.15.44:0/191879352 >> v2:172.21.15.175:3300/0 conn(0x7fc478068860 0x7fc4780b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 -- 172.21.15.44:0/191879352 >> 172.21.15.44:0/191879352 conn(0x7fc4780cd490 msgr2=0x7fc47806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 -- 172.21.15.44:0/191879352 shutdown_connections 2024-07-17T08:12:23.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.399+0000 7fc47d600640 1 -- 172.21.15.44:0/191879352 wait complete. 2024-07-17T08:12:23.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.401+0000 7fc47d600640 1 Processor -- start 2024-07-17T08:12:23.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.401+0000 7fc47d600640 1 -- start start 2024-07-17T08:12:23.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc478068c40 0x7fc47806f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4780b3270 0x7fc4781588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc47813e660 con 0x7fc478068860 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc47813e4e0 con 0x7fc478068c40 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc47813e7e0 con 0x7fc4780b3270 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57224/0 (socket says 172.21.15.44:57224) 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 -- 172.21.15.44:0/3796031695 learned_addr learned my addr 172.21.15.44:0/3796031695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc47cc00640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4780b3270 0x7fc4781588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc46f400640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc478068c40 0x7fc47806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4780b3270 msgr2=0x7fc4781588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4780b3270 0x7fc4781588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc478068c40 msgr2=0x7fc47806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:23.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc478068c40 0x7fc47806f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.402+0000 7fc477e00640 1 -- 172.21.15.44:0/3796031695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4781f8920 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc477e00640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9f0 tx=0x7fc46800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800cd80 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc468010430 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468010840 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4781f50a0 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.403+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47806eb30 con 0x7fc478068860 2024-07-17T08:12:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.404+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4780b2d30 con 0x7fc478068860 2024-07-17T08:12:23.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.407+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7fc4680109e0 con 0x7fc478068860 2024-07-17T08:12:23.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.408+0000 7fc474c00640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 0x7fc454079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:23.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.408+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 7081+0+0 (secure 0 0 0) 0x7fc468099600 con 0x7fc478068860 2024-07-17T08:12:23.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.408+0000 7fc46f400640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 msgr2=0x7fc454079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:23.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.408+0000 7fc46f400640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 0x7fc454079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.412+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc4781f8920 con 0x7fc478068860 2024-07-17T08:12:23.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.610+0000 7fc46f400640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 msgr2=0x7fc454079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:23.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.610+0000 7fc46f400640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 0x7fc454079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:23.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.636+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4781e8340 con 0x7fc478068860 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.637+0000 7fc474c00640 1 -- 172.21.15.44:0/3796031695 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v161) ==== 80+0+475 (secure 0 0 0) 0x7fc468062400 con 0x7fc478068860 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4073021880 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/564005492 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1767233865 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4152324913 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3712769329 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6801/729654308 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.639 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6800/729654308 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.640 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1576245277 2024-07-18T07:16:18.744944+0000 2024-07-17T08:12:23.640 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:12:23.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.640+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 msgr2=0x7fc454079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:23.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.640+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 0x7fc454079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.640+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 msgr2=0x7fc4781f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.640+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc46800e9f0 tx=0x7fc46800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 shutdown_connections 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fc454076cd0 0x7fc454079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4780b3270 0x7fc4781588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc478068c40 0x7fc47806f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 --2- 172.21.15.44:0/3796031695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc478068860 0x7fc4781f83e0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 >> 172.21.15.44:0/3796031695 conn(0x7fc4780cd490 msgr2=0x7fc47813c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 shutdown_connections 2024-07-17T08:12:23.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:23.641+0000 7fc47d600640 1 -- 172.21.15.44:0/3796031695 wait complete. 2024-07-17T08:12:23.661 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:12:23.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:23.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:12:23.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:12:23.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:23.745 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:12:23.786 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:12:23.786 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:12:23.787 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:23.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:12:23.835 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:12:23.835 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:23.835 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:23.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:12:23.885 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:12:23.885 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:12:23.885 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:23.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:12:23.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 -- 172.21.15.44:0/318329501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e00b1160 msgr2=0x7f42e00738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 --2- 172.21.15.44:0/318329501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e00b1160 0x7f42e00738c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f42e00725c0 tx=0x7f42d802aa30 comp rx=0 tx=0).stop 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 -- 172.21.15.44:0/318329501 shutdown_connections 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 --2- 172.21.15.44:0/318329501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e0072a10 0x7f42e00b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 --2- 172.21.15.44:0/318329501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e00b1160 0x7f42e00738c0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 --2- 172.21.15.44:0/318329501 >> v2:172.21.15.175:3300/0 conn(0x7f42e015d620 0x7f42e006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 -- 172.21.15.44:0/318329501 >> 172.21.15.44:0/318329501 conn(0x7f42e00b5890 msgr2=0x7f42e00b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 -- 172.21.15.44:0/318329501 shutdown_connections 2024-07-17T08:12:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.139+0000 7f42df200640 1 -- 172.21.15.44:0/318329501 wait complete. 2024-07-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.141+0000 7f42df200640 1 Processor -- start 2024-07-17T08:12:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.141+0000 7f42df200640 1 -- start start 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42e00b1160 0x7f42e014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 0x7f42e00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0073f80 con 0x7f42e0072a10 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0073e00 con 0x7f42e00b1160 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42df200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42e0074100 con 0x7f42e015d620 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42dde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57250/0 (socket says 172.21.15.44:57250) 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42dde00640 1 -- 172.21.15.44:0/3556401523 learned_addr learned my addr 172.21.15.44:0/3556401523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42de800640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 0x7f42e00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.142+0000 7f42d5400640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42e00b1160 0x7f42e014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 msgr2=0x7f42e00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 0x7f42e00adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42e00b1160 msgr2=0x7f42e014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42e00b1160 0x7f42e014dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 -- 172.21.15.44:0/3556401523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e014fa50 con 0x7f42e0072a10 2024-07-17T08:12:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42de800640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 0x7f42e00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42dde00640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d770 tx=0x7f42d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0017070 con 0x7f42e0072a10 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42d0009d20 con 0x7f42e0072a10 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e00ca1c0 con 0x7f42e0072a10 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00106c0 con 0x7f42e0072a10 2024-07-17T08:12:24.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.143+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e00ca780 con 0x7f42e0072a10 2024-07-17T08:12:24.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.144+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e0074690 con 0x7f42e0072a10 2024-07-17T08:12:24.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.147+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7f42d001a070 con 0x7f42e0072a10 2024-07-17T08:12:24.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.148+0000 7f42d6a00640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 0x7f42bc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.148+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 7081+0+0 (secure 0 0 0) 0x7f42d0098c40 con 0x7f42e0072a10 2024-07-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.148+0000 7f42d5400640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 msgr2=0x7f42bc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.148+0000 7f42d5400640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 0x7f42bc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.152+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f42d00618f0 con 0x7f42e0072a10 2024-07-17T08:12:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.350+0000 7f42d5400640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 msgr2=0x7f42bc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.350+0000 7f42d5400640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 0x7f42bc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:24.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.403+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42e00b0a50 con 0x7f42e0072a10 2024-07-17T08:12:24.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.405+0000 7f42d6a00640 1 -- 172.21.15.44:0/3556401523 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7f42d00611f0 con 0x7f42e0072a10 2024-07-17T08:12:24.406 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:24.407 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T08:12:24.407 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 msgr2=0x7f42bc079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 0x7f42bc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 msgr2=0x7f42e00afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d770 tx=0x7f42d000dc30 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 shutdown_connections 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f42bc076cd0 0x7f42bc079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42e015d620 0x7f42e00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42e00b1160 0x7f42e014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 --2- 172.21.15.44:0/3556401523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42e0072a10 0x7f42e00afe10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 >> 172.21.15.44:0/3556401523 conn(0x7f42e00b5890 msgr2=0x7f42e015b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:24.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 shutdown_connections 2024-07-17T08:12:24.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.408+0000 7f42df200640 1 -- 172.21.15.44:0/3556401523 wait complete. 2024-07-17T08:12:24.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.626+0000 7f7ecc200640 1 -- 172.21.15.44:0/3587431425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 msgr2=0x7f7ec406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.626+0000 7f7ecc200640 1 --2- 172.21.15.44:0/3587431425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec406e4a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7ec41576d0 tx=0x7f7ebc02aa30 comp rx=0 tx=0).stop 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 -- 172.21.15.44:0/3587431425 shutdown_connections 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 --2- 172.21.15.44:0/3587431425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec406e4a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 --2- 172.21.15.44:0/3587431425 >> v2:172.21.15.175:3300/0 conn(0x7f7ec40ac3e0 0x7f7ec40ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 -- 172.21.15.44:0/3587431425 >> 172.21.15.44:0/3587431425 conn(0x7f7ec40b0fb0 msgr2=0x7f7ec40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 -- 172.21.15.44:0/3587431425 shutdown_connections 2024-07-17T08:12:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.627+0000 7f7ecc200640 1 -- 172.21.15.44:0/3587431425 wait complete. 2024-07-17T08:12:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.628+0000 7f7ecc200640 1 Processor -- start 2024-07-17T08:12:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 -- start start 2024-07-17T08:12:24.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 0x7f7ec40976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec40ac3e0 0x7f7ec409bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4159c80 con 0x7f7ec406e090 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4159b00 con 0x7f7ec406f710 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.629+0000 7f7ecc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec4159e00 con 0x7f7ec40ac3e0 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57266/0 (socket says 172.21.15.44:57266) 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 -- 172.21.15.44:0/245813672 learned_addr learned my addr 172.21.15.44:0/245813672 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec8e00640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 0x7f7ec40976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7eca200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec40ac3e0 0x7f7ec409bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:24.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec40ac3e0 msgr2=0x7f7ec409bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec40ac3e0 0x7f7ec409bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 msgr2=0x7f7ec40976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 0x7f7ec40976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec9800640 1 -- 172.21.15.44:0/245813672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec413f430 con 0x7f7ec406e090 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.630+0000 7f7ec8e00640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 0x7f7ec40976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ec9800640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800c430 tx=0x7f7eb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8011790 con 0x7f7ec406e090 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eb8017070 con 0x7f7ec406e090 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8011cc0 con 0x7f7ec406e090 2024-07-17T08:12:24.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec4072bb0 con 0x7f7ec406e090 2024-07-17T08:12:24.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.631+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec4073160 con 0x7f7ec406e090 2024-07-17T08:12:24.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.632+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec4050670 con 0x7f7ec406e090 2024-07-17T08:12:24.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.633+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7f7eb801c460 con 0x7f7ec406e090 2024-07-17T08:12:24.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.634+0000 7f7ec1600640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 0x7f7ea4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:24.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.634+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 7081+0+0 (secure 0 0 0) 0x7f7eb80989f0 con 0x7f7ec406e090 2024-07-17T08:12:24.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.635+0000 7f7ec8e00640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 msgr2=0x7f7ea4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:24.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.635+0000 7f7ec8e00640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 0x7f7ea4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:24.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.642+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7ec4050670 con 0x7f7ec406e090 2024-07-17T08:12:24.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.836+0000 7f7ec8e00640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 msgr2=0x7f7ea4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:24.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.836+0000 7f7ec8e00640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 0x7f7ea4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:24.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.913+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec40afb10 con 0x7f7ec406e090 2024-07-17T08:12:24.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.914+0000 7f7ec1600640 1 -- 172.21.15.44:0/245813672 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7f7eb80617b0 con 0x7f7ec406e090 2024-07-17T08:12:24.916 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:24.916 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T08:12:24.916 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T08:12:24.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.916+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 msgr2=0x7f7ea4079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:24.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.916+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 0x7f7ea4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.916+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 msgr2=0x7f7ec4072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:24.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.916+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800c430 tx=0x7f7eb800c8f0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 shutdown_connections 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f7ea4076cd0 0x7f7ea4079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec40ac3e0 0x7f7ec409bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec406f710 0x7f7ec40976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 --2- 172.21.15.44:0/245813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ec406e090 0x7f7ec4072670 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 >> 172.21.15.44:0/245813672 conn(0x7f7ec40b0fb0 msgr2=0x7f7ec4098760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 shutdown_connections 2024-07-17T08:12:24.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:24.917+0000 7f7ecc200640 1 -- 172.21.15.44:0/245813672 wait complete. 2024-07-17T08:12:24.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 -- 172.21.15.44:0/2687878265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 msgr2=0x7ff38815d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 --2- 172.21.15.44:0/2687878265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff38815d9d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff380009860 tx=0x7ff38002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 -- 172.21.15.44:0/2687878265 shutdown_connections 2024-07-17T08:12:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 --2- 172.21.15.44:0/2687878265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff38815d9d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 -- 172.21.15.44:0/2687878265 >> 172.21.15.44:0/2687878265 conn(0x7ff3880cfec0 msgr2=0x7ff3880d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 -- 172.21.15.44:0/2687878265 shutdown_connections 2024-07-17T08:12:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.123+0000 7ff38e400640 1 -- 172.21.15.44:0/2687878265 wait complete. 2024-07-17T08:12:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.125+0000 7ff38e400640 1 Processor -- start 2024-07-17T08:12:25.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.125+0000 7ff38e400640 1 -- start start 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3880b8c80 0x7ff3880cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 0x7ff3880c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff38815e2a0 con 0x7ff3880b9060 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff38815e120 con 0x7ff3880ce5c0 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff38815e420 con 0x7ff3880b8c80 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff37fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff37fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57284/0 (socket says 172.21.15.44:57284) 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff37fe00640 1 -- 172.21.15.44:0/289251793 learned_addr learned my addr 172.21.15.44:0/289251793 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:25.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.126+0000 7ff38d000640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3880b8c80 0x7ff3880cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff38da00640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 0x7ff3880c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3880b8c80 msgr2=0x7ff3880cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3880b8c80 0x7ff3880cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 msgr2=0x7ff3880c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 0x7ff3880c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 -- 172.21.15.44:0/289251793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3880cbaa0 con 0x7ff3880b9060 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff38da00640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 0x7ff3880c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff37fe00640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c370 tx=0x7ff37400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374017070 con 0x7ff3880b9060 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3740053c0 con 0x7ff3880b9060 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374003850 con 0x7ff3880b9060 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3881f75e0 con 0x7ff3880b9060 2024-07-17T08:12:25.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.127+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3881f7ba0 con 0x7ff3880b9060 2024-07-17T08:12:25.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.129+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3880b4ec0 con 0x7ff3880b9060 2024-07-17T08:12:25.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.131+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7ff37401a070 con 0x7ff3880b9060 2024-07-17T08:12:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.132+0000 7ff385600640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 0x7ff368079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.132+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 7081+0+0 (secure 0 0 0) 0x7ff374097d80 con 0x7ff3880b9060 2024-07-17T08:12:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.132+0000 7ff38d000640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 msgr2=0x7ff368079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:25.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.132+0000 7ff38d000640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 0x7ff368079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:25.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.140+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff3740604f0 con 0x7ff3880b9060 2024-07-17T08:12:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.334+0000 7ff38d000640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 msgr2=0x7ff368079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.334+0000 7ff38d000640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 0x7ff368079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:25.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.350+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff388074bc0 con 0x7ff3880b9060 2024-07-17T08:12:25.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.352+0000 7ff385600640 1 -- 172.21.15.44:0/289251793 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+18391 (secure 0 0 0) 0x7ff374064a40 con 0x7ff3880b9060 2024-07-17T08:12:25.354 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:25.354 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":162,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:24.781397+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:16:01.823652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-07-17T07:16:02.840012+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4073021880":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/564005492":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/1767233865":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/4152324913":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/3712769329":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6801/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6800/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/1576245277":"2024-07-18T07:16:18.744944+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:25.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 msgr2=0x7ff368079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 0x7ff368079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 msgr2=0x7ff3880ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c370 tx=0x7ff37400c830 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 shutdown_connections 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff368076cd0 0x7ff368079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3880ce5c0 0x7ff3880c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3880b9060 0x7ff3880ccd90 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 --2- 172.21.15.44:0/289251793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3880b8c80 0x7ff3880cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 >> 172.21.15.44:0/289251793 conn(0x7ff3880cfec0 msgr2=0x7ff3880d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:25.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 shutdown_connections 2024-07-17T08:12:25.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.355+0000 7ff38e400640 1 -- 172.21.15.44:0/289251793 wait complete. 2024-07-17T08:12:25.373 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:12:25.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 -- 172.21.15.44:0/932479084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 msgr2=0x7fcb180cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/932479084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cdcf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005fd0 tx=0x7fcb1002ab30 comp rx=0 tx=0).stop 2024-07-17T08:12:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 -- 172.21.15.44:0/932479084 shutdown_connections 2024-07-17T08:12:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/932479084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cdcf0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/932479084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 0x7fcb180b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 -- 172.21.15.44:0/932479084 >> 172.21.15.44:0/932479084 conn(0x7fcb180d00a0 msgr2=0x7fcb180d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 -- 172.21.15.44:0/932479084 shutdown_connections 2024-07-17T08:12:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.565+0000 7fcb1ec00640 1 -- 172.21.15.44:0/932479084 wait complete. 2024-07-17T08:12:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.567+0000 7fcb1ec00640 1 Processor -- start 2024-07-17T08:12:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.567+0000 7fcb1ec00640 1 -- start start 2024-07-17T08:12:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 0x7fcb181570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180ce700 con 0x7fcb1815d600 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180ce580 con 0x7fcb180b5740 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180ce880 con 0x7fcb180b2ad0 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57300/0 (socket says 172.21.15.44:57300) 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb1cc00640 1 -- 172.21.15.44:0/418198413 learned_addr learned my addr 172.21.15.44:0/418198413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.568+0000 7fcb0fe00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb17e00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 0x7fcb181570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 msgr2=0x7fcb181570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 0x7fcb181570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 msgr2=0x7fcb180cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 -- 172.21.15.44:0/418198413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181f7850 con 0x7fcb1815d600 2024-07-17T08:12:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1cc00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009d90 tx=0x7fcb1002cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10044070 con 0x7fcb1815d600 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb10004250 con 0x7fcb1815d600 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10004680 con 0x7fcb1815d600 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.569+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181f7b30 con 0x7fcb1815d600 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.570+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181f80f0 con 0x7fcb1815d600 2024-07-17T08:12:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.571+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb180ced30 con 0x7fcb1815d600 2024-07-17T08:12:25.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.572+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7fcb1003f410 con 0x7fcb1815d600 2024-07-17T08:12:25.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.573+0000 7fcb14c00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 0x7fcaf40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.573+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 7081+0+0 (secure 0 0 0) 0x7fcb100b9040 con 0x7fcb1815d600 2024-07-17T08:12:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.573+0000 7fcb17e00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 msgr2=0x7fcaf40791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.573+0000 7fcb17e00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 0x7fcaf40791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.577+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcb10032030 con 0x7fcb1815d600 2024-07-17T08:12:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.775+0000 7fcb17e00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 msgr2=0x7fcaf40791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.775+0000 7fcb17e00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 0x7fcaf40791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:25.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.827+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb180b6120 con 0x7fcb1815d600 2024-07-17T08:12:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.828+0000 7fcb14c00640 1 -- 172.21.15.44:0/418198413 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7fcb10004820 con 0x7fcb1815d600 2024-07-17T08:12:25.830 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:25.830 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T08:12:25.830 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 msgr2=0x7fcaf40791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 0x7fcaf40791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 msgr2=0x7fcb180b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009d90 tx=0x7fcb1002cf00 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 shutdown_connections 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7fcaf4076d00 0x7fcaf40791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb1815d600 0x7fcb180b4230 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb180b5740 0x7fcb180cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 --2- 172.21.15.44:0/418198413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb180b2ad0 0x7fcb181570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:25.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 >> 172.21.15.44:0/418198413 conn(0x7fcb180d00a0 msgr2=0x7fcb1806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.831+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 shutdown_connections 2024-07-17T08:12:25.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:25.832+0000 7fcb1ec00640 1 -- 172.21.15.44:0/418198413 wait complete. 2024-07-17T08:12:25.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 -- 172.21.15.44:0/4081556087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b01522c0 msgr2=0x7f10b00ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 --2- 172.21.15.44:0/4081556087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b01522c0 0x7f10b00ad790 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f10b015c950 tx=0x7f10a002a560 comp rx=0 tx=0).stop 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 -- 172.21.15.44:0/4081556087 shutdown_connections 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 --2- 172.21.15.44:0/4081556087 >> v2:172.21.15.175:3300/0 conn(0x7f10b00adcd0 0x7f10b00ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 --2- 172.21.15.44:0/4081556087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b01522c0 0x7f10b00ad790 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 -- 172.21.15.44:0/4081556087 >> 172.21.15.44:0/4081556087 conn(0x7f10b00b0fb0 msgr2=0x7f10b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 -- 172.21.15.44:0/4081556087 shutdown_connections 2024-07-17T08:12:26.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.060+0000 7f10b7200640 1 -- 172.21.15.44:0/4081556087 wait complete. 2024-07-17T08:12:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.062+0000 7f10b7200640 1 Processor -- start 2024-07-17T08:12:26.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.062+0000 7f10b7200640 1 -- start start 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 0x7f10b0155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10b01522c0 0x7f10b01f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00ce820 con 0x7f10b00adcd0 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00ce6a0 con 0x7f10b00ac760 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10b00ce9a0 con 0x7f10b01522c0 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:26.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57326/0 (socket says 172.21.15.44:57326) 2024-07-17T08:12:26.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.063+0000 7f10a7400640 1 -- 172.21.15.44:0/3952774888 learned_addr learned my addr 172.21.15.44:0/3952774888 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10afe00640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 0x7f10b0155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10b01522c0 msgr2=0x7f10b01f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10b01522c0 0x7f10b01f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 msgr2=0x7f10b0155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 0x7f10b0155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 -- 172.21.15.44:0/3952774888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b0154480 con 0x7f10b00adcd0 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10afe00640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 0x7f10b0155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10a7400640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f109800c9a0 tx=0x7f109800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098007bb0 con 0x7f10b00adcd0 2024-07-17T08:12:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1098005020 con 0x7f10b00adcd0 2024-07-17T08:12:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.064+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098007e60 con 0x7f10b00adcd0 2024-07-17T08:12:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.065+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b0154760 con 0x7f10b00adcd0 2024-07-17T08:12:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.065+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b0154d20 con 0x7f10b00adcd0 2024-07-17T08:12:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.066+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b00ac220 con 0x7f10b00adcd0 2024-07-17T08:12:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.067+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7f1098005470 con 0x7f10b00adcd0 2024-07-17T08:12:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.068+0000 7f10acc00640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 0x7f108c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.068+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 7081+0+0 (secure 0 0 0) 0x7f1098098ff0 con 0x7f10b00adcd0 2024-07-17T08:12:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.068+0000 7f10afe00640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 msgr2=0x7f108c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:26.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.068+0000 7f10afe00640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 0x7f108c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:26.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.072+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1098014030 con 0x7f10b00adcd0 2024-07-17T08:12:26.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.270+0000 7f10afe00640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 msgr2=0x7f108c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:26.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.270+0000 7f10afe00640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 0x7f108c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:26.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.331+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10b015b8a0 con 0x7f10b00adcd0 2024-07-17T08:12:26.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.332+0000 7f10acc00640 1 -- 172.21.15.44:0/3952774888 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5228 (secure 0 0 0) 0x7f1098061ce0 con 0x7f10b00adcd0 2024-07-17T08:12:26.334 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:26.334 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":79,"btime":"2024-07-17T07:16:07:444751+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T07:16:03.849433+0000","modified":"2024-07-17T07:16:06.948093+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6949":{"gid":6949,"name":"d","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/569905425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":569905425},{"type":"v1","addr":"172.21.15.175:6837","nonce":569905425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6949,"qdb_cluster":[6949]},"id":9}]} 2024-07-17T08:12:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 79 2024-07-17T08:12:26.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.334+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 msgr2=0x7f108c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.334+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 0x7f108c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.334+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 msgr2=0x7f10b0144b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.334+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f109800c9a0 tx=0x7f109800ce60 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 shutdown_connections 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f108c076cd0 0x7f108c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10b01522c0 0x7f10b01f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10b00adcd0 0x7f10b0144b70 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 --2- 172.21.15.44:0/3952774888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10b00ac760 0x7f10b0155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 >> 172.21.15.44:0/3952774888 conn(0x7f10b00b0fb0 msgr2=0x7f10b00cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 shutdown_connections 2024-07-17T08:12:26.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.335+0000 7f10b7200640 1 -- 172.21.15.44:0/3952774888 wait complete. 2024-07-17T08:12:26.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.552+0000 7f589c200640 1 -- 172.21.15.44:0/396511424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c0051c0 msgr2=0x7f587c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.552+0000 7f589c200640 1 --2- 172.21.15.44:0/396511424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587c0051c0 0x7f587c00b9c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f588c00c610 tx=0x7f587c005830 comp rx=0 tx=0).stop 2024-07-17T08:12:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.552+0000 7f589a200640 1 -- 172.21.15.44:0/396511424 reap_dead start 2024-07-17T08:12:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.553+0000 7f589c200640 1 -- 172.21.15.44:0/396511424 shutdown_connections 2024-07-17T08:12:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.553+0000 7f589c200640 1 -- 172.21.15.44:0/396511424 >> 172.21.15.44:0/396511424 conn(0x7f58940b3020 msgr2=0x7f58940b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:26.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.553+0000 7f589c200640 1 -- 172.21.15.44:0/396511424 shutdown_connections 2024-07-17T08:12:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.553+0000 7f589c200640 1 -- 172.21.15.44:0/396511424 wait complete. 2024-07-17T08:12:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.555+0000 7f589c200640 1 Processor -- start 2024-07-17T08:12:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.555+0000 7f589c200640 1 -- start start 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f587c0051c0 0x7f589415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58940d1880 0x7f58940d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58940c7810 con 0x7f589409e830 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5894071140 con 0x7f587c0051c0 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f589c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58940c7990 con 0x7f58940d1880 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f5898e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f5898e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57344/0 (socket says 172.21.15.44:57344) 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f5898e00640 1 -- 172.21.15.44:0/1089858307 learned_addr learned my addr 172.21.15.44:0/1089858307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.556+0000 7f5899800640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f587c0051c0 0x7f589415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f589a200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58940d1880 0x7f58940d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58940d1880 msgr2=0x7f58940d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58940d1880 0x7f58940d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f587c0051c0 msgr2=0x7f589415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f587c0051c0 0x7f589415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 -- 172.21.15.44:0/1089858307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894152770 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5898e00640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f588000e710 tx=0x7f588000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588000c9f0 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f588000cc80 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880011470 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.557+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589409f030 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.558+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589409b890 con 0x7f589409e830 2024-07-17T08:12:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.559+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58940c7f20 con 0x7f589409e830 2024-07-17T08:12:26.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.560+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7f5880011820 con 0x7f589409e830 2024-07-17T08:12:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.561+0000 7f5891600640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 0x7f5874079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.561+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 7081+0+0 (secure 0 0 0) 0x7f58800996a0 con 0x7f589409e830 2024-07-17T08:12:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.561+0000 7f5899800640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 msgr2=0x7f5874079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.561+0000 7f5899800640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 0x7f5874079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.565+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5880016030 con 0x7f589409e830 2024-07-17T08:12:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.762+0000 7f5899800640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 msgr2=0x7f5874079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.762+0000 7f5899800640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 0x7f5874079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.777+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58940d2240 con 0x7f589409e830 2024-07-17T08:12:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.779+0000 7f5891600640 1 -- 172.21.15.44:0/1089858307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+18391 (secure 0 0 0) 0x7f5880062350 con 0x7f589409e830 2024-07-17T08:12:26.781 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:26.781 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":162,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:24.781397+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-07-17T07:16:01.823652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-07-17T07:16:02.840012+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4073021880":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/564005492":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/1767233865":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/4152324913":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/3712769329":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6801/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6800/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/1576245277":"2024-07-18T07:16:18.744944+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.781+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 msgr2=0x7f5874079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.781+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 0x7f5874079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.781+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 msgr2=0x7f5894152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.781+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f588000e710 tx=0x7f588000ebd0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 shutdown_connections 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7f5874076cd0 0x7f5874079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58940d1880 0x7f58940d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f589409e830 0x7f5894152230 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 --2- 172.21.15.44:0/1089858307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f587c0051c0 0x7f589415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 >> 172.21.15.44:0/1089858307 conn(0x7f58940b3020 msgr2=0x7f58940b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 shutdown_connections 2024-07-17T08:12:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.782+0000 7f589c200640 1 -- 172.21.15.44:0/1089858307 wait complete. 2024-07-17T08:12:26.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 -- 172.21.15.44:0/737338608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0ce1d0 msgr2=0x7ff9dc0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 --2- 172.21.15.44:0/737338608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc0ce5b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc005fd0 tx=0x7ff9cc02a670 comp rx=0 tx=0).stop 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 -- 172.21.15.44:0/737338608 shutdown_connections 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 --2- 172.21.15.44:0/737338608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc0ce5b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 -- 172.21.15.44:0/737338608 >> 172.21.15.44:0/737338608 conn(0x7ff9dc0d3700 msgr2=0x7ff9dc0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 -- 172.21.15.44:0/737338608 shutdown_connections 2024-07-17T08:12:27.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:26.999+0000 7ff9e1000640 1 -- 172.21.15.44:0/737338608 wait complete. 2024-07-17T08:12:27.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.001+0000 7ff9e1000640 1 Processor -- start 2024-07-17T08:12:27.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.001+0000 7ff9e1000640 1 -- start start 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9dc0b29e0 0x7ff9dc14a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc168aa0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc168920 con 0x7ff9dc0ce1d0 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9e1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc168c20 con 0x7ff9dc0b29e0 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57366/0 (socket says 172.21.15.44:57366) 2024-07-17T08:12:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.002+0000 7ff9d9400640 1 -- 172.21.15.44:0/328605299 learned_addr learned my addr 172.21.15.44:0/328605299 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9da800640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9dc0b29e0 msgr2=0x7ff9dc14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9dc0b29e0 0x7ff9dc14a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9dc0ce1d0 msgr2=0x7ff9dc157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 -- 172.21.15.44:0/328605299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0c7770 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d9400640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c450 tx=0x7ff9c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80052d0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9c8005470 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8012870 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc15b130 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.003+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0c8e10 con 0x7ff9dc0b6790 2024-07-17T08:12:27.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.005+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc0ce5b0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.006+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 87) ==== 98943+0+0 (secure 0 0 0) 0x7ff9c80035f0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.007+0000 7ff9d2000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 0x7ff9bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.007+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 7081+0+0 (secure 0 0 0) 0x7ff9c8098600 con 0x7ff9dc0b6790 2024-07-17T08:12:27.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.007+0000 7ff9d9e00640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 msgr2=0x7ff9bc0791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:27.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.007+0000 7ff9d9e00640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 0x7ff9bc0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:27.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.011+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff9c80613d0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.208+0000 7ff9d9e00640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 msgr2=0x7ff9bc0791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2098888629 2024-07-17T08:12:27.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.208+0000 7ff9d9e00640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 0x7ff9bc0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.256+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff9dc0c8be0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.258 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:12:27.257+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:12:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.259+0000 7ff9d2000640 1 -- 172.21.15.44:0/328605299 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7ff9c8060cd0 con 0x7ff9dc0b6790 2024-07-17T08:12:27.261 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:12:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.262+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 msgr2=0x7ff9bc0791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.262+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 0x7ff9bc0791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.262+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 msgr2=0x7ff9dc0c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.262+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c450 tx=0x7ff9c800c910 comp rx=0 tx=0).stop 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 shutdown_connections 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:6800/2098888629,v1:172.21.15.175:6801/2098888629] conn(0x7ff9bc076d00 0x7ff9bc0791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9dc0ce1d0 0x7ff9dc157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9dc0b6790 0x7ff9dc0c7230 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 --2- 172.21.15.44:0/328605299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9dc0b29e0 0x7ff9dc14a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 >> 172.21.15.44:0/328605299 conn(0x7ff9dc0d3700 msgr2=0x7ff9dc0d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 shutdown_connections 2024-07-17T08:12:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.263+0000 7ff9e1000640 1 -- 172.21.15.44:0/328605299 wait complete. 2024-07-17T08:12:27.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:12:27.382 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:12:27.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.445+0000 7f00b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.445+0000 7f00b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.445+0000 7f00b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.445+0000 7f00b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.446 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.445+0000 7f00b5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.447 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.446+0000 7f00b9400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.447 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.446+0000 7f00b9400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.447 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.446+0000 7f00b9400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.447 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.446+0000 7f00b9400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.447 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:27.446+0000 7f00b9400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.476+0000 7f1a27000640 1 -- 172.21.15.44:0/2698064869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 msgr2=0x7f1a2015a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.476+0000 7f1a27000640 1 --2- 172.21.15.44:0/2698064869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015a5a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1a2015aae0 tx=0x7f1a1002aa30 comp rx=0 tx=0).stop 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 -- 172.21.15.44:0/2698064869 shutdown_connections 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 --2- 172.21.15.44:0/2698064869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200b3280 0x7f1a2006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 --2- 172.21.15.44:0/2698064869 >> v2:172.21.15.175:3300/0 conn(0x7f1a200ae560 0x7f1a200ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 --2- 172.21.15.44:0/2698064869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015a5a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 -- 172.21.15.44:0/2698064869 >> 172.21.15.44:0/2698064869 conn(0x7f1a200b0eb0 msgr2=0x7f1a200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 -- 172.21.15.44:0/2698064869 shutdown_connections 2024-07-17T08:12:27.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.477+0000 7f1a27000640 1 -- 172.21.15.44:0/2698064869 wait complete. 2024-07-17T08:12:27.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.478+0000 7f1a27000640 1 Processor -- start 2024-07-17T08:12:27.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 -- start start 2024-07-17T08:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200ae560 0x7f1a201f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 0x7f1a201f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200aca60 con 0x7f1a2006f6d0 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200ac8e0 con 0x7f1a200b3280 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.479+0000 7f1a27000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a200acbe0 con 0x7f1a200ae560 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57382/0 (socket says 172.21.15.44:57382) 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 -- 172.21.15.44:0/2287865956 learned_addr learned my addr 172.21.15.44:0/2287865956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a17400640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200ae560 0x7f1a201f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a25000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 0x7f1a201f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 -- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200ae560 msgr2=0x7f1a201f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200ae560 0x7f1a201f1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 -- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 msgr2=0x7f1a201f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 0x7f1a201f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a1fe00640 1 -- 172.21.15.44:0/2287865956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a201f6390 con 0x7f1a2006f6d0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.480+0000 7f1a25000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 0x7f1a201f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a1fe00640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009d60 tx=0x7f1a10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10033050 con 0x7f1a2006f6d0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a100044f0 con 0x7f1a2006f6d0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10038450 con 0x7f1a2006f6d0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a201484d0 con 0x7f1a2006f6d0 2024-07-17T08:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.481+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a20148a90 con 0x7f1a2006f6d0 2024-07-17T08:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.482+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a200ac3a0 con 0x7f1a2006f6d0 2024-07-17T08:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.483+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 88) ==== 98540+0+0 (secure 0 0 0) 0x7f1a10036070 con 0x7f1a2006f6d0 2024-07-17T08:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.484+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 7467+0+0 (secure 0 0 0) 0x7f1a100b9830 con 0x7f1a2006f6d0 2024-07-17T08:12:27.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.489+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1a10082980 con 0x7f1a2006f6d0 2024-07-17T08:12:27.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:27.734+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1a200b4200 con 0x7f1a2006f6d0 2024-07-17T08:12:28.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.382+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 89) ==== 98667+0+0 (secure 0 0 0) 0x7f1a1002d070 con 0x7f1a2006f6d0 2024-07-17T08:12:28.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.383+0000 7f1a1cc00640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 0x7f19fc07f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:28.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.384+0000 7f1a17400640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 0x7f19fc07f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:28.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.388+0000 7f1a17400640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 0x7f19fc07f530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1a201f53e0 tx=0x7f1a0800b410 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:28.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.402+0000 7f1a1cc00640 1 -- 172.21.15.44:0/2287865956 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7f1a100b3a30 con 0x7f1a2006f6d0 2024-07-17T08:12:28.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.405+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 msgr2=0x7f19fc07f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:28.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.405+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 0x7f19fc07f530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1a201f53e0 tx=0x7f1a0800b410 comp rx=0 tx=0).stop 2024-07-17T08:12:28.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.405+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 msgr2=0x7f1a2015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.405+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009d60 tx=0x7f1a10003f10 comp rx=0 tx=0).stop 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 shutdown_connections 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f19fc07d140 0x7f19fc07f530 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a200b3280 0x7f1a201f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a200ae560 0x7f1a201f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 --2- 172.21.15.44:0/2287865956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2006f6d0 0x7f1a2015f0e0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 >> 172.21.15.44:0/2287865956 conn(0x7f1a200b0eb0 msgr2=0x7f1a200aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 shutdown_connections 2024-07-17T08:12:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.406+0000 7f1a27000640 1 -- 172.21.15.44:0/2287865956 wait complete. 2024-07-17T08:12:28.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:12:28.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 -- 172.21.15.44:0/674255986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 msgr2=0x7f460806bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:28.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 --2- 172.21.15.44:0/674255986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f460806bf40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009830 tx=0x7f45f402a520 comp rx=0 tx=0).stop 2024-07-17T08:12:28.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 -- 172.21.15.44:0/674255986 shutdown_connections 2024-07-17T08:12:28.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 --2- 172.21.15.44:0/674255986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 0x7f46080b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 --2- 172.21.15.44:0/674255986 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f460806bf40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 -- 172.21.15.44:0/674255986 >> 172.21.15.44:0/674255986 conn(0x7f46080b2160 msgr2=0x7f46080b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:28.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 -- 172.21.15.44:0/674255986 shutdown_connections 2024-07-17T08:12:28.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.652+0000 7f460e600640 1 -- 172.21.15.44:0/674255986 wait complete. 2024-07-17T08:12:28.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.654+0000 7f460e600640 1 Processor -- start 2024-07-17T08:12:28.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.655+0000 7f460e600640 1 -- start start 2024-07-17T08:12:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 0x7f460813c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f4608154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46080b4570 con 0x7f46080cfcf0 2024-07-17T08:12:28.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46080b43f0 con 0x7f460815e720 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f460e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46080b46f0 con 0x7f460806c480 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f4606a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f4606a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57388/0 (socket says 172.21.15.44:57388) 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.656+0000 7f4606a00640 1 -- 172.21.15.44:0/4267923169 learned_addr learned my addr 172.21.15.44:0/4267923169 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4607e00640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f4608154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4607400640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 0x7f460813c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 -- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 msgr2=0x7f460813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 0x7f460813c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 -- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 msgr2=0x7f4608154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f4608154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 -- 172.21.15.44:0/4267923169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460813cb20 con 0x7f46080cfcf0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.657+0000 7f4606a00640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009950 tx=0x7f45f4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.658+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031d60 con 0x7f46080cfcf0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.658+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460814e250 con 0x7f46080cfcf0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.658+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f4033e10 con 0x7f46080cfcf0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.658+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460814eaf0 con 0x7f46080cfcf0 2024-07-17T08:12:28.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.658+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4048020 con 0x7f46080cfcf0 2024-07-17T08:12:28.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.659+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46080b3c20 con 0x7f46080cfcf0 2024-07-17T08:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.660+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 89) ==== 98667+0+0 (secure 0 0 0) 0x7f45f40332d0 con 0x7f46080cfcf0 2024-07-17T08:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.661+0000 7f45ff400640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 0x7f45e8078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.661+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 7467+0+0 (secure 0 0 0) 0x7f45f40b99c0 con 0x7f46080cfcf0 2024-07-17T08:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.662+0000 7f4607400640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 0x7f45e8078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.663+0000 7f4607400640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 0x7f45e8078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45f80046b0 tx=0x7f45f8009040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:28.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.665+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f45f403f030 con 0x7f46080cfcf0 2024-07-17T08:12:28.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:28.874+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46080afeb0 con 0x7f46080cfcf0 2024-07-17T08:12:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.713+0000 7f45ff400640 1 -- 172.21.15.44:0/4267923169 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v165) ==== 181+0+0 (secure 0 0 0) 0x7f45f40829a0 con 0x7f46080cfcf0 2024-07-17T08:12:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:12:29.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.720+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 msgr2=0x7f45e8078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:29.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.720+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 0x7f45e8078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45f80046b0 tx=0x7f45f8009040 comp rx=0 tx=0).stop 2024-07-17T08:12:29.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.720+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 msgr2=0x7f460813cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:29.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.720+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009950 tx=0x7f45f4033c20 comp rx=0 tx=0).stop 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 shutdown_connections 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f45e8076a00 0x7f45e8078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f460815e720 0x7f4608154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46080cfcf0 0x7f460813cdf0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 --2- 172.21.15.44:0/4267923169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f460806c480 0x7f460813c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.721+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 >> 172.21.15.44:0/4267923169 conn(0x7f46080b2160 msgr2=0x7f46080adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:29.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.722+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 shutdown_connections 2024-07-17T08:12:29.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.723+0000 7f460e600640 1 -- 172.21.15.44:0/4267923169 wait complete. 2024-07-17T08:12:29.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:12:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.945+0000 7f95c9c00640 1 -- 172.21.15.44:0/2869714194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95a8003a00 msgr2=0x7f95a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.945+0000 7f95c9c00640 1 --2- 172.21.15.44:0/2869714194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95a8003a00 0x7f95a8003e20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f95b400baf0 tx=0x7f95a8005830 comp rx=0 tx=0).stop 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.945+0000 7f95c9200640 1 -- 172.21.15.44:0/2869714194 reap_dead start 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.945+0000 7f95c9c00640 1 -- 172.21.15.44:0/2869714194 shutdown_connections 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.945+0000 7f95c9c00640 1 -- 172.21.15.44:0/2869714194 >> 172.21.15.44:0/2869714194 conn(0x7f95c40cf050 msgr2=0x7f95c40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.946+0000 7f95c9c00640 1 -- 172.21.15.44:0/2869714194 shutdown_connections 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.946+0000 7f95c9c00640 1 -- 172.21.15.44:0/2869714194 wait complete. 2024-07-17T08:12:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.947+0000 7f95c9c00640 1 Processor -- start 2024-07-17T08:12:29.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.947+0000 7f95c9c00640 1 -- start start 2024-07-17T08:12:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a8003a00 0x7f95c41f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95c41537a0 0x7f95c406e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:29.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40add80 con 0x7f95c406ed70 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4073bd0 con 0x7f95a8003a00 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40adf00 con 0x7f95c41537a0 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.948+0000 7f95c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60742/0 (socket says 172.21.15.44:60742) 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 -- 172.21.15.44:0/3455251520 learned_addr learned my addr 172.21.15.44:0/3455251520 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 -- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95c41537a0 msgr2=0x7f95c406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c3400640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95c41537a0 0x7f95c406e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95c41537a0 0x7f95c406e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 -- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a8003a00 msgr2=0x7f95c41f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a8003a00 0x7f95c41f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:29.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 -- 172.21.15.44:0/3455251520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41f8010 con 0x7f95c406ed70 2024-07-17T08:12:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95c9200640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000f80 tx=0x7f95bc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc015070 con 0x7f95c406ed70 2024-07-17T08:12:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95bc00cd50 con 0x7f95c406ed70 2024-07-17T08:12:29.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.949+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0116c0 con 0x7f95c406ed70 2024-07-17T08:12:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.950+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c41f82f0 con 0x7f95c406ed70 2024-07-17T08:12:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.950+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c4146a30 con 0x7f95c406ed70 2024-07-17T08:12:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.951+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c40abfa0 con 0x7f95c406ed70 2024-07-17T08:12:29.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.952+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 90) ==== 98773+0+0 (secure 0 0 0) 0x7f95bc018070 con 0x7f95c406ed70 2024-07-17T08:12:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.953+0000 7f95bbe00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 0x7f95a4078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:29.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.953+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 7029+0+0 (secure 0 0 0) 0x7f95bc098b10 con 0x7f95c406ed70 2024-07-17T08:12:29.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.954+0000 7f95c3e00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 0x7f95a4078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:29.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.955+0000 7f95c3e00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 0x7f95a4078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95b4036000 tx=0x7f95b4004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:29.957+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f95bc061b80 con 0x7f95c406ed70 2024-07-17T08:12:30.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.167+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f95c415af60 con 0x7f95c406ed70 2024-07-17T08:12:30.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.817+0000 7f95bbe00640 1 -- 172.21.15.44:0/3455251520 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v166) ==== 169+0+0 (secure 0 0 0) 0x7f95bc061480 con 0x7f95c406ed70 2024-07-17T08:12:30.819 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:12:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.822+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 msgr2=0x7f95a4078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.822+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 0x7f95a4078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95b4036000 tx=0x7f95b4004730 comp rx=0 tx=0).stop 2024-07-17T08:12:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 msgr2=0x7f95c409c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:30.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000f80 tx=0x7f95bc00ed10 comp rx=0 tx=0).stop 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 shutdown_connections 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f95a4076b20 0x7f95a4078fe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c406ed70 0x7f95c409c8a0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95c41537a0 0x7f95c406e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 --2- 172.21.15.44:0/3455251520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95a8003a00 0x7f95c41f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:30.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.823+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 >> 172.21.15.44:0/3455251520 conn(0x7f95c40cf050 msgr2=0x7f95c40af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:30.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.825+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 shutdown_connections 2024-07-17T08:12:30.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:30.826+0000 7f95c9c00640 1 -- 172.21.15.44:0/3455251520 wait complete. 2024-07-17T08:12:30.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 -- 172.21.15.44:0/115142815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c0003a50 msgr2=0x7f65c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 --2- 172.21.15.44:0/115142815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c0003a50 0x7f65c0003e50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c290 tx=0x7f65c0005830 comp rx=0 tx=0).stop 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65de000640 1 -- 172.21.15.44:0/115142815 reap_dead start 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 -- 172.21.15.44:0/115142815 shutdown_connections 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 -- 172.21.15.44:0/115142815 >> 172.21.15.44:0/115142815 conn(0x7f65d80cf0d0 msgr2=0x7f65d80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:31.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 -- 172.21.15.44:0/115142815 shutdown_connections 2024-07-17T08:12:31.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.055+0000 7f65dea00640 1 -- 172.21.15.44:0/115142815 wait complete. 2024-07-17T08:12:31.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.057+0000 7f65dea00640 1 Processor -- start 2024-07-17T08:12:31.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.057+0000 7f65dea00640 1 -- start start 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0003a50 0x7f65d80b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d80cbc90 0x7f65d81f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81503c0 con 0x7f65d80b07a0 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65d80b3690 con 0x7f65c0003a50 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.058+0000 7f65dea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65d8150540 con 0x7f65d80cbc90 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.059+0000 7f65dcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.059+0000 7f65dcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60750/0 (socket says 172.21.15.44:60750) 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.059+0000 7f65dcc00640 1 -- 172.21.15.44:0/628968274 learned_addr learned my addr 172.21.15.44:0/628968274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.059+0000 7f65de000640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d80cbc90 0x7f65d81f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.059+0000 7f65dd600640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0003a50 0x7f65d80b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 -- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d80cbc90 msgr2=0x7f65d81f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d80cbc90 0x7f65d81f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 -- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0003a50 msgr2=0x7f65d80b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0003a50 0x7f65d80b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 -- 172.21.15.44:0/628968274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81f40b0 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dcc00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f65c800ea20 tx=0x7f65c800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800cde0 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c8004c30 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81f4370 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8005280 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.060+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81f4930 con 0x7f65d80b07a0 2024-07-17T08:12:31.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.062+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d80cc0a0 con 0x7f65d80b07a0 2024-07-17T08:12:31.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.063+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 90) ==== 98773+0+0 (secure 0 0 0) 0x7f65c8005420 con 0x7f65d80b07a0 2024-07-17T08:12:31.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.064+0000 7f65d5600640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 0x7f65bc078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.064+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f65c8099040 con 0x7f65d80b07a0 2024-07-17T08:12:31.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.065+0000 7f65dd600640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 0x7f65bc078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.066+0000 7f65dd600640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 0x7f65bc078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65d80b41f0 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:31.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.068+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f65d80cc0a0 con 0x7f65d80b07a0 2024-07-17T08:12:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.281+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f65d80ae340 con 0x7f65d80b07a0 2024-07-17T08:12:31.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.287+0000 7f65d5600640 1 -- 172.21.15.44:0/628968274 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v59) ==== 71+0+0 (secure 0 0 0) 0x7f65c8016030 con 0x7f65d80b07a0 2024-07-17T08:12:31.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 msgr2=0x7f65bc078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 0x7f65bc078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65d80b41f0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 msgr2=0x7f65d80cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f65c800ea20 tx=0x7f65c800eee0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 shutdown_connections 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f65bc076b20 0x7f65bc078fe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d80cbc90 0x7f65d81f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65d80b07a0 0x7f65d80cb750 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 --2- 172.21.15.44:0/628968274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c0003a50 0x7f65d80b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.290+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 >> 172.21.15.44:0/628968274 conn(0x7f65d80cf0d0 msgr2=0x7f65d80cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.291+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 shutdown_connections 2024-07-17T08:12:31.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.291+0000 7f65dea00640 1 -- 172.21.15.44:0/628968274 wait complete. 2024-07-17T08:12:31.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2024-07-17T08:12:31.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.527+0000 7f8b5e800640 1 -- 172.21.15.44:0/3927131128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b3c004800 msgr2=0x7f8b3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.527+0000 7f8b5e800640 1 --2- 172.21.15.44:0/3927131128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8b580ad200 tx=0x7f8b3c005830 comp rx=0 tx=0).stop 2024-07-17T08:12:31.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.527+0000 7f8b57e00640 1 -- 172.21.15.44:0/3927131128 reap_dead start 2024-07-17T08:12:31.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.528+0000 7f8b5e800640 1 -- 172.21.15.44:0/3927131128 shutdown_connections 2024-07-17T08:12:31.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.528+0000 7f8b5e800640 1 -- 172.21.15.44:0/3927131128 >> 172.21.15.44:0/3927131128 conn(0x7f8b580b0fb0 msgr2=0x7f8b580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:31.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.528+0000 7f8b5e800640 1 -- 172.21.15.44:0/3927131128 shutdown_connections 2024-07-17T08:12:31.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.528+0000 7f8b5e800640 1 -- 172.21.15.44:0/3927131128 wait complete. 2024-07-17T08:12:31.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.530+0000 7f8b5e800640 1 Processor -- start 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.530+0000 7f8b5e800640 1 -- start start 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b3c004800 0x7f8b580b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b58154440 0x7f8b580af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b580adb50 con 0x7f8b581548a0 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b580ad9d0 con 0x7f8b3c004800 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b5e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b580cdda0 con 0x7f8b58154440 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b57e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b57e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60764/0 (socket says 172.21.15.44:60764) 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.531+0000 7f8b57e00640 1 -- 172.21.15.44:0/2592700842 learned_addr learned my addr 172.21.15.44:0/2592700842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57400640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b3c004800 0x7f8b580b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b4fe00640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b58154440 0x7f8b580af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 -- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b58154440 msgr2=0x7f8b580af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b58154440 0x7f8b580af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 -- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b3c004800 msgr2=0x7f8b580b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b3c004800 0x7f8b580b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 -- 172.21.15.44:0/2592700842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b581533e0 con 0x7f8b581548a0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b57e00640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c004780 tx=0x7f8b500019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.532+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50015020 con 0x7f8b581548a0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.533+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b50011960 con 0x7f8b581548a0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.533+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50011db0 con 0x7f8b581548a0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.533+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b58153680 con 0x7f8b581548a0 2024-07-17T08:12:31.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.533+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b58153c40 con 0x7f8b581548a0 2024-07-17T08:12:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.534+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b580cd690 con 0x7f8b581548a0 2024-07-17T08:12:31.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.535+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 90) ==== 98773+0+0 (secure 0 0 0) 0x7f8b50018020 con 0x7f8b581548a0 2024-07-17T08:12:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.536+0000 7f8b4f400640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 0x7f8b34078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.537+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f8b50098c00 con 0x7f8b581548a0 2024-07-17T08:12:31.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.537+0000 7f8b57400640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 0x7f8b34078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:31.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.538+0000 7f8b57400640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 0x7f8b34078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009b20 tx=0x7f8b48005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:31.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.541+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8b5001f030 con 0x7f8b581548a0 2024-07-17T08:12:31.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.832+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) -- 0x7f8b580cbd00 con 0x7f8b581548a0 2024-07-17T08:12:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.836+0000 7f8b4f400640 1 -- 172.21.15.44:0/2592700842 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2528) ==== 178+0+0 (secure 0 0 0) 0x7f8b50061db0 con 0x7f8b581548a0 2024-07-17T08:12:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.838+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 msgr2=0x7f8b34078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.838+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 0x7f8b34078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009b20 tx=0x7f8b48005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.838+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 msgr2=0x7f8b580afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.838+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c004780 tx=0x7f8b500019b0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 shutdown_connections 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8b34076b20 0x7f8b34078fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b581548a0 0x7f8b580afc60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b58154440 0x7f8b580af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 --2- 172.21.15.44:0/2592700842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b3c004800 0x7f8b580b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 >> 172.21.15.44:0/2592700842 conn(0x7f8b580b0fb0 msgr2=0x7f8b5806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 shutdown_connections 2024-07-17T08:12:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:31.839+0000 7f8b5e800640 1 -- 172.21.15.44:0/2592700842 wait complete. 2024-07-17T08:12:31.860 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2024-07-17T08:12:31.860 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2024-07-17T08:12:31.860 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T08:12:31.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 -- 172.21.15.44:0/1133963408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae40ac760 msgr2=0x7fbae415ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 --2- 172.21.15.44:0/1133963408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae40ac760 0x7fbae415ee90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fbae4070bf0 tx=0x7fbadc02aa10 comp rx=0 tx=0).stop 2024-07-17T08:12:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 -- 172.21.15.44:0/1133963408 shutdown_connections 2024-07-17T08:12:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 --2- 172.21.15.44:0/1133963408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 0x7fbae40ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 --2- 172.21.15.44:0/1133963408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae40ac760 0x7fbae415ee90 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 --2- 172.21.15.44:0/1133963408 >> v2:172.21.15.175:3300/0 conn(0x7fbae41522c0 0x7fbae40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 -- 172.21.15.44:0/1133963408 >> 172.21.15.44:0/1133963408 conn(0x7fbae40b0fb0 msgr2=0x7fbae40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 -- 172.21.15.44:0/1133963408 shutdown_connections 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.069+0000 7fbaeb800640 1 -- 172.21.15.44:0/1133963408 wait complete. 2024-07-17T08:12:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.071+0000 7fbaeb800640 1 Processor -- start 2024-07-17T08:12:32.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.071+0000 7fbaeb800640 1 -- start start 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 0x7fbae414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 0x7fbae4072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbae415f760 con 0x7fbae41522c0 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbae415f5e0 con 0x7fbae40ac760 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.072+0000 7fbaeb800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbae415f8e0 con 0x7fbae40adcd0 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60786/0 (socket says 172.21.15.44:60786) 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 -- 172.21.15.44:0/21571667 learned_addr learned my addr 172.21.15.44:0/21571667 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae8e00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 0x7fbae414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae3e00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 0x7fbae4072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 -- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 msgr2=0x7fbae4072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 0x7fbae4072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 -- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 msgr2=0x7fbae414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 0x7fbae414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae9800640 1 -- 172.21.15.44:0/21571667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae41f4930 con 0x7fbae41522c0 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.073+0000 7fbae8e00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 0x7fbae414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:12:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbae9800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fbadc02b000 tx=0x7fbadc003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc046070 con 0x7fbae41522c0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbadc02f670 con 0x7fbae41522c0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc033050 con 0x7fbae41522c0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae41f4c10 con 0x7fbae41522c0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.074+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae414c710 con 0x7fbae41522c0 2024-07-17T08:12:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.075+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae41f4e50 con 0x7fbae41522c0 2024-07-17T08:12:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.076+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 90) ==== 98773+0+0 (secure 0 0 0) 0x7fbadc049070 con 0x7fbae41522c0 2024-07-17T08:12:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.078+0000 7fbae0c00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 0x7fbac8078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.078+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7fbadc0b9390 con 0x7fbae41522c0 2024-07-17T08:12:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.078+0000 7fbae8e00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 0x7fbac8078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.080+0000 7fbae8e00640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 0x7fbac8078fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005fd0 tx=0x7fbad800f360 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.082+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbadc082550 con 0x7fbae41522c0 2024-07-17T08:12:32.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.295+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbae406e410 con 0x7fbae41522c0 2024-07-17T08:12:32.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.297+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fbadc036070 con 0x7fbae41522c0 2024-07-17T08:12:32.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.301+0000 7fbae0c00640 1 -- 172.21.15.44:0/21571667 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+15139 (secure 0 0 0) 0x7fbadc09d940 con 0x7fbae41522c0 2024-07-17T08:12:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":166,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:30.815202+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2578713851":"2024-07-18T08:12:27.357263+0000","172.21.15.175:0/3939123149":"2024-07-18T08:12:27.357263+0000","172.21.15.175:0/2798255355":"2024-07-18T08:12:27.357263+0000","172.21.15.175:6801/2098888629":"2024-07-18T08:12:27.357263+0000","172.21.15.175:6837/569905425":"2024-07-18T08:12:27.258418+0000","172.21.15.175:6834/569905425":"2024-07-18T08:12:27.258418+0000","172.21.15.175:0/4073021880":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/564005492":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/1341686368":"2024-07-18T08:12:27.357263+0000","172.21.15.175:0/1767233865":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/4152324913":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/3712769329":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6801/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:6800/729654308":"2024-07-18T07:16:18.744944+0000","172.21.15.175:0/4030590413":"2024-07-18T08:12:27.357263+0000","172.21.15.175:6800/2098888629":"2024-07-18T08:12:27.357263+0000","172.21.15.175:0/1983576462":"2024-07-18T08:12:27.357263+0000","172.21.15.175:0/1576245277":"2024-07-18T07:16:18.744944+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 msgr2=0x7fbac8078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 0x7fbac8078fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005fd0 tx=0x7fbad800f360 comp rx=0 tx=0).stop 2024-07-17T08:12:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 msgr2=0x7fbae41f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fbadc02b000 tx=0x7fbadc003ef0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 shutdown_connections 2024-07-17T08:12:32.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fbac8076b20 0x7fbac8078fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae41522c0 0x7fbae41f3e30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae40adcd0 0x7fbae4072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 --2- 172.21.15.44:0/21571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40ac760 0x7fbae414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.305+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 >> 172.21.15.44:0/21571667 conn(0x7fbae40b0fb0 msgr2=0x7fbae40cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.306+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 shutdown_connections 2024-07-17T08:12:32.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.306+0000 7fbaeb800640 1 -- 172.21.15.44:0/21571667 wait complete. 2024-07-17T08:12:32.329 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:12:32.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:12:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.539+0000 7f3c1a200640 1 -- 172.21.15.44:0/3357392282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 msgr2=0x7f3bf8004f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.539+0000 7f3c1a200640 1 --2- 172.21.15.44:0/3357392282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3bf8004f10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400baf0 tx=0x7f3bf8006940 comp rx=0 tx=0).stop 2024-07-17T08:12:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.539+0000 7f3c1a200640 1 -- 172.21.15.44:0/3357392282 shutdown_connections 2024-07-17T08:12:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.539+0000 7f3c1a200640 1 --2- 172.21.15.44:0/3357392282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3bf8004f10 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.539+0000 7f3c1a200640 1 -- 172.21.15.44:0/3357392282 >> 172.21.15.44:0/3357392282 conn(0x7f3c140b0eb0 msgr2=0x7f3c140b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:32.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.540+0000 7f3c1a200640 1 -- 172.21.15.44:0/3357392282 shutdown_connections 2024-07-17T08:12:32.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.540+0000 7f3c1a200640 1 -- 172.21.15.44:0/3357392282 wait complete. 2024-07-17T08:12:32.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.541+0000 7f3c1a200640 1 Processor -- start 2024-07-17T08:12:32.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.542+0000 7f3c1a200640 1 -- start start 2024-07-17T08:12:32.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.542+0000 7f3c1a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c1a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1414afa0 0x7f3c1415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c1a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c141f2ab0 0x7f3c141f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c1a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1406ed10 con 0x7f3bf8004b10 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c1a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1406eb90 con 0x7f3c141f2ab0 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c1a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1406ee90 con 0x7f3c1414afa0 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60800/0 (socket says 172.21.15.44:60800) 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 -- 172.21.15.44:0/2144412818 learned_addr learned my addr 172.21.15.44:0/2144412818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c0aa00640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1414afa0 0x7f3c1415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13e00640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c141f2ab0 0x7f3c141f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 -- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1414afa0 msgr2=0x7f3c1415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1414afa0 0x7f3c1415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 -- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c141f2ab0 msgr2=0x7f3c141f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c141f2ab0 0x7f3c141f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.543+0000 7f3c13400640 1 -- 172.21.15.44:0/2144412818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1415bcc0 con 0x7f3bf8004b10 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c13400640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3c04034040 tx=0x7f3c04032760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04041070 con 0x7f3bf8004b10 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c04032d60 con 0x7f3bf8004b10 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04004290 con 0x7f3bf8004b10 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1414b420 con 0x7f3bf8004b10 2024-07-17T08:12:32.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.544+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c141f6350 con 0x7f3bf8004b10 2024-07-17T08:12:32.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.545+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c140ac510 con 0x7f3bf8004b10 2024-07-17T08:12:32.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.547+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f3c040035f0 con 0x7f3bf8004b10 2024-07-17T08:12:32.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.548+0000 7f3c0be00640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 0x7f3bf0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:32.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.548+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f3c040b9130 con 0x7f3bf8004b10 2024-07-17T08:12:32.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.548+0000 7f3c0aa00640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 0x7f3bf0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:32.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.549+0000 7f3c0aa00640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 0x7f3bf0079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc001860 tx=0x7f3bfc001b20 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:32.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.552+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3c04082290 con 0x7f3bf8004b10 2024-07-17T08:12:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.763+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c14158530 con 0x7f3bf8004b10 2024-07-17T08:12:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.764+0000 7f3c0be00640 1 -- 172.21.15.44:0/2144412818 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) ==== 93+0+10 (secure 0 0 0) 0x7f3c04081b90 con 0x7f3bf8004b10 2024-07-17T08:12:32.766 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 msgr2=0x7f3bf0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 0x7f3bf0079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc001860 tx=0x7f3bfc001b20 comp rx=0 tx=0).stop 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 msgr2=0x7f3c141f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3c04034040 tx=0x7f3c04032760 comp rx=0 tx=0).stop 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 shutdown_connections 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3bf0076bb0 0x7f3bf0079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c141f2ab0 0x7f3c141f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1414afa0 0x7f3c1415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 --2- 172.21.15.44:0/2144412818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bf8004b10 0x7f3c141f5970 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:32.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 >> 172.21.15.44:0/2144412818 conn(0x7f3c140b0eb0 msgr2=0x7f3c14157b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:32.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 shutdown_connections 2024-07-17T08:12:32.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:32.767+0000 7f3c1a200640 1 -- 172.21.15.44:0/2144412818 wait complete. 2024-07-17T08:12:32.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2024-07-17T08:12:32.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:12:32.891 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:12:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.002+0000 7f6a45a00640 1 -- 172.21.15.44:0/1786855688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 msgr2=0x7f6a4006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.002+0000 7f6a45a00640 1 --2- 172.21.15.44:0/1786855688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a4006e3c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009830 tx=0x7f6a3002a3f0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 -- 172.21.15.44:0/1786855688 shutdown_connections 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 --2- 172.21.15.44:0/1786855688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a400b4ec0 0x7f6a400b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 --2- 172.21.15.44:0/1786855688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a4006e3c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 -- 172.21.15.44:0/1786855688 >> 172.21.15.44:0/1786855688 conn(0x7f6a400d00a0 msgr2=0x7f6a400d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 -- 172.21.15.44:0/1786855688 shutdown_connections 2024-07-17T08:12:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.003+0000 7f6a45a00640 1 -- 172.21.15.44:0/1786855688 wait complete. 2024-07-17T08:12:33.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.005+0000 7f6a45a00640 1 Processor -- start 2024-07-17T08:12:33.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.005+0000 7f6a45a00640 1 -- start start 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a400c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4015d6e0 0x7f6a401f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400d1660 con 0x7f6a400b4ec0 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400d14e0 con 0x7f6a4015d300 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.006+0000 7f6a45a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400d17e0 con 0x7f6a4015d6e0 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60812/0 (socket says 172.21.15.44:60812) 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 -- 172.21.15.44:0/195935662 learned_addr learned my addr 172.21.15.44:0/195935662 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:33.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3f400640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a400c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a45000640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4015d6e0 0x7f6a401f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 -- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4015d6e0 msgr2=0x7f6a401f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4015d6e0 0x7f6a401f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 -- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 msgr2=0x7f6a400c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a400c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.007+0000 7f6a3fe00640 1 -- 172.21.15.44:0/195935662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a401f75b0 con 0x7f6a400b4ec0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a3fe00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009950 tx=0x7f6a300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30040070 con 0x7f6a400b4ec0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a30005310 con 0x7f6a400b4ec0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a300044c0 con 0x7f6a400b4ec0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a401f7890 con 0x7f6a400b4ec0 2024-07-17T08:12:33.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.008+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a400c9c70 con 0x7f6a400b4ec0 2024-07-17T08:12:33.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.010+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a400b5300 con 0x7f6a400b4ec0 2024-07-17T08:12:33.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.011+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f6a30039040 con 0x7f6a400b4ec0 2024-07-17T08:12:33.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.012+0000 7f6a37e00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 0x7f6a20079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:33.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.012+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f6a300b9630 con 0x7f6a400b4ec0 2024-07-17T08:12:33.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.013+0000 7f6a3f400640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 0x7f6a20079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:33.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.014+0000 7f6a3f400640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 0x7f6a20079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a401f6e80 tx=0x7f6a2c005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:33.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.016+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6a30031770 con 0x7f6a400b4ec0 2024-07-17T08:12:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.305+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) -- 0x7f6a4015ca20 con 0x7f6a400b4ec0 2024-07-17T08:12:33.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.849+0000 7f6a37e00640 1 -- 172.21.15.44:0/195935662 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2530) ==== 177+0+0 (secure 0 0 0) 0x7f6a30082700 con 0x7f6a400b4ec0 2024-07-17T08:12:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.853+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 msgr2=0x7f6a20079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.853+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 0x7f6a20079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6a401f6e80 tx=0x7f6a2c005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.853+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 msgr2=0x7f6a401f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:33.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.853+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009950 tx=0x7f6a300040c0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.853+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 shutdown_connections 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6a20076bb0 0x7f6a20079070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4015d6e0 0x7f6a401f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4015d300 0x7f6a400c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 --2- 172.21.15.44:0/195935662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a400b4ec0 0x7f6a401f6f20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 >> 172.21.15.44:0/195935662 conn(0x7f6a400d00a0 msgr2=0x7f6a4006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 shutdown_connections 2024-07-17T08:12:33.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:33.854+0000 7f6a45a00640 1 -- 172.21.15.44:0/195935662 wait complete. 2024-07-17T08:12:33.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 -- 172.21.15.44:0/976367338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c0cf310 msgr2=0x7f818c0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 --2- 172.21.15.44:0/976367338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c0cf310 0x7f818c0cf780 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f818c06f960 tx=0x7f817802a530 comp rx=0 tx=0).stop 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 -- 172.21.15.44:0/976367338 shutdown_connections 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 --2- 172.21.15.44:0/976367338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c0cf310 0x7f818c0cf780 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 -- 172.21.15.44:0/976367338 >> 172.21.15.44:0/976367338 conn(0x7f818c072770 msgr2=0x7f818c072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 -- 172.21.15.44:0/976367338 shutdown_connections 2024-07-17T08:12:34.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.136+0000 7f8194000640 1 -- 172.21.15.44:0/976367338 wait complete. 2024-07-17T08:12:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.138+0000 7f8194000640 1 Processor -- start 2024-07-17T08:12:34.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.138+0000 7f8194000640 1 -- start start 2024-07-17T08:12:34.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 0x7f818c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f818c0cf310 0x7f818c15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f818c15c410 con 0x7f818c15b420 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f818c15c290 con 0x7f818c0b5870 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.139+0000 7f8194000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f818c0b5ca0 con 0x7f818c0cf310 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60840/0 (socket says 172.21.15.44:60840) 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 -- 172.21.15.44:0/1451950374 learned_addr learned my addr 172.21.15.44:0/1451950374 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8191600640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 0x7f818c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 -- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f818c0cf310 msgr2=0x7f818c15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f818c0cf310 0x7f818c15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 -- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 msgr2=0x7f818c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 0x7f818c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 -- 172.21.15.44:0/1451950374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1f7b40 con 0x7f818c15b420 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8191600640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 0x7f818c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.140+0000 7f8192000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f818400c780 tx=0x7f818400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.141+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184004b60 con 0x7f818c15b420 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.141+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8184004df0 con 0x7f818c15b420 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.141+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818400aa70 con 0x7f818c15b420 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.141+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1f7e20 con 0x7f818c15b420 2024-07-17T08:12:34.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.141+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1f83e0 con 0x7f818c15b420 2024-07-17T08:12:34.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.142+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c0762e0 con 0x7f818c15b420 2024-07-17T08:12:34.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.143+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f818400ac10 con 0x7f818c15b420 2024-07-17T08:12:34.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.144+0000 7f8189600640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 0x7f816c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.144+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f8184098990 con 0x7f818c15b420 2024-07-17T08:12:34.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.145+0000 7f8191600640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 0x7f816c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.146+0000 7f8191600640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 0x7f816c0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f817c0016a0 tx=0x7f817c009040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:34.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.149+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f818c0762e0 con 0x7f818c15b420 2024-07-17T08:12:34.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.360+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f818c0b6150 con 0x7f818c15b420 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.361+0000 7f8189600640 1 -- 172.21.15.44:0/1451950374 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v59) ==== 59+0+1998 (secure 0 0 0) 0x7f8184020070 con 0x7f818c15b420 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:12:34.363 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:12:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:12:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:12:34.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 msgr2=0x7f816c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 0x7f816c0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f817c0016a0 tx=0x7f817c009040 comp rx=0 tx=0).stop 2024-07-17T08:12:34.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 msgr2=0x7f818c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f818400c780 tx=0x7f818400cc40 comp rx=0 tx=0).stop 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 shutdown_connections 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f816c076be0 0x7f816c0790a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f818c15b420 0x7f818c15d090 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f818c0cf310 0x7f818c15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.364+0000 7f8194000640 1 --2- 172.21.15.44:0/1451950374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f818c0b5870 0x7f818c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.365+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 >> 172.21.15.44:0/1451950374 conn(0x7f818c072770 msgr2=0x7f818c0b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.365+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 shutdown_connections 2024-07-17T08:12:34.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.365+0000 7f8194000640 1 -- 172.21.15.44:0/1451950374 wait complete. 2024-07-17T08:12:34.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:12:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.602+0000 7f0fb6000640 1 -- 172.21.15.44:0/2095066768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 msgr2=0x7f0fb006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.602+0000 7f0fb6000640 1 --2- 172.21.15.44:0/2095066768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb006d620 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00cce50 tx=0x7f0fa002a560 comp rx=0 tx=0).stop 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 -- 172.21.15.44:0/2095066768 shutdown_connections 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 --2- 172.21.15.44:0/2095066768 >> v2:172.21.15.175:3300/0 conn(0x7f0fb00cbab0 0x7f0fb015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 --2- 172.21.15.44:0/2095066768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb006d620 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 -- 172.21.15.44:0/2095066768 >> 172.21.15.44:0/2095066768 conn(0x7f0fb00b0eb0 msgr2=0x7f0fb00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 -- 172.21.15.44:0/2095066768 shutdown_connections 2024-07-17T08:12:34.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.603+0000 7f0fb6000640 1 -- 172.21.15.44:0/2095066768 wait complete. 2024-07-17T08:12:34.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.604+0000 7f0fb6000640 1 Processor -- start 2024-07-17T08:12:34.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.605+0000 7f0fb6000640 1 -- start start 2024-07-17T08:12:34.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 0x7f0fb014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb00cbab0 0x7f0fb01f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb00cd780 con 0x7f0fb0152220 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb00cd600 con 0x7f0fb006ed30 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fb6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb00cd900 con 0x7f0fb00cbab0 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60850/0 (socket says 172.21.15.44:60850) 2024-07-17T08:12:34.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fafe00640 1 -- 172.21.15.44:0/3647916960 learned_addr learned my addr 172.21.15.44:0/3647916960 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0faf400640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 0x7f0fb014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.606+0000 7f0fafe00640 1 -- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb00cbab0 msgr2=0x7f0fb01f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fa7e00640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb00cbab0 0x7f0fb01f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fafe00640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb00cbab0 0x7f0fb01f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fafe00640 1 -- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 msgr2=0x7f0fb014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fafe00640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 0x7f0fb014e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fafe00640 1 -- 172.21.15.44:0/3647916960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb01f7f90 con 0x7f0fb0152220 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0faf400640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 0x7f0fb014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fafe00640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800eb40 tx=0x7f0fa800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:34.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800ce50 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa8004c10 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb01f8210 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8005020 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.607+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb01f87d0 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.609+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb00cde20 con 0x7f0fb0152220 2024-07-17T08:12:34.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.610+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f0fa80051c0 con 0x7f0fb0152220 2024-07-17T08:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.611+0000 7f0fa7400640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 0x7f0f8c079240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:34.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.611+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f0fa8099460 con 0x7f0fb0152220 2024-07-17T08:12:34.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.611+0000 7f0faf400640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 0x7f0f8c079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:34.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.613+0000 7f0faf400640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 0x7f0f8c079240 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0033000 tx=0x7f0fa0004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:34.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.615+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0fb01f7f90 con 0x7f0fb0152220 2024-07-17T08:12:34.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.827+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0fb0068510 con 0x7f0fb0152220 2024-07-17T08:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.834+0000 7f0fa7400640 1 -- 172.21.15.44:0/3647916960 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v60)=0 v60) ==== 122+0+0 (secure 0 0 0) 0x7f0fa8062530 con 0x7f0fb0152220 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 msgr2=0x7f0f8c079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 0x7f0f8c079240 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0033000 tx=0x7f0fa0004730 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 msgr2=0x7f0fb01f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800eb40 tx=0x7f0fa800c2d0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 shutdown_connections 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0f8c076d80 0x7f0f8c079240 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb0152220 0x7f0fb01f7a50 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fb00cbab0 0x7f0fb01f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 --2- 172.21.15.44:0/3647916960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fb006ed30 0x7f0fb014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 >> 172.21.15.44:0/3647916960 conn(0x7f0fb00b0eb0 msgr2=0x7f0fb00b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 shutdown_connections 2024-07-17T08:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:34.837+0000 7f0fb6000640 1 -- 172.21.15.44:0/3647916960 wait complete. 2024-07-17T08:12:34.860 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:12:34.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:12:34.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:12:35.091 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:12:35.091 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:35.091 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:35.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:12:35.148 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:12:35.148 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:35.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:12:35.204 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:35.204 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:35.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:12:35.261 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:12:35.261 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:35.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:12:35.316 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:12:35.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 -- 172.21.15.44:0/386005495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 msgr2=0x7f708400c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 --2- 172.21.15.44:0/386005495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f708400c410 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f709800a790 tx=0x7f7084003810 comp rx=0 tx=0).stop 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a6200640 1 -- 172.21.15.44:0/386005495 reap_dead start 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 -- 172.21.15.44:0/386005495 shutdown_connections 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 -- 172.21.15.44:0/386005495 >> 172.21.15.44:0/386005495 conn(0x7f70a00b0fb0 msgr2=0x7f70a00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 -- 172.21.15.44:0/386005495 shutdown_connections 2024-07-17T08:12:35.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.532+0000 7f70a8200640 1 -- 172.21.15.44:0/386005495 wait complete. 2024-07-17T08:12:35.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.534+0000 7f70a8200640 1 Processor -- start 2024-07-17T08:12:35.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.534+0000 7f70a8200640 1 -- start start 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70a015b6d0 0x7f70a01f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a01f5780 0x7f70a0157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70a0153fb0 con 0x7f7084009f80 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70a0153e30 con 0x7f70a015b6d0 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a8200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70a006f710 con 0x7f70a01f5780 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60858/0 (socket says 172.21.15.44:60858) 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a5800640 1 -- 172.21.15.44:0/3522950531 learned_addr learned my addr 172.21.15.44:0/3522950531 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:35.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a4e00640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70a015b6d0 0x7f70a01f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a6200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a01f5780 0x7f70a0157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.535+0000 7f70a5800640 1 -- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a01f5780 msgr2=0x7f70a0157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a5800640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a01f5780 0x7f70a0157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a5800640 1 -- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70a015b6d0 msgr2=0x7f70a01f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a5800640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70a015b6d0 0x7f70a01f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a5800640 1 -- 172.21.15.44:0/3522950531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a0151100 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a5800640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7090004e20 tx=0x7f7090001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090015020 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7090011960 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7090011d70 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a01513a0 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.536+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a0151960 con 0x7f7084009f80 2024-07-17T08:12:35.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.537+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a00b4010 con 0x7f7084009f80 2024-07-17T08:12:35.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.539+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f7090018020 con 0x7f7084009f80 2024-07-17T08:12:35.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.540+0000 7f709d600640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 0x7f70800790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:35.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.540+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f7090098ff0 con 0x7f7084009f80 2024-07-17T08:12:35.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.540+0000 7f70a4e00640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 0x7f70800790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:35.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.542+0000 7f70a4e00640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 0x7f70800790a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70a015b660 tx=0x7f708c005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:35.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.544+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f709001f020 con 0x7f7084009f80 2024-07-17T08:12:35.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.800+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70a00b3730 con 0x7f7084009f80 2024-07-17T08:12:35.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.801+0000 7f709d600640 1 -- 172.21.15.44:0/3522950531 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+3914 (secure 0 0 0) 0x7f70900620e0 con 0x7f7084009f80 2024-07-17T08:12:35.803 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:35.803 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":81,"btime":"2024-07-17T08:12:28:367627+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6518,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3734078433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3734078433},{"type":"v1","addr":"172.21.15.44:6833","nonce":3734078433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4233651924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4233651924},{"type":"v1","addr":"172.21.15.44:6835","nonce":4233651924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2501363942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2501363942},{"type":"v1","addr":"172.21.15.175:6836","nonce":2501363942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":7147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/564572997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":564572997},{"type":"v1","addr":"172.21.15.175:6801","nonce":564572997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-07-17T08:12:35.803 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 81 2024-07-17T08:12:35.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 msgr2=0x7f70800790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 0x7f70800790a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70a015b660 tx=0x7f708c005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 msgr2=0x7f70a01f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7090004e20 tx=0x7f7090001630 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 shutdown_connections 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7080076be0 0x7f70800790a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70a01f5780 0x7f70a0157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70a015b6d0 0x7f70a01f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 --2- 172.21.15.44:0/3522950531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7084009f80 0x7f70a01f4e60 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:35.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 >> 172.21.15.44:0/3522950531 conn(0x7f70a00b0fb0 msgr2=0x7f70a015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:35.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.804+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 shutdown_connections 2024-07-17T08:12:35.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:35.805+0000 7f70a8200640 1 -- 172.21.15.44:0/3522950531 wait complete. 2024-07-17T08:12:35.828 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:12:35.828 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:12:35.828 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:12:35.829 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:12:35.829 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:12:35.829 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:12:35.829 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:12:35.829 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:12:35.829 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:12:35.829 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:12:35.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:12:35.830 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:12:35.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:12:35.830 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:12:35.830 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:12:35.830 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:12:35.853 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:12:35.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:12:35.858 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:12:35.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:12:35.860 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:12:35.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:12:35.865 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:12:35.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:12:35.905 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:12:35.906 DEBUG:teuthology.parallel:result is None 2024-07-17T08:12:35.909 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:12:35.909 DEBUG:teuthology.parallel:result is None 2024-07-17T08:12:35.911 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:12:35.911 DEBUG:teuthology.parallel:result is None 2024-07-17T08:12:35.915 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:12:35.915 DEBUG:teuthology.parallel:result is None 2024-07-17T08:12:35.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:12:36.052 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:12:36.103 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:12:36.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-07-17T08:12:36.104 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:12:36.104 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 -- 172.21.15.44:0/2353869729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 msgr2=0x7f975c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 --2- 172.21.15.44:0/2353869729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c0ce5b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f974c009830 tx=0x7f974c02a530 comp rx=0 tx=0).stop 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 -- 172.21.15.44:0/2353869729 shutdown_connections 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 --2- 172.21.15.44:0/2353869729 >> v2:172.21.15.175:3300/0 conn(0x7f975c0b6790 0x7f975c161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 --2- 172.21.15.44:0/2353869729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c0ce5b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 -- 172.21.15.44:0/2353869729 >> 172.21.15.44:0/2353869729 conn(0x7f975c0d3700 msgr2=0x7f975c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 -- 172.21.15.44:0/2353869729 shutdown_connections 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.098+0000 7f9763600640 1 -- 172.21.15.44:0/2353869729 wait complete. 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.100+0000 7f9763600640 1 Processor -- start 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.100+0000 7f9763600640 1 -- start start 2024-07-17T08:12:36.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f975c0b29e0 0x7f975c150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c1f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f975c162420 con 0x7f975c0b6790 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1622a0 con 0x7f975c0b29e0 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9763600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f975c167910 con 0x7f975c0ce1d0 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9753e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9753e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60902/0 (socket says 172.21.15.44:60902) 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9753e00640 1 -- 172.21.15.44:0/875542058 learned_addr learned my addr 172.21.15.44:0/875542058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.101+0000 7f9761600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c1f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9760c00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f975c0b29e0 0x7f975c150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 -- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 msgr2=0x7f975c1f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c1f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 -- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f975c0b29e0 msgr2=0x7f975c150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f975c0b29e0 0x7f975c150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 -- 172.21.15.44:0/875542058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c15b160 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9753e00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f974400e9f0 tx=0x7f974400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974400cd80 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c14d420 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9744005020 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.102+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744005450 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.103+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c14dfb0 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.104+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c0ce5b0 con 0x7f975c0b6790 2024-07-17T08:12:36.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.105+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f97440047a0 con 0x7f975c0b6790 2024-07-17T08:12:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.107+0000 7f9759600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 0x7f9738079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.107+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6571+0+0 (secure 0 0 0) 0x7f9744099120 con 0x7f975c0b6790 2024-07-17T08:12:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.107+0000 7f9760c00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 0x7f9738079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:36.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.108+0000 7f9760c00640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 0x7f9738079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f974c034000 tx=0x7f974c033040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:36.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.111+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f97440622c0 con 0x7f975c0b6790 2024-07-17T08:12:36.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.334+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f975c157810 con 0x7f975c0b6790 2024-07-17T08:12:36.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.890+0000 7f9759600640 1 -- 172.21.15.44:0/875542058 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v168) ==== 97+0+0 (secure 0 0 0) 0x7f9744061bc0 con 0x7f975c0b6790 2024-07-17T08:12:36.892 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:12:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 msgr2=0x7f9738079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 0x7f9738079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f974c034000 tx=0x7f974c033040 comp rx=0 tx=0).stop 2024-07-17T08:12:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 msgr2=0x7f975c1f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f974400e9f0 tx=0x7f974400eeb0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 shutdown_connections 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f9738076bb0 0x7f9738079070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f975c0ce1d0 0x7f975c1f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f975c0b6790 0x7f975c1f70a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 --2- 172.21.15.44:0/875542058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f975c0b29e0 0x7f975c150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 >> 172.21.15.44:0/875542058 conn(0x7f975c0d3700 msgr2=0x7f975c157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:36.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 shutdown_connections 2024-07-17T08:12:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:36.894+0000 7f9763600640 1 -- 172.21.15.44:0/875542058 wait complete. 2024-07-17T08:12:36.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:12:37.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.119+0000 7f2738e00640 1 -- 172.21.15.44:0/3703010471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27340b39c0 msgr2=0x7f27340b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.119+0000 7f2738e00640 1 --2- 172.21.15.44:0/3703010471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27340b39c0 0x7f27340b3da0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2724005fd0 tx=0x7f272402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.119+0000 7f2738e00640 1 -- 172.21.15.44:0/3703010471 shutdown_connections 2024-07-17T08:12:37.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.119+0000 7f2738e00640 1 --2- 172.21.15.44:0/3703010471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27340b39c0 0x7f27340b3da0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.119+0000 7f2738e00640 1 -- 172.21.15.44:0/3703010471 >> 172.21.15.44:0/3703010471 conn(0x7f27340b15b0 msgr2=0x7f27340b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:37.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.120+0000 7f2738e00640 1 -- 172.21.15.44:0/3703010471 shutdown_connections 2024-07-17T08:12:37.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.120+0000 7f2738e00640 1 -- 172.21.15.44:0/3703010471 wait complete. 2024-07-17T08:12:37.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.121+0000 7f2738e00640 1 Processor -- start 2024-07-17T08:12:37.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.122+0000 7f2738e00640 1 -- start start 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 0x7f27340cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2734148c00 0x7f2734148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f273415a370 con 0x7f273406ff40 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f273415a4f0 con 0x7f27340b39c0 2024-07-17T08:12:37.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2738e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27340cf050 con 0x7f2734148c00 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2731e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2731e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60922/0 (socket says 172.21.15.44:60922) 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2731e00640 1 -- 172.21.15.44:0/3693410326 learned_addr learned my addr 172.21.15.44:0/3693410326 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.123+0000 7f2731400640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 0x7f27340cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 -- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2734148c00 msgr2=0x7f2734148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2734148c00 0x7f2734148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 -- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 msgr2=0x7f27340cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 0x7f27340cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 -- 172.21.15.44:0/3693410326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734149af0 con 0x7f273406ff40 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731400640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 0x7f27340cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2731e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2724034040 tx=0x7f27240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272403e030 con 0x7f273406ff40 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2724003ca0 con 0x7f273406ff40 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2734149dd0 con 0x7f273406ff40 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724035040 con 0x7f273406ff40 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.124+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273414a390 con 0x7f273406ff40 2024-07-17T08:12:37.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.125+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27340b3da0 con 0x7f273406ff40 2024-07-17T08:12:37.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.127+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f2724041030 con 0x7f273406ff40 2024-07-17T08:12:37.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.128+0000 7f272a000640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 0x7f27140790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.128+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5797+0+0 (secure 0 0 0) 0x7f27240b8800 con 0x7f273406ff40 2024-07-17T08:12:37.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.129+0000 7f2731400640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 0x7f27140790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.130+0000 7f2731400640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 0x7f27140790a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f273414cf80 tx=0x7f2720009040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:37.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.132+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2724081bd0 con 0x7f273406ff40 2024-07-17T08:12:37.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.429+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f273415b220 con 0x7f273406ff40 2024-07-17T08:12:37.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.430+0000 7f272a000640 1 -- 172.21.15.44:0/3693410326 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v60) ==== 80+0+5304 (secure 0 0 0) 0x7f27240814d0 con 0x7f273406ff40 2024-07-17T08:12:37.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:37.432 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.433 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:12:37.435 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:12:37.436 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:12:37.438 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.440 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.442 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:12:37.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:12:37.444 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:12:37.445 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:12:37.447 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:12:37.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.433+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 msgr2=0x7f27140790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.433+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 0x7f27140790a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f273414cf80 tx=0x7f2720009040 comp rx=0 tx=0).stop 2024-07-17T08:12:37.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.433+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 msgr2=0x7f27341f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.433+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2724034040 tx=0x7f27240038a0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 shutdown_connections 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f2714076be0 0x7f27140790a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2734148c00 0x7f2734148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27340b39c0 0x7f27340cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 --2- 172.21.15.44:0/3693410326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f273406ff40 0x7f27341f6030 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 >> 172.21.15.44:0/3693410326 conn(0x7f27340b15b0 msgr2=0x7f273406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 shutdown_connections 2024-07-17T08:12:37.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.434+0000 7f2738e00640 1 -- 172.21.15.44:0/3693410326 wait complete. 2024-07-17T08:12:37.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 -- 172.21.15.44:0/1771493122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 msgr2=0x7efcd40738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 --2- 172.21.15.44:0/1771493122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd40738c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005fd0 tx=0x7efcc402aa10 comp rx=0 tx=0).stop 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 -- 172.21.15.44:0/1771493122 shutdown_connections 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 --2- 172.21.15.44:0/1771493122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd40738c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 --2- 172.21.15.44:0/1771493122 >> v2:172.21.15.175:3300/0 conn(0x7efcd415b1d0 0x7efcd40add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 -- 172.21.15.44:0/1771493122 >> 172.21.15.44:0/1771493122 conn(0x7efcd40b5890 msgr2=0x7efcd40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 -- 172.21.15.44:0/1771493122 shutdown_connections 2024-07-17T08:12:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.656+0000 7efcda600640 1 -- 172.21.15.44:0/1771493122 wait complete. 2024-07-17T08:12:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.658+0000 7efcda600640 1 Processor -- start 2024-07-17T08:12:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.658+0000 7efcda600640 1 -- start start 2024-07-17T08:12:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4073e00 0x7efcd41f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcd40745e0 con 0x7efcd415b1d0 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4074460 con 0x7efcd4073e00 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcda600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efcd415f4c0 con 0x7efcd40b2a00 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcd9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcd9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60930/0 (socket says 172.21.15.44:60930) 2024-07-17T08:12:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcd9c00640 1 -- 172.21.15.44:0/3134230872 learned_addr learned my addr 172.21.15.44:0/3134230872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcd3e00640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.659+0000 7efcd9200640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4073e00 0x7efcd41f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 -- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 msgr2=0x7efcd414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 -- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4073e00 msgr2=0x7efcd41f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4073e00 0x7efcd41f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 -- 172.21.15.44:0/3134230872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd414e5d0 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd9c00640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c3c0 tx=0x7efccc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc017070 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efccc0053c0 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc003850 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd414e8b0 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.660+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd414ee70 con 0x7efcd415b1d0 2024-07-17T08:12:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.662+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd415f9e0 con 0x7efcd415b1d0 2024-07-17T08:12:37.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.663+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7efccc01a070 con 0x7efcd415b1d0 2024-07-17T08:12:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.664+0000 7efcd0c00640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 0x7efcb0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.664+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5797+0+0 (secure 0 0 0) 0x7efccc097740 con 0x7efcd415b1d0 2024-07-17T08:12:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.664+0000 7efcd9200640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 0x7efcb0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:37.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.665+0000 7efcd9200640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 0x7efcb0079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efcd415e4b0 tx=0x7efcbc005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:37.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.668+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efccc060ba0 con 0x7efcd415b1d0 2024-07-17T08:12:37.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.885+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efcd40aeac0 con 0x7efcd415b1d0 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.886+0000 7efcd0c00640 1 -- 172.21.15.44:0/3134230872 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v170) ==== 84+0+65 (secure 0 0 0) 0x7efccc0604a0 con 0x7efcd415b1d0 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:37.888 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 msgr2=0x7efcb0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 0x7efcb0079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efcd415e4b0 tx=0x7efcbc005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 msgr2=0x7efcd414d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c3c0 tx=0x7efccc00c880 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 shutdown_connections 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7efcb0076bb0 0x7efcb0079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcd415b1d0 0x7efcd414d640 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40b2a00 0x7efcd414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 --2- 172.21.15.44:0/3134230872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4073e00 0x7efcd41f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:37.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 >> 172.21.15.44:0/3134230872 conn(0x7efcd40b5890 msgr2=0x7efcd40add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:37.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 shutdown_connections 2024-07-17T08:12:37.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:37.889+0000 7efcda600640 1 -- 172.21.15.44:0/3134230872 wait complete. 2024-07-17T08:12:37.911 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:12:37.911 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:12:38.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.114+0000 7fdd86200640 1 -- 172.21.15.44:0/3702154783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd64005160 msgr2=0x7fdd64005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.114+0000 7fdd86200640 1 --2- 172.21.15.44:0/3702154783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd64005160 0x7fdd64005910 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdd800cf170 tx=0x7fdd64006940 comp rx=0 tx=0).stop 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.115+0000 7fdd7fe00640 1 -- 172.21.15.44:0/3702154783 reap_dead start 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.115+0000 7fdd86200640 1 -- 172.21.15.44:0/3702154783 shutdown_connections 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.115+0000 7fdd86200640 1 -- 172.21.15.44:0/3702154783 >> 172.21.15.44:0/3702154783 conn(0x7fdd800b0eb0 msgr2=0x7fdd800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.115+0000 7fdd86200640 1 -- 172.21.15.44:0/3702154783 shutdown_connections 2024-07-17T08:12:38.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.115+0000 7fdd86200640 1 -- 172.21.15.44:0/3702154783 wait complete. 2024-07-17T08:12:38.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.116+0000 7fdd86200640 1 Processor -- start 2024-07-17T08:12:38.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.117+0000 7fdd86200640 1 -- start start 2024-07-17T08:12:38.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.117+0000 7fdd86200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd801f3dc0 0x7fdd8014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:38.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd86200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:38.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd86200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd801f7f80 0x7fdd800cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:38.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd86200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800ac810 con 0x7fdd8014eb50 2024-07-17T08:12:38.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd86200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800ac690 con 0x7fdd801f3dc0 2024-07-17T08:12:38.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd86200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd800ac990 con 0x7fdd801f7f80 2024-07-17T08:12:38.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:38.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60944/0 (socket says 172.21.15.44:60944) 2024-07-17T08:12:38.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 -- 172.21.15.44:0/2915815409 learned_addr learned my addr 172.21.15.44:0/2915815409 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7fe00640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd801f7f80 0x7fdd800cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 -- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd801f7f80 msgr2=0x7fdd800cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7f400640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd801f3dc0 0x7fdd8014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd801f7f80 0x7fdd800cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 -- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd801f3dc0 msgr2=0x7fdd8014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd801f3dc0 0x7fdd8014f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.118+0000 7fdd7ea00640 1 -- 172.21.15.44:0/2915815409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd801f8930 con 0x7fdd8014eb50 2024-07-17T08:12:38.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd7ea00640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c001880 tx=0x7fdd6c00f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c001c50 con 0x7fdd8014eb50 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd6c0039c0 con 0x7fdd8014eb50 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c003dd0 con 0x7fdd8014eb50 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd80099340 con 0x7fdd8014eb50 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.119+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801f41a0 con 0x7fdd8014eb50 2024-07-17T08:12:38.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.120+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd80099860 con 0x7fdd8014eb50 2024-07-17T08:12:38.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.122+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fdd6c02d560 con 0x7fdd8014eb50 2024-07-17T08:12:38.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.123+0000 7fdd77400640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 0x7fdd60079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:38.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.123+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5797+0+0 (secure 0 0 0) 0x7fdd6c0b8190 con 0x7fdd8014eb50 2024-07-17T08:12:38.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.124+0000 7fdd7f400640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 0x7fdd60079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:38.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.125+0000 7fdd7f400640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 0x7fdd60079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006490 tx=0x7fdd70006a40 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:38.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.128+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdd80099860 con 0x7fdd8014eb50 2024-07-17T08:12:38.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.340+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdd8006d620 con 0x7fdd8014eb50 2024-07-17T08:12:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.919+0000 7fdd77400640 1 -- 172.21.15.44:0/2915815409 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v171) ==== 138+0+0 (secure 0 0 0) 0x7fdd6c0815e0 con 0x7fdd8014eb50 2024-07-17T08:12:38.921 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:12:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.922+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 msgr2=0x7fdd60079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.922+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 0x7fdd60079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006490 tx=0x7fdd70006a40 comp rx=0 tx=0).stop 2024-07-17T08:12:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.922+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 msgr2=0x7fdd801f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:38.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.922+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c001880 tx=0x7fdd6c00f470 comp rx=0 tx=0).stop 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 shutdown_connections 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fdd60076bb0 0x7fdd60079070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd801f7f80 0x7fdd800cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd8014eb50 0x7fdd801f7a40 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 --2- 172.21.15.44:0/2915815409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd801f3dc0 0x7fdd8014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 >> 172.21.15.44:0/2915815409 conn(0x7fdd800b0eb0 msgr2=0x7fdd80158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 shutdown_connections 2024-07-17T08:12:38.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:38.923+0000 7fdd86200640 1 -- 172.21.15.44:0/2915815409 wait complete. 2024-07-17T08:12:38.953 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:12:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 -- 172.21.15.44:0/4179906226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f757415d740 msgr2=0x7f757406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 --2- 172.21.15.44:0/4179906226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f757415d740 0x7f757406c410 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f757408c9f0 tx=0x7f756402a440 comp rx=0 tx=0).stop 2024-07-17T08:12:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 -- 172.21.15.44:0/4179906226 shutdown_connections 2024-07-17T08:12:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 --2- 172.21.15.44:0/4179906226 >> v2:172.21.15.175:3300/0 conn(0x7f75740ce710 0x7f757406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 --2- 172.21.15.44:0/4179906226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f757415d740 0x7f757406c410 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 -- 172.21.15.44:0/4179906226 >> 172.21.15.44:0/4179906226 conn(0x7f75740d00f0 msgr2=0x7f75740d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 -- 172.21.15.44:0/4179906226 shutdown_connections 2024-07-17T08:12:39.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.169+0000 7f7579600640 1 -- 172.21.15.44:0/4179906226 wait complete. 2024-07-17T08:12:39.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.170+0000 7f7579600640 1 Processor -- start 2024-07-17T08:12:39.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.170+0000 7f7579600640 1 -- start start 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 0x7f75741f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 0x7f7574157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f757415dca0 con 0x7f75740d11b0 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f757415db20 con 0x7f757415d740 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7579600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75740ca370 con 0x7f75740ce710 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:39.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48454/0 (socket says 172.21.15.44:48454) 2024-07-17T08:12:39.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 -- 172.21.15.44:0/805176252 learned_addr learned my addr 172.21.15.44:0/805176252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572a00640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 0x7f75741f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7573400640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 0x7f7574157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 -- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 msgr2=0x7f75741f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 0x7f75741f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 -- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 msgr2=0x7f7574157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 0x7f7574157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 -- 172.21.15.44:0/805176252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740cc720 con 0x7f75740d11b0 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7573400640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 0x7f7574157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:39.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572a00640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 0x7f75741f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.172+0000 7f7572000640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f755c00d7a0 tx=0x7f755c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.173+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c017070 con 0x7f75740d11b0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.173+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f755c009d20 con 0x7f75740d11b0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.173+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c005280 con 0x7f75740d11b0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.173+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75740cca00 con 0x7f75740d11b0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.173+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75740caa30 con 0x7f75740d11b0 2024-07-17T08:12:39.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.174+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757415de20 con 0x7f75740d11b0 2024-07-17T08:12:39.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.177+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f755c005420 con 0x7f75740d11b0 2024-07-17T08:12:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.178+0000 7f756aa00640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 0x7f7550079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:39.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.178+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 6167+0+0 (secure 0 0 0) 0x7f755c098b00 con 0x7f75740d11b0 2024-07-17T08:12:39.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.178+0000 7f7572a00640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 0x7f7550079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:39.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.180+0000 7f7572a00640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 0x7f7550079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7564033000 tx=0x7f7564004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:39.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.181+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f755c061de0 con 0x7f75740d11b0 2024-07-17T08:12:39.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.395+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f75740b6fb0 con 0x7f75740d11b0 2024-07-17T08:12:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.937+0000 7f756aa00640 1 -- 172.21.15.44:0/805176252 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v172) ==== 170+0+0 (secure 0 0 0) 0x7f755c0616e0 con 0x7f75740d11b0 2024-07-17T08:12:39.938 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:12:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.942+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 msgr2=0x7f7550079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:39.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 0x7f7550079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7564033000 tx=0x7f7564004730 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 msgr2=0x7f75740cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f755c00d7a0 tx=0x7f755c00dc60 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 shutdown_connections 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7550076bb0 0x7f7550079070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f757415d740 0x7f7574157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75740d11b0 0x7f75740cbfd0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 --2- 172.21.15.44:0/805176252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75740ce710 0x7f75741f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:39.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 >> 172.21.15.44:0/805176252 conn(0x7f75740d00f0 msgr2=0x7f757415fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.943+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 shutdown_connections 2024-07-17T08:12:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:39.944+0000 7f7579600640 1 -- 172.21.15.44:0/805176252 wait complete. 2024-07-17T08:12:39.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.165+0000 7f7027c00640 1 -- 172.21.15.44:0/3536368347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 msgr2=0x7f70200b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.165+0000 7f7027c00640 1 --2- 172.21.15.44:0/3536368347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f70200b4f10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f70200b0620 tx=0x7f700802a560 comp rx=0 tx=0).stop 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 -- 172.21.15.44:0/3536368347 shutdown_connections 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 --2- 172.21.15.44:0/3536368347 >> v2:172.21.15.175:3300/0 conn(0x7f70200b0fe0 0x7f70200b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 --2- 172.21.15.44:0/3536368347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f70200b4f10 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 -- 172.21.15.44:0/3536368347 >> 172.21.15.44:0/3536368347 conn(0x7f7020073d50 msgr2=0x7f702009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 -- 172.21.15.44:0/3536368347 shutdown_connections 2024-07-17T08:12:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.166+0000 7f7027c00640 1 -- 172.21.15.44:0/3536368347 wait complete. 2024-07-17T08:12:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.167+0000 7f7027c00640 1 Processor -- start 2024-07-17T08:12:40.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.168+0000 7f7027c00640 1 -- start start 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70200b0fe0 0x7f70201571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f702015b5d0 0x7f70200b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70201653a0 con 0x7f70200b01b0 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7020165220 con 0x7f70200b0fe0 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7027c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7020165520 con 0x7f702015b5d0 2024-07-17T08:12:40.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7025200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7025200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48486/0 (socket says 172.21.15.44:48486) 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7025200640 1 -- 172.21.15.44:0/408901263 learned_addr learned my addr 172.21.15.44:0/408901263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f701fe00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70200b0fe0 0x7f70201571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7025c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f702015b5d0 0x7f70200b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.169+0000 7f7025200640 1 -- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f702015b5d0 msgr2=0x7f70200b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7025200640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f702015b5d0 0x7f70200b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7025200640 1 -- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70200b0fe0 msgr2=0x7f70201571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7025200640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70200b0fe0 0x7f70201571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7025200640 1 -- 172.21.15.44:0/408901263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70201f51e0 con 0x7f70200b01b0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7025200640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f701000ba50 tx=0x7f701000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010019070 con 0x7f70200b01b0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f701000cd10 con 0x7f70200b01b0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70100146a0 con 0x7f70200b01b0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702014c3c0 con 0x7f70200b01b0 2024-07-17T08:12:40.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.170+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702014d3e0 con 0x7f70200b01b0 2024-07-17T08:12:40.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.172+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70200b1f00 con 0x7f70200b01b0 2024-07-17T08:12:40.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.174+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f701001c070 con 0x7f70200b01b0 2024-07-17T08:12:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.175+0000 7f701cc00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 0x7f6ffc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:40.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.175+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 6549+0+0 (secure 0 0 0) 0x7f7010098b00 con 0x7f70200b01b0 2024-07-17T08:12:40.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.175+0000 7f701fe00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 0x7f6ffc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:40.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.177+0000 7f701fe00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 0x7f6ffc079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7008033000 tx=0x7f7008004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:40.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.179+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f70100616a0 con 0x7f70200b01b0 2024-07-17T08:12:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.435+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f702014b400 con 0x7f70200b01b0 2024-07-17T08:12:40.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.941+0000 7f701cc00640 1 -- 172.21.15.44:0/408901263 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v84) ==== 174+0+0 (secure 0 0 0) 0x7f7010068110 con 0x7f70200b01b0 2024-07-17T08:12:40.942 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 21 and data pool 22 2024-07-17T08:12:40.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.946+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 msgr2=0x7f6ffc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:40.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.946+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 0x7f6ffc079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7008033000 tx=0x7f7008004730 comp rx=0 tx=0).stop 2024-07-17T08:12:40.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.946+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 msgr2=0x7f702014cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:40.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.946+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f701000ba50 tx=0x7f701000bf10 comp rx=0 tx=0).stop 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 shutdown_connections 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6ffc076bb0 0x7f6ffc079070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f702015b5d0 0x7f70200b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70200b0fe0 0x7f70201571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 --2- 172.21.15.44:0/408901263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70200b01b0 0x7f702014cea0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:40.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.947+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 >> 172.21.15.44:0/408901263 conn(0x7f7020073d50 msgr2=0x7f70200b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:40.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.948+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 shutdown_connections 2024-07-17T08:12:40.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:40.949+0000 7f7027c00640 1 -- 172.21.15.44:0/408901263 wait complete. 2024-07-17T08:12:40.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:41.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.167+0000 7f7d03c00640 1 -- 172.21.15.44:0/3543525820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ce0004050 msgr2=0x7f7ce0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.167+0000 7f7d03c00640 1 --2- 172.21.15.44:0/3543525820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ce0004050 0x7f7ce0004800 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b4920 tx=0x7f7ce0005830 comp rx=0 tx=0).stop 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.167+0000 7f7d01c00640 1 -- 172.21.15.44:0/3543525820 reap_dead start 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.167+0000 7f7d03c00640 1 -- 172.21.15.44:0/3543525820 shutdown_connections 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.167+0000 7f7d03c00640 1 -- 172.21.15.44:0/3543525820 >> 172.21.15.44:0/3543525820 conn(0x7f7cfc0b0fb0 msgr2=0x7f7cfc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.168+0000 7f7d03c00640 1 -- 172.21.15.44:0/3543525820 shutdown_connections 2024-07-17T08:12:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.168+0000 7f7d03c00640 1 -- 172.21.15.44:0/3543525820 wait complete. 2024-07-17T08:12:41.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.170+0000 7f7d03c00640 1 Processor -- start 2024-07-17T08:12:41.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.170+0000 7f7d03c00640 1 -- start start 2024-07-17T08:12:41.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ce0004050 0x7f7cfc1e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc151700 0x7f7cfc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc0ac650 con 0x7f7cfc151b60 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc15cfd0 con 0x7f7ce0004050 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d03c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc0ac7d0 con 0x7f7cfc151700 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d01c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d01c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48492/0 (socket says 172.21.15.44:48492) 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d01c00640 1 -- 172.21.15.44:0/2897051666 learned_addr learned my addr 172.21.15.44:0/2897051666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7d01200640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ce0004050 0x7f7cfc1e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.171+0000 7f7cfbe00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc151700 0x7f7cfc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 -- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc151700 msgr2=0x7f7cfc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc151700 0x7f7cfc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 -- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ce0004050 msgr2=0x7f7cfc1e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ce0004050 0x7f7cfc1e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 -- 172.21.15.44:0/2897051666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0b3420 con 0x7f7cfc151b60 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d01c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15dcf0 tx=0x7f7cf400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400f040 con 0x7f7cfc151b60 2024-07-17T08:12:41.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf4018070 con 0x7f7cfc151b60 2024-07-17T08:12:41.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40138b0 con 0x7f7cfc151b60 2024-07-17T08:12:41.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc071a50 con 0x7f7cfc151b60 2024-07-17T08:12:41.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.172+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc0b30b0 con 0x7f7cfc151b60 2024-07-17T08:12:41.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.174+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc153e40 con 0x7f7cfc151b60 2024-07-17T08:12:41.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.175+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f7cf4013a50 con 0x7f7cfc151b60 2024-07-17T08:12:41.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.176+0000 7f7cf8c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 0x7f7cd8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.176+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6649+0+0 (secure 0 0 0) 0x7f7cf4098180 con 0x7f7cfc151b60 2024-07-17T08:12:41.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.176+0000 7f7d01200640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 0x7f7cd8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.177+0000 7f7d01200640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 0x7f7cd8079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009aa0 tx=0x7f7cec005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:41.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.180+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7cf4061210 con 0x7f7cfc151b60 2024-07-17T08:12:41.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.383+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cfc06f070 con 0x7f7cfc151b60 2024-07-17T08:12:41.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.386+0000 7f7cf8c00640 1 -- 172.21.15.44:0/2897051666 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+17724 (secure 0 0 0) 0x7f7cf4060b10 con 0x7f7cfc151b60 2024-07-17T08:12:41.388 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:41.388 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":173,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:40.436722+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:12:38.365198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-07-17T08:12:39.424163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 msgr2=0x7f7cd8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 0x7f7cd8079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009aa0 tx=0x7f7cec005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 msgr2=0x7f7cfc06e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15dcf0 tx=0x7f7cf400e9f0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 shutdown_connections 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7cd8076bb0 0x7f7cd8079070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cfc151b60 0x7f7cfc06e450 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cfc151700 0x7f7cfc06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 --2- 172.21.15.44:0/2897051666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ce0004050 0x7f7cfc1e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 >> 172.21.15.44:0/2897051666 conn(0x7f7cfc0b0fb0 msgr2=0x7f7cfc0aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:41.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 shutdown_connections 2024-07-17T08:12:41.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.389+0000 7f7d03c00640 1 -- 172.21.15.44:0/2897051666 wait complete. 2024-07-17T08:12:41.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 -- 172.21.15.44:0/131033196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f337809bee0 msgr2=0x7f33780b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 --2- 172.21.15.44:0/131033196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f337809bee0 0x7f33780b31b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f337806dc90 tx=0x7f337002a560 comp rx=0 tx=0).stop 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 -- 172.21.15.44:0/131033196 shutdown_connections 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 --2- 172.21.15.44:0/131033196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33780cf8e0 0x7f337809b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 --2- 172.21.15.44:0/131033196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f337809bee0 0x7f33780b31b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 --2- 172.21.15.44:0/131033196 >> v2:172.21.15.175:3300/0 conn(0x7f3378157ba0 0x7f33780cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 -- 172.21.15.44:0/131033196 >> 172.21.15.44:0/131033196 conn(0x7f33780b0e80 msgr2=0x7f33780b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 -- 172.21.15.44:0/131033196 shutdown_connections 2024-07-17T08:12:41.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.606+0000 7f3377e00640 1 -- 172.21.15.44:0/131033196 wait complete. 2024-07-17T08:12:41.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.607+0000 7f3377e00640 1 Processor -- start 2024-07-17T08:12:41.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 -- start start 2024-07-17T08:12:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f337809bee0 0x7f33781f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 0x7f33781f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33780ac930 con 0x7f33780cf8e0 2024-07-17T08:12:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33780ac7b0 con 0x7f337809bee0 2024-07-17T08:12:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.608+0000 7f3377e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33780b36f0 con 0x7f3378157ba0 2024-07-17T08:12:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48504/0 (socket says 172.21.15.44:48504) 2024-07-17T08:12:41.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 -- 172.21.15.44:0/1654881607 learned_addr learned my addr 172.21.15.44:0/1654881607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f3377400640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 0x7f33781f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f3376a00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f337809bee0 0x7f33781f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 -- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 msgr2=0x7f33781f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 0x7f33781f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 -- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f337809bee0 msgr2=0x7f33781f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f337809bee0 0x7f33781f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f336e000640 1 -- 172.21.15.44:0/1654881607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33781f1660 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.609+0000 7f3377400640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 0x7f33781f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f336e000640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f336000ea80 tx=0x7f336000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000ce40 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3360004c10 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360005020 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33781f56d0 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.610+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781f1920 con 0x7f33780cf8e0 2024-07-17T08:12:41.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.611+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337809b9a0 con 0x7f33780cf8e0 2024-07-17T08:12:41.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.612+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f33600051c0 con 0x7f33780cf8e0 2024-07-17T08:12:41.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.613+0000 7f336f400640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 0x7f3354079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:41.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.613+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6649+0+0 (secure 0 0 0) 0x7f3360099090 con 0x7f33780cf8e0 2024-07-17T08:12:41.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.614+0000 7f3376a00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 0x7f3354079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:41.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.615+0000 7f3376a00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 0x7f3354079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3368009960 tx=0x7f3368005c90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:41.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.617+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3360016030 con 0x7f33780cf8e0 2024-07-17T08:12:41.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.831+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33780b15e0 con 0x7f33780cf8e0 2024-07-17T08:12:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.834+0000 7f336f400640 1 -- 172.21.15.44:0/1654881607 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+17724 (secure 0 0 0) 0x7f3360062110 con 0x7f33780cf8e0 2024-07-17T08:12:41.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:41.835 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":173,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:40.436722+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:12:38.365198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-07-17T08:12:39.424163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:41.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 msgr2=0x7f3354079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 0x7f3354079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3368009960 tx=0x7f3368005c90 comp rx=0 tx=0).stop 2024-07-17T08:12:41.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 msgr2=0x7f3378073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:41.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f336000ea80 tx=0x7f336000ef40 comp rx=0 tx=0).stop 2024-07-17T08:12:41.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 shutdown_connections 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3354076bb0 0x7f3354079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3378157ba0 0x7f33781f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33780cf8e0 0x7f3378073bc0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 --2- 172.21.15.44:0/1654881607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f337809bee0 0x7f33781f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.837+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 >> 172.21.15.44:0/1654881607 conn(0x7f33780b0e80 msgr2=0x7f33780adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.838+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 shutdown_connections 2024-07-17T08:12:41.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:41.838+0000 7f3377e00640 1 -- 172.21.15.44:0/1654881607 wait complete. 2024-07-17T08:12:41.857 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 -- 172.21.15.44:0/2863838212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328161120 msgr2=0x7f332806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 --2- 172.21.15.44:0/2863838212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328161120 0x7f332806d620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3314009900 tx=0x7f331402a600 comp rx=0 tx=0).stop 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 -- 172.21.15.44:0/2863838212 shutdown_connections 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 --2- 172.21.15.44:0/2863838212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328161120 0x7f332806d620 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 --2- 172.21.15.44:0/2863838212 >> v2:172.21.15.175:3300/0 conn(0x7f3328154600 0x7f33280b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 -- 172.21.15.44:0/2863838212 >> 172.21.15.44:0/2863838212 conn(0x7f33280b0eb0 msgr2=0x7f33280b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 -- 172.21.15.44:0/2863838212 shutdown_connections 2024-07-17T08:12:42.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.085+0000 7f332ec00640 1 -- 172.21.15.44:0/2863838212 wait complete. 2024-07-17T08:12:42.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.087+0000 7f332ec00640 1 Processor -- start 2024-07-17T08:12:42.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.087+0000 7f332ec00640 1 -- start start 2024-07-17T08:12:42.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 0x7f332814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:42.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:42.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328159bd0 0x7f3328071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:42.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f332815e220 con 0x7f3328161120 2024-07-17T08:12:42.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f332815e0a0 con 0x7f3328154600 2024-07-17T08:12:42.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.088+0000 7f332ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f332815e3a0 con 0x7f3328159bd0 2024-07-17T08:12:42.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:42.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48520/0 (socket says 172.21.15.44:48520) 2024-07-17T08:12:42.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 -- 172.21.15.44:0/3491576294 learned_addr learned my addr 172.21.15.44:0/3491576294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f332cc00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328159bd0 0x7f3328071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327e00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 0x7f332814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 -- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328159bd0 msgr2=0x7f3328071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328159bd0 0x7f3328071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 -- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 msgr2=0x7f332814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 0x7f332814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327400640 1 -- 172.21.15.44:0/3491576294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281f4820 con 0x7f3328161120 2024-07-17T08:12:42.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.089+0000 7f3327e00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 0x7f332814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f3327400640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3314009230 tx=0x7f33140038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331403e030 con 0x7f3328161120 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3314003d90 con 0x7f3328161120 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314035040 con 0x7f3328161120 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3328153850 con 0x7f3328161120 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.090+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281f4da0 con 0x7f3328161120 2024-07-17T08:12:42.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.091+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328159fe0 con 0x7f3328161120 2024-07-17T08:12:42.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.093+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f3314041030 con 0x7f3328161120 2024-07-17T08:12:42.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.093+0000 7f331fe00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 0x7f3308079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:42.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.094+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f33140b9080 con 0x7f3328161120 2024-07-17T08:12:42.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.094+0000 7f3327e00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 0x7f3308079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:42.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.096+0000 7f3327e00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 0x7f3308079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3318009b50 tx=0x7f3318005f90 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:42.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.098+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3328159fe0 con 0x7f3328161120 2024-07-17T08:12:42.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.349+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f332815e8c0 con 0x7f3328161120 2024-07-17T08:12:42.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-07-17T08:12:42.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.969+0000 7f331fe00640 1 -- 172.21.15.44:0/3491576294 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) ==== 118+0+0 (secure 0 0 0) 0x7f3314082100 con 0x7f3328161120 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 msgr2=0x7f3308079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 0x7f3308079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3318009b50 tx=0x7f3318005f90 comp rx=0 tx=0).stop 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 msgr2=0x7f3328159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3314009230 tx=0x7f33140038a0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 shutdown_connections 2024-07-17T08:12:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f3308076bb0 0x7f3308079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3328159bd0 0x7f3328071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3328161120 0x7f3328159690 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 --2- 172.21.15.44:0/3491576294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3328154600 0x7f332814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.973+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 >> 172.21.15.44:0/3491576294 conn(0x7f33280b0eb0 msgr2=0x7f332806df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.974+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 shutdown_connections 2024-07-17T08:12:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:42.974+0000 7f332ec00640 1 -- 172.21.15.44:0/3491576294 wait complete. 2024-07-17T08:12:42.996 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:12:42.996 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:12:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.216+0000 7f1e5a800640 1 -- 172.21.15.44:0/3573324709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e3c0040f0 msgr2=0x7f1e3c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.216+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3573324709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e3c0040f0 0x7f1e3c004800 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c290 tx=0x7f1e3c005830 comp rx=0 tx=0).stop 2024-07-17T08:12:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.216+0000 7f1e59e00640 1 -- 172.21.15.44:0/3573324709 reap_dead start 2024-07-17T08:12:43.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.216+0000 7f1e5a800640 1 -- 172.21.15.44:0/3573324709 shutdown_connections 2024-07-17T08:12:43.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.216+0000 7f1e5a800640 1 -- 172.21.15.44:0/3573324709 >> 172.21.15.44:0/3573324709 conn(0x7f1e540b5560 msgr2=0x7f1e540b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:43.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.217+0000 7f1e5a800640 1 -- 172.21.15.44:0/3573324709 shutdown_connections 2024-07-17T08:12:43.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.217+0000 7f1e5a800640 1 -- 172.21.15.44:0/3573324709 wait complete. 2024-07-17T08:12:43.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.218+0000 7f1e5a800640 1 Processor -- start 2024-07-17T08:12:43.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.219+0000 7f1e5a800640 1 -- start start 2024-07-17T08:12:43.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540b1350 0x7f1e5414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:43.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e541f4aa0 0x7f1e54151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5406ead0 con 0x7f1e5414ef60 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5406e950 con 0x7f1e541f4aa0 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e5a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5406ec50 con 0x7f1e540b1350 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e58a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48546/0 (socket says 172.21.15.44:48546) 2024-07-17T08:12:43.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e58a00640 1 -- 172.21.15.44:0/3687956381 learned_addr learned my addr 172.21.15.44:0/3687956381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:43.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e59e00640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e541f4aa0 0x7f1e54151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.220+0000 7f1e59400640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540b1350 0x7f1e5414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 -- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540b1350 msgr2=0x7f1e5414f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540b1350 0x7f1e5414f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 -- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e541f4aa0 msgr2=0x7f1e54151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e541f4aa0 0x7f1e54151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 -- 172.21.15.44:0/3687956381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5414fe70 con 0x7f1e5414ef60 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e58a00640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c380 tx=0x7f1e44001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44044070 con 0x7f1e5414ef60 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e440036d0 con 0x7f1e5414ef60 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44031070 con 0x7f1e5414ef60 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541f3fc0 con 0x7f1e5414ef60 2024-07-17T08:12:43.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.221+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541f4580 con 0x7f1e5414ef60 2024-07-17T08:12:43.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.223+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5415cef0 con 0x7f1e5414ef60 2024-07-17T08:12:43.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.224+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f1e44034070 con 0x7f1e5414ef60 2024-07-17T08:12:43.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.225+0000 7f1e51600640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 0x7f1e380790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:43.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.225+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f1e440b82f0 con 0x7f1e5414ef60 2024-07-17T08:12:43.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.226+0000 7f1e59400640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 0x7f1e380790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:43.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.227+0000 7f1e59400640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 0x7f1e380790a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005ee0 tx=0x7f1e48005df0 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:43.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.229+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e5415cef0 con 0x7f1e5414ef60 2024-07-17T08:12:43.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.491+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1e540b30d0 con 0x7f1e5414ef60 2024-07-17T08:12:43.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.995+0000 7f1e51600640 1 -- 172.21.15.44:0/3687956381 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7f1e44081360 con 0x7f1e5414ef60 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 msgr2=0x7f1e380790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 0x7f1e380790a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e48005ee0 tx=0x7f1e48005df0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 msgr2=0x7f1e54151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c380 tx=0x7f1e44001cb0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 shutdown_connections 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e38076be0 0x7f1e380790a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e541f4aa0 0x7f1e54151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5414ef60 0x7f1e54151640 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 --2- 172.21.15.44:0/3687956381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e540b1350 0x7f1e5414f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 >> 172.21.15.44:0/3687956381 conn(0x7f1e540b5560 msgr2=0x7f1e5415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 shutdown_connections 2024-07-17T08:12:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:43.999+0000 7f1e5a800640 1 -- 172.21.15.44:0/3687956381 wait complete. 2024-07-17T08:12:44.020 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:12:44.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:44.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.228+0000 7fcbd9a00640 1 -- 172.21.15.44:0/3865326741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 msgr2=0x7fcbd40b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.228+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/3865326741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40b4a30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4005fd0 tx=0x7fcbc402a4a0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 -- 172.21.15.44:0/3865326741 shutdown_connections 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/3865326741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40cded0 0x7fcbd40ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/3865326741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40b4a30 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 -- 172.21.15.44:0/3865326741 >> 172.21.15.44:0/3865326741 conn(0x7fcbd408ca20 msgr2=0x7fcbd408ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 -- 172.21.15.44:0/3865326741 shutdown_connections 2024-07-17T08:12:44.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.229+0000 7fcbd9a00640 1 -- 172.21.15.44:0/3865326741 wait complete. 2024-07-17T08:12:44.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.230+0000 7fcbd9a00640 1 Processor -- start 2024-07-17T08:12:44.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.231+0000 7fcbd9a00640 1 -- start start 2024-07-17T08:12:44.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.231+0000 7fcbd9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd40cded0 0x7fcbd415e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40d3800 0x7fcbd41f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40ce8a0 con 0x7fcbd40d3420 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40cea20 con 0x7fcbd40cded0 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd40ceba0 con 0x7fcbd40d3800 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48564/0 (socket says 172.21.15.44:48564) 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 -- 172.21.15.44:0/977360371 learned_addr learned my addr 172.21.15.44:0/977360371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd3e00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd40cded0 0x7fcbd415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbd9000640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40d3800 0x7fcbd41f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 -- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40d3800 msgr2=0x7fcbd41f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40d3800 0x7fcbd41f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 -- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd40cded0 msgr2=0x7fcbd415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd40cded0 0x7fcbd415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.232+0000 7fcbcb400640 1 -- 172.21.15.44:0/977360371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4160d20 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbcb400640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00b3f0 tx=0x7fcbbc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc015020 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbbc011920 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc011d70 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd415ea80 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.233+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd415f040 con 0x7fcbd40d3420 2024-07-17T08:12:44.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.234+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd40ce360 con 0x7fcbd40d3420 2024-07-17T08:12:44.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.236+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fcbbc018020 con 0x7fcbd40d3420 2024-07-17T08:12:44.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.237+0000 7fcbd0c00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 0x7fcbb0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.237+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7fcbbc098440 con 0x7fcbd40d3420 2024-07-17T08:12:44.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.237+0000 7fcbd3e00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 0x7fcbb0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.238+0000 7fcbd3e00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 0x7fcbb0079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4033000 tx=0x7fcbc4004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:44.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.241+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcbbc01f030 con 0x7fcbd40d3420 2024-07-17T08:12:44.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.490+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbd40d4080 con 0x7fcbd40d3420 2024-07-17T08:12:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.491+0000 7fcbd0c00640 1 -- 172.21.15.44:0/977360371 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7fcbbc0614c0 con 0x7fcbd40d3420 2024-07-17T08:12:44.492 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:44.492 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:12:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 msgr2=0x7fcbb0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 0x7fcbb0079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4033000 tx=0x7fcbc4004730 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 msgr2=0x7fcbd40a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00b3f0 tx=0x7fcbbc00b8b0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 shutdown_connections 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fcbb0076bb0 0x7fcbb0079070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcbd40d3800 0x7fcbd41f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbd40d3420 0x7fcbd40a09d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 --2- 172.21.15.44:0/977360371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbd40cded0 0x7fcbd415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 >> 172.21.15.44:0/977360371 conn(0x7fcbd408ca20 msgr2=0x7fcbd406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 shutdown_connections 2024-07-17T08:12:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.494+0000 7fcbd9a00640 1 -- 172.21.15.44:0/977360371 wait complete. 2024-07-17T08:12:44.513 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 -- 172.21.15.44:0/1117024798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f340adcd0 msgr2=0x7f5f340ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/1117024798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f340adcd0 0x7f5f340ac220 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f2002a540 comp rx=0 tx=0).stop 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 -- 172.21.15.44:0/1117024798 shutdown_connections 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/1117024798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f340adcd0 0x7f5f340ac220 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/1117024798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f341522c0 0x7f5f340ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 -- 172.21.15.44:0/1117024798 >> 172.21.15.44:0/1117024798 conn(0x7f5f340b0fb0 msgr2=0x7f5f340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 -- 172.21.15.44:0/1117024798 shutdown_connections 2024-07-17T08:12:44.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.722+0000 7f5f3ae00640 1 -- 172.21.15.44:0/1117024798 wait complete. 2024-07-17T08:12:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.724+0000 7f5f3ae00640 1 Processor -- start 2024-07-17T08:12:44.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.724+0000 7f5f3ae00640 1 -- start start 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340ac760 0x7f5f3414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340adcd0 0x7f5f3409c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415f5a0 con 0x7f5f341522c0 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340ce8b0 con 0x7f5f340adcd0 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f3ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3415f720 con 0x7f5f340ac760 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48578/0 (socket says 172.21.15.44:48578) 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 -- 172.21.15.44:0/270187681 learned_addr learned my addr 172.21.15.44:0/270187681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:44.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f33e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340ac760 0x7f5f3414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f33400640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340adcd0 0x7f5f3409c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 -- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340ac760 msgr2=0x7f5f3414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340ac760 0x7f5f3414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 -- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340adcd0 msgr2=0x7f5f3409c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340adcd0 0x7f5f3409c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.725+0000 7f5f38e00640 1 -- 172.21.15.44:0/270187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341f19e0 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f38e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c780 tx=0x7f5f2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004b10 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f2c004da0 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00aa50 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3409ccc0 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.726+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3414dc60 con 0x7f5f341522c0 2024-07-17T08:12:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.727+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3415fc40 con 0x7f5f341522c0 2024-07-17T08:12:44.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.728+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f5f2c00abf0 con 0x7f5f341522c0 2024-07-17T08:12:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.730+0000 7f5f2be00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 0x7f5f14079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:44.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.730+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f5f2c098950 con 0x7f5f341522c0 2024-07-17T08:12:44.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.731+0000 7f5f33e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 0x7f5f14079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.732+0000 7f5f33e00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 0x7f5f14079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b9a0 tx=0x7f5f24005c20 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:44.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.735+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f2c020070 con 0x7f5f341522c0 2024-07-17T08:12:44.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.946+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f340ad610 con 0x7f5f341522c0 2024-07-17T08:12:44.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.947+0000 7f5f2be00640 1 -- 172.21.15.44:0/270187681 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) ==== 74+0+17711 (secure 0 0 0) 0x7f5f2c061a60 con 0x7f5f341522c0 2024-07-17T08:12:44.948 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:44.949 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":174,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:12:41.941694+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:12:38.365198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-07-17T08:12:39.424163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:12:44.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 msgr2=0x7f5f14079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 0x7f5f14079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b9a0 tx=0x7f5f24005c20 comp rx=0 tx=0).stop 2024-07-17T08:12:44.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 msgr2=0x7f5f3406e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:44.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c780 tx=0x7f5f2c00cc40 comp rx=0 tx=0).stop 2024-07-17T08:12:44.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 shutdown_connections 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f5f14076bb0 0x7f5f14079070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f341522c0 0x7f5f3406e990 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340adcd0 0x7f5f3409c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 --2- 172.21.15.44:0/270187681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340ac760 0x7f5f3414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.950+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 >> 172.21.15.44:0/270187681 conn(0x7f5f340b0fb0 msgr2=0x7f5f340aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.951+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 shutdown_connections 2024-07-17T08:12:44.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:44.951+0000 7f5f3ae00640 1 -- 172.21.15.44:0/270187681 wait complete. 2024-07-17T08:12:44.972 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:12:44.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 -- 172.21.15.44:0/10549534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc31415d600 msgr2=0x7fc31406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 --2- 172.21.15.44:0/10549534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc31415d600 0x7fc31406e480 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc3140d0000 tx=0x7fc30802a680 comp rx=0 tx=0).stop 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 -- 172.21.15.44:0/10549534 shutdown_connections 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 --2- 172.21.15.44:0/10549534 >> v2:172.21.15.175:3300/0 conn(0x7fc3140b5910 0x7fc3140cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 --2- 172.21.15.44:0/10549534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc31415d600 0x7fc31406e480 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 -- 172.21.15.44:0/10549534 >> 172.21.15.44:0/10549534 conn(0x7fc3140d00a0 msgr2=0x7fc3140d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 -- 172.21.15.44:0/10549534 shutdown_connections 2024-07-17T08:12:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.179+0000 7fc31be00640 1 -- 172.21.15.44:0/10549534 wait complete. 2024-07-17T08:12:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.181+0000 7fc31be00640 1 Processor -- start 2024-07-17T08:12:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.181+0000 7fc31be00640 1 -- start start 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.182+0000 7fc31be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3140b2ad0 0x7fc314155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc31be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc31be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 0x7fc31414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc31be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3140ce640 con 0x7fc3140b5910 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc31be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3140ce4c0 con 0x7fc31415d600 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc31be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3140ce7c0 con 0x7fc3140b2ad0 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48600/0 (socket says 172.21.15.44:48600) 2024-07-17T08:12:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 -- 172.21.15.44:0/460008765 learned_addr learned my addr 172.21.15.44:0/460008765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc319400640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3140b2ad0 0x7fc314155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc319e00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 0x7fc31414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 -- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3140b2ad0 msgr2=0x7fc314155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3140b2ad0 0x7fc314155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 -- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 msgr2=0x7fc31414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 0x7fc31414a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc318a00640 1 -- 172.21.15.44:0/460008765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31415b930 con 0x7fc3140b5910 2024-07-17T08:12:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.183+0000 7fc319e00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 0x7fc31414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc318a00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c450 tx=0x7fc30000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3000052d0 con 0x7fc3140b5910 2024-07-17T08:12:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc300005470 con 0x7fc3140b5910 2024-07-17T08:12:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300012870 con 0x7fc3140b5910 2024-07-17T08:12:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31414b070 con 0x7fc3140b5910 2024-07-17T08:12:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.184+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31415bbd0 con 0x7fc3140b5910 2024-07-17T08:12:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.186+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3140cdcf0 con 0x7fc3140b5910 2024-07-17T08:12:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.187+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fc3000035f0 con 0x7fc3140b5910 2024-07-17T08:12:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.188+0000 7fc311600640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 0x7fc2f4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.188+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7fc300098320 con 0x7fc3140b5910 2024-07-17T08:12:45.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.188+0000 7fc319400640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 0x7fc2f4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.190+0000 7fc319400640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 0x7fc2f4079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc308033000 tx=0x7fc308004730 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:45.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.192+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc3000613a0 con 0x7fc3140b5910 2024-07-17T08:12:45.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.400+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 --> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3140b4c90 con 0x7fc2f4076bb0 2024-07-17T08:12:45.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.416+0000 7fc311600640 1 -- 172.21.15.44:0/460008765 <== mgr.6653 v2:172.21.15.175:6834/3617669990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7fc3140b4c90 con 0x7fc2f4076bb0 2024-07-17T08:12:45.421 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:45.423 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.422+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 msgr2=0x7fc2f4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.422+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 0x7fc2f4079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc308033000 tx=0x7fc308004730 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.422+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 msgr2=0x7fc314159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.422+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c450 tx=0x7fc30000c910 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.422+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 shutdown_connections 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc2f4076bb0 0x7fc2f4079070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc31415d600 0x7fc31414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3140b5910 0x7fc314159d30 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 --2- 172.21.15.44:0/460008765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3140b2ad0 0x7fc314155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 >> 172.21.15.44:0/460008765 conn(0x7fc3140d00a0 msgr2=0x7fc31406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:45.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 shutdown_connections 2024-07-17T08:12:45.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.423+0000 7fc31be00640 1 -- 172.21.15.44:0/460008765 wait complete. 2024-07-17T08:12:45.445 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-07-17T08:12:45.399394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":320,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2501,"ondisk_log_size":2501,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":373056,"kb_used_data":18116,"kb_used_omap":12,"kb_used_meta":354803,"kb_avail":754601664,"statfs":{"total":773094113280,"available":772712103936,"internally_reserved":0,"allocated":18550784,"data_stored":10484268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":363318832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002039"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662283+0000","last_change":"2024-07-17T08:12:40.982016+0000","last_active":"2024-07-17T08:12:42.662283+0000","last_peered":"2024-07-17T08:12:42.662283+0000","last_clean":"2024-07-17T08:12:42.662283+0000","last_became_active":"2024-07-17T08:12:40.977962+0000","last_became_peered":"2024-07-17T08:12:40.977962+0000","last_unstale":"2024-07-17T08:12:42.662283+0000","last_undegraded":"2024-07-17T08:12:42.662283+0000","last_fullsized":"2024-07-17T08:12:42.662283+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:29.889194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367925+0000","last_change":"2024-07-17T08:12:40.972918+0000","last_active":"2024-07-17T08:12:43.367925+0000","last_peered":"2024-07-17T08:12:43.367925+0000","last_clean":"2024-07-17T08:12:43.367925+0000","last_became_active":"2024-07-17T08:12:40.972131+0000","last_became_peered":"2024-07-17T08:12:40.972131+0000","last_unstale":"2024-07-17T08:12:43.367925+0000","last_undegraded":"2024-07-17T08:12:43.367925+0000","last_fullsized":"2024-07-17T08:12:43.367925+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:01.340713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368653+0000","last_change":"2024-07-17T08:12:40.984995+0000","last_active":"2024-07-17T08:12:43.368653+0000","last_peered":"2024-07-17T08:12:43.368653+0000","last_clean":"2024-07-17T08:12:43.368653+0000","last_became_active":"2024-07-17T08:12:40.984600+0000","last_became_peered":"2024-07-17T08:12:40.984600+0000","last_unstale":"2024-07-17T08:12:43.368653+0000","last_undegraded":"2024-07-17T08:12:43.368653+0000","last_fullsized":"2024-07-17T08:12:43.368653+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:23:26.303837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955142+0000","last_change":"2024-07-17T08:12:40.973444+0000","last_active":"2024-07-17T08:12:41.955142+0000","last_peered":"2024-07-17T08:12:41.955142+0000","last_clean":"2024-07-17T08:12:41.955142+0000","last_became_active":"2024-07-17T08:12:40.973173+0000","last_became_peered":"2024-07-17T08:12:40.973173+0000","last_unstale":"2024-07-17T08:12:41.955142+0000","last_undegraded":"2024-07-17T08:12:41.955142+0000","last_fullsized":"2024-07-17T08:12:41.955142+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:01.340713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382598+0000","last_change":"2024-07-17T08:12:40.976129+0000","last_active":"2024-07-17T08:12:43.382598+0000","last_peered":"2024-07-17T08:12:43.382598+0000","last_clean":"2024-07-17T08:12:43.382598+0000","last_became_active":"2024-07-17T08:12:40.974654+0000","last_became_peered":"2024-07-17T08:12:40.974654+0000","last_unstale":"2024-07-17T08:12:43.382598+0000","last_undegraded":"2024-07-17T08:12:43.382598+0000","last_fullsized":"2024-07-17T08:12:43.382598+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:39.600897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381752+0000","last_change":"2024-07-17T08:12:40.959361+0000","last_active":"2024-07-17T08:12:43.381752+0000","last_peered":"2024-07-17T08:12:43.381752+0000","last_clean":"2024-07-17T08:12:43.381752+0000","last_became_active":"2024-07-17T08:12:40.958577+0000","last_became_peered":"2024-07-17T08:12:40.958577+0000","last_unstale":"2024-07-17T08:12:43.381752+0000","last_undegraded":"2024-07-17T08:12:43.381752+0000","last_fullsized":"2024-07-17T08:12:43.381752+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:22.940653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661918+0000","last_change":"2024-07-17T08:12:40.964253+0000","last_active":"2024-07-17T08:12:42.661918+0000","last_peered":"2024-07-17T08:12:42.661918+0000","last_clean":"2024-07-17T08:12:42.661918+0000","last_became_active":"2024-07-17T08:12:40.963955+0000","last_became_peered":"2024-07-17T08:12:40.963955+0000","last_unstale":"2024-07-17T08:12:42.661918+0000","last_undegraded":"2024-07-17T08:12:42.661918+0000","last_fullsized":"2024-07-17T08:12:42.661918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:51.660772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951378+0000","last_change":"2024-07-17T08:12:40.978851+0000","last_active":"2024-07-17T08:12:41.951378+0000","last_peered":"2024-07-17T08:12:41.951378+0000","last_clean":"2024-07-17T08:12:41.951378+0000","last_became_active":"2024-07-17T08:12:40.977412+0000","last_became_peered":"2024-07-17T08:12:40.977412+0000","last_unstale":"2024-07-17T08:12:41.951378+0000","last_undegraded":"2024-07-17T08:12:41.951378+0000","last_fullsized":"2024-07-17T08:12:41.951378+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.127724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382909+0000","last_change":"2024-07-17T08:12:40.974678+0000","last_active":"2024-07-17T08:12:43.382909+0000","last_peered":"2024-07-17T08:12:43.382909+0000","last_clean":"2024-07-17T08:12:43.382909+0000","last_became_active":"2024-07-17T08:12:40.974358+0000","last_became_peered":"2024-07-17T08:12:40.974358+0000","last_unstale":"2024-07-17T08:12:43.382909+0000","last_undegraded":"2024-07-17T08:12:43.382909+0000","last_fullsized":"2024-07-17T08:12:43.382909+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:57.343043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382953+0000","last_change":"2024-07-17T08:12:40.970873+0000","last_active":"2024-07-17T08:12:43.382953+0000","last_peered":"2024-07-17T08:12:43.382953+0000","last_clean":"2024-07-17T08:12:43.382953+0000","last_became_active":"2024-07-17T08:12:40.969332+0000","last_became_peered":"2024-07-17T08:12:40.969332+0000","last_unstale":"2024-07-17T08:12:43.382953+0000","last_undegraded":"2024-07-17T08:12:43.382953+0000","last_fullsized":"2024-07-17T08:12:43.382953+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:31:30.695204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.948136+0000","last_change":"2024-07-17T08:12:40.976236+0000","last_active":"2024-07-17T08:12:41.948136+0000","last_peered":"2024-07-17T08:12:41.948136+0000","last_clean":"2024-07-17T08:12:41.948136+0000","last_became_active":"2024-07-17T08:12:40.975956+0000","last_became_peered":"2024-07-17T08:12:40.975956+0000","last_unstale":"2024-07-17T08:12:41.948136+0000","last_undegraded":"2024-07-17T08:12:41.948136+0000","last_fullsized":"2024-07-17T08:12:41.948136+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:16.879818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954538+0000","last_change":"2024-07-17T08:12:40.973318+0000","last_active":"2024-07-17T08:12:41.954538+0000","last_peered":"2024-07-17T08:12:41.954538+0000","last_clean":"2024-07-17T08:12:41.954538+0000","last_became_active":"2024-07-17T08:12:40.972876+0000","last_became_peered":"2024-07-17T08:12:40.972876+0000","last_unstale":"2024-07-17T08:12:41.954538+0000","last_undegraded":"2024-07-17T08:12:41.954538+0000","last_fullsized":"2024-07-17T08:12:41.954538+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:26.695183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382036+0000","last_change":"2024-07-17T08:12:40.967437+0000","last_active":"2024-07-17T08:12:43.382036+0000","last_peered":"2024-07-17T08:12:43.382036+0000","last_clean":"2024-07-17T08:12:43.382036+0000","last_became_active":"2024-07-17T08:12:40.965913+0000","last_became_peered":"2024-07-17T08:12:40.965913+0000","last_unstale":"2024-07-17T08:12:43.382036+0000","last_undegraded":"2024-07-17T08:12:43.382036+0000","last_fullsized":"2024-07-17T08:12:43.382036+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:58.867645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.953442+0000","last_change":"2024-07-17T08:12:40.970072+0000","last_active":"2024-07-17T08:12:41.953442+0000","last_peered":"2024-07-17T08:12:41.953442+0000","last_clean":"2024-07-17T08:12:41.953442+0000","last_became_active":"2024-07-17T08:12:40.969743+0000","last_became_peered":"2024-07-17T08:12:40.969743+0000","last_unstale":"2024-07-17T08:12:41.953442+0000","last_undegraded":"2024-07-17T08:12:41.953442+0000","last_fullsized":"2024-07-17T08:12:41.953442+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.738801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367454+0000","last_change":"2024-07-17T08:12:40.986760+0000","last_active":"2024-07-17T08:12:43.367454+0000","last_peered":"2024-07-17T08:12:43.367454+0000","last_clean":"2024-07-17T08:12:43.367454+0000","last_became_active":"2024-07-17T08:12:40.986278+0000","last_became_peered":"2024-07-17T08:12:40.986278+0000","last_unstale":"2024-07-17T08:12:43.367454+0000","last_undegraded":"2024-07-17T08:12:43.367454+0000","last_fullsized":"2024-07-17T08:12:43.367454+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.738801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381576+0000","last_change":"2024-07-17T08:12:40.973444+0000","last_active":"2024-07-17T08:12:43.381576+0000","last_peered":"2024-07-17T08:12:43.381576+0000","last_clean":"2024-07-17T08:12:43.381576+0000","last_became_active":"2024-07-17T08:12:40.973117+0000","last_became_peered":"2024-07-17T08:12:40.973117+0000","last_unstale":"2024-07-17T08:12:43.381576+0000","last_undegraded":"2024-07-17T08:12:43.381576+0000","last_fullsized":"2024-07-17T08:12:43.381576+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:14.536558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955273+0000","last_change":"2024-07-17T08:12:40.976137+0000","last_active":"2024-07-17T08:12:41.955273+0000","last_peered":"2024-07-17T08:12:41.955273+0000","last_clean":"2024-07-17T08:12:41.955273+0000","last_became_active":"2024-07-17T08:12:40.975847+0000","last_became_peered":"2024-07-17T08:12:40.975847+0000","last_unstale":"2024-07-17T08:12:41.955273+0000","last_undegraded":"2024-07-17T08:12:41.955273+0000","last_fullsized":"2024-07-17T08:12:41.955273+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:57.189144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947966+0000","last_change":"2024-07-17T08:12:40.971971+0000","last_active":"2024-07-17T08:12:41.947966+0000","last_peered":"2024-07-17T08:12:41.947966+0000","last_clean":"2024-07-17T08:12:41.947966+0000","last_became_active":"2024-07-17T08:12:40.971676+0000","last_became_peered":"2024-07-17T08:12:40.971676+0000","last_unstale":"2024-07-17T08:12:41.947966+0000","last_undegraded":"2024-07-17T08:12:41.947966+0000","last_fullsized":"2024-07-17T08:12:41.947966+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:39.256096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063119+0000","last_change":"2024-07-17T08:12:40.992984+0000","last_active":"2024-07-17T08:12:42.063119+0000","last_peered":"2024-07-17T08:12:42.063119+0000","last_clean":"2024-07-17T08:12:42.063119+0000","last_became_active":"2024-07-17T08:12:40.992422+0000","last_became_peered":"2024-07-17T08:12:40.992422+0000","last_unstale":"2024-07-17T08:12:42.063119+0000","last_undegraded":"2024-07-17T08:12:42.063119+0000","last_fullsized":"2024-07-17T08:12:42.063119+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:38:06.900480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875848+0000","last_change":"2024-07-17T08:12:40.969842+0000","last_active":"2024-07-17T08:12:42.875848+0000","last_peered":"2024-07-17T08:12:42.875848+0000","last_clean":"2024-07-17T08:12:42.875848+0000","last_became_active":"2024-07-17T08:12:40.969541+0000","last_became_peered":"2024-07-17T08:12:40.969541+0000","last_unstale":"2024-07-17T08:12:42.875848+0000","last_undegraded":"2024-07-17T08:12:42.875848+0000","last_fullsized":"2024-07-17T08:12:42.875848+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:14.073148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382307+0000","last_change":"2024-07-17T08:12:40.963212+0000","last_active":"2024-07-17T08:12:43.382307+0000","last_peered":"2024-07-17T08:12:43.382307+0000","last_clean":"2024-07-17T08:12:43.382307+0000","last_became_active":"2024-07-17T08:12:40.962844+0000","last_became_peered":"2024-07-17T08:12:40.962844+0000","last_unstale":"2024-07-17T08:12:43.382307+0000","last_undegraded":"2024-07-17T08:12:43.382307+0000","last_fullsized":"2024-07-17T08:12:43.382307+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:27.861293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661853+0000","last_change":"2024-07-17T08:12:40.963785+0000","last_active":"2024-07-17T08:12:42.661853+0000","last_peered":"2024-07-17T08:12:42.661853+0000","last_clean":"2024-07-17T08:12:42.661853+0000","last_became_active":"2024-07-17T08:12:40.963463+0000","last_became_peered":"2024-07-17T08:12:40.963463+0000","last_unstale":"2024-07-17T08:12:42.661853+0000","last_undegraded":"2024-07-17T08:12:42.661853+0000","last_fullsized":"2024-07-17T08:12:42.661853+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:19.022860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951862+0000","last_change":"2024-07-17T08:12:40.982334+0000","last_active":"2024-07-17T08:12:41.951862+0000","last_peered":"2024-07-17T08:12:41.951862+0000","last_clean":"2024-07-17T08:12:41.951862+0000","last_became_active":"2024-07-17T08:12:40.982055+0000","last_became_peered":"2024-07-17T08:12:40.982055+0000","last_unstale":"2024-07-17T08:12:41.951862+0000","last_undegraded":"2024-07-17T08:12:41.951862+0000","last_fullsized":"2024-07-17T08:12:41.951862+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:01.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947580+0000","last_change":"2024-07-17T08:12:40.956802+0000","last_active":"2024-07-17T08:12:41.947580+0000","last_peered":"2024-07-17T08:12:41.947580+0000","last_clean":"2024-07-17T08:12:41.947580+0000","last_became_active":"2024-07-17T08:12:40.956418+0000","last_became_peered":"2024-07-17T08:12:40.956418+0000","last_unstale":"2024-07-17T08:12:41.947580+0000","last_undegraded":"2024-07-17T08:12:41.947580+0000","last_fullsized":"2024-07-17T08:12:41.947580+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:24.391300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662483+0000","last_change":"2024-07-17T08:12:40.968684+0000","last_active":"2024-07-17T08:12:42.662483+0000","last_peered":"2024-07-17T08:12:42.662483+0000","last_clean":"2024-07-17T08:12:42.662483+0000","last_became_active":"2024-07-17T08:12:40.968348+0000","last_became_peered":"2024-07-17T08:12:40.968348+0000","last_unstale":"2024-07-17T08:12:42.662483+0000","last_undegraded":"2024-07-17T08:12:42.662483+0000","last_fullsized":"2024-07-17T08:12:42.662483+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:31.015455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662206+0000","last_change":"2024-07-17T08:12:40.968483+0000","last_active":"2024-07-17T08:12:42.662206+0000","last_peered":"2024-07-17T08:12:42.662206+0000","last_clean":"2024-07-17T08:12:42.662206+0000","last_became_active":"2024-07-17T08:12:40.968018+0000","last_became_peered":"2024-07-17T08:12:40.968018+0000","last_unstale":"2024-07-17T08:12:42.662206+0000","last_undegraded":"2024-07-17T08:12:42.662206+0000","last_fullsized":"2024-07-17T08:12:42.662206+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:24:51.007876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382402+0000","last_change":"2024-07-17T08:12:40.981064+0000","last_active":"2024-07-17T08:12:43.382402+0000","last_peered":"2024-07-17T08:12:43.382402+0000","last_clean":"2024-07-17T08:12:43.382402+0000","last_became_active":"2024-07-17T08:12:40.977277+0000","last_became_peered":"2024-07-17T08:12:40.977277+0000","last_unstale":"2024-07-17T08:12:43.382402+0000","last_undegraded":"2024-07-17T08:12:43.382402+0000","last_fullsized":"2024-07-17T08:12:43.382402+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:46.256356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947664+0000","last_change":"2024-07-17T08:12:40.962461+0000","last_active":"2024-07-17T08:12:41.947664+0000","last_peered":"2024-07-17T08:12:41.947664+0000","last_clean":"2024-07-17T08:12:41.947664+0000","last_became_active":"2024-07-17T08:12:40.962135+0000","last_became_peered":"2024-07-17T08:12:40.962135+0000","last_unstale":"2024-07-17T08:12:41.947664+0000","last_undegraded":"2024-07-17T08:12:41.947664+0000","last_fullsized":"2024-07-17T08:12:41.947664+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:25.201683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954370+0000","last_change":"2024-07-17T08:12:40.980549+0000","last_active":"2024-07-17T08:12:41.954370+0000","last_peered":"2024-07-17T08:12:41.954370+0000","last_clean":"2024-07-17T08:12:41.954370+0000","last_became_active":"2024-07-17T08:12:40.980275+0000","last_became_peered":"2024-07-17T08:12:40.980275+0000","last_unstale":"2024-07-17T08:12:41.954370+0000","last_undegraded":"2024-07-17T08:12:41.954370+0000","last_fullsized":"2024-07-17T08:12:41.954370+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:58.143265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947582+0000","last_change":"2024-07-17T08:12:40.972560+0000","last_active":"2024-07-17T08:12:41.947582+0000","last_peered":"2024-07-17T08:12:41.947582+0000","last_clean":"2024-07-17T08:12:41.947582+0000","last_became_active":"2024-07-17T08:12:40.972264+0000","last_became_peered":"2024-07-17T08:12:40.972264+0000","last_unstale":"2024-07-17T08:12:41.947582+0000","last_undegraded":"2024-07-17T08:12:41.947582+0000","last_fullsized":"2024-07-17T08:12:41.947582+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:00.095637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381659+0000","last_change":"2024-07-17T08:12:40.977885+0000","last_active":"2024-07-17T08:12:43.381659+0000","last_peered":"2024-07-17T08:12:43.381659+0000","last_clean":"2024-07-17T08:12:43.381659+0000","last_became_active":"2024-07-17T08:12:40.977585+0000","last_became_peered":"2024-07-17T08:12:40.977585+0000","last_unstale":"2024-07-17T08:12:43.381659+0000","last_undegraded":"2024-07-17T08:12:43.381659+0000","last_fullsized":"2024-07-17T08:12:43.381659+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:43.367116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951899+0000","last_change":"2024-07-17T08:12:40.976239+0000","last_active":"2024-07-17T08:12:41.951899+0000","last_peered":"2024-07-17T08:12:41.951899+0000","last_clean":"2024-07-17T08:12:41.951899+0000","last_became_active":"2024-07-17T08:12:40.972980+0000","last_became_peered":"2024-07-17T08:12:40.972980+0000","last_unstale":"2024-07-17T08:12:41.951899+0000","last_undegraded":"2024-07-17T08:12:41.951899+0000","last_fullsized":"2024-07-17T08:12:41.951899+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:04.654259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368101+0000","last_change":"2024-07-17T08:12:40.978280+0000","last_active":"2024-07-17T08:12:43.368101+0000","last_peered":"2024-07-17T08:12:43.368101+0000","last_clean":"2024-07-17T08:12:43.368101+0000","last_became_active":"2024-07-17T08:12:40.976608+0000","last_became_peered":"2024-07-17T08:12:40.976608+0000","last_unstale":"2024-07-17T08:12:43.368101+0000","last_undegraded":"2024-07-17T08:12:43.368101+0000","last_fullsized":"2024-07-17T08:12:43.368101+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:44:24.265138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"173'4","reported_seq":18,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.383077+0000","last_change":"2024-07-17T08:12:39.968428+0000","last_active":"2024-07-17T08:12:43.383077+0000","last_peered":"2024-07-17T08:12:43.383077+0000","last_clean":"2024-07-17T08:12:43.383077+0000","last_became_active":"2024-07-17T08:12:39.967776+0000","last_became_peered":"2024-07-17T08:12:39.967776+0000","last_unstale":"2024-07-17T08:12:43.383077+0000","last_undegraded":"2024-07-17T08:12:43.383077+0000","last_fullsized":"2024-07-17T08:12:43.383077+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:48.246489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368706+0000","last_change":"2024-07-17T08:12:40.987069+0000","last_active":"2024-07-17T08:12:43.368706+0000","last_peered":"2024-07-17T08:12:43.368706+0000","last_clean":"2024-07-17T08:12:43.368706+0000","last_became_active":"2024-07-17T08:12:40.986749+0000","last_became_peered":"2024-07-17T08:12:40.986749+0000","last_unstale":"2024-07-17T08:12:43.368706+0000","last_undegraded":"2024-07-17T08:12:43.368706+0000","last_fullsized":"2024-07-17T08:12:43.368706+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:17.704894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062807+0000","last_change":"2024-07-17T08:12:39.980229+0000","last_active":"2024-07-17T08:12:42.062807+0000","last_peered":"2024-07-17T08:12:42.062807+0000","last_clean":"2024-07-17T08:12:42.062807+0000","last_became_active":"2024-07-17T08:12:39.979929+0000","last_became_peered":"2024-07-17T08:12:39.979929+0000","last_unstale":"2024-07-17T08:12:42.062807+0000","last_undegraded":"2024-07-17T08:12:42.062807+0000","last_fullsized":"2024-07-17T08:12:42.062807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:44.198820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955804+0000","last_change":"2024-07-17T08:12:40.989072+0000","last_active":"2024-07-17T08:12:41.955804+0000","last_peered":"2024-07-17T08:12:41.955804+0000","last_clean":"2024-07-17T08:12:41.955804+0000","last_became_active":"2024-07-17T08:12:40.988783+0000","last_became_peered":"2024-07-17T08:12:40.988783+0000","last_unstale":"2024-07-17T08:12:41.955804+0000","last_undegraded":"2024-07-17T08:12:41.955804+0000","last_fullsized":"2024-07-17T08:12:41.955804+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:36.859840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063215+0000","last_change":"2024-07-17T08:12:39.981106+0000","last_active":"2024-07-17T08:12:42.063215+0000","last_peered":"2024-07-17T08:12:42.063215+0000","last_clean":"2024-07-17T08:12:42.063215+0000","last_became_active":"2024-07-17T08:12:39.980786+0000","last_became_peered":"2024-07-17T08:12:39.980786+0000","last_unstale":"2024-07-17T08:12:42.063215+0000","last_undegraded":"2024-07-17T08:12:42.063215+0000","last_fullsized":"2024-07-17T08:12:42.063215+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:30.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063195+0000","last_change":"2024-07-17T08:12:40.991340+0000","last_active":"2024-07-17T08:12:42.063195+0000","last_peered":"2024-07-17T08:12:42.063195+0000","last_clean":"2024-07-17T08:12:42.063195+0000","last_became_active":"2024-07-17T08:12:40.991067+0000","last_became_peered":"2024-07-17T08:12:40.991067+0000","last_unstale":"2024-07-17T08:12:42.063195+0000","last_undegraded":"2024-07-17T08:12:42.063195+0000","last_fullsized":"2024-07-17T08:12:42.063195+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:54.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368399+0000","last_change":"2024-07-17T08:12:39.976857+0000","last_active":"2024-07-17T08:12:43.368399+0000","last_peered":"2024-07-17T08:12:43.368399+0000","last_clean":"2024-07-17T08:12:43.368399+0000","last_became_active":"2024-07-17T08:12:39.976522+0000","last_became_peered":"2024-07-17T08:12:39.976522+0000","last_unstale":"2024-07-17T08:12:43.368399+0000","last_undegraded":"2024-07-17T08:12:43.368399+0000","last_fullsized":"2024-07-17T08:12:43.368399+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:34.216344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875835+0000","last_change":"2024-07-17T08:12:40.969061+0000","last_active":"2024-07-17T08:12:42.875835+0000","last_peered":"2024-07-17T08:12:42.875835+0000","last_clean":"2024-07-17T08:12:42.875835+0000","last_became_active":"2024-07-17T08:12:40.966503+0000","last_became_peered":"2024-07-17T08:12:40.966503+0000","last_unstale":"2024-07-17T08:12:42.875835+0000","last_undegraded":"2024-07-17T08:12:42.875835+0000","last_fullsized":"2024-07-17T08:12:42.875835+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:18.607103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954392+0000","last_change":"2024-07-17T08:12:39.978775+0000","last_active":"2024-07-17T08:12:41.954392+0000","last_peered":"2024-07-17T08:12:41.954392+0000","last_clean":"2024-07-17T08:12:41.954392+0000","last_became_active":"2024-07-17T08:12:39.978325+0000","last_became_peered":"2024-07-17T08:12:39.978325+0000","last_unstale":"2024-07-17T08:12:41.954392+0000","last_undegraded":"2024-07-17T08:12:41.954392+0000","last_fullsized":"2024-07-17T08:12:41.954392+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:43.478766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952617+0000","last_change":"2024-07-17T08:12:40.976416+0000","last_active":"2024-07-17T08:12:41.952617+0000","last_peered":"2024-07-17T08:12:41.952617+0000","last_clean":"2024-07-17T08:12:41.952617+0000","last_became_active":"2024-07-17T08:12:40.975684+0000","last_became_peered":"2024-07-17T08:12:40.975684+0000","last_unstale":"2024-07-17T08:12:41.952617+0000","last_undegraded":"2024-07-17T08:12:41.952617+0000","last_fullsized":"2024-07-17T08:12:41.952617+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:13.812536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382066+0000","last_change":"2024-07-17T08:12:39.969801+0000","last_active":"2024-07-17T08:12:43.382066+0000","last_peered":"2024-07-17T08:12:43.382066+0000","last_clean":"2024-07-17T08:12:43.382066+0000","last_became_active":"2024-07-17T08:12:39.969439+0000","last_became_peered":"2024-07-17T08:12:39.969439+0000","last_unstale":"2024-07-17T08:12:43.382066+0000","last_undegraded":"2024-07-17T08:12:43.382066+0000","last_fullsized":"2024-07-17T08:12:43.382066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:33.780901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368370+0000","last_change":"2024-07-17T08:12:40.972273+0000","last_active":"2024-07-17T08:12:43.368370+0000","last_peered":"2024-07-17T08:12:43.368370+0000","last_clean":"2024-07-17T08:12:43.368370+0000","last_became_active":"2024-07-17T08:12:40.971994+0000","last_became_peered":"2024-07-17T08:12:40.971994+0000","last_unstale":"2024-07-17T08:12:43.368370+0000","last_undegraded":"2024-07-17T08:12:43.368370+0000","last_fullsized":"2024-07-17T08:12:43.368370+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:14.066482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661741+0000","last_change":"2024-07-17T08:12:39.966501+0000","last_active":"2024-07-17T08:12:42.661741+0000","last_peered":"2024-07-17T08:12:42.661741+0000","last_clean":"2024-07-17T08:12:42.661741+0000","last_became_active":"2024-07-17T08:12:39.966175+0000","last_became_peered":"2024-07-17T08:12:39.966175+0000","last_unstale":"2024-07-17T08:12:42.661741+0000","last_undegraded":"2024-07-17T08:12:42.661741+0000","last_fullsized":"2024-07-17T08:12:42.661741+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:13.060339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661857+0000","last_change":"2024-07-17T08:12:40.979752+0000","last_active":"2024-07-17T08:12:42.661857+0000","last_peered":"2024-07-17T08:12:42.661857+0000","last_clean":"2024-07-17T08:12:42.661857+0000","last_became_active":"2024-07-17T08:12:40.976470+0000","last_became_peered":"2024-07-17T08:12:40.976470+0000","last_unstale":"2024-07-17T08:12:42.661857+0000","last_undegraded":"2024-07-17T08:12:42.661857+0000","last_fullsized":"2024-07-17T08:12:42.661857+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:17.220124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381318+0000","last_change":"2024-07-17T08:12:39.959993+0000","last_active":"2024-07-17T08:12:43.381318+0000","last_peered":"2024-07-17T08:12:43.381318+0000","last_clean":"2024-07-17T08:12:43.381318+0000","last_became_active":"2024-07-17T08:12:39.959636+0000","last_became_peered":"2024-07-17T08:12:39.959636+0000","last_unstale":"2024-07-17T08:12:43.381318+0000","last_undegraded":"2024-07-17T08:12:43.381318+0000","last_fullsized":"2024-07-17T08:12:43.381318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:49.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954544+0000","last_change":"2024-07-17T08:12:40.990145+0000","last_active":"2024-07-17T08:12:41.954544+0000","last_peered":"2024-07-17T08:12:41.954544+0000","last_clean":"2024-07-17T08:12:41.954544+0000","last_became_active":"2024-07-17T08:12:40.989839+0000","last_became_peered":"2024-07-17T08:12:40.989839+0000","last_unstale":"2024-07-17T08:12:41.954544+0000","last_undegraded":"2024-07-17T08:12:41.954544+0000","last_fullsized":"2024-07-17T08:12:41.954544+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:23.033075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063183+0000","last_change":"2024-07-17T08:12:39.979609+0000","last_active":"2024-07-17T08:12:42.063183+0000","last_peered":"2024-07-17T08:12:42.063183+0000","last_clean":"2024-07-17T08:12:42.063183+0000","last_became_active":"2024-07-17T08:12:39.979336+0000","last_became_peered":"2024-07-17T08:12:39.979336+0000","last_unstale":"2024-07-17T08:12:42.063183+0000","last_undegraded":"2024-07-17T08:12:42.063183+0000","last_fullsized":"2024-07-17T08:12:42.063183+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:17.605220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947963+0000","last_change":"2024-07-17T08:12:40.973490+0000","last_active":"2024-07-17T08:12:41.947963+0000","last_peered":"2024-07-17T08:12:41.947963+0000","last_clean":"2024-07-17T08:12:41.947963+0000","last_became_active":"2024-07-17T08:12:40.973222+0000","last_became_peered":"2024-07-17T08:12:40.973222+0000","last_unstale":"2024-07-17T08:12:41.947963+0000","last_undegraded":"2024-07-17T08:12:41.947963+0000","last_fullsized":"2024-07-17T08:12:41.947963+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:38.341914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662792+0000","last_change":"2024-07-17T08:12:39.971697+0000","last_active":"2024-07-17T08:12:42.662792+0000","last_peered":"2024-07-17T08:12:42.662792+0000","last_clean":"2024-07-17T08:12:42.662792+0000","last_became_active":"2024-07-17T08:12:39.971061+0000","last_became_peered":"2024-07-17T08:12:39.971061+0000","last_unstale":"2024-07-17T08:12:42.662792+0000","last_undegraded":"2024-07-17T08:12:42.662792+0000","last_fullsized":"2024-07-17T08:12:42.662792+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:17:15.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947920+0000","last_change":"2024-07-17T08:12:40.971297+0000","last_active":"2024-07-17T08:12:41.947920+0000","last_peered":"2024-07-17T08:12:41.947920+0000","last_clean":"2024-07-17T08:12:41.947920+0000","last_became_active":"2024-07-17T08:12:40.969652+0000","last_became_peered":"2024-07-17T08:12:40.969652+0000","last_unstale":"2024-07-17T08:12:41.947920+0000","last_undegraded":"2024-07-17T08:12:41.947920+0000","last_fullsized":"2024-07-17T08:12:41.947920+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:20.156544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"173'3","reported_seq":17,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.948112+0000","last_change":"2024-07-17T08:12:39.965917+0000","last_active":"2024-07-17T08:12:41.948112+0000","last_peered":"2024-07-17T08:12:41.948112+0000","last_clean":"2024-07-17T08:12:41.948112+0000","last_became_active":"2024-07-17T08:12:39.963869+0000","last_became_peered":"2024-07-17T08:12:39.963869+0000","last_unstale":"2024-07-17T08:12:41.948112+0000","last_undegraded":"2024-07-17T08:12:41.948112+0000","last_fullsized":"2024-07-17T08:12:41.948112+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:29.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951948+0000","last_change":"2024-07-17T08:12:40.979069+0000","last_active":"2024-07-17T08:12:41.951948+0000","last_peered":"2024-07-17T08:12:41.951948+0000","last_clean":"2024-07-17T08:12:41.951948+0000","last_became_active":"2024-07-17T08:12:40.978341+0000","last_became_peered":"2024-07-17T08:12:40.978341+0000","last_unstale":"2024-07-17T08:12:41.951948+0000","last_undegraded":"2024-07-17T08:12:41.951948+0000","last_fullsized":"2024-07-17T08:12:41.951948+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:45.794369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382633+0000","last_change":"2024-07-17T08:12:39.965736+0000","last_active":"2024-07-17T08:12:43.382633+0000","last_peered":"2024-07-17T08:12:43.382633+0000","last_clean":"2024-07-17T08:12:43.382633+0000","last_became_active":"2024-07-17T08:12:39.964796+0000","last_became_peered":"2024-07-17T08:12:39.964796+0000","last_unstale":"2024-07-17T08:12:43.382633+0000","last_undegraded":"2024-07-17T08:12:43.382633+0000","last_fullsized":"2024-07-17T08:12:43.382633+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:26.826945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367723+0000","last_change":"2024-07-17T08:12:40.986769+0000","last_active":"2024-07-17T08:12:43.367723+0000","last_peered":"2024-07-17T08:12:43.367723+0000","last_clean":"2024-07-17T08:12:43.367723+0000","last_became_active":"2024-07-17T08:12:40.986483+0000","last_became_peered":"2024-07-17T08:12:40.986483+0000","last_unstale":"2024-07-17T08:12:43.367723+0000","last_undegraded":"2024-07-17T08:12:43.367723+0000","last_fullsized":"2024-07-17T08:12:43.367723+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:26.695183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.953477+0000","last_change":"2024-07-17T08:12:39.975115+0000","last_active":"2024-07-17T08:12:41.953477+0000","last_peered":"2024-07-17T08:12:41.953477+0000","last_clean":"2024-07-17T08:12:41.953477+0000","last_became_active":"2024-07-17T08:12:39.974804+0000","last_became_peered":"2024-07-17T08:12:39.974804+0000","last_unstale":"2024-07-17T08:12:41.953477+0000","last_undegraded":"2024-07-17T08:12:41.953477+0000","last_fullsized":"2024-07-17T08:12:41.953477+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:33.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381899+0000","last_change":"2024-07-17T08:12:40.953908+0000","last_active":"2024-07-17T08:12:43.381899+0000","last_peered":"2024-07-17T08:12:43.381899+0000","last_clean":"2024-07-17T08:12:43.381899+0000","last_became_active":"2024-07-17T08:12:40.953536+0000","last_became_peered":"2024-07-17T08:12:40.953536+0000","last_unstale":"2024-07-17T08:12:43.381899+0000","last_undegraded":"2024-07-17T08:12:43.381899+0000","last_fullsized":"2024-07-17T08:12:43.381899+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:08.957400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952887+0000","last_change":"2024-07-17T08:12:39.971111+0000","last_active":"2024-07-17T08:12:41.952887+0000","last_peered":"2024-07-17T08:12:41.952887+0000","last_clean":"2024-07-17T08:12:41.952887+0000","last_became_active":"2024-07-17T08:12:39.969608+0000","last_became_peered":"2024-07-17T08:12:39.969608+0000","last_unstale":"2024-07-17T08:12:41.952887+0000","last_undegraded":"2024-07-17T08:12:41.952887+0000","last_fullsized":"2024-07-17T08:12:41.952887+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:30.667648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367826+0000","last_change":"2024-07-17T08:12:40.972863+0000","last_active":"2024-07-17T08:12:43.367826+0000","last_peered":"2024-07-17T08:12:43.367826+0000","last_clean":"2024-07-17T08:12:43.367826+0000","last_became_active":"2024-07-17T08:12:40.970381+0000","last_became_peered":"2024-07-17T08:12:40.970381+0000","last_unstale":"2024-07-17T08:12:43.367826+0000","last_undegraded":"2024-07-17T08:12:43.367826+0000","last_fullsized":"2024-07-17T08:12:43.367826+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:24.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":22,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661801+0000","last_change":"2024-07-17T08:12:39.981503+0000","last_active":"2024-07-17T08:12:42.661801+0000","last_peered":"2024-07-17T08:12:42.661801+0000","last_clean":"2024-07-17T08:12:42.661801+0000","last_became_active":"2024-07-17T08:12:39.981126+0000","last_became_peered":"2024-07-17T08:12:39.981126+0000","last_unstale":"2024-07-17T08:12:42.661801+0000","last_undegraded":"2024-07-17T08:12:42.661801+0000","last_fullsized":"2024-07-17T08:12:42.661801+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:48.132769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062583+0000","last_change":"2024-07-17T08:12:40.986405+0000","last_active":"2024-07-17T08:12:42.062583+0000","last_peered":"2024-07-17T08:12:42.062583+0000","last_clean":"2024-07-17T08:12:42.062583+0000","last_became_active":"2024-07-17T08:12:40.985861+0000","last_became_peered":"2024-07-17T08:12:40.985861+0000","last_unstale":"2024-07-17T08:12:42.062583+0000","last_undegraded":"2024-07-17T08:12:42.062583+0000","last_fullsized":"2024-07-17T08:12:42.062583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:30.295306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947973+0000","last_change":"2024-07-17T08:12:39.966931+0000","last_active":"2024-07-17T08:12:41.947973+0000","last_peered":"2024-07-17T08:12:41.947973+0000","last_clean":"2024-07-17T08:12:41.947973+0000","last_became_active":"2024-07-17T08:12:39.966630+0000","last_became_peered":"2024-07-17T08:12:39.966630+0000","last_unstale":"2024-07-17T08:12:41.947973+0000","last_undegraded":"2024-07-17T08:12:41.947973+0000","last_fullsized":"2024-07-17T08:12:41.947973+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:32:26.581691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063047+0000","last_change":"2024-07-17T08:12:40.994300+0000","last_active":"2024-07-17T08:12:42.063047+0000","last_peered":"2024-07-17T08:12:42.063047+0000","last_clean":"2024-07-17T08:12:42.063047+0000","last_became_active":"2024-07-17T08:12:40.994053+0000","last_became_peered":"2024-07-17T08:12:40.994053+0000","last_unstale":"2024-07-17T08:12:42.063047+0000","last_undegraded":"2024-07-17T08:12:42.063047+0000","last_fullsized":"2024-07-17T08:12:42.063047+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:50.343054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062985+0000","last_change":"2024-07-17T08:12:39.980185+0000","last_active":"2024-07-17T08:12:42.062985+0000","last_peered":"2024-07-17T08:12:42.062985+0000","last_clean":"2024-07-17T08:12:42.062985+0000","last_became_active":"2024-07-17T08:12:39.979010+0000","last_became_peered":"2024-07-17T08:12:39.979010+0000","last_unstale":"2024-07-17T08:12:42.062985+0000","last_undegraded":"2024-07-17T08:12:42.062985+0000","last_fullsized":"2024-07-17T08:12:42.062985+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:13.229833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662611+0000","last_change":"2024-07-17T08:12:40.968751+0000","last_active":"2024-07-17T08:12:42.662611+0000","last_peered":"2024-07-17T08:12:42.662611+0000","last_clean":"2024-07-17T08:12:42.662611+0000","last_became_active":"2024-07-17T08:12:40.966006+0000","last_became_peered":"2024-07-17T08:12:40.966006+0000","last_unstale":"2024-07-17T08:12:42.662611+0000","last_undegraded":"2024-07-17T08:12:42.662611+0000","last_fullsized":"2024-07-17T08:12:42.662611+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:50.432523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368945+0000","last_change":"2024-07-17T08:12:39.970173+0000","last_active":"2024-07-17T08:12:43.368945+0000","last_peered":"2024-07-17T08:12:43.368945+0000","last_clean":"2024-07-17T08:12:43.368945+0000","last_became_active":"2024-07-17T08:12:39.969804+0000","last_became_peered":"2024-07-17T08:12:39.969804+0000","last_unstale":"2024-07-17T08:12:43.368945+0000","last_undegraded":"2024-07-17T08:12:43.368945+0000","last_fullsized":"2024-07-17T08:12:43.368945+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:33.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662510+0000","last_change":"2024-07-17T08:12:40.980821+0000","last_active":"2024-07-17T08:12:42.662510+0000","last_peered":"2024-07-17T08:12:42.662510+0000","last_clean":"2024-07-17T08:12:42.662510+0000","last_became_active":"2024-07-17T08:12:40.978280+0000","last_became_peered":"2024-07-17T08:12:40.978280+0000","last_unstale":"2024-07-17T08:12:42.662510+0000","last_undegraded":"2024-07-17T08:12:42.662510+0000","last_fullsized":"2024-07-17T08:12:42.662510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:30:24.899223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954840+0000","last_change":"2024-07-17T08:12:39.978041+0000","last_active":"2024-07-17T08:12:41.954840+0000","last_peered":"2024-07-17T08:12:41.954840+0000","last_clean":"2024-07-17T08:12:41.954840+0000","last_became_active":"2024-07-17T08:12:39.977732+0000","last_became_peered":"2024-07-17T08:12:39.977732+0000","last_unstale":"2024-07-17T08:12:41.954840+0000","last_undegraded":"2024-07-17T08:12:41.954840+0000","last_fullsized":"2024-07-17T08:12:41.954840+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:15.428828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062281+0000","last_change":"2024-07-17T08:12:40.990242+0000","last_active":"2024-07-17T08:12:42.062281+0000","last_peered":"2024-07-17T08:12:42.062281+0000","last_clean":"2024-07-17T08:12:42.062281+0000","last_became_active":"2024-07-17T08:12:40.989971+0000","last_became_peered":"2024-07-17T08:12:40.989971+0000","last_unstale":"2024-07-17T08:12:42.062281+0000","last_undegraded":"2024-07-17T08:12:42.062281+0000","last_fullsized":"2024-07-17T08:12:42.062281+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:02.030798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875680+0000","last_change":"2024-07-17T08:12:39.978456+0000","last_active":"2024-07-17T08:12:42.875680+0000","last_peered":"2024-07-17T08:12:42.875680+0000","last_clean":"2024-07-17T08:12:42.875680+0000","last_became_active":"2024-07-17T08:12:39.978154+0000","last_became_peered":"2024-07-17T08:12:39.978154+0000","last_unstale":"2024-07-17T08:12:42.875680+0000","last_undegraded":"2024-07-17T08:12:42.875680+0000","last_fullsized":"2024-07-17T08:12:42.875680+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:56.336900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367928+0000","last_change":"2024-07-17T08:12:40.977335+0000","last_active":"2024-07-17T08:12:43.367928+0000","last_peered":"2024-07-17T08:12:43.367928+0000","last_clean":"2024-07-17T08:12:43.367928+0000","last_became_active":"2024-07-17T08:12:40.975406+0000","last_became_peered":"2024-07-17T08:12:40.975406+0000","last_unstale":"2024-07-17T08:12:43.367928+0000","last_undegraded":"2024-07-17T08:12:43.367928+0000","last_fullsized":"2024-07-17T08:12:43.367928+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:57.189144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952396+0000","last_change":"2024-07-17T08:12:39.973638+0000","last_active":"2024-07-17T08:12:41.952396+0000","last_peered":"2024-07-17T08:12:41.952396+0000","last_clean":"2024-07-17T08:12:41.952396+0000","last_became_active":"2024-07-17T08:12:39.973137+0000","last_became_peered":"2024-07-17T08:12:39.973137+0000","last_unstale":"2024-07-17T08:12:41.952396+0000","last_undegraded":"2024-07-17T08:12:41.952396+0000","last_fullsized":"2024-07-17T08:12:41.952396+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:50:40.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.061970+0000","last_change":"2024-07-17T08:12:40.985007+0000","last_active":"2024-07-17T08:12:42.061970+0000","last_peered":"2024-07-17T08:12:42.061970+0000","last_clean":"2024-07-17T08:12:42.061970+0000","last_became_active":"2024-07-17T08:12:40.984730+0000","last_became_peered":"2024-07-17T08:12:40.984730+0000","last_unstale":"2024-07-17T08:12:42.061970+0000","last_undegraded":"2024-07-17T08:12:42.061970+0000","last_fullsized":"2024-07-17T08:12:42.061970+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:01.189730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875314+0000","last_change":"2024-07-17T08:12:39.969706+0000","last_active":"2024-07-17T08:12:42.875314+0000","last_peered":"2024-07-17T08:12:42.875314+0000","last_clean":"2024-07-17T08:12:42.875314+0000","last_became_active":"2024-07-17T08:12:39.969274+0000","last_became_peered":"2024-07-17T08:12:39.969274+0000","last_unstale":"2024-07-17T08:12:42.875314+0000","last_undegraded":"2024-07-17T08:12:42.875314+0000","last_fullsized":"2024-07-17T08:12:42.875314+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:36.963194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368244+0000","last_change":"2024-07-17T08:12:40.986353+0000","last_active":"2024-07-17T08:12:43.368244+0000","last_peered":"2024-07-17T08:12:43.368244+0000","last_clean":"2024-07-17T08:12:43.368244+0000","last_became_active":"2024-07-17T08:12:40.985975+0000","last_became_peered":"2024-07-17T08:12:40.985975+0000","last_unstale":"2024-07-17T08:12:43.368244+0000","last_undegraded":"2024-07-17T08:12:43.368244+0000","last_fullsized":"2024-07-17T08:12:43.368244+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:49.138913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952238+0000","last_change":"2024-07-17T08:12:39.973183+0000","last_active":"2024-07-17T08:12:41.952238+0000","last_peered":"2024-07-17T08:12:41.952238+0000","last_clean":"2024-07-17T08:12:41.952238+0000","last_became_active":"2024-07-17T08:12:39.972882+0000","last_became_peered":"2024-07-17T08:12:39.972882+0000","last_unstale":"2024-07-17T08:12:41.952238+0000","last_undegraded":"2024-07-17T08:12:41.952238+0000","last_fullsized":"2024-07-17T08:12:41.952238+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:48.284682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382577+0000","last_change":"2024-07-17T08:12:40.967497+0000","last_active":"2024-07-17T08:12:43.382577+0000","last_peered":"2024-07-17T08:12:43.382577+0000","last_clean":"2024-07-17T08:12:43.382577+0000","last_became_active":"2024-07-17T08:12:40.967137+0000","last_became_peered":"2024-07-17T08:12:40.967137+0000","last_unstale":"2024-07-17T08:12:43.382577+0000","last_undegraded":"2024-07-17T08:12:43.382577+0000","last_fullsized":"2024-07-17T08:12:43.382577+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:37.969337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955079+0000","last_change":"2024-07-17T08:12:39.975659+0000","last_active":"2024-07-17T08:12:41.955079+0000","last_peered":"2024-07-17T08:12:41.955079+0000","last_clean":"2024-07-17T08:12:41.955079+0000","last_became_active":"2024-07-17T08:12:39.975352+0000","last_became_peered":"2024-07-17T08:12:39.975352+0000","last_unstale":"2024-07-17T08:12:41.955079+0000","last_undegraded":"2024-07-17T08:12:41.955079+0000","last_fullsized":"2024-07-17T08:12:41.955079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:05.504874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954878+0000","last_change":"2024-07-17T08:12:40.979905+0000","last_active":"2024-07-17T08:12:41.954878+0000","last_peered":"2024-07-17T08:12:41.954878+0000","last_clean":"2024-07-17T08:12:41.954878+0000","last_became_active":"2024-07-17T08:12:40.979619+0000","last_became_peered":"2024-07-17T08:12:40.979619+0000","last_unstale":"2024-07-17T08:12:41.954878+0000","last_undegraded":"2024-07-17T08:12:41.954878+0000","last_fullsized":"2024-07-17T08:12:41.954878+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:24.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"174'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.959944+0000","last_change":"2024-07-17T08:12:39.971941+0000","last_active":"2024-07-17T08:12:42.959944+0000","last_peered":"2024-07-17T08:12:42.959944+0000","last_clean":"2024-07-17T08:12:42.959944+0000","last_became_active":"2024-07-17T08:12:39.971410+0000","last_became_peered":"2024-07-17T08:12:39.971410+0000","last_unstale":"2024-07-17T08:12:42.959944+0000","last_undegraded":"2024-07-17T08:12:42.959944+0000","last_fullsized":"2024-07-17T08:12:42.959944+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:47.400806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662704+0000","last_change":"2024-07-17T08:12:40.981113+0000","last_active":"2024-07-17T08:12:42.662704+0000","last_peered":"2024-07-17T08:12:42.662704+0000","last_clean":"2024-07-17T08:12:42.662704+0000","last_became_active":"2024-07-17T08:12:40.980833+0000","last_became_peered":"2024-07-17T08:12:40.980833+0000","last_unstale":"2024-07-17T08:12:42.662704+0000","last_undegraded":"2024-07-17T08:12:42.662704+0000","last_fullsized":"2024-07-17T08:12:42.662704+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:27.833088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368844+0000","last_change":"2024-07-17T08:12:39.974518+0000","last_active":"2024-07-17T08:12:43.368844+0000","last_peered":"2024-07-17T08:12:43.368844+0000","last_clean":"2024-07-17T08:12:43.368844+0000","last_became_active":"2024-07-17T08:12:39.974238+0000","last_became_peered":"2024-07-17T08:12:39.974238+0000","last_unstale":"2024-07-17T08:12:43.368844+0000","last_undegraded":"2024-07-17T08:12:43.368844+0000","last_fullsized":"2024-07-17T08:12:43.368844+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:30.667648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382911+0000","last_change":"2024-07-17T08:12:40.983976+0000","last_active":"2024-07-17T08:12:43.382911+0000","last_peered":"2024-07-17T08:12:43.382911+0000","last_clean":"2024-07-17T08:12:43.382911+0000","last_became_active":"2024-07-17T08:12:40.983512+0000","last_became_peered":"2024-07-17T08:12:40.983512+0000","last_unstale":"2024-07-17T08:12:43.382911+0000","last_undegraded":"2024-07-17T08:12:43.382911+0000","last_fullsized":"2024-07-17T08:12:43.382911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:27.937363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382461+0000","last_change":"2024-07-17T08:12:39.978198+0000","last_active":"2024-07-17T08:12:43.382461+0000","last_peered":"2024-07-17T08:12:43.382461+0000","last_clean":"2024-07-17T08:12:43.382461+0000","last_became_active":"2024-07-17T08:12:39.977463+0000","last_became_peered":"2024-07-17T08:12:39.977463+0000","last_unstale":"2024-07-17T08:12:43.382461+0000","last_undegraded":"2024-07-17T08:12:43.382461+0000","last_fullsized":"2024-07-17T08:12:43.382461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:10.667095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952320+0000","last_change":"2024-07-17T08:12:40.978609+0000","last_active":"2024-07-17T08:12:41.952320+0000","last_peered":"2024-07-17T08:12:41.952320+0000","last_clean":"2024-07-17T08:12:41.952320+0000","last_became_active":"2024-07-17T08:12:40.976699+0000","last_became_peered":"2024-07-17T08:12:40.976699+0000","last_unstale":"2024-07-17T08:12:41.952320+0000","last_undegraded":"2024-07-17T08:12:41.952320+0000","last_fullsized":"2024-07-17T08:12:41.952320+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:11:27.950859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662143+0000","last_change":"2024-07-17T08:12:39.969612+0000","last_active":"2024-07-17T08:12:42.662143+0000","last_peered":"2024-07-17T08:12:42.662143+0000","last_clean":"2024-07-17T08:12:42.662143+0000","last_became_active":"2024-07-17T08:12:39.969335+0000","last_became_peered":"2024-07-17T08:12:39.969335+0000","last_unstale":"2024-07-17T08:12:42.662143+0000","last_undegraded":"2024-07-17T08:12:42.662143+0000","last_fullsized":"2024-07-17T08:12:42.662143+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:16.698750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952984+0000","last_change":"2024-07-17T08:12:40.982802+0000","last_active":"2024-07-17T08:12:41.952984+0000","last_peered":"2024-07-17T08:12:41.952984+0000","last_clean":"2024-07-17T08:12:41.952984+0000","last_became_active":"2024-07-17T08:12:40.976004+0000","last_became_peered":"2024-07-17T08:12:40.976004+0000","last_unstale":"2024-07-17T08:12:41.952984+0000","last_undegraded":"2024-07-17T08:12:41.952984+0000","last_fullsized":"2024-07-17T08:12:41.952984+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:45:00.464498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947795+0000","last_change":"2024-07-17T08:12:39.973111+0000","last_active":"2024-07-17T08:12:41.947795+0000","last_peered":"2024-07-17T08:12:41.947795+0000","last_clean":"2024-07-17T08:12:41.947795+0000","last_became_active":"2024-07-17T08:12:39.972828+0000","last_became_peered":"2024-07-17T08:12:39.972828+0000","last_unstale":"2024-07-17T08:12:41.947795+0000","last_undegraded":"2024-07-17T08:12:41.947795+0000","last_fullsized":"2024-07-17T08:12:41.947795+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:57:16.640714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368468+0000","last_change":"2024-07-17T08:12:40.971677+0000","last_active":"2024-07-17T08:12:43.368468+0000","last_peered":"2024-07-17T08:12:43.368468+0000","last_clean":"2024-07-17T08:12:43.368468+0000","last_became_active":"2024-07-17T08:12:40.971383+0000","last_became_peered":"2024-07-17T08:12:40.971383+0000","last_unstale":"2024-07-17T08:12:43.368468+0000","last_undegraded":"2024-07-17T08:12:43.368468+0000","last_fullsized":"2024-07-17T08:12:43.368468+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:57.737087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662018+0000","last_change":"2024-07-17T08:12:39.968980+0000","last_active":"2024-07-17T08:12:42.662018+0000","last_peered":"2024-07-17T08:12:42.662018+0000","last_clean":"2024-07-17T08:12:42.662018+0000","last_became_active":"2024-07-17T08:12:39.968644+0000","last_became_peered":"2024-07-17T08:12:39.968644+0000","last_unstale":"2024-07-17T08:12:42.662018+0000","last_undegraded":"2024-07-17T08:12:42.662018+0000","last_fullsized":"2024-07-17T08:12:42.662018+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:56.730943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952423+0000","last_change":"2024-07-17T08:12:40.980487+0000","last_active":"2024-07-17T08:12:41.952423+0000","last_peered":"2024-07-17T08:12:41.952423+0000","last_clean":"2024-07-17T08:12:41.952423+0000","last_became_active":"2024-07-17T08:12:40.979907+0000","last_became_peered":"2024-07-17T08:12:40.979907+0000","last_unstale":"2024-07-17T08:12:41.952423+0000","last_undegraded":"2024-07-17T08:12:41.952423+0000","last_fullsized":"2024-07-17T08:12:41.952423+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:50.638303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062109+0000","last_change":"2024-07-17T08:12:39.977423+0000","last_active":"2024-07-17T08:12:42.062109+0000","last_peered":"2024-07-17T08:12:42.062109+0000","last_clean":"2024-07-17T08:12:42.062109+0000","last_became_active":"2024-07-17T08:12:39.977037+0000","last_became_peered":"2024-07-17T08:12:39.977037+0000","last_unstale":"2024-07-17T08:12:42.062109+0000","last_undegraded":"2024-07-17T08:12:42.062109+0000","last_fullsized":"2024-07-17T08:12:42.062109+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:26:47.332136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955942+0000","last_change":"2024-07-17T08:12:40.973812+0000","last_active":"2024-07-17T08:12:41.955942+0000","last_peered":"2024-07-17T08:12:41.955942+0000","last_clean":"2024-07-17T08:12:41.955942+0000","last_became_active":"2024-07-17T08:12:40.973112+0000","last_became_peered":"2024-07-17T08:12:40.973112+0000","last_unstale":"2024-07-17T08:12:41.955942+0000","last_undegraded":"2024-07-17T08:12:41.955942+0000","last_fullsized":"2024-07-17T08:12:41.955942+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:34.853668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":20,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875772+0000","last_change":"2024-07-17T08:12:39.976810+0000","last_active":"2024-07-17T08:12:42.875772+0000","last_peered":"2024-07-17T08:12:42.875772+0000","last_clean":"2024-07-17T08:12:42.875772+0000","last_became_active":"2024-07-17T08:12:39.976078+0000","last_became_peered":"2024-07-17T08:12:39.976078+0000","last_unstale":"2024-07-17T08:12:42.875772+0000","last_undegraded":"2024-07-17T08:12:42.875772+0000","last_fullsized":"2024-07-17T08:12:42.875772+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:38.594753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"164'2462","reported_seq":3105,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062250+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:12:42.062250+0000","last_peered":"2024-07-17T08:12:42.062250+0000","last_clean":"2024-07-17T08:12:42.062250+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:12:42.062250+0000","last_undegraded":"2024-07-17T08:12:42.062250+0000","last_fullsized":"2024-07-17T08:12:42.062250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2462,"log_dups_size":0,"ondisk_log_size":2462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":275,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":275,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2462,"ondisk_log_size":2462,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543073,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60248,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311592,"statfs":{"total":96636764160,"available":96575070208,"internally_reserved":0,"allocated":2764800,"data_stored":1755444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510368,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41624,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2170880,"data_stored":1165124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510369,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47184,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324656,"statfs":{"total":96636764160,"available":96588447744,"internally_reserved":0,"allocated":2166784,"data_stored":1165054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543074,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41636,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330204,"statfs":{"total":96636764160,"available":96594128896,"internally_reserved":0,"allocated":2179072,"data_stored":1166666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510368,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59668,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312172,"statfs":{"total":96636764160,"available":96575664128,"internally_reserved":0,"allocated":2162688,"data_stored":1158430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543072,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47192,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324648,"statfs":{"total":96636764160,"available":96588439552,"internally_reserved":0,"allocated":2174976,"data_stored":1165166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41612,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330228,"statfs":{"total":96636764160,"available":96594153472,"internally_reserved":0,"allocated":2158592,"data_stored":1157884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510368,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1750500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:12:45.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:12:45.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 -- 172.21.15.44:0/244834792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b5400 msgr2=0x7f412c06def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 --2- 172.21.15.44:0/244834792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b5400 0x7f412c06def0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4124005fd0 tx=0x7f412402a580 comp rx=0 tx=0).stop 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 -- 172.21.15.44:0/244834792 shutdown_connections 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 --2- 172.21.15.44:0/244834792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c06d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 --2- 172.21.15.44:0/244834792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b5400 0x7f412c06def0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 --2- 172.21.15.44:0/244834792 >> v2:172.21.15.175:3300/0 conn(0x7f412c0b8c80 0x7f412c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 -- 172.21.15.44:0/244834792 >> 172.21.15.44:0/244834792 conn(0x7f412c0cfec0 msgr2=0x7f412c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 -- 172.21.15.44:0/244834792 shutdown_connections 2024-07-17T08:12:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.727+0000 7f412b200640 1 -- 172.21.15.44:0/244834792 wait complete. 2024-07-17T08:12:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.729+0000 7f412b200640 1 Processor -- start 2024-07-17T08:12:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.729+0000 7f412b200640 1 -- start start 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f412c0b5400 0x7f412c0ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f412c06e5b0 con 0x7f412c0b8c80 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f412c06e430 con 0x7f412c0b5400 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.730+0000 7f412b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f412c06e730 con 0x7f412c0ce5c0 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48604/0 (socket says 172.21.15.44:48604) 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 -- 172.21.15.44:0/2444069656 learned_addr learned my addr 172.21.15.44:0/2444069656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129e00640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f412c0b5400 0x7f412c0ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f412a800640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 -- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 msgr2=0x7f412c151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 -- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f412c0b5400 msgr2=0x7f412c0ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f412c0b5400 0x7f412c0ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f4129400640 1 -- 172.21.15.44:0/2444069656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c0ab210 con 0x7f412c0b8c80 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.731+0000 7f412a800640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:45.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f4129400640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f411c00ea50 tx=0x7f411c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:45.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c00cd60 con 0x7f412c0b8c80 2024-07-17T08:12:45.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f411c011470 con 0x7f412c0b8c80 2024-07-17T08:12:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c0118a0 con 0x7f412c0b8c80 2024-07-17T08:12:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c0ab4f0 con 0x7f412c0b8c80 2024-07-17T08:12:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.732+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c0abab0 con 0x7f412c0b8c80 2024-07-17T08:12:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.734+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c0b4ec0 con 0x7f412c0b8c80 2024-07-17T08:12:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.735+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f411c011a40 con 0x7f412c0b8c80 2024-07-17T08:12:45.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.736+0000 7f4122000640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 0x7f4108079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:45.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.736+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f411c0991a0 con 0x7f412c0b8c80 2024-07-17T08:12:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.737+0000 7f4129e00640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 0x7f4108079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.738+0000 7f4129e00640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 0x7f4108079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4114005e00 tx=0x7f4114005c50 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:45.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.740+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f411c016030 con 0x7f412c0b8c80 2024-07-17T08:12:45.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.948+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 --> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f412c0aafc0 con 0x7f4108076bb0 2024-07-17T08:12:45.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.962+0000 7f4122000640 1 -- 172.21.15.44:0/2444069656 <== mgr.6653 v2:172.21.15.175:6834/3617669990 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7f412c0aafc0 con 0x7f4108076bb0 2024-07-17T08:12:45.967 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:45.968 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:12:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 msgr2=0x7f4108079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 0x7f4108079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4114005e00 tx=0x7f4114005c50 comp rx=0 tx=0).stop 2024-07-17T08:12:45.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 msgr2=0x7f412c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f411c00ea50 tx=0x7f411c00ef10 comp rx=0 tx=0).stop 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 shutdown_connections 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f4108076bb0 0x7f4108079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f412c0ce5c0 0x7f412c151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f412c0b8c80 0x7f412c0b1740 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 --2- 172.21.15.44:0/2444069656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f412c0b5400 0x7f412c0ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.968+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 >> 172.21.15.44:0/2444069656 conn(0x7f412c0cfec0 msgr2=0x7f412c0b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.969+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 shutdown_connections 2024-07-17T08:12:45.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:45.969+0000 7f412b200640 1 -- 172.21.15.44:0/2444069656 wait complete. 2024-07-17T08:12:45.988 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-07-17T08:12:45.399394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":320,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2501,"ondisk_log_size":2501,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":373056,"kb_used_data":18116,"kb_used_omap":12,"kb_used_meta":354803,"kb_avail":754601664,"statfs":{"total":773094113280,"available":772712103936,"internally_reserved":0,"allocated":18550784,"data_stored":10484268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":363318832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002039"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662283+0000","last_change":"2024-07-17T08:12:40.982016+0000","last_active":"2024-07-17T08:12:42.662283+0000","last_peered":"2024-07-17T08:12:42.662283+0000","last_clean":"2024-07-17T08:12:42.662283+0000","last_became_active":"2024-07-17T08:12:40.977962+0000","last_became_peered":"2024-07-17T08:12:40.977962+0000","last_unstale":"2024-07-17T08:12:42.662283+0000","last_undegraded":"2024-07-17T08:12:42.662283+0000","last_fullsized":"2024-07-17T08:12:42.662283+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:29.889194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367925+0000","last_change":"2024-07-17T08:12:40.972918+0000","last_active":"2024-07-17T08:12:43.367925+0000","last_peered":"2024-07-17T08:12:43.367925+0000","last_clean":"2024-07-17T08:12:43.367925+0000","last_became_active":"2024-07-17T08:12:40.972131+0000","last_became_peered":"2024-07-17T08:12:40.972131+0000","last_unstale":"2024-07-17T08:12:43.367925+0000","last_undegraded":"2024-07-17T08:12:43.367925+0000","last_fullsized":"2024-07-17T08:12:43.367925+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:01.340713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368653+0000","last_change":"2024-07-17T08:12:40.984995+0000","last_active":"2024-07-17T08:12:43.368653+0000","last_peered":"2024-07-17T08:12:43.368653+0000","last_clean":"2024-07-17T08:12:43.368653+0000","last_became_active":"2024-07-17T08:12:40.984600+0000","last_became_peered":"2024-07-17T08:12:40.984600+0000","last_unstale":"2024-07-17T08:12:43.368653+0000","last_undegraded":"2024-07-17T08:12:43.368653+0000","last_fullsized":"2024-07-17T08:12:43.368653+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:23:26.303837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955142+0000","last_change":"2024-07-17T08:12:40.973444+0000","last_active":"2024-07-17T08:12:41.955142+0000","last_peered":"2024-07-17T08:12:41.955142+0000","last_clean":"2024-07-17T08:12:41.955142+0000","last_became_active":"2024-07-17T08:12:40.973173+0000","last_became_peered":"2024-07-17T08:12:40.973173+0000","last_unstale":"2024-07-17T08:12:41.955142+0000","last_undegraded":"2024-07-17T08:12:41.955142+0000","last_fullsized":"2024-07-17T08:12:41.955142+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:01.340713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382598+0000","last_change":"2024-07-17T08:12:40.976129+0000","last_active":"2024-07-17T08:12:43.382598+0000","last_peered":"2024-07-17T08:12:43.382598+0000","last_clean":"2024-07-17T08:12:43.382598+0000","last_became_active":"2024-07-17T08:12:40.974654+0000","last_became_peered":"2024-07-17T08:12:40.974654+0000","last_unstale":"2024-07-17T08:12:43.382598+0000","last_undegraded":"2024-07-17T08:12:43.382598+0000","last_fullsized":"2024-07-17T08:12:43.382598+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:39.600897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381752+0000","last_change":"2024-07-17T08:12:40.959361+0000","last_active":"2024-07-17T08:12:43.381752+0000","last_peered":"2024-07-17T08:12:43.381752+0000","last_clean":"2024-07-17T08:12:43.381752+0000","last_became_active":"2024-07-17T08:12:40.958577+0000","last_became_peered":"2024-07-17T08:12:40.958577+0000","last_unstale":"2024-07-17T08:12:43.381752+0000","last_undegraded":"2024-07-17T08:12:43.381752+0000","last_fullsized":"2024-07-17T08:12:43.381752+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:22.940653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661918+0000","last_change":"2024-07-17T08:12:40.964253+0000","last_active":"2024-07-17T08:12:42.661918+0000","last_peered":"2024-07-17T08:12:42.661918+0000","last_clean":"2024-07-17T08:12:42.661918+0000","last_became_active":"2024-07-17T08:12:40.963955+0000","last_became_peered":"2024-07-17T08:12:40.963955+0000","last_unstale":"2024-07-17T08:12:42.661918+0000","last_undegraded":"2024-07-17T08:12:42.661918+0000","last_fullsized":"2024-07-17T08:12:42.661918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:51.660772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951378+0000","last_change":"2024-07-17T08:12:40.978851+0000","last_active":"2024-07-17T08:12:41.951378+0000","last_peered":"2024-07-17T08:12:41.951378+0000","last_clean":"2024-07-17T08:12:41.951378+0000","last_became_active":"2024-07-17T08:12:40.977412+0000","last_became_peered":"2024-07-17T08:12:40.977412+0000","last_unstale":"2024-07-17T08:12:41.951378+0000","last_undegraded":"2024-07-17T08:12:41.951378+0000","last_fullsized":"2024-07-17T08:12:41.951378+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.127724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382909+0000","last_change":"2024-07-17T08:12:40.974678+0000","last_active":"2024-07-17T08:12:43.382909+0000","last_peered":"2024-07-17T08:12:43.382909+0000","last_clean":"2024-07-17T08:12:43.382909+0000","last_became_active":"2024-07-17T08:12:40.974358+0000","last_became_peered":"2024-07-17T08:12:40.974358+0000","last_unstale":"2024-07-17T08:12:43.382909+0000","last_undegraded":"2024-07-17T08:12:43.382909+0000","last_fullsized":"2024-07-17T08:12:43.382909+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:57.343043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382953+0000","last_change":"2024-07-17T08:12:40.970873+0000","last_active":"2024-07-17T08:12:43.382953+0000","last_peered":"2024-07-17T08:12:43.382953+0000","last_clean":"2024-07-17T08:12:43.382953+0000","last_became_active":"2024-07-17T08:12:40.969332+0000","last_became_peered":"2024-07-17T08:12:40.969332+0000","last_unstale":"2024-07-17T08:12:43.382953+0000","last_undegraded":"2024-07-17T08:12:43.382953+0000","last_fullsized":"2024-07-17T08:12:43.382953+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:31:30.695204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.948136+0000","last_change":"2024-07-17T08:12:40.976236+0000","last_active":"2024-07-17T08:12:41.948136+0000","last_peered":"2024-07-17T08:12:41.948136+0000","last_clean":"2024-07-17T08:12:41.948136+0000","last_became_active":"2024-07-17T08:12:40.975956+0000","last_became_peered":"2024-07-17T08:12:40.975956+0000","last_unstale":"2024-07-17T08:12:41.948136+0000","last_undegraded":"2024-07-17T08:12:41.948136+0000","last_fullsized":"2024-07-17T08:12:41.948136+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:16.879818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954538+0000","last_change":"2024-07-17T08:12:40.973318+0000","last_active":"2024-07-17T08:12:41.954538+0000","last_peered":"2024-07-17T08:12:41.954538+0000","last_clean":"2024-07-17T08:12:41.954538+0000","last_became_active":"2024-07-17T08:12:40.972876+0000","last_became_peered":"2024-07-17T08:12:40.972876+0000","last_unstale":"2024-07-17T08:12:41.954538+0000","last_undegraded":"2024-07-17T08:12:41.954538+0000","last_fullsized":"2024-07-17T08:12:41.954538+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:26.695183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382036+0000","last_change":"2024-07-17T08:12:40.967437+0000","last_active":"2024-07-17T08:12:43.382036+0000","last_peered":"2024-07-17T08:12:43.382036+0000","last_clean":"2024-07-17T08:12:43.382036+0000","last_became_active":"2024-07-17T08:12:40.965913+0000","last_became_peered":"2024-07-17T08:12:40.965913+0000","last_unstale":"2024-07-17T08:12:43.382036+0000","last_undegraded":"2024-07-17T08:12:43.382036+0000","last_fullsized":"2024-07-17T08:12:43.382036+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:58.867645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.953442+0000","last_change":"2024-07-17T08:12:40.970072+0000","last_active":"2024-07-17T08:12:41.953442+0000","last_peered":"2024-07-17T08:12:41.953442+0000","last_clean":"2024-07-17T08:12:41.953442+0000","last_became_active":"2024-07-17T08:12:40.969743+0000","last_became_peered":"2024-07-17T08:12:40.969743+0000","last_unstale":"2024-07-17T08:12:41.953442+0000","last_undegraded":"2024-07-17T08:12:41.953442+0000","last_fullsized":"2024-07-17T08:12:41.953442+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.738801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367454+0000","last_change":"2024-07-17T08:12:40.986760+0000","last_active":"2024-07-17T08:12:43.367454+0000","last_peered":"2024-07-17T08:12:43.367454+0000","last_clean":"2024-07-17T08:12:43.367454+0000","last_became_active":"2024-07-17T08:12:40.986278+0000","last_became_peered":"2024-07-17T08:12:40.986278+0000","last_unstale":"2024-07-17T08:12:43.367454+0000","last_undegraded":"2024-07-17T08:12:43.367454+0000","last_fullsized":"2024-07-17T08:12:43.367454+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:36.738801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381576+0000","last_change":"2024-07-17T08:12:40.973444+0000","last_active":"2024-07-17T08:12:43.381576+0000","last_peered":"2024-07-17T08:12:43.381576+0000","last_clean":"2024-07-17T08:12:43.381576+0000","last_became_active":"2024-07-17T08:12:40.973117+0000","last_became_peered":"2024-07-17T08:12:40.973117+0000","last_unstale":"2024-07-17T08:12:43.381576+0000","last_undegraded":"2024-07-17T08:12:43.381576+0000","last_fullsized":"2024-07-17T08:12:43.381576+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:14.536558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955273+0000","last_change":"2024-07-17T08:12:40.976137+0000","last_active":"2024-07-17T08:12:41.955273+0000","last_peered":"2024-07-17T08:12:41.955273+0000","last_clean":"2024-07-17T08:12:41.955273+0000","last_became_active":"2024-07-17T08:12:40.975847+0000","last_became_peered":"2024-07-17T08:12:40.975847+0000","last_unstale":"2024-07-17T08:12:41.955273+0000","last_undegraded":"2024-07-17T08:12:41.955273+0000","last_fullsized":"2024-07-17T08:12:41.955273+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:57.189144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947966+0000","last_change":"2024-07-17T08:12:40.971971+0000","last_active":"2024-07-17T08:12:41.947966+0000","last_peered":"2024-07-17T08:12:41.947966+0000","last_clean":"2024-07-17T08:12:41.947966+0000","last_became_active":"2024-07-17T08:12:40.971676+0000","last_became_peered":"2024-07-17T08:12:40.971676+0000","last_unstale":"2024-07-17T08:12:41.947966+0000","last_undegraded":"2024-07-17T08:12:41.947966+0000","last_fullsized":"2024-07-17T08:12:41.947966+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:39.256096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063119+0000","last_change":"2024-07-17T08:12:40.992984+0000","last_active":"2024-07-17T08:12:42.063119+0000","last_peered":"2024-07-17T08:12:42.063119+0000","last_clean":"2024-07-17T08:12:42.063119+0000","last_became_active":"2024-07-17T08:12:40.992422+0000","last_became_peered":"2024-07-17T08:12:40.992422+0000","last_unstale":"2024-07-17T08:12:42.063119+0000","last_undegraded":"2024-07-17T08:12:42.063119+0000","last_fullsized":"2024-07-17T08:12:42.063119+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:38:06.900480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875848+0000","last_change":"2024-07-17T08:12:40.969842+0000","last_active":"2024-07-17T08:12:42.875848+0000","last_peered":"2024-07-17T08:12:42.875848+0000","last_clean":"2024-07-17T08:12:42.875848+0000","last_became_active":"2024-07-17T08:12:40.969541+0000","last_became_peered":"2024-07-17T08:12:40.969541+0000","last_unstale":"2024-07-17T08:12:42.875848+0000","last_undegraded":"2024-07-17T08:12:42.875848+0000","last_fullsized":"2024-07-17T08:12:42.875848+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:14.073148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382307+0000","last_change":"2024-07-17T08:12:40.963212+0000","last_active":"2024-07-17T08:12:43.382307+0000","last_peered":"2024-07-17T08:12:43.382307+0000","last_clean":"2024-07-17T08:12:43.382307+0000","last_became_active":"2024-07-17T08:12:40.962844+0000","last_became_peered":"2024-07-17T08:12:40.962844+0000","last_unstale":"2024-07-17T08:12:43.382307+0000","last_undegraded":"2024-07-17T08:12:43.382307+0000","last_fullsized":"2024-07-17T08:12:43.382307+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:27.861293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661853+0000","last_change":"2024-07-17T08:12:40.963785+0000","last_active":"2024-07-17T08:12:42.661853+0000","last_peered":"2024-07-17T08:12:42.661853+0000","last_clean":"2024-07-17T08:12:42.661853+0000","last_became_active":"2024-07-17T08:12:40.963463+0000","last_became_peered":"2024-07-17T08:12:40.963463+0000","last_unstale":"2024-07-17T08:12:42.661853+0000","last_undegraded":"2024-07-17T08:12:42.661853+0000","last_fullsized":"2024-07-17T08:12:42.661853+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:19.022860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951862+0000","last_change":"2024-07-17T08:12:40.982334+0000","last_active":"2024-07-17T08:12:41.951862+0000","last_peered":"2024-07-17T08:12:41.951862+0000","last_clean":"2024-07-17T08:12:41.951862+0000","last_became_active":"2024-07-17T08:12:40.982055+0000","last_became_peered":"2024-07-17T08:12:40.982055+0000","last_unstale":"2024-07-17T08:12:41.951862+0000","last_undegraded":"2024-07-17T08:12:41.951862+0000","last_fullsized":"2024-07-17T08:12:41.951862+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:01.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947580+0000","last_change":"2024-07-17T08:12:40.956802+0000","last_active":"2024-07-17T08:12:41.947580+0000","last_peered":"2024-07-17T08:12:41.947580+0000","last_clean":"2024-07-17T08:12:41.947580+0000","last_became_active":"2024-07-17T08:12:40.956418+0000","last_became_peered":"2024-07-17T08:12:40.956418+0000","last_unstale":"2024-07-17T08:12:41.947580+0000","last_undegraded":"2024-07-17T08:12:41.947580+0000","last_fullsized":"2024-07-17T08:12:41.947580+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:24.391300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662483+0000","last_change":"2024-07-17T08:12:40.968684+0000","last_active":"2024-07-17T08:12:42.662483+0000","last_peered":"2024-07-17T08:12:42.662483+0000","last_clean":"2024-07-17T08:12:42.662483+0000","last_became_active":"2024-07-17T08:12:40.968348+0000","last_became_peered":"2024-07-17T08:12:40.968348+0000","last_unstale":"2024-07-17T08:12:42.662483+0000","last_undegraded":"2024-07-17T08:12:42.662483+0000","last_fullsized":"2024-07-17T08:12:42.662483+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:31.015455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662206+0000","last_change":"2024-07-17T08:12:40.968483+0000","last_active":"2024-07-17T08:12:42.662206+0000","last_peered":"2024-07-17T08:12:42.662206+0000","last_clean":"2024-07-17T08:12:42.662206+0000","last_became_active":"2024-07-17T08:12:40.968018+0000","last_became_peered":"2024-07-17T08:12:40.968018+0000","last_unstale":"2024-07-17T08:12:42.662206+0000","last_undegraded":"2024-07-17T08:12:42.662206+0000","last_fullsized":"2024-07-17T08:12:42.662206+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:24:51.007876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382402+0000","last_change":"2024-07-17T08:12:40.981064+0000","last_active":"2024-07-17T08:12:43.382402+0000","last_peered":"2024-07-17T08:12:43.382402+0000","last_clean":"2024-07-17T08:12:43.382402+0000","last_became_active":"2024-07-17T08:12:40.977277+0000","last_became_peered":"2024-07-17T08:12:40.977277+0000","last_unstale":"2024-07-17T08:12:43.382402+0000","last_undegraded":"2024-07-17T08:12:43.382402+0000","last_fullsized":"2024-07-17T08:12:43.382402+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:46.256356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947664+0000","last_change":"2024-07-17T08:12:40.962461+0000","last_active":"2024-07-17T08:12:41.947664+0000","last_peered":"2024-07-17T08:12:41.947664+0000","last_clean":"2024-07-17T08:12:41.947664+0000","last_became_active":"2024-07-17T08:12:40.962135+0000","last_became_peered":"2024-07-17T08:12:40.962135+0000","last_unstale":"2024-07-17T08:12:41.947664+0000","last_undegraded":"2024-07-17T08:12:41.947664+0000","last_fullsized":"2024-07-17T08:12:41.947664+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:25.201683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954370+0000","last_change":"2024-07-17T08:12:40.980549+0000","last_active":"2024-07-17T08:12:41.954370+0000","last_peered":"2024-07-17T08:12:41.954370+0000","last_clean":"2024-07-17T08:12:41.954370+0000","last_became_active":"2024-07-17T08:12:40.980275+0000","last_became_peered":"2024-07-17T08:12:40.980275+0000","last_unstale":"2024-07-17T08:12:41.954370+0000","last_undegraded":"2024-07-17T08:12:41.954370+0000","last_fullsized":"2024-07-17T08:12:41.954370+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:58.143265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947582+0000","last_change":"2024-07-17T08:12:40.972560+0000","last_active":"2024-07-17T08:12:41.947582+0000","last_peered":"2024-07-17T08:12:41.947582+0000","last_clean":"2024-07-17T08:12:41.947582+0000","last_became_active":"2024-07-17T08:12:40.972264+0000","last_became_peered":"2024-07-17T08:12:40.972264+0000","last_unstale":"2024-07-17T08:12:41.947582+0000","last_undegraded":"2024-07-17T08:12:41.947582+0000","last_fullsized":"2024-07-17T08:12:41.947582+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:00.095637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381659+0000","last_change":"2024-07-17T08:12:40.977885+0000","last_active":"2024-07-17T08:12:43.381659+0000","last_peered":"2024-07-17T08:12:43.381659+0000","last_clean":"2024-07-17T08:12:43.381659+0000","last_became_active":"2024-07-17T08:12:40.977585+0000","last_became_peered":"2024-07-17T08:12:40.977585+0000","last_unstale":"2024-07-17T08:12:43.381659+0000","last_undegraded":"2024-07-17T08:12:43.381659+0000","last_fullsized":"2024-07-17T08:12:43.381659+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:43.367116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951899+0000","last_change":"2024-07-17T08:12:40.976239+0000","last_active":"2024-07-17T08:12:41.951899+0000","last_peered":"2024-07-17T08:12:41.951899+0000","last_clean":"2024-07-17T08:12:41.951899+0000","last_became_active":"2024-07-17T08:12:40.972980+0000","last_became_peered":"2024-07-17T08:12:40.972980+0000","last_unstale":"2024-07-17T08:12:41.951899+0000","last_undegraded":"2024-07-17T08:12:41.951899+0000","last_fullsized":"2024-07-17T08:12:41.951899+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:04.654259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368101+0000","last_change":"2024-07-17T08:12:40.978280+0000","last_active":"2024-07-17T08:12:43.368101+0000","last_peered":"2024-07-17T08:12:43.368101+0000","last_clean":"2024-07-17T08:12:43.368101+0000","last_became_active":"2024-07-17T08:12:40.976608+0000","last_became_peered":"2024-07-17T08:12:40.976608+0000","last_unstale":"2024-07-17T08:12:43.368101+0000","last_undegraded":"2024-07-17T08:12:43.368101+0000","last_fullsized":"2024-07-17T08:12:43.368101+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:44:24.265138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"173'4","reported_seq":18,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.383077+0000","last_change":"2024-07-17T08:12:39.968428+0000","last_active":"2024-07-17T08:12:43.383077+0000","last_peered":"2024-07-17T08:12:43.383077+0000","last_clean":"2024-07-17T08:12:43.383077+0000","last_became_active":"2024-07-17T08:12:39.967776+0000","last_became_peered":"2024-07-17T08:12:39.967776+0000","last_unstale":"2024-07-17T08:12:43.383077+0000","last_undegraded":"2024-07-17T08:12:43.383077+0000","last_fullsized":"2024-07-17T08:12:43.383077+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:09:48.246489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368706+0000","last_change":"2024-07-17T08:12:40.987069+0000","last_active":"2024-07-17T08:12:43.368706+0000","last_peered":"2024-07-17T08:12:43.368706+0000","last_clean":"2024-07-17T08:12:43.368706+0000","last_became_active":"2024-07-17T08:12:40.986749+0000","last_became_peered":"2024-07-17T08:12:40.986749+0000","last_unstale":"2024-07-17T08:12:43.368706+0000","last_undegraded":"2024-07-17T08:12:43.368706+0000","last_fullsized":"2024-07-17T08:12:43.368706+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:17.704894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062807+0000","last_change":"2024-07-17T08:12:39.980229+0000","last_active":"2024-07-17T08:12:42.062807+0000","last_peered":"2024-07-17T08:12:42.062807+0000","last_clean":"2024-07-17T08:12:42.062807+0000","last_became_active":"2024-07-17T08:12:39.979929+0000","last_became_peered":"2024-07-17T08:12:39.979929+0000","last_unstale":"2024-07-17T08:12:42.062807+0000","last_undegraded":"2024-07-17T08:12:42.062807+0000","last_fullsized":"2024-07-17T08:12:42.062807+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:49:44.198820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955804+0000","last_change":"2024-07-17T08:12:40.989072+0000","last_active":"2024-07-17T08:12:41.955804+0000","last_peered":"2024-07-17T08:12:41.955804+0000","last_clean":"2024-07-17T08:12:41.955804+0000","last_became_active":"2024-07-17T08:12:40.988783+0000","last_became_peered":"2024-07-17T08:12:40.988783+0000","last_unstale":"2024-07-17T08:12:41.955804+0000","last_undegraded":"2024-07-17T08:12:41.955804+0000","last_fullsized":"2024-07-17T08:12:41.955804+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:36.859840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063215+0000","last_change":"2024-07-17T08:12:39.981106+0000","last_active":"2024-07-17T08:12:42.063215+0000","last_peered":"2024-07-17T08:12:42.063215+0000","last_clean":"2024-07-17T08:12:42.063215+0000","last_became_active":"2024-07-17T08:12:39.980786+0000","last_became_peered":"2024-07-17T08:12:39.980786+0000","last_unstale":"2024-07-17T08:12:42.063215+0000","last_undegraded":"2024-07-17T08:12:42.063215+0000","last_fullsized":"2024-07-17T08:12:42.063215+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:30.528182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063195+0000","last_change":"2024-07-17T08:12:40.991340+0000","last_active":"2024-07-17T08:12:42.063195+0000","last_peered":"2024-07-17T08:12:42.063195+0000","last_clean":"2024-07-17T08:12:42.063195+0000","last_became_active":"2024-07-17T08:12:40.991067+0000","last_became_peered":"2024-07-17T08:12:40.991067+0000","last_unstale":"2024-07-17T08:12:42.063195+0000","last_undegraded":"2024-07-17T08:12:42.063195+0000","last_fullsized":"2024-07-17T08:12:42.063195+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:54.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368399+0000","last_change":"2024-07-17T08:12:39.976857+0000","last_active":"2024-07-17T08:12:43.368399+0000","last_peered":"2024-07-17T08:12:43.368399+0000","last_clean":"2024-07-17T08:12:43.368399+0000","last_became_active":"2024-07-17T08:12:39.976522+0000","last_became_peered":"2024-07-17T08:12:39.976522+0000","last_unstale":"2024-07-17T08:12:43.368399+0000","last_undegraded":"2024-07-17T08:12:43.368399+0000","last_fullsized":"2024-07-17T08:12:43.368399+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:51:34.216344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875835+0000","last_change":"2024-07-17T08:12:40.969061+0000","last_active":"2024-07-17T08:12:42.875835+0000","last_peered":"2024-07-17T08:12:42.875835+0000","last_clean":"2024-07-17T08:12:42.875835+0000","last_became_active":"2024-07-17T08:12:40.966503+0000","last_became_peered":"2024-07-17T08:12:40.966503+0000","last_unstale":"2024-07-17T08:12:42.875835+0000","last_undegraded":"2024-07-17T08:12:42.875835+0000","last_fullsized":"2024-07-17T08:12:42.875835+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:18.607103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954392+0000","last_change":"2024-07-17T08:12:39.978775+0000","last_active":"2024-07-17T08:12:41.954392+0000","last_peered":"2024-07-17T08:12:41.954392+0000","last_clean":"2024-07-17T08:12:41.954392+0000","last_became_active":"2024-07-17T08:12:39.978325+0000","last_became_peered":"2024-07-17T08:12:39.978325+0000","last_unstale":"2024-07-17T08:12:41.954392+0000","last_undegraded":"2024-07-17T08:12:41.954392+0000","last_fullsized":"2024-07-17T08:12:41.954392+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:43.478766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952617+0000","last_change":"2024-07-17T08:12:40.976416+0000","last_active":"2024-07-17T08:12:41.952617+0000","last_peered":"2024-07-17T08:12:41.952617+0000","last_clean":"2024-07-17T08:12:41.952617+0000","last_became_active":"2024-07-17T08:12:40.975684+0000","last_became_peered":"2024-07-17T08:12:40.975684+0000","last_unstale":"2024-07-17T08:12:41.952617+0000","last_undegraded":"2024-07-17T08:12:41.952617+0000","last_fullsized":"2024-07-17T08:12:41.952617+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:13.812536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382066+0000","last_change":"2024-07-17T08:12:39.969801+0000","last_active":"2024-07-17T08:12:43.382066+0000","last_peered":"2024-07-17T08:12:43.382066+0000","last_clean":"2024-07-17T08:12:43.382066+0000","last_became_active":"2024-07-17T08:12:39.969439+0000","last_became_peered":"2024-07-17T08:12:39.969439+0000","last_unstale":"2024-07-17T08:12:43.382066+0000","last_undegraded":"2024-07-17T08:12:43.382066+0000","last_fullsized":"2024-07-17T08:12:43.382066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:33.780901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368370+0000","last_change":"2024-07-17T08:12:40.972273+0000","last_active":"2024-07-17T08:12:43.368370+0000","last_peered":"2024-07-17T08:12:43.368370+0000","last_clean":"2024-07-17T08:12:43.368370+0000","last_became_active":"2024-07-17T08:12:40.971994+0000","last_became_peered":"2024-07-17T08:12:40.971994+0000","last_unstale":"2024-07-17T08:12:43.368370+0000","last_undegraded":"2024-07-17T08:12:43.368370+0000","last_fullsized":"2024-07-17T08:12:43.368370+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:14.066482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661741+0000","last_change":"2024-07-17T08:12:39.966501+0000","last_active":"2024-07-17T08:12:42.661741+0000","last_peered":"2024-07-17T08:12:42.661741+0000","last_clean":"2024-07-17T08:12:42.661741+0000","last_became_active":"2024-07-17T08:12:39.966175+0000","last_became_peered":"2024-07-17T08:12:39.966175+0000","last_unstale":"2024-07-17T08:12:42.661741+0000","last_undegraded":"2024-07-17T08:12:42.661741+0000","last_fullsized":"2024-07-17T08:12:42.661741+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:13.060339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661857+0000","last_change":"2024-07-17T08:12:40.979752+0000","last_active":"2024-07-17T08:12:42.661857+0000","last_peered":"2024-07-17T08:12:42.661857+0000","last_clean":"2024-07-17T08:12:42.661857+0000","last_became_active":"2024-07-17T08:12:40.976470+0000","last_became_peered":"2024-07-17T08:12:40.976470+0000","last_unstale":"2024-07-17T08:12:42.661857+0000","last_undegraded":"2024-07-17T08:12:42.661857+0000","last_fullsized":"2024-07-17T08:12:42.661857+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:17.220124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381318+0000","last_change":"2024-07-17T08:12:39.959993+0000","last_active":"2024-07-17T08:12:43.381318+0000","last_peered":"2024-07-17T08:12:43.381318+0000","last_clean":"2024-07-17T08:12:43.381318+0000","last_became_active":"2024-07-17T08:12:39.959636+0000","last_became_peered":"2024-07-17T08:12:39.959636+0000","last_unstale":"2024-07-17T08:12:43.381318+0000","last_undegraded":"2024-07-17T08:12:43.381318+0000","last_fullsized":"2024-07-17T08:12:43.381318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:49.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954544+0000","last_change":"2024-07-17T08:12:40.990145+0000","last_active":"2024-07-17T08:12:41.954544+0000","last_peered":"2024-07-17T08:12:41.954544+0000","last_clean":"2024-07-17T08:12:41.954544+0000","last_became_active":"2024-07-17T08:12:40.989839+0000","last_became_peered":"2024-07-17T08:12:40.989839+0000","last_unstale":"2024-07-17T08:12:41.954544+0000","last_undegraded":"2024-07-17T08:12:41.954544+0000","last_fullsized":"2024-07-17T08:12:41.954544+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:23.033075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063183+0000","last_change":"2024-07-17T08:12:39.979609+0000","last_active":"2024-07-17T08:12:42.063183+0000","last_peered":"2024-07-17T08:12:42.063183+0000","last_clean":"2024-07-17T08:12:42.063183+0000","last_became_active":"2024-07-17T08:12:39.979336+0000","last_became_peered":"2024-07-17T08:12:39.979336+0000","last_unstale":"2024-07-17T08:12:42.063183+0000","last_undegraded":"2024-07-17T08:12:42.063183+0000","last_fullsized":"2024-07-17T08:12:42.063183+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:17.605220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947963+0000","last_change":"2024-07-17T08:12:40.973490+0000","last_active":"2024-07-17T08:12:41.947963+0000","last_peered":"2024-07-17T08:12:41.947963+0000","last_clean":"2024-07-17T08:12:41.947963+0000","last_became_active":"2024-07-17T08:12:40.973222+0000","last_became_peered":"2024-07-17T08:12:40.973222+0000","last_unstale":"2024-07-17T08:12:41.947963+0000","last_undegraded":"2024-07-17T08:12:41.947963+0000","last_fullsized":"2024-07-17T08:12:41.947963+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:38.341914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662792+0000","last_change":"2024-07-17T08:12:39.971697+0000","last_active":"2024-07-17T08:12:42.662792+0000","last_peered":"2024-07-17T08:12:42.662792+0000","last_clean":"2024-07-17T08:12:42.662792+0000","last_became_active":"2024-07-17T08:12:39.971061+0000","last_became_peered":"2024-07-17T08:12:39.971061+0000","last_unstale":"2024-07-17T08:12:42.662792+0000","last_undegraded":"2024-07-17T08:12:42.662792+0000","last_fullsized":"2024-07-17T08:12:42.662792+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:17:15.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947920+0000","last_change":"2024-07-17T08:12:40.971297+0000","last_active":"2024-07-17T08:12:41.947920+0000","last_peered":"2024-07-17T08:12:41.947920+0000","last_clean":"2024-07-17T08:12:41.947920+0000","last_became_active":"2024-07-17T08:12:40.969652+0000","last_became_peered":"2024-07-17T08:12:40.969652+0000","last_unstale":"2024-07-17T08:12:41.947920+0000","last_undegraded":"2024-07-17T08:12:41.947920+0000","last_fullsized":"2024-07-17T08:12:41.947920+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:20.156544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"173'3","reported_seq":17,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.948112+0000","last_change":"2024-07-17T08:12:39.965917+0000","last_active":"2024-07-17T08:12:41.948112+0000","last_peered":"2024-07-17T08:12:41.948112+0000","last_clean":"2024-07-17T08:12:41.948112+0000","last_became_active":"2024-07-17T08:12:39.963869+0000","last_became_peered":"2024-07-17T08:12:39.963869+0000","last_unstale":"2024-07-17T08:12:41.948112+0000","last_undegraded":"2024-07-17T08:12:41.948112+0000","last_fullsized":"2024-07-17T08:12:41.948112+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:29.332747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.951948+0000","last_change":"2024-07-17T08:12:40.979069+0000","last_active":"2024-07-17T08:12:41.951948+0000","last_peered":"2024-07-17T08:12:41.951948+0000","last_clean":"2024-07-17T08:12:41.951948+0000","last_became_active":"2024-07-17T08:12:40.978341+0000","last_became_peered":"2024-07-17T08:12:40.978341+0000","last_unstale":"2024-07-17T08:12:41.951948+0000","last_undegraded":"2024-07-17T08:12:41.951948+0000","last_fullsized":"2024-07-17T08:12:41.951948+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:45.794369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382633+0000","last_change":"2024-07-17T08:12:39.965736+0000","last_active":"2024-07-17T08:12:43.382633+0000","last_peered":"2024-07-17T08:12:43.382633+0000","last_clean":"2024-07-17T08:12:43.382633+0000","last_became_active":"2024-07-17T08:12:39.964796+0000","last_became_peered":"2024-07-17T08:12:39.964796+0000","last_unstale":"2024-07-17T08:12:43.382633+0000","last_undegraded":"2024-07-17T08:12:43.382633+0000","last_fullsized":"2024-07-17T08:12:43.382633+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:26.826945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367723+0000","last_change":"2024-07-17T08:12:40.986769+0000","last_active":"2024-07-17T08:12:43.367723+0000","last_peered":"2024-07-17T08:12:43.367723+0000","last_clean":"2024-07-17T08:12:43.367723+0000","last_became_active":"2024-07-17T08:12:40.986483+0000","last_became_peered":"2024-07-17T08:12:40.986483+0000","last_unstale":"2024-07-17T08:12:43.367723+0000","last_undegraded":"2024-07-17T08:12:43.367723+0000","last_fullsized":"2024-07-17T08:12:43.367723+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:26.695183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.953477+0000","last_change":"2024-07-17T08:12:39.975115+0000","last_active":"2024-07-17T08:12:41.953477+0000","last_peered":"2024-07-17T08:12:41.953477+0000","last_clean":"2024-07-17T08:12:41.953477+0000","last_became_active":"2024-07-17T08:12:39.974804+0000","last_became_peered":"2024-07-17T08:12:39.974804+0000","last_unstale":"2024-07-17T08:12:41.953477+0000","last_undegraded":"2024-07-17T08:12:41.953477+0000","last_fullsized":"2024-07-17T08:12:41.953477+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:33.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.381899+0000","last_change":"2024-07-17T08:12:40.953908+0000","last_active":"2024-07-17T08:12:43.381899+0000","last_peered":"2024-07-17T08:12:43.381899+0000","last_clean":"2024-07-17T08:12:43.381899+0000","last_became_active":"2024-07-17T08:12:40.953536+0000","last_became_peered":"2024-07-17T08:12:40.953536+0000","last_unstale":"2024-07-17T08:12:43.381899+0000","last_undegraded":"2024-07-17T08:12:43.381899+0000","last_fullsized":"2024-07-17T08:12:43.381899+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:08.957400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952887+0000","last_change":"2024-07-17T08:12:39.971111+0000","last_active":"2024-07-17T08:12:41.952887+0000","last_peered":"2024-07-17T08:12:41.952887+0000","last_clean":"2024-07-17T08:12:41.952887+0000","last_became_active":"2024-07-17T08:12:39.969608+0000","last_became_peered":"2024-07-17T08:12:39.969608+0000","last_unstale":"2024-07-17T08:12:41.952887+0000","last_undegraded":"2024-07-17T08:12:41.952887+0000","last_fullsized":"2024-07-17T08:12:41.952887+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:30.667648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367826+0000","last_change":"2024-07-17T08:12:40.972863+0000","last_active":"2024-07-17T08:12:43.367826+0000","last_peered":"2024-07-17T08:12:43.367826+0000","last_clean":"2024-07-17T08:12:43.367826+0000","last_became_active":"2024-07-17T08:12:40.970381+0000","last_became_peered":"2024-07-17T08:12:40.970381+0000","last_unstale":"2024-07-17T08:12:43.367826+0000","last_undegraded":"2024-07-17T08:12:43.367826+0000","last_fullsized":"2024-07-17T08:12:43.367826+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:24.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"173'8","reported_seq":22,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.661801+0000","last_change":"2024-07-17T08:12:39.981503+0000","last_active":"2024-07-17T08:12:42.661801+0000","last_peered":"2024-07-17T08:12:42.661801+0000","last_clean":"2024-07-17T08:12:42.661801+0000","last_became_active":"2024-07-17T08:12:39.981126+0000","last_became_peered":"2024-07-17T08:12:39.981126+0000","last_unstale":"2024-07-17T08:12:42.661801+0000","last_undegraded":"2024-07-17T08:12:42.661801+0000","last_fullsized":"2024-07-17T08:12:42.661801+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:48.132769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062583+0000","last_change":"2024-07-17T08:12:40.986405+0000","last_active":"2024-07-17T08:12:42.062583+0000","last_peered":"2024-07-17T08:12:42.062583+0000","last_clean":"2024-07-17T08:12:42.062583+0000","last_became_active":"2024-07-17T08:12:40.985861+0000","last_became_peered":"2024-07-17T08:12:40.985861+0000","last_unstale":"2024-07-17T08:12:42.062583+0000","last_undegraded":"2024-07-17T08:12:42.062583+0000","last_fullsized":"2024-07-17T08:12:42.062583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:30.295306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947973+0000","last_change":"2024-07-17T08:12:39.966931+0000","last_active":"2024-07-17T08:12:41.947973+0000","last_peered":"2024-07-17T08:12:41.947973+0000","last_clean":"2024-07-17T08:12:41.947973+0000","last_became_active":"2024-07-17T08:12:39.966630+0000","last_became_peered":"2024-07-17T08:12:39.966630+0000","last_unstale":"2024-07-17T08:12:41.947973+0000","last_undegraded":"2024-07-17T08:12:41.947973+0000","last_fullsized":"2024-07-17T08:12:41.947973+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:32:26.581691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.063047+0000","last_change":"2024-07-17T08:12:40.994300+0000","last_active":"2024-07-17T08:12:42.063047+0000","last_peered":"2024-07-17T08:12:42.063047+0000","last_clean":"2024-07-17T08:12:42.063047+0000","last_became_active":"2024-07-17T08:12:40.994053+0000","last_became_peered":"2024-07-17T08:12:40.994053+0000","last_unstale":"2024-07-17T08:12:42.063047+0000","last_undegraded":"2024-07-17T08:12:42.063047+0000","last_fullsized":"2024-07-17T08:12:42.063047+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:50.343054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062985+0000","last_change":"2024-07-17T08:12:39.980185+0000","last_active":"2024-07-17T08:12:42.062985+0000","last_peered":"2024-07-17T08:12:42.062985+0000","last_clean":"2024-07-17T08:12:42.062985+0000","last_became_active":"2024-07-17T08:12:39.979010+0000","last_became_peered":"2024-07-17T08:12:39.979010+0000","last_unstale":"2024-07-17T08:12:42.062985+0000","last_undegraded":"2024-07-17T08:12:42.062985+0000","last_fullsized":"2024-07-17T08:12:42.062985+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:13.229833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662611+0000","last_change":"2024-07-17T08:12:40.968751+0000","last_active":"2024-07-17T08:12:42.662611+0000","last_peered":"2024-07-17T08:12:42.662611+0000","last_clean":"2024-07-17T08:12:42.662611+0000","last_became_active":"2024-07-17T08:12:40.966006+0000","last_became_peered":"2024-07-17T08:12:40.966006+0000","last_unstale":"2024-07-17T08:12:42.662611+0000","last_undegraded":"2024-07-17T08:12:42.662611+0000","last_fullsized":"2024-07-17T08:12:42.662611+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:50.432523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"173'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368945+0000","last_change":"2024-07-17T08:12:39.970173+0000","last_active":"2024-07-17T08:12:43.368945+0000","last_peered":"2024-07-17T08:12:43.368945+0000","last_clean":"2024-07-17T08:12:43.368945+0000","last_became_active":"2024-07-17T08:12:39.969804+0000","last_became_peered":"2024-07-17T08:12:39.969804+0000","last_unstale":"2024-07-17T08:12:43.368945+0000","last_undegraded":"2024-07-17T08:12:43.368945+0000","last_fullsized":"2024-07-17T08:12:43.368945+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:33.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662510+0000","last_change":"2024-07-17T08:12:40.980821+0000","last_active":"2024-07-17T08:12:42.662510+0000","last_peered":"2024-07-17T08:12:42.662510+0000","last_clean":"2024-07-17T08:12:42.662510+0000","last_became_active":"2024-07-17T08:12:40.978280+0000","last_became_peered":"2024-07-17T08:12:40.978280+0000","last_unstale":"2024-07-17T08:12:42.662510+0000","last_undegraded":"2024-07-17T08:12:42.662510+0000","last_fullsized":"2024-07-17T08:12:42.662510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:30:24.899223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954840+0000","last_change":"2024-07-17T08:12:39.978041+0000","last_active":"2024-07-17T08:12:41.954840+0000","last_peered":"2024-07-17T08:12:41.954840+0000","last_clean":"2024-07-17T08:12:41.954840+0000","last_became_active":"2024-07-17T08:12:39.977732+0000","last_became_peered":"2024-07-17T08:12:39.977732+0000","last_unstale":"2024-07-17T08:12:41.954840+0000","last_undegraded":"2024-07-17T08:12:41.954840+0000","last_fullsized":"2024-07-17T08:12:41.954840+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:15.428828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062281+0000","last_change":"2024-07-17T08:12:40.990242+0000","last_active":"2024-07-17T08:12:42.062281+0000","last_peered":"2024-07-17T08:12:42.062281+0000","last_clean":"2024-07-17T08:12:42.062281+0000","last_became_active":"2024-07-17T08:12:40.989971+0000","last_became_peered":"2024-07-17T08:12:40.989971+0000","last_unstale":"2024-07-17T08:12:42.062281+0000","last_undegraded":"2024-07-17T08:12:42.062281+0000","last_fullsized":"2024-07-17T08:12:42.062281+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:32:02.030798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875680+0000","last_change":"2024-07-17T08:12:39.978456+0000","last_active":"2024-07-17T08:12:42.875680+0000","last_peered":"2024-07-17T08:12:42.875680+0000","last_clean":"2024-07-17T08:12:42.875680+0000","last_became_active":"2024-07-17T08:12:39.978154+0000","last_became_peered":"2024-07-17T08:12:39.978154+0000","last_unstale":"2024-07-17T08:12:42.875680+0000","last_undegraded":"2024-07-17T08:12:42.875680+0000","last_fullsized":"2024-07-17T08:12:42.875680+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:56.336900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.367928+0000","last_change":"2024-07-17T08:12:40.977335+0000","last_active":"2024-07-17T08:12:43.367928+0000","last_peered":"2024-07-17T08:12:43.367928+0000","last_clean":"2024-07-17T08:12:43.367928+0000","last_became_active":"2024-07-17T08:12:40.975406+0000","last_became_peered":"2024-07-17T08:12:40.975406+0000","last_unstale":"2024-07-17T08:12:43.367928+0000","last_undegraded":"2024-07-17T08:12:43.367928+0000","last_fullsized":"2024-07-17T08:12:43.367928+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:57.189144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952396+0000","last_change":"2024-07-17T08:12:39.973638+0000","last_active":"2024-07-17T08:12:41.952396+0000","last_peered":"2024-07-17T08:12:41.952396+0000","last_clean":"2024-07-17T08:12:41.952396+0000","last_became_active":"2024-07-17T08:12:39.973137+0000","last_became_peered":"2024-07-17T08:12:39.973137+0000","last_unstale":"2024-07-17T08:12:41.952396+0000","last_undegraded":"2024-07-17T08:12:41.952396+0000","last_fullsized":"2024-07-17T08:12:41.952396+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:50:40.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.061970+0000","last_change":"2024-07-17T08:12:40.985007+0000","last_active":"2024-07-17T08:12:42.061970+0000","last_peered":"2024-07-17T08:12:42.061970+0000","last_clean":"2024-07-17T08:12:42.061970+0000","last_became_active":"2024-07-17T08:12:40.984730+0000","last_became_peered":"2024-07-17T08:12:40.984730+0000","last_unstale":"2024-07-17T08:12:42.061970+0000","last_undegraded":"2024-07-17T08:12:42.061970+0000","last_fullsized":"2024-07-17T08:12:42.061970+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:01.189730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875314+0000","last_change":"2024-07-17T08:12:39.969706+0000","last_active":"2024-07-17T08:12:42.875314+0000","last_peered":"2024-07-17T08:12:42.875314+0000","last_clean":"2024-07-17T08:12:42.875314+0000","last_became_active":"2024-07-17T08:12:39.969274+0000","last_became_peered":"2024-07-17T08:12:39.969274+0000","last_unstale":"2024-07-17T08:12:42.875314+0000","last_undegraded":"2024-07-17T08:12:42.875314+0000","last_fullsized":"2024-07-17T08:12:42.875314+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:36.963194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368244+0000","last_change":"2024-07-17T08:12:40.986353+0000","last_active":"2024-07-17T08:12:43.368244+0000","last_peered":"2024-07-17T08:12:43.368244+0000","last_clean":"2024-07-17T08:12:43.368244+0000","last_became_active":"2024-07-17T08:12:40.985975+0000","last_became_peered":"2024-07-17T08:12:40.985975+0000","last_unstale":"2024-07-17T08:12:43.368244+0000","last_undegraded":"2024-07-17T08:12:43.368244+0000","last_fullsized":"2024-07-17T08:12:43.368244+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:49.138913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952238+0000","last_change":"2024-07-17T08:12:39.973183+0000","last_active":"2024-07-17T08:12:41.952238+0000","last_peered":"2024-07-17T08:12:41.952238+0000","last_clean":"2024-07-17T08:12:41.952238+0000","last_became_active":"2024-07-17T08:12:39.972882+0000","last_became_peered":"2024-07-17T08:12:39.972882+0000","last_unstale":"2024-07-17T08:12:41.952238+0000","last_undegraded":"2024-07-17T08:12:41.952238+0000","last_fullsized":"2024-07-17T08:12:41.952238+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:48.284682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382577+0000","last_change":"2024-07-17T08:12:40.967497+0000","last_active":"2024-07-17T08:12:43.382577+0000","last_peered":"2024-07-17T08:12:43.382577+0000","last_clean":"2024-07-17T08:12:43.382577+0000","last_became_active":"2024-07-17T08:12:40.967137+0000","last_became_peered":"2024-07-17T08:12:40.967137+0000","last_unstale":"2024-07-17T08:12:43.382577+0000","last_undegraded":"2024-07-17T08:12:43.382577+0000","last_fullsized":"2024-07-17T08:12:43.382577+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:37.969337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"173'2","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955079+0000","last_change":"2024-07-17T08:12:39.975659+0000","last_active":"2024-07-17T08:12:41.955079+0000","last_peered":"2024-07-17T08:12:41.955079+0000","last_clean":"2024-07-17T08:12:41.955079+0000","last_became_active":"2024-07-17T08:12:39.975352+0000","last_became_peered":"2024-07-17T08:12:39.975352+0000","last_unstale":"2024-07-17T08:12:41.955079+0000","last_undegraded":"2024-07-17T08:12:41.955079+0000","last_fullsized":"2024-07-17T08:12:41.955079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:05.504874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.954878+0000","last_change":"2024-07-17T08:12:40.979905+0000","last_active":"2024-07-17T08:12:41.954878+0000","last_peered":"2024-07-17T08:12:41.954878+0000","last_clean":"2024-07-17T08:12:41.954878+0000","last_became_active":"2024-07-17T08:12:40.979619+0000","last_became_peered":"2024-07-17T08:12:40.979619+0000","last_unstale":"2024-07-17T08:12:41.954878+0000","last_undegraded":"2024-07-17T08:12:41.954878+0000","last_fullsized":"2024-07-17T08:12:41.954878+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:23:24.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"174'1","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.959944+0000","last_change":"2024-07-17T08:12:39.971941+0000","last_active":"2024-07-17T08:12:42.959944+0000","last_peered":"2024-07-17T08:12:42.959944+0000","last_clean":"2024-07-17T08:12:42.959944+0000","last_became_active":"2024-07-17T08:12:39.971410+0000","last_became_peered":"2024-07-17T08:12:39.971410+0000","last_unstale":"2024-07-17T08:12:42.959944+0000","last_undegraded":"2024-07-17T08:12:42.959944+0000","last_fullsized":"2024-07-17T08:12:42.959944+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:47.400806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662704+0000","last_change":"2024-07-17T08:12:40.981113+0000","last_active":"2024-07-17T08:12:42.662704+0000","last_peered":"2024-07-17T08:12:42.662704+0000","last_clean":"2024-07-17T08:12:42.662704+0000","last_became_active":"2024-07-17T08:12:40.980833+0000","last_became_peered":"2024-07-17T08:12:40.980833+0000","last_unstale":"2024-07-17T08:12:42.662704+0000","last_undegraded":"2024-07-17T08:12:42.662704+0000","last_fullsized":"2024-07-17T08:12:42.662704+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:27.833088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368844+0000","last_change":"2024-07-17T08:12:39.974518+0000","last_active":"2024-07-17T08:12:43.368844+0000","last_peered":"2024-07-17T08:12:43.368844+0000","last_clean":"2024-07-17T08:12:43.368844+0000","last_became_active":"2024-07-17T08:12:39.974238+0000","last_became_peered":"2024-07-17T08:12:39.974238+0000","last_unstale":"2024-07-17T08:12:43.368844+0000","last_undegraded":"2024-07-17T08:12:43.368844+0000","last_fullsized":"2024-07-17T08:12:43.368844+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:30.667648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382911+0000","last_change":"2024-07-17T08:12:40.983976+0000","last_active":"2024-07-17T08:12:43.382911+0000","last_peered":"2024-07-17T08:12:43.382911+0000","last_clean":"2024-07-17T08:12:43.382911+0000","last_became_active":"2024-07-17T08:12:40.983512+0000","last_became_peered":"2024-07-17T08:12:40.983512+0000","last_unstale":"2024-07-17T08:12:43.382911+0000","last_undegraded":"2024-07-17T08:12:43.382911+0000","last_fullsized":"2024-07-17T08:12:43.382911+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:24:27.937363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.382461+0000","last_change":"2024-07-17T08:12:39.978198+0000","last_active":"2024-07-17T08:12:43.382461+0000","last_peered":"2024-07-17T08:12:43.382461+0000","last_clean":"2024-07-17T08:12:43.382461+0000","last_became_active":"2024-07-17T08:12:39.977463+0000","last_became_peered":"2024-07-17T08:12:39.977463+0000","last_unstale":"2024-07-17T08:12:43.382461+0000","last_undegraded":"2024-07-17T08:12:43.382461+0000","last_fullsized":"2024-07-17T08:12:43.382461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:10.667095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952320+0000","last_change":"2024-07-17T08:12:40.978609+0000","last_active":"2024-07-17T08:12:41.952320+0000","last_peered":"2024-07-17T08:12:41.952320+0000","last_clean":"2024-07-17T08:12:41.952320+0000","last_became_active":"2024-07-17T08:12:40.976699+0000","last_became_peered":"2024-07-17T08:12:40.976699+0000","last_unstale":"2024-07-17T08:12:41.952320+0000","last_undegraded":"2024-07-17T08:12:41.952320+0000","last_fullsized":"2024-07-17T08:12:41.952320+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:11:27.950859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662143+0000","last_change":"2024-07-17T08:12:39.969612+0000","last_active":"2024-07-17T08:12:42.662143+0000","last_peered":"2024-07-17T08:12:42.662143+0000","last_clean":"2024-07-17T08:12:42.662143+0000","last_became_active":"2024-07-17T08:12:39.969335+0000","last_became_peered":"2024-07-17T08:12:39.969335+0000","last_unstale":"2024-07-17T08:12:42.662143+0000","last_undegraded":"2024-07-17T08:12:42.662143+0000","last_fullsized":"2024-07-17T08:12:42.662143+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:16.698750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952984+0000","last_change":"2024-07-17T08:12:40.982802+0000","last_active":"2024-07-17T08:12:41.952984+0000","last_peered":"2024-07-17T08:12:41.952984+0000","last_clean":"2024-07-17T08:12:41.952984+0000","last_became_active":"2024-07-17T08:12:40.976004+0000","last_became_peered":"2024-07-17T08:12:40.976004+0000","last_unstale":"2024-07-17T08:12:41.952984+0000","last_undegraded":"2024-07-17T08:12:41.952984+0000","last_fullsized":"2024-07-17T08:12:41.952984+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:45:00.464498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.947795+0000","last_change":"2024-07-17T08:12:39.973111+0000","last_active":"2024-07-17T08:12:41.947795+0000","last_peered":"2024-07-17T08:12:41.947795+0000","last_clean":"2024-07-17T08:12:41.947795+0000","last_became_active":"2024-07-17T08:12:39.972828+0000","last_became_peered":"2024-07-17T08:12:39.972828+0000","last_unstale":"2024-07-17T08:12:41.947795+0000","last_undegraded":"2024-07-17T08:12:41.947795+0000","last_fullsized":"2024-07-17T08:12:41.947795+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:57:16.640714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:43.368468+0000","last_change":"2024-07-17T08:12:40.971677+0000","last_active":"2024-07-17T08:12:43.368468+0000","last_peered":"2024-07-17T08:12:43.368468+0000","last_clean":"2024-07-17T08:12:43.368468+0000","last_became_active":"2024-07-17T08:12:40.971383+0000","last_became_peered":"2024-07-17T08:12:40.971383+0000","last_unstale":"2024-07-17T08:12:43.368468+0000","last_undegraded":"2024-07-17T08:12:43.368468+0000","last_fullsized":"2024-07-17T08:12:43.368468+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:57.737087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.662018+0000","last_change":"2024-07-17T08:12:39.968980+0000","last_active":"2024-07-17T08:12:42.662018+0000","last_peered":"2024-07-17T08:12:42.662018+0000","last_clean":"2024-07-17T08:12:42.662018+0000","last_became_active":"2024-07-17T08:12:39.968644+0000","last_became_peered":"2024-07-17T08:12:39.968644+0000","last_unstale":"2024-07-17T08:12:42.662018+0000","last_undegraded":"2024-07-17T08:12:42.662018+0000","last_fullsized":"2024-07-17T08:12:42.662018+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:56.730943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.952423+0000","last_change":"2024-07-17T08:12:40.980487+0000","last_active":"2024-07-17T08:12:41.952423+0000","last_peered":"2024-07-17T08:12:41.952423+0000","last_clean":"2024-07-17T08:12:41.952423+0000","last_became_active":"2024-07-17T08:12:40.979907+0000","last_became_peered":"2024-07-17T08:12:40.979907+0000","last_unstale":"2024-07-17T08:12:41.952423+0000","last_undegraded":"2024-07-17T08:12:41.952423+0000","last_fullsized":"2024-07-17T08:12:41.952423+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:31:50.638303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062109+0000","last_change":"2024-07-17T08:12:39.977423+0000","last_active":"2024-07-17T08:12:42.062109+0000","last_peered":"2024-07-17T08:12:42.062109+0000","last_clean":"2024-07-17T08:12:42.062109+0000","last_became_active":"2024-07-17T08:12:39.977037+0000","last_became_peered":"2024-07-17T08:12:39.977037+0000","last_unstale":"2024-07-17T08:12:42.062109+0000","last_undegraded":"2024-07-17T08:12:42.062109+0000","last_fullsized":"2024-07-17T08:12:42.062109+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:26:47.332136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:41.955942+0000","last_change":"2024-07-17T08:12:40.973812+0000","last_active":"2024-07-17T08:12:41.955942+0000","last_peered":"2024-07-17T08:12:41.955942+0000","last_clean":"2024-07-17T08:12:41.955942+0000","last_became_active":"2024-07-17T08:12:40.973112+0000","last_became_peered":"2024-07-17T08:12:40.973112+0000","last_unstale":"2024-07-17T08:12:41.955942+0000","last_undegraded":"2024-07-17T08:12:41.955942+0000","last_fullsized":"2024-07-17T08:12:41.955942+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:39.921168+0000","last_clean_scrub_stamp":"2024-07-17T08:12:39.921168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:34.853668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"173'6","reported_seq":20,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.875772+0000","last_change":"2024-07-17T08:12:39.976810+0000","last_active":"2024-07-17T08:12:42.875772+0000","last_peered":"2024-07-17T08:12:42.875772+0000","last_clean":"2024-07-17T08:12:42.875772+0000","last_became_active":"2024-07-17T08:12:39.976078+0000","last_became_peered":"2024-07-17T08:12:39.976078+0000","last_unstale":"2024-07-17T08:12:42.875772+0000","last_undegraded":"2024-07-17T08:12:42.875772+0000","last_fullsized":"2024-07-17T08:12:42.875772+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:12:38.915025+0000","last_clean_scrub_stamp":"2024-07-17T08:12:38.915025+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:38.594753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"164'2462","reported_seq":3105,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-07-17T08:12:42.062250+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:12:42.062250+0000","last_peered":"2024-07-17T08:12:42.062250+0000","last_clean":"2024-07-17T08:12:42.062250+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:12:42.062250+0000","last_undegraded":"2024-07-17T08:12:42.062250+0000","last_fullsized":"2024-07-17T08:12:42.062250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2462,"log_dups_size":0,"ondisk_log_size":2462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":275,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":967,"num_read_kb":2639,"num_write":275,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2462,"ondisk_log_size":2462,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543073,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60248,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311592,"statfs":{"total":96636764160,"available":96575070208,"internally_reserved":0,"allocated":2764800,"data_stored":1755444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510368,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41624,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330216,"statfs":{"total":96636764160,"available":96594141184,"internally_reserved":0,"allocated":2170880,"data_stored":1165124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510369,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47184,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324656,"statfs":{"total":96636764160,"available":96588447744,"internally_reserved":0,"allocated":2166784,"data_stored":1165054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543074,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41636,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330204,"statfs":{"total":96636764160,"available":96594128896,"internally_reserved":0,"allocated":2179072,"data_stored":1166666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510368,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59668,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312172,"statfs":{"total":96636764160,"available":96575664128,"internally_reserved":0,"allocated":2162688,"data_stored":1158430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543072,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47192,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324648,"statfs":{"total":96636764160,"available":96588439552,"internally_reserved":0,"allocated":2174976,"data_stored":1165166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41612,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330228,"statfs":{"total":96636764160,"available":96594153472,"internally_reserved":0,"allocated":2158592,"data_stored":1157884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510368,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1750500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:12:45.992 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:12:45.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:12:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 -- 172.21.15.44:0/331507813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 msgr2=0x7fb7280d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 --2- 172.21.15.44:0/331507813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7280d1e30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb720005fd0 tx=0x7fb72002a820 comp rx=0 tx=0).stop 2024-07-17T08:12:46.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 -- 172.21.15.44:0/331507813 shutdown_connections 2024-07-17T08:12:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 --2- 172.21.15.44:0/331507813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7280d1e30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 --2- 172.21.15.44:0/331507813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 0x7fb7280d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 -- 172.21.15.44:0/331507813 >> 172.21.15.44:0/331507813 conn(0x7fb7280cfec0 msgr2=0x7fb7280d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 -- 172.21.15.44:0/331507813 shutdown_connections 2024-07-17T08:12:46.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.255+0000 7fb72de00640 1 -- 172.21.15.44:0/331507813 wait complete. 2024-07-17T08:12:46.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.257+0000 7fb72de00640 1 Processor -- start 2024-07-17T08:12:46.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.257+0000 7fb72de00640 1 -- start start 2024-07-17T08:12:46.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 0x7fb72815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7281f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280d1220 con 0x7fb72815d300 2024-07-17T08:12:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280d2370 con 0x7fb7280d2fa0 2024-07-17T08:12:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7281f5020 con 0x7fb7280b55d0 2024-07-17T08:12:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48614/0 (socket says 172.21.15.44:48614) 2024-07-17T08:12:46.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.258+0000 7fb72d400640 1 -- 172.21.15.44:0/2734952098 learned_addr learned my addr 172.21.15.44:0/2734952098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb727e00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7281f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72ca00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 0x7fb72815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 -- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 msgr2=0x7fb72815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 0x7fb72815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 -- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 msgr2=0x7fb7281f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7281f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 -- 172.21.15.44:0/2734952098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281f4a60 con 0x7fb72815d300 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.259+0000 7fb72d400640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb720009b30 tx=0x7fb720033ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.260+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720037050 con 0x7fb72815d300 2024-07-17T08:12:46.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.260+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb720043070 con 0x7fb72815d300 2024-07-17T08:12:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.260+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200044b0 con 0x7fb72815d300 2024-07-17T08:12:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.260+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7281f4d40 con 0x7fb72815d300 2024-07-17T08:12:46.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.260+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72815dd40 con 0x7fb72815d300 2024-07-17T08:12:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.261+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7280d2850 con 0x7fb72815d300 2024-07-17T08:12:46.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.262+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fb720004650 con 0x7fb72815d300 2024-07-17T08:12:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.264+0000 7fb724c00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 0x7fb708079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.264+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7fb7200b9760 con 0x7fb72815d300 2024-07-17T08:12:46.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.265+0000 7fb72ca00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 0x7fb708079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.266+0000 7fb72ca00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 0x7fb708079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0015e0 tx=0x7fb71c009040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.268+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb7200827e0 con 0x7fb72815d300 2024-07-17T08:12:46.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.563+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb7280b93c0 con 0x7fb72815d300 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.568+0000 7fb724c00640 1 -- 172.21.15.44:0/2734952098 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v61) ==== 190+0+147 (secure 0 0 0) 0x7fb7200820e0 con 0x7fb72815d300 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:12:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:12:46.570 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:12:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 msgr2=0x7fb708079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 0x7fb708079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0015e0 tx=0x7fb71c009040 comp rx=0 tx=0).stop 2024-07-17T08:12:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 msgr2=0x7fb7280ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb720009b30 tx=0x7fb720033ca0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 shutdown_connections 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fb708076bb0 0x7fb708079070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb72815d300 0x7fb7280ca510 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7280d2fa0 0x7fb7281f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 --2- 172.21.15.44:0/2734952098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7280b55d0 0x7fb72815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 >> 172.21.15.44:0/2734952098 conn(0x7fb7280cfec0 msgr2=0x7fb7280b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 shutdown_connections 2024-07-17T08:12:46.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.571+0000 7fb72de00640 1 -- 172.21.15.44:0/2734952098 wait complete. 2024-07-17T08:12:46.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 -- 172.21.15.44:0/916251077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 msgr2=0x7f0ca8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/916251077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0ca8004c80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000bb20 tx=0x7f0ca8005830 comp rx=0 tx=0).stop 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3400640 1 -- 172.21.15.44:0/916251077 reap_dead start 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 -- 172.21.15.44:0/916251077 shutdown_connections 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 -- 172.21.15.44:0/916251077 >> 172.21.15.44:0/916251077 conn(0x7f0cc40b0e80 msgr2=0x7f0cc40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 -- 172.21.15.44:0/916251077 shutdown_connections 2024-07-17T08:12:46.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.794+0000 7f0cc3e00640 1 -- 172.21.15.44:0/916251077 wait complete. 2024-07-17T08:12:46.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.796+0000 7f0cc3e00640 1 Processor -- start 2024-07-17T08:12:46.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.796+0000 7f0cc3e00640 1 -- start start 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 0x7f0cc41f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc4073ac0 0x7f0cc4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc4140030 con 0x7f0ca8004800 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc40ac270 con 0x7f0cc4073640 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc41401b0 con 0x7f0cc4073ac0 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48642/0 (socket says 172.21.15.44:48642) 2024-07-17T08:12:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.797+0000 7f0cc2a00640 1 -- 172.21.15.44:0/3018027473 learned_addr learned my addr 172.21.15.44:0/3018027473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2000640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 0x7f0cc41f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc3400640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc4073ac0 0x7f0cc4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 -- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc4073ac0 msgr2=0x7f0cc4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc4073ac0 0x7f0cc4157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 -- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 msgr2=0x7f0cc41f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 0x7f0cc41f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 -- 172.21.15.44:0/3018027473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc41f4360 con 0x7f0ca8004800 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2000640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 0x7f0cc41f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.798+0000 7f0cc2a00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400d8a0 tx=0x7f0cb400dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.799+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400b830 con 0x7f0ca8004800 2024-07-17T08:12:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.799+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cb400bac0 con 0x7f0ca8004800 2024-07-17T08:12:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.799+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40035f0 con 0x7f0ca8004800 2024-07-17T08:12:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.799+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc41f8410 con 0x7f0ca8004800 2024-07-17T08:12:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.799+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc41f45e0 con 0x7f0ca8004800 2024-07-17T08:12:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.800+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc41411f0 con 0x7f0ca8004800 2024-07-17T08:12:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.801+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f0cb4003790 con 0x7f0ca8004800 2024-07-17T08:12:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.803+0000 7f0cbaa00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 0x7f0ca40790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.803+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f0cb4098ce0 con 0x7f0ca8004800 2024-07-17T08:12:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.803+0000 7f0cc2000640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 0x7f0ca40790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.805+0000 7f0cc2000640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 0x7f0ca40790d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0036000 tx=0x7f0cb00046c0 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:46.807+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0cc41f4360 con 0x7f0ca8004800 2024-07-17T08:12:46.899 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T08:12:46.897+0000 7fb652a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:12:47.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.058+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cc40ce2a0 con 0x7f0ca8004800 2024-07-17T08:12:47.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.059+0000 7f0cbaa00640 1 -- 172.21.15.44:0/3018027473 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7f0cb4061d60 con 0x7f0ca8004800 2024-07-17T08:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:12:47.060 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:12:47.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.061+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 msgr2=0x7f0ca40790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.061+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 0x7f0ca40790d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0036000 tx=0x7f0cb00046c0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.061+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 msgr2=0x7f0cc41f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.061+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400d8a0 tx=0x7f0cb400dd60 comp rx=0 tx=0).stop 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 shutdown_connections 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f0ca4076c10 0x7f0ca40790d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc4073ac0 0x7f0cc4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc4073640 0x7f0cc41f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 --2- 172.21.15.44:0/3018027473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ca8004800 0x7f0cc41f4de0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 >> 172.21.15.44:0/3018027473 conn(0x7f0cc40b0e80 msgr2=0x7f0cc409c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 shutdown_connections 2024-07-17T08:12:47.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.062+0000 7f0cc3e00640 1 -- 172.21.15.44:0/3018027473 wait complete. 2024-07-17T08:12:47.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:12:40.436688+0000', 'modified': '2024-07-17T08:12:43.492715+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7258}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7258': {'gid': 7258, 'name': 'b', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6836/808460181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 808460181}, {'type': 'v1', 'addr': '172.21.15.175:6836', 'nonce': 808460181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7258, 'qdb_cluster': [7258]} 2024-07-17T08:12:47.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:12:47.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 -- 172.21.15.44:0/3472194804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a80cd0b0 msgr2=0x7f83a806ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 --2- 172.21.15.44:0/3472194804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a80cd0b0 0x7f83a806ec70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f8394005fd0 tx=0x7f839402a4a0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 -- 172.21.15.44:0/3472194804 shutdown_connections 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 --2- 172.21.15.44:0/3472194804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a80cd0b0 0x7f83a806ec70 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 --2- 172.21.15.44:0/3472194804 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8151f20 0x7f83a815c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 -- 172.21.15.44:0/3472194804 >> 172.21.15.44:0/3472194804 conn(0x7f83a80cf0d0 msgr2=0x7f83a80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 -- 172.21.15.44:0/3472194804 shutdown_connections 2024-07-17T08:12:47.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.712+0000 7f83af000640 1 -- 172.21.15.44:0/3472194804 wait complete. 2024-07-17T08:12:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.714+0000 7f83af000640 1 Processor -- start 2024-07-17T08:12:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.714+0000 7f83af000640 1 -- start start 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83a80cd0b0 0x7f83a8099190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8152300 0x7f83a81f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80cb790 con 0x7f83a8151f20 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83a806f550 con 0x7f83a80cd0b0 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83af000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83a80cb910 con 0x7f83a8152300 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48664/0 (socket says 172.21.15.44:48664) 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 -- 172.21.15.44:0/3244107119 learned_addr learned my addr 172.21.15.44:0/3244107119 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83adc00640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83a80cd0b0 0x7f83a8099190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ae600640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8152300 0x7f83a81f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 -- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8152300 msgr2=0x7f83a81f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8152300 0x7f83a81f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 -- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83a80cd0b0 msgr2=0x7f83a8099190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.716+0000 7f83ad200640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83a80cd0b0 0x7f83a8099190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83ad200640 1 -- 172.21.15.44:0/3244107119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81f4db0 con 0x7f83a8151f20 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83ad200640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8394009850 tx=0x7f83940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83a5600640 1 -- 172.21.15.44:0/3244107119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839403f070 con 0x7f83a8151f20 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83a5600640 1 -- 172.21.15.44:0/3244107119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8394004890 con 0x7f83a8151f20 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83a5600640 1 -- 172.21.15.44:0/3244107119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394035030 con 0x7f83a8151f20 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81f4ad0 con 0x7f83a8151f20 2024-07-17T08:12:47.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.717+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81514e0 con 0x7f83a8151f20 2024-07-17T08:12:47.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.720+0000 7f83a5600640 1 -- 172.21.15.44:0/3244107119 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f8394042070 con 0x7f83a8151f20 2024-07-17T08:12:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.721+0000 7f83a5600640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 0x7f83880790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.721+0000 7f83a5600640 1 -- 172.21.15.44:0/3244107119 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f83940b94f0 con 0x7f83a8151f20 2024-07-17T08:12:47.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.721+0000 7f83adc00640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 0x7f83880790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.721+0000 7f83b3432740 1 Processor -- start 2024-07-17T08:12:47.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.722+0000 7f83b3432740 1 -- start start 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83adc00640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 0x7f83880790a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f839c009890 tx=0x7f839c008040 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9177fc0 0x5614b91783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 0x5614b9174250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5614b90bfd60 con 0x5614b9174790 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5614b90af260 con 0x5614b914f280 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5614b90b32a0 con 0x5614b9177fc0 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83ae600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83ae600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48666/0 (socket says 172.21.15.44:48666) 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83ae600640 1 -- 172.21.15.44:0/3103694746 learned_addr learned my addr 172.21.15.44:0/3103694746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83ad200640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 0x5614b9174250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.723+0000 7f83adc00640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9177fc0 0x5614b91783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 -- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9177fc0 msgr2=0x5614b91783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9177fc0 0x5614b91783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 -- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 msgr2=0x5614b9174250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 0x5614b9174250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 -- 172.21.15.44:0/3103694746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b8e13850 con 0x5614b9174790 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ad200640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 0x5614b9174250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f83ae600640 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009f30 tx=0x7f83a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddefbe99d5ea478 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f839a000640 1 -- 172.21.15.44:0/3103694746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a001a070 con 0x5614b9174790 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f839a000640 1 -- 172.21.15.44:0/3103694746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83a0015410 con 0x5614b9174790 2024-07-17T08:12:47.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.724+0000 7f839a000640 1 -- 172.21.15.44:0/3103694746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0015800 con 0x5614b9174790 2024-07-17T08:12:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 -- 172.21.15.44:0/3103694746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 msgr2=0x5614b9174c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009f30 tx=0x7f83a000c8b0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 -- 172.21.15.44:0/3103694746 shutdown_connections 2024-07-17T08:12:47.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9174790 0x5614b9174c40 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b914f280 0x5614b9174250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 --2- 172.21.15.44:0/3103694746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9177fc0 0x5614b91783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 -- 172.21.15.44:0/3103694746 >> 172.21.15.44:0/3103694746 conn(0x5614b914a460 msgr2=0x5614b914de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 -- 172.21.15.44:0/3103694746 shutdown_connections 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.725+0000 7f83b3432740 1 -- 172.21.15.44:0/3103694746 wait complete. 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.726+0000 7f83b3432740 1 -- start start 2024-07-17T08:12:47.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.726+0000 7f83b3432740 1 Processor -- start 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b91722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b916d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5614b8e22620 con 0x5614b9177fc0 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5614b9056f90 con 0x5614b9174790 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5614b9148b60 con 0x5614b91a3720 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.727+0000 7f83b3432740 10 client.0 _ll_register_callbacks cb 0x5614b907cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:12:47.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48678/0 (socket says 172.21.15.44:48678) 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 -- 172.21.15.44:0/2651505085 learned_addr learned my addr 172.21.15.44:0/2651505085 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83adc00640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b91722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 -- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 msgr2=0x5614b916d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b916d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 -- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 msgr2=0x5614b91722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b91722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 -- 172.21.15.44:0/2651505085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b8e8a6b0 con 0x5614b9177fc0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f83ad200640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f839407df80 tx=0x7f8394081e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426e4ada24307bc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f838ea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.728+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839409b020 con 0x5614b9177fc0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f839408aa90 con 0x5614b9177fc0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5614b9086900 con 0x5614b9177fc0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8394049070 con 0x5614b9177fc0 2024-07-17T08:12:47.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 10 client.8895 fetch_fsmap learned FSMap version 88 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5614b9172800 con 0x5614b9177fc0 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5614b9086900 con 0x5614b9177fc0 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7f83940b70d0 con 0x5614b9177fc0 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 10 client.8895 fetch_fsmap finished waiting for FSMap version 88 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 10 client.8895 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 20 client.8895 populate_metadata read hostname 'smithi044' 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b915db90 0x5614b915df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 4 client.8895 mds_command: new command op to 7258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.729+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 --> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b90c1cb0 con 0x5614b915db90 2024-07-17T08:12:47.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.730+0000 7f83adc00640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b915db90 0x5614b915df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.731+0000 7f83adc00640 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b915db90 0x5614b915df70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff9222cda82e4d server_cookie=3ada76b6cde6b5a in_seq=0 out_seq=0 2024-07-17T08:12:47.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.731+0000 7f838ea00640 10 client.8895 ms_handle_connect on v2:172.21.15.175:6801/808460181 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f838ea00640 1 -- 172.21.15.44:0/2651505085 <== mds.0 v2:172.21.15.175:6801/808460181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5614b90c1cb0 con 0x5614b915db90 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f838ea00640 10 client.8895 handle_command_reply: tid=0 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 client.8895 shutdown 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 msgr2=0x5614b91a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f839407df80 tx=0x7f8394081e00 comp rx=0 tx=0).stop 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 shutdown_connections 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b915db90 0x5614b915df70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b916d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a6320 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 --2- 172.21.15.44:0/2651505085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b91722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f838ea00640 0 client.8895 ms_handle_reset on v2:172.21.15.175:6801/808460181 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 >> 172.21.15.44:0/2651505085 conn(0x5614b914f280 msgr2=0x5614b91783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 shutdown_connections 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.733+0000 7f83b3432740 1 -- 172.21.15.44:0/2651505085 wait complete. 2024-07-17T08:12:47.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.734+0000 7f83b3432740 20 client.8895 trim_cache size 0 max 16384 2024-07-17T08:12:47.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.755+0000 7f83b3432740 1 Processor -- start 2024-07-17T08:12:47.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 -- start start 2024-07-17T08:12:47.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b916d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b91a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8200550 con 0x5614b9177fc0 2024-07-17T08:12:47.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ff8b0 con 0x5614b9174790 2024-07-17T08:12:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.756+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83a81fff00 con 0x5614b91a3720 2024-07-17T08:12:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48686/0 (socket says 172.21.15.44:48686) 2024-07-17T08:12:47.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 -- 172.21.15.44:0/3836217551 learned_addr learned my addr 172.21.15.44:0/3836217551 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83adc00640 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b916d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 -- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 msgr2=0x5614b91a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b91a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 -- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 msgr2=0x5614b916d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b916d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83ad200640 1 -- 172.21.15.44:0/3836217551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b91738d0 con 0x5614b9177fc0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.757+0000 7f83adc00640 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b916d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f83ad200640 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f839407f170 tx=0x7f8394005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f4fb10b54cab47 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f8399600640 1 -- 172.21.15.44:0/3836217551 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839409b040 con 0x5614b9177fc0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f8399600640 1 -- 172.21.15.44:0/3836217551 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8394086500 con 0x5614b9177fc0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f8399600640 1 -- 172.21.15.44:0/3836217551 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839408af20 con 0x5614b9177fc0 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f83b3432740 1 -- 172.21.15.44:0/3836217551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 msgr2=0x5614b91a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.758+0000 7f83b3432740 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f839407f170 tx=0x7f8394005a40 comp rx=0 tx=0).stop 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 -- 172.21.15.44:0/3836217551 shutdown_connections 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b91a3720 0x5614b91a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b9177fc0 0x5614b91a2100 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 --2- 172.21.15.44:0/3836217551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9174790 0x5614b916d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 -- 172.21.15.44:0/3836217551 >> 172.21.15.44:0/3836217551 conn(0x5614b914f280 msgr2=0x5614b91b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 -- 172.21.15.44:0/3836217551 shutdown_connections 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 -- 172.21.15.44:0/3836217551 wait complete. 2024-07-17T08:12:47.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.759+0000 7f83b3432740 1 -- start start 2024-07-17T08:12:47.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.760+0000 7f83b3432740 1 Processor -- start 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9174790 0x5614b920df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9177fc0 0x5614b9195700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5614b91a6d70 con 0x5614b91a3720 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5614b8e22620 con 0x5614b9177fc0 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5614b916e010 con 0x5614b9174790 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83b3432740 10 client.0 _ll_register_callbacks cb 0x7f8394005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48700/0 (socket says 172.21.15.44:48700) 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 -- 172.21.15.44:0/643206749 learned_addr learned my addr 172.21.15.44:0/643206749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83adc00640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9174790 0x5614b920df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ad200640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9177fc0 0x5614b9195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 -- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9174790 msgr2=0x5614b920df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9174790 0x5614b920df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 -- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9177fc0 msgr2=0x5614b9195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9177fc0 0x5614b9195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.761+0000 7f83ae600640 1 -- 172.21.15.44:0/643206749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b9149e50 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f83ae600640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c8f0 tx=0x7f83a0009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34d4d9f6313d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f838ea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a001a040 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8394086500 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f83a0012040 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f83a0003970 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f83b3432740 10 client.8901 fetch_fsmap learned FSMap version 88 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.762+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5614b920e480 con 0x5614b91a3720 2024-07-17T08:12:47.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8394086500 con 0x5614b91a3720 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7f83a0003c00 con 0x5614b91a3720 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 10 client.8901 fetch_fsmap finished waiting for FSMap version 88 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 10 client.8901 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 20 client.8901 populate_metadata read hostname 'smithi044' 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b91a28c0 0x5614b9188420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 4 client.8901 mds_command: new command op to 7258 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:12:47.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 --> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] -- command(tid 0: {"prefix": "status"}) -- 0x5614b90bd750 con 0x5614b91a28c0 2024-07-17T08:12:47.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.763+0000 7f83adc00640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b91a28c0 0x5614b9188420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.764+0000 7f83adc00640 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b91a28c0 0x5614b9188420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41211c1944f433a1 server_cookie=f7a8b068e90c8986 in_seq=0 out_seq=0 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.764+0000 7f838ea00640 10 client.8901 ms_handle_connect on v2:172.21.15.175:6801/808460181 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f838ea00640 1 -- 172.21.15.44:0/643206749 <== mds.0 v2:172.21.15.175:6801/808460181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5614b90bd750 con 0x5614b91a28c0 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f838ea00640 10 client.8901 handle_command_reply: tid=0 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 client.8901 shutdown 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 msgr2=0x5614b9195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c8f0 tx=0x7f83a0009730 comp rx=0 tx=0).stop 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 shutdown_connections 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614b91a3720 0x5614b9195c40 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5614b9177fc0 0x5614b9195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:6801/808460181,v1:172.21.15.175:6836/808460181] conn(0x5614b91a28c0 0x5614b9188420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 --2- 172.21.15.44:0/643206749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5614b9174790 0x5614b920df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 >> 172.21.15.44:0/643206749 conn(0x5614b914f280 msgr2=0x5614b916d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.765+0000 7f838ea00640 0 client.8901 ms_handle_reset on v2:172.21.15.175:6801/808460181 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.766+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 shutdown_connections 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.766+0000 7f83b3432740 1 -- 172.21.15.44:0/643206749 wait complete. 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.766+0000 7f83b3432740 20 client.8901 trim_cache size 0 max 16384 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 7258, 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:12:47.769 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8205168230000002, 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 88, 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 174, 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 174, 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.711152654999999 2024-07-17T08:12:47.770 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.766+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 msgr2=0x7f83880790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.766+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 0x7f83880790a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f839c009890 tx=0x7f839c008040 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 msgr2=0x7f83a81f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8394009850 tx=0x7f83940039e0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 shutdown_connections 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f8388076be0 0x7f83880790a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83a8152300 0x7f83a81f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a8151f20 0x7f83a81f4120 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 --2- 172.21.15.44:0/3244107119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83a80cd0b0 0x7f83a8099190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 >> 172.21.15.44:0/3244107119 conn(0x7f83a80cf0d0 msgr2=0x7f83a815d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 shutdown_connections 2024-07-17T08:12:47.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.767+0000 7f83af000640 1 -- 172.21.15.44:0/3244107119 wait complete. 2024-07-17T08:12:47.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:12:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 -- 172.21.15.44:0/2785513443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 msgr2=0x7fc6400ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 --2- 172.21.15.44:0/2785513443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc6400ac840 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc630005fd0 tx=0x7fc63002b910 comp rx=0 tx=0).stop 2024-07-17T08:12:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 -- 172.21.15.44:0/2785513443 shutdown_connections 2024-07-17T08:12:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 --2- 172.21.15.44:0/2785513443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc6400ac840 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 --2- 172.21.15.44:0/2785513443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64006f710 0x7fc640153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 -- 172.21.15.44:0/2785513443 >> 172.21.15.44:0/2785513443 conn(0x7fc6400b0fb0 msgr2=0x7fc6400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:47.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 -- 172.21.15.44:0/2785513443 shutdown_connections 2024-07-17T08:12:47.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.993+0000 7fc646800640 1 -- 172.21.15.44:0/2785513443 wait complete. 2024-07-17T08:12:47.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.995+0000 7fc646800640 1 Processor -- start 2024-07-17T08:12:47.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.995+0000 7fc646800640 1 -- start start 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 0x7fc64014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64013ed60 0x7fc6401f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6400ae800 con 0x7fc6400ac3e0 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc64006f0a0 con 0x7fc64006f710 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc646800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6400ae980 con 0x7fc64013ed60 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.996+0000 7fc63ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48706/0 (socket says 172.21.15.44:48706) 2024-07-17T08:12:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 -- 172.21.15.44:0/1093815334 learned_addr learned my addr 172.21.15.44:0/1093815334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63fe00640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64013ed60 0x7fc6401f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63f400640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 0x7fc64014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 -- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64013ed60 msgr2=0x7fc6401f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64013ed60 0x7fc6401f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 -- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 msgr2=0x7fc64014f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 0x7fc64014f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 -- 172.21.15.44:0/1093815334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc640071cc0 con 0x7fc6400ac3e0 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63f400640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 0x7fc64014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.997+0000 7fc63ea00640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc63000a960 tx=0x7fc630004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.998+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630004e20 con 0x7fc6400ac3e0 2024-07-17T08:12:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.998+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc630031700 con 0x7fc6400ac3e0 2024-07-17T08:12:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.998+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630031af0 con 0x7fc6400ac3e0 2024-07-17T08:12:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.998+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6401411b0 con 0x7fc6400ac3e0 2024-07-17T08:12:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.998+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc640141770 con 0x7fc6400ac3e0 2024-07-17T08:12:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:47.999+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64013fad0 con 0x7fc6400ac3e0 2024-07-17T08:12:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.000+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7fc630038070 con 0x7fc6400ac3e0 2024-07-17T08:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.001+0000 7fc637400640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 0x7fc61c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.001+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7fc6300b9a90 con 0x7fc6400ac3e0 2024-07-17T08:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.002+0000 7fc63f400640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 0x7fc61c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.003+0000 7fc63f400640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 0x7fc61c079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc628009b40 tx=0x7fc628005e50 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.006+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc63003f030 con 0x7fc6400ac3e0 2024-07-17T08:12:48.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.262+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc6400cb650 con 0x7fc6400ac3e0 2024-07-17T08:12:48.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.263+0000 7fc637400640 1 -- 172.21.15.44:0/1093815334 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7fc630082b10 con 0x7fc6400ac3e0 2024-07-17T08:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:12:48.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 msgr2=0x7fc61c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 0x7fc61c079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc628009b40 tx=0x7fc628005e50 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 msgr2=0x7fc640072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc63000a960 tx=0x7fc630004820 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 shutdown_connections 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7fc61c076bb0 0x7fc61c079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc64013ed60 0x7fc6401f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6400ac3e0 0x7fc640072010 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 --2- 172.21.15.44:0/1093815334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc64006f710 0x7fc64014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 >> 172.21.15.44:0/1093815334 conn(0x7fc6400b0fb0 msgr2=0x7fc6400af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 shutdown_connections 2024-07-17T08:12:48.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48.266+0000 7fc646800640 1 -- 172.21.15.44:0/1093815334 wait complete. 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:12:48.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:12:48.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.289 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:12:48.339 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 37889sec preferred_lft 37889sec 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:12:48.340 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:12:48.341 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:12:48.342 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:12:48.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.343 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:12:48.417 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:12:48.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.421 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:12:48.475 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:12:48.475 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:12:48.476 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:12:48.476 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:12:48.476 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:12:48.476 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:12:48.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:12:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:12:48.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.700 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:12:48.746 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:12:48.746 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:12:48.746 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:12:48.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:12:48.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:12:48.877 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:12:48.933 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:12:48.933 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:12:48.933 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:12:48.991 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:12:48.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:48.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:12:49.107 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:12:49.107 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:12:49.107 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:12:49.129 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:12:49.129 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:12:49.129 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:12:49.129 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:12:49.130 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:12:49.158 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:12:49.159 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:12:49.161 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:12:49.162 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:12:49.162 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:12:49.162 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:12:49.162 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:12:49.166 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:12:49.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:49.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:12:49.245 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:12:49.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:12:49.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:12:49.390 INFO:teuthology.orchestra.run.smithi044.stdout:client8910 2024-07-17T08:12:49.391 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:12:49.391 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:12:49.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:12:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.659+0000 7f1e4f200640 1 -- 172.21.15.44:0/3050281877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500b33f0 msgr2=0x7f1e5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.659+0000 7f1e4f200640 1 --2- 172.21.15.44:0/3050281877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500b33f0 0x7f1e5006d420 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1e50070830 tx=0x7f1e4802aa30 comp rx=0 tx=0).stop 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 -- 172.21.15.44:0/3050281877 shutdown_connections 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 --2- 172.21.15.44:0/3050281877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e500ae340 0x7f1e500b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 --2- 172.21.15.44:0/3050281877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500b33f0 0x7f1e5006d420 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 --2- 172.21.15.44:0/3050281877 >> v2:172.21.15.175:3300/0 conn(0x7f1e501503c0 0x7f1e5015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 -- 172.21.15.44:0/3050281877 >> 172.21.15.44:0/3050281877 conn(0x7f1e500cf0d0 msgr2=0x7f1e500cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 -- 172.21.15.44:0/3050281877 shutdown_connections 2024-07-17T08:12:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.660+0000 7f1e4f200640 1 -- 172.21.15.44:0/3050281877 wait complete. 2024-07-17T08:12:49.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.661+0000 7f1e4f200640 1 Processor -- start 2024-07-17T08:12:49.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.662+0000 7f1e4f200640 1 -- start start 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e500b33f0 0x7f1e5006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e501503c0 0x7f1e5006c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5006dae0 con 0x7f1e500ae340 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5006d960 con 0x7f1e500b33f0 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5006dc60 con 0x7f1e501503c0 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36284/0 (socket says 172.21.15.44:36284) 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 -- 172.21.15.44:0/931652336 learned_addr learned my addr 172.21.15.44:0/931652336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:49.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4e800640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e501503c0 0x7f1e5006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4d400640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e500b33f0 0x7f1e5006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 -- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e501503c0 msgr2=0x7f1e5006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e501503c0 0x7f1e5006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 -- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e500b33f0 msgr2=0x7f1e5006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e500b33f0 0x7f1e5006bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.663+0000 7f1e4de00640 1 -- 172.21.15.44:0/931652336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501f7200 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e4de00640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000d740 tx=0x7f1e4000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40017070 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e40009d20 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400106c0 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501f74e0 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.664+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501f7aa0 con 0x7f1e500ae340 2024-07-17T08:12:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.666+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e500cffe0 con 0x7f1e500ae340 2024-07-17T08:12:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.667+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f1e4001a070 con 0x7f1e500ae340 2024-07-17T08:12:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.668+0000 7f1e46000640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 0x7f1e2c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.668+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f1e400986e0 con 0x7f1e500ae340 2024-07-17T08:12:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.669+0000 7f1e4d400640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 0x7f1e2c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.670+0000 7f1e4d400640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 0x7f1e2c079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e500af2e0 tx=0x7f1e38005e50 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.673+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e40061890 con 0x7f1e500ae340 2024-07-17T08:12:49.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:49.910+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 --> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1e50159410 con 0x7f1e2c076bb0 2024-07-17T08:12:57.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.466+0000 7f00b3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.466+0000 7f00b3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.466+0000 7f00b3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.466+0000 7f00b3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.466+0000 7f00b3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.468+0000 7f00b8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.468+0000 7f00b8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.468+0000 7f00b8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.468+0000 7f00b8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.470 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:12:57.468+0000 7f00b8a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:12:57.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.482+0000 7f1e46000640 1 -- 172.21.15.44:0/931652336 <== mgr.6653 v2:172.21.15.175:6834/3617669990 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1e50159410 con 0x7f1e2c076bb0 2024-07-17T08:12:57.483 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:12:57.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:12:57.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:12:57.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:12:57.483 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:12:57.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.485+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 msgr2=0x7f1e2c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:57.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.485+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 0x7f1e2c079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e500af2e0 tx=0x7f1e38005e50 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.485+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 msgr2=0x7f1e5014f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.485+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000d740 tx=0x7f1e4000dc00 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 shutdown_connections 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f1e2c076bb0 0x7f1e2c079070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e501503c0 0x7f1e5006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e500b33f0 0x7f1e5006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 --2- 172.21.15.44:0/931652336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e500ae340 0x7f1e5014f5e0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 >> 172.21.15.44:0/931652336 conn(0x7f1e500cf0d0 msgr2=0x7f1e500b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 shutdown_connections 2024-07-17T08:12:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.486+0000 7f1e4f200640 1 -- 172.21.15.44:0/931652336 wait complete. 2024-07-17T08:12:57.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 -- 172.21.15.44:0/1214195563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 msgr2=0x7f6d940051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 --2- 172.21.15.44:0/1214195563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6d940051a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009ed0 tx=0x7f6d94005830 comp rx=0 tx=0).stop 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db1800640 1 -- 172.21.15.44:0/1214195563 reap_dead start 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 -- 172.21.15.44:0/1214195563 shutdown_connections 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 -- 172.21.15.44:0/1214195563 >> 172.21.15.44:0/1214195563 conn(0x7f6dac0b0eb0 msgr2=0x7f6dac0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 -- 172.21.15.44:0/1214195563 shutdown_connections 2024-07-17T08:12:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.729+0000 7f6db3800640 1 -- 172.21.15.44:0/1214195563 wait complete. 2024-07-17T08:12:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.731+0000 7f6db3800640 1 Processor -- start 2024-07-17T08:12:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.731+0000 7f6db3800640 1 -- start start 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dac14a4f0 0x7f6dac148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dac1f4540 0x7f6dac1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac15c470 con 0x7f6d94004d40 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac15c2f0 con 0x7f6dac14a4f0 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac15c5f0 con 0x7f6dac1f4540 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36298/0 (socket says 172.21.15.44:36298) 2024-07-17T08:12:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.732+0000 7f6db0e00640 1 -- 172.21.15.44:0/3791437307 learned_addr learned my addr 172.21.15.44:0/3791437307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 -- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dac1f4540 msgr2=0x7f6dac1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dac1f4540 0x7f6dac1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 -- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dac14a4f0 msgr2=0x7f6dac148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dac14a4f0 0x7f6dac148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 -- 172.21.15.44:0/3791437307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac14a970 con 0x7f6d94004d40 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db0e00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6da0004d60 tx=0x7f6da0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00107d0 con 0x7f6d94004d40 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da0016070 con 0x7f6d94004d40 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0010d00 con 0x7f6d94004d40 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.733+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac14ac50 con 0x7f6d94004d40 2024-07-17T08:12:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.734+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac14b210 con 0x7f6d94004d40 2024-07-17T08:12:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.735+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dac14b650 con 0x7f6d94004d40 2024-07-17T08:12:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.736+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 91) ==== 98829+0+0 (secure 0 0 0) 0x7f6da001b460 con 0x7f6d94004d40 2024-07-17T08:12:57.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.737+0000 7f6da8c00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 0x7f6d8c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:57.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.737+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f6da0098cf0 con 0x7f6d94004d40 2024-07-17T08:12:57.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.738+0000 7f6dabe00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 0x7f6d8c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.739+0000 7f6dabe00640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 0x7f6d8c079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d98001010 tx=0x7f6d98001580 comp rx=0 tx=0).ready entity=mgr.6653 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:57.742+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6da009d070 con 0x7f6d94004d40 2024-07-17T08:12:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.004+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f6dac0cd2e0 con 0x7f6d94004d40 2024-07-17T08:12:58.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.057+0000 7f6da8c00640 1 -- 172.21.15.44:0/3791437307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v92) ==== 92+0+0 (secure 0 0 0) 0x7f6da0061ea0 con 0x7f6d94004d40 2024-07-17T08:12:58.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.060+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 msgr2=0x7f6d8c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.060+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 0x7f6d8c079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d98001010 tx=0x7f6d98001580 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.060+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 msgr2=0x7f6dac148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.060+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6da0004d60 tx=0x7f6da0001630 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 shutdown_connections 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f6d8c076bb0 0x7f6d8c079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dac1f4540 0x7f6dac1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dac14a4f0 0x7f6dac148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 --2- 172.21.15.44:0/3791437307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d94004d40 0x7f6dac148450 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 >> 172.21.15.44:0/3791437307 conn(0x7f6dac0b0eb0 msgr2=0x7f6dac0b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:58.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 shutdown_connections 2024-07-17T08:12:58.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.061+0000 7f6db3800640 1 -- 172.21.15.44:0/3791437307 wait complete. 2024-07-17T08:12:58.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:12:58.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 -- 172.21.15.44:0/941779051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 msgr2=0x7f7df40ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 --2- 172.21.15.44:0/941779051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40ce5b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009830 tx=0x7f7de402a5a0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 -- 172.21.15.44:0/941779051 shutdown_connections 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 --2- 172.21.15.44:0/941779051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 0x7f7df4161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 --2- 172.21.15.44:0/941779051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40ce5b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 -- 172.21.15.44:0/941779051 >> 172.21.15.44:0/941779051 conn(0x7f7df40d3700 msgr2=0x7f7df40d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 -- 172.21.15.44:0/941779051 shutdown_connections 2024-07-17T08:12:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.275+0000 7f7df9c00640 1 -- 172.21.15.44:0/941779051 wait complete. 2024-07-17T08:12:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.277+0000 7f7df9c00640 1 Processor -- start 2024-07-17T08:12:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.277+0000 7f7df9c00640 1 -- start start 2024-07-17T08:12:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 0x7f7df4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 0x7f7df40c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4162420 con 0x7f7df40ce1d0 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41622a0 con 0x7f7df40b29e0 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7df4167910 con 0x7f7df40b6790 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.278+0000 7f7df3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36314/0 (socket says 172.21.15.44:36314) 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 -- 172.21.15.44:0/1707642711 learned_addr learned my addr 172.21.15.44:0/1707642711 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df2a00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 0x7f7df4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df2000640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 0x7f7df40c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 msgr2=0x7f7df40c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 0x7f7df40c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 msgr2=0x7f7df4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 0x7f7df4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 -- 172.21.15.44:0/1707642711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df41f4a50 con 0x7f7df40ce1d0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df2a00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 0x7f7df4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.279+0000 7f7df3400640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00cbc0 tx=0x7f7dec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.280+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec016020 con 0x7f7df40ce1d0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.280+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dec00ae30 con 0x7f7df40ce1d0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.280+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec012660 con 0x7f7df40ce1d0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.280+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df41f4d30 con 0x7f7df40ce1d0 2024-07-17T08:12:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.280+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40c7510 con 0x7f7df40ce1d0 2024-07-17T08:12:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.281+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df40ce5b0 con 0x7f7df40ce1d0 2024-07-17T08:12:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.282+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 92) ==== 98846+0+0 (secure 0 0 0) 0x7f7dec019020 con 0x7f7df40ce1d0 2024-07-17T08:12:58.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.283+0000 7f7deaa00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 0x7f7dd40790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.283+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f7dec098eb0 con 0x7f7df40ce1d0 2024-07-17T08:12:58.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.284+0000 7f7df2a00640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 msgr2=0x7f7dd40790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3617669990 2024-07-17T08:12:58.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.284+0000 7f7df2a00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 0x7f7dd40790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:58.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.287+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7dec020020 con 0x7f7df40ce1d0 2024-07-17T08:12:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.485+0000 7f7df2a00640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 msgr2=0x7f7dd40790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3617669990 2024-07-17T08:12:58.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.485+0000 7f7df2a00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 0x7f7dd40790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.491+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f7df40b8530 con 0x7f7df40ce1d0 2024-07-17T08:12:58.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.498+0000 7f7deaa00640 1 -- 172.21.15.44:0/1707642711 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v61)=0 v61) ==== 138+0+0 (secure 0 0 0) 0x7f7dec061f30 con 0x7f7df40ce1d0 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 msgr2=0x7f7dd40790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 0x7f7dd40790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 msgr2=0x7f7df40c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00cbc0 tx=0x7f7dec00a430 comp rx=0 tx=0).stop 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 shutdown_connections 2024-07-17T08:12:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f7dd4076c00 0x7f7dd40790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df40ce1d0 0x7f7df40c8620 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df40b6790 0x7f7df40c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 --2- 172.21.15.44:0/1707642711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df40b29e0 0x7f7df4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 >> 172.21.15.44:0/1707642711 conn(0x7f7df40d3700 msgr2=0x7f7df40b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.501+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 shutdown_connections 2024-07-17T08:12:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.502+0000 7f7df9c00640 1 -- 172.21.15.44:0/1707642711 wait complete. 2024-07-17T08:12:58.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:12:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 -- 172.21.15.44:0/2123087001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 msgr2=0x7f50a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 --2- 172.21.15.44:0/2123087001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50a400b9c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ace90 tx=0x7f50a4005830 comp rx=0 tx=0).stop 2024-07-17T08:12:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 -- 172.21.15.44:0/2123087001 shutdown_connections 2024-07-17T08:12:58.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 --2- 172.21.15.44:0/2123087001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50a400b9c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 -- 172.21.15.44:0/2123087001 >> 172.21.15.44:0/2123087001 conn(0x7f50c00b0eb0 msgr2=0x7f50c00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:58.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 -- 172.21.15.44:0/2123087001 shutdown_connections 2024-07-17T08:12:58.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.720+0000 7f50c7200640 1 -- 172.21.15.44:0/2123087001 wait complete. 2024-07-17T08:12:58.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.722+0000 7f50c7200640 1 Processor -- start 2024-07-17T08:12:58.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.722+0000 7f50c7200640 1 -- start start 2024-07-17T08:12:58.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50c0154040 0x7f50c0072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50c00ca670 0x7f50c00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50c00b2540 con 0x7f50a40052e0 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50c00b23c0 con 0x7f50c00ca670 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50c00ac190 con 0x7f50c0154040 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36336/0 (socket says 172.21.15.44:36336) 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.723+0000 7f50bfe00640 1 -- 172.21.15.44:0/1847302636 learned_addr learned my addr 172.21.15.44:0/1847302636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50c0154040 msgr2=0x7f50c0072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50b7e00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50c0154040 0x7f50c0072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50c5200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50c00ca670 0x7f50c00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50c0154040 0x7f50c0072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50c00ca670 msgr2=0x7f50c00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50c00ca670 0x7f50c00caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 -- 172.21.15.44:0/1847302636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01f8680 con 0x7f50a40052e0 2024-07-17T08:12:58.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bfe00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f50b000f7d0 tx=0x7f50b000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:12:58.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000bcb0 con 0x7f50a40052e0 2024-07-17T08:12:58.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50b0005020 con 0x7f50a40052e0 2024-07-17T08:12:58.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0005470 con 0x7f50a40052e0 2024-07-17T08:12:58.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.724+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c0155660 con 0x7f50a40052e0 2024-07-17T08:12:58.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.725+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c01548a0 con 0x7f50a40052e0 2024-07-17T08:12:58.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.726+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c015ac10 con 0x7f50a40052e0 2024-07-17T08:12:58.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.727+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 92) ==== 98846+0+0 (secure 0 0 0) 0x7f50b0019030 con 0x7f50a40052e0 2024-07-17T08:12:58.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.728+0000 7f50bcc00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 0x7f509c0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:12:58.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.728+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6649+0+0 (secure 0 0 0) 0x7f50b0098db0 con 0x7f50a40052e0 2024-07-17T08:12:58.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.728+0000 7f50b7e00640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 msgr2=0x7f509c0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3617669990 2024-07-17T08:12:58.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.728+0000 7f50b7e00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 0x7f509c0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:12:58.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.732+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f50b009d070 con 0x7f50a40052e0 2024-07-17T08:12:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.930+0000 7f50b7e00640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 msgr2=0x7f509c0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3617669990 2024-07-17T08:12:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.930+0000 7f50b7e00640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 0x7f509c0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:12:58.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.934+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f50c00cb030 con 0x7f50a40052e0 2024-07-17T08:12:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.940+0000 7f50bcc00640 1 -- 172.21.15.44:0/1847302636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v62)=0 v62) ==== 133+0+0 (secure 0 0 0) 0x7f50c01f8680 con 0x7f50a40052e0 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 msgr2=0x7f509c0790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 0x7f509c0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 msgr2=0x7f50c01f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f50b000f7d0 tx=0x7f50b000fc90 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 shutdown_connections 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:6834/3617669990,v1:172.21.15.175:6837/3617669990] conn(0x7f509c076c00 0x7f509c0790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50c00ca670 0x7f50c00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50c0154040 0x7f50c0072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 --2- 172.21.15.44:0/1847302636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50a40052e0 0x7f50c01f8140 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 >> 172.21.15.44:0/1847302636 conn(0x7f50c00b0eb0 msgr2=0x7f50c015cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:12:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 shutdown_connections 2024-07-17T08:12:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:12:58.943+0000 7f50c7200640 1 -- 172.21.15.44:0/1847302636 wait complete. 2024-07-17T08:12:58.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:12:58.963 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.576 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:02.577 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:13:02.575+0000 7f9f3e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:13:03.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:13:03.021 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:13:33.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:13:33.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:14:01.071 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2024-07-17T08:14:01.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.294+0000 7f6e2c800640 1 -- 172.21.15.44:0/949160865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e24152220 msgr2=0x7f6e2406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.294+0000 7f6e2c800640 1 --2- 172.21.15.44:0/949160865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e24152220 0x7f6e2406d620 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009830 tx=0x7f6e1802a480 comp rx=0 tx=0).stop 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.294+0000 7f6e2c800640 1 -- 172.21.15.44:0/949160865 shutdown_connections 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.294+0000 7f6e2c800640 1 --2- 172.21.15.44:0/949160865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e24152220 0x7f6e2406d620 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.294+0000 7f6e2c800640 1 -- 172.21.15.44:0/949160865 >> 172.21.15.44:0/949160865 conn(0x7f6e240b0eb0 msgr2=0x7f6e240b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.295+0000 7f6e2c800640 1 -- 172.21.15.44:0/949160865 shutdown_connections 2024-07-17T08:14:01.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.295+0000 7f6e2c800640 1 -- 172.21.15.44:0/949160865 wait complete. 2024-07-17T08:14:01.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.296+0000 7f6e2c800640 1 Processor -- start 2024-07-17T08:14:01.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.296+0000 7f6e2c800640 1 -- start start 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e240cbab0 0x7f6e241f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24152220 0x7f6e241548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240cd9f0 con 0x7f6e2406ed30 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2415c800 con 0x7f6e24152220 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.297+0000 7f6e2c800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240cdb70 con 0x7f6e240cbab0 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48518/0 (socket says 172.21.15.44:48518) 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 -- 172.21.15.44:0/4035468818 learned_addr learned my addr 172.21.15.44:0/4035468818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29400640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e240cbab0 0x7f6e241f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 -- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e240cbab0 msgr2=0x7f6e241f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e240cbab0 0x7f6e241f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e2a800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24152220 0x7f6e241548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 -- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24152220 msgr2=0x7f6e241548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24152220 0x7f6e241548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.298+0000 7f6e29e00640 1 -- 172.21.15.44:0/4035468818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241553d0 con 0x7f6e2406ed30 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e29e00640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e18033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031d60 con 0x7f6e2406ed30 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e180330d0 con 0x7f6e2406ed30 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18033520 con 0x7f6e2406ed30 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241556b0 con 0x7f6e2406ed30 2024-07-17T08:14:01.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.299+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e241f7d00 con 0x7f6e2406ed30 2024-07-17T08:14:01.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.301+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e240ce090 con 0x7f6e2406ed30 2024-07-17T08:14:01.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.302+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 96) ==== 99005+0+0 (secure 0 0 0) 0x7f6e180035f0 con 0x7f6e2406ed30 2024-07-17T08:14:01.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.303+0000 7f6e1e000640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 0x7f6e04079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.303+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6907+0+0 (secure 0 0 0) 0x7f6e180ba0c0 con 0x7f6e2406ed30 2024-07-17T08:14:01.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.303+0000 7f6e29400640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 0x7f6e04079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.305+0000 7f6e29400640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 0x7f6e04079270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6e241504d0 tx=0x7f6e10009040 comp rx=0 tx=0).ready entity=mgr.6875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:01.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.307+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6e1803f030 con 0x7f6e2406ed30 2024-07-17T08:14:01.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.540+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 --> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) -- 0x7f6e240cb890 con 0x7f6e04076db0 2024-07-17T08:14:01.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.718+0000 7f6e1e000640 1 -- 172.21.15.44:0/4035468818 <== mgr.6875 v2:172.21.15.175:6834/3180589979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e240cb890 con 0x7f6e04076db0 2024-07-17T08:14:01.719 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T08:14:01.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.721+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 msgr2=0x7f6e04079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.721+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 0x7f6e04079270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6e241504d0 tx=0x7f6e10009040 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.721+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 msgr2=0x7f6e241f8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.721+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e18033d60 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 shutdown_connections 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24152220 0x7f6e241548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f6e04076db0 0x7f6e04079270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e240cbab0 0x7f6e241f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 --2- 172.21.15.44:0/4035468818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e2406ed30 0x7f6e241f8220 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 >> 172.21.15.44:0/4035468818 conn(0x7f6e240b0eb0 msgr2=0x7f6e240b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 shutdown_connections 2024-07-17T08:14:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.722+0000 7f6e2c800640 1 -- 172.21.15.44:0/4035468818 wait complete. 2024-07-17T08:14:01.744 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T08:14:01.744 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-07-17T08:14:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 -- 172.21.15.44:0/3245378202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480ce3b0 msgr2=0x7f22480d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 --2- 172.21.15.44:0/3245378202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480ce3b0 0x7f22480d1820 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2240005fd0 tx=0x7f224002a820 comp rx=0 tx=0).stop 2024-07-17T08:14:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 -- 172.21.15.44:0/3245378202 shutdown_connections 2024-07-17T08:14:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 --2- 172.21.15.44:0/3245378202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480ce3b0 0x7f22480d1820 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 -- 172.21.15.44:0/3245378202 >> 172.21.15.44:0/3245378202 conn(0x7f22480d04c0 msgr2=0x7f22480d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 -- 172.21.15.44:0/3245378202 shutdown_connections 2024-07-17T08:14:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.955+0000 7f224f000640 1 -- 172.21.15.44:0/3245378202 wait complete. 2024-07-17T08:14:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.957+0000 7f224f000640 1 Processor -- start 2024-07-17T08:14:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.957+0000 7f224f000640 1 -- start start 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22480ce3b0 0x7f22481f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f224815d350 0x7f2248157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f224806e5d0 con 0x7f22480d1440 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f224806e450 con 0x7f224815d350 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f224806e750 con 0x7f22480ce3b0 2024-07-17T08:14:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48528/0 (socket says 172.21.15.44:48528) 2024-07-17T08:14:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.958+0000 7f224d200640 1 -- 172.21.15.44:0/1883906817 learned_addr learned my addr 172.21.15.44:0/1883906817 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224dc00640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22480ce3b0 0x7f22481f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224e600640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f224815d350 0x7f2248157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 -- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22480ce3b0 msgr2=0x7f22481f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22480ce3b0 0x7f22481f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 -- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f224815d350 msgr2=0x7f2248157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f224815d350 0x7f2248157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 -- 172.21.15.44:0/1883906817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22481f7b10 con 0x7f22480d1440 2024-07-17T08:14:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f224d200640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f223400c430 tx=0x7f223400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.959+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234011790 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.960+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2234017070 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.960+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234011d00 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.960+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22481f7df0 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.960+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2248158280 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.961+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22480d1820 con 0x7f22480d1440 2024-07-17T08:14:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.962+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 96) ==== 99005+0+0 (secure 0 0 0) 0x7f223401c460 con 0x7f22480d1440 2024-07-17T08:14:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.963+0000 7f2245600640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 0x7f2228079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.963+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6907+0+0 (secure 0 0 0) 0x7f2234098a80 con 0x7f22480d1440 2024-07-17T08:14:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.964+0000 7f224dc00640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 0x7f2228079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.965+0000 7f224dc00640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 0x7f2228079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2238007fd0 tx=0x7f2238009040 comp rx=0 tx=0).ready entity=mgr.6875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:01.968+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2234061820 con 0x7f22480d1440 2024-07-17T08:14:02.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.193+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 --> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f22480b29f0 con 0x7f2228076db0 2024-07-17T08:14:02.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.389+0000 7f2245600640 1 -- 172.21.15.44:0/1883906817 <== mgr.6875 v2:172.21.15.175:6834/3180589979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f22480b29f0 con 0x7f2228076db0 2024-07-17T08:14:02.390 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path snap_test_dir1 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 msgr2=0x7f2228079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 0x7f2228079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2238007fd0 tx=0x7f2238009040 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 msgr2=0x7f22480b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f223400c430 tx=0x7f223400c8f0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 shutdown_connections 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f224815d350 0x7f2248157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22480d1440 0x7f22480b8480 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2228076db0 0x7f2228079270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 --2- 172.21.15.44:0/1883906817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22480ce3b0 0x7f22481f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 >> 172.21.15.44:0/1883906817 conn(0x7f22480d04c0 msgr2=0x7f2248160870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:02.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 shutdown_connections 2024-07-17T08:14:02.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.392+0000 7f224f000640 1 -- 172.21.15.44:0/1883906817 wait complete. 2024-07-17T08:14:02.413 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-07-17T08:14:02.413 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2024-07-17T08:14:02.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 -- 172.21.15.44:0/2286576402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 msgr2=0x7ff750161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:02.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 --2- 172.21.15.44:0/2286576402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff750161370 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7ff748005fd0 tx=0x7ff74802aa50 comp rx=0 tx=0).stop 2024-07-17T08:14:02.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 -- 172.21.15.44:0/2286576402 shutdown_connections 2024-07-17T08:14:02.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 --2- 172.21.15.44:0/2286576402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff750161370 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 --2- 172.21.15.44:0/2286576402 >> v2:172.21.15.175:3300/0 conn(0x7ff7500ad8a0 0x7ff7500adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 --2- 172.21.15.44:0/2286576402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 0x7ff7500aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 -- 172.21.15.44:0/2286576402 >> 172.21.15.44:0/2286576402 conn(0x7ff75006bf40 msgr2=0x7ff75006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 -- 172.21.15.44:0/2286576402 shutdown_connections 2024-07-17T08:14:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.629+0000 7ff755200640 1 -- 172.21.15.44:0/2286576402 wait complete. 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.631+0000 7ff755200640 1 Processor -- start 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.631+0000 7ff755200640 1 -- start start 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500ad8a0 0x7ff750153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 0x7ff75014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff7501f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501618b0 con 0x7ff750160ec0 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7500ae8f0 con 0x7ff7500ad8a0 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff755200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff750161a30 con 0x7ff750154350 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff74e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff7501f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff74e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff7501f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48548/0 (socket says 172.21.15.44:48548) 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.632+0000 7ff74e800640 1 -- 172.21.15.44:0/860532350 learned_addr learned my addr 172.21.15.44:0/860532350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74de00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500ad8a0 0x7ff750153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff747e00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 0x7ff75014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 -- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 msgr2=0x7ff75014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 0x7ff75014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 -- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500ad8a0 msgr2=0x7ff750153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500ad8a0 0x7ff750153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 -- 172.21.15.44:0/860532350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7501f4350 con 0x7ff750160ec0 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74de00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7500ad8a0 0x7ff750153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff747e00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff750154350 0x7ff75014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.633+0000 7ff74e800640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff750160ec0 0x7ff7501f74f0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7ff748006010 tx=0x7ff748003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.634+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748046070 con 0x7ff750160ec0 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.634+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7480045c0 con 0x7ff750160ec0 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.634+0000 7ff755200640 1 -- 172.21.15.44:0/860532350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7501f4630 con 0x7ff750160ec0 2024-07-17T08:14:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.634+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748033070 con 0x7ff750160ec0 2024-07-17T08:14:02.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.634+0000 7ff755200640 1 -- 172.21.15.44:0/860532350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501f4bf0 con 0x7ff750160ec0 2024-07-17T08:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.635+0000 7ff755200640 1 -- 172.21.15.44:0/860532350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7500aebf0 con 0x7ff750160ec0 2024-07-17T08:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.636+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 96) ==== 99005+0+0 (secure 0 0 0) 0x7ff748049070 con 0x7ff750160ec0 2024-07-17T08:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.637+0000 7ff746000640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7ff72c076db0 0x7ff72c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.637+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 6965+0+0 (secure 0 0 0) 0x7ff7480b9cc0 con 0x7ff750160ec0 2024-07-17T08:14:02.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.638+0000 7ff74de00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7ff72c076db0 0x7ff72c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:02.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.639+0000 7ff74de00640 1 --2- 172.21.15.44:0/860532350 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7ff72c076db0 0x7ff72c079270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff740006fd0 tx=0x7ff740008040 comp rx=0 tx=0).ready entity=mgr.6875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:02.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.641+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff7480c0080 con 0x7ff750160ec0 2024-07-17T08:14:02.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.881+0000 7ff755200640 1 -- 172.21.15.44:0/860532350 --> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7ff7500cd3b0 con 0x7ff72c076db0 2024-07-17T08:14:02.899 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:02.898+0000 7f9f47400640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2024-07-17T08:14:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:02.899+0000 7ff746000640 1 -- 172.21.15.44:0/860532350 <== mgr.6875 v2:172.21.15.175:6834/3180589979 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) ==== 49+0+0 (secure 0 0 0) 0x7ff74000ae20 con 0x7ff72c076db0 2024-07-17T08:14:02.899 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2024-07-17T08:14:02.915 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-07-17T08:14:02.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:02.915 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T08:14:02.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:14:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 -- 172.21.15.44:0/3969817878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c0cf6f0 msgr2=0x7fd33c06d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 --2- 172.21.15.44:0/3969817878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c0cf6f0 0x7fd33c06d610 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0d0150 tx=0x7fd33402b900 comp rx=0 tx=0).stop 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 -- 172.21.15.44:0/3969817878 shutdown_connections 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 --2- 172.21.15.44:0/3969817878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c0cf6f0 0x7fd33c06d610 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 --2- 172.21.15.44:0/3969817878 >> v2:172.21.15.175:3300/0 conn(0x7fd33c06f5a0 0x7fd33c0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 -- 172.21.15.44:0/3969817878 >> 172.21.15.44:0/3969817878 conn(0x7fd33c06c120 msgr2=0x7fd33c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 -- 172.21.15.44:0/3969817878 shutdown_connections 2024-07-17T08:14:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.194+0000 7fd341200640 1 -- 172.21.15.44:0/3969817878 wait complete. 2024-07-17T08:14:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.196+0000 7fd341200640 1 Processor -- start 2024-07-17T08:14:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.196+0000 7fd341200640 1 -- start start 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c06f5a0 0x7fd33c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c0ae810 con 0x7fd33c156d40 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c0ae690 con 0x7fd33c06f5a0 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.197+0000 7fd341200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c0ae990 con 0x7fd33c0cf6f0 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:03.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48564/0 (socket says 172.21.15.44:48564) 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 -- 172.21.15.44:0/392137799 learned_addr learned my addr 172.21.15.44:0/392137799 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33b400640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c06f5a0 0x7fd33c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33aa00640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 -- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0cf6f0 msgr2=0x7fd33c0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 -- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c06f5a0 msgr2=0x7fd33c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c06f5a0 0x7fd33c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.198+0000 7fd33be00640 1 -- 172.21.15.44:0/392137799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c152490 con 0x7fd33c156d40 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd33be00640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd33400ae70 tx=0x7fd334003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334035040 con 0x7fd33c156d40 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3340042c0 con 0x7fd33c156d40 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340046f0 con 0x7fd33c156d40 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c152d30 con 0x7fd33c156d40 2024-07-17T08:14:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.199+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c0b18d0 con 0x7fd33c156d40 2024-07-17T08:14:03.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.201+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33c0b2140 con 0x7fd33c156d40 2024-07-17T08:14:03.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.202+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 96) ==== 99005+0+0 (secure 0 0 0) 0x7fd334038020 con 0x7fd33c156d40 2024-07-17T08:14:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.203+0000 7fd333400640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 0x7fd31c079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.203+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 6965+0+0 (secure 0 0 0) 0x7fd3340ba0a0 con 0x7fd33c156d40 2024-07-17T08:14:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.204+0000 7fd33b400640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 0x7fd31c079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:03.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.205+0000 7fd33b400640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 0x7fd31c079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0058c0 tx=0x7fd32c00a4c0 comp rx=0 tx=0).ready entity=mgr.6875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:03.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.208+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd334042e40 con 0x7fd33c156d40 2024-07-17T08:14:03.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:14:03.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:14:03.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.476+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fd33c158850 con 0x7fd33c156d40 2024-07-17T08:14:03.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.556+0000 7fd333400640 1 -- 172.21.15.44:0/392137799 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v97) ==== 93+0+0 (secure 0 0 0) 0x7fd334082d60 con 0x7fd33c156d40 2024-07-17T08:14:03.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.559+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 msgr2=0x7fd31c079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:03.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.559+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 0x7fd31c079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0058c0 tx=0x7fd32c00a4c0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.559+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 msgr2=0x7fd33c1e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:03.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.559+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd33400ae70 tx=0x7fd334003f10 comp rx=0 tx=0).stop 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 shutdown_connections 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33c156d40 0x7fd33c1e3ae0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fd31c076db0 0x7fd31c079270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0cf6f0 0x7fd33c0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 --2- 172.21.15.44:0/392137799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c06f5a0 0x7fd33c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 >> 172.21.15.44:0/392137799 conn(0x7fd33c06c120 msgr2=0x7fd33c157860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 shutdown_connections 2024-07-17T08:14:03.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:03.560+0000 7fd341200640 1 -- 172.21.15.44:0/392137799 wait complete. 2024-07-17T08:14:03.584 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:14:03.584 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:14:03.681 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:14:03.767 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:14:03.767 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:14:03.767 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:03.767 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:03.815 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:03.815 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:03.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.110+0000 7f16a6200640 1 -- 172.21.15.44:0/4082354087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a0161be0 msgr2=0x7f16a015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.110+0000 7f16a6200640 1 --2- 172.21.15.44:0/4082354087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a0161be0 0x7f16a015d510 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f168c005fd0 tx=0x7f168c02a4a0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 -- 172.21.15.44:0/4082354087 shutdown_connections 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 --2- 172.21.15.44:0/4082354087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a00ce690 0x7f16a00cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 --2- 172.21.15.44:0/4082354087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a0161be0 0x7f16a015d510 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 -- 172.21.15.44:0/4082354087 >> 172.21.15.44:0/4082354087 conn(0x7f16a00b4ff0 msgr2=0x7f16a00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 -- 172.21.15.44:0/4082354087 shutdown_connections 2024-07-17T08:14:04.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.111+0000 7f16a6200640 1 -- 172.21.15.44:0/4082354087 wait complete. 2024-07-17T08:14:04.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.113+0000 7f16a6200640 1 Processor -- start 2024-07-17T08:14:04.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.114+0000 7f16a6200640 1 -- start start 2024-07-17T08:14:04.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16a015d130 0x7f16a00cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a0161be0 0x7f16a00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00d1a60 con 0x7f16a00ce690 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16a00d18e0 con 0x7f16a015d130 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16a00d1be0 con 0x7f16a0161be0 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.115+0000 7f16a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48584/0 (socket says 172.21.15.44:48584) 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 -- 172.21.15.44:0/1011393105 learned_addr learned my addr 172.21.15.44:0/1011393105 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f169fe00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16a015d130 0x7f16a00cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a5800640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a0161be0 0x7f16a00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a0161be0 msgr2=0x7f16a00cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a0161be0 0x7f16a00cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16a015d130 msgr2=0x7f16a00cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16a015d130 0x7f16a00cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 -- 172.21.15.44:0/1011393105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a00aa010 con 0x7f16a00ce690 2024-07-17T08:14:04.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.116+0000 7f16a4e00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f169400c460 tx=0x7f169400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:04.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.117+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694016020 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.117+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1694012960 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.117+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694012db0 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.117+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a00aa2f0 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.117+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a00aa8b0 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.118+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a00762e0 con 0x7f16a00ce690 2024-07-17T08:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.119+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f1694019020 con 0x7f16a00ce690 2024-07-17T08:14:04.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.120+0000 7f169cc00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 0x7f1680079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.120+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f1694098960 con 0x7f16a00ce690 2024-07-17T08:14:04.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.120+0000 7f169fe00640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 msgr2=0x7f1680079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:04.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.120+0000 7f169fe00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 0x7f1680079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:04.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.124+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f16a00762e0 con 0x7f16a00ce690 2024-07-17T08:14:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.321+0000 7f169fe00640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 msgr2=0x7f1680079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.321+0000 7f169fe00640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 0x7f1680079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:04.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.380+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f16a00d0e10 con 0x7f16a00ce690 2024-07-17T08:14:04.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.381+0000 7f169cc00640 1 -- 172.21.15.44:0/1011393105 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v177) ==== 80+0+358 (secure 0 0 0) 0x7f1694020030 con 0x7f16a00ce690 2024-07-17T08:14:04.381 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/253461221 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2425717984 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1618489471 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/3617669990 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/3617669990 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/408471903 2024-07-18T08:13:02.263003+0000 2024-07-17T08:14:04.382 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T08:14:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 msgr2=0x7f1680079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 0x7f1680079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 msgr2=0x7f16a00cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f169400c460 tx=0x7f169400c920 comp rx=0 tx=0).stop 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 shutdown_connections 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f1680076d90 0x7f1680079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16a0161be0 0x7f16a00cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16a015d130 0x7f16a00cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 --2- 172.21.15.44:0/1011393105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16a00ce690 0x7f16a00cb5c0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.383+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 >> 172.21.15.44:0/1011393105 conn(0x7f16a00b4ff0 msgr2=0x7f16a00b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.384+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 shutdown_connections 2024-07-17T08:14:04.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.384+0000 7f16a6200640 1 -- 172.21.15.44:0/1011393105 wait complete. 2024-07-17T08:14:04.404 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:14:04.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:04.404 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:14:04.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:14:04.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:04.487 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:14:04.528 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:14:04.528 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:14:04.528 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:04.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:14:04.567 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:14:04.567 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:04.567 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:04.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:14:04.608 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:14:04.608 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:14:04.609 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:04.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:14:04.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 -- 172.21.15.44:0/4128474173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b8c80 msgr2=0x7f47540b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 --2- 172.21.15.44:0/4128474173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b8c80 0x7f47540b4ec0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4744005fd0 tx=0x7f474402aa50 comp rx=0 tx=0).stop 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 -- 172.21.15.44:0/4128474173 shutdown_connections 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 --2- 172.21.15.44:0/4128474173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f475406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 --2- 172.21.15.44:0/4128474173 >> v2:172.21.15.175:3300/0 conn(0x7f47540b5400 0x7f475406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 --2- 172.21.15.44:0/4128474173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b8c80 0x7f47540b4ec0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 -- 172.21.15.44:0/4128474173 >> 172.21.15.44:0/4128474173 conn(0x7f47540cfec0 msgr2=0x7f47540d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 -- 172.21.15.44:0/4128474173 shutdown_connections 2024-07-17T08:14:04.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.861+0000 7f4759400640 1 -- 172.21.15.44:0/4128474173 wait complete. 2024-07-17T08:14:04.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.863+0000 7f4759400640 1 Processor -- start 2024-07-17T08:14:04.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.863+0000 7f4759400640 1 -- start start 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 0x7f47541f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f4754151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f475406e5b0 con 0x7f47540b5400 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f475406e430 con 0x7f47540b8c80 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4759400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f475406e730 con 0x7f47540ce5c0 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48606/0 (socket says 172.21.15.44:48606) 2024-07-17T08:14:04.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753e00640 1 -- 172.21.15.44:0/3476122698 learned_addr learned my addr 172.21.15.44:0/3476122698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 0x7f47541f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4758a00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f4754151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753e00640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 msgr2=0x7f4754151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.864+0000 7f4753e00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f4754151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4753e00640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 msgr2=0x7f47541f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4753e00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 0x7f47541f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4753e00640 1 -- 172.21.15.44:0/3476122698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475415ac90 con 0x7f47540b5400 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4758a00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f4754151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4753400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 0x7f47541f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4753e00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4744000f80 tx=0x7f4744003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744047070 con 0x7f47540b5400 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47440042d0 con 0x7f47540b5400 2024-07-17T08:14:04.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744033050 con 0x7f47540b5400 2024-07-17T08:14:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541f7f70 con 0x7f47540b5400 2024-07-17T08:14:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.865+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541f8530 con 0x7f47540b5400 2024-07-17T08:14:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.867+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47540b4ec0 con 0x7f47540b5400 2024-07-17T08:14:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.868+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f474404a070 con 0x7f47540b5400 2024-07-17T08:14:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.869+0000 7f474be00640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 0x7f4730079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.869+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f47440ba080 con 0x7f47540b5400 2024-07-17T08:14:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.870+0000 7f4753400640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 msgr2=0x7f4730079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.870+0000 7f4753400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 0x7f4730079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:04.873+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4744051080 con 0x7f47540b5400 2024-07-17T08:14:05.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.071+0000 7f4753400640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 msgr2=0x7f4730079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:05.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.071+0000 7f4753400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 0x7f4730079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.117+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47540b5d70 con 0x7f47540b5400 2024-07-17T08:14:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.118+0000 7f474be00640 1 -- 172.21.15.44:0/3476122698 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7f475415ac90 con 0x7f47540b5400 2024-07-17T08:14:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:05.119 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:14:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 msgr2=0x7f4730079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 0x7f4730079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 msgr2=0x7f475415e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4744000f80 tx=0x7f4744003f10 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 shutdown_connections 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f4730076d60 0x7f4730079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47540ce5c0 0x7f4754151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540b8c80 0x7f47541f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 --2- 172.21.15.44:0/3476122698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47540b5400 0x7f475415e120 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 >> 172.21.15.44:0/3476122698 conn(0x7f47540cfec0 msgr2=0x7f47540b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 shutdown_connections 2024-07-17T08:14:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.121+0000 7f4759400640 1 -- 172.21.15.44:0/3476122698 wait complete. 2024-07-17T08:14:05.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 -- 172.21.15.44:0/1752569676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b40b2ad0 msgr2=0x7f27b40b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 --2- 172.21.15.44:0/1752569676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b40b2ad0 0x7f27b40b5200 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009900 tx=0x7f27a002a690 comp rx=0 tx=0).stop 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 -- 172.21.15.44:0/1752569676 shutdown_connections 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 --2- 172.21.15.44:0/1752569676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b40b5740 0x7f27b40cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 --2- 172.21.15.44:0/1752569676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b40b2ad0 0x7f27b40b5200 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 -- 172.21.15.44:0/1752569676 >> 172.21.15.44:0/1752569676 conn(0x7f27b40d00a0 msgr2=0x7f27b40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 -- 172.21.15.44:0/1752569676 shutdown_connections 2024-07-17T08:14:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.337+0000 7f27ba800640 1 -- 172.21.15.44:0/1752569676 wait complete. 2024-07-17T08:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.339+0000 7f27ba800640 1 Processor -- start 2024-07-17T08:14:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.339+0000 7f27ba800640 1 -- start start 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b40b5740 0x7f27b4153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b4143160 0x7f27b41429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27b40ce760 con 0x7f27b415d600 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27b40ce5e0 con 0x7f27b40b5740 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27ba800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27b40ce8e0 con 0x7f27b4143160 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48620/0 (socket says 172.21.15.44:48620) 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b2a00640 1 -- 172.21.15.44:0/889031908 learned_addr learned my addr 172.21.15.44:0/889031908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b3400640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b40b5740 0x7f27b4153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b3e00640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b4143160 0x7f27b41429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b2a00640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b4143160 msgr2=0x7f27b41429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.340+0000 7f27b2a00640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b4143160 0x7f27b41429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27b2a00640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b40b5740 msgr2=0x7f27b4153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27b2a00640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b40b5740 0x7f27b4153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27b2a00640 1 -- 172.21.15.44:0/889031908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b415f210 con 0x7f27b415d600 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27b2a00640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009900 tx=0x7f27a00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0035070 con 0x7f27b415d600 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27a0041070 con 0x7f27b415d600 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b4154a80 con 0x7f27b415d600 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0004a00 con 0x7f27b415d600 2024-07-17T08:14:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.341+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b415f4b0 con 0x7f27b415d600 2024-07-17T08:14:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.343+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b40ced90 con 0x7f27b415d600 2024-07-17T08:14:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.344+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f27a0004ba0 con 0x7f27b415d600 2024-07-17T08:14:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.345+0000 7f27ab400640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 0x7f2794079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.345+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f27a00b9910 con 0x7f27b415d600 2024-07-17T08:14:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.345+0000 7f27b3400640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 msgr2=0x7f2794079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.345+0000 7f27b3400640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 0x7f2794079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.349+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f27a00824c0 con 0x7f27b415d600 2024-07-17T08:14:05.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.547+0000 7f27b3400640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 msgr2=0x7f2794079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.547+0000 7f27b3400640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 0x7f2794079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:05.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.599+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27b4142f00 con 0x7f27b415d600 2024-07-17T08:14:05.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.600+0000 7f27ab400640 1 -- 172.21.15.44:0/889031908 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7f27a0081dc0 con 0x7f27b415d600 2024-07-17T08:14:05.601 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:05.601 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:14:05.601 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 msgr2=0x7f2794079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 0x7f2794079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 msgr2=0x7f27b4154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009900 tx=0x7f27a00038a0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 shutdown_connections 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f2794076d90 0x7f2794079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b4143160 0x7f27b41429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27b415d600 0x7f27b4154540 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 --2- 172.21.15.44:0/889031908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b40b5740 0x7f27b4153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 >> 172.21.15.44:0/889031908 conn(0x7f27b40d00a0 msgr2=0x7f27b41587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 shutdown_connections 2024-07-17T08:14:05.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.603+0000 7f27ba800640 1 -- 172.21.15.44:0/889031908 wait complete. 2024-07-17T08:14:05.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:05.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.808+0000 7fee00e00640 1 -- 172.21.15.44:0/2090905231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0709e0 msgr2=0x7fedfc0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.808+0000 7fee00e00640 1 --2- 172.21.15.44:0/2090905231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0709e0 0x7fedfc0ce110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005fd0 tx=0x7fedf402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 -- 172.21.15.44:0/2090905231 shutdown_connections 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 --2- 172.21.15.44:0/2090905231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0709e0 0x7fedfc0ce110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 --2- 172.21.15.44:0/2090905231 >> v2:172.21.15.175:3300/0 conn(0x7fedfc157da0 0x7fedfc06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 -- 172.21.15.44:0/2090905231 >> 172.21.15.44:0/2090905231 conn(0x7fedfc0b2240 msgr2=0x7fedfc0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 -- 172.21.15.44:0/2090905231 shutdown_connections 2024-07-17T08:14:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.809+0000 7fee00e00640 1 -- 172.21.15.44:0/2090905231 wait complete. 2024-07-17T08:14:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.810+0000 7fee00e00640 1 Processor -- start 2024-07-17T08:14:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 -- start start 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0afb30 0x7fedfc1f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc157da0 0x7fedfc0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc15b3e0 con 0x7fedfc0709e0 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc0ce650 con 0x7fedfc157da0 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.811+0000 7fee00e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fedfc156480 con 0x7fedfc0afb30 2024-07-17T08:14:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48632/0 (socket says 172.21.15.44:48632) 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 -- 172.21.15.44:0/3134892605 learned_addr learned my addr 172.21.15.44:0/3134892605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9400640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0afb30 0x7fedfc1f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedfa800640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc157da0 0x7fedfc0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0afb30 msgr2=0x7fedfc1f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0afb30 0x7fedfc1f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc157da0 msgr2=0x7fedfc0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc157da0 0x7fedfc0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.812+0000 7fedf9e00640 1 -- 172.21.15.44:0/3134892605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc1f7840 con 0x7fedfc0709e0 2024-07-17T08:14:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fedf9e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fedec00e9f0 tx=0x7fedec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec00cd80 con 0x7fedfc0709e0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc1f7b20 con 0x7fedfc0709e0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedec004020 con 0x7fedfc0709e0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec004470 con 0x7fedfc0709e0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.813+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1f80e0 con 0x7fedfc0709e0 2024-07-17T08:14:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.814+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc0ceb00 con 0x7fedfc0709e0 2024-07-17T08:14:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.815+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7fedec0035f0 con 0x7fedfc0709e0 2024-07-17T08:14:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.816+0000 7fedf2000640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 0x7fedd8079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.817+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7fedec0996f0 con 0x7fedfc0709e0 2024-07-17T08:14:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.817+0000 7fedf9400640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 msgr2=0x7fedd8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.817+0000 7fedf9400640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 0x7fedd8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:05.820+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fedec062220 con 0x7fedfc0709e0 2024-07-17T08:14:06.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.018+0000 7fedf9400640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 msgr2=0x7fedd8079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:06.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.019+0000 7fedf9400640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 0x7fedd8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:06.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.028+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedfc075550 con 0x7fedfc0709e0 2024-07-17T08:14:06.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.030+0000 7fedf2000640 1 -- 172.21.15.44:0/3134892605 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) ==== 74+0+18249 (secure 0 0 0) 0x7fedec061b20 con 0x7fedfc0709e0 2024-07-17T08:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":177,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:03.552532+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:12:38.365198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-07-17T08:12:39.424163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/253461221":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/2425717984":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/1618489471":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6837/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6834/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/408471903":"2024-07-18T08:13:02.263003+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 msgr2=0x7fedd8079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 0x7fedd8079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 msgr2=0x7fedfc1f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fedec00e9f0 tx=0x7fedec00eeb0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 shutdown_connections 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedfc157da0 0x7fedfc0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fedd8076d90 0x7fedd8079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedfc0afb30 0x7fedfc1f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 --2- 172.21.15.44:0/3134892605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedfc0709e0 0x7fedfc1f7260 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 >> 172.21.15.44:0/3134892605 conn(0x7fedfc0b2240 msgr2=0x7fedfc0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:06.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 shutdown_connections 2024-07-17T08:14:06.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.033+0000 7fee00e00640 1 -- 172.21.15.44:0/3134892605 wait complete. 2024-07-17T08:14:06.053 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:14:06.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 -- 172.21.15.44:0/77060961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec8073fd0 msgr2=0x7f3ec80b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 --2- 172.21.15.44:0/77060961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec8073fd0 0x7f3ec80b0610 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005fd0 tx=0x7f3ec002a820 comp rx=0 tx=0).stop 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 -- 172.21.15.44:0/77060961 shutdown_connections 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 --2- 172.21.15.44:0/77060961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec8073fd0 0x7f3ec80b0610 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 --2- 172.21.15.44:0/77060961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec80b2a00 0x7f3ec80add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 -- 172.21.15.44:0/77060961 >> 172.21.15.44:0/77060961 conn(0x7f3ec80b5890 msgr2=0x7f3ec80b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 -- 172.21.15.44:0/77060961 shutdown_connections 2024-07-17T08:14:06.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.238+0000 7f3ecca00640 1 -- 172.21.15.44:0/77060961 wait complete. 2024-07-17T08:14:06.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.240+0000 7f3ecca00640 1 Processor -- start 2024-07-17T08:14:06.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.240+0000 7f3ecca00640 1 -- start start 2024-07-17T08:14:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec8073fd0 0x7f3ec80cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ec80b2a00 0x7f3ec8150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b0cd0 con 0x7f3ec815b1d0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80b0b50 con 0x7f3ec80b2a00 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ecca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80b0e50 con 0x7f3ec8073fd0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ec7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ec7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48650/0 (socket says 172.21.15.44:48650) 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ec7200640 1 -- 172.21.15.44:0/1474439467 learned_addr learned my addr 172.21.15.44:0/1474439467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ec6800640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec8073fd0 0x7f3ec80cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.241+0000 7f3ec5e00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ec80b2a00 0x7f3ec8150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec8073fd0 msgr2=0x7f3ec80cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec8073fd0 0x7f3ec80cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ec80b2a00 msgr2=0x7f3ec8150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ec80b2a00 0x7f3ec8150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 -- 172.21.15.44:0/1474439467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8150c40 con 0x7f3ec815b1d0 2024-07-17T08:14:06.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ec7200640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009cd0 tx=0x7f3ec0032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0037040 con 0x7f3ec815b1d0 2024-07-17T08:14:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec0042070 con 0x7f3ec815b1d0 2024-07-17T08:14:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0003e60 con 0x7f3ec815b1d0 2024-07-17T08:14:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec8157940 con 0x7f3ec815b1d0 2024-07-17T08:14:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.242+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec8157f00 con 0x7f3ec815b1d0 2024-07-17T08:14:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.243+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec80cb310 con 0x7f3ec815b1d0 2024-07-17T08:14:06.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.245+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f3ec0004280 con 0x7f3ec815b1d0 2024-07-17T08:14:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.246+0000 7f3ebea00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 0x7f3ea8079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.246+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f3ec00b97f0 con 0x7f3ec815b1d0 2024-07-17T08:14:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.246+0000 7f3ec6800640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 msgr2=0x7f3ea8079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:06.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.246+0000 7f3ec6800640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 0x7f3ea8079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:06.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.250+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3ec80cb310 con 0x7f3ec815b1d0 2024-07-17T08:14:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.447+0000 7f3ec6800640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 msgr2=0x7f3ea8079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:06.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.447+0000 7f3ec6800640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 0x7f3ea8079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:06.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.497+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ec80b1300 con 0x7f3ec815b1d0 2024-07-17T08:14:06.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.498+0000 7f3ebea00640 1 -- 172.21.15.44:0/1474439467 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7f3ec8150c40 con 0x7f3ec815b1d0 2024-07-17T08:14:06.499 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:06.500 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:14:06.500 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 msgr2=0x7f3ea8079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 0x7f3ea8079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 msgr2=0x7f3ec80ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009cd0 tx=0x7f3ec0032c60 comp rx=0 tx=0).stop 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 shutdown_connections 2024-07-17T08:14:06.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ec815b1d0 0x7f3ec80ca520 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ec80b2a00 0x7f3ec8150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f3ea8076d60 0x7f3ea8079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 --2- 172.21.15.44:0/1474439467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ec8073fd0 0x7f3ec80cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 >> 172.21.15.44:0/1474439467 conn(0x7f3ec80b5890 msgr2=0x7f3ec806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 shutdown_connections 2024-07-17T08:14:06.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.501+0000 7f3ecca00640 1 -- 172.21.15.44:0/1474439467 wait complete. 2024-07-17T08:14:06.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.706+0000 7f5ab7000640 1 -- 172.21.15.44:0/115246983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 msgr2=0x7f5ab00b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.706+0000 7f5ab7000640 1 --2- 172.21.15.44:0/115246983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00b4ec0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005fd0 tx=0x7f5aa002a620 comp rx=0 tx=0).stop 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 -- 172.21.15.44:0/115246983 shutdown_connections 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 --2- 172.21.15.44:0/115246983 >> v2:172.21.15.175:3300/0 conn(0x7f5ab00b5400 0x7f5ab015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 --2- 172.21.15.44:0/115246983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00b4ec0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 -- 172.21.15.44:0/115246983 >> 172.21.15.44:0/115246983 conn(0x7f5ab00cfec0 msgr2=0x7f5ab00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 -- 172.21.15.44:0/115246983 shutdown_connections 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.707+0000 7f5ab7000640 1 -- 172.21.15.44:0/115246983 wait complete. 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.708+0000 7f5ab7000640 1 Processor -- start 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.708+0000 7f5ab7000640 1 -- start start 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 0x7f5ab015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 0x7f5ab00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab006e0c0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab00b92f0 con 0x7f5ab00b5400 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.709+0000 7f5ab7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab006e240 con 0x7f5ab00ce5c0 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48680/0 (socket says 172.21.15.44:48680) 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 -- 172.21.15.44:0/4125901542 learned_addr learned my addr 172.21.15.44:0/4125901542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5c00640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 0x7f5ab015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab6600640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 0x7f5ab00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 msgr2=0x7f5ab00cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 0x7f5ab00cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 msgr2=0x7f5ab015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 0x7f5ab015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5200640 1 -- 172.21.15.44:0/4125901542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab00c7920 con 0x7f5ab00b8c80 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab5c00640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 0x7f5ab015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.710+0000 7f5ab6600640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 0x7f5ab00cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5ab5200640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c730 tx=0x7f5aa400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4003b70 con 0x7f5ab00b8c80 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5aa4003d10 con 0x7f5ab00b8c80 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400aab0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab01593f0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.711+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab015b840 con 0x7f5ab00b8c80 2024-07-17T08:14:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.712+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab01591e0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.714+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f5aa400ac50 con 0x7f5ab00b8c80 2024-07-17T08:14:06.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.716+0000 7f5aad600640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 0x7f5a94079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.716+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f5aa4098a00 con 0x7f5ab00b8c80 2024-07-17T08:14:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.716+0000 7f5ab5c00640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 msgr2=0x7f5a94079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:06.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.716+0000 7f5ab5c00640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 0x7f5a94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:06.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.720+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5aa40616c0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.918+0000 7f5ab5c00640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 msgr2=0x7f5a94079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:06.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.918+0000 7f5ab5c00640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 0x7f5a94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:06.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.970+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab006e6f0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.971+0000 7f5aad600640 1 -- 172.21.15.44:0/4125901542 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5227 (secure 0 0 0) 0x7f5aa4060fc0 con 0x7f5ab00b8c80 2024-07-17T08:14:06.971 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:06.972 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":88,"btime":"2024-07-17T08:12:43:978967+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:12:40.436688+0000","modified":"2024-07-17T08:12:43.492715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7258":{"gid":7258,"name":"b","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/808460181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":808460181},{"type":"v1","addr":"172.21.15.175:6836","nonce":808460181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7258,"qdb_cluster":[7258]},"id":10}]} 2024-07-17T08:14:06.972 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 88 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.973+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 msgr2=0x7f5a94079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 0x7f5a94079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 msgr2=0x7f5ab00cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c730 tx=0x7f5aa400cbf0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 shutdown_connections 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f5a94076d60 0x7f5a94079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab00ce5c0 0x7f5ab00cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab00b8c80 0x7f5ab00cb560 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 --2- 172.21.15.44:0/4125901542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab00b5400 0x7f5ab015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 >> 172.21.15.44:0/4125901542 conn(0x7f5ab00cfec0 msgr2=0x7f5ab00d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:06.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 shutdown_connections 2024-07-17T08:14:06.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:06.974+0000 7f5ab7000640 1 -- 172.21.15.44:0/4125901542 wait complete. 2024-07-17T08:14:06.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:07.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 -- 172.21.15.44:0/3832102877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa754004000 msgr2=0x7fa754004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 --2- 172.21.15.44:0/3832102877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa754004000 0x7fa754004800 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa758009b50 tx=0x7fa754005830 comp rx=0 tx=0).stop 2024-07-17T08:14:07.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa76f400640 1 -- 172.21.15.44:0/3832102877 reap_dead start 2024-07-17T08:14:07.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 -- 172.21.15.44:0/3832102877 shutdown_connections 2024-07-17T08:14:07.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 -- 172.21.15.44:0/3832102877 >> 172.21.15.44:0/3832102877 conn(0x7fa7700b2240 msgr2=0x7fa7700b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:07.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 -- 172.21.15.44:0/3832102877 shutdown_connections 2024-07-17T08:14:07.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.183+0000 7fa775600640 1 -- 172.21.15.44:0/3832102877 wait complete. 2024-07-17T08:14:07.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.185+0000 7fa775600640 1 Processor -- start 2024-07-17T08:14:07.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.185+0000 7fa775600640 1 -- start start 2024-07-17T08:14:07.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa754004000 0x7fa770154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 0x7fa77014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700cee60 con 0x7fa7701523d0 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700cece0 con 0x7fa754004000 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.186+0000 7fa775600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7700cefe0 con 0x7fa770151830 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48698/0 (socket says 172.21.15.44:48698) 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 -- 172.21.15.44:0/3706597482 learned_addr learned my addr 172.21.15.44:0/3706597482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa767e00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 0x7fa77014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76ea00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa754004000 0x7fa770154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 msgr2=0x7fa77014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 0x7fa77014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa754004000 msgr2=0x7fa770154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa754004000 0x7fa770154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 -- 172.21.15.44:0/3706597482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa770154ab0 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa767e00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 0x7fa77014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.187+0000 7fa76f400640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa768004a50 tx=0x7fa76800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.188+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800b8a0 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.188+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa768018070 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.188+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800bd00 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.188+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7700ae350 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.188+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7700ae910 con 0x7fa7701523d0 2024-07-17T08:14:07.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.189+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770151c10 con 0x7fa7701523d0 2024-07-17T08:14:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.191+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7fa7680263d0 con 0x7fa7701523d0 2024-07-17T08:14:07.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.192+0000 7fa766a00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 0x7fa74c079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.192+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7fa768099510 con 0x7fa7701523d0 2024-07-17T08:14:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.192+0000 7fa76ea00640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 msgr2=0x7fa74c079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.192+0000 7fa76ea00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 0x7fa74c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:07.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.194+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa768062250 con 0x7fa7701523d0 2024-07-17T08:14:07.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.394+0000 7fa76ea00640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 msgr2=0x7fa74c079250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:07.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.394+0000 7fa76ea00640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 0x7fa74c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:07.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.407+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7700b5940 con 0x7fa7701523d0 2024-07-17T08:14:07.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.410+0000 7fa766a00640 1 -- 172.21.15.44:0/3706597482 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) ==== 74+0+18249 (secure 0 0 0) 0x7fa768061b50 con 0x7fa7701523d0 2024-07-17T08:14:07.411 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:07.411 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":177,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:03.552532+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:12:38.365198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-07-17T08:12:39.424163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/253461221":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/2425717984":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/1618489471":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6837/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6834/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/408471903":"2024-07-18T08:13:02.263003+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 msgr2=0x7fa74c079250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 0x7fa74c079250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 msgr2=0x7fa770152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa768004a50 tx=0x7fa76800dd50 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 shutdown_connections 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7701523d0 0x7fa770152850 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7fa74c076d90 0x7fa74c079250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa770151830 0x7fa77014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 --2- 172.21.15.44:0/3706597482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa754004000 0x7fa770154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 >> 172.21.15.44:0/3706597482 conn(0x7fa7700b2240 msgr2=0x7fa7700b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 shutdown_connections 2024-07-17T08:14:07.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.413+0000 7fa775600640 1 -- 172.21.15.44:0/3706597482 wait complete. 2024-07-17T08:14:07.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.629+0000 7f7d7f200640 1 -- 172.21.15.44:0/3603603475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d64003ac0 msgr2=0x7f7d64003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.629+0000 7f7d7f200640 1 --2- 172.21.15.44:0/3603603475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d64003ac0 0x7f7d64003ee0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c310 tx=0x7f7d64005830 comp rx=0 tx=0).stop 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.630+0000 7f7d7f200640 1 -- 172.21.15.44:0/3603603475 shutdown_connections 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.630+0000 7f7d7f200640 1 --2- 172.21.15.44:0/3603603475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d64003ac0 0x7f7d64003ee0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.630+0000 7f7d7f200640 1 -- 172.21.15.44:0/3603603475 >> 172.21.15.44:0/3603603475 conn(0x7f7d800b4ff0 msgr2=0x7f7d800b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.630+0000 7f7d7f200640 1 -- 172.21.15.44:0/3603603475 shutdown_connections 2024-07-17T08:14:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.630+0000 7f7d7f200640 1 -- 172.21.15.44:0/3603603475 wait complete. 2024-07-17T08:14:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.631+0000 7f7d7f200640 1 Processor -- start 2024-07-17T08:14:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.632+0000 7f7d7f200640 1 -- start start 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 0x7f7d800c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d80157110 0x7f7d801f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800b2860 con 0x7f7d801f4880 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800b26e0 con 0x7f7d64003ac0 2024-07-17T08:14:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d800cfdf0 con 0x7f7d80157110 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48714/0 (socket says 172.21.15.44:48714) 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 -- 172.21.15.44:0/4097483813 learned_addr learned my addr 172.21.15.44:0/4097483813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7e800640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d80157110 0x7f7d801f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7de00640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 0x7f7d800c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d80157110 msgr2=0x7f7d801f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d80157110 0x7f7d801f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 msgr2=0x7f7d800c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 0x7f7d800c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7d400640 1 -- 172.21.15.44:0/4097483813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d80158ce0 con 0x7f7d801f4880 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.633+0000 7f7d7de00640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 0x7f7d800c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d7d400640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004eb0 tx=0x7f7d6c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c015020 con 0x7f7d801f4880 2024-07-17T08:14:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d6c011960 con 0x7f7d801f4880 2024-07-17T08:14:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c011d90 con 0x7f7d801f4880 2024-07-17T08:14:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d800c9ed0 con 0x7f7d801f4880 2024-07-17T08:14:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.634+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d80159240 con 0x7f7d801f4880 2024-07-17T08:14:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.635+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d800d02c0 con 0x7f7d801f4880 2024-07-17T08:14:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.637+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 97) ==== 98988+0+0 (secure 0 0 0) 0x7f7d6c018020 con 0x7f7d801f4880 2024-07-17T08:14:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.638+0000 7f7d76000640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 0x7f7d60079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.638+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 6965+0+0 (secure 0 0 0) 0x7f7d6c099710 con 0x7f7d801f4880 2024-07-17T08:14:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.638+0000 7f7d7de00640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 msgr2=0x7f7d60079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:07.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.638+0000 7f7d7de00640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 0x7f7d60079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:07.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.642+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7d6c01f020 con 0x7f7d801f4880 2024-07-17T08:14:07.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.840+0000 7f7d7de00640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 msgr2=0x7f7d60079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3180589979 2024-07-17T08:14:07.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.840+0000 7f7d7de00640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 0x7f7d60079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:07.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.871+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 98) ==== 98540+0+0 (secure 0 0 0) 0x7f7d6c05de50 con 0x7f7d801f4880 2024-07-17T08:14:07.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.872+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 msgr2=0x7f7d60079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:07.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.872+0000 7f7d76000640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 0x7f7d60079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.898+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d800d0e10 con 0x7f7d801f4880 2024-07-17T08:14:07.899 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:14:07.899+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:14:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.901+0000 7f7d76000640 1 -- 172.21.15.44:0/4097483813 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f7d80158ce0 con 0x7f7d801f4880 2024-07-17T08:14:07.901 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:14:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.905+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 msgr2=0x7f7d80156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:07.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.905+0000 7f7d7f200640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004eb0 tx=0x7f7d6c001630 comp rx=0 tx=0).stop 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 shutdown_connections 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:6834/3180589979,v1:172.21.15.175:6837/3180589979] conn(0x7f7d60076d60 0x7f7d60079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d80157110 0x7f7d801f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d801f4880 0x7f7d80156bd0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 --2- 172.21.15.44:0/4097483813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d64003ac0 0x7f7d800c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 >> 172.21.15.44:0/4097483813 conn(0x7f7d800b4ff0 msgr2=0x7f7d800d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:07.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.906+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 shutdown_connections 2024-07-17T08:14:07.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:07.907+0000 7f7d7f200640 1 -- 172.21.15.44:0/4097483813 wait complete. 2024-07-17T08:14:07.917 INFO:tasks.ceph.mds.b.smithi175.stderr: -8537> 2024-07-17T08:12:46.897+0000 7fb652a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:14:07.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb063600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb063600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb063600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb063600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb063600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb068600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb068600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb068600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb068600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:07.949+0000 7fb068600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:07.977 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:14:08.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.123+0000 7f2c7d800640 1 -- 172.21.15.44:0/3943873223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c004800 msgr2=0x7f2c5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.123+0000 7f2c7d800640 1 --2- 172.21.15.44:0/3943873223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c004800 0x7f2c5c004c80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2c640016c0 tx=0x7f2c5c005830 comp rx=0 tx=0).stop 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.124+0000 7f2c7d800640 1 -- 172.21.15.44:0/3943873223 shutdown_connections 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.124+0000 7f2c7d800640 1 --2- 172.21.15.44:0/3943873223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c004800 0x7f2c5c004c80 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.124+0000 7f2c7d800640 1 -- 172.21.15.44:0/3943873223 >> 172.21.15.44:0/3943873223 conn(0x7f2c780d04c0 msgr2=0x7f2c780d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.124+0000 7f2c7d800640 1 -- 172.21.15.44:0/3943873223 shutdown_connections 2024-07-17T08:14:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.124+0000 7f2c7d800640 1 -- 172.21.15.44:0/3943873223 wait complete. 2024-07-17T08:14:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.126+0000 7f2c7d800640 1 Processor -- start 2024-07-17T08:14:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.127+0000 7f2c7d800640 1 -- start start 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c5c004800 0x7f2c78147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c78154db0 0x7f2c781ea170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78161b00 con 0x7f2c7815b3e0 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78161980 con 0x7f2c78154db0 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.128+0000 7f2c7d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780d1820 con 0x7f2c5c004800 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48740/0 (socket says 172.21.15.44:48740) 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 -- 172.21.15.44:0/2571101017 learned_addr learned my addr 172.21.15.44:0/2571101017 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c7ce00640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c78154db0 0x7f2c781ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77e00640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c5c004800 0x7f2c78147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 -- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c5c004800 msgr2=0x7f2c78147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c5c004800 0x7f2c78147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 -- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c78154db0 msgr2=0x7f2c781ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c78154db0 0x7f2c781ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.129+0000 7f2c77400640 1 -- 172.21.15.44:0/2571101017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c781e8960 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c77400640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402a910 tx=0x7f2c640063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64003a00 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c64039070 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64033420 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c781e8c40 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.130+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781e9200 con 0x7f2c7815b3e0 2024-07-17T08:14:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.132+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c78068cf0 con 0x7f2c7815b3e0 2024-07-17T08:14:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.133+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 98) ==== 98540+0+0 (secure 0 0 0) 0x7f2c64044020 con 0x7f2c7815b3e0 2024-07-17T08:14:08.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.134+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 7438+0+0 (secure 0 0 0) 0x7f2c640b7c30 con 0x7f2c7815b3e0 2024-07-17T08:14:08.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.138+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2c78068cf0 con 0x7f2c7815b3e0 2024-07-17T08:14:08.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.420+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2c780cdca0 con 0x7f2c7815b3e0 2024-07-17T08:14:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.918+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 99) ==== 98667+0+0 (secure 0 0 0) 0x7f2c6403b030 con 0x7f2c7815b3e0 2024-07-17T08:14:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.919+0000 7f2c6fe00640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 0x7f2c5807f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.921+0000 7f2c77e00640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 0x7f2c5807f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.924+0000 7f2c77e00640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 0x7f2c5807f590 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2c7806ae90 tx=0x7f2c68005f90 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:08.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.937+0000 7f2c6fe00640 1 -- 172.21.15.44:0/2571101017 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) ==== 104+0+0 (secure 0 0 0) 0x7f2c64080da0 con 0x7f2c7815b3e0 2024-07-17T08:14:08.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.941+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 msgr2=0x7f2c5807f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:08.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.941+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 0x7f2c5807f590 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2c7806ae90 tx=0x7f2c68005f90 comp rx=0 tx=0).stop 2024-07-17T08:14:08.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 msgr2=0x7f2c781e9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402a910 tx=0x7f2c640063a0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 shutdown_connections 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f2c5807d1a0 0x7f2c5807f590 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c78154db0 0x7f2c781ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c7815b3e0 0x7f2c781e9c30 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 --2- 172.21.15.44:0/2571101017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c5c004800 0x7f2c78147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 >> 172.21.15.44:0/2571101017 conn(0x7f2c780d04c0 msgr2=0x7f2c780b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 shutdown_connections 2024-07-17T08:14:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:08.942+0000 7f2c7d800640 1 -- 172.21.15.44:0/2571101017 wait complete. 2024-07-17T08:14:08.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:14:09.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.179+0000 7f0a8a600640 1 -- 172.21.15.44:0/872902043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 msgr2=0x7f0a840b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.179+0000 7f0a8a600640 1 --2- 172.21.15.44:0/872902043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b8670 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f0a840766b0 tx=0x7f0a7c02a8f0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.179+0000 7f0a8a600640 1 -- 172.21.15.44:0/872902043 shutdown_connections 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.180+0000 7f0a8a600640 1 --2- 172.21.15.44:0/872902043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b8670 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.180+0000 7f0a8a600640 1 --2- 172.21.15.44:0/872902043 >> v2:172.21.15.175:3300/0 conn(0x7f0a8415e370 0x7f0a840ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.180+0000 7f0a8a600640 1 -- 172.21.15.44:0/872902043 >> 172.21.15.44:0/872902043 conn(0x7f0a84073a60 msgr2=0x7f0a84073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.180+0000 7f0a8a600640 1 -- 172.21.15.44:0/872902043 shutdown_connections 2024-07-17T08:14:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.180+0000 7f0a8a600640 1 -- 172.21.15.44:0/872902043 wait complete. 2024-07-17T08:14:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.181+0000 7f0a8a600640 1 Processor -- start 2024-07-17T08:14:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.181+0000 7f0a8a600640 1 -- start start 2024-07-17T08:14:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a8415e370 0x7f0a8414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a84161b50 0x7f0a840c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a840b8d30 con 0x7f0a840d0b40 2024-07-17T08:14:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a840b8bb0 con 0x7f0a84161b50 2024-07-17T08:14:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.182+0000 7f0a8a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a840b8eb0 con 0x7f0a8415e370 2024-07-17T08:14:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43774/0 (socket says 172.21.15.44:43774) 2024-07-17T08:14:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 -- 172.21.15.44:0/1862390681 learned_addr learned my addr 172.21.15.44:0/1862390681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 -- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a8415e370 msgr2=0x7f0a8414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a83e00640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a8415e370 0x7f0a8414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89c00640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a84161b50 0x7f0a840c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a8415e370 0x7f0a8414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 -- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a84161b50 msgr2=0x7f0a840c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a84161b50 0x7f0a840c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 -- 172.21.15.44:0/1862390681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a840cb1b0 con 0x7f0a840d0b40 2024-07-17T08:14:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.183+0000 7f0a89200640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800b4a0 tx=0x7f0a7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.184+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78014070 con 0x7f0a840d0b40 2024-07-17T08:14:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.184+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a78010700 con 0x7f0a840d0b40 2024-07-17T08:14:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.184+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78010b50 con 0x7f0a840d0b40 2024-07-17T08:14:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.184+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a840b7190 con 0x7f0a840d0b40 2024-07-17T08:14:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.184+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a840caea0 con 0x7f0a840d0b40 2024-07-17T08:14:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.185+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a840b93a0 con 0x7f0a840d0b40 2024-07-17T08:14:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.186+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 99) ==== 98667+0+0 (secure 0 0 0) 0x7f0a78017070 con 0x7f0a840d0b40 2024-07-17T08:14:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.187+0000 7f0a80c00640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 0x7f0a68078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.187+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 7438+0+0 (secure 0 0 0) 0x7f0a78098210 con 0x7f0a840d0b40 2024-07-17T08:14:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.188+0000 7f0a83e00640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 0x7f0a68078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.189+0000 7f0a83e00640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 0x7f0a68078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0a840b2210 tx=0x7f0a74008040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.191+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0a78061290 con 0x7f0a840d0b40 2024-07-17T08:14:09.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.400+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a84069460 con 0x7f0a840d0b40 2024-07-17T08:14:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.948+0000 7f0a80c00640 1 -- 172.21.15.44:0/1862390681 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v180) ==== 181+0+0 (secure 0 0 0) 0x7f0a78060b90 con 0x7f0a840d0b40 2024-07-17T08:14:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:14:09.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 msgr2=0x7f0a68078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:09.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 0x7f0a68078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0a840b2210 tx=0x7f0a74008040 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 msgr2=0x7f0a840b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800b4a0 tx=0x7f0a7800b960 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 shutdown_connections 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f0a68076a00 0x7f0a68078ec0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0a840b2210 tx=0x7f0a74008040 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a84161b50 0x7f0a840c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a8415e370 0x7f0a8414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 --2- 172.21.15.44:0/1862390681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a840d0b40 0x7f0a840b7c70 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.955+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 >> 172.21.15.44:0/1862390681 conn(0x7f0a84073a60 msgr2=0x7f0a8415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.956+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 shutdown_connections 2024-07-17T08:14:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:09.956+0000 7f0a8a600640 1 -- 172.21.15.44:0/1862390681 wait complete. 2024-07-17T08:14:09.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 -- 172.21.15.44:0/4084618161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc80b5910 msgr2=0x7f7cc80cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 --2- 172.21.15.44:0/4084618161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc80b5910 0x7f7cc80cdcf0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005fd0 tx=0x7f7cc002a5c0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 -- 172.21.15.44:0/4084618161 shutdown_connections 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 --2- 172.21.15.44:0/4084618161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc80b5910 0x7f7cc80cdcf0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 --2- 172.21.15.44:0/4084618161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc815d600 0x7f7cc806e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 -- 172.21.15.44:0/4084618161 >> 172.21.15.44:0/4084618161 conn(0x7f7cc80d00a0 msgr2=0x7f7cc80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 -- 172.21.15.44:0/4084618161 shutdown_connections 2024-07-17T08:14:10.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.185+0000 7f7cce000640 1 -- 172.21.15.44:0/4084618161 wait complete. 2024-07-17T08:14:10.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.187+0000 7f7cce000640 1 Processor -- start 2024-07-17T08:14:10.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.187+0000 7f7cce000640 1 -- start start 2024-07-17T08:14:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 0x7f7cc814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 0x7f7cc80ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80ce6b0 con 0x7f7cc815d600 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80ce530 con 0x7f7cc80b2ad0 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cce000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80ce830 con 0x7f7cc80b5910 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43792/0 (socket says 172.21.15.44:43792) 2024-07-17T08:14:10.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 -- 172.21.15.44:0/1162318073 learned_addr learned my addr 172.21.15.44:0/1162318073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:10.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc6a00640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 0x7f7cc80ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:10.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7400640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 0x7f7cc814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 -- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 msgr2=0x7f7cc80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 0x7f7cc80ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 -- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 msgr2=0x7f7cc814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 0x7f7cc814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7e00640 1 -- 172.21.15.44:0/1162318073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc814ff60 con 0x7f7cc815d600 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc6a00640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 0x7f7cc80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.189+0000 7f7cc7400640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 0x7f7cc814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cc7e00640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009230 tx=0x7f7cc0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc003f070 con 0x7f7cc815d600 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cc0004890 con 0x7f7cc815d600 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc81fba50 con 0x7f7cc815d600 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0035030 con 0x7f7cc815d600 2024-07-17T08:14:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.190+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc81fc010 con 0x7f7cc815d600 2024-07-17T08:14:10.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.192+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc80b5ef0 con 0x7f7cc815d600 2024-07-17T08:14:10.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.193+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 99) ==== 98667+0+0 (secure 0 0 0) 0x7f7cc0042070 con 0x7f7cc815d600 2024-07-17T08:14:10.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.194+0000 7f7cbf400640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 0x7f7ca4078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:10.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.194+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 7000+0+0 (secure 0 0 0) 0x7f7cc00b9360 con 0x7f7cc815d600 2024-07-17T08:14:10.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.195+0000 7f7cc7400640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 0x7f7ca4078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:10.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.196+0000 7f7cc7400640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 0x7f7ca4078ec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005fd0 tx=0x7f7cb80082b0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.200+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7cc80b5ef0 con 0x7f7cc815d600 2024-07-17T08:14:10.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.422+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7cc806e1f0 con 0x7f7cc815d600 2024-07-17T08:14:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.606+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 100) ==== 98773+0+0 (secure 0 0 0) 0x7f7cc007e120 con 0x7f7cc815d600 2024-07-17T08:14:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.952+0000 7f7cbf400640 1 -- 172.21.15.44:0/1162318073 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v181) ==== 169+0+0 (secure 0 0 0) 0x7f7cc806e1f0 con 0x7f7cc815d600 2024-07-17T08:14:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:14:10.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.958+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 msgr2=0x7f7ca4078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.958+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 0x7f7ca4078ec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005fd0 tx=0x7f7cb80082b0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 msgr2=0x7f7cc814e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009230 tx=0x7f7cc0003960 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 shutdown_connections 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7ca4076a00 0x7f7ca4078ec0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005fd0 tx=0x7f7cb80082b0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc815d600 0x7f7cc814e470 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7cc80b5910 0x7f7cc80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 --2- 172.21.15.44:0/1162318073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc80b2ad0 0x7f7cc814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:10.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.959+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 >> 172.21.15.44:0/1162318073 conn(0x7f7cc80d00a0 msgr2=0x7f7cc80b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.960+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 shutdown_connections 2024-07-17T08:14:10.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:10.960+0000 7f7cce000640 1 -- 172.21.15.44:0/1162318073 wait complete. 2024-07-17T08:14:10.983 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:14:11.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 -- 172.21.15.44:0/1906913421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 msgr2=0x7fbdf806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 --2- 172.21.15.44:0/1906913421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf806d620 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fbdf815b870 tx=0x7fbde802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 -- 172.21.15.44:0/1906913421 shutdown_connections 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 --2- 172.21.15.44:0/1906913421 >> v2:172.21.15.175:3300/0 conn(0x7fbdf806ed30 0x7fbdf815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 --2- 172.21.15.44:0/1906913421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf806d620 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.203+0000 7fbe00800640 1 -- 172.21.15.44:0/1906913421 >> 172.21.15.44:0/1906913421 conn(0x7fbdf80b0eb0 msgr2=0x7fbdf80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.204+0000 7fbe00800640 1 -- 172.21.15.44:0/1906913421 shutdown_connections 2024-07-17T08:14:11.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.204+0000 7fbe00800640 1 -- 172.21.15.44:0/1906913421 wait complete. 2024-07-17T08:14:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.205+0000 7fbe00800640 1 Processor -- start 2024-07-17T08:14:11.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.205+0000 7fbe00800640 1 -- start start 2024-07-17T08:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 0x7fbdf81503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 0x7fbdf8073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cd780 con 0x7fbdf80cbab0 2024-07-17T08:14:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cd600 con 0x7fbdf806ed30 2024-07-17T08:14:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.206+0000 7fbe00800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf80cd900 con 0x7fbdf8152220 2024-07-17T08:14:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43808/0 (socket says 172.21.15.44:43808) 2024-07-17T08:14:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 -- 172.21.15.44:0/3352392702 learned_addr learned my addr 172.21.15.44:0/3352392702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfde00640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 0x7fbdf81503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfe800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 0x7fbdf8073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 -- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 msgr2=0x7fbdf8073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 0x7fbdf8073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 -- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 msgr2=0x7fbdf81503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 0x7fbdf81503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfd400640 1 -- 172.21.15.44:0/3352392702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf80ac230 con 0x7fbdf80cbab0 2024-07-17T08:14:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfde00640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 0x7fbdf81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.207+0000 7fbdfe800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 0x7fbdf8073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbdfd400640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098f0 tx=0x7fbde8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8040070 con 0x7fbdf80cbab0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbde8005310 con 0x7fbdf80cbab0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8005c90 con 0x7fbdf80cbab0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf8073b80 con 0x7fbdf80cbab0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.208+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf8073870 con 0x7fbdf80cbab0 2024-07-17T08:14:11.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.210+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf80cde20 con 0x7fbdf80cbab0 2024-07-17T08:14:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.210+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 100) ==== 98773+0+0 (secure 0 0 0) 0x7fbde8039040 con 0x7fbdf80cbab0 2024-07-17T08:14:11.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.211+0000 7fbdf2000640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 0x7fbddc078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.212+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7fbde80b9720 con 0x7fbdf80cbab0 2024-07-17T08:14:11.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.212+0000 7fbdfde00640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 0x7fbddc078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.213+0000 7fbdfde00640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 0x7fbddc078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbdec001220 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:11.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.216+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbde80829b0 con 0x7fbdf80cbab0 2024-07-17T08:14:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.427+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fbdf8155a30 con 0x7fbdf80cbab0 2024-07-17T08:14:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.436+0000 7fbdf2000640 1 -- 172.21.15.44:0/3352392702 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v63) ==== 71+0+0 (secure 0 0 0) 0x7fbde80822b0 con 0x7fbdf80cbab0 2024-07-17T08:14:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 msgr2=0x7fbddc078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 0x7fbddc078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbdec001220 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2024-07-17T08:14:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 msgr2=0x7fbdf80accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098f0 tx=0x7fbde8032c10 comp rx=0 tx=0).stop 2024-07-17T08:14:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 shutdown_connections 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbddc076b20 0x7fbddc078fe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdf8152220 0x7fbdf8073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdf80cbab0 0x7fbdf80accd0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 --2- 172.21.15.44:0/3352392702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf806ed30 0x7fbdf81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.439+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 >> 172.21.15.44:0/3352392702 conn(0x7fbdf80b0eb0 msgr2=0x7fbdf80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.440+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 shutdown_connections 2024-07-17T08:14:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.440+0000 7fbe00800640 1 -- 172.21.15.44:0/3352392702 wait complete. 2024-07-17T08:14:11.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.672+0000 7f7468e00640 1 -- 172.21.15.44:0/458747604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74640ac840 msgr2=0x7f746413f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.672+0000 7f7468e00640 1 --2- 172.21.15.44:0/458747604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74640ac840 0x7f746413f470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f745c005fd0 tx=0x7f745c02bbc0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 -- 172.21.15.44:0/458747604 shutdown_connections 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 --2- 172.21.15.44:0/458747604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74640ac840 0x7f746413f470 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 --2- 172.21.15.44:0/458747604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f746406f710 0x7f7464153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 -- 172.21.15.44:0/458747604 >> 172.21.15.44:0/458747604 conn(0x7f74640b0fb0 msgr2=0x7f74640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 -- 172.21.15.44:0/458747604 shutdown_connections 2024-07-17T08:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.673+0000 7f7468e00640 1 -- 172.21.15.44:0/458747604 wait complete. 2024-07-17T08:14:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.675+0000 7f7468e00640 1 Processor -- start 2024-07-17T08:14:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.675+0000 7f7468e00640 1 -- start start 2024-07-17T08:14:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.676+0000 7f7468e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f746406f710 0x7f74641423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.676+0000 7f7468e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74640ac3e0 0x7f74641e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.676+0000 7f7468e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7468e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74640ae800 con 0x7f74640ac840 2024-07-17T08:14:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7468e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f746406f0a0 con 0x7f746406f710 2024-07-17T08:14:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7468e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74640ae980 con 0x7f74640ac3e0 2024-07-17T08:14:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43810/0 (socket says 172.21.15.44:43810) 2024-07-17T08:14:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 -- 172.21.15.44:0/2563645519 learned_addr learned my addr 172.21.15.44:0/2563645519 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7461e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f746406f710 0x7f74641423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f745be00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74640ac3e0 0x7f74641e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 -- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74640ac3e0 msgr2=0x7f74641e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74640ac3e0 0x7f74641e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 -- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f746406f710 msgr2=0x7f74641423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f746406f710 0x7f74641423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.677+0000 7f7462800640 1 -- 172.21.15.44:0/2563645519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74640b4310 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f7462800640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f745c00ade0 tx=0x7f745c02d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c041020 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745c004250 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c004660 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74640b3270 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.678+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74640b3830 con 0x7f74640ac840 2024-07-17T08:14:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.679+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746413fad0 con 0x7f74640ac840 2024-07-17T08:14:11.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.681+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 100) ==== 98773+0+0 (secure 0 0 0) 0x7f745c044020 con 0x7f74640ac840 2024-07-17T08:14:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.682+0000 7f745a000640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 0x7f7440078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.682+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7f745c0b9ba0 con 0x7f74640ac840 2024-07-17T08:14:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.683+0000 7f7461e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 0x7f7440078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.684+0000 7f7461e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 0x7f7440078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7454005fd0 tx=0x7f7454009260 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:11.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.686+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f745c04b020 con 0x7f74640ac840 2024-07-17T08:14:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:11.970+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) -- 0x7f74641e9880 con 0x7f74640ac840 2024-07-17T08:14:12.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.637+0000 7f745a000640 1 -- 172.21.15.44:0/2563645519 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2593) ==== 174+0+0 (secure 0 0 0) 0x7f745c082e30 con 0x7f74640ac840 2024-07-17T08:14:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.641+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 msgr2=0x7f7440078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.641+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 0x7f7440078fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7454005fd0 tx=0x7f7454009260 comp rx=0 tx=0).stop 2024-07-17T08:14:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.641+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 msgr2=0x7f74641e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.641+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f745c00ade0 tx=0x7f745c02d950 comp rx=0 tx=0).stop 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 shutdown_connections 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7440076b20 0x7f7440078fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74640ac840 0x7f74641e9090 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74640ac3e0 0x7f74641e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 --2- 172.21.15.44:0/2563645519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f746406f710 0x7f74641423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 >> 172.21.15.44:0/2563645519 conn(0x7f74640b0fb0 msgr2=0x7f74640af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 shutdown_connections 2024-07-17T08:14:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.642+0000 7f7468e00640 1 -- 172.21.15.44:0/2563645519 wait complete. 2024-07-17T08:14:12.664 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2024-07-17T08:14:12.664 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2024-07-17T08:14:12.664 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T08:14:12.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 -- 172.21.15.44:0/2420077446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 msgr2=0x7f1de8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 --2- 172.21.15.44:0/2420077446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1de8003e70 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1df4006f80 tx=0x7f1de8005830 comp rx=0 tx=0).stop 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e02800640 1 -- 172.21.15.44:0/2420077446 reap_dead start 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 -- 172.21.15.44:0/2420077446 shutdown_connections 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 -- 172.21.15.44:0/2420077446 >> 172.21.15.44:0/2420077446 conn(0x7f1e040b0fb0 msgr2=0x7f1e040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 -- 172.21.15.44:0/2420077446 shutdown_connections 2024-07-17T08:14:12.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.863+0000 7f1e08e00640 1 -- 172.21.15.44:0/2420077446 wait complete. 2024-07-17T08:14:12.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.865+0000 7f1e08e00640 1 Processor -- start 2024-07-17T08:14:12.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.865+0000 7f1e08e00640 1 -- start start 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e04144190 0x7f1e041f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e041f76d0 0x7f1e041f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e040cdca0 con 0x7f1de8003a50 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e040cdb20 con 0x7f1e04144190 2024-07-17T08:14:12.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e08e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e040cde20 con 0x7f1e041f76d0 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43834/0 (socket says 172.21.15.44:43834) 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.866+0000 7f1e01e00640 1 -- 172.21.15.44:0/117998128 learned_addr learned my addr 172.21.15.44:0/117998128 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1dfbe00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e04144190 0x7f1e041f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 -- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e041f76d0 msgr2=0x7f1e041f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e041f76d0 0x7f1e041f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 -- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e04144190 msgr2=0x7f1e041f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e04144190 0x7f1e041f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:12.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 -- 172.21.15.44:0/117998128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e04072810 con 0x7f1de8003a50 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e01e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c350 tx=0x7f1df4006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4032030 con 0x7f1de8003a50 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1df4003af0 con 0x7f1de8003a50 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e04072af0 con 0x7f1de8003a50 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.867+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4042430 con 0x7f1de8003a50 2024-07-17T08:14:12.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.868+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e040730b0 con 0x7f1de8003a50 2024-07-17T08:14:12.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.869+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e041f7af0 con 0x7f1de8003a50 2024-07-17T08:14:12.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.870+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f1df4035020 con 0x7f1de8003a50 2024-07-17T08:14:12.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.871+0000 7f1dfa000640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 0x7f1de0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:12.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.872+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7f1df40b8940 con 0x7f1de8003a50 2024-07-17T08:14:12.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.872+0000 7f1dfbe00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 0x7f1de0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:12.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.873+0000 7f1dfbe00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 0x7f1de0079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e041472e0 tx=0x7f1dec005f90 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:12.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:12.876+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1df4047030 con 0x7f1de8003a50 2024-07-17T08:14:13.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.090+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e040ac690 con 0x7f1de8003a50 2024-07-17T08:14:13.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.091+0000 7f1dfa000640 1 -- 172.21.15.44:0/117998128 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+15076 (secure 0 0 0) 0x7f1df4081a60 con 0x7f1de8003a50 2024-07-17T08:14:13.092 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:13.092 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:10.948429+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6801/808460181":"2024-07-18T08:14:07.899625+0000","172.21.15.175:0/1391523042":"2024-07-18T08:14:07.854030+0000","172.21.15.175:6836/808460181":"2024-07-18T08:14:07.899625+0000","172.21.15.175:6837/3180589979":"2024-07-18T08:14:07.854030+0000","172.21.15.175:6834/3180589979":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/4228999230":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/3232933147":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/3973287790":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/543236369":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/1266546946":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/4056293303":"2024-07-18T08:14:07.854030+0000","172.21.15.175:0/253461221":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/2425717984":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/1618489471":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6837/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:6834/3617669990":"2024-07-18T08:13:02.263003+0000","172.21.15.175:0/408471903":"2024-07-18T08:13:02.263003+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 msgr2=0x7f1de0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 0x7f1de0079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e041472e0 tx=0x7f1dec005f90 comp rx=0 tx=0).stop 2024-07-17T08:14:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 msgr2=0x7f1e040c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c350 tx=0x7f1df4006c30 comp rx=0 tx=0).stop 2024-07-17T08:14:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 shutdown_connections 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f1de0076bb0 0x7f1de0079070 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e041f76d0 0x7f1e041f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e04144190 0x7f1e041f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 --2- 172.21.15.44:0/117998128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1de8003a50 0x7f1e040c86e0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 >> 172.21.15.44:0/117998128 conn(0x7f1e040b0fb0 msgr2=0x7f1e040afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 shutdown_connections 2024-07-17T08:14:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.094+0000 7f1e08e00640 1 -- 172.21.15.44:0/117998128 wait complete. 2024-07-17T08:14:13.114 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:14:13.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:14:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 -- 172.21.15.44:0/3153415212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf200ac190 msgr2=0x7fcf200ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 --2- 172.21.15.44:0/3153415212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf200ac190 0x7fcf200ac600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005fd0 tx=0x7fcf1802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 -- 172.21.15.44:0/3153415212 shutdown_connections 2024-07-17T08:14:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 --2- 172.21.15.44:0/3153415212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf200ac190 0x7fcf200ac600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 --2- 172.21.15.44:0/3153415212 >> v2:172.21.15.175:3300/0 conn(0x7fcf20154600 0x7fcf200b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 -- 172.21.15.44:0/3153415212 >> 172.21.15.44:0/3153415212 conn(0x7fcf200b0eb0 msgr2=0x7fcf200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 -- 172.21.15.44:0/3153415212 shutdown_connections 2024-07-17T08:14:13.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.331+0000 7fcf26600640 1 -- 172.21.15.44:0/3153415212 wait complete. 2024-07-17T08:14:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.333+0000 7fcf26600640 1 Processor -- start 2024-07-17T08:14:13.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.334+0000 7fcf26600640 1 -- start start 2024-07-17T08:14:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf20154600 0x7fcf201f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 0x7fcf20145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2015e220 con 0x7fcf200ac190 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2015e0a0 con 0x7fcf20161120 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf26600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2015e3a0 con 0x7fcf20154600 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43850/0 (socket says 172.21.15.44:43850) 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.335+0000 7fcf1f400640 1 -- 172.21.15.44:0/2818461097 learned_addr learned my addr 172.21.15.44:0/2818461097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 0x7fcf20145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf20154600 0x7fcf201f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 -- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf20154600 msgr2=0x7fcf201f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf20154600 0x7fcf201f7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 -- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 msgr2=0x7fcf20145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 0x7fcf20145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 -- 172.21.15.44:0/2818461097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf201569c0 con 0x7fcf200ac190 2024-07-17T08:14:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 0x7fcf20145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf1f400640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9c0 tx=0x7fcf1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000ccc0 con 0x7fcf200ac190 2024-07-17T08:14:13.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf10005020 con 0x7fcf200ac190 2024-07-17T08:14:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000ce60 con 0x7fcf200ac190 2024-07-17T08:14:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.336+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf20145fb0 con 0x7fcf200ac190 2024-07-17T08:14:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.337+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf20156c40 con 0x7fcf200ac190 2024-07-17T08:14:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.338+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2015e930 con 0x7fcf200ac190 2024-07-17T08:14:13.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.339+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fcf10005450 con 0x7fcf200ac190 2024-07-17T08:14:13.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.340+0000 7fcf17400640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 0x7fcf00079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.340+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7fcf10098f20 con 0x7fcf200ac190 2024-07-17T08:14:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.341+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 0x7fcf00079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.342+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 0x7fcf00079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf20074140 tx=0x7fcf0c009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.344+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcf201569c0 con 0x7fcf200ac190 2024-07-17T08:14:13.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.555+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf2015f7c0 con 0x7fcf200ac190 2024-07-17T08:14:13.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.556+0000 7fcf17400640 1 -- 172.21.15.44:0/2818461097 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) ==== 93+0+10 (secure 0 0 0) 0x7fcf100620d0 con 0x7fcf200ac190 2024-07-17T08:14:13.557 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 msgr2=0x7fcf00079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 0x7fcf00079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf20074140 tx=0x7fcf0c009040 comp rx=0 tx=0).stop 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 msgr2=0x7fcf20072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9c0 tx=0x7fcf1000ee80 comp rx=0 tx=0).stop 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 shutdown_connections 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fcf00076bb0 0x7fcf00079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf20161120 0x7fcf20145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf20154600 0x7fcf201f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 --2- 172.21.15.44:0/2818461097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ac190 0x7fcf20072630 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 >> 172.21.15.44:0/2818461097 conn(0x7fcf200b0eb0 msgr2=0x7fcf200b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 shutdown_connections 2024-07-17T08:14:13.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.559+0000 7fcf26600640 1 -- 172.21.15.44:0/2818461097 wait complete. 2024-07-17T08:14:13.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 -- 172.21.15.44:0/521765573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc288004800 msgr2=0x7fc288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/521765573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc288004800 0x7fc288004c80 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006f30 tx=0x7fc288005830 comp rx=0 tx=0).stop 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 -- 172.21.15.44:0/521765573 shutdown_connections 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/521765573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc288004800 0x7fc288004c80 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 -- 172.21.15.44:0/521765573 >> 172.21.15.44:0/521765573 conn(0x7fc2a00b0fb0 msgr2=0x7fc2a00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:13.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 -- 172.21.15.44:0/521765573 shutdown_connections 2024-07-17T08:14:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.783+0000 7fc2a7e00640 1 -- 172.21.15.44:0/521765573 wait complete. 2024-07-17T08:14:13.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.785+0000 7fc2a7e00640 1 Processor -- start 2024-07-17T08:14:13.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.785+0000 7fc2a7e00640 1 -- start start 2024-07-17T08:14:13.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.786+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc288004800 0x7fc2a014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.786+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.786+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a015acf0 0x7fc2a015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a00ad910 con 0x7fc2a01544b0 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a00ad790 con 0x7fc2a015acf0 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a00ada90 con 0x7fc288004800 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43870/0 (socket says 172.21.15.44:43870) 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 -- 172.21.15.44:0/683901871 learned_addr learned my addr 172.21.15.44:0/683901871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a5e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a015acf0 0x7fc2a015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a5400640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc288004800 0x7fc2a014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 -- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc288004800 msgr2=0x7fc2a014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc288004800 0x7fc2a014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 -- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a015acf0 msgr2=0x7fc2a015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a015acf0 0x7fc2a015b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.787+0000 7fc2a4a00640 1 -- 172.21.15.44:0/683901871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a014acb0 con 0x7fc2a01544b0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc2a4a00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006f30 tx=0x7fc28c001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c032030 con 0x7fc2a01544b0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc28c003af0 con 0x7fc2a01544b0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c038430 con 0x7fc2a01544b0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a00b2600 con 0x7fc2a01544b0 2024-07-17T08:14:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.788+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a00b2bc0 con 0x7fc2a01544b0 2024-07-17T08:14:13.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.790+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a015ee90 con 0x7fc2a01544b0 2024-07-17T08:14:13.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.791+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fc28c035020 con 0x7fc2a01544b0 2024-07-17T08:14:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.792+0000 7fc29d600640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 0x7fc2800790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:13.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.792+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7fc28c0b8470 con 0x7fc2a01544b0 2024-07-17T08:14:13.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.792+0000 7fc2a5400640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 0x7fc2800790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:13.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.794+0000 7fc2a5400640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 0x7fc2800790f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc294000c00 tx=0x7fc294006a80 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:13.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:13.796+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc28c0816a0 con 0x7fc2a01544b0 2024-07-17T08:14:14.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.083+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) -- 0x7fc2a015caa0 con 0x7fc2a01544b0 2024-07-17T08:14:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.650+0000 7fc29d600640 1 -- 172.21.15.44:0/683901871 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2595) ==== 168+0+0 (secure 0 0 0) 0x7fc28c080fa0 con 0x7fc2a01544b0 2024-07-17T08:14:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.653+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 msgr2=0x7fc2800790f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.653+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 0x7fc2800790f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc294000c00 tx=0x7fc294006a80 comp rx=0 tx=0).stop 2024-07-17T08:14:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.653+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 msgr2=0x7fc2a0154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:14.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.653+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006f30 tx=0x7fc28c001d10 comp rx=0 tx=0).stop 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 shutdown_connections 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fc280076c30 0x7fc2800790f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a015acf0 0x7fc2a015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2a01544b0 0x7fc2a0154930 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 --2- 172.21.15.44:0/683901871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc288004800 0x7fc2a014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 >> 172.21.15.44:0/683901871 conn(0x7fc2a00b0fb0 msgr2=0x7fc2a006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 shutdown_connections 2024-07-17T08:14:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.654+0000 7fc2a7e00640 1 -- 172.21.15.44:0/683901871 wait complete. 2024-07-17T08:14:14.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:14:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.887+0000 7f7455c00640 1 -- 172.21.15.44:0/3681425474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745015d430 msgr2=0x7f74500ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.887+0000 7f7455c00640 1 --2- 172.21.15.44:0/3681425474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745015d430 0x7f74500ceb70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7450071030 tx=0x7f744802bb10 comp rx=0 tx=0).stop 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 -- 172.21.15.44:0/3681425474 shutdown_connections 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 --2- 172.21.15.44:0/3681425474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745015d430 0x7f74500ceb70 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 --2- 172.21.15.44:0/3681425474 >> v2:172.21.15.175:3300/0 conn(0x7f745006f750 0x7f7450071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 -- 172.21.15.44:0/3681425474 >> 172.21.15.44:0/3681425474 conn(0x7f745006bf40 msgr2=0x7f745006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 -- 172.21.15.44:0/3681425474 shutdown_connections 2024-07-17T08:14:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.888+0000 7f7455c00640 1 -- 172.21.15.44:0/3681425474 wait complete. 2024-07-17T08:14:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.889+0000 7f7455c00640 1 Processor -- start 2024-07-17T08:14:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.890+0000 7f7455c00640 1 -- start start 2024-07-17T08:14:14.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.890+0000 7f7455c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:14.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.890+0000 7f7455c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745015d000 0x7f74501f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:14.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f7455c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745015d430 0x7f7450158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:14.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f7455c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74500b31d0 con 0x7f745006f750 2024-07-17T08:14:14.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f7455c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f745015d9b0 con 0x7f745015d000 2024-07-17T08:14:14.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f7455c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74500b3350 con 0x7f745015d430 2024-07-17T08:14:14.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:14.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43890/0 (socket says 172.21.15.44:43890) 2024-07-17T08:14:14.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 -- 172.21.15.44:0/2473808731 learned_addr learned my addr 172.21.15.44:0/2473808731 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:14.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744f400640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745015d430 0x7f7450158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:14.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744e000640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745015d000 0x7f74501f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:14.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 -- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745015d430 msgr2=0x7f7450158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:14.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745015d430 0x7f7450158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 -- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745015d000 msgr2=0x7f74501f1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745015d000 0x7f74501f1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.891+0000 7f744ea00640 1 -- 172.21.15.44:0/2473808731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7450151cc0 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f744ea00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440017070 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7440009cc0 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400106c0 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7450151fa0 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.892+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7450152560 con 0x7f745006f750 2024-07-17T08:14:14.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.893+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74500b3800 con 0x7f745006f750 2024-07-17T08:14:14.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.895+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f7440010860 con 0x7f745006f750 2024-07-17T08:14:14.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.896+0000 7f7446a00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 0x7f742c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:14.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.896+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7f7440098910 con 0x7f745006f750 2024-07-17T08:14:14.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.896+0000 7f744e000640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 0x7f742c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:14.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.897+0000 7f744e000640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 0x7f742c0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7450073e20 tx=0x7f7438009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:14.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:14.900+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7440061ac0 con 0x7f745006f750 2024-07-17T08:14:15.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.119+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7450073900 con 0x7f745006f750 2024-07-17T08:14:15.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.120+0000 7f7446a00640 1 -- 172.21.15.44:0/2473808731 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v63) ==== 59+0+1998 (secure 0 0 0) 0x7f74400613c0 con 0x7f745006f750 2024-07-17T08:14:15.121 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:14:15.121 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:14:15.121 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:14:15.122 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:14:15.123 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:14:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:14:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:14:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:14:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:14:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 msgr2=0x7f742c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 0x7f742c0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7450073e20 tx=0x7f7438009040 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 msgr2=0x7f74501515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 shutdown_connections 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f742c076be0 0x7f742c0790a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745015d430 0x7f7450158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745015d000 0x7f74501f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 --2- 172.21.15.44:0/2473808731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f745006f750 0x7f74501515e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 >> 172.21.15.44:0/2473808731 conn(0x7f745006bf40 msgr2=0x7f7450150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.124+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 shutdown_connections 2024-07-17T08:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.125+0000 7f7455c00640 1 -- 172.21.15.44:0/2473808731 wait complete. 2024-07-17T08:14:15.146 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:14:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 -- 172.21.15.44:0/823174787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600ac8a0 msgr2=0x7f796015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 --2- 172.21.15.44:0/823174787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600ac8a0 0x7f796015ac10 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f79600d0260 tx=0x7f795802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 -- 172.21.15.44:0/823174787 shutdown_connections 2024-07-17T08:14:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 --2- 172.21.15.44:0/823174787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79600ac8a0 0x7f796015ac10 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 --2- 172.21.15.44:0/823174787 >> v2:172.21.15.175:3300/0 conn(0x7f796015a280 0x7f79600b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 -- 172.21.15.44:0/823174787 >> 172.21.15.44:0/823174787 conn(0x7f79600b0eb0 msgr2=0x7f79600b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 -- 172.21.15.44:0/823174787 shutdown_connections 2024-07-17T08:14:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.358+0000 7f7966e00640 1 -- 172.21.15.44:0/823174787 wait complete. 2024-07-17T08:14:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.360+0000 7f7966e00640 1 Processor -- start 2024-07-17T08:14:15.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.360+0000 7f7966e00640 1 -- start start 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600ac8a0 0x7f79601537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 0x7f79600ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f796006d760 con 0x7f796015a280 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f796006f1d0 con 0x7f79600b1af0 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7966e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f796015b150 con 0x7f79600ac8a0 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.361+0000 7f7964e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43906/0 (socket says 172.21.15.44:43906) 2024-07-17T08:14:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 -- 172.21.15.44:0/1760685381 learned_addr learned my addr 172.21.15.44:0/1760685381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f795f400640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 0x7f79600ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f795fe00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600ac8a0 0x7f79601537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 -- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600ac8a0 msgr2=0x7f79601537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600ac8a0 0x7f79601537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 -- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 msgr2=0x7f79600ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 0x7f79600ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 -- 172.21.15.44:0/1760685381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7960158ea0 con 0x7f796015a280 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f795f400640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 0x7f79600ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.362+0000 7f7964e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7958009ca0 tx=0x7f7958031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.363+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958040070 con 0x7f796015a280 2024-07-17T08:14:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.363+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7958004040 con 0x7f796015a280 2024-07-17T08:14:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.363+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958004470 con 0x7f796015a280 2024-07-17T08:14:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.363+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7960159180 con 0x7f796015a280 2024-07-17T08:14:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.363+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7960159740 con 0x7f796015a280 2024-07-17T08:14:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.364+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79600ae320 con 0x7f796015a280 2024-07-17T08:14:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.366+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f7958039040 con 0x7f796015a280 2024-07-17T08:14:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.367+0000 7f7957e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 0x7f79400790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.367+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7f79580b9bb0 con 0x7f796015a280 2024-07-17T08:14:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.367+0000 7f795fe00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 0x7f79400790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.369+0000 7f795fe00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 0x7f79400790c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f79500058c0 tx=0x7f7950005850 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.371+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f79600ae320 con 0x7f796015a280 2024-07-17T08:14:15.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.581+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f796015b6e0 con 0x7f796015a280 2024-07-17T08:14:15.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.589+0000 7f7957e00640 1 -- 172.21.15.44:0/1760685381 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7f7958082de0 con 0x7f796015a280 2024-07-17T08:14:15.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 msgr2=0x7f79400790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 0x7f79400790c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f79500058c0 tx=0x7f7950005850 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 msgr2=0x7f79600ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7958009ca0 tx=0x7f7958031720 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 shutdown_connections 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7940076c00 0x7f79400790c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796015a280 0x7f79600ad6d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1af0 0x7f79600ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 --2- 172.21.15.44:0/1760685381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600ac8a0 0x7f79601537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 >> 172.21.15.44:0/1760685381 conn(0x7f79600b0eb0 msgr2=0x7f79600cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:15.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.592+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 shutdown_connections 2024-07-17T08:14:15.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:15.593+0000 7f7966e00640 1 -- 172.21.15.44:0/1760685381 wait complete. 2024-07-17T08:14:15.615 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:14:15.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:14:15.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:14:15.838 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:14:15.839 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:15.839 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:15.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:14:15.896 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:14:15.897 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:15.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:14:15.953 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:15.953 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:16.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:14:16.011 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:14:16.011 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:16.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:14:16.065 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:14:16.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:16.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 -- 172.21.15.44:0/4025200273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9740cf380 msgr2=0x7fd9740cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 --2- 172.21.15.44:0/4025200273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9740cf380 0x7fd9740cf810 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd960005fd0 tx=0x7fd96002a580 comp rx=0 tx=0).stop 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 -- 172.21.15.44:0/4025200273 shutdown_connections 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 --2- 172.21.15.44:0/4025200273 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd974159800 0x7fd974159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 --2- 172.21.15.44:0/4025200273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9740cf380 0x7fd9740cf810 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 --2- 172.21.15.44:0/4025200273 >> v2:172.21.15.175:3300/0 conn(0x7fd97406f4b0 0x7fd97406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.290+0000 7fd97aa00640 1 -- 172.21.15.44:0/4025200273 >> 172.21.15.44:0/4025200273 conn(0x7fd9740b0e80 msgr2=0x7fd9740b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.291+0000 7fd97aa00640 1 -- 172.21.15.44:0/4025200273 shutdown_connections 2024-07-17T08:14:16.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.291+0000 7fd97aa00640 1 -- 172.21.15.44:0/4025200273 wait complete. 2024-07-17T08:14:16.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.293+0000 7fd97aa00640 1 Processor -- start 2024-07-17T08:14:16.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.293+0000 7fd97aa00640 1 -- start start 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97406f4b0 0x7fd9741f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9740cf380 0x7fd974165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd97409b880 con 0x7fd974159800 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740b1fa0 con 0x7fd9740cf380 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9740cc6a0 con 0x7fd97406f4b0 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43926/0 (socket says 172.21.15.44:43926) 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97a000640 1 -- 172.21.15.44:0/337754976 learned_addr learned my addr 172.21.15.44:0/337754976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd979600640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97406f4b0 0x7fd9741f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd978c00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9740cf380 0x7fd974165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97a000640 1 -- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97406f4b0 msgr2=0x7fd9741f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.295+0000 7fd97a000640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97406f4b0 0x7fd9741f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97a000640 1 -- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9740cf380 msgr2=0x7fd974165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97a000640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9740cf380 0x7fd974165060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97a000640 1 -- 172.21.15.44:0/337754976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd974165a40 con 0x7fd974159800 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97a000640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00cad0 tx=0x7fd96c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00ae20 con 0x7fd974159800 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd96c00f450 con 0x7fd974159800 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00f8c0 con 0x7fd974159800 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd974165d20 con 0x7fd974159800 2024-07-17T08:14:16.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.296+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd974166000 con 0x7fd974159800 2024-07-17T08:14:16.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.298+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9740cf810 con 0x7fd974159800 2024-07-17T08:14:16.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.299+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fd96c00fa80 con 0x7fd974159800 2024-07-17T08:14:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.300+0000 7fd971600640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 0x7fd954079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.300+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7fd96c099100 con 0x7fd974159800 2024-07-17T08:14:16.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.301+0000 7fd979600640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 0x7fd954079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.302+0000 7fd979600640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 0x7fd954079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd968005fd0 tx=0x7fd968005dc0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:16.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.305+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd96c014030 con 0x7fd974159800 2024-07-17T08:14:16.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.557+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97415c810 con 0x7fd974159800 2024-07-17T08:14:16.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.559+0000 7fd971600640 1 -- 172.21.15.44:0/337754976 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+3914 (secure 0 0 0) 0x7fd96c0622a0 con 0x7fd974159800 2024-07-17T08:14:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:16.561 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":90,"btime":"2024-07-17T08:14:08:905330+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6773,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/309860854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":309860854},{"type":"v1","addr":"172.21.15.44:6834","nonce":309860854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6776,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1728343648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1728343648},{"type":"v1","addr":"172.21.15.44:6835","nonce":1728343648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3496039092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3496039092},{"type":"v1","addr":"172.21.15.175:6835","nonce":3496039092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1179426860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1179426860},{"type":"v1","addr":"172.21.15.175:6836","nonce":1179426860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-07-17T08:14:16.561 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 90 2024-07-17T08:14:16.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.561+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 msgr2=0x7fd954079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.561+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 0x7fd954079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd968005fd0 tx=0x7fd968005dc0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.561+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 msgr2=0x7fd974073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.561+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00cad0 tx=0x7fd96c00cf90 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 shutdown_connections 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fd954076bb0 0x7fd954079070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd974159800 0x7fd974073590 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9740cf380 0x7fd974165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 --2- 172.21.15.44:0/337754976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd97406f4b0 0x7fd9741f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 >> 172.21.15.44:0/337754976 conn(0x7fd9740b0e80 msgr2=0x7fd9740adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 shutdown_connections 2024-07-17T08:14:16.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.562+0000 7fd97aa00640 1 -- 172.21.15.44:0/337754976 wait complete. 2024-07-17T08:14:16.584 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:14:16.585 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:14:16.585 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:14:16.585 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:14:16.585 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:14:16.585 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:14:16.586 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:14:16.586 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:14:16.586 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:14:16.586 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:14:16.586 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:14:16.586 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:14:16.587 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:14:16.587 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:14:16.587 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:14:16.587 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:14:16.608 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:14:16.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:14:16.609 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:14:16.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:14:16.610 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:14:16.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:14:16.611 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:14:16.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:14:16.656 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:14:16.656 DEBUG:teuthology.parallel:result is None 2024-07-17T08:14:16.658 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:14:16.658 DEBUG:teuthology.parallel:result is None 2024-07-17T08:14:16.662 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:14:16.662 DEBUG:teuthology.parallel:result is None 2024-07-17T08:14:16.664 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:14:16.665 DEBUG:teuthology.parallel:result is None 2024-07-17T08:14:16.665 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:14:16.804 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:14:16.805 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:14:16.808 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:14:16.811 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 -- 172.21.15.44:0/4188485628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95815b5d0 msgr2=0x7fe9580b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 --2- 172.21.15.44:0/4188485628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95815b5d0 0x7fe9580b1f00 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fe9580b5bf0 tx=0x7fe94802a5f0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 -- 172.21.15.44:0/4188485628 shutdown_connections 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 --2- 172.21.15.44:0/4188485628 >> v2:172.21.15.175:3300/0 conn(0x7fe9580b0fe0 0x7fe9580b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 --2- 172.21.15.44:0/4188485628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe95815b5d0 0x7fe9580b1f00 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 -- 172.21.15.44:0/4188485628 >> 172.21.15.44:0/4188485628 conn(0x7fe958073d50 msgr2=0x7fe95809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 -- 172.21.15.44:0/4188485628 shutdown_connections 2024-07-17T08:14:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.845+0000 7fe95d600640 1 -- 172.21.15.44:0/4188485628 wait complete. 2024-07-17T08:14:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.847+0000 7fe95d600640 1 Processor -- start 2024-07-17T08:14:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.847+0000 7fe95d600640 1 -- start start 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9580b0fe0 0x7fe95814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 0x7fe9581f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9580afc40 con 0x7fe9580b5450 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9580afac0 con 0x7fe9580b0fe0 2024-07-17T08:14:16.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe95d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9580afdc0 con 0x7fe95815b5d0 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43988/0 (socket says 172.21.15.44:43988) 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 -- 172.21.15.44:0/2549900864 learned_addr learned my addr 172.21.15.44:0/2549900864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe956a00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9580b0fe0 0x7fe95814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe957400640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 0x7fe9581f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 -- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 msgr2=0x7fe9581f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 0x7fe9581f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 -- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9580b0fe0 msgr2=0x7fe95814b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9580b0fe0 0x7fe95814b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.848+0000 7fe94fe00640 1 -- 172.21.15.44:0/2549900864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95814ca00 con 0x7fe9580b5450 2024-07-17T08:14:16.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe957400640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 0x7fe9581f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe94fe00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe94000e9f0 tx=0x7fe94000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94000cd20 con 0x7fe9580b5450 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe940010430 con 0x7fe9580b5450 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940010840 con 0x7fe9580b5450 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95814cce0 con 0x7fe9580b5450 2024-07-17T08:14:16.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.849+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9581f3e90 con 0x7fe9580b5450 2024-07-17T08:14:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.851+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9580b0350 con 0x7fe9580b5450 2024-07-17T08:14:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.852+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fe9400109e0 con 0x7fe9580b5450 2024-07-17T08:14:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.853+0000 7fe94ea00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 0x7fe9340790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.853+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6528+0+0 (secure 0 0 0) 0x7fe940099020 con 0x7fe9580b5450 2024-07-17T08:14:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.853+0000 7fe956a00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 0x7fe9340790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.854+0000 7fe956a00640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 0x7fe9340790a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe948033000 tx=0x7fe948004730 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:16.857+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe940062310 con 0x7fe9580b5450 2024-07-17T08:14:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.080+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe9580b0780 con 0x7fe9580b5450 2024-07-17T08:14:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.688+0000 7fe94ea00640 1 -- 172.21.15.44:0/2549900864 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v183) ==== 97+0+0 (secure 0 0 0) 0x7fe940061c10 con 0x7fe9580b5450 2024-07-17T08:14:17.690 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:14:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 msgr2=0x7fe9340790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 0x7fe9340790a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe948033000 tx=0x7fe948004730 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 msgr2=0x7fe95814d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe94000e9f0 tx=0x7fe94000eeb0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 shutdown_connections 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe934076be0 0x7fe9340790a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe95815b5d0 0x7fe9581f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9580b5450 0x7fe95814d2a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 --2- 172.21.15.44:0/2549900864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9580b0fe0 0x7fe95814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.692+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 >> 172.21.15.44:0/2549900864 conn(0x7fe958073d50 msgr2=0x7fe9580b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.693+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 shutdown_connections 2024-07-17T08:14:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.693+0000 7fe95d600640 1 -- 172.21.15.44:0/2549900864 wait complete. 2024-07-17T08:14:17.719 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:14:17.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 -- 172.21.15.44:0/721126930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 msgr2=0x7f83d0159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 --2- 172.21.15.44:0/721126930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0159f60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f83d015d030 tx=0x7f83bc02aa30 comp rx=0 tx=0).stop 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 -- 172.21.15.44:0/721126930 shutdown_connections 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 --2- 172.21.15.44:0/721126930 >> v2:172.21.15.175:3300/0 conn(0x7f83d00ac3e0 0x7f83d00ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 --2- 172.21.15.44:0/721126930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0159f60 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 --2- 172.21.15.44:0/721126930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.923+0000 7f83d5400640 1 -- 172.21.15.44:0/721126930 >> 172.21.15.44:0/721126930 conn(0x7f83d00b0fb0 msgr2=0x7f83d00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.924+0000 7f83d5400640 1 -- 172.21.15.44:0/721126930 shutdown_connections 2024-07-17T08:14:17.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.924+0000 7f83d5400640 1 -- 172.21.15.44:0/721126930 wait complete. 2024-07-17T08:14:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.925+0000 7f83d5400640 1 Processor -- start 2024-07-17T08:14:17.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.925+0000 7f83d5400640 1 -- start start 2024-07-17T08:14:17.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83d00ac3e0 0x7f83d00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83d00ae980 con 0x7f83d0159b00 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83d00ae800 con 0x7f83d00ac3e0 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.926+0000 7f83d5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83d013ed60 con 0x7f83d006f710 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44012/0 (socket says 172.21.15.44:44012) 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 -- 172.21.15.44:0/3667237776 learned_addr learned my addr 172.21.15.44:0/3667237776 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83ce800640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cde00640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83d00ac3e0 0x7f83d00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 -- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 msgr2=0x7f83d0098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:17.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 -- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83d00ac3e0 msgr2=0x7f83d00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83d00ac3e0 0x7f83d00b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83cf200640 1 -- 172.21.15.44:0/3667237776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d0073560 con 0x7f83d0159b00 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.927+0000 7f83ce800640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83cf200640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d680 tx=0x7f83c800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8017070 con 0x7f83d0159b00 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83c8005310 con 0x7f83d0159b00 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d00737e0 con 0x7f83d0159b00 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8009820 con 0x7f83d0159b00 2024-07-17T08:14:17.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.928+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d0073da0 con 0x7f83d0159b00 2024-07-17T08:14:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.929+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d00ac810 con 0x7f83d0159b00 2024-07-17T08:14:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.930+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f83c80099c0 con 0x7f83d0159b00 2024-07-17T08:14:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.931+0000 7f83c6a00640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 0x7f83b00790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.931+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5797+0+0 (secure 0 0 0) 0x7f83c80982b0 con 0x7f83d0159b00 2024-07-17T08:14:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.932+0000 7f83ce800640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 0x7f83b00790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.933+0000 7f83ce800640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 0x7f83b00790d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005fd0 tx=0x7f83c0005850 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:17.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:17.936+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f83d0073560 con 0x7f83d0159b00 2024-07-17T08:14:18.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.227+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f83d015a110 con 0x7f83d0159b00 2024-07-17T08:14:18.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.228+0000 7f83c6a00640 1 -- 172.21.15.44:0/3667237776 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v61) ==== 80+0+5304 (secure 0 0 0) 0x7f83c8061710 con 0x7f83d0159b00 2024-07-17T08:14:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:14:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:14:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:14:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:14:18.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:14:18.240 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:14:18.241 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:14:18.242 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:14:18.244 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:14:18.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 msgr2=0x7f83b00790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 0x7f83b00790d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005fd0 tx=0x7f83c0005850 comp rx=0 tx=0).stop 2024-07-17T08:14:18.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 msgr2=0x7f83d0152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d680 tx=0x7f83c800db40 comp rx=0 tx=0).stop 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 shutdown_connections 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f83b0076c10 0x7f83b00790d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83d0159b00 0x7f83d0152890 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83d00ac3e0 0x7f83d00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 --2- 172.21.15.44:0/3667237776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83d006f710 0x7f83d0098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.231+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 >> 172.21.15.44:0/3667237776 conn(0x7f83d00b0fb0 msgr2=0x7f83d01e9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.232+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 shutdown_connections 2024-07-17T08:14:18.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.232+0000 7f83d5400640 1 -- 172.21.15.44:0/3667237776 wait complete. 2024-07-17T08:14:18.254 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 -- 172.21.15.44:0/2948046643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ef00053c0 msgr2=0x7f3ef000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 --2- 172.21.15.44:0/2948046643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ef00053c0 0x7f3ef000b9c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400adb0 tx=0x7f3ef0005830 comp rx=0 tx=0).stop 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 -- 172.21.15.44:0/2948046643 shutdown_connections 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 --2- 172.21.15.44:0/2948046643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ef00053c0 0x7f3ef000b9c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 -- 172.21.15.44:0/2948046643 >> 172.21.15.44:0/2948046643 conn(0x7f3f0c0b0e80 msgr2=0x7f3f0c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:18.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 -- 172.21.15.44:0/2948046643 shutdown_connections 2024-07-17T08:14:18.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.472+0000 7f3f11e00640 1 -- 172.21.15.44:0/2948046643 wait complete. 2024-07-17T08:14:18.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.474+0000 7f3f11e00640 1 Processor -- start 2024-07-17T08:14:18.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.475+0000 7f3f11e00640 1 -- start start 2024-07-17T08:14:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 0x7f3f0c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 0x7f3f0c073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c0b3c00 con 0x7f3f0c1548a0 2024-07-17T08:14:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c0b3a80 con 0x7f3f0c0737d0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0c0b3d80 con 0x7f3ef00053c0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44016/0 (socket says 172.21.15.44:44016) 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.476+0000 7f3f11400640 1 -- 172.21.15.44:0/1195560110 learned_addr learned my addr 172.21.15.44:0/1195560110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f0be00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 0x7f3f0c073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f10a00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 0x7f3f0c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 -- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 msgr2=0x7f3f0c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 0x7f3f0c0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 -- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 msgr2=0x7f3f0c073c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 0x7f3f0c073c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 -- 172.21.15.44:0/1195560110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0c0b0050 con 0x7f3f0c1548a0 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f0be00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 0x7f3f0c073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f10a00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 0x7f3f0c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.477+0000 7f3f11400640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400aed0 tx=0x7f3f04036e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.478+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400a290 con 0x7f3f0c1548a0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.478+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f04004b20 con 0x7f3f0c1548a0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.478+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402e880 con 0x7f3f0c1548a0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.478+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0c0b0330 con 0x7f3f0c1548a0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.478+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0c0718f0 con 0x7f3f0c1548a0 2024-07-17T08:14:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.479+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0c13c9d0 con 0x7f3f0c1548a0 2024-07-17T08:14:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.480+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f3f0402ea20 con 0x7f3f0c1548a0 2024-07-17T08:14:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.481+0000 7f3f08c00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 0x7f3eec079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.482+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5797+0+0 (secure 0 0 0) 0x7f3f040b86f0 con 0x7f3f0c1548a0 2024-07-17T08:14:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.482+0000 7f3f10a00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 0x7f3eec079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.483+0000 7f3f10a00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 0x7f3eec079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3efc001880 tx=0x7f3efc000ed0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.486+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3f04047030 con 0x7f3f0c1548a0 2024-07-17T08:14:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.706+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3f0c13c6f0 con 0x7f3f0c1548a0 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.707+0000 7f3f08c00640 1 -- 172.21.15.44:0/1195560110 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v185) ==== 84+0+65 (secure 0 0 0) 0x7f3f04032030 con 0x7f3f0c1548a0 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:18.709 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 msgr2=0x7f3eec079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 0x7f3eec079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3efc001880 tx=0x7f3efc000ed0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 msgr2=0x7f3f0c154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400aed0 tx=0x7f3f04036e50 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 shutdown_connections 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f3eec076bb0 0x7f3eec079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f0c1548a0 0x7f3f0c154d50 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f0c0737d0 0x7f3f0c073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 --2- 172.21.15.44:0/1195560110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ef00053c0 0x7f3f0c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 >> 172.21.15.44:0/1195560110 conn(0x7f3f0c0b0e80 msgr2=0x7f3f0c13d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 shutdown_connections 2024-07-17T08:14:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.710+0000 7f3f11e00640 1 -- 172.21.15.44:0/1195560110 wait complete. 2024-07-17T08:14:18.733 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:14:18.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:14:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 -- 172.21.15.44:0/3003888098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f980ae490 msgr2=0x7f7f98165e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/3003888098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f980ae490 0x7f7f98165e90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005fd0 tx=0x7f7f9002ab30 comp rx=0 tx=0).stop 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 -- 172.21.15.44:0/3003888098 shutdown_connections 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/3003888098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f980ae490 0x7f7f98165e90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 -- 172.21.15.44:0/3003888098 >> 172.21.15.44:0/3003888098 conn(0x7f7f980cf050 msgr2=0x7f7f980cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 -- 172.21.15.44:0/3003888098 shutdown_connections 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.945+0000 7f7f9ca00640 1 -- 172.21.15.44:0/3003888098 wait complete. 2024-07-17T08:14:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.947+0000 7f7f9ca00640 1 Processor -- start 2024-07-17T08:14:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.947+0000 7f7f9ca00640 1 -- start start 2024-07-17T08:14:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f980ad960 0x7f7f98147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 0x7f7f9814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98166550 con 0x7f7f98155860 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981663d0 con 0x7f7f980ae490 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.948+0000 7f7f9ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f981666d0 con 0x7f7f980ad960 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44882/0 (socket says 172.21.15.44:44882) 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 -- 172.21.15.44:0/176679855 learned_addr learned my addr 172.21.15.44:0/176679855 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97400640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f980ad960 0x7f7f98147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f96a00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 0x7f7f9814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 -- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f980ad960 msgr2=0x7f7f98147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f980ad960 0x7f7f98147350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 -- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 msgr2=0x7f7f9814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 0x7f7f9814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 -- 172.21.15.44:0/176679855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f98147890 con 0x7f7f98155860 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f96a00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 0x7f7f9814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.949+0000 7f7f97e00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006010 tx=0x7f7f9002cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.950+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90044070 con 0x7f7f98155860 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.950+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f90004250 con 0x7f7f98155860 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.950+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f980b2da0 con 0x7f7f98155860 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.950+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90004680 con 0x7f7f98155860 2024-07-17T08:14:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.950+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f980b3e20 con 0x7f7f98155860 2024-07-17T08:14:18.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.951+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f980add80 con 0x7f7f98155860 2024-07-17T08:14:18.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.952+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f7f90036400 con 0x7f7f98155860 2024-07-17T08:14:18.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.953+0000 7f7f8f400640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 0x7f7f78079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:18.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.953+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5797+0+0 (secure 0 0 0) 0x7f7f900b8ad0 con 0x7f7f98155860 2024-07-17T08:14:18.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.954+0000 7f7f97400640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 0x7f7f78079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:18.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.955+0000 7f7f97400640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 0x7f7f78079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7f880015e0 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:18.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:18.958+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7f90032030 con 0x7f7f98155860 2024-07-17T08:14:19.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.169+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7f9814ba60 con 0x7f7f98155860 2024-07-17T08:14:19.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.717+0000 7f7f8f400640 1 -- 172.21.15.44:0/176679855 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v186) ==== 138+0+0 (secure 0 0 0) 0x7f7f90004820 con 0x7f7f98155860 2024-07-17T08:14:19.719 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:14:19.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 msgr2=0x7f7f78079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 0x7f7f78079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7f880015e0 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 msgr2=0x7f7f981f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006010 tx=0x7f7f9002cf00 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 shutdown_connections 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.722+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f7f78076bb0 0x7f7f78079070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f98155860 0x7f7f981f7660 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f980ae490 0x7f7f9814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 --2- 172.21.15.44:0/176679855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f980ad960 0x7f7f98147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 >> 172.21.15.44:0/176679855 conn(0x7f7f980cf050 msgr2=0x7f7f9814e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 shutdown_connections 2024-07-17T08:14:19.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.723+0000 7f7f9ca00640 1 -- 172.21.15.44:0/176679855 wait complete. 2024-07-17T08:14:19.756 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:14:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 -- 172.21.15.44:0/2577038268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f415a450 msgr2=0x7f11f40b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 --2- 172.21.15.44:0/2577038268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f415a450 0x7f11f40b2eb0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f11dc005fd0 tx=0x7f11dc02b8f0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 -- 172.21.15.44:0/2577038268 shutdown_connections 2024-07-17T08:14:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 --2- 172.21.15.44:0/2577038268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f40b33f0 0x7f11f406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 --2- 172.21.15.44:0/2577038268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f415a450 0x7f11f40b2eb0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 -- 172.21.15.44:0/2577038268 >> 172.21.15.44:0/2577038268 conn(0x7f11f40cf0d0 msgr2=0x7f11f40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 -- 172.21.15.44:0/2577038268 shutdown_connections 2024-07-17T08:14:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.956+0000 7f11f9000640 1 -- 172.21.15.44:0/2577038268 wait complete. 2024-07-17T08:14:19.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.958+0000 7f11f9000640 1 Processor -- start 2024-07-17T08:14:19.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.958+0000 7f11f9000640 1 -- start start 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 0x7f11f414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 0x7f11f40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11f406dae0 con 0x7f11f40b33f0 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11f406d960 con 0x7f11f415a450 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f9000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11f406dc60 con 0x7f11f41503c0 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:19.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44892/0 (socket says 172.21.15.44:44892) 2024-07-17T08:14:19.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.959+0000 7f11f3400640 1 -- 172.21.15.44:0/478021012 learned_addr learned my addr 172.21.15.44:0/478021012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3e00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 0x7f11f40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 -- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 msgr2=0x7f11f414e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f2a00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 0x7f11f414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 0x7f11f414e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 -- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 msgr2=0x7f11f40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 0x7f11f40af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 -- 172.21.15.44:0/478021012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41f8020 con 0x7f11f40b33f0 2024-07-17T08:14:19.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3e00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 0x7f11f40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f2a00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 0x7f11f414e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.960+0000 7f11f3400640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c870 tx=0x7f11e400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.961+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4003b70 con 0x7f11f40b33f0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.961+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e4003d10 con 0x7f11f40b33f0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.961+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e400aab0 con 0x7f11f40b33f0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.961+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41f3b20 con 0x7f11f40b33f0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.961+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f40cd130 con 0x7f11f40b33f0 2024-07-17T08:14:19.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.962+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f40cffe0 con 0x7f11f40b33f0 2024-07-17T08:14:19.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.963+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f11e400ac50 con 0x7f11f40b33f0 2024-07-17T08:14:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.964+0000 7f11eb400640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 0x7f11d00790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.964+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6167+0+0 (secure 0 0 0) 0x7f11e4098e90 con 0x7f11f40b33f0 2024-07-17T08:14:19.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.965+0000 7f11f2a00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 0x7f11d00790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:19.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.966+0000 7f11f2a00640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 0x7f11d00790a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11dc035000 tx=0x7f11dc004730 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:19.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:19.968+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f11e4062200 con 0x7f11f40b33f0 2024-07-17T08:14:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.180+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f11f406d3a0 con 0x7f11f40b33f0 2024-07-17T08:14:20.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.724+0000 7f11eb400640 1 -- 172.21.15.44:0/478021012 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v187) ==== 170+0+0 (secure 0 0 0) 0x7f11e4061b00 con 0x7f11f40b33f0 2024-07-17T08:14:20.727 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.728+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 msgr2=0x7f11d00790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.728+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 0x7f11d00790a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11dc035000 tx=0x7f11dc004730 comp rx=0 tx=0).stop 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.728+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 msgr2=0x7f11f41f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.728+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c870 tx=0x7f11e400cd30 comp rx=0 tx=0).stop 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 shutdown_connections 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f11d0076be0 0x7f11d00790a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11f415a450 0x7f11f40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11f41503c0 0x7f11f414e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 --2- 172.21.15.44:0/478021012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f40b33f0 0x7f11f41f7ae0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.729+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 >> 172.21.15.44:0/478021012 conn(0x7f11f40cf0d0 msgr2=0x7f11f415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.730+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 shutdown_connections 2024-07-17T08:14:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.730+0000 7f11f9000640 1 -- 172.21.15.44:0/478021012 wait complete. 2024-07-17T08:14:20.771 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:14:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 -- 172.21.15.44:0/1051677593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80b6790 msgr2=0x7efbf81648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:20.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 --2- 172.21.15.44:0/1051677593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80b6790 0x7efbf81648f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efbe4005fd0 tx=0x7efbe402a660 comp rx=0 tx=0).stop 2024-07-17T08:14:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 -- 172.21.15.44:0/1051677593 shutdown_connections 2024-07-17T08:14:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 --2- 172.21.15.44:0/1051677593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80b6790 0x7efbf81648f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 -- 172.21.15.44:0/1051677593 >> 172.21.15.44:0/1051677593 conn(0x7efbf80d3700 msgr2=0x7efbf80d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 -- 172.21.15.44:0/1051677593 shutdown_connections 2024-07-17T08:14:20.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.952+0000 7efc00400640 1 -- 172.21.15.44:0/1051677593 wait complete. 2024-07-17T08:14:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.954+0000 7efc00400640 1 Processor -- start 2024-07-17T08:14:20.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.954+0000 7efc00400640 1 -- start start 2024-07-17T08:14:20.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b29e0 0x7efbf80c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:20.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 0x7efbf81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81688f0 con 0x7efbf80ce1d0 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8168770 con 0x7efbf80b6790 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efc00400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efbf8168a70 con 0x7efbf80b29e0 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efbfe400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efbfe400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44924/0 (socket says 172.21.15.44:44924) 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.955+0000 7efbfe400640 1 -- 172.21.15.44:0/4183317051 learned_addr learned my addr 172.21.15.44:0/4183317051 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfd000640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 0x7efbf81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:20.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 -- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b29e0 msgr2=0x7efbf80c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfda00640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b29e0 0x7efbf80c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b29e0 0x7efbf80c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 -- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 msgr2=0x7efbf81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 0x7efbf81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 -- 172.21.15.44:0/4183317051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8151c30 con 0x7efbf80ce1d0 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfd000640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 0x7efbf81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbfe400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c7b0 tx=0x7efbf000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.956+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0004b60 con 0x7efbf80ce1d0 2024-07-17T08:14:20.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.957+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf80c7360 con 0x7efbf80ce1d0 2024-07-17T08:14:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.957+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbf0004df0 con 0x7efbf80ce1d0 2024-07-17T08:14:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.957+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000aa90 con 0x7efbf80ce1d0 2024-07-17T08:14:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.957+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf80c7600 con 0x7efbf80ce1d0 2024-07-17T08:14:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.958+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf80ce5b0 con 0x7efbf80ce1d0 2024-07-17T08:14:20.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.959+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7efbf000ac30 con 0x7efbf80ce1d0 2024-07-17T08:14:20.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.961+0000 7efbf5600640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 0x7efbd8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:20.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.961+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 6549+0+0 (secure 0 0 0) 0x7efbf0099180 con 0x7efbf80ce1d0 2024-07-17T08:14:20.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.961+0000 7efbfda00640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 0x7efbd8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.963+0000 7efbfda00640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 0x7efbd8079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efbec0015a0 tx=0x7efbec009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:20.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:20.964+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efbf0020070 con 0x7efbf80ce1d0 2024-07-17T08:14:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.220+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efbf80b8020 con 0x7efbf80ce1d0 2024-07-17T08:14:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.768+0000 7efbf5600640 1 -- 172.21.15.44:0/4183317051 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v93) ==== 174+0+0 (secure 0 0 0) 0x7efbf00622f0 con 0x7efbf80ce1d0 2024-07-17T08:14:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 23 and data pool 24 2024-07-17T08:14:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.777+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 msgr2=0x7efbd8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.777+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 0x7efbd8079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efbec0015a0 tx=0x7efbec009040 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.777+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 msgr2=0x7efbf8155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.777+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c7b0 tx=0x7efbf000cc70 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 shutdown_connections 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efbd8076bb0 0x7efbd8079070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efbf80ce1d0 0x7efbf8155ae0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf80b6790 0x7efbf81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 --2- 172.21.15.44:0/4183317051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b29e0 0x7efbf80c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.778+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 >> 172.21.15.44:0/4183317051 conn(0x7efbf80d3700 msgr2=0x7efbf80d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.780+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 shutdown_connections 2024-07-17T08:14:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.780+0000 7efc00400640 1 -- 172.21.15.44:0/4183317051 wait complete. 2024-07-17T08:14:21.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 -- 172.21.15.44:0/3381471209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 msgr2=0x7f8b54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 --2- 172.21.15.44:0/3381471209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b54004c80 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c2e0 tx=0x7f8b54005830 comp rx=0 tx=0).stop 2024-07-17T08:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 -- 172.21.15.44:0/3381471209 shutdown_connections 2024-07-17T08:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 --2- 172.21.15.44:0/3381471209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b54004c80 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 -- 172.21.15.44:0/3381471209 >> 172.21.15.44:0/3381471209 conn(0x7f8b700b0fb0 msgr2=0x7f8b700b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 -- 172.21.15.44:0/3381471209 shutdown_connections 2024-07-17T08:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.994+0000 7f8b78400640 1 -- 172.21.15.44:0/3381471209 wait complete. 2024-07-17T08:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.996+0000 7f8b78400640 1 Processor -- start 2024-07-17T08:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.996+0000 7f8b78400640 1 -- start start 2024-07-17T08:14:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b701555b0 0x7f8b7015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7015d440 0x7f8b7013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b700adb20 con 0x7f8b54004800 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b700ad9a0 con 0x7f8b701555b0 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.997+0000 7f8b78400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b700ac220 con 0x7f8b7015d440 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44946/0 (socket says 172.21.15.44:44946) 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 -- 172.21.15.44:0/312630655 learned_addr learned my addr 172.21.15.44:0/312630655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75000640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b701555b0 0x7f8b7015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b76400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7015d440 0x7f8b7013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 -- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7015d440 msgr2=0x7f8b7013e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7015d440 0x7f8b7013e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 -- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b701555b0 msgr2=0x7f8b7015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b701555b0 0x7f8b7015cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 -- 172.21.15.44:0/312630655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b70074540 con 0x7f8b54004800 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.998+0000 7f8b75a00640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e690 tx=0x7f8b6000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.999+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ca50 con 0x7f8b54004800 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.999+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b6000cce0 con 0x7f8b54004800 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.999+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b70141300 con 0x7f8b54004800 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.999+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60005280 con 0x7f8b54004800 2024-07-17T08:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:21.999+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b70154db0 con 0x7f8b54004800 2024-07-17T08:14:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.000+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7015ee90 con 0x7f8b54004800 2024-07-17T08:14:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.001+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f8b60005420 con 0x7f8b54004800 2024-07-17T08:14:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.002+0000 7f8b6d600640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 0x7f8b500790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.002+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6649+0+0 (secure 0 0 0) 0x7f8b60098e30 con 0x7f8b54004800 2024-07-17T08:14:22.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.003+0000 7f8b75000640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 0x7f8b500790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.004+0000 7f8b75000640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 0x7f8b500790a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c033000 tx=0x7f8b5c02d000 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.006+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8b60016030 con 0x7f8b54004800 2024-07-17T08:14:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.214+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b7015d820 con 0x7f8b54004800 2024-07-17T08:14:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.216+0000 7f8b6d600640 1 -- 172.21.15.44:0/312630655 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17716 (secure 0 0 0) 0x7f8b60061eb0 con 0x7f8b54004800 2024-07-17T08:14:22.218 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:22.218 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":188,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:21.223395+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:14:19.191608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-07-17T08:14:20.205318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.218+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 msgr2=0x7f8b500790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.218+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 0x7f8b500790a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c033000 tx=0x7f8b5c02d000 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 msgr2=0x7f8b70074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e690 tx=0x7f8b6000eb50 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 shutdown_connections 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8b50076be0 0x7f8b500790a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7015d440 0x7f8b7013e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b701555b0 0x7f8b7015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 --2- 172.21.15.44:0/312630655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b54004800 0x7f8b70074000 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 >> 172.21.15.44:0/312630655 conn(0x7f8b700b0fb0 msgr2=0x7f8b700b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 shutdown_connections 2024-07-17T08:14:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.219+0000 7f8b78400640 1 -- 172.21.15.44:0/312630655 wait complete. 2024-07-17T08:14:22.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:22.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 -- 172.21.15.44:0/31885187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b8c80 msgr2=0x7fbbfc0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 --2- 172.21.15.44:0/31885187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc0b4ec0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002a610 comp rx=0 tx=0).stop 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 -- 172.21.15.44:0/31885187 shutdown_connections 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 --2- 172.21.15.44:0/31885187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0b5400 0x7fbbfc15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 --2- 172.21.15.44:0/31885187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc0b4ec0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.432+0000 7fbc03200640 1 -- 172.21.15.44:0/31885187 >> 172.21.15.44:0/31885187 conn(0x7fbbfc0cfec0 msgr2=0x7fbbfc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.433+0000 7fbc03200640 1 -- 172.21.15.44:0/31885187 shutdown_connections 2024-07-17T08:14:22.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.433+0000 7fbc03200640 1 -- 172.21.15.44:0/31885187 wait complete. 2024-07-17T08:14:22.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.435+0000 7fbc03200640 1 Processor -- start 2024-07-17T08:14:22.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.435+0000 7fbc03200640 1 -- start start 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.436+0000 7fbc03200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0ce5c0 0x7fbbfc0aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc03200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc06e0c0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc0b92f0 con 0x7fbbfc0b8c80 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbfc06e240 con 0x7fbbfc0ce5c0 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc01e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44960/0 (socket says 172.21.15.44:44960) 2024-07-17T08:14:22.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc01e00640 1 -- 172.21.15.44:0/3141235899 learned_addr learned my addr 172.21.15.44:0/3141235899 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:22.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc01400640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.437+0000 7fbc02800640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0ce5c0 0x7fbbfc0aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 -- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0ce5c0 msgr2=0x7fbbfc0aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0ce5c0 0x7fbbfc0aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 -- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 msgr2=0x7fbbfc15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 -- 172.21.15.44:0/3141235899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc0aacd0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01400640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc01e00640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009a20 tx=0x7fbbf00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf003f070 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf00047a0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0035030 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc0aafb0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.438+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc1f4320 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.439+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbfc06e6f0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.441+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fbbf0042070 con 0x7fbbfc0b5400 2024-07-17T08:14:22.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.442+0000 7fbbf6000640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 0x7fbbe0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.442+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 6649+0+0 (secure 0 0 0) 0x7fbbf00b9200 con 0x7fbbfc0b5400 2024-07-17T08:14:22.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.442+0000 7fbc01400640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 0x7fbbe0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.444+0000 7fbc01400640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 0x7fbbe0079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0b1d50 tx=0x7fbbec009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.446+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbbf00823b0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.646+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbfc0b5d70 con 0x7fbbfc0b5400 2024-07-17T08:14:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.649+0000 7fbbf6000640 1 -- 172.21.15.44:0/3141235899 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) ==== 74+0+17716 (secure 0 0 0) 0x7fbbf0081cb0 con 0x7fbbfc0b5400 2024-07-17T08:14:22.652 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:22.652 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":188,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:21.223395+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:14:19.191608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-07-17T08:14:20.205318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 msgr2=0x7fbbe0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 0x7fbbe0079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0b1d50 tx=0x7fbbec009040 comp rx=0 tx=0).stop 2024-07-17T08:14:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 msgr2=0x7fbbfc1f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009a20 tx=0x7fbbf00038a0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 shutdown_connections 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fbbe0076bb0 0x7fbbe0079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbfc0ce5c0 0x7fbbfc0aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbfc0b8c80 0x7fbbfc15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 --2- 172.21.15.44:0/3141235899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbfc0b5400 0x7fbbfc1f4b20 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.652+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 >> 172.21.15.44:0/3141235899 conn(0x7fbbfc0cfec0 msgr2=0x7fbbfc0b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.653+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 shutdown_connections 2024-07-17T08:14:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.653+0000 7fbc03200640 1 -- 172.21.15.44:0/3141235899 wait complete. 2024-07-17T08:14:22.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:14:22.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 -- 172.21.15.44:0/3346242285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 msgr2=0x7f653815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 --2- 172.21.15.44:0/3346242285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f653815c140 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6528009830 tx=0x7f652802a480 comp rx=0 tx=0).stop 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 -- 172.21.15.44:0/3346242285 shutdown_connections 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 --2- 172.21.15.44:0/3346242285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f653815c140 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 -- 172.21.15.44:0/3346242285 >> 172.21.15.44:0/3346242285 conn(0x7f65380cf0d0 msgr2=0x7f65380cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 -- 172.21.15.44:0/3346242285 shutdown_connections 2024-07-17T08:14:22.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.886+0000 7f653d800640 1 -- 172.21.15.44:0/3346242285 wait complete. 2024-07-17T08:14:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.888+0000 7f653d800640 1 Processor -- start 2024-07-17T08:14:22.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.888+0000 7f653d800640 1 -- start start 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f653806f380 0x7f653815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 0x7f653815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f653806e730 con 0x7f6538151f20 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6538152510 con 0x7f653806f380 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65380cb790 con 0x7f65380cd0b0 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.889+0000 7f653ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44974/0 (socket says 172.21.15.44:44974) 2024-07-17T08:14:22.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 -- 172.21.15.44:0/4032218204 learned_addr learned my addr 172.21.15.44:0/4032218204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f6537e00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f653806f380 0x7f653815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f652fe00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 0x7f653815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 -- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 msgr2=0x7f653815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 0x7f653815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 -- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f653806f380 msgr2=0x7f653815a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f653806f380 0x7f653815a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 -- 172.21.15.44:0/4032218204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380ac010 con 0x7f6538151f20 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f652fe00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 0x7f653815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:22.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.890+0000 7f653ce00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f653000c700 tx=0x7f653000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.891+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530016020 con 0x7f6538151f20 2024-07-17T08:14:22.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.891+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f653000ab40 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.891+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380ac2f0 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.891+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000ce10 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.891+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65380ac8b0 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.892+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653815c140 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.893+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f6530019020 con 0x7f6538151f20 2024-07-17T08:14:22.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.894+0000 7f6534c00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 0x7f6514079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.894+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f65300989d0 con 0x7f6538151f20 2024-07-17T08:14:22.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.895+0000 7f6537e00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 0x7f6514079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:22.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.896+0000 7f6537e00640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 0x7f6514079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f6528004730 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:22.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:22.898+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6530020020 con 0x7f6538151f20 2024-07-17T08:14:23.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.147+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6538074ee0 con 0x7f6538151f20 2024-07-17T08:14:23.706 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:14:23.705+0000 7f97a9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:14:23.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.793+0000 7f6534c00640 1 -- 172.21.15.44:0/4032218204 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) ==== 118+0+0 (secure 0 0 0) 0x7f6530061af0 con 0x7f6538151f20 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 msgr2=0x7f6514079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 0x7f6514079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6528034000 tx=0x7f6528004730 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 msgr2=0x7f65381f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f653000c700 tx=0x7f653000cbc0 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 shutdown_connections 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6514076bb0 0x7f6514079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6538151f20 0x7f65381f3fc0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380cd0b0 0x7f653815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 --2- 172.21.15.44:0/4032218204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f653806f380 0x7f653815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.798+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 >> 172.21.15.44:0/4032218204 conn(0x7f65380cf0d0 msgr2=0x7f653806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.799+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 shutdown_connections 2024-07-17T08:14:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:23.799+0000 7f653d800640 1 -- 172.21.15.44:0/4032218204 wait complete. 2024-07-17T08:14:23.823 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:14:23.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 -- 172.21.15.44:0/42954393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 msgr2=0x7effc4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 --2- 172.21.15.44:0/42954393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effc4003f10 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7effcc006f80 tx=0x7effc4005830 comp rx=0 tx=0).stop 2024-07-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe1e00640 1 -- 172.21.15.44:0/42954393 reap_dead start 2024-07-17T08:14:24.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 -- 172.21.15.44:0/42954393 shutdown_connections 2024-07-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 -- 172.21.15.44:0/42954393 >> 172.21.15.44:0/42954393 conn(0x7effdc0cf0d0 msgr2=0x7effdc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 -- 172.21.15.44:0/42954393 shutdown_connections 2024-07-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.041+0000 7effe2800640 1 -- 172.21.15.44:0/42954393 wait complete. 2024-07-17T08:14:24.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.043+0000 7effe2800640 1 Processor -- start 2024-07-17T08:14:24.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.043+0000 7effe2800640 1 -- start start 2024-07-17T08:14:24.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effdc09cab0 0x7effdc13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effdc09cf10 0x7effdc13f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effdc0cb080 con 0x7effc4003af0 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effdc152510 con 0x7effdc09cab0 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effdc0cb200 con 0x7effdc09cf10 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44996/0 (socket says 172.21.15.44:44996) 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.044+0000 7effe1400640 1 -- 172.21.15.44:0/3068378639 learned_addr learned my addr 172.21.15.44:0/3068378639 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1e00640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effdc09cf10 0x7effdc13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:24.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 -- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effdc09cf10 msgr2=0x7effdc13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effdc09cf10 0x7effdc13f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe0a00640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effdc09cab0 0x7effdc13f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 -- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effdc09cab0 msgr2=0x7effdc13f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effdc09cab0 0x7effdc13f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 -- 172.21.15.44:0/3068378639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc073090 con 0x7effc4003af0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effe1400640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7effcc02b0a0 tx=0x7effcc006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.045+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc032030 con 0x7effc4003af0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.046+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effcc003a00 con 0x7effc4003af0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.046+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc003e30 con 0x7effc4003af0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.046+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc073370 con 0x7effc4003af0 2024-07-17T08:14:24.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.046+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc073930 con 0x7effc4003af0 2024-07-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.047+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effdc073d10 con 0x7effc4003af0 2024-07-17T08:14:24.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.048+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7effcc035020 con 0x7effc4003af0 2024-07-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.049+0000 7effd9600640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 0x7effc00790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.049+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7effcc0b8dd0 con 0x7effc4003af0 2024-07-17T08:14:24.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.050+0000 7effe0a00640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 0x7effc00790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:24.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.051+0000 7effe0a00640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 0x7effc00790a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effd0000f80 tx=0x7effd0004b10 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:24.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.054+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7effdc073d10 con 0x7effc4003af0 2024-07-17T08:14:24.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.306+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7effdc0b0600 con 0x7effc4003af0 2024-07-17T08:14:24.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.807+0000 7effd9600640 1 -- 172.21.15.44:0/3068378639 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) ==== 122+0+0 (secure 0 0 0) 0x7effdc0b0600 con 0x7effc4003af0 2024-07-17T08:14:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 msgr2=0x7effc00790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 0x7effc00790a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7effd0000f80 tx=0x7effd0004b10 comp rx=0 tx=0).stop 2024-07-17T08:14:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 msgr2=0x7effdc14b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:24.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7effcc02b0a0 tx=0x7effcc006cf0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 shutdown_connections 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7effc0076be0 0x7effc00790a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effdc09cf10 0x7effdc13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effdc09cab0 0x7effdc13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 --2- 172.21.15.44:0/3068378639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effc4003af0 0x7effdc14b760 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 >> 172.21.15.44:0/3068378639 conn(0x7effdc0cf0d0 msgr2=0x7effdc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 shutdown_connections 2024-07-17T08:14:24.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:24.811+0000 7effe2800640 1 -- 172.21.15.44:0/3068378639 wait complete. 2024-07-17T08:14:24.837 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:14:24.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 -- 172.21.15.44:0/3612718876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 msgr2=0x7fe0cc0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/3612718876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc0cf3a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc06d5d0 tx=0x7fe0c402a8c0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 -- 172.21.15.44:0/3612718876 shutdown_connections 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/3612718876 >> v2:172.21.15.175:3300/0 conn(0x7fe0cc09c0b0 0x7fe0cc15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/3612718876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc0cf3a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.058+0000 7fe0d2e00640 1 -- 172.21.15.44:0/3612718876 >> 172.21.15.44:0/3612718876 conn(0x7fe0cc0b0e80 msgr2=0x7fe0cc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.059+0000 7fe0d2e00640 1 -- 172.21.15.44:0/3612718876 shutdown_connections 2024-07-17T08:14:25.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.059+0000 7fe0d2e00640 1 -- 172.21.15.44:0/3612718876 wait complete. 2024-07-17T08:14:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.060+0000 7fe0d2e00640 1 Processor -- start 2024-07-17T08:14:25.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.060+0000 7fe0d2e00640 1 -- start start 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 0x7fe0cc158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc0cf8e0 0x7fe0cc1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc15e070 con 0x7fe0cc157ba0 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc15e1f0 con 0x7fe0cc0cf8e0 2024-07-17T08:14:25.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.061+0000 7fe0d2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc15e370 con 0x7fe0cc09c0b0 2024-07-17T08:14:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45008/0 (socket says 172.21.15.44:45008) 2024-07-17T08:14:25.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 -- 172.21.15.44:0/71387354 learned_addr learned my addr 172.21.15.44:0/71387354 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d1000640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc0cf8e0 0x7fe0cc1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d1a00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 0x7fe0cc158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 -- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 msgr2=0x7fe0cc158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 0x7fe0cc158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 -- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc0cf8e0 msgr2=0x7fe0cc1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc0cf8e0 0x7fe0cc1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d2400640 1 -- 172.21.15.44:0/71387354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc1e76c0 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.062+0000 7fe0d1a00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 0x7fe0cc158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0d2400640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009fd0 tx=0x7fe0bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc011830 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0bc016070 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc011d40 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc1e79a0 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.063+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc1e7f60 con 0x7fe0cc157ba0 2024-07-17T08:14:25.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.064+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0cc15e840 con 0x7fe0cc157ba0 2024-07-17T08:14:25.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.065+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fe0bc01b460 con 0x7fe0cc157ba0 2024-07-17T08:14:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.066+0000 7fe0c9600640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 0x7fe0ac079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.066+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7fe0bc098470 con 0x7fe0cc157ba0 2024-07-17T08:14:25.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.067+0000 7fe0d1a00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 0x7fe0ac079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.068+0000 7fe0d1a00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 0x7fe0ac079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4034000 tx=0x7fe0c4004ab0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:25.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.073+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0bc061590 con 0x7fe0cc157ba0 2024-07-17T08:14:25.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.329+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0cc0ad6b0 con 0x7fe0cc157ba0 2024-07-17T08:14:25.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.331+0000 7fe0c9600640 1 -- 172.21.15.44:0/71387354 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7fe0bc060e90 con 0x7fe0cc157ba0 2024-07-17T08:14:25.333 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:25.333 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:14:25.333 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 msgr2=0x7fe0ac079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 0x7fe0ac079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4034000 tx=0x7fe0c4004ab0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 msgr2=0x7fe0cc159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009fd0 tx=0x7fe0bc00b8e0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 shutdown_connections 2024-07-17T08:14:25.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fe0ac076bb0 0x7fe0ac079070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0cc157ba0 0x7fe0cc159be0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0cc0cf8e0 0x7fe0cc1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 --2- 172.21.15.44:0/71387354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0cc09c0b0 0x7fe0cc158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.334+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 >> 172.21.15.44:0/71387354 conn(0x7fe0cc0b0e80 msgr2=0x7fe0cc0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.335+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 shutdown_connections 2024-07-17T08:14:25.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.335+0000 7fe0d2e00640 1 -- 172.21.15.44:0/71387354 wait complete. 2024-07-17T08:14:25.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:14:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.568+0000 7f6912e00640 1 -- 172.21.15.44:0/812990846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 msgr2=0x7f690c0cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.568+0000 7f6912e00640 1 --2- 172.21.15.44:0/812990846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c0cf2d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f690c0cf040 tx=0x7f68f802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 -- 172.21.15.44:0/812990846 shutdown_connections 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 --2- 172.21.15.44:0/812990846 >> v2:172.21.15.175:3300/0 conn(0x7f690c0cf810 0x7f690c0ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 --2- 172.21.15.44:0/812990846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c0cf2d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 -- 172.21.15.44:0/812990846 >> 172.21.15.44:0/812990846 conn(0x7f690c0b0e80 msgr2=0x7f690c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 -- 172.21.15.44:0/812990846 shutdown_connections 2024-07-17T08:14:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.569+0000 7f6912e00640 1 -- 172.21.15.44:0/812990846 wait complete. 2024-07-17T08:14:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.571+0000 7f6912e00640 1 Processor -- start 2024-07-17T08:14:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.571+0000 7f6912e00640 1 -- start start 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0cf810 0x7f690c14d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c15d240 0x7f690c14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f690c0ace30 con 0x7f690c09ca20 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f690c0accb0 con 0x7f690c0cf810 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6912e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f690c0acfb0 con 0x7f690c15d240 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45016/0 (socket says 172.21.15.44:45016) 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 -- 172.21.15.44:0/1582974475 learned_addr learned my addr 172.21.15.44:0/1582974475 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 -- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c15d240 msgr2=0x7f690c14dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c15d240 0x7f690c14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 -- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0cf810 msgr2=0x7f690c14d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0cf810 0x7f690c14d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.572+0000 7f6911a00640 1 -- 172.21.15.44:0/1582974475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c15d840 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6911a00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f690000c3a0 tx=0x7f690000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011790 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6900017070 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c15db20 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011cc0 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.573+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c15e0e0 con 0x7f690c09ca20 2024-07-17T08:14:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.574+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690c050670 con 0x7f690c09ca20 2024-07-17T08:14:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.575+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f690001c460 con 0x7f690c09ca20 2024-07-17T08:14:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.576+0000 7f6909600640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 0x7f68ec0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.577+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f6900098420 con 0x7f690c09ca20 2024-07-17T08:14:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.577+0000 7f6911000640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 0x7f68ec0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.578+0000 7f6911000640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 0x7f68ec0790a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68f8034000 tx=0x7f68f80046c0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.581+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f690c050670 con 0x7f690c09ca20 2024-07-17T08:14:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.792+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f690c0ad500 con 0x7f690c09ca20 2024-07-17T08:14:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.795+0000 7f6909600640 1 -- 172.21.15.44:0/1582974475 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) ==== 74+0+17703 (secure 0 0 0) 0x7f690c0cfc20 con 0x7f690c09ca20 2024-07-17T08:14:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:25.797 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":189,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:14:22.769735+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:14:19.191608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-07-17T08:14:20.205318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.797+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 msgr2=0x7f68ec0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.797+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 0x7f68ec0790a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68f8034000 tx=0x7f68f80046c0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.797+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 msgr2=0x7f690c15f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.797+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f690000c3a0 tx=0x7f690000c860 comp rx=0 tx=0).stop 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 shutdown_connections 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f68ec076be0 0x7f68ec0790a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c15d240 0x7f690c14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0cf810 0x7f690c14d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 --2- 172.21.15.44:0/1582974475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c09ca20 0x7f690c15f680 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 >> 172.21.15.44:0/1582974475 conn(0x7f690c0b0e80 msgr2=0x7f690c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 shutdown_connections 2024-07-17T08:14:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:25.798+0000 7f6912e00640 1 -- 172.21.15.44:0/1582974475 wait complete. 2024-07-17T08:14:25.821 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:14:25.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 -- 172.21.15.44:0/1289133349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c0cfa90 msgr2=0x7f667c159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 --2- 172.21.15.44:0/1289133349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c0cfa90 0x7f667c159800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6674005fd0 tx=0x7f667402a8b0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 -- 172.21.15.44:0/1289133349 shutdown_connections 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 --2- 172.21.15.44:0/1289133349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c0cfa90 0x7f667c159800 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 --2- 172.21.15.44:0/1289133349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 0x7f667c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 -- 172.21.15.44:0/1289133349 >> 172.21.15.44:0/1289133349 conn(0x7f667c0b0e80 msgr2=0x7f667c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 -- 172.21.15.44:0/1289133349 shutdown_connections 2024-07-17T08:14:26.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.031+0000 7f667b200640 1 -- 172.21.15.44:0/1289133349 wait complete. 2024-07-17T08:14:26.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.033+0000 7f667b200640 1 Processor -- start 2024-07-17T08:14:26.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.033+0000 7f667b200640 1 -- start start 2024-07-17T08:14:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c06f4b0 0x7f667c1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 0x7f667c1e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0b2120 con 0x7f667c0cfa90 2024-07-17T08:14:26.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0b1fa0 con 0x7f667c06f4b0 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0b22a0 con 0x7f667c09b300 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45020/0 (socket says 172.21.15.44:45020) 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f667a800640 1 -- 172.21.15.44:0/1504608708 learned_addr learned my addr 172.21.15.44:0/1504608708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f6679e00640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c06f4b0 0x7f667c1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.035+0000 7f6679400640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 0x7f667c1e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 -- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 msgr2=0x7f667c1e57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 0x7f667c1e57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 -- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c06f4b0 msgr2=0x7f667c1f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c06f4b0 0x7f667c1f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 -- 172.21.15.44:0/1504608708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c1e5d30 con 0x7f667c0cfa90 2024-07-17T08:14:26.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667a800640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f6674009cd0 tx=0x7f66740316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:26.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674031da0 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6674003a60 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740330d0 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.036+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c1f7b80 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.037+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c1f8140 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.038+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c06e980 con 0x7f667c0cfa90 2024-07-17T08:14:26.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.039+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f6674033270 con 0x7f667c0cfa90 2024-07-17T08:14:26.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.040+0000 7f6672000640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 0x7f665c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.041+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f66740b9980 con 0x7f667c0cfa90 2024-07-17T08:14:26.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.041+0000 7f6679e00640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 0x7f665c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.042+0000 7f6679e00640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 0x7f665c079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f666c005fd0 tx=0x7f666c009260 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:26.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.045+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f667c06e980 con 0x7f667c0cfa90 2024-07-17T08:14:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.251+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 --> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f667c06e060 con 0x7f665c076bb0 2024-07-17T08:14:26.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.266+0000 7f6672000640 1 -- 172.21.15.44:0/1504608708 <== mgr.7435 v2:172.21.15.175:6834/2760073115 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f667c06e060 con 0x7f665c076bb0 2024-07-17T08:14:26.271 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:26.273 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:14:26.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 msgr2=0x7f665c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 0x7f665c079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f666c005fd0 tx=0x7f666c009260 comp rx=0 tx=0).stop 2024-07-17T08:14:26.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 msgr2=0x7f667c072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f6674009cd0 tx=0x7f66740316f0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 shutdown_connections 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f665c076bb0 0x7f665c079070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667c0cfa90 0x7f667c072040 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667c09b300 0x7f667c1e57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 --2- 172.21.15.44:0/1504608708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f667c06f4b0 0x7f667c1f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 >> 172.21.15.44:0/1504608708 conn(0x7f667c0b0e80 msgr2=0x7f667c158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 shutdown_connections 2024-07-17T08:14:26.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.272+0000 7f667b200640 1 -- 172.21.15.44:0/1504608708 wait complete. 2024-07-17T08:14:26.293 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:14:25.896042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":344,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2515,"ondisk_log_size":2515,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378724,"kb_used_data":19620,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754595996,"statfs":{"total":773094113280,"available":772706299904,"internally_reserved":0,"allocated":20090880,"data_stored":11336228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":367578672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002062"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336897+0000","last_change":"2024-07-17T08:14:21.785290+0000","last_active":"2024-07-17T08:14:23.336897+0000","last_peered":"2024-07-17T08:14:23.336897+0000","last_clean":"2024-07-17T08:14:23.336897+0000","last_became_active":"2024-07-17T08:14:21.785002+0000","last_became_peered":"2024-07-17T08:14:21.785002+0000","last_unstale":"2024-07-17T08:14:23.336897+0000","last_undegraded":"2024-07-17T08:14:23.336897+0000","last_fullsized":"2024-07-17T08:14:23.336897+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:24.571970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.789930+0000","last_change":"2024-07-17T08:14:21.783478+0000","last_active":"2024-07-17T08:14:21.789930+0000","last_peered":"2024-07-17T08:14:21.789930+0000","last_clean":"2024-07-17T08:14:21.789930+0000","last_became_active":"2024-07-17T08:14:21.781759+0000","last_became_peered":"2024-07-17T08:14:21.781759+0000","last_unstale":"2024-07-17T08:14:21.789930+0000","last_undegraded":"2024-07-17T08:14:21.789930+0000","last_fullsized":"2024-07-17T08:14:21.789930+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:38.053464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337754+0000","last_change":"2024-07-17T08:14:21.779561+0000","last_active":"2024-07-17T08:14:23.337754+0000","last_peered":"2024-07-17T08:14:23.337754+0000","last_clean":"2024-07-17T08:14:23.337754+0000","last_became_active":"2024-07-17T08:14:21.779053+0000","last_became_peered":"2024-07-17T08:14:21.779053+0000","last_unstale":"2024-07-17T08:14:23.337754+0000","last_undegraded":"2024-07-17T08:14:23.337754+0000","last_fullsized":"2024-07-17T08:14:23.337754+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:17.390393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337340+0000","last_change":"2024-07-17T08:14:21.781033+0000","last_active":"2024-07-17T08:14:23.337340+0000","last_peered":"2024-07-17T08:14:23.337340+0000","last_clean":"2024-07-17T08:14:23.337340+0000","last_became_active":"2024-07-17T08:14:21.780662+0000","last_became_peered":"2024-07-17T08:14:21.780662+0000","last_unstale":"2024-07-17T08:14:23.337340+0000","last_undegraded":"2024-07-17T08:14:23.337340+0000","last_fullsized":"2024-07-17T08:14:23.337340+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:20.699644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.802302+0000","last_change":"2024-07-17T08:14:21.795330+0000","last_active":"2024-07-17T08:14:21.802302+0000","last_peered":"2024-07-17T08:14:21.802302+0000","last_clean":"2024-07-17T08:14:21.802302+0000","last_became_active":"2024-07-17T08:14:21.794604+0000","last_became_peered":"2024-07-17T08:14:21.794604+0000","last_unstale":"2024-07-17T08:14:21.802302+0000","last_undegraded":"2024-07-17T08:14:21.802302+0000","last_fullsized":"2024-07-17T08:14:21.802302+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:28.727845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795458+0000","last_change":"2024-07-17T08:14:21.787840+0000","last_active":"2024-07-17T08:14:21.795458+0000","last_peered":"2024-07-17T08:14:21.795458+0000","last_clean":"2024-07-17T08:14:21.795458+0000","last_became_active":"2024-07-17T08:14:21.787521+0000","last_became_peered":"2024-07-17T08:14:21.787521+0000","last_unstale":"2024-07-17T08:14:21.795458+0000","last_undegraded":"2024-07-17T08:14:21.795458+0000","last_fullsized":"2024-07-17T08:14:21.795458+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:34:23.468535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792820+0000","last_change":"2024-07-17T08:14:21.793555+0000","last_active":"2024-07-17T08:14:22.792820+0000","last_peered":"2024-07-17T08:14:22.792820+0000","last_clean":"2024-07-17T08:14:22.792820+0000","last_became_active":"2024-07-17T08:14:21.792889+0000","last_became_peered":"2024-07-17T08:14:21.792889+0000","last_unstale":"2024-07-17T08:14:22.792820+0000","last_undegraded":"2024-07-17T08:14:22.792820+0000","last_fullsized":"2024-07-17T08:14:22.792820+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:43.890699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792783+0000","last_change":"2024-07-17T08:14:21.789138+0000","last_active":"2024-07-17T08:14:22.792783+0000","last_peered":"2024-07-17T08:14:22.792783+0000","last_clean":"2024-07-17T08:14:22.792783+0000","last_became_active":"2024-07-17T08:14:21.788876+0000","last_became_peered":"2024-07-17T08:14:21.788876+0000","last_unstale":"2024-07-17T08:14:22.792783+0000","last_undegraded":"2024-07-17T08:14:22.792783+0000","last_fullsized":"2024-07-17T08:14:22.792783+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:08.736092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793681+0000","last_change":"2024-07-17T08:14:21.803641+0000","last_active":"2024-07-17T08:14:22.793681+0000","last_peered":"2024-07-17T08:14:22.793681+0000","last_clean":"2024-07-17T08:14:22.793681+0000","last_became_active":"2024-07-17T08:14:21.802406+0000","last_became_peered":"2024-07-17T08:14:21.802406+0000","last_unstale":"2024-07-17T08:14:22.793681+0000","last_undegraded":"2024-07-17T08:14:22.793681+0000","last_fullsized":"2024-07-17T08:14:22.793681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:08.005085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793528+0000","last_change":"2024-07-17T08:14:21.802094+0000","last_active":"2024-07-17T08:14:22.793528+0000","last_peered":"2024-07-17T08:14:22.793528+0000","last_clean":"2024-07-17T08:14:22.793528+0000","last_became_active":"2024-07-17T08:14:21.801297+0000","last_became_peered":"2024-07-17T08:14:21.801297+0000","last_unstale":"2024-07-17T08:14:22.793528+0000","last_undegraded":"2024-07-17T08:14:22.793528+0000","last_fullsized":"2024-07-17T08:14:22.793528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:32.217139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789568+0000","last_change":"2024-07-17T08:14:21.799764+0000","last_active":"2024-07-17T08:14:22.789568+0000","last_peered":"2024-07-17T08:14:22.789568+0000","last_clean":"2024-07-17T08:14:22.789568+0000","last_became_active":"2024-07-17T08:14:21.799516+0000","last_became_peered":"2024-07-17T08:14:21.799516+0000","last_unstale":"2024-07-17T08:14:22.789568+0000","last_undegraded":"2024-07-17T08:14:22.789568+0000","last_fullsized":"2024-07-17T08:14:22.789568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:58.190239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795715+0000","last_change":"2024-07-17T08:14:21.791201+0000","last_active":"2024-07-17T08:14:21.795715+0000","last_peered":"2024-07-17T08:14:21.795715+0000","last_clean":"2024-07-17T08:14:21.795715+0000","last_became_active":"2024-07-17T08:14:21.790207+0000","last_became_peered":"2024-07-17T08:14:21.790207+0000","last_unstale":"2024-07-17T08:14:21.795715+0000","last_undegraded":"2024-07-17T08:14:21.795715+0000","last_fullsized":"2024-07-17T08:14:21.795715+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:25.483748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785350+0000","last_change":"2024-07-17T08:14:21.780565+0000","last_active":"2024-07-17T08:14:22.785350+0000","last_peered":"2024-07-17T08:14:22.785350+0000","last_clean":"2024-07-17T08:14:22.785350+0000","last_became_active":"2024-07-17T08:14:21.780205+0000","last_became_peered":"2024-07-17T08:14:21.780205+0000","last_unstale":"2024-07-17T08:14:22.785350+0000","last_undegraded":"2024-07-17T08:14:22.785350+0000","last_fullsized":"2024-07-17T08:14:22.785350+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:16:06.860324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.794496+0000","last_change":"2024-07-17T08:14:21.786961+0000","last_active":"2024-07-17T08:14:21.794496+0000","last_peered":"2024-07-17T08:14:21.794496+0000","last_clean":"2024-07-17T08:14:21.794496+0000","last_became_active":"2024-07-17T08:14:21.784107+0000","last_became_peered":"2024-07-17T08:14:21.784107+0000","last_unstale":"2024-07-17T08:14:21.794496+0000","last_undegraded":"2024-07-17T08:14:21.794496+0000","last_fullsized":"2024-07-17T08:14:21.794496+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:26.340215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790579+0000","last_change":"2024-07-17T08:14:21.789019+0000","last_active":"2024-07-17T08:14:22.790579+0000","last_peered":"2024-07-17T08:14:22.790579+0000","last_clean":"2024-07-17T08:14:22.790579+0000","last_became_active":"2024-07-17T08:14:21.788581+0000","last_became_peered":"2024-07-17T08:14:21.788581+0000","last_unstale":"2024-07-17T08:14:22.790579+0000","last_undegraded":"2024-07-17T08:14:22.790579+0000","last_fullsized":"2024-07-17T08:14:22.790579+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:20.582643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336513+0000","last_change":"2024-07-17T08:14:21.781022+0000","last_active":"2024-07-17T08:14:23.336513+0000","last_peered":"2024-07-17T08:14:23.336513+0000","last_clean":"2024-07-17T08:14:23.336513+0000","last_became_active":"2024-07-17T08:14:21.780331+0000","last_became_peered":"2024-07-17T08:14:21.780331+0000","last_unstale":"2024-07-17T08:14:23.336513+0000","last_undegraded":"2024-07-17T08:14:23.336513+0000","last_fullsized":"2024-07-17T08:14:23.336513+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:46.616389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793767+0000","last_change":"2024-07-17T08:14:21.793907+0000","last_active":"2024-07-17T08:14:22.793767+0000","last_peered":"2024-07-17T08:14:22.793767+0000","last_clean":"2024-07-17T08:14:22.793767+0000","last_became_active":"2024-07-17T08:14:21.793398+0000","last_became_peered":"2024-07-17T08:14:21.793398+0000","last_unstale":"2024-07-17T08:14:22.793767+0000","last_undegraded":"2024-07-17T08:14:22.793767+0000","last_fullsized":"2024-07-17T08:14:22.793767+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:23.721805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.788477+0000","last_change":"2024-07-17T08:14:21.793900+0000","last_active":"2024-07-17T08:14:22.788477+0000","last_peered":"2024-07-17T08:14:22.788477+0000","last_clean":"2024-07-17T08:14:22.788477+0000","last_became_active":"2024-07-17T08:14:21.793582+0000","last_became_peered":"2024-07-17T08:14:21.793582+0000","last_unstale":"2024-07-17T08:14:22.788477+0000","last_undegraded":"2024-07-17T08:14:22.788477+0000","last_fullsized":"2024-07-17T08:14:22.788477+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:18:33.274175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790033+0000","last_change":"2024-07-17T08:14:21.784903+0000","last_active":"2024-07-17T08:14:22.790033+0000","last_peered":"2024-07-17T08:14:22.790033+0000","last_clean":"2024-07-17T08:14:22.790033+0000","last_became_active":"2024-07-17T08:14:21.784586+0000","last_became_peered":"2024-07-17T08:14:21.784586+0000","last_unstale":"2024-07-17T08:14:22.790033+0000","last_undegraded":"2024-07-17T08:14:22.790033+0000","last_fullsized":"2024-07-17T08:14:22.790033+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:01.636186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786618+0000","last_change":"2024-07-17T08:14:21.781461+0000","last_active":"2024-07-17T08:14:21.786618+0000","last_peered":"2024-07-17T08:14:21.786618+0000","last_clean":"2024-07-17T08:14:21.786618+0000","last_became_active":"2024-07-17T08:14:21.778131+0000","last_became_peered":"2024-07-17T08:14:21.778131+0000","last_unstale":"2024-07-17T08:14:21.786618+0000","last_undegraded":"2024-07-17T08:14:21.786618+0000","last_fullsized":"2024-07-17T08:14:21.786618+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:36.646927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785574+0000","last_change":"2024-07-17T08:14:21.772831+0000","last_active":"2024-07-17T08:14:22.785574+0000","last_peered":"2024-07-17T08:14:22.785574+0000","last_clean":"2024-07-17T08:14:22.785574+0000","last_became_active":"2024-07-17T08:14:21.771958+0000","last_became_peered":"2024-07-17T08:14:21.771958+0000","last_unstale":"2024-07-17T08:14:22.785574+0000","last_undegraded":"2024-07-17T08:14:22.785574+0000","last_fullsized":"2024-07-17T08:14:22.785574+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:53.349723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.797995+0000","last_change":"2024-07-17T08:14:21.797468+0000","last_active":"2024-07-17T08:14:21.797995+0000","last_peered":"2024-07-17T08:14:21.797995+0000","last_clean":"2024-07-17T08:14:21.797995+0000","last_became_active":"2024-07-17T08:14:21.797053+0000","last_became_peered":"2024-07-17T08:14:21.797053+0000","last_unstale":"2024-07-17T08:14:21.797995+0000","last_undegraded":"2024-07-17T08:14:21.797995+0000","last_fullsized":"2024-07-17T08:14:21.797995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:56.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.801079+0000","last_change":"2024-07-17T08:14:21.798809+0000","last_active":"2024-07-17T08:14:21.801079+0000","last_peered":"2024-07-17T08:14:21.801079+0000","last_clean":"2024-07-17T08:14:21.801079+0000","last_became_active":"2024-07-17T08:14:21.792158+0000","last_became_peered":"2024-07-17T08:14:21.792158+0000","last_unstale":"2024-07-17T08:14:21.801079+0000","last_undegraded":"2024-07-17T08:14:21.801079+0000","last_fullsized":"2024-07-17T08:14:21.801079+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:06.381735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336805+0000","last_change":"2024-07-17T08:14:21.793060+0000","last_active":"2024-07-17T08:14:23.336805+0000","last_peered":"2024-07-17T08:14:23.336805+0000","last_clean":"2024-07-17T08:14:23.336805+0000","last_became_active":"2024-07-17T08:14:21.791797+0000","last_became_peered":"2024-07-17T08:14:21.791797+0000","last_unstale":"2024-07-17T08:14:23.336805+0000","last_undegraded":"2024-07-17T08:14:23.336805+0000","last_fullsized":"2024-07-17T08:14:23.336805+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:07.048034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.786037+0000","last_change":"2024-07-17T08:14:21.778734+0000","last_active":"2024-07-17T08:14:22.786037+0000","last_peered":"2024-07-17T08:14:22.786037+0000","last_clean":"2024-07-17T08:14:22.786037+0000","last_became_active":"2024-07-17T08:14:21.778407+0000","last_became_peered":"2024-07-17T08:14:21.778407+0000","last_unstale":"2024-07-17T08:14:22.786037+0000","last_undegraded":"2024-07-17T08:14:22.786037+0000","last_fullsized":"2024-07-17T08:14:22.786037+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:37.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.787874+0000","last_change":"2024-07-17T08:14:21.779540+0000","last_active":"2024-07-17T08:14:21.787874+0000","last_peered":"2024-07-17T08:14:21.787874+0000","last_clean":"2024-07-17T08:14:21.787874+0000","last_became_active":"2024-07-17T08:14:21.779222+0000","last_became_peered":"2024-07-17T08:14:21.779222+0000","last_unstale":"2024-07-17T08:14:21.787874+0000","last_undegraded":"2024-07-17T08:14:21.787874+0000","last_fullsized":"2024-07-17T08:14:21.787874+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:06.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795741+0000","last_change":"2024-07-17T08:14:21.789034+0000","last_active":"2024-07-17T08:14:21.795741+0000","last_peered":"2024-07-17T08:14:21.795741+0000","last_clean":"2024-07-17T08:14:21.795741+0000","last_became_active":"2024-07-17T08:14:21.785800+0000","last_became_peered":"2024-07-17T08:14:21.785800+0000","last_unstale":"2024-07-17T08:14:21.795741+0000","last_undegraded":"2024-07-17T08:14:21.795741+0000","last_fullsized":"2024-07-17T08:14:21.795741+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:07.075917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785529+0000","last_change":"2024-07-17T08:14:21.789653+0000","last_active":"2024-07-17T08:14:22.785529+0000","last_peered":"2024-07-17T08:14:22.785529+0000","last_clean":"2024-07-17T08:14:22.785529+0000","last_became_active":"2024-07-17T08:14:21.789319+0000","last_became_peered":"2024-07-17T08:14:21.789319+0000","last_unstale":"2024-07-17T08:14:22.785529+0000","last_undegraded":"2024-07-17T08:14:22.785529+0000","last_fullsized":"2024-07-17T08:14:22.785529+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:08.403272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796510+0000","last_change":"2024-07-17T08:14:21.795390+0000","last_active":"2024-07-17T08:14:21.796510+0000","last_peered":"2024-07-17T08:14:21.796510+0000","last_clean":"2024-07-17T08:14:21.796510+0000","last_became_active":"2024-07-17T08:14:21.793637+0000","last_became_peered":"2024-07-17T08:14:21.793637+0000","last_unstale":"2024-07-17T08:14:21.796510+0000","last_undegraded":"2024-07-17T08:14:21.796510+0000","last_fullsized":"2024-07-17T08:14:21.796510+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:23.622454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787894+0000","last_change":"2024-07-17T08:14:21.792364+0000","last_active":"2024-07-17T08:14:22.787894+0000","last_peered":"2024-07-17T08:14:22.787894+0000","last_clean":"2024-07-17T08:14:22.787894+0000","last_became_active":"2024-07-17T08:14:21.789378+0000","last_became_peered":"2024-07-17T08:14:21.789378+0000","last_unstale":"2024-07-17T08:14:22.787894+0000","last_undegraded":"2024-07-17T08:14:22.787894+0000","last_fullsized":"2024-07-17T08:14:22.787894+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:27.503110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789897+0000","last_change":"2024-07-17T08:14:21.786905+0000","last_active":"2024-07-17T08:14:22.789897+0000","last_peered":"2024-07-17T08:14:22.789897+0000","last_clean":"2024-07-17T08:14:22.789897+0000","last_became_active":"2024-07-17T08:14:21.786599+0000","last_became_peered":"2024-07-17T08:14:21.786599+0000","last_unstale":"2024-07-17T08:14:22.789897+0000","last_undegraded":"2024-07-17T08:14:22.789897+0000","last_fullsized":"2024-07-17T08:14:22.789897+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:05.121612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787831+0000","last_change":"2024-07-17T08:14:21.781480+0000","last_active":"2024-07-17T08:14:22.787831+0000","last_peered":"2024-07-17T08:14:22.787831+0000","last_clean":"2024-07-17T08:14:22.787831+0000","last_became_active":"2024-07-17T08:14:21.781219+0000","last_became_peered":"2024-07-17T08:14:21.781219+0000","last_unstale":"2024-07-17T08:14:22.787831+0000","last_undegraded":"2024-07-17T08:14:22.787831+0000","last_fullsized":"2024-07-17T08:14:22.787831+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:26.652431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795752+0000","last_change":"2024-07-17T08:14:21.793883+0000","last_active":"2024-07-17T08:14:21.795752+0000","last_peered":"2024-07-17T08:14:21.795752+0000","last_clean":"2024-07-17T08:14:21.795752+0000","last_became_active":"2024-07-17T08:14:21.793558+0000","last_became_peered":"2024-07-17T08:14:21.793558+0000","last_unstale":"2024-07-17T08:14:21.795752+0000","last_undegraded":"2024-07-17T08:14:21.795752+0000","last_fullsized":"2024-07-17T08:14:21.795752+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:18:47.779249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"188'4","reported_seq":18,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337361+0000","last_change":"2024-07-17T08:14:20.752063+0000","last_active":"2024-07-17T08:14:23.337361+0000","last_peered":"2024-07-17T08:14:23.337361+0000","last_clean":"2024-07-17T08:14:23.337361+0000","last_became_active":"2024-07-17T08:14:20.751784+0000","last_became_peered":"2024-07-17T08:14:20.751784+0000","last_unstale":"2024-07-17T08:14:23.337361+0000","last_undegraded":"2024-07-17T08:14:23.337361+0000","last_fullsized":"2024-07-17T08:14:23.337361+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:57.779035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785072+0000","last_change":"2024-07-17T08:14:21.782352+0000","last_active":"2024-07-17T08:14:22.785072+0000","last_peered":"2024-07-17T08:14:22.785072+0000","last_clean":"2024-07-17T08:14:22.785072+0000","last_became_active":"2024-07-17T08:14:21.782061+0000","last_became_peered":"2024-07-17T08:14:21.782061+0000","last_unstale":"2024-07-17T08:14:22.785072+0000","last_undegraded":"2024-07-17T08:14:22.785072+0000","last_fullsized":"2024-07-17T08:14:22.785072+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:31.207100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815571+0000","last_change":"2024-07-17T08:14:20.767918+0000","last_active":"2024-07-17T08:14:21.815571+0000","last_peered":"2024-07-17T08:14:21.815571+0000","last_clean":"2024-07-17T08:14:21.815571+0000","last_became_active":"2024-07-17T08:14:20.767516+0000","last_became_peered":"2024-07-17T08:14:20.767516+0000","last_unstale":"2024-07-17T08:14:21.815571+0000","last_undegraded":"2024-07-17T08:14:21.815571+0000","last_fullsized":"2024-07-17T08:14:21.815571+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:22.917406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.800764+0000","last_change":"2024-07-17T08:14:21.787465+0000","last_active":"2024-07-17T08:14:21.800764+0000","last_peered":"2024-07-17T08:14:21.800764+0000","last_clean":"2024-07-17T08:14:21.800764+0000","last_became_active":"2024-07-17T08:14:21.783683+0000","last_became_peered":"2024-07-17T08:14:21.783683+0000","last_unstale":"2024-07-17T08:14:21.800764+0000","last_undegraded":"2024-07-17T08:14:21.800764+0000","last_fullsized":"2024-07-17T08:14:21.800764+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:51.256299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790423+0000","last_change":"2024-07-17T08:14:20.760499+0000","last_active":"2024-07-17T08:14:22.790423+0000","last_peered":"2024-07-17T08:14:22.790423+0000","last_clean":"2024-07-17T08:14:22.790423+0000","last_became_active":"2024-07-17T08:14:20.760181+0000","last_became_peered":"2024-07-17T08:14:20.760181+0000","last_unstale":"2024-07-17T08:14:22.790423+0000","last_undegraded":"2024-07-17T08:14:22.790423+0000","last_fullsized":"2024-07-17T08:14:22.790423+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:53.865648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796788+0000","last_change":"2024-07-17T08:14:21.796084+0000","last_active":"2024-07-17T08:14:21.796788+0000","last_peered":"2024-07-17T08:14:21.796788+0000","last_clean":"2024-07-17T08:14:21.796788+0000","last_became_active":"2024-07-17T08:14:21.794683+0000","last_became_peered":"2024-07-17T08:14:21.794683+0000","last_unstale":"2024-07-17T08:14:21.796788+0000","last_undegraded":"2024-07-17T08:14:21.796788+0000","last_fullsized":"2024-07-17T08:14:21.796788+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:20.663185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812541+0000","last_change":"2024-07-17T08:14:20.760206+0000","last_active":"2024-07-17T08:14:21.812541+0000","last_peered":"2024-07-17T08:14:21.812541+0000","last_clean":"2024-07-17T08:14:21.812541+0000","last_became_active":"2024-07-17T08:14:20.759131+0000","last_became_peered":"2024-07-17T08:14:20.759131+0000","last_unstale":"2024-07-17T08:14:21.812541+0000","last_undegraded":"2024-07-17T08:14:21.812541+0000","last_fullsized":"2024-07-17T08:14:21.812541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:48:29.692936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793495+0000","last_change":"2024-07-17T08:14:21.800183+0000","last_active":"2024-07-17T08:14:22.793495+0000","last_peered":"2024-07-17T08:14:22.793495+0000","last_clean":"2024-07-17T08:14:22.793495+0000","last_became_active":"2024-07-17T08:14:21.799919+0000","last_became_peered":"2024-07-17T08:14:21.799919+0000","last_unstale":"2024-07-17T08:14:22.793495+0000","last_undegraded":"2024-07-17T08:14:22.793495+0000","last_fullsized":"2024-07-17T08:14:22.793495+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:50.347629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793135+0000","last_change":"2024-07-17T08:14:20.764705+0000","last_active":"2024-07-17T08:14:22.793135+0000","last_peered":"2024-07-17T08:14:22.793135+0000","last_clean":"2024-07-17T08:14:22.793135+0000","last_became_active":"2024-07-17T08:14:20.764402+0000","last_became_peered":"2024-07-17T08:14:20.764402+0000","last_unstale":"2024-07-17T08:14:22.793135+0000","last_undegraded":"2024-07-17T08:14:22.793135+0000","last_fullsized":"2024-07-17T08:14:22.793135+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:02.733153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.779418+0000","last_change":"2024-07-17T08:14:21.777319+0000","last_active":"2024-07-17T08:14:21.779418+0000","last_peered":"2024-07-17T08:14:21.779418+0000","last_clean":"2024-07-17T08:14:21.779418+0000","last_became_active":"2024-07-17T08:14:21.774244+0000","last_became_peered":"2024-07-17T08:14:21.774244+0000","last_unstale":"2024-07-17T08:14:21.779418+0000","last_undegraded":"2024-07-17T08:14:21.779418+0000","last_fullsized":"2024-07-17T08:14:21.779418+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:01:14.809600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815179+0000","last_change":"2024-07-17T08:14:20.750186+0000","last_active":"2024-07-17T08:14:21.815179+0000","last_peered":"2024-07-17T08:14:21.815179+0000","last_clean":"2024-07-17T08:14:21.815179+0000","last_became_active":"2024-07-17T08:14:20.749069+0000","last_became_peered":"2024-07-17T08:14:20.749069+0000","last_unstale":"2024-07-17T08:14:21.815179+0000","last_undegraded":"2024-07-17T08:14:21.815179+0000","last_fullsized":"2024-07-17T08:14:21.815179+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:30.257455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795048+0000","last_change":"2024-07-17T08:14:21.791654+0000","last_active":"2024-07-17T08:14:21.795048+0000","last_peered":"2024-07-17T08:14:21.795048+0000","last_clean":"2024-07-17T08:14:21.795048+0000","last_became_active":"2024-07-17T08:14:21.788459+0000","last_became_peered":"2024-07-17T08:14:21.788459+0000","last_unstale":"2024-07-17T08:14:21.795048+0000","last_undegraded":"2024-07-17T08:14:21.795048+0000","last_fullsized":"2024-07-17T08:14:21.795048+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:24.676597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.764998+0000","last_change":"2024-07-17T08:14:20.760709+0000","last_active":"2024-07-17T08:14:21.764998+0000","last_peered":"2024-07-17T08:14:21.764998+0000","last_clean":"2024-07-17T08:14:21.764998+0000","last_became_active":"2024-07-17T08:14:20.760392+0000","last_became_peered":"2024-07-17T08:14:20.760392+0000","last_unstale":"2024-07-17T08:14:21.764998+0000","last_undegraded":"2024-07-17T08:14:21.764998+0000","last_fullsized":"2024-07-17T08:14:21.764998+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:57.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337039+0000","last_change":"2024-07-17T08:14:21.782855+0000","last_active":"2024-07-17T08:14:23.337039+0000","last_peered":"2024-07-17T08:14:23.337039+0000","last_clean":"2024-07-17T08:14:23.337039+0000","last_became_active":"2024-07-17T08:14:21.779257+0000","last_became_peered":"2024-07-17T08:14:21.779257+0000","last_unstale":"2024-07-17T08:14:23.337039+0000","last_undegraded":"2024-07-17T08:14:23.337039+0000","last_fullsized":"2024-07-17T08:14:23.337039+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:56.644834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792845+0000","last_change":"2024-07-17T08:14:20.766884+0000","last_active":"2024-07-17T08:14:22.792845+0000","last_peered":"2024-07-17T08:14:22.792845+0000","last_clean":"2024-07-17T08:14:22.792845+0000","last_became_active":"2024-07-17T08:14:20.765185+0000","last_became_peered":"2024-07-17T08:14:20.765185+0000","last_unstale":"2024-07-17T08:14:22.792845+0000","last_undegraded":"2024-07-17T08:14:22.792845+0000","last_fullsized":"2024-07-17T08:14:22.792845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:54.329058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795733+0000","last_change":"2024-07-17T08:14:21.791691+0000","last_active":"2024-07-17T08:14:21.795733+0000","last_peered":"2024-07-17T08:14:21.795733+0000","last_clean":"2024-07-17T08:14:21.795733+0000","last_became_active":"2024-07-17T08:14:21.791359+0000","last_became_peered":"2024-07-17T08:14:21.791359+0000","last_unstale":"2024-07-17T08:14:21.795733+0000","last_undegraded":"2024-07-17T08:14:21.795733+0000","last_fullsized":"2024-07-17T08:14:21.795733+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:08:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785840+0000","last_change":"2024-07-17T08:14:20.767778+0000","last_active":"2024-07-17T08:14:22.785840+0000","last_peered":"2024-07-17T08:14:22.785840+0000","last_clean":"2024-07-17T08:14:22.785840+0000","last_became_active":"2024-07-17T08:14:20.765968+0000","last_became_peered":"2024-07-17T08:14:20.765968+0000","last_unstale":"2024-07-17T08:14:22.785840+0000","last_undegraded":"2024-07-17T08:14:22.785840+0000","last_fullsized":"2024-07-17T08:14:22.785840+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:42.971063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785148+0000","last_change":"2024-07-17T08:14:21.791533+0000","last_active":"2024-07-17T08:14:22.785148+0000","last_peered":"2024-07-17T08:14:22.785148+0000","last_clean":"2024-07-17T08:14:22.785148+0000","last_became_active":"2024-07-17T08:14:21.791084+0000","last_became_peered":"2024-07-17T08:14:21.791084+0000","last_unstale":"2024-07-17T08:14:22.785148+0000","last_undegraded":"2024-07-17T08:14:22.785148+0000","last_fullsized":"2024-07-17T08:14:22.785148+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:07.421409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"188'3","reported_seq":17,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337617+0000","last_change":"2024-07-17T08:14:20.762621+0000","last_active":"2024-07-17T08:14:23.337617+0000","last_peered":"2024-07-17T08:14:23.337617+0000","last_clean":"2024-07-17T08:14:23.337617+0000","last_became_active":"2024-07-17T08:14:20.762159+0000","last_became_peered":"2024-07-17T08:14:20.762159+0000","last_unstale":"2024-07-17T08:14:23.337617+0000","last_undegraded":"2024-07-17T08:14:23.337617+0000","last_fullsized":"2024-07-17T08:14:23.337617+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:05.037549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.805193+0000","last_change":"2024-07-17T08:14:21.798202+0000","last_active":"2024-07-17T08:14:21.805193+0000","last_peered":"2024-07-17T08:14:21.805193+0000","last_clean":"2024-07-17T08:14:21.805193+0000","last_became_active":"2024-07-17T08:14:21.797921+0000","last_became_peered":"2024-07-17T08:14:21.797921+0000","last_unstale":"2024-07-17T08:14:21.805193+0000","last_undegraded":"2024-07-17T08:14:21.805193+0000","last_fullsized":"2024-07-17T08:14:21.805193+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:30:41.464272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"188'6","reported_seq":20,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785878+0000","last_change":"2024-07-17T08:14:20.764762+0000","last_active":"2024-07-17T08:14:22.785878+0000","last_peered":"2024-07-17T08:14:22.785878+0000","last_clean":"2024-07-17T08:14:22.785878+0000","last_became_active":"2024-07-17T08:14:20.764456+0000","last_became_peered":"2024-07-17T08:14:20.764456+0000","last_unstale":"2024-07-17T08:14:22.785878+0000","last_undegraded":"2024-07-17T08:14:22.785878+0000","last_fullsized":"2024-07-17T08:14:22.785878+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:30.257455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789291+0000","last_change":"2024-07-17T08:14:21.786361+0000","last_active":"2024-07-17T08:14:22.789291+0000","last_peered":"2024-07-17T08:14:22.789291+0000","last_clean":"2024-07-17T08:14:22.789291+0000","last_became_active":"2024-07-17T08:14:21.786090+0000","last_became_peered":"2024-07-17T08:14:21.786090+0000","last_unstale":"2024-07-17T08:14:22.789291+0000","last_undegraded":"2024-07-17T08:14:22.789291+0000","last_fullsized":"2024-07-17T08:14:22.789291+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:55.334083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815063+0000","last_change":"2024-07-17T08:14:20.762399+0000","last_active":"2024-07-17T08:14:21.815063+0000","last_peered":"2024-07-17T08:14:21.815063+0000","last_clean":"2024-07-17T08:14:21.815063+0000","last_became_active":"2024-07-17T08:14:20.760113+0000","last_became_peered":"2024-07-17T08:14:20.760113+0000","last_unstale":"2024-07-17T08:14:21.815063+0000","last_undegraded":"2024-07-17T08:14:21.815063+0000","last_fullsized":"2024-07-17T08:14:21.815063+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:23.477566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793233+0000","last_change":"2024-07-17T08:14:21.803922+0000","last_active":"2024-07-17T08:14:22.793233+0000","last_peered":"2024-07-17T08:14:22.793233+0000","last_clean":"2024-07-17T08:14:22.793233+0000","last_became_active":"2024-07-17T08:14:21.803327+0000","last_became_peered":"2024-07-17T08:14:21.803327+0000","last_unstale":"2024-07-17T08:14:22.793233+0000","last_undegraded":"2024-07-17T08:14:22.793233+0000","last_fullsized":"2024-07-17T08:14:22.793233+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:07.056039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785218+0000","last_change":"2024-07-17T08:14:20.759090+0000","last_active":"2024-07-17T08:14:22.785218+0000","last_peered":"2024-07-17T08:14:22.785218+0000","last_clean":"2024-07-17T08:14:22.785218+0000","last_became_active":"2024-07-17T08:14:20.757816+0000","last_became_peered":"2024-07-17T08:14:20.757816+0000","last_unstale":"2024-07-17T08:14:22.785218+0000","last_undegraded":"2024-07-17T08:14:22.785218+0000","last_fullsized":"2024-07-17T08:14:22.785218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:50.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790375+0000","last_change":"2024-07-17T08:14:21.782081+0000","last_active":"2024-07-17T08:14:22.790375+0000","last_peered":"2024-07-17T08:14:22.790375+0000","last_clean":"2024-07-17T08:14:22.790375+0000","last_became_active":"2024-07-17T08:14:21.778876+0000","last_became_peered":"2024-07-17T08:14:21.778876+0000","last_unstale":"2024-07-17T08:14:22.790375+0000","last_undegraded":"2024-07-17T08:14:22.790375+0000","last_fullsized":"2024-07-17T08:14:22.790375+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:03.177998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790181+0000","last_change":"2024-07-17T08:14:20.761932+0000","last_active":"2024-07-17T08:14:22.790181+0000","last_peered":"2024-07-17T08:14:22.790181+0000","last_clean":"2024-07-17T08:14:22.790181+0000","last_became_active":"2024-07-17T08:14:20.761629+0000","last_became_peered":"2024-07-17T08:14:20.761629+0000","last_unstale":"2024-07-17T08:14:22.790181+0000","last_undegraded":"2024-07-17T08:14:22.790181+0000","last_fullsized":"2024-07-17T08:14:22.790181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:13.107054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786003+0000","last_change":"2024-07-17T08:14:21.780807+0000","last_active":"2024-07-17T08:14:21.786003+0000","last_peered":"2024-07-17T08:14:21.786003+0000","last_clean":"2024-07-17T08:14:21.786003+0000","last_became_active":"2024-07-17T08:14:21.780468+0000","last_became_peered":"2024-07-17T08:14:21.780468+0000","last_unstale":"2024-07-17T08:14:21.786003+0000","last_undegraded":"2024-07-17T08:14:21.786003+0000","last_fullsized":"2024-07-17T08:14:21.786003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:28.497122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"188'8","reported_seq":22,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793255+0000","last_change":"2024-07-17T08:14:20.758057+0000","last_active":"2024-07-17T08:14:22.793255+0000","last_peered":"2024-07-17T08:14:22.793255+0000","last_clean":"2024-07-17T08:14:22.793255+0000","last_became_active":"2024-07-17T08:14:20.757228+0000","last_became_peered":"2024-07-17T08:14:20.757228+0000","last_unstale":"2024-07-17T08:14:22.793255+0000","last_undegraded":"2024-07-17T08:14:22.793255+0000","last_fullsized":"2024-07-17T08:14:22.793255+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:23.159616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792712+0000","last_change":"2024-07-17T08:14:21.791654+0000","last_active":"2024-07-17T08:14:22.792712+0000","last_peered":"2024-07-17T08:14:22.792712+0000","last_clean":"2024-07-17T08:14:22.792712+0000","last_became_active":"2024-07-17T08:14:21.791038+0000","last_became_peered":"2024-07-17T08:14:21.791038+0000","last_unstale":"2024-07-17T08:14:22.792712+0000","last_undegraded":"2024-07-17T08:14:22.792712+0000","last_fullsized":"2024-07-17T08:14:22.792712+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:14.112079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.786007+0000","last_change":"2024-07-17T08:14:20.752532+0000","last_active":"2024-07-17T08:14:22.786007+0000","last_peered":"2024-07-17T08:14:22.786007+0000","last_clean":"2024-07-17T08:14:22.786007+0000","last_became_active":"2024-07-17T08:14:20.751822+0000","last_became_peered":"2024-07-17T08:14:20.751822+0000","last_unstale":"2024-07-17T08:14:22.786007+0000","last_undegraded":"2024-07-17T08:14:22.786007+0000","last_fullsized":"2024-07-17T08:14:22.786007+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:25.335190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.788875+0000","last_change":"2024-07-17T08:14:21.783002+0000","last_active":"2024-07-17T08:14:22.788875+0000","last_peered":"2024-07-17T08:14:22.788875+0000","last_clean":"2024-07-17T08:14:22.788875+0000","last_became_active":"2024-07-17T08:14:21.782604+0000","last_became_peered":"2024-07-17T08:14:21.782604+0000","last_unstale":"2024-07-17T08:14:22.788875+0000","last_undegraded":"2024-07-17T08:14:22.788875+0000","last_fullsized":"2024-07-17T08:14:22.788875+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793659+0000","last_change":"2024-07-17T08:14:20.764814+0000","last_active":"2024-07-17T08:14:22.793659+0000","last_peered":"2024-07-17T08:14:22.793659+0000","last_clean":"2024-07-17T08:14:22.793659+0000","last_became_active":"2024-07-17T08:14:20.764505+0000","last_became_peered":"2024-07-17T08:14:20.764505+0000","last_unstale":"2024-07-17T08:14:22.793659+0000","last_undegraded":"2024-07-17T08:14:22.793659+0000","last_fullsized":"2024-07-17T08:14:22.793659+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:28:07.653793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790543+0000","last_change":"2024-07-17T08:14:21.787713+0000","last_active":"2024-07-17T08:14:22.790543+0000","last_peered":"2024-07-17T08:14:22.790543+0000","last_clean":"2024-07-17T08:14:22.790543+0000","last_became_active":"2024-07-17T08:14:21.787378+0000","last_became_peered":"2024-07-17T08:14:21.787378+0000","last_unstale":"2024-07-17T08:14:22.790543+0000","last_undegraded":"2024-07-17T08:14:22.790543+0000","last_fullsized":"2024-07-17T08:14:22.790543+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:33:34.157844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812603+0000","last_change":"2024-07-17T08:14:20.765656+0000","last_active":"2024-07-17T08:14:21.812603+0000","last_peered":"2024-07-17T08:14:21.812603+0000","last_clean":"2024-07-17T08:14:21.812603+0000","last_became_active":"2024-07-17T08:14:20.760900+0000","last_became_peered":"2024-07-17T08:14:20.760900+0000","last_unstale":"2024-07-17T08:14:21.812603+0000","last_undegraded":"2024-07-17T08:14:21.812603+0000","last_fullsized":"2024-07-17T08:14:21.812603+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:56.988606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789894+0000","last_change":"2024-07-17T08:14:21.801957+0000","last_active":"2024-07-17T08:14:22.789894+0000","last_peered":"2024-07-17T08:14:22.789894+0000","last_clean":"2024-07-17T08:14:22.789894+0000","last_became_active":"2024-07-17T08:14:21.801749+0000","last_became_peered":"2024-07-17T08:14:21.801749+0000","last_unstale":"2024-07-17T08:14:22.789894+0000","last_undegraded":"2024-07-17T08:14:22.789894+0000","last_fullsized":"2024-07-17T08:14:22.789894+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:20.158513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793521+0000","last_change":"2024-07-17T08:14:20.765387+0000","last_active":"2024-07-17T08:14:22.793521+0000","last_peered":"2024-07-17T08:14:22.793521+0000","last_clean":"2024-07-17T08:14:22.793521+0000","last_became_active":"2024-07-17T08:14:20.765076+0000","last_became_peered":"2024-07-17T08:14:20.765076+0000","last_unstale":"2024-07-17T08:14:22.793521+0000","last_undegraded":"2024-07-17T08:14:22.793521+0000","last_fullsized":"2024-07-17T08:14:22.793521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:48.749900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785732+0000","last_change":"2024-07-17T08:14:21.787981+0000","last_active":"2024-07-17T08:14:22.785732+0000","last_peered":"2024-07-17T08:14:22.785732+0000","last_clean":"2024-07-17T08:14:22.785732+0000","last_became_active":"2024-07-17T08:14:21.785399+0000","last_became_peered":"2024-07-17T08:14:21.785399+0000","last_unstale":"2024-07-17T08:14:22.785732+0000","last_undegraded":"2024-07-17T08:14:22.785732+0000","last_fullsized":"2024-07-17T08:14:22.785732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:27.713497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815560+0000","last_change":"2024-07-17T08:14:20.762072+0000","last_active":"2024-07-17T08:14:21.815560+0000","last_peered":"2024-07-17T08:14:21.815560+0000","last_clean":"2024-07-17T08:14:21.815560+0000","last_became_active":"2024-07-17T08:14:20.759687+0000","last_became_peered":"2024-07-17T08:14:20.759687+0000","last_unstale":"2024-07-17T08:14:21.815560+0000","last_undegraded":"2024-07-17T08:14:21.815560+0000","last_fullsized":"2024-07-17T08:14:21.815560+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:12.585687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.800753+0000","last_change":"2024-07-17T08:14:21.789710+0000","last_active":"2024-07-17T08:14:21.800753+0000","last_peered":"2024-07-17T08:14:21.800753+0000","last_clean":"2024-07-17T08:14:21.800753+0000","last_became_active":"2024-07-17T08:14:21.789422+0000","last_became_peered":"2024-07-17T08:14:21.789422+0000","last_unstale":"2024-07-17T08:14:21.800753+0000","last_undegraded":"2024-07-17T08:14:21.800753+0000","last_fullsized":"2024-07-17T08:14:21.800753+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:03.803183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790655+0000","last_change":"2024-07-17T08:14:20.763024+0000","last_active":"2024-07-17T08:14:22.790655+0000","last_peered":"2024-07-17T08:14:22.790655+0000","last_clean":"2024-07-17T08:14:22.790655+0000","last_became_active":"2024-07-17T08:14:20.761934+0000","last_became_peered":"2024-07-17T08:14:20.761934+0000","last_unstale":"2024-07-17T08:14:22.790655+0000","last_undegraded":"2024-07-17T08:14:22.790655+0000","last_fullsized":"2024-07-17T08:14:22.790655+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:42.885674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.805507+0000","last_change":"2024-07-17T08:14:21.803712+0000","last_active":"2024-07-17T08:14:21.805507+0000","last_peered":"2024-07-17T08:14:21.805507+0000","last_clean":"2024-07-17T08:14:21.805507+0000","last_became_active":"2024-07-17T08:14:21.803434+0000","last_became_peered":"2024-07-17T08:14:21.803434+0000","last_unstale":"2024-07-17T08:14:21.805507+0000","last_undegraded":"2024-07-17T08:14:21.805507+0000","last_fullsized":"2024-07-17T08:14:21.805507+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:11.432781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.767951+0000","last_change":"2024-07-17T08:14:20.765798+0000","last_active":"2024-07-17T08:14:21.767951+0000","last_peered":"2024-07-17T08:14:21.767951+0000","last_clean":"2024-07-17T08:14:21.767951+0000","last_became_active":"2024-07-17T08:14:20.765499+0000","last_became_peered":"2024-07-17T08:14:20.765499+0000","last_unstale":"2024-07-17T08:14:21.767951+0000","last_undegraded":"2024-07-17T08:14:21.767951+0000","last_fullsized":"2024-07-17T08:14:21.767951+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:38:09.285542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796105+0000","last_change":"2024-07-17T08:14:21.793958+0000","last_active":"2024-07-17T08:14:21.796105+0000","last_peered":"2024-07-17T08:14:21.796105+0000","last_clean":"2024-07-17T08:14:21.796105+0000","last_became_active":"2024-07-17T08:14:21.792126+0000","last_became_peered":"2024-07-17T08:14:21.792126+0000","last_unstale":"2024-07-17T08:14:21.796105+0000","last_undegraded":"2024-07-17T08:14:21.796105+0000","last_fullsized":"2024-07-17T08:14:21.796105+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:35.916073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.775734+0000","last_change":"2024-07-17T08:14:20.760401+0000","last_active":"2024-07-17T08:14:21.775734+0000","last_peered":"2024-07-17T08:14:21.775734+0000","last_clean":"2024-07-17T08:14:21.775734+0000","last_became_active":"2024-07-17T08:14:20.760092+0000","last_became_peered":"2024-07-17T08:14:20.760092+0000","last_unstale":"2024-07-17T08:14:21.775734+0000","last_undegraded":"2024-07-17T08:14:21.775734+0000","last_fullsized":"2024-07-17T08:14:21.775734+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:57.172385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787972+0000","last_change":"2024-07-17T08:14:21.788380+0000","last_active":"2024-07-17T08:14:22.787972+0000","last_peered":"2024-07-17T08:14:22.787972+0000","last_clean":"2024-07-17T08:14:22.787972+0000","last_became_active":"2024-07-17T08:14:21.787728+0000","last_became_peered":"2024-07-17T08:14:21.787728+0000","last_unstale":"2024-07-17T08:14:22.787972+0000","last_undegraded":"2024-07-17T08:14:22.787972+0000","last_fullsized":"2024-07-17T08:14:22.787972+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:44:30.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812792+0000","last_change":"2024-07-17T08:14:20.764995+0000","last_active":"2024-07-17T08:14:21.812792+0000","last_peered":"2024-07-17T08:14:21.812792+0000","last_clean":"2024-07-17T08:14:21.812792+0000","last_became_active":"2024-07-17T08:14:20.761411+0000","last_became_peered":"2024-07-17T08:14:20.761411+0000","last_unstale":"2024-07-17T08:14:21.812792+0000","last_undegraded":"2024-07-17T08:14:21.812792+0000","last_fullsized":"2024-07-17T08:14:21.812792+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:08.640294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793028+0000","last_change":"2024-07-17T08:14:21.793248+0000","last_active":"2024-07-17T08:14:22.793028+0000","last_peered":"2024-07-17T08:14:22.793028+0000","last_clean":"2024-07-17T08:14:22.793028+0000","last_became_active":"2024-07-17T08:14:21.792508+0000","last_became_peered":"2024-07-17T08:14:21.792508+0000","last_unstale":"2024-07-17T08:14:22.793028+0000","last_undegraded":"2024-07-17T08:14:22.793028+0000","last_fullsized":"2024-07-17T08:14:22.793028+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:27.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"189'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.791932+0000","last_change":"2024-07-17T08:14:20.767034+0000","last_active":"2024-07-17T08:14:22.791932+0000","last_peered":"2024-07-17T08:14:22.791932+0000","last_clean":"2024-07-17T08:14:22.791932+0000","last_became_active":"2024-07-17T08:14:20.766400+0000","last_became_peered":"2024-07-17T08:14:20.766400+0000","last_unstale":"2024-07-17T08:14:22.791932+0000","last_undegraded":"2024-07-17T08:14:22.791932+0000","last_fullsized":"2024-07-17T08:14:22.791932+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:23.477566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.791887+0000","last_change":"2024-07-17T08:14:21.783480+0000","last_active":"2024-07-17T08:14:21.791887+0000","last_peered":"2024-07-17T08:14:21.791887+0000","last_clean":"2024-07-17T08:14:21.791887+0000","last_became_active":"2024-07-17T08:14:21.783156+0000","last_became_peered":"2024-07-17T08:14:21.783156+0000","last_unstale":"2024-07-17T08:14:21.791887+0000","last_undegraded":"2024-07-17T08:14:21.791887+0000","last_fullsized":"2024-07-17T08:14:21.791887+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:34.370323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785622+0000","last_change":"2024-07-17T08:14:20.756607+0000","last_active":"2024-07-17T08:14:22.785622+0000","last_peered":"2024-07-17T08:14:22.785622+0000","last_clean":"2024-07-17T08:14:22.785622+0000","last_became_active":"2024-07-17T08:14:20.755139+0000","last_became_peered":"2024-07-17T08:14:20.755139+0000","last_unstale":"2024-07-17T08:14:22.785622+0000","last_undegraded":"2024-07-17T08:14:22.785622+0000","last_fullsized":"2024-07-17T08:14:22.785622+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:59.795046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792994+0000","last_change":"2024-07-17T08:14:21.799265+0000","last_active":"2024-07-17T08:14:22.792994+0000","last_peered":"2024-07-17T08:14:22.792994+0000","last_clean":"2024-07-17T08:14:22.792994+0000","last_became_active":"2024-07-17T08:14:21.798771+0000","last_became_peered":"2024-07-17T08:14:21.798771+0000","last_unstale":"2024-07-17T08:14:22.792994+0000","last_undegraded":"2024-07-17T08:14:22.792994+0000","last_fullsized":"2024-07-17T08:14:22.792994+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:53.321905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.765672+0000","last_change":"2024-07-17T08:14:20.767416+0000","last_active":"2024-07-17T08:14:21.765672+0000","last_peered":"2024-07-17T08:14:21.765672+0000","last_clean":"2024-07-17T08:14:21.765672+0000","last_became_active":"2024-07-17T08:14:20.767120+0000","last_became_peered":"2024-07-17T08:14:20.767120+0000","last_unstale":"2024-07-17T08:14:21.765672+0000","last_undegraded":"2024-07-17T08:14:21.765672+0000","last_fullsized":"2024-07-17T08:14:21.765672+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:53:30.693634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337409+0000","last_change":"2024-07-17T08:14:21.778545+0000","last_active":"2024-07-17T08:14:23.337409+0000","last_peered":"2024-07-17T08:14:23.337409+0000","last_clean":"2024-07-17T08:14:23.337409+0000","last_became_active":"2024-07-17T08:14:21.778158+0000","last_became_peered":"2024-07-17T08:14:21.778158+0000","last_unstale":"2024-07-17T08:14:23.337409+0000","last_undegraded":"2024-07-17T08:14:23.337409+0000","last_fullsized":"2024-07-17T08:14:23.337409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:28.529855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337264+0000","last_change":"2024-07-17T08:14:20.764148+0000","last_active":"2024-07-17T08:14:23.337264+0000","last_peered":"2024-07-17T08:14:23.337264+0000","last_clean":"2024-07-17T08:14:23.337264+0000","last_became_active":"2024-07-17T08:14:20.763764+0000","last_became_peered":"2024-07-17T08:14:20.763764+0000","last_unstale":"2024-07-17T08:14:23.337264+0000","last_undegraded":"2024-07-17T08:14:23.337264+0000","last_fullsized":"2024-07-17T08:14:23.337264+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:55.832810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796568+0000","last_change":"2024-07-17T08:14:21.795571+0000","last_active":"2024-07-17T08:14:21.796568+0000","last_peered":"2024-07-17T08:14:21.796568+0000","last_clean":"2024-07-17T08:14:21.796568+0000","last_became_active":"2024-07-17T08:14:21.794561+0000","last_became_peered":"2024-07-17T08:14:21.794561+0000","last_unstale":"2024-07-17T08:14:21.796568+0000","last_undegraded":"2024-07-17T08:14:21.796568+0000","last_fullsized":"2024-07-17T08:14:21.796568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:26.714903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785289+0000","last_change":"2024-07-17T08:14:20.753576+0000","last_active":"2024-07-17T08:14:22.785289+0000","last_peered":"2024-07-17T08:14:22.785289+0000","last_clean":"2024-07-17T08:14:22.785289+0000","last_became_active":"2024-07-17T08:14:20.753229+0000","last_became_peered":"2024-07-17T08:14:20.753229+0000","last_unstale":"2024-07-17T08:14:22.785289+0000","last_undegraded":"2024-07-17T08:14:22.785289+0000","last_fullsized":"2024-07-17T08:14:22.785289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:12.585687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785986+0000","last_change":"2024-07-17T08:14:21.791663+0000","last_active":"2024-07-17T08:14:22.785986+0000","last_peered":"2024-07-17T08:14:22.785986+0000","last_clean":"2024-07-17T08:14:22.785986+0000","last_became_active":"2024-07-17T08:14:21.791154+0000","last_became_peered":"2024-07-17T08:14:21.791154+0000","last_unstale":"2024-07-17T08:14:22.785986+0000","last_undegraded":"2024-07-17T08:14:22.785986+0000","last_fullsized":"2024-07-17T08:14:22.785986+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:47.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790432+0000","last_change":"2024-07-17T08:14:20.755453+0000","last_active":"2024-07-17T08:14:22.790432+0000","last_peered":"2024-07-17T08:14:22.790432+0000","last_clean":"2024-07-17T08:14:22.790432+0000","last_became_active":"2024-07-17T08:14:20.755119+0000","last_became_peered":"2024-07-17T08:14:20.755119+0000","last_unstale":"2024-07-17T08:14:22.790432+0000","last_undegraded":"2024-07-17T08:14:22.790432+0000","last_fullsized":"2024-07-17T08:14:22.790432+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:07.731068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786226+0000","last_change":"2024-07-17T08:14:21.781679+0000","last_active":"2024-07-17T08:14:21.786226+0000","last_peered":"2024-07-17T08:14:21.786226+0000","last_clean":"2024-07-17T08:14:21.786226+0000","last_became_active":"2024-07-17T08:14:21.781364+0000","last_became_peered":"2024-07-17T08:14:21.781364+0000","last_unstale":"2024-07-17T08:14:21.786226+0000","last_undegraded":"2024-07-17T08:14:21.786226+0000","last_fullsized":"2024-07-17T08:14:21.786226+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:30.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.777274+0000","last_change":"2024-07-17T08:14:20.754748+0000","last_active":"2024-07-17T08:14:21.777274+0000","last_peered":"2024-07-17T08:14:21.777274+0000","last_clean":"2024-07-17T08:14:21.777274+0000","last_became_active":"2024-07-17T08:14:20.754475+0000","last_became_peered":"2024-07-17T08:14:20.754475+0000","last_unstale":"2024-07-17T08:14:21.777274+0000","last_undegraded":"2024-07-17T08:14:21.777274+0000","last_fullsized":"2024-07-17T08:14:21.777274+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:59.795046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795778+0000","last_change":"2024-07-17T08:14:21.789040+0000","last_active":"2024-07-17T08:14:21.795778+0000","last_peered":"2024-07-17T08:14:21.795778+0000","last_clean":"2024-07-17T08:14:21.795778+0000","last_became_active":"2024-07-17T08:14:21.785072+0000","last_became_peered":"2024-07-17T08:14:21.785072+0000","last_unstale":"2024-07-17T08:14:21.795778+0000","last_undegraded":"2024-07-17T08:14:21.795778+0000","last_fullsized":"2024-07-17T08:14:21.795778+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:47.533791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785750+0000","last_change":"2024-07-17T08:14:20.767261+0000","last_active":"2024-07-17T08:14:22.785750+0000","last_peered":"2024-07-17T08:14:22.785750+0000","last_clean":"2024-07-17T08:14:22.785750+0000","last_became_active":"2024-07-17T08:14:20.766770+0000","last_became_peered":"2024-07-17T08:14:20.766770+0000","last_unstale":"2024-07-17T08:14:22.785750+0000","last_undegraded":"2024-07-17T08:14:22.785750+0000","last_fullsized":"2024-07-17T08:14:22.785750+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:02.798158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"180'2476","reported_seq":3180,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.775793+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:14:21.775793+0000","last_peered":"2024-07-17T08:14:21.775793+0000","last_clean":"2024-07-17T08:14:21.775793+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:14:21.775793+0000","last_undegraded":"2024-07-17T08:14:21.775793+0000","last_fullsized":"2024-07-17T08:14:21.775793+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":299,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":299,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2476,"ondisk_log_size":2476,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543092,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60420,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311420,"statfs":{"total":96636764160,"available":96574894080,"internally_reserved":0,"allocated":2940928,"data_stored":1854677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510387,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":2359296,"data_stored":1271549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510388,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47384,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324456,"statfs":{"total":96636764160,"available":96588242944,"internally_reserved":0,"allocated":2367488,"data_stored":1271673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543093,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41820,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330020,"statfs":{"total":96636764160,"available":96593940480,"internally_reserved":0,"allocated":2375680,"data_stored":1272313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510387,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64036,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307804,"statfs":{"total":96636764160,"available":96571191296,"internally_reserved":0,"allocated":2379776,"data_stored":1273765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543091,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2347008,"data_stored":1264309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41824,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330016,"statfs":{"total":96636764160,"available":96593936384,"internally_reserved":0,"allocated":2371584,"data_stored":1273161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510387,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34068,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337772,"statfs":{"total":96636764160,"available":96601878528,"internally_reserved":0,"allocated":2949120,"data_stored":1854781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:14:26.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:14:26.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 -- 172.21.15.44:0/572579373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b5400 msgr2=0x7f710815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 --2- 172.21.15.44:0/572579373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b5400 0x7f710815e2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7100009ca0 tx=0x7f710002aac0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 -- 172.21.15.44:0/572579373 shutdown_connections 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 --2- 172.21.15.44:0/572579373 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b5400 0x7f710815e2d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 --2- 172.21.15.44:0/572579373 >> v2:172.21.15.175:3300/0 conn(0x7f71080b8c80 0x7f71080b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.555+0000 7f710e200640 1 -- 172.21.15.44:0/572579373 >> 172.21.15.44:0/572579373 conn(0x7f71080cfec0 msgr2=0x7f71080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.556+0000 7f710e200640 1 -- 172.21.15.44:0/572579373 shutdown_connections 2024-07-17T08:14:26.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.556+0000 7f710e200640 1 -- 172.21.15.44:0/572579373 wait complete. 2024-07-17T08:14:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.557+0000 7f710e200640 1 Processor -- start 2024-07-17T08:14:26.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.558+0000 7f710e200640 1 -- start start 2024-07-17T08:14:26.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.558+0000 7f710e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71080b5400 0x7f71080cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b8c80 0x7f71080aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f710806e0c0 con 0x7f71080ce5c0 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71080b9270 con 0x7f71080b5400 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f710806e240 con 0x7f71080b8c80 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45022/0 (socket says 172.21.15.44:45022) 2024-07-17T08:14:26.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 -- 172.21.15.44:0/2232138740 learned_addr learned my addr 172.21.15.44:0/2232138740 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:26.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f7107e00640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b8c80 0x7f71080aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710ce00640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71080b5400 0x7f71080cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 -- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b8c80 msgr2=0x7f71080aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b8c80 0x7f71080aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 -- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71080b5400 msgr2=0x7f71080cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.559+0000 7f710d800640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71080b5400 0x7f71080cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f710d800640 1 -- 172.21.15.44:0/2232138740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081e7d70 con 0x7f71080ce5c0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f710d800640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7100009bf0 tx=0x7f7100003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100046070 con 0x7f71080ce5c0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7100003e00 con 0x7f71080ce5c0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000044b0 con 0x7f71080ce5c0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710814f370 con 0x7f71080ce5c0 2024-07-17T08:14:26.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.560+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710814f930 con 0x7f71080ce5c0 2024-07-17T08:14:26.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.561+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710806e6f0 con 0x7f71080ce5c0 2024-07-17T08:14:26.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.563+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f7100049070 con 0x7f71080ce5c0 2024-07-17T08:14:26.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.564+0000 7f7104c00640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 0x7f70e4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:26.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.564+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f71000b9230 con 0x7f71080ce5c0 2024-07-17T08:14:26.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.565+0000 7f710ce00640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 0x7f70e4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.566+0000 7f710ce00640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 0x7f70e4079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009b60 tx=0x7f70f0005f90 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.568+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7100032030 con 0x7f71080ce5c0 2024-07-17T08:14:26.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.783+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 --> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71080b87f0 con 0x7f70e4076bb0 2024-07-17T08:14:26.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.795+0000 7f7104c00640 1 -- 172.21.15.44:0/2232138740 <== mgr.7435 v2:172.21.15.175:6834/2760073115 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f71080b87f0 con 0x7f70e4076bb0 2024-07-17T08:14:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.802+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 msgr2=0x7f70e4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 0x7f70e4079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009b60 tx=0x7f70f0005f90 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 msgr2=0x7f71080aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7100009bf0 tx=0x7f7100003c10 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 shutdown_connections 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f70e4076bb0 0x7f70e4079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71080ce5c0 0x7f71080aa800 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71080b8c80 0x7f71080aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 --2- 172.21.15.44:0/2232138740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71080b5400 0x7f71080cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 >> 172.21.15.44:0/2232138740 conn(0x7f71080cfec0 msgr2=0x7f71080d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 shutdown_connections 2024-07-17T08:14:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:26.803+0000 7f710e200640 1 -- 172.21.15.44:0/2232138740 wait complete. 2024-07-17T08:14:26.825 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:14:25.896042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":344,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2515,"ondisk_log_size":2515,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378724,"kb_used_data":19620,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754595996,"statfs":{"total":773094113280,"available":772706299904,"internally_reserved":0,"allocated":20090880,"data_stored":11336228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":367578672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002062"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336897+0000","last_change":"2024-07-17T08:14:21.785290+0000","last_active":"2024-07-17T08:14:23.336897+0000","last_peered":"2024-07-17T08:14:23.336897+0000","last_clean":"2024-07-17T08:14:23.336897+0000","last_became_active":"2024-07-17T08:14:21.785002+0000","last_became_peered":"2024-07-17T08:14:21.785002+0000","last_unstale":"2024-07-17T08:14:23.336897+0000","last_undegraded":"2024-07-17T08:14:23.336897+0000","last_fullsized":"2024-07-17T08:14:23.336897+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:24.571970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.789930+0000","last_change":"2024-07-17T08:14:21.783478+0000","last_active":"2024-07-17T08:14:21.789930+0000","last_peered":"2024-07-17T08:14:21.789930+0000","last_clean":"2024-07-17T08:14:21.789930+0000","last_became_active":"2024-07-17T08:14:21.781759+0000","last_became_peered":"2024-07-17T08:14:21.781759+0000","last_unstale":"2024-07-17T08:14:21.789930+0000","last_undegraded":"2024-07-17T08:14:21.789930+0000","last_fullsized":"2024-07-17T08:14:21.789930+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:38.053464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337754+0000","last_change":"2024-07-17T08:14:21.779561+0000","last_active":"2024-07-17T08:14:23.337754+0000","last_peered":"2024-07-17T08:14:23.337754+0000","last_clean":"2024-07-17T08:14:23.337754+0000","last_became_active":"2024-07-17T08:14:21.779053+0000","last_became_peered":"2024-07-17T08:14:21.779053+0000","last_unstale":"2024-07-17T08:14:23.337754+0000","last_undegraded":"2024-07-17T08:14:23.337754+0000","last_fullsized":"2024-07-17T08:14:23.337754+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:17.390393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337340+0000","last_change":"2024-07-17T08:14:21.781033+0000","last_active":"2024-07-17T08:14:23.337340+0000","last_peered":"2024-07-17T08:14:23.337340+0000","last_clean":"2024-07-17T08:14:23.337340+0000","last_became_active":"2024-07-17T08:14:21.780662+0000","last_became_peered":"2024-07-17T08:14:21.780662+0000","last_unstale":"2024-07-17T08:14:23.337340+0000","last_undegraded":"2024-07-17T08:14:23.337340+0000","last_fullsized":"2024-07-17T08:14:23.337340+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:20.699644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.802302+0000","last_change":"2024-07-17T08:14:21.795330+0000","last_active":"2024-07-17T08:14:21.802302+0000","last_peered":"2024-07-17T08:14:21.802302+0000","last_clean":"2024-07-17T08:14:21.802302+0000","last_became_active":"2024-07-17T08:14:21.794604+0000","last_became_peered":"2024-07-17T08:14:21.794604+0000","last_unstale":"2024-07-17T08:14:21.802302+0000","last_undegraded":"2024-07-17T08:14:21.802302+0000","last_fullsized":"2024-07-17T08:14:21.802302+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:28.727845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795458+0000","last_change":"2024-07-17T08:14:21.787840+0000","last_active":"2024-07-17T08:14:21.795458+0000","last_peered":"2024-07-17T08:14:21.795458+0000","last_clean":"2024-07-17T08:14:21.795458+0000","last_became_active":"2024-07-17T08:14:21.787521+0000","last_became_peered":"2024-07-17T08:14:21.787521+0000","last_unstale":"2024-07-17T08:14:21.795458+0000","last_undegraded":"2024-07-17T08:14:21.795458+0000","last_fullsized":"2024-07-17T08:14:21.795458+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:34:23.468535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792820+0000","last_change":"2024-07-17T08:14:21.793555+0000","last_active":"2024-07-17T08:14:22.792820+0000","last_peered":"2024-07-17T08:14:22.792820+0000","last_clean":"2024-07-17T08:14:22.792820+0000","last_became_active":"2024-07-17T08:14:21.792889+0000","last_became_peered":"2024-07-17T08:14:21.792889+0000","last_unstale":"2024-07-17T08:14:22.792820+0000","last_undegraded":"2024-07-17T08:14:22.792820+0000","last_fullsized":"2024-07-17T08:14:22.792820+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:43.890699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792783+0000","last_change":"2024-07-17T08:14:21.789138+0000","last_active":"2024-07-17T08:14:22.792783+0000","last_peered":"2024-07-17T08:14:22.792783+0000","last_clean":"2024-07-17T08:14:22.792783+0000","last_became_active":"2024-07-17T08:14:21.788876+0000","last_became_peered":"2024-07-17T08:14:21.788876+0000","last_unstale":"2024-07-17T08:14:22.792783+0000","last_undegraded":"2024-07-17T08:14:22.792783+0000","last_fullsized":"2024-07-17T08:14:22.792783+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:08.736092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793681+0000","last_change":"2024-07-17T08:14:21.803641+0000","last_active":"2024-07-17T08:14:22.793681+0000","last_peered":"2024-07-17T08:14:22.793681+0000","last_clean":"2024-07-17T08:14:22.793681+0000","last_became_active":"2024-07-17T08:14:21.802406+0000","last_became_peered":"2024-07-17T08:14:21.802406+0000","last_unstale":"2024-07-17T08:14:22.793681+0000","last_undegraded":"2024-07-17T08:14:22.793681+0000","last_fullsized":"2024-07-17T08:14:22.793681+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:19:08.005085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793528+0000","last_change":"2024-07-17T08:14:21.802094+0000","last_active":"2024-07-17T08:14:22.793528+0000","last_peered":"2024-07-17T08:14:22.793528+0000","last_clean":"2024-07-17T08:14:22.793528+0000","last_became_active":"2024-07-17T08:14:21.801297+0000","last_became_peered":"2024-07-17T08:14:21.801297+0000","last_unstale":"2024-07-17T08:14:22.793528+0000","last_undegraded":"2024-07-17T08:14:22.793528+0000","last_fullsized":"2024-07-17T08:14:22.793528+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:32.217139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789568+0000","last_change":"2024-07-17T08:14:21.799764+0000","last_active":"2024-07-17T08:14:22.789568+0000","last_peered":"2024-07-17T08:14:22.789568+0000","last_clean":"2024-07-17T08:14:22.789568+0000","last_became_active":"2024-07-17T08:14:21.799516+0000","last_became_peered":"2024-07-17T08:14:21.799516+0000","last_unstale":"2024-07-17T08:14:22.789568+0000","last_undegraded":"2024-07-17T08:14:22.789568+0000","last_fullsized":"2024-07-17T08:14:22.789568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:33:58.190239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795715+0000","last_change":"2024-07-17T08:14:21.791201+0000","last_active":"2024-07-17T08:14:21.795715+0000","last_peered":"2024-07-17T08:14:21.795715+0000","last_clean":"2024-07-17T08:14:21.795715+0000","last_became_active":"2024-07-17T08:14:21.790207+0000","last_became_peered":"2024-07-17T08:14:21.790207+0000","last_unstale":"2024-07-17T08:14:21.795715+0000","last_undegraded":"2024-07-17T08:14:21.795715+0000","last_fullsized":"2024-07-17T08:14:21.795715+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:25.483748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785350+0000","last_change":"2024-07-17T08:14:21.780565+0000","last_active":"2024-07-17T08:14:22.785350+0000","last_peered":"2024-07-17T08:14:22.785350+0000","last_clean":"2024-07-17T08:14:22.785350+0000","last_became_active":"2024-07-17T08:14:21.780205+0000","last_became_peered":"2024-07-17T08:14:21.780205+0000","last_unstale":"2024-07-17T08:14:22.785350+0000","last_undegraded":"2024-07-17T08:14:22.785350+0000","last_fullsized":"2024-07-17T08:14:22.785350+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:16:06.860324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.794496+0000","last_change":"2024-07-17T08:14:21.786961+0000","last_active":"2024-07-17T08:14:21.794496+0000","last_peered":"2024-07-17T08:14:21.794496+0000","last_clean":"2024-07-17T08:14:21.794496+0000","last_became_active":"2024-07-17T08:14:21.784107+0000","last_became_peered":"2024-07-17T08:14:21.784107+0000","last_unstale":"2024-07-17T08:14:21.794496+0000","last_undegraded":"2024-07-17T08:14:21.794496+0000","last_fullsized":"2024-07-17T08:14:21.794496+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:26.340215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790579+0000","last_change":"2024-07-17T08:14:21.789019+0000","last_active":"2024-07-17T08:14:22.790579+0000","last_peered":"2024-07-17T08:14:22.790579+0000","last_clean":"2024-07-17T08:14:22.790579+0000","last_became_active":"2024-07-17T08:14:21.788581+0000","last_became_peered":"2024-07-17T08:14:21.788581+0000","last_unstale":"2024-07-17T08:14:22.790579+0000","last_undegraded":"2024-07-17T08:14:22.790579+0000","last_fullsized":"2024-07-17T08:14:22.790579+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:20.582643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336513+0000","last_change":"2024-07-17T08:14:21.781022+0000","last_active":"2024-07-17T08:14:23.336513+0000","last_peered":"2024-07-17T08:14:23.336513+0000","last_clean":"2024-07-17T08:14:23.336513+0000","last_became_active":"2024-07-17T08:14:21.780331+0000","last_became_peered":"2024-07-17T08:14:21.780331+0000","last_unstale":"2024-07-17T08:14:23.336513+0000","last_undegraded":"2024-07-17T08:14:23.336513+0000","last_fullsized":"2024-07-17T08:14:23.336513+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:46.616389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793767+0000","last_change":"2024-07-17T08:14:21.793907+0000","last_active":"2024-07-17T08:14:22.793767+0000","last_peered":"2024-07-17T08:14:22.793767+0000","last_clean":"2024-07-17T08:14:22.793767+0000","last_became_active":"2024-07-17T08:14:21.793398+0000","last_became_peered":"2024-07-17T08:14:21.793398+0000","last_unstale":"2024-07-17T08:14:22.793767+0000","last_undegraded":"2024-07-17T08:14:22.793767+0000","last_fullsized":"2024-07-17T08:14:22.793767+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:23.721805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.788477+0000","last_change":"2024-07-17T08:14:21.793900+0000","last_active":"2024-07-17T08:14:22.788477+0000","last_peered":"2024-07-17T08:14:22.788477+0000","last_clean":"2024-07-17T08:14:22.788477+0000","last_became_active":"2024-07-17T08:14:21.793582+0000","last_became_peered":"2024-07-17T08:14:21.793582+0000","last_unstale":"2024-07-17T08:14:22.788477+0000","last_undegraded":"2024-07-17T08:14:22.788477+0000","last_fullsized":"2024-07-17T08:14:22.788477+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:18:33.274175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790033+0000","last_change":"2024-07-17T08:14:21.784903+0000","last_active":"2024-07-17T08:14:22.790033+0000","last_peered":"2024-07-17T08:14:22.790033+0000","last_clean":"2024-07-17T08:14:22.790033+0000","last_became_active":"2024-07-17T08:14:21.784586+0000","last_became_peered":"2024-07-17T08:14:21.784586+0000","last_unstale":"2024-07-17T08:14:22.790033+0000","last_undegraded":"2024-07-17T08:14:22.790033+0000","last_fullsized":"2024-07-17T08:14:22.790033+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:44:01.636186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786618+0000","last_change":"2024-07-17T08:14:21.781461+0000","last_active":"2024-07-17T08:14:21.786618+0000","last_peered":"2024-07-17T08:14:21.786618+0000","last_clean":"2024-07-17T08:14:21.786618+0000","last_became_active":"2024-07-17T08:14:21.778131+0000","last_became_peered":"2024-07-17T08:14:21.778131+0000","last_unstale":"2024-07-17T08:14:21.786618+0000","last_undegraded":"2024-07-17T08:14:21.786618+0000","last_fullsized":"2024-07-17T08:14:21.786618+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:36.646927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785574+0000","last_change":"2024-07-17T08:14:21.772831+0000","last_active":"2024-07-17T08:14:22.785574+0000","last_peered":"2024-07-17T08:14:22.785574+0000","last_clean":"2024-07-17T08:14:22.785574+0000","last_became_active":"2024-07-17T08:14:21.771958+0000","last_became_peered":"2024-07-17T08:14:21.771958+0000","last_unstale":"2024-07-17T08:14:22.785574+0000","last_undegraded":"2024-07-17T08:14:22.785574+0000","last_fullsized":"2024-07-17T08:14:22.785574+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:53.349723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.797995+0000","last_change":"2024-07-17T08:14:21.797468+0000","last_active":"2024-07-17T08:14:21.797995+0000","last_peered":"2024-07-17T08:14:21.797995+0000","last_clean":"2024-07-17T08:14:21.797995+0000","last_became_active":"2024-07-17T08:14:21.797053+0000","last_became_peered":"2024-07-17T08:14:21.797053+0000","last_unstale":"2024-07-17T08:14:21.797995+0000","last_undegraded":"2024-07-17T08:14:21.797995+0000","last_fullsized":"2024-07-17T08:14:21.797995+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:56.150796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.801079+0000","last_change":"2024-07-17T08:14:21.798809+0000","last_active":"2024-07-17T08:14:21.801079+0000","last_peered":"2024-07-17T08:14:21.801079+0000","last_clean":"2024-07-17T08:14:21.801079+0000","last_became_active":"2024-07-17T08:14:21.792158+0000","last_became_peered":"2024-07-17T08:14:21.792158+0000","last_unstale":"2024-07-17T08:14:21.801079+0000","last_undegraded":"2024-07-17T08:14:21.801079+0000","last_fullsized":"2024-07-17T08:14:21.801079+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:06.381735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.336805+0000","last_change":"2024-07-17T08:14:21.793060+0000","last_active":"2024-07-17T08:14:23.336805+0000","last_peered":"2024-07-17T08:14:23.336805+0000","last_clean":"2024-07-17T08:14:23.336805+0000","last_became_active":"2024-07-17T08:14:21.791797+0000","last_became_peered":"2024-07-17T08:14:21.791797+0000","last_unstale":"2024-07-17T08:14:23.336805+0000","last_undegraded":"2024-07-17T08:14:23.336805+0000","last_fullsized":"2024-07-17T08:14:23.336805+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:07.048034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.786037+0000","last_change":"2024-07-17T08:14:21.778734+0000","last_active":"2024-07-17T08:14:22.786037+0000","last_peered":"2024-07-17T08:14:22.786037+0000","last_clean":"2024-07-17T08:14:22.786037+0000","last_became_active":"2024-07-17T08:14:21.778407+0000","last_became_peered":"2024-07-17T08:14:21.778407+0000","last_unstale":"2024-07-17T08:14:22.786037+0000","last_undegraded":"2024-07-17T08:14:22.786037+0000","last_fullsized":"2024-07-17T08:14:22.786037+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:37.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.787874+0000","last_change":"2024-07-17T08:14:21.779540+0000","last_active":"2024-07-17T08:14:21.787874+0000","last_peered":"2024-07-17T08:14:21.787874+0000","last_clean":"2024-07-17T08:14:21.787874+0000","last_became_active":"2024-07-17T08:14:21.779222+0000","last_became_peered":"2024-07-17T08:14:21.779222+0000","last_unstale":"2024-07-17T08:14:21.787874+0000","last_undegraded":"2024-07-17T08:14:21.787874+0000","last_fullsized":"2024-07-17T08:14:21.787874+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:06.014027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795741+0000","last_change":"2024-07-17T08:14:21.789034+0000","last_active":"2024-07-17T08:14:21.795741+0000","last_peered":"2024-07-17T08:14:21.795741+0000","last_clean":"2024-07-17T08:14:21.795741+0000","last_became_active":"2024-07-17T08:14:21.785800+0000","last_became_peered":"2024-07-17T08:14:21.785800+0000","last_unstale":"2024-07-17T08:14:21.795741+0000","last_undegraded":"2024-07-17T08:14:21.795741+0000","last_fullsized":"2024-07-17T08:14:21.795741+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:07.075917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785529+0000","last_change":"2024-07-17T08:14:21.789653+0000","last_active":"2024-07-17T08:14:22.785529+0000","last_peered":"2024-07-17T08:14:22.785529+0000","last_clean":"2024-07-17T08:14:22.785529+0000","last_became_active":"2024-07-17T08:14:21.789319+0000","last_became_peered":"2024-07-17T08:14:21.789319+0000","last_unstale":"2024-07-17T08:14:22.785529+0000","last_undegraded":"2024-07-17T08:14:22.785529+0000","last_fullsized":"2024-07-17T08:14:22.785529+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:08.403272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796510+0000","last_change":"2024-07-17T08:14:21.795390+0000","last_active":"2024-07-17T08:14:21.796510+0000","last_peered":"2024-07-17T08:14:21.796510+0000","last_clean":"2024-07-17T08:14:21.796510+0000","last_became_active":"2024-07-17T08:14:21.793637+0000","last_became_peered":"2024-07-17T08:14:21.793637+0000","last_unstale":"2024-07-17T08:14:21.796510+0000","last_undegraded":"2024-07-17T08:14:21.796510+0000","last_fullsized":"2024-07-17T08:14:21.796510+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:23.622454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787894+0000","last_change":"2024-07-17T08:14:21.792364+0000","last_active":"2024-07-17T08:14:22.787894+0000","last_peered":"2024-07-17T08:14:22.787894+0000","last_clean":"2024-07-17T08:14:22.787894+0000","last_became_active":"2024-07-17T08:14:21.789378+0000","last_became_peered":"2024-07-17T08:14:21.789378+0000","last_unstale":"2024-07-17T08:14:22.787894+0000","last_undegraded":"2024-07-17T08:14:22.787894+0000","last_fullsized":"2024-07-17T08:14:22.787894+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:27.503110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789897+0000","last_change":"2024-07-17T08:14:21.786905+0000","last_active":"2024-07-17T08:14:22.789897+0000","last_peered":"2024-07-17T08:14:22.789897+0000","last_clean":"2024-07-17T08:14:22.789897+0000","last_became_active":"2024-07-17T08:14:21.786599+0000","last_became_peered":"2024-07-17T08:14:21.786599+0000","last_unstale":"2024-07-17T08:14:22.789897+0000","last_undegraded":"2024-07-17T08:14:22.789897+0000","last_fullsized":"2024-07-17T08:14:22.789897+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:42:05.121612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787831+0000","last_change":"2024-07-17T08:14:21.781480+0000","last_active":"2024-07-17T08:14:22.787831+0000","last_peered":"2024-07-17T08:14:22.787831+0000","last_clean":"2024-07-17T08:14:22.787831+0000","last_became_active":"2024-07-17T08:14:21.781219+0000","last_became_peered":"2024-07-17T08:14:21.781219+0000","last_unstale":"2024-07-17T08:14:22.787831+0000","last_undegraded":"2024-07-17T08:14:22.787831+0000","last_fullsized":"2024-07-17T08:14:22.787831+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:26.652431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795752+0000","last_change":"2024-07-17T08:14:21.793883+0000","last_active":"2024-07-17T08:14:21.795752+0000","last_peered":"2024-07-17T08:14:21.795752+0000","last_clean":"2024-07-17T08:14:21.795752+0000","last_became_active":"2024-07-17T08:14:21.793558+0000","last_became_peered":"2024-07-17T08:14:21.793558+0000","last_unstale":"2024-07-17T08:14:21.795752+0000","last_undegraded":"2024-07-17T08:14:21.795752+0000","last_fullsized":"2024-07-17T08:14:21.795752+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:18:47.779249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"188'4","reported_seq":18,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337361+0000","last_change":"2024-07-17T08:14:20.752063+0000","last_active":"2024-07-17T08:14:23.337361+0000","last_peered":"2024-07-17T08:14:23.337361+0000","last_clean":"2024-07-17T08:14:23.337361+0000","last_became_active":"2024-07-17T08:14:20.751784+0000","last_became_peered":"2024-07-17T08:14:20.751784+0000","last_unstale":"2024-07-17T08:14:23.337361+0000","last_undegraded":"2024-07-17T08:14:23.337361+0000","last_fullsized":"2024-07-17T08:14:23.337361+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:57.779035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785072+0000","last_change":"2024-07-17T08:14:21.782352+0000","last_active":"2024-07-17T08:14:22.785072+0000","last_peered":"2024-07-17T08:14:22.785072+0000","last_clean":"2024-07-17T08:14:22.785072+0000","last_became_active":"2024-07-17T08:14:21.782061+0000","last_became_peered":"2024-07-17T08:14:21.782061+0000","last_unstale":"2024-07-17T08:14:22.785072+0000","last_undegraded":"2024-07-17T08:14:22.785072+0000","last_fullsized":"2024-07-17T08:14:22.785072+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:31.207100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815571+0000","last_change":"2024-07-17T08:14:20.767918+0000","last_active":"2024-07-17T08:14:21.815571+0000","last_peered":"2024-07-17T08:14:21.815571+0000","last_clean":"2024-07-17T08:14:21.815571+0000","last_became_active":"2024-07-17T08:14:20.767516+0000","last_became_peered":"2024-07-17T08:14:20.767516+0000","last_unstale":"2024-07-17T08:14:21.815571+0000","last_undegraded":"2024-07-17T08:14:21.815571+0000","last_fullsized":"2024-07-17T08:14:21.815571+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:22.917406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.800764+0000","last_change":"2024-07-17T08:14:21.787465+0000","last_active":"2024-07-17T08:14:21.800764+0000","last_peered":"2024-07-17T08:14:21.800764+0000","last_clean":"2024-07-17T08:14:21.800764+0000","last_became_active":"2024-07-17T08:14:21.783683+0000","last_became_peered":"2024-07-17T08:14:21.783683+0000","last_unstale":"2024-07-17T08:14:21.800764+0000","last_undegraded":"2024-07-17T08:14:21.800764+0000","last_fullsized":"2024-07-17T08:14:21.800764+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:51.256299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790423+0000","last_change":"2024-07-17T08:14:20.760499+0000","last_active":"2024-07-17T08:14:22.790423+0000","last_peered":"2024-07-17T08:14:22.790423+0000","last_clean":"2024-07-17T08:14:22.790423+0000","last_became_active":"2024-07-17T08:14:20.760181+0000","last_became_peered":"2024-07-17T08:14:20.760181+0000","last_unstale":"2024-07-17T08:14:22.790423+0000","last_undegraded":"2024-07-17T08:14:22.790423+0000","last_fullsized":"2024-07-17T08:14:22.790423+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:53.865648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796788+0000","last_change":"2024-07-17T08:14:21.796084+0000","last_active":"2024-07-17T08:14:21.796788+0000","last_peered":"2024-07-17T08:14:21.796788+0000","last_clean":"2024-07-17T08:14:21.796788+0000","last_became_active":"2024-07-17T08:14:21.794683+0000","last_became_peered":"2024-07-17T08:14:21.794683+0000","last_unstale":"2024-07-17T08:14:21.796788+0000","last_undegraded":"2024-07-17T08:14:21.796788+0000","last_fullsized":"2024-07-17T08:14:21.796788+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:20.663185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812541+0000","last_change":"2024-07-17T08:14:20.760206+0000","last_active":"2024-07-17T08:14:21.812541+0000","last_peered":"2024-07-17T08:14:21.812541+0000","last_clean":"2024-07-17T08:14:21.812541+0000","last_became_active":"2024-07-17T08:14:20.759131+0000","last_became_peered":"2024-07-17T08:14:20.759131+0000","last_unstale":"2024-07-17T08:14:21.812541+0000","last_undegraded":"2024-07-17T08:14:21.812541+0000","last_fullsized":"2024-07-17T08:14:21.812541+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:48:29.692936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793495+0000","last_change":"2024-07-17T08:14:21.800183+0000","last_active":"2024-07-17T08:14:22.793495+0000","last_peered":"2024-07-17T08:14:22.793495+0000","last_clean":"2024-07-17T08:14:22.793495+0000","last_became_active":"2024-07-17T08:14:21.799919+0000","last_became_peered":"2024-07-17T08:14:21.799919+0000","last_unstale":"2024-07-17T08:14:22.793495+0000","last_undegraded":"2024-07-17T08:14:22.793495+0000","last_fullsized":"2024-07-17T08:14:22.793495+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:50.347629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793135+0000","last_change":"2024-07-17T08:14:20.764705+0000","last_active":"2024-07-17T08:14:22.793135+0000","last_peered":"2024-07-17T08:14:22.793135+0000","last_clean":"2024-07-17T08:14:22.793135+0000","last_became_active":"2024-07-17T08:14:20.764402+0000","last_became_peered":"2024-07-17T08:14:20.764402+0000","last_unstale":"2024-07-17T08:14:22.793135+0000","last_undegraded":"2024-07-17T08:14:22.793135+0000","last_fullsized":"2024-07-17T08:14:22.793135+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:02.733153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.779418+0000","last_change":"2024-07-17T08:14:21.777319+0000","last_active":"2024-07-17T08:14:21.779418+0000","last_peered":"2024-07-17T08:14:21.779418+0000","last_clean":"2024-07-17T08:14:21.779418+0000","last_became_active":"2024-07-17T08:14:21.774244+0000","last_became_peered":"2024-07-17T08:14:21.774244+0000","last_unstale":"2024-07-17T08:14:21.779418+0000","last_undegraded":"2024-07-17T08:14:21.779418+0000","last_fullsized":"2024-07-17T08:14:21.779418+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:01:14.809600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815179+0000","last_change":"2024-07-17T08:14:20.750186+0000","last_active":"2024-07-17T08:14:21.815179+0000","last_peered":"2024-07-17T08:14:21.815179+0000","last_clean":"2024-07-17T08:14:21.815179+0000","last_became_active":"2024-07-17T08:14:20.749069+0000","last_became_peered":"2024-07-17T08:14:20.749069+0000","last_unstale":"2024-07-17T08:14:21.815179+0000","last_undegraded":"2024-07-17T08:14:21.815179+0000","last_fullsized":"2024-07-17T08:14:21.815179+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:30.257455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795048+0000","last_change":"2024-07-17T08:14:21.791654+0000","last_active":"2024-07-17T08:14:21.795048+0000","last_peered":"2024-07-17T08:14:21.795048+0000","last_clean":"2024-07-17T08:14:21.795048+0000","last_became_active":"2024-07-17T08:14:21.788459+0000","last_became_peered":"2024-07-17T08:14:21.788459+0000","last_unstale":"2024-07-17T08:14:21.795048+0000","last_undegraded":"2024-07-17T08:14:21.795048+0000","last_fullsized":"2024-07-17T08:14:21.795048+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:24.676597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.764998+0000","last_change":"2024-07-17T08:14:20.760709+0000","last_active":"2024-07-17T08:14:21.764998+0000","last_peered":"2024-07-17T08:14:21.764998+0000","last_clean":"2024-07-17T08:14:21.764998+0000","last_became_active":"2024-07-17T08:14:20.760392+0000","last_became_peered":"2024-07-17T08:14:20.760392+0000","last_unstale":"2024-07-17T08:14:21.764998+0000","last_undegraded":"2024-07-17T08:14:21.764998+0000","last_fullsized":"2024-07-17T08:14:21.764998+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:57.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337039+0000","last_change":"2024-07-17T08:14:21.782855+0000","last_active":"2024-07-17T08:14:23.337039+0000","last_peered":"2024-07-17T08:14:23.337039+0000","last_clean":"2024-07-17T08:14:23.337039+0000","last_became_active":"2024-07-17T08:14:21.779257+0000","last_became_peered":"2024-07-17T08:14:21.779257+0000","last_unstale":"2024-07-17T08:14:23.337039+0000","last_undegraded":"2024-07-17T08:14:23.337039+0000","last_fullsized":"2024-07-17T08:14:23.337039+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:56.644834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792845+0000","last_change":"2024-07-17T08:14:20.766884+0000","last_active":"2024-07-17T08:14:22.792845+0000","last_peered":"2024-07-17T08:14:22.792845+0000","last_clean":"2024-07-17T08:14:22.792845+0000","last_became_active":"2024-07-17T08:14:20.765185+0000","last_became_peered":"2024-07-17T08:14:20.765185+0000","last_unstale":"2024-07-17T08:14:22.792845+0000","last_undegraded":"2024-07-17T08:14:22.792845+0000","last_fullsized":"2024-07-17T08:14:22.792845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:54.329058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795733+0000","last_change":"2024-07-17T08:14:21.791691+0000","last_active":"2024-07-17T08:14:21.795733+0000","last_peered":"2024-07-17T08:14:21.795733+0000","last_clean":"2024-07-17T08:14:21.795733+0000","last_became_active":"2024-07-17T08:14:21.791359+0000","last_became_peered":"2024-07-17T08:14:21.791359+0000","last_unstale":"2024-07-17T08:14:21.795733+0000","last_undegraded":"2024-07-17T08:14:21.795733+0000","last_fullsized":"2024-07-17T08:14:21.795733+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:08:42.707288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785840+0000","last_change":"2024-07-17T08:14:20.767778+0000","last_active":"2024-07-17T08:14:22.785840+0000","last_peered":"2024-07-17T08:14:22.785840+0000","last_clean":"2024-07-17T08:14:22.785840+0000","last_became_active":"2024-07-17T08:14:20.765968+0000","last_became_peered":"2024-07-17T08:14:20.765968+0000","last_unstale":"2024-07-17T08:14:22.785840+0000","last_undegraded":"2024-07-17T08:14:22.785840+0000","last_fullsized":"2024-07-17T08:14:22.785840+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:42.971063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785148+0000","last_change":"2024-07-17T08:14:21.791533+0000","last_active":"2024-07-17T08:14:22.785148+0000","last_peered":"2024-07-17T08:14:22.785148+0000","last_clean":"2024-07-17T08:14:22.785148+0000","last_became_active":"2024-07-17T08:14:21.791084+0000","last_became_peered":"2024-07-17T08:14:21.791084+0000","last_unstale":"2024-07-17T08:14:22.785148+0000","last_undegraded":"2024-07-17T08:14:22.785148+0000","last_fullsized":"2024-07-17T08:14:22.785148+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:07.421409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"188'3","reported_seq":17,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337617+0000","last_change":"2024-07-17T08:14:20.762621+0000","last_active":"2024-07-17T08:14:23.337617+0000","last_peered":"2024-07-17T08:14:23.337617+0000","last_clean":"2024-07-17T08:14:23.337617+0000","last_became_active":"2024-07-17T08:14:20.762159+0000","last_became_peered":"2024-07-17T08:14:20.762159+0000","last_unstale":"2024-07-17T08:14:23.337617+0000","last_undegraded":"2024-07-17T08:14:23.337617+0000","last_fullsized":"2024-07-17T08:14:23.337617+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:05.037549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.805193+0000","last_change":"2024-07-17T08:14:21.798202+0000","last_active":"2024-07-17T08:14:21.805193+0000","last_peered":"2024-07-17T08:14:21.805193+0000","last_clean":"2024-07-17T08:14:21.805193+0000","last_became_active":"2024-07-17T08:14:21.797921+0000","last_became_peered":"2024-07-17T08:14:21.797921+0000","last_unstale":"2024-07-17T08:14:21.805193+0000","last_undegraded":"2024-07-17T08:14:21.805193+0000","last_fullsized":"2024-07-17T08:14:21.805193+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:30:41.464272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"188'6","reported_seq":20,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785878+0000","last_change":"2024-07-17T08:14:20.764762+0000","last_active":"2024-07-17T08:14:22.785878+0000","last_peered":"2024-07-17T08:14:22.785878+0000","last_clean":"2024-07-17T08:14:22.785878+0000","last_became_active":"2024-07-17T08:14:20.764456+0000","last_became_peered":"2024-07-17T08:14:20.764456+0000","last_unstale":"2024-07-17T08:14:22.785878+0000","last_undegraded":"2024-07-17T08:14:22.785878+0000","last_fullsized":"2024-07-17T08:14:22.785878+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:57:30.257455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789291+0000","last_change":"2024-07-17T08:14:21.786361+0000","last_active":"2024-07-17T08:14:22.789291+0000","last_peered":"2024-07-17T08:14:22.789291+0000","last_clean":"2024-07-17T08:14:22.789291+0000","last_became_active":"2024-07-17T08:14:21.786090+0000","last_became_peered":"2024-07-17T08:14:21.786090+0000","last_unstale":"2024-07-17T08:14:22.789291+0000","last_undegraded":"2024-07-17T08:14:22.789291+0000","last_fullsized":"2024-07-17T08:14:22.789291+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:55.334083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815063+0000","last_change":"2024-07-17T08:14:20.762399+0000","last_active":"2024-07-17T08:14:21.815063+0000","last_peered":"2024-07-17T08:14:21.815063+0000","last_clean":"2024-07-17T08:14:21.815063+0000","last_became_active":"2024-07-17T08:14:20.760113+0000","last_became_peered":"2024-07-17T08:14:20.760113+0000","last_unstale":"2024-07-17T08:14:21.815063+0000","last_undegraded":"2024-07-17T08:14:21.815063+0000","last_fullsized":"2024-07-17T08:14:21.815063+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:23.477566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793233+0000","last_change":"2024-07-17T08:14:21.803922+0000","last_active":"2024-07-17T08:14:22.793233+0000","last_peered":"2024-07-17T08:14:22.793233+0000","last_clean":"2024-07-17T08:14:22.793233+0000","last_became_active":"2024-07-17T08:14:21.803327+0000","last_became_peered":"2024-07-17T08:14:21.803327+0000","last_unstale":"2024-07-17T08:14:22.793233+0000","last_undegraded":"2024-07-17T08:14:22.793233+0000","last_fullsized":"2024-07-17T08:14:22.793233+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:07.056039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785218+0000","last_change":"2024-07-17T08:14:20.759090+0000","last_active":"2024-07-17T08:14:22.785218+0000","last_peered":"2024-07-17T08:14:22.785218+0000","last_clean":"2024-07-17T08:14:22.785218+0000","last_became_active":"2024-07-17T08:14:20.757816+0000","last_became_peered":"2024-07-17T08:14:20.757816+0000","last_unstale":"2024-07-17T08:14:22.785218+0000","last_undegraded":"2024-07-17T08:14:22.785218+0000","last_fullsized":"2024-07-17T08:14:22.785218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:50.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790375+0000","last_change":"2024-07-17T08:14:21.782081+0000","last_active":"2024-07-17T08:14:22.790375+0000","last_peered":"2024-07-17T08:14:22.790375+0000","last_clean":"2024-07-17T08:14:22.790375+0000","last_became_active":"2024-07-17T08:14:21.778876+0000","last_became_peered":"2024-07-17T08:14:21.778876+0000","last_unstale":"2024-07-17T08:14:22.790375+0000","last_undegraded":"2024-07-17T08:14:22.790375+0000","last_fullsized":"2024-07-17T08:14:22.790375+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:03.177998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790181+0000","last_change":"2024-07-17T08:14:20.761932+0000","last_active":"2024-07-17T08:14:22.790181+0000","last_peered":"2024-07-17T08:14:22.790181+0000","last_clean":"2024-07-17T08:14:22.790181+0000","last_became_active":"2024-07-17T08:14:20.761629+0000","last_became_peered":"2024-07-17T08:14:20.761629+0000","last_unstale":"2024-07-17T08:14:22.790181+0000","last_undegraded":"2024-07-17T08:14:22.790181+0000","last_fullsized":"2024-07-17T08:14:22.790181+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:13.107054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786003+0000","last_change":"2024-07-17T08:14:21.780807+0000","last_active":"2024-07-17T08:14:21.786003+0000","last_peered":"2024-07-17T08:14:21.786003+0000","last_clean":"2024-07-17T08:14:21.786003+0000","last_became_active":"2024-07-17T08:14:21.780468+0000","last_became_peered":"2024-07-17T08:14:21.780468+0000","last_unstale":"2024-07-17T08:14:21.786003+0000","last_undegraded":"2024-07-17T08:14:21.786003+0000","last_fullsized":"2024-07-17T08:14:21.786003+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:28.497122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"188'8","reported_seq":22,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793255+0000","last_change":"2024-07-17T08:14:20.758057+0000","last_active":"2024-07-17T08:14:22.793255+0000","last_peered":"2024-07-17T08:14:22.793255+0000","last_clean":"2024-07-17T08:14:22.793255+0000","last_became_active":"2024-07-17T08:14:20.757228+0000","last_became_peered":"2024-07-17T08:14:20.757228+0000","last_unstale":"2024-07-17T08:14:22.793255+0000","last_undegraded":"2024-07-17T08:14:22.793255+0000","last_fullsized":"2024-07-17T08:14:22.793255+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:23.159616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792712+0000","last_change":"2024-07-17T08:14:21.791654+0000","last_active":"2024-07-17T08:14:22.792712+0000","last_peered":"2024-07-17T08:14:22.792712+0000","last_clean":"2024-07-17T08:14:22.792712+0000","last_became_active":"2024-07-17T08:14:21.791038+0000","last_became_peered":"2024-07-17T08:14:21.791038+0000","last_unstale":"2024-07-17T08:14:22.792712+0000","last_undegraded":"2024-07-17T08:14:22.792712+0000","last_fullsized":"2024-07-17T08:14:22.792712+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:14.112079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"188'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.786007+0000","last_change":"2024-07-17T08:14:20.752532+0000","last_active":"2024-07-17T08:14:22.786007+0000","last_peered":"2024-07-17T08:14:22.786007+0000","last_clean":"2024-07-17T08:14:22.786007+0000","last_became_active":"2024-07-17T08:14:20.751822+0000","last_became_peered":"2024-07-17T08:14:20.751822+0000","last_unstale":"2024-07-17T08:14:22.786007+0000","last_undegraded":"2024-07-17T08:14:22.786007+0000","last_fullsized":"2024-07-17T08:14:22.786007+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:25.335190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.788875+0000","last_change":"2024-07-17T08:14:21.783002+0000","last_active":"2024-07-17T08:14:22.788875+0000","last_peered":"2024-07-17T08:14:22.788875+0000","last_clean":"2024-07-17T08:14:22.788875+0000","last_became_active":"2024-07-17T08:14:21.782604+0000","last_became_peered":"2024-07-17T08:14:21.782604+0000","last_unstale":"2024-07-17T08:14:22.788875+0000","last_undegraded":"2024-07-17T08:14:22.788875+0000","last_fullsized":"2024-07-17T08:14:22.788875+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:26.938403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"188'2","reported_seq":16,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793659+0000","last_change":"2024-07-17T08:14:20.764814+0000","last_active":"2024-07-17T08:14:22.793659+0000","last_peered":"2024-07-17T08:14:22.793659+0000","last_clean":"2024-07-17T08:14:22.793659+0000","last_became_active":"2024-07-17T08:14:20.764505+0000","last_became_peered":"2024-07-17T08:14:20.764505+0000","last_unstale":"2024-07-17T08:14:22.793659+0000","last_undegraded":"2024-07-17T08:14:22.793659+0000","last_fullsized":"2024-07-17T08:14:22.793659+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:28:07.653793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790543+0000","last_change":"2024-07-17T08:14:21.787713+0000","last_active":"2024-07-17T08:14:22.790543+0000","last_peered":"2024-07-17T08:14:22.790543+0000","last_clean":"2024-07-17T08:14:22.790543+0000","last_became_active":"2024-07-17T08:14:21.787378+0000","last_became_peered":"2024-07-17T08:14:21.787378+0000","last_unstale":"2024-07-17T08:14:22.790543+0000","last_undegraded":"2024-07-17T08:14:22.790543+0000","last_fullsized":"2024-07-17T08:14:22.790543+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:33:34.157844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"188'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812603+0000","last_change":"2024-07-17T08:14:20.765656+0000","last_active":"2024-07-17T08:14:21.812603+0000","last_peered":"2024-07-17T08:14:21.812603+0000","last_clean":"2024-07-17T08:14:21.812603+0000","last_became_active":"2024-07-17T08:14:20.760900+0000","last_became_peered":"2024-07-17T08:14:20.760900+0000","last_unstale":"2024-07-17T08:14:21.812603+0000","last_undegraded":"2024-07-17T08:14:21.812603+0000","last_fullsized":"2024-07-17T08:14:21.812603+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:56.988606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.789894+0000","last_change":"2024-07-17T08:14:21.801957+0000","last_active":"2024-07-17T08:14:22.789894+0000","last_peered":"2024-07-17T08:14:22.789894+0000","last_clean":"2024-07-17T08:14:22.789894+0000","last_became_active":"2024-07-17T08:14:21.801749+0000","last_became_peered":"2024-07-17T08:14:21.801749+0000","last_unstale":"2024-07-17T08:14:22.789894+0000","last_undegraded":"2024-07-17T08:14:22.789894+0000","last_fullsized":"2024-07-17T08:14:22.789894+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:20.158513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793521+0000","last_change":"2024-07-17T08:14:20.765387+0000","last_active":"2024-07-17T08:14:22.793521+0000","last_peered":"2024-07-17T08:14:22.793521+0000","last_clean":"2024-07-17T08:14:22.793521+0000","last_became_active":"2024-07-17T08:14:20.765076+0000","last_became_peered":"2024-07-17T08:14:20.765076+0000","last_unstale":"2024-07-17T08:14:22.793521+0000","last_undegraded":"2024-07-17T08:14:22.793521+0000","last_fullsized":"2024-07-17T08:14:22.793521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:48.749900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785732+0000","last_change":"2024-07-17T08:14:21.787981+0000","last_active":"2024-07-17T08:14:22.785732+0000","last_peered":"2024-07-17T08:14:22.785732+0000","last_clean":"2024-07-17T08:14:22.785732+0000","last_became_active":"2024-07-17T08:14:21.785399+0000","last_became_peered":"2024-07-17T08:14:21.785399+0000","last_unstale":"2024-07-17T08:14:22.785732+0000","last_undegraded":"2024-07-17T08:14:22.785732+0000","last_fullsized":"2024-07-17T08:14:22.785732+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:36:27.713497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.815560+0000","last_change":"2024-07-17T08:14:20.762072+0000","last_active":"2024-07-17T08:14:21.815560+0000","last_peered":"2024-07-17T08:14:21.815560+0000","last_clean":"2024-07-17T08:14:21.815560+0000","last_became_active":"2024-07-17T08:14:20.759687+0000","last_became_peered":"2024-07-17T08:14:20.759687+0000","last_unstale":"2024-07-17T08:14:21.815560+0000","last_undegraded":"2024-07-17T08:14:21.815560+0000","last_fullsized":"2024-07-17T08:14:21.815560+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:12.585687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.800753+0000","last_change":"2024-07-17T08:14:21.789710+0000","last_active":"2024-07-17T08:14:21.800753+0000","last_peered":"2024-07-17T08:14:21.800753+0000","last_clean":"2024-07-17T08:14:21.800753+0000","last_became_active":"2024-07-17T08:14:21.789422+0000","last_became_peered":"2024-07-17T08:14:21.789422+0000","last_unstale":"2024-07-17T08:14:21.800753+0000","last_undegraded":"2024-07-17T08:14:21.800753+0000","last_fullsized":"2024-07-17T08:14:21.800753+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:03.803183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790655+0000","last_change":"2024-07-17T08:14:20.763024+0000","last_active":"2024-07-17T08:14:22.790655+0000","last_peered":"2024-07-17T08:14:22.790655+0000","last_clean":"2024-07-17T08:14:22.790655+0000","last_became_active":"2024-07-17T08:14:20.761934+0000","last_became_peered":"2024-07-17T08:14:20.761934+0000","last_unstale":"2024-07-17T08:14:22.790655+0000","last_undegraded":"2024-07-17T08:14:22.790655+0000","last_fullsized":"2024-07-17T08:14:22.790655+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:42.885674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.805507+0000","last_change":"2024-07-17T08:14:21.803712+0000","last_active":"2024-07-17T08:14:21.805507+0000","last_peered":"2024-07-17T08:14:21.805507+0000","last_clean":"2024-07-17T08:14:21.805507+0000","last_became_active":"2024-07-17T08:14:21.803434+0000","last_became_peered":"2024-07-17T08:14:21.803434+0000","last_unstale":"2024-07-17T08:14:21.805507+0000","last_undegraded":"2024-07-17T08:14:21.805507+0000","last_fullsized":"2024-07-17T08:14:21.805507+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:11.432781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.767951+0000","last_change":"2024-07-17T08:14:20.765798+0000","last_active":"2024-07-17T08:14:21.767951+0000","last_peered":"2024-07-17T08:14:21.767951+0000","last_clean":"2024-07-17T08:14:21.767951+0000","last_became_active":"2024-07-17T08:14:20.765499+0000","last_became_peered":"2024-07-17T08:14:20.765499+0000","last_unstale":"2024-07-17T08:14:21.767951+0000","last_undegraded":"2024-07-17T08:14:21.767951+0000","last_fullsized":"2024-07-17T08:14:21.767951+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:38:09.285542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796105+0000","last_change":"2024-07-17T08:14:21.793958+0000","last_active":"2024-07-17T08:14:21.796105+0000","last_peered":"2024-07-17T08:14:21.796105+0000","last_clean":"2024-07-17T08:14:21.796105+0000","last_became_active":"2024-07-17T08:14:21.792126+0000","last_became_peered":"2024-07-17T08:14:21.792126+0000","last_unstale":"2024-07-17T08:14:21.796105+0000","last_undegraded":"2024-07-17T08:14:21.796105+0000","last_fullsized":"2024-07-17T08:14:21.796105+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:35.916073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.775734+0000","last_change":"2024-07-17T08:14:20.760401+0000","last_active":"2024-07-17T08:14:21.775734+0000","last_peered":"2024-07-17T08:14:21.775734+0000","last_clean":"2024-07-17T08:14:21.775734+0000","last_became_active":"2024-07-17T08:14:20.760092+0000","last_became_peered":"2024-07-17T08:14:20.760092+0000","last_unstale":"2024-07-17T08:14:21.775734+0000","last_undegraded":"2024-07-17T08:14:21.775734+0000","last_fullsized":"2024-07-17T08:14:21.775734+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:57.172385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.787972+0000","last_change":"2024-07-17T08:14:21.788380+0000","last_active":"2024-07-17T08:14:22.787972+0000","last_peered":"2024-07-17T08:14:22.787972+0000","last_clean":"2024-07-17T08:14:22.787972+0000","last_became_active":"2024-07-17T08:14:21.787728+0000","last_became_peered":"2024-07-17T08:14:21.787728+0000","last_unstale":"2024-07-17T08:14:22.787972+0000","last_undegraded":"2024-07-17T08:14:22.787972+0000","last_fullsized":"2024-07-17T08:14:22.787972+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:44:30.506222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"188'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.812792+0000","last_change":"2024-07-17T08:14:20.764995+0000","last_active":"2024-07-17T08:14:21.812792+0000","last_peered":"2024-07-17T08:14:21.812792+0000","last_clean":"2024-07-17T08:14:21.812792+0000","last_became_active":"2024-07-17T08:14:20.761411+0000","last_became_peered":"2024-07-17T08:14:20.761411+0000","last_unstale":"2024-07-17T08:14:21.812792+0000","last_undegraded":"2024-07-17T08:14:21.812792+0000","last_fullsized":"2024-07-17T08:14:21.812792+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:08.640294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.793028+0000","last_change":"2024-07-17T08:14:21.793248+0000","last_active":"2024-07-17T08:14:22.793028+0000","last_peered":"2024-07-17T08:14:22.793028+0000","last_clean":"2024-07-17T08:14:22.793028+0000","last_became_active":"2024-07-17T08:14:21.792508+0000","last_became_peered":"2024-07-17T08:14:21.792508+0000","last_unstale":"2024-07-17T08:14:22.793028+0000","last_undegraded":"2024-07-17T08:14:22.793028+0000","last_fullsized":"2024-07-17T08:14:22.793028+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:27.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"189'1","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.791932+0000","last_change":"2024-07-17T08:14:20.767034+0000","last_active":"2024-07-17T08:14:22.791932+0000","last_peered":"2024-07-17T08:14:22.791932+0000","last_clean":"2024-07-17T08:14:22.791932+0000","last_became_active":"2024-07-17T08:14:20.766400+0000","last_became_peered":"2024-07-17T08:14:20.766400+0000","last_unstale":"2024-07-17T08:14:22.791932+0000","last_undegraded":"2024-07-17T08:14:22.791932+0000","last_fullsized":"2024-07-17T08:14:22.791932+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:23.477566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.791887+0000","last_change":"2024-07-17T08:14:21.783480+0000","last_active":"2024-07-17T08:14:21.791887+0000","last_peered":"2024-07-17T08:14:21.791887+0000","last_clean":"2024-07-17T08:14:21.791887+0000","last_became_active":"2024-07-17T08:14:21.783156+0000","last_became_peered":"2024-07-17T08:14:21.783156+0000","last_unstale":"2024-07-17T08:14:21.791887+0000","last_undegraded":"2024-07-17T08:14:21.791887+0000","last_fullsized":"2024-07-17T08:14:21.791887+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:34.370323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785622+0000","last_change":"2024-07-17T08:14:20.756607+0000","last_active":"2024-07-17T08:14:22.785622+0000","last_peered":"2024-07-17T08:14:22.785622+0000","last_clean":"2024-07-17T08:14:22.785622+0000","last_became_active":"2024-07-17T08:14:20.755139+0000","last_became_peered":"2024-07-17T08:14:20.755139+0000","last_unstale":"2024-07-17T08:14:22.785622+0000","last_undegraded":"2024-07-17T08:14:22.785622+0000","last_fullsized":"2024-07-17T08:14:22.785622+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:59.795046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.792994+0000","last_change":"2024-07-17T08:14:21.799265+0000","last_active":"2024-07-17T08:14:22.792994+0000","last_peered":"2024-07-17T08:14:22.792994+0000","last_clean":"2024-07-17T08:14:22.792994+0000","last_became_active":"2024-07-17T08:14:21.798771+0000","last_became_peered":"2024-07-17T08:14:21.798771+0000","last_unstale":"2024-07-17T08:14:22.792994+0000","last_undegraded":"2024-07-17T08:14:22.792994+0000","last_fullsized":"2024-07-17T08:14:22.792994+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:53.321905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.765672+0000","last_change":"2024-07-17T08:14:20.767416+0000","last_active":"2024-07-17T08:14:21.765672+0000","last_peered":"2024-07-17T08:14:21.765672+0000","last_clean":"2024-07-17T08:14:21.765672+0000","last_became_active":"2024-07-17T08:14:20.767120+0000","last_became_peered":"2024-07-17T08:14:20.767120+0000","last_unstale":"2024-07-17T08:14:21.765672+0000","last_undegraded":"2024-07-17T08:14:21.765672+0000","last_fullsized":"2024-07-17T08:14:21.765672+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:53:30.693634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337409+0000","last_change":"2024-07-17T08:14:21.778545+0000","last_active":"2024-07-17T08:14:23.337409+0000","last_peered":"2024-07-17T08:14:23.337409+0000","last_clean":"2024-07-17T08:14:23.337409+0000","last_became_active":"2024-07-17T08:14:21.778158+0000","last_became_peered":"2024-07-17T08:14:21.778158+0000","last_unstale":"2024-07-17T08:14:23.337409+0000","last_undegraded":"2024-07-17T08:14:23.337409+0000","last_fullsized":"2024-07-17T08:14:23.337409+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:28.529855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:23.337264+0000","last_change":"2024-07-17T08:14:20.764148+0000","last_active":"2024-07-17T08:14:23.337264+0000","last_peered":"2024-07-17T08:14:23.337264+0000","last_clean":"2024-07-17T08:14:23.337264+0000","last_became_active":"2024-07-17T08:14:20.763764+0000","last_became_peered":"2024-07-17T08:14:20.763764+0000","last_unstale":"2024-07-17T08:14:23.337264+0000","last_undegraded":"2024-07-17T08:14:23.337264+0000","last_fullsized":"2024-07-17T08:14:23.337264+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:55.832810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.796568+0000","last_change":"2024-07-17T08:14:21.795571+0000","last_active":"2024-07-17T08:14:21.796568+0000","last_peered":"2024-07-17T08:14:21.796568+0000","last_clean":"2024-07-17T08:14:21.796568+0000","last_became_active":"2024-07-17T08:14:21.794561+0000","last_became_peered":"2024-07-17T08:14:21.794561+0000","last_unstale":"2024-07-17T08:14:21.796568+0000","last_undegraded":"2024-07-17T08:14:21.796568+0000","last_fullsized":"2024-07-17T08:14:21.796568+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:26.714903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785289+0000","last_change":"2024-07-17T08:14:20.753576+0000","last_active":"2024-07-17T08:14:22.785289+0000","last_peered":"2024-07-17T08:14:22.785289+0000","last_clean":"2024-07-17T08:14:22.785289+0000","last_became_active":"2024-07-17T08:14:20.753229+0000","last_became_peered":"2024-07-17T08:14:20.753229+0000","last_unstale":"2024-07-17T08:14:22.785289+0000","last_undegraded":"2024-07-17T08:14:22.785289+0000","last_fullsized":"2024-07-17T08:14:22.785289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:52:12.585687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785986+0000","last_change":"2024-07-17T08:14:21.791663+0000","last_active":"2024-07-17T08:14:22.785986+0000","last_peered":"2024-07-17T08:14:22.785986+0000","last_clean":"2024-07-17T08:14:22.785986+0000","last_became_active":"2024-07-17T08:14:21.791154+0000","last_became_peered":"2024-07-17T08:14:21.791154+0000","last_unstale":"2024-07-17T08:14:22.785986+0000","last_undegraded":"2024-07-17T08:14:22.785986+0000","last_fullsized":"2024-07-17T08:14:22.785986+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:47.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.790432+0000","last_change":"2024-07-17T08:14:20.755453+0000","last_active":"2024-07-17T08:14:22.790432+0000","last_peered":"2024-07-17T08:14:22.790432+0000","last_clean":"2024-07-17T08:14:22.790432+0000","last_became_active":"2024-07-17T08:14:20.755119+0000","last_became_peered":"2024-07-17T08:14:20.755119+0000","last_unstale":"2024-07-17T08:14:22.790432+0000","last_undegraded":"2024-07-17T08:14:22.790432+0000","last_fullsized":"2024-07-17T08:14:22.790432+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:07.731068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.786226+0000","last_change":"2024-07-17T08:14:21.781679+0000","last_active":"2024-07-17T08:14:21.786226+0000","last_peered":"2024-07-17T08:14:21.786226+0000","last_clean":"2024-07-17T08:14:21.786226+0000","last_became_active":"2024-07-17T08:14:21.781364+0000","last_became_peered":"2024-07-17T08:14:21.781364+0000","last_unstale":"2024-07-17T08:14:21.786226+0000","last_undegraded":"2024-07-17T08:14:21.786226+0000","last_fullsized":"2024-07-17T08:14:21.786226+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:11:30.747138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.777274+0000","last_change":"2024-07-17T08:14:20.754748+0000","last_active":"2024-07-17T08:14:21.777274+0000","last_peered":"2024-07-17T08:14:21.777274+0000","last_clean":"2024-07-17T08:14:21.777274+0000","last_became_active":"2024-07-17T08:14:20.754475+0000","last_became_peered":"2024-07-17T08:14:20.754475+0000","last_unstale":"2024-07-17T08:14:21.777274+0000","last_undegraded":"2024-07-17T08:14:21.777274+0000","last_fullsized":"2024-07-17T08:14:21.777274+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:59.795046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.795778+0000","last_change":"2024-07-17T08:14:21.789040+0000","last_active":"2024-07-17T08:14:21.795778+0000","last_peered":"2024-07-17T08:14:21.795778+0000","last_clean":"2024-07-17T08:14:21.795778+0000","last_became_active":"2024-07-17T08:14:21.785072+0000","last_became_peered":"2024-07-17T08:14:21.785072+0000","last_unstale":"2024-07-17T08:14:21.795778+0000","last_undegraded":"2024-07-17T08:14:21.795778+0000","last_fullsized":"2024-07-17T08:14:21.795778+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:20.718693+0000","last_clean_scrub_stamp":"2024-07-17T08:14:20.718693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:47.533791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-07-17T08:14:22.785750+0000","last_change":"2024-07-17T08:14:20.767261+0000","last_active":"2024-07-17T08:14:22.785750+0000","last_peered":"2024-07-17T08:14:22.785750+0000","last_clean":"2024-07-17T08:14:22.785750+0000","last_became_active":"2024-07-17T08:14:20.766770+0000","last_became_peered":"2024-07-17T08:14:20.766770+0000","last_unstale":"2024-07-17T08:14:22.785750+0000","last_undegraded":"2024-07-17T08:14:22.785750+0000","last_fullsized":"2024-07-17T08:14:22.785750+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:14:19.713668+0000","last_clean_scrub_stamp":"2024-07-17T08:14:19.713668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:02.798158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"180'2476","reported_seq":3180,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-07-17T08:14:21.775793+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:14:21.775793+0000","last_peered":"2024-07-17T08:14:21.775793+0000","last_clean":"2024-07-17T08:14:21.775793+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:14:21.775793+0000","last_undegraded":"2024-07-17T08:14:21.775793+0000","last_fullsized":"2024-07-17T08:14:21.775793+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":299,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1087,"num_read_kb":2991,"num_write":299,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2476,"ondisk_log_size":2476,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543092,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60420,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311420,"statfs":{"total":96636764160,"available":96574894080,"internally_reserved":0,"allocated":2940928,"data_stored":1854677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510387,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":2359296,"data_stored":1271549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510388,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47384,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324456,"statfs":{"total":96636764160,"available":96588242944,"internally_reserved":0,"allocated":2367488,"data_stored":1271673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543093,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41820,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330020,"statfs":{"total":96636764160,"available":96593940480,"internally_reserved":0,"allocated":2375680,"data_stored":1272313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510387,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64036,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307804,"statfs":{"total":96636764160,"available":96571191296,"internally_reserved":0,"allocated":2379776,"data_stored":1273765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543091,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47364,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324476,"statfs":{"total":96636764160,"available":96588263424,"internally_reserved":0,"allocated":2347008,"data_stored":1264309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41824,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330016,"statfs":{"total":96636764160,"available":96593936384,"internally_reserved":0,"allocated":2371584,"data_stored":1273161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510387,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34068,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337772,"statfs":{"total":96636764160,"available":96601878528,"internally_reserved":0,"allocated":2949120,"data_stored":1854781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:14:26.828 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:14:26.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:14:27.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 -- 172.21.15.44:0/405567128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400b33f0 msgr2=0x7ff94006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 --2- 172.21.15.44:0/405567128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400b33f0 0x7ff94006d420 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff940070a70 tx=0x7ff93802aa10 comp rx=0 tx=0).stop 2024-07-17T08:14:27.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 -- 172.21.15.44:0/405567128 shutdown_connections 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 --2- 172.21.15.44:0/405567128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400b33f0 0x7ff94006d420 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 --2- 172.21.15.44:0/405567128 >> v2:172.21.15.175:3300/0 conn(0x7ff9400ae340 0x7ff9400b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 --2- 172.21.15.44:0/405567128 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 0x7ff94015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 -- 172.21.15.44:0/405567128 >> 172.21.15.44:0/405567128 conn(0x7ff9400cf0d0 msgr2=0x7ff9400cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 -- 172.21.15.44:0/405567128 shutdown_connections 2024-07-17T08:14:27.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.093+0000 7ff944c00640 1 -- 172.21.15.44:0/405567128 wait complete. 2024-07-17T08:14:27.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.095+0000 7ff944c00640 1 Processor -- start 2024-07-17T08:14:27.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.095+0000 7ff944c00640 1 -- start start 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9400b33f0 0x7ff9401583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 0x7ff940158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff94006dae0 con 0x7ff9400ae340 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff94006d960 con 0x7ff9400b33f0 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff944c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff94006dc60 con 0x7ff9401503c0 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff93f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff93f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45032/0 (socket says 172.21.15.44:45032) 2024-07-17T08:14:27.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.096+0000 7ff93f400640 1 -- 172.21.15.44:0/1645173648 learned_addr learned my addr 172.21.15.44:0/1645173648 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93fe00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 0x7ff940158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff937e00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9400b33f0 0x7ff9401583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 -- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 msgr2=0x7ff940158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 0x7ff940158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 -- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9400b33f0 msgr2=0x7ff9401583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9400b33f0 0x7ff9401583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 -- 172.21.15.44:0/1645173648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff940152d70 con 0x7ff9400ae340 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff93f400640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b440 tx=0x7ff93000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.097+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930015020 con 0x7ff9400ae340 2024-07-17T08:14:27.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.098+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff930011980 con 0x7ff9400ae340 2024-07-17T08:14:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.098+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930011dd0 con 0x7ff9400ae340 2024-07-17T08:14:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.098+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff940153050 con 0x7ff9400ae340 2024-07-17T08:14:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.098+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff940146770 con 0x7ff9400ae340 2024-07-17T08:14:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.099+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9401596e0 con 0x7ff9400ae340 2024-07-17T08:14:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.100+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7ff930018020 con 0x7ff9400ae340 2024-07-17T08:14:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.102+0000 7ff937400640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 0x7ff91c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.102+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7ff9300988a0 con 0x7ff9400ae340 2024-07-17T08:14:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.102+0000 7ff937e00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 0x7ff91c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.104+0000 7ff937e00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 0x7ff91c079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff928005fd0 tx=0x7ff928005ce0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:27.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.106+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff93001f020 con 0x7ff9400ae340 2024-07-17T08:14:27.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.396+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff9400cffe0 con 0x7ff9400ae340 2024-07-17T08:14:27.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.402+0000 7ff937400640 1 -- 172.21.15.44:0/1645173648 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v62) ==== 190+0+147 (secure 0 0 0) 0x7ff93000d040 con 0x7ff9400ae340 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:14:27.404 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 msgr2=0x7ff91c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 0x7ff91c079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff928005fd0 tx=0x7ff928005ce0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 msgr2=0x7ff9400989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b440 tx=0x7ff93000b900 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 shutdown_connections 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff91c076bb0 0x7ff91c079070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9401503c0 0x7ff940158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9400b33f0 0x7ff9401583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 --2- 172.21.15.44:0/1645173648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9400ae340 0x7ff9400989e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 >> 172.21.15.44:0/1645173648 conn(0x7ff9400cf0d0 msgr2=0x7ff94006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:27.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 shutdown_connections 2024-07-17T08:14:27.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.406+0000 7ff944c00640 1 -- 172.21.15.44:0/1645173648 wait complete. 2024-07-17T08:14:27.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:14:27.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 -- 172.21.15.44:0/3771636846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b5400 msgr2=0x7fa8dc15e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/3771636846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b5400 0x7fa8dc15e2d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009860 tx=0x7fa8d402a540 comp rx=0 tx=0).stop 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 -- 172.21.15.44:0/3771636846 shutdown_connections 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/3771636846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b5400 0x7fa8dc15e2d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/3771636846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b8c80 0x7fa8dc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.638+0000 7fa8dbe00640 1 -- 172.21.15.44:0/3771636846 >> 172.21.15.44:0/3771636846 conn(0x7fa8dc0cfec0 msgr2=0x7fa8dc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.639+0000 7fa8dbe00640 1 -- 172.21.15.44:0/3771636846 shutdown_connections 2024-07-17T08:14:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.639+0000 7fa8dbe00640 1 -- 172.21.15.44:0/3771636846 wait complete. 2024-07-17T08:14:27.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.641+0000 7fa8dbe00640 1 Processor -- start 2024-07-17T08:14:27.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.641+0000 7fa8dbe00640 1 -- start start 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.642+0000 7fa8dbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b5400 0x7fa8dc1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b8c80 0x7fa8dc0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8dbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8dbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc06e0c0 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8dbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc0b92f0 con 0x7fa8dc0b8c80 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8dbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8dc06e240 con 0x7fa8dc0b5400 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45034/0 (socket says 172.21.15.44:45034) 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8db400640 1 -- 172.21.15.44:0/2008437665 learned_addr learned my addr 172.21.15.44:0/2008437665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8daa00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b5400 0x7fa8dc1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.643+0000 7fa8da000640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b8c80 0x7fa8dc0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 -- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b5400 msgr2=0x7fa8dc1f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b5400 0x7fa8dc1f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 -- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b8c80 msgr2=0x7fa8dc0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b8c80 0x7fa8dc0b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 -- 172.21.15.44:0/2008437665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc14ea70 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.644+0000 7fa8db400640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402ab30 tx=0x7fa8d4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.645+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4040070 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.645+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8d4005310 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.645+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005c90 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.645+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc14ed50 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.645+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc14f310 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.646+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc06e6f0 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.647+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fa8d4039040 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.648+0000 7fa8d3400640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 0x7fa8b8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.648+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7fa8d40b9830 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.649+0000 7fa8daa00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 0x7fa8b8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.650+0000 7fa8daa00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 0x7fa8b8079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc005fd0 tx=0x7fa8cc009500 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.653+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa8d4082270 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.904+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8dc0cb560 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.905+0000 7fa8d3400640 1 -- 172.21.15.44:0/2008437665 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7fa8d40867c0 con 0x7fa8dc0ce5c0 2024-07-17T08:14:27.907 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:14:27.908 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:14:27.908 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:14:27.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 msgr2=0x7fa8b8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 0x7fa8b8079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc005fd0 tx=0x7fa8cc009500 comp rx=0 tx=0).stop 2024-07-17T08:14:27.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 msgr2=0x7fa8dc0ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:27.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402ab30 tx=0x7fa8d4032c10 comp rx=0 tx=0).stop 2024-07-17T08:14:27.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 shutdown_connections 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fa8b8076bb0 0x7fa8b8079070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc0ce5c0 0x7fa8dc0ac800 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8dc0b8c80 0x7fa8dc0b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 --2- 172.21.15.44:0/2008437665 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8dc0b5400 0x7fa8dc1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.908+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 >> 172.21.15.44:0/2008437665 conn(0x7fa8dc0cfec0 msgr2=0x7fa8dc0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.909+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 shutdown_connections 2024-07-17T08:14:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:27.909+0000 7fa8dbe00640 1 -- 172.21.15.44:0/2008437665 wait complete. 2024-07-17T08:14:28.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:14:21.223311+0000', 'modified': '2024-07-17T08:14:24.308288+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9183}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9183': {'gid': 9183, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/1832122622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 1832122622}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1832122622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9183, 'qdb_cluster': [9183]} 2024-07-17T08:14:28.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:14:28.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:14:28.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.550+0000 7fadae800640 1 -- 172.21.15.44:0/1579787858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 msgr2=0x7fad90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 --2- 172.21.15.44:0/1579787858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fad90004c80 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fad9400c2e0 tx=0x7fad90005830 comp rx=0 tx=0).stop 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 -- 172.21.15.44:0/1579787858 shutdown_connections 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 --2- 172.21.15.44:0/1579787858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fad90004c80 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 -- 172.21.15.44:0/1579787858 >> 172.21.15.44:0/1579787858 conn(0x7fada80cf0d0 msgr2=0x7fada80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 -- 172.21.15.44:0/1579787858 shutdown_connections 2024-07-17T08:14:28.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.551+0000 7fadae800640 1 -- 172.21.15.44:0/1579787858 wait complete. 2024-07-17T08:14:28.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.553+0000 7fadae800640 1 Processor -- start 2024-07-17T08:14:28.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.553+0000 7fadae800640 1 -- start start 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada806c570 0x7fada81f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada8073ca0 0x7fada8074130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fada806d9d0 con 0x7fad90004800 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fada806d850 con 0x7fada8073ca0 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fada806db50 con 0x7fada806c570 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45038/0 (socket says 172.21.15.44:45038) 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 -- 172.21.15.44:0/1381784410 learned_addr learned my addr 172.21.15.44:0/1381784410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 -- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada806c570 msgr2=0x7fada81f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada806c570 0x7fada81f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 -- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada8073ca0 msgr2=0x7fada8074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada8073ca0 0x7fada8074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.554+0000 7fadad400640 1 -- 172.21.15.44:0/1381784410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada8074670 con 0x7fad90004800 2024-07-17T08:14:28.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fadad400640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00e690 tx=0x7fad9c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fada5600640 1 -- 172.21.15.44:0/1381784410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00ca50 con 0x7fad90004800 2024-07-17T08:14:28.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fada5600640 1 -- 172.21.15.44:0/1381784410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c00cce0 con 0x7fad90004800 2024-07-17T08:14:28.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fada5600640 1 -- 172.21.15.44:0/1381784410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0116d0 con 0x7fad90004800 2024-07-17T08:14:28.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada8146420 con 0x7fad90004800 2024-07-17T08:14:28.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.555+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada81469e0 con 0x7fad90004800 2024-07-17T08:14:28.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.558+0000 7fada5600640 1 -- 172.21.15.44:0/1381784410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7fad9c011870 con 0x7fad90004800 2024-07-17T08:14:28.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.559+0000 7fada5600640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 0x7fad88079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.559+0000 7fada5600640 1 -- 172.21.15.44:0/1381784410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7fad9c099050 con 0x7fad90004800 2024-07-17T08:14:28.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.559+0000 7fadb2d7e740 1 Processor -- start 2024-07-17T08:14:28.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.559+0000 7fadaca00640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 0x7fad88079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.559+0000 7fadb2d7e740 1 -- start start 2024-07-17T08:14:28.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff20f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 0x55e1ff20b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e1ff156d60 con 0x55e1ff20b790 2024-07-17T08:14:28.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e1ff146260 con 0x55e1ff1e6280 2024-07-17T08:14:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.560+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e1ff14a2a0 con 0x55e1ff20efc0 2024-07-17T08:14:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadaca00640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 0x7fad88079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad94033000 tx=0x7fada80b3f10 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45040/0 (socket says 172.21.15.44:45040) 2024-07-17T08:14:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 -- 172.21.15.44:0/4126411649 learned_addr learned my addr 172.21.15.44:0/4126411649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadad400640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff20f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 -- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 msgr2=0x55e1ff20f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff20f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 -- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 msgr2=0x55e1ff20b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadaca00640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 0x55e1ff20b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 0x55e1ff20b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 -- 172.21.15.44:0/4126411649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1feeaa850 con 0x55e1ff20b790 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadaca00640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 0x55e1ff20b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.561+0000 7fadade00640 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fada81507a0 tx=0x7fada0002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c669ca907d3fdb48 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fad99c00640 1 -- 172.21.15.44:0/4126411649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0017070 con 0x55e1ff20b790 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fad99c00640 1 -- 172.21.15.44:0/4126411649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada0003ca0 con 0x55e1ff20b790 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fad99c00640 1 -- 172.21.15.44:0/4126411649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada000cbc0 con 0x55e1ff20b790 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 -- 172.21.15.44:0/4126411649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 msgr2=0x55e1ff20bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fada81507a0 tx=0x7fada0002020 comp rx=0 tx=0).stop 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 -- 172.21.15.44:0/4126411649 shutdown_connections 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff20bc40 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff1e6280 0x55e1ff20b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/4126411649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff20f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.562+0000 7fadb2d7e740 1 -- 172.21.15.44:0/4126411649 >> 172.21.15.44:0/4126411649 conn(0x55e1ff1e1460 msgr2=0x55e1ff1e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.563+0000 7fadb2d7e740 1 -- 172.21.15.44:0/4126411649 shutdown_connections 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.563+0000 7fadb2d7e740 1 -- 172.21.15.44:0/4126411649 wait complete. 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.563+0000 7fadb2d7e740 1 -- start start 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.563+0000 7fadb2d7e740 1 Processor -- start 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.564+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20efc0 0x55e1ff209c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff207680 0x55e1ff20a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e1feeb9620 con 0x55e1ff20b790 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e1ff0edf90 con 0x55e1ff20efc0 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e1ff1dfb60 con 0x55e1ff207680 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadb2d7e740 10 client.0 _ll_register_callbacks cb 0x55e1ff113f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:14:28.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45054/0 (socket says 172.21.15.44:45054) 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 -- 172.21.15.44:0/516093619 learned_addr learned my addr 172.21.15.44:0/516093619 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 -- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff207680 msgr2=0x55e1ff20a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff207680 0x55e1ff20a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 -- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20efc0 msgr2=0x55e1ff209c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20efc0 0x55e1ff209c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.565+0000 7fadad400640 1 -- 172.21.15.44:0/516093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1fef216b0 con 0x55e1ff20b790 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadad400640 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fad9c061770 tx=0x7fad9c062090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6868d183d4e338 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c01f040 con 0x55e1ff20b790 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7fad9c060cd0 con 0x55e1ff20b790 2024-07-17T08:14:28.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1ff11d900 con 0x55e1ff20b790 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fad9c062170 con 0x55e1ff20b790 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 10 client.9306 fetch_fsmap learned FSMap version 97 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x55e1ff225300 con 0x55e1ff20b790 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e1ff11d900 con 0x55e1ff20b790 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fad9c00f040 con 0x55e1ff20b790 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 10 client.9306 fetch_fsmap finished waiting for FSMap version 97 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 10 client.9306 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 20 client.9306 populate_metadata read hostname 'smithi044' 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff205e00 0x55e1ff2061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 4 client.9306 mds_command: new command op to 9183 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.566+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 --> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1ff158cb0 con 0x55e1ff205e00 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.567+0000 7fadaca00640 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff205e00 0x55e1ff2061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.567+0000 7fadaca00640 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff205e00 0x55e1ff2061e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da623811633f530 server_cookie=323485cae3a6447a in_seq=0 out_seq=0 2024-07-17T08:14:28.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.567+0000 7fad8e000640 10 client.9306 ms_handle_connect on v2:172.21.15.44:6833/1832122622 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.568+0000 7fad8e000640 1 -- 172.21.15.44:0/516093619 <== mds.0 v2:172.21.15.44:6833/1832122622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e1ff158cb0 con 0x55e1ff205e00 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.568+0000 7fad8e000640 10 client.9306 handle_command_reply: tid=0 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.568+0000 7fadb2d7e740 1 client.9306 shutdown 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.568+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 msgr2=0x55e1ff2033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.568+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fad9c061770 tx=0x7fad9c062090 comp rx=0 tx=0).stop 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 shutdown_connections 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff205e00 0x55e1ff2061e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff207680 0x55e1ff20a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20efc0 0x55e1ff209c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/516093619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff20b790 0x55e1ff2033d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fad8e000640 0 client.9306 ms_handle_reset on v2:172.21.15.44:6833/1832122622 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 >> 172.21.15.44:0/516093619 conn(0x55e1ff1e6280 msgr2=0x55e1ff1dff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 shutdown_connections 2024-07-17T08:14:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 1 -- 172.21.15.44:0/516093619 wait complete. 2024-07-17T08:14:28.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.569+0000 7fadb2d7e740 20 client.9306 trim_cache size 0 max 16384 2024-07-17T08:14:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 Processor -- start 2024-07-17T08:14:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 -- start start 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff209c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff203ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fada8200b50 con 0x55e1ff207680 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fada80ce370 con 0x55e1ff20b790 2024-07-17T08:14:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.590+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fada8200500 con 0x55e1ff20efc0 2024-07-17T08:14:28.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45070/0 (socket says 172.21.15.44:45070) 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 -- 172.21.15.44:0/3674865292 learned_addr learned my addr 172.21.15.44:0/3674865292 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 -- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 msgr2=0x55e1ff203ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff203ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 -- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 msgr2=0x55e1ff209c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff209c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.591+0000 7fadad400640 1 -- 172.21.15.44:0/3674865292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ff23cec0 con 0x55e1ff207680 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadad400640 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fad9c023040 tx=0x7fad9c0613e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b03e0a08b11cfba server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fad99200640 1 -- 172.21.15.44:0/3674865292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c01f040 con 0x55e1ff207680 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fad99200640 1 -- 172.21.15.44:0/3674865292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c07a070 con 0x55e1ff207680 2024-07-17T08:14:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fad99200640 1 -- 172.21.15.44:0/3674865292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c066070 con 0x55e1ff207680 2024-07-17T08:14:28.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 -- 172.21.15.44:0/3674865292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 msgr2=0x55e1ff20de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fad9c023040 tx=0x7fad9c0613e0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 -- 172.21.15.44:0/3674865292 shutdown_connections 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff203ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff209c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/3674865292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff20de30 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.592+0000 7fadb2d7e740 1 -- 172.21.15.44:0/3674865292 >> 172.21.15.44:0/3674865292 conn(0x55e1ff1e6280 msgr2=0x55e1ff1df940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.593+0000 7fadb2d7e740 1 -- 172.21.15.44:0/3674865292 shutdown_connections 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.593+0000 7fadb2d7e740 1 -- 172.21.15.44:0/3674865292 wait complete. 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.593+0000 7fadb2d7e740 1 -- start start 2024-07-17T08:14:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.594+0000 7fadb2d7e740 1 Processor -- start 2024-07-17T08:14:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.594+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff2a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff2abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e1ff206cf0 con 0x55e1ff207680 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e1ff223dc0 con 0x55e1ff20b790 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e1ff224b20 con 0x55e1ff20efc0 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadb2d7e740 10 client.0 _ll_register_callbacks cb 0x7fad9c00ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45078/0 (socket says 172.21.15.44:45078) 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 -- 172.21.15.44:0/1494149571 learned_addr learned my addr 172.21.15.44:0/1494149571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 -- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 msgr2=0x55e1ff2abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff2abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 -- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 msgr2=0x55e1ff2a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff2a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 -- 172.21.15.44:0/1494149571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ff1e3500 con 0x55e1ff207680 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fadad400640 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad9c061770 tx=0x7fad9c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2306075bb5829a2c server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.595+0000 7fad8e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c06f060 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7fad9c060ed0 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c07a070 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fad9c06aa90 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 10 client.9312 fetch_fsmap learned FSMap version 97 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x55e1ff23f770 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c07a070 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fad9c06ace0 con 0x55e1ff207680 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 10 client.9312 fetch_fsmap finished waiting for FSMap version 97 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 10 client.9312 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 20 client.9312 populate_metadata read hostname 'smithi044' 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff23e6a0 0x55e1ff23ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 4 client.9312 mds_command: new command op to 9183 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:14:28.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.596+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 --> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] -- command(tid 0: {"prefix": "status"}) -- 0x55e1ff227b60 con 0x55e1ff23e6a0 2024-07-17T08:14:28.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fadaca00640 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff23e6a0 0x55e1ff23ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fadaca00640 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff23e6a0 0x55e1ff23ea80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce34ac568f97c88 server_cookie=232de927299501a8 in_seq=0 out_seq=0 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fad8e000640 10 client.9312 ms_handle_connect on v2:172.21.15.44:6833/1832122622 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fad8e000640 1 -- 172.21.15.44:0/1494149571 <== mds.0 v2:172.21.15.44:6833/1832122622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e1ff227b60 con 0x55e1ff23e6a0 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fad8e000640 10 client.9312 handle_command_reply: tid=0 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.597+0000 7fadb2d7e740 1 client.9312 shutdown 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 msgr2=0x55e1ff29edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad9c061770 tx=0x7fad9c00cf60 comp rx=0 tx=0).stop 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 shutdown_connections 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:6833/1832122622,v1:172.21.15.44:6835/1832122622] conn(0x55e1ff23e6a0 0x55e1ff23ea80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e1ff20efc0 0x55e1ff2abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e1ff20b790 0x55e1ff2a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 --2- 172.21.15.44:0/1494149571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1ff207680 0x55e1ff29edd0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fad8e000640 0 client.9312 ms_handle_reset on v2:172.21.15.44:6833/1832122622 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 >> 172.21.15.44:0/1494149571 conn(0x55e1ff1e6280 msgr2=0x55e1ff209c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 shutdown_connections 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 1 -- 172.21.15.44:0/1494149571 wait complete. 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.598+0000 7fadb2d7e740 20 client.9312 trim_cache size 0 max 16384 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:14:28.603 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 9183, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8269628420000004, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 97, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 189, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 189, 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.789025836 2024-07-17T08:14:28.604 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 msgr2=0x7fad88079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 0x7fad88079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad94033000 tx=0x7fada80b3f10 comp rx=0 tx=0).stop 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 msgr2=0x7fada81440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00e690 tx=0x7fad9c00eb50 comp rx=0 tx=0).stop 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 shutdown_connections 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7fad88076bb0 0x7fad88079070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada8073ca0 0x7fada8074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada806c570 0x7fada81f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 --2- 172.21.15.44:0/1381784410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad90004800 0x7fada81440b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 >> 172.21.15.44:0/1381784410 conn(0x7fada80cf0d0 msgr2=0x7fada80b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 shutdown_connections 2024-07-17T08:14:28.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.599+0000 7fadae800640 1 -- 172.21.15.44:0/1381784410 wait complete. 2024-07-17T08:14:28.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 -- 172.21.15.44:0/2942301422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 msgr2=0x7efcf4005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 --2- 172.21.15.44:0/2942301422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efcf4005d90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009b50 tx=0x7efcf4006940 comp rx=0 tx=0).stop 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 -- 172.21.15.44:0/2942301422 shutdown_connections 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 --2- 172.21.15.44:0/2942301422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efcf4005d90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 -- 172.21.15.44:0/2942301422 >> 172.21.15.44:0/2942301422 conn(0x7efd0c0b0e80 msgr2=0x7efd0c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:28.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 -- 172.21.15.44:0/2942301422 shutdown_connections 2024-07-17T08:14:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.804+0000 7efd12a00640 1 -- 172.21.15.44:0/2942301422 wait complete. 2024-07-17T08:14:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.806+0000 7efd12a00640 1 Processor -- start 2024-07-17T08:14:28.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.806+0000 7efd12a00640 1 -- start start 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 0x7efd0c14e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0c0720b0 0x7efd0c072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c09b580 con 0x7efcf4005910 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c09b400 con 0x7efd0c14eda0 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd12a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd0c0cf350 con 0x7efd0c0720b0 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd11600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd11600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45098/0 (socket says 172.21.15.44:45098) 2024-07-17T08:14:28.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.807+0000 7efd11600640 1 -- 172.21.15.44:0/4232044314 learned_addr learned my addr 172.21.15.44:0/4232044314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd12000640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0c0720b0 0x7efd0c072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 -- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0c0720b0 msgr2=0x7efd0c072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd10c00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 0x7efd0c14e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0c0720b0 0x7efd0c072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 -- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 msgr2=0x7efd0c14e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 0x7efd0c14e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 -- 172.21.15.44:0/4232044314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c072b20 con 0x7efcf4005910 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd10c00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 0x7efd0c14e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:14:28.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd11600640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7efd0000d760 tx=0x7efd0000dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00018070 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd000050d0 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0000b6c0 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.808+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c072e00 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.809+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c159d70 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.810+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c0cf8b0 con 0x7efcf4005910 2024-07-17T08:14:28.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.811+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7efd0001b070 con 0x7efcf4005910 2024-07-17T08:14:28.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.812+0000 7efd09600640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 0x7efcec079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:28.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.812+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7efd00097cf0 con 0x7efcf4005910 2024-07-17T08:14:28.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.813+0000 7efd10c00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 0x7efcec079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:28.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.814+0000 7efd10c00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 0x7efcec079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcf8035000 tx=0x7efcf80046c0 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:28.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:28.816+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efd000607b0 con 0x7efcf4005910 2024-07-17T08:14:29.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.069+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd0c0b06f0 con 0x7efcf4005910 2024-07-17T08:14:29.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.070+0000 7efd09600640 1 -- 172.21.15.44:0/4232044314 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7efd00069040 con 0x7efcf4005910 2024-07-17T08:14:29.072 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 msgr2=0x7efcec079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 0x7efcec079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcf8035000 tx=0x7efcf80046c0 comp rx=0 tx=0).stop 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 msgr2=0x7efd0c0ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7efd0000d760 tx=0x7efd0000dc20 comp rx=0 tx=0).stop 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 shutdown_connections 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7efcec076bb0 0x7efcec079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:29.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd0c0720b0 0x7efd0c072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:29.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c14eda0 0x7efd0c14e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:29.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 --2- 172.21.15.44:0/4232044314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcf4005910 0x7efd0c0ac590 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:29.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 >> 172.21.15.44:0/4232044314 conn(0x7efd0c0b0e80 msgr2=0x7efd0c1580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:29.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 shutdown_connections 2024-07-17T08:14:29.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29.073+0000 7efd12a00640 1 -- 172.21.15.44:0/4232044314 wait complete. 2024-07-17T08:14:29.093 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:14:29.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:14:29.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:14:29.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:14:29.094 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:14:29.094 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:14:29.094 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:14:29.094 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:14:29.094 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:14:29.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.094 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.145 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 37788sec preferred_lft 37788sec 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:14:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.147 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:14:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:14:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:14:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:14:29.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.149 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:14:29.223 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:14:29.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.228 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:14:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:14:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:14:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:14:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:14:29.281 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:14:29.282 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:14:29.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:14:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:14:29.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.499 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:14:29.550 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:14:29.550 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:14:29.551 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:14:29.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:14:29.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:14:29.641 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:14:29.697 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:14:29.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:14:29.697 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:14:29.751 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:14:29.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:14:29.940 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:14:29.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:29.940 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:14:30.020 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:14:30.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:14:30.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:14:30.162 INFO:teuthology.orchestra.run.smithi044.stdout:client7097 2024-07-17T08:14:30.162 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:14:30.162 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:14:30.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:14:30.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 -- 172.21.15.44:0/4020398775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1a0004800 msgr2=0x7ff1a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:30.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/4020398775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1a0004800 0x7ff1a0004c80 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80016c0 tx=0x7ff1a0005830 comp rx=0 tx=0).stop 2024-07-17T08:14:30.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 -- 172.21.15.44:0/4020398775 shutdown_connections 2024-07-17T08:14:30.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/4020398775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1a0004800 0x7ff1a0004c80 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:30.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 -- 172.21.15.44:0/4020398775 >> 172.21.15.44:0/4020398775 conn(0x7ff1bc0cf0d0 msgr2=0x7ff1bc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:30.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.429+0000 7ff1c0e00640 1 -- 172.21.15.44:0/4020398775 shutdown_connections 2024-07-17T08:14:30.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.430+0000 7ff1c0e00640 1 -- 172.21.15.44:0/4020398775 wait complete. 2024-07-17T08:14:30.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.431+0000 7ff1c0e00640 1 Processor -- start 2024-07-17T08:14:30.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.431+0000 7ff1c0e00640 1 -- start start 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0004800 0x7ff1bc1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc0afb20 0x7ff1bc0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc06f000 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc06ee80 con 0x7ff1bc0afb20 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.432+0000 7ff1c0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc06f180 con 0x7ff1a0004800 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48744/0 (socket says 172.21.15.44:48744) 2024-07-17T08:14:30.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 -- 172.21.15.44:0/3584425164 learned_addr learned my addr 172.21.15.44:0/3584425164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1bb400640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0004800 0x7ff1bc1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1bbe00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc0afb20 0x7ff1bc0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 -- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0004800 msgr2=0x7ff1bc1f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0004800 0x7ff1bc1f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 -- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc0afb20 msgr2=0x7ff1bc0affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc0afb20 0x7ff1bc0affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:30.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.433+0000 7ff1baa00640 1 -- 172.21.15.44:0/3584425164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc1f6f80 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1baa00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80017e0 tx=0x7ff1a80063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8042070 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1a8039070 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc1f7260 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8003d90 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.434+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc1f7820 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.435+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc14dde0 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.437+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7ff1a803b070 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.439+0000 7ff1b3400640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 0x7ff19c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:30.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.439+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7ff1a80b7b10 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.439+0000 7ff1bb400640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 0x7ff19c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:30.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.441+0000 7ff1bb400640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 0x7ff19c0790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac001860 tx=0x7ff1ac001670 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.445+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff1bc14dde0 con 0x7ff1bc1e7d60 2024-07-17T08:14:30.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:30.681+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 --> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff1bc15c430 con 0x7ff19c076be0 2024-07-17T08:14:33.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:14:33.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:14:37.975 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.974+0000 7fb067c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.975 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.974+0000 7fb067c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.975 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.974+0000 7fb067c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.975 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.974+0000 7fb067c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.975 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.974+0000 7fb067c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.976 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.975+0000 7fb064000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.976 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.975+0000 7fb064000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.976 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.975+0000 7fb064000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.976 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.975+0000 7fb064000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.976 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:37.975+0000 7fb064000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.994+0000 7ff1b3400640 1 -- 172.21.15.44:0/3584425164 <== mgr.7435 v2:172.21.15.175:6834/2760073115 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff1bc15c430 con 0x7ff19c076be0 2024-07-17T08:14:37.996 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:14:37.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:14:37.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:14:37.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:14:37.997 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:14:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 msgr2=0x7ff19c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 0x7ff19c0790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac001860 tx=0x7ff1ac001670 comp rx=0 tx=0).stop 2024-07-17T08:14:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 msgr2=0x7ff1bc1e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80017e0 tx=0x7ff1a80063a0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 shutdown_connections 2024-07-17T08:14:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7ff19c076be0 0x7ff19c0790a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1bc0afb20 0x7ff1bc0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1bc1e7d60 0x7ff1bc1e81e0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 --2- 172.21.15.44:0/3584425164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a0004800 0x7ff1bc1f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 >> 172.21.15.44:0/3584425164 conn(0x7ff1bc0cf0d0 msgr2=0x7ff1bc15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 shutdown_connections 2024-07-17T08:14:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:37.999+0000 7ff1c0e00640 1 -- 172.21.15.44:0/3584425164 wait complete. 2024-07-17T08:14:38.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:14:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 -- 172.21.15.44:0/2155537347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a815b490 msgr2=0x7f61a80b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 --2- 172.21.15.44:0/2155537347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a815b490 0x7f61a80b3e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6190005fd0 tx=0x7f619002aa90 comp rx=0 tx=0).stop 2024-07-17T08:14:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 -- 172.21.15.44:0/2155537347 shutdown_connections 2024-07-17T08:14:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 --2- 172.21.15.44:0/2155537347 >> v2:172.21.15.175:3300/0 conn(0x7f61a80b4350 0x7f61a806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 --2- 172.21.15.44:0/2155537347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a815b490 0x7f61a80b3e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 -- 172.21.15.44:0/2155537347 >> 172.21.15.44:0/2155537347 conn(0x7f61a806ce80 msgr2=0x7f61a806d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 -- 172.21.15.44:0/2155537347 shutdown_connections 2024-07-17T08:14:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.238+0000 7f61ad600640 1 -- 172.21.15.44:0/2155537347 wait complete. 2024-07-17T08:14:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.240+0000 7f61ad600640 1 Processor -- start 2024-07-17T08:14:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.240+0000 7f61ad600640 1 -- start start 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a80b4350 0x7f61a814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a8154940 0x7f61a81f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61a806f4b0 con 0x7f61a815b490 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61a806f330 con 0x7f61a80b4350 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.241+0000 7f61ad600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61a806f630 con 0x7f61a8154940 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48758/0 (socket says 172.21.15.44:48758) 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 -- 172.21.15.44:0/2254278465 learned_addr learned my addr 172.21.15.44:0/2254278465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a6a00640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a80b4350 0x7f61a814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a6000640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a8154940 0x7f61a81f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 -- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a8154940 msgr2=0x7f61a81f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a8154940 0x7f61a81f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 -- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a80b4350 msgr2=0x7f61a814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a80b4350 0x7f61a814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.242+0000 7f61a7400640 1 -- 172.21.15.44:0/2254278465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a81531b0 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f61a7400640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f61a000d950 tx=0x7f61a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0017030 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61a0009a30 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0009e40 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a8153490 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.243+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81f55f0 con 0x7f61a815b490 2024-07-17T08:14:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.244+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a806fb50 con 0x7f61a815b490 2024-07-17T08:14:38.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.246+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 101) ==== 98829+0+0 (secure 0 0 0) 0x7f61a001a070 con 0x7f61a815b490 2024-07-17T08:14:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.247+0000 7f619ea00640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 0x7f6184079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.247+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f61a0098340 con 0x7f61a815b490 2024-07-17T08:14:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.248+0000 7f61a6a00640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 0x7f6184079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.249+0000 7f61a6a00640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 0x7f6184079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6198001680 tx=0x7f6198009040 comp rx=0 tx=0).ready entity=mgr.7435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:38.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.252+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f61a0021070 con 0x7f61a815b490 2024-07-17T08:14:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.521+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f61a8073280 con 0x7f61a815b490 2024-07-17T08:14:38.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.987+0000 7f619ea00640 1 -- 172.21.15.44:0/2254278465 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v102) ==== 92+0+0 (secure 0 0 0) 0x7f61a0061460 con 0x7f61a815b490 2024-07-17T08:14:38.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.991+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 msgr2=0x7f6184079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.991+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 0x7f6184079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6198001680 tx=0x7f6198009040 comp rx=0 tx=0).stop 2024-07-17T08:14:38.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.991+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 msgr2=0x7f61a81528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.991+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f61a000d950 tx=0x7f61a000de10 comp rx=0 tx=0).stop 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 shutdown_connections 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f6184076bb0 0x7f6184079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61a815b490 0x7f61a81528b0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61a8154940 0x7f61a81f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 --2- 172.21.15.44:0/2254278465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a80b4350 0x7f61a814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 >> 172.21.15.44:0/2254278465 conn(0x7f61a806ce80 msgr2=0x7f61a815ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 shutdown_connections 2024-07-17T08:14:38.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:38.992+0000 7f61ad600640 1 -- 172.21.15.44:0/2254278465 wait complete. 2024-07-17T08:14:39.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:14:39.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 -- 172.21.15.44:0/1161697869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 msgr2=0x7f8c38068860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 --2- 172.21.15.44:0/1161697869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c38068860 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8c3815b020 tx=0x7f8c2802a530 comp rx=0 tx=0).stop 2024-07-17T08:14:39.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 -- 172.21.15.44:0/1161697869 shutdown_connections 2024-07-17T08:14:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 --2- 172.21.15.44:0/1161697869 >> v2:172.21.15.175:3300/0 conn(0x7f8c380735a0 0x7f8c380b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 --2- 172.21.15.44:0/1161697869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c38068860 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 -- 172.21.15.44:0/1161697869 >> 172.21.15.44:0/1161697869 conn(0x7f8c3806ce80 msgr2=0x7f8c3806d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 -- 172.21.15.44:0/1161697869 shutdown_connections 2024-07-17T08:14:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.218+0000 7f8c3da00640 1 -- 172.21.15.44:0/1161697869 wait complete. 2024-07-17T08:14:39.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.220+0000 7f8c3da00640 1 Processor -- start 2024-07-17T08:14:39.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.220+0000 7f8c3da00640 1 -- start start 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c380735a0 0x7f8c3813a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c380b55a0 0x7f8c3806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38164530 con 0x7f8c38159990 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3815c590 con 0x7f8c380735a0 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.221+0000 7f8c3da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c381646b0 con 0x7f8c380b55a0 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34904/0 (socket says 172.21.15.44:34904) 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 -- 172.21.15.44:0/617122291 learned_addr learned my addr 172.21.15.44:0/617122291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c36a00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c380735a0 0x7f8c3813a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c36000640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c380b55a0 0x7f8c3806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c380b55a0 msgr2=0x7f8c3806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c380b55a0 0x7f8c3806e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c380735a0 msgr2=0x7f8c3813a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c380735a0 0x7f8c3813a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.222+0000 7f8c37400640 1 -- 172.21.15.44:0/617122291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c381f7c70 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c37400640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000da10 tx=0x7f8c3000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30016070 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c30003970 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30003dc0 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c381f7f50 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.223+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381f8510 con 0x7f8c38159990 2024-07-17T08:14:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.224+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c380ad960 con 0x7f8c38159990 2024-07-17T08:14:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.226+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 102) ==== 98846+0+0 (secure 0 0 0) 0x7f8c30003f60 con 0x7f8c38159990 2024-07-17T08:14:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.227+0000 7f8c2ea00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 0x7f8c180790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.227+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f8c30099060 con 0x7f8c38159990 2024-07-17T08:14:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.227+0000 7f8c36a00640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 msgr2=0x7f8c180790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2760073115 2024-07-17T08:14:39.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.227+0000 7f8c36a00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 0x7f8c180790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.230+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8c380ad960 con 0x7f8c38159990 2024-07-17T08:14:39.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.429+0000 7f8c36a00640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 msgr2=0x7f8c180790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2760073115 2024-07-17T08:14:39.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.429+0000 7f8c36a00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 0x7f8c180790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:39.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.441+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8c380b5a10 con 0x7f8c38159990 2024-07-17T08:14:39.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.447+0000 7f8c2ea00640 1 -- 172.21.15.44:0/617122291 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) ==== 138+0+0 (secure 0 0 0) 0x7f8c30061ba0 con 0x7f8c38159990 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 msgr2=0x7f8c180790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 0x7f8c180790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 msgr2=0x7f8c380afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000da10 tx=0x7f8c3000ded0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 shutdown_connections 2024-07-17T08:14:39.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f8c18076c00 0x7f8c180790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c38159990 0x7f8c380afb50 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c380b55a0 0x7f8c3806e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 --2- 172.21.15.44:0/617122291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c380735a0 0x7f8c3813a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 >> 172.21.15.44:0/617122291 conn(0x7f8c3806ce80 msgr2=0x7f8c38072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:39.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 shutdown_connections 2024-07-17T08:14:39.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.450+0000 7f8c3da00640 1 -- 172.21.15.44:0/617122291 wait complete. 2024-07-17T08:14:39.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:14:39.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.675+0000 7f10db200640 1 -- 172.21.15.44:0/2353082117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d415d130 msgr2=0x7f10d415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.675+0000 7f10db200640 1 --2- 172.21.15.44:0/2353082117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d415d130 0x7f10d415d510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f10cc005fd0 tx=0x7f10cc02aa10 comp rx=0 tx=0).stop 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 -- 172.21.15.44:0/2353082117 shutdown_connections 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 --2- 172.21.15.44:0/2353082117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d4161be0 0x7f10d40b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 --2- 172.21.15.44:0/2353082117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d415d130 0x7f10d415d510 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 -- 172.21.15.44:0/2353082117 >> 172.21.15.44:0/2353082117 conn(0x7f10d40b4ff0 msgr2=0x7f10d40b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 -- 172.21.15.44:0/2353082117 shutdown_connections 2024-07-17T08:14:39.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.676+0000 7f10db200640 1 -- 172.21.15.44:0/2353082117 wait complete. 2024-07-17T08:14:39.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.677+0000 7f10db200640 1 Processor -- start 2024-07-17T08:14:39.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.677+0000 7f10db200640 1 -- start start 2024-07-17T08:14:39.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d40ce690 0x7f10d4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d415d130 0x7f10d41540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40d1a10 con 0x7f10d4161be0 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40d1890 con 0x7f10d40ce690 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.678+0000 7f10db200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10d40d1b90 con 0x7f10d415d130 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.679+0000 7f10da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.679+0000 7f10d9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d415d130 0x7f10d41540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.679+0000 7f10da800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34920/0 (socket says 172.21.15.44:34920) 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.679+0000 7f10da800640 1 -- 172.21.15.44:0/3411667254 learned_addr learned my addr 172.21.15.44:0/3411667254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:14:39.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.679+0000 7f10d9e00640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d40ce690 0x7f10d4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d415d130 msgr2=0x7f10d41540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d415d130 0x7f10d41540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d40ce690 msgr2=0x7f10d4152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d40ce690 0x7f10d4152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 -- 172.21.15.44:0/3411667254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d41551d0 con 0x7f10d4161be0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10da800640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c3c0 tx=0x7f10d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0017070 con 0x7f10d4161be0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d00053c0 con 0x7f10d4161be0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d41554b0 con 0x7f10d4161be0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0003850 con 0x7f10d4161be0 2024-07-17T08:14:39.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.680+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d4155d80 con 0x7f10d4161be0 2024-07-17T08:14:39.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.682+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d40b2200 con 0x7f10d4161be0 2024-07-17T08:14:39.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.683+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 102) ==== 98846+0+0 (secure 0 0 0) 0x7f10d001a070 con 0x7f10d4161be0 2024-07-17T08:14:39.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.684+0000 7f10ca000640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 0x7f10b80790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:14:39.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.684+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 6649+0+0 (secure 0 0 0) 0x7f10d00979e0 con 0x7f10d4161be0 2024-07-17T08:14:39.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.684+0000 7f10d9e00640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 msgr2=0x7f10b80790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2760073115 2024-07-17T08:14:39.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.685+0000 7f10d9e00640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 0x7f10b80790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:14:39.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.688+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f10d0060410 con 0x7f10d4161be0 2024-07-17T08:14:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.886+0000 7f10d9e00640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 msgr2=0x7f10b80790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2760073115 2024-07-17T08:14:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.886+0000 7f10d9e00640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 0x7f10b80790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:14:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.897+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f10d40b3bc0 con 0x7f10d4161be0 2024-07-17T08:14:39.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.903+0000 7f10ca000640 1 -- 172.21.15.44:0/3411667254 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) ==== 133+0+0 (secure 0 0 0) 0x7f10d0064960 con 0x7f10d4161be0 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 msgr2=0x7f10b80790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 0x7f10b80790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 msgr2=0x7f10d41562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c3c0 tx=0x7f10d000c880 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 shutdown_connections 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:6834/2760073115,v1:172.21.15.175:6837/2760073115] conn(0x7f10b8076c00 0x7f10b80790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d4161be0 0x7f10d41562a0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10d415d130 0x7f10d41540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 --2- 172.21.15.44:0/3411667254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10d40ce690 0x7f10d4152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:14:39.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 >> 172.21.15.44:0/3411667254 conn(0x7f10d40b4ff0 msgr2=0x7f10d40b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:14:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 shutdown_connections 2024-07-17T08:14:39.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:14:39.906+0000 7f10db200640 1 -- 172.21.15.44:0/3411667254 wait complete. 2024-07-17T08:14:39.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:14:39.928 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T08:14:43.595 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.593+0000 7f6a94400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a94400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a94400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a94400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a94400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a98000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a98000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.597 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a98000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.597 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a98000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:14:43.597 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:14:43.594+0000 7f6a98000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:15:03.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:15:03.542 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:15:33.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:15:33.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:15:41.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:15:41.827 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2024-07-17T08:15:41.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2024-07-17T08:15:42.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 -- 172.21.15.44:0/3866829519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5300040c0 msgr2=0x7fd530004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 --2- 172.21.15.44:0/3866829519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5300040c0 0x7fd530004800 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fd538009b50 tx=0x7fd530005830 comp rx=0 tx=0).stop 2024-07-17T08:15:42.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54a800640 1 -- 172.21.15.44:0/3866829519 reap_dead start 2024-07-17T08:15:42.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 -- 172.21.15.44:0/3866829519 shutdown_connections 2024-07-17T08:15:42.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 -- 172.21.15.44:0/3866829519 >> 172.21.15.44:0/3866829519 conn(0x7fd54c0b4ff0 msgr2=0x7fd54c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:15:42.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 -- 172.21.15.44:0/3866829519 shutdown_connections 2024-07-17T08:15:42.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.105+0000 7fd54b200640 1 -- 172.21.15.44:0/3866829519 wait complete. 2024-07-17T08:15:42.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.107+0000 7fd54b200640 1 Processor -- start 2024-07-17T08:15:42.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.107+0000 7fd54b200640 1 -- start start 2024-07-17T08:15:42.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 0x7fd54c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 0x7fd54c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1643f0 con 0x7fd54c15fe40 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c164270 con 0x7fd54c157a60 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.108+0000 7fd54b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c164570 con 0x7fd5300040c0 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51424/0 (socket says 172.21.15.44:51424) 2024-07-17T08:15:42.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 -- 172.21.15.44:0/837408142 learned_addr learned my addr 172.21.15.44:0/837408142 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:15:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd549400640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 0x7fd54c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 -- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 msgr2=0x7fd54c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd549e00640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 0x7fd54c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 0x7fd54c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 -- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 msgr2=0x7fd54c14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 0x7fd54c14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd54a800640 1 -- 172.21.15.44:0/837408142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c0b87c0 con 0x7fd54c15fe40 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd549400640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 0x7fd54c14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.109+0000 7fd549e00640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 0x7fd54c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd54a800640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd54c06c300 tx=0x7fd54400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54400f040 con 0x7fd54c15fe40 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd544018070 con 0x7fd54c15fe40 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54400c8a0 con 0x7fd54c15fe40 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c14bd10 con 0x7fd54c15fe40 2024-07-17T08:15:42.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.110+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c0cce10 con 0x7fd54c15fe40 2024-07-17T08:15:42.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.111+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c0cd010 con 0x7fd54c15fe40 2024-07-17T08:15:42.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.113+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 106) ==== 99005+0+0 (secure 0 0 0) 0x7fd54400ca40 con 0x7fd54c15fe40 2024-07-17T08:15:42.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.113+0000 7fd542000640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 0x7fd52c0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.114+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 6907+0+0 (secure 0 0 0) 0x7fd5440981b0 con 0x7fd54c15fe40 2024-07-17T08:15:42.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.114+0000 7fd549e00640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 0x7fd52c0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.115+0000 7fd549e00640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 0x7fd52c0792a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd53c001860 tx=0x7fd53c009040 comp rx=0 tx=0).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:15:42.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.118+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd54c0cd010 con 0x7fd54c15fe40 2024-07-17T08:15:42.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.358+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 --> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd54c0cfdf0 con 0x7fd52c076de0 2024-07-17T08:15:42.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.548+0000 7fd542000640 1 -- 172.21.15.44:0/837408142 <== mgr.7127 v2:172.21.15.175:6834/4042077236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd54c0cfdf0 con 0x7fd52c076de0 2024-07-17T08:15:42.549 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /snap_test_dir1/test_restart 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 msgr2=0x7fd52c0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 0x7fd52c0792a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd53c001860 tx=0x7fd53c009040 comp rx=0 tx=0).stop 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 msgr2=0x7fd54c1602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fd54c06c300 tx=0x7fd54400ec20 comp rx=0 tx=0).stop 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 shutdown_connections 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fd52c076de0 0x7fd52c0792a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd54c15fe40 0x7fd54c1602c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd54c157a60 0x7fd54c14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 --2- 172.21.15.44:0/837408142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5300040c0 0x7fd54c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 >> 172.21.15.44:0/837408142 conn(0x7fd54c0b4ff0 msgr2=0x7fd54c0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:15:42.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 shutdown_connections 2024-07-17T08:15:42.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.551+0000 7fd54b200640 1 -- 172.21.15.44:0/837408142 wait complete. 2024-07-17T08:15:42.575 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2024-07-17T08:15:42.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 -- 172.21.15.44:0/1927569463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 msgr2=0x7fed400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 --2- 172.21.15.44:0/1927569463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed400051c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fed54005fd0 tx=0x7fed40005830 comp rx=0 tx=0).stop 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed5a800640 1 -- 172.21.15.44:0/1927569463 reap_dead start 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 -- 172.21.15.44:0/1927569463 shutdown_connections 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 -- 172.21.15.44:0/1927569463 >> 172.21.15.44:0/1927569463 conn(0x7fed5c0b0eb0 msgr2=0x7fed5c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 -- 172.21.15.44:0/1927569463 shutdown_connections 2024-07-17T08:15:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.789+0000 7fed61000640 1 -- 172.21.15.44:0/1927569463 wait complete. 2024-07-17T08:15:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.791+0000 7fed61000640 1 Processor -- start 2024-07-17T08:15:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.791+0000 7fed61000640 1 -- start start 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5c152ed0 0x7fed5c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5c1e4d80 0x7fed5c1e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0b3280 con 0x7fed40004d40 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c06f930 con 0x7fed5c152ed0 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed61000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c0b3400 con 0x7fed5c1e4d80 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed59e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed59e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51448/0 (socket says 172.21.15.44:51448) 2024-07-17T08:15:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.792+0000 7fed59e00640 1 -- 172.21.15.44:0/3360699530 learned_addr learned my addr 172.21.15.44:0/3360699530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:15:42.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 -- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5c1e4d80 msgr2=0x7fed5c1e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5c1e4d80 0x7fed5c1e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 -- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5c152ed0 msgr2=0x7fed5c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5c152ed0 0x7fed5c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 -- 172.21.15.44:0/3360699530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c1e3e20 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed59e00640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00d5d0 tx=0x7fed4c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c014070 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed4c00f990 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00fde0 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c1e3060 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.793+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1e3620 con 0x7fed40004d40 2024-07-17T08:15:42.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.794+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c153890 con 0x7fed40004d40 2024-07-17T08:15:42.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.796+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 106) ==== 99005+0+0 (secure 0 0 0) 0x7fed4c017070 con 0x7fed40004d40 2024-07-17T08:15:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.797+0000 7fed52000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 0x7fed3c0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:15:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.797+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 6907+0+0 (secure 0 0 0) 0x7fed4c098e10 con 0x7fed40004d40 2024-07-17T08:15:42.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.798+0000 7fed59400640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 0x7fed3c0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:15:42.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.799+0000 7fed59400640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 0x7fed3c0792a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fed48001eb0 tx=0x7fed48001d10 comp rx=0 tx=0).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:15:42.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:42.802+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fed4c09c030 con 0x7fed40004d40 2024-07-17T08:15:43.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.039+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 --> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fed5c1e2e10 con 0x7fed3c076de0 2024-07-17T08:15:43.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.192+0000 7fed52000640 1 -- 172.21.15.44:0/3360699530 <== mgr.7127 v2:172.21.15.175:6834/4042077236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7fed5c1e2e10 con 0x7fed3c076de0 2024-07-17T08:15:43.194 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path /snap_test_dir1/test_restart 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 msgr2=0x7fed3c0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 0x7fed3c0792a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fed48001eb0 tx=0x7fed48001d10 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 msgr2=0x7fed5c150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00d5d0 tx=0x7fed4c00da90 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 shutdown_connections 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.195+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fed3c076de0 0x7fed3c0792a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5c1e4d80 0x7fed5c1e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5c152ed0 0x7fed5c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 --2- 172.21.15.44:0/3360699530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed40004d40 0x7fed5c150ba0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 >> 172.21.15.44:0/3360699530 conn(0x7fed5c0b0eb0 msgr2=0x7fed5c0b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 shutdown_connections 2024-07-17T08:15:43.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:15:43.196+0000 7fed61000640 1 -- 172.21.15.44:0/3360699530 wait complete. 2024-07-17T08:15:43.219 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2024-07-17T08:16:00.219 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:16:00.216+0000 7f6a7e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:16:00.219 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:16:00.216+0000 7f6a7e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:16:00.219 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:16:00.216+0000 7f6a7e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:16:00.219 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:16:00.216+0000 7f6a7e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:16:00.219 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:16:00.216+0000 7f6a7e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:16:03.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:16:03.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:16:33.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:16:33.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:17:04.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:17:04.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:17:34.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:17:34.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:18:04.352 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:18:04.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:18:34.515 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:18:34.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:19:04.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:19:04.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:19:34.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:19:34.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:20:04.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:20:04.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:20:35.093 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:20:35.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:21:05.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:21:05.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:21:35.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:21:35.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:22:05.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:22:05.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:22:35.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:22:35.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:23:05.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:23:05.796 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:23:35.931 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:23:35.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:24:06.068 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:24:06.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:24:36.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:24:36.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:25:06.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:25:06.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:25:36.473 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:25:36.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:26:06.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:26:06.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:26:16.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:26:16.221 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-07-17T08:26:16.281 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_16_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_17_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_18_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_19_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_20_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_21_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_22_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_23_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_24_00_UTC 2024-07-17T08:26:16.282 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_25_00_UTC 2024-07-17T08:26:16.283 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_26_00_UTC 2024-07-17T08:26:16.283 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-08_16_00_UTC', 'scheduled-2024-07-17-08_17_00_UTC', 'scheduled-2024-07-17-08_18_00_UTC', 'scheduled-2024-07-17-08_19_00_UTC', 'scheduled-2024-07-17-08_20_00_UTC', 'scheduled-2024-07-17-08_21_00_UTC', 'scheduled-2024-07-17-08_22_00_UTC', 'scheduled-2024-07-17-08_23_00_UTC', 'scheduled-2024-07-17-08_24_00_UTC', 'scheduled-2024-07-17-08_25_00_UTC', 'scheduled-2024-07-17-08_26_00_UTC'] 2024-07-17T08:26:16.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2024-07-17T08:26:16.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 -- 172.21.15.44:0/1990042396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 msgr2=0x7fb14c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:16.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 --2- 172.21.15.44:0/1990042396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c06d620 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb14c06f610 tx=0x7fb13c02a980 comp rx=0 tx=0).stop 2024-07-17T08:26:16.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 -- 172.21.15.44:0/1990042396 shutdown_connections 2024-07-17T08:26:16.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 --2- 172.21.15.44:0/1990042396 >> v2:172.21.15.175:3300/0 conn(0x7fb14c06ed30 0x7fb14c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 --2- 172.21.15.44:0/1990042396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c06d620 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 -- 172.21.15.44:0/1990042396 >> 172.21.15.44:0/1990042396 conn(0x7fb14c0b0eb0 msgr2=0x7fb14c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:26:16.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 -- 172.21.15.44:0/1990042396 shutdown_connections 2024-07-17T08:26:16.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.502+0000 7fb152400640 1 -- 172.21.15.44:0/1990042396 wait complete. 2024-07-17T08:26:16.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.504+0000 7fb152400640 1 Processor -- start 2024-07-17T08:26:16.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.504+0000 7fb152400640 1 -- start start 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 0x7fb14c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb14c0cb3a0 0x7fb14c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0cbc30 con 0x7fb14c152220 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c0cbab0 con 0x7fb14c0cb3a0 2024-07-17T08:26:16.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.505+0000 7fb152400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c15f570 con 0x7fb14c06ed30 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54834/0 (socket says 172.21.15.44:54834) 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 -- 172.21.15.44:0/3172426204 learned_addr learned my addr 172.21.15.44:0/3172426204 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14b400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 0x7fb14c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14aa00640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb14c0cb3a0 0x7fb14c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 -- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 msgr2=0x7fb14c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 0x7fb14c15e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 -- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb14c0cb3a0 msgr2=0x7fb14c0734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb14c0cb3a0 0x7fb14c0734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14be00640 1 -- 172.21.15.44:0/3172426204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c0739f0 con 0x7fb14c152220 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.506+0000 7fb14b400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 0x7fb14c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:26:16.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb14be00640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb14400c390 tx=0x7fb14400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144017070 con 0x7fb14c152220 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1440035f0 con 0x7fb14c152220 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144003a40 con 0x7fb14c152220 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1f71c0 con 0x7fb14c152220 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.507+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1f7a60 con 0x7fb14c152220 2024-07-17T08:26:16.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.508+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c15fb10 con 0x7fb14c152220 2024-07-17T08:26:16.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.509+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 108) ==== 98960+0+0 (secure 0 0 0) 0x7fb14401a070 con 0x7fb14c152220 2024-07-17T08:26:16.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.510+0000 7fb143400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 0x7fb128079230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:16.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.511+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 6993+0+0 (secure 0 0 0) 0x7fb144097e50 con 0x7fb14c152220 2024-07-17T08:26:16.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.511+0000 7fb14b400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 0x7fb128079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:16.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.512+0000 7fb14b400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 0x7fb128079230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb13c035000 tx=0x7fb13c004ab0 comp rx=0 tx=0).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:26:16.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.516+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb144060ce0 con 0x7fb14c152220 2024-07-17T08:26:16.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.756+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 --> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb14c15ef80 con 0x7fb128076d70 2024-07-17T08:26:16.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.774+0000 7fb143400640 1 -- 172.21.15.44:0/3172426204 <== mgr.7127 v2:172.21.15.175:6834/4042077236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+375 (secure 0 0 0) 0x7fb14c15ef80 con 0x7fb128076d70 2024-07-17T08:26:16.775 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:26:16.775 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:15:42", "first": "2024-07-17T08:16:00", "last": "2024-07-17T08:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.776+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 msgr2=0x7fb128079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.776+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 0x7fb128079230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb13c035000 tx=0x7fb13c004ab0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.776+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 msgr2=0x7fb14c1505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.776+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb14400c390 tx=0x7fb14400c850 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 shutdown_connections 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7fb128076d70 0x7fb128079230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c152220 0x7fb14c1505e0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb14c0cb3a0 0x7fb14c0734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 --2- 172.21.15.44:0/3172426204 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb14c06ed30 0x7fb14c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 >> 172.21.15.44:0/3172426204 conn(0x7fb14c0b0eb0 msgr2=0x7fb14c0aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:26:16.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 shutdown_connections 2024-07-17T08:26:16.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:16.777+0000 7fb152400640 1 -- 172.21.15.44:0/3172426204 wait complete. 2024-07-17T08:26:16.797 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:15:42", "first": "2024-07-17T08:16:00", "last": "2024-07-17T08:26:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2024-07-17T08:26:16.797 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T08:15:42', 'first': '2024-07-17T08:16:00', 'last': '2024-07-17T08:26:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2024-07-17T08:26:16.797 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2024-07-17T08:26:16.797 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2024-07-17T08:26:16.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.013+0000 7f726ea00640 1 -- 172.21.15.44:0/3940787540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f724c003a00 msgr2=0x7f724c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.013+0000 7f726ea00640 1 --2- 172.21.15.44:0/3940787540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f724c003a00 0x7f724c003e20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f725800bbb0 tx=0x7f724c005830 comp rx=0 tx=0).stop 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.014+0000 7f726ca00640 1 -- 172.21.15.44:0/3940787540 reap_dead start 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.014+0000 7f726ea00640 1 -- 172.21.15.44:0/3940787540 shutdown_connections 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.014+0000 7f726ea00640 1 -- 172.21.15.44:0/3940787540 >> 172.21.15.44:0/3940787540 conn(0x7f72680b3020 msgr2=0x7f72680b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.014+0000 7f726ea00640 1 -- 172.21.15.44:0/3940787540 shutdown_connections 2024-07-17T08:26:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.014+0000 7f726ea00640 1 -- 172.21.15.44:0/3940787540 wait complete. 2024-07-17T08:26:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.016+0000 7f726ea00640 1 Processor -- start 2024-07-17T08:26:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.016+0000 7f726ea00640 1 -- start start 2024-07-17T08:26:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f724c003a00 0x7f72680b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72681f4900 0x7f72680b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72680ae480 con 0x7f726815d600 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72680ae300 con 0x7f72681f4900 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72680ae600 con 0x7f724c003a00 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54856/0 (socket says 172.21.15.44:54856) 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f726ca00640 1 -- 172.21.15.44:0/3954486138 learned_addr learned my addr 172.21.15.44:0/3954486138 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f7267e00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f724c003a00 0x7f72680b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.017+0000 7f725fe00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72681f4900 0x7f72680b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 -- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f724c003a00 msgr2=0x7f72680b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f724c003a00 0x7f72680b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 -- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72681f4900 msgr2=0x7f72680b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72681f4900 0x7f72680b6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 -- 172.21.15.44:0/3954486138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72680b6010 con 0x7f726815d600 2024-07-17T08:26:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ca00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7268071520 tx=0x7f726000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:26:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726000dda0 con 0x7f726815d600 2024-07-17T08:26:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7260011440 con 0x7f726815d600 2024-07-17T08:26:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726815bb70 con 0x7f726815d600 2024-07-17T08:26:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260011890 con 0x7f726815d600 2024-07-17T08:26:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.018+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726815c130 con 0x7f726815d600 2024-07-17T08:26:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.020+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72680cf140 con 0x7f726815d600 2024-07-17T08:26:17.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.021+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 108) ==== 98960+0+0 (secure 0 0 0) 0x7f7260011a30 con 0x7f726815d600 2024-07-17T08:26:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.023+0000 7f7264c00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 0x7f7244079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:26:17.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.023+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 6993+0+0 (secure 0 0 0) 0x7f72600995e0 con 0x7f726815d600 2024-07-17T08:26:17.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.023+0000 7f7267e00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 0x7f7244079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:26:17.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.025+0000 7f7267e00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 0x7f7244079210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f72580046c0 comp rx=0 tx=0).ready entity=mgr.7127 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:26:17.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.027+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f72680cf140 con 0x7f726815d600 2024-07-17T08:26:17.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.289+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f72680cf560 con 0x7f726815d600 2024-07-17T08:26:17.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.294+0000 7f7264c00640 1 -- 172.21.15.44:0/3954486138 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v108) ==== 74+0+190801 (secure 0 0 0) 0x7f7260021060 con 0x7f726815d600 2024-07-17T08:26:17.298 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 msgr2=0x7f7244079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 0x7f7244079210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f72580046c0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 msgr2=0x7f726815da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7268071520 tx=0x7f726000fe20 comp rx=0 tx=0).stop 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 shutdown_connections 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:6834/4042077236,v1:172.21.15.175:6837/4042077236] conn(0x7f7244076d50 0x7f7244079210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f726815d600 0x7f726815da90 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72681f4900 0x7f72680b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 --2- 172.21.15.44:0/3954486138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f724c003a00 0x7f72680b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:26:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 >> 172.21.15.44:0/3954486138 conn(0x7f72680b3020 msgr2=0x7f72680d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:26:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 shutdown_connections 2024-07-17T08:26:17.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:26:17.299+0000 7f726ea00640 1 -- 172.21.15.44:0/3954486138 wait complete. 2024-07-17T08:26:17.321 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":108,"flags":0,"active_gid":7127,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4042077236},{"type":"v1","addr":"172.21.15.175:6837","nonce":4042077236}]},"active_addr":"172.21.15.175:6837/4042077236","active_change":"2024-07-17T08:14:43.235394+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":9336,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":190,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2945977477}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3253828084}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":256403424}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2650508982}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2961426739}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1323057578}]}]} 2024-07-17T08:26:17.325 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2024-07-17T08:26:17.325 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-07-17T08:26:17.325 INFO:tasks.ceph.mgr.x:Stopping old one... 2024-07-17T08:26:17.325 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-07-17T08:26:17.325 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:26:17.420 INFO:tasks.ceph.mgr.x:Stopped 2024-07-17T08:26:17.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-07-17T08:26:17.425 INFO:tasks.ceph.mgr.x:Started 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.070+0000 7ff5af200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.070+0000 7ff5af200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.070+0000 7ff5af200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.070+0000 7ff5af200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.070+0000 7ff5af200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.071+0000 7ff5aac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.071+0000 7ff5aac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.071+0000 7ff5aac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.071+0000 7ff5aac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:22.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:26:22.071+0000 7ff5aac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:26:36.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:26:36.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:27:00.071 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:27:00.070+0000 7ff5b9200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:27:00.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:27:00.070+0000 7ff5b9200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:27:00.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:27:00.070+0000 7ff5b9200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:27:00.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:27:00.070+0000 7ff5b9200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:27:00.072 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:27:00.070+0000 7ff5b9200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:27:06.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:27:06.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:27:37.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:27:37.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:28:07.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:28:07.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:28:37.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:28:37.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:29:07.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:29:07.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:29:37.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:29:37.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:30:07.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:30:07.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:30:37.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:30:37.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:31:08.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:31:08.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:31:17.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2024-07-17T08:31:17.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 -- 172.21.15.44:0/1655714174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61680ac190 msgr2=0x7f61680ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 --2- 172.21.15.44:0/1655714174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61680ac190 0x7f61680ac640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6160005fd0 tx=0x7f616002aa90 comp rx=0 tx=0).stop 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 -- 172.21.15.44:0/1655714174 shutdown_connections 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 --2- 172.21.15.44:0/1655714174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61680ac190 0x7f61680ac640 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 --2- 172.21.15.44:0/1655714174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6168154600 0x7f61680b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 -- 172.21.15.44:0/1655714174 >> 172.21.15.44:0/1655714174 conn(0x7f61680b0eb0 msgr2=0x7f61680b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 -- 172.21.15.44:0/1655714174 shutdown_connections 2024-07-17T08:31:17.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.650+0000 7f6170000640 1 -- 172.21.15.44:0/1655714174 wait complete. 2024-07-17T08:31:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.652+0000 7f6170000640 1 Processor -- start 2024-07-17T08:31:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.652+0000 7f6170000640 1 -- start start 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680ac190 0x7f6168143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6168154600 0x7f616815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f616815e220 con 0x7f6168161120 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f616815e0a0 con 0x7f6168154600 2024-07-17T08:31:17.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.653+0000 7f6170000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f616815e3a0 con 0x7f61680ac190 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35562/0 (socket says 172.21.15.44:35562) 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 -- 172.21.15.44:0/2951762175 learned_addr learned my addr 172.21.15.44:0/2951762175 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616cc00640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6168154600 0x7f616815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616d600640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680ac190 0x7f6168143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 -- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680ac190 msgr2=0x7f6168143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680ac190 0x7f6168143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 -- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6168154600 msgr2=0x7f616815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:17.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6168154600 0x7f616815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.654+0000 7f616e000640 1 -- 172.21.15.44:0/2951762175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681f4230 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f616e000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f6160003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160046070 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6160004040 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160004430 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681f4510 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.655+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681f4ad0 con 0x7f6168161120 2024-07-17T08:31:17.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.657+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616815e930 con 0x7f6168161120 2024-07-17T08:31:17.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.658+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 114) ==== 98960+0+0 (secure 0 0 0) 0x7f6160049070 con 0x7f6168161120 2024-07-17T08:31:17.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.658+0000 7f6165600640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 0x7f61480791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:17.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.659+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 7337+0+0 (secure 0 0 0) 0x7f61600b98c0 con 0x7f6168161120 2024-07-17T08:31:17.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.659+0000 7f616d600640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 0x7f61480791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:17.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.660+0000 7f616d600640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 0x7f61480791e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f61680b03d0 tx=0x7f615c005f90 comp rx=0 tx=0).ready entity=mgr.7777 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:17.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.663+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6160032030 con 0x7f6168161120 2024-07-17T08:31:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:17.909+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 --> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6168160e90 con 0x7f6148076d20 2024-07-17T08:31:18.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.091+0000 7f6165600640 1 -- 172.21.15.44:0/2951762175 <== mgr.7777 v2:172.21.15.175:6834/1237700724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6168160e90 con 0x7f6148076d20 2024-07-17T08:31:18.093 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 msgr2=0x7f61480791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 0x7f61480791e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f61680b03d0 tx=0x7f615c005f90 comp rx=0 tx=0).stop 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 msgr2=0x7f616815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f6160003ce0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 shutdown_connections 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6148076d20 0x7f61480791e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168161120 0x7f616815b160 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6168154600 0x7f616815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 --2- 172.21.15.44:0/2951762175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680ac190 0x7f6168143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 >> 172.21.15.44:0/2951762175 conn(0x7f61680b0eb0 msgr2=0x7f616815cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:18.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 shutdown_connections 2024-07-17T08:31:18.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.096+0000 7f6170000640 1 -- 172.21.15.44:0/2951762175 wait complete. 2024-07-17T08:31:18.120 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2024-07-17T08:31:18.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:18.121 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-07-17T08:31:18.173 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_16_00_UTC 2024-07-17T08:31:18.173 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_17_00_UTC 2024-07-17T08:31:18.173 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_18_00_UTC 2024-07-17T08:31:18.173 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_19_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_20_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_21_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_22_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_23_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_24_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_25_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_26_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_27_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_28_00_UTC 2024-07-17T08:31:18.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_29_00_UTC 2024-07-17T08:31:18.175 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_30_00_UTC 2024-07-17T08:31:18.175 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_31_00_UTC 2024-07-17T08:31:18.176 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2024-07-17-08_16_00_UTC', 'scheduled-2024-07-17-08_17_00_UTC', 'scheduled-2024-07-17-08_18_00_UTC', 'scheduled-2024-07-17-08_19_00_UTC', 'scheduled-2024-07-17-08_20_00_UTC', 'scheduled-2024-07-17-08_21_00_UTC', 'scheduled-2024-07-17-08_22_00_UTC', 'scheduled-2024-07-17-08_23_00_UTC', 'scheduled-2024-07-17-08_24_00_UTC', 'scheduled-2024-07-17-08_25_00_UTC', 'scheduled-2024-07-17-08_26_00_UTC', 'scheduled-2024-07-17-08_27_00_UTC', 'scheduled-2024-07-17-08_28_00_UTC', 'scheduled-2024-07-17-08_29_00_UTC', 'scheduled-2024-07-17-08_30_00_UTC', 'scheduled-2024-07-17-08_31_00_UTC'] 2024-07-17T08:31:18.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2024-07-17T08:31:18.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.387+0000 7f8bb0400640 1 -- 172.21.15.44:0/1922523399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90004050 msgr2=0x7f8b90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.387+0000 7f8bb0400640 1 --2- 172.21.15.44:0/1922523399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90004050 0x7f8b90004800 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006f80 tx=0x7f8b90005830 comp rx=0 tx=0).stop 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.387+0000 7f8bae400640 1 -- 172.21.15.44:0/1922523399 reap_dead start 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.387+0000 7f8bb0400640 1 -- 172.21.15.44:0/1922523399 shutdown_connections 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.387+0000 7f8bb0400640 1 -- 172.21.15.44:0/1922523399 >> 172.21.15.44:0/1922523399 conn(0x7f8ba80b0eb0 msgr2=0x7f8ba80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.388+0000 7f8bb0400640 1 -- 172.21.15.44:0/1922523399 shutdown_connections 2024-07-17T08:31:18.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.388+0000 7f8bb0400640 1 -- 172.21.15.44:0/1922523399 wait complete. 2024-07-17T08:31:18.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.389+0000 7f8bb0400640 1 Processor -- start 2024-07-17T08:31:18.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.390+0000 7f8bb0400640 1 -- start start 2024-07-17T08:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.390+0000 7f8bb0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b90004050 0x7f8ba8072ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.390+0000 7f8bb0400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bb0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 0x7f8ba80cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bb0400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba815c470 con 0x7f8ba814d8d0 2024-07-17T08:31:18.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bb0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba815c2f0 con 0x7f8b90004050 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bb0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba815c5f0 con 0x7f8ba814ff20 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35574/0 (socket says 172.21.15.44:35574) 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 -- 172.21.15.44:0/3855277519 learned_addr learned my addr 172.21.15.44:0/3855277519 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bada00640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b90004050 0x7f8ba8072ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bae400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 0x7f8ba80cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 -- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 msgr2=0x7f8ba80cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 0x7f8ba80cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 -- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b90004050 msgr2=0x7f8ba8072ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b90004050 0x7f8ba8072ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bad000640 1 -- 172.21.15.44:0/3855277519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba81f3c20 con 0x7f8ba814d8d0 2024-07-17T08:31:18.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.391+0000 7f8bae400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 0x7f8ba80cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8bad000640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8b98031040 tx=0x7f8b98006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98032030 con 0x7f8ba814d8d0 2024-07-17T08:31:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b98003af0 con 0x7f8ba814d8d0 2024-07-17T08:31:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9802e440 con 0x7f8ba814d8d0 2024-07-17T08:31:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba81f3ec0 con 0x7f8ba814d8d0 2024-07-17T08:31:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.392+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81f4480 con 0x7f8ba814d8d0 2024-07-17T08:31:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.393+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba80cd2e0 con 0x7f8ba814d8d0 2024-07-17T08:31:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.395+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 114) ==== 98960+0+0 (secure 0 0 0) 0x7f8b98035020 con 0x7f8ba814d8d0 2024-07-17T08:31:18.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.396+0000 7f8ba5600640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 0x7f8b8c079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.396+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 7337+0+0 (secure 0 0 0) 0x7f8b980b8fc0 con 0x7f8ba814d8d0 2024-07-17T08:31:18.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.397+0000 7f8bada00640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 0x7f8b8c079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.398+0000 7f8bada00640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 0x7f8b8c079210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c000c00 tx=0x7f8b9c006b50 comp rx=0 tx=0).ready entity=mgr.7777 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:18.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.400+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8b98047030 con 0x7f8ba814d8d0 2024-07-17T08:31:18.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.637+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 --> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ba8150300 con 0x7f8b8c076d50 2024-07-17T08:31:18.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.653+0000 7f8ba5600640 1 -- 172.21.15.44:0/3855277519 <== mgr.7777 v2:172.21.15.175:6834/1237700724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+376 (secure 0 0 0) 0x7f8ba8150300 con 0x7f8b8c076d50 2024-07-17T08:31:18.655 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:18.655 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:15:42", "first": "2024-07-17T08:16:00", "last": "2024-07-17T08:31:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2024-07-17T08:31:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 msgr2=0x7f8b8c079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 0x7f8b8c079210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c000c00 tx=0x7f8b9c006b50 comp rx=0 tx=0).stop 2024-07-17T08:31:18.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 msgr2=0x7f8ba814f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8b98031040 tx=0x7f8b98006cc0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 shutdown_connections 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f8b8c076d50 0x7f8b8c079210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba814ff20 0x7f8ba80cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ba814d8d0 0x7f8ba814f9e0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 --2- 172.21.15.44:0/3855277519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b90004050 0x7f8ba8072ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 >> 172.21.15.44:0/3855277519 conn(0x7f8ba80b0eb0 msgr2=0x7f8ba80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:18.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 shutdown_connections 2024-07-17T08:31:18.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.656+0000 7f8bb0400640 1 -- 172.21.15.44:0/3855277519 wait complete. 2024-07-17T08:31:18.678 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:15:42", "first": "2024-07-17T08:16:00", "last": "2024-07-17T08:31:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2024-07-17T08:31:18.678 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2024-07-17T00:00:00', 'created': '2024-07-17T08:15:42', 'first': '2024-07-17T08:16:00', 'last': '2024-07-17T08:31:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2024-07-17T08:31:18.678 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2024-07-17T08:31:18.678 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2024-07-17T08:31:18.678 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2024-07-17T08:31:18.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 -- 172.21.15.44:0/2935020630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 msgr2=0x7f9714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 --2- 172.21.15.44:0/2935020630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f9714004c80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f971c001eb0 tx=0x7f9714005830 comp rx=0 tx=0).stop 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f972fe00640 1 -- 172.21.15.44:0/2935020630 reap_dead start 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 -- 172.21.15.44:0/2935020630 shutdown_connections 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 -- 172.21.15.44:0/2935020630 >> 172.21.15.44:0/2935020630 conn(0x7f97300cf0d0 msgr2=0x7f97300cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 -- 172.21.15.44:0/2935020630 shutdown_connections 2024-07-17T08:31:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.889+0000 7f9735000640 1 -- 172.21.15.44:0/2935020630 wait complete. 2024-07-17T08:31:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.891+0000 7f9735000640 1 Processor -- start 2024-07-17T08:31:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.891+0000 7f9735000640 1 -- start start 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 0x7f97301e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97301e6600 0x7f9730141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f973006d5a0 con 0x7f9714004800 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f973006d420 con 0x7f97300cded0 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f9735000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f973006d720 con 0x7f97301e6600 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.892+0000 7f972f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45066/0 (socket says 172.21.15.44:45066) 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 -- 172.21.15.44:0/3774842713 learned_addr learned my addr 172.21.15.44:0/3774842713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 -- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97301e6600 msgr2=0x7f9730141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97301e6600 0x7f9730141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 -- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 msgr2=0x7f97301e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972ea00640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 0x7f97301e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 0x7f97301e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 -- 172.21.15.44:0/3774842713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301e6b10 con 0x7f9714004800 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972ea00640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 0x7f97301e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.893+0000 7f972f400640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9720010950 tx=0x7f9720010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.894+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972000eda0 con 0x7f9714004800 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.894+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9720004030 con 0x7f9714004800 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.894+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97301e6df0 con 0x7f9714004800 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.894+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720004460 con 0x7f9714004800 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.894+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97300cb6b0 con 0x7f9714004800 2024-07-17T08:31:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.895+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973006dc60 con 0x7f9714004800 2024-07-17T08:31:18.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.896+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 114) ==== 98960+0+0 (secure 0 0 0) 0x7f97200035f0 con 0x7f9714004800 2024-07-17T08:31:18.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.898+0000 7f9727400640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 0x7f97100791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:18.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.898+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 7337+0+0 (secure 0 0 0) 0x7f9720099af0 con 0x7f9714004800 2024-07-17T08:31:18.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.899+0000 7f972ea00640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 0x7f97100791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:18.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.900+0000 7f972ea00640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 0x7f97100791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f971c033000 tx=0x7f973006f8b0 comp rx=0 tx=0).ready entity=mgr.7777 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:18.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:18.902+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f97200626a0 con 0x7f9714004800 2024-07-17T08:31:19.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.138+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 --> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f97300cffe0 con 0x7f9710076d20 2024-07-17T08:31:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.332+0000 7f9727400640 1 -- 172.21.15.44:0/3774842713 <== mgr.7777 v2:172.21.15.175:6834/1237700724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+55 (secure 0 0 0) 0x7f97300cffe0 con 0x7f9710076d20 2024-07-17T08:31:19.334 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /snap_test_dir1/test_restart 2024-07-17T08:31:19.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 msgr2=0x7f97100791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:19.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 0x7f97100791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f971c033000 tx=0x7f973006f8b0 comp rx=0 tx=0).stop 2024-07-17T08:31:19.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 msgr2=0x7f97300cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9720010950 tx=0x7f9720010e10 comp rx=0 tx=0).stop 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 shutdown_connections 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f9710076d20 0x7f97100791e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97301e6600 0x7f9730141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97300cded0 0x7f97301e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 --2- 172.21.15.44:0/3774842713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9714004800 0x7f97300cd990 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.335+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 >> 172.21.15.44:0/3774842713 conn(0x7f97300cf0d0 msgr2=0x7f97300680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.336+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 shutdown_connections 2024-07-17T08:31:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:19.336+0000 7f9735000640 1 -- 172.21.15.44:0/3774842713 wait complete. 2024-07-17T08:31:19.359 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2024-07-17T08:31:19.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.359 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2024-07-17T08:31:19.410 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_16_00_UTC 2024-07-17T08:31:19.410 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_17_00_UTC 2024-07-17T08:31:19.410 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_18_00_UTC 2024-07-17T08:31:19.410 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_19_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_20_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_21_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_22_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_23_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_24_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_25_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_26_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_27_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_28_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_29_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_30_00_UTC 2024-07-17T08:31:19.411 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_31_00_UTC 2024-07-17T08:31:19.412 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_16_00_UTC 2024-07-17T08:31:19.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.413 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_16_00_UTC) 2024-07-17T08:31:19.469 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_17_00_UTC 2024-07-17T08:31:19.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.470 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_17_00_UTC) 2024-07-17T08:31:19.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_18_00_UTC 2024-07-17T08:31:19.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.528 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_18_00_UTC) 2024-07-17T08:31:19.590 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_19_00_UTC 2024-07-17T08:31:19.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.591 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_19_00_UTC) 2024-07-17T08:31:19.645 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_20_00_UTC 2024-07-17T08:31:19.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.645 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_20_00_UTC) 2024-07-17T08:31:19.699 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_21_00_UTC 2024-07-17T08:31:19.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.699 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_21_00_UTC) 2024-07-17T08:31:19.755 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_22_00_UTC 2024-07-17T08:31:19.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.755 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_22_00_UTC) 2024-07-17T08:31:19.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_23_00_UTC 2024-07-17T08:31:19.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.815 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_23_00_UTC) 2024-07-17T08:31:19.876 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_24_00_UTC 2024-07-17T08:31:19.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.876 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_24_00_UTC) 2024-07-17T08:31:19.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_25_00_UTC 2024-07-17T08:31:19.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:19.936 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_25_00_UTC) 2024-07-17T08:31:20.000 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_26_00_UTC 2024-07-17T08:31:20.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.001 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_26_00_UTC) 2024-07-17T08:31:20.063 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_27_00_UTC 2024-07-17T08:31:20.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.064 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_27_00_UTC) 2024-07-17T08:31:20.129 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_28_00_UTC 2024-07-17T08:31:20.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.130 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_28_00_UTC) 2024-07-17T08:31:20.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_29_00_UTC 2024-07-17T08:31:20.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.194 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_29_00_UTC) 2024-07-17T08:31:20.259 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_30_00_UTC 2024-07-17T08:31:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.259 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_30_00_UTC) 2024-07-17T08:31:20.317 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_31_00_UTC 2024-07-17T08:31:20.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.317 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2024-07-17-08_31_00_UTC) 2024-07-17T08:31:20.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:20.376 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2024-07-17T08:31:20.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:31:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 -- 172.21.15.44:0/774186991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc86815d350 msgr2=0x7fc868161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 --2- 172.21.15.44:0/774186991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc86815d350 0x7fc868161770 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009830 tx=0x7fc85c02a480 comp rx=0 tx=0).stop 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 -- 172.21.15.44:0/774186991 shutdown_connections 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 --2- 172.21.15.44:0/774186991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8680d1440 0x7fc8680d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 --2- 172.21.15.44:0/774186991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc86815d350 0x7fc868161770 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 -- 172.21.15.44:0/774186991 >> 172.21.15.44:0/774186991 conn(0x7fc8680d04c0 msgr2=0x7fc8680d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 -- 172.21.15.44:0/774186991 shutdown_connections 2024-07-17T08:31:20.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.639+0000 7fc86ea00640 1 -- 172.21.15.44:0/774186991 wait complete. 2024-07-17T08:31:20.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.641+0000 7fc86ea00640 1 Processor -- start 2024-07-17T08:31:20.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.641+0000 7fc86ea00640 1 -- start start 2024-07-17T08:31:20.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:20.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 0x7fc8680b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc86815d350 0x7fc8681467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc86806ead0 con 0x7fc8680cdca0 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc86806e950 con 0x7fc8680d1440 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8680ce150 con 0x7fc86815d350 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45080/0 (socket says 172.21.15.44:45080) 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.642+0000 7fc86d600640 1 -- 172.21.15.44:0/467518081 learned_addr learned my addr 172.21.15.44:0/467518081 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86e000640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc86815d350 0x7fc8681467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86cc00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 0x7fc8680b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:20.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 -- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc86815d350 msgr2=0x7fc8681467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc86815d350 0x7fc8681467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 -- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 msgr2=0x7fc8680b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 0x7fc8680b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 -- 172.21.15.44:0/467518081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680b6ef0 con 0x7fc8680cdca0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86cc00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 0x7fc8680b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.643+0000 7fc86d600640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009950 tx=0x7fc85c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.644+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c031d60 con 0x7fc8680cdca0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.644+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc85c033e10 con 0x7fc8680cdca0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.644+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c005260 con 0x7fc8680cdca0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.644+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8680b71d0 con 0x7fc8680cdca0 2024-07-17T08:31:20.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.644+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8680b7790 con 0x7fc8680cdca0 2024-07-17T08:31:20.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.645+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8680d1820 con 0x7fc8680cdca0 2024-07-17T08:31:20.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.646+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 114) ==== 98960+0+0 (secure 0 0 0) 0x7fc85c005400 con 0x7fc8680cdca0 2024-07-17T08:31:20.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.647+0000 7fc865600640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 0x7fc84c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:20.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.648+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 7337+0+0 (secure 0 0 0) 0x7fc85c0b9be0 con 0x7fc8680cdca0 2024-07-17T08:31:20.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.648+0000 7fc86cc00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 0x7fc84c0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.649+0000 7fc86cc00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 0x7fc84c0791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc86814a030 tx=0x7fc858009500 comp rx=0 tx=0).ready entity=mgr.7777 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:20.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.652+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc85c03f030 con 0x7fc8680cdca0 2024-07-17T08:31:20.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:20.916+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fc8680b29f0 con 0x7fc8680cdca0 2024-07-17T08:31:21.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.238+0000 7fc865600640 1 -- 172.21.15.44:0/467518081 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v115) ==== 93+0+0 (secure 0 0 0) 0x7fc85c031770 con 0x7fc8680cdca0 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 msgr2=0x7fc84c0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 0x7fc84c0791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc86814a030 tx=0x7fc858009500 comp rx=0 tx=0).stop 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 msgr2=0x7fc8680b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009950 tx=0x7fc85c033c20 comp rx=0 tx=0).stop 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 shutdown_connections 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc84c076d20 0x7fc84c0791e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc86815d350 0x7fc8681467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8680d1440 0x7fc8680b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 --2- 172.21.15.44:0/467518081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8680cdca0 0x7fc8680b6910 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.243+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 >> 172.21.15.44:0/467518081 conn(0x7fc8680d04c0 msgr2=0x7fc8680cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.244+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 shutdown_connections 2024-07-17T08:31:21.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.244+0000 7fc86ea00640 1 -- 172.21.15.44:0/467518081 wait complete. 2024-07-17T08:31:21.270 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:31:21.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:31:21.375 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:31:21.455 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:31:21.455 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:31:21.455 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:21.455 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:21.504 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:21.504 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:21.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 -- 172.21.15.44:0/3702837013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 msgr2=0x7fe94815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 --2- 172.21.15.44:0/3702837013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe94815d510 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fe9480d0de0 tx=0x7fe93802a4d0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 -- 172.21.15.44:0/3702837013 shutdown_connections 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 --2- 172.21.15.44:0/3702837013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe948161be0 0x7fe9480b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 --2- 172.21.15.44:0/3702837013 >> v2:172.21.15.175:3300/0 conn(0x7fe9480ce690 0x7fe9480cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 --2- 172.21.15.44:0/3702837013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe94815d510 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 -- 172.21.15.44:0/3702837013 >> 172.21.15.44:0/3702837013 conn(0x7fe9480b4ff0 msgr2=0x7fe9480b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 -- 172.21.15.44:0/3702837013 shutdown_connections 2024-07-17T08:31:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.792+0000 7fe94da00640 1 -- 172.21.15.44:0/3702837013 wait complete. 2024-07-17T08:31:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.794+0000 7fe94da00640 1 Processor -- start 2024-07-17T08:31:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.795+0000 7fe94da00640 1 -- start start 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9480ce690 0x7fe9481f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948161be0 0x7fe94815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe94806c7a0 con 0x7fe94815d130 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe94806c620 con 0x7fe948161be0 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe94da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe94806c920 con 0x7fe9480ce690 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe947400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe947400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45110/0 (socket says 172.21.15.44:45110) 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.796+0000 7fe947400640 1 -- 172.21.15.44:0/2001241941 learned_addr learned my addr 172.21.15.44:0/2001241941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947e00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9480ce690 0x7fe9481f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe94d000640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948161be0 0x7fe94815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9480ce690 msgr2=0x7fe9481f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9480ce690 0x7fe9481f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948161be0 msgr2=0x7fe94815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948161be0 0x7fe94815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 -- 172.21.15.44:0/2001241941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480add40 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.797+0000 7fe947400640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe93400d770 tx=0x7fe93400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.798+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934017070 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.798+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe934009d20 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.798+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934005280 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.798+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94814ec50 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.798+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94814fa60 con 0x7fe94815d130 2024-07-17T08:31:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.799+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9480d3120 con 0x7fe94815d130 2024-07-17T08:31:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.801+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7fe934005420 con 0x7fe94815d130 2024-07-17T08:31:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.802+0000 7fe93fe00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 0x7fe928079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.802+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 7337+0+0 (secure 0 0 0) 0x7fe9340991b0 con 0x7fe94815d130 2024-07-17T08:31:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.802+0000 7fe947e00640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 msgr2=0x7fe928079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.803+0000 7fe947e00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 0x7fe928079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:21.808+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe9480d3120 con 0x7fe94815d130 2024-07-17T08:31:22.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.004+0000 7fe947e00640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 msgr2=0x7fe928079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:22.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.004+0000 7fe947e00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 0x7fe928079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.033+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9480cfdf0 con 0x7fe94815d130 2024-07-17T08:31:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.034+0000 7fe93fe00640 1 -- 172.21.15.44:0/2001241941 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v192) ==== 81+0+953 (secure 0 0 0) 0x7fe9480add40 con 0x7fe94815d130 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/256403424 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3253828084 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1323057578 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2945977477 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/4042077236 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.036 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2650508982 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2961426739 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2162593019 2024-07-17T09:19:41.823667+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1442043557 2024-07-17T09:19:41.824057+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/2760073115 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/2760073115 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4121236115 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/818614358 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/657674782 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/4042077236 2024-07-18T08:26:21.647492+0000 2024-07-17T08:31:22.037 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1169415444 2024-07-18T08:14:43.235053+0000 2024-07-17T08:31:22.038 INFO:teuthology.orchestra.run.smithi044.stderr:listed 16 entries 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.036+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 msgr2=0x7fe928079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.036+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 0x7fe928079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 msgr2=0x7fe9480acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe93400d770 tx=0x7fe93400dc30 comp rx=0 tx=0).stop 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 shutdown_connections 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fe928076cd0 0x7fe928079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe948161be0 0x7fe94815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe94815d130 0x7fe9480acfa0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 --2- 172.21.15.44:0/2001241941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9480ce690 0x7fe9481f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 >> 172.21.15.44:0/2001241941 conn(0x7fe9480b4ff0 msgr2=0x7fe9480b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:22.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 shutdown_connections 2024-07-17T08:31:22.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.037+0000 7fe94da00640 1 -- 172.21.15.44:0/2001241941 wait complete. 2024-07-17T08:31:22.057 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:31:22.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:22.058 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:31:22.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:31:22.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:22.133 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:31:22.174 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:31:22.174 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:31:22.174 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:22.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:31:22.214 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:31:22.214 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:22.214 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:22.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:31:22.265 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:31:22.265 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:31:22.265 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:22.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:31:22.324 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:22.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 -- 172.21.15.44:0/2611585722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b8c80 msgr2=0x7f7b900b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:22.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 --2- 172.21.15.44:0/2611585722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b8c80 0x7f7b900b4ec0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009be0 tx=0x7f7b8002a970 comp rx=0 tx=0).stop 2024-07-17T08:31:22.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 -- 172.21.15.44:0/2611585722 shutdown_connections 2024-07-17T08:31:22.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 --2- 172.21.15.44:0/2611585722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b8c80 0x7f7b900b4ec0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 -- 172.21.15.44:0/2611585722 >> 172.21.15.44:0/2611585722 conn(0x7f7b900cfec0 msgr2=0x7f7b900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:22.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 -- 172.21.15.44:0/2611585722 shutdown_connections 2024-07-17T08:31:22.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.508+0000 7f7b95800640 1 -- 172.21.15.44:0/2611585722 wait complete. 2024-07-17T08:31:22.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.510+0000 7f7b95800640 1 Processor -- start 2024-07-17T08:31:22.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.510+0000 7f7b95800640 1 -- start start 2024-07-17T08:31:22.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900b8c80 0x7f7b90154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b900ce5c0 0x7f7b90154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9015e2d0 con 0x7f7b900b5400 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900b9270 con 0x7f7b900b8c80 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b95800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9015e450 con 0x7f7b900ce5c0 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b8fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45124/0 (socket says 172.21.15.44:45124) 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b8fe00640 1 -- 172.21.15.44:0/3388039096 learned_addr learned my addr 172.21.15.44:0/3388039096 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.511+0000 7f7b94e00640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b900ce5c0 0x7f7b90154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8f400640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900b8c80 0x7f7b90154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b900ce5c0 msgr2=0x7f7b90154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:22.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b900ce5c0 0x7f7b90154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900b8c80 msgr2=0x7f7b90154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:22.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900b8c80 0x7f7b90154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 -- 172.21.15.44:0/3388039096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901f7cb0 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b8fe00640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009b30 tx=0x7f7b80003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.512+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80003e20 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.513+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b80004040 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.513+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004490 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.513+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b901f7f90 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.513+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901f8550 con 0x7f7b900b5400 2024-07-17T08:31:22.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.514+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9015e8b0 con 0x7f7b900b5400 2024-07-17T08:31:22.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.515+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7f7b80004630 con 0x7f7b900b5400 2024-07-17T08:31:22.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.516+0000 7f7b87e00640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 0x7f7b6c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:22.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.516+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 7425+0+0 (secure 0 0 0) 0x7f7b800ba5a0 con 0x7f7b900b5400 2024-07-17T08:31:22.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.516+0000 7f7b8f400640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 msgr2=0x7f7b6c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:22.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.516+0000 7f7b8f400640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 0x7f7b6c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.520+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7b9015e8b0 con 0x7f7b900b5400 2024-07-17T08:31:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.717+0000 7f7b8f400640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 msgr2=0x7f7b6c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:22.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.717+0000 7f7b8f400640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 0x7f7b6c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:22.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.769+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b900b4ec0 con 0x7f7b900b5400 2024-07-17T08:31:22.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.771+0000 7f7b87e00640 1 -- 172.21.15.44:0/3388039096 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7f7b80083290 con 0x7f7b900b5400 2024-07-17T08:31:22.773 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:22.773 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:31:22.773 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.774+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 msgr2=0x7f7b6c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.774+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 0x7f7b6c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.774+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 msgr2=0x7f7b900c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.774+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009b30 tx=0x7f7b80003970 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 shutdown_connections 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f7b6c076cd0 0x7f7b6c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b900ce5c0 0x7f7b90154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b900b8c80 0x7f7b90154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 --2- 172.21.15.44:0/3388039096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b900b5400 0x7f7b900c8760 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:22.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 >> 172.21.15.44:0/3388039096 conn(0x7f7b900cfec0 msgr2=0x7f7b900b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 shutdown_connections 2024-07-17T08:31:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:22.775+0000 7f7b95800640 1 -- 172.21.15.44:0/3388039096 wait complete. 2024-07-17T08:31:22.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.075+0000 7f6a97800640 1 -- 172.21.15.44:0/272183057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900ac220 msgr2=0x7f6a900ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.075+0000 7f6a97800640 1 --2- 172.21.15.44:0/272183057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900ac220 0x7f6a900ac690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009830 tx=0x7f6a7c02a5c0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.076+0000 7f6a97800640 1 -- 172.21.15.44:0/272183057 shutdown_connections 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.076+0000 7f6a97800640 1 --2- 172.21.15.44:0/272183057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900ac220 0x7f6a900ac690 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.076+0000 7f6a97800640 1 -- 172.21.15.44:0/272183057 >> 172.21.15.44:0/272183057 conn(0x7f6a900b0fb0 msgr2=0x7f6a900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.076+0000 7f6a97800640 1 -- 172.21.15.44:0/272183057 shutdown_connections 2024-07-17T08:31:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.076+0000 7f6a97800640 1 -- 172.21.15.44:0/272183057 wait complete. 2024-07-17T08:31:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.077+0000 7f6a97800640 1 Processor -- start 2024-07-17T08:31:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.077+0000 7f6a97800640 1 -- start start 2024-07-17T08:31:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a900ac220 0x7f6a90145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900cd690 0x7f6a9009cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900ce370 con 0x7f6a901522c0 2024-07-17T08:31:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900ce1f0 con 0x7f6a900cd690 2024-07-17T08:31:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.078+0000 7f6a97800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a900ce4f0 con 0x7f6a900ac220 2024-07-17T08:31:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45132/0 (socket says 172.21.15.44:45132) 2024-07-17T08:31:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 -- 172.21.15.44:0/3656064040 learned_addr learned my addr 172.21.15.44:0/3656064040 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a8fe00640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900cd690 0x7f6a9009cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a94e00640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a900ac220 0x7f6a90145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a900ac220 msgr2=0x7f6a90145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a900ac220 0x7f6a90145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900cd690 msgr2=0x7f6a9009cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900cd690 0x7f6a9009cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.079+0000 7f6a95800640 1 -- 172.21.15.44:0/3656064040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a901f7b70 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a95800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800cb10 tx=0x7f6a8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88016020 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a8800ad80 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a901f7e50 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88012680 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.080+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901454e0 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.081+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a900ac690 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.082+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7f6a88019020 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.083+0000 7f6a8cc00640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 0x7f6a70079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.083+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 7425+0+0 (secure 0 0 0) 0x7f6a88099130 con 0x7f6a901522c0 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.083+0000 7f6a94e00640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 msgr2=0x7f6a70079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.083+0000 7f6a94e00640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 0x7f6a70079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.088+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6a88020020 con 0x7f6a901522c0 2024-07-17T08:31:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.284+0000 7f6a94e00640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 msgr2=0x7f6a70079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.284+0000 7f6a94e00640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 0x7f6a70079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.345+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a9006f260 con 0x7f6a901522c0 2024-07-17T08:31:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.346+0000 7f6a8cc00640 1 -- 172.21.15.44:0/3656064040 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7f6a88061750 con 0x7f6a901522c0 2024-07-17T08:31:23.348 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:23.348 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:31:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 msgr2=0x7f6a70079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 0x7f6a70079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 msgr2=0x7f6a901f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800cb10 tx=0x7f6a8800a060 comp rx=0 tx=0).stop 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 shutdown_connections 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f6a70076cd0 0x7f6a70079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a901522c0 0x7f6a901f70b0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a900cd690 0x7f6a9009cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 --2- 172.21.15.44:0/3656064040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a900ac220 0x7f6a90145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 >> 172.21.15.44:0/3656064040 conn(0x7f6a900b0fb0 msgr2=0x7f6a9015a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 shutdown_connections 2024-07-17T08:31:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.349+0000 7f6a97800640 1 -- 172.21.15.44:0/3656064040 wait complete. 2024-07-17T08:31:23.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 -- 172.21.15.44:0/1738771412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340b8c80 msgr2=0x7fa6340b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 --2- 172.21.15.44:0/1738771412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340b8c80 0x7fa6340b4ec0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa628009900 tx=0x7fa62802a560 comp rx=0 tx=0).stop 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 -- 172.21.15.44:0/1738771412 shutdown_connections 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 --2- 172.21.15.44:0/1738771412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340ce5c0 0x7fa63406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 --2- 172.21.15.44:0/1738771412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340b8c80 0x7fa6340b4ec0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 -- 172.21.15.44:0/1738771412 >> 172.21.15.44:0/1738771412 conn(0x7fa6340cfec0 msgr2=0x7fa6340d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 -- 172.21.15.44:0/1738771412 shutdown_connections 2024-07-17T08:31:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.554+0000 7fa63b000640 1 -- 172.21.15.44:0/1738771412 wait complete. 2024-07-17T08:31:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.556+0000 7fa63b000640 1 Processor -- start 2024-07-17T08:31:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.556+0000 7fa63b000640 1 -- start start 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6340b5400 0x7fa6340d2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340b8c80 0x7fa6341f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406e780 con 0x7fa6340ce5c0 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406e600 con 0x7fa6340b5400 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa63406e900 con 0x7fa6340b8c80 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45156/0 (socket says 172.21.15.44:45156) 2024-07-17T08:31:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.557+0000 7fa63a600640 1 -- 172.21.15.44:0/3551278748 learned_addr learned my addr 172.21.15.44:0/3551278748 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa639c00640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6340b5400 0x7fa6340d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa639200640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340b8c80 0x7fa6341f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340b8c80 msgr2=0x7fa6341f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340b8c80 0x7fa6341f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6340b5400 msgr2=0x7fa6340d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6340b5400 0x7fa6340d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 -- 172.21.15.44:0/3551278748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341f4d50 con 0x7fa6340ce5c0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63a600640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c780 tx=0x7fa62c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c017070 con 0x7fa6340ce5c0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa62c005070 con 0x7fa6340ce5c0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341f5020 con 0x7fa6340ce5c0 2024-07-17T08:31:23.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.558+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c005500 con 0x7fa6340ce5c0 2024-07-17T08:31:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.559+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6340b6550 con 0x7fa6340ce5c0 2024-07-17T08:31:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.560+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63415e570 con 0x7fa6340ce5c0 2024-07-17T08:31:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.561+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7fa62c01a070 con 0x7fa6340ce5c0 2024-07-17T08:31:23.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.562+0000 7fa631600640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:23.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.562+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 7409+0+0 (secure 0 0 0) 0x7fa62c098270 con 0x7fa6340ce5c0 2024-07-17T08:31:23.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.562+0000 7fa639c00640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 msgr2=0x7fa614079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:23.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.562+0000 7fa639c00640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:23.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.567+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa62c060f70 con 0x7fa6340ce5c0 2024-07-17T08:31:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.763+0000 7fa639c00640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 msgr2=0x7fa614079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.763+0000 7fa639c00640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:23.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.778+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6340b68f0 con 0x7fa6340ce5c0 2024-07-17T08:31:23.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.781+0000 7fa631600640 1 -- 172.21.15.44:0/3551278748 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+18858 (secure 0 0 0) 0x7fa62c0691b0 con 0x7fa6340ce5c0 2024-07-17T08:31:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:23.784 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":194,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:23.249169+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:14:19.191608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-07-17T08:14:20.205318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":194,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/256403424":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/3253828084":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1323057578":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2945977477":"2024-07-18T08:26:21.647492+0000","172.21.15.175:6837/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2650508982":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2961426739":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2162593019":"2024-07-17T09:19:41.823667+0000","172.21.15.175:0/1442043557":"2024-07-17T09:19:41.824057+0000","172.21.15.175:6837/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:6834/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/4121236115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/818614358":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/657674782":"2024-07-18T08:14:43.235053+0000","172.21.15.175:6834/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1169415444":"2024-07-18T08:14:43.235053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.784+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 msgr2=0x7fa614079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 msgr2=0x7fa6341f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c780 tx=0x7fa62c00cc40 comp rx=0 tx=0).stop 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 shutdown_connections 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6340ce5c0 0x7fa6341f4810 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6340b8c80 0x7fa6341f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 --2- 172.21.15.44:0/3551278748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6340b5400 0x7fa6340d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 >> 172.21.15.44:0/3551278748 conn(0x7fa6340cfec0 msgr2=0x7fa6340b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 shutdown_connections 2024-07-17T08:31:23.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:23.785+0000 7fa63b000640 1 -- 172.21.15.44:0/3551278748 wait complete. 2024-07-17T08:31:23.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:31:23.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.003+0000 7fc9ff200640 1 -- 172.21.15.44:0/766618775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b8c80 msgr2=0x7fc9f80b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.003+0000 7fc9ff200640 1 --2- 172.21.15.44:0/766618775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b8c80 0x7fc9f80b4ec0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009860 tx=0x7fc9ec02a540 comp rx=0 tx=0).stop 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 -- 172.21.15.44:0/766618775 shutdown_connections 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 --2- 172.21.15.44:0/766618775 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80b5400 0x7fc9f815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 --2- 172.21.15.44:0/766618775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b8c80 0x7fc9f80b4ec0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 -- 172.21.15.44:0/766618775 >> 172.21.15.44:0/766618775 conn(0x7fc9f80cfec0 msgr2=0x7fc9f80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 -- 172.21.15.44:0/766618775 shutdown_connections 2024-07-17T08:31:24.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.004+0000 7fc9ff200640 1 -- 172.21.15.44:0/766618775 wait complete. 2024-07-17T08:31:24.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.005+0000 7fc9ff200640 1 Processor -- start 2024-07-17T08:31:24.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 -- start start 2024-07-17T08:31:24.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f80b8c80 0x7fc9f814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80ce5c0 0x7fc9f814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f806e0c0 con 0x7fc9f80b5400 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80b92f0 con 0x7fc9f80b8c80 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.006+0000 7fc9ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f806e240 con 0x7fc9f80ce5c0 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45174/0 (socket says 172.21.15.44:45174) 2024-07-17T08:31:24.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 -- 172.21.15.44:0/467766431 learned_addr learned my addr 172.21.15.44:0/467766431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:24.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fe800640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80ce5c0 0x7fc9f814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fd400640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f80b8c80 0x7fc9f814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80ce5c0 msgr2=0x7fc9f814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80ce5c0 0x7fc9f814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f80b8c80 msgr2=0x7fc9f814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f80b8c80 0x7fc9f814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.007+0000 7fc9fde00640 1 -- 172.21.15.44:0/467766431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f8149980 con 0x7fc9f80b5400 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9fde00640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009980 tx=0x7fc9ec032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec040070 con 0x7fc9f80b5400 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ec003af0 con 0x7fc9f80b5400 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f8149c60 con 0x7fc9f80b5400 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec045460 con 0x7fc9f80b5400 2024-07-17T08:31:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.008+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f814a220 con 0x7fc9f80b5400 2024-07-17T08:31:24.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.009+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f80cefe0 con 0x7fc9f80b5400 2024-07-17T08:31:24.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.011+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7fc9ec039040 con 0x7fc9f80b5400 2024-07-17T08:31:24.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.012+0000 7fc9f2000640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 0x7fc9dc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.013+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 7409+0+0 (secure 0 0 0) 0x7fc9ec0ba250 con 0x7fc9f80b5400 2024-07-17T08:31:24.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.013+0000 7fc9fd400640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 msgr2=0x7fc9dc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:24.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.013+0000 7fc9fd400640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 0x7fc9dc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:24.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.017+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc9f80cefe0 con 0x7fc9f80b5400 2024-07-17T08:31:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.214+0000 7fc9fd400640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 msgr2=0x7fc9dc079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:24.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.215+0000 7fc9fd400640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 0x7fc9dc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:24.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.257+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9f806e6f0 con 0x7fc9f80b5400 2024-07-17T08:31:24.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.258+0000 7fc9f2000640 1 -- 172.21.15.44:0/467766431 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7fc9ec082e40 con 0x7fc9f80b5400 2024-07-17T08:31:24.260 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:24.260 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:31:24.261 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 msgr2=0x7fc9dc079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 0x7fc9dc079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 msgr2=0x7fc9f81596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009980 tx=0x7fc9ec032d50 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 shutdown_connections 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc9dc076cd0 0x7fc9dc079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9f80ce5c0 0x7fc9f814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f80b8c80 0x7fc9f814ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 --2- 172.21.15.44:0/467766431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9f80b5400 0x7fc9f81596d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 >> 172.21.15.44:0/467766431 conn(0x7fc9f80cfec0 msgr2=0x7fc9f80b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:24.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 shutdown_connections 2024-07-17T08:31:24.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.261+0000 7fc9ff200640 1 -- 172.21.15.44:0/467766431 wait complete. 2024-07-17T08:31:24.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:24.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 -- 172.21.15.44:0/124185884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc35006e090 msgr2=0x7fc35006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 --2- 172.21.15.44:0/124185884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc35006e090 0x7fc35006e500 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc348009830 tx=0x7fc34802a520 comp rx=0 tx=0).stop 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 -- 172.21.15.44:0/124185884 shutdown_connections 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 --2- 172.21.15.44:0/124185884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc35006e090 0x7fc35006e500 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 --2- 172.21.15.44:0/124185884 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3500ac3e0 0x7fc3500ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 -- 172.21.15.44:0/124185884 >> 172.21.15.44:0/124185884 conn(0x7fc3500b0fb0 msgr2=0x7fc3500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 -- 172.21.15.44:0/124185884 shutdown_connections 2024-07-17T08:31:24.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.472+0000 7fc357000640 1 -- 172.21.15.44:0/124185884 wait complete. 2024-07-17T08:31:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.474+0000 7fc357000640 1 Processor -- start 2024-07-17T08:31:24.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.474+0000 7fc357000640 1 -- start start 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35006e090 0x7fc3501e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc350153a50 0x7fc35015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc350153fb0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc350153e30 con 0x7fc350153a50 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc357000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc35006f710 con 0x7fc35006e090 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.475+0000 7fc347400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45190/0 (socket says 172.21.15.44:45190) 2024-07-17T08:31:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 -- 172.21.15.44:0/2464591148 learned_addr learned my addr 172.21.15.44:0/2464591148 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc34fe00640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35006e090 0x7fc3501e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc355000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc350153a50 0x7fc35015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35006e090 msgr2=0x7fc3501e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35006e090 0x7fc3501e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc350153a50 msgr2=0x7fc35015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc350153a50 0x7fc35015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 -- 172.21.15.44:0/2464591148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35015c8b0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.476+0000 7fc347400640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3380097d0 tx=0x7fc338009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.477+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33800e8b0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.477+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc338019070 con 0x7fc3500ac3e0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.477+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33800edc0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.477+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35015cb90 con 0x7fc3500ac3e0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.477+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35015b320 con 0x7fc3500ac3e0 2024-07-17T08:31:24.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.478+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35015cdd0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.480+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7fc33801d410 con 0x7fc3500ac3e0 2024-07-17T08:31:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.480+0000 7fc34cc00640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 0x7fc32c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.481+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 7409+0+0 (secure 0 0 0) 0x7fc338098ff0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.481+0000 7fc34fe00640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 msgr2=0x7fc32c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:24.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.481+0000 7fc34fe00640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 0x7fc32c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.483+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc35015cdd0 con 0x7fc3500ac3e0 2024-07-17T08:31:24.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.682+0000 7fc34fe00640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 msgr2=0x7fc32c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:24.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.683+0000 7fc34fe00640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 0x7fc32c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.739+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3500ac840 con 0x7fc3500ac3e0 2024-07-17T08:31:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.740+0000 7fc34cc00640 1 -- 172.21.15.44:0/2464591148 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7fc338022020 con 0x7fc3500ac3e0 2024-07-17T08:31:24.742 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:24.742 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":97,"btime":"2024-07-17T08:14:24:795751+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:14:21.223311+0000","modified":"2024-07-17T08:14:24.308288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9183":{"gid":9183,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1832122622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1832122622},{"type":"v1","addr":"172.21.15.44:6835","nonce":1832122622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9183,"qdb_cluster":[9183]},"id":11}]} 2024-07-17T08:31:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 97 2024-07-17T08:31:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 msgr2=0x7fc32c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:24.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 0x7fc32c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 msgr2=0x7fc35014f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3380097d0 tx=0x7fc338009c90 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 shutdown_connections 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7fc32c076cd0 0x7fc32c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc350153a50 0x7fc35015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500ac3e0 0x7fc35014f5d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 --2- 172.21.15.44:0/2464591148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35006e090 0x7fc3501e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.742+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 >> 172.21.15.44:0/2464591148 conn(0x7fc3500b0fb0 msgr2=0x7fc35013eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.743+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 shutdown_connections 2024-07-17T08:31:24.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.743+0000 7fc357000640 1 -- 172.21.15.44:0/2464591148 wait complete. 2024-07-17T08:31:24.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:24.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 -- 172.21.15.44:0/73604124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4a8004800 msgr2=0x7ff4a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/73604124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4a8004800 0x7ff4a8004c80 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0016c0 tx=0x7ff4a8005830 comp rx=0 tx=0).stop 2024-07-17T08:31:24.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c5c00640 1 -- 172.21.15.44:0/73604124 reap_dead start 2024-07-17T08:31:24.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 -- 172.21.15.44:0/73604124 shutdown_connections 2024-07-17T08:31:24.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 -- 172.21.15.44:0/73604124 >> 172.21.15.44:0/73604124 conn(0x7ff4c00af940 msgr2=0x7ff4c00afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:24.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 -- 172.21.15.44:0/73604124 shutdown_connections 2024-07-17T08:31:24.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.946+0000 7ff4c7c00640 1 -- 172.21.15.44:0/73604124 wait complete. 2024-07-17T08:31:24.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.948+0000 7ff4c7c00640 1 Processor -- start 2024-07-17T08:31:24.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.948+0000 7ff4c7c00640 1 -- start start 2024-07-17T08:31:24.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a8004800 0x7ff4c01f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c0150f30 0x7ff4c014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c006f5e0 con 0x7ff4c01534a0 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00cf860 con 0x7ff4c0150f30 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c006f760 con 0x7ff4a8004800 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45196/0 (socket says 172.21.15.44:45196) 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.949+0000 7ff4c5c00640 1 -- 172.21.15.44:0/2307037091 learned_addr learned my addr 172.21.15.44:0/2307037091 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5200640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a8004800 0x7ff4c01f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4bfe00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c0150f30 0x7ff4c014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a8004800 msgr2=0x7ff4c01f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a8004800 0x7ff4c01f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c0150f30 msgr2=0x7ff4c014c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c0150f30 0x7ff4c014c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 -- 172.21.15.44:0/2307037091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0151830 con 0x7ff4c01534a0 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.950+0000 7ff4c5c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80015c0 tx=0x7ff4b800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.951+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8015070 con 0x7ff4c01534a0 2024-07-17T08:31:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.951+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b800cb00 con 0x7ff4c01534a0 2024-07-17T08:31:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.951+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8011460 con 0x7ff4c01534a0 2024-07-17T08:31:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.951+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c0151b10 con 0x7ff4c01534a0 2024-07-17T08:31:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.951+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c014f140 con 0x7ff4c01534a0 2024-07-17T08:31:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.952+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c0152030 con 0x7ff4c01534a0 2024-07-17T08:31:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.954+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7ff4b8018070 con 0x7ff4c01534a0 2024-07-17T08:31:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.954+0000 7ff4bcc00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 0x7ff4a00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.955+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 7409+0+0 (secure 0 0 0) 0x7ff4b8098eb0 con 0x7ff4c01534a0 2024-07-17T08:31:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.955+0000 7ff4c5200640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 msgr2=0x7ff4a00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.955+0000 7ff4c5200640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 0x7ff4a00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:24.959+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4b80614f0 con 0x7ff4c01534a0 2024-07-17T08:31:25.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.156+0000 7ff4c5200640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 msgr2=0x7ff4a00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:25.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.157+0000 7ff4c5200640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 0x7ff4a00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:25.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.168+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4c006fcc0 con 0x7ff4c01534a0 2024-07-17T08:31:25.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.171+0000 7ff4bcc00640 1 -- 172.21.15.44:0/2307037091 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+18858 (secure 0 0 0) 0x7ff4b8065a40 con 0x7ff4c01534a0 2024-07-17T08:31:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:25.173 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":194,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:23.249169+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:14:19.191608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-07-17T08:14:20.205318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":194,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/256403424":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/3253828084":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1323057578":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2945977477":"2024-07-18T08:26:21.647492+0000","172.21.15.175:6837/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2650508982":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2961426739":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2162593019":"2024-07-17T09:19:41.823667+0000","172.21.15.175:0/1442043557":"2024-07-17T09:19:41.824057+0000","172.21.15.175:6837/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:6834/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/4121236115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/818614358":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/657674782":"2024-07-18T08:14:43.235053+0000","172.21.15.175:6834/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1169415444":"2024-07-18T08:14:43.235053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 msgr2=0x7ff4a00791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 0x7ff4a00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 msgr2=0x7ff4c01538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80015c0 tx=0x7ff4b800eb80 comp rx=0 tx=0).stop 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 shutdown_connections 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7ff4a0076d00 0x7ff4a00791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c01534a0 0x7ff4c01538f0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4c0150f30 0x7ff4c014c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 --2- 172.21.15.44:0/2307037091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4a8004800 0x7ff4c01f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 >> 172.21.15.44:0/2307037091 conn(0x7ff4c00af940 msgr2=0x7ff4c00ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 shutdown_connections 2024-07-17T08:31:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.174+0000 7ff4c7c00640 1 -- 172.21.15.44:0/2307037091 wait complete. 2024-07-17T08:31:25.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 -- 172.21.15.44:0/2246757004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 msgr2=0x7f549c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 --2- 172.21.15.44:0/2246757004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c0b2200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5488005fd0 tx=0x7f548802a540 comp rx=0 tx=0).stop 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 -- 172.21.15.44:0/2246757004 shutdown_connections 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 --2- 172.21.15.44:0/2246757004 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c0b2200 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 --2- 172.21.15.44:0/2246757004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 0x7f549c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 -- 172.21.15.44:0/2246757004 >> 172.21.15.44:0/2246757004 conn(0x7f549c0b4ff0 msgr2=0x7f549c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 -- 172.21.15.44:0/2246757004 shutdown_connections 2024-07-17T08:31:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.399+0000 7f54a2e00640 1 -- 172.21.15.44:0/2246757004 wait complete. 2024-07-17T08:31:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.401+0000 7f54a2e00640 1 Processor -- start 2024-07-17T08:31:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.401+0000 7f54a2e00640 1 -- start start 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 0x7f549c152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f549c0d1a10 con 0x7f549c0ce690 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f549c0d1890 con 0x7f549c161be0 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f549c0d1b90 con 0x7f549c15d130 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45198/0 (socket says 172.21.15.44:45198) 2024-07-17T08:31:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a1a00640 1 -- 172.21.15.44:0/197541787 learned_addr learned my addr 172.21.15.44:0/197541787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a1000640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 0x7f549c152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.402+0000 7f54a2400640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 msgr2=0x7f549c152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 0x7f549c152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 msgr2=0x7f549c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 -- 172.21.15.44:0/197541787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c15dec0 con 0x7f549c0ce690 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a2400640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a1a00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f549000ea60 tx=0x7f549000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000cde0 con 0x7f549c0ce690 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5490004020 con 0x7f549c0ce690 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c15e1a0 con 0x7f549c0ce690 2024-07-17T08:31:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490004450 con 0x7f549c0ce690 2024-07-17T08:31:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.403+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c15e760 con 0x7f549c0ce690 2024-07-17T08:31:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.405+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c0b2200 con 0x7f549c0ce690 2024-07-17T08:31:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.406+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 115) ==== 98943+0+0 (secure 0 0 0) 0x7f54900035f0 con 0x7f549c0ce690 2024-07-17T08:31:25.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.407+0000 7f5499600640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 0x7f547c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.407+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 7409+0+0 (secure 0 0 0) 0x7f54900995b0 con 0x7f549c0ce690 2024-07-17T08:31:25.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.407+0000 7f54a1000640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 msgr2=0x7f547c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:25.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.407+0000 7f54a1000640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 0x7f547c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.411+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f54900621a0 con 0x7f549c0ce690 2024-07-17T08:31:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.609+0000 7f54a1000640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 msgr2=0x7f547c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1237700724 2024-07-17T08:31:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.609+0000 7f54a1000640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 0x7f547c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:25.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.623+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 116) ==== 98540+0+0 (secure 0 0 0) 0x7f549005dd30 con 0x7f549c0ce690 2024-07-17T08:31:25.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.623+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 msgr2=0x7f547c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:25.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.623+0000 7f5499600640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 0x7f547c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.666+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f549c0d0e10 con 0x7f549c0ce690 2024-07-17T08:31:25.668 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:31:25.667+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:31:25.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.669+0000 7f5499600640 1 -- 172.21.15.44:0/197541787 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f549006a3e0 con 0x7f549c0ce690 2024-07-17T08:31:25.671 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:31:25.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 msgr2=0x7f549c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f549000ea60 tx=0x7f549000ef20 comp rx=0 tx=0).stop 2024-07-17T08:31:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 shutdown_connections 2024-07-17T08:31:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:6834/1237700724,v1:172.21.15.175:6837/1237700724] conn(0x7f547c076cd0 0x7f547c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f549c161be0 0x7f549c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f549c15d130 0x7f549c152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 --2- 172.21.15.44:0/197541787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f549c0ce690 0x7f549c15d510 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.673+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 >> 172.21.15.44:0/197541787 conn(0x7f549c0b4ff0 msgr2=0x7f549c0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:25.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.674+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 shutdown_connections 2024-07-17T08:31:25.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.674+0000 7f54a2e00640 1 -- 172.21.15.44:0/197541787 wait complete. 2024-07-17T08:31:25.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30b8c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30b8c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30b8c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30b8c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.702 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:25.700+0000 7f30b8c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:25.781 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:31:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.871+0000 7f6927e00640 1 -- 172.21.15.44:0/1754064863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c0053c0 msgr2=0x7f690c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.871+0000 7f6927e00640 1 --2- 172.21.15.44:0/1754064863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c0053c0 0x7f690c00b9c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f6920001dc0 tx=0x7f690c005830 comp rx=0 tx=0).stop 2024-07-17T08:31:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.872+0000 7f6927e00640 1 -- 172.21.15.44:0/1754064863 shutdown_connections 2024-07-17T08:31:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.872+0000 7f6927e00640 1 --2- 172.21.15.44:0/1754064863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f690c0053c0 0x7f690c00b9c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.872+0000 7f6927e00640 1 -- 172.21.15.44:0/1754064863 >> 172.21.15.44:0/1754064863 conn(0x7f69280d04c0 msgr2=0x7f69280d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.872+0000 7f6927e00640 1 -- 172.21.15.44:0/1754064863 shutdown_connections 2024-07-17T08:31:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.872+0000 7f6927e00640 1 -- 172.21.15.44:0/1754064863 wait complete. 2024-07-17T08:31:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.873+0000 7f6927e00640 1 Processor -- start 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.873+0000 7f6927e00640 1 -- start start 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 0x7f69280ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69281f7e70 0x7f69281f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69280d1bb0 con 0x7f69280cac60 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69280d1a30 con 0x7f690c0053c0 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.874+0000 7f6927e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69280ce130 con 0x7f69281f7e70 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45228/0 (socket says 172.21.15.44:45228) 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 -- 172.21.15.44:0/3583859371 learned_addr learned my addr 172.21.15.44:0/3583859371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f6926a00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 0x7f69280ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f6927400640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69281f7e70 0x7f69281f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 -- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69281f7e70 msgr2=0x7f69281f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69281f7e70 0x7f69281f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 -- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 msgr2=0x7f69280ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 0x7f69280ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 -- 172.21.15.44:0/3583859371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69280ccb90 con 0x7f69280cac60 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f6926a00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 0x7f69280ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.875+0000 7f691fe00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f691000f7d0 tx=0x7f691000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.876+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691000bcb0 con 0x7f69280cac60 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.876+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6910004030 con 0x7f69280cac60 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.876+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910004480 con 0x7f69280cac60 2024-07-17T08:31:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.876+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69280cce00 con 0x7f69280cac60 2024-07-17T08:31:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.876+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69280cd3c0 con 0x7f69280cac60 2024-07-17T08:31:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.877+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6928161770 con 0x7f69280cac60 2024-07-17T08:31:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.878+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 116) ==== 98540+0+0 (secure 0 0 0) 0x7f69100035f0 con 0x7f69280cac60 2024-07-17T08:31:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.879+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 7795+0+0 (secure 0 0 0) 0x7f6910098ec0 con 0x7f69280cac60 2024-07-17T08:31:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:25.884+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6910061870 con 0x7f69280cac60 2024-07-17T08:31:26.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.137+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f69280b29f0 con 0x7f69280cac60 2024-07-17T08:31:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.633+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 117) ==== 98667+0+0 (secure 0 0 0) 0x7f691005d1e0 con 0x7f69280cac60 2024-07-17T08:31:26.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.633+0000 7f691ea00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 0x7f690407f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:26.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.635+0000 7f6926a00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 0x7f690407f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:26.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.639+0000 7f6926a00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 0x7f690407f7e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f69281468b0 tx=0x7f69180067d0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.673+0000 7f691ea00640 1 -- 172.21.15.44:0/3583859371 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) ==== 104+0+0 (secure 0 0 0) 0x7f691006bb10 con 0x7f69280cac60 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 msgr2=0x7f690407f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 0x7f690407f7e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f69281468b0 tx=0x7f69180067d0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 msgr2=0x7f69281f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f691000f7d0 tx=0x7f691000fc90 comp rx=0 tx=0).stop 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 shutdown_connections 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69281f7e70 0x7f69281f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69280cac60 0x7f69281f7930 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f690407d3f0 0x7f690407f7e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 --2- 172.21.15.44:0/3583859371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c0053c0 0x7f69280ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 >> 172.21.15.44:0/3583859371 conn(0x7f69280d04c0 msgr2=0x7f69280d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:26.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 shutdown_connections 2024-07-17T08:31:26.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.677+0000 7f6927e00640 1 -- 172.21.15.44:0/3583859371 wait complete. 2024-07-17T08:31:26.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:31:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 -- 172.21.15.44:0/2946772450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 msgr2=0x7f992c0ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 --2- 172.21.15.44:0/2946772450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c0ac3f0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f992402b910 comp rx=0 tx=0).stop 2024-07-17T08:31:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 -- 172.21.15.44:0/2946772450 shutdown_connections 2024-07-17T08:31:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 --2- 172.21.15.44:0/2946772450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c0ac3f0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 --2- 172.21.15.44:0/2946772450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c0ae110 0x7f992c0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 -- 172.21.15.44:0/2946772450 >> 172.21.15.44:0/2946772450 conn(0x7f992c0cf050 msgr2=0x7f992c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 -- 172.21.15.44:0/2946772450 shutdown_connections 2024-07-17T08:31:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.909+0000 7f9932c00640 1 -- 172.21.15.44:0/2946772450 wait complete. 2024-07-17T08:31:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.911+0000 7f9932c00640 1 Processor -- start 2024-07-17T08:31:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.911+0000 7f9932c00640 1 -- start start 2024-07-17T08:31:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c06f460 0x7f992c149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f992c0ae110 0x7f992c071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f992c06eaa0 con 0x7f992c0abfa0 2024-07-17T08:31:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f992c06e920 con 0x7f992c0ae110 2024-07-17T08:31:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9932c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f992c06ec20 con 0x7f992c06f460 2024-07-17T08:31:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.912+0000 7f9930e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45238/0 (socket says 172.21.15.44:45238) 2024-07-17T08:31:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 -- 172.21.15.44:0/13930295 learned_addr learned my addr 172.21.15.44:0/13930295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9932200640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f992c0ae110 0x7f992c071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9931800640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c06f460 0x7f992c149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 -- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c06f460 msgr2=0x7f992c149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c06f460 0x7f992c149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 -- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f992c0ae110 msgr2=0x7f992c071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f992c0ae110 0x7f992c071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 -- 172.21.15.44:0/13930295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c14dd90 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.913+0000 7f9930e00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f991800c970 tx=0x7f991800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.914+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918007bb0 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.914+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9918007e40 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.914+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991800f690 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.914+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c14e070 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.914+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c14e630 con 0x7f992c0abfa0 2024-07-17T08:31:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.915+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c0ac3d0 con 0x7f992c0abfa0 2024-07-17T08:31:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.916+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 117) ==== 98667+0+0 (secure 0 0 0) 0x7f991800f830 con 0x7f992c0abfa0 2024-07-17T08:31:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.917+0000 7f9929600640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 0x7f990c078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.917+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 7795+0+0 (secure 0 0 0) 0x7f9918098f30 con 0x7f992c0abfa0 2024-07-17T08:31:26.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.918+0000 7f9931800640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 0x7f990c078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:26.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.919+0000 7f9931800640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 0x7f990c078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f992c0b09e0 tx=0x7f9920009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:26.922+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9918014030 con 0x7f992c0abfa0 2024-07-17T08:31:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.131+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f992c1e7e50 con 0x7f992c0abfa0 2024-07-17T08:31:27.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.677+0000 7f9929600640 1 -- 172.21.15.44:0/13930295 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v197) ==== 181+0+0 (secure 0 0 0) 0x7f9918061e40 con 0x7f992c0abfa0 2024-07-17T08:31:27.679 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:31:27.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.683+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 msgr2=0x7f990c078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:27.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.683+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 0x7f990c078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f992c0b09e0 tx=0x7f9920009040 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.683+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 msgr2=0x7f992c14ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.683+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f991800c970 tx=0x7f991800ce30 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 shutdown_connections 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f990c076a00 0x7f990c078ec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f992c0ae110 0x7f992c071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f992c0abfa0 0x7f992c14ee30 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 --2- 172.21.15.44:0/13930295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c06f460 0x7f992c149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 >> 172.21.15.44:0/13930295 conn(0x7f992c0cf050 msgr2=0x7f992c15b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:27.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 shutdown_connections 2024-07-17T08:31:27.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.684+0000 7f9932c00640 1 -- 172.21.15.44:0/13930295 wait complete. 2024-07-17T08:31:27.711 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.913+0000 7fe0ad000640 1 -- 172.21.15.44:0/2028876445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 msgr2=0x7fe0a8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.913+0000 7fe0ad000640 1 --2- 172.21.15.44:0/2028876445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a8070290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe098005fd0 tx=0x7fe09802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 -- 172.21.15.44:0/2028876445 shutdown_connections 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 --2- 172.21.15.44:0/2028876445 >> v2:172.21.15.175:3300/0 conn(0x7fe0a80cddf0 0x7fe0a80ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 --2- 172.21.15.44:0/2028876445 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a8070290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 -- 172.21.15.44:0/2028876445 >> 172.21.15.44:0/2028876445 conn(0x7fe0a806ce80 msgr2=0x7fe0a806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 -- 172.21.15.44:0/2028876445 shutdown_connections 2024-07-17T08:31:27.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.914+0000 7fe0ad000640 1 -- 172.21.15.44:0/2028876445 wait complete. 2024-07-17T08:31:27.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.916+0000 7fe0ad000640 1 Processor -- start 2024-07-17T08:31:27.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.916+0000 7fe0ad000640 1 -- start start 2024-07-17T08:31:27.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:27.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a80cddf0 0x7fe0a806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a81f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a80ce960 con 0x7fe0a8072e90 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a80ce7e0 con 0x7fe0a80cddf0 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.917+0000 7fe0ad000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a80ceae0 con 0x7fe0a815f060 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45248/0 (socket says 172.21.15.44:45248) 2024-07-17T08:31:27.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 -- 172.21.15.44:0/1350555390 learned_addr learned my addr 172.21.15.44:0/1350555390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 -- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 msgr2=0x7fe0a81f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a6800640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a81f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a81f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 -- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a80cddf0 msgr2=0x7fe0a806f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:27.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a80cddf0 0x7fe0a806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.918+0000 7fe0a5e00640 1 -- 172.21.15.44:0/1350555390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a806fad0 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe0a5e00640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe098000c00 tx=0x7fe0980035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09803e030 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe098003ae0 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a80b49e0 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098035040 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.919+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a80b4fa0 con 0x7fe0a8072e90 2024-07-17T08:31:27.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.920+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a80ce2a0 con 0x7fe0a8072e90 2024-07-17T08:31:27.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.921+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 118) ==== 98773+0+0 (secure 0 0 0) 0x7fe098041030 con 0x7fe0a8072e90 2024-07-17T08:31:27.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.923+0000 7fe09e000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 0x7fe088078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:27.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.923+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(197..197 src has 1..197) ==== 7357+0+0 (secure 0 0 0) 0x7fe0980b8cc0 con 0x7fe0a8072e90 2024-07-17T08:31:27.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.924+0000 7fe0a5400640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 0x7fe088078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:27.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.925+0000 7fe0a5400640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 0x7fe088078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe094007fd0 tx=0x7fe0940079a0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:27.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:27.927+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0a806fad0 con 0x7fe0a8072e90 2024-07-17T08:31:28.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.139+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe0a8068e20 con 0x7fe0a8072e90 2024-07-17T08:31:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.693+0000 7fe09e000640 1 -- 172.21.15.44:0/1350555390 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v198) ==== 169+0+0 (secure 0 0 0) 0x7fe098081b60 con 0x7fe0a8072e90 2024-07-17T08:31:28.695 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:31:28.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.696+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 msgr2=0x7fe088078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 0x7fe088078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe094007fd0 tx=0x7fe0940079a0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 msgr2=0x7fe0a80b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:28.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe098000c00 tx=0x7fe0980035f0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 shutdown_connections 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0a815f060 0x7fe0a81f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fe088076b20 0x7fe088078fe0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe094007fd0 tx=0x7fe0940079a0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0a80cddf0 0x7fe0a806f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 --2- 172.21.15.44:0/1350555390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0a8072e90 0x7fe0a80b4340 secure :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe098000c00 tx=0x7fe0980035f0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.697+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 >> 172.21.15.44:0/1350555390 conn(0x7fe0a806ce80 msgr2=0x7fe0a81451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.699+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 shutdown_connections 2024-07-17T08:31:28.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.699+0000 7fe0ad000640 1 -- 172.21.15.44:0/1350555390 wait complete. 2024-07-17T08:31:28.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:31:28.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 -- 172.21.15.44:0/1620672368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 msgr2=0x7f8cd0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/1620672368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cd0004800 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400bb20 tx=0x7f8cd0005830 comp rx=0 tx=0).stop 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8ceb200640 1 -- 172.21.15.44:0/1620672368 reap_dead start 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 -- 172.21.15.44:0/1620672368 shutdown_connections 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 -- 172.21.15.44:0/1620672368 >> 172.21.15.44:0/1620672368 conn(0x7f8cec0b4ff0 msgr2=0x7f8cec0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 -- 172.21.15.44:0/1620672368 shutdown_connections 2024-07-17T08:31:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.922+0000 7f8cf0a00640 1 -- 172.21.15.44:0/1620672368 wait complete. 2024-07-17T08:31:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.924+0000 7f8cf0a00640 1 Processor -- start 2024-07-17T08:31:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.924+0000 7f8cf0a00640 1 -- start start 2024-07-17T08:31:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cec15d720 0x7f8cec15f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cec15dba0 0x7f8cec0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0d15c0 con 0x7f8cd0004140 2024-07-17T08:31:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0b27d0 con 0x7f8cec15d720 2024-07-17T08:31:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.925+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec0d1740 con 0x7f8cec15dba0 2024-07-17T08:31:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41130/0 (socket says 172.21.15.44:41130) 2024-07-17T08:31:28.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 -- 172.21.15.44:0/4235607141 learned_addr learned my addr 172.21.15.44:0/4235607141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:28.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 -- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cec15dba0 msgr2=0x7f8cec0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:28.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cec15dba0 0x7f8cec0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 -- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cec15d720 msgr2=0x7f8cec15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:28.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cec15d720 0x7f8cec15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:28.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 -- 172.21.15.44:0/4235607141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec160330 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.926+0000 7f8cea800640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009a30 tx=0x7f8cdc009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.927+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004d20 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.927+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cdc00faf0 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.927+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc005020 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.927+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec160610 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.927+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec1f4030 con 0x7f8cd0004140 2024-07-17T08:31:28.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.928+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cec0ca590 con 0x7f8cd0004140 2024-07-17T08:31:28.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.929+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 118) ==== 98773+0+0 (secure 0 0 0) 0x7f8cdc0051c0 con 0x7f8cd0004140 2024-07-17T08:31:28.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.930+0000 7f8ce3400640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 0x7f8cc8079010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:28.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.931+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7f8cdc0994b0 con 0x7f8cd0004140 2024-07-17T08:31:28.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.931+0000 7f8ce1e00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 0x7f8cc8079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:28.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.932+0000 7f8ce1e00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 0x7f8cc8079010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4036000 tx=0x7f8cd4004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:28.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:28.935+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8cec0ca590 con 0x7f8cd0004140 2024-07-17T08:31:29.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.152+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f8cec15d510 con 0x7f8cd0004140 2024-07-17T08:31:29.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.158+0000 7f8ce3400640 1 -- 172.21.15.44:0/4235607141 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v67) ==== 71+0+0 (secure 0 0 0) 0x7f8cdc0624f0 con 0x7f8cd0004140 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 msgr2=0x7f8cc8079010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 0x7f8cc8079010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4036000 tx=0x7f8cd4004730 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 msgr2=0x7f8cec157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009a30 tx=0x7f8cdc009ef0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 shutdown_connections 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cec15dba0 0x7f8cec0cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f8cc8076b50 0x7f8cc8079010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cec15d720 0x7f8cec15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 --2- 172.21.15.44:0/4235607141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd0004140 0x7f8cec157de0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 >> 172.21.15.44:0/4235607141 conn(0x7f8cec0b4ff0 msgr2=0x7f8cec06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:29.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 shutdown_connections 2024-07-17T08:31:29.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.161+0000 7f8cf0a00640 1 -- 172.21.15.44:0/4235607141 wait complete. 2024-07-17T08:31:29.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 -- 172.21.15.44:0/554114708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf94004e40 msgr2=0x7fdf940052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/554114708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf94004e40 0x7fdf940052c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8001dc0 tx=0x7fdf94005830 comp rx=0 tx=0).stop 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfafe00640 1 -- 172.21.15.44:0/554114708 reap_dead start 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 -- 172.21.15.44:0/554114708 shutdown_connections 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 -- 172.21.15.44:0/554114708 >> 172.21.15.44:0/554114708 conn(0x7fdfb00b0e80 msgr2=0x7fdfb00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 -- 172.21.15.44:0/554114708 shutdown_connections 2024-07-17T08:31:29.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.389+0000 7fdfb4e00640 1 -- 172.21.15.44:0/554114708 wait complete. 2024-07-17T08:31:29.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.391+0000 7fdfb4e00640 1 Processor -- start 2024-07-17T08:31:29.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.391+0000 7fdfb4e00640 1 -- start start 2024-07-17T08:31:29.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 0x7fdfb0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:29.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfb01f2f30 0x7fdfb0073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:29.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:29.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015a2a0 con 0x7fdfb0071fc0 2024-07-17T08:31:29.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb015a120 con 0x7fdf94004e40 2024-07-17T08:31:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00b1fa0 con 0x7fdfb01f2f30 2024-07-17T08:31:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41150/0 (socket says 172.21.15.44:41150) 2024-07-17T08:31:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.392+0000 7fdfafe00640 1 -- 172.21.15.44:0/1965577902 learned_addr learned my addr 172.21.15.44:0/1965577902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:29.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 -- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfb01f2f30 msgr2=0x7fdfb0073190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:29.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfaf400640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 0x7fdfb0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfaea00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfb01f2f30 0x7fdfb0073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfb01f2f30 0x7fdfb0073190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 -- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 msgr2=0x7fdfb0153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 0x7fdfb0153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 -- 172.21.15.44:0/1965577902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb00736d0 con 0x7fdfb0071fc0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfaf400640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 0x7fdfb0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfafe00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdfa802acc0 tx=0x7fdfa8006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80062b0 con 0x7fdfb0071fc0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa8006450 con 0x7fdfb0071fc0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80068a0 con 0x7fdfb0071fc0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.393+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb01f36a0 con 0x7fdfb0071fc0 2024-07-17T08:31:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.394+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01f3c60 con 0x7fdfb0071fc0 2024-07-17T08:31:29.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.395+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb00cf380 con 0x7fdfb0071fc0 2024-07-17T08:31:29.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.396+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 118) ==== 98773+0+0 (secure 0 0 0) 0x7fdfa8003a30 con 0x7fdfb0071fc0 2024-07-17T08:31:29.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.397+0000 7fdfa7400640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 0x7fdf90079010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.397+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7fdfa80b8170 con 0x7fdfb0071fc0 2024-07-17T08:31:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.397+0000 7fdfaf400640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 0x7fdf90079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.399+0000 7fdfaf400640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 0x7fdf90079010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005fd0 tx=0x7fdfa0005d10 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:29.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.401+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdfa8081240 con 0x7fdfb0071fc0 2024-07-17T08:31:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:29.699+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) -- 0x7fdfb0158b10 con 0x7fdfb0071fc0 2024-07-17T08:31:30.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.176+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fdfa807cdd0 con 0x7fdfb0071fc0 2024-07-17T08:31:30.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.699+0000 7fdfa7400640 1 -- 172.21.15.44:0/1965577902 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3131) ==== 165+0+0 (secure 0 0 0) 0x7fdfa80646d0 con 0x7fdfb0071fc0 2024-07-17T08:31:30.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 msgr2=0x7fdf90079010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:30.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 0x7fdf90079010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0005fd0 tx=0x7fdfa0005d10 comp rx=0 tx=0).stop 2024-07-17T08:31:30.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 msgr2=0x7fdfb00cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:30.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdfa802acc0 tx=0x7fdfa8006ca0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 shutdown_connections 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfb0071fc0 0x7fdfb00cbb90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfb01f2f30 0x7fdfb0073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fdf90076b50 0x7fdf90079010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 --2- 172.21.15.44:0/1965577902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf94004e40 0x7fdfb0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 >> 172.21.15.44:0/1965577902 conn(0x7fdfb00b0e80 msgr2=0x7fdfb00ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.703+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 shutdown_connections 2024-07-17T08:31:30.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.704+0000 7fdfb4e00640 1 -- 172.21.15.44:0/1965577902 wait complete. 2024-07-17T08:31:30.729 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2024-07-17T08:31:30.729 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2024-07-17T08:31:30.729 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2024-07-17T08:31:30.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:30.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 -- 172.21.15.44:0/1133907555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540b9060 msgr2=0x7fd95415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:30.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 --2- 172.21.15.44:0/1133907555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540b9060 0x7fd95415d800 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd94c005fd0 tx=0x7fd94c02a620 comp rx=0 tx=0).stop 2024-07-17T08:31:30.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 -- 172.21.15.44:0/1133907555 shutdown_connections 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 --2- 172.21.15.44:0/1133907555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540b9060 0x7fd95415d800 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 --2- 172.21.15.44:0/1133907555 >> v2:172.21.15.175:3300/0 conn(0x7fd9540ce5c0 0x7fd95415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 -- 172.21.15.44:0/1133907555 >> 172.21.15.44:0/1133907555 conn(0x7fd9540cfec0 msgr2=0x7fd9540d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 -- 172.21.15.44:0/1133907555 shutdown_connections 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.944+0000 7fd958a00640 1 -- 172.21.15.44:0/1133907555 wait complete. 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.946+0000 7fd958a00640 1 Processor -- start 2024-07-17T08:31:30.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.946+0000 7fd958a00640 1 -- start start 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540b8c80 0x7fd95415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 0x7fd95414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd95415e0d0 con 0x7fd9540ce5c0 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd95415df50 con 0x7fd9540b9060 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd958a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd95415e250 con 0x7fd9540b8c80 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd953200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd953200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41170/0 (socket says 172.21.15.44:41170) 2024-07-17T08:31:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.947+0000 7fd953200640 1 -- 172.21.15.44:0/1839712416 learned_addr learned my addr 172.21.15.44:0/1839712416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd951e00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 0x7fd95414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 -- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540b8c80 msgr2=0x7fd95415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd952800640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540b8c80 0x7fd95415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540b8c80 0x7fd95415cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 -- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 msgr2=0x7fd95414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 0x7fd95414ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 -- 172.21.15.44:0/1839712416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95414d980 con 0x7fd9540ce5c0 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd951e00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 0x7fd95414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd953200640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c03f070 con 0x7fd9540ce5c0 2024-07-17T08:31:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd94c004da0 con 0x7fd9540ce5c0 2024-07-17T08:31:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c035030 con 0x7fd9540ce5c0 2024-07-17T08:31:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.948+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95414dc60 con 0x7fd9540ce5c0 2024-07-17T08:31:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.949+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95414e220 con 0x7fd9540ce5c0 2024-07-17T08:31:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.950+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9540b5d70 con 0x7fd9540ce5c0 2024-07-17T08:31:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.951+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fd94c042070 con 0x7fd9540ce5c0 2024-07-17T08:31:30.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.952+0000 7fd94aa00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 0x7fd930079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:30.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.952+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7fd94c0b9530 con 0x7fd9540ce5c0 2024-07-17T08:31:30.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.953+0000 7fd952800640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 0x7fd930079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:30.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.954+0000 7fd952800640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 0x7fd930079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd944007fd0 tx=0x7fd944009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:30.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:30.957+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd94c082490 con 0x7fd9540ce5c0 2024-07-17T08:31:31.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.172+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9540d0580 con 0x7fd9540ce5c0 2024-07-17T08:31:31.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.174+0000 7fd94aa00640 1 -- 172.21.15.44:0/1839712416 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) ==== 74+0+15691 (secure 0 0 0) 0x7fd94c081d90 con 0x7fd9540ce5c0 2024-07-17T08:31:31.176 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:31.177 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":198,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:28.690860+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1832122622":"2024-07-18T08:31:25.668354+0000","172.21.15.44:6833/1832122622":"2024-07-18T08:31:25.668354+0000","172.21.15.175:6837/1237700724":"2024-07-18T08:31:25.610430+0000","172.21.15.175:6834/1237700724":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/1367557679":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/14624268":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/256403424":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/3253828084":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1323057578":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2945977477":"2024-07-18T08:26:21.647492+0000","172.21.15.175:6837/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2650508982":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2961426739":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/2162593019":"2024-07-17T09:19:41.823667+0000","172.21.15.175:0/3794138141":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/1442043557":"2024-07-17T09:19:41.824057+0000","172.21.15.175:6837/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:6834/2760073115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/55451808":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/3817507359":"2024-07-18T08:31:25.610430+0000","172.21.15.175:0/4121236115":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/818614358":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/657674782":"2024-07-18T08:14:43.235053+0000","172.21.15.175:0/3708979139":"2024-07-18T08:31:25.610430+0000","172.21.15.175:6834/4042077236":"2024-07-18T08:26:21.647492+0000","172.21.15.175:0/1169415444":"2024-07-18T08:14:43.235053+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 msgr2=0x7fd930079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 0x7fd930079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd944007fd0 tx=0x7fd944009040 comp rx=0 tx=0).stop 2024-07-17T08:31:31.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 msgr2=0x7fd9540af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c0045e0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 shutdown_connections 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9540ce5c0 0x7fd9540af4f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd930076bb0 0x7fd930079070 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9540b9060 0x7fd95414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 --2- 172.21.15.44:0/1839712416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9540b8c80 0x7fd95415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.177+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 >> 172.21.15.44:0/1839712416 conn(0x7fd9540cfec0 msgr2=0x7fd9540b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.178+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 shutdown_connections 2024-07-17T08:31:31.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.178+0000 7fd958a00640 1 -- 172.21.15.44:0/1839712416 wait complete. 2024-07-17T08:31:31.200 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:31:31.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 -- 172.21.15.44:0/2154019139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 msgr2=0x7f161006d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 --2- 172.21.15.44:0/2154019139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f161006d850 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1610070830 tx=0x7f160802b9a0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 -- 172.21.15.44:0/2154019139 shutdown_connections 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 --2- 172.21.15.44:0/2154019139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f161006d850 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 --2- 172.21.15.44:0/2154019139 >> v2:172.21.15.175:3300/0 conn(0x7f161006f4d0 0x7f161015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 -- 172.21.15.44:0/2154019139 >> 172.21.15.44:0/2154019139 conn(0x7f16100cf0d0 msgr2=0x7f16100cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 -- 172.21.15.44:0/2154019139 shutdown_connections 2024-07-17T08:31:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.410+0000 7f1614a00640 1 -- 172.21.15.44:0/2154019139 wait complete. 2024-07-17T08:31:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.412+0000 7f1614a00640 1 Processor -- start 2024-07-17T08:31:31.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.412+0000 7f1614a00640 1 -- start start 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f161006f4d0 0x7f16100cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16101503c0 0x7f16100cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16100b3380 con 0x7f161006d420 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16100b3200 con 0x7f161006f4d0 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f1614a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16100b3500 con 0x7f16101503c0 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f160e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f160e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41176/0 (socket says 172.21.15.44:41176) 2024-07-17T08:31:31.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.413+0000 7f160e800640 1 -- 172.21.15.44:0/2109494332 learned_addr learned my addr 172.21.15.44:0/2109494332 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160de00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f161006f4d0 0x7f16100cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 -- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16101503c0 msgr2=0x7f16100cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16101503c0 0x7f16100cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 -- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f161006f4d0 msgr2=0x7f16100cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f161006f4d0 0x7f16100cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 -- 172.21.15.44:0/2109494332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16100cbf90 con 0x7f161006d420 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f160e800640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f160000c8c0 tx=0x7f160000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600004220 con 0x7f161006d420 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16000079b0 con 0x7f161006d420 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600007de0 con 0x7f161006d420 2024-07-17T08:31:31.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.414+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16100cc270 con 0x7f161006d420 2024-07-17T08:31:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.415+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16100cc830 con 0x7f161006d420 2024-07-17T08:31:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.416+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161006d850 con 0x7f161006d420 2024-07-17T08:31:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.417+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f1600005020 con 0x7f161006d420 2024-07-17T08:31:31.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.418+0000 7f1606a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 0x7f15f0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.418+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7f1600098c20 con 0x7f161006d420 2024-07-17T08:31:31.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.419+0000 7f160de00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 0x7f15f0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.420+0000 7f160de00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 0x7f15f0079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16100995f0 tx=0x7f15fc009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:31.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.422+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1600014030 con 0x7f161006d420 2024-07-17T08:31:31.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.630+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16100cffe0 con 0x7f161006d420 2024-07-17T08:31:31.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.631+0000 7f1606a00640 1 -- 172.21.15.44:0/2109494332 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) ==== 93+0+10 (secure 0 0 0) 0x7f1600061c10 con 0x7f161006d420 2024-07-17T08:31:31.633 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:31:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 msgr2=0x7f15f0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 0x7f15f0079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16100995f0 tx=0x7f15fc009040 comp rx=0 tx=0).stop 2024-07-17T08:31:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 msgr2=0x7f16101f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f160000c8c0 tx=0x7f160000cd80 comp rx=0 tx=0).stop 2024-07-17T08:31:31.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 shutdown_connections 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16101503c0 0x7f16100cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f15f0076bb0 0x7f15f0079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f161006f4d0 0x7f16100cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 --2- 172.21.15.44:0/2109494332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f161006d420 0x7f16101f3c70 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 >> 172.21.15.44:0/2109494332 conn(0x7f16100cf0d0 msgr2=0x7f16100aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 shutdown_connections 2024-07-17T08:31:31.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.634+0000 7f1614a00640 1 -- 172.21.15.44:0/2109494332 wait complete. 2024-07-17T08:31:31.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2024-07-17T08:31:31.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1762542923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 msgr2=0x7fa5c81660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1762542923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c81660d0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c002a8b0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1762542923 shutdown_connections 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1762542923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c81660d0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1762542923 >> 172.21.15.44:0/1762542923 conn(0x7fa5c80b0e80 msgr2=0x7fa5c80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1762542923 shutdown_connections 2024-07-17T08:31:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.874+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1762542923 wait complete. 2024-07-17T08:31:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.876+0000 7fa5c7e00640 1 Processor -- start 2024-07-17T08:31:31.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.876+0000 7fa5c7e00640 1 -- start start 2024-07-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 0x7fa5c8073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 0x7fa5c813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80aea40 con 0x7fa5c8163ce0 2024-07-17T08:31:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80ac270 con 0x7fa5c80cf540 2024-07-17T08:31:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c8166610 con 0x7fa5c80cf920 2024-07-17T08:31:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41194/0 (socket says 172.21.15.44:41194) 2024-07-17T08:31:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.877+0000 7fa5c7400640 1 -- 172.21.15.44:0/1369055464 learned_addr learned my addr 172.21.15.44:0/1369055464 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c6a00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 0x7fa5c8073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c6000640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 0x7fa5c813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 -- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 msgr2=0x7fa5c813cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 0x7fa5c813cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 -- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 msgr2=0x7fa5c8073e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 0x7fa5c8073e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 -- 172.21.15.44:0/1369055464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c813d510 con 0x7fa5c8163ce0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c6000640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 0x7fa5c813cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c6a00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 0x7fa5c8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.878+0000 7fa5c7400640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009cd0 tx=0x7fa5c00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.879+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0031990 con 0x7fa5c8163ce0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.879+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5c0031c20 con 0x7fa5c8163ce0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.879+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00330d0 con 0x7fa5c8163ce0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.879+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c8156120 con 0x7fa5c8163ce0 2024-07-17T08:31:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.879+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81587e0 con 0x7fa5c8163ce0 2024-07-17T08:31:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.880+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c80b3870 con 0x7fa5c8163ce0 2024-07-17T08:31:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.881+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fa5c0033270 con 0x7fa5c8163ce0 2024-07-17T08:31:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.883+0000 7fa5bea00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 0x7fa5a8079090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.883+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7fa5c00b9af0 con 0x7fa5c8163ce0 2024-07-17T08:31:31.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.883+0000 7fa5c6a00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 0x7fa5a8079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.885+0000 7fa5c6a00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 0x7fa5a8079090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8003eb0 tx=0x7fa5b8009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:31.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:31.887+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa5c003f030 con 0x7fa5c8163ce0 2024-07-17T08:31:32.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.177+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) -- 0x7fa5c81407b0 con 0x7fa5c8163ce0 2024-07-17T08:31:32.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.714+0000 7fa5bea00640 1 -- 172.21.15.44:0/1369055464 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3133) ==== 166+0+0 (secure 0 0 0) 0x7fa5c0082a50 con 0x7fa5c8163ce0 2024-07-17T08:31:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 msgr2=0x7fa5a8079090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:32.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 0x7fa5a8079090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8003eb0 tx=0x7fa5b8009040 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 msgr2=0x7fa5c813e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009cd0 tx=0x7fa5c00316d0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 shutdown_connections 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c8163ce0 0x7fa5c813e9d0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5c80cf920 0x7fa5c813cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa5a8076bd0 0x7fa5a8079090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 --2- 172.21.15.44:0/1369055464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5c80cf540 0x7fa5c8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.717+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 >> 172.21.15.44:0/1369055464 conn(0x7fa5c80b0e80 msgr2=0x7fa5c80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.718+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 shutdown_connections 2024-07-17T08:31:32.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.718+0000 7fa5c7e00640 1 -- 172.21.15.44:0/1369055464 wait complete. 2024-07-17T08:31:32.743 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 -- 172.21.15.44:0/3694444167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac48003a30 msgr2=0x7fac48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 --2- 172.21.15.44:0/3694444167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac48003a30 0x7fac48003e10 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fac54001740 tx=0x7fac48005830 comp rx=0 tx=0).stop 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac63200640 1 -- 172.21.15.44:0/3694444167 reap_dead start 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 -- 172.21.15.44:0/3694444167 shutdown_connections 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 -- 172.21.15.44:0/3694444167 >> 172.21.15.44:0/3694444167 conn(0x7fac6406c120 msgr2=0x7fac6406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 -- 172.21.15.44:0/3694444167 shutdown_connections 2024-07-17T08:31:32.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.953+0000 7fac68a00640 1 -- 172.21.15.44:0/3694444167 wait complete. 2024-07-17T08:31:32.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.955+0000 7fac68a00640 1 Processor -- start 2024-07-17T08:31:32.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.955+0000 7fac68a00640 1 -- start start 2024-07-17T08:31:32.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac48003a30 0x7fac6409c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:32.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 0x7fac64074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:32.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:32.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac6415dab0 con 0x7fac64143f10 2024-07-17T08:31:32.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fac6415dc30 con 0x7fac48003a30 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac68a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac6415ddb0 con 0x7fac64153f10 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac63200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac63200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41216/0 (socket says 172.21.15.44:41216) 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.956+0000 7fac63200640 1 -- 172.21.15.44:0/2207164457 learned_addr learned my addr 172.21.15.44:0/2207164457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac5be00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 0x7fac64074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:32.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac62800640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac48003a30 0x7fac6409c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 -- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 msgr2=0x7fac64074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 0x7fac64074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 -- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac48003a30 msgr2=0x7fac6409c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac48003a30 0x7fac6409c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:32.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 -- 172.21.15.44:0/2207164457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6409ca70 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac5be00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 0x7fac64074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.957+0000 7fac63200640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fac6406f540 tx=0x7fac5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.958+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c017070 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.958+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac5c00ad20 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.958+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6409ba90 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.958+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c007330 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.958+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac64144920 con 0x7fac64143f10 2024-07-17T08:31:32.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.959+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac640ced20 con 0x7fac64143f10 2024-07-17T08:31:32.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.960+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fac5c01a070 con 0x7fac64143f10 2024-07-17T08:31:32.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.961+0000 7fac5aa00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 0x7fac40079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:32.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.961+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7fac5c0985d0 con 0x7fac64143f10 2024-07-17T08:31:32.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.962+0000 7fac62800640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 0x7fac40079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:32.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.963+0000 7fac62800640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 0x7fac40079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac54001710 tx=0x7fac54001270 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:32.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:32.965+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fac5c061530 con 0x7fac64143f10 2024-07-17T08:31:33.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.185+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fac640b1730 con 0x7fac64143f10 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.186+0000 7fac5aa00640 1 -- 172.21.15.44:0/2207164457 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v67) ==== 59+0+1998 (secure 0 0 0) 0x7fac5c060e30 con 0x7fac64143f10 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:31:33.188 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:31:33.189 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:31:33.190 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:31:33.190 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:31:33.190 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:31:33.190 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:31:33.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 msgr2=0x7fac40079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 0x7fac40079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac54001710 tx=0x7fac54001270 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 msgr2=0x7fac64144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fac6406f540 tx=0x7fac5c00a060 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 shutdown_connections 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac64143f10 0x7fac64144390 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac64153f10 0x7fac64074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fac40076bb0 0x7fac40079070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 --2- 172.21.15.44:0/2207164457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac48003a30 0x7fac6409c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 >> 172.21.15.44:0/2207164457 conn(0x7fac6406c120 msgr2=0x7fac640acba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 shutdown_connections 2024-07-17T08:31:33.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.189+0000 7fac68a00640 1 -- 172.21.15.44:0/2207164457 wait complete. 2024-07-17T08:31:33.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 -- 172.21.15.44:0/2468988446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880b29e0 msgr2=0x7fa088161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 --2- 172.21.15.44:0/2468988446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880b29e0 0x7fa088161d60 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa088075790 tx=0x7fa08002a940 comp rx=0 tx=0).stop 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 -- 172.21.15.44:0/2468988446 shutdown_connections 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 --2- 172.21.15.44:0/2468988446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880b29e0 0x7fa088161d60 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 --2- 172.21.15.44:0/2468988446 >> v2:172.21.15.175:3300/0 conn(0x7fa0880ce1d0 0x7fa0880ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 -- 172.21.15.44:0/2468988446 >> 172.21.15.44:0/2468988446 conn(0x7fa0880d3700 msgr2=0x7fa0880d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 -- 172.21.15.44:0/2468988446 shutdown_connections 2024-07-17T08:31:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.426+0000 7fa08d000640 1 -- 172.21.15.44:0/2468988446 wait complete. 2024-07-17T08:31:33.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 Processor -- start 2024-07-17T08:31:33.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 -- start start 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 0x7fa08814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0880b6790 0x7fa088152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa088162420 con 0x7fa0880ce1d0 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881622a0 con 0x7fa0880b29e0 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa08d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa088167910 con 0x7fa0880b6790 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa086800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:33.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa086800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41238/0 (socket says 172.21.15.44:41238) 2024-07-17T08:31:33.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.429+0000 7fa086800640 1 -- 172.21.15.44:0/2032393869 learned_addr learned my addr 172.21.15.44:0/2032393869 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa07d400640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0880b6790 0x7fa088152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa085e00640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 0x7fa08814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 -- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0880b6790 msgr2=0x7fa088152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0880b6790 0x7fa088152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 -- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 msgr2=0x7fa08814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 0x7fa08814d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 -- 172.21.15.44:0/2032393869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08814d550 con 0x7fa0880ce1d0 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa085e00640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 0x7fa08814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:33.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa086800640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa08002af30 tx=0x7fa080031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.430+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080030430 con 0x7fa0880ce1d0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.431+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa080004240 con 0x7fa0880ce1d0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.431+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080004690 con 0x7fa0880ce1d0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.431+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa088159d30 con 0x7fa0880ce1d0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.431+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08815a2b0 con 0x7fa0880ce1d0 2024-07-17T08:31:33.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.432+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08806ca60 con 0x7fa0880ce1d0 2024-07-17T08:31:33.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.433+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fa08004c440 con 0x7fa0880ce1d0 2024-07-17T08:31:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.434+0000 7fa07ea00640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 0x7fa064079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.434+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7fa0800ba3b0 con 0x7fa0880ce1d0 2024-07-17T08:31:33.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.435+0000 7fa085e00640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 0x7fa064079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:33.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.437+0000 7fa085e00640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 0x7fa064079070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa088154e30 tx=0x7fa078005dd0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:33.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.438+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa08806ca60 con 0x7fa0880ce1d0 2024-07-17T08:31:33.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.649+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa0880ce5b0 con 0x7fa0880ce1d0 2024-07-17T08:31:33.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.657+0000 7fa07ea00640 1 -- 172.21.15.44:0/2032393869 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v68)=0 v68) ==== 122+0+0 (secure 0 0 0) 0x7fa08814d550 con 0x7fa0880ce1d0 2024-07-17T08:31:33.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 msgr2=0x7fa064079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 0x7fa064079070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa088154e30 tx=0x7fa078005dd0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 msgr2=0x7fa0880c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:33.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa08002af30 tx=0x7fa080031e20 comp rx=0 tx=0).stop 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 shutdown_connections 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0880ce1d0 0x7fa0880c7b80 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0880b6790 0x7fa088152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa064076bb0 0x7fa064079070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 --2- 172.21.15.44:0/2032393869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0880b29e0 0x7fa08814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.660+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 >> 172.21.15.44:0/2032393869 conn(0x7fa0880d3700 msgr2=0x7fa08815b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.661+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 shutdown_connections 2024-07-17T08:31:33.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:33.661+0000 7fa08d000640 1 -- 172.21.15.44:0/2032393869 wait complete. 2024-07-17T08:31:33.681 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:31:33.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:31:33.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:31:33.923 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:31:33.923 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:33.923 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:33.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:31:33.980 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:31:33.980 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:34.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:31:34.040 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:34.040 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:34.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:31:34.097 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:31:34.097 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:34.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:31:34.152 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:31:34.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.375+0000 7feef7000640 1 -- 172.21.15.44:0/390207673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef015f810 msgr2=0x7feef0070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.375+0000 7feef7000640 1 --2- 172.21.15.44:0/390207673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef015f810 0x7feef0070de0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7feedc009830 tx=0x7feedc02a520 comp rx=0 tx=0).stop 2024-07-17T08:31:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 -- 172.21.15.44:0/390207673 shutdown_connections 2024-07-17T08:31:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 --2- 172.21.15.44:0/390207673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef015f810 0x7feef0070de0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 --2- 172.21.15.44:0/390207673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0140070 0x7feef015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 -- 172.21.15.44:0/390207673 >> 172.21.15.44:0/390207673 conn(0x7feef00b3090 msgr2=0x7feef00b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 -- 172.21.15.44:0/390207673 shutdown_connections 2024-07-17T08:31:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.376+0000 7feef7000640 1 -- 172.21.15.44:0/390207673 wait complete. 2024-07-17T08:31:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.377+0000 7feef7000640 1 Processor -- start 2024-07-17T08:31:34.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.378+0000 7feef7000640 1 -- start start 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0071320 0x7feef01f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 0x7feef006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feef0073d70 con 0x7feef0140070 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feef00d1010 con 0x7feef015f810 2024-07-17T08:31:34.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feef0073ef0 con 0x7feef0071320 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41248/0 (socket says 172.21.15.44:41248) 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 -- 172.21.15.44:0/1936310113 learned_addr learned my addr 172.21.15.44:0/1936310113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5c00640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0071320 0x7feef01f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef6600640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 0x7feef006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 -- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0071320 msgr2=0x7feef01f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0071320 0x7feef01f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 -- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 msgr2=0x7feef006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 0x7feef006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.379+0000 7feef5200640 1 -- 172.21.15.44:0/1936310113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef01f3df0 con 0x7feef0140070 2024-07-17T08:31:34.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feef6600640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 0x7feef006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feef5200640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7feedc009950 tx=0x7feedc033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc005c90 con 0x7feef0140070 2024-07-17T08:31:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feedc031e50 con 0x7feef0140070 2024-07-17T08:31:34.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0332f0 con 0x7feef0140070 2024-07-17T08:31:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01f40d0 con 0x7feef0140070 2024-07-17T08:31:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.380+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01f4690 con 0x7feef0140070 2024-07-17T08:31:34.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.381+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0074410 con 0x7feef0140070 2024-07-17T08:31:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.382+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7feedc033490 con 0x7feef0140070 2024-07-17T08:31:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.383+0000 7feeed600640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 0x7feed0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.383+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7feedc0b9b40 con 0x7feef0140070 2024-07-17T08:31:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.384+0000 7feef5c00640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 0x7feed0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.385+0000 7feef5c00640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 0x7feed0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feee4005d80 tx=0x7feee4005c70 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:34.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.388+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feedc03f030 con 0x7feef0140070 2024-07-17T08:31:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.640+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feef0153e60 con 0x7feef0140070 2024-07-17T08:31:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.641+0000 7feeed600640 1 -- 172.21.15.44:0/1936310113 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+3917 (secure 0 0 0) 0x7feedc082b30 con 0x7feef0140070 2024-07-17T08:31:34.642 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:34.642 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":99,"btime":"2024-07-17T08:31:26:671245+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4201401270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4201401270},{"type":"v1","addr":"172.21.15.175:6801","nonce":4201401270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7055,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4149092855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4149092855},{"type":"v1","addr":"172.21.15.175:6836","nonce":4149092855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1726732342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1726732342},{"type":"v1","addr":"172.21.15.44:6834","nonce":1726732342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3584548947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3584548947},{"type":"v1","addr":"172.21.15.44:6835","nonce":3584548947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[]} 2024-07-17T08:31:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 99 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.646+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 msgr2=0x7feed0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.646+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 0x7feed0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feee4005d80 tx=0x7feee4005c70 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 msgr2=0x7feef01f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7feedc009950 tx=0x7feedc033c90 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 shutdown_connections 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7feed0076bb0 0x7feed0079070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef015f810 0x7feef006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef0140070 0x7feef01f35d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 --2- 172.21.15.44:0/1936310113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef0071320 0x7feef01f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 >> 172.21.15.44:0/1936310113 conn(0x7feef00b3090 msgr2=0x7feef015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 shutdown_connections 2024-07-17T08:31:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.647+0000 7feef7000640 1 -- 172.21.15.44:0/1936310113 wait complete. 2024-07-17T08:31:34.668 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:31:34.668 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:31:34.668 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:31:34.668 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:31:34.668 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:31:34.668 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:31:34.669 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:31:34.669 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:31:34.669 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:31:34.669 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:31:34.669 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:31:34.669 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:31:34.669 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:31:34.669 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:31:34.670 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:31:34.670 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:31:34.685 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:31:34.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:31:34.691 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:31:34.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:31:34.693 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:31:34.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:31:34.704 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:31:34.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:31:34.735 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:31:34.736 DEBUG:teuthology.parallel:result is None 2024-07-17T08:31:34.739 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:31:34.740 DEBUG:teuthology.parallel:result is None 2024-07-17T08:31:34.751 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:31:34.752 DEBUG:teuthology.parallel:result is None 2024-07-17T08:31:34.757 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:31:34.758 DEBUG:teuthology.parallel:result is None 2024-07-17T08:31:34.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:31:34.878 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:31:34.879 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:31:34.898 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:31:34.899 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:31:34.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.927+0000 7f4148400640 1 -- 172.21.15.44:0/785547153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 msgr2=0x7f41400d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.927+0000 7f4148400640 1 --2- 172.21.15.44:0/785547153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f41400d0580 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f41400b30e0 tx=0x7f413002a4d0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 -- 172.21.15.44:0/785547153 shutdown_connections 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 --2- 172.21.15.44:0/785547153 >> v2:172.21.15.175:3300/0 conn(0x7f41400d41a0 0x7f41400d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 --2- 172.21.15.44:0/785547153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f41400d0580 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 --2- 172.21.15.44:0/785547153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41400d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 -- 172.21.15.44:0/785547153 >> 172.21.15.44:0/785547153 conn(0x7f414006c830 msgr2=0x7f414006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 -- 172.21.15.44:0/785547153 shutdown_connections 2024-07-17T08:31:34.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.928+0000 7f4148400640 1 -- 172.21.15.44:0/785547153 wait complete. 2024-07-17T08:31:34.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.929+0000 7f4148400640 1 Processor -- start 2024-07-17T08:31:34.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.929+0000 7f4148400640 1 -- start start 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41400d41a0 0x7f41400c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41401f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41400d1ad0 con 0x7f41400ceae0 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41400d1950 con 0x7f41400d41a0 2024-07-17T08:31:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.930+0000 7f4148400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41400d1c50 con 0x7f41401611d0 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41282/0 (socket says 172.21.15.44:41282) 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 -- 172.21.15.44:0/2283809243 learned_addr learned my addr 172.21.15.44:0/2283809243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4146400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41401f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145000640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41400d41a0 0x7f41400c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 -- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 msgr2=0x7f41401f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41401f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 -- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41400d41a0 msgr2=0x7f41400c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41400d41a0 0x7f41400c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 -- 172.21.15.44:0/2283809243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414014d1b0 con 0x7f41400ceae0 2024-07-17T08:31:34.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4146400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41401f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.931+0000 7f4145a00640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f413400b410 tx=0x7f413400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.932+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134015020 con 0x7f41400ceae0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.932+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4134011940 con 0x7f41400ceae0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.932+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134011d90 con 0x7f41400ceae0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.932+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41401f5460 con 0x7f41400ceae0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.932+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401f5180 con 0x7f41400ceae0 2024-07-17T08:31:34.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.933+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41400d2200 con 0x7f41400ceae0 2024-07-17T08:31:34.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.934+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f4134018020 con 0x7f41400ceae0 2024-07-17T08:31:34.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.936+0000 7f413d600640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 0x7f4124079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:34.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.936+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6943+0+0 (secure 0 0 0) 0x7f4134098850 con 0x7f41400ceae0 2024-07-17T08:31:34.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.936+0000 7f4145000640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 0x7f4124079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:34.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.937+0000 7f4145000640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 0x7f4124079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4130033000 tx=0x7f4130004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:34.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:34.939+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f413401f020 con 0x7f41400ceae0 2024-07-17T08:31:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.156+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f41400b95b0 con 0x7f41400ceae0 2024-07-17T08:31:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.750+0000 7f413d600640 1 -- 172.21.15.44:0/2283809243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v200) ==== 97+0+0 (secure 0 0 0) 0x7f41340617b0 con 0x7f41400ceae0 2024-07-17T08:31:35.752 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 msgr2=0x7f4124079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 0x7f4124079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4130033000 tx=0x7f4130004730 comp rx=0 tx=0).stop 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 msgr2=0x7f414014cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f413400b410 tx=0x7f413400b8d0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 shutdown_connections 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41401611d0 0x7f41401f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f4124076bb0 0x7f4124079070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41400d41a0 0x7f41400c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 --2- 172.21.15.44:0/2283809243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ceae0 0x7f414014cc70 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 >> 172.21.15.44:0/2283809243 conn(0x7f414006c830 msgr2=0x7f4140160110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 shutdown_connections 2024-07-17T08:31:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.755+0000 7f4148400640 1 -- 172.21.15.44:0/2283809243 wait complete. 2024-07-17T08:31:35.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.966+0000 7f154f200640 1 -- 172.21.15.44:0/354834272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1550140570 msgr2=0x7f1550140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.966+0000 7f154f200640 1 --2- 172.21.15.44:0/354834272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1550140570 0x7f1550140a00 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1548005fd0 tx=0x7f154802a840 comp rx=0 tx=0).stop 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.966+0000 7f154f200640 1 -- 172.21.15.44:0/354834272 shutdown_connections 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.966+0000 7f154f200640 1 --2- 172.21.15.44:0/354834272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1550140570 0x7f1550140a00 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.967+0000 7f154f200640 1 --2- 172.21.15.44:0/354834272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15500cf920 0x7f1550140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.967+0000 7f154f200640 1 -- 172.21.15.44:0/354834272 >> 172.21.15.44:0/354834272 conn(0x7f15500b0e80 msgr2=0x7f15500b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.967+0000 7f154f200640 1 -- 172.21.15.44:0/354834272 shutdown_connections 2024-07-17T08:31:35.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.967+0000 7f154f200640 1 -- 172.21.15.44:0/354834272 wait complete. 2024-07-17T08:31:35.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.968+0000 7f154f200640 1 Processor -- start 2024-07-17T08:31:35.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 -- start start 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 0x7f15501f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1550140570 0x7f15500cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15500aea10 con 0x7f15500cf920 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15500ac270 con 0x7f15500cf540 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.969+0000 7f154f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1550140f40 con 0x7f1550140570 2024-07-17T08:31:35.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41292/0 (socket says 172.21.15.44:41292) 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 -- 172.21.15.44:0/1379486484 learned_addr learned my addr 172.21.15.44:0/1379486484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154e800640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1550140570 0x7f15500cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 -- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1550140570 msgr2=0x7f15500cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154de00640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 0x7f15501f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1550140570 0x7f15500cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 -- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 msgr2=0x7f15501f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 0x7f15501f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 -- 172.21.15.44:0/1379486484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15500718f0 con 0x7f15500cf920 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154de00640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 0x7f15501f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:35.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.970+0000 7f154d400640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f153800e9f0 tx=0x7f153800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.971+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153800cd80 con 0x7f15500cf920 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.971+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1538010430 con 0x7f15500cf920 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.971+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538010840 con 0x7f15500cf920 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.971+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1550071bd0 con 0x7f15500cf920 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.971+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1550072190 con 0x7f15500cf920 2024-07-17T08:31:35.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.972+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15501413f0 con 0x7f15500cf920 2024-07-17T08:31:35.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.973+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f15380109e0 con 0x7f15500cf920 2024-07-17T08:31:35.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.975+0000 7f1546000640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 0x7f152c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:35.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.975+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5825+0+0 (secure 0 0 0) 0x7f1538098fd0 con 0x7f15500cf920 2024-07-17T08:31:35.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.975+0000 7f154de00640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 0x7f152c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:35.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.977+0000 7f154de00640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 0x7f152c079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1540001220 tx=0x7f1540009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:35.979+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f15380623b0 con 0x7f15500cf920 2024-07-17T08:31:36.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.282+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f155006cc30 con 0x7f15500cf920 2024-07-17T08:31:36.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.283+0000 7f1546000640 1 -- 172.21.15.44:0/1379486484 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v68) ==== 80+0+5304 (secure 0 0 0) 0x7f1538061cb0 con 0x7f15500cf920 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:31:36.284 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:31:36.286 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:31:36.288 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:31:36.290 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:31:36.292 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:31:36.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:31:36.297 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:31:36.298 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 msgr2=0x7f152c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 0x7f152c079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1540001220 tx=0x7f1540009040 comp rx=0 tx=0).stop 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 msgr2=0x7f15500af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f153800e9f0 tx=0x7f153800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 shutdown_connections 2024-07-17T08:31:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1550140570 0x7f15500cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15500cf920 0x7f15500af350 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f152c076bb0 0x7f152c079070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 --2- 172.21.15.44:0/1379486484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15500cf540 0x7f15501f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.285+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 >> 172.21.15.44:0/1379486484 conn(0x7f15500b0e80 msgr2=0x7f155009ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.286+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 shutdown_connections 2024-07-17T08:31:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.286+0000 7f154f200640 1 -- 172.21.15.44:0/1379486484 wait complete. 2024-07-17T08:31:36.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:31:36.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 -- 172.21.15.44:0/3520052398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a341522c0 msgr2=0x7f6a340ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 --2- 172.21.15.44:0/3520052398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a341522c0 0x7f6a340ad790 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6a3415c950 tx=0x7f6a2402a540 comp rx=0 tx=0).stop 2024-07-17T08:31:36.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 -- 172.21.15.44:0/3520052398 shutdown_connections 2024-07-17T08:31:36.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 --2- 172.21.15.44:0/3520052398 >> v2:172.21.15.175:3300/0 conn(0x7f6a340adcd0 0x7f6a340ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 --2- 172.21.15.44:0/3520052398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a341522c0 0x7f6a340ad790 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 -- 172.21.15.44:0/3520052398 >> 172.21.15.44:0/3520052398 conn(0x7f6a340b0fb0 msgr2=0x7f6a340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 -- 172.21.15.44:0/3520052398 shutdown_connections 2024-07-17T08:31:36.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.519+0000 7f6a3a200640 1 -- 172.21.15.44:0/3520052398 wait complete. 2024-07-17T08:31:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.521+0000 7f6a3a200640 1 Processor -- start 2024-07-17T08:31:36.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.521+0000 7f6a3a200640 1 -- start start 2024-07-17T08:31:36.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 0x7f6a340b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a341522c0 0x7f6a341f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a340ce820 con 0x7f6a340adcd0 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a340ce6a0 con 0x7f6a340ac760 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.522+0000 7f6a3a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a340ce9a0 con 0x7f6a341522c0 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41304/0 (socket says 172.21.15.44:41304) 2024-07-17T08:31:36.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 -- 172.21.15.44:0/2292166864 learned_addr learned my addr 172.21.15.44:0/2292166864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a33400640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 0x7f6a340b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 -- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a341522c0 msgr2=0x7f6a341f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a33e00640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a341522c0 0x7f6a341f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:36.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a341522c0 0x7f6a341f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 -- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 msgr2=0x7f6a340b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 0x7f6a340b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a2be00640 1 -- 172.21.15.44:0/2292166864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341f4820 con 0x7f6a340adcd0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.523+0000 7f6a33400640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 0x7f6a340b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a2be00640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d950 tx=0x7f6a1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c017070 con 0x7f6a340adcd0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a1c009d20 con 0x7f6a340adcd0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004280 con 0x7f6a340adcd0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341f7810 con 0x7f6a340adcd0 2024-07-17T08:31:36.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.524+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341f3ae0 con 0x7f6a340adcd0 2024-07-17T08:31:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.525+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a341f38d0 con 0x7f6a340adcd0 2024-07-17T08:31:36.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.526+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f6a1c0035f0 con 0x7f6a340adcd0 2024-07-17T08:31:36.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.527+0000 7f6a2b400640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 0x7f6a10079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.527+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5825+0+0 (secure 0 0 0) 0x7f6a1c0984e0 con 0x7f6a340adcd0 2024-07-17T08:31:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.528+0000 7f6a33400640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 0x7f6a10079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:36.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.529+0000 7f6a33400640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 0x7f6a10079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a24034000 tx=0x7f6a24004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:36.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.537+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6a341f4820 con 0x7f6a340adcd0 2024-07-17T08:31:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.752+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6a340ac220 con 0x7f6a340adcd0 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.753+0000 7f6a2b400640 1 -- 172.21.15.44:0/2292166864 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v202) ==== 84+0+65 (secure 0 0 0) 0x7f6a1c061890 con 0x7f6a340adcd0 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:36.754 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 msgr2=0x7f6a10079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 0x7f6a10079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a24034000 tx=0x7f6a24004730 comp rx=0 tx=0).stop 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 msgr2=0x7f6a341f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d950 tx=0x7f6a1c00de10 comp rx=0 tx=0).stop 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 shutdown_connections 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a341522c0 0x7f6a341f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a340adcd0 0x7f6a341f42e0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6a10076bb0 0x7f6a10079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 --2- 172.21.15.44:0/2292166864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a340ac760 0x7f6a340b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.757+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 >> 172.21.15.44:0/2292166864 conn(0x7f6a340b0fb0 msgr2=0x7f6a340af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.758+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 shutdown_connections 2024-07-17T08:31:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.758+0000 7f6a3a200640 1 -- 172.21.15.44:0/2292166864 wait complete. 2024-07-17T08:31:36.778 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:31:36.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.994+0000 7f0077200640 1 -- 172.21.15.44:0/92084780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 msgr2=0x7f005c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.994+0000 7f0077200640 1 --2- 172.21.15.44:0/92084780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f005c004800 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0060009b50 tx=0x7f005c005830 comp rx=0 tx=0).stop 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.995+0000 7f0076800640 1 -- 172.21.15.44:0/92084780 reap_dead start 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.995+0000 7f0077200640 1 -- 172.21.15.44:0/92084780 shutdown_connections 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.995+0000 7f0077200640 1 -- 172.21.15.44:0/92084780 >> 172.21.15.44:0/92084780 conn(0x7f00780cf050 msgr2=0x7f00780cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.995+0000 7f0077200640 1 -- 172.21.15.44:0/92084780 shutdown_connections 2024-07-17T08:31:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.995+0000 7f0077200640 1 -- 172.21.15.44:0/92084780 wait complete. 2024-07-17T08:31:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.997+0000 7f0077200640 1 Processor -- start 2024-07-17T08:31:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.997+0000 7f0077200640 1 -- start start 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781f4b30 0x7f007814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00781501c0 0x7f00781f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00780add80 con 0x7f005c003fd0 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0078073bd0 con 0x7f00781f4b30 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0077200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00780adf00 con 0x7f00781501c0 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41314/0 (socket says 172.21.15.44:41314) 2024-07-17T08:31:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 -- 172.21.15.44:0/130375568 learned_addr learned my addr 172.21.15.44:0/130375568 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 -- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00781501c0 msgr2=0x7f00781f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00781501c0 0x7f00781f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f0075e00640 1 -- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781f4b30 msgr2=0x7f007814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.998+0000 7f006fe00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781f4b30 0x7f007814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f0075e00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781f4b30 0x7f007814fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f0075e00640 1 -- 172.21.15.44:0/130375568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0078150620 con 0x7f005c003fd0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f0075e00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f006800e970 tx=0x7f006800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800ccc0 con 0x7f005c003fd0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f006800ce60 con 0x7f005c003fd0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781524f0 con 0x7f005c003fd0 2024-07-17T08:31:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068005260 con 0x7f005c003fd0 2024-07-17T08:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:36.999+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0078152ab0 con 0x7f005c003fd0 2024-07-17T08:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.000+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007815dd70 con 0x7f005c003fd0 2024-07-17T08:31:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.002+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f0068005400 con 0x7f005c003fd0 2024-07-17T08:31:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.003+0000 7f006e000640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 0x7f00540790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.003+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 5825+0+0 (secure 0 0 0) 0x7f0068098d40 con 0x7f005c003fd0 2024-07-17T08:31:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.003+0000 7f006fe00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 0x7f00540790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.004+0000 7f006fe00640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 0x7f00540790a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0060035000 tx=0x7f00600046c0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.007+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0078150620 con 0x7f005c003fd0 2024-07-17T08:31:37.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.219+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f00780ad250 con 0x7f005c003fd0 2024-07-17T08:31:37.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.761+0000 7f006e000640 1 -- 172.21.15.44:0/130375568 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v203) ==== 138+0+0 (secure 0 0 0) 0x7f0068062180 con 0x7f005c003fd0 2024-07-17T08:31:37.762 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 msgr2=0x7f00540790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 0x7f00540790a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0060035000 tx=0x7f00600046c0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 msgr2=0x7f00780b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f006800e970 tx=0x7f006800ee30 comp rx=0 tx=0).stop 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 shutdown_connections 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00781501c0 0x7f00781f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f0054076be0 0x7f00540790a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781f4b30 0x7f007814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 --2- 172.21.15.44:0/130375568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f005c003fd0 0x7f00780b0020 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.765+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 >> 172.21.15.44:0/130375568 conn(0x7f00780cf050 msgr2=0x7f007806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:37.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.766+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 shutdown_connections 2024-07-17T08:31:37.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.766+0000 7f0077200640 1 -- 172.21.15.44:0/130375568 wait complete. 2024-07-17T08:31:37.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.986+0000 7ffac8000640 1 -- 172.21.15.44:0/1053911330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00ce1d0 msgr2=0x7ffac00ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.986+0000 7ffac8000640 1 --2- 172.21.15.44:0/1053911330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00ce1d0 0x7ffac00ce5b0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffac008c1e0 tx=0x7ffab402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 -- 172.21.15.44:0/1053911330 shutdown_connections 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 --2- 172.21.15.44:0/1053911330 >> v2:172.21.15.175:3300/0 conn(0x7ffac00b29e0 0x7ffac0161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 --2- 172.21.15.44:0/1053911330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00ce1d0 0x7ffac00ce5b0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 -- 172.21.15.44:0/1053911330 >> 172.21.15.44:0/1053911330 conn(0x7ffac00d3700 msgr2=0x7ffac00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 -- 172.21.15.44:0/1053911330 shutdown_connections 2024-07-17T08:31:37.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.987+0000 7ffac8000640 1 -- 172.21.15.44:0/1053911330 wait complete. 2024-07-17T08:31:37.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.988+0000 7ffac8000640 1 Processor -- start 2024-07-17T08:31:37.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.988+0000 7ffac8000640 1 -- start start 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac00b6790 0x7ffac00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac00ce1d0 0x7ffac01566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0162420 con 0x7ffac00b29e0 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01622a0 con 0x7ffac00b6790 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.989+0000 7ffac8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffac0167910 con 0x7ffac00ce1d0 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41342/0 (socket says 172.21.15.44:41342) 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 -- 172.21.15.44:0/1151081419 learned_addr learned my addr 172.21.15.44:0/1151081419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac4c00640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac00b6790 0x7ffac00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac6000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac00ce1d0 0x7ffac01566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 -- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac00ce1d0 msgr2=0x7ffac01566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac00ce1d0 0x7ffac01566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 -- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac00b6790 msgr2=0x7ffac00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:37.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac00b6790 0x7ffac00c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.990+0000 7ffac5600640 1 -- 172.21.15.44:0/1151081419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac00c9050 con 0x7ffac00b29e0 2024-07-17T08:31:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffac5600640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009920 tx=0x7ffab4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4031d60 con 0x7ffac00b29e0 2024-07-17T08:31:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffab4033e10 con 0x7ffac00b29e0 2024-07-17T08:31:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac00c9330 con 0x7ffac00b29e0 2024-07-17T08:31:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005280 con 0x7ffac00b29e0 2024-07-17T08:31:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.991+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac00c98f0 con 0x7ffac00b29e0 2024-07-17T08:31:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.993+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac00ce5b0 con 0x7ffac00b29e0 2024-07-17T08:31:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.994+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7ffab4005420 con 0x7ffac00b29e0 2024-07-17T08:31:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.995+0000 7ffabd600640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 0x7ffaa0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.995+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(203..203 src has 1..203) ==== 6195+0+0 (secure 0 0 0) 0x7ffab40b9760 con 0x7ffac00b29e0 2024-07-17T08:31:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.996+0000 7ffac4c00640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 0x7ffaa0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.997+0000 7ffac4c00640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 0x7ffaa0079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffaac007fd0 tx=0x7ffaac009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:37.999+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffab403f030 con 0x7ffac00b29e0 2024-07-17T08:31:38.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:31:38.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:31:38.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.220+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffac015dc80 con 0x7ffac00b29e0 2024-07-17T08:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.779+0000 7ffabd600640 1 -- 172.21.15.44:0/1151081419 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v204) ==== 170+0+0 (secure 0 0 0) 0x7ffab4082460 con 0x7ffac00b29e0 2024-07-17T08:31:38.780 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.785+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 msgr2=0x7ffaa0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.785+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 0x7ffaa0079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffaac007fd0 tx=0x7ffaac009040 comp rx=0 tx=0).stop 2024-07-17T08:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.785+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 msgr2=0x7ffac014fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:38.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.785+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009920 tx=0x7ffab4033c20 comp rx=0 tx=0).stop 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 shutdown_connections 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac00ce1d0 0x7ffac01566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7ffaa0076bb0 0x7ffaa0079070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac00b6790 0x7ffac00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 --2- 172.21.15.44:0/1151081419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac00b29e0 0x7ffac014fef0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 >> 172.21.15.44:0/1151081419 conn(0x7ffac00d3700 msgr2=0x7ffac0157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 shutdown_connections 2024-07-17T08:31:38.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:38.786+0000 7ffac8000640 1 -- 172.21.15.44:0/1151081419 wait complete. 2024-07-17T08:31:38.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.006+0000 7f1cb3e00640 1 -- 172.21.15.44:0/4211444885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94005210 msgr2=0x7f1c9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.006+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/4211444885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94005210 0x7f1c9400b9c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c470 tx=0x7f1c94005830 comp rx=0 tx=0).stop 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.007+0000 7f1cb3e00640 1 -- 172.21.15.44:0/4211444885 shutdown_connections 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.007+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/4211444885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94005210 0x7f1c9400b9c0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.007+0000 7f1cb3e00640 1 -- 172.21.15.44:0/4211444885 >> 172.21.15.44:0/4211444885 conn(0x7f1cac0b0eb0 msgr2=0x7f1cac0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.007+0000 7f1cb3e00640 1 -- 172.21.15.44:0/4211444885 shutdown_connections 2024-07-17T08:31:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.007+0000 7f1cb3e00640 1 -- 172.21.15.44:0/4211444885 wait complete. 2024-07-17T08:31:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.009+0000 7f1cb3e00640 1 Processor -- start 2024-07-17T08:31:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.009+0000 7f1cb3e00640 1 -- start start 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 0x7f1cac14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 0x7f1cac0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac0b3e60 con 0x7f1cac0992a0 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac0b3ce0 con 0x7f1c94005210 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac0b3fe0 con 0x7f1cac099720 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50514/0 (socket says 172.21.15.44:50514) 2024-07-17T08:31:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 -- 172.21.15.44:0/364612211 learned_addr learned my addr 172.21.15.44:0/364612211 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb1e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 0x7f1cac0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb1400640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 0x7f1cac14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 -- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 msgr2=0x7f1cac0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 0x7f1cac0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 -- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 msgr2=0x7f1cac14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.010+0000 7f1cb0a00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 0x7f1cac14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb0a00640 1 -- 172.21.15.44:0/364612211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac09b6f0 con 0x7f1cac0992a0 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb1400640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 0x7f1cac14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb1e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 0x7f1cac0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb0a00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1c98004d60 tx=0x7f1c98001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980107d0 con 0x7f1cac0992a0 2024-07-17T08:31:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c98016070 con 0x7f1cac0992a0 2024-07-17T08:31:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98010d20 con 0x7f1cac0992a0 2024-07-17T08:31:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac099cd0 con 0x7f1cac0992a0 2024-07-17T08:31:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.011+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac099fb0 con 0x7f1cac0992a0 2024-07-17T08:31:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.012+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac0ac940 con 0x7f1cac0992a0 2024-07-17T08:31:39.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.014+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f1c9801b460 con 0x7f1cac0992a0 2024-07-17T08:31:39.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.015+0000 7f1ca9600640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 0x7f1c8c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:39.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.015+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(204..204 src has 1..204) ==== 6577+0+0 (secure 0 0 0) 0x7f1c98099850 con 0x7f1cac0992a0 2024-07-17T08:31:39.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.016+0000 7f1cb1400640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 0x7f1c8c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:39.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.017+0000 7f1cb1400640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 0x7f1c8c0790a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1cac074760 tx=0x7f1ca0009500 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.019+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1c98062340 con 0x7f1cac0992a0 2024-07-17T08:31:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.270+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1cac160e90 con 0x7f1cac0992a0 2024-07-17T08:31:39.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.784+0000 7f1ca9600640 1 -- 172.21.15.44:0/364612211 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v102) ==== 174+0+0 (secure 0 0 0) 0x7f1c98066890 con 0x7f1cac0992a0 2024-07-17T08:31:39.785 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 25 and data pool 26 2024-07-17T08:31:39.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.788+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 msgr2=0x7f1c8c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:39.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.788+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 0x7f1c8c0790a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1cac074760 tx=0x7f1ca0009500 comp rx=0 tx=0).stop 2024-07-17T08:31:39.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.788+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 msgr2=0x7f1cac0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:39.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.788+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1c98004d60 tx=0x7f1c98001630 comp rx=0 tx=0).stop 2024-07-17T08:31:39.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.789+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 shutdown_connections 2024-07-17T08:31:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.789+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1cac099720 0x7f1cac0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.790+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cac0992a0 0x7f1cac0b0260 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.790+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1c8c076be0 0x7f1c8c0790a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.790+0000 7f1cb3e00640 1 --2- 172.21.15.44:0/364612211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94005210 0x7f1cac14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:39.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.790+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 >> 172.21.15.44:0/364612211 conn(0x7f1cac0b0eb0 msgr2=0x7f1cac15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:39.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.791+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 shutdown_connections 2024-07-17T08:31:39.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:39.792+0000 7f1cb3e00640 1 -- 172.21.15.44:0/364612211 wait complete. 2024-07-17T08:31:39.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 -- 172.21.15.44:0/1239029984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c004020 msgr2=0x7f614c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 --2- 172.21.15.44:0/1239029984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f614c004020 0x7f614c004800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f615000ac60 tx=0x7f614c005830 comp rx=0 tx=0).stop 2024-07-17T08:31:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f6167400640 1 -- 172.21.15.44:0/1239029984 reap_dead start 2024-07-17T08:31:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 -- 172.21.15.44:0/1239029984 shutdown_connections 2024-07-17T08:31:40.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 -- 172.21.15.44:0/1239029984 >> 172.21.15.44:0/1239029984 conn(0x7f61680b0fb0 msgr2=0x7f61680b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:40.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 -- 172.21.15.44:0/1239029984 shutdown_connections 2024-07-17T08:31:40.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.009+0000 7f616d800640 1 -- 172.21.15.44:0/1239029984 wait complete. 2024-07-17T08:31:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.011+0000 7f616d800640 1 Processor -- start 2024-07-17T08:31:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.011+0000 7f616d800640 1 -- start start 2024-07-17T08:31:40.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f614c004020 0x7f61681f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680c9450 0x7f61680c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61680ac9c0 con 0x7f61680aef00 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61680ac840 con 0x7f614c004020 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f616d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61680acb40 con 0x7f61680c9450 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.012+0000 7f615fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50542/0 (socket says 172.21.15.44:50542) 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 -- 172.21.15.44:0/1484238390 learned_addr learned my addr 172.21.15.44:0/1484238390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f6167400640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680c9450 0x7f61680c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f6166a00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f614c004020 0x7f61681f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 -- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680c9450 msgr2=0x7f61680c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680c9450 0x7f61680c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 -- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f614c004020 msgr2=0x7f61681f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f614c004020 0x7f61681f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 -- 172.21.15.44:0/1484238390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681f50e0 con 0x7f61680aef00 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.013+0000 7f615fe00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f615000ab80 tx=0x7f6150034ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.014+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150039040 con 0x7f61680aef00 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.014+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6150004310 con 0x7f61680aef00 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.014+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150003850 con 0x7f61680aef00 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.014+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61680af470 con 0x7f61680aef00 2024-07-17T08:31:40.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.014+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61680afa30 con 0x7f61680aef00 2024-07-17T08:31:40.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.015+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168159d80 con 0x7f61680aef00 2024-07-17T08:31:40.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.016+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f61500039f0 con 0x7f61680aef00 2024-07-17T08:31:40.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.017+0000 7f615ea00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 0x7f6144079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.017+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(205..205 src has 1..205) ==== 6677+0+0 (secure 0 0 0) 0x7f61500b9320 con 0x7f61680aef00 2024-07-17T08:31:40.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.018+0000 7f6166a00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 0x7f6144079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.019+0000 7f6166a00640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 0x7f6144079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6168074ae0 tx=0x7f6158009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.023+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f61681f50e0 con 0x7f61680aef00 2024-07-17T08:31:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.227+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f616813f1e0 con 0x7f61680aef00 2024-07-17T08:31:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.230+0000 7f615ea00640 1 -- 172.21.15.44:0/1484238390 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) ==== 74+0+17766 (secure 0 0 0) 0x7f6150082410 con 0x7f61680aef00 2024-07-17T08:31:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:40.232 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":205,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:39.271852+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:31:37.247087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-07-17T08:31:38.238209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 msgr2=0x7f6144079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 0x7f6144079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6168074ae0 tx=0x7f6158009040 comp rx=0 tx=0).stop 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 msgr2=0x7f61681f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f615000ab80 tx=0x7f6150034ea0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 shutdown_connections 2024-07-17T08:31:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61680c9450 0x7f61680c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61680aef00 0x7f61681f3890 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f6144076bb0 0x7f6144079070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 --2- 172.21.15.44:0/1484238390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f614c004020 0x7f61681f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.233+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 >> 172.21.15.44:0/1484238390 conn(0x7f61680b0fb0 msgr2=0x7f616814e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.234+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 shutdown_connections 2024-07-17T08:31:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.234+0000 7f616d800640 1 -- 172.21.15.44:0/1484238390 wait complete. 2024-07-17T08:31:40.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.453+0000 7f2aa5400640 1 -- 172.21.15.44:0/1417701492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa006ea50 msgr2=0x7f2aa00ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.453+0000 7f2aa5400640 1 --2- 172.21.15.44:0/1417701492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa006ea50 0x7f2aa00ac220 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009900 tx=0x7f2a8c02a580 comp rx=0 tx=0).stop 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.453+0000 7f2aa5400640 1 -- 172.21.15.44:0/1417701492 shutdown_connections 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.453+0000 7f2aa5400640 1 --2- 172.21.15.44:0/1417701492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa006ea50 0x7f2aa00ac220 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.453+0000 7f2aa5400640 1 -- 172.21.15.44:0/1417701492 >> 172.21.15.44:0/1417701492 conn(0x7f2aa00b0fb0 msgr2=0x7f2aa00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.454+0000 7f2aa5400640 1 -- 172.21.15.44:0/1417701492 shutdown_connections 2024-07-17T08:31:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.454+0000 7f2aa5400640 1 -- 172.21.15.44:0/1417701492 wait complete. 2024-07-17T08:31:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.456+0000 7f2aa5400640 1 Processor -- start 2024-07-17T08:31:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.456+0000 7f2aa5400640 1 -- start start 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2aa006ea50 0x7f2aa01f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2aa006eec0 0x7f2aa01f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa00b3070 con 0x7f2aa00ae810 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa00b2ef0 con 0x7f2aa006eec0 2024-07-17T08:31:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2aa5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa00aed30 con 0x7f2aa006ea50 2024-07-17T08:31:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2a9f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2a9f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50548/0 (socket says 172.21.15.44:50548) 2024-07-17T08:31:40.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.457+0000 7f2a9f200640 1 -- 172.21.15.44:0/703044871 learned_addr learned my addr 172.21.15.44:0/703044871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9de00640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2aa006eec0 0x7f2aa01f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9e800640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2aa006ea50 0x7f2aa01f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 -- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2aa006ea50 msgr2=0x7f2aa01f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2aa006ea50 0x7f2aa01f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 -- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2aa006eec0 msgr2=0x7f2aa01f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2aa006eec0 0x7f2aa01f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 -- 172.21.15.44:0/703044871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa01422a0 con 0x7f2aa00ae810 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.458+0000 7f2a9f200640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c780 tx=0x7f2a9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.459+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98017070 con 0x7f2aa00ae810 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.459+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a9800ce30 con 0x7f2aa00ae810 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.459+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98005290 con 0x7f2aa00ae810 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.459+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa0142580 con 0x7f2aa00ae810 2024-07-17T08:31:40.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.459+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa0142b40 con 0x7f2aa00ae810 2024-07-17T08:31:40.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.460+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa00aca70 con 0x7f2aa00ae810 2024-07-17T08:31:40.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.461+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f2a98005430 con 0x7f2aa00ae810 2024-07-17T08:31:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.462+0000 7f2a96a00640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 0x7f2a80079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.462+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(205..205 src has 1..205) ==== 6677+0+0 (secure 0 0 0) 0x7f2a98097c70 con 0x7f2aa00ae810 2024-07-17T08:31:40.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.463+0000 7f2a9e800640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 0x7f2a80079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.464+0000 7f2a9e800640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 0x7f2a80079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a900015e0 tx=0x7f2a90009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.466+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2a98060cd0 con 0x7f2aa00ae810 2024-07-17T08:31:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.677+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2aa00b23b0 con 0x7f2aa00ae810 2024-07-17T08:31:40.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.680+0000 7f2a96a00640 1 -- 172.21.15.44:0/703044871 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) ==== 74+0+17766 (secure 0 0 0) 0x7f2a980605d0 con 0x7f2aa00ae810 2024-07-17T08:31:40.681 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:40.681 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":205,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:39.271852+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:31:37.247087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-07-17T08:31:38.238209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:40.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.682+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 msgr2=0x7f2a80079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.682+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 0x7f2a80079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a900015e0 tx=0x7f2a90009040 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.682+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 msgr2=0x7f2aa0143340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.682+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c780 tx=0x7f2a9800cc40 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 shutdown_connections 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2aa00ae810 0x7f2aa0143340 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2a80076bb0 0x7f2a80079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2aa006eec0 0x7f2aa01f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 --2- 172.21.15.44:0/703044871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2aa006ea50 0x7f2aa01f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 >> 172.21.15.44:0/703044871 conn(0x7f2aa00b0fb0 msgr2=0x7f2aa00cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:40.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 shutdown_connections 2024-07-17T08:31:40.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.683+0000 7f2aa5400640 1 -- 172.21.15.44:0/703044871 wait complete. 2024-07-17T08:31:40.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:31:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 -- 172.21.15.44:0/4106877350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e80ac3a0 msgr2=0x7fa8e80ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 --2- 172.21.15.44:0/4106877350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e80ac3a0 0x7fa8e80ac810 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005fd0 tx=0x7fa8e002b000 comp rx=0 tx=0).stop 2024-07-17T08:31:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 -- 172.21.15.44:0/4106877350 shutdown_connections 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 --2- 172.21.15.44:0/4106877350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e80ac3a0 0x7fa8e80ac810 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 --2- 172.21.15.44:0/4106877350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e815a5a0 0x7fa8e80b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 -- 172.21.15.44:0/4106877350 >> 172.21.15.44:0/4106877350 conn(0x7fa8e80b0eb0 msgr2=0x7fa8e80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 -- 172.21.15.44:0/4106877350 shutdown_connections 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.917+0000 7fa8ef800640 1 -- 172.21.15.44:0/4106877350 wait complete. 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.919+0000 7fa8ef800640 1 Processor -- start 2024-07-17T08:31:40.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.919+0000 7fa8ef800640 1 -- start start 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 0x7fa8e814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 0x7fa8e80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e806f080 con 0x7fa8e815a5a0 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e806f8e0 con 0x7fa8e806eb90 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ef800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e814fd60 con 0x7fa8e80ac3a0 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ed800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ed800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50566/0 (socket says 172.21.15.44:50566) 2024-07-17T08:31:40.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.920+0000 7fa8ed800640 1 -- 172.21.15.44:0/2137644481 learned_addr learned my addr 172.21.15.44:0/2137644481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 -- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 msgr2=0x7fa8e80737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ece00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 0x7fa8e814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8e7e00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 0x7fa8e80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 0x7fa8e80737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 -- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 msgr2=0x7fa8e814f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 0x7fa8e814f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 -- 172.21.15.44:0/2137644481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e80cf8e0 con 0x7fa8e815a5a0 2024-07-17T08:31:40.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ece00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 0x7fa8e814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8e7e00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 0x7fa8e80737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8ed800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036040 tx=0x7fa8e0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.921+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0040020 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.922+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8e0004040 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.922+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0038040 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.922+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e80cfbc0 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.922+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e80c9870 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.923+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e80cfe00 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.924+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fa8e0043020 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.925+0000 7fa8e4c00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 0x7fa8c8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.925+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7fa8e00b8ec0 con 0x7fa8e815a5a0 2024-07-17T08:31:40.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.925+0000 7fa8ece00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 0x7fa8c8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:40.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.927+0000 7fa8ece00640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 0x7fa8c8079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8001740 tx=0x7fa8d8009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:40.929+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa8e004a030 con 0x7fa8e815a5a0 2024-07-17T08:31:41.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.173+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8e80ac810 con 0x7fa8e815a5a0 2024-07-17T08:31:41.759 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-07-17T08:31:41.758+0000 7f0b56a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:31:41.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.829+0000 7fa8e4c00640 1 -- 172.21.15.44:0/2137644481 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) ==== 118+0+0 (secure 0 0 0) 0x7fa8e0081f20 con 0x7fa8e815a5a0 2024-07-17T08:31:41.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 msgr2=0x7fa8c8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:41.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 0x7fa8c8079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8001740 tx=0x7fa8d8009040 comp rx=0 tx=0).stop 2024-07-17T08:31:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 msgr2=0x7fa8e8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036040 tx=0x7fa8e0009900 comp rx=0 tx=0).stop 2024-07-17T08:31:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 shutdown_connections 2024-07-17T08:31:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8e815a5a0 0x7fa8e8073cf0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:41.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8e80ac3a0 0x7fa8e80737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa8c8076bb0 0x7fa8c8079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 --2- 172.21.15.44:0/2137644481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8e806eb90 0x7fa8e814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 >> 172.21.15.44:0/2137644481 conn(0x7fa8e80b0eb0 msgr2=0x7fa8e8152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 shutdown_connections 2024-07-17T08:31:41.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:41.833+0000 7fa8ef800640 1 -- 172.21.15.44:0/2137644481 wait complete. 2024-07-17T08:31:41.855 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:31:41.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:31:42.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 -- 172.21.15.44:0/3523563900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde406e5c0 msgr2=0x7efde40acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 --2- 172.21.15.44:0/3523563900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde406e5c0 0x7efde40acb30 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7efde40ae340 tx=0x7efdd402a5f0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 -- 172.21.15.44:0/3523563900 shutdown_connections 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 --2- 172.21.15.44:0/3523563900 >> v2:172.21.15.175:3300/0 conn(0x7efde40ad070 0x7efde40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 --2- 172.21.15.44:0/3523563900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde406e5c0 0x7efde40acb30 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 -- 172.21.15.44:0/3523563900 >> 172.21.15.44:0/3523563900 conn(0x7efde40b0eb0 msgr2=0x7efde40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.068+0000 7efdeb600640 1 -- 172.21.15.44:0/3523563900 shutdown_connections 2024-07-17T08:31:42.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.069+0000 7efdeb600640 1 -- 172.21.15.44:0/3523563900 wait complete. 2024-07-17T08:31:42.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.070+0000 7efdeb600640 1 Processor -- start 2024-07-17T08:31:42.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.070+0000 7efdeb600640 1 -- start start 2024-07-17T08:31:42.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 0x7efde41f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:42.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 0x7efde414e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:42.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:42.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efde406d9b0 con 0x7efde40b3c20 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efde406d830 con 0x7efde406e5c0 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.071+0000 7efdeb600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efde406db30 con 0x7efde40ad070 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50586/0 (socket says 172.21.15.44:50586) 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 -- 172.21.15.44:0/1910069167 learned_addr learned my addr 172.21.15.44:0/1910069167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde8c00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 0x7efde41f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efddbe00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 0x7efde414e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 -- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 msgr2=0x7efde414e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 0x7efde414e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 -- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 msgr2=0x7efde41f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 0x7efde41f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde9600640 1 -- 172.21.15.44:0/1910069167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde414e720 con 0x7efde40b3c20 2024-07-17T08:31:42.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efde8c00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 0x7efde41f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:31:42.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.072+0000 7efddbe00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 0x7efde414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:31:42.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efde9600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7efddc00cce0 tx=0x7efddc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:42.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00f450 con 0x7efde40b3c20 2024-07-17T08:31:42.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efddc00f700 con 0x7efde40b3c20 2024-07-17T08:31:42.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00fb50 con 0x7efde40b3c20 2024-07-17T08:31:42.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde41f2450 con 0x7efde40b3c20 2024-07-17T08:31:42.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.073+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41f2a10 con 0x7efde40b3c20 2024-07-17T08:31:42.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.074+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde40acb30 con 0x7efde40b3c20 2024-07-17T08:31:42.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.076+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7efddc00fcf0 con 0x7efde40b3c20 2024-07-17T08:31:42.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.077+0000 7efde1600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 0x7efdc0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:42.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.077+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7efddc099240 con 0x7efde40b3c20 2024-07-17T08:31:42.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.077+0000 7efde8c00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 0x7efdc0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:42.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.079+0000 7efde8c00640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 0x7efdc0079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efdcc007fd0 tx=0x7efdcc009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:42.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.081+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efddc014030 con 0x7efde40b3c20 2024-07-17T08:31:42.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.341+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efde40af910 con 0x7efde40b3c20 2024-07-17T08:31:42.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.832+0000 7efde1600640 1 -- 172.21.15.44:0/1910069167 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) ==== 122+0+0 (secure 0 0 0) 0x7efddc062330 con 0x7efde40b3c20 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 msgr2=0x7efdc0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 0x7efdc0079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efdcc007fd0 tx=0x7efdcc009040 comp rx=0 tx=0).stop 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 msgr2=0x7efde409b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7efddc00cce0 tx=0x7efddc00a620 comp rx=0 tx=0).stop 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 shutdown_connections 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde40b3c20 0x7efde409b2b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde40ad070 0x7efde414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7efdc0076bb0 0x7efdc0079070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 --2- 172.21.15.44:0/1910069167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde406e5c0 0x7efde41f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:42.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 >> 172.21.15.44:0/1910069167 conn(0x7efde40b0eb0 msgr2=0x7efde40742a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:42.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 shutdown_connections 2024-07-17T08:31:42.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:42.836+0000 7efdeb600640 1 -- 172.21.15.44:0/1910069167 wait complete. 2024-07-17T08:31:42.859 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:31:42.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 -- 172.21.15.44:0/4256223754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b64003ac0 msgr2=0x7f9b64003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 --2- 172.21.15.44:0/4256223754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b64003ac0 0x7f9b64003ee0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000c310 tx=0x7f9b64005830 comp rx=0 tx=0).stop 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 -- 172.21.15.44:0/4256223754 shutdown_connections 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 --2- 172.21.15.44:0/4256223754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b64003ac0 0x7f9b64003ee0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 -- 172.21.15.44:0/4256223754 >> 172.21.15.44:0/4256223754 conn(0x7f9b8006ce80 msgr2=0x7f9b8006d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 -- 172.21.15.44:0/4256223754 shutdown_connections 2024-07-17T08:31:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.073+0000 7f9b86600640 1 -- 172.21.15.44:0/4256223754 wait complete. 2024-07-17T08:31:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.075+0000 7f9b86600640 1 Processor -- start 2024-07-17T08:31:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.075+0000 7f9b86600640 1 -- start start 2024-07-17T08:31:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 0x7f9b8015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b80158e00 0x7f9b80159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8015bc10 con 0x7f9b80141cb0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8015ba90 con 0x7f9b64003ac0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b86600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8015bd90 con 0x7f9b80158e00 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.076+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50616/0 (socket says 172.21.15.44:50616) 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 -- 172.21.15.44:0/2046342657 learned_addr learned my addr 172.21.15.44:0/2046342657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b85200640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 0x7f9b8015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b85c00640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b80158e00 0x7f9b80159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 -- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b80158e00 msgr2=0x7f9b80159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b80158e00 0x7f9b80159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 -- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 msgr2=0x7f9b8015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 0x7f9b8015ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 -- 172.21.15.44:0/2046342657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8015b390 con 0x7f9b80141cb0 2024-07-17T08:31:43.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b85200640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 0x7f9b8015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.077+0000 7f9b7fe00640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c480 tx=0x7f9b6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.078+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0052d0 con 0x7f9b80141cb0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.078+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b6c005490 con 0x7f9b80141cb0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.078+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c012890 con 0x7f9b80141cb0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.078+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b801f1ca0 con 0x7f9b80141cb0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.078+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b801f2260 con 0x7f9b80141cb0 2024-07-17T08:31:43.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.079+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b800cf330 con 0x7f9b80141cb0 2024-07-17T08:31:43.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.080+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f9b6c0035f0 con 0x7f9b80141cb0 2024-07-17T08:31:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.081+0000 7f9b7cc00640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 0x7f9b60079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.081+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f9b6c0982b0 con 0x7f9b80141cb0 2024-07-17T08:31:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.082+0000 7f9b85200640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 0x7f9b60079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.083+0000 7f9b85200640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 0x7f9b60079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b70033000 tx=0x7f9b70001a60 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.085+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9b6c0613a0 con 0x7f9b80141cb0 2024-07-17T08:31:43.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.332+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b8014f810 con 0x7f9b80141cb0 2024-07-17T08:31:43.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.333+0000 7f9b7cc00640 1 -- 172.21.15.44:0/2046342657 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7f9b6c060ca0 con 0x7f9b80141cb0 2024-07-17T08:31:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:43.334 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:31:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 msgr2=0x7f9b60079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 0x7f9b60079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b70033000 tx=0x7f9b70001a60 comp rx=0 tx=0).stop 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 msgr2=0x7f9b8013ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c480 tx=0x7f9b6c00c940 comp rx=0 tx=0).stop 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 shutdown_connections 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b80158e00 0x7f9b80159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b80141cb0 0x7f9b8013ea90 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f9b60076bb0 0x7f9b60079070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 --2- 172.21.15.44:0/2046342657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b64003ac0 0x7f9b8015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 >> 172.21.15.44:0/2046342657 conn(0x7f9b8006ce80 msgr2=0x7f9b800b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:43.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 shutdown_connections 2024-07-17T08:31:43.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.336+0000 7f9b86600640 1 -- 172.21.15.44:0/2046342657 wait complete. 2024-07-17T08:31:43.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 -- 172.21.15.44:0/1912650705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac870 msgr2=0x7f2b2813f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1912650705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac870 0x7f2b2813f470 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2b28070c50 tx=0x7f2b2002a8b0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 -- 172.21.15.44:0/1912650705 shutdown_connections 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1912650705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac870 0x7f2b2813f470 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1912650705 >> v2:172.21.15.175:3300/0 conn(0x7f2b2806f710 0x7f2b28153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.561+0000 7f2b2d600640 1 -- 172.21.15.44:0/1912650705 >> 172.21.15.44:0/1912650705 conn(0x7f2b280b0fb0 msgr2=0x7f2b280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.562+0000 7f2b2d600640 1 -- 172.21.15.44:0/1912650705 shutdown_connections 2024-07-17T08:31:43.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.562+0000 7f2b2d600640 1 -- 172.21.15.44:0/1912650705 wait complete. 2024-07-17T08:31:43.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.563+0000 7f2b2d600640 1 Processor -- start 2024-07-17T08:31:43.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.563+0000 7f2b2d600640 1 -- start start 2024-07-17T08:31:43.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 0x7f2b28155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 0x7f2b281f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b280ae800 con 0x7f2b280ac3e0 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2806f0a0 con 0x7f2b2806f710 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.564+0000 7f2b2d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b280ae980 con 0x7f2b280ac870 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50634/0 (socket says 172.21.15.44:50634) 2024-07-17T08:31:43.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 -- 172.21.15.44:0/1378747281 learned_addr learned my addr 172.21.15.44:0/1378747281 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b26a00640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 0x7f2b28155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b27400640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 0x7f2b281f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 -- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 msgr2=0x7f2b281f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 0x7f2b281f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 -- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 msgr2=0x7f2b28155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 0x7f2b28155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b1e000640 1 -- 172.21.15.44:0/1378747281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28157040 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b27400640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 0x7f2b281f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.565+0000 7f2b26a00640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 0x7f2b28155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b1e000640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c730 tx=0x7f2b1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10016020 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b1000a930 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b28156d60 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000ad80 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.566+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b281423d0 con 0x7f2b280ac3e0 2024-07-17T08:31:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.567+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2814fc60 con 0x7f2b280ac3e0 2024-07-17T08:31:43.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.568+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f2b10019020 con 0x7f2b280ac3e0 2024-07-17T08:31:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.569+0000 7f2b1f400640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 0x7f2b04079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.569+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f2b10098700 con 0x7f2b280ac3e0 2024-07-17T08:31:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.570+0000 7f2b26a00640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 0x7f2b04079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:43.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.571+0000 7f2b26a00640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 0x7f2b04079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800ba60 tx=0x7f2b18005cf0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:43.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.574+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2b10020020 con 0x7f2b280ac3e0 2024-07-17T08:31:43.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.779+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b280b03c0 con 0x7f2b280ac3e0 2024-07-17T08:31:43.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.781+0000 7f2b1f400640 1 -- 172.21.15.44:0/1378747281 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) ==== 74+0+17753 (secure 0 0 0) 0x7f2b2815acc0 con 0x7f2b280ac3e0 2024-07-17T08:31:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:43.782 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":206,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:31:40.803226+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:31:37.247087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-07-17T08:31:38.238209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:31:43.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 msgr2=0x7f2b04079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 0x7f2b04079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800ba60 tx=0x7f2b18005cf0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 msgr2=0x7f2b281562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:43.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c730 tx=0x7f2b1000cbf0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 shutdown_connections 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b280ac870 0x7f2b281f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b280ac3e0 0x7f2b281562a0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f2b04076bb0 0x7f2b04079070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 --2- 172.21.15.44:0/1378747281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b2806f710 0x7f2b28155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.783+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 >> 172.21.15.44:0/1378747281 conn(0x7f2b280b0fb0 msgr2=0x7f2b28158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.784+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 shutdown_connections 2024-07-17T08:31:43.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:43.784+0000 7f2b2d600640 1 -- 172.21.15.44:0/1378747281 wait complete. 2024-07-17T08:31:43.805 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:31:43.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 -- 172.21.15.44:0/4161670810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 msgr2=0x7face406f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 --2- 172.21.15.44:0/4161670810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face406f4a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7face4070ff0 tx=0x7facd402a580 comp rx=0 tx=0).stop 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 -- 172.21.15.44:0/4161670810 shutdown_connections 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 --2- 172.21.15.44:0/4161670810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 0x7face40b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 --2- 172.21.15.44:0/4161670810 >> v2:172.21.15.175:3300/0 conn(0x7face40b30b0 0x7face40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 --2- 172.21.15.44:0/4161670810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face406f4a0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 -- 172.21.15.44:0/4161670810 >> 172.21.15.44:0/4161670810 conn(0x7face40b0fb0 msgr2=0x7face40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 -- 172.21.15.44:0/4161670810 shutdown_connections 2024-07-17T08:31:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.004+0000 7face8e00640 1 -- 172.21.15.44:0/4161670810 wait complete. 2024-07-17T08:31:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.005+0000 7face8e00640 1 Processor -- start 2024-07-17T08:31:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.006+0000 7face8e00640 1 -- start start 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 0x7face40993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face40b30b0 0x7face4158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7face406dcc0 con 0x7face4159910 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7face406db40 con 0x7face40b30b0 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7face4159cf0 con 0x7face40acc30 2024-07-17T08:31:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50652/0 (socket says 172.21.15.44:50652) 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 -- 172.21.15.44:0/4183419189 learned_addr learned my addr 172.21.15.44:0/4183419189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face1e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 0x7face40993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7facdbe00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face40b30b0 0x7face4158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 -- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 msgr2=0x7face40993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 0x7face40993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 -- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face40b30b0 msgr2=0x7face4158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face40b30b0 0x7face4158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.007+0000 7face2800640 1 -- 172.21.15.44:0/4183419189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face4099a60 con 0x7face4159910 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7face2800640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b9e0 tx=0x7facdc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc019070 con 0x7face4159910 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc00cb90 con 0x7face4159910 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face4099d40 con 0x7face4159910 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc014440 con 0x7face4159910 2024-07-17T08:31:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.008+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face409a020 con 0x7face4159910 2024-07-17T08:31:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.009+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face41f5bf0 con 0x7face4159910 2024-07-17T08:31:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.010+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7facdc01c070 con 0x7face4159910 2024-07-17T08:31:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.011+0000 7facda000640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 0x7facc0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.012+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7facdc0983d0 con 0x7face4159910 2024-07-17T08:31:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.012+0000 7face1e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 0x7facc0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.013+0000 7face1e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 0x7facc0079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facd4033000 tx=0x7facd4004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:44.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.016+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7facdc0614c0 con 0x7face4159910 2024-07-17T08:31:44.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.228+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 --> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7face4155830 con 0x7facc0076bb0 2024-07-17T08:31:44.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.244+0000 7facda000640 1 -- 172.21.15.44:0/4183419189 <== mgr.7280 v2:172.21.15.175:6834/1104780009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7face4155830 con 0x7facc0076bb0 2024-07-17T08:31:44.249 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:44.250 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 msgr2=0x7facc0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 0x7facc0079070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facd4033000 tx=0x7facd4004730 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 msgr2=0x7face41552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b9e0 tx=0x7facdc00bea0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 shutdown_connections 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7face4159910 0x7face41552f0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7facc0076bb0 0x7facc0079070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7face40b30b0 0x7face4158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 --2- 172.21.15.44:0/4183419189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7face40acc30 0x7face40993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.249+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 >> 172.21.15.44:0/4183419189 conn(0x7face40b0fb0 msgr2=0x7face4073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.250+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 shutdown_connections 2024-07-17T08:31:44.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.250+0000 7face8e00640 1 -- 172.21.15.44:0/4183419189 wait complete. 2024-07-17T08:31:44.272 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:31:43.651795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":380,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2536,"ondisk_log_size":2536,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422068,"kb_used_data":21304,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":754552652,"statfs":{"total":773094113280,"available":772661915648,"internally_reserved":0,"allocated":21815296,"data_stored":12334460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":410242608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.887505"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821976+0000","last_change":"2024-07-17T08:31:39.823959+0000","last_active":"2024-07-17T08:31:40.821976+0000","last_peered":"2024-07-17T08:31:40.821976+0000","last_clean":"2024-07-17T08:31:40.821976+0000","last_became_active":"2024-07-17T08:31:39.823622+0000","last_became_peered":"2024-07-17T08:31:39.823622+0000","last_unstale":"2024-07-17T08:31:40.821976+0000","last_undegraded":"2024-07-17T08:31:40.821976+0000","last_fullsized":"2024-07-17T08:31:40.821976+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:09.489756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821990+0000","last_change":"2024-07-17T08:31:39.834589+0000","last_active":"2024-07-17T08:31:40.821990+0000","last_peered":"2024-07-17T08:31:40.821990+0000","last_clean":"2024-07-17T08:31:40.821990+0000","last_became_active":"2024-07-17T08:31:39.834318+0000","last_became_peered":"2024-07-17T08:31:39.834318+0000","last_unstale":"2024-07-17T08:31:40.821990+0000","last_undegraded":"2024-07-17T08:31:40.821990+0000","last_fullsized":"2024-07-17T08:31:40.821990+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:28.977482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823377+0000","last_change":"2024-07-17T08:31:39.810119+0000","last_active":"2024-07-17T08:31:40.823377+0000","last_peered":"2024-07-17T08:31:40.823377+0000","last_clean":"2024-07-17T08:31:40.823377+0000","last_became_active":"2024-07-17T08:31:39.809661+0000","last_became_peered":"2024-07-17T08:31:39.809661+0000","last_unstale":"2024-07-17T08:31:40.823377+0000","last_undegraded":"2024-07-17T08:31:40.823377+0000","last_fullsized":"2024-07-17T08:31:40.823377+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:02:39.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588980+0000","last_change":"2024-07-17T08:31:39.829466+0000","last_active":"2024-07-17T08:31:41.588980+0000","last_peered":"2024-07-17T08:31:41.588980+0000","last_clean":"2024-07-17T08:31:41.588980+0000","last_became_active":"2024-07-17T08:31:39.828590+0000","last_became_peered":"2024-07-17T08:31:39.828590+0000","last_unstale":"2024-07-17T08:31:41.588980+0000","last_undegraded":"2024-07-17T08:31:41.588980+0000","last_fullsized":"2024-07-17T08:31:41.588980+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:52:27.370045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589516+0000","last_change":"2024-07-17T08:31:39.825045+0000","last_active":"2024-07-17T08:31:41.589516+0000","last_peered":"2024-07-17T08:31:41.589516+0000","last_clean":"2024-07-17T08:31:41.589516+0000","last_became_active":"2024-07-17T08:31:39.824424+0000","last_became_peered":"2024-07-17T08:31:39.824424+0000","last_unstale":"2024-07-17T08:31:41.589516+0000","last_undegraded":"2024-07-17T08:31:41.589516+0000","last_fullsized":"2024-07-17T08:31:41.589516+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:23.537611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589296+0000","last_change":"2024-07-17T08:31:39.805661+0000","last_active":"2024-07-17T08:31:41.589296+0000","last_peered":"2024-07-17T08:31:41.589296+0000","last_clean":"2024-07-17T08:31:41.589296+0000","last_became_active":"2024-07-17T08:31:39.805408+0000","last_became_peered":"2024-07-17T08:31:39.805408+0000","last_unstale":"2024-07-17T08:31:41.589296+0000","last_undegraded":"2024-07-17T08:31:41.589296+0000","last_fullsized":"2024-07-17T08:31:41.589296+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:31.040807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822018+0000","last_change":"2024-07-17T08:31:39.820719+0000","last_active":"2024-07-17T08:31:40.822018+0000","last_peered":"2024-07-17T08:31:40.822018+0000","last_clean":"2024-07-17T08:31:40.822018+0000","last_became_active":"2024-07-17T08:31:39.820422+0000","last_became_peered":"2024-07-17T08:31:39.820422+0000","last_unstale":"2024-07-17T08:31:40.822018+0000","last_undegraded":"2024-07-17T08:31:40.822018+0000","last_fullsized":"2024-07-17T08:31:40.822018+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:10.937482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161506+0000","last_change":"2024-07-17T08:31:39.836492+0000","last_active":"2024-07-17T08:31:41.161506+0000","last_peered":"2024-07-17T08:31:41.161506+0000","last_clean":"2024-07-17T08:31:41.161506+0000","last_became_active":"2024-07-17T08:31:39.836108+0000","last_became_peered":"2024-07-17T08:31:39.836108+0000","last_unstale":"2024-07-17T08:31:41.161506+0000","last_undegraded":"2024-07-17T08:31:41.161506+0000","last_fullsized":"2024-07-17T08:31:41.161506+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:36.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823514+0000","last_change":"2024-07-17T08:31:39.817001+0000","last_active":"2024-07-17T08:31:40.823514+0000","last_peered":"2024-07-17T08:31:40.823514+0000","last_clean":"2024-07-17T08:31:40.823514+0000","last_became_active":"2024-07-17T08:31:39.816712+0000","last_became_peered":"2024-07-17T08:31:39.816712+0000","last_unstale":"2024-07-17T08:31:40.823514+0000","last_undegraded":"2024-07-17T08:31:40.823514+0000","last_fullsized":"2024-07-17T08:31:40.823514+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:36:10.825056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590634+0000","last_change":"2024-07-17T08:31:39.818917+0000","last_active":"2024-07-17T08:31:41.590634+0000","last_peered":"2024-07-17T08:31:41.590634+0000","last_clean":"2024-07-17T08:31:41.590634+0000","last_became_active":"2024-07-17T08:31:39.818644+0000","last_became_peered":"2024-07-17T08:31:39.818644+0000","last_unstale":"2024-07-17T08:31:41.590634+0000","last_undegraded":"2024-07-17T08:31:41.590634+0000","last_fullsized":"2024-07-17T08:31:41.590634+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:18:33.054231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588081+0000","last_change":"2024-07-17T08:31:39.825586+0000","last_active":"2024-07-17T08:31:41.588081+0000","last_peered":"2024-07-17T08:31:41.588081+0000","last_clean":"2024-07-17T08:31:41.588081+0000","last_became_active":"2024-07-17T08:31:39.818539+0000","last_became_peered":"2024-07-17T08:31:39.818539+0000","last_unstale":"2024-07-17T08:31:41.588081+0000","last_undegraded":"2024-07-17T08:31:41.588081+0000","last_fullsized":"2024-07-17T08:31:41.588081+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:45.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589059+0000","last_change":"2024-07-17T08:31:39.823951+0000","last_active":"2024-07-17T08:31:41.589059+0000","last_peered":"2024-07-17T08:31:41.589059+0000","last_clean":"2024-07-17T08:31:41.589059+0000","last_became_active":"2024-07-17T08:31:39.823672+0000","last_became_peered":"2024-07-17T08:31:39.823672+0000","last_unstale":"2024-07-17T08:31:41.589059+0000","last_undegraded":"2024-07-17T08:31:41.589059+0000","last_fullsized":"2024-07-17T08:31:41.589059+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:47.136915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590210+0000","last_change":"2024-07-17T08:31:39.816567+0000","last_active":"2024-07-17T08:31:41.590210+0000","last_peered":"2024-07-17T08:31:41.590210+0000","last_clean":"2024-07-17T08:31:41.590210+0000","last_became_active":"2024-07-17T08:31:39.816217+0000","last_became_peered":"2024-07-17T08:31:39.816217+0000","last_unstale":"2024-07-17T08:31:41.590210+0000","last_undegraded":"2024-07-17T08:31:41.590210+0000","last_fullsized":"2024-07-17T08:31:41.590210+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:20.066592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822614+0000","last_change":"2024-07-17T08:31:39.816531+0000","last_active":"2024-07-17T08:31:40.822614+0000","last_peered":"2024-07-17T08:31:40.822614+0000","last_clean":"2024-07-17T08:31:40.822614+0000","last_became_active":"2024-07-17T08:31:39.815950+0000","last_became_peered":"2024-07-17T08:31:39.815950+0000","last_unstale":"2024-07-17T08:31:40.822614+0000","last_undegraded":"2024-07-17T08:31:40.822614+0000","last_fullsized":"2024-07-17T08:31:40.822614+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:07.028191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821048+0000","last_change":"2024-07-17T08:31:39.812583+0000","last_active":"2024-07-17T08:31:40.821048+0000","last_peered":"2024-07-17T08:31:40.821048+0000","last_clean":"2024-07-17T08:31:40.821048+0000","last_became_active":"2024-07-17T08:31:39.812310+0000","last_became_peered":"2024-07-17T08:31:39.812310+0000","last_unstale":"2024-07-17T08:31:40.821048+0000","last_undegraded":"2024-07-17T08:31:40.821048+0000","last_fullsized":"2024-07-17T08:31:40.821048+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:30.031284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588357+0000","last_change":"2024-07-17T08:31:39.830646+0000","last_active":"2024-07-17T08:31:41.588357+0000","last_peered":"2024-07-17T08:31:41.588357+0000","last_clean":"2024-07-17T08:31:41.588357+0000","last_became_active":"2024-07-17T08:31:39.830291+0000","last_became_peered":"2024-07-17T08:31:39.830291+0000","last_unstale":"2024-07-17T08:31:41.588357+0000","last_undegraded":"2024-07-17T08:31:41.588357+0000","last_fullsized":"2024-07-17T08:31:41.588357+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.475040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590818+0000","last_change":"2024-07-17T08:31:39.823282+0000","last_active":"2024-07-17T08:31:41.590818+0000","last_peered":"2024-07-17T08:31:41.590818+0000","last_clean":"2024-07-17T08:31:41.590818+0000","last_became_active":"2024-07-17T08:31:39.822975+0000","last_became_peered":"2024-07-17T08:31:39.822975+0000","last_unstale":"2024-07-17T08:31:41.590818+0000","last_undegraded":"2024-07-17T08:31:41.590818+0000","last_fullsized":"2024-07-17T08:31:41.590818+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:21:45.472080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591519+0000","last_change":"2024-07-17T08:31:39.824803+0000","last_active":"2024-07-17T08:31:41.591519+0000","last_peered":"2024-07-17T08:31:41.591519+0000","last_clean":"2024-07-17T08:31:41.591519+0000","last_became_active":"2024-07-17T08:31:39.824486+0000","last_became_peered":"2024-07-17T08:31:39.824486+0000","last_unstale":"2024-07-17T08:31:41.591519+0000","last_undegraded":"2024-07-17T08:31:41.591519+0000","last_fullsized":"2024-07-17T08:31:41.591519+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:07.257594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591373+0000","last_change":"2024-07-17T08:31:39.816790+0000","last_active":"2024-07-17T08:31:41.591373+0000","last_peered":"2024-07-17T08:31:41.591373+0000","last_clean":"2024-07-17T08:31:41.591373+0000","last_became_active":"2024-07-17T08:31:39.816433+0000","last_became_peered":"2024-07-17T08:31:39.816433+0000","last_unstale":"2024-07-17T08:31:41.591373+0000","last_undegraded":"2024-07-17T08:31:41.591373+0000","last_fullsized":"2024-07-17T08:31:41.591373+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:26:07.835685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589613+0000","last_change":"2024-07-17T08:31:39.825621+0000","last_active":"2024-07-17T08:31:41.589613+0000","last_peered":"2024-07-17T08:31:41.589613+0000","last_clean":"2024-07-17T08:31:41.589613+0000","last_became_active":"2024-07-17T08:31:39.825025+0000","last_became_peered":"2024-07-17T08:31:39.825025+0000","last_unstale":"2024-07-17T08:31:41.589613+0000","last_undegraded":"2024-07-17T08:31:41.589613+0000","last_fullsized":"2024-07-17T08:31:41.589613+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:00.813468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823486+0000","last_change":"2024-07-17T08:31:39.815870+0000","last_active":"2024-07-17T08:31:40.823486+0000","last_peered":"2024-07-17T08:31:40.823486+0000","last_clean":"2024-07-17T08:31:40.823486+0000","last_became_active":"2024-07-17T08:31:39.815392+0000","last_became_peered":"2024-07-17T08:31:39.815392+0000","last_unstale":"2024-07-17T08:31:40.823486+0000","last_undegraded":"2024-07-17T08:31:40.823486+0000","last_fullsized":"2024-07-17T08:31:40.823486+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:32.930492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823397+0000","last_change":"2024-07-17T08:31:39.821225+0000","last_active":"2024-07-17T08:31:40.823397+0000","last_peered":"2024-07-17T08:31:40.823397+0000","last_clean":"2024-07-17T08:31:40.823397+0000","last_became_active":"2024-07-17T08:31:39.820907+0000","last_became_peered":"2024-07-17T08:31:39.820907+0000","last_unstale":"2024-07-17T08:31:40.823397+0000","last_undegraded":"2024-07-17T08:31:40.823397+0000","last_fullsized":"2024-07-17T08:31:40.823397+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:34.335739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161347+0000","last_change":"2024-07-17T08:31:39.830948+0000","last_active":"2024-07-17T08:31:41.161347+0000","last_peered":"2024-07-17T08:31:41.161347+0000","last_clean":"2024-07-17T08:31:41.161347+0000","last_became_active":"2024-07-17T08:31:39.830680+0000","last_became_peered":"2024-07-17T08:31:39.830680+0000","last_unstale":"2024-07-17T08:31:41.161347+0000","last_undegraded":"2024-07-17T08:31:41.161347+0000","last_fullsized":"2024-07-17T08:31:41.161347+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:39.346717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821022+0000","last_change":"2024-07-17T08:31:39.824738+0000","last_active":"2024-07-17T08:31:40.821022+0000","last_peered":"2024-07-17T08:31:40.821022+0000","last_clean":"2024-07-17T08:31:40.821022+0000","last_became_active":"2024-07-17T08:31:39.824464+0000","last_became_peered":"2024-07-17T08:31:39.824464+0000","last_unstale":"2024-07-17T08:31:40.821022+0000","last_undegraded":"2024-07-17T08:31:40.821022+0000","last_fullsized":"2024-07-17T08:31:40.821022+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:45.832140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823131+0000","last_change":"2024-07-17T08:31:39.803470+0000","last_active":"2024-07-17T08:31:40.823131+0000","last_peered":"2024-07-17T08:31:40.823131+0000","last_clean":"2024-07-17T08:31:40.823131+0000","last_became_active":"2024-07-17T08:31:39.802960+0000","last_became_peered":"2024-07-17T08:31:39.802960+0000","last_unstale":"2024-07-17T08:31:40.823131+0000","last_undegraded":"2024-07-17T08:31:40.823131+0000","last_fullsized":"2024-07-17T08:31:40.823131+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:09.994022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588804+0000","last_change":"2024-07-17T08:31:39.827030+0000","last_active":"2024-07-17T08:31:41.588804+0000","last_peered":"2024-07-17T08:31:41.588804+0000","last_clean":"2024-07-17T08:31:41.588804+0000","last_became_active":"2024-07-17T08:31:39.823457+0000","last_became_peered":"2024-07-17T08:31:39.823457+0000","last_unstale":"2024-07-17T08:31:41.588804+0000","last_undegraded":"2024-07-17T08:31:41.588804+0000","last_fullsized":"2024-07-17T08:31:41.588804+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:30.031284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.162094+0000","last_change":"2024-07-17T08:31:39.829525+0000","last_active":"2024-07-17T08:31:41.162094+0000","last_peered":"2024-07-17T08:31:41.162094+0000","last_clean":"2024-07-17T08:31:41.162094+0000","last_became_active":"2024-07-17T08:31:39.829239+0000","last_became_peered":"2024-07-17T08:31:39.829239+0000","last_unstale":"2024-07-17T08:31:41.162094+0000","last_undegraded":"2024-07-17T08:31:41.162094+0000","last_fullsized":"2024-07-17T08:31:41.162094+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:24.251749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589840+0000","last_change":"2024-07-17T08:31:39.820594+0000","last_active":"2024-07-17T08:31:41.589840+0000","last_peered":"2024-07-17T08:31:41.589840+0000","last_clean":"2024-07-17T08:31:41.589840+0000","last_became_active":"2024-07-17T08:31:39.820304+0000","last_became_peered":"2024-07-17T08:31:39.820304+0000","last_unstale":"2024-07-17T08:31:41.589840+0000","last_undegraded":"2024-07-17T08:31:41.589840+0000","last_fullsized":"2024-07-17T08:31:41.589840+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:06:18.162316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587981+0000","last_change":"2024-07-17T08:31:39.819430+0000","last_active":"2024-07-17T08:31:41.587981+0000","last_peered":"2024-07-17T08:31:41.587981+0000","last_clean":"2024-07-17T08:31:41.587981+0000","last_became_active":"2024-07-17T08:31:39.818610+0000","last_became_peered":"2024-07-17T08:31:39.818610+0000","last_unstale":"2024-07-17T08:31:41.587981+0000","last_undegraded":"2024-07-17T08:31:41.587981+0000","last_fullsized":"2024-07-17T08:31:41.587981+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:41.158567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588163+0000","last_change":"2024-07-17T08:31:39.832236+0000","last_active":"2024-07-17T08:31:41.588163+0000","last_peered":"2024-07-17T08:31:41.588163+0000","last_clean":"2024-07-17T08:31:41.588163+0000","last_became_active":"2024-07-17T08:31:39.831761+0000","last_became_peered":"2024-07-17T08:31:39.831761+0000","last_unstale":"2024-07-17T08:31:41.588163+0000","last_undegraded":"2024-07-17T08:31:41.588163+0000","last_fullsized":"2024-07-17T08:31:41.588163+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:45.832140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822359+0000","last_change":"2024-07-17T08:31:39.808245+0000","last_active":"2024-07-17T08:31:40.822359+0000","last_peered":"2024-07-17T08:31:40.822359+0000","last_clean":"2024-07-17T08:31:40.822359+0000","last_became_active":"2024-07-17T08:31:39.807981+0000","last_became_peered":"2024-07-17T08:31:39.807981+0000","last_unstale":"2024-07-17T08:31:40.822359+0000","last_undegraded":"2024-07-17T08:31:40.822359+0000","last_fullsized":"2024-07-17T08:31:40.822359+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:12.354179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588813+0000","last_change":"2024-07-17T08:31:39.805478+0000","last_active":"2024-07-17T08:31:41.588813+0000","last_peered":"2024-07-17T08:31:41.588813+0000","last_clean":"2024-07-17T08:31:41.588813+0000","last_became_active":"2024-07-17T08:31:39.805195+0000","last_became_peered":"2024-07-17T08:31:39.805195+0000","last_unstale":"2024-07-17T08:31:41.588813+0000","last_undegraded":"2024-07-17T08:31:41.588813+0000","last_fullsized":"2024-07-17T08:31:41.588813+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:58:36.223909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161849+0000","last_change":"2024-07-17T08:31:39.828329+0000","last_active":"2024-07-17T08:31:41.161849+0000","last_peered":"2024-07-17T08:31:41.161849+0000","last_clean":"2024-07-17T08:31:41.161849+0000","last_became_active":"2024-07-17T08:31:39.827150+0000","last_became_peered":"2024-07-17T08:31:39.827150+0000","last_unstale":"2024-07-17T08:31:41.161849+0000","last_undegraded":"2024-07-17T08:31:41.161849+0000","last_fullsized":"2024-07-17T08:31:41.161849+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:43.414458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"205'4","reported_seq":18,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588171+0000","last_change":"2024-07-17T08:31:38.819115+0000","last_active":"2024-07-17T08:31:41.588171+0000","last_peered":"2024-07-17T08:31:41.588171+0000","last_clean":"2024-07-17T08:31:41.588171+0000","last_became_active":"2024-07-17T08:31:38.818794+0000","last_became_peered":"2024-07-17T08:31:38.818794+0000","last_unstale":"2024-07-17T08:31:41.588171+0000","last_undegraded":"2024-07-17T08:31:41.588171+0000","last_fullsized":"2024-07-17T08:31:41.588171+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:28.448310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823897+0000","last_change":"2024-07-17T08:31:39.823269+0000","last_active":"2024-07-17T08:31:40.823897+0000","last_peered":"2024-07-17T08:31:40.823897+0000","last_clean":"2024-07-17T08:31:40.823897+0000","last_became_active":"2024-07-17T08:31:39.823004+0000","last_became_peered":"2024-07-17T08:31:39.823004+0000","last_unstale":"2024-07-17T08:31:40.823897+0000","last_undegraded":"2024-07-17T08:31:40.823897+0000","last_fullsized":"2024-07-17T08:31:40.823897+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:44.491921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590718+0000","last_change":"2024-07-17T08:31:38.811816+0000","last_active":"2024-07-17T08:31:41.590718+0000","last_peered":"2024-07-17T08:31:41.590718+0000","last_clean":"2024-07-17T08:31:41.590718+0000","last_became_active":"2024-07-17T08:31:38.810227+0000","last_became_peered":"2024-07-17T08:31:38.810227+0000","last_unstale":"2024-07-17T08:31:41.590718+0000","last_undegraded":"2024-07-17T08:31:41.590718+0000","last_fullsized":"2024-07-17T08:31:41.590718+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:28.125965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589001+0000","last_change":"2024-07-17T08:31:39.829059+0000","last_active":"2024-07-17T08:31:41.589001+0000","last_peered":"2024-07-17T08:31:41.589001+0000","last_clean":"2024-07-17T08:31:41.589001+0000","last_became_active":"2024-07-17T08:31:39.828514+0000","last_became_peered":"2024-07-17T08:31:39.828514+0000","last_unstale":"2024-07-17T08:31:41.589001+0000","last_undegraded":"2024-07-17T08:31:41.589001+0000","last_fullsized":"2024-07-17T08:31:41.589001+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:52.662697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588739+0000","last_change":"2024-07-17T08:31:38.818035+0000","last_active":"2024-07-17T08:31:41.588739+0000","last_peered":"2024-07-17T08:31:41.588739+0000","last_clean":"2024-07-17T08:31:41.588739+0000","last_became_active":"2024-07-17T08:31:38.817709+0000","last_became_peered":"2024-07-17T08:31:38.817709+0000","last_unstale":"2024-07-17T08:31:41.588739+0000","last_undegraded":"2024-07-17T08:31:41.588739+0000","last_fullsized":"2024-07-17T08:31:41.588739+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:44.973632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822248+0000","last_change":"2024-07-17T08:31:39.814181+0000","last_active":"2024-07-17T08:31:40.822248+0000","last_peered":"2024-07-17T08:31:40.822248+0000","last_clean":"2024-07-17T08:31:40.822248+0000","last_became_active":"2024-07-17T08:31:39.813884+0000","last_became_peered":"2024-07-17T08:31:39.813884+0000","last_unstale":"2024-07-17T08:31:40.822248+0000","last_undegraded":"2024-07-17T08:31:40.822248+0000","last_fullsized":"2024-07-17T08:31:40.822248+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:07.419250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591222+0000","last_change":"2024-07-17T08:31:38.820462+0000","last_active":"2024-07-17T08:31:41.591222+0000","last_peered":"2024-07-17T08:31:41.591222+0000","last_clean":"2024-07-17T08:31:41.591222+0000","last_became_active":"2024-07-17T08:31:38.820060+0000","last_became_peered":"2024-07-17T08:31:38.820060+0000","last_unstale":"2024-07-17T08:31:41.591222+0000","last_undegraded":"2024-07-17T08:31:41.591222+0000","last_fullsized":"2024-07-17T08:31:41.591222+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:40.318529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589386+0000","last_change":"2024-07-17T08:31:39.815242+0000","last_active":"2024-07-17T08:31:41.589386+0000","last_peered":"2024-07-17T08:31:41.589386+0000","last_clean":"2024-07-17T08:31:41.589386+0000","last_became_active":"2024-07-17T08:31:39.814953+0000","last_became_peered":"2024-07-17T08:31:39.814953+0000","last_unstale":"2024-07-17T08:31:41.589386+0000","last_undegraded":"2024-07-17T08:31:41.589386+0000","last_fullsized":"2024-07-17T08:31:41.589386+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:13.155112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589839+0000","last_change":"2024-07-17T08:31:38.817373+0000","last_active":"2024-07-17T08:31:41.589839+0000","last_peered":"2024-07-17T08:31:41.589839+0000","last_clean":"2024-07-17T08:31:41.589839+0000","last_became_active":"2024-07-17T08:31:38.812250+0000","last_became_peered":"2024-07-17T08:31:38.812250+0000","last_unstale":"2024-07-17T08:31:41.589839+0000","last_undegraded":"2024-07-17T08:31:41.589839+0000","last_fullsized":"2024-07-17T08:31:41.589839+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:35.674590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822823+0000","last_change":"2024-07-17T08:31:39.804245+0000","last_active":"2024-07-17T08:31:40.822823+0000","last_peered":"2024-07-17T08:31:40.822823+0000","last_clean":"2024-07-17T08:31:40.822823+0000","last_became_active":"2024-07-17T08:31:39.803945+0000","last_became_peered":"2024-07-17T08:31:39.803945+0000","last_unstale":"2024-07-17T08:31:40.822823+0000","last_undegraded":"2024-07-17T08:31:40.822823+0000","last_fullsized":"2024-07-17T08:31:40.822823+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:42.061189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588073+0000","last_change":"2024-07-17T08:31:38.819410+0000","last_active":"2024-07-17T08:31:41.588073+0000","last_peered":"2024-07-17T08:31:41.588073+0000","last_clean":"2024-07-17T08:31:41.588073+0000","last_became_active":"2024-07-17T08:31:38.818839+0000","last_became_peered":"2024-07-17T08:31:38.818839+0000","last_unstale":"2024-07-17T08:31:41.588073+0000","last_undegraded":"2024-07-17T08:31:41.588073+0000","last_fullsized":"2024-07-17T08:31:41.588073+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:35.674590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822289+0000","last_change":"2024-07-17T08:31:39.819899+0000","last_active":"2024-07-17T08:31:40.822289+0000","last_peered":"2024-07-17T08:31:40.822289+0000","last_clean":"2024-07-17T08:31:40.822289+0000","last_became_active":"2024-07-17T08:31:39.818790+0000","last_became_peered":"2024-07-17T08:31:39.818790+0000","last_unstale":"2024-07-17T08:31:40.822289+0000","last_undegraded":"2024-07-17T08:31:40.822289+0000","last_fullsized":"2024-07-17T08:31:40.822289+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:16.094871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590106+0000","last_change":"2024-07-17T08:31:38.818130+0000","last_active":"2024-07-17T08:31:41.590106+0000","last_peered":"2024-07-17T08:31:41.590106+0000","last_clean":"2024-07-17T08:31:41.590106+0000","last_became_active":"2024-07-17T08:31:38.817779+0000","last_became_peered":"2024-07-17T08:31:38.817779+0000","last_unstale":"2024-07-17T08:31:41.590106+0000","last_undegraded":"2024-07-17T08:31:41.590106+0000","last_fullsized":"2024-07-17T08:31:41.590106+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:34.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161998+0000","last_change":"2024-07-17T08:31:39.832741+0000","last_active":"2024-07-17T08:31:41.161998+0000","last_peered":"2024-07-17T08:31:41.161998+0000","last_clean":"2024-07-17T08:31:41.161998+0000","last_became_active":"2024-07-17T08:31:39.832446+0000","last_became_peered":"2024-07-17T08:31:39.832446+0000","last_unstale":"2024-07-17T08:31:41.161998+0000","last_undegraded":"2024-07-17T08:31:41.161998+0000","last_fullsized":"2024-07-17T08:31:41.161998+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:59.375338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590873+0000","last_change":"2024-07-17T08:31:38.808755+0000","last_active":"2024-07-17T08:31:41.590873+0000","last_peered":"2024-07-17T08:31:41.590873+0000","last_clean":"2024-07-17T08:31:41.590873+0000","last_became_active":"2024-07-17T08:31:38.808084+0000","last_became_peered":"2024-07-17T08:31:38.808084+0000","last_unstale":"2024-07-17T08:31:41.590873+0000","last_undegraded":"2024-07-17T08:31:41.590873+0000","last_fullsized":"2024-07-17T08:31:41.590873+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:32.690011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590485+0000","last_change":"2024-07-17T08:31:39.817816+0000","last_active":"2024-07-17T08:31:41.590485+0000","last_peered":"2024-07-17T08:31:41.590485+0000","last_clean":"2024-07-17T08:31:41.590485+0000","last_became_active":"2024-07-17T08:31:39.817324+0000","last_became_peered":"2024-07-17T08:31:39.817324+0000","last_unstale":"2024-07-17T08:31:41.590485+0000","last_undegraded":"2024-07-17T08:31:41.590485+0000","last_fullsized":"2024-07-17T08:31:41.590485+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:18:16.348784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822179+0000","last_change":"2024-07-17T08:31:38.811624+0000","last_active":"2024-07-17T08:31:40.822179+0000","last_peered":"2024-07-17T08:31:40.822179+0000","last_clean":"2024-07-17T08:31:40.822179+0000","last_became_active":"2024-07-17T08:31:38.811313+0000","last_became_peered":"2024-07-17T08:31:38.811313+0000","last_unstale":"2024-07-17T08:31:40.822179+0000","last_undegraded":"2024-07-17T08:31:40.822179+0000","last_fullsized":"2024-07-17T08:31:40.822179+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:10:06.763657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591474+0000","last_change":"2024-07-17T08:31:39.835436+0000","last_active":"2024-07-17T08:31:41.591474+0000","last_peered":"2024-07-17T08:31:41.591474+0000","last_clean":"2024-07-17T08:31:41.591474+0000","last_became_active":"2024-07-17T08:31:39.834756+0000","last_became_peered":"2024-07-17T08:31:39.834756+0000","last_unstale":"2024-07-17T08:31:41.591474+0000","last_undegraded":"2024-07-17T08:31:41.591474+0000","last_fullsized":"2024-07-17T08:31:41.591474+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:12.079894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"205'3","reported_seq":17,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587645+0000","last_change":"2024-07-17T08:31:38.812205+0000","last_active":"2024-07-17T08:31:41.587645+0000","last_peered":"2024-07-17T08:31:41.587645+0000","last_clean":"2024-07-17T08:31:41.587645+0000","last_became_active":"2024-07-17T08:31:38.811933+0000","last_became_peered":"2024-07-17T08:31:38.811933+0000","last_unstale":"2024-07-17T08:31:41.587645+0000","last_undegraded":"2024-07-17T08:31:41.587645+0000","last_fullsized":"2024-07-17T08:31:41.587645+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:18:12.732868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823160+0000","last_change":"2024-07-17T08:31:39.825589+0000","last_active":"2024-07-17T08:31:40.823160+0000","last_peered":"2024-07-17T08:31:40.823160+0000","last_clean":"2024-07-17T08:31:40.823160+0000","last_became_active":"2024-07-17T08:31:39.823167+0000","last_became_peered":"2024-07-17T08:31:39.823167+0000","last_unstale":"2024-07-17T08:31:40.823160+0000","last_undegraded":"2024-07-17T08:31:40.823160+0000","last_fullsized":"2024-07-17T08:31:40.823160+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:45.432822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"205'6","reported_seq":20,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591186+0000","last_change":"2024-07-17T08:31:38.818164+0000","last_active":"2024-07-17T08:31:41.591186+0000","last_peered":"2024-07-17T08:31:41.591186+0000","last_clean":"2024-07-17T08:31:41.591186+0000","last_became_active":"2024-07-17T08:31:38.817822+0000","last_became_peered":"2024-07-17T08:31:38.817822+0000","last_unstale":"2024-07-17T08:31:41.591186+0000","last_undegraded":"2024-07-17T08:31:41.591186+0000","last_fullsized":"2024-07-17T08:31:41.591186+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:47.798226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821319+0000","last_change":"2024-07-17T08:31:39.822416+0000","last_active":"2024-07-17T08:31:40.821319+0000","last_peered":"2024-07-17T08:31:40.821319+0000","last_clean":"2024-07-17T08:31:40.821319+0000","last_became_active":"2024-07-17T08:31:39.822139+0000","last_became_peered":"2024-07-17T08:31:39.822139+0000","last_unstale":"2024-07-17T08:31:40.821319+0000","last_undegraded":"2024-07-17T08:31:40.821319+0000","last_fullsized":"2024-07-17T08:31:40.821319+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:46.672165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590643+0000","last_change":"2024-07-17T08:31:38.820600+0000","last_active":"2024-07-17T08:31:41.590643+0000","last_peered":"2024-07-17T08:31:41.590643+0000","last_clean":"2024-07-17T08:31:41.590643+0000","last_became_active":"2024-07-17T08:31:38.820243+0000","last_became_peered":"2024-07-17T08:31:38.820243+0000","last_unstale":"2024-07-17T08:31:41.590643+0000","last_undegraded":"2024-07-17T08:31:41.590643+0000","last_fullsized":"2024-07-17T08:31:41.590643+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:53.393603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587729+0000","last_change":"2024-07-17T08:31:39.806750+0000","last_active":"2024-07-17T08:31:41.587729+0000","last_peered":"2024-07-17T08:31:41.587729+0000","last_clean":"2024-07-17T08:31:41.587729+0000","last_became_active":"2024-07-17T08:31:39.806445+0000","last_became_peered":"2024-07-17T08:31:39.806445+0000","last_unstale":"2024-07-17T08:31:41.587729+0000","last_undegraded":"2024-07-17T08:31:41.587729+0000","last_fullsized":"2024-07-17T08:31:41.587729+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:36.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822582+0000","last_change":"2024-07-17T08:31:38.815059+0000","last_active":"2024-07-17T08:31:40.822582+0000","last_peered":"2024-07-17T08:31:40.822582+0000","last_clean":"2024-07-17T08:31:40.822582+0000","last_became_active":"2024-07-17T08:31:38.812349+0000","last_became_peered":"2024-07-17T08:31:38.812349+0000","last_unstale":"2024-07-17T08:31:40.822582+0000","last_undegraded":"2024-07-17T08:31:40.822582+0000","last_fullsized":"2024-07-17T08:31:40.822582+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:45:53.901805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590377+0000","last_change":"2024-07-17T08:31:39.818766+0000","last_active":"2024-07-17T08:31:41.590377+0000","last_peered":"2024-07-17T08:31:41.590377+0000","last_clean":"2024-07-17T08:31:41.590377+0000","last_became_active":"2024-07-17T08:31:39.818405+0000","last_became_peered":"2024-07-17T08:31:39.818405+0000","last_unstale":"2024-07-17T08:31:41.590377+0000","last_undegraded":"2024-07-17T08:31:41.590377+0000","last_fullsized":"2024-07-17T08:31:41.590377+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:04.170195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587802+0000","last_change":"2024-07-17T08:31:38.820427+0000","last_active":"2024-07-17T08:31:41.587802+0000","last_peered":"2024-07-17T08:31:41.587802+0000","last_clean":"2024-07-17T08:31:41.587802+0000","last_became_active":"2024-07-17T08:31:38.820096+0000","last_became_peered":"2024-07-17T08:31:38.820096+0000","last_unstale":"2024-07-17T08:31:41.587802+0000","last_undegraded":"2024-07-17T08:31:41.587802+0000","last_fullsized":"2024-07-17T08:31:41.587802+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:52.639287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590586+0000","last_change":"2024-07-17T08:31:39.809525+0000","last_active":"2024-07-17T08:31:41.590586+0000","last_peered":"2024-07-17T08:31:41.590586+0000","last_clean":"2024-07-17T08:31:41.590586+0000","last_became_active":"2024-07-17T08:31:39.809213+0000","last_became_peered":"2024-07-17T08:31:39.809213+0000","last_unstale":"2024-07-17T08:31:41.590586+0000","last_undegraded":"2024-07-17T08:31:41.590586+0000","last_fullsized":"2024-07-17T08:31:41.590586+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:56:35.082577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"205'8","reported_seq":22,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587896+0000","last_change":"2024-07-17T08:31:38.809345+0000","last_active":"2024-07-17T08:31:41.587896+0000","last_peered":"2024-07-17T08:31:41.587896+0000","last_clean":"2024-07-17T08:31:41.587896+0000","last_became_active":"2024-07-17T08:31:38.808447+0000","last_became_peered":"2024-07-17T08:31:38.808447+0000","last_unstale":"2024-07-17T08:31:41.587896+0000","last_undegraded":"2024-07-17T08:31:41.587896+0000","last_fullsized":"2024-07-17T08:31:41.587896+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:58.014602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822759+0000","last_change":"2024-07-17T08:31:39.809364+0000","last_active":"2024-07-17T08:31:40.822759+0000","last_peered":"2024-07-17T08:31:40.822759+0000","last_clean":"2024-07-17T08:31:40.822759+0000","last_became_active":"2024-07-17T08:31:39.806864+0000","last_became_peered":"2024-07-17T08:31:39.806864+0000","last_unstale":"2024-07-17T08:31:40.822759+0000","last_undegraded":"2024-07-17T08:31:40.822759+0000","last_fullsized":"2024-07-17T08:31:40.822759+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:16.393319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589290+0000","last_change":"2024-07-17T08:31:38.815102+0000","last_active":"2024-07-17T08:31:41.589290+0000","last_peered":"2024-07-17T08:31:41.589290+0000","last_clean":"2024-07-17T08:31:41.589290+0000","last_became_active":"2024-07-17T08:31:38.814791+0000","last_became_peered":"2024-07-17T08:31:38.814791+0000","last_unstale":"2024-07-17T08:31:41.589290+0000","last_undegraded":"2024-07-17T08:31:41.589290+0000","last_fullsized":"2024-07-17T08:31:41.589290+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:34.762772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590160+0000","last_change":"2024-07-17T08:31:39.819374+0000","last_active":"2024-07-17T08:31:41.590160+0000","last_peered":"2024-07-17T08:31:41.590160+0000","last_clean":"2024-07-17T08:31:41.590160+0000","last_became_active":"2024-07-17T08:31:39.818158+0000","last_became_peered":"2024-07-17T08:31:39.818158+0000","last_unstale":"2024-07-17T08:31:41.590160+0000","last_undegraded":"2024-07-17T08:31:41.590160+0000","last_fullsized":"2024-07-17T08:31:41.590160+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:20:54.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588377+0000","last_change":"2024-07-17T08:31:38.819774+0000","last_active":"2024-07-17T08:31:41.588377+0000","last_peered":"2024-07-17T08:31:41.588377+0000","last_clean":"2024-07-17T08:31:41.588377+0000","last_became_active":"2024-07-17T08:31:38.818991+0000","last_became_peered":"2024-07-17T08:31:38.818991+0000","last_unstale":"2024-07-17T08:31:41.588377+0000","last_undegraded":"2024-07-17T08:31:41.588377+0000","last_fullsized":"2024-07-17T08:31:41.588377+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:58.591656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589155+0000","last_change":"2024-07-17T08:31:39.826773+0000","last_active":"2024-07-17T08:31:41.589155+0000","last_peered":"2024-07-17T08:31:41.589155+0000","last_clean":"2024-07-17T08:31:41.589155+0000","last_became_active":"2024-07-17T08:31:39.823063+0000","last_became_peered":"2024-07-17T08:31:39.823063+0000","last_unstale":"2024-07-17T08:31:41.589155+0000","last_undegraded":"2024-07-17T08:31:41.589155+0000","last_fullsized":"2024-07-17T08:31:41.589155+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:29.302320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823608+0000","last_change":"2024-07-17T08:31:38.808909+0000","last_active":"2024-07-17T08:31:40.823608+0000","last_peered":"2024-07-17T08:31:40.823608+0000","last_clean":"2024-07-17T08:31:40.823608+0000","last_became_active":"2024-07-17T08:31:38.808527+0000","last_became_peered":"2024-07-17T08:31:38.808527+0000","last_unstale":"2024-07-17T08:31:40.823608+0000","last_undegraded":"2024-07-17T08:31:40.823608+0000","last_fullsized":"2024-07-17T08:31:40.823608+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:20.292751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823329+0000","last_change":"2024-07-17T08:31:39.818523+0000","last_active":"2024-07-17T08:31:40.823329+0000","last_peered":"2024-07-17T08:31:40.823329+0000","last_clean":"2024-07-17T08:31:40.823329+0000","last_became_active":"2024-07-17T08:31:39.818260+0000","last_became_peered":"2024-07-17T08:31:39.818260+0000","last_unstale":"2024-07-17T08:31:40.823329+0000","last_undegraded":"2024-07-17T08:31:40.823329+0000","last_fullsized":"2024-07-17T08:31:40.823329+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:39.178144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588184+0000","last_change":"2024-07-17T08:31:38.815982+0000","last_active":"2024-07-17T08:31:41.588184+0000","last_peered":"2024-07-17T08:31:41.588184+0000","last_clean":"2024-07-17T08:31:41.588184+0000","last_became_active":"2024-07-17T08:31:38.815678+0000","last_became_peered":"2024-07-17T08:31:38.815678+0000","last_unstale":"2024-07-17T08:31:41.588184+0000","last_undegraded":"2024-07-17T08:31:41.588184+0000","last_fullsized":"2024-07-17T08:31:41.588184+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:23.606160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821130+0000","last_change":"2024-07-17T08:31:39.819106+0000","last_active":"2024-07-17T08:31:40.821130+0000","last_peered":"2024-07-17T08:31:40.821130+0000","last_clean":"2024-07-17T08:31:40.821130+0000","last_became_active":"2024-07-17T08:31:39.818816+0000","last_became_peered":"2024-07-17T08:31:39.818816+0000","last_unstale":"2024-07-17T08:31:40.821130+0000","last_undegraded":"2024-07-17T08:31:40.821130+0000","last_fullsized":"2024-07-17T08:31:40.821130+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:52.662697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590901+0000","last_change":"2024-07-17T08:31:38.819807+0000","last_active":"2024-07-17T08:31:41.590901+0000","last_peered":"2024-07-17T08:31:41.590901+0000","last_clean":"2024-07-17T08:31:41.590901+0000","last_became_active":"2024-07-17T08:31:38.819330+0000","last_became_peered":"2024-07-17T08:31:38.819330+0000","last_unstale":"2024-07-17T08:31:41.590901+0000","last_undegraded":"2024-07-17T08:31:41.590901+0000","last_fullsized":"2024-07-17T08:31:41.590901+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:50.816517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587442+0000","last_change":"2024-07-17T08:31:39.830116+0000","last_active":"2024-07-17T08:31:41.587442+0000","last_peered":"2024-07-17T08:31:41.587442+0000","last_clean":"2024-07-17T08:31:41.587442+0000","last_became_active":"2024-07-17T08:31:39.828879+0000","last_became_peered":"2024-07-17T08:31:39.828879+0000","last_unstale":"2024-07-17T08:31:41.587442+0000","last_undegraded":"2024-07-17T08:31:41.587442+0000","last_fullsized":"2024-07-17T08:31:41.587442+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:36:33.236703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588361+0000","last_change":"2024-07-17T08:31:38.819129+0000","last_active":"2024-07-17T08:31:41.588361+0000","last_peered":"2024-07-17T08:31:41.588361+0000","last_clean":"2024-07-17T08:31:41.588361+0000","last_became_active":"2024-07-17T08:31:38.818368+0000","last_became_peered":"2024-07-17T08:31:38.818368+0000","last_unstale":"2024-07-17T08:31:41.588361+0000","last_undegraded":"2024-07-17T08:31:41.588361+0000","last_fullsized":"2024-07-17T08:31:41.588361+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:28.125965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587749+0000","last_change":"2024-07-17T08:31:39.834431+0000","last_active":"2024-07-17T08:31:41.587749+0000","last_peered":"2024-07-17T08:31:41.587749+0000","last_clean":"2024-07-17T08:31:41.587749+0000","last_became_active":"2024-07-17T08:31:39.834136+0000","last_became_peered":"2024-07-17T08:31:39.834136+0000","last_unstale":"2024-07-17T08:31:41.587749+0000","last_undegraded":"2024-07-17T08:31:41.587749+0000","last_fullsized":"2024-07-17T08:31:41.587749+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:02.330164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822620+0000","last_change":"2024-07-17T08:31:38.809351+0000","last_active":"2024-07-17T08:31:40.822620+0000","last_peered":"2024-07-17T08:31:40.822620+0000","last_clean":"2024-07-17T08:31:40.822620+0000","last_became_active":"2024-07-17T08:31:38.809079+0000","last_became_peered":"2024-07-17T08:31:38.809079+0000","last_unstale":"2024-07-17T08:31:40.822620+0000","last_undegraded":"2024-07-17T08:31:40.822620+0000","last_fullsized":"2024-07-17T08:31:40.822620+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:09.389453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589765+0000","last_change":"2024-07-17T08:31:39.806959+0000","last_active":"2024-07-17T08:31:41.589765+0000","last_peered":"2024-07-17T08:31:41.589765+0000","last_clean":"2024-07-17T08:31:41.589765+0000","last_became_active":"2024-07-17T08:31:39.806674+0000","last_became_peered":"2024-07-17T08:31:39.806674+0000","last_unstale":"2024-07-17T08:31:41.589765+0000","last_undegraded":"2024-07-17T08:31:41.589765+0000","last_fullsized":"2024-07-17T08:31:41.589765+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:31:41.363288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821762+0000","last_change":"2024-07-17T08:31:38.809809+0000","last_active":"2024-07-17T08:31:40.821762+0000","last_peered":"2024-07-17T08:31:40.821762+0000","last_clean":"2024-07-17T08:31:40.821762+0000","last_became_active":"2024-07-17T08:31:38.807439+0000","last_became_peered":"2024-07-17T08:31:38.807439+0000","last_unstale":"2024-07-17T08:31:40.821762+0000","last_undegraded":"2024-07-17T08:31:40.821762+0000","last_fullsized":"2024-07-17T08:31:40.821762+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:20.523594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823197+0000","last_change":"2024-07-17T08:31:39.824716+0000","last_active":"2024-07-17T08:31:40.823197+0000","last_peered":"2024-07-17T08:31:40.823197+0000","last_clean":"2024-07-17T08:31:40.823197+0000","last_became_active":"2024-07-17T08:31:39.822968+0000","last_became_peered":"2024-07-17T08:31:39.822968+0000","last_unstale":"2024-07-17T08:31:40.823197+0000","last_undegraded":"2024-07-17T08:31:40.823197+0000","last_fullsized":"2024-07-17T08:31:40.823197+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:41.543111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588423+0000","last_change":"2024-07-17T08:31:38.811899+0000","last_active":"2024-07-17T08:31:41.588423+0000","last_peered":"2024-07-17T08:31:41.588423+0000","last_clean":"2024-07-17T08:31:41.588423+0000","last_became_active":"2024-07-17T08:31:38.811557+0000","last_became_peered":"2024-07-17T08:31:38.811557+0000","last_unstale":"2024-07-17T08:31:41.588423+0000","last_undegraded":"2024-07-17T08:31:41.588423+0000","last_fullsized":"2024-07-17T08:31:41.588423+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:49.729504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821517+0000","last_change":"2024-07-17T08:31:39.814407+0000","last_active":"2024-07-17T08:31:40.821517+0000","last_peered":"2024-07-17T08:31:40.821517+0000","last_clean":"2024-07-17T08:31:40.821517+0000","last_became_active":"2024-07-17T08:31:39.814139+0000","last_became_peered":"2024-07-17T08:31:39.814139+0000","last_unstale":"2024-07-17T08:31:40.821517+0000","last_undegraded":"2024-07-17T08:31:40.821517+0000","last_fullsized":"2024-07-17T08:31:40.821517+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:29.302320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823755+0000","last_change":"2024-07-17T08:31:38.815082+0000","last_active":"2024-07-17T08:31:40.823755+0000","last_peered":"2024-07-17T08:31:40.823755+0000","last_clean":"2024-07-17T08:31:40.823755+0000","last_became_active":"2024-07-17T08:31:38.812303+0000","last_became_peered":"2024-07-17T08:31:38.812303+0000","last_unstale":"2024-07-17T08:31:40.823755+0000","last_undegraded":"2024-07-17T08:31:40.823755+0000","last_fullsized":"2024-07-17T08:31:40.823755+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:04.222394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.162243+0000","last_change":"2024-07-17T08:31:39.830815+0000","last_active":"2024-07-17T08:31:41.162243+0000","last_peered":"2024-07-17T08:31:41.162243+0000","last_clean":"2024-07-17T08:31:41.162243+0000","last_became_active":"2024-07-17T08:31:39.827586+0000","last_became_peered":"2024-07-17T08:31:39.827586+0000","last_unstale":"2024-07-17T08:31:41.162243+0000","last_undegraded":"2024-07-17T08:31:41.162243+0000","last_fullsized":"2024-07-17T08:31:41.162243+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:10.822211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590428+0000","last_change":"2024-07-17T08:31:38.816782+0000","last_active":"2024-07-17T08:31:41.590428+0000","last_peered":"2024-07-17T08:31:41.590428+0000","last_clean":"2024-07-17T08:31:41.590428+0000","last_became_active":"2024-07-17T08:31:38.815186+0000","last_became_peered":"2024-07-17T08:31:38.815186+0000","last_unstale":"2024-07-17T08:31:41.590428+0000","last_undegraded":"2024-07-17T08:31:41.590428+0000","last_fullsized":"2024-07-17T08:31:41.590428+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:25.972011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823462+0000","last_change":"2024-07-17T08:31:39.818433+0000","last_active":"2024-07-17T08:31:40.823462+0000","last_peered":"2024-07-17T08:31:40.823462+0000","last_clean":"2024-07-17T08:31:40.823462+0000","last_became_active":"2024-07-17T08:31:39.816321+0000","last_became_peered":"2024-07-17T08:31:39.816321+0000","last_unstale":"2024-07-17T08:31:40.823462+0000","last_undegraded":"2024-07-17T08:31:40.823462+0000","last_fullsized":"2024-07-17T08:31:40.823462+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:09.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589920+0000","last_change":"2024-07-17T08:31:38.805212+0000","last_active":"2024-07-17T08:31:41.589920+0000","last_peered":"2024-07-17T08:31:41.589920+0000","last_clean":"2024-07-17T08:31:41.589920+0000","last_became_active":"2024-07-17T08:31:38.804912+0000","last_became_peered":"2024-07-17T08:31:38.804912+0000","last_unstale":"2024-07-17T08:31:41.589920+0000","last_undegraded":"2024-07-17T08:31:41.589920+0000","last_fullsized":"2024-07-17T08:31:41.589920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:06.697072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821741+0000","last_change":"2024-07-17T08:31:39.815480+0000","last_active":"2024-07-17T08:31:40.821741+0000","last_peered":"2024-07-17T08:31:40.821741+0000","last_clean":"2024-07-17T08:31:40.821741+0000","last_became_active":"2024-07-17T08:31:39.815206+0000","last_became_peered":"2024-07-17T08:31:39.815206+0000","last_unstale":"2024-07-17T08:31:40.821741+0000","last_undegraded":"2024-07-17T08:31:40.821741+0000","last_fullsized":"2024-07-17T08:31:40.821741+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:38:15.504242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161313+0000","last_change":"2024-07-17T08:31:38.817064+0000","last_active":"2024-07-17T08:31:41.161313+0000","last_peered":"2024-07-17T08:31:41.161313+0000","last_clean":"2024-07-17T08:31:41.161313+0000","last_became_active":"2024-07-17T08:31:38.816291+0000","last_became_peered":"2024-07-17T08:31:38.816291+0000","last_unstale":"2024-07-17T08:31:41.161313+0000","last_undegraded":"2024-07-17T08:31:41.161313+0000","last_fullsized":"2024-07-17T08:31:41.161313+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:28.650063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590200+0000","last_change":"2024-07-17T08:31:39.816913+0000","last_active":"2024-07-17T08:31:41.590200+0000","last_peered":"2024-07-17T08:31:41.590200+0000","last_clean":"2024-07-17T08:31:41.590200+0000","last_became_active":"2024-07-17T08:31:39.816604+0000","last_became_peered":"2024-07-17T08:31:39.816604+0000","last_unstale":"2024-07-17T08:31:41.590200+0000","last_undegraded":"2024-07-17T08:31:41.590200+0000","last_fullsized":"2024-07-17T08:31:41.590200+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:42.596428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161133+0000","last_change":"2024-07-17T08:31:38.812443+0000","last_active":"2024-07-17T08:31:41.161133+0000","last_peered":"2024-07-17T08:31:41.161133+0000","last_clean":"2024-07-17T08:31:41.161133+0000","last_became_active":"2024-07-17T08:31:38.812147+0000","last_became_peered":"2024-07-17T08:31:38.812147+0000","last_unstale":"2024-07-17T08:31:41.161133+0000","last_undegraded":"2024-07-17T08:31:41.161133+0000","last_fullsized":"2024-07-17T08:31:41.161133+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:51.869401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822456+0000","last_change":"2024-07-17T08:31:39.813472+0000","last_active":"2024-07-17T08:31:40.822456+0000","last_peered":"2024-07-17T08:31:40.822456+0000","last_clean":"2024-07-17T08:31:40.822456+0000","last_became_active":"2024-07-17T08:31:39.813173+0000","last_became_peered":"2024-07-17T08:31:39.813173+0000","last_unstale":"2024-07-17T08:31:40.822456+0000","last_undegraded":"2024-07-17T08:31:40.822456+0000","last_fullsized":"2024-07-17T08:31:40.822456+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:31.724471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822661+0000","last_change":"2024-07-17T08:31:38.804439+0000","last_active":"2024-07-17T08:31:40.822661+0000","last_peered":"2024-07-17T08:31:40.822661+0000","last_clean":"2024-07-17T08:31:40.822661+0000","last_became_active":"2024-07-17T08:31:38.803725+0000","last_became_peered":"2024-07-17T08:31:38.803725+0000","last_unstale":"2024-07-17T08:31:40.822661+0000","last_undegraded":"2024-07-17T08:31:40.822661+0000","last_fullsized":"2024-07-17T08:31:40.822661+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:17.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161427+0000","last_change":"2024-07-17T08:31:39.829402+0000","last_active":"2024-07-17T08:31:41.161427+0000","last_peered":"2024-07-17T08:31:41.161427+0000","last_clean":"2024-07-17T08:31:41.161427+0000","last_became_active":"2024-07-17T08:31:39.829135+0000","last_became_peered":"2024-07-17T08:31:39.829135+0000","last_unstale":"2024-07-17T08:31:41.161427+0000","last_undegraded":"2024-07-17T08:31:41.161427+0000","last_fullsized":"2024-07-17T08:31:41.161427+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:24.298669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822033+0000","last_change":"2024-07-17T08:31:38.806916+0000","last_active":"2024-07-17T08:31:40.822033+0000","last_peered":"2024-07-17T08:31:40.822033+0000","last_clean":"2024-07-17T08:31:40.822033+0000","last_became_active":"2024-07-17T08:31:38.806157+0000","last_became_peered":"2024-07-17T08:31:38.806157+0000","last_unstale":"2024-07-17T08:31:40.822033+0000","last_undegraded":"2024-07-17T08:31:40.822033+0000","last_fullsized":"2024-07-17T08:31:40.822033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:14.516253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822547+0000","last_change":"2024-07-17T08:31:39.811208+0000","last_active":"2024-07-17T08:31:40.822547+0000","last_peered":"2024-07-17T08:31:40.822547+0000","last_clean":"2024-07-17T08:31:40.822547+0000","last_became_active":"2024-07-17T08:31:39.810572+0000","last_became_peered":"2024-07-17T08:31:39.810572+0000","last_unstale":"2024-07-17T08:31:40.822547+0000","last_undegraded":"2024-07-17T08:31:40.822547+0000","last_fullsized":"2024-07-17T08:31:40.822547+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:29.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588005+0000","last_change":"2024-07-17T08:31:38.809339+0000","last_active":"2024-07-17T08:31:41.588005+0000","last_peered":"2024-07-17T08:31:41.588005+0000","last_clean":"2024-07-17T08:31:41.588005+0000","last_became_active":"2024-07-17T08:31:38.809017+0000","last_became_peered":"2024-07-17T08:31:38.809017+0000","last_unstale":"2024-07-17T08:31:41.588005+0000","last_undegraded":"2024-07-17T08:31:41.588005+0000","last_fullsized":"2024-07-17T08:31:41.588005+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:06.697072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"197'2497","reported_seq":3296,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161325+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:31:41.161325+0000","last_peered":"2024-07-17T08:31:41.161325+0000","last_clean":"2024-07-17T08:31:41.161325+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:31:41.161325+0000","last_undegraded":"2024-07-17T08:31:41.161325+0000","last_fullsized":"2024-07-17T08:31:41.161325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":335,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":335,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2497,"ondisk_log_size":2497,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543298,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60644,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311196,"statfs":{"total":96636764160,"available":96574664704,"internally_reserved":0,"allocated":3170304,"data_stored":1987615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42020,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329820,"statfs":{"total":96636764160,"available":96593735680,"internally_reserved":0,"allocated":2580480,"data_stored":1397317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510594,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":2580480,"data_stored":1391613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543299,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2580480,"data_stored":1391625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510593,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2572288,"data_stored":1390139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2580480,"data_stored":1397829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510594,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1397247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510593,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56492,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315348,"statfs":{"total":96636764160,"available":96578916352,"internally_reserved":0,"allocated":3174400,"data_stored":1981075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:31:44.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 -- 172.21.15.44:0/320135889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c4153a50 msgr2=0x7f73c4159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 --2- 172.21.15.44:0/320135889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c4153a50 0x7f73c4159b00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f73c40aeda0 tx=0x7f73b402a5f0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 -- 172.21.15.44:0/320135889 shutdown_connections 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 --2- 172.21.15.44:0/320135889 >> v2:172.21.15.175:3300/0 conn(0x7f73c40acaf0 0x7f73c406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 --2- 172.21.15.44:0/320135889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c4153a50 0x7f73c4159b00 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 -- 172.21.15.44:0/320135889 >> 172.21.15.44:0/320135889 conn(0x7f73c40b0fb0 msgr2=0x7f73c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 -- 172.21.15.44:0/320135889 shutdown_connections 2024-07-17T08:31:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.534+0000 7f73c9a00640 1 -- 172.21.15.44:0/320135889 wait complete. 2024-07-17T08:31:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.536+0000 7f73c9a00640 1 Processor -- start 2024-07-17T08:31:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.536+0000 7f73c9a00640 1 -- start start 2024-07-17T08:31:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c40acaf0 0x7f73c40988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73c4153a50 0x7f73c41429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73c406f710 con 0x7f73c415a040 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4154040 con 0x7f73c40acaf0 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73c406f890 con 0x7f73c4153a50 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50672/0 (socket says 172.21.15.44:50672) 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.537+0000 7f73c3400640 1 -- 172.21.15.44:0/4290640487 learned_addr learned my addr 172.21.15.44:0/4290640487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c2a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c40acaf0 0x7f73c40988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c2000640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73c4153a50 0x7f73c41429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 -- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73c4153a50 msgr2=0x7f73c41429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73c4153a50 0x7f73c41429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 -- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c40acaf0 msgr2=0x7f73c40988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c40acaf0 0x7f73c40988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 -- 172.21.15.44:0/4290640487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c4143050 con 0x7f73c415a040 2024-07-17T08:31:44.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c3400640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ddf0 tx=0x7f73bc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc004d20 con 0x7f73c415a040 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73bc00be40 con 0x7f73c415a040 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00e8a0 con 0x7f73c415a040 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.538+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c4072760 con 0x7f73c415a040 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.539+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c4072d20 con 0x7f73c415a040 2024-07-17T08:31:44.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.540+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c406e090 con 0x7f73c415a040 2024-07-17T08:31:44.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.541+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f73bc00ea40 con 0x7f73c415a040 2024-07-17T08:31:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.542+0000 7f73baa00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 0x7f73a0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.542+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f73bc0991f0 con 0x7f73c415a040 2024-07-17T08:31:44.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.543+0000 7f73c2a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 0x7f73a0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:44.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.544+0000 7f73c2a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 0x7f73a0079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73b4033000 tx=0x7f73b4004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:44.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.546+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f73bc0622e0 con 0x7f73c415a040 2024-07-17T08:31:44.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.744+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 --> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73c41e3470 con 0x7f73a0076bb0 2024-07-17T08:31:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.757+0000 7f73baa00640 1 -- 172.21.15.44:0/4290640487 <== mgr.7280 v2:172.21.15.175:6834/1104780009 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f73c41e3470 con 0x7f73a0076bb0 2024-07-17T08:31:44.760 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:31:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 msgr2=0x7f73a0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 0x7f73a0079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f73b4033000 tx=0x7f73b4004730 comp rx=0 tx=0).stop 2024-07-17T08:31:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 msgr2=0x7f73c41e2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ddf0 tx=0x7f73bc00b730 comp rx=0 tx=0).stop 2024-07-17T08:31:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 shutdown_connections 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73c415a040 0x7f73c41e2f30 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73c4153a50 0x7f73c41429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f73a0076bb0 0x7f73a0079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 --2- 172.21.15.44:0/4290640487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73c40acaf0 0x7f73c40988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 >> 172.21.15.44:0/4290640487 conn(0x7f73c40b0fb0 msgr2=0x7f73c40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 shutdown_connections 2024-07-17T08:31:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:44.763+0000 7f73c9a00640 1 -- 172.21.15.44:0/4290640487 wait complete. 2024-07-17T08:31:44.782 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:31:43.651795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":380,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2536,"ondisk_log_size":2536,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422068,"kb_used_data":21304,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":754552652,"statfs":{"total":773094113280,"available":772661915648,"internally_reserved":0,"allocated":21815296,"data_stored":12334460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":410242608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.887505"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821976+0000","last_change":"2024-07-17T08:31:39.823959+0000","last_active":"2024-07-17T08:31:40.821976+0000","last_peered":"2024-07-17T08:31:40.821976+0000","last_clean":"2024-07-17T08:31:40.821976+0000","last_became_active":"2024-07-17T08:31:39.823622+0000","last_became_peered":"2024-07-17T08:31:39.823622+0000","last_unstale":"2024-07-17T08:31:40.821976+0000","last_undegraded":"2024-07-17T08:31:40.821976+0000","last_fullsized":"2024-07-17T08:31:40.821976+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:09.489756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821990+0000","last_change":"2024-07-17T08:31:39.834589+0000","last_active":"2024-07-17T08:31:40.821990+0000","last_peered":"2024-07-17T08:31:40.821990+0000","last_clean":"2024-07-17T08:31:40.821990+0000","last_became_active":"2024-07-17T08:31:39.834318+0000","last_became_peered":"2024-07-17T08:31:39.834318+0000","last_unstale":"2024-07-17T08:31:40.821990+0000","last_undegraded":"2024-07-17T08:31:40.821990+0000","last_fullsized":"2024-07-17T08:31:40.821990+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:28.977482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823377+0000","last_change":"2024-07-17T08:31:39.810119+0000","last_active":"2024-07-17T08:31:40.823377+0000","last_peered":"2024-07-17T08:31:40.823377+0000","last_clean":"2024-07-17T08:31:40.823377+0000","last_became_active":"2024-07-17T08:31:39.809661+0000","last_became_peered":"2024-07-17T08:31:39.809661+0000","last_unstale":"2024-07-17T08:31:40.823377+0000","last_undegraded":"2024-07-17T08:31:40.823377+0000","last_fullsized":"2024-07-17T08:31:40.823377+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:02:39.114234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588980+0000","last_change":"2024-07-17T08:31:39.829466+0000","last_active":"2024-07-17T08:31:41.588980+0000","last_peered":"2024-07-17T08:31:41.588980+0000","last_clean":"2024-07-17T08:31:41.588980+0000","last_became_active":"2024-07-17T08:31:39.828590+0000","last_became_peered":"2024-07-17T08:31:39.828590+0000","last_unstale":"2024-07-17T08:31:41.588980+0000","last_undegraded":"2024-07-17T08:31:41.588980+0000","last_fullsized":"2024-07-17T08:31:41.588980+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:52:27.370045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589516+0000","last_change":"2024-07-17T08:31:39.825045+0000","last_active":"2024-07-17T08:31:41.589516+0000","last_peered":"2024-07-17T08:31:41.589516+0000","last_clean":"2024-07-17T08:31:41.589516+0000","last_became_active":"2024-07-17T08:31:39.824424+0000","last_became_peered":"2024-07-17T08:31:39.824424+0000","last_unstale":"2024-07-17T08:31:41.589516+0000","last_undegraded":"2024-07-17T08:31:41.589516+0000","last_fullsized":"2024-07-17T08:31:41.589516+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:23.537611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589296+0000","last_change":"2024-07-17T08:31:39.805661+0000","last_active":"2024-07-17T08:31:41.589296+0000","last_peered":"2024-07-17T08:31:41.589296+0000","last_clean":"2024-07-17T08:31:41.589296+0000","last_became_active":"2024-07-17T08:31:39.805408+0000","last_became_peered":"2024-07-17T08:31:39.805408+0000","last_unstale":"2024-07-17T08:31:41.589296+0000","last_undegraded":"2024-07-17T08:31:41.589296+0000","last_fullsized":"2024-07-17T08:31:41.589296+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:31.040807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822018+0000","last_change":"2024-07-17T08:31:39.820719+0000","last_active":"2024-07-17T08:31:40.822018+0000","last_peered":"2024-07-17T08:31:40.822018+0000","last_clean":"2024-07-17T08:31:40.822018+0000","last_became_active":"2024-07-17T08:31:39.820422+0000","last_became_peered":"2024-07-17T08:31:39.820422+0000","last_unstale":"2024-07-17T08:31:40.822018+0000","last_undegraded":"2024-07-17T08:31:40.822018+0000","last_fullsized":"2024-07-17T08:31:40.822018+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:10.937482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161506+0000","last_change":"2024-07-17T08:31:39.836492+0000","last_active":"2024-07-17T08:31:41.161506+0000","last_peered":"2024-07-17T08:31:41.161506+0000","last_clean":"2024-07-17T08:31:41.161506+0000","last_became_active":"2024-07-17T08:31:39.836108+0000","last_became_peered":"2024-07-17T08:31:39.836108+0000","last_unstale":"2024-07-17T08:31:41.161506+0000","last_undegraded":"2024-07-17T08:31:41.161506+0000","last_fullsized":"2024-07-17T08:31:41.161506+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:36.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823514+0000","last_change":"2024-07-17T08:31:39.817001+0000","last_active":"2024-07-17T08:31:40.823514+0000","last_peered":"2024-07-17T08:31:40.823514+0000","last_clean":"2024-07-17T08:31:40.823514+0000","last_became_active":"2024-07-17T08:31:39.816712+0000","last_became_peered":"2024-07-17T08:31:39.816712+0000","last_unstale":"2024-07-17T08:31:40.823514+0000","last_undegraded":"2024-07-17T08:31:40.823514+0000","last_fullsized":"2024-07-17T08:31:40.823514+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:36:10.825056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590634+0000","last_change":"2024-07-17T08:31:39.818917+0000","last_active":"2024-07-17T08:31:41.590634+0000","last_peered":"2024-07-17T08:31:41.590634+0000","last_clean":"2024-07-17T08:31:41.590634+0000","last_became_active":"2024-07-17T08:31:39.818644+0000","last_became_peered":"2024-07-17T08:31:39.818644+0000","last_unstale":"2024-07-17T08:31:41.590634+0000","last_undegraded":"2024-07-17T08:31:41.590634+0000","last_fullsized":"2024-07-17T08:31:41.590634+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:18:33.054231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588081+0000","last_change":"2024-07-17T08:31:39.825586+0000","last_active":"2024-07-17T08:31:41.588081+0000","last_peered":"2024-07-17T08:31:41.588081+0000","last_clean":"2024-07-17T08:31:41.588081+0000","last_became_active":"2024-07-17T08:31:39.818539+0000","last_became_peered":"2024-07-17T08:31:39.818539+0000","last_unstale":"2024-07-17T08:31:41.588081+0000","last_undegraded":"2024-07-17T08:31:41.588081+0000","last_fullsized":"2024-07-17T08:31:41.588081+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:44:45.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589059+0000","last_change":"2024-07-17T08:31:39.823951+0000","last_active":"2024-07-17T08:31:41.589059+0000","last_peered":"2024-07-17T08:31:41.589059+0000","last_clean":"2024-07-17T08:31:41.589059+0000","last_became_active":"2024-07-17T08:31:39.823672+0000","last_became_peered":"2024-07-17T08:31:39.823672+0000","last_unstale":"2024-07-17T08:31:41.589059+0000","last_undegraded":"2024-07-17T08:31:41.589059+0000","last_fullsized":"2024-07-17T08:31:41.589059+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:47.136915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590210+0000","last_change":"2024-07-17T08:31:39.816567+0000","last_active":"2024-07-17T08:31:41.590210+0000","last_peered":"2024-07-17T08:31:41.590210+0000","last_clean":"2024-07-17T08:31:41.590210+0000","last_became_active":"2024-07-17T08:31:39.816217+0000","last_became_peered":"2024-07-17T08:31:39.816217+0000","last_unstale":"2024-07-17T08:31:41.590210+0000","last_undegraded":"2024-07-17T08:31:41.590210+0000","last_fullsized":"2024-07-17T08:31:41.590210+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:20.066592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822614+0000","last_change":"2024-07-17T08:31:39.816531+0000","last_active":"2024-07-17T08:31:40.822614+0000","last_peered":"2024-07-17T08:31:40.822614+0000","last_clean":"2024-07-17T08:31:40.822614+0000","last_became_active":"2024-07-17T08:31:39.815950+0000","last_became_peered":"2024-07-17T08:31:39.815950+0000","last_unstale":"2024-07-17T08:31:40.822614+0000","last_undegraded":"2024-07-17T08:31:40.822614+0000","last_fullsized":"2024-07-17T08:31:40.822614+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:07.028191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821048+0000","last_change":"2024-07-17T08:31:39.812583+0000","last_active":"2024-07-17T08:31:40.821048+0000","last_peered":"2024-07-17T08:31:40.821048+0000","last_clean":"2024-07-17T08:31:40.821048+0000","last_became_active":"2024-07-17T08:31:39.812310+0000","last_became_peered":"2024-07-17T08:31:39.812310+0000","last_unstale":"2024-07-17T08:31:40.821048+0000","last_undegraded":"2024-07-17T08:31:40.821048+0000","last_fullsized":"2024-07-17T08:31:40.821048+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:30.031284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588357+0000","last_change":"2024-07-17T08:31:39.830646+0000","last_active":"2024-07-17T08:31:41.588357+0000","last_peered":"2024-07-17T08:31:41.588357+0000","last_clean":"2024-07-17T08:31:41.588357+0000","last_became_active":"2024-07-17T08:31:39.830291+0000","last_became_peered":"2024-07-17T08:31:39.830291+0000","last_unstale":"2024-07-17T08:31:41.588357+0000","last_undegraded":"2024-07-17T08:31:41.588357+0000","last_fullsized":"2024-07-17T08:31:41.588357+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.475040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590818+0000","last_change":"2024-07-17T08:31:39.823282+0000","last_active":"2024-07-17T08:31:41.590818+0000","last_peered":"2024-07-17T08:31:41.590818+0000","last_clean":"2024-07-17T08:31:41.590818+0000","last_became_active":"2024-07-17T08:31:39.822975+0000","last_became_peered":"2024-07-17T08:31:39.822975+0000","last_unstale":"2024-07-17T08:31:41.590818+0000","last_undegraded":"2024-07-17T08:31:41.590818+0000","last_fullsized":"2024-07-17T08:31:41.590818+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:21:45.472080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591519+0000","last_change":"2024-07-17T08:31:39.824803+0000","last_active":"2024-07-17T08:31:41.591519+0000","last_peered":"2024-07-17T08:31:41.591519+0000","last_clean":"2024-07-17T08:31:41.591519+0000","last_became_active":"2024-07-17T08:31:39.824486+0000","last_became_peered":"2024-07-17T08:31:39.824486+0000","last_unstale":"2024-07-17T08:31:41.591519+0000","last_undegraded":"2024-07-17T08:31:41.591519+0000","last_fullsized":"2024-07-17T08:31:41.591519+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:07.257594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591373+0000","last_change":"2024-07-17T08:31:39.816790+0000","last_active":"2024-07-17T08:31:41.591373+0000","last_peered":"2024-07-17T08:31:41.591373+0000","last_clean":"2024-07-17T08:31:41.591373+0000","last_became_active":"2024-07-17T08:31:39.816433+0000","last_became_peered":"2024-07-17T08:31:39.816433+0000","last_unstale":"2024-07-17T08:31:41.591373+0000","last_undegraded":"2024-07-17T08:31:41.591373+0000","last_fullsized":"2024-07-17T08:31:41.591373+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:26:07.835685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589613+0000","last_change":"2024-07-17T08:31:39.825621+0000","last_active":"2024-07-17T08:31:41.589613+0000","last_peered":"2024-07-17T08:31:41.589613+0000","last_clean":"2024-07-17T08:31:41.589613+0000","last_became_active":"2024-07-17T08:31:39.825025+0000","last_became_peered":"2024-07-17T08:31:39.825025+0000","last_unstale":"2024-07-17T08:31:41.589613+0000","last_undegraded":"2024-07-17T08:31:41.589613+0000","last_fullsized":"2024-07-17T08:31:41.589613+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:05:00.813468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823486+0000","last_change":"2024-07-17T08:31:39.815870+0000","last_active":"2024-07-17T08:31:40.823486+0000","last_peered":"2024-07-17T08:31:40.823486+0000","last_clean":"2024-07-17T08:31:40.823486+0000","last_became_active":"2024-07-17T08:31:39.815392+0000","last_became_peered":"2024-07-17T08:31:39.815392+0000","last_unstale":"2024-07-17T08:31:40.823486+0000","last_undegraded":"2024-07-17T08:31:40.823486+0000","last_fullsized":"2024-07-17T08:31:40.823486+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:32.930492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823397+0000","last_change":"2024-07-17T08:31:39.821225+0000","last_active":"2024-07-17T08:31:40.823397+0000","last_peered":"2024-07-17T08:31:40.823397+0000","last_clean":"2024-07-17T08:31:40.823397+0000","last_became_active":"2024-07-17T08:31:39.820907+0000","last_became_peered":"2024-07-17T08:31:39.820907+0000","last_unstale":"2024-07-17T08:31:40.823397+0000","last_undegraded":"2024-07-17T08:31:40.823397+0000","last_fullsized":"2024-07-17T08:31:40.823397+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:34.335739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161347+0000","last_change":"2024-07-17T08:31:39.830948+0000","last_active":"2024-07-17T08:31:41.161347+0000","last_peered":"2024-07-17T08:31:41.161347+0000","last_clean":"2024-07-17T08:31:41.161347+0000","last_became_active":"2024-07-17T08:31:39.830680+0000","last_became_peered":"2024-07-17T08:31:39.830680+0000","last_unstale":"2024-07-17T08:31:41.161347+0000","last_undegraded":"2024-07-17T08:31:41.161347+0000","last_fullsized":"2024-07-17T08:31:41.161347+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:39.346717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821022+0000","last_change":"2024-07-17T08:31:39.824738+0000","last_active":"2024-07-17T08:31:40.821022+0000","last_peered":"2024-07-17T08:31:40.821022+0000","last_clean":"2024-07-17T08:31:40.821022+0000","last_became_active":"2024-07-17T08:31:39.824464+0000","last_became_peered":"2024-07-17T08:31:39.824464+0000","last_unstale":"2024-07-17T08:31:40.821022+0000","last_undegraded":"2024-07-17T08:31:40.821022+0000","last_fullsized":"2024-07-17T08:31:40.821022+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:45.832140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823131+0000","last_change":"2024-07-17T08:31:39.803470+0000","last_active":"2024-07-17T08:31:40.823131+0000","last_peered":"2024-07-17T08:31:40.823131+0000","last_clean":"2024-07-17T08:31:40.823131+0000","last_became_active":"2024-07-17T08:31:39.802960+0000","last_became_peered":"2024-07-17T08:31:39.802960+0000","last_unstale":"2024-07-17T08:31:40.823131+0000","last_undegraded":"2024-07-17T08:31:40.823131+0000","last_fullsized":"2024-07-17T08:31:40.823131+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:09.994022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588804+0000","last_change":"2024-07-17T08:31:39.827030+0000","last_active":"2024-07-17T08:31:41.588804+0000","last_peered":"2024-07-17T08:31:41.588804+0000","last_clean":"2024-07-17T08:31:41.588804+0000","last_became_active":"2024-07-17T08:31:39.823457+0000","last_became_peered":"2024-07-17T08:31:39.823457+0000","last_unstale":"2024-07-17T08:31:41.588804+0000","last_undegraded":"2024-07-17T08:31:41.588804+0000","last_fullsized":"2024-07-17T08:31:41.588804+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:30.031284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.162094+0000","last_change":"2024-07-17T08:31:39.829525+0000","last_active":"2024-07-17T08:31:41.162094+0000","last_peered":"2024-07-17T08:31:41.162094+0000","last_clean":"2024-07-17T08:31:41.162094+0000","last_became_active":"2024-07-17T08:31:39.829239+0000","last_became_peered":"2024-07-17T08:31:39.829239+0000","last_unstale":"2024-07-17T08:31:41.162094+0000","last_undegraded":"2024-07-17T08:31:41.162094+0000","last_fullsized":"2024-07-17T08:31:41.162094+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:24.251749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589840+0000","last_change":"2024-07-17T08:31:39.820594+0000","last_active":"2024-07-17T08:31:41.589840+0000","last_peered":"2024-07-17T08:31:41.589840+0000","last_clean":"2024-07-17T08:31:41.589840+0000","last_became_active":"2024-07-17T08:31:39.820304+0000","last_became_peered":"2024-07-17T08:31:39.820304+0000","last_unstale":"2024-07-17T08:31:41.589840+0000","last_undegraded":"2024-07-17T08:31:41.589840+0000","last_fullsized":"2024-07-17T08:31:41.589840+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:06:18.162316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587981+0000","last_change":"2024-07-17T08:31:39.819430+0000","last_active":"2024-07-17T08:31:41.587981+0000","last_peered":"2024-07-17T08:31:41.587981+0000","last_clean":"2024-07-17T08:31:41.587981+0000","last_became_active":"2024-07-17T08:31:39.818610+0000","last_became_peered":"2024-07-17T08:31:39.818610+0000","last_unstale":"2024-07-17T08:31:41.587981+0000","last_undegraded":"2024-07-17T08:31:41.587981+0000","last_fullsized":"2024-07-17T08:31:41.587981+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:41.158567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588163+0000","last_change":"2024-07-17T08:31:39.832236+0000","last_active":"2024-07-17T08:31:41.588163+0000","last_peered":"2024-07-17T08:31:41.588163+0000","last_clean":"2024-07-17T08:31:41.588163+0000","last_became_active":"2024-07-17T08:31:39.831761+0000","last_became_peered":"2024-07-17T08:31:39.831761+0000","last_unstale":"2024-07-17T08:31:41.588163+0000","last_undegraded":"2024-07-17T08:31:41.588163+0000","last_fullsized":"2024-07-17T08:31:41.588163+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:45.832140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822359+0000","last_change":"2024-07-17T08:31:39.808245+0000","last_active":"2024-07-17T08:31:40.822359+0000","last_peered":"2024-07-17T08:31:40.822359+0000","last_clean":"2024-07-17T08:31:40.822359+0000","last_became_active":"2024-07-17T08:31:39.807981+0000","last_became_peered":"2024-07-17T08:31:39.807981+0000","last_unstale":"2024-07-17T08:31:40.822359+0000","last_undegraded":"2024-07-17T08:31:40.822359+0000","last_fullsized":"2024-07-17T08:31:40.822359+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:12.354179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588813+0000","last_change":"2024-07-17T08:31:39.805478+0000","last_active":"2024-07-17T08:31:41.588813+0000","last_peered":"2024-07-17T08:31:41.588813+0000","last_clean":"2024-07-17T08:31:41.588813+0000","last_became_active":"2024-07-17T08:31:39.805195+0000","last_became_peered":"2024-07-17T08:31:39.805195+0000","last_unstale":"2024-07-17T08:31:41.588813+0000","last_undegraded":"2024-07-17T08:31:41.588813+0000","last_fullsized":"2024-07-17T08:31:41.588813+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:58:36.223909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161849+0000","last_change":"2024-07-17T08:31:39.828329+0000","last_active":"2024-07-17T08:31:41.161849+0000","last_peered":"2024-07-17T08:31:41.161849+0000","last_clean":"2024-07-17T08:31:41.161849+0000","last_became_active":"2024-07-17T08:31:39.827150+0000","last_became_peered":"2024-07-17T08:31:39.827150+0000","last_unstale":"2024-07-17T08:31:41.161849+0000","last_undegraded":"2024-07-17T08:31:41.161849+0000","last_fullsized":"2024-07-17T08:31:41.161849+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:34:43.414458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"205'4","reported_seq":18,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588171+0000","last_change":"2024-07-17T08:31:38.819115+0000","last_active":"2024-07-17T08:31:41.588171+0000","last_peered":"2024-07-17T08:31:41.588171+0000","last_clean":"2024-07-17T08:31:41.588171+0000","last_became_active":"2024-07-17T08:31:38.818794+0000","last_became_peered":"2024-07-17T08:31:38.818794+0000","last_unstale":"2024-07-17T08:31:41.588171+0000","last_undegraded":"2024-07-17T08:31:41.588171+0000","last_fullsized":"2024-07-17T08:31:41.588171+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:28.448310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823897+0000","last_change":"2024-07-17T08:31:39.823269+0000","last_active":"2024-07-17T08:31:40.823897+0000","last_peered":"2024-07-17T08:31:40.823897+0000","last_clean":"2024-07-17T08:31:40.823897+0000","last_became_active":"2024-07-17T08:31:39.823004+0000","last_became_peered":"2024-07-17T08:31:39.823004+0000","last_unstale":"2024-07-17T08:31:40.823897+0000","last_undegraded":"2024-07-17T08:31:40.823897+0000","last_fullsized":"2024-07-17T08:31:40.823897+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:44.491921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590718+0000","last_change":"2024-07-17T08:31:38.811816+0000","last_active":"2024-07-17T08:31:41.590718+0000","last_peered":"2024-07-17T08:31:41.590718+0000","last_clean":"2024-07-17T08:31:41.590718+0000","last_became_active":"2024-07-17T08:31:38.810227+0000","last_became_peered":"2024-07-17T08:31:38.810227+0000","last_unstale":"2024-07-17T08:31:41.590718+0000","last_undegraded":"2024-07-17T08:31:41.590718+0000","last_fullsized":"2024-07-17T08:31:41.590718+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:28.125965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589001+0000","last_change":"2024-07-17T08:31:39.829059+0000","last_active":"2024-07-17T08:31:41.589001+0000","last_peered":"2024-07-17T08:31:41.589001+0000","last_clean":"2024-07-17T08:31:41.589001+0000","last_became_active":"2024-07-17T08:31:39.828514+0000","last_became_peered":"2024-07-17T08:31:39.828514+0000","last_unstale":"2024-07-17T08:31:41.589001+0000","last_undegraded":"2024-07-17T08:31:41.589001+0000","last_fullsized":"2024-07-17T08:31:41.589001+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:52.662697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588739+0000","last_change":"2024-07-17T08:31:38.818035+0000","last_active":"2024-07-17T08:31:41.588739+0000","last_peered":"2024-07-17T08:31:41.588739+0000","last_clean":"2024-07-17T08:31:41.588739+0000","last_became_active":"2024-07-17T08:31:38.817709+0000","last_became_peered":"2024-07-17T08:31:38.817709+0000","last_unstale":"2024-07-17T08:31:41.588739+0000","last_undegraded":"2024-07-17T08:31:41.588739+0000","last_fullsized":"2024-07-17T08:31:41.588739+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:44.973632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822248+0000","last_change":"2024-07-17T08:31:39.814181+0000","last_active":"2024-07-17T08:31:40.822248+0000","last_peered":"2024-07-17T08:31:40.822248+0000","last_clean":"2024-07-17T08:31:40.822248+0000","last_became_active":"2024-07-17T08:31:39.813884+0000","last_became_peered":"2024-07-17T08:31:39.813884+0000","last_unstale":"2024-07-17T08:31:40.822248+0000","last_undegraded":"2024-07-17T08:31:40.822248+0000","last_fullsized":"2024-07-17T08:31:40.822248+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:07.419250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591222+0000","last_change":"2024-07-17T08:31:38.820462+0000","last_active":"2024-07-17T08:31:41.591222+0000","last_peered":"2024-07-17T08:31:41.591222+0000","last_clean":"2024-07-17T08:31:41.591222+0000","last_became_active":"2024-07-17T08:31:38.820060+0000","last_became_peered":"2024-07-17T08:31:38.820060+0000","last_unstale":"2024-07-17T08:31:41.591222+0000","last_undegraded":"2024-07-17T08:31:41.591222+0000","last_fullsized":"2024-07-17T08:31:41.591222+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:40.318529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589386+0000","last_change":"2024-07-17T08:31:39.815242+0000","last_active":"2024-07-17T08:31:41.589386+0000","last_peered":"2024-07-17T08:31:41.589386+0000","last_clean":"2024-07-17T08:31:41.589386+0000","last_became_active":"2024-07-17T08:31:39.814953+0000","last_became_peered":"2024-07-17T08:31:39.814953+0000","last_unstale":"2024-07-17T08:31:41.589386+0000","last_undegraded":"2024-07-17T08:31:41.589386+0000","last_fullsized":"2024-07-17T08:31:41.589386+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:58:13.155112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589839+0000","last_change":"2024-07-17T08:31:38.817373+0000","last_active":"2024-07-17T08:31:41.589839+0000","last_peered":"2024-07-17T08:31:41.589839+0000","last_clean":"2024-07-17T08:31:41.589839+0000","last_became_active":"2024-07-17T08:31:38.812250+0000","last_became_peered":"2024-07-17T08:31:38.812250+0000","last_unstale":"2024-07-17T08:31:41.589839+0000","last_undegraded":"2024-07-17T08:31:41.589839+0000","last_fullsized":"2024-07-17T08:31:41.589839+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:35.674590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822823+0000","last_change":"2024-07-17T08:31:39.804245+0000","last_active":"2024-07-17T08:31:40.822823+0000","last_peered":"2024-07-17T08:31:40.822823+0000","last_clean":"2024-07-17T08:31:40.822823+0000","last_became_active":"2024-07-17T08:31:39.803945+0000","last_became_peered":"2024-07-17T08:31:39.803945+0000","last_unstale":"2024-07-17T08:31:40.822823+0000","last_undegraded":"2024-07-17T08:31:40.822823+0000","last_fullsized":"2024-07-17T08:31:40.822823+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:42.061189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588073+0000","last_change":"2024-07-17T08:31:38.819410+0000","last_active":"2024-07-17T08:31:41.588073+0000","last_peered":"2024-07-17T08:31:41.588073+0000","last_clean":"2024-07-17T08:31:41.588073+0000","last_became_active":"2024-07-17T08:31:38.818839+0000","last_became_peered":"2024-07-17T08:31:38.818839+0000","last_unstale":"2024-07-17T08:31:41.588073+0000","last_undegraded":"2024-07-17T08:31:41.588073+0000","last_fullsized":"2024-07-17T08:31:41.588073+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:35.674590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822289+0000","last_change":"2024-07-17T08:31:39.819899+0000","last_active":"2024-07-17T08:31:40.822289+0000","last_peered":"2024-07-17T08:31:40.822289+0000","last_clean":"2024-07-17T08:31:40.822289+0000","last_became_active":"2024-07-17T08:31:39.818790+0000","last_became_peered":"2024-07-17T08:31:39.818790+0000","last_unstale":"2024-07-17T08:31:40.822289+0000","last_undegraded":"2024-07-17T08:31:40.822289+0000","last_fullsized":"2024-07-17T08:31:40.822289+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:16.094871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590106+0000","last_change":"2024-07-17T08:31:38.818130+0000","last_active":"2024-07-17T08:31:41.590106+0000","last_peered":"2024-07-17T08:31:41.590106+0000","last_clean":"2024-07-17T08:31:41.590106+0000","last_became_active":"2024-07-17T08:31:38.817779+0000","last_became_peered":"2024-07-17T08:31:38.817779+0000","last_unstale":"2024-07-17T08:31:41.590106+0000","last_undegraded":"2024-07-17T08:31:41.590106+0000","last_fullsized":"2024-07-17T08:31:41.590106+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:34.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161998+0000","last_change":"2024-07-17T08:31:39.832741+0000","last_active":"2024-07-17T08:31:41.161998+0000","last_peered":"2024-07-17T08:31:41.161998+0000","last_clean":"2024-07-17T08:31:41.161998+0000","last_became_active":"2024-07-17T08:31:39.832446+0000","last_became_peered":"2024-07-17T08:31:39.832446+0000","last_unstale":"2024-07-17T08:31:41.161998+0000","last_undegraded":"2024-07-17T08:31:41.161998+0000","last_fullsized":"2024-07-17T08:31:41.161998+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:59.375338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590873+0000","last_change":"2024-07-17T08:31:38.808755+0000","last_active":"2024-07-17T08:31:41.590873+0000","last_peered":"2024-07-17T08:31:41.590873+0000","last_clean":"2024-07-17T08:31:41.590873+0000","last_became_active":"2024-07-17T08:31:38.808084+0000","last_became_peered":"2024-07-17T08:31:38.808084+0000","last_unstale":"2024-07-17T08:31:41.590873+0000","last_undegraded":"2024-07-17T08:31:41.590873+0000","last_fullsized":"2024-07-17T08:31:41.590873+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:32.690011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590485+0000","last_change":"2024-07-17T08:31:39.817816+0000","last_active":"2024-07-17T08:31:41.590485+0000","last_peered":"2024-07-17T08:31:41.590485+0000","last_clean":"2024-07-17T08:31:41.590485+0000","last_became_active":"2024-07-17T08:31:39.817324+0000","last_became_peered":"2024-07-17T08:31:39.817324+0000","last_unstale":"2024-07-17T08:31:41.590485+0000","last_undegraded":"2024-07-17T08:31:41.590485+0000","last_fullsized":"2024-07-17T08:31:41.590485+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:18:16.348784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822179+0000","last_change":"2024-07-17T08:31:38.811624+0000","last_active":"2024-07-17T08:31:40.822179+0000","last_peered":"2024-07-17T08:31:40.822179+0000","last_clean":"2024-07-17T08:31:40.822179+0000","last_became_active":"2024-07-17T08:31:38.811313+0000","last_became_peered":"2024-07-17T08:31:38.811313+0000","last_unstale":"2024-07-17T08:31:40.822179+0000","last_undegraded":"2024-07-17T08:31:40.822179+0000","last_fullsized":"2024-07-17T08:31:40.822179+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:10:06.763657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591474+0000","last_change":"2024-07-17T08:31:39.835436+0000","last_active":"2024-07-17T08:31:41.591474+0000","last_peered":"2024-07-17T08:31:41.591474+0000","last_clean":"2024-07-17T08:31:41.591474+0000","last_became_active":"2024-07-17T08:31:39.834756+0000","last_became_peered":"2024-07-17T08:31:39.834756+0000","last_unstale":"2024-07-17T08:31:41.591474+0000","last_undegraded":"2024-07-17T08:31:41.591474+0000","last_fullsized":"2024-07-17T08:31:41.591474+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:12.079894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"205'3","reported_seq":17,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587645+0000","last_change":"2024-07-17T08:31:38.812205+0000","last_active":"2024-07-17T08:31:41.587645+0000","last_peered":"2024-07-17T08:31:41.587645+0000","last_clean":"2024-07-17T08:31:41.587645+0000","last_became_active":"2024-07-17T08:31:38.811933+0000","last_became_peered":"2024-07-17T08:31:38.811933+0000","last_unstale":"2024-07-17T08:31:41.587645+0000","last_undegraded":"2024-07-17T08:31:41.587645+0000","last_fullsized":"2024-07-17T08:31:41.587645+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:18:12.732868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823160+0000","last_change":"2024-07-17T08:31:39.825589+0000","last_active":"2024-07-17T08:31:40.823160+0000","last_peered":"2024-07-17T08:31:40.823160+0000","last_clean":"2024-07-17T08:31:40.823160+0000","last_became_active":"2024-07-17T08:31:39.823167+0000","last_became_peered":"2024-07-17T08:31:39.823167+0000","last_unstale":"2024-07-17T08:31:40.823160+0000","last_undegraded":"2024-07-17T08:31:40.823160+0000","last_fullsized":"2024-07-17T08:31:40.823160+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:45.432822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"205'6","reported_seq":20,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.591186+0000","last_change":"2024-07-17T08:31:38.818164+0000","last_active":"2024-07-17T08:31:41.591186+0000","last_peered":"2024-07-17T08:31:41.591186+0000","last_clean":"2024-07-17T08:31:41.591186+0000","last_became_active":"2024-07-17T08:31:38.817822+0000","last_became_peered":"2024-07-17T08:31:38.817822+0000","last_unstale":"2024-07-17T08:31:41.591186+0000","last_undegraded":"2024-07-17T08:31:41.591186+0000","last_fullsized":"2024-07-17T08:31:41.591186+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:47.798226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821319+0000","last_change":"2024-07-17T08:31:39.822416+0000","last_active":"2024-07-17T08:31:40.821319+0000","last_peered":"2024-07-17T08:31:40.821319+0000","last_clean":"2024-07-17T08:31:40.821319+0000","last_became_active":"2024-07-17T08:31:39.822139+0000","last_became_peered":"2024-07-17T08:31:39.822139+0000","last_unstale":"2024-07-17T08:31:40.821319+0000","last_undegraded":"2024-07-17T08:31:40.821319+0000","last_fullsized":"2024-07-17T08:31:40.821319+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:46.672165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590643+0000","last_change":"2024-07-17T08:31:38.820600+0000","last_active":"2024-07-17T08:31:41.590643+0000","last_peered":"2024-07-17T08:31:41.590643+0000","last_clean":"2024-07-17T08:31:41.590643+0000","last_became_active":"2024-07-17T08:31:38.820243+0000","last_became_peered":"2024-07-17T08:31:38.820243+0000","last_unstale":"2024-07-17T08:31:41.590643+0000","last_undegraded":"2024-07-17T08:31:41.590643+0000","last_fullsized":"2024-07-17T08:31:41.590643+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:53.393603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587729+0000","last_change":"2024-07-17T08:31:39.806750+0000","last_active":"2024-07-17T08:31:41.587729+0000","last_peered":"2024-07-17T08:31:41.587729+0000","last_clean":"2024-07-17T08:31:41.587729+0000","last_became_active":"2024-07-17T08:31:39.806445+0000","last_became_peered":"2024-07-17T08:31:39.806445+0000","last_unstale":"2024-07-17T08:31:41.587729+0000","last_undegraded":"2024-07-17T08:31:41.587729+0000","last_fullsized":"2024-07-17T08:31:41.587729+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:36.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822582+0000","last_change":"2024-07-17T08:31:38.815059+0000","last_active":"2024-07-17T08:31:40.822582+0000","last_peered":"2024-07-17T08:31:40.822582+0000","last_clean":"2024-07-17T08:31:40.822582+0000","last_became_active":"2024-07-17T08:31:38.812349+0000","last_became_peered":"2024-07-17T08:31:38.812349+0000","last_unstale":"2024-07-17T08:31:40.822582+0000","last_undegraded":"2024-07-17T08:31:40.822582+0000","last_fullsized":"2024-07-17T08:31:40.822582+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:45:53.901805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590377+0000","last_change":"2024-07-17T08:31:39.818766+0000","last_active":"2024-07-17T08:31:41.590377+0000","last_peered":"2024-07-17T08:31:41.590377+0000","last_clean":"2024-07-17T08:31:41.590377+0000","last_became_active":"2024-07-17T08:31:39.818405+0000","last_became_peered":"2024-07-17T08:31:39.818405+0000","last_unstale":"2024-07-17T08:31:41.590377+0000","last_undegraded":"2024-07-17T08:31:41.590377+0000","last_fullsized":"2024-07-17T08:31:41.590377+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:04.170195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587802+0000","last_change":"2024-07-17T08:31:38.820427+0000","last_active":"2024-07-17T08:31:41.587802+0000","last_peered":"2024-07-17T08:31:41.587802+0000","last_clean":"2024-07-17T08:31:41.587802+0000","last_became_active":"2024-07-17T08:31:38.820096+0000","last_became_peered":"2024-07-17T08:31:38.820096+0000","last_unstale":"2024-07-17T08:31:41.587802+0000","last_undegraded":"2024-07-17T08:31:41.587802+0000","last_fullsized":"2024-07-17T08:31:41.587802+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:52.639287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590586+0000","last_change":"2024-07-17T08:31:39.809525+0000","last_active":"2024-07-17T08:31:41.590586+0000","last_peered":"2024-07-17T08:31:41.590586+0000","last_clean":"2024-07-17T08:31:41.590586+0000","last_became_active":"2024-07-17T08:31:39.809213+0000","last_became_peered":"2024-07-17T08:31:39.809213+0000","last_unstale":"2024-07-17T08:31:41.590586+0000","last_undegraded":"2024-07-17T08:31:41.590586+0000","last_fullsized":"2024-07-17T08:31:41.590586+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:56:35.082577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"205'8","reported_seq":22,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587896+0000","last_change":"2024-07-17T08:31:38.809345+0000","last_active":"2024-07-17T08:31:41.587896+0000","last_peered":"2024-07-17T08:31:41.587896+0000","last_clean":"2024-07-17T08:31:41.587896+0000","last_became_active":"2024-07-17T08:31:38.808447+0000","last_became_peered":"2024-07-17T08:31:38.808447+0000","last_unstale":"2024-07-17T08:31:41.587896+0000","last_undegraded":"2024-07-17T08:31:41.587896+0000","last_fullsized":"2024-07-17T08:31:41.587896+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:22:58.014602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822759+0000","last_change":"2024-07-17T08:31:39.809364+0000","last_active":"2024-07-17T08:31:40.822759+0000","last_peered":"2024-07-17T08:31:40.822759+0000","last_clean":"2024-07-17T08:31:40.822759+0000","last_became_active":"2024-07-17T08:31:39.806864+0000","last_became_peered":"2024-07-17T08:31:39.806864+0000","last_unstale":"2024-07-17T08:31:40.822759+0000","last_undegraded":"2024-07-17T08:31:40.822759+0000","last_fullsized":"2024-07-17T08:31:40.822759+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:16.393319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589290+0000","last_change":"2024-07-17T08:31:38.815102+0000","last_active":"2024-07-17T08:31:41.589290+0000","last_peered":"2024-07-17T08:31:41.589290+0000","last_clean":"2024-07-17T08:31:41.589290+0000","last_became_active":"2024-07-17T08:31:38.814791+0000","last_became_peered":"2024-07-17T08:31:38.814791+0000","last_unstale":"2024-07-17T08:31:41.589290+0000","last_undegraded":"2024-07-17T08:31:41.589290+0000","last_fullsized":"2024-07-17T08:31:41.589290+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:34.762772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590160+0000","last_change":"2024-07-17T08:31:39.819374+0000","last_active":"2024-07-17T08:31:41.590160+0000","last_peered":"2024-07-17T08:31:41.590160+0000","last_clean":"2024-07-17T08:31:41.590160+0000","last_became_active":"2024-07-17T08:31:39.818158+0000","last_became_peered":"2024-07-17T08:31:39.818158+0000","last_unstale":"2024-07-17T08:31:41.590160+0000","last_undegraded":"2024-07-17T08:31:41.590160+0000","last_fullsized":"2024-07-17T08:31:41.590160+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:20:54.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588377+0000","last_change":"2024-07-17T08:31:38.819774+0000","last_active":"2024-07-17T08:31:41.588377+0000","last_peered":"2024-07-17T08:31:41.588377+0000","last_clean":"2024-07-17T08:31:41.588377+0000","last_became_active":"2024-07-17T08:31:38.818991+0000","last_became_peered":"2024-07-17T08:31:38.818991+0000","last_unstale":"2024-07-17T08:31:41.588377+0000","last_undegraded":"2024-07-17T08:31:41.588377+0000","last_fullsized":"2024-07-17T08:31:41.588377+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:58.591656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589155+0000","last_change":"2024-07-17T08:31:39.826773+0000","last_active":"2024-07-17T08:31:41.589155+0000","last_peered":"2024-07-17T08:31:41.589155+0000","last_clean":"2024-07-17T08:31:41.589155+0000","last_became_active":"2024-07-17T08:31:39.823063+0000","last_became_peered":"2024-07-17T08:31:39.823063+0000","last_unstale":"2024-07-17T08:31:41.589155+0000","last_undegraded":"2024-07-17T08:31:41.589155+0000","last_fullsized":"2024-07-17T08:31:41.589155+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:29.302320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823608+0000","last_change":"2024-07-17T08:31:38.808909+0000","last_active":"2024-07-17T08:31:40.823608+0000","last_peered":"2024-07-17T08:31:40.823608+0000","last_clean":"2024-07-17T08:31:40.823608+0000","last_became_active":"2024-07-17T08:31:38.808527+0000","last_became_peered":"2024-07-17T08:31:38.808527+0000","last_unstale":"2024-07-17T08:31:40.823608+0000","last_undegraded":"2024-07-17T08:31:40.823608+0000","last_fullsized":"2024-07-17T08:31:40.823608+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:20.292751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823329+0000","last_change":"2024-07-17T08:31:39.818523+0000","last_active":"2024-07-17T08:31:40.823329+0000","last_peered":"2024-07-17T08:31:40.823329+0000","last_clean":"2024-07-17T08:31:40.823329+0000","last_became_active":"2024-07-17T08:31:39.818260+0000","last_became_peered":"2024-07-17T08:31:39.818260+0000","last_unstale":"2024-07-17T08:31:40.823329+0000","last_undegraded":"2024-07-17T08:31:40.823329+0000","last_fullsized":"2024-07-17T08:31:40.823329+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:39.178144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588184+0000","last_change":"2024-07-17T08:31:38.815982+0000","last_active":"2024-07-17T08:31:41.588184+0000","last_peered":"2024-07-17T08:31:41.588184+0000","last_clean":"2024-07-17T08:31:41.588184+0000","last_became_active":"2024-07-17T08:31:38.815678+0000","last_became_peered":"2024-07-17T08:31:38.815678+0000","last_unstale":"2024-07-17T08:31:41.588184+0000","last_undegraded":"2024-07-17T08:31:41.588184+0000","last_fullsized":"2024-07-17T08:31:41.588184+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:23.606160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821130+0000","last_change":"2024-07-17T08:31:39.819106+0000","last_active":"2024-07-17T08:31:40.821130+0000","last_peered":"2024-07-17T08:31:40.821130+0000","last_clean":"2024-07-17T08:31:40.821130+0000","last_became_active":"2024-07-17T08:31:39.818816+0000","last_became_peered":"2024-07-17T08:31:39.818816+0000","last_unstale":"2024-07-17T08:31:40.821130+0000","last_undegraded":"2024-07-17T08:31:40.821130+0000","last_fullsized":"2024-07-17T08:31:40.821130+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:52.662697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590901+0000","last_change":"2024-07-17T08:31:38.819807+0000","last_active":"2024-07-17T08:31:41.590901+0000","last_peered":"2024-07-17T08:31:41.590901+0000","last_clean":"2024-07-17T08:31:41.590901+0000","last_became_active":"2024-07-17T08:31:38.819330+0000","last_became_peered":"2024-07-17T08:31:38.819330+0000","last_unstale":"2024-07-17T08:31:41.590901+0000","last_undegraded":"2024-07-17T08:31:41.590901+0000","last_fullsized":"2024-07-17T08:31:41.590901+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:50.816517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587442+0000","last_change":"2024-07-17T08:31:39.830116+0000","last_active":"2024-07-17T08:31:41.587442+0000","last_peered":"2024-07-17T08:31:41.587442+0000","last_clean":"2024-07-17T08:31:41.587442+0000","last_became_active":"2024-07-17T08:31:39.828879+0000","last_became_peered":"2024-07-17T08:31:39.828879+0000","last_unstale":"2024-07-17T08:31:41.587442+0000","last_undegraded":"2024-07-17T08:31:41.587442+0000","last_fullsized":"2024-07-17T08:31:41.587442+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:36:33.236703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588361+0000","last_change":"2024-07-17T08:31:38.819129+0000","last_active":"2024-07-17T08:31:41.588361+0000","last_peered":"2024-07-17T08:31:41.588361+0000","last_clean":"2024-07-17T08:31:41.588361+0000","last_became_active":"2024-07-17T08:31:38.818368+0000","last_became_peered":"2024-07-17T08:31:38.818368+0000","last_unstale":"2024-07-17T08:31:41.588361+0000","last_undegraded":"2024-07-17T08:31:41.588361+0000","last_fullsized":"2024-07-17T08:31:41.588361+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:28.125965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.587749+0000","last_change":"2024-07-17T08:31:39.834431+0000","last_active":"2024-07-17T08:31:41.587749+0000","last_peered":"2024-07-17T08:31:41.587749+0000","last_clean":"2024-07-17T08:31:41.587749+0000","last_became_active":"2024-07-17T08:31:39.834136+0000","last_became_peered":"2024-07-17T08:31:39.834136+0000","last_unstale":"2024-07-17T08:31:41.587749+0000","last_undegraded":"2024-07-17T08:31:41.587749+0000","last_fullsized":"2024-07-17T08:31:41.587749+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:02.330164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822620+0000","last_change":"2024-07-17T08:31:38.809351+0000","last_active":"2024-07-17T08:31:40.822620+0000","last_peered":"2024-07-17T08:31:40.822620+0000","last_clean":"2024-07-17T08:31:40.822620+0000","last_became_active":"2024-07-17T08:31:38.809079+0000","last_became_peered":"2024-07-17T08:31:38.809079+0000","last_unstale":"2024-07-17T08:31:40.822620+0000","last_undegraded":"2024-07-17T08:31:40.822620+0000","last_fullsized":"2024-07-17T08:31:40.822620+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:09.389453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589765+0000","last_change":"2024-07-17T08:31:39.806959+0000","last_active":"2024-07-17T08:31:41.589765+0000","last_peered":"2024-07-17T08:31:41.589765+0000","last_clean":"2024-07-17T08:31:41.589765+0000","last_became_active":"2024-07-17T08:31:39.806674+0000","last_became_peered":"2024-07-17T08:31:39.806674+0000","last_unstale":"2024-07-17T08:31:41.589765+0000","last_undegraded":"2024-07-17T08:31:41.589765+0000","last_fullsized":"2024-07-17T08:31:41.589765+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:31:41.363288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821762+0000","last_change":"2024-07-17T08:31:38.809809+0000","last_active":"2024-07-17T08:31:40.821762+0000","last_peered":"2024-07-17T08:31:40.821762+0000","last_clean":"2024-07-17T08:31:40.821762+0000","last_became_active":"2024-07-17T08:31:38.807439+0000","last_became_peered":"2024-07-17T08:31:38.807439+0000","last_unstale":"2024-07-17T08:31:40.821762+0000","last_undegraded":"2024-07-17T08:31:40.821762+0000","last_fullsized":"2024-07-17T08:31:40.821762+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:20.523594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823197+0000","last_change":"2024-07-17T08:31:39.824716+0000","last_active":"2024-07-17T08:31:40.823197+0000","last_peered":"2024-07-17T08:31:40.823197+0000","last_clean":"2024-07-17T08:31:40.823197+0000","last_became_active":"2024-07-17T08:31:39.822968+0000","last_became_peered":"2024-07-17T08:31:39.822968+0000","last_unstale":"2024-07-17T08:31:40.823197+0000","last_undegraded":"2024-07-17T08:31:40.823197+0000","last_fullsized":"2024-07-17T08:31:40.823197+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:41.543111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"205'2","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588423+0000","last_change":"2024-07-17T08:31:38.811899+0000","last_active":"2024-07-17T08:31:41.588423+0000","last_peered":"2024-07-17T08:31:41.588423+0000","last_clean":"2024-07-17T08:31:41.588423+0000","last_became_active":"2024-07-17T08:31:38.811557+0000","last_became_peered":"2024-07-17T08:31:38.811557+0000","last_unstale":"2024-07-17T08:31:41.588423+0000","last_undegraded":"2024-07-17T08:31:41.588423+0000","last_fullsized":"2024-07-17T08:31:41.588423+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:49.729504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821517+0000","last_change":"2024-07-17T08:31:39.814407+0000","last_active":"2024-07-17T08:31:40.821517+0000","last_peered":"2024-07-17T08:31:40.821517+0000","last_clean":"2024-07-17T08:31:40.821517+0000","last_became_active":"2024-07-17T08:31:39.814139+0000","last_became_peered":"2024-07-17T08:31:39.814139+0000","last_unstale":"2024-07-17T08:31:40.821517+0000","last_undegraded":"2024-07-17T08:31:40.821517+0000","last_fullsized":"2024-07-17T08:31:40.821517+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:34:29.302320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"205'1","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823755+0000","last_change":"2024-07-17T08:31:38.815082+0000","last_active":"2024-07-17T08:31:40.823755+0000","last_peered":"2024-07-17T08:31:40.823755+0000","last_clean":"2024-07-17T08:31:40.823755+0000","last_became_active":"2024-07-17T08:31:38.812303+0000","last_became_peered":"2024-07-17T08:31:38.812303+0000","last_unstale":"2024-07-17T08:31:40.823755+0000","last_undegraded":"2024-07-17T08:31:40.823755+0000","last_fullsized":"2024-07-17T08:31:40.823755+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:04.222394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.162243+0000","last_change":"2024-07-17T08:31:39.830815+0000","last_active":"2024-07-17T08:31:41.162243+0000","last_peered":"2024-07-17T08:31:41.162243+0000","last_clean":"2024-07-17T08:31:41.162243+0000","last_became_active":"2024-07-17T08:31:39.827586+0000","last_became_peered":"2024-07-17T08:31:39.827586+0000","last_unstale":"2024-07-17T08:31:41.162243+0000","last_undegraded":"2024-07-17T08:31:41.162243+0000","last_fullsized":"2024-07-17T08:31:41.162243+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:10.822211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590428+0000","last_change":"2024-07-17T08:31:38.816782+0000","last_active":"2024-07-17T08:31:41.590428+0000","last_peered":"2024-07-17T08:31:41.590428+0000","last_clean":"2024-07-17T08:31:41.590428+0000","last_became_active":"2024-07-17T08:31:38.815186+0000","last_became_peered":"2024-07-17T08:31:38.815186+0000","last_unstale":"2024-07-17T08:31:41.590428+0000","last_undegraded":"2024-07-17T08:31:41.590428+0000","last_fullsized":"2024-07-17T08:31:41.590428+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:25.972011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.823462+0000","last_change":"2024-07-17T08:31:39.818433+0000","last_active":"2024-07-17T08:31:40.823462+0000","last_peered":"2024-07-17T08:31:40.823462+0000","last_clean":"2024-07-17T08:31:40.823462+0000","last_became_active":"2024-07-17T08:31:39.816321+0000","last_became_peered":"2024-07-17T08:31:39.816321+0000","last_unstale":"2024-07-17T08:31:40.823462+0000","last_undegraded":"2024-07-17T08:31:40.823462+0000","last_fullsized":"2024-07-17T08:31:40.823462+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:09.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.589920+0000","last_change":"2024-07-17T08:31:38.805212+0000","last_active":"2024-07-17T08:31:41.589920+0000","last_peered":"2024-07-17T08:31:41.589920+0000","last_clean":"2024-07-17T08:31:41.589920+0000","last_became_active":"2024-07-17T08:31:38.804912+0000","last_became_peered":"2024-07-17T08:31:38.804912+0000","last_unstale":"2024-07-17T08:31:41.589920+0000","last_undegraded":"2024-07-17T08:31:41.589920+0000","last_fullsized":"2024-07-17T08:31:41.589920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:06.697072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.821741+0000","last_change":"2024-07-17T08:31:39.815480+0000","last_active":"2024-07-17T08:31:40.821741+0000","last_peered":"2024-07-17T08:31:40.821741+0000","last_clean":"2024-07-17T08:31:40.821741+0000","last_became_active":"2024-07-17T08:31:39.815206+0000","last_became_peered":"2024-07-17T08:31:39.815206+0000","last_unstale":"2024-07-17T08:31:40.821741+0000","last_undegraded":"2024-07-17T08:31:40.821741+0000","last_fullsized":"2024-07-17T08:31:40.821741+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:38:15.504242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161313+0000","last_change":"2024-07-17T08:31:38.817064+0000","last_active":"2024-07-17T08:31:41.161313+0000","last_peered":"2024-07-17T08:31:41.161313+0000","last_clean":"2024-07-17T08:31:41.161313+0000","last_became_active":"2024-07-17T08:31:38.816291+0000","last_became_peered":"2024-07-17T08:31:38.816291+0000","last_unstale":"2024-07-17T08:31:41.161313+0000","last_undegraded":"2024-07-17T08:31:41.161313+0000","last_fullsized":"2024-07-17T08:31:41.161313+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:28.650063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.590200+0000","last_change":"2024-07-17T08:31:39.816913+0000","last_active":"2024-07-17T08:31:41.590200+0000","last_peered":"2024-07-17T08:31:41.590200+0000","last_clean":"2024-07-17T08:31:41.590200+0000","last_became_active":"2024-07-17T08:31:39.816604+0000","last_became_peered":"2024-07-17T08:31:39.816604+0000","last_unstale":"2024-07-17T08:31:41.590200+0000","last_undegraded":"2024-07-17T08:31:41.590200+0000","last_fullsized":"2024-07-17T08:31:41.590200+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:42.596428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161133+0000","last_change":"2024-07-17T08:31:38.812443+0000","last_active":"2024-07-17T08:31:41.161133+0000","last_peered":"2024-07-17T08:31:41.161133+0000","last_clean":"2024-07-17T08:31:41.161133+0000","last_became_active":"2024-07-17T08:31:38.812147+0000","last_became_peered":"2024-07-17T08:31:38.812147+0000","last_unstale":"2024-07-17T08:31:41.161133+0000","last_undegraded":"2024-07-17T08:31:41.161133+0000","last_fullsized":"2024-07-17T08:31:41.161133+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:51.869401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822456+0000","last_change":"2024-07-17T08:31:39.813472+0000","last_active":"2024-07-17T08:31:40.822456+0000","last_peered":"2024-07-17T08:31:40.822456+0000","last_clean":"2024-07-17T08:31:40.822456+0000","last_became_active":"2024-07-17T08:31:39.813173+0000","last_became_peered":"2024-07-17T08:31:39.813173+0000","last_unstale":"2024-07-17T08:31:40.822456+0000","last_undegraded":"2024-07-17T08:31:40.822456+0000","last_fullsized":"2024-07-17T08:31:40.822456+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:31.724471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822661+0000","last_change":"2024-07-17T08:31:38.804439+0000","last_active":"2024-07-17T08:31:40.822661+0000","last_peered":"2024-07-17T08:31:40.822661+0000","last_clean":"2024-07-17T08:31:40.822661+0000","last_became_active":"2024-07-17T08:31:38.803725+0000","last_became_peered":"2024-07-17T08:31:38.803725+0000","last_unstale":"2024-07-17T08:31:40.822661+0000","last_undegraded":"2024-07-17T08:31:40.822661+0000","last_fullsized":"2024-07-17T08:31:40.822661+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:17.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161427+0000","last_change":"2024-07-17T08:31:39.829402+0000","last_active":"2024-07-17T08:31:41.161427+0000","last_peered":"2024-07-17T08:31:41.161427+0000","last_clean":"2024-07-17T08:31:41.161427+0000","last_became_active":"2024-07-17T08:31:39.829135+0000","last_became_peered":"2024-07-17T08:31:39.829135+0000","last_unstale":"2024-07-17T08:31:41.161427+0000","last_undegraded":"2024-07-17T08:31:41.161427+0000","last_fullsized":"2024-07-17T08:31:41.161427+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:24.298669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822033+0000","last_change":"2024-07-17T08:31:38.806916+0000","last_active":"2024-07-17T08:31:40.822033+0000","last_peered":"2024-07-17T08:31:40.822033+0000","last_clean":"2024-07-17T08:31:40.822033+0000","last_became_active":"2024-07-17T08:31:38.806157+0000","last_became_peered":"2024-07-17T08:31:38.806157+0000","last_unstale":"2024-07-17T08:31:40.822033+0000","last_undegraded":"2024-07-17T08:31:40.822033+0000","last_fullsized":"2024-07-17T08:31:40.822033+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:14.516253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:40.822547+0000","last_change":"2024-07-17T08:31:39.811208+0000","last_active":"2024-07-17T08:31:40.822547+0000","last_peered":"2024-07-17T08:31:40.822547+0000","last_clean":"2024-07-17T08:31:40.822547+0000","last_became_active":"2024-07-17T08:31:39.810572+0000","last_became_peered":"2024-07-17T08:31:39.810572+0000","last_unstale":"2024-07-17T08:31:40.822547+0000","last_undegraded":"2024-07-17T08:31:40.822547+0000","last_fullsized":"2024-07-17T08:31:40.822547+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:38.762465+0000","last_clean_scrub_stamp":"2024-07-17T08:31:38.762465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:29.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.588005+0000","last_change":"2024-07-17T08:31:38.809339+0000","last_active":"2024-07-17T08:31:41.588005+0000","last_peered":"2024-07-17T08:31:41.588005+0000","last_clean":"2024-07-17T08:31:41.588005+0000","last_became_active":"2024-07-17T08:31:38.809017+0000","last_became_peered":"2024-07-17T08:31:38.809017+0000","last_unstale":"2024-07-17T08:31:41.588005+0000","last_undegraded":"2024-07-17T08:31:41.588005+0000","last_fullsized":"2024-07-17T08:31:41.588005+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:31:37.747351+0000","last_clean_scrub_stamp":"2024-07-17T08:31:37.747351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:06.697072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"197'2497","reported_seq":3296,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-07-17T08:31:41.161325+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:31:41.161325+0000","last_peered":"2024-07-17T08:31:41.161325+0000","last_clean":"2024-07-17T08:31:41.161325+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:31:41.161325+0000","last_undegraded":"2024-07-17T08:31:41.161325+0000","last_fullsized":"2024-07-17T08:31:41.161325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":335,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1267,"num_read_kb":3519,"num_write":335,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2497,"ondisk_log_size":2497,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543298,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60644,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311196,"statfs":{"total":96636764160,"available":96574664704,"internally_reserved":0,"allocated":3170304,"data_stored":1987615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510593,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42020,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329820,"statfs":{"total":96636764160,"available":96593735680,"internally_reserved":0,"allocated":2580480,"data_stored":1397317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510594,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":2580480,"data_stored":1391613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543299,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47592,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324248,"statfs":{"total":96636764160,"available":96588029952,"internally_reserved":0,"allocated":2580480,"data_stored":1391625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510593,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2572288,"data_stored":1390139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2580480,"data_stored":1397829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510594,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1397247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510593,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56492,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315348,"statfs":{"total":96636764160,"available":96578916352,"internally_reserved":0,"allocated":3174400,"data_stored":1981075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:31:44.786 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:31:44.786 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 -- 172.21.15.44:0/4206326236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 msgr2=0x7f3308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 --2- 172.21.15.44:0/4206326236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f3308004c80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f330c006f60 tx=0x7f3308005830 comp rx=0 tx=0).stop 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 -- 172.21.15.44:0/4206326236 shutdown_connections 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 --2- 172.21.15.44:0/4206326236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f3308004c80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 -- 172.21.15.44:0/4206326236 >> 172.21.15.44:0/4206326236 conn(0x7f332406ce80 msgr2=0x7f332406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 -- 172.21.15.44:0/4206326236 shutdown_connections 2024-07-17T08:31:45.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.045+0000 7f3329600640 1 -- 172.21.15.44:0/4206326236 wait complete. 2024-07-17T08:31:45.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.046+0000 7f3329600640 1 Processor -- start 2024-07-17T08:31:45.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.047+0000 7f3329600640 1 -- start start 2024-07-17T08:31:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33241f7420 0x7f33241f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 0x7f3324146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f332406f240 con 0x7f3308004800 2024-07-17T08:31:45.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f332406f0c0 con 0x7f33241f7420 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3329600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f332406f3c0 con 0x7f33241462a0 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50692/0 (socket says 172.21.15.44:50692) 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 -- 172.21.15.44:0/4045370306 learned_addr learned my addr 172.21.15.44:0/4045370306 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322000640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33241f7420 0x7f33241f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3323400640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 0x7f3324146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 -- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 msgr2=0x7f3324146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 0x7f3324146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 -- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33241f7420 msgr2=0x7f33241f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33241f7420 0x7f33241f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3322a00640 1 -- 172.21.15.44:0/4045370306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33241f8190 con 0x7f3308004800 2024-07-17T08:31:45.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.048+0000 7f3323400640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 0x7f3324146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:45.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f3322a00640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3314010910 tx=0x7f3314010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:45.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400ece0 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f331400ee80 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314005240 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33241f40a0 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.049+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33241f3d50 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.051+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33240ada10 con 0x7f3308004800 2024-07-17T08:31:45.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.052+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f33140053e0 con 0x7f3308004800 2024-07-17T08:31:45.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.053+0000 7f331aa00640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 0x7f33000790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.053+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f3314099370 con 0x7f3308004800 2024-07-17T08:31:45.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.054+0000 7f3322000640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 0x7f33000790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.055+0000 7f3322000640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 0x7f33000790a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f330c033000 tx=0x7f33240b5ce0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:45.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.057+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3314062460 con 0x7f3308004800 2024-07-17T08:31:45.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.347+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3324068c90 con 0x7f3308004800 2024-07-17T08:31:45.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.352+0000 7f331aa00640 1 -- 172.21.15.44:0/4045370306 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v69) ==== 190+0+147 (secure 0 0 0) 0x7f3314099130 con 0x7f3308004800 2024-07-17T08:31:45.353 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:31:45.354 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:31:45.354 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:31:45.354 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:31:45.354 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:31:45.354 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.355+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 msgr2=0x7f33000790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.355+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 0x7f33000790a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f330c033000 tx=0x7f33240b5ce0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.355+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 msgr2=0x7f33240b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.355+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3314010910 tx=0x7f3314010dd0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 shutdown_connections 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33241462a0 0x7f3324146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f3300076be0 0x7f33000790a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33241f7420 0x7f33241f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 --2- 172.21.15.44:0/4045370306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3308004800 0x7f33240b4710 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 >> 172.21.15.44:0/4045370306 conn(0x7f332406ce80 msgr2=0x7f3324073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 shutdown_connections 2024-07-17T08:31:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.356+0000 7f3329600640 1 -- 172.21.15.44:0/4045370306 wait complete. 2024-07-17T08:31:45.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 -- 172.21.15.44:0/635537174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 msgr2=0x7fa2700ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 --2- 172.21.15.44:0/635537174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa2700ad8e0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa2700d00a0 tx=0x7fa25c02a4a0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 -- 172.21.15.44:0/635537174 shutdown_connections 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 --2- 172.21.15.44:0/635537174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa2700ad8e0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 --2- 172.21.15.44:0/635537174 >> v2:172.21.15.175:3300/0 conn(0x7fa2700cf540 0x7fa2700b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.589+0000 7fa276c00640 1 -- 172.21.15.44:0/635537174 >> 172.21.15.44:0/635537174 conn(0x7fa2700b0e80 msgr2=0x7fa2700b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.590+0000 7fa276c00640 1 -- 172.21.15.44:0/635537174 shutdown_connections 2024-07-17T08:31:45.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.590+0000 7fa276c00640 1 -- 172.21.15.44:0/635537174 wait complete. 2024-07-17T08:31:45.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.591+0000 7fa276c00640 1 Processor -- start 2024-07-17T08:31:45.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.592+0000 7fa276c00640 1 -- start start 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 0x7fa2701598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 0x7fa270159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700ae8c0 con 0x7fa2700b3db0 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700ae740 con 0x7fa2700cf920 2024-07-17T08:31:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700aea40 con 0x7fa2700cf540 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50710/0 (socket says 172.21.15.44:50710) 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 -- 172.21.15.44:0/1178359466 learned_addr learned my addr 172.21.15.44:0/1178359466 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa274e00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 0x7fa2701598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa276200640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 0x7fa270159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 -- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 msgr2=0x7fa2701598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 0x7fa2701598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 -- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 msgr2=0x7fa270159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 0x7fa270159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.593+0000 7fa275800640 1 -- 172.21.15.44:0/1178359466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa270072390 con 0x7fa2700b3db0 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa274e00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 0x7fa2701598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:45.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa276200640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 0x7fa270159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa275800640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa26400d980 tx=0x7fa26400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264017070 con 0x7fa2700b3db0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa264009d20 con 0x7fa2700b3db0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640106c0 con 0x7fa2700b3db0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2700715d0 con 0x7fa2700b3db0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.594+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa270071b90 con 0x7fa2700b3db0 2024-07-17T08:31:45.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.596+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2701412e0 con 0x7fa2700b3db0 2024-07-17T08:31:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.597+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fa264010860 con 0x7fa2700b3db0 2024-07-17T08:31:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.598+0000 7fa26d600640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 0x7fa2500790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.598+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7fa264098a70 con 0x7fa2700b3db0 2024-07-17T08:31:45.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.598+0000 7fa274e00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 0x7fa2500790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:45.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.600+0000 7fa274e00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 0x7fa2500790c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa25c034000 tx=0x7fa25c004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:45.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.602+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa264061b50 con 0x7fa2700b3db0 2024-07-17T08:31:45.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.846+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa270140180 con 0x7fa2700b3db0 2024-07-17T08:31:45.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.854+0000 7fa26d600640 1 -- 172.21.15.44:0/1178359466 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7fa264061450 con 0x7fa2700b3db0 2024-07-17T08:31:45.855 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:31:45.855 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:31:45.856 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 msgr2=0x7fa2500790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 0x7fa2500790c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa25c034000 tx=0x7fa25c004730 comp rx=0 tx=0).stop 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 msgr2=0x7fa27013da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa26400d980 tx=0x7fa26400de40 comp rx=0 tx=0).stop 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 shutdown_connections 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa250076c00 0x7fa2500790c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2700cf920 0x7fa270159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2700cf540 0x7fa2701598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 --2- 172.21.15.44:0/1178359466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2700b3db0 0x7fa27013da00 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 >> 172.21.15.44:0/1178359466 conn(0x7fa2700b0e80 msgr2=0x7fa2701406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 shutdown_connections 2024-07-17T08:31:45.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:45.857+0000 7fa276c00640 1 -- 172.21.15.44:0/1178359466 wait complete. 2024-07-17T08:31:46.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:31:39.271781+0000', 'modified': '2024-07-17T08:31:42.343316+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9612}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9612': {'gid': 9612, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/271909788', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 271909788}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 271909788}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9612, 'qdb_cluster': [9612]} 2024-07-17T08:31:46.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:31:46.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-17T08:31:46.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 -- 172.21.15.44:0/354239834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a8154490 msgr2=0x7f59a815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 --2- 172.21.15.44:0/354239834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a8154490 0x7f59a815d830 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5998009830 tx=0x7f599802a510 comp rx=0 tx=0).stop 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 -- 172.21.15.44:0/354239834 shutdown_connections 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 --2- 172.21.15.44:0/354239834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a8154490 0x7f59a815d830 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 -- 172.21.15.44:0/354239834 >> 172.21.15.44:0/354239834 conn(0x7f59a80cf0d0 msgr2=0x7f59a80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.500+0000 7f59ade00640 1 -- 172.21.15.44:0/354239834 shutdown_connections 2024-07-17T08:31:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.501+0000 7f59ade00640 1 -- 172.21.15.44:0/354239834 wait complete. 2024-07-17T08:31:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.502+0000 7f59ade00640 1 Processor -- start 2024-07-17T08:31:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.502+0000 7f59ade00640 1 -- start start 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 0x7f59a8155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59a815dd70 0x7f59a81f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59a80b2430 con 0x7f59a806f4f0 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59a80b22b0 con 0x7f59a8154490 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.503+0000 7f59ade00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59a80b25b0 con 0x7f59a815dd70 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50718/0 (socket says 172.21.15.44:50718) 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 -- 172.21.15.44:0/2417762678 learned_addr learned my addr 172.21.15.44:0/2417762678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59a7e00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 0x7f59a8155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59ad400640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59a815dd70 0x7f59a81f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 -- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59a815dd70 msgr2=0x7f59a81f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59a815dd70 0x7f59a81f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 -- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 msgr2=0x7f59a8155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 0x7f59a8155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59aca00640 1 -- 172.21.15.44:0/2417762678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a81f39a0 con 0x7f59a806f4f0 2024-07-17T08:31:46.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.504+0000 7f59a7e00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 0x7f59a8155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59aca00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59a4c00640 1 -- 172.21.15.44:0/2417762678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998040070 con 0x7f59a806f4f0 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59a4c00640 1 -- 172.21.15.44:0/2417762678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59980038e0 con 0x7f59a806f4f0 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59a4c00640 1 -- 172.21.15.44:0/2417762678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998003d30 con 0x7f59a806f4f0 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a81f79e0 con 0x7f59a806f4f0 2024-07-17T08:31:46.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.505+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a81f3c60 con 0x7f59a806f4f0 2024-07-17T08:31:46.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.507+0000 7f59a4c00640 1 -- 172.21.15.44:0/2417762678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f5998039040 con 0x7f59a806f4f0 2024-07-17T08:31:46.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.508+0000 7f59a4c00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 0x7f59840790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.509+0000 7f59a4c00640 1 -- 172.21.15.44:0/2417762678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f59980b9be0 con 0x7f59a806f4f0 2024-07-17T08:31:46.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.509+0000 7f59b2277740 1 Processor -- start 2024-07-17T08:31:46.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.509+0000 7f59b2277740 1 -- start start 2024-07-17T08:31:46.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.509+0000 7f59a7e00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 0x7f59840790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9d1280 0x55908f9f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f6790 0x55908f9f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55908f941d60 con 0x55908f9f9fc0 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55908f931260 con 0x55908f9f6790 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55908f9352a0 con 0x55908f9d1280 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50722/0 (socket says 172.21.15.44:50722) 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 -- 172.21.15.44:0/2700534093 learned_addr learned my addr 172.21.15.44:0/2700534093 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59a7e00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 0x7f59840790a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59900015a0 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 -- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9d1280 msgr2=0x55908f9f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9d1280 0x55908f9f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 -- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f6790 msgr2=0x55908f9f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f6790 0x55908f9f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.510+0000 7f59aca00640 1 -- 172.21.15.44:0/2700534093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908f695850 con 0x55908f9f9fc0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.511+0000 7f59aca00640 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f59980824e0 tx=0x7f5998081f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d376299a9796944 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.511+0000 7f599d200640 1 -- 172.21.15.44:0/2700534093 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599804a030 con 0x55908f9f9fc0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.511+0000 7f599d200640 1 -- 172.21.15.44:0/2700534093 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5998082c50 con 0x55908f9f9fc0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.511+0000 7f599d200640 1 -- 172.21.15.44:0/2700534093 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599808b340 con 0x55908f9f9fc0 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 -- 172.21.15.44:0/2700534093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 msgr2=0x55908f9fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f59980824e0 tx=0x7f5998081f60 comp rx=0 tx=0).stop 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 -- 172.21.15.44:0/2700534093 shutdown_connections 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f6790 0x55908f9f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9d1280 0x55908f9f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 --2- 172.21.15.44:0/2700534093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f9fc0 0x55908f9fa3a0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 -- 172.21.15.44:0/2700534093 >> 172.21.15.44:0/2700534093 conn(0x55908f9cc460 msgr2=0x55908f9cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 -- 172.21.15.44:0/2700534093 shutdown_connections 2024-07-17T08:31:46.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.512+0000 7f59b2277740 1 -- 172.21.15.44:0/2700534093 wait complete. 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.513+0000 7f59b2277740 1 -- start start 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.513+0000 7f59b2277740 1 Processor -- start 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f6790 0x55908f9dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55908f6a4620 con 0x55908f9e2d00 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55908f8d8f90 con 0x55908f9f9fc0 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55908f9cab60 con 0x55908f9f6790 2024-07-17T08:31:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59b2277740 10 client.0 _ll_register_callbacks cb 0x55908f8fef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59ad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59ad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50730/0 (socket says 172.21.15.44:50730) 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59ad400640 1 -- 172.21.15.44:0/1898125591 learned_addr learned my addr 172.21.15.44:0/1898125591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.514+0000 7f59aca00640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f6790 0x55908f9dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59a7e00640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 -- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f6790 msgr2=0x55908f9dd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f6790 0x55908f9dd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 -- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 msgr2=0x55908f9f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 -- 172.21.15.44:0/1898125591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908f70c6b0 con 0x55908f9e2d00 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59a7e00640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59ad400640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f59a0001ce0 tx=0x7f59a000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ccd9b98ea43260 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f5995600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0018060 con 0x55908f9e2d00 2024-07-17T08:31:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f59a00052d0 con 0x55908f9e2d00 2024-07-17T08:31:46.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55908f908900 con 0x55908f9e2d00 2024-07-17T08:31:46.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f59a001e050 con 0x55908f9e2d00 2024-07-17T08:31:46.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.515+0000 7f59b2277740 10 client.9756 fetch_fsmap learned FSMap version 105 2024-07-17T08:31:46.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55908f9e3b70 con 0x55908f9e2d00 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55908f908900 con 0x55908f9e2d00 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 105) ==== 3502+0+0 (secure 0 0 0) 0x7f59a0021020 con 0x55908f9e2d00 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 10 client.9756 fetch_fsmap finished waiting for FSMap version 105 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 10 client.9756 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 20 client.9756 populate_metadata read hostname 'smithi044' 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9e0020 0x55908fa28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 4 client.9756 mds_command: new command op to 9612 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:31:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 --> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55908f943cb0 con 0x55908f9e0020 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.516+0000 7f59aca00640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9e0020 0x55908fa28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.517+0000 7f59aca00640 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9e0020 0x55908fa28370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732340e8d269bcb3 server_cookie=aff1bee510931edc in_seq=0 out_seq=0 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.517+0000 7f5995600640 10 client.9756 ms_handle_connect on v2:172.21.15.44:6833/271909788 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f5995600640 1 -- 172.21.15.44:0/1898125591 <== mds.0 v2:172.21.15.44:6833/271909788 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55908f943cb0 con 0x55908f9e0020 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f5995600640 10 client.9756 handle_command_reply: tid=0 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 client.9756 shutdown 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 msgr2=0x55908f9e3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f59a0001ce0 tx=0x7f59a000d520 comp rx=0 tx=0).stop 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 shutdown_connections 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9e0020 0x55908fa28370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9e2d00 0x55908f9e3180 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 --2- 172.21.15.44:0/1898125591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f6790 0x55908f9dd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 >> 172.21.15.44:0/1898125591 conn(0x55908f9d1280 msgr2=0x55908f9fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f5995600640 0 client.9756 ms_handle_reset on v2:172.21.15.44:6833/271909788 2024-07-17T08:31:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 shutdown_connections 2024-07-17T08:31:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.518+0000 7f59b2277740 1 -- 172.21.15.44:0/1898125591 wait complete. 2024-07-17T08:31:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.519+0000 7f59b2277740 20 client.9756 trim_cache size 0 max 16384 2024-07-17T08:31:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.539+0000 7f59b2277740 1 Processor -- start 2024-07-17T08:31:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.539+0000 7f59b2277740 1 -- start start 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 0x55908fa23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f9fc0 0x55908f9f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59a82003b0 con 0x55908f9f6790 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8071f60 con 0x55908f9e2d00 2024-07-17T08:31:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59a81ffdc0 con 0x55908f9f9fc0 2024-07-17T08:31:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50746/0 (socket says 172.21.15.44:50746) 2024-07-17T08:31:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.540+0000 7f59a7e00640 1 -- 172.21.15.44:0/3145968218 learned_addr learned my addr 172.21.15.44:0/3145968218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 -- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f9fc0 msgr2=0x55908f9f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f9fc0 0x55908f9f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 -- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 msgr2=0x55908fa23c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59aca00640 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 0x55908fa23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 0x55908fa23c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 -- 172.21.15.44:0/3145968218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908f9f4090 con 0x55908f9f6790 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59aca00640 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 0x55908fa23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f59a7e00640 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5990014420 tx=0x7f5990010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b290a2c8211e2a2a server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.541+0000 7f5997e00640 1 -- 172.21.15.44:0/3145968218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900127e0 con 0x55908f9f6790 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f5997e00640 1 -- 172.21.15.44:0/3145968218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5990024070 con 0x55908f9f6790 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f5997e00640 1 -- 172.21.15.44:0/3145968218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990012bc0 con 0x55908f9f6790 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 -- 172.21.15.44:0/3145968218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 msgr2=0x55908f9dd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5990014420 tx=0x7f5990010000 comp rx=0 tx=0).stop 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 -- 172.21.15.44:0/3145968218 shutdown_connections 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9f9fc0 0x55908f9f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9dd500 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 --2- 172.21.15.44:0/3145968218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9e2d00 0x55908fa23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 -- 172.21.15.44:0/3145968218 >> 172.21.15.44:0/3145968218 conn(0x55908f9d1280 msgr2=0x55908f9c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 -- 172.21.15.44:0/3145968218 shutdown_connections 2024-07-17T08:31:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.542+0000 7f59b2277740 1 -- 172.21.15.44:0/3145968218 wait complete. 2024-07-17T08:31:46.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.543+0000 7f59b2277740 1 -- start start 2024-07-17T08:31:46.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.543+0000 7f59b2277740 1 Processor -- start 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9e2d00 0x55908fa90940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55908f9eff90 con 0x55908f9f6790 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55908f9ef9d0 con 0x55908f9f9fc0 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55908f9de020 con 0x55908f9e2d00 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.544+0000 7f59b2277740 10 client.0 _ll_register_callbacks cb 0x7f59a0007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50748/0 (socket says 172.21.15.44:50748) 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 -- 172.21.15.44:0/904458510 learned_addr learned my addr 172.21.15.44:0/904458510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59ad400640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59aca00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9e2d00 0x55908fa90940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 -- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9e2d00 msgr2=0x55908fa90940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9e2d00 0x55908fa90940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 -- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 msgr2=0x55908f9c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 -- 172.21.15.44:0/904458510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908f9d1a60 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.545+0000 7f59a7e00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5990020000 tx=0x7f5990012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33821af87769ad77 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990012e90 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f5990034480 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5990024070 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f599003c050 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f59b2277740 10 client.9762 fetch_fsmap learned FSMap version 105 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55908fa24300 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5990024070 con 0x55908f9f6790 2024-07-17T08:31:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 105) ==== 3502+0+0 (secure 0 0 0) 0x7f599003f020 con 0x55908f9f6790 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.546+0000 7f59b2277740 10 client.9762 fetch_fsmap finished waiting for FSMap version 105 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59b2277740 10 client.9762 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59b2277740 20 client.9762 populate_metadata read hostname 'smithi044' 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9f9110 0x55908f9f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59b2277740 4 client.9762 mds_command: new command op to 9612 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 --> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] -- command(tid 0: {"prefix": "status"}) -- 0x55908f93f750 con 0x55908f9f9110 2024-07-17T08:31:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59aca00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9f9110 0x55908f9f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f59aca00640 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9f9110 0x55908f9f94f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d86a2b22e660f8 server_cookie=adb027e15956ab09 in_seq=0 out_seq=0 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.547+0000 7f5995600640 10 client.9762 ms_handle_connect on v2:172.21.15.44:6833/271909788 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f5995600640 1 -- 172.21.15.44:0/904458510 <== mds.0 v2:172.21.15.44:6833/271909788 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55908f93f750 con 0x55908f9f9110 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f5995600640 10 client.9762 handle_command_reply: tid=0 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 client.9762 shutdown 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 msgr2=0x55908f9c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5990020000 tx=0x7f5990012890 comp rx=0 tx=0).stop 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 shutdown_connections 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:6833/271909788,v1:172.21.15.44:6835/271909788] conn(0x55908f9f9110 0x55908f9f94f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55908f9f9fc0 0x55908f9c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55908f9f6790 0x55908f9c98e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 --2- 172.21.15.44:0/904458510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55908f9e2d00 0x55908fa90940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 >> 172.21.15.44:0/904458510 conn(0x55908f9d1280 msgr2=0x55908f9f4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f5995600640 0 client.9762 ms_handle_reset on v2:172.21.15.44:6833/271909788 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 shutdown_connections 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 1 -- 172.21.15.44:0/904458510 wait complete. 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.548+0000 7f59b2277740 20 client.9762 trim_cache size 0 max 16384 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:31:46.552 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 9612, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7611664779999998, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 105, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 206, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 206, 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.664104115000001 2024-07-17T08:31:46.553 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 msgr2=0x7f59840790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 0x7f59840790a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59900015a0 tx=0x7f5990009040 comp rx=0 tx=0).stop 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 msgr2=0x7f59a81f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5998000c00 tx=0x7f5998032c10 comp rx=0 tx=0).stop 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 shutdown_connections 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59a815dd70 0x7f59a81f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5984076be0 0x7f59840790a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59a8154490 0x7f59a8155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 --2- 172.21.15.44:0/2417762678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59a806f4f0 0x7f59a81f4460 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 >> 172.21.15.44:0/2417762678 conn(0x7f59a80cf0d0 msgr2=0x7f59a80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 shutdown_connections 2024-07-17T08:31:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.549+0000 7f59ade00640 1 -- 172.21.15.44:0/2417762678 wait complete. 2024-07-17T08:31:46.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:31:46.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 -- 172.21.15.44:0/3354937043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1458003c80 msgr2=0x7f1458004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 --2- 172.21.15.44:0/3354937043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1458003c80 0x7f1458004080 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f1468009b50 tx=0x7f1458005830 comp rx=0 tx=0).stop 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477400640 1 -- 172.21.15.44:0/3354937043 reap_dead start 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 -- 172.21.15.44:0/3354937043 shutdown_connections 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 -- 172.21.15.44:0/3354937043 >> 172.21.15.44:0/3354937043 conn(0x7f14780cfec0 msgr2=0x7f14780d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 -- 172.21.15.44:0/3354937043 shutdown_connections 2024-07-17T08:31:46.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.772+0000 7f1477e00640 1 -- 172.21.15.44:0/3354937043 wait complete. 2024-07-17T08:31:46.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.774+0000 7f1477e00640 1 Processor -- start 2024-07-17T08:31:46.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.774+0000 7f1477e00640 1 -- start start 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 0x7f14781557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f147815a8f0 0x7f14780727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14780b4ec0 con 0x7f147815ef90 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14780b9270 con 0x7f147815a8f0 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.775+0000 7f1477e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f14780b5040 con 0x7f1458003c80 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50764/0 (socket says 172.21.15.44:50764) 2024-07-17T08:31:46.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 -- 172.21.15.44:0/776296690 learned_addr learned my addr 172.21.15.44:0/776296690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1476000640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f147815a8f0 0x7f14780727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1476a00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 0x7f14781557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 -- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 msgr2=0x7f14781557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 0x7f14781557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 -- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f147815a8f0 msgr2=0x7f14780727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f147815a8f0 0x7f14780727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 -- 172.21.15.44:0/776296690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147815b0f0 con 0x7f147815ef90 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1476a00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 0x7f14781557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.776+0000 7f1477400640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f147000d580 tx=0x7f147000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.777+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700035f0 con 0x7f147815ef90 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.777+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1470003880 con 0x7f147815ef90 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.777+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1478072d20 con 0x7f147815ef90 2024-07-17T08:31:46.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.777+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470003cd0 con 0x7f147815ef90 2024-07-17T08:31:46.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.777+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14780732c0 con 0x7f147815ef90 2024-07-17T08:31:46.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.778+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14780d42b0 con 0x7f147815ef90 2024-07-17T08:31:46.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.779+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f147001b470 con 0x7f147815ef90 2024-07-17T08:31:46.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.780+0000 7f146f400640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 0x7f1454079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:46.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.780+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f14700989d0 con 0x7f147815ef90 2024-07-17T08:31:46.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.781+0000 7f1476a00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 0x7f1454079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:46.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.782+0000 7f1476a00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 0x7f1454079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1468035000 tx=0x7f146802d040 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:46.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:46.784+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f14780d42b0 con 0x7f147815ef90 2024-07-17T08:31:47.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.048+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f14780b5d70 con 0x7f147815ef90 2024-07-17T08:31:47.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.049+0000 7f146f400640 1 -- 172.21.15.44:0/776296690 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7f1470061ac0 con 0x7f147815ef90 2024-07-17T08:31:47.050 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.051+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 msgr2=0x7f1454079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.051+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 0x7f1454079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1468035000 tx=0x7f146802d040 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.051+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 msgr2=0x7f147815f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f147000d580 tx=0x7f147000d980 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 shutdown_connections 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f147815ef90 0x7f147815f410 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f1454076bb0 0x7f1454079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f147815a8f0 0x7f14780727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 --2- 172.21.15.44:0/776296690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1458003c80 0x7f14781557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:47.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 >> 172.21.15.44:0/776296690 conn(0x7f14780cfec0 msgr2=0x7f147815fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:47.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 shutdown_connections 2024-07-17T08:31:47.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47.052+0000 7f1477e00640 1 -- 172.21.15.44:0/776296690 wait complete. 2024-07-17T08:31:47.073 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:31:47.073 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:31:47.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:31:47.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.074 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:31:47.122 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:31:47.123 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36751sec preferred_lft 36751sec 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.124 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:31:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:31:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:31:47.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.127 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:31:47.207 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:31:47.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.211 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:31:47.264 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:31:47.264 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:31:47.264 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:31:47.264 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:31:47.265 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:31:47.265 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:31:47.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:31:47.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:31:47.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.479 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:31:47.529 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:31:47.529 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:31:47.530 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:31:47.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:31:47.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:31:47.617 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:31:47.671 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:31:47.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:31:47.671 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:31:47.723 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:31:47.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:31:47.863 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:31:47.863 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:31:47.863 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:31:47.885 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:31:47.885 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:31:47.885 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:31:47.885 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:31:47.885 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:31:47.914 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:31:47.915 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:31:47.917 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:31:47.918 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:31:47.918 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:31:47.918 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:31:47.918 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:31:47.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:31:47.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:47.923 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:31:48.001 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:31:48.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:31:48.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:31:48.149 INFO:teuthology.orchestra.run.smithi044.stdout:client7442 2024-07-17T08:31:48.150 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:31:48.150 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:31:48.205 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:31:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.412+0000 7f5fbd200640 1 -- 172.21.15.44:0/3759101925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb806f4d0 msgr2=0x7f5fb815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.412+0000 7f5fbd200640 1 --2- 172.21.15.44:0/3759101925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb806f4d0 0x7f5fb815a450 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8005fd0 tx=0x7f5fa802a540 comp rx=0 tx=0).stop 2024-07-17T08:31:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 -- 172.21.15.44:0/3759101925 shutdown_connections 2024-07-17T08:31:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 --2- 172.21.15.44:0/3759101925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb81503c0 0x7f5fb80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 --2- 172.21.15.44:0/3759101925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb806f4d0 0x7f5fb815a450 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 -- 172.21.15.44:0/3759101925 >> 172.21.15.44:0/3759101925 conn(0x7f5fb80cf0d0 msgr2=0x7f5fb80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 -- 172.21.15.44:0/3759101925 shutdown_connections 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.413+0000 7f5fbd200640 1 -- 172.21.15.44:0/3759101925 wait complete. 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.415+0000 7f5fbd200640 1 Processor -- start 2024-07-17T08:31:48.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.415+0000 7f5fbd200640 1 -- start start 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb806f4d0 0x7f5fb81f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 0x7f5fb8158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80b36d0 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80b3550 con 0x7f5fb81503c0 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fbd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb806dd50 con 0x7f5fb806f4d0 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fb6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fb6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50778/0 (socket says 172.21.15.44:50778) 2024-07-17T08:31:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fb6a00640 1 -- 172.21.15.44:0/4014752458 learned_addr learned my addr 172.21.15.44:0/4014752458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fb7400640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb806f4d0 0x7f5fb81f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.416+0000 7f5fb7e00640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 0x7f5fb8158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 -- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb806f4d0 msgr2=0x7f5fb81f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb806f4d0 0x7f5fb81f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 -- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 msgr2=0x7f5fb8158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 0x7f5fb8158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 -- 172.21.15.44:0/4014752458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81f7c80 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb7e00640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 0x7f5fb8158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fb6a00640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c970 tx=0x7f5fa400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4007bb0 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fa4005020 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb815c5f0 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4007e60 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.417+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81f7f40 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.419+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb81f8a00 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.420+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7f5fa4005470 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.421+0000 7f5faf400640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 0x7f5f98079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:48.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.421+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f5fa4098af0 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.422+0000 7f5fb7400640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 0x7f5f98079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:48.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.423+0000 7f5fb7400640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 0x7f5f98079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033000 tx=0x7f5fa8004730 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:48.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.425+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5fa4014030 con 0x7f5fb80b2eb0 2024-07-17T08:31:48.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:48.667+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 --> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5fb815a450 con 0x7f5f98076bb0 2024-07-17T08:31:55.727 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30b6e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.728 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:31:55.726+0000 7f30bb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:31:55.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.741+0000 7f5faf400640 1 -- 172.21.15.44:0/4014752458 <== mgr.7280 v2:172.21.15.175:6834/1104780009 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5fb815a450 con 0x7f5f98076bb0 2024-07-17T08:31:55.742 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:31:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:31:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:31:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:31:55.742 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.744+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 msgr2=0x7f5f98079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.744+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 0x7f5f98079070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033000 tx=0x7f5fa8004730 comp rx=0 tx=0).stop 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 msgr2=0x7f5fb80cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c970 tx=0x7f5fa400ce30 comp rx=0 tx=0).stop 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 shutdown_connections 2024-07-17T08:31:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f5f98076bb0 0x7f5f98079070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fb81503c0 0x7f5fb8158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb80b2eb0 0x7f5fb80cb830 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 --2- 172.21.15.44:0/4014752458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fb806f4d0 0x7f5fb81f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 >> 172.21.15.44:0/4014752458 conn(0x7f5fb80cf0d0 msgr2=0x7f5fb80aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 shutdown_connections 2024-07-17T08:31:55.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.745+0000 7f5fbd200640 1 -- 172.21.15.44:0/4014752458 wait complete. 2024-07-17T08:31:55.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 -- 172.21.15.44:0/4144437604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d40052e0 msgr2=0x7fa6d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/4144437604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d40052e0 0x7fa6d400b9c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c530 tx=0x7fa6d4005830 comp rx=0 tx=0).stop 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 -- 172.21.15.44:0/4144437604 shutdown_connections 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/4144437604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6d40052e0 0x7fa6d400b9c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 -- 172.21.15.44:0/4144437604 >> 172.21.15.44:0/4144437604 conn(0x7fa6f40d04c0 msgr2=0x7fa6f40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 -- 172.21.15.44:0/4144437604 shutdown_connections 2024-07-17T08:31:55.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.996+0000 7fa6f3e00640 1 -- 172.21.15.44:0/4144437604 wait complete. 2024-07-17T08:31:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.998+0000 7fa6f3e00640 1 Processor -- start 2024-07-17T08:31:55.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.998+0000 7fa6f3e00640 1 -- start start 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d40052e0 0x7fa6f415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6f41f41a0 0x7fa6f41f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40d1bb0 con 0x7fa6f40ca720 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40d1a30 con 0x7fa6f41f41a0 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f40ce130 con 0x7fa6d40052e0 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59640/0 (socket says 172.21.15.44:59640) 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f2000640 1 -- 172.21.15.44:0/3862022309 learned_addr learned my addr 172.21.15.44:0/3862022309 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:56.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:55.999+0000 7fa6f2a00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d40052e0 0x7fa6f415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:56.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f3400640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6f41f41a0 0x7fa6f41f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 -- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d40052e0 msgr2=0x7fa6f415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d40052e0 0x7fa6f415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 -- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6f41f41a0 msgr2=0x7fa6f41f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6f41f41a0 0x7fa6f41f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 -- 172.21.15.44:0/3862022309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40cc660 con 0x7fa6f40ca720 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f2000640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e710 tx=0x7fa6dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc00cab0 con 0x7fa6f40ca720 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6dc00cc50 con 0x7fa6f40ca720 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc0052b0 con 0x7fa6f40ca720 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.000+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f40cc920 con 0x7fa6f40ca720 2024-07-17T08:31:56.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.001+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f40ccee0 con 0x7fa6f40ca720 2024-07-17T08:31:56.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.002+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f4161770 con 0x7fa6f40ca720 2024-07-17T08:31:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.003+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 119) ==== 98829+0+0 (secure 0 0 0) 0x7fa6dc004730 con 0x7fa6f40ca720 2024-07-17T08:31:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.004+0000 7fa6eb400640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 0x7fa6d0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.004+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7fa6dc0992a0 con 0x7fa6f40ca720 2024-07-17T08:31:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.005+0000 7fa6f2a00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 0x7fa6d0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:56.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.006+0000 7fa6f2a00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 0x7fa6d0079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4004600 tx=0x7fa6e4006bf0 comp rx=0 tx=0).ready entity=mgr.7280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.009+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa6dc014070 con 0x7fa6f40ca720 2024-07-17T08:31:56.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.282+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa6f40b29f0 con 0x7fa6f40ca720 2024-07-17T08:31:56.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.909+0000 7fa6eb400640 1 -- 172.21.15.44:0/3862022309 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v120) ==== 92+0+0 (secure 0 0 0) 0x7fa6dc062390 con 0x7fa6f40ca720 2024-07-17T08:31:56.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 msgr2=0x7fa6d0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:56.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 0x7fa6d0079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4004600 tx=0x7fa6e4006bf0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 msgr2=0x7fa6f40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:56.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e710 tx=0x7fa6dc00ebd0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 shutdown_connections 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fa6d0076bb0 0x7fa6d0079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6f41f41a0 0x7fa6f41f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6f40ca720 0x7fa6f40caba0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 --2- 172.21.15.44:0/3862022309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d40052e0 0x7fa6f415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 >> 172.21.15.44:0/3862022309 conn(0x7fa6f40d04c0 msgr2=0x7fa6f40d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.913+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 shutdown_connections 2024-07-17T08:31:56.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:56.914+0000 7fa6f3e00640 1 -- 172.21.15.44:0/3862022309 wait complete. 2024-07-17T08:31:56.940 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.129+0000 7f3a0a400640 1 -- 172.21.15.44:0/2613083450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39e8003a50 msgr2=0x7f39e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.129+0000 7f3a0a400640 1 --2- 172.21.15.44:0/2613083450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39e8003a50 0x7f39e8003e70 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009b80 tx=0x7f39e8005830 comp rx=0 tx=0).stop 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.130+0000 7f3a03e00640 1 -- 172.21.15.44:0/2613083450 reap_dead start 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.130+0000 7f3a0a400640 1 -- 172.21.15.44:0/2613083450 shutdown_connections 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.130+0000 7f3a0a400640 1 -- 172.21.15.44:0/2613083450 >> 172.21.15.44:0/2613083450 conn(0x7f3a040b0eb0 msgr2=0x7f3a040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.130+0000 7f3a0a400640 1 -- 172.21.15.44:0/2613083450 shutdown_connections 2024-07-17T08:31:57.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.130+0000 7f3a0a400640 1 -- 172.21.15.44:0/2613083450 wait complete. 2024-07-17T08:31:57.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.132+0000 7f3a0a400640 1 Processor -- start 2024-07-17T08:31:57.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.132+0000 7f3a0a400640 1 -- start start 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 0x7f3a041f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a041f70e0 0x7f3a04074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0406f850 con 0x7f3a04140ee0 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0406f6d0 con 0x7f39e8003a50 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a0a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0415a5a0 con 0x7f3a041f70e0 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a03e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59682/0 (socket says 172.21.15.44:59682) 2024-07-17T08:31:57.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a03e00640 1 -- 172.21.15.44:0/3540230638 learned_addr learned my addr 172.21.15.44:0/3540230638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:57.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a03400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 0x7f3a041f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.133+0000 7f3a02a00640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a041f70e0 0x7f3a04074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a041f70e0 msgr2=0x7f3a04074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a041f70e0 0x7f3a04074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 msgr2=0x7f3a041f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 0x7f3a041f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 -- 172.21.15.44:0/3540230638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041f75e0 con 0x7f3a04140ee0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 0x7f3a041f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a03e00640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cf170 tx=0x7f39fc00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00c8f0 con 0x7f3a04140ee0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39fc018070 con 0x7f3a04140ee0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00ce40 con 0x7f3a04140ee0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a041f78c0 con 0x7f3a04140ee0 2024-07-17T08:31:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.134+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041f7e80 con 0x7f3a04140ee0 2024-07-17T08:31:57.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.136+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a040ae560 con 0x7f3a04140ee0 2024-07-17T08:31:57.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.137+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 120) ==== 98846+0+0 (secure 0 0 0) 0x7f39fc01d460 con 0x7f3a04140ee0 2024-07-17T08:31:57.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.138+0000 7f39fb400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 0x7f39e40790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.138+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7f39fc098ad0 con 0x7f3a04140ee0 2024-07-17T08:31:57.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.138+0000 7f3a03400640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 msgr2=0x7f39e40790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1104780009 2024-07-17T08:31:57.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.138+0000 7f3a03400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 0x7f39e40790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:57.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.142+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f39fc061b30 con 0x7f3a04140ee0 2024-07-17T08:31:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.340+0000 7f3a03400640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 msgr2=0x7f39e40790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1104780009 2024-07-17T08:31:57.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.340+0000 7f3a03400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 0x7f39e40790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.352+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f3a040b41a0 con 0x7f3a04140ee0 2024-07-17T08:31:57.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.359+0000 7f39fb400640 1 -- 172.21.15.44:0/3540230638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v69)=0 v69) ==== 138+0+0 (secure 0 0 0) 0x7f39fc061430 con 0x7f3a04140ee0 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 msgr2=0x7f39e40790f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 0x7f39e40790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 msgr2=0x7f3a04141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cf170 tx=0x7f39fc00eae0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 shutdown_connections 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a04140ee0 0x7f3a04141370 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a041f70e0 0x7f3a04074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7f39e4076c30 0x7f39e40790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 --2- 172.21.15.44:0/3540230638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39e8003a50 0x7f3a041f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 >> 172.21.15.44:0/3540230638 conn(0x7f3a040b0eb0 msgr2=0x7f3a040b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.363+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 shutdown_connections 2024-07-17T08:31:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.364+0000 7f3a0a400640 1 -- 172.21.15.44:0/3540230638 wait complete. 2024-07-17T08:31:57.391 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:31:57.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 -- 172.21.15.44:0/1279717492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b29e0 msgr2=0x7fd1d0166e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1279717492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b29e0 0x7fd1d0166e40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8005fd0 tx=0x7fd1c802a670 comp rx=0 tx=0).stop 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 -- 172.21.15.44:0/1279717492 shutdown_connections 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1279717492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b29e0 0x7fd1d0166e40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1279717492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00b6790 0x7fd1d01648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 -- 172.21.15.44:0/1279717492 >> 172.21.15.44:0/1279717492 conn(0x7fd1d00d3700 msgr2=0x7fd1d00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 -- 172.21.15.44:0/1279717492 shutdown_connections 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.587+0000 7fd1d8400640 1 -- 172.21.15.44:0/1279717492 wait complete. 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.589+0000 7fd1d8400640 1 Processor -- start 2024-07-17T08:31:57.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.589+0000 7fd1d8400640 1 -- start start 2024-07-17T08:31:57.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b6790 0x7fd1d015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00ce1d0 0x7fd1d00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d0167710 con 0x7fd1d00b29e0 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d0167590 con 0x7fd1d00b6790 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d0167890 con 0x7fd1d00ce1d0 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59698/0 (socket says 172.21.15.44:59698) 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d5a00640 1 -- 172.21.15.44:0/1276601496 learned_addr learned my addr 172.21.15.44:0/1276601496 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d5000640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b6790 0x7fd1d015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.590+0000 7fd1d6400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00ce1d0 0x7fd1d00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00ce1d0 msgr2=0x7fd1d00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00ce1d0 0x7fd1d00cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b6790 msgr2=0x7fd1d015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b6790 0x7fd1d015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 -- 172.21.15.44:0/1276601496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d0160550 con 0x7fd1d00b29e0 2024-07-17T08:31:57.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d5a00640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c730 tx=0x7fd1c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:31:57.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4003b70 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1c4003d30 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400aab0 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.591+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d0160830 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.592+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d0153c90 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.593+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d00ce5b0 con 0x7fd1d00b29e0 2024-07-17T08:31:57.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.594+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 120) ==== 98846+0+0 (secure 0 0 0) 0x7fd1c400ac50 con 0x7fd1d00b29e0 2024-07-17T08:31:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.595+0000 7fd1cd600640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 0x7fd1b00790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:31:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.596+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6677+0+0 (secure 0 0 0) 0x7fd1c4098600 con 0x7fd1d00b29e0 2024-07-17T08:31:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.596+0000 7fd1d5000640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 msgr2=0x7fd1b00790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1104780009 2024-07-17T08:31:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.596+0000 7fd1d5000640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 0x7fd1b00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:31:57.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.600+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd1c4061660 con 0x7fd1d00b29e0 2024-07-17T08:31:57.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.797+0000 7fd1d5000640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 msgr2=0x7fd1b00790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1104780009 2024-07-17T08:31:57.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.798+0000 7fd1d5000640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 0x7fd1b00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:31:57.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.812+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd1d00d20b0 con 0x7fd1d00b29e0 2024-07-17T08:31:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.817+0000 7fd1cd600640 1 -- 172.21.15.44:0/1276601496 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v70)=0 v70) ==== 133+0+0 (secure 0 0 0) 0x7fd1c4060f60 con 0x7fd1d00b29e0 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 msgr2=0x7fd1b00790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 0x7fd1b00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 msgr2=0x7fd1d01f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400c730 tx=0x7fd1c400cbf0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 shutdown_connections 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1d00ce1d0 0x7fd1d00cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:6834/1104780009,v1:172.21.15.175:6837/1104780009] conn(0x7fd1b0076c00 0x7fd1b00790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1d00b6790 0x7fd1d015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 --2- 172.21.15.44:0/1276601496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1d00b29e0 0x7fd1d01f7e90 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:31:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 >> 172.21.15.44:0/1276601496 conn(0x7fd1d00d3700 msgr2=0x7fd1d00d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:31:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 shutdown_connections 2024-07-17T08:31:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:31:57.820+0000 7fd1d8400640 1 -- 172.21.15.44:0/1276601496 wait complete. 2024-07-17T08:31:57.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:31:57.839 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T08:32:01.488 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.487+0000 7fdba1800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.487+0000 7fdba1800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.487+0000 7fdba1800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.487+0000 7fdba1800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.487+0000 7fdba1800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.488+0000 7fdb9fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.488+0000 7fdb9fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.488+0000 7fdb9fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.488+0000 7fdb9fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:01.489 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:32:01.488+0000 7fdb9fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:32:08.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:32:08.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:32:38.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:32:38.497 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:32:59.901 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-07-17T08:33:00.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 -- 172.21.15.44:0/3078623678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c152220 msgr2=0x7f313c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 --2- 172.21.15.44:0/3078623678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c152220 0x7f313c06d620 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f3124005fd0 tx=0x7f312402a540 comp rx=0 tx=0).stop 2024-07-17T08:33:00.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 -- 172.21.15.44:0/3078623678 shutdown_connections 2024-07-17T08:33:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 --2- 172.21.15.44:0/3078623678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c06ed30 0x7f313c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 --2- 172.21.15.44:0/3078623678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c152220 0x7f313c06d620 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 -- 172.21.15.44:0/3078623678 >> 172.21.15.44:0/3078623678 conn(0x7f313c0b0eb0 msgr2=0x7f313c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 -- 172.21.15.44:0/3078623678 shutdown_connections 2024-07-17T08:33:00.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.121+0000 7f3141e00640 1 -- 172.21.15.44:0/3078623678 wait complete. 2024-07-17T08:33:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.123+0000 7f3141e00640 1 Processor -- start 2024-07-17T08:33:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.123+0000 7f3141e00640 1 -- start start 2024-07-17T08:33:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f313c06ed30 0x7f313c073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 0x7f313c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0cbbd0 con 0x7f313c0cb3a0 2024-07-17T08:33:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f313c0cba50 con 0x7f313c06ed30 2024-07-17T08:33:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f3141e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f313c0cbd50 con 0x7f313c152220 2024-07-17T08:33:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f313a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f313a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51462/0 (socket says 172.21.15.44:51462) 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.124+0000 7f313a000640 1 -- 172.21.15.44:0/1810781783 learned_addr learned my addr 172.21.15.44:0/1810781783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313aa00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f313c06ed30 0x7f313c073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313b400640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 0x7f313c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 -- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 msgr2=0x7f313c1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 0x7f313c1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 -- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f313c06ed30 msgr2=0x7f313c073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f313c06ed30 0x7f313c073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 -- 172.21.15.44:0/1810781783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c071ff0 con 0x7f313c0cb3a0 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313b400640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 0x7f313c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.125+0000 7f313a000640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f312c00eab0 tx=0x7f312c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.126+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00ce40 con 0x7f313c0cb3a0 2024-07-17T08:33:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.126+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f312c011560 con 0x7f313c0cb3a0 2024-07-17T08:33:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.126+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c011930 con 0x7f313c0cb3a0 2024-07-17T08:33:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.126+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c0722d0 con 0x7f313c0cb3a0 2024-07-17T08:33:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.126+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c072890 con 0x7f313c0cb3a0 2024-07-17T08:33:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.127+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c072f50 con 0x7f313c0cb3a0 2024-07-17T08:33:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.128+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 124) ==== 99005+0+0 (secure 0 0 0) 0x7f312c011ad0 con 0x7f313c0cb3a0 2024-07-17T08:33:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.129+0000 7f3133400640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 0x7f3118079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.129+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 6935+0+0 (secure 0 0 0) 0x7f312c0998e0 con 0x7f313c0cb3a0 2024-07-17T08:33:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.130+0000 7f313aa00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 0x7f3118079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.131+0000 7f313aa00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 0x7f3118079270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3124033000 tx=0x7f3124004730 comp rx=0 tx=0).ready entity=mgr.8101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.134+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f313c072f50 con 0x7f313c0cb3a0 2024-07-17T08:33:00.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.369+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 --> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f313c074490 con 0x7f3118076db0 2024-07-17T08:33:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.565+0000 7f3133400640 1 -- 172.21.15.44:0/1810781783 <== mgr.8101 v2:172.21.15.175:6834/3642155749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f31240042f0 con 0x7f3118076db0 2024-07-17T08:33:00.567 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 msgr2=0x7f3118079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 0x7f3118079270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3124033000 tx=0x7f3124004730 comp rx=0 tx=0).stop 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 msgr2=0x7f313c1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f312c00eab0 tx=0x7f312c00ef70 comp rx=0 tx=0).stop 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 shutdown_connections 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f3118076db0 0x7f3118079270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313c152220 0x7f313c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f313c0cb3a0 0x7f313c1f4900 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 --2- 172.21.15.44:0/1810781783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f313c06ed30 0x7f313c073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 >> 172.21.15.44:0/1810781783 conn(0x7f313c0b0eb0 msgr2=0x7f313c15e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 shutdown_connections 2024-07-17T08:33:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.569+0000 7f3141e00640 1 -- 172.21.15.44:0/1810781783 wait complete. 2024-07-17T08:33:00.590 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T08:33:00.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2024-07-17T08:33:00.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 -- 172.21.15.44:0/3309612884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff0155860 msgr2=0x7f5ff00ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/3309612884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff0155860 0x7f5ff00ad250 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00d0050 tx=0x7f5fe002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 -- 172.21.15.44:0/3309612884 shutdown_connections 2024-07-17T08:33:00.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/3309612884 >> v2:172.21.15.175:3300/0 conn(0x7f5ff00ad790 0x7f5ff00add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/3309612884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff0155860 0x7f5ff00ad250 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 -- 172.21.15.44:0/3309612884 >> 172.21.15.44:0/3309612884 conn(0x7f5ff00cf050 msgr2=0x7f5ff00cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 -- 172.21.15.44:0/3309612884 shutdown_connections 2024-07-17T08:33:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.809+0000 7f5ff5a00640 1 -- 172.21.15.44:0/3309612884 wait complete. 2024-07-17T08:33:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.811+0000 7f5ff5a00640 1 Processor -- start 2024-07-17T08:33:00.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.811+0000 7f5ff5a00640 1 -- start start 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff00ae2c0 0x7f5ff015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff0155860 0x7f5ff015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0073b40 con 0x7f5ff00ad790 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff00739c0 con 0x7f5ff0155860 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5ff5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff0073cc0 con 0x7f5ff00ae2c0 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5fefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51474/0 (socket says 172.21.15.44:51474) 2024-07-17T08:33:00.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.812+0000 7f5fefe00640 1 -- 172.21.15.44:0/2098475119 learned_addr learned my addr 172.21.15.44:0/2098475119 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:33:00.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 -- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff00ae2c0 msgr2=0x7f5ff015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff00ae2c0 0x7f5ff015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 -- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff0155860 msgr2=0x7f5ff015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff0155860 0x7f5ff015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 -- 172.21.15.44:0/2098475119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff00c3fd0 con 0x7f5ff00ad790 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fefe00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0000c00 tx=0x7f5fe0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe003e030 con 0x7f5ff00ad790 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fe0003e00 con 0x7f5ff00ad790 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0035040 con 0x7f5ff00ad790 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff00c42b0 con 0x7f5ff00ad790 2024-07-17T08:33:00.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.813+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff009a5f0 con 0x7f5ff00ad790 2024-07-17T08:33:00.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.815+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff00add80 con 0x7f5ff00ad790 2024-07-17T08:33:00.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.816+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 124) ==== 99005+0+0 (secure 0 0 0) 0x7f5fe0041030 con 0x7f5ff00ad790 2024-07-17T08:33:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.817+0000 7f5fe7e00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 0x7f5fd0079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.817+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 6935+0+0 (secure 0 0 0) 0x7f5fe00b9230 con 0x7f5ff00ad790 2024-07-17T08:33:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.818+0000 7f5fef400640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 0x7f5fd0079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:00.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.819+0000 7f5fef400640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 0x7f5fd0079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5ff006cce0 tx=0x7f5fdc005e50 comp rx=0 tx=0).ready entity=mgr.8101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:00.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:00.821+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5fe0081f30 con 0x7f5ff00ad790 2024-07-17T08:33:01.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.058+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 --> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5ff0153350 con 0x7f5fd0076db0 2024-07-17T08:33:01.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.249+0000 7f5fe7e00640 1 -- 172.21.15.44:0/2098475119 <== mgr.8101 v2:172.21.15.175:6834/3642155749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+39 (secure 0 0 0) 0x7f5ff0153350 con 0x7f5fd0076db0 2024-07-17T08:33:01.250 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path snap_test_dir1 2024-07-17T08:33:01.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.252+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 msgr2=0x7f5fd0079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.252+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 0x7f5fd0079270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5ff006cce0 tx=0x7f5fdc005e50 comp rx=0 tx=0).stop 2024-07-17T08:33:01.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.252+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 msgr2=0x7f5ff0152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.252+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0000c00 tx=0x7f5fe0003910 comp rx=0 tx=0).stop 2024-07-17T08:33:01.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 shutdown_connections 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff0155860 0x7f5ff015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5fd0076db0 0x7f5fd0079270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff00ae2c0 0x7f5ff015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 --2- 172.21.15.44:0/2098475119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff00ad790 0x7f5ff0152e10 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 >> 172.21.15.44:0/2098475119 conn(0x7f5ff00cf050 msgr2=0x7f5ff015cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 shutdown_connections 2024-07-17T08:33:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.253+0000 7f5ff5a00640 1 -- 172.21.15.44:0/2098475119 wait complete. 2024-07-17T08:33:01.276 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-07-17T08:33:01.276 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-07-17-08_34_00 in ~59s... 2024-07-17T08:33:01.276 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2024-07-17T08:33:01.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-07-17T08:33:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 -- 172.21.15.44:0/543903374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440cefe0 msgr2=0x7fc5440cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 --2- 172.21.15.44:0/543903374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440cefe0 0x7fc5440cf470 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fc53c005fd0 tx=0x7fc53c02aa90 comp rx=0 tx=0).stop 2024-07-17T08:33:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 -- 172.21.15.44:0/543903374 shutdown_connections 2024-07-17T08:33:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 --2- 172.21.15.44:0/543903374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5440cefe0 0x7fc5440cf470 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 --2- 172.21.15.44:0/543903374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc54406f5a0 0x7fc5440b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 -- 172.21.15.44:0/543903374 >> 172.21.15.44:0/543903374 conn(0x7fc54406c120 msgr2=0x7fc54406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 -- 172.21.15.44:0/543903374 shutdown_connections 2024-07-17T08:33:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.497+0000 7fc549400640 1 -- 172.21.15.44:0/543903374 wait complete. 2024-07-17T08:33:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.499+0000 7fc549400640 1 Processor -- start 2024-07-17T08:33:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.499+0000 7fc549400640 1 -- start start 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5440cefe0 0x7fc5441f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544156d40 0x7fc544150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc54406d790 con 0x7fc54406f5a0 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc54406d610 con 0x7fc544156d40 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc549400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc54406d910 con 0x7fc5440cefe0 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc543e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc543e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51504/0 (socket says 172.21.15.44:51504) 2024-07-17T08:33:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.500+0000 7fc543e00640 1 -- 172.21.15.44:0/1905719279 learned_addr learned my addr 172.21.15.44:0/1905719279 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc53be00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5440cefe0 0x7fc5441f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 -- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5440cefe0 msgr2=0x7fc5441f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5440cefe0 0x7fc5441f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 -- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544156d40 msgr2=0x7fc544150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544156d40 0x7fc544150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 -- 172.21.15.44:0/1905719279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5441f5e50 con 0x7fc54406f5a0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc543e00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c890 tx=0x7fc53400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004220 con 0x7fc54406f5a0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5340079b0 con 0x7fc54406f5a0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534007de0 con 0x7fc54406f5a0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.501+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5441f2870 con 0x7fc54406f5a0 2024-07-17T08:33:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.502+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5441f1d90 con 0x7fc54406f5a0 2024-07-17T08:33:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.503+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54406f340 con 0x7fc54406f5a0 2024-07-17T08:33:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.504+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 124) ==== 99005+0+0 (secure 0 0 0) 0x7fc534005020 con 0x7fc54406f5a0 2024-07-17T08:33:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.505+0000 7fc540c00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 0x7fc5200792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:33:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.505+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(208..208 src has 1..208) ==== 7051+0+0 (secure 0 0 0) 0x7fc534099130 con 0x7fc54406f5a0 2024-07-17T08:33:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.506+0000 7fc53be00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 0x7fc5200792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:33:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.507+0000 7fc53be00640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 0x7fc5200792a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc544073bc0 tx=0x7fc52c009040 comp rx=0 tx=0).ready entity=mgr.8101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:33:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.509+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc54406f340 con 0x7fc54406f5a0 2024-07-17T08:33:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.742+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 --> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5440b2140 con 0x7fc520076de0 2024-07-17T08:33:01.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.757+0000 7fc540c00640 1 -- 172.21.15.44:0/1905719279 <== mgr.8101 v2:172.21.15.175:6834/3642155749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7fc5440b2140 con 0x7fc520076de0 2024-07-17T08:33:01.758 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:33:01.759 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-07-17T08:33:01.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.760+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 msgr2=0x7fc5200792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.760+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 0x7fc5200792a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc544073bc0 tx=0x7fc52c009040 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.760+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 msgr2=0x7fc544073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.760+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c890 tx=0x7fc53400cd50 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 shutdown_connections 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc544156d40 0x7fc544150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc520076de0 0x7fc5200792a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5440cefe0 0x7fc5441f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 --2- 172.21.15.44:0/1905719279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc54406f5a0 0x7fc544073dd0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 >> 172.21.15.44:0/1905719279 conn(0x7fc54406c120 msgr2=0x7fc544151750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 shutdown_connections 2024-07-17T08:33:01.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:33:01.761+0000 7fc549400640 1 -- 172.21.15.44:0/1905719279 wait complete. 2024-07-17T08:33:01.781 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-07-17T08:33:01.781 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2024-07-17T08:33:01.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:01.782 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:01.837 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:01.837 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:01.838 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:02.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:02.839 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:02.901 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:02.901 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:02.901 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:03.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:03.901 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:03.963 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:03.963 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:03.963 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:04.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:04.965 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:05.027 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:05.027 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:05.027 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:06.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:06.028 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:06.092 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:06.092 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:06.092 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:07.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:07.093 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:07.155 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:07.155 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:07.156 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:08.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:08.157 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:08.219 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:08.219 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:08.219 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:08.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:33:08.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:33:09.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:09.220 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:09.282 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:09.282 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:09.282 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:10.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:10.283 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:10.345 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:10.345 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:10.345 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:11.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:11.346 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:11.404 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:11.404 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:11.405 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:12.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:12.405 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:12.464 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:12.464 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:12.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:13.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:13.465 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:13.526 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:13.526 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:13.526 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:14.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:14.528 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:14.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:14.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:14.584 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:15.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:15.585 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:15.646 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:15.646 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:15.646 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:16.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:16.648 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:16.709 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:16.710 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:16.710 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:17.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:17.711 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:17.772 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:17.773 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:17.773 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:18.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:18.774 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:18.838 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:18.838 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:18.838 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:19.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:19.839 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:19.899 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:19.900 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:19.900 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:20.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:20.900 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:20.961 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:20.961 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:20.961 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:21.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:21.962 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:22.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:22.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:22.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:23.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:23.026 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:23.089 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:23.089 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:23.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:24.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:24.090 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:24.152 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:24.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:24.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:25.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:25.154 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:25.216 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:25.216 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:25.216 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:26.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:26.217 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:26.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:26.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:26.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:27.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:27.277 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:27.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:27.337 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:27.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:28.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:28.338 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:28.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:28.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:28.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:29.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:29.402 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:29.460 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:29.461 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:29.461 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:30.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:30.462 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:30.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:30.523 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:30.523 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:31.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:31.524 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:31.585 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:31.585 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:31.585 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:32.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:32.586 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:32.648 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:32.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:32.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:33.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:33.649 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:33.707 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:33.707 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:33.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:34.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:34.709 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:34.773 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:34.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:34.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:35.774 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:35.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:35.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:35.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:36.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:36.836 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:36.897 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:36.898 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:36.898 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:37.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:37.899 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:37.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:37.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:37.959 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:38.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:38.961 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:39.023 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:39.023 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:39.023 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:39.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:33:39.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:33:40.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:40.024 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:40.086 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:40.086 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:40.086 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:41.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:41.087 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:41.148 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:41.148 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:41.148 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:42.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:42.150 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:42.211 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:42.212 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:42.212 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:43.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:43.213 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:43.274 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:43.274 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:43.274 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:44.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:44.275 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:44.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:44.337 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:44.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:45.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:45.338 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:45.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:45.400 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:45.400 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:46.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:46.401 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:46.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:46.464 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:46.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:47.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:47.465 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:47.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:47.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:47.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:48.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:48.529 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:48.591 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:48.591 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:48.591 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:49.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:49.592 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:49.653 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:49.653 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:49.653 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:50.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:50.655 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:50.716 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:50.716 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:50.716 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:51.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:51.718 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:51.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:51.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:51.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:52.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:52.781 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:52.843 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:52.843 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:52.843 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:53.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:53.844 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:53.906 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:53.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:53.907 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:54.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:54.908 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:54.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:54.970 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:54.970 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:55.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:55.971 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:56.033 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:56.033 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:56.034 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:57.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:57.035 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:57.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:57.098 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:57.098 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:58.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:58.098 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:58.161 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:58.161 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:58.161 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:33:59.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:33:59.162 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:33:59.225 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2024-07-17T08:33:59.225 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:33:59.225 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:00.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:00.226 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:00.271 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:34:00.270+0000 7fdb89200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:34:00.271 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:34:00.270+0000 7fdb89200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:34:00.271 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:34:00.270+0000 7fdb89200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:34:00.271 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:34:00.270+0000 7fdb89200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:34:00.271 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:34:00.270+0000 7fdb89200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:34:00.291 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:00.293 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:00.293 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:00.293 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:00.293 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2024-07-17-08_34_00_UTC'] 2024-07-17T08:34:00.293 DEBUG:tasks.cephfs.test_snap_schedules:exec=1721205181.276475, now = 1721205240.2936976, timo = 59 2024-07-17T08:34:01.294 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2024-07-17-08_34_00 in ~119s... 2024-07-17T08:34:01.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:01.295 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:01.354 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:01.356 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:01.356 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:01.356 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:02.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:02.357 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:02.418 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:02.419 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:02.419 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:02.419 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:03.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:03.421 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:03.482 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:03.484 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:03.484 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:03.484 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:04.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:04.485 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:04.546 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:04.548 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:04.548 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:04.548 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:05.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:05.549 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:05.610 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:05.611 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:05.611 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:05.611 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:06.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:06.613 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:06.672 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:06.674 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:06.674 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:06.674 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:07.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:07.675 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:07.738 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:07.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:07.739 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:07.739 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:08.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:08.740 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:08.799 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:08.801 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:08.801 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:08.801 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:09.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:34:09.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:34:09.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:09.802 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:09.864 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:09.865 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:09.866 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:09.866 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:10.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:10.867 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:10.926 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:10.928 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:10.928 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:10.928 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:11.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:11.929 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:11.989 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:11.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:11.991 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:11.991 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:12.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:12.992 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:13.052 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:13.055 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:13.055 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:13.055 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:14.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:14.056 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:14.115 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:14.116 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:14.116 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:14.116 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:15.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:15.118 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:15.178 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:15.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:15.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:15.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:16.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:16.182 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:16.241 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:16.242 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:16.242 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:16.243 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:17.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:17.243 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:17.304 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:17.305 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:17.305 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:17.305 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:18.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:18.307 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:18.367 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:18.369 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:18.369 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:18.369 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:19.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:19.370 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:19.430 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:19.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:19.432 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:19.432 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:20.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:20.433 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:20.488 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:20.489 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:20.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:20.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:21.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:21.491 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:21.551 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:21.553 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:21.553 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:21.553 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:22.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:22.554 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:22.612 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:22.613 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:22.613 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:22.613 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:23.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:23.615 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:23.675 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:23.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:23.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:23.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:24.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:24.678 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:24.736 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:24.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:24.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:24.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:25.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:25.738 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:25.797 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:25.797 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:25.797 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:26.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:26.799 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:26.860 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:26.861 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:26.861 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:26.861 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:27.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:27.862 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:27.921 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:27.923 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:27.923 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:27.923 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:28.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:28.924 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:28.983 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:28.985 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:28.985 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:28.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:29.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:29.986 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:30.046 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:30.047 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:30.047 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:30.047 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:31.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:31.049 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:31.107 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:31.108 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:31.108 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:31.108 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:32.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:32.109 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:32.169 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:32.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:32.171 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:32.171 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:33.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:33.171 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:33.231 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:33.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:33.233 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:33.233 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:34.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:34.234 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:34.295 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:34.297 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:34.297 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:34.297 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:35.298 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:35.357 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:35.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:35.358 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:35.358 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:36.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:36.359 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:36.420 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:36.422 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:36.422 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:36.422 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:37.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:37.423 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:37.483 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:37.485 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:37.485 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:37.485 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:38.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:38.486 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:38.545 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:38.547 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:38.547 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:38.547 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:39.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:34:39.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:34:39.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:39.548 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:39.608 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:39.609 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:39.609 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:39.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:40.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:40.610 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:40.669 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:40.671 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:40.671 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:40.671 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:41.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:41.672 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:41.733 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:41.734 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:41.734 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:41.734 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:42.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:42.736 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:42.796 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:42.797 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:42.797 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:42.797 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:43.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:43.799 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:43.855 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:43.856 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:43.857 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:43.857 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:44.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:44.858 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:44.919 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:44.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:44.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:44.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:45.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:45.922 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:45.984 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:45.986 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:45.986 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:45.986 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:46.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:46.988 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:47.047 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:47.049 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:47.049 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:47.049 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:48.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:48.050 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:48.111 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:48.113 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:48.113 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:48.113 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:49.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:49.114 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:49.174 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:49.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:49.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:49.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:50.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:50.176 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:50.236 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:50.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:50.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:50.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:51.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:51.239 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:51.300 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:51.302 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:51.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:51.302 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:52.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:52.303 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:52.364 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:52.366 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:52.366 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:52.366 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:53.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:53.367 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:53.426 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:53.428 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:53.428 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:53.428 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:54.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:54.429 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:54.490 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:54.491 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:54.491 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:54.491 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:55.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:55.493 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:55.553 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:55.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:55.554 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:55.554 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:56.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:56.556 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:56.616 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:56.618 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:56.618 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:56.618 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:57.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:57.619 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:57.679 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:57.680 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:57.680 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:57.680 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:58.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:58.682 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:58.744 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:58.746 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:58.746 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:58.746 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:34:59.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:34:59.747 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:34:59.808 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_34_00_UTC 2024-07-17T08:34:59.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:34:59.809 INFO:tasks.cephfs.test_snap_schedules:added: set() 2024-07-17T08:34:59.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2024-07-17T08:35:00.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:00.811 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:35:00.872 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_35_00_UTC 2024-07-17T08:35:00.873 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2024-07-17-08_35_00_UTC'} 2024-07-17T08:35:00.873 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2024-07-17-08_35_00_UTC'} 2024-07-17T08:35:00.874 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2024-07-17-08_34_00_UTC'} 2024-07-17T08:35:00.874 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2024-07-17-08_34_00_UTC'] 2024-07-17T08:35:00.874 DEBUG:tasks.cephfs.test_snap_schedules:exec=1721205181.276475, now = 1721205300.8742507, timo = 119 2024-07-17T08:35:01.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-07-17T08:35:02.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 -- 172.21.15.44:0/4126088662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 msgr2=0x7f6770072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 --2- 172.21.15.44:0/4126088662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f6770072d10 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f67700735e0 tx=0x7f676002a410 comp rx=0 tx=0).stop 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 -- 172.21.15.44:0/4126088662 shutdown_connections 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 --2- 172.21.15.44:0/4126088662 >> v2:172.21.15.175:3300/0 conn(0x7f677015e1d0 0x7f67700b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 --2- 172.21.15.44:0/4126088662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f6770072d10 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 -- 172.21.15.44:0/4126088662 >> 172.21.15.44:0/4126088662 conn(0x7f677006bf40 msgr2=0x7f677006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.096+0000 7f6776c00640 1 -- 172.21.15.44:0/4126088662 shutdown_connections 2024-07-17T08:35:02.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.097+0000 7f6776c00640 1 -- 172.21.15.44:0/4126088662 wait complete. 2024-07-17T08:35:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.098+0000 7f6776c00640 1 Processor -- start 2024-07-17T08:35:02.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.099+0000 7f6776c00640 1 -- start start 2024-07-17T08:35:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 0x7f67701488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 0x7f6770156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67700ad8a0 con 0x7f67701544f0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67701548d0 con 0x7f677015e1d0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6776c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67700ada20 con 0x7f67700b2cd0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f676f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f676f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55856/0 (socket says 172.21.15.44:55856) 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f676f400640 1 -- 172.21.15.44:0/3364132564 learned_addr learned my addr 172.21.15.44:0/3364132564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f6774c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 0x7f6770156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f676f400640 1 -- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 msgr2=0x7f67701488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.100+0000 7f676fe00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 0x7f67701488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676f400640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 0x7f67701488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676f400640 1 -- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 msgr2=0x7f6770156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676f400640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 0x7f6770156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676f400640 1 -- 172.21.15.44:0/3364132564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67701f4590 con 0x7f67701544f0 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6774c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 0x7f6770156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676fe00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 0x7f67701488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f676f400640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c730 tx=0x7f675c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c016020 con 0x7f67701544f0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f675c00a930 con 0x7f67701544f0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67701f4870 con 0x7f67701544f0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c00ad80 con 0x7f67701544f0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.101+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701f4e30 con 0x7f67701544f0 2024-07-17T08:35:02.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.102+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6770072d10 con 0x7f67701544f0 2024-07-17T08:35:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.104+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 126) ==== 98960+0+0 (secure 0 0 0) 0x7f675c019020 con 0x7f67701544f0 2024-07-17T08:35:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.105+0000 7f6767e00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 0x7f67500791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.105+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 7051+0+0 (secure 0 0 0) 0x7f675c098b40 con 0x7f67701544f0 2024-07-17T08:35:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.105+0000 7f676fe00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 0x7f67500791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.107+0000 7f676fe00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 0x7f67500791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f6760004730 comp rx=0 tx=0).ready entity=mgr.8101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:02.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.109+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f675c020020 con 0x7f67701544f0 2024-07-17T08:35:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.346+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 --> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f6770070510 con 0x7f6750076d20 2024-07-17T08:35:02.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.549+0000 7f6767e00640 1 -- 172.21.15.44:0/3364132564 <== mgr.8101 v2:172.21.15.175:6834/3642155749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6770070510 con 0x7f6750076d20 2024-07-17T08:35:02.551 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path snap_test_dir1 2024-07-17T08:35:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 msgr2=0x7f67500791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 0x7f67500791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f6760004730 comp rx=0 tx=0).stop 2024-07-17T08:35:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 msgr2=0x7f677015a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c730 tx=0x7f675c00cbf0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 shutdown_connections 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f677015e1d0 0x7f6770156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67701544f0 0x7f677015a040 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f6750076d20 0x7f67500791e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 --2- 172.21.15.44:0/3364132564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67700b2cd0 0x7f67701488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.553+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 >> 172.21.15.44:0/3364132564 conn(0x7f677006bf40 msgr2=0x7f677006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:02.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.554+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 shutdown_connections 2024-07-17T08:35:02.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.554+0000 7f6776c00640 1 -- 172.21.15.44:0/3364132564 wait complete. 2024-07-17T08:35:02.579 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-07-17T08:35:02.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:02.580 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:35:02.634 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_35_00_UTC 2024-07-17T08:35:02.635 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2024-07-17-08_35_00_UTC 2024-07-17T08:35:02.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:02.635 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2024-07-17-08_35_00_UTC) 2024-07-17T08:35:02.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:02.698 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T08:35:02.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 -- 172.21.15.44:0/1753598545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c15d350 msgr2=0x7fc84c161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 --2- 172.21.15.44:0/1753598545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c15d350 0x7fc84c161770 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b3050 tx=0x7fc84002a4f0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 -- 172.21.15.44:0/1753598545 shutdown_connections 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 --2- 172.21.15.44:0/1753598545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c0d1440 0x7fc84c0d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 --2- 172.21.15.44:0/1753598545 >> v2:172.21.15.175:3300/0 conn(0x7fc84c0cdca0 0x7fc84c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 --2- 172.21.15.44:0/1753598545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c15d350 0x7fc84c161770 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 -- 172.21.15.44:0/1753598545 >> 172.21.15.44:0/1753598545 conn(0x7fc84c0d04c0 msgr2=0x7fc84c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 -- 172.21.15.44:0/1753598545 shutdown_connections 2024-07-17T08:35:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.982+0000 7fc853200640 1 -- 172.21.15.44:0/1753598545 wait complete. 2024-07-17T08:35:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.984+0000 7fc853200640 1 Processor -- start 2024-07-17T08:35:02.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.984+0000 7fc853200640 1 -- start start 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 0x7fc84c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 0x7fc84c0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06e3c0 con 0x7fc84c0d1440 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06e240 con 0x7fc84c0cdca0 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc853200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c06e540 con 0x7fc84c15d350 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc851400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc851400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55868/0 (socket says 172.21.15.44:55868) 2024-07-17T08:35:02.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.985+0000 7fc851400640 1 -- 172.21.15.44:0/1913210765 learned_addr learned my addr 172.21.15.44:0/1913210765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:02.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 -- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 msgr2=0x7fc84c0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851e00640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 0x7fc84c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc852800640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 0x7fc84c0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 0x7fc84c0b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 -- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 msgr2=0x7fc84c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 0x7fc84c15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 -- 172.21.15.44:0/1913210765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c154170 con 0x7fc84c0d1440 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc852800640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 0x7fc84c0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851e00640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 0x7fc84c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc851400640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00d770 tx=0x7fc83c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c017070 con 0x7fc84c0d1440 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc83c009d20 con 0x7fc84c0d1440 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c005280 con 0x7fc84c0d1440 2024-07-17T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.986+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c154450 con 0x7fc84c0d1440 2024-07-17T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.987+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c153140 con 0x7fc84c0d1440 2024-07-17T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.988+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c0c9ff0 con 0x7fc84c0d1440 2024-07-17T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.989+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 126) ==== 98960+0+0 (secure 0 0 0) 0x7fc83c005420 con 0x7fc84c0d1440 2024-07-17T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.990+0000 7fc846000640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 0x7fc8300791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.990+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 7051+0+0 (secure 0 0 0) 0x7fc83c098c50 con 0x7fc84c0d1440 2024-07-17T08:35:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.990+0000 7fc851e00640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 0x7fc8300791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.992+0000 7fc851e00640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 0x7fc8300791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc840033000 tx=0x7fc840004730 comp rx=0 tx=0).ready entity=mgr.8101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:02.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:02.996+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc84c0c9ff0 con 0x7fc84c0d1440 2024-07-17T08:35:03.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.261+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fc84c06ead0 con 0x7fc84c0d1440 2024-07-17T08:35:03.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.383+0000 7fc846000640 1 -- 172.21.15.44:0/1913210765 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v127) ==== 93+0+0 (secure 0 0 0) 0x7fc83c061920 con 0x7fc84c0d1440 2024-07-17T08:35:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.386+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 msgr2=0x7fc8300791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.386+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 0x7fc8300791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc840033000 tx=0x7fc840004730 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.386+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 msgr2=0x7fc84c149910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.386+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00d770 tx=0x7fc83c00dc30 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 shutdown_connections 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc830076d20 0x7fc8300791e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc84c15d350 0x7fc84c0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc84c0d1440 0x7fc84c149910 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 --2- 172.21.15.44:0/1913210765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc84c0cdca0 0x7fc84c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 >> 172.21.15.44:0/1913210765 conn(0x7fc84c0d04c0 msgr2=0x7fc84c0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.387+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 shutdown_connections 2024-07-17T08:35:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:03.388+0000 7fc853200640 1 -- 172.21.15.44:0/1913210765 wait complete. 2024-07-17T08:35:03.412 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:35:03.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:03.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:03.629 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:35:03.629 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:03.629 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:03.629 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:03.679 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:03.679 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:03.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:35:04.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 -- 172.21.15.44:0/1659070415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 msgr2=0x7f76b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 --2- 172.21.15.44:0/1659070415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76b4004800 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009b20 tx=0x7f76b4005830 comp rx=0 tx=0).stop 2024-07-17T08:35:04.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d5c00640 1 -- 172.21.15.44:0/1659070415 reap_dead start 2024-07-17T08:35:04.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 -- 172.21.15.44:0/1659070415 shutdown_connections 2024-07-17T08:35:04.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 -- 172.21.15.44:0/1659070415 >> 172.21.15.44:0/1659070415 conn(0x7f76d009aa30 msgr2=0x7f76d00b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:04.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 -- 172.21.15.44:0/1659070415 shutdown_connections 2024-07-17T08:35:04.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.018+0000 7f76d7c00640 1 -- 172.21.15.44:0/1659070415 wait complete. 2024-07-17T08:35:04.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.020+0000 7f76d7c00640 1 Processor -- start 2024-07-17T08:35:04.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.020+0000 7f76d7c00640 1 -- start start 2024-07-17T08:35:04.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76d01f4c80 0x7f76d015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76d00aee80 0x7f76d00af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76d00ae720 con 0x7f76b4003fd0 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76d00ae5a0 con 0x7f76d01f4c80 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76d00cab10 con 0x7f76d00aee80 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.021+0000 7f76d5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55910/0 (socket says 172.21.15.44:55910) 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 -- 172.21.15.44:0/457563894 learned_addr learned my addr 172.21.15.44:0/457563894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76cfe00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76d01f4c80 0x7f76d015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76d00aee80 0x7f76d00af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76d00aee80 msgr2=0x7f76d00af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76d00aee80 0x7f76d00af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76d01f4c80 msgr2=0x7f76d015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76d01f4c80 0x7f76d015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 -- 172.21.15.44:0/457563894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d00b5b30 con 0x7f76b4003fd0 2024-07-17T08:35:04.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.022+0000 7f76d5200640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f76c000e930 tx=0x7f76c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.023+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0015070 con 0x7f76b4003fd0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.023+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c000ce00 con 0x7f76b4003fd0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.023+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00116c0 con 0x7f76b4003fd0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.023+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d015eb20 con 0x7f76b4003fd0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.023+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d015f0d0 con 0x7f76b4003fd0 2024-07-17T08:35:04.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.024+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d00b37c0 con 0x7f76b4003fd0 2024-07-17T08:35:04.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.025+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f76c0018070 con 0x7f76b4003fd0 2024-07-17T08:35:04.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.026+0000 7f76ccc00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 0x7f76b00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.027+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 7051+0+0 (secure 0 0 0) 0x7f76c0099170 con 0x7f76b4003fd0 2024-07-17T08:35:04.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.027+0000 7f76cfe00640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 msgr2=0x7f76b00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:04.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.027+0000 7f76cfe00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 0x7f76b00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:04.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.031+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f76c0061890 con 0x7f76b4003fd0 2024-07-17T08:35:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.228+0000 7f76cfe00640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 msgr2=0x7f76b00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:04.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.229+0000 7f76cfe00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 0x7f76b00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:04.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.255+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76d00b06b0 con 0x7f76b4003fd0 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.256+0000 7f76ccc00640 1 -- 172.21.15.44:0/457563894 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v209) ==== 80+0+357 (secure 0 0 0) 0x7f76c006a120 con 0x7f76b4003fd0 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/1104780009 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/1104780009 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/340287962 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3551582397 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/693817936 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.257 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/464369211 2024-07-18T08:32:01.140259+0000 2024-07-17T08:35:04.258 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 msgr2=0x7f76b00791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 0x7f76b00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 msgr2=0x7f76d0154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f76c000e930 tx=0x7f76c000edf0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 shutdown_connections 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f76b0076d00 0x7f76b00791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76d00aee80 0x7f76d00af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76d01f4c80 0x7f76d015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 --2- 172.21.15.44:0/457563894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b4003fd0 0x7f76d0154140 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 >> 172.21.15.44:0/457563894 conn(0x7f76d009aa30 msgr2=0x7f76d006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:04.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 shutdown_connections 2024-07-17T08:35:04.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.259+0000 7f76d7c00640 1 -- 172.21.15.44:0/457563894 wait complete. 2024-07-17T08:35:04.278 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:35:04.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:04.278 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:04.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:35:04.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:04.361 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:35:04.401 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:04.401 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:35:04.401 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:04.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:35:04.445 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:04.445 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:04.445 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:04.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:35:04.489 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:04.489 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:35:04.489 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:04.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:35:04.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 -- 172.21.15.44:0/3455672331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b18003a50 msgr2=0x7f1b18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 --2- 172.21.15.44:0/3455672331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b18003a50 0x7f1b18003e70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1b240016c0 tx=0x7f1b18005830 comp rx=0 tx=0).stop 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 -- 172.21.15.44:0/3455672331 shutdown_connections 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 --2- 172.21.15.44:0/3455672331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b18003a50 0x7f1b18003e70 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 -- 172.21.15.44:0/3455672331 >> 172.21.15.44:0/3455672331 conn(0x7f1b340d00a0 msgr2=0x7f1b340d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:04.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 -- 172.21.15.44:0/3455672331 shutdown_connections 2024-07-17T08:35:04.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.738+0000 7f1b38a00640 1 -- 172.21.15.44:0/3455672331 wait complete. 2024-07-17T08:35:04.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.741+0000 7f1b38a00640 1 Processor -- start 2024-07-17T08:35:04.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.741+0000 7f1b38a00640 1 -- start start 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.742+0000 7f1b38a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 0x7f1b3415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.742+0000 7f1b38a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b340cc460 0x7f1b341f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b38a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b38a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3406e480 con 0x7f1b341f7e50 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b38a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b340b5740 con 0x7f1b18003a50 2024-07-17T08:35:04.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b38a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3406e600 con 0x7f1b340cc460 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55930/0 (socket says 172.21.15.44:55930) 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 -- 172.21.15.44:0/1619327158 learned_addr learned my addr 172.21.15.44:0/1619327158 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b31e00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 0x7f1b3415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b31400640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b340cc460 0x7f1b341f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b340cc460 msgr2=0x7f1b341f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b340cc460 0x7f1b341f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 msgr2=0x7f1b3415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 0x7f1b3415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b32800640 1 -- 172.21.15.44:0/1619327158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b340cca00 con 0x7f1b341f7e50 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.743+0000 7f1b31e00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 0x7f1b3415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:04.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b32800640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f1b340d2cf0 tx=0x7f1b2c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:04.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c003e40 con 0x7f1b341f7e50 2024-07-17T08:35:04.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b2c015070 con 0x7f1b341f7e50 2024-07-17T08:35:04.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e9a0 con 0x7f1b341f7e50 2024-07-17T08:35:04.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b340ccce0 con 0x7f1b341f7e50 2024-07-17T08:35:04.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.744+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b340cd2a0 con 0x7f1b341f7e50 2024-07-17T08:35:04.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.745+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b34161020 con 0x7f1b341f7e50 2024-07-17T08:35:04.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.746+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f1b2c00eb40 con 0x7f1b341f7e50 2024-07-17T08:35:04.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.748+0000 7f1b2a000640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 0x7f1b140791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:04.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.748+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 7051+0+0 (secure 0 0 0) 0x7f1b2c099520 con 0x7f1b341f7e50 2024-07-17T08:35:04.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.748+0000 7f1b31e00640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 msgr2=0x7f1b140791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:04.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.748+0000 7f1b31e00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 0x7f1b140791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:04.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.752+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1b2c062300 con 0x7f1b341f7e50 2024-07-17T08:35:04.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.950+0000 7f1b31e00640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 msgr2=0x7f1b140791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:04.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:04.950+0000 7f1b31e00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 0x7f1b140791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:05.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.003+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b34158770 con 0x7f1b341f7e50 2024-07-17T08:35:05.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.004+0000 7f1b2a000640 1 -- 172.21.15.44:0/1619327158 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7f1b2c061c00 con 0x7f1b341f7e50 2024-07-17T08:35:05.005 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:05.006 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:35:05.006 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:35:05.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.008+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 msgr2=0x7f1b140791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.008+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 0x7f1b140791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.008+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 msgr2=0x7f1b341f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.008+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f1b340d2cf0 tx=0x7f1b2c010f40 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 shutdown_connections 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b341f7e50 0x7f1b341f82d0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f1b14076d00 0x7f1b140791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b340cc460 0x7f1b341f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 --2- 172.21.15.44:0/1619327158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b18003a50 0x7f1b3415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 >> 172.21.15.44:0/1619327158 conn(0x7f1b340d00a0 msgr2=0x7f1b340b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 shutdown_connections 2024-07-17T08:35:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.009+0000 7f1b38a00640 1 -- 172.21.15.44:0/1619327158 wait complete. 2024-07-17T08:35:05.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 -- 172.21.15.44:0/2912084749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 msgr2=0x7f03340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 --2- 172.21.15.44:0/2912084749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f03340051c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0344009ed0 tx=0x7f0334005830 comp rx=0 tx=0).stop 2024-07-17T08:35:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0351e00640 1 -- 172.21.15.44:0/2912084749 reap_dead start 2024-07-17T08:35:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 -- 172.21.15.44:0/2912084749 shutdown_connections 2024-07-17T08:35:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 -- 172.21.15.44:0/2912084749 >> 172.21.15.44:0/2912084749 conn(0x7f034c0cfec0 msgr2=0x7f034c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 -- 172.21.15.44:0/2912084749 shutdown_connections 2024-07-17T08:35:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.232+0000 7f0352800640 1 -- 172.21.15.44:0/2912084749 wait complete. 2024-07-17T08:35:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.234+0000 7f0352800640 1 Processor -- start 2024-07-17T08:35:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.234+0000 7f0352800640 1 -- start start 2024-07-17T08:35:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034c0cb560 0x7f034c1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034c0cb9e0 0x7f034c1f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f034c15d980 con 0x7f0334004d40 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f034c15d800 con 0x7f034c0cb560 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0352800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f034c15db00 con 0x7f034c0cb9e0 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55944/0 (socket says 172.21.15.44:55944) 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 -- 172.21.15.44:0/1370603079 learned_addr learned my addr 172.21.15.44:0/1370603079 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0350a00640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034c0cb560 0x7f034c1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351e00640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034c0cb9e0 0x7f034c1f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034c0cb9e0 msgr2=0x7f034c1f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034c0cb9e0 0x7f034c1f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034c0cb560 msgr2=0x7f034c1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.235+0000 7f0351400640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034c0cb560 0x7f034c1f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0351400640 1 -- 172.21.15.44:0/1370603079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c0cbfb0 con 0x7f0334004d40 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0351400640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0340006680 tx=0x7f03400066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340015070 con 0x7f0334004d40 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c0cc290 con 0x7f0334004d40 2024-07-17T08:35:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0340011920 con 0x7f0334004d40 2024-07-17T08:35:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340011d70 con 0x7f0334004d40 2024-07-17T08:35:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.236+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c0cc850 con 0x7f0334004d40 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.237+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c0b5d30 con 0x7f0334004d40 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.239+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f0340010420 con 0x7f0334004d40 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.239+0000 7f0349600640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 0x7f032c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.240+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 7051+0+0 (secure 0 0 0) 0x7f03400998e0 con 0x7f0334004d40 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.240+0000 7f0350a00640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 msgr2=0x7f032c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:05.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.240+0000 7f0350a00640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 0x7f032c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.248+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0340062530 con 0x7f0334004d40 2024-07-17T08:35:05.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.441+0000 7f0350a00640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 msgr2=0x7f032c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:05.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.442+0000 7f0350a00640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 0x7f032c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:05.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.508+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f034c15d2c0 con 0x7f0334004d40 2024-07-17T08:35:05.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.509+0000 7f0349600640 1 -- 172.21.15.44:0/1370603079 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7f0340061e30 con 0x7f0334004d40 2024-07-17T08:35:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:05.511 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:35:05.511 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:35:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 msgr2=0x7f032c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 0x7f032c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 msgr2=0x7f034c0ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0340006680 tx=0x7f03400066b0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 shutdown_connections 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f032c076cd0 0x7f032c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034c0cb9e0 0x7f034c1f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034c0cb560 0x7f034c1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.512+0000 7f0352800640 1 --2- 172.21.15.44:0/1370603079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0334004d40 0x7f034c0ad3d0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.513+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 >> 172.21.15.44:0/1370603079 conn(0x7f034c0cfec0 msgr2=0x7f034c0b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:05.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.513+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 shutdown_connections 2024-07-17T08:35:05.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.513+0000 7f0352800640 1 -- 172.21.15.44:0/1370603079 wait complete. 2024-07-17T08:35:05.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 -- 172.21.15.44:0/515981533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 msgr2=0x7f0fec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 --2- 172.21.15.44:0/515981533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f0fec004800 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0006f80 tx=0x7f0fec005830 comp rx=0 tx=0).stop 2024-07-17T08:35:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f1007400640 1 -- 172.21.15.44:0/515981533 reap_dead start 2024-07-17T08:35:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 -- 172.21.15.44:0/515981533 shutdown_connections 2024-07-17T08:35:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 -- 172.21.15.44:0/515981533 >> 172.21.15.44:0/515981533 conn(0x7f10080d00a0 msgr2=0x7f10080d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:05.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 -- 172.21.15.44:0/515981533 shutdown_connections 2024-07-17T08:35:05.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.720+0000 7f100dc00640 1 -- 172.21.15.44:0/515981533 wait complete. 2024-07-17T08:35:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.722+0000 7f100dc00640 1 Processor -- start 2024-07-17T08:35:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.722+0000 7f100dc00640 1 -- start start 2024-07-17T08:35:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1008156280 0x7f10081f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10080cd6f0 0x7f10081f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10080ce110 con 0x7f0fec004020 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10080cdf90 con 0x7f1008156280 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f100dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10080ce290 con 0x7f10080cd6f0 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f1006a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f1006a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55956/0 (socket says 172.21.15.44:55956) 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f1006a00640 1 -- 172.21.15.44:0/4199447741 learned_addr learned my addr 172.21.15.44:0/4199447741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.723+0000 7f1006000640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1008156280 0x7f10081f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1007400640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10080cd6f0 0x7f10081f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10080cd6f0 msgr2=0x7f10081f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10080cd6f0 0x7f10081f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1008156280 msgr2=0x7f10081f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1008156280 0x7f10081f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 -- 172.21.15.44:0/4199447741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10080ca7e0 con 0x7f0fec004020 2024-07-17T08:35:05.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f1006a00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8010a70 tx=0x7f0ff8010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800eda0 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ff8005020 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8005450 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.724+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10080caaa0 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.725+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10080cd1d0 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.726+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100806e480 con 0x7f0fec004020 2024-07-17T08:35:05.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.727+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f0ff80047a0 con 0x7f0fec004020 2024-07-17T08:35:05.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.728+0000 7f0ffea00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 0x7f0fe4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:05.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.728+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7107+0+0 (secure 0 0 0) 0x7f0ff8099890 con 0x7f0fec004020 2024-07-17T08:35:05.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.728+0000 7f1006000640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 msgr2=0x7f0fe4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:05.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.728+0000 7f1006000640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 0x7f0fe4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:05.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.732+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0ff80625b0 con 0x7f0fec004020 2024-07-17T08:35:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.930+0000 7f1006000640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 msgr2=0x7f0fe4079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.930+0000 7f1006000640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 0x7f0fe4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:05.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.942+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10081576f0 con 0x7f0fec004020 2024-07-17T08:35:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.944+0000 7f0ffea00640 1 -- 172.21.15.44:0/4199447741 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) ==== 74+0+18434 (secure 0 0 0) 0x7f0ff8061eb0 con 0x7f0fec004020 2024-07-17T08:35:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":210,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:05.384354+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:31:37.247087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-07-17T08:31:38.238209+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.e","mappings":[{"from":3,"to":6}]},{"pgid":"26.22","mappings":[{"from":1,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.3f","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.175:6834/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/340287962":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/3551582397":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/693817936":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/464369211":"2024-07-18T08:32:01.140259+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.946+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 msgr2=0x7f0fe4079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.946+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 0x7f0fe4079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 msgr2=0x7f1008147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8010a70 tx=0x7f0ff8010e70 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 shutdown_connections 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f0fe4076cd0 0x7f0fe4079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10080cd6f0 0x7f10081f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1008156280 0x7f10081f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 --2- 172.21.15.44:0/4199447741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fec004020 0x7f1008147a10 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 >> 172.21.15.44:0/4199447741 conn(0x7f10080d00a0 msgr2=0x7f1008157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 shutdown_connections 2024-07-17T08:35:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:05.947+0000 7f100dc00640 1 -- 172.21.15.44:0/4199447741 wait complete. 2024-07-17T08:35:05.967 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:35:05.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:06.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 -- 172.21.15.44:0/50539807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb8005310 msgr2=0x7f5eb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/50539807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb8005310 0x7f5eb800b9c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc001dc0 tx=0x7f5eb8005830 comp rx=0 tx=0).stop 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 -- 172.21.15.44:0/50539807 shutdown_connections 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/50539807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb8005310 0x7f5eb800b9c0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 -- 172.21.15.44:0/50539807 >> 172.21.15.44:0/50539807 conn(0x7f5ed406ce80 msgr2=0x7f5ed406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 -- 172.21.15.44:0/50539807 shutdown_connections 2024-07-17T08:35:06.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.161+0000 7f5ed3e00640 1 -- 172.21.15.44:0/50539807 wait complete. 2024-07-17T08:35:06.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.163+0000 7f5ed3e00640 1 Processor -- start 2024-07-17T08:35:06.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.163+0000 7f5ed3e00640 1 -- start start 2024-07-17T08:35:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 0x7f5ed406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed4145b00 0x7f5ed4145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed415bc10 con 0x7f5ed41f7560 2024-07-17T08:35:06.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed415ba90 con 0x7f5eb8005310 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed415bd90 con 0x7f5ed4145b00 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ecbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 0x7f5ed406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ed2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 0x7f5ed406f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.44:47332/0 (socket says 172.21.15.44:47332) 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ed2a00640 1 -- 172.21.15.44:0/754758681 learned_addr learned my addr 172.21.15.44:0/754758681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.164+0000 7f5ecbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55968/0 (socket says 172.21.15.44:55968) 2024-07-17T08:35:06.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ed3400640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed4145b00 0x7f5ed4145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed4145b00 msgr2=0x7f5ed4145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed4145b00 0x7f5ed4145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 msgr2=0x7f5ed406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 0x7f5ed406f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 -- 172.21.15.44:0/754758681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41498e0 con 0x7f5ed41f7560 2024-07-17T08:35:06.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecbe00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc011780 tx=0x7f5ebc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc004460 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ebc016070 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed4148b20 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.165+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00fb40 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.166+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41490e0 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.167+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4073230 con 0x7f5ed41f7560 2024-07-17T08:35:06.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.168+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f5ebc018030 con 0x7f5ed41f7560 2024-07-17T08:35:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.169+0000 7f5ecb400640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 0x7f5eb00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.169+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7107+0+0 (secure 0 0 0) 0x7f5ebc0990c0 con 0x7f5ed41f7560 2024-07-17T08:35:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.169+0000 7f5ed2a00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 msgr2=0x7f5eb00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:06.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.169+0000 7f5ed2a00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 0x7f5eb00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.174+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5ebc061e60 con 0x7f5ed41f7560 2024-07-17T08:35:06.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.370+0000 7f5ed2a00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 msgr2=0x7f5eb00791c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:06.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.370+0000 7f5ed2a00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 0x7f5eb00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:06.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.421+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ed415ac00 con 0x7f5ed41f7560 2024-07-17T08:35:06.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.423+0000 7f5ecb400640 1 -- 172.21.15.44:0/754758681 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7f5ebc06a0a0 con 0x7f5ed41f7560 2024-07-17T08:35:06.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:06.424 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:35:06.424 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 msgr2=0x7f5eb00791c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 0x7f5eb00791c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 msgr2=0x7f5ed41f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc011780 tx=0x7f5ebc011c40 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 shutdown_connections 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f5eb0076d00 0x7f5eb00791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.425+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed4145b00 0x7f5ed4145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.426+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ed41f7560 0x7f5ed41f79e0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.426+0000 7f5ed3e00640 1 --2- 172.21.15.44:0/754758681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb8005310 0x7f5ed406f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.426+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 >> 172.21.15.44:0/754758681 conn(0x7f5ed406ce80 msgr2=0x7f5ed40b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.426+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 shutdown_connections 2024-07-17T08:35:06.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.426+0000 7f5ed3e00640 1 -- 172.21.15.44:0/754758681 wait complete. 2024-07-17T08:35:06.445 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:06.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1320611353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d01522c0 msgr2=0x7fc4d00ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1320611353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d01522c0 0x7fc4d00ad790 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fc4d015edd0 tx=0x7fc4c402a530 comp rx=0 tx=0).stop 2024-07-17T08:35:06.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1320611353 shutdown_connections 2024-07-17T08:35:06.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1320611353 >> v2:172.21.15.175:3300/0 conn(0x7fc4d00ac930 0x7fc4d00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1320611353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d01522c0 0x7fc4d00ad790 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1320611353 >> 172.21.15.44:0/1320611353 conn(0x7fc4d00b0fb0 msgr2=0x7fc4d00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1320611353 shutdown_connections 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.630+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1320611353 wait complete. 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.632+0000 7fc4d7c00640 1 Processor -- start 2024-07-17T08:35:06.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.632+0000 7fc4d7c00640 1 -- start start 2024-07-17T08:35:06.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4d00adcd0 0x7fc4d014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4d01522c0 0x7fc4d014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00cdf60 con 0x7fc4d00ac930 2024-07-17T08:35:06.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00cdde0 con 0x7fc4d00adcd0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d00ce0e0 con 0x7fc4d01522c0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55978/0 (socket says 172.21.15.44:55978) 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.633+0000 7fc4d5200640 1 -- 172.21.15.44:0/1362911882 learned_addr learned my addr 172.21.15.44:0/1362911882 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4cfe00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4d00adcd0 0x7fc4d014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4d01522c0 0x7fc4d014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4d01522c0 msgr2=0x7fc4d014bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4d01522c0 0x7fc4d014bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4d00adcd0 msgr2=0x7fc4d014b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4d00adcd0 0x7fc4d014b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 -- 172.21.15.44:0/1362911882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d014d580 con 0x7fc4d00ac930 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d5200640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009920 tx=0x7fc4c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4031d60 con 0x7fc4d00ac930 2024-07-17T08:35:06.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4c4033e10 con 0x7fc4d00ac930 2024-07-17T08:35:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4005260 con 0x7fc4d00ac930 2024-07-17T08:35:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d014d860 con 0x7fc4d00ac930 2024-07-17T08:35:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.634+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d014db40 con 0x7fc4d00ac930 2024-07-17T08:35:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.636+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d00cd690 con 0x7fc4d00ac930 2024-07-17T08:35:06.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.637+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7fc4c4005400 con 0x7fc4d00ac930 2024-07-17T08:35:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.637+0000 7fc4ccc00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 0x7fc4b00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.638+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7107+0+0 (secure 0 0 0) 0x7fc4c40b9b60 con 0x7fc4d00ac930 2024-07-17T08:35:06.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.638+0000 7fc4cfe00640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 msgr2=0x7fc4b00791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:06.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.638+0000 7fc4cfe00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 0x7fc4b00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:06.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.642+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc4c403f030 con 0x7fc4d00ac930 2024-07-17T08:35:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.839+0000 7fc4cfe00640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 msgr2=0x7fc4b00791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:06.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.840+0000 7fc4cfe00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 0x7fc4b00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.886+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d0072d20 con 0x7fc4d00ac930 2024-07-17T08:35:06.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.888+0000 7fc4ccc00640 1 -- 172.21.15.44:0/1362911882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5233 (secure 0 0 0) 0x7fc4c4082900 con 0x7fc4d00ac930 2024-07-17T08:35:06.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:06.889 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":105,"btime":"2024-07-17T08:31:42:828974+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:31:39.271781+0000","modified":"2024-07-17T08:31:42.343316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9612},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9612":{"gid":9612,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/271909788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":271909788},{"type":"v1","addr":"172.21.15.44:6835","nonce":271909788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9612,"qdb_cluster":[9612]},"id":12}]} 2024-07-17T08:35:06.889 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 105 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.890+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 msgr2=0x7fc4b00791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.890+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 0x7fc4b00791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.890+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 msgr2=0x7fc4d0148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009920 tx=0x7fc4c4033c20 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 shutdown_connections 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fc4b0076d20 0x7fc4b00791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4d01522c0 0x7fc4d014bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4d00adcd0 0x7fc4d014b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 --2- 172.21.15.44:0/1362911882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d00ac930 0x7fc4d0148890 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:06.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 >> 172.21.15.44:0/1362911882 conn(0x7fc4d00b0fb0 msgr2=0x7fc4d015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:06.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 shutdown_connections 2024-07-17T08:35:06.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:06.891+0000 7fc4d7c00640 1 -- 172.21.15.44:0/1362911882 wait complete. 2024-07-17T08:35:06.910 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:07.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 -- 172.21.15.44:0/2838082895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ac930 msgr2=0x7f83440b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 --2- 172.21.15.44:0/2838082895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ac930 0x7f83440b25d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f833c009830 tx=0x7f833c02a510 comp rx=0 tx=0).stop 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 -- 172.21.15.44:0/2838082895 shutdown_connections 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 --2- 172.21.15.44:0/2838082895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ac930 0x7f83440b25d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 --2- 172.21.15.44:0/2838082895 >> v2:172.21.15.175:3300/0 conn(0x7f83440ae810 0x7f834415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 -- 172.21.15.44:0/2838082895 >> 172.21.15.44:0/2838082895 conn(0x7f83440b0fb0 msgr2=0x7f83440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 -- 172.21.15.44:0/2838082895 shutdown_connections 2024-07-17T08:35:07.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.104+0000 7f8349e00640 1 -- 172.21.15.44:0/2838082895 wait complete. 2024-07-17T08:35:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.106+0000 7f8349e00640 1 Processor -- start 2024-07-17T08:35:07.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.107+0000 7f8349e00640 1 -- start start 2024-07-17T08:35:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834406ea50 0x7f8344143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ae810 0x7f83440737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f834406f040 con 0x7f83440ac930 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f834406eec0 con 0x7f834406ea50 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8349e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83440b2b10 con 0x7f83440ae810 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8342000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8342000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55992/0 (socket says 172.21.15.44:55992) 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8342000640 1 -- 172.21.15.44:0/1581910403 learned_addr learned my addr 172.21.15.44:0/1581910403 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8343400640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ae810 0x7f83440737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.108+0000 7f8342a00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834406ea50 0x7f8344143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ae810 msgr2=0x7f83440737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ae810 0x7f83440737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834406ea50 msgr2=0x7f8344143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834406ea50 0x7f8344143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 -- 172.21.15.44:0/1581910403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834413aa00 con 0x7f83440ac930 2024-07-17T08:35:07.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8342000640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c450 tx=0x7f832c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c0052d0 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f832c005470 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.109+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83441f6500 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.110+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c012870 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.110+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83441f6ac0 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.111+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834406a1a0 con 0x7f83440ac930 2024-07-17T08:35:07.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.112+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7f832c012a10 con 0x7f83440ac930 2024-07-17T08:35:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.113+0000 7f833b400640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 0x7f8320079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.113+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7107+0+0 (secure 0 0 0) 0x7f832c0985c0 con 0x7f83440ac930 2024-07-17T08:35:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.113+0000 7f8342a00640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 msgr2=0x7f8320079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:07.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.113+0000 7f8342a00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 0x7f8320079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:07.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.117+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f834415ea50 con 0x7f83440ac930 2024-07-17T08:35:07.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.315+0000 7f8342a00640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 msgr2=0x7f8320079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:07.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.315+0000 7f8342a00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 0x7f8320079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.324+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83440b3030 con 0x7f83440ac930 2024-07-17T08:35:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.327+0000 7f833b400640 1 -- 172.21.15.44:0/1581910403 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) ==== 74+0+18434 (secure 0 0 0) 0x7f832c0613f0 con 0x7f83440ac930 2024-07-17T08:35:07.328 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:07.328 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":210,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:05.384354+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:31:37.247087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-07-17T08:31:38.238209+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.e","mappings":[{"from":3,"to":6}]},{"pgid":"26.22","mappings":[{"from":1,"to":0}]},{"pgid":"26.31","mappings":[{"from":1,"to":0}]},{"pgid":"26.3f","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.175:6834/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/340287962":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/3551582397":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/693817936":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/464369211":"2024-07-18T08:32:01.140259+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.329+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 msgr2=0x7f8320079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.329+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 0x7f8320079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.329+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 msgr2=0x7f834413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.329+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c450 tx=0x7f832c00c910 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.329+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 shutdown_connections 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7f8320076cd0 0x7f8320079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83440ae810 0x7f83440737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83440ac930 0x7f834413a4c0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 --2- 172.21.15.44:0/1581910403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f834406ea50 0x7f8344143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 >> 172.21.15.44:0/1581910403 conn(0x7f83440b0fb0 msgr2=0x7f83440b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 shutdown_connections 2024-07-17T08:35:07.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.330+0000 7f8349e00640 1 -- 172.21.15.44:0/1581910403 wait complete. 2024-07-17T08:35:07.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:35:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 -- 172.21.15.44:0/296256442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c1503c0 msgr2=0x7fe89c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 --2- 172.21.15.44:0/296256442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c1503c0 0x7fe89c15a450 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0cffb0 tx=0x7fe88c02a560 comp rx=0 tx=0).stop 2024-07-17T08:35:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 -- 172.21.15.44:0/296256442 shutdown_connections 2024-07-17T08:35:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 --2- 172.21.15.44:0/296256442 >> v2:172.21.15.175:3300/0 conn(0x7fe89c0ae340 0x7fe89c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 --2- 172.21.15.44:0/296256442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c1503c0 0x7fe89c15a450 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 -- 172.21.15.44:0/296256442 >> 172.21.15.44:0/296256442 conn(0x7fe89c0cf0d0 msgr2=0x7fe89c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 -- 172.21.15.44:0/296256442 shutdown_connections 2024-07-17T08:35:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.537+0000 7fe8a1800640 1 -- 172.21.15.44:0/296256442 wait complete. 2024-07-17T08:35:07.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.540+0000 7fe8a1800640 1 Processor -- start 2024-07-17T08:35:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.540+0000 7fe8a1800640 1 -- start start 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 0x7fe89c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe89c1503c0 0x7fe89c0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c06f4d0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c06dd40 con 0x7fe89c0ae340 2024-07-17T08:35:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c06f650 con 0x7fe89c1503c0 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56010/0 (socket says 172.21.15.44:56010) 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 -- 172.21.15.44:0/1527198496 learned_addr learned my addr 172.21.15.44:0/1527198496 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89be00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 0x7fe89c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 -- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe89c1503c0 msgr2=0x7fe89c0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe8a0e00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe89c1503c0 0x7fe89c0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe89c1503c0 0x7fe89c0b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 -- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 msgr2=0x7fe89c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 0x7fe89c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89b400640 1 -- 172.21.15.44:0/1527198496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0ce6b0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.542+0000 7fe89be00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 0x7fe89c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe89b400640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe88800dbf0 tx=0x7fe88800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888005220 con 0x7fe89c0b33f0 2024-07-17T08:35:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe88800ae50 con 0x7fe89c0b33f0 2024-07-17T08:35:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c1f4f80 con 0x7fe89c0b33f0 2024-07-17T08:35:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880047a0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.543+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c1f5540 con 0x7fe89c0b33f0 2024-07-17T08:35:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.545+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c0cffe0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.545+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 127) ==== 98943+0+0 (secure 0 0 0) 0x7fe888004940 con 0x7fe89c0b33f0 2024-07-17T08:35:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.547+0000 7fe893e00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 0x7fe87c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.547+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7107+0+0 (secure 0 0 0) 0x7fe8880991c0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.547+0000 7fe89be00640 1 -- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 msgr2=0x7fe87c079190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3642155749 2024-07-17T08:35:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.547+0000 7fe89be00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 0x7fe87c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.550+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe888061f60 con 0x7fe89c0b33f0 2024-07-17T08:35:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.735+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 128) ==== 98540+0+0 (secure 0 0 0) 0x7fe88805daf0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.735+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 msgr2=0x7fe87c079190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:07.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.735+0000 7fe893e00640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 0x7fe87c079190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.812+0000 7f1897e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.812+0000 7f1897e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.812+0000 7f1897e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.812+0000 7f1897e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.812+0000 7f1897e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.813 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.813+0000 7f1894200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.814 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.813+0000 7f1894200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.814 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.813+0000 7f1894200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.814 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.813+0000 7f1894200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.814 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:07.813+0000 7f1894200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:07.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.835+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe89c0b2e90 con 0x7fe89c0b33f0 2024-07-17T08:35:07.838 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:35:07.837+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:35:07.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.840+0000 7fe893e00640 1 -- 172.21.15.44:0/1527198496 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7fe888013ce0 con 0x7fe89c0b33f0 2024-07-17T08:35:07.841 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 msgr2=0x7fe89c0afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe88800dbf0 tx=0x7fe88800a620 comp rx=0 tx=0).stop 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 shutdown_connections 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:6834/3642155749,v1:172.21.15.175:6837/3642155749] conn(0x7fe87c076cd0 0x7fe87c079190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe89c1503c0 0x7fe89c0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe89c0b33f0 0x7fe89c0afd90 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 --2- 172.21.15.44:0/1527198496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe89c0ae340 0x7fe89c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:07.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 >> 172.21.15.44:0/1527198496 conn(0x7fe89c0cf0d0 msgr2=0x7fe89c1583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:07.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 shutdown_connections 2024-07-17T08:35:07.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:07.845+0000 7fe8a1800640 1 -- 172.21.15.44:0/1527198496 wait complete. 2024-07-17T08:35:07.863 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:35:07.954 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.049+0000 7fba17800640 1 -- 172.21.15.44:0/2385788341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba100ce1d0 msgr2=0x7fba100ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.049+0000 7fba17800640 1 --2- 172.21.15.44:0/2385788341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba100ce1d0 0x7fba100ce5b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fba10074630 tx=0x7fba0002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 -- 172.21.15.44:0/2385788341 shutdown_connections 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 --2- 172.21.15.44:0/2385788341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba100b6790 0x7fba10161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 --2- 172.21.15.44:0/2385788341 >> v2:172.21.15.175:3300/0 conn(0x7fba100b29e0 0x7fba10161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 --2- 172.21.15.44:0/2385788341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba100ce1d0 0x7fba100ce5b0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 -- 172.21.15.44:0/2385788341 >> 172.21.15.44:0/2385788341 conn(0x7fba100d3700 msgr2=0x7fba100d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 -- 172.21.15.44:0/2385788341 shutdown_connections 2024-07-17T08:35:08.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.050+0000 7fba17800640 1 -- 172.21.15.44:0/2385788341 wait complete. 2024-07-17T08:35:08.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.051+0000 7fba17800640 1 Processor -- start 2024-07-17T08:35:08.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.051+0000 7fba17800640 1 -- start start 2024-07-17T08:35:08.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba100b29e0 0x7fba10151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba100b6790 0x7fba101f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba10162250 con 0x7fba1014f710 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba101620d0 con 0x7fba100b6790 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.052+0000 7fba17800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba101623d0 con 0x7fba100b29e0 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56066/0 (socket says 172.21.15.44:56066) 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 -- 172.21.15.44:0/1059190000 learned_addr learned my addr 172.21.15.44:0/1059190000 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 -- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba100b29e0 msgr2=0x7fba10151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba100b29e0 0x7fba10151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba0fe00640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba100b6790 0x7fba101f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:08.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 -- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba100b6790 msgr2=0x7fba101f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba100b6790 0x7fba101f7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 -- 172.21.15.44:0/1059190000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba10156280 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba15800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e980 tx=0x7fba0800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.053+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08004d20 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.054+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba0800cb20 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.054+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08005020 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.054+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba10155a40 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.054+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba10155cc0 con 0x7fba1014f710 2024-07-17T08:35:08.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.055+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba100ca590 con 0x7fba1014f710 2024-07-17T08:35:08.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.056+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 128) ==== 98540+0+0 (secure 0 0 0) 0x7fba080051c0 con 0x7fba1014f710 2024-07-17T08:35:08.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.057+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(212..212 src has 1..212) ==== 7509+0+0 (secure 0 0 0) 0x7fba08098e10 con 0x7fba1014f710 2024-07-17T08:35:08.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.061+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fba100ca590 con 0x7fba1014f710 2024-07-17T08:35:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.312+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fba100ce5b0 con 0x7fba1014f710 2024-07-17T08:35:08.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.743+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 129) ==== 98667+0+0 (secure 0 0 0) 0x7fba0805d1a0 con 0x7fba1014f710 2024-07-17T08:35:08.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.744+0000 7fba0cc00640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 0x7fb9f407f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:08.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.746+0000 7fba14e00640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 0x7fb9f407f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:08.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.763+0000 7fba14e00640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 0x7fb9f407f730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba00033000 tx=0x7fba000046c0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.844+0000 7fba0cc00640 1 -- 172.21.15.44:0/1059190000 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) ==== 104+0+0 (secure 0 0 0) 0x7fba10155a40 con 0x7fba1014f710 2024-07-17T08:35:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 msgr2=0x7fb9f407f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 0x7fb9f407f730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba00033000 tx=0x7fba000046c0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 msgr2=0x7fba101f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e980 tx=0x7fba0800ee40 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 shutdown_connections 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba1014f710 0x7fba101f7c60 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.848+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb9f407d340 0x7fb9f407f730 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.849+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba100b6790 0x7fba101f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.849+0000 7fba17800640 1 --2- 172.21.15.44:0/1059190000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba100b29e0 0x7fba10151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.849+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 >> 172.21.15.44:0/1059190000 conn(0x7fba100d3700 msgr2=0x7fba100b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.849+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 shutdown_connections 2024-07-17T08:35:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:08.849+0000 7fba17800640 1 -- 172.21.15.44:0/1059190000 wait complete. 2024-07-17T08:35:08.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:35:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 -- 172.21.15.44:0/3488946692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 msgr2=0x7fe6fc0b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 --2- 172.21.15.44:0/3488946692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc0b8670 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0766b0 tx=0x7fe6f402a8f0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 -- 172.21.15.44:0/3488946692 shutdown_connections 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 --2- 172.21.15.44:0/3488946692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc161b50 0x7fe6fc0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 --2- 172.21.15.44:0/3488946692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc0b8670 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 --2- 172.21.15.44:0/3488946692 >> v2:172.21.15.175:3300/0 conn(0x7fe6fc15e370 0x7fe6fc0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 -- 172.21.15.44:0/3488946692 >> 172.21.15.44:0/3488946692 conn(0x7fe6fc073a60 msgr2=0x7fe6fc073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 -- 172.21.15.44:0/3488946692 shutdown_connections 2024-07-17T08:35:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.070+0000 7fe702a00640 1 -- 172.21.15.44:0/3488946692 wait complete. 2024-07-17T08:35:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.072+0000 7fe702a00640 1 Processor -- start 2024-07-17T08:35:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.072+0000 7fe702a00640 1 -- start start 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 0x7fe6fc1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc161b50 0x7fe6fc0b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc15e8d0 con 0x7fe6fc0d0970 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc15e750 con 0x7fe6fc161b50 2024-07-17T08:35:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.073+0000 7fe702a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc0b8bb0 con 0x7fe6fc15e370 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52640/0 (socket says 172.21.15.44:52640) 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 -- 172.21.15.44:0/2138626408 learned_addr learned my addr 172.21.15.44:0/2138626408 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe700c00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 0x7fe6fc1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe702000640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc161b50 0x7fe6fc0b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 -- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 msgr2=0x7fe6fc1f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 0x7fe6fc1f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 -- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc161b50 msgr2=0x7fe6fc0b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc161b50 0x7fe6fc0b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe701600640 1 -- 172.21.15.44:0/2138626408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1f8230 con 0x7fe6fc0d0970 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.074+0000 7fe700c00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 0x7fe6fc1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe701600640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c940 tx=0x7fe6f000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0007bb0 con 0x7fe6fc0d0970 2024-07-17T08:35:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f0007e40 con 0x7fe6fc0d0970 2024-07-17T08:35:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0005260 con 0x7fe6fc0d0970 2024-07-17T08:35:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc0cb960 con 0x7fe6fc0d0970 2024-07-17T08:35:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.075+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc0cbf20 con 0x7fe6fc0d0970 2024-07-17T08:35:09.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.076+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc15d7c0 con 0x7fe6fc0d0970 2024-07-17T08:35:09.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.078+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 129) ==== 98667+0+0 (secure 0 0 0) 0x7fe6f0005400 con 0x7fe6fc0d0970 2024-07-17T08:35:09.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.079+0000 7fe6f9600640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 0x7fe6dc078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:09.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.079+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(212..212 src has 1..212) ==== 7509+0+0 (secure 0 0 0) 0x7fe6f0098fb0 con 0x7fe6fc0d0970 2024-07-17T08:35:09.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.079+0000 7fe700c00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 0x7fe6dc078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.080+0000 7fe700c00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 0x7fe6dc078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc154a40 tx=0x7fe6e8005cf0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:09.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.082+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe6fc15d7c0 con 0x7fe6fc0d0970 2024-07-17T08:35:09.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.292+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6fc069460 con 0x7fe6fc0d0970 2024-07-17T08:35:09.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:35:09.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:35:09.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.848+0000 7fe6f9600640 1 -- 172.21.15.44:0/2138626408 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v213) ==== 181+0+0 (secure 0 0 0) 0x7fe6fc1f8230 con 0x7fe6fc0d0970 2024-07-17T08:35:09.849 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:35:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.852+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 msgr2=0x7fe6dc078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.852+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 0x7fe6dc078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc154a40 tx=0x7fe6e8005cf0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.852+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 msgr2=0x7fe6fc1f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.852+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c940 tx=0x7fe6f000ce00 comp rx=0 tx=0).stop 2024-07-17T08:35:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 shutdown_connections 2024-07-17T08:35:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe6dc076a00 0x7fe6dc078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6fc161b50 0x7fe6fc0b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6fc15e370 0x7fe6fc1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 --2- 172.21.15.44:0/2138626408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6fc0d0970 0x7fe6fc1f7cf0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 >> 172.21.15.44:0/2138626408 conn(0x7fe6fc073a60 msgr2=0x7fe6fc0d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 shutdown_connections 2024-07-17T08:35:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:09.854+0000 7fe702a00640 1 -- 172.21.15.44:0/2138626408 wait complete. 2024-07-17T08:35:09.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:35:10.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 -- 172.21.15.44:0/1984863667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f507406e690 msgr2=0x7f507415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:10.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 --2- 172.21.15.44:0/1984863667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f507406e690 0x7f507415dad0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5060005fd0 tx=0x7f506002a8b0 comp rx=0 tx=0).stop 2024-07-17T08:35:10.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 -- 172.21.15.44:0/1984863667 shutdown_connections 2024-07-17T08:35:10.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 --2- 172.21.15.44:0/1984863667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f507406e690 0x7f507415dad0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 -- 172.21.15.44:0/1984863667 >> 172.21.15.44:0/1984863667 conn(0x7f50740b0eb0 msgr2=0x7f50740b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 -- 172.21.15.44:0/1984863667 shutdown_connections 2024-07-17T08:35:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.095+0000 7f507c600640 1 -- 172.21.15.44:0/1984863667 wait complete. 2024-07-17T08:35:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.097+0000 7f507c600640 1 Processor -- start 2024-07-17T08:35:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.097+0000 7f507c600640 1 -- start start 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 0x7f50741f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5074154560 0x7f50740b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b28e0 con 0x7f50740ae8b0 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b2760 con 0x7f5074154560 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f507c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50740b2a60 con 0x7f507406e690 2024-07-17T08:35:10.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.098+0000 7f5079200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:10.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52646/0 (socket says 172.21.15.44:52646) 2024-07-17T08:35:10.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 -- 172.21.15.44:0/2850812218 learned_addr learned my addr 172.21.15.44:0/2850812218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079c00640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 0x7f50741f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f507a600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5074154560 0x7f50740b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 -- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 msgr2=0x7f50741f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 0x7f50741f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 -- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5074154560 msgr2=0x7f50740b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5074154560 0x7f50740b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 -- 172.21.15.44:0/2850812218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50741f4c50 con 0x7f50740ae8b0 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079c00640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 0x7f50741f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.099+0000 7f5079200640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5060009cd0 tx=0x7f50600039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.100+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060040070 con 0x7f50740ae8b0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.100+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5060004130 con 0x7f50740ae8b0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.100+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50741f43b0 con 0x7f50740ae8b0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.100+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060004540 con 0x7f50740ae8b0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.100+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741f4970 con 0x7f50740ae8b0 2024-07-17T08:35:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.101+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50740ae1a0 con 0x7f50740ae8b0 2024-07-17T08:35:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.102+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 130) ==== 98875+0+0 (secure 0 0 0) 0x7f5060039040 con 0x7f50740ae8b0 2024-07-17T08:35:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.103+0000 7f5071600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 0x7f5054079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.104+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(213..213 src has 1..213) ==== 7071+0+0 (secure 0 0 0) 0x7f50600ba130 con 0x7f50740ae8b0 2024-07-17T08:35:10.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.104+0000 7f5079c00640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 0x7f5054079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.105+0000 7f5079c00640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 0x7f5054079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50640046b0 tx=0x7f5064009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:10.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.108+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5060082f40 con 0x7f50740ae8b0 2024-07-17T08:35:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:10.329+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50741f81a0 con 0x7f50740ae8b0 2024-07-17T08:35:11.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.190+0000 7f5071600640 1 -- 172.21.15.44:0/2850812218 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v215) ==== 169+0+0 (secure 0 0 0) 0x7f5060082840 con 0x7f50740ae8b0 2024-07-17T08:35:11.191 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:35:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 msgr2=0x7f5054079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 0x7f5054079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50640046b0 tx=0x7f5064009040 comp rx=0 tx=0).stop 2024-07-17T08:35:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 msgr2=0x7f5074141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5060009cd0 tx=0x7f50600039f0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 shutdown_connections 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5054076c40 0x7f5054079100 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50640046b0 tx=0x7f5064009040 comp rx=0 tx=0).stop 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5074154560 0x7f50740b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50740ae8b0 0x7f5074141460 secure :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5060009cd0 tx=0x7f50600039f0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 --2- 172.21.15.44:0/2850812218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f507406e690 0x7f50741f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.196+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 >> 172.21.15.44:0/2850812218 conn(0x7f50740b0eb0 msgr2=0x7f50740b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.198+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 shutdown_connections 2024-07-17T08:35:11.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.199+0000 7f507c600640 1 -- 172.21.15.44:0/2850812218 wait complete. 2024-07-17T08:35:11.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:35:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 -- 172.21.15.44:0/4093922338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef40d2fa0 msgr2=0x7feef40d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 --2- 172.21.15.44:0/4093922338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef40d2fa0 0x7feef40d1e30 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7feeec005fd0 tx=0x7feeec02a970 comp rx=0 tx=0).stop 2024-07-17T08:35:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 -- 172.21.15.44:0/4093922338 shutdown_connections 2024-07-17T08:35:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 --2- 172.21.15.44:0/4093922338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef40d2fa0 0x7feef40d1e30 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 --2- 172.21.15.44:0/4093922338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 0x7feef40d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 -- 172.21.15.44:0/4093922338 >> 172.21.15.44:0/4093922338 conn(0x7feef40cfec0 msgr2=0x7feef40d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 -- 172.21.15.44:0/4093922338 shutdown_connections 2024-07-17T08:35:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.421+0000 7feef9c00640 1 -- 172.21.15.44:0/4093922338 wait complete. 2024-07-17T08:35:11.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.423+0000 7feef9c00640 1 Processor -- start 2024-07-17T08:35:11.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.423+0000 7feef9c00640 1 -- start start 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 0x7feef40a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef40d2fa0 0x7feef415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feef40d1220 con 0x7feef415d300 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feef40d2370 con 0x7feef40d2fa0 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feef40c9ba0 con 0x7feef40b55d0 2024-07-17T08:35:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.424+0000 7feef9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52674/0 (socket says 172.21.15.44:52674) 2024-07-17T08:35:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 -- 172.21.15.44:0/383690606 learned_addr learned my addr 172.21.15.44:0/383690606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef3e00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 0x7feef40a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef3400640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef40d2fa0 0x7feef415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 -- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 msgr2=0x7feef40a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 0x7feef40a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 -- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef40d2fa0 msgr2=0x7feef415ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef40d2fa0 0x7feef415ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 -- 172.21.15.44:0/383690606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef40c9e80 con 0x7feef415d300 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.425+0000 7feef9200640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7feeec006010 tx=0x7feeec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.426+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec047070 con 0x7feef415d300 2024-07-17T08:35:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.426+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeec004540 con 0x7feef415d300 2024-07-17T08:35:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.426+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec034070 con 0x7feef415d300 2024-07-17T08:35:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.426+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef415f700 con 0x7feef415d300 2024-07-17T08:35:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.426+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef4160440 con 0x7feef415d300 2024-07-17T08:35:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.427+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef40d2850 con 0x7feef415d300 2024-07-17T08:35:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.428+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 130) ==== 98875+0+0 (secure 0 0 0) 0x7feeec04a070 con 0x7feef415d300 2024-07-17T08:35:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.429+0000 7feeebe00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 0x7feed0079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.429+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7feeec0b9860 con 0x7feef415d300 2024-07-17T08:35:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.430+0000 7feef3e00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 0x7feed0079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.431+0000 7feef3e00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 0x7feed0079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feee4006fd0 tx=0x7feee4008040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.434+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feeec03f070 con 0x7feef415d300 2024-07-17T08:35:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.645+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7feef40b93c0 con 0x7feef415d300 2024-07-17T08:35:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.653+0000 7feeebe00640 1 -- 172.21.15.44:0/383690606 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v71) ==== 71+0+0 (secure 0 0 0) 0x7feeec082890 con 0x7feef415d300 2024-07-17T08:35:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 msgr2=0x7feed0079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 0x7feed0079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feee4006fd0 tx=0x7feee4008040 comp rx=0 tx=0).stop 2024-07-17T08:35:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 msgr2=0x7feef415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7feeec006010 tx=0x7feeec003f10 comp rx=0 tx=0).stop 2024-07-17T08:35:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 shutdown_connections 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feef415d300 0x7feef415eba0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feed0076c40 0x7feed0079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feef40d2fa0 0x7feef415ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 --2- 172.21.15.44:0/383690606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feef40b55d0 0x7feef40a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.656+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 >> 172.21.15.44:0/383690606 conn(0x7feef40cfec0 msgr2=0x7feef40b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.657+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 shutdown_connections 2024-07-17T08:35:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.657+0000 7feef9c00640 1 -- 172.21.15.44:0/383690606 wait complete. 2024-07-17T08:35:11.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 -- 172.21.15.44:0/694223812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6434004800 msgr2=0x7f6434004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 --2- 172.21.15.44:0/694223812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6434004800 0x7f6434004c80 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f643800bb20 tx=0x7f6434005830 comp rx=0 tx=0).stop 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 -- 172.21.15.44:0/694223812 shutdown_connections 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 --2- 172.21.15.44:0/694223812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6434004800 0x7f6434004c80 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 -- 172.21.15.44:0/694223812 >> 172.21.15.44:0/694223812 conn(0x7f64500b3120 msgr2=0x7f64500b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 -- 172.21.15.44:0/694223812 shutdown_connections 2024-07-17T08:35:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.891+0000 7f6457800640 1 -- 172.21.15.44:0/694223812 wait complete. 2024-07-17T08:35:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.893+0000 7f6457800640 1 Processor -- start 2024-07-17T08:35:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.893+0000 7f6457800640 1 -- start start 2024-07-17T08:35:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434004800 0x7f6450141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 0x7f64501f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64500cd660 con 0x7f64501f2d10 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64500cf960 con 0x7f6434004800 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6457800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64500cd7e0 con 0x7f64500b5f50 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6447e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6447e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52702/0 (socket says 172.21.15.44:52702) 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.894+0000 7f6447e00640 1 -- 172.21.15.44:0/4026334732 learned_addr learned my addr 172.21.15.44:0/4026334732 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6454e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434004800 0x7f6450141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6455800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 0x7f64501f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 -- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 msgr2=0x7f64501f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 0x7f64501f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 -- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434004800 msgr2=0x7f6450141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434004800 0x7f6450141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 -- 172.21.15.44:0/4026334732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6450141ac0 con 0x7f64501f2d10 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6455800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 0x7f64501f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.895+0000 7f6447e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f643800ba40 tx=0x7f643800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.896+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438040070 con 0x7f64501f2d10 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.896+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6438004250 con 0x7f64501f2d10 2024-07-17T08:35:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.896+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438035040 con 0x7f64501f2d10 2024-07-17T08:35:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.896+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64501f6320 con 0x7f64501f2d10 2024-07-17T08:35:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.896+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501f2510 con 0x7f64501f2d10 2024-07-17T08:35:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.897+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64500b6330 con 0x7f64501f2d10 2024-07-17T08:35:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.898+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 130) ==== 98875+0+0 (secure 0 0 0) 0x7f6438038020 con 0x7f64501f2d10 2024-07-17T08:35:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.899+0000 7f644d600640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 0x7f642c079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.900+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7f64380b92a0 con 0x7f64501f2d10 2024-07-17T08:35:11.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.900+0000 7f6454e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 0x7f642c079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:11.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.901+0000 7f6454e00640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 0x7f642c079130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6440001eb0 tx=0x7f6440001ca0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:11.904+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f643802c040 con 0x7f64501f2d10 2024-07-17T08:35:12.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:12.203+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) -- 0x7f64501f1ff0 con 0x7f64501f2d10 2024-07-17T08:35:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:12.681+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f643802e030 con 0x7f64501f2d10 2024-07-17T08:35:13.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.196+0000 7f644d600640 1 -- 172.21.15.44:0/4026334732 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3261) ==== 163+0+0 (secure 0 0 0) 0x7f64380822c0 con 0x7f64501f2d10 2024-07-17T08:35:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.199+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 msgr2=0x7f642c079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.199+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 0x7f642c079130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6440001eb0 tx=0x7f6440001ca0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 msgr2=0x7f64500b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f643800ba40 tx=0x7f643800b6f0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 shutdown_connections 2024-07-17T08:35:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64500b5f50 0x7f64501f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64501f2d10 0x7f64500b5a10 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f642c076c70 0x7f642c079130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 --2- 172.21.15.44:0/4026334732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6434004800 0x7f6450141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 >> 172.21.15.44:0/4026334732 conn(0x7f64500b3120 msgr2=0x7f64500cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:13.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 shutdown_connections 2024-07-17T08:35:13.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.200+0000 7f6457800640 1 -- 172.21.15.44:0/4026334732 wait complete. 2024-07-17T08:35:13.223 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2024-07-17T08:35:13.223 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2024-07-17T08:35:13.223 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2024-07-17T08:35:13.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 -- 172.21.15.44:0/3236413238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 msgr2=0x7fb6e00cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/3236413238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e00cf7e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fb6e006f890 tx=0x7fb6cc02a560 comp rx=0 tx=0).stop 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 -- 172.21.15.44:0/3236413238 shutdown_connections 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/3236413238 >> v2:172.21.15.175:3300/0 conn(0x7fb6e0159800 0x7fb6e0159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/3236413238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e00cf7e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/3236413238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e006f4b0 0x7fb6e006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 -- 172.21.15.44:0/3236413238 >> 172.21.15.44:0/3236413238 conn(0x7fb6e00b0e80 msgr2=0x7fb6e00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 -- 172.21.15.44:0/3236413238 shutdown_connections 2024-07-17T08:35:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.436+0000 7fb6e4e00640 1 -- 172.21.15.44:0/3236413238 wait complete. 2024-07-17T08:35:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.438+0000 7fb6e4e00640 1 Processor -- start 2024-07-17T08:35:13.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.438+0000 7fb6e4e00640 1 -- start start 2024-07-17T08:35:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 0x7fb6e013da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e0159800 0x7fb6e01404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e009b9f0 con 0x7fb6e00cf380 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e009b870 con 0x7fb6e006f4b0 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6e4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e00b1fa0 con 0x7fb6e0159800 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52726/0 (socket says 172.21.15.44:52726) 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 -- 172.21.15.44:0/2304236465 learned_addr learned my addr 172.21.15.44:0/2304236465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dfe00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e0159800 0x7fb6e01404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6df400640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 0x7fb6e013da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 -- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e0159800 msgr2=0x7fb6e01404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e0159800 0x7fb6e01404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.439+0000 7fb6dea00640 1 -- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 msgr2=0x7fb6e013da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6dea00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 0x7fb6e013da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6dea00640 1 -- 172.21.15.44:0/2304236465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e0150a30 con 0x7fb6e00cf380 2024-07-17T08:35:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6df400640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 0x7fb6e013da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6dea00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009850 tx=0x7fb6cc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc03f070 con 0x7fb6e00cf380 2024-07-17T08:35:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6cc004890 con 0x7fb6e00cf380 2024-07-17T08:35:13.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc035030 con 0x7fb6e00cf380 2024-07-17T08:35:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e013d730 con 0x7fb6e00cf380 2024-07-17T08:35:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.440+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e01409e0 con 0x7fb6e00cf380 2024-07-17T08:35:13.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.442+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e00cf7e0 con 0x7fb6e00cf380 2024-07-17T08:35:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.443+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fb6cc042070 con 0x7fb6e00cf380 2024-07-17T08:35:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.444+0000 7fb6d7400640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 0x7fb6c00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.444+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7fb6cc0b9770 con 0x7fb6e00cf380 2024-07-17T08:35:13.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.445+0000 7fb6df400640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 0x7fb6c00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.446+0000 7fb6df400640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 0x7fb6c00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005940 tx=0x7fb6d0005c90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:13.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.448+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb6e0150a30 con 0x7fb6e00cf380 2024-07-17T08:35:13.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.666+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e0150750 con 0x7fb6e00cf380 2024-07-17T08:35:13.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.668+0000 7fb6d7400640 1 -- 172.21.15.44:0/2304236465 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) ==== 74+0+15056 (secure 0 0 0) 0x7fb6cc082760 con 0x7fb6e00cf380 2024-07-17T08:35:13.669 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:13.670 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":215,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:11.186502+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":208,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/271909788":"2024-07-18T08:35:07.837293+0000","172.21.15.175:6837/3642155749":"2024-07-18T08:35:07.717943+0000","172.21.15.175:6834/3642155749":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/255024555":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/2404136309":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/2971217224":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/2333353903":"2024-07-18T08:35:07.717943+0000","172.21.15.175:6837/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.175:6834/1104780009":"2024-07-18T08:32:01.140259+0000","172.21.15.44:6833/271909788":"2024-07-18T08:35:07.837293+0000","172.21.15.175:0/3942952311":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/340287962":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/1137184888":"2024-07-18T08:35:07.717943+0000","172.21.15.175:0/3551582397":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/693817936":"2024-07-18T08:32:01.140259+0000","172.21.15.175:0/464369211":"2024-07-18T08:32:01.140259+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 msgr2=0x7fb6c00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 0x7fb6c00791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005940 tx=0x7fb6d0005c90 comp rx=0 tx=0).stop 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 msgr2=0x7fb6e013d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009850 tx=0x7fb6cc0039e0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 shutdown_connections 2024-07-17T08:35:13.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6e0159800 0x7fb6e01404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e00cf380 0x7fb6e013d1f0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb6c0076d20 0x7fb6c00791e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 --2- 172.21.15.44:0/2304236465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6e006f4b0 0x7fb6e013da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.671+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 >> 172.21.15.44:0/2304236465 conn(0x7fb6e00b0e80 msgr2=0x7fb6e015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.672+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 shutdown_connections 2024-07-17T08:35:13.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.672+0000 7fb6e4e00640 1 -- 172.21.15.44:0/2304236465 wait complete. 2024-07-17T08:35:13.692 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:35:13.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.906+0000 7f96ef800640 1 -- 172.21.15.44:0/712559780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e80b3280 msgr2=0x7f96e80ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.906+0000 7f96ef800640 1 --2- 172.21.15.44:0/712559780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e80b3280 0x7f96e80ae560 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f96e80708b0 tx=0x7f96d402a560 comp rx=0 tx=0).stop 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.906+0000 7f96ef800640 1 -- 172.21.15.44:0/712559780 shutdown_connections 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.907+0000 7f96ef800640 1 --2- 172.21.15.44:0/712559780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e80b3280 0x7f96e80ae560 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.907+0000 7f96ef800640 1 --2- 172.21.15.44:0/712559780 >> v2:172.21.15.175:3300/0 conn(0x7f96e806eb90 0x7f96e815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.907+0000 7f96ef800640 1 -- 172.21.15.44:0/712559780 >> 172.21.15.44:0/712559780 conn(0x7f96e80b0eb0 msgr2=0x7f96e80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.907+0000 7f96ef800640 1 -- 172.21.15.44:0/712559780 shutdown_connections 2024-07-17T08:35:13.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.907+0000 7f96ef800640 1 -- 172.21.15.44:0/712559780 wait complete. 2024-07-17T08:35:13.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.909+0000 7f96ef800640 1 Processor -- start 2024-07-17T08:35:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.909+0000 7f96ef800640 1 -- start start 2024-07-17T08:35:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96e80ac3a0 0x7f96e8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e80b3280 0x7f96e8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f96e806f850 con 0x7f96e806eb90 2024-07-17T08:35:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96e806f6d0 con 0x7f96e80b3280 2024-07-17T08:35:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.910+0000 7f96ef800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f96e814fd60 con 0x7f96e80ac3a0 2024-07-17T08:35:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52742/0 (socket says 172.21.15.44:52742) 2024-07-17T08:35:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 -- 172.21.15.44:0/847218109 learned_addr learned my addr 172.21.15.44:0/847218109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:13.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96e7e00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96e80ac3a0 0x7f96e8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ed800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e80b3280 0x7f96e8152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 -- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96e80ac3a0 msgr2=0x7f96e8073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96e80ac3a0 0x7f96e8073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 -- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e80b3280 msgr2=0x7f96e8152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e80b3280 0x7f96e8152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.911+0000 7f96ece00640 1 -- 172.21.15.44:0/847218109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e8151220 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96ece00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f96d800dbf0 tx=0x7f96d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8005220 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96d800ae50 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d80047a0 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e80c9790 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.912+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e80c9d50 con 0x7f96e806eb90 2024-07-17T08:35:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.914+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e815a5a0 con 0x7f96e806eb90 2024-07-17T08:35:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.915+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f96d8004940 con 0x7f96e806eb90 2024-07-17T08:35:13.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.916+0000 7f96e4c00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 0x7f96c8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:13.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.917+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7f96d8098f70 con 0x7f96e806eb90 2024-07-17T08:35:13.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.917+0000 7f96e7e00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 0x7f96c8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:13.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.919+0000 7f96e7e00640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 0x7f96c8079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96d4033000 tx=0x7f96d4004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:13.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:13.921+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f96d8061f60 con 0x7f96e806eb90 2024-07-17T08:35:14.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.132+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96e80b4200 con 0x7f96e806eb90 2024-07-17T08:35:14.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.133+0000 7f96e4c00640 1 -- 172.21.15.44:0/847218109 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v215) ==== 93+0+10 (secure 0 0 0) 0x7f96d806a1a0 con 0x7f96e806eb90 2024-07-17T08:35:14.134 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.136+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 msgr2=0x7f96c8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:14.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.136+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 0x7f96c8079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96d4033000 tx=0x7f96d4004730 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.136+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 msgr2=0x7f96e8072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.136+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f96d800dbf0 tx=0x7f96d800a620 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 shutdown_connections 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f96c8076cd0 0x7f96c8079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96e80b3280 0x7f96e8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96e80ac3a0 0x7f96e8073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 --2- 172.21.15.44:0/847218109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96e806eb90 0x7f96e8072330 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 >> 172.21.15.44:0/847218109 conn(0x7f96e80b0eb0 msgr2=0x7f96e80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 shutdown_connections 2024-07-17T08:35:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.137+0000 7f96ef800640 1 -- 172.21.15.44:0/847218109 wait complete. 2024-07-17T08:35:14.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2024-07-17T08:35:14.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 -- 172.21.15.44:0/967715652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004800 msgr2=0x7fe798004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:14.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 --2- 172.21.15.44:0/967715652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004800 0x7fe798004c80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00bb20 tx=0x7fe798005830 comp rx=0 tx=0).stop 2024-07-17T08:35:14.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b5600640 1 -- 172.21.15.44:0/967715652 reap_dead start 2024-07-17T08:35:14.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 -- 172.21.15.44:0/967715652 shutdown_connections 2024-07-17T08:35:14.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 -- 172.21.15.44:0/967715652 >> 172.21.15.44:0/967715652 conn(0x7fe7b00b0fb0 msgr2=0x7fe7b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 -- 172.21.15.44:0/967715652 shutdown_connections 2024-07-17T08:35:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.362+0000 7fe7b7600640 1 -- 172.21.15.44:0/967715652 wait complete. 2024-07-17T08:35:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.364+0000 7fe7b7600640 1 Processor -- start 2024-07-17T08:35:14.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.364+0000 7fe7b7600640 1 -- start start 2024-07-17T08:35:14.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe798004800 0x7fe7b00b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:14.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b01485f0 0x7fe7b0147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b00ac7e0 con 0x7fe7b00b24b0 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b00ac660 con 0x7fe7b01485f0 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.365+0000 7fe7b7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b00ac960 con 0x7fe798004800 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52766/0 (socket says 172.21.15.44:52766) 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 -- 172.21.15.44:0/3722446862 learned_addr learned my addr 172.21.15.44:0/3722446862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b4c00640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe798004800 0x7fe7b00b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7afe00640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b01485f0 0x7fe7b0147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:14.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 -- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe798004800 msgr2=0x7fe7b00b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:14.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe798004800 0x7fe7b00b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 -- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b01485f0 msgr2=0x7fe7b0147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:14.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b01485f0 0x7fe7b0147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.366+0000 7fe7b5600640 1 -- 172.21.15.44:0/3722446862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b014b800 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7b5600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe7b006d5d0 tx=0x7fe7a800e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8018070 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7a80135f0 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800f040 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b014ba70 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.367+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b014c030 con 0x7fe7b00b24b0 2024-07-17T08:35:14.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.368+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b00cdb20 con 0x7fe7b00b24b0 2024-07-17T08:35:14.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.369+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fe7a801b070 con 0x7fe7b00b24b0 2024-07-17T08:35:14.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.370+0000 7fe7acc00640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 0x7fe7900791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:14.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.370+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7fe7a80979f0 con 0x7fe7b00b24b0 2024-07-17T08:35:14.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.371+0000 7fe7b4c00640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 0x7fe7900791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:14.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.373+0000 7fe7b4c00640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 0x7fe7900791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4001eb0 tx=0x7fe7a40016d0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:14.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.374+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7a8050f50 con 0x7fe7b00b24b0 2024-07-17T08:35:14.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:14.661+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) -- 0x7fe7b015c2e0 con 0x7fe7b00b24b0 2024-07-17T08:35:15.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.208+0000 7fe7acc00640 1 -- 172.21.15.44:0/3722446862 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3263) ==== 189+0+0 (secure 0 0 0) 0x7fe7a8060990 con 0x7fe7b00b24b0 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.212+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 msgr2=0x7fe7900791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.212+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 0x7fe7900791c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4001eb0 tx=0x7fe7a40016d0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.212+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 msgr2=0x7fe7b01e7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.212+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe7b006d5d0 tx=0x7fe7a800e920 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 shutdown_connections 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7b00b24b0 0x7fe7b01e7f40 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fe790076d00 0x7fe7900791c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7b01485f0 0x7fe7b0147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 --2- 172.21.15.44:0/3722446862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe798004800 0x7fe7b00b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 >> 172.21.15.44:0/3722446862 conn(0x7fe7b00b0fb0 msgr2=0x7fe7b00ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:15.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 shutdown_connections 2024-07-17T08:35:15.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.213+0000 7fe7b7600640 1 -- 172.21.15.44:0/3722446862 wait complete. 2024-07-17T08:35:15.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:35:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 -- 172.21.15.44:0/2617542732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc161450 msgr2=0x7f77fc0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 --2- 172.21.15.44:0/2617542732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc161450 0x7f77fc0ce240 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f77fc074a00 tx=0x7f77f402a820 comp rx=0 tx=0).stop 2024-07-17T08:35:15.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 -- 172.21.15.44:0/2617542732 shutdown_connections 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 --2- 172.21.15.44:0/2617542732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc161450 0x7f77fc0ce240 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 --2- 172.21.15.44:0/2617542732 >> v2:172.21.15.175:3300/0 conn(0x7f77fc1579d0 0x7f77fc0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 -- 172.21.15.44:0/2617542732 >> 172.21.15.44:0/2617542732 conn(0x7f77fc06c390 msgr2=0x7f77fc06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 -- 172.21.15.44:0/2617542732 shutdown_connections 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.442+0000 7f77fb200640 1 -- 172.21.15.44:0/2617542732 wait complete. 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.444+0000 7f77fb200640 1 Processor -- start 2024-07-17T08:35:15.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.444+0000 7f77fb200640 1 -- start start 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77fc15e3d0 0x7f77fc0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77fc161450 0x7f77fc14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0d1800 con 0x7f77fc1579d0 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0d1680 con 0x7f77fc161450 2024-07-17T08:35:15.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77fc0d1980 con 0x7f77fc15e3d0 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77f9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52784/0 (socket says 172.21.15.44:52784) 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.445+0000 7f77f9e00640 1 -- 172.21.15.44:0/3601761100 learned_addr learned my addr 172.21.15.44:0/3601761100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 -- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77fc15e3d0 msgr2=0x7f77fc0c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f3e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77fc15e3d0 0x7f77fc0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77fc15e3d0 0x7f77fc0c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 -- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77fc161450 msgr2=0x7f77fc14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77fa800640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77fc161450 0x7f77fc14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77fc161450 0x7f77fc14aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 -- 172.21.15.44:0/3601761100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc0ca120 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f9e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c890 tx=0x7f77ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec004220 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ec0079b0 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec007de0 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.446+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1f7700 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.447+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1f7cc0 con 0x7f77fc1579d0 2024-07-17T08:35:15.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.448+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc06a1a0 con 0x7f77fc1579d0 2024-07-17T08:35:15.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.449+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f77ec005020 con 0x7f77fc1579d0 2024-07-17T08:35:15.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.450+0000 7f77f2000640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 0x7f77d8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.450+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7f77ec098c80 con 0x7f77fc1579d0 2024-07-17T08:35:15.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.450+0000 7f77f3e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 0x7f77d8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.452+0000 7f77f3e00640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 0x7f77d8079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0c7530 tx=0x7f77e4009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:15.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.455+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f77fc06a1a0 con 0x7f77fc1579d0 2024-07-17T08:35:15.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.672+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f77fc0d1e30 con 0x7f77fc1579d0 2024-07-17T08:35:15.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.674+0000 7f77f2000640 1 -- 172.21.15.44:0/3601761100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v71) ==== 59+0+1998 (secure 0 0 0) 0x7f77ec014030 con 0x7f77fc1579d0 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:35:15.675 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:35:15.676 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:35:15.677 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:35:15.677 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:35:15.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 msgr2=0x7f77d8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 0x7f77d8079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0c7530 tx=0x7f77e4009040 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 msgr2=0x7f77fc1587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c890 tx=0x7f77ec00cd50 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 shutdown_connections 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f77d8076cd0 0x7f77d8079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77fc161450 0x7f77fc14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77fc15e3d0 0x7f77fc0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 --2- 172.21.15.44:0/3601761100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77fc1579d0 0x7f77fc1587e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 >> 172.21.15.44:0/3601761100 conn(0x7f77fc06c390 msgr2=0x7f77fc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 shutdown_connections 2024-07-17T08:35:15.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.677+0000 7f77fb200640 1 -- 172.21.15.44:0/3601761100 wait complete. 2024-07-17T08:35:15.697 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 -- 172.21.15.44:0/1201916713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 msgr2=0x7fc1ec00cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 --2- 172.21.15.44:0/1201916713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc1ec00cad0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c640 tx=0x7fc1ec006940 comp rx=0 tx=0).stop 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc206800640 1 -- 172.21.15.44:0/1201916713 reap_dead start 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 -- 172.21.15.44:0/1201916713 shutdown_connections 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 -- 172.21.15.44:0/1201916713 >> 172.21.15.44:0/1201916713 conn(0x7fc2080cf0d0 msgr2=0x7fc2080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 -- 172.21.15.44:0/1201916713 shutdown_connections 2024-07-17T08:35:15.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.906+0000 7fc207200640 1 -- 172.21.15.44:0/1201916713 wait complete. 2024-07-17T08:35:15.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.908+0000 7fc207200640 1 Processor -- start 2024-07-17T08:35:15.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.908+0000 7fc207200640 1 -- start start 2024-07-17T08:35:15.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc208144170 0x7fc2080ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2080ce7d0 0x7fc2081f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc20815a7e0 con 0x7fc1ec0063f0 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc20815a660 con 0x7fc208144170 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc207200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc20806d8b0 con 0x7fc2080ce7d0 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc205e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc205e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52806/0 (socket says 172.21.15.44:52806) 2024-07-17T08:35:15.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.909+0000 7fc205e00640 1 -- 172.21.15.44:0/1956667522 learned_addr learned my addr 172.21.15.44:0/1956667522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc208144170 0x7fc2080ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 -- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2080ce7d0 msgr2=0x7fc2081f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2080ce7d0 0x7fc2081f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 -- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc208144170 msgr2=0x7fc2080ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc208144170 0x7fc2080ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 -- 172.21.15.44:0/1956667522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20814fa70 con 0x7fc1ec0063f0 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc205e00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e6e0 tx=0x7fc1f800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:15.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800c9f0 con 0x7fc1ec0063f0 2024-07-17T08:35:15.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f800cc80 con 0x7fc1ec0063f0 2024-07-17T08:35:15.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081f4e10 con 0x7fc1ec0063f0 2024-07-17T08:35:15.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80116d0 con 0x7fc1ec0063f0 2024-07-17T08:35:15.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.910+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc208072390 con 0x7fc1ec0063f0 2024-07-17T08:35:15.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.912+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2080ac010 con 0x7fc1ec0063f0 2024-07-17T08:35:15.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.913+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fc1f8011870 con 0x7fc1ec0063f0 2024-07-17T08:35:15.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.914+0000 7fc1fe000640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 0x7fc1e4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:15.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.914+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7fc1f80992e0 con 0x7fc1ec0063f0 2024-07-17T08:35:15.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.914+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 0x7fc1e4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.916+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 0x7fc1e4079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0006a40 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:15.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:15.919+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc20814fa70 con 0x7fc1ec0063f0 2024-07-17T08:35:16.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.130+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc2080cffe0 con 0x7fc1ec0063f0 2024-07-17T08:35:16.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.136+0000 7fc1fe000640 1 -- 172.21.15.44:0/1956667522 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v72)=0 v72) ==== 122+0+0 (secure 0 0 0) 0x7fc1f8016030 con 0x7fc1ec0063f0 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.139+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 msgr2=0x7fc1e4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.139+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 0x7fc1e4079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0006a40 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.139+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 msgr2=0x7fc20809ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.139+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e6e0 tx=0x7fc1f800eba0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 shutdown_connections 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2080ce7d0 0x7fc2081f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc1e4076cd0 0x7fc1e4079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc208144170 0x7fc2080ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 --2- 172.21.15.44:0/1956667522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1ec0063f0 0x7fc20809ccb0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 >> 172.21.15.44:0/1956667522 conn(0x7fc2080cf0d0 msgr2=0x7fc20806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:16.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 shutdown_connections 2024-07-17T08:35:16.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.140+0000 7fc207200640 1 -- 172.21.15.44:0/1956667522 wait complete. 2024-07-17T08:35:16.162 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:35:16.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:16.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:16.403 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:35:16.404 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:16.404 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:16.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:35:16.462 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:35:16.462 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:16.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:35:16.518 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:16.518 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:16.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:35:16.577 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:35:16.577 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:16.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:35:16.633 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:35:16.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.853+0000 7fd3a9000640 1 -- 172.21.15.44:0/536772321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 msgr2=0x7fd38800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.853+0000 7fd3a9000640 1 --2- 172.21.15.44:0/536772321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd38800cad0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009e20 tx=0x7fd388006940 comp rx=0 tx=0).stop 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.853+0000 7fd3a3e00640 1 -- 172.21.15.44:0/536772321 reap_dead start 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.854+0000 7fd3a9000640 1 -- 172.21.15.44:0/536772321 shutdown_connections 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.854+0000 7fd3a9000640 1 -- 172.21.15.44:0/536772321 >> 172.21.15.44:0/536772321 conn(0x7fd3a40b0e80 msgr2=0x7fd3a40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.854+0000 7fd3a9000640 1 -- 172.21.15.44:0/536772321 shutdown_connections 2024-07-17T08:35:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.854+0000 7fd3a9000640 1 -- 172.21.15.44:0/536772321 wait complete. 2024-07-17T08:35:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.855+0000 7fd3a9000640 1 Processor -- start 2024-07-17T08:35:16.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.856+0000 7fd3a9000640 1 -- start start 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 0x7fd3a40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4150be0 0x7fd3a4151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a409b690 con 0x7fd3880062d0 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a409b510 con 0x7fd3a4150be0 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a9000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a409b810 con 0x7fd3a4158a50 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52816/0 (socket says 172.21.15.44:52816) 2024-07-17T08:35:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 -- 172.21.15.44:0/2240620358 learned_addr learned my addr 172.21.15.44:0/2240620358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd39be00640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 0x7fd3a40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3e00640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4150be0 0x7fd3a4151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 -- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 msgr2=0x7fd3a40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 0x7fd3a40cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 -- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4150be0 msgr2=0x7fd3a4151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4150be0 0x7fd3a4151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd3a3400640 1 -- 172.21.15.44:0/2240620358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40cb920 con 0x7fd3880062d0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.857+0000 7fd39be00640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 0x7fd3a40cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd3a3400640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d700 tx=0x7fd39400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd3880062d0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd394004310 con 0x7fd3880062d0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394009a20 con 0x7fd3880062d0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a40cbbf0 con 0x7fd3880062d0 2024-07-17T08:35:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.858+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a40cc1b0 con 0x7fd3880062d0 2024-07-17T08:35:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.859+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a40cf7e0 con 0x7fd3880062d0 2024-07-17T08:35:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.860+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fd394009bc0 con 0x7fd3880062d0 2024-07-17T08:35:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.861+0000 7fd39b400640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 0x7fd3800791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.861+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7fd394098a70 con 0x7fd3880062d0 2024-07-17T08:35:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.862+0000 7fd39be00640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 0x7fd3800791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:16.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.863+0000 7fd39be00640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 0x7fd3800791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40b30c0 tx=0x7fd38c001660 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:16.866+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd394020030 con 0x7fd3880062d0 2024-07-17T08:35:17.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.119+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3a40b2ad0 con 0x7fd3880062d0 2024-07-17T08:35:17.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.119+0000 7fd39b400640 1 -- 172.21.15.44:0/2240620358 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+3919 (secure 0 0 0) 0x7fd3940619e0 con 0x7fd3880062d0 2024-07-17T08:35:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:17.121 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":107,"btime":"2024-07-17T08:35:08:841240+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2935110837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2935110837},{"type":"v1","addr":"172.21.15.175:6801","nonce":2935110837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":7403,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/258803827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":258803827},{"type":"v1","addr":"172.21.15.175:6836","nonce":258803827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":9609,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1942518219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1942518219},{"type":"v1","addr":"172.21.15.44:6834","nonce":1942518219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2721712643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2721712643},{"type":"v1","addr":"172.21.15.44:6835","nonce":2721712643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[]} 2024-07-17T08:35:17.121 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 107 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 msgr2=0x7fd3800791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 0x7fd3800791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40b30c0 tx=0x7fd38c001660 comp rx=0 tx=0).stop 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 msgr2=0x7fd3a414edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d700 tx=0x7fd39400dbc0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 shutdown_connections 2024-07-17T08:35:17.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd380076d00 0x7fd3800791c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a4150be0 0x7fd3a4151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a4158a50 0x7fd3a40cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 --2- 172.21.15.44:0/2240620358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3880062d0 0x7fd3a414edc0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 >> 172.21.15.44:0/2240620358 conn(0x7fd3a40b0e80 msgr2=0x7fd3a40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.122+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 shutdown_connections 2024-07-17T08:35:17.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.123+0000 7fd3a9000640 1 -- 172.21.15.44:0/2240620358 wait complete. 2024-07-17T08:35:17.144 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:35:17.144 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:35:17.144 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:35:17.145 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:35:17.145 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:35:17.145 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:35:17.145 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:35:17.145 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:35:17.145 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:35:17.146 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:35:17.146 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:35:17.146 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:35:17.146 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:35:17.146 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:35:17.146 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:35:17.147 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:35:17.161 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:35:17.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:35:17.166 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:35:17.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:35:17.176 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:35:17.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:35:17.178 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:35:17.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:35:17.223 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:35:17.223 DEBUG:teuthology.parallel:result is None 2024-07-17T08:35:17.224 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:35:17.224 DEBUG:teuthology.parallel:result is None 2024-07-17T08:35:17.229 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:35:17.230 DEBUG:teuthology.parallel:result is None 2024-07-17T08:35:17.230 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:35:17.230 DEBUG:teuthology.parallel:result is None 2024-07-17T08:35:17.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:35:17.372 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:35:17.372 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:35:17.374 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:35:17.375 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:35:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 -- 172.21.15.44:0/1992579295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c0b3280 msgr2=0x7f541c0ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:17.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 --2- 172.21.15.44:0/1992579295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c0b3280 0x7f541c0ae560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5408009830 tx=0x7f540802a530 comp rx=0 tx=0).stop 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 -- 172.21.15.44:0/1992579295 shutdown_connections 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 --2- 172.21.15.44:0/1992579295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c0b3280 0x7f541c0ae560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 --2- 172.21.15.44:0/1992579295 >> v2:172.21.15.175:3300/0 conn(0x7f541c06eb90 0x7f541c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 -- 172.21.15.44:0/1992579295 >> 172.21.15.44:0/1992579295 conn(0x7f541c0b0eb0 msgr2=0x7f541c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 -- 172.21.15.44:0/1992579295 shutdown_connections 2024-07-17T08:35:17.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.415+0000 7f5423e00640 1 -- 172.21.15.44:0/1992579295 wait complete. 2024-07-17T08:35:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.417+0000 7f5423e00640 1 Processor -- start 2024-07-17T08:35:17.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.417+0000 7f5423e00640 1 -- start start 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c06eb90 0x7f541c14ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c0ac3a0 0x7f541c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f541c06f850 con 0x7f541c0b3280 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f541c06f6d0 con 0x7f541c0ac3a0 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5423e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f541c150470 con 0x7f541c06eb90 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.418+0000 7f5421e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52852/0 (socket says 172.21.15.44:52852) 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 -- 172.21.15.44:0/2295815694 learned_addr learned my addr 172.21.15.44:0/2295815694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421400640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c06eb90 0x7f541c14ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:17.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5420a00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c0ac3a0 0x7f541c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 -- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c06eb90 msgr2=0x7f541c14ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c06eb90 0x7f541c14ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 -- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c0ac3a0 msgr2=0x7f541c14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c0ac3a0 0x7f541c14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 -- 172.21.15.44:0/2295815694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c074010 con 0x7f541c0b3280 2024-07-17T08:35:17.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5421e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f541400cbc0 tx=0x7f541400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.419+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414016020 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.420+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f541400ad90 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.420+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414012680 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.420+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c156630 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.420+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1568d0 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.421+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c15a5a0 con 0x7f541c0b3280 2024-07-17T08:35:17.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.422+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f5414019020 con 0x7f541c0b3280 2024-07-17T08:35:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.423+0000 7f5419600640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 0x7f53fc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.423+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6513+0+0 (secure 0 0 0) 0x7f5414098f90 con 0x7f541c0b3280 2024-07-17T08:35:17.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.423+0000 7f5421400640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 0x7f53fc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.424+0000 7f5421400640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 0x7f53fc0791c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f541c14f0d0 tx=0x7f5410005f90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:17.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.427+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5414020020 con 0x7f541c0b3280 2024-07-17T08:35:17.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:17.653+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f541c0744f0 con 0x7f541c0b3280 2024-07-17T08:35:18.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.248+0000 7f5419600640 1 -- 172.21.15.44:0/2295815694 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v217) ==== 97+0+0 (secure 0 0 0) 0x7f5414061f80 con 0x7f541c0b3280 2024-07-17T08:35:18.249 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:35:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 msgr2=0x7f53fc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 0x7f53fc0791c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f541c14f0d0 tx=0x7f5410005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 msgr2=0x7f541c155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f541400cbc0 tx=0x7f541400a060 comp rx=0 tx=0).stop 2024-07-17T08:35:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 shutdown_connections 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f541c0b3280 0x7f541c155860 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f53fc076d00 0x7f53fc0791c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541c0ac3a0 0x7f541c14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 --2- 172.21.15.44:0/2295815694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f541c06eb90 0x7f541c14ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.252+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 >> 172.21.15.44:0/2295815694 conn(0x7f541c0b0eb0 msgr2=0x7f541c15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.253+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 shutdown_connections 2024-07-17T08:35:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.253+0000 7f5423e00640 1 -- 172.21.15.44:0/2295815694 wait complete. 2024-07-17T08:35:18.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:35:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.477+0000 7f49bb400640 1 -- 172.21.15.44:0/501740690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 msgr2=0x7f49b40af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.477+0000 7f49bb400640 1 --2- 172.21.15.44:0/501740690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b40af520 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f49b4073c70 tx=0x7f49ac02aa10 comp rx=0 tx=0).stop 2024-07-17T08:35:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 -- 172.21.15.44:0/501740690 shutdown_connections 2024-07-17T08:35:18.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 --2- 172.21.15.44:0/501740690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b40af520 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 --2- 172.21.15.44:0/501740690 >> v2:172.21.15.175:3300/0 conn(0x7f49b415b6e0 0x7f49b40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 -- 172.21.15.44:0/501740690 >> 172.21.15.44:0/501740690 conn(0x7f49b40b3120 msgr2=0x7f49b40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 -- 172.21.15.44:0/501740690 shutdown_connections 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.478+0000 7f49bb400640 1 -- 172.21.15.44:0/501740690 wait complete. 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.480+0000 7f49bb400640 1 Processor -- start 2024-07-17T08:35:18.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.480+0000 7f49bb400640 1 -- start start 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40b3cb0 0x7f49b40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415b6e0 0x7f49b40b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49b415f320 con 0x7f49b40b06b0 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49b415f1a0 con 0x7f49b415b6e0 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49bb400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49b415f4a0 con 0x7f49b40b3cb0 2024-07-17T08:35:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49b8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49b8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52872/0 (socket says 172.21.15.44:52872) 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49b8a00640 1 -- 172.21.15.44:0/2083495005 learned_addr learned my addr 172.21.15.44:0/2083495005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49b3e00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40b3cb0 0x7f49b40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.481+0000 7f49b9400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415b6e0 0x7f49b40b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 -- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40b3cb0 msgr2=0x7f49b40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40b3cb0 0x7f49b40cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 -- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415b6e0 msgr2=0x7f49b40b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415b6e0 0x7f49b40b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 -- 172.21.15.44:0/2083495005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40ce5f0 con 0x7f49b40b06b0 2024-07-17T08:35:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b8a00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b410 tx=0x7f49a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4015020 con 0x7f49b40b06b0 2024-07-17T08:35:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49a4011940 con 0x7f49b40b06b0 2024-07-17T08:35:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4011d90 con 0x7f49b40b06b0 2024-07-17T08:35:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b40ce8d0 con 0x7f49b40b06b0 2024-07-17T08:35:18.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.482+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b40cbd20 con 0x7f49b40b06b0 2024-07-17T08:35:18.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.483+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b40702c0 con 0x7f49b40b06b0 2024-07-17T08:35:18.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.485+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f49a4018020 con 0x7f49b40b06b0 2024-07-17T08:35:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.485+0000 7f49b0c00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 0x7f4990079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.486+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 5825+0+0 (secure 0 0 0) 0x7f49a4098500 con 0x7f49b40b06b0 2024-07-17T08:35:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.486+0000 7f49b3e00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 0x7f4990079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.487+0000 7f49b3e00640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 0x7f4990079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f499c007fd0 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.490+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f49a401f020 con 0x7f49b40b06b0 2024-07-17T08:35:18.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.777+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f49b414fdd0 con 0x7f49b40b06b0 2024-07-17T08:35:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.779+0000 7f49b0c00640 1 -- 172.21.15.44:0/2083495005 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v71) ==== 80+0+5304 (secure 0 0 0) 0x7f49a4061830 con 0x7f49b40b06b0 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.780 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.781 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:35:18.784 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:35:18.788 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:35:18.789 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:35:18.790 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:35:18.791 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.792 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:35:18.793 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:35:18.794 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.794 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:18.794 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:35:18.794 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.781+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 msgr2=0x7f4990079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.781+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 0x7f4990079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f499c007fd0 tx=0x7f499c009040 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.781+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 msgr2=0x7f49b4157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.781+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b410 tx=0x7f49a400b8d0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 shutdown_connections 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4990076cd0 0x7f4990079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b415b6e0 0x7f49b40b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49b40b3cb0 0x7f49b40cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 --2- 172.21.15.44:0/2083495005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b40b06b0 0x7f49b4157ba0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 >> 172.21.15.44:0/2083495005 conn(0x7f49b40b3120 msgr2=0x7f49b40b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 shutdown_connections 2024-07-17T08:35:18.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.782+0000 7f49bb400640 1 -- 172.21.15.44:0/2083495005 wait complete. 2024-07-17T08:35:18.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:35:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3829435889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b300 msgr2=0x7f5eb806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3829435889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b300 0x7f5eb806e060 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8153e90 tx=0x7f5ea802a560 comp rx=0 tx=0).stop 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3829435889 shutdown_connections 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3829435889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb809b6e0 0x7f5eb80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3829435889 >> v2:172.21.15.175:3300/0 conn(0x7f5eb80cf8c0 0x7f5eb8159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3829435889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b300 0x7f5eb806e060 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3829435889 >> 172.21.15.44:0/3829435889 conn(0x7f5eb80b0e80 msgr2=0x7f5eb80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3829435889 shutdown_connections 2024-07-17T08:35:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.994+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3829435889 wait complete. 2024-07-17T08:35:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.996+0000 7f5ebcc00640 1 Processor -- start 2024-07-17T08:35:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.996+0000 7f5ebcc00640 1 -- start start 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb809b300 0x7f5eb81447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80cf8c0 0x7f5eb81585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815a2d0 con 0x7f5eb809b6e0 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815a150 con 0x7f5eb80cf8c0 2024-07-17T08:35:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5ebcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb80b1fa0 con 0x7f5eb809b300 2024-07-17T08:35:18.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5eaea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:18.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5eaea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59922/0 (socket says 172.21.15.44:59922) 2024-07-17T08:35:18.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.997+0000 7f5eaea00640 1 -- 172.21.15.44:0/3625437560 learned_addr learned my addr 172.21.15.44:0/3625437560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 -- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb809b300 msgr2=0x7f5eb81447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb809b300 0x7f5eb81447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 -- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80cf8c0 msgr2=0x7f5eb81585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eb7e00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80cf8c0 0x7f5eb81585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80cf8c0 0x7f5eb81585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 -- 172.21.15.44:0/3625437560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb8156ef0 con 0x7f5eb809b6e0 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eaea00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000b4a0 tx=0x7f5ea000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.998+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0014070 con 0x7f5eb809b6e0 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.999+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea0010700 con 0x7f5eb809b6e0 2024-07-17T08:35:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.999+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0010b50 con 0x7f5eb809b6e0 2024-07-17T08:35:19.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.999+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81f2400 con 0x7f5eb809b6e0 2024-07-17T08:35:19.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:18.999+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81f2100 con 0x7f5eb809b6e0 2024-07-17T08:35:19.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.000+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb80cf380 con 0x7f5eb809b6e0 2024-07-17T08:35:19.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.001+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f5ea0017070 con 0x7f5eb809b6e0 2024-07-17T08:35:19.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.002+0000 7f5eafe00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 0x7f5e94079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:19.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.002+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 5825+0+0 (secure 0 0 0) 0x7f5ea0097f20 con 0x7f5eb809b6e0 2024-07-17T08:35:19.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.003+0000 7f5eb7400640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 0x7f5e94079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:19.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.004+0000 7f5eb7400640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 0x7f5e94079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8033000 tx=0x7f5ea80046c0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:19.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.006+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5ea00612f0 con 0x7f5eb809b6e0 2024-07-17T08:35:19.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.206+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5eb81f25d0 con 0x7f5eb809b6e0 2024-07-17T08:35:19.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.207+0000 7f5eafe00640 1 -- 172.21.15.44:0/3625437560 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v219) ==== 84+0+65 (secure 0 0 0) 0x7f5ea0060bf0 con 0x7f5eb809b6e0 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:19.209 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:35:19.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 msgr2=0x7f5e94079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 0x7f5e94079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8033000 tx=0x7f5ea80046c0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 msgr2=0x7f5eb8156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000b4a0 tx=0x7f5ea000b960 comp rx=0 tx=0).stop 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 shutdown_connections 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5e94076cd0 0x7f5e94079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eb80cf8c0 0x7f5eb81585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5eb809b6e0 0x7f5eb8156550 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 --2- 172.21.15.44:0/3625437560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5eb809b300 0x7f5eb81447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.209+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 >> 172.21.15.44:0/3625437560 conn(0x7f5eb80b0e80 msgr2=0x7f5eb80abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.210+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 shutdown_connections 2024-07-17T08:35:19.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.210+0000 7f5ebcc00640 1 -- 172.21.15.44:0/3625437560 wait complete. 2024-07-17T08:35:19.231 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:35:19.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.431+0000 7efe90e00640 1 -- 172.21.15.44:0/3170561657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c0ad960 msgr2=0x7efe8c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.431+0000 7efe90e00640 1 --2- 172.21.15.44:0/3170561657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c0ad960 0x7efe8c0add80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fd0 tx=0x7efe7402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.432+0000 7efe90e00640 1 -- 172.21.15.44:0/3170561657 shutdown_connections 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.432+0000 7efe90e00640 1 --2- 172.21.15.44:0/3170561657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c0ad960 0x7efe8c0add80 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.432+0000 7efe90e00640 1 -- 172.21.15.44:0/3170561657 >> 172.21.15.44:0/3170561657 conn(0x7efe8c0cf050 msgr2=0x7efe8c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.432+0000 7efe90e00640 1 -- 172.21.15.44:0/3170561657 shutdown_connections 2024-07-17T08:35:19.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.432+0000 7efe90e00640 1 -- 172.21.15.44:0/3170561657 wait complete. 2024-07-17T08:35:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.434+0000 7efe90e00640 1 Processor -- start 2024-07-17T08:35:19.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.434+0000 7efe90e00640 1 -- start start 2024-07-17T08:35:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0ad960 0x7efe8c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c0ae2c0 0x7efe8c149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c073450 con 0x7efe8c155860 2024-07-17T08:35:19.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c0732d0 con 0x7efe8c0ae2c0 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe90e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe8c0735d0 con 0x7efe8c0ad960 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59930/0 (socket says 172.21.15.44:59930) 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.435+0000 7efe8be00640 1 -- 172.21.15.44:0/139682829 learned_addr learned my addr 172.21.15.44:0/139682829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe83e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c0ae2c0 0x7efe8c149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 -- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0ad960 msgr2=0x7efe8c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0ad960 0x7efe8c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 -- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c0ae2c0 msgr2=0x7efe8c149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c0ae2c0 0x7efe8c149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:19.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 -- 172.21.15.44:0/139682829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c159980 con 0x7efe8c155860 2024-07-17T08:35:19.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe8be00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c7a0 tx=0x7efe8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:19.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84003b70 con 0x7efe8c155860 2024-07-17T08:35:19.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe84003d10 con 0x7efe8c155860 2024-07-17T08:35:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c159c60 con 0x7efe8c155860 2024-07-17T08:35:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400aab0 con 0x7efe8c155860 2024-07-17T08:35:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.436+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c15a220 con 0x7efe8c155860 2024-07-17T08:35:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.437+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c073a50 con 0x7efe8c155860 2024-07-17T08:35:19.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.439+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7efe8400ac50 con 0x7efe8c155860 2024-07-17T08:35:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.440+0000 7efe83400640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 0x7efe68079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.440+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 5825+0+0 (secure 0 0 0) 0x7efe840984a0 con 0x7efe8c155860 2024-07-17T08:35:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.441+0000 7efe8b400640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 0x7efe68079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:19.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.442+0000 7efe8b400640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 0x7efe68079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006fd0 tx=0x7efe7c006bc0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:19.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.444+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efe84061740 con 0x7efe8c155860 2024-07-17T08:35:19.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:19.658+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efe8c06f430 con 0x7efe8c155860 2024-07-17T08:35:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.259+0000 7efe83400640 1 -- 172.21.15.44:0/139682829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v220) ==== 138+0+0 (secure 0 0 0) 0x7efe84061040 con 0x7efe8c155860 2024-07-17T08:35:20.261 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:35:20.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 msgr2=0x7efe68079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:20.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 0x7efe68079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe7c006fd0 tx=0x7efe7c006bc0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 msgr2=0x7efe8c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:20.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c7a0 tx=0x7efe8400cc60 comp rx=0 tx=0).stop 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 shutdown_connections 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe8c155860 0x7efe8c1565d0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7efe68076cd0 0x7efe68079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c0ae2c0 0x7efe8c149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.263+0000 7efe90e00640 1 --2- 172.21.15.44:0/139682829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe8c0ad960 0x7efe8c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.264+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 >> 172.21.15.44:0/139682829 conn(0x7efe8c0cf050 msgr2=0x7efe8c0b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.264+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 shutdown_connections 2024-07-17T08:35:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.264+0000 7efe90e00640 1 -- 172.21.15.44:0/139682829 wait complete. 2024-07-17T08:35:20.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.503+0000 7f8ee2c00640 1 -- 172.21.15.44:0/2183174721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ec4005230 msgr2=0x7f8ec400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.503+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/2183174721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ec4005230 0x7f8ec400b9c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c310 tx=0x7f8ec4005830 comp rx=0 tx=0).stop 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.504+0000 7f8ee2c00640 1 -- 172.21.15.44:0/2183174721 shutdown_connections 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.504+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/2183174721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ec4005230 0x7f8ec400b9c0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.504+0000 7f8ee2c00640 1 -- 172.21.15.44:0/2183174721 >> 172.21.15.44:0/2183174721 conn(0x7f8edc0cd490 msgr2=0x7f8edc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.504+0000 7f8ee2c00640 1 -- 172.21.15.44:0/2183174721 shutdown_connections 2024-07-17T08:35:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.504+0000 7f8ee2c00640 1 -- 172.21.15.44:0/2183174721 wait complete. 2024-07-17T08:35:20.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.505+0000 7f8ee2c00640 1 Processor -- start 2024-07-17T08:35:20.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.506+0000 7f8ee2c00640 1 -- start start 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ec4005230 0x7f8edc06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8edc1f3aa0 0x7f8edc0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc0aec60 con 0x7f8edc14fe00 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc06f720 con 0x7f8edc1f3aa0 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc0aede0 con 0x7f8ec4005230 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59946/0 (socket says 172.21.15.44:59946) 2024-07-17T08:35:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee2200640 1 -- 172.21.15.44:0/4156722699 learned_addr learned my addr 172.21.15.44:0/4156722699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee0e00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8edc1f3aa0 0x7f8edc0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.507+0000 7f8ee1800640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ec4005230 0x7f8edc06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 -- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ec4005230 msgr2=0x7f8edc06d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ec4005230 0x7f8edc06d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 -- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8edc1f3aa0 msgr2=0x7f8edc0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8edc1f3aa0 0x7f8edc0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 -- 172.21.15.44:0/4156722699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc1f4220 con 0x7f8edc14fe00 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2200640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c370 tx=0x7f8ed4001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4032050 con 0x7f8edc14fe00 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ed4003af0 con 0x7f8edc14fe00 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed402e440 con 0x7f8edc14fe00 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.508+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1f4500 con 0x7f8edc14fe00 2024-07-17T08:35:20.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.509+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1f4ac0 con 0x7f8edc14fe00 2024-07-17T08:35:20.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.510+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc15ecf0 con 0x7f8edc14fe00 2024-07-17T08:35:20.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.511+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f8ed4035020 con 0x7f8edc14fe00 2024-07-17T08:35:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.512+0000 7f8ed9600640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 0x7f8ebc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:20.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.513+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(220..220 src has 1..220) ==== 6195+0+0 (secure 0 0 0) 0x7f8ed40b8680 con 0x7f8edc14fe00 2024-07-17T08:35:20.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.513+0000 7f8ee1800640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 0x7f8ebc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:20.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.514+0000 7f8ee1800640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 0x7f8ebc079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8edc150de0 tx=0x7f8ed0009500 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.518+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8ed4081840 con 0x7f8edc14fe00 2024-07-17T08:35:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:20.728+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8edc068ff0 con 0x7f8edc14fe00 2024-07-17T08:35:21.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.278+0000 7f8ed9600640 1 -- 172.21.15.44:0/4156722699 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v221) ==== 170+0+0 (secure 0 0 0) 0x7f8ed4081140 con 0x7f8edc14fe00 2024-07-17T08:35:21.280 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.283+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 msgr2=0x7f8ebc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.283+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 0x7f8ebc079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8edc150de0 tx=0x7f8ed0009500 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.283+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 msgr2=0x7f8edc1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.283+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c370 tx=0x7f8ed4001de0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 shutdown_connections 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8edc14fe00 0x7f8edc1502b0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8ebc076cd0 0x7f8ebc079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8edc1f3aa0 0x7f8edc0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 --2- 172.21.15.44:0/4156722699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ec4005230 0x7f8edc06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 >> 172.21.15.44:0/4156722699 conn(0x7f8edc0cd490 msgr2=0x7f8edc0af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:21.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.284+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 shutdown_connections 2024-07-17T08:35:21.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.285+0000 7f8ee2c00640 1 -- 172.21.15.44:0/4156722699 wait complete. 2024-07-17T08:35:21.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:35:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 -- 172.21.15.44:0/1084503921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc004800 msgr2=0x7f3bbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 --2- 172.21.15.44:0/1084503921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b9d0 tx=0x7f3bbc005830 comp rx=0 tx=0).stop 2024-07-17T08:35:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bda800640 1 -- 172.21.15.44:0/1084503921 reap_dead start 2024-07-17T08:35:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 -- 172.21.15.44:0/1084503921 shutdown_connections 2024-07-17T08:35:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 -- 172.21.15.44:0/1084503921 >> 172.21.15.44:0/1084503921 conn(0x7f3bd40b0eb0 msgr2=0x7f3bd40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 -- 172.21.15.44:0/1084503921 shutdown_connections 2024-07-17T08:35:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.502+0000 7f3bdc800640 1 -- 172.21.15.44:0/1084503921 wait complete. 2024-07-17T08:35:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.504+0000 7f3bdc800640 1 Processor -- start 2024-07-17T08:35:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.504+0000 7f3bdc800640 1 -- start start 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc004800 0x7f3bd4155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 0x7f3bd41edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40b3280 con 0x7f3bd41f4630 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40b3400 con 0x7f3bd4158df0 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bdc800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd40b3580 con 0x7f3bbc004800 2024-07-17T08:35:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bd9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bd9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59956/0 (socket says 172.21.15.44:59956) 2024-07-17T08:35:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.505+0000 7f3bd9400640 1 -- 172.21.15.44:0/676246958 learned_addr learned my addr 172.21.15.44:0/676246958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bda800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 0x7f3bd41edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 -- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc004800 msgr2=0x7f3bd4155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9e00640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc004800 0x7f3bd4155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc004800 0x7f3bd4155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 -- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 msgr2=0x7f3bd41edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 0x7f3bd41edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 -- 172.21.15.44:0/676246958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd4071680 con 0x7f3bd41f4630 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bda800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 0x7f3bd41edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bd9400640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f3bc402aea0 tx=0x7f3bc4034e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.506+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4043030 con 0x7f3bd41f4630 2024-07-17T08:35:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.507+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bc4034470 con 0x7f3bd41f4630 2024-07-17T08:35:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.507+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4039040 con 0x7f3bd41f4630 2024-07-17T08:35:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.507+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41f8ac0 con 0x7f3bd41f4630 2024-07-17T08:35:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.507+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd41f4ed0 con 0x7f3bd41f4630 2024-07-17T08:35:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.508+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd40ac3a0 con 0x7f3bd41f4630 2024-07-17T08:35:21.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.509+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f3bc4046070 con 0x7f3bd41f4630 2024-07-17T08:35:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.510+0000 7f3bce000640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 0x7f3bb8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.510+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 6577+0+0 (secure 0 0 0) 0x7f3bc40b91d0 con 0x7f3bd41f4630 2024-07-17T08:35:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.511+0000 7f3bd9e00640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 0x7f3bb8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.512+0000 7f3bd9e00640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 0x7f3bb8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000c00 tx=0x7f3bc8006a40 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.514+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bc4030050 con 0x7f3bd41f4630 2024-07-17T08:35:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:21.765+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3bd40b4200 con 0x7f3bd41f4630 2024-07-17T08:35:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.303+0000 7f3bce000640 1 -- 172.21.15.44:0/676246958 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v110) ==== 174+0+0 (secure 0 0 0) 0x7f3bc408a2b0 con 0x7f3bd41f4630 2024-07-17T08:35:22.318 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 27 and data pool 28 2024-07-17T08:35:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.317+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 msgr2=0x7f3bb8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.317+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 0x7f3bb8079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000c00 tx=0x7f3bc8006a40 comp rx=0 tx=0).stop 2024-07-17T08:35:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.317+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 msgr2=0x7f3bd41f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.317+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f3bc402aea0 tx=0x7f3bc4034e60 comp rx=0 tx=0).stop 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 shutdown_connections 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3bb8076cd0 0x7f3bb8079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd4158df0 0x7f3bd41edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd41f4630 0x7f3bd41f8580 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 --2- 172.21.15.44:0/676246958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bbc004800 0x7f3bd4155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 >> 172.21.15.44:0/676246958 conn(0x7f3bd40b0eb0 msgr2=0x7f3bd415b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:22.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.318+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 shutdown_connections 2024-07-17T08:35:22.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.319+0000 7f3bdc800640 1 -- 172.21.15.44:0/676246958 wait complete. 2024-07-17T08:35:22.353 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:22.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.525+0000 7f4c29000640 1 -- 172.21.15.44:0/1109828882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c2406f710 msgr2=0x7f4c24153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.525+0000 7f4c29000640 1 --2- 172.21.15.44:0/1109828882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c2406f710 0x7f4c24153a50 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f4c240d0340 tx=0x7f4c1402a540 comp rx=0 tx=0).stop 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 -- 172.21.15.44:0/1109828882 shutdown_connections 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 --2- 172.21.15.44:0/1109828882 >> v2:172.21.15.175:3300/0 conn(0x7f4c240ac870 0x7f4c2413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 --2- 172.21.15.44:0/1109828882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c2406f710 0x7f4c24153a50 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 -- 172.21.15.44:0/1109828882 >> 172.21.15.44:0/1109828882 conn(0x7f4c240b0fb0 msgr2=0x7f4c240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 -- 172.21.15.44:0/1109828882 shutdown_connections 2024-07-17T08:35:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.526+0000 7f4c29000640 1 -- 172.21.15.44:0/1109828882 wait complete. 2024-07-17T08:35:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.527+0000 7f4c29000640 1 Processor -- start 2024-07-17T08:35:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.528+0000 7f4c29000640 1 -- start start 2024-07-17T08:35:22.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.528+0000 7f4c29000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 0x7f4c24154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.528+0000 7f4c29000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c29000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c240ac870 0x7f4c241f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c29000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240ae980 con 0x7f4c240ac3e0 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c29000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240ae800 con 0x7f4c240ac870 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c29000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2413f9b0 con 0x7f4c2406f710 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59970/0 (socket says 172.21.15.44:59970) 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 -- 172.21.15.44:0/2597076944 learned_addr learned my addr 172.21.15.44:0/2597076944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c21e00640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 0x7f4c24154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 -- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 msgr2=0x7f4c24154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 0x7f4c24154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 -- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c240ac870 msgr2=0x7f4c241f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c240ac870 0x7f4c241f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c19400640 1 -- 172.21.15.44:0/2597076944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c241f4a50 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.529+0000 7f4c21e00640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 0x7f4c24154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c19400640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c760 tx=0x7f4c0c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c017070 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c0c005180 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241f4d30 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00a640 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.530+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c24156830 con 0x7f4c240ac3e0 2024-07-17T08:35:22.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.531+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24153a50 con 0x7f4c240ac3e0 2024-07-17T08:35:22.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.533+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f4c0c01a070 con 0x7f4c240ac3e0 2024-07-17T08:35:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.534+0000 7f4c1aa00640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 0x7f4c000791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.534+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 6677+0+0 (secure 0 0 0) 0x7f4c0c097c50 con 0x7f4c240ac3e0 2024-07-17T08:35:22.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.534+0000 7f4c21e00640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 0x7f4c000791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.535+0000 7f4c21e00640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 0x7f4c000791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c14004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:22.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.538+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4c2415acc0 con 0x7f4c240ac3e0 2024-07-17T08:35:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.748+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c24156a00 con 0x7f4c240ac3e0 2024-07-17T08:35:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.750+0000 7f4c1aa00640 1 -- 172.21.15.44:0/2597076944 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) ==== 74+0+17830 (secure 0 0 0) 0x7f4c0c067e20 con 0x7f4c240ac3e0 2024-07-17T08:35:22.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:22.753 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":222,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:21.767850+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 msgr2=0x7f4c000791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 0x7f4c000791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c14004730 comp rx=0 tx=0).stop 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 msgr2=0x7f4c24156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c760 tx=0x7f4c0c00cc20 comp rx=0 tx=0).stop 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 shutdown_connections 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4c00076d00 0x7f4c000791c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c240ac870 0x7f4c241f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c240ac3e0 0x7f4c24156d50 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 --2- 172.21.15.44:0/2597076944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c2406f710 0x7f4c24154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.753+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 >> 172.21.15.44:0/2597076944 conn(0x7f4c240b0fb0 msgr2=0x7f4c24157720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.754+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 shutdown_connections 2024-07-17T08:35:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.754+0000 7f4c29000640 1 -- 172.21.15.44:0/2597076944 wait complete. 2024-07-17T08:35:22.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:22.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.977+0000 7f027fe00640 1 -- 172.21.15.44:0/3441981773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028009ca20 msgr2=0x7f02800cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.977+0000 7f027fe00640 1 --2- 172.21.15.44:0/3441981773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028009ca20 0x7f02800cf2d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f026c005fd0 tx=0x7f026c02a540 comp rx=0 tx=0).stop 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.977+0000 7f027fe00640 1 -- 172.21.15.44:0/3441981773 shutdown_connections 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.977+0000 7f027fe00640 1 --2- 172.21.15.44:0/3441981773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028009ca20 0x7f02800cf2d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.977+0000 7f027fe00640 1 -- 172.21.15.44:0/3441981773 >> 172.21.15.44:0/3441981773 conn(0x7f02800b0e80 msgr2=0x7f02800b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.978+0000 7f027fe00640 1 -- 172.21.15.44:0/3441981773 shutdown_connections 2024-07-17T08:35:22.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.978+0000 7f027fe00640 1 -- 172.21.15.44:0/3441981773 wait complete. 2024-07-17T08:35:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.979+0000 7f027fe00640 1 Processor -- start 2024-07-17T08:35:22.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.979+0000 7f027fe00640 1 -- start start 2024-07-17T08:35:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f028009ca20 0x7f02801f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02800cf810 0x7f028015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02800ad080 con 0x7f028015d240 2024-07-17T08:35:22.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02800acf00 con 0x7f028009ca20 2024-07-17T08:35:22.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02800ad200 con 0x7f02800cf810 2024-07-17T08:35:22.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.980+0000 7f027f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59986/0 (socket says 172.21.15.44:59986) 2024-07-17T08:35:22.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 -- 172.21.15.44:0/1143007072 learned_addr learned my addr 172.21.15.44:0/1143007072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:22.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027ea00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f028009ca20 0x7f02801f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027e000640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02800cf810 0x7f028015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 -- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02800cf810 msgr2=0x7f028015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02800cf810 0x7f028015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 -- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f028009ca20 msgr2=0x7f02801f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f028009ca20 0x7f02801f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 -- 172.21.15.44:0/1143007072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028015dd30 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f027f400640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f027800c7a0 tx=0x7f027800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.981+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278003b70 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.982+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0278003d10 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.982+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027800aab0 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.982+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02800ce120 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.982+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0280071610 con 0x7f028015d240 2024-07-17T08:35:22.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.983+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02800ad6f0 con 0x7f028015d240 2024-07-17T08:35:22.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.985+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f027800ac50 con 0x7f028015d240 2024-07-17T08:35:22.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.986+0000 7f0276a00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 0x7f02600791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:22.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.986+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 6677+0+0 (secure 0 0 0) 0x7f0278099200 con 0x7f028015d240 2024-07-17T08:35:22.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.987+0000 7f027ea00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 0x7f02600791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:22.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.988+0000 7f027ea00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 0x7f02600791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f027000a750 tx=0x7f0270008040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:22.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:22.989+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f02800ce120 con 0x7f028015d240 2024-07-17T08:35:23.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.200+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f028015a4b0 con 0x7f028015d240 2024-07-17T08:35:23.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.203+0000 7f0276a00640 1 -- 172.21.15.44:0/1143007072 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) ==== 74+0+17830 (secure 0 0 0) 0x7f02800cfc20 con 0x7f028015d240 2024-07-17T08:35:23.205 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:23.205 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":222,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:21.767850+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 msgr2=0x7f02600791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 0x7f02600791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f027000a750 tx=0x7f0270008040 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 msgr2=0x7f0280156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f027800c7a0 tx=0x7f027800cc60 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 shutdown_connections 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f028015d240 0x7f0280156960 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02800cf810 0x7f028015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f0260076d00 0x7f02600791c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 --2- 172.21.15.44:0/1143007072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f028009ca20 0x7f02801f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 >> 172.21.15.44:0/1143007072 conn(0x7f02800b0e80 msgr2=0x7f0280157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.206+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 shutdown_connections 2024-07-17T08:35:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.207+0000 7f027fe00640 1 -- 172.21.15.44:0/1143007072 wait complete. 2024-07-17T08:35:23.227 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:35:23.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.439+0000 7f2f0b200640 1 -- 172.21.15.44:0/1863140825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 msgr2=0x7f2f0c0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.439+0000 7f2f0b200640 1 --2- 172.21.15.44:0/1863140825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c0b5a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8005fd0 tx=0x7f2ef802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 -- 172.21.15.44:0/1863140825 shutdown_connections 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 --2- 172.21.15.44:0/1863140825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c0b5a70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 --2- 172.21.15.44:0/1863140825 >> v2:172.21.15.175:3300/0 conn(0x7f2f0c15b030 0x7f2f0c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 -- 172.21.15.44:0/1863140825 >> 172.21.15.44:0/1863140825 conn(0x7f2f0c0b3090 msgr2=0x7f2f0c0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 -- 172.21.15.44:0/1863140825 shutdown_connections 2024-07-17T08:35:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.440+0000 7f2f0b200640 1 -- 172.21.15.44:0/1863140825 wait complete. 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.442+0000 7f2f0b200640 1 Processor -- start 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.442+0000 7f2f0b200640 1 -- start start 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15b030 0x7f2f0c146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c0b3d90 con 0x7f2f0c15eb60 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c074720 con 0x7f2f0c15b030 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.443+0000 7f2f0b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0c0b3f10 con 0x7f2f0c0b09a0 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59992/0 (socket says 172.21.15.44:59992) 2024-07-17T08:35:23.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 -- 172.21.15.44:0/2366589380 learned_addr learned my addr 172.21.15.44:0/2366589380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f09400640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15b030 0x7f2f0c146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f09e00640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 -- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 msgr2=0x7f2f0c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c1499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 -- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15b030 msgr2=0x7f2f0c146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15b030 0x7f2f0c146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 -- 172.21.15.44:0/2366589380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c146b60 con 0x7f2f0c15eb60 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.444+0000 7f2f0a800640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b970 tx=0x7f2f0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.445+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04019070 con 0x7f2f0c15eb60 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.445+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f0400cc30 con 0x7f2f0c15eb60 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.445+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040146a0 con 0x7f2f0c15eb60 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.445+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c157870 con 0x7f2f0c15eb60 2024-07-17T08:35:23.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.445+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c148ee0 con 0x7f2f0c15eb60 2024-07-17T08:35:23.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.446+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c0ce3d0 con 0x7f2f0c15eb60 2024-07-17T08:35:23.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.448+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f2f0401c070 con 0x7f2f0c15eb60 2024-07-17T08:35:23.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.450+0000 7f2f02000640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 0x7f2eec0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:23.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.450+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f2f040988b0 con 0x7f2f0c15eb60 2024-07-17T08:35:23.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.450+0000 7f2f09e00640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 0x7f2eec0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:23.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.451+0000 7f2f09e00640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 0x7f2eec0791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009b60 tx=0x7f2efc005f90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:23.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.453+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2f0c0ce3d0 con 0x7f2f0c15eb60 2024-07-17T08:35:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:23.705+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2f0c0cd8c0 con 0x7f2f0c15eb60 2024-07-17T08:35:24.253 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:35:24.251+0000 7f788ac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:35:24.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.329+0000 7f2f02000640 1 -- 172.21.15.44:0/2366589380 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v113) ==== 118+0+0 (secure 0 0 0) 0x7f2f04061880 con 0x7f2f0c15eb60 2024-07-17T08:35:24.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.332+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 msgr2=0x7f2eec0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:24.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.332+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 0x7f2eec0791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009b60 tx=0x7f2efc005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.332+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 msgr2=0x7f2f0c1f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.332+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b970 tx=0x7f2f0400be30 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 shutdown_connections 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f0c15eb60 0x7f2f0c1f7d10 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2eec076d00 0x7f2eec0791c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f0c15b030 0x7f2f0c146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 --2- 172.21.15.44:0/2366589380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f0c0b09a0 0x7f2f0c1499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 >> 172.21.15.44:0/2366589380 conn(0x7f2f0c0b3090 msgr2=0x7f2f0c0b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 shutdown_connections 2024-07-17T08:35:24.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.333+0000 7f2f0b200640 1 -- 172.21.15.44:0/2366589380 wait complete. 2024-07-17T08:35:24.358 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:35:24.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:35:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 -- 172.21.15.44:0/1126628670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a4161be0 msgr2=0x7fc5a415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1126628670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a4161be0 0x7fc5a415d510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc590009830 tx=0x7fc59002a480 comp rx=0 tx=0).stop 2024-07-17T08:35:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 -- 172.21.15.44:0/1126628670 shutdown_connections 2024-07-17T08:35:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1126628670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a40ce690 0x7fc5a40cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1126628670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a4161be0 0x7fc5a415d510 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 -- 172.21.15.44:0/1126628670 >> 172.21.15.44:0/1126628670 conn(0x7fc5a40b4ff0 msgr2=0x7fc5a40b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 -- 172.21.15.44:0/1126628670 shutdown_connections 2024-07-17T08:35:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.576+0000 7fc5aa200640 1 -- 172.21.15.44:0/1126628670 wait complete. 2024-07-17T08:35:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.578+0000 7fc5aa200640 1 Processor -- start 2024-07-17T08:35:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.578+0000 7fc5aa200640 1 -- start start 2024-07-17T08:35:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 0x7fc5a415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a415d130 0x7fc5a4150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d1a60 con 0x7fc5a4161be0 2024-07-17T08:35:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d18e0 con 0x7fc5a40ce690 2024-07-17T08:35:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.579+0000 7fc5aa200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40d1be0 con 0x7fc5a415d130 2024-07-17T08:35:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59998/0 (socket says 172.21.15.44:59998) 2024-07-17T08:35:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 -- 172.21.15.44:0/1933324794 learned_addr learned my addr 172.21.15.44:0/1933324794 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a3e00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a415d130 0x7fc5a4150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a8e00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 0x7fc5a415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 -- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a415d130 msgr2=0x7fc5a4150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a415d130 0x7fc5a4150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 -- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 msgr2=0x7fc5a415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 0x7fc5a415bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 -- 172.21.15.44:0/1933324794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40b2420 con 0x7fc5a4161be0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a8e00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 0x7fc5a415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.580+0000 7fc5a9800640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00ea80 tx=0x7fc59c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.581+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c004b20 con 0x7fc5a4161be0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.581+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59c004ce0 con 0x7fc5a4161be0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.581+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005280 con 0x7fc5a4161be0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.581+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a40aa7b0 con 0x7fc5a4161be0 2024-07-17T08:35:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.581+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a40aad70 con 0x7fc5a4161be0 2024-07-17T08:35:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.582+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a40cfdf0 con 0x7fc5a4161be0 2024-07-17T08:35:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.583+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fc59c005420 con 0x7fc5a4161be0 2024-07-17T08:35:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.585+0000 7fc5a0c00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 0x7fc5840791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.585+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7fc59c0994d0 con 0x7fc5a4161be0 2024-07-17T08:35:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.585+0000 7fc5a8e00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 0x7fc5840791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.586+0000 7fc5a8e00640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 0x7fc5840791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc598000f60 tx=0x7fc598009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.589+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc59c0623a0 con 0x7fc5a4161be0 2024-07-17T08:35:24.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:24.838+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc5a41608e0 con 0x7fc5a4161be0 2024-07-17T08:35:25.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.340+0000 7fc5a0c00640 1 -- 172.21.15.44:0/1933324794 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v114) ==== 122+0+0 (secure 0 0 0) 0x7fc59c061ca0 con 0x7fc5a4161be0 2024-07-17T08:35:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 msgr2=0x7fc5840791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 0x7fc5840791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc598000f60 tx=0x7fc598009040 comp rx=0 tx=0).stop 2024-07-17T08:35:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 msgr2=0x7fc5a40b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00ea80 tx=0x7fc59c00ef40 comp rx=0 tx=0).stop 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 shutdown_connections 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a4161be0 0x7fc5a40b1ee0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a415d130 0x7fc5a4150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc584076d00 0x7fc5840791c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 --2- 172.21.15.44:0/1933324794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a40ce690 0x7fc5a415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 >> 172.21.15.44:0/1933324794 conn(0x7fc5a40b4ff0 msgr2=0x7fc5a40b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 shutdown_connections 2024-07-17T08:35:25.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.344+0000 7fc5aa200640 1 -- 172.21.15.44:0/1933324794 wait complete. 2024-07-17T08:35:25.368 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:35:25.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.581+0000 7f124b400640 1 -- 172.21.15.44:0/2300294124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 msgr2=0x7f12440cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.581+0000 7f124b400640 1 --2- 172.21.15.44:0/2300294124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f12440cfe30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1230005fd0 tx=0x7f123002a4a0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 -- 172.21.15.44:0/2300294124 shutdown_connections 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 --2- 172.21.15.44:0/2300294124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12440cd610 0x7f12440d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 --2- 172.21.15.44:0/2300294124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f12440cfe30 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 -- 172.21.15.44:0/2300294124 >> 172.21.15.44:0/2300294124 conn(0x7f12440b3120 msgr2=0x7f12440b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 -- 172.21.15.44:0/2300294124 shutdown_connections 2024-07-17T08:35:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.582+0000 7f124b400640 1 -- 172.21.15.44:0/2300294124 wait complete. 2024-07-17T08:35:25.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.584+0000 7f124b400640 1 Processor -- start 2024-07-17T08:35:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.584+0000 7f124b400640 1 -- start start 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124406fec0 0x7f124414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12440cd610 0x7f1244158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12440d0a30 con 0x7f12440ce530 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12440d08b0 con 0x7f12440cd610 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.585+0000 7f124b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12440d0bb0 con 0x7f124406fec0 2024-07-17T08:35:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60008/0 (socket says 172.21.15.44:60008) 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 -- 172.21.15.44:0/2951247875 learned_addr learned my addr 172.21.15.44:0/2951247875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1243e00640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12440cd610 0x7f1244158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1248a00640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124406fec0 0x7f124414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 -- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124406fec0 msgr2=0x7f124414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124406fec0 0x7f124414ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 -- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12440cd610 msgr2=0x7f1244158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12440cd610 0x7f1244158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.586+0000 7f1249400640 1 -- 172.21.15.44:0/2951247875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1244156b70 con 0x7f12440ce530 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f1249400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f123c00eaa0 tx=0x7f123c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c00cdb0 con 0x7f12440ce530 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f123c004c10 con 0x7f12440ce530 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1244150910 con 0x7f12440ce530 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c005020 con 0x7f12440ce530 2024-07-17T08:35:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.587+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1244159900 con 0x7f12440ce530 2024-07-17T08:35:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.588+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12441576e0 con 0x7f12440ce530 2024-07-17T08:35:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.590+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f123c0051c0 con 0x7f12440ce530 2024-07-17T08:35:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.591+0000 7f1240c00640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 0x7f1224079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:25.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.591+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f123c099220 con 0x7f12440ce530 2024-07-17T08:35:25.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.592+0000 7f1248a00640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 0x7f1224079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.593+0000 7f1248a00640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 0x7f1224079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234008040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:25.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.596+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f12441576e0 con 0x7f12440ce530 2024-07-17T08:35:25.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.843+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12440702a0 con 0x7f12440ce530 2024-07-17T08:35:25.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.844+0000 7f1240c00640 1 -- 172.21.15.44:0/2951247875 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5235 (secure 0 0 0) 0x7f123c016030 con 0x7f12440ce530 2024-07-17T08:35:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:25.846 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":114,"btime":"2024-07-17T08:35:25:338230+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":8329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13}]} 2024-07-17T08:35:25.846 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 114 2024-07-17T08:35:25.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.846+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 msgr2=0x7f1224079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.846+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 0x7f1224079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234008040 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.846+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 msgr2=0x7f1244159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.846+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f123c00eaa0 tx=0x7f123c00ef60 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 shutdown_connections 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12440ce530 0x7f1244159080 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f1224076cd0 0x7f1224079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12440cd610 0x7f1244158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 --2- 172.21.15.44:0/2951247875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124406fec0 0x7f124414ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 >> 172.21.15.44:0/2951247875 conn(0x7f12440b3120 msgr2=0x7f12440cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 shutdown_connections 2024-07-17T08:35:25.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:25.847+0000 7f124b400640 1 -- 172.21.15.44:0/2951247875 wait complete. 2024-07-17T08:35:25.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 -- 172.21.15.44:0/3394321924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 msgr2=0x7fc604004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 --2- 172.21.15.44:0/3394321924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc604004c80 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc608009b20 tx=0x7fc604005830 comp rx=0 tx=0).stop 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc621600640 1 -- 172.21.15.44:0/3394321924 reap_dead start 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 -- 172.21.15.44:0/3394321924 shutdown_connections 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 -- 172.21.15.44:0/3394321924 >> 172.21.15.44:0/3394321924 conn(0x7fc61c0b0e80 msgr2=0x7fc61c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:26.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 -- 172.21.15.44:0/3394321924 shutdown_connections 2024-07-17T08:35:26.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.074+0000 7fc622000640 1 -- 172.21.15.44:0/3394321924 wait complete. 2024-07-17T08:35:26.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.076+0000 7fc622000640 1 Processor -- start 2024-07-17T08:35:26.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.076+0000 7fc622000640 1 -- start start 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 0x7fc61c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc61c158ad0 0x7fc61c1f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0ac1e0 con 0x7fc604004800 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0ac060 con 0x7fc61c0739f0 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc622000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c140a00 con 0x7fc61c158ad0 2024-07-17T08:35:26.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc620c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc620c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60028/0 (socket says 172.21.15.44:60028) 2024-07-17T08:35:26.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.077+0000 7fc620c00640 1 -- 172.21.15.44:0/4140348337 learned_addr learned my addr 172.21.15.44:0/4140348337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:26.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc61be00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 0x7fc61c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc621600640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc61c158ad0 0x7fc61c1f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 -- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc61c158ad0 msgr2=0x7fc61c1f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc61c158ad0 0x7fc61c1f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 -- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 msgr2=0x7fc61c158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 0x7fc61c158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 -- 172.21.15.44:0/4140348337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c1565a0 con 0x7fc604004800 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc61be00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 0x7fc61c158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc620c00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc61000e5c0 tx=0x7fc61000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610015070 con 0x7fc604004800 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc61000ca00 con 0x7fc604004800 2024-07-17T08:35:26.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c156880 con 0x7fc604004800 2024-07-17T08:35:26.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.078+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61000ce50 con 0x7fc604004800 2024-07-17T08:35:26.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.079+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c156e40 con 0x7fc604004800 2024-07-17T08:35:26.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.080+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c140030 con 0x7fc604004800 2024-07-17T08:35:26.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.081+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fc610018070 con 0x7fc604004800 2024-07-17T08:35:26.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.082+0000 7fc618c00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 0x7fc5fc079360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.082+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7fc610098ce0 con 0x7fc604004800 2024-07-17T08:35:26.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.083+0000 7fc61be00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 0x7fc5fc079360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.084+0000 7fc61be00640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 0x7fc5fc079360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc608035000 tx=0x7fc60802d040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:26.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.087+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc610061b20 con 0x7fc604004800 2024-07-17T08:35:26.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.300+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc61c09b4e0 con 0x7fc604004800 2024-07-17T08:35:26.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.304+0000 7fc618c00640 1 -- 172.21.15.44:0/4140348337 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) ==== 74+0+17817 (secure 0 0 0) 0x7fc610061420 con 0x7fc604004800 2024-07-17T08:35:26.306 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":223,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:23.301904+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:26.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 msgr2=0x7fc5fc079360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 0x7fc5fc079360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc608035000 tx=0x7fc60802d040 comp rx=0 tx=0).stop 2024-07-17T08:35:26.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 msgr2=0x7fc61c15a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc61000e5c0 tx=0x7fc61000ea80 comp rx=0 tx=0).stop 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 shutdown_connections 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.306+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc61c158ad0 0x7fc61c1f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc5fc076ea0 0x7fc5fc079360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc61c0739f0 0x7fc61c158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 --2- 172.21.15.44:0/4140348337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc604004800 0x7fc61c15a6d0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 >> 172.21.15.44:0/4140348337 conn(0x7fc61c0b0e80 msgr2=0x7fc61c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 shutdown_connections 2024-07-17T08:35:26.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.307+0000 7fc622000640 1 -- 172.21.15.44:0/4140348337 wait complete. 2024-07-17T08:35:26.328 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:35:26.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 -- 172.21.15.44:0/2715540054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29480adcd0 msgr2=0x7f29480ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 --2- 172.21.15.44:0/2715540054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29480adcd0 0x7f29480ac220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 -- 172.21.15.44:0/2715540054 shutdown_connections 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 --2- 172.21.15.44:0/2715540054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29480adcd0 0x7f29480ac220 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 --2- 172.21.15.44:0/2715540054 >> v2:172.21.15.175:3300/0 conn(0x7f29481522c0 0x7f29480ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 -- 172.21.15.44:0/2715540054 >> 172.21.15.44:0/2715540054 conn(0x7f29480b0fb0 msgr2=0x7f29480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 -- 172.21.15.44:0/2715540054 shutdown_connections 2024-07-17T08:35:26.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.533+0000 7f294dc00640 1 -- 172.21.15.44:0/2715540054 wait complete. 2024-07-17T08:35:26.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.535+0000 7f294dc00640 1 Processor -- start 2024-07-17T08:35:26.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.535+0000 7f294dc00640 1 -- start start 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 0x7f29481476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29481522c0 0x7f29480b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f294815f5a0 con 0x7f29480ac760 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29480ce8b0 con 0x7f29480adcd0 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f294dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f294815f720 con 0x7f29481522c0 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f2946a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f2946a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60036/0 (socket says 172.21.15.44:60036) 2024-07-17T08:35:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f2946a00640 1 -- 172.21.15.44:0/1890074896 learned_addr learned my addr 172.21.15.44:0/1890074896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.536+0000 7f2947400640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29481522c0 0x7f29480b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946000640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 0x7f29481476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 -- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29481522c0 msgr2=0x7f29480b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29481522c0 0x7f29480b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 -- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 msgr2=0x7f29481476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 0x7f29481476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 -- 172.21.15.44:0/1890074896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2948147c00 con 0x7f29480ac760 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946000640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 0x7f29481476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f2946a00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f293800ecb0 tx=0x7f293800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938003b70 con 0x7f29480ac760 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2938003d10 con 0x7f29480ac760 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938004280 con 0x7f29480ac760 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2948149f80 con 0x7f29480ac760 2024-07-17T08:35:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.537+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294814a540 con 0x7f29480ac760 2024-07-17T08:35:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.539+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294815fc40 con 0x7f29480ac760 2024-07-17T08:35:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.540+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f2938004420 con 0x7f29480ac760 2024-07-17T08:35:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.541+0000 7f293ea00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 0x7f2924079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.541+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f29380995e0 con 0x7f29480ac760 2024-07-17T08:35:26.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.541+0000 7f2946000640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 0x7f2924079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:26.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.542+0000 7f2946000640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 0x7f2924079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2930033000 tx=0x7f2930004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.547+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f29380624b0 con 0x7f29480ac760 2024-07-17T08:35:26.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.758+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f294806e130 con 0x7f2924076cd0 2024-07-17T08:35:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.772+0000 7f293ea00640 1 -- 172.21.15.44:0/1890074896 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7f294806e130 con 0x7f2924076cd0 2024-07-17T08:35:26.778 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 msgr2=0x7f2924079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 0x7f2924079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2930033000 tx=0x7f2930004730 comp rx=0 tx=0).stop 2024-07-17T08:35:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 msgr2=0x7f2948073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f293800ecb0 tx=0x7f293800c6a0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 shutdown_connections 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29481522c0 0x7f29480b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2924076cd0 0x7f2924079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29480adcd0 0x7f29481476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 --2- 172.21.15.44:0/1890074896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29480ac760 0x7f2948073080 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 >> 172.21.15.44:0/1890074896 conn(0x7f29480b0fb0 msgr2=0x7f29480aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 shutdown_connections 2024-07-17T08:35:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:26.779+0000 7f294dc00640 1 -- 172.21.15.44:0/1890074896 wait complete. 2024-07-17T08:35:26.798 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-07-17T08:35:25.761046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":404,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2550,"ondisk_log_size":2550,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":415432,"kb_used_data":22976,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754559288,"statfs":{"total":773094113280,"available":772668710912,"internally_reserved":0,"allocated":23527424,"data_stored":13296892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.497899"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339517+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:22.339517+0000","last_peered":"2024-07-17T08:35:22.339517+0000","last_clean":"2024-07-17T08:35:22.339517+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:22.339517+0000","last_undegraded":"2024-07-17T08:35:22.339517+0000","last_fullsized":"2024-07-17T08:35:22.339517+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:07.170768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.313966+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:22.313966+0000","last_peered":"2024-07-17T08:35:22.313966+0000","last_clean":"2024-07-17T08:35:22.313966+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:22.313966+0000","last_undegraded":"2024-07-17T08:35:22.313966+0000","last_fullsized":"2024-07-17T08:35:22.313966+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326699+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:23.326699+0000","last_peered":"2024-07-17T08:35:23.326699+0000","last_clean":"2024-07-17T08:35:23.326699+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:23.326699+0000","last_undegraded":"2024-07-17T08:35:23.326699+0000","last_fullsized":"2024-07-17T08:35:23.326699+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332815+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:22.332815+0000","last_peered":"2024-07-17T08:35:22.332815+0000","last_clean":"2024-07-17T08:35:22.332815+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:22.332815+0000","last_undegraded":"2024-07-17T08:35:22.332815+0000","last_fullsized":"2024-07-17T08:35:22.332815+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:45.612904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317917+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:23.317917+0000","last_peered":"2024-07-17T08:35:23.317917+0000","last_clean":"2024-07-17T08:35:23.317917+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:23.317917+0000","last_undegraded":"2024-07-17T08:35:23.317917+0000","last_fullsized":"2024-07-17T08:35:23.317917+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.325243+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:22.325243+0000","last_peered":"2024-07-17T08:35:22.325243+0000","last_clean":"2024-07-17T08:35:22.325243+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:22.325243+0000","last_undegraded":"2024-07-17T08:35:22.325243+0000","last_fullsized":"2024-07-17T08:35:22.325243+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:17.385054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317048+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:23.317048+0000","last_peered":"2024-07-17T08:35:23.317048+0000","last_clean":"2024-07-17T08:35:23.317048+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:23.317048+0000","last_undegraded":"2024-07-17T08:35:23.317048+0000","last_fullsized":"2024-07-17T08:35:23.317048+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320757+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:23.320757+0000","last_peered":"2024-07-17T08:35:23.320757+0000","last_clean":"2024-07-17T08:35:23.320757+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:23.320757+0000","last_undegraded":"2024-07-17T08:35:23.320757+0000","last_fullsized":"2024-07-17T08:35:23.320757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319829+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:23.319829+0000","last_peered":"2024-07-17T08:35:23.319829+0000","last_clean":"2024-07-17T08:35:23.319829+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:23.319829+0000","last_undegraded":"2024-07-17T08:35:23.319829+0000","last_fullsized":"2024-07-17T08:35:23.319829+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322580+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:23.322580+0000","last_peered":"2024-07-17T08:35:23.322580+0000","last_clean":"2024-07-17T08:35:23.322580+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:23.322580+0000","last_undegraded":"2024-07-17T08:35:23.322580+0000","last_fullsized":"2024-07-17T08:35:23.322580+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326844+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:23.326844+0000","last_peered":"2024-07-17T08:35:23.326844+0000","last_clean":"2024-07-17T08:35:23.326844+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:23.326844+0000","last_undegraded":"2024-07-17T08:35:23.326844+0000","last_fullsized":"2024-07-17T08:35:23.326844+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317307+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:23.317307+0000","last_peered":"2024-07-17T08:35:23.317307+0000","last_clean":"2024-07-17T08:35:23.317307+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:23.317307+0000","last_undegraded":"2024-07-17T08:35:23.317307+0000","last_fullsized":"2024-07-17T08:35:23.317307+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332706+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:22.332706+0000","last_peered":"2024-07-17T08:35:22.332706+0000","last_clean":"2024-07-17T08:35:22.332706+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:22.332706+0000","last_undegraded":"2024-07-17T08:35:22.332706+0000","last_fullsized":"2024-07-17T08:35:22.332706+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:43.220621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320816+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:23.320816+0000","last_peered":"2024-07-17T08:35:23.320816+0000","last_clean":"2024-07-17T08:35:23.320816+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:23.320816+0000","last_undegraded":"2024-07-17T08:35:23.320816+0000","last_fullsized":"2024-07-17T08:35:23.320816+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332375+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:22.332375+0000","last_peered":"2024-07-17T08:35:22.332375+0000","last_clean":"2024-07-17T08:35:22.332375+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:22.332375+0000","last_undegraded":"2024-07-17T08:35:22.332375+0000","last_fullsized":"2024-07-17T08:35:22.332375+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:47.900589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326124+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:23.326124+0000","last_peered":"2024-07-17T08:35:23.326124+0000","last_clean":"2024-07-17T08:35:23.326124+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:23.326124+0000","last_undegraded":"2024-07-17T08:35:23.326124+0000","last_fullsized":"2024-07-17T08:35:23.326124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320251+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:23.320251+0000","last_peered":"2024-07-17T08:35:23.320251+0000","last_clean":"2024-07-17T08:35:23.320251+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:23.320251+0000","last_undegraded":"2024-07-17T08:35:23.320251+0000","last_fullsized":"2024-07-17T08:35:23.320251+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319506+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:23.319506+0000","last_peered":"2024-07-17T08:35:23.319506+0000","last_clean":"2024-07-17T08:35:23.319506+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:23.319506+0000","last_undegraded":"2024-07-17T08:35:23.319506+0000","last_fullsized":"2024-07-17T08:35:23.319506+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322345+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:23.322345+0000","last_peered":"2024-07-17T08:35:23.322345+0000","last_clean":"2024-07-17T08:35:23.322345+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:23.322345+0000","last_undegraded":"2024-07-17T08:35:23.322345+0000","last_fullsized":"2024-07-17T08:35:23.322345+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322122+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:23.322122+0000","last_peered":"2024-07-17T08:35:23.322122+0000","last_clean":"2024-07-17T08:35:23.322122+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:23.322122+0000","last_undegraded":"2024-07-17T08:35:23.322122+0000","last_fullsized":"2024-07-17T08:35:23.322122+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321838+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:23.321838+0000","last_peered":"2024-07-17T08:35:23.321838+0000","last_clean":"2024-07-17T08:35:23.321838+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:23.321838+0000","last_undegraded":"2024-07-17T08:35:23.321838+0000","last_fullsized":"2024-07-17T08:35:23.321838+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321999+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:23.321999+0000","last_peered":"2024-07-17T08:35:23.321999+0000","last_clean":"2024-07-17T08:35:23.321999+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:23.321999+0000","last_undegraded":"2024-07-17T08:35:23.321999+0000","last_fullsized":"2024-07-17T08:35:23.321999+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.333880+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:22.333880+0000","last_peered":"2024-07-17T08:35:22.333880+0000","last_clean":"2024-07-17T08:35:22.333880+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:22.333880+0000","last_undegraded":"2024-07-17T08:35:22.333880+0000","last_fullsized":"2024-07-17T08:35:22.333880+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:40.484299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316881+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:23.316881+0000","last_peered":"2024-07-17T08:35:23.316881+0000","last_clean":"2024-07-17T08:35:23.316881+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:23.316881+0000","last_undegraded":"2024-07-17T08:35:23.316881+0000","last_fullsized":"2024-07-17T08:35:23.316881+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320212+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:23.320212+0000","last_peered":"2024-07-17T08:35:23.320212+0000","last_clean":"2024-07-17T08:35:23.320212+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:23.320212+0000","last_undegraded":"2024-07-17T08:35:23.320212+0000","last_fullsized":"2024-07-17T08:35:23.320212+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336680+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:22.336680+0000","last_peered":"2024-07-17T08:35:22.336680+0000","last_clean":"2024-07-17T08:35:22.336680+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:22.336680+0000","last_undegraded":"2024-07-17T08:35:22.336680+0000","last_fullsized":"2024-07-17T08:35:22.336680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:26.810568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326818+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:23.326818+0000","last_peered":"2024-07-17T08:35:23.326818+0000","last_clean":"2024-07-17T08:35:23.326818+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:23.326818+0000","last_undegraded":"2024-07-17T08:35:23.326818+0000","last_fullsized":"2024-07-17T08:35:23.326818+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317375+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:23.317375+0000","last_peered":"2024-07-17T08:35:23.317375+0000","last_clean":"2024-07-17T08:35:23.317375+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:23.317375+0000","last_undegraded":"2024-07-17T08:35:23.317375+0000","last_fullsized":"2024-07-17T08:35:23.317375+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319032+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:23.319032+0000","last_peered":"2024-07-17T08:35:23.319032+0000","last_clean":"2024-07-17T08:35:23.319032+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:23.319032+0000","last_undegraded":"2024-07-17T08:35:23.319032+0000","last_fullsized":"2024-07-17T08:35:23.319032+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315473+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:22.315473+0000","last_peered":"2024-07-17T08:35:22.315473+0000","last_clean":"2024-07-17T08:35:22.315473+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:22.315473+0000","last_undegraded":"2024-07-17T08:35:22.315473+0000","last_fullsized":"2024-07-17T08:35:22.315473+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:34.228783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332902+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:22.332902+0000","last_peered":"2024-07-17T08:35:22.332902+0000","last_clean":"2024-07-17T08:35:22.332902+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:22.332902+0000","last_undegraded":"2024-07-17T08:35:22.332902+0000","last_fullsized":"2024-07-17T08:35:22.332902+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:45.694196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.319697+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:22.319697+0000","last_peered":"2024-07-17T08:35:22.319697+0000","last_clean":"2024-07-17T08:35:22.319697+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:22.319697+0000","last_undegraded":"2024-07-17T08:35:22.319697+0000","last_fullsized":"2024-07-17T08:35:22.319697+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:58:40.977604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.331426+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:22.331426+0000","last_peered":"2024-07-17T08:35:22.331426+0000","last_clean":"2024-07-17T08:35:22.331426+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:22.331426+0000","last_undegraded":"2024-07-17T08:35:22.331426+0000","last_fullsized":"2024-07-17T08:35:22.331426+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":16,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.341018+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:22.341018+0000","last_peered":"2024-07-17T08:35:22.341018+0000","last_clean":"2024-07-17T08:35:22.341018+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:22.341018+0000","last_undegraded":"2024-07-17T08:35:22.341018+0000","last_fullsized":"2024-07-17T08:35:22.341018+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.317524+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:22.317524+0000","last_peered":"2024-07-17T08:35:22.317524+0000","last_clean":"2024-07-17T08:35:22.317524+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:22.317524+0000","last_undegraded":"2024-07-17T08:35:22.317524+0000","last_fullsized":"2024-07-17T08:35:22.317524+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:48.338537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336710+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:22.336710+0000","last_peered":"2024-07-17T08:35:22.336710+0000","last_clean":"2024-07-17T08:35:22.336710+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:22.336710+0000","last_undegraded":"2024-07-17T08:35:22.336710+0000","last_fullsized":"2024-07-17T08:35:22.336710+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321880+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:23.321880+0000","last_peered":"2024-07-17T08:35:23.321880+0000","last_clean":"2024-07-17T08:35:23.321880+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:23.321880+0000","last_undegraded":"2024-07-17T08:35:23.321880+0000","last_fullsized":"2024-07-17T08:35:23.321880+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336285+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:22.336285+0000","last_peered":"2024-07-17T08:35:22.336285+0000","last_clean":"2024-07-17T08:35:22.336285+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:22.336285+0000","last_undegraded":"2024-07-17T08:35:22.336285+0000","last_fullsized":"2024-07-17T08:35:22.336285+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317968+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:23.317968+0000","last_peered":"2024-07-17T08:35:23.317968+0000","last_clean":"2024-07-17T08:35:23.317968+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:23.317968+0000","last_undegraded":"2024-07-17T08:35:23.317968+0000","last_fullsized":"2024-07-17T08:35:23.317968+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319236+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:23.319236+0000","last_peered":"2024-07-17T08:35:23.319236+0000","last_clean":"2024-07-17T08:35:23.319236+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:23.319236+0000","last_undegraded":"2024-07-17T08:35:23.319236+0000","last_fullsized":"2024-07-17T08:35:23.319236+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.312930+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:22.312930+0000","last_peered":"2024-07-17T08:35:22.312930+0000","last_clean":"2024-07-17T08:35:22.312930+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:22.312930+0000","last_undegraded":"2024-07-17T08:35:22.312930+0000","last_fullsized":"2024-07-17T08:35:22.312930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:19.775392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.338521+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:22.338521+0000","last_peered":"2024-07-17T08:35:22.338521+0000","last_clean":"2024-07-17T08:35:22.338521+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:22.338521+0000","last_undegraded":"2024-07-17T08:35:22.338521+0000","last_fullsized":"2024-07-17T08:35:22.338521+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.312997+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:22.312997+0000","last_peered":"2024-07-17T08:35:22.312997+0000","last_clean":"2024-07-17T08:35:22.312997+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:22.312997+0000","last_undegraded":"2024-07-17T08:35:22.312997+0000","last_fullsized":"2024-07-17T08:35:22.312997+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:52.676809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.317228+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:22.317228+0000","last_peered":"2024-07-17T08:35:22.317228+0000","last_clean":"2024-07-17T08:35:22.317228+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:22.317228+0000","last_undegraded":"2024-07-17T08:35:22.317228+0000","last_fullsized":"2024-07-17T08:35:22.317228+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320332+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:23.320332+0000","last_peered":"2024-07-17T08:35:23.320332+0000","last_clean":"2024-07-17T08:35:23.320332+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:23.320332+0000","last_undegraded":"2024-07-17T08:35:23.320332+0000","last_fullsized":"2024-07-17T08:35:23.320332+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326134+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:23.326134+0000","last_peered":"2024-07-17T08:35:23.326134+0000","last_clean":"2024-07-17T08:35:23.326134+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:23.326134+0000","last_undegraded":"2024-07-17T08:35:23.326134+0000","last_fullsized":"2024-07-17T08:35:23.326134+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.320389+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:22.320389+0000","last_peered":"2024-07-17T08:35:22.320389+0000","last_clean":"2024-07-17T08:35:22.320389+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:22.320389+0000","last_undegraded":"2024-07-17T08:35:22.320389+0000","last_fullsized":"2024-07-17T08:35:22.320389+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:12.044847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317528+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:23.317528+0000","last_peered":"2024-07-17T08:35:23.317528+0000","last_clean":"2024-07-17T08:35:23.317528+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:23.317528+0000","last_undegraded":"2024-07-17T08:35:23.317528+0000","last_fullsized":"2024-07-17T08:35:23.317528+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326134+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:23.326134+0000","last_peered":"2024-07-17T08:35:23.326134+0000","last_clean":"2024-07-17T08:35:23.326134+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:23.326134+0000","last_undegraded":"2024-07-17T08:35:23.326134+0000","last_fullsized":"2024-07-17T08:35:23.326134+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315342+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:22.315342+0000","last_peered":"2024-07-17T08:35:22.315342+0000","last_clean":"2024-07-17T08:35:22.315342+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:22.315342+0000","last_undegraded":"2024-07-17T08:35:22.315342+0000","last_fullsized":"2024-07-17T08:35:22.315342+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319286+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:23.319286+0000","last_peered":"2024-07-17T08:35:23.319286+0000","last_clean":"2024-07-17T08:35:23.319286+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:23.319286+0000","last_undegraded":"2024-07-17T08:35:23.319286+0000","last_fullsized":"2024-07-17T08:35:23.319286+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319631+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:23.319631+0000","last_peered":"2024-07-17T08:35:23.319631+0000","last_clean":"2024-07-17T08:35:23.319631+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:23.319631+0000","last_undegraded":"2024-07-17T08:35:23.319631+0000","last_fullsized":"2024-07-17T08:35:23.319631+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318124+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:23.318124+0000","last_peered":"2024-07-17T08:35:23.318124+0000","last_clean":"2024-07-17T08:35:23.318124+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:23.318124+0000","last_undegraded":"2024-07-17T08:35:23.318124+0000","last_fullsized":"2024-07-17T08:35:23.318124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"222'6","reported_seq":18,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.343996+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:22.343996+0000","last_peered":"2024-07-17T08:35:22.343996+0000","last_clean":"2024-07-17T08:35:22.343996+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:22.343996+0000","last_undegraded":"2024-07-17T08:35:22.343996+0000","last_fullsized":"2024-07-17T08:35:22.343996+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309577+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:22.309577+0000","last_peered":"2024-07-17T08:35:22.309577+0000","last_clean":"2024-07-17T08:35:22.309577+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:22.309577+0000","last_undegraded":"2024-07-17T08:35:22.309577+0000","last_fullsized":"2024-07-17T08:35:22.309577+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.340736+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:22.340736+0000","last_peered":"2024-07-17T08:35:22.340736+0000","last_clean":"2024-07-17T08:35:22.340736+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:22.340736+0000","last_undegraded":"2024-07-17T08:35:22.340736+0000","last_fullsized":"2024-07-17T08:35:22.340736+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326986+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:23.326986+0000","last_peered":"2024-07-17T08:35:23.326986+0000","last_clean":"2024-07-17T08:35:23.326986+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:23.326986+0000","last_undegraded":"2024-07-17T08:35:23.326986+0000","last_fullsized":"2024-07-17T08:35:23.326986+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326612+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:23.326612+0000","last_peered":"2024-07-17T08:35:23.326612+0000","last_clean":"2024-07-17T08:35:23.326612+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:23.326612+0000","last_undegraded":"2024-07-17T08:35:23.326612+0000","last_fullsized":"2024-07-17T08:35:23.326612+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319872+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:23.319872+0000","last_peered":"2024-07-17T08:35:23.319872+0000","last_clean":"2024-07-17T08:35:23.319872+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:23.319872+0000","last_undegraded":"2024-07-17T08:35:23.319872+0000","last_fullsized":"2024-07-17T08:35:23.319872+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318614+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:23.318614+0000","last_peered":"2024-07-17T08:35:23.318614+0000","last_clean":"2024-07-17T08:35:23.318614+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:23.318614+0000","last_undegraded":"2024-07-17T08:35:23.318614+0000","last_fullsized":"2024-07-17T08:35:23.318614+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332744+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:22.332744+0000","last_peered":"2024-07-17T08:35:22.332744+0000","last_clean":"2024-07-17T08:35:22.332744+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:22.332744+0000","last_undegraded":"2024-07-17T08:35:22.332744+0000","last_fullsized":"2024-07-17T08:35:22.332744+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:46.008752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":22,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316788+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:23.316788+0000","last_peered":"2024-07-17T08:35:23.316788+0000","last_clean":"2024-07-17T08:35:23.316788+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:23.316788+0000","last_undegraded":"2024-07-17T08:35:23.316788+0000","last_fullsized":"2024-07-17T08:35:23.316788+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322296+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:23.322296+0000","last_peered":"2024-07-17T08:35:23.322296+0000","last_clean":"2024-07-17T08:35:23.322296+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:23.322296+0000","last_undegraded":"2024-07-17T08:35:23.322296+0000","last_fullsized":"2024-07-17T08:35:23.322296+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339191+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:22.339191+0000","last_peered":"2024-07-17T08:35:22.339191+0000","last_clean":"2024-07-17T08:35:22.339191+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:22.339191+0000","last_undegraded":"2024-07-17T08:35:22.339191+0000","last_fullsized":"2024-07-17T08:35:22.339191+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336684+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:22.336684+0000","last_peered":"2024-07-17T08:35:22.336684+0000","last_clean":"2024-07-17T08:35:22.336684+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:22.336684+0000","last_undegraded":"2024-07-17T08:35:22.336684+0000","last_fullsized":"2024-07-17T08:35:22.336684+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:48.340873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339161+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:22.339161+0000","last_peered":"2024-07-17T08:35:22.339161+0000","last_clean":"2024-07-17T08:35:22.339161+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:22.339161+0000","last_undegraded":"2024-07-17T08:35:22.339161+0000","last_fullsized":"2024-07-17T08:35:22.339161+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.324302+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:22.324302+0000","last_peered":"2024-07-17T08:35:22.324302+0000","last_clean":"2024-07-17T08:35:22.324302+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:22.324302+0000","last_undegraded":"2024-07-17T08:35:22.324302+0000","last_fullsized":"2024-07-17T08:35:22.324302+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:27.503999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.334218+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:22.334218+0000","last_peered":"2024-07-17T08:35:22.334218+0000","last_clean":"2024-07-17T08:35:22.334218+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:22.334218+0000","last_undegraded":"2024-07-17T08:35:22.334218+0000","last_fullsized":"2024-07-17T08:35:22.334218+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321958+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:23.321958+0000","last_peered":"2024-07-17T08:35:23.321958+0000","last_clean":"2024-07-17T08:35:23.321958+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:23.321958+0000","last_undegraded":"2024-07-17T08:35:23.321958+0000","last_fullsized":"2024-07-17T08:35:23.321958+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317450+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:23.317450+0000","last_peered":"2024-07-17T08:35:23.317450+0000","last_clean":"2024-07-17T08:35:23.317450+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:23.317450+0000","last_undegraded":"2024-07-17T08:35:23.317450+0000","last_fullsized":"2024-07-17T08:35:23.317450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309536+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:22.309536+0000","last_peered":"2024-07-17T08:35:22.309536+0000","last_clean":"2024-07-17T08:35:22.309536+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:22.309536+0000","last_undegraded":"2024-07-17T08:35:22.309536+0000","last_fullsized":"2024-07-17T08:35:22.309536+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318125+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:23.318125+0000","last_peered":"2024-07-17T08:35:23.318125+0000","last_clean":"2024-07-17T08:35:23.318125+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:23.318125+0000","last_undegraded":"2024-07-17T08:35:23.318125+0000","last_fullsized":"2024-07-17T08:35:23.318125+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.338063+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:22.338063+0000","last_peered":"2024-07-17T08:35:22.338063+0000","last_clean":"2024-07-17T08:35:22.338063+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:22.338063+0000","last_undegraded":"2024-07-17T08:35:22.338063+0000","last_fullsized":"2024-07-17T08:35:22.338063+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:35:13.946033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.296229+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:22.296229+0000","last_peered":"2024-07-17T08:35:22.296229+0000","last_clean":"2024-07-17T08:35:22.296229+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:22.296229+0000","last_undegraded":"2024-07-17T08:35:22.296229+0000","last_fullsized":"2024-07-17T08:35:22.296229+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320022+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:23.320022+0000","last_peered":"2024-07-17T08:35:23.320022+0000","last_clean":"2024-07-17T08:35:23.320022+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:23.320022+0000","last_undegraded":"2024-07-17T08:35:23.320022+0000","last_fullsized":"2024-07-17T08:35:23.320022+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.298829+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:22.298829+0000","last_peered":"2024-07-17T08:35:22.298829+0000","last_clean":"2024-07-17T08:35:22.298829+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:22.298829+0000","last_undegraded":"2024-07-17T08:35:22.298829+0000","last_fullsized":"2024-07-17T08:35:22.298829+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.323334+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:23.323334+0000","last_peered":"2024-07-17T08:35:23.323334+0000","last_clean":"2024-07-17T08:35:23.323334+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:23.323334+0000","last_undegraded":"2024-07-17T08:35:23.323334+0000","last_fullsized":"2024-07-17T08:35:23.323334+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309707+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:22.309707+0000","last_peered":"2024-07-17T08:35:22.309707+0000","last_clean":"2024-07-17T08:35:22.309707+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:22.309707+0000","last_undegraded":"2024-07-17T08:35:22.309707+0000","last_fullsized":"2024-07-17T08:35:22.309707+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322022+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:23.322022+0000","last_peered":"2024-07-17T08:35:23.322022+0000","last_clean":"2024-07-17T08:35:23.322022+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:23.322022+0000","last_undegraded":"2024-07-17T08:35:23.322022+0000","last_fullsized":"2024-07-17T08:35:23.322022+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320401+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:23.320401+0000","last_peered":"2024-07-17T08:35:23.320401+0000","last_clean":"2024-07-17T08:35:23.320401+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:23.320401+0000","last_undegraded":"2024-07-17T08:35:23.320401+0000","last_fullsized":"2024-07-17T08:35:23.320401+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326648+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:23.326648+0000","last_peered":"2024-07-17T08:35:23.326648+0000","last_clean":"2024-07-17T08:35:23.326648+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:23.326648+0000","last_undegraded":"2024-07-17T08:35:23.326648+0000","last_fullsized":"2024-07-17T08:35:23.326648+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326032+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:23.326032+0000","last_peered":"2024-07-17T08:35:23.326032+0000","last_clean":"2024-07-17T08:35:23.326032+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:23.326032+0000","last_undegraded":"2024-07-17T08:35:23.326032+0000","last_fullsized":"2024-07-17T08:35:23.326032+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318899+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:23.318899+0000","last_peered":"2024-07-17T08:35:23.318899+0000","last_clean":"2024-07-17T08:35:23.318899+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:23.318899+0000","last_undegraded":"2024-07-17T08:35:23.318899+0000","last_fullsized":"2024-07-17T08:35:23.318899+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.298499+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:22.298499+0000","last_peered":"2024-07-17T08:35:22.298499+0000","last_clean":"2024-07-17T08:35:22.298499+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:22.298499+0000","last_undegraded":"2024-07-17T08:35:22.298499+0000","last_fullsized":"2024-07-17T08:35:22.298499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.323796+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:22.323796+0000","last_peered":"2024-07-17T08:35:22.323796+0000","last_clean":"2024-07-17T08:35:22.323796+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:22.323796+0000","last_undegraded":"2024-07-17T08:35:22.323796+0000","last_fullsized":"2024-07-17T08:35:22.323796+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:35.373523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.306117+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:22.306117+0000","last_peered":"2024-07-17T08:35:22.306117+0000","last_clean":"2024-07-17T08:35:22.306117+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:22.306117+0000","last_undegraded":"2024-07-17T08:35:22.306117+0000","last_fullsized":"2024-07-17T08:35:22.306117+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321848+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:23.321848+0000","last_peered":"2024-07-17T08:35:23.321848+0000","last_clean":"2024-07-17T08:35:23.321848+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:23.321848+0000","last_undegraded":"2024-07-17T08:35:23.321848+0000","last_fullsized":"2024-07-17T08:35:23.321848+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322119+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:23.322119+0000","last_peered":"2024-07-17T08:35:23.322119+0000","last_clean":"2024-07-17T08:35:23.322119+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:23.322119+0000","last_undegraded":"2024-07-17T08:35:23.322119+0000","last_fullsized":"2024-07-17T08:35:23.322119+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332783+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:22.332783+0000","last_peered":"2024-07-17T08:35:22.332783+0000","last_clean":"2024-07-17T08:35:22.332783+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:22.332783+0000","last_undegraded":"2024-07-17T08:35:22.332783+0000","last_fullsized":"2024-07-17T08:35:22.332783+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:25.679004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319246+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:23.319246+0000","last_peered":"2024-07-17T08:35:23.319246+0000","last_clean":"2024-07-17T08:35:23.319246+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:23.319246+0000","last_undegraded":"2024-07-17T08:35:23.319246+0000","last_fullsized":"2024-07-17T08:35:23.319246+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332659+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:22.332659+0000","last_peered":"2024-07-17T08:35:22.332659+0000","last_clean":"2024-07-17T08:35:22.332659+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:22.332659+0000","last_undegraded":"2024-07-17T08:35:22.332659+0000","last_fullsized":"2024-07-17T08:35:22.332659+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:53.406049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309861+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:22.309861+0000","last_peered":"2024-07-17T08:35:22.309861+0000","last_clean":"2024-07-17T08:35:22.309861+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:22.309861+0000","last_undegraded":"2024-07-17T08:35:22.309861+0000","last_fullsized":"2024-07-17T08:35:22.309861+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326714+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:23.326714+0000","last_peered":"2024-07-17T08:35:23.326714+0000","last_clean":"2024-07-17T08:35:23.326714+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:23.326714+0000","last_undegraded":"2024-07-17T08:35:23.326714+0000","last_fullsized":"2024-07-17T08:35:23.326714+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316939+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:23.316939+0000","last_peered":"2024-07-17T08:35:23.316939+0000","last_clean":"2024-07-17T08:35:23.316939+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:23.316939+0000","last_undegraded":"2024-07-17T08:35:23.316939+0000","last_fullsized":"2024-07-17T08:35:23.316939+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339029+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:22.339029+0000","last_peered":"2024-07-17T08:35:22.339029+0000","last_clean":"2024-07-17T08:35:22.339029+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:22.339029+0000","last_undegraded":"2024-07-17T08:35:22.339029+0000","last_fullsized":"2024-07-17T08:35:22.339029+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:52.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319119+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:23.319119+0000","last_peered":"2024-07-17T08:35:23.319119+0000","last_clean":"2024-07-17T08:35:23.319119+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:23.319119+0000","last_undegraded":"2024-07-17T08:35:23.319119+0000","last_fullsized":"2024-07-17T08:35:23.319119+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3376,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315183+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:22.315183+0000","last_peered":"2024-07-17T08:35:22.315183+0000","last_clean":"2024-07-17T08:35:22.315183+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:22.315183+0000","last_undegraded":"2024-07-17T08:35:22.315183+0000","last_fullsized":"2024-07-17T08:35:22.315183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543342,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65028,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3391488,"data_stored":2104150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510637,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":2809856,"data_stored":1521562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510638,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2789376,"data_stored":1512156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47792,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324048,"statfs":{"total":96636764160,"available":96587825152,"internally_reserved":0,"allocated":2785280,"data_stored":1512682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":2793472,"data_stored":1512226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51948,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319892,"statfs":{"total":96636764160,"available":96583569408,"internally_reserved":0,"allocated":2785280,"data_stored":1512190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510638,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47784,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324056,"statfs":{"total":96636764160,"available":96587833344,"internally_reserved":0,"allocated":2781184,"data_stored":1512100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510637,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56708,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315132,"statfs":{"total":96636764160,"available":96578695168,"internally_reserved":0,"allocated":3391488,"data_stored":2109826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:26.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 -- 172.21.15.44:0/763760857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 msgr2=0x7f48d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 --2- 172.21.15.44:0/763760857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48d0004c80 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c310 tx=0x7f48d0005830 comp rx=0 tx=0).stop 2024-07-17T08:35:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 -- 172.21.15.44:0/763760857 shutdown_connections 2024-07-17T08:35:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 --2- 172.21.15.44:0/763760857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48d0004c80 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 -- 172.21.15.44:0/763760857 >> 172.21.15.44:0/763760857 conn(0x7f48e80b0fb0 msgr2=0x7f48e80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 -- 172.21.15.44:0/763760857 shutdown_connections 2024-07-17T08:35:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.048+0000 7f48ef600640 1 -- 172.21.15.44:0/763760857 wait complete. 2024-07-17T08:35:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.050+0000 7f48ef600640 1 Processor -- start 2024-07-17T08:35:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.050+0000 7f48ef600640 1 -- start start 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 0x7f48e81e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 0x7f48e80ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48e80adb20 con 0x7f48d0004800 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48e80ad9a0 con 0x7f48e8151040 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ef600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48e80ac220 con 0x7f48e81e8d50 2024-07-17T08:35:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ecc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ecc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60046/0 (socket says 172.21.15.44:60046) 2024-07-17T08:35:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.051+0000 7f48ecc00640 1 -- 172.21.15.44:0/1836023774 learned_addr learned my addr 172.21.15.44:0/1836023774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48e7e00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 0x7f48e81e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 -- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 msgr2=0x7f48e80ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ed600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 0x7f48e80ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 0x7f48e80ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 -- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 msgr2=0x7f48e81e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 0x7f48e81e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 -- 172.21.15.44:0/1836023774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e81e7810 con 0x7f48d0004800 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48e7e00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 0x7f48e81e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ed600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 0x7f48e80ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48ecc00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f48dc010940 tx=0x7f48dc010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.052+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00ece0 con 0x7f48d0004800 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.053+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48dc00ee80 con 0x7f48d0004800 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.053+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc004290 con 0x7f48d0004800 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.053+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e81f7a80 con 0x7f48d0004800 2024-07-17T08:35:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.053+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e81f8040 con 0x7f48d0004800 2024-07-17T08:35:27.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.054+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e815ee90 con 0x7f48d0004800 2024-07-17T08:35:27.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.055+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f48dc0035f0 con 0x7f48d0004800 2024-07-17T08:35:27.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.056+0000 7f48e4c00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 0x7f48c80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.056+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f48dc099590 con 0x7f48d0004800 2024-07-17T08:35:27.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.057+0000 7f48e7e00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 0x7f48c80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.058+0000 7f48e7e00640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 0x7f48c80791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48d4033000 tx=0x7f48d40019f0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:27.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.061+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48dc062470 con 0x7f48d0004800 2024-07-17T08:35:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.267+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48e81e9c80 con 0x7f48c8076d00 2024-07-17T08:35:27.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.281+0000 7f48e4c00640 1 -- 172.21.15.44:0/1836023774 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7f48e81e9c80 con 0x7f48c8076d00 2024-07-17T08:35:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 msgr2=0x7f48c80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 0x7f48c80791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48d4033000 tx=0x7f48d40019f0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 msgr2=0x7f48e81e72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f48dc010940 tx=0x7f48dc010e00 comp rx=0 tx=0).stop 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 shutdown_connections 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48e81e8d50 0x7f48e80ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f48c8076d00 0x7f48c80791c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48e8151040 0x7f48e81e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 --2- 172.21.15.44:0/1836023774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48d0004800 0x7f48e81e72d0 secure :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f48dc010940 tx=0x7f48dc010e00 comp rx=0 tx=0).stop 2024-07-17T08:35:27.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 >> 172.21.15.44:0/1836023774 conn(0x7f48e80b0fb0 msgr2=0x7f48e815dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:27.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 shutdown_connections 2024-07-17T08:35:27.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.287+0000 7f48ef600640 1 -- 172.21.15.44:0/1836023774 wait complete. 2024-07-17T08:35:27.310 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-07-17T08:35:25.761046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":404,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2550,"ondisk_log_size":2550,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":415432,"kb_used_data":22976,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754559288,"statfs":{"total":773094113280,"available":772668710912,"internally_reserved":0,"allocated":23527424,"data_stored":13296892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.497899"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339517+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:22.339517+0000","last_peered":"2024-07-17T08:35:22.339517+0000","last_clean":"2024-07-17T08:35:22.339517+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:22.339517+0000","last_undegraded":"2024-07-17T08:35:22.339517+0000","last_fullsized":"2024-07-17T08:35:22.339517+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:07.170768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.313966+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:22.313966+0000","last_peered":"2024-07-17T08:35:22.313966+0000","last_clean":"2024-07-17T08:35:22.313966+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:22.313966+0000","last_undegraded":"2024-07-17T08:35:22.313966+0000","last_fullsized":"2024-07-17T08:35:22.313966+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326699+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:23.326699+0000","last_peered":"2024-07-17T08:35:23.326699+0000","last_clean":"2024-07-17T08:35:23.326699+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:23.326699+0000","last_undegraded":"2024-07-17T08:35:23.326699+0000","last_fullsized":"2024-07-17T08:35:23.326699+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332815+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:22.332815+0000","last_peered":"2024-07-17T08:35:22.332815+0000","last_clean":"2024-07-17T08:35:22.332815+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:22.332815+0000","last_undegraded":"2024-07-17T08:35:22.332815+0000","last_fullsized":"2024-07-17T08:35:22.332815+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:45.612904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317917+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:23.317917+0000","last_peered":"2024-07-17T08:35:23.317917+0000","last_clean":"2024-07-17T08:35:23.317917+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:23.317917+0000","last_undegraded":"2024-07-17T08:35:23.317917+0000","last_fullsized":"2024-07-17T08:35:23.317917+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.325243+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:22.325243+0000","last_peered":"2024-07-17T08:35:22.325243+0000","last_clean":"2024-07-17T08:35:22.325243+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:22.325243+0000","last_undegraded":"2024-07-17T08:35:22.325243+0000","last_fullsized":"2024-07-17T08:35:22.325243+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:17.385054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317048+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:23.317048+0000","last_peered":"2024-07-17T08:35:23.317048+0000","last_clean":"2024-07-17T08:35:23.317048+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:23.317048+0000","last_undegraded":"2024-07-17T08:35:23.317048+0000","last_fullsized":"2024-07-17T08:35:23.317048+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320757+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:23.320757+0000","last_peered":"2024-07-17T08:35:23.320757+0000","last_clean":"2024-07-17T08:35:23.320757+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:23.320757+0000","last_undegraded":"2024-07-17T08:35:23.320757+0000","last_fullsized":"2024-07-17T08:35:23.320757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319829+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:23.319829+0000","last_peered":"2024-07-17T08:35:23.319829+0000","last_clean":"2024-07-17T08:35:23.319829+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:23.319829+0000","last_undegraded":"2024-07-17T08:35:23.319829+0000","last_fullsized":"2024-07-17T08:35:23.319829+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322580+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:23.322580+0000","last_peered":"2024-07-17T08:35:23.322580+0000","last_clean":"2024-07-17T08:35:23.322580+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:23.322580+0000","last_undegraded":"2024-07-17T08:35:23.322580+0000","last_fullsized":"2024-07-17T08:35:23.322580+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326844+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:23.326844+0000","last_peered":"2024-07-17T08:35:23.326844+0000","last_clean":"2024-07-17T08:35:23.326844+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:23.326844+0000","last_undegraded":"2024-07-17T08:35:23.326844+0000","last_fullsized":"2024-07-17T08:35:23.326844+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317307+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:23.317307+0000","last_peered":"2024-07-17T08:35:23.317307+0000","last_clean":"2024-07-17T08:35:23.317307+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:23.317307+0000","last_undegraded":"2024-07-17T08:35:23.317307+0000","last_fullsized":"2024-07-17T08:35:23.317307+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332706+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:22.332706+0000","last_peered":"2024-07-17T08:35:22.332706+0000","last_clean":"2024-07-17T08:35:22.332706+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:22.332706+0000","last_undegraded":"2024-07-17T08:35:22.332706+0000","last_fullsized":"2024-07-17T08:35:22.332706+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:43.220621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320816+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:23.320816+0000","last_peered":"2024-07-17T08:35:23.320816+0000","last_clean":"2024-07-17T08:35:23.320816+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:23.320816+0000","last_undegraded":"2024-07-17T08:35:23.320816+0000","last_fullsized":"2024-07-17T08:35:23.320816+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332375+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:22.332375+0000","last_peered":"2024-07-17T08:35:22.332375+0000","last_clean":"2024-07-17T08:35:22.332375+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:22.332375+0000","last_undegraded":"2024-07-17T08:35:22.332375+0000","last_fullsized":"2024-07-17T08:35:22.332375+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:47.900589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326124+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:23.326124+0000","last_peered":"2024-07-17T08:35:23.326124+0000","last_clean":"2024-07-17T08:35:23.326124+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:23.326124+0000","last_undegraded":"2024-07-17T08:35:23.326124+0000","last_fullsized":"2024-07-17T08:35:23.326124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320251+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:23.320251+0000","last_peered":"2024-07-17T08:35:23.320251+0000","last_clean":"2024-07-17T08:35:23.320251+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:23.320251+0000","last_undegraded":"2024-07-17T08:35:23.320251+0000","last_fullsized":"2024-07-17T08:35:23.320251+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319506+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:23.319506+0000","last_peered":"2024-07-17T08:35:23.319506+0000","last_clean":"2024-07-17T08:35:23.319506+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:23.319506+0000","last_undegraded":"2024-07-17T08:35:23.319506+0000","last_fullsized":"2024-07-17T08:35:23.319506+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322345+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:23.322345+0000","last_peered":"2024-07-17T08:35:23.322345+0000","last_clean":"2024-07-17T08:35:23.322345+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:23.322345+0000","last_undegraded":"2024-07-17T08:35:23.322345+0000","last_fullsized":"2024-07-17T08:35:23.322345+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322122+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:23.322122+0000","last_peered":"2024-07-17T08:35:23.322122+0000","last_clean":"2024-07-17T08:35:23.322122+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:23.322122+0000","last_undegraded":"2024-07-17T08:35:23.322122+0000","last_fullsized":"2024-07-17T08:35:23.322122+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321838+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:23.321838+0000","last_peered":"2024-07-17T08:35:23.321838+0000","last_clean":"2024-07-17T08:35:23.321838+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:23.321838+0000","last_undegraded":"2024-07-17T08:35:23.321838+0000","last_fullsized":"2024-07-17T08:35:23.321838+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321999+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:23.321999+0000","last_peered":"2024-07-17T08:35:23.321999+0000","last_clean":"2024-07-17T08:35:23.321999+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:23.321999+0000","last_undegraded":"2024-07-17T08:35:23.321999+0000","last_fullsized":"2024-07-17T08:35:23.321999+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.333880+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:22.333880+0000","last_peered":"2024-07-17T08:35:22.333880+0000","last_clean":"2024-07-17T08:35:22.333880+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:22.333880+0000","last_undegraded":"2024-07-17T08:35:22.333880+0000","last_fullsized":"2024-07-17T08:35:22.333880+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:40.484299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316881+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:23.316881+0000","last_peered":"2024-07-17T08:35:23.316881+0000","last_clean":"2024-07-17T08:35:23.316881+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:23.316881+0000","last_undegraded":"2024-07-17T08:35:23.316881+0000","last_fullsized":"2024-07-17T08:35:23.316881+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320212+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:23.320212+0000","last_peered":"2024-07-17T08:35:23.320212+0000","last_clean":"2024-07-17T08:35:23.320212+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:23.320212+0000","last_undegraded":"2024-07-17T08:35:23.320212+0000","last_fullsized":"2024-07-17T08:35:23.320212+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336680+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:22.336680+0000","last_peered":"2024-07-17T08:35:22.336680+0000","last_clean":"2024-07-17T08:35:22.336680+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:22.336680+0000","last_undegraded":"2024-07-17T08:35:22.336680+0000","last_fullsized":"2024-07-17T08:35:22.336680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:26.810568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326818+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:23.326818+0000","last_peered":"2024-07-17T08:35:23.326818+0000","last_clean":"2024-07-17T08:35:23.326818+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:23.326818+0000","last_undegraded":"2024-07-17T08:35:23.326818+0000","last_fullsized":"2024-07-17T08:35:23.326818+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317375+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:23.317375+0000","last_peered":"2024-07-17T08:35:23.317375+0000","last_clean":"2024-07-17T08:35:23.317375+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:23.317375+0000","last_undegraded":"2024-07-17T08:35:23.317375+0000","last_fullsized":"2024-07-17T08:35:23.317375+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319032+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:23.319032+0000","last_peered":"2024-07-17T08:35:23.319032+0000","last_clean":"2024-07-17T08:35:23.319032+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:23.319032+0000","last_undegraded":"2024-07-17T08:35:23.319032+0000","last_fullsized":"2024-07-17T08:35:23.319032+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315473+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:22.315473+0000","last_peered":"2024-07-17T08:35:22.315473+0000","last_clean":"2024-07-17T08:35:22.315473+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:22.315473+0000","last_undegraded":"2024-07-17T08:35:22.315473+0000","last_fullsized":"2024-07-17T08:35:22.315473+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:34.228783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332902+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:22.332902+0000","last_peered":"2024-07-17T08:35:22.332902+0000","last_clean":"2024-07-17T08:35:22.332902+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:22.332902+0000","last_undegraded":"2024-07-17T08:35:22.332902+0000","last_fullsized":"2024-07-17T08:35:22.332902+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:46:45.694196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.319697+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:22.319697+0000","last_peered":"2024-07-17T08:35:22.319697+0000","last_clean":"2024-07-17T08:35:22.319697+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:22.319697+0000","last_undegraded":"2024-07-17T08:35:22.319697+0000","last_fullsized":"2024-07-17T08:35:22.319697+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:58:40.977604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.331426+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:22.331426+0000","last_peered":"2024-07-17T08:35:22.331426+0000","last_clean":"2024-07-17T08:35:22.331426+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:22.331426+0000","last_undegraded":"2024-07-17T08:35:22.331426+0000","last_fullsized":"2024-07-17T08:35:22.331426+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":16,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.341018+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:22.341018+0000","last_peered":"2024-07-17T08:35:22.341018+0000","last_clean":"2024-07-17T08:35:22.341018+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:22.341018+0000","last_undegraded":"2024-07-17T08:35:22.341018+0000","last_fullsized":"2024-07-17T08:35:22.341018+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.317524+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:22.317524+0000","last_peered":"2024-07-17T08:35:22.317524+0000","last_clean":"2024-07-17T08:35:22.317524+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:22.317524+0000","last_undegraded":"2024-07-17T08:35:22.317524+0000","last_fullsized":"2024-07-17T08:35:22.317524+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:48.338537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336710+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:22.336710+0000","last_peered":"2024-07-17T08:35:22.336710+0000","last_clean":"2024-07-17T08:35:22.336710+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:22.336710+0000","last_undegraded":"2024-07-17T08:35:22.336710+0000","last_fullsized":"2024-07-17T08:35:22.336710+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321880+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:23.321880+0000","last_peered":"2024-07-17T08:35:23.321880+0000","last_clean":"2024-07-17T08:35:23.321880+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:23.321880+0000","last_undegraded":"2024-07-17T08:35:23.321880+0000","last_fullsized":"2024-07-17T08:35:23.321880+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336285+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:22.336285+0000","last_peered":"2024-07-17T08:35:22.336285+0000","last_clean":"2024-07-17T08:35:22.336285+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:22.336285+0000","last_undegraded":"2024-07-17T08:35:22.336285+0000","last_fullsized":"2024-07-17T08:35:22.336285+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317968+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:23.317968+0000","last_peered":"2024-07-17T08:35:23.317968+0000","last_clean":"2024-07-17T08:35:23.317968+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:23.317968+0000","last_undegraded":"2024-07-17T08:35:23.317968+0000","last_fullsized":"2024-07-17T08:35:23.317968+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319236+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:23.319236+0000","last_peered":"2024-07-17T08:35:23.319236+0000","last_clean":"2024-07-17T08:35:23.319236+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:23.319236+0000","last_undegraded":"2024-07-17T08:35:23.319236+0000","last_fullsized":"2024-07-17T08:35:23.319236+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.312930+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:22.312930+0000","last_peered":"2024-07-17T08:35:22.312930+0000","last_clean":"2024-07-17T08:35:22.312930+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:22.312930+0000","last_undegraded":"2024-07-17T08:35:22.312930+0000","last_fullsized":"2024-07-17T08:35:22.312930+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:19.775392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.338521+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:22.338521+0000","last_peered":"2024-07-17T08:35:22.338521+0000","last_clean":"2024-07-17T08:35:22.338521+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:22.338521+0000","last_undegraded":"2024-07-17T08:35:22.338521+0000","last_fullsized":"2024-07-17T08:35:22.338521+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.312997+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:22.312997+0000","last_peered":"2024-07-17T08:35:22.312997+0000","last_clean":"2024-07-17T08:35:22.312997+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:22.312997+0000","last_undegraded":"2024-07-17T08:35:22.312997+0000","last_fullsized":"2024-07-17T08:35:22.312997+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:52.676809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.317228+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:22.317228+0000","last_peered":"2024-07-17T08:35:22.317228+0000","last_clean":"2024-07-17T08:35:22.317228+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:22.317228+0000","last_undegraded":"2024-07-17T08:35:22.317228+0000","last_fullsized":"2024-07-17T08:35:22.317228+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320332+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:23.320332+0000","last_peered":"2024-07-17T08:35:23.320332+0000","last_clean":"2024-07-17T08:35:23.320332+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:23.320332+0000","last_undegraded":"2024-07-17T08:35:23.320332+0000","last_fullsized":"2024-07-17T08:35:23.320332+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326134+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:23.326134+0000","last_peered":"2024-07-17T08:35:23.326134+0000","last_clean":"2024-07-17T08:35:23.326134+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:23.326134+0000","last_undegraded":"2024-07-17T08:35:23.326134+0000","last_fullsized":"2024-07-17T08:35:23.326134+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.320389+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:22.320389+0000","last_peered":"2024-07-17T08:35:22.320389+0000","last_clean":"2024-07-17T08:35:22.320389+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:22.320389+0000","last_undegraded":"2024-07-17T08:35:22.320389+0000","last_fullsized":"2024-07-17T08:35:22.320389+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:12.044847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317528+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:23.317528+0000","last_peered":"2024-07-17T08:35:23.317528+0000","last_clean":"2024-07-17T08:35:23.317528+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:23.317528+0000","last_undegraded":"2024-07-17T08:35:23.317528+0000","last_fullsized":"2024-07-17T08:35:23.317528+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326134+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:23.326134+0000","last_peered":"2024-07-17T08:35:23.326134+0000","last_clean":"2024-07-17T08:35:23.326134+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:23.326134+0000","last_undegraded":"2024-07-17T08:35:23.326134+0000","last_fullsized":"2024-07-17T08:35:23.326134+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315342+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:22.315342+0000","last_peered":"2024-07-17T08:35:22.315342+0000","last_clean":"2024-07-17T08:35:22.315342+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:22.315342+0000","last_undegraded":"2024-07-17T08:35:22.315342+0000","last_fullsized":"2024-07-17T08:35:22.315342+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319286+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:23.319286+0000","last_peered":"2024-07-17T08:35:23.319286+0000","last_clean":"2024-07-17T08:35:23.319286+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:23.319286+0000","last_undegraded":"2024-07-17T08:35:23.319286+0000","last_fullsized":"2024-07-17T08:35:23.319286+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":17,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319631+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:23.319631+0000","last_peered":"2024-07-17T08:35:23.319631+0000","last_clean":"2024-07-17T08:35:23.319631+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:23.319631+0000","last_undegraded":"2024-07-17T08:35:23.319631+0000","last_fullsized":"2024-07-17T08:35:23.319631+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318124+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:23.318124+0000","last_peered":"2024-07-17T08:35:23.318124+0000","last_clean":"2024-07-17T08:35:23.318124+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:23.318124+0000","last_undegraded":"2024-07-17T08:35:23.318124+0000","last_fullsized":"2024-07-17T08:35:23.318124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"222'6","reported_seq":18,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.343996+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:22.343996+0000","last_peered":"2024-07-17T08:35:22.343996+0000","last_clean":"2024-07-17T08:35:22.343996+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:22.343996+0000","last_undegraded":"2024-07-17T08:35:22.343996+0000","last_fullsized":"2024-07-17T08:35:22.343996+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309577+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:22.309577+0000","last_peered":"2024-07-17T08:35:22.309577+0000","last_clean":"2024-07-17T08:35:22.309577+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:22.309577+0000","last_undegraded":"2024-07-17T08:35:22.309577+0000","last_fullsized":"2024-07-17T08:35:22.309577+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.340736+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:22.340736+0000","last_peered":"2024-07-17T08:35:22.340736+0000","last_clean":"2024-07-17T08:35:22.340736+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:22.340736+0000","last_undegraded":"2024-07-17T08:35:22.340736+0000","last_fullsized":"2024-07-17T08:35:22.340736+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326986+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:23.326986+0000","last_peered":"2024-07-17T08:35:23.326986+0000","last_clean":"2024-07-17T08:35:23.326986+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:23.326986+0000","last_undegraded":"2024-07-17T08:35:23.326986+0000","last_fullsized":"2024-07-17T08:35:23.326986+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326612+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:23.326612+0000","last_peered":"2024-07-17T08:35:23.326612+0000","last_clean":"2024-07-17T08:35:23.326612+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:23.326612+0000","last_undegraded":"2024-07-17T08:35:23.326612+0000","last_fullsized":"2024-07-17T08:35:23.326612+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319872+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:23.319872+0000","last_peered":"2024-07-17T08:35:23.319872+0000","last_clean":"2024-07-17T08:35:23.319872+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:23.319872+0000","last_undegraded":"2024-07-17T08:35:23.319872+0000","last_fullsized":"2024-07-17T08:35:23.319872+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318614+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:23.318614+0000","last_peered":"2024-07-17T08:35:23.318614+0000","last_clean":"2024-07-17T08:35:23.318614+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:23.318614+0000","last_undegraded":"2024-07-17T08:35:23.318614+0000","last_fullsized":"2024-07-17T08:35:23.318614+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332744+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:22.332744+0000","last_peered":"2024-07-17T08:35:22.332744+0000","last_clean":"2024-07-17T08:35:22.332744+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:22.332744+0000","last_undegraded":"2024-07-17T08:35:22.332744+0000","last_fullsized":"2024-07-17T08:35:22.332744+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:46.008752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":22,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316788+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:23.316788+0000","last_peered":"2024-07-17T08:35:23.316788+0000","last_clean":"2024-07-17T08:35:23.316788+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:23.316788+0000","last_undegraded":"2024-07-17T08:35:23.316788+0000","last_fullsized":"2024-07-17T08:35:23.316788+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322296+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:23.322296+0000","last_peered":"2024-07-17T08:35:23.322296+0000","last_clean":"2024-07-17T08:35:23.322296+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:23.322296+0000","last_undegraded":"2024-07-17T08:35:23.322296+0000","last_fullsized":"2024-07-17T08:35:23.322296+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339191+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:22.339191+0000","last_peered":"2024-07-17T08:35:22.339191+0000","last_clean":"2024-07-17T08:35:22.339191+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:22.339191+0000","last_undegraded":"2024-07-17T08:35:22.339191+0000","last_fullsized":"2024-07-17T08:35:22.339191+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.336684+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:22.336684+0000","last_peered":"2024-07-17T08:35:22.336684+0000","last_clean":"2024-07-17T08:35:22.336684+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:22.336684+0000","last_undegraded":"2024-07-17T08:35:22.336684+0000","last_fullsized":"2024-07-17T08:35:22.336684+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:48.340873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339161+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:22.339161+0000","last_peered":"2024-07-17T08:35:22.339161+0000","last_clean":"2024-07-17T08:35:22.339161+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:22.339161+0000","last_undegraded":"2024-07-17T08:35:22.339161+0000","last_fullsized":"2024-07-17T08:35:22.339161+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.324302+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:22.324302+0000","last_peered":"2024-07-17T08:35:22.324302+0000","last_clean":"2024-07-17T08:35:22.324302+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:22.324302+0000","last_undegraded":"2024-07-17T08:35:22.324302+0000","last_fullsized":"2024-07-17T08:35:22.324302+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:27.503999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.334218+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:22.334218+0000","last_peered":"2024-07-17T08:35:22.334218+0000","last_clean":"2024-07-17T08:35:22.334218+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:22.334218+0000","last_undegraded":"2024-07-17T08:35:22.334218+0000","last_fullsized":"2024-07-17T08:35:22.334218+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321958+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:23.321958+0000","last_peered":"2024-07-17T08:35:23.321958+0000","last_clean":"2024-07-17T08:35:23.321958+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:23.321958+0000","last_undegraded":"2024-07-17T08:35:23.321958+0000","last_fullsized":"2024-07-17T08:35:23.321958+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.317450+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:23.317450+0000","last_peered":"2024-07-17T08:35:23.317450+0000","last_clean":"2024-07-17T08:35:23.317450+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:23.317450+0000","last_undegraded":"2024-07-17T08:35:23.317450+0000","last_fullsized":"2024-07-17T08:35:23.317450+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309536+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:22.309536+0000","last_peered":"2024-07-17T08:35:22.309536+0000","last_clean":"2024-07-17T08:35:22.309536+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:22.309536+0000","last_undegraded":"2024-07-17T08:35:22.309536+0000","last_fullsized":"2024-07-17T08:35:22.309536+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318125+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:23.318125+0000","last_peered":"2024-07-17T08:35:23.318125+0000","last_clean":"2024-07-17T08:35:23.318125+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:23.318125+0000","last_undegraded":"2024-07-17T08:35:23.318125+0000","last_fullsized":"2024-07-17T08:35:23.318125+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.338063+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:22.338063+0000","last_peered":"2024-07-17T08:35:22.338063+0000","last_clean":"2024-07-17T08:35:22.338063+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:22.338063+0000","last_undegraded":"2024-07-17T08:35:22.338063+0000","last_fullsized":"2024-07-17T08:35:22.338063+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:35:13.946033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.296229+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:22.296229+0000","last_peered":"2024-07-17T08:35:22.296229+0000","last_clean":"2024-07-17T08:35:22.296229+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:22.296229+0000","last_undegraded":"2024-07-17T08:35:22.296229+0000","last_fullsized":"2024-07-17T08:35:22.296229+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320022+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:23.320022+0000","last_peered":"2024-07-17T08:35:23.320022+0000","last_clean":"2024-07-17T08:35:23.320022+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:23.320022+0000","last_undegraded":"2024-07-17T08:35:23.320022+0000","last_fullsized":"2024-07-17T08:35:23.320022+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.298829+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:22.298829+0000","last_peered":"2024-07-17T08:35:22.298829+0000","last_clean":"2024-07-17T08:35:22.298829+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:22.298829+0000","last_undegraded":"2024-07-17T08:35:22.298829+0000","last_fullsized":"2024-07-17T08:35:22.298829+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.323334+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:23.323334+0000","last_peered":"2024-07-17T08:35:23.323334+0000","last_clean":"2024-07-17T08:35:23.323334+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:23.323334+0000","last_undegraded":"2024-07-17T08:35:23.323334+0000","last_fullsized":"2024-07-17T08:35:23.323334+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309707+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:22.309707+0000","last_peered":"2024-07-17T08:35:22.309707+0000","last_clean":"2024-07-17T08:35:22.309707+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:22.309707+0000","last_undegraded":"2024-07-17T08:35:22.309707+0000","last_fullsized":"2024-07-17T08:35:22.309707+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322022+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:23.322022+0000","last_peered":"2024-07-17T08:35:23.322022+0000","last_clean":"2024-07-17T08:35:23.322022+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:23.322022+0000","last_undegraded":"2024-07-17T08:35:23.322022+0000","last_fullsized":"2024-07-17T08:35:23.322022+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":16,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.320401+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:23.320401+0000","last_peered":"2024-07-17T08:35:23.320401+0000","last_clean":"2024-07-17T08:35:23.320401+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:23.320401+0000","last_undegraded":"2024-07-17T08:35:23.320401+0000","last_fullsized":"2024-07-17T08:35:23.320401+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326648+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:23.326648+0000","last_peered":"2024-07-17T08:35:23.326648+0000","last_clean":"2024-07-17T08:35:23.326648+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:23.326648+0000","last_undegraded":"2024-07-17T08:35:23.326648+0000","last_fullsized":"2024-07-17T08:35:23.326648+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326032+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:23.326032+0000","last_peered":"2024-07-17T08:35:23.326032+0000","last_clean":"2024-07-17T08:35:23.326032+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:23.326032+0000","last_undegraded":"2024-07-17T08:35:23.326032+0000","last_fullsized":"2024-07-17T08:35:23.326032+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.318899+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:23.318899+0000","last_peered":"2024-07-17T08:35:23.318899+0000","last_clean":"2024-07-17T08:35:23.318899+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:23.318899+0000","last_undegraded":"2024-07-17T08:35:23.318899+0000","last_fullsized":"2024-07-17T08:35:23.318899+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.298499+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:22.298499+0000","last_peered":"2024-07-17T08:35:22.298499+0000","last_clean":"2024-07-17T08:35:22.298499+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:22.298499+0000","last_undegraded":"2024-07-17T08:35:22.298499+0000","last_fullsized":"2024-07-17T08:35:22.298499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.323796+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:22.323796+0000","last_peered":"2024-07-17T08:35:22.323796+0000","last_clean":"2024-07-17T08:35:22.323796+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:22.323796+0000","last_undegraded":"2024-07-17T08:35:22.323796+0000","last_fullsized":"2024-07-17T08:35:22.323796+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:35.373523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.306117+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:22.306117+0000","last_peered":"2024-07-17T08:35:22.306117+0000","last_clean":"2024-07-17T08:35:22.306117+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:22.306117+0000","last_undegraded":"2024-07-17T08:35:22.306117+0000","last_fullsized":"2024-07-17T08:35:22.306117+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.321848+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:23.321848+0000","last_peered":"2024-07-17T08:35:23.321848+0000","last_clean":"2024-07-17T08:35:23.321848+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:23.321848+0000","last_undegraded":"2024-07-17T08:35:23.321848+0000","last_fullsized":"2024-07-17T08:35:23.321848+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.322119+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:23.322119+0000","last_peered":"2024-07-17T08:35:23.322119+0000","last_clean":"2024-07-17T08:35:23.322119+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:23.322119+0000","last_undegraded":"2024-07-17T08:35:23.322119+0000","last_fullsized":"2024-07-17T08:35:23.322119+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332783+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:22.332783+0000","last_peered":"2024-07-17T08:35:22.332783+0000","last_clean":"2024-07-17T08:35:22.332783+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:22.332783+0000","last_undegraded":"2024-07-17T08:35:22.332783+0000","last_fullsized":"2024-07-17T08:35:22.332783+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:25.679004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319246+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:23.319246+0000","last_peered":"2024-07-17T08:35:23.319246+0000","last_clean":"2024-07-17T08:35:23.319246+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:23.319246+0000","last_undegraded":"2024-07-17T08:35:23.319246+0000","last_fullsized":"2024-07-17T08:35:23.319246+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.332659+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:22.332659+0000","last_peered":"2024-07-17T08:35:22.332659+0000","last_clean":"2024-07-17T08:35:22.332659+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:22.332659+0000","last_undegraded":"2024-07-17T08:35:22.332659+0000","last_fullsized":"2024-07-17T08:35:22.332659+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:53.406049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.309861+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:22.309861+0000","last_peered":"2024-07-17T08:35:22.309861+0000","last_clean":"2024-07-17T08:35:22.309861+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:22.309861+0000","last_undegraded":"2024-07-17T08:35:22.309861+0000","last_fullsized":"2024-07-17T08:35:22.309861+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.326714+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:23.326714+0000","last_peered":"2024-07-17T08:35:23.326714+0000","last_clean":"2024-07-17T08:35:23.326714+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:23.326714+0000","last_undegraded":"2024-07-17T08:35:23.326714+0000","last_fullsized":"2024-07-17T08:35:23.326714+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.316939+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:23.316939+0000","last_peered":"2024-07-17T08:35:23.316939+0000","last_clean":"2024-07-17T08:35:23.316939+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:23.316939+0000","last_undegraded":"2024-07-17T08:35:23.316939+0000","last_fullsized":"2024-07-17T08:35:23.316939+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.339029+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:22.339029+0000","last_peered":"2024-07-17T08:35:22.339029+0000","last_clean":"2024-07-17T08:35:22.339029+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:22.339029+0000","last_undegraded":"2024-07-17T08:35:22.339029+0000","last_fullsized":"2024-07-17T08:35:22.339029+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:52.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":15,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-07-17T08:35:23.319119+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:23.319119+0000","last_peered":"2024-07-17T08:35:23.319119+0000","last_clean":"2024-07-17T08:35:23.319119+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:23.319119+0000","last_undegraded":"2024-07-17T08:35:23.319119+0000","last_fullsized":"2024-07-17T08:35:23.319119+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3376,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-07-17T08:35:22.315183+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:22.315183+0000","last_peered":"2024-07-17T08:35:22.315183+0000","last_clean":"2024-07-17T08:35:22.315183+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:22.315183+0000","last_undegraded":"2024-07-17T08:35:22.315183+0000","last_fullsized":"2024-07-17T08:35:22.315183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543342,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65028,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306812,"statfs":{"total":96636764160,"available":96570175488,"internally_reserved":0,"allocated":3391488,"data_stored":2104150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510637,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":2809856,"data_stored":1521562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510638,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2789376,"data_stored":1512156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47792,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324048,"statfs":{"total":96636764160,"available":96587825152,"internally_reserved":0,"allocated":2785280,"data_stored":1512682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":2793472,"data_stored":1512226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51948,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319892,"statfs":{"total":96636764160,"available":96583569408,"internally_reserved":0,"allocated":2785280,"data_stored":1512190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510638,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47784,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324056,"statfs":{"total":96636764160,"available":96587833344,"internally_reserved":0,"allocated":2781184,"data_stored":1512100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510637,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56708,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315132,"statfs":{"total":96636764160,"available":96578695168,"internally_reserved":0,"allocated":3391488,"data_stored":2109826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:27.315 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:35:27.315 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.573+0000 7f2a2e000640 1 -- 172.21.15.44:0/1660194679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280d0ca0 msgr2=0x7f2a280b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.573+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1660194679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280d0ca0 0x7f2a280b1cd0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2a28071920 tx=0x7f2a1402a5f0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 -- 172.21.15.44:0/1660194679 shutdown_connections 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1660194679 >> v2:172.21.15.175:3300/0 conn(0x7f2a280b2210 0x7f2a280ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1660194679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280d0ca0 0x7f2a280b1cd0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 -- 172.21.15.44:0/1660194679 >> 172.21.15.44:0/1660194679 conn(0x7f2a280b5560 msgr2=0x7f2a280b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 -- 172.21.15.44:0/1660194679 shutdown_connections 2024-07-17T08:35:27.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.574+0000 7f2a2e000640 1 -- 172.21.15.44:0/1660194679 wait complete. 2024-07-17T08:35:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.575+0000 7f2a2e000640 1 Processor -- start 2024-07-17T08:35:27.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.576+0000 7f2a2e000640 1 -- start start 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a280d0ca0 0x7f2a2814c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a2815b3a0 0x7f2a281572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280aeff0 con 0x7f2a280b2210 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280aee70 con 0x7f2a2815b3a0 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a280af170 con 0x7f2a280d0ca0 2024-07-17T08:35:27.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60060/0 (socket says 172.21.15.44:60060) 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2cc00640 1 -- 172.21.15.44:0/1715072421 learned_addr learned my addr 172.21.15.44:0/1715072421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a27e00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a280d0ca0 0x7f2a2814c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.577+0000 7f2a2d600640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a2815b3a0 0x7f2a281572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 -- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a280d0ca0 msgr2=0x7f2a2814c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a280d0ca0 0x7f2a2814c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 -- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a2815b3a0 msgr2=0x7f2a281572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a2815b3a0 0x7f2a281572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 -- 172.21.15.44:0/1715072421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a281f3b60 con 0x7f2a280b2210 2024-07-17T08:35:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2cc00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c730 tx=0x7f2a1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18016020 con 0x7f2a280b2210 2024-07-17T08:35:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a1800ce40 con 0x7f2a280b2210 2024-07-17T08:35:27.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2814da00 con 0x7f2a280b2210 2024-07-17T08:35:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18006e60 con 0x7f2a280b2210 2024-07-17T08:35:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.578+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2814e1b0 con 0x7f2a280b2210 2024-07-17T08:35:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.580+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a280af670 con 0x7f2a280b2210 2024-07-17T08:35:27.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.581+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f2a18003bf0 con 0x7f2a280b2210 2024-07-17T08:35:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.582+0000 7f2a24c00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 0x7f2a08079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:27.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.582+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f2a18098570 con 0x7f2a280b2210 2024-07-17T08:35:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.582+0000 7f2a27e00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 0x7f2a08079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.583+0000 7f2a27e00640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 0x7f2a08079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a14033000 tx=0x7f2a14004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.586+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2a18020020 con 0x7f2a280b2210 2024-07-17T08:35:27.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.873+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2a280ae730 con 0x7f2a280b2210 2024-07-17T08:35:27.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.878+0000 7f2a24c00640 1 -- 172.21.15.44:0/1715072421 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v72) ==== 190+0+147 (secure 0 0 0) 0x7f2a180615f0 con 0x7f2a280b2210 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:35:27.880 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:35:27.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 msgr2=0x7f2a08079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 0x7f2a08079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a14033000 tx=0x7f2a14004730 comp rx=0 tx=0).stop 2024-07-17T08:35:27.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 msgr2=0x7f2a280b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:27.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c730 tx=0x7f2a1800cbf0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 shutdown_connections 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2a08076cd0 0x7f2a08079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a2815b3a0 0x7f2a281572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a280d0ca0 0x7f2a2814c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 --2- 172.21.15.44:0/1715072421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a280b2210 0x7f2a280b1ca0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 >> 172.21.15.44:0/1715072421 conn(0x7f2a280b5560 msgr2=0x7f2a280b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.881+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 shutdown_connections 2024-07-17T08:35:27.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:27.882+0000 7f2a2e000640 1 -- 172.21.15.44:0/1715072421 wait complete. 2024-07-17T08:35:27.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:28.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 -- 172.21.15.44:0/882329856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 msgr2=0x7f3ffc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:28.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 --2- 172.21.15.44:0/882329856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc06e9f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005fd0 tx=0x7f3fe802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 -- 172.21.15.44:0/882329856 shutdown_connections 2024-07-17T08:35:28.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 --2- 172.21.15.44:0/882329856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc06e9f0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 -- 172.21.15.44:0/882329856 >> 172.21.15.44:0/882329856 conn(0x7f3ffc06ce80 msgr2=0x7f3ffc06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:28.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 -- 172.21.15.44:0/882329856 shutdown_connections 2024-07-17T08:35:28.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.101+0000 7f4000a00640 1 -- 172.21.15.44:0/882329856 wait complete. 2024-07-17T08:35:28.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.103+0000 7f4000a00640 1 Processor -- start 2024-07-17T08:35:28.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.103+0000 7f4000a00640 1 -- start start 2024-07-17T08:35:28.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ffc06e610 0x7f3ffc140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:28.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 0x7f3ffc155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:28.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15c9d0 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15c850 con 0x7f3ffc06e610 2024-07-17T08:35:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.104+0000 7f4000a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc15cb50 con 0x7f3ffc06ef30 2024-07-17T08:35:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:28.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60070/0 (socket says 172.21.15.44:60070) 2024-07-17T08:35:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 -- 172.21.15.44:0/1159049988 learned_addr learned my addr 172.21.15.44:0/1159049988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffa800640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ffc06e610 0x7f3ffc140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ff9e00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 0x7f3ffc155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 -- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 msgr2=0x7f3ffc155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 0x7f3ffc155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 -- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ffc06e610 msgr2=0x7f3ffc140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ffc06e610 0x7f3ffc140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 -- 172.21.15.44:0/1159049988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1f2870 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ff9e00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 0x7f3ffc155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.105+0000 7f3ffb200640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c410 tx=0x7f3ff400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.106+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4011790 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.106+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ff4017070 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.106+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4011cc0 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.106+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc154ca0 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.106+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc154f40 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.107+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc15cfa0 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.108+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f3ff401c460 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.109+0000 7f3ff2a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 0x7f3fdc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.109+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f3ff4098560 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.110+0000 7f3ffa800640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 0x7f3fdc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.111+0000 7f3ffa800640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 0x7f3fdc079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0016b0 tx=0x7f3fec009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:28.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.114+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3ff4021020 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.362+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ffc068c10 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.363+0000 7f3ff2a00640 1 -- 172.21.15.44:0/1159049988 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5235 (secure 0 0 0) 0x7f3ff4061540 con 0x7f3ffc15e1b0 2024-07-17T08:35:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:28.365 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":114,"btime":"2024-07-17T08:35:25:338230+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":8329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13}]} 2024-07-17T08:35:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 114 2024-07-17T08:35:28.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 msgr2=0x7f3fdc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 0x7f3fdc079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0016b0 tx=0x7f3fec009040 comp rx=0 tx=0).stop 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 msgr2=0x7f3ffc1e54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c410 tx=0x7f3ff400c8d0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 shutdown_connections 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ffc15e1b0 0x7f3ffc1e54a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ffc06ef30 0x7f3ffc155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fdc076cd0 0x7f3fdc079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 --2- 172.21.15.44:0/1159049988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ffc06e610 0x7f3ffc140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 >> 172.21.15.44:0/1159049988 conn(0x7f3ffc06ce80 msgr2=0x7f3ffc1426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.366+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 shutdown_connections 2024-07-17T08:35:28.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:28.367+0000 7f4000a00640 1 -- 172.21.15.44:0/1159049988 wait complete. 2024-07-17T08:35:28.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:35:21.767746+0000', 'modified': '2024-07-17T08:35:24.841167+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10083': {'gid': 10083, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/1413771882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 1413771882}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1413771882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10083, 'qdb_cluster': [10083]} 2024-07-17T08:35:28.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:35:28.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:35:29.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 -- 172.21.15.44:0/1398668285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca80cd0b0 msgr2=0x7f2ca806ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 --2- 172.21.15.44:0/1398668285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca806ec70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2c94005fd0 tx=0x7f2c9402a4f0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 -- 172.21.15.44:0/1398668285 shutdown_connections 2024-07-17T08:35:29.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 --2- 172.21.15.44:0/1398668285 >> v2:172.21.15.175:3300/0 conn(0x7f2ca8152300 0x7f2ca806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 --2- 172.21.15.44:0/1398668285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca806ec70 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 -- 172.21.15.44:0/1398668285 >> 172.21.15.44:0/1398668285 conn(0x7f2ca80cf0d0 msgr2=0x7f2ca80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 -- 172.21.15.44:0/1398668285 shutdown_connections 2024-07-17T08:35:29.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.008+0000 7f2cad600640 1 -- 172.21.15.44:0/1398668285 wait complete. 2024-07-17T08:35:29.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.010+0000 7f2cad600640 1 Processor -- start 2024-07-17T08:35:29.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.010+0000 7f2cad600640 1 -- start start 2024-07-17T08:35:29.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca80adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 0x7f2ca80713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80cb6e0 con 0x7f2ca8152300 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80cb560 con 0x7f2ca80cd0b0 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cad600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80cb860 con 0x7f2ca8151f20 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.011+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44868/0 (socket says 172.21.15.44:44868) 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 -- 172.21.15.44:0/710713284 learned_addr learned my addr 172.21.15.44:0/710713284 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca80adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2ca7400640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 0x7f2ca80713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 -- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 msgr2=0x7f2ca80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 0x7f2ca80713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 -- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ca80cd0b0 msgr2=0x7f2ca80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca80adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 -- 172.21.15.44:0/710713284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81f3e90 con 0x7f2ca8152300 2024-07-17T08:35:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2ca7400640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 0x7f2ca80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.012+0000 7f2cacc00640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e990 tx=0x7f2ca000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.013+0000 7f2c9fe00640 1 -- 172.21.15.44:0/710713284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0004d20 con 0x7f2ca8152300 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.013+0000 7f2c9fe00640 1 -- 172.21.15.44:0/710713284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca000cb00 con 0x7f2ca8152300 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.013+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81f4170 con 0x7f2ca8152300 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.013+0000 7f2c9fe00640 1 -- 172.21.15.44:0/710713284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0005020 con 0x7f2ca8152300 2024-07-17T08:35:29.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.013+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81f4730 con 0x7f2ca8152300 2024-07-17T08:35:29.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.016+0000 7f2c9fe00640 1 -- 172.21.15.44:0/710713284 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f2ca00051c0 con 0x7f2ca8152300 2024-07-17T08:35:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.017+0000 7f2cb19bd740 1 Processor -- start 2024-07-17T08:35:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.017+0000 7f2c9fe00640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 0x7f2c88079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.017+0000 7f2cb19bd740 1 -- start start 2024-07-17T08:35:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.017+0000 7f2c9fe00640 1 -- 172.21.15.44:0/710713284 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f2ca0099180 con 0x7f2ca8152300 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.017+0000 7f2ca7400640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 0x7f2c88079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55906399cfc0 0x55906399d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x559063999c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5590638e4d60 con 0x559063974280 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5590638d4260 con 0x559063999790 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5590638d82a0 con 0x55906399cfc0 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44884/0 (socket says 172.21.15.44:44884) 2024-07-17T08:35:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 -- 172.21.15.44:0/2162538310 learned_addr learned my addr 172.21.15.44:0/2162538310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55906399cfc0 0x55906399d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2cacc00640 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x559063999c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 -- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55906399cfc0 msgr2=0x55906399d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55906399cfc0 0x55906399d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 -- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 msgr2=0x559063999c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.018+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x559063999c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.019+0000 7f2ca7400640 1 -- 172.21.15.44:0/2162538310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559063638850 con 0x559063974280 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.019+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2c940099f0 tx=0x7f2c94003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a192fc26ecbac3 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.019+0000 7f2c93e00640 1 -- 172.21.15.44:0/2162538310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940397e0 con 0x559063974280 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.019+0000 7f2c93e00640 1 -- 172.21.15.44:0/2162538310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c94039980 con 0x559063974280 2024-07-17T08:35:29.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.019+0000 7f2c93e00640 1 -- 172.21.15.44:0/2162538310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94039d50 con 0x559063974280 2024-07-17T08:35:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2162538310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 msgr2=0x559063999250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2c940099f0 tx=0x7f2c94003a00 comp rx=0 tx=0).stop 2024-07-17T08:35:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2ca7400640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 0x7f2c88079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c9403d000 tx=0x7f2c9403a000 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2162538310 shutdown_connections 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x559063999c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063974280 0x559063999250 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2162538310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55906399cfc0 0x55906399d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2162538310 >> 172.21.15.44:0/2162538310 conn(0x55906396f460 msgr2=0x559063972e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2162538310 shutdown_connections 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.020+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2162538310 wait complete. 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.021+0000 7f2cb19bd740 1 -- start start 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.021+0000 7f2cb19bd740 1 Processor -- start 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x5590639cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906399ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559063647620 con 0x55906399cfc0 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55906387bf90 con 0x559063999790 2024-07-17T08:35:29.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55906396db60 con 0x5590639c7a20 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2cb19bd740 10 client.0 _ll_register_callbacks cb 0x5590638a1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2ca7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44886/0 (socket says 172.21.15.44:44886) 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.022+0000 7f2ca7400640 1 -- 172.21.15.44:0/2912506640 learned_addr learned my addr 172.21.15.44:0/2912506640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 -- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 msgr2=0x55906399ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906399ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 -- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 msgr2=0x5590639cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x5590639cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 -- 172.21.15.44:0/2912506640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590636af6b0 con 0x55906399cfc0 2024-07-17T08:35:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2ca7400640 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2c9402aae0 tx=0x7f2c94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b9fe5514eab429 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2c90c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94035070 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f2c94039c20 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.023+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590638ab900 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f2c9405d050 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 10 client.10200 fetch_fsmap learned FSMap version 114 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5590639c8f80 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5590638ab900 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 114) ==== 3502+0+0 (secure 0 0 0) 0x7f2c94060020 con 0x55906399cfc0 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 10 client.10200 fetch_fsmap finished waiting for FSMap version 114 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 10 client.10200 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 20 client.10200 populate_metadata read hostname 'smithi044' 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063983cd0 0x5590639840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 4 client.10200 mds_command: new command op to 10083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.024+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 --> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590638e6cb0 con 0x559063983cd0 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.025+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063983cd0 0x5590639840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.025+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063983cd0 0x5590639840b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a458c7c06fe3f4ea server_cookie=c0bdb20d48f574f3 in_seq=0 out_seq=0 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.025+0000 7f2c90c00640 10 client.10200 ms_handle_connect on v2:172.21.15.44:6832/1413771882 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2c90c00640 1 -- 172.21.15.44:0/2912506640 <== mds.0 v2:172.21.15.44:6832/1413771882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5590638e6cb0 con 0x559063983cd0 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2c90c00640 10 client.10200 handle_command_reply: tid=0 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 client.10200 shutdown 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 msgr2=0x5590639ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2c9402aae0 tx=0x7f2c94009260 comp rx=0 tx=0).stop 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 shutdown_connections 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063983cd0 0x5590639840b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906399ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55906399cfc0 0x5590639ae490 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/2912506640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559063999790 0x5590639cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2c90c00640 0 client.10200 ms_handle_reset on v2:172.21.15.44:6832/1413771882 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.026+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 >> 172.21.15.44:0/2912506640 conn(0x559063974280 msgr2=0x559063970d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.027+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 shutdown_connections 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.027+0000 7f2cb19bd740 1 -- 172.21.15.44:0/2912506640 wait complete. 2024-07-17T08:35:29.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.027+0000 7f2cb19bd740 20 client.10200 trim_cache size 0 max 16384 2024-07-17T08:35:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.047+0000 7f2cb19bd740 1 Processor -- start 2024-07-17T08:35:29.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.047+0000 7f2cb19bd740 1 -- start start 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559063999790 0x559063970ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8200430 con 0x5590639c7a20 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80acc90 con 0x55906399cfc0 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.048+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca81ffde0 con 0x559063999790 2024-07-17T08:35:29.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44890/0 (socket says 172.21.15.44:44890) 2024-07-17T08:35:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 -- 172.21.15.44:0/145021881 learned_addr learned my addr 172.21.15.44:0/145021881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559063999790 0x559063970ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2ca7400640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 -- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559063999790 msgr2=0x559063970ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559063999790 0x559063970ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 -- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 msgr2=0x5590639828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 -- 172.21.15.44:0/145021881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55906357e970 con 0x5590639c7a20 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2ca7400640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.049+0000 7f2cacc00640 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00970a0 tx=0x7f2ca00610c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95ff6b350fe47ca server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2c93400640 1 -- 172.21.15.44:0/145021881 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca001f030 con 0x5590639c7a20 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2c93400640 1 -- 172.21.15.44:0/145021881 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca007b070 con 0x5590639c7a20 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2c93400640 1 -- 172.21.15.44:0/145021881 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0060e10 con 0x5590639c7a20 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 -- 172.21.15.44:0/145021881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 msgr2=0x559063982e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00970a0 tx=0x7f2ca00610c0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 -- 172.21.15.44:0/145021881 shutdown_connections 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5590639c7a20 0x559063982e00 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/145021881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559063999790 0x559063970ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 -- 172.21.15.44:0/145021881 >> 172.21.15.44:0/145021881 conn(0x559063974280 msgr2=0x5590639adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.050+0000 7f2cb19bd740 1 -- 172.21.15.44:0/145021881 shutdown_connections 2024-07-17T08:35:29.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.051+0000 7f2cb19bd740 1 -- 172.21.15.44:0/145021881 wait complete. 2024-07-17T08:35:29.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.051+0000 7f2cb19bd740 1 -- start start 2024-07-17T08:35:29.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.051+0000 7f2cb19bd740 1 Processor -- start 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.052+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.052+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cb19bd740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906398e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5590639c8c80 con 0x559063999790 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559063647620 con 0x55906399cfc0 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cb19bd740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x559063998d80 con 0x5590639c7a20 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cb19bd740 10 client.0 _ll_register_callbacks cb 0x7f2c94003900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44902/0 (socket says 172.21.15.44:44902) 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 -- 172.21.15.44:0/682656852 learned_addr learned my addr 172.21.15.44:0/682656852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7400640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2cacc00640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906398e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 -- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 msgr2=0x55906398e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906398e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 -- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 msgr2=0x5590639d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.053+0000 7f2ca7e00640 1 -- 172.21.15.44:0/682656852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55906398f9d0 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2ca7e00640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x559063993560 tx=0x7f2c980051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0109f42265034fd server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2c90c00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98007030 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f2c980071d0 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca007b070 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f2c9802b050 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2cb19bd740 10 client.10206 fetch_fsmap learned FSMap version 114 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.054+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5590639d1b40 con 0x559063999790 2024-07-17T08:35:29.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca007b070 con 0x559063999790 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 114) ==== 3502+0+0 (secure 0 0 0) 0x7f2c9802e020 con 0x559063999790 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 10 client.10206 fetch_fsmap finished waiting for FSMap version 114 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 10 client.10206 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 20 client.10206 populate_metadata read hostname 'smithi044' 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063a34ee0 0x5590639b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 4 client.10206 mds_command: new command op to 10083 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 --> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] -- command(tid 0: {"prefix": "status"}) -- 0x5590638e2750 con 0x559063a34ee0 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.055+0000 7f2ca7400640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063a34ee0 0x5590639b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2ca7400640 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063a34ee0 0x5590639b0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef55a35001593ba server_cookie=30f0e249696dbb3f in_seq=0 out_seq=0 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2c90c00640 10 client.10206 ms_handle_connect on v2:172.21.15.44:6832/1413771882 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2c90c00640 1 -- 172.21.15.44:0/682656852 <== mds.0 v2:172.21.15.44:6832/1413771882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x5590638e2750 con 0x559063a34ee0 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2c90c00640 10 client.10206 handle_command_reply: tid=0 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 client.10206 shutdown 2024-07-17T08:35:29.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 msgr2=0x5590639dcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x559063993560 tx=0x7f2c980051e0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 shutdown_connections 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:6832/1413771882,v1:172.21.15.44:6835/1413771882] conn(0x559063a34ee0 0x5590639b0d70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5590639c7a20 0x55906398e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55906399cfc0 0x5590639d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 --2- 172.21.15.44:0/682656852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559063999790 0x5590639dcb30 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2c90c00640 0 client.10206 ms_handle_reset on v2:172.21.15.44:6832/1413771882 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 >> 172.21.15.44:0/682656852 conn(0x559063974280 msgr2=0x5590639d28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 shutdown_connections 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.056+0000 7f2cb19bd740 1 -- 172.21.15.44:0/682656852 wait complete. 2024-07-17T08:35:29.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cb19bd740 20 client.10206 trim_cache size 0 max 16384 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 10083, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7512439119999996, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 114, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 223, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 223, 2024-07-17T08:35:29.063 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.680415812 2024-07-17T08:35:29.064 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:35:29.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 msgr2=0x7f2c88079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 0x7f2c88079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c9403d000 tx=0x7f2c9403a000 comp rx=0 tx=0).stop 2024-07-17T08:35:29.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 msgr2=0x7f2ca81f7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e990 tx=0x7f2ca000ee50 comp rx=0 tx=0).stop 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 shutdown_connections 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ca8152300 0x7f2ca81f7440 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ca8151f20 0x7f2ca80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2c88076cd0 0x7f2c88079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 --2- 172.21.15.44:0/710713284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ca80cd0b0 0x7f2ca80adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 >> 172.21.15.44:0/710713284 conn(0x7f2ca80cf0d0 msgr2=0x7f2ca806df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 shutdown_connections 2024-07-17T08:35:29.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.057+0000 7f2cad600640 1 -- 172.21.15.44:0/710713284 wait complete. 2024-07-17T08:35:29.078 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 -- 172.21.15.44:0/3521710419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 msgr2=0x7f310c0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 --2- 172.21.15.44:0/3521710419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c0ad790 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f310c15edd0 tx=0x7f30fc02a560 comp rx=0 tx=0).stop 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 -- 172.21.15.44:0/3521710419 shutdown_connections 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 --2- 172.21.15.44:0/3521710419 >> v2:172.21.15.175:3300/0 conn(0x7f310c0ac930 0x7f310c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 --2- 172.21.15.44:0/3521710419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c0ad790 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 -- 172.21.15.44:0/3521710419 >> 172.21.15.44:0/3521710419 conn(0x7f310c0b0fb0 msgr2=0x7f310c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 -- 172.21.15.44:0/3521710419 shutdown_connections 2024-07-17T08:35:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.276+0000 7f3112200640 1 -- 172.21.15.44:0/3521710419 wait complete. 2024-07-17T08:35:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.278+0000 7f3112200640 1 Processor -- start 2024-07-17T08:35:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.278+0000 7f3112200640 1 -- start start 2024-07-17T08:35:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f310c0ac930 0x7f310c1ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f310c0adcd0 0x7f310c14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f310c0cdf60 con 0x7f310c1522c0 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f310c0cdde0 con 0x7f310c0adcd0 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f3112200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f310c0ce0e0 con 0x7f310c0ac930 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f310be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f310be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44918/0 (socket says 172.21.15.44:44918) 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.279+0000 7f310be00640 1 -- 172.21.15.44:0/2505925048 learned_addr learned my addr 172.21.15.44:0/2505925048 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310aa00640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f310c0adcd0 0x7f310c14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 -- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f310c0ac930 msgr2=0x7f310c1ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f310c0ac930 0x7f310c1ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 -- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f310c0adcd0 msgr2=0x7f310c14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f310c0adcd0 0x7f310c14b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 -- 172.21.15.44:0/2505925048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c074580 con 0x7f310c1522c0 2024-07-17T08:35:29.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f310be00640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f310400b9e0 tx=0x7f310400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104019070 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f310400cbb0 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104014440 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.280+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c1f53d0 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.281+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c14aa80 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.282+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c0af740 con 0x7f310c1522c0 2024-07-17T08:35:29.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.283+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f310401c070 con 0x7f310c1522c0 2024-07-17T08:35:29.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.284+0000 7f3103400640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 0x7f30e8079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:29.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.284+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f3104098600 con 0x7f310c1522c0 2024-07-17T08:35:29.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.285+0000 7f310b400640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 0x7f30e8079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:29.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.286+0000 7f310b400640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 0x7f30e8079210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30fc033000 tx=0x7f30fc0046c0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:29.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.288+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f310c0af740 con 0x7f310c1522c0 2024-07-17T08:35:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.554+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f310c0ce630 con 0x7f310c1522c0 2024-07-17T08:35:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.555+0000 7f3103400640 1 -- 172.21.15.44:0/2505925048 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) ==== 53+0+73 (secure 0 0 0) 0x7f31040615d0 con 0x7f310c1522c0 2024-07-17T08:35:29.556 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 msgr2=0x7f30e8079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 0x7f30e8079210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30fc033000 tx=0x7f30fc0046c0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 msgr2=0x7f310c071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f310400b9e0 tx=0x7f310400bea0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 shutdown_connections 2024-07-17T08:35:29.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f310c1522c0 0x7f310c071620 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f30e8076d50 0x7f30e8079210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f310c0adcd0 0x7f310c14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 --2- 172.21.15.44:0/2505925048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f310c0ac930 0x7f310c1ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 >> 172.21.15.44:0/2505925048 conn(0x7f310c0b0fb0 msgr2=0x7f310c15e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.557+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 shutdown_connections 2024-07-17T08:35:29.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29.558+0000 7f3112200640 1 -- 172.21.15.44:0/2505925048 wait complete. 2024-07-17T08:35:29.579 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:35:29.579 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:35:29.579 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:35:29.580 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:35:29.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:29.580 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:35:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36528sec preferred_lft 36528sec 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:35:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:35:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:35:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:35:29.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:29.635 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:35:29.714 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:35:29.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:29.719 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:35:29.775 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:35:29.775 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:35:29.775 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:35:29.775 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:35:29.775 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:35:29.776 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:35:29.776 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:35:29.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:35:29.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:35:29.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:35:29.777 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:35:29.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:35:29.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:35:30.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:30.005 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:35:30.056 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:35:30.056 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:35:30.057 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:35:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:30.058 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:35:30.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:35:30.139 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:30.194 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:35:30.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:35:30.195 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:30.246 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:35:30.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:30.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:35:30.386 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:35:30.386 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:35:30.386 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:35:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:35:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:35:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:35:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:35:30.409 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:35:30.439 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:35:30.445 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:35:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:35:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:35:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:35:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:35:30.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:35:30.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:30.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:30.528 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:35:30.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:35:30.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:35:30.677 INFO:teuthology.orchestra.run.smithi044.stdout:client7694 2024-07-17T08:35:30.678 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:30.678 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:30.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-17T08:35:30.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 -- 172.21.15.44:0/2570462071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de80ad100 msgr2=0x7f8de815aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:30.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 --2- 172.21.15.44:0/2570462071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de80ad100 0x7f8de815aa00 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005fd0 tx=0x7f8dd002b910 comp rx=0 tx=0).stop 2024-07-17T08:35:30.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 -- 172.21.15.44:0/2570462071 shutdown_connections 2024-07-17T08:35:30.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 --2- 172.21.15.44:0/2570462071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de80ad100 0x7f8de815aa00 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:30.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 -- 172.21.15.44:0/2570462071 >> 172.21.15.44:0/2570462071 conn(0x7f8de80b0eb0 msgr2=0x7f8de80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:30.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.942+0000 7f8decc00640 1 -- 172.21.15.44:0/2570462071 shutdown_connections 2024-07-17T08:35:30.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.943+0000 7f8decc00640 1 -- 172.21.15.44:0/2570462071 wait complete. 2024-07-17T08:35:30.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.944+0000 7f8decc00640 1 Processor -- start 2024-07-17T08:35:30.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.944+0000 7f8decc00640 1 -- start start 2024-07-17T08:35:30.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80ad100 0x7f8de8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 0x7f8de81f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80b32e0 con 0x7f8de806f510 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80b3160 con 0x7f8de80b3e20 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.945+0000 7f8decc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8de80b3460 con 0x7f8de80ad100 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44938/0 (socket says 172.21.15.44:44938) 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 -- 172.21.15.44:0/30038032 learned_addr learned my addr 172.21.15.44:0/30038032 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8ddfe00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80ad100 0x7f8de8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:30.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de6800640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 0x7f8de81f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 -- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80ad100 msgr2=0x7f8de8156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80ad100 0x7f8de8156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 -- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 msgr2=0x7f8de81f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 0x7f8de81f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de5e00640 1 -- 172.21.15.44:0/30038032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de81e8010 con 0x7f8de806f510 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.946+0000 7f8de6800640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 0x7f8de81f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8de5e00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c3d0 tx=0x7f8dd800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8017070 con 0x7f8de806f510 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd80053c0 con 0x7f8de806f510 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8003850 con 0x7f8de806f510 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de81e82f0 con 0x7f8de806f510 2024-07-17T08:35:30.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.947+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de81e88b0 con 0x7f8de806f510 2024-07-17T08:35:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.948+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de815aa00 con 0x7f8de806f510 2024-07-17T08:35:30.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.950+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f8dd801a070 con 0x7f8de806f510 2024-07-17T08:35:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.951+0000 7f8dde000640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 0x7f8dc4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:30.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.951+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f8dd8097c30 con 0x7f8de806f510 2024-07-17T08:35:30.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.951+0000 7f8ddfe00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 0x7f8dc4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:30.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.953+0000 7f8ddfe00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 0x7f8dc4079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0035000 tx=0x7f8dd0004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:30.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:30.955+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8dd8060c10 con 0x7f8de806f510 2024-07-17T08:35:31.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.213+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8de81577f0 con 0x7f8de806f510 2024-07-17T08:35:31.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.383+0000 7f8dde000640 1 -- 172.21.15.44:0/30038032 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v115) ==== 136+0+0 (secure 0 0 0) 0x7f8dd8060510 con 0x7f8de806f510 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 msgr2=0x7f8dc4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 0x7f8dc4079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0035000 tx=0x7f8dd0004730 comp rx=0 tx=0).stop 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 msgr2=0x7f8de81f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c3d0 tx=0x7f8dd800c890 comp rx=0 tx=0).stop 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 shutdown_connections 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8dc4076cd0 0x7f8dc4079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80b3e20 0x7f8de81f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80ad100 0x7f8de8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 --2- 172.21.15.44:0/30038032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8de806f510 0x7f8de81f6fb0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 >> 172.21.15.44:0/30038032 conn(0x7f8de80b0eb0 msgr2=0x7f8de81ea150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 shutdown_connections 2024-07-17T08:35:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.387+0000 7f8decc00640 1 -- 172.21.15.44:0/30038032 wait complete. 2024-07-17T08:35:31.411 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:35:31.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 -- 172.21.15.44:0/445223054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa50003a50 msgr2=0x7faa50003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 --2- 172.21.15.44:0/445223054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa50003a50 0x7faa50003e70 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009b80 tx=0x7faa50005830 comp rx=0 tx=0).stop 2024-07-17T08:35:31.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6de00640 1 -- 172.21.15.44:0/445223054 reap_dead start 2024-07-17T08:35:31.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 -- 172.21.15.44:0/445223054 shutdown_connections 2024-07-17T08:35:31.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 -- 172.21.15.44:0/445223054 >> 172.21.15.44:0/445223054 conn(0x7faa680b3020 msgr2=0x7faa680b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:31.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 -- 172.21.15.44:0/445223054 shutdown_connections 2024-07-17T08:35:31.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.607+0000 7faa6fe00640 1 -- 172.21.15.44:0/445223054 wait complete. 2024-07-17T08:35:31.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.609+0000 7faa6fe00640 1 Processor -- start 2024-07-17T08:35:31.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.609+0000 7faa6fe00640 1 -- start start 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50003a50 0x7faa6815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa6814f020 0x7faa680b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faa6806fde0 con 0x7faa680b4ad0 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa680cffd0 con 0x7faa50003a50 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faa6806ff60 con 0x7faa6814f020 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.610+0000 7faa6de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:31.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44958/0 (socket says 172.21.15.44:44958) 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 -- 172.21.15.44:0/3409886573 learned_addr learned my addr 172.21.15.44:0/3409886573 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6d400640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50003a50 0x7faa6815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6ca00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa6814f020 0x7faa680b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 -- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa6814f020 msgr2=0x7faa680b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa6814f020 0x7faa680b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 -- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50003a50 msgr2=0x7faa6815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50003a50 0x7faa6815ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 -- 172.21.15.44:0/3409886573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa680b4fe0 con 0x7faa680b4ad0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.611+0000 7faa6de00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7faa60004a50 tx=0x7faa6000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.612+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60018070 con 0x7faa680b4ad0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.612+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa6000ae40 con 0x7faa680b4ad0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.612+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600072d0 con 0x7faa680b4ad0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.612+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa680b52c0 con 0x7faa680b4ad0 2024-07-17T08:35:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.612+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa680b5880 con 0x7faa680b4ad0 2024-07-17T08:35:31.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.613+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa680cf560 con 0x7faa680b4ad0 2024-07-17T08:35:31.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.614+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7faa60007470 con 0x7faa680b4ad0 2024-07-17T08:35:31.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.615+0000 7faa65600640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 0x7faa48079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:31.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.616+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7faa600986b0 con 0x7faa680b4ad0 2024-07-17T08:35:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.616+0000 7faa6d400640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 0x7faa48079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.617+0000 7faa6d400640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 0x7faa48079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa5c035000 tx=0x7faa5c02d040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:31.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.620+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faa60061600 con 0x7faa680b4ad0 2024-07-17T08:35:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.830+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa6815bf80 con 0x7faa680b4ad0 2024-07-17T08:35:31.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.831+0000 7faa65600640 1 -- 172.21.15.44:0/3409886573 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v223) ==== 84+0+209 (secure 0 0 0) 0x7faa60060f00 con 0x7faa680b4ad0 2024-07-17T08:35:31.833 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:31.833 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:35:31.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 27, 2024-07-17T08:35:31.834 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "cephfs_metadata" 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 28, 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "cephfs_data" 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:31.835 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:35:31.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 msgr2=0x7faa48079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 0x7faa48079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa5c035000 tx=0x7faa5c02d040 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 msgr2=0x7faa680b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7faa60004a50 tx=0x7faa6000a430 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 shutdown_connections 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa680b4ad0 0x7faa680b6080 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa6814f020 0x7faa680b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa48076cd0 0x7faa48079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 --2- 172.21.15.44:0/3409886573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa50003a50 0x7faa6815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 >> 172.21.15.44:0/3409886573 conn(0x7faa680b3020 msgr2=0x7faa680b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:31.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 shutdown_connections 2024-07-17T08:35:31.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:31.834+0000 7faa6fe00640 1 -- 172.21.15.44:0/3409886573 wait complete. 2024-07-17T08:35:31.856 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-17T08:35:31.857 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 -- 172.21.15.44:0/1405163628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 msgr2=0x7f08000ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 --2- 172.21.15.44:0/1405163628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f08000ae110 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f07f0005fd0 tx=0x7f07f002ae30 comp rx=0 tx=0).stop 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 -- 172.21.15.44:0/1405163628 shutdown_connections 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 --2- 172.21.15.44:0/1405163628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f08000ae110 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 --2- 172.21.15.44:0/1405163628 >> v2:172.21.15.175:3300/0 conn(0x7f08001537d0 0x7f080006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 -- 172.21.15.44:0/1405163628 >> 172.21.15.44:0/1405163628 conn(0x7f08000cf050 msgr2=0x7f08000cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 -- 172.21.15.44:0/1405163628 shutdown_connections 2024-07-17T08:35:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.072+0000 7f07ff200640 1 -- 172.21.15.44:0/1405163628 wait complete. 2024-07-17T08:35:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.074+0000 7f07ff200640 1 Processor -- start 2024-07-17T08:35:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.074+0000 7f07ff200640 1 -- start start 2024-07-17T08:35:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08001537d0 0x7f08000b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800153bb0 0x7f08000b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08000ae7d0 con 0x7f080006ec10 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08000ae650 con 0x7f08001537d0 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08000ae950 con 0x7f0800153bb0 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44970/0 (socket says 172.21.15.44:44970) 2024-07-17T08:35:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 -- 172.21.15.44:0/3161633608 learned_addr learned my addr 172.21.15.44:0/3161633608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fd400640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08001537d0 0x7f08000b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 -- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800153bb0 msgr2=0x7f08000b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800153bb0 0x7f08000b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 -- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08001537d0 msgr2=0x7f08000b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08001537d0 0x7f08000b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.075+0000 7f07fde00640 1 -- 172.21.15.44:0/3161633608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001f77c0 con 0x7f080006ec10 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07fde00640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c8c0 tx=0x7f07e800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8004220 con 0x7f080006ec10 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07e80079b0 con 0x7f080006ec10 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001f7aa0 con 0x7f080006ec10 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8007de0 con 0x7f080006ec10 2024-07-17T08:35:32.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.076+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001f8060 con 0x7f080006ec10 2024-07-17T08:35:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.078+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08000ae110 con 0x7f080006ec10 2024-07-17T08:35:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.079+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f07e8005020 con 0x7f080006ec10 2024-07-17T08:35:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.080+0000 7f07f6000640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 0x7f07dc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.081+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 6677+0+0 (secure 0 0 0) 0x7f07e8098e70 con 0x7f080006ec10 2024-07-17T08:35:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.081+0000 7f07fd400640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 0x7f07dc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.082+0000 7f07fd400640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 0x7f07dc079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f0004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:32.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.085+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f07e8014030 con 0x7f080006ec10 2024-07-17T08:35:32.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.293+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f08000cee30 con 0x7f080006ec10 2024-07-17T08:35:32.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.408+0000 7f07f6000640 1 -- 172.21.15.44:0/3161633608 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v224) ==== 144+0+0 (secure 0 0 0) 0x7f07e8061cb0 con 0x7f080006ec10 2024-07-17T08:35:32.411 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_metadata' created 2024-07-17T08:35:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.412+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 msgr2=0x7f07dc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.412+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 0x7f07dc079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f0004730 comp rx=0 tx=0).stop 2024-07-17T08:35:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.412+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 msgr2=0x7f0800152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.412+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c8c0 tx=0x7f07e800cd80 comp rx=0 tx=0).stop 2024-07-17T08:35:32.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 shutdown_connections 2024-07-17T08:35:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800153bb0 0x7f08000b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f07dc076cd0 0x7f07dc079190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08001537d0 0x7f08000b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 --2- 172.21.15.44:0/3161633608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f080006ec10 0x7f0800152970 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 >> 172.21.15.44:0/3161633608 conn(0x7f08000cf050 msgr2=0x7f080006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:32.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 shutdown_connections 2024-07-17T08:35:32.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.413+0000 7f07ff200640 1 -- 172.21.15.44:0/3161633608 wait complete. 2024-07-17T08:35:32.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:35:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 -- 172.21.15.44:0/313077763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0614151f20 msgr2=0x7f061415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 --2- 172.21.15.44:0/313077763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0614151f20 0x7f061415c140 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f06140ccc20 tx=0x7f060402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 -- 172.21.15.44:0/313077763 shutdown_connections 2024-07-17T08:35:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 --2- 172.21.15.44:0/313077763 >> v2:172.21.15.175:3300/0 conn(0x7f061406f380 0x7f06140cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 --2- 172.21.15.44:0/313077763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0614151f20 0x7f061415c140 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 -- 172.21.15.44:0/313077763 >> 172.21.15.44:0/313077763 conn(0x7f06140cf0d0 msgr2=0x7f06140cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 -- 172.21.15.44:0/313077763 shutdown_connections 2024-07-17T08:35:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.664+0000 7f0613200640 1 -- 172.21.15.44:0/313077763 wait complete. 2024-07-17T08:35:32.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.666+0000 7f0613200640 1 Processor -- start 2024-07-17T08:35:32.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.666+0000 7f0613200640 1 -- start start 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06140cd0b0 0x7f0614154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 0x7f061409c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06140cb740 con 0x7f061406f380 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06140cb5c0 con 0x7f06140cd0b0 2024-07-17T08:35:32.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0613200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06140cb8c0 con 0x7f0614151f20 2024-07-17T08:35:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0611e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0611e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44988/0 (socket says 172.21.15.44:44988) 2024-07-17T08:35:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0611e00640 1 -- 172.21.15.44:0/966546579 learned_addr learned my addr 172.21.15.44:0/966546579 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.667+0000 7f0612800640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 0x7f061409c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 -- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 msgr2=0x7f061409c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 0x7f061409c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611400640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06140cd0b0 0x7f0614154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 -- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06140cd0b0 msgr2=0x7f0614154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06140cd0b0 0x7f0614154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 -- 172.21.15.44:0/966546579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06141439f0 con 0x7f061406f380 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0612800640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 0x7f061409c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f0611e00640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f06040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.668+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060403e030 con 0x7f061406f380 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.669+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06040047a0 con 0x7f061406f380 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.669+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0614155310 con 0x7f061406f380 2024-07-17T08:35:32.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.669+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604035040 con 0x7f061406f380 2024-07-17T08:35:32.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.669+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141542f0 con 0x7f061406f380 2024-07-17T08:35:32.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.670+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06140cb080 con 0x7f061406f380 2024-07-17T08:35:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.671+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f0604041030 con 0x7f061406f380 2024-07-17T08:35:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.672+0000 7f060a000640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 0x7f05f40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.673+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(224..224 src has 1..224) ==== 7050+0+0 (secure 0 0 0) 0x7f06040b96f0 con 0x7f061406f380 2024-07-17T08:35:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.673+0000 7f0611400640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 0x7f05f40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.674+0000 7f0611400640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 0x7f05f40791c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f061409d210 tx=0x7f0600009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:32.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.676+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f06141439f0 con 0x7f061406f380 2024-07-17T08:35:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:32.899+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0614074ee0 con 0x7f061406f380 2024-07-17T08:35:33.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.416+0000 7f060a000640 1 -- 172.21.15.44:0/966546579 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v225) ==== 176+0+0 (secure 0 0 0) 0x7f061415c680 con 0x7f061406f380 2024-07-17T08:35:33.418 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_data' created 2024-07-17T08:35:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.421+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 msgr2=0x7f05f40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.422+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 0x7f05f40791c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f061409d210 tx=0x7f0600009040 comp rx=0 tx=0).stop 2024-07-17T08:35:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.422+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 msgr2=0x7f06141434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:33.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.422+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f06040039e0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 shutdown_connections 2024-07-17T08:35:33.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0614151f20 0x7f061409c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f05f4076d00 0x7f05f40791c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06140cd0b0 0x7f0614154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 --2- 172.21.15.44:0/966546579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f061406f380 0x7f06141434b0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.423+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 >> 172.21.15.44:0/966546579 conn(0x7f06140cf0d0 msgr2=0x7f06140cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.424+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 shutdown_connections 2024-07-17T08:35:33.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.424+0000 7f0613200640 1 -- 172.21.15.44:0/966546579 wait complete. 2024-07-17T08:35:33.463 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.642+0000 7f8261e00640 1 -- 172.21.15.44:0/441230297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c06eb90 msgr2=0x7f825c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.642+0000 7f8261e00640 1 --2- 172.21.15.44:0/441230297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c06eb90 0x7f825c15a5a0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f824c009fd0 tx=0x7f824c02b120 comp rx=0 tx=0).stop 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.643+0000 7f8261e00640 1 -- 172.21.15.44:0/441230297 shutdown_connections 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.643+0000 7f8261e00640 1 --2- 172.21.15.44:0/441230297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c06eb90 0x7f825c15a5a0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.643+0000 7f8261e00640 1 -- 172.21.15.44:0/441230297 >> 172.21.15.44:0/441230297 conn(0x7f825c0b0eb0 msgr2=0x7f825c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.643+0000 7f8261e00640 1 -- 172.21.15.44:0/441230297 shutdown_connections 2024-07-17T08:35:33.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.643+0000 7f8261e00640 1 -- 172.21.15.44:0/441230297 wait complete. 2024-07-17T08:35:33.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.645+0000 7f8261e00640 1 Processor -- start 2024-07-17T08:35:33.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.645+0000 7f8261e00640 1 -- start start 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c06eb90 0x7f825c14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f825c0ac3a0 0x7f825c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f825c14fd60 con 0x7f825c0b3280 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f825c06f8e0 con 0x7f825c06eb90 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f8261e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f825c14fee0 con 0x7f825c0ac3a0 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f825b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f825b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45018/0 (socket says 172.21.15.44:45018) 2024-07-17T08:35:33.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.646+0000 7f825b400640 1 -- 172.21.15.44:0/2562513364 learned_addr learned my addr 172.21.15.44:0/2562513364 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825a000640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f825c0ac3a0 0x7f825c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825aa00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c06eb90 0x7f825c14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 -- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f825c0ac3a0 msgr2=0x7f825c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f825c0ac3a0 0x7f825c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 -- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c06eb90 msgr2=0x7f825c14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c06eb90 0x7f825c14e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 -- 172.21.15.44:0/2562513364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1e73a0 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f825b400640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f825400c410 tx=0x7f825400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254011790 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8254017070 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254011d00 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.647+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1f5340 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.648+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c1e7940 con 0x7f825c0b3280 2024-07-17T08:35:33.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.649+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c15a5a0 con 0x7f825c0b3280 2024-07-17T08:35:33.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.650+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f8254007020 con 0x7f825c0b3280 2024-07-17T08:35:33.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.651+0000 7f8253400640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 0x7f82380791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:33.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.651+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 7435+0+0 (secure 0 0 0) 0x7f8254099010 con 0x7f825c0b3280 2024-07-17T08:35:33.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.652+0000 7f825aa00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 0x7f82380791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:33.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.653+0000 7f825aa00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 0x7f82380791c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824c033000 tx=0x7f824c004ab0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:33.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.656+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8254021020 con 0x7f825c0b3280 2024-07-17T08:35:33.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:33.906+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f825c1e7b10 con 0x7f825c0b3280 2024-07-17T08:35:34.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.465+0000 7f8253400640 1 -- 172.21.15.44:0/2562513364 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 29 and data pool 30 v116) ==== 183+0+0 (secure 0 0 0) 0x7f8254061e80 con 0x7f825c0b3280 2024-07-17T08:35:34.469 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 29 and data pool 30 2024-07-17T08:35:34.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.476+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 msgr2=0x7f82380791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.476+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 0x7f82380791c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824c033000 tx=0x7f824c004ab0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.476+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 msgr2=0x7f825c158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.476+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f825400c410 tx=0x7f825400c8d0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 shutdown_connections 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f825c0b3280 0x7f825c158a90 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f825c0ac3a0 0x7f825c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f8238076d00 0x7f82380791c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 --2- 172.21.15.44:0/2562513364 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f825c06eb90 0x7f825c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.478+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 >> 172.21.15.44:0/2562513364 conn(0x7f825c0b0eb0 msgr2=0x7f825c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.479+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 shutdown_connections 2024-07-17T08:35:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.479+0000 7f8261e00640 1 -- 172.21.15.44:0/2562513364 wait complete. 2024-07-17T08:35:34.513 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.687+0000 7f7630400640 1 -- 172.21.15.44:0/3593681825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76281522c0 msgr2=0x7f76280ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.687+0000 7f7630400640 1 --2- 172.21.15.44:0/3593681825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76281522c0 0x7f76280ad790 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f761c009830 tx=0x7f761c02a510 comp rx=0 tx=0).stop 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.687+0000 7f7630400640 1 -- 172.21.15.44:0/3593681825 shutdown_connections 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.687+0000 7f7630400640 1 --2- 172.21.15.44:0/3593681825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76281522c0 0x7f76280ad790 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.687+0000 7f7630400640 1 -- 172.21.15.44:0/3593681825 >> 172.21.15.44:0/3593681825 conn(0x7f76280b0fb0 msgr2=0x7f76280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.688+0000 7f7630400640 1 -- 172.21.15.44:0/3593681825 shutdown_connections 2024-07-17T08:35:34.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.688+0000 7f7630400640 1 -- 172.21.15.44:0/3593681825 wait complete. 2024-07-17T08:35:34.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.689+0000 7f7630400640 1 Processor -- start 2024-07-17T08:35:34.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.690+0000 7f7630400640 1 -- start start 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 0x7f76281f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76281522c0 0x7f762814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76280ce130 con 0x7f76280adcd0 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76280cdfb0 con 0x7f76280ac930 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f7630400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76280ce2b0 con 0x7f76281522c0 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:34.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45040/0 (socket says 172.21.15.44:45040) 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 -- 172.21.15.44:0/2101836838 learned_addr learned my addr 172.21.15.44:0/2101836838 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762da00640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 0x7f76281f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762e400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76281522c0 0x7f762814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 -- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76281522c0 msgr2=0x7f762814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76281522c0 0x7f762814c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 -- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 msgr2=0x7f76281f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 0x7f76281f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762d000640 1 -- 172.21.15.44:0/2101836838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762814c800 con 0x7f76280adcd0 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.691+0000 7f762da00640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 0x7f76281f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f762d000640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f761800c7c0 tx=0x7f761800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:34.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618016020 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f761800ab40 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618012420 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762814a160 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.692+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762814a720 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.694+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762815c490 con 0x7f76280adcd0 2024-07-17T08:35:34.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.695+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f7618019020 con 0x7f76280adcd0 2024-07-17T08:35:34.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.696+0000 7f7625600640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:34.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.696+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(226..226 src has 1..226) ==== 7541+0+0 (secure 0 0 0) 0x7f7618098ad0 con 0x7f76280adcd0 2024-07-17T08:35:34.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.697+0000 7f762da00640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:34.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.698+0000 7f762da00640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 0x7f760c079190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:34.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.700+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7618020070 con 0x7f76280adcd0 2024-07-17T08:35:34.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.918+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f762815cb70 con 0x7f76280adcd0 2024-07-17T08:35:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.920+0000 7f7625600640 1 -- 172.21.15.44:0/2101836838 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) ==== 74+0+21552 (secure 0 0 0) 0x7f76180616c0 con 0x7f76280adcd0 2024-07-17T08:35:34.922 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:34.922 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":226,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:33.909039+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 msgr2=0x7f760c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 0x7f760c079190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c004730 comp rx=0 tx=0).stop 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 msgr2=0x7f762814bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f761800c7c0 tx=0x7f761800cc80 comp rx=0 tx=0).stop 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 shutdown_connections 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76281522c0 0x7f762814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76280adcd0 0x7f762814bc90 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 --2- 172.21.15.44:0/2101836838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76280ac930 0x7f76281f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 >> 172.21.15.44:0/2101836838 conn(0x7f76280b0fb0 msgr2=0x7f76280afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 shutdown_connections 2024-07-17T08:35:34.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:34.923+0000 7f7630400640 1 -- 172.21.15.44:0/2101836838 wait complete. 2024-07-17T08:35:34.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.157+0000 7f0502000640 1 -- 172.21.15.44:0/487747161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 msgr2=0x7f04fc06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.157+0000 7f0502000640 1 --2- 172.21.15.44:0/487747161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc06f020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f04e4005fd0 tx=0x7f04e402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 -- 172.21.15.44:0/487747161 shutdown_connections 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 --2- 172.21.15.44:0/487747161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc06f020 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 --2- 172.21.15.44:0/487747161 >> v2:172.21.15.175:3300/0 conn(0x7f04fc15b2e0 0x7f04fc0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 -- 172.21.15.44:0/487747161 >> 172.21.15.44:0/487747161 conn(0x7f04fc0b0fb0 msgr2=0x7f04fc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 -- 172.21.15.44:0/487747161 shutdown_connections 2024-07-17T08:35:35.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.158+0000 7f0502000640 1 -- 172.21.15.44:0/487747161 wait complete. 2024-07-17T08:35:35.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.160+0000 7f0502000640 1 Processor -- start 2024-07-17T08:35:35.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.161+0000 7f0502000640 1 -- start start 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 0x7f04fc0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc1548e0 con 0x7f04fc06f560 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc154760 con 0x7f04fc15b2e0 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f0502000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc154a60 con 0x7f04fc0acb90 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f04fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f04fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45050/0 (socket says 172.21.15.44:45050) 2024-07-17T08:35:35.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.162+0000 7f04fb400640 1 -- 172.21.15.44:0/3989514255 learned_addr learned my addr 172.21.15.44:0/3989514255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04faa00640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 -- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 msgr2=0x7f04fc099ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc099ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fbe00640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 0x7f04fc0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 -- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 msgr2=0x7f04fc0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 0x7f04fc0cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 -- 172.21.15.44:0/3989514255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc09cac0 con 0x7f04fc06f560 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fbe00640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 0x7f04fc0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:35.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04fb400640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00e9f0 tx=0x7f04ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00cd80 con 0x7f04fc06f560 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04ec004020 con 0x7f04fc06f560 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec004470 con 0x7f04fc06f560 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.163+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc09cda0 con 0x7f04fc06f560 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.164+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc072b90 con 0x7f04fc06f560 2024-07-17T08:35:35.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.165+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc06f020 con 0x7f04fc06f560 2024-07-17T08:35:35.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.166+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f04ec0035f0 con 0x7f04fc06f560 2024-07-17T08:35:35.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.168+0000 7f04f3400640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 0x7f04d8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.168+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(226..226 src has 1..226) ==== 7541+0+0 (secure 0 0 0) 0x7f04ec099580 con 0x7f04fc06f560 2024-07-17T08:35:35.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.168+0000 7f04faa00640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 0x7f04d8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.170+0000 7f04faa00640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 0x7f04d8079190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04e4033000 tx=0x7f04e4004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:35.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.172+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f04ec0620f0 con 0x7f04fc06f560 2024-07-17T08:35:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.389+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04fc09cf70 con 0x7f04fc06f560 2024-07-17T08:35:35.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.393+0000 7f04f3400640 1 -- 172.21.15.44:0/3989514255 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) ==== 74+0+21552 (secure 0 0 0) 0x7f04ec0619f0 con 0x7f04fc06f560 2024-07-17T08:35:35.395 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:35.395 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":226,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:33.909039+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 msgr2=0x7f04d8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 0x7f04d8079190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04e4033000 tx=0x7f04e4004730 comp rx=0 tx=0).stop 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 msgr2=0x7f04fc150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00e9f0 tx=0x7f04ec00eeb0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 shutdown_connections 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f04d8076cd0 0x7f04d8079190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04fc15b2e0 0x7f04fc0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04fc0acb90 0x7f04fc099ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 --2- 172.21.15.44:0/3989514255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04fc06f560 0x7f04fc150980 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 >> 172.21.15.44:0/3989514255 conn(0x7f04fc0b0fb0 msgr2=0x7f04fc0b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 shutdown_connections 2024-07-17T08:35:35.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.396+0000 7f0502000640 1 -- 172.21.15.44:0/3989514255 wait complete. 2024-07-17T08:35:35.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-17T08:35:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 -- 172.21.15.44:0/3886838771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 msgr2=0x7f5f5c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 --2- 172.21.15.44:0/3886838771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f5c003e70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009b50 tx=0x7f5f5c005830 comp rx=0 tx=0).stop 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f77e00640 1 -- 172.21.15.44:0/3886838771 reap_dead start 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 -- 172.21.15.44:0/3886838771 shutdown_connections 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 -- 172.21.15.44:0/3886838771 >> 172.21.15.44:0/3886838771 conn(0x7f5f780b0eb0 msgr2=0x7f5f780b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 -- 172.21.15.44:0/3886838771 shutdown_connections 2024-07-17T08:35:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.611+0000 7f5f7e600640 1 -- 172.21.15.44:0/3886838771 wait complete. 2024-07-17T08:35:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.613+0000 7f5f7e600640 1 Processor -- start 2024-07-17T08:35:35.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.613+0000 7f5f7e600640 1 -- start start 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f78154920 0x7f5f781590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f780c8d30 0x7f5f780c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7815ae20 con 0x7f5f5c003a50 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7806d7e0 con 0x7f5f780c8d30 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f7e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7815afa0 con 0x7f5f78154920 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45066/0 (socket says 172.21.15.44:45066) 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.614+0000 7f5f77400640 1 -- 172.21.15.44:0/1700628418 learned_addr learned my addr 172.21.15.44:0/1700628418 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:35.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 -- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f78154920 msgr2=0x7f5f781590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f78154920 0x7f5f781590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 -- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f780c8d30 msgr2=0x7f5f780c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f780c8d30 0x7f5f780c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 -- 172.21.15.44:0/1700628418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f781595e0 con 0x7f5f5c003a50 2024-07-17T08:35:35.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f77400640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009c40 tx=0x7f5f68033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68038040 con 0x7f5f5c003a50 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f68004020 con 0x7f5f5c003a50 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004450 con 0x7f5f5c003a50 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.615+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f78154fb0 con 0x7f5f5c003a50 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.616+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f78155570 con 0x7f5f5c003a50 2024-07-17T08:35:35.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.617+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7815b530 con 0x7f5f5c003a50 2024-07-17T08:35:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.618+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f5f68045020 con 0x7f5f5c003a50 2024-07-17T08:35:35.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.619+0000 7f5f6f400640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 0x7f5f540791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:35.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.619+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f5f680b93c0 con 0x7f5f5c003a50 2024-07-17T08:35:35.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.620+0000 7f5f6fe00640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 0x7f5f540791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:35.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.621+0000 7f5f6fe00640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 0x7f5f540791c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f780700f0 tx=0x7f5f600014d0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:35.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.623+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f781595e0 con 0x7f5f5c003a50 2024-07-17T08:35:35.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:35.875+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5f78156340 con 0x7f5f5c003a50 2024-07-17T08:35:36.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.473+0000 7f5f6f400640 1 -- 172.21.15.44:0/1700628418 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v118) ==== 121+0+0 (secure 0 0 0) 0x7f5f68081fb0 con 0x7f5f5c003a50 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 msgr2=0x7f5f540791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 0x7f5f540791c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f780700f0 tx=0x7f5f600014d0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 msgr2=0x7f5f7814dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009c40 tx=0x7f5f68033670 comp rx=0 tx=0).stop 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 shutdown_connections 2024-07-17T08:35:36.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5f54076d00 0x7f5f540791c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f780c8d30 0x7f5f780c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f78154920 0x7f5f781590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 --2- 172.21.15.44:0/1700628418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f5c003a50 0x7f5f7814dc70 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.477+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 >> 172.21.15.44:0/1700628418 conn(0x7f5f780b0eb0 msgr2=0x7f5f780ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.478+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 shutdown_connections 2024-07-17T08:35:36.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.478+0000 7f5f7e600640 1 -- 172.21.15.44:0/1700628418 wait complete. 2024-07-17T08:35:36.501 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:35:36.501 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-17T08:35:36.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.706+0000 7fad8f000640 1 -- 172.21.15.44:0/815541495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 msgr2=0x7fad6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.706+0000 7fad8f000640 1 --2- 172.21.15.44:0/815541495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad6c004c80 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fad700016c0 tx=0x7fad6c005830 comp rx=0 tx=0).stop 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.706+0000 7fad8f000640 1 -- 172.21.15.44:0/815541495 shutdown_connections 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.706+0000 7fad8f000640 1 --2- 172.21.15.44:0/815541495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad6c004c80 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.706+0000 7fad8f000640 1 -- 172.21.15.44:0/815541495 >> 172.21.15.44:0/815541495 conn(0x7fad8806d050 msgr2=0x7fad8806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.707+0000 7fad8f000640 1 -- 172.21.15.44:0/815541495 shutdown_connections 2024-07-17T08:35:36.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.707+0000 7fad8f000640 1 -- 172.21.15.44:0/815541495 wait complete. 2024-07-17T08:35:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.708+0000 7fad8f000640 1 Processor -- start 2024-07-17T08:35:36.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.708+0000 7fad8f000640 1 -- start start 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad88153e80 0x7fad88143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad881f7ab0 0x7fad881e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad8806e730 con 0x7fad6c004800 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad880b3b70 con 0x7fad88153e80 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.709+0000 7fad8f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad8806e8b0 con 0x7fad881f7ab0 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:36.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45102/0 (socket says 172.21.15.44:45102) 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 -- 172.21.15.44:0/1776826036 learned_addr learned my addr 172.21.15.44:0/1776826036 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87400640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad88153e80 0x7fad88143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad8d000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad881f7ab0 0x7fad881e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 -- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad881f7ab0 msgr2=0x7fad881e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad881f7ab0 0x7fad881e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 -- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad88153e80 msgr2=0x7fad88143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad88153e80 0x7fad88143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 -- 172.21.15.44:0/1776826036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88152de0 con 0x7fad6c004800 2024-07-17T08:35:36.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.710+0000 7fad87e00640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fad78011780 tx=0x7fad78011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.711+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7801f070 con 0x7fad6c004800 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.711+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad78011e90 con 0x7fad6c004800 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.711+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780126b0 con 0x7fad6c004800 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.711+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad881530c0 con 0x7fad6c004800 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.711+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad88153680 con 0x7fad6c004800 2024-07-17T08:35:36.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.712+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad880727d0 con 0x7fad6c004800 2024-07-17T08:35:36.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.714+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fad78022070 con 0x7fad6c004800 2024-07-17T08:35:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.715+0000 7fad7fe00640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 0x7fad64079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:36.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.715+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fad78099dd0 con 0x7fad6c004800 2024-07-17T08:35:36.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.716+0000 7fad87400640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 0x7fad64079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:36.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.718+0000 7fad87400640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 0x7fad64079190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad70032000 tx=0x7fad70001190 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:36.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.719+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fad78062480 con 0x7fad6c004800 2024-07-17T08:35:36.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:36.969+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fad88073850 con 0x7fad6c004800 2024-07-17T08:35:37.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.476+0000 7fad7fe00640 1 -- 172.21.15.44:0/1776826036 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v119) ==== 125+0+0 (secure 0 0 0) 0x7fad780669d0 con 0x7fad6c004800 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.480+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 msgr2=0x7fad64079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.480+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 0x7fad64079190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad70032000 tx=0x7fad70001190 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.480+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 msgr2=0x7fad88152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.480+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fad78011780 tx=0x7fad78011c40 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 shutdown_connections 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad881f7ab0 0x7fad881e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fad64076cd0 0x7fad64079190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad88153e80 0x7fad88143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 --2- 172.21.15.44:0/1776826036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad6c004800 0x7fad88152700 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 >> 172.21.15.44:0/1776826036 conn(0x7fad8806d050 msgr2=0x7fad8815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:37.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 shutdown_connections 2024-07-17T08:35:37.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.481+0000 7fad8f000640 1 -- 172.21.15.44:0/1776826036 wait complete. 2024-07-17T08:35:37.505 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-17T08:35:37.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:37.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 -- 172.21.15.44:0/3755983789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be01522c0 msgr2=0x7f2be00ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3755983789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be01522c0 0x7f2be00ad790 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2be015c950 tx=0x7f2bd002aac0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 -- 172.21.15.44:0/3755983789 shutdown_connections 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3755983789 >> v2:172.21.15.175:3300/0 conn(0x7f2be00adcd0 0x7f2be00ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3755983789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be01522c0 0x7f2be00ad790 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.709+0000 7f2be5a00640 1 -- 172.21.15.44:0/3755983789 >> 172.21.15.44:0/3755983789 conn(0x7f2be00b0fb0 msgr2=0x7f2be00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.710+0000 7f2be5a00640 1 -- 172.21.15.44:0/3755983789 shutdown_connections 2024-07-17T08:35:37.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.710+0000 7f2be5a00640 1 -- 172.21.15.44:0/3755983789 wait complete. 2024-07-17T08:35:37.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.711+0000 7f2be5a00640 1 Processor -- start 2024-07-17T08:35:37.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.711+0000 7f2be5a00640 1 -- start start 2024-07-17T08:35:37.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:37.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be00adcd0 0x7f2be0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:37.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 0x7f2be01f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:37.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2be015bc60 con 0x7f2be00ac760 2024-07-17T08:35:37.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00ce8f0 con 0x7f2be01522c0 2024-07-17T08:35:37.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.712+0000 7f2be5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2be015f5a0 con 0x7f2be00adcd0 2024-07-17T08:35:37.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:37.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45126/0 (socket says 172.21.15.44:45126) 2024-07-17T08:35:37.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 -- 172.21.15.44:0/3725901648 learned_addr learned my addr 172.21.15.44:0/3725901648 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdf400640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 0x7f2be01f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 -- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be00adcd0 msgr2=0x7f2be0153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be00adcd0 0x7f2be0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 -- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 msgr2=0x7f2be01f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 0x7f2be01f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 -- 172.21.15.44:0/3725901648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be0145d50 con 0x7f2be00ac760 2024-07-17T08:35:37.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdf400640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 0x7f2be01f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.713+0000 7f2bdea00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009d60 tx=0x7f2bd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.714+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004250 con 0x7f2be00ac760 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.714+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd00043f0 con 0x7f2be00ac760 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.714+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004800 con 0x7f2be00ac760 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.714+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be0145ff0 con 0x7f2be00ac760 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.714+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01465b0 con 0x7f2be00ac760 2024-07-17T08:35:37.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.715+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be015bfe0 con 0x7f2be00ac760 2024-07-17T08:35:37.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.716+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f2bd002c940 con 0x7f2be00ac760 2024-07-17T08:35:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.717+0000 7f2bd6a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 0x7f2bbc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.717+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f2bd00b9b20 con 0x7f2be00ac760 2024-07-17T08:35:37.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.718+0000 7f2bde000640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 0x7f2bbc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:37.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.719+0000 7f2bde000640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 0x7f2bbc0791c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2be0155400 tx=0x7f2bc8005bf0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:37.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.721+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2be015bfe0 con 0x7f2be00ac760 2024-07-17T08:35:37.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.963+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2be015fa50 con 0x7f2be00ac760 2024-07-17T08:35:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.964+0000 7f2bd6a00640 1 -- 172.21.15.44:0/3725901648 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f2bd0032030 con 0x7f2be00ac760 2024-07-17T08:35:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:37.966 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:35:37.966 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:35:37.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.967+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 msgr2=0x7f2bbc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.967+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 0x7f2bbc0791c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2be0155400 tx=0x7f2bc8005bf0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.967+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 msgr2=0x7f2be0145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:37.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.967+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009d60 tx=0x7f2bd0003f10 comp rx=0 tx=0).stop 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 shutdown_connections 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f2bbc076d00 0x7f2bbc0791c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be01522c0 0x7f2be01f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be00adcd0 0x7f2be0153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 --2- 172.21.15.44:0/3725901648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be00ac760 0x7f2be0145810 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 >> 172.21.15.44:0/3725901648 conn(0x7f2be00b0fb0 msgr2=0x7f2be015a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 shutdown_connections 2024-07-17T08:35:37.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:37.968+0000 7f2be5a00640 1 -- 172.21.15.44:0/3725901648 wait complete. 2024-07-17T08:35:37.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.179+0000 7f5e18200640 1 -- 172.21.15.44:0/1863368376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e101522c0 msgr2=0x7f5e100ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.179+0000 7f5e18200640 1 --2- 172.21.15.44:0/1863368376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e101522c0 0x7f5e100ad790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009860 tx=0x7f5e0402a540 comp rx=0 tx=0).stop 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 -- 172.21.15.44:0/1863368376 shutdown_connections 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 --2- 172.21.15.44:0/1863368376 >> v2:172.21.15.175:3300/0 conn(0x7f5e100ac930 0x7f5e100cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 --2- 172.21.15.44:0/1863368376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e101522c0 0x7f5e100ad790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 -- 172.21.15.44:0/1863368376 >> 172.21.15.44:0/1863368376 conn(0x7f5e100b0fb0 msgr2=0x7f5e100b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 -- 172.21.15.44:0/1863368376 shutdown_connections 2024-07-17T08:35:38.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.180+0000 7f5e18200640 1 -- 172.21.15.44:0/1863368376 wait complete. 2024-07-17T08:35:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.181+0000 7f5e18200640 1 Processor -- start 2024-07-17T08:35:38.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.182+0000 7f5e18200640 1 -- start start 2024-07-17T08:35:38.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.182+0000 7f5e18200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e100ac930 0x7f5e10073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.182+0000 7f5e18200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e100adcd0 0x7f5e1014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.182+0000 7f5e18200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e18200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cdf90 con 0x7f5e101522c0 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e18200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cde10 con 0x7f5e100ac930 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e18200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5e100ce110 con 0x7f5e100adcd0 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45132/0 (socket says 172.21.15.44:45132) 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 -- 172.21.15.44:0/1822569063 learned_addr learned my addr 172.21.15.44:0/1822569063 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e14e00640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e100adcd0 0x7f5e1014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e15800640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e100ac930 0x7f5e10073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 -- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e100adcd0 msgr2=0x7f5e1014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e100adcd0 0x7f5e1014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 -- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e100ac930 msgr2=0x7f5e10073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e100ac930 0x7f5e10073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.183+0000 7f5e16200640 1 -- 172.21.15.44:0/1822569063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e10073590 con 0x7f5e101522c0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e16200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800ee80 tx=0x7f5e0800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080099a0 con 0x7f5e101522c0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e08004c30 con 0x7f5e101522c0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1014fbb0 con 0x7f5e101522c0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003bd0 con 0x7f5e101522c0 2024-07-17T08:35:38.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.184+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e10073b20 con 0x7f5e101522c0 2024-07-17T08:35:38.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.185+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e100ce660 con 0x7f5e101522c0 2024-07-17T08:35:38.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.186+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f5e0801d020 con 0x7f5e101522c0 2024-07-17T08:35:38.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.188+0000 7f5e0d600640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 0x7f5df4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.188+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f5e080993a0 con 0x7f5e101522c0 2024-07-17T08:35:38.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.188+0000 7f5e15800640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 0x7f5df4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.190+0000 7f5e15800640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 0x7f5df4079190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e04034000 tx=0x7f5e04004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:38.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.192+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5e08061e80 con 0x7f5e101522c0 2024-07-17T08:35:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.405+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e1015b280 con 0x7f5e101522c0 2024-07-17T08:35:38.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.408+0000 7f5e0d600640 1 -- 172.21.15.44:0/1822569063 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v227) ==== 74+0+21539 (secure 0 0 0) 0x7f5e08061780 con 0x7f5e101522c0 2024-07-17T08:35:38.410 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:38.410 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":227,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:35.442232+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.410+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 msgr2=0x7f5df4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.410+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 0x7f5df4079190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e04034000 tx=0x7f5e04004730 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.410+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 msgr2=0x7f5e100993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.410+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800ee80 tx=0x7f5e0800c740 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 shutdown_connections 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e101522c0 0x7f5e100993a0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e100adcd0 0x7f5e1014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f5df4076cd0 0x7f5df4079190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 --2- 172.21.15.44:0/1822569063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e100ac930 0x7f5e10073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 >> 172.21.15.44:0/1822569063 conn(0x7f5e100b0fb0 msgr2=0x7f5e100aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:38.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 shutdown_connections 2024-07-17T08:35:38.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.411+0000 7f5e18200640 1 -- 172.21.15.44:0/1822569063 wait complete. 2024-07-17T08:35:38.431 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:35:38.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:38.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 -- 172.21.15.44:0/204792541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc1522c0 msgr2=0x7fcdbc0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 --2- 172.21.15.44:0/204792541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0ad790 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fcdac005fd0 tx=0x7fcdac02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 -- 172.21.15.44:0/204792541 shutdown_connections 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 --2- 172.21.15.44:0/204792541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0ad790 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 -- 172.21.15.44:0/204792541 >> 172.21.15.44:0/204792541 conn(0x7fcdbc0b0fb0 msgr2=0x7fcdbc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.644+0000 7fcdc2000640 1 -- 172.21.15.44:0/204792541 shutdown_connections 2024-07-17T08:35:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.645+0000 7fcdc2000640 1 -- 172.21.15.44:0/204792541 wait complete. 2024-07-17T08:35:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.646+0000 7fcdc2000640 1 Processor -- start 2024-07-17T08:35:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.646+0000 7fcdc2000640 1 -- start start 2024-07-17T08:35:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 0x7fcdbc0993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0ce130 con 0x7fcdbc0ac930 2024-07-17T08:35:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0cdfb0 con 0x7fcdbc1522c0 2024-07-17T08:35:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdc2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc0ce2b0 con 0x7fcdbc0adcd0 2024-07-17T08:35:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.647+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45152/0 (socket says 172.21.15.44:45152) 2024-07-17T08:35:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 -- 172.21.15.44:0/745112501 learned_addr learned my addr 172.21.15.44:0/745112501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbbe00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 -- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 msgr2=0x7fcdbc0993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbaa00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 0x7fcdbc0993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 0x7fcdbc0993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 -- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 msgr2=0x7fcdbc0998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 -- 172.21.15.44:0/745112501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc099f80 con 0x7fcdbc0ac930 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbaa00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 0x7fcdbc0993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbbe00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.648+0000 7fcdbb400640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034040 tx=0x7fcdac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.649+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac03e030 con 0x7fcdbc0ac930 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.649+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdac003ca0 con 0x7fcdbc0ac930 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.649+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac035040 con 0x7fcdbc0ac930 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.649+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc09a260 con 0x7fcdbc0ac930 2024-07-17T08:35:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.649+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc09acb0 con 0x7fcdbc0ac930 2024-07-17T08:35:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.650+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc0ce7f0 con 0x7fcdbc0ac930 2024-07-17T08:35:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.651+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fcdac041030 con 0x7fcdbc0ac930 2024-07-17T08:35:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.652+0000 7fcdb3400640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 0x7fcd9c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.652+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fcdac0b9160 con 0x7fcdbc0ac930 2024-07-17T08:35:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.653+0000 7fcdbaa00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 0x7fcd9c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.654+0000 7fcdbaa00640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 0x7fcd9c079190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcda8007fd0 tx=0x7fcda8009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.657+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcdac081cd0 con 0x7fcdbc0ac930 2024-07-17T08:35:38.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.867+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdbc143cc0 con 0x7fcd9c076cd0 2024-07-17T08:35:38.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.896+0000 7fcdb3400640 1 -- 172.21.15.44:0/745112501 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+486104 (secure 0 0 0) 0x7fcda8001c80 con 0x7fcd9c076cd0 2024-07-17T08:35:38.907 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:38.911 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.908+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 msgr2=0x7fcd9c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.908+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 0x7fcd9c079190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcda8007fd0 tx=0x7fcda8009040 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.908+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 msgr2=0x7fcdbc06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.908+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034040 tx=0x7fcdac0038a0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 shutdown_connections 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fcd9c076cd0 0x7fcd9c079190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc1522c0 0x7fcdbc0998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc0adcd0 0x7fcdbc0993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 --2- 172.21.15.44:0/745112501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ac930 0x7fcdbc06e940 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 >> 172.21.15.44:0/745112501 conn(0x7fcdbc0b0fb0 msgr2=0x7fcdbc0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:38.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 shutdown_connections 2024-07-17T08:35:38.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:38.909+0000 7fcdc2000640 1 -- 172.21.15.44:0/745112501 wait complete. 2024-07-17T08:35:38.936 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-07-17T08:35:37.765627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594024,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":406,"num_write_kb":1644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2552,"ondisk_log_size":2552,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398944,"kb_used_data":23168,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754575776,"statfs":{"total":773094113280,"available":772685594624,"internally_reserved":0,"allocated":23724032,"data_stored":13420344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1246,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.351964"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.444506+0000","last_change":"2024-07-17T08:35:33.444506+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.444506+0000","last_undegraded":"2024-07-17T08:35:33.444506+0000","last_fullsized":"2024-07-17T08:35:33.444506+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3382,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450159+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:33.450159+0000","last_peered":"2024-07-17T08:35:33.450159+0000","last_clean":"2024-07-17T08:35:33.450159+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:33.450159+0000","last_undegraded":"2024-07-17T08:35:33.450159+0000","last_fullsized":"2024-07-17T08:35:33.450159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441990+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:33.441990+0000","last_peered":"2024-07-17T08:35:33.441990+0000","last_clean":"2024-07-17T08:35:33.441990+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:33.441990+0000","last_undegraded":"2024-07-17T08:35:33.441990+0000","last_fullsized":"2024-07-17T08:35:33.441990+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448569+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:33.448569+0000","last_peered":"2024-07-17T08:35:33.448569+0000","last_clean":"2024-07-17T08:35:33.448569+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:33.448569+0000","last_undegraded":"2024-07-17T08:35:33.448569+0000","last_fullsized":"2024-07-17T08:35:33.448569+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:44.992431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.458346+0000","last_change":"2024-07-17T08:35:33.455926+0000","last_active":"2024-07-17T08:35:33.458346+0000","last_peered":"2024-07-17T08:35:33.458346+0000","last_clean":"2024-07-17T08:35:33.458346+0000","last_became_active":"2024-07-17T08:35:33.455626+0000","last_became_peered":"2024-07-17T08:35:33.455626+0000","last_unstale":"2024-07-17T08:35:33.458346+0000","last_undegraded":"2024-07-17T08:35:33.458346+0000","last_fullsized":"2024-07-17T08:35:33.458346+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:59.073309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441144+0000","last_change":"2024-07-17T08:35:33.441144+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441144+0000","last_undegraded":"2024-07-17T08:35:33.441144+0000","last_fullsized":"2024-07-17T08:35:33.441144+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424194+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:33.424194+0000","last_peered":"2024-07-17T08:35:33.424194+0000","last_clean":"2024-07-17T08:35:33.424194+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:33.424194+0000","last_undegraded":"2024-07-17T08:35:33.424194+0000","last_fullsized":"2024-07-17T08:35:33.424194+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451029+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:33.451029+0000","last_peered":"2024-07-17T08:35:33.451029+0000","last_clean":"2024-07-17T08:35:33.451029+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:33.451029+0000","last_undegraded":"2024-07-17T08:35:33.451029+0000","last_fullsized":"2024-07-17T08:35:33.451029+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453562+0000","last_change":"2024-07-17T08:35:33.451371+0000","last_active":"2024-07-17T08:35:33.453562+0000","last_peered":"2024-07-17T08:35:33.453562+0000","last_clean":"2024-07-17T08:35:33.453562+0000","last_became_active":"2024-07-17T08:35:33.451107+0000","last_became_peered":"2024-07-17T08:35:33.451107+0000","last_unstale":"2024-07-17T08:35:33.453562+0000","last_undegraded":"2024-07-17T08:35:33.453562+0000","last_fullsized":"2024-07-17T08:35:33.453562+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:28.506108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435146+0000","last_change":"2024-07-17T08:35:33.435146+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435146+0000","last_undegraded":"2024-07-17T08:35:33.435146+0000","last_fullsized":"2024-07-17T08:35:33.435146+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448125+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:33.448125+0000","last_peered":"2024-07-17T08:35:33.448125+0000","last_clean":"2024-07-17T08:35:33.448125+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:33.448125+0000","last_undegraded":"2024-07-17T08:35:33.448125+0000","last_fullsized":"2024-07-17T08:35:33.448125+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.452598+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:33.452598+0000","last_peered":"2024-07-17T08:35:33.452598+0000","last_clean":"2024-07-17T08:35:33.452598+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:33.452598+0000","last_undegraded":"2024-07-17T08:35:33.452598+0000","last_fullsized":"2024-07-17T08:35:33.452598+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:08.117588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464320+0000","last_change":"2024-07-17T08:35:33.459624+0000","last_active":"2024-07-17T08:35:33.464320+0000","last_peered":"2024-07-17T08:35:33.464320+0000","last_clean":"2024-07-17T08:35:33.464320+0000","last_became_active":"2024-07-17T08:35:33.459359+0000","last_became_peered":"2024-07-17T08:35:33.459359+0000","last_unstale":"2024-07-17T08:35:33.464320+0000","last_undegraded":"2024-07-17T08:35:33.464320+0000","last_fullsized":"2024-07-17T08:35:33.464320+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:37.256027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429954+0000","last_change":"2024-07-17T08:35:33.429954+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429954+0000","last_undegraded":"2024-07-17T08:35:33.429954+0000","last_fullsized":"2024-07-17T08:35:33.429954+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442098+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:33.442098+0000","last_peered":"2024-07-17T08:35:33.442098+0000","last_clean":"2024-07-17T08:35:33.442098+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:33.442098+0000","last_undegraded":"2024-07-17T08:35:33.442098+0000","last_fullsized":"2024-07-17T08:35:33.442098+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451543+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:33.451543+0000","last_peered":"2024-07-17T08:35:33.451543+0000","last_clean":"2024-07-17T08:35:33.451543+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:33.451543+0000","last_undegraded":"2024-07-17T08:35:33.451543+0000","last_fullsized":"2024-07-17T08:35:33.451543+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:20.349285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461817+0000","last_change":"2024-07-17T08:35:33.457849+0000","last_active":"2024-07-17T08:35:33.461817+0000","last_peered":"2024-07-17T08:35:33.461817+0000","last_clean":"2024-07-17T08:35:33.461817+0000","last_became_active":"2024-07-17T08:35:33.453123+0000","last_became_peered":"2024-07-17T08:35:33.453123+0000","last_unstale":"2024-07-17T08:35:33.461817+0000","last_undegraded":"2024-07-17T08:35:33.461817+0000","last_fullsized":"2024-07-17T08:35:33.461817+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:12.277866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.438424+0000","last_change":"2024-07-17T08:35:33.438424+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.438424+0000","last_undegraded":"2024-07-17T08:35:33.438424+0000","last_fullsized":"2024-07-17T08:35:33.438424+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.431997+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:33.431997+0000","last_peered":"2024-07-17T08:35:33.431997+0000","last_clean":"2024-07-17T08:35:33.431997+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:33.431997+0000","last_undegraded":"2024-07-17T08:35:33.431997+0000","last_fullsized":"2024-07-17T08:35:33.431997+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.429975+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:33.429975+0000","last_peered":"2024-07-17T08:35:33.429975+0000","last_clean":"2024-07-17T08:35:33.429975+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:33.429975+0000","last_undegraded":"2024-07-17T08:35:33.429975+0000","last_fullsized":"2024-07-17T08:35:33.429975+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453756+0000","last_change":"2024-07-17T08:35:33.452063+0000","last_active":"2024-07-17T08:35:33.453756+0000","last_peered":"2024-07-17T08:35:33.453756+0000","last_clean":"2024-07-17T08:35:33.453756+0000","last_became_active":"2024-07-17T08:35:33.450965+0000","last_became_peered":"2024-07-17T08:35:33.450965+0000","last_unstale":"2024-07-17T08:35:33.453756+0000","last_undegraded":"2024-07-17T08:35:33.453756+0000","last_fullsized":"2024-07-17T08:35:33.453756+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:16.564132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431703+0000","last_change":"2024-07-17T08:35:33.431703+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431703+0000","last_undegraded":"2024-07-17T08:35:33.431703+0000","last_fullsized":"2024-07-17T08:35:33.431703+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448163+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:33.448163+0000","last_peered":"2024-07-17T08:35:33.448163+0000","last_clean":"2024-07-17T08:35:33.448163+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:33.448163+0000","last_undegraded":"2024-07-17T08:35:33.448163+0000","last_fullsized":"2024-07-17T08:35:33.448163+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446939+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:33.446939+0000","last_peered":"2024-07-17T08:35:33.446939+0000","last_clean":"2024-07-17T08:35:33.446939+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:33.446939+0000","last_undegraded":"2024-07-17T08:35:33.446939+0000","last_fullsized":"2024-07-17T08:35:33.446939+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:32.339990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446768+0000","last_change":"2024-07-17T08:35:33.443001+0000","last_active":"2024-07-17T08:35:33.446768+0000","last_peered":"2024-07-17T08:35:33.446768+0000","last_clean":"2024-07-17T08:35:33.446768+0000","last_became_active":"2024-07-17T08:35:33.442630+0000","last_became_peered":"2024-07-17T08:35:33.442630+0000","last_unstale":"2024-07-17T08:35:33.446768+0000","last_undegraded":"2024-07-17T08:35:33.446768+0000","last_fullsized":"2024-07-17T08:35:33.446768+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:57.867743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433338+0000","last_change":"2024-07-17T08:35:33.433338+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433338+0000","last_undegraded":"2024-07-17T08:35:33.433338+0000","last_fullsized":"2024-07-17T08:35:33.433338+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436851+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:33.436851+0000","last_peered":"2024-07-17T08:35:33.436851+0000","last_clean":"2024-07-17T08:35:33.436851+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:33.436851+0000","last_undegraded":"2024-07-17T08:35:33.436851+0000","last_fullsized":"2024-07-17T08:35:33.436851+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441736+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:33.441736+0000","last_peered":"2024-07-17T08:35:33.441736+0000","last_clean":"2024-07-17T08:35:33.441736+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:33.441736+0000","last_undegraded":"2024-07-17T08:35:33.441736+0000","last_fullsized":"2024-07-17T08:35:33.441736+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454181+0000","last_change":"2024-07-17T08:35:33.447597+0000","last_active":"2024-07-17T08:35:33.454181+0000","last_peered":"2024-07-17T08:35:33.454181+0000","last_clean":"2024-07-17T08:35:33.454181+0000","last_became_active":"2024-07-17T08:35:33.446794+0000","last_became_peered":"2024-07-17T08:35:33.446794+0000","last_unstale":"2024-07-17T08:35:33.454181+0000","last_undegraded":"2024-07-17T08:35:33.454181+0000","last_fullsized":"2024-07-17T08:35:33.454181+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:24.117005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430173+0000","last_change":"2024-07-17T08:35:33.430173+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430173+0000","last_undegraded":"2024-07-17T08:35:33.430173+0000","last_fullsized":"2024-07-17T08:35:33.430173+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436157+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:33.436157+0000","last_peered":"2024-07-17T08:35:33.436157+0000","last_clean":"2024-07-17T08:35:33.436157+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:33.436157+0000","last_undegraded":"2024-07-17T08:35:33.436157+0000","last_fullsized":"2024-07-17T08:35:33.436157+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448059+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:33.448059+0000","last_peered":"2024-07-17T08:35:33.448059+0000","last_clean":"2024-07-17T08:35:33.448059+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:33.448059+0000","last_undegraded":"2024-07-17T08:35:33.448059+0000","last_fullsized":"2024-07-17T08:35:33.448059+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464485+0000","last_change":"2024-07-17T08:35:33.461285+0000","last_active":"2024-07-17T08:35:33.464485+0000","last_peered":"2024-07-17T08:35:33.464485+0000","last_clean":"2024-07-17T08:35:33.464485+0000","last_became_active":"2024-07-17T08:35:33.460353+0000","last_became_peered":"2024-07-17T08:35:33.460353+0000","last_unstale":"2024-07-17T08:35:33.464485+0000","last_undegraded":"2024-07-17T08:35:33.464485+0000","last_fullsized":"2024-07-17T08:35:33.464485+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:53.232196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430826+0000","last_change":"2024-07-17T08:35:33.430826+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430826+0000","last_undegraded":"2024-07-17T08:35:33.430826+0000","last_fullsized":"2024-07-17T08:35:33.430826+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442600+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:33.442600+0000","last_peered":"2024-07-17T08:35:33.442600+0000","last_clean":"2024-07-17T08:35:33.442600+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:33.442600+0000","last_undegraded":"2024-07-17T08:35:33.442600+0000","last_fullsized":"2024-07-17T08:35:33.442600+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.433201+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:33.433201+0000","last_peered":"2024-07-17T08:35:33.433201+0000","last_clean":"2024-07-17T08:35:33.433201+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:33.433201+0000","last_undegraded":"2024-07-17T08:35:33.433201+0000","last_fullsized":"2024-07-17T08:35:33.433201+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460912+0000","last_change":"2024-07-17T08:35:33.458859+0000","last_active":"2024-07-17T08:35:33.460912+0000","last_peered":"2024-07-17T08:35:33.460912+0000","last_clean":"2024-07-17T08:35:33.460912+0000","last_became_active":"2024-07-17T08:35:33.458540+0000","last_became_peered":"2024-07-17T08:35:33.458540+0000","last_unstale":"2024-07-17T08:35:33.460912+0000","last_undegraded":"2024-07-17T08:35:33.460912+0000","last_fullsized":"2024-07-17T08:35:33.460912+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:59.715580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.444703+0000","last_change":"2024-07-17T08:35:33.444703+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.444703+0000","last_undegraded":"2024-07-17T08:35:33.444703+0000","last_fullsized":"2024-07-17T08:35:33.444703+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.438968+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:33.438968+0000","last_peered":"2024-07-17T08:35:33.438968+0000","last_clean":"2024-07-17T08:35:33.438968+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:33.438968+0000","last_undegraded":"2024-07-17T08:35:33.438968+0000","last_fullsized":"2024-07-17T08:35:33.438968+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.438487+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:33.438487+0000","last_peered":"2024-07-17T08:35:33.438487+0000","last_clean":"2024-07-17T08:35:33.438487+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:33.438487+0000","last_undegraded":"2024-07-17T08:35:33.438487+0000","last_fullsized":"2024-07-17T08:35:33.438487+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451248+0000","last_change":"2024-07-17T08:35:33.449985+0000","last_active":"2024-07-17T08:35:33.451248+0000","last_peered":"2024-07-17T08:35:33.451248+0000","last_clean":"2024-07-17T08:35:33.451248+0000","last_became_active":"2024-07-17T08:35:33.449677+0000","last_became_peered":"2024-07-17T08:35:33.449677+0000","last_unstale":"2024-07-17T08:35:33.451248+0000","last_undegraded":"2024-07-17T08:35:33.451248+0000","last_fullsized":"2024-07-17T08:35:33.451248+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:45.362621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433025+0000","last_change":"2024-07-17T08:35:33.433025+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433025+0000","last_undegraded":"2024-07-17T08:35:33.433025+0000","last_fullsized":"2024-07-17T08:35:33.433025+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436626+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:33.436626+0000","last_peered":"2024-07-17T08:35:33.436626+0000","last_clean":"2024-07-17T08:35:33.436626+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:33.436626+0000","last_undegraded":"2024-07-17T08:35:33.436626+0000","last_fullsized":"2024-07-17T08:35:33.436626+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440415+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:33.440415+0000","last_peered":"2024-07-17T08:35:33.440415+0000","last_clean":"2024-07-17T08:35:33.440415+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:33.440415+0000","last_undegraded":"2024-07-17T08:35:33.440415+0000","last_fullsized":"2024-07-17T08:35:33.440415+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.455745+0000","last_change":"2024-07-17T08:35:33.455119+0000","last_active":"2024-07-17T08:35:33.455745+0000","last_peered":"2024-07-17T08:35:33.455745+0000","last_clean":"2024-07-17T08:35:33.455745+0000","last_became_active":"2024-07-17T08:35:33.454773+0000","last_became_peered":"2024-07-17T08:35:33.454773+0000","last_unstale":"2024-07-17T08:35:33.455745+0000","last_undegraded":"2024-07-17T08:35:33.455745+0000","last_fullsized":"2024-07-17T08:35:33.455745+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:02:33.574937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430713+0000","last_change":"2024-07-17T08:35:33.430713+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430713+0000","last_undegraded":"2024-07-17T08:35:33.430713+0000","last_fullsized":"2024-07-17T08:35:33.430713+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443666+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:33.443666+0000","last_peered":"2024-07-17T08:35:33.443666+0000","last_clean":"2024-07-17T08:35:33.443666+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:33.443666+0000","last_undegraded":"2024-07-17T08:35:33.443666+0000","last_fullsized":"2024-07-17T08:35:33.443666+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451420+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:33.451420+0000","last_peered":"2024-07-17T08:35:33.451420+0000","last_clean":"2024-07-17T08:35:33.451420+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:33.451420+0000","last_undegraded":"2024-07-17T08:35:33.451420+0000","last_fullsized":"2024-07-17T08:35:33.451420+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:36.700604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461838+0000","last_change":"2024-07-17T08:35:33.457888+0000","last_active":"2024-07-17T08:35:33.461838+0000","last_peered":"2024-07-17T08:35:33.461838+0000","last_clean":"2024-07-17T08:35:33.461838+0000","last_became_active":"2024-07-17T08:35:33.457510+0000","last_became_peered":"2024-07-17T08:35:33.457510+0000","last_unstale":"2024-07-17T08:35:33.461838+0000","last_undegraded":"2024-07-17T08:35:33.461838+0000","last_fullsized":"2024-07-17T08:35:33.461838+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:01.792237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439003+0000","last_change":"2024-07-17T08:35:33.439003+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439003+0000","last_undegraded":"2024-07-17T08:35:33.439003+0000","last_fullsized":"2024-07-17T08:35:33.439003+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424352+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:33.424352+0000","last_peered":"2024-07-17T08:35:33.424352+0000","last_clean":"2024-07-17T08:35:33.424352+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:33.424352+0000","last_undegraded":"2024-07-17T08:35:33.424352+0000","last_fullsized":"2024-07-17T08:35:33.424352+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443960+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:33.443960+0000","last_peered":"2024-07-17T08:35:33.443960+0000","last_clean":"2024-07-17T08:35:33.443960+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:33.443960+0000","last_undegraded":"2024-07-17T08:35:33.443960+0000","last_fullsized":"2024-07-17T08:35:33.443960+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:53.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460619+0000","last_change":"2024-07-17T08:35:33.456857+0000","last_active":"2024-07-17T08:35:33.460619+0000","last_peered":"2024-07-17T08:35:33.460619+0000","last_clean":"2024-07-17T08:35:33.460619+0000","last_became_active":"2024-07-17T08:35:33.456290+0000","last_became_peered":"2024-07-17T08:35:33.456290+0000","last_unstale":"2024-07-17T08:35:33.460619+0000","last_undegraded":"2024-07-17T08:35:33.460619+0000","last_fullsized":"2024-07-17T08:35:33.460619+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:47.684683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439099+0000","last_change":"2024-07-17T08:35:33.439099+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439099+0000","last_undegraded":"2024-07-17T08:35:33.439099+0000","last_fullsized":"2024-07-17T08:35:33.439099+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.425122+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:33.425122+0000","last_peered":"2024-07-17T08:35:33.425122+0000","last_clean":"2024-07-17T08:35:33.425122+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:33.425122+0000","last_undegraded":"2024-07-17T08:35:33.425122+0000","last_fullsized":"2024-07-17T08:35:33.425122+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.428509+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:33.428509+0000","last_peered":"2024-07-17T08:35:33.428509+0000","last_clean":"2024-07-17T08:35:33.428509+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:33.428509+0000","last_undegraded":"2024-07-17T08:35:33.428509+0000","last_fullsized":"2024-07-17T08:35:33.428509+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454922+0000","last_change":"2024-07-17T08:35:33.454020+0000","last_active":"2024-07-17T08:35:33.454922+0000","last_peered":"2024-07-17T08:35:33.454922+0000","last_clean":"2024-07-17T08:35:33.454922+0000","last_became_active":"2024-07-17T08:35:33.453743+0000","last_became_peered":"2024-07-17T08:35:33.453743+0000","last_unstale":"2024-07-17T08:35:33.454922+0000","last_undegraded":"2024-07-17T08:35:33.454922+0000","last_fullsized":"2024-07-17T08:35:33.454922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:52.231608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434167+0000","last_change":"2024-07-17T08:35:33.434167+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434167+0000","last_undegraded":"2024-07-17T08:35:33.434167+0000","last_fullsized":"2024-07-17T08:35:33.434167+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.444327+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:33.444327+0000","last_peered":"2024-07-17T08:35:33.444327+0000","last_clean":"2024-07-17T08:35:33.444327+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:33.444327+0000","last_undegraded":"2024-07-17T08:35:33.444327+0000","last_fullsized":"2024-07-17T08:35:33.444327+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447324+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:33.447324+0000","last_peered":"2024-07-17T08:35:33.447324+0000","last_clean":"2024-07-17T08:35:33.447324+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:33.447324+0000","last_undegraded":"2024-07-17T08:35:33.447324+0000","last_fullsized":"2024-07-17T08:35:33.447324+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:07.288084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454222+0000","last_change":"2024-07-17T08:35:33.451492+0000","last_active":"2024-07-17T08:35:33.454222+0000","last_peered":"2024-07-17T08:35:33.454222+0000","last_clean":"2024-07-17T08:35:33.454222+0000","last_became_active":"2024-07-17T08:35:33.451016+0000","last_became_peered":"2024-07-17T08:35:33.451016+0000","last_unstale":"2024-07-17T08:35:33.454222+0000","last_undegraded":"2024-07-17T08:35:33.454222+0000","last_fullsized":"2024-07-17T08:35:33.454222+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:53:03.903417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432858+0000","last_change":"2024-07-17T08:35:33.432858+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432858+0000","last_undegraded":"2024-07-17T08:35:33.432858+0000","last_fullsized":"2024-07-17T08:35:33.432858+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448902+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:33.448902+0000","last_peered":"2024-07-17T08:35:33.448902+0000","last_clean":"2024-07-17T08:35:33.448902+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:33.448902+0000","last_undegraded":"2024-07-17T08:35:33.448902+0000","last_fullsized":"2024-07-17T08:35:33.448902+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448027+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:33.448027+0000","last_peered":"2024-07-17T08:35:33.448027+0000","last_clean":"2024-07-17T08:35:33.448027+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:33.448027+0000","last_undegraded":"2024-07-17T08:35:33.448027+0000","last_fullsized":"2024-07-17T08:35:33.448027+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:44.865077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443689+0000","last_change":"2024-07-17T08:35:33.440197+0000","last_active":"2024-07-17T08:35:33.443689+0000","last_peered":"2024-07-17T08:35:33.443689+0000","last_clean":"2024-07-17T08:35:33.443689+0000","last_became_active":"2024-07-17T08:35:33.439893+0000","last_became_peered":"2024-07-17T08:35:33.439893+0000","last_unstale":"2024-07-17T08:35:33.443689+0000","last_undegraded":"2024-07-17T08:35:33.443689+0000","last_fullsized":"2024-07-17T08:35:33.443689+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:03.810647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432715+0000","last_change":"2024-07-17T08:35:33.432715+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432715+0000","last_undegraded":"2024-07-17T08:35:33.432715+0000","last_fullsized":"2024-07-17T08:35:33.432715+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449207+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:33.449207+0000","last_peered":"2024-07-17T08:35:33.449207+0000","last_clean":"2024-07-17T08:35:33.449207+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:33.449207+0000","last_undegraded":"2024-07-17T08:35:33.449207+0000","last_fullsized":"2024-07-17T08:35:33.449207+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.432996+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:33.432996+0000","last_peered":"2024-07-17T08:35:33.432996+0000","last_clean":"2024-07-17T08:35:33.432996+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:33.432996+0000","last_undegraded":"2024-07-17T08:35:33.432996+0000","last_fullsized":"2024-07-17T08:35:33.432996+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453504+0000","last_change":"2024-07-17T08:35:33.451532+0000","last_active":"2024-07-17T08:35:33.453504+0000","last_peered":"2024-07-17T08:35:33.453504+0000","last_clean":"2024-07-17T08:35:33.453504+0000","last_became_active":"2024-07-17T08:35:33.449530+0000","last_became_peered":"2024-07-17T08:35:33.449530+0000","last_unstale":"2024-07-17T08:35:33.453504+0000","last_undegraded":"2024-07-17T08:35:33.453504+0000","last_fullsized":"2024-07-17T08:35:33.453504+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:28:19.068669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442530+0000","last_change":"2024-07-17T08:35:33.442530+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442530+0000","last_undegraded":"2024-07-17T08:35:33.442530+0000","last_fullsized":"2024-07-17T08:35:33.442530+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":26,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424081+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:33.424081+0000","last_peered":"2024-07-17T08:35:33.424081+0000","last_clean":"2024-07-17T08:35:33.424081+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:33.424081+0000","last_undegraded":"2024-07-17T08:35:33.424081+0000","last_fullsized":"2024-07-17T08:35:33.424081+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454471+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:33.454471+0000","last_peered":"2024-07-17T08:35:33.454471+0000","last_clean":"2024-07-17T08:35:33.454471+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:33.454471+0000","last_undegraded":"2024-07-17T08:35:33.454471+0000","last_fullsized":"2024-07-17T08:35:33.454471+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:38.094519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.457534+0000","last_change":"2024-07-17T08:35:33.456460+0000","last_active":"2024-07-17T08:35:33.457534+0000","last_peered":"2024-07-17T08:35:33.457534+0000","last_clean":"2024-07-17T08:35:33.457534+0000","last_became_active":"2024-07-17T08:35:33.456193+0000","last_became_peered":"2024-07-17T08:35:33.456193+0000","last_unstale":"2024-07-17T08:35:33.457534+0000","last_undegraded":"2024-07-17T08:35:33.457534+0000","last_fullsized":"2024-07-17T08:35:33.457534+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:11.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.445347+0000","last_change":"2024-07-17T08:35:33.445347+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.445347+0000","last_undegraded":"2024-07-17T08:35:33.445347+0000","last_fullsized":"2024-07-17T08:35:33.445347+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.439847+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:33.439847+0000","last_peered":"2024-07-17T08:35:33.439847+0000","last_clean":"2024-07-17T08:35:33.439847+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:33.439847+0000","last_undegraded":"2024-07-17T08:35:33.439847+0000","last_fullsized":"2024-07-17T08:35:33.439847+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440307+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:33.440307+0000","last_peered":"2024-07-17T08:35:33.440307+0000","last_clean":"2024-07-17T08:35:33.440307+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:33.440307+0000","last_undegraded":"2024-07-17T08:35:33.440307+0000","last_fullsized":"2024-07-17T08:35:33.440307+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.462214+0000","last_change":"2024-07-17T08:35:33.460434+0000","last_active":"2024-07-17T08:35:33.462214+0000","last_peered":"2024-07-17T08:35:33.462214+0000","last_clean":"2024-07-17T08:35:33.462214+0000","last_became_active":"2024-07-17T08:35:33.460174+0000","last_became_peered":"2024-07-17T08:35:33.460174+0000","last_unstale":"2024-07-17T08:35:33.462214+0000","last_undegraded":"2024-07-17T08:35:33.462214+0000","last_fullsized":"2024-07-17T08:35:33.462214+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:17.721570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433830+0000","last_change":"2024-07-17T08:35:33.433830+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433830+0000","last_undegraded":"2024-07-17T08:35:33.433830+0000","last_fullsized":"2024-07-17T08:35:33.433830+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448041+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:33.448041+0000","last_peered":"2024-07-17T08:35:33.448041+0000","last_clean":"2024-07-17T08:35:33.448041+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:33.448041+0000","last_undegraded":"2024-07-17T08:35:33.448041+0000","last_fullsized":"2024-07-17T08:35:33.448041+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449170+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:33.449170+0000","last_peered":"2024-07-17T08:35:33.449170+0000","last_clean":"2024-07-17T08:35:33.449170+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:33.449170+0000","last_undegraded":"2024-07-17T08:35:33.449170+0000","last_fullsized":"2024-07-17T08:35:33.449170+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461042+0000","last_change":"2024-07-17T08:35:33.458710+0000","last_active":"2024-07-17T08:35:33.461042+0000","last_peered":"2024-07-17T08:35:33.461042+0000","last_clean":"2024-07-17T08:35:33.461042+0000","last_became_active":"2024-07-17T08:35:33.458420+0000","last_became_peered":"2024-07-17T08:35:33.458420+0000","last_unstale":"2024-07-17T08:35:33.461042+0000","last_undegraded":"2024-07-17T08:35:33.461042+0000","last_fullsized":"2024-07-17T08:35:33.461042+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.438593+0000","last_change":"2024-07-17T08:35:33.438593+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.438593+0000","last_undegraded":"2024-07-17T08:35:33.438593+0000","last_fullsized":"2024-07-17T08:35:33.438593+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443790+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:33.443790+0000","last_peered":"2024-07-17T08:35:33.443790+0000","last_clean":"2024-07-17T08:35:33.443790+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:33.443790+0000","last_undegraded":"2024-07-17T08:35:33.443790+0000","last_fullsized":"2024-07-17T08:35:33.443790+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443850+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:33.443850+0000","last_peered":"2024-07-17T08:35:33.443850+0000","last_clean":"2024-07-17T08:35:33.443850+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:33.443850+0000","last_undegraded":"2024-07-17T08:35:33.443850+0000","last_fullsized":"2024-07-17T08:35:33.443850+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:07:04.780778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.462456+0000","last_change":"2024-07-17T08:35:33.459431+0000","last_active":"2024-07-17T08:35:33.462456+0000","last_peered":"2024-07-17T08:35:33.462456+0000","last_clean":"2024-07-17T08:35:33.462456+0000","last_became_active":"2024-07-17T08:35:33.459129+0000","last_became_peered":"2024-07-17T08:35:33.459129+0000","last_unstale":"2024-07-17T08:35:33.462456+0000","last_undegraded":"2024-07-17T08:35:33.462456+0000","last_fullsized":"2024-07-17T08:35:33.462456+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:48:15.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439921+0000","last_change":"2024-07-17T08:35:33.439921+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439921+0000","last_undegraded":"2024-07-17T08:35:33.439921+0000","last_fullsized":"2024-07-17T08:35:33.439921+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"223'8","reported_seq":26,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451621+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:33.451621+0000","last_peered":"2024-07-17T08:35:33.451621+0000","last_clean":"2024-07-17T08:35:33.451621+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:33.451621+0000","last_undegraded":"2024-07-17T08:35:33.451621+0000","last_fullsized":"2024-07-17T08:35:33.451621+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424179+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:33.424179+0000","last_peered":"2024-07-17T08:35:33.424179+0000","last_clean":"2024-07-17T08:35:33.424179+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:33.424179+0000","last_undegraded":"2024-07-17T08:35:33.424179+0000","last_fullsized":"2024-07-17T08:35:33.424179+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442922+0000","last_change":"2024-07-17T08:35:33.438513+0000","last_active":"2024-07-17T08:35:33.442922+0000","last_peered":"2024-07-17T08:35:33.442922+0000","last_clean":"2024-07-17T08:35:33.442922+0000","last_became_active":"2024-07-17T08:35:33.438129+0000","last_became_peered":"2024-07-17T08:35:33.438129+0000","last_unstale":"2024-07-17T08:35:33.442922+0000","last_undegraded":"2024-07-17T08:35:33.442922+0000","last_fullsized":"2024-07-17T08:35:33.442922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:53.693969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434590+0000","last_change":"2024-07-17T08:35:33.434590+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434590+0000","last_undegraded":"2024-07-17T08:35:33.434590+0000","last_fullsized":"2024-07-17T08:35:33.434590+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":21,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441590+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:33.441590+0000","last_peered":"2024-07-17T08:35:33.441590+0000","last_clean":"2024-07-17T08:35:33.441590+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:33.441590+0000","last_undegraded":"2024-07-17T08:35:33.441590+0000","last_fullsized":"2024-07-17T08:35:33.441590+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441886+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:33.441886+0000","last_peered":"2024-07-17T08:35:33.441886+0000","last_clean":"2024-07-17T08:35:33.441886+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:33.441886+0000","last_undegraded":"2024-07-17T08:35:33.441886+0000","last_fullsized":"2024-07-17T08:35:33.441886+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454203+0000","last_change":"2024-07-17T08:35:33.451275+0000","last_active":"2024-07-17T08:35:33.454203+0000","last_peered":"2024-07-17T08:35:33.454203+0000","last_clean":"2024-07-17T08:35:33.454203+0000","last_became_active":"2024-07-17T08:35:33.450592+0000","last_became_peered":"2024-07-17T08:35:33.450592+0000","last_unstale":"2024-07-17T08:35:33.454203+0000","last_undegraded":"2024-07-17T08:35:33.454203+0000","last_fullsized":"2024-07-17T08:35:33.454203+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:29.313577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.443204+0000","last_change":"2024-07-17T08:35:33.443204+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.443204+0000","last_undegraded":"2024-07-17T08:35:33.443204+0000","last_fullsized":"2024-07-17T08:35:33.443204+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.452712+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:33.452712+0000","last_peered":"2024-07-17T08:35:33.452712+0000","last_clean":"2024-07-17T08:35:33.452712+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:33.452712+0000","last_undegraded":"2024-07-17T08:35:33.452712+0000","last_fullsized":"2024-07-17T08:35:33.452712+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447124+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:33.447124+0000","last_peered":"2024-07-17T08:35:33.447124+0000","last_clean":"2024-07-17T08:35:33.447124+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:33.447124+0000","last_undegraded":"2024-07-17T08:35:33.447124+0000","last_fullsized":"2024-07-17T08:35:33.447124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464465+0000","last_change":"2024-07-17T08:35:33.460898+0000","last_active":"2024-07-17T08:35:33.464465+0000","last_peered":"2024-07-17T08:35:33.464465+0000","last_clean":"2024-07-17T08:35:33.464465+0000","last_became_active":"2024-07-17T08:35:33.460626+0000","last_became_peered":"2024-07-17T08:35:33.460626+0000","last_unstale":"2024-07-17T08:35:33.464465+0000","last_undegraded":"2024-07-17T08:35:33.464465+0000","last_fullsized":"2024-07-17T08:35:33.464465+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:10.867332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433820+0000","last_change":"2024-07-17T08:35:33.433820+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433820+0000","last_undegraded":"2024-07-17T08:35:33.433820+0000","last_fullsized":"2024-07-17T08:35:33.433820+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.426266+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:33.426266+0000","last_peered":"2024-07-17T08:35:33.426266+0000","last_clean":"2024-07-17T08:35:33.426266+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:33.426266+0000","last_undegraded":"2024-07-17T08:35:33.426266+0000","last_fullsized":"2024-07-17T08:35:33.426266+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.434425+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:33.434425+0000","last_peered":"2024-07-17T08:35:33.434425+0000","last_clean":"2024-07-17T08:35:33.434425+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:33.434425+0000","last_undegraded":"2024-07-17T08:35:33.434425+0000","last_fullsized":"2024-07-17T08:35:33.434425+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:21.845715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453162+0000","last_change":"2024-07-17T08:35:33.452238+0000","last_active":"2024-07-17T08:35:33.453162+0000","last_peered":"2024-07-17T08:35:33.453162+0000","last_clean":"2024-07-17T08:35:33.453162+0000","last_became_active":"2024-07-17T08:35:33.450790+0000","last_became_peered":"2024-07-17T08:35:33.450790+0000","last_unstale":"2024-07-17T08:35:33.453162+0000","last_undegraded":"2024-07-17T08:35:33.453162+0000","last_fullsized":"2024-07-17T08:35:33.453162+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:34.886482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440290+0000","last_change":"2024-07-17T08:35:33.440290+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440290+0000","last_undegraded":"2024-07-17T08:35:33.440290+0000","last_fullsized":"2024-07-17T08:35:33.440290+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460635+0000","last_change":"2024-07-17T08:35:33.456972+0000","last_active":"2024-07-17T08:35:33.460635+0000","last_peered":"2024-07-17T08:35:33.460635+0000","last_clean":"2024-07-17T08:35:33.460635+0000","last_became_active":"2024-07-17T08:35:33.452779+0000","last_became_peered":"2024-07-17T08:35:33.452779+0000","last_unstale":"2024-07-17T08:35:33.460635+0000","last_undegraded":"2024-07-17T08:35:33.460635+0000","last_fullsized":"2024-07-17T08:35:33.460635+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:17.921136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446639+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:33.446639+0000","last_peered":"2024-07-17T08:35:33.446639+0000","last_clean":"2024-07-17T08:35:33.446639+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:33.446639+0000","last_undegraded":"2024-07-17T08:35:33.446639+0000","last_fullsized":"2024-07-17T08:35:33.446639+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442475+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:33.442475+0000","last_peered":"2024-07-17T08:35:33.442475+0000","last_clean":"2024-07-17T08:35:33.442475+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:33.442475+0000","last_undegraded":"2024-07-17T08:35:33.442475+0000","last_fullsized":"2024-07-17T08:35:33.442475+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437006+0000","last_change":"2024-07-17T08:35:33.437006+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437006+0000","last_undegraded":"2024-07-17T08:35:33.437006+0000","last_fullsized":"2024-07-17T08:35:33.437006+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448097+0000","last_change":"2024-07-17T08:35:33.446100+0000","last_active":"2024-07-17T08:35:33.448097+0000","last_peered":"2024-07-17T08:35:33.448097+0000","last_clean":"2024-07-17T08:35:33.448097+0000","last_became_active":"2024-07-17T08:35:33.445751+0000","last_became_peered":"2024-07-17T08:35:33.445751+0000","last_unstale":"2024-07-17T08:35:33.448097+0000","last_undegraded":"2024-07-17T08:35:33.448097+0000","last_fullsized":"2024-07-17T08:35:33.448097+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:30.909230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449048+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:33.449048+0000","last_peered":"2024-07-17T08:35:33.449048+0000","last_clean":"2024-07-17T08:35:33.449048+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:33.449048+0000","last_undegraded":"2024-07-17T08:35:33.449048+0000","last_fullsized":"2024-07-17T08:35:33.449048+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437299+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:33.437299+0000","last_peered":"2024-07-17T08:35:33.437299+0000","last_clean":"2024-07-17T08:35:33.437299+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:33.437299+0000","last_undegraded":"2024-07-17T08:35:33.437299+0000","last_fullsized":"2024-07-17T08:35:33.437299+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:19.204797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432982+0000","last_change":"2024-07-17T08:35:33.432982+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432982+0000","last_undegraded":"2024-07-17T08:35:33.432982+0000","last_fullsized":"2024-07-17T08:35:33.432982+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450133+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:33.450133+0000","last_peered":"2024-07-17T08:35:33.450133+0000","last_clean":"2024-07-17T08:35:33.450133+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:33.450133+0000","last_undegraded":"2024-07-17T08:35:33.450133+0000","last_fullsized":"2024-07-17T08:35:33.450133+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437444+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:33.437444+0000","last_peered":"2024-07-17T08:35:33.437444+0000","last_clean":"2024-07-17T08:35:33.437444+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:33.437444+0000","last_undegraded":"2024-07-17T08:35:33.437444+0000","last_fullsized":"2024-07-17T08:35:33.437444+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:41.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460617+0000","last_change":"2024-07-17T08:35:33.458703+0000","last_active":"2024-07-17T08:35:33.460617+0000","last_peered":"2024-07-17T08:35:33.460617+0000","last_clean":"2024-07-17T08:35:33.460617+0000","last_became_active":"2024-07-17T08:35:33.458365+0000","last_became_peered":"2024-07-17T08:35:33.458365+0000","last_unstale":"2024-07-17T08:35:33.460617+0000","last_undegraded":"2024-07-17T08:35:33.460617+0000","last_fullsized":"2024-07-17T08:35:33.460617+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:30.562586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442707+0000","last_change":"2024-07-17T08:35:33.442707+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442707+0000","last_undegraded":"2024-07-17T08:35:33.442707+0000","last_fullsized":"2024-07-17T08:35:33.442707+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440401+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:33.440401+0000","last_peered":"2024-07-17T08:35:33.440401+0000","last_clean":"2024-07-17T08:35:33.440401+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:33.440401+0000","last_undegraded":"2024-07-17T08:35:33.440401+0000","last_fullsized":"2024-07-17T08:35:33.440401+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423664+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:33.423664+0000","last_peered":"2024-07-17T08:35:33.423664+0000","last_clean":"2024-07-17T08:35:33.423664+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:33.423664+0000","last_undegraded":"2024-07-17T08:35:33.423664+0000","last_fullsized":"2024-07-17T08:35:33.423664+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464592+0000","last_change":"2024-07-17T08:35:33.462435+0000","last_active":"2024-07-17T08:35:33.464592+0000","last_peered":"2024-07-17T08:35:33.464592+0000","last_clean":"2024-07-17T08:35:33.464592+0000","last_became_active":"2024-07-17T08:35:33.462140+0000","last_became_peered":"2024-07-17T08:35:33.462140+0000","last_unstale":"2024-07-17T08:35:33.464592+0000","last_undegraded":"2024-07-17T08:35:33.464592+0000","last_fullsized":"2024-07-17T08:35:33.464592+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.853381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435109+0000","last_change":"2024-07-17T08:35:33.435109+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435109+0000","last_undegraded":"2024-07-17T08:35:33.435109+0000","last_fullsized":"2024-07-17T08:35:33.435109+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.444182+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:33.444182+0000","last_peered":"2024-07-17T08:35:33.444182+0000","last_clean":"2024-07-17T08:35:33.444182+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:33.444182+0000","last_undegraded":"2024-07-17T08:35:33.444182+0000","last_fullsized":"2024-07-17T08:35:33.444182+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427463+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:33.427463+0000","last_peered":"2024-07-17T08:35:33.427463+0000","last_clean":"2024-07-17T08:35:33.427463+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:33.427463+0000","last_undegraded":"2024-07-17T08:35:33.427463+0000","last_fullsized":"2024-07-17T08:35:33.427463+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.465443+0000","last_change":"2024-07-17T08:35:33.464947+0000","last_active":"2024-07-17T08:35:33.465443+0000","last_peered":"2024-07-17T08:35:33.465443+0000","last_clean":"2024-07-17T08:35:33.465443+0000","last_became_active":"2024-07-17T08:35:33.464637+0000","last_became_peered":"2024-07-17T08:35:33.464637+0000","last_unstale":"2024-07-17T08:35:33.465443+0000","last_undegraded":"2024-07-17T08:35:33.465443+0000","last_fullsized":"2024-07-17T08:35:33.465443+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:06.263177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429924+0000","last_change":"2024-07-17T08:35:33.429924+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429924+0000","last_undegraded":"2024-07-17T08:35:33.429924+0000","last_fullsized":"2024-07-17T08:35:33.429924+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448794+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:33.448794+0000","last_peered":"2024-07-17T08:35:33.448794+0000","last_clean":"2024-07-17T08:35:33.448794+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:33.448794+0000","last_undegraded":"2024-07-17T08:35:33.448794+0000","last_fullsized":"2024-07-17T08:35:33.448794+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447914+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:33.447914+0000","last_peered":"2024-07-17T08:35:33.447914+0000","last_clean":"2024-07-17T08:35:33.447914+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:33.447914+0000","last_undegraded":"2024-07-17T08:35:33.447914+0000","last_fullsized":"2024-07-17T08:35:33.447914+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:18.513656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454136+0000","last_change":"2024-07-17T08:35:33.450591+0000","last_active":"2024-07-17T08:35:33.454136+0000","last_peered":"2024-07-17T08:35:33.454136+0000","last_clean":"2024-07-17T08:35:33.454136+0000","last_became_active":"2024-07-17T08:35:33.450291+0000","last_became_peered":"2024-07-17T08:35:33.450291+0000","last_unstale":"2024-07-17T08:35:33.454136+0000","last_undegraded":"2024-07-17T08:35:33.454136+0000","last_fullsized":"2024-07-17T08:35:33.454136+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:19.108338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442079+0000","last_change":"2024-07-17T08:35:33.442079+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442079+0000","last_undegraded":"2024-07-17T08:35:33.442079+0000","last_fullsized":"2024-07-17T08:35:33.442079+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":22,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437146+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:33.437146+0000","last_peered":"2024-07-17T08:35:33.437146+0000","last_clean":"2024-07-17T08:35:33.437146+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:33.437146+0000","last_undegraded":"2024-07-17T08:35:33.437146+0000","last_fullsized":"2024-07-17T08:35:33.437146+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441680+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:33.441680+0000","last_peered":"2024-07-17T08:35:33.441680+0000","last_clean":"2024-07-17T08:35:33.441680+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:33.441680+0000","last_undegraded":"2024-07-17T08:35:33.441680+0000","last_fullsized":"2024-07-17T08:35:33.441680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:42.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.455912+0000","last_change":"2024-07-17T08:35:33.454797+0000","last_active":"2024-07-17T08:35:33.455912+0000","last_peered":"2024-07-17T08:35:33.455912+0000","last_clean":"2024-07-17T08:35:33.455912+0000","last_became_active":"2024-07-17T08:35:33.454327+0000","last_became_peered":"2024-07-17T08:35:33.454327+0000","last_unstale":"2024-07-17T08:35:33.455912+0000","last_undegraded":"2024-07-17T08:35:33.455912+0000","last_fullsized":"2024-07-17T08:35:33.455912+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:31:33.235826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441435+0000","last_change":"2024-07-17T08:35:33.441435+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441435+0000","last_undegraded":"2024-07-17T08:35:33.441435+0000","last_fullsized":"2024-07-17T08:35:33.441435+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450033+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:33.450033+0000","last_peered":"2024-07-17T08:35:33.450033+0000","last_clean":"2024-07-17T08:35:33.450033+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:33.450033+0000","last_undegraded":"2024-07-17T08:35:33.450033+0000","last_fullsized":"2024-07-17T08:35:33.450033+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:09:55.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432942+0000","last_change":"2024-07-17T08:35:33.432942+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432942+0000","last_undegraded":"2024-07-17T08:35:33.432942+0000","last_fullsized":"2024-07-17T08:35:33.432942+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.445897+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:33.445897+0000","last_peered":"2024-07-17T08:35:33.445897+0000","last_clean":"2024-07-17T08:35:33.445897+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:33.445897+0000","last_undegraded":"2024-07-17T08:35:33.445897+0000","last_fullsized":"2024-07-17T08:35:33.445897+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:01.628007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434996+0000","last_change":"2024-07-17T08:35:33.434996+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434996+0000","last_undegraded":"2024-07-17T08:35:33.434996+0000","last_fullsized":"2024-07-17T08:35:33.434996+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437023+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:33.437023+0000","last_peered":"2024-07-17T08:35:33.437023+0000","last_clean":"2024-07-17T08:35:33.437023+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:33.437023+0000","last_undegraded":"2024-07-17T08:35:33.437023+0000","last_fullsized":"2024-07-17T08:35:33.437023+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:25.913456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429840+0000","last_change":"2024-07-17T08:35:33.429840+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429840+0000","last_undegraded":"2024-07-17T08:35:33.429840+0000","last_fullsized":"2024-07-17T08:35:33.429840+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442127+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:33.442127+0000","last_peered":"2024-07-17T08:35:33.442127+0000","last_clean":"2024-07-17T08:35:33.442127+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:33.442127+0000","last_undegraded":"2024-07-17T08:35:33.442127+0000","last_fullsized":"2024-07-17T08:35:33.442127+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440642+0000","last_change":"2024-07-17T08:35:33.440642+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440642+0000","last_undegraded":"2024-07-17T08:35:33.440642+0000","last_fullsized":"2024-07-17T08:35:33.440642+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.425002+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:33.425002+0000","last_peered":"2024-07-17T08:35:33.425002+0000","last_clean":"2024-07-17T08:35:33.425002+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:33.425002+0000","last_undegraded":"2024-07-17T08:35:33.425002+0000","last_fullsized":"2024-07-17T08:35:33.425002+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442247+0000","last_change":"2024-07-17T08:35:33.442247+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442247+0000","last_undegraded":"2024-07-17T08:35:33.442247+0000","last_fullsized":"2024-07-17T08:35:33.442247+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448866+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:33.448866+0000","last_peered":"2024-07-17T08:35:33.448866+0000","last_clean":"2024-07-17T08:35:33.448866+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:33.448866+0000","last_undegraded":"2024-07-17T08:35:33.448866+0000","last_fullsized":"2024-07-17T08:35:33.448866+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435729+0000","last_change":"2024-07-17T08:35:33.435729+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435729+0000","last_undegraded":"2024-07-17T08:35:33.435729+0000","last_fullsized":"2024-07-17T08:35:33.435729+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449009+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:33.449009+0000","last_peered":"2024-07-17T08:35:33.449009+0000","last_clean":"2024-07-17T08:35:33.449009+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:33.449009+0000","last_undegraded":"2024-07-17T08:35:33.449009+0000","last_fullsized":"2024-07-17T08:35:33.449009+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:37.046641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429884+0000","last_change":"2024-07-17T08:35:33.429884+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429884+0000","last_undegraded":"2024-07-17T08:35:33.429884+0000","last_fullsized":"2024-07-17T08:35:33.429884+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442377+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:33.442377+0000","last_peered":"2024-07-17T08:35:33.442377+0000","last_clean":"2024-07-17T08:35:33.442377+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:33.442377+0000","last_undegraded":"2024-07-17T08:35:33.442377+0000","last_fullsized":"2024-07-17T08:35:33.442377+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.445996+0000","last_change":"2024-07-17T08:35:33.445996+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.445996+0000","last_undegraded":"2024-07-17T08:35:33.445996+0000","last_fullsized":"2024-07-17T08:35:33.445996+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423937+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:33.423937+0000","last_peered":"2024-07-17T08:35:33.423937+0000","last_clean":"2024-07-17T08:35:33.423937+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:33.423937+0000","last_undegraded":"2024-07-17T08:35:33.423937+0000","last_fullsized":"2024-07-17T08:35:33.423937+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433327+0000","last_change":"2024-07-17T08:35:33.433327+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433327+0000","last_undegraded":"2024-07-17T08:35:33.433327+0000","last_fullsized":"2024-07-17T08:35:33.433327+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443422+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:33.443422+0000","last_peered":"2024-07-17T08:35:33.443422+0000","last_clean":"2024-07-17T08:35:33.443422+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:33.443422+0000","last_undegraded":"2024-07-17T08:35:33.443422+0000","last_fullsized":"2024-07-17T08:35:33.443422+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:00.653492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437539+0000","last_change":"2024-07-17T08:35:33.437539+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437539+0000","last_undegraded":"2024-07-17T08:35:33.437539+0000","last_fullsized":"2024-07-17T08:35:33.437539+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427175+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:33.427175+0000","last_peered":"2024-07-17T08:35:33.427175+0000","last_clean":"2024-07-17T08:35:33.427175+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:33.427175+0000","last_undegraded":"2024-07-17T08:35:33.427175+0000","last_fullsized":"2024-07-17T08:35:33.427175+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431947+0000","last_change":"2024-07-17T08:35:33.431947+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431947+0000","last_undegraded":"2024-07-17T08:35:33.431947+0000","last_fullsized":"2024-07-17T08:35:33.431947+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427344+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:33.427344+0000","last_peered":"2024-07-17T08:35:33.427344+0000","last_clean":"2024-07-17T08:35:33.427344+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:33.427344+0000","last_undegraded":"2024-07-17T08:35:33.427344+0000","last_fullsized":"2024-07-17T08:35:33.427344+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441140+0000","last_change":"2024-07-17T08:35:33.441140+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441140+0000","last_undegraded":"2024-07-17T08:35:33.441140+0000","last_fullsized":"2024-07-17T08:35:33.441140+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.429646+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:33.429646+0000","last_peered":"2024-07-17T08:35:33.429646+0000","last_clean":"2024-07-17T08:35:33.429646+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:33.429646+0000","last_undegraded":"2024-07-17T08:35:33.429646+0000","last_fullsized":"2024-07-17T08:35:33.429646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433840+0000","last_change":"2024-07-17T08:35:33.433840+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433840+0000","last_undegraded":"2024-07-17T08:35:33.433840+0000","last_fullsized":"2024-07-17T08:35:33.433840+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.431825+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:33.431825+0000","last_peered":"2024-07-17T08:35:33.431825+0000","last_clean":"2024-07-17T08:35:33.431825+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:33.431825+0000","last_undegraded":"2024-07-17T08:35:33.431825+0000","last_fullsized":"2024-07-17T08:35:33.431825+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442277+0000","last_change":"2024-07-17T08:35:33.442277+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442277+0000","last_undegraded":"2024-07-17T08:35:33.442277+0000","last_fullsized":"2024-07-17T08:35:33.442277+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440184+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:33.440184+0000","last_peered":"2024-07-17T08:35:33.440184+0000","last_clean":"2024-07-17T08:35:33.440184+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:33.440184+0000","last_undegraded":"2024-07-17T08:35:33.440184+0000","last_fullsized":"2024-07-17T08:35:33.440184+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440659+0000","last_change":"2024-07-17T08:35:33.440659+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440659+0000","last_undegraded":"2024-07-17T08:35:33.440659+0000","last_fullsized":"2024-07-17T08:35:33.440659+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441938+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:33.441938+0000","last_peered":"2024-07-17T08:35:33.441938+0000","last_clean":"2024-07-17T08:35:33.441938+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:33.441938+0000","last_undegraded":"2024-07-17T08:35:33.441938+0000","last_fullsized":"2024-07-17T08:35:33.441938+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432116+0000","last_change":"2024-07-17T08:35:33.432116+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432116+0000","last_undegraded":"2024-07-17T08:35:33.432116+0000","last_fullsized":"2024-07-17T08:35:33.432116+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441757+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:33.441757+0000","last_peered":"2024-07-17T08:35:33.441757+0000","last_clean":"2024-07-17T08:35:33.441757+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:33.441757+0000","last_undegraded":"2024-07-17T08:35:33.441757+0000","last_fullsized":"2024-07-17T08:35:33.441757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442310+0000","last_change":"2024-07-17T08:35:33.442310+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442310+0000","last_undegraded":"2024-07-17T08:35:33.442310+0000","last_fullsized":"2024-07-17T08:35:33.442310+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436435+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:33.436435+0000","last_peered":"2024-07-17T08:35:33.436435+0000","last_clean":"2024-07-17T08:35:33.436435+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:33.436435+0000","last_undegraded":"2024-07-17T08:35:33.436435+0000","last_fullsized":"2024-07-17T08:35:33.436435+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:06.797667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.436288+0000","last_change":"2024-07-17T08:35:33.436288+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.436288+0000","last_undegraded":"2024-07-17T08:35:33.436288+0000","last_fullsized":"2024-07-17T08:35:33.436288+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.426835+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:33.426835+0000","last_peered":"2024-07-17T08:35:33.426835+0000","last_clean":"2024-07-17T08:35:33.426835+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:33.426835+0000","last_undegraded":"2024-07-17T08:35:33.426835+0000","last_fullsized":"2024-07-17T08:35:33.426835+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439473+0000","last_change":"2024-07-17T08:35:33.439473+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439473+0000","last_undegraded":"2024-07-17T08:35:33.439473+0000","last_fullsized":"2024-07-17T08:35:33.439473+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450024+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:33.450024+0000","last_peered":"2024-07-17T08:35:33.450024+0000","last_clean":"2024-07-17T08:35:33.450024+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:33.450024+0000","last_undegraded":"2024-07-17T08:35:33.450024+0000","last_fullsized":"2024-07-17T08:35:33.450024+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:41.247825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441612+0000","last_change":"2024-07-17T08:35:33.441612+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441612+0000","last_undegraded":"2024-07-17T08:35:33.441612+0000","last_fullsized":"2024-07-17T08:35:33.441612+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424947+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:33.424947+0000","last_peered":"2024-07-17T08:35:33.424947+0000","last_clean":"2024-07-17T08:35:33.424947+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:33.424947+0000","last_undegraded":"2024-07-17T08:35:33.424947+0000","last_fullsized":"2024-07-17T08:35:33.424947+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431955+0000","last_change":"2024-07-17T08:35:33.431955+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431955+0000","last_undegraded":"2024-07-17T08:35:33.431955+0000","last_fullsized":"2024-07-17T08:35:33.431955+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448704+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:33.448704+0000","last_peered":"2024-07-17T08:35:33.448704+0000","last_clean":"2024-07-17T08:35:33.448704+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:33.448704+0000","last_undegraded":"2024-07-17T08:35:33.448704+0000","last_fullsized":"2024-07-17T08:35:33.448704+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.436574+0000","last_change":"2024-07-17T08:35:33.436574+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.436574+0000","last_undegraded":"2024-07-17T08:35:33.436574+0000","last_fullsized":"2024-07-17T08:35:33.436574+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.435998+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:33.435998+0000","last_peered":"2024-07-17T08:35:33.435998+0000","last_clean":"2024-07-17T08:35:33.435998+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:33.435998+0000","last_undegraded":"2024-07-17T08:35:33.435998+0000","last_fullsized":"2024-07-17T08:35:33.435998+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431069+0000","last_change":"2024-07-17T08:35:33.431069+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431069+0000","last_undegraded":"2024-07-17T08:35:33.431069+0000","last_fullsized":"2024-07-17T08:35:33.431069+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440838+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:33.440838+0000","last_peered":"2024-07-17T08:35:33.440838+0000","last_clean":"2024-07-17T08:35:33.440838+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:33.440838+0000","last_undegraded":"2024-07-17T08:35:33.440838+0000","last_fullsized":"2024-07-17T08:35:33.440838+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439842+0000","last_change":"2024-07-17T08:35:33.439842+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439842+0000","last_undegraded":"2024-07-17T08:35:33.439842+0000","last_fullsized":"2024-07-17T08:35:33.439842+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.432767+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:33.432767+0000","last_peered":"2024-07-17T08:35:33.432767+0000","last_clean":"2024-07-17T08:35:33.432767+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:33.432767+0000","last_undegraded":"2024-07-17T08:35:33.432767+0000","last_fullsized":"2024-07-17T08:35:33.432767+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440554+0000","last_change":"2024-07-17T08:35:33.440554+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440554+0000","last_undegraded":"2024-07-17T08:35:33.440554+0000","last_fullsized":"2024-07-17T08:35:33.440554+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423143+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:33.423143+0000","last_peered":"2024-07-17T08:35:33.423143+0000","last_clean":"2024-07-17T08:35:33.423143+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:33.423143+0000","last_undegraded":"2024-07-17T08:35:33.423143+0000","last_fullsized":"2024-07-17T08:35:33.423143+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437241+0000","last_change":"2024-07-17T08:35:33.437241+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437241+0000","last_undegraded":"2024-07-17T08:35:33.437241+0000","last_fullsized":"2024-07-17T08:35:33.437241+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449482+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:33.449482+0000","last_peered":"2024-07-17T08:35:33.449482+0000","last_clean":"2024-07-17T08:35:33.449482+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:33.449482+0000","last_undegraded":"2024-07-17T08:35:33.449482+0000","last_fullsized":"2024-07-17T08:35:33.449482+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:52.497030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432242+0000","last_change":"2024-07-17T08:35:33.432242+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432242+0000","last_undegraded":"2024-07-17T08:35:33.432242+0000","last_fullsized":"2024-07-17T08:35:33.432242+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423373+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:33.423373+0000","last_peered":"2024-07-17T08:35:33.423373+0000","last_clean":"2024-07-17T08:35:33.423373+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:33.423373+0000","last_undegraded":"2024-07-17T08:35:33.423373+0000","last_fullsized":"2024-07-17T08:35:33.423373+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433338+0000","last_change":"2024-07-17T08:35:33.433338+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433338+0000","last_undegraded":"2024-07-17T08:35:33.433338+0000","last_fullsized":"2024-07-17T08:35:33.433338+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451507+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:33.451507+0000","last_peered":"2024-07-17T08:35:33.451507+0000","last_clean":"2024-07-17T08:35:33.451507+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:33.451507+0000","last_undegraded":"2024-07-17T08:35:33.451507+0000","last_fullsized":"2024-07-17T08:35:33.451507+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:34.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442878+0000","last_change":"2024-07-17T08:35:33.442878+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442878+0000","last_undegraded":"2024-07-17T08:35:33.442878+0000","last_fullsized":"2024-07-17T08:35:33.442878+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448535+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:33.448535+0000","last_peered":"2024-07-17T08:35:33.448535+0000","last_clean":"2024-07-17T08:35:33.448535+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:33.448535+0000","last_undegraded":"2024-07-17T08:35:33.448535+0000","last_fullsized":"2024-07-17T08:35:33.448535+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.427411+0000","last_change":"2024-07-17T08:35:33.427411+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.427411+0000","last_undegraded":"2024-07-17T08:35:33.427411+0000","last_fullsized":"2024-07-17T08:35:33.427411+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448060+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:33.448060+0000","last_peered":"2024-07-17T08:35:33.448060+0000","last_clean":"2024-07-17T08:35:33.448060+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:33.448060+0000","last_undegraded":"2024-07-17T08:35:33.448060+0000","last_fullsized":"2024-07-17T08:35:33.448060+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:11.863693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439822+0000","last_change":"2024-07-17T08:35:33.439822+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439822+0000","last_undegraded":"2024-07-17T08:35:33.439822+0000","last_fullsized":"2024-07-17T08:35:33.439822+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449109+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:33.449109+0000","last_peered":"2024-07-17T08:35:33.449109+0000","last_clean":"2024-07-17T08:35:33.449109+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:33.449109+0000","last_undegraded":"2024-07-17T08:35:33.449109+0000","last_fullsized":"2024-07-17T08:35:33.449109+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:22.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543344,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65044,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306796,"statfs":{"total":96636764160,"available":96570159104,"internally_reserved":0,"allocated":3416064,"data_stored":2120307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510639,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29788,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342052,"statfs":{"total":96636764160,"available":96606261248,"internally_reserved":0,"allocated":2834432,"data_stored":1538555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510640,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47816,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324024,"statfs":{"total":96636764160,"available":96587800576,"internally_reserved":0,"allocated":2813952,"data_stored":1527067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47812,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324028,"statfs":{"total":96636764160,"available":96587804672,"internally_reserved":0,"allocated":2809856,"data_stored":1527593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510639,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51980,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319860,"statfs":{"total":96636764160,"available":96583536640,"internally_reserved":0,"allocated":2818048,"data_stored":1527137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543343,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2809856,"data_stored":1527101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510640,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47808,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324032,"statfs":{"total":96636764160,"available":96587808768,"internally_reserved":0,"allocated":2805760,"data_stored":1527011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510639,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56724,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315116,"statfs":{"total":96636764160,"available":96578678784,"internally_reserved":0,"allocated":3416064,"data_stored":2125573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:38.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:39.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.197+0000 7f55dae00640 1 -- 172.21.15.44:0/102860362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d40b09a0 msgr2=0x7f55d4073d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.197+0000 7f55dae00640 1 --2- 172.21.15.44:0/102860362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d40b09a0 0x7f55d4073d70 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009900 tx=0x7f55cc02a690 comp rx=0 tx=0).stop 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.197+0000 7f55dae00640 1 -- 172.21.15.44:0/102860362 shutdown_connections 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.197+0000 7f55dae00640 1 --2- 172.21.15.44:0/102860362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d40b09a0 0x7f55d4073d70 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.197+0000 7f55dae00640 1 -- 172.21.15.44:0/102860362 >> 172.21.15.44:0/102860362 conn(0x7f55d40b3090 msgr2=0x7f55d40b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.198+0000 7f55dae00640 1 -- 172.21.15.44:0/102860362 shutdown_connections 2024-07-17T08:35:39.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.198+0000 7f55dae00640 1 -- 172.21.15.44:0/102860362 wait complete. 2024-07-17T08:35:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.199+0000 7f55dae00640 1 Processor -- start 2024-07-17T08:35:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.200+0000 7f55dae00640 1 -- start start 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d40b09a0 0x7f55d414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d40b5c40 0x7f55d414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40b3f10 con 0x7f55d415b030 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40b3d90 con 0x7f55d40b5c40 2024-07-17T08:35:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55dae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55d40742b0 con 0x7f55d40b09a0 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39720/0 (socket says 172.21.15.44:39720) 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 -- 172.21.15.44:0/511357700 learned_addr learned my addr 172.21.15.44:0/511357700 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55d9a00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d40b09a0 0x7f55d414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55d9000640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d40b5c40 0x7f55d414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 -- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d40b09a0 msgr2=0x7f55d414eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d40b09a0 0x7f55d414eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 -- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d40b5c40 msgr2=0x7f55d414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d40b5c40 0x7f55d414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.201+0000 7f55da400640 1 -- 172.21.15.44:0/511357700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d414f110 con 0x7f55d415b030 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55da400640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009230 tx=0x7f55cc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc03f070 con 0x7f55d415b030 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55cc004890 con 0x7f55d415b030 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d414f3f0 con 0x7f55d415b030 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc035030 con 0x7f55d415b030 2024-07-17T08:35:39.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.202+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d414c8d0 con 0x7f55d415b030 2024-07-17T08:35:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.204+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d40694a0 con 0x7f55d415b030 2024-07-17T08:35:39.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.205+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f55cc042070 con 0x7f55d415b030 2024-07-17T08:35:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.206+0000 7f55d1600640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 0x7f55b40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.206+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f55cc0b9b00 con 0x7f55d415b030 2024-07-17T08:35:39.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.206+0000 7f55d9a00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 0x7f55b40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.208+0000 7f55d9a00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 0x7f55b40791c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001ce0 tx=0x7f55c4009040 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:39.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.211+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55d40694a0 con 0x7f55d415b030 2024-07-17T08:35:39.258 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-07-17T08:35:39.256+0000 7f93fe200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:35:39.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.421+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55d414f910 con 0x7f55b4076d00 2024-07-17T08:35:39.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.443+0000 7f55d1600640 1 -- 172.21.15.44:0/511357700 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+486104 (secure 0 0 0) 0x7f55d414f910 con 0x7f55b4076d00 2024-07-17T08:35:39.455 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:39.458 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 msgr2=0x7f55b40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 0x7f55b40791c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001ce0 tx=0x7f55c4009040 comp rx=0 tx=0).stop 2024-07-17T08:35:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 msgr2=0x7f55d414cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009230 tx=0x7f55cc003960 comp rx=0 tx=0).stop 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 shutdown_connections 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d415b030 0x7f55d414cdf0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f55b4076d00 0x7f55b40791c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55d40b5c40 0x7f55d414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 --2- 172.21.15.44:0/511357700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55d40b09a0 0x7f55d414eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 >> 172.21.15.44:0/511357700 conn(0x7f55d40b3090 msgr2=0x7f55d4158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.454+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 shutdown_connections 2024-07-17T08:35:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.455+0000 7f55dae00640 1 -- 172.21.15.44:0/511357700 wait complete. 2024-07-17T08:35:39.477 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-07-17T08:35:37.765627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594024,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":406,"num_write_kb":1644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2552,"ondisk_log_size":2552,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398944,"kb_used_data":23168,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754575776,"statfs":{"total":773094113280,"available":772685594624,"internally_reserved":0,"allocated":23724032,"data_stored":13420344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1246,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.351964"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.444506+0000","last_change":"2024-07-17T08:35:33.444506+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.444506+0000","last_undegraded":"2024-07-17T08:35:33.444506+0000","last_fullsized":"2024-07-17T08:35:33.444506+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3382,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450159+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:33.450159+0000","last_peered":"2024-07-17T08:35:33.450159+0000","last_clean":"2024-07-17T08:35:33.450159+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:33.450159+0000","last_undegraded":"2024-07-17T08:35:33.450159+0000","last_fullsized":"2024-07-17T08:35:33.450159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441990+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:33.441990+0000","last_peered":"2024-07-17T08:35:33.441990+0000","last_clean":"2024-07-17T08:35:33.441990+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:33.441990+0000","last_undegraded":"2024-07-17T08:35:33.441990+0000","last_fullsized":"2024-07-17T08:35:33.441990+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448569+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:33.448569+0000","last_peered":"2024-07-17T08:35:33.448569+0000","last_clean":"2024-07-17T08:35:33.448569+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:33.448569+0000","last_undegraded":"2024-07-17T08:35:33.448569+0000","last_fullsized":"2024-07-17T08:35:33.448569+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:44.992431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.458346+0000","last_change":"2024-07-17T08:35:33.455926+0000","last_active":"2024-07-17T08:35:33.458346+0000","last_peered":"2024-07-17T08:35:33.458346+0000","last_clean":"2024-07-17T08:35:33.458346+0000","last_became_active":"2024-07-17T08:35:33.455626+0000","last_became_peered":"2024-07-17T08:35:33.455626+0000","last_unstale":"2024-07-17T08:35:33.458346+0000","last_undegraded":"2024-07-17T08:35:33.458346+0000","last_fullsized":"2024-07-17T08:35:33.458346+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:59.073309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441144+0000","last_change":"2024-07-17T08:35:33.441144+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441144+0000","last_undegraded":"2024-07-17T08:35:33.441144+0000","last_fullsized":"2024-07-17T08:35:33.441144+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424194+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:33.424194+0000","last_peered":"2024-07-17T08:35:33.424194+0000","last_clean":"2024-07-17T08:35:33.424194+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:33.424194+0000","last_undegraded":"2024-07-17T08:35:33.424194+0000","last_fullsized":"2024-07-17T08:35:33.424194+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451029+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:33.451029+0000","last_peered":"2024-07-17T08:35:33.451029+0000","last_clean":"2024-07-17T08:35:33.451029+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:33.451029+0000","last_undegraded":"2024-07-17T08:35:33.451029+0000","last_fullsized":"2024-07-17T08:35:33.451029+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453562+0000","last_change":"2024-07-17T08:35:33.451371+0000","last_active":"2024-07-17T08:35:33.453562+0000","last_peered":"2024-07-17T08:35:33.453562+0000","last_clean":"2024-07-17T08:35:33.453562+0000","last_became_active":"2024-07-17T08:35:33.451107+0000","last_became_peered":"2024-07-17T08:35:33.451107+0000","last_unstale":"2024-07-17T08:35:33.453562+0000","last_undegraded":"2024-07-17T08:35:33.453562+0000","last_fullsized":"2024-07-17T08:35:33.453562+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:28.506108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435146+0000","last_change":"2024-07-17T08:35:33.435146+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435146+0000","last_undegraded":"2024-07-17T08:35:33.435146+0000","last_fullsized":"2024-07-17T08:35:33.435146+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448125+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:33.448125+0000","last_peered":"2024-07-17T08:35:33.448125+0000","last_clean":"2024-07-17T08:35:33.448125+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:33.448125+0000","last_undegraded":"2024-07-17T08:35:33.448125+0000","last_fullsized":"2024-07-17T08:35:33.448125+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.452598+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:33.452598+0000","last_peered":"2024-07-17T08:35:33.452598+0000","last_clean":"2024-07-17T08:35:33.452598+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:33.452598+0000","last_undegraded":"2024-07-17T08:35:33.452598+0000","last_fullsized":"2024-07-17T08:35:33.452598+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:08.117588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464320+0000","last_change":"2024-07-17T08:35:33.459624+0000","last_active":"2024-07-17T08:35:33.464320+0000","last_peered":"2024-07-17T08:35:33.464320+0000","last_clean":"2024-07-17T08:35:33.464320+0000","last_became_active":"2024-07-17T08:35:33.459359+0000","last_became_peered":"2024-07-17T08:35:33.459359+0000","last_unstale":"2024-07-17T08:35:33.464320+0000","last_undegraded":"2024-07-17T08:35:33.464320+0000","last_fullsized":"2024-07-17T08:35:33.464320+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:37.256027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429954+0000","last_change":"2024-07-17T08:35:33.429954+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429954+0000","last_undegraded":"2024-07-17T08:35:33.429954+0000","last_fullsized":"2024-07-17T08:35:33.429954+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442098+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:33.442098+0000","last_peered":"2024-07-17T08:35:33.442098+0000","last_clean":"2024-07-17T08:35:33.442098+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:33.442098+0000","last_undegraded":"2024-07-17T08:35:33.442098+0000","last_fullsized":"2024-07-17T08:35:33.442098+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451543+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:33.451543+0000","last_peered":"2024-07-17T08:35:33.451543+0000","last_clean":"2024-07-17T08:35:33.451543+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:33.451543+0000","last_undegraded":"2024-07-17T08:35:33.451543+0000","last_fullsized":"2024-07-17T08:35:33.451543+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:20.349285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461817+0000","last_change":"2024-07-17T08:35:33.457849+0000","last_active":"2024-07-17T08:35:33.461817+0000","last_peered":"2024-07-17T08:35:33.461817+0000","last_clean":"2024-07-17T08:35:33.461817+0000","last_became_active":"2024-07-17T08:35:33.453123+0000","last_became_peered":"2024-07-17T08:35:33.453123+0000","last_unstale":"2024-07-17T08:35:33.461817+0000","last_undegraded":"2024-07-17T08:35:33.461817+0000","last_fullsized":"2024-07-17T08:35:33.461817+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:06:12.277866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.438424+0000","last_change":"2024-07-17T08:35:33.438424+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.438424+0000","last_undegraded":"2024-07-17T08:35:33.438424+0000","last_fullsized":"2024-07-17T08:35:33.438424+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.431997+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:33.431997+0000","last_peered":"2024-07-17T08:35:33.431997+0000","last_clean":"2024-07-17T08:35:33.431997+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:33.431997+0000","last_undegraded":"2024-07-17T08:35:33.431997+0000","last_fullsized":"2024-07-17T08:35:33.431997+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.429975+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:33.429975+0000","last_peered":"2024-07-17T08:35:33.429975+0000","last_clean":"2024-07-17T08:35:33.429975+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:33.429975+0000","last_undegraded":"2024-07-17T08:35:33.429975+0000","last_fullsized":"2024-07-17T08:35:33.429975+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453756+0000","last_change":"2024-07-17T08:35:33.452063+0000","last_active":"2024-07-17T08:35:33.453756+0000","last_peered":"2024-07-17T08:35:33.453756+0000","last_clean":"2024-07-17T08:35:33.453756+0000","last_became_active":"2024-07-17T08:35:33.450965+0000","last_became_peered":"2024-07-17T08:35:33.450965+0000","last_unstale":"2024-07-17T08:35:33.453756+0000","last_undegraded":"2024-07-17T08:35:33.453756+0000","last_fullsized":"2024-07-17T08:35:33.453756+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:16.564132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431703+0000","last_change":"2024-07-17T08:35:33.431703+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431703+0000","last_undegraded":"2024-07-17T08:35:33.431703+0000","last_fullsized":"2024-07-17T08:35:33.431703+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448163+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:33.448163+0000","last_peered":"2024-07-17T08:35:33.448163+0000","last_clean":"2024-07-17T08:35:33.448163+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:33.448163+0000","last_undegraded":"2024-07-17T08:35:33.448163+0000","last_fullsized":"2024-07-17T08:35:33.448163+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446939+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:33.446939+0000","last_peered":"2024-07-17T08:35:33.446939+0000","last_clean":"2024-07-17T08:35:33.446939+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:33.446939+0000","last_undegraded":"2024-07-17T08:35:33.446939+0000","last_fullsized":"2024-07-17T08:35:33.446939+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:32.339990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446768+0000","last_change":"2024-07-17T08:35:33.443001+0000","last_active":"2024-07-17T08:35:33.446768+0000","last_peered":"2024-07-17T08:35:33.446768+0000","last_clean":"2024-07-17T08:35:33.446768+0000","last_became_active":"2024-07-17T08:35:33.442630+0000","last_became_peered":"2024-07-17T08:35:33.442630+0000","last_unstale":"2024-07-17T08:35:33.446768+0000","last_undegraded":"2024-07-17T08:35:33.446768+0000","last_fullsized":"2024-07-17T08:35:33.446768+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:57.867743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433338+0000","last_change":"2024-07-17T08:35:33.433338+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433338+0000","last_undegraded":"2024-07-17T08:35:33.433338+0000","last_fullsized":"2024-07-17T08:35:33.433338+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436851+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:33.436851+0000","last_peered":"2024-07-17T08:35:33.436851+0000","last_clean":"2024-07-17T08:35:33.436851+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:33.436851+0000","last_undegraded":"2024-07-17T08:35:33.436851+0000","last_fullsized":"2024-07-17T08:35:33.436851+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441736+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:33.441736+0000","last_peered":"2024-07-17T08:35:33.441736+0000","last_clean":"2024-07-17T08:35:33.441736+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:33.441736+0000","last_undegraded":"2024-07-17T08:35:33.441736+0000","last_fullsized":"2024-07-17T08:35:33.441736+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454181+0000","last_change":"2024-07-17T08:35:33.447597+0000","last_active":"2024-07-17T08:35:33.454181+0000","last_peered":"2024-07-17T08:35:33.454181+0000","last_clean":"2024-07-17T08:35:33.454181+0000","last_became_active":"2024-07-17T08:35:33.446794+0000","last_became_peered":"2024-07-17T08:35:33.446794+0000","last_unstale":"2024-07-17T08:35:33.454181+0000","last_undegraded":"2024-07-17T08:35:33.454181+0000","last_fullsized":"2024-07-17T08:35:33.454181+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:24.117005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430173+0000","last_change":"2024-07-17T08:35:33.430173+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430173+0000","last_undegraded":"2024-07-17T08:35:33.430173+0000","last_fullsized":"2024-07-17T08:35:33.430173+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436157+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:33.436157+0000","last_peered":"2024-07-17T08:35:33.436157+0000","last_clean":"2024-07-17T08:35:33.436157+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:33.436157+0000","last_undegraded":"2024-07-17T08:35:33.436157+0000","last_fullsized":"2024-07-17T08:35:33.436157+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448059+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:33.448059+0000","last_peered":"2024-07-17T08:35:33.448059+0000","last_clean":"2024-07-17T08:35:33.448059+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:33.448059+0000","last_undegraded":"2024-07-17T08:35:33.448059+0000","last_fullsized":"2024-07-17T08:35:33.448059+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464485+0000","last_change":"2024-07-17T08:35:33.461285+0000","last_active":"2024-07-17T08:35:33.464485+0000","last_peered":"2024-07-17T08:35:33.464485+0000","last_clean":"2024-07-17T08:35:33.464485+0000","last_became_active":"2024-07-17T08:35:33.460353+0000","last_became_peered":"2024-07-17T08:35:33.460353+0000","last_unstale":"2024-07-17T08:35:33.464485+0000","last_undegraded":"2024-07-17T08:35:33.464485+0000","last_fullsized":"2024-07-17T08:35:33.464485+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:53.232196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430826+0000","last_change":"2024-07-17T08:35:33.430826+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430826+0000","last_undegraded":"2024-07-17T08:35:33.430826+0000","last_fullsized":"2024-07-17T08:35:33.430826+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442600+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:33.442600+0000","last_peered":"2024-07-17T08:35:33.442600+0000","last_clean":"2024-07-17T08:35:33.442600+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:33.442600+0000","last_undegraded":"2024-07-17T08:35:33.442600+0000","last_fullsized":"2024-07-17T08:35:33.442600+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.433201+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:33.433201+0000","last_peered":"2024-07-17T08:35:33.433201+0000","last_clean":"2024-07-17T08:35:33.433201+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:33.433201+0000","last_undegraded":"2024-07-17T08:35:33.433201+0000","last_fullsized":"2024-07-17T08:35:33.433201+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460912+0000","last_change":"2024-07-17T08:35:33.458859+0000","last_active":"2024-07-17T08:35:33.460912+0000","last_peered":"2024-07-17T08:35:33.460912+0000","last_clean":"2024-07-17T08:35:33.460912+0000","last_became_active":"2024-07-17T08:35:33.458540+0000","last_became_peered":"2024-07-17T08:35:33.458540+0000","last_unstale":"2024-07-17T08:35:33.460912+0000","last_undegraded":"2024-07-17T08:35:33.460912+0000","last_fullsized":"2024-07-17T08:35:33.460912+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:59.715580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.444703+0000","last_change":"2024-07-17T08:35:33.444703+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.444703+0000","last_undegraded":"2024-07-17T08:35:33.444703+0000","last_fullsized":"2024-07-17T08:35:33.444703+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.438968+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:33.438968+0000","last_peered":"2024-07-17T08:35:33.438968+0000","last_clean":"2024-07-17T08:35:33.438968+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:33.438968+0000","last_undegraded":"2024-07-17T08:35:33.438968+0000","last_fullsized":"2024-07-17T08:35:33.438968+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.438487+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:33.438487+0000","last_peered":"2024-07-17T08:35:33.438487+0000","last_clean":"2024-07-17T08:35:33.438487+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:33.438487+0000","last_undegraded":"2024-07-17T08:35:33.438487+0000","last_fullsized":"2024-07-17T08:35:33.438487+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451248+0000","last_change":"2024-07-17T08:35:33.449985+0000","last_active":"2024-07-17T08:35:33.451248+0000","last_peered":"2024-07-17T08:35:33.451248+0000","last_clean":"2024-07-17T08:35:33.451248+0000","last_became_active":"2024-07-17T08:35:33.449677+0000","last_became_peered":"2024-07-17T08:35:33.449677+0000","last_unstale":"2024-07-17T08:35:33.451248+0000","last_undegraded":"2024-07-17T08:35:33.451248+0000","last_fullsized":"2024-07-17T08:35:33.451248+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:45.362621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433025+0000","last_change":"2024-07-17T08:35:33.433025+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433025+0000","last_undegraded":"2024-07-17T08:35:33.433025+0000","last_fullsized":"2024-07-17T08:35:33.433025+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436626+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:33.436626+0000","last_peered":"2024-07-17T08:35:33.436626+0000","last_clean":"2024-07-17T08:35:33.436626+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:33.436626+0000","last_undegraded":"2024-07-17T08:35:33.436626+0000","last_fullsized":"2024-07-17T08:35:33.436626+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440415+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:33.440415+0000","last_peered":"2024-07-17T08:35:33.440415+0000","last_clean":"2024-07-17T08:35:33.440415+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:33.440415+0000","last_undegraded":"2024-07-17T08:35:33.440415+0000","last_fullsized":"2024-07-17T08:35:33.440415+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.455745+0000","last_change":"2024-07-17T08:35:33.455119+0000","last_active":"2024-07-17T08:35:33.455745+0000","last_peered":"2024-07-17T08:35:33.455745+0000","last_clean":"2024-07-17T08:35:33.455745+0000","last_became_active":"2024-07-17T08:35:33.454773+0000","last_became_peered":"2024-07-17T08:35:33.454773+0000","last_unstale":"2024-07-17T08:35:33.455745+0000","last_undegraded":"2024-07-17T08:35:33.455745+0000","last_fullsized":"2024-07-17T08:35:33.455745+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:02:33.574937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.430713+0000","last_change":"2024-07-17T08:35:33.430713+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.430713+0000","last_undegraded":"2024-07-17T08:35:33.430713+0000","last_fullsized":"2024-07-17T08:35:33.430713+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443666+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:33.443666+0000","last_peered":"2024-07-17T08:35:33.443666+0000","last_clean":"2024-07-17T08:35:33.443666+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:33.443666+0000","last_undegraded":"2024-07-17T08:35:33.443666+0000","last_fullsized":"2024-07-17T08:35:33.443666+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451420+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:33.451420+0000","last_peered":"2024-07-17T08:35:33.451420+0000","last_clean":"2024-07-17T08:35:33.451420+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:33.451420+0000","last_undegraded":"2024-07-17T08:35:33.451420+0000","last_fullsized":"2024-07-17T08:35:33.451420+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:36.700604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461838+0000","last_change":"2024-07-17T08:35:33.457888+0000","last_active":"2024-07-17T08:35:33.461838+0000","last_peered":"2024-07-17T08:35:33.461838+0000","last_clean":"2024-07-17T08:35:33.461838+0000","last_became_active":"2024-07-17T08:35:33.457510+0000","last_became_peered":"2024-07-17T08:35:33.457510+0000","last_unstale":"2024-07-17T08:35:33.461838+0000","last_undegraded":"2024-07-17T08:35:33.461838+0000","last_fullsized":"2024-07-17T08:35:33.461838+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:01.792237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439003+0000","last_change":"2024-07-17T08:35:33.439003+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439003+0000","last_undegraded":"2024-07-17T08:35:33.439003+0000","last_fullsized":"2024-07-17T08:35:33.439003+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424352+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:33.424352+0000","last_peered":"2024-07-17T08:35:33.424352+0000","last_clean":"2024-07-17T08:35:33.424352+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:33.424352+0000","last_undegraded":"2024-07-17T08:35:33.424352+0000","last_fullsized":"2024-07-17T08:35:33.424352+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443960+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:33.443960+0000","last_peered":"2024-07-17T08:35:33.443960+0000","last_clean":"2024-07-17T08:35:33.443960+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:33.443960+0000","last_undegraded":"2024-07-17T08:35:33.443960+0000","last_fullsized":"2024-07-17T08:35:33.443960+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:53.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460619+0000","last_change":"2024-07-17T08:35:33.456857+0000","last_active":"2024-07-17T08:35:33.460619+0000","last_peered":"2024-07-17T08:35:33.460619+0000","last_clean":"2024-07-17T08:35:33.460619+0000","last_became_active":"2024-07-17T08:35:33.456290+0000","last_became_peered":"2024-07-17T08:35:33.456290+0000","last_unstale":"2024-07-17T08:35:33.460619+0000","last_undegraded":"2024-07-17T08:35:33.460619+0000","last_fullsized":"2024-07-17T08:35:33.460619+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:47.684683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439099+0000","last_change":"2024-07-17T08:35:33.439099+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439099+0000","last_undegraded":"2024-07-17T08:35:33.439099+0000","last_fullsized":"2024-07-17T08:35:33.439099+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.425122+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:33.425122+0000","last_peered":"2024-07-17T08:35:33.425122+0000","last_clean":"2024-07-17T08:35:33.425122+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:33.425122+0000","last_undegraded":"2024-07-17T08:35:33.425122+0000","last_fullsized":"2024-07-17T08:35:33.425122+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.428509+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:33.428509+0000","last_peered":"2024-07-17T08:35:33.428509+0000","last_clean":"2024-07-17T08:35:33.428509+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:33.428509+0000","last_undegraded":"2024-07-17T08:35:33.428509+0000","last_fullsized":"2024-07-17T08:35:33.428509+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454922+0000","last_change":"2024-07-17T08:35:33.454020+0000","last_active":"2024-07-17T08:35:33.454922+0000","last_peered":"2024-07-17T08:35:33.454922+0000","last_clean":"2024-07-17T08:35:33.454922+0000","last_became_active":"2024-07-17T08:35:33.453743+0000","last_became_peered":"2024-07-17T08:35:33.453743+0000","last_unstale":"2024-07-17T08:35:33.454922+0000","last_undegraded":"2024-07-17T08:35:33.454922+0000","last_fullsized":"2024-07-17T08:35:33.454922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:52.231608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434167+0000","last_change":"2024-07-17T08:35:33.434167+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434167+0000","last_undegraded":"2024-07-17T08:35:33.434167+0000","last_fullsized":"2024-07-17T08:35:33.434167+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.444327+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:33.444327+0000","last_peered":"2024-07-17T08:35:33.444327+0000","last_clean":"2024-07-17T08:35:33.444327+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:33.444327+0000","last_undegraded":"2024-07-17T08:35:33.444327+0000","last_fullsized":"2024-07-17T08:35:33.444327+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447324+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:33.447324+0000","last_peered":"2024-07-17T08:35:33.447324+0000","last_clean":"2024-07-17T08:35:33.447324+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:33.447324+0000","last_undegraded":"2024-07-17T08:35:33.447324+0000","last_fullsized":"2024-07-17T08:35:33.447324+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:07.288084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454222+0000","last_change":"2024-07-17T08:35:33.451492+0000","last_active":"2024-07-17T08:35:33.454222+0000","last_peered":"2024-07-17T08:35:33.454222+0000","last_clean":"2024-07-17T08:35:33.454222+0000","last_became_active":"2024-07-17T08:35:33.451016+0000","last_became_peered":"2024-07-17T08:35:33.451016+0000","last_unstale":"2024-07-17T08:35:33.454222+0000","last_undegraded":"2024-07-17T08:35:33.454222+0000","last_fullsized":"2024-07-17T08:35:33.454222+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:53:03.903417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432858+0000","last_change":"2024-07-17T08:35:33.432858+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432858+0000","last_undegraded":"2024-07-17T08:35:33.432858+0000","last_fullsized":"2024-07-17T08:35:33.432858+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448902+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:33.448902+0000","last_peered":"2024-07-17T08:35:33.448902+0000","last_clean":"2024-07-17T08:35:33.448902+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:33.448902+0000","last_undegraded":"2024-07-17T08:35:33.448902+0000","last_fullsized":"2024-07-17T08:35:33.448902+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448027+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:33.448027+0000","last_peered":"2024-07-17T08:35:33.448027+0000","last_clean":"2024-07-17T08:35:33.448027+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:33.448027+0000","last_undegraded":"2024-07-17T08:35:33.448027+0000","last_fullsized":"2024-07-17T08:35:33.448027+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:44.865077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443689+0000","last_change":"2024-07-17T08:35:33.440197+0000","last_active":"2024-07-17T08:35:33.443689+0000","last_peered":"2024-07-17T08:35:33.443689+0000","last_clean":"2024-07-17T08:35:33.443689+0000","last_became_active":"2024-07-17T08:35:33.439893+0000","last_became_peered":"2024-07-17T08:35:33.439893+0000","last_unstale":"2024-07-17T08:35:33.443689+0000","last_undegraded":"2024-07-17T08:35:33.443689+0000","last_fullsized":"2024-07-17T08:35:33.443689+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:03.810647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432715+0000","last_change":"2024-07-17T08:35:33.432715+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432715+0000","last_undegraded":"2024-07-17T08:35:33.432715+0000","last_fullsized":"2024-07-17T08:35:33.432715+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449207+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:33.449207+0000","last_peered":"2024-07-17T08:35:33.449207+0000","last_clean":"2024-07-17T08:35:33.449207+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:33.449207+0000","last_undegraded":"2024-07-17T08:35:33.449207+0000","last_fullsized":"2024-07-17T08:35:33.449207+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.432996+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:33.432996+0000","last_peered":"2024-07-17T08:35:33.432996+0000","last_clean":"2024-07-17T08:35:33.432996+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:33.432996+0000","last_undegraded":"2024-07-17T08:35:33.432996+0000","last_fullsized":"2024-07-17T08:35:33.432996+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453504+0000","last_change":"2024-07-17T08:35:33.451532+0000","last_active":"2024-07-17T08:35:33.453504+0000","last_peered":"2024-07-17T08:35:33.453504+0000","last_clean":"2024-07-17T08:35:33.453504+0000","last_became_active":"2024-07-17T08:35:33.449530+0000","last_became_peered":"2024-07-17T08:35:33.449530+0000","last_unstale":"2024-07-17T08:35:33.453504+0000","last_undegraded":"2024-07-17T08:35:33.453504+0000","last_fullsized":"2024-07-17T08:35:33.453504+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:28:19.068669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442530+0000","last_change":"2024-07-17T08:35:33.442530+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442530+0000","last_undegraded":"2024-07-17T08:35:33.442530+0000","last_fullsized":"2024-07-17T08:35:33.442530+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":26,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424081+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:33.424081+0000","last_peered":"2024-07-17T08:35:33.424081+0000","last_clean":"2024-07-17T08:35:33.424081+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:33.424081+0000","last_undegraded":"2024-07-17T08:35:33.424081+0000","last_fullsized":"2024-07-17T08:35:33.424081+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454471+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:33.454471+0000","last_peered":"2024-07-17T08:35:33.454471+0000","last_clean":"2024-07-17T08:35:33.454471+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:33.454471+0000","last_undegraded":"2024-07-17T08:35:33.454471+0000","last_fullsized":"2024-07-17T08:35:33.454471+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:38.094519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.457534+0000","last_change":"2024-07-17T08:35:33.456460+0000","last_active":"2024-07-17T08:35:33.457534+0000","last_peered":"2024-07-17T08:35:33.457534+0000","last_clean":"2024-07-17T08:35:33.457534+0000","last_became_active":"2024-07-17T08:35:33.456193+0000","last_became_peered":"2024-07-17T08:35:33.456193+0000","last_unstale":"2024-07-17T08:35:33.457534+0000","last_undegraded":"2024-07-17T08:35:33.457534+0000","last_fullsized":"2024-07-17T08:35:33.457534+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:11.023729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.445347+0000","last_change":"2024-07-17T08:35:33.445347+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.445347+0000","last_undegraded":"2024-07-17T08:35:33.445347+0000","last_fullsized":"2024-07-17T08:35:33.445347+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.439847+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:33.439847+0000","last_peered":"2024-07-17T08:35:33.439847+0000","last_clean":"2024-07-17T08:35:33.439847+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:33.439847+0000","last_undegraded":"2024-07-17T08:35:33.439847+0000","last_fullsized":"2024-07-17T08:35:33.439847+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440307+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:33.440307+0000","last_peered":"2024-07-17T08:35:33.440307+0000","last_clean":"2024-07-17T08:35:33.440307+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:33.440307+0000","last_undegraded":"2024-07-17T08:35:33.440307+0000","last_fullsized":"2024-07-17T08:35:33.440307+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.462214+0000","last_change":"2024-07-17T08:35:33.460434+0000","last_active":"2024-07-17T08:35:33.462214+0000","last_peered":"2024-07-17T08:35:33.462214+0000","last_clean":"2024-07-17T08:35:33.462214+0000","last_became_active":"2024-07-17T08:35:33.460174+0000","last_became_peered":"2024-07-17T08:35:33.460174+0000","last_unstale":"2024-07-17T08:35:33.462214+0000","last_undegraded":"2024-07-17T08:35:33.462214+0000","last_fullsized":"2024-07-17T08:35:33.462214+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:17.721570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433830+0000","last_change":"2024-07-17T08:35:33.433830+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433830+0000","last_undegraded":"2024-07-17T08:35:33.433830+0000","last_fullsized":"2024-07-17T08:35:33.433830+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448041+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:33.448041+0000","last_peered":"2024-07-17T08:35:33.448041+0000","last_clean":"2024-07-17T08:35:33.448041+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:33.448041+0000","last_undegraded":"2024-07-17T08:35:33.448041+0000","last_fullsized":"2024-07-17T08:35:33.448041+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449170+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:33.449170+0000","last_peered":"2024-07-17T08:35:33.449170+0000","last_clean":"2024-07-17T08:35:33.449170+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:33.449170+0000","last_undegraded":"2024-07-17T08:35:33.449170+0000","last_fullsized":"2024-07-17T08:35:33.449170+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.461042+0000","last_change":"2024-07-17T08:35:33.458710+0000","last_active":"2024-07-17T08:35:33.461042+0000","last_peered":"2024-07-17T08:35:33.461042+0000","last_clean":"2024-07-17T08:35:33.461042+0000","last_became_active":"2024-07-17T08:35:33.458420+0000","last_became_peered":"2024-07-17T08:35:33.458420+0000","last_unstale":"2024-07-17T08:35:33.461042+0000","last_undegraded":"2024-07-17T08:35:33.461042+0000","last_fullsized":"2024-07-17T08:35:33.461042+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.438593+0000","last_change":"2024-07-17T08:35:33.438593+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.438593+0000","last_undegraded":"2024-07-17T08:35:33.438593+0000","last_fullsized":"2024-07-17T08:35:33.438593+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443790+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:33.443790+0000","last_peered":"2024-07-17T08:35:33.443790+0000","last_clean":"2024-07-17T08:35:33.443790+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:33.443790+0000","last_undegraded":"2024-07-17T08:35:33.443790+0000","last_fullsized":"2024-07-17T08:35:33.443790+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443850+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:33.443850+0000","last_peered":"2024-07-17T08:35:33.443850+0000","last_clean":"2024-07-17T08:35:33.443850+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:33.443850+0000","last_undegraded":"2024-07-17T08:35:33.443850+0000","last_fullsized":"2024-07-17T08:35:33.443850+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:07:04.780778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.462456+0000","last_change":"2024-07-17T08:35:33.459431+0000","last_active":"2024-07-17T08:35:33.462456+0000","last_peered":"2024-07-17T08:35:33.462456+0000","last_clean":"2024-07-17T08:35:33.462456+0000","last_became_active":"2024-07-17T08:35:33.459129+0000","last_became_peered":"2024-07-17T08:35:33.459129+0000","last_unstale":"2024-07-17T08:35:33.462456+0000","last_undegraded":"2024-07-17T08:35:33.462456+0000","last_fullsized":"2024-07-17T08:35:33.462456+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:48:15.650144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439921+0000","last_change":"2024-07-17T08:35:33.439921+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439921+0000","last_undegraded":"2024-07-17T08:35:33.439921+0000","last_fullsized":"2024-07-17T08:35:33.439921+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"223'8","reported_seq":26,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451621+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:33.451621+0000","last_peered":"2024-07-17T08:35:33.451621+0000","last_clean":"2024-07-17T08:35:33.451621+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:33.451621+0000","last_undegraded":"2024-07-17T08:35:33.451621+0000","last_fullsized":"2024-07-17T08:35:33.451621+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424179+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:33.424179+0000","last_peered":"2024-07-17T08:35:33.424179+0000","last_clean":"2024-07-17T08:35:33.424179+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:33.424179+0000","last_undegraded":"2024-07-17T08:35:33.424179+0000","last_fullsized":"2024-07-17T08:35:33.424179+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442922+0000","last_change":"2024-07-17T08:35:33.438513+0000","last_active":"2024-07-17T08:35:33.442922+0000","last_peered":"2024-07-17T08:35:33.442922+0000","last_clean":"2024-07-17T08:35:33.442922+0000","last_became_active":"2024-07-17T08:35:33.438129+0000","last_became_peered":"2024-07-17T08:35:33.438129+0000","last_unstale":"2024-07-17T08:35:33.442922+0000","last_undegraded":"2024-07-17T08:35:33.442922+0000","last_fullsized":"2024-07-17T08:35:33.442922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:53.693969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434590+0000","last_change":"2024-07-17T08:35:33.434590+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434590+0000","last_undegraded":"2024-07-17T08:35:33.434590+0000","last_fullsized":"2024-07-17T08:35:33.434590+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":21,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441590+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:33.441590+0000","last_peered":"2024-07-17T08:35:33.441590+0000","last_clean":"2024-07-17T08:35:33.441590+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:33.441590+0000","last_undegraded":"2024-07-17T08:35:33.441590+0000","last_fullsized":"2024-07-17T08:35:33.441590+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441886+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:33.441886+0000","last_peered":"2024-07-17T08:35:33.441886+0000","last_clean":"2024-07-17T08:35:33.441886+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:33.441886+0000","last_undegraded":"2024-07-17T08:35:33.441886+0000","last_fullsized":"2024-07-17T08:35:33.441886+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454203+0000","last_change":"2024-07-17T08:35:33.451275+0000","last_active":"2024-07-17T08:35:33.454203+0000","last_peered":"2024-07-17T08:35:33.454203+0000","last_clean":"2024-07-17T08:35:33.454203+0000","last_became_active":"2024-07-17T08:35:33.450592+0000","last_became_peered":"2024-07-17T08:35:33.450592+0000","last_unstale":"2024-07-17T08:35:33.454203+0000","last_undegraded":"2024-07-17T08:35:33.454203+0000","last_fullsized":"2024-07-17T08:35:33.454203+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:29.313577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.443204+0000","last_change":"2024-07-17T08:35:33.443204+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.443204+0000","last_undegraded":"2024-07-17T08:35:33.443204+0000","last_fullsized":"2024-07-17T08:35:33.443204+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.452712+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:33.452712+0000","last_peered":"2024-07-17T08:35:33.452712+0000","last_clean":"2024-07-17T08:35:33.452712+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:33.452712+0000","last_undegraded":"2024-07-17T08:35:33.452712+0000","last_fullsized":"2024-07-17T08:35:33.452712+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447124+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:33.447124+0000","last_peered":"2024-07-17T08:35:33.447124+0000","last_clean":"2024-07-17T08:35:33.447124+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:33.447124+0000","last_undegraded":"2024-07-17T08:35:33.447124+0000","last_fullsized":"2024-07-17T08:35:33.447124+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464465+0000","last_change":"2024-07-17T08:35:33.460898+0000","last_active":"2024-07-17T08:35:33.464465+0000","last_peered":"2024-07-17T08:35:33.464465+0000","last_clean":"2024-07-17T08:35:33.464465+0000","last_became_active":"2024-07-17T08:35:33.460626+0000","last_became_peered":"2024-07-17T08:35:33.460626+0000","last_unstale":"2024-07-17T08:35:33.464465+0000","last_undegraded":"2024-07-17T08:35:33.464465+0000","last_fullsized":"2024-07-17T08:35:33.464465+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:10.867332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433820+0000","last_change":"2024-07-17T08:35:33.433820+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433820+0000","last_undegraded":"2024-07-17T08:35:33.433820+0000","last_fullsized":"2024-07-17T08:35:33.433820+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.426266+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:33.426266+0000","last_peered":"2024-07-17T08:35:33.426266+0000","last_clean":"2024-07-17T08:35:33.426266+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:33.426266+0000","last_undegraded":"2024-07-17T08:35:33.426266+0000","last_fullsized":"2024-07-17T08:35:33.426266+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.434425+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:33.434425+0000","last_peered":"2024-07-17T08:35:33.434425+0000","last_clean":"2024-07-17T08:35:33.434425+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:33.434425+0000","last_undegraded":"2024-07-17T08:35:33.434425+0000","last_fullsized":"2024-07-17T08:35:33.434425+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:21.845715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.453162+0000","last_change":"2024-07-17T08:35:33.452238+0000","last_active":"2024-07-17T08:35:33.453162+0000","last_peered":"2024-07-17T08:35:33.453162+0000","last_clean":"2024-07-17T08:35:33.453162+0000","last_became_active":"2024-07-17T08:35:33.450790+0000","last_became_peered":"2024-07-17T08:35:33.450790+0000","last_unstale":"2024-07-17T08:35:33.453162+0000","last_undegraded":"2024-07-17T08:35:33.453162+0000","last_fullsized":"2024-07-17T08:35:33.453162+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:34.886482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440290+0000","last_change":"2024-07-17T08:35:33.440290+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440290+0000","last_undegraded":"2024-07-17T08:35:33.440290+0000","last_fullsized":"2024-07-17T08:35:33.440290+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460635+0000","last_change":"2024-07-17T08:35:33.456972+0000","last_active":"2024-07-17T08:35:33.460635+0000","last_peered":"2024-07-17T08:35:33.460635+0000","last_clean":"2024-07-17T08:35:33.460635+0000","last_became_active":"2024-07-17T08:35:33.452779+0000","last_became_peered":"2024-07-17T08:35:33.452779+0000","last_unstale":"2024-07-17T08:35:33.460635+0000","last_undegraded":"2024-07-17T08:35:33.460635+0000","last_fullsized":"2024-07-17T08:35:33.460635+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:17.921136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.446639+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:33.446639+0000","last_peered":"2024-07-17T08:35:33.446639+0000","last_clean":"2024-07-17T08:35:33.446639+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:33.446639+0000","last_undegraded":"2024-07-17T08:35:33.446639+0000","last_fullsized":"2024-07-17T08:35:33.446639+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442475+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:33.442475+0000","last_peered":"2024-07-17T08:35:33.442475+0000","last_clean":"2024-07-17T08:35:33.442475+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:33.442475+0000","last_undegraded":"2024-07-17T08:35:33.442475+0000","last_fullsized":"2024-07-17T08:35:33.442475+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437006+0000","last_change":"2024-07-17T08:35:33.437006+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437006+0000","last_undegraded":"2024-07-17T08:35:33.437006+0000","last_fullsized":"2024-07-17T08:35:33.437006+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448097+0000","last_change":"2024-07-17T08:35:33.446100+0000","last_active":"2024-07-17T08:35:33.448097+0000","last_peered":"2024-07-17T08:35:33.448097+0000","last_clean":"2024-07-17T08:35:33.448097+0000","last_became_active":"2024-07-17T08:35:33.445751+0000","last_became_peered":"2024-07-17T08:35:33.445751+0000","last_unstale":"2024-07-17T08:35:33.448097+0000","last_undegraded":"2024-07-17T08:35:33.448097+0000","last_fullsized":"2024-07-17T08:35:33.448097+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:30.909230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":18,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449048+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:33.449048+0000","last_peered":"2024-07-17T08:35:33.449048+0000","last_clean":"2024-07-17T08:35:33.449048+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:33.449048+0000","last_undegraded":"2024-07-17T08:35:33.449048+0000","last_fullsized":"2024-07-17T08:35:33.449048+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437299+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:33.437299+0000","last_peered":"2024-07-17T08:35:33.437299+0000","last_clean":"2024-07-17T08:35:33.437299+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:33.437299+0000","last_undegraded":"2024-07-17T08:35:33.437299+0000","last_fullsized":"2024-07-17T08:35:33.437299+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:19.204797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432982+0000","last_change":"2024-07-17T08:35:33.432982+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432982+0000","last_undegraded":"2024-07-17T08:35:33.432982+0000","last_fullsized":"2024-07-17T08:35:33.432982+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450133+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:33.450133+0000","last_peered":"2024-07-17T08:35:33.450133+0000","last_clean":"2024-07-17T08:35:33.450133+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:33.450133+0000","last_undegraded":"2024-07-17T08:35:33.450133+0000","last_fullsized":"2024-07-17T08:35:33.450133+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437444+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:33.437444+0000","last_peered":"2024-07-17T08:35:33.437444+0000","last_clean":"2024-07-17T08:35:33.437444+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:33.437444+0000","last_undegraded":"2024-07-17T08:35:33.437444+0000","last_fullsized":"2024-07-17T08:35:33.437444+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:41.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.460617+0000","last_change":"2024-07-17T08:35:33.458703+0000","last_active":"2024-07-17T08:35:33.460617+0000","last_peered":"2024-07-17T08:35:33.460617+0000","last_clean":"2024-07-17T08:35:33.460617+0000","last_became_active":"2024-07-17T08:35:33.458365+0000","last_became_peered":"2024-07-17T08:35:33.458365+0000","last_unstale":"2024-07-17T08:35:33.460617+0000","last_undegraded":"2024-07-17T08:35:33.460617+0000","last_fullsized":"2024-07-17T08:35:33.460617+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:30.562586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442707+0000","last_change":"2024-07-17T08:35:33.442707+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442707+0000","last_undegraded":"2024-07-17T08:35:33.442707+0000","last_fullsized":"2024-07-17T08:35:33.442707+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440401+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:33.440401+0000","last_peered":"2024-07-17T08:35:33.440401+0000","last_clean":"2024-07-17T08:35:33.440401+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:33.440401+0000","last_undegraded":"2024-07-17T08:35:33.440401+0000","last_fullsized":"2024-07-17T08:35:33.440401+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423664+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:33.423664+0000","last_peered":"2024-07-17T08:35:33.423664+0000","last_clean":"2024-07-17T08:35:33.423664+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:33.423664+0000","last_undegraded":"2024-07-17T08:35:33.423664+0000","last_fullsized":"2024-07-17T08:35:33.423664+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.464592+0000","last_change":"2024-07-17T08:35:33.462435+0000","last_active":"2024-07-17T08:35:33.464592+0000","last_peered":"2024-07-17T08:35:33.464592+0000","last_clean":"2024-07-17T08:35:33.464592+0000","last_became_active":"2024-07-17T08:35:33.462140+0000","last_became_peered":"2024-07-17T08:35:33.462140+0000","last_unstale":"2024-07-17T08:35:33.464592+0000","last_undegraded":"2024-07-17T08:35:33.464592+0000","last_fullsized":"2024-07-17T08:35:33.464592+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.853381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435109+0000","last_change":"2024-07-17T08:35:33.435109+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435109+0000","last_undegraded":"2024-07-17T08:35:33.435109+0000","last_fullsized":"2024-07-17T08:35:33.435109+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":20,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.444182+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:33.444182+0000","last_peered":"2024-07-17T08:35:33.444182+0000","last_clean":"2024-07-17T08:35:33.444182+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:33.444182+0000","last_undegraded":"2024-07-17T08:35:33.444182+0000","last_fullsized":"2024-07-17T08:35:33.444182+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427463+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:33.427463+0000","last_peered":"2024-07-17T08:35:33.427463+0000","last_clean":"2024-07-17T08:35:33.427463+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:33.427463+0000","last_undegraded":"2024-07-17T08:35:33.427463+0000","last_fullsized":"2024-07-17T08:35:33.427463+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.465443+0000","last_change":"2024-07-17T08:35:33.464947+0000","last_active":"2024-07-17T08:35:33.465443+0000","last_peered":"2024-07-17T08:35:33.465443+0000","last_clean":"2024-07-17T08:35:33.465443+0000","last_became_active":"2024-07-17T08:35:33.464637+0000","last_became_peered":"2024-07-17T08:35:33.464637+0000","last_unstale":"2024-07-17T08:35:33.465443+0000","last_undegraded":"2024-07-17T08:35:33.465443+0000","last_fullsized":"2024-07-17T08:35:33.465443+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:36:06.263177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429924+0000","last_change":"2024-07-17T08:35:33.429924+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429924+0000","last_undegraded":"2024-07-17T08:35:33.429924+0000","last_fullsized":"2024-07-17T08:35:33.429924+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":19,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448794+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:33.448794+0000","last_peered":"2024-07-17T08:35:33.448794+0000","last_clean":"2024-07-17T08:35:33.448794+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:33.448794+0000","last_undegraded":"2024-07-17T08:35:33.448794+0000","last_fullsized":"2024-07-17T08:35:33.448794+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.447914+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:33.447914+0000","last_peered":"2024-07-17T08:35:33.447914+0000","last_clean":"2024-07-17T08:35:33.447914+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:33.447914+0000","last_undegraded":"2024-07-17T08:35:33.447914+0000","last_fullsized":"2024-07-17T08:35:33.447914+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:18.513656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.454136+0000","last_change":"2024-07-17T08:35:33.450591+0000","last_active":"2024-07-17T08:35:33.454136+0000","last_peered":"2024-07-17T08:35:33.454136+0000","last_clean":"2024-07-17T08:35:33.454136+0000","last_became_active":"2024-07-17T08:35:33.450291+0000","last_became_peered":"2024-07-17T08:35:33.450291+0000","last_unstale":"2024-07-17T08:35:33.454136+0000","last_undegraded":"2024-07-17T08:35:33.454136+0000","last_fullsized":"2024-07-17T08:35:33.454136+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:19.108338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442079+0000","last_change":"2024-07-17T08:35:33.442079+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442079+0000","last_undegraded":"2024-07-17T08:35:33.442079+0000","last_fullsized":"2024-07-17T08:35:33.442079+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":22,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437146+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:33.437146+0000","last_peered":"2024-07-17T08:35:33.437146+0000","last_clean":"2024-07-17T08:35:33.437146+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:33.437146+0000","last_undegraded":"2024-07-17T08:35:33.437146+0000","last_fullsized":"2024-07-17T08:35:33.437146+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441680+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:33.441680+0000","last_peered":"2024-07-17T08:35:33.441680+0000","last_clean":"2024-07-17T08:35:33.441680+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:33.441680+0000","last_undegraded":"2024-07-17T08:35:33.441680+0000","last_fullsized":"2024-07-17T08:35:33.441680+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:42.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.455912+0000","last_change":"2024-07-17T08:35:33.454797+0000","last_active":"2024-07-17T08:35:33.455912+0000","last_peered":"2024-07-17T08:35:33.455912+0000","last_clean":"2024-07-17T08:35:33.455912+0000","last_became_active":"2024-07-17T08:35:33.454327+0000","last_became_peered":"2024-07-17T08:35:33.454327+0000","last_unstale":"2024-07-17T08:35:33.455912+0000","last_undegraded":"2024-07-17T08:35:33.455912+0000","last_fullsized":"2024-07-17T08:35:33.455912+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:31:33.235826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441435+0000","last_change":"2024-07-17T08:35:33.441435+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441435+0000","last_undegraded":"2024-07-17T08:35:33.441435+0000","last_fullsized":"2024-07-17T08:35:33.441435+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450033+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:33.450033+0000","last_peered":"2024-07-17T08:35:33.450033+0000","last_clean":"2024-07-17T08:35:33.450033+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:33.450033+0000","last_undegraded":"2024-07-17T08:35:33.450033+0000","last_fullsized":"2024-07-17T08:35:33.450033+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:09:55.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432942+0000","last_change":"2024-07-17T08:35:33.432942+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432942+0000","last_undegraded":"2024-07-17T08:35:33.432942+0000","last_fullsized":"2024-07-17T08:35:33.432942+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.445897+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:33.445897+0000","last_peered":"2024-07-17T08:35:33.445897+0000","last_clean":"2024-07-17T08:35:33.445897+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:33.445897+0000","last_undegraded":"2024-07-17T08:35:33.445897+0000","last_fullsized":"2024-07-17T08:35:33.445897+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:01.628007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.434996+0000","last_change":"2024-07-17T08:35:33.434996+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.434996+0000","last_undegraded":"2024-07-17T08:35:33.434996+0000","last_fullsized":"2024-07-17T08:35:33.434996+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.437023+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:33.437023+0000","last_peered":"2024-07-17T08:35:33.437023+0000","last_clean":"2024-07-17T08:35:33.437023+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:33.437023+0000","last_undegraded":"2024-07-17T08:35:33.437023+0000","last_fullsized":"2024-07-17T08:35:33.437023+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:25.913456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429840+0000","last_change":"2024-07-17T08:35:33.429840+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429840+0000","last_undegraded":"2024-07-17T08:35:33.429840+0000","last_fullsized":"2024-07-17T08:35:33.429840+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442127+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:33.442127+0000","last_peered":"2024-07-17T08:35:33.442127+0000","last_clean":"2024-07-17T08:35:33.442127+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:33.442127+0000","last_undegraded":"2024-07-17T08:35:33.442127+0000","last_fullsized":"2024-07-17T08:35:33.442127+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440642+0000","last_change":"2024-07-17T08:35:33.440642+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440642+0000","last_undegraded":"2024-07-17T08:35:33.440642+0000","last_fullsized":"2024-07-17T08:35:33.440642+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.425002+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:33.425002+0000","last_peered":"2024-07-17T08:35:33.425002+0000","last_clean":"2024-07-17T08:35:33.425002+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:33.425002+0000","last_undegraded":"2024-07-17T08:35:33.425002+0000","last_fullsized":"2024-07-17T08:35:33.425002+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442247+0000","last_change":"2024-07-17T08:35:33.442247+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442247+0000","last_undegraded":"2024-07-17T08:35:33.442247+0000","last_fullsized":"2024-07-17T08:35:33.442247+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448866+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:33.448866+0000","last_peered":"2024-07-17T08:35:33.448866+0000","last_clean":"2024-07-17T08:35:33.448866+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:33.448866+0000","last_undegraded":"2024-07-17T08:35:33.448866+0000","last_fullsized":"2024-07-17T08:35:33.448866+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.435729+0000","last_change":"2024-07-17T08:35:33.435729+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.435729+0000","last_undegraded":"2024-07-17T08:35:33.435729+0000","last_fullsized":"2024-07-17T08:35:33.435729+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449009+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:33.449009+0000","last_peered":"2024-07-17T08:35:33.449009+0000","last_clean":"2024-07-17T08:35:33.449009+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:33.449009+0000","last_undegraded":"2024-07-17T08:35:33.449009+0000","last_fullsized":"2024-07-17T08:35:33.449009+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:37.046641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.429884+0000","last_change":"2024-07-17T08:35:33.429884+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.429884+0000","last_undegraded":"2024-07-17T08:35:33.429884+0000","last_fullsized":"2024-07-17T08:35:33.429884+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.442377+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:33.442377+0000","last_peered":"2024-07-17T08:35:33.442377+0000","last_clean":"2024-07-17T08:35:33.442377+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:33.442377+0000","last_undegraded":"2024-07-17T08:35:33.442377+0000","last_fullsized":"2024-07-17T08:35:33.442377+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.445996+0000","last_change":"2024-07-17T08:35:33.445996+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.445996+0000","last_undegraded":"2024-07-17T08:35:33.445996+0000","last_fullsized":"2024-07-17T08:35:33.445996+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423937+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:33.423937+0000","last_peered":"2024-07-17T08:35:33.423937+0000","last_clean":"2024-07-17T08:35:33.423937+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:33.423937+0000","last_undegraded":"2024-07-17T08:35:33.423937+0000","last_fullsized":"2024-07-17T08:35:33.423937+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433327+0000","last_change":"2024-07-17T08:35:33.433327+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433327+0000","last_undegraded":"2024-07-17T08:35:33.433327+0000","last_fullsized":"2024-07-17T08:35:33.433327+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.443422+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:33.443422+0000","last_peered":"2024-07-17T08:35:33.443422+0000","last_clean":"2024-07-17T08:35:33.443422+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:33.443422+0000","last_undegraded":"2024-07-17T08:35:33.443422+0000","last_fullsized":"2024-07-17T08:35:33.443422+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:00.653492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437539+0000","last_change":"2024-07-17T08:35:33.437539+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437539+0000","last_undegraded":"2024-07-17T08:35:33.437539+0000","last_fullsized":"2024-07-17T08:35:33.437539+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427175+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:33.427175+0000","last_peered":"2024-07-17T08:35:33.427175+0000","last_clean":"2024-07-17T08:35:33.427175+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:33.427175+0000","last_undegraded":"2024-07-17T08:35:33.427175+0000","last_fullsized":"2024-07-17T08:35:33.427175+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431947+0000","last_change":"2024-07-17T08:35:33.431947+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431947+0000","last_undegraded":"2024-07-17T08:35:33.431947+0000","last_fullsized":"2024-07-17T08:35:33.431947+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.427344+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:33.427344+0000","last_peered":"2024-07-17T08:35:33.427344+0000","last_clean":"2024-07-17T08:35:33.427344+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:33.427344+0000","last_undegraded":"2024-07-17T08:35:33.427344+0000","last_fullsized":"2024-07-17T08:35:33.427344+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441140+0000","last_change":"2024-07-17T08:35:33.441140+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441140+0000","last_undegraded":"2024-07-17T08:35:33.441140+0000","last_fullsized":"2024-07-17T08:35:33.441140+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.429646+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:33.429646+0000","last_peered":"2024-07-17T08:35:33.429646+0000","last_clean":"2024-07-17T08:35:33.429646+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:33.429646+0000","last_undegraded":"2024-07-17T08:35:33.429646+0000","last_fullsized":"2024-07-17T08:35:33.429646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433840+0000","last_change":"2024-07-17T08:35:33.433840+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433840+0000","last_undegraded":"2024-07-17T08:35:33.433840+0000","last_fullsized":"2024-07-17T08:35:33.433840+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.431825+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:33.431825+0000","last_peered":"2024-07-17T08:35:33.431825+0000","last_clean":"2024-07-17T08:35:33.431825+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:33.431825+0000","last_undegraded":"2024-07-17T08:35:33.431825+0000","last_fullsized":"2024-07-17T08:35:33.431825+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442277+0000","last_change":"2024-07-17T08:35:33.442277+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442277+0000","last_undegraded":"2024-07-17T08:35:33.442277+0000","last_fullsized":"2024-07-17T08:35:33.442277+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440184+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:33.440184+0000","last_peered":"2024-07-17T08:35:33.440184+0000","last_clean":"2024-07-17T08:35:33.440184+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:33.440184+0000","last_undegraded":"2024-07-17T08:35:33.440184+0000","last_fullsized":"2024-07-17T08:35:33.440184+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440659+0000","last_change":"2024-07-17T08:35:33.440659+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440659+0000","last_undegraded":"2024-07-17T08:35:33.440659+0000","last_fullsized":"2024-07-17T08:35:33.440659+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441938+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:33.441938+0000","last_peered":"2024-07-17T08:35:33.441938+0000","last_clean":"2024-07-17T08:35:33.441938+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:33.441938+0000","last_undegraded":"2024-07-17T08:35:33.441938+0000","last_fullsized":"2024-07-17T08:35:33.441938+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432116+0000","last_change":"2024-07-17T08:35:33.432116+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432116+0000","last_undegraded":"2024-07-17T08:35:33.432116+0000","last_fullsized":"2024-07-17T08:35:33.432116+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.441757+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:33.441757+0000","last_peered":"2024-07-17T08:35:33.441757+0000","last_clean":"2024-07-17T08:35:33.441757+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:33.441757+0000","last_undegraded":"2024-07-17T08:35:33.441757+0000","last_fullsized":"2024-07-17T08:35:33.441757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442310+0000","last_change":"2024-07-17T08:35:33.442310+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442310+0000","last_undegraded":"2024-07-17T08:35:33.442310+0000","last_fullsized":"2024-07-17T08:35:33.442310+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.436435+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:33.436435+0000","last_peered":"2024-07-17T08:35:33.436435+0000","last_clean":"2024-07-17T08:35:33.436435+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:33.436435+0000","last_undegraded":"2024-07-17T08:35:33.436435+0000","last_fullsized":"2024-07-17T08:35:33.436435+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:06.797667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.436288+0000","last_change":"2024-07-17T08:35:33.436288+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.436288+0000","last_undegraded":"2024-07-17T08:35:33.436288+0000","last_fullsized":"2024-07-17T08:35:33.436288+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.426835+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:33.426835+0000","last_peered":"2024-07-17T08:35:33.426835+0000","last_clean":"2024-07-17T08:35:33.426835+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:33.426835+0000","last_undegraded":"2024-07-17T08:35:33.426835+0000","last_fullsized":"2024-07-17T08:35:33.426835+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439473+0000","last_change":"2024-07-17T08:35:33.439473+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439473+0000","last_undegraded":"2024-07-17T08:35:33.439473+0000","last_fullsized":"2024-07-17T08:35:33.439473+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.450024+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:33.450024+0000","last_peered":"2024-07-17T08:35:33.450024+0000","last_clean":"2024-07-17T08:35:33.450024+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:33.450024+0000","last_undegraded":"2024-07-17T08:35:33.450024+0000","last_fullsized":"2024-07-17T08:35:33.450024+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:41.247825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.441612+0000","last_change":"2024-07-17T08:35:33.441612+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.441612+0000","last_undegraded":"2024-07-17T08:35:33.441612+0000","last_fullsized":"2024-07-17T08:35:33.441612+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.424947+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:33.424947+0000","last_peered":"2024-07-17T08:35:33.424947+0000","last_clean":"2024-07-17T08:35:33.424947+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:33.424947+0000","last_undegraded":"2024-07-17T08:35:33.424947+0000","last_fullsized":"2024-07-17T08:35:33.424947+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431955+0000","last_change":"2024-07-17T08:35:33.431955+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431955+0000","last_undegraded":"2024-07-17T08:35:33.431955+0000","last_fullsized":"2024-07-17T08:35:33.431955+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448704+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:33.448704+0000","last_peered":"2024-07-17T08:35:33.448704+0000","last_clean":"2024-07-17T08:35:33.448704+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:33.448704+0000","last_undegraded":"2024-07-17T08:35:33.448704+0000","last_fullsized":"2024-07-17T08:35:33.448704+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.436574+0000","last_change":"2024-07-17T08:35:33.436574+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.436574+0000","last_undegraded":"2024-07-17T08:35:33.436574+0000","last_fullsized":"2024-07-17T08:35:33.436574+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.435998+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:33.435998+0000","last_peered":"2024-07-17T08:35:33.435998+0000","last_clean":"2024-07-17T08:35:33.435998+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:33.435998+0000","last_undegraded":"2024-07-17T08:35:33.435998+0000","last_fullsized":"2024-07-17T08:35:33.435998+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.431069+0000","last_change":"2024-07-17T08:35:33.431069+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.431069+0000","last_undegraded":"2024-07-17T08:35:33.431069+0000","last_fullsized":"2024-07-17T08:35:33.431069+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.440838+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:33.440838+0000","last_peered":"2024-07-17T08:35:33.440838+0000","last_clean":"2024-07-17T08:35:33.440838+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:33.440838+0000","last_undegraded":"2024-07-17T08:35:33.440838+0000","last_fullsized":"2024-07-17T08:35:33.440838+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439842+0000","last_change":"2024-07-17T08:35:33.439842+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439842+0000","last_undegraded":"2024-07-17T08:35:33.439842+0000","last_fullsized":"2024-07-17T08:35:33.439842+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.432767+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:33.432767+0000","last_peered":"2024-07-17T08:35:33.432767+0000","last_clean":"2024-07-17T08:35:33.432767+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:33.432767+0000","last_undegraded":"2024-07-17T08:35:33.432767+0000","last_fullsized":"2024-07-17T08:35:33.432767+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.440554+0000","last_change":"2024-07-17T08:35:33.440554+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.440554+0000","last_undegraded":"2024-07-17T08:35:33.440554+0000","last_fullsized":"2024-07-17T08:35:33.440554+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423143+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:33.423143+0000","last_peered":"2024-07-17T08:35:33.423143+0000","last_clean":"2024-07-17T08:35:33.423143+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:33.423143+0000","last_undegraded":"2024-07-17T08:35:33.423143+0000","last_fullsized":"2024-07-17T08:35:33.423143+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.437241+0000","last_change":"2024-07-17T08:35:33.437241+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.437241+0000","last_undegraded":"2024-07-17T08:35:33.437241+0000","last_fullsized":"2024-07-17T08:35:33.437241+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449482+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:33.449482+0000","last_peered":"2024-07-17T08:35:33.449482+0000","last_clean":"2024-07-17T08:35:33.449482+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:33.449482+0000","last_undegraded":"2024-07-17T08:35:33.449482+0000","last_fullsized":"2024-07-17T08:35:33.449482+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:52.497030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.432242+0000","last_change":"2024-07-17T08:35:33.432242+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.432242+0000","last_undegraded":"2024-07-17T08:35:33.432242+0000","last_fullsized":"2024-07-17T08:35:33.432242+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.423373+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:33.423373+0000","last_peered":"2024-07-17T08:35:33.423373+0000","last_clean":"2024-07-17T08:35:33.423373+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:33.423373+0000","last_undegraded":"2024-07-17T08:35:33.423373+0000","last_fullsized":"2024-07-17T08:35:33.423373+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.433338+0000","last_change":"2024-07-17T08:35:33.433338+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.433338+0000","last_undegraded":"2024-07-17T08:35:33.433338+0000","last_fullsized":"2024-07-17T08:35:33.433338+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.451507+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:33.451507+0000","last_peered":"2024-07-17T08:35:33.451507+0000","last_clean":"2024-07-17T08:35:33.451507+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:33.451507+0000","last_undegraded":"2024-07-17T08:35:33.451507+0000","last_fullsized":"2024-07-17T08:35:33.451507+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:34.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.442878+0000","last_change":"2024-07-17T08:35:33.442878+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.442878+0000","last_undegraded":"2024-07-17T08:35:33.442878+0000","last_fullsized":"2024-07-17T08:35:33.442878+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448535+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:33.448535+0000","last_peered":"2024-07-17T08:35:33.448535+0000","last_clean":"2024-07-17T08:35:33.448535+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:33.448535+0000","last_undegraded":"2024-07-17T08:35:33.448535+0000","last_fullsized":"2024-07-17T08:35:33.448535+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.427411+0000","last_change":"2024-07-17T08:35:33.427411+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.427411+0000","last_undegraded":"2024-07-17T08:35:33.427411+0000","last_fullsized":"2024-07-17T08:35:33.427411+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.448060+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:33.448060+0000","last_peered":"2024-07-17T08:35:33.448060+0000","last_clean":"2024-07-17T08:35:33.448060+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:33.448060+0000","last_undegraded":"2024-07-17T08:35:33.448060+0000","last_fullsized":"2024-07-17T08:35:33.448060+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:11.863693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":225,"state":"creating+peering","last_fresh":"2024-07-17T08:35:33.439822+0000","last_change":"2024-07-17T08:35:33.439822+0000","last_active":"2024-07-17T08:35:33.409871+0000","last_peered":"2024-07-17T08:35:33.409871+0000","last_clean":"2024-07-17T08:35:33.409871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:35:33.439822+0000","last_undegraded":"2024-07-17T08:35:33.439822+0000","last_fullsized":"2024-07-17T08:35:33.439822+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":16,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-07-17T08:35:33.449109+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:33.449109+0000","last_peered":"2024-07-17T08:35:33.449109+0000","last_clean":"2024-07-17T08:35:33.449109+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:33.449109+0000","last_undegraded":"2024-07-17T08:35:33.449109+0000","last_fullsized":"2024-07-17T08:35:33.449109+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:22.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543344,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65044,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306796,"statfs":{"total":96636764160,"available":96570159104,"internally_reserved":0,"allocated":3416064,"data_stored":2120307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510639,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29788,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342052,"statfs":{"total":96636764160,"available":96606261248,"internally_reserved":0,"allocated":2834432,"data_stored":1538555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510640,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47816,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324024,"statfs":{"total":96636764160,"available":96587800576,"internally_reserved":0,"allocated":2813952,"data_stored":1527067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47812,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324028,"statfs":{"total":96636764160,"available":96587804672,"internally_reserved":0,"allocated":2809856,"data_stored":1527593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510639,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51980,"kb_used_data":2752,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319860,"statfs":{"total":96636764160,"available":96583536640,"internally_reserved":0,"allocated":2818048,"data_stored":1527137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543343,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2809856,"data_stored":1527101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510640,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47808,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324032,"statfs":{"total":96636764160,"available":96587808768,"internally_reserved":0,"allocated":2805760,"data_stored":1527011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510639,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56724,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315116,"statfs":{"total":96636764160,"available":96578678784,"internally_reserved":0,"allocated":3416064,"data_stored":2125573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:39.483 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:39.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:35:39.632 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:35:39.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.723+0000 7fd6f6400640 1 -- 172.21.15.44:0/2190721145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f0154650 msgr2=0x7fd6f00cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.723+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2190721145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f0154650 0x7fd6f00cf340 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0005fd0 tx=0x7fd6e002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 -- 172.21.15.44:0/2190721145 shutdown_connections 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2190721145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f006efd0 0x7fd6f015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2190721145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f0154650 0x7fd6f00cf340 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 -- 172.21.15.44:0/2190721145 >> 172.21.15.44:0/2190721145 conn(0x7fd6f00af940 msgr2=0x7fd6f00afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 -- 172.21.15.44:0/2190721145 shutdown_connections 2024-07-17T08:35:39.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.724+0000 7fd6f6400640 1 -- 172.21.15.44:0/2190721145 wait complete. 2024-07-17T08:35:39.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.725+0000 7fd6f6400640 1 Processor -- start 2024-07-17T08:35:39.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 -- start start 2024-07-17T08:35:39.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6f0154650 0x7fd6f0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 0x7fd6f01f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00cd860 con 0x7fd6f006efd0 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00b5890 con 0x7fd6f0154650 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.726+0000 7fd6f6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f00cd9e0 con 0x7fd6f015b3c0 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39736/0 (socket says 172.21.15.44:39736) 2024-07-17T08:35:39.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 -- 172.21.15.44:0/2257314302 learned_addr learned my addr 172.21.15.44:0/2257314302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:39.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6efe00640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 0x7fd6f01f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6eea00640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6f0154650 0x7fd6f0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 -- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 msgr2=0x7fd6f01f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 0x7fd6f01f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 -- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6f0154650 msgr2=0x7fd6f0155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6f0154650 0x7fd6f0155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6ef400640 1 -- 172.21.15.44:0/2257314302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f00aa460 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.727+0000 7fd6efe00640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 0x7fd6f01f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6ef400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009850 tx=0x7fd6e00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e003f070 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e0003ce0 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035030 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f0156490 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.728+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f0156a50 con 0x7fd6f006efd0 2024-07-17T08:35:39.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.730+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f015ae80 con 0x7fd6f006efd0 2024-07-17T08:35:39.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.731+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fd6e0042070 con 0x7fd6f006efd0 2024-07-17T08:35:39.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.731+0000 7fd6e7400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 0x7fd6d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:39.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.732+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fd6e00b9820 con 0x7fd6f006efd0 2024-07-17T08:35:39.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.732+0000 7fd6eea00640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 0x7fd6d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:39.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.733+0000 7fd6eea00640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 0x7fd6d0079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0153750 tx=0x7fd6dc005f90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:39.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.736+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd6e00823a0 con 0x7fd6f006efd0 2024-07-17T08:35:39.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.943+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6f00cc250 con 0x7fd6d0076cd0 2024-07-17T08:35:39.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.966+0000 7fd6e7400640 1 -- 172.21.15.44:0/2257314302 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491482 (secure 0 0 0) 0x7fd6f00cc250 con 0x7fd6d0076cd0 2024-07-17T08:35:39.970 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:39.977 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 msgr2=0x7fd6d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 0x7fd6d0079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0153750 tx=0x7fd6dc005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 msgr2=0x7fd6f014e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009850 tx=0x7fd6e00035f0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 shutdown_connections 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6f015b3c0 0x7fd6f01f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fd6d0076cd0 0x7fd6d0079190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6f0154650 0x7fd6f0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 --2- 172.21.15.44:0/2257314302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6f006efd0 0x7fd6f014e250 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 >> 172.21.15.44:0/2257314302 conn(0x7fd6f00af940 msgr2=0x7fd6f00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 shutdown_connections 2024-07-17T08:35:39.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:39.977+0000 7fd6f6400640 1 -- 172.21.15.44:0/2257314302 wait complete. 2024-07-17T08:35:40.000 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-07-17T08:35:39.766335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596434,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":451,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2591,"ondisk_log_size":2591,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399316,"kb_used_data":23496,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754575404,"statfs":{"total":773094113280,"available":772685213696,"internally_reserved":0,"allocated":24059904,"data_stored":13607772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.346364"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554981+0000","last_change":"2024-07-17T08:35:34.486327+0000","last_active":"2024-07-17T08:35:35.554981+0000","last_peered":"2024-07-17T08:35:35.554981+0000","last_clean":"2024-07-17T08:35:35.554981+0000","last_became_active":"2024-07-17T08:35:34.486069+0000","last_became_peered":"2024-07-17T08:35:34.486069+0000","last_unstale":"2024-07-17T08:35:35.554981+0000","last_undegraded":"2024-07-17T08:35:35.554981+0000","last_fullsized":"2024-07-17T08:35:35.554981+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:20.240831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3386,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457819+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:35.457819+0000","last_peered":"2024-07-17T08:35:35.457819+0000","last_clean":"2024-07-17T08:35:35.457819+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:35.457819+0000","last_undegraded":"2024-07-17T08:35:35.457819+0000","last_fullsized":"2024-07-17T08:35:35.457819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:44.992431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553498+0000","last_change":"2024-07-17T08:35:33.455926+0000","last_active":"2024-07-17T08:35:35.553498+0000","last_peered":"2024-07-17T08:35:35.553498+0000","last_clean":"2024-07-17T08:35:35.553498+0000","last_became_active":"2024-07-17T08:35:33.455626+0000","last_became_peered":"2024-07-17T08:35:33.455626+0000","last_unstale":"2024-07-17T08:35:35.553498+0000","last_undegraded":"2024-07-17T08:35:35.553498+0000","last_fullsized":"2024-07-17T08:35:35.553498+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:18.891188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554140+0000","last_change":"2024-07-17T08:35:34.499379+0000","last_active":"2024-07-17T08:35:35.554140+0000","last_peered":"2024-07-17T08:35:35.554140+0000","last_clean":"2024-07-17T08:35:35.554140+0000","last_became_active":"2024-07-17T08:35:34.498469+0000","last_became_peered":"2024-07-17T08:35:34.498469+0000","last_unstale":"2024-07-17T08:35:35.554140+0000","last_undegraded":"2024-07-17T08:35:35.554140+0000","last_fullsized":"2024-07-17T08:35:35.554140+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:18.359384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449166+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:35.449166+0000","last_peered":"2024-07-17T08:35:35.449166+0000","last_clean":"2024-07-17T08:35:35.449166+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:35.449166+0000","last_undegraded":"2024-07-17T08:35:35.449166+0000","last_fullsized":"2024-07-17T08:35:35.449166+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.487031+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:35.487031+0000","last_peered":"2024-07-17T08:35:35.487031+0000","last_clean":"2024-07-17T08:35:35.487031+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:35.487031+0000","last_undegraded":"2024-07-17T08:35:35.487031+0000","last_fullsized":"2024-07-17T08:35:35.487031+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"227'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486853+0000","last_change":"2024-07-17T08:35:33.451371+0000","last_active":"2024-07-17T08:35:35.486853+0000","last_peered":"2024-07-17T08:35:35.486853+0000","last_clean":"2024-07-17T08:35:35.486853+0000","last_became_active":"2024-07-17T08:35:33.451107+0000","last_became_peered":"2024-07-17T08:35:33.451107+0000","last_unstale":"2024-07-17T08:35:35.486853+0000","last_undegraded":"2024-07-17T08:35:35.486853+0000","last_fullsized":"2024-07-17T08:35:35.486853+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:15.595003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553500+0000","last_change":"2024-07-17T08:35:34.480720+0000","last_active":"2024-07-17T08:35:35.553500+0000","last_peered":"2024-07-17T08:35:35.553500+0000","last_clean":"2024-07-17T08:35:35.553500+0000","last_became_active":"2024-07-17T08:35:34.480430+0000","last_became_peered":"2024-07-17T08:35:34.480430+0000","last_unstale":"2024-07-17T08:35:35.553500+0000","last_undegraded":"2024-07-17T08:35:35.553500+0000","last_fullsized":"2024-07-17T08:35:35.553500+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:08.457362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456016+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:35.456016+0000","last_peered":"2024-07-17T08:35:35.456016+0000","last_clean":"2024-07-17T08:35:35.456016+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:35.456016+0000","last_undegraded":"2024-07-17T08:35:35.456016+0000","last_fullsized":"2024-07-17T08:35:35.456016+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555195+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:35.555195+0000","last_peered":"2024-07-17T08:35:35.555195+0000","last_clean":"2024-07-17T08:35:35.555195+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:35.555195+0000","last_undegraded":"2024-07-17T08:35:35.555195+0000","last_fullsized":"2024-07-17T08:35:35.555195+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:08.117588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460518+0000","last_change":"2024-07-17T08:35:33.459624+0000","last_active":"2024-07-17T08:35:35.460518+0000","last_peered":"2024-07-17T08:35:35.460518+0000","last_clean":"2024-07-17T08:35:35.460518+0000","last_became_active":"2024-07-17T08:35:33.459359+0000","last_became_peered":"2024-07-17T08:35:33.459359+0000","last_unstale":"2024-07-17T08:35:35.460518+0000","last_undegraded":"2024-07-17T08:35:35.460518+0000","last_fullsized":"2024-07-17T08:35:35.460518+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:23.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451151+0000","last_change":"2024-07-17T08:35:34.474086+0000","last_active":"2024-07-17T08:35:35.451151+0000","last_peered":"2024-07-17T08:35:35.451151+0000","last_clean":"2024-07-17T08:35:35.451151+0000","last_became_active":"2024-07-17T08:35:34.473771+0000","last_became_peered":"2024-07-17T08:35:34.473771+0000","last_unstale":"2024-07-17T08:35:35.451151+0000","last_undegraded":"2024-07-17T08:35:35.451151+0000","last_fullsized":"2024-07-17T08:35:35.451151+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:12.038298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553735+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:35.553735+0000","last_peered":"2024-07-17T08:35:35.553735+0000","last_clean":"2024-07-17T08:35:35.553735+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:35.553735+0000","last_undegraded":"2024-07-17T08:35:35.553735+0000","last_fullsized":"2024-07-17T08:35:35.553735+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554875+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:35.554875+0000","last_peered":"2024-07-17T08:35:35.554875+0000","last_clean":"2024-07-17T08:35:35.554875+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:35.554875+0000","last_undegraded":"2024-07-17T08:35:35.554875+0000","last_fullsized":"2024-07-17T08:35:35.554875+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:20.349285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461693+0000","last_change":"2024-07-17T08:35:33.457849+0000","last_active":"2024-07-17T08:35:35.461693+0000","last_peered":"2024-07-17T08:35:35.461693+0000","last_clean":"2024-07-17T08:35:35.461693+0000","last_became_active":"2024-07-17T08:35:33.453123+0000","last_became_peered":"2024-07-17T08:35:33.453123+0000","last_unstale":"2024-07-17T08:35:35.461693+0000","last_undegraded":"2024-07-17T08:35:35.461693+0000","last_fullsized":"2024-07-17T08:35:35.461693+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:56:02.265477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457387+0000","last_change":"2024-07-17T08:35:34.485825+0000","last_active":"2024-07-17T08:35:35.457387+0000","last_peered":"2024-07-17T08:35:35.457387+0000","last_clean":"2024-07-17T08:35:35.457387+0000","last_became_active":"2024-07-17T08:35:34.485520+0000","last_became_peered":"2024-07-17T08:35:34.485520+0000","last_unstale":"2024-07-17T08:35:35.457387+0000","last_undegraded":"2024-07-17T08:35:35.457387+0000","last_fullsized":"2024-07-17T08:35:35.457387+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:39.903925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460664+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:35.460664+0000","last_peered":"2024-07-17T08:35:35.460664+0000","last_clean":"2024-07-17T08:35:35.460664+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:35.460664+0000","last_undegraded":"2024-07-17T08:35:35.460664+0000","last_fullsized":"2024-07-17T08:35:35.460664+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462215+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:35.462215+0000","last_peered":"2024-07-17T08:35:35.462215+0000","last_clean":"2024-07-17T08:35:35.462215+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:35.462215+0000","last_undegraded":"2024-07-17T08:35:35.462215+0000","last_fullsized":"2024-07-17T08:35:35.462215+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486627+0000","last_change":"2024-07-17T08:35:33.452063+0000","last_active":"2024-07-17T08:35:35.486627+0000","last_peered":"2024-07-17T08:35:35.486627+0000","last_clean":"2024-07-17T08:35:35.486627+0000","last_became_active":"2024-07-17T08:35:33.450965+0000","last_became_peered":"2024-07-17T08:35:33.450965+0000","last_unstale":"2024-07-17T08:35:35.486627+0000","last_undegraded":"2024-07-17T08:35:35.486627+0000","last_fullsized":"2024-07-17T08:35:35.486627+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:01.482943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552456+0000","last_change":"2024-07-17T08:35:34.485471+0000","last_active":"2024-07-17T08:35:35.552456+0000","last_peered":"2024-07-17T08:35:35.552456+0000","last_clean":"2024-07-17T08:35:35.552456+0000","last_became_active":"2024-07-17T08:35:34.485180+0000","last_became_peered":"2024-07-17T08:35:34.485180+0000","last_unstale":"2024-07-17T08:35:35.552456+0000","last_undegraded":"2024-07-17T08:35:35.552456+0000","last_fullsized":"2024-07-17T08:35:35.552456+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:52.825712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458117+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:35.458117+0000","last_peered":"2024-07-17T08:35:35.458117+0000","last_clean":"2024-07-17T08:35:35.458117+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:35.458117+0000","last_undegraded":"2024-07-17T08:35:35.458117+0000","last_fullsized":"2024-07-17T08:35:35.458117+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457075+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:35.457075+0000","last_peered":"2024-07-17T08:35:35.457075+0000","last_clean":"2024-07-17T08:35:35.457075+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:35.457075+0000","last_undegraded":"2024-07-17T08:35:35.457075+0000","last_fullsized":"2024-07-17T08:35:35.457075+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:32.339990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491696+0000","last_change":"2024-07-17T08:35:33.443001+0000","last_active":"2024-07-17T08:35:35.491696+0000","last_peered":"2024-07-17T08:35:35.491696+0000","last_clean":"2024-07-17T08:35:35.491696+0000","last_became_active":"2024-07-17T08:35:33.442630+0000","last_became_peered":"2024-07-17T08:35:33.442630+0000","last_unstale":"2024-07-17T08:35:35.491696+0000","last_undegraded":"2024-07-17T08:35:35.491696+0000","last_fullsized":"2024-07-17T08:35:35.491696+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:44.333702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552490+0000","last_change":"2024-07-17T08:35:34.484841+0000","last_active":"2024-07-17T08:35:35.552490+0000","last_peered":"2024-07-17T08:35:35.552490+0000","last_clean":"2024-07-17T08:35:35.552490+0000","last_became_active":"2024-07-17T08:35:34.484580+0000","last_became_peered":"2024-07-17T08:35:34.484580+0000","last_unstale":"2024-07-17T08:35:35.552490+0000","last_undegraded":"2024-07-17T08:35:35.552490+0000","last_fullsized":"2024-07-17T08:35:35.552490+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:20:14.336249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493351+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:35.493351+0000","last_peered":"2024-07-17T08:35:35.493351+0000","last_clean":"2024-07-17T08:35:35.493351+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:35.493351+0000","last_undegraded":"2024-07-17T08:35:35.493351+0000","last_fullsized":"2024-07-17T08:35:35.493351+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553061+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:35.553061+0000","last_peered":"2024-07-17T08:35:35.553061+0000","last_clean":"2024-07-17T08:35:35.553061+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:35.553061+0000","last_undegraded":"2024-07-17T08:35:35.553061+0000","last_fullsized":"2024-07-17T08:35:35.553061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459525+0000","last_change":"2024-07-17T08:35:33.447597+0000","last_active":"2024-07-17T08:35:35.459525+0000","last_peered":"2024-07-17T08:35:35.459525+0000","last_clean":"2024-07-17T08:35:35.459525+0000","last_became_active":"2024-07-17T08:35:33.446794+0000","last_became_peered":"2024-07-17T08:35:33.446794+0000","last_unstale":"2024-07-17T08:35:35.459525+0000","last_undegraded":"2024-07-17T08:35:35.459525+0000","last_fullsized":"2024-07-17T08:35:35.459525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:26.440472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459908+0000","last_change":"2024-07-17T08:35:34.476099+0000","last_active":"2024-07-17T08:35:35.459908+0000","last_peered":"2024-07-17T08:35:35.459908+0000","last_clean":"2024-07-17T08:35:35.459908+0000","last_became_active":"2024-07-17T08:35:34.475823+0000","last_became_peered":"2024-07-17T08:35:34.475823+0000","last_unstale":"2024-07-17T08:35:35.459908+0000","last_undegraded":"2024-07-17T08:35:35.459908+0000","last_fullsized":"2024-07-17T08:35:35.459908+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:00.486944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460588+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:35.460588+0000","last_peered":"2024-07-17T08:35:35.460588+0000","last_clean":"2024-07-17T08:35:35.460588+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:35.460588+0000","last_undegraded":"2024-07-17T08:35:35.460588+0000","last_fullsized":"2024-07-17T08:35:35.460588+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486047+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:35.486047+0000","last_peered":"2024-07-17T08:35:35.486047+0000","last_clean":"2024-07-17T08:35:35.486047+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:35.486047+0000","last_undegraded":"2024-07-17T08:35:35.486047+0000","last_fullsized":"2024-07-17T08:35:35.486047+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455982+0000","last_change":"2024-07-17T08:35:33.461285+0000","last_active":"2024-07-17T08:35:35.455982+0000","last_peered":"2024-07-17T08:35:35.455982+0000","last_clean":"2024-07-17T08:35:35.455982+0000","last_became_active":"2024-07-17T08:35:33.460353+0000","last_became_peered":"2024-07-17T08:35:33.460353+0000","last_unstale":"2024-07-17T08:35:35.455982+0000","last_undegraded":"2024-07-17T08:35:35.455982+0000","last_fullsized":"2024-07-17T08:35:35.455982+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:46.221656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449383+0000","last_change":"2024-07-17T08:35:34.459938+0000","last_active":"2024-07-17T08:35:35.449383+0000","last_peered":"2024-07-17T08:35:35.449383+0000","last_clean":"2024-07-17T08:35:35.449383+0000","last_became_active":"2024-07-17T08:35:34.459039+0000","last_became_peered":"2024-07-17T08:35:34.459039+0000","last_unstale":"2024-07-17T08:35:35.449383+0000","last_undegraded":"2024-07-17T08:35:35.449383+0000","last_fullsized":"2024-07-17T08:35:35.449383+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:05.469616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555507+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:35.555507+0000","last_peered":"2024-07-17T08:35:35.555507+0000","last_clean":"2024-07-17T08:35:35.555507+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:35.555507+0000","last_undegraded":"2024-07-17T08:35:35.555507+0000","last_fullsized":"2024-07-17T08:35:35.555507+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457232+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:35.457232+0000","last_peered":"2024-07-17T08:35:35.457232+0000","last_clean":"2024-07-17T08:35:35.457232+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:35.457232+0000","last_undegraded":"2024-07-17T08:35:35.457232+0000","last_fullsized":"2024-07-17T08:35:35.457232+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553475+0000","last_change":"2024-07-17T08:35:33.458859+0000","last_active":"2024-07-17T08:35:35.553475+0000","last_peered":"2024-07-17T08:35:35.553475+0000","last_clean":"2024-07-17T08:35:35.553475+0000","last_became_active":"2024-07-17T08:35:33.458540+0000","last_became_peered":"2024-07-17T08:35:33.458540+0000","last_unstale":"2024-07-17T08:35:35.553475+0000","last_undegraded":"2024-07-17T08:35:35.553475+0000","last_fullsized":"2024-07-17T08:35:35.553475+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:00.336730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553819+0000","last_change":"2024-07-17T08:35:34.479421+0000","last_active":"2024-07-17T08:35:35.553819+0000","last_peered":"2024-07-17T08:35:35.553819+0000","last_clean":"2024-07-17T08:35:35.553819+0000","last_became_active":"2024-07-17T08:35:34.477979+0000","last_became_peered":"2024-07-17T08:35:34.477979+0000","last_unstale":"2024-07-17T08:35:35.553819+0000","last_undegraded":"2024-07-17T08:35:35.553819+0000","last_fullsized":"2024-07-17T08:35:35.553819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.053432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456089+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:35.456089+0000","last_peered":"2024-07-17T08:35:35.456089+0000","last_clean":"2024-07-17T08:35:35.456089+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:35.456089+0000","last_undegraded":"2024-07-17T08:35:35.456089+0000","last_fullsized":"2024-07-17T08:35:35.456089+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461819+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:35.461819+0000","last_peered":"2024-07-17T08:35:35.461819+0000","last_clean":"2024-07-17T08:35:35.461819+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:35.461819+0000","last_undegraded":"2024-07-17T08:35:35.461819+0000","last_fullsized":"2024-07-17T08:35:35.461819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450729+0000","last_change":"2024-07-17T08:35:33.449985+0000","last_active":"2024-07-17T08:35:35.450729+0000","last_peered":"2024-07-17T08:35:35.450729+0000","last_clean":"2024-07-17T08:35:35.450729+0000","last_became_active":"2024-07-17T08:35:33.449677+0000","last_became_peered":"2024-07-17T08:35:33.449677+0000","last_unstale":"2024-07-17T08:35:35.450729+0000","last_undegraded":"2024-07-17T08:35:35.450729+0000","last_fullsized":"2024-07-17T08:35:35.450729+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:59.034427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552631+0000","last_change":"2024-07-17T08:35:34.488611+0000","last_active":"2024-07-17T08:35:35.552631+0000","last_peered":"2024-07-17T08:35:35.552631+0000","last_clean":"2024-07-17T08:35:35.552631+0000","last_became_active":"2024-07-17T08:35:34.486963+0000","last_became_peered":"2024-07-17T08:35:34.486963+0000","last_unstale":"2024-07-17T08:35:35.552631+0000","last_undegraded":"2024-07-17T08:35:35.552631+0000","last_fullsized":"2024-07-17T08:35:35.552631+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:00.632819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493160+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:35.493160+0000","last_peered":"2024-07-17T08:35:35.493160+0000","last_clean":"2024-07-17T08:35:35.493160+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:35.493160+0000","last_undegraded":"2024-07-17T08:35:35.493160+0000","last_fullsized":"2024-07-17T08:35:35.493160+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554092+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:35.554092+0000","last_peered":"2024-07-17T08:35:35.554092+0000","last_clean":"2024-07-17T08:35:35.554092+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:35.554092+0000","last_undegraded":"2024-07-17T08:35:35.554092+0000","last_fullsized":"2024-07-17T08:35:35.554092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555232+0000","last_change":"2024-07-17T08:35:33.455119+0000","last_active":"2024-07-17T08:35:35.555232+0000","last_peered":"2024-07-17T08:35:35.555232+0000","last_clean":"2024-07-17T08:35:35.555232+0000","last_became_active":"2024-07-17T08:35:33.454773+0000","last_became_peered":"2024-07-17T08:35:33.454773+0000","last_unstale":"2024-07-17T08:35:35.555232+0000","last_undegraded":"2024-07-17T08:35:35.555232+0000","last_fullsized":"2024-07-17T08:35:35.555232+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:38.944624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461250+0000","last_change":"2024-07-17T08:35:34.474579+0000","last_active":"2024-07-17T08:35:35.461250+0000","last_peered":"2024-07-17T08:35:35.461250+0000","last_clean":"2024-07-17T08:35:35.461250+0000","last_became_active":"2024-07-17T08:35:34.474286+0000","last_became_peered":"2024-07-17T08:35:34.474286+0000","last_unstale":"2024-07-17T08:35:35.461250+0000","last_undegraded":"2024-07-17T08:35:35.461250+0000","last_fullsized":"2024-07-17T08:35:35.461250+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:04.679552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555839+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:35.555839+0000","last_peered":"2024-07-17T08:35:35.555839+0000","last_clean":"2024-07-17T08:35:35.555839+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:35.555839+0000","last_undegraded":"2024-07-17T08:35:35.555839+0000","last_fullsized":"2024-07-17T08:35:35.555839+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554654+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:35.554654+0000","last_peered":"2024-07-17T08:35:35.554654+0000","last_clean":"2024-07-17T08:35:35.554654+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:35.554654+0000","last_undegraded":"2024-07-17T08:35:35.554654+0000","last_fullsized":"2024-07-17T08:35:35.554654+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:36.700604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461450+0000","last_change":"2024-07-17T08:35:33.457888+0000","last_active":"2024-07-17T08:35:35.461450+0000","last_peered":"2024-07-17T08:35:35.461450+0000","last_clean":"2024-07-17T08:35:35.461450+0000","last_became_active":"2024-07-17T08:35:33.457510+0000","last_became_peered":"2024-07-17T08:35:33.457510+0000","last_unstale":"2024-07-17T08:35:35.461450+0000","last_undegraded":"2024-07-17T08:35:35.461450+0000","last_fullsized":"2024-07-17T08:35:35.461450+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:51.342532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555200+0000","last_change":"2024-07-17T08:35:34.498472+0000","last_active":"2024-07-17T08:35:35.555200+0000","last_peered":"2024-07-17T08:35:35.555200+0000","last_clean":"2024-07-17T08:35:35.555200+0000","last_became_active":"2024-07-17T08:35:34.492593+0000","last_became_peered":"2024-07-17T08:35:34.492593+0000","last_unstale":"2024-07-17T08:35:35.555200+0000","last_undegraded":"2024-07-17T08:35:35.555200+0000","last_fullsized":"2024-07-17T08:35:35.555200+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:28.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450203+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:35.450203+0000","last_peered":"2024-07-17T08:35:35.450203+0000","last_clean":"2024-07-17T08:35:35.450203+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:35.450203+0000","last_undegraded":"2024-07-17T08:35:35.450203+0000","last_fullsized":"2024-07-17T08:35:35.450203+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554679+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:35.554679+0000","last_peered":"2024-07-17T08:35:35.554679+0000","last_clean":"2024-07-17T08:35:35.554679+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:35.554679+0000","last_undegraded":"2024-07-17T08:35:35.554679+0000","last_fullsized":"2024-07-17T08:35:35.554679+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:53.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459946+0000","last_change":"2024-07-17T08:35:33.456857+0000","last_active":"2024-07-17T08:35:35.459946+0000","last_peered":"2024-07-17T08:35:35.459946+0000","last_clean":"2024-07-17T08:35:35.459946+0000","last_became_active":"2024-07-17T08:35:33.456290+0000","last_became_peered":"2024-07-17T08:35:33.456290+0000","last_unstale":"2024-07-17T08:35:35.459946+0000","last_undegraded":"2024-07-17T08:35:35.459946+0000","last_fullsized":"2024-07-17T08:35:35.459946+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:12.755522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553191+0000","last_change":"2024-07-17T08:35:34.491188+0000","last_active":"2024-07-17T08:35:35.553191+0000","last_peered":"2024-07-17T08:35:35.553191+0000","last_clean":"2024-07-17T08:35:35.553191+0000","last_became_active":"2024-07-17T08:35:34.489934+0000","last_became_peered":"2024-07-17T08:35:34.489934+0000","last_unstale":"2024-07-17T08:35:35.553191+0000","last_undegraded":"2024-07-17T08:35:35.553191+0000","last_fullsized":"2024-07-17T08:35:35.553191+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:04.970179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450322+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:35.450322+0000","last_peered":"2024-07-17T08:35:35.450322+0000","last_clean":"2024-07-17T08:35:35.450322+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:35.450322+0000","last_undegraded":"2024-07-17T08:35:35.450322+0000","last_fullsized":"2024-07-17T08:35:35.450322+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460400+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:35.460400+0000","last_peered":"2024-07-17T08:35:35.460400+0000","last_clean":"2024-07-17T08:35:35.460400+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:35.460400+0000","last_undegraded":"2024-07-17T08:35:35.460400+0000","last_fullsized":"2024-07-17T08:35:35.460400+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486062+0000","last_change":"2024-07-17T08:35:33.454020+0000","last_active":"2024-07-17T08:35:35.486062+0000","last_peered":"2024-07-17T08:35:35.486062+0000","last_clean":"2024-07-17T08:35:35.486062+0000","last_became_active":"2024-07-17T08:35:33.453743+0000","last_became_peered":"2024-07-17T08:35:33.453743+0000","last_unstale":"2024-07-17T08:35:35.486062+0000","last_undegraded":"2024-07-17T08:35:35.486062+0000","last_fullsized":"2024-07-17T08:35:35.486062+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:59.652207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455403+0000","last_change":"2024-07-17T08:35:34.486630+0000","last_active":"2024-07-17T08:35:35.455403+0000","last_peered":"2024-07-17T08:35:35.455403+0000","last_clean":"2024-07-17T08:35:35.455403+0000","last_became_active":"2024-07-17T08:35:34.485558+0000","last_became_peered":"2024-07-17T08:35:34.485558+0000","last_unstale":"2024-07-17T08:35:35.455403+0000","last_undegraded":"2024-07-17T08:35:35.455403+0000","last_fullsized":"2024-07-17T08:35:35.455403+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:20.604380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554451+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:35.554451+0000","last_peered":"2024-07-17T08:35:35.554451+0000","last_clean":"2024-07-17T08:35:35.554451+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:35.554451+0000","last_undegraded":"2024-07-17T08:35:35.554451+0000","last_fullsized":"2024-07-17T08:35:35.554451+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458964+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:35.458964+0000","last_peered":"2024-07-17T08:35:35.458964+0000","last_clean":"2024-07-17T08:35:35.458964+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:35.458964+0000","last_undegraded":"2024-07-17T08:35:35.458964+0000","last_fullsized":"2024-07-17T08:35:35.458964+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:07.288084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459274+0000","last_change":"2024-07-17T08:35:33.451492+0000","last_active":"2024-07-17T08:35:35.459274+0000","last_peered":"2024-07-17T08:35:35.459274+0000","last_clean":"2024-07-17T08:35:35.459274+0000","last_became_active":"2024-07-17T08:35:33.451016+0000","last_became_peered":"2024-07-17T08:35:33.451016+0000","last_unstale":"2024-07-17T08:35:35.459274+0000","last_undegraded":"2024-07-17T08:35:35.459274+0000","last_fullsized":"2024-07-17T08:35:35.459274+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:32.267947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553479+0000","last_change":"2024-07-17T08:35:34.497474+0000","last_active":"2024-07-17T08:35:35.553479+0000","last_peered":"2024-07-17T08:35:35.553479+0000","last_clean":"2024-07-17T08:35:35.553479+0000","last_became_active":"2024-07-17T08:35:34.486892+0000","last_became_peered":"2024-07-17T08:35:34.486892+0000","last_unstale":"2024-07-17T08:35:35.553479+0000","last_undegraded":"2024-07-17T08:35:35.553479+0000","last_fullsized":"2024-07-17T08:35:35.553479+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:35.295643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458654+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:35.458654+0000","last_peered":"2024-07-17T08:35:35.458654+0000","last_clean":"2024-07-17T08:35:35.458654+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:35.458654+0000","last_undegraded":"2024-07-17T08:35:35.458654+0000","last_fullsized":"2024-07-17T08:35:35.458654+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455763+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:35.455763+0000","last_peered":"2024-07-17T08:35:35.455763+0000","last_clean":"2024-07-17T08:35:35.455763+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:35.455763+0000","last_undegraded":"2024-07-17T08:35:35.455763+0000","last_fullsized":"2024-07-17T08:35:35.455763+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:44.865077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.448996+0000","last_change":"2024-07-17T08:35:33.440197+0000","last_active":"2024-07-17T08:35:35.448996+0000","last_peered":"2024-07-17T08:35:35.448996+0000","last_clean":"2024-07-17T08:35:35.448996+0000","last_became_active":"2024-07-17T08:35:33.439893+0000","last_became_peered":"2024-07-17T08:35:33.439893+0000","last_unstale":"2024-07-17T08:35:35.448996+0000","last_undegraded":"2024-07-17T08:35:35.448996+0000","last_fullsized":"2024-07-17T08:35:35.448996+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:23.178685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552047+0000","last_change":"2024-07-17T08:35:34.486149+0000","last_active":"2024-07-17T08:35:35.552047+0000","last_peered":"2024-07-17T08:35:35.552047+0000","last_clean":"2024-07-17T08:35:35.552047+0000","last_became_active":"2024-07-17T08:35:34.485165+0000","last_became_peered":"2024-07-17T08:35:34.485165+0000","last_unstale":"2024-07-17T08:35:35.552047+0000","last_undegraded":"2024-07-17T08:35:35.552047+0000","last_fullsized":"2024-07-17T08:35:35.552047+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:33.375893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459284+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:35.459284+0000","last_peered":"2024-07-17T08:35:35.459284+0000","last_clean":"2024-07-17T08:35:35.459284+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:35.459284+0000","last_undegraded":"2024-07-17T08:35:35.459284+0000","last_fullsized":"2024-07-17T08:35:35.459284+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456973+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:35.456973+0000","last_peered":"2024-07-17T08:35:35.456973+0000","last_clean":"2024-07-17T08:35:35.456973+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:35.456973+0000","last_undegraded":"2024-07-17T08:35:35.456973+0000","last_fullsized":"2024-07-17T08:35:35.456973+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485924+0000","last_change":"2024-07-17T08:35:33.451532+0000","last_active":"2024-07-17T08:35:35.485924+0000","last_peered":"2024-07-17T08:35:35.485924+0000","last_clean":"2024-07-17T08:35:35.485924+0000","last_became_active":"2024-07-17T08:35:33.449530+0000","last_became_peered":"2024-07-17T08:35:33.449530+0000","last_unstale":"2024-07-17T08:35:35.485924+0000","last_undegraded":"2024-07-17T08:35:35.485924+0000","last_fullsized":"2024-07-17T08:35:35.485924+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:53.409692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486540+0000","last_change":"2024-07-17T08:35:34.484134+0000","last_active":"2024-07-17T08:35:35.486540+0000","last_peered":"2024-07-17T08:35:35.486540+0000","last_clean":"2024-07-17T08:35:35.486540+0000","last_became_active":"2024-07-17T08:35:34.480964+0000","last_became_peered":"2024-07-17T08:35:34.480964+0000","last_unstale":"2024-07-17T08:35:35.486540+0000","last_undegraded":"2024-07-17T08:35:35.486540+0000","last_fullsized":"2024-07-17T08:35:35.486540+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:22.103527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449153+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:35.449153+0000","last_peered":"2024-07-17T08:35:35.449153+0000","last_clean":"2024-07-17T08:35:35.449153+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:35.449153+0000","last_undegraded":"2024-07-17T08:35:35.449153+0000","last_fullsized":"2024-07-17T08:35:35.449153+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555620+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:35.555620+0000","last_peered":"2024-07-17T08:35:35.555620+0000","last_clean":"2024-07-17T08:35:35.555620+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:35.555620+0000","last_undegraded":"2024-07-17T08:35:35.555620+0000","last_fullsized":"2024-07-17T08:35:35.555620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:38.094519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492867+0000","last_change":"2024-07-17T08:35:33.456460+0000","last_active":"2024-07-17T08:35:35.492867+0000","last_peered":"2024-07-17T08:35:35.492867+0000","last_clean":"2024-07-17T08:35:35.492867+0000","last_became_active":"2024-07-17T08:35:33.456193+0000","last_became_peered":"2024-07-17T08:35:33.456193+0000","last_unstale":"2024-07-17T08:35:35.492867+0000","last_undegraded":"2024-07-17T08:35:35.492867+0000","last_fullsized":"2024-07-17T08:35:35.492867+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:00.094046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486268+0000","last_change":"2024-07-17T08:35:34.485589+0000","last_active":"2024-07-17T08:35:35.486268+0000","last_peered":"2024-07-17T08:35:35.486268+0000","last_clean":"2024-07-17T08:35:35.486268+0000","last_became_active":"2024-07-17T08:35:34.485287+0000","last_became_peered":"2024-07-17T08:35:34.485287+0000","last_unstale":"2024-07-17T08:35:35.486268+0000","last_undegraded":"2024-07-17T08:35:35.486268+0000","last_fullsized":"2024-07-17T08:35:35.486268+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:13.147791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554416+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:35.554416+0000","last_peered":"2024-07-17T08:35:35.554416+0000","last_clean":"2024-07-17T08:35:35.554416+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:35.554416+0000","last_undegraded":"2024-07-17T08:35:35.554416+0000","last_fullsized":"2024-07-17T08:35:35.554416+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553997+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:35.553997+0000","last_peered":"2024-07-17T08:35:35.553997+0000","last_clean":"2024-07-17T08:35:35.553997+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:35.553997+0000","last_undegraded":"2024-07-17T08:35:35.553997+0000","last_fullsized":"2024-07-17T08:35:35.553997+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555546+0000","last_change":"2024-07-17T08:35:33.460434+0000","last_active":"2024-07-17T08:35:35.555546+0000","last_peered":"2024-07-17T08:35:35.555546+0000","last_clean":"2024-07-17T08:35:35.555546+0000","last_became_active":"2024-07-17T08:35:33.460174+0000","last_became_peered":"2024-07-17T08:35:33.460174+0000","last_unstale":"2024-07-17T08:35:35.555546+0000","last_undegraded":"2024-07-17T08:35:35.555546+0000","last_fullsized":"2024-07-17T08:35:35.555546+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:33.539981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492894+0000","last_change":"2024-07-17T08:35:34.478029+0000","last_active":"2024-07-17T08:35:35.492894+0000","last_peered":"2024-07-17T08:35:35.492894+0000","last_clean":"2024-07-17T08:35:35.492894+0000","last_became_active":"2024-07-17T08:35:34.474129+0000","last_became_peered":"2024-07-17T08:35:34.474129+0000","last_unstale":"2024-07-17T08:35:35.492894+0000","last_undegraded":"2024-07-17T08:35:35.492894+0000","last_fullsized":"2024-07-17T08:35:35.492894+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:46.401539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486426+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:35.486426+0000","last_peered":"2024-07-17T08:35:35.486426+0000","last_clean":"2024-07-17T08:35:35.486426+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:35.486426+0000","last_undegraded":"2024-07-17T08:35:35.486426+0000","last_fullsized":"2024-07-17T08:35:35.486426+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486816+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:35.486816+0000","last_peered":"2024-07-17T08:35:35.486816+0000","last_clean":"2024-07-17T08:35:35.486816+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:35.486816+0000","last_undegraded":"2024-07-17T08:35:35.486816+0000","last_fullsized":"2024-07-17T08:35:35.486816+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"226'6","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458001+0000","last_change":"2024-07-17T08:35:33.458710+0000","last_active":"2024-07-17T08:35:35.458001+0000","last_peered":"2024-07-17T08:35:35.458001+0000","last_clean":"2024-07-17T08:35:35.458001+0000","last_became_active":"2024-07-17T08:35:33.458420+0000","last_became_peered":"2024-07-17T08:35:33.458420+0000","last_unstale":"2024-07-17T08:35:35.458001+0000","last_undegraded":"2024-07-17T08:35:35.458001+0000","last_fullsized":"2024-07-17T08:35:35.458001+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:01.698884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555436+0000","last_change":"2024-07-17T08:35:34.483586+0000","last_active":"2024-07-17T08:35:35.555436+0000","last_peered":"2024-07-17T08:35:35.555436+0000","last_clean":"2024-07-17T08:35:35.555436+0000","last_became_active":"2024-07-17T08:35:34.483162+0000","last_became_peered":"2024-07-17T08:35:34.483162+0000","last_unstale":"2024-07-17T08:35:35.555436+0000","last_undegraded":"2024-07-17T08:35:35.555436+0000","last_fullsized":"2024-07-17T08:35:35.555436+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:20.206854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553417+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:35.553417+0000","last_peered":"2024-07-17T08:35:35.553417+0000","last_clean":"2024-07-17T08:35:35.553417+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:35.553417+0000","last_undegraded":"2024-07-17T08:35:35.553417+0000","last_fullsized":"2024-07-17T08:35:35.553417+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554601+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:35.554601+0000","last_peered":"2024-07-17T08:35:35.554601+0000","last_clean":"2024-07-17T08:35:35.554601+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:35.554601+0000","last_undegraded":"2024-07-17T08:35:35.554601+0000","last_fullsized":"2024-07-17T08:35:35.554601+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:07:04.780778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"226'3","reported_seq":17,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554549+0000","last_change":"2024-07-17T08:35:33.459431+0000","last_active":"2024-07-17T08:35:35.554549+0000","last_peered":"2024-07-17T08:35:35.554549+0000","last_clean":"2024-07-17T08:35:35.554549+0000","last_became_active":"2024-07-17T08:35:33.459129+0000","last_became_peered":"2024-07-17T08:35:33.459129+0000","last_unstale":"2024-07-17T08:35:35.554549+0000","last_undegraded":"2024-07-17T08:35:35.554549+0000","last_fullsized":"2024-07-17T08:35:35.554549+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:13.823820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455802+0000","last_change":"2024-07-17T08:35:34.493124+0000","last_active":"2024-07-17T08:35:35.455802+0000","last_peered":"2024-07-17T08:35:35.455802+0000","last_clean":"2024-07-17T08:35:35.455802+0000","last_became_active":"2024-07-17T08:35:34.492795+0000","last_became_peered":"2024-07-17T08:35:34.492795+0000","last_unstale":"2024-07-17T08:35:35.455802+0000","last_undegraded":"2024-07-17T08:35:35.455802+0000","last_fullsized":"2024-07-17T08:35:35.455802+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:10.255450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"223'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554137+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:35.554137+0000","last_peered":"2024-07-17T08:35:35.554137+0000","last_clean":"2024-07-17T08:35:35.554137+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:35.554137+0000","last_undegraded":"2024-07-17T08:35:35.554137+0000","last_fullsized":"2024-07-17T08:35:35.554137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450149+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:35.450149+0000","last_peered":"2024-07-17T08:35:35.450149+0000","last_clean":"2024-07-17T08:35:35.450149+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:35.450149+0000","last_undegraded":"2024-07-17T08:35:35.450149+0000","last_fullsized":"2024-07-17T08:35:35.450149+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552761+0000","last_change":"2024-07-17T08:35:33.438513+0000","last_active":"2024-07-17T08:35:35.552761+0000","last_peered":"2024-07-17T08:35:35.552761+0000","last_clean":"2024-07-17T08:35:35.552761+0000","last_became_active":"2024-07-17T08:35:33.438129+0000","last_became_peered":"2024-07-17T08:35:33.438129+0000","last_unstale":"2024-07-17T08:35:35.552761+0000","last_undegraded":"2024-07-17T08:35:35.552761+0000","last_fullsized":"2024-07-17T08:35:35.552761+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:20:04.565470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551555+0000","last_change":"2024-07-17T08:35:34.483491+0000","last_active":"2024-07-17T08:35:35.551555+0000","last_peered":"2024-07-17T08:35:35.551555+0000","last_clean":"2024-07-17T08:35:35.551555+0000","last_became_active":"2024-07-17T08:35:34.483129+0000","last_became_peered":"2024-07-17T08:35:34.483129+0000","last_unstale":"2024-07-17T08:35:35.551555+0000","last_undegraded":"2024-07-17T08:35:35.551555+0000","last_fullsized":"2024-07-17T08:35:35.551555+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:39.959193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":25,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552904+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:35.552904+0000","last_peered":"2024-07-17T08:35:35.552904+0000","last_clean":"2024-07-17T08:35:35.552904+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:35.552904+0000","last_undegraded":"2024-07-17T08:35:35.552904+0000","last_fullsized":"2024-07-17T08:35:35.552904+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553318+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:35.553318+0000","last_peered":"2024-07-17T08:35:35.553318+0000","last_clean":"2024-07-17T08:35:35.553318+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:35.553318+0000","last_undegraded":"2024-07-17T08:35:35.553318+0000","last_fullsized":"2024-07-17T08:35:35.553318+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458967+0000","last_change":"2024-07-17T08:35:33.451275+0000","last_active":"2024-07-17T08:35:35.458967+0000","last_peered":"2024-07-17T08:35:35.458967+0000","last_clean":"2024-07-17T08:35:35.458967+0000","last_became_active":"2024-07-17T08:35:33.450592+0000","last_became_peered":"2024-07-17T08:35:33.450592+0000","last_unstale":"2024-07-17T08:35:35.458967+0000","last_undegraded":"2024-07-17T08:35:35.458967+0000","last_fullsized":"2024-07-17T08:35:35.458967+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:47.040622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486251+0000","last_change":"2024-07-17T08:35:34.496715+0000","last_active":"2024-07-17T08:35:35.486251+0000","last_peered":"2024-07-17T08:35:35.486251+0000","last_clean":"2024-07-17T08:35:35.486251+0000","last_became_active":"2024-07-17T08:35:34.496413+0000","last_became_peered":"2024-07-17T08:35:34.496413+0000","last_unstale":"2024-07-17T08:35:35.486251+0000","last_undegraded":"2024-07-17T08:35:35.486251+0000","last_fullsized":"2024-07-17T08:35:35.486251+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:04.804268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555332+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:35.555332+0000","last_peered":"2024-07-17T08:35:35.555332+0000","last_clean":"2024-07-17T08:35:35.555332+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:35.555332+0000","last_undegraded":"2024-07-17T08:35:35.555332+0000","last_fullsized":"2024-07-17T08:35:35.555332+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485967+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:35.485967+0000","last_peered":"2024-07-17T08:35:35.485967+0000","last_clean":"2024-07-17T08:35:35.485967+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:35.485967+0000","last_undegraded":"2024-07-17T08:35:35.485967+0000","last_fullsized":"2024-07-17T08:35:35.485967+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"226'8","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455662+0000","last_change":"2024-07-17T08:35:33.460898+0000","last_active":"2024-07-17T08:35:35.455662+0000","last_peered":"2024-07-17T08:35:35.455662+0000","last_clean":"2024-07-17T08:35:35.455662+0000","last_became_active":"2024-07-17T08:35:33.460626+0000","last_became_peered":"2024-07-17T08:35:33.460626+0000","last_unstale":"2024-07-17T08:35:35.455662+0000","last_undegraded":"2024-07-17T08:35:35.455662+0000","last_fullsized":"2024-07-17T08:35:35.455662+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:10.216965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555230+0000","last_change":"2024-07-17T08:35:34.497427+0000","last_active":"2024-07-17T08:35:35.555230+0000","last_peered":"2024-07-17T08:35:35.555230+0000","last_clean":"2024-07-17T08:35:35.555230+0000","last_became_active":"2024-07-17T08:35:34.496490+0000","last_became_peered":"2024-07-17T08:35:34.496490+0000","last_unstale":"2024-07-17T08:35:35.555230+0000","last_undegraded":"2024-07-17T08:35:35.555230+0000","last_fullsized":"2024-07-17T08:35:35.555230+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:27.078421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451056+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:35.451056+0000","last_peered":"2024-07-17T08:35:35.451056+0000","last_clean":"2024-07-17T08:35:35.451056+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:35.451056+0000","last_undegraded":"2024-07-17T08:35:35.451056+0000","last_fullsized":"2024-07-17T08:35:35.451056+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491687+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:35.491687+0000","last_peered":"2024-07-17T08:35:35.491687+0000","last_clean":"2024-07-17T08:35:35.491687+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:35.491687+0000","last_undegraded":"2024-07-17T08:35:35.491687+0000","last_fullsized":"2024-07-17T08:35:35.491687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:21.845715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454949+0000","last_change":"2024-07-17T08:35:33.452238+0000","last_active":"2024-07-17T08:35:35.454949+0000","last_peered":"2024-07-17T08:35:35.454949+0000","last_clean":"2024-07-17T08:35:35.454949+0000","last_became_active":"2024-07-17T08:35:33.450790+0000","last_became_peered":"2024-07-17T08:35:33.450790+0000","last_unstale":"2024-07-17T08:35:35.454949+0000","last_undegraded":"2024-07-17T08:35:35.454949+0000","last_fullsized":"2024-07-17T08:35:35.454949+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:24.729759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553337+0000","last_change":"2024-07-17T08:35:34.483916+0000","last_active":"2024-07-17T08:35:35.553337+0000","last_peered":"2024-07-17T08:35:35.553337+0000","last_clean":"2024-07-17T08:35:35.553337+0000","last_became_active":"2024-07-17T08:35:34.483101+0000","last_became_peered":"2024-07-17T08:35:34.483101+0000","last_unstale":"2024-07-17T08:35:35.553337+0000","last_undegraded":"2024-07-17T08:35:35.553337+0000","last_fullsized":"2024-07-17T08:35:35.553337+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:46.506398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460048+0000","last_change":"2024-07-17T08:35:33.456972+0000","last_active":"2024-07-17T08:35:35.460048+0000","last_peered":"2024-07-17T08:35:35.460048+0000","last_clean":"2024-07-17T08:35:35.460048+0000","last_became_active":"2024-07-17T08:35:33.452779+0000","last_became_peered":"2024-07-17T08:35:33.452779+0000","last_unstale":"2024-07-17T08:35:35.460048+0000","last_undegraded":"2024-07-17T08:35:35.460048+0000","last_fullsized":"2024-07-17T08:35:35.460048+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:29:43.620854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485766+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:35.485766+0000","last_peered":"2024-07-17T08:35:35.485766+0000","last_clean":"2024-07-17T08:35:35.485766+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:35.485766+0000","last_undegraded":"2024-07-17T08:35:35.485766+0000","last_fullsized":"2024-07-17T08:35:35.485766+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555255+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:35.555255+0000","last_peered":"2024-07-17T08:35:35.555255+0000","last_clean":"2024-07-17T08:35:35.555255+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:35.555255+0000","last_undegraded":"2024-07-17T08:35:35.555255+0000","last_fullsized":"2024-07-17T08:35:35.555255+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554158+0000","last_change":"2024-07-17T08:35:34.491080+0000","last_active":"2024-07-17T08:35:35.554158+0000","last_peered":"2024-07-17T08:35:35.554158+0000","last_clean":"2024-07-17T08:35:35.554158+0000","last_became_active":"2024-07-17T08:35:34.490804+0000","last_became_peered":"2024-07-17T08:35:34.490804+0000","last_unstale":"2024-07-17T08:35:35.554158+0000","last_undegraded":"2024-07-17T08:35:35.554158+0000","last_fullsized":"2024-07-17T08:35:35.554158+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:50.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"226'4","reported_seq":18,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450844+0000","last_change":"2024-07-17T08:35:33.446100+0000","last_active":"2024-07-17T08:35:35.450844+0000","last_peered":"2024-07-17T08:35:35.450844+0000","last_clean":"2024-07-17T08:35:35.450844+0000","last_became_active":"2024-07-17T08:35:33.445751+0000","last_became_peered":"2024-07-17T08:35:33.445751+0000","last_unstale":"2024-07-17T08:35:35.450844+0000","last_undegraded":"2024-07-17T08:35:35.450844+0000","last_fullsized":"2024-07-17T08:35:35.450844+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:18.271351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553518+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:35.553518+0000","last_peered":"2024-07-17T08:35:35.553518+0000","last_clean":"2024-07-17T08:35:35.553518+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:35.553518+0000","last_undegraded":"2024-07-17T08:35:35.553518+0000","last_fullsized":"2024-07-17T08:35:35.553518+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493201+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:35.493201+0000","last_peered":"2024-07-17T08:35:35.493201+0000","last_clean":"2024-07-17T08:35:35.493201+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:35.493201+0000","last_undegraded":"2024-07-17T08:35:35.493201+0000","last_fullsized":"2024-07-17T08:35:35.493201+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:19.204797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461020+0000","last_change":"2024-07-17T08:35:34.495578+0000","last_active":"2024-07-17T08:35:35.461020+0000","last_peered":"2024-07-17T08:35:35.461020+0000","last_clean":"2024-07-17T08:35:35.461020+0000","last_became_active":"2024-07-17T08:35:34.495255+0000","last_became_peered":"2024-07-17T08:35:34.495255+0000","last_unstale":"2024-07-17T08:35:35.461020+0000","last_undegraded":"2024-07-17T08:35:35.461020+0000","last_fullsized":"2024-07-17T08:35:35.461020+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:38.958975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554180+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:35.554180+0000","last_peered":"2024-07-17T08:35:35.554180+0000","last_clean":"2024-07-17T08:35:35.554180+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:35.554180+0000","last_undegraded":"2024-07-17T08:35:35.554180+0000","last_fullsized":"2024-07-17T08:35:35.554180+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492633+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:35.492633+0000","last_peered":"2024-07-17T08:35:35.492633+0000","last_clean":"2024-07-17T08:35:35.492633+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:35.492633+0000","last_undegraded":"2024-07-17T08:35:35.492633+0000","last_fullsized":"2024-07-17T08:35:35.492633+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:41.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457525+0000","last_change":"2024-07-17T08:35:33.458703+0000","last_active":"2024-07-17T08:35:35.457525+0000","last_peered":"2024-07-17T08:35:35.457525+0000","last_clean":"2024-07-17T08:35:35.457525+0000","last_became_active":"2024-07-17T08:35:33.458365+0000","last_became_peered":"2024-07-17T08:35:33.458365+0000","last_unstale":"2024-07-17T08:35:35.457525+0000","last_undegraded":"2024-07-17T08:35:35.457525+0000","last_fullsized":"2024-07-17T08:35:35.457525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:25.173461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455182+0000","last_change":"2024-07-17T08:35:34.472507+0000","last_active":"2024-07-17T08:35:35.455182+0000","last_peered":"2024-07-17T08:35:35.455182+0000","last_clean":"2024-07-17T08:35:35.455182+0000","last_became_active":"2024-07-17T08:35:34.472042+0000","last_became_peered":"2024-07-17T08:35:34.472042+0000","last_unstale":"2024-07-17T08:35:35.455182+0000","last_undegraded":"2024-07-17T08:35:35.455182+0000","last_fullsized":"2024-07-17T08:35:35.455182+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:57.135370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554169+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:35.554169+0000","last_peered":"2024-07-17T08:35:35.554169+0000","last_clean":"2024-07-17T08:35:35.554169+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:35.554169+0000","last_undegraded":"2024-07-17T08:35:35.554169+0000","last_fullsized":"2024-07-17T08:35:35.554169+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450061+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:35.450061+0000","last_peered":"2024-07-17T08:35:35.450061+0000","last_clean":"2024-07-17T08:35:35.450061+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:35.450061+0000","last_undegraded":"2024-07-17T08:35:35.450061+0000","last_fullsized":"2024-07-17T08:35:35.450061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553207+0000","last_change":"2024-07-17T08:35:33.462435+0000","last_active":"2024-07-17T08:35:35.553207+0000","last_peered":"2024-07-17T08:35:35.553207+0000","last_clean":"2024-07-17T08:35:35.553207+0000","last_became_active":"2024-07-17T08:35:33.462140+0000","last_became_peered":"2024-07-17T08:35:33.462140+0000","last_unstale":"2024-07-17T08:35:35.553207+0000","last_undegraded":"2024-07-17T08:35:35.553207+0000","last_fullsized":"2024-07-17T08:35:35.553207+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:49.267551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551332+0000","last_change":"2024-07-17T08:35:34.503686+0000","last_active":"2024-07-17T08:35:35.551332+0000","last_peered":"2024-07-17T08:35:35.551332+0000","last_clean":"2024-07-17T08:35:35.551332+0000","last_became_active":"2024-07-17T08:35:34.502946+0000","last_became_peered":"2024-07-17T08:35:34.502946+0000","last_unstale":"2024-07-17T08:35:35.551332+0000","last_undegraded":"2024-07-17T08:35:35.551332+0000","last_fullsized":"2024-07-17T08:35:35.551332+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:34.944240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553744+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:35.553744+0000","last_peered":"2024-07-17T08:35:35.553744+0000","last_clean":"2024-07-17T08:35:35.553744+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:35.553744+0000","last_undegraded":"2024-07-17T08:35:35.553744+0000","last_fullsized":"2024-07-17T08:35:35.553744+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486237+0000","last_change":"2024-07-17T08:35:33.464947+0000","last_active":"2024-07-17T08:35:35.486237+0000","last_peered":"2024-07-17T08:35:35.486237+0000","last_clean":"2024-07-17T08:35:35.486237+0000","last_became_active":"2024-07-17T08:35:33.464637+0000","last_became_peered":"2024-07-17T08:35:33.464637+0000","last_unstale":"2024-07-17T08:35:35.486237+0000","last_undegraded":"2024-07-17T08:35:35.486237+0000","last_fullsized":"2024-07-17T08:35:35.486237+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:33.340251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449931+0000","last_change":"2024-07-17T08:35:34.485339+0000","last_active":"2024-07-17T08:35:35.449931+0000","last_peered":"2024-07-17T08:35:35.449931+0000","last_clean":"2024-07-17T08:35:35.449931+0000","last_became_active":"2024-07-17T08:35:34.485071+0000","last_became_peered":"2024-07-17T08:35:34.485071+0000","last_unstale":"2024-07-17T08:35:35.449931+0000","last_undegraded":"2024-07-17T08:35:35.449931+0000","last_fullsized":"2024-07-17T08:35:35.449931+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:58.882312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458370+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:35.458370+0000","last_peered":"2024-07-17T08:35:35.458370+0000","last_clean":"2024-07-17T08:35:35.458370+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:35.458370+0000","last_undegraded":"2024-07-17T08:35:35.458370+0000","last_fullsized":"2024-07-17T08:35:35.458370+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455489+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:35.455489+0000","last_peered":"2024-07-17T08:35:35.455489+0000","last_clean":"2024-07-17T08:35:35.455489+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:35.455489+0000","last_undegraded":"2024-07-17T08:35:35.455489+0000","last_fullsized":"2024-07-17T08:35:35.455489+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:18.513656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551486+0000","last_change":"2024-07-17T08:35:33.450591+0000","last_active":"2024-07-17T08:35:35.551486+0000","last_peered":"2024-07-17T08:35:35.551486+0000","last_clean":"2024-07-17T08:35:35.551486+0000","last_became_active":"2024-07-17T08:35:33.450291+0000","last_became_peered":"2024-07-17T08:35:33.450291+0000","last_unstale":"2024-07-17T08:35:35.551486+0000","last_undegraded":"2024-07-17T08:35:35.551486+0000","last_fullsized":"2024-07-17T08:35:35.551486+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:11.346934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486484+0000","last_change":"2024-07-17T08:35:34.495666+0000","last_active":"2024-07-17T08:35:35.486484+0000","last_peered":"2024-07-17T08:35:35.486484+0000","last_clean":"2024-07-17T08:35:35.486484+0000","last_became_active":"2024-07-17T08:35:34.493428+0000","last_became_peered":"2024-07-17T08:35:34.493428+0000","last_unstale":"2024-07-17T08:35:35.486484+0000","last_undegraded":"2024-07-17T08:35:35.486484+0000","last_fullsized":"2024-07-17T08:35:35.486484+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:14.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":26,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493209+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:35.493209+0000","last_peered":"2024-07-17T08:35:35.493209+0000","last_clean":"2024-07-17T08:35:35.493209+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:35.493209+0000","last_undegraded":"2024-07-17T08:35:35.493209+0000","last_fullsized":"2024-07-17T08:35:35.493209+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552776+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:35.552776+0000","last_peered":"2024-07-17T08:35:35.552776+0000","last_clean":"2024-07-17T08:35:35.552776+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:35.552776+0000","last_undegraded":"2024-07-17T08:35:35.552776+0000","last_fullsized":"2024-07-17T08:35:35.552776+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:42.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491432+0000","last_change":"2024-07-17T08:35:33.454797+0000","last_active":"2024-07-17T08:35:35.491432+0000","last_peered":"2024-07-17T08:35:35.491432+0000","last_clean":"2024-07-17T08:35:35.491432+0000","last_became_active":"2024-07-17T08:35:33.454327+0000","last_became_peered":"2024-07-17T08:35:33.454327+0000","last_unstale":"2024-07-17T08:35:35.491432+0000","last_undegraded":"2024-07-17T08:35:35.491432+0000","last_fullsized":"2024-07-17T08:35:35.491432+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:51.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455439+0000","last_change":"2024-07-17T08:35:34.490533+0000","last_active":"2024-07-17T08:35:35.455439+0000","last_peered":"2024-07-17T08:35:35.455439+0000","last_clean":"2024-07-17T08:35:35.455439+0000","last_became_active":"2024-07-17T08:35:34.488112+0000","last_became_peered":"2024-07-17T08:35:34.488112+0000","last_unstale":"2024-07-17T08:35:35.455439+0000","last_undegraded":"2024-07-17T08:35:35.455439+0000","last_fullsized":"2024-07-17T08:35:35.455439+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:15:00.666776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553336+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:35.553336+0000","last_peered":"2024-07-17T08:35:35.553336+0000","last_clean":"2024-07-17T08:35:35.553336+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:35.553336+0000","last_undegraded":"2024-07-17T08:35:35.553336+0000","last_fullsized":"2024-07-17T08:35:35.553336+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:09:55.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552022+0000","last_change":"2024-07-17T08:35:34.484796+0000","last_active":"2024-07-17T08:35:35.552022+0000","last_peered":"2024-07-17T08:35:35.552022+0000","last_clean":"2024-07-17T08:35:35.552022+0000","last_became_active":"2024-07-17T08:35:34.484514+0000","last_became_peered":"2024-07-17T08:35:34.484514+0000","last_unstale":"2024-07-17T08:35:35.552022+0000","last_undegraded":"2024-07-17T08:35:35.552022+0000","last_fullsized":"2024-07-17T08:35:35.552022+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:09.046835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553249+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:35.553249+0000","last_peered":"2024-07-17T08:35:35.553249+0000","last_clean":"2024-07-17T08:35:35.553249+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:35.553249+0000","last_undegraded":"2024-07-17T08:35:35.553249+0000","last_fullsized":"2024-07-17T08:35:35.553249+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:01.628007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455254+0000","last_change":"2024-07-17T08:35:34.488202+0000","last_active":"2024-07-17T08:35:35.455254+0000","last_peered":"2024-07-17T08:35:35.455254+0000","last_clean":"2024-07-17T08:35:35.455254+0000","last_became_active":"2024-07-17T08:35:34.487794+0000","last_became_peered":"2024-07-17T08:35:34.487794+0000","last_unstale":"2024-07-17T08:35:35.455254+0000","last_undegraded":"2024-07-17T08:35:35.455254+0000","last_fullsized":"2024-07-17T08:35:35.455254+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:34.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492044+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:35.492044+0000","last_peered":"2024-07-17T08:35:35.492044+0000","last_clean":"2024-07-17T08:35:35.492044+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:35.492044+0000","last_undegraded":"2024-07-17T08:35:35.492044+0000","last_fullsized":"2024-07-17T08:35:35.492044+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:25.913456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449659+0000","last_change":"2024-07-17T08:35:34.489886+0000","last_active":"2024-07-17T08:35:35.449659+0000","last_peered":"2024-07-17T08:35:35.449659+0000","last_clean":"2024-07-17T08:35:35.449659+0000","last_became_active":"2024-07-17T08:35:34.489612+0000","last_became_peered":"2024-07-17T08:35:34.489612+0000","last_unstale":"2024-07-17T08:35:35.449659+0000","last_undegraded":"2024-07-17T08:35:35.449659+0000","last_fullsized":"2024-07-17T08:35:35.449659+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:10.418879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552117+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:35.552117+0000","last_peered":"2024-07-17T08:35:35.552117+0000","last_clean":"2024-07-17T08:35:35.552117+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:35.552117+0000","last_undegraded":"2024-07-17T08:35:35.552117+0000","last_fullsized":"2024-07-17T08:35:35.552117+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554848+0000","last_change":"2024-07-17T08:35:34.497991+0000","last_active":"2024-07-17T08:35:35.554848+0000","last_peered":"2024-07-17T08:35:35.554848+0000","last_clean":"2024-07-17T08:35:35.554848+0000","last_became_active":"2024-07-17T08:35:34.491521+0000","last_became_peered":"2024-07-17T08:35:34.491521+0000","last_unstale":"2024-07-17T08:35:35.554848+0000","last_undegraded":"2024-07-17T08:35:35.554848+0000","last_fullsized":"2024-07-17T08:35:35.554848+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:52.030910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450264+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:35.450264+0000","last_peered":"2024-07-17T08:35:35.450264+0000","last_clean":"2024-07-17T08:35:35.450264+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:35.450264+0000","last_undegraded":"2024-07-17T08:35:35.450264+0000","last_fullsized":"2024-07-17T08:35:35.450264+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555288+0000","last_change":"2024-07-17T08:35:34.503875+0000","last_active":"2024-07-17T08:35:35.555288+0000","last_peered":"2024-07-17T08:35:35.555288+0000","last_clean":"2024-07-17T08:35:35.555288+0000","last_became_active":"2024-07-17T08:35:34.503583+0000","last_became_peered":"2024-07-17T08:35:34.503583+0000","last_unstale":"2024-07-17T08:35:35.555288+0000","last_undegraded":"2024-07-17T08:35:35.555288+0000","last_fullsized":"2024-07-17T08:35:35.555288+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:00:33.798525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486757+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:35.486757+0000","last_peered":"2024-07-17T08:35:35.486757+0000","last_clean":"2024-07-17T08:35:35.486757+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:35.486757+0000","last_undegraded":"2024-07-17T08:35:35.486757+0000","last_fullsized":"2024-07-17T08:35:35.486757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461219+0000","last_change":"2024-07-17T08:35:34.471198+0000","last_active":"2024-07-17T08:35:35.461219+0000","last_peered":"2024-07-17T08:35:35.461219+0000","last_clean":"2024-07-17T08:35:35.461219+0000","last_became_active":"2024-07-17T08:35:34.470800+0000","last_became_peered":"2024-07-17T08:35:34.470800+0000","last_unstale":"2024-07-17T08:35:35.461219+0000","last_undegraded":"2024-07-17T08:35:35.461219+0000","last_fullsized":"2024-07-17T08:35:35.461219+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:00.489281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458092+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:35.458092+0000","last_peered":"2024-07-17T08:35:35.458092+0000","last_clean":"2024-07-17T08:35:35.458092+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:35.458092+0000","last_undegraded":"2024-07-17T08:35:35.458092+0000","last_fullsized":"2024-07-17T08:35:35.458092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:37.046641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493189+0000","last_change":"2024-07-17T08:35:34.481733+0000","last_active":"2024-07-17T08:35:35.493189+0000","last_peered":"2024-07-17T08:35:35.493189+0000","last_clean":"2024-07-17T08:35:35.493189+0000","last_became_active":"2024-07-17T08:35:34.481419+0000","last_became_peered":"2024-07-17T08:35:34.481419+0000","last_unstale":"2024-07-17T08:35:35.493189+0000","last_undegraded":"2024-07-17T08:35:35.493189+0000","last_fullsized":"2024-07-17T08:35:35.493189+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:29.214921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555009+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:35.555009+0000","last_peered":"2024-07-17T08:35:35.555009+0000","last_clean":"2024-07-17T08:35:35.555009+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:35.555009+0000","last_undegraded":"2024-07-17T08:35:35.555009+0000","last_fullsized":"2024-07-17T08:35:35.555009+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486172+0000","last_change":"2024-07-17T08:35:34.494972+0000","last_active":"2024-07-17T08:35:35.486172+0000","last_peered":"2024-07-17T08:35:35.486172+0000","last_clean":"2024-07-17T08:35:35.486172+0000","last_became_active":"2024-07-17T08:35:34.493982+0000","last_became_peered":"2024-07-17T08:35:34.493982+0000","last_unstale":"2024-07-17T08:35:35.486172+0000","last_undegraded":"2024-07-17T08:35:35.486172+0000","last_fullsized":"2024-07-17T08:35:35.486172+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:11.025000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449054+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:35.449054+0000","last_peered":"2024-07-17T08:35:35.449054+0000","last_clean":"2024-07-17T08:35:35.449054+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:35.449054+0000","last_undegraded":"2024-07-17T08:35:35.449054+0000","last_fullsized":"2024-07-17T08:35:35.449054+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552737+0000","last_change":"2024-07-17T08:35:34.498453+0000","last_active":"2024-07-17T08:35:35.552737+0000","last_peered":"2024-07-17T08:35:35.552737+0000","last_clean":"2024-07-17T08:35:35.552737+0000","last_became_active":"2024-07-17T08:35:34.498160+0000","last_became_peered":"2024-07-17T08:35:34.498160+0000","last_unstale":"2024-07-17T08:35:35.552737+0000","last_undegraded":"2024-07-17T08:35:35.552737+0000","last_fullsized":"2024-07-17T08:35:35.552737+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:45:36.345027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552918+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:35.552918+0000","last_peered":"2024-07-17T08:35:35.552918+0000","last_clean":"2024-07-17T08:35:35.552918+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:35.552918+0000","last_undegraded":"2024-07-17T08:35:35.552918+0000","last_fullsized":"2024-07-17T08:35:35.552918+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:00.653492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553040+0000","last_change":"2024-07-17T08:35:34.504906+0000","last_active":"2024-07-17T08:35:35.553040+0000","last_peered":"2024-07-17T08:35:35.553040+0000","last_clean":"2024-07-17T08:35:35.553040+0000","last_became_active":"2024-07-17T08:35:34.504288+0000","last_became_peered":"2024-07-17T08:35:34.504288+0000","last_unstale":"2024-07-17T08:35:35.553040+0000","last_undegraded":"2024-07-17T08:35:35.553040+0000","last_fullsized":"2024-07-17T08:35:35.553040+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:19.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459230+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:35.459230+0000","last_peered":"2024-07-17T08:35:35.459230+0000","last_clean":"2024-07-17T08:35:35.459230+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:35.459230+0000","last_undegraded":"2024-07-17T08:35:35.459230+0000","last_fullsized":"2024-07-17T08:35:35.459230+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449377+0000","last_change":"2024-07-17T08:35:34.481895+0000","last_active":"2024-07-17T08:35:35.449377+0000","last_peered":"2024-07-17T08:35:35.449377+0000","last_clean":"2024-07-17T08:35:35.449377+0000","last_became_active":"2024-07-17T08:35:34.481567+0000","last_became_peered":"2024-07-17T08:35:34.481567+0000","last_unstale":"2024-07-17T08:35:35.449377+0000","last_undegraded":"2024-07-17T08:35:35.449377+0000","last_fullsized":"2024-07-17T08:35:35.449377+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:45:18.596493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461653+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:35.461653+0000","last_peered":"2024-07-17T08:35:35.461653+0000","last_clean":"2024-07-17T08:35:35.461653+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:35.461653+0000","last_undegraded":"2024-07-17T08:35:35.461653+0000","last_fullsized":"2024-07-17T08:35:35.461653+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554398+0000","last_change":"2024-07-17T08:35:34.499163+0000","last_active":"2024-07-17T08:35:35.554398+0000","last_peered":"2024-07-17T08:35:35.554398+0000","last_clean":"2024-07-17T08:35:35.554398+0000","last_became_active":"2024-07-17T08:35:34.491959+0000","last_became_peered":"2024-07-17T08:35:34.491959+0000","last_unstale":"2024-07-17T08:35:35.554398+0000","last_undegraded":"2024-07-17T08:35:35.554398+0000","last_fullsized":"2024-07-17T08:35:35.554398+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:22.008659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462086+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:35.462086+0000","last_peered":"2024-07-17T08:35:35.462086+0000","last_clean":"2024-07-17T08:35:35.462086+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:35.462086+0000","last_undegraded":"2024-07-17T08:35:35.462086+0000","last_fullsized":"2024-07-17T08:35:35.462086+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460233+0000","last_change":"2024-07-17T08:35:34.492707+0000","last_active":"2024-07-17T08:35:35.460233+0000","last_peered":"2024-07-17T08:35:35.460233+0000","last_clean":"2024-07-17T08:35:35.460233+0000","last_became_active":"2024-07-17T08:35:34.490372+0000","last_became_peered":"2024-07-17T08:35:34.490372+0000","last_unstale":"2024-07-17T08:35:35.460233+0000","last_undegraded":"2024-07-17T08:35:35.460233+0000","last_fullsized":"2024-07-17T08:35:35.460233+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:39.652406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460378+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:35.460378+0000","last_peered":"2024-07-17T08:35:35.460378+0000","last_clean":"2024-07-17T08:35:35.460378+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:35.460378+0000","last_undegraded":"2024-07-17T08:35:35.460378+0000","last_fullsized":"2024-07-17T08:35:35.460378+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457473+0000","last_change":"2024-07-17T08:35:34.486721+0000","last_active":"2024-07-17T08:35:35.457473+0000","last_peered":"2024-07-17T08:35:35.457473+0000","last_clean":"2024-07-17T08:35:35.457473+0000","last_became_active":"2024-07-17T08:35:34.486259+0000","last_became_peered":"2024-07-17T08:35:34.486259+0000","last_unstale":"2024-07-17T08:35:35.457473+0000","last_undegraded":"2024-07-17T08:35:35.457473+0000","last_fullsized":"2024-07-17T08:35:35.457473+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:27.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553127+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:35.553127+0000","last_peered":"2024-07-17T08:35:35.553127+0000","last_clean":"2024-07-17T08:35:35.553127+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:35.553127+0000","last_undegraded":"2024-07-17T08:35:35.553127+0000","last_fullsized":"2024-07-17T08:35:35.553127+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555034+0000","last_change":"2024-07-17T08:35:34.499538+0000","last_active":"2024-07-17T08:35:35.555034+0000","last_peered":"2024-07-17T08:35:35.555034+0000","last_clean":"2024-07-17T08:35:35.555034+0000","last_became_active":"2024-07-17T08:35:34.499236+0000","last_became_peered":"2024-07-17T08:35:34.499236+0000","last_unstale":"2024-07-17T08:35:35.555034+0000","last_undegraded":"2024-07-17T08:35:35.555034+0000","last_fullsized":"2024-07-17T08:35:35.555034+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:55:40.339503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554755+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:35.554755+0000","last_peered":"2024-07-17T08:35:35.554755+0000","last_clean":"2024-07-17T08:35:35.554755+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:35.554755+0000","last_undegraded":"2024-07-17T08:35:35.554755+0000","last_fullsized":"2024-07-17T08:35:35.554755+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458749+0000","last_change":"2024-07-17T08:35:34.484162+0000","last_active":"2024-07-17T08:35:35.458749+0000","last_peered":"2024-07-17T08:35:35.458749+0000","last_clean":"2024-07-17T08:35:35.458749+0000","last_became_active":"2024-07-17T08:35:34.483488+0000","last_became_peered":"2024-07-17T08:35:34.483488+0000","last_unstale":"2024-07-17T08:35:35.458749+0000","last_undegraded":"2024-07-17T08:35:35.458749+0000","last_fullsized":"2024-07-17T08:35:35.458749+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:42.661226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485758+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:35.485758+0000","last_peered":"2024-07-17T08:35:35.485758+0000","last_clean":"2024-07-17T08:35:35.485758+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:35.485758+0000","last_undegraded":"2024-07-17T08:35:35.485758+0000","last_fullsized":"2024-07-17T08:35:35.485758+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456828+0000","last_change":"2024-07-17T08:35:34.485105+0000","last_active":"2024-07-17T08:35:35.456828+0000","last_peered":"2024-07-17T08:35:35.456828+0000","last_clean":"2024-07-17T08:35:35.456828+0000","last_became_active":"2024-07-17T08:35:34.484821+0000","last_became_peered":"2024-07-17T08:35:34.484821+0000","last_unstale":"2024-07-17T08:35:35.456828+0000","last_undegraded":"2024-07-17T08:35:35.456828+0000","last_fullsized":"2024-07-17T08:35:35.456828+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:16.609572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492853+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:35.492853+0000","last_peered":"2024-07-17T08:35:35.492853+0000","last_clean":"2024-07-17T08:35:35.492853+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:35.492853+0000","last_undegraded":"2024-07-17T08:35:35.492853+0000","last_fullsized":"2024-07-17T08:35:35.492853+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:06.797667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454814+0000","last_change":"2024-07-17T08:35:34.488768+0000","last_active":"2024-07-17T08:35:35.454814+0000","last_peered":"2024-07-17T08:35:35.454814+0000","last_clean":"2024-07-17T08:35:35.454814+0000","last_became_active":"2024-07-17T08:35:34.488485+0000","last_became_peered":"2024-07-17T08:35:34.488485+0000","last_unstale":"2024-07-17T08:35:35.454814+0000","last_undegraded":"2024-07-17T08:35:35.454814+0000","last_fullsized":"2024-07-17T08:35:35.454814+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:02.497512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458986+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:35.458986+0000","last_peered":"2024-07-17T08:35:35.458986+0000","last_clean":"2024-07-17T08:35:35.458986+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:35.458986+0000","last_undegraded":"2024-07-17T08:35:35.458986+0000","last_fullsized":"2024-07-17T08:35:35.458986+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485819+0000","last_change":"2024-07-17T08:35:34.490716+0000","last_active":"2024-07-17T08:35:35.485819+0000","last_peered":"2024-07-17T08:35:35.485819+0000","last_clean":"2024-07-17T08:35:35.485819+0000","last_became_active":"2024-07-17T08:35:34.490402+0000","last_became_peered":"2024-07-17T08:35:34.490402+0000","last_unstale":"2024-07-17T08:35:35.485819+0000","last_undegraded":"2024-07-17T08:35:35.485819+0000","last_fullsized":"2024-07-17T08:35:35.485819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:49.544177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554175+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:35.554175+0000","last_peered":"2024-07-17T08:35:35.554175+0000","last_clean":"2024-07-17T08:35:35.554175+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:35.554175+0000","last_undegraded":"2024-07-17T08:35:35.554175+0000","last_fullsized":"2024-07-17T08:35:35.554175+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:41.247825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552756+0000","last_change":"2024-07-17T08:35:34.498232+0000","last_active":"2024-07-17T08:35:35.552756+0000","last_peered":"2024-07-17T08:35:35.552756+0000","last_clean":"2024-07-17T08:35:35.552756+0000","last_became_active":"2024-07-17T08:35:34.492256+0000","last_became_peered":"2024-07-17T08:35:34.492256+0000","last_unstale":"2024-07-17T08:35:35.552756+0000","last_undegraded":"2024-07-17T08:35:35.552756+0000","last_fullsized":"2024-07-17T08:35:35.552756+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:44.693413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450194+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:35.450194+0000","last_peered":"2024-07-17T08:35:35.450194+0000","last_clean":"2024-07-17T08:35:35.450194+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:35.450194+0000","last_undegraded":"2024-07-17T08:35:35.450194+0000","last_fullsized":"2024-07-17T08:35:35.450194+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450743+0000","last_change":"2024-07-17T08:35:34.484320+0000","last_active":"2024-07-17T08:35:35.450743+0000","last_peered":"2024-07-17T08:35:35.450743+0000","last_clean":"2024-07-17T08:35:35.450743+0000","last_became_active":"2024-07-17T08:35:34.483670+0000","last_became_peered":"2024-07-17T08:35:34.483670+0000","last_unstale":"2024-07-17T08:35:35.450743+0000","last_undegraded":"2024-07-17T08:35:35.450743+0000","last_fullsized":"2024-07-17T08:35:35.450743+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:31:34.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486717+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:35.486717+0000","last_peered":"2024-07-17T08:35:35.486717+0000","last_clean":"2024-07-17T08:35:35.486717+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:35.486717+0000","last_undegraded":"2024-07-17T08:35:35.486717+0000","last_fullsized":"2024-07-17T08:35:35.486717+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460839+0000","last_change":"2024-07-17T08:35:34.489939+0000","last_active":"2024-07-17T08:35:35.460839+0000","last_peered":"2024-07-17T08:35:35.460839+0000","last_clean":"2024-07-17T08:35:35.460839+0000","last_became_active":"2024-07-17T08:35:34.489661+0000","last_became_peered":"2024-07-17T08:35:34.489661+0000","last_unstale":"2024-07-17T08:35:35.460839+0000","last_undegraded":"2024-07-17T08:35:35.460839+0000","last_fullsized":"2024-07-17T08:35:35.460839+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:47.521931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460334+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:35.460334+0000","last_peered":"2024-07-17T08:35:35.460334+0000","last_clean":"2024-07-17T08:35:35.460334+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:35.460334+0000","last_undegraded":"2024-07-17T08:35:35.460334+0000","last_fullsized":"2024-07-17T08:35:35.460334+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493380+0000","last_change":"2024-07-17T08:35:34.490263+0000","last_active":"2024-07-17T08:35:35.493380+0000","last_peered":"2024-07-17T08:35:35.493380+0000","last_clean":"2024-07-17T08:35:35.493380+0000","last_became_active":"2024-07-17T08:35:34.489975+0000","last_became_peered":"2024-07-17T08:35:34.489975+0000","last_unstale":"2024-07-17T08:35:35.493380+0000","last_undegraded":"2024-07-17T08:35:35.493380+0000","last_fullsized":"2024-07-17T08:35:35.493380+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:06.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554243+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:35.554243+0000","last_peered":"2024-07-17T08:35:35.554243+0000","last_clean":"2024-07-17T08:35:35.554243+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:35.554243+0000","last_undegraded":"2024-07-17T08:35:35.554243+0000","last_fullsized":"2024-07-17T08:35:35.554243+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458638+0000","last_change":"2024-07-17T08:35:34.492433+0000","last_active":"2024-07-17T08:35:35.458638+0000","last_peered":"2024-07-17T08:35:35.458638+0000","last_clean":"2024-07-17T08:35:35.458638+0000","last_became_active":"2024-07-17T08:35:34.490894+0000","last_became_peered":"2024-07-17T08:35:34.490894+0000","last_unstale":"2024-07-17T08:35:35.458638+0000","last_undegraded":"2024-07-17T08:35:35.458638+0000","last_fullsized":"2024-07-17T08:35:35.458638+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:47.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456454+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:35.456454+0000","last_peered":"2024-07-17T08:35:35.456454+0000","last_clean":"2024-07-17T08:35:35.456454+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:35.456454+0000","last_undegraded":"2024-07-17T08:35:35.456454+0000","last_fullsized":"2024-07-17T08:35:35.456454+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553404+0000","last_change":"2024-07-17T08:35:34.488358+0000","last_active":"2024-07-17T08:35:35.553404+0000","last_peered":"2024-07-17T08:35:35.553404+0000","last_clean":"2024-07-17T08:35:35.553404+0000","last_became_active":"2024-07-17T08:35:34.488076+0000","last_became_peered":"2024-07-17T08:35:34.488076+0000","last_unstale":"2024-07-17T08:35:35.553404+0000","last_undegraded":"2024-07-17T08:35:35.553404+0000","last_fullsized":"2024-07-17T08:35:35.553404+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449620+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:35.449620+0000","last_peered":"2024-07-17T08:35:35.449620+0000","last_clean":"2024-07-17T08:35:35.449620+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:35.449620+0000","last_undegraded":"2024-07-17T08:35:35.449620+0000","last_fullsized":"2024-07-17T08:35:35.449620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:34.483905+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:34.481531+0000","last_became_peered":"2024-07-17T08:35:34.481531+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:25.813209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553820+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:35.553820+0000","last_peered":"2024-07-17T08:35:35.553820+0000","last_clean":"2024-07-17T08:35:35.553820+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:35.553820+0000","last_undegraded":"2024-07-17T08:35:35.553820+0000","last_fullsized":"2024-07-17T08:35:35.553820+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:52.497030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552031+0000","last_change":"2024-07-17T08:35:34.495772+0000","last_active":"2024-07-17T08:35:35.552031+0000","last_peered":"2024-07-17T08:35:35.552031+0000","last_clean":"2024-07-17T08:35:35.552031+0000","last_became_active":"2024-07-17T08:35:34.495454+0000","last_became_peered":"2024-07-17T08:35:34.495454+0000","last_unstale":"2024-07-17T08:35:35.552031+0000","last_undegraded":"2024-07-17T08:35:35.552031+0000","last_fullsized":"2024-07-17T08:35:35.552031+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:23.394746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450049+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:35.450049+0000","last_peered":"2024-07-17T08:35:35.450049+0000","last_clean":"2024-07-17T08:35:35.450049+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:35.450049+0000","last_undegraded":"2024-07-17T08:35:35.450049+0000","last_fullsized":"2024-07-17T08:35:35.450049+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552363+0000","last_change":"2024-07-17T08:35:34.492325+0000","last_active":"2024-07-17T08:35:35.552363+0000","last_peered":"2024-07-17T08:35:35.552363+0000","last_clean":"2024-07-17T08:35:35.552363+0000","last_became_active":"2024-07-17T08:35:34.491403+0000","last_became_peered":"2024-07-17T08:35:34.491403+0000","last_unstale":"2024-07-17T08:35:35.552363+0000","last_undegraded":"2024-07-17T08:35:35.552363+0000","last_fullsized":"2024-07-17T08:35:35.552363+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:42:09.402312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554043+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:35.554043+0000","last_peered":"2024-07-17T08:35:35.554043+0000","last_clean":"2024-07-17T08:35:35.554043+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:35.554043+0000","last_undegraded":"2024-07-17T08:35:35.554043+0000","last_fullsized":"2024-07-17T08:35:35.554043+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:34.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555266+0000","last_change":"2024-07-17T08:35:34.494219+0000","last_active":"2024-07-17T08:35:35.555266+0000","last_peered":"2024-07-17T08:35:35.555266+0000","last_clean":"2024-07-17T08:35:35.555266+0000","last_became_active":"2024-07-17T08:35:34.493904+0000","last_became_peered":"2024-07-17T08:35:34.493904+0000","last_unstale":"2024-07-17T08:35:35.555266+0000","last_undegraded":"2024-07-17T08:35:35.555266+0000","last_fullsized":"2024-07-17T08:35:35.555266+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:11.039773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486687+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:35.486687+0000","last_peered":"2024-07-17T08:35:35.486687+0000","last_clean":"2024-07-17T08:35:35.486687+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:35.486687+0000","last_undegraded":"2024-07-17T08:35:35.486687+0000","last_fullsized":"2024-07-17T08:35:35.486687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492520+0000","last_change":"2024-07-17T08:35:34.489926+0000","last_active":"2024-07-17T08:35:35.492520+0000","last_peered":"2024-07-17T08:35:35.492520+0000","last_clean":"2024-07-17T08:35:35.492520+0000","last_became_active":"2024-07-17T08:35:34.489578+0000","last_became_peered":"2024-07-17T08:35:34.489578+0000","last_unstale":"2024-07-17T08:35:35.492520+0000","last_undegraded":"2024-07-17T08:35:35.492520+0000","last_fullsized":"2024-07-17T08:35:35.492520+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:52.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555446+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:35.555446+0000","last_peered":"2024-07-17T08:35:35.555446+0000","last_clean":"2024-07-17T08:35:35.555446+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:35.555446+0000","last_undegraded":"2024-07-17T08:35:35.555446+0000","last_fullsized":"2024-07-17T08:35:35.555446+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:11.863693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554183+0000","last_change":"2024-07-17T08:35:34.489451+0000","last_active":"2024-07-17T08:35:35.554183+0000","last_peered":"2024-07-17T08:35:35.554183+0000","last_clean":"2024-07-17T08:35:35.554183+0000","last_became_active":"2024-07-17T08:35:34.489177+0000","last_became_peered":"2024-07-17T08:35:34.489177+0000","last_unstale":"2024-07-17T08:35:35.554183+0000","last_undegraded":"2024-07-17T08:35:35.554183+0000","last_fullsized":"2024-07-17T08:35:35.554183+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:41.289199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458899+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:35.458899+0000","last_peered":"2024-07-17T08:35:35.458899+0000","last_clean":"2024-07-17T08:35:35.458899+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:35.458899+0000","last_undegraded":"2024-07-17T08:35:35.458899+0000","last_fullsized":"2024-07-17T08:35:35.458899+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:22.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543345,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65088,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306752,"statfs":{"total":96636764160,"available":96570114048,"internally_reserved":0,"allocated":3461120,"data_stored":2145831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510640,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29844,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341996,"statfs":{"total":96636764160,"available":96606203904,"internally_reserved":0,"allocated":2883584,"data_stored":1557041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510641,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1553601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543346,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2850816,"data_stored":1552535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510640,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52028,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319812,"statfs":{"total":96636764160,"available":96583487488,"internally_reserved":0,"allocated":2859008,"data_stored":1552011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543344,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2854912,"data_stored":1552569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510641,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2842624,"data_stored":1551863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56756,"kb_used_data":3360,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315084,"statfs":{"total":96636764160,"available":96578646016,"internally_reserved":0,"allocated":3440640,"data_stored":2142321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:40.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 -- 172.21.15.44:0/4187883088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 msgr2=0x7f3fb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4187883088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fb800b9c0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0159ca0 tx=0x7f3fb8005830 comp rx=0 tx=0).stop 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 -- 172.21.15.44:0/4187883088 shutdown_connections 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4187883088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fb800b9c0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 -- 172.21.15.44:0/4187883088 >> 172.21.15.44:0/4187883088 conn(0x7f3fd00b0e80 msgr2=0x7f3fd00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 -- 172.21.15.44:0/4187883088 shutdown_connections 2024-07-17T08:35:40.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.259+0000 7f3fd7000640 1 -- 172.21.15.44:0/4187883088 wait complete. 2024-07-17T08:35:40.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.261+0000 7f3fd7000640 1 Processor -- start 2024-07-17T08:35:40.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.261+0000 7f3fd7000640 1 -- start start 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd014f880 0x7f3fd00ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 0x7f3fd01e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd009b550 con 0x7f3fb80052c0 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd009b3d0 con 0x7f3fd014f880 2024-07-17T08:35:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.262+0000 7f3fd7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00cf350 con 0x7f3fd01506d0 2024-07-17T08:35:40.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39740/0 (socket says 172.21.15.44:39740) 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 -- 172.21.15.44:0/4200707422 learned_addr learned my addr 172.21.15.44:0/4200707422 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5200640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd014f880 0x7f3fd00ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd6600640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 0x7f3fd01e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 -- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 msgr2=0x7f3fd01e6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 0x7f3fd01e6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 -- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd014f880 msgr2=0x7f3fd00ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd014f880 0x7f3fd00ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 -- 172.21.15.44:0/4200707422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd014fd00 con 0x7f3fb80052c0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd6600640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 0x7f3fd01e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.263+0000 7f3fd5c00640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400d640 tx=0x7f3fc400db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.264+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40062e0 con 0x7f3fb80052c0 2024-07-17T08:35:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.264+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fc4015020 con 0x7f3fb80052c0 2024-07-17T08:35:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.264+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd00acce0 con 0x7f3fb80052c0 2024-07-17T08:35:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.264+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4003870 con 0x7f3fb80052c0 2024-07-17T08:35:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.264+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd00ad2a0 con 0x7f3fb80052c0 2024-07-17T08:35:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.265+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd00cf8b0 con 0x7f3fb80052c0 2024-07-17T08:35:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.266+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f3fc4003a10 con 0x7f3fb80052c0 2024-07-17T08:35:40.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.267+0000 7f3fcd600640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 0x7f3fb00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.267+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f3fc4099320 con 0x7f3fb80052c0 2024-07-17T08:35:40.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.268+0000 7f3fd5200640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 0x7f3fb00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.269+0000 7f3fd5200640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 0x7f3fb00791c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00ce790 tx=0x7f3fbc005f90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:40.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.272+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3fc401f020 con 0x7f3fb80052c0 2024-07-17T08:35:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.561+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3fd00af600 con 0x7f3fb80052c0 2024-07-17T08:35:40.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.562+0000 7f3fcd600640 1 -- 172.21.15.44:0/4200707422 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1167 (secure 0 0 0) 0x7f3fd014fd00 con 0x7f3fb80052c0 2024-07-17T08:35:40.564 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:40.564 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":6072,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":227,"num_osds":8,"num_up_osds":8,"osd_up_since":1721199282,"num_in_osds":8,"osd_in_since":1721199273,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":193}],"num_pgs":193,"num_pools":5,"num_objects":46,"data_bytes":596434,"bytes_used":408899584,"bytes_avail":772685213696,"bytes_total":773094113280,"write_bytes_sec":2581,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","id":13,"up":1,"in":1,"max":1,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"c","status":"up:active","gid":8329},{"filesystem_id":13,"rank":0,"name":"a","status":"up:active","gid":10083}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-07-17T08:33:03.196734+0000","services":{}},"progress_events":{}} 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.564+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 msgr2=0x7f3fb00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.564+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 0x7f3fb00791c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00ce790 tx=0x7f3fbc005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 msgr2=0x7f3fd00ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400d640 tx=0x7f3fc400db00 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 shutdown_connections 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd01506d0 0x7f3fd01e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f3fb0076d00 0x7f3fb00791c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd014f880 0x7f3fd00ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 --2- 172.21.15.44:0/4200707422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fb80052c0 0x7f3fd00ae100 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 >> 172.21.15.44:0/4200707422 conn(0x7f3fd00b0e80 msgr2=0x7f3fd006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:40.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 shutdown_connections 2024-07-17T08:35:40.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.565+0000 7f3fd7000640 1 -- 172.21.15.44:0/4200707422 wait complete. 2024-07-17T08:35:40.584 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-07-17T08:35:40.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:40.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.789+0000 7fc672200640 1 -- 172.21.15.44:0/415535743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c15b560 msgr2=0x7fc66c0cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.789+0000 7fc672200640 1 --2- 172.21.15.44:0/415535743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c15b560 0x7fc66c0cf560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005fd0 tx=0x7fc65c02a540 comp rx=0 tx=0).stop 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 -- 172.21.15.44:0/415535743 shutdown_connections 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 --2- 172.21.15.44:0/415535743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0704f0 0x7fc66c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 --2- 172.21.15.44:0/415535743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c15b560 0x7fc66c0cf560 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 -- 172.21.15.44:0/415535743 >> 172.21.15.44:0/415535743 conn(0x7fc66c0b3020 msgr2=0x7fc66c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 -- 172.21.15.44:0/415535743 shutdown_connections 2024-07-17T08:35:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.790+0000 7fc672200640 1 -- 172.21.15.44:0/415535743 wait complete. 2024-07-17T08:35:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.792+0000 7fc672200640 1 Processor -- start 2024-07-17T08:35:40.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.792+0000 7fc672200640 1 -- start start 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c0704f0 0x7fc66c1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 0x7fc66c09ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c0ae9c0 con 0x7fc66c15b560 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c0ae840 con 0x7fc66c0704f0 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc672200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc66c0aeb40 con 0x7fc66c0cfaa0 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc66be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc66be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39754/0 (socket says 172.21.15.44:39754) 2024-07-17T08:35:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.794+0000 7fc66be00640 1 -- 172.21.15.44:0/895531398 learned_addr learned my addr 172.21.15.44:0/895531398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66b400640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c0704f0 0x7fc66c1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc663e00640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 0x7fc66c09ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 -- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 msgr2=0x7fc66c09ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 0x7fc66c09ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 -- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c0704f0 msgr2=0x7fc66c1f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c0704f0 0x7fc66c1f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 -- 172.21.15.44:0/895531398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c09d960 con 0x7fc66c15b560 2024-07-17T08:35:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc663e00640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 0x7fc66c09ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.795+0000 7fc66be00640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc66400efd0 tx=0x7fc66400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.796+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664003b70 con 0x7fc66c15b560 2024-07-17T08:35:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.796+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc664003e00 con 0x7fc66c15b560 2024-07-17T08:35:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.796+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664004280 con 0x7fc66c15b560 2024-07-17T08:35:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.796+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c09dbe0 con 0x7fc66c15b560 2024-07-17T08:35:40.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.796+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c09e1a0 con 0x7fc66c15b560 2024-07-17T08:35:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.798+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c0d0e80 con 0x7fc66c15b560 2024-07-17T08:35:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.798+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7fc664004420 con 0x7fc66c15b560 2024-07-17T08:35:40.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.799+0000 7fc663400640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 0x7fc648079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.800+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fc664099a60 con 0x7fc66c15b560 2024-07-17T08:35:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.800+0000 7fc66b400640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 0x7fc648079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.801+0000 7fc66b400640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 0x7fc648079190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc65c033000 tx=0x7fc65c004730 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:40.804+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc66c09d960 con 0x7fc66c15b560 2024-07-17T08:35:41.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.009+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc66c09f1c0 con 0x7fc648076cd0 2024-07-17T08:35:41.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.033+0000 7fc663400640 1 -- 172.21.15.44:0/895531398 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491482 (secure 0 0 0) 0x7fc66c09f1c0 con 0x7fc648076cd0 2024-07-17T08:35:41.039 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:41.044 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 msgr2=0x7fc648079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 0x7fc648079190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc65c033000 tx=0x7fc65c004730 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 msgr2=0x7fc66c0b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc66400efd0 tx=0x7fc66400c6a0 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 shutdown_connections 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc66c15b560 0x7fc66c0b6760 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc66c0cfaa0 0x7fc66c09ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fc648076cd0 0x7fc648079190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 --2- 172.21.15.44:0/895531398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc66c0704f0 0x7fc66c1f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 >> 172.21.15.44:0/895531398 conn(0x7fc66c0b3020 msgr2=0x7fc66c156a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 shutdown_connections 2024-07-17T08:35:41.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:41.045+0000 7fc672200640 1 -- 172.21.15.44:0/895531398 wait complete. 2024-07-17T08:35:41.068 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-07-17T08:35:39.766335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596434,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":451,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2591,"ondisk_log_size":2591,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399316,"kb_used_data":23496,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754575404,"statfs":{"total":773094113280,"available":772685213696,"internally_reserved":0,"allocated":24059904,"data_stored":13607772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.346364"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554981+0000","last_change":"2024-07-17T08:35:34.486327+0000","last_active":"2024-07-17T08:35:35.554981+0000","last_peered":"2024-07-17T08:35:35.554981+0000","last_clean":"2024-07-17T08:35:35.554981+0000","last_became_active":"2024-07-17T08:35:34.486069+0000","last_became_peered":"2024-07-17T08:35:34.486069+0000","last_unstale":"2024-07-17T08:35:35.554981+0000","last_undegraded":"2024-07-17T08:35:35.554981+0000","last_fullsized":"2024-07-17T08:35:35.554981+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:20.240831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3386,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457819+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:35.457819+0000","last_peered":"2024-07-17T08:35:35.457819+0000","last_clean":"2024-07-17T08:35:35.457819+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:35.457819+0000","last_undegraded":"2024-07-17T08:35:35.457819+0000","last_fullsized":"2024-07-17T08:35:35.457819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:44.992431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553498+0000","last_change":"2024-07-17T08:35:33.455926+0000","last_active":"2024-07-17T08:35:35.553498+0000","last_peered":"2024-07-17T08:35:35.553498+0000","last_clean":"2024-07-17T08:35:35.553498+0000","last_became_active":"2024-07-17T08:35:33.455626+0000","last_became_peered":"2024-07-17T08:35:33.455626+0000","last_unstale":"2024-07-17T08:35:35.553498+0000","last_undegraded":"2024-07-17T08:35:35.553498+0000","last_fullsized":"2024-07-17T08:35:35.553498+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:18.891188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554140+0000","last_change":"2024-07-17T08:35:34.499379+0000","last_active":"2024-07-17T08:35:35.554140+0000","last_peered":"2024-07-17T08:35:35.554140+0000","last_clean":"2024-07-17T08:35:35.554140+0000","last_became_active":"2024-07-17T08:35:34.498469+0000","last_became_peered":"2024-07-17T08:35:34.498469+0000","last_unstale":"2024-07-17T08:35:35.554140+0000","last_undegraded":"2024-07-17T08:35:35.554140+0000","last_fullsized":"2024-07-17T08:35:35.554140+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:18.359384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449166+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:35.449166+0000","last_peered":"2024-07-17T08:35:35.449166+0000","last_clean":"2024-07-17T08:35:35.449166+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:35.449166+0000","last_undegraded":"2024-07-17T08:35:35.449166+0000","last_fullsized":"2024-07-17T08:35:35.449166+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.487031+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:35.487031+0000","last_peered":"2024-07-17T08:35:35.487031+0000","last_clean":"2024-07-17T08:35:35.487031+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:35.487031+0000","last_undegraded":"2024-07-17T08:35:35.487031+0000","last_fullsized":"2024-07-17T08:35:35.487031+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"227'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486853+0000","last_change":"2024-07-17T08:35:33.451371+0000","last_active":"2024-07-17T08:35:35.486853+0000","last_peered":"2024-07-17T08:35:35.486853+0000","last_clean":"2024-07-17T08:35:35.486853+0000","last_became_active":"2024-07-17T08:35:33.451107+0000","last_became_peered":"2024-07-17T08:35:33.451107+0000","last_unstale":"2024-07-17T08:35:35.486853+0000","last_undegraded":"2024-07-17T08:35:35.486853+0000","last_fullsized":"2024-07-17T08:35:35.486853+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:15.595003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553500+0000","last_change":"2024-07-17T08:35:34.480720+0000","last_active":"2024-07-17T08:35:35.553500+0000","last_peered":"2024-07-17T08:35:35.553500+0000","last_clean":"2024-07-17T08:35:35.553500+0000","last_became_active":"2024-07-17T08:35:34.480430+0000","last_became_peered":"2024-07-17T08:35:34.480430+0000","last_unstale":"2024-07-17T08:35:35.553500+0000","last_undegraded":"2024-07-17T08:35:35.553500+0000","last_fullsized":"2024-07-17T08:35:35.553500+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:08.457362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456016+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:35.456016+0000","last_peered":"2024-07-17T08:35:35.456016+0000","last_clean":"2024-07-17T08:35:35.456016+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:35.456016+0000","last_undegraded":"2024-07-17T08:35:35.456016+0000","last_fullsized":"2024-07-17T08:35:35.456016+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555195+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:35.555195+0000","last_peered":"2024-07-17T08:35:35.555195+0000","last_clean":"2024-07-17T08:35:35.555195+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:35.555195+0000","last_undegraded":"2024-07-17T08:35:35.555195+0000","last_fullsized":"2024-07-17T08:35:35.555195+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:08.117588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460518+0000","last_change":"2024-07-17T08:35:33.459624+0000","last_active":"2024-07-17T08:35:35.460518+0000","last_peered":"2024-07-17T08:35:35.460518+0000","last_clean":"2024-07-17T08:35:35.460518+0000","last_became_active":"2024-07-17T08:35:33.459359+0000","last_became_peered":"2024-07-17T08:35:33.459359+0000","last_unstale":"2024-07-17T08:35:35.460518+0000","last_undegraded":"2024-07-17T08:35:35.460518+0000","last_fullsized":"2024-07-17T08:35:35.460518+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:23.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451151+0000","last_change":"2024-07-17T08:35:34.474086+0000","last_active":"2024-07-17T08:35:35.451151+0000","last_peered":"2024-07-17T08:35:35.451151+0000","last_clean":"2024-07-17T08:35:35.451151+0000","last_became_active":"2024-07-17T08:35:34.473771+0000","last_became_peered":"2024-07-17T08:35:34.473771+0000","last_unstale":"2024-07-17T08:35:35.451151+0000","last_undegraded":"2024-07-17T08:35:35.451151+0000","last_fullsized":"2024-07-17T08:35:35.451151+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:12.038298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553735+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:35.553735+0000","last_peered":"2024-07-17T08:35:35.553735+0000","last_clean":"2024-07-17T08:35:35.553735+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:35.553735+0000","last_undegraded":"2024-07-17T08:35:35.553735+0000","last_fullsized":"2024-07-17T08:35:35.553735+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554875+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:35.554875+0000","last_peered":"2024-07-17T08:35:35.554875+0000","last_clean":"2024-07-17T08:35:35.554875+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:35.554875+0000","last_undegraded":"2024-07-17T08:35:35.554875+0000","last_fullsized":"2024-07-17T08:35:35.554875+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:20.349285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461693+0000","last_change":"2024-07-17T08:35:33.457849+0000","last_active":"2024-07-17T08:35:35.461693+0000","last_peered":"2024-07-17T08:35:35.461693+0000","last_clean":"2024-07-17T08:35:35.461693+0000","last_became_active":"2024-07-17T08:35:33.453123+0000","last_became_peered":"2024-07-17T08:35:33.453123+0000","last_unstale":"2024-07-17T08:35:35.461693+0000","last_undegraded":"2024-07-17T08:35:35.461693+0000","last_fullsized":"2024-07-17T08:35:35.461693+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:56:02.265477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457387+0000","last_change":"2024-07-17T08:35:34.485825+0000","last_active":"2024-07-17T08:35:35.457387+0000","last_peered":"2024-07-17T08:35:35.457387+0000","last_clean":"2024-07-17T08:35:35.457387+0000","last_became_active":"2024-07-17T08:35:34.485520+0000","last_became_peered":"2024-07-17T08:35:34.485520+0000","last_unstale":"2024-07-17T08:35:35.457387+0000","last_undegraded":"2024-07-17T08:35:35.457387+0000","last_fullsized":"2024-07-17T08:35:35.457387+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:39.903925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460664+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:35.460664+0000","last_peered":"2024-07-17T08:35:35.460664+0000","last_clean":"2024-07-17T08:35:35.460664+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:35.460664+0000","last_undegraded":"2024-07-17T08:35:35.460664+0000","last_fullsized":"2024-07-17T08:35:35.460664+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462215+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:35.462215+0000","last_peered":"2024-07-17T08:35:35.462215+0000","last_clean":"2024-07-17T08:35:35.462215+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:35.462215+0000","last_undegraded":"2024-07-17T08:35:35.462215+0000","last_fullsized":"2024-07-17T08:35:35.462215+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486627+0000","last_change":"2024-07-17T08:35:33.452063+0000","last_active":"2024-07-17T08:35:35.486627+0000","last_peered":"2024-07-17T08:35:35.486627+0000","last_clean":"2024-07-17T08:35:35.486627+0000","last_became_active":"2024-07-17T08:35:33.450965+0000","last_became_peered":"2024-07-17T08:35:33.450965+0000","last_unstale":"2024-07-17T08:35:35.486627+0000","last_undegraded":"2024-07-17T08:35:35.486627+0000","last_fullsized":"2024-07-17T08:35:35.486627+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:01.482943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552456+0000","last_change":"2024-07-17T08:35:34.485471+0000","last_active":"2024-07-17T08:35:35.552456+0000","last_peered":"2024-07-17T08:35:35.552456+0000","last_clean":"2024-07-17T08:35:35.552456+0000","last_became_active":"2024-07-17T08:35:34.485180+0000","last_became_peered":"2024-07-17T08:35:34.485180+0000","last_unstale":"2024-07-17T08:35:35.552456+0000","last_undegraded":"2024-07-17T08:35:35.552456+0000","last_fullsized":"2024-07-17T08:35:35.552456+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:52.825712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458117+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:35.458117+0000","last_peered":"2024-07-17T08:35:35.458117+0000","last_clean":"2024-07-17T08:35:35.458117+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:35.458117+0000","last_undegraded":"2024-07-17T08:35:35.458117+0000","last_fullsized":"2024-07-17T08:35:35.458117+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457075+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:35.457075+0000","last_peered":"2024-07-17T08:35:35.457075+0000","last_clean":"2024-07-17T08:35:35.457075+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:35.457075+0000","last_undegraded":"2024-07-17T08:35:35.457075+0000","last_fullsized":"2024-07-17T08:35:35.457075+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:32.339990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491696+0000","last_change":"2024-07-17T08:35:33.443001+0000","last_active":"2024-07-17T08:35:35.491696+0000","last_peered":"2024-07-17T08:35:35.491696+0000","last_clean":"2024-07-17T08:35:35.491696+0000","last_became_active":"2024-07-17T08:35:33.442630+0000","last_became_peered":"2024-07-17T08:35:33.442630+0000","last_unstale":"2024-07-17T08:35:35.491696+0000","last_undegraded":"2024-07-17T08:35:35.491696+0000","last_fullsized":"2024-07-17T08:35:35.491696+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:44.333702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552490+0000","last_change":"2024-07-17T08:35:34.484841+0000","last_active":"2024-07-17T08:35:35.552490+0000","last_peered":"2024-07-17T08:35:35.552490+0000","last_clean":"2024-07-17T08:35:35.552490+0000","last_became_active":"2024-07-17T08:35:34.484580+0000","last_became_peered":"2024-07-17T08:35:34.484580+0000","last_unstale":"2024-07-17T08:35:35.552490+0000","last_undegraded":"2024-07-17T08:35:35.552490+0000","last_fullsized":"2024-07-17T08:35:35.552490+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:20:14.336249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493351+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:35.493351+0000","last_peered":"2024-07-17T08:35:35.493351+0000","last_clean":"2024-07-17T08:35:35.493351+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:35.493351+0000","last_undegraded":"2024-07-17T08:35:35.493351+0000","last_fullsized":"2024-07-17T08:35:35.493351+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553061+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:35.553061+0000","last_peered":"2024-07-17T08:35:35.553061+0000","last_clean":"2024-07-17T08:35:35.553061+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:35.553061+0000","last_undegraded":"2024-07-17T08:35:35.553061+0000","last_fullsized":"2024-07-17T08:35:35.553061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459525+0000","last_change":"2024-07-17T08:35:33.447597+0000","last_active":"2024-07-17T08:35:35.459525+0000","last_peered":"2024-07-17T08:35:35.459525+0000","last_clean":"2024-07-17T08:35:35.459525+0000","last_became_active":"2024-07-17T08:35:33.446794+0000","last_became_peered":"2024-07-17T08:35:33.446794+0000","last_unstale":"2024-07-17T08:35:35.459525+0000","last_undegraded":"2024-07-17T08:35:35.459525+0000","last_fullsized":"2024-07-17T08:35:35.459525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:26.440472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459908+0000","last_change":"2024-07-17T08:35:34.476099+0000","last_active":"2024-07-17T08:35:35.459908+0000","last_peered":"2024-07-17T08:35:35.459908+0000","last_clean":"2024-07-17T08:35:35.459908+0000","last_became_active":"2024-07-17T08:35:34.475823+0000","last_became_peered":"2024-07-17T08:35:34.475823+0000","last_unstale":"2024-07-17T08:35:35.459908+0000","last_undegraded":"2024-07-17T08:35:35.459908+0000","last_fullsized":"2024-07-17T08:35:35.459908+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:00.486944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460588+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:35.460588+0000","last_peered":"2024-07-17T08:35:35.460588+0000","last_clean":"2024-07-17T08:35:35.460588+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:35.460588+0000","last_undegraded":"2024-07-17T08:35:35.460588+0000","last_fullsized":"2024-07-17T08:35:35.460588+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486047+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:35.486047+0000","last_peered":"2024-07-17T08:35:35.486047+0000","last_clean":"2024-07-17T08:35:35.486047+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:35.486047+0000","last_undegraded":"2024-07-17T08:35:35.486047+0000","last_fullsized":"2024-07-17T08:35:35.486047+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455982+0000","last_change":"2024-07-17T08:35:33.461285+0000","last_active":"2024-07-17T08:35:35.455982+0000","last_peered":"2024-07-17T08:35:35.455982+0000","last_clean":"2024-07-17T08:35:35.455982+0000","last_became_active":"2024-07-17T08:35:33.460353+0000","last_became_peered":"2024-07-17T08:35:33.460353+0000","last_unstale":"2024-07-17T08:35:35.455982+0000","last_undegraded":"2024-07-17T08:35:35.455982+0000","last_fullsized":"2024-07-17T08:35:35.455982+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:46.221656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449383+0000","last_change":"2024-07-17T08:35:34.459938+0000","last_active":"2024-07-17T08:35:35.449383+0000","last_peered":"2024-07-17T08:35:35.449383+0000","last_clean":"2024-07-17T08:35:35.449383+0000","last_became_active":"2024-07-17T08:35:34.459039+0000","last_became_peered":"2024-07-17T08:35:34.459039+0000","last_unstale":"2024-07-17T08:35:35.449383+0000","last_undegraded":"2024-07-17T08:35:35.449383+0000","last_fullsized":"2024-07-17T08:35:35.449383+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:05.469616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555507+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:35.555507+0000","last_peered":"2024-07-17T08:35:35.555507+0000","last_clean":"2024-07-17T08:35:35.555507+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:35.555507+0000","last_undegraded":"2024-07-17T08:35:35.555507+0000","last_fullsized":"2024-07-17T08:35:35.555507+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457232+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:35.457232+0000","last_peered":"2024-07-17T08:35:35.457232+0000","last_clean":"2024-07-17T08:35:35.457232+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:35.457232+0000","last_undegraded":"2024-07-17T08:35:35.457232+0000","last_fullsized":"2024-07-17T08:35:35.457232+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553475+0000","last_change":"2024-07-17T08:35:33.458859+0000","last_active":"2024-07-17T08:35:35.553475+0000","last_peered":"2024-07-17T08:35:35.553475+0000","last_clean":"2024-07-17T08:35:35.553475+0000","last_became_active":"2024-07-17T08:35:33.458540+0000","last_became_peered":"2024-07-17T08:35:33.458540+0000","last_unstale":"2024-07-17T08:35:35.553475+0000","last_undegraded":"2024-07-17T08:35:35.553475+0000","last_fullsized":"2024-07-17T08:35:35.553475+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:00.336730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553819+0000","last_change":"2024-07-17T08:35:34.479421+0000","last_active":"2024-07-17T08:35:35.553819+0000","last_peered":"2024-07-17T08:35:35.553819+0000","last_clean":"2024-07-17T08:35:35.553819+0000","last_became_active":"2024-07-17T08:35:34.477979+0000","last_became_peered":"2024-07-17T08:35:34.477979+0000","last_unstale":"2024-07-17T08:35:35.553819+0000","last_undegraded":"2024-07-17T08:35:35.553819+0000","last_fullsized":"2024-07-17T08:35:35.553819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.053432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456089+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:35.456089+0000","last_peered":"2024-07-17T08:35:35.456089+0000","last_clean":"2024-07-17T08:35:35.456089+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:35.456089+0000","last_undegraded":"2024-07-17T08:35:35.456089+0000","last_fullsized":"2024-07-17T08:35:35.456089+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461819+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:35.461819+0000","last_peered":"2024-07-17T08:35:35.461819+0000","last_clean":"2024-07-17T08:35:35.461819+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:35.461819+0000","last_undegraded":"2024-07-17T08:35:35.461819+0000","last_fullsized":"2024-07-17T08:35:35.461819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450729+0000","last_change":"2024-07-17T08:35:33.449985+0000","last_active":"2024-07-17T08:35:35.450729+0000","last_peered":"2024-07-17T08:35:35.450729+0000","last_clean":"2024-07-17T08:35:35.450729+0000","last_became_active":"2024-07-17T08:35:33.449677+0000","last_became_peered":"2024-07-17T08:35:33.449677+0000","last_unstale":"2024-07-17T08:35:35.450729+0000","last_undegraded":"2024-07-17T08:35:35.450729+0000","last_fullsized":"2024-07-17T08:35:35.450729+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:59.034427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552631+0000","last_change":"2024-07-17T08:35:34.488611+0000","last_active":"2024-07-17T08:35:35.552631+0000","last_peered":"2024-07-17T08:35:35.552631+0000","last_clean":"2024-07-17T08:35:35.552631+0000","last_became_active":"2024-07-17T08:35:34.486963+0000","last_became_peered":"2024-07-17T08:35:34.486963+0000","last_unstale":"2024-07-17T08:35:35.552631+0000","last_undegraded":"2024-07-17T08:35:35.552631+0000","last_fullsized":"2024-07-17T08:35:35.552631+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:00.632819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493160+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:35.493160+0000","last_peered":"2024-07-17T08:35:35.493160+0000","last_clean":"2024-07-17T08:35:35.493160+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:35.493160+0000","last_undegraded":"2024-07-17T08:35:35.493160+0000","last_fullsized":"2024-07-17T08:35:35.493160+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554092+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:35.554092+0000","last_peered":"2024-07-17T08:35:35.554092+0000","last_clean":"2024-07-17T08:35:35.554092+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:35.554092+0000","last_undegraded":"2024-07-17T08:35:35.554092+0000","last_fullsized":"2024-07-17T08:35:35.554092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555232+0000","last_change":"2024-07-17T08:35:33.455119+0000","last_active":"2024-07-17T08:35:35.555232+0000","last_peered":"2024-07-17T08:35:35.555232+0000","last_clean":"2024-07-17T08:35:35.555232+0000","last_became_active":"2024-07-17T08:35:33.454773+0000","last_became_peered":"2024-07-17T08:35:33.454773+0000","last_unstale":"2024-07-17T08:35:35.555232+0000","last_undegraded":"2024-07-17T08:35:35.555232+0000","last_fullsized":"2024-07-17T08:35:35.555232+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:38.944624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461250+0000","last_change":"2024-07-17T08:35:34.474579+0000","last_active":"2024-07-17T08:35:35.461250+0000","last_peered":"2024-07-17T08:35:35.461250+0000","last_clean":"2024-07-17T08:35:35.461250+0000","last_became_active":"2024-07-17T08:35:34.474286+0000","last_became_peered":"2024-07-17T08:35:34.474286+0000","last_unstale":"2024-07-17T08:35:35.461250+0000","last_undegraded":"2024-07-17T08:35:35.461250+0000","last_fullsized":"2024-07-17T08:35:35.461250+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:04.679552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555839+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:35.555839+0000","last_peered":"2024-07-17T08:35:35.555839+0000","last_clean":"2024-07-17T08:35:35.555839+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:35.555839+0000","last_undegraded":"2024-07-17T08:35:35.555839+0000","last_fullsized":"2024-07-17T08:35:35.555839+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554654+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:35.554654+0000","last_peered":"2024-07-17T08:35:35.554654+0000","last_clean":"2024-07-17T08:35:35.554654+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:35.554654+0000","last_undegraded":"2024-07-17T08:35:35.554654+0000","last_fullsized":"2024-07-17T08:35:35.554654+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:36.700604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461450+0000","last_change":"2024-07-17T08:35:33.457888+0000","last_active":"2024-07-17T08:35:35.461450+0000","last_peered":"2024-07-17T08:35:35.461450+0000","last_clean":"2024-07-17T08:35:35.461450+0000","last_became_active":"2024-07-17T08:35:33.457510+0000","last_became_peered":"2024-07-17T08:35:33.457510+0000","last_unstale":"2024-07-17T08:35:35.461450+0000","last_undegraded":"2024-07-17T08:35:35.461450+0000","last_fullsized":"2024-07-17T08:35:35.461450+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:51.342532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555200+0000","last_change":"2024-07-17T08:35:34.498472+0000","last_active":"2024-07-17T08:35:35.555200+0000","last_peered":"2024-07-17T08:35:35.555200+0000","last_clean":"2024-07-17T08:35:35.555200+0000","last_became_active":"2024-07-17T08:35:34.492593+0000","last_became_peered":"2024-07-17T08:35:34.492593+0000","last_unstale":"2024-07-17T08:35:35.555200+0000","last_undegraded":"2024-07-17T08:35:35.555200+0000","last_fullsized":"2024-07-17T08:35:35.555200+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:28.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450203+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:35.450203+0000","last_peered":"2024-07-17T08:35:35.450203+0000","last_clean":"2024-07-17T08:35:35.450203+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:35.450203+0000","last_undegraded":"2024-07-17T08:35:35.450203+0000","last_fullsized":"2024-07-17T08:35:35.450203+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554679+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:35.554679+0000","last_peered":"2024-07-17T08:35:35.554679+0000","last_clean":"2024-07-17T08:35:35.554679+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:35.554679+0000","last_undegraded":"2024-07-17T08:35:35.554679+0000","last_fullsized":"2024-07-17T08:35:35.554679+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:53.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459946+0000","last_change":"2024-07-17T08:35:33.456857+0000","last_active":"2024-07-17T08:35:35.459946+0000","last_peered":"2024-07-17T08:35:35.459946+0000","last_clean":"2024-07-17T08:35:35.459946+0000","last_became_active":"2024-07-17T08:35:33.456290+0000","last_became_peered":"2024-07-17T08:35:33.456290+0000","last_unstale":"2024-07-17T08:35:35.459946+0000","last_undegraded":"2024-07-17T08:35:35.459946+0000","last_fullsized":"2024-07-17T08:35:35.459946+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:12.755522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553191+0000","last_change":"2024-07-17T08:35:34.491188+0000","last_active":"2024-07-17T08:35:35.553191+0000","last_peered":"2024-07-17T08:35:35.553191+0000","last_clean":"2024-07-17T08:35:35.553191+0000","last_became_active":"2024-07-17T08:35:34.489934+0000","last_became_peered":"2024-07-17T08:35:34.489934+0000","last_unstale":"2024-07-17T08:35:35.553191+0000","last_undegraded":"2024-07-17T08:35:35.553191+0000","last_fullsized":"2024-07-17T08:35:35.553191+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:04.970179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450322+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:35.450322+0000","last_peered":"2024-07-17T08:35:35.450322+0000","last_clean":"2024-07-17T08:35:35.450322+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:35.450322+0000","last_undegraded":"2024-07-17T08:35:35.450322+0000","last_fullsized":"2024-07-17T08:35:35.450322+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460400+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:35.460400+0000","last_peered":"2024-07-17T08:35:35.460400+0000","last_clean":"2024-07-17T08:35:35.460400+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:35.460400+0000","last_undegraded":"2024-07-17T08:35:35.460400+0000","last_fullsized":"2024-07-17T08:35:35.460400+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486062+0000","last_change":"2024-07-17T08:35:33.454020+0000","last_active":"2024-07-17T08:35:35.486062+0000","last_peered":"2024-07-17T08:35:35.486062+0000","last_clean":"2024-07-17T08:35:35.486062+0000","last_became_active":"2024-07-17T08:35:33.453743+0000","last_became_peered":"2024-07-17T08:35:33.453743+0000","last_unstale":"2024-07-17T08:35:35.486062+0000","last_undegraded":"2024-07-17T08:35:35.486062+0000","last_fullsized":"2024-07-17T08:35:35.486062+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:59.652207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455403+0000","last_change":"2024-07-17T08:35:34.486630+0000","last_active":"2024-07-17T08:35:35.455403+0000","last_peered":"2024-07-17T08:35:35.455403+0000","last_clean":"2024-07-17T08:35:35.455403+0000","last_became_active":"2024-07-17T08:35:34.485558+0000","last_became_peered":"2024-07-17T08:35:34.485558+0000","last_unstale":"2024-07-17T08:35:35.455403+0000","last_undegraded":"2024-07-17T08:35:35.455403+0000","last_fullsized":"2024-07-17T08:35:35.455403+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:20.604380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554451+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:35.554451+0000","last_peered":"2024-07-17T08:35:35.554451+0000","last_clean":"2024-07-17T08:35:35.554451+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:35.554451+0000","last_undegraded":"2024-07-17T08:35:35.554451+0000","last_fullsized":"2024-07-17T08:35:35.554451+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458964+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:35.458964+0000","last_peered":"2024-07-17T08:35:35.458964+0000","last_clean":"2024-07-17T08:35:35.458964+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:35.458964+0000","last_undegraded":"2024-07-17T08:35:35.458964+0000","last_fullsized":"2024-07-17T08:35:35.458964+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:07.288084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459274+0000","last_change":"2024-07-17T08:35:33.451492+0000","last_active":"2024-07-17T08:35:35.459274+0000","last_peered":"2024-07-17T08:35:35.459274+0000","last_clean":"2024-07-17T08:35:35.459274+0000","last_became_active":"2024-07-17T08:35:33.451016+0000","last_became_peered":"2024-07-17T08:35:33.451016+0000","last_unstale":"2024-07-17T08:35:35.459274+0000","last_undegraded":"2024-07-17T08:35:35.459274+0000","last_fullsized":"2024-07-17T08:35:35.459274+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:32.267947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553479+0000","last_change":"2024-07-17T08:35:34.497474+0000","last_active":"2024-07-17T08:35:35.553479+0000","last_peered":"2024-07-17T08:35:35.553479+0000","last_clean":"2024-07-17T08:35:35.553479+0000","last_became_active":"2024-07-17T08:35:34.486892+0000","last_became_peered":"2024-07-17T08:35:34.486892+0000","last_unstale":"2024-07-17T08:35:35.553479+0000","last_undegraded":"2024-07-17T08:35:35.553479+0000","last_fullsized":"2024-07-17T08:35:35.553479+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:35.295643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458654+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:35.458654+0000","last_peered":"2024-07-17T08:35:35.458654+0000","last_clean":"2024-07-17T08:35:35.458654+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:35.458654+0000","last_undegraded":"2024-07-17T08:35:35.458654+0000","last_fullsized":"2024-07-17T08:35:35.458654+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455763+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:35.455763+0000","last_peered":"2024-07-17T08:35:35.455763+0000","last_clean":"2024-07-17T08:35:35.455763+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:35.455763+0000","last_undegraded":"2024-07-17T08:35:35.455763+0000","last_fullsized":"2024-07-17T08:35:35.455763+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:44.865077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.448996+0000","last_change":"2024-07-17T08:35:33.440197+0000","last_active":"2024-07-17T08:35:35.448996+0000","last_peered":"2024-07-17T08:35:35.448996+0000","last_clean":"2024-07-17T08:35:35.448996+0000","last_became_active":"2024-07-17T08:35:33.439893+0000","last_became_peered":"2024-07-17T08:35:33.439893+0000","last_unstale":"2024-07-17T08:35:35.448996+0000","last_undegraded":"2024-07-17T08:35:35.448996+0000","last_fullsized":"2024-07-17T08:35:35.448996+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:23.178685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552047+0000","last_change":"2024-07-17T08:35:34.486149+0000","last_active":"2024-07-17T08:35:35.552047+0000","last_peered":"2024-07-17T08:35:35.552047+0000","last_clean":"2024-07-17T08:35:35.552047+0000","last_became_active":"2024-07-17T08:35:34.485165+0000","last_became_peered":"2024-07-17T08:35:34.485165+0000","last_unstale":"2024-07-17T08:35:35.552047+0000","last_undegraded":"2024-07-17T08:35:35.552047+0000","last_fullsized":"2024-07-17T08:35:35.552047+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:33.375893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459284+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:35.459284+0000","last_peered":"2024-07-17T08:35:35.459284+0000","last_clean":"2024-07-17T08:35:35.459284+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:35.459284+0000","last_undegraded":"2024-07-17T08:35:35.459284+0000","last_fullsized":"2024-07-17T08:35:35.459284+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456973+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:35.456973+0000","last_peered":"2024-07-17T08:35:35.456973+0000","last_clean":"2024-07-17T08:35:35.456973+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:35.456973+0000","last_undegraded":"2024-07-17T08:35:35.456973+0000","last_fullsized":"2024-07-17T08:35:35.456973+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485924+0000","last_change":"2024-07-17T08:35:33.451532+0000","last_active":"2024-07-17T08:35:35.485924+0000","last_peered":"2024-07-17T08:35:35.485924+0000","last_clean":"2024-07-17T08:35:35.485924+0000","last_became_active":"2024-07-17T08:35:33.449530+0000","last_became_peered":"2024-07-17T08:35:33.449530+0000","last_unstale":"2024-07-17T08:35:35.485924+0000","last_undegraded":"2024-07-17T08:35:35.485924+0000","last_fullsized":"2024-07-17T08:35:35.485924+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:53.409692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486540+0000","last_change":"2024-07-17T08:35:34.484134+0000","last_active":"2024-07-17T08:35:35.486540+0000","last_peered":"2024-07-17T08:35:35.486540+0000","last_clean":"2024-07-17T08:35:35.486540+0000","last_became_active":"2024-07-17T08:35:34.480964+0000","last_became_peered":"2024-07-17T08:35:34.480964+0000","last_unstale":"2024-07-17T08:35:35.486540+0000","last_undegraded":"2024-07-17T08:35:35.486540+0000","last_fullsized":"2024-07-17T08:35:35.486540+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:22.103527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449153+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:35.449153+0000","last_peered":"2024-07-17T08:35:35.449153+0000","last_clean":"2024-07-17T08:35:35.449153+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:35.449153+0000","last_undegraded":"2024-07-17T08:35:35.449153+0000","last_fullsized":"2024-07-17T08:35:35.449153+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555620+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:35.555620+0000","last_peered":"2024-07-17T08:35:35.555620+0000","last_clean":"2024-07-17T08:35:35.555620+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:35.555620+0000","last_undegraded":"2024-07-17T08:35:35.555620+0000","last_fullsized":"2024-07-17T08:35:35.555620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:38.094519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492867+0000","last_change":"2024-07-17T08:35:33.456460+0000","last_active":"2024-07-17T08:35:35.492867+0000","last_peered":"2024-07-17T08:35:35.492867+0000","last_clean":"2024-07-17T08:35:35.492867+0000","last_became_active":"2024-07-17T08:35:33.456193+0000","last_became_peered":"2024-07-17T08:35:33.456193+0000","last_unstale":"2024-07-17T08:35:35.492867+0000","last_undegraded":"2024-07-17T08:35:35.492867+0000","last_fullsized":"2024-07-17T08:35:35.492867+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:00.094046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486268+0000","last_change":"2024-07-17T08:35:34.485589+0000","last_active":"2024-07-17T08:35:35.486268+0000","last_peered":"2024-07-17T08:35:35.486268+0000","last_clean":"2024-07-17T08:35:35.486268+0000","last_became_active":"2024-07-17T08:35:34.485287+0000","last_became_peered":"2024-07-17T08:35:34.485287+0000","last_unstale":"2024-07-17T08:35:35.486268+0000","last_undegraded":"2024-07-17T08:35:35.486268+0000","last_fullsized":"2024-07-17T08:35:35.486268+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:13.147791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554416+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:35.554416+0000","last_peered":"2024-07-17T08:35:35.554416+0000","last_clean":"2024-07-17T08:35:35.554416+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:35.554416+0000","last_undegraded":"2024-07-17T08:35:35.554416+0000","last_fullsized":"2024-07-17T08:35:35.554416+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553997+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:35.553997+0000","last_peered":"2024-07-17T08:35:35.553997+0000","last_clean":"2024-07-17T08:35:35.553997+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:35.553997+0000","last_undegraded":"2024-07-17T08:35:35.553997+0000","last_fullsized":"2024-07-17T08:35:35.553997+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555546+0000","last_change":"2024-07-17T08:35:33.460434+0000","last_active":"2024-07-17T08:35:35.555546+0000","last_peered":"2024-07-17T08:35:35.555546+0000","last_clean":"2024-07-17T08:35:35.555546+0000","last_became_active":"2024-07-17T08:35:33.460174+0000","last_became_peered":"2024-07-17T08:35:33.460174+0000","last_unstale":"2024-07-17T08:35:35.555546+0000","last_undegraded":"2024-07-17T08:35:35.555546+0000","last_fullsized":"2024-07-17T08:35:35.555546+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:33.539981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492894+0000","last_change":"2024-07-17T08:35:34.478029+0000","last_active":"2024-07-17T08:35:35.492894+0000","last_peered":"2024-07-17T08:35:35.492894+0000","last_clean":"2024-07-17T08:35:35.492894+0000","last_became_active":"2024-07-17T08:35:34.474129+0000","last_became_peered":"2024-07-17T08:35:34.474129+0000","last_unstale":"2024-07-17T08:35:35.492894+0000","last_undegraded":"2024-07-17T08:35:35.492894+0000","last_fullsized":"2024-07-17T08:35:35.492894+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:46.401539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486426+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:35.486426+0000","last_peered":"2024-07-17T08:35:35.486426+0000","last_clean":"2024-07-17T08:35:35.486426+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:35.486426+0000","last_undegraded":"2024-07-17T08:35:35.486426+0000","last_fullsized":"2024-07-17T08:35:35.486426+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486816+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:35.486816+0000","last_peered":"2024-07-17T08:35:35.486816+0000","last_clean":"2024-07-17T08:35:35.486816+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:35.486816+0000","last_undegraded":"2024-07-17T08:35:35.486816+0000","last_fullsized":"2024-07-17T08:35:35.486816+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"226'6","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458001+0000","last_change":"2024-07-17T08:35:33.458710+0000","last_active":"2024-07-17T08:35:35.458001+0000","last_peered":"2024-07-17T08:35:35.458001+0000","last_clean":"2024-07-17T08:35:35.458001+0000","last_became_active":"2024-07-17T08:35:33.458420+0000","last_became_peered":"2024-07-17T08:35:33.458420+0000","last_unstale":"2024-07-17T08:35:35.458001+0000","last_undegraded":"2024-07-17T08:35:35.458001+0000","last_fullsized":"2024-07-17T08:35:35.458001+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:01.698884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555436+0000","last_change":"2024-07-17T08:35:34.483586+0000","last_active":"2024-07-17T08:35:35.555436+0000","last_peered":"2024-07-17T08:35:35.555436+0000","last_clean":"2024-07-17T08:35:35.555436+0000","last_became_active":"2024-07-17T08:35:34.483162+0000","last_became_peered":"2024-07-17T08:35:34.483162+0000","last_unstale":"2024-07-17T08:35:35.555436+0000","last_undegraded":"2024-07-17T08:35:35.555436+0000","last_fullsized":"2024-07-17T08:35:35.555436+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:20.206854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553417+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:35.553417+0000","last_peered":"2024-07-17T08:35:35.553417+0000","last_clean":"2024-07-17T08:35:35.553417+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:35.553417+0000","last_undegraded":"2024-07-17T08:35:35.553417+0000","last_fullsized":"2024-07-17T08:35:35.553417+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554601+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:35.554601+0000","last_peered":"2024-07-17T08:35:35.554601+0000","last_clean":"2024-07-17T08:35:35.554601+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:35.554601+0000","last_undegraded":"2024-07-17T08:35:35.554601+0000","last_fullsized":"2024-07-17T08:35:35.554601+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:07:04.780778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"226'3","reported_seq":17,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554549+0000","last_change":"2024-07-17T08:35:33.459431+0000","last_active":"2024-07-17T08:35:35.554549+0000","last_peered":"2024-07-17T08:35:35.554549+0000","last_clean":"2024-07-17T08:35:35.554549+0000","last_became_active":"2024-07-17T08:35:33.459129+0000","last_became_peered":"2024-07-17T08:35:33.459129+0000","last_unstale":"2024-07-17T08:35:35.554549+0000","last_undegraded":"2024-07-17T08:35:35.554549+0000","last_fullsized":"2024-07-17T08:35:35.554549+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:13.823820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455802+0000","last_change":"2024-07-17T08:35:34.493124+0000","last_active":"2024-07-17T08:35:35.455802+0000","last_peered":"2024-07-17T08:35:35.455802+0000","last_clean":"2024-07-17T08:35:35.455802+0000","last_became_active":"2024-07-17T08:35:34.492795+0000","last_became_peered":"2024-07-17T08:35:34.492795+0000","last_unstale":"2024-07-17T08:35:35.455802+0000","last_undegraded":"2024-07-17T08:35:35.455802+0000","last_fullsized":"2024-07-17T08:35:35.455802+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:10.255450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"223'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554137+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:35.554137+0000","last_peered":"2024-07-17T08:35:35.554137+0000","last_clean":"2024-07-17T08:35:35.554137+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:35.554137+0000","last_undegraded":"2024-07-17T08:35:35.554137+0000","last_fullsized":"2024-07-17T08:35:35.554137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450149+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:35.450149+0000","last_peered":"2024-07-17T08:35:35.450149+0000","last_clean":"2024-07-17T08:35:35.450149+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:35.450149+0000","last_undegraded":"2024-07-17T08:35:35.450149+0000","last_fullsized":"2024-07-17T08:35:35.450149+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552761+0000","last_change":"2024-07-17T08:35:33.438513+0000","last_active":"2024-07-17T08:35:35.552761+0000","last_peered":"2024-07-17T08:35:35.552761+0000","last_clean":"2024-07-17T08:35:35.552761+0000","last_became_active":"2024-07-17T08:35:33.438129+0000","last_became_peered":"2024-07-17T08:35:33.438129+0000","last_unstale":"2024-07-17T08:35:35.552761+0000","last_undegraded":"2024-07-17T08:35:35.552761+0000","last_fullsized":"2024-07-17T08:35:35.552761+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:20:04.565470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551555+0000","last_change":"2024-07-17T08:35:34.483491+0000","last_active":"2024-07-17T08:35:35.551555+0000","last_peered":"2024-07-17T08:35:35.551555+0000","last_clean":"2024-07-17T08:35:35.551555+0000","last_became_active":"2024-07-17T08:35:34.483129+0000","last_became_peered":"2024-07-17T08:35:34.483129+0000","last_unstale":"2024-07-17T08:35:35.551555+0000","last_undegraded":"2024-07-17T08:35:35.551555+0000","last_fullsized":"2024-07-17T08:35:35.551555+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:39.959193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":25,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552904+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:35.552904+0000","last_peered":"2024-07-17T08:35:35.552904+0000","last_clean":"2024-07-17T08:35:35.552904+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:35.552904+0000","last_undegraded":"2024-07-17T08:35:35.552904+0000","last_fullsized":"2024-07-17T08:35:35.552904+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553318+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:35.553318+0000","last_peered":"2024-07-17T08:35:35.553318+0000","last_clean":"2024-07-17T08:35:35.553318+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:35.553318+0000","last_undegraded":"2024-07-17T08:35:35.553318+0000","last_fullsized":"2024-07-17T08:35:35.553318+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458967+0000","last_change":"2024-07-17T08:35:33.451275+0000","last_active":"2024-07-17T08:35:35.458967+0000","last_peered":"2024-07-17T08:35:35.458967+0000","last_clean":"2024-07-17T08:35:35.458967+0000","last_became_active":"2024-07-17T08:35:33.450592+0000","last_became_peered":"2024-07-17T08:35:33.450592+0000","last_unstale":"2024-07-17T08:35:35.458967+0000","last_undegraded":"2024-07-17T08:35:35.458967+0000","last_fullsized":"2024-07-17T08:35:35.458967+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:47.040622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486251+0000","last_change":"2024-07-17T08:35:34.496715+0000","last_active":"2024-07-17T08:35:35.486251+0000","last_peered":"2024-07-17T08:35:35.486251+0000","last_clean":"2024-07-17T08:35:35.486251+0000","last_became_active":"2024-07-17T08:35:34.496413+0000","last_became_peered":"2024-07-17T08:35:34.496413+0000","last_unstale":"2024-07-17T08:35:35.486251+0000","last_undegraded":"2024-07-17T08:35:35.486251+0000","last_fullsized":"2024-07-17T08:35:35.486251+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:04.804268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555332+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:35.555332+0000","last_peered":"2024-07-17T08:35:35.555332+0000","last_clean":"2024-07-17T08:35:35.555332+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:35.555332+0000","last_undegraded":"2024-07-17T08:35:35.555332+0000","last_fullsized":"2024-07-17T08:35:35.555332+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485967+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:35.485967+0000","last_peered":"2024-07-17T08:35:35.485967+0000","last_clean":"2024-07-17T08:35:35.485967+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:35.485967+0000","last_undegraded":"2024-07-17T08:35:35.485967+0000","last_fullsized":"2024-07-17T08:35:35.485967+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"226'8","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455662+0000","last_change":"2024-07-17T08:35:33.460898+0000","last_active":"2024-07-17T08:35:35.455662+0000","last_peered":"2024-07-17T08:35:35.455662+0000","last_clean":"2024-07-17T08:35:35.455662+0000","last_became_active":"2024-07-17T08:35:33.460626+0000","last_became_peered":"2024-07-17T08:35:33.460626+0000","last_unstale":"2024-07-17T08:35:35.455662+0000","last_undegraded":"2024-07-17T08:35:35.455662+0000","last_fullsized":"2024-07-17T08:35:35.455662+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:10.216965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555230+0000","last_change":"2024-07-17T08:35:34.497427+0000","last_active":"2024-07-17T08:35:35.555230+0000","last_peered":"2024-07-17T08:35:35.555230+0000","last_clean":"2024-07-17T08:35:35.555230+0000","last_became_active":"2024-07-17T08:35:34.496490+0000","last_became_peered":"2024-07-17T08:35:34.496490+0000","last_unstale":"2024-07-17T08:35:35.555230+0000","last_undegraded":"2024-07-17T08:35:35.555230+0000","last_fullsized":"2024-07-17T08:35:35.555230+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:27.078421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451056+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:35.451056+0000","last_peered":"2024-07-17T08:35:35.451056+0000","last_clean":"2024-07-17T08:35:35.451056+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:35.451056+0000","last_undegraded":"2024-07-17T08:35:35.451056+0000","last_fullsized":"2024-07-17T08:35:35.451056+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491687+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:35.491687+0000","last_peered":"2024-07-17T08:35:35.491687+0000","last_clean":"2024-07-17T08:35:35.491687+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:35.491687+0000","last_undegraded":"2024-07-17T08:35:35.491687+0000","last_fullsized":"2024-07-17T08:35:35.491687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:21.845715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454949+0000","last_change":"2024-07-17T08:35:33.452238+0000","last_active":"2024-07-17T08:35:35.454949+0000","last_peered":"2024-07-17T08:35:35.454949+0000","last_clean":"2024-07-17T08:35:35.454949+0000","last_became_active":"2024-07-17T08:35:33.450790+0000","last_became_peered":"2024-07-17T08:35:33.450790+0000","last_unstale":"2024-07-17T08:35:35.454949+0000","last_undegraded":"2024-07-17T08:35:35.454949+0000","last_fullsized":"2024-07-17T08:35:35.454949+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:24.729759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553337+0000","last_change":"2024-07-17T08:35:34.483916+0000","last_active":"2024-07-17T08:35:35.553337+0000","last_peered":"2024-07-17T08:35:35.553337+0000","last_clean":"2024-07-17T08:35:35.553337+0000","last_became_active":"2024-07-17T08:35:34.483101+0000","last_became_peered":"2024-07-17T08:35:34.483101+0000","last_unstale":"2024-07-17T08:35:35.553337+0000","last_undegraded":"2024-07-17T08:35:35.553337+0000","last_fullsized":"2024-07-17T08:35:35.553337+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:46.506398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460048+0000","last_change":"2024-07-17T08:35:33.456972+0000","last_active":"2024-07-17T08:35:35.460048+0000","last_peered":"2024-07-17T08:35:35.460048+0000","last_clean":"2024-07-17T08:35:35.460048+0000","last_became_active":"2024-07-17T08:35:33.452779+0000","last_became_peered":"2024-07-17T08:35:33.452779+0000","last_unstale":"2024-07-17T08:35:35.460048+0000","last_undegraded":"2024-07-17T08:35:35.460048+0000","last_fullsized":"2024-07-17T08:35:35.460048+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:29:43.620854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485766+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:35.485766+0000","last_peered":"2024-07-17T08:35:35.485766+0000","last_clean":"2024-07-17T08:35:35.485766+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:35.485766+0000","last_undegraded":"2024-07-17T08:35:35.485766+0000","last_fullsized":"2024-07-17T08:35:35.485766+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555255+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:35.555255+0000","last_peered":"2024-07-17T08:35:35.555255+0000","last_clean":"2024-07-17T08:35:35.555255+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:35.555255+0000","last_undegraded":"2024-07-17T08:35:35.555255+0000","last_fullsized":"2024-07-17T08:35:35.555255+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554158+0000","last_change":"2024-07-17T08:35:34.491080+0000","last_active":"2024-07-17T08:35:35.554158+0000","last_peered":"2024-07-17T08:35:35.554158+0000","last_clean":"2024-07-17T08:35:35.554158+0000","last_became_active":"2024-07-17T08:35:34.490804+0000","last_became_peered":"2024-07-17T08:35:34.490804+0000","last_unstale":"2024-07-17T08:35:35.554158+0000","last_undegraded":"2024-07-17T08:35:35.554158+0000","last_fullsized":"2024-07-17T08:35:35.554158+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:50.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"226'4","reported_seq":18,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450844+0000","last_change":"2024-07-17T08:35:33.446100+0000","last_active":"2024-07-17T08:35:35.450844+0000","last_peered":"2024-07-17T08:35:35.450844+0000","last_clean":"2024-07-17T08:35:35.450844+0000","last_became_active":"2024-07-17T08:35:33.445751+0000","last_became_peered":"2024-07-17T08:35:33.445751+0000","last_unstale":"2024-07-17T08:35:35.450844+0000","last_undegraded":"2024-07-17T08:35:35.450844+0000","last_fullsized":"2024-07-17T08:35:35.450844+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:18.271351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553518+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:35.553518+0000","last_peered":"2024-07-17T08:35:35.553518+0000","last_clean":"2024-07-17T08:35:35.553518+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:35.553518+0000","last_undegraded":"2024-07-17T08:35:35.553518+0000","last_fullsized":"2024-07-17T08:35:35.553518+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493201+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:35.493201+0000","last_peered":"2024-07-17T08:35:35.493201+0000","last_clean":"2024-07-17T08:35:35.493201+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:35.493201+0000","last_undegraded":"2024-07-17T08:35:35.493201+0000","last_fullsized":"2024-07-17T08:35:35.493201+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:19.204797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461020+0000","last_change":"2024-07-17T08:35:34.495578+0000","last_active":"2024-07-17T08:35:35.461020+0000","last_peered":"2024-07-17T08:35:35.461020+0000","last_clean":"2024-07-17T08:35:35.461020+0000","last_became_active":"2024-07-17T08:35:34.495255+0000","last_became_peered":"2024-07-17T08:35:34.495255+0000","last_unstale":"2024-07-17T08:35:35.461020+0000","last_undegraded":"2024-07-17T08:35:35.461020+0000","last_fullsized":"2024-07-17T08:35:35.461020+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:38.958975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554180+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:35.554180+0000","last_peered":"2024-07-17T08:35:35.554180+0000","last_clean":"2024-07-17T08:35:35.554180+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:35.554180+0000","last_undegraded":"2024-07-17T08:35:35.554180+0000","last_fullsized":"2024-07-17T08:35:35.554180+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492633+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:35.492633+0000","last_peered":"2024-07-17T08:35:35.492633+0000","last_clean":"2024-07-17T08:35:35.492633+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:35.492633+0000","last_undegraded":"2024-07-17T08:35:35.492633+0000","last_fullsized":"2024-07-17T08:35:35.492633+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:41.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457525+0000","last_change":"2024-07-17T08:35:33.458703+0000","last_active":"2024-07-17T08:35:35.457525+0000","last_peered":"2024-07-17T08:35:35.457525+0000","last_clean":"2024-07-17T08:35:35.457525+0000","last_became_active":"2024-07-17T08:35:33.458365+0000","last_became_peered":"2024-07-17T08:35:33.458365+0000","last_unstale":"2024-07-17T08:35:35.457525+0000","last_undegraded":"2024-07-17T08:35:35.457525+0000","last_fullsized":"2024-07-17T08:35:35.457525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:25.173461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455182+0000","last_change":"2024-07-17T08:35:34.472507+0000","last_active":"2024-07-17T08:35:35.455182+0000","last_peered":"2024-07-17T08:35:35.455182+0000","last_clean":"2024-07-17T08:35:35.455182+0000","last_became_active":"2024-07-17T08:35:34.472042+0000","last_became_peered":"2024-07-17T08:35:34.472042+0000","last_unstale":"2024-07-17T08:35:35.455182+0000","last_undegraded":"2024-07-17T08:35:35.455182+0000","last_fullsized":"2024-07-17T08:35:35.455182+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:57.135370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554169+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:35.554169+0000","last_peered":"2024-07-17T08:35:35.554169+0000","last_clean":"2024-07-17T08:35:35.554169+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:35.554169+0000","last_undegraded":"2024-07-17T08:35:35.554169+0000","last_fullsized":"2024-07-17T08:35:35.554169+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450061+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:35.450061+0000","last_peered":"2024-07-17T08:35:35.450061+0000","last_clean":"2024-07-17T08:35:35.450061+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:35.450061+0000","last_undegraded":"2024-07-17T08:35:35.450061+0000","last_fullsized":"2024-07-17T08:35:35.450061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553207+0000","last_change":"2024-07-17T08:35:33.462435+0000","last_active":"2024-07-17T08:35:35.553207+0000","last_peered":"2024-07-17T08:35:35.553207+0000","last_clean":"2024-07-17T08:35:35.553207+0000","last_became_active":"2024-07-17T08:35:33.462140+0000","last_became_peered":"2024-07-17T08:35:33.462140+0000","last_unstale":"2024-07-17T08:35:35.553207+0000","last_undegraded":"2024-07-17T08:35:35.553207+0000","last_fullsized":"2024-07-17T08:35:35.553207+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:49.267551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551332+0000","last_change":"2024-07-17T08:35:34.503686+0000","last_active":"2024-07-17T08:35:35.551332+0000","last_peered":"2024-07-17T08:35:35.551332+0000","last_clean":"2024-07-17T08:35:35.551332+0000","last_became_active":"2024-07-17T08:35:34.502946+0000","last_became_peered":"2024-07-17T08:35:34.502946+0000","last_unstale":"2024-07-17T08:35:35.551332+0000","last_undegraded":"2024-07-17T08:35:35.551332+0000","last_fullsized":"2024-07-17T08:35:35.551332+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:34.944240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553744+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:35.553744+0000","last_peered":"2024-07-17T08:35:35.553744+0000","last_clean":"2024-07-17T08:35:35.553744+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:35.553744+0000","last_undegraded":"2024-07-17T08:35:35.553744+0000","last_fullsized":"2024-07-17T08:35:35.553744+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486237+0000","last_change":"2024-07-17T08:35:33.464947+0000","last_active":"2024-07-17T08:35:35.486237+0000","last_peered":"2024-07-17T08:35:35.486237+0000","last_clean":"2024-07-17T08:35:35.486237+0000","last_became_active":"2024-07-17T08:35:33.464637+0000","last_became_peered":"2024-07-17T08:35:33.464637+0000","last_unstale":"2024-07-17T08:35:35.486237+0000","last_undegraded":"2024-07-17T08:35:35.486237+0000","last_fullsized":"2024-07-17T08:35:35.486237+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:33.340251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449931+0000","last_change":"2024-07-17T08:35:34.485339+0000","last_active":"2024-07-17T08:35:35.449931+0000","last_peered":"2024-07-17T08:35:35.449931+0000","last_clean":"2024-07-17T08:35:35.449931+0000","last_became_active":"2024-07-17T08:35:34.485071+0000","last_became_peered":"2024-07-17T08:35:34.485071+0000","last_unstale":"2024-07-17T08:35:35.449931+0000","last_undegraded":"2024-07-17T08:35:35.449931+0000","last_fullsized":"2024-07-17T08:35:35.449931+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:58.882312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458370+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:35.458370+0000","last_peered":"2024-07-17T08:35:35.458370+0000","last_clean":"2024-07-17T08:35:35.458370+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:35.458370+0000","last_undegraded":"2024-07-17T08:35:35.458370+0000","last_fullsized":"2024-07-17T08:35:35.458370+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455489+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:35.455489+0000","last_peered":"2024-07-17T08:35:35.455489+0000","last_clean":"2024-07-17T08:35:35.455489+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:35.455489+0000","last_undegraded":"2024-07-17T08:35:35.455489+0000","last_fullsized":"2024-07-17T08:35:35.455489+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:18.513656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551486+0000","last_change":"2024-07-17T08:35:33.450591+0000","last_active":"2024-07-17T08:35:35.551486+0000","last_peered":"2024-07-17T08:35:35.551486+0000","last_clean":"2024-07-17T08:35:35.551486+0000","last_became_active":"2024-07-17T08:35:33.450291+0000","last_became_peered":"2024-07-17T08:35:33.450291+0000","last_unstale":"2024-07-17T08:35:35.551486+0000","last_undegraded":"2024-07-17T08:35:35.551486+0000","last_fullsized":"2024-07-17T08:35:35.551486+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:11.346934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486484+0000","last_change":"2024-07-17T08:35:34.495666+0000","last_active":"2024-07-17T08:35:35.486484+0000","last_peered":"2024-07-17T08:35:35.486484+0000","last_clean":"2024-07-17T08:35:35.486484+0000","last_became_active":"2024-07-17T08:35:34.493428+0000","last_became_peered":"2024-07-17T08:35:34.493428+0000","last_unstale":"2024-07-17T08:35:35.486484+0000","last_undegraded":"2024-07-17T08:35:35.486484+0000","last_fullsized":"2024-07-17T08:35:35.486484+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:14.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":26,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493209+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:35.493209+0000","last_peered":"2024-07-17T08:35:35.493209+0000","last_clean":"2024-07-17T08:35:35.493209+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:35.493209+0000","last_undegraded":"2024-07-17T08:35:35.493209+0000","last_fullsized":"2024-07-17T08:35:35.493209+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552776+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:35.552776+0000","last_peered":"2024-07-17T08:35:35.552776+0000","last_clean":"2024-07-17T08:35:35.552776+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:35.552776+0000","last_undegraded":"2024-07-17T08:35:35.552776+0000","last_fullsized":"2024-07-17T08:35:35.552776+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:42.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491432+0000","last_change":"2024-07-17T08:35:33.454797+0000","last_active":"2024-07-17T08:35:35.491432+0000","last_peered":"2024-07-17T08:35:35.491432+0000","last_clean":"2024-07-17T08:35:35.491432+0000","last_became_active":"2024-07-17T08:35:33.454327+0000","last_became_peered":"2024-07-17T08:35:33.454327+0000","last_unstale":"2024-07-17T08:35:35.491432+0000","last_undegraded":"2024-07-17T08:35:35.491432+0000","last_fullsized":"2024-07-17T08:35:35.491432+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:51.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455439+0000","last_change":"2024-07-17T08:35:34.490533+0000","last_active":"2024-07-17T08:35:35.455439+0000","last_peered":"2024-07-17T08:35:35.455439+0000","last_clean":"2024-07-17T08:35:35.455439+0000","last_became_active":"2024-07-17T08:35:34.488112+0000","last_became_peered":"2024-07-17T08:35:34.488112+0000","last_unstale":"2024-07-17T08:35:35.455439+0000","last_undegraded":"2024-07-17T08:35:35.455439+0000","last_fullsized":"2024-07-17T08:35:35.455439+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:15:00.666776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553336+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:35.553336+0000","last_peered":"2024-07-17T08:35:35.553336+0000","last_clean":"2024-07-17T08:35:35.553336+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:35.553336+0000","last_undegraded":"2024-07-17T08:35:35.553336+0000","last_fullsized":"2024-07-17T08:35:35.553336+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:09:55.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552022+0000","last_change":"2024-07-17T08:35:34.484796+0000","last_active":"2024-07-17T08:35:35.552022+0000","last_peered":"2024-07-17T08:35:35.552022+0000","last_clean":"2024-07-17T08:35:35.552022+0000","last_became_active":"2024-07-17T08:35:34.484514+0000","last_became_peered":"2024-07-17T08:35:34.484514+0000","last_unstale":"2024-07-17T08:35:35.552022+0000","last_undegraded":"2024-07-17T08:35:35.552022+0000","last_fullsized":"2024-07-17T08:35:35.552022+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:09.046835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553249+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:35.553249+0000","last_peered":"2024-07-17T08:35:35.553249+0000","last_clean":"2024-07-17T08:35:35.553249+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:35.553249+0000","last_undegraded":"2024-07-17T08:35:35.553249+0000","last_fullsized":"2024-07-17T08:35:35.553249+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:01.628007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455254+0000","last_change":"2024-07-17T08:35:34.488202+0000","last_active":"2024-07-17T08:35:35.455254+0000","last_peered":"2024-07-17T08:35:35.455254+0000","last_clean":"2024-07-17T08:35:35.455254+0000","last_became_active":"2024-07-17T08:35:34.487794+0000","last_became_peered":"2024-07-17T08:35:34.487794+0000","last_unstale":"2024-07-17T08:35:35.455254+0000","last_undegraded":"2024-07-17T08:35:35.455254+0000","last_fullsized":"2024-07-17T08:35:35.455254+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:34.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492044+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:35.492044+0000","last_peered":"2024-07-17T08:35:35.492044+0000","last_clean":"2024-07-17T08:35:35.492044+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:35.492044+0000","last_undegraded":"2024-07-17T08:35:35.492044+0000","last_fullsized":"2024-07-17T08:35:35.492044+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:25.913456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449659+0000","last_change":"2024-07-17T08:35:34.489886+0000","last_active":"2024-07-17T08:35:35.449659+0000","last_peered":"2024-07-17T08:35:35.449659+0000","last_clean":"2024-07-17T08:35:35.449659+0000","last_became_active":"2024-07-17T08:35:34.489612+0000","last_became_peered":"2024-07-17T08:35:34.489612+0000","last_unstale":"2024-07-17T08:35:35.449659+0000","last_undegraded":"2024-07-17T08:35:35.449659+0000","last_fullsized":"2024-07-17T08:35:35.449659+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:10.418879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552117+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:35.552117+0000","last_peered":"2024-07-17T08:35:35.552117+0000","last_clean":"2024-07-17T08:35:35.552117+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:35.552117+0000","last_undegraded":"2024-07-17T08:35:35.552117+0000","last_fullsized":"2024-07-17T08:35:35.552117+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554848+0000","last_change":"2024-07-17T08:35:34.497991+0000","last_active":"2024-07-17T08:35:35.554848+0000","last_peered":"2024-07-17T08:35:35.554848+0000","last_clean":"2024-07-17T08:35:35.554848+0000","last_became_active":"2024-07-17T08:35:34.491521+0000","last_became_peered":"2024-07-17T08:35:34.491521+0000","last_unstale":"2024-07-17T08:35:35.554848+0000","last_undegraded":"2024-07-17T08:35:35.554848+0000","last_fullsized":"2024-07-17T08:35:35.554848+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:52.030910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450264+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:35.450264+0000","last_peered":"2024-07-17T08:35:35.450264+0000","last_clean":"2024-07-17T08:35:35.450264+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:35.450264+0000","last_undegraded":"2024-07-17T08:35:35.450264+0000","last_fullsized":"2024-07-17T08:35:35.450264+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555288+0000","last_change":"2024-07-17T08:35:34.503875+0000","last_active":"2024-07-17T08:35:35.555288+0000","last_peered":"2024-07-17T08:35:35.555288+0000","last_clean":"2024-07-17T08:35:35.555288+0000","last_became_active":"2024-07-17T08:35:34.503583+0000","last_became_peered":"2024-07-17T08:35:34.503583+0000","last_unstale":"2024-07-17T08:35:35.555288+0000","last_undegraded":"2024-07-17T08:35:35.555288+0000","last_fullsized":"2024-07-17T08:35:35.555288+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:00:33.798525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486757+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:35.486757+0000","last_peered":"2024-07-17T08:35:35.486757+0000","last_clean":"2024-07-17T08:35:35.486757+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:35.486757+0000","last_undegraded":"2024-07-17T08:35:35.486757+0000","last_fullsized":"2024-07-17T08:35:35.486757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461219+0000","last_change":"2024-07-17T08:35:34.471198+0000","last_active":"2024-07-17T08:35:35.461219+0000","last_peered":"2024-07-17T08:35:35.461219+0000","last_clean":"2024-07-17T08:35:35.461219+0000","last_became_active":"2024-07-17T08:35:34.470800+0000","last_became_peered":"2024-07-17T08:35:34.470800+0000","last_unstale":"2024-07-17T08:35:35.461219+0000","last_undegraded":"2024-07-17T08:35:35.461219+0000","last_fullsized":"2024-07-17T08:35:35.461219+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:00.489281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458092+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:35.458092+0000","last_peered":"2024-07-17T08:35:35.458092+0000","last_clean":"2024-07-17T08:35:35.458092+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:35.458092+0000","last_undegraded":"2024-07-17T08:35:35.458092+0000","last_fullsized":"2024-07-17T08:35:35.458092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:37.046641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493189+0000","last_change":"2024-07-17T08:35:34.481733+0000","last_active":"2024-07-17T08:35:35.493189+0000","last_peered":"2024-07-17T08:35:35.493189+0000","last_clean":"2024-07-17T08:35:35.493189+0000","last_became_active":"2024-07-17T08:35:34.481419+0000","last_became_peered":"2024-07-17T08:35:34.481419+0000","last_unstale":"2024-07-17T08:35:35.493189+0000","last_undegraded":"2024-07-17T08:35:35.493189+0000","last_fullsized":"2024-07-17T08:35:35.493189+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:29.214921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555009+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:35.555009+0000","last_peered":"2024-07-17T08:35:35.555009+0000","last_clean":"2024-07-17T08:35:35.555009+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:35.555009+0000","last_undegraded":"2024-07-17T08:35:35.555009+0000","last_fullsized":"2024-07-17T08:35:35.555009+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486172+0000","last_change":"2024-07-17T08:35:34.494972+0000","last_active":"2024-07-17T08:35:35.486172+0000","last_peered":"2024-07-17T08:35:35.486172+0000","last_clean":"2024-07-17T08:35:35.486172+0000","last_became_active":"2024-07-17T08:35:34.493982+0000","last_became_peered":"2024-07-17T08:35:34.493982+0000","last_unstale":"2024-07-17T08:35:35.486172+0000","last_undegraded":"2024-07-17T08:35:35.486172+0000","last_fullsized":"2024-07-17T08:35:35.486172+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:11.025000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449054+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:35.449054+0000","last_peered":"2024-07-17T08:35:35.449054+0000","last_clean":"2024-07-17T08:35:35.449054+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:35.449054+0000","last_undegraded":"2024-07-17T08:35:35.449054+0000","last_fullsized":"2024-07-17T08:35:35.449054+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552737+0000","last_change":"2024-07-17T08:35:34.498453+0000","last_active":"2024-07-17T08:35:35.552737+0000","last_peered":"2024-07-17T08:35:35.552737+0000","last_clean":"2024-07-17T08:35:35.552737+0000","last_became_active":"2024-07-17T08:35:34.498160+0000","last_became_peered":"2024-07-17T08:35:34.498160+0000","last_unstale":"2024-07-17T08:35:35.552737+0000","last_undegraded":"2024-07-17T08:35:35.552737+0000","last_fullsized":"2024-07-17T08:35:35.552737+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:45:36.345027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552918+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:35.552918+0000","last_peered":"2024-07-17T08:35:35.552918+0000","last_clean":"2024-07-17T08:35:35.552918+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:35.552918+0000","last_undegraded":"2024-07-17T08:35:35.552918+0000","last_fullsized":"2024-07-17T08:35:35.552918+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:00.653492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553040+0000","last_change":"2024-07-17T08:35:34.504906+0000","last_active":"2024-07-17T08:35:35.553040+0000","last_peered":"2024-07-17T08:35:35.553040+0000","last_clean":"2024-07-17T08:35:35.553040+0000","last_became_active":"2024-07-17T08:35:34.504288+0000","last_became_peered":"2024-07-17T08:35:34.504288+0000","last_unstale":"2024-07-17T08:35:35.553040+0000","last_undegraded":"2024-07-17T08:35:35.553040+0000","last_fullsized":"2024-07-17T08:35:35.553040+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:19.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459230+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:35.459230+0000","last_peered":"2024-07-17T08:35:35.459230+0000","last_clean":"2024-07-17T08:35:35.459230+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:35.459230+0000","last_undegraded":"2024-07-17T08:35:35.459230+0000","last_fullsized":"2024-07-17T08:35:35.459230+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449377+0000","last_change":"2024-07-17T08:35:34.481895+0000","last_active":"2024-07-17T08:35:35.449377+0000","last_peered":"2024-07-17T08:35:35.449377+0000","last_clean":"2024-07-17T08:35:35.449377+0000","last_became_active":"2024-07-17T08:35:34.481567+0000","last_became_peered":"2024-07-17T08:35:34.481567+0000","last_unstale":"2024-07-17T08:35:35.449377+0000","last_undegraded":"2024-07-17T08:35:35.449377+0000","last_fullsized":"2024-07-17T08:35:35.449377+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:45:18.596493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461653+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:35.461653+0000","last_peered":"2024-07-17T08:35:35.461653+0000","last_clean":"2024-07-17T08:35:35.461653+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:35.461653+0000","last_undegraded":"2024-07-17T08:35:35.461653+0000","last_fullsized":"2024-07-17T08:35:35.461653+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554398+0000","last_change":"2024-07-17T08:35:34.499163+0000","last_active":"2024-07-17T08:35:35.554398+0000","last_peered":"2024-07-17T08:35:35.554398+0000","last_clean":"2024-07-17T08:35:35.554398+0000","last_became_active":"2024-07-17T08:35:34.491959+0000","last_became_peered":"2024-07-17T08:35:34.491959+0000","last_unstale":"2024-07-17T08:35:35.554398+0000","last_undegraded":"2024-07-17T08:35:35.554398+0000","last_fullsized":"2024-07-17T08:35:35.554398+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:22.008659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462086+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:35.462086+0000","last_peered":"2024-07-17T08:35:35.462086+0000","last_clean":"2024-07-17T08:35:35.462086+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:35.462086+0000","last_undegraded":"2024-07-17T08:35:35.462086+0000","last_fullsized":"2024-07-17T08:35:35.462086+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460233+0000","last_change":"2024-07-17T08:35:34.492707+0000","last_active":"2024-07-17T08:35:35.460233+0000","last_peered":"2024-07-17T08:35:35.460233+0000","last_clean":"2024-07-17T08:35:35.460233+0000","last_became_active":"2024-07-17T08:35:34.490372+0000","last_became_peered":"2024-07-17T08:35:34.490372+0000","last_unstale":"2024-07-17T08:35:35.460233+0000","last_undegraded":"2024-07-17T08:35:35.460233+0000","last_fullsized":"2024-07-17T08:35:35.460233+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:39.652406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460378+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:35.460378+0000","last_peered":"2024-07-17T08:35:35.460378+0000","last_clean":"2024-07-17T08:35:35.460378+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:35.460378+0000","last_undegraded":"2024-07-17T08:35:35.460378+0000","last_fullsized":"2024-07-17T08:35:35.460378+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457473+0000","last_change":"2024-07-17T08:35:34.486721+0000","last_active":"2024-07-17T08:35:35.457473+0000","last_peered":"2024-07-17T08:35:35.457473+0000","last_clean":"2024-07-17T08:35:35.457473+0000","last_became_active":"2024-07-17T08:35:34.486259+0000","last_became_peered":"2024-07-17T08:35:34.486259+0000","last_unstale":"2024-07-17T08:35:35.457473+0000","last_undegraded":"2024-07-17T08:35:35.457473+0000","last_fullsized":"2024-07-17T08:35:35.457473+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:27.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553127+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:35.553127+0000","last_peered":"2024-07-17T08:35:35.553127+0000","last_clean":"2024-07-17T08:35:35.553127+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:35.553127+0000","last_undegraded":"2024-07-17T08:35:35.553127+0000","last_fullsized":"2024-07-17T08:35:35.553127+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555034+0000","last_change":"2024-07-17T08:35:34.499538+0000","last_active":"2024-07-17T08:35:35.555034+0000","last_peered":"2024-07-17T08:35:35.555034+0000","last_clean":"2024-07-17T08:35:35.555034+0000","last_became_active":"2024-07-17T08:35:34.499236+0000","last_became_peered":"2024-07-17T08:35:34.499236+0000","last_unstale":"2024-07-17T08:35:35.555034+0000","last_undegraded":"2024-07-17T08:35:35.555034+0000","last_fullsized":"2024-07-17T08:35:35.555034+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:55:40.339503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554755+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:35.554755+0000","last_peered":"2024-07-17T08:35:35.554755+0000","last_clean":"2024-07-17T08:35:35.554755+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:35.554755+0000","last_undegraded":"2024-07-17T08:35:35.554755+0000","last_fullsized":"2024-07-17T08:35:35.554755+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458749+0000","last_change":"2024-07-17T08:35:34.484162+0000","last_active":"2024-07-17T08:35:35.458749+0000","last_peered":"2024-07-17T08:35:35.458749+0000","last_clean":"2024-07-17T08:35:35.458749+0000","last_became_active":"2024-07-17T08:35:34.483488+0000","last_became_peered":"2024-07-17T08:35:34.483488+0000","last_unstale":"2024-07-17T08:35:35.458749+0000","last_undegraded":"2024-07-17T08:35:35.458749+0000","last_fullsized":"2024-07-17T08:35:35.458749+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:42.661226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485758+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:35.485758+0000","last_peered":"2024-07-17T08:35:35.485758+0000","last_clean":"2024-07-17T08:35:35.485758+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:35.485758+0000","last_undegraded":"2024-07-17T08:35:35.485758+0000","last_fullsized":"2024-07-17T08:35:35.485758+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456828+0000","last_change":"2024-07-17T08:35:34.485105+0000","last_active":"2024-07-17T08:35:35.456828+0000","last_peered":"2024-07-17T08:35:35.456828+0000","last_clean":"2024-07-17T08:35:35.456828+0000","last_became_active":"2024-07-17T08:35:34.484821+0000","last_became_peered":"2024-07-17T08:35:34.484821+0000","last_unstale":"2024-07-17T08:35:35.456828+0000","last_undegraded":"2024-07-17T08:35:35.456828+0000","last_fullsized":"2024-07-17T08:35:35.456828+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:16.609572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492853+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:35.492853+0000","last_peered":"2024-07-17T08:35:35.492853+0000","last_clean":"2024-07-17T08:35:35.492853+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:35.492853+0000","last_undegraded":"2024-07-17T08:35:35.492853+0000","last_fullsized":"2024-07-17T08:35:35.492853+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:06.797667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454814+0000","last_change":"2024-07-17T08:35:34.488768+0000","last_active":"2024-07-17T08:35:35.454814+0000","last_peered":"2024-07-17T08:35:35.454814+0000","last_clean":"2024-07-17T08:35:35.454814+0000","last_became_active":"2024-07-17T08:35:34.488485+0000","last_became_peered":"2024-07-17T08:35:34.488485+0000","last_unstale":"2024-07-17T08:35:35.454814+0000","last_undegraded":"2024-07-17T08:35:35.454814+0000","last_fullsized":"2024-07-17T08:35:35.454814+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:02.497512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458986+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:35.458986+0000","last_peered":"2024-07-17T08:35:35.458986+0000","last_clean":"2024-07-17T08:35:35.458986+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:35.458986+0000","last_undegraded":"2024-07-17T08:35:35.458986+0000","last_fullsized":"2024-07-17T08:35:35.458986+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485819+0000","last_change":"2024-07-17T08:35:34.490716+0000","last_active":"2024-07-17T08:35:35.485819+0000","last_peered":"2024-07-17T08:35:35.485819+0000","last_clean":"2024-07-17T08:35:35.485819+0000","last_became_active":"2024-07-17T08:35:34.490402+0000","last_became_peered":"2024-07-17T08:35:34.490402+0000","last_unstale":"2024-07-17T08:35:35.485819+0000","last_undegraded":"2024-07-17T08:35:35.485819+0000","last_fullsized":"2024-07-17T08:35:35.485819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:49.544177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554175+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:35.554175+0000","last_peered":"2024-07-17T08:35:35.554175+0000","last_clean":"2024-07-17T08:35:35.554175+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:35.554175+0000","last_undegraded":"2024-07-17T08:35:35.554175+0000","last_fullsized":"2024-07-17T08:35:35.554175+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:41.247825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552756+0000","last_change":"2024-07-17T08:35:34.498232+0000","last_active":"2024-07-17T08:35:35.552756+0000","last_peered":"2024-07-17T08:35:35.552756+0000","last_clean":"2024-07-17T08:35:35.552756+0000","last_became_active":"2024-07-17T08:35:34.492256+0000","last_became_peered":"2024-07-17T08:35:34.492256+0000","last_unstale":"2024-07-17T08:35:35.552756+0000","last_undegraded":"2024-07-17T08:35:35.552756+0000","last_fullsized":"2024-07-17T08:35:35.552756+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:44.693413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450194+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:35.450194+0000","last_peered":"2024-07-17T08:35:35.450194+0000","last_clean":"2024-07-17T08:35:35.450194+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:35.450194+0000","last_undegraded":"2024-07-17T08:35:35.450194+0000","last_fullsized":"2024-07-17T08:35:35.450194+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450743+0000","last_change":"2024-07-17T08:35:34.484320+0000","last_active":"2024-07-17T08:35:35.450743+0000","last_peered":"2024-07-17T08:35:35.450743+0000","last_clean":"2024-07-17T08:35:35.450743+0000","last_became_active":"2024-07-17T08:35:34.483670+0000","last_became_peered":"2024-07-17T08:35:34.483670+0000","last_unstale":"2024-07-17T08:35:35.450743+0000","last_undegraded":"2024-07-17T08:35:35.450743+0000","last_fullsized":"2024-07-17T08:35:35.450743+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:31:34.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486717+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:35.486717+0000","last_peered":"2024-07-17T08:35:35.486717+0000","last_clean":"2024-07-17T08:35:35.486717+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:35.486717+0000","last_undegraded":"2024-07-17T08:35:35.486717+0000","last_fullsized":"2024-07-17T08:35:35.486717+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460839+0000","last_change":"2024-07-17T08:35:34.489939+0000","last_active":"2024-07-17T08:35:35.460839+0000","last_peered":"2024-07-17T08:35:35.460839+0000","last_clean":"2024-07-17T08:35:35.460839+0000","last_became_active":"2024-07-17T08:35:34.489661+0000","last_became_peered":"2024-07-17T08:35:34.489661+0000","last_unstale":"2024-07-17T08:35:35.460839+0000","last_undegraded":"2024-07-17T08:35:35.460839+0000","last_fullsized":"2024-07-17T08:35:35.460839+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:47.521931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460334+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:35.460334+0000","last_peered":"2024-07-17T08:35:35.460334+0000","last_clean":"2024-07-17T08:35:35.460334+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:35.460334+0000","last_undegraded":"2024-07-17T08:35:35.460334+0000","last_fullsized":"2024-07-17T08:35:35.460334+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493380+0000","last_change":"2024-07-17T08:35:34.490263+0000","last_active":"2024-07-17T08:35:35.493380+0000","last_peered":"2024-07-17T08:35:35.493380+0000","last_clean":"2024-07-17T08:35:35.493380+0000","last_became_active":"2024-07-17T08:35:34.489975+0000","last_became_peered":"2024-07-17T08:35:34.489975+0000","last_unstale":"2024-07-17T08:35:35.493380+0000","last_undegraded":"2024-07-17T08:35:35.493380+0000","last_fullsized":"2024-07-17T08:35:35.493380+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:06.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554243+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:35.554243+0000","last_peered":"2024-07-17T08:35:35.554243+0000","last_clean":"2024-07-17T08:35:35.554243+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:35.554243+0000","last_undegraded":"2024-07-17T08:35:35.554243+0000","last_fullsized":"2024-07-17T08:35:35.554243+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458638+0000","last_change":"2024-07-17T08:35:34.492433+0000","last_active":"2024-07-17T08:35:35.458638+0000","last_peered":"2024-07-17T08:35:35.458638+0000","last_clean":"2024-07-17T08:35:35.458638+0000","last_became_active":"2024-07-17T08:35:34.490894+0000","last_became_peered":"2024-07-17T08:35:34.490894+0000","last_unstale":"2024-07-17T08:35:35.458638+0000","last_undegraded":"2024-07-17T08:35:35.458638+0000","last_fullsized":"2024-07-17T08:35:35.458638+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:47.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456454+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:35.456454+0000","last_peered":"2024-07-17T08:35:35.456454+0000","last_clean":"2024-07-17T08:35:35.456454+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:35.456454+0000","last_undegraded":"2024-07-17T08:35:35.456454+0000","last_fullsized":"2024-07-17T08:35:35.456454+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553404+0000","last_change":"2024-07-17T08:35:34.488358+0000","last_active":"2024-07-17T08:35:35.553404+0000","last_peered":"2024-07-17T08:35:35.553404+0000","last_clean":"2024-07-17T08:35:35.553404+0000","last_became_active":"2024-07-17T08:35:34.488076+0000","last_became_peered":"2024-07-17T08:35:34.488076+0000","last_unstale":"2024-07-17T08:35:35.553404+0000","last_undegraded":"2024-07-17T08:35:35.553404+0000","last_fullsized":"2024-07-17T08:35:35.553404+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449620+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:35.449620+0000","last_peered":"2024-07-17T08:35:35.449620+0000","last_clean":"2024-07-17T08:35:35.449620+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:35.449620+0000","last_undegraded":"2024-07-17T08:35:35.449620+0000","last_fullsized":"2024-07-17T08:35:35.449620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:34.483905+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:34.481531+0000","last_became_peered":"2024-07-17T08:35:34.481531+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:25.813209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553820+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:35.553820+0000","last_peered":"2024-07-17T08:35:35.553820+0000","last_clean":"2024-07-17T08:35:35.553820+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:35.553820+0000","last_undegraded":"2024-07-17T08:35:35.553820+0000","last_fullsized":"2024-07-17T08:35:35.553820+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:52.497030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552031+0000","last_change":"2024-07-17T08:35:34.495772+0000","last_active":"2024-07-17T08:35:35.552031+0000","last_peered":"2024-07-17T08:35:35.552031+0000","last_clean":"2024-07-17T08:35:35.552031+0000","last_became_active":"2024-07-17T08:35:34.495454+0000","last_became_peered":"2024-07-17T08:35:34.495454+0000","last_unstale":"2024-07-17T08:35:35.552031+0000","last_undegraded":"2024-07-17T08:35:35.552031+0000","last_fullsized":"2024-07-17T08:35:35.552031+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:23.394746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450049+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:35.450049+0000","last_peered":"2024-07-17T08:35:35.450049+0000","last_clean":"2024-07-17T08:35:35.450049+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:35.450049+0000","last_undegraded":"2024-07-17T08:35:35.450049+0000","last_fullsized":"2024-07-17T08:35:35.450049+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552363+0000","last_change":"2024-07-17T08:35:34.492325+0000","last_active":"2024-07-17T08:35:35.552363+0000","last_peered":"2024-07-17T08:35:35.552363+0000","last_clean":"2024-07-17T08:35:35.552363+0000","last_became_active":"2024-07-17T08:35:34.491403+0000","last_became_peered":"2024-07-17T08:35:34.491403+0000","last_unstale":"2024-07-17T08:35:35.552363+0000","last_undegraded":"2024-07-17T08:35:35.552363+0000","last_fullsized":"2024-07-17T08:35:35.552363+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:42:09.402312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554043+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:35.554043+0000","last_peered":"2024-07-17T08:35:35.554043+0000","last_clean":"2024-07-17T08:35:35.554043+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:35.554043+0000","last_undegraded":"2024-07-17T08:35:35.554043+0000","last_fullsized":"2024-07-17T08:35:35.554043+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:34.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555266+0000","last_change":"2024-07-17T08:35:34.494219+0000","last_active":"2024-07-17T08:35:35.555266+0000","last_peered":"2024-07-17T08:35:35.555266+0000","last_clean":"2024-07-17T08:35:35.555266+0000","last_became_active":"2024-07-17T08:35:34.493904+0000","last_became_peered":"2024-07-17T08:35:34.493904+0000","last_unstale":"2024-07-17T08:35:35.555266+0000","last_undegraded":"2024-07-17T08:35:35.555266+0000","last_fullsized":"2024-07-17T08:35:35.555266+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:11.039773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486687+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:35.486687+0000","last_peered":"2024-07-17T08:35:35.486687+0000","last_clean":"2024-07-17T08:35:35.486687+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:35.486687+0000","last_undegraded":"2024-07-17T08:35:35.486687+0000","last_fullsized":"2024-07-17T08:35:35.486687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492520+0000","last_change":"2024-07-17T08:35:34.489926+0000","last_active":"2024-07-17T08:35:35.492520+0000","last_peered":"2024-07-17T08:35:35.492520+0000","last_clean":"2024-07-17T08:35:35.492520+0000","last_became_active":"2024-07-17T08:35:34.489578+0000","last_became_peered":"2024-07-17T08:35:34.489578+0000","last_unstale":"2024-07-17T08:35:35.492520+0000","last_undegraded":"2024-07-17T08:35:35.492520+0000","last_fullsized":"2024-07-17T08:35:35.492520+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:52.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555446+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:35.555446+0000","last_peered":"2024-07-17T08:35:35.555446+0000","last_clean":"2024-07-17T08:35:35.555446+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:35.555446+0000","last_undegraded":"2024-07-17T08:35:35.555446+0000","last_fullsized":"2024-07-17T08:35:35.555446+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:11.863693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554183+0000","last_change":"2024-07-17T08:35:34.489451+0000","last_active":"2024-07-17T08:35:35.554183+0000","last_peered":"2024-07-17T08:35:35.554183+0000","last_clean":"2024-07-17T08:35:35.554183+0000","last_became_active":"2024-07-17T08:35:34.489177+0000","last_became_peered":"2024-07-17T08:35:34.489177+0000","last_unstale":"2024-07-17T08:35:35.554183+0000","last_undegraded":"2024-07-17T08:35:35.554183+0000","last_fullsized":"2024-07-17T08:35:35.554183+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:41.289199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458899+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:35.458899+0000","last_peered":"2024-07-17T08:35:35.458899+0000","last_clean":"2024-07-17T08:35:35.458899+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:35.458899+0000","last_undegraded":"2024-07-17T08:35:35.458899+0000","last_fullsized":"2024-07-17T08:35:35.458899+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:22.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543345,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65088,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306752,"statfs":{"total":96636764160,"available":96570114048,"internally_reserved":0,"allocated":3461120,"data_stored":2145831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510640,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29844,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341996,"statfs":{"total":96636764160,"available":96606203904,"internally_reserved":0,"allocated":2883584,"data_stored":1557041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510641,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1553601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543346,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2850816,"data_stored":1552535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510640,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52028,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319812,"statfs":{"total":96636764160,"available":96583487488,"internally_reserved":0,"allocated":2859008,"data_stored":1552011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543344,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2854912,"data_stored":1552569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510641,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2842624,"data_stored":1551863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56756,"kb_used_data":3360,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315084,"statfs":{"total":96636764160,"available":96578646016,"internally_reserved":0,"allocated":3440640,"data_stored":2142321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:44.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:35:44.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 -- 172.21.15.44:0/3693470173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 msgr2=0x7feb88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 --2- 172.21.15.44:0/3693470173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feb88004c80 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00c4a0 tx=0x7feb88005830 comp rx=0 tx=0).stop 2024-07-17T08:35:44.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7feba9a00640 1 -- 172.21.15.44:0/3693470173 reap_dead start 2024-07-17T08:35:44.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 -- 172.21.15.44:0/3693470173 shutdown_connections 2024-07-17T08:35:44.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 -- 172.21.15.44:0/3693470173 >> 172.21.15.44:0/3693470173 conn(0x7feba40b0fb0 msgr2=0x7feba40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:44.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 -- 172.21.15.44:0/3693470173 shutdown_connections 2024-07-17T08:35:44.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.300+0000 7febaba00640 1 -- 172.21.15.44:0/3693470173 wait complete. 2024-07-17T08:35:44.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.302+0000 7febaba00640 1 Processor -- start 2024-07-17T08:35:44.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.302+0000 7febaba00640 1 -- start start 2024-07-17T08:35:44.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 0x7feba41f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba41f46c0 0x7feba41f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feba4153bd0 con 0x7feb88004800 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feba4153a50 con 0x7feba41f46c0 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7febaba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feba4153d50 con 0x7feba41f6ef0 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.303+0000 7feba9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39770/0 (socket says 172.21.15.44:39770) 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 -- 172.21.15.44:0/3765248108 learned_addr learned my addr 172.21.15.44:0/3765248108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba3e00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 0x7feba41f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 -- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 msgr2=0x7feba41f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 0x7feba41f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 -- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba41f46c0 msgr2=0x7feba41f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba41f46c0 0x7feba41f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 -- 172.21.15.44:0/3765248108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40748b0 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba3e00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 0x7feba41f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.304+0000 7feba9000640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7feb9400eab0 tx=0x7feb9400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.305+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9400cde0 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.305+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb94004b20 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.305+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba4072160 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.305+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94005020 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.305+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba4072720 con 0x7feb88004800 2024-07-17T08:35:44.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.306+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba415a250 con 0x7feb88004800 2024-07-17T08:35:44.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.307+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7feb940051c0 con 0x7feb88004800 2024-07-17T08:35:44.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.309+0000 7feba0c00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 0x7feb80079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.309+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7feb94099840 con 0x7feb88004800 2024-07-17T08:35:44.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.310+0000 7feba3e00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 0x7feb80079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.311+0000 7feba3e00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 0x7feb80079190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb8c033000 tx=0x7feb8c001c30 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:44.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.313+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feb94016030 con 0x7feb88004800 2024-07-17T08:35:44.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.519+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feba4158cc0 con 0x7feb80076cd0 2024-07-17T08:35:44.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.545+0000 7feba0c00640 1 -- 172.21.15.44:0/3765248108 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491458 (secure 0 0 0) 0x7feb8c0b9fe0 con 0x7feb80076cd0 2024-07-17T08:35:44.551 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:44.560 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:35:44.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 msgr2=0x7feb80079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 0x7feb80079190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb8c033000 tx=0x7feb8c001c30 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 msgr2=0x7feba4074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7feb9400eab0 tx=0x7feb9400ef70 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 shutdown_connections 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7feb80076cd0 0x7feb80079190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feba41f46c0 0x7feba41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feba41f6ef0 0x7feba41f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 --2- 172.21.15.44:0/3765248108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb88004800 0x7feba4074370 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 >> 172.21.15.44:0/3765248108 conn(0x7feba40b0fb0 msgr2=0x7feba415b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.557+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 shutdown_connections 2024-07-17T08:35:44.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.558+0000 7febaba00640 1 -- 172.21.15.44:0/3765248108 wait complete. 2024-07-17T08:35:44.584 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-07-17T08:35:43.768463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596434,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":451,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2591,"ondisk_log_size":2591,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":399316,"kb_used_data":23496,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754575404,"statfs":{"total":773094113280,"available":772685213696,"internally_reserved":0,"allocated":24059904,"data_stored":13607772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.322724"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554981+0000","last_change":"2024-07-17T08:35:34.486327+0000","last_active":"2024-07-17T08:35:35.554981+0000","last_peered":"2024-07-17T08:35:35.554981+0000","last_clean":"2024-07-17T08:35:35.554981+0000","last_became_active":"2024-07-17T08:35:34.486069+0000","last_became_peered":"2024-07-17T08:35:34.486069+0000","last_unstale":"2024-07-17T08:35:35.554981+0000","last_undegraded":"2024-07-17T08:35:35.554981+0000","last_fullsized":"2024-07-17T08:35:35.554981+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:20.240831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"212'2511","reported_seq":3386,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2511,"log_dups_size":0,"ondisk_log_size":2511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553610+0000","last_change":"2024-07-17T08:35:21.307737+0000","last_active":"2024-07-17T08:35:35.553610+0000","last_peered":"2024-07-17T08:35:35.553610+0000","last_clean":"2024-07-17T08:35:35.553610+0000","last_became_active":"2024-07-17T08:35:21.307456+0000","last_became_peered":"2024-07-17T08:35:21.307456+0000","last_unstale":"2024-07-17T08:35:35.553610+0000","last_undegraded":"2024-07-17T08:35:35.553610+0000","last_fullsized":"2024-07-17T08:35:35.553610+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:49.547967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457819+0000","last_change":"2024-07-17T08:35:22.335722+0000","last_active":"2024-07-17T08:35:35.457819+0000","last_peered":"2024-07-17T08:35:35.457819+0000","last_clean":"2024-07-17T08:35:35.457819+0000","last_became_active":"2024-07-17T08:35:22.334572+0000","last_became_peered":"2024-07-17T08:35:22.334572+0000","last_unstale":"2024-07-17T08:35:35.457819+0000","last_undegraded":"2024-07-17T08:35:35.457819+0000","last_fullsized":"2024-07-17T08:35:35.457819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:44.992431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553498+0000","last_change":"2024-07-17T08:35:33.455926+0000","last_active":"2024-07-17T08:35:35.553498+0000","last_peered":"2024-07-17T08:35:35.553498+0000","last_clean":"2024-07-17T08:35:35.553498+0000","last_became_active":"2024-07-17T08:35:33.455626+0000","last_became_peered":"2024-07-17T08:35:33.455626+0000","last_unstale":"2024-07-17T08:35:35.553498+0000","last_undegraded":"2024-07-17T08:35:35.553498+0000","last_fullsized":"2024-07-17T08:35:35.553498+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:18.891188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554140+0000","last_change":"2024-07-17T08:35:34.499379+0000","last_active":"2024-07-17T08:35:35.554140+0000","last_peered":"2024-07-17T08:35:35.554140+0000","last_clean":"2024-07-17T08:35:35.554140+0000","last_became_active":"2024-07-17T08:35:34.498469+0000","last_became_peered":"2024-07-17T08:35:34.498469+0000","last_unstale":"2024-07-17T08:35:35.554140+0000","last_undegraded":"2024-07-17T08:35:35.554140+0000","last_fullsized":"2024-07-17T08:35:35.554140+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:18.359384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449166+0000","last_change":"2024-07-17T08:35:21.300946+0000","last_active":"2024-07-17T08:35:35.449166+0000","last_peered":"2024-07-17T08:35:35.449166+0000","last_clean":"2024-07-17T08:35:35.449166+0000","last_became_active":"2024-07-17T08:35:21.296706+0000","last_became_peered":"2024-07-17T08:35:21.296706+0000","last_unstale":"2024-07-17T08:35:35.449166+0000","last_undegraded":"2024-07-17T08:35:35.449166+0000","last_fullsized":"2024-07-17T08:35:35.449166+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:55.321074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.487031+0000","last_change":"2024-07-17T08:35:22.336599+0000","last_active":"2024-07-17T08:35:35.487031+0000","last_peered":"2024-07-17T08:35:35.487031+0000","last_clean":"2024-07-17T08:35:35.487031+0000","last_became_active":"2024-07-17T08:35:22.336309+0000","last_became_peered":"2024-07-17T08:35:22.336309+0000","last_unstale":"2024-07-17T08:35:35.487031+0000","last_undegraded":"2024-07-17T08:35:35.487031+0000","last_fullsized":"2024-07-17T08:35:35.487031+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:27:23.337295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"227'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486853+0000","last_change":"2024-07-17T08:35:33.451371+0000","last_active":"2024-07-17T08:35:35.486853+0000","last_peered":"2024-07-17T08:35:35.486853+0000","last_clean":"2024-07-17T08:35:35.486853+0000","last_became_active":"2024-07-17T08:35:33.451107+0000","last_became_peered":"2024-07-17T08:35:33.451107+0000","last_unstale":"2024-07-17T08:35:35.486853+0000","last_undegraded":"2024-07-17T08:35:35.486853+0000","last_fullsized":"2024-07-17T08:35:35.486853+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:15.595003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553500+0000","last_change":"2024-07-17T08:35:34.480720+0000","last_active":"2024-07-17T08:35:35.553500+0000","last_peered":"2024-07-17T08:35:35.553500+0000","last_clean":"2024-07-17T08:35:35.553500+0000","last_became_active":"2024-07-17T08:35:34.480430+0000","last_became_peered":"2024-07-17T08:35:34.480430+0000","last_unstale":"2024-07-17T08:35:35.553500+0000","last_undegraded":"2024-07-17T08:35:35.553500+0000","last_fullsized":"2024-07-17T08:35:35.553500+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:08.457362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456016+0000","last_change":"2024-07-17T08:35:21.319884+0000","last_active":"2024-07-17T08:35:35.456016+0000","last_peered":"2024-07-17T08:35:35.456016+0000","last_clean":"2024-07-17T08:35:35.456016+0000","last_became_active":"2024-07-17T08:35:21.318930+0000","last_became_peered":"2024-07-17T08:35:21.318930+0000","last_unstale":"2024-07-17T08:35:35.456016+0000","last_undegraded":"2024-07-17T08:35:35.456016+0000","last_fullsized":"2024-07-17T08:35:35.456016+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:29.495739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555195+0000","last_change":"2024-07-17T08:35:22.325415+0000","last_active":"2024-07-17T08:35:35.555195+0000","last_peered":"2024-07-17T08:35:35.555195+0000","last_clean":"2024-07-17T08:35:35.555195+0000","last_became_active":"2024-07-17T08:35:22.324765+0000","last_became_peered":"2024-07-17T08:35:22.324765+0000","last_unstale":"2024-07-17T08:35:35.555195+0000","last_undegraded":"2024-07-17T08:35:35.555195+0000","last_fullsized":"2024-07-17T08:35:35.555195+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:08.117588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460518+0000","last_change":"2024-07-17T08:35:33.459624+0000","last_active":"2024-07-17T08:35:35.460518+0000","last_peered":"2024-07-17T08:35:35.460518+0000","last_clean":"2024-07-17T08:35:35.460518+0000","last_became_active":"2024-07-17T08:35:33.459359+0000","last_became_peered":"2024-07-17T08:35:33.459359+0000","last_unstale":"2024-07-17T08:35:35.460518+0000","last_undegraded":"2024-07-17T08:35:35.460518+0000","last_fullsized":"2024-07-17T08:35:35.460518+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:16:23.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451151+0000","last_change":"2024-07-17T08:35:34.474086+0000","last_active":"2024-07-17T08:35:35.451151+0000","last_peered":"2024-07-17T08:35:35.451151+0000","last_clean":"2024-07-17T08:35:35.451151+0000","last_became_active":"2024-07-17T08:35:34.473771+0000","last_became_peered":"2024-07-17T08:35:34.473771+0000","last_unstale":"2024-07-17T08:35:35.451151+0000","last_undegraded":"2024-07-17T08:35:35.451151+0000","last_fullsized":"2024-07-17T08:35:35.451151+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:12.038298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553735+0000","last_change":"2024-07-17T08:35:21.312211+0000","last_active":"2024-07-17T08:35:35.553735+0000","last_peered":"2024-07-17T08:35:35.553735+0000","last_clean":"2024-07-17T08:35:35.553735+0000","last_became_active":"2024-07-17T08:35:21.311667+0000","last_became_peered":"2024-07-17T08:35:21.311667+0000","last_unstale":"2024-07-17T08:35:35.553735+0000","last_undegraded":"2024-07-17T08:35:35.553735+0000","last_fullsized":"2024-07-17T08:35:35.553735+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:51.231358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554875+0000","last_change":"2024-07-17T08:35:22.328800+0000","last_active":"2024-07-17T08:35:35.554875+0000","last_peered":"2024-07-17T08:35:35.554875+0000","last_clean":"2024-07-17T08:35:35.554875+0000","last_became_active":"2024-07-17T08:35:22.327847+0000","last_became_peered":"2024-07-17T08:35:22.327847+0000","last_unstale":"2024-07-17T08:35:35.554875+0000","last_undegraded":"2024-07-17T08:35:35.554875+0000","last_fullsized":"2024-07-17T08:35:35.554875+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:20.349285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461693+0000","last_change":"2024-07-17T08:35:33.457849+0000","last_active":"2024-07-17T08:35:35.461693+0000","last_peered":"2024-07-17T08:35:35.461693+0000","last_clean":"2024-07-17T08:35:35.461693+0000","last_became_active":"2024-07-17T08:35:33.453123+0000","last_became_peered":"2024-07-17T08:35:33.453123+0000","last_unstale":"2024-07-17T08:35:35.461693+0000","last_undegraded":"2024-07-17T08:35:35.461693+0000","last_fullsized":"2024-07-17T08:35:35.461693+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:56:02.265477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457387+0000","last_change":"2024-07-17T08:35:34.485825+0000","last_active":"2024-07-17T08:35:35.457387+0000","last_peered":"2024-07-17T08:35:35.457387+0000","last_clean":"2024-07-17T08:35:35.457387+0000","last_became_active":"2024-07-17T08:35:34.485520+0000","last_became_peered":"2024-07-17T08:35:34.485520+0000","last_unstale":"2024-07-17T08:35:35.457387+0000","last_undegraded":"2024-07-17T08:35:35.457387+0000","last_fullsized":"2024-07-17T08:35:35.457387+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:39.903925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460664+0000","last_change":"2024-07-17T08:35:21.302094+0000","last_active":"2024-07-17T08:35:35.460664+0000","last_peered":"2024-07-17T08:35:35.460664+0000","last_clean":"2024-07-17T08:35:35.460664+0000","last_became_active":"2024-07-17T08:35:21.301525+0000","last_became_peered":"2024-07-17T08:35:21.301525+0000","last_unstale":"2024-07-17T08:35:35.460664+0000","last_undegraded":"2024-07-17T08:35:35.460664+0000","last_fullsized":"2024-07-17T08:35:35.460664+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:17.583166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462215+0000","last_change":"2024-07-17T08:35:22.329411+0000","last_active":"2024-07-17T08:35:35.462215+0000","last_peered":"2024-07-17T08:35:35.462215+0000","last_clean":"2024-07-17T08:35:35.462215+0000","last_became_active":"2024-07-17T08:35:22.327651+0000","last_became_peered":"2024-07-17T08:35:22.327651+0000","last_unstale":"2024-07-17T08:35:35.462215+0000","last_undegraded":"2024-07-17T08:35:35.462215+0000","last_fullsized":"2024-07-17T08:35:35.462215+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:37.270119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486627+0000","last_change":"2024-07-17T08:35:33.452063+0000","last_active":"2024-07-17T08:35:35.486627+0000","last_peered":"2024-07-17T08:35:35.486627+0000","last_clean":"2024-07-17T08:35:35.486627+0000","last_became_active":"2024-07-17T08:35:33.450965+0000","last_became_peered":"2024-07-17T08:35:33.450965+0000","last_unstale":"2024-07-17T08:35:35.486627+0000","last_undegraded":"2024-07-17T08:35:35.486627+0000","last_fullsized":"2024-07-17T08:35:35.486627+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:01.482943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552456+0000","last_change":"2024-07-17T08:35:34.485471+0000","last_active":"2024-07-17T08:35:35.552456+0000","last_peered":"2024-07-17T08:35:35.552456+0000","last_clean":"2024-07-17T08:35:35.552456+0000","last_became_active":"2024-07-17T08:35:34.485180+0000","last_became_peered":"2024-07-17T08:35:34.485180+0000","last_unstale":"2024-07-17T08:35:35.552456+0000","last_undegraded":"2024-07-17T08:35:35.552456+0000","last_fullsized":"2024-07-17T08:35:35.552456+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:52.825712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458117+0000","last_change":"2024-07-17T08:35:21.312506+0000","last_active":"2024-07-17T08:35:35.458117+0000","last_peered":"2024-07-17T08:35:35.458117+0000","last_clean":"2024-07-17T08:35:35.458117+0000","last_became_active":"2024-07-17T08:35:21.310774+0000","last_became_peered":"2024-07-17T08:35:21.310774+0000","last_unstale":"2024-07-17T08:35:35.458117+0000","last_undegraded":"2024-07-17T08:35:35.458117+0000","last_fullsized":"2024-07-17T08:35:35.458117+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:35.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457075+0000","last_change":"2024-07-17T08:35:22.321270+0000","last_active":"2024-07-17T08:35:35.457075+0000","last_peered":"2024-07-17T08:35:35.457075+0000","last_clean":"2024-07-17T08:35:35.457075+0000","last_became_active":"2024-07-17T08:35:22.320558+0000","last_became_peered":"2024-07-17T08:35:22.320558+0000","last_unstale":"2024-07-17T08:35:35.457075+0000","last_undegraded":"2024-07-17T08:35:35.457075+0000","last_fullsized":"2024-07-17T08:35:35.457075+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:02:32.339990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491696+0000","last_change":"2024-07-17T08:35:33.443001+0000","last_active":"2024-07-17T08:35:35.491696+0000","last_peered":"2024-07-17T08:35:35.491696+0000","last_clean":"2024-07-17T08:35:35.491696+0000","last_became_active":"2024-07-17T08:35:33.442630+0000","last_became_peered":"2024-07-17T08:35:33.442630+0000","last_unstale":"2024-07-17T08:35:35.491696+0000","last_undegraded":"2024-07-17T08:35:35.491696+0000","last_fullsized":"2024-07-17T08:35:35.491696+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:44.333702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552490+0000","last_change":"2024-07-17T08:35:34.484841+0000","last_active":"2024-07-17T08:35:35.552490+0000","last_peered":"2024-07-17T08:35:35.552490+0000","last_clean":"2024-07-17T08:35:35.552490+0000","last_became_active":"2024-07-17T08:35:34.484580+0000","last_became_peered":"2024-07-17T08:35:34.484580+0000","last_unstale":"2024-07-17T08:35:35.552490+0000","last_undegraded":"2024-07-17T08:35:35.552490+0000","last_fullsized":"2024-07-17T08:35:35.552490+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:20:14.336249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493351+0000","last_change":"2024-07-17T08:35:21.317033+0000","last_active":"2024-07-17T08:35:35.493351+0000","last_peered":"2024-07-17T08:35:35.493351+0000","last_clean":"2024-07-17T08:35:35.493351+0000","last_became_active":"2024-07-17T08:35:21.316701+0000","last_became_peered":"2024-07-17T08:35:21.316701+0000","last_unstale":"2024-07-17T08:35:35.493351+0000","last_undegraded":"2024-07-17T08:35:35.493351+0000","last_fullsized":"2024-07-17T08:35:35.493351+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.012371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553061+0000","last_change":"2024-07-17T08:35:22.314891+0000","last_active":"2024-07-17T08:35:35.553061+0000","last_peered":"2024-07-17T08:35:35.553061+0000","last_clean":"2024-07-17T08:35:35.553061+0000","last_became_active":"2024-07-17T08:35:22.314531+0000","last_became_peered":"2024-07-17T08:35:22.314531+0000","last_unstale":"2024-07-17T08:35:35.553061+0000","last_undegraded":"2024-07-17T08:35:35.553061+0000","last_fullsized":"2024-07-17T08:35:35.553061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:35.730846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459525+0000","last_change":"2024-07-17T08:35:33.447597+0000","last_active":"2024-07-17T08:35:35.459525+0000","last_peered":"2024-07-17T08:35:35.459525+0000","last_clean":"2024-07-17T08:35:35.459525+0000","last_became_active":"2024-07-17T08:35:33.446794+0000","last_became_peered":"2024-07-17T08:35:33.446794+0000","last_unstale":"2024-07-17T08:35:35.459525+0000","last_undegraded":"2024-07-17T08:35:35.459525+0000","last_fullsized":"2024-07-17T08:35:35.459525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:26.440472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459908+0000","last_change":"2024-07-17T08:35:34.476099+0000","last_active":"2024-07-17T08:35:35.459908+0000","last_peered":"2024-07-17T08:35:35.459908+0000","last_clean":"2024-07-17T08:35:35.459908+0000","last_became_active":"2024-07-17T08:35:34.475823+0000","last_became_peered":"2024-07-17T08:35:34.475823+0000","last_unstale":"2024-07-17T08:35:35.459908+0000","last_undegraded":"2024-07-17T08:35:35.459908+0000","last_fullsized":"2024-07-17T08:35:35.459908+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:00.486944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"223'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460588+0000","last_change":"2024-07-17T08:35:21.314364+0000","last_active":"2024-07-17T08:35:35.460588+0000","last_peered":"2024-07-17T08:35:35.460588+0000","last_clean":"2024-07-17T08:35:35.460588+0000","last_became_active":"2024-07-17T08:35:21.311480+0000","last_became_peered":"2024-07-17T08:35:21.311480+0000","last_unstale":"2024-07-17T08:35:35.460588+0000","last_undegraded":"2024-07-17T08:35:35.460588+0000","last_fullsized":"2024-07-17T08:35:35.460588+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:14.718622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486047+0000","last_change":"2024-07-17T08:35:22.330512+0000","last_active":"2024-07-17T08:35:35.486047+0000","last_peered":"2024-07-17T08:35:35.486047+0000","last_clean":"2024-07-17T08:35:35.486047+0000","last_became_active":"2024-07-17T08:35:22.330260+0000","last_became_peered":"2024-07-17T08:35:22.330260+0000","last_unstale":"2024-07-17T08:35:35.486047+0000","last_undegraded":"2024-07-17T08:35:35.486047+0000","last_fullsized":"2024-07-17T08:35:35.486047+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:43:23.752644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455982+0000","last_change":"2024-07-17T08:35:33.461285+0000","last_active":"2024-07-17T08:35:35.455982+0000","last_peered":"2024-07-17T08:35:35.455982+0000","last_clean":"2024-07-17T08:35:35.455982+0000","last_became_active":"2024-07-17T08:35:33.460353+0000","last_became_peered":"2024-07-17T08:35:33.460353+0000","last_unstale":"2024-07-17T08:35:35.455982+0000","last_undegraded":"2024-07-17T08:35:35.455982+0000","last_fullsized":"2024-07-17T08:35:35.455982+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:29:46.221656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449383+0000","last_change":"2024-07-17T08:35:34.459938+0000","last_active":"2024-07-17T08:35:35.449383+0000","last_peered":"2024-07-17T08:35:35.449383+0000","last_clean":"2024-07-17T08:35:35.449383+0000","last_became_active":"2024-07-17T08:35:34.459039+0000","last_became_peered":"2024-07-17T08:35:34.459039+0000","last_unstale":"2024-07-17T08:35:35.449383+0000","last_undegraded":"2024-07-17T08:35:35.449383+0000","last_fullsized":"2024-07-17T08:35:35.449383+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:05.469616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555507+0000","last_change":"2024-07-17T08:35:21.311835+0000","last_active":"2024-07-17T08:35:35.555507+0000","last_peered":"2024-07-17T08:35:35.555507+0000","last_clean":"2024-07-17T08:35:35.555507+0000","last_became_active":"2024-07-17T08:35:21.311517+0000","last_became_peered":"2024-07-17T08:35:21.311517+0000","last_unstale":"2024-07-17T08:35:35.555507+0000","last_undegraded":"2024-07-17T08:35:35.555507+0000","last_fullsized":"2024-07-17T08:35:35.555507+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:49.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457232+0000","last_change":"2024-07-17T08:35:22.325233+0000","last_active":"2024-07-17T08:35:35.457232+0000","last_peered":"2024-07-17T08:35:35.457232+0000","last_clean":"2024-07-17T08:35:35.457232+0000","last_became_active":"2024-07-17T08:35:22.324220+0000","last_became_peered":"2024-07-17T08:35:22.324220+0000","last_unstale":"2024-07-17T08:35:35.457232+0000","last_undegraded":"2024-07-17T08:35:35.457232+0000","last_fullsized":"2024-07-17T08:35:35.457232+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:27.971078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553475+0000","last_change":"2024-07-17T08:35:33.458859+0000","last_active":"2024-07-17T08:35:35.553475+0000","last_peered":"2024-07-17T08:35:35.553475+0000","last_clean":"2024-07-17T08:35:35.553475+0000","last_became_active":"2024-07-17T08:35:33.458540+0000","last_became_peered":"2024-07-17T08:35:33.458540+0000","last_unstale":"2024-07-17T08:35:35.553475+0000","last_undegraded":"2024-07-17T08:35:35.553475+0000","last_fullsized":"2024-07-17T08:35:35.553475+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:00.336730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553819+0000","last_change":"2024-07-17T08:35:34.479421+0000","last_active":"2024-07-17T08:35:35.553819+0000","last_peered":"2024-07-17T08:35:35.553819+0000","last_clean":"2024-07-17T08:35:35.553819+0000","last_became_active":"2024-07-17T08:35:34.477979+0000","last_became_peered":"2024-07-17T08:35:34.477979+0000","last_unstale":"2024-07-17T08:35:35.553819+0000","last_undegraded":"2024-07-17T08:35:35.553819+0000","last_fullsized":"2024-07-17T08:35:35.553819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:51.053432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456089+0000","last_change":"2024-07-17T08:35:21.307478+0000","last_active":"2024-07-17T08:35:35.456089+0000","last_peered":"2024-07-17T08:35:35.456089+0000","last_clean":"2024-07-17T08:35:35.456089+0000","last_became_active":"2024-07-17T08:35:21.307175+0000","last_became_peered":"2024-07-17T08:35:21.307175+0000","last_unstale":"2024-07-17T08:35:35.456089+0000","last_undegraded":"2024-07-17T08:35:35.456089+0000","last_fullsized":"2024-07-17T08:35:35.456089+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:57.089964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461819+0000","last_change":"2024-07-17T08:35:22.329861+0000","last_active":"2024-07-17T08:35:35.461819+0000","last_peered":"2024-07-17T08:35:35.461819+0000","last_clean":"2024-07-17T08:35:35.461819+0000","last_became_active":"2024-07-17T08:35:22.329107+0000","last_became_peered":"2024-07-17T08:35:22.329107+0000","last_unstale":"2024-07-17T08:35:35.461819+0000","last_undegraded":"2024-07-17T08:35:35.461819+0000","last_fullsized":"2024-07-17T08:35:35.461819+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:56.485103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450729+0000","last_change":"2024-07-17T08:35:33.449985+0000","last_active":"2024-07-17T08:35:35.450729+0000","last_peered":"2024-07-17T08:35:35.450729+0000","last_clean":"2024-07-17T08:35:35.450729+0000","last_became_active":"2024-07-17T08:35:33.449677+0000","last_became_peered":"2024-07-17T08:35:33.449677+0000","last_unstale":"2024-07-17T08:35:35.450729+0000","last_undegraded":"2024-07-17T08:35:35.450729+0000","last_fullsized":"2024-07-17T08:35:35.450729+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:58:59.034427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552631+0000","last_change":"2024-07-17T08:35:34.488611+0000","last_active":"2024-07-17T08:35:35.552631+0000","last_peered":"2024-07-17T08:35:35.552631+0000","last_clean":"2024-07-17T08:35:35.552631+0000","last_became_active":"2024-07-17T08:35:34.486963+0000","last_became_peered":"2024-07-17T08:35:34.486963+0000","last_unstale":"2024-07-17T08:35:35.552631+0000","last_undegraded":"2024-07-17T08:35:35.552631+0000","last_fullsized":"2024-07-17T08:35:35.552631+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:00.632819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493160+0000","last_change":"2024-07-17T08:35:21.320073+0000","last_active":"2024-07-17T08:35:35.493160+0000","last_peered":"2024-07-17T08:35:35.493160+0000","last_clean":"2024-07-17T08:35:35.493160+0000","last_became_active":"2024-07-17T08:35:21.319143+0000","last_became_peered":"2024-07-17T08:35:21.319143+0000","last_unstale":"2024-07-17T08:35:35.493160+0000","last_undegraded":"2024-07-17T08:35:35.493160+0000","last_fullsized":"2024-07-17T08:35:35.493160+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:58.157402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554092+0000","last_change":"2024-07-17T08:35:22.326868+0000","last_active":"2024-07-17T08:35:35.554092+0000","last_peered":"2024-07-17T08:35:35.554092+0000","last_clean":"2024-07-17T08:35:35.554092+0000","last_became_active":"2024-07-17T08:35:22.325671+0000","last_became_peered":"2024-07-17T08:35:22.325671+0000","last_unstale":"2024-07-17T08:35:35.554092+0000","last_undegraded":"2024-07-17T08:35:35.554092+0000","last_fullsized":"2024-07-17T08:35:35.554092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:52.655860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555232+0000","last_change":"2024-07-17T08:35:33.455119+0000","last_active":"2024-07-17T08:35:35.555232+0000","last_peered":"2024-07-17T08:35:35.555232+0000","last_clean":"2024-07-17T08:35:35.555232+0000","last_became_active":"2024-07-17T08:35:33.454773+0000","last_became_peered":"2024-07-17T08:35:33.454773+0000","last_unstale":"2024-07-17T08:35:35.555232+0000","last_undegraded":"2024-07-17T08:35:35.555232+0000","last_fullsized":"2024-07-17T08:35:35.555232+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:38.944624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461250+0000","last_change":"2024-07-17T08:35:34.474579+0000","last_active":"2024-07-17T08:35:35.461250+0000","last_peered":"2024-07-17T08:35:35.461250+0000","last_clean":"2024-07-17T08:35:35.461250+0000","last_became_active":"2024-07-17T08:35:34.474286+0000","last_became_peered":"2024-07-17T08:35:34.474286+0000","last_unstale":"2024-07-17T08:35:35.461250+0000","last_undegraded":"2024-07-17T08:35:35.461250+0000","last_fullsized":"2024-07-17T08:35:35.461250+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:04.679552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555839+0000","last_change":"2024-07-17T08:35:21.309083+0000","last_active":"2024-07-17T08:35:35.555839+0000","last_peered":"2024-07-17T08:35:35.555839+0000","last_clean":"2024-07-17T08:35:35.555839+0000","last_became_active":"2024-07-17T08:35:21.308727+0000","last_became_peered":"2024-07-17T08:35:21.308727+0000","last_unstale":"2024-07-17T08:35:35.555839+0000","last_undegraded":"2024-07-17T08:35:35.555839+0000","last_fullsized":"2024-07-17T08:35:35.555839+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:00.607340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554654+0000","last_change":"2024-07-17T08:35:22.334411+0000","last_active":"2024-07-17T08:35:35.554654+0000","last_peered":"2024-07-17T08:35:35.554654+0000","last_clean":"2024-07-17T08:35:35.554654+0000","last_became_active":"2024-07-17T08:35:22.334114+0000","last_became_peered":"2024-07-17T08:35:22.334114+0000","last_unstale":"2024-07-17T08:35:35.554654+0000","last_undegraded":"2024-07-17T08:35:35.554654+0000","last_fullsized":"2024-07-17T08:35:35.554654+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:36:36.700604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461450+0000","last_change":"2024-07-17T08:35:33.457888+0000","last_active":"2024-07-17T08:35:35.461450+0000","last_peered":"2024-07-17T08:35:35.461450+0000","last_clean":"2024-07-17T08:35:35.461450+0000","last_became_active":"2024-07-17T08:35:33.457510+0000","last_became_peered":"2024-07-17T08:35:33.457510+0000","last_unstale":"2024-07-17T08:35:35.461450+0000","last_undegraded":"2024-07-17T08:35:35.461450+0000","last_fullsized":"2024-07-17T08:35:35.461450+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:51.342532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555200+0000","last_change":"2024-07-17T08:35:34.498472+0000","last_active":"2024-07-17T08:35:35.555200+0000","last_peered":"2024-07-17T08:35:35.555200+0000","last_clean":"2024-07-17T08:35:35.555200+0000","last_became_active":"2024-07-17T08:35:34.492593+0000","last_became_peered":"2024-07-17T08:35:34.492593+0000","last_unstale":"2024-07-17T08:35:35.555200+0000","last_undegraded":"2024-07-17T08:35:35.555200+0000","last_fullsized":"2024-07-17T08:35:35.555200+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:28.558002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450203+0000","last_change":"2024-07-17T08:35:21.306491+0000","last_active":"2024-07-17T08:35:35.450203+0000","last_peered":"2024-07-17T08:35:35.450203+0000","last_clean":"2024-07-17T08:35:35.450203+0000","last_became_active":"2024-07-17T08:35:21.306189+0000","last_became_peered":"2024-07-17T08:35:21.306189+0000","last_unstale":"2024-07-17T08:35:35.450203+0000","last_undegraded":"2024-07-17T08:35:35.450203+0000","last_fullsized":"2024-07-17T08:35:35.450203+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:07.790314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554679+0000","last_change":"2024-07-17T08:35:22.307834+0000","last_active":"2024-07-17T08:35:35.554679+0000","last_peered":"2024-07-17T08:35:35.554679+0000","last_clean":"2024-07-17T08:35:35.554679+0000","last_became_active":"2024-07-17T08:35:22.307123+0000","last_became_peered":"2024-07-17T08:35:22.307123+0000","last_unstale":"2024-07-17T08:35:35.554679+0000","last_undegraded":"2024-07-17T08:35:35.554679+0000","last_fullsized":"2024-07-17T08:35:35.554679+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:53.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459946+0000","last_change":"2024-07-17T08:35:33.456857+0000","last_active":"2024-07-17T08:35:35.459946+0000","last_peered":"2024-07-17T08:35:35.459946+0000","last_clean":"2024-07-17T08:35:35.459946+0000","last_became_active":"2024-07-17T08:35:33.456290+0000","last_became_peered":"2024-07-17T08:35:33.456290+0000","last_unstale":"2024-07-17T08:35:35.459946+0000","last_undegraded":"2024-07-17T08:35:35.459946+0000","last_fullsized":"2024-07-17T08:35:35.459946+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:12.755522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553191+0000","last_change":"2024-07-17T08:35:34.491188+0000","last_active":"2024-07-17T08:35:35.553191+0000","last_peered":"2024-07-17T08:35:35.553191+0000","last_clean":"2024-07-17T08:35:35.553191+0000","last_became_active":"2024-07-17T08:35:34.489934+0000","last_became_peered":"2024-07-17T08:35:34.489934+0000","last_unstale":"2024-07-17T08:35:35.553191+0000","last_undegraded":"2024-07-17T08:35:35.553191+0000","last_fullsized":"2024-07-17T08:35:35.553191+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:04.970179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450322+0000","last_change":"2024-07-17T08:35:21.306965+0000","last_active":"2024-07-17T08:35:35.450322+0000","last_peered":"2024-07-17T08:35:35.450322+0000","last_clean":"2024-07-17T08:35:35.450322+0000","last_became_active":"2024-07-17T08:35:21.306687+0000","last_became_peered":"2024-07-17T08:35:21.306687+0000","last_unstale":"2024-07-17T08:35:35.450322+0000","last_undegraded":"2024-07-17T08:35:35.450322+0000","last_fullsized":"2024-07-17T08:35:35.450322+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:47.652137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460400+0000","last_change":"2024-07-17T08:35:22.328315+0000","last_active":"2024-07-17T08:35:35.460400+0000","last_peered":"2024-07-17T08:35:35.460400+0000","last_clean":"2024-07-17T08:35:35.460400+0000","last_became_active":"2024-07-17T08:35:22.327994+0000","last_became_peered":"2024-07-17T08:35:22.327994+0000","last_unstale":"2024-07-17T08:35:35.460400+0000","last_undegraded":"2024-07-17T08:35:35.460400+0000","last_fullsized":"2024-07-17T08:35:35.460400+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:21:58.847782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486062+0000","last_change":"2024-07-17T08:35:33.454020+0000","last_active":"2024-07-17T08:35:35.486062+0000","last_peered":"2024-07-17T08:35:35.486062+0000","last_clean":"2024-07-17T08:35:35.486062+0000","last_became_active":"2024-07-17T08:35:33.453743+0000","last_became_peered":"2024-07-17T08:35:33.453743+0000","last_unstale":"2024-07-17T08:35:35.486062+0000","last_undegraded":"2024-07-17T08:35:35.486062+0000","last_fullsized":"2024-07-17T08:35:35.486062+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:59.652207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455403+0000","last_change":"2024-07-17T08:35:34.486630+0000","last_active":"2024-07-17T08:35:35.455403+0000","last_peered":"2024-07-17T08:35:35.455403+0000","last_clean":"2024-07-17T08:35:35.455403+0000","last_became_active":"2024-07-17T08:35:34.485558+0000","last_became_peered":"2024-07-17T08:35:34.485558+0000","last_unstale":"2024-07-17T08:35:35.455403+0000","last_undegraded":"2024-07-17T08:35:35.455403+0000","last_fullsized":"2024-07-17T08:35:35.455403+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:20.604380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554451+0000","last_change":"2024-07-17T08:35:21.306855+0000","last_active":"2024-07-17T08:35:35.554451+0000","last_peered":"2024-07-17T08:35:35.554451+0000","last_clean":"2024-07-17T08:35:35.554451+0000","last_became_active":"2024-07-17T08:35:21.304982+0000","last_became_peered":"2024-07-17T08:35:21.304982+0000","last_unstale":"2024-07-17T08:35:35.554451+0000","last_undegraded":"2024-07-17T08:35:35.554451+0000","last_fullsized":"2024-07-17T08:35:35.554451+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:57.859513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458964+0000","last_change":"2024-07-17T08:35:22.319680+0000","last_active":"2024-07-17T08:35:35.458964+0000","last_peered":"2024-07-17T08:35:35.458964+0000","last_clean":"2024-07-17T08:35:35.458964+0000","last_became_active":"2024-07-17T08:35:22.319373+0000","last_became_peered":"2024-07-17T08:35:22.319373+0000","last_unstale":"2024-07-17T08:35:35.458964+0000","last_undegraded":"2024-07-17T08:35:35.458964+0000","last_fullsized":"2024-07-17T08:35:35.458964+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:07.288084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459274+0000","last_change":"2024-07-17T08:35:33.451492+0000","last_active":"2024-07-17T08:35:35.459274+0000","last_peered":"2024-07-17T08:35:35.459274+0000","last_clean":"2024-07-17T08:35:35.459274+0000","last_became_active":"2024-07-17T08:35:33.451016+0000","last_became_peered":"2024-07-17T08:35:33.451016+0000","last_unstale":"2024-07-17T08:35:35.459274+0000","last_undegraded":"2024-07-17T08:35:35.459274+0000","last_fullsized":"2024-07-17T08:35:35.459274+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:08:32.267947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553479+0000","last_change":"2024-07-17T08:35:34.497474+0000","last_active":"2024-07-17T08:35:35.553479+0000","last_peered":"2024-07-17T08:35:35.553479+0000","last_clean":"2024-07-17T08:35:35.553479+0000","last_became_active":"2024-07-17T08:35:34.486892+0000","last_became_peered":"2024-07-17T08:35:34.486892+0000","last_unstale":"2024-07-17T08:35:35.553479+0000","last_undegraded":"2024-07-17T08:35:35.553479+0000","last_fullsized":"2024-07-17T08:35:35.553479+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:35.295643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458654+0000","last_change":"2024-07-17T08:35:21.318267+0000","last_active":"2024-07-17T08:35:35.458654+0000","last_peered":"2024-07-17T08:35:35.458654+0000","last_clean":"2024-07-17T08:35:35.458654+0000","last_became_active":"2024-07-17T08:35:21.317956+0000","last_became_peered":"2024-07-17T08:35:21.317956+0000","last_unstale":"2024-07-17T08:35:35.458654+0000","last_undegraded":"2024-07-17T08:35:35.458654+0000","last_fullsized":"2024-07-17T08:35:35.458654+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:07.438893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455763+0000","last_change":"2024-07-17T08:35:22.331346+0000","last_active":"2024-07-17T08:35:35.455763+0000","last_peered":"2024-07-17T08:35:35.455763+0000","last_clean":"2024-07-17T08:35:35.455763+0000","last_became_active":"2024-07-17T08:35:22.331070+0000","last_became_peered":"2024-07-17T08:35:22.331070+0000","last_unstale":"2024-07-17T08:35:35.455763+0000","last_undegraded":"2024-07-17T08:35:35.455763+0000","last_fullsized":"2024-07-17T08:35:35.455763+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:44.865077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.448996+0000","last_change":"2024-07-17T08:35:33.440197+0000","last_active":"2024-07-17T08:35:35.448996+0000","last_peered":"2024-07-17T08:35:35.448996+0000","last_clean":"2024-07-17T08:35:35.448996+0000","last_became_active":"2024-07-17T08:35:33.439893+0000","last_became_peered":"2024-07-17T08:35:33.439893+0000","last_unstale":"2024-07-17T08:35:35.448996+0000","last_undegraded":"2024-07-17T08:35:35.448996+0000","last_fullsized":"2024-07-17T08:35:35.448996+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:23.178685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552047+0000","last_change":"2024-07-17T08:35:34.486149+0000","last_active":"2024-07-17T08:35:35.552047+0000","last_peered":"2024-07-17T08:35:35.552047+0000","last_clean":"2024-07-17T08:35:35.552047+0000","last_became_active":"2024-07-17T08:35:34.485165+0000","last_became_peered":"2024-07-17T08:35:34.485165+0000","last_unstale":"2024-07-17T08:35:35.552047+0000","last_undegraded":"2024-07-17T08:35:35.552047+0000","last_fullsized":"2024-07-17T08:35:35.552047+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:33.375893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459284+0000","last_change":"2024-07-17T08:35:21.313000+0000","last_active":"2024-07-17T08:35:35.459284+0000","last_peered":"2024-07-17T08:35:35.459284+0000","last_clean":"2024-07-17T08:35:35.459284+0000","last_became_active":"2024-07-17T08:35:21.310447+0000","last_became_peered":"2024-07-17T08:35:21.310447+0000","last_unstale":"2024-07-17T08:35:35.459284+0000","last_undegraded":"2024-07-17T08:35:35.459284+0000","last_fullsized":"2024-07-17T08:35:35.459284+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:41.800582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456973+0000","last_change":"2024-07-17T08:35:22.325720+0000","last_active":"2024-07-17T08:35:35.456973+0000","last_peered":"2024-07-17T08:35:35.456973+0000","last_clean":"2024-07-17T08:35:35.456973+0000","last_became_active":"2024-07-17T08:35:22.325341+0000","last_became_peered":"2024-07-17T08:35:22.325341+0000","last_unstale":"2024-07-17T08:35:35.456973+0000","last_undegraded":"2024-07-17T08:35:35.456973+0000","last_fullsized":"2024-07-17T08:35:35.456973+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:27.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485924+0000","last_change":"2024-07-17T08:35:33.451532+0000","last_active":"2024-07-17T08:35:35.485924+0000","last_peered":"2024-07-17T08:35:35.485924+0000","last_clean":"2024-07-17T08:35:35.485924+0000","last_became_active":"2024-07-17T08:35:33.449530+0000","last_became_peered":"2024-07-17T08:35:33.449530+0000","last_unstale":"2024-07-17T08:35:35.485924+0000","last_undegraded":"2024-07-17T08:35:35.485924+0000","last_fullsized":"2024-07-17T08:35:35.485924+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:53.409692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486540+0000","last_change":"2024-07-17T08:35:34.484134+0000","last_active":"2024-07-17T08:35:35.486540+0000","last_peered":"2024-07-17T08:35:35.486540+0000","last_clean":"2024-07-17T08:35:35.486540+0000","last_became_active":"2024-07-17T08:35:34.480964+0000","last_became_peered":"2024-07-17T08:35:34.480964+0000","last_unstale":"2024-07-17T08:35:35.486540+0000","last_undegraded":"2024-07-17T08:35:35.486540+0000","last_fullsized":"2024-07-17T08:35:35.486540+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:22.103527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"222'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449153+0000","last_change":"2024-07-17T08:35:21.299629+0000","last_active":"2024-07-17T08:35:35.449153+0000","last_peered":"2024-07-17T08:35:35.449153+0000","last_clean":"2024-07-17T08:35:35.449153+0000","last_became_active":"2024-07-17T08:35:21.297262+0000","last_became_peered":"2024-07-17T08:35:21.297262+0000","last_unstale":"2024-07-17T08:35:35.449153+0000","last_undegraded":"2024-07-17T08:35:35.449153+0000","last_fullsized":"2024-07-17T08:35:35.449153+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:32.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555620+0000","last_change":"2024-07-17T08:35:22.327177+0000","last_active":"2024-07-17T08:35:35.555620+0000","last_peered":"2024-07-17T08:35:35.555620+0000","last_clean":"2024-07-17T08:35:35.555620+0000","last_became_active":"2024-07-17T08:35:22.326884+0000","last_became_peered":"2024-07-17T08:35:22.326884+0000","last_unstale":"2024-07-17T08:35:35.555620+0000","last_undegraded":"2024-07-17T08:35:35.555620+0000","last_fullsized":"2024-07-17T08:35:35.555620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:38.094519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492867+0000","last_change":"2024-07-17T08:35:33.456460+0000","last_active":"2024-07-17T08:35:35.492867+0000","last_peered":"2024-07-17T08:35:35.492867+0000","last_clean":"2024-07-17T08:35:35.492867+0000","last_became_active":"2024-07-17T08:35:33.456193+0000","last_became_peered":"2024-07-17T08:35:33.456193+0000","last_unstale":"2024-07-17T08:35:35.492867+0000","last_undegraded":"2024-07-17T08:35:35.492867+0000","last_fullsized":"2024-07-17T08:35:35.492867+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:00.094046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486268+0000","last_change":"2024-07-17T08:35:34.485589+0000","last_active":"2024-07-17T08:35:35.486268+0000","last_peered":"2024-07-17T08:35:35.486268+0000","last_clean":"2024-07-17T08:35:35.486268+0000","last_became_active":"2024-07-17T08:35:34.485287+0000","last_became_peered":"2024-07-17T08:35:34.485287+0000","last_unstale":"2024-07-17T08:35:35.486268+0000","last_undegraded":"2024-07-17T08:35:35.486268+0000","last_fullsized":"2024-07-17T08:35:35.486268+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:13.147791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554416+0000","last_change":"2024-07-17T08:35:21.315619+0000","last_active":"2024-07-17T08:35:35.554416+0000","last_peered":"2024-07-17T08:35:35.554416+0000","last_clean":"2024-07-17T08:35:35.554416+0000","last_became_active":"2024-07-17T08:35:21.315304+0000","last_became_peered":"2024-07-17T08:35:21.315304+0000","last_unstale":"2024-07-17T08:35:35.554416+0000","last_undegraded":"2024-07-17T08:35:35.554416+0000","last_fullsized":"2024-07-17T08:35:35.554416+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:16.355191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553997+0000","last_change":"2024-07-17T08:35:22.331108+0000","last_active":"2024-07-17T08:35:35.553997+0000","last_peered":"2024-07-17T08:35:35.553997+0000","last_clean":"2024-07-17T08:35:35.553997+0000","last_became_active":"2024-07-17T08:35:22.326982+0000","last_became_peered":"2024-07-17T08:35:22.326982+0000","last_unstale":"2024-07-17T08:35:35.553997+0000","last_undegraded":"2024-07-17T08:35:35.553997+0000","last_fullsized":"2024-07-17T08:35:35.553997+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:58.876592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555546+0000","last_change":"2024-07-17T08:35:33.460434+0000","last_active":"2024-07-17T08:35:35.555546+0000","last_peered":"2024-07-17T08:35:35.555546+0000","last_clean":"2024-07-17T08:35:35.555546+0000","last_became_active":"2024-07-17T08:35:33.460174+0000","last_became_peered":"2024-07-17T08:35:33.460174+0000","last_unstale":"2024-07-17T08:35:35.555546+0000","last_undegraded":"2024-07-17T08:35:35.555546+0000","last_fullsized":"2024-07-17T08:35:35.555546+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:33.539981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492894+0000","last_change":"2024-07-17T08:35:34.478029+0000","last_active":"2024-07-17T08:35:35.492894+0000","last_peered":"2024-07-17T08:35:35.492894+0000","last_clean":"2024-07-17T08:35:35.492894+0000","last_became_active":"2024-07-17T08:35:34.474129+0000","last_became_peered":"2024-07-17T08:35:34.474129+0000","last_unstale":"2024-07-17T08:35:35.492894+0000","last_undegraded":"2024-07-17T08:35:35.492894+0000","last_fullsized":"2024-07-17T08:35:35.492894+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:11:46.401539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486426+0000","last_change":"2024-07-17T08:35:21.308765+0000","last_active":"2024-07-17T08:35:35.486426+0000","last_peered":"2024-07-17T08:35:35.486426+0000","last_clean":"2024-07-17T08:35:35.486426+0000","last_became_active":"2024-07-17T08:35:21.308223+0000","last_became_peered":"2024-07-17T08:35:21.308223+0000","last_unstale":"2024-07-17T08:35:35.486426+0000","last_undegraded":"2024-07-17T08:35:35.486426+0000","last_fullsized":"2024-07-17T08:35:35.486426+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:51.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486816+0000","last_change":"2024-07-17T08:35:22.320351+0000","last_active":"2024-07-17T08:35:35.486816+0000","last_peered":"2024-07-17T08:35:35.486816+0000","last_clean":"2024-07-17T08:35:35.486816+0000","last_became_active":"2024-07-17T08:35:22.319981+0000","last_became_peered":"2024-07-17T08:35:22.319981+0000","last_unstale":"2024-07-17T08:35:35.486816+0000","last_undegraded":"2024-07-17T08:35:35.486816+0000","last_fullsized":"2024-07-17T08:35:35.486816+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:50.565046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"226'6","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458001+0000","last_change":"2024-07-17T08:35:33.458710+0000","last_active":"2024-07-17T08:35:35.458001+0000","last_peered":"2024-07-17T08:35:35.458001+0000","last_clean":"2024-07-17T08:35:35.458001+0000","last_became_active":"2024-07-17T08:35:33.458420+0000","last_became_peered":"2024-07-17T08:35:33.458420+0000","last_unstale":"2024-07-17T08:35:35.458001+0000","last_undegraded":"2024-07-17T08:35:35.458001+0000","last_fullsized":"2024-07-17T08:35:35.458001+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:01.698884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555436+0000","last_change":"2024-07-17T08:35:34.483586+0000","last_active":"2024-07-17T08:35:35.555436+0000","last_peered":"2024-07-17T08:35:35.555436+0000","last_clean":"2024-07-17T08:35:35.555436+0000","last_became_active":"2024-07-17T08:35:34.483162+0000","last_became_peered":"2024-07-17T08:35:34.483162+0000","last_unstale":"2024-07-17T08:35:35.555436+0000","last_undegraded":"2024-07-17T08:35:35.555436+0000","last_fullsized":"2024-07-17T08:35:35.555436+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:20.206854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553417+0000","last_change":"2024-07-17T08:35:21.307808+0000","last_active":"2024-07-17T08:35:35.553417+0000","last_peered":"2024-07-17T08:35:35.553417+0000","last_clean":"2024-07-17T08:35:35.553417+0000","last_became_active":"2024-07-17T08:35:21.307146+0000","last_became_peered":"2024-07-17T08:35:21.307146+0000","last_unstale":"2024-07-17T08:35:35.553417+0000","last_undegraded":"2024-07-17T08:35:35.553417+0000","last_fullsized":"2024-07-17T08:35:35.553417+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:35.853604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554601+0000","last_change":"2024-07-17T08:35:22.308092+0000","last_active":"2024-07-17T08:35:35.554601+0000","last_peered":"2024-07-17T08:35:35.554601+0000","last_clean":"2024-07-17T08:35:35.554601+0000","last_became_active":"2024-07-17T08:35:22.307506+0000","last_became_peered":"2024-07-17T08:35:22.307506+0000","last_unstale":"2024-07-17T08:35:35.554601+0000","last_undegraded":"2024-07-17T08:35:35.554601+0000","last_fullsized":"2024-07-17T08:35:35.554601+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:07:04.780778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"226'3","reported_seq":17,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554549+0000","last_change":"2024-07-17T08:35:33.459431+0000","last_active":"2024-07-17T08:35:35.554549+0000","last_peered":"2024-07-17T08:35:35.554549+0000","last_clean":"2024-07-17T08:35:35.554549+0000","last_became_active":"2024-07-17T08:35:33.459129+0000","last_became_peered":"2024-07-17T08:35:33.459129+0000","last_unstale":"2024-07-17T08:35:35.554549+0000","last_undegraded":"2024-07-17T08:35:35.554549+0000","last_fullsized":"2024-07-17T08:35:35.554549+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:13.823820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455802+0000","last_change":"2024-07-17T08:35:34.493124+0000","last_active":"2024-07-17T08:35:35.455802+0000","last_peered":"2024-07-17T08:35:35.455802+0000","last_clean":"2024-07-17T08:35:35.455802+0000","last_became_active":"2024-07-17T08:35:34.492795+0000","last_became_peered":"2024-07-17T08:35:34.492795+0000","last_unstale":"2024-07-17T08:35:35.455802+0000","last_undegraded":"2024-07-17T08:35:35.455802+0000","last_fullsized":"2024-07-17T08:35:35.455802+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:10.255450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"223'8","reported_seq":30,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554137+0000","last_change":"2024-07-17T08:35:21.321190+0000","last_active":"2024-07-17T08:35:35.554137+0000","last_peered":"2024-07-17T08:35:35.554137+0000","last_clean":"2024-07-17T08:35:35.554137+0000","last_became_active":"2024-07-17T08:35:21.320605+0000","last_became_peered":"2024-07-17T08:35:21.320605+0000","last_unstale":"2024-07-17T08:35:35.554137+0000","last_undegraded":"2024-07-17T08:35:35.554137+0000","last_fullsized":"2024-07-17T08:35:35.554137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:58.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450149+0000","last_change":"2024-07-17T08:35:22.333426+0000","last_active":"2024-07-17T08:35:35.450149+0000","last_peered":"2024-07-17T08:35:35.450149+0000","last_clean":"2024-07-17T08:35:35.450149+0000","last_became_active":"2024-07-17T08:35:22.329914+0000","last_became_peered":"2024-07-17T08:35:22.329914+0000","last_unstale":"2024-07-17T08:35:35.450149+0000","last_undegraded":"2024-07-17T08:35:35.450149+0000","last_fullsized":"2024-07-17T08:35:35.450149+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:51.346013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552761+0000","last_change":"2024-07-17T08:35:33.438513+0000","last_active":"2024-07-17T08:35:35.552761+0000","last_peered":"2024-07-17T08:35:35.552761+0000","last_clean":"2024-07-17T08:35:35.552761+0000","last_became_active":"2024-07-17T08:35:33.438129+0000","last_became_peered":"2024-07-17T08:35:33.438129+0000","last_unstale":"2024-07-17T08:35:35.552761+0000","last_undegraded":"2024-07-17T08:35:35.552761+0000","last_fullsized":"2024-07-17T08:35:35.552761+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:20:04.565470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551555+0000","last_change":"2024-07-17T08:35:34.483491+0000","last_active":"2024-07-17T08:35:35.551555+0000","last_peered":"2024-07-17T08:35:35.551555+0000","last_clean":"2024-07-17T08:35:35.551555+0000","last_became_active":"2024-07-17T08:35:34.483129+0000","last_became_peered":"2024-07-17T08:35:34.483129+0000","last_unstale":"2024-07-17T08:35:35.551555+0000","last_undegraded":"2024-07-17T08:35:35.551555+0000","last_fullsized":"2024-07-17T08:35:35.551555+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:25:39.959193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"222'3","reported_seq":25,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552904+0000","last_change":"2024-07-17T08:35:21.313395+0000","last_active":"2024-07-17T08:35:35.552904+0000","last_peered":"2024-07-17T08:35:35.552904+0000","last_clean":"2024-07-17T08:35:35.552904+0000","last_became_active":"2024-07-17T08:35:21.309858+0000","last_became_peered":"2024-07-17T08:35:21.309858+0000","last_unstale":"2024-07-17T08:35:35.552904+0000","last_undegraded":"2024-07-17T08:35:35.552904+0000","last_fullsized":"2024-07-17T08:35:35.552904+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:42.857038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553318+0000","last_change":"2024-07-17T08:35:22.326199+0000","last_active":"2024-07-17T08:35:35.553318+0000","last_peered":"2024-07-17T08:35:35.553318+0000","last_clean":"2024-07-17T08:35:35.553318+0000","last_became_active":"2024-07-17T08:35:22.325620+0000","last_became_peered":"2024-07-17T08:35:22.325620+0000","last_unstale":"2024-07-17T08:35:35.553318+0000","last_undegraded":"2024-07-17T08:35:35.553318+0000","last_fullsized":"2024-07-17T08:35:35.553318+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:01.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458967+0000","last_change":"2024-07-17T08:35:33.451275+0000","last_active":"2024-07-17T08:35:35.458967+0000","last_peered":"2024-07-17T08:35:35.458967+0000","last_clean":"2024-07-17T08:35:35.458967+0000","last_became_active":"2024-07-17T08:35:33.450592+0000","last_became_peered":"2024-07-17T08:35:33.450592+0000","last_unstale":"2024-07-17T08:35:35.458967+0000","last_undegraded":"2024-07-17T08:35:35.458967+0000","last_fullsized":"2024-07-17T08:35:35.458967+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:47.040622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486251+0000","last_change":"2024-07-17T08:35:34.496715+0000","last_active":"2024-07-17T08:35:35.486251+0000","last_peered":"2024-07-17T08:35:35.486251+0000","last_clean":"2024-07-17T08:35:35.486251+0000","last_became_active":"2024-07-17T08:35:34.496413+0000","last_became_peered":"2024-07-17T08:35:34.496413+0000","last_unstale":"2024-07-17T08:35:35.486251+0000","last_undegraded":"2024-07-17T08:35:35.486251+0000","last_fullsized":"2024-07-17T08:35:35.486251+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:04.804268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555332+0000","last_change":"2024-07-17T08:35:21.320195+0000","last_active":"2024-07-17T08:35:35.555332+0000","last_peered":"2024-07-17T08:35:35.555332+0000","last_clean":"2024-07-17T08:35:35.555332+0000","last_became_active":"2024-07-17T08:35:21.319928+0000","last_became_peered":"2024-07-17T08:35:21.319928+0000","last_unstale":"2024-07-17T08:35:35.555332+0000","last_undegraded":"2024-07-17T08:35:35.555332+0000","last_fullsized":"2024-07-17T08:35:35.555332+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:33.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485967+0000","last_change":"2024-07-17T08:35:22.327230+0000","last_active":"2024-07-17T08:35:35.485967+0000","last_peered":"2024-07-17T08:35:35.485967+0000","last_clean":"2024-07-17T08:35:35.485967+0000","last_became_active":"2024-07-17T08:35:22.326914+0000","last_became_peered":"2024-07-17T08:35:22.326914+0000","last_unstale":"2024-07-17T08:35:35.485967+0000","last_undegraded":"2024-07-17T08:35:35.485967+0000","last_fullsized":"2024-07-17T08:35:35.485967+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:00.746379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"226'8","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455662+0000","last_change":"2024-07-17T08:35:33.460898+0000","last_active":"2024-07-17T08:35:35.455662+0000","last_peered":"2024-07-17T08:35:35.455662+0000","last_clean":"2024-07-17T08:35:35.455662+0000","last_became_active":"2024-07-17T08:35:33.460626+0000","last_became_peered":"2024-07-17T08:35:33.460626+0000","last_unstale":"2024-07-17T08:35:35.455662+0000","last_undegraded":"2024-07-17T08:35:35.455662+0000","last_fullsized":"2024-07-17T08:35:35.455662+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:10.216965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555230+0000","last_change":"2024-07-17T08:35:34.497427+0000","last_active":"2024-07-17T08:35:35.555230+0000","last_peered":"2024-07-17T08:35:35.555230+0000","last_clean":"2024-07-17T08:35:35.555230+0000","last_became_active":"2024-07-17T08:35:34.496490+0000","last_became_peered":"2024-07-17T08:35:34.496490+0000","last_unstale":"2024-07-17T08:35:35.555230+0000","last_undegraded":"2024-07-17T08:35:35.555230+0000","last_fullsized":"2024-07-17T08:35:35.555230+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:14:27.078421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.451056+0000","last_change":"2024-07-17T08:35:21.312028+0000","last_active":"2024-07-17T08:35:35.451056+0000","last_peered":"2024-07-17T08:35:35.451056+0000","last_clean":"2024-07-17T08:35:35.451056+0000","last_became_active":"2024-07-17T08:35:21.311720+0000","last_became_peered":"2024-07-17T08:35:21.311720+0000","last_unstale":"2024-07-17T08:35:35.451056+0000","last_undegraded":"2024-07-17T08:35:35.451056+0000","last_fullsized":"2024-07-17T08:35:35.451056+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:09:24.312105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491687+0000","last_change":"2024-07-17T08:35:22.317853+0000","last_active":"2024-07-17T08:35:35.491687+0000","last_peered":"2024-07-17T08:35:35.491687+0000","last_clean":"2024-07-17T08:35:35.491687+0000","last_became_active":"2024-07-17T08:35:22.317276+0000","last_became_peered":"2024-07-17T08:35:22.317276+0000","last_unstale":"2024-07-17T08:35:35.491687+0000","last_undegraded":"2024-07-17T08:35:35.491687+0000","last_fullsized":"2024-07-17T08:35:35.491687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:21.845715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454949+0000","last_change":"2024-07-17T08:35:33.452238+0000","last_active":"2024-07-17T08:35:35.454949+0000","last_peered":"2024-07-17T08:35:35.454949+0000","last_clean":"2024-07-17T08:35:35.454949+0000","last_became_active":"2024-07-17T08:35:33.450790+0000","last_became_peered":"2024-07-17T08:35:33.450790+0000","last_unstale":"2024-07-17T08:35:35.454949+0000","last_undegraded":"2024-07-17T08:35:35.454949+0000","last_fullsized":"2024-07-17T08:35:35.454949+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:24.729759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553337+0000","last_change":"2024-07-17T08:35:34.483916+0000","last_active":"2024-07-17T08:35:35.553337+0000","last_peered":"2024-07-17T08:35:35.553337+0000","last_clean":"2024-07-17T08:35:35.553337+0000","last_became_active":"2024-07-17T08:35:34.483101+0000","last_became_peered":"2024-07-17T08:35:34.483101+0000","last_unstale":"2024-07-17T08:35:35.553337+0000","last_undegraded":"2024-07-17T08:35:35.553337+0000","last_fullsized":"2024-07-17T08:35:35.553337+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:46.506398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460048+0000","last_change":"2024-07-17T08:35:33.456972+0000","last_active":"2024-07-17T08:35:35.460048+0000","last_peered":"2024-07-17T08:35:35.460048+0000","last_clean":"2024-07-17T08:35:35.460048+0000","last_became_active":"2024-07-17T08:35:33.452779+0000","last_became_peered":"2024-07-17T08:35:33.452779+0000","last_unstale":"2024-07-17T08:35:35.460048+0000","last_undegraded":"2024-07-17T08:35:35.460048+0000","last_fullsized":"2024-07-17T08:35:35.460048+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:29:43.620854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485766+0000","last_change":"2024-07-17T08:35:21.314342+0000","last_active":"2024-07-17T08:35:35.485766+0000","last_peered":"2024-07-17T08:35:35.485766+0000","last_clean":"2024-07-17T08:35:35.485766+0000","last_became_active":"2024-07-17T08:35:21.314009+0000","last_became_peered":"2024-07-17T08:35:21.314009+0000","last_unstale":"2024-07-17T08:35:35.485766+0000","last_undegraded":"2024-07-17T08:35:35.485766+0000","last_fullsized":"2024-07-17T08:35:35.485766+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:47.323794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555255+0000","last_change":"2024-07-17T08:35:22.327523+0000","last_active":"2024-07-17T08:35:35.555255+0000","last_peered":"2024-07-17T08:35:35.555255+0000","last_clean":"2024-07-17T08:35:35.555255+0000","last_became_active":"2024-07-17T08:35:22.323209+0000","last_became_peered":"2024-07-17T08:35:22.323209+0000","last_unstale":"2024-07-17T08:35:35.555255+0000","last_undegraded":"2024-07-17T08:35:35.555255+0000","last_fullsized":"2024-07-17T08:35:35.555255+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:02.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554158+0000","last_change":"2024-07-17T08:35:34.491080+0000","last_active":"2024-07-17T08:35:35.554158+0000","last_peered":"2024-07-17T08:35:35.554158+0000","last_clean":"2024-07-17T08:35:35.554158+0000","last_became_active":"2024-07-17T08:35:34.490804+0000","last_became_peered":"2024-07-17T08:35:34.490804+0000","last_unstale":"2024-07-17T08:35:35.554158+0000","last_undegraded":"2024-07-17T08:35:35.554158+0000","last_fullsized":"2024-07-17T08:35:35.554158+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:50.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"226'4","reported_seq":18,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450844+0000","last_change":"2024-07-17T08:35:33.446100+0000","last_active":"2024-07-17T08:35:35.450844+0000","last_peered":"2024-07-17T08:35:35.450844+0000","last_clean":"2024-07-17T08:35:35.450844+0000","last_became_active":"2024-07-17T08:35:33.445751+0000","last_became_peered":"2024-07-17T08:35:33.445751+0000","last_unstale":"2024-07-17T08:35:35.450844+0000","last_undegraded":"2024-07-17T08:35:35.450844+0000","last_fullsized":"2024-07-17T08:35:35.450844+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:18.271351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":22,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553518+0000","last_change":"2024-07-17T08:35:21.314490+0000","last_active":"2024-07-17T08:35:35.553518+0000","last_peered":"2024-07-17T08:35:35.553518+0000","last_clean":"2024-07-17T08:35:35.553518+0000","last_became_active":"2024-07-17T08:35:21.314232+0000","last_became_peered":"2024-07-17T08:35:21.314232+0000","last_unstale":"2024-07-17T08:35:35.553518+0000","last_undegraded":"2024-07-17T08:35:35.553518+0000","last_fullsized":"2024-07-17T08:35:35.553518+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:50.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493201+0000","last_change":"2024-07-17T08:35:22.310511+0000","last_active":"2024-07-17T08:35:35.493201+0000","last_peered":"2024-07-17T08:35:35.493201+0000","last_clean":"2024-07-17T08:35:35.493201+0000","last_became_active":"2024-07-17T08:35:22.310207+0000","last_became_peered":"2024-07-17T08:35:22.310207+0000","last_unstale":"2024-07-17T08:35:35.493201+0000","last_undegraded":"2024-07-17T08:35:35.493201+0000","last_fullsized":"2024-07-17T08:35:35.493201+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:19.204797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461020+0000","last_change":"2024-07-17T08:35:34.495578+0000","last_active":"2024-07-17T08:35:35.461020+0000","last_peered":"2024-07-17T08:35:35.461020+0000","last_clean":"2024-07-17T08:35:35.461020+0000","last_became_active":"2024-07-17T08:35:34.495255+0000","last_became_peered":"2024-07-17T08:35:34.495255+0000","last_unstale":"2024-07-17T08:35:35.461020+0000","last_undegraded":"2024-07-17T08:35:35.461020+0000","last_fullsized":"2024-07-17T08:35:35.461020+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:38.958975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554180+0000","last_change":"2024-07-17T08:35:21.319503+0000","last_active":"2024-07-17T08:35:35.554180+0000","last_peered":"2024-07-17T08:35:35.554180+0000","last_clean":"2024-07-17T08:35:35.554180+0000","last_became_active":"2024-07-17T08:35:21.319064+0000","last_became_peered":"2024-07-17T08:35:21.319064+0000","last_unstale":"2024-07-17T08:35:35.554180+0000","last_undegraded":"2024-07-17T08:35:35.554180+0000","last_fullsized":"2024-07-17T08:35:35.554180+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:21.414002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492633+0000","last_change":"2024-07-17T08:35:22.310961+0000","last_active":"2024-07-17T08:35:35.492633+0000","last_peered":"2024-07-17T08:35:35.492633+0000","last_clean":"2024-07-17T08:35:35.492633+0000","last_became_active":"2024-07-17T08:35:22.310559+0000","last_became_peered":"2024-07-17T08:35:22.310559+0000","last_unstale":"2024-07-17T08:35:35.492633+0000","last_undegraded":"2024-07-17T08:35:35.492633+0000","last_fullsized":"2024-07-17T08:35:35.492633+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:41.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"226'2","reported_seq":16,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457525+0000","last_change":"2024-07-17T08:35:33.458703+0000","last_active":"2024-07-17T08:35:35.457525+0000","last_peered":"2024-07-17T08:35:35.457525+0000","last_clean":"2024-07-17T08:35:35.457525+0000","last_became_active":"2024-07-17T08:35:33.458365+0000","last_became_peered":"2024-07-17T08:35:33.458365+0000","last_unstale":"2024-07-17T08:35:35.457525+0000","last_undegraded":"2024-07-17T08:35:35.457525+0000","last_fullsized":"2024-07-17T08:35:35.457525+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:25.173461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455182+0000","last_change":"2024-07-17T08:35:34.472507+0000","last_active":"2024-07-17T08:35:35.455182+0000","last_peered":"2024-07-17T08:35:35.455182+0000","last_clean":"2024-07-17T08:35:35.455182+0000","last_became_active":"2024-07-17T08:35:34.472042+0000","last_became_peered":"2024-07-17T08:35:34.472042+0000","last_unstale":"2024-07-17T08:35:35.455182+0000","last_undegraded":"2024-07-17T08:35:35.455182+0000","last_fullsized":"2024-07-17T08:35:35.455182+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:57.135370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554169+0000","last_change":"2024-07-17T08:35:21.314425+0000","last_active":"2024-07-17T08:35:35.554169+0000","last_peered":"2024-07-17T08:35:35.554169+0000","last_clean":"2024-07-17T08:35:35.554169+0000","last_became_active":"2024-07-17T08:35:21.313302+0000","last_became_peered":"2024-07-17T08:35:21.313302+0000","last_unstale":"2024-07-17T08:35:35.554169+0000","last_undegraded":"2024-07-17T08:35:35.554169+0000","last_fullsized":"2024-07-17T08:35:35.554169+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:04.413215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450061+0000","last_change":"2024-07-17T08:35:22.304051+0000","last_active":"2024-07-17T08:35:35.450061+0000","last_peered":"2024-07-17T08:35:35.450061+0000","last_clean":"2024-07-17T08:35:35.450061+0000","last_became_active":"2024-07-17T08:35:22.303298+0000","last_became_peered":"2024-07-17T08:35:22.303298+0000","last_unstale":"2024-07-17T08:35:35.450061+0000","last_undegraded":"2024-07-17T08:35:35.450061+0000","last_fullsized":"2024-07-17T08:35:35.450061+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:52.484718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553207+0000","last_change":"2024-07-17T08:35:33.462435+0000","last_active":"2024-07-17T08:35:35.553207+0000","last_peered":"2024-07-17T08:35:35.553207+0000","last_clean":"2024-07-17T08:35:35.553207+0000","last_became_active":"2024-07-17T08:35:33.462140+0000","last_became_peered":"2024-07-17T08:35:33.462140+0000","last_unstale":"2024-07-17T08:35:35.553207+0000","last_undegraded":"2024-07-17T08:35:35.553207+0000","last_fullsized":"2024-07-17T08:35:35.553207+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:49.267551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551332+0000","last_change":"2024-07-17T08:35:34.503686+0000","last_active":"2024-07-17T08:35:35.551332+0000","last_peered":"2024-07-17T08:35:35.551332+0000","last_clean":"2024-07-17T08:35:35.551332+0000","last_became_active":"2024-07-17T08:35:34.502946+0000","last_became_peered":"2024-07-17T08:35:34.502946+0000","last_unstale":"2024-07-17T08:35:35.551332+0000","last_undegraded":"2024-07-17T08:35:35.551332+0000","last_fullsized":"2024-07-17T08:35:35.551332+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:34.944240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"222'2","reported_seq":24,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553744+0000","last_change":"2024-07-17T08:35:21.306146+0000","last_active":"2024-07-17T08:35:35.553744+0000","last_peered":"2024-07-17T08:35:35.553744+0000","last_clean":"2024-07-17T08:35:35.553744+0000","last_became_active":"2024-07-17T08:35:21.303755+0000","last_became_peered":"2024-07-17T08:35:21.303755+0000","last_unstale":"2024-07-17T08:35:35.553744+0000","last_undegraded":"2024-07-17T08:35:35.553744+0000","last_fullsized":"2024-07-17T08:35:35.553744+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:36.378691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:22.330485+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:22.330205+0000","last_became_peered":"2024-07-17T08:35:22.330205+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:00.661313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"226'1","reported_seq":15,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486237+0000","last_change":"2024-07-17T08:35:33.464947+0000","last_active":"2024-07-17T08:35:35.486237+0000","last_peered":"2024-07-17T08:35:35.486237+0000","last_clean":"2024-07-17T08:35:35.486237+0000","last_became_active":"2024-07-17T08:35:33.464637+0000","last_became_peered":"2024-07-17T08:35:33.464637+0000","last_unstale":"2024-07-17T08:35:35.486237+0000","last_undegraded":"2024-07-17T08:35:35.486237+0000","last_fullsized":"2024-07-17T08:35:35.486237+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:33.340251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449931+0000","last_change":"2024-07-17T08:35:34.485339+0000","last_active":"2024-07-17T08:35:35.449931+0000","last_peered":"2024-07-17T08:35:35.449931+0000","last_clean":"2024-07-17T08:35:35.449931+0000","last_became_active":"2024-07-17T08:35:34.485071+0000","last_became_peered":"2024-07-17T08:35:34.485071+0000","last_unstale":"2024-07-17T08:35:35.449931+0000","last_undegraded":"2024-07-17T08:35:35.449931+0000","last_fullsized":"2024-07-17T08:35:35.449931+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:40:58.882312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"222'1","reported_seq":23,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458370+0000","last_change":"2024-07-17T08:35:21.315598+0000","last_active":"2024-07-17T08:35:35.458370+0000","last_peered":"2024-07-17T08:35:35.458370+0000","last_clean":"2024-07-17T08:35:35.458370+0000","last_became_active":"2024-07-17T08:35:21.314041+0000","last_became_peered":"2024-07-17T08:35:21.314041+0000","last_unstale":"2024-07-17T08:35:35.458370+0000","last_undegraded":"2024-07-17T08:35:35.458370+0000","last_fullsized":"2024-07-17T08:35:35.458370+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:43.969884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455489+0000","last_change":"2024-07-17T08:35:22.314566+0000","last_active":"2024-07-17T08:35:35.455489+0000","last_peered":"2024-07-17T08:35:35.455489+0000","last_clean":"2024-07-17T08:35:35.455489+0000","last_became_active":"2024-07-17T08:35:22.314306+0000","last_became_peered":"2024-07-17T08:35:22.314306+0000","last_unstale":"2024-07-17T08:35:35.455489+0000","last_undegraded":"2024-07-17T08:35:35.455489+0000","last_fullsized":"2024-07-17T08:35:35.455489+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:18.513656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.551486+0000","last_change":"2024-07-17T08:35:33.450591+0000","last_active":"2024-07-17T08:35:35.551486+0000","last_peered":"2024-07-17T08:35:35.551486+0000","last_clean":"2024-07-17T08:35:35.551486+0000","last_became_active":"2024-07-17T08:35:33.450291+0000","last_became_peered":"2024-07-17T08:35:33.450291+0000","last_unstale":"2024-07-17T08:35:35.551486+0000","last_undegraded":"2024-07-17T08:35:35.551486+0000","last_fullsized":"2024-07-17T08:35:35.551486+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:11.346934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486484+0000","last_change":"2024-07-17T08:35:34.495666+0000","last_active":"2024-07-17T08:35:35.486484+0000","last_peered":"2024-07-17T08:35:35.486484+0000","last_clean":"2024-07-17T08:35:35.486484+0000","last_became_active":"2024-07-17T08:35:34.493428+0000","last_became_peered":"2024-07-17T08:35:34.493428+0000","last_unstale":"2024-07-17T08:35:35.486484+0000","last_undegraded":"2024-07-17T08:35:35.486484+0000","last_fullsized":"2024-07-17T08:35:35.486484+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:14.634595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"222'4","reported_seq":26,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493209+0000","last_change":"2024-07-17T08:35:21.317528+0000","last_active":"2024-07-17T08:35:35.493209+0000","last_peered":"2024-07-17T08:35:35.493209+0000","last_clean":"2024-07-17T08:35:35.493209+0000","last_became_active":"2024-07-17T08:35:21.316801+0000","last_became_peered":"2024-07-17T08:35:21.316801+0000","last_unstale":"2024-07-17T08:35:35.493209+0000","last_undegraded":"2024-07-17T08:35:35.493209+0000","last_fullsized":"2024-07-17T08:35:35.493209+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:20.244384+0000","last_clean_scrub_stamp":"2024-07-17T08:35:20.244384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:34.366434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552776+0000","last_change":"2024-07-17T08:35:22.321872+0000","last_active":"2024-07-17T08:35:35.552776+0000","last_peered":"2024-07-17T08:35:35.552776+0000","last_clean":"2024-07-17T08:35:35.552776+0000","last_became_active":"2024-07-17T08:35:22.321609+0000","last_became_peered":"2024-07-17T08:35:22.321609+0000","last_unstale":"2024-07-17T08:35:35.552776+0000","last_undegraded":"2024-07-17T08:35:35.552776+0000","last_fullsized":"2024-07-17T08:35:35.552776+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:44:42.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.491432+0000","last_change":"2024-07-17T08:35:33.454797+0000","last_active":"2024-07-17T08:35:35.491432+0000","last_peered":"2024-07-17T08:35:35.491432+0000","last_clean":"2024-07-17T08:35:35.491432+0000","last_became_active":"2024-07-17T08:35:33.454327+0000","last_became_peered":"2024-07-17T08:35:33.454327+0000","last_unstale":"2024-07-17T08:35:35.491432+0000","last_undegraded":"2024-07-17T08:35:35.491432+0000","last_fullsized":"2024-07-17T08:35:35.491432+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":224,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:32.395301+0000","last_clean_scrub_stamp":"2024-07-17T08:35:32.395301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:51.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455439+0000","last_change":"2024-07-17T08:35:34.490533+0000","last_active":"2024-07-17T08:35:35.455439+0000","last_peered":"2024-07-17T08:35:35.455439+0000","last_clean":"2024-07-17T08:35:35.455439+0000","last_became_active":"2024-07-17T08:35:34.488112+0000","last_became_peered":"2024-07-17T08:35:34.488112+0000","last_unstale":"2024-07-17T08:35:35.455439+0000","last_undegraded":"2024-07-17T08:35:35.455439+0000","last_fullsized":"2024-07-17T08:35:35.455439+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:15:00.666776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553336+0000","last_change":"2024-07-17T08:35:22.317582+0000","last_active":"2024-07-17T08:35:35.553336+0000","last_peered":"2024-07-17T08:35:35.553336+0000","last_clean":"2024-07-17T08:35:35.553336+0000","last_became_active":"2024-07-17T08:35:22.317162+0000","last_became_peered":"2024-07-17T08:35:22.317162+0000","last_unstale":"2024-07-17T08:35:35.553336+0000","last_undegraded":"2024-07-17T08:35:35.553336+0000","last_fullsized":"2024-07-17T08:35:35.553336+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:09:55.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552022+0000","last_change":"2024-07-17T08:35:34.484796+0000","last_active":"2024-07-17T08:35:35.552022+0000","last_peered":"2024-07-17T08:35:35.552022+0000","last_clean":"2024-07-17T08:35:35.552022+0000","last_became_active":"2024-07-17T08:35:34.484514+0000","last_became_peered":"2024-07-17T08:35:34.484514+0000","last_unstale":"2024-07-17T08:35:35.552022+0000","last_undegraded":"2024-07-17T08:35:35.552022+0000","last_fullsized":"2024-07-17T08:35:35.552022+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:09.046835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553249+0000","last_change":"2024-07-17T08:35:22.330963+0000","last_active":"2024-07-17T08:35:35.553249+0000","last_peered":"2024-07-17T08:35:35.553249+0000","last_clean":"2024-07-17T08:35:35.553249+0000","last_became_active":"2024-07-17T08:35:22.330648+0000","last_became_peered":"2024-07-17T08:35:22.330648+0000","last_unstale":"2024-07-17T08:35:35.553249+0000","last_undegraded":"2024-07-17T08:35:35.553249+0000","last_fullsized":"2024-07-17T08:35:35.553249+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:01.628007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.455254+0000","last_change":"2024-07-17T08:35:34.488202+0000","last_active":"2024-07-17T08:35:35.455254+0000","last_peered":"2024-07-17T08:35:35.455254+0000","last_clean":"2024-07-17T08:35:35.455254+0000","last_became_active":"2024-07-17T08:35:34.487794+0000","last_became_peered":"2024-07-17T08:35:34.487794+0000","last_unstale":"2024-07-17T08:35:35.455254+0000","last_undegraded":"2024-07-17T08:35:35.455254+0000","last_fullsized":"2024-07-17T08:35:35.455254+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:34.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492044+0000","last_change":"2024-07-17T08:35:22.314010+0000","last_active":"2024-07-17T08:35:35.492044+0000","last_peered":"2024-07-17T08:35:35.492044+0000","last_clean":"2024-07-17T08:35:35.492044+0000","last_became_active":"2024-07-17T08:35:22.313432+0000","last_became_peered":"2024-07-17T08:35:22.313432+0000","last_unstale":"2024-07-17T08:35:35.492044+0000","last_undegraded":"2024-07-17T08:35:35.492044+0000","last_fullsized":"2024-07-17T08:35:35.492044+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:25.913456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449659+0000","last_change":"2024-07-17T08:35:34.489886+0000","last_active":"2024-07-17T08:35:35.449659+0000","last_peered":"2024-07-17T08:35:35.449659+0000","last_clean":"2024-07-17T08:35:35.449659+0000","last_became_active":"2024-07-17T08:35:34.489612+0000","last_became_peered":"2024-07-17T08:35:34.489612+0000","last_unstale":"2024-07-17T08:35:35.449659+0000","last_undegraded":"2024-07-17T08:35:35.449659+0000","last_fullsized":"2024-07-17T08:35:35.449659+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:10.418879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552117+0000","last_change":"2024-07-17T08:35:22.325381+0000","last_active":"2024-07-17T08:35:35.552117+0000","last_peered":"2024-07-17T08:35:35.552117+0000","last_clean":"2024-07-17T08:35:35.552117+0000","last_became_active":"2024-07-17T08:35:22.325069+0000","last_became_peered":"2024-07-17T08:35:22.325069+0000","last_unstale":"2024-07-17T08:35:35.552117+0000","last_undegraded":"2024-07-17T08:35:35.552117+0000","last_fullsized":"2024-07-17T08:35:35.552117+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:13.664802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554848+0000","last_change":"2024-07-17T08:35:34.497991+0000","last_active":"2024-07-17T08:35:35.554848+0000","last_peered":"2024-07-17T08:35:35.554848+0000","last_clean":"2024-07-17T08:35:35.554848+0000","last_became_active":"2024-07-17T08:35:34.491521+0000","last_became_peered":"2024-07-17T08:35:34.491521+0000","last_unstale":"2024-07-17T08:35:35.554848+0000","last_undegraded":"2024-07-17T08:35:35.554848+0000","last_fullsized":"2024-07-17T08:35:35.554848+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:52.030910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450264+0000","last_change":"2024-07-17T08:35:22.320798+0000","last_active":"2024-07-17T08:35:35.450264+0000","last_peered":"2024-07-17T08:35:35.450264+0000","last_clean":"2024-07-17T08:35:35.450264+0000","last_became_active":"2024-07-17T08:35:22.320495+0000","last_became_peered":"2024-07-17T08:35:22.320495+0000","last_unstale":"2024-07-17T08:35:35.450264+0000","last_undegraded":"2024-07-17T08:35:35.450264+0000","last_fullsized":"2024-07-17T08:35:35.450264+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:36.469858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555288+0000","last_change":"2024-07-17T08:35:34.503875+0000","last_active":"2024-07-17T08:35:35.555288+0000","last_peered":"2024-07-17T08:35:35.555288+0000","last_clean":"2024-07-17T08:35:35.555288+0000","last_became_active":"2024-07-17T08:35:34.503583+0000","last_became_peered":"2024-07-17T08:35:34.503583+0000","last_unstale":"2024-07-17T08:35:35.555288+0000","last_undegraded":"2024-07-17T08:35:35.555288+0000","last_fullsized":"2024-07-17T08:35:35.555288+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:00:33.798525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486757+0000","last_change":"2024-07-17T08:35:22.340190+0000","last_active":"2024-07-17T08:35:35.486757+0000","last_peered":"2024-07-17T08:35:35.486757+0000","last_clean":"2024-07-17T08:35:35.486757+0000","last_became_active":"2024-07-17T08:35:22.339303+0000","last_became_peered":"2024-07-17T08:35:22.339303+0000","last_unstale":"2024-07-17T08:35:35.486757+0000","last_undegraded":"2024-07-17T08:35:35.486757+0000","last_fullsized":"2024-07-17T08:35:35.486757+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:13.595921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461219+0000","last_change":"2024-07-17T08:35:34.471198+0000","last_active":"2024-07-17T08:35:35.461219+0000","last_peered":"2024-07-17T08:35:35.461219+0000","last_clean":"2024-07-17T08:35:35.461219+0000","last_became_active":"2024-07-17T08:35:34.470800+0000","last_became_peered":"2024-07-17T08:35:34.470800+0000","last_unstale":"2024-07-17T08:35:35.461219+0000","last_undegraded":"2024-07-17T08:35:35.461219+0000","last_fullsized":"2024-07-17T08:35:35.461219+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:00.489281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458092+0000","last_change":"2024-07-17T08:35:22.330952+0000","last_active":"2024-07-17T08:35:35.458092+0000","last_peered":"2024-07-17T08:35:35.458092+0000","last_clean":"2024-07-17T08:35:35.458092+0000","last_became_active":"2024-07-17T08:35:22.330655+0000","last_became_peered":"2024-07-17T08:35:22.330655+0000","last_unstale":"2024-07-17T08:35:35.458092+0000","last_undegraded":"2024-07-17T08:35:35.458092+0000","last_fullsized":"2024-07-17T08:35:35.458092+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:37.046641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493189+0000","last_change":"2024-07-17T08:35:34.481733+0000","last_active":"2024-07-17T08:35:35.493189+0000","last_peered":"2024-07-17T08:35:35.493189+0000","last_clean":"2024-07-17T08:35:35.493189+0000","last_became_active":"2024-07-17T08:35:34.481419+0000","last_became_peered":"2024-07-17T08:35:34.481419+0000","last_unstale":"2024-07-17T08:35:35.493189+0000","last_undegraded":"2024-07-17T08:35:35.493189+0000","last_fullsized":"2024-07-17T08:35:35.493189+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:18:29.214921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555009+0000","last_change":"2024-07-17T08:35:22.329059+0000","last_active":"2024-07-17T08:35:35.555009+0000","last_peered":"2024-07-17T08:35:35.555009+0000","last_clean":"2024-07-17T08:35:35.555009+0000","last_became_active":"2024-07-17T08:35:22.328454+0000","last_became_peered":"2024-07-17T08:35:22.328454+0000","last_unstale":"2024-07-17T08:35:35.555009+0000","last_undegraded":"2024-07-17T08:35:35.555009+0000","last_fullsized":"2024-07-17T08:35:35.555009+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:00.999383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486172+0000","last_change":"2024-07-17T08:35:34.494972+0000","last_active":"2024-07-17T08:35:35.486172+0000","last_peered":"2024-07-17T08:35:35.486172+0000","last_clean":"2024-07-17T08:35:35.486172+0000","last_became_active":"2024-07-17T08:35:34.493982+0000","last_became_peered":"2024-07-17T08:35:34.493982+0000","last_unstale":"2024-07-17T08:35:35.486172+0000","last_undegraded":"2024-07-17T08:35:35.486172+0000","last_fullsized":"2024-07-17T08:35:35.486172+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:11.025000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449054+0000","last_change":"2024-07-17T08:35:22.314331+0000","last_active":"2024-07-17T08:35:35.449054+0000","last_peered":"2024-07-17T08:35:35.449054+0000","last_clean":"2024-07-17T08:35:35.449054+0000","last_became_active":"2024-07-17T08:35:22.314009+0000","last_became_peered":"2024-07-17T08:35:22.314009+0000","last_unstale":"2024-07-17T08:35:35.449054+0000","last_undegraded":"2024-07-17T08:35:35.449054+0000","last_fullsized":"2024-07-17T08:35:35.449054+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:11:10.331760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552737+0000","last_change":"2024-07-17T08:35:34.498453+0000","last_active":"2024-07-17T08:35:35.552737+0000","last_peered":"2024-07-17T08:35:35.552737+0000","last_clean":"2024-07-17T08:35:35.552737+0000","last_became_active":"2024-07-17T08:35:34.498160+0000","last_became_peered":"2024-07-17T08:35:34.498160+0000","last_unstale":"2024-07-17T08:35:35.552737+0000","last_undegraded":"2024-07-17T08:35:35.552737+0000","last_fullsized":"2024-07-17T08:35:35.552737+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:45:36.345027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552918+0000","last_change":"2024-07-17T08:35:22.324018+0000","last_active":"2024-07-17T08:35:35.552918+0000","last_peered":"2024-07-17T08:35:35.552918+0000","last_clean":"2024-07-17T08:35:35.552918+0000","last_became_active":"2024-07-17T08:35:22.323058+0000","last_became_peered":"2024-07-17T08:35:22.323058+0000","last_unstale":"2024-07-17T08:35:35.552918+0000","last_undegraded":"2024-07-17T08:35:35.552918+0000","last_fullsized":"2024-07-17T08:35:35.552918+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:30:00.653492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553040+0000","last_change":"2024-07-17T08:35:34.504906+0000","last_active":"2024-07-17T08:35:35.553040+0000","last_peered":"2024-07-17T08:35:35.553040+0000","last_clean":"2024-07-17T08:35:35.553040+0000","last_became_active":"2024-07-17T08:35:34.504288+0000","last_became_peered":"2024-07-17T08:35:34.504288+0000","last_unstale":"2024-07-17T08:35:35.553040+0000","last_undegraded":"2024-07-17T08:35:35.553040+0000","last_fullsized":"2024-07-17T08:35:35.553040+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:19.817886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459230+0000","last_change":"2024-07-17T08:35:22.326654+0000","last_active":"2024-07-17T08:35:35.459230+0000","last_peered":"2024-07-17T08:35:35.459230+0000","last_clean":"2024-07-17T08:35:35.459230+0000","last_became_active":"2024-07-17T08:35:22.326088+0000","last_became_peered":"2024-07-17T08:35:22.326088+0000","last_unstale":"2024-07-17T08:35:35.459230+0000","last_undegraded":"2024-07-17T08:35:35.459230+0000","last_fullsized":"2024-07-17T08:35:35.459230+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:15.553229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449377+0000","last_change":"2024-07-17T08:35:34.481895+0000","last_active":"2024-07-17T08:35:35.449377+0000","last_peered":"2024-07-17T08:35:35.449377+0000","last_clean":"2024-07-17T08:35:35.449377+0000","last_became_active":"2024-07-17T08:35:34.481567+0000","last_became_peered":"2024-07-17T08:35:34.481567+0000","last_unstale":"2024-07-17T08:35:35.449377+0000","last_undegraded":"2024-07-17T08:35:35.449377+0000","last_fullsized":"2024-07-17T08:35:35.449377+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:45:18.596493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.461653+0000","last_change":"2024-07-17T08:35:22.335005+0000","last_active":"2024-07-17T08:35:35.461653+0000","last_peered":"2024-07-17T08:35:35.461653+0000","last_clean":"2024-07-17T08:35:35.461653+0000","last_became_active":"2024-07-17T08:35:22.334641+0000","last_became_peered":"2024-07-17T08:35:22.334641+0000","last_unstale":"2024-07-17T08:35:35.461653+0000","last_undegraded":"2024-07-17T08:35:35.461653+0000","last_fullsized":"2024-07-17T08:35:35.461653+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:23.862286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554398+0000","last_change":"2024-07-17T08:35:34.499163+0000","last_active":"2024-07-17T08:35:35.554398+0000","last_peered":"2024-07-17T08:35:35.554398+0000","last_clean":"2024-07-17T08:35:35.554398+0000","last_became_active":"2024-07-17T08:35:34.491959+0000","last_became_peered":"2024-07-17T08:35:34.491959+0000","last_unstale":"2024-07-17T08:35:35.554398+0000","last_undegraded":"2024-07-17T08:35:35.554398+0000","last_fullsized":"2024-07-17T08:35:35.554398+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:22.008659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.462086+0000","last_change":"2024-07-17T08:35:22.325751+0000","last_active":"2024-07-17T08:35:35.462086+0000","last_peered":"2024-07-17T08:35:35.462086+0000","last_clean":"2024-07-17T08:35:35.462086+0000","last_became_active":"2024-07-17T08:35:22.325447+0000","last_became_peered":"2024-07-17T08:35:22.325447+0000","last_unstale":"2024-07-17T08:35:35.462086+0000","last_undegraded":"2024-07-17T08:35:35.462086+0000","last_fullsized":"2024-07-17T08:35:35.462086+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:19.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460233+0000","last_change":"2024-07-17T08:35:34.492707+0000","last_active":"2024-07-17T08:35:35.460233+0000","last_peered":"2024-07-17T08:35:35.460233+0000","last_clean":"2024-07-17T08:35:35.460233+0000","last_became_active":"2024-07-17T08:35:34.490372+0000","last_became_peered":"2024-07-17T08:35:34.490372+0000","last_unstale":"2024-07-17T08:35:35.460233+0000","last_undegraded":"2024-07-17T08:35:35.460233+0000","last_fullsized":"2024-07-17T08:35:35.460233+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:39.652406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460378+0000","last_change":"2024-07-17T08:35:22.323343+0000","last_active":"2024-07-17T08:35:35.460378+0000","last_peered":"2024-07-17T08:35:35.460378+0000","last_clean":"2024-07-17T08:35:35.460378+0000","last_became_active":"2024-07-17T08:35:22.323073+0000","last_became_peered":"2024-07-17T08:35:22.323073+0000","last_unstale":"2024-07-17T08:35:35.460378+0000","last_undegraded":"2024-07-17T08:35:35.460378+0000","last_fullsized":"2024-07-17T08:35:35.460378+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:59:25.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.457473+0000","last_change":"2024-07-17T08:35:34.486721+0000","last_active":"2024-07-17T08:35:35.457473+0000","last_peered":"2024-07-17T08:35:35.457473+0000","last_clean":"2024-07-17T08:35:35.457473+0000","last_became_active":"2024-07-17T08:35:34.486259+0000","last_became_peered":"2024-07-17T08:35:34.486259+0000","last_unstale":"2024-07-17T08:35:35.457473+0000","last_undegraded":"2024-07-17T08:35:35.457473+0000","last_fullsized":"2024-07-17T08:35:35.457473+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:27.746206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553127+0000","last_change":"2024-07-17T08:35:22.330814+0000","last_active":"2024-07-17T08:35:35.553127+0000","last_peered":"2024-07-17T08:35:35.553127+0000","last_clean":"2024-07-17T08:35:35.553127+0000","last_became_active":"2024-07-17T08:35:22.327206+0000","last_became_peered":"2024-07-17T08:35:22.327206+0000","last_unstale":"2024-07-17T08:35:35.553127+0000","last_undegraded":"2024-07-17T08:35:35.553127+0000","last_fullsized":"2024-07-17T08:35:35.553127+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:36.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555034+0000","last_change":"2024-07-17T08:35:34.499538+0000","last_active":"2024-07-17T08:35:35.555034+0000","last_peered":"2024-07-17T08:35:35.555034+0000","last_clean":"2024-07-17T08:35:35.555034+0000","last_became_active":"2024-07-17T08:35:34.499236+0000","last_became_peered":"2024-07-17T08:35:34.499236+0000","last_unstale":"2024-07-17T08:35:35.555034+0000","last_undegraded":"2024-07-17T08:35:35.555034+0000","last_fullsized":"2024-07-17T08:35:35.555034+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:55:40.339503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554755+0000","last_change":"2024-07-17T08:35:22.341905+0000","last_active":"2024-07-17T08:35:35.554755+0000","last_peered":"2024-07-17T08:35:35.554755+0000","last_clean":"2024-07-17T08:35:35.554755+0000","last_became_active":"2024-07-17T08:35:22.338210+0000","last_became_peered":"2024-07-17T08:35:22.338210+0000","last_unstale":"2024-07-17T08:35:35.554755+0000","last_undegraded":"2024-07-17T08:35:35.554755+0000","last_fullsized":"2024-07-17T08:35:35.554755+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:09.955119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458749+0000","last_change":"2024-07-17T08:35:34.484162+0000","last_active":"2024-07-17T08:35:35.458749+0000","last_peered":"2024-07-17T08:35:35.458749+0000","last_clean":"2024-07-17T08:35:35.458749+0000","last_became_active":"2024-07-17T08:35:34.483488+0000","last_became_peered":"2024-07-17T08:35:34.483488+0000","last_unstale":"2024-07-17T08:35:35.458749+0000","last_undegraded":"2024-07-17T08:35:35.458749+0000","last_fullsized":"2024-07-17T08:35:35.458749+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:59:42.661226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485758+0000","last_change":"2024-07-17T08:35:22.335216+0000","last_active":"2024-07-17T08:35:35.485758+0000","last_peered":"2024-07-17T08:35:35.485758+0000","last_clean":"2024-07-17T08:35:35.485758+0000","last_became_active":"2024-07-17T08:35:22.333195+0000","last_became_peered":"2024-07-17T08:35:22.333195+0000","last_unstale":"2024-07-17T08:35:35.485758+0000","last_undegraded":"2024-07-17T08:35:35.485758+0000","last_fullsized":"2024-07-17T08:35:35.485758+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:52:31.091811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456828+0000","last_change":"2024-07-17T08:35:34.485105+0000","last_active":"2024-07-17T08:35:35.456828+0000","last_peered":"2024-07-17T08:35:35.456828+0000","last_clean":"2024-07-17T08:35:35.456828+0000","last_became_active":"2024-07-17T08:35:34.484821+0000","last_became_peered":"2024-07-17T08:35:34.484821+0000","last_unstale":"2024-07-17T08:35:35.456828+0000","last_undegraded":"2024-07-17T08:35:35.456828+0000","last_fullsized":"2024-07-17T08:35:35.456828+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:16.609572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492853+0000","last_change":"2024-07-17T08:35:22.331324+0000","last_active":"2024-07-17T08:35:35.492853+0000","last_peered":"2024-07-17T08:35:35.492853+0000","last_clean":"2024-07-17T08:35:35.492853+0000","last_became_active":"2024-07-17T08:35:22.329709+0000","last_became_peered":"2024-07-17T08:35:22.329709+0000","last_unstale":"2024-07-17T08:35:35.492853+0000","last_undegraded":"2024-07-17T08:35:35.492853+0000","last_fullsized":"2024-07-17T08:35:35.492853+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:06.797667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.454814+0000","last_change":"2024-07-17T08:35:34.488768+0000","last_active":"2024-07-17T08:35:35.454814+0000","last_peered":"2024-07-17T08:35:35.454814+0000","last_clean":"2024-07-17T08:35:35.454814+0000","last_became_active":"2024-07-17T08:35:34.488485+0000","last_became_peered":"2024-07-17T08:35:34.488485+0000","last_unstale":"2024-07-17T08:35:35.454814+0000","last_undegraded":"2024-07-17T08:35:35.454814+0000","last_fullsized":"2024-07-17T08:35:35.454814+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:02.497512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458986+0000","last_change":"2024-07-17T08:35:22.326895+0000","last_active":"2024-07-17T08:35:35.458986+0000","last_peered":"2024-07-17T08:35:35.458986+0000","last_clean":"2024-07-17T08:35:35.458986+0000","last_became_active":"2024-07-17T08:35:22.326402+0000","last_became_peered":"2024-07-17T08:35:22.326402+0000","last_unstale":"2024-07-17T08:35:35.458986+0000","last_undegraded":"2024-07-17T08:35:35.458986+0000","last_fullsized":"2024-07-17T08:35:35.458986+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:43.312465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.485819+0000","last_change":"2024-07-17T08:35:34.490716+0000","last_active":"2024-07-17T08:35:35.485819+0000","last_peered":"2024-07-17T08:35:35.485819+0000","last_clean":"2024-07-17T08:35:35.485819+0000","last_became_active":"2024-07-17T08:35:34.490402+0000","last_became_peered":"2024-07-17T08:35:34.490402+0000","last_unstale":"2024-07-17T08:35:35.485819+0000","last_undegraded":"2024-07-17T08:35:35.485819+0000","last_fullsized":"2024-07-17T08:35:35.485819+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:49.544177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554175+0000","last_change":"2024-07-17T08:35:22.326996+0000","last_active":"2024-07-17T08:35:35.554175+0000","last_peered":"2024-07-17T08:35:35.554175+0000","last_clean":"2024-07-17T08:35:35.554175+0000","last_became_active":"2024-07-17T08:35:22.325603+0000","last_became_peered":"2024-07-17T08:35:22.325603+0000","last_unstale":"2024-07-17T08:35:35.554175+0000","last_undegraded":"2024-07-17T08:35:35.554175+0000","last_fullsized":"2024-07-17T08:35:35.554175+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:41.247825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552756+0000","last_change":"2024-07-17T08:35:34.498232+0000","last_active":"2024-07-17T08:35:35.552756+0000","last_peered":"2024-07-17T08:35:35.552756+0000","last_clean":"2024-07-17T08:35:35.552756+0000","last_became_active":"2024-07-17T08:35:34.492256+0000","last_became_peered":"2024-07-17T08:35:34.492256+0000","last_unstale":"2024-07-17T08:35:35.552756+0000","last_undegraded":"2024-07-17T08:35:35.552756+0000","last_fullsized":"2024-07-17T08:35:35.552756+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:44.693413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450194+0000","last_change":"2024-07-17T08:35:22.315660+0000","last_active":"2024-07-17T08:35:35.450194+0000","last_peered":"2024-07-17T08:35:35.450194+0000","last_clean":"2024-07-17T08:35:35.450194+0000","last_became_active":"2024-07-17T08:35:22.315370+0000","last_became_peered":"2024-07-17T08:35:22.315370+0000","last_unstale":"2024-07-17T08:35:35.450194+0000","last_undegraded":"2024-07-17T08:35:35.450194+0000","last_fullsized":"2024-07-17T08:35:35.450194+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:45:56.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450743+0000","last_change":"2024-07-17T08:35:34.484320+0000","last_active":"2024-07-17T08:35:35.450743+0000","last_peered":"2024-07-17T08:35:35.450743+0000","last_clean":"2024-07-17T08:35:35.450743+0000","last_became_active":"2024-07-17T08:35:34.483670+0000","last_became_peered":"2024-07-17T08:35:34.483670+0000","last_unstale":"2024-07-17T08:35:35.450743+0000","last_undegraded":"2024-07-17T08:35:35.450743+0000","last_fullsized":"2024-07-17T08:35:35.450743+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:31:34.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486717+0000","last_change":"2024-07-17T08:35:22.340596+0000","last_active":"2024-07-17T08:35:35.486717+0000","last_peered":"2024-07-17T08:35:35.486717+0000","last_clean":"2024-07-17T08:35:35.486717+0000","last_became_active":"2024-07-17T08:35:22.339735+0000","last_became_peered":"2024-07-17T08:35:22.339735+0000","last_unstale":"2024-07-17T08:35:35.486717+0000","last_undegraded":"2024-07-17T08:35:35.486717+0000","last_fullsized":"2024-07-17T08:35:35.486717+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:14.039622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460839+0000","last_change":"2024-07-17T08:35:34.489939+0000","last_active":"2024-07-17T08:35:35.460839+0000","last_peered":"2024-07-17T08:35:35.460839+0000","last_clean":"2024-07-17T08:35:35.460839+0000","last_became_active":"2024-07-17T08:35:34.489661+0000","last_became_peered":"2024-07-17T08:35:34.489661+0000","last_unstale":"2024-07-17T08:35:35.460839+0000","last_undegraded":"2024-07-17T08:35:35.460839+0000","last_fullsized":"2024-07-17T08:35:35.460839+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:47.521931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.460334+0000","last_change":"2024-07-17T08:35:22.324447+0000","last_active":"2024-07-17T08:35:35.460334+0000","last_peered":"2024-07-17T08:35:35.460334+0000","last_clean":"2024-07-17T08:35:35.460334+0000","last_became_active":"2024-07-17T08:35:22.324156+0000","last_became_peered":"2024-07-17T08:35:22.324156+0000","last_unstale":"2024-07-17T08:35:35.460334+0000","last_undegraded":"2024-07-17T08:35:35.460334+0000","last_fullsized":"2024-07-17T08:35:35.460334+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:56:10.070701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.493380+0000","last_change":"2024-07-17T08:35:34.490263+0000","last_active":"2024-07-17T08:35:35.493380+0000","last_peered":"2024-07-17T08:35:35.493380+0000","last_clean":"2024-07-17T08:35:35.493380+0000","last_became_active":"2024-07-17T08:35:34.489975+0000","last_became_peered":"2024-07-17T08:35:34.489975+0000","last_unstale":"2024-07-17T08:35:35.493380+0000","last_undegraded":"2024-07-17T08:35:35.493380+0000","last_fullsized":"2024-07-17T08:35:35.493380+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:06.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554243+0000","last_change":"2024-07-17T08:35:22.329218+0000","last_active":"2024-07-17T08:35:35.554243+0000","last_peered":"2024-07-17T08:35:35.554243+0000","last_clean":"2024-07-17T08:35:35.554243+0000","last_became_active":"2024-07-17T08:35:22.328768+0000","last_became_peered":"2024-07-17T08:35:22.328768+0000","last_unstale":"2024-07-17T08:35:35.554243+0000","last_undegraded":"2024-07-17T08:35:35.554243+0000","last_fullsized":"2024-07-17T08:35:35.554243+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:37.395770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458638+0000","last_change":"2024-07-17T08:35:34.492433+0000","last_active":"2024-07-17T08:35:35.458638+0000","last_peered":"2024-07-17T08:35:35.458638+0000","last_clean":"2024-07-17T08:35:35.458638+0000","last_became_active":"2024-07-17T08:35:34.490894+0000","last_became_peered":"2024-07-17T08:35:34.490894+0000","last_unstale":"2024-07-17T08:35:35.458638+0000","last_undegraded":"2024-07-17T08:35:35.458638+0000","last_fullsized":"2024-07-17T08:35:35.458638+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:47.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.456454+0000","last_change":"2024-07-17T08:35:22.326358+0000","last_active":"2024-07-17T08:35:35.456454+0000","last_peered":"2024-07-17T08:35:35.456454+0000","last_clean":"2024-07-17T08:35:35.456454+0000","last_became_active":"2024-07-17T08:35:22.326078+0000","last_became_peered":"2024-07-17T08:35:22.326078+0000","last_unstale":"2024-07-17T08:35:35.456454+0000","last_undegraded":"2024-07-17T08:35:35.456454+0000","last_fullsized":"2024-07-17T08:35:35.456454+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:49:06.036609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553404+0000","last_change":"2024-07-17T08:35:34.488358+0000","last_active":"2024-07-17T08:35:35.553404+0000","last_peered":"2024-07-17T08:35:35.553404+0000","last_clean":"2024-07-17T08:35:35.553404+0000","last_became_active":"2024-07-17T08:35:34.488076+0000","last_became_peered":"2024-07-17T08:35:34.488076+0000","last_unstale":"2024-07-17T08:35:35.553404+0000","last_undegraded":"2024-07-17T08:35:35.553404+0000","last_fullsized":"2024-07-17T08:35:35.553404+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.483378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.449620+0000","last_change":"2024-07-17T08:35:22.312298+0000","last_active":"2024-07-17T08:35:35.449620+0000","last_peered":"2024-07-17T08:35:35.449620+0000","last_clean":"2024-07-17T08:35:35.449620+0000","last_became_active":"2024-07-17T08:35:22.309483+0000","last_became_peered":"2024-07-17T08:35:22.309483+0000","last_unstale":"2024-07-17T08:35:35.449620+0000","last_undegraded":"2024-07-17T08:35:35.449620+0000","last_fullsized":"2024-07-17T08:35:35.449620+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:54.102827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.459478+0000","last_change":"2024-07-17T08:35:34.483905+0000","last_active":"2024-07-17T08:35:35.459478+0000","last_peered":"2024-07-17T08:35:35.459478+0000","last_clean":"2024-07-17T08:35:35.459478+0000","last_became_active":"2024-07-17T08:35:34.481531+0000","last_became_peered":"2024-07-17T08:35:34.481531+0000","last_unstale":"2024-07-17T08:35:35.459478+0000","last_undegraded":"2024-07-17T08:35:35.459478+0000","last_fullsized":"2024-07-17T08:35:35.459478+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:25.813209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.553820+0000","last_change":"2024-07-17T08:35:22.323100+0000","last_active":"2024-07-17T08:35:35.553820+0000","last_peered":"2024-07-17T08:35:35.553820+0000","last_clean":"2024-07-17T08:35:35.553820+0000","last_became_active":"2024-07-17T08:35:22.322805+0000","last_became_peered":"2024-07-17T08:35:22.322805+0000","last_unstale":"2024-07-17T08:35:35.553820+0000","last_undegraded":"2024-07-17T08:35:35.553820+0000","last_fullsized":"2024-07-17T08:35:35.553820+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:52.497030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552031+0000","last_change":"2024-07-17T08:35:34.495772+0000","last_active":"2024-07-17T08:35:35.552031+0000","last_peered":"2024-07-17T08:35:35.552031+0000","last_clean":"2024-07-17T08:35:35.552031+0000","last_became_active":"2024-07-17T08:35:34.495454+0000","last_became_peered":"2024-07-17T08:35:34.495454+0000","last_unstale":"2024-07-17T08:35:35.552031+0000","last_undegraded":"2024-07-17T08:35:35.552031+0000","last_fullsized":"2024-07-17T08:35:35.552031+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:23.394746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.450049+0000","last_change":"2024-07-17T08:35:22.309692+0000","last_active":"2024-07-17T08:35:35.450049+0000","last_peered":"2024-07-17T08:35:35.450049+0000","last_clean":"2024-07-17T08:35:35.450049+0000","last_became_active":"2024-07-17T08:35:22.309210+0000","last_became_peered":"2024-07-17T08:35:22.309210+0000","last_unstale":"2024-07-17T08:35:35.450049+0000","last_undegraded":"2024-07-17T08:35:35.450049+0000","last_fullsized":"2024-07-17T08:35:35.450049+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:34.535907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.552363+0000","last_change":"2024-07-17T08:35:34.492325+0000","last_active":"2024-07-17T08:35:35.552363+0000","last_peered":"2024-07-17T08:35:35.552363+0000","last_clean":"2024-07-17T08:35:35.552363+0000","last_became_active":"2024-07-17T08:35:34.491403+0000","last_became_peered":"2024-07-17T08:35:34.491403+0000","last_unstale":"2024-07-17T08:35:35.552363+0000","last_undegraded":"2024-07-17T08:35:35.552363+0000","last_fullsized":"2024-07-17T08:35:35.552363+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:42:09.402312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554043+0000","last_change":"2024-07-17T08:35:22.329337+0000","last_active":"2024-07-17T08:35:35.554043+0000","last_peered":"2024-07-17T08:35:35.554043+0000","last_clean":"2024-07-17T08:35:35.554043+0000","last_became_active":"2024-07-17T08:35:22.329060+0000","last_became_peered":"2024-07-17T08:35:22.329060+0000","last_unstale":"2024-07-17T08:35:35.554043+0000","last_undegraded":"2024-07-17T08:35:35.554043+0000","last_fullsized":"2024-07-17T08:35:35.554043+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:34.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555266+0000","last_change":"2024-07-17T08:35:34.494219+0000","last_active":"2024-07-17T08:35:35.555266+0000","last_peered":"2024-07-17T08:35:35.555266+0000","last_clean":"2024-07-17T08:35:35.555266+0000","last_became_active":"2024-07-17T08:35:34.493904+0000","last_became_peered":"2024-07-17T08:35:34.493904+0000","last_unstale":"2024-07-17T08:35:35.555266+0000","last_undegraded":"2024-07-17T08:35:35.555266+0000","last_fullsized":"2024-07-17T08:35:35.555266+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:33:11.039773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.486687+0000","last_change":"2024-07-17T08:35:22.321610+0000","last_active":"2024-07-17T08:35:35.486687+0000","last_peered":"2024-07-17T08:35:35.486687+0000","last_clean":"2024-07-17T08:35:35.486687+0000","last_became_active":"2024-07-17T08:35:22.321282+0000","last_became_peered":"2024-07-17T08:35:22.321282+0000","last_unstale":"2024-07-17T08:35:35.486687+0000","last_undegraded":"2024-07-17T08:35:35.486687+0000","last_fullsized":"2024-07-17T08:35:35.486687+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:19.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.492520+0000","last_change":"2024-07-17T08:35:34.489926+0000","last_active":"2024-07-17T08:35:35.492520+0000","last_peered":"2024-07-17T08:35:35.492520+0000","last_clean":"2024-07-17T08:35:35.492520+0000","last_became_active":"2024-07-17T08:35:34.489578+0000","last_became_peered":"2024-07-17T08:35:34.489578+0000","last_unstale":"2024-07-17T08:35:35.492520+0000","last_undegraded":"2024-07-17T08:35:35.492520+0000","last_fullsized":"2024-07-17T08:35:35.492520+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:58:52.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.555446+0000","last_change":"2024-07-17T08:35:22.312477+0000","last_active":"2024-07-17T08:35:35.555446+0000","last_peered":"2024-07-17T08:35:35.555446+0000","last_clean":"2024-07-17T08:35:35.555446+0000","last_became_active":"2024-07-17T08:35:22.312192+0000","last_became_peered":"2024-07-17T08:35:22.312192+0000","last_unstale":"2024-07-17T08:35:35.555446+0000","last_undegraded":"2024-07-17T08:35:35.555446+0000","last_fullsized":"2024-07-17T08:35:35.555446+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:13:11.863693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.554183+0000","last_change":"2024-07-17T08:35:34.489451+0000","last_active":"2024-07-17T08:35:35.554183+0000","last_peered":"2024-07-17T08:35:35.554183+0000","last_clean":"2024-07-17T08:35:35.554183+0000","last_became_active":"2024-07-17T08:35:34.489177+0000","last_became_peered":"2024-07-17T08:35:34.489177+0000","last_unstale":"2024-07-17T08:35:35.554183+0000","last_undegraded":"2024-07-17T08:35:35.554183+0000","last_fullsized":"2024-07-17T08:35:35.554183+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":225,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:33.409871+0000","last_clean_scrub_stamp":"2024-07-17T08:35:33.409871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:41.289199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":20,"reported_epoch":227,"state":"active+clean","last_fresh":"2024-07-17T08:35:35.458899+0000","last_change":"2024-07-17T08:35:22.337420+0000","last_active":"2024-07-17T08:35:35.458899+0000","last_peered":"2024-07-17T08:35:35.458899+0000","last_clean":"2024-07-17T08:35:35.458899+0000","last_became_active":"2024-07-17T08:35:22.337154+0000","last_became_peered":"2024-07-17T08:35:22.337154+0000","last_unstale":"2024-07-17T08:35:35.458899+0000","last_undegraded":"2024-07-17T08:35:35.458899+0000","last_fullsized":"2024-07-17T08:35:35.458899+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:35:21.261463+0000","last_clean_scrub_stamp":"2024-07-17T08:35:21.261463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:22.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1387,"num_read_kb":3871,"num_write":359,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2511,"ondisk_log_size":2511,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543345,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65088,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306752,"statfs":{"total":96636764160,"available":96570114048,"internally_reserved":0,"allocated":3461120,"data_stored":2145831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510640,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29844,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341996,"statfs":{"total":96636764160,"available":96606203904,"internally_reserved":0,"allocated":2883584,"data_stored":1557041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510641,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1553601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543346,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2850816,"data_stored":1552535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510641,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52028,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319812,"statfs":{"total":96636764160,"available":96583487488,"internally_reserved":0,"allocated":2859008,"data_stored":1552011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2854912,"data_stored":1552569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510641,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2842624,"data_stored":1551863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56756,"kb_used_data":3360,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315084,"statfs":{"total":96636764160,"available":96578646016,"internally_reserved":0,"allocated":3440640,"data_stored":2142321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:35:44.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:35:44.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:35:44.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 -- 172.21.15.44:0/3170457889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc15d600 msgr2=0x7f18dc06e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 --2- 172.21.15.44:0/3170457889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc15d600 0x7f18dc06e480 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005fd0 tx=0x7f18cc02a940 comp rx=0 tx=0).stop 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 -- 172.21.15.44:0/3170457889 shutdown_connections 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 --2- 172.21.15.44:0/3170457889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc0b2ad0 0x7f18dc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 --2- 172.21.15.44:0/3170457889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc15d600 0x7f18dc06e480 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 -- 172.21.15.44:0/3170457889 >> 172.21.15.44:0/3170457889 conn(0x7f18dc0d00a0 msgr2=0x7f18dc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 -- 172.21.15.44:0/3170457889 shutdown_connections 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.844+0000 7f18e0c00640 1 -- 172.21.15.44:0/3170457889 wait complete. 2024-07-17T08:35:44.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.846+0000 7f18e0c00640 1 Processor -- start 2024-07-17T08:35:44.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.846+0000 7f18e0c00640 1 -- start start 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 0x7f18dc15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc15d600 0x7f18dc1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc161730 con 0x7f18dc0b5740 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc0cef10 con 0x7f18dc0b2ad0 2024-07-17T08:35:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18e0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18dc1618b0 con 0x7f18dc15d600 2024-07-17T08:35:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39776/0 (socket says 172.21.15.44:39776) 2024-07-17T08:35:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.847+0000 7f18d9400640 1 -- 172.21.15.44:0/2555501167 learned_addr learned my addr 172.21.15.44:0/2555501167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9e00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 0x7f18dc15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18da800640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc15d600 0x7f18dc1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 -- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc15d600 msgr2=0x7f18dc1f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc15d600 0x7f18dc1f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 -- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 msgr2=0x7f18dc15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 0x7f18dc15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 -- 172.21.15.44:0/2555501167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1f5000 con 0x7f18dc0b5740 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9e00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 0x7f18dc15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d9400640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f18c400ea60 tx=0x7f18c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.848+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c400cde0 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.849+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18c4004020 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.849+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4004450 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.849+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc159d10 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.849+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc15a2d0 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.850+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc161d60 con 0x7f18dc0b5740 2024-07-17T08:35:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.851+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f18c40035f0 con 0x7f18dc0b5740 2024-07-17T08:35:44.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.852+0000 7f18d2000640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 0x7f18b8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:44.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.852+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f18c4099700 con 0x7f18dc0b5740 2024-07-17T08:35:44.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.853+0000 7f18d9e00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 0x7f18b8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:44.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.854+0000 7f18d9e00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 0x7f18b8079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc004ab0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:44.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:44.856+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f18c40621e0 con 0x7f18dc0b5740 2024-07-17T08:35:45.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.107+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18dc0b9030 con 0x7f18dc0b5740 2024-07-17T08:35:45.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.109+0000 7f18d2000640 1 -- 172.21.15.44:0/2555501167 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f18c4061ae0 con 0x7f18dc0b5740 2024-07-17T08:35:45.111 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:45.111 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:35:45.111 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 msgr2=0x7f18b8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 0x7f18b8079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc004ab0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 msgr2=0x7f18dc1f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f18c400ea60 tx=0x7f18c400ef20 comp rx=0 tx=0).stop 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 shutdown_connections 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18dc15d600 0x7f18dc1f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18dc0b5740 0x7f18dc1f4260 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f18b8076cd0 0x7f18b8079190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 --2- 172.21.15.44:0/2555501167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18dc0b2ad0 0x7f18dc15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 >> 172.21.15.44:0/2555501167 conn(0x7f18dc0d00a0 msgr2=0x7f18dc0c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 shutdown_connections 2024-07-17T08:35:45.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.112+0000 7f18e0c00640 1 -- 172.21.15.44:0/2555501167 wait complete. 2024-07-17T08:35:45.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:35:33.908968+0000', 'modified': '2024-07-17T08:35:36.972151+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8329}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8329': {'gid': 8329, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.44:6834/3288748832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 3288748832}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 3288748832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8329, 'qdb_cluster': [8329]} 2024-07-17T08:35:45.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:35:45.537 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-17T08:35:45.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 -- 172.21.15.44:0/4083248302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0152220 msgr2=0x7f28f006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 --2- 172.21.15.44:0/4083248302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0152220 0x7f28f006d620 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f28f006f610 tx=0x7f28e402a4b0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 -- 172.21.15.44:0/4083248302 shutdown_connections 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 --2- 172.21.15.44:0/4083248302 >> v2:172.21.15.175:3300/0 conn(0x7f28f006ed30 0x7f28f015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 --2- 172.21.15.44:0/4083248302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f0152220 0x7f28f006d620 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.757+0000 7f28f7a00640 1 -- 172.21.15.44:0/4083248302 >> 172.21.15.44:0/4083248302 conn(0x7f28f00b0eb0 msgr2=0x7f28f00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.758+0000 7f28f7a00640 1 -- 172.21.15.44:0/4083248302 shutdown_connections 2024-07-17T08:35:45.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.758+0000 7f28f7a00640 1 -- 172.21.15.44:0/4083248302 wait complete. 2024-07-17T08:35:45.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.760+0000 7f28f7a00640 1 Processor -- start 2024-07-17T08:35:45.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.760+0000 7f28f7a00640 1 -- start start 2024-07-17T08:35:45.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 0x7f28f00ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28f0152220 0x7f28f0073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28f00cbc30 con 0x7f28f006ed30 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28f00cbab0 con 0x7f28f00cb3a0 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.761+0000 7f28f7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28f015f570 con 0x7f28f0152220 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39796/0 (socket says 172.21.15.44:39796) 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 -- 172.21.15.44:0/3796253891 learned_addr learned my addr 172.21.15.44:0/3796253891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28efe00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 0x7f28f00ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 -- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28f0152220 msgr2=0x7f28f0073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28f0152220 0x7f28f0073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 -- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 msgr2=0x7f28f00ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 0x7f28f00ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 -- 172.21.15.44:0/3796253891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f0072510 con 0x7f28f006ed30 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28efe00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 0x7f28f00ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.762+0000 7f28f5000640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009780 tx=0x7f28e4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.763+0000 7f28ecc00640 1 -- 172.21.15.44:0/3796253891 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4031d60 con 0x7f28f006ed30 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.763+0000 7f28ecc00640 1 -- 172.21.15.44:0/3796253891 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e4005020 con 0x7f28f006ed30 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.763+0000 7f28ecc00640 1 -- 172.21.15.44:0/3796253891 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4005470 con 0x7f28f006ed30 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.763+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01f4f40 con 0x7f28f006ed30 2024-07-17T08:35:45.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.763+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f00acb30 con 0x7f28f006ed30 2024-07-17T08:35:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.766+0000 7f28ecc00640 1 -- 172.21.15.44:0/3796253891 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f28e40047a0 con 0x7f28f006ed30 2024-07-17T08:35:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.766+0000 7f28ecc00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 0x7f28d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.767+0000 7f28fa83b740 1 Processor -- start 2024-07-17T08:35:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.767+0000 7f28ecc00640 1 -- 172.21.15.44:0/3796253891 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f28e40b9e90 con 0x7f28f006ed30 2024-07-17T08:35:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.767+0000 7f28fa83b740 1 -- start start 2024-07-17T08:35:45.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.767+0000 7f28efe00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 0x7f28d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02ede0 0x5640fb02f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02b5b0 0x5640fb02ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640faf76d60 con 0x5640fb0060a0 2024-07-17T08:35:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640faf66260 con 0x5640fb02b5b0 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5640faf6a2a0 con 0x5640fb02ede0 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39804/0 (socket says 172.21.15.44:39804) 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 -- 172.21.15.44:0/1032056240 learned_addr learned my addr 172.21.15.44:0/1032056240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 -- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02ede0 msgr2=0x5640fb02f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02ede0 0x5640fb02f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 -- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02b5b0 msgr2=0x5640fb02ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.768+0000 7f28efe00640 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02b5b0 0x5640fb02ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28efe00640 1 -- 172.21.15.44:0/1032056240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640facca850 con 0x5640fb0060a0 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28efe00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 0x7f28d0079190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28f01f7530 tx=0x7f28dc00bec0 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28efe00640 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00b3d0 tx=0x7f28dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9563800b19f301 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28e1600640 1 -- 172.21.15.44:0/1032056240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01c650 con 0x5640fb0060a0 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28e1600640 1 -- 172.21.15.44:0/1032056240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28dc01c8e0 con 0x5640fb0060a0 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.769+0000 7f28e1600640 1 -- 172.21.15.44:0/1032056240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01cd10 con 0x5640fb0060a0 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 -- 172.21.15.44:0/1032056240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 msgr2=0x5640fb02b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00b3d0 tx=0x7f28dc018000 comp rx=0 tx=0).stop 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 -- 172.21.15.44:0/1032056240 shutdown_connections 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02b5b0 0x5640fb02ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb0060a0 0x5640fb02b070 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 --2- 172.21.15.44:0/1032056240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02ede0 0x5640fb02f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 -- 172.21.15.44:0/1032056240 >> 172.21.15.44:0/1032056240 conn(0x5640fb001280 msgr2=0x5640fb004ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 -- 172.21.15.44:0/1032056240 shutdown_connections 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.770+0000 7f28fa83b740 1 -- 172.21.15.44:0/1032056240 wait complete. 2024-07-17T08:35:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.771+0000 7f28fa83b740 1 -- start start 2024-07-17T08:35:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.771+0000 7f28fa83b740 1 Processor -- start 2024-07-17T08:35:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb027a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb027fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640facd9620 con 0x5640fb02b5b0 2024-07-17T08:35:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640faf0df90 con 0x5640fb02ede0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5640fafff980 con 0x5640fb0230c0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28fa83b740 10 client.0 _ll_register_callbacks cb 0x5640faf33f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39820/0 (socket says 172.21.15.44:39820) 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 -- 172.21.15.44:0/509778037 learned_addr learned my addr 172.21.15.44:0/509778037 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 -- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 msgr2=0x5640fb027fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb027fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.772+0000 7f28f5000640 1 -- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 msgr2=0x5640fb027a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28f5000640 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb027a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28f5000640 1 -- 172.21.15.44:0/509778037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640fad416b0 con 0x5640fb02b5b0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28f5000640 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f28e4081f70 tx=0x7f28e4081c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78299ff808e8bb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28d6000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e409b020 con 0x5640fb02b5b0 2024-07-17T08:35:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f28e408ae00 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5640faf3d900 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f28e4048040 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28fa83b740 10 client.10350 fetch_fsmap learned FSMap version 119 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) -- 0x5640fb027440 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.773+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5640faf3d900 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 119) ==== 4149+0+0 (secure 0 0 0) 0x7f28e40b0b70 con 0x5640fb02b5b0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 10 client.10350 fetch_fsmap finished waiting for FSMap version 119 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 10 client.10350 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 20 client.10350 populate_metadata read hostname 'smithi044' 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb024de0 0x5640fb05d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 4 client.10350 mds_command: new command op to 8329 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 --> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640faf78cb0 con 0x5640fb024de0 2024-07-17T08:35:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.774+0000 7f28efe00640 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb024de0 0x5640fb05d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.775+0000 7f28efe00640 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb024de0 0x5640fb05d620 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a19963543705af server_cookie=7f00f3fdb3234bf1 in_seq=0 out_seq=0 2024-07-17T08:35:45.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.775+0000 7f28d6000640 10 client.10350 ms_handle_connect on v2:172.21.15.44:6833/3288748832 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28d6000640 1 -- 172.21.15.44:0/509778037 <== mds.0 v2:172.21.15.44:6833/3288748832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640faf78cb0 con 0x5640fb024de0 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28d6000640 10 client.10350 handle_command_reply: tid=0 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 client.10350 shutdown 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 msgr2=0x5640fb017ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f28e4081f70 tx=0x7f28e4081c80 comp rx=0 tx=0).stop 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 shutdown_connections 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb024de0 0x5640fb05d620 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb027fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb027a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28d6000640 0 client.10350 ms_handle_reset on v2:172.21.15.44:6833/3288748832 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 --2- 172.21.15.44:0/509778037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb017ef0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 >> 172.21.15.44:0/509778037 conn(0x5640fb0060a0 msgr2=0x5640fb0238d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 shutdown_connections 2024-07-17T08:35:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 1 -- 172.21.15.44:0/509778037 wait complete. 2024-07-17T08:35:45.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.776+0000 7f28fa83b740 20 client.10350 trim_cache size 0 max 16384 2024-07-17T08:35:45.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.797+0000 7f28fa83b740 1 Processor -- start 2024-07-17T08:35:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.797+0000 7f28fa83b740 1 -- start start 2024-07-17T08:35:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb0230c0 0x5640fb0028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 0x5640fb017700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28f02046b0 con 0x5640fb02ede0 2024-07-17T08:35:45.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28f02020a0 con 0x5640fb0230c0 2024-07-17T08:35:45.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28f02026f0 con 0x5640fb02b5b0 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39822/0 (socket says 172.21.15.44:39822) 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 -- 172.21.15.44:0/2399404876 learned_addr learned my addr 172.21.15.44:0/2399404876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28efe00640 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 0x5640fb017700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 -- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 msgr2=0x5640fb017700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 0x5640fb017700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 -- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb0230c0 msgr2=0x5640fb0028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb0230c0 0x5640fb0028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.798+0000 7f28f5a00640 1 -- 172.21.15.44:0/2399404876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640faffec10 con 0x5640fb02ede0 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.799+0000 7f28efe00640 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 0x5640fb017700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:45.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.799+0000 7f28f5a00640 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f28e8001f90 tx=0x7f28e800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cca2e752838a22f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.799+0000 7f28e0c00640 1 -- 172.21.15.44:0/2399404876 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8017070 con 0x5640fb02ede0 2024-07-17T08:35:45.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.799+0000 7f28e0c00640 1 -- 172.21.15.44:0/2399404876 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e8004a20 con 0x5640fb02ede0 2024-07-17T08:35:45.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.799+0000 7f28e0c00640 1 -- 172.21.15.44:0/2399404876 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004e50 con 0x5640fb02ede0 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 -- 172.21.15.44:0/2399404876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 msgr2=0x5640fb017c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f28e8001f90 tx=0x7f28e800c4b0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 -- 172.21.15.44:0/2399404876 shutdown_connections 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02ede0 0x5640fb017c40 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb02b5b0 0x5640fb017700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2399404876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb0230c0 0x5640fb0028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 -- 172.21.15.44:0/2399404876 >> 172.21.15.44:0/2399404876 conn(0x5640fb0060a0 msgr2=0x5640fb005030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 -- 172.21.15.44:0/2399404876 shutdown_connections 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.800+0000 7f28fa83b740 1 -- 172.21.15.44:0/2399404876 wait complete. 2024-07-17T08:35:45.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.801+0000 7f28fa83b740 1 -- start start 2024-07-17T08:35:45.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.801+0000 7f28fa83b740 1 Processor -- start 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640fb024c60 con 0x5640fb02b5b0 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640fb027720 con 0x5640fb02ede0 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5640fb05cfc0 con 0x5640fb0230c0 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28fa83b740 10 client.0 _ll_register_callbacks cb 0x7f28e4033f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39828/0 (socket says 172.21.15.44:39828) 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 -- 172.21.15.44:0/2350754156 learned_addr learned my addr 172.21.15.44:0/2350754156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 -- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 msgr2=0x5640fb0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 -- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 msgr2=0x5640fb06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.802+0000 7f28efe00640 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28efe00640 1 -- 172.21.15.44:0/2350754156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640fb0000c0 con 0x5640fb02b5b0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28efe00640 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0058c0 tx=0x7f28dc00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62948e4e83c4fb8e server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28d6000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc035060 con 0x5640fb02b5b0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f28dc01cae0 con 0x5640fb02b5b0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e8004a20 con 0x5640fb02b5b0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f28dc03b050 con 0x5640fb02b5b0 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28fa83b740 10 client.10356 fetch_fsmap learned FSMap version 119 2024-07-17T08:35:45.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.803+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) -- 0x5640fb05be00 con 0x5640fb02b5b0 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e8004a20 con 0x5640fb02b5b0 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 119) ==== 4149+0+0 (secure 0 0 0) 0x7f28dc03e020 con 0x5640fb02b5b0 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 10 client.10356 fetch_fsmap finished waiting for FSMap version 119 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 10 client.10356 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 20 client.10356 populate_metadata read hostname 'smithi044' 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb059340 0x5640fb06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 4 client.10356 mds_command: new command op to 8329 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 --> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] -- command(tid 0: {"prefix": "status"}) -- 0x5640faf74750 con 0x5640fb059340 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28f5000640 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb059340 0x5640fb06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28f5000640 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb059340 0x5640fb06c380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d944569cc910f3 server_cookie=4811a048da1ebefa in_seq=0 out_seq=0 2024-07-17T08:35:45.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.804+0000 7f28d6000640 10 client.10356 ms_handle_connect on v2:172.21.15.44:6833/3288748832 2024-07-17T08:35:45.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28d6000640 1 -- 172.21.15.44:0/2350754156 <== mds.0 v2:172.21.15.44:6833/3288748832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5640faf74750 con 0x5640fb059340 2024-07-17T08:35:45.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28d6000640 10 client.10356 handle_command_reply: tid=0 2024-07-17T08:35:45.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 client.10356 shutdown 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 msgr2=0x5640fb06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0058c0 tx=0x7f28dc00b710 comp rx=0 tx=0).stop 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 shutdown_connections 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640fb02ede0 0x5640fb06dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640fb02b5b0 0x5640fb06d5a0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.44:6833/3288748832,v1:172.21.15.44:6834/3288748832] conn(0x5640fb059340 0x5640fb06c380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 --2- 172.21.15.44:0/2350754156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5640fb0230c0 0x5640fb0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28d6000640 0 client.10356 ms_handle_reset on v2:172.21.15.44:6833/3288748832 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 >> 172.21.15.44:0/2350754156 conn(0x5640fb0060a0 msgr2=0x5640fb027d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 shutdown_connections 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 1 -- 172.21.15.44:0/2350754156 wait complete. 2024-07-17T08:35:45.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.805+0000 7f28fa83b740 20 client.10356 trim_cache size 0 max 16384 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8329, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "backup_fs", 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 11.335736123, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 119, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 227, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 227, 2024-07-17T08:35:45.811 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 28.429677295000001 2024-07-17T08:35:45.812 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:35:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 msgr2=0x7f28d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 0x7f28d0079190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28f01f7530 tx=0x7f28dc00bec0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 msgr2=0x7f28f0071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009780 tx=0x7f28e4033d60 comp rx=0 tx=0).stop 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 shutdown_connections 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28f0152220 0x7f28f0073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f28d0076cd0 0x7f28d0079190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28f00cb3a0 0x7f28f00ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 --2- 172.21.15.44:0/3796253891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28f006ed30 0x7f28f0071fd0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 >> 172.21.15.44:0/3796253891 conn(0x7f28f00b0eb0 msgr2=0x7f28f015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 shutdown_connections 2024-07-17T08:35:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:45.806+0000 7f28f7a00640 1 -- 172.21.15.44:0/3796253891 wait complete. 2024-07-17T08:35:45.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 -- 172.21.15.44:0/1073400506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900ac8a0 msgr2=0x7f749015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 --2- 172.21.15.44:0/1073400506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900ac8a0 0x7f749015ac10 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f74900d0260 tx=0x7f748802a560 comp rx=0 tx=0).stop 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 -- 172.21.15.44:0/1073400506 shutdown_connections 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 --2- 172.21.15.44:0/1073400506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900ac8a0 0x7f749015ac10 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 --2- 172.21.15.44:0/1073400506 >> v2:172.21.15.175:3300/0 conn(0x7f749015a280 0x7f74900b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 -- 172.21.15.44:0/1073400506 >> 172.21.15.44:0/1073400506 conn(0x7f74900b0eb0 msgr2=0x7f74900b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 -- 172.21.15.44:0/1073400506 shutdown_connections 2024-07-17T08:35:46.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.019+0000 7f7498200640 1 -- 172.21.15.44:0/1073400506 wait complete. 2024-07-17T08:35:46.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.021+0000 7f7498200640 1 Processor -- start 2024-07-17T08:35:46.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.021+0000 7f7498200640 1 -- start start 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 0x7f7490071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749015a280 0x7f7490148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f749006d760 con 0x7f74900b1af0 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f749006f1d0 con 0x7f749015a280 2024-07-17T08:35:46.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.022+0000 7f7498200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f749015b150 con 0x7f74900ac8a0 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39848/0 (socket says 172.21.15.44:39848) 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 -- 172.21.15.44:0/1340864040 learned_addr learned my addr 172.21.15.44:0/1340864040 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7496200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749015a280 0x7f7490148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7495800640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 0x7f7490071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 -- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 msgr2=0x7f7490071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 0x7f7490071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 -- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749015a280 msgr2=0x7f7490148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749015a280 0x7f7490148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7494e00640 1 -- 172.21.15.44:0/1340864040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7490156740 con 0x7f74900b1af0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.023+0000 7f7495800640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 0x7f7490071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f7494e00640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c440 tx=0x7f747c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c017070 con 0x7f74900b1af0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f747c0035f0 con 0x7f74900b1af0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7490156a00 con 0x7f74900b1af0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c003a40 con 0x7f74900b1af0 2024-07-17T08:35:46.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.024+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7490154b60 con 0x7f74900b1af0 2024-07-17T08:35:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.026+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74900ae040 con 0x7f74900b1af0 2024-07-17T08:35:46.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.027+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f747c01a070 con 0x7f74900b1af0 2024-07-17T08:35:46.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.028+0000 7f748d600640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 0x7f7470079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.028+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f747c097f50 con 0x7f74900b1af0 2024-07-17T08:35:46.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.029+0000 7f7495800640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 0x7f7470079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.030+0000 7f7495800640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 0x7f7470079190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7484005c40 tx=0x7f748400a410 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:46.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.033+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f74900ae040 con 0x7f74900b1af0 2024-07-17T08:35:46.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.276+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7490156f20 con 0x7f7470076cd0 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.281+0000 7f748d600640 1 -- 172.21.15.44:0/1340864040 <== mgr.8179 v2:172.21.15.175:6834/2798563150 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+82 (secure 0 0 0) 0x7f7490156f20 con 0x7f7470076cd0 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "backup_fs" 2024-07-17T08:35:46.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:35:46.283 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 msgr2=0x7f7470079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 0x7f7470079190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7484005c40 tx=0x7f748400a410 comp rx=0 tx=0).stop 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 msgr2=0x7f74901595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c440 tx=0x7f747c00c900 comp rx=0 tx=0).stop 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 shutdown_connections 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f7470076cd0 0x7f7470079190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749015a280 0x7f7490148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74900b1af0 0x7f74901595f0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 --2- 172.21.15.44:0/1340864040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900ac8a0 0x7f7490071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 >> 172.21.15.44:0/1340864040 conn(0x7f74900b0eb0 msgr2=0x7f74900aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 shutdown_connections 2024-07-17T08:35:46.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.283+0000 7f7498200640 1 -- 172.21.15.44:0/1340864040 wait complete. 2024-07-17T08:35:46.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:35:46.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 -- 172.21.15.44:0/166571231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f80004800 msgr2=0x7f4f80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 --2- 172.21.15.44:0/166571231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f80004800 0x7f4f80004c80 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c260 tx=0x7f4f80005830 comp rx=0 tx=0).stop 2024-07-17T08:35:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4f9e600640 1 -- 172.21.15.44:0/166571231 reap_dead start 2024-07-17T08:35:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 -- 172.21.15.44:0/166571231 shutdown_connections 2024-07-17T08:35:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 -- 172.21.15.44:0/166571231 >> 172.21.15.44:0/166571231 conn(0x7f4f980b0eb0 msgr2=0x7f4f980b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 -- 172.21.15.44:0/166571231 shutdown_connections 2024-07-17T08:35:46.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.513+0000 7f4fa0600640 1 -- 172.21.15.44:0/166571231 wait complete. 2024-07-17T08:35:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.515+0000 7f4fa0600640 1 Processor -- start 2024-07-17T08:35:46.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.515+0000 7f4fa0600640 1 -- start start 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f80004800 0x7f4f9815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9815cb10 0x7f4f9815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f980cd2e0 con 0x7f4f981f1d20 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9806d8e0 con 0x7f4f9815cb10 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.516+0000 7f4fa0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980cd460 con 0x7f4f80004800 2024-07-17T08:35:46.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39876/0 (socket says 172.21.15.44:39876) 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 -- 172.21.15.44:0/288936151 learned_addr learned my addr 172.21.15.44:0/288936151 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9dc00640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f80004800 0x7f4f9815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9e600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9815cb10 0x7f4f9815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 -- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f80004800 msgr2=0x7f4f9815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f80004800 0x7f4f9815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 -- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9815cb10 msgr2=0x7f4f9815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9815cb10 0x7f4f9815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.517+0000 7f4f9d200640 1 -- 172.21.15.44:0/288936151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f981f2c50 con 0x7f4f981f1d20 2024-07-17T08:35:46.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4f9d200640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402b110 tx=0x7f4f84006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84031040 con 0x7f4f981f1d20 2024-07-17T08:35:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f840035f0 con 0x7f4f981f1d20 2024-07-17T08:35:46.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84003a00 con 0x7f4f981f1d20 2024-07-17T08:35:46.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9815f310 con 0x7f4f981f1d20 2024-07-17T08:35:46.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.518+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9815f8d0 con 0x7f4f981f1d20 2024-07-17T08:35:46.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.519+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f980b0060 con 0x7f4f981f1d20 2024-07-17T08:35:46.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.520+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 131) ==== 98931+0+0 (secure 0 0 0) 0x7f4f8403e030 con 0x7f4f981f1d20 2024-07-17T08:35:46.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.521+0000 7f4f95600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 0x7f4f78079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:46.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.522+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7f4f840b8870 con 0x7f4f981f1d20 2024-07-17T08:35:46.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.522+0000 7f4f9dc00640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 0x7f4f78079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:46.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.523+0000 7f4f9dc00640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 0x7f4f78079190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f98155d30 tx=0x7f4f8c005f90 comp rx=0 tx=0).ready entity=mgr.8179 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:46.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.526+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4f840813f0 con 0x7f4f981f1d20 2024-07-17T08:35:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:46.797+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4f980afd80 con 0x7f4f981f1d20 2024-07-17T08:35:47.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.565+0000 7f4f95600640 1 -- 172.21.15.44:0/288936151 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v132) ==== 92+0+0 (secure 0 0 0) 0x7f4f84080cf0 con 0x7f4f981f1d20 2024-07-17T08:35:47.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.569+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 msgr2=0x7f4f78079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:47.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.569+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 0x7f4f78079190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f98155d30 tx=0x7f4f8c005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:47.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.569+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 msgr2=0x7f4f981f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:47.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.569+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f4f8402b110 tx=0x7f4f84006d70 comp rx=0 tx=0).stop 2024-07-17T08:35:47.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 shutdown_connections 2024-07-17T08:35:47.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7f4f78076cd0 0x7f4f78079190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f9815cb10 0x7f4f9815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f981f1d20 0x7f4f981f2710 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 --2- 172.21.15.44:0/288936151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f80004800 0x7f4f9815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.570+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 >> 172.21.15.44:0/288936151 conn(0x7f4f980b0eb0 msgr2=0x7f4f980b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:47.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.571+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 shutdown_connections 2024-07-17T08:35:47.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.572+0000 7f4fa0600640 1 -- 172.21.15.44:0/288936151 wait complete. 2024-07-17T08:35:47.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:35:47.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 -- 172.21.15.44:0/2520376114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e80d0ca0 msgr2=0x7fb8e80b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 --2- 172.21.15.44:0/2520376114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e80b1cd0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009830 tx=0x7fb8d402a5c0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 -- 172.21.15.44:0/2520376114 shutdown_connections 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 --2- 172.21.15.44:0/2520376114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e80b1cd0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 -- 172.21.15.44:0/2520376114 >> 172.21.15.44:0/2520376114 conn(0x7fb8e80b5560 msgr2=0x7fb8e80b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 -- 172.21.15.44:0/2520376114 shutdown_connections 2024-07-17T08:35:47.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.785+0000 7fb8e7200640 1 -- 172.21.15.44:0/2520376114 wait complete. 2024-07-17T08:35:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.787+0000 7fb8e7200640 1 Processor -- start 2024-07-17T08:35:47.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.787+0000 7fb8e7200640 1 -- start start 2024-07-17T08:35:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8e80b2210 0x7fb8e81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:47.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80af1c0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80af040 con 0x7fb8e80d0ca0 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.788+0000 7fb8e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e80af340 con 0x7fb8e80b2210 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39910/0 (socket says 172.21.15.44:39910) 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 -- 172.21.15.44:0/3266041481 learned_addr learned my addr 172.21.15.44:0/3266041481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e5400640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:47.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e5e00640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8e80b2210 0x7fb8e81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:47.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8e80b2210 msgr2=0x7fb8e81570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:47.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8e80b2210 0x7fb8e81570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8e80d0ca0 msgr2=0x7fb8e8157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:47.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e8157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.789+0000 7fb8e6800640 1 -- 172.21.15.44:0/3266041481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e815e8c0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8e6800640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c710 tx=0x7fb8e000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0016020 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e000ce20 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e815e5e0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0006e60 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.790+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81559c0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.791+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e80af7d0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.793+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 132) ==== 98948+0+0 (secure 0 0 0) 0x7fb8e0003bd0 con 0x7fb8e815b3a0 2024-07-17T08:35:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.794+0000 7fb8de000640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 0x7fb8c8079260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.794+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fb8e0098870 con 0x7fb8e815b3a0 2024-07-17T08:35:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.794+0000 7fb8e5e00640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 msgr2=0x7fb8c8079260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:47.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.794+0000 7fb8e5e00640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 0x7fb8c8079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:47.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.798+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb8e0020020 con 0x7fb8e815b3a0 2024-07-17T08:35:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.995+0000 7fb8e5e00640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 msgr2=0x7fb8c8079260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:47.995+0000 7fb8e5e00640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 0x7fb8c8079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.008+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb8e8156e30 con 0x7fb8e815b3a0 2024-07-17T08:35:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.014+0000 7fb8de000640 1 -- 172.21.15.44:0/3266041481 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v73)=0 v73) ==== 138+0+0 (secure 0 0 0) 0x7fb8e0061570 con 0x7fb8e815b3a0 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 msgr2=0x7fb8c8079260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 0x7fb8c8079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 msgr2=0x7fb8e8157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c710 tx=0x7fb8e000cbd0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 shutdown_connections 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e815b3a0 0x7fb8e8157b50 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.016+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fb8c8076da0 0x7fb8c8079260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.017+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8e80d0ca0 0x7fb8e8157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.017+0000 7fb8e7200640 1 --2- 172.21.15.44:0/3266041481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8e80b2210 0x7fb8e81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.017+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 >> 172.21.15.44:0/3266041481 conn(0x7fb8e80b5560 msgr2=0x7fb8e80cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.017+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 shutdown_connections 2024-07-17T08:35:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.017+0000 7fb8e7200640 1 -- 172.21.15.44:0/3266041481 wait complete. 2024-07-17T08:35:48.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 -- 172.21.15.44:0/2124637026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc15d130 msgr2=0x7fecfc15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 --2- 172.21.15.44:0/2124637026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc15d130 0x7fecfc15d510 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fece4009830 tx=0x7fece402a510 comp rx=0 tx=0).stop 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 -- 172.21.15.44:0/2124637026 shutdown_connections 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 --2- 172.21.15.44:0/2124637026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc15d130 0x7fecfc15d510 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 -- 172.21.15.44:0/2124637026 >> 172.21.15.44:0/2124637026 conn(0x7fecfc0b4ff0 msgr2=0x7fecfc0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 -- 172.21.15.44:0/2124637026 shutdown_connections 2024-07-17T08:35:48.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.234+0000 7fed01800640 1 -- 172.21.15.44:0/2124637026 wait complete. 2024-07-17T08:35:48.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.236+0000 7fed01800640 1 Processor -- start 2024-07-17T08:35:48.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.236+0000 7fed01800640 1 -- start start 2024-07-17T08:35:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 0x7fecfc14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 0x7fecfc14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc168860 con 0x7fecfc0ce690 2024-07-17T08:35:48.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1686e0 con 0x7fecfc161be0 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed01800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecfc1689e0 con 0x7fecfc15d130 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39924/0 (socket says 172.21.15.44:39924) 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 -- 172.21.15.44:0/2671786101 learned_addr learned my addr 172.21.15.44:0/2671786101 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfb400640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 0x7fecfc14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed00e00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 0x7fecfc14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 msgr2=0x7fecfc14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 0x7fecfc14b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 msgr2=0x7fecfc14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 0x7fecfc14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfbe00640 1 -- 172.21.15.44:0/2671786101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc14e060 con 0x7fecfc0ce690 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fed00e00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 0x7fecfc14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:48.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.237+0000 7fecfb400640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 0x7fecfc14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fecfbe00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4040070 con 0x7fecfc0ce690 2024-07-17T08:35:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece40038e0 con 0x7fecfc0ce690 2024-07-17T08:35:48.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4003d30 con 0x7fecfc0ce690 2024-07-17T08:35:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc154be0 con 0x7fecfc0ce690 2024-07-17T08:35:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.238+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc151910 con 0x7fecfc0ce690 2024-07-17T08:35:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.240+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecfc15d510 con 0x7fecfc0ce690 2024-07-17T08:35:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.240+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 132) ==== 98948+0+0 (secure 0 0 0) 0x7fece4039040 con 0x7fecfc0ce690 2024-07-17T08:35:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.241+0000 7fecf3e00640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 0x7fecd8079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.241+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7fece40ba1c0 con 0x7fecfc0ce690 2024-07-17T08:35:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.242+0000 7fecfb400640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 msgr2=0x7fecd8079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:48.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.242+0000 7fecfb400640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 0x7fecd8079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:48.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.245+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fece404c030 con 0x7fecfc0ce690 2024-07-17T08:35:48.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.443+0000 7fecfb400640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 msgr2=0x7fecd8079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:48.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.443+0000 7fecfb400640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 0x7fecd8079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:48.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.480+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fecfc0d0e10 con 0x7fecfc0ce690 2024-07-17T08:35:48.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.485+0000 7fecf3e00640 1 -- 172.21.15.44:0/2671786101 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v74)=0 v74) ==== 133+0+0 (secure 0 0 0) 0x7fece4082c10 con 0x7fecfc0ce690 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 msgr2=0x7fecd8079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 0x7fecd8079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 msgr2=0x7fecfc15c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fece4000c00 tx=0x7fece4032c10 comp rx=0 tx=0).stop 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 shutdown_connections 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7fecd8076d50 0x7fecd8079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecfc161be0 0x7fecfc14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecfc15d130 0x7fecfc14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 --2- 172.21.15.44:0/2671786101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecfc0ce690 0x7fecfc15c730 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 >> 172.21.15.44:0/2671786101 conn(0x7fecfc0b4ff0 msgr2=0x7fecfc155800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:48.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 shutdown_connections 2024-07-17T08:35:48.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.488+0000 7fed01800640 1 -- 172.21.15.44:0/2671786101 wait complete. 2024-07-17T08:35:48.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:48.509 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2024-07-17T08:35:48.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 -- 172.21.15.44:0/3066782114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa015e370 msgr2=0x7faaa00ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 --2- 172.21.15.44:0/3066782114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa015e370 0x7faaa00ce500 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7faa94005fd0 tx=0x7faa9402a4b0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 -- 172.21.15.44:0/3066782114 shutdown_connections 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 --2- 172.21.15.44:0/3066782114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa015e370 0x7faaa00ce500 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 -- 172.21.15.44:0/3066782114 >> 172.21.15.44:0/3066782114 conn(0x7faaa0073a60 msgr2=0x7faaa0073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:48.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 -- 172.21.15.44:0/3066782114 shutdown_connections 2024-07-17T08:35:48.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.756+0000 7faaa6c00640 1 -- 172.21.15.44:0/3066782114 wait complete. 2024-07-17T08:35:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.759+0000 7faaa6c00640 1 Processor -- start 2024-07-17T08:35:48.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.759+0000 7faaa6c00640 1 -- start start 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 0x7faaa00c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaa015e370 0x7faaa00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0161b50 0x7faaa00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00b8d80 con 0x7faaa00d0b40 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00cec10 con 0x7faaa0161b50 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.760+0000 7faaa6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faaa00b8f00 con 0x7faaa015e370 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 0x7faaa00c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 0x7faaa00c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39942/0 (socket says 172.21.15.44:39942) 2024-07-17T08:35:48.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 -- 172.21.15.44:0/2183680112 learned_addr learned my addr 172.21.15.44:0/2183680112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa6200640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0161b50 0x7faaa00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaa015e370 0x7faaa00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaa015e370 msgr2=0x7faaa00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faaa015e370 0x7faaa00b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0161b50 msgr2=0x7faaa00b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0161b50 0x7faaa00b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa5800640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa01f7c80 con 0x7faaa00d0b40 2024-07-17T08:35:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.761+0000 7faaa6200640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0161b50 0x7faaa00b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faaa5800640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 0x7faaa00c8310 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9403f070 con 0x7faaa00d0b40 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa94004890 con 0x7faaa00d0b40 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa01f7f00 con 0x7faaa00d0b40 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94035030 con 0x7faaa00d0b40 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.762+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa01f41a0 con 0x7faaa00d0b40 2024-07-17T08:35:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.763+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa00b93f0 con 0x7faaa00d0b40 2024-07-17T08:35:48.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.764+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 132) ==== 98948+0+0 (secure 0 0 0) 0x7faa94042070 con 0x7faaa00d0b40 2024-07-17T08:35:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.766+0000 7faa9d600640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.766+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 7541+0+0 (secure 0 0 0) 0x7faa940b9b30 con 0x7faaa00d0b40 2024-07-17T08:35:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.767+0000 7faaa4e00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:48.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.767+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.769+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faaa01f7c80 con 0x7faaa00d0b40 2024-07-17T08:35:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.968+0000 7faaa4e00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:48.968+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:49.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:49.025+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) -- 0x7faaa00d1090 con 0x7faa80076d70 2024-07-17T08:35:49.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:49.369+0000 7faaa4e00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:49.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:49.369+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:35:50.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:50.171+0000 7faaa4e00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:50.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:50.171+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:35:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:51.773+0000 7faaa4e00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2798563150 2024-07-17T08:35:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:51.773+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:35:51.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:51.834+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 133) ==== 98557+0+0 (secure 0 0 0) 0x7faa9407e2b0 con 0x7faaa00d0b40 2024-07-17T08:35:51.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:51.836+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 msgr2=0x7faa80079230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:51.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:51.836+0000 7faa9d600640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/2798563150,v1:172.21.15.175:6837/2798563150] conn(0x7faa80076d70 0x7faa80079230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:52.389 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.390 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.388+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.408+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.408+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.408+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.408+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.408 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.408+0000 7fdda7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.409 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.409+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.410 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.409+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.410 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.409+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.410 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.409+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.410 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:35:52.409+0000 7fdda4c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:35:52.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:52.842+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 134) ==== 98684+0+0 (secure 0 0 0) 0x7faa940b7470 con 0x7faaa00d0b40 2024-07-17T08:35:52.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:52.843+0000 7faa9d600640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7faa800820a0 0x7faa80084490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:52.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:52.843+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) -- 0x7faa8000c8d0 con 0x7faa800820a0 2024-07-17T08:35:52.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:52.844+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7faa800820a0 0x7faa80084490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:52.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:52.848+0000 7faaa4e00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7faa800820a0 0x7faa80084490 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faaa00ca060 tx=0x7faa8c009040 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.030+0000 7faa9d600640 1 -- 172.21.15.44:0/2183680112 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+53 (secure 0 0 0) 0x7faa8000c8d0 con 0x7faa800820a0 2024-07-17T08:35:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path mandatory_fs_argument_test_dir 2024-07-17T08:35:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7faa800820a0 msgr2=0x7faa80084490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7faa800820a0 0x7faa80084490 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faaa00ca060 tx=0x7faa8c009040 comp rx=0 tx=0).stop 2024-07-17T08:35:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 msgr2=0x7faaa00c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 --2- 172.21.15.44:0/2183680112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faaa00d0b40 0x7faaa00c8310 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7faa94000c00 tx=0x7faa940039e0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6200640 1 -- 172.21.15.44:0/2183680112 reap_dead start 2024-07-17T08:35:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 shutdown_connections 2024-07-17T08:35:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.034+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 >> 172.21.15.44:0/2183680112 conn(0x7faaa0073a60 msgr2=0x7faaa015c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.035+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 shutdown_connections 2024-07-17T08:35:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.035+0000 7faaa6c00640 1 -- 172.21.15.44:0/2183680112 wait complete. 2024-07-17T08:35:53.056 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2024-07-17T08:35:53.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.275+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1343883036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7a8003ad0 msgr2=0x7fe7a8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.275+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1343883036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7a8003ad0 0x7fe7a8003ef0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4001d90 tx=0x7fe7a8005830 comp rx=0 tx=0).stop 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.275+0000 7fe7c3e00640 1 -- 172.21.15.44:0/1343883036 reap_dead start 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.276+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1343883036 shutdown_connections 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.276+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1343883036 >> 172.21.15.44:0/1343883036 conn(0x7fe7c406c360 msgr2=0x7fe7c406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.276+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1343883036 shutdown_connections 2024-07-17T08:35:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.276+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1343883036 wait complete. 2024-07-17T08:35:53.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.277+0000 7fe7c8a00640 1 Processor -- start 2024-07-17T08:35:53.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.277+0000 7fe7c8a00640 1 -- start start 2024-07-17T08:35:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8003ad0 0x7fe7c414f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4072d20 0x7fe7c4152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c40b2330 con 0x7fe7c40b0b80 2024-07-17T08:35:53.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c40b21b0 con 0x7fe7a8003ad0 2024-07-17T08:35:53.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.278+0000 7fe7c8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c40cfa10 con 0x7fe7c4072d20 2024-07-17T08:35:53.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57330/0 (socket says 172.21.15.44:57330) 2024-07-17T08:35:53.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 -- 172.21.15.44:0/1832160510 learned_addr learned my addr 172.21.15.44:0/1832160510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c2a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4072d20 0x7fe7c4152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3400640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8003ad0 0x7fe7c414f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 -- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4072d20 msgr2=0x7fe7c4152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4072d20 0x7fe7c4152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 -- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8003ad0 msgr2=0x7fe7c414f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8003ad0 0x7fe7c414f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.279+0000 7fe7c3e00640 1 -- 172.21.15.44:0/1832160510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c414fd00 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7c3e00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe7c406d770 tx=0x7fe7bc00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc016070 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7bc010990 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c40732c0 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc010de0 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.280+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c41f76a0 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.281+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c41f71b0 con 0x7fe7c40b0b80 2024-07-17T08:35:53.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.282+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 134) ==== 98684+0+0 (secure 0 0 0) 0x7fe7bc019070 con 0x7fe7c40b0b80 2024-07-17T08:35:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.284+0000 7fe7bb400640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 0x7fe7a4078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.284+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fe7bc098980 con 0x7fe7c40b0b80 2024-07-17T08:35:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.284+0000 7fe7c3400640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 0x7fe7a4078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.286+0000 7fe7c3400640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 0x7fe7a4078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4032000 tx=0x7fe7b4001510 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:53.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.291+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7bc061840 con 0x7fe7c40b0b80 2024-07-17T08:35:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.532+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) -- 0x7fe7c415e070 con 0x7fe7a4076a50 2024-07-17T08:35:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.692+0000 7fe7bb400640 1 -- 172.21.15.44:0/1832160510 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe7c415e070 con 0x7fe7a4076a50 2024-07-17T08:35:53.693 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.695+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 msgr2=0x7fe7a4078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.695+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 0x7fe7a4078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4032000 tx=0x7fe7b4001510 comp rx=0 tx=0).stop 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.695+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 msgr2=0x7fe7c40b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe7c406d770 tx=0x7fe7bc00ea30 comp rx=0 tx=0).stop 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 shutdown_connections 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7a4076a50 0x7fe7a4078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40b0b80 0x7fe7c40b1000 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4072d20 0x7fe7c4152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 --2- 172.21.15.44:0/1832160510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7a8003ad0 0x7fe7c414f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 >> 172.21.15.44:0/1832160510 conn(0x7fe7c406c360 msgr2=0x7fe7c406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 shutdown_connections 2024-07-17T08:35:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.696+0000 7fe7c8a00640 1 -- 172.21.15.44:0/1832160510 wait complete. 2024-07-17T08:35:53.721 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2024-07-17T08:35:53.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:53.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 -- 172.21.15.44:0/3998538813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 msgr2=0x7f71c006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 --2- 172.21.15.44:0/3998538813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c006bf40 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 -- 172.21.15.44:0/3998538813 shutdown_connections 2024-07-17T08:35:53.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 --2- 172.21.15.44:0/3998538813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c006bf40 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 --2- 172.21.15.44:0/3998538813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 0x7f71c015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 -- 172.21.15.44:0/3998538813 >> 172.21.15.44:0/3998538813 conn(0x7f71c006d050 msgr2=0x7f71c006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:53.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 -- 172.21.15.44:0/3998538813 shutdown_connections 2024-07-17T08:35:53.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.943+0000 7f71c5c00640 1 -- 172.21.15.44:0/3998538813 wait complete. 2024-07-17T08:35:53.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.945+0000 7f71c5c00640 1 Processor -- start 2024-07-17T08:35:53.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.945+0000 7f71c5c00640 1 -- start start 2024-07-17T08:35:53.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 0x7f71c01f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c013e860 0x7f71c00b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71c006c810 con 0x7f71c015f9f0 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71c006c690 con 0x7f71c013e860 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71c5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71c01410e0 con 0x7f71c00afa90 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.946+0000 7f71bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57348/0 (socket says 172.21.15.44:57348) 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 -- 172.21.15.44:0/205805711 learned_addr learned my addr 172.21.15.44:0/205805711 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71be000640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c013e860 0x7f71c00b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bea00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 0x7f71c01f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 -- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 msgr2=0x7f71c01f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 0x7f71c01f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 -- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c013e860 msgr2=0x7f71c00b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:53.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c013e860 0x7f71c00b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:53.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 -- 172.21.15.44:0/205805711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c013de30 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.947+0000 7f71bf400640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c770 tx=0x7f71b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.948+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8003b70 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.948+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71b8003d10 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.948+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800aab0 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.948+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01e5970 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.948+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01e5f30 con 0x7f71c015f9f0 2024-07-17T08:35:53.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.949+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c015f810 con 0x7f71c015f9f0 2024-07-17T08:35:53.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.950+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 135) ==== 99051+0+0 (secure 0 0 0) 0x7f71b800ac50 con 0x7f71c015f9f0 2024-07-17T08:35:53.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.952+0000 7f71b6a00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 0x7f719c079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:53.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.952+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f71b8098e10 con 0x7f71c015f9f0 2024-07-17T08:35:53.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.952+0000 7f71bea00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 0x7f719c079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:53.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.953+0000 7f71bea00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 0x7f719c079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009bd0 tx=0x7f71b0005f90 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:53.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:53.956+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f71c015f810 con 0x7f71c015f9f0 2024-07-17T08:35:54.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.193+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) -- 0x7f71c006ca70 con 0x7f719c076e40 2024-07-17T08:35:54.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.369+0000 7f71b6a00640 1 -- 172.21.15.44:0/205805711 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+55 (secure 0 0 0) 0x7f71c006ca70 con 0x7f719c076e40 2024-07-17T08:35:54.370 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path mandatory_fs_argument_test_dir 2024-07-17T08:35:54.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.372+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 msgr2=0x7f719c079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.372+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 0x7f719c079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009bd0 tx=0x7f71b0005f90 comp rx=0 tx=0).stop 2024-07-17T08:35:54.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.372+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 msgr2=0x7f71c01544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.372+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c770 tx=0x7f71b800cc30 comp rx=0 tx=0).stop 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 shutdown_connections 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f719c076e40 0x7f719c079300 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c015f9f0 0x7f71c01544b0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c013e860 0x7f71c00b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 --2- 172.21.15.44:0/205805711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c00afa90 0x7f71c01f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 >> 172.21.15.44:0/205805711 conn(0x7f71c006d050 msgr2=0x7f71c015e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 shutdown_connections 2024-07-17T08:35:54.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.373+0000 7f71c5c00640 1 -- 172.21.15.44:0/205805711 wait complete. 2024-07-17T08:35:54.396 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2024-07-17T08:35:54.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.611+0000 7fcebf200640 1 -- 172.21.15.44:0/2922672726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 msgr2=0x7fceb80cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.611+0000 7fcebf200640 1 --2- 172.21.15.44:0/2922672726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb80cd300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcea4005fd0 tx=0x7fcea402a540 comp rx=0 tx=0).stop 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 -- 172.21.15.44:0/2922672726 shutdown_connections 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 --2- 172.21.15.44:0/2922672726 >> v2:172.21.15.175:3300/0 conn(0x7fceb80cb880 0x7fceb80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 --2- 172.21.15.44:0/2922672726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb80cd300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 -- 172.21.15.44:0/2922672726 >> 172.21.15.44:0/2922672726 conn(0x7fceb80b0eb0 msgr2=0x7fceb80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 -- 172.21.15.44:0/2922672726 shutdown_connections 2024-07-17T08:35:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.612+0000 7fcebf200640 1 -- 172.21.15.44:0/2922672726 wait complete. 2024-07-17T08:35:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.614+0000 7fcebf200640 1 Processor -- start 2024-07-17T08:35:54.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.614+0000 7fcebf200640 1 -- start start 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cb880 0x7fceb81506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fceb815b630 con 0x7fceb815b060 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceb815c470 con 0x7fceb80cb880 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.615+0000 7fcebf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fceb80cdd80 con 0x7fceb806dd30 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57352/0 (socket says 172.21.15.44:57352) 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 -- 172.21.15.44:0/1573251245 learned_addr learned my addr 172.21.15.44:0/1573251245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fceb7e00640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fceb7400640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cb880 0x7fceb81506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 -- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 msgr2=0x7fceb8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 -- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cb880 msgr2=0x7fceb81506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cb880 0x7fceb81506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.616+0000 7fcebd200640 1 -- 172.21.15.44:0/1573251245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb8159400 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fcebd200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fceb000eb70 tx=0x7fceb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0004d20 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb81596e0 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fceb000ce70 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0005260 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.617+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb8159ca0 con 0x7fceb815b060 2024-07-17T08:35:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.619+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb80ae490 con 0x7fceb815b060 2024-07-17T08:35:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.620+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 135) ==== 99051+0+0 (secure 0 0 0) 0x7fceb0005400 con 0x7fceb815b060 2024-07-17T08:35:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.621+0000 7fceafe00640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 0x7fce98079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.621+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fceb0099f50 con 0x7fceb815b060 2024-07-17T08:35:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.621+0000 7fceb7e00640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 0x7fce98079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:54.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.623+0000 7fceb7e00640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 0x7fce98079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcea80015e0 tx=0x7fcea8009040 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:54.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.625+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fceb80ae490 con 0x7fceb815b060 2024-07-17T08:35:54.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.860+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fceb80ce1d0 con 0x7fce98076e40 2024-07-17T08:35:54.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.878+0000 7fceafe00640 1 -- 172.21.15.44:0/1573251245 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+88 (secure 0 0 0) 0x7fceb80ce1d0 con 0x7fce98076e40 2024-07-17T08:35:54.879 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:54.879 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.880+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 msgr2=0x7fce98079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.880+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 0x7fce98079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcea80015e0 tx=0x7fcea8009040 comp rx=0 tx=0).stop 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.880+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 msgr2=0x7fceb8073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.880+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fceb000eb70 tx=0x7fceb000c6a0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 shutdown_connections 2024-07-17T08:35:54.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fce98076e40 0x7fce98079300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb815b060 0x7fceb8073980 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cb880 0x7fceb81506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 --2- 172.21.15.44:0/1573251245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb806dd30 0x7fceb8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 >> 172.21.15.44:0/1573251245 conn(0x7fceb80b0eb0 msgr2=0x7fceb80aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 shutdown_connections 2024-07-17T08:35:54.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:54.881+0000 7fcebf200640 1 -- 172.21.15.44:0/1573251245 wait complete. 2024-07-17T08:35:54.904 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2024-07-17T08:35:54.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 -- 172.21.15.44:0/608951684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c161b50 msgr2=0x7f594c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 --2- 172.21.15.44:0/608951684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c161b50 0x7f594c0d0430 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f5934005fd0 tx=0x7f593402a4b0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 -- 172.21.15.44:0/608951684 shutdown_connections 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 --2- 172.21.15.44:0/608951684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c161b50 0x7f594c0d0430 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 -- 172.21.15.44:0/608951684 >> 172.21.15.44:0/608951684 conn(0x7f594c073a60 msgr2=0x7f594c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 -- 172.21.15.44:0/608951684 shutdown_connections 2024-07-17T08:35:55.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.124+0000 7f5952600640 1 -- 172.21.15.44:0/608951684 wait complete. 2024-07-17T08:35:55.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.126+0000 7f5952600640 1 Processor -- start 2024-07-17T08:35:55.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.126+0000 7f5952600640 1 -- start start 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 0x7f594c0b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 0x7f594c0b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f594c0b8f00 con 0x7f594c0d0970 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f594c0b8d80 con 0x7f594c161b50 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5952600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f594c0b9080 con 0x7f594c15e370 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5951200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5951200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57364/0 (socket says 172.21.15.44:57364) 2024-07-17T08:35:55.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.127+0000 7f5951200640 1 -- 172.21.15.44:0/3509896106 learned_addr learned my addr 172.21.15.44:0/3509896106 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5943e00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 0x7f594c0b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951c00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 0x7f594c0b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 -- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 msgr2=0x7f594c0b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 0x7f594c0b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 -- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 msgr2=0x7f594c0b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 0x7f594c0b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 -- 172.21.15.44:0/3509896106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c0cc440 con 0x7f594c0d0970 2024-07-17T08:35:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5943e00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 0x7f594c0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951c00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 0x7f594c0b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:55.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5951200640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c440 tx=0x7f593c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.128+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c017070 con 0x7f594c0d0970 2024-07-17T08:35:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.129+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f593c0035f0 con 0x7f594c0d0970 2024-07-17T08:35:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.129+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c003a40 con 0x7f594c0d0970 2024-07-17T08:35:55.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.129+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c0cc720 con 0x7f594c0d0970 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.129+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c0ccce0 con 0x7f594c0d0970 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.130+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594c0d0430 con 0x7f594c0d0970 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.131+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 135) ==== 99051+0+0 (secure 0 0 0) 0x7f593c01a070 con 0x7f594c0d0970 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.132+0000 7f5949600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 0x7f5928079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.132+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f593c098460 con 0x7f594c0d0970 2024-07-17T08:35:55.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.133+0000 7f5943e00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 0x7f5928079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.134+0000 7f5943e00640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 0x7f5928079300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f5934004730 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:55.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.138+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f593c060d40 con 0x7f594c0d0970 2024-07-17T08:35:55.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.383+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f594c069460 con 0x7f5928076e40 2024-07-17T08:35:55.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.400+0000 7f5949600640 1 -- 172.21.15.44:0/3509896106 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+353 (secure 0 0 0) 0x7f594c069460 con 0x7f5928076e40 2024-07-17T08:35:55.402 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:35:55.402 INFO:teuthology.orchestra.run.smithi044.stdout:[{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:35:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-07-17T08:35:55.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.403+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 msgr2=0x7f5928079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.403+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 0x7f5928079300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f5934004730 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.403+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 msgr2=0x7f594c0b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.403+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c440 tx=0x7f593c00c900 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 shutdown_connections 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f5928076e40 0x7f5928079300 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c161b50 0x7f594c0b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f594c15e370 0x7f594c0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 --2- 172.21.15.44:0/3509896106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f594c0d0970 0x7f594c0b4aa0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 >> 172.21.15.44:0/3509896106 conn(0x7f594c073a60 msgr2=0x7f594c15bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:55.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 shutdown_connections 2024-07-17T08:35:55.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.404+0000 7f5952600640 1 -- 172.21.15.44:0/3509896106 wait complete. 2024-07-17T08:35:55.423 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2024-07-17T00:00:00", "created": "2024-07-17T08:35:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2024-07-17T08:35:55.424 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:55.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.646+0000 7f16cb200640 1 -- 172.21.15.44:0/2859687800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc15a450 msgr2=0x7f16cc0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.646+0000 7f16cb200640 1 --2- 172.21.15.44:0/2859687800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc15a450 0x7f16cc0b2eb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005fd0 tx=0x7f16b402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 -- 172.21.15.44:0/2859687800 shutdown_connections 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 --2- 172.21.15.44:0/2859687800 >> v2:172.21.15.175:3300/0 conn(0x7f16cc0b33f0 0x7f16cc06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 --2- 172.21.15.44:0/2859687800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc15a450 0x7f16cc0b2eb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 -- 172.21.15.44:0/2859687800 >> 172.21.15.44:0/2859687800 conn(0x7f16cc0cf0d0 msgr2=0x7f16cc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 -- 172.21.15.44:0/2859687800 shutdown_connections 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.647+0000 7f16cb200640 1 -- 172.21.15.44:0/2859687800 wait complete. 2024-07-17T08:35:55.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.649+0000 7f16cb200640 1 Processor -- start 2024-07-17T08:35:55.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.649+0000 7f16cb200640 1 -- start start 2024-07-17T08:35:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc0b33f0 0x7f16cc14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc1503c0 0x7f16cc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc06dcf0 con 0x7f16cc15a450 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc06db70 con 0x7f16cc0b33f0 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16cc06f780 con 0x7f16cc1503c0 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57376/0 (socket says 172.21.15.44:57376) 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16ca800640 1 -- 172.21.15.44:0/2018962173 learned_addr learned my addr 172.21.15.44:0/2018962173 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16c3e00640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc1503c0 0x7f16cc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.650+0000 7f16c9e00640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc0b33f0 0x7f16cc14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 -- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc1503c0 msgr2=0x7f16cc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc1503c0 0x7f16cc0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 -- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc0b33f0 msgr2=0x7f16cc14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:55.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc0b33f0 0x7f16cc14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:55.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 -- 172.21.15.44:0/2018962173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc0ccd90 con 0x7f16cc15a450 2024-07-17T08:35:55.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16ca800640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f16c400a660 tx=0x7f16c400ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4018020 con 0x7f16cc15a450 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16c4004d20 con 0x7f16cc15a450 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc0cd070 con 0x7f16cc15a450 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.651+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c400ca10 con 0x7f16cc15a450 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.652+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc0cd630 con 0x7f16cc15a450 2024-07-17T08:35:55.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.653+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc0cffe0 con 0x7f16cc15a450 2024-07-17T08:35:55.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.655+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 135) ==== 99051+0+0 (secure 0 0 0) 0x7f16c400cbb0 con 0x7f16cc15a450 2024-07-17T08:35:55.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.656+0000 7f16c2000640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 0x7f16a8079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:55.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.656+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f16c4099200 con 0x7f16cc15a450 2024-07-17T08:35:55.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.657+0000 7f16c9e00640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 0x7f16a8079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:55.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.658+0000 7f16c9e00640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 0x7f16a8079300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16bc001810 tx=0x7f16bc009040 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:55.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.661+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f16c4022020 con 0x7f16cc15a450 2024-07-17T08:35:55.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.900+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) -- 0x7f16cc1ea020 con 0x7f16a8076e40 2024-07-17T08:35:55.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:55.909+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 136) ==== 99107+0+0 (secure 0 0 0) 0x7f16c405d4e0 con 0x7f16cc15a450 2024-07-17T08:35:56.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.087+0000 7f16c2000640 1 -- 172.21.15.44:0/2018962173 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f16cc1ea020 con 0x7f16a8076e40 2024-07-17T08:35:56.089 INFO:teuthology.orchestra.run.smithi044.stdout:Retention removed from path mandatory_fs_argument_test_dir 2024-07-17T08:35:56.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 msgr2=0x7f16a8079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 0x7f16a8079300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16bc001810 tx=0x7f16bc009040 comp rx=0 tx=0).stop 2024-07-17T08:35:56.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 msgr2=0x7f16cc06df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f16c400a660 tx=0x7f16c400ab20 comp rx=0 tx=0).stop 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 shutdown_connections 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f16a8076e40 0x7f16a8079300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc15a450 0x7f16cc06df00 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16cc1503c0 0x7f16cc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 --2- 172.21.15.44:0/2018962173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc0b33f0 0x7f16cc14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 >> 172.21.15.44:0/2018962173 conn(0x7f16cc0cf0d0 msgr2=0x7f16cc068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:56.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 shutdown_connections 2024-07-17T08:35:56.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.091+0000 7f16cb200640 1 -- 172.21.15.44:0/2018962173 wait complete. 2024-07-17T08:35:56.116 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2024-07-17T08:35:56.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.342+0000 7fde20a00640 1 -- 172.21.15.44:0/3552106217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 msgr2=0x7fde1c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.342+0000 7fde20a00640 1 --2- 172.21.15.44:0/3552106217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c1539f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fde04009830 tx=0x7fde0402a520 comp rx=0 tx=0).stop 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 -- 172.21.15.44:0/3552106217 shutdown_connections 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 --2- 172.21.15.44:0/3552106217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c0ac3a0 0x7fde1c0ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 --2- 172.21.15.44:0/3552106217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c1539f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 -- 172.21.15.44:0/3552106217 >> 172.21.15.44:0/3552106217 conn(0x7fde1c0b0eb0 msgr2=0x7fde1c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 -- 172.21.15.44:0/3552106217 shutdown_connections 2024-07-17T08:35:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.343+0000 7fde20a00640 1 -- 172.21.15.44:0/3552106217 wait complete. 2024-07-17T08:35:56.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.344+0000 7fde20a00640 1 Processor -- start 2024-07-17T08:35:56.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.345+0000 7fde20a00640 1 -- start start 2024-07-17T08:35:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c06f6d0 0x7fde1c1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1644c0 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c164340 con 0x7fde1c15a5d0 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde20a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde1c164640 con 0x7fde1c06f6d0 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57380/0 (socket says 172.21.15.44:57380) 2024-07-17T08:35:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19400640 1 -- 172.21.15.44:0/1025596229 learned_addr learned my addr 172.21.15.44:0/1025596229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19e00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c06f6d0 0x7fde1c1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde1a800640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19400640 1 -- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c06f6d0 msgr2=0x7fde1c1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.346+0000 7fde19400640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c06f6d0 0x7fde1c1420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde19400640 1 -- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 msgr2=0x7fde1c141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde19400640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde19400640 1 -- 172.21.15.44:0/1025596229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c0aea70 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde1a800640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde19400640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fde0402ab10 tx=0x7fde04033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04005c90 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde04031e50 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04005260 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c159c70 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.347+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c0b0430 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.348+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c0ae8a0 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.350+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 136) ==== 99107+0+0 (secure 0 0 0) 0x7fde04005400 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.351+0000 7fde12000640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 0x7fddf80793c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.351+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fde040ba090 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.351+0000 7fde19e00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 0x7fddf80793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.353+0000 7fde19e00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 0x7fddf80793c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde1c159920 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.355+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fde0403f030 con 0x7fde1c0ac3a0 2024-07-17T08:35:56.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.594+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) -- 0x7fde1c0ac810 con 0x7fddf8076f00 2024-07-17T08:35:56.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.766+0000 7fde12000640 1 -- 172.21.15.44:0/1025596229 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+61 (secure 0 0 0) 0x7fde1c0ac810 con 0x7fddf8076f00 2024-07-17T08:35:56.767 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2024-07-17T08:35:56.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 msgr2=0x7fddf80793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 0x7fddf80793c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde1c159920 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 msgr2=0x7fde1c1504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fde0402ab10 tx=0x7fde04033c20 comp rx=0 tx=0).stop 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 shutdown_connections 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fddf8076f00 0x7fddf80793c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c15a5d0 0x7fde1c141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde1c0ac3a0 0x7fde1c1504d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 --2- 172.21.15.44:0/1025596229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde1c06f6d0 0x7fde1c1420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 >> 172.21.15.44:0/1025596229 conn(0x7fde1c0b0eb0 msgr2=0x7fde1c151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 shutdown_connections 2024-07-17T08:35:56.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:56.770+0000 7fde20a00640 1 -- 172.21.15.44:0/1025596229 wait complete. 2024-07-17T08:35:56.791 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2024-07-17T08:35:56.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2024-07-17T08:35:57.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 -- 172.21.15.44:0/3630998577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc0cfcb0 msgr2=0x7f29dc06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 --2- 172.21.15.44:0/3630998577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc0cfcb0 0x7f29dc06ef00 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f29d4005fd0 tx=0x7f29d402aa00 comp rx=0 tx=0).stop 2024-07-17T08:35:57.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 -- 172.21.15.44:0/3630998577 shutdown_connections 2024-07-17T08:35:57.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 --2- 172.21.15.44:0/3630998577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc0cfcb0 0x7f29dc06ef00 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 --2- 172.21.15.44:0/3630998577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29dc152180 0x7f29dc0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 -- 172.21.15.44:0/3630998577 >> 172.21.15.44:0/3630998577 conn(0x7f29dc06c360 msgr2=0x7f29dc06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 -- 172.21.15.44:0/3630998577 shutdown_connections 2024-07-17T08:35:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.009+0000 7f29e0c00640 1 -- 172.21.15.44:0/3630998577 wait complete. 2024-07-17T08:35:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.011+0000 7f29e0c00640 1 Processor -- start 2024-07-17T08:35:57.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.011+0000 7f29e0c00640 1 -- start start 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29dc0cfcb0 0x7f29dc09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29dc152180 0x7f29dc073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc15bc90 con 0x7f29dc06eac0 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc15bb10 con 0x7f29dc0cfcb0 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29e0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29dc15be10 con 0x7f29dc152180 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57396/0 (socket says 172.21.15.44:57396) 2024-07-17T08:35:57.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.012+0000 7f29db400640 1 -- 172.21.15.44:0/1281034779 learned_addr learned my addr 172.21.15.44:0/1281034779 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 -- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29dc152180 msgr2=0x7f29dc073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29dc152180 0x7f29dc073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 -- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29dc0cfcb0 msgr2=0x7f29dc09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29dc0cfcb0 0x7f29dc09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 -- 172.21.15.44:0/1281034779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc142120 con 0x7f29dc06eac0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29db400640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c970 tx=0x7f29cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc007bb0 con 0x7f29dc06eac0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29cc005020 con 0x7f29dc06eac0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc007e60 con 0x7f29dc06eac0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.013+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1423c0 con 0x7f29dc06eac0 2024-07-17T08:35:57.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.014+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc142980 con 0x7f29dc06eac0 2024-07-17T08:35:57.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.015+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc06ef00 con 0x7f29dc06eac0 2024-07-17T08:35:57.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.016+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 136) ==== 99107+0+0 (secure 0 0 0) 0x7f29cc005470 con 0x7f29dc06eac0 2024-07-17T08:35:57.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.017+0000 7f29d3400640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 0x7f29bc079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.017+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f29cc099680 con 0x7f29dc06eac0 2024-07-17T08:35:57.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.018+0000 7f29daa00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 0x7f29bc079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.019+0000 7f29daa00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 0x7f29bc079390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29dc1430a0 tx=0x7f29c80079b0 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:57.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.022+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f29cc014030 con 0x7f29dc06eac0 2024-07-17T08:35:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.263+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 --> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) -- 0x7f29dc069450 con 0x7f29bc076ed0 2024-07-17T08:35:57.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.470+0000 7f29d3400640 1 -- 172.21.15.44:0/1281034779 <== mgr.8485 v2:172.21.15.175:6834/1605633467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f29dc069450 con 0x7f29bc076ed0 2024-07-17T08:35:57.471 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2024-07-17T08:35:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 msgr2=0x7f29bc079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 0x7f29bc079390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29dc1430a0 tx=0x7f29c80079b0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 msgr2=0x7f29dc1f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c970 tx=0x7f29cc00ce30 comp rx=0 tx=0).stop 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 shutdown_connections 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f29bc076ed0 0x7f29bc079390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29dc152180 0x7f29dc073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29dc0cfcb0 0x7f29dc09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 --2- 172.21.15.44:0/1281034779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29dc06eac0 0x7f29dc1f2100 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.474+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 >> 172.21.15.44:0/1281034779 conn(0x7f29dc06c360 msgr2=0x7f29dc15db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.475+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 shutdown_connections 2024-07-17T08:35:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.475+0000 7f29e0c00640 1 -- 172.21.15.44:0/1281034779 wait complete. 2024-07-17T08:35:57.501 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2024-07-17T08:35:57.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:57.501 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls mandatory_fs_argument_test_dir/.snap) 2024-07-17T08:35:57.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:57.559 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2024-07-17T08:35:57.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 -- 172.21.15.44:0/4026229209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 msgr2=0x7f988c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 --2- 172.21.15.44:0/4026229209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f988c004c80 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f989000baf0 tx=0x7f988c005830 comp rx=0 tx=0).stop 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 -- 172.21.15.44:0/4026229209 shutdown_connections 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 --2- 172.21.15.44:0/4026229209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f988c004c80 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 -- 172.21.15.44:0/4026229209 >> 172.21.15.44:0/4026229209 conn(0x7f98a406c120 msgr2=0x7f98a406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 -- 172.21.15.44:0/4026229209 shutdown_connections 2024-07-17T08:35:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.824+0000 7f98aa800640 1 -- 172.21.15.44:0/4026229209 wait complete. 2024-07-17T08:35:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.826+0000 7f98aa800640 1 Processor -- start 2024-07-17T08:35:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.826+0000 7f98aa800640 1 -- start start 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98a4150db0 0x7f98a41e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 0x7f98a41e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4156ec0 con 0x7f988c004800 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4156d40 con 0x7f98a4151210 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98aa800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98a4157040 con 0x7f98a4150db0 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57428/0 (socket says 172.21.15.44:57428) 2024-07-17T08:35:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.827+0000 7f98a9400640 1 -- 172.21.15.44:0/876734948 learned_addr learned my addr 172.21.15.44:0/876734948 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a8a00640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98a4150db0 0x7f98a41e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9e00640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 0x7f98a41e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 -- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98a4150db0 msgr2=0x7f98a41e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98a4150db0 0x7f98a41e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 -- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 msgr2=0x7f98a41e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 0x7f98a41e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 -- 172.21.15.44:0/876734948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40b1400 con 0x7f988c004800 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9e00640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 0x7f98a41e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a9400640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f989800d800 tx=0x7f989800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898018070 con 0x7f988c004800 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98980052e0 con 0x7f988c004800 2024-07-17T08:35:57.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989800b900 con 0x7f988c004800 2024-07-17T08:35:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a40b1c80 con 0x7f988c004800 2024-07-17T08:35:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.828+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a40b19a0 con 0x7f988c004800 2024-07-17T08:35:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.830+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a406a1a0 con 0x7f988c004800 2024-07-17T08:35:57.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.831+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 136) ==== 99107+0+0 (secure 0 0 0) 0x7f989801b070 con 0x7f988c004800 2024-07-17T08:35:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.832+0000 7f98a1600640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 0x7f9884079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:57.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.832+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f98980987a0 con 0x7f988c004800 2024-07-17T08:35:57.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.832+0000 7f98a8a00640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 0x7f9884079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:57.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.834+0000 7f98a8a00640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 0x7f9884079390 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f98900046c0 comp rx=0 tx=0).ready entity=mgr.8485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:57.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:57.836+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98a406a1a0 con 0x7f988c004800 2024-07-17T08:35:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.104+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f98a40cf610 con 0x7f988c004800 2024-07-17T08:35:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.931+0000 7f98a1600640 1 -- 172.21.15.44:0/876734948 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v137) ==== 93+0+0 (secure 0 0 0) 0x7f98980692a0 con 0x7f988c004800 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 msgr2=0x7f9884079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 0x7f9884079390 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f98900046c0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 msgr2=0x7f98a4149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f989800d800 tx=0x7f989800dcc0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 shutdown_connections 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f9884076ed0 0x7f9884079390 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98a4151210 0x7f98a41e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98a4150db0 0x7f98a41e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 --2- 172.21.15.44:0/876734948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c004800 0x7f98a4149040 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 >> 172.21.15.44:0/876734948 conn(0x7f98a406c120 msgr2=0x7f98a41e6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.935+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 shutdown_connections 2024-07-17T08:35:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:58.936+0000 7f98aa800640 1 -- 172.21.15.44:0/876734948 wait complete. 2024-07-17T08:35:58.961 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:35:58.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:59.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:35:59.144 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:35:59.144 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:59.145 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:59.145 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:59.193 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:59.193 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:59.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.537+0000 7fe7da000640 1 -- 172.21.15.44:0/2237178836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d40ac190 msgr2=0x7fe7d40ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.537+0000 7fe7da000640 1 --2- 172.21.15.44:0/2237178836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d40ac190 0x7fe7d40ac640 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc005fd0 tx=0x7fe7cc02a840 comp rx=0 tx=0).stop 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 -- 172.21.15.44:0/2237178836 shutdown_connections 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 --2- 172.21.15.44:0/2237178836 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d40ac190 0x7fe7d40ac640 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 --2- 172.21.15.44:0/2237178836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4154600 0x7fe7d40b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 -- 172.21.15.44:0/2237178836 >> 172.21.15.44:0/2237178836 conn(0x7fe7d40b0eb0 msgr2=0x7fe7d40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:59.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 -- 172.21.15.44:0/2237178836 shutdown_connections 2024-07-17T08:35:59.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.538+0000 7fe7da000640 1 -- 172.21.15.44:0/2237178836 wait complete. 2024-07-17T08:35:59.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.540+0000 7fe7da000640 1 Processor -- start 2024-07-17T08:35:59.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.540+0000 7fe7da000640 1 -- start start 2024-07-17T08:35:59.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d40ac190 0x7fe7d414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 0x7fe7d4156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d415e220 con 0x7fe7d4154600 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d415e0a0 con 0x7fe7d4161120 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7da000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d415e3a0 con 0x7fe7d40ac190 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7cbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46054/0 (socket says 172.21.15.44:46054) 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7cbe00640 1 -- 172.21.15.44:0/1299689084 learned_addr learned my addr 172.21.15.44:0/1299689084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7d3e00640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 0x7fe7d4156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.541+0000 7fe7d3400640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d40ac190 0x7fe7d414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d40ac190 msgr2=0x7fe7d414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:59.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d40ac190 0x7fe7d414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 msgr2=0x7fe7d4156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:59.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 0x7fe7d4156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 -- 172.21.15.44:0/1299689084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4155bd0 con 0x7fe7d4154600 2024-07-17T08:35:59.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7d3e00640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 0x7fe7d4156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:35:59.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cbe00640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c500 tx=0x7fe7bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc016020 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7bc012940 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc012d90 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.542+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d4154e10 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.543+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d41553d0 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.544+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d415e8e0 con 0x7fe7d4154600 2024-07-17T08:35:59.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.545+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7fe7bc019020 con 0x7fe7d4154600 2024-07-17T08:35:59.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.546+0000 7fe7cb400640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 0x7fe7b0079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:35:59.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.546+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fe7bc0990f0 con 0x7fe7d4154600 2024-07-17T08:35:59.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.546+0000 7fe7d3400640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 msgr2=0x7fe7b0079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:35:59.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.546+0000 7fe7d3400640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 0x7fe7b0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:35:59.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.550+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7bc020020 con 0x7fe7d4154600 2024-07-17T08:35:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.747+0000 7fe7d3400640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 msgr2=0x7fe7b0079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:35:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.747+0000 7fe7d3400640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 0x7fe7b0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:35:59.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.774+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe7d4160e90 con 0x7fe7d4154600 2024-07-17T08:35:59.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.775+0000 7fe7cb400640 1 -- 172.21.15.44:0/1299689084 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v228) ==== 80+0+477 (secure 0 0 0) 0x7fe7d41549e0 con 0x7fe7d4154600 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1458277022 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1949400205 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/2798563150 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3512613843 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/2798563150 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/101784654 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4286139695 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1591261735 2024-07-18T08:35:51.813306+0000 2024-07-17T08:35:59.776 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 msgr2=0x7fe7b0079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 0x7fe7b0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 msgr2=0x7fe7d4156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c500 tx=0x7fe7bc00c9c0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 shutdown_connections 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fe7b0076e80 0x7fe7b0079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4161120 0x7fe7d4156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7d4154600 0x7fe7d4156920 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 --2- 172.21.15.44:0/1299689084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d40ac190 0x7fe7d414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.777+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 >> 172.21.15.44:0/1299689084 conn(0x7fe7d40b0eb0 msgr2=0x7fe7d406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.778+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 shutdown_connections 2024-07-17T08:35:59.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:35:59.778+0000 7fe7da000640 1 -- 172.21.15.44:0/1299689084 wait complete. 2024-07-17T08:35:59.798 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:35:59.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:59.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:35:59.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:35:59.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:35:59.886 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:35:59.926 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:59.926 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:35:59.926 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:35:59.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:35:59.971 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:35:59.971 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:35:59.971 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:00.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:36:00.018 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:36:00.018 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:36:00.018 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:00.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:36:00.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:00.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 -- 172.21.15.44:0/1531461119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c81444d0 msgr2=0x7f70c81448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 --2- 172.21.15.44:0/1531461119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c81444d0 0x7f70c81448b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009830 tx=0x7f70b802a410 comp rx=0 tx=0).stop 2024-07-17T08:36:00.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 -- 172.21.15.44:0/1531461119 shutdown_connections 2024-07-17T08:36:00.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 --2- 172.21.15.44:0/1531461119 >> v2:172.21.15.175:3300/0 conn(0x7f70c815eff0 0x7f70c806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 --2- 172.21.15.44:0/1531461119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c81444d0 0x7f70c81448b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 -- 172.21.15.44:0/1531461119 >> 172.21.15.44:0/1531461119 conn(0x7f70c8072730 msgr2=0x7f70c8072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:00.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 -- 172.21.15.44:0/1531461119 shutdown_connections 2024-07-17T08:36:00.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.267+0000 7f70ce200640 1 -- 172.21.15.44:0/1531461119 wait complete. 2024-07-17T08:36:00.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.269+0000 7f70ce200640 1 Processor -- start 2024-07-17T08:36:00.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.269+0000 7f70ce200640 1 -- start start 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70c81444d0 0x7f70c815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c815eff0 0x7f70c806e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8070070 con 0x7f70c80cd8e0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70c806fef0 con 0x7f70c81444d0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.270+0000 7f70ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f70c80b5850 con 0x7f70c815eff0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46068/0 (socket says 172.21.15.44:46068) 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 -- 172.21.15.44:0/965570114 learned_addr learned my addr 172.21.15.44:0/965570114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7e00640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c815eff0 0x7f70c806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70bea00640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70c81444d0 0x7f70c815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c815eff0 msgr2=0x7f70c806e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c815eff0 0x7f70c806e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70c81444d0 msgr2=0x7f70c815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70c81444d0 0x7f70c815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 -- 172.21.15.44:0/965570114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c80af890 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.271+0000 7f70c7400640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009950 tx=0x7f70b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.272+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b803f030 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.272+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70b8004510 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.272+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8037040 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.272+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c80ad730 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.272+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81f5700 con 0x7f70c80cd8e0 2024-07-17T08:36:00.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.273+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c81448b0 con 0x7f70c80cd8e0 2024-07-17T08:36:00.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.274+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7f70b8042070 con 0x7f70c80cd8e0 2024-07-17T08:36:00.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.275+0000 7f70bfe00640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 0x7f70a4079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.275+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f70b80b8bd0 con 0x7f70c80cd8e0 2024-07-17T08:36:00.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.275+0000 7f70bea00640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 msgr2=0x7f70a4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:00.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.275+0000 7f70bea00640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 0x7f70a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:00.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.280+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f70b8049070 con 0x7f70c80cd8e0 2024-07-17T08:36:00.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.476+0000 7f70bea00640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 msgr2=0x7f70a4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:00.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.476+0000 7f70bea00640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 0x7f70a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:00.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.534+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70c806f510 con 0x7f70c80cd8e0 2024-07-17T08:36:00.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.535+0000 7f70bfe00640 1 -- 172.21.15.44:0/965570114 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f70b8081560 con 0x7f70c80cd8e0 2024-07-17T08:36:00.536 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:00.536 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:00.536 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 msgr2=0x7f70a4079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 0x7f70a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 msgr2=0x7f70c8141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009950 tx=0x7f70b8003c10 comp rx=0 tx=0).stop 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 shutdown_connections 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f70a4076e80 0x7f70a4079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f70c815eff0 0x7f70c806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70c81444d0 0x7f70c815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 --2- 172.21.15.44:0/965570114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70c80cd8e0 0x7f70c8141370 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 >> 172.21.15.44:0/965570114 conn(0x7f70c8072730 msgr2=0x7f70c80cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.538+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 shutdown_connections 2024-07-17T08:36:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.539+0000 7f70ce200640 1 -- 172.21.15.44:0/965570114 wait complete. 2024-07-17T08:36:00.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.745+0000 7f421b000640 1 -- 172.21.15.44:0/760634092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 msgr2=0x7f421415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.745+0000 7f421b000640 1 --2- 172.21.15.44:0/760634092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f421415d800 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f420c009ca0 tx=0x7f420c02aac0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 -- 172.21.15.44:0/760634092 shutdown_connections 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 --2- 172.21.15.44:0/760634092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f421415d800 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 --2- 172.21.15.44:0/760634092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140ce5c0 0x7f421415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 -- 172.21.15.44:0/760634092 >> 172.21.15.44:0/760634092 conn(0x7f42140cfec0 msgr2=0x7f42140d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 -- 172.21.15.44:0/760634092 shutdown_connections 2024-07-17T08:36:00.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.746+0000 7f421b000640 1 -- 172.21.15.44:0/760634092 wait complete. 2024-07-17T08:36:00.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.748+0000 7f421b000640 1 Processor -- start 2024-07-17T08:36:00.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.748+0000 7f421b000640 1 -- start start 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140b8c80 0x7f42141e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42140ce5c0 0x7f421414bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f421415e0d0 con 0x7f42140b9060 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f421415df50 con 0x7f42140ce5c0 2024-07-17T08:36:00.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.749+0000 7f421b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f421415e250 con 0x7f42140b8c80 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140b8c80 0x7f42141e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46074/0 (socket says 172.21.15.44:46074) 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 -- 172.21.15.44:0/4013538049 learned_addr learned my addr 172.21.15.44:0/4013538049 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f421a600640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42140ce5c0 0x7f421414bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140b8c80 msgr2=0x7f42141e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140b8c80 0x7f42141e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42140ce5c0 msgr2=0x7f421414bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42140ce5c0 0x7f421414bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.750+0000 7f4219200640 1 -- 172.21.15.44:0/4013538049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4214149560 con 0x7f42140b9060 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f4219200640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f420000e7c0 tx=0x7f420000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:00.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420000ccc0 con 0x7f42140b9060 2024-07-17T08:36:00.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f420000ce80 con 0x7f42140b9060 2024-07-17T08:36:00.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200005280 con 0x7f42140b9060 2024-07-17T08:36:00.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4214149840 con 0x7f42140b9060 2024-07-17T08:36:00.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.751+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4214149e00 con 0x7f42140b9060 2024-07-17T08:36:00.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.752+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421415f880 con 0x7f42140b9060 2024-07-17T08:36:00.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.753+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7f4200005420 con 0x7f42140b9060 2024-07-17T08:36:00.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.754+0000 7f4211600640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 0x7f41f4079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:00.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.754+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f4200099880 con 0x7f42140b9060 2024-07-17T08:36:00.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.754+0000 7f4219c00640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 msgr2=0x7f41f4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:00.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.754+0000 7f4219c00640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 0x7f41f4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:00.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.758+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4200061a70 con 0x7f42140b9060 2024-07-17T08:36:00.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.955+0000 7f4219c00640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 msgr2=0x7f41f4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:00.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:00.955+0000 7f4219c00640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 0x7f41f4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.002+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4214160110 con 0x7f42140b9060 2024-07-17T08:36:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.003+0000 7f4211600640 1 -- 172.21.15.44:0/4013538049 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f420006a300 con 0x7f42140b9060 2024-07-17T08:36:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:01.004 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 msgr2=0x7f41f4079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 0x7f41f4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 msgr2=0x7f42140a9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f420000e7c0 tx=0x7f420000ec80 comp rx=0 tx=0).stop 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 shutdown_connections 2024-07-17T08:36:01.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f41f4076e80 0x7f41f4079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42140ce5c0 0x7f421414bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42140b9060 0x7f42140a9fe0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 --2- 172.21.15.44:0/4013538049 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42140b8c80 0x7f42141e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.006+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 >> 172.21.15.44:0/4013538049 conn(0x7f42140cfec0 msgr2=0x7f42140d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.007+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 shutdown_connections 2024-07-17T08:36:01.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.007+0000 7f421b000640 1 -- 172.21.15.44:0/4013538049 wait complete. 2024-07-17T08:36:01.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:01.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 -- 172.21.15.44:0/3067669539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 msgr2=0x7fc47c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 --2- 172.21.15.44:0/3067669539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c0cdae0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc47c15c950 tx=0x7fc47402a8b0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 -- 172.21.15.44:0/3067669539 shutdown_connections 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 --2- 172.21.15.44:0/3067669539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c0cdae0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 --2- 172.21.15.44:0/3067669539 >> v2:172.21.15.175:3300/0 conn(0x7fc47c0ac220 0x7fc47c0ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 -- 172.21.15.44:0/3067669539 >> 172.21.15.44:0/3067669539 conn(0x7fc47c0b0fb0 msgr2=0x7fc47c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 -- 172.21.15.44:0/3067669539 shutdown_connections 2024-07-17T08:36:01.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.227+0000 7fc481e00640 1 -- 172.21.15.44:0/3067669539 wait complete. 2024-07-17T08:36:01.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.230+0000 7fc481e00640 1 Processor -- start 2024-07-17T08:36:01.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.230+0000 7fc481e00640 1 -- start start 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 0x7fc47c1f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc47c1522c0 0x7fc47c1f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c0ce020 con 0x7fc47c0cd690 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c0adf80 con 0x7fc47c0ac220 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc481e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc47c0ce1a0 con 0x7fc47c1522c0 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc473e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc473e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46086/0 (socket says 172.21.15.44:46086) 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc473e00640 1 -- 172.21.15.44:0/2870584734 learned_addr learned my addr 172.21.15.44:0/2870584734 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc47b400640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc47c1522c0 0x7fc47c1f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.232+0000 7fc47aa00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 0x7fc47c1f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc47c1522c0 msgr2=0x7fc47c1f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc47c1522c0 0x7fc47c1f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 msgr2=0x7fc47c1f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 0x7fc47c1f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 -- 172.21.15.44:0/2870584734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c1f7800 con 0x7fc47c0cd690 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc47aa00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 0x7fc47c1f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc46400d950 tx=0x7fc46400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464017070 con 0x7fc47c0cd690 2024-07-17T08:36:01.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc464009d20 con 0x7fc47c0cd690 2024-07-17T08:36:01.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4640106c0 con 0x7fc47c0cd690 2024-07-17T08:36:01.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c1f7dc0 con 0x7fc47c0cd690 2024-07-17T08:36:01.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.233+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1f3d20 con 0x7fc47c0cd690 2024-07-17T08:36:01.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.235+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c0acd50 con 0x7fc47c0cd690 2024-07-17T08:36:01.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.236+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7fc464010860 con 0x7fc47c0cd690 2024-07-17T08:36:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.237+0000 7fc473400640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 0x7fc458079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.238+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fc464099420 con 0x7fc47c0cd690 2024-07-17T08:36:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.238+0000 7fc47aa00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 msgr2=0x7fc458079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.238+0000 7fc47aa00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 0x7fc458079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.241+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc47c0acd50 con 0x7fc47c0cd690 2024-07-17T08:36:01.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.439+0000 7fc47aa00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 msgr2=0x7fc458079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:01.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.440+0000 7fc47aa00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 0x7fc458079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.468+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc47c0ce6c0 con 0x7fc47c0cd690 2024-07-17T08:36:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.471+0000 7fc473400640 1 -- 172.21.15.44:0/2870584734 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) ==== 74+0+22047 (secure 0 0 0) 0x7fc464061c60 con 0x7fc47c0cd690 2024-07-17T08:36:01.472 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:01.472 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":228,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:51.813374+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1458277022":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1949400205":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6837/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/3512613843":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6834/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/101784654":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/4286139695":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1591261735":"2024-07-18T08:35:51.813306+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.473+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 msgr2=0x7fc458079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.473+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 0x7fc458079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.473+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 msgr2=0x7fc47c155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.473+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc46400d950 tx=0x7fc46400de10 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 shutdown_connections 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fc458076e80 0x7fc458079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc47c1522c0 0x7fc47c1f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc47c0cd690 0x7fc47c155680 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 --2- 172.21.15.44:0/2870584734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc47c0ac220 0x7fc47c1f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 >> 172.21.15.44:0/2870584734 conn(0x7fc47c0b0fb0 msgr2=0x7fc47c0ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:01.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 shutdown_connections 2024-07-17T08:36:01.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.474+0000 7fc481e00640 1 -- 172.21.15.44:0/2870584734 wait complete. 2024-07-17T08:36:01.493 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:36:01.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 -- 172.21.15.44:0/1229576489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e34004800 msgr2=0x7f0e34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 --2- 172.21.15.44:0/1229576489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e34004800 0x7f0e34004c80 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800bb20 tx=0x7f0e34005830 comp rx=0 tx=0).stop 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 -- 172.21.15.44:0/1229576489 shutdown_connections 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 --2- 172.21.15.44:0/1229576489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e34004800 0x7f0e34004c80 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 -- 172.21.15.44:0/1229576489 >> 172.21.15.44:0/1229576489 conn(0x7f0e500b3020 msgr2=0x7f0e500b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:01.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 -- 172.21.15.44:0/1229576489 shutdown_connections 2024-07-17T08:36:01.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.683+0000 7f0e55000640 1 -- 172.21.15.44:0/1229576489 wait complete. 2024-07-17T08:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.686+0000 7f0e55000640 1 Processor -- start 2024-07-17T08:36:01.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.686+0000 7f0e55000640 1 -- start start 2024-07-17T08:36:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e34004800 0x7f0e5014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50153d60 0x7f0e501f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e500711f0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50071070 con 0x7f0e50153d60 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e55000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e500cefa0 con 0x7f0e34004800 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e4e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46098/0 (socket says 172.21.15.44:46098) 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e4e800640 1 -- 172.21.15.44:0/2864388494 learned_addr learned my addr 172.21.15.44:0/2864388494 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e4de00640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e34004800 0x7f0e5014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e45400640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50153d60 0x7f0e501f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.688+0000 7f0e4e800640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e34004800 msgr2=0x7f0e5014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e4e800640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e34004800 0x7f0e5014d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e4e800640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50153d60 msgr2=0x7f0e501f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e4e800640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50153d60 0x7f0e501f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e4e800640 1 -- 172.21.15.44:0/2864388494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5014ded0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e4e800640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f0e500747a0 tx=0x7f0e4800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:01.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e480062e0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e480035f0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501f84a0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48003a40 con 0x7f0e5015ecf0 2024-07-17T08:36:01.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.689+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5015df30 con 0x7f0e5015ecf0 2024-07-17T08:36:01.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.691+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e500cf4a0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.692+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7f0e48018020 con 0x7f0e5015ecf0 2024-07-17T08:36:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.693+0000 7f0e46a00640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 0x7f0e2c079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.693+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f0e48098aa0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.693+0000 7f0e4de00640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 msgr2=0x7f0e2c079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:01.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.693+0000 7f0e4de00640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 0x7f0e2c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:01.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.697+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0e48060c90 con 0x7f0e5015ecf0 2024-07-17T08:36:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.894+0000 7f0e4de00640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 msgr2=0x7f0e2c079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:01.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.894+0000 7f0e4de00640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 0x7f0e2c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:01.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.948+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e500af420 con 0x7f0e5015ecf0 2024-07-17T08:36:01.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.949+0000 7f0e46a00640 1 -- 172.21.15.44:0/2864388494 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f0e480651e0 con 0x7f0e5015ecf0 2024-07-17T08:36:01.950 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:01.950 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:01.950 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.951+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 msgr2=0x7f0e2c079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.951+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 0x7f0e2c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.951+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 msgr2=0x7f0e5014e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.951+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f0e500747a0 tx=0x7f0e4800da10 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 shutdown_connections 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f0e2c076eb0 0x7f0e2c079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e5015ecf0 0x7f0e5014e1a0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e50153d60 0x7f0e501f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 --2- 172.21.15.44:0/2864388494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e34004800 0x7f0e5014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 >> 172.21.15.44:0/2864388494 conn(0x7f0e500b3020 msgr2=0x7f0e500afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 shutdown_connections 2024-07-17T08:36:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:01.952+0000 7f0e55000640 1 -- 172.21.15.44:0/2864388494 wait complete. 2024-07-17T08:36:01.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 -- 172.21.15.44:0/954546262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0b9060 msgr2=0x7f864c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 --2- 172.21.15.44:0/954546262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0b9060 0x7f864c15d800 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f8644009ca0 tx=0x7f864402aac0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 -- 172.21.15.44:0/954546262 shutdown_connections 2024-07-17T08:36:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 --2- 172.21.15.44:0/954546262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0b9060 0x7f864c15d800 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 --2- 172.21.15.44:0/954546262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0ce5c0 0x7f864c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 -- 172.21.15.44:0/954546262 >> 172.21.15.44:0/954546262 conn(0x7f864c0cfec0 msgr2=0x7f864c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 -- 172.21.15.44:0/954546262 shutdown_connections 2024-07-17T08:36:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.163+0000 7f8652c00640 1 -- 172.21.15.44:0/954546262 wait complete. 2024-07-17T08:36:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.165+0000 7f8652c00640 1 Processor -- start 2024-07-17T08:36:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.165+0000 7f8652c00640 1 -- start start 2024-07-17T08:36:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 0x7f864c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f864c0b9060 0x7f864c1e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f864c15e0d0 con 0x7f864c0ce5c0 2024-07-17T08:36:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f864c15df50 con 0x7f864c0b9060 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f864c15e250 con 0x7f864c0b8c80 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.166+0000 7f8652200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8651800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 0x7f864c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46108/0 (socket says 172.21.15.44:46108) 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8651800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 0x7f864c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.44:48120/0 (socket says 172.21.15.44:48120) 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 -- 172.21.15.44:0/2250834956 learned_addr learned my addr 172.21.15.44:0/2250834956 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8650e00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f864c0b9060 0x7f864c1e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 msgr2=0x7f864c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 0x7f864c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f864c0b9060 msgr2=0x7f864c1e85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f864c0b9060 0x7f864c1e85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 -- 172.21.15.44:0/2250834956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c0ca510 con 0x7f864c0ce5c0 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.167+0000 7f8652200640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.168+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644046070 con 0x7f864c0ce5c0 2024-07-17T08:36:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.168+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8644004250 con 0x7f864c0ce5c0 2024-07-17T08:36:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.168+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644004680 con 0x7f864c0ce5c0 2024-07-17T08:36:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.168+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c1e79c0 con 0x7f864c0ce5c0 2024-07-17T08:36:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.168+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c0ca7d0 con 0x7f864c0ce5c0 2024-07-17T08:36:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.169+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864c15e700 con 0x7f864c0ce5c0 2024-07-17T08:36:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.170+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7f8644036400 con 0x7f864c0ce5c0 2024-07-17T08:36:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.171+0000 7f8649600640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 0x7f862c079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.171+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f86440b9f50 con 0x7f864c0ce5c0 2024-07-17T08:36:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.171+0000 7f8651800640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 msgr2=0x7f862c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.171+0000 7f8651800640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 0x7f862c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.175+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8644032030 con 0x7f864c0ce5c0 2024-07-17T08:36:02.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.373+0000 7f8651800640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 msgr2=0x7f862c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:02.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.373+0000 7f8651800640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 0x7f862c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:02.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.428+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f864c0b5d70 con 0x7f864c0ce5c0 2024-07-17T08:36:02.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.429+0000 7f8649600640 1 -- 172.21.15.44:0/2250834956 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+6556 (secure 0 0 0) 0x7f8644004820 con 0x7f864c0ce5c0 2024-07-17T08:36:02.430 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:02.430 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":119,"btime":"2024-07-17T08:35:37:474455+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:21.767746+0000","modified":"2024-07-17T08:35:24.841167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10083":{"gid":10083,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1413771882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1413771882},{"type":"v1","addr":"172.21.15.44:6835","nonce":1413771882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10083,"qdb_cluster":[10083]},"id":13},{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:02.430 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 119 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.431+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 msgr2=0x7f862c079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.431+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 0x7f862c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 msgr2=0x7f864c1e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644003f10 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 shutdown_connections 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f862c076e80 0x7f862c079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f864c0ce5c0 0x7f864c1e8ae0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f864c0b9060 0x7f864c1e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 --2- 172.21.15.44:0/2250834956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f864c0b8c80 0x7f864c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 >> 172.21.15.44:0/2250834956 conn(0x7f864c0cfec0 msgr2=0x7f864c0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:02.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 shutdown_connections 2024-07-17T08:36:02.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.432+0000 7f8652c00640 1 -- 172.21.15.44:0/2250834956 wait complete. 2024-07-17T08:36:02.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 -- 172.21.15.44:0/115465060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc06e560 msgr2=0x7f18fc073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 --2- 172.21.15.44:0/115465060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc06e560 0x7f18fc073370 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f18fc06e9f0 tx=0x7f18e802a560 comp rx=0 tx=0).stop 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 -- 172.21.15.44:0/115465060 shutdown_connections 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 --2- 172.21.15.44:0/115465060 >> v2:172.21.15.175:3300/0 conn(0x7f18fc153990 0x7f18fc153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 --2- 172.21.15.44:0/115465060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc06e560 0x7f18fc073370 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 --2- 172.21.15.44:0/115465060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 -- 172.21.15.44:0/115465060 >> 172.21.15.44:0/115465060 conn(0x7f18fc06bf40 msgr2=0x7f18fc06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:02.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 -- 172.21.15.44:0/115465060 shutdown_connections 2024-07-17T08:36:02.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.648+0000 7f1901e00640 1 -- 172.21.15.44:0/115465060 wait complete. 2024-07-17T08:36:02.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.649+0000 7f1901e00640 1 Processor -- start 2024-07-17T08:36:02.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.650+0000 7f1901e00640 1 -- start start 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc14a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18fc06e560 0x7f18fc14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc068860 con 0x7f18fc153990 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc072090 con 0x7f18fc06e560 2024-07-17T08:36:02.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f1901e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18fc0689e0 con 0x7f18fc06e180 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f18fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f18fb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46128/0 (socket says 172.21.15.44:46128) 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f18fb400640 1 -- 172.21.15.44:0/231849910 learned_addr learned my addr 172.21.15.44:0/231849910 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f18faa00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc14a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.651+0000 7f18fa000640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18fc06e560 0x7f18fc14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 msgr2=0x7f18fc14a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc14a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18fc06e560 msgr2=0x7f18fc14e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18fc06e560 0x7f18fc14e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 -- 172.21.15.44:0/231849910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc14aa40 con 0x7f18fc153990 2024-07-17T08:36:02.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18faa00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc14a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18fb400640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cae0 tx=0x7f18f4007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4007da0 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f4004c30 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4005280 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc148590 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.652+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc14a630 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.654+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18fc073370 con 0x7f18fc153990 2024-07-17T08:36:02.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.655+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7f18f4005420 con 0x7f18fc153990 2024-07-17T08:36:02.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.656+0000 7f18f3400640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 0x7f18dc079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:02.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.656+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7f18f4099a60 con 0x7f18fc153990 2024-07-17T08:36:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.656+0000 7f18faa00640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 msgr2=0x7f18dc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.656+0000 7f18faa00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 0x7f18dc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.660+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f18f4014070 con 0x7f18fc153990 2024-07-17T08:36:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.858+0000 7f18faa00640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 msgr2=0x7f18dc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.858+0000 7f18faa00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 0x7f18dc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:02.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.870+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18fc0b3990 con 0x7f18fc153990 2024-07-17T08:36:02.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.874+0000 7f18f3400640 1 -- 172.21.15.44:0/231849910 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) ==== 74+0+22047 (secure 0 0 0) 0x7f18f40622a0 con 0x7f18fc153990 2024-07-17T08:36:02.875 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:02.875 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":228,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:35:51.813374+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:35:19.685232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-07-17T08:35:20.753963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1458277022":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1949400205":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6837/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/3512613843":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6834/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/101784654":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/4286139695":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1591261735":"2024-07-18T08:35:51.813306+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 msgr2=0x7f18dc079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 0x7f18dc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 msgr2=0x7f18fc14e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cae0 tx=0x7f18f4007220 comp rx=0 tx=0).stop 2024-07-17T08:36:02.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 shutdown_connections 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7f18dc076e80 0x7f18dc079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18fc153990 0x7f18fc14e710 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18fc06e560 0x7f18fc14e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 --2- 172.21.15.44:0/231849910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18fc06e180 0x7f18fc14a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 >> 172.21.15.44:0/231849910 conn(0x7f18fc06bf40 msgr2=0x7f18fc152910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.877+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 shutdown_connections 2024-07-17T08:36:02.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:02.878+0000 7f1901e00640 1 -- 172.21.15.44:0/231849910 wait complete. 2024-07-17T08:36:02.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 -- 172.21.15.44:0/550395666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea40733d0 msgr2=0x7fdea4163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 --2- 172.21.15.44:0/550395666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea40733d0 0x7fdea4163520 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fdea4071e30 tx=0x7fde9c02bc00 comp rx=0 tx=0).stop 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 -- 172.21.15.44:0/550395666 shutdown_connections 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 --2- 172.21.15.44:0/550395666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea40733d0 0x7fdea4163520 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 --2- 172.21.15.44:0/550395666 >> v2:172.21.15.175:3300/0 conn(0x7fdea415c280 0x7fdea4072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.101+0000 7fdeab400640 1 -- 172.21.15.44:0/550395666 >> 172.21.15.44:0/550395666 conn(0x7fdea406ce80 msgr2=0x7fdea406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.102+0000 7fdeab400640 1 -- 172.21.15.44:0/550395666 shutdown_connections 2024-07-17T08:36:03.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.102+0000 7fdeab400640 1 -- 172.21.15.44:0/550395666 wait complete. 2024-07-17T08:36:03.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.103+0000 7fdeab400640 1 Processor -- start 2024-07-17T08:36:03.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.103+0000 7fdeab400640 1 -- start start 2024-07-17T08:36:03.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea40733d0 0x7fdea40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 0x7fdea40ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdea40689e0 con 0x7fdea4159990 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdea4068860 con 0x7fdea40733d0 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdeab400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdea4068b60 con 0x7fdea415c280 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.104+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46150/0 (socket says 172.21.15.44:46150) 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 -- 172.21.15.44:0/67579710 learned_addr learned my addr 172.21.15.44:0/67579710 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea8a00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea40733d0 0x7fdea40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea9400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 0x7fdea40ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 -- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 msgr2=0x7fdea40ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 0x7fdea40ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 -- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea40733d0 msgr2=0x7fdea40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea40733d0 0x7fdea40ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 -- 172.21.15.44:0/67579710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea40cdb50 con 0x7fdea4159990 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea9400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 0x7fdea40ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.105+0000 7fdea3e00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fde9000d740 tx=0x7fde9000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.106+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90017070 con 0x7fdea4159990 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.106+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea4155590 con 0x7fdea4159990 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.106+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde90009d20 con 0x7fdea4159990 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.106+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900106c0 con 0x7fdea4159990 2024-07-17T08:36:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.106+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea40cd870 con 0x7fdea4159990 2024-07-17T08:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.107+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea4072e90 con 0x7fdea4159990 2024-07-17T08:36:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.108+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 137) ==== 99090+0+0 (secure 0 0 0) 0x7fde9001a070 con 0x7fdea4159990 2024-07-17T08:36:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.110+0000 7fdea0c00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 0x7fde84079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.110+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 7885+0+0 (secure 0 0 0) 0x7fde90099260 con 0x7fdea4159990 2024-07-17T08:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.110+0000 7fdea8a00640 1 -- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 msgr2=0x7fde84079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1605633467 2024-07-17T08:36:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.110+0000 7fdea8a00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 0x7fde84079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.114+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fde90061a20 con 0x7fdea4159990 2024-07-17T08:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.249+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 138) ==== 98540+0+0 (secure 0 0 0) 0x7fde9005d5b0 con 0x7fdea4159990 2024-07-17T08:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.250+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 msgr2=0x7fde84079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:03.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.250+0000 7fdea0c00640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 0x7fde84079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.333+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.333+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.333+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.333+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.333+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.334+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.334+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.335 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.334+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.336 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.334+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.336 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.334+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.349+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.349+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.349+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.350 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.349+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.351 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.349+0000 7f7d5da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.351 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.350+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.351 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.350+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.351 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.350+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.352 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.350+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.352 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:03.350+0000 7f7d59e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:03.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.374+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdea4158960 con 0x7fdea4159990 2024-07-17T08:36:03.376 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:36:03.375+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:36:03.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.377+0000 7fdea0c00640 1 -- 172.21.15.44:0/67579710 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) ==== 161+0+0 (secure 0 0 0) 0x7fde9000f3c0 con 0x7fdea4159990 2024-07-17T08:36:03.378 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:36:03.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.380+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 msgr2=0x7fdea40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.380+0000 7fdeab400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fde9000d740 tx=0x7fde9000dc00 comp rx=0 tx=0).stop 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 shutdown_connections 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:6834/1605633467,v1:172.21.15.175:6837/1605633467] conn(0x7fde84076e80 0x7fde84079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdea415c280 0x7fdea40ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdea4159990 0x7fdea40ce0d0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 --2- 172.21.15.44:0/67579710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdea40733d0 0x7fdea40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.381+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 >> 172.21.15.44:0/67579710 conn(0x7fdea406ce80 msgr2=0x7fdea40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:03.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.384+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 shutdown_connections 2024-07-17T08:36:03.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.384+0000 7fdeab400640 1 -- 172.21.15.44:0/67579710 wait complete. 2024-07-17T08:36:03.395 INFO:tasks.ceph.mds.a.smithi044.stderr: -5009> 2024-07-17T08:35:24.251+0000 7f788ac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:36:03.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:36:03.451 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:36:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 -- 172.21.15.44:0/2499323337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce34152220 msgr2=0x7fce3406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 --2- 172.21.15.44:0/2499323337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce34152220 0x7fce3406d620 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fce340cce50 tx=0x7fce2402a4b0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 -- 172.21.15.44:0/2499323337 shutdown_connections 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 --2- 172.21.15.44:0/2499323337 >> v2:172.21.15.175:3300/0 conn(0x7fce340cbab0 0x7fce3415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 --2- 172.21.15.44:0/2499323337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce34152220 0x7fce3406d620 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 -- 172.21.15.44:0/2499323337 >> 172.21.15.44:0/2499323337 conn(0x7fce340b0eb0 msgr2=0x7fce340b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 -- 172.21.15.44:0/2499323337 shutdown_connections 2024-07-17T08:36:03.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.590+0000 7fce39600640 1 -- 172.21.15.44:0/2499323337 wait complete. 2024-07-17T08:36:03.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.592+0000 7fce39600640 1 Processor -- start 2024-07-17T08:36:03.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.592+0000 7fce39600640 1 -- start start 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3406ed30 0x7fce3415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 0x7fce341f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce340cd780 con 0x7fce340cbab0 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce340cd600 con 0x7fce34152220 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.593+0000 7fce39600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce340cd900 con 0x7fce3406ed30 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46172/0 (socket says 172.21.15.44:46172) 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 -- 172.21.15.44:0/1696122699 learned_addr learned my addr 172.21.15.44:0/1696122699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce32a00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3406ed30 0x7fce3415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce33400640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 0x7fce341f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 -- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3406ed30 msgr2=0x7fce3415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3406ed30 0x7fce3415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 -- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 msgr2=0x7fce341f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 0x7fce341f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce2be00640 1 -- 172.21.15.44:0/1696122699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce341597c0 con 0x7fce340cbab0 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.594+0000 7fce33400640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 0x7fce341f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce2be00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c450 tx=0x7fce1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c016020 con 0x7fce340cbab0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce1c012920 con 0x7fce340cbab0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce34159a80 con 0x7fce340cbab0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c012d70 con 0x7fce340cbab0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.595+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3415a040 con 0x7fce340cbab0 2024-07-17T08:36:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.596+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce340ce5e0 con 0x7fce340cbab0 2024-07-17T08:36:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.597+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 138) ==== 98540+0+0 (secure 0 0 0) 0x7fce1c019020 con 0x7fce340cbab0 2024-07-17T08:36:03.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.599+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(230..230 src has 1..230) ==== 8444+0+0 (secure 0 0 0) 0x7fce1c0986e0 con 0x7fce340cbab0 2024-07-17T08:36:03.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.604+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fce1c020030 con 0x7fce340cbab0 2024-07-17T08:36:03.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:03.853+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fce3415d720 con 0x7fce340cbab0 2024-07-17T08:36:04.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.258+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 139) ==== 98667+0+0 (secure 0 0 0) 0x7fce1c05c6c0 con 0x7fce340cbab0 2024-07-17T08:36:04.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.259+0000 7fce2aa00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 0x7fce10081cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:04.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.266+0000 7fce32a00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 0x7fce10081cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:04.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.281+0000 7fce32a00640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 0x7fce10081cf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce24034000 tx=0x7fce24004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:04.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.381+0000 7fce2aa00640 1 -- 172.21.15.44:0/1696122699 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v121) ==== 104+0+0 (secure 0 0 0) 0x7fce1c061450 con 0x7fce340cbab0 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 msgr2=0x7fce10081cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 0x7fce10081cf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce24034000 tx=0x7fce24004730 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 msgr2=0x7fce341f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00c450 tx=0x7fce1c00c910 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 shutdown_connections 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fce1007f900 0x7fce10081cf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce34152220 0x7fce341f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce340cbab0 0x7fce341f7240 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 --2- 172.21.15.44:0/1696122699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce3406ed30 0x7fce3415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 >> 172.21.15.44:0/1696122699 conn(0x7fce340b0eb0 msgr2=0x7fce340b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:04.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 shutdown_connections 2024-07-17T08:36:04.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.385+0000 7fce39600640 1 -- 172.21.15.44:0/1696122699 wait complete. 2024-07-17T08:36:04.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:36:04.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.664+0000 7f0763e00640 1 -- 172.21.15.44:0/1030044173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 msgr2=0x7f07640b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.664+0000 7f0763e00640 1 --2- 172.21.15.44:0/1030044173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f07640b8670 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f075c005fd0 tx=0x7f075c02a820 comp rx=0 tx=0).stop 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.665+0000 7f0763e00640 1 -- 172.21.15.44:0/1030044173 shutdown_connections 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.665+0000 7f0763e00640 1 --2- 172.21.15.44:0/1030044173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f07640b8670 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.665+0000 7f0763e00640 1 -- 172.21.15.44:0/1030044173 >> 172.21.15.44:0/1030044173 conn(0x7f0764073a60 msgr2=0x7f0764073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.665+0000 7f0763e00640 1 -- 172.21.15.44:0/1030044173 shutdown_connections 2024-07-17T08:36:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.665+0000 7f0763e00640 1 -- 172.21.15.44:0/1030044173 wait complete. 2024-07-17T08:36:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.666+0000 7f0763e00640 1 Processor -- start 2024-07-17T08:36:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 -- start start 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076415e370 0x7f076415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0764161b50 0x7f076415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07640b8d30 con 0x7f07640d0b40 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07640b8bb0 con 0x7f076415e370 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.667+0000 7f0763e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07640b8eb0 con 0x7f0764161b50 2024-07-17T08:36:04.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46194/0 (socket says 172.21.15.44:46194) 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 -- 172.21.15.44:0/2435332643 learned_addr learned my addr 172.21.15.44:0/2435332643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0763400640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0764161b50 0x7f076415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762000640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076415e370 0x7f076415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 -- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0764161b50 msgr2=0x7f076415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0764161b50 0x7f076415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 -- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076415e370 msgr2=0x7f076415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076415e370 0x7f076415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.668+0000 7f0762a00640 1 -- 172.21.15.44:0/2435332643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07640c8040 con 0x7f07640d0b40 2024-07-17T08:36:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f0762a00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f075400c460 tx=0x7f075400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754016020 con 0x7f07640d0b40 2024-07-17T08:36:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0754012920 con 0x7f07640d0b40 2024-07-17T08:36:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754012d70 con 0x7f07640d0b40 2024-07-17T08:36:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07640b4e50 con 0x7f07640d0b40 2024-07-17T08:36:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.669+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07640c82e0 con 0x7f07640d0b40 2024-07-17T08:36:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.670+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07640b9360 con 0x7f07640d0b40 2024-07-17T08:36:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.671+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 139) ==== 98667+0+0 (secure 0 0 0) 0x7f0754019020 con 0x7f07640d0b40 2024-07-17T08:36:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.672+0000 7f075aa00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 0x7f0740078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.673+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(230..230 src has 1..230) ==== 8444+0+0 (secure 0 0 0) 0x7f0754098510 con 0x7f07640d0b40 2024-07-17T08:36:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.673+0000 7f0762000640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 0x7f0740078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:04.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.674+0000 7f0762000640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 0x7f0740078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f074c000c00 tx=0x7f074c009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:04.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.677+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0754020070 con 0x7f07640d0b40 2024-07-17T08:36:04.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:04.887+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07640d1090 con 0x7f07640d0b40 2024-07-17T08:36:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.385+0000 7f075aa00640 1 -- 172.21.15.44:0/2435332643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v231) ==== 181+0+0 (secure 0 0 0) 0x7f0754061190 con 0x7f07640d0b40 2024-07-17T08:36:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:36:05.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 msgr2=0x7f0740078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 0x7f0740078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f074c000c00 tx=0x7f074c009040 comp rx=0 tx=0).stop 2024-07-17T08:36:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 msgr2=0x7f0764157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:05.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f075400c460 tx=0x7f075400c920 comp rx=0 tx=0).stop 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 shutdown_connections 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0740076a00 0x7f0740078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0764161b50 0x7f076415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076415e370 0x7f076415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 --2- 172.21.15.44:0/2435332643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07640d0b40 0x7f0764157980 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.389+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 >> 172.21.15.44:0/2435332643 conn(0x7f0764073a60 msgr2=0x7f076415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:05.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.390+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 shutdown_connections 2024-07-17T08:36:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.390+0000 7f0763e00640 1 -- 172.21.15.44:0/2435332643 wait complete. 2024-07-17T08:36:05.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:36:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 -- 172.21.15.44:0/2079703589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb940003a30 msgr2=0x7fb940003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 --2- 172.21.15.44:0/2079703589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb940003a30 0x7fb940003e50 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00baf0 tx=0x7fb940005830 comp rx=0 tx=0).stop 2024-07-17T08:36:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb95e200640 1 -- 172.21.15.44:0/2079703589 reap_dead start 2024-07-17T08:36:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 -- 172.21.15.44:0/2079703589 shutdown_connections 2024-07-17T08:36:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 -- 172.21.15.44:0/2079703589 >> 172.21.15.44:0/2079703589 conn(0x7fb9580b0eb0 msgr2=0x7fb9580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:05.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 -- 172.21.15.44:0/2079703589 shutdown_connections 2024-07-17T08:36:05.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.612+0000 7fb960200640 1 -- 172.21.15.44:0/2079703589 wait complete. 2024-07-17T08:36:05.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.614+0000 7fb960200640 1 Processor -- start 2024-07-17T08:36:05.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.614+0000 7fb960200640 1 -- start start 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 0x7fb95809af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 0x7fb958141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9580ac7f0 con 0x7fb958099cb0 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb958153d40 con 0x7fb940003a30 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.615+0000 7fb960200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9580ac970 con 0x7fb9580aee20 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46206/0 (socket says 172.21.15.44:46206) 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 -- 172.21.15.44:0/4086403752 learned_addr learned my addr 172.21.15.44:0/4086403752 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:05.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95d800640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 0x7fb95809af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95e200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 0x7fb958141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 -- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 msgr2=0x7fb958141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 0x7fb958141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 -- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 msgr2=0x7fb95809af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 0x7fb95809af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 -- 172.21.15.44:0/4086403752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95809a220 con 0x7fb958099cb0 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95d800640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 0x7fb95809af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95e200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 0x7fb958141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.616+0000 7fb95ce00640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d820 tx=0x7fb94400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.617+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944019070 con 0x7fb958099cb0 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.617+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9440052e0 con 0x7fb958099cb0 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.617+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95809a500 con 0x7fb958099cb0 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.617+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400b8a0 con 0x7fb958099cb0 2024-07-17T08:36:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.617+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581f50d0 con 0x7fb958099cb0 2024-07-17T08:36:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.618+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95809a740 con 0x7fb958099cb0 2024-07-17T08:36:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.619+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 140) ==== 98977+0+0 (secure 0 0 0) 0x7fb94401c070 con 0x7fb958099cb0 2024-07-17T08:36:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.620+0000 7fb955600640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 0x7fb938079280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.620+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(231..231 src has 1..231) ==== 8006+0+0 (secure 0 0 0) 0x7fb9440987d0 con 0x7fb958099cb0 2024-07-17T08:36:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.621+0000 7fb95d800640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 0x7fb938079280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.622+0000 7fb95d800640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 0x7fb938079280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb94c036000 tx=0x7fb94c0046c0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.625+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb944061080 con 0x7fb958099cb0 2024-07-17T08:36:05.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:05.844+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb95806d620 con 0x7fb958099cb0 2024-07-17T08:36:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.405+0000 7fb955600640 1 -- 172.21.15.44:0/4086403752 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v232) ==== 169+0+0 (secure 0 0 0) 0x7fb944060980 con 0x7fb958099cb0 2024-07-17T08:36:06.406 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:36:06.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 msgr2=0x7fb938079280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 0x7fb938079280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb94c036000 tx=0x7fb94c0046c0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 msgr2=0x7fb95809b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d820 tx=0x7fb94400dce0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 shutdown_connections 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb938076dc0 0x7fb938079280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9580aee20 0x7fb958141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb958099cb0 0x7fb95809b450 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 --2- 172.21.15.44:0/4086403752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb940003a30 0x7fb95809af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 >> 172.21.15.44:0/4086403752 conn(0x7fb9580b0eb0 msgr2=0x7fb9580735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:06.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 shutdown_connections 2024-07-17T08:36:06.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.409+0000 7fb960200640 1 -- 172.21.15.44:0/4086403752 wait complete. 2024-07-17T08:36:06.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:06.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 -- 172.21.15.44:0/1594351074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb8004800 msgr2=0x7f4eb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 --2- 172.21.15.44:0/1594351074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb8004800 0x7f4eb8004c80 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00baf0 tx=0x7f4eb8005830 comp rx=0 tx=0).stop 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 -- 172.21.15.44:0/1594351074 shutdown_connections 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 --2- 172.21.15.44:0/1594351074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4eb8004800 0x7f4eb8004c80 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 -- 172.21.15.44:0/1594351074 >> 172.21.15.44:0/1594351074 conn(0x7f4ed40b0fb0 msgr2=0x7f4ed40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.647+0000 7f4edaa00640 1 -- 172.21.15.44:0/1594351074 shutdown_connections 2024-07-17T08:36:06.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.648+0000 7f4edaa00640 1 -- 172.21.15.44:0/1594351074 wait complete. 2024-07-17T08:36:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.649+0000 7f4edaa00640 1 Processor -- start 2024-07-17T08:36:06.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.650+0000 7f4edaa00640 1 -- start start 2024-07-17T08:36:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 0x7f4ed414e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed415da00 0x7f4ed415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:06.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40ac810 con 0x7f4ed40733a0 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40ac690 con 0x7f4eb8004800 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4edaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40ac990 con 0x7f4ed415da00 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4ed8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4ed8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46232/0 (socket says 172.21.15.44:46232) 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4ed8a00640 1 -- 172.21.15.44:0/2069103832 learned_addr learned my addr 172.21.15.44:0/2069103832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4ecbe00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed415da00 0x7f4ed415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.651+0000 7f4ed3e00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 0x7f4ed414e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 -- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed415da00 msgr2=0x7f4ed415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed415da00 0x7f4ed415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 -- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 msgr2=0x7f4ed414e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 0x7f4ed414e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 -- 172.21.15.44:0/2069103832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed414ea50 con 0x7f4ed40733a0 2024-07-17T08:36:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed3e00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 0x7f4ed414e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed8a00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c450 tx=0x7f4ecc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc011790 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ecc017070 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc011cc0 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41f3ac0 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.652+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41f4060 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.654+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed415e3c0 con 0x7f4ed40733a0 2024-07-17T08:36:06.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.655+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 140) ==== 98977+0+0 (secure 0 0 0) 0x7f4ecc01c460 con 0x7f4ed40733a0 2024-07-17T08:36:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.656+0000 7f4ed0c00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 0x7f4eb0079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.656+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7f4ecc098aa0 con 0x7f4ed40733a0 2024-07-17T08:36:06.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.657+0000 7f4ed3e00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 0x7f4eb0079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:06.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.658+0000 7f4ed3e00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 0x7f4eb0079250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4001860 tx=0x7f4ec4001bd0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:06.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.660+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4ecc061580 con 0x7f4ed40733a0 2024-07-17T08:36:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.926+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed40adaa0 con 0x7f4ed40733a0 2024-07-17T08:36:06.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.927+0000 7f4ed0c00640 1 -- 172.21.15.44:0/2069103832 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5234 (secure 0 0 0) 0x7f4ecc060e80 con 0x7f4ed40733a0 2024-07-17T08:36:06.928 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:06.928 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":121,"btime":"2024-07-17T08:36:04:377961+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":10587,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3444850603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3444850603},{"type":"v1","addr":"172.21.15.44:6835","nonce":3444850603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:06.928 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 121 2024-07-17T08:36:06.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 msgr2=0x7f4eb0079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 0x7f4eb0079250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4001860 tx=0x7f4ec4001bd0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 msgr2=0x7f4ed4073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:06.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c450 tx=0x7f4ecc00c910 comp rx=0 tx=0).stop 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 shutdown_connections 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4eb0076d90 0x7f4eb0079250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ed40733a0 0x7f4ed4073850 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ed415da00 0x7f4ed415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 --2- 172.21.15.44:0/2069103832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eb8004800 0x7f4ed414e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 >> 172.21.15.44:0/2069103832 conn(0x7f4ed40b0fb0 msgr2=0x7f4ed406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.930+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 shutdown_connections 2024-07-17T08:36:06.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:06.931+0000 7f4edaa00640 1 -- 172.21.15.44:0/2069103832 wait complete. 2024-07-17T08:36:06.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 -- 172.21.15.44:0/957655849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40ae810 msgr2=0x7f88d415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 --2- 172.21.15.44:0/957655849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40ae810 0x7f88d415e510 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009830 tx=0x7f88c402a510 comp rx=0 tx=0).stop 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 -- 172.21.15.44:0/957655849 shutdown_connections 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 --2- 172.21.15.44:0/957655849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d40ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 --2- 172.21.15.44:0/957655849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40ae810 0x7f88d415e510 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 -- 172.21.15.44:0/957655849 >> 172.21.15.44:0/957655849 conn(0x7f88d40b0fb0 msgr2=0x7f88d40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 -- 172.21.15.44:0/957655849 shutdown_connections 2024-07-17T08:36:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.157+0000 7f88db200640 1 -- 172.21.15.44:0/957655849 wait complete. 2024-07-17T08:36:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.159+0000 7f88db200640 1 Processor -- start 2024-07-17T08:36:07.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.159+0000 7f88db200640 1 -- start start 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d41f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 0x7f88d41f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40b2e60 con 0x7f88d406ee90 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40b2ce0 con 0x7f88d40ae810 2024-07-17T08:36:07.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88db200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88d40b2fe0 con 0x7f88d406ea50 2024-07-17T08:36:07.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.160+0000 7f88d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46238/0 (socket says 172.21.15.44:46238) 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 -- 172.21.15.44:0/1625921254 learned_addr learned my addr 172.21.15.44:0/1625921254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3e00640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d41f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d9200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 0x7f88d41f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 -- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 msgr2=0x7f88d41f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d41f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 -- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 msgr2=0x7f88d41f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 0x7f88d41f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 -- 172.21.15.44:0/1625921254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d41f46f0 con 0x7f88d406ee90 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3e00640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d41f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d9200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 0x7f88d41f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.161+0000 7f88d3400640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f88c000d950 tx=0x7f88c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:07.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.162+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88d406ee90 2024-07-17T08:36:07.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.162+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c0009d20 con 0x7f88d406ee90 2024-07-17T08:36:07.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.162+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00106c0 con 0x7f88d406ee90 2024-07-17T08:36:07.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.162+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d41f49d0 con 0x7f88d406ee90 2024-07-17T08:36:07.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.162+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41432d0 con 0x7f88d406ee90 2024-07-17T08:36:07.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.163+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d40ac840 con 0x7f88d406ee90 2024-07-17T08:36:07.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.164+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 140) ==== 98977+0+0 (secure 0 0 0) 0x7f88c0010860 con 0x7f88d406ee90 2024-07-17T08:36:07.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.165+0000 7f88cbe00640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 0x7f88b4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.165+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7f88c00992b0 con 0x7f88d406ee90 2024-07-17T08:36:07.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.166+0000 7f88d3e00640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 0x7f88b4079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.167+0000 7f88d3e00640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 0x7f88b4079220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034000 tx=0x7f88c4004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:07.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.170+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f88c0061bf0 con 0x7f88d406ee90 2024-07-17T08:36:07.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.383+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88d40b23b0 con 0x7f88d406ee90 2024-07-17T08:36:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.386+0000 7f88cbe00640 1 -- 172.21.15.44:0/1625921254 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) ==== 74+0+19095 (secure 0 0 0) 0x7f88c00614f0 con 0x7f88d406ee90 2024-07-17T08:36:07.388 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:07.388 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":232,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:06.401323+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/2462391516":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1904359110":"2024-07-18T08:36:03.234736+0000","172.21.15.44:6832/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/3038109628":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6837/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6834/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1799654570":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/2557831081":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/749128801":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/485731160":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/363211359":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/3582055205":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1458277022":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1949400205":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6837/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/3512613843":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6834/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/101784654":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/4286139695":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1591261735":"2024-07-18T08:35:51.813306+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:07.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.389+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 msgr2=0x7f88b4079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.389+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 0x7f88b4079220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034000 tx=0x7f88c4004730 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.389+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 msgr2=0x7f88d41414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.389+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f88c000d950 tx=0x7f88c000de10 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 shutdown_connections 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f88b4076d60 0x7f88b4079220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88d40ae810 0x7f88d41f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d406ee90 0x7f88d41414f0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 --2- 172.21.15.44:0/1625921254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88d406ea50 0x7f88d41f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 >> 172.21.15.44:0/1625921254 conn(0x7f88d40b0fb0 msgr2=0x7f88d40aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 shutdown_connections 2024-07-17T08:36:07.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.390+0000 7f88db200640 1 -- 172.21.15.44:0/1625921254 wait complete. 2024-07-17T08:36:07.409 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-17T08:36:07.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 -- 172.21.15.44:0/1736220696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0ce5c0 msgr2=0x7fb51c06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 --2- 172.21.15.44:0/1736220696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0ce5c0 0x7fb51c06d9b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb504009900 tx=0x7fb50402a610 comp rx=0 tx=0).stop 2024-07-17T08:36:07.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 -- 172.21.15.44:0/1736220696 shutdown_connections 2024-07-17T08:36:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 --2- 172.21.15.44:0/1736220696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0ce5c0 0x7fb51c06d9b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 --2- 172.21.15.44:0/1736220696 >> v2:172.21.15.175:3300/0 conn(0x7fb51c0b8c80 0x7fb51c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.620+0000 7fb51be00640 1 -- 172.21.15.44:0/1736220696 >> 172.21.15.44:0/1736220696 conn(0x7fb51c0cfec0 msgr2=0x7fb51c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.621+0000 7fb51be00640 1 -- 172.21.15.44:0/1736220696 shutdown_connections 2024-07-17T08:36:07.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.621+0000 7fb51be00640 1 -- 172.21.15.44:0/1736220696 wait complete. 2024-07-17T08:36:07.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.623+0000 7fb51be00640 1 Processor -- start 2024-07-17T08:36:07.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.623+0000 7fb51be00640 1 -- start start 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0b5400 0x7fb51c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 0x7fb51c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c06e780 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c06e600 con 0x7fb51c0b8c80 2024-07-17T08:36:07.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.624+0000 7fb51be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c06e900 con 0x7fb51c0b5400 2024-07-17T08:36:07.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46252/0 (socket says 172.21.15.44:46252) 2024-07-17T08:36:07.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 -- 172.21.15.44:0/3917633652 learned_addr learned my addr 172.21.15.44:0/3917633652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51aa00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0b5400 0x7fb51c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51a000640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 0x7fb51c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 -- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0b5400 msgr2=0x7fb51c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0b5400 0x7fb51c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 -- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 msgr2=0x7fb51c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 0x7fb51c0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51b400640 1 -- 172.21.15.44:0/3917633652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c0ca510 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.625+0000 7fb51a000640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 0x7fb51c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb51b400640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c7a0 tx=0x7fb51400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514003b70 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb514003d30 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400aab0 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c1e7ce0 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.626+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c1e82a0 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.628+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51c0b2840 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.629+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fb51400ac50 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.630+0000 7fb512a00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 0x7fb4f80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:07.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.630+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7fb514098d10 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.630+0000 7fb51aa00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 0x7fb4f80792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.632+0000 7fb51aa00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 0x7fb4f80792b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009bd0 tx=0x7fb50c005f90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.634+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb51c0b2840 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.886+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb51c15e520 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.888+0000 7fb512a00640 1 -- 172.21.15.44:0/3917633652 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5234 (secure 0 0 0) 0x7fb51409e020 con 0x7fb51c0ce5c0 2024-07-17T08:36:07.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:07.889 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":121,"btime":"2024-07-17T08:36:04:377961+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":10587,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3444850603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3444850603},{"type":"v1","addr":"172.21.15.44:6835","nonce":3444850603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:07.889 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 121 2024-07-17T08:36:07.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 msgr2=0x7fb4f80792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 0x7fb4f80792b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009bd0 tx=0x7fb50c005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:07.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 msgr2=0x7fb51c1e8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:07.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c7a0 tx=0x7fb51400cc60 comp rx=0 tx=0).stop 2024-07-17T08:36:07.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 shutdown_connections 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb4f8076df0 0x7fb4f80792b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb51c0ce5c0 0x7fb51c1e8b10 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb51c0b8c80 0x7fb51c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 --2- 172.21.15.44:0/3917633652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51c0b5400 0x7fb51c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 >> 172.21.15.44:0/3917633652 conn(0x7fb51c0cfec0 msgr2=0x7fb51c0c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.891+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 shutdown_connections 2024-07-17T08:36:07.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:07.892+0000 7fb51be00640 1 -- 172.21.15.44:0/3917633652 wait complete. 2024-07-17T08:36:07.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.125+0000 7f62ae800640 1 -- 172.21.15.44:0/4167847777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b5400 msgr2=0x7f62a815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.125+0000 7f62ae800640 1 --2- 172.21.15.44:0/4167847777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b5400 0x7f62a815e2d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009ca0 tx=0x7f62a002aac0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 -- 172.21.15.44:0/4167847777 shutdown_connections 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 --2- 172.21.15.44:0/4167847777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b5400 0x7f62a815e2d0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 --2- 172.21.15.44:0/4167847777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80b8c80 0x7f62a80b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 -- 172.21.15.44:0/4167847777 >> 172.21.15.44:0/4167847777 conn(0x7f62a80cfec0 msgr2=0x7f62a80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 -- 172.21.15.44:0/4167847777 shutdown_connections 2024-07-17T08:36:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.126+0000 7f62ae800640 1 -- 172.21.15.44:0/4167847777 wait complete. 2024-07-17T08:36:08.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.127+0000 7f62ae800640 1 Processor -- start 2024-07-17T08:36:08.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.128+0000 7f62ae800640 1 -- start start 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.128+0000 7f62ae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80b5400 0x7f62a814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ae800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ae800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80ce5c0 0x7f62a80aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ae800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62a806e0c0 con 0x7f62a80b8c80 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ae800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80b9380 con 0x7f62a80b5400 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ae800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62a806e240 con 0x7f62a80ce5c0 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46270/0 (socket says 172.21.15.44:46270) 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 -- 172.21.15.44:0/3822632225 learned_addr learned my addr 172.21.15.44:0/3822632225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:08.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ad400640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80b5400 0x7f62a814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62ade00640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80ce5c0 0x7f62a80aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 -- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80ce5c0 msgr2=0x7f62a80aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80ce5c0 0x7f62a80aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 -- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80b5400 msgr2=0x7f62a814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80b5400 0x7f62a814ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.129+0000 7f62aca00640 1 -- 172.21.15.44:0/3822632225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a814fdf0 con 0x7f62a80b8c80 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62aca00640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f629800c3a0 tx=0x7f629800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298017070 con 0x7f62a80b8c80 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62980053c0 con 0x7f62a80b8c80 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298003850 con 0x7f62a80b8c80 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81500d0 con 0x7f62a80b8c80 2024-07-17T08:36:08.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.130+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a8150690 con 0x7f62a80b8c80 2024-07-17T08:36:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.132+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a806e6f0 con 0x7f62a80b8c80 2024-07-17T08:36:08.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.133+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f629801a070 con 0x7f62a80b8c80 2024-07-17T08:36:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.134+0000 7f62a5600640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 0x7f628c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.134+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7f6298098210 con 0x7f62a80b8c80 2024-07-17T08:36:08.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.135+0000 7f62ad400640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 0x7f628c0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.136+0000 7f62ad400640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 0x7f628c0792e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c005850 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:08.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.139+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6298060c50 con 0x7f62a80b8c80 2024-07-17T08:36:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.390+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a815e5f0 con 0x7f62a80b8c80 2024-07-17T08:36:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.391+0000 7f62a5600640 1 -- 172.21.15.44:0/3822632225 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5234 (secure 0 0 0) 0x7f6298060550 con 0x7f62a80b8c80 2024-07-17T08:36:08.392 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:08.392 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":121,"btime":"2024-07-17T08:36:04:377961+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":10587,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3444850603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3444850603},{"type":"v1","addr":"172.21.15.44:6835","nonce":3444850603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:35:33.908968+0000","modified":"2024-07-17T08:35:36.972151+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8329":{"gid":8329,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6834/3288748832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3288748832},{"type":"v1","addr":"172.21.15.44:6834","nonce":3288748832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8329,"qdb_cluster":[8329]},"id":14}]} 2024-07-17T08:36:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 121 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 msgr2=0x7f628c0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 0x7f628c0792e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c005850 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 msgr2=0x7f62a814f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f629800c3a0 tx=0x7f629800c860 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 shutdown_connections 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f628c076e20 0x7f628c0792e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a80ce5c0 0x7f62a80aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a80b8c80 0x7f62a814f370 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 --2- 172.21.15.44:0/3822632225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80b5400 0x7f62a814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 >> 172.21.15.44:0/3822632225 conn(0x7f62a80cfec0 msgr2=0x7f62a80b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 shutdown_connections 2024-07-17T08:36:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.394+0000 7f62ae800640 1 -- 172.21.15.44:0/3822632225 wait complete. 2024-07-17T08:36:08.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:08.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.632+0000 7efef2c00640 1 -- 172.21.15.44:0/1418668990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed4003aa0 msgr2=0x7efed4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.632+0000 7efef2c00640 1 --2- 172.21.15.44:0/1418668990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efed4003aa0 0x7efed4003ec0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7efee000c310 tx=0x7efed4005830 comp rx=0 tx=0).stop 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.632+0000 7efef2200640 1 -- 172.21.15.44:0/1418668990 reap_dead start 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.633+0000 7efef2c00640 1 -- 172.21.15.44:0/1418668990 shutdown_connections 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.633+0000 7efef2c00640 1 -- 172.21.15.44:0/1418668990 >> 172.21.15.44:0/1418668990 conn(0x7efeec0cfec0 msgr2=0x7efeec0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.633+0000 7efef2c00640 1 -- 172.21.15.44:0/1418668990 shutdown_connections 2024-07-17T08:36:08.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.633+0000 7efef2c00640 1 -- 172.21.15.44:0/1418668990 wait complete. 2024-07-17T08:36:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.634+0000 7efef2c00640 1 Processor -- start 2024-07-17T08:36:08.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.635+0000 7efef2c00640 1 -- start start 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed4003aa0 0x7efeec0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeec0cc520 0x7efeec0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efeec15d650 con 0x7efeec0b6c10 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efeec15d4d0 con 0x7efeec0cc520 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efeec0b5740 con 0x7efed4003aa0 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46286/0 (socket says 172.21.15.44:46286) 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef0e00640 1 -- 172.21.15.44:0/3743966889 learned_addr learned my addr 172.21.15.44:0/3743966889 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef2200640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeec0cc520 0x7efeec0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.636+0000 7efef1800640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed4003aa0 0x7efeec0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 -- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed4003aa0 msgr2=0x7efeec0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed4003aa0 0x7efeec0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 -- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeec0cc520 msgr2=0x7efeec0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeec0cc520 0x7efeec0cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 -- 172.21.15.44:0/3743966889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec1f41d0 con 0x7efeec0b6c10 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef0e00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7efed800e770 tx=0x7efed800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed800cc60 con 0x7efeec0b6c10 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed800ee80 con 0x7efeec0b6c10 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8005280 con 0x7efeec0b6c10 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec0b7780 con 0x7efeec0b6c10 2024-07-17T08:36:08.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.637+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec1f5020 con 0x7efeec0b6c10 2024-07-17T08:36:08.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.639+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeec0ccee0 con 0x7efeec0b6c10 2024-07-17T08:36:08.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.640+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7efed8005420 con 0x7efeec0b6c10 2024-07-17T08:36:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.642+0000 7efee9600640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 0x7efecc0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:08.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.642+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7efed8099690 con 0x7efeec0b6c10 2024-07-17T08:36:08.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.642+0000 7efef1800640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 0x7efecc0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:08.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.644+0000 7efef1800640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 0x7efecc0792e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efee0033000 tx=0x7efeec161940 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:08.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.646+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efed8016030 con 0x7efeec0b6c10 2024-07-17T08:36:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.859+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efeec15e2f0 con 0x7efeec0b6c10 2024-07-17T08:36:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.861+0000 7efee9600640 1 -- 172.21.15.44:0/3743966889 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) ==== 74+0+19095 (secure 0 0 0) 0x7efed8061ef0 con 0x7efeec0b6c10 2024-07-17T08:36:08.862 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:08.862 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":232,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:06.401323+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:35:32.313593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:35:32.921150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"227","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/2462391516":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1904359110":"2024-07-18T08:36:03.234736+0000","172.21.15.44:6832/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/3038109628":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6837/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6834/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1799654570":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/2557831081":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/749128801":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/485731160":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/363211359":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/3582055205":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1458277022":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1949400205":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6837/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/3512613843":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6834/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/101784654":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/4286139695":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1591261735":"2024-07-18T08:35:51.813306+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 msgr2=0x7efecc0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 0x7efecc0792e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efee0033000 tx=0x7efeec161940 comp rx=0 tx=0).stop 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 msgr2=0x7efeec0b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7efed800e770 tx=0x7efed800ec30 comp rx=0 tx=0).stop 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 shutdown_connections 2024-07-17T08:36:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efecc076e20 0x7efecc0792e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeec0cc520 0x7efeec0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efeec0b6c10 0x7efeec0b7090 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 --2- 172.21.15.44:0/3743966889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efed4003aa0 0x7efeec0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 >> 172.21.15.44:0/3743966889 conn(0x7efeec0cfec0 msgr2=0x7efeec0b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.864+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 shutdown_connections 2024-07-17T08:36:08.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:08.865+0000 7efef2c00640 1 -- 172.21.15.44:0/3743966889 wait complete. 2024-07-17T08:36:08.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 -- 172.21.15.44:0/1396477474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 msgr2=0x7f32700b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 --2- 172.21.15.44:0/1396477474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700b5400 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f326c005fd0 tx=0x7f326c02a820 comp rx=0 tx=0).stop 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 -- 172.21.15.44:0/1396477474 shutdown_connections 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 --2- 172.21.15.44:0/1396477474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700b5400 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 --2- 172.21.15.44:0/1396477474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327015d300 0x7f32700b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 -- 172.21.15.44:0/1396477474 >> 172.21.15.44:0/1396477474 conn(0x7f32700cfec0 msgr2=0x7f32700d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 -- 172.21.15.44:0/1396477474 shutdown_connections 2024-07-17T08:36:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.094+0000 7f3277200640 1 -- 172.21.15.44:0/1396477474 wait complete. 2024-07-17T08:36:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.096+0000 7f3277200640 1 Processor -- start 2024-07-17T08:36:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.096+0000 7f3277200640 1 -- start start 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700d2fa0 0x7f32701f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327015d300 0x7f32700ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32700d1220 con 0x7f32700d2900 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f327006e0c0 con 0x7f32700d2fa0 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3277200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f32700c7490 con 0x7f327015d300 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3275e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3275e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52098/0 (socket says 172.21.15.44:52098) 2024-07-17T08:36:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.097+0000 7f3275e00640 1 -- 172.21.15.44:0/4036041419 learned_addr learned my addr 172.21.15.44:0/4036041419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 -- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327015d300 msgr2=0x7f32700ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327015d300 0x7f32700ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 -- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700d2fa0 msgr2=0x7f32701f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275400640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700d2fa0 0x7f32701f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700d2fa0 0x7f32701f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 -- 172.21.15.44:0/4036041419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701f3fe0 con 0x7f32700d2900 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3275e00640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f326400c970 tx=0x7f326400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264007bb0 con 0x7f32700d2900 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3264007e40 con 0x7f32700d2900 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264005280 con 0x7f32700d2900 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.098+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701f42c0 con 0x7f32700d2900 2024-07-17T08:36:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.099+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701f4880 con 0x7f32700d2900 2024-07-17T08:36:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.100+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327006e510 con 0x7f32700d2900 2024-07-17T08:36:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.101+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f3264005420 con 0x7f32700d2900 2024-07-17T08:36:09.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.102+0000 7f326a000640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 0x7f32540792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.102+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7592+0+0 (secure 0 0 0) 0x7f32640993b0 con 0x7f32700d2900 2024-07-17T08:36:09.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.103+0000 7f3275400640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 0x7f32540792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.104+0000 7f3275400640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 0x7f32540792b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f327015ccd0 tx=0x7f3260009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.106+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3264014030 con 0x7f32700d2900 2024-07-17T08:36:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.357+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7f32700cb560 con 0x7f32700d2900 2024-07-17T08:36:09.359 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:36:09.358+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:36:09.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.361+0000 7f326a000640 1 -- 172.21.15.44:0/4036041419 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v122) ==== 167+0+0 (secure 0 0 0) 0x7f3264061d60 con 0x7f32700d2900 2024-07-17T08:36:09.362 INFO:teuthology.orchestra.run.smithi044.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:36:09.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.364+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 msgr2=0x7f32540792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 0x7f32540792b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f327015ccd0 tx=0x7f3260009040 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 msgr2=0x7f32700c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f326400c970 tx=0x7f326400ce30 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 shutdown_connections 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3254076df0 0x7f32540792b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327015d300 0x7f32700ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32700d2fa0 0x7f32701f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 --2- 172.21.15.44:0/4036041419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32700d2900 0x7f32700c6f50 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 >> 172.21.15.44:0/4036041419 conn(0x7f32700cfec0 msgr2=0x7f32700d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:09.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 shutdown_connections 2024-07-17T08:36:09.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.365+0000 7f3277200640 1 -- 172.21.15.44:0/4036041419 wait complete. 2024-07-17T08:36:09.383 INFO:tasks.ceph.mds.c.smithi044.stderr: -1923> 2024-07-17T08:35:39.256+0000 7f93fe200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:36:09.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-17T08:36:09.444 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:36:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 -- 172.21.15.44:0/629268329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 msgr2=0x7f436815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 --2- 172.21.15.44:0/629268329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f436815a5a0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f43680b1f20 tx=0x7f435c02ade0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 -- 172.21.15.44:0/629268329 shutdown_connections 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 --2- 172.21.15.44:0/629268329 >> v2:172.21.15.175:3300/0 conn(0x7f43680ac3a0 0x7f43680ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 --2- 172.21.15.44:0/629268329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f436815a5a0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 -- 172.21.15.44:0/629268329 >> 172.21.15.44:0/629268329 conn(0x7f43680b0eb0 msgr2=0x7f43680b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 -- 172.21.15.44:0/629268329 shutdown_connections 2024-07-17T08:36:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.580+0000 7f4370200640 1 -- 172.21.15.44:0/629268329 wait complete. 2024-07-17T08:36:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.582+0000 7f4370200640 1 Processor -- start 2024-07-17T08:36:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.582+0000 7f4370200640 1 -- start start 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43680ac3a0 0x7f436814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 0x7f4368073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f436806f850 con 0x7f436806eb90 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f436806f6d0 con 0x7f43680b3280 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f4370200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f436814fd60 con 0x7f43680ac3a0 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f436d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f436d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52130/0 (socket says 172.21.15.44:52130) 2024-07-17T08:36:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.583+0000 7f436d800640 1 -- 172.21.15.44:0/2511517842 learned_addr learned my addr 172.21.15.44:0/2511517842 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 -- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43680ac3a0 msgr2=0x7f436814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43680ac3a0 0x7f436814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 -- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 msgr2=0x7f4368073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436e200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 0x7f4368073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 0x7f4368073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 -- 172.21.15.44:0/2511517842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43680726d0 con 0x7f436806eb90 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436e200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 0x7f4368073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f436d800640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b3c0 tx=0x7f435c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.584+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c040020 con 0x7f436806eb90 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.585+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f435c02b930 con 0x7f436806eb90 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.585+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43680729b0 con 0x7f436806eb90 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.585+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c02bd40 con 0x7f436806eb90 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.585+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4368072f70 con 0x7f436806eb90 2024-07-17T08:36:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.586+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43680b4200 con 0x7f436806eb90 2024-07-17T08:36:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.587+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f435c043020 con 0x7f436806eb90 2024-07-17T08:36:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.588+0000 7f4365600640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 0x7f43480792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.588+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7678+0+0 (secure 0 0 0) 0x7f435c0b9410 con 0x7f436806eb90 2024-07-17T08:36:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.589+0000 7f436ce00640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 0x7f43480792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:09.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.590+0000 7f436ce00640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 0x7f43480792b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f436815a860 tx=0x7f4354006ba0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.593+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f435c04a030 con 0x7f436806eb90 2024-07-17T08:36:09.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:36:09.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:36:09.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:09.850+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f436815ae10 con 0x7f436806eb90 2024-07-17T08:36:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.451+0000 7f4365600640 1 -- 172.21.15.44:0/2511517842 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v123) ==== 107+0+0 (secure 0 0 0) 0x7f435c081dd0 con 0x7f436806eb90 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 msgr2=0x7f43480792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 0x7f43480792b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f436815a860 tx=0x7f4354006ba0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 msgr2=0x7f4368073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b3c0 tx=0x7f435c009850 comp rx=0 tx=0).stop 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 shutdown_connections 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4348076df0 0x7f43480792b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43680b3280 0x7f4368073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43680ac3a0 0x7f436814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 --2- 172.21.15.44:0/2511517842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f436806eb90 0x7f4368073770 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 >> 172.21.15.44:0/2511517842 conn(0x7f43680b0eb0 msgr2=0x7f436815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 shutdown_connections 2024-07-17T08:36:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.455+0000 7f4370200640 1 -- 172.21.15.44:0/2511517842 wait complete. 2024-07-17T08:36:10.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-17T08:36:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.676+0000 7f0af3200640 1 -- 172.21.15.44:0/4294422577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 msgr2=0x7f0ad8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.676+0000 7f0af3200640 1 --2- 172.21.15.44:0/4294422577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0ad8003e50 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40016c0 tx=0x7f0ad8005830 comp rx=0 tx=0).stop 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.676+0000 7f0af2800640 1 -- 172.21.15.44:0/4294422577 reap_dead start 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.677+0000 7f0af3200640 1 -- 172.21.15.44:0/4294422577 shutdown_connections 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.677+0000 7f0af3200640 1 -- 172.21.15.44:0/4294422577 >> 172.21.15.44:0/4294422577 conn(0x7f0af40cf050 msgr2=0x7f0af40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.677+0000 7f0af3200640 1 -- 172.21.15.44:0/4294422577 shutdown_connections 2024-07-17T08:36:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.677+0000 7f0af3200640 1 -- 172.21.15.44:0/4294422577 wait complete. 2024-07-17T08:36:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.678+0000 7f0af3200640 1 Processor -- start 2024-07-17T08:36:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.679+0000 7f0af3200640 1 -- start start 2024-07-17T08:36:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 0x7f0af41f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 0x7f0af4156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40ae870 con 0x7f0ad8003a30 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40ae6f0 con 0x7f0af41f4640 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0af4153e60 con 0x7f0af41535b0 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52154/0 (socket says 172.21.15.44:52154) 2024-07-17T08:36:10.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 -- 172.21.15.44:0/3535773065 learned_addr learned my addr 172.21.15.44:0/3535773065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1400640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 0x7f0af41f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af2800640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 0x7f0af4156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 -- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 msgr2=0x7f0af41f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 0x7f0af41f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 -- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 msgr2=0x7f0af4156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 0x7f0af4156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1e00640 1 -- 172.21.15.44:0/3535773065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af41f50a0 con 0x7f0ad8003a30 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af1400640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 0x7f0af41f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.680+0000 7f0af2800640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 0x7f0af4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0af1e00640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40017e0 tx=0x7f0ae4006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4041070 con 0x7f0ad8003a30 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ae4003af0 con 0x7f0ad8003a30 2024-07-17T08:36:10.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4038070 con 0x7f0ad8003a30 2024-07-17T08:36:10.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af41f4ae0 con 0x7f0ad8003a30 2024-07-17T08:36:10.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.681+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af41f7800 con 0x7f0ad8003a30 2024-07-17T08:36:10.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.683+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af406f720 con 0x7f0ad8003a30 2024-07-17T08:36:10.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.684+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f0ae4030070 con 0x7f0ad8003a30 2024-07-17T08:36:10.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.685+0000 7f0aea000640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 0x7f0ad40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:10.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.685+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7678+0+0 (secure 0 0 0) 0x7f0ae40b82c0 con 0x7f0ad8003a30 2024-07-17T08:36:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.685+0000 7f0af1400640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 0x7f0ad40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:10.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.686+0000 7f0af1400640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 0x7f0ad40792b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005d80 tx=0x7f0ae0005c90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:10.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.689+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0af41f50a0 con 0x7f0ad8003a30 2024-07-17T08:36:10.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:10.889+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0af415ef30 con 0x7f0ad8003a30 2024-07-17T08:36:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.462+0000 7f0aea000640 1 -- 172.21.15.44:0/3535773065 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v234) ==== 190+0+0 (secure 0 0 0) 0x7f0ae4080d10 con 0x7f0ad8003a30 2024-07-17T08:36:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_metadata' removed 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 msgr2=0x7f0ad40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 0x7f0ad40792b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005d80 tx=0x7f0ae0005c90 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 msgr2=0x7f0af41f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40017e0 tx=0x7f0ae4006360 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 shutdown_connections 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ad4076df0 0x7f0ad40792b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0af41f4640 0x7f0af4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0af41535b0 0x7f0af41f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 --2- 172.21.15.44:0/3535773065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad8003a30 0x7f0af41f8000 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 >> 172.21.15.44:0/3535773065 conn(0x7f0af40cf050 msgr2=0x7f0af415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.466+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 shutdown_connections 2024-07-17T08:36:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.467+0000 7f0af3200640 1 -- 172.21.15.44:0/3535773065 wait complete. 2024-07-17T08:36:11.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-17T08:36:11.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.696+0000 7faa09a00640 1 -- 172.21.15.44:0/4027844582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 msgr2=0x7faa040cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.696+0000 7faa09a00640 1 --2- 172.21.15.44:0/4027844582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040cdc70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009830 tx=0x7fa9f002a400 comp rx=0 tx=0).stop 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 -- 172.21.15.44:0/4027844582 shutdown_connections 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 --2- 172.21.15.44:0/4027844582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040cdc70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 --2- 172.21.15.44:0/4027844582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa04160980 0x7faa041619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 -- 172.21.15.44:0/4027844582 >> 172.21.15.44:0/4027844582 conn(0x7faa040d0100 msgr2=0x7faa040d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 -- 172.21.15.44:0/4027844582 shutdown_connections 2024-07-17T08:36:11.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.697+0000 7faa09a00640 1 -- 172.21.15.44:0/4027844582 wait complete. 2024-07-17T08:36:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.699+0000 7faa09a00640 1 Processor -- start 2024-07-17T08:36:11.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.699+0000 7faa09a00640 1 -- start start 2024-07-17T08:36:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.700+0000 7faa09a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 0x7faa040ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:11.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.700+0000 7faa09a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faa040b5760 con 0x7faa04160980 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa040b55e0 con 0x7faa040d2a40 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faa040d1af0 con 0x7faa040b4f20 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52162/0 (socket says 172.21.15.44:52162) 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 -- 172.21.15.44:0/142517902 learned_addr learned my addr 172.21.15.44:0/142517902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa03400640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa03e00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 0x7faa040ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 -- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 msgr2=0x7faa040ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 0x7faa040ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 -- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 msgr2=0x7faa040ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.701+0000 7faa09000640 1 -- 172.21.15.44:0/142517902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa040ab620 con 0x7faa04160980 2024-07-17T08:36:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7faa03e00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 0x7faa040ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:36:11.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7faa09000640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c7a0 tx=0x7fa9fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc003b70 con 0x7faa04160980 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9fc003d30 con 0x7faa04160980 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0414d910 con 0x7faa04160980 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00aa50 con 0x7faa04160980 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.702+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041f76b0 con 0x7faa04160980 2024-07-17T08:36:11.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.704+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa040d1fa0 con 0x7faa04160980 2024-07-17T08:36:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.705+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fa9fc00abf0 con 0x7faa04160980 2024-07-17T08:36:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.706+0000 7fa9fbe00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 0x7fa9e40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.706+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(234..234 src has 1..234) ==== 7234+0+0 (secure 0 0 0) 0x7fa9fc098b50 con 0x7faa04160980 2024-07-17T08:36:11.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.707+0000 7faa03e00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 0x7fa9e40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:11.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.708+0000 7faa03e00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 0x7fa9e40792b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005fd0 tx=0x7fa9f4009260 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.710+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa9fc0616b0 con 0x7faa04160980 2024-07-17T08:36:11.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:11.922+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faa040b7f90 con 0x7faa04160980 2024-07-17T08:36:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.475+0000 7fa9fbe00640 1 -- 172.21.15.44:0/142517902 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v235) ==== 178+0+0 (secure 0 0 0) 0x7fa9fc060fb0 con 0x7faa04160980 2024-07-17T08:36:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_data' removed 2024-07-17T08:36:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 msgr2=0x7fa9e40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 0x7fa9e40792b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005fd0 tx=0x7fa9f4009260 comp rx=0 tx=0).stop 2024-07-17T08:36:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 msgr2=0x7faa040b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c7a0 tx=0x7fa9fc00cc60 comp rx=0 tx=0).stop 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 shutdown_connections 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa9e4076df0 0x7fa9e40792b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa04160980 0x7faa040b84b0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.479+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa040d2a40 0x7faa040ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.480+0000 7faa09a00640 1 --2- 172.21.15.44:0/142517902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa040b4f20 0x7faa040ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.480+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 >> 172.21.15.44:0/142517902 conn(0x7faa040d0100 msgr2=0x7faa040b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.480+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 shutdown_connections 2024-07-17T08:36:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.481+0000 7faa09a00640 1 -- 172.21.15.44:0/142517902 wait complete. 2024-07-17T08:36:12.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 -- 172.21.15.44:0/1617505725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ac003af0 msgr2=0x7f65ac003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 --2- 172.21.15.44:0/1617505725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ac003af0 0x7f65ac003f10 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fc0 tx=0x7f65ac005830 comp rx=0 tx=0).stop 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c6800640 1 -- 172.21.15.44:0/1617505725 reap_dead start 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 -- 172.21.15.44:0/1617505725 shutdown_connections 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 -- 172.21.15.44:0/1617505725 >> 172.21.15.44:0/1617505725 conn(0x7f65c80cf0d0 msgr2=0x7f65c80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 -- 172.21.15.44:0/1617505725 shutdown_connections 2024-07-17T08:36:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.712+0000 7f65c7200640 1 -- 172.21.15.44:0/1617505725 wait complete. 2024-07-17T08:36:12.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.714+0000 7f65c7200640 1 Processor -- start 2024-07-17T08:36:12.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.714+0000 7f65c7200640 1 -- start start 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65ac003af0 0x7f65c814e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c80cd040 0x7f65c8148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65c815b3d0 con 0x7f65c80cd8d0 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65c815b250 con 0x7f65c80cd040 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65c815b550 con 0x7f65ac003af0 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:12.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52186/0 (socket says 172.21.15.44:52186) 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c6800640 1 -- 172.21.15.44:0/158593098 learned_addr learned my addr 172.21.15.44:0/158593098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c5e00640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65ac003af0 0x7f65c814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65bd400640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c80cd040 0x7f65c8148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c6800640 1 -- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65ac003af0 msgr2=0x7f65c814e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.715+0000 7f65c6800640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65ac003af0 0x7f65c814e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c6800640 1 -- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c80cd040 msgr2=0x7f65c8148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c6800640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c80cd040 0x7f65c8148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c6800640 1 -- 172.21.15.44:0/158593098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c814eff0 con 0x7f65c80cd8d0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c6800640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ceb10 tx=0x7f65c000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0017070 con 0x7f65c80cd8d0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c0005e50 con 0x7f65c80cd8d0 2024-07-17T08:36:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c000a870 con 0x7f65c80cd8d0 2024-07-17T08:36:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c814f2d0 con 0x7f65c80cd8d0 2024-07-17T08:36:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.716+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c80cd4a0 con 0x7f65c80cd8d0 2024-07-17T08:36:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.718+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c80b20a0 con 0x7f65c80cd8d0 2024-07-17T08:36:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.719+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f65c000aa10 con 0x7f65c80cd8d0 2024-07-17T08:36:12.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.720+0000 7f65bea00640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 0x7f65a40792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:12.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.720+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f65c0097d60 con 0x7f65c80cd8d0 2024-07-17T08:36:12.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.721+0000 7f65c5e00640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 0x7f65a40792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:12.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.722+0000 7f65c5e00640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 0x7f65a40792e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65b8033000 tx=0x7f65c80ace00 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:12.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.724+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f65c0060a70 con 0x7f65c80cd8d0 2024-07-17T08:36:12.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.934+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f65c80b2560 con 0x7f65c80cd8d0 2024-07-17T08:36:12.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.941+0000 7f65bea00640 1 -- 172.21.15.44:0/158593098 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v75) ==== 71+0+0 (secure 0 0 0) 0x7f65c0060370 con 0x7f65c80cd8d0 2024-07-17T08:36:12.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.944+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 msgr2=0x7f65a40792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.944+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 0x7f65a40792e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65b8033000 tx=0x7f65c80ace00 comp rx=0 tx=0).stop 2024-07-17T08:36:12.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.944+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 msgr2=0x7f65c80cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:12.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.944+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ceb10 tx=0x7f65c000cb90 comp rx=0 tx=0).stop 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 shutdown_connections 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f65a4076e20 0x7f65a40792e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65c80cd8d0 0x7f65c80cdd10 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65c80cd040 0x7f65c8148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 --2- 172.21.15.44:0/158593098 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65ac003af0 0x7f65c814e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 >> 172.21.15.44:0/158593098 conn(0x7f65c80cf0d0 msgr2=0x7f65c80b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 shutdown_connections 2024-07-17T08:36:12.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:12.945+0000 7f65c7200640 1 -- 172.21.15.44:0/158593098 wait complete. 2024-07-17T08:36:12.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2024-07-17T08:36:13.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 -- 172.21.15.44:0/4189122436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f369c004800 msgr2=0x7f369c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 --2- 172.21.15.44:0/4189122436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f369c004800 0x7f369c004c80 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c710 tx=0x7f369c005830 comp rx=0 tx=0).stop 2024-07-17T08:36:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 -- 172.21.15.44:0/4189122436 shutdown_connections 2024-07-17T08:36:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 --2- 172.21.15.44:0/4189122436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f369c004800 0x7f369c004c80 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 -- 172.21.15.44:0/4189122436 >> 172.21.15.44:0/4189122436 conn(0x7f36b80d00a0 msgr2=0x7f36b80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 -- 172.21.15.44:0/4189122436 shutdown_connections 2024-07-17T08:36:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.159+0000 7f36bea00640 1 -- 172.21.15.44:0/4189122436 wait complete. 2024-07-17T08:36:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.161+0000 7f36bea00640 1 Processor -- start 2024-07-17T08:36:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.161+0000 7f36bea00640 1 -- start start 2024-07-17T08:36:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c004800 0x7f36b81e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36b81f7650 0x7f36b80b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f36b806e480 con 0x7f36b81f7ff0 2024-07-17T08:36:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80b56f0 con 0x7f369c004800 2024-07-17T08:36:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36b806e600 con 0x7f36b81f7650 2024-07-17T08:36:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52198/0 (socket says 172.21.15.44:52198) 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36b7400640 1 -- 172.21.15.44:0/1409960286 learned_addr learned my addr 172.21.15.44:0/1409960286 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36b7400640 1 -- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36b81f7650 msgr2=0x7f36b80b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.162+0000 7f36bca00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36b81f7650 0x7f36b80b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36b7400640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36b81f7650 0x7f36b80b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36b7400640 1 -- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c004800 msgr2=0x7f36b81e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36b7400640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c004800 0x7f36b81e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36b7400640 1 -- 172.21.15.44:0/1409960286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b815e020 con 0x7f36b81f7ff0 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36b7400640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c830 tx=0x7f36a4006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4004830 con 0x7f36b81f7ff0 2024-07-17T08:36:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a4039070 con 0x7f36b81f7ff0 2024-07-17T08:36:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b8160760 con 0x7f36b81f7ff0 2024-07-17T08:36:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4004d60 con 0x7f36b81f7ff0 2024-07-17T08:36:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.163+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b81e7c60 con 0x7f36b81f7ff0 2024-07-17T08:36:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.165+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b81e7a70 con 0x7f36b81f7ff0 2024-07-17T08:36:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.166+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f36a4043030 con 0x7f36b81f7ff0 2024-07-17T08:36:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.167+0000 7f36afe00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 0x7f36980792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.167+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f36a40b8370 con 0x7f36b81f7ff0 2024-07-17T08:36:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.168+0000 7f36b7e00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 0x7f36980792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.169+0000 7f36b7e00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 0x7f36980792b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36a8001860 tx=0x7f36a80014f0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.172+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f36a40811d0 con 0x7f36b81f7ff0 2024-07-17T08:36:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.464+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) -- 0x7f36b8161020 con 0x7f36b81f7ff0 2024-07-17T08:36:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.478+0000 7f36afe00640 1 -- 172.21.15.44:0/1409960286 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3318) ==== 186+0+0 (secure 0 0 0) 0x7f36a4080ad0 con 0x7f36b81f7ff0 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.481+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 msgr2=0x7f36980792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.481+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 0x7f36980792b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f36a8001860 tx=0x7f36a80014f0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.481+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 msgr2=0x7f36b815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.481+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c830 tx=0x7f36a4006380 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 shutdown_connections 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3698076df0 0x7f36980792b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36b81f7650 0x7f36b80b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b81f7ff0 0x7f36b815dae0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 --2- 172.21.15.44:0/1409960286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c004800 0x7f36b81e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 >> 172.21.15.44:0/1409960286 conn(0x7f36b80d00a0 msgr2=0x7f36b80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 shutdown_connections 2024-07-17T08:36:13.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.482+0000 7f36bea00640 1 -- 172.21.15.44:0/1409960286 wait complete. 2024-07-17T08:36:13.502 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2024-07-17T08:36:13.502 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2024-07-17T08:36:13.502 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2024-07-17T08:36:13.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.696+0000 7f166aa00640 1 -- 172.21.15.44:0/1014618519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 msgr2=0x7f164800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.696+0000 7f166aa00640 1 --2- 172.21.15.44:0/1014618519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f164800b9c0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c610 tx=0x7f1648005830 comp rx=0 tx=0).stop 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.696+0000 7f166aa00640 1 -- 172.21.15.44:0/1014618519 shutdown_connections 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.696+0000 7f166aa00640 1 --2- 172.21.15.44:0/1014618519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f164800b9c0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.696+0000 7f166aa00640 1 -- 172.21.15.44:0/1014618519 >> 172.21.15.44:0/1014618519 conn(0x7f16640b0fb0 msgr2=0x7f16640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.697+0000 7f166aa00640 1 -- 172.21.15.44:0/1014618519 shutdown_connections 2024-07-17T08:36:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.697+0000 7f166aa00640 1 -- 172.21.15.44:0/1014618519 wait complete. 2024-07-17T08:36:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.698+0000 7f166aa00640 1 Processor -- start 2024-07-17T08:36:13.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.698+0000 7f166aa00640 1 -- start start 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16641f3b80 0x7f166415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f166414f2d0 0x7f166414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16640ac810 con 0x7f16480051c0 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16640ac690 con 0x7f16641f3b80 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.699+0000 7f166aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16640ac990 con 0x7f166414f2d0 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52224/0 (socket says 172.21.15.44:52224) 2024-07-17T08:36:13.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 -- 172.21.15.44:0/2573624123 learned_addr learned my addr 172.21.15.44:0/2573624123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 -- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f166414f2d0 msgr2=0x7f166414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f166414f2d0 0x7f166414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 -- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16641f3b80 msgr2=0x7f166415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16641f3b80 0x7f166415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 -- 172.21.15.44:0/2573624123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166414fc90 con 0x7f16480051c0 2024-07-17T08:36:13.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.700+0000 7f1663e00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f165400c430 tx=0x7f165400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.701+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654017070 con 0x7f16480051c0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.701+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16540053c0 con 0x7f16480051c0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.701+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654003850 con 0x7f16480051c0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.701+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166415e150 con 0x7f16480051c0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.701+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1664152ff0 con 0x7f16480051c0 2024-07-17T08:36:13.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.702+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16640adaa0 con 0x7f16480051c0 2024-07-17T08:36:13.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.703+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f165401a070 con 0x7f16480051c0 2024-07-17T08:36:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.705+0000 7f165be00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 0x7f16440792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:13.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.705+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f1654097f00 con 0x7f16480051c0 2024-07-17T08:36:13.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.705+0000 7f1663400640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 0x7f16440792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:13.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.706+0000 7f1663400640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 0x7f16440792b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f166415b950 tx=0x7f1650005740 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:13.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.709+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f166414fc90 con 0x7f16480051c0 2024-07-17T08:36:13.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.921+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f166415d8a0 con 0x7f16480051c0 2024-07-17T08:36:13.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.923+0000 7f165be00640 1 -- 172.21.15.44:0/2573624123 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) ==== 74+0+15503 (secure 0 0 0) 0x7f1654060ce0 con 0x7f16480051c0 2024-07-17T08:36:13.924 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:13.924 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":235,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:12.463230+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/2462391516":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1904359110":"2024-07-18T08:36:03.234736+0000","172.21.15.44:6832/1413771882":"2024-07-18T08:36:03.375398+0000","172.21.15.175:0/3038109628":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6837/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.175:6834/1605633467":"2024-07-18T08:36:03.234736+0000","172.21.15.44:6834/3288748832":"2024-07-18T08:36:09.358821+0000","172.21.15.175:0/1799654570":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/2557831081":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/749128801":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/485731160":"2024-07-18T08:36:03.234736+0000","172.21.15.44:6833/3288748832":"2024-07-18T08:36:09.358821+0000","172.21.15.175:0/363211359":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/3582055205":"2024-07-18T08:36:03.234736+0000","172.21.15.175:0/1458277022":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1949400205":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6837/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/3512613843":"2024-07-18T08:35:51.813306+0000","172.21.15.175:6834/2798563150":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/101784654":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/4286139695":"2024-07-18T08:35:51.813306+0000","172.21.15.175:0/1591261735":"2024-07-18T08:35:51.813306+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:13.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.926+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 msgr2=0x7f16440792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.926+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 0x7f16440792b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f166415b950 tx=0x7f1650005740 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.926+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 msgr2=0x7f166415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.926+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f165400c430 tx=0x7f165400c8f0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 shutdown_connections 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f1644076df0 0x7f16440792b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f166414f2d0 0x7f166414f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16641f3b80 0x7f166415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 --2- 172.21.15.44:0/2573624123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16480051c0 0x7f166415d360 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 >> 172.21.15.44:0/2573624123 conn(0x7f16640b0fb0 msgr2=0x7f16640af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 shutdown_connections 2024-07-17T08:36:13.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:13.927+0000 7f166aa00640 1 -- 172.21.15.44:0/2573624123 wait complete. 2024-07-17T08:36:13.948 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:36:13.948 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:36:14.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 -- 172.21.15.44:0/857216854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 msgr2=0x7f404415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 --2- 172.21.15.44:0/857216854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f404415d830 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f4034005fd0 tx=0x7f403402a9f0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 -- 172.21.15.44:0/857216854 shutdown_connections 2024-07-17T08:36:14.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 --2- 172.21.15.44:0/857216854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f404406f4f0 0x7f40440ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 --2- 172.21.15.44:0/857216854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f404415d830 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 -- 172.21.15.44:0/857216854 >> 172.21.15.44:0/857216854 conn(0x7f40440cf0d0 msgr2=0x7f40440cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:14.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 -- 172.21.15.44:0/857216854 shutdown_connections 2024-07-17T08:36:14.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.165+0000 7f4043200640 1 -- 172.21.15.44:0/857216854 wait complete. 2024-07-17T08:36:14.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.168+0000 7f4043200640 1 Processor -- start 2024-07-17T08:36:14.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.168+0000 7f4043200640 1 -- start start 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.169+0000 7f4043200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f404406f4f0 0x7f404414a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.169+0000 7f4043200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.169+0000 7f4043200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f404415dd70 0x7f4044073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4043200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40440b2220 con 0x7f4044154490 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4043200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40440b20a0 con 0x7f404406f4f0 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4043200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40440b23a0 con 0x7f404415dd70 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52246/0 (socket says 172.21.15.44:52246) 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 -- 172.21.15.44:0/1243840245 learned_addr learned my addr 172.21.15.44:0/1243840245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 -- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f404415dd70 msgr2=0x7f4044073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f404415dd70 0x7f4044073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 -- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f404406f4f0 msgr2=0x7f404414a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041e00640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f404406f4f0 0x7f404414a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f404406f4f0 0x7f404414a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.170+0000 7f4041400640 1 -- 172.21.15.44:0/1243840245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440721b0 con 0x7f4044154490 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f4041400640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b410 tx=0x7f402c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c015020 con 0x7f4044154490 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f402c011920 con 0x7f4044154490 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044072490 con 0x7f4044154490 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c011d70 con 0x7f4044154490 2024-07-17T08:36:14.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.171+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044072a50 con 0x7f4044154490 2024-07-17T08:36:14.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.172+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40440b2990 con 0x7f4044154490 2024-07-17T08:36:14.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.173+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f402c018020 con 0x7f4044154490 2024-07-17T08:36:14.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.174+0000 7f403a000640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 0x7f40200792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.175+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f402c098830 con 0x7f4044154490 2024-07-17T08:36:14.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.175+0000 7f4041e00640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 0x7f40200792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.176+0000 7f4041e00640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 0x7f40200792b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4034035000 tx=0x7f4034004ab0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.179+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f402c01f030 con 0x7f4044154490 2024-07-17T08:36:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.394+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4044152790 con 0x7f4044154490 2024-07-17T08:36:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.395+0000 7f403a000640 1 -- 172.21.15.44:0/1243840245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v235) ==== 93+0+10 (secure 0 0 0) 0x7f402c061540 con 0x7f4044154490 2024-07-17T08:36:14.396 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:36:14.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.397+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 msgr2=0x7f40200792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.397+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 0x7f40200792b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4034035000 tx=0x7f4034004ab0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.397+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 msgr2=0x7f4044073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.397+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b410 tx=0x7f402c00b8d0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 shutdown_connections 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f4020076df0 0x7f40200792b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f404415dd70 0x7f4044073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4044154490 0x7f4044073230 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 --2- 172.21.15.44:0/1243840245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f404406f4f0 0x7f404414a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 >> 172.21.15.44:0/1243840245 conn(0x7f40440cf0d0 msgr2=0x7f4044152cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 shutdown_connections 2024-07-17T08:36:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.398+0000 7f4043200640 1 -- 172.21.15.44:0/1243840245 wait complete. 2024-07-17T08:36:14.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 -- 172.21.15.44:0/2208642573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 msgr2=0x7f0d700ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 --2- 172.21.15.44:0/2208642573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d700ad790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009900 tx=0x7f0d6002a610 comp rx=0 tx=0).stop 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 -- 172.21.15.44:0/2208642573 shutdown_connections 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 --2- 172.21.15.44:0/2208642573 >> v2:172.21.15.175:3300/0 conn(0x7f0d700ac930 0x7f0d700cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 --2- 172.21.15.44:0/2208642573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d700ad790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 -- 172.21.15.44:0/2208642573 >> 172.21.15.44:0/2208642573 conn(0x7f0d700b0fb0 msgr2=0x7f0d700b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:14.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 -- 172.21.15.44:0/2208642573 shutdown_connections 2024-07-17T08:36:14.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.610+0000 7f0d77600640 1 -- 172.21.15.44:0/2208642573 wait complete. 2024-07-17T08:36:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.612+0000 7f0d77600640 1 Processor -- start 2024-07-17T08:36:14.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.613+0000 7f0d77600640 1 -- start start 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d700adcd0 0x7f0d7014d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d7015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700cdf90 con 0x7f0d700ac930 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700cde10 con 0x7f0d700adcd0 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d77600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700ce110 con 0x7f0d701522c0 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d74c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52258/0 (socket says 172.21.15.44:52258) 2024-07-17T08:36:14.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.614+0000 7f0d74c00640 1 -- 172.21.15.44:0/3642336161 learned_addr learned my addr 172.21.15.44:0/3642336161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d75600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d7015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d700adcd0 0x7f0d7014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 -- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 msgr2=0x7f0d7015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d7015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 -- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d700adcd0 msgr2=0x7f0d7014d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d700adcd0 0x7f0d7014d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 -- 172.21.15.44:0/3642336161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d70154c10 con 0x7f0d700ac930 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d74c00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f0d60006010 tx=0x7f0d600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.615+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6003e030 con 0x7f0d700ac930 2024-07-17T08:36:14.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.616+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701f80b0 con 0x7f0d700ac930 2024-07-17T08:36:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.616+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d60003ae0 con 0x7f0d700ac930 2024-07-17T08:36:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.616+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60035040 con 0x7f0d700ac930 2024-07-17T08:36:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.616+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d70154650 con 0x7f0d700ac930 2024-07-17T08:36:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.617+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d700ce660 con 0x7f0d700ac930 2024-07-17T08:36:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.618+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f0d60041030 con 0x7f0d700ac930 2024-07-17T08:36:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.619+0000 7f0d6cc00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 0x7f0d500792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.619+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f0d600b93c0 con 0x7f0d700ac930 2024-07-17T08:36:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.620+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 0x7f0d500792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:14.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.621+0000 7f0d6fe00640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 0x7f0d500792e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cbef0 tx=0x7f0d5c009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.624+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0d70154c10 con 0x7f0d700ac930 2024-07-17T08:36:14.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:14.920+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) -- 0x7f0d70075550 con 0x7f0d700ac930 2024-07-17T08:36:15.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.494+0000 7f0d6cc00640 1 -- 172.21.15.44:0/3642336161 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3320) ==== 190+0+0 (secure 0 0 0) 0x7f0d60082010 con 0x7f0d700ac930 2024-07-17T08:36:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 msgr2=0x7f0d500792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 0x7f0d500792e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cbef0 tx=0x7f0d5c009040 comp rx=0 tx=0).stop 2024-07-17T08:36:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 msgr2=0x7f0d701f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f0d60006010 tx=0x7f0d600035f0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 shutdown_connections 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0d50076e20 0x7f0d500792e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d701522c0 0x7f0d7015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d700adcd0 0x7f0d7014d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.498+0000 7f0d77600640 1 --2- 172.21.15.44:0/3642336161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d700ac930 0x7f0d701f7b70 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.499+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 >> 172.21.15.44:0/3642336161 conn(0x7f0d700b0fb0 msgr2=0x7f0d700afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.499+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 shutdown_connections 2024-07-17T08:36:15.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.499+0000 7f0d77600640 1 -- 172.21.15.44:0/3642336161 wait complete. 2024-07-17T08:36:15.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:36:15.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 -- 172.21.15.44:0/3050033462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 msgr2=0x7fa6940cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 --2- 172.21.15.44:0/3050033462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa6940cc370 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fa694073a30 tx=0x7fa68c02aa10 comp rx=0 tx=0).stop 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 -- 172.21.15.44:0/3050033462 shutdown_connections 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 --2- 172.21.15.44:0/3050033462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa6940cc370 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 --2- 172.21.15.44:0/3050033462 >> v2:172.21.15.175:3300/0 conn(0x7fa69415c3b0 0x7fa6940cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 --2- 172.21.15.44:0/3050033462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 0x7fa69415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 -- 172.21.15.44:0/3050033462 >> 172.21.15.44:0/3050033462 conn(0x7fa69406bf40 msgr2=0x7fa69406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 -- 172.21.15.44:0/3050033462 shutdown_connections 2024-07-17T08:36:15.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.733+0000 7fa699e00640 1 -- 172.21.15.44:0/3050033462 wait complete. 2024-07-17T08:36:15.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.735+0000 7fa699e00640 1 Processor -- start 2024-07-17T08:36:15.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.735+0000 7fa699e00640 1 -- start start 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 0x7fa694154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa694139f80 0x7fa6940b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415f390 con 0x7fa69415b910 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415f210 con 0x7fa694139f80 2024-07-17T08:36:15.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.736+0000 7fa699e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa69415f510 con 0x7fa6940aea80 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52278/0 (socket says 172.21.15.44:52278) 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 -- 172.21.15.44:0/534577386 learned_addr learned my addr 172.21.15.44:0/534577386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa693400640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa694139f80 0x7fa6940b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692a00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 0x7fa694154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 -- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 msgr2=0x7fa694154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 0x7fa694154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 -- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa694139f80 msgr2=0x7fa6940b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa694139f80 0x7fa6940b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 -- 172.21.15.44:0/534577386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69413ea40 con 0x7fa69415b910 2024-07-17T08:36:15.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.737+0000 7fa692000640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b410 tx=0x7fa68000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.738+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680015020 con 0x7fa69415b910 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.738+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa680011920 con 0x7fa69415b910 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.738+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69413ed20 con 0x7fa69415b910 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.738+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011d70 con 0x7fa69415b910 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.738+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69413f2e0 con 0x7fa69415b910 2024-07-17T08:36:15.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.739+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69413dc20 con 0x7fa69415b910 2024-07-17T08:36:15.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.740+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fa680018020 con 0x7fa69415b910 2024-07-17T08:36:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.741+0000 7fa68b400640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 0x7fa6740792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.741+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7fa680098b20 con 0x7fa69415b910 2024-07-17T08:36:15.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.742+0000 7fa692a00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 0x7fa6740792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:15.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.743+0000 7fa692a00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 0x7fa6740792e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa684009890 tx=0x7fa684008040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:15.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.746+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa69413dc20 con 0x7fa69415b910 2024-07-17T08:36:15.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.960+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa69415fa90 con 0x7fa69415b910 2024-07-17T08:36:15.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.961+0000 7fa68b400640 1 -- 172.21.15.44:0/534577386 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v75) ==== 59+0+1998 (secure 0 0 0) 0x7fa68001f020 con 0x7fa69415b910 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:36:15.962 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:36:15.963 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:36:15.964 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:36:15.964 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:36:15.964 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:36:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 msgr2=0x7fa6740792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 0x7fa6740792e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa684009890 tx=0x7fa684008040 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 msgr2=0x7fa694139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b410 tx=0x7fa68000b8d0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 shutdown_connections 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fa674076e20 0x7fa6740792e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa694139f80 0x7fa6940b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69415b910 0x7fa694139a40 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 --2- 172.21.15.44:0/534577386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6940aea80 0x7fa694154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 >> 172.21.15.44:0/534577386 conn(0x7fa69406bf40 msgr2=0x7fa694072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 shutdown_connections 2024-07-17T08:36:15.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:15.964+0000 7fa699e00640 1 -- 172.21.15.44:0/534577386 wait complete. 2024-07-17T08:36:15.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:36:16.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 -- 172.21.15.44:0/1266324747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c0ce690 msgr2=0x7efc1c0cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 --2- 172.21.15.44:0/1266324747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c0ce690 0x7efc1c0cfdf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc14005fd0 tx=0x7efc1402aa10 comp rx=0 tx=0).stop 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 -- 172.21.15.44:0/1266324747 shutdown_connections 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 --2- 172.21.15.44:0/1266324747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c0ce690 0x7efc1c0cfdf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 --2- 172.21.15.44:0/1266324747 >> v2:172.21.15.175:3300/0 conn(0x7efc1c15d130 0x7efc1c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.194+0000 7efc1b200640 1 -- 172.21.15.44:0/1266324747 >> 172.21.15.44:0/1266324747 conn(0x7efc1c0b4ff0 msgr2=0x7efc1c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.195+0000 7efc1b200640 1 -- 172.21.15.44:0/1266324747 shutdown_connections 2024-07-17T08:36:16.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.195+0000 7efc1b200640 1 -- 172.21.15.44:0/1266324747 wait complete. 2024-07-17T08:36:16.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.196+0000 7efc1b200640 1 Processor -- start 2024-07-17T08:36:16.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.197+0000 7efc1b200640 1 -- start start 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1c15d130 0x7efc1c1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c161be0 0x7efc1c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c0d1a60 con 0x7efc1c0ce690 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c0d18e0 con 0x7efc1c15d130 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c0d1be0 con 0x7efc1c161be0 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52292/0 (socket says 172.21.15.44:52292) 2024-07-17T08:36:16.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 -- 172.21.15.44:0/367738723 learned_addr learned my addr 172.21.15.44:0/367738723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19400640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1c15d130 0x7efc1c1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc1a800640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c161be0 0x7efc1c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 -- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c161be0 msgr2=0x7efc1c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c161be0 0x7efc1c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 -- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1c15d130 msgr2=0x7efc1c1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1c15d130 0x7efc1c1f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.198+0000 7efc19e00640 1 -- 172.21.15.44:0/367738723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c1f44d0 con 0x7efc1c0ce690 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc19e00640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ea20 tx=0x7efc0c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00cd80 con 0x7efc1c0ce690 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc0c005020 con 0x7efc1c0ce690 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c005450 con 0x7efc1c0ce690 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c1f47b0 con 0x7efc1c0ce690 2024-07-17T08:36:16.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.199+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c0ca7c0 con 0x7efc1c0ce690 2024-07-17T08:36:16.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.200+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1c0cfdf0 con 0x7efc1c0ce690 2024-07-17T08:36:16.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.202+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7efc0c0047a0 con 0x7efc1c0ce690 2024-07-17T08:36:16.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.203+0000 7efc12000640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 0x7efbf80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:16.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.203+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7efc0c0997c0 con 0x7efc1c0ce690 2024-07-17T08:36:16.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.203+0000 7efc19400640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 0x7efbf80792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.204+0000 7efc19400640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 0x7efbf80792b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc1c1f8350 tx=0x7efc04009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.207+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efc0c062330 con 0x7efc1c0ce690 2024-07-17T08:36:16.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.421+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efc1c0ca990 con 0x7efc1c0ce690 2024-07-17T08:36:16.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.430+0000 7efc12000640 1 -- 172.21.15.44:0/367738723 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7efc0c061c30 con 0x7efc1c0ce690 2024-07-17T08:36:16.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 msgr2=0x7efbf80792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:16.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 0x7efbf80792b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efc1c1f8350 tx=0x7efc04009040 comp rx=0 tx=0).stop 2024-07-17T08:36:16.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 msgr2=0x7efc1c1603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ea20 tx=0x7efc0c00ee20 comp rx=0 tx=0).stop 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 shutdown_connections 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7efbf8076df0 0x7efbf80792b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc1c161be0 0x7efc1c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc1c15d130 0x7efc1c1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 --2- 172.21.15.44:0/367738723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc1c0ce690 0x7efc1c1603b0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.433+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 >> 172.21.15.44:0/367738723 conn(0x7efc1c0b4ff0 msgr2=0x7efc1c0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.434+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 shutdown_connections 2024-07-17T08:36:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:16.434+0000 7efc1b200640 1 -- 172.21.15.44:0/367738723 wait complete. 2024-07-17T08:36:16.453 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:36:16.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:36:16.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:36:16.680 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:36:16.681 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:16.681 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:16.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:36:16.737 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:36:16.737 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:16.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:36:16.791 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:16.791 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:16.849 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:36:16.849 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:36:16.849 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:16.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:36:16.904 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:36:16.904 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 -- 172.21.15.44:0/1371612295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 msgr2=0x7f49b0003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 --2- 172.21.15.44:0/1371612295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49b0003f30 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009ed0 tx=0x7f49b0005830 comp rx=0 tx=0).stop 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 -- 172.21.15.44:0/1371612295 shutdown_connections 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 --2- 172.21.15.44:0/1371612295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49b0003f30 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 -- 172.21.15.44:0/1371612295 >> 172.21.15.44:0/1371612295 conn(0x7f49cc0d00a0 msgr2=0x7f49cc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 -- 172.21.15.44:0/1371612295 shutdown_connections 2024-07-17T08:36:17.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.129+0000 7f49d2400640 1 -- 172.21.15.44:0/1371612295 wait complete. 2024-07-17T08:36:17.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.131+0000 7f49d2400640 1 Processor -- start 2024-07-17T08:36:17.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.131+0000 7f49d2400640 1 -- start start 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49cc0c7880 0x7f49cc0b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc0c7d00 0x7f49cc0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc06e750 con 0x7f49b0003b10 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc06e5d0 con 0x7f49cc0c7d00 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0b5330 con 0x7f49cc0c7880 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52300/0 (socket says 172.21.15.44:52300) 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 -- 172.21.15.44:0/2817750559 learned_addr learned my addr 172.21.15.44:0/2817750559 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 -- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49cc0c7880 msgr2=0x7f49cc0b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49cbe00640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49cc0c7880 0x7f49cc0b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49cc0c7880 0x7f49cc0b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 -- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc0c7d00 msgr2=0x7f49cc0b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc0c7d00 0x7f49cc0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.132+0000 7f49d1000640 1 -- 172.21.15.44:0/2817750559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc1f4d00 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49d1000640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009e20 tx=0x7f49bc033cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc038040 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49bc041070 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc03c670 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1f3f40 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.133+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1f4500 con 0x7f49b0003b10 2024-07-17T08:36:17.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.135+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc160390 con 0x7f49b0003b10 2024-07-17T08:36:17.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.136+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f49bc03c810 con 0x7f49b0003b10 2024-07-17T08:36:17.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.137+0000 7f49c8c00640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 0x7f49a80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.137+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f49bc0b98a0 con 0x7f49b0003b10 2024-07-17T08:36:17.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.137+0000 7f49cbe00640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 0x7f49a80792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.139+0000 7f49cbe00640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 0x7f49a80792b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49b4001eb0 tx=0x7f49b4001c40 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:17.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.141+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f49cc160390 con 0x7f49b0003b10 2024-07-17T08:36:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.392+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49cc0c6af0 con 0x7f49b0003b10 2024-07-17T08:36:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.393+0000 7f49c8c00640 1 -- 172.21.15.44:0/2817750559 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+3915 (secure 0 0 0) 0x7f49cc0c6af0 con 0x7f49b0003b10 2024-07-17T08:36:17.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:17.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":123,"btime":"2024-07-17T08:36:10:436139+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/79388186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":79388186},{"type":"v1","addr":"172.21.15.175:6836","nonce":79388186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109},{"gid":8326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/854486949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":854486949},{"type":"v1","addr":"172.21.15.175:6801","nonce":854486949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108},{"gid":10587,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3444850603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3444850603},{"type":"v1","addr":"172.21.15.44:6835","nonce":3444850603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121},{"gid":10656,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2214529917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2214529917},{"type":"v1","addr":"172.21.15.44:6834","nonce":2214529917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[]} 2024-07-17T08:36:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 123 2024-07-17T08:36:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.396+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 msgr2=0x7f49a80792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.396+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 0x7f49a80792b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49b4001eb0 tx=0x7f49b4001c40 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.396+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 msgr2=0x7f49cc1598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.396+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009e20 tx=0x7f49bc033cb0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 shutdown_connections 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49a8076df0 0x7f49a80792b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc0c7d00 0x7f49cc0b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49cc0c7880 0x7f49cc0b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 --2- 172.21.15.44:0/2817750559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49b0003b10 0x7f49cc1598b0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 >> 172.21.15.44:0/2817750559 conn(0x7f49cc0d00a0 msgr2=0x7f49cc15f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 shutdown_connections 2024-07-17T08:36:17.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.397+0000 7f49d2400640 1 -- 172.21.15.44:0/2817750559 wait complete. 2024-07-17T08:36:17.417 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:36:17.417 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:36:17.418 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:36:17.418 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:36:17.418 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:36:17.418 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:36:17.419 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:36:17.419 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:36:17.419 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:36:17.419 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:36:17.419 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:36:17.419 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:36:17.419 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:36:17.419 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:36:17.420 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:36:17.420 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:36:17.435 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:36:17.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:36:17.437 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:36:17.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:36:17.442 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:36:17.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:36:17.444 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:36:17.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:36:17.484 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:36:17.484 DEBUG:teuthology.parallel:result is None 2024-07-17T08:36:17.490 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:36:17.490 DEBUG:teuthology.parallel:result is None 2024-07-17T08:36:17.491 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:36:17.491 DEBUG:teuthology.parallel:result is None 2024-07-17T08:36:17.497 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:36:17.497 DEBUG:teuthology.parallel:result is None 2024-07-17T08:36:17.498 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:36:17.634 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:36:17.634 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:36:17.643 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:36:17.648 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:36:17.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 -- 172.21.15.44:0/1032955871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c0ae340 msgr2=0x7f962c0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 --2- 172.21.15.44:0/1032955871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c0ae340 0x7f962c0b2eb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cffb0 tx=0x7f961802b270 comp rx=0 tx=0).stop 2024-07-17T08:36:17.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 -- 172.21.15.44:0/1032955871 shutdown_connections 2024-07-17T08:36:17.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 --2- 172.21.15.44:0/1032955871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c0ae340 0x7f962c0b2eb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 --2- 172.21.15.44:0/1032955871 >> v2:172.21.15.44:3300/0 conn(0x7f962c1503c0 0x7f962c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 -- 172.21.15.44:0/1032955871 >> 172.21.15.44:0/1032955871 conn(0x7f962c0cf0d0 msgr2=0x7f962c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:17.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.679+0000 7f9630a00640 1 -- 172.21.15.44:0/1032955871 shutdown_connections 2024-07-17T08:36:17.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.680+0000 7f9630a00640 1 -- 172.21.15.44:0/1032955871 wait complete. 2024-07-17T08:36:17.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.681+0000 7f9630a00640 1 Processor -- start 2024-07-17T08:36:17.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.681+0000 7f9630a00640 1 -- start start 2024-07-17T08:36:17.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f962c0ae340 0x7f962c1f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 0x7f962c0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f962c06f4d0 con 0x7f962c0b33f0 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f962c06dd40 con 0x7f962c1503c0 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.682+0000 7f9630a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f962c06f650 con 0x7f962c0ae340 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52342/0 (socket says 172.21.15.44:52342) 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 -- 172.21.15.44:0/3024833022 learned_addr learned my addr 172.21.15.44:0/3024833022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962b400640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f962c0ae340 0x7f962c1f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962be00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 0x7f962c0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 -- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f962c0ae340 msgr2=0x7f962c1f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f962c0ae340 0x7f962c1f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 -- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 msgr2=0x7f962c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 0x7f962c0713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962aa00640 1 -- 172.21.15.44:0/3024833022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1f79c0 con 0x7f962c0b33f0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.683+0000 7f962be00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 0x7f962c0713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f962aa00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f96180075a0 tx=0x7f96180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618041070 con 0x7f962c0b33f0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9618003db0 con 0x7f962c0b33f0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618005280 con 0x7f962c0b33f0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1f7ca0 con 0x7f962c0b33f0 2024-07-17T08:36:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.684+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1f3ee0 con 0x7f962c0b33f0 2024-07-17T08:36:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.685+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0b2eb0 con 0x7f962c0b33f0 2024-07-17T08:36:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.686+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f9618044070 con 0x7f962c0b33f0 2024-07-17T08:36:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.687+0000 7f9623400640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 0x7f960c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.687+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 6814+0+0 (secure 0 0 0) 0x7f96180b8a00 con 0x7f962c0b33f0 2024-07-17T08:36:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.688+0000 7f962b400640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 0x7f960c0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.688+0000 7f962b400640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 0x7f960c0792b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cf040 tx=0x7f961c005f90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:17.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.691+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9618081690 con 0x7f962c0b33f0 2024-07-17T08:36:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:17.911+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f962c0cffe0 con 0x7f962c0b33f0 2024-07-17T08:36:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.537+0000 7f9623400640 1 -- 172.21.15.44:0/3024833022 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v237) ==== 97+0+0 (secure 0 0 0) 0x7f9618080f90 con 0x7f962c0b33f0 2024-07-17T08:36:18.538 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:36:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 msgr2=0x7f960c0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 0x7f960c0792b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cf040 tx=0x7f961c005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 msgr2=0x7f962c14f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f96180075a0 tx=0x7f96180038a0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 shutdown_connections 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f960c076df0 0x7f960c0792b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f962c1503c0 0x7f962c0713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f962c0b33f0 0x7f962c14f670 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 --2- 172.21.15.44:0/3024833022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f962c0ae340 0x7f962c1f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.542+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 >> 172.21.15.44:0/3024833022 conn(0x7f962c0cf0d0 msgr2=0x7f962c15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.543+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 shutdown_connections 2024-07-17T08:36:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.543+0000 7f9630a00640 1 -- 172.21.15.44:0/3024833022 wait complete. 2024-07-17T08:36:18.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.777+0000 7f942d400640 1 -- 172.21.15.44:0/3991413328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f942806ea50 msgr2=0x7f94280ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.777+0000 7f942d400640 1 --2- 172.21.15.44:0/3991413328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f942806ea50 0x7f94280ac220 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f9410005fd0 tx=0x7f941002a4f0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 -- 172.21.15.44:0/3991413328 shutdown_connections 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 --2- 172.21.15.44:0/3991413328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f942806ea50 0x7f94280ac220 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 --2- 172.21.15.44:0/3991413328 >> v2:172.21.15.175:3300/0 conn(0x7f94280ae810 0x7f942815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 -- 172.21.15.44:0/3991413328 >> 172.21.15.44:0/3991413328 conn(0x7f94280b0fb0 msgr2=0x7f94280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 -- 172.21.15.44:0/3991413328 shutdown_connections 2024-07-17T08:36:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.778+0000 7f942d400640 1 -- 172.21.15.44:0/3991413328 wait complete. 2024-07-17T08:36:18.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.779+0000 7f942d400640 1 Processor -- start 2024-07-17T08:36:18.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.780+0000 7f942d400640 1 -- start start 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.780+0000 7f942d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942806ea50 0x7f9428143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.780+0000 7f942d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f942806eec0 0x7f9428143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.780+0000 7f942d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.780+0000 7f942d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94280b2e60 con 0x7f94280ae810 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f942d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94280b2ce0 con 0x7f942806ea50 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f942d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94280b2fe0 con 0x7f942806eec0 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52358/0 (socket says 172.21.15.44:52358) 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 -- 172.21.15.44:0/2723509303 learned_addr learned my addr 172.21.15.44:0/2723509303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 -- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f942806eec0 msgr2=0x7f9428143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:18.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f942806eec0 0x7f9428143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 -- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942806ea50 msgr2=0x7f9428143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:18.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942806ea50 0x7f9428143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:18.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 -- 172.21.15.44:0/2723509303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280735b0 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.781+0000 7f9427200640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f942000c7b0 tx=0x7f942000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.782+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420017070 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.782+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9420005070 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.782+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000ce60 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.782+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9428073890 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.782+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9428073e50 con 0x7f94280ae810 2024-07-17T08:36:18.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.783+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94280ac920 con 0x7f94280ae810 2024-07-17T08:36:18.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.785+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f94200054c0 con 0x7f94280ae810 2024-07-17T08:36:18.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.786+0000 7f941f400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 0x7f94040792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:18.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.786+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 5825+0+0 (secure 0 0 0) 0x7f9420097de0 con 0x7f94280ae810 2024-07-17T08:36:18.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.787+0000 7f9426800640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 0x7f94040792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:18.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.788+0000 7f9426800640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 0x7f94040792b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9418009670 tx=0x7f9418006bf0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:18.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:18.790+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f94200057b0 con 0x7f94280ae810 2024-07-17T08:36:19.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.076+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f94280b23b0 con 0x7f94280ae810 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.077+0000 7f941f400640 1 -- 172.21.15.44:0/2723509303 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v72) ==== 80+0+5304 (secure 0 0 0) 0x7f9420060da0 con 0x7f94280ae810 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:36:19.078 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.079 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:36:19.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.082 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:36:19.083 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:36:19.083 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.083 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.083 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.083 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.084 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.084 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:36:19.084 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.086 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.087 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:36:19.088 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:36:19.088 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.088 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.088 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.089 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.090 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.091 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:36:19.091 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:36:19.091 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:36:19.093 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:36:19.094 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:36:19.095 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.096 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.097 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:36:19.098 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:36:19.099 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:36:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 msgr2=0x7f94040792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 0x7f94040792b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9418009670 tx=0x7f9418006bf0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 msgr2=0x7f9428071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f942000c7b0 tx=0x7f942000cc70 comp rx=0 tx=0).stop 2024-07-17T08:36:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 shutdown_connections 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9404076df0 0x7f94040792b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94280ae810 0x7f9428071620 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f942806eec0 0x7f9428143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 --2- 172.21.15.44:0/2723509303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f942806ea50 0x7f9428143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.080+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 >> 172.21.15.44:0/2723509303 conn(0x7f94280b0fb0 msgr2=0x7f94280b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.081+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 shutdown_connections 2024-07-17T08:36:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.081+0000 7f942d400640 1 -- 172.21.15.44:0/2723509303 wait complete. 2024-07-17T08:36:19.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:36:19.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 -- 172.21.15.44:0/2388207490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76740ac3e0 msgr2=0x7f76740ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 --2- 172.21.15.44:0/2388207490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76740ac3e0 0x7f76740ac810 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7674070c50 tx=0x7f766c02aa30 comp rx=0 tx=0).stop 2024-07-17T08:36:19.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 -- 172.21.15.44:0/2388207490 shutdown_connections 2024-07-17T08:36:19.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 --2- 172.21.15.44:0/2388207490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7674159b00 0x7f7674159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 --2- 172.21.15.44:0/2388207490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76740ac3e0 0x7f76740ac810 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 --2- 172.21.15.44:0/2388207490 >> v2:172.21.15.175:3300/0 conn(0x7f767406f710 0x7f7674153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 -- 172.21.15.44:0/2388207490 >> 172.21.15.44:0/2388207490 conn(0x7f76740b0fb0 msgr2=0x7f76740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:19.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 -- 172.21.15.44:0/2388207490 shutdown_connections 2024-07-17T08:36:19.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.311+0000 7f7679000640 1 -- 172.21.15.44:0/2388207490 wait complete. 2024-07-17T08:36:19.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.314+0000 7f7679000640 1 Processor -- start 2024-07-17T08:36:19.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.314+0000 7f7679000640 1 -- start start 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76740ac3e0 0x7f767414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7674159b00 0x7f767414f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76740ae980 con 0x7f767406f710 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76740ae800 con 0x7f7674159b00 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.315+0000 7f7679000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f767413ed60 con 0x7f76740ac3e0 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33806/0 (socket says 172.21.15.44:33806) 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 -- 172.21.15.44:0/1027139078 learned_addr learned my addr 172.21.15.44:0/1027139078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671400640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76740ac3e0 0x7f767414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7672800640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7674159b00 0x7f767414f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 -- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76740ac3e0 msgr2=0x7f767414eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76740ac3e0 0x7f767414eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 -- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7674159b00 msgr2=0x7f767414f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7674159b00 0x7f767414f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.316+0000 7f7671e00640 1 -- 172.21.15.44:0/1027139078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7674153020 con 0x7f767406f710 2024-07-17T08:36:19.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f7671e00640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:19.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664017070 con 0x7f767406f710 2024-07-17T08:36:19.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7664009d20 con 0x7f767406f710 2024-07-17T08:36:19.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7674152a60 con 0x7f767406f710 2024-07-17T08:36:19.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640106c0 con 0x7f767406f710 2024-07-17T08:36:19.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.317+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7674151af0 con 0x7f767406f710 2024-07-17T08:36:19.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.318+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767406a1a0 con 0x7f767406f710 2024-07-17T08:36:19.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.319+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f7664010860 con 0x7f767406f710 2024-07-17T08:36:19.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.320+0000 7f766a000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 0x7f76540792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.321+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 5825+0+0 (secure 0 0 0) 0x7f7664098c80 con 0x7f767406f710 2024-07-17T08:36:19.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.321+0000 7f7671400640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 0x7f76540792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.322+0000 7f7671400640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 0x7f76540792e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7660005fd0 tx=0x7f766000d4b0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:19.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.325+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f767406a1a0 con 0x7f767406f710 2024-07-17T08:36:19.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.540+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f76740ac810 con 0x7f767406f710 2024-07-17T08:36:19.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.540+0000 7f766a000640 1 -- 172.21.15.44:0/1027139078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v239) ==== 84+0+65 (secure 0 0 0) 0x7f7664061bc0 con 0x7f767406f710 2024-07-17T08:36:19.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:19.541 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:36:19.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:19.542 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:36:19.542 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:36:19.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:19.542 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:36:19.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 msgr2=0x7f76540792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 0x7f76540792e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7660005fd0 tx=0x7f766000d4b0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 msgr2=0x7f76740afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400de10 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 shutdown_connections 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7654076e20 0x7f76540792e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7674159b00 0x7f767414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76740ac3e0 0x7f767414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 --2- 172.21.15.44:0/1027139078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f767406f710 0x7f76740afa00 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.543+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 >> 172.21.15.44:0/1027139078 conn(0x7f76740b0fb0 msgr2=0x7f7674158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.544+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 shutdown_connections 2024-07-17T08:36:19.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.544+0000 7f7679000640 1 -- 172.21.15.44:0/1027139078 wait complete. 2024-07-17T08:36:19.563 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:36:19.563 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 -- 172.21.15.44:0/1329098958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc004e60 msgr2=0x7f5fcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 --2- 172.21.15.44:0/1329098958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc004e60 0x7f5fcc00b9c0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c590 tx=0x7f5fcc005830 comp rx=0 tx=0).stop 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fecc00640 1 -- 172.21.15.44:0/1329098958 reap_dead start 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 -- 172.21.15.44:0/1329098958 shutdown_connections 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 -- 172.21.15.44:0/1329098958 >> 172.21.15.44:0/1329098958 conn(0x7f5fe80d0100 msgr2=0x7f5fe80d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:19.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 -- 172.21.15.44:0/1329098958 shutdown_connections 2024-07-17T08:36:19.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.779+0000 7f5fed600640 1 -- 172.21.15.44:0/1329098958 wait complete. 2024-07-17T08:36:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.781+0000 7f5fed600640 1 Processor -- start 2024-07-17T08:36:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.781+0000 7f5fed600640 1 -- start start 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc004e60 0x7f5fe81f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe814e9a0 0x7f5fe814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81619c0 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80b5850 con 0x7f5fe814e9a0 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.782+0000 7f5fed600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe8161b40 con 0x7f5fcc004e60 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33832/0 (socket says 172.21.15.44:33832) 2024-07-17T08:36:19.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 -- 172.21.15.44:0/2264721456 learned_addr learned my addr 172.21.15.44:0/2264721456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fe7e00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc004e60 0x7f5fe81f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fecc00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe814e9a0 0x7f5fe814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 -- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc004e60 msgr2=0x7f5fe81f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc004e60 0x7f5fe81f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 -- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe814e9a0 msgr2=0x7f5fe814ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe814e9a0 0x7f5fe814ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.783+0000 7f5fdfe00640 1 -- 172.21.15.44:0/2264721456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81f3ac0 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fdfe00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000d400 tx=0x7f5fd000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0014070 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fd000f970 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe80cb960 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000fdc0 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.784+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe80cbf20 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.785+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe80cf6c0 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.786+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f5fd0017070 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.787+0000 7f5fe4c00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 0x7f5fc40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.787+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 5825+0+0 (secure 0 0 0) 0x7f5fd0098b30 con 0x7f5fe80cb3a0 2024-07-17T08:36:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.788+0000 7f5fe7e00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 0x7f5fc40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.789+0000 7f5fe7e00640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 0x7f5fc40792b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fb40 tx=0x7f5fd8005f30 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:19.792+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5fe80cf6c0 con 0x7f5fe80cb3a0 2024-07-17T08:36:20.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.003+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5fe80cdc70 con 0x7f5fe80cb3a0 2024-07-17T08:36:20.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.566+0000 7f5fe4c00640 1 -- 172.21.15.44:0/2264721456 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v240) ==== 138+0+0 (secure 0 0 0) 0x7f5fd0061a70 con 0x7f5fe80cb3a0 2024-07-17T08:36:20.567 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:36:20.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.570+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 msgr2=0x7f5fc40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:20.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.570+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 0x7f5fc40792b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fb40 tx=0x7f5fd8005f30 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.570+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 msgr2=0x7f5fe814e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.570+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000d400 tx=0x7f5fd000d8c0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 shutdown_connections 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5fc4076df0 0x7f5fc40792b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe814e9a0 0x7f5fe814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe80cb3a0 0x7f5fe814e460 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 --2- 172.21.15.44:0/2264721456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc004e60 0x7f5fe81f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 >> 172.21.15.44:0/2264721456 conn(0x7f5fe80d0100 msgr2=0x7f5fe806e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:20.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 shutdown_connections 2024-07-17T08:36:20.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.571+0000 7f5fed600640 1 -- 172.21.15.44:0/2264721456 wait complete. 2024-07-17T08:36:20.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 -- 172.21.15.44:0/1816265119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f006ed30 msgr2=0x7f79f015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 --2- 172.21.15.44:0/1816265119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f006ed30 0x7f79f015c2f0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b43d0 tx=0x7f79dc02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 -- 172.21.15.44:0/1816265119 shutdown_connections 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 --2- 172.21.15.44:0/1816265119 >> v2:172.21.15.175:3300/0 conn(0x7f79f00cb3a0 0x7f79f00cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 --2- 172.21.15.44:0/1816265119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f006ed30 0x7f79f015c2f0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 --2- 172.21.15.44:0/1816265119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 0x7f79f006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 -- 172.21.15.44:0/1816265119 >> 172.21.15.44:0/1816265119 conn(0x7f79f00b0eb0 msgr2=0x7f79f00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:20.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 -- 172.21.15.44:0/1816265119 shutdown_connections 2024-07-17T08:36:20.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.807+0000 7f79f4c00640 1 -- 172.21.15.44:0/1816265119 wait complete. 2024-07-17T08:36:20.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.810+0000 7f79f4c00640 1 Processor -- start 2024-07-17T08:36:20.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.810+0000 7f79f4c00640 1 -- start start 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79f006ed30 0x7f79f01546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 0x7f79f00b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79f015f1f0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79f015f070 con 0x7f79f006ed30 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79f4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79f015f370 con 0x7f79f0152220 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.811+0000 7f79ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33850/0 (socket says 172.21.15.44:33850) 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 -- 172.21.15.44:0/4289115769 learned_addr learned my addr 172.21.15.44:0/4289115769 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:20.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ede00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79f006ed30 0x7f79f01546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ee800640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 0x7f79f00b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 -- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 msgr2=0x7f79f00b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 0x7f79f00b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 -- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79f006ed30 msgr2=0x7f79f01546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79f006ed30 0x7f79f01546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:20.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.812+0000 7f79ed400640 1 -- 172.21.15.44:0/4289115769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f0154fe0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79ed400640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc03f070 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79dc0047a0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc035030 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f01f77e0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.813+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01f7da0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.815+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f00cb830 con 0x7f79f00cb3a0 2024-07-17T08:36:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.817+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f79dc042070 con 0x7f79f00cb3a0 2024-07-17T08:36:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.818+0000 7f79e6000640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 0x7f79d00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.818+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 6195+0+0 (secure 0 0 0) 0x7f79dc0b9be0 con 0x7f79f00cb3a0 2024-07-17T08:36:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.819+0000 7f79ede00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 0x7f79d00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.820+0000 7f79ede00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 0x7f79d00792b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e0005850 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:20.822+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f79dc082b90 con 0x7f79f00cb3a0 2024-07-17T08:36:21.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.035+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f79f01f7f70 con 0x7f79f00cb3a0 2024-07-17T08:36:21.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.584+0000 7f79e6000640 1 -- 172.21.15.44:0/4289115769 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v241) ==== 170+0+0 (secure 0 0 0) 0x7f79dc082490 con 0x7f79f00cb3a0 2024-07-17T08:36:21.585 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:36:21.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.588+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 msgr2=0x7f79d00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.588+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 0x7f79d00792b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79e0005fd0 tx=0x7f79e0005850 comp rx=0 tx=0).stop 2024-07-17T08:36:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.588+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 msgr2=0x7f79f01493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.588+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc003910 comp rx=0 tx=0).stop 2024-07-17T08:36:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 shutdown_connections 2024-07-17T08:36:21.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f79d0076df0 0x7f79d00792b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79f0152220 0x7f79f00b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79f00cb3a0 0x7f79f01493b0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 --2- 172.21.15.44:0/4289115769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79f006ed30 0x7f79f01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 >> 172.21.15.44:0/4289115769 conn(0x7f79f00b0eb0 msgr2=0x7f79f0068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 shutdown_connections 2024-07-17T08:36:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.589+0000 7f79f4c00640 1 -- 172.21.15.44:0/4289115769 wait complete. 2024-07-17T08:36:21.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:36:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 -- 172.21.15.44:0/3828898550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b8152220 msgr2=0x7f22b806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 --2- 172.21.15.44:0/3828898550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b8152220 0x7f22b806d620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009830 tx=0x7f22a802a480 comp rx=0 tx=0).stop 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 -- 172.21.15.44:0/3828898550 shutdown_connections 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 --2- 172.21.15.44:0/3828898550 >> v2:172.21.15.175:3300/0 conn(0x7f22b80cbab0 0x7f22b815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 --2- 172.21.15.44:0/3828898550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b8152220 0x7f22b806d620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 -- 172.21.15.44:0/3828898550 >> 172.21.15.44:0/3828898550 conn(0x7f22b80b0eb0 msgr2=0x7f22b80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 -- 172.21.15.44:0/3828898550 shutdown_connections 2024-07-17T08:36:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.807+0000 7f22bdc00640 1 -- 172.21.15.44:0/3828898550 wait complete. 2024-07-17T08:36:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.809+0000 7f22bdc00640 1 Processor -- start 2024-07-17T08:36:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.809+0000 7f22bdc00640 1 -- start start 2024-07-17T08:36:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.810+0000 7f22bdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b806ed30 0x7f22b81f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.810+0000 7f22bdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 0x7f22b8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.810+0000 7f22bdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.810+0000 7f22bdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22b80cd780 con 0x7f22b8152220 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22bdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22b80cd600 con 0x7f22b80cbab0 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22bdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22b80cd900 con 0x7f22b806ed30 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22b7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33864/0 (socket says 172.21.15.44:33864) 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22b7400640 1 -- 172.21.15.44:0/4227949312 learned_addr learned my addr 172.21.15.44:0/4227949312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22b6000640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 0x7f22b8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.811+0000 7f22b6a00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b806ed30 0x7f22b81f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 -- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b806ed30 msgr2=0x7f22b81f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b806ed30 0x7f22b81f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 -- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 msgr2=0x7f22b8073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 0x7f22b8073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 -- 172.21.15.44:0/4227949312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b8073c00 con 0x7f22b8152220 2024-07-17T08:36:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b6000640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 0x7f22b8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:36:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22b7400640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009fa0 tx=0x7f22b000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004b20 con 0x7f22b8152220 2024-07-17T08:36:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22b0004db0 con 0x7f22b8152220 2024-07-17T08:36:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0003bd0 con 0x7f22b8152220 2024-07-17T08:36:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b8073ee0 con 0x7f22b8152220 2024-07-17T08:36:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.812+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b815cd20 con 0x7f22b8152220 2024-07-17T08:36:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.814+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b80cde20 con 0x7f22b8152220 2024-07-17T08:36:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.816+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f22b001d020 con 0x7f22b8152220 2024-07-17T08:36:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.818+0000 7f22aea00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 0x7f22940792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.818+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 6577+0+0 (secure 0 0 0) 0x7f22b0099a50 con 0x7f22b8152220 2024-07-17T08:36:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.819+0000 7f22b6a00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 0x7f22940792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:21.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.820+0000 7f22b6a00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 0x7f22940792e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22a8034000 tx=0x7f22a8004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:21.821+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f22b0024070 con 0x7f22b8152220 2024-07-17T08:36:22.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.074+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f22b80afb30 con 0x7f22b8152220 2024-07-17T08:36:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.612+0000 7f22aea00640 1 -- 172.21.15.44:0/4227949312 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v126) ==== 174+0+0 (secure 0 0 0) 0x7f22b0062720 con 0x7f22b8152220 2024-07-17T08:36:22.617 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 31 and data pool 32 2024-07-17T08:36:22.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.622+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 msgr2=0x7f22940792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:22.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.622+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 0x7f22940792e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22a8034000 tx=0x7f22a8004730 comp rx=0 tx=0).stop 2024-07-17T08:36:22.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.622+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 msgr2=0x7f22b8073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:22.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.622+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009fa0 tx=0x7f22b000c800 comp rx=0 tx=0).stop 2024-07-17T08:36:22.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 shutdown_connections 2024-07-17T08:36:22.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2294076e20 0x7f22940792e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22b8152220 0x7f22b8073580 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b80cbab0 0x7f22b8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 --2- 172.21.15.44:0/4227949312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b806ed30 0x7f22b81f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.625+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 >> 172.21.15.44:0/4227949312 conn(0x7f22b80b0eb0 msgr2=0x7f22b80b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.627+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 shutdown_connections 2024-07-17T08:36:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.628+0000 7f22bdc00640 1 -- 172.21.15.44:0/4227949312 wait complete. 2024-07-17T08:36:22.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 -- 172.21.15.44:0/1545534701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76f4003a50 msgr2=0x7f76f4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 --2- 172.21.15.44:0/1545534701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76f4003a50 0x7f76f4003e70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f770000c290 tx=0x7f76f4005830 comp rx=0 tx=0).stop 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f770e800640 1 -- 172.21.15.44:0/1545534701 reap_dead start 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 -- 172.21.15.44:0/1545534701 shutdown_connections 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 -- 172.21.15.44:0/1545534701 >> 172.21.15.44:0/1545534701 conn(0x7f77100d00a0 msgr2=0x7f77100d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 -- 172.21.15.44:0/1545534701 shutdown_connections 2024-07-17T08:36:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.840+0000 7f7714c00640 1 -- 172.21.15.44:0/1545534701 wait complete. 2024-07-17T08:36:22.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.842+0000 7f7714c00640 1 Processor -- start 2024-07-17T08:36:22.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.842+0000 7f7714c00640 1 -- start start 2024-07-17T08:36:22.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f4003a50 0x7f771015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:22.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:22.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 0x7f77101561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:22.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77100ce230 con 0x7f771014a7d0 2024-07-17T08:36:22.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77100b54c0 con 0x7f7710155d40 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f7714c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77100ce3b0 con 0x7f76f4003a50 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f770d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f770d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33886/0 (socket says 172.21.15.44:33886) 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f770d400640 1 -- 172.21.15.44:0/288267120 learned_addr learned my addr 172.21.15.44:0/288267120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.843+0000 7f770de00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f4003a50 0x7f771015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770e800640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 0x7f77101561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 -- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f4003a50 msgr2=0x7f771015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f4003a50 0x7f771015c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 -- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 msgr2=0x7f77101561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 0x7f77101561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 -- 172.21.15.44:0/288267120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77100cc160 con 0x7f771014a7d0 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770e800640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 0x7f77101561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f770d400640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f76fc010a10 tx=0x7f76fc010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc00ece0 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76fc00ee80 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc005280 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.844+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77100cc3d0 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.845+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77100cc990 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.846+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77100cdcf0 con 0x7f771014a7d0 2024-07-17T08:36:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.847+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f76fc005420 con 0x7f771014a7d0 2024-07-17T08:36:22.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.848+0000 7f7706000640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 0x7f76f00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:22.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.848+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(242..242 src has 1..242) ==== 6677+0+0 (secure 0 0 0) 0x7f76fc0999b0 con 0x7f771014a7d0 2024-07-17T08:36:22.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.849+0000 7f770de00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 0x7f76f00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:22.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.850+0000 7f770de00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 0x7f76f00792b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7700033000 tx=0x7f77100d13d0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:22.853+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f76fc0625a0 con 0x7f771014a7d0 2024-07-17T08:36:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.063+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77100cf3f0 con 0x7f771014a7d0 2024-07-17T08:36:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.066+0000 7f7706000640 1 -- 172.21.15.44:0/288267120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) ==== 74+0+17830 (secure 0 0 0) 0x7f76fc061ea0 con 0x7f771014a7d0 2024-07-17T08:36:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:23.068 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":242,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:22.075429+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.068+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 msgr2=0x7f76f00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.068+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 0x7f76f00792b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7700033000 tx=0x7f77100d13d0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.068+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 msgr2=0x7f771014ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.068+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f76fc010a10 tx=0x7f76fc010ed0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 shutdown_connections 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f76f0076df0 0x7f76f00792b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7710155d40 0x7f77101561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f771014a7d0 0x7f771014ac50 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 --2- 172.21.15.44:0/288267120 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f4003a50 0x7f771015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 >> 172.21.15.44:0/288267120 conn(0x7f77100d00a0 msgr2=0x7f77100b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 shutdown_connections 2024-07-17T08:36:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.069+0000 7f7714c00640 1 -- 172.21.15.44:0/288267120 wait complete. 2024-07-17T08:36:23.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 -- 172.21.15.44:0/175290639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41448b0 msgr2=0x7f3fd40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 --2- 172.21.15.44:0/175290639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41448b0 0x7f3fd40cd8e0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40cea70 tx=0x7f3fbc02a580 comp rx=0 tx=0).stop 2024-07-17T08:36:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 -- 172.21.15.44:0/175290639 shutdown_connections 2024-07-17T08:36:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 --2- 172.21.15.44:0/175290639 >> v2:172.21.15.175:3300/0 conn(0x7f3fd415eff0 0x7f3fd406f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 --2- 172.21.15.44:0/175290639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41448b0 0x7f3fd40cd8e0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 -- 172.21.15.44:0/175290639 >> 172.21.15.44:0/175290639 conn(0x7f3fd4072730 msgr2=0x7f3fd4072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 -- 172.21.15.44:0/175290639 shutdown_connections 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.312+0000 7f3fd9800640 1 -- 172.21.15.44:0/175290639 wait complete. 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.314+0000 7f3fd9800640 1 Processor -- start 2024-07-17T08:36:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.314+0000 7f3fd9800640 1 -- start start 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd41448b0 0x7f3fd40ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd415eff0 0x7f3fd40abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40700d0 con 0x7f3fd41444d0 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd406ff50 con 0x7f3fd41448b0 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd40b5850 con 0x7f3fd415eff0 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33900/0 (socket says 172.21.15.44:33900) 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd2a00640 1 -- 172.21.15.44:0/8080843 learned_addr learned my addr 172.21.15.44:0/8080843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fcbe00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd41448b0 0x7f3fd40ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd2a00640 1 -- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd415eff0 msgr2=0x7f3fd40abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.315+0000 7f3fd2a00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd415eff0 0x7f3fd40abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd2a00640 1 -- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd41448b0 msgr2=0x7f3fd40ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd2a00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd41448b0 0x7f3fd40ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd2a00640 1 -- 172.21.15.44:0/8080843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd4135af0 con 0x7f3fd41444d0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd2a00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400c3a0 tx=0x7f3fc400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4017070 con 0x7f3fd41444d0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fc40053c0 con 0x7f3fd41444d0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4003850 con 0x7f3fd41444d0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd4135dd0 con 0x7f3fd41444d0 2024-07-17T08:36:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.316+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd41f76c0 con 0x7f3fd41444d0 2024-07-17T08:36:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.317+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd40cd8e0 con 0x7f3fd41444d0 2024-07-17T08:36:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.319+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f3fc401a070 con 0x7f3fd41444d0 2024-07-17T08:36:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.320+0000 7f3fcaa00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 0x7f3fb00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.320+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(242..242 src has 1..242) ==== 6677+0+0 (secure 0 0 0) 0x7f3fc4097e30 con 0x7f3fd41444d0 2024-07-17T08:36:23.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.320+0000 7f3fcbe00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 0x7f3fb00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.321+0000 7f3fcbe00640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 0x7f3fb00792b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc033000 tx=0x7f3fbc004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:23.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.324+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3fc4060c90 con 0x7f3fd41444d0 2024-07-17T08:36:23.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.537+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd40ccfd0 con 0x7f3fd41444d0 2024-07-17T08:36:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.539+0000 7f3fcaa00640 1 -- 172.21.15.44:0/8080843 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) ==== 74+0+17830 (secure 0 0 0) 0x7f3fc4060590 con 0x7f3fd41444d0 2024-07-17T08:36:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:23.542 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":242,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:22.075429+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 msgr2=0x7f3fb00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 0x7f3fb00792b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc033000 tx=0x7f3fbc004730 comp rx=0 tx=0).stop 2024-07-17T08:36:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 msgr2=0x7f3fd41f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400c3a0 tx=0x7f3fc400c860 comp rx=0 tx=0).stop 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 shutdown_connections 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3fb0076df0 0x7f3fb00792b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fd415eff0 0x7f3fd40abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fd41448b0 0x7f3fd40ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 --2- 172.21.15.44:0/8080843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fd41444d0 0x7f3fd41f4960 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.542+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 >> 172.21.15.44:0/8080843 conn(0x7f3fd4072730 msgr2=0x7f3fd40b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.543+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 shutdown_connections 2024-07-17T08:36:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.543+0000 7f3fd9800640 1 -- 172.21.15.44:0/8080843 wait complete. 2024-07-17T08:36:23.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:36:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 -- 172.21.15.44:0/2268173748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374006f4d0 msgr2=0x7f374015b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 --2- 172.21.15.44:0/2268173748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374006f4d0 0x7f374015b040 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f37400b1e60 tx=0x7f373802a820 comp rx=0 tx=0).stop 2024-07-17T08:36:23.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 -- 172.21.15.44:0/2268173748 shutdown_connections 2024-07-17T08:36:23.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 --2- 172.21.15.44:0/2268173748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374006f4d0 0x7f374015b040 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 --2- 172.21.15.44:0/2268173748 >> v2:172.21.15.175:3300/0 conn(0x7f3740150ed0 0x7f37400ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 -- 172.21.15.44:0/2268173748 >> 172.21.15.44:0/2268173748 conn(0x7f37400cf0d0 msgr2=0x7f37400cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:23.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 -- 172.21.15.44:0/2268173748 shutdown_connections 2024-07-17T08:36:23.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.768+0000 7f3746800640 1 -- 172.21.15.44:0/2268173748 wait complete. 2024-07-17T08:36:23.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.770+0000 7f3746800640 1 Processor -- start 2024-07-17T08:36:23.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.770+0000 7f3746800640 1 -- start start 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 0x7f37401f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740150ed0 0x7f374015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37400b2680 con 0x7f37401512b0 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37400b2500 con 0x7f374006f4d0 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.771+0000 7f3746800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37400b2800 con 0x7f3740150ed0 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33912/0 (socket says 172.21.15.44:33912) 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 -- 172.21.15.44:0/2931515557 learned_addr learned my addr 172.21.15.44:0/2931515557 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745400640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 0x7f37401f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 -- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740150ed0 msgr2=0x7f374015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3744a00640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740150ed0 0x7f374015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740150ed0 0x7f374015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 -- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 msgr2=0x7f37401f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 0x7f37401f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 -- 172.21.15.44:0/2931515557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401f7200 con 0x7f37401512b0 2024-07-17T08:36:23.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745400640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 0x7f37401f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.772+0000 7f3745e00640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f3738009ca0 tx=0x7f3738032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.773+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738037040 con 0x7f37401512b0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.773+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3738042070 con 0x7f37401512b0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.773+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37400cc4a0 con 0x7f37401512b0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.773+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380042a0 con 0x7f37401512b0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.773+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37400cc780 con 0x7f37401512b0 2024-07-17T08:36:23.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.774+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374015b040 con 0x7f37401512b0 2024-07-17T08:36:23.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.775+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f3738004440 con 0x7f37401512b0 2024-07-17T08:36:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.776+0000 7f373d600640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 0x7f37200792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.776+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f37380b98f0 con 0x7f37401512b0 2024-07-17T08:36:23.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.777+0000 7f3745400640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 0x7f37200792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:23.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.778+0000 7f3745400640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 0x7f37200792e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3734009b40 tx=0x7f3734005e50 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:23.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:23.780+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3738082570 con 0x7f37401512b0 2024-07-17T08:36:24.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.029+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f374006d520 con 0x7f37401512b0 2024-07-17T08:36:24.549 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-07-17T08:36:24.547+0000 7f3a45e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:36:24.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.623+0000 7f373d600640 1 -- 172.21.15.44:0/2931515557 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v129) ==== 118+0+0 (secure 0 0 0) 0x7f3738081e70 con 0x7f37401512b0 2024-07-17T08:36:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 msgr2=0x7f37200792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 0x7f37200792e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3734009b40 tx=0x7f3734005e50 comp rx=0 tx=0).stop 2024-07-17T08:36:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 msgr2=0x7f3740153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:24.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f3738009ca0 tx=0x7f3738032d10 comp rx=0 tx=0).stop 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 shutdown_connections 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f3720076e20 0x7f37200792e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37401512b0 0x7f3740153ed0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3740150ed0 0x7f374015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 --2- 172.21.15.44:0/2931515557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f374006f4d0 0x7f37401f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.627+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 >> 172.21.15.44:0/2931515557 conn(0x7f37400cf0d0 msgr2=0x7f374015bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:24.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.628+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 shutdown_connections 2024-07-17T08:36:24.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.628+0000 7f3746800640 1 -- 172.21.15.44:0/2931515557 wait complete. 2024-07-17T08:36:24.651 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:36:24.651 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:36:24.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 -- 172.21.15.44:0/2866611781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0b6790 msgr2=0x7fe22c161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 --2- 172.21.15.44:0/2866611781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0b6790 0x7fe22c161650 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fe22c075790 tx=0x7fe21402a5f0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 -- 172.21.15.44:0/2866611781 shutdown_connections 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 --2- 172.21.15.44:0/2866611781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b29e0 0x7fe22c161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 --2- 172.21.15.44:0/2866611781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0b6790 0x7fe22c161650 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 --2- 172.21.15.44:0/2866611781 >> v2:172.21.15.175:3300/0 conn(0x7fe22c0ce1d0 0x7fe22c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 -- 172.21.15.44:0/2866611781 >> 172.21.15.44:0/2866611781 conn(0x7fe22c0d3700 msgr2=0x7fe22c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 -- 172.21.15.44:0/2866611781 shutdown_connections 2024-07-17T08:36:24.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.853+0000 7fe230c00640 1 -- 172.21.15.44:0/2866611781 wait complete. 2024-07-17T08:36:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.855+0000 7fe230c00640 1 Processor -- start 2024-07-17T08:36:24.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.855+0000 7fe230c00640 1 -- start start 2024-07-17T08:36:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22c0b29e0 0x7fe22c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 0x7fe22c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:24.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c162250 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1620d0 con 0x7fe22c0b29e0 2024-07-17T08:36:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.856+0000 7fe230c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c1623d0 con 0x7fe22c0b6790 2024-07-17T08:36:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33930/0 (socket says 172.21.15.44:33930) 2024-07-17T08:36:24.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 -- 172.21.15.44:0/2841806275 learned_addr learned my addr 172.21.15.44:0/2841806275 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe229e00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22c0b29e0 0x7fe22c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe229400640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 0x7fe22c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 -- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 msgr2=0x7fe22c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 0x7fe22c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 -- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22c0b29e0 msgr2=0x7fe22c156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22c0b29e0 0x7fe22c156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe22a800640 1 -- 172.21.15.44:0/2841806275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c155490 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.857+0000 7fe229400640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 0x7fe22c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe22a800640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fe22400ca70 tx=0x7fe22400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22400adc0 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe224004c10 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c0c9af0 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224005280 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.858+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c158080 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.859+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c0ce5b0 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.860+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fe224005420 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.862+0000 7fe222000640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 0x7fe2080792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:24.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.862+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fe224099330 con 0x7fe22c0ce1d0 2024-07-17T08:36:24.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.862+0000 7fe229e00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 0x7fe2080792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:24.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.864+0000 7fe229e00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 0x7fe2080792b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005d70 tx=0x7fe21c005d00 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:24.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:24.866+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe224014030 con 0x7fe22c0ce1d0 2024-07-17T08:36:25.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.117+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe22c0b3060 con 0x7fe22c0ce1d0 2024-07-17T08:36:25.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.635+0000 7fe222000640 1 -- 172.21.15.44:0/2841806275 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v130) ==== 122+0+0 (secure 0 0 0) 0x7fe224061fb0 con 0x7fe22c0ce1d0 2024-07-17T08:36:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 msgr2=0x7fe2080792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 0x7fe2080792b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005d70 tx=0x7fe21c005d00 comp rx=0 tx=0).stop 2024-07-17T08:36:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 msgr2=0x7fe22c157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fe22400ca70 tx=0x7fe22400cf30 comp rx=0 tx=0).stop 2024-07-17T08:36:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 shutdown_connections 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fe208076df0 0x7fe2080792b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe22c0ce1d0 0x7fe22c157b40 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe22c0b6790 0x7fe22c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 --2- 172.21.15.44:0/2841806275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe22c0b29e0 0x7fe22c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.639+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 >> 172.21.15.44:0/2841806275 conn(0x7fe22c0d3700 msgr2=0x7fe22c0b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.640+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 shutdown_connections 2024-07-17T08:36:25.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.640+0000 7fe230c00640 1 -- 172.21.15.44:0/2841806275 wait complete. 2024-07-17T08:36:25.665 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:36:25.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 -- 172.21.15.44:0/3462903194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc157ba0 msgr2=0x7f8ffc0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 --2- 172.21.15.44:0/3462903194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc157ba0 0x7f8ffc0cf3a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009830 tx=0x7f8fec02a510 comp rx=0 tx=0).stop 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 -- 172.21.15.44:0/3462903194 shutdown_connections 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 --2- 172.21.15.44:0/3462903194 >> v2:172.21.15.175:3300/0 conn(0x7f8ffc09c0b0 0x7f8ffc15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 --2- 172.21.15.44:0/3462903194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc157ba0 0x7f8ffc0cf3a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 -- 172.21.15.44:0/3462903194 >> 172.21.15.44:0/3462903194 conn(0x7f8ffc0b0e80 msgr2=0x7f8ffc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:25.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 -- 172.21.15.44:0/3462903194 shutdown_connections 2024-07-17T08:36:25.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.873+0000 7f9001800640 1 -- 172.21.15.44:0/3462903194 wait complete. 2024-07-17T08:36:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.875+0000 7f9001800640 1 Processor -- start 2024-07-17T08:36:25.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.875+0000 7f9001800640 1 -- start start 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc09c0b0 0x7f8ffc1f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0cf8e0 0x7f8ffc073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc15e070 con 0x7f8ffc157ba0 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc15e1f0 con 0x7f8ffc0cf8e0 2024-07-17T08:36:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.876+0000 7f9001800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ffc15e370 con 0x7f8ffc09c0b0 2024-07-17T08:36:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33946/0 (socket says 172.21.15.44:33946) 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 -- 172.21.15.44:0/260637971 learned_addr learned my addr 172.21.15.44:0/260637971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f8ffb400640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0cf8e0 0x7f8ffc073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f8ffbe00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc09c0b0 0x7f8ffc1f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 -- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc09c0b0 msgr2=0x7f8ffc1f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc09c0b0 0x7f8ffc1f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 -- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0cf8e0 msgr2=0x7f8ffc073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0cf8e0 0x7f8ffc073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.877+0000 7f9000e00640 1 -- 172.21.15.44:0/260637971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc158170 con 0x7f8ffc157ba0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f9000e00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400eb10 tx=0x7f8ff400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4004b20 con 0x7f8ffc157ba0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff4004ce0 con 0x7f8ffc157ba0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4005280 con 0x7f8ffc157ba0 2024-07-17T08:36:25.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc1f4c60 con 0x7f8ffc157ba0 2024-07-17T08:36:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.878+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1586c0 con 0x7f8ffc157ba0 2024-07-17T08:36:25.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.879+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc15e840 con 0x7f8ffc157ba0 2024-07-17T08:36:25.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.880+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f8ff4005420 con 0x7f8ffc157ba0 2024-07-17T08:36:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.881+0000 7f8ff3e00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 0x7f8fdc0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.882+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f8ff4099970 con 0x7f8ffc157ba0 2024-07-17T08:36:25.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.882+0000 7f8ffbe00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 0x7f8fdc0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:25.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.883+0000 7f8ffbe00640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 0x7f8fdc0792b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:25.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:25.886+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8ffc1f4c60 con 0x7f8ffc157ba0 2024-07-17T08:36:26.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.139+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ffc0b1300 con 0x7f8ffc157ba0 2024-07-17T08:36:26.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.140+0000 7f8ff3e00640 1 -- 172.21.15.44:0/260637971 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) ==== 95+0+5236 (secure 0 0 0) 0x7f8ff4062560 con 0x7f8ffc157ba0 2024-07-17T08:36:26.142 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:26.142 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":130,"btime":"2024-07-17T08:36:25:625513+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":8785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15}]} 2024-07-17T08:36:26.143 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 130 2024-07-17T08:36:26.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 msgr2=0x7f8fdc0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 0x7f8fdc0792b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec004730 comp rx=0 tx=0).stop 2024-07-17T08:36:26.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 msgr2=0x7f8ffc0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400eb10 tx=0x7f8ff400c2d0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 shutdown_connections 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fdc076df0 0x7f8fdc0792b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ffc157ba0 0x7f8ffc0cd5a0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0cf8e0 0x7f8ffc073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 --2- 172.21.15.44:0/260637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc09c0b0 0x7f8ffc1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.143+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 >> 172.21.15.44:0/260637971 conn(0x7f8ffc0b0e80 msgr2=0x7f8ffc15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.144+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 shutdown_connections 2024-07-17T08:36:26.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.144+0000 7f9001800640 1 -- 172.21.15.44:0/260637971 wait complete. 2024-07-17T08:36:26.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 -- 172.21.15.44:0/1136221939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49cc003a80 msgr2=0x7f49cc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 --2- 172.21.15.44:0/1136221939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49cc003a80 0x7f49cc003ea0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c310 tx=0x7f49cc005830 comp rx=0 tx=0).stop 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49e7400640 1 -- 172.21.15.44:0/1136221939 reap_dead start 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 -- 172.21.15.44:0/1136221939 shutdown_connections 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 -- 172.21.15.44:0/1136221939 >> 172.21.15.44:0/1136221939 conn(0x7f49e80b0fb0 msgr2=0x7f49e80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 -- 172.21.15.44:0/1136221939 shutdown_connections 2024-07-17T08:36:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.374+0000 7f49eda00640 1 -- 172.21.15.44:0/1136221939 wait complete. 2024-07-17T08:36:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.376+0000 7f49eda00640 1 Processor -- start 2024-07-17T08:36:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.376+0000 7f49eda00640 1 -- start start 2024-07-17T08:36:26.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 0x7f49e8073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 0x7f49e8145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49e806f480 con 0x7f49e81553f0 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49e815b8d0 con 0x7f49cc003a80 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49eda00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49e806f600 con 0x7f49e81ef600 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33958/0 (socket says 172.21.15.44:33958) 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.377+0000 7f49e7400640 1 -- 172.21.15.44:0/1031417501 learned_addr learned my addr 172.21.15.44:0/1031417501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e6a00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 0x7f49e8073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e6000640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 0x7f49e8145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 -- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 msgr2=0x7f49e8145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 0x7f49e8145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 -- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 msgr2=0x7f49e8073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 0x7f49e8073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 -- 172.21.15.44:0/1031417501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80742f0 con 0x7f49e81553f0 2024-07-17T08:36:26.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e6a00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 0x7f49e8073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:26.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e6000640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 0x7f49e8145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.378+0000 7f49e7400640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f49e806dbc0 tx=0x7f49e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.379+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e00035f0 con 0x7f49e81553f0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.379+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49e0016070 con 0x7f49e81553f0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.379+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0003b20 con 0x7f49e81553f0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.379+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80721b0 con 0x7f49e81553f0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.379+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e8145910 con 0x7f49e81553f0 2024-07-17T08:36:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.380+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e8153e40 con 0x7f49e81553f0 2024-07-17T08:36:26.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.381+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f49e0003cc0 con 0x7f49e81553f0 2024-07-17T08:36:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.382+0000 7f49dea00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 0x7f49c40792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.382+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f49e00993a0 con 0x7f49e81553f0 2024-07-17T08:36:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.383+0000 7f49e6a00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 0x7f49c40792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.384+0000 7f49e6a00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 0x7f49c40792e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49d0033000 tx=0x7f49d00019f0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.386+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f49e0062130 con 0x7f49e81553f0 2024-07-17T08:36:26.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.600+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49e806dbf0 con 0x7f49e81553f0 2024-07-17T08:36:26.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.603+0000 7f49dea00640 1 -- 172.21.15.44:0/1031417501 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) ==== 74+0+17817 (secure 0 0 0) 0x7f49e006a370 con 0x7f49e81553f0 2024-07-17T08:36:26.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:26.605 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":243,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:23.611829+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":241,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:26.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 msgr2=0x7f49c40792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 0x7f49c40792e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f49d0033000 tx=0x7f49d00019f0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 msgr2=0x7f49e80b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f49e806dbc0 tx=0x7f49e0011aa0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 shutdown_connections 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f49c4076e20 0x7f49c40792e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49e81553f0 0x7f49e80b38a0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49e81ef600 0x7f49e8145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 --2- 172.21.15.44:0/1031417501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49cc003a80 0x7f49e8073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 >> 172.21.15.44:0/1031417501 conn(0x7f49e80b0fb0 msgr2=0x7f49e80b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.606+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 shutdown_connections 2024-07-17T08:36:26.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.607+0000 7f49eda00640 1 -- 172.21.15.44:0/1031417501 wait complete. 2024-07-17T08:36:26.629 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:36:26.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 -- 172.21.15.44:0/2525481266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b8c80 msgr2=0x7fbd0c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 --2- 172.21.15.44:0/2525481266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c0b4ec0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009900 tx=0x7fbd0002a610 comp rx=0 tx=0).stop 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 -- 172.21.15.44:0/2525481266 shutdown_connections 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 --2- 172.21.15.44:0/2525481266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 0x7fbd0c06d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 --2- 172.21.15.44:0/2525481266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c0b4ec0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 -- 172.21.15.44:0/2525481266 >> 172.21.15.44:0/2525481266 conn(0x7fbd0c0cfec0 msgr2=0x7fbd0c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 -- 172.21.15.44:0/2525481266 shutdown_connections 2024-07-17T08:36:26.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.842+0000 7fbd12e00640 1 -- 172.21.15.44:0/2525481266 wait complete. 2024-07-17T08:36:26.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.844+0000 7fbd12e00640 1 Processor -- start 2024-07-17T08:36:26.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.844+0000 7fbd12e00640 1 -- start start 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 0x7fbd0c0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c06e780 con 0x7fbd0c0b5400 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c06e600 con 0x7fbd0c0b8c80 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd12e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0c06e900 con 0x7fbd0c0ce5c0 2024-07-17T08:36:26.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd11a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd11a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33968/0 (socket says 172.21.15.44:33968) 2024-07-17T08:36:26.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.845+0000 7fbd11a00640 1 -- 172.21.15.44:0/3788733170 learned_addr learned my addr 172.21.15.44:0/3788733170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11000640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd12400640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 0x7fbd0c0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 -- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 msgr2=0x7fbd0c0b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 0x7fbd0c0b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 -- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0b8c80 msgr2=0x7fbd0c1f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c1f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 -- 172.21.15.44:0/3788733170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c0b2260 con 0x7fbd0c0b5400 2024-07-17T08:36:26.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd11a00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd00003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0003f070 con 0x7fbd0c0b5400 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.846+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd00005600 con 0x7fbd0c0b5400 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.847+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c0cc490 con 0x7fbd0c0b5400 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.847+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00035030 con 0x7fbd0c0b5400 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.847+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c0cbed0 con 0x7fbd0c0b5400 2024-07-17T08:36:26.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.848+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c15e520 con 0x7fbd0c0b5400 2024-07-17T08:36:26.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.849+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fbd00042070 con 0x7fbd0c0b5400 2024-07-17T08:36:26.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.850+0000 7fbd09600640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 0x7fbcf00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:26.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.851+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fbd000b9710 con 0x7fbd0c0b5400 2024-07-17T08:36:26.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.851+0000 7fbd11000640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 0x7fbcf00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:26.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.852+0000 7fbd11000640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 0x7fbcf00792b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0b29f0 tx=0x7fbcfc005f90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:26.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:26.855+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbd00082460 con 0x7fbd0c0b5400 2024-07-17T08:36:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.064+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd0c0cd850 con 0x7fbcf0076df0 2024-07-17T08:36:27.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.080+0000 7fbd09600640 1 -- 172.21.15.44:0/3788733170 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7fbd0c0cd850 con 0x7fbcf0076df0 2024-07-17T08:36:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:27.088 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:27.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.087+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 msgr2=0x7fbcf00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.087+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 0x7fbcf00792b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0b29f0 tx=0x7fbcfc005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.087+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 msgr2=0x7fbd0c0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.087+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd00003820 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 shutdown_connections 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fbcf0076df0 0x7fbcf00792b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd0c0ce5c0 0x7fbd0c0b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd0c0b8c80 0x7fbd0c1f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 --2- 172.21.15.44:0/3788733170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c0b5400 0x7fbd0c0cb560 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 >> 172.21.15.44:0/3788733170 conn(0x7fbd0c0cfec0 msgr2=0x7fbd0c0c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 shutdown_connections 2024-07-17T08:36:27.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.088+0000 7fbd12e00640 1 -- 172.21.15.44:0/3788733170 wait complete. 2024-07-17T08:36:27.111 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-07-17T08:36:25.281688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":428,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2564,"ondisk_log_size":2564,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406492,"kb_used_data":25124,"kb_used_omap":12,"kb_used_meta":381235,"kb_avail":754568228,"statfs":{"total":773094113280,"available":772677865472,"internally_reserved":0,"allocated":25726976,"data_stored":14548904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390385200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.712823"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850426+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:23.850426+0000","last_peered":"2024-07-17T08:36:23.850426+0000","last_clean":"2024-07-17T08:36:23.850426+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:23.850426+0000","last_undegraded":"2024-07-17T08:36:23.850426+0000","last_fullsized":"2024-07-17T08:36:23.850426+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625143+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:23.625143+0000","last_peered":"2024-07-17T08:36:23.625143+0000","last_clean":"2024-07-17T08:36:23.625143+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:23.625143+0000","last_undegraded":"2024-07-17T08:36:23.625143+0000","last_fullsized":"2024-07-17T08:36:23.625143+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854093+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:23.854093+0000","last_peered":"2024-07-17T08:36:23.854093+0000","last_clean":"2024-07-17T08:36:23.854093+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:23.854093+0000","last_undegraded":"2024-07-17T08:36:23.854093+0000","last_fullsized":"2024-07-17T08:36:23.854093+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854686+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:23.854686+0000","last_peered":"2024-07-17T08:36:23.854686+0000","last_clean":"2024-07-17T08:36:23.854686+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:23.854686+0000","last_undegraded":"2024-07-17T08:36:23.854686+0000","last_fullsized":"2024-07-17T08:36:23.854686+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629764+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:23.629764+0000","last_peered":"2024-07-17T08:36:23.629764+0000","last_clean":"2024-07-17T08:36:23.629764+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:23.629764+0000","last_undegraded":"2024-07-17T08:36:23.629764+0000","last_fullsized":"2024-07-17T08:36:23.629764+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849288+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:23.849288+0000","last_peered":"2024-07-17T08:36:23.849288+0000","last_clean":"2024-07-17T08:36:23.849288+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:23.849288+0000","last_undegraded":"2024-07-17T08:36:23.849288+0000","last_fullsized":"2024-07-17T08:36:23.849288+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.635916+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:23.635916+0000","last_peered":"2024-07-17T08:36:23.635916+0000","last_clean":"2024-07-17T08:36:23.635916+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:23.635916+0000","last_undegraded":"2024-07-17T08:36:23.635916+0000","last_fullsized":"2024-07-17T08:36:23.635916+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628524+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:23.628524+0000","last_peered":"2024-07-17T08:36:23.628524+0000","last_clean":"2024-07-17T08:36:23.628524+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:23.628524+0000","last_undegraded":"2024-07-17T08:36:23.628524+0000","last_fullsized":"2024-07-17T08:36:23.628524+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850711+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:23.850711+0000","last_peered":"2024-07-17T08:36:23.850711+0000","last_clean":"2024-07-17T08:36:23.850711+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:23.850711+0000","last_undegraded":"2024-07-17T08:36:23.850711+0000","last_fullsized":"2024-07-17T08:36:23.850711+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854296+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:23.854296+0000","last_peered":"2024-07-17T08:36:23.854296+0000","last_clean":"2024-07-17T08:36:23.854296+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:23.854296+0000","last_undegraded":"2024-07-17T08:36:23.854296+0000","last_fullsized":"2024-07-17T08:36:23.854296+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854250+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:23.854250+0000","last_peered":"2024-07-17T08:36:23.854250+0000","last_clean":"2024-07-17T08:36:23.854250+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:23.854250+0000","last_undegraded":"2024-07-17T08:36:23.854250+0000","last_fullsized":"2024-07-17T08:36:23.854250+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849906+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:23.849906+0000","last_peered":"2024-07-17T08:36:23.849906+0000","last_clean":"2024-07-17T08:36:23.849906+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:23.849906+0000","last_undegraded":"2024-07-17T08:36:23.849906+0000","last_fullsized":"2024-07-17T08:36:23.849906+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627593+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:23.627593+0000","last_peered":"2024-07-17T08:36:23.627593+0000","last_clean":"2024-07-17T08:36:23.627593+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:23.627593+0000","last_undegraded":"2024-07-17T08:36:23.627593+0000","last_fullsized":"2024-07-17T08:36:23.627593+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851174+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:23.851174+0000","last_peered":"2024-07-17T08:36:23.851174+0000","last_clean":"2024-07-17T08:36:23.851174+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:23.851174+0000","last_undegraded":"2024-07-17T08:36:23.851174+0000","last_fullsized":"2024-07-17T08:36:23.851174+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641220+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:23.641220+0000","last_peered":"2024-07-17T08:36:23.641220+0000","last_clean":"2024-07-17T08:36:23.641220+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:23.641220+0000","last_undegraded":"2024-07-17T08:36:23.641220+0000","last_fullsized":"2024-07-17T08:36:23.641220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851329+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:23.851329+0000","last_peered":"2024-07-17T08:36:23.851329+0000","last_clean":"2024-07-17T08:36:23.851329+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:23.851329+0000","last_undegraded":"2024-07-17T08:36:23.851329+0000","last_fullsized":"2024-07-17T08:36:23.851329+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627694+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:23.627694+0000","last_peered":"2024-07-17T08:36:23.627694+0000","last_clean":"2024-07-17T08:36:23.627694+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:23.627694+0000","last_undegraded":"2024-07-17T08:36:23.627694+0000","last_fullsized":"2024-07-17T08:36:23.627694+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854574+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:23.854574+0000","last_peered":"2024-07-17T08:36:23.854574+0000","last_clean":"2024-07-17T08:36:23.854574+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:23.854574+0000","last_undegraded":"2024-07-17T08:36:23.854574+0000","last_fullsized":"2024-07-17T08:36:23.854574+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641939+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:23.641939+0000","last_peered":"2024-07-17T08:36:23.641939+0000","last_clean":"2024-07-17T08:36:23.641939+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:23.641939+0000","last_undegraded":"2024-07-17T08:36:23.641939+0000","last_fullsized":"2024-07-17T08:36:23.641939+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625190+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:23.625190+0000","last_peered":"2024-07-17T08:36:23.625190+0000","last_clean":"2024-07-17T08:36:23.625190+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:23.625190+0000","last_undegraded":"2024-07-17T08:36:23.625190+0000","last_fullsized":"2024-07-17T08:36:23.625190+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636273+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:23.636273+0000","last_peered":"2024-07-17T08:36:23.636273+0000","last_clean":"2024-07-17T08:36:23.636273+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:23.636273+0000","last_undegraded":"2024-07-17T08:36:23.636273+0000","last_fullsized":"2024-07-17T08:36:23.636273+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627847+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:23.627847+0000","last_peered":"2024-07-17T08:36:23.627847+0000","last_clean":"2024-07-17T08:36:23.627847+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:23.627847+0000","last_undegraded":"2024-07-17T08:36:23.627847+0000","last_fullsized":"2024-07-17T08:36:23.627847+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641422+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:23.641422+0000","last_peered":"2024-07-17T08:36:23.641422+0000","last_clean":"2024-07-17T08:36:23.641422+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:23.641422+0000","last_undegraded":"2024-07-17T08:36:23.641422+0000","last_fullsized":"2024-07-17T08:36:23.641422+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626651+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:23.626651+0000","last_peered":"2024-07-17T08:36:23.626651+0000","last_clean":"2024-07-17T08:36:23.626651+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:23.626651+0000","last_undegraded":"2024-07-17T08:36:23.626651+0000","last_fullsized":"2024-07-17T08:36:23.626651+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625711+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:23.625711+0000","last_peered":"2024-07-17T08:36:23.625711+0000","last_clean":"2024-07-17T08:36:23.625711+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:23.625711+0000","last_undegraded":"2024-07-17T08:36:23.625711+0000","last_fullsized":"2024-07-17T08:36:23.625711+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636787+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:23.636787+0000","last_peered":"2024-07-17T08:36:23.636787+0000","last_clean":"2024-07-17T08:36:23.636787+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:23.636787+0000","last_undegraded":"2024-07-17T08:36:23.636787+0000","last_fullsized":"2024-07-17T08:36:23.636787+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629715+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:23.629715+0000","last_peered":"2024-07-17T08:36:23.629715+0000","last_clean":"2024-07-17T08:36:23.629715+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:23.629715+0000","last_undegraded":"2024-07-17T08:36:23.629715+0000","last_fullsized":"2024-07-17T08:36:23.629715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849683+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:23.849683+0000","last_peered":"2024-07-17T08:36:23.849683+0000","last_clean":"2024-07-17T08:36:23.849683+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:23.849683+0000","last_undegraded":"2024-07-17T08:36:23.849683+0000","last_fullsized":"2024-07-17T08:36:23.849683+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852488+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:23.852488+0000","last_peered":"2024-07-17T08:36:23.852488+0000","last_clean":"2024-07-17T08:36:23.852488+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:23.852488+0000","last_undegraded":"2024-07-17T08:36:23.852488+0000","last_fullsized":"2024-07-17T08:36:23.852488+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852622+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:23.852622+0000","last_peered":"2024-07-17T08:36:23.852622+0000","last_clean":"2024-07-17T08:36:23.852622+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:23.852622+0000","last_undegraded":"2024-07-17T08:36:23.852622+0000","last_fullsized":"2024-07-17T08:36:23.852622+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624724+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:23.624724+0000","last_peered":"2024-07-17T08:36:23.624724+0000","last_clean":"2024-07-17T08:36:23.624724+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:23.624724+0000","last_undegraded":"2024-07-17T08:36:23.624724+0000","last_fullsized":"2024-07-17T08:36:23.624724+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849335+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:23.849335+0000","last_peered":"2024-07-17T08:36:23.849335+0000","last_clean":"2024-07-17T08:36:23.849335+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:23.849335+0000","last_undegraded":"2024-07-17T08:36:23.849335+0000","last_fullsized":"2024-07-17T08:36:23.849335+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624626+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:23.624626+0000","last_peered":"2024-07-17T08:36:23.624626+0000","last_clean":"2024-07-17T08:36:23.624626+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:23.624626+0000","last_undegraded":"2024-07-17T08:36:23.624626+0000","last_fullsized":"2024-07-17T08:36:23.624626+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850538+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:23.850538+0000","last_peered":"2024-07-17T08:36:23.850538+0000","last_clean":"2024-07-17T08:36:23.850538+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:23.850538+0000","last_undegraded":"2024-07-17T08:36:23.850538+0000","last_fullsized":"2024-07-17T08:36:23.850538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641461+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:23.641461+0000","last_peered":"2024-07-17T08:36:23.641461+0000","last_clean":"2024-07-17T08:36:23.641461+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:23.641461+0000","last_undegraded":"2024-07-17T08:36:23.641461+0000","last_fullsized":"2024-07-17T08:36:23.641461+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627014+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:23.627014+0000","last_peered":"2024-07-17T08:36:23.627014+0000","last_clean":"2024-07-17T08:36:23.627014+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:23.627014+0000","last_undegraded":"2024-07-17T08:36:23.627014+0000","last_fullsized":"2024-07-17T08:36:23.627014+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850486+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:23.850486+0000","last_peered":"2024-07-17T08:36:23.850486+0000","last_clean":"2024-07-17T08:36:23.850486+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:23.850486+0000","last_undegraded":"2024-07-17T08:36:23.850486+0000","last_fullsized":"2024-07-17T08:36:23.850486+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853381+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:23.853381+0000","last_peered":"2024-07-17T08:36:23.853381+0000","last_clean":"2024-07-17T08:36:23.853381+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:23.853381+0000","last_undegraded":"2024-07-17T08:36:23.853381+0000","last_fullsized":"2024-07-17T08:36:23.853381+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853089+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:23.853089+0000","last_peered":"2024-07-17T08:36:23.853089+0000","last_clean":"2024-07-17T08:36:23.853089+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:23.853089+0000","last_undegraded":"2024-07-17T08:36:23.853089+0000","last_fullsized":"2024-07-17T08:36:23.853089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641339+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:23.641339+0000","last_peered":"2024-07-17T08:36:23.641339+0000","last_clean":"2024-07-17T08:36:23.641339+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:23.641339+0000","last_undegraded":"2024-07-17T08:36:23.641339+0000","last_fullsized":"2024-07-17T08:36:23.641339+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636508+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:23.636508+0000","last_peered":"2024-07-17T08:36:23.636508+0000","last_clean":"2024-07-17T08:36:23.636508+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:23.636508+0000","last_undegraded":"2024-07-17T08:36:23.636508+0000","last_fullsized":"2024-07-17T08:36:23.636508+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852160+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:23.852160+0000","last_peered":"2024-07-17T08:36:23.852160+0000","last_clean":"2024-07-17T08:36:23.852160+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:23.852160+0000","last_undegraded":"2024-07-17T08:36:23.852160+0000","last_fullsized":"2024-07-17T08:36:23.852160+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854233+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:23.854233+0000","last_peered":"2024-07-17T08:36:23.854233+0000","last_clean":"2024-07-17T08:36:23.854233+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:23.854233+0000","last_undegraded":"2024-07-17T08:36:23.854233+0000","last_fullsized":"2024-07-17T08:36:23.854233+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851733+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:23.851733+0000","last_peered":"2024-07-17T08:36:23.851733+0000","last_clean":"2024-07-17T08:36:23.851733+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:23.851733+0000","last_undegraded":"2024-07-17T08:36:23.851733+0000","last_fullsized":"2024-07-17T08:36:23.851733+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629917+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:23.629917+0000","last_peered":"2024-07-17T08:36:23.629917+0000","last_clean":"2024-07-17T08:36:23.629917+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:23.629917+0000","last_undegraded":"2024-07-17T08:36:23.629917+0000","last_fullsized":"2024-07-17T08:36:23.629917+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849532+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:23.849532+0000","last_peered":"2024-07-17T08:36:23.849532+0000","last_clean":"2024-07-17T08:36:23.849532+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:23.849532+0000","last_undegraded":"2024-07-17T08:36:23.849532+0000","last_fullsized":"2024-07-17T08:36:23.849532+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636337+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:23.636337+0000","last_peered":"2024-07-17T08:36:23.636337+0000","last_clean":"2024-07-17T08:36:23.636337+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:23.636337+0000","last_undegraded":"2024-07-17T08:36:23.636337+0000","last_fullsized":"2024-07-17T08:36:23.636337+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853354+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:23.853354+0000","last_peered":"2024-07-17T08:36:23.853354+0000","last_clean":"2024-07-17T08:36:23.853354+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:23.853354+0000","last_undegraded":"2024-07-17T08:36:23.853354+0000","last_fullsized":"2024-07-17T08:36:23.853354+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626897+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:23.626897+0000","last_peered":"2024-07-17T08:36:23.626897+0000","last_clean":"2024-07-17T08:36:23.626897+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:23.626897+0000","last_undegraded":"2024-07-17T08:36:23.626897+0000","last_fullsized":"2024-07-17T08:36:23.626897+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854490+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:23.854490+0000","last_peered":"2024-07-17T08:36:23.854490+0000","last_clean":"2024-07-17T08:36:23.854490+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:23.854490+0000","last_undegraded":"2024-07-17T08:36:23.854490+0000","last_fullsized":"2024-07-17T08:36:23.854490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852336+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:23.852336+0000","last_peered":"2024-07-17T08:36:23.852336+0000","last_clean":"2024-07-17T08:36:23.852336+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:23.852336+0000","last_undegraded":"2024-07-17T08:36:23.852336+0000","last_fullsized":"2024-07-17T08:36:23.852336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629866+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:23.629866+0000","last_peered":"2024-07-17T08:36:23.629866+0000","last_clean":"2024-07-17T08:36:23.629866+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:23.629866+0000","last_undegraded":"2024-07-17T08:36:23.629866+0000","last_fullsized":"2024-07-17T08:36:23.629866+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.635920+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:23.635920+0000","last_peered":"2024-07-17T08:36:23.635920+0000","last_clean":"2024-07-17T08:36:23.635920+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:23.635920+0000","last_undegraded":"2024-07-17T08:36:23.635920+0000","last_fullsized":"2024-07-17T08:36:23.635920+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"242'6","reported_seq":20,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849794+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:23.849794+0000","last_peered":"2024-07-17T08:36:23.849794+0000","last_clean":"2024-07-17T08:36:23.849794+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:23.849794+0000","last_undegraded":"2024-07-17T08:36:23.849794+0000","last_fullsized":"2024-07-17T08:36:23.849794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625005+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:23.625005+0000","last_peered":"2024-07-17T08:36:23.625005+0000","last_clean":"2024-07-17T08:36:23.625005+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:23.625005+0000","last_undegraded":"2024-07-17T08:36:23.625005+0000","last_fullsized":"2024-07-17T08:36:23.625005+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641428+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:23.641428+0000","last_peered":"2024-07-17T08:36:23.641428+0000","last_clean":"2024-07-17T08:36:23.641428+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:23.641428+0000","last_undegraded":"2024-07-17T08:36:23.641428+0000","last_fullsized":"2024-07-17T08:36:23.641428+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636485+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:23.636485+0000","last_peered":"2024-07-17T08:36:23.636485+0000","last_clean":"2024-07-17T08:36:23.636485+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:23.636485+0000","last_undegraded":"2024-07-17T08:36:23.636485+0000","last_fullsized":"2024-07-17T08:36:23.636485+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629236+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:23.629236+0000","last_peered":"2024-07-17T08:36:23.629236+0000","last_clean":"2024-07-17T08:36:23.629236+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:23.629236+0000","last_undegraded":"2024-07-17T08:36:23.629236+0000","last_fullsized":"2024-07-17T08:36:23.629236+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641706+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:23.641706+0000","last_peered":"2024-07-17T08:36:23.641706+0000","last_clean":"2024-07-17T08:36:23.641706+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:23.641706+0000","last_undegraded":"2024-07-17T08:36:23.641706+0000","last_fullsized":"2024-07-17T08:36:23.641706+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628693+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:23.628693+0000","last_peered":"2024-07-17T08:36:23.628693+0000","last_clean":"2024-07-17T08:36:23.628693+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:23.628693+0000","last_undegraded":"2024-07-17T08:36:23.628693+0000","last_fullsized":"2024-07-17T08:36:23.628693+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629696+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:23.629696+0000","last_peered":"2024-07-17T08:36:23.629696+0000","last_clean":"2024-07-17T08:36:23.629696+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:23.629696+0000","last_undegraded":"2024-07-17T08:36:23.629696+0000","last_fullsized":"2024-07-17T08:36:23.629696+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624520+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:23.624520+0000","last_peered":"2024-07-17T08:36:23.624520+0000","last_clean":"2024-07-17T08:36:23.624520+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:23.624520+0000","last_undegraded":"2024-07-17T08:36:23.624520+0000","last_fullsized":"2024-07-17T08:36:23.624520+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628528+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:23.628528+0000","last_peered":"2024-07-17T08:36:23.628528+0000","last_clean":"2024-07-17T08:36:23.628528+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:23.628528+0000","last_undegraded":"2024-07-17T08:36:23.628528+0000","last_fullsized":"2024-07-17T08:36:23.628528+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850814+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:23.850814+0000","last_peered":"2024-07-17T08:36:23.850814+0000","last_clean":"2024-07-17T08:36:23.850814+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:23.850814+0000","last_undegraded":"2024-07-17T08:36:23.850814+0000","last_fullsized":"2024-07-17T08:36:23.850814+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628540+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:23.628540+0000","last_peered":"2024-07-17T08:36:23.628540+0000","last_clean":"2024-07-17T08:36:23.628540+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:23.628540+0000","last_undegraded":"2024-07-17T08:36:23.628540+0000","last_fullsized":"2024-07-17T08:36:23.628540+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625278+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:23.625278+0000","last_peered":"2024-07-17T08:36:23.625278+0000","last_clean":"2024-07-17T08:36:23.625278+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:23.625278+0000","last_undegraded":"2024-07-17T08:36:23.625278+0000","last_fullsized":"2024-07-17T08:36:23.625278+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624759+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:23.624759+0000","last_peered":"2024-07-17T08:36:23.624759+0000","last_clean":"2024-07-17T08:36:23.624759+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:23.624759+0000","last_undegraded":"2024-07-17T08:36:23.624759+0000","last_fullsized":"2024-07-17T08:36:23.624759+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627159+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:23.627159+0000","last_peered":"2024-07-17T08:36:23.627159+0000","last_clean":"2024-07-17T08:36:23.627159+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:23.627159+0000","last_undegraded":"2024-07-17T08:36:23.627159+0000","last_fullsized":"2024-07-17T08:36:23.627159+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851698+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:23.851698+0000","last_peered":"2024-07-17T08:36:23.851698+0000","last_clean":"2024-07-17T08:36:23.851698+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:23.851698+0000","last_undegraded":"2024-07-17T08:36:23.851698+0000","last_fullsized":"2024-07-17T08:36:23.851698+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641445+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:23.641445+0000","last_peered":"2024-07-17T08:36:23.641445+0000","last_clean":"2024-07-17T08:36:23.641445+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:23.641445+0000","last_undegraded":"2024-07-17T08:36:23.641445+0000","last_fullsized":"2024-07-17T08:36:23.641445+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854305+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:23.854305+0000","last_peered":"2024-07-17T08:36:23.854305+0000","last_clean":"2024-07-17T08:36:23.854305+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:23.854305+0000","last_undegraded":"2024-07-17T08:36:23.854305+0000","last_fullsized":"2024-07-17T08:36:23.854305+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853967+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:23.853967+0000","last_peered":"2024-07-17T08:36:23.853967+0000","last_clean":"2024-07-17T08:36:23.853967+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:23.853967+0000","last_undegraded":"2024-07-17T08:36:23.853967+0000","last_fullsized":"2024-07-17T08:36:23.853967+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625164+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:23.625164+0000","last_peered":"2024-07-17T08:36:23.625164+0000","last_clean":"2024-07-17T08:36:23.625164+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:23.625164+0000","last_undegraded":"2024-07-17T08:36:23.625164+0000","last_fullsized":"2024-07-17T08:36:23.625164+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627369+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:23.627369+0000","last_peered":"2024-07-17T08:36:23.627369+0000","last_clean":"2024-07-17T08:36:23.627369+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:23.627369+0000","last_undegraded":"2024-07-17T08:36:23.627369+0000","last_fullsized":"2024-07-17T08:36:23.627369+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627991+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:23.627991+0000","last_peered":"2024-07-17T08:36:23.627991+0000","last_clean":"2024-07-17T08:36:23.627991+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:23.627991+0000","last_undegraded":"2024-07-17T08:36:23.627991+0000","last_fullsized":"2024-07-17T08:36:23.627991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626871+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:23.626871+0000","last_peered":"2024-07-17T08:36:23.626871+0000","last_clean":"2024-07-17T08:36:23.626871+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:23.626871+0000","last_undegraded":"2024-07-17T08:36:23.626871+0000","last_fullsized":"2024-07-17T08:36:23.626871+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627454+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:23.627454+0000","last_peered":"2024-07-17T08:36:23.627454+0000","last_clean":"2024-07-17T08:36:23.627454+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:23.627454+0000","last_undegraded":"2024-07-17T08:36:23.627454+0000","last_fullsized":"2024-07-17T08:36:23.627454+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850851+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:23.850851+0000","last_peered":"2024-07-17T08:36:23.850851+0000","last_clean":"2024-07-17T08:36:23.850851+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:23.850851+0000","last_undegraded":"2024-07-17T08:36:23.850851+0000","last_fullsized":"2024-07-17T08:36:23.850851+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636089+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:23.636089+0000","last_peered":"2024-07-17T08:36:23.636089+0000","last_clean":"2024-07-17T08:36:23.636089+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:23.636089+0000","last_undegraded":"2024-07-17T08:36:23.636089+0000","last_fullsized":"2024-07-17T08:36:23.636089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854540+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:23.854540+0000","last_peered":"2024-07-17T08:36:23.854540+0000","last_clean":"2024-07-17T08:36:23.854540+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:23.854540+0000","last_undegraded":"2024-07-17T08:36:23.854540+0000","last_fullsized":"2024-07-17T08:36:23.854540+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641130+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:23.641130+0000","last_peered":"2024-07-17T08:36:23.641130+0000","last_clean":"2024-07-17T08:36:23.641130+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:23.641130+0000","last_undegraded":"2024-07-17T08:36:23.641130+0000","last_fullsized":"2024-07-17T08:36:23.641130+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.634790+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:23.634790+0000","last_peered":"2024-07-17T08:36:23.634790+0000","last_clean":"2024-07-17T08:36:23.634790+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:23.634790+0000","last_undegraded":"2024-07-17T08:36:23.634790+0000","last_fullsized":"2024-07-17T08:36:23.634790+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852058+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:23.852058+0000","last_peered":"2024-07-17T08:36:23.852058+0000","last_clean":"2024-07-17T08:36:23.852058+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:23.852058+0000","last_undegraded":"2024-07-17T08:36:23.852058+0000","last_fullsized":"2024-07-17T08:36:23.852058+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852842+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:23.852842+0000","last_peered":"2024-07-17T08:36:23.852842+0000","last_clean":"2024-07-17T08:36:23.852842+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:23.852842+0000","last_undegraded":"2024-07-17T08:36:23.852842+0000","last_fullsized":"2024-07-17T08:36:23.852842+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629563+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:23.629563+0000","last_peered":"2024-07-17T08:36:23.629563+0000","last_clean":"2024-07-17T08:36:23.629563+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:23.629563+0000","last_undegraded":"2024-07-17T08:36:23.629563+0000","last_fullsized":"2024-07-17T08:36:23.629563+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853415+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:23.853415+0000","last_peered":"2024-07-17T08:36:23.853415+0000","last_clean":"2024-07-17T08:36:23.853415+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:23.853415+0000","last_undegraded":"2024-07-17T08:36:23.853415+0000","last_fullsized":"2024-07-17T08:36:23.853415+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852973+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:23.852973+0000","last_peered":"2024-07-17T08:36:23.852973+0000","last_clean":"2024-07-17T08:36:23.852973+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:23.852973+0000","last_undegraded":"2024-07-17T08:36:23.852973+0000","last_fullsized":"2024-07-17T08:36:23.852973+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627572+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:23.627572+0000","last_peered":"2024-07-17T08:36:23.627572+0000","last_clean":"2024-07-17T08:36:23.627572+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:23.627572+0000","last_undegraded":"2024-07-17T08:36:23.627572+0000","last_fullsized":"2024-07-17T08:36:23.627572+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850003+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:23.850003+0000","last_peered":"2024-07-17T08:36:23.850003+0000","last_clean":"2024-07-17T08:36:23.850003+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:23.850003+0000","last_undegraded":"2024-07-17T08:36:23.850003+0000","last_fullsized":"2024-07-17T08:36:23.850003+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629378+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:23.629378+0000","last_peered":"2024-07-17T08:36:23.629378+0000","last_clean":"2024-07-17T08:36:23.629378+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:23.629378+0000","last_undegraded":"2024-07-17T08:36:23.629378+0000","last_fullsized":"2024-07-17T08:36:23.629378+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625225+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:23.625225+0000","last_peered":"2024-07-17T08:36:23.625225+0000","last_clean":"2024-07-17T08:36:23.625225+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:23.625225+0000","last_undegraded":"2024-07-17T08:36:23.625225+0000","last_fullsized":"2024-07-17T08:36:23.625225+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851791+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:23.851791+0000","last_peered":"2024-07-17T08:36:23.851791+0000","last_clean":"2024-07-17T08:36:23.851791+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:23.851791+0000","last_undegraded":"2024-07-17T08:36:23.851791+0000","last_fullsized":"2024-07-17T08:36:23.851791+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627227+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:23.627227+0000","last_peered":"2024-07-17T08:36:23.627227+0000","last_clean":"2024-07-17T08:36:23.627227+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:23.627227+0000","last_undegraded":"2024-07-17T08:36:23.627227+0000","last_fullsized":"2024-07-17T08:36:23.627227+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626830+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:23.626830+0000","last_peered":"2024-07-17T08:36:23.626830+0000","last_clean":"2024-07-17T08:36:23.626830+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:23.626830+0000","last_undegraded":"2024-07-17T08:36:23.626830+0000","last_fullsized":"2024-07-17T08:36:23.626830+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850392+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:23.850392+0000","last_peered":"2024-07-17T08:36:23.850392+0000","last_clean":"2024-07-17T08:36:23.850392+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:23.850392+0000","last_undegraded":"2024-07-17T08:36:23.850392+0000","last_fullsized":"2024-07-17T08:36:23.850392+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636583+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:23.636583+0000","last_peered":"2024-07-17T08:36:23.636583+0000","last_clean":"2024-07-17T08:36:23.636583+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:23.636583+0000","last_undegraded":"2024-07-17T08:36:23.636583+0000","last_fullsized":"2024-07-17T08:36:23.636583+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3467,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.640930+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:23.640930+0000","last_peered":"2024-07-17T08:36:23.640930+0000","last_clean":"2024-07-17T08:36:23.640930+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:23.640930+0000","last_undegraded":"2024-07-17T08:36:23.640930+0000","last_fullsized":"2024-07-17T08:36:23.640930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48648,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323192,"statfs":{"total":96636764160,"available":96586948608,"internally_reserved":0,"allocated":3665920,"data_stored":2263636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48056,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323784,"statfs":{"total":96636764160,"available":96587554816,"internally_reserved":0,"allocated":3059712,"data_stored":1665988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3063808,"data_stored":1673144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543354,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3055616,"data_stored":1665946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52244,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319596,"statfs":{"total":96636764160,"available":96583266304,"internally_reserved":0,"allocated":3088384,"data_stored":1675394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543352,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52232,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319608,"statfs":{"total":96636764160,"available":96583278592,"internally_reserved":0,"allocated":3072000,"data_stored":1673796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510649,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48068,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323772,"statfs":{"total":96636764160,"available":96587542528,"internally_reserved":0,"allocated":3063808,"data_stored":1673144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510648,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56964,"kb_used_data":3572,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314876,"statfs":{"total":96636764160,"available":96578433024,"internally_reserved":0,"allocated":3657728,"data_stored":2257856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:27.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 -- 172.21.15.44:0/3819626789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 msgr2=0x7fd288159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 --2- 172.21.15.44:0/3819626789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288159800 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fd280005fd0 tx=0x7fd28002a9f0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 -- 172.21.15.44:0/3819626789 shutdown_connections 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 --2- 172.21.15.44:0/3819626789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288159800 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 --2- 172.21.15.44:0/3819626789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b300 0x7fd28806e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 -- 172.21.15.44:0/3819626789 >> 172.21.15.44:0/3819626789 conn(0x7fd2880b0e80 msgr2=0x7fd2880b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 -- 172.21.15.44:0/3819626789 shutdown_connections 2024-07-17T08:36:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.368+0000 7fd28dc00640 1 -- 172.21.15.44:0/3819626789 wait complete. 2024-07-17T08:36:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.370+0000 7fd28dc00640 1 Processor -- start 2024-07-17T08:36:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.370+0000 7fd28dc00640 1 -- start start 2024-07-17T08:36:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28809b300 0x7fd2881f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 0x7fd288073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd288159ec0 con 0x7fd2880cfa90 2024-07-17T08:36:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd288159d40 con 0x7fd28809b300 2024-07-17T08:36:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.371+0000 7fd28dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd28815a040 con 0x7fd28809b6e0 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33988/0 (socket says 172.21.15.44:33988) 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 -- 172.21.15.44:0/2696094122 learned_addr learned my addr 172.21.15.44:0/2696094122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd287e00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28809b300 0x7fd2881f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd27fe00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 0x7fd288073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 -- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 msgr2=0x7fd288073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 0x7fd288073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 -- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28809b300 msgr2=0x7fd2881f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28809b300 0x7fd2881f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd28d200640 1 -- 172.21.15.44:0/2696094122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2881f6150 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.372+0000 7fd27fe00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 0x7fd288073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.373+0000 7fd28d200640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd280009d90 tx=0x7fd280003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.373+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280033050 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.373+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd280004550 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.373+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280038450 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.373+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28815b9b0 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.374+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881f5410 con 0x7fd2880cfa90 2024-07-17T08:36:27.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.375+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2881f5230 con 0x7fd2880cfa90 2024-07-17T08:36:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.376+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fd280036070 con 0x7fd2880cfa90 2024-07-17T08:36:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.377+0000 7fd284c00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 0x7fd2640792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.377+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fd2800b9fe0 con 0x7fd2880cfa90 2024-07-17T08:36:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.378+0000 7fd287e00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 0x7fd2640792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.379+0000 7fd287e00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 0x7fd2640792b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd278005fd0 tx=0x7fd278005e40 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.381+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd2881f5230 con 0x7fd2880cfa90 2024-07-17T08:36:27.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.587+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd28806e060 con 0x7fd264076df0 2024-07-17T08:36:27.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.600+0000 7fd284c00640 1 -- 172.21.15.44:0/2696094122 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7fd28806e060 con 0x7fd264076df0 2024-07-17T08:36:27.606 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:27.608 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 msgr2=0x7fd2640792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 0x7fd2640792b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd278005fd0 tx=0x7fd278005e40 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 msgr2=0x7fd288150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd280009d90 tx=0x7fd280003f10 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 shutdown_connections 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd264076df0 0x7fd2640792b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2880cfa90 0x7fd288150190 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd28809b6e0 0x7fd288073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 --2- 172.21.15.44:0/2696094122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd28809b300 0x7fd2881f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.607+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 >> 172.21.15.44:0/2696094122 conn(0x7fd2880b0e80 msgr2=0x7fd2880cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.608+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 shutdown_connections 2024-07-17T08:36:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.608+0000 7fd28dc00640 1 -- 172.21.15.44:0/2696094122 wait complete. 2024-07-17T08:36:27.629 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-07-17T08:36:27.282747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":428,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2564,"ondisk_log_size":2564,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406492,"kb_used_data":25124,"kb_used_omap":12,"kb_used_meta":381235,"kb_avail":754568228,"statfs":{"total":773094113280,"available":772677865472,"internally_reserved":0,"allocated":25726976,"data_stored":14548904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":390385200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002359"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850426+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:23.850426+0000","last_peered":"2024-07-17T08:36:23.850426+0000","last_clean":"2024-07-17T08:36:23.850426+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:23.850426+0000","last_undegraded":"2024-07-17T08:36:23.850426+0000","last_fullsized":"2024-07-17T08:36:23.850426+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625143+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:23.625143+0000","last_peered":"2024-07-17T08:36:23.625143+0000","last_clean":"2024-07-17T08:36:23.625143+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:23.625143+0000","last_undegraded":"2024-07-17T08:36:23.625143+0000","last_fullsized":"2024-07-17T08:36:23.625143+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854093+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:23.854093+0000","last_peered":"2024-07-17T08:36:23.854093+0000","last_clean":"2024-07-17T08:36:23.854093+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:23.854093+0000","last_undegraded":"2024-07-17T08:36:23.854093+0000","last_fullsized":"2024-07-17T08:36:23.854093+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854686+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:23.854686+0000","last_peered":"2024-07-17T08:36:23.854686+0000","last_clean":"2024-07-17T08:36:23.854686+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:23.854686+0000","last_undegraded":"2024-07-17T08:36:23.854686+0000","last_fullsized":"2024-07-17T08:36:23.854686+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629764+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:23.629764+0000","last_peered":"2024-07-17T08:36:23.629764+0000","last_clean":"2024-07-17T08:36:23.629764+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:23.629764+0000","last_undegraded":"2024-07-17T08:36:23.629764+0000","last_fullsized":"2024-07-17T08:36:23.629764+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849288+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:23.849288+0000","last_peered":"2024-07-17T08:36:23.849288+0000","last_clean":"2024-07-17T08:36:23.849288+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:23.849288+0000","last_undegraded":"2024-07-17T08:36:23.849288+0000","last_fullsized":"2024-07-17T08:36:23.849288+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.635916+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:23.635916+0000","last_peered":"2024-07-17T08:36:23.635916+0000","last_clean":"2024-07-17T08:36:23.635916+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:23.635916+0000","last_undegraded":"2024-07-17T08:36:23.635916+0000","last_fullsized":"2024-07-17T08:36:23.635916+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628524+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:23.628524+0000","last_peered":"2024-07-17T08:36:23.628524+0000","last_clean":"2024-07-17T08:36:23.628524+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:23.628524+0000","last_undegraded":"2024-07-17T08:36:23.628524+0000","last_fullsized":"2024-07-17T08:36:23.628524+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850711+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:23.850711+0000","last_peered":"2024-07-17T08:36:23.850711+0000","last_clean":"2024-07-17T08:36:23.850711+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:23.850711+0000","last_undegraded":"2024-07-17T08:36:23.850711+0000","last_fullsized":"2024-07-17T08:36:23.850711+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854296+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:23.854296+0000","last_peered":"2024-07-17T08:36:23.854296+0000","last_clean":"2024-07-17T08:36:23.854296+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:23.854296+0000","last_undegraded":"2024-07-17T08:36:23.854296+0000","last_fullsized":"2024-07-17T08:36:23.854296+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854250+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:23.854250+0000","last_peered":"2024-07-17T08:36:23.854250+0000","last_clean":"2024-07-17T08:36:23.854250+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:23.854250+0000","last_undegraded":"2024-07-17T08:36:23.854250+0000","last_fullsized":"2024-07-17T08:36:23.854250+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849906+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:23.849906+0000","last_peered":"2024-07-17T08:36:23.849906+0000","last_clean":"2024-07-17T08:36:23.849906+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:23.849906+0000","last_undegraded":"2024-07-17T08:36:23.849906+0000","last_fullsized":"2024-07-17T08:36:23.849906+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627593+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:23.627593+0000","last_peered":"2024-07-17T08:36:23.627593+0000","last_clean":"2024-07-17T08:36:23.627593+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:23.627593+0000","last_undegraded":"2024-07-17T08:36:23.627593+0000","last_fullsized":"2024-07-17T08:36:23.627593+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851174+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:23.851174+0000","last_peered":"2024-07-17T08:36:23.851174+0000","last_clean":"2024-07-17T08:36:23.851174+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:23.851174+0000","last_undegraded":"2024-07-17T08:36:23.851174+0000","last_fullsized":"2024-07-17T08:36:23.851174+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641220+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:23.641220+0000","last_peered":"2024-07-17T08:36:23.641220+0000","last_clean":"2024-07-17T08:36:23.641220+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:23.641220+0000","last_undegraded":"2024-07-17T08:36:23.641220+0000","last_fullsized":"2024-07-17T08:36:23.641220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851329+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:23.851329+0000","last_peered":"2024-07-17T08:36:23.851329+0000","last_clean":"2024-07-17T08:36:23.851329+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:23.851329+0000","last_undegraded":"2024-07-17T08:36:23.851329+0000","last_fullsized":"2024-07-17T08:36:23.851329+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627694+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:23.627694+0000","last_peered":"2024-07-17T08:36:23.627694+0000","last_clean":"2024-07-17T08:36:23.627694+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:23.627694+0000","last_undegraded":"2024-07-17T08:36:23.627694+0000","last_fullsized":"2024-07-17T08:36:23.627694+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854574+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:23.854574+0000","last_peered":"2024-07-17T08:36:23.854574+0000","last_clean":"2024-07-17T08:36:23.854574+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:23.854574+0000","last_undegraded":"2024-07-17T08:36:23.854574+0000","last_fullsized":"2024-07-17T08:36:23.854574+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641939+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:23.641939+0000","last_peered":"2024-07-17T08:36:23.641939+0000","last_clean":"2024-07-17T08:36:23.641939+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:23.641939+0000","last_undegraded":"2024-07-17T08:36:23.641939+0000","last_fullsized":"2024-07-17T08:36:23.641939+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625190+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:23.625190+0000","last_peered":"2024-07-17T08:36:23.625190+0000","last_clean":"2024-07-17T08:36:23.625190+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:23.625190+0000","last_undegraded":"2024-07-17T08:36:23.625190+0000","last_fullsized":"2024-07-17T08:36:23.625190+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636273+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:23.636273+0000","last_peered":"2024-07-17T08:36:23.636273+0000","last_clean":"2024-07-17T08:36:23.636273+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:23.636273+0000","last_undegraded":"2024-07-17T08:36:23.636273+0000","last_fullsized":"2024-07-17T08:36:23.636273+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627847+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:23.627847+0000","last_peered":"2024-07-17T08:36:23.627847+0000","last_clean":"2024-07-17T08:36:23.627847+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:23.627847+0000","last_undegraded":"2024-07-17T08:36:23.627847+0000","last_fullsized":"2024-07-17T08:36:23.627847+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641422+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:23.641422+0000","last_peered":"2024-07-17T08:36:23.641422+0000","last_clean":"2024-07-17T08:36:23.641422+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:23.641422+0000","last_undegraded":"2024-07-17T08:36:23.641422+0000","last_fullsized":"2024-07-17T08:36:23.641422+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626651+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:23.626651+0000","last_peered":"2024-07-17T08:36:23.626651+0000","last_clean":"2024-07-17T08:36:23.626651+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:23.626651+0000","last_undegraded":"2024-07-17T08:36:23.626651+0000","last_fullsized":"2024-07-17T08:36:23.626651+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625711+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:23.625711+0000","last_peered":"2024-07-17T08:36:23.625711+0000","last_clean":"2024-07-17T08:36:23.625711+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:23.625711+0000","last_undegraded":"2024-07-17T08:36:23.625711+0000","last_fullsized":"2024-07-17T08:36:23.625711+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636787+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:23.636787+0000","last_peered":"2024-07-17T08:36:23.636787+0000","last_clean":"2024-07-17T08:36:23.636787+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:23.636787+0000","last_undegraded":"2024-07-17T08:36:23.636787+0000","last_fullsized":"2024-07-17T08:36:23.636787+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629715+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:23.629715+0000","last_peered":"2024-07-17T08:36:23.629715+0000","last_clean":"2024-07-17T08:36:23.629715+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:23.629715+0000","last_undegraded":"2024-07-17T08:36:23.629715+0000","last_fullsized":"2024-07-17T08:36:23.629715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849683+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:23.849683+0000","last_peered":"2024-07-17T08:36:23.849683+0000","last_clean":"2024-07-17T08:36:23.849683+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:23.849683+0000","last_undegraded":"2024-07-17T08:36:23.849683+0000","last_fullsized":"2024-07-17T08:36:23.849683+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852488+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:23.852488+0000","last_peered":"2024-07-17T08:36:23.852488+0000","last_clean":"2024-07-17T08:36:23.852488+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:23.852488+0000","last_undegraded":"2024-07-17T08:36:23.852488+0000","last_fullsized":"2024-07-17T08:36:23.852488+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852622+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:23.852622+0000","last_peered":"2024-07-17T08:36:23.852622+0000","last_clean":"2024-07-17T08:36:23.852622+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:23.852622+0000","last_undegraded":"2024-07-17T08:36:23.852622+0000","last_fullsized":"2024-07-17T08:36:23.852622+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624724+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:23.624724+0000","last_peered":"2024-07-17T08:36:23.624724+0000","last_clean":"2024-07-17T08:36:23.624724+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:23.624724+0000","last_undegraded":"2024-07-17T08:36:23.624724+0000","last_fullsized":"2024-07-17T08:36:23.624724+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849335+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:23.849335+0000","last_peered":"2024-07-17T08:36:23.849335+0000","last_clean":"2024-07-17T08:36:23.849335+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:23.849335+0000","last_undegraded":"2024-07-17T08:36:23.849335+0000","last_fullsized":"2024-07-17T08:36:23.849335+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624626+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:23.624626+0000","last_peered":"2024-07-17T08:36:23.624626+0000","last_clean":"2024-07-17T08:36:23.624626+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:23.624626+0000","last_undegraded":"2024-07-17T08:36:23.624626+0000","last_fullsized":"2024-07-17T08:36:23.624626+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":18,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850538+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:23.850538+0000","last_peered":"2024-07-17T08:36:23.850538+0000","last_clean":"2024-07-17T08:36:23.850538+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:23.850538+0000","last_undegraded":"2024-07-17T08:36:23.850538+0000","last_fullsized":"2024-07-17T08:36:23.850538+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641461+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:23.641461+0000","last_peered":"2024-07-17T08:36:23.641461+0000","last_clean":"2024-07-17T08:36:23.641461+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:23.641461+0000","last_undegraded":"2024-07-17T08:36:23.641461+0000","last_fullsized":"2024-07-17T08:36:23.641461+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627014+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:23.627014+0000","last_peered":"2024-07-17T08:36:23.627014+0000","last_clean":"2024-07-17T08:36:23.627014+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:23.627014+0000","last_undegraded":"2024-07-17T08:36:23.627014+0000","last_fullsized":"2024-07-17T08:36:23.627014+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850486+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:23.850486+0000","last_peered":"2024-07-17T08:36:23.850486+0000","last_clean":"2024-07-17T08:36:23.850486+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:23.850486+0000","last_undegraded":"2024-07-17T08:36:23.850486+0000","last_fullsized":"2024-07-17T08:36:23.850486+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853381+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:23.853381+0000","last_peered":"2024-07-17T08:36:23.853381+0000","last_clean":"2024-07-17T08:36:23.853381+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:23.853381+0000","last_undegraded":"2024-07-17T08:36:23.853381+0000","last_fullsized":"2024-07-17T08:36:23.853381+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853089+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:23.853089+0000","last_peered":"2024-07-17T08:36:23.853089+0000","last_clean":"2024-07-17T08:36:23.853089+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:23.853089+0000","last_undegraded":"2024-07-17T08:36:23.853089+0000","last_fullsized":"2024-07-17T08:36:23.853089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641339+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:23.641339+0000","last_peered":"2024-07-17T08:36:23.641339+0000","last_clean":"2024-07-17T08:36:23.641339+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:23.641339+0000","last_undegraded":"2024-07-17T08:36:23.641339+0000","last_fullsized":"2024-07-17T08:36:23.641339+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636508+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:23.636508+0000","last_peered":"2024-07-17T08:36:23.636508+0000","last_clean":"2024-07-17T08:36:23.636508+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:23.636508+0000","last_undegraded":"2024-07-17T08:36:23.636508+0000","last_fullsized":"2024-07-17T08:36:23.636508+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852160+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:23.852160+0000","last_peered":"2024-07-17T08:36:23.852160+0000","last_clean":"2024-07-17T08:36:23.852160+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:23.852160+0000","last_undegraded":"2024-07-17T08:36:23.852160+0000","last_fullsized":"2024-07-17T08:36:23.852160+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854233+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:23.854233+0000","last_peered":"2024-07-17T08:36:23.854233+0000","last_clean":"2024-07-17T08:36:23.854233+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:23.854233+0000","last_undegraded":"2024-07-17T08:36:23.854233+0000","last_fullsized":"2024-07-17T08:36:23.854233+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851733+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:23.851733+0000","last_peered":"2024-07-17T08:36:23.851733+0000","last_clean":"2024-07-17T08:36:23.851733+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:23.851733+0000","last_undegraded":"2024-07-17T08:36:23.851733+0000","last_fullsized":"2024-07-17T08:36:23.851733+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629917+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:23.629917+0000","last_peered":"2024-07-17T08:36:23.629917+0000","last_clean":"2024-07-17T08:36:23.629917+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:23.629917+0000","last_undegraded":"2024-07-17T08:36:23.629917+0000","last_fullsized":"2024-07-17T08:36:23.629917+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849532+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:23.849532+0000","last_peered":"2024-07-17T08:36:23.849532+0000","last_clean":"2024-07-17T08:36:23.849532+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:23.849532+0000","last_undegraded":"2024-07-17T08:36:23.849532+0000","last_fullsized":"2024-07-17T08:36:23.849532+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636337+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:23.636337+0000","last_peered":"2024-07-17T08:36:23.636337+0000","last_clean":"2024-07-17T08:36:23.636337+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:23.636337+0000","last_undegraded":"2024-07-17T08:36:23.636337+0000","last_fullsized":"2024-07-17T08:36:23.636337+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853354+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:23.853354+0000","last_peered":"2024-07-17T08:36:23.853354+0000","last_clean":"2024-07-17T08:36:23.853354+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:23.853354+0000","last_undegraded":"2024-07-17T08:36:23.853354+0000","last_fullsized":"2024-07-17T08:36:23.853354+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626897+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:23.626897+0000","last_peered":"2024-07-17T08:36:23.626897+0000","last_clean":"2024-07-17T08:36:23.626897+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:23.626897+0000","last_undegraded":"2024-07-17T08:36:23.626897+0000","last_fullsized":"2024-07-17T08:36:23.626897+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854490+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:23.854490+0000","last_peered":"2024-07-17T08:36:23.854490+0000","last_clean":"2024-07-17T08:36:23.854490+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:23.854490+0000","last_undegraded":"2024-07-17T08:36:23.854490+0000","last_fullsized":"2024-07-17T08:36:23.854490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852336+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:23.852336+0000","last_peered":"2024-07-17T08:36:23.852336+0000","last_clean":"2024-07-17T08:36:23.852336+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:23.852336+0000","last_undegraded":"2024-07-17T08:36:23.852336+0000","last_fullsized":"2024-07-17T08:36:23.852336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":17,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629866+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:23.629866+0000","last_peered":"2024-07-17T08:36:23.629866+0000","last_clean":"2024-07-17T08:36:23.629866+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:23.629866+0000","last_undegraded":"2024-07-17T08:36:23.629866+0000","last_fullsized":"2024-07-17T08:36:23.629866+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.635920+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:23.635920+0000","last_peered":"2024-07-17T08:36:23.635920+0000","last_clean":"2024-07-17T08:36:23.635920+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:23.635920+0000","last_undegraded":"2024-07-17T08:36:23.635920+0000","last_fullsized":"2024-07-17T08:36:23.635920+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"242'6","reported_seq":20,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.849794+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:23.849794+0000","last_peered":"2024-07-17T08:36:23.849794+0000","last_clean":"2024-07-17T08:36:23.849794+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:23.849794+0000","last_undegraded":"2024-07-17T08:36:23.849794+0000","last_fullsized":"2024-07-17T08:36:23.849794+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625005+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:23.625005+0000","last_peered":"2024-07-17T08:36:23.625005+0000","last_clean":"2024-07-17T08:36:23.625005+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:23.625005+0000","last_undegraded":"2024-07-17T08:36:23.625005+0000","last_fullsized":"2024-07-17T08:36:23.625005+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641428+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:23.641428+0000","last_peered":"2024-07-17T08:36:23.641428+0000","last_clean":"2024-07-17T08:36:23.641428+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:23.641428+0000","last_undegraded":"2024-07-17T08:36:23.641428+0000","last_fullsized":"2024-07-17T08:36:23.641428+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636485+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:23.636485+0000","last_peered":"2024-07-17T08:36:23.636485+0000","last_clean":"2024-07-17T08:36:23.636485+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:23.636485+0000","last_undegraded":"2024-07-17T08:36:23.636485+0000","last_fullsized":"2024-07-17T08:36:23.636485+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629236+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:23.629236+0000","last_peered":"2024-07-17T08:36:23.629236+0000","last_clean":"2024-07-17T08:36:23.629236+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:23.629236+0000","last_undegraded":"2024-07-17T08:36:23.629236+0000","last_fullsized":"2024-07-17T08:36:23.629236+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641706+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:23.641706+0000","last_peered":"2024-07-17T08:36:23.641706+0000","last_clean":"2024-07-17T08:36:23.641706+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:23.641706+0000","last_undegraded":"2024-07-17T08:36:23.641706+0000","last_fullsized":"2024-07-17T08:36:23.641706+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628693+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:23.628693+0000","last_peered":"2024-07-17T08:36:23.628693+0000","last_clean":"2024-07-17T08:36:23.628693+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:23.628693+0000","last_undegraded":"2024-07-17T08:36:23.628693+0000","last_fullsized":"2024-07-17T08:36:23.628693+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629696+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:23.629696+0000","last_peered":"2024-07-17T08:36:23.629696+0000","last_clean":"2024-07-17T08:36:23.629696+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:23.629696+0000","last_undegraded":"2024-07-17T08:36:23.629696+0000","last_fullsized":"2024-07-17T08:36:23.629696+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":22,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624520+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:23.624520+0000","last_peered":"2024-07-17T08:36:23.624520+0000","last_clean":"2024-07-17T08:36:23.624520+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:23.624520+0000","last_undegraded":"2024-07-17T08:36:23.624520+0000","last_fullsized":"2024-07-17T08:36:23.624520+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628528+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:23.628528+0000","last_peered":"2024-07-17T08:36:23.628528+0000","last_clean":"2024-07-17T08:36:23.628528+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:23.628528+0000","last_undegraded":"2024-07-17T08:36:23.628528+0000","last_fullsized":"2024-07-17T08:36:23.628528+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850814+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:23.850814+0000","last_peered":"2024-07-17T08:36:23.850814+0000","last_clean":"2024-07-17T08:36:23.850814+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:23.850814+0000","last_undegraded":"2024-07-17T08:36:23.850814+0000","last_fullsized":"2024-07-17T08:36:23.850814+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.628540+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:23.628540+0000","last_peered":"2024-07-17T08:36:23.628540+0000","last_clean":"2024-07-17T08:36:23.628540+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:23.628540+0000","last_undegraded":"2024-07-17T08:36:23.628540+0000","last_fullsized":"2024-07-17T08:36:23.628540+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625278+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:23.625278+0000","last_peered":"2024-07-17T08:36:23.625278+0000","last_clean":"2024-07-17T08:36:23.625278+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:23.625278+0000","last_undegraded":"2024-07-17T08:36:23.625278+0000","last_fullsized":"2024-07-17T08:36:23.625278+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.624759+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:23.624759+0000","last_peered":"2024-07-17T08:36:23.624759+0000","last_clean":"2024-07-17T08:36:23.624759+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:23.624759+0000","last_undegraded":"2024-07-17T08:36:23.624759+0000","last_fullsized":"2024-07-17T08:36:23.624759+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627159+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:23.627159+0000","last_peered":"2024-07-17T08:36:23.627159+0000","last_clean":"2024-07-17T08:36:23.627159+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:23.627159+0000","last_undegraded":"2024-07-17T08:36:23.627159+0000","last_fullsized":"2024-07-17T08:36:23.627159+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851698+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:23.851698+0000","last_peered":"2024-07-17T08:36:23.851698+0000","last_clean":"2024-07-17T08:36:23.851698+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:23.851698+0000","last_undegraded":"2024-07-17T08:36:23.851698+0000","last_fullsized":"2024-07-17T08:36:23.851698+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641445+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:23.641445+0000","last_peered":"2024-07-17T08:36:23.641445+0000","last_clean":"2024-07-17T08:36:23.641445+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:23.641445+0000","last_undegraded":"2024-07-17T08:36:23.641445+0000","last_fullsized":"2024-07-17T08:36:23.641445+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854305+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:23.854305+0000","last_peered":"2024-07-17T08:36:23.854305+0000","last_clean":"2024-07-17T08:36:23.854305+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:23.854305+0000","last_undegraded":"2024-07-17T08:36:23.854305+0000","last_fullsized":"2024-07-17T08:36:23.854305+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853967+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:23.853967+0000","last_peered":"2024-07-17T08:36:23.853967+0000","last_clean":"2024-07-17T08:36:23.853967+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:23.853967+0000","last_undegraded":"2024-07-17T08:36:23.853967+0000","last_fullsized":"2024-07-17T08:36:23.853967+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625164+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:23.625164+0000","last_peered":"2024-07-17T08:36:23.625164+0000","last_clean":"2024-07-17T08:36:23.625164+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:23.625164+0000","last_undegraded":"2024-07-17T08:36:23.625164+0000","last_fullsized":"2024-07-17T08:36:23.625164+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627369+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:23.627369+0000","last_peered":"2024-07-17T08:36:23.627369+0000","last_clean":"2024-07-17T08:36:23.627369+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:23.627369+0000","last_undegraded":"2024-07-17T08:36:23.627369+0000","last_fullsized":"2024-07-17T08:36:23.627369+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627991+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:23.627991+0000","last_peered":"2024-07-17T08:36:23.627991+0000","last_clean":"2024-07-17T08:36:23.627991+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:23.627991+0000","last_undegraded":"2024-07-17T08:36:23.627991+0000","last_fullsized":"2024-07-17T08:36:23.627991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626871+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:23.626871+0000","last_peered":"2024-07-17T08:36:23.626871+0000","last_clean":"2024-07-17T08:36:23.626871+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:23.626871+0000","last_undegraded":"2024-07-17T08:36:23.626871+0000","last_fullsized":"2024-07-17T08:36:23.626871+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627454+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:23.627454+0000","last_peered":"2024-07-17T08:36:23.627454+0000","last_clean":"2024-07-17T08:36:23.627454+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:23.627454+0000","last_undegraded":"2024-07-17T08:36:23.627454+0000","last_fullsized":"2024-07-17T08:36:23.627454+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850851+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:23.850851+0000","last_peered":"2024-07-17T08:36:23.850851+0000","last_clean":"2024-07-17T08:36:23.850851+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:23.850851+0000","last_undegraded":"2024-07-17T08:36:23.850851+0000","last_fullsized":"2024-07-17T08:36:23.850851+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636089+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:23.636089+0000","last_peered":"2024-07-17T08:36:23.636089+0000","last_clean":"2024-07-17T08:36:23.636089+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:23.636089+0000","last_undegraded":"2024-07-17T08:36:23.636089+0000","last_fullsized":"2024-07-17T08:36:23.636089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":16,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.854540+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:23.854540+0000","last_peered":"2024-07-17T08:36:23.854540+0000","last_clean":"2024-07-17T08:36:23.854540+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:23.854540+0000","last_undegraded":"2024-07-17T08:36:23.854540+0000","last_fullsized":"2024-07-17T08:36:23.854540+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.641130+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:23.641130+0000","last_peered":"2024-07-17T08:36:23.641130+0000","last_clean":"2024-07-17T08:36:23.641130+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:23.641130+0000","last_undegraded":"2024-07-17T08:36:23.641130+0000","last_fullsized":"2024-07-17T08:36:23.641130+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.634790+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:23.634790+0000","last_peered":"2024-07-17T08:36:23.634790+0000","last_clean":"2024-07-17T08:36:23.634790+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:23.634790+0000","last_undegraded":"2024-07-17T08:36:23.634790+0000","last_fullsized":"2024-07-17T08:36:23.634790+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852058+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:23.852058+0000","last_peered":"2024-07-17T08:36:23.852058+0000","last_clean":"2024-07-17T08:36:23.852058+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:23.852058+0000","last_undegraded":"2024-07-17T08:36:23.852058+0000","last_fullsized":"2024-07-17T08:36:23.852058+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852842+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:23.852842+0000","last_peered":"2024-07-17T08:36:23.852842+0000","last_clean":"2024-07-17T08:36:23.852842+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:23.852842+0000","last_undegraded":"2024-07-17T08:36:23.852842+0000","last_fullsized":"2024-07-17T08:36:23.852842+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629563+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:23.629563+0000","last_peered":"2024-07-17T08:36:23.629563+0000","last_clean":"2024-07-17T08:36:23.629563+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:23.629563+0000","last_undegraded":"2024-07-17T08:36:23.629563+0000","last_fullsized":"2024-07-17T08:36:23.629563+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.853415+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:23.853415+0000","last_peered":"2024-07-17T08:36:23.853415+0000","last_clean":"2024-07-17T08:36:23.853415+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:23.853415+0000","last_undegraded":"2024-07-17T08:36:23.853415+0000","last_fullsized":"2024-07-17T08:36:23.853415+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.852973+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:23.852973+0000","last_peered":"2024-07-17T08:36:23.852973+0000","last_clean":"2024-07-17T08:36:23.852973+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:23.852973+0000","last_undegraded":"2024-07-17T08:36:23.852973+0000","last_fullsized":"2024-07-17T08:36:23.852973+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627572+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:23.627572+0000","last_peered":"2024-07-17T08:36:23.627572+0000","last_clean":"2024-07-17T08:36:23.627572+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:23.627572+0000","last_undegraded":"2024-07-17T08:36:23.627572+0000","last_fullsized":"2024-07-17T08:36:23.627572+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850003+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:23.850003+0000","last_peered":"2024-07-17T08:36:23.850003+0000","last_clean":"2024-07-17T08:36:23.850003+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:23.850003+0000","last_undegraded":"2024-07-17T08:36:23.850003+0000","last_fullsized":"2024-07-17T08:36:23.850003+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.629378+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:23.629378+0000","last_peered":"2024-07-17T08:36:23.629378+0000","last_clean":"2024-07-17T08:36:23.629378+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:23.629378+0000","last_undegraded":"2024-07-17T08:36:23.629378+0000","last_fullsized":"2024-07-17T08:36:23.629378+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.625225+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:23.625225+0000","last_peered":"2024-07-17T08:36:23.625225+0000","last_clean":"2024-07-17T08:36:23.625225+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:23.625225+0000","last_undegraded":"2024-07-17T08:36:23.625225+0000","last_fullsized":"2024-07-17T08:36:23.625225+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.851791+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:23.851791+0000","last_peered":"2024-07-17T08:36:23.851791+0000","last_clean":"2024-07-17T08:36:23.851791+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:23.851791+0000","last_undegraded":"2024-07-17T08:36:23.851791+0000","last_fullsized":"2024-07-17T08:36:23.851791+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.627227+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:23.627227+0000","last_peered":"2024-07-17T08:36:23.627227+0000","last_clean":"2024-07-17T08:36:23.627227+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:23.627227+0000","last_undegraded":"2024-07-17T08:36:23.627227+0000","last_fullsized":"2024-07-17T08:36:23.627227+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.626830+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:23.626830+0000","last_peered":"2024-07-17T08:36:23.626830+0000","last_clean":"2024-07-17T08:36:23.626830+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:23.626830+0000","last_undegraded":"2024-07-17T08:36:23.626830+0000","last_fullsized":"2024-07-17T08:36:23.626830+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.850392+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:23.850392+0000","last_peered":"2024-07-17T08:36:23.850392+0000","last_clean":"2024-07-17T08:36:23.850392+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:23.850392+0000","last_undegraded":"2024-07-17T08:36:23.850392+0000","last_fullsized":"2024-07-17T08:36:23.850392+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":15,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.636583+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:23.636583+0000","last_peered":"2024-07-17T08:36:23.636583+0000","last_clean":"2024-07-17T08:36:23.636583+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:23.636583+0000","last_undegraded":"2024-07-17T08:36:23.636583+0000","last_fullsized":"2024-07-17T08:36:23.636583+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3467,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-07-17T08:36:23.640930+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:23.640930+0000","last_peered":"2024-07-17T08:36:23.640930+0000","last_clean":"2024-07-17T08:36:23.640930+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:23.640930+0000","last_undegraded":"2024-07-17T08:36:23.640930+0000","last_fullsized":"2024-07-17T08:36:23.640930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48648,"kb_used_data":3580,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323192,"statfs":{"total":96636764160,"available":96586948608,"internally_reserved":0,"allocated":3665920,"data_stored":2263636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48056,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323784,"statfs":{"total":96636764160,"available":96587554816,"internally_reserved":0,"allocated":3059712,"data_stored":1665988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48064,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323776,"statfs":{"total":96636764160,"available":96587546624,"internally_reserved":0,"allocated":3063808,"data_stored":1673144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543354,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3055616,"data_stored":1665946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52244,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319596,"statfs":{"total":96636764160,"available":96583266304,"internally_reserved":0,"allocated":3088384,"data_stored":1675394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543352,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52232,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319608,"statfs":{"total":96636764160,"available":96583278592,"internally_reserved":0,"allocated":3072000,"data_stored":1673796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510649,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48068,"kb_used_data":2992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323772,"statfs":{"total":96636764160,"available":96587542528,"internally_reserved":0,"allocated":3063808,"data_stored":1673144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510648,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56964,"kb_used_data":3572,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314876,"statfs":{"total":96636764160,"available":96578433024,"internally_reserved":0,"allocated":3657728,"data_stored":2257856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:27.632 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:36:27.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 -- 172.21.15.44:0/739740280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 msgr2=0x7fd1e40b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 --2- 172.21.15.44:0/739740280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e40b3ce0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4005fd0 tx=0x7fd1d402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 -- 172.21.15.44:0/739740280 shutdown_connections 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 --2- 172.21.15.44:0/739740280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e40b3ce0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 -- 172.21.15.44:0/739740280 >> 172.21.15.44:0/739740280 conn(0x7fd1e40b0eb0 msgr2=0x7fd1e40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 -- 172.21.15.44:0/739740280 shutdown_connections 2024-07-17T08:36:27.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.893+0000 7fd1e9200640 1 -- 172.21.15.44:0/739740280 wait complete. 2024-07-17T08:36:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.895+0000 7fd1e9200640 1 Processor -- start 2024-07-17T08:36:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.895+0000 7fd1e9200640 1 -- start start 2024-07-17T08:36:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1e40ac190 0x7fd1e41f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 0x7fd1e41f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e415e9c0 con 0x7fd1e4154600 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e415dc40 con 0x7fd1e4161120 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e40ac5c0 con 0x7fd1e40ac190 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34006/0 (socket says 172.21.15.44:34006) 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.896+0000 7fd1e1400640 1 -- 172.21.15.44:0/671239101 learned_addr learned my addr 172.21.15.44:0/671239101 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e2800640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 0x7fd1e41f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 -- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1e40ac190 msgr2=0x7fd1e41f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1e40ac190 0x7fd1e41f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 -- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 msgr2=0x7fd1e41f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 0x7fd1e41f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:27.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 -- 172.21.15.44:0/671239101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e41f7ff0 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e2800640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 0x7fd1e41f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1e1400640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c730 tx=0x7fd1cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.897+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc003b70 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.898+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1cc003d10 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.898+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00aab0 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.898+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e41f82d0 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.898+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e41f45b0 con 0x7fd1e4154600 2024-07-17T08:36:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.899+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e40b3ce0 con 0x7fd1e4154600 2024-07-17T08:36:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.900+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fd1cc00ac50 con 0x7fd1e4154600 2024-07-17T08:36:27.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.901+0000 7fd1da000640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 0x7fd1c00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:27.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.901+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fd1cc0989f0 con 0x7fd1e4154600 2024-07-17T08:36:27.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.902+0000 7fd1e1e00640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 0x7fd1c00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.903+0000 7fd1e1e00640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 0x7fd1c00792b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4033000 tx=0x7fd1d40046c0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:27.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:27.906+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd1e41549e0 con 0x7fd1e4154600 2024-07-17T08:36:28.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.203+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd1e4160e90 con 0x7fd1e4154600 2024-07-17T08:36:28.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.208+0000 7fd1da000640 1 -- 172.21.15.44:0/671239101 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v73) ==== 190+0+147 (secure 0 0 0) 0x7fd1cc0616f0 con 0x7fd1e4154600 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:36:28.210 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 msgr2=0x7fd1c00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 0x7fd1c00792b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4033000 tx=0x7fd1d40046c0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 msgr2=0x7fd1e41573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c730 tx=0x7fd1cc00cbf0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 shutdown_connections 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd1c0076df0 0x7fd1c00792b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1e4161120 0x7fd1e41f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1e4154600 0x7fd1e41573c0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 --2- 172.21.15.44:0/671239101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1e40ac190 0x7fd1e41f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 >> 172.21.15.44:0/671239101 conn(0x7fd1e40b0eb0 msgr2=0x7fd1e40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:28.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 shutdown_connections 2024-07-17T08:36:28.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.211+0000 7fd1e9200640 1 -- 172.21.15.44:0/671239101 wait complete. 2024-07-17T08:36:28.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 -- 172.21.15.44:0/757596286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 msgr2=0x7f5a6415e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 --2- 172.21.15.44:0/757596286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a6415e470 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f5a54005fd0 tx=0x7f5a5402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 -- 172.21.15.44:0/757596286 shutdown_connections 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 --2- 172.21.15.44:0/757596286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406f370 0x7f5a6406dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 --2- 172.21.15.44:0/757596286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a6415e470 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 -- 172.21.15.44:0/757596286 >> 172.21.15.44:0/757596286 conn(0x7f5a640af940 msgr2=0x7f5a640afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 -- 172.21.15.44:0/757596286 shutdown_connections 2024-07-17T08:36:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.438+0000 7f5a69000640 1 -- 172.21.15.44:0/757596286 wait complete. 2024-07-17T08:36:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.439+0000 7f5a69000640 1 Processor -- start 2024-07-17T08:36:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.440+0000 7f5a69000640 1 -- start start 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406e400 0x7f5a6414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a6406f370 0x7f5a64153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a640aea30 con 0x7f5a640b58a0 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a640ae8b0 con 0x7f5a6406f370 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a69000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a640aebb0 con 0x7f5a6406e400 2024-07-17T08:36:28.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34026/0 (socket says 172.21.15.44:34026) 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 -- 172.21.15.44:0/82178361 learned_addr learned my addr 172.21.15.44:0/82178361 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a5be00640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a6406f370 0x7f5a64153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a61e00640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406e400 0x7f5a6414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 -- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406e400 msgr2=0x7f5a6414f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406e400 0x7f5a6414f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 -- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a6406f370 msgr2=0x7f5a64153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a6406f370 0x7f5a64153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.441+0000 7f5a62800640 1 -- 172.21.15.44:0/82178361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a641547c0 con 0x7f5a640b58a0 2024-07-17T08:36:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a62800640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c7c0 tx=0x7f5a5c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003b70 con 0x7f5a640b58a0 2024-07-17T08:36:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a5c003d30 con 0x7f5a640b58a0 2024-07-17T08:36:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00aab0 con 0x7f5a640b58a0 2024-07-17T08:36:28.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a640cc390 con 0x7f5a640b58a0 2024-07-17T08:36:28.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.442+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6414fc20 con 0x7f5a640b58a0 2024-07-17T08:36:28.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.443+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6406dec0 con 0x7f5a640b58a0 2024-07-17T08:36:28.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.445+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f5a5c00ac50 con 0x7f5a640b58a0 2024-07-17T08:36:28.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.446+0000 7f5a5a000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 0x7f5a400792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:28.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.446+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f5a5c0989f0 con 0x7f5a640b58a0 2024-07-17T08:36:28.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.446+0000 7f5a61e00640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 0x7f5a400792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:28.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.447+0000 7f5a61e00640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 0x7f5a400792b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a54033000 tx=0x7f5a54004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:28.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.450+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5a5c061780 con 0x7f5a640b58a0 2024-07-17T08:36:28.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.702+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a640b2df0 con 0x7f5a640b58a0 2024-07-17T08:36:28.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.703+0000 7f5a5a000640 1 -- 172.21.15.44:0/82178361 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) ==== 95+0+5236 (secure 0 0 0) 0x7f5a5c061080 con 0x7f5a640b58a0 2024-07-17T08:36:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:28.705 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":130,"btime":"2024-07-17T08:36:25:625513+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":8785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15}]} 2024-07-17T08:36:28.706 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 130 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.706+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 msgr2=0x7f5a400792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.706+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 0x7f5a400792b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a54033000 tx=0x7f5a54004730 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.706+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 msgr2=0x7f5a64156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.706+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c7c0 tx=0x7f5a5c00cc80 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 shutdown_connections 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f5a40076df0 0x7f5a400792b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a640b58a0 0x7f5a64156a90 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a6406f370 0x7f5a64153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 --2- 172.21.15.44:0/82178361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a6406e400 0x7f5a6414f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:28.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 >> 172.21.15.44:0/82178361 conn(0x7f5a640af940 msgr2=0x7f5a6415daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 shutdown_connections 2024-07-17T08:36:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:28.707+0000 7f5a69000640 1 -- 172.21.15.44:0/82178361 wait complete. 2024-07-17T08:36:29.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:36:22.075389+0000', 'modified': '2024-07-17T08:36:25.120303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8788}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8788': {'gid': 8788, 'name': 'd', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6836/1736454513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 1736454513}, {'type': 'v1', 'addr': '172.21.15.175:6836', 'nonce': 1736454513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8788, 'qdb_cluster': [8788]} 2024-07-17T08:36:29.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:36:29.130 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 -- 172.21.15.44:0/2044026242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f40b3280 msgr2=0x7fc3f406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 --2- 172.21.15.44:0/2044026242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f40b3280 0x7fc3f406eb90 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0005fd0 tx=0x7fc3e002a530 comp rx=0 tx=0).stop 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 -- 172.21.15.44:0/2044026242 shutdown_connections 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 --2- 172.21.15.44:0/2044026242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f40b3280 0x7fc3f406eb90 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 --2- 172.21.15.44:0/2044026242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f406f6d0 0x7fc3f415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.353+0000 7fc3fba00640 1 -- 172.21.15.44:0/2044026242 >> 172.21.15.44:0/2044026242 conn(0x7fc3f40b0eb0 msgr2=0x7fc3f40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.354+0000 7fc3fba00640 1 -- 172.21.15.44:0/2044026242 shutdown_connections 2024-07-17T08:36:29.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.354+0000 7fc3fba00640 1 -- 172.21.15.44:0/2044026242 wait complete. 2024-07-17T08:36:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.355+0000 7fc3fba00640 1 Processor -- start 2024-07-17T08:36:29.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.355+0000 7fc3fba00640 1 -- start start 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 0x7fc3f414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 0x7fc3f41f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f40ac810 con 0x7fc3f406f6d0 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4150810 con 0x7fc3f40ac3a0 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.356+0000 7fc3fba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f40ac990 con 0x7fc3f40b3280 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52456/0 (socket says 172.21.15.44:52456) 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 -- 172.21.15.44:0/1776899109 learned_addr learned my addr 172.21.15.44:0/1776899109 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 0x7fc3f414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 0x7fc3f41f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 -- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 msgr2=0x7fc3f41f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 0x7fc3f41f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 -- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 msgr2=0x7fc3f414f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 0x7fc3f414f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 -- 172.21.15.44:0/1776899109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f4158350 con 0x7fc3f406f6d0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 0x7fc3f414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 0x7fc3f41f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.357+0000 7fc3f9000640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800dbc0 tx=0x7fc3e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.358+0000 7fc3f0c00640 1 -- 172.21.15.44:0/1776899109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005020 con 0x7fc3f406f6d0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.358+0000 7fc3f0c00640 1 -- 172.21.15.44:0/1776899109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3e80052b0 con 0x7fc3f406f6d0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.358+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f41585f0 con 0x7fc3f406f6d0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.358+0000 7fc3f0c00640 1 -- 172.21.15.44:0/1776899109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80047a0 con 0x7fc3f406f6d0 2024-07-17T08:36:29.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.358+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f40ae9a0 con 0x7fc3f406f6d0 2024-07-17T08:36:29.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.360+0000 7fc3f0c00640 1 -- 172.21.15.44:0/1776899109 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fc3e8004940 con 0x7fc3f406f6d0 2024-07-17T08:36:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.361+0000 7fc3f0c00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 0x7fc3d40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.362+0000 7fc3f0c00640 1 -- 172.21.15.44:0/1776899109 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fc3e8099240 con 0x7fc3f406f6d0 2024-07-17T08:36:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.362+0000 7fc3fe82c740 1 Processor -- start 2024-07-17T08:36:29.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.362+0000 7fc3fe82c740 1 -- start start 2024-07-17T08:36:29.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.362+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 0x7fc3d40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cbf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbf2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e50cb3dd60 con 0x55e50cbcd0a0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e50cb2d260 con 0x55e50cbf25b0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e50cb312a0 con 0x55e50cbf5de0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52464/0 (socket says 172.21.15.44:52464) 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3f3e00640 1 -- 172.21.15.44:0/1667325729 learned_addr learned my addr 172.21.15.44:0/1667325729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbf2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.363+0000 7fc3f9000640 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cbf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 -- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 msgr2=0x55e50cbf61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cbf61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 -- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 msgr2=0x55e50cbf2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbf2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 -- 172.21.15.44:0/1667325729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50c891850 con 0x55e50cbcd0a0 2024-07-17T08:36:29.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0034040 tx=0x7fc3e0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805348b7364af912 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 0x7fc3d40792b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0000c00 tx=0x7fc3e0004420 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3e5600640 1 -- 172.21.15.44:0/1667325729 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0045070 con 0x55e50cbcd0a0 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3e5600640 1 -- 172.21.15.44:0/1667325729 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3e004d070 con 0x55e50cbcd0a0 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.364+0000 7fc3e5600640 1 -- 172.21.15.44:0/1667325729 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0035040 con 0x55e50cbcd0a0 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 -- 172.21.15.44:0/1667325729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 msgr2=0x55e50cbf2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0034040 tx=0x7fc3e0041000 comp rx=0 tx=0).stop 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 -- 172.21.15.44:0/1667325729 shutdown_connections 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbf2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbcd0a0 0x55e50cbf2070 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/1667325729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cbf61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 -- 172.21.15.44:0/1667325729 >> 172.21.15.44:0/1667325729 conn(0x55e50cbc8280 msgr2=0x55e50cbcbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 -- 172.21.15.44:0/1667325729 shutdown_connections 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.365+0000 7fc3fe82c740 1 -- 172.21.15.44:0/1667325729 wait complete. 2024-07-17T08:36:29.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.366+0000 7fc3fe82c740 1 -- start start 2024-07-17T08:36:29.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.366+0000 7fc3fe82c740 1 Processor -- start 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbdd450 0x55e50cc09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e50c8a0620 con 0x55e50cbf25b0 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e50cad4f90 con 0x55e50cbdd450 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e50cbc6980 con 0x55e50cbf5de0 2024-07-17T08:36:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.367+0000 7fc3fe82c740 10 client.0 _ll_register_callbacks cb 0x55e50cafaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52470/0 (socket says 172.21.15.44:52470) 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 -- 172.21.15.44:0/710889723 learned_addr learned my addr 172.21.15.44:0/710889723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbdd450 0x55e50cc09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 -- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 msgr2=0x55e50cc070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 -- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbdd450 msgr2=0x55e50cc09f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbdd450 0x55e50cc09f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 -- 172.21.15.44:0/710889723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50c9086b0 con 0x55e50cbf25b0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3f9000640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fc3e805eab0 tx=0x7fc3e8061de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191113827bebc696 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.368+0000 7fc3da000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8061350 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fc3e80680c0 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e50cb04900 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) ==== 24+0+0 (secure 0 0 0) 0x7fc3e8061f40 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 10 client.10845 fetch_fsmap learned FSMap version 130 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) -- 0x55e50cbf1f80 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e50cb04900 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 130) ==== 3502+0+0 (secure 0 0 0) 0x7fc3e800b040 con 0x55e50cbf25b0 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 10 client.10845 fetch_fsmap finished waiting for FSMap version 130 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 10 client.10845 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:36:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 20 client.10845 populate_metadata read hostname 'smithi044' 2024-07-17T08:36:29.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbf4df0 0x55e50cbf1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 4 client.10845 mds_command: new command op to 8788 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:36:29.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.369+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 --> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e50cb3fcb0 con 0x55e50cbf4df0 2024-07-17T08:36:29.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.370+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbf4df0 0x55e50cbf1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.371+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbf4df0 0x55e50cbf1580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9be0ef8129c9b1d server_cookie=34f06b265454a7f in_seq=0 out_seq=0 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.371+0000 7fc3da000640 10 client.10845 ms_handle_connect on v2:172.21.15.175:6835/1736454513 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.373+0000 7fc3da000640 1 -- 172.21.15.44:0/710889723 <== mds.0 v2:172.21.15.175:6835/1736454513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e50cb3fcb0 con 0x55e50cbf4df0 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.373+0000 7fc3da000640 10 client.10845 handle_command_reply: tid=0 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.373+0000 7fc3fe82c740 1 client.10845 shutdown 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 msgr2=0x55e50cc241a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fc3e805eab0 tx=0x7fc3e8061de0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 shutdown_connections 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbf4df0 0x55e50cbf1580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbdd450 0x55e50cc09f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/710889723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf25b0 0x55e50cc241a0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 >> 172.21.15.44:0/710889723 conn(0x55e50cbcd0a0 msgr2=0x55e50cbf61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3da000640 0 client.10845 ms_handle_reset on v2:172.21.15.175:6835/1736454513 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 shutdown_connections 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 1 -- 172.21.15.44:0/710889723 wait complete. 2024-07-17T08:36:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.374+0000 7fc3fe82c740 20 client.10845 trim_cache size 0 max 16384 2024-07-17T08:36:29.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.396+0000 7fc3fe82c740 1 Processor -- start 2024-07-17T08:36:29.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.396+0000 7fc3fe82c740 1 -- start start 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbdd450 0x55e50cbc98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbc5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f42004b0 con 0x55e50cbf5de0 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41ff810 con 0x55e50cbf25b0 2024-07-17T08:36:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.397+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f41ffe60 con 0x55e50cbdd450 2024-07-17T08:36:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52478/0 (socket says 172.21.15.44:52478) 2024-07-17T08:36:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 -- 172.21.15.44:0/3610655062 learned_addr learned my addr 172.21.15.44:0/3610655062 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9000640 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbdd450 0x55e50cbc98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 -- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbdd450 msgr2=0x55e50cbc98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbdd450 0x55e50cbc98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 -- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 msgr2=0x55e50cbc5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.398+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbc5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.399+0000 7fc3f9a00640 1 -- 172.21.15.44:0/3610655062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50cbcb9a0 con 0x55e50cbf5de0 2024-07-17T08:36:29.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.399+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0054a0 tx=0x7fc3ec005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b63e7aa2a6da4f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.400+0000 7fc3e4c00640 1 -- 172.21.15.44:0/3610655062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec019070 con 0x55e50cbf5de0 2024-07-17T08:36:29.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.400+0000 7fc3e4c00640 1 -- 172.21.15.44:0/3610655062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3ec011d10 con 0x55e50cbf5de0 2024-07-17T08:36:29.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.400+0000 7fc3e4c00640 1 -- 172.21.15.44:0/3610655062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec015690 con 0x55e50cbf5de0 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.400+0000 7fc3fe82c740 1 -- 172.21.15.44:0/3610655062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 msgr2=0x55e50cbc56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0054a0 tx=0x7fc3ec005960 comp rx=0 tx=0).stop 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 -- 172.21.15.44:0/3610655062 shutdown_connections 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbf5de0 0x55e50cbc56d0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cbc5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/3610655062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbdd450 0x55e50cbc98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 -- 172.21.15.44:0/3610655062 >> 172.21.15.44:0/3610655062 conn(0x55e50cbcd0a0 msgr2=0x55e50cc23210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 -- 172.21.15.44:0/3610655062 shutdown_connections 2024-07-17T08:36:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 -- 172.21.15.44:0/3610655062 wait complete. 2024-07-17T08:36:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.401+0000 7fc3fe82c740 1 -- start start 2024-07-17T08:36:29.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.402+0000 7fc3fe82c740 1 Processor -- start 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cc85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e50cbf0700 con 0x55e50cbdd450 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e50cc24b10 con 0x55e50cbf25b0 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e50cbf5220 con 0x55e50cbf5de0 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3fe82c740 10 client.0 _ll_register_callbacks cb 0x7fc3e805de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52482/0 (socket says 172.21.15.44:52482) 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 -- 172.21.15.44:0/2181965755 learned_addr learned my addr 172.21.15.44:0/2181965755 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cc85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9a00640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 -- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 msgr2=0x55e50cc33610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc33610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 -- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 msgr2=0x55e50cc85d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.403+0000 7fc3f9000640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cc85d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3f9000640 1 -- 172.21.15.44:0/2181965755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50cbf23e0 con 0x55e50cbdd450 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cc85d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3f9000640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800ab20 tx=0x7fc3e800acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f985410f50d5190 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3da000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8060fb0 con 0x55e50cbdd450 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7fc3e8062d80 con 0x55e50cbdd450 2024-07-17T08:36:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ec011d10 con 0x55e50cbdd450 2024-07-17T08:36:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) ==== 24+0+0 (secure 0 0 0) 0x7fc3e8096f40 con 0x55e50cbdd450 2024-07-17T08:36:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3fe82c740 10 client.10851 fetch_fsmap learned FSMap version 130 2024-07-17T08:36:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.404+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) -- 0x55e50cc25420 con 0x55e50cbdd450 2024-07-17T08:36:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3ec011d10 con 0x55e50cbdd450 2024-07-17T08:36:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 130) ==== 3502+0+0 (secure 0 0 0) 0x7fc3e80680b0 con 0x55e50cbdd450 2024-07-17T08:36:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 10 client.10851 fetch_fsmap finished waiting for FSMap version 130 2024-07-17T08:36:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 10 client.10851 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:36:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 20 client.10851 populate_metadata read hostname 'smithi044' 2024-07-17T08:36:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbc67c0 0x55e50cbc6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 4 client.10851 mds_command: new command op to 8788 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:36:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.405+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 --> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] -- command(tid 0: {"prefix": "status"}) -- 0x55e50cc246f0 con 0x55e50cbc67c0 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.406+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbc67c0 0x55e50cbc6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.406+0000 7fc3f3e00640 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbc67c0 0x55e50cbc6ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675988a830d12ad1 server_cookie=22961fc6d1a869a7 in_seq=0 out_seq=0 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.406+0000 7fc3da000640 10 client.10851 ms_handle_connect on v2:172.21.15.175:6835/1736454513 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.407+0000 7fc3da000640 1 -- 172.21.15.44:0/2181965755 <== mds.0 v2:172.21.15.175:6835/1736454513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55e50cc246f0 con 0x55e50cbc67c0 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.407+0000 7fc3da000640 10 client.10851 handle_command_reply: tid=0 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.407+0000 7fc3fe82c740 1 client.10851 shutdown 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 msgr2=0x55e50cc349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800ab20 tx=0x7fc3e800acc0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 shutdown_connections 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e50cbf5de0 0x55e50cc33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:6835/1736454513,v1:172.21.15.175:6836/1736454513] conn(0x55e50cbc67c0 0x55e50cbc6ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e50cbf25b0 0x55e50cc85d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 --2- 172.21.15.44:0/2181965755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e50cbdd450 0x55e50cc349c0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 >> 172.21.15.44:0/2181965755 conn(0x55e50cbcd0a0 msgr2=0x55e50cbf13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3da000640 0 client.10851 ms_handle_reset on v2:172.21.15.175:6835/1736454513 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 shutdown_connections 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 1 -- 172.21.15.44:0/2181965755 wait complete. 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.408+0000 7fc3fe82c740 20 client.10851 trim_cache size 0 max 16384 2024-07-17T08:36:29.415 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8788, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7901431900000002, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 130, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 243, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 243, 2024-07-17T08:36:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.757008771000001 2024-07-17T08:36:29.417 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:36:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 msgr2=0x7fc3d40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 0x7fc3d40792b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0000c00 tx=0x7fc3e0004420 comp rx=0 tx=0).stop 2024-07-17T08:36:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 msgr2=0x7fc3f41f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800dbc0 tx=0x7fc3e800a620 comp rx=0 tx=0).stop 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 shutdown_connections 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc3d4076df0 0x7fc3d40792b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3f40b3280 0x7fc3f41f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3f40ac3a0 0x7fc3f414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 --2- 172.21.15.44:0/1776899109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3f406f6d0 0x7fc3f41f47c0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 >> 172.21.15.44:0/1776899109 conn(0x7fc3f40b0eb0 msgr2=0x7fc3f415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 shutdown_connections 2024-07-17T08:36:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.409+0000 7fc3fba00640 1 -- 172.21.15.44:0/1776899109 wait complete. 2024-07-17T08:36:29.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.633+0000 7f2aa0e00640 1 -- 172.21.15.44:0/2169482766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80005910 msgr2=0x7f2a80005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.633+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/2169482766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a80005910 0x7f2a80005d90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c290 tx=0x7f2a80006940 comp rx=0 tx=0).stop 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.633+0000 7f2a9be00640 1 -- 172.21.15.44:0/2169482766 reap_dead start 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.634+0000 7f2aa0e00640 1 -- 172.21.15.44:0/2169482766 shutdown_connections 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.634+0000 7f2aa0e00640 1 -- 172.21.15.44:0/2169482766 >> 172.21.15.44:0/2169482766 conn(0x7f2a9c0b0e80 msgr2=0x7f2a9c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.634+0000 7f2aa0e00640 1 -- 172.21.15.44:0/2169482766 shutdown_connections 2024-07-17T08:36:29.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.634+0000 7f2aa0e00640 1 -- 172.21.15.44:0/2169482766 wait complete. 2024-07-17T08:36:29.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.635+0000 7f2aa0e00640 1 Processor -- start 2024-07-17T08:36:29.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.636+0000 7f2aa0e00640 1 -- start start 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a80005910 0x7f2a9c1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a9c0acfb0 0x7f2a9c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c09b690 con 0x7f2a9c15b290 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c09b510 con 0x7f2a80005910 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2aa0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c09b810 con 0x7f2a9c0acfb0 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52508/0 (socket says 172.21.15.44:52508) 2024-07-17T08:36:29.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 -- 172.21.15.44:0/3280193637 learned_addr learned my addr 172.21.15.44:0/3280193637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:29.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9be00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a9c0acfb0 0x7f2a9c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 -- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a9c0acfb0 msgr2=0x7f2a9c157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a9c0acfb0 0x7f2a9c157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 -- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a80005910 msgr2=0x7f2a9c1f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a80005910 0x7f2a9c1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.637+0000 7f2a9aa00640 1 -- 172.21.15.44:0/3280193637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0cc4d0 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2a9aa00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2a8802b110 tx=0x7f2a88001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88031040 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a88003800 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c0cc7b0 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88006600 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.638+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c0ccd70 con 0x7f2a9c15b290 2024-07-17T08:36:29.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.639+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c06e060 con 0x7f2a9c15b290 2024-07-17T08:36:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.641+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f2a88034070 con 0x7f2a9c15b290 2024-07-17T08:36:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.642+0000 7f2a93400640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 0x7f2a7c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.642+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f2a880b8d50 con 0x7f2a9c15b290 2024-07-17T08:36:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.642+0000 7f2a9b400640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 0x7f2a7c0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.643+0000 7f2a9b400640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 0x7f2a7c0792e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c005d00 tx=0x7f2a8c009530 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:29.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.646+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2a88081a50 con 0x7f2a9c15b290 2024-07-17T08:36:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.905+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2a9c0ad390 con 0x7f2a9c15b290 2024-07-17T08:36:29.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.906+0000 7f2a93400640 1 -- 172.21.15.44:0/3280193637 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v130) ==== 53+0+73 (secure 0 0 0) 0x7f2a88081350 con 0x7f2a9c15b290 2024-07-17T08:36:29.908 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:36:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 msgr2=0x7f2a7c0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 0x7f2a7c0792e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c005d00 tx=0x7f2a8c009530 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 msgr2=0x7f2a9c0aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2a8802b110 tx=0x7f2a88001dd0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 shutdown_connections 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2a7c076e20 0x7f2a7c0792e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a9c0acfb0 0x7f2a9c157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a9c15b290 0x7f2a9c0aca70 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 --2- 172.21.15.44:0/3280193637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a80005910 0x7f2a9c1f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.909+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 >> 172.21.15.44:0/3280193637 conn(0x7f2a9c0b0e80 msgr2=0x7f2a9c0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.910+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 shutdown_connections 2024-07-17T08:36:29.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:29.910+0000 7f2aa0e00640 1 -- 172.21.15.44:0/3280193637 wait complete. 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:36:29.932 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:36:29.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:29.933 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:36:29.981 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36468sec preferred_lft 36468sec 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:36:29.982 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:36:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:36:29.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:29.984 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:36:30.058 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:36:30.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:30.062 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:36:30.115 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:36:30.115 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:36:30.116 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:36:30.116 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:36:30.116 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:36:30.117 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:36:30.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:36:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:36:30.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:30.337 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:36:30.384 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:36:30.384 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:36:30.385 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:36:30.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:30.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:36:30.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:36:30.519 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:36:30.574 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:36:30.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:36:30.574 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:36:30.629 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:36:30.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:30.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:36:30.758 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:36:30.758 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:36:30.758 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:36:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:36:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:36:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:36:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:36:30.781 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:36:30.808 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:36:30.808 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:36:30.809 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:36:30.811 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:36:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:36:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:36:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:36:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:36:30.817 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:36:30.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:30.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:36:30.899 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:36:31.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:36:31.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:36:31.047 INFO:teuthology.orchestra.run.smithi044.stdout:client8081 2024-07-17T08:36:31.047 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:31.047 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:31.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-17T08:36:31.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.320+0000 7ff407200640 1 -- 172.21.15.44:0/1073109507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e8003a50 msgr2=0x7ff3e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.320+0000 7ff407200640 1 --2- 172.21.15.44:0/1073109507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3e8003a50 0x7ff3e8003e70 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009b50 tx=0x7ff3e8005830 comp rx=0 tx=0).stop 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.320+0000 7ff406800640 1 -- 172.21.15.44:0/1073109507 reap_dead start 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.320+0000 7ff407200640 1 -- 172.21.15.44:0/1073109507 shutdown_connections 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.320+0000 7ff407200640 1 -- 172.21.15.44:0/1073109507 >> 172.21.15.44:0/1073109507 conn(0x7ff4000cf050 msgr2=0x7ff4000cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.321+0000 7ff407200640 1 -- 172.21.15.44:0/1073109507 shutdown_connections 2024-07-17T08:36:31.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.321+0000 7ff407200640 1 -- 172.21.15.44:0/1073109507 wait complete. 2024-07-17T08:36:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.322+0000 7ff407200640 1 Processor -- start 2024-07-17T08:36:31.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.322+0000 7ff407200640 1 -- start start 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 0x7ff40014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 0x7ff40015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff400072d40 con 0x7ff4000b39f0 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff400072bc0 con 0x7ff3e8003a50 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.323+0000 7ff407200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff400072ec0 con 0x7ff4000b3e30 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52518/0 (socket says 172.21.15.44:52518) 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 -- 172.21.15.44:0/338126661 learned_addr learned my addr 172.21.15.44:0/338126661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff406800640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 0x7ff40015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 -- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 msgr2=0x7ff40015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405e00640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 0x7ff40014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 0x7ff40015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 -- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 msgr2=0x7ff40014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 0x7ff40014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 -- 172.21.15.44:0/338126661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400153e30 con 0x7ff4000b39f0 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff406800640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 0x7ff40015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:31.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405e00640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 0x7ff40014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.324+0000 7ff405400640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d7a0 tx=0x7ff3f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.325+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0018070 con 0x7ff4000b39f0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.325+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3f000de50 con 0x7ff4000b39f0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.325+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000b6c0 con 0x7ff4000b39f0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.325+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4000b20c0 con 0x7ff4000b39f0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.325+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4000b2660 con 0x7ff4000b39f0 2024-07-17T08:36:31.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.326+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4000aee80 con 0x7ff4000b39f0 2024-07-17T08:36:31.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.327+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7ff3f000b860 con 0x7ff4000b39f0 2024-07-17T08:36:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.329+0000 7ff3fa000640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 0x7ff3e40792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.329+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7ff3f0098180 con 0x7ff4000b39f0 2024-07-17T08:36:31.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.330+0000 7ff405e00640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 0x7ff3e40792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.331+0000 7ff405e00640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 0x7ff3e40792e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4035000 tx=0x7ff3f402d040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.333+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4000aee80 con 0x7ff4000b39f0 2024-07-17T08:36:31.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.584+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff4000ad250 con 0x7ff4000b39f0 2024-07-17T08:36:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.679+0000 7ff3fa000640 1 -- 172.21.15.44:0/338126661 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v131) ==== 136+0+0 (secure 0 0 0) 0x7ff3f00608c0 con 0x7ff4000b39f0 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 msgr2=0x7ff3e40792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 0x7ff3e40792e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4035000 tx=0x7ff3f402d040 comp rx=0 tx=0).stop 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 msgr2=0x7ff40015d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d7a0 tx=0x7ff3f000dc60 comp rx=0 tx=0).stop 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 shutdown_connections 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7ff3e4076e20 0x7ff3e40792e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4000b3e30 0x7ff40015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4000b39f0 0x7ff40015d080 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 --2- 172.21.15.44:0/338126661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3e8003a50 0x7ff40014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 >> 172.21.15.44:0/338126661 conn(0x7ff4000cf050 msgr2=0x7ff4000af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.682+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 shutdown_connections 2024-07-17T08:36:31.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.683+0000 7ff407200640 1 -- 172.21.15.44:0/338126661 wait complete. 2024-07-17T08:36:31.709 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:36:31.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.922+0000 7f7ba7a00640 1 -- 172.21.15.44:0/2337114900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba0153e40 msgr2=0x7f7ba01542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.922+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/2337114900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba0153e40 0x7f7ba01542f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7b98005fd0 tx=0x7f7b9802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 -- 172.21.15.44:0/2337114900 shutdown_connections 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/2337114900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba0153e40 0x7f7ba01542f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/2337114900 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba006f020 0x7f7ba006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 -- 172.21.15.44:0/2337114900 >> 172.21.15.44:0/2337114900 conn(0x7f7ba00b0fb0 msgr2=0x7f7ba00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 -- 172.21.15.44:0/2337114900 shutdown_connections 2024-07-17T08:36:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.923+0000 7f7ba7a00640 1 -- 172.21.15.44:0/2337114900 wait complete. 2024-07-17T08:36:31.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.924+0000 7f7ba7a00640 1 Processor -- start 2024-07-17T08:36:31.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.925+0000 7f7ba7a00640 1 -- start start 2024-07-17T08:36:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.925+0000 7f7ba7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba006f020 0x7f7ba01f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.925+0000 7f7ba7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba0153e40 0x7f7ba00b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0154830 con 0x7f7ba015b2e0 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba015b8d0 con 0x7f7ba006f020 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba01549b0 con 0x7f7ba0153e40 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52522/0 (socket says 172.21.15.44:52522) 2024-07-17T08:36:31.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 -- 172.21.15.44:0/3146621130 learned_addr learned my addr 172.21.15.44:0/3146621130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5000640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba006f020 0x7f7ba01f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7b97e00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba0153e40 0x7f7ba00b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 -- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba0153e40 msgr2=0x7f7ba00b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba0153e40 0x7f7ba00b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 -- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba006f020 msgr2=0x7f7ba01f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba006f020 0x7f7ba01f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:31.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.926+0000 7f7ba5a00640 1 -- 172.21.15.44:0/3146621130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba01f29e0 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7ba5a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006010 tx=0x7f7b98031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98040070 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b98004040 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba0071750 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98004450 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.927+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba0071d10 con 0x7f7ba015b2e0 2024-07-17T08:36:31.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.929+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba006f480 con 0x7f7ba015b2e0 2024-07-17T08:36:31.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.930+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f7b98039040 con 0x7f7ba015b2e0 2024-07-17T08:36:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.931+0000 7f7b9d600640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 0x7f7b7c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.931+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7f7b980ba550 con 0x7f7ba015b2e0 2024-07-17T08:36:31.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.932+0000 7f7ba5000640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 0x7f7b7c0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:31.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.933+0000 7f7ba5000640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 0x7f7b7c0792b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00731e0 tx=0x7f7b88009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:31.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:31.935+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7ba01f29e0 con 0x7f7ba015b2e0 2024-07-17T08:36:32.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.149+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7ba006dbf0 con 0x7f7ba015b2e0 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.150+0000 7f7b9d600640 1 -- 172.21.15.44:0/3146621130 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v243) ==== 84+0+209 (secure 0 0 0) 0x7f7b980832f0 con 0x7f7ba015b2e0 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:36:32.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 31, 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "cephfs_metadata" 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 32, 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "cephfs_data" 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:32.153 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:36:32.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.153+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 msgr2=0x7f7b7c0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.153+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 0x7f7b7c0792b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00731e0 tx=0x7f7b88009040 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.153+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 msgr2=0x7f7ba0151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.153+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006010 tx=0x7f7b98031670 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 shutdown_connections 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7b7c076df0 0x7f7b7c0792b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba015b2e0 0x7f7ba0151040 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ba0153e40 0x7f7ba00b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 --2- 172.21.15.44:0/3146621130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ba006f020 0x7f7ba01f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 >> 172.21.15.44:0/3146621130 conn(0x7f7ba00b0fb0 msgr2=0x7f7ba006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 shutdown_connections 2024-07-17T08:36:32.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.154+0000 7f7ba7a00640 1 -- 172.21.15.44:0/3146621130 wait complete. 2024-07-17T08:36:32.178 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-17T08:36:32.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.392+0000 7faf4e200640 1 -- 172.21.15.44:0/2033481322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815e1d0 msgr2=0x7faf480b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.392+0000 7faf4e200640 1 --2- 172.21.15.44:0/2033481322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815e1d0 0x7faf480b4f20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faf40005fd0 tx=0x7faf4002a820 comp rx=0 tx=0).stop 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 -- 172.21.15.44:0/2033481322 shutdown_connections 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 --2- 172.21.15.44:0/2033481322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815e1d0 0x7faf480b4f20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 --2- 172.21.15.44:0/2033481322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d740 0x7faf480ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 -- 172.21.15.44:0/2033481322 >> 172.21.15.44:0/2033481322 conn(0x7faf480d0100 msgr2=0x7faf480d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 -- 172.21.15.44:0/2033481322 shutdown_connections 2024-07-17T08:36:32.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.393+0000 7faf4e200640 1 -- 172.21.15.44:0/2033481322 wait complete. 2024-07-17T08:36:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.395+0000 7faf4e200640 1 Processor -- start 2024-07-17T08:36:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.396+0000 7faf4e200640 1 -- start start 2024-07-17T08:36:32.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d360 0x7faf480c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 0x7faf481f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf480d1700 con 0x7faf4815e1d0 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf480d1580 con 0x7faf4815d740 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf480d1880 con 0x7faf4815d360 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52526/0 (socket says 172.21.15.44:52526) 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.397+0000 7faf4d800640 1 -- 172.21.15.44:0/1983229822 learned_addr learned my addr 172.21.15.44:0/1983229822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf47e00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 0x7faf481f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4ce00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d360 0x7faf480c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 -- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d360 msgr2=0x7faf480c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d360 0x7faf480c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 -- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 msgr2=0x7faf481f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 0x7faf481f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 -- 172.21.15.44:0/1983229822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480c8ac0 con 0x7faf4815e1d0 2024-07-17T08:36:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf47e00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 0x7faf481f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:32.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf4d800640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7faf40006010 tx=0x7faf40032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.398+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037040 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.399+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf40042070 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.399+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400042a0 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.399+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf480c8da0 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.399+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf480c9360 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.400+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf480d1da0 con 0x7faf4815e1d0 2024-07-17T08:36:32.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.401+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7faf40004440 con 0x7faf4815e1d0 2024-07-17T08:36:32.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.403+0000 7faf44c00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 0x7faf280792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.403+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 6677+0+0 (secure 0 0 0) 0x7faf400b9910 con 0x7faf4815e1d0 2024-07-17T08:36:32.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.403+0000 7faf4ce00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 0x7faf280792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.404+0000 7faf4ce00640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 0x7faf280792b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf3c001680 tx=0x7faf3c009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:32.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.407+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faf40082480 con 0x7faf4815e1d0 2024-07-17T08:36:32.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.622+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7faf480b9420 con 0x7faf4815e1d0 2024-07-17T08:36:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.699+0000 7faf44c00640 1 -- 172.21.15.44:0/1983229822 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v244) ==== 144+0+0 (secure 0 0 0) 0x7faf40081d80 con 0x7faf4815e1d0 2024-07-17T08:36:32.701 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_metadata' created 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 msgr2=0x7faf280792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 0x7faf280792b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf3c001680 tx=0x7faf3c009040 comp rx=0 tx=0).stop 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 msgr2=0x7faf481f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7faf40006010 tx=0x7faf40032c90 comp rx=0 tx=0).stop 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 shutdown_connections 2024-07-17T08:36:32.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7faf28076df0 0x7faf280792b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf4815e1d0 0x7faf481f7890 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf4815d740 0x7faf481f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 --2- 172.21.15.44:0/1983229822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf4815d360 0x7faf480c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.702+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 >> 172.21.15.44:0/1983229822 conn(0x7faf480d0100 msgr2=0x7faf480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.703+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 shutdown_connections 2024-07-17T08:36:32.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.703+0000 7faf4e200640 1 -- 172.21.15.44:0/1983229822 wait complete. 2024-07-17T08:36:32.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.922+0000 7f7146e00640 1 -- 172.21.15.44:0/1029028949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7124003fd0 msgr2=0x7f7124004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.922+0000 7f7146e00640 1 --2- 172.21.15.44:0/1029028949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7124003fd0 0x7f7124004800 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c4d0 tx=0x7f7124005830 comp rx=0 tx=0).stop 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.922+0000 7f7144e00640 1 -- 172.21.15.44:0/1029028949 reap_dead start 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.923+0000 7f7146e00640 1 -- 172.21.15.44:0/1029028949 shutdown_connections 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.923+0000 7f7146e00640 1 -- 172.21.15.44:0/1029028949 >> 172.21.15.44:0/1029028949 conn(0x7f71400b0eb0 msgr2=0x7f71400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.923+0000 7f7146e00640 1 -- 172.21.15.44:0/1029028949 shutdown_connections 2024-07-17T08:36:32.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.923+0000 7f7146e00640 1 -- 172.21.15.44:0/1029028949 wait complete. 2024-07-17T08:36:32.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.925+0000 7f7146e00640 1 Processor -- start 2024-07-17T08:36:32.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.925+0000 7f7146e00640 1 -- start start 2024-07-17T08:36:32.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7124003fd0 0x7f71401f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714014adf0 0x7f7140072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f714006d9b0 con 0x7f7140149970 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f714006d830 con 0x7f714014adf0 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7146e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f714006db30 con 0x7f7124003fd0 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f713f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f713f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52530/0 (socket says 172.21.15.44:52530) 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f713f400640 1 -- 172.21.15.44:0/1247629676 learned_addr learned my addr 172.21.15.44:0/1247629676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f713fe00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7124003fd0 0x7f71401f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.926+0000 7f7144e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714014adf0 0x7f7140072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 -- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7124003fd0 msgr2=0x7f71401f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7124003fd0 0x7f71401f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 -- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714014adf0 msgr2=0x7f7140072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714014adf0 0x7f7140072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 -- 172.21.15.44:0/1247629676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714014b290 con 0x7f7140149970 2024-07-17T08:36:32.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f713f400640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c5f0 tx=0x7f712c006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:32.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c02b660 con 0x7f7140149970 2024-07-17T08:36:32.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f712c02b800 con 0x7f7140149970 2024-07-17T08:36:32.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c02bbf0 con 0x7f7140149970 2024-07-17T08:36:32.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714014b570 con 0x7f7140149970 2024-07-17T08:36:32.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.927+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71401f3910 con 0x7f7140149970 2024-07-17T08:36:32.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.929+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714006deb0 con 0x7f7140149970 2024-07-17T08:36:32.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.931+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f712c034470 con 0x7f7140149970 2024-07-17T08:36:32.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.931+0000 7f7137e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 0x7f71200792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:32.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.932+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(244..244 src has 1..244) ==== 7050+0+0 (secure 0 0 0) 0x7f712c0b8ec0 con 0x7f7140149970 2024-07-17T08:36:32.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.932+0000 7f713fe00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 0x7f71200792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:32.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.933+0000 7f713fe00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 0x7f71200792e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7130009b20 tx=0x7f7130005f90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:32.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:32.935+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f714006e800 con 0x7f7140149970 2024-07-17T08:36:33.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.148+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f71400acff0 con 0x7f7140149970 2024-07-17T08:36:33.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.717+0000 7f7137e00640 1 -- 172.21.15.44:0/1247629676 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v245) ==== 176+0+0 (secure 0 0 0) 0x7f712c081b60 con 0x7f7140149970 2024-07-17T08:36:33.719 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_data' created 2024-07-17T08:36:33.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.721+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 msgr2=0x7f71200792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:33.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.721+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 0x7f71200792e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7130009b20 tx=0x7f7130005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:33.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.721+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 msgr2=0x7f714014a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:33.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.721+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c5f0 tx=0x7f712c006060 comp rx=0 tx=0).stop 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 shutdown_connections 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7120076e20 0x7f71200792e0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7130009b20 tx=0x7f7130005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714014adf0 0x7f7140072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7140149970 0x7f714014a8b0 secure :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c5f0 tx=0x7f712c006060 comp rx=0 tx=0).stop 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 --2- 172.21.15.44:0/1247629676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7124003fd0 0x7f71401f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.722+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 >> 172.21.15.44:0/1247629676 conn(0x7f71400b0eb0 msgr2=0x7f71400cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:33.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.725+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 shutdown_connections 2024-07-17T08:36:33.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.725+0000 7f7146e00640 1 -- 172.21.15.44:0/1247629676 wait complete. 2024-07-17T08:36:33.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2611825652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce006ed30 msgr2=0x7f2ce015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2611825652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce006ed30 0x7f2ce015c2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8005fd0 tx=0x7f2cc802a540 comp rx=0 tx=0).stop 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2611825652 shutdown_connections 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2611825652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce006ed30 0x7f2ce015c2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2611825652 >> v2:172.21.15.175:3300/0 conn(0x7f2ce0152220 0x7f2ce006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.954+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2611825652 >> 172.21.15.44:0/2611825652 conn(0x7f2ce00b0eb0 msgr2=0x7f2ce00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.955+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2611825652 shutdown_connections 2024-07-17T08:36:33.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.955+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2611825652 wait complete. 2024-07-17T08:36:33.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.956+0000 7f2ce6c00640 1 Processor -- start 2024-07-17T08:36:33.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.957+0000 7f2ce6c00640 1 -- start start 2024-07-17T08:36:33.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.957+0000 7f2ce6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 0x7f2ce01f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:33.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.957+0000 7f2ce6c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.957+0000 7f2ce6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 0x7f2ce0073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2ce6c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00cbbd0 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2ce6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00cba50 con 0x7f2ce006ed30 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2ce6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00cbd50 con 0x7f2ce0152220 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52546/0 (socket says 172.21.15.44:52546) 2024-07-17T08:36:33.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 -- 172.21.15.44:0/2115387769 learned_addr learned my addr 172.21.15.44:0/2115387769 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:33.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2ce4c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 0x7f2ce0073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:33.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 0x7f2ce01f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:33.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 -- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 msgr2=0x7f2ce0073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:33.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 0x7f2ce0073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 -- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 msgr2=0x7f2ce01f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 0x7f2ce01f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cd7400640 1 -- 172.21.15.44:0/2115387769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce01539a0 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 0x7f2ce01f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.958+0000 7f2ce4c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 0x7f2ce0073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2cd7400640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009850 tx=0x7f2cc80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc803f070 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc80047a0 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8035030 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce0071680 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.959+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce0071c00 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.960+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce015f940 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.962+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f2cc8042070 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.963+0000 7f2cdcc00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 0x7f2cbc0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:33.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.963+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7435+0+0 (secure 0 0 0) 0x7f2cc80b9ff0 con 0x7f2ce00cb3a0 2024-07-17T08:36:33.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.964+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 0x7f2cbc0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:33.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.965+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 0x7f2cbc0792e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009ab0 tx=0x7f2cd0005f90 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:33.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:33.967+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2ce01539a0 con 0x7f2ce00cb3a0 2024-07-17T08:36:34.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.216+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f2ce00b3f30 con 0x7f2ce00cb3a0 2024-07-17T08:36:34.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.721+0000 7f2cdcc00640 1 -- 172.21.15.44:0/2115387769 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 33 and data pool 34 v132) ==== 183+0+0 (secure 0 0 0) 0x7f2cc8082ad0 con 0x7f2ce00cb3a0 2024-07-17T08:36:34.726 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 33 and data pool 34 2024-07-17T08:36:34.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 msgr2=0x7f2cbc0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 0x7f2cbc0792e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009ab0 tx=0x7f2cd0005f90 comp rx=0 tx=0).stop 2024-07-17T08:36:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 msgr2=0x7f2ce01505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009850 tx=0x7f2cc80038a0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 shutdown_connections 2024-07-17T08:36:34.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f2cbc076e20 0x7f2cbc0792e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0152220 0x7f2ce0073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce00cb3a0 0x7f2ce01505a0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 --2- 172.21.15.44:0/2115387769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006ed30 0x7f2ce01f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 >> 172.21.15.44:0/2115387769 conn(0x7f2ce00b0eb0 msgr2=0x7f2ce00b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 shutdown_connections 2024-07-17T08:36:34.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.725+0000 7f2ce6c00640 1 -- 172.21.15.44:0/2115387769 wait complete. 2024-07-17T08:36:34.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.947+0000 7fd192000640 1 -- 172.21.15.44:0/2151034270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06e090 msgr2=0x7fd18c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.947+0000 7fd192000640 1 --2- 172.21.15.44:0/2151034270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06e090 0x7fd18c06e4a0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fd18c1576d0 tx=0x7fd18402b9a0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 -- 172.21.15.44:0/2151034270 shutdown_connections 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 --2- 172.21.15.44:0/2151034270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06e090 0x7fd18c06e4a0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 --2- 172.21.15.44:0/2151034270 >> v2:172.21.15.175:3300/0 conn(0x7fd18c0ac3e0 0x7fd18c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 -- 172.21.15.44:0/2151034270 >> 172.21.15.44:0/2151034270 conn(0x7fd18c0b0fb0 msgr2=0x7fd18c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 -- 172.21.15.44:0/2151034270 shutdown_connections 2024-07-17T08:36:34.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.948+0000 7fd192000640 1 -- 172.21.15.44:0/2151034270 wait complete. 2024-07-17T08:36:34.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.949+0000 7fd192000640 1 Processor -- start 2024-07-17T08:36:34.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.950+0000 7fd192000640 1 -- start start 2024-07-17T08:36:34.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.950+0000 7fd192000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18c06e090 0x7fd18c0975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:34.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.950+0000 7fd192000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:34.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd192000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 0x7fd18c1f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd192000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c159c80 con 0x7fd18c06f710 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd192000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c159b00 con 0x7fd18c0ac3e0 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd192000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd18c159e00 con 0x7fd18c06e090 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52568/0 (socket says 172.21.15.44:52568) 2024-07-17T08:36:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 -- 172.21.15.44:0/219636387 learned_addr learned my addr 172.21.15.44:0/219636387 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 -- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18c06e090 msgr2=0x7fd18c0975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18be00640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 0x7fd18c1f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18b400640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18c06e090 0x7fd18c0975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18c06e090 0x7fd18c0975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 -- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 msgr2=0x7fd18c1f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 0x7fd18c1f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18aa00640 1 -- 172.21.15.44:0/219636387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c095f00 con 0x7fd18c06f710 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.951+0000 7fd18be00640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 0x7fd18c1f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd18aa00640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fd17800b410 tx=0x7fd17800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178015020 con 0x7fd18c06f710 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd178011940 con 0x7fd18c06f710 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c0961e0 con 0x7fd18c06f710 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178011d90 con 0x7fd18c06f710 2024-07-17T08:36:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.952+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c0967a0 con 0x7fd18c06f710 2024-07-17T08:36:34.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.953+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c050670 con 0x7fd18c06f710 2024-07-17T08:36:34.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.955+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fd178018020 con 0x7fd18c06f710 2024-07-17T08:36:34.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.956+0000 7fd183400640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 0x7fd16c0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.956+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7541+0+0 (secure 0 0 0) 0x7fd178098f70 con 0x7fd18c06f710 2024-07-17T08:36:34.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.957+0000 7fd18b400640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 0x7fd16c0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:34.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.958+0000 7fd18b400640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 0x7fd16c0792b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd17c001640 tx=0x7fd17c009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:34.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:34.960+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd18c050670 con 0x7fd18c06f710 2024-07-17T08:36:35.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.166+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd18c0ac870 con 0x7fd18c06f710 2024-07-17T08:36:35.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.170+0000 7fd183400640 1 -- 172.21.15.44:0/219636387 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) ==== 74+0+21556 (secure 0 0 0) 0x7fd17801f020 con 0x7fd18c06f710 2024-07-17T08:36:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:35.172 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":246,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:34.218408+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 msgr2=0x7fd16c0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 0x7fd16c0792b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd17c001640 tx=0x7fd17c009040 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 msgr2=0x7fd18c1f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fd17800b410 tx=0x7fd17800b8d0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 shutdown_connections 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fd16c076df0 0x7fd16c0792b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd18c0ac3e0 0x7fd18c1f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd18c06f710 0x7fd18c1f7af0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 --2- 172.21.15.44:0/219636387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd18c06e090 0x7fd18c0975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 >> 172.21.15.44:0/219636387 conn(0x7fd18c0b0fb0 msgr2=0x7fd18c097e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:35.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 shutdown_connections 2024-07-17T08:36:35.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.173+0000 7fd192000640 1 -- 172.21.15.44:0/219636387 wait complete. 2024-07-17T08:36:35.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 -- 172.21.15.44:0/741445627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ac930 msgr2=0x7f0cf00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/741445627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ac930 0x7f0cf00b25d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005fd0 tx=0x7f0ce802aa90 comp rx=0 tx=0).stop 2024-07-17T08:36:35.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 -- 172.21.15.44:0/741445627 shutdown_connections 2024-07-17T08:36:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/741445627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ac930 0x7f0cf00b25d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/741445627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf00ae810 0x7f0cf015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 -- 172.21.15.44:0/741445627 >> 172.21.15.44:0/741445627 conn(0x7f0cf00b0fb0 msgr2=0x7f0cf00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 -- 172.21.15.44:0/741445627 shutdown_connections 2024-07-17T08:36:35.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.382+0000 7f0cf4e00640 1 -- 172.21.15.44:0/741445627 wait complete. 2024-07-17T08:36:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.384+0000 7f0cf4e00640 1 Processor -- start 2024-07-17T08:36:35.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.385+0000 7f0cf4e00640 1 -- start start 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf00ac930 0x7f0cf0144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ae810 0x7f0cf0151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006f010 con 0x7f0cf006ea50 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf006ee90 con 0x7f0cf00ae810 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cf4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf00b2b10 con 0x7f0cf00ac930 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52574/0 (socket says 172.21.15.44:52574) 2024-07-17T08:36:35.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.386+0000 7f0cede00640 1 -- 172.21.15.44:0/1267436629 learned_addr learned my addr 172.21.15.44:0/1267436629 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:35.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 -- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf00ac930 msgr2=0x7f0cf0144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf00ac930 0x7f0cf0144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 -- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ae810 msgr2=0x7f0cf0151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cee800640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ae810 0x7f0cf0151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ae810 0x7f0cf0151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 -- 172.21.15.44:0/1267436629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf0155b20 con 0x7f0cf006ea50 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cede00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e790 tx=0x7f0ce000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000eea0 con 0x7f0cf006ea50 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ce000cdb0 con 0x7f0cf006ea50 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0010690 con 0x7f0cf006ea50 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf0154d60 con 0x7f0cf006ea50 2024-07-17T08:36:35.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.387+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf0155320 con 0x7f0cf006ea50 2024-07-17T08:36:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.389+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf006a1a0 con 0x7f0cf006ea50 2024-07-17T08:36:35.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.390+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f0ce0010830 con 0x7f0cf006ea50 2024-07-17T08:36:35.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.391+0000 7f0ce6000640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 0x7f0cd00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.391+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7541+0+0 (secure 0 0 0) 0x7f0ce0099840 con 0x7f0cf006ea50 2024-07-17T08:36:35.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.392+0000 7f0ced400640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 0x7f0cd00792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.393+0000 7f0ced400640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 0x7f0cd00792b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000f60 tx=0x7f0cdc0079b0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.396+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0ce0062200 con 0x7f0cf006ea50 2024-07-17T08:36:35.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.604+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cf00b23b0 con 0x7f0cf006ea50 2024-07-17T08:36:35.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.608+0000 7f0ce6000640 1 -- 172.21.15.44:0/1267436629 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) ==== 74+0+21556 (secure 0 0 0) 0x7f0ce0061b00 con 0x7f0cf006ea50 2024-07-17T08:36:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:35.610 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":246,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:34.218408+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 msgr2=0x7f0cd00792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 0x7f0cd00792b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000f60 tx=0x7f0cdc0079b0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 msgr2=0x7f0cf0150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e790 tx=0x7f0ce000ec50 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 shutdown_connections 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0cd0076df0 0x7f0cd00792b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf00ae810 0x7f0cf0151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf00ac930 0x7f0cf0144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 --2- 172.21.15.44:0/1267436629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf006ea50 0x7f0cf0150c30 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 >> 172.21.15.44:0/1267436629 conn(0x7f0cf00b0fb0 msgr2=0x7f0cf00aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:35.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 shutdown_connections 2024-07-17T08:36:35.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.611+0000 7f0cf4e00640 1 -- 172.21.15.44:0/1267436629 wait complete. 2024-07-17T08:36:35.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 -- 172.21.15.44:0/1922480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee8004800 msgr2=0x7f0ee8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 --2- 172.21.15.44:0/1922480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee8004800 0x7f0ee8004c80 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000bb20 tx=0x7f0ee8005830 comp rx=0 tx=0).stop 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 -- 172.21.15.44:0/1922480391 shutdown_connections 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 --2- 172.21.15.44:0/1922480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ee8004800 0x7f0ee8004c80 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 -- 172.21.15.44:0/1922480391 >> 172.21.15.44:0/1922480391 conn(0x7f0f000b0eb0 msgr2=0x7f0f000b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 -- 172.21.15.44:0/1922480391 shutdown_connections 2024-07-17T08:36:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.844+0000 7f0f08000640 1 -- 172.21.15.44:0/1922480391 wait complete. 2024-07-17T08:36:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.846+0000 7f0f08000640 1 Processor -- start 2024-07-17T08:36:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.846+0000 7f0f08000640 1 -- start start 2024-07-17T08:36:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ee8004800 0x7f0f001477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f00073f30 0x7f0f001f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0015c470 con 0x7f0f001f7f00 2024-07-17T08:36:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0015c2f0 con 0x7f0ee8004800 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f08000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0015c5f0 con 0x7f0f00073f30 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f06000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f06000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52594/0 (socket says 172.21.15.44:52594) 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f06000640 1 -- 172.21.15.44:0/1501381493 learned_addr learned my addr 172.21.15.44:0/1501381493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f05600640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ee8004800 0x7f0f001477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f04c00640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f00073f30 0x7f0f001f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f06000640 1 -- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f00073f30 msgr2=0x7f0f001f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.847+0000 7f0f06000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f00073f30 0x7f0f001f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f06000640 1 -- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ee8004800 msgr2=0x7f0f001477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f06000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ee8004800 0x7f0f001477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f06000640 1 -- 172.21.15.44:0/1501381493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0014fef0 con 0x7f0f001f7f00 2024-07-17T08:36:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f06000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f0f000cce20 tx=0x7f0ef800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8018070 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ef8003970 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8003da0 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0015af40 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.848+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f00150440 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.850+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f000cb4d0 con 0x7f0f001f7f00 2024-07-17T08:36:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.851+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f0ef801b070 con 0x7f0f001f7f00 2024-07-17T08:36:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.852+0000 7f0efd600640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 0x7f0ee40792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.853+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f0ef8098420 con 0x7f0f001f7f00 2024-07-17T08:36:35.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.853+0000 7f0f05600640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 0x7f0ee40792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:35.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.854+0000 7f0f05600640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 0x7f0ee40792e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4001eb0 tx=0x7f0ef4009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:35.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:35.857+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0ef8060dc0 con 0x7f0f001f7f00 2024-07-17T08:36:36.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.113+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0f0015d2e0 con 0x7f0f001f7f00 2024-07-17T08:36:36.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.746+0000 7f0efd600640 1 -- 172.21.15.44:0/1501381493 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v134) ==== 121+0+0 (secure 0 0 0) 0x7f0ef80606c0 con 0x7f0f001f7f00 2024-07-17T08:36:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 msgr2=0x7f0ee40792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 0x7f0ee40792e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4001eb0 tx=0x7f0ef4009040 comp rx=0 tx=0).stop 2024-07-17T08:36:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 msgr2=0x7f0f001f83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f0f000cce20 tx=0x7f0ef800dc10 comp rx=0 tx=0).stop 2024-07-17T08:36:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 shutdown_connections 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f0ee4076e20 0x7f0ee40792e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f001f7f00 0x7f0f001f83b0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f00073f30 0x7f0f001f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 --2- 172.21.15.44:0/1501381493 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ee8004800 0x7f0f001477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.750+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 >> 172.21.15.44:0/1501381493 conn(0x7f0f000b0eb0 msgr2=0x7f0f0015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.751+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 shutdown_connections 2024-07-17T08:36:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.751+0000 7f0f08000640 1 -- 172.21.15.44:0/1501381493 wait complete. 2024-07-17T08:36:36.778 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:36:36.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-17T08:36:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 -- 172.21.15.44:0/1268122963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc218003ad0 msgr2=0x7fc218003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 --2- 172.21.15.44:0/1268122963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc218003ad0 0x7fc218003ef0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c290 tx=0x7fc218005830 comp rx=0 tx=0).stop 2024-07-17T08:36:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc233400640 1 -- 172.21.15.44:0/1268122963 reap_dead start 2024-07-17T08:36:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 -- 172.21.15.44:0/1268122963 shutdown_connections 2024-07-17T08:36:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 -- 172.21.15.44:0/1268122963 >> 172.21.15.44:0/1268122963 conn(0x7fc234073d50 msgr2=0x7fc23409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 -- 172.21.15.44:0/1268122963 shutdown_connections 2024-07-17T08:36:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.994+0000 7fc239a00640 1 -- 172.21.15.44:0/1268122963 wait complete. 2024-07-17T08:36:36.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.996+0000 7fc239a00640 1 Processor -- start 2024-07-17T08:36:36.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.996+0000 7fc239a00640 1 -- start start 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ad0 0x7fc2341f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc234150c20 0x7fc23415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340ae5f0 con 0x7fc23414c640 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340ae470 con 0x7fc234150c20 2024-07-17T08:36:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.997+0000 7fc239a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2340ae770 con 0x7fc218003ad0 2024-07-17T08:36:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52606/0 (socket says 172.21.15.44:52606) 2024-07-17T08:36:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 -- 172.21.15.44:0/2614917849 learned_addr learned my addr 172.21.15.44:0/2614917849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc22a000640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc234150c20 0x7fc23415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 -- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ad0 msgr2=0x7fc2341f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ad0 0x7fc2341f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 -- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc234150c20 msgr2=0x7fc23415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc234150c20 0x7fc23415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 -- 172.21.15.44:0/2614917849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23415fec0 con 0x7fc23414c640 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.998+0000 7fc233400640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc2341611f0 tx=0x7fc22c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.999+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c016070 con 0x7fc23414c640 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.999+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc22c004e30 con 0x7fc23414c640 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.999+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00cc70 con 0x7fc23414c640 2024-07-17T08:36:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.999+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc234160140 con 0x7fc23414c640 2024-07-17T08:36:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:36.999+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc234160700 con 0x7fc23414c640 2024-07-17T08:36:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.000+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23415de30 con 0x7fc23414c640 2024-07-17T08:36:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.001+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fc22c005000 con 0x7fc23414c640 2024-07-17T08:36:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.002+0000 7fc22b400640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 0x7fc2100792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.002+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fc22c099480 con 0x7fc23414c640 2024-07-17T08:36:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.003+0000 7fc232a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 0x7fc2100792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:37.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.004+0000 7fc232a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 0x7fc2100792b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc224033000 tx=0x7fc2340b2130 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:37.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.007+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc23415de30 con 0x7fc23414c640 2024-07-17T08:36:37.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.259+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc23406d690 con 0x7fc23414c640 2024-07-17T08:36:37.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.769+0000 7fc22b400640 1 -- 172.21.15.44:0/2614917849 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v135) ==== 125+0+0 (secure 0 0 0) 0x7fc23415fec0 con 0x7fc23414c640 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.774+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 msgr2=0x7fc2100792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.774+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 0x7fc2100792b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc224033000 tx=0x7fc2340b2130 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.774+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 msgr2=0x7fc23415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.774+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc2341611f0 tx=0x7fc22c00ed20 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 shutdown_connections 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fc210076df0 0x7fc2100792b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23414c640 0x7fc23415c2f0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc234150c20 0x7fc23415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 --2- 172.21.15.44:0/2614917849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ad0 0x7fc2341f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 >> 172.21.15.44:0/2614917849 conn(0x7fc234073d50 msgr2=0x7fc23409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 shutdown_connections 2024-07-17T08:36:37.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:37.775+0000 7fc239a00640 1 -- 172.21.15.44:0/2614917849 wait complete. 2024-07-17T08:36:37.800 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-17T08:36:37.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 -- 172.21.15.44:0/972263812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 msgr2=0x7fdeb00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 --2- 172.21.15.44:0/972263812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb00ad8e0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00d00a0 tx=0x7fde9c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 -- 172.21.15.44:0/972263812 shutdown_connections 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 --2- 172.21.15.44:0/972263812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb00ad8e0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 --2- 172.21.15.44:0/972263812 >> v2:172.21.15.175:3300/0 conn(0x7fdeb00cf540 0x7fdeb00b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.010+0000 7fdeb6400640 1 -- 172.21.15.44:0/972263812 >> 172.21.15.44:0/972263812 conn(0x7fdeb00b0e80 msgr2=0x7fdeb00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.011+0000 7fdeb6400640 1 -- 172.21.15.44:0/972263812 shutdown_connections 2024-07-17T08:36:38.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.011+0000 7fdeb6400640 1 -- 172.21.15.44:0/972263812 wait complete. 2024-07-17T08:36:38.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.012+0000 7fdeb6400640 1 Processor -- start 2024-07-17T08:36:38.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.012+0000 7fdeb6400640 1 -- start start 2024-07-17T08:36:38.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdeb00cf540 0x7fdeb006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdeb00cf920 0x7fdeb013ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb00ae8c0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb00ae740 con 0x7fdeb00cf920 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.013+0000 7fdeb6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb00aea40 con 0x7fdeb00cf540 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52622/0 (socket says 172.21.15.44:52622) 2024-07-17T08:36:38.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 -- 172.21.15.44:0/3035727061 learned_addr learned my addr 172.21.15.44:0/3035727061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:38.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeafe00640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdeb00cf540 0x7fdeb006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 -- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdeb00cf540 msgr2=0x7fdeb006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdeb00cf540 0x7fdeb006ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 -- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdeb00cf920 msgr2=0x7fdeb013ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdeb00cf920 0x7fdeb013ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.014+0000 7fdeb5000640 1 -- 172.21.15.44:0/3035727061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb0158b50 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeb5000640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c8c0 tx=0x7fdea400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4004220 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdea40079b0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4007de0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb0158e30 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.015+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb006f1b0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.016+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb01412e0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.017+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fdea4005020 con 0x7fdeb00b3db0 2024-07-17T08:36:38.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.018+0000 7fdeacc00640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 0x7fde900792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.018+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fdea40993f0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.019+0000 7fdeafe00640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 0x7fde900792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.020+0000 7fdeafe00640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 0x7fde900792b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde9c033000 tx=0x7fde9c004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:38.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.023+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdea4014030 con 0x7fdeb00b3db0 2024-07-17T08:36:38.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.281+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeb006fcd0 con 0x7fdeb00b3db0 2024-07-17T08:36:38.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.282+0000 7fdeacc00640 1 -- 172.21.15.44:0/3035727061 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7fdea4061d10 con 0x7fdeb00b3db0 2024-07-17T08:36:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:38.284 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:38.285 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 msgr2=0x7fde900792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 0x7fde900792b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde9c033000 tx=0x7fde9c004730 comp rx=0 tx=0).stop 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 msgr2=0x7fdeb01571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c8c0 tx=0x7fdea400cd80 comp rx=0 tx=0).stop 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 shutdown_connections 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fde90076df0 0x7fde900792b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdeb00cf920 0x7fdeb013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdeb00cf540 0x7fdeb006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 --2- 172.21.15.44:0/3035727061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdeb00b3db0 0x7fdeb01571c0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 >> 172.21.15.44:0/3035727061 conn(0x7fdeb00b0e80 msgr2=0x7fdeb013c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 shutdown_connections 2024-07-17T08:36:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.285+0000 7fdeb6400640 1 -- 172.21.15.44:0/3035727061 wait complete. 2024-07-17T08:36:38.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 -- 172.21.15.44:0/3079245743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf540 msgr2=0x7f11400b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 --2- 172.21.15.44:0/3079245743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf540 0x7f11400b3870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1130005fd0 tx=0x7f113002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 -- 172.21.15.44:0/3079245743 shutdown_connections 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 --2- 172.21.15.44:0/3079245743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 0x7f11400ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 --2- 172.21.15.44:0/3079245743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf540 0x7f11400b3870 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.520+0000 7f113f200640 1 -- 172.21.15.44:0/3079245743 >> 172.21.15.44:0/3079245743 conn(0x7f11400b0e80 msgr2=0x7f11400b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.521+0000 7f113f200640 1 -- 172.21.15.44:0/3079245743 shutdown_connections 2024-07-17T08:36:38.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.521+0000 7f113f200640 1 -- 172.21.15.44:0/3079245743 wait complete. 2024-07-17T08:36:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.522+0000 7f113f200640 1 Processor -- start 2024-07-17T08:36:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.523+0000 7f113f200640 1 -- start start 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.523+0000 7f113f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 0x7f11401f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 0x7f1140072370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f11400ae940 con 0x7f11400cf540 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11400ae7c0 con 0x7f11400cf920 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1140141040 con 0x7f11400b3db0 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52632/0 (socket says 172.21.15.44:52632) 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 -- 172.21.15.44:0/1004941240 learned_addr learned my addr 172.21.15.44:0/1004941240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113de00640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 0x7f11401f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113e800640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 0x7f1140072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 -- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 msgr2=0x7f11401f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 0x7f11401f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 -- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 msgr2=0x7f1140072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 0x7f1140072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113d400640 1 -- 172.21.15.44:0/1004941240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401f4fd0 con 0x7f11400cf540 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.524+0000 7f113e800640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 0x7f1140072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f113d400640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f112800ea20 tx=0x7f112800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112800cd80 con 0x7f11400cf540 2024-07-17T08:36:38.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1128010430 con 0x7f11400cf540 2024-07-17T08:36:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128010840 con 0x7f11400cf540 2024-07-17T08:36:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1140071760 con 0x7f11400cf540 2024-07-17T08:36:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.525+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1140071d20 con 0x7f11400cf540 2024-07-17T08:36:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.526+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1140050670 con 0x7f11400cf540 2024-07-17T08:36:38.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.527+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f11280109e0 con 0x7f11400cf540 2024-07-17T08:36:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.528+0000 7f1136000640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 0x7f111c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:38.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.529+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f1128099ad0 con 0x7f11400cf540 2024-07-17T08:36:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.529+0000 7f113de00640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 0x7f111c0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:38.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.530+0000 7f113de00640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 0x7f111c0792e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1130033000 tx=0x7f1130004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:38.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.533+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1140050670 con 0x7f11400cf540 2024-07-17T08:36:38.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.758+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11401414c0 con 0x7f11400cf540 2024-07-17T08:36:38.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.762+0000 7f1136000640 1 -- 172.21.15.44:0/1004941240 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v247) ==== 74+0+21543 (secure 0 0 0) 0x7f1128062360 con 0x7f11400cf540 2024-07-17T08:36:38.764 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:38.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":247,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:35.722533+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 msgr2=0x7f111c0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 0x7f111c0792e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1130033000 tx=0x7f1130004730 comp rx=0 tx=0).stop 2024-07-17T08:36:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 msgr2=0x7f114015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f112800ea20 tx=0x7f112800eee0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 shutdown_connections 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f111c076e20 0x7f111c0792e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11400cf920 0x7f1140072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11400cf540 0x7f114015e3b0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 --2- 172.21.15.44:0/1004941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11400b3db0 0x7f11401f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 >> 172.21.15.44:0/1004941240 conn(0x7f11400b0e80 msgr2=0x7f114013e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.765+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 shutdown_connections 2024-07-17T08:36:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.766+0000 7f113f200640 1 -- 172.21.15.44:0/1004941240 wait complete. 2024-07-17T08:36:38.789 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:36:38.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:39.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.999+0000 7f8fefe00640 1 -- 172.21.15.44:0/2665269666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff00cfab0 msgr2=0x7f8ff009b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.999+0000 7f8fefe00640 1 --2- 172.21.15.44:0/2665269666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff00cfab0 0x7f8ff009b9a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005fd0 tx=0x7f8fd802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.999+0000 7f8fefe00640 1 -- 172.21.15.44:0/2665269666 shutdown_connections 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.999+0000 7f8fefe00640 1 --2- 172.21.15.44:0/2665269666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff00cfab0 0x7f8ff009b9a0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:38.999+0000 7f8fefe00640 1 -- 172.21.15.44:0/2665269666 >> 172.21.15.44:0/2665269666 conn(0x7f8ff00b0e80 msgr2=0x7f8ff00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.000+0000 7f8fefe00640 1 -- 172.21.15.44:0/2665269666 shutdown_connections 2024-07-17T08:36:39.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.000+0000 7f8fefe00640 1 -- 172.21.15.44:0/2665269666 wait complete. 2024-07-17T08:36:39.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.001+0000 7f8fefe00640 1 Processor -- start 2024-07-17T08:36:39.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.001+0000 7f8fefe00640 1 -- start start 2024-07-17T08:36:39.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff009bee0 0x7f8ff0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff00cfab0 0x7f8ff01566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff015e240 con 0x7f8ff0157ba0 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff015e3c0 con 0x7f8ff00cfab0 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.002+0000 7f8fefe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff015e540 con 0x7f8ff009bee0 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54180/0 (socket says 172.21.15.44:54180) 2024-07-17T08:36:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 -- 172.21.15.44:0/865264225 learned_addr learned my addr 172.21.15.44:0/865264225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8feea00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff009bee0 0x7f8ff0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 -- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff009bee0 msgr2=0x7f8ff0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fe7e00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff00cfab0 0x7f8ff01566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff009bee0 0x7f8ff0156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 -- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff00cfab0 msgr2=0x7f8ff01566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff00cfab0 0x7f8ff01566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.003+0000 7f8fef400640 1 -- 172.21.15.44:0/865264225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0072af0 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fef400640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c780 tx=0x7f8fe800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004b60 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe8004df0 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800aa90 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff0072dd0 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.004+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff0073390 con 0x7f8ff0157ba0 2024-07-17T08:36:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.006+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff009b9a0 con 0x7f8ff0157ba0 2024-07-17T08:36:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.007+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f8fe800ac30 con 0x7f8ff0157ba0 2024-07-17T08:36:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.008+0000 7f8fe7400640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 0x7f8fcc0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.008+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f8fe8099260 con 0x7f8ff0157ba0 2024-07-17T08:36:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.008+0000 7f8feea00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 0x7f8fcc0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.010+0000 7f8feea00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 0x7f8fcc0792b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00015a0 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.013+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8fe8020070 con 0x7f8ff0157ba0 2024-07-17T08:36:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.225+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ff00ce920 con 0x7f8fcc076df0 2024-07-17T08:36:39.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.251+0000 7f8fe7400640 1 -- 172.21.15.44:0/865264225 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+486100 (secure 0 0 0) 0x7f8ff00ce920 con 0x7f8fcc076df0 2024-07-17T08:36:39.260 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:39.264 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 msgr2=0x7f8fcc0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 0x7f8fcc0792b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00015a0 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 msgr2=0x7f8ff01580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c780 tx=0x7f8fe800cc40 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 shutdown_connections 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fcc076df0 0x7f8fcc0792b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff0157ba0 0x7f8ff01580d0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff00cfab0 0x7f8ff01566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 --2- 172.21.15.44:0/865264225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff009bee0 0x7f8ff0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 >> 172.21.15.44:0/865264225 conn(0x7f8ff00b0e80 msgr2=0x7f8ff015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 shutdown_connections 2024-07-17T08:36:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.260+0000 7f8fefe00640 1 -- 172.21.15.44:0/865264225 wait complete. 2024-07-17T08:36:39.282 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-07-17T08:36:37.286889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594024,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":430,"num_write_kb":1644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2566,"ondisk_log_size":2566,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410820,"kb_used_data":25304,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754563900,"statfs":{"total":773094113280,"available":772673433600,"internally_reserved":0,"allocated":25911296,"data_stored":14666760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1246,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.583645"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739038+0000","last_change":"2024-07-17T08:36:33.739038+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739038+0000","last_undegraded":"2024-07-17T08:36:33.739038+0000","last_fullsized":"2024-07-17T08:36:33.739038+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750705+0000","last_change":"2024-07-17T08:36:33.750014+0000","last_active":"2024-07-17T08:36:33.750705+0000","last_peered":"2024-07-17T08:36:33.750705+0000","last_clean":"2024-07-17T08:36:33.750705+0000","last_became_active":"2024-07-17T08:36:33.749632+0000","last_became_peered":"2024-07-17T08:36:33.749632+0000","last_unstale":"2024-07-17T08:36:33.750705+0000","last_undegraded":"2024-07-17T08:36:33.750705+0000","last_fullsized":"2024-07-17T08:36:33.750705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:12.073787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749733+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:33.749733+0000","last_peered":"2024-07-17T08:36:33.749733+0000","last_clean":"2024-07-17T08:36:33.749733+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:33.749733+0000","last_undegraded":"2024-07-17T08:36:33.749733+0000","last_fullsized":"2024-07-17T08:36:33.749733+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727993+0000","last_change":"2024-07-17T08:36:33.727993+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727993+0000","last_undegraded":"2024-07-17T08:36:33.727993+0000","last_fullsized":"2024-07-17T08:36:33.727993+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.759795+0000","last_change":"2024-07-17T08:36:33.758375+0000","last_active":"2024-07-17T08:36:33.759795+0000","last_peered":"2024-07-17T08:36:33.759795+0000","last_clean":"2024-07-17T08:36:33.759795+0000","last_became_active":"2024-07-17T08:36:33.758093+0000","last_became_peered":"2024-07-17T08:36:33.758093+0000","last_unstale":"2024-07-17T08:36:33.759795+0000","last_undegraded":"2024-07-17T08:36:33.759795+0000","last_fullsized":"2024-07-17T08:36:33.759795+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:01.888503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.725621+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:33.725621+0000","last_peered":"2024-07-17T08:36:33.725621+0000","last_clean":"2024-07-17T08:36:33.725621+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:33.725621+0000","last_undegraded":"2024-07-17T08:36:33.725621+0000","last_fullsized":"2024-07-17T08:36:33.725621+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.731661+0000","last_change":"2024-07-17T08:36:33.731661+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.731661+0000","last_undegraded":"2024-07-17T08:36:33.731661+0000","last_fullsized":"2024-07-17T08:36:33.731661+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748357+0000","last_change":"2024-07-17T08:36:33.746302+0000","last_active":"2024-07-17T08:36:33.748357+0000","last_peered":"2024-07-17T08:36:33.748357+0000","last_clean":"2024-07-17T08:36:33.748357+0000","last_became_active":"2024-07-17T08:36:33.745779+0000","last_became_peered":"2024-07-17T08:36:33.745779+0000","last_unstale":"2024-07-17T08:36:33.748357+0000","last_undegraded":"2024-07-17T08:36:33.748357+0000","last_fullsized":"2024-07-17T08:36:33.748357+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:43:23.866594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737655+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:33.737655+0000","last_peered":"2024-07-17T08:36:33.737655+0000","last_clean":"2024-07-17T08:36:33.737655+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:33.737655+0000","last_undegraded":"2024-07-17T08:36:33.737655+0000","last_fullsized":"2024-07-17T08:36:33.737655+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739208+0000","last_change":"2024-07-17T08:36:33.739208+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739208+0000","last_undegraded":"2024-07-17T08:36:33.739208+0000","last_fullsized":"2024-07-17T08:36:33.739208+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757949+0000","last_change":"2024-07-17T08:36:33.755800+0000","last_active":"2024-07-17T08:36:33.757949+0000","last_peered":"2024-07-17T08:36:33.757949+0000","last_clean":"2024-07-17T08:36:33.757949+0000","last_became_active":"2024-07-17T08:36:33.754310+0000","last_became_peered":"2024-07-17T08:36:33.754310+0000","last_unstale":"2024-07-17T08:36:33.757949+0000","last_undegraded":"2024-07-17T08:36:33.757949+0000","last_fullsized":"2024-07-17T08:36:33.757949+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739306+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:33.739306+0000","last_peered":"2024-07-17T08:36:33.739306+0000","last_clean":"2024-07-17T08:36:33.739306+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:33.739306+0000","last_undegraded":"2024-07-17T08:36:33.739306+0000","last_fullsized":"2024-07-17T08:36:33.739306+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.731677+0000","last_change":"2024-07-17T08:36:33.731677+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.731677+0000","last_undegraded":"2024-07-17T08:36:33.731677+0000","last_fullsized":"2024-07-17T08:36:33.731677+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750372+0000","last_change":"2024-07-17T08:36:33.747145+0000","last_active":"2024-07-17T08:36:33.750372+0000","last_peered":"2024-07-17T08:36:33.750372+0000","last_clean":"2024-07-17T08:36:33.750372+0000","last_became_active":"2024-07-17T08:36:33.746842+0000","last_became_peered":"2024-07-17T08:36:33.746842+0000","last_unstale":"2024-07-17T08:36:33.750372+0000","last_undegraded":"2024-07-17T08:36:33.750372+0000","last_fullsized":"2024-07-17T08:36:33.750372+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:17.951527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734142+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:33.734142+0000","last_peered":"2024-07-17T08:36:33.734142+0000","last_clean":"2024-07-17T08:36:33.734142+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:33.734142+0000","last_undegraded":"2024-07-17T08:36:33.734142+0000","last_fullsized":"2024-07-17T08:36:33.734142+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728530+0000","last_change":"2024-07-17T08:36:33.728530+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728530+0000","last_undegraded":"2024-07-17T08:36:33.728530+0000","last_fullsized":"2024-07-17T08:36:33.728530+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757187+0000","last_change":"2024-07-17T08:36:33.755670+0000","last_active":"2024-07-17T08:36:33.757187+0000","last_peered":"2024-07-17T08:36:33.757187+0000","last_clean":"2024-07-17T08:36:33.757187+0000","last_became_active":"2024-07-17T08:36:33.755363+0000","last_became_peered":"2024-07-17T08:36:33.755363+0000","last_unstale":"2024-07-17T08:36:33.757187+0000","last_undegraded":"2024-07-17T08:36:33.757187+0000","last_fullsized":"2024-07-17T08:36:33.757187+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:19.719037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748287+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:33.748287+0000","last_peered":"2024-07-17T08:36:33.748287+0000","last_clean":"2024-07-17T08:36:33.748287+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:33.748287+0000","last_undegraded":"2024-07-17T08:36:33.748287+0000","last_fullsized":"2024-07-17T08:36:33.748287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740704+0000","last_change":"2024-07-17T08:36:33.740704+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740704+0000","last_undegraded":"2024-07-17T08:36:33.740704+0000","last_fullsized":"2024-07-17T08:36:33.740704+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753956+0000","last_change":"2024-07-17T08:36:33.748383+0000","last_active":"2024-07-17T08:36:33.753956+0000","last_peered":"2024-07-17T08:36:33.753956+0000","last_clean":"2024-07-17T08:36:33.753956+0000","last_became_active":"2024-07-17T08:36:33.748082+0000","last_became_peered":"2024-07-17T08:36:33.748082+0000","last_unstale":"2024-07-17T08:36:33.753956+0000","last_undegraded":"2024-07-17T08:36:33.753956+0000","last_fullsized":"2024-07-17T08:36:33.753956+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:08.260953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741041+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:33.741041+0000","last_peered":"2024-07-17T08:36:33.741041+0000","last_clean":"2024-07-17T08:36:33.741041+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:33.741041+0000","last_undegraded":"2024-07-17T08:36:33.741041+0000","last_fullsized":"2024-07-17T08:36:33.741041+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742246+0000","last_change":"2024-07-17T08:36:33.742246+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742246+0000","last_undegraded":"2024-07-17T08:36:33.742246+0000","last_fullsized":"2024-07-17T08:36:33.742246+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758343+0000","last_change":"2024-07-17T08:36:33.756246+0000","last_active":"2024-07-17T08:36:33.758343+0000","last_peered":"2024-07-17T08:36:33.758343+0000","last_clean":"2024-07-17T08:36:33.758343+0000","last_became_active":"2024-07-17T08:36:33.754426+0000","last_became_peered":"2024-07-17T08:36:33.754426+0000","last_unstale":"2024-07-17T08:36:33.758343+0000","last_undegraded":"2024-07-17T08:36:33.758343+0000","last_fullsized":"2024-07-17T08:36:33.758343+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:16.201072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732948+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:33.732948+0000","last_peered":"2024-07-17T08:36:33.732948+0000","last_clean":"2024-07-17T08:36:33.732948+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:33.732948+0000","last_undegraded":"2024-07-17T08:36:33.732948+0000","last_fullsized":"2024-07-17T08:36:33.732948+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.737445+0000","last_change":"2024-07-17T08:36:33.737445+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.737445+0000","last_undegraded":"2024-07-17T08:36:33.737445+0000","last_fullsized":"2024-07-17T08:36:33.737445+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.756455+0000","last_change":"2024-07-17T08:36:33.754739+0000","last_active":"2024-07-17T08:36:33.756455+0000","last_peered":"2024-07-17T08:36:33.756455+0000","last_clean":"2024-07-17T08:36:33.756455+0000","last_became_active":"2024-07-17T08:36:33.754416+0000","last_became_peered":"2024-07-17T08:36:33.754416+0000","last_unstale":"2024-07-17T08:36:33.756455+0000","last_undegraded":"2024-07-17T08:36:33.756455+0000","last_fullsized":"2024-07-17T08:36:33.756455+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:45:53.980425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749593+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:33.749593+0000","last_peered":"2024-07-17T08:36:33.749593+0000","last_clean":"2024-07-17T08:36:33.749593+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:33.749593+0000","last_undegraded":"2024-07-17T08:36:33.749593+0000","last_fullsized":"2024-07-17T08:36:33.749593+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744410+0000","last_change":"2024-07-17T08:36:33.744410+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744410+0000","last_undegraded":"2024-07-17T08:36:33.744410+0000","last_fullsized":"2024-07-17T08:36:33.744410+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757512+0000","last_change":"2024-07-17T08:36:33.754818+0000","last_active":"2024-07-17T08:36:33.757512+0000","last_peered":"2024-07-17T08:36:33.757512+0000","last_clean":"2024-07-17T08:36:33.757512+0000","last_became_active":"2024-07-17T08:36:33.753426+0000","last_became_peered":"2024-07-17T08:36:33.753426+0000","last_unstale":"2024-07-17T08:36:33.757512+0000","last_undegraded":"2024-07-17T08:36:33.757512+0000","last_fullsized":"2024-07-17T08:36:33.757512+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:17:29.423459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740088+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:33.740088+0000","last_peered":"2024-07-17T08:36:33.740088+0000","last_clean":"2024-07-17T08:36:33.740088+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:33.740088+0000","last_undegraded":"2024-07-17T08:36:33.740088+0000","last_fullsized":"2024-07-17T08:36:33.740088+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748161+0000","last_change":"2024-07-17T08:36:33.748161+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748161+0000","last_undegraded":"2024-07-17T08:36:33.748161+0000","last_fullsized":"2024-07-17T08:36:33.748161+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.751998+0000","last_change":"2024-07-17T08:36:33.751230+0000","last_active":"2024-07-17T08:36:33.751998+0000","last_peered":"2024-07-17T08:36:33.751998+0000","last_clean":"2024-07-17T08:36:33.751998+0000","last_became_active":"2024-07-17T08:36:33.750539+0000","last_became_peered":"2024-07-17T08:36:33.750539+0000","last_unstale":"2024-07-17T08:36:33.751998+0000","last_undegraded":"2024-07-17T08:36:33.751998+0000","last_fullsized":"2024-07-17T08:36:33.751998+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:44.514396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738509+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:33.738509+0000","last_peered":"2024-07-17T08:36:33.738509+0000","last_clean":"2024-07-17T08:36:33.738509+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:33.738509+0000","last_undegraded":"2024-07-17T08:36:33.738509+0000","last_fullsized":"2024-07-17T08:36:33.738509+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747898+0000","last_change":"2024-07-17T08:36:33.747898+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747898+0000","last_undegraded":"2024-07-17T08:36:33.747898+0000","last_fullsized":"2024-07-17T08:36:33.747898+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758193+0000","last_change":"2024-07-17T08:36:33.754294+0000","last_active":"2024-07-17T08:36:33.758193+0000","last_peered":"2024-07-17T08:36:33.758193+0000","last_clean":"2024-07-17T08:36:33.758193+0000","last_became_active":"2024-07-17T08:36:33.753457+0000","last_became_peered":"2024-07-17T08:36:33.753457+0000","last_unstale":"2024-07-17T08:36:33.758193+0000","last_undegraded":"2024-07-17T08:36:33.758193+0000","last_fullsized":"2024-07-17T08:36:33.758193+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:11.308002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748312+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:33.748312+0000","last_peered":"2024-07-17T08:36:33.748312+0000","last_clean":"2024-07-17T08:36:33.748312+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:33.748312+0000","last_undegraded":"2024-07-17T08:36:33.748312+0000","last_fullsized":"2024-07-17T08:36:33.748312+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742681+0000","last_change":"2024-07-17T08:36:33.742681+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742681+0000","last_undegraded":"2024-07-17T08:36:33.742681+0000","last_fullsized":"2024-07-17T08:36:33.742681+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.759296+0000","last_change":"2024-07-17T08:36:33.758000+0000","last_active":"2024-07-17T08:36:33.759296+0000","last_peered":"2024-07-17T08:36:33.759296+0000","last_clean":"2024-07-17T08:36:33.759296+0000","last_became_active":"2024-07-17T08:36:33.757029+0000","last_became_peered":"2024-07-17T08:36:33.757029+0000","last_unstale":"2024-07-17T08:36:33.759296+0000","last_undegraded":"2024-07-17T08:36:33.759296+0000","last_fullsized":"2024-07-17T08:36:33.759296+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:27:44.036829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753985+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:33.753985+0000","last_peered":"2024-07-17T08:36:33.753985+0000","last_clean":"2024-07-17T08:36:33.753985+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:33.753985+0000","last_undegraded":"2024-07-17T08:36:33.753985+0000","last_fullsized":"2024-07-17T08:36:33.753985+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735212+0000","last_change":"2024-07-17T08:36:33.735212+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735212+0000","last_undegraded":"2024-07-17T08:36:33.735212+0000","last_fullsized":"2024-07-17T08:36:33.735212+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754676+0000","last_change":"2024-07-17T08:36:33.749435+0000","last_active":"2024-07-17T08:36:33.754676+0000","last_peered":"2024-07-17T08:36:33.754676+0000","last_clean":"2024-07-17T08:36:33.754676+0000","last_became_active":"2024-07-17T08:36:33.749168+0000","last_became_peered":"2024-07-17T08:36:33.749168+0000","last_unstale":"2024-07-17T08:36:33.754676+0000","last_undegraded":"2024-07-17T08:36:33.754676+0000","last_fullsized":"2024-07-17T08:36:33.754676+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:55.609527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.729786+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:33.729786+0000","last_peered":"2024-07-17T08:36:33.729786+0000","last_clean":"2024-07-17T08:36:33.729786+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:33.729786+0000","last_undegraded":"2024-07-17T08:36:33.729786+0000","last_fullsized":"2024-07-17T08:36:33.729786+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.738574+0000","last_change":"2024-07-17T08:36:33.738574+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.738574+0000","last_undegraded":"2024-07-17T08:36:33.738574+0000","last_fullsized":"2024-07-17T08:36:33.738574+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757878+0000","last_change":"2024-07-17T08:36:33.754851+0000","last_active":"2024-07-17T08:36:33.757878+0000","last_peered":"2024-07-17T08:36:33.757878+0000","last_clean":"2024-07-17T08:36:33.757878+0000","last_became_active":"2024-07-17T08:36:33.754191+0000","last_became_peered":"2024-07-17T08:36:33.754191+0000","last_unstale":"2024-07-17T08:36:33.757878+0000","last_undegraded":"2024-07-17T08:36:33.757878+0000","last_fullsized":"2024-07-17T08:36:33.757878+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:43:24.572407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758424+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:33.758424+0000","last_peered":"2024-07-17T08:36:33.758424+0000","last_clean":"2024-07-17T08:36:33.758424+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:33.758424+0000","last_undegraded":"2024-07-17T08:36:33.758424+0000","last_fullsized":"2024-07-17T08:36:33.758424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735656+0000","last_change":"2024-07-17T08:36:33.735656+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735656+0000","last_undegraded":"2024-07-17T08:36:33.735656+0000","last_fullsized":"2024-07-17T08:36:33.735656+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.760529+0000","last_change":"2024-07-17T08:36:33.758968+0000","last_active":"2024-07-17T08:36:33.760529+0000","last_peered":"2024-07-17T08:36:33.760529+0000","last_clean":"2024-07-17T08:36:33.760529+0000","last_became_active":"2024-07-17T08:36:33.757393+0000","last_became_peered":"2024-07-17T08:36:33.757393+0000","last_unstale":"2024-07-17T08:36:33.760529+0000","last_undegraded":"2024-07-17T08:36:33.760529+0000","last_fullsized":"2024-07-17T08:36:33.760529+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:34.664243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735516+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:33.735516+0000","last_peered":"2024-07-17T08:36:33.735516+0000","last_clean":"2024-07-17T08:36:33.735516+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:33.735516+0000","last_undegraded":"2024-07-17T08:36:33.735516+0000","last_fullsized":"2024-07-17T08:36:33.735516+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745690+0000","last_change":"2024-07-17T08:36:33.745690+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745690+0000","last_undegraded":"2024-07-17T08:36:33.745690+0000","last_fullsized":"2024-07-17T08:36:33.745690+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.761062+0000","last_change":"2024-07-17T08:36:33.759374+0000","last_active":"2024-07-17T08:36:33.761062+0000","last_peered":"2024-07-17T08:36:33.761062+0000","last_clean":"2024-07-17T08:36:33.761062+0000","last_became_active":"2024-07-17T08:36:33.759075+0000","last_became_peered":"2024-07-17T08:36:33.759075+0000","last_unstale":"2024-07-17T08:36:33.761062+0000","last_undegraded":"2024-07-17T08:36:33.761062+0000","last_fullsized":"2024-07-17T08:36:33.761062+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:34.449961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758562+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:33.758562+0000","last_peered":"2024-07-17T08:36:33.758562+0000","last_clean":"2024-07-17T08:36:33.758562+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:33.758562+0000","last_undegraded":"2024-07-17T08:36:33.758562+0000","last_fullsized":"2024-07-17T08:36:33.758562+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744930+0000","last_change":"2024-07-17T08:36:33.744930+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744930+0000","last_undegraded":"2024-07-17T08:36:33.744930+0000","last_fullsized":"2024-07-17T08:36:33.744930+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.762453+0000","last_change":"2024-07-17T08:36:33.761952+0000","last_active":"2024-07-17T08:36:33.762453+0000","last_peered":"2024-07-17T08:36:33.762453+0000","last_clean":"2024-07-17T08:36:33.762453+0000","last_became_active":"2024-07-17T08:36:33.761707+0000","last_became_peered":"2024-07-17T08:36:33.761707+0000","last_unstale":"2024-07-17T08:36:33.762453+0000","last_undegraded":"2024-07-17T08:36:33.762453+0000","last_fullsized":"2024-07-17T08:36:33.762453+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:53.151763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740188+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:33.740188+0000","last_peered":"2024-07-17T08:36:33.740188+0000","last_clean":"2024-07-17T08:36:33.740188+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:33.740188+0000","last_undegraded":"2024-07-17T08:36:33.740188+0000","last_fullsized":"2024-07-17T08:36:33.740188+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745954+0000","last_change":"2024-07-17T08:36:33.745954+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745954+0000","last_undegraded":"2024-07-17T08:36:33.745954+0000","last_fullsized":"2024-07-17T08:36:33.745954+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.761078+0000","last_change":"2024-07-17T08:36:33.760094+0000","last_active":"2024-07-17T08:36:33.761078+0000","last_peered":"2024-07-17T08:36:33.761078+0000","last_clean":"2024-07-17T08:36:33.761078+0000","last_became_active":"2024-07-17T08:36:33.757800+0000","last_became_peered":"2024-07-17T08:36:33.757800+0000","last_unstale":"2024-07-17T08:36:33.761078+0000","last_undegraded":"2024-07-17T08:36:33.761078+0000","last_fullsized":"2024-07-17T08:36:33.761078+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:34.924754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758273+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:33.758273+0000","last_peered":"2024-07-17T08:36:33.758273+0000","last_clean":"2024-07-17T08:36:33.758273+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:33.758273+0000","last_undegraded":"2024-07-17T08:36:33.758273+0000","last_fullsized":"2024-07-17T08:36:33.758273+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.743335+0000","last_change":"2024-07-17T08:36:33.743335+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.743335+0000","last_undegraded":"2024-07-17T08:36:33.743335+0000","last_fullsized":"2024-07-17T08:36:33.743335+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754739+0000","last_change":"2024-07-17T08:36:33.750553+0000","last_active":"2024-07-17T08:36:33.754739+0000","last_peered":"2024-07-17T08:36:33.754739+0000","last_clean":"2024-07-17T08:36:33.754739+0000","last_became_active":"2024-07-17T08:36:33.750105+0000","last_became_peered":"2024-07-17T08:36:33.750105+0000","last_unstale":"2024-07-17T08:36:33.754739+0000","last_undegraded":"2024-07-17T08:36:33.754739+0000","last_fullsized":"2024-07-17T08:36:33.754739+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:22.666633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.725048+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:33.725048+0000","last_peered":"2024-07-17T08:36:33.725048+0000","last_clean":"2024-07-17T08:36:33.725048+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:33.725048+0000","last_undegraded":"2024-07-17T08:36:33.725048+0000","last_fullsized":"2024-07-17T08:36:33.725048+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741992+0000","last_change":"2024-07-17T08:36:33.741992+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741992+0000","last_undegraded":"2024-07-17T08:36:33.741992+0000","last_fullsized":"2024-07-17T08:36:33.741992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750538+0000","last_change":"2024-07-17T08:36:33.747393+0000","last_active":"2024-07-17T08:36:33.750538+0000","last_peered":"2024-07-17T08:36:33.750538+0000","last_clean":"2024-07-17T08:36:33.750538+0000","last_became_active":"2024-07-17T08:36:33.746617+0000","last_became_peered":"2024-07-17T08:36:33.746617+0000","last_unstale":"2024-07-17T08:36:33.750538+0000","last_undegraded":"2024-07-17T08:36:33.750538+0000","last_fullsized":"2024-07-17T08:36:33.750538+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:44.829224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741308+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:33.741308+0000","last_peered":"2024-07-17T08:36:33.741308+0000","last_clean":"2024-07-17T08:36:33.741308+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:33.741308+0000","last_undegraded":"2024-07-17T08:36:33.741308+0000","last_fullsized":"2024-07-17T08:36:33.741308+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728066+0000","last_change":"2024-07-17T08:36:33.728066+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728066+0000","last_undegraded":"2024-07-17T08:36:33.728066+0000","last_fullsized":"2024-07-17T08:36:33.728066+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757430+0000","last_change":"2024-07-17T08:36:33.756251+0000","last_active":"2024-07-17T08:36:33.757430+0000","last_peered":"2024-07-17T08:36:33.757430+0000","last_clean":"2024-07-17T08:36:33.757430+0000","last_became_active":"2024-07-17T08:36:33.755982+0000","last_became_peered":"2024-07-17T08:36:33.755982+0000","last_unstale":"2024-07-17T08:36:33.757430+0000","last_undegraded":"2024-07-17T08:36:33.757430+0000","last_fullsized":"2024-07-17T08:36:33.757430+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:10.183173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753830+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:33.753830+0000","last_peered":"2024-07-17T08:36:33.753830+0000","last_clean":"2024-07-17T08:36:33.753830+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:33.753830+0000","last_undegraded":"2024-07-17T08:36:33.753830+0000","last_fullsized":"2024-07-17T08:36:33.753830+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.726377+0000","last_change":"2024-07-17T08:36:33.726377+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.726377+0000","last_undegraded":"2024-07-17T08:36:33.726377+0000","last_fullsized":"2024-07-17T08:36:33.726377+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.755844+0000","last_change":"2024-07-17T08:36:33.753248+0000","last_active":"2024-07-17T08:36:33.755844+0000","last_peered":"2024-07-17T08:36:33.755844+0000","last_clean":"2024-07-17T08:36:33.755844+0000","last_became_active":"2024-07-17T08:36:33.752921+0000","last_became_peered":"2024-07-17T08:36:33.752921+0000","last_unstale":"2024-07-17T08:36:33.755844+0000","last_undegraded":"2024-07-17T08:36:33.755844+0000","last_fullsized":"2024-07-17T08:36:33.755844+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:25.764114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758519+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:33.758519+0000","last_peered":"2024-07-17T08:36:33.758519+0000","last_clean":"2024-07-17T08:36:33.758519+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:33.758519+0000","last_undegraded":"2024-07-17T08:36:33.758519+0000","last_fullsized":"2024-07-17T08:36:33.758519+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745800+0000","last_change":"2024-07-17T08:36:33.745800+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745800+0000","last_undegraded":"2024-07-17T08:36:33.745800+0000","last_fullsized":"2024-07-17T08:36:33.745800+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736181+0000","last_change":"2024-07-17T08:36:33.735145+0000","last_active":"2024-07-17T08:36:33.736181+0000","last_peered":"2024-07-17T08:36:33.736181+0000","last_clean":"2024-07-17T08:36:33.736181+0000","last_became_active":"2024-07-17T08:36:33.734773+0000","last_became_peered":"2024-07-17T08:36:33.734773+0000","last_unstale":"2024-07-17T08:36:33.736181+0000","last_undegraded":"2024-07-17T08:36:33.736181+0000","last_fullsized":"2024-07-17T08:36:33.736181+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:41.505196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752774+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:33.752774+0000","last_peered":"2024-07-17T08:36:33.752774+0000","last_clean":"2024-07-17T08:36:33.752774+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:33.752774+0000","last_undegraded":"2024-07-17T08:36:33.752774+0000","last_fullsized":"2024-07-17T08:36:33.752774+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741385+0000","last_change":"2024-07-17T08:36:33.741385+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741385+0000","last_undegraded":"2024-07-17T08:36:33.741385+0000","last_fullsized":"2024-07-17T08:36:33.741385+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.759735+0000","last_change":"2024-07-17T08:36:33.758508+0000","last_active":"2024-07-17T08:36:33.759735+0000","last_peered":"2024-07-17T08:36:33.759735+0000","last_clean":"2024-07-17T08:36:33.759735+0000","last_became_active":"2024-07-17T08:36:33.757683+0000","last_became_peered":"2024-07-17T08:36:33.757683+0000","last_unstale":"2024-07-17T08:36:33.759735+0000","last_undegraded":"2024-07-17T08:36:33.759735+0000","last_fullsized":"2024-07-17T08:36:33.759735+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:39.427439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.726775+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:33.726775+0000","last_peered":"2024-07-17T08:36:33.726775+0000","last_clean":"2024-07-17T08:36:33.726775+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:33.726775+0000","last_undegraded":"2024-07-17T08:36:33.726775+0000","last_fullsized":"2024-07-17T08:36:33.726775+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747512+0000","last_change":"2024-07-17T08:36:33.747512+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747512+0000","last_undegraded":"2024-07-17T08:36:33.747512+0000","last_fullsized":"2024-07-17T08:36:33.747512+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758842+0000","last_change":"2024-07-17T08:36:33.757914+0000","last_active":"2024-07-17T08:36:33.758842+0000","last_peered":"2024-07-17T08:36:33.758842+0000","last_clean":"2024-07-17T08:36:33.758842+0000","last_became_active":"2024-07-17T08:36:33.757588+0000","last_became_peered":"2024-07-17T08:36:33.757588+0000","last_unstale":"2024-07-17T08:36:33.758842+0000","last_undegraded":"2024-07-17T08:36:33.758842+0000","last_fullsized":"2024-07-17T08:36:33.758842+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:15.993632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747362+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:33.747362+0000","last_peered":"2024-07-17T08:36:33.747362+0000","last_clean":"2024-07-17T08:36:33.747362+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:33.747362+0000","last_undegraded":"2024-07-17T08:36:33.747362+0000","last_fullsized":"2024-07-17T08:36:33.747362+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747183+0000","last_change":"2024-07-17T08:36:33.747183+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747183+0000","last_undegraded":"2024-07-17T08:36:33.747183+0000","last_fullsized":"2024-07-17T08:36:33.747183+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757640+0000","last_change":"2024-07-17T08:36:33.756054+0000","last_active":"2024-07-17T08:36:33.757640+0000","last_peered":"2024-07-17T08:36:33.757640+0000","last_clean":"2024-07-17T08:36:33.757640+0000","last_became_active":"2024-07-17T08:36:33.755748+0000","last_became_peered":"2024-07-17T08:36:33.755748+0000","last_unstale":"2024-07-17T08:36:33.757640+0000","last_undegraded":"2024-07-17T08:36:33.757640+0000","last_fullsized":"2024-07-17T08:36:33.757640+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:52.097599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737089+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:33.737089+0000","last_peered":"2024-07-17T08:36:33.737089+0000","last_clean":"2024-07-17T08:36:33.737089+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:33.737089+0000","last_undegraded":"2024-07-17T08:36:33.737089+0000","last_fullsized":"2024-07-17T08:36:33.737089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741947+0000","last_change":"2024-07-17T08:36:33.741947+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741947+0000","last_undegraded":"2024-07-17T08:36:33.741947+0000","last_fullsized":"2024-07-17T08:36:33.741947+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757705+0000","last_change":"2024-07-17T08:36:33.756539+0000","last_active":"2024-07-17T08:36:33.757705+0000","last_peered":"2024-07-17T08:36:33.757705+0000","last_clean":"2024-07-17T08:36:33.757705+0000","last_became_active":"2024-07-17T08:36:33.755822+0000","last_became_peered":"2024-07-17T08:36:33.755822+0000","last_unstale":"2024-07-17T08:36:33.757705+0000","last_undegraded":"2024-07-17T08:36:33.757705+0000","last_fullsized":"2024-07-17T08:36:33.757705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:13.608136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748410+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:33.748410+0000","last_peered":"2024-07-17T08:36:33.748410+0000","last_clean":"2024-07-17T08:36:33.748410+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:33.748410+0000","last_undegraded":"2024-07-17T08:36:33.748410+0000","last_fullsized":"2024-07-17T08:36:33.748410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.726437+0000","last_change":"2024-07-17T08:36:33.726437+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.726437+0000","last_undegraded":"2024-07-17T08:36:33.726437+0000","last_fullsized":"2024-07-17T08:36:33.726437+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.756228+0000","last_change":"2024-07-17T08:36:33.755126+0000","last_active":"2024-07-17T08:36:33.756228+0000","last_peered":"2024-07-17T08:36:33.756228+0000","last_clean":"2024-07-17T08:36:33.756228+0000","last_became_active":"2024-07-17T08:36:33.753919+0000","last_became_peered":"2024-07-17T08:36:33.753919+0000","last_unstale":"2024-07-17T08:36:33.756228+0000","last_undegraded":"2024-07-17T08:36:33.756228+0000","last_fullsized":"2024-07-17T08:36:33.756228+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:25.943916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.726945+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:33.726945+0000","last_peered":"2024-07-17T08:36:33.726945+0000","last_clean":"2024-07-17T08:36:33.726945+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:33.726945+0000","last_undegraded":"2024-07-17T08:36:33.726945+0000","last_fullsized":"2024-07-17T08:36:33.726945+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740049+0000","last_change":"2024-07-17T08:36:33.740049+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740049+0000","last_undegraded":"2024-07-17T08:36:33.740049+0000","last_fullsized":"2024-07-17T08:36:33.740049+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.759728+0000","last_change":"2024-07-17T08:36:33.758782+0000","last_active":"2024-07-17T08:36:33.759728+0000","last_peered":"2024-07-17T08:36:33.759728+0000","last_clean":"2024-07-17T08:36:33.759728+0000","last_became_active":"2024-07-17T08:36:33.757603+0000","last_became_peered":"2024-07-17T08:36:33.757603+0000","last_unstale":"2024-07-17T08:36:33.759728+0000","last_undegraded":"2024-07-17T08:36:33.759728+0000","last_fullsized":"2024-07-17T08:36:33.759728+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:50.856604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.729891+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:33.729891+0000","last_peered":"2024-07-17T08:36:33.729891+0000","last_clean":"2024-07-17T08:36:33.729891+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:33.729891+0000","last_undegraded":"2024-07-17T08:36:33.729891+0000","last_fullsized":"2024-07-17T08:36:33.729891+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727974+0000","last_change":"2024-07-17T08:36:33.727974+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727974+0000","last_undegraded":"2024-07-17T08:36:33.727974+0000","last_fullsized":"2024-07-17T08:36:33.727974+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736441+0000","last_change":"2024-07-17T08:36:33.735347+0000","last_active":"2024-07-17T08:36:33.736441+0000","last_peered":"2024-07-17T08:36:33.736441+0000","last_clean":"2024-07-17T08:36:33.736441+0000","last_became_active":"2024-07-17T08:36:33.734974+0000","last_became_peered":"2024-07-17T08:36:33.734974+0000","last_unstale":"2024-07-17T08:36:33.736441+0000","last_undegraded":"2024-07-17T08:36:33.736441+0000","last_fullsized":"2024-07-17T08:36:33.736441+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:15.203135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.723944+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:33.723944+0000","last_peered":"2024-07-17T08:36:33.723944+0000","last_clean":"2024-07-17T08:36:33.723944+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:33.723944+0000","last_undegraded":"2024-07-17T08:36:33.723944+0000","last_fullsized":"2024-07-17T08:36:33.723944+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728509+0000","last_change":"2024-07-17T08:36:33.728509+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728509+0000","last_undegraded":"2024-07-17T08:36:33.728509+0000","last_fullsized":"2024-07-17T08:36:33.728509+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.751645+0000","last_change":"2024-07-17T08:36:33.749168+0000","last_active":"2024-07-17T08:36:33.751645+0000","last_peered":"2024-07-17T08:36:33.751645+0000","last_clean":"2024-07-17T08:36:33.751645+0000","last_became_active":"2024-07-17T08:36:33.748825+0000","last_became_peered":"2024-07-17T08:36:33.748825+0000","last_unstale":"2024-07-17T08:36:33.751645+0000","last_undegraded":"2024-07-17T08:36:33.751645+0000","last_fullsized":"2024-07-17T08:36:33.751645+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:26:34.216127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748774+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:33.748774+0000","last_peered":"2024-07-17T08:36:33.748774+0000","last_clean":"2024-07-17T08:36:33.748774+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:33.748774+0000","last_undegraded":"2024-07-17T08:36:33.748774+0000","last_fullsized":"2024-07-17T08:36:33.748774+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740842+0000","last_change":"2024-07-17T08:36:33.740842+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740842+0000","last_undegraded":"2024-07-17T08:36:33.740842+0000","last_fullsized":"2024-07-17T08:36:33.740842+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.723361+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:33.723361+0000","last_peered":"2024-07-17T08:36:33.723361+0000","last_clean":"2024-07-17T08:36:33.723361+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:33.723361+0000","last_undegraded":"2024-07-17T08:36:33.723361+0000","last_fullsized":"2024-07-17T08:36:33.723361+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":22,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749462+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:33.749462+0000","last_peered":"2024-07-17T08:36:33.749462+0000","last_clean":"2024-07-17T08:36:33.749462+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:33.749462+0000","last_undegraded":"2024-07-17T08:36:33.749462+0000","last_fullsized":"2024-07-17T08:36:33.749462+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.729271+0000","last_change":"2024-07-17T08:36:33.729271+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.729271+0000","last_undegraded":"2024-07-17T08:36:33.729271+0000","last_fullsized":"2024-07-17T08:36:33.729271+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757749+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:33.757749+0000","last_peered":"2024-07-17T08:36:33.757749+0000","last_clean":"2024-07-17T08:36:33.757749+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:33.757749+0000","last_undegraded":"2024-07-17T08:36:33.757749+0000","last_fullsized":"2024-07-17T08:36:33.757749+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757682+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:33.757682+0000","last_peered":"2024-07-17T08:36:33.757682+0000","last_clean":"2024-07-17T08:36:33.757682+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:33.757682+0000","last_undegraded":"2024-07-17T08:36:33.757682+0000","last_fullsized":"2024-07-17T08:36:33.757682+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.732219+0000","last_change":"2024-07-17T08:36:33.732219+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.732219+0000","last_undegraded":"2024-07-17T08:36:33.732219+0000","last_fullsized":"2024-07-17T08:36:33.732219+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753353+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:33.753353+0000","last_peered":"2024-07-17T08:36:33.753353+0000","last_clean":"2024-07-17T08:36:33.753353+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:33.753353+0000","last_undegraded":"2024-07-17T08:36:33.753353+0000","last_fullsized":"2024-07-17T08:36:33.753353+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.727884+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:33.727884+0000","last_peered":"2024-07-17T08:36:33.727884+0000","last_clean":"2024-07-17T08:36:33.727884+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:33.727884+0000","last_undegraded":"2024-07-17T08:36:33.727884+0000","last_fullsized":"2024-07-17T08:36:33.727884+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727296+0000","last_change":"2024-07-17T08:36:33.727296+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727296+0000","last_undegraded":"2024-07-17T08:36:33.727296+0000","last_fullsized":"2024-07-17T08:36:33.727296+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.729074+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:33.729074+0000","last_peered":"2024-07-17T08:36:33.729074+0000","last_clean":"2024-07-17T08:36:33.729074+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:33.729074+0000","last_undegraded":"2024-07-17T08:36:33.729074+0000","last_fullsized":"2024-07-17T08:36:33.729074+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757771+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:33.757771+0000","last_peered":"2024-07-17T08:36:33.757771+0000","last_clean":"2024-07-17T08:36:33.757771+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:33.757771+0000","last_undegraded":"2024-07-17T08:36:33.757771+0000","last_fullsized":"2024-07-17T08:36:33.757771+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.749442+0000","last_change":"2024-07-17T08:36:33.749442+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.749442+0000","last_undegraded":"2024-07-17T08:36:33.749442+0000","last_fullsized":"2024-07-17T08:36:33.749442+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749616+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:33.749616+0000","last_peered":"2024-07-17T08:36:33.749616+0000","last_clean":"2024-07-17T08:36:33.749616+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:33.749616+0000","last_undegraded":"2024-07-17T08:36:33.749616+0000","last_fullsized":"2024-07-17T08:36:33.749616+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.730048+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:33.730048+0000","last_peered":"2024-07-17T08:36:33.730048+0000","last_clean":"2024-07-17T08:36:33.730048+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:33.730048+0000","last_undegraded":"2024-07-17T08:36:33.730048+0000","last_fullsized":"2024-07-17T08:36:33.730048+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739783+0000","last_change":"2024-07-17T08:36:33.739783+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739783+0000","last_undegraded":"2024-07-17T08:36:33.739783+0000","last_fullsized":"2024-07-17T08:36:33.739783+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739977+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:33.739977+0000","last_peered":"2024-07-17T08:36:33.739977+0000","last_clean":"2024-07-17T08:36:33.739977+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:33.739977+0000","last_undegraded":"2024-07-17T08:36:33.739977+0000","last_fullsized":"2024-07-17T08:36:33.739977+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736947+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:33.736947+0000","last_peered":"2024-07-17T08:36:33.736947+0000","last_clean":"2024-07-17T08:36:33.736947+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:33.736947+0000","last_undegraded":"2024-07-17T08:36:33.736947+0000","last_fullsized":"2024-07-17T08:36:33.736947+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.746117+0000","last_change":"2024-07-17T08:36:33.746117+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.746117+0000","last_undegraded":"2024-07-17T08:36:33.746117+0000","last_fullsized":"2024-07-17T08:36:33.746117+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738917+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:33.738917+0000","last_peered":"2024-07-17T08:36:33.738917+0000","last_clean":"2024-07-17T08:36:33.738917+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:33.738917+0000","last_undegraded":"2024-07-17T08:36:33.738917+0000","last_fullsized":"2024-07-17T08:36:33.738917+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748964+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:33.748964+0000","last_peered":"2024-07-17T08:36:33.748964+0000","last_clean":"2024-07-17T08:36:33.748964+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:33.748964+0000","last_undegraded":"2024-07-17T08:36:33.748964+0000","last_fullsized":"2024-07-17T08:36:33.748964+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744976+0000","last_change":"2024-07-17T08:36:33.744976+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744976+0000","last_undegraded":"2024-07-17T08:36:33.744976+0000","last_fullsized":"2024-07-17T08:36:33.744976+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739832+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:33.739832+0000","last_peered":"2024-07-17T08:36:33.739832+0000","last_clean":"2024-07-17T08:36:33.739832+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:33.739832+0000","last_undegraded":"2024-07-17T08:36:33.739832+0000","last_fullsized":"2024-07-17T08:36:33.739832+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.733129+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:33.733129+0000","last_peered":"2024-07-17T08:36:33.733129+0000","last_clean":"2024-07-17T08:36:33.733129+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:33.733129+0000","last_undegraded":"2024-07-17T08:36:33.733129+0000","last_fullsized":"2024-07-17T08:36:33.733129+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741478+0000","last_change":"2024-07-17T08:36:33.741478+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741478+0000","last_undegraded":"2024-07-17T08:36:33.741478+0000","last_fullsized":"2024-07-17T08:36:33.741478+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754424+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:33.754424+0000","last_peered":"2024-07-17T08:36:33.754424+0000","last_clean":"2024-07-17T08:36:33.754424+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:33.754424+0000","last_undegraded":"2024-07-17T08:36:33.754424+0000","last_fullsized":"2024-07-17T08:36:33.754424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740288+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:33.740288+0000","last_peered":"2024-07-17T08:36:33.740288+0000","last_clean":"2024-07-17T08:36:33.740288+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:33.740288+0000","last_undegraded":"2024-07-17T08:36:33.740288+0000","last_fullsized":"2024-07-17T08:36:33.740288+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739689+0000","last_change":"2024-07-17T08:36:33.739689+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739689+0000","last_undegraded":"2024-07-17T08:36:33.739689+0000","last_fullsized":"2024-07-17T08:36:33.739689+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744328+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:33.744328+0000","last_peered":"2024-07-17T08:36:33.744328+0000","last_clean":"2024-07-17T08:36:33.744328+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:33.744328+0000","last_undegraded":"2024-07-17T08:36:33.744328+0000","last_fullsized":"2024-07-17T08:36:33.744328+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":21,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737223+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:33.737223+0000","last_peered":"2024-07-17T08:36:33.737223+0000","last_clean":"2024-07-17T08:36:33.737223+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:33.737223+0000","last_undegraded":"2024-07-17T08:36:33.737223+0000","last_fullsized":"2024-07-17T08:36:33.737223+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.732693+0000","last_change":"2024-07-17T08:36:33.732693+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.732693+0000","last_undegraded":"2024-07-17T08:36:33.732693+0000","last_fullsized":"2024-07-17T08:36:33.732693+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741099+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:33.741099+0000","last_peered":"2024-07-17T08:36:33.741099+0000","last_clean":"2024-07-17T08:36:33.741099+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:33.741099+0000","last_undegraded":"2024-07-17T08:36:33.741099+0000","last_fullsized":"2024-07-17T08:36:33.741099+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":26,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748500+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:33.748500+0000","last_peered":"2024-07-17T08:36:33.748500+0000","last_clean":"2024-07-17T08:36:33.748500+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:33.748500+0000","last_undegraded":"2024-07-17T08:36:33.748500+0000","last_fullsized":"2024-07-17T08:36:33.748500+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740337+0000","last_change":"2024-07-17T08:36:33.740337+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740337+0000","last_undegraded":"2024-07-17T08:36:33.740337+0000","last_fullsized":"2024-07-17T08:36:33.740337+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.724977+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:33.724977+0000","last_peered":"2024-07-17T08:36:33.724977+0000","last_clean":"2024-07-17T08:36:33.724977+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:33.724977+0000","last_undegraded":"2024-07-17T08:36:33.724977+0000","last_fullsized":"2024-07-17T08:36:33.724977+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757876+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:33.757876+0000","last_peered":"2024-07-17T08:36:33.757876+0000","last_clean":"2024-07-17T08:36:33.757876+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:33.757876+0000","last_undegraded":"2024-07-17T08:36:33.757876+0000","last_fullsized":"2024-07-17T08:36:33.757876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.729240+0000","last_change":"2024-07-17T08:36:33.729240+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.729240+0000","last_undegraded":"2024-07-17T08:36:33.729240+0000","last_fullsized":"2024-07-17T08:36:33.729240+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749373+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:33.749373+0000","last_peered":"2024-07-17T08:36:33.749373+0000","last_clean":"2024-07-17T08:36:33.749373+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:33.749373+0000","last_undegraded":"2024-07-17T08:36:33.749373+0000","last_fullsized":"2024-07-17T08:36:33.749373+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735885+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:33.735885+0000","last_peered":"2024-07-17T08:36:33.735885+0000","last_clean":"2024-07-17T08:36:33.735885+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:33.735885+0000","last_undegraded":"2024-07-17T08:36:33.735885+0000","last_fullsized":"2024-07-17T08:36:33.735885+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747960+0000","last_change":"2024-07-17T08:36:33.747960+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747960+0000","last_undegraded":"2024-07-17T08:36:33.747960+0000","last_fullsized":"2024-07-17T08:36:33.747960+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.760565+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:33.760565+0000","last_peered":"2024-07-17T08:36:33.760565+0000","last_clean":"2024-07-17T08:36:33.760565+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:33.760565+0000","last_undegraded":"2024-07-17T08:36:33.760565+0000","last_fullsized":"2024-07-17T08:36:33.760565+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732603+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:33.732603+0000","last_peered":"2024-07-17T08:36:33.732603+0000","last_clean":"2024-07-17T08:36:33.732603+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:33.732603+0000","last_undegraded":"2024-07-17T08:36:33.732603+0000","last_fullsized":"2024-07-17T08:36:33.732603+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728163+0000","last_change":"2024-07-17T08:36:33.728163+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728163+0000","last_undegraded":"2024-07-17T08:36:33.728163+0000","last_fullsized":"2024-07-17T08:36:33.728163+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738796+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:33.738796+0000","last_peered":"2024-07-17T08:36:33.738796+0000","last_clean":"2024-07-17T08:36:33.738796+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:33.738796+0000","last_undegraded":"2024-07-17T08:36:33.738796+0000","last_fullsized":"2024-07-17T08:36:33.738796+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":26,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.723697+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:33.723697+0000","last_peered":"2024-07-17T08:36:33.723697+0000","last_clean":"2024-07-17T08:36:33.723697+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:33.723697+0000","last_undegraded":"2024-07-17T08:36:33.723697+0000","last_fullsized":"2024-07-17T08:36:33.723697+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.730718+0000","last_change":"2024-07-17T08:36:33.730718+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.730718+0000","last_undegraded":"2024-07-17T08:36:33.730718+0000","last_fullsized":"2024-07-17T08:36:33.730718+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732714+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:33.732714+0000","last_peered":"2024-07-17T08:36:33.732714+0000","last_clean":"2024-07-17T08:36:33.732714+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:33.732714+0000","last_undegraded":"2024-07-17T08:36:33.732714+0000","last_fullsized":"2024-07-17T08:36:33.732714+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749696+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:33.749696+0000","last_peered":"2024-07-17T08:36:33.749696+0000","last_clean":"2024-07-17T08:36:33.749696+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:33.749696+0000","last_undegraded":"2024-07-17T08:36:33.749696+0000","last_fullsized":"2024-07-17T08:36:33.749696+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.750091+0000","last_change":"2024-07-17T08:36:33.750091+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.750091+0000","last_undegraded":"2024-07-17T08:36:33.750091+0000","last_fullsized":"2024-07-17T08:36:33.750091+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.731581+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:33.731581+0000","last_peered":"2024-07-17T08:36:33.731581+0000","last_clean":"2024-07-17T08:36:33.731581+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:33.731581+0000","last_undegraded":"2024-07-17T08:36:33.731581+0000","last_fullsized":"2024-07-17T08:36:33.731581+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.725843+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:33.725843+0000","last_peered":"2024-07-17T08:36:33.725843+0000","last_clean":"2024-07-17T08:36:33.725843+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:33.725843+0000","last_undegraded":"2024-07-17T08:36:33.725843+0000","last_fullsized":"2024-07-17T08:36:33.725843+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.730370+0000","last_change":"2024-07-17T08:36:33.730370+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.730370+0000","last_undegraded":"2024-07-17T08:36:33.730370+0000","last_fullsized":"2024-07-17T08:36:33.730370+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.723946+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:33.723946+0000","last_peered":"2024-07-17T08:36:33.723946+0000","last_clean":"2024-07-17T08:36:33.723946+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:33.723946+0000","last_undegraded":"2024-07-17T08:36:33.723946+0000","last_fullsized":"2024-07-17T08:36:33.723946+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757835+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:33.757835+0000","last_peered":"2024-07-17T08:36:33.757835+0000","last_clean":"2024-07-17T08:36:33.757835+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:33.757835+0000","last_undegraded":"2024-07-17T08:36:33.757835+0000","last_fullsized":"2024-07-17T08:36:33.757835+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.733250+0000","last_change":"2024-07-17T08:36:33.733250+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.733250+0000","last_undegraded":"2024-07-17T08:36:33.733250+0000","last_fullsized":"2024-07-17T08:36:33.733250+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744175+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:33.744175+0000","last_peered":"2024-07-17T08:36:33.744175+0000","last_clean":"2024-07-17T08:36:33.744175+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:33.744175+0000","last_undegraded":"2024-07-17T08:36:33.744175+0000","last_fullsized":"2024-07-17T08:36:33.744175+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757866+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:33.757866+0000","last_peered":"2024-07-17T08:36:33.757866+0000","last_clean":"2024-07-17T08:36:33.757866+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:33.757866+0000","last_undegraded":"2024-07-17T08:36:33.757866+0000","last_fullsized":"2024-07-17T08:36:33.757866+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.732033+0000","last_change":"2024-07-17T08:36:33.732033+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.732033+0000","last_undegraded":"2024-07-17T08:36:33.732033+0000","last_fullsized":"2024-07-17T08:36:33.732033+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739171+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:33.739171+0000","last_peered":"2024-07-17T08:36:33.739171+0000","last_clean":"2024-07-17T08:36:33.739171+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:33.739171+0000","last_undegraded":"2024-07-17T08:36:33.739171+0000","last_fullsized":"2024-07-17T08:36:33.739171+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744670+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:33.744670+0000","last_peered":"2024-07-17T08:36:33.744670+0000","last_clean":"2024-07-17T08:36:33.744670+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:33.744670+0000","last_undegraded":"2024-07-17T08:36:33.744670+0000","last_fullsized":"2024-07-17T08:36:33.744670+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.750569+0000","last_change":"2024-07-17T08:36:33.750569+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.750569+0000","last_undegraded":"2024-07-17T08:36:33.750569+0000","last_fullsized":"2024-07-17T08:36:33.750569+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.724589+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:33.724589+0000","last_peered":"2024-07-17T08:36:33.724589+0000","last_clean":"2024-07-17T08:36:33.724589+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:33.724589+0000","last_undegraded":"2024-07-17T08:36:33.724589+0000","last_fullsized":"2024-07-17T08:36:33.724589+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754718+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:33.754718+0000","last_peered":"2024-07-17T08:36:33.754718+0000","last_clean":"2024-07-17T08:36:33.754718+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:33.754718+0000","last_undegraded":"2024-07-17T08:36:33.754718+0000","last_fullsized":"2024-07-17T08:36:33.754718+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.752613+0000","last_change":"2024-07-17T08:36:33.752613+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.752613+0000","last_undegraded":"2024-07-17T08:36:33.752613+0000","last_fullsized":"2024-07-17T08:36:33.752613+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758148+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:33.758148+0000","last_peered":"2024-07-17T08:36:33.758148+0000","last_clean":"2024-07-17T08:36:33.758148+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:33.758148+0000","last_undegraded":"2024-07-17T08:36:33.758148+0000","last_fullsized":"2024-07-17T08:36:33.758148+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754545+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:33.754545+0000","last_peered":"2024-07-17T08:36:33.754545+0000","last_clean":"2024-07-17T08:36:33.754545+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:33.754545+0000","last_undegraded":"2024-07-17T08:36:33.754545+0000","last_fullsized":"2024-07-17T08:36:33.754545+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748774+0000","last_change":"2024-07-17T08:36:33.748774+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748774+0000","last_undegraded":"2024-07-17T08:36:33.748774+0000","last_fullsized":"2024-07-17T08:36:33.748774+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758417+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:33.758417+0000","last_peered":"2024-07-17T08:36:33.758417+0000","last_clean":"2024-07-17T08:36:33.758417+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:33.758417+0000","last_undegraded":"2024-07-17T08:36:33.758417+0000","last_fullsized":"2024-07-17T08:36:33.758417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.723912+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:33.723912+0000","last_peered":"2024-07-17T08:36:33.723912+0000","last_clean":"2024-07-17T08:36:33.723912+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:33.723912+0000","last_undegraded":"2024-07-17T08:36:33.723912+0000","last_fullsized":"2024-07-17T08:36:33.723912+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741816+0000","last_change":"2024-07-17T08:36:33.741816+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741816+0000","last_undegraded":"2024-07-17T08:36:33.741816+0000","last_fullsized":"2024-07-17T08:36:33.741816+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739627+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:33.739627+0000","last_peered":"2024-07-17T08:36:33.739627+0000","last_clean":"2024-07-17T08:36:33.739627+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:33.739627+0000","last_undegraded":"2024-07-17T08:36:33.739627+0000","last_fullsized":"2024-07-17T08:36:33.739627+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738733+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:33.738733+0000","last_peered":"2024-07-17T08:36:33.738733+0000","last_clean":"2024-07-17T08:36:33.738733+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:33.738733+0000","last_undegraded":"2024-07-17T08:36:33.738733+0000","last_fullsized":"2024-07-17T08:36:33.738733+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.736575+0000","last_change":"2024-07-17T08:36:33.736575+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.736575+0000","last_undegraded":"2024-07-17T08:36:33.736575+0000","last_fullsized":"2024-07-17T08:36:33.736575+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758322+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:33.758322+0000","last_peered":"2024-07-17T08:36:33.758322+0000","last_clean":"2024-07-17T08:36:33.758322+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:33.758322+0000","last_undegraded":"2024-07-17T08:36:33.758322+0000","last_fullsized":"2024-07-17T08:36:33.758322+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737876+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:33.737876+0000","last_peered":"2024-07-17T08:36:33.737876+0000","last_clean":"2024-07-17T08:36:33.737876+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:33.737876+0000","last_undegraded":"2024-07-17T08:36:33.737876+0000","last_fullsized":"2024-07-17T08:36:33.737876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.746340+0000","last_change":"2024-07-17T08:36:33.746340+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.746340+0000","last_undegraded":"2024-07-17T08:36:33.746340+0000","last_fullsized":"2024-07-17T08:36:33.746340+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741540+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:33.741540+0000","last_peered":"2024-07-17T08:36:33.741540+0000","last_clean":"2024-07-17T08:36:33.741540+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:33.741540+0000","last_undegraded":"2024-07-17T08:36:33.741540+0000","last_fullsized":"2024-07-17T08:36:33.741540+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.727753+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:33.727753+0000","last_peered":"2024-07-17T08:36:33.727753+0000","last_clean":"2024-07-17T08:36:33.727753+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:33.727753+0000","last_undegraded":"2024-07-17T08:36:33.727753+0000","last_fullsized":"2024-07-17T08:36:33.727753+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739158+0000","last_change":"2024-07-17T08:36:33.739158+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739158+0000","last_undegraded":"2024-07-17T08:36:33.739158+0000","last_fullsized":"2024-07-17T08:36:33.739158+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736298+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:33.736298+0000","last_peered":"2024-07-17T08:36:33.736298+0000","last_clean":"2024-07-17T08:36:33.736298+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:33.736298+0000","last_undegraded":"2024-07-17T08:36:33.736298+0000","last_fullsized":"2024-07-17T08:36:33.736298+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.728196+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:33.728196+0000","last_peered":"2024-07-17T08:36:33.728196+0000","last_clean":"2024-07-17T08:36:33.728196+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:33.728196+0000","last_undegraded":"2024-07-17T08:36:33.728196+0000","last_fullsized":"2024-07-17T08:36:33.728196+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728103+0000","last_change":"2024-07-17T08:36:33.728103+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728103+0000","last_undegraded":"2024-07-17T08:36:33.728103+0000","last_fullsized":"2024-07-17T08:36:33.728103+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.727317+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:33.727317+0000","last_peered":"2024-07-17T08:36:33.727317+0000","last_clean":"2024-07-17T08:36:33.727317+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:33.727317+0000","last_undegraded":"2024-07-17T08:36:33.727317+0000","last_fullsized":"2024-07-17T08:36:33.727317+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754091+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:33.754091+0000","last_peered":"2024-07-17T08:36:33.754091+0000","last_clean":"2024-07-17T08:36:33.754091+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:33.754091+0000","last_undegraded":"2024-07-17T08:36:33.754091+0000","last_fullsized":"2024-07-17T08:36:33.754091+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748648+0000","last_change":"2024-07-17T08:36:33.748648+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748648+0000","last_undegraded":"2024-07-17T08:36:33.748648+0000","last_fullsized":"2024-07-17T08:36:33.748648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748758+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:33.748758+0000","last_peered":"2024-07-17T08:36:33.748758+0000","last_clean":"2024-07-17T08:36:33.748758+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:33.748758+0000","last_undegraded":"2024-07-17T08:36:33.748758+0000","last_fullsized":"2024-07-17T08:36:33.748758+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734495+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:33.734495+0000","last_peered":"2024-07-17T08:36:33.734495+0000","last_clean":"2024-07-17T08:36:33.734495+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:33.734495+0000","last_undegraded":"2024-07-17T08:36:33.734495+0000","last_fullsized":"2024-07-17T08:36:33.734495+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745630+0000","last_change":"2024-07-17T08:36:33.745630+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745630+0000","last_undegraded":"2024-07-17T08:36:33.745630+0000","last_fullsized":"2024-07-17T08:36:33.745630+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.725728+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:33.725728+0000","last_peered":"2024-07-17T08:36:33.725728+0000","last_clean":"2024-07-17T08:36:33.725728+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:33.725728+0000","last_undegraded":"2024-07-17T08:36:33.725728+0000","last_fullsized":"2024-07-17T08:36:33.725728+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737087+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:33.737087+0000","last_peered":"2024-07-17T08:36:33.737087+0000","last_clean":"2024-07-17T08:36:33.737087+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:33.737087+0000","last_undegraded":"2024-07-17T08:36:33.737087+0000","last_fullsized":"2024-07-17T08:36:33.737087+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.743109+0000","last_change":"2024-07-17T08:36:33.743109+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.743109+0000","last_undegraded":"2024-07-17T08:36:33.743109+0000","last_fullsized":"2024-07-17T08:36:33.743109+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758410+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:33.758410+0000","last_peered":"2024-07-17T08:36:33.758410+0000","last_clean":"2024-07-17T08:36:33.758410+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:33.758410+0000","last_undegraded":"2024-07-17T08:36:33.758410+0000","last_fullsized":"2024-07-17T08:36:33.758410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752926+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:33.752926+0000","last_peered":"2024-07-17T08:36:33.752926+0000","last_clean":"2024-07-17T08:36:33.752926+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:33.752926+0000","last_undegraded":"2024-07-17T08:36:33.752926+0000","last_fullsized":"2024-07-17T08:36:33.752926+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747615+0000","last_change":"2024-07-17T08:36:33.747615+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747615+0000","last_undegraded":"2024-07-17T08:36:33.747615+0000","last_fullsized":"2024-07-17T08:36:33.747615+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748081+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:33.748081+0000","last_peered":"2024-07-17T08:36:33.748081+0000","last_clean":"2024-07-17T08:36:33.748081+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:33.748081+0000","last_undegraded":"2024-07-17T08:36:33.748081+0000","last_fullsized":"2024-07-17T08:36:33.748081+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747574+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:33.747574+0000","last_peered":"2024-07-17T08:36:33.747574+0000","last_clean":"2024-07-17T08:36:33.747574+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:33.747574+0000","last_undegraded":"2024-07-17T08:36:33.747574+0000","last_fullsized":"2024-07-17T08:36:33.747574+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3471,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754355+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:33.754355+0000","last_peered":"2024-07-17T08:36:33.754355+0000","last_clean":"2024-07-17T08:36:33.754355+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:33.754355+0000","last_undegraded":"2024-07-17T08:36:33.754355+0000","last_fullsized":"2024-07-17T08:36:33.754355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543355,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48676,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323164,"statfs":{"total":96636764160,"available":96586919936,"internally_reserved":0,"allocated":3690496,"data_stored":2279383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510650,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48080,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323760,"statfs":{"total":96636764160,"available":96587530240,"internally_reserved":0,"allocated":3084288,"data_stored":1680899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52248,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319592,"statfs":{"total":96636764160,"available":96583262208,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543356,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52236,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319604,"statfs":{"total":96636764160,"available":96583274496,"internally_reserved":0,"allocated":3080192,"data_stored":1680857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510650,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52256,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3100672,"data_stored":1684283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543354,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52256,"kb_used_data":3024,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3096576,"data_stored":1689543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510651,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48084,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323756,"statfs":{"total":96636764160,"available":96587526144,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510650,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56984,"kb_used_data":3596,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314856,"statfs":{"total":96636764160,"available":96578412544,"internally_reserved":0,"allocated":3682304,"data_stored":2274013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:39.290 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 -- 172.21.15.44:0/2707300060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d009b6e0 msgr2=0x7f30d00cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 --2- 172.21.15.44:0/2707300060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d009b6e0 0x7f30d00cf380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f30bc007350 tx=0x7f30bc02b1a0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 -- 172.21.15.44:0/2707300060 shutdown_connections 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 --2- 172.21.15.44:0/2707300060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d009b6e0 0x7f30d00cf380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 --2- 172.21.15.44:0/2707300060 >> v2:172.21.15.175:3300/0 conn(0x7f30d009b300 0x7f30d006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 -- 172.21.15.44:0/2707300060 >> 172.21.15.44:0/2707300060 conn(0x7f30d00b0e80 msgr2=0x7f30d00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 -- 172.21.15.44:0/2707300060 shutdown_connections 2024-07-17T08:36:39.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.536+0000 7f30d5600640 1 -- 172.21.15.44:0/2707300060 wait complete. 2024-07-17T08:36:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.538+0000 7f30d5600640 1 Processor -- start 2024-07-17T08:36:39.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.538+0000 7f30d5600640 1 -- start start 2024-07-17T08:36:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30d009b300 0x7f30d01e8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d00cf8c0 0x7f30d00adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00b2120 con 0x7f30d009b6e0 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30d00b1fa0 con 0x7f30d009b300 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.539+0000 7f30d5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00b22a0 con 0x7f30d00cf8c0 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54190/0 (socket says 172.21.15.44:54190) 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 -- 172.21.15.44:0/3187569258 learned_addr learned my addr 172.21.15.44:0/3187569258 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cfe00640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30d009b300 0x7f30d01e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30d4c00640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d00cf8c0 0x7f30d00adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 -- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d00cf8c0 msgr2=0x7f30d00adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d00cf8c0 0x7f30d00adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 -- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30d009b300 msgr2=0x7f30d01e8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30d009b300 0x7f30d01e8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.540+0000 7f30cf400640 1 -- 172.21.15.44:0/3187569258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d01e8670 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30cf400640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f30bc02b790 tx=0x7f30bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc042070 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30bc007c60 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc005c90 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d01e73d0 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.541+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d01e7c10 con 0x7f30d009b6e0 2024-07-17T08:36:39.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.542+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d01e75b0 con 0x7f30d009b6e0 2024-07-17T08:36:39.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.544+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f30bc03a070 con 0x7f30d009b6e0 2024-07-17T08:36:39.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.545+0000 7f30c7e00640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 0x7f30b00792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:39.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.545+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f30bc0b8b00 con 0x7f30d009b6e0 2024-07-17T08:36:39.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.546+0000 7f30cfe00640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 0x7f30b00792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:39.548 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T08:36:39.547+0000 7f6a35c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:36:39.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.548+0000 7f30cfe00640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 0x7f30b00792e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30c0005fd0 tx=0x7f30c00095c0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:39.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.550+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f30d01e73d0 con 0x7f30d009b6e0 2024-07-17T08:36:39.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.759+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30d006ebb0 con 0x7f30b0076e20 2024-07-17T08:36:39.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.786+0000 7f30c7e00640 1 -- 172.21.15.44:0/3187569258 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+488391 (secure 0 0 0) 0x7f30d006ebb0 con 0x7f30b0076e20 2024-07-17T08:36:39.795 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:39.798 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 msgr2=0x7f30b00792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 0x7f30b00792e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30c0005fd0 tx=0x7f30c00095c0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 msgr2=0x7f30d014db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f30bc02b790 tx=0x7f30bc003c10 comp rx=0 tx=0).stop 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 shutdown_connections 2024-07-17T08:36:39.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.794+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f30b0076e20 0x7f30b00792e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30d00cf8c0 0x7f30d00adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30d009b6e0 0x7f30d014db00 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 --2- 172.21.15.44:0/3187569258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30d009b300 0x7f30d01e8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 >> 172.21.15.44:0/3187569258 conn(0x7f30d00b0e80 msgr2=0x7f30d00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 shutdown_connections 2024-07-17T08:36:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:39.795+0000 7f30d5600640 1 -- 172.21.15.44:0/3187569258 wait complete. 2024-07-17T08:36:39.818 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-07-17T08:36:39.287554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596308,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":463,"num_write_kb":1649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2593,"ondisk_log_size":2593,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":297,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410944,"kb_used_data":25428,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754563776,"statfs":{"total":773094113280,"available":772673306624,"internally_reserved":0,"allocated":26038272,"data_stored":14735588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3530,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002161"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731112+0000","last_change":"2024-07-17T08:36:34.769780+0000","last_active":"2024-07-17T08:36:35.731112+0000","last_peered":"2024-07-17T08:36:35.731112+0000","last_clean":"2024-07-17T08:36:35.731112+0000","last_became_active":"2024-07-17T08:36:34.769495+0000","last_became_peered":"2024-07-17T08:36:34.769495+0000","last_unstale":"2024-07-17T08:36:35.731112+0000","last_undegraded":"2024-07-17T08:36:35.731112+0000","last_fullsized":"2024-07-17T08:36:35.731112+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:30:35.751750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750705+0000","last_change":"2024-07-17T08:36:33.750014+0000","last_active":"2024-07-17T08:36:33.750705+0000","last_peered":"2024-07-17T08:36:33.750705+0000","last_clean":"2024-07-17T08:36:33.750705+0000","last_became_active":"2024-07-17T08:36:33.749632+0000","last_became_peered":"2024-07-17T08:36:33.749632+0000","last_unstale":"2024-07-17T08:36:33.750705+0000","last_undegraded":"2024-07-17T08:36:33.750705+0000","last_fullsized":"2024-07-17T08:36:33.750705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:12.073787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560241+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:36.560241+0000","last_peered":"2024-07-17T08:36:36.560241+0000","last_clean":"2024-07-17T08:36:36.560241+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:36.560241+0000","last_undegraded":"2024-07-17T08:36:36.560241+0000","last_fullsized":"2024-07-17T08:36:36.560241+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727993+0000","last_change":"2024-07-17T08:36:33.727993+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727993+0000","last_undegraded":"2024-07-17T08:36:33.727993+0000","last_fullsized":"2024-07-17T08:36:33.727993+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"246'4","reported_seq":18,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731482+0000","last_change":"2024-07-17T08:36:33.758375+0000","last_active":"2024-07-17T08:36:35.731482+0000","last_peered":"2024-07-17T08:36:35.731482+0000","last_clean":"2024-07-17T08:36:35.731482+0000","last_became_active":"2024-07-17T08:36:33.758093+0000","last_became_peered":"2024-07-17T08:36:33.758093+0000","last_unstale":"2024-07-17T08:36:35.731482+0000","last_undegraded":"2024-07-17T08:36:35.731482+0000","last_fullsized":"2024-07-17T08:36:35.731482+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:20.948787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732496+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:35.732496+0000","last_peered":"2024-07-17T08:36:35.732496+0000","last_clean":"2024-07-17T08:36:35.732496+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:35.732496+0000","last_undegraded":"2024-07-17T08:36:35.732496+0000","last_fullsized":"2024-07-17T08:36:35.732496+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406884+0000","last_change":"2024-07-17T08:36:34.766332+0000","last_active":"2024-07-17T08:36:36.406884+0000","last_peered":"2024-07-17T08:36:36.406884+0000","last_clean":"2024-07-17T08:36:36.406884+0000","last_became_active":"2024-07-17T08:36:34.763769+0000","last_became_peered":"2024-07-17T08:36:34.763769+0000","last_unstale":"2024-07-17T08:36:36.406884+0000","last_undegraded":"2024-07-17T08:36:36.406884+0000","last_fullsized":"2024-07-17T08:36:36.406884+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:56.756485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748357+0000","last_change":"2024-07-17T08:36:33.746302+0000","last_active":"2024-07-17T08:36:33.748357+0000","last_peered":"2024-07-17T08:36:33.748357+0000","last_clean":"2024-07-17T08:36:33.748357+0000","last_became_active":"2024-07-17T08:36:33.745779+0000","last_became_peered":"2024-07-17T08:36:33.745779+0000","last_unstale":"2024-07-17T08:36:33.748357+0000","last_undegraded":"2024-07-17T08:36:33.748357+0000","last_fullsized":"2024-07-17T08:36:33.748357+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:43:23.866594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737655+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:33.737655+0000","last_peered":"2024-07-17T08:36:33.737655+0000","last_clean":"2024-07-17T08:36:33.737655+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:33.737655+0000","last_undegraded":"2024-07-17T08:36:33.737655+0000","last_fullsized":"2024-07-17T08:36:33.737655+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731650+0000","last_change":"2024-07-17T08:36:34.737756+0000","last_active":"2024-07-17T08:36:35.731650+0000","last_peered":"2024-07-17T08:36:35.731650+0000","last_clean":"2024-07-17T08:36:35.731650+0000","last_became_active":"2024-07-17T08:36:34.737369+0000","last_became_peered":"2024-07-17T08:36:34.737369+0000","last_unstale":"2024-07-17T08:36:35.731650+0000","last_undegraded":"2024-07-17T08:36:35.731650+0000","last_fullsized":"2024-07-17T08:36:35.731650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:30.548489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757949+0000","last_change":"2024-07-17T08:36:33.755800+0000","last_active":"2024-07-17T08:36:33.757949+0000","last_peered":"2024-07-17T08:36:33.757949+0000","last_clean":"2024-07-17T08:36:33.757949+0000","last_became_active":"2024-07-17T08:36:33.754310+0000","last_became_peered":"2024-07-17T08:36:33.754310+0000","last_unstale":"2024-07-17T08:36:33.757949+0000","last_undegraded":"2024-07-17T08:36:33.757949+0000","last_fullsized":"2024-07-17T08:36:33.757949+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739306+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:33.739306+0000","last_peered":"2024-07-17T08:36:33.739306+0000","last_clean":"2024-07-17T08:36:33.739306+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:33.739306+0000","last_undegraded":"2024-07-17T08:36:33.739306+0000","last_fullsized":"2024-07-17T08:36:33.739306+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.767946+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.767625+0000","last_became_peered":"2024-07-17T08:36:34.767625+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:12.760846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:33.747145+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:33.746842+0000","last_became_peered":"2024-07-17T08:36:33.746842+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:31.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734142+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:33.734142+0000","last_peered":"2024-07-17T08:36:33.734142+0000","last_clean":"2024-07-17T08:36:33.734142+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:33.734142+0000","last_undegraded":"2024-07-17T08:36:33.734142+0000","last_fullsized":"2024-07-17T08:36:33.734142+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404806+0000","last_change":"2024-07-17T08:36:34.770512+0000","last_active":"2024-07-17T08:36:36.404806+0000","last_peered":"2024-07-17T08:36:36.404806+0000","last_clean":"2024-07-17T08:36:36.404806+0000","last_became_active":"2024-07-17T08:36:34.770193+0000","last_became_peered":"2024-07-17T08:36:34.770193+0000","last_unstale":"2024-07-17T08:36:36.404806+0000","last_undegraded":"2024-07-17T08:36:36.404806+0000","last_fullsized":"2024-07-17T08:36:36.404806+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:56.938896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757187+0000","last_change":"2024-07-17T08:36:33.755670+0000","last_active":"2024-07-17T08:36:33.757187+0000","last_peered":"2024-07-17T08:36:33.757187+0000","last_clean":"2024-07-17T08:36:33.757187+0000","last_became_active":"2024-07-17T08:36:33.755363+0000","last_became_peered":"2024-07-17T08:36:33.755363+0000","last_unstale":"2024-07-17T08:36:33.757187+0000","last_undegraded":"2024-07-17T08:36:33.757187+0000","last_fullsized":"2024-07-17T08:36:33.757187+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:19.719037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558689+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:36.558689+0000","last_peered":"2024-07-17T08:36:36.558689+0000","last_clean":"2024-07-17T08:36:36.558689+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:36.558689+0000","last_undegraded":"2024-07-17T08:36:36.558689+0000","last_fullsized":"2024-07-17T08:36:36.558689+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558650+0000","last_change":"2024-07-17T08:36:34.757341+0000","last_active":"2024-07-17T08:36:36.558650+0000","last_peered":"2024-07-17T08:36:36.558650+0000","last_clean":"2024-07-17T08:36:36.558650+0000","last_became_active":"2024-07-17T08:36:34.755978+0000","last_became_peered":"2024-07-17T08:36:34.755978+0000","last_unstale":"2024-07-17T08:36:36.558650+0000","last_undegraded":"2024-07-17T08:36:36.558650+0000","last_fullsized":"2024-07-17T08:36:36.558650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:04:27.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753956+0000","last_change":"2024-07-17T08:36:33.748383+0000","last_active":"2024-07-17T08:36:33.753956+0000","last_peered":"2024-07-17T08:36:33.753956+0000","last_clean":"2024-07-17T08:36:33.753956+0000","last_became_active":"2024-07-17T08:36:33.748082+0000","last_became_peered":"2024-07-17T08:36:33.748082+0000","last_unstale":"2024-07-17T08:36:33.753956+0000","last_undegraded":"2024-07-17T08:36:33.753956+0000","last_fullsized":"2024-07-17T08:36:33.753956+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:08.260953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741041+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:33.741041+0000","last_peered":"2024-07-17T08:36:33.741041+0000","last_clean":"2024-07-17T08:36:33.741041+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:33.741041+0000","last_undegraded":"2024-07-17T08:36:33.741041+0000","last_fullsized":"2024-07-17T08:36:33.741041+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742246+0000","last_change":"2024-07-17T08:36:33.742246+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742246+0000","last_undegraded":"2024-07-17T08:36:33.742246+0000","last_fullsized":"2024-07-17T08:36:33.742246+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758343+0000","last_change":"2024-07-17T08:36:33.756246+0000","last_active":"2024-07-17T08:36:33.758343+0000","last_peered":"2024-07-17T08:36:33.758343+0000","last_clean":"2024-07-17T08:36:33.758343+0000","last_became_active":"2024-07-17T08:36:33.754426+0000","last_became_peered":"2024-07-17T08:36:33.754426+0000","last_unstale":"2024-07-17T08:36:33.758343+0000","last_undegraded":"2024-07-17T08:36:33.758343+0000","last_fullsized":"2024-07-17T08:36:33.758343+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:16.201072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732948+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:33.732948+0000","last_peered":"2024-07-17T08:36:33.732948+0000","last_clean":"2024-07-17T08:36:33.732948+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:33.732948+0000","last_undegraded":"2024-07-17T08:36:33.732948+0000","last_fullsized":"2024-07-17T08:36:33.732948+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731147+0000","last_change":"2024-07-17T08:36:34.757574+0000","last_active":"2024-07-17T08:36:35.731147+0000","last_peered":"2024-07-17T08:36:35.731147+0000","last_clean":"2024-07-17T08:36:35.731147+0000","last_became_active":"2024-07-17T08:36:34.756063+0000","last_became_peered":"2024-07-17T08:36:34.756063+0000","last_unstale":"2024-07-17T08:36:35.731147+0000","last_undegraded":"2024-07-17T08:36:35.731147+0000","last_fullsized":"2024-07-17T08:36:35.731147+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:45.933512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.756455+0000","last_change":"2024-07-17T08:36:33.754739+0000","last_active":"2024-07-17T08:36:33.756455+0000","last_peered":"2024-07-17T08:36:33.756455+0000","last_clean":"2024-07-17T08:36:33.756455+0000","last_became_active":"2024-07-17T08:36:33.754416+0000","last_became_peered":"2024-07-17T08:36:33.754416+0000","last_unstale":"2024-07-17T08:36:33.756455+0000","last_undegraded":"2024-07-17T08:36:33.756455+0000","last_fullsized":"2024-07-17T08:36:33.756455+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:45:53.980425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560417+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:36.560417+0000","last_peered":"2024-07-17T08:36:36.560417+0000","last_clean":"2024-07-17T08:36:36.560417+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:36.560417+0000","last_undegraded":"2024-07-17T08:36:36.560417+0000","last_fullsized":"2024-07-17T08:36:36.560417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559256+0000","last_change":"2024-07-17T08:36:34.766632+0000","last_active":"2024-07-17T08:36:36.559256+0000","last_peered":"2024-07-17T08:36:36.559256+0000","last_clean":"2024-07-17T08:36:36.559256+0000","last_became_active":"2024-07-17T08:36:34.766314+0000","last_became_peered":"2024-07-17T08:36:34.766314+0000","last_unstale":"2024-07-17T08:36:36.559256+0000","last_undegraded":"2024-07-17T08:36:36.559256+0000","last_fullsized":"2024-07-17T08:36:36.559256+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:20.443444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559692+0000","last_change":"2024-07-17T08:36:33.754818+0000","last_active":"2024-07-17T08:36:36.559692+0000","last_peered":"2024-07-17T08:36:36.559692+0000","last_clean":"2024-07-17T08:36:36.559692+0000","last_became_active":"2024-07-17T08:36:33.753426+0000","last_became_peered":"2024-07-17T08:36:33.753426+0000","last_unstale":"2024-07-17T08:36:36.559692+0000","last_undegraded":"2024-07-17T08:36:36.559692+0000","last_fullsized":"2024-07-17T08:36:36.559692+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:47.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740088+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:33.740088+0000","last_peered":"2024-07-17T08:36:33.740088+0000","last_clean":"2024-07-17T08:36:33.740088+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:33.740088+0000","last_undegraded":"2024-07-17T08:36:33.740088+0000","last_fullsized":"2024-07-17T08:36:33.740088+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748161+0000","last_change":"2024-07-17T08:36:33.748161+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748161+0000","last_undegraded":"2024-07-17T08:36:33.748161+0000","last_fullsized":"2024-07-17T08:36:33.748161+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"246'6","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730737+0000","last_change":"2024-07-17T08:36:33.751230+0000","last_active":"2024-07-17T08:36:35.730737+0000","last_peered":"2024-07-17T08:36:35.730737+0000","last_clean":"2024-07-17T08:36:35.730737+0000","last_became_active":"2024-07-17T08:36:33.750539+0000","last_became_peered":"2024-07-17T08:36:33.750539+0000","last_unstale":"2024-07-17T08:36:35.730737+0000","last_undegraded":"2024-07-17T08:36:35.730737+0000","last_fullsized":"2024-07-17T08:36:35.730737+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:35.074061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738509+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:33.738509+0000","last_peered":"2024-07-17T08:36:33.738509+0000","last_clean":"2024-07-17T08:36:33.738509+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:33.738509+0000","last_undegraded":"2024-07-17T08:36:33.738509+0000","last_fullsized":"2024-07-17T08:36:33.738509+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747898+0000","last_change":"2024-07-17T08:36:33.747898+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747898+0000","last_undegraded":"2024-07-17T08:36:33.747898+0000","last_fullsized":"2024-07-17T08:36:33.747898+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"246'3","reported_seq":17,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559354+0000","last_change":"2024-07-17T08:36:33.754294+0000","last_active":"2024-07-17T08:36:36.559354+0000","last_peered":"2024-07-17T08:36:36.559354+0000","last_clean":"2024-07-17T08:36:36.559354+0000","last_became_active":"2024-07-17T08:36:33.753457+0000","last_became_peered":"2024-07-17T08:36:33.753457+0000","last_unstale":"2024-07-17T08:36:36.559354+0000","last_undegraded":"2024-07-17T08:36:36.559354+0000","last_fullsized":"2024-07-17T08:36:36.559354+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:47.335972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559369+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:36.559369+0000","last_peered":"2024-07-17T08:36:36.559369+0000","last_clean":"2024-07-17T08:36:36.559369+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:36.559369+0000","last_undegraded":"2024-07-17T08:36:36.559369+0000","last_fullsized":"2024-07-17T08:36:36.559369+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742681+0000","last_change":"2024-07-17T08:36:33.742681+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742681+0000","last_undegraded":"2024-07-17T08:36:33.742681+0000","last_fullsized":"2024-07-17T08:36:33.742681+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558126+0000","last_change":"2024-07-17T08:36:33.758000+0000","last_active":"2024-07-17T08:36:36.558126+0000","last_peered":"2024-07-17T08:36:36.558126+0000","last_clean":"2024-07-17T08:36:36.558126+0000","last_became_active":"2024-07-17T08:36:33.757029+0000","last_became_peered":"2024-07-17T08:36:33.757029+0000","last_unstale":"2024-07-17T08:36:36.558126+0000","last_undegraded":"2024-07-17T08:36:36.558126+0000","last_fullsized":"2024-07-17T08:36:36.558126+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:48.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753985+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:33.753985+0000","last_peered":"2024-07-17T08:36:33.753985+0000","last_clean":"2024-07-17T08:36:33.753985+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:33.753985+0000","last_undegraded":"2024-07-17T08:36:33.753985+0000","last_fullsized":"2024-07-17T08:36:33.753985+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735212+0000","last_change":"2024-07-17T08:36:33.735212+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735212+0000","last_undegraded":"2024-07-17T08:36:33.735212+0000","last_fullsized":"2024-07-17T08:36:33.735212+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754676+0000","last_change":"2024-07-17T08:36:33.749435+0000","last_active":"2024-07-17T08:36:33.754676+0000","last_peered":"2024-07-17T08:36:33.754676+0000","last_clean":"2024-07-17T08:36:33.754676+0000","last_became_active":"2024-07-17T08:36:33.749168+0000","last_became_peered":"2024-07-17T08:36:33.749168+0000","last_unstale":"2024-07-17T08:36:33.754676+0000","last_undegraded":"2024-07-17T08:36:33.754676+0000","last_fullsized":"2024-07-17T08:36:33.754676+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:55.609527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406087+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:36.406087+0000","last_peered":"2024-07-17T08:36:36.406087+0000","last_clean":"2024-07-17T08:36:36.406087+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:36.406087+0000","last_undegraded":"2024-07-17T08:36:36.406087+0000","last_fullsized":"2024-07-17T08:36:36.406087+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.738574+0000","last_change":"2024-07-17T08:36:33.738574+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.738574+0000","last_undegraded":"2024-07-17T08:36:33.738574+0000","last_fullsized":"2024-07-17T08:36:33.738574+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"246'8","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557060+0000","last_change":"2024-07-17T08:36:33.754851+0000","last_active":"2024-07-17T08:36:36.557060+0000","last_peered":"2024-07-17T08:36:36.557060+0000","last_clean":"2024-07-17T08:36:36.557060+0000","last_became_active":"2024-07-17T08:36:33.754191+0000","last_became_peered":"2024-07-17T08:36:33.754191+0000","last_unstale":"2024-07-17T08:36:36.557060+0000","last_undegraded":"2024-07-17T08:36:36.557060+0000","last_fullsized":"2024-07-17T08:36:36.557060+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:11.746726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758424+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:33.758424+0000","last_peered":"2024-07-17T08:36:33.758424+0000","last_clean":"2024-07-17T08:36:33.758424+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:33.758424+0000","last_undegraded":"2024-07-17T08:36:33.758424+0000","last_fullsized":"2024-07-17T08:36:33.758424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735656+0000","last_change":"2024-07-17T08:36:33.735656+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735656+0000","last_undegraded":"2024-07-17T08:36:33.735656+0000","last_fullsized":"2024-07-17T08:36:33.735656+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557662+0000","last_change":"2024-07-17T08:36:33.758968+0000","last_active":"2024-07-17T08:36:36.557662+0000","last_peered":"2024-07-17T08:36:36.557662+0000","last_clean":"2024-07-17T08:36:36.557662+0000","last_became_active":"2024-07-17T08:36:33.757393+0000","last_became_peered":"2024-07-17T08:36:33.757393+0000","last_unstale":"2024-07-17T08:36:36.557662+0000","last_undegraded":"2024-07-17T08:36:36.557662+0000","last_fullsized":"2024-07-17T08:36:36.557662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:54:23.614399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735516+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:33.735516+0000","last_peered":"2024-07-17T08:36:33.735516+0000","last_clean":"2024-07-17T08:36:33.735516+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:33.735516+0000","last_undegraded":"2024-07-17T08:36:33.735516+0000","last_fullsized":"2024-07-17T08:36:33.735516+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745690+0000","last_change":"2024-07-17T08:36:33.745690+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745690+0000","last_undegraded":"2024-07-17T08:36:33.745690+0000","last_fullsized":"2024-07-17T08:36:33.745690+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731725+0000","last_change":"2024-07-17T08:36:33.759374+0000","last_active":"2024-07-17T08:36:35.731725+0000","last_peered":"2024-07-17T08:36:35.731725+0000","last_clean":"2024-07-17T08:36:35.731725+0000","last_became_active":"2024-07-17T08:36:33.759075+0000","last_became_peered":"2024-07-17T08:36:33.759075+0000","last_unstale":"2024-07-17T08:36:35.731725+0000","last_undegraded":"2024-07-17T08:36:35.731725+0000","last_fullsized":"2024-07-17T08:36:35.731725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758562+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:33.758562+0000","last_peered":"2024-07-17T08:36:33.758562+0000","last_clean":"2024-07-17T08:36:33.758562+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:33.758562+0000","last_undegraded":"2024-07-17T08:36:33.758562+0000","last_fullsized":"2024-07-17T08:36:33.758562+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744930+0000","last_change":"2024-07-17T08:36:33.744930+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744930+0000","last_undegraded":"2024-07-17T08:36:33.744930+0000","last_fullsized":"2024-07-17T08:36:33.744930+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.762453+0000","last_change":"2024-07-17T08:36:33.761952+0000","last_active":"2024-07-17T08:36:33.762453+0000","last_peered":"2024-07-17T08:36:33.762453+0000","last_clean":"2024-07-17T08:36:33.762453+0000","last_became_active":"2024-07-17T08:36:33.761707+0000","last_became_peered":"2024-07-17T08:36:33.761707+0000","last_unstale":"2024-07-17T08:36:33.762453+0000","last_undegraded":"2024-07-17T08:36:33.762453+0000","last_fullsized":"2024-07-17T08:36:33.762453+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:53.151763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740188+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:33.740188+0000","last_peered":"2024-07-17T08:36:33.740188+0000","last_clean":"2024-07-17T08:36:33.740188+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:33.740188+0000","last_undegraded":"2024-07-17T08:36:33.740188+0000","last_fullsized":"2024-07-17T08:36:33.740188+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745954+0000","last_change":"2024-07-17T08:36:33.745954+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745954+0000","last_undegraded":"2024-07-17T08:36:33.745954+0000","last_fullsized":"2024-07-17T08:36:33.745954+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730838+0000","last_change":"2024-07-17T08:36:33.760094+0000","last_active":"2024-07-17T08:36:35.730838+0000","last_peered":"2024-07-17T08:36:35.730838+0000","last_clean":"2024-07-17T08:36:35.730838+0000","last_became_active":"2024-07-17T08:36:33.757800+0000","last_became_peered":"2024-07-17T08:36:33.757800+0000","last_unstale":"2024-07-17T08:36:35.730838+0000","last_undegraded":"2024-07-17T08:36:35.730838+0000","last_fullsized":"2024-07-17T08:36:35.730838+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:32.367201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758273+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:33.758273+0000","last_peered":"2024-07-17T08:36:33.758273+0000","last_clean":"2024-07-17T08:36:33.758273+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:33.758273+0000","last_undegraded":"2024-07-17T08:36:33.758273+0000","last_fullsized":"2024-07-17T08:36:33.758273+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559766+0000","last_change":"2024-07-17T08:36:34.767120+0000","last_active":"2024-07-17T08:36:36.559766+0000","last_peered":"2024-07-17T08:36:36.559766+0000","last_clean":"2024-07-17T08:36:36.559766+0000","last_became_active":"2024-07-17T08:36:34.766506+0000","last_became_peered":"2024-07-17T08:36:34.766506+0000","last_unstale":"2024-07-17T08:36:36.559766+0000","last_undegraded":"2024-07-17T08:36:36.559766+0000","last_fullsized":"2024-07-17T08:36:36.559766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:50.347213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754739+0000","last_change":"2024-07-17T08:36:33.750553+0000","last_active":"2024-07-17T08:36:33.754739+0000","last_peered":"2024-07-17T08:36:33.754739+0000","last_clean":"2024-07-17T08:36:33.754739+0000","last_became_active":"2024-07-17T08:36:33.750105+0000","last_became_peered":"2024-07-17T08:36:33.750105+0000","last_unstale":"2024-07-17T08:36:33.754739+0000","last_undegraded":"2024-07-17T08:36:33.754739+0000","last_fullsized":"2024-07-17T08:36:33.754739+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:22.666633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731232+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:35.731232+0000","last_peered":"2024-07-17T08:36:35.731232+0000","last_clean":"2024-07-17T08:36:35.731232+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:35.731232+0000","last_undegraded":"2024-07-17T08:36:35.731232+0000","last_fullsized":"2024-07-17T08:36:35.731232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741992+0000","last_change":"2024-07-17T08:36:33.741992+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741992+0000","last_undegraded":"2024-07-17T08:36:33.741992+0000","last_fullsized":"2024-07-17T08:36:33.741992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750538+0000","last_change":"2024-07-17T08:36:33.747393+0000","last_active":"2024-07-17T08:36:33.750538+0000","last_peered":"2024-07-17T08:36:33.750538+0000","last_clean":"2024-07-17T08:36:33.750538+0000","last_became_active":"2024-07-17T08:36:33.746617+0000","last_became_peered":"2024-07-17T08:36:33.746617+0000","last_unstale":"2024-07-17T08:36:33.750538+0000","last_undegraded":"2024-07-17T08:36:33.750538+0000","last_fullsized":"2024-07-17T08:36:33.750538+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:44.829224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741308+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:33.741308+0000","last_peered":"2024-07-17T08:36:33.741308+0000","last_clean":"2024-07-17T08:36:33.741308+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:33.741308+0000","last_undegraded":"2024-07-17T08:36:33.741308+0000","last_fullsized":"2024-07-17T08:36:33.741308+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405145+0000","last_change":"2024-07-17T08:36:34.763807+0000","last_active":"2024-07-17T08:36:36.405145+0000","last_peered":"2024-07-17T08:36:36.405145+0000","last_clean":"2024-07-17T08:36:36.405145+0000","last_became_active":"2024-07-17T08:36:34.763502+0000","last_became_peered":"2024-07-17T08:36:34.763502+0000","last_unstale":"2024-07-17T08:36:36.405145+0000","last_undegraded":"2024-07-17T08:36:36.405145+0000","last_fullsized":"2024-07-17T08:36:36.405145+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:52.862035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757430+0000","last_change":"2024-07-17T08:36:33.756251+0000","last_active":"2024-07-17T08:36:33.757430+0000","last_peered":"2024-07-17T08:36:33.757430+0000","last_clean":"2024-07-17T08:36:33.757430+0000","last_became_active":"2024-07-17T08:36:33.755982+0000","last_became_peered":"2024-07-17T08:36:33.755982+0000","last_unstale":"2024-07-17T08:36:33.757430+0000","last_undegraded":"2024-07-17T08:36:33.757430+0000","last_fullsized":"2024-07-17T08:36:33.757430+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:10.183173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753830+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:33.753830+0000","last_peered":"2024-07-17T08:36:33.753830+0000","last_clean":"2024-07-17T08:36:33.753830+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:33.753830+0000","last_undegraded":"2024-07-17T08:36:33.753830+0000","last_fullsized":"2024-07-17T08:36:33.753830+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.761196+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.760906+0000","last_became_peered":"2024-07-17T08:36:34.760906+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:39.899314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557265+0000","last_change":"2024-07-17T08:36:33.753248+0000","last_active":"2024-07-17T08:36:36.557265+0000","last_peered":"2024-07-17T08:36:36.557265+0000","last_clean":"2024-07-17T08:36:36.557265+0000","last_became_active":"2024-07-17T08:36:33.752921+0000","last_became_peered":"2024-07-17T08:36:33.752921+0000","last_unstale":"2024-07-17T08:36:36.557265+0000","last_undegraded":"2024-07-17T08:36:36.557265+0000","last_fullsized":"2024-07-17T08:36:36.557265+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:46.641811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758519+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:33.758519+0000","last_peered":"2024-07-17T08:36:33.758519+0000","last_clean":"2024-07-17T08:36:33.758519+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:33.758519+0000","last_undegraded":"2024-07-17T08:36:33.758519+0000","last_fullsized":"2024-07-17T08:36:33.758519+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557766+0000","last_change":"2024-07-17T08:36:34.766089+0000","last_active":"2024-07-17T08:36:36.557766+0000","last_peered":"2024-07-17T08:36:36.557766+0000","last_clean":"2024-07-17T08:36:36.557766+0000","last_became_active":"2024-07-17T08:36:34.762344+0000","last_became_peered":"2024-07-17T08:36:34.762344+0000","last_unstale":"2024-07-17T08:36:36.557766+0000","last_undegraded":"2024-07-17T08:36:36.557766+0000","last_fullsized":"2024-07-17T08:36:36.557766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:43.102170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406148+0000","last_change":"2024-07-17T08:36:33.735145+0000","last_active":"2024-07-17T08:36:36.406148+0000","last_peered":"2024-07-17T08:36:36.406148+0000","last_clean":"2024-07-17T08:36:36.406148+0000","last_became_active":"2024-07-17T08:36:33.734773+0000","last_became_peered":"2024-07-17T08:36:33.734773+0000","last_unstale":"2024-07-17T08:36:36.406148+0000","last_undegraded":"2024-07-17T08:36:36.406148+0000","last_fullsized":"2024-07-17T08:36:36.406148+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:06.368749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752774+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:33.752774+0000","last_peered":"2024-07-17T08:36:33.752774+0000","last_clean":"2024-07-17T08:36:33.752774+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:33.752774+0000","last_undegraded":"2024-07-17T08:36:33.752774+0000","last_fullsized":"2024-07-17T08:36:33.752774+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741385+0000","last_change":"2024-07-17T08:36:33.741385+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741385+0000","last_undegraded":"2024-07-17T08:36:33.741385+0000","last_fullsized":"2024-07-17T08:36:33.741385+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"247'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732163+0000","last_change":"2024-07-17T08:36:33.758508+0000","last_active":"2024-07-17T08:36:35.732163+0000","last_peered":"2024-07-17T08:36:35.732163+0000","last_clean":"2024-07-17T08:36:35.732163+0000","last_became_active":"2024-07-17T08:36:33.757683+0000","last_became_peered":"2024-07-17T08:36:33.757683+0000","last_unstale":"2024-07-17T08:36:35.732163+0000","last_undegraded":"2024-07-17T08:36:35.732163+0000","last_fullsized":"2024-07-17T08:36:35.732163+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:13.951810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732029+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:35.732029+0000","last_peered":"2024-07-17T08:36:35.732029+0000","last_clean":"2024-07-17T08:36:35.732029+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:35.732029+0000","last_undegraded":"2024-07-17T08:36:35.732029+0000","last_fullsized":"2024-07-17T08:36:35.732029+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747512+0000","last_change":"2024-07-17T08:36:33.747512+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747512+0000","last_undegraded":"2024-07-17T08:36:33.747512+0000","last_fullsized":"2024-07-17T08:36:33.747512+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559978+0000","last_change":"2024-07-17T08:36:33.757914+0000","last_active":"2024-07-17T08:36:36.559978+0000","last_peered":"2024-07-17T08:36:36.559978+0000","last_clean":"2024-07-17T08:36:36.559978+0000","last_became_active":"2024-07-17T08:36:33.757588+0000","last_became_peered":"2024-07-17T08:36:33.757588+0000","last_unstale":"2024-07-17T08:36:36.559978+0000","last_undegraded":"2024-07-17T08:36:36.559978+0000","last_fullsized":"2024-07-17T08:36:36.559978+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:32.359840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747362+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:33.747362+0000","last_peered":"2024-07-17T08:36:33.747362+0000","last_clean":"2024-07-17T08:36:33.747362+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:33.747362+0000","last_undegraded":"2024-07-17T08:36:33.747362+0000","last_fullsized":"2024-07-17T08:36:33.747362+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747183+0000","last_change":"2024-07-17T08:36:33.747183+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747183+0000","last_undegraded":"2024-07-17T08:36:33.747183+0000","last_fullsized":"2024-07-17T08:36:33.747183+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757640+0000","last_change":"2024-07-17T08:36:33.756054+0000","last_active":"2024-07-17T08:36:33.757640+0000","last_peered":"2024-07-17T08:36:33.757640+0000","last_clean":"2024-07-17T08:36:33.757640+0000","last_became_active":"2024-07-17T08:36:33.755748+0000","last_became_peered":"2024-07-17T08:36:33.755748+0000","last_unstale":"2024-07-17T08:36:33.757640+0000","last_undegraded":"2024-07-17T08:36:33.757640+0000","last_fullsized":"2024-07-17T08:36:33.757640+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:52.097599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737089+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:33.737089+0000","last_peered":"2024-07-17T08:36:33.737089+0000","last_clean":"2024-07-17T08:36:33.737089+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:33.737089+0000","last_undegraded":"2024-07-17T08:36:33.737089+0000","last_fullsized":"2024-07-17T08:36:33.737089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741947+0000","last_change":"2024-07-17T08:36:33.741947+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741947+0000","last_undegraded":"2024-07-17T08:36:33.741947+0000","last_fullsized":"2024-07-17T08:36:33.741947+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757705+0000","last_change":"2024-07-17T08:36:33.756539+0000","last_active":"2024-07-17T08:36:33.757705+0000","last_peered":"2024-07-17T08:36:33.757705+0000","last_clean":"2024-07-17T08:36:33.757705+0000","last_became_active":"2024-07-17T08:36:33.755822+0000","last_became_peered":"2024-07-17T08:36:33.755822+0000","last_unstale":"2024-07-17T08:36:33.757705+0000","last_undegraded":"2024-07-17T08:36:33.757705+0000","last_fullsized":"2024-07-17T08:36:33.757705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:13.608136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559586+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:36.559586+0000","last_peered":"2024-07-17T08:36:36.559586+0000","last_clean":"2024-07-17T08:36:36.559586+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:36.559586+0000","last_undegraded":"2024-07-17T08:36:36.559586+0000","last_fullsized":"2024-07-17T08:36:36.559586+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405849+0000","last_change":"2024-07-17T08:36:34.772888+0000","last_active":"2024-07-17T08:36:36.405849+0000","last_peered":"2024-07-17T08:36:36.405849+0000","last_clean":"2024-07-17T08:36:36.405849+0000","last_became_active":"2024-07-17T08:36:34.771828+0000","last_became_peered":"2024-07-17T08:36:34.771828+0000","last_unstale":"2024-07-17T08:36:36.405849+0000","last_undegraded":"2024-07-17T08:36:36.405849+0000","last_fullsized":"2024-07-17T08:36:36.405849+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:24.603843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730262+0000","last_change":"2024-07-17T08:36:33.755126+0000","last_active":"2024-07-17T08:36:35.730262+0000","last_peered":"2024-07-17T08:36:35.730262+0000","last_clean":"2024-07-17T08:36:35.730262+0000","last_became_active":"2024-07-17T08:36:33.753919+0000","last_became_peered":"2024-07-17T08:36:33.753919+0000","last_unstale":"2024-07-17T08:36:35.730262+0000","last_undegraded":"2024-07-17T08:36:35.730262+0000","last_fullsized":"2024-07-17T08:36:35.730262+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405878+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:36.405878+0000","last_peered":"2024-07-17T08:36:36.405878+0000","last_clean":"2024-07-17T08:36:36.405878+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:36.405878+0000","last_undegraded":"2024-07-17T08:36:36.405878+0000","last_fullsized":"2024-07-17T08:36:36.405878+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557257+0000","last_change":"2024-07-17T08:36:34.757792+0000","last_active":"2024-07-17T08:36:36.557257+0000","last_peered":"2024-07-17T08:36:36.557257+0000","last_clean":"2024-07-17T08:36:36.557257+0000","last_became_active":"2024-07-17T08:36:34.757510+0000","last_became_peered":"2024-07-17T08:36:34.757510+0000","last_unstale":"2024-07-17T08:36:36.557257+0000","last_undegraded":"2024-07-17T08:36:36.557257+0000","last_fullsized":"2024-07-17T08:36:36.557257+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:05.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730760+0000","last_change":"2024-07-17T08:36:33.758782+0000","last_active":"2024-07-17T08:36:35.730760+0000","last_peered":"2024-07-17T08:36:35.730760+0000","last_clean":"2024-07-17T08:36:35.730760+0000","last_became_active":"2024-07-17T08:36:33.757603+0000","last_became_peered":"2024-07-17T08:36:33.757603+0000","last_unstale":"2024-07-17T08:36:35.730760+0000","last_undegraded":"2024-07-17T08:36:35.730760+0000","last_fullsized":"2024-07-17T08:36:35.730760+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:01.305456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727974+0000","last_change":"2024-07-17T08:36:33.727974+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727974+0000","last_undegraded":"2024-07-17T08:36:33.727974+0000","last_fullsized":"2024-07-17T08:36:33.727974+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.729622+0000","last_change":"2024-07-17T08:36:33.735347+0000","last_active":"2024-07-17T08:36:35.729622+0000","last_peered":"2024-07-17T08:36:35.729622+0000","last_clean":"2024-07-17T08:36:35.729622+0000","last_became_active":"2024-07-17T08:36:33.734974+0000","last_became_peered":"2024-07-17T08:36:33.734974+0000","last_unstale":"2024-07-17T08:36:35.729622+0000","last_undegraded":"2024-07-17T08:36:35.729622+0000","last_fullsized":"2024-07-17T08:36:35.729622+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:04.954732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730287+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:35.730287+0000","last_peered":"2024-07-17T08:36:35.730287+0000","last_clean":"2024-07-17T08:36:35.730287+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:35.730287+0000","last_undegraded":"2024-07-17T08:36:35.730287+0000","last_fullsized":"2024-07-17T08:36:35.730287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405245+0000","last_change":"2024-07-17T08:36:34.771421+0000","last_active":"2024-07-17T08:36:36.405245+0000","last_peered":"2024-07-17T08:36:36.405245+0000","last_clean":"2024-07-17T08:36:36.405245+0000","last_became_active":"2024-07-17T08:36:34.770940+0000","last_became_peered":"2024-07-17T08:36:34.770940+0000","last_unstale":"2024-07-17T08:36:36.405245+0000","last_undegraded":"2024-07-17T08:36:36.405245+0000","last_fullsized":"2024-07-17T08:36:36.405245+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:32.518091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.751645+0000","last_change":"2024-07-17T08:36:33.749168+0000","last_active":"2024-07-17T08:36:33.751645+0000","last_peered":"2024-07-17T08:36:33.751645+0000","last_clean":"2024-07-17T08:36:33.751645+0000","last_became_active":"2024-07-17T08:36:33.748825+0000","last_became_peered":"2024-07-17T08:36:33.748825+0000","last_unstale":"2024-07-17T08:36:33.751645+0000","last_undegraded":"2024-07-17T08:36:33.751645+0000","last_fullsized":"2024-07-17T08:36:33.751645+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:26:34.216127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558155+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:36.558155+0000","last_peered":"2024-07-17T08:36:36.558155+0000","last_clean":"2024-07-17T08:36:36.558155+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:36.558155+0000","last_undegraded":"2024-07-17T08:36:36.558155+0000","last_fullsized":"2024-07-17T08:36:36.558155+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740842+0000","last_change":"2024-07-17T08:36:33.740842+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740842+0000","last_undegraded":"2024-07-17T08:36:33.740842+0000","last_fullsized":"2024-07-17T08:36:33.740842+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730220+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:35.730220+0000","last_peered":"2024-07-17T08:36:35.730220+0000","last_clean":"2024-07-17T08:36:35.730220+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:35.730220+0000","last_undegraded":"2024-07-17T08:36:35.730220+0000","last_fullsized":"2024-07-17T08:36:35.730220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":26,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560226+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:36.560226+0000","last_peered":"2024-07-17T08:36:36.560226+0000","last_clean":"2024-07-17T08:36:36.560226+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:36.560226+0000","last_undegraded":"2024-07-17T08:36:36.560226+0000","last_fullsized":"2024-07-17T08:36:36.560226+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404648+0000","last_change":"2024-07-17T08:36:34.756329+0000","last_active":"2024-07-17T08:36:36.404648+0000","last_peered":"2024-07-17T08:36:36.404648+0000","last_clean":"2024-07-17T08:36:36.404648+0000","last_became_active":"2024-07-17T08:36:34.756056+0000","last_became_peered":"2024-07-17T08:36:34.756056+0000","last_unstale":"2024-07-17T08:36:36.404648+0000","last_undegraded":"2024-07-17T08:36:36.404648+0000","last_fullsized":"2024-07-17T08:36:36.404648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:00.211357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757749+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:33.757749+0000","last_peered":"2024-07-17T08:36:33.757749+0000","last_clean":"2024-07-17T08:36:33.757749+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:33.757749+0000","last_undegraded":"2024-07-17T08:36:33.757749+0000","last_fullsized":"2024-07-17T08:36:33.757749+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757682+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:33.757682+0000","last_peered":"2024-07-17T08:36:33.757682+0000","last_clean":"2024-07-17T08:36:33.757682+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:33.757682+0000","last_undegraded":"2024-07-17T08:36:33.757682+0000","last_fullsized":"2024-07-17T08:36:33.757682+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406536+0000","last_change":"2024-07-17T08:36:34.776531+0000","last_active":"2024-07-17T08:36:36.406536+0000","last_peered":"2024-07-17T08:36:36.406536+0000","last_clean":"2024-07-17T08:36:36.406536+0000","last_became_active":"2024-07-17T08:36:34.776010+0000","last_became_peered":"2024-07-17T08:36:34.776010+0000","last_unstale":"2024-07-17T08:36:36.406536+0000","last_undegraded":"2024-07-17T08:36:36.406536+0000","last_fullsized":"2024-07-17T08:36:36.406536+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:27.058250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559158+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:36.559158+0000","last_peered":"2024-07-17T08:36:36.559158+0000","last_clean":"2024-07-17T08:36:36.559158+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:36.559158+0000","last_undegraded":"2024-07-17T08:36:36.559158+0000","last_fullsized":"2024-07-17T08:36:36.559158+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406585+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:36.406585+0000","last_peered":"2024-07-17T08:36:36.406585+0000","last_clean":"2024-07-17T08:36:36.406585+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:36.406585+0000","last_undegraded":"2024-07-17T08:36:36.406585+0000","last_fullsized":"2024-07-17T08:36:36.406585+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407015+0000","last_change":"2024-07-17T08:36:34.768405+0000","last_active":"2024-07-17T08:36:36.407015+0000","last_peered":"2024-07-17T08:36:36.407015+0000","last_clean":"2024-07-17T08:36:36.407015+0000","last_became_active":"2024-07-17T08:36:34.768128+0000","last_became_peered":"2024-07-17T08:36:34.768128+0000","last_unstale":"2024-07-17T08:36:36.407015+0000","last_undegraded":"2024-07-17T08:36:36.407015+0000","last_fullsized":"2024-07-17T08:36:36.407015+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:26.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405806+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:36.405806+0000","last_peered":"2024-07-17T08:36:36.405806+0000","last_clean":"2024-07-17T08:36:36.405806+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:36.405806+0000","last_undegraded":"2024-07-17T08:36:36.405806+0000","last_fullsized":"2024-07-17T08:36:36.405806+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757771+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:33.757771+0000","last_peered":"2024-07-17T08:36:33.757771+0000","last_clean":"2024-07-17T08:36:33.757771+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:33.757771+0000","last_undegraded":"2024-07-17T08:36:33.757771+0000","last_fullsized":"2024-07-17T08:36:33.757771+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.749442+0000","last_change":"2024-07-17T08:36:33.749442+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.749442+0000","last_undegraded":"2024-07-17T08:36:33.749442+0000","last_fullsized":"2024-07-17T08:36:33.749442+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749616+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:33.749616+0000","last_peered":"2024-07-17T08:36:33.749616+0000","last_clean":"2024-07-17T08:36:33.749616+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:33.749616+0000","last_undegraded":"2024-07-17T08:36:33.749616+0000","last_fullsized":"2024-07-17T08:36:33.749616+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407040+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:36.407040+0000","last_peered":"2024-07-17T08:36:36.407040+0000","last_clean":"2024-07-17T08:36:36.407040+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:36.407040+0000","last_undegraded":"2024-07-17T08:36:36.407040+0000","last_fullsized":"2024-07-17T08:36:36.407040+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739783+0000","last_change":"2024-07-17T08:36:33.739783+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739783+0000","last_undegraded":"2024-07-17T08:36:33.739783+0000","last_fullsized":"2024-07-17T08:36:33.739783+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739977+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:33.739977+0000","last_peered":"2024-07-17T08:36:33.739977+0000","last_clean":"2024-07-17T08:36:33.739977+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:33.739977+0000","last_undegraded":"2024-07-17T08:36:33.739977+0000","last_fullsized":"2024-07-17T08:36:33.739977+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736947+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:33.736947+0000","last_peered":"2024-07-17T08:36:33.736947+0000","last_clean":"2024-07-17T08:36:33.736947+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:33.736947+0000","last_undegraded":"2024-07-17T08:36:33.736947+0000","last_fullsized":"2024-07-17T08:36:33.736947+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.746117+0000","last_change":"2024-07-17T08:36:33.746117+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.746117+0000","last_undegraded":"2024-07-17T08:36:33.746117+0000","last_fullsized":"2024-07-17T08:36:33.746117+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738917+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:33.738917+0000","last_peered":"2024-07-17T08:36:33.738917+0000","last_clean":"2024-07-17T08:36:33.738917+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:33.738917+0000","last_undegraded":"2024-07-17T08:36:33.738917+0000","last_fullsized":"2024-07-17T08:36:33.738917+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558367+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:36.558367+0000","last_peered":"2024-07-17T08:36:36.558367+0000","last_clean":"2024-07-17T08:36:36.558367+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:36.558367+0000","last_undegraded":"2024-07-17T08:36:36.558367+0000","last_fullsized":"2024-07-17T08:36:36.558367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744976+0000","last_change":"2024-07-17T08:36:33.744976+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744976+0000","last_undegraded":"2024-07-17T08:36:33.744976+0000","last_fullsized":"2024-07-17T08:36:33.744976+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739832+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:33.739832+0000","last_peered":"2024-07-17T08:36:33.739832+0000","last_clean":"2024-07-17T08:36:33.739832+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:33.739832+0000","last_undegraded":"2024-07-17T08:36:33.739832+0000","last_fullsized":"2024-07-17T08:36:33.739832+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407193+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:36.407193+0000","last_peered":"2024-07-17T08:36:36.407193+0000","last_clean":"2024-07-17T08:36:36.407193+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:36.407193+0000","last_undegraded":"2024-07-17T08:36:36.407193+0000","last_fullsized":"2024-07-17T08:36:36.407193+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741478+0000","last_change":"2024-07-17T08:36:33.741478+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741478+0000","last_undegraded":"2024-07-17T08:36:33.741478+0000","last_fullsized":"2024-07-17T08:36:33.741478+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754424+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:33.754424+0000","last_peered":"2024-07-17T08:36:33.754424+0000","last_clean":"2024-07-17T08:36:33.754424+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:33.754424+0000","last_undegraded":"2024-07-17T08:36:33.754424+0000","last_fullsized":"2024-07-17T08:36:33.754424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740288+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:33.740288+0000","last_peered":"2024-07-17T08:36:33.740288+0000","last_clean":"2024-07-17T08:36:33.740288+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:33.740288+0000","last_undegraded":"2024-07-17T08:36:33.740288+0000","last_fullsized":"2024-07-17T08:36:33.740288+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739689+0000","last_change":"2024-07-17T08:36:33.739689+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739689+0000","last_undegraded":"2024-07-17T08:36:33.739689+0000","last_fullsized":"2024-07-17T08:36:33.739689+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744328+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:33.744328+0000","last_peered":"2024-07-17T08:36:33.744328+0000","last_clean":"2024-07-17T08:36:33.744328+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:33.744328+0000","last_undegraded":"2024-07-17T08:36:33.744328+0000","last_fullsized":"2024-07-17T08:36:33.744328+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":21,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737223+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:33.737223+0000","last_peered":"2024-07-17T08:36:33.737223+0000","last_clean":"2024-07-17T08:36:33.737223+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:33.737223+0000","last_undegraded":"2024-07-17T08:36:33.737223+0000","last_fullsized":"2024-07-17T08:36:33.737223+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404878+0000","last_change":"2024-07-17T08:36:34.761187+0000","last_active":"2024-07-17T08:36:36.404878+0000","last_peered":"2024-07-17T08:36:36.404878+0000","last_clean":"2024-07-17T08:36:36.404878+0000","last_became_active":"2024-07-17T08:36:34.760833+0000","last_became_peered":"2024-07-17T08:36:34.760833+0000","last_unstale":"2024-07-17T08:36:36.404878+0000","last_undegraded":"2024-07-17T08:36:36.404878+0000","last_fullsized":"2024-07-17T08:36:36.404878+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:49.833097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741099+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:33.741099+0000","last_peered":"2024-07-17T08:36:33.741099+0000","last_clean":"2024-07-17T08:36:33.741099+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:33.741099+0000","last_undegraded":"2024-07-17T08:36:33.741099+0000","last_fullsized":"2024-07-17T08:36:33.741099+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559818+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:36.559818+0000","last_peered":"2024-07-17T08:36:36.559818+0000","last_clean":"2024-07-17T08:36:36.559818+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:36.559818+0000","last_undegraded":"2024-07-17T08:36:36.559818+0000","last_fullsized":"2024-07-17T08:36:36.559818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732279+0000","last_change":"2024-07-17T08:36:34.759236+0000","last_active":"2024-07-17T08:36:35.732279+0000","last_peered":"2024-07-17T08:36:35.732279+0000","last_clean":"2024-07-17T08:36:35.732279+0000","last_became_active":"2024-07-17T08:36:34.751420+0000","last_became_peered":"2024-07-17T08:36:34.751420+0000","last_unstale":"2024-07-17T08:36:35.732279+0000","last_undegraded":"2024-07-17T08:36:35.732279+0000","last_fullsized":"2024-07-17T08:36:35.732279+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731166+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:35.731166+0000","last_peered":"2024-07-17T08:36:35.731166+0000","last_clean":"2024-07-17T08:36:35.731166+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:35.731166+0000","last_undegraded":"2024-07-17T08:36:35.731166+0000","last_fullsized":"2024-07-17T08:36:35.731166+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757876+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:33.757876+0000","last_peered":"2024-07-17T08:36:33.757876+0000","last_clean":"2024-07-17T08:36:33.757876+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:33.757876+0000","last_undegraded":"2024-07-17T08:36:33.757876+0000","last_fullsized":"2024-07-17T08:36:33.757876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.729240+0000","last_change":"2024-07-17T08:36:33.729240+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.729240+0000","last_undegraded":"2024-07-17T08:36:33.729240+0000","last_fullsized":"2024-07-17T08:36:33.729240+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749373+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:33.749373+0000","last_peered":"2024-07-17T08:36:33.749373+0000","last_clean":"2024-07-17T08:36:33.749373+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:33.749373+0000","last_undegraded":"2024-07-17T08:36:33.749373+0000","last_fullsized":"2024-07-17T08:36:33.749373+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735885+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:33.735885+0000","last_peered":"2024-07-17T08:36:33.735885+0000","last_clean":"2024-07-17T08:36:33.735885+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:33.735885+0000","last_undegraded":"2024-07-17T08:36:33.735885+0000","last_fullsized":"2024-07-17T08:36:33.735885+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747960+0000","last_change":"2024-07-17T08:36:33.747960+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747960+0000","last_undegraded":"2024-07-17T08:36:33.747960+0000","last_fullsized":"2024-07-17T08:36:33.747960+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.760565+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:33.760565+0000","last_peered":"2024-07-17T08:36:33.760565+0000","last_clean":"2024-07-17T08:36:33.760565+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:33.760565+0000","last_undegraded":"2024-07-17T08:36:33.760565+0000","last_fullsized":"2024-07-17T08:36:33.760565+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732603+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:33.732603+0000","last_peered":"2024-07-17T08:36:33.732603+0000","last_clean":"2024-07-17T08:36:33.732603+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:33.732603+0000","last_undegraded":"2024-07-17T08:36:33.732603+0000","last_fullsized":"2024-07-17T08:36:33.732603+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728163+0000","last_change":"2024-07-17T08:36:33.728163+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728163+0000","last_undegraded":"2024-07-17T08:36:33.728163+0000","last_fullsized":"2024-07-17T08:36:33.728163+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738796+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:33.738796+0000","last_peered":"2024-07-17T08:36:33.738796+0000","last_clean":"2024-07-17T08:36:33.738796+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:33.738796+0000","last_undegraded":"2024-07-17T08:36:33.738796+0000","last_fullsized":"2024-07-17T08:36:33.738796+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730519+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:35.730519+0000","last_peered":"2024-07-17T08:36:35.730519+0000","last_clean":"2024-07-17T08:36:35.730519+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:35.730519+0000","last_undegraded":"2024-07-17T08:36:35.730519+0000","last_fullsized":"2024-07-17T08:36:35.730519+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405657+0000","last_change":"2024-07-17T08:36:34.762727+0000","last_active":"2024-07-17T08:36:36.405657+0000","last_peered":"2024-07-17T08:36:36.405657+0000","last_clean":"2024-07-17T08:36:36.405657+0000","last_became_active":"2024-07-17T08:36:34.762396+0000","last_became_peered":"2024-07-17T08:36:34.762396+0000","last_unstale":"2024-07-17T08:36:36.405657+0000","last_undegraded":"2024-07-17T08:36:36.405657+0000","last_fullsized":"2024-07-17T08:36:36.405657+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:10.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732714+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:33.732714+0000","last_peered":"2024-07-17T08:36:33.732714+0000","last_clean":"2024-07-17T08:36:33.732714+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:33.732714+0000","last_undegraded":"2024-07-17T08:36:33.732714+0000","last_fullsized":"2024-07-17T08:36:33.732714+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560433+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:36.560433+0000","last_peered":"2024-07-17T08:36:36.560433+0000","last_clean":"2024-07-17T08:36:36.560433+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:36.560433+0000","last_undegraded":"2024-07-17T08:36:36.560433+0000","last_fullsized":"2024-07-17T08:36:36.560433+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558118+0000","last_change":"2024-07-17T08:36:34.757902+0000","last_active":"2024-07-17T08:36:36.558118+0000","last_peered":"2024-07-17T08:36:36.558118+0000","last_clean":"2024-07-17T08:36:36.558118+0000","last_became_active":"2024-07-17T08:36:34.756374+0000","last_became_peered":"2024-07-17T08:36:34.756374+0000","last_unstale":"2024-07-17T08:36:36.558118+0000","last_undegraded":"2024-07-17T08:36:36.558118+0000","last_fullsized":"2024-07-17T08:36:36.558118+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:04:56.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.731581+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:33.731581+0000","last_peered":"2024-07-17T08:36:33.731581+0000","last_clean":"2024-07-17T08:36:33.731581+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:33.731581+0000","last_undegraded":"2024-07-17T08:36:33.731581+0000","last_fullsized":"2024-07-17T08:36:33.731581+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.733076+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:35.733076+0000","last_peered":"2024-07-17T08:36:35.733076+0000","last_clean":"2024-07-17T08:36:35.733076+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:35.733076+0000","last_undegraded":"2024-07-17T08:36:35.733076+0000","last_fullsized":"2024-07-17T08:36:35.733076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.730370+0000","last_change":"2024-07-17T08:36:33.730370+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.730370+0000","last_undegraded":"2024-07-17T08:36:33.730370+0000","last_fullsized":"2024-07-17T08:36:33.730370+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730514+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:35.730514+0000","last_peered":"2024-07-17T08:36:35.730514+0000","last_clean":"2024-07-17T08:36:35.730514+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:35.730514+0000","last_undegraded":"2024-07-17T08:36:35.730514+0000","last_fullsized":"2024-07-17T08:36:35.730514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757835+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:33.757835+0000","last_peered":"2024-07-17T08:36:33.757835+0000","last_clean":"2024-07-17T08:36:33.757835+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:33.757835+0000","last_undegraded":"2024-07-17T08:36:33.757835+0000","last_fullsized":"2024-07-17T08:36:33.757835+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404661+0000","last_change":"2024-07-17T08:36:34.767357+0000","last_active":"2024-07-17T08:36:36.404661+0000","last_peered":"2024-07-17T08:36:36.404661+0000","last_clean":"2024-07-17T08:36:36.404661+0000","last_became_active":"2024-07-17T08:36:34.766645+0000","last_became_peered":"2024-07-17T08:36:34.766645+0000","last_unstale":"2024-07-17T08:36:36.404661+0000","last_undegraded":"2024-07-17T08:36:36.404661+0000","last_fullsized":"2024-07-17T08:36:36.404661+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:40.714568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744175+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:33.744175+0000","last_peered":"2024-07-17T08:36:33.744175+0000","last_clean":"2024-07-17T08:36:33.744175+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:33.744175+0000","last_undegraded":"2024-07-17T08:36:33.744175+0000","last_fullsized":"2024-07-17T08:36:33.744175+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757866+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:33.757866+0000","last_peered":"2024-07-17T08:36:33.757866+0000","last_clean":"2024-07-17T08:36:33.757866+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:33.757866+0000","last_undegraded":"2024-07-17T08:36:33.757866+0000","last_fullsized":"2024-07-17T08:36:33.757866+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.732033+0000","last_change":"2024-07-17T08:36:33.732033+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.732033+0000","last_undegraded":"2024-07-17T08:36:33.732033+0000","last_fullsized":"2024-07-17T08:36:33.732033+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739171+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:33.739171+0000","last_peered":"2024-07-17T08:36:33.739171+0000","last_clean":"2024-07-17T08:36:33.739171+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:33.739171+0000","last_undegraded":"2024-07-17T08:36:33.739171+0000","last_fullsized":"2024-07-17T08:36:33.739171+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744670+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:33.744670+0000","last_peered":"2024-07-17T08:36:33.744670+0000","last_clean":"2024-07-17T08:36:33.744670+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:33.744670+0000","last_undegraded":"2024-07-17T08:36:33.744670+0000","last_fullsized":"2024-07-17T08:36:33.744670+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.750569+0000","last_change":"2024-07-17T08:36:33.750569+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.750569+0000","last_undegraded":"2024-07-17T08:36:33.750569+0000","last_fullsized":"2024-07-17T08:36:33.750569+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731483+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:35.731483+0000","last_peered":"2024-07-17T08:36:35.731483+0000","last_clean":"2024-07-17T08:36:35.731483+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:35.731483+0000","last_undegraded":"2024-07-17T08:36:35.731483+0000","last_fullsized":"2024-07-17T08:36:35.731483+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754718+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:33.754718+0000","last_peered":"2024-07-17T08:36:33.754718+0000","last_clean":"2024-07-17T08:36:33.754718+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:33.754718+0000","last_undegraded":"2024-07-17T08:36:33.754718+0000","last_fullsized":"2024-07-17T08:36:33.754718+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.752613+0000","last_change":"2024-07-17T08:36:33.752613+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.752613+0000","last_undegraded":"2024-07-17T08:36:33.752613+0000","last_fullsized":"2024-07-17T08:36:33.752613+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758148+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:33.758148+0000","last_peered":"2024-07-17T08:36:33.758148+0000","last_clean":"2024-07-17T08:36:33.758148+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:33.758148+0000","last_undegraded":"2024-07-17T08:36:33.758148+0000","last_fullsized":"2024-07-17T08:36:33.758148+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754545+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:33.754545+0000","last_peered":"2024-07-17T08:36:33.754545+0000","last_clean":"2024-07-17T08:36:33.754545+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:33.754545+0000","last_undegraded":"2024-07-17T08:36:33.754545+0000","last_fullsized":"2024-07-17T08:36:33.754545+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748774+0000","last_change":"2024-07-17T08:36:33.748774+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748774+0000","last_undegraded":"2024-07-17T08:36:33.748774+0000","last_fullsized":"2024-07-17T08:36:33.748774+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758417+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:33.758417+0000","last_peered":"2024-07-17T08:36:33.758417+0000","last_clean":"2024-07-17T08:36:33.758417+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:33.758417+0000","last_undegraded":"2024-07-17T08:36:33.758417+0000","last_fullsized":"2024-07-17T08:36:33.758417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405097+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:36.405097+0000","last_peered":"2024-07-17T08:36:36.405097+0000","last_clean":"2024-07-17T08:36:36.405097+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:36.405097+0000","last_undegraded":"2024-07-17T08:36:36.405097+0000","last_fullsized":"2024-07-17T08:36:36.405097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741816+0000","last_change":"2024-07-17T08:36:33.741816+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741816+0000","last_undegraded":"2024-07-17T08:36:33.741816+0000","last_fullsized":"2024-07-17T08:36:33.741816+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739627+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:33.739627+0000","last_peered":"2024-07-17T08:36:33.739627+0000","last_clean":"2024-07-17T08:36:33.739627+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:33.739627+0000","last_undegraded":"2024-07-17T08:36:33.739627+0000","last_fullsized":"2024-07-17T08:36:33.739627+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738733+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:33.738733+0000","last_peered":"2024-07-17T08:36:33.738733+0000","last_clean":"2024-07-17T08:36:33.738733+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:33.738733+0000","last_undegraded":"2024-07-17T08:36:33.738733+0000","last_fullsized":"2024-07-17T08:36:33.738733+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.736575+0000","last_change":"2024-07-17T08:36:33.736575+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.736575+0000","last_undegraded":"2024-07-17T08:36:33.736575+0000","last_fullsized":"2024-07-17T08:36:33.736575+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758322+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:33.758322+0000","last_peered":"2024-07-17T08:36:33.758322+0000","last_clean":"2024-07-17T08:36:33.758322+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:33.758322+0000","last_undegraded":"2024-07-17T08:36:33.758322+0000","last_fullsized":"2024-07-17T08:36:33.758322+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737876+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:33.737876+0000","last_peered":"2024-07-17T08:36:33.737876+0000","last_clean":"2024-07-17T08:36:33.737876+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:33.737876+0000","last_undegraded":"2024-07-17T08:36:33.737876+0000","last_fullsized":"2024-07-17T08:36:33.737876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557817+0000","last_change":"2024-07-17T08:36:34.739794+0000","last_active":"2024-07-17T08:36:36.557817+0000","last_peered":"2024-07-17T08:36:36.557817+0000","last_clean":"2024-07-17T08:36:36.557817+0000","last_became_active":"2024-07-17T08:36:34.739282+0000","last_became_peered":"2024-07-17T08:36:34.739282+0000","last_unstale":"2024-07-17T08:36:36.557817+0000","last_undegraded":"2024-07-17T08:36:36.557817+0000","last_fullsized":"2024-07-17T08:36:36.557817+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:18.364494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741540+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:33.741540+0000","last_peered":"2024-07-17T08:36:33.741540+0000","last_clean":"2024-07-17T08:36:33.741540+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:33.741540+0000","last_undegraded":"2024-07-17T08:36:33.741540+0000","last_fullsized":"2024-07-17T08:36:33.741540+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406502+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:36.406502+0000","last_peered":"2024-07-17T08:36:36.406502+0000","last_clean":"2024-07-17T08:36:36.406502+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:36.406502+0000","last_undegraded":"2024-07-17T08:36:36.406502+0000","last_fullsized":"2024-07-17T08:36:36.406502+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739158+0000","last_change":"2024-07-17T08:36:33.739158+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739158+0000","last_undegraded":"2024-07-17T08:36:33.739158+0000","last_fullsized":"2024-07-17T08:36:33.739158+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736298+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:33.736298+0000","last_peered":"2024-07-17T08:36:33.736298+0000","last_clean":"2024-07-17T08:36:33.736298+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:33.736298+0000","last_undegraded":"2024-07-17T08:36:33.736298+0000","last_fullsized":"2024-07-17T08:36:33.736298+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406872+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:36.406872+0000","last_peered":"2024-07-17T08:36:36.406872+0000","last_clean":"2024-07-17T08:36:36.406872+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:36.406872+0000","last_undegraded":"2024-07-17T08:36:36.406872+0000","last_fullsized":"2024-07-17T08:36:36.406872+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406507+0000","last_change":"2024-07-17T08:36:34.772879+0000","last_active":"2024-07-17T08:36:36.406507+0000","last_peered":"2024-07-17T08:36:36.406507+0000","last_clean":"2024-07-17T08:36:36.406507+0000","last_became_active":"2024-07-17T08:36:34.772086+0000","last_became_peered":"2024-07-17T08:36:34.772086+0000","last_unstale":"2024-07-17T08:36:36.406507+0000","last_undegraded":"2024-07-17T08:36:36.406507+0000","last_fullsized":"2024-07-17T08:36:36.406507+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:17.960713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405403+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:36.405403+0000","last_peered":"2024-07-17T08:36:36.405403+0000","last_clean":"2024-07-17T08:36:36.405403+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:36.405403+0000","last_undegraded":"2024-07-17T08:36:36.405403+0000","last_fullsized":"2024-07-17T08:36:36.405403+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754091+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:33.754091+0000","last_peered":"2024-07-17T08:36:33.754091+0000","last_clean":"2024-07-17T08:36:33.754091+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:33.754091+0000","last_undegraded":"2024-07-17T08:36:33.754091+0000","last_fullsized":"2024-07-17T08:36:33.754091+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748648+0000","last_change":"2024-07-17T08:36:33.748648+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748648+0000","last_undegraded":"2024-07-17T08:36:33.748648+0000","last_fullsized":"2024-07-17T08:36:33.748648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559753+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:36.559753+0000","last_peered":"2024-07-17T08:36:36.559753+0000","last_clean":"2024-07-17T08:36:36.559753+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:36.559753+0000","last_undegraded":"2024-07-17T08:36:36.559753+0000","last_fullsized":"2024-07-17T08:36:36.559753+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734495+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:33.734495+0000","last_peered":"2024-07-17T08:36:33.734495+0000","last_clean":"2024-07-17T08:36:33.734495+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:33.734495+0000","last_undegraded":"2024-07-17T08:36:33.734495+0000","last_fullsized":"2024-07-17T08:36:33.734495+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558755+0000","last_change":"2024-07-17T08:36:34.753399+0000","last_active":"2024-07-17T08:36:36.558755+0000","last_peered":"2024-07-17T08:36:36.558755+0000","last_clean":"2024-07-17T08:36:36.558755+0000","last_became_active":"2024-07-17T08:36:34.753124+0000","last_became_peered":"2024-07-17T08:36:34.753124+0000","last_unstale":"2024-07-17T08:36:36.558755+0000","last_undegraded":"2024-07-17T08:36:36.558755+0000","last_fullsized":"2024-07-17T08:36:36.558755+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:19.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732588+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:35.732588+0000","last_peered":"2024-07-17T08:36:35.732588+0000","last_clean":"2024-07-17T08:36:35.732588+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:35.732588+0000","last_undegraded":"2024-07-17T08:36:35.732588+0000","last_fullsized":"2024-07-17T08:36:35.732588+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737087+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:33.737087+0000","last_peered":"2024-07-17T08:36:33.737087+0000","last_clean":"2024-07-17T08:36:33.737087+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:33.737087+0000","last_undegraded":"2024-07-17T08:36:33.737087+0000","last_fullsized":"2024-07-17T08:36:33.737087+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.743109+0000","last_change":"2024-07-17T08:36:33.743109+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.743109+0000","last_undegraded":"2024-07-17T08:36:33.743109+0000","last_fullsized":"2024-07-17T08:36:33.743109+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758410+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:33.758410+0000","last_peered":"2024-07-17T08:36:33.758410+0000","last_clean":"2024-07-17T08:36:33.758410+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:33.758410+0000","last_undegraded":"2024-07-17T08:36:33.758410+0000","last_fullsized":"2024-07-17T08:36:33.758410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752926+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:33.752926+0000","last_peered":"2024-07-17T08:36:33.752926+0000","last_clean":"2024-07-17T08:36:33.752926+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:33.752926+0000","last_undegraded":"2024-07-17T08:36:33.752926+0000","last_fullsized":"2024-07-17T08:36:33.752926+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558684+0000","last_change":"2024-07-17T08:36:34.758236+0000","last_active":"2024-07-17T08:36:36.558684+0000","last_peered":"2024-07-17T08:36:36.558684+0000","last_clean":"2024-07-17T08:36:36.558684+0000","last_became_active":"2024-07-17T08:36:34.756897+0000","last_became_peered":"2024-07-17T08:36:34.756897+0000","last_unstale":"2024-07-17T08:36:36.558684+0000","last_undegraded":"2024-07-17T08:36:36.558684+0000","last_fullsized":"2024-07-17T08:36:36.558684+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.720969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558229+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:36.558229+0000","last_peered":"2024-07-17T08:36:36.558229+0000","last_clean":"2024-07-17T08:36:36.558229+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:36.558229+0000","last_undegraded":"2024-07-17T08:36:36.558229+0000","last_fullsized":"2024-07-17T08:36:36.558229+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747574+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:33.747574+0000","last_peered":"2024-07-17T08:36:33.747574+0000","last_clean":"2024-07-17T08:36:33.747574+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:33.747574+0000","last_undegraded":"2024-07-17T08:36:33.747574+0000","last_fullsized":"2024-07-17T08:36:33.747574+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3471,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754355+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:33.754355+0000","last_peered":"2024-07-17T08:36:33.754355+0000","last_clean":"2024-07-17T08:36:33.754355+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:33.754355+0000","last_undegraded":"2024-07-17T08:36:33.754355+0000","last_fullsized":"2024-07-17T08:36:33.754355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2284,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543355,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48676,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323164,"statfs":{"total":96636764160,"available":96586919936,"internally_reserved":0,"allocated":3690496,"data_stored":2279383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510650,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48080,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323760,"statfs":{"total":96636764160,"available":96587530240,"internally_reserved":0,"allocated":3084288,"data_stored":1680899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52248,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319592,"statfs":{"total":96636764160,"available":96583262208,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543356,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52236,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319604,"statfs":{"total":96636764160,"available":96583274496,"internally_reserved":0,"allocated":3080192,"data_stored":1680857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510651,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52296,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319544,"statfs":{"total":96636764160,"available":96583213056,"internally_reserved":0,"allocated":3141632,"data_stored":1709225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543355,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52300,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319540,"statfs":{"total":96636764160,"available":96583208960,"internally_reserved":0,"allocated":3141632,"data_stored":1708507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510651,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48084,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323756,"statfs":{"total":96636764160,"available":96587526144,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510651,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57024,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314816,"statfs":{"total":96636764160,"available":96578371584,"internally_reserved":0,"allocated":3723264,"data_stored":2298935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:39.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:39.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:36:40.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.071+0000 7f8ffe800640 1 -- 172.21.15.44:0/2311742673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80cd690 msgr2=0x7f8ff80cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.071+0000 7f8ffe800640 1 --2- 172.21.15.44:0/2311742673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80cd690 0x7f8ff80cdb20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005fd0 tx=0x7f8ff002aa90 comp rx=0 tx=0).stop 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 -- 172.21.15.44:0/2311742673 shutdown_connections 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 --2- 172.21.15.44:0/2311742673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80cd690 0x7f8ff80cdb20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 --2- 172.21.15.44:0/2311742673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80ac220 0x7f8ff80ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 -- 172.21.15.44:0/2311742673 >> 172.21.15.44:0/2311742673 conn(0x7f8ff80b0fb0 msgr2=0x7f8ff80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 -- 172.21.15.44:0/2311742673 shutdown_connections 2024-07-17T08:36:40.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.072+0000 7f8ffe800640 1 -- 172.21.15.44:0/2311742673 wait complete. 2024-07-17T08:36:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.073+0000 7f8ffe800640 1 Processor -- start 2024-07-17T08:36:40.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.074+0000 7f8ffe800640 1 -- start start 2024-07-17T08:36:40.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80ac220 0x7f8ff81f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 0x7f8ff81f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80ce1e0 con 0x7f8ff81522c0 2024-07-17T08:36:40.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80ce060 con 0x7f8ff80ac220 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ffe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff80ce360 con 0x7f8ff80cd690 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54192/0 (socket says 172.21.15.44:54192) 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ff7e00640 1 -- 172.21.15.44:0/4192991454 learned_addr learned my addr 172.21.15.44:0/4192991454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ff7400640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80ac220 0x7f8ff81f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8fefe00640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 0x7f8ff81f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.075+0000 7f8ff7e00640 1 -- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 msgr2=0x7f8ff81f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ff7e00640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 0x7f8ff81f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ff7e00640 1 -- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80ac220 msgr2=0x7f8ff81f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ff7e00640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80ac220 0x7f8ff81f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ff7e00640 1 -- 172.21.15.44:0/4192991454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff8153c50 con 0x7f8ff81522c0 2024-07-17T08:36:40.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8fefe00640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 0x7f8ff81f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:40.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ff7e00640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009d90 tx=0x7f8ff0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:40.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0046070 con 0x7f8ff81522c0 2024-07-17T08:36:40.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff0004040 con 0x7f8ff81522c0 2024-07-17T08:36:40.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004450 con 0x7f8ff81522c0 2024-07-17T08:36:40.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff8150550 con 0x7f8ff81522c0 2024-07-17T08:36:40.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.076+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff8153ef0 con 0x7f8ff81522c0 2024-07-17T08:36:40.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.078+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff8151170 con 0x7f8ff81522c0 2024-07-17T08:36:40.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.079+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f8ff0049070 con 0x7f8ff81522c0 2024-07-17T08:36:40.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.080+0000 7f8fef400640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 0x7f8fd40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.080+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f8ff00b9cf0 con 0x7f8ff81522c0 2024-07-17T08:36:40.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.080+0000 7f8ff7400640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 0x7f8fd40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.082+0000 7f8ff7400640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 0x7f8fd40792b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80015e0 tx=0x7f8fe8009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:40.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.084+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8ff0032030 con 0x7f8ff81522c0 2024-07-17T08:36:40.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.294+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ff8075550 con 0x7f8fd4076df0 2024-07-17T08:36:40.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.316+0000 7f8fef400640 1 -- 172.21.15.44:0/4192991454 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+488391 (secure 0 0 0) 0x7f8ff8075550 con 0x7f8fd4076df0 2024-07-17T08:36:40.323 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:40.328 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.326+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 msgr2=0x7f8fd40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.326+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 0x7f8fd40792b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80015e0 tx=0x7f8fe8009040 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.326+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 msgr2=0x7f8ff814f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.326+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009d90 tx=0x7f8ff0003ba0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 shutdown_connections 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f8fd4076df0 0x7f8fd40792b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ff81522c0 0x7f8ff814f8d0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ff80cd690 0x7f8ff81f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 --2- 172.21.15.44:0/4192991454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ff80ac220 0x7f8ff81f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 >> 172.21.15.44:0/4192991454 conn(0x7f8ff80b0fb0 msgr2=0x7f8ff806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:40.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 shutdown_connections 2024-07-17T08:36:40.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.327+0000 7f8ffe800640 1 -- 172.21.15.44:0/4192991454 wait complete. 2024-07-17T08:36:40.352 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-07-17T08:36:39.287554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596308,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":463,"num_write_kb":1649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2593,"ondisk_log_size":2593,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":297,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410944,"kb_used_data":25428,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754563776,"statfs":{"total":773094113280,"available":772673306624,"internally_reserved":0,"allocated":26038272,"data_stored":14735588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3530,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002161"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731112+0000","last_change":"2024-07-17T08:36:34.769780+0000","last_active":"2024-07-17T08:36:35.731112+0000","last_peered":"2024-07-17T08:36:35.731112+0000","last_clean":"2024-07-17T08:36:35.731112+0000","last_became_active":"2024-07-17T08:36:34.769495+0000","last_became_peered":"2024-07-17T08:36:34.769495+0000","last_unstale":"2024-07-17T08:36:35.731112+0000","last_undegraded":"2024-07-17T08:36:35.731112+0000","last_fullsized":"2024-07-17T08:36:35.731112+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:30:35.751750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750705+0000","last_change":"2024-07-17T08:36:33.750014+0000","last_active":"2024-07-17T08:36:33.750705+0000","last_peered":"2024-07-17T08:36:33.750705+0000","last_clean":"2024-07-17T08:36:33.750705+0000","last_became_active":"2024-07-17T08:36:33.749632+0000","last_became_peered":"2024-07-17T08:36:33.749632+0000","last_unstale":"2024-07-17T08:36:33.750705+0000","last_undegraded":"2024-07-17T08:36:33.750705+0000","last_fullsized":"2024-07-17T08:36:33.750705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:12.073787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560241+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:36.560241+0000","last_peered":"2024-07-17T08:36:36.560241+0000","last_clean":"2024-07-17T08:36:36.560241+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:36.560241+0000","last_undegraded":"2024-07-17T08:36:36.560241+0000","last_fullsized":"2024-07-17T08:36:36.560241+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727993+0000","last_change":"2024-07-17T08:36:33.727993+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727993+0000","last_undegraded":"2024-07-17T08:36:33.727993+0000","last_fullsized":"2024-07-17T08:36:33.727993+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"246'4","reported_seq":18,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731482+0000","last_change":"2024-07-17T08:36:33.758375+0000","last_active":"2024-07-17T08:36:35.731482+0000","last_peered":"2024-07-17T08:36:35.731482+0000","last_clean":"2024-07-17T08:36:35.731482+0000","last_became_active":"2024-07-17T08:36:33.758093+0000","last_became_peered":"2024-07-17T08:36:33.758093+0000","last_unstale":"2024-07-17T08:36:35.731482+0000","last_undegraded":"2024-07-17T08:36:35.731482+0000","last_fullsized":"2024-07-17T08:36:35.731482+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:20.948787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732496+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:35.732496+0000","last_peered":"2024-07-17T08:36:35.732496+0000","last_clean":"2024-07-17T08:36:35.732496+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:35.732496+0000","last_undegraded":"2024-07-17T08:36:35.732496+0000","last_fullsized":"2024-07-17T08:36:35.732496+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406884+0000","last_change":"2024-07-17T08:36:34.766332+0000","last_active":"2024-07-17T08:36:36.406884+0000","last_peered":"2024-07-17T08:36:36.406884+0000","last_clean":"2024-07-17T08:36:36.406884+0000","last_became_active":"2024-07-17T08:36:34.763769+0000","last_became_peered":"2024-07-17T08:36:34.763769+0000","last_unstale":"2024-07-17T08:36:36.406884+0000","last_undegraded":"2024-07-17T08:36:36.406884+0000","last_fullsized":"2024-07-17T08:36:36.406884+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:56.756485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.748357+0000","last_change":"2024-07-17T08:36:33.746302+0000","last_active":"2024-07-17T08:36:33.748357+0000","last_peered":"2024-07-17T08:36:33.748357+0000","last_clean":"2024-07-17T08:36:33.748357+0000","last_became_active":"2024-07-17T08:36:33.745779+0000","last_became_peered":"2024-07-17T08:36:33.745779+0000","last_unstale":"2024-07-17T08:36:33.748357+0000","last_undegraded":"2024-07-17T08:36:33.748357+0000","last_fullsized":"2024-07-17T08:36:33.748357+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:43:23.866594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737655+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:33.737655+0000","last_peered":"2024-07-17T08:36:33.737655+0000","last_clean":"2024-07-17T08:36:33.737655+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:33.737655+0000","last_undegraded":"2024-07-17T08:36:33.737655+0000","last_fullsized":"2024-07-17T08:36:33.737655+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731650+0000","last_change":"2024-07-17T08:36:34.737756+0000","last_active":"2024-07-17T08:36:35.731650+0000","last_peered":"2024-07-17T08:36:35.731650+0000","last_clean":"2024-07-17T08:36:35.731650+0000","last_became_active":"2024-07-17T08:36:34.737369+0000","last_became_peered":"2024-07-17T08:36:34.737369+0000","last_unstale":"2024-07-17T08:36:35.731650+0000","last_undegraded":"2024-07-17T08:36:35.731650+0000","last_fullsized":"2024-07-17T08:36:35.731650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:30.548489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757949+0000","last_change":"2024-07-17T08:36:33.755800+0000","last_active":"2024-07-17T08:36:33.757949+0000","last_peered":"2024-07-17T08:36:33.757949+0000","last_clean":"2024-07-17T08:36:33.757949+0000","last_became_active":"2024-07-17T08:36:33.754310+0000","last_became_peered":"2024-07-17T08:36:33.754310+0000","last_unstale":"2024-07-17T08:36:33.757949+0000","last_undegraded":"2024-07-17T08:36:33.757949+0000","last_fullsized":"2024-07-17T08:36:33.757949+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739306+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:33.739306+0000","last_peered":"2024-07-17T08:36:33.739306+0000","last_clean":"2024-07-17T08:36:33.739306+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:33.739306+0000","last_undegraded":"2024-07-17T08:36:33.739306+0000","last_fullsized":"2024-07-17T08:36:33.739306+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.767946+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.767625+0000","last_became_peered":"2024-07-17T08:36:34.767625+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:12.760846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:33.747145+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:33.746842+0000","last_became_peered":"2024-07-17T08:36:33.746842+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:31.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734142+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:33.734142+0000","last_peered":"2024-07-17T08:36:33.734142+0000","last_clean":"2024-07-17T08:36:33.734142+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:33.734142+0000","last_undegraded":"2024-07-17T08:36:33.734142+0000","last_fullsized":"2024-07-17T08:36:33.734142+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404806+0000","last_change":"2024-07-17T08:36:34.770512+0000","last_active":"2024-07-17T08:36:36.404806+0000","last_peered":"2024-07-17T08:36:36.404806+0000","last_clean":"2024-07-17T08:36:36.404806+0000","last_became_active":"2024-07-17T08:36:34.770193+0000","last_became_peered":"2024-07-17T08:36:34.770193+0000","last_unstale":"2024-07-17T08:36:36.404806+0000","last_undegraded":"2024-07-17T08:36:36.404806+0000","last_fullsized":"2024-07-17T08:36:36.404806+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:56.938896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757187+0000","last_change":"2024-07-17T08:36:33.755670+0000","last_active":"2024-07-17T08:36:33.757187+0000","last_peered":"2024-07-17T08:36:33.757187+0000","last_clean":"2024-07-17T08:36:33.757187+0000","last_became_active":"2024-07-17T08:36:33.755363+0000","last_became_peered":"2024-07-17T08:36:33.755363+0000","last_unstale":"2024-07-17T08:36:33.757187+0000","last_undegraded":"2024-07-17T08:36:33.757187+0000","last_fullsized":"2024-07-17T08:36:33.757187+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:19.719037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558689+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:36.558689+0000","last_peered":"2024-07-17T08:36:36.558689+0000","last_clean":"2024-07-17T08:36:36.558689+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:36.558689+0000","last_undegraded":"2024-07-17T08:36:36.558689+0000","last_fullsized":"2024-07-17T08:36:36.558689+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558650+0000","last_change":"2024-07-17T08:36:34.757341+0000","last_active":"2024-07-17T08:36:36.558650+0000","last_peered":"2024-07-17T08:36:36.558650+0000","last_clean":"2024-07-17T08:36:36.558650+0000","last_became_active":"2024-07-17T08:36:34.755978+0000","last_became_peered":"2024-07-17T08:36:34.755978+0000","last_unstale":"2024-07-17T08:36:36.558650+0000","last_undegraded":"2024-07-17T08:36:36.558650+0000","last_fullsized":"2024-07-17T08:36:36.558650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:04:27.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753956+0000","last_change":"2024-07-17T08:36:33.748383+0000","last_active":"2024-07-17T08:36:33.753956+0000","last_peered":"2024-07-17T08:36:33.753956+0000","last_clean":"2024-07-17T08:36:33.753956+0000","last_became_active":"2024-07-17T08:36:33.748082+0000","last_became_peered":"2024-07-17T08:36:33.748082+0000","last_unstale":"2024-07-17T08:36:33.753956+0000","last_undegraded":"2024-07-17T08:36:33.753956+0000","last_fullsized":"2024-07-17T08:36:33.753956+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:08.260953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741041+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:33.741041+0000","last_peered":"2024-07-17T08:36:33.741041+0000","last_clean":"2024-07-17T08:36:33.741041+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:33.741041+0000","last_undegraded":"2024-07-17T08:36:33.741041+0000","last_fullsized":"2024-07-17T08:36:33.741041+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742246+0000","last_change":"2024-07-17T08:36:33.742246+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742246+0000","last_undegraded":"2024-07-17T08:36:33.742246+0000","last_fullsized":"2024-07-17T08:36:33.742246+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758343+0000","last_change":"2024-07-17T08:36:33.756246+0000","last_active":"2024-07-17T08:36:33.758343+0000","last_peered":"2024-07-17T08:36:33.758343+0000","last_clean":"2024-07-17T08:36:33.758343+0000","last_became_active":"2024-07-17T08:36:33.754426+0000","last_became_peered":"2024-07-17T08:36:33.754426+0000","last_unstale":"2024-07-17T08:36:33.758343+0000","last_undegraded":"2024-07-17T08:36:33.758343+0000","last_fullsized":"2024-07-17T08:36:33.758343+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:16.201072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732948+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:33.732948+0000","last_peered":"2024-07-17T08:36:33.732948+0000","last_clean":"2024-07-17T08:36:33.732948+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:33.732948+0000","last_undegraded":"2024-07-17T08:36:33.732948+0000","last_fullsized":"2024-07-17T08:36:33.732948+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731147+0000","last_change":"2024-07-17T08:36:34.757574+0000","last_active":"2024-07-17T08:36:35.731147+0000","last_peered":"2024-07-17T08:36:35.731147+0000","last_clean":"2024-07-17T08:36:35.731147+0000","last_became_active":"2024-07-17T08:36:34.756063+0000","last_became_peered":"2024-07-17T08:36:34.756063+0000","last_unstale":"2024-07-17T08:36:35.731147+0000","last_undegraded":"2024-07-17T08:36:35.731147+0000","last_fullsized":"2024-07-17T08:36:35.731147+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:45.933512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.756455+0000","last_change":"2024-07-17T08:36:33.754739+0000","last_active":"2024-07-17T08:36:33.756455+0000","last_peered":"2024-07-17T08:36:33.756455+0000","last_clean":"2024-07-17T08:36:33.756455+0000","last_became_active":"2024-07-17T08:36:33.754416+0000","last_became_peered":"2024-07-17T08:36:33.754416+0000","last_unstale":"2024-07-17T08:36:33.756455+0000","last_undegraded":"2024-07-17T08:36:33.756455+0000","last_fullsized":"2024-07-17T08:36:33.756455+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:45:53.980425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560417+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:36.560417+0000","last_peered":"2024-07-17T08:36:36.560417+0000","last_clean":"2024-07-17T08:36:36.560417+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:36.560417+0000","last_undegraded":"2024-07-17T08:36:36.560417+0000","last_fullsized":"2024-07-17T08:36:36.560417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559256+0000","last_change":"2024-07-17T08:36:34.766632+0000","last_active":"2024-07-17T08:36:36.559256+0000","last_peered":"2024-07-17T08:36:36.559256+0000","last_clean":"2024-07-17T08:36:36.559256+0000","last_became_active":"2024-07-17T08:36:34.766314+0000","last_became_peered":"2024-07-17T08:36:34.766314+0000","last_unstale":"2024-07-17T08:36:36.559256+0000","last_undegraded":"2024-07-17T08:36:36.559256+0000","last_fullsized":"2024-07-17T08:36:36.559256+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:20.443444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559692+0000","last_change":"2024-07-17T08:36:33.754818+0000","last_active":"2024-07-17T08:36:36.559692+0000","last_peered":"2024-07-17T08:36:36.559692+0000","last_clean":"2024-07-17T08:36:36.559692+0000","last_became_active":"2024-07-17T08:36:33.753426+0000","last_became_peered":"2024-07-17T08:36:33.753426+0000","last_unstale":"2024-07-17T08:36:36.559692+0000","last_undegraded":"2024-07-17T08:36:36.559692+0000","last_fullsized":"2024-07-17T08:36:36.559692+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:47.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740088+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:33.740088+0000","last_peered":"2024-07-17T08:36:33.740088+0000","last_clean":"2024-07-17T08:36:33.740088+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:33.740088+0000","last_undegraded":"2024-07-17T08:36:33.740088+0000","last_fullsized":"2024-07-17T08:36:33.740088+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748161+0000","last_change":"2024-07-17T08:36:33.748161+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748161+0000","last_undegraded":"2024-07-17T08:36:33.748161+0000","last_fullsized":"2024-07-17T08:36:33.748161+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"246'6","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730737+0000","last_change":"2024-07-17T08:36:33.751230+0000","last_active":"2024-07-17T08:36:35.730737+0000","last_peered":"2024-07-17T08:36:35.730737+0000","last_clean":"2024-07-17T08:36:35.730737+0000","last_became_active":"2024-07-17T08:36:33.750539+0000","last_became_peered":"2024-07-17T08:36:33.750539+0000","last_unstale":"2024-07-17T08:36:35.730737+0000","last_undegraded":"2024-07-17T08:36:35.730737+0000","last_fullsized":"2024-07-17T08:36:35.730737+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:35.074061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738509+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:33.738509+0000","last_peered":"2024-07-17T08:36:33.738509+0000","last_clean":"2024-07-17T08:36:33.738509+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:33.738509+0000","last_undegraded":"2024-07-17T08:36:33.738509+0000","last_fullsized":"2024-07-17T08:36:33.738509+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747898+0000","last_change":"2024-07-17T08:36:33.747898+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747898+0000","last_undegraded":"2024-07-17T08:36:33.747898+0000","last_fullsized":"2024-07-17T08:36:33.747898+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"246'3","reported_seq":17,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559354+0000","last_change":"2024-07-17T08:36:33.754294+0000","last_active":"2024-07-17T08:36:36.559354+0000","last_peered":"2024-07-17T08:36:36.559354+0000","last_clean":"2024-07-17T08:36:36.559354+0000","last_became_active":"2024-07-17T08:36:33.753457+0000","last_became_peered":"2024-07-17T08:36:33.753457+0000","last_unstale":"2024-07-17T08:36:36.559354+0000","last_undegraded":"2024-07-17T08:36:36.559354+0000","last_fullsized":"2024-07-17T08:36:36.559354+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:47.335972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559369+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:36.559369+0000","last_peered":"2024-07-17T08:36:36.559369+0000","last_clean":"2024-07-17T08:36:36.559369+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:36.559369+0000","last_undegraded":"2024-07-17T08:36:36.559369+0000","last_fullsized":"2024-07-17T08:36:36.559369+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.742681+0000","last_change":"2024-07-17T08:36:33.742681+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.742681+0000","last_undegraded":"2024-07-17T08:36:33.742681+0000","last_fullsized":"2024-07-17T08:36:33.742681+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558126+0000","last_change":"2024-07-17T08:36:33.758000+0000","last_active":"2024-07-17T08:36:36.558126+0000","last_peered":"2024-07-17T08:36:36.558126+0000","last_clean":"2024-07-17T08:36:36.558126+0000","last_became_active":"2024-07-17T08:36:33.757029+0000","last_became_peered":"2024-07-17T08:36:33.757029+0000","last_unstale":"2024-07-17T08:36:36.558126+0000","last_undegraded":"2024-07-17T08:36:36.558126+0000","last_fullsized":"2024-07-17T08:36:36.558126+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:48.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753985+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:33.753985+0000","last_peered":"2024-07-17T08:36:33.753985+0000","last_clean":"2024-07-17T08:36:33.753985+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:33.753985+0000","last_undegraded":"2024-07-17T08:36:33.753985+0000","last_fullsized":"2024-07-17T08:36:33.753985+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735212+0000","last_change":"2024-07-17T08:36:33.735212+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735212+0000","last_undegraded":"2024-07-17T08:36:33.735212+0000","last_fullsized":"2024-07-17T08:36:33.735212+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754676+0000","last_change":"2024-07-17T08:36:33.749435+0000","last_active":"2024-07-17T08:36:33.754676+0000","last_peered":"2024-07-17T08:36:33.754676+0000","last_clean":"2024-07-17T08:36:33.754676+0000","last_became_active":"2024-07-17T08:36:33.749168+0000","last_became_peered":"2024-07-17T08:36:33.749168+0000","last_unstale":"2024-07-17T08:36:33.754676+0000","last_undegraded":"2024-07-17T08:36:33.754676+0000","last_fullsized":"2024-07-17T08:36:33.754676+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:55.609527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406087+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:36.406087+0000","last_peered":"2024-07-17T08:36:36.406087+0000","last_clean":"2024-07-17T08:36:36.406087+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:36.406087+0000","last_undegraded":"2024-07-17T08:36:36.406087+0000","last_fullsized":"2024-07-17T08:36:36.406087+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.738574+0000","last_change":"2024-07-17T08:36:33.738574+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.738574+0000","last_undegraded":"2024-07-17T08:36:33.738574+0000","last_fullsized":"2024-07-17T08:36:33.738574+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"246'8","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557060+0000","last_change":"2024-07-17T08:36:33.754851+0000","last_active":"2024-07-17T08:36:36.557060+0000","last_peered":"2024-07-17T08:36:36.557060+0000","last_clean":"2024-07-17T08:36:36.557060+0000","last_became_active":"2024-07-17T08:36:33.754191+0000","last_became_peered":"2024-07-17T08:36:33.754191+0000","last_unstale":"2024-07-17T08:36:36.557060+0000","last_undegraded":"2024-07-17T08:36:36.557060+0000","last_fullsized":"2024-07-17T08:36:36.557060+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:11.746726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758424+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:33.758424+0000","last_peered":"2024-07-17T08:36:33.758424+0000","last_clean":"2024-07-17T08:36:33.758424+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:33.758424+0000","last_undegraded":"2024-07-17T08:36:33.758424+0000","last_fullsized":"2024-07-17T08:36:33.758424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.735656+0000","last_change":"2024-07-17T08:36:33.735656+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.735656+0000","last_undegraded":"2024-07-17T08:36:33.735656+0000","last_fullsized":"2024-07-17T08:36:33.735656+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557662+0000","last_change":"2024-07-17T08:36:33.758968+0000","last_active":"2024-07-17T08:36:36.557662+0000","last_peered":"2024-07-17T08:36:36.557662+0000","last_clean":"2024-07-17T08:36:36.557662+0000","last_became_active":"2024-07-17T08:36:33.757393+0000","last_became_peered":"2024-07-17T08:36:33.757393+0000","last_unstale":"2024-07-17T08:36:36.557662+0000","last_undegraded":"2024-07-17T08:36:36.557662+0000","last_fullsized":"2024-07-17T08:36:36.557662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:54:23.614399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735516+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:33.735516+0000","last_peered":"2024-07-17T08:36:33.735516+0000","last_clean":"2024-07-17T08:36:33.735516+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:33.735516+0000","last_undegraded":"2024-07-17T08:36:33.735516+0000","last_fullsized":"2024-07-17T08:36:33.735516+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745690+0000","last_change":"2024-07-17T08:36:33.745690+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745690+0000","last_undegraded":"2024-07-17T08:36:33.745690+0000","last_fullsized":"2024-07-17T08:36:33.745690+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731725+0000","last_change":"2024-07-17T08:36:33.759374+0000","last_active":"2024-07-17T08:36:35.731725+0000","last_peered":"2024-07-17T08:36:35.731725+0000","last_clean":"2024-07-17T08:36:35.731725+0000","last_became_active":"2024-07-17T08:36:33.759075+0000","last_became_peered":"2024-07-17T08:36:33.759075+0000","last_unstale":"2024-07-17T08:36:35.731725+0000","last_undegraded":"2024-07-17T08:36:35.731725+0000","last_fullsized":"2024-07-17T08:36:35.731725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758562+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:33.758562+0000","last_peered":"2024-07-17T08:36:33.758562+0000","last_clean":"2024-07-17T08:36:33.758562+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:33.758562+0000","last_undegraded":"2024-07-17T08:36:33.758562+0000","last_fullsized":"2024-07-17T08:36:33.758562+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744930+0000","last_change":"2024-07-17T08:36:33.744930+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744930+0000","last_undegraded":"2024-07-17T08:36:33.744930+0000","last_fullsized":"2024-07-17T08:36:33.744930+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.762453+0000","last_change":"2024-07-17T08:36:33.761952+0000","last_active":"2024-07-17T08:36:33.762453+0000","last_peered":"2024-07-17T08:36:33.762453+0000","last_clean":"2024-07-17T08:36:33.762453+0000","last_became_active":"2024-07-17T08:36:33.761707+0000","last_became_peered":"2024-07-17T08:36:33.761707+0000","last_unstale":"2024-07-17T08:36:33.762453+0000","last_undegraded":"2024-07-17T08:36:33.762453+0000","last_fullsized":"2024-07-17T08:36:33.762453+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:55:53.151763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740188+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:33.740188+0000","last_peered":"2024-07-17T08:36:33.740188+0000","last_clean":"2024-07-17T08:36:33.740188+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:33.740188+0000","last_undegraded":"2024-07-17T08:36:33.740188+0000","last_fullsized":"2024-07-17T08:36:33.740188+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.745954+0000","last_change":"2024-07-17T08:36:33.745954+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.745954+0000","last_undegraded":"2024-07-17T08:36:33.745954+0000","last_fullsized":"2024-07-17T08:36:33.745954+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730838+0000","last_change":"2024-07-17T08:36:33.760094+0000","last_active":"2024-07-17T08:36:35.730838+0000","last_peered":"2024-07-17T08:36:35.730838+0000","last_clean":"2024-07-17T08:36:35.730838+0000","last_became_active":"2024-07-17T08:36:33.757800+0000","last_became_peered":"2024-07-17T08:36:33.757800+0000","last_unstale":"2024-07-17T08:36:35.730838+0000","last_undegraded":"2024-07-17T08:36:35.730838+0000","last_fullsized":"2024-07-17T08:36:35.730838+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:32.367201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758273+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:33.758273+0000","last_peered":"2024-07-17T08:36:33.758273+0000","last_clean":"2024-07-17T08:36:33.758273+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:33.758273+0000","last_undegraded":"2024-07-17T08:36:33.758273+0000","last_fullsized":"2024-07-17T08:36:33.758273+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559766+0000","last_change":"2024-07-17T08:36:34.767120+0000","last_active":"2024-07-17T08:36:36.559766+0000","last_peered":"2024-07-17T08:36:36.559766+0000","last_clean":"2024-07-17T08:36:36.559766+0000","last_became_active":"2024-07-17T08:36:34.766506+0000","last_became_peered":"2024-07-17T08:36:34.766506+0000","last_unstale":"2024-07-17T08:36:36.559766+0000","last_undegraded":"2024-07-17T08:36:36.559766+0000","last_fullsized":"2024-07-17T08:36:36.559766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:50.347213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754739+0000","last_change":"2024-07-17T08:36:33.750553+0000","last_active":"2024-07-17T08:36:33.754739+0000","last_peered":"2024-07-17T08:36:33.754739+0000","last_clean":"2024-07-17T08:36:33.754739+0000","last_became_active":"2024-07-17T08:36:33.750105+0000","last_became_peered":"2024-07-17T08:36:33.750105+0000","last_unstale":"2024-07-17T08:36:33.754739+0000","last_undegraded":"2024-07-17T08:36:33.754739+0000","last_fullsized":"2024-07-17T08:36:33.754739+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:22.666633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731232+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:35.731232+0000","last_peered":"2024-07-17T08:36:35.731232+0000","last_clean":"2024-07-17T08:36:35.731232+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:35.731232+0000","last_undegraded":"2024-07-17T08:36:35.731232+0000","last_fullsized":"2024-07-17T08:36:35.731232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741992+0000","last_change":"2024-07-17T08:36:33.741992+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741992+0000","last_undegraded":"2024-07-17T08:36:33.741992+0000","last_fullsized":"2024-07-17T08:36:33.741992+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.750538+0000","last_change":"2024-07-17T08:36:33.747393+0000","last_active":"2024-07-17T08:36:33.750538+0000","last_peered":"2024-07-17T08:36:33.750538+0000","last_clean":"2024-07-17T08:36:33.750538+0000","last_became_active":"2024-07-17T08:36:33.746617+0000","last_became_peered":"2024-07-17T08:36:33.746617+0000","last_unstale":"2024-07-17T08:36:33.750538+0000","last_undegraded":"2024-07-17T08:36:33.750538+0000","last_fullsized":"2024-07-17T08:36:33.750538+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:44.829224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741308+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:33.741308+0000","last_peered":"2024-07-17T08:36:33.741308+0000","last_clean":"2024-07-17T08:36:33.741308+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:33.741308+0000","last_undegraded":"2024-07-17T08:36:33.741308+0000","last_fullsized":"2024-07-17T08:36:33.741308+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405145+0000","last_change":"2024-07-17T08:36:34.763807+0000","last_active":"2024-07-17T08:36:36.405145+0000","last_peered":"2024-07-17T08:36:36.405145+0000","last_clean":"2024-07-17T08:36:36.405145+0000","last_became_active":"2024-07-17T08:36:34.763502+0000","last_became_peered":"2024-07-17T08:36:34.763502+0000","last_unstale":"2024-07-17T08:36:36.405145+0000","last_undegraded":"2024-07-17T08:36:36.405145+0000","last_fullsized":"2024-07-17T08:36:36.405145+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:52.862035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757430+0000","last_change":"2024-07-17T08:36:33.756251+0000","last_active":"2024-07-17T08:36:33.757430+0000","last_peered":"2024-07-17T08:36:33.757430+0000","last_clean":"2024-07-17T08:36:33.757430+0000","last_became_active":"2024-07-17T08:36:33.755982+0000","last_became_peered":"2024-07-17T08:36:33.755982+0000","last_unstale":"2024-07-17T08:36:33.757430+0000","last_undegraded":"2024-07-17T08:36:33.757430+0000","last_fullsized":"2024-07-17T08:36:33.757430+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:10.183173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.753830+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:33.753830+0000","last_peered":"2024-07-17T08:36:33.753830+0000","last_clean":"2024-07-17T08:36:33.753830+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:33.753830+0000","last_undegraded":"2024-07-17T08:36:33.753830+0000","last_fullsized":"2024-07-17T08:36:33.753830+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.761196+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.760906+0000","last_became_peered":"2024-07-17T08:36:34.760906+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:39.899314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557265+0000","last_change":"2024-07-17T08:36:33.753248+0000","last_active":"2024-07-17T08:36:36.557265+0000","last_peered":"2024-07-17T08:36:36.557265+0000","last_clean":"2024-07-17T08:36:36.557265+0000","last_became_active":"2024-07-17T08:36:33.752921+0000","last_became_peered":"2024-07-17T08:36:33.752921+0000","last_unstale":"2024-07-17T08:36:36.557265+0000","last_undegraded":"2024-07-17T08:36:36.557265+0000","last_fullsized":"2024-07-17T08:36:36.557265+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:46.641811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758519+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:33.758519+0000","last_peered":"2024-07-17T08:36:33.758519+0000","last_clean":"2024-07-17T08:36:33.758519+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:33.758519+0000","last_undegraded":"2024-07-17T08:36:33.758519+0000","last_fullsized":"2024-07-17T08:36:33.758519+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557766+0000","last_change":"2024-07-17T08:36:34.766089+0000","last_active":"2024-07-17T08:36:36.557766+0000","last_peered":"2024-07-17T08:36:36.557766+0000","last_clean":"2024-07-17T08:36:36.557766+0000","last_became_active":"2024-07-17T08:36:34.762344+0000","last_became_peered":"2024-07-17T08:36:34.762344+0000","last_unstale":"2024-07-17T08:36:36.557766+0000","last_undegraded":"2024-07-17T08:36:36.557766+0000","last_fullsized":"2024-07-17T08:36:36.557766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:43.102170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406148+0000","last_change":"2024-07-17T08:36:33.735145+0000","last_active":"2024-07-17T08:36:36.406148+0000","last_peered":"2024-07-17T08:36:36.406148+0000","last_clean":"2024-07-17T08:36:36.406148+0000","last_became_active":"2024-07-17T08:36:33.734773+0000","last_became_peered":"2024-07-17T08:36:33.734773+0000","last_unstale":"2024-07-17T08:36:36.406148+0000","last_undegraded":"2024-07-17T08:36:36.406148+0000","last_fullsized":"2024-07-17T08:36:36.406148+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:06.368749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752774+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:33.752774+0000","last_peered":"2024-07-17T08:36:33.752774+0000","last_clean":"2024-07-17T08:36:33.752774+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:33.752774+0000","last_undegraded":"2024-07-17T08:36:33.752774+0000","last_fullsized":"2024-07-17T08:36:33.752774+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741385+0000","last_change":"2024-07-17T08:36:33.741385+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741385+0000","last_undegraded":"2024-07-17T08:36:33.741385+0000","last_fullsized":"2024-07-17T08:36:33.741385+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"247'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732163+0000","last_change":"2024-07-17T08:36:33.758508+0000","last_active":"2024-07-17T08:36:35.732163+0000","last_peered":"2024-07-17T08:36:35.732163+0000","last_clean":"2024-07-17T08:36:35.732163+0000","last_became_active":"2024-07-17T08:36:33.757683+0000","last_became_peered":"2024-07-17T08:36:33.757683+0000","last_unstale":"2024-07-17T08:36:35.732163+0000","last_undegraded":"2024-07-17T08:36:35.732163+0000","last_fullsized":"2024-07-17T08:36:35.732163+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:13.951810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732029+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:35.732029+0000","last_peered":"2024-07-17T08:36:35.732029+0000","last_clean":"2024-07-17T08:36:35.732029+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:35.732029+0000","last_undegraded":"2024-07-17T08:36:35.732029+0000","last_fullsized":"2024-07-17T08:36:35.732029+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747512+0000","last_change":"2024-07-17T08:36:33.747512+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747512+0000","last_undegraded":"2024-07-17T08:36:33.747512+0000","last_fullsized":"2024-07-17T08:36:33.747512+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559978+0000","last_change":"2024-07-17T08:36:33.757914+0000","last_active":"2024-07-17T08:36:36.559978+0000","last_peered":"2024-07-17T08:36:36.559978+0000","last_clean":"2024-07-17T08:36:36.559978+0000","last_became_active":"2024-07-17T08:36:33.757588+0000","last_became_peered":"2024-07-17T08:36:33.757588+0000","last_unstale":"2024-07-17T08:36:36.559978+0000","last_undegraded":"2024-07-17T08:36:36.559978+0000","last_fullsized":"2024-07-17T08:36:36.559978+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:32.359840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747362+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:33.747362+0000","last_peered":"2024-07-17T08:36:33.747362+0000","last_clean":"2024-07-17T08:36:33.747362+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:33.747362+0000","last_undegraded":"2024-07-17T08:36:33.747362+0000","last_fullsized":"2024-07-17T08:36:33.747362+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747183+0000","last_change":"2024-07-17T08:36:33.747183+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747183+0000","last_undegraded":"2024-07-17T08:36:33.747183+0000","last_fullsized":"2024-07-17T08:36:33.747183+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757640+0000","last_change":"2024-07-17T08:36:33.756054+0000","last_active":"2024-07-17T08:36:33.757640+0000","last_peered":"2024-07-17T08:36:33.757640+0000","last_clean":"2024-07-17T08:36:33.757640+0000","last_became_active":"2024-07-17T08:36:33.755748+0000","last_became_peered":"2024-07-17T08:36:33.755748+0000","last_unstale":"2024-07-17T08:36:33.757640+0000","last_undegraded":"2024-07-17T08:36:33.757640+0000","last_fullsized":"2024-07-17T08:36:33.757640+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:52.097599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737089+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:33.737089+0000","last_peered":"2024-07-17T08:36:33.737089+0000","last_clean":"2024-07-17T08:36:33.737089+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:33.737089+0000","last_undegraded":"2024-07-17T08:36:33.737089+0000","last_fullsized":"2024-07-17T08:36:33.737089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741947+0000","last_change":"2024-07-17T08:36:33.741947+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741947+0000","last_undegraded":"2024-07-17T08:36:33.741947+0000","last_fullsized":"2024-07-17T08:36:33.741947+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757705+0000","last_change":"2024-07-17T08:36:33.756539+0000","last_active":"2024-07-17T08:36:33.757705+0000","last_peered":"2024-07-17T08:36:33.757705+0000","last_clean":"2024-07-17T08:36:33.757705+0000","last_became_active":"2024-07-17T08:36:33.755822+0000","last_became_peered":"2024-07-17T08:36:33.755822+0000","last_unstale":"2024-07-17T08:36:33.757705+0000","last_undegraded":"2024-07-17T08:36:33.757705+0000","last_fullsized":"2024-07-17T08:36:33.757705+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:21:13.608136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559586+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:36.559586+0000","last_peered":"2024-07-17T08:36:36.559586+0000","last_clean":"2024-07-17T08:36:36.559586+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:36.559586+0000","last_undegraded":"2024-07-17T08:36:36.559586+0000","last_fullsized":"2024-07-17T08:36:36.559586+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405849+0000","last_change":"2024-07-17T08:36:34.772888+0000","last_active":"2024-07-17T08:36:36.405849+0000","last_peered":"2024-07-17T08:36:36.405849+0000","last_clean":"2024-07-17T08:36:36.405849+0000","last_became_active":"2024-07-17T08:36:34.771828+0000","last_became_peered":"2024-07-17T08:36:34.771828+0000","last_unstale":"2024-07-17T08:36:36.405849+0000","last_undegraded":"2024-07-17T08:36:36.405849+0000","last_fullsized":"2024-07-17T08:36:36.405849+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:24.603843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730262+0000","last_change":"2024-07-17T08:36:33.755126+0000","last_active":"2024-07-17T08:36:35.730262+0000","last_peered":"2024-07-17T08:36:35.730262+0000","last_clean":"2024-07-17T08:36:35.730262+0000","last_became_active":"2024-07-17T08:36:33.753919+0000","last_became_peered":"2024-07-17T08:36:33.753919+0000","last_unstale":"2024-07-17T08:36:35.730262+0000","last_undegraded":"2024-07-17T08:36:35.730262+0000","last_fullsized":"2024-07-17T08:36:35.730262+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405878+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:36.405878+0000","last_peered":"2024-07-17T08:36:36.405878+0000","last_clean":"2024-07-17T08:36:36.405878+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:36.405878+0000","last_undegraded":"2024-07-17T08:36:36.405878+0000","last_fullsized":"2024-07-17T08:36:36.405878+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557257+0000","last_change":"2024-07-17T08:36:34.757792+0000","last_active":"2024-07-17T08:36:36.557257+0000","last_peered":"2024-07-17T08:36:36.557257+0000","last_clean":"2024-07-17T08:36:36.557257+0000","last_became_active":"2024-07-17T08:36:34.757510+0000","last_became_peered":"2024-07-17T08:36:34.757510+0000","last_unstale":"2024-07-17T08:36:36.557257+0000","last_undegraded":"2024-07-17T08:36:36.557257+0000","last_fullsized":"2024-07-17T08:36:36.557257+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:05.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730760+0000","last_change":"2024-07-17T08:36:33.758782+0000","last_active":"2024-07-17T08:36:35.730760+0000","last_peered":"2024-07-17T08:36:35.730760+0000","last_clean":"2024-07-17T08:36:35.730760+0000","last_became_active":"2024-07-17T08:36:33.757603+0000","last_became_peered":"2024-07-17T08:36:33.757603+0000","last_unstale":"2024-07-17T08:36:35.730760+0000","last_undegraded":"2024-07-17T08:36:35.730760+0000","last_fullsized":"2024-07-17T08:36:35.730760+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:01.305456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.727974+0000","last_change":"2024-07-17T08:36:33.727974+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.727974+0000","last_undegraded":"2024-07-17T08:36:33.727974+0000","last_fullsized":"2024-07-17T08:36:33.727974+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.729622+0000","last_change":"2024-07-17T08:36:33.735347+0000","last_active":"2024-07-17T08:36:35.729622+0000","last_peered":"2024-07-17T08:36:35.729622+0000","last_clean":"2024-07-17T08:36:35.729622+0000","last_became_active":"2024-07-17T08:36:33.734974+0000","last_became_peered":"2024-07-17T08:36:33.734974+0000","last_unstale":"2024-07-17T08:36:35.729622+0000","last_undegraded":"2024-07-17T08:36:35.729622+0000","last_fullsized":"2024-07-17T08:36:35.729622+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:04.954732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730287+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:35.730287+0000","last_peered":"2024-07-17T08:36:35.730287+0000","last_clean":"2024-07-17T08:36:35.730287+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:35.730287+0000","last_undegraded":"2024-07-17T08:36:35.730287+0000","last_fullsized":"2024-07-17T08:36:35.730287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405245+0000","last_change":"2024-07-17T08:36:34.771421+0000","last_active":"2024-07-17T08:36:36.405245+0000","last_peered":"2024-07-17T08:36:36.405245+0000","last_clean":"2024-07-17T08:36:36.405245+0000","last_became_active":"2024-07-17T08:36:34.770940+0000","last_became_peered":"2024-07-17T08:36:34.770940+0000","last_unstale":"2024-07-17T08:36:36.405245+0000","last_undegraded":"2024-07-17T08:36:36.405245+0000","last_fullsized":"2024-07-17T08:36:36.405245+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:32.518091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.751645+0000","last_change":"2024-07-17T08:36:33.749168+0000","last_active":"2024-07-17T08:36:33.751645+0000","last_peered":"2024-07-17T08:36:33.751645+0000","last_clean":"2024-07-17T08:36:33.751645+0000","last_became_active":"2024-07-17T08:36:33.748825+0000","last_became_peered":"2024-07-17T08:36:33.748825+0000","last_unstale":"2024-07-17T08:36:33.751645+0000","last_undegraded":"2024-07-17T08:36:33.751645+0000","last_fullsized":"2024-07-17T08:36:33.751645+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:26:34.216127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558155+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:36.558155+0000","last_peered":"2024-07-17T08:36:36.558155+0000","last_clean":"2024-07-17T08:36:36.558155+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:36.558155+0000","last_undegraded":"2024-07-17T08:36:36.558155+0000","last_fullsized":"2024-07-17T08:36:36.558155+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.740842+0000","last_change":"2024-07-17T08:36:33.740842+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.740842+0000","last_undegraded":"2024-07-17T08:36:33.740842+0000","last_fullsized":"2024-07-17T08:36:33.740842+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730220+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:35.730220+0000","last_peered":"2024-07-17T08:36:35.730220+0000","last_clean":"2024-07-17T08:36:35.730220+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:35.730220+0000","last_undegraded":"2024-07-17T08:36:35.730220+0000","last_fullsized":"2024-07-17T08:36:35.730220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":26,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560226+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:36.560226+0000","last_peered":"2024-07-17T08:36:36.560226+0000","last_clean":"2024-07-17T08:36:36.560226+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:36.560226+0000","last_undegraded":"2024-07-17T08:36:36.560226+0000","last_fullsized":"2024-07-17T08:36:36.560226+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404648+0000","last_change":"2024-07-17T08:36:34.756329+0000","last_active":"2024-07-17T08:36:36.404648+0000","last_peered":"2024-07-17T08:36:36.404648+0000","last_clean":"2024-07-17T08:36:36.404648+0000","last_became_active":"2024-07-17T08:36:34.756056+0000","last_became_peered":"2024-07-17T08:36:34.756056+0000","last_unstale":"2024-07-17T08:36:36.404648+0000","last_undegraded":"2024-07-17T08:36:36.404648+0000","last_fullsized":"2024-07-17T08:36:36.404648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:00.211357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757749+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:33.757749+0000","last_peered":"2024-07-17T08:36:33.757749+0000","last_clean":"2024-07-17T08:36:33.757749+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:33.757749+0000","last_undegraded":"2024-07-17T08:36:33.757749+0000","last_fullsized":"2024-07-17T08:36:33.757749+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757682+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:33.757682+0000","last_peered":"2024-07-17T08:36:33.757682+0000","last_clean":"2024-07-17T08:36:33.757682+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:33.757682+0000","last_undegraded":"2024-07-17T08:36:33.757682+0000","last_fullsized":"2024-07-17T08:36:33.757682+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406536+0000","last_change":"2024-07-17T08:36:34.776531+0000","last_active":"2024-07-17T08:36:36.406536+0000","last_peered":"2024-07-17T08:36:36.406536+0000","last_clean":"2024-07-17T08:36:36.406536+0000","last_became_active":"2024-07-17T08:36:34.776010+0000","last_became_peered":"2024-07-17T08:36:34.776010+0000","last_unstale":"2024-07-17T08:36:36.406536+0000","last_undegraded":"2024-07-17T08:36:36.406536+0000","last_fullsized":"2024-07-17T08:36:36.406536+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:27.058250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559158+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:36.559158+0000","last_peered":"2024-07-17T08:36:36.559158+0000","last_clean":"2024-07-17T08:36:36.559158+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:36.559158+0000","last_undegraded":"2024-07-17T08:36:36.559158+0000","last_fullsized":"2024-07-17T08:36:36.559158+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406585+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:36.406585+0000","last_peered":"2024-07-17T08:36:36.406585+0000","last_clean":"2024-07-17T08:36:36.406585+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:36.406585+0000","last_undegraded":"2024-07-17T08:36:36.406585+0000","last_fullsized":"2024-07-17T08:36:36.406585+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407015+0000","last_change":"2024-07-17T08:36:34.768405+0000","last_active":"2024-07-17T08:36:36.407015+0000","last_peered":"2024-07-17T08:36:36.407015+0000","last_clean":"2024-07-17T08:36:36.407015+0000","last_became_active":"2024-07-17T08:36:34.768128+0000","last_became_peered":"2024-07-17T08:36:34.768128+0000","last_unstale":"2024-07-17T08:36:36.407015+0000","last_undegraded":"2024-07-17T08:36:36.407015+0000","last_fullsized":"2024-07-17T08:36:36.407015+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:26.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405806+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:36.405806+0000","last_peered":"2024-07-17T08:36:36.405806+0000","last_clean":"2024-07-17T08:36:36.405806+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:36.405806+0000","last_undegraded":"2024-07-17T08:36:36.405806+0000","last_fullsized":"2024-07-17T08:36:36.405806+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757771+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:33.757771+0000","last_peered":"2024-07-17T08:36:33.757771+0000","last_clean":"2024-07-17T08:36:33.757771+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:33.757771+0000","last_undegraded":"2024-07-17T08:36:33.757771+0000","last_fullsized":"2024-07-17T08:36:33.757771+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.749442+0000","last_change":"2024-07-17T08:36:33.749442+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.749442+0000","last_undegraded":"2024-07-17T08:36:33.749442+0000","last_fullsized":"2024-07-17T08:36:33.749442+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749616+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:33.749616+0000","last_peered":"2024-07-17T08:36:33.749616+0000","last_clean":"2024-07-17T08:36:33.749616+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:33.749616+0000","last_undegraded":"2024-07-17T08:36:33.749616+0000","last_fullsized":"2024-07-17T08:36:33.749616+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407040+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:36.407040+0000","last_peered":"2024-07-17T08:36:36.407040+0000","last_clean":"2024-07-17T08:36:36.407040+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:36.407040+0000","last_undegraded":"2024-07-17T08:36:36.407040+0000","last_fullsized":"2024-07-17T08:36:36.407040+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739783+0000","last_change":"2024-07-17T08:36:33.739783+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739783+0000","last_undegraded":"2024-07-17T08:36:33.739783+0000","last_fullsized":"2024-07-17T08:36:33.739783+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739977+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:33.739977+0000","last_peered":"2024-07-17T08:36:33.739977+0000","last_clean":"2024-07-17T08:36:33.739977+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:33.739977+0000","last_undegraded":"2024-07-17T08:36:33.739977+0000","last_fullsized":"2024-07-17T08:36:33.739977+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736947+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:33.736947+0000","last_peered":"2024-07-17T08:36:33.736947+0000","last_clean":"2024-07-17T08:36:33.736947+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:33.736947+0000","last_undegraded":"2024-07-17T08:36:33.736947+0000","last_fullsized":"2024-07-17T08:36:33.736947+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.746117+0000","last_change":"2024-07-17T08:36:33.746117+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.746117+0000","last_undegraded":"2024-07-17T08:36:33.746117+0000","last_fullsized":"2024-07-17T08:36:33.746117+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738917+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:33.738917+0000","last_peered":"2024-07-17T08:36:33.738917+0000","last_clean":"2024-07-17T08:36:33.738917+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:33.738917+0000","last_undegraded":"2024-07-17T08:36:33.738917+0000","last_fullsized":"2024-07-17T08:36:33.738917+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558367+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:36.558367+0000","last_peered":"2024-07-17T08:36:36.558367+0000","last_clean":"2024-07-17T08:36:36.558367+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:36.558367+0000","last_undegraded":"2024-07-17T08:36:36.558367+0000","last_fullsized":"2024-07-17T08:36:36.558367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.744976+0000","last_change":"2024-07-17T08:36:33.744976+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.744976+0000","last_undegraded":"2024-07-17T08:36:33.744976+0000","last_fullsized":"2024-07-17T08:36:33.744976+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739832+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:33.739832+0000","last_peered":"2024-07-17T08:36:33.739832+0000","last_clean":"2024-07-17T08:36:33.739832+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:33.739832+0000","last_undegraded":"2024-07-17T08:36:33.739832+0000","last_fullsized":"2024-07-17T08:36:33.739832+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407193+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:36.407193+0000","last_peered":"2024-07-17T08:36:36.407193+0000","last_clean":"2024-07-17T08:36:36.407193+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:36.407193+0000","last_undegraded":"2024-07-17T08:36:36.407193+0000","last_fullsized":"2024-07-17T08:36:36.407193+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741478+0000","last_change":"2024-07-17T08:36:33.741478+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741478+0000","last_undegraded":"2024-07-17T08:36:33.741478+0000","last_fullsized":"2024-07-17T08:36:33.741478+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754424+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:33.754424+0000","last_peered":"2024-07-17T08:36:33.754424+0000","last_clean":"2024-07-17T08:36:33.754424+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:33.754424+0000","last_undegraded":"2024-07-17T08:36:33.754424+0000","last_fullsized":"2024-07-17T08:36:33.754424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.740288+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:33.740288+0000","last_peered":"2024-07-17T08:36:33.740288+0000","last_clean":"2024-07-17T08:36:33.740288+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:33.740288+0000","last_undegraded":"2024-07-17T08:36:33.740288+0000","last_fullsized":"2024-07-17T08:36:33.740288+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739689+0000","last_change":"2024-07-17T08:36:33.739689+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739689+0000","last_undegraded":"2024-07-17T08:36:33.739689+0000","last_fullsized":"2024-07-17T08:36:33.739689+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744328+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:33.744328+0000","last_peered":"2024-07-17T08:36:33.744328+0000","last_clean":"2024-07-17T08:36:33.744328+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:33.744328+0000","last_undegraded":"2024-07-17T08:36:33.744328+0000","last_fullsized":"2024-07-17T08:36:33.744328+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":21,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737223+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:33.737223+0000","last_peered":"2024-07-17T08:36:33.737223+0000","last_clean":"2024-07-17T08:36:33.737223+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:33.737223+0000","last_undegraded":"2024-07-17T08:36:33.737223+0000","last_fullsized":"2024-07-17T08:36:33.737223+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404878+0000","last_change":"2024-07-17T08:36:34.761187+0000","last_active":"2024-07-17T08:36:36.404878+0000","last_peered":"2024-07-17T08:36:36.404878+0000","last_clean":"2024-07-17T08:36:36.404878+0000","last_became_active":"2024-07-17T08:36:34.760833+0000","last_became_peered":"2024-07-17T08:36:34.760833+0000","last_unstale":"2024-07-17T08:36:36.404878+0000","last_undegraded":"2024-07-17T08:36:36.404878+0000","last_fullsized":"2024-07-17T08:36:36.404878+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:49.833097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741099+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:33.741099+0000","last_peered":"2024-07-17T08:36:33.741099+0000","last_clean":"2024-07-17T08:36:33.741099+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:33.741099+0000","last_undegraded":"2024-07-17T08:36:33.741099+0000","last_fullsized":"2024-07-17T08:36:33.741099+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559818+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:36.559818+0000","last_peered":"2024-07-17T08:36:36.559818+0000","last_clean":"2024-07-17T08:36:36.559818+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:36.559818+0000","last_undegraded":"2024-07-17T08:36:36.559818+0000","last_fullsized":"2024-07-17T08:36:36.559818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732279+0000","last_change":"2024-07-17T08:36:34.759236+0000","last_active":"2024-07-17T08:36:35.732279+0000","last_peered":"2024-07-17T08:36:35.732279+0000","last_clean":"2024-07-17T08:36:35.732279+0000","last_became_active":"2024-07-17T08:36:34.751420+0000","last_became_peered":"2024-07-17T08:36:34.751420+0000","last_unstale":"2024-07-17T08:36:35.732279+0000","last_undegraded":"2024-07-17T08:36:35.732279+0000","last_fullsized":"2024-07-17T08:36:35.732279+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731166+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:35.731166+0000","last_peered":"2024-07-17T08:36:35.731166+0000","last_clean":"2024-07-17T08:36:35.731166+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:35.731166+0000","last_undegraded":"2024-07-17T08:36:35.731166+0000","last_fullsized":"2024-07-17T08:36:35.731166+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757876+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:33.757876+0000","last_peered":"2024-07-17T08:36:33.757876+0000","last_clean":"2024-07-17T08:36:33.757876+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:33.757876+0000","last_undegraded":"2024-07-17T08:36:33.757876+0000","last_fullsized":"2024-07-17T08:36:33.757876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.729240+0000","last_change":"2024-07-17T08:36:33.729240+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.729240+0000","last_undegraded":"2024-07-17T08:36:33.729240+0000","last_fullsized":"2024-07-17T08:36:33.729240+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.749373+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:33.749373+0000","last_peered":"2024-07-17T08:36:33.749373+0000","last_clean":"2024-07-17T08:36:33.749373+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:33.749373+0000","last_undegraded":"2024-07-17T08:36:33.749373+0000","last_fullsized":"2024-07-17T08:36:33.749373+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.735885+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:33.735885+0000","last_peered":"2024-07-17T08:36:33.735885+0000","last_clean":"2024-07-17T08:36:33.735885+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:33.735885+0000","last_undegraded":"2024-07-17T08:36:33.735885+0000","last_fullsized":"2024-07-17T08:36:33.735885+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.747960+0000","last_change":"2024-07-17T08:36:33.747960+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.747960+0000","last_undegraded":"2024-07-17T08:36:33.747960+0000","last_fullsized":"2024-07-17T08:36:33.747960+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.760565+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:33.760565+0000","last_peered":"2024-07-17T08:36:33.760565+0000","last_clean":"2024-07-17T08:36:33.760565+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:33.760565+0000","last_undegraded":"2024-07-17T08:36:33.760565+0000","last_fullsized":"2024-07-17T08:36:33.760565+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732603+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:33.732603+0000","last_peered":"2024-07-17T08:36:33.732603+0000","last_clean":"2024-07-17T08:36:33.732603+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:33.732603+0000","last_undegraded":"2024-07-17T08:36:33.732603+0000","last_fullsized":"2024-07-17T08:36:33.732603+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.728163+0000","last_change":"2024-07-17T08:36:33.728163+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.728163+0000","last_undegraded":"2024-07-17T08:36:33.728163+0000","last_fullsized":"2024-07-17T08:36:33.728163+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738796+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:33.738796+0000","last_peered":"2024-07-17T08:36:33.738796+0000","last_clean":"2024-07-17T08:36:33.738796+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:33.738796+0000","last_undegraded":"2024-07-17T08:36:33.738796+0000","last_fullsized":"2024-07-17T08:36:33.738796+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730519+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:35.730519+0000","last_peered":"2024-07-17T08:36:35.730519+0000","last_clean":"2024-07-17T08:36:35.730519+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:35.730519+0000","last_undegraded":"2024-07-17T08:36:35.730519+0000","last_fullsized":"2024-07-17T08:36:35.730519+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405657+0000","last_change":"2024-07-17T08:36:34.762727+0000","last_active":"2024-07-17T08:36:36.405657+0000","last_peered":"2024-07-17T08:36:36.405657+0000","last_clean":"2024-07-17T08:36:36.405657+0000","last_became_active":"2024-07-17T08:36:34.762396+0000","last_became_peered":"2024-07-17T08:36:34.762396+0000","last_unstale":"2024-07-17T08:36:36.405657+0000","last_undegraded":"2024-07-17T08:36:36.405657+0000","last_fullsized":"2024-07-17T08:36:36.405657+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:10.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.732714+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:33.732714+0000","last_peered":"2024-07-17T08:36:33.732714+0000","last_clean":"2024-07-17T08:36:33.732714+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:33.732714+0000","last_undegraded":"2024-07-17T08:36:33.732714+0000","last_fullsized":"2024-07-17T08:36:33.732714+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560433+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:36.560433+0000","last_peered":"2024-07-17T08:36:36.560433+0000","last_clean":"2024-07-17T08:36:36.560433+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:36.560433+0000","last_undegraded":"2024-07-17T08:36:36.560433+0000","last_fullsized":"2024-07-17T08:36:36.560433+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558118+0000","last_change":"2024-07-17T08:36:34.757902+0000","last_active":"2024-07-17T08:36:36.558118+0000","last_peered":"2024-07-17T08:36:36.558118+0000","last_clean":"2024-07-17T08:36:36.558118+0000","last_became_active":"2024-07-17T08:36:34.756374+0000","last_became_peered":"2024-07-17T08:36:34.756374+0000","last_unstale":"2024-07-17T08:36:36.558118+0000","last_undegraded":"2024-07-17T08:36:36.558118+0000","last_fullsized":"2024-07-17T08:36:36.558118+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:04:56.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.731581+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:33.731581+0000","last_peered":"2024-07-17T08:36:33.731581+0000","last_clean":"2024-07-17T08:36:33.731581+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:33.731581+0000","last_undegraded":"2024-07-17T08:36:33.731581+0000","last_fullsized":"2024-07-17T08:36:33.731581+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.733076+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:35.733076+0000","last_peered":"2024-07-17T08:36:35.733076+0000","last_clean":"2024-07-17T08:36:35.733076+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:35.733076+0000","last_undegraded":"2024-07-17T08:36:35.733076+0000","last_fullsized":"2024-07-17T08:36:35.733076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.730370+0000","last_change":"2024-07-17T08:36:33.730370+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.730370+0000","last_undegraded":"2024-07-17T08:36:33.730370+0000","last_fullsized":"2024-07-17T08:36:33.730370+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730514+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:35.730514+0000","last_peered":"2024-07-17T08:36:35.730514+0000","last_clean":"2024-07-17T08:36:35.730514+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:35.730514+0000","last_undegraded":"2024-07-17T08:36:35.730514+0000","last_fullsized":"2024-07-17T08:36:35.730514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757835+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:33.757835+0000","last_peered":"2024-07-17T08:36:33.757835+0000","last_clean":"2024-07-17T08:36:33.757835+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:33.757835+0000","last_undegraded":"2024-07-17T08:36:33.757835+0000","last_fullsized":"2024-07-17T08:36:33.757835+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404661+0000","last_change":"2024-07-17T08:36:34.767357+0000","last_active":"2024-07-17T08:36:36.404661+0000","last_peered":"2024-07-17T08:36:36.404661+0000","last_clean":"2024-07-17T08:36:36.404661+0000","last_became_active":"2024-07-17T08:36:34.766645+0000","last_became_peered":"2024-07-17T08:36:34.766645+0000","last_unstale":"2024-07-17T08:36:36.404661+0000","last_undegraded":"2024-07-17T08:36:36.404661+0000","last_fullsized":"2024-07-17T08:36:36.404661+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:40.714568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744175+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:33.744175+0000","last_peered":"2024-07-17T08:36:33.744175+0000","last_clean":"2024-07-17T08:36:33.744175+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:33.744175+0000","last_undegraded":"2024-07-17T08:36:33.744175+0000","last_fullsized":"2024-07-17T08:36:33.744175+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.757866+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:33.757866+0000","last_peered":"2024-07-17T08:36:33.757866+0000","last_clean":"2024-07-17T08:36:33.757866+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:33.757866+0000","last_undegraded":"2024-07-17T08:36:33.757866+0000","last_fullsized":"2024-07-17T08:36:33.757866+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.732033+0000","last_change":"2024-07-17T08:36:33.732033+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.732033+0000","last_undegraded":"2024-07-17T08:36:33.732033+0000","last_fullsized":"2024-07-17T08:36:33.732033+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739171+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:33.739171+0000","last_peered":"2024-07-17T08:36:33.739171+0000","last_clean":"2024-07-17T08:36:33.739171+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:33.739171+0000","last_undegraded":"2024-07-17T08:36:33.739171+0000","last_fullsized":"2024-07-17T08:36:33.739171+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.744670+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:33.744670+0000","last_peered":"2024-07-17T08:36:33.744670+0000","last_clean":"2024-07-17T08:36:33.744670+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:33.744670+0000","last_undegraded":"2024-07-17T08:36:33.744670+0000","last_fullsized":"2024-07-17T08:36:33.744670+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.750569+0000","last_change":"2024-07-17T08:36:33.750569+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.750569+0000","last_undegraded":"2024-07-17T08:36:33.750569+0000","last_fullsized":"2024-07-17T08:36:33.750569+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731483+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:35.731483+0000","last_peered":"2024-07-17T08:36:35.731483+0000","last_clean":"2024-07-17T08:36:35.731483+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:35.731483+0000","last_undegraded":"2024-07-17T08:36:35.731483+0000","last_fullsized":"2024-07-17T08:36:35.731483+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754718+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:33.754718+0000","last_peered":"2024-07-17T08:36:33.754718+0000","last_clean":"2024-07-17T08:36:33.754718+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:33.754718+0000","last_undegraded":"2024-07-17T08:36:33.754718+0000","last_fullsized":"2024-07-17T08:36:33.754718+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.752613+0000","last_change":"2024-07-17T08:36:33.752613+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.752613+0000","last_undegraded":"2024-07-17T08:36:33.752613+0000","last_fullsized":"2024-07-17T08:36:33.752613+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758148+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:33.758148+0000","last_peered":"2024-07-17T08:36:33.758148+0000","last_clean":"2024-07-17T08:36:33.758148+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:33.758148+0000","last_undegraded":"2024-07-17T08:36:33.758148+0000","last_fullsized":"2024-07-17T08:36:33.758148+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754545+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:33.754545+0000","last_peered":"2024-07-17T08:36:33.754545+0000","last_clean":"2024-07-17T08:36:33.754545+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:33.754545+0000","last_undegraded":"2024-07-17T08:36:33.754545+0000","last_fullsized":"2024-07-17T08:36:33.754545+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748774+0000","last_change":"2024-07-17T08:36:33.748774+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748774+0000","last_undegraded":"2024-07-17T08:36:33.748774+0000","last_fullsized":"2024-07-17T08:36:33.748774+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758417+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:33.758417+0000","last_peered":"2024-07-17T08:36:33.758417+0000","last_clean":"2024-07-17T08:36:33.758417+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:33.758417+0000","last_undegraded":"2024-07-17T08:36:33.758417+0000","last_fullsized":"2024-07-17T08:36:33.758417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405097+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:36.405097+0000","last_peered":"2024-07-17T08:36:36.405097+0000","last_clean":"2024-07-17T08:36:36.405097+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:36.405097+0000","last_undegraded":"2024-07-17T08:36:36.405097+0000","last_fullsized":"2024-07-17T08:36:36.405097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.741816+0000","last_change":"2024-07-17T08:36:33.741816+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.741816+0000","last_undegraded":"2024-07-17T08:36:33.741816+0000","last_fullsized":"2024-07-17T08:36:33.741816+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.739627+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:33.739627+0000","last_peered":"2024-07-17T08:36:33.739627+0000","last_clean":"2024-07-17T08:36:33.739627+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:33.739627+0000","last_undegraded":"2024-07-17T08:36:33.739627+0000","last_fullsized":"2024-07-17T08:36:33.739627+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":20,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.738733+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:33.738733+0000","last_peered":"2024-07-17T08:36:33.738733+0000","last_clean":"2024-07-17T08:36:33.738733+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:33.738733+0000","last_undegraded":"2024-07-17T08:36:33.738733+0000","last_fullsized":"2024-07-17T08:36:33.738733+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.736575+0000","last_change":"2024-07-17T08:36:33.736575+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.736575+0000","last_undegraded":"2024-07-17T08:36:33.736575+0000","last_fullsized":"2024-07-17T08:36:33.736575+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758322+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:33.758322+0000","last_peered":"2024-07-17T08:36:33.758322+0000","last_clean":"2024-07-17T08:36:33.758322+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:33.758322+0000","last_undegraded":"2024-07-17T08:36:33.758322+0000","last_fullsized":"2024-07-17T08:36:33.758322+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737876+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:33.737876+0000","last_peered":"2024-07-17T08:36:33.737876+0000","last_clean":"2024-07-17T08:36:33.737876+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:33.737876+0000","last_undegraded":"2024-07-17T08:36:33.737876+0000","last_fullsized":"2024-07-17T08:36:33.737876+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557817+0000","last_change":"2024-07-17T08:36:34.739794+0000","last_active":"2024-07-17T08:36:36.557817+0000","last_peered":"2024-07-17T08:36:36.557817+0000","last_clean":"2024-07-17T08:36:36.557817+0000","last_became_active":"2024-07-17T08:36:34.739282+0000","last_became_peered":"2024-07-17T08:36:34.739282+0000","last_unstale":"2024-07-17T08:36:36.557817+0000","last_undegraded":"2024-07-17T08:36:36.557817+0000","last_fullsized":"2024-07-17T08:36:36.557817+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:18.364494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.741540+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:33.741540+0000","last_peered":"2024-07-17T08:36:33.741540+0000","last_clean":"2024-07-17T08:36:33.741540+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:33.741540+0000","last_undegraded":"2024-07-17T08:36:33.741540+0000","last_fullsized":"2024-07-17T08:36:33.741540+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406502+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:36.406502+0000","last_peered":"2024-07-17T08:36:36.406502+0000","last_clean":"2024-07-17T08:36:36.406502+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:36.406502+0000","last_undegraded":"2024-07-17T08:36:36.406502+0000","last_fullsized":"2024-07-17T08:36:36.406502+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.739158+0000","last_change":"2024-07-17T08:36:33.739158+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.739158+0000","last_undegraded":"2024-07-17T08:36:33.739158+0000","last_fullsized":"2024-07-17T08:36:33.739158+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.736298+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:33.736298+0000","last_peered":"2024-07-17T08:36:33.736298+0000","last_clean":"2024-07-17T08:36:33.736298+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:33.736298+0000","last_undegraded":"2024-07-17T08:36:33.736298+0000","last_fullsized":"2024-07-17T08:36:33.736298+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406872+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:36.406872+0000","last_peered":"2024-07-17T08:36:36.406872+0000","last_clean":"2024-07-17T08:36:36.406872+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:36.406872+0000","last_undegraded":"2024-07-17T08:36:36.406872+0000","last_fullsized":"2024-07-17T08:36:36.406872+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406507+0000","last_change":"2024-07-17T08:36:34.772879+0000","last_active":"2024-07-17T08:36:36.406507+0000","last_peered":"2024-07-17T08:36:36.406507+0000","last_clean":"2024-07-17T08:36:36.406507+0000","last_became_active":"2024-07-17T08:36:34.772086+0000","last_became_peered":"2024-07-17T08:36:34.772086+0000","last_unstale":"2024-07-17T08:36:36.406507+0000","last_undegraded":"2024-07-17T08:36:36.406507+0000","last_fullsized":"2024-07-17T08:36:36.406507+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:17.960713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405403+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:36.405403+0000","last_peered":"2024-07-17T08:36:36.405403+0000","last_clean":"2024-07-17T08:36:36.405403+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:36.405403+0000","last_undegraded":"2024-07-17T08:36:36.405403+0000","last_fullsized":"2024-07-17T08:36:36.405403+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754091+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:33.754091+0000","last_peered":"2024-07-17T08:36:33.754091+0000","last_clean":"2024-07-17T08:36:33.754091+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:33.754091+0000","last_undegraded":"2024-07-17T08:36:33.754091+0000","last_fullsized":"2024-07-17T08:36:33.754091+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.748648+0000","last_change":"2024-07-17T08:36:33.748648+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.748648+0000","last_undegraded":"2024-07-17T08:36:33.748648+0000","last_fullsized":"2024-07-17T08:36:33.748648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559753+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:36.559753+0000","last_peered":"2024-07-17T08:36:36.559753+0000","last_clean":"2024-07-17T08:36:36.559753+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:36.559753+0000","last_undegraded":"2024-07-17T08:36:36.559753+0000","last_fullsized":"2024-07-17T08:36:36.559753+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.734495+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:33.734495+0000","last_peered":"2024-07-17T08:36:33.734495+0000","last_clean":"2024-07-17T08:36:33.734495+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:33.734495+0000","last_undegraded":"2024-07-17T08:36:33.734495+0000","last_fullsized":"2024-07-17T08:36:33.734495+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558755+0000","last_change":"2024-07-17T08:36:34.753399+0000","last_active":"2024-07-17T08:36:36.558755+0000","last_peered":"2024-07-17T08:36:36.558755+0000","last_clean":"2024-07-17T08:36:36.558755+0000","last_became_active":"2024-07-17T08:36:34.753124+0000","last_became_peered":"2024-07-17T08:36:34.753124+0000","last_unstale":"2024-07-17T08:36:36.558755+0000","last_undegraded":"2024-07-17T08:36:36.558755+0000","last_fullsized":"2024-07-17T08:36:36.558755+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:19.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732588+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:35.732588+0000","last_peered":"2024-07-17T08:36:35.732588+0000","last_clean":"2024-07-17T08:36:35.732588+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:35.732588+0000","last_undegraded":"2024-07-17T08:36:35.732588+0000","last_fullsized":"2024-07-17T08:36:35.732588+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.737087+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:33.737087+0000","last_peered":"2024-07-17T08:36:33.737087+0000","last_clean":"2024-07-17T08:36:33.737087+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:33.737087+0000","last_undegraded":"2024-07-17T08:36:33.737087+0000","last_fullsized":"2024-07-17T08:36:33.737087+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":1,"reported_epoch":245,"state":"creating+peering","last_fresh":"2024-07-17T08:36:33.743109+0000","last_change":"2024-07-17T08:36:33.743109+0000","last_active":"2024-07-17T08:36:33.701795+0000","last_peered":"2024-07-17T08:36:33.701795+0000","last_clean":"2024-07-17T08:36:33.701795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-17T08:36:33.743109+0000","last_undegraded":"2024-07-17T08:36:33.743109+0000","last_fullsized":"2024-07-17T08:36:33.743109+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":16,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.758410+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:33.758410+0000","last_peered":"2024-07-17T08:36:33.758410+0000","last_clean":"2024-07-17T08:36:33.758410+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:33.758410+0000","last_undegraded":"2024-07-17T08:36:33.758410+0000","last_fullsized":"2024-07-17T08:36:33.758410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":18,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.752926+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:33.752926+0000","last_peered":"2024-07-17T08:36:33.752926+0000","last_clean":"2024-07-17T08:36:33.752926+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:33.752926+0000","last_undegraded":"2024-07-17T08:36:33.752926+0000","last_fullsized":"2024-07-17T08:36:33.752926+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558684+0000","last_change":"2024-07-17T08:36:34.758236+0000","last_active":"2024-07-17T08:36:36.558684+0000","last_peered":"2024-07-17T08:36:36.558684+0000","last_clean":"2024-07-17T08:36:36.558684+0000","last_became_active":"2024-07-17T08:36:34.756897+0000","last_became_peered":"2024-07-17T08:36:34.756897+0000","last_unstale":"2024-07-17T08:36:36.558684+0000","last_undegraded":"2024-07-17T08:36:36.558684+0000","last_fullsized":"2024-07-17T08:36:36.558684+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.720969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558229+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:36.558229+0000","last_peered":"2024-07-17T08:36:36.558229+0000","last_clean":"2024-07-17T08:36:36.558229+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:36.558229+0000","last_undegraded":"2024-07-17T08:36:36.558229+0000","last_fullsized":"2024-07-17T08:36:36.558229+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":19,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.747574+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:33.747574+0000","last_peered":"2024-07-17T08:36:33.747574+0000","last_clean":"2024-07-17T08:36:33.747574+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:33.747574+0000","last_undegraded":"2024-07-17T08:36:33.747574+0000","last_fullsized":"2024-07-17T08:36:33.747574+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3471,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-07-17T08:36:33.754355+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:33.754355+0000","last_peered":"2024-07-17T08:36:33.754355+0000","last_clean":"2024-07-17T08:36:33.754355+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:33.754355+0000","last_undegraded":"2024-07-17T08:36:33.754355+0000","last_fullsized":"2024-07-17T08:36:33.754355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2284,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543355,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48676,"kb_used_data":3604,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323164,"statfs":{"total":96636764160,"available":96586919936,"internally_reserved":0,"allocated":3690496,"data_stored":2279383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510650,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48080,"kb_used_data":3012,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323760,"statfs":{"total":96636764160,"available":96587530240,"internally_reserved":0,"allocated":3084288,"data_stored":1680899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52248,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319592,"statfs":{"total":96636764160,"available":96583262208,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543356,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52236,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319604,"statfs":{"total":96636764160,"available":96583274496,"internally_reserved":0,"allocated":3080192,"data_stored":1680857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510651,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52296,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319544,"statfs":{"total":96636764160,"available":96583213056,"internally_reserved":0,"allocated":3141632,"data_stored":1709225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543355,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52300,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319540,"statfs":{"total":96636764160,"available":96583208960,"internally_reserved":0,"allocated":3141632,"data_stored":1708507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510651,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48084,"kb_used_data":3016,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323756,"statfs":{"total":96636764160,"available":96587526144,"internally_reserved":0,"allocated":3088384,"data_stored":1688891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510651,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57024,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314816,"statfs":{"total":96636764160,"available":96578371584,"internally_reserved":0,"allocated":3723264,"data_stored":2298935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:PG 34.1c is not active+clean 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.727993+0000', 'last_change': '2024-07-17T08:36:33.727993+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.727993+0000', 'last_undegraded': '2024-07-17T08:36:33.727993+0000', 'last_fullsized': '2024-07-17T08:36:33.727993+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:PG 34.1a is not active+clean 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.742246+0000', 'last_change': '2024-07-17T08:36:33.742246+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.742246+0000', 'last_undegraded': '2024-07-17T08:36:33.742246+0000', 'last_fullsized': '2024-07-17T08:36:33.742246+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:PG 34.17 is not active+clean 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.748161+0000', 'last_change': '2024-07-17T08:36:33.748161+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.748161+0000', 'last_undegraded': '2024-07-17T08:36:33.748161+0000', 'last_fullsized': '2024-07-17T08:36:33.748161+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:PG 34.16 is not active+clean 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.747898+0000', 'last_change': '2024-07-17T08:36:33.747898+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.747898+0000', 'last_undegraded': '2024-07-17T08:36:33.747898+0000', 'last_fullsized': '2024-07-17T08:36:33.747898+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:PG 34.11 is not active+clean 2024-07-17T08:36:40.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.742681+0000', 'last_change': '2024-07-17T08:36:33.742681+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.742681+0000', 'last_undegraded': '2024-07-17T08:36:33.742681+0000', 'last_fullsized': '2024-07-17T08:36:33.742681+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.10 is not active+clean 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.735212+0000', 'last_change': '2024-07-17T08:36:33.735212+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.735212+0000', 'last_undegraded': '2024-07-17T08:36:33.735212+0000', 'last_fullsized': '2024-07-17T08:36:33.735212+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.13 is not active+clean 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.738574+0000', 'last_change': '2024-07-17T08:36:33.738574+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.738574+0000', 'last_undegraded': '2024-07-17T08:36:33.738574+0000', 'last_fullsized': '2024-07-17T08:36:33.738574+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.12 is not active+clean 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.735656+0000', 'last_change': '2024-07-17T08:36:33.735656+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.735656+0000', 'last_undegraded': '2024-07-17T08:36:33.735656+0000', 'last_fullsized': '2024-07-17T08:36:33.735656+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.d is not active+clean 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.745690+0000', 'last_change': '2024-07-17T08:36:33.745690+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.745690+0000', 'last_undegraded': '2024-07-17T08:36:33.745690+0000', 'last_fullsized': '2024-07-17T08:36:33.745690+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.c is not active+clean 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.744930+0000', 'last_change': '2024-07-17T08:36:33.744930+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.744930+0000', 'last_undegraded': '2024-07-17T08:36:33.744930+0000', 'last_fullsized': '2024-07-17T08:36:33.744930+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.361 INFO:tasks.ceph.ceph_manager.ceph:PG 34.f is not active+clean 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.745954+0000', 'last_change': '2024-07-17T08:36:33.745954+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.745954+0000', 'last_undegraded': '2024-07-17T08:36:33.745954+0000', 'last_fullsized': '2024-07-17T08:36:33.745954+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:PG 34.9 is not active+clean 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.741992+0000', 'last_change': '2024-07-17T08:36:33.741992+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.741992+0000', 'last_undegraded': '2024-07-17T08:36:33.741992+0000', 'last_fullsized': '2024-07-17T08:36:33.741992+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:PG 34.5 is not active+clean 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.741385+0000', 'last_change': '2024-07-17T08:36:33.741385+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.741385+0000', 'last_undegraded': '2024-07-17T08:36:33.741385+0000', 'last_fullsized': '2024-07-17T08:36:33.741385+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:PG 34.4 is not active+clean 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.747512+0000', 'last_change': '2024-07-17T08:36:33.747512+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.747512+0000', 'last_undegraded': '2024-07-17T08:36:33.747512+0000', 'last_fullsized': '2024-07-17T08:36:33.747512+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:PG 34.7 is not active+clean 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.747183+0000', 'last_change': '2024-07-17T08:36:33.747183+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.747183+0000', 'last_undegraded': '2024-07-17T08:36:33.747183+0000', 'last_fullsized': '2024-07-17T08:36:33.747183+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.362 INFO:tasks.ceph.ceph_manager.ceph:PG 34.6 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.741947+0000', 'last_change': '2024-07-17T08:36:33.741947+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.741947+0000', 'last_undegraded': '2024-07-17T08:36:33.741947+0000', 'last_fullsized': '2024-07-17T08:36:33.741947+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:PG 34.3 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.727974+0000', 'last_change': '2024-07-17T08:36:33.727974+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.727974+0000', 'last_undegraded': '2024-07-17T08:36:33.727974+0000', 'last_fullsized': '2024-07-17T08:36:33.727974+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:PG 34.22 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.740842+0000', 'last_change': '2024-07-17T08:36:33.740842+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.740842+0000', 'last_undegraded': '2024-07-17T08:36:33.740842+0000', 'last_fullsized': '2024-07-17T08:36:33.740842+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:PG 34.27 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.749442+0000', 'last_change': '2024-07-17T08:36:33.749442+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.749442+0000', 'last_undegraded': '2024-07-17T08:36:33.749442+0000', 'last_fullsized': '2024-07-17T08:36:33.749442+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:PG 34.24 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.739783+0000', 'last_change': '2024-07-17T08:36:33.739783+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.739783+0000', 'last_undegraded': '2024-07-17T08:36:33.739783+0000', 'last_fullsized': '2024-07-17T08:36:33.739783+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:PG 34.25 is not active+clean 2024-07-17T08:36:40.363 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.746117+0000', 'last_change': '2024-07-17T08:36:33.746117+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.746117+0000', 'last_undegraded': '2024-07-17T08:36:33.746117+0000', 'last_fullsized': '2024-07-17T08:36:33.746117+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:PG 34.2a is not active+clean 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.744976+0000', 'last_change': '2024-07-17T08:36:33.744976+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.744976+0000', 'last_undegraded': '2024-07-17T08:36:33.744976+0000', 'last_fullsized': '2024-07-17T08:36:33.744976+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:PG 34.2b is not active+clean 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.741478+0000', 'last_change': '2024-07-17T08:36:33.741478+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.741478+0000', 'last_undegraded': '2024-07-17T08:36:33.741478+0000', 'last_fullsized': '2024-07-17T08:36:33.741478+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:PG 34.28 is not active+clean 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.739689+0000', 'last_change': '2024-07-17T08:36:33.739689+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.739689+0000', 'last_undegraded': '2024-07-17T08:36:33.739689+0000', 'last_fullsized': '2024-07-17T08:36:33.739689+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:PG 34.2f is not active+clean 2024-07-17T08:36:40.364 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.729240+0000', 'last_change': '2024-07-17T08:36:33.729240+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.729240+0000', 'last_undegraded': '2024-07-17T08:36:33.729240+0000', 'last_fullsized': '2024-07-17T08:36:33.729240+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.2c is not active+clean 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.747960+0000', 'last_change': '2024-07-17T08:36:33.747960+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.747960+0000', 'last_undegraded': '2024-07-17T08:36:33.747960+0000', 'last_fullsized': '2024-07-17T08:36:33.747960+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.2d is not active+clean 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.728163+0000', 'last_change': '2024-07-17T08:36:33.728163+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.728163+0000', 'last_undegraded': '2024-07-17T08:36:33.728163+0000', 'last_fullsized': '2024-07-17T08:36:33.728163+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.30 is not active+clean 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.730370+0000', 'last_change': '2024-07-17T08:36:33.730370+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.730370+0000', 'last_undegraded': '2024-07-17T08:36:33.730370+0000', 'last_fullsized': '2024-07-17T08:36:33.730370+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.36 is not active+clean 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.732033+0000', 'last_change': '2024-07-17T08:36:33.732033+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.732033+0000', 'last_undegraded': '2024-07-17T08:36:33.732033+0000', 'last_fullsized': '2024-07-17T08:36:33.732033+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.37 is not active+clean 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.750569+0000', 'last_change': '2024-07-17T08:36:33.750569+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.750569+0000', 'last_undegraded': '2024-07-17T08:36:33.750569+0000', 'last_fullsized': '2024-07-17T08:36:33.750569+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.365 INFO:tasks.ceph.ceph_manager.ceph:PG 34.34 is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.752613+0000', 'last_change': '2024-07-17T08:36:33.752613+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.752613+0000', 'last_undegraded': '2024-07-17T08:36:33.752613+0000', 'last_fullsized': '2024-07-17T08:36:33.752613+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:PG 34.35 is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.748774+0000', 'last_change': '2024-07-17T08:36:33.748774+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.748774+0000', 'last_undegraded': '2024-07-17T08:36:33.748774+0000', 'last_fullsized': '2024-07-17T08:36:33.748774+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:PG 34.3a is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.741816+0000', 'last_change': '2024-07-17T08:36:33.741816+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.741816+0000', 'last_undegraded': '2024-07-17T08:36:33.741816+0000', 'last_fullsized': '2024-07-17T08:36:33.741816+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:PG 34.3b is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.736575+0000', 'last_change': '2024-07-17T08:36:33.736575+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.736575+0000', 'last_undegraded': '2024-07-17T08:36:33.736575+0000', 'last_fullsized': '2024-07-17T08:36:33.736575+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:PG 34.39 is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.739158+0000', 'last_change': '2024-07-17T08:36:33.739158+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.739158+0000', 'last_undegraded': '2024-07-17T08:36:33.739158+0000', 'last_fullsized': '2024-07-17T08:36:33.739158+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:PG 34.3f is not active+clean 2024-07-17T08:36:40.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.748648+0000', 'last_change': '2024-07-17T08:36:33.748648+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.748648+0000', 'last_undegraded': '2024-07-17T08:36:33.748648+0000', 'last_fullsized': '2024-07-17T08:36:33.748648+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-07-17T08:36:40.367 INFO:tasks.ceph.ceph_manager.ceph:PG 34.3d is not active+clean 2024-07-17T08:36:40.367 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '34.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 245, 'state': 'creating+peering', 'last_fresh': '2024-07-17T08:36:33.743109+0000', 'last_change': '2024-07-17T08:36:33.743109+0000', 'last_active': '2024-07-17T08:36:33.701795+0000', 'last_peered': '2024-07-17T08:36:33.701795+0000', 'last_clean': '2024-07-17T08:36:33.701795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-07-17T08:36:33.743109+0000', 'last_undegraded': '2024-07-17T08:36:33.743109+0000', 'last_fullsized': '2024-07-17T08:36:33.743109+0000', 'mapping_epoch': 245, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 245, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'last_clean_scrub_stamp': '2024-07-17T08:36:33.701795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-07-17T08:36:40.368 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-07-17T08:36:40.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 -- 172.21.15.44:0/3570108477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c0ae810 msgr2=0x7f728c15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 --2- 172.21.15.44:0/3570108477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c0ae810 0x7f728c15e510 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f728c154b50 tx=0x7f727c02a560 comp rx=0 tx=0).stop 2024-07-17T08:36:40.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 -- 172.21.15.44:0/3570108477 shutdown_connections 2024-07-17T08:36:40.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 --2- 172.21.15.44:0/3570108477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c06ea50 0x7f728c0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 --2- 172.21.15.44:0/3570108477 >> v2:172.21.15.175:3300/0 conn(0x7f728c06ee90 0x7f728c0ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 --2- 172.21.15.44:0/3570108477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c0ae810 0x7f728c15e510 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 -- 172.21.15.44:0/3570108477 >> 172.21.15.44:0/3570108477 conn(0x7f728c0b0fb0 msgr2=0x7f728c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:40.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 -- 172.21.15.44:0/3570108477 shutdown_connections 2024-07-17T08:36:40.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.609+0000 7f7292a00640 1 -- 172.21.15.44:0/3570108477 wait complete. 2024-07-17T08:36:40.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.611+0000 7f7292a00640 1 Processor -- start 2024-07-17T08:36:40.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.611+0000 7f7292a00640 1 -- start start 2024-07-17T08:36:40.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.612+0000 7f7292a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c06ea50 0x7f728c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7292a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7292a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae810 0x7f728c144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7292a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0aed70 con 0x7f728c06ee90 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7292a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0aebf0 con 0x7f728c06ea50 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7292a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0b25d0 con 0x7f728c0ae810 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54208/0 (socket says 172.21.15.44:54208) 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 -- 172.21.15.44:0/2077777415 learned_addr learned my addr 172.21.15.44:0/2077777415 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 -- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae810 msgr2=0x7f728c144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae810 0x7f728c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 -- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c06ea50 msgr2=0x7f728c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c06ea50 0x7f728c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.613+0000 7f7283e00640 1 -- 172.21.15.44:0/2077777415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c153c30 con 0x7f728c06ee90 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7283e00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f727400d770 tx=0x7f727400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:40.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274017070 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7274009d20 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274005280 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c0b1850 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.614+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1542a0 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.615+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c0b1a90 con 0x7f728c06ee90 2024-07-17T08:36:40.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.617+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f7274005420 con 0x7f728c06ee90 2024-07-17T08:36:40.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.618+0000 7f7288c00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 0x7f72680792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:40.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.618+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f7274099090 con 0x7f728c06ee90 2024-07-17T08:36:40.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.618+0000 7f728be00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 0x7f72680792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:40.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.620+0000 7f728be00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 0x7f72680792b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f727c033000 tx=0x7f727c0046c0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:40.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.623+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f72740619b0 con 0x7f728c06ee90 2024-07-17T08:36:40.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.923+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f728c15e510 con 0x7f728c06ee90 2024-07-17T08:36:40.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.924+0000 7f7288c00640 1 -- 172.21.15.44:0/2077777415 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1252 (secure 0 0 0) 0x7f72740612b0 con 0x7f728c06ee90 2024-07-17T08:36:40.926 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:40.926 INFO:teuthology.orchestra.run.smithi044.stdout:{"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":6132,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":247,"num_osds":8,"num_up_osds":8,"osd_up_since":1721199282,"num_in_osds":8,"osd_in_since":1721199273,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":156},{"state_name":"creating+peering","count":37}],"num_pgs":193,"num_pools":5,"num_objects":39,"data_bytes":596308,"bytes_used":420806656,"bytes_avail":772673306624,"bytes_total":773094113280,"inactive_pgs_ratio":0.19170984625816345,"write_bytes_sec":1194,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","id":15,"up":1,"in":1,"max":1,"id":16,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":16,"rank":0,"name":"b","status":"up:active","gid":8785},{"filesystem_id":15,"rank":0,"name":"d","status":"up:active","gid":8788}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-07-17T08:33:03.196734+0000","services":{}},"progress_events":{}} 2024-07-17T08:36:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 msgr2=0x7f72680792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 0x7f72680792b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f727c033000 tx=0x7f727c0046c0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 msgr2=0x7f728c144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f727400d770 tx=0x7f727400dc30 comp rx=0 tx=0).stop 2024-07-17T08:36:40.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 shutdown_connections 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7268076df0 0x7f72680792b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae810 0x7f728c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c06ee90 0x7f728c144340 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 --2- 172.21.15.44:0/2077777415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c06ea50 0x7f728c09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 >> 172.21.15.44:0/2077777415 conn(0x7f728c0b0fb0 msgr2=0x7f728c0ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.927+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 shutdown_connections 2024-07-17T08:36:40.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:40.928+0000 7f7292a00640 1 -- 172.21.15.44:0/2077777415 wait complete. 2024-07-17T08:36:40.950 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-07-17T08:36:40.950 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 -- 172.21.15.44:0/2782817324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815b7f0 msgr2=0x7f20e806e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 --2- 172.21.15.44:0/2782817324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815b7f0 0x7f20e806e790 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005fd0 tx=0x7f20e002a8a0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 -- 172.21.15.44:0/2782817324 shutdown_connections 2024-07-17T08:36:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 --2- 172.21.15.44:0/2782817324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815b7f0 0x7f20e806e790 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 --2- 172.21.15.44:0/2782817324 >> v2:172.21.15.175:3300/0 conn(0x7f20e806bf40 0x7f20e80cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 -- 172.21.15.44:0/2782817324 >> 172.21.15.44:0/2782817324 conn(0x7f20e80b2240 msgr2=0x7f20e80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 -- 172.21.15.44:0/2782817324 shutdown_connections 2024-07-17T08:36:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.154+0000 7f20ed800640 1 -- 172.21.15.44:0/2782817324 wait complete. 2024-07-17T08:36:41.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.157+0000 7f20ed800640 1 Processor -- start 2024-07-17T08:36:41.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.157+0000 7f20ed800640 1 -- start start 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 0x7f20e81f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ef10 0x7f20e8141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20e806ecd0 con 0x7f20e806bf40 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20e815bf20 con 0x7f20e815b7f0 2024-07-17T08:36:41.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.158+0000 7f20ed800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20e806ee50 con 0x7f20e815ef10 2024-07-17T08:36:41.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:41.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54220/0 (socket says 172.21.15.44:54220) 2024-07-17T08:36:41.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 -- 172.21.15.44:0/2956785072 learned_addr learned my addr 172.21.15.44:0/2956785072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:41.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e7400640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ef10 0x7f20e8141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:41.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6000640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 0x7f20e81f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:41.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 -- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ef10 msgr2=0x7f20e8141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:41.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ef10 0x7f20e8141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 -- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 msgr2=0x7f20e81f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 0x7f20e81f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6a00640 1 -- 172.21.15.44:0/2956785072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e8142340 con 0x7f20e806bf40 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.159+0000 7f20e6000640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 0x7f20e81f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20e6a00640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f20d800e9f0 tx=0x7f20d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d800ccc0 con 0x7f20e806bf40 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20d800ce60 con 0x7f20e806bf40 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8005280 con 0x7f20e806bf40 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e81f3620 con 0x7f20e806bf40 2024-07-17T08:36:41.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.160+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81f3be0 con 0x7f20e806bf40 2024-07-17T08:36:41.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.161+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e80cfd00 con 0x7f20e806bf40 2024-07-17T08:36:41.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.162+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f20d8005420 con 0x7f20e806bf40 2024-07-17T08:36:41.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.163+0000 7f20dea00640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 0x7f20c80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:41.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.163+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f20d8099990 con 0x7f20e806bf40 2024-07-17T08:36:41.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.164+0000 7f20e6000640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 0x7f20c80792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:41.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.165+0000 7f20e6000640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 0x7f20c80792b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20e8155d00 tx=0x7f20d4009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:41.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.168+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f20d8062220 con 0x7f20e806bf40 2024-07-17T08:36:41.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.377+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20e815fcb0 con 0x7f20c8076df0 2024-07-17T08:36:41.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.400+0000 7f20dea00640 1 -- 172.21.15.44:0/2956785072 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491486 (secure 0 0 0) 0x7f20e815fcb0 con 0x7f20c8076df0 2024-07-17T08:36:41.408 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:41.414 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:41.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.412+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 msgr2=0x7f20c80792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.412+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 0x7f20c80792b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20e8155d00 tx=0x7f20d4009040 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.412+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 msgr2=0x7f20e81e4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.412+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f20d800e9f0 tx=0x7f20d800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.412+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 shutdown_connections 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f20c8076df0 0x7f20c80792b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20e815ef10 0x7f20e8141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20e815b7f0 0x7f20e81f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 --2- 172.21.15.44:0/2956785072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e806bf40 0x7f20e81e4be0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 >> 172.21.15.44:0/2956785072 conn(0x7f20e80b2240 msgr2=0x7f20e80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 shutdown_connections 2024-07-17T08:36:41.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:41.413+0000 7f20ed800640 1 -- 172.21.15.44:0/2956785072 wait complete. 2024-07-17T08:36:41.437 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-07-17T08:36:41.288569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596434,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":475,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2605,"ondisk_log_size":2605,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411132,"kb_used_data":25608,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754563588,"statfs":{"total":773094113280,"available":772673114112,"internally_reserved":0,"allocated":26222592,"data_stored":14837980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.566968"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731112+0000","last_change":"2024-07-17T08:36:34.769780+0000","last_active":"2024-07-17T08:36:35.731112+0000","last_peered":"2024-07-17T08:36:35.731112+0000","last_clean":"2024-07-17T08:36:35.731112+0000","last_became_active":"2024-07-17T08:36:34.769495+0000","last_became_peered":"2024-07-17T08:36:34.769495+0000","last_unstale":"2024-07-17T08:36:35.731112+0000","last_undegraded":"2024-07-17T08:36:35.731112+0000","last_fullsized":"2024-07-17T08:36:35.731112+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:30:35.751750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737302+0000","last_change":"2024-07-17T08:36:33.750014+0000","last_active":"2024-07-17T08:36:35.737302+0000","last_peered":"2024-07-17T08:36:35.737302+0000","last_clean":"2024-07-17T08:36:35.737302+0000","last_became_active":"2024-07-17T08:36:33.749632+0000","last_became_peered":"2024-07-17T08:36:33.749632+0000","last_unstale":"2024-07-17T08:36:35.737302+0000","last_undegraded":"2024-07-17T08:36:35.737302+0000","last_fullsized":"2024-07-17T08:36:35.737302+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:47.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560241+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:36.560241+0000","last_peered":"2024-07-17T08:36:36.560241+0000","last_clean":"2024-07-17T08:36:36.560241+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:36.560241+0000","last_undegraded":"2024-07-17T08:36:36.560241+0000","last_fullsized":"2024-07-17T08:36:36.560241+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736980+0000","last_change":"2024-07-17T08:36:34.763237+0000","last_active":"2024-07-17T08:36:35.736980+0000","last_peered":"2024-07-17T08:36:35.736980+0000","last_clean":"2024-07-17T08:36:35.736980+0000","last_became_active":"2024-07-17T08:36:34.756313+0000","last_became_peered":"2024-07-17T08:36:34.756313+0000","last_unstale":"2024-07-17T08:36:35.736980+0000","last_undegraded":"2024-07-17T08:36:35.736980+0000","last_fullsized":"2024-07-17T08:36:35.736980+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:16.188290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"246'4","reported_seq":18,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731482+0000","last_change":"2024-07-17T08:36:33.758375+0000","last_active":"2024-07-17T08:36:35.731482+0000","last_peered":"2024-07-17T08:36:35.731482+0000","last_clean":"2024-07-17T08:36:35.731482+0000","last_became_active":"2024-07-17T08:36:33.758093+0000","last_became_peered":"2024-07-17T08:36:33.758093+0000","last_unstale":"2024-07-17T08:36:35.731482+0000","last_undegraded":"2024-07-17T08:36:35.731482+0000","last_fullsized":"2024-07-17T08:36:35.731482+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:20.948787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732496+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:35.732496+0000","last_peered":"2024-07-17T08:36:35.732496+0000","last_clean":"2024-07-17T08:36:35.732496+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:35.732496+0000","last_undegraded":"2024-07-17T08:36:35.732496+0000","last_fullsized":"2024-07-17T08:36:35.732496+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406884+0000","last_change":"2024-07-17T08:36:34.766332+0000","last_active":"2024-07-17T08:36:36.406884+0000","last_peered":"2024-07-17T08:36:36.406884+0000","last_clean":"2024-07-17T08:36:36.406884+0000","last_became_active":"2024-07-17T08:36:34.763769+0000","last_became_peered":"2024-07-17T08:36:34.763769+0000","last_unstale":"2024-07-17T08:36:36.406884+0000","last_undegraded":"2024-07-17T08:36:36.406884+0000","last_fullsized":"2024-07-17T08:36:36.406884+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:56.756485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736610+0000","last_change":"2024-07-17T08:36:33.746302+0000","last_active":"2024-07-17T08:36:35.736610+0000","last_peered":"2024-07-17T08:36:35.736610+0000","last_clean":"2024-07-17T08:36:35.736610+0000","last_became_active":"2024-07-17T08:36:33.745779+0000","last_became_peered":"2024-07-17T08:36:33.745779+0000","last_unstale":"2024-07-17T08:36:35.736610+0000","last_undegraded":"2024-07-17T08:36:35.736610+0000","last_fullsized":"2024-07-17T08:36:35.736610+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:21:04.851926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736131+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:35.736131+0000","last_peered":"2024-07-17T08:36:35.736131+0000","last_clean":"2024-07-17T08:36:35.736131+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:35.736131+0000","last_undegraded":"2024-07-17T08:36:35.736131+0000","last_fullsized":"2024-07-17T08:36:35.736131+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731650+0000","last_change":"2024-07-17T08:36:34.737756+0000","last_active":"2024-07-17T08:36:35.731650+0000","last_peered":"2024-07-17T08:36:35.731650+0000","last_clean":"2024-07-17T08:36:35.731650+0000","last_became_active":"2024-07-17T08:36:34.737369+0000","last_became_peered":"2024-07-17T08:36:34.737369+0000","last_unstale":"2024-07-17T08:36:35.731650+0000","last_undegraded":"2024-07-17T08:36:35.731650+0000","last_fullsized":"2024-07-17T08:36:35.731650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:30.548489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560259+0000","last_change":"2024-07-17T08:36:33.755800+0000","last_active":"2024-07-17T08:36:36.560259+0000","last_peered":"2024-07-17T08:36:36.560259+0000","last_clean":"2024-07-17T08:36:36.560259+0000","last_became_active":"2024-07-17T08:36:33.754310+0000","last_became_peered":"2024-07-17T08:36:33.754310+0000","last_unstale":"2024-07-17T08:36:36.560259+0000","last_undegraded":"2024-07-17T08:36:36.560259+0000","last_fullsized":"2024-07-17T08:36:36.560259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736234+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:35.736234+0000","last_peered":"2024-07-17T08:36:35.736234+0000","last_clean":"2024-07-17T08:36:35.736234+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:35.736234+0000","last_undegraded":"2024-07-17T08:36:35.736234+0000","last_fullsized":"2024-07-17T08:36:35.736234+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.767946+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.767625+0000","last_became_peered":"2024-07-17T08:36:34.767625+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:12.760846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:33.747145+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:33.746842+0000","last_became_peered":"2024-07-17T08:36:33.746842+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:31.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737281+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:35.737281+0000","last_peered":"2024-07-17T08:36:35.737281+0000","last_clean":"2024-07-17T08:36:35.737281+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:35.737281+0000","last_undegraded":"2024-07-17T08:36:35.737281+0000","last_fullsized":"2024-07-17T08:36:35.737281+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404806+0000","last_change":"2024-07-17T08:36:34.770512+0000","last_active":"2024-07-17T08:36:36.404806+0000","last_peered":"2024-07-17T08:36:36.404806+0000","last_clean":"2024-07-17T08:36:36.404806+0000","last_became_active":"2024-07-17T08:36:34.770193+0000","last_became_peered":"2024-07-17T08:36:34.770193+0000","last_unstale":"2024-07-17T08:36:36.404806+0000","last_undegraded":"2024-07-17T08:36:36.404806+0000","last_fullsized":"2024-07-17T08:36:36.404806+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:56.938896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735822+0000","last_change":"2024-07-17T08:36:33.755670+0000","last_active":"2024-07-17T08:36:35.735822+0000","last_peered":"2024-07-17T08:36:35.735822+0000","last_clean":"2024-07-17T08:36:35.735822+0000","last_became_active":"2024-07-17T08:36:33.755363+0000","last_became_peered":"2024-07-17T08:36:33.755363+0000","last_unstale":"2024-07-17T08:36:35.735822+0000","last_undegraded":"2024-07-17T08:36:35.735822+0000","last_fullsized":"2024-07-17T08:36:35.735822+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:41.505196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558689+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:36.558689+0000","last_peered":"2024-07-17T08:36:36.558689+0000","last_clean":"2024-07-17T08:36:36.558689+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:36.558689+0000","last_undegraded":"2024-07-17T08:36:36.558689+0000","last_fullsized":"2024-07-17T08:36:36.558689+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558650+0000","last_change":"2024-07-17T08:36:34.757341+0000","last_active":"2024-07-17T08:36:36.558650+0000","last_peered":"2024-07-17T08:36:36.558650+0000","last_clean":"2024-07-17T08:36:36.558650+0000","last_became_active":"2024-07-17T08:36:34.755978+0000","last_became_peered":"2024-07-17T08:36:34.755978+0000","last_unstale":"2024-07-17T08:36:36.558650+0000","last_undegraded":"2024-07-17T08:36:36.558650+0000","last_fullsized":"2024-07-17T08:36:36.558650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:04:27.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735299+0000","last_change":"2024-07-17T08:36:33.748383+0000","last_active":"2024-07-17T08:36:35.735299+0000","last_peered":"2024-07-17T08:36:35.735299+0000","last_clean":"2024-07-17T08:36:35.735299+0000","last_became_active":"2024-07-17T08:36:33.748082+0000","last_became_peered":"2024-07-17T08:36:33.748082+0000","last_unstale":"2024-07-17T08:36:35.735299+0000","last_undegraded":"2024-07-17T08:36:35.735299+0000","last_fullsized":"2024-07-17T08:36:35.735299+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:47:10.743177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560090+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:36.560090+0000","last_peered":"2024-07-17T08:36:36.560090+0000","last_clean":"2024-07-17T08:36:36.560090+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:36.560090+0000","last_undegraded":"2024-07-17T08:36:36.560090+0000","last_fullsized":"2024-07-17T08:36:36.560090+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559030+0000","last_change":"2024-07-17T08:36:34.774215+0000","last_active":"2024-07-17T08:36:36.559030+0000","last_peered":"2024-07-17T08:36:36.559030+0000","last_clean":"2024-07-17T08:36:36.559030+0000","last_became_active":"2024-07-17T08:36:34.773518+0000","last_became_peered":"2024-07-17T08:36:34.773518+0000","last_unstale":"2024-07-17T08:36:36.559030+0000","last_undegraded":"2024-07-17T08:36:36.559030+0000","last_fullsized":"2024-07-17T08:36:36.559030+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:27.370346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734842+0000","last_change":"2024-07-17T08:36:33.756246+0000","last_active":"2024-07-17T08:36:35.734842+0000","last_peered":"2024-07-17T08:36:35.734842+0000","last_clean":"2024-07-17T08:36:35.734842+0000","last_became_active":"2024-07-17T08:36:33.754426+0000","last_became_peered":"2024-07-17T08:36:33.754426+0000","last_unstale":"2024-07-17T08:36:35.734842+0000","last_undegraded":"2024-07-17T08:36:35.734842+0000","last_fullsized":"2024-07-17T08:36:35.734842+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:23.283987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735514+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:35.735514+0000","last_peered":"2024-07-17T08:36:35.735514+0000","last_clean":"2024-07-17T08:36:35.735514+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:35.735514+0000","last_undegraded":"2024-07-17T08:36:35.735514+0000","last_fullsized":"2024-07-17T08:36:35.735514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731147+0000","last_change":"2024-07-17T08:36:34.757574+0000","last_active":"2024-07-17T08:36:35.731147+0000","last_peered":"2024-07-17T08:36:35.731147+0000","last_clean":"2024-07-17T08:36:35.731147+0000","last_became_active":"2024-07-17T08:36:34.756063+0000","last_became_peered":"2024-07-17T08:36:34.756063+0000","last_unstale":"2024-07-17T08:36:35.731147+0000","last_undegraded":"2024-07-17T08:36:35.731147+0000","last_fullsized":"2024-07-17T08:36:35.731147+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:45.933512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737630+0000","last_change":"2024-07-17T08:36:33.754739+0000","last_active":"2024-07-17T08:36:35.737630+0000","last_peered":"2024-07-17T08:36:35.737630+0000","last_clean":"2024-07-17T08:36:35.737630+0000","last_became_active":"2024-07-17T08:36:33.754416+0000","last_became_peered":"2024-07-17T08:36:33.754416+0000","last_unstale":"2024-07-17T08:36:35.737630+0000","last_undegraded":"2024-07-17T08:36:35.737630+0000","last_fullsized":"2024-07-17T08:36:35.737630+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:27.464559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560417+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:36.560417+0000","last_peered":"2024-07-17T08:36:36.560417+0000","last_clean":"2024-07-17T08:36:36.560417+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:36.560417+0000","last_undegraded":"2024-07-17T08:36:36.560417+0000","last_fullsized":"2024-07-17T08:36:36.560417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559256+0000","last_change":"2024-07-17T08:36:34.766632+0000","last_active":"2024-07-17T08:36:36.559256+0000","last_peered":"2024-07-17T08:36:36.559256+0000","last_clean":"2024-07-17T08:36:36.559256+0000","last_became_active":"2024-07-17T08:36:34.766314+0000","last_became_peered":"2024-07-17T08:36:34.766314+0000","last_unstale":"2024-07-17T08:36:36.559256+0000","last_undegraded":"2024-07-17T08:36:36.559256+0000","last_fullsized":"2024-07-17T08:36:36.559256+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:20.443444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559692+0000","last_change":"2024-07-17T08:36:33.754818+0000","last_active":"2024-07-17T08:36:36.559692+0000","last_peered":"2024-07-17T08:36:36.559692+0000","last_clean":"2024-07-17T08:36:36.559692+0000","last_became_active":"2024-07-17T08:36:33.753426+0000","last_became_peered":"2024-07-17T08:36:33.753426+0000","last_unstale":"2024-07-17T08:36:36.559692+0000","last_undegraded":"2024-07-17T08:36:36.559692+0000","last_fullsized":"2024-07-17T08:36:36.559692+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:47.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736433+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:35.736433+0000","last_peered":"2024-07-17T08:36:35.736433+0000","last_clean":"2024-07-17T08:36:35.736433+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:35.736433+0000","last_undegraded":"2024-07-17T08:36:35.736433+0000","last_fullsized":"2024-07-17T08:36:35.736433+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561462+0000","last_change":"2024-07-17T08:36:34.763712+0000","last_active":"2024-07-17T08:36:36.561462+0000","last_peered":"2024-07-17T08:36:36.561462+0000","last_clean":"2024-07-17T08:36:36.561462+0000","last_became_active":"2024-07-17T08:36:34.761551+0000","last_became_peered":"2024-07-17T08:36:34.761551+0000","last_unstale":"2024-07-17T08:36:36.561462+0000","last_undegraded":"2024-07-17T08:36:36.561462+0000","last_fullsized":"2024-07-17T08:36:36.561462+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:05.846381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"246'6","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730737+0000","last_change":"2024-07-17T08:36:33.751230+0000","last_active":"2024-07-17T08:36:35.730737+0000","last_peered":"2024-07-17T08:36:35.730737+0000","last_clean":"2024-07-17T08:36:35.730737+0000","last_became_active":"2024-07-17T08:36:33.750539+0000","last_became_peered":"2024-07-17T08:36:33.750539+0000","last_unstale":"2024-07-17T08:36:35.730737+0000","last_undegraded":"2024-07-17T08:36:35.730737+0000","last_fullsized":"2024-07-17T08:36:35.730737+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:35.074061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736286+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:35.736286+0000","last_peered":"2024-07-17T08:36:35.736286+0000","last_clean":"2024-07-17T08:36:35.736286+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:35.736286+0000","last_undegraded":"2024-07-17T08:36:35.736286+0000","last_fullsized":"2024-07-17T08:36:35.736286+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743570+0000","last_change":"2024-07-17T08:36:34.768425+0000","last_active":"2024-07-17T08:36:35.743570+0000","last_peered":"2024-07-17T08:36:35.743570+0000","last_clean":"2024-07-17T08:36:35.743570+0000","last_became_active":"2024-07-17T08:36:34.767292+0000","last_became_peered":"2024-07-17T08:36:34.767292+0000","last_unstale":"2024-07-17T08:36:35.743570+0000","last_undegraded":"2024-07-17T08:36:35.743570+0000","last_fullsized":"2024-07-17T08:36:35.743570+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:01:05.763645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"246'3","reported_seq":17,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559354+0000","last_change":"2024-07-17T08:36:33.754294+0000","last_active":"2024-07-17T08:36:36.559354+0000","last_peered":"2024-07-17T08:36:36.559354+0000","last_clean":"2024-07-17T08:36:36.559354+0000","last_became_active":"2024-07-17T08:36:33.753457+0000","last_became_peered":"2024-07-17T08:36:33.753457+0000","last_unstale":"2024-07-17T08:36:36.559354+0000","last_undegraded":"2024-07-17T08:36:36.559354+0000","last_fullsized":"2024-07-17T08:36:36.559354+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:47.335972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559369+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:36.559369+0000","last_peered":"2024-07-17T08:36:36.559369+0000","last_clean":"2024-07-17T08:36:36.559369+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:36.559369+0000","last_undegraded":"2024-07-17T08:36:36.559369+0000","last_fullsized":"2024-07-17T08:36:36.559369+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558019+0000","last_change":"2024-07-17T08:36:34.772658+0000","last_active":"2024-07-17T08:36:36.558019+0000","last_peered":"2024-07-17T08:36:36.558019+0000","last_clean":"2024-07-17T08:36:36.558019+0000","last_became_active":"2024-07-17T08:36:34.772287+0000","last_became_peered":"2024-07-17T08:36:34.772287+0000","last_unstale":"2024-07-17T08:36:36.558019+0000","last_undegraded":"2024-07-17T08:36:36.558019+0000","last_fullsized":"2024-07-17T08:36:36.558019+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:52.350056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558126+0000","last_change":"2024-07-17T08:36:33.758000+0000","last_active":"2024-07-17T08:36:36.558126+0000","last_peered":"2024-07-17T08:36:36.558126+0000","last_clean":"2024-07-17T08:36:36.558126+0000","last_became_active":"2024-07-17T08:36:33.757029+0000","last_became_peered":"2024-07-17T08:36:33.757029+0000","last_unstale":"2024-07-17T08:36:36.558126+0000","last_undegraded":"2024-07-17T08:36:36.558126+0000","last_fullsized":"2024-07-17T08:36:36.558126+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:48.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560728+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:36.560728+0000","last_peered":"2024-07-17T08:36:36.560728+0000","last_clean":"2024-07-17T08:36:36.560728+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:36.560728+0000","last_undegraded":"2024-07-17T08:36:36.560728+0000","last_fullsized":"2024-07-17T08:36:36.560728+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734977+0000","last_change":"2024-07-17T08:36:34.756176+0000","last_active":"2024-07-17T08:36:35.734977+0000","last_peered":"2024-07-17T08:36:35.734977+0000","last_clean":"2024-07-17T08:36:35.734977+0000","last_became_active":"2024-07-17T08:36:34.755839+0000","last_became_peered":"2024-07-17T08:36:34.755839+0000","last_unstale":"2024-07-17T08:36:35.734977+0000","last_undegraded":"2024-07-17T08:36:35.734977+0000","last_fullsized":"2024-07-17T08:36:35.734977+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:01.411712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735459+0000","last_change":"2024-07-17T08:36:33.749435+0000","last_active":"2024-07-17T08:36:35.735459+0000","last_peered":"2024-07-17T08:36:35.735459+0000","last_clean":"2024-07-17T08:36:35.735459+0000","last_became_active":"2024-07-17T08:36:33.749168+0000","last_became_peered":"2024-07-17T08:36:33.749168+0000","last_unstale":"2024-07-17T08:36:35.735459+0000","last_undegraded":"2024-07-17T08:36:35.735459+0000","last_fullsized":"2024-07-17T08:36:35.735459+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:49.720522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406087+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:36.406087+0000","last_peered":"2024-07-17T08:36:36.406087+0000","last_clean":"2024-07-17T08:36:36.406087+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:36.406087+0000","last_undegraded":"2024-07-17T08:36:36.406087+0000","last_fullsized":"2024-07-17T08:36:36.406087+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735009+0000","last_change":"2024-07-17T08:36:34.772376+0000","last_active":"2024-07-17T08:36:35.735009+0000","last_peered":"2024-07-17T08:36:35.735009+0000","last_clean":"2024-07-17T08:36:35.735009+0000","last_became_active":"2024-07-17T08:36:34.772066+0000","last_became_peered":"2024-07-17T08:36:34.772066+0000","last_unstale":"2024-07-17T08:36:35.735009+0000","last_undegraded":"2024-07-17T08:36:35.735009+0000","last_fullsized":"2024-07-17T08:36:35.735009+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:41.322562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"246'8","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557060+0000","last_change":"2024-07-17T08:36:33.754851+0000","last_active":"2024-07-17T08:36:36.557060+0000","last_peered":"2024-07-17T08:36:36.557060+0000","last_clean":"2024-07-17T08:36:36.557060+0000","last_became_active":"2024-07-17T08:36:33.754191+0000","last_became_peered":"2024-07-17T08:36:33.754191+0000","last_unstale":"2024-07-17T08:36:36.557060+0000","last_undegraded":"2024-07-17T08:36:36.557060+0000","last_fullsized":"2024-07-17T08:36:36.557060+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:11.746726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742072+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:35.742072+0000","last_peered":"2024-07-17T08:36:35.742072+0000","last_clean":"2024-07-17T08:36:35.742072+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:35.742072+0000","last_undegraded":"2024-07-17T08:36:35.742072+0000","last_fullsized":"2024-07-17T08:36:35.742072+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734906+0000","last_change":"2024-07-17T08:36:34.767985+0000","last_active":"2024-07-17T08:36:35.734906+0000","last_peered":"2024-07-17T08:36:35.734906+0000","last_clean":"2024-07-17T08:36:35.734906+0000","last_became_active":"2024-07-17T08:36:34.767692+0000","last_became_peered":"2024-07-17T08:36:34.767692+0000","last_unstale":"2024-07-17T08:36:35.734906+0000","last_undegraded":"2024-07-17T08:36:35.734906+0000","last_fullsized":"2024-07-17T08:36:35.734906+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:43.793611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557662+0000","last_change":"2024-07-17T08:36:33.758968+0000","last_active":"2024-07-17T08:36:36.557662+0000","last_peered":"2024-07-17T08:36:36.557662+0000","last_clean":"2024-07-17T08:36:36.557662+0000","last_became_active":"2024-07-17T08:36:33.757393+0000","last_became_peered":"2024-07-17T08:36:33.757393+0000","last_unstale":"2024-07-17T08:36:36.557662+0000","last_undegraded":"2024-07-17T08:36:36.557662+0000","last_fullsized":"2024-07-17T08:36:36.557662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:54:23.614399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735634+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:35.735634+0000","last_peered":"2024-07-17T08:36:35.735634+0000","last_clean":"2024-07-17T08:36:35.735634+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:35.735634+0000","last_undegraded":"2024-07-17T08:36:35.735634+0000","last_fullsized":"2024-07-17T08:36:35.735634+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559990+0000","last_change":"2024-07-17T08:36:34.769496+0000","last_active":"2024-07-17T08:36:36.559990+0000","last_peered":"2024-07-17T08:36:36.559990+0000","last_clean":"2024-07-17T08:36:36.559990+0000","last_became_active":"2024-07-17T08:36:34.767061+0000","last_became_peered":"2024-07-17T08:36:34.767061+0000","last_unstale":"2024-07-17T08:36:36.559990+0000","last_undegraded":"2024-07-17T08:36:36.559990+0000","last_fullsized":"2024-07-17T08:36:36.559990+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:53.417936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731725+0000","last_change":"2024-07-17T08:36:33.759374+0000","last_active":"2024-07-17T08:36:35.731725+0000","last_peered":"2024-07-17T08:36:35.731725+0000","last_clean":"2024-07-17T08:36:35.731725+0000","last_became_active":"2024-07-17T08:36:33.759075+0000","last_became_peered":"2024-07-17T08:36:33.759075+0000","last_unstale":"2024-07-17T08:36:35.731725+0000","last_undegraded":"2024-07-17T08:36:35.731725+0000","last_fullsized":"2024-07-17T08:36:35.731725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561997+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:36.561997+0000","last_peered":"2024-07-17T08:36:36.561997+0000","last_clean":"2024-07-17T08:36:36.561997+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:36.561997+0000","last_undegraded":"2024-07-17T08:36:36.561997+0000","last_fullsized":"2024-07-17T08:36:36.561997+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558626+0000","last_change":"2024-07-17T08:36:34.764651+0000","last_active":"2024-07-17T08:36:36.558626+0000","last_peered":"2024-07-17T08:36:36.558626+0000","last_clean":"2024-07-17T08:36:36.558626+0000","last_became_active":"2024-07-17T08:36:34.764374+0000","last_became_peered":"2024-07-17T08:36:34.764374+0000","last_unstale":"2024-07-17T08:36:36.558626+0000","last_undegraded":"2024-07-17T08:36:36.558626+0000","last_fullsized":"2024-07-17T08:36:36.558626+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:38:18.388381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743901+0000","last_change":"2024-07-17T08:36:33.761952+0000","last_active":"2024-07-17T08:36:35.743901+0000","last_peered":"2024-07-17T08:36:35.743901+0000","last_clean":"2024-07-17T08:36:35.743901+0000","last_became_active":"2024-07-17T08:36:33.761707+0000","last_became_peered":"2024-07-17T08:36:33.761707+0000","last_unstale":"2024-07-17T08:36:35.743901+0000","last_undegraded":"2024-07-17T08:36:35.743901+0000","last_fullsized":"2024-07-17T08:36:35.743901+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:26.511292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736609+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:35.736609+0000","last_peered":"2024-07-17T08:36:35.736609+0000","last_clean":"2024-07-17T08:36:35.736609+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:35.736609+0000","last_undegraded":"2024-07-17T08:36:35.736609+0000","last_fullsized":"2024-07-17T08:36:35.736609+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561491+0000","last_change":"2024-07-17T08:36:34.765860+0000","last_active":"2024-07-17T08:36:36.561491+0000","last_peered":"2024-07-17T08:36:36.561491+0000","last_clean":"2024-07-17T08:36:36.561491+0000","last_became_active":"2024-07-17T08:36:34.765534+0000","last_became_peered":"2024-07-17T08:36:34.765534+0000","last_unstale":"2024-07-17T08:36:36.561491+0000","last_undegraded":"2024-07-17T08:36:36.561491+0000","last_fullsized":"2024-07-17T08:36:36.561491+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:55.660953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730838+0000","last_change":"2024-07-17T08:36:33.760094+0000","last_active":"2024-07-17T08:36:35.730838+0000","last_peered":"2024-07-17T08:36:35.730838+0000","last_clean":"2024-07-17T08:36:35.730838+0000","last_became_active":"2024-07-17T08:36:33.757800+0000","last_became_peered":"2024-07-17T08:36:33.757800+0000","last_unstale":"2024-07-17T08:36:35.730838+0000","last_undegraded":"2024-07-17T08:36:35.730838+0000","last_fullsized":"2024-07-17T08:36:35.730838+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:32.367201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743167+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:35.743167+0000","last_peered":"2024-07-17T08:36:35.743167+0000","last_clean":"2024-07-17T08:36:35.743167+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:35.743167+0000","last_undegraded":"2024-07-17T08:36:35.743167+0000","last_fullsized":"2024-07-17T08:36:35.743167+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559766+0000","last_change":"2024-07-17T08:36:34.767120+0000","last_active":"2024-07-17T08:36:36.559766+0000","last_peered":"2024-07-17T08:36:36.559766+0000","last_clean":"2024-07-17T08:36:36.559766+0000","last_became_active":"2024-07-17T08:36:34.766506+0000","last_became_peered":"2024-07-17T08:36:34.766506+0000","last_unstale":"2024-07-17T08:36:36.559766+0000","last_undegraded":"2024-07-17T08:36:36.559766+0000","last_fullsized":"2024-07-17T08:36:36.559766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:50.347213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560996+0000","last_change":"2024-07-17T08:36:33.750553+0000","last_active":"2024-07-17T08:36:36.560996+0000","last_peered":"2024-07-17T08:36:36.560996+0000","last_clean":"2024-07-17T08:36:36.560996+0000","last_became_active":"2024-07-17T08:36:33.750105+0000","last_became_peered":"2024-07-17T08:36:33.750105+0000","last_unstale":"2024-07-17T08:36:36.560996+0000","last_undegraded":"2024-07-17T08:36:36.560996+0000","last_fullsized":"2024-07-17T08:36:36.560996+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:45.778285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731232+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:35.731232+0000","last_peered":"2024-07-17T08:36:35.731232+0000","last_clean":"2024-07-17T08:36:35.731232+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:35.731232+0000","last_undegraded":"2024-07-17T08:36:35.731232+0000","last_fullsized":"2024-07-17T08:36:35.731232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558254+0000","last_change":"2024-07-17T08:36:34.762489+0000","last_active":"2024-07-17T08:36:36.558254+0000","last_peered":"2024-07-17T08:36:36.558254+0000","last_clean":"2024-07-17T08:36:36.558254+0000","last_became_active":"2024-07-17T08:36:34.762152+0000","last_became_peered":"2024-07-17T08:36:34.762152+0000","last_unstale":"2024-07-17T08:36:36.558254+0000","last_undegraded":"2024-07-17T08:36:36.558254+0000","last_fullsized":"2024-07-17T08:36:36.558254+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:34.839616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735072+0000","last_change":"2024-07-17T08:36:33.747393+0000","last_active":"2024-07-17T08:36:35.735072+0000","last_peered":"2024-07-17T08:36:35.735072+0000","last_clean":"2024-07-17T08:36:35.735072+0000","last_became_active":"2024-07-17T08:36:33.746617+0000","last_became_peered":"2024-07-17T08:36:33.746617+0000","last_unstale":"2024-07-17T08:36:35.735072+0000","last_undegraded":"2024-07-17T08:36:35.735072+0000","last_fullsized":"2024-07-17T08:36:35.735072+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:44.524775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559848+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:36.559848+0000","last_peered":"2024-07-17T08:36:36.559848+0000","last_clean":"2024-07-17T08:36:36.559848+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:36.559848+0000","last_undegraded":"2024-07-17T08:36:36.559848+0000","last_fullsized":"2024-07-17T08:36:36.559848+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405145+0000","last_change":"2024-07-17T08:36:34.763807+0000","last_active":"2024-07-17T08:36:36.405145+0000","last_peered":"2024-07-17T08:36:36.405145+0000","last_clean":"2024-07-17T08:36:36.405145+0000","last_became_active":"2024-07-17T08:36:34.763502+0000","last_became_peered":"2024-07-17T08:36:34.763502+0000","last_unstale":"2024-07-17T08:36:36.405145+0000","last_undegraded":"2024-07-17T08:36:36.405145+0000","last_fullsized":"2024-07-17T08:36:36.405145+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:52.862035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735458+0000","last_change":"2024-07-17T08:36:33.756251+0000","last_active":"2024-07-17T08:36:35.735458+0000","last_peered":"2024-07-17T08:36:35.735458+0000","last_clean":"2024-07-17T08:36:35.735458+0000","last_became_active":"2024-07-17T08:36:33.755982+0000","last_became_peered":"2024-07-17T08:36:33.755982+0000","last_unstale":"2024-07-17T08:36:35.735458+0000","last_undegraded":"2024-07-17T08:36:35.735458+0000","last_fullsized":"2024-07-17T08:36:35.735458+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:04.834399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558966+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:36.558966+0000","last_peered":"2024-07-17T08:36:36.558966+0000","last_clean":"2024-07-17T08:36:36.558966+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:36.558966+0000","last_undegraded":"2024-07-17T08:36:36.558966+0000","last_fullsized":"2024-07-17T08:36:36.558966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.761196+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.760906+0000","last_became_peered":"2024-07-17T08:36:34.760906+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:39.899314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557265+0000","last_change":"2024-07-17T08:36:33.753248+0000","last_active":"2024-07-17T08:36:36.557265+0000","last_peered":"2024-07-17T08:36:36.557265+0000","last_clean":"2024-07-17T08:36:36.557265+0000","last_became_active":"2024-07-17T08:36:33.752921+0000","last_became_peered":"2024-07-17T08:36:33.752921+0000","last_unstale":"2024-07-17T08:36:36.557265+0000","last_undegraded":"2024-07-17T08:36:36.557265+0000","last_fullsized":"2024-07-17T08:36:36.557265+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:46.641811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742343+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:35.742343+0000","last_peered":"2024-07-17T08:36:35.742343+0000","last_clean":"2024-07-17T08:36:35.742343+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:35.742343+0000","last_undegraded":"2024-07-17T08:36:35.742343+0000","last_fullsized":"2024-07-17T08:36:35.742343+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557766+0000","last_change":"2024-07-17T08:36:34.766089+0000","last_active":"2024-07-17T08:36:36.557766+0000","last_peered":"2024-07-17T08:36:36.557766+0000","last_clean":"2024-07-17T08:36:36.557766+0000","last_became_active":"2024-07-17T08:36:34.762344+0000","last_became_peered":"2024-07-17T08:36:34.762344+0000","last_unstale":"2024-07-17T08:36:36.557766+0000","last_undegraded":"2024-07-17T08:36:36.557766+0000","last_fullsized":"2024-07-17T08:36:36.557766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:43.102170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406148+0000","last_change":"2024-07-17T08:36:33.735145+0000","last_active":"2024-07-17T08:36:36.406148+0000","last_peered":"2024-07-17T08:36:36.406148+0000","last_clean":"2024-07-17T08:36:36.406148+0000","last_became_active":"2024-07-17T08:36:33.734773+0000","last_became_peered":"2024-07-17T08:36:33.734773+0000","last_unstale":"2024-07-17T08:36:36.406148+0000","last_undegraded":"2024-07-17T08:36:36.406148+0000","last_fullsized":"2024-07-17T08:36:36.406148+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:06.368749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557818+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:36.557818+0000","last_peered":"2024-07-17T08:36:36.557818+0000","last_clean":"2024-07-17T08:36:36.557818+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:36.557818+0000","last_undegraded":"2024-07-17T08:36:36.557818+0000","last_fullsized":"2024-07-17T08:36:36.557818+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742864+0000","last_change":"2024-07-17T08:36:34.769570+0000","last_active":"2024-07-17T08:36:35.742864+0000","last_peered":"2024-07-17T08:36:35.742864+0000","last_clean":"2024-07-17T08:36:35.742864+0000","last_became_active":"2024-07-17T08:36:34.769300+0000","last_became_peered":"2024-07-17T08:36:34.769300+0000","last_unstale":"2024-07-17T08:36:35.742864+0000","last_undegraded":"2024-07-17T08:36:35.742864+0000","last_fullsized":"2024-07-17T08:36:35.742864+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:54.576193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"247'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732163+0000","last_change":"2024-07-17T08:36:33.758508+0000","last_active":"2024-07-17T08:36:35.732163+0000","last_peered":"2024-07-17T08:36:35.732163+0000","last_clean":"2024-07-17T08:36:35.732163+0000","last_became_active":"2024-07-17T08:36:33.757683+0000","last_became_peered":"2024-07-17T08:36:33.757683+0000","last_unstale":"2024-07-17T08:36:35.732163+0000","last_undegraded":"2024-07-17T08:36:35.732163+0000","last_fullsized":"2024-07-17T08:36:35.732163+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:13.951810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732029+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:35.732029+0000","last_peered":"2024-07-17T08:36:35.732029+0000","last_clean":"2024-07-17T08:36:35.732029+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:35.732029+0000","last_undegraded":"2024-07-17T08:36:35.732029+0000","last_fullsized":"2024-07-17T08:36:35.732029+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743580+0000","last_change":"2024-07-17T08:36:34.762378+0000","last_active":"2024-07-17T08:36:35.743580+0000","last_peered":"2024-07-17T08:36:35.743580+0000","last_clean":"2024-07-17T08:36:35.743580+0000","last_became_active":"2024-07-17T08:36:34.761575+0000","last_became_peered":"2024-07-17T08:36:34.761575+0000","last_unstale":"2024-07-17T08:36:35.743580+0000","last_undegraded":"2024-07-17T08:36:35.743580+0000","last_fullsized":"2024-07-17T08:36:35.743580+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:32.981640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559978+0000","last_change":"2024-07-17T08:36:33.757914+0000","last_active":"2024-07-17T08:36:36.559978+0000","last_peered":"2024-07-17T08:36:36.559978+0000","last_clean":"2024-07-17T08:36:36.559978+0000","last_became_active":"2024-07-17T08:36:33.757588+0000","last_became_peered":"2024-07-17T08:36:33.757588+0000","last_unstale":"2024-07-17T08:36:36.559978+0000","last_undegraded":"2024-07-17T08:36:36.559978+0000","last_fullsized":"2024-07-17T08:36:36.559978+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:32.359840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560887+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:36.560887+0000","last_peered":"2024-07-17T08:36:36.560887+0000","last_clean":"2024-07-17T08:36:36.560887+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:36.560887+0000","last_undegraded":"2024-07-17T08:36:36.560887+0000","last_fullsized":"2024-07-17T08:36:36.560887+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743750+0000","last_change":"2024-07-17T08:36:34.771365+0000","last_active":"2024-07-17T08:36:35.743750+0000","last_peered":"2024-07-17T08:36:35.743750+0000","last_clean":"2024-07-17T08:36:35.743750+0000","last_became_active":"2024-07-17T08:36:34.770738+0000","last_became_peered":"2024-07-17T08:36:34.770738+0000","last_unstale":"2024-07-17T08:36:35.743750+0000","last_undegraded":"2024-07-17T08:36:35.743750+0000","last_fullsized":"2024-07-17T08:36:35.743750+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:17:43.891881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561664+0000","last_change":"2024-07-17T08:36:33.756054+0000","last_active":"2024-07-17T08:36:36.561664+0000","last_peered":"2024-07-17T08:36:36.561664+0000","last_clean":"2024-07-17T08:36:36.561664+0000","last_became_active":"2024-07-17T08:36:33.755748+0000","last_became_peered":"2024-07-17T08:36:33.755748+0000","last_unstale":"2024-07-17T08:36:36.561664+0000","last_undegraded":"2024-07-17T08:36:36.561664+0000","last_fullsized":"2024-07-17T08:36:36.561664+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:43.965300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736609+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:35.736609+0000","last_peered":"2024-07-17T08:36:35.736609+0000","last_clean":"2024-07-17T08:36:35.736609+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:35.736609+0000","last_undegraded":"2024-07-17T08:36:35.736609+0000","last_fullsized":"2024-07-17T08:36:35.736609+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560249+0000","last_change":"2024-07-17T08:36:34.761601+0000","last_active":"2024-07-17T08:36:36.560249+0000","last_peered":"2024-07-17T08:36:36.560249+0000","last_clean":"2024-07-17T08:36:36.560249+0000","last_became_active":"2024-07-17T08:36:34.761306+0000","last_became_peered":"2024-07-17T08:36:34.761306+0000","last_unstale":"2024-07-17T08:36:36.560249+0000","last_undegraded":"2024-07-17T08:36:36.560249+0000","last_fullsized":"2024-07-17T08:36:36.560249+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:04.486870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560154+0000","last_change":"2024-07-17T08:36:33.756539+0000","last_active":"2024-07-17T08:36:36.560154+0000","last_peered":"2024-07-17T08:36:36.560154+0000","last_clean":"2024-07-17T08:36:36.560154+0000","last_became_active":"2024-07-17T08:36:33.755822+0000","last_became_peered":"2024-07-17T08:36:33.755822+0000","last_unstale":"2024-07-17T08:36:36.560154+0000","last_undegraded":"2024-07-17T08:36:36.560154+0000","last_fullsized":"2024-07-17T08:36:36.560154+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:34.567530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559586+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:36.559586+0000","last_peered":"2024-07-17T08:36:36.559586+0000","last_clean":"2024-07-17T08:36:36.559586+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:36.559586+0000","last_undegraded":"2024-07-17T08:36:36.559586+0000","last_fullsized":"2024-07-17T08:36:36.559586+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405849+0000","last_change":"2024-07-17T08:36:34.772888+0000","last_active":"2024-07-17T08:36:36.405849+0000","last_peered":"2024-07-17T08:36:36.405849+0000","last_clean":"2024-07-17T08:36:36.405849+0000","last_became_active":"2024-07-17T08:36:34.771828+0000","last_became_peered":"2024-07-17T08:36:34.771828+0000","last_unstale":"2024-07-17T08:36:36.405849+0000","last_undegraded":"2024-07-17T08:36:36.405849+0000","last_fullsized":"2024-07-17T08:36:36.405849+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:24.603843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730262+0000","last_change":"2024-07-17T08:36:33.755126+0000","last_active":"2024-07-17T08:36:35.730262+0000","last_peered":"2024-07-17T08:36:35.730262+0000","last_clean":"2024-07-17T08:36:35.730262+0000","last_became_active":"2024-07-17T08:36:33.753919+0000","last_became_peered":"2024-07-17T08:36:33.753919+0000","last_unstale":"2024-07-17T08:36:35.730262+0000","last_undegraded":"2024-07-17T08:36:35.730262+0000","last_fullsized":"2024-07-17T08:36:35.730262+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405878+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:36.405878+0000","last_peered":"2024-07-17T08:36:36.405878+0000","last_clean":"2024-07-17T08:36:36.405878+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:36.405878+0000","last_undegraded":"2024-07-17T08:36:36.405878+0000","last_fullsized":"2024-07-17T08:36:36.405878+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557257+0000","last_change":"2024-07-17T08:36:34.757792+0000","last_active":"2024-07-17T08:36:36.557257+0000","last_peered":"2024-07-17T08:36:36.557257+0000","last_clean":"2024-07-17T08:36:36.557257+0000","last_became_active":"2024-07-17T08:36:34.757510+0000","last_became_peered":"2024-07-17T08:36:34.757510+0000","last_unstale":"2024-07-17T08:36:36.557257+0000","last_undegraded":"2024-07-17T08:36:36.557257+0000","last_fullsized":"2024-07-17T08:36:36.557257+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:05.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730760+0000","last_change":"2024-07-17T08:36:33.758782+0000","last_active":"2024-07-17T08:36:35.730760+0000","last_peered":"2024-07-17T08:36:35.730760+0000","last_clean":"2024-07-17T08:36:35.730760+0000","last_became_active":"2024-07-17T08:36:33.757603+0000","last_became_peered":"2024-07-17T08:36:33.757603+0000","last_unstale":"2024-07-17T08:36:35.730760+0000","last_undegraded":"2024-07-17T08:36:35.730760+0000","last_fullsized":"2024-07-17T08:36:35.730760+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:01.305456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736740+0000","last_change":"2024-07-17T08:36:34.747846+0000","last_active":"2024-07-17T08:36:35.736740+0000","last_peered":"2024-07-17T08:36:35.736740+0000","last_clean":"2024-07-17T08:36:35.736740+0000","last_became_active":"2024-07-17T08:36:34.746565+0000","last_became_peered":"2024-07-17T08:36:34.746565+0000","last_unstale":"2024-07-17T08:36:35.736740+0000","last_undegraded":"2024-07-17T08:36:35.736740+0000","last_fullsized":"2024-07-17T08:36:35.736740+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:52.538461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.729622+0000","last_change":"2024-07-17T08:36:33.735347+0000","last_active":"2024-07-17T08:36:35.729622+0000","last_peered":"2024-07-17T08:36:35.729622+0000","last_clean":"2024-07-17T08:36:35.729622+0000","last_became_active":"2024-07-17T08:36:33.734974+0000","last_became_peered":"2024-07-17T08:36:33.734974+0000","last_unstale":"2024-07-17T08:36:35.729622+0000","last_undegraded":"2024-07-17T08:36:35.729622+0000","last_fullsized":"2024-07-17T08:36:35.729622+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:04.954732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730287+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:35.730287+0000","last_peered":"2024-07-17T08:36:35.730287+0000","last_clean":"2024-07-17T08:36:35.730287+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:35.730287+0000","last_undegraded":"2024-07-17T08:36:35.730287+0000","last_fullsized":"2024-07-17T08:36:35.730287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405245+0000","last_change":"2024-07-17T08:36:34.771421+0000","last_active":"2024-07-17T08:36:36.405245+0000","last_peered":"2024-07-17T08:36:36.405245+0000","last_clean":"2024-07-17T08:36:36.405245+0000","last_became_active":"2024-07-17T08:36:34.770940+0000","last_became_peered":"2024-07-17T08:36:34.770940+0000","last_unstale":"2024-07-17T08:36:36.405245+0000","last_undegraded":"2024-07-17T08:36:36.405245+0000","last_fullsized":"2024-07-17T08:36:36.405245+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:32.518091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559823+0000","last_change":"2024-07-17T08:36:33.749168+0000","last_active":"2024-07-17T08:36:36.559823+0000","last_peered":"2024-07-17T08:36:36.559823+0000","last_clean":"2024-07-17T08:36:36.559823+0000","last_became_active":"2024-07-17T08:36:33.748825+0000","last_became_peered":"2024-07-17T08:36:33.748825+0000","last_unstale":"2024-07-17T08:36:36.559823+0000","last_undegraded":"2024-07-17T08:36:36.559823+0000","last_fullsized":"2024-07-17T08:36:36.559823+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:35.616914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558155+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:36.558155+0000","last_peered":"2024-07-17T08:36:36.558155+0000","last_clean":"2024-07-17T08:36:36.558155+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:36.558155+0000","last_undegraded":"2024-07-17T08:36:36.558155+0000","last_fullsized":"2024-07-17T08:36:36.558155+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742019+0000","last_change":"2024-07-17T08:36:34.759506+0000","last_active":"2024-07-17T08:36:35.742019+0000","last_peered":"2024-07-17T08:36:35.742019+0000","last_clean":"2024-07-17T08:36:35.742019+0000","last_became_active":"2024-07-17T08:36:34.757200+0000","last_became_peered":"2024-07-17T08:36:34.757200+0000","last_unstale":"2024-07-17T08:36:35.742019+0000","last_undegraded":"2024-07-17T08:36:35.742019+0000","last_fullsized":"2024-07-17T08:36:35.742019+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:29:59.072833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730220+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:35.730220+0000","last_peered":"2024-07-17T08:36:35.730220+0000","last_clean":"2024-07-17T08:36:35.730220+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:35.730220+0000","last_undegraded":"2024-07-17T08:36:35.730220+0000","last_fullsized":"2024-07-17T08:36:35.730220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":26,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560226+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:36.560226+0000","last_peered":"2024-07-17T08:36:36.560226+0000","last_clean":"2024-07-17T08:36:36.560226+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:36.560226+0000","last_undegraded":"2024-07-17T08:36:36.560226+0000","last_fullsized":"2024-07-17T08:36:36.560226+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404648+0000","last_change":"2024-07-17T08:36:34.756329+0000","last_active":"2024-07-17T08:36:36.404648+0000","last_peered":"2024-07-17T08:36:36.404648+0000","last_clean":"2024-07-17T08:36:36.404648+0000","last_became_active":"2024-07-17T08:36:34.756056+0000","last_became_peered":"2024-07-17T08:36:34.756056+0000","last_unstale":"2024-07-17T08:36:36.404648+0000","last_undegraded":"2024-07-17T08:36:36.404648+0000","last_fullsized":"2024-07-17T08:36:36.404648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:00.211357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742286+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:35.742286+0000","last_peered":"2024-07-17T08:36:35.742286+0000","last_clean":"2024-07-17T08:36:35.742286+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:35.742286+0000","last_undegraded":"2024-07-17T08:36:35.742286+0000","last_fullsized":"2024-07-17T08:36:35.742286+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560752+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:36.560752+0000","last_peered":"2024-07-17T08:36:36.560752+0000","last_clean":"2024-07-17T08:36:36.560752+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:36.560752+0000","last_undegraded":"2024-07-17T08:36:36.560752+0000","last_fullsized":"2024-07-17T08:36:36.560752+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406536+0000","last_change":"2024-07-17T08:36:34.776531+0000","last_active":"2024-07-17T08:36:36.406536+0000","last_peered":"2024-07-17T08:36:36.406536+0000","last_clean":"2024-07-17T08:36:36.406536+0000","last_became_active":"2024-07-17T08:36:34.776010+0000","last_became_peered":"2024-07-17T08:36:34.776010+0000","last_unstale":"2024-07-17T08:36:36.406536+0000","last_undegraded":"2024-07-17T08:36:36.406536+0000","last_fullsized":"2024-07-17T08:36:36.406536+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:27.058250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559158+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:36.559158+0000","last_peered":"2024-07-17T08:36:36.559158+0000","last_clean":"2024-07-17T08:36:36.559158+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:36.559158+0000","last_undegraded":"2024-07-17T08:36:36.559158+0000","last_fullsized":"2024-07-17T08:36:36.559158+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406585+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:36.406585+0000","last_peered":"2024-07-17T08:36:36.406585+0000","last_clean":"2024-07-17T08:36:36.406585+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:36.406585+0000","last_undegraded":"2024-07-17T08:36:36.406585+0000","last_fullsized":"2024-07-17T08:36:36.406585+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407015+0000","last_change":"2024-07-17T08:36:34.768405+0000","last_active":"2024-07-17T08:36:36.407015+0000","last_peered":"2024-07-17T08:36:36.407015+0000","last_clean":"2024-07-17T08:36:36.407015+0000","last_became_active":"2024-07-17T08:36:34.768128+0000","last_became_peered":"2024-07-17T08:36:34.768128+0000","last_unstale":"2024-07-17T08:36:36.407015+0000","last_undegraded":"2024-07-17T08:36:36.407015+0000","last_fullsized":"2024-07-17T08:36:36.407015+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:26.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405806+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:36.405806+0000","last_peered":"2024-07-17T08:36:36.405806+0000","last_clean":"2024-07-17T08:36:36.405806+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:36.405806+0000","last_undegraded":"2024-07-17T08:36:36.405806+0000","last_fullsized":"2024-07-17T08:36:36.405806+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742501+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:35.742501+0000","last_peered":"2024-07-17T08:36:35.742501+0000","last_clean":"2024-07-17T08:36:35.742501+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:35.742501+0000","last_undegraded":"2024-07-17T08:36:35.742501+0000","last_fullsized":"2024-07-17T08:36:35.742501+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561721+0000","last_change":"2024-07-17T08:36:34.762335+0000","last_active":"2024-07-17T08:36:36.561721+0000","last_peered":"2024-07-17T08:36:36.561721+0000","last_clean":"2024-07-17T08:36:36.561721+0000","last_became_active":"2024-07-17T08:36:34.762073+0000","last_became_peered":"2024-07-17T08:36:34.762073+0000","last_unstale":"2024-07-17T08:36:36.561721+0000","last_undegraded":"2024-07-17T08:36:36.561721+0000","last_fullsized":"2024-07-17T08:36:36.561721+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:47.721727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561037+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:36.561037+0000","last_peered":"2024-07-17T08:36:36.561037+0000","last_clean":"2024-07-17T08:36:36.561037+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:36.561037+0000","last_undegraded":"2024-07-17T08:36:36.561037+0000","last_fullsized":"2024-07-17T08:36:36.561037+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407040+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:36.407040+0000","last_peered":"2024-07-17T08:36:36.407040+0000","last_clean":"2024-07-17T08:36:36.407040+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:36.407040+0000","last_undegraded":"2024-07-17T08:36:36.407040+0000","last_fullsized":"2024-07-17T08:36:36.407040+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559358+0000","last_change":"2024-07-17T08:36:34.772217+0000","last_active":"2024-07-17T08:36:36.559358+0000","last_peered":"2024-07-17T08:36:36.559358+0000","last_clean":"2024-07-17T08:36:36.559358+0000","last_became_active":"2024-07-17T08:36:34.771921+0000","last_became_peered":"2024-07-17T08:36:34.771921+0000","last_unstale":"2024-07-17T08:36:36.559358+0000","last_undegraded":"2024-07-17T08:36:36.559358+0000","last_fullsized":"2024-07-17T08:36:36.559358+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:46.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736365+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:35.736365+0000","last_peered":"2024-07-17T08:36:35.736365+0000","last_clean":"2024-07-17T08:36:35.736365+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:35.736365+0000","last_undegraded":"2024-07-17T08:36:35.736365+0000","last_fullsized":"2024-07-17T08:36:35.736365+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734862+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:35.734862+0000","last_peered":"2024-07-17T08:36:35.734862+0000","last_clean":"2024-07-17T08:36:35.734862+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:35.734862+0000","last_undegraded":"2024-07-17T08:36:35.734862+0000","last_fullsized":"2024-07-17T08:36:35.734862+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559537+0000","last_change":"2024-07-17T08:36:34.772619+0000","last_active":"2024-07-17T08:36:36.559537+0000","last_peered":"2024-07-17T08:36:36.559537+0000","last_clean":"2024-07-17T08:36:36.559537+0000","last_became_active":"2024-07-17T08:36:34.772316+0000","last_became_peered":"2024-07-17T08:36:34.772316+0000","last_unstale":"2024-07-17T08:36:36.559537+0000","last_undegraded":"2024-07-17T08:36:36.559537+0000","last_fullsized":"2024-07-17T08:36:36.559537+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:04.078178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737996+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:35.737996+0000","last_peered":"2024-07-17T08:36:35.737996+0000","last_clean":"2024-07-17T08:36:35.737996+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:35.737996+0000","last_undegraded":"2024-07-17T08:36:35.737996+0000","last_fullsized":"2024-07-17T08:36:35.737996+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558367+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:36.558367+0000","last_peered":"2024-07-17T08:36:36.558367+0000","last_clean":"2024-07-17T08:36:36.558367+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:36.558367+0000","last_undegraded":"2024-07-17T08:36:36.558367+0000","last_fullsized":"2024-07-17T08:36:36.558367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559393+0000","last_change":"2024-07-17T08:36:34.775531+0000","last_active":"2024-07-17T08:36:36.559393+0000","last_peered":"2024-07-17T08:36:36.559393+0000","last_clean":"2024-07-17T08:36:36.559393+0000","last_became_active":"2024-07-17T08:36:34.775165+0000","last_became_peered":"2024-07-17T08:36:34.775165+0000","last_unstale":"2024-07-17T08:36:36.559393+0000","last_undegraded":"2024-07-17T08:36:36.559393+0000","last_fullsized":"2024-07-17T08:36:36.559393+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:06:11.650539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558966+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:36.558966+0000","last_peered":"2024-07-17T08:36:36.558966+0000","last_clean":"2024-07-17T08:36:36.558966+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:36.558966+0000","last_undegraded":"2024-07-17T08:36:36.558966+0000","last_fullsized":"2024-07-17T08:36:36.558966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407193+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:36.407193+0000","last_peered":"2024-07-17T08:36:36.407193+0000","last_clean":"2024-07-17T08:36:36.407193+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:36.407193+0000","last_undegraded":"2024-07-17T08:36:36.407193+0000","last_fullsized":"2024-07-17T08:36:36.407193+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735122+0000","last_change":"2024-07-17T08:36:34.750275+0000","last_active":"2024-07-17T08:36:35.735122+0000","last_peered":"2024-07-17T08:36:35.735122+0000","last_clean":"2024-07-17T08:36:35.735122+0000","last_became_active":"2024-07-17T08:36:34.749980+0000","last_became_peered":"2024-07-17T08:36:34.749980+0000","last_unstale":"2024-07-17T08:36:35.735122+0000","last_undegraded":"2024-07-17T08:36:35.735122+0000","last_fullsized":"2024-07-17T08:36:35.735122+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:45.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558551+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:36.558551+0000","last_peered":"2024-07-17T08:36:36.558551+0000","last_clean":"2024-07-17T08:36:36.558551+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:36.558551+0000","last_undegraded":"2024-07-17T08:36:36.558551+0000","last_fullsized":"2024-07-17T08:36:36.558551+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736687+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:35.736687+0000","last_peered":"2024-07-17T08:36:35.736687+0000","last_clean":"2024-07-17T08:36:35.736687+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:35.736687+0000","last_undegraded":"2024-07-17T08:36:35.736687+0000","last_fullsized":"2024-07-17T08:36:35.736687+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557833+0000","last_change":"2024-07-17T08:36:34.764300+0000","last_active":"2024-07-17T08:36:36.557833+0000","last_peered":"2024-07-17T08:36:36.557833+0000","last_clean":"2024-07-17T08:36:36.557833+0000","last_became_active":"2024-07-17T08:36:34.764033+0000","last_became_peered":"2024-07-17T08:36:34.764033+0000","last_unstale":"2024-07-17T08:36:36.557833+0000","last_undegraded":"2024-07-17T08:36:36.557833+0000","last_fullsized":"2024-07-17T08:36:36.557833+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:00:23.399806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735197+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:35.735197+0000","last_peered":"2024-07-17T08:36:35.735197+0000","last_clean":"2024-07-17T08:36:35.735197+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:35.735197+0000","last_undegraded":"2024-07-17T08:36:35.735197+0000","last_fullsized":"2024-07-17T08:36:35.735197+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":25,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736803+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:35.736803+0000","last_peered":"2024-07-17T08:36:35.736803+0000","last_clean":"2024-07-17T08:36:35.736803+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:35.736803+0000","last_undegraded":"2024-07-17T08:36:35.736803+0000","last_fullsized":"2024-07-17T08:36:35.736803+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404878+0000","last_change":"2024-07-17T08:36:34.761187+0000","last_active":"2024-07-17T08:36:36.404878+0000","last_peered":"2024-07-17T08:36:36.404878+0000","last_clean":"2024-07-17T08:36:36.404878+0000","last_became_active":"2024-07-17T08:36:34.760833+0000","last_became_peered":"2024-07-17T08:36:34.760833+0000","last_unstale":"2024-07-17T08:36:36.404878+0000","last_undegraded":"2024-07-17T08:36:36.404878+0000","last_fullsized":"2024-07-17T08:36:36.404878+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:49.833097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559560+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:36.559560+0000","last_peered":"2024-07-17T08:36:36.559560+0000","last_clean":"2024-07-17T08:36:36.559560+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:36.559560+0000","last_undegraded":"2024-07-17T08:36:36.559560+0000","last_fullsized":"2024-07-17T08:36:36.559560+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559818+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:36.559818+0000","last_peered":"2024-07-17T08:36:36.559818+0000","last_clean":"2024-07-17T08:36:36.559818+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:36.559818+0000","last_undegraded":"2024-07-17T08:36:36.559818+0000","last_fullsized":"2024-07-17T08:36:36.559818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732279+0000","last_change":"2024-07-17T08:36:34.759236+0000","last_active":"2024-07-17T08:36:35.732279+0000","last_peered":"2024-07-17T08:36:35.732279+0000","last_clean":"2024-07-17T08:36:35.732279+0000","last_became_active":"2024-07-17T08:36:34.751420+0000","last_became_peered":"2024-07-17T08:36:34.751420+0000","last_unstale":"2024-07-17T08:36:35.732279+0000","last_undegraded":"2024-07-17T08:36:35.732279+0000","last_fullsized":"2024-07-17T08:36:35.732279+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731166+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:35.731166+0000","last_peered":"2024-07-17T08:36:35.731166+0000","last_clean":"2024-07-17T08:36:35.731166+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:35.731166+0000","last_undegraded":"2024-07-17T08:36:35.731166+0000","last_fullsized":"2024-07-17T08:36:35.731166+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742818+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:35.742818+0000","last_peered":"2024-07-17T08:36:35.742818+0000","last_clean":"2024-07-17T08:36:35.742818+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:35.742818+0000","last_undegraded":"2024-07-17T08:36:35.742818+0000","last_fullsized":"2024-07-17T08:36:35.742818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737727+0000","last_change":"2024-07-17T08:36:34.756689+0000","last_active":"2024-07-17T08:36:35.737727+0000","last_peered":"2024-07-17T08:36:35.737727+0000","last_clean":"2024-07-17T08:36:35.737727+0000","last_became_active":"2024-07-17T08:36:34.755385+0000","last_became_peered":"2024-07-17T08:36:34.755385+0000","last_unstale":"2024-07-17T08:36:35.737727+0000","last_undegraded":"2024-07-17T08:36:35.737727+0000","last_fullsized":"2024-07-17T08:36:35.737727+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:55.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560741+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:36.560741+0000","last_peered":"2024-07-17T08:36:36.560741+0000","last_clean":"2024-07-17T08:36:36.560741+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:36.560741+0000","last_undegraded":"2024-07-17T08:36:36.560741+0000","last_fullsized":"2024-07-17T08:36:36.560741+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736105+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:35.736105+0000","last_peered":"2024-07-17T08:36:35.736105+0000","last_clean":"2024-07-17T08:36:35.736105+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:35.736105+0000","last_undegraded":"2024-07-17T08:36:35.736105+0000","last_fullsized":"2024-07-17T08:36:35.736105+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743307+0000","last_change":"2024-07-17T08:36:34.762630+0000","last_active":"2024-07-17T08:36:35.743307+0000","last_peered":"2024-07-17T08:36:35.743307+0000","last_clean":"2024-07-17T08:36:35.743307+0000","last_became_active":"2024-07-17T08:36:34.762294+0000","last_became_peered":"2024-07-17T08:36:34.762294+0000","last_unstale":"2024-07-17T08:36:35.743307+0000","last_undegraded":"2024-07-17T08:36:35.743307+0000","last_fullsized":"2024-07-17T08:36:35.743307+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:35.702183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743816+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:35.743816+0000","last_peered":"2024-07-17T08:36:35.743816+0000","last_clean":"2024-07-17T08:36:35.743816+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:35.743816+0000","last_undegraded":"2024-07-17T08:36:35.743816+0000","last_fullsized":"2024-07-17T08:36:35.743816+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735336+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:35.735336+0000","last_peered":"2024-07-17T08:36:35.735336+0000","last_clean":"2024-07-17T08:36:35.735336+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:35.735336+0000","last_undegraded":"2024-07-17T08:36:35.735336+0000","last_fullsized":"2024-07-17T08:36:35.735336+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736222+0000","last_change":"2024-07-17T08:36:34.766601+0000","last_active":"2024-07-17T08:36:35.736222+0000","last_peered":"2024-07-17T08:36:35.736222+0000","last_clean":"2024-07-17T08:36:35.736222+0000","last_became_active":"2024-07-17T08:36:34.766306+0000","last_became_peered":"2024-07-17T08:36:34.766306+0000","last_unstale":"2024-07-17T08:36:35.736222+0000","last_undegraded":"2024-07-17T08:36:35.736222+0000","last_fullsized":"2024-07-17T08:36:35.736222+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:00.924743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738018+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:35.738018+0000","last_peered":"2024-07-17T08:36:35.738018+0000","last_clean":"2024-07-17T08:36:35.738018+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:35.738018+0000","last_undegraded":"2024-07-17T08:36:35.738018+0000","last_fullsized":"2024-07-17T08:36:35.738018+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730519+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:35.730519+0000","last_peered":"2024-07-17T08:36:35.730519+0000","last_clean":"2024-07-17T08:36:35.730519+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:35.730519+0000","last_undegraded":"2024-07-17T08:36:35.730519+0000","last_fullsized":"2024-07-17T08:36:35.730519+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405657+0000","last_change":"2024-07-17T08:36:34.762727+0000","last_active":"2024-07-17T08:36:36.405657+0000","last_peered":"2024-07-17T08:36:36.405657+0000","last_clean":"2024-07-17T08:36:36.405657+0000","last_became_active":"2024-07-17T08:36:34.762396+0000","last_became_peered":"2024-07-17T08:36:34.762396+0000","last_unstale":"2024-07-17T08:36:36.405657+0000","last_undegraded":"2024-07-17T08:36:36.405657+0000","last_fullsized":"2024-07-17T08:36:36.405657+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:10.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735410+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:35.735410+0000","last_peered":"2024-07-17T08:36:35.735410+0000","last_clean":"2024-07-17T08:36:35.735410+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:35.735410+0000","last_undegraded":"2024-07-17T08:36:35.735410+0000","last_fullsized":"2024-07-17T08:36:35.735410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560433+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:36.560433+0000","last_peered":"2024-07-17T08:36:36.560433+0000","last_clean":"2024-07-17T08:36:36.560433+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:36.560433+0000","last_undegraded":"2024-07-17T08:36:36.560433+0000","last_fullsized":"2024-07-17T08:36:36.560433+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558118+0000","last_change":"2024-07-17T08:36:34.757902+0000","last_active":"2024-07-17T08:36:36.558118+0000","last_peered":"2024-07-17T08:36:36.558118+0000","last_clean":"2024-07-17T08:36:36.558118+0000","last_became_active":"2024-07-17T08:36:34.756374+0000","last_became_peered":"2024-07-17T08:36:34.756374+0000","last_unstale":"2024-07-17T08:36:36.558118+0000","last_undegraded":"2024-07-17T08:36:36.558118+0000","last_fullsized":"2024-07-17T08:36:36.558118+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:04:56.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734799+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:35.734799+0000","last_peered":"2024-07-17T08:36:35.734799+0000","last_clean":"2024-07-17T08:36:35.734799+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:35.734799+0000","last_undegraded":"2024-07-17T08:36:35.734799+0000","last_fullsized":"2024-07-17T08:36:35.734799+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.733076+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:35.733076+0000","last_peered":"2024-07-17T08:36:35.733076+0000","last_clean":"2024-07-17T08:36:35.733076+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:35.733076+0000","last_undegraded":"2024-07-17T08:36:35.733076+0000","last_fullsized":"2024-07-17T08:36:35.733076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735096+0000","last_change":"2024-07-17T08:36:34.759035+0000","last_active":"2024-07-17T08:36:35.735096+0000","last_peered":"2024-07-17T08:36:35.735096+0000","last_clean":"2024-07-17T08:36:35.735096+0000","last_became_active":"2024-07-17T08:36:34.758729+0000","last_became_peered":"2024-07-17T08:36:34.758729+0000","last_unstale":"2024-07-17T08:36:35.735096+0000","last_undegraded":"2024-07-17T08:36:35.735096+0000","last_fullsized":"2024-07-17T08:36:35.735096+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:09.694236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730514+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:35.730514+0000","last_peered":"2024-07-17T08:36:35.730514+0000","last_clean":"2024-07-17T08:36:35.730514+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:35.730514+0000","last_undegraded":"2024-07-17T08:36:35.730514+0000","last_fullsized":"2024-07-17T08:36:35.730514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561013+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:36.561013+0000","last_peered":"2024-07-17T08:36:36.561013+0000","last_clean":"2024-07-17T08:36:36.561013+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:36.561013+0000","last_undegraded":"2024-07-17T08:36:36.561013+0000","last_fullsized":"2024-07-17T08:36:36.561013+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404661+0000","last_change":"2024-07-17T08:36:34.767357+0000","last_active":"2024-07-17T08:36:36.404661+0000","last_peered":"2024-07-17T08:36:36.404661+0000","last_clean":"2024-07-17T08:36:36.404661+0000","last_became_active":"2024-07-17T08:36:34.766645+0000","last_became_peered":"2024-07-17T08:36:34.766645+0000","last_unstale":"2024-07-17T08:36:36.404661+0000","last_undegraded":"2024-07-17T08:36:36.404661+0000","last_fullsized":"2024-07-17T08:36:36.404661+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:40.714568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735996+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:35.735996+0000","last_peered":"2024-07-17T08:36:35.735996+0000","last_clean":"2024-07-17T08:36:35.735996+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:35.735996+0000","last_undegraded":"2024-07-17T08:36:35.735996+0000","last_fullsized":"2024-07-17T08:36:35.735996+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743773+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:35.743773+0000","last_peered":"2024-07-17T08:36:35.743773+0000","last_clean":"2024-07-17T08:36:35.743773+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:35.743773+0000","last_undegraded":"2024-07-17T08:36:35.743773+0000","last_fullsized":"2024-07-17T08:36:35.743773+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738001+0000","last_change":"2024-07-17T08:36:34.751102+0000","last_active":"2024-07-17T08:36:35.738001+0000","last_peered":"2024-07-17T08:36:35.738001+0000","last_clean":"2024-07-17T08:36:35.738001+0000","last_became_active":"2024-07-17T08:36:34.750817+0000","last_became_peered":"2024-07-17T08:36:34.750817+0000","last_unstale":"2024-07-17T08:36:35.738001+0000","last_undegraded":"2024-07-17T08:36:35.738001+0000","last_fullsized":"2024-07-17T08:36:35.738001+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:07.007824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735924+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:35.735924+0000","last_peered":"2024-07-17T08:36:35.735924+0000","last_clean":"2024-07-17T08:36:35.735924+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:35.735924+0000","last_undegraded":"2024-07-17T08:36:35.735924+0000","last_fullsized":"2024-07-17T08:36:35.735924+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736034+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:35.736034+0000","last_peered":"2024-07-17T08:36:35.736034+0000","last_clean":"2024-07-17T08:36:35.736034+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:35.736034+0000","last_undegraded":"2024-07-17T08:36:35.736034+0000","last_fullsized":"2024-07-17T08:36:35.736034+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743869+0000","last_change":"2024-07-17T08:36:34.762388+0000","last_active":"2024-07-17T08:36:35.743869+0000","last_peered":"2024-07-17T08:36:35.743869+0000","last_clean":"2024-07-17T08:36:35.743869+0000","last_became_active":"2024-07-17T08:36:34.762062+0000","last_became_peered":"2024-07-17T08:36:34.762062+0000","last_unstale":"2024-07-17T08:36:35.743869+0000","last_undegraded":"2024-07-17T08:36:35.743869+0000","last_fullsized":"2024-07-17T08:36:35.743869+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:31.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731483+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:35.731483+0000","last_peered":"2024-07-17T08:36:35.731483+0000","last_clean":"2024-07-17T08:36:35.731483+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:35.731483+0000","last_undegraded":"2024-07-17T08:36:35.731483+0000","last_fullsized":"2024-07-17T08:36:35.731483+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559699+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:36.559699+0000","last_peered":"2024-07-17T08:36:36.559699+0000","last_clean":"2024-07-17T08:36:36.559699+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:36.559699+0000","last_undegraded":"2024-07-17T08:36:36.559699+0000","last_fullsized":"2024-07-17T08:36:36.559699+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743167+0000","last_change":"2024-07-17T08:36:34.758037+0000","last_active":"2024-07-17T08:36:35.743167+0000","last_peered":"2024-07-17T08:36:35.743167+0000","last_clean":"2024-07-17T08:36:35.743167+0000","last_became_active":"2024-07-17T08:36:34.757760+0000","last_became_peered":"2024-07-17T08:36:34.757760+0000","last_unstale":"2024-07-17T08:36:35.743167+0000","last_undegraded":"2024-07-17T08:36:35.743167+0000","last_fullsized":"2024-07-17T08:36:35.743167+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:56.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560149+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:36.560149+0000","last_peered":"2024-07-17T08:36:36.560149+0000","last_clean":"2024-07-17T08:36:36.560149+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:36.560149+0000","last_undegraded":"2024-07-17T08:36:36.560149+0000","last_fullsized":"2024-07-17T08:36:36.560149+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558622+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:36.558622+0000","last_peered":"2024-07-17T08:36:36.558622+0000","last_clean":"2024-07-17T08:36:36.558622+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:36.558622+0000","last_undegraded":"2024-07-17T08:36:36.558622+0000","last_fullsized":"2024-07-17T08:36:36.558622+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559072+0000","last_change":"2024-07-17T08:36:34.769778+0000","last_active":"2024-07-17T08:36:36.559072+0000","last_peered":"2024-07-17T08:36:36.559072+0000","last_clean":"2024-07-17T08:36:36.559072+0000","last_became_active":"2024-07-17T08:36:34.767090+0000","last_became_peered":"2024-07-17T08:36:34.767090+0000","last_unstale":"2024-07-17T08:36:36.559072+0000","last_undegraded":"2024-07-17T08:36:36.559072+0000","last_fullsized":"2024-07-17T08:36:36.559072+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:34.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561823+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:36.561823+0000","last_peered":"2024-07-17T08:36:36.561823+0000","last_clean":"2024-07-17T08:36:36.561823+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:36.561823+0000","last_undegraded":"2024-07-17T08:36:36.561823+0000","last_fullsized":"2024-07-17T08:36:36.561823+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405097+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:36.405097+0000","last_peered":"2024-07-17T08:36:36.405097+0000","last_clean":"2024-07-17T08:36:36.405097+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:36.405097+0000","last_undegraded":"2024-07-17T08:36:36.405097+0000","last_fullsized":"2024-07-17T08:36:36.405097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741921+0000","last_change":"2024-07-17T08:36:34.764621+0000","last_active":"2024-07-17T08:36:35.741921+0000","last_peered":"2024-07-17T08:36:35.741921+0000","last_clean":"2024-07-17T08:36:35.741921+0000","last_became_active":"2024-07-17T08:36:34.764293+0000","last_became_peered":"2024-07-17T08:36:34.764293+0000","last_unstale":"2024-07-17T08:36:35.741921+0000","last_undegraded":"2024-07-17T08:36:35.741921+0000","last_fullsized":"2024-07-17T08:36:35.741921+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:32.986386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558641+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:36.558641+0000","last_peered":"2024-07-17T08:36:36.558641+0000","last_clean":"2024-07-17T08:36:36.558641+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:36.558641+0000","last_undegraded":"2024-07-17T08:36:36.558641+0000","last_fullsized":"2024-07-17T08:36:36.558641+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736688+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:35.736688+0000","last_peered":"2024-07-17T08:36:35.736688+0000","last_clean":"2024-07-17T08:36:35.736688+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:35.736688+0000","last_undegraded":"2024-07-17T08:36:35.736688+0000","last_fullsized":"2024-07-17T08:36:35.736688+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735803+0000","last_change":"2024-07-17T08:36:34.749678+0000","last_active":"2024-07-17T08:36:35.735803+0000","last_peered":"2024-07-17T08:36:35.735803+0000","last_clean":"2024-07-17T08:36:35.735803+0000","last_became_active":"2024-07-17T08:36:34.749045+0000","last_became_peered":"2024-07-17T08:36:34.749045+0000","last_unstale":"2024-07-17T08:36:35.735803+0000","last_undegraded":"2024-07-17T08:36:35.735803+0000","last_fullsized":"2024-07-17T08:36:35.735803+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:58.763986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741764+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:35.741764+0000","last_peered":"2024-07-17T08:36:35.741764+0000","last_clean":"2024-07-17T08:36:35.741764+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:35.741764+0000","last_undegraded":"2024-07-17T08:36:35.741764+0000","last_fullsized":"2024-07-17T08:36:35.741764+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738256+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:35.738256+0000","last_peered":"2024-07-17T08:36:35.738256+0000","last_clean":"2024-07-17T08:36:35.738256+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:35.738256+0000","last_undegraded":"2024-07-17T08:36:35.738256+0000","last_fullsized":"2024-07-17T08:36:35.738256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557817+0000","last_change":"2024-07-17T08:36:34.739794+0000","last_active":"2024-07-17T08:36:36.557817+0000","last_peered":"2024-07-17T08:36:36.557817+0000","last_clean":"2024-07-17T08:36:36.557817+0000","last_became_active":"2024-07-17T08:36:34.739282+0000","last_became_peered":"2024-07-17T08:36:34.739282+0000","last_unstale":"2024-07-17T08:36:36.557817+0000","last_undegraded":"2024-07-17T08:36:36.557817+0000","last_fullsized":"2024-07-17T08:36:36.557817+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:18.364494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735138+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:35.735138+0000","last_peered":"2024-07-17T08:36:35.735138+0000","last_clean":"2024-07-17T08:36:35.735138+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:35.735138+0000","last_undegraded":"2024-07-17T08:36:35.735138+0000","last_fullsized":"2024-07-17T08:36:35.735138+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406502+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:36.406502+0000","last_peered":"2024-07-17T08:36:36.406502+0000","last_clean":"2024-07-17T08:36:36.406502+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:36.406502+0000","last_undegraded":"2024-07-17T08:36:36.406502+0000","last_fullsized":"2024-07-17T08:36:36.406502+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559322+0000","last_change":"2024-07-17T08:36:34.757365+0000","last_active":"2024-07-17T08:36:36.559322+0000","last_peered":"2024-07-17T08:36:36.559322+0000","last_clean":"2024-07-17T08:36:36.559322+0000","last_became_active":"2024-07-17T08:36:34.757077+0000","last_became_peered":"2024-07-17T08:36:34.757077+0000","last_unstale":"2024-07-17T08:36:36.559322+0000","last_undegraded":"2024-07-17T08:36:36.559322+0000","last_fullsized":"2024-07-17T08:36:36.559322+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:20.498778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736547+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:35.736547+0000","last_peered":"2024-07-17T08:36:35.736547+0000","last_clean":"2024-07-17T08:36:35.736547+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:35.736547+0000","last_undegraded":"2024-07-17T08:36:35.736547+0000","last_fullsized":"2024-07-17T08:36:35.736547+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406872+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:36.406872+0000","last_peered":"2024-07-17T08:36:36.406872+0000","last_clean":"2024-07-17T08:36:36.406872+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:36.406872+0000","last_undegraded":"2024-07-17T08:36:36.406872+0000","last_fullsized":"2024-07-17T08:36:36.406872+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406507+0000","last_change":"2024-07-17T08:36:34.772879+0000","last_active":"2024-07-17T08:36:36.406507+0000","last_peered":"2024-07-17T08:36:36.406507+0000","last_clean":"2024-07-17T08:36:36.406507+0000","last_became_active":"2024-07-17T08:36:34.772086+0000","last_became_peered":"2024-07-17T08:36:34.772086+0000","last_unstale":"2024-07-17T08:36:36.406507+0000","last_undegraded":"2024-07-17T08:36:36.406507+0000","last_fullsized":"2024-07-17T08:36:36.406507+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:17.960713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405403+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:36.405403+0000","last_peered":"2024-07-17T08:36:36.405403+0000","last_clean":"2024-07-17T08:36:36.405403+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:36.405403+0000","last_undegraded":"2024-07-17T08:36:36.405403+0000","last_fullsized":"2024-07-17T08:36:36.405403+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560972+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:36.560972+0000","last_peered":"2024-07-17T08:36:36.560972+0000","last_clean":"2024-07-17T08:36:36.560972+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:36.560972+0000","last_undegraded":"2024-07-17T08:36:36.560972+0000","last_fullsized":"2024-07-17T08:36:36.560972+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561083+0000","last_change":"2024-07-17T08:36:34.772407+0000","last_active":"2024-07-17T08:36:36.561083+0000","last_peered":"2024-07-17T08:36:36.561083+0000","last_clean":"2024-07-17T08:36:36.561083+0000","last_became_active":"2024-07-17T08:36:34.772076+0000","last_became_peered":"2024-07-17T08:36:34.772076+0000","last_unstale":"2024-07-17T08:36:36.561083+0000","last_undegraded":"2024-07-17T08:36:36.561083+0000","last_fullsized":"2024-07-17T08:36:36.561083+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:29.825386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559753+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:36.559753+0000","last_peered":"2024-07-17T08:36:36.559753+0000","last_clean":"2024-07-17T08:36:36.559753+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:36.559753+0000","last_undegraded":"2024-07-17T08:36:36.559753+0000","last_fullsized":"2024-07-17T08:36:36.559753+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736187+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:35.736187+0000","last_peered":"2024-07-17T08:36:35.736187+0000","last_clean":"2024-07-17T08:36:35.736187+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:35.736187+0000","last_undegraded":"2024-07-17T08:36:35.736187+0000","last_fullsized":"2024-07-17T08:36:35.736187+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558755+0000","last_change":"2024-07-17T08:36:34.753399+0000","last_active":"2024-07-17T08:36:36.558755+0000","last_peered":"2024-07-17T08:36:36.558755+0000","last_clean":"2024-07-17T08:36:36.558755+0000","last_became_active":"2024-07-17T08:36:34.753124+0000","last_became_peered":"2024-07-17T08:36:34.753124+0000","last_unstale":"2024-07-17T08:36:36.558755+0000","last_undegraded":"2024-07-17T08:36:36.558755+0000","last_fullsized":"2024-07-17T08:36:36.558755+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:19.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732588+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:35.732588+0000","last_peered":"2024-07-17T08:36:35.732588+0000","last_clean":"2024-07-17T08:36:35.732588+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:35.732588+0000","last_undegraded":"2024-07-17T08:36:35.732588+0000","last_fullsized":"2024-07-17T08:36:35.732588+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734906+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:35.734906+0000","last_peered":"2024-07-17T08:36:35.734906+0000","last_clean":"2024-07-17T08:36:35.734906+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:35.734906+0000","last_undegraded":"2024-07-17T08:36:35.734906+0000","last_fullsized":"2024-07-17T08:36:35.734906+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560794+0000","last_change":"2024-07-17T08:36:34.774271+0000","last_active":"2024-07-17T08:36:36.560794+0000","last_peered":"2024-07-17T08:36:36.560794+0000","last_clean":"2024-07-17T08:36:36.560794+0000","last_became_active":"2024-07-17T08:36:34.773736+0000","last_became_peered":"2024-07-17T08:36:34.773736+0000","last_unstale":"2024-07-17T08:36:36.560794+0000","last_undegraded":"2024-07-17T08:36:36.560794+0000","last_fullsized":"2024-07-17T08:36:36.560794+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:03.362787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561767+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:36.561767+0000","last_peered":"2024-07-17T08:36:36.561767+0000","last_clean":"2024-07-17T08:36:36.561767+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:36.561767+0000","last_undegraded":"2024-07-17T08:36:36.561767+0000","last_fullsized":"2024-07-17T08:36:36.561767+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558115+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:36.558115+0000","last_peered":"2024-07-17T08:36:36.558115+0000","last_clean":"2024-07-17T08:36:36.558115+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:36.558115+0000","last_undegraded":"2024-07-17T08:36:36.558115+0000","last_fullsized":"2024-07-17T08:36:36.558115+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558684+0000","last_change":"2024-07-17T08:36:34.758236+0000","last_active":"2024-07-17T08:36:36.558684+0000","last_peered":"2024-07-17T08:36:36.558684+0000","last_clean":"2024-07-17T08:36:36.558684+0000","last_became_active":"2024-07-17T08:36:34.756897+0000","last_became_peered":"2024-07-17T08:36:34.756897+0000","last_unstale":"2024-07-17T08:36:36.558684+0000","last_undegraded":"2024-07-17T08:36:36.558684+0000","last_fullsized":"2024-07-17T08:36:36.558684+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.720969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558229+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:36.558229+0000","last_peered":"2024-07-17T08:36:36.558229+0000","last_clean":"2024-07-17T08:36:36.558229+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:36.558229+0000","last_undegraded":"2024-07-17T08:36:36.558229+0000","last_fullsized":"2024-07-17T08:36:36.558229+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561312+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:36.561312+0000","last_peered":"2024-07-17T08:36:36.561312+0000","last_clean":"2024-07-17T08:36:36.561312+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:36.561312+0000","last_undegraded":"2024-07-17T08:36:36.561312+0000","last_fullsized":"2024-07-17T08:36:36.561312+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3475,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741587+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:35.741587+0000","last_peered":"2024-07-17T08:36:35.741587+0000","last_clean":"2024-07-17T08:36:35.741587+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:35.741587+0000","last_undegraded":"2024-07-17T08:36:35.741587+0000","last_fullsized":"2024-07-17T08:36:35.741587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543356,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48704,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323136,"statfs":{"total":96636764160,"available":96586891264,"internally_reserved":0,"allocated":3723264,"data_stored":2296735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510651,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48124,"kb_used_data":3052,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323716,"statfs":{"total":96636764160,"available":96587485184,"internally_reserved":0,"allocated":3125248,"data_stored":1705785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510652,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52276,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319564,"statfs":{"total":96636764160,"available":96583233536,"internally_reserved":0,"allocated":3117056,"data_stored":1705709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543357,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52284,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3129344,"data_stored":1707321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510651,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52296,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319544,"statfs":{"total":96636764160,"available":96583213056,"internally_reserved":0,"allocated":3141632,"data_stored":1709225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543355,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52300,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319540,"statfs":{"total":96636764160,"available":96583208960,"internally_reserved":0,"allocated":3141632,"data_stored":1708507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510652,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48124,"kb_used_data":3048,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323716,"statfs":{"total":96636764160,"available":96587485184,"internally_reserved":0,"allocated":3121152,"data_stored":1705763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510651,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57024,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314816,"statfs":{"total":96636764160,"available":96578371584,"internally_reserved":0,"allocated":3723264,"data_stored":2298935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:44.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 -- 172.21.15.44:0/2311375009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c2406e090 msgr2=0x7f7c2406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 --2- 172.21.15.44:0/2311375009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c2406e090 0x7f7c2406e4a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f7c241576d0 tx=0x7f7c1c02bbe0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 -- 172.21.15.44:0/2311375009 shutdown_connections 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 --2- 172.21.15.44:0/2311375009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c2406e090 0x7f7c2406e4a0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 --2- 172.21.15.44:0/2311375009 >> v2:172.21.15.175:3300/0 conn(0x7f7c240ac3e0 0x7f7c240ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 -- 172.21.15.44:0/2311375009 >> 172.21.15.44:0/2311375009 conn(0x7f7c240b0fb0 msgr2=0x7f7c240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:44.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 -- 172.21.15.44:0/2311375009 shutdown_connections 2024-07-17T08:36:44.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.665+0000 7f7c28c00640 1 -- 172.21.15.44:0/2311375009 wait complete. 2024-07-17T08:36:44.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.667+0000 7f7c28c00640 1 Processor -- start 2024-07-17T08:36:44.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.667+0000 7f7c28c00640 1 -- start start 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 0x7f7c241f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2406f710 0x7f7c240b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24159c80 con 0x7f7c240ac3e0 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24159b00 con 0x7f7c2406f710 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.668+0000 7f7c28c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c24159e00 con 0x7f7c2406e090 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:44.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54234/0 (socket says 172.21.15.44:54234) 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 -- 172.21.15.44:0/1836873539 learned_addr learned my addr 172.21.15.44:0/1836873539 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c21e00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 0x7f7c241f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 -- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 msgr2=0x7f7c241f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 0x7f7c241f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 -- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2406f710 msgr2=0x7f7c240b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2406f710 0x7f7c240b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 -- 172.21.15.44:0/1836873539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2415c0c0 con 0x7f7c240ac3e0 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c21e00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 0x7f7c241f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:44.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.669+0000 7f7c22800640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00adb0 tx=0x7f7c1c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.670+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c041020 con 0x7f7c240ac3e0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.670+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c1c03d440 con 0x7f7c240ac3e0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.670+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c03d870 con 0x7f7c240ac3e0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.670+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2415c360 con 0x7f7c240ac3e0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.670+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241f7a40 con 0x7f7c240ac3e0 2024-07-17T08:36:44.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.671+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c24050670 con 0x7f7c240ac3e0 2024-07-17T08:36:44.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.673+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f7c1c044020 con 0x7f7c240ac3e0 2024-07-17T08:36:44.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.674+0000 7f7c1a000640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 0x7f7c040792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:44.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.674+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f7c1c0ba910 con 0x7f7c240ac3e0 2024-07-17T08:36:44.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.674+0000 7f7c21e00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 0x7f7c040792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:44.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.675+0000 7f7c21e00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 0x7f7c040792b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c240cfdb0 tx=0x7f7c14009040 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:44.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.679+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7c24050670 con 0x7f7c240ac3e0 2024-07-17T08:36:44.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.889+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c240ac870 con 0x7f7c04076df0 2024-07-17T08:36:44.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.912+0000 7f7c1a000640 1 -- 172.21.15.44:0/1836873539 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+491486 (secure 0 0 0) 0x7f7c240ac870 con 0x7f7c04076df0 2024-07-17T08:36:44.921 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:44.924 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 msgr2=0x7f7c040792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 0x7f7c040792b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c240cfdb0 tx=0x7f7c14009040 comp rx=0 tx=0).stop 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 msgr2=0x7f7c240b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00adb0 tx=0x7f7c1c004250 comp rx=0 tx=0).stop 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 shutdown_connections 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f7c04076df0 0x7f7c040792b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c240ac3e0 0x7f7c240b27f0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2406f710 0x7f7c240b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 --2- 172.21.15.44:0/1836873539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2406e090 0x7f7c241f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.919+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 >> 172.21.15.44:0/1836873539 conn(0x7f7c240b0fb0 msgr2=0x7f7c2415b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.920+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 shutdown_connections 2024-07-17T08:36:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:44.920+0000 7f7c28c00640 1 -- 172.21.15.44:0/1836873539 wait complete. 2024-07-17T08:36:44.943 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-07-17T08:36:43.289766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596434,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":475,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2605,"ondisk_log_size":2605,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":411132,"kb_used_data":25608,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754563588,"statfs":{"total":773094113280,"available":772673114112,"internally_reserved":0,"allocated":26222592,"data_stored":14837980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.562006"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731112+0000","last_change":"2024-07-17T08:36:34.769780+0000","last_active":"2024-07-17T08:36:35.731112+0000","last_peered":"2024-07-17T08:36:35.731112+0000","last_clean":"2024-07-17T08:36:35.731112+0000","last_became_active":"2024-07-17T08:36:34.769495+0000","last_became_peered":"2024-07-17T08:36:34.769495+0000","last_unstale":"2024-07-17T08:36:35.731112+0000","last_undegraded":"2024-07-17T08:36:35.731112+0000","last_fullsized":"2024-07-17T08:36:35.731112+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:30:35.751750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737302+0000","last_change":"2024-07-17T08:36:33.750014+0000","last_active":"2024-07-17T08:36:35.737302+0000","last_peered":"2024-07-17T08:36:35.737302+0000","last_clean":"2024-07-17T08:36:35.737302+0000","last_became_active":"2024-07-17T08:36:33.749632+0000","last_became_peered":"2024-07-17T08:36:33.749632+0000","last_unstale":"2024-07-17T08:36:35.737302+0000","last_undegraded":"2024-07-17T08:36:35.737302+0000","last_fullsized":"2024-07-17T08:36:35.737302+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:47.188026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560241+0000","last_change":"2024-07-17T08:36:22.636006+0000","last_active":"2024-07-17T08:36:36.560241+0000","last_peered":"2024-07-17T08:36:36.560241+0000","last_clean":"2024-07-17T08:36:36.560241+0000","last_became_active":"2024-07-17T08:36:22.635711+0000","last_became_peered":"2024-07-17T08:36:22.635711+0000","last_unstale":"2024-07-17T08:36:36.560241+0000","last_undegraded":"2024-07-17T08:36:36.560241+0000","last_fullsized":"2024-07-17T08:36:36.560241+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:00.626551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736980+0000","last_change":"2024-07-17T08:36:34.763237+0000","last_active":"2024-07-17T08:36:35.736980+0000","last_peered":"2024-07-17T08:36:35.736980+0000","last_clean":"2024-07-17T08:36:35.736980+0000","last_became_active":"2024-07-17T08:36:34.756313+0000","last_became_peered":"2024-07-17T08:36:34.756313+0000","last_unstale":"2024-07-17T08:36:35.736980+0000","last_undegraded":"2024-07-17T08:36:35.736980+0000","last_fullsized":"2024-07-17T08:36:35.736980+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:16.188290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"246'4","reported_seq":18,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731482+0000","last_change":"2024-07-17T08:36:33.758375+0000","last_active":"2024-07-17T08:36:35.731482+0000","last_peered":"2024-07-17T08:36:35.731482+0000","last_clean":"2024-07-17T08:36:35.731482+0000","last_became_active":"2024-07-17T08:36:33.758093+0000","last_became_peered":"2024-07-17T08:36:33.758093+0000","last_unstale":"2024-07-17T08:36:35.731482+0000","last_undegraded":"2024-07-17T08:36:35.731482+0000","last_fullsized":"2024-07-17T08:36:35.731482+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:15:20.948787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732496+0000","last_change":"2024-07-17T08:36:22.632573+0000","last_active":"2024-07-17T08:36:35.732496+0000","last_peered":"2024-07-17T08:36:35.732496+0000","last_clean":"2024-07-17T08:36:35.732496+0000","last_became_active":"2024-07-17T08:36:22.629998+0000","last_became_peered":"2024-07-17T08:36:22.629998+0000","last_unstale":"2024-07-17T08:36:35.732496+0000","last_undegraded":"2024-07-17T08:36:35.732496+0000","last_fullsized":"2024-07-17T08:36:35.732496+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:18:29.663419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406884+0000","last_change":"2024-07-17T08:36:34.766332+0000","last_active":"2024-07-17T08:36:36.406884+0000","last_peered":"2024-07-17T08:36:36.406884+0000","last_clean":"2024-07-17T08:36:36.406884+0000","last_became_active":"2024-07-17T08:36:34.763769+0000","last_became_peered":"2024-07-17T08:36:34.763769+0000","last_unstale":"2024-07-17T08:36:36.406884+0000","last_undegraded":"2024-07-17T08:36:36.406884+0000","last_fullsized":"2024-07-17T08:36:36.406884+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:56.756485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736610+0000","last_change":"2024-07-17T08:36:33.746302+0000","last_active":"2024-07-17T08:36:35.736610+0000","last_peered":"2024-07-17T08:36:35.736610+0000","last_clean":"2024-07-17T08:36:35.736610+0000","last_became_active":"2024-07-17T08:36:33.745779+0000","last_became_peered":"2024-07-17T08:36:33.745779+0000","last_unstale":"2024-07-17T08:36:35.736610+0000","last_undegraded":"2024-07-17T08:36:35.736610+0000","last_fullsized":"2024-07-17T08:36:35.736610+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:21:04.851926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736131+0000","last_change":"2024-07-17T08:36:22.634280+0000","last_active":"2024-07-17T08:36:35.736131+0000","last_peered":"2024-07-17T08:36:35.736131+0000","last_clean":"2024-07-17T08:36:35.736131+0000","last_became_active":"2024-07-17T08:36:22.633985+0000","last_became_peered":"2024-07-17T08:36:22.633985+0000","last_unstale":"2024-07-17T08:36:35.736131+0000","last_undegraded":"2024-07-17T08:36:35.736131+0000","last_fullsized":"2024-07-17T08:36:35.736131+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:10.166288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731650+0000","last_change":"2024-07-17T08:36:34.737756+0000","last_active":"2024-07-17T08:36:35.731650+0000","last_peered":"2024-07-17T08:36:35.731650+0000","last_clean":"2024-07-17T08:36:35.731650+0000","last_became_active":"2024-07-17T08:36:34.737369+0000","last_became_peered":"2024-07-17T08:36:34.737369+0000","last_unstale":"2024-07-17T08:36:35.731650+0000","last_undegraded":"2024-07-17T08:36:35.731650+0000","last_fullsized":"2024-07-17T08:36:35.731650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:30.548489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560259+0000","last_change":"2024-07-17T08:36:33.755800+0000","last_active":"2024-07-17T08:36:36.560259+0000","last_peered":"2024-07-17T08:36:36.560259+0000","last_clean":"2024-07-17T08:36:36.560259+0000","last_became_active":"2024-07-17T08:36:33.754310+0000","last_became_peered":"2024-07-17T08:36:33.754310+0000","last_unstale":"2024-07-17T08:36:36.560259+0000","last_undegraded":"2024-07-17T08:36:36.560259+0000","last_fullsized":"2024-07-17T08:36:36.560259+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736234+0000","last_change":"2024-07-17T08:36:22.649499+0000","last_active":"2024-07-17T08:36:35.736234+0000","last_peered":"2024-07-17T08:36:35.736234+0000","last_clean":"2024-07-17T08:36:35.736234+0000","last_became_active":"2024-07-17T08:36:22.641857+0000","last_became_peered":"2024-07-17T08:36:22.641857+0000","last_unstale":"2024-07-17T08:36:35.736234+0000","last_undegraded":"2024-07-17T08:36:35.736234+0000","last_fullsized":"2024-07-17T08:36:35.736234+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.352575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.767946+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.767625+0000","last_became_peered":"2024-07-17T08:36:34.767625+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:12.760846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:33.747145+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:33.746842+0000","last_became_peered":"2024-07-17T08:36:33.746842+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:31.490051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737281+0000","last_change":"2024-07-17T08:36:22.627540+0000","last_active":"2024-07-17T08:36:35.737281+0000","last_peered":"2024-07-17T08:36:35.737281+0000","last_clean":"2024-07-17T08:36:35.737281+0000","last_became_active":"2024-07-17T08:36:22.625232+0000","last_became_peered":"2024-07-17T08:36:22.625232+0000","last_unstale":"2024-07-17T08:36:35.737281+0000","last_undegraded":"2024-07-17T08:36:35.737281+0000","last_fullsized":"2024-07-17T08:36:35.737281+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:59.182629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404806+0000","last_change":"2024-07-17T08:36:34.770512+0000","last_active":"2024-07-17T08:36:36.404806+0000","last_peered":"2024-07-17T08:36:36.404806+0000","last_clean":"2024-07-17T08:36:36.404806+0000","last_became_active":"2024-07-17T08:36:34.770193+0000","last_became_peered":"2024-07-17T08:36:34.770193+0000","last_unstale":"2024-07-17T08:36:36.404806+0000","last_undegraded":"2024-07-17T08:36:36.404806+0000","last_fullsized":"2024-07-17T08:36:36.404806+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:56.938896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735822+0000","last_change":"2024-07-17T08:36:33.755670+0000","last_active":"2024-07-17T08:36:35.735822+0000","last_peered":"2024-07-17T08:36:35.735822+0000","last_clean":"2024-07-17T08:36:35.735822+0000","last_became_active":"2024-07-17T08:36:33.755363+0000","last_became_peered":"2024-07-17T08:36:33.755363+0000","last_unstale":"2024-07-17T08:36:35.735822+0000","last_undegraded":"2024-07-17T08:36:35.735822+0000","last_fullsized":"2024-07-17T08:36:35.735822+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:41.505196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558689+0000","last_change":"2024-07-17T08:36:22.620680+0000","last_active":"2024-07-17T08:36:36.558689+0000","last_peered":"2024-07-17T08:36:36.558689+0000","last_clean":"2024-07-17T08:36:36.558689+0000","last_became_active":"2024-07-17T08:36:22.620404+0000","last_became_peered":"2024-07-17T08:36:22.620404+0000","last_unstale":"2024-07-17T08:36:36.558689+0000","last_undegraded":"2024-07-17T08:36:36.558689+0000","last_fullsized":"2024-07-17T08:36:36.558689+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:48.077061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558650+0000","last_change":"2024-07-17T08:36:34.757341+0000","last_active":"2024-07-17T08:36:36.558650+0000","last_peered":"2024-07-17T08:36:36.558650+0000","last_clean":"2024-07-17T08:36:36.558650+0000","last_became_active":"2024-07-17T08:36:34.755978+0000","last_became_peered":"2024-07-17T08:36:34.755978+0000","last_unstale":"2024-07-17T08:36:36.558650+0000","last_undegraded":"2024-07-17T08:36:36.558650+0000","last_fullsized":"2024-07-17T08:36:36.558650+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:04:27.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735299+0000","last_change":"2024-07-17T08:36:33.748383+0000","last_active":"2024-07-17T08:36:35.735299+0000","last_peered":"2024-07-17T08:36:35.735299+0000","last_clean":"2024-07-17T08:36:35.735299+0000","last_became_active":"2024-07-17T08:36:33.748082+0000","last_became_peered":"2024-07-17T08:36:33.748082+0000","last_unstale":"2024-07-17T08:36:35.735299+0000","last_undegraded":"2024-07-17T08:36:35.735299+0000","last_fullsized":"2024-07-17T08:36:35.735299+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:47:10.743177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560090+0000","last_change":"2024-07-17T08:36:22.630800+0000","last_active":"2024-07-17T08:36:36.560090+0000","last_peered":"2024-07-17T08:36:36.560090+0000","last_clean":"2024-07-17T08:36:36.560090+0000","last_became_active":"2024-07-17T08:36:22.628609+0000","last_became_peered":"2024-07-17T08:36:22.628609+0000","last_unstale":"2024-07-17T08:36:36.560090+0000","last_undegraded":"2024-07-17T08:36:36.560090+0000","last_fullsized":"2024-07-17T08:36:36.560090+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:04.053995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559030+0000","last_change":"2024-07-17T08:36:34.774215+0000","last_active":"2024-07-17T08:36:36.559030+0000","last_peered":"2024-07-17T08:36:36.559030+0000","last_clean":"2024-07-17T08:36:36.559030+0000","last_became_active":"2024-07-17T08:36:34.773518+0000","last_became_peered":"2024-07-17T08:36:34.773518+0000","last_unstale":"2024-07-17T08:36:36.559030+0000","last_undegraded":"2024-07-17T08:36:36.559030+0000","last_fullsized":"2024-07-17T08:36:36.559030+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:27.370346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734842+0000","last_change":"2024-07-17T08:36:33.756246+0000","last_active":"2024-07-17T08:36:35.734842+0000","last_peered":"2024-07-17T08:36:35.734842+0000","last_clean":"2024-07-17T08:36:35.734842+0000","last_became_active":"2024-07-17T08:36:33.754426+0000","last_became_peered":"2024-07-17T08:36:33.754426+0000","last_unstale":"2024-07-17T08:36:35.734842+0000","last_undegraded":"2024-07-17T08:36:35.734842+0000","last_fullsized":"2024-07-17T08:36:35.734842+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:23.283987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735514+0000","last_change":"2024-07-17T08:36:22.613734+0000","last_active":"2024-07-17T08:36:35.735514+0000","last_peered":"2024-07-17T08:36:35.735514+0000","last_clean":"2024-07-17T08:36:35.735514+0000","last_became_active":"2024-07-17T08:36:22.612544+0000","last_became_peered":"2024-07-17T08:36:22.612544+0000","last_unstale":"2024-07-17T08:36:35.735514+0000","last_undegraded":"2024-07-17T08:36:35.735514+0000","last_fullsized":"2024-07-17T08:36:35.735514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:15.903836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731147+0000","last_change":"2024-07-17T08:36:34.757574+0000","last_active":"2024-07-17T08:36:35.731147+0000","last_peered":"2024-07-17T08:36:35.731147+0000","last_clean":"2024-07-17T08:36:35.731147+0000","last_became_active":"2024-07-17T08:36:34.756063+0000","last_became_peered":"2024-07-17T08:36:34.756063+0000","last_unstale":"2024-07-17T08:36:35.731147+0000","last_undegraded":"2024-07-17T08:36:35.731147+0000","last_fullsized":"2024-07-17T08:36:35.731147+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:45.933512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737630+0000","last_change":"2024-07-17T08:36:33.754739+0000","last_active":"2024-07-17T08:36:35.737630+0000","last_peered":"2024-07-17T08:36:35.737630+0000","last_clean":"2024-07-17T08:36:35.737630+0000","last_became_active":"2024-07-17T08:36:33.754416+0000","last_became_peered":"2024-07-17T08:36:33.754416+0000","last_unstale":"2024-07-17T08:36:35.737630+0000","last_undegraded":"2024-07-17T08:36:35.737630+0000","last_fullsized":"2024-07-17T08:36:35.737630+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:27.464559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560417+0000","last_change":"2024-07-17T08:36:22.626646+0000","last_active":"2024-07-17T08:36:36.560417+0000","last_peered":"2024-07-17T08:36:36.560417+0000","last_clean":"2024-07-17T08:36:36.560417+0000","last_became_active":"2024-07-17T08:36:22.626267+0000","last_became_peered":"2024-07-17T08:36:22.626267+0000","last_unstale":"2024-07-17T08:36:36.560417+0000","last_undegraded":"2024-07-17T08:36:36.560417+0000","last_fullsized":"2024-07-17T08:36:36.560417+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:39:57.885211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559256+0000","last_change":"2024-07-17T08:36:34.766632+0000","last_active":"2024-07-17T08:36:36.559256+0000","last_peered":"2024-07-17T08:36:36.559256+0000","last_clean":"2024-07-17T08:36:36.559256+0000","last_became_active":"2024-07-17T08:36:34.766314+0000","last_became_peered":"2024-07-17T08:36:34.766314+0000","last_unstale":"2024-07-17T08:36:36.559256+0000","last_undegraded":"2024-07-17T08:36:36.559256+0000","last_fullsized":"2024-07-17T08:36:36.559256+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:40:20.443444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559692+0000","last_change":"2024-07-17T08:36:33.754818+0000","last_active":"2024-07-17T08:36:36.559692+0000","last_peered":"2024-07-17T08:36:36.559692+0000","last_clean":"2024-07-17T08:36:36.559692+0000","last_became_active":"2024-07-17T08:36:33.753426+0000","last_became_peered":"2024-07-17T08:36:33.753426+0000","last_unstale":"2024-07-17T08:36:36.559692+0000","last_undegraded":"2024-07-17T08:36:36.559692+0000","last_fullsized":"2024-07-17T08:36:36.559692+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:46:47.218187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736433+0000","last_change":"2024-07-17T08:36:22.636371+0000","last_active":"2024-07-17T08:36:35.736433+0000","last_peered":"2024-07-17T08:36:35.736433+0000","last_clean":"2024-07-17T08:36:35.736433+0000","last_became_active":"2024-07-17T08:36:22.633111+0000","last_became_peered":"2024-07-17T08:36:22.633111+0000","last_unstale":"2024-07-17T08:36:35.736433+0000","last_undegraded":"2024-07-17T08:36:35.736433+0000","last_fullsized":"2024-07-17T08:36:35.736433+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:15:15.236051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561462+0000","last_change":"2024-07-17T08:36:34.763712+0000","last_active":"2024-07-17T08:36:36.561462+0000","last_peered":"2024-07-17T08:36:36.561462+0000","last_clean":"2024-07-17T08:36:36.561462+0000","last_became_active":"2024-07-17T08:36:34.761551+0000","last_became_peered":"2024-07-17T08:36:34.761551+0000","last_unstale":"2024-07-17T08:36:36.561462+0000","last_undegraded":"2024-07-17T08:36:36.561462+0000","last_fullsized":"2024-07-17T08:36:36.561462+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:05.846381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"246'6","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730737+0000","last_change":"2024-07-17T08:36:33.751230+0000","last_active":"2024-07-17T08:36:35.730737+0000","last_peered":"2024-07-17T08:36:35.730737+0000","last_clean":"2024-07-17T08:36:35.730737+0000","last_became_active":"2024-07-17T08:36:33.750539+0000","last_became_peered":"2024-07-17T08:36:33.750539+0000","last_unstale":"2024-07-17T08:36:35.730737+0000","last_undegraded":"2024-07-17T08:36:35.730737+0000","last_fullsized":"2024-07-17T08:36:35.730737+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:35.074061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736286+0000","last_change":"2024-07-17T08:36:22.629481+0000","last_active":"2024-07-17T08:36:35.736286+0000","last_peered":"2024-07-17T08:36:35.736286+0000","last_clean":"2024-07-17T08:36:35.736286+0000","last_became_active":"2024-07-17T08:36:22.628291+0000","last_became_peered":"2024-07-17T08:36:22.628291+0000","last_unstale":"2024-07-17T08:36:35.736286+0000","last_undegraded":"2024-07-17T08:36:35.736286+0000","last_fullsized":"2024-07-17T08:36:35.736286+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:08.398461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743570+0000","last_change":"2024-07-17T08:36:34.768425+0000","last_active":"2024-07-17T08:36:35.743570+0000","last_peered":"2024-07-17T08:36:35.743570+0000","last_clean":"2024-07-17T08:36:35.743570+0000","last_became_active":"2024-07-17T08:36:34.767292+0000","last_became_peered":"2024-07-17T08:36:34.767292+0000","last_unstale":"2024-07-17T08:36:35.743570+0000","last_undegraded":"2024-07-17T08:36:35.743570+0000","last_fullsized":"2024-07-17T08:36:35.743570+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:01:05.763645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"246'3","reported_seq":17,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559354+0000","last_change":"2024-07-17T08:36:33.754294+0000","last_active":"2024-07-17T08:36:36.559354+0000","last_peered":"2024-07-17T08:36:36.559354+0000","last_clean":"2024-07-17T08:36:36.559354+0000","last_became_active":"2024-07-17T08:36:33.753457+0000","last_became_peered":"2024-07-17T08:36:33.753457+0000","last_unstale":"2024-07-17T08:36:36.559354+0000","last_undegraded":"2024-07-17T08:36:36.559354+0000","last_fullsized":"2024-07-17T08:36:36.559354+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:47.335972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559369+0000","last_change":"2024-07-17T08:36:22.625779+0000","last_active":"2024-07-17T08:36:36.559369+0000","last_peered":"2024-07-17T08:36:36.559369+0000","last_clean":"2024-07-17T08:36:36.559369+0000","last_became_active":"2024-07-17T08:36:22.625270+0000","last_became_peered":"2024-07-17T08:36:22.625270+0000","last_unstale":"2024-07-17T08:36:36.559369+0000","last_undegraded":"2024-07-17T08:36:36.559369+0000","last_fullsized":"2024-07-17T08:36:36.559369+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:37.698802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558019+0000","last_change":"2024-07-17T08:36:34.772658+0000","last_active":"2024-07-17T08:36:36.558019+0000","last_peered":"2024-07-17T08:36:36.558019+0000","last_clean":"2024-07-17T08:36:36.558019+0000","last_became_active":"2024-07-17T08:36:34.772287+0000","last_became_peered":"2024-07-17T08:36:34.772287+0000","last_unstale":"2024-07-17T08:36:36.558019+0000","last_undegraded":"2024-07-17T08:36:36.558019+0000","last_fullsized":"2024-07-17T08:36:36.558019+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:52.350056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558126+0000","last_change":"2024-07-17T08:36:33.758000+0000","last_active":"2024-07-17T08:36:36.558126+0000","last_peered":"2024-07-17T08:36:36.558126+0000","last_clean":"2024-07-17T08:36:36.558126+0000","last_became_active":"2024-07-17T08:36:33.757029+0000","last_became_peered":"2024-07-17T08:36:33.757029+0000","last_unstale":"2024-07-17T08:36:36.558126+0000","last_undegraded":"2024-07-17T08:36:36.558126+0000","last_fullsized":"2024-07-17T08:36:36.558126+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:48.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560728+0000","last_change":"2024-07-17T08:36:22.638165+0000","last_active":"2024-07-17T08:36:36.560728+0000","last_peered":"2024-07-17T08:36:36.560728+0000","last_clean":"2024-07-17T08:36:36.560728+0000","last_became_active":"2024-07-17T08:36:22.637601+0000","last_became_peered":"2024-07-17T08:36:22.637601+0000","last_unstale":"2024-07-17T08:36:36.560728+0000","last_undegraded":"2024-07-17T08:36:36.560728+0000","last_fullsized":"2024-07-17T08:36:36.560728+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:39.742346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734977+0000","last_change":"2024-07-17T08:36:34.756176+0000","last_active":"2024-07-17T08:36:35.734977+0000","last_peered":"2024-07-17T08:36:35.734977+0000","last_clean":"2024-07-17T08:36:35.734977+0000","last_became_active":"2024-07-17T08:36:34.755839+0000","last_became_peered":"2024-07-17T08:36:34.755839+0000","last_unstale":"2024-07-17T08:36:35.734977+0000","last_undegraded":"2024-07-17T08:36:35.734977+0000","last_fullsized":"2024-07-17T08:36:35.734977+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:16:01.411712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735459+0000","last_change":"2024-07-17T08:36:33.749435+0000","last_active":"2024-07-17T08:36:35.735459+0000","last_peered":"2024-07-17T08:36:35.735459+0000","last_clean":"2024-07-17T08:36:35.735459+0000","last_became_active":"2024-07-17T08:36:33.749168+0000","last_became_peered":"2024-07-17T08:36:33.749168+0000","last_unstale":"2024-07-17T08:36:35.735459+0000","last_undegraded":"2024-07-17T08:36:35.735459+0000","last_fullsized":"2024-07-17T08:36:35.735459+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:49.720522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406087+0000","last_change":"2024-07-17T08:36:22.625803+0000","last_active":"2024-07-17T08:36:36.406087+0000","last_peered":"2024-07-17T08:36:36.406087+0000","last_clean":"2024-07-17T08:36:36.406087+0000","last_became_active":"2024-07-17T08:36:22.625241+0000","last_became_peered":"2024-07-17T08:36:22.625241+0000","last_unstale":"2024-07-17T08:36:36.406087+0000","last_undegraded":"2024-07-17T08:36:36.406087+0000","last_fullsized":"2024-07-17T08:36:36.406087+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:33.714966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735009+0000","last_change":"2024-07-17T08:36:34.772376+0000","last_active":"2024-07-17T08:36:35.735009+0000","last_peered":"2024-07-17T08:36:35.735009+0000","last_clean":"2024-07-17T08:36:35.735009+0000","last_became_active":"2024-07-17T08:36:34.772066+0000","last_became_peered":"2024-07-17T08:36:34.772066+0000","last_unstale":"2024-07-17T08:36:35.735009+0000","last_undegraded":"2024-07-17T08:36:35.735009+0000","last_fullsized":"2024-07-17T08:36:35.735009+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:41.322562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"246'8","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557060+0000","last_change":"2024-07-17T08:36:33.754851+0000","last_active":"2024-07-17T08:36:36.557060+0000","last_peered":"2024-07-17T08:36:36.557060+0000","last_clean":"2024-07-17T08:36:36.557060+0000","last_became_active":"2024-07-17T08:36:33.754191+0000","last_became_peered":"2024-07-17T08:36:33.754191+0000","last_unstale":"2024-07-17T08:36:36.557060+0000","last_undegraded":"2024-07-17T08:36:36.557060+0000","last_fullsized":"2024-07-17T08:36:36.557060+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:11.746726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742072+0000","last_change":"2024-07-17T08:36:22.631340+0000","last_active":"2024-07-17T08:36:35.742072+0000","last_peered":"2024-07-17T08:36:35.742072+0000","last_clean":"2024-07-17T08:36:35.742072+0000","last_became_active":"2024-07-17T08:36:22.631024+0000","last_became_peered":"2024-07-17T08:36:22.631024+0000","last_unstale":"2024-07-17T08:36:35.742072+0000","last_undegraded":"2024-07-17T08:36:35.742072+0000","last_fullsized":"2024-07-17T08:36:35.742072+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:14.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734906+0000","last_change":"2024-07-17T08:36:34.767985+0000","last_active":"2024-07-17T08:36:35.734906+0000","last_peered":"2024-07-17T08:36:35.734906+0000","last_clean":"2024-07-17T08:36:35.734906+0000","last_became_active":"2024-07-17T08:36:34.767692+0000","last_became_peered":"2024-07-17T08:36:34.767692+0000","last_unstale":"2024-07-17T08:36:35.734906+0000","last_undegraded":"2024-07-17T08:36:35.734906+0000","last_fullsized":"2024-07-17T08:36:35.734906+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:43.793611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557662+0000","last_change":"2024-07-17T08:36:33.758968+0000","last_active":"2024-07-17T08:36:36.557662+0000","last_peered":"2024-07-17T08:36:36.557662+0000","last_clean":"2024-07-17T08:36:36.557662+0000","last_became_active":"2024-07-17T08:36:33.757393+0000","last_became_peered":"2024-07-17T08:36:33.757393+0000","last_unstale":"2024-07-17T08:36:36.557662+0000","last_undegraded":"2024-07-17T08:36:36.557662+0000","last_fullsized":"2024-07-17T08:36:36.557662+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:54:23.614399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735634+0000","last_change":"2024-07-17T08:36:22.627265+0000","last_active":"2024-07-17T08:36:35.735634+0000","last_peered":"2024-07-17T08:36:35.735634+0000","last_clean":"2024-07-17T08:36:35.735634+0000","last_became_active":"2024-07-17T08:36:22.626441+0000","last_became_peered":"2024-07-17T08:36:22.626441+0000","last_unstale":"2024-07-17T08:36:35.735634+0000","last_undegraded":"2024-07-17T08:36:35.735634+0000","last_fullsized":"2024-07-17T08:36:35.735634+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:34.664027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559990+0000","last_change":"2024-07-17T08:36:34.769496+0000","last_active":"2024-07-17T08:36:36.559990+0000","last_peered":"2024-07-17T08:36:36.559990+0000","last_clean":"2024-07-17T08:36:36.559990+0000","last_became_active":"2024-07-17T08:36:34.767061+0000","last_became_peered":"2024-07-17T08:36:34.767061+0000","last_unstale":"2024-07-17T08:36:36.559990+0000","last_undegraded":"2024-07-17T08:36:36.559990+0000","last_fullsized":"2024-07-17T08:36:36.559990+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:53.417936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731725+0000","last_change":"2024-07-17T08:36:33.759374+0000","last_active":"2024-07-17T08:36:35.731725+0000","last_peered":"2024-07-17T08:36:35.731725+0000","last_clean":"2024-07-17T08:36:35.731725+0000","last_became_active":"2024-07-17T08:36:33.759075+0000","last_became_peered":"2024-07-17T08:36:33.759075+0000","last_unstale":"2024-07-17T08:36:35.731725+0000","last_undegraded":"2024-07-17T08:36:35.731725+0000","last_fullsized":"2024-07-17T08:36:35.731725+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:48.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561997+0000","last_change":"2024-07-17T08:36:22.617620+0000","last_active":"2024-07-17T08:36:36.561997+0000","last_peered":"2024-07-17T08:36:36.561997+0000","last_clean":"2024-07-17T08:36:36.561997+0000","last_became_active":"2024-07-17T08:36:22.616996+0000","last_became_peered":"2024-07-17T08:36:22.616996+0000","last_unstale":"2024-07-17T08:36:36.561997+0000","last_undegraded":"2024-07-17T08:36:36.561997+0000","last_fullsized":"2024-07-17T08:36:36.561997+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:51:23.335703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558626+0000","last_change":"2024-07-17T08:36:34.764651+0000","last_active":"2024-07-17T08:36:36.558626+0000","last_peered":"2024-07-17T08:36:36.558626+0000","last_clean":"2024-07-17T08:36:36.558626+0000","last_became_active":"2024-07-17T08:36:34.764374+0000","last_became_peered":"2024-07-17T08:36:34.764374+0000","last_unstale":"2024-07-17T08:36:36.558626+0000","last_undegraded":"2024-07-17T08:36:36.558626+0000","last_fullsized":"2024-07-17T08:36:36.558626+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:38:18.388381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743901+0000","last_change":"2024-07-17T08:36:33.761952+0000","last_active":"2024-07-17T08:36:35.743901+0000","last_peered":"2024-07-17T08:36:35.743901+0000","last_clean":"2024-07-17T08:36:35.743901+0000","last_became_active":"2024-07-17T08:36:33.761707+0000","last_became_peered":"2024-07-17T08:36:33.761707+0000","last_unstale":"2024-07-17T08:36:35.743901+0000","last_undegraded":"2024-07-17T08:36:35.743901+0000","last_fullsized":"2024-07-17T08:36:35.743901+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:26.511292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736609+0000","last_change":"2024-07-17T08:36:22.633485+0000","last_active":"2024-07-17T08:36:35.736609+0000","last_peered":"2024-07-17T08:36:35.736609+0000","last_clean":"2024-07-17T08:36:35.736609+0000","last_became_active":"2024-07-17T08:36:22.632802+0000","last_became_peered":"2024-07-17T08:36:22.632802+0000","last_unstale":"2024-07-17T08:36:35.736609+0000","last_undegraded":"2024-07-17T08:36:35.736609+0000","last_fullsized":"2024-07-17T08:36:35.736609+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:08.364483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561491+0000","last_change":"2024-07-17T08:36:34.765860+0000","last_active":"2024-07-17T08:36:36.561491+0000","last_peered":"2024-07-17T08:36:36.561491+0000","last_clean":"2024-07-17T08:36:36.561491+0000","last_became_active":"2024-07-17T08:36:34.765534+0000","last_became_peered":"2024-07-17T08:36:34.765534+0000","last_unstale":"2024-07-17T08:36:36.561491+0000","last_undegraded":"2024-07-17T08:36:36.561491+0000","last_fullsized":"2024-07-17T08:36:36.561491+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:55.660953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730838+0000","last_change":"2024-07-17T08:36:33.760094+0000","last_active":"2024-07-17T08:36:35.730838+0000","last_peered":"2024-07-17T08:36:35.730838+0000","last_clean":"2024-07-17T08:36:35.730838+0000","last_became_active":"2024-07-17T08:36:33.757800+0000","last_became_peered":"2024-07-17T08:36:33.757800+0000","last_unstale":"2024-07-17T08:36:35.730838+0000","last_undegraded":"2024-07-17T08:36:35.730838+0000","last_fullsized":"2024-07-17T08:36:35.730838+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:32.367201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743167+0000","last_change":"2024-07-17T08:36:22.636353+0000","last_active":"2024-07-17T08:36:35.743167+0000","last_peered":"2024-07-17T08:36:35.743167+0000","last_clean":"2024-07-17T08:36:35.743167+0000","last_became_active":"2024-07-17T08:36:22.636090+0000","last_became_peered":"2024-07-17T08:36:22.636090+0000","last_unstale":"2024-07-17T08:36:35.743167+0000","last_undegraded":"2024-07-17T08:36:35.743167+0000","last_fullsized":"2024-07-17T08:36:35.743167+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:32.993581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559766+0000","last_change":"2024-07-17T08:36:34.767120+0000","last_active":"2024-07-17T08:36:36.559766+0000","last_peered":"2024-07-17T08:36:36.559766+0000","last_clean":"2024-07-17T08:36:36.559766+0000","last_became_active":"2024-07-17T08:36:34.766506+0000","last_became_peered":"2024-07-17T08:36:34.766506+0000","last_unstale":"2024-07-17T08:36:36.559766+0000","last_undegraded":"2024-07-17T08:36:36.559766+0000","last_fullsized":"2024-07-17T08:36:36.559766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:50.347213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"246'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560996+0000","last_change":"2024-07-17T08:36:33.750553+0000","last_active":"2024-07-17T08:36:36.560996+0000","last_peered":"2024-07-17T08:36:36.560996+0000","last_clean":"2024-07-17T08:36:36.560996+0000","last_became_active":"2024-07-17T08:36:33.750105+0000","last_became_peered":"2024-07-17T08:36:33.750105+0000","last_unstale":"2024-07-17T08:36:36.560996+0000","last_undegraded":"2024-07-17T08:36:36.560996+0000","last_fullsized":"2024-07-17T08:36:36.560996+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:45.778285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731232+0000","last_change":"2024-07-17T08:36:22.614476+0000","last_active":"2024-07-17T08:36:35.731232+0000","last_peered":"2024-07-17T08:36:35.731232+0000","last_clean":"2024-07-17T08:36:35.731232+0000","last_became_active":"2024-07-17T08:36:22.613980+0000","last_became_peered":"2024-07-17T08:36:22.613980+0000","last_unstale":"2024-07-17T08:36:35.731232+0000","last_undegraded":"2024-07-17T08:36:35.731232+0000","last_fullsized":"2024-07-17T08:36:35.731232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:03.804813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558254+0000","last_change":"2024-07-17T08:36:34.762489+0000","last_active":"2024-07-17T08:36:36.558254+0000","last_peered":"2024-07-17T08:36:36.558254+0000","last_clean":"2024-07-17T08:36:36.558254+0000","last_became_active":"2024-07-17T08:36:34.762152+0000","last_became_peered":"2024-07-17T08:36:34.762152+0000","last_unstale":"2024-07-17T08:36:36.558254+0000","last_undegraded":"2024-07-17T08:36:36.558254+0000","last_fullsized":"2024-07-17T08:36:36.558254+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:34.839616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735072+0000","last_change":"2024-07-17T08:36:33.747393+0000","last_active":"2024-07-17T08:36:35.735072+0000","last_peered":"2024-07-17T08:36:35.735072+0000","last_clean":"2024-07-17T08:36:35.735072+0000","last_became_active":"2024-07-17T08:36:33.746617+0000","last_became_peered":"2024-07-17T08:36:33.746617+0000","last_unstale":"2024-07-17T08:36:35.735072+0000","last_undegraded":"2024-07-17T08:36:35.735072+0000","last_fullsized":"2024-07-17T08:36:35.735072+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:44.524775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559848+0000","last_change":"2024-07-17T08:36:22.634890+0000","last_active":"2024-07-17T08:36:36.559848+0000","last_peered":"2024-07-17T08:36:36.559848+0000","last_clean":"2024-07-17T08:36:36.559848+0000","last_became_active":"2024-07-17T08:36:22.634508+0000","last_became_peered":"2024-07-17T08:36:22.634508+0000","last_unstale":"2024-07-17T08:36:36.559848+0000","last_undegraded":"2024-07-17T08:36:36.559848+0000","last_fullsized":"2024-07-17T08:36:36.559848+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:56.457313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405145+0000","last_change":"2024-07-17T08:36:34.763807+0000","last_active":"2024-07-17T08:36:36.405145+0000","last_peered":"2024-07-17T08:36:36.405145+0000","last_clean":"2024-07-17T08:36:36.405145+0000","last_became_active":"2024-07-17T08:36:34.763502+0000","last_became_peered":"2024-07-17T08:36:34.763502+0000","last_unstale":"2024-07-17T08:36:36.405145+0000","last_undegraded":"2024-07-17T08:36:36.405145+0000","last_fullsized":"2024-07-17T08:36:36.405145+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:52.862035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735458+0000","last_change":"2024-07-17T08:36:33.756251+0000","last_active":"2024-07-17T08:36:35.735458+0000","last_peered":"2024-07-17T08:36:35.735458+0000","last_clean":"2024-07-17T08:36:35.735458+0000","last_became_active":"2024-07-17T08:36:33.755982+0000","last_became_peered":"2024-07-17T08:36:33.755982+0000","last_unstale":"2024-07-17T08:36:35.735458+0000","last_undegraded":"2024-07-17T08:36:35.735458+0000","last_fullsized":"2024-07-17T08:36:35.735458+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:12:04.834399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558966+0000","last_change":"2024-07-17T08:36:22.632594+0000","last_active":"2024-07-17T08:36:36.558966+0000","last_peered":"2024-07-17T08:36:36.558966+0000","last_clean":"2024-07-17T08:36:36.558966+0000","last_became_active":"2024-07-17T08:36:22.632283+0000","last_became_peered":"2024-07-17T08:36:22.632283+0000","last_unstale":"2024-07-17T08:36:36.558966+0000","last_undegraded":"2024-07-17T08:36:36.558966+0000","last_fullsized":"2024-07-17T08:36:36.558966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:24:23.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405320+0000","last_change":"2024-07-17T08:36:34.761196+0000","last_active":"2024-07-17T08:36:36.405320+0000","last_peered":"2024-07-17T08:36:36.405320+0000","last_clean":"2024-07-17T08:36:36.405320+0000","last_became_active":"2024-07-17T08:36:34.760906+0000","last_became_peered":"2024-07-17T08:36:34.760906+0000","last_unstale":"2024-07-17T08:36:36.405320+0000","last_undegraded":"2024-07-17T08:36:36.405320+0000","last_fullsized":"2024-07-17T08:36:36.405320+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:39.899314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557265+0000","last_change":"2024-07-17T08:36:33.753248+0000","last_active":"2024-07-17T08:36:36.557265+0000","last_peered":"2024-07-17T08:36:36.557265+0000","last_clean":"2024-07-17T08:36:36.557265+0000","last_became_active":"2024-07-17T08:36:33.752921+0000","last_became_peered":"2024-07-17T08:36:33.752921+0000","last_unstale":"2024-07-17T08:36:36.557265+0000","last_undegraded":"2024-07-17T08:36:36.557265+0000","last_fullsized":"2024-07-17T08:36:36.557265+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:46.641811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742343+0000","last_change":"2024-07-17T08:36:22.638957+0000","last_active":"2024-07-17T08:36:35.742343+0000","last_peered":"2024-07-17T08:36:35.742343+0000","last_clean":"2024-07-17T08:36:35.742343+0000","last_became_active":"2024-07-17T08:36:22.638634+0000","last_became_peered":"2024-07-17T08:36:22.638634+0000","last_unstale":"2024-07-17T08:36:35.742343+0000","last_undegraded":"2024-07-17T08:36:35.742343+0000","last_fullsized":"2024-07-17T08:36:35.742343+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:46:26.976107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557766+0000","last_change":"2024-07-17T08:36:34.766089+0000","last_active":"2024-07-17T08:36:36.557766+0000","last_peered":"2024-07-17T08:36:36.557766+0000","last_clean":"2024-07-17T08:36:36.557766+0000","last_became_active":"2024-07-17T08:36:34.762344+0000","last_became_peered":"2024-07-17T08:36:34.762344+0000","last_unstale":"2024-07-17T08:36:36.557766+0000","last_undegraded":"2024-07-17T08:36:36.557766+0000","last_fullsized":"2024-07-17T08:36:36.557766+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:10:43.102170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406148+0000","last_change":"2024-07-17T08:36:33.735145+0000","last_active":"2024-07-17T08:36:36.406148+0000","last_peered":"2024-07-17T08:36:36.406148+0000","last_clean":"2024-07-17T08:36:36.406148+0000","last_became_active":"2024-07-17T08:36:33.734773+0000","last_became_peered":"2024-07-17T08:36:33.734773+0000","last_unstale":"2024-07-17T08:36:36.406148+0000","last_undegraded":"2024-07-17T08:36:36.406148+0000","last_fullsized":"2024-07-17T08:36:36.406148+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:02:06.368749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557818+0000","last_change":"2024-07-17T08:36:22.626305+0000","last_active":"2024-07-17T08:36:36.557818+0000","last_peered":"2024-07-17T08:36:36.557818+0000","last_clean":"2024-07-17T08:36:36.557818+0000","last_became_active":"2024-07-17T08:36:22.625168+0000","last_became_peered":"2024-07-17T08:36:22.625168+0000","last_unstale":"2024-07-17T08:36:36.557818+0000","last_undegraded":"2024-07-17T08:36:36.557818+0000","last_fullsized":"2024-07-17T08:36:36.557818+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:37:33.400138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742864+0000","last_change":"2024-07-17T08:36:34.769570+0000","last_active":"2024-07-17T08:36:35.742864+0000","last_peered":"2024-07-17T08:36:35.742864+0000","last_clean":"2024-07-17T08:36:35.742864+0000","last_became_active":"2024-07-17T08:36:34.769300+0000","last_became_peered":"2024-07-17T08:36:34.769300+0000","last_unstale":"2024-07-17T08:36:35.742864+0000","last_undegraded":"2024-07-17T08:36:35.742864+0000","last_fullsized":"2024-07-17T08:36:35.742864+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:54.576193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"247'1","reported_seq":15,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732163+0000","last_change":"2024-07-17T08:36:33.758508+0000","last_active":"2024-07-17T08:36:35.732163+0000","last_peered":"2024-07-17T08:36:35.732163+0000","last_clean":"2024-07-17T08:36:35.732163+0000","last_became_active":"2024-07-17T08:36:33.757683+0000","last_became_peered":"2024-07-17T08:36:33.757683+0000","last_unstale":"2024-07-17T08:36:35.732163+0000","last_undegraded":"2024-07-17T08:36:35.732163+0000","last_fullsized":"2024-07-17T08:36:35.732163+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:13.951810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732029+0000","last_change":"2024-07-17T08:36:22.621045+0000","last_active":"2024-07-17T08:36:35.732029+0000","last_peered":"2024-07-17T08:36:35.732029+0000","last_clean":"2024-07-17T08:36:35.732029+0000","last_became_active":"2024-07-17T08:36:22.620730+0000","last_became_peered":"2024-07-17T08:36:22.620730+0000","last_unstale":"2024-07-17T08:36:35.732029+0000","last_undegraded":"2024-07-17T08:36:35.732029+0000","last_fullsized":"2024-07-17T08:36:35.732029+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:56.556105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743580+0000","last_change":"2024-07-17T08:36:34.762378+0000","last_active":"2024-07-17T08:36:35.743580+0000","last_peered":"2024-07-17T08:36:35.743580+0000","last_clean":"2024-07-17T08:36:35.743580+0000","last_became_active":"2024-07-17T08:36:34.761575+0000","last_became_peered":"2024-07-17T08:36:34.761575+0000","last_unstale":"2024-07-17T08:36:35.743580+0000","last_undegraded":"2024-07-17T08:36:35.743580+0000","last_fullsized":"2024-07-17T08:36:35.743580+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:32.981640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"246'2","reported_seq":16,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559978+0000","last_change":"2024-07-17T08:36:33.757914+0000","last_active":"2024-07-17T08:36:36.559978+0000","last_peered":"2024-07-17T08:36:36.559978+0000","last_clean":"2024-07-17T08:36:36.559978+0000","last_became_active":"2024-07-17T08:36:33.757588+0000","last_became_peered":"2024-07-17T08:36:33.757588+0000","last_unstale":"2024-07-17T08:36:36.559978+0000","last_undegraded":"2024-07-17T08:36:36.559978+0000","last_fullsized":"2024-07-17T08:36:36.559978+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:32.359840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560887+0000","last_change":"2024-07-17T08:36:22.639834+0000","last_active":"2024-07-17T08:36:36.560887+0000","last_peered":"2024-07-17T08:36:36.560887+0000","last_clean":"2024-07-17T08:36:36.560887+0000","last_became_active":"2024-07-17T08:36:22.638754+0000","last_became_peered":"2024-07-17T08:36:22.638754+0000","last_unstale":"2024-07-17T08:36:36.560887+0000","last_undegraded":"2024-07-17T08:36:36.560887+0000","last_fullsized":"2024-07-17T08:36:36.560887+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:01.104869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743750+0000","last_change":"2024-07-17T08:36:34.771365+0000","last_active":"2024-07-17T08:36:35.743750+0000","last_peered":"2024-07-17T08:36:35.743750+0000","last_clean":"2024-07-17T08:36:35.743750+0000","last_became_active":"2024-07-17T08:36:34.770738+0000","last_became_peered":"2024-07-17T08:36:34.770738+0000","last_unstale":"2024-07-17T08:36:35.743750+0000","last_undegraded":"2024-07-17T08:36:35.743750+0000","last_fullsized":"2024-07-17T08:36:35.743750+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:17:43.891881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561664+0000","last_change":"2024-07-17T08:36:33.756054+0000","last_active":"2024-07-17T08:36:36.561664+0000","last_peered":"2024-07-17T08:36:36.561664+0000","last_clean":"2024-07-17T08:36:36.561664+0000","last_became_active":"2024-07-17T08:36:33.755748+0000","last_became_peered":"2024-07-17T08:36:33.755748+0000","last_unstale":"2024-07-17T08:36:36.561664+0000","last_undegraded":"2024-07-17T08:36:36.561664+0000","last_fullsized":"2024-07-17T08:36:36.561664+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:43.965300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736609+0000","last_change":"2024-07-17T08:36:22.628504+0000","last_active":"2024-07-17T08:36:35.736609+0000","last_peered":"2024-07-17T08:36:35.736609+0000","last_clean":"2024-07-17T08:36:35.736609+0000","last_became_active":"2024-07-17T08:36:22.627199+0000","last_became_peered":"2024-07-17T08:36:22.627199+0000","last_unstale":"2024-07-17T08:36:35.736609+0000","last_undegraded":"2024-07-17T08:36:35.736609+0000","last_fullsized":"2024-07-17T08:36:35.736609+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:37.701806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560249+0000","last_change":"2024-07-17T08:36:34.761601+0000","last_active":"2024-07-17T08:36:36.560249+0000","last_peered":"2024-07-17T08:36:36.560249+0000","last_clean":"2024-07-17T08:36:36.560249+0000","last_became_active":"2024-07-17T08:36:34.761306+0000","last_became_peered":"2024-07-17T08:36:34.761306+0000","last_unstale":"2024-07-17T08:36:36.560249+0000","last_undegraded":"2024-07-17T08:36:36.560249+0000","last_fullsized":"2024-07-17T08:36:36.560249+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:04.486870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560154+0000","last_change":"2024-07-17T08:36:33.756539+0000","last_active":"2024-07-17T08:36:36.560154+0000","last_peered":"2024-07-17T08:36:36.560154+0000","last_clean":"2024-07-17T08:36:36.560154+0000","last_became_active":"2024-07-17T08:36:33.755822+0000","last_became_peered":"2024-07-17T08:36:33.755822+0000","last_unstale":"2024-07-17T08:36:36.560154+0000","last_undegraded":"2024-07-17T08:36:36.560154+0000","last_fullsized":"2024-07-17T08:36:36.560154+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:34.567530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559586+0000","last_change":"2024-07-17T08:36:22.625531+0000","last_active":"2024-07-17T08:36:36.559586+0000","last_peered":"2024-07-17T08:36:36.559586+0000","last_clean":"2024-07-17T08:36:36.559586+0000","last_became_active":"2024-07-17T08:36:22.624907+0000","last_became_peered":"2024-07-17T08:36:22.624907+0000","last_unstale":"2024-07-17T08:36:36.559586+0000","last_undegraded":"2024-07-17T08:36:36.559586+0000","last_fullsized":"2024-07-17T08:36:36.559586+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:44.804601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405849+0000","last_change":"2024-07-17T08:36:34.772888+0000","last_active":"2024-07-17T08:36:36.405849+0000","last_peered":"2024-07-17T08:36:36.405849+0000","last_clean":"2024-07-17T08:36:36.405849+0000","last_became_active":"2024-07-17T08:36:34.771828+0000","last_became_peered":"2024-07-17T08:36:34.771828+0000","last_unstale":"2024-07-17T08:36:36.405849+0000","last_undegraded":"2024-07-17T08:36:36.405849+0000","last_fullsized":"2024-07-17T08:36:36.405849+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:24.603843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730262+0000","last_change":"2024-07-17T08:36:33.755126+0000","last_active":"2024-07-17T08:36:35.730262+0000","last_peered":"2024-07-17T08:36:35.730262+0000","last_clean":"2024-07-17T08:36:35.730262+0000","last_became_active":"2024-07-17T08:36:33.753919+0000","last_became_peered":"2024-07-17T08:36:33.753919+0000","last_unstale":"2024-07-17T08:36:35.730262+0000","last_undegraded":"2024-07-17T08:36:35.730262+0000","last_fullsized":"2024-07-17T08:36:35.730262+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:07.283093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405878+0000","last_change":"2024-07-17T08:36:22.632503+0000","last_active":"2024-07-17T08:36:36.405878+0000","last_peered":"2024-07-17T08:36:36.405878+0000","last_clean":"2024-07-17T08:36:36.405878+0000","last_became_active":"2024-07-17T08:36:22.632216+0000","last_became_peered":"2024-07-17T08:36:22.632216+0000","last_unstale":"2024-07-17T08:36:36.405878+0000","last_undegraded":"2024-07-17T08:36:36.405878+0000","last_fullsized":"2024-07-17T08:36:36.405878+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:57.146695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557257+0000","last_change":"2024-07-17T08:36:34.757792+0000","last_active":"2024-07-17T08:36:36.557257+0000","last_peered":"2024-07-17T08:36:36.557257+0000","last_clean":"2024-07-17T08:36:36.557257+0000","last_became_active":"2024-07-17T08:36:34.757510+0000","last_became_peered":"2024-07-17T08:36:34.757510+0000","last_unstale":"2024-07-17T08:36:36.557257+0000","last_undegraded":"2024-07-17T08:36:36.557257+0000","last_fullsized":"2024-07-17T08:36:36.557257+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:05.662586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730760+0000","last_change":"2024-07-17T08:36:33.758782+0000","last_active":"2024-07-17T08:36:35.730760+0000","last_peered":"2024-07-17T08:36:35.730760+0000","last_clean":"2024-07-17T08:36:35.730760+0000","last_became_active":"2024-07-17T08:36:33.757603+0000","last_became_peered":"2024-07-17T08:36:33.757603+0000","last_unstale":"2024-07-17T08:36:35.730760+0000","last_undegraded":"2024-07-17T08:36:35.730760+0000","last_fullsized":"2024-07-17T08:36:35.730760+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:01.305456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406624+0000","last_change":"2024-07-17T08:36:22.629725+0000","last_active":"2024-07-17T08:36:36.406624+0000","last_peered":"2024-07-17T08:36:36.406624+0000","last_clean":"2024-07-17T08:36:36.406624+0000","last_became_active":"2024-07-17T08:36:22.626785+0000","last_became_peered":"2024-07-17T08:36:22.626785+0000","last_unstale":"2024-07-17T08:36:36.406624+0000","last_undegraded":"2024-07-17T08:36:36.406624+0000","last_fullsized":"2024-07-17T08:36:36.406624+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:44.495269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736740+0000","last_change":"2024-07-17T08:36:34.747846+0000","last_active":"2024-07-17T08:36:35.736740+0000","last_peered":"2024-07-17T08:36:35.736740+0000","last_clean":"2024-07-17T08:36:35.736740+0000","last_became_active":"2024-07-17T08:36:34.746565+0000","last_became_peered":"2024-07-17T08:36:34.746565+0000","last_unstale":"2024-07-17T08:36:35.736740+0000","last_undegraded":"2024-07-17T08:36:35.736740+0000","last_fullsized":"2024-07-17T08:36:35.736740+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:52.538461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.729622+0000","last_change":"2024-07-17T08:36:33.735347+0000","last_active":"2024-07-17T08:36:35.729622+0000","last_peered":"2024-07-17T08:36:35.729622+0000","last_clean":"2024-07-17T08:36:35.729622+0000","last_became_active":"2024-07-17T08:36:33.734974+0000","last_became_peered":"2024-07-17T08:36:33.734974+0000","last_unstale":"2024-07-17T08:36:35.729622+0000","last_undegraded":"2024-07-17T08:36:35.729622+0000","last_fullsized":"2024-07-17T08:36:35.729622+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:04.954732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730287+0000","last_change":"2024-07-17T08:36:22.622222+0000","last_active":"2024-07-17T08:36:35.730287+0000","last_peered":"2024-07-17T08:36:35.730287+0000","last_clean":"2024-07-17T08:36:35.730287+0000","last_became_active":"2024-07-17T08:36:22.621799+0000","last_became_peered":"2024-07-17T08:36:22.621799+0000","last_unstale":"2024-07-17T08:36:35.730287+0000","last_undegraded":"2024-07-17T08:36:35.730287+0000","last_fullsized":"2024-07-17T08:36:35.730287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:38.035505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405245+0000","last_change":"2024-07-17T08:36:34.771421+0000","last_active":"2024-07-17T08:36:36.405245+0000","last_peered":"2024-07-17T08:36:36.405245+0000","last_clean":"2024-07-17T08:36:36.405245+0000","last_became_active":"2024-07-17T08:36:34.770940+0000","last_became_peered":"2024-07-17T08:36:34.770940+0000","last_unstale":"2024-07-17T08:36:36.405245+0000","last_undegraded":"2024-07-17T08:36:36.405245+0000","last_fullsized":"2024-07-17T08:36:36.405245+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:32.518091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":14,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559823+0000","last_change":"2024-07-17T08:36:33.749168+0000","last_active":"2024-07-17T08:36:36.559823+0000","last_peered":"2024-07-17T08:36:36.559823+0000","last_clean":"2024-07-17T08:36:36.559823+0000","last_became_active":"2024-07-17T08:36:33.748825+0000","last_became_peered":"2024-07-17T08:36:33.748825+0000","last_unstale":"2024-07-17T08:36:36.559823+0000","last_undegraded":"2024-07-17T08:36:36.559823+0000","last_fullsized":"2024-07-17T08:36:36.559823+0000","mapping_epoch":244,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:32.681758+0000","last_clean_scrub_stamp":"2024-07-17T08:36:32.681758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:46:35.616914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558155+0000","last_change":"2024-07-17T08:36:22.626802+0000","last_active":"2024-07-17T08:36:36.558155+0000","last_peered":"2024-07-17T08:36:36.558155+0000","last_clean":"2024-07-17T08:36:36.558155+0000","last_became_active":"2024-07-17T08:36:22.626509+0000","last_became_peered":"2024-07-17T08:36:22.626509+0000","last_unstale":"2024-07-17T08:36:36.558155+0000","last_undegraded":"2024-07-17T08:36:36.558155+0000","last_fullsized":"2024-07-17T08:36:36.558155+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:28.580273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742019+0000","last_change":"2024-07-17T08:36:34.759506+0000","last_active":"2024-07-17T08:36:35.742019+0000","last_peered":"2024-07-17T08:36:35.742019+0000","last_clean":"2024-07-17T08:36:35.742019+0000","last_became_active":"2024-07-17T08:36:34.757200+0000","last_became_peered":"2024-07-17T08:36:34.757200+0000","last_unstale":"2024-07-17T08:36:35.742019+0000","last_undegraded":"2024-07-17T08:36:35.742019+0000","last_fullsized":"2024-07-17T08:36:35.742019+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:29:59.072833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730220+0000","last_change":"2024-07-17T08:36:22.635775+0000","last_active":"2024-07-17T08:36:35.730220+0000","last_peered":"2024-07-17T08:36:35.730220+0000","last_clean":"2024-07-17T08:36:35.730220+0000","last_became_active":"2024-07-17T08:36:22.634977+0000","last_became_peered":"2024-07-17T08:36:22.634977+0000","last_unstale":"2024-07-17T08:36:35.730220+0000","last_undegraded":"2024-07-17T08:36:35.730220+0000","last_fullsized":"2024-07-17T08:36:35.730220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:53.198970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"242'4","reported_seq":26,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560226+0000","last_change":"2024-07-17T08:36:21.602986+0000","last_active":"2024-07-17T08:36:36.560226+0000","last_peered":"2024-07-17T08:36:36.560226+0000","last_clean":"2024-07-17T08:36:36.560226+0000","last_became_active":"2024-07-17T08:36:21.602561+0000","last_became_peered":"2024-07-17T08:36:21.602561+0000","last_unstale":"2024-07-17T08:36:36.560226+0000","last_undegraded":"2024-07-17T08:36:36.560226+0000","last_fullsized":"2024-07-17T08:36:36.560226+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:28.181800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404648+0000","last_change":"2024-07-17T08:36:34.756329+0000","last_active":"2024-07-17T08:36:36.404648+0000","last_peered":"2024-07-17T08:36:36.404648+0000","last_clean":"2024-07-17T08:36:36.404648+0000","last_became_active":"2024-07-17T08:36:34.756056+0000","last_became_peered":"2024-07-17T08:36:34.756056+0000","last_unstale":"2024-07-17T08:36:36.404648+0000","last_undegraded":"2024-07-17T08:36:36.404648+0000","last_fullsized":"2024-07-17T08:36:36.404648+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:00.211357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742286+0000","last_change":"2024-07-17T08:36:22.634962+0000","last_active":"2024-07-17T08:36:35.742286+0000","last_peered":"2024-07-17T08:36:35.742286+0000","last_clean":"2024-07-17T08:36:35.742286+0000","last_became_active":"2024-07-17T08:36:22.634558+0000","last_became_peered":"2024-07-17T08:36:22.634558+0000","last_unstale":"2024-07-17T08:36:35.742286+0000","last_undegraded":"2024-07-17T08:36:35.742286+0000","last_fullsized":"2024-07-17T08:36:35.742286+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:26.433601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560752+0000","last_change":"2024-07-17T08:36:21.614964+0000","last_active":"2024-07-17T08:36:36.560752+0000","last_peered":"2024-07-17T08:36:36.560752+0000","last_clean":"2024-07-17T08:36:36.560752+0000","last_became_active":"2024-07-17T08:36:21.611104+0000","last_became_peered":"2024-07-17T08:36:21.611104+0000","last_unstale":"2024-07-17T08:36:36.560752+0000","last_undegraded":"2024-07-17T08:36:36.560752+0000","last_fullsized":"2024-07-17T08:36:36.560752+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:02.798346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406536+0000","last_change":"2024-07-17T08:36:34.776531+0000","last_active":"2024-07-17T08:36:36.406536+0000","last_peered":"2024-07-17T08:36:36.406536+0000","last_clean":"2024-07-17T08:36:36.406536+0000","last_became_active":"2024-07-17T08:36:34.776010+0000","last_became_peered":"2024-07-17T08:36:34.776010+0000","last_unstale":"2024-07-17T08:36:36.406536+0000","last_undegraded":"2024-07-17T08:36:36.406536+0000","last_fullsized":"2024-07-17T08:36:36.406536+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:27.058250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559158+0000","last_change":"2024-07-17T08:36:22.631210+0000","last_active":"2024-07-17T08:36:36.559158+0000","last_peered":"2024-07-17T08:36:36.559158+0000","last_clean":"2024-07-17T08:36:36.559158+0000","last_became_active":"2024-07-17T08:36:22.630922+0000","last_became_peered":"2024-07-17T08:36:22.630922+0000","last_unstale":"2024-07-17T08:36:36.559158+0000","last_undegraded":"2024-07-17T08:36:36.559158+0000","last_fullsized":"2024-07-17T08:36:36.559158+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:27.765019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406585+0000","last_change":"2024-07-17T08:36:21.617732+0000","last_active":"2024-07-17T08:36:36.406585+0000","last_peered":"2024-07-17T08:36:36.406585+0000","last_clean":"2024-07-17T08:36:36.406585+0000","last_became_active":"2024-07-17T08:36:21.615523+0000","last_became_peered":"2024-07-17T08:36:21.615523+0000","last_unstale":"2024-07-17T08:36:36.406585+0000","last_undegraded":"2024-07-17T08:36:36.406585+0000","last_fullsized":"2024-07-17T08:36:36.406585+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:39.182073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407015+0000","last_change":"2024-07-17T08:36:34.768405+0000","last_active":"2024-07-17T08:36:36.407015+0000","last_peered":"2024-07-17T08:36:36.407015+0000","last_clean":"2024-07-17T08:36:36.407015+0000","last_became_active":"2024-07-17T08:36:34.768128+0000","last_became_peered":"2024-07-17T08:36:34.768128+0000","last_unstale":"2024-07-17T08:36:36.407015+0000","last_undegraded":"2024-07-17T08:36:36.407015+0000","last_fullsized":"2024-07-17T08:36:36.407015+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:26.784151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405806+0000","last_change":"2024-07-17T08:36:22.623312+0000","last_active":"2024-07-17T08:36:36.405806+0000","last_peered":"2024-07-17T08:36:36.405806+0000","last_clean":"2024-07-17T08:36:36.405806+0000","last_became_active":"2024-07-17T08:36:22.619422+0000","last_became_peered":"2024-07-17T08:36:22.619422+0000","last_unstale":"2024-07-17T08:36:36.405806+0000","last_undegraded":"2024-07-17T08:36:36.405806+0000","last_fullsized":"2024-07-17T08:36:36.405806+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:06:59.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742501+0000","last_change":"2024-07-17T08:36:21.622068+0000","last_active":"2024-07-17T08:36:35.742501+0000","last_peered":"2024-07-17T08:36:35.742501+0000","last_clean":"2024-07-17T08:36:35.742501+0000","last_became_active":"2024-07-17T08:36:21.621662+0000","last_became_peered":"2024-07-17T08:36:21.621662+0000","last_unstale":"2024-07-17T08:36:35.742501+0000","last_undegraded":"2024-07-17T08:36:35.742501+0000","last_fullsized":"2024-07-17T08:36:35.742501+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:53:05.996618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561721+0000","last_change":"2024-07-17T08:36:34.762335+0000","last_active":"2024-07-17T08:36:36.561721+0000","last_peered":"2024-07-17T08:36:36.561721+0000","last_clean":"2024-07-17T08:36:36.561721+0000","last_became_active":"2024-07-17T08:36:34.762073+0000","last_became_peered":"2024-07-17T08:36:34.762073+0000","last_unstale":"2024-07-17T08:36:36.561721+0000","last_undegraded":"2024-07-17T08:36:36.561721+0000","last_fullsized":"2024-07-17T08:36:36.561721+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:47.721727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561037+0000","last_change":"2024-07-17T08:36:22.634974+0000","last_active":"2024-07-17T08:36:36.561037+0000","last_peered":"2024-07-17T08:36:36.561037+0000","last_clean":"2024-07-17T08:36:36.561037+0000","last_became_active":"2024-07-17T08:36:22.634653+0000","last_became_peered":"2024-07-17T08:36:22.634653+0000","last_unstale":"2024-07-17T08:36:36.561037+0000","last_undegraded":"2024-07-17T08:36:36.561037+0000","last_fullsized":"2024-07-17T08:36:36.561037+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:12.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407040+0000","last_change":"2024-07-17T08:36:21.617119+0000","last_active":"2024-07-17T08:36:36.407040+0000","last_peered":"2024-07-17T08:36:36.407040+0000","last_clean":"2024-07-17T08:36:36.407040+0000","last_became_active":"2024-07-17T08:36:21.616850+0000","last_became_peered":"2024-07-17T08:36:21.616850+0000","last_unstale":"2024-07-17T08:36:36.407040+0000","last_undegraded":"2024-07-17T08:36:36.407040+0000","last_fullsized":"2024-07-17T08:36:36.407040+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:15.709164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559358+0000","last_change":"2024-07-17T08:36:34.772217+0000","last_active":"2024-07-17T08:36:36.559358+0000","last_peered":"2024-07-17T08:36:36.559358+0000","last_clean":"2024-07-17T08:36:36.559358+0000","last_became_active":"2024-07-17T08:36:34.771921+0000","last_became_peered":"2024-07-17T08:36:34.771921+0000","last_unstale":"2024-07-17T08:36:36.559358+0000","last_undegraded":"2024-07-17T08:36:36.559358+0000","last_fullsized":"2024-07-17T08:36:36.559358+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:46.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736365+0000","last_change":"2024-07-17T08:36:22.646727+0000","last_active":"2024-07-17T08:36:35.736365+0000","last_peered":"2024-07-17T08:36:35.736365+0000","last_clean":"2024-07-17T08:36:35.736365+0000","last_became_active":"2024-07-17T08:36:22.646414+0000","last_became_peered":"2024-07-17T08:36:22.646414+0000","last_unstale":"2024-07-17T08:36:35.736365+0000","last_undegraded":"2024-07-17T08:36:35.736365+0000","last_fullsized":"2024-07-17T08:36:35.736365+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:40.188539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734862+0000","last_change":"2024-07-17T08:36:21.621097+0000","last_active":"2024-07-17T08:36:35.734862+0000","last_peered":"2024-07-17T08:36:35.734862+0000","last_clean":"2024-07-17T08:36:35.734862+0000","last_became_active":"2024-07-17T08:36:21.620829+0000","last_became_peered":"2024-07-17T08:36:21.620829+0000","last_unstale":"2024-07-17T08:36:35.734862+0000","last_undegraded":"2024-07-17T08:36:35.734862+0000","last_fullsized":"2024-07-17T08:36:35.734862+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:42.120317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559537+0000","last_change":"2024-07-17T08:36:34.772619+0000","last_active":"2024-07-17T08:36:36.559537+0000","last_peered":"2024-07-17T08:36:36.559537+0000","last_clean":"2024-07-17T08:36:36.559537+0000","last_became_active":"2024-07-17T08:36:34.772316+0000","last_became_peered":"2024-07-17T08:36:34.772316+0000","last_unstale":"2024-07-17T08:36:36.559537+0000","last_undegraded":"2024-07-17T08:36:36.559537+0000","last_fullsized":"2024-07-17T08:36:36.559537+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:26:04.078178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737996+0000","last_change":"2024-07-17T08:36:22.631296+0000","last_active":"2024-07-17T08:36:35.737996+0000","last_peered":"2024-07-17T08:36:35.737996+0000","last_clean":"2024-07-17T08:36:35.737996+0000","last_became_active":"2024-07-17T08:36:22.630247+0000","last_became_peered":"2024-07-17T08:36:22.630247+0000","last_unstale":"2024-07-17T08:36:35.737996+0000","last_undegraded":"2024-07-17T08:36:35.737996+0000","last_fullsized":"2024-07-17T08:36:35.737996+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:52.961897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558367+0000","last_change":"2024-07-17T08:36:21.615638+0000","last_active":"2024-07-17T08:36:36.558367+0000","last_peered":"2024-07-17T08:36:36.558367+0000","last_clean":"2024-07-17T08:36:36.558367+0000","last_became_active":"2024-07-17T08:36:21.615340+0000","last_became_peered":"2024-07-17T08:36:21.615340+0000","last_unstale":"2024-07-17T08:36:36.558367+0000","last_undegraded":"2024-07-17T08:36:36.558367+0000","last_fullsized":"2024-07-17T08:36:36.558367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:35.174424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559393+0000","last_change":"2024-07-17T08:36:34.775531+0000","last_active":"2024-07-17T08:36:36.559393+0000","last_peered":"2024-07-17T08:36:36.559393+0000","last_clean":"2024-07-17T08:36:36.559393+0000","last_became_active":"2024-07-17T08:36:34.775165+0000","last_became_peered":"2024-07-17T08:36:34.775165+0000","last_unstale":"2024-07-17T08:36:36.559393+0000","last_undegraded":"2024-07-17T08:36:36.559393+0000","last_fullsized":"2024-07-17T08:36:36.559393+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:06:11.650539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558966+0000","last_change":"2024-07-17T08:36:22.633208+0000","last_active":"2024-07-17T08:36:36.558966+0000","last_peered":"2024-07-17T08:36:36.558966+0000","last_clean":"2024-07-17T08:36:36.558966+0000","last_became_active":"2024-07-17T08:36:22.632929+0000","last_became_peered":"2024-07-17T08:36:22.632929+0000","last_unstale":"2024-07-17T08:36:36.558966+0000","last_undegraded":"2024-07-17T08:36:36.558966+0000","last_fullsized":"2024-07-17T08:36:36.558966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:23.489720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.407193+0000","last_change":"2024-07-17T08:36:21.622663+0000","last_active":"2024-07-17T08:36:36.407193+0000","last_peered":"2024-07-17T08:36:36.407193+0000","last_clean":"2024-07-17T08:36:36.407193+0000","last_became_active":"2024-07-17T08:36:21.622022+0000","last_became_peered":"2024-07-17T08:36:21.622022+0000","last_unstale":"2024-07-17T08:36:36.407193+0000","last_undegraded":"2024-07-17T08:36:36.407193+0000","last_fullsized":"2024-07-17T08:36:36.407193+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:00:09.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735122+0000","last_change":"2024-07-17T08:36:34.750275+0000","last_active":"2024-07-17T08:36:35.735122+0000","last_peered":"2024-07-17T08:36:35.735122+0000","last_clean":"2024-07-17T08:36:35.735122+0000","last_became_active":"2024-07-17T08:36:34.749980+0000","last_became_peered":"2024-07-17T08:36:34.749980+0000","last_unstale":"2024-07-17T08:36:35.735122+0000","last_undegraded":"2024-07-17T08:36:35.735122+0000","last_fullsized":"2024-07-17T08:36:35.735122+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:45.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558551+0000","last_change":"2024-07-17T08:36:22.632890+0000","last_active":"2024-07-17T08:36:36.558551+0000","last_peered":"2024-07-17T08:36:36.558551+0000","last_clean":"2024-07-17T08:36:36.558551+0000","last_became_active":"2024-07-17T08:36:22.632563+0000","last_became_peered":"2024-07-17T08:36:22.632563+0000","last_unstale":"2024-07-17T08:36:36.558551+0000","last_undegraded":"2024-07-17T08:36:36.558551+0000","last_fullsized":"2024-07-17T08:36:36.558551+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:14.829658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736687+0000","last_change":"2024-07-17T08:36:21.622513+0000","last_active":"2024-07-17T08:36:35.736687+0000","last_peered":"2024-07-17T08:36:35.736687+0000","last_clean":"2024-07-17T08:36:35.736687+0000","last_became_active":"2024-07-17T08:36:21.622206+0000","last_became_peered":"2024-07-17T08:36:21.622206+0000","last_unstale":"2024-07-17T08:36:35.736687+0000","last_undegraded":"2024-07-17T08:36:35.736687+0000","last_fullsized":"2024-07-17T08:36:35.736687+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:11.585153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557833+0000","last_change":"2024-07-17T08:36:34.764300+0000","last_active":"2024-07-17T08:36:36.557833+0000","last_peered":"2024-07-17T08:36:36.557833+0000","last_clean":"2024-07-17T08:36:36.557833+0000","last_became_active":"2024-07-17T08:36:34.764033+0000","last_became_peered":"2024-07-17T08:36:34.764033+0000","last_unstale":"2024-07-17T08:36:36.557833+0000","last_undegraded":"2024-07-17T08:36:36.557833+0000","last_fullsized":"2024-07-17T08:36:36.557833+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:00:23.399806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735197+0000","last_change":"2024-07-17T08:36:22.642780+0000","last_active":"2024-07-17T08:36:35.735197+0000","last_peered":"2024-07-17T08:36:35.735197+0000","last_clean":"2024-07-17T08:36:35.735197+0000","last_became_active":"2024-07-17T08:36:22.642138+0000","last_became_peered":"2024-07-17T08:36:22.642138+0000","last_unstale":"2024-07-17T08:36:35.735197+0000","last_undegraded":"2024-07-17T08:36:35.735197+0000","last_fullsized":"2024-07-17T08:36:35.735197+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:29.446828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"242'3","reported_seq":25,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736803+0000","last_change":"2024-07-17T08:36:21.618560+0000","last_active":"2024-07-17T08:36:35.736803+0000","last_peered":"2024-07-17T08:36:35.736803+0000","last_clean":"2024-07-17T08:36:35.736803+0000","last_became_active":"2024-07-17T08:36:21.618180+0000","last_became_peered":"2024-07-17T08:36:21.618180+0000","last_unstale":"2024-07-17T08:36:35.736803+0000","last_undegraded":"2024-07-17T08:36:35.736803+0000","last_fullsized":"2024-07-17T08:36:35.736803+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:04.729864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404878+0000","last_change":"2024-07-17T08:36:34.761187+0000","last_active":"2024-07-17T08:36:36.404878+0000","last_peered":"2024-07-17T08:36:36.404878+0000","last_clean":"2024-07-17T08:36:36.404878+0000","last_became_active":"2024-07-17T08:36:34.760833+0000","last_became_peered":"2024-07-17T08:36:34.760833+0000","last_unstale":"2024-07-17T08:36:36.404878+0000","last_undegraded":"2024-07-17T08:36:36.404878+0000","last_fullsized":"2024-07-17T08:36:36.404878+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:05:49.833097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559560+0000","last_change":"2024-07-17T08:36:22.637133+0000","last_active":"2024-07-17T08:36:36.559560+0000","last_peered":"2024-07-17T08:36:36.559560+0000","last_clean":"2024-07-17T08:36:36.559560+0000","last_became_active":"2024-07-17T08:36:22.636707+0000","last_became_peered":"2024-07-17T08:36:22.636707+0000","last_unstale":"2024-07-17T08:36:36.559560+0000","last_undegraded":"2024-07-17T08:36:36.559560+0000","last_fullsized":"2024-07-17T08:36:36.559560+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:43.126784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"243'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559818+0000","last_change":"2024-07-17T08:36:21.616880+0000","last_active":"2024-07-17T08:36:36.559818+0000","last_peered":"2024-07-17T08:36:36.559818+0000","last_clean":"2024-07-17T08:36:36.559818+0000","last_became_active":"2024-07-17T08:36:21.616622+0000","last_became_peered":"2024-07-17T08:36:21.616622+0000","last_unstale":"2024-07-17T08:36:36.559818+0000","last_undegraded":"2024-07-17T08:36:36.559818+0000","last_fullsized":"2024-07-17T08:36:36.559818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:41:32.538601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732279+0000","last_change":"2024-07-17T08:36:34.759236+0000","last_active":"2024-07-17T08:36:35.732279+0000","last_peered":"2024-07-17T08:36:35.732279+0000","last_clean":"2024-07-17T08:36:35.732279+0000","last_became_active":"2024-07-17T08:36:34.751420+0000","last_became_peered":"2024-07-17T08:36:34.751420+0000","last_unstale":"2024-07-17T08:36:35.732279+0000","last_undegraded":"2024-07-17T08:36:35.732279+0000","last_fullsized":"2024-07-17T08:36:35.732279+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:52.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731166+0000","last_change":"2024-07-17T08:36:22.640753+0000","last_active":"2024-07-17T08:36:35.731166+0000","last_peered":"2024-07-17T08:36:35.731166+0000","last_clean":"2024-07-17T08:36:35.731166+0000","last_became_active":"2024-07-17T08:36:22.640392+0000","last_became_peered":"2024-07-17T08:36:22.640392+0000","last_unstale":"2024-07-17T08:36:35.731166+0000","last_undegraded":"2024-07-17T08:36:35.731166+0000","last_fullsized":"2024-07-17T08:36:35.731166+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:20.897694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.742818+0000","last_change":"2024-07-17T08:36:21.620050+0000","last_active":"2024-07-17T08:36:35.742818+0000","last_peered":"2024-07-17T08:36:35.742818+0000","last_clean":"2024-07-17T08:36:35.742818+0000","last_became_active":"2024-07-17T08:36:21.619765+0000","last_became_peered":"2024-07-17T08:36:21.619765+0000","last_unstale":"2024-07-17T08:36:35.742818+0000","last_undegraded":"2024-07-17T08:36:35.742818+0000","last_fullsized":"2024-07-17T08:36:35.742818+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:05:56.290437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.737727+0000","last_change":"2024-07-17T08:36:34.756689+0000","last_active":"2024-07-17T08:36:35.737727+0000","last_peered":"2024-07-17T08:36:35.737727+0000","last_clean":"2024-07-17T08:36:35.737727+0000","last_became_active":"2024-07-17T08:36:34.755385+0000","last_became_peered":"2024-07-17T08:36:34.755385+0000","last_unstale":"2024-07-17T08:36:35.737727+0000","last_undegraded":"2024-07-17T08:36:35.737727+0000","last_fullsized":"2024-07-17T08:36:35.737727+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:55.261079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560741+0000","last_change":"2024-07-17T08:36:22.634924+0000","last_active":"2024-07-17T08:36:36.560741+0000","last_peered":"2024-07-17T08:36:36.560741+0000","last_clean":"2024-07-17T08:36:36.560741+0000","last_became_active":"2024-07-17T08:36:22.634593+0000","last_became_peered":"2024-07-17T08:36:22.634593+0000","last_unstale":"2024-07-17T08:36:36.560741+0000","last_undegraded":"2024-07-17T08:36:36.560741+0000","last_fullsized":"2024-07-17T08:36:36.560741+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:10.482748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736105+0000","last_change":"2024-07-17T08:36:21.600821+0000","last_active":"2024-07-17T08:36:35.736105+0000","last_peered":"2024-07-17T08:36:35.736105+0000","last_clean":"2024-07-17T08:36:35.736105+0000","last_became_active":"2024-07-17T08:36:21.598009+0000","last_became_peered":"2024-07-17T08:36:21.598009+0000","last_unstale":"2024-07-17T08:36:35.736105+0000","last_undegraded":"2024-07-17T08:36:35.736105+0000","last_fullsized":"2024-07-17T08:36:35.736105+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:52:43.173687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743307+0000","last_change":"2024-07-17T08:36:34.762630+0000","last_active":"2024-07-17T08:36:35.743307+0000","last_peered":"2024-07-17T08:36:35.743307+0000","last_clean":"2024-07-17T08:36:35.743307+0000","last_became_active":"2024-07-17T08:36:34.762294+0000","last_became_peered":"2024-07-17T08:36:34.762294+0000","last_unstale":"2024-07-17T08:36:35.743307+0000","last_undegraded":"2024-07-17T08:36:35.743307+0000","last_fullsized":"2024-07-17T08:36:35.743307+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:35.702183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743816+0000","last_change":"2024-07-17T08:36:22.644258+0000","last_active":"2024-07-17T08:36:35.743816+0000","last_peered":"2024-07-17T08:36:35.743816+0000","last_clean":"2024-07-17T08:36:35.743816+0000","last_became_active":"2024-07-17T08:36:22.643986+0000","last_became_peered":"2024-07-17T08:36:22.643986+0000","last_unstale":"2024-07-17T08:36:35.743816+0000","last_undegraded":"2024-07-17T08:36:35.743816+0000","last_fullsized":"2024-07-17T08:36:35.743816+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:57:15.095345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735336+0000","last_change":"2024-07-17T08:36:21.604938+0000","last_active":"2024-07-17T08:36:35.735336+0000","last_peered":"2024-07-17T08:36:35.735336+0000","last_clean":"2024-07-17T08:36:35.735336+0000","last_became_active":"2024-07-17T08:36:21.604534+0000","last_became_peered":"2024-07-17T08:36:21.604534+0000","last_unstale":"2024-07-17T08:36:35.735336+0000","last_undegraded":"2024-07-17T08:36:35.735336+0000","last_fullsized":"2024-07-17T08:36:35.735336+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:38:51.548007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736222+0000","last_change":"2024-07-17T08:36:34.766601+0000","last_active":"2024-07-17T08:36:35.736222+0000","last_peered":"2024-07-17T08:36:35.736222+0000","last_clean":"2024-07-17T08:36:35.736222+0000","last_became_active":"2024-07-17T08:36:34.766306+0000","last_became_peered":"2024-07-17T08:36:34.766306+0000","last_unstale":"2024-07-17T08:36:35.736222+0000","last_undegraded":"2024-07-17T08:36:35.736222+0000","last_fullsized":"2024-07-17T08:36:35.736222+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:19:00.924743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738018+0000","last_change":"2024-07-17T08:36:22.613146+0000","last_active":"2024-07-17T08:36:35.738018+0000","last_peered":"2024-07-17T08:36:35.738018+0000","last_clean":"2024-07-17T08:36:35.738018+0000","last_became_active":"2024-07-17T08:36:22.608402+0000","last_became_peered":"2024-07-17T08:36:22.608402+0000","last_unstale":"2024-07-17T08:36:35.738018+0000","last_undegraded":"2024-07-17T08:36:35.738018+0000","last_fullsized":"2024-07-17T08:36:35.738018+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:37.176719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"242'8","reported_seq":30,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730519+0000","last_change":"2024-07-17T08:36:21.617338+0000","last_active":"2024-07-17T08:36:35.730519+0000","last_peered":"2024-07-17T08:36:35.730519+0000","last_clean":"2024-07-17T08:36:35.730519+0000","last_became_active":"2024-07-17T08:36:21.617084+0000","last_became_peered":"2024-07-17T08:36:21.617084+0000","last_unstale":"2024-07-17T08:36:35.730519+0000","last_undegraded":"2024-07-17T08:36:35.730519+0000","last_fullsized":"2024-07-17T08:36:35.730519+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:53.122471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405657+0000","last_change":"2024-07-17T08:36:34.762727+0000","last_active":"2024-07-17T08:36:36.405657+0000","last_peered":"2024-07-17T08:36:36.405657+0000","last_clean":"2024-07-17T08:36:36.405657+0000","last_became_active":"2024-07-17T08:36:34.762396+0000","last_became_peered":"2024-07-17T08:36:34.762396+0000","last_unstale":"2024-07-17T08:36:36.405657+0000","last_undegraded":"2024-07-17T08:36:36.405657+0000","last_fullsized":"2024-07-17T08:36:36.405657+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:40:10.019506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735410+0000","last_change":"2024-07-17T08:36:22.608501+0000","last_active":"2024-07-17T08:36:35.735410+0000","last_peered":"2024-07-17T08:36:35.735410+0000","last_clean":"2024-07-17T08:36:35.735410+0000","last_became_active":"2024-07-17T08:36:22.607712+0000","last_became_peered":"2024-07-17T08:36:22.607712+0000","last_unstale":"2024-07-17T08:36:35.735410+0000","last_undegraded":"2024-07-17T08:36:35.735410+0000","last_fullsized":"2024-07-17T08:36:35.735410+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:36.036883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560433+0000","last_change":"2024-07-17T08:36:21.611765+0000","last_active":"2024-07-17T08:36:36.560433+0000","last_peered":"2024-07-17T08:36:36.560433+0000","last_clean":"2024-07-17T08:36:36.560433+0000","last_became_active":"2024-07-17T08:36:21.611464+0000","last_became_peered":"2024-07-17T08:36:21.611464+0000","last_unstale":"2024-07-17T08:36:36.560433+0000","last_undegraded":"2024-07-17T08:36:36.560433+0000","last_fullsized":"2024-07-17T08:36:36.560433+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:48.270950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558118+0000","last_change":"2024-07-17T08:36:34.757902+0000","last_active":"2024-07-17T08:36:36.558118+0000","last_peered":"2024-07-17T08:36:36.558118+0000","last_clean":"2024-07-17T08:36:36.558118+0000","last_became_active":"2024-07-17T08:36:34.756374+0000","last_became_peered":"2024-07-17T08:36:34.756374+0000","last_unstale":"2024-07-17T08:36:36.558118+0000","last_undegraded":"2024-07-17T08:36:36.558118+0000","last_fullsized":"2024-07-17T08:36:36.558118+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:04:56.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734799+0000","last_change":"2024-07-17T08:36:22.613967+0000","last_active":"2024-07-17T08:36:35.734799+0000","last_peered":"2024-07-17T08:36:35.734799+0000","last_clean":"2024-07-17T08:36:35.734799+0000","last_became_active":"2024-07-17T08:36:22.613378+0000","last_became_peered":"2024-07-17T08:36:22.613378+0000","last_unstale":"2024-07-17T08:36:35.734799+0000","last_undegraded":"2024-07-17T08:36:35.734799+0000","last_fullsized":"2024-07-17T08:36:35.734799+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:13.970838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.733076+0000","last_change":"2024-07-17T08:36:21.614892+0000","last_active":"2024-07-17T08:36:35.733076+0000","last_peered":"2024-07-17T08:36:35.733076+0000","last_clean":"2024-07-17T08:36:35.733076+0000","last_became_active":"2024-07-17T08:36:21.614398+0000","last_became_peered":"2024-07-17T08:36:21.614398+0000","last_unstale":"2024-07-17T08:36:35.733076+0000","last_undegraded":"2024-07-17T08:36:35.733076+0000","last_fullsized":"2024-07-17T08:36:35.733076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:14:16.058348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735096+0000","last_change":"2024-07-17T08:36:34.759035+0000","last_active":"2024-07-17T08:36:35.735096+0000","last_peered":"2024-07-17T08:36:35.735096+0000","last_clean":"2024-07-17T08:36:35.735096+0000","last_became_active":"2024-07-17T08:36:34.758729+0000","last_became_peered":"2024-07-17T08:36:34.758729+0000","last_unstale":"2024-07-17T08:36:35.735096+0000","last_undegraded":"2024-07-17T08:36:35.735096+0000","last_fullsized":"2024-07-17T08:36:35.735096+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:43:09.694236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.730514+0000","last_change":"2024-07-17T08:36:22.609333+0000","last_active":"2024-07-17T08:36:35.730514+0000","last_peered":"2024-07-17T08:36:35.730514+0000","last_clean":"2024-07-17T08:36:35.730514+0000","last_became_active":"2024-07-17T08:36:22.608260+0000","last_became_peered":"2024-07-17T08:36:22.608260+0000","last_unstale":"2024-07-17T08:36:35.730514+0000","last_undegraded":"2024-07-17T08:36:35.730514+0000","last_fullsized":"2024-07-17T08:36:35.730514+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:50.594772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561013+0000","last_change":"2024-07-17T08:36:21.612976+0000","last_active":"2024-07-17T08:36:36.561013+0000","last_peered":"2024-07-17T08:36:36.561013+0000","last_clean":"2024-07-17T08:36:36.561013+0000","last_became_active":"2024-07-17T08:36:21.610553+0000","last_became_peered":"2024-07-17T08:36:21.610553+0000","last_unstale":"2024-07-17T08:36:36.561013+0000","last_undegraded":"2024-07-17T08:36:36.561013+0000","last_fullsized":"2024-07-17T08:36:36.561013+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:55.549639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.404661+0000","last_change":"2024-07-17T08:36:34.767357+0000","last_active":"2024-07-17T08:36:36.404661+0000","last_peered":"2024-07-17T08:36:36.404661+0000","last_clean":"2024-07-17T08:36:36.404661+0000","last_became_active":"2024-07-17T08:36:34.766645+0000","last_became_peered":"2024-07-17T08:36:34.766645+0000","last_unstale":"2024-07-17T08:36:36.404661+0000","last_undegraded":"2024-07-17T08:36:36.404661+0000","last_fullsized":"2024-07-17T08:36:36.404661+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:40.714568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735996+0000","last_change":"2024-07-17T08:36:22.635021+0000","last_active":"2024-07-17T08:36:35.735996+0000","last_peered":"2024-07-17T08:36:35.735996+0000","last_clean":"2024-07-17T08:36:35.735996+0000","last_became_active":"2024-07-17T08:36:22.633066+0000","last_became_peered":"2024-07-17T08:36:22.633066+0000","last_unstale":"2024-07-17T08:36:35.735996+0000","last_undegraded":"2024-07-17T08:36:35.735996+0000","last_fullsized":"2024-07-17T08:36:35.735996+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:23.453272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743773+0000","last_change":"2024-07-17T08:36:21.622354+0000","last_active":"2024-07-17T08:36:35.743773+0000","last_peered":"2024-07-17T08:36:35.743773+0000","last_clean":"2024-07-17T08:36:35.743773+0000","last_became_active":"2024-07-17T08:36:21.622074+0000","last_became_peered":"2024-07-17T08:36:21.622074+0000","last_unstale":"2024-07-17T08:36:35.743773+0000","last_undegraded":"2024-07-17T08:36:35.743773+0000","last_fullsized":"2024-07-17T08:36:35.743773+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:19.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738001+0000","last_change":"2024-07-17T08:36:34.751102+0000","last_active":"2024-07-17T08:36:35.738001+0000","last_peered":"2024-07-17T08:36:35.738001+0000","last_clean":"2024-07-17T08:36:35.738001+0000","last_became_active":"2024-07-17T08:36:34.750817+0000","last_became_peered":"2024-07-17T08:36:34.750817+0000","last_unstale":"2024-07-17T08:36:35.738001+0000","last_undegraded":"2024-07-17T08:36:35.738001+0000","last_fullsized":"2024-07-17T08:36:35.738001+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:07.007824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735924+0000","last_change":"2024-07-17T08:36:22.634541+0000","last_active":"2024-07-17T08:36:35.735924+0000","last_peered":"2024-07-17T08:36:35.735924+0000","last_clean":"2024-07-17T08:36:35.735924+0000","last_became_active":"2024-07-17T08:36:22.634192+0000","last_became_peered":"2024-07-17T08:36:22.634192+0000","last_unstale":"2024-07-17T08:36:35.735924+0000","last_undegraded":"2024-07-17T08:36:35.735924+0000","last_fullsized":"2024-07-17T08:36:35.735924+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:16.715631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736034+0000","last_change":"2024-07-17T08:36:21.621397+0000","last_active":"2024-07-17T08:36:35.736034+0000","last_peered":"2024-07-17T08:36:35.736034+0000","last_clean":"2024-07-17T08:36:35.736034+0000","last_became_active":"2024-07-17T08:36:21.621108+0000","last_became_peered":"2024-07-17T08:36:21.621108+0000","last_unstale":"2024-07-17T08:36:35.736034+0000","last_undegraded":"2024-07-17T08:36:35.736034+0000","last_fullsized":"2024-07-17T08:36:35.736034+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:55.450846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743869+0000","last_change":"2024-07-17T08:36:34.762388+0000","last_active":"2024-07-17T08:36:35.743869+0000","last_peered":"2024-07-17T08:36:35.743869+0000","last_clean":"2024-07-17T08:36:35.743869+0000","last_became_active":"2024-07-17T08:36:34.762062+0000","last_became_peered":"2024-07-17T08:36:34.762062+0000","last_unstale":"2024-07-17T08:36:35.743869+0000","last_undegraded":"2024-07-17T08:36:35.743869+0000","last_fullsized":"2024-07-17T08:36:35.743869+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:31.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.731483+0000","last_change":"2024-07-17T08:36:22.639694+0000","last_active":"2024-07-17T08:36:35.731483+0000","last_peered":"2024-07-17T08:36:35.731483+0000","last_clean":"2024-07-17T08:36:35.731483+0000","last_became_active":"2024-07-17T08:36:22.639260+0000","last_became_peered":"2024-07-17T08:36:22.639260+0000","last_unstale":"2024-07-17T08:36:35.731483+0000","last_undegraded":"2024-07-17T08:36:35.731483+0000","last_fullsized":"2024-07-17T08:36:35.731483+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:37.963821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559699+0000","last_change":"2024-07-17T08:36:21.619736+0000","last_active":"2024-07-17T08:36:36.559699+0000","last_peered":"2024-07-17T08:36:36.559699+0000","last_clean":"2024-07-17T08:36:36.559699+0000","last_became_active":"2024-07-17T08:36:21.615843+0000","last_became_peered":"2024-07-17T08:36:21.615843+0000","last_unstale":"2024-07-17T08:36:36.559699+0000","last_undegraded":"2024-07-17T08:36:36.559699+0000","last_fullsized":"2024-07-17T08:36:36.559699+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:19.891228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.743167+0000","last_change":"2024-07-17T08:36:34.758037+0000","last_active":"2024-07-17T08:36:35.743167+0000","last_peered":"2024-07-17T08:36:35.743167+0000","last_clean":"2024-07-17T08:36:35.743167+0000","last_became_active":"2024-07-17T08:36:34.757760+0000","last_became_peered":"2024-07-17T08:36:34.757760+0000","last_unstale":"2024-07-17T08:36:35.743167+0000","last_undegraded":"2024-07-17T08:36:35.743167+0000","last_fullsized":"2024-07-17T08:36:35.743167+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:56.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560149+0000","last_change":"2024-07-17T08:36:22.622881+0000","last_active":"2024-07-17T08:36:36.560149+0000","last_peered":"2024-07-17T08:36:36.560149+0000","last_clean":"2024-07-17T08:36:36.560149+0000","last_became_active":"2024-07-17T08:36:22.622539+0000","last_became_peered":"2024-07-17T08:36:22.622539+0000","last_unstale":"2024-07-17T08:36:36.560149+0000","last_undegraded":"2024-07-17T08:36:36.560149+0000","last_fullsized":"2024-07-17T08:36:36.560149+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:17:50.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558622+0000","last_change":"2024-07-17T08:36:21.610743+0000","last_active":"2024-07-17T08:36:36.558622+0000","last_peered":"2024-07-17T08:36:36.558622+0000","last_clean":"2024-07-17T08:36:36.558622+0000","last_became_active":"2024-07-17T08:36:21.610311+0000","last_became_peered":"2024-07-17T08:36:21.610311+0000","last_unstale":"2024-07-17T08:36:36.558622+0000","last_undegraded":"2024-07-17T08:36:36.558622+0000","last_fullsized":"2024-07-17T08:36:36.558622+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:52.192504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559072+0000","last_change":"2024-07-17T08:36:34.769778+0000","last_active":"2024-07-17T08:36:36.559072+0000","last_peered":"2024-07-17T08:36:36.559072+0000","last_clean":"2024-07-17T08:36:36.559072+0000","last_became_active":"2024-07-17T08:36:34.767090+0000","last_became_peered":"2024-07-17T08:36:34.767090+0000","last_unstale":"2024-07-17T08:36:36.559072+0000","last_undegraded":"2024-07-17T08:36:36.559072+0000","last_fullsized":"2024-07-17T08:36:36.559072+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:34.871413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561823+0000","last_change":"2024-07-17T08:36:22.621339+0000","last_active":"2024-07-17T08:36:36.561823+0000","last_peered":"2024-07-17T08:36:36.561823+0000","last_clean":"2024-07-17T08:36:36.561823+0000","last_became_active":"2024-07-17T08:36:22.620185+0000","last_became_peered":"2024-07-17T08:36:22.620185+0000","last_unstale":"2024-07-17T08:36:36.561823+0000","last_undegraded":"2024-07-17T08:36:36.561823+0000","last_fullsized":"2024-07-17T08:36:36.561823+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:46:28.313181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405097+0000","last_change":"2024-07-17T08:36:21.620288+0000","last_active":"2024-07-17T08:36:36.405097+0000","last_peered":"2024-07-17T08:36:36.405097+0000","last_clean":"2024-07-17T08:36:36.405097+0000","last_became_active":"2024-07-17T08:36:21.620007+0000","last_became_peered":"2024-07-17T08:36:21.620007+0000","last_unstale":"2024-07-17T08:36:36.405097+0000","last_undegraded":"2024-07-17T08:36:36.405097+0000","last_fullsized":"2024-07-17T08:36:36.405097+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:38.002458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741921+0000","last_change":"2024-07-17T08:36:34.764621+0000","last_active":"2024-07-17T08:36:35.741921+0000","last_peered":"2024-07-17T08:36:35.741921+0000","last_clean":"2024-07-17T08:36:35.741921+0000","last_became_active":"2024-07-17T08:36:34.764293+0000","last_became_peered":"2024-07-17T08:36:34.764293+0000","last_unstale":"2024-07-17T08:36:35.741921+0000","last_undegraded":"2024-07-17T08:36:35.741921+0000","last_fullsized":"2024-07-17T08:36:35.741921+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:32.986386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558641+0000","last_change":"2024-07-17T08:36:22.628977+0000","last_active":"2024-07-17T08:36:36.558641+0000","last_peered":"2024-07-17T08:36:36.558641+0000","last_clean":"2024-07-17T08:36:36.558641+0000","last_became_active":"2024-07-17T08:36:22.628557+0000","last_became_peered":"2024-07-17T08:36:22.628557+0000","last_unstale":"2024-07-17T08:36:36.558641+0000","last_undegraded":"2024-07-17T08:36:36.558641+0000","last_fullsized":"2024-07-17T08:36:36.558641+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:54.873528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"242'2","reported_seq":24,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736688+0000","last_change":"2024-07-17T08:36:21.625092+0000","last_active":"2024-07-17T08:36:35.736688+0000","last_peered":"2024-07-17T08:36:35.736688+0000","last_clean":"2024-07-17T08:36:35.736688+0000","last_became_active":"2024-07-17T08:36:21.624606+0000","last_became_peered":"2024-07-17T08:36:21.624606+0000","last_unstale":"2024-07-17T08:36:35.736688+0000","last_undegraded":"2024-07-17T08:36:35.736688+0000","last_fullsized":"2024-07-17T08:36:35.736688+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:37:22.483253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735803+0000","last_change":"2024-07-17T08:36:34.749678+0000","last_active":"2024-07-17T08:36:35.735803+0000","last_peered":"2024-07-17T08:36:35.735803+0000","last_clean":"2024-07-17T08:36:35.735803+0000","last_became_active":"2024-07-17T08:36:34.749045+0000","last_became_peered":"2024-07-17T08:36:34.749045+0000","last_unstale":"2024-07-17T08:36:35.735803+0000","last_undegraded":"2024-07-17T08:36:35.735803+0000","last_fullsized":"2024-07-17T08:36:35.735803+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:58.763986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741764+0000","last_change":"2024-07-17T08:36:22.626750+0000","last_active":"2024-07-17T08:36:35.741764+0000","last_peered":"2024-07-17T08:36:35.741764+0000","last_clean":"2024-07-17T08:36:35.741764+0000","last_became_active":"2024-07-17T08:36:22.626471+0000","last_became_peered":"2024-07-17T08:36:22.626471+0000","last_unstale":"2024-07-17T08:36:35.741764+0000","last_undegraded":"2024-07-17T08:36:35.741764+0000","last_fullsized":"2024-07-17T08:36:35.741764+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:48.803678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"243'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.738256+0000","last_change":"2024-07-17T08:36:21.612266+0000","last_active":"2024-07-17T08:36:35.738256+0000","last_peered":"2024-07-17T08:36:35.738256+0000","last_clean":"2024-07-17T08:36:35.738256+0000","last_became_active":"2024-07-17T08:36:21.611949+0000","last_became_peered":"2024-07-17T08:36:21.611949+0000","last_unstale":"2024-07-17T08:36:35.738256+0000","last_undegraded":"2024-07-17T08:36:35.738256+0000","last_fullsized":"2024-07-17T08:36:35.738256+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:29:07.234400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.557817+0000","last_change":"2024-07-17T08:36:34.739794+0000","last_active":"2024-07-17T08:36:36.557817+0000","last_peered":"2024-07-17T08:36:36.557817+0000","last_clean":"2024-07-17T08:36:36.557817+0000","last_became_active":"2024-07-17T08:36:34.739282+0000","last_became_peered":"2024-07-17T08:36:34.739282+0000","last_unstale":"2024-07-17T08:36:36.557817+0000","last_undegraded":"2024-07-17T08:36:36.557817+0000","last_fullsized":"2024-07-17T08:36:36.557817+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:18.364494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.735138+0000","last_change":"2024-07-17T08:36:22.643008+0000","last_active":"2024-07-17T08:36:35.735138+0000","last_peered":"2024-07-17T08:36:35.735138+0000","last_clean":"2024-07-17T08:36:35.735138+0000","last_became_active":"2024-07-17T08:36:22.642495+0000","last_became_peered":"2024-07-17T08:36:22.642495+0000","last_unstale":"2024-07-17T08:36:35.735138+0000","last_undegraded":"2024-07-17T08:36:35.735138+0000","last_fullsized":"2024-07-17T08:36:35.735138+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:39.008924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406502+0000","last_change":"2024-07-17T08:36:21.614139+0000","last_active":"2024-07-17T08:36:36.406502+0000","last_peered":"2024-07-17T08:36:36.406502+0000","last_clean":"2024-07-17T08:36:36.406502+0000","last_became_active":"2024-07-17T08:36:21.612452+0000","last_became_peered":"2024-07-17T08:36:21.612452+0000","last_unstale":"2024-07-17T08:36:36.406502+0000","last_undegraded":"2024-07-17T08:36:36.406502+0000","last_fullsized":"2024-07-17T08:36:36.406502+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:28.440362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559322+0000","last_change":"2024-07-17T08:36:34.757365+0000","last_active":"2024-07-17T08:36:36.559322+0000","last_peered":"2024-07-17T08:36:36.559322+0000","last_clean":"2024-07-17T08:36:36.559322+0000","last_became_active":"2024-07-17T08:36:34.757077+0000","last_became_peered":"2024-07-17T08:36:34.757077+0000","last_unstale":"2024-07-17T08:36:36.559322+0000","last_undegraded":"2024-07-17T08:36:36.559322+0000","last_fullsized":"2024-07-17T08:36:36.559322+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:21:20.498778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736547+0000","last_change":"2024-07-17T08:36:22.608254+0000","last_active":"2024-07-17T08:36:35.736547+0000","last_peered":"2024-07-17T08:36:35.736547+0000","last_clean":"2024-07-17T08:36:35.736547+0000","last_became_active":"2024-07-17T08:36:22.607420+0000","last_became_peered":"2024-07-17T08:36:22.607420+0000","last_unstale":"2024-07-17T08:36:35.736547+0000","last_undegraded":"2024-07-17T08:36:35.736547+0000","last_fullsized":"2024-07-17T08:36:35.736547+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:35:01.930456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406872+0000","last_change":"2024-07-17T08:36:21.618840+0000","last_active":"2024-07-17T08:36:36.406872+0000","last_peered":"2024-07-17T08:36:36.406872+0000","last_clean":"2024-07-17T08:36:36.406872+0000","last_became_active":"2024-07-17T08:36:21.618526+0000","last_became_peered":"2024-07-17T08:36:21.618526+0000","last_unstale":"2024-07-17T08:36:36.406872+0000","last_undegraded":"2024-07-17T08:36:36.406872+0000","last_fullsized":"2024-07-17T08:36:36.406872+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:07.391994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.406507+0000","last_change":"2024-07-17T08:36:34.772879+0000","last_active":"2024-07-17T08:36:36.406507+0000","last_peered":"2024-07-17T08:36:36.406507+0000","last_clean":"2024-07-17T08:36:36.406507+0000","last_became_active":"2024-07-17T08:36:34.772086+0000","last_became_peered":"2024-07-17T08:36:34.772086+0000","last_unstale":"2024-07-17T08:36:36.406507+0000","last_undegraded":"2024-07-17T08:36:36.406507+0000","last_fullsized":"2024-07-17T08:36:36.406507+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:17.960713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.405403+0000","last_change":"2024-07-17T08:36:22.632310+0000","last_active":"2024-07-17T08:36:36.405403+0000","last_peered":"2024-07-17T08:36:36.405403+0000","last_clean":"2024-07-17T08:36:36.405403+0000","last_became_active":"2024-07-17T08:36:22.632016+0000","last_became_peered":"2024-07-17T08:36:22.632016+0000","last_unstale":"2024-07-17T08:36:36.405403+0000","last_undegraded":"2024-07-17T08:36:36.405403+0000","last_fullsized":"2024-07-17T08:36:36.405403+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:08.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560972+0000","last_change":"2024-07-17T08:36:21.617106+0000","last_active":"2024-07-17T08:36:36.560972+0000","last_peered":"2024-07-17T08:36:36.560972+0000","last_clean":"2024-07-17T08:36:36.560972+0000","last_became_active":"2024-07-17T08:36:21.615901+0000","last_became_peered":"2024-07-17T08:36:21.615901+0000","last_unstale":"2024-07-17T08:36:36.560972+0000","last_undegraded":"2024-07-17T08:36:36.560972+0000","last_fullsized":"2024-07-17T08:36:36.560972+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:36.957355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561083+0000","last_change":"2024-07-17T08:36:34.772407+0000","last_active":"2024-07-17T08:36:36.561083+0000","last_peered":"2024-07-17T08:36:36.561083+0000","last_clean":"2024-07-17T08:36:36.561083+0000","last_became_active":"2024-07-17T08:36:34.772076+0000","last_became_peered":"2024-07-17T08:36:34.772076+0000","last_unstale":"2024-07-17T08:36:36.561083+0000","last_undegraded":"2024-07-17T08:36:36.561083+0000","last_fullsized":"2024-07-17T08:36:36.561083+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:47:29.825386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.559753+0000","last_change":"2024-07-17T08:36:22.627923+0000","last_active":"2024-07-17T08:36:36.559753+0000","last_peered":"2024-07-17T08:36:36.559753+0000","last_clean":"2024-07-17T08:36:36.559753+0000","last_became_active":"2024-07-17T08:36:22.622868+0000","last_became_peered":"2024-07-17T08:36:22.622868+0000","last_unstale":"2024-07-17T08:36:36.559753+0000","last_undegraded":"2024-07-17T08:36:36.559753+0000","last_fullsized":"2024-07-17T08:36:36.559753+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:40.727739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.736187+0000","last_change":"2024-07-17T08:36:21.614104+0000","last_active":"2024-07-17T08:36:35.736187+0000","last_peered":"2024-07-17T08:36:35.736187+0000","last_clean":"2024-07-17T08:36:35.736187+0000","last_became_active":"2024-07-17T08:36:21.610791+0000","last_became_peered":"2024-07-17T08:36:21.610791+0000","last_unstale":"2024-07-17T08:36:35.736187+0000","last_undegraded":"2024-07-17T08:36:35.736187+0000","last_fullsized":"2024-07-17T08:36:35.736187+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:16.671840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558755+0000","last_change":"2024-07-17T08:36:34.753399+0000","last_active":"2024-07-17T08:36:36.558755+0000","last_peered":"2024-07-17T08:36:36.558755+0000","last_clean":"2024-07-17T08:36:36.558755+0000","last_became_active":"2024-07-17T08:36:34.753124+0000","last_became_peered":"2024-07-17T08:36:34.753124+0000","last_unstale":"2024-07-17T08:36:36.558755+0000","last_undegraded":"2024-07-17T08:36:36.558755+0000","last_fullsized":"2024-07-17T08:36:36.558755+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:20:19.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.732588+0000","last_change":"2024-07-17T08:36:22.638056+0000","last_active":"2024-07-17T08:36:35.732588+0000","last_peered":"2024-07-17T08:36:35.732588+0000","last_clean":"2024-07-17T08:36:35.732588+0000","last_became_active":"2024-07-17T08:36:22.637567+0000","last_became_peered":"2024-07-17T08:36:22.637567+0000","last_unstale":"2024-07-17T08:36:35.732588+0000","last_undegraded":"2024-07-17T08:36:35.732588+0000","last_fullsized":"2024-07-17T08:36:35.732588+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:34.565758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.734906+0000","last_change":"2024-07-17T08:36:21.618627+0000","last_active":"2024-07-17T08:36:35.734906+0000","last_peered":"2024-07-17T08:36:35.734906+0000","last_clean":"2024-07-17T08:36:35.734906+0000","last_became_active":"2024-07-17T08:36:21.618107+0000","last_became_peered":"2024-07-17T08:36:21.618107+0000","last_unstale":"2024-07-17T08:36:35.734906+0000","last_undegraded":"2024-07-17T08:36:35.734906+0000","last_fullsized":"2024-07-17T08:36:35.734906+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:53.867062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.560794+0000","last_change":"2024-07-17T08:36:34.774271+0000","last_active":"2024-07-17T08:36:36.560794+0000","last_peered":"2024-07-17T08:36:36.560794+0000","last_clean":"2024-07-17T08:36:36.560794+0000","last_became_active":"2024-07-17T08:36:34.773736+0000","last_became_peered":"2024-07-17T08:36:34.773736+0000","last_unstale":"2024-07-17T08:36:36.560794+0000","last_undegraded":"2024-07-17T08:36:36.560794+0000","last_fullsized":"2024-07-17T08:36:36.560794+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:03.362787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561767+0000","last_change":"2024-07-17T08:36:22.640373+0000","last_active":"2024-07-17T08:36:36.561767+0000","last_peered":"2024-07-17T08:36:36.561767+0000","last_clean":"2024-07-17T08:36:36.561767+0000","last_became_active":"2024-07-17T08:36:22.639197+0000","last_became_peered":"2024-07-17T08:36:22.639197+0000","last_unstale":"2024-07-17T08:36:36.561767+0000","last_undegraded":"2024-07-17T08:36:36.561767+0000","last_fullsized":"2024-07-17T08:36:36.561767+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:50.774245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":22,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558115+0000","last_change":"2024-07-17T08:36:21.612719+0000","last_active":"2024-07-17T08:36:36.558115+0000","last_peered":"2024-07-17T08:36:36.558115+0000","last_clean":"2024-07-17T08:36:36.558115+0000","last_became_active":"2024-07-17T08:36:21.612396+0000","last_became_peered":"2024-07-17T08:36:21.612396+0000","last_unstale":"2024-07-17T08:36:36.558115+0000","last_undegraded":"2024-07-17T08:36:36.558115+0000","last_fullsized":"2024-07-17T08:36:36.558115+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:37.029038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558684+0000","last_change":"2024-07-17T08:36:34.758236+0000","last_active":"2024-07-17T08:36:36.558684+0000","last_peered":"2024-07-17T08:36:36.558684+0000","last_clean":"2024-07-17T08:36:36.558684+0000","last_became_active":"2024-07-17T08:36:34.756897+0000","last_became_peered":"2024-07-17T08:36:34.756897+0000","last_unstale":"2024-07-17T08:36:36.558684+0000","last_undegraded":"2024-07-17T08:36:36.558684+0000","last_fullsized":"2024-07-17T08:36:36.558684+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":245,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:33.701795+0000","last_clean_scrub_stamp":"2024-07-17T08:36:33.701795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:11.720969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":20,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.558229+0000","last_change":"2024-07-17T08:36:22.619369+0000","last_active":"2024-07-17T08:36:36.558229+0000","last_peered":"2024-07-17T08:36:36.558229+0000","last_clean":"2024-07-17T08:36:36.558229+0000","last_became_active":"2024-07-17T08:36:22.619052+0000","last_became_peered":"2024-07-17T08:36:22.619052+0000","last_unstale":"2024-07-17T08:36:36.558229+0000","last_undegraded":"2024-07-17T08:36:36.558229+0000","last_fullsized":"2024-07-17T08:36:36.558229+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":241,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:21.567500+0000","last_clean_scrub_stamp":"2024-07-17T08:36:21.567500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:20.383796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"242'1","reported_seq":23,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:36.561312+0000","last_change":"2024-07-17T08:36:21.618154+0000","last_active":"2024-07-17T08:36:36.561312+0000","last_peered":"2024-07-17T08:36:36.561312+0000","last_clean":"2024-07-17T08:36:36.561312+0000","last_became_active":"2024-07-17T08:36:21.617353+0000","last_became_peered":"2024-07-17T08:36:21.617353+0000","last_unstale":"2024-07-17T08:36:36.561312+0000","last_undegraded":"2024-07-17T08:36:36.561312+0000","last_fullsized":"2024-07-17T08:36:36.561312+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:36:20.561033+0000","last_clean_scrub_stamp":"2024-07-17T08:36:20.561033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:03:21.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"230'2525","reported_seq":3475,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-07-17T08:36:35.741587+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:36:35.741587+0000","last_peered":"2024-07-17T08:36:35.741587+0000","last_clean":"2024-07-17T08:36:35.741587+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:36:35.741587+0000","last_undegraded":"2024-07-17T08:36:35.741587+0000","last_fullsized":"2024-07-17T08:36:35.741587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3656,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1507,"num_read_kb":4223,"num_write":383,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2525,"ondisk_log_size":2525,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543356,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48704,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323136,"statfs":{"total":96636764160,"available":96586891264,"internally_reserved":0,"allocated":3723264,"data_stored":2296735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510651,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48124,"kb_used_data":3052,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323716,"statfs":{"total":96636764160,"available":96587485184,"internally_reserved":0,"allocated":3125248,"data_stored":1705785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510652,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52276,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319564,"statfs":{"total":96636764160,"available":96583233536,"internally_reserved":0,"allocated":3117056,"data_stored":1705709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543357,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52284,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3129344,"data_stored":1707321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510651,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52296,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319544,"statfs":{"total":96636764160,"available":96583213056,"internally_reserved":0,"allocated":3141632,"data_stored":1709225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543355,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52300,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319540,"statfs":{"total":96636764160,"available":96583208960,"internally_reserved":0,"allocated":3141632,"data_stored":1708507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510652,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48124,"kb_used_data":3048,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323716,"statfs":{"total":96636764160,"available":96587485184,"internally_reserved":0,"allocated":3121152,"data_stored":1705763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510651,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57024,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314816,"statfs":{"total":96636764160,"available":96578371584,"internally_reserved":0,"allocated":3723264,"data_stored":2298935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:36:44.951 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:36:44.952 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 -- 172.21.15.44:0/1511003875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 msgr2=0x7fae0c06f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 --2- 172.21.15.44:0/1511003875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c06f4a0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005fd0 tx=0x7fadf402b910 comp rx=0 tx=0).stop 2024-07-17T08:36:45.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 -- 172.21.15.44:0/1511003875 shutdown_connections 2024-07-17T08:36:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 --2- 172.21.15.44:0/1511003875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 --2- 172.21.15.44:0/1511003875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c06f4a0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 -- 172.21.15.44:0/1511003875 >> 172.21.15.44:0/1511003875 conn(0x7fae0c0b0fb0 msgr2=0x7fae0c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 -- 172.21.15.44:0/1511003875 shutdown_connections 2024-07-17T08:36:45.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.198+0000 7fae12a00640 1 -- 172.21.15.44:0/1511003875 wait complete. 2024-07-17T08:36:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.200+0000 7fae12a00640 1 Processor -- start 2024-07-17T08:36:45.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.200+0000 7fae12a00640 1 -- start start 2024-07-17T08:36:45.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:45.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:45.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 0x7fae0c071b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:45.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c06d800 con 0x7fae0c0acc30 2024-07-17T08:36:45.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c06d680 con 0x7fae0c159910 2024-07-17T08:36:45.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.201+0000 7fae12a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c06d980 con 0x7fae0c0b3280 2024-07-17T08:36:45.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:45.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54242/0 (socket says 172.21.15.44:54242) 2024-07-17T08:36:45.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 -- 172.21.15.44:0/2687072184 learned_addr learned my addr 172.21.15.44:0/2687072184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae03400640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae10a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 0x7fae0c071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 -- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 msgr2=0x7fae0c071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 -- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 msgr2=0x7fae0c071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 0x7fae0c071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae0be00640 1 -- 172.21.15.44:0/2687072184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c072f40 con 0x7fae0c0acc30 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae03400640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.202+0000 7fae10a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 0x7fae0c071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae0be00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c3a0 tx=0x7fadfc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc017070 con 0x7fae0c0acc30 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadfc0053c0 con 0x7fae0c0acc30 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc003850 con 0x7fae0c0acc30 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c0731c0 con 0x7fae0c0acc30 2024-07-17T08:36:45.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.203+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c073780 con 0x7fae0c0acc30 2024-07-17T08:36:45.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.204+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c06f4a0 con 0x7fae0c0acc30 2024-07-17T08:36:45.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.206+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fadfc01a070 con 0x7fae0c0acc30 2024-07-17T08:36:45.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.207+0000 7fae08c00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 0x7fade80792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:45.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.207+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fadfc098100 con 0x7fae0c0acc30 2024-07-17T08:36:45.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.207+0000 7fae03400640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 0x7fade80792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:45.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.208+0000 7fae03400640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 0x7fade80792e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fadf4035000 tx=0x7fadf4004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:45.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.211+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fae0c072f40 con 0x7fae0c0acc30 2024-07-17T08:36:45.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.460+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae0c0b27a0 con 0x7fae0c0acc30 2024-07-17T08:36:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.462+0000 7fae08c00640 1 -- 172.21.15.44:0/2687072184 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7fadfc060c00 con 0x7fae0c0acc30 2024-07-17T08:36:45.464 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:45.464 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:45.464 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.464+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 msgr2=0x7fade80792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.464+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 0x7fade80792e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fadf4035000 tx=0x7fadf4004730 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 msgr2=0x7fae0c14f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c3a0 tx=0x7fadfc00c860 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 shutdown_connections 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fade8076e20 0x7fade80792e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae0c159910 0x7fae0c071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0c0b3280 0x7fae0c071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 --2- 172.21.15.44:0/2687072184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae0c0acc30 0x7fae0c14f650 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 >> 172.21.15.44:0/2687072184 conn(0x7fae0c0b0fb0 msgr2=0x7fae0c06dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 shutdown_connections 2024-07-17T08:36:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:45.465+0000 7fae12a00640 1 -- 172.21.15.44:0/2687072184 wait complete. 2024-07-17T08:36:45.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 135, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:36:34.218363+0000', 'modified': '2024-07-17T08:36:37.261234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8785}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8785': {'gid': 8785, 'name': 'b', 'rank': 0, 'incarnation': 132, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.175:6801/3620068660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 3620068660}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 3620068660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8785, 'qdb_cluster': [8785]} 2024-07-17T08:36:45.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:36:45.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.118+0000 7fb712c00640 1 -- 172.21.15.44:0/2550357789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c06f4d0 msgr2=0x7fb70c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.118+0000 7fb712c00640 1 --2- 172.21.15.44:0/2550357789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c06f4d0 0x7fb70c15a450 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0b2560 tx=0x7fb70002b860 comp rx=0 tx=0).stop 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 -- 172.21.15.44:0/2550357789 shutdown_connections 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 --2- 172.21.15.44:0/2550357789 >> v2:172.21.15.175:3300/0 conn(0x7fb70c06d420 0x7fb70c06d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 --2- 172.21.15.44:0/2550357789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c06f4d0 0x7fb70c15a450 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 -- 172.21.15.44:0/2550357789 >> 172.21.15.44:0/2550357789 conn(0x7fb70c0cf0d0 msgr2=0x7fb70c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 -- 172.21.15.44:0/2550357789 shutdown_connections 2024-07-17T08:36:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.119+0000 7fb712c00640 1 -- 172.21.15.44:0/2550357789 wait complete. 2024-07-17T08:36:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.120+0000 7fb712c00640 1 Processor -- start 2024-07-17T08:36:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.121+0000 7fb712c00640 1 -- start start 2024-07-17T08:36:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c06d420 0x7fb70c151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c06f4d0 0x7fb70c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0b3380 con 0x7fb70c1503c0 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0b3200 con 0x7fb70c06f4d0 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c0b3500 con 0x7fb70c06d420 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54260/0 (socket says 172.21.15.44:54260) 2024-07-17T08:36:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 -- 172.21.15.44:0/3138621550 learned_addr learned my addr 172.21.15.44:0/3138621550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb710e00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c06f4d0 0x7fb70c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb711800640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c06d420 0x7fb70c151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 -- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c06d420 msgr2=0x7fb70c151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c06d420 0x7fb70c151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 -- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c06f4d0 msgr2=0x7fb70c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c06f4d0 0x7fb70c1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.122+0000 7fb712200640 1 -- 172.21.15.44:0/3138621550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c150f90 con 0x7fb70c1503c0 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb712200640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb70400da70 tx=0x7fb70400df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb709600640 1 -- 172.21.15.44:0/3138621550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704017070 con 0x7fb70c1503c0 2024-07-17T08:36:46.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb709600640 1 -- 172.21.15.44:0/3138621550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb704003970 con 0x7fb70c1503c0 2024-07-17T08:36:46.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c151270 con 0x7fb70c1503c0 2024-07-17T08:36:46.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb709600640 1 -- 172.21.15.44:0/3138621550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704003dc0 con 0x7fb70c1503c0 2024-07-17T08:36:46.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.123+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c151830 con 0x7fb70c1503c0 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.126+0000 7fb709600640 1 -- 172.21.15.44:0/3138621550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fb704003f60 con 0x7fb70c1503c0 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.127+0000 7fb709600640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 0x7fb6ec0792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.127+0000 7fb709600640 1 -- 172.21.15.44:0/3138621550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fb7040993f0 con 0x7fb70c1503c0 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.127+0000 7fb716fe9740 1 Processor -- start 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.127+0000 7fb716fe9740 1 -- start start 2024-07-17T08:36:46.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.127+0000 7fb711800640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 0x7fb6ec0792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f707f280 0x5618f70a4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a4790 0x5618f70a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618f6fefd60 con 0x5618f70a7fc0 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5618f6fdf260 con 0x5618f707f280 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.128+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5618f6fe32a0 con 0x5618f70a4790 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54266/0 (socket says 172.21.15.44:54266) 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 -- 172.21.15.44:0/758856920 learned_addr learned my addr 172.21.15.44:0/758856920 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 0x7fb6ec0792b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb700036000 tx=0x7fb700004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 -- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a4790 msgr2=0x5618f70a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a4790 0x5618f70a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 -- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f707f280 msgr2=0x5618f70a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f707f280 0x5618f70a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 -- 172.21.15.44:0/758856920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f6d43850 con 0x5618f70a7fc0 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.129+0000 7fb711800640 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb70002be50 tx=0x7fb70003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4492d24ef44b7ff server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb6fe000640 1 -- 172.21.15.44:0/758856920 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700003e00 con 0x5618f70a7fc0 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb6fe000640 1 -- 172.21.15.44:0/758856920 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb700037050 con 0x5618f70a7fc0 2024-07-17T08:36:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb6fe000640 1 -- 172.21.15.44:0/758856920 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000318b0 con 0x5618f70a7fc0 2024-07-17T08:36:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 -- 172.21.15.44:0/758856920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 msgr2=0x5618f70a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fb70002be50 tx=0x7fb70003e000 comp rx=0 tx=0).stop 2024-07-17T08:36:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 -- 172.21.15.44:0/758856920 shutdown_connections 2024-07-17T08:36:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a4790 0x5618f70a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f707f280 0x5618f70a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 --2- 172.21.15.44:0/758856920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70a83a0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.130+0000 7fb716fe9740 1 -- 172.21.15.44:0/758856920 >> 172.21.15.44:0/758856920 conn(0x5618f707a460 msgr2=0x5618f707de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.131+0000 7fb716fe9740 1 -- 172.21.15.44:0/758856920 shutdown_connections 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.131+0000 7fb716fe9740 1 -- 172.21.15.44:0/758856920 wait complete. 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.131+0000 7fb716fe9740 1 -- start start 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.131+0000 7fb716fe9740 1 Processor -- start 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.132+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.132+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f70b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.132+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.132+0000 7fb716fe9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618f6d52620 con 0x5618f70a4790 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.132+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5618f6f86f90 con 0x5618f709d740 2024-07-17T08:36:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5618f7078b60 con 0x5618f70a7fc0 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb716fe9740 10 client.0 _ll_register_callbacks cb 0x5618f6facf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54270/0 (socket says 172.21.15.44:54270) 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 -- 172.21.15.44:0/1142107253 learned_addr learned my addr 172.21.15.44:0/1142107253 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 -- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 msgr2=0x5618f70b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f70b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 -- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 msgr2=0x5618f70a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 -- 172.21.15.44:0/1142107253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f6dba6b0 con 0x5618f70a4790 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.133+0000 7fb711800640 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fb700034040 tx=0x7fb700004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909cb5b07e483666 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700031e10 con 0x5618f70a4790 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618f6fb6900 con 0x5618f70a4790 2024-07-17T08:36:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fb700049420 con 0x5618f70a4790 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=135) ==== 24+0+0 (secure 0 0 0) 0x7fb700049620 con 0x5618f70a4790 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb716fe9740 10 client.10962 fetch_fsmap learned FSMap version 135 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=135,monmap=2+}) -- 0x5618f709e6e0 con 0x5618f70a4790 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5618f6fb6900 con 0x5618f70a4790 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.134+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 135) ==== 4149+0+0 (secure 0 0 0) 0x7fb70000b040 con 0x5618f70a4790 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 10 client.10962 fetch_fsmap finished waiting for FSMap version 135 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 10 client.10962 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 20 client.10962 populate_metadata read hostname 'smithi044' 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f70d6760 0x5618f70d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 4 client.10962 mds_command: new command op to 8785 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:36:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 --> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f6ff1cb0 con 0x5618f70d6760 2024-07-17T08:36:46.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.135+0000 7fb710e00640 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f70d6760 0x5618f70d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.136+0000 7fb710e00640 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f70d6760 0x5618f70d6b40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9668f79131096a63 server_cookie=c6294a78690fd46 in_seq=0 out_seq=0 2024-07-17T08:36:46.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.136+0000 7fb6f2a00640 10 client.10962 ms_handle_connect on v2:172.21.15.175:6800/3620068660 2024-07-17T08:36:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.139+0000 7fb6f2a00640 1 -- 172.21.15.44:0/1142107253 <== mds.0 v2:172.21.15.175:6800/3620068660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5618f6ff1cb0 con 0x5618f70d6760 2024-07-17T08:36:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.139+0000 7fb6f2a00640 10 client.10962 handle_command_reply: tid=0 2024-07-17T08:36:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.139+0000 7fb716fe9740 1 client.10962 shutdown 2024-07-17T08:36:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 msgr2=0x5618f70d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fb700034040 tx=0x7fb700004050 comp rx=0 tx=0).stop 2024-07-17T08:36:46.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 shutdown_connections 2024-07-17T08:36:46.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f70d6760 0x5618f70d6b40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f70b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 --2- 172.21.15.44:0/1142107253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f70d37c0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb6f2a00640 0 client.10962 ms_handle_reset on v2:172.21.15.175:6800/3620068660 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 >> 172.21.15.44:0/1142107253 conn(0x5618f707f280 msgr2=0x5618f707de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 shutdown_connections 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 1 -- 172.21.15.44:0/1142107253 wait complete. 2024-07-17T08:36:46.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.140+0000 7fb716fe9740 20 client.10962 trim_cache size 0 max 16384 2024-07-17T08:36:46.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.161+0000 7fb716fe9740 1 Processor -- start 2024-07-17T08:36:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.161+0000 7fb716fe9740 1 -- start start 2024-07-17T08:36:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f707e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c1ffc10 con 0x5618f70a4790 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0732a0 con 0x5618f709d740 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.162+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c1fdcf0 con 0x5618f70a7fc0 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54276/0 (socket says 172.21.15.44:54276) 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 -- 172.21.15.44:0/2900643522 learned_addr learned my addr 172.21.15.44:0/2900643522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb711800640 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb712200640 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f707e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 -- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 msgr2=0x5618f707e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f707e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 -- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 msgr2=0x5618f70eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 -- 172.21.15.44:0/2900643522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f70d4420 con 0x5618f70a4790 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.163+0000 7fb710e00640 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fb70c068340 tx=0x7fb6f80018c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7b1d871bb3f312 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb6fd600640 1 -- 172.21.15.44:0/2900643522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8005220 con 0x5618f70a4790 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb6fd600640 1 -- 172.21.15.44:0/2900643522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6f80053e0 con 0x5618f70a4790 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb6fd600640 1 -- 172.21.15.44:0/2900643522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004920 con 0x5618f70a4790 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 -- 172.21.15.44:0/2900643522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 msgr2=0x5618f707dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fb70c068340 tx=0x7fb6f80018c0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 -- 172.21.15.44:0/2900643522 shutdown_connections 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f70a7fc0 0x5618f707e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a4790 0x5618f707dc60 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 --2- 172.21.15.44:0/2900643522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f709d740 0x5618f70eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.164+0000 7fb716fe9740 1 -- 172.21.15.44:0/2900643522 >> 172.21.15.44:0/2900643522 conn(0x5618f707f280 msgr2=0x5618f70c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.165+0000 7fb716fe9740 1 -- 172.21.15.44:0/2900643522 shutdown_connections 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.165+0000 7fb716fe9740 1 -- 172.21.15.44:0/2900643522 wait complete. 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.165+0000 7fb716fe9740 1 -- start start 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.166+0000 7fb716fe9740 1 Processor -- start 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.166+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f709d740 0x5618f71453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.166+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f70a4790 0x5618f70e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.166+0000 7fb716fe9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.166+0000 7fb716fe9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5618f70b97e0 con 0x5618f70a7fc0 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5618f6d52620 con 0x5618f70a4790 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb716fe9740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5618f70d47e0 con 0x5618f709d740 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb716fe9740 10 client.0 _ll_register_callbacks cb 0x7fb6f40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:36:46.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54282/0 (socket says 172.21.15.44:54282) 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 -- 172.21.15.44:0/3226670014 learned_addr learned my addr 172.21.15.44:0/3226670014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 -- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f709d740 msgr2=0x5618f71453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f709d740 0x5618f71453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 -- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f70a4790 msgr2=0x5618f70e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f70a4790 0x5618f70e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.167+0000 7fb712200640 1 -- 172.21.15.44:0/3226670014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f70d5ea0 con 0x5618f70a7fc0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb712200640 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fb7040615a0 tx=0x7fb704061020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f90d70f5e10ba8 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704020070 con 0x5618f70a7fc0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fb704097220 con 0x5618f70a7fc0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f80053e0 con 0x5618f70a7fc0 2024-07-17T08:36:46.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=135) ==== 24+0+0 (secure 0 0 0) 0x7fb704061610 con 0x5618f70a7fc0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 10 client.10968 fetch_fsmap learned FSMap version 135 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=135,monmap=2+}) -- 0x5618f70c6da0 con 0x5618f70a7fc0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6f80053e0 con 0x5618f70a7fc0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 135) ==== 4149+0+0 (secure 0 0 0) 0x7fb704060870 con 0x5618f70a7fc0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 10 client.10968 fetch_fsmap finished waiting for FSMap version 135 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 10 client.10968 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 20 client.10968 populate_metadata read hostname 'smithi044' 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f709bbb0 0x5618f709bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 4 client.10968 mds_command: new command op to 8785 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.168+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 --> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] -- command(tid 0: {"prefix": "status"}) -- 0x5618f707e7a0 con 0x5618f709bbb0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.169+0000 7fb711800640 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f709bbb0 0x5618f709bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.170+0000 7fb711800640 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f709bbb0 0x5618f709bf90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdc0129ea3395b8 server_cookie=8e4978f8ba288c1 in_seq=0 out_seq=0 2024-07-17T08:36:46.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.170+0000 7fb6f2a00640 10 client.10968 ms_handle_connect on v2:172.21.15.175:6800/3620068660 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.170+0000 7fb6f2a00640 1 -- 172.21.15.44:0/3226670014 <== mds.0 v2:172.21.15.175:6800/3620068660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+337 (crc 0 0 0) 0x5618f707e7a0 con 0x5618f709bbb0 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb6f2a00640 10 client.10968 handle_command_reply: tid=0 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 client.10968 shutdown 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 msgr2=0x5618f70ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fb7040615a0 tx=0x7fb704061020 comp rx=0 tx=0).stop 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 shutdown_connections 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:6800/3620068660,v1:172.21.15.175:6801/3620068660] conn(0x5618f709bbb0 0x5618f709bf90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5618f70a7fc0 0x5618f70ea2c0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5618f70a4790 0x5618f70e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 --2- 172.21.15.44:0/3226670014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5618f709d740 0x5618f71453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb6f2a00640 0 client.10968 ms_handle_reset on v2:172.21.15.175:6800/3620068660 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 >> 172.21.15.44:0/3226670014 conn(0x5618f707f280 msgr2=0x5618f70bde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 shutdown_connections 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 1 -- 172.21.15.44:0/3226670014 wait complete. 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.171+0000 7fb716fe9740 20 client.10968 trim_cache size 0 max 16384 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 8785, 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:36:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "backup_fs", 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 11.445513760000001, 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 135, 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 247, 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 247, 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 28.528612170999999 2024-07-17T08:36:46.182 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 msgr2=0x7fb6ec0792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 0x7fb6ec0792b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb700036000 tx=0x7fb700004730 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 msgr2=0x7fb70c13ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fb70400da70 tx=0x7fb70400df30 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 shutdown_connections 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fb6ec076df0 0x7fb6ec0792b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb70c1503c0 0x7fb70c13ea60 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c06f4d0 0x7fb70c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 --2- 172.21.15.44:0/3138621550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c06d420 0x7fb70c151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 >> 172.21.15.44:0/3138621550 conn(0x7fb70c0cf0d0 msgr2=0x7fb70c073e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 shutdown_connections 2024-07-17T08:36:46.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.172+0000 7fb712c00640 1 -- 172.21.15.44:0/3138621550 wait complete. 2024-07-17T08:36:46.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 -- 172.21.15.44:0/3471625946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff806eb90 msgr2=0x7fcff815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 --2- 172.21.15.44:0/3471625946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff806eb90 0x7fcff815a5a0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0005fd0 tx=0x7fcfe002b910 comp rx=0 tx=0).stop 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 -- 172.21.15.44:0/3471625946 shutdown_connections 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 --2- 172.21.15.44:0/3471625946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff80ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 --2- 172.21.15.44:0/3471625946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff806eb90 0x7fcff815a5a0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 -- 172.21.15.44:0/3471625946 >> 172.21.15.44:0/3471625946 conn(0x7fcff80b0eb0 msgr2=0x7fcff80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 -- 172.21.15.44:0/3471625946 shutdown_connections 2024-07-17T08:36:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.390+0000 7fcffcc00640 1 -- 172.21.15.44:0/3471625946 wait complete. 2024-07-17T08:36:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.392+0000 7fcffcc00640 1 Processor -- start 2024-07-17T08:36:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.392+0000 7fcffcc00640 1 -- start start 2024-07-17T08:36:46.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 0x7fcff81f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff81f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcff806f850 con 0x7fcff80ac3a0 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcff806f6d0 con 0x7fcff806eb90 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcffcc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcff8150470 con 0x7fcff80b3280 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcff5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcff5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54306/0 (socket says 172.21.15.44:54306) 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.393+0000 7fcff5400640 1 -- 172.21.15.44:0/4253149030 learned_addr learned my addr 172.21.15.44:0/4253149030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 -- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 msgr2=0x7fcff81f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5e00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 0x7fcff81f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff6800640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff81f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff81f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 -- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 msgr2=0x7fcff81f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 0x7fcff81f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 -- 172.21.15.44:0/4253149030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81f1a60 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff6800640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff81f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5e00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 0x7fcff81f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcff5400640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ea20 tx=0x7fcfe800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cde0 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe8011560 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8011990 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.394+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81f7ce0 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.395+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81f82a0 con 0x7fcff80ac3a0 2024-07-17T08:36:46.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.396+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff806a1a0 con 0x7fcff80ac3a0 2024-07-17T08:36:46.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.397+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7fcfe8011b30 con 0x7fcff80ac3a0 2024-07-17T08:36:46.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.398+0000 7fcfee000640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 0x7fcfd40792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.398+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7fcfe8099c00 con 0x7fcff80ac3a0 2024-07-17T08:36:46.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.399+0000 7fcff5e00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 0x7fcfd40792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.400+0000 7fcff5e00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 0x7fcfd40792b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0035000 tx=0x7fcfe0004730 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.403+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcff806a1a0 con 0x7fcff80ac3a0 2024-07-17T08:36:46.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.650+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fcff815a5a0 con 0x7fcfd4076df0 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.654+0000 7fcfee000640 1 -- 172.21.15.44:0/4253149030 <== mgr.8593 v2:172.21.15.175:6834/189954344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+82 (secure 0 0 0) 0x7fcff815a5a0 con 0x7fcfd4076df0 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:36:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:36:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "backup_fs" 2024-07-17T08:36:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:36:46.657 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 msgr2=0x7fcfd40792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 0x7fcfd40792b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0035000 tx=0x7fcfe0004730 comp rx=0 tx=0).stop 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 msgr2=0x7fcff80cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ea20 tx=0x7fcfe800eee0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 shutdown_connections 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fcfd4076df0 0x7fcfd40792b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff80b3280 0x7fcff81f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff80ac3a0 0x7fcff80cfb40 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 --2- 172.21.15.44:0/4253149030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff806eb90 0x7fcff81f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.657+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 >> 172.21.15.44:0/4253149030 conn(0x7fcff80b0eb0 msgr2=0x7fcff815bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.658+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 shutdown_connections 2024-07-17T08:36:46.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.658+0000 7fcffcc00640 1 -- 172.21.15.44:0/4253149030 wait complete. 2024-07-17T08:36:46.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:36:46.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.886+0000 7f75e4e00640 1 -- 172.21.15.44:0/2115540089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c40062d0 msgr2=0x7f75c400cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.886+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2115540089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c40062d0 0x7f75c400cad0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f75d8005fd0 tx=0x7f75c4006940 comp rx=0 tx=0).stop 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.886+0000 7f75de800640 1 -- 172.21.15.44:0/2115540089 reap_dead start 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.887+0000 7f75e4e00640 1 -- 172.21.15.44:0/2115540089 shutdown_connections 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.887+0000 7f75e4e00640 1 -- 172.21.15.44:0/2115540089 >> 172.21.15.44:0/2115540089 conn(0x7f75e00b0eb0 msgr2=0x7f75e00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.887+0000 7f75e4e00640 1 -- 172.21.15.44:0/2115540089 shutdown_connections 2024-07-17T08:36:46.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.887+0000 7f75e4e00640 1 -- 172.21.15.44:0/2115540089 wait complete. 2024-07-17T08:36:46.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.889+0000 7f75e4e00640 1 Processor -- start 2024-07-17T08:36:46.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.889+0000 7f75e4e00640 1 -- start start 2024-07-17T08:36:46.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c40062d0 0x7f75e0074180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e0071ff0 0x7f75e01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75e00ae560 con 0x7f75e0156950 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75e015a8f0 con 0x7f75e0071ff0 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75e4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75e00ae6e0 con 0x7f75c40062d0 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75de800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54324/0 (socket says 172.21.15.44:54324) 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.890+0000 7f75de800640 1 -- 172.21.15.44:0/2119308152 learned_addr learned my addr 172.21.15.44:0/2119308152 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75dde00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c40062d0 0x7f75e0074180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75d5400640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e0071ff0 0x7f75e01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 -- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c40062d0 msgr2=0x7f75e0074180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c40062d0 0x7f75e0074180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 -- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e0071ff0 msgr2=0x7f75e01f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e0071ff0 0x7f75e01f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 -- 172.21.15.44:0/2119308152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e00b0be0 con 0x7f75e0156950 2024-07-17T08:36:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75de800640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f75d802b360 tx=0x7f75d8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8038040 con 0x7f75e0156950 2024-07-17T08:36:46.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75d8004b20 con 0x7f75e0156950 2024-07-17T08:36:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8003970 con 0x7f75e0156950 2024-07-17T08:36:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.891+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e00739c0 con 0x7f75e0156950 2024-07-17T08:36:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.892+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e00b0900 con 0x7f75e0156950 2024-07-17T08:36:46.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.893+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e00b4200 con 0x7f75e0156950 2024-07-17T08:36:46.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.894+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 141) ==== 99033+0+0 (secure 0 0 0) 0x7f75d8043020 con 0x7f75e0156950 2024-07-17T08:36:46.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.895+0000 7f75d6a00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 0x7f75bc0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:46.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.895+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f75d80b8aa0 con 0x7f75e0156950 2024-07-17T08:36:46.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.896+0000 7f75dde00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 0x7f75bc0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.897+0000 7f75dde00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 0x7f75bc0792e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005d00 tx=0x7f75d0005ed0 comp rx=0 tx=0).ready entity=mgr.8593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:46.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:46.900+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f75d80814d0 con 0x7f75e0156950 2024-07-17T08:36:47.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.168+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f75e0073790 con 0x7f75e0156950 2024-07-17T08:36:47.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.822+0000 7f75d6a00640 1 -- 172.21.15.44:0/2119308152 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v142) ==== 92+0+0 (secure 0 0 0) 0x7f75d8080dd0 con 0x7f75e0156950 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 msgr2=0x7f75bc0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 0x7f75bc0792e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005d00 tx=0x7f75d0005ed0 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 msgr2=0x7f75e015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f75d802b360 tx=0x7f75d8033670 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 shutdown_connections 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f75bc076e20 0x7f75bc0792e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e0156950 0x7f75e015d560 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75e0071ff0 0x7f75e01f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 --2- 172.21.15.44:0/2119308152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c40062d0 0x7f75e0074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 >> 172.21.15.44:0/2119308152 conn(0x7f75e00b0eb0 msgr2=0x7f75e015c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 shutdown_connections 2024-07-17T08:36:47.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:47.826+0000 7f75e4e00640 1 -- 172.21.15.44:0/2119308152 wait complete. 2024-07-17T08:36:47.854 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:36:48.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 -- 172.21.15.44:0/36962519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200ce1d0 msgr2=0x7f48200ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 --2- 172.21.15.44:0/36962519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200ce1d0 0x7f48200ce5b0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f48200cdce0 tx=0x7f480802a680 comp rx=0 tx=0).stop 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 -- 172.21.15.44:0/36962519 shutdown_connections 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 --2- 172.21.15.44:0/36962519 >> v2:172.21.15.175:3300/0 conn(0x7f48200b6790 0x7f4820161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 --2- 172.21.15.44:0/36962519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200ce1d0 0x7f48200ce5b0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 -- 172.21.15.44:0/36962519 >> 172.21.15.44:0/36962519 conn(0x7f48200d3700 msgr2=0x7f48200d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 -- 172.21.15.44:0/36962519 shutdown_connections 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.049+0000 7f4824a00640 1 -- 172.21.15.44:0/36962519 wait complete. 2024-07-17T08:36:48.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.051+0000 7f4824a00640 1 Processor -- start 2024-07-17T08:36:48.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.051+0000 7f4824a00640 1 -- start start 2024-07-17T08:36:48.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 0x7f48200ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48200ce1d0 0x7f482014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4820162420 con 0x7f48200b6790 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48201622a0 con 0x7f48200ce1d0 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f4824a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4820167910 con 0x7f48200b29e0 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f481d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f481d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54356/0 (socket says 172.21.15.44:54356) 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f481d400640 1 -- 172.21.15.44:0/1121162174 learned_addr learned my addr 172.21.15.44:0/1121162174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f481de00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 0x7f48200ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.052+0000 7f481e800640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48200ce1d0 0x7f482014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 msgr2=0x7f48200ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 0x7f48200ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48200ce1d0 msgr2=0x7f482014cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48200ce1d0 0x7f482014cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 -- 172.21.15.44:0/1121162174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200cba80 con 0x7f48200b6790 2024-07-17T08:36:48.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481de00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 0x7f48200ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:48.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f481d400640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f481000c8c0 tx=0x7f481000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810004220 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48100079b0 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810007de0 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.053+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48200cbd40 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.054+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48200cc300 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.055+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48200ce5b0 con 0x7f48200b6790 2024-07-17T08:36:48.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.056+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 142) ==== 99050+0+0 (secure 0 0 0) 0x7f4810005020 con 0x7f48200b6790 2024-07-17T08:36:48.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.058+0000 7f4816000640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 0x7f47fc079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.058+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f4810099460 con 0x7f48200b6790 2024-07-17T08:36:48.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.058+0000 7f481de00640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 msgr2=0x7f47fc079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:48.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.058+0000 7f481de00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 0x7f47fc079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:48.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.062+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4810014030 con 0x7f48200b6790 2024-07-17T08:36:48.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.260+0000 7f481de00640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 msgr2=0x7f47fc079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:48.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.260+0000 7f481de00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 0x7f47fc079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.272+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f48200d19f0 con 0x7f48200b6790 2024-07-17T08:36:48.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.280+0000 7f4816000640 1 -- 172.21.15.44:0/1121162174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) ==== 138+0+0 (secure 0 0 0) 0x7f4810061cf0 con 0x7f48200b6790 2024-07-17T08:36:48.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 msgr2=0x7f47fc079300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 0x7f47fc079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 msgr2=0x7f48201526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f481000c8c0 tx=0x7f481000cd80 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 shutdown_connections 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f47fc076e40 0x7f47fc079300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48200ce1d0 0x7f482014cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48200b6790 0x7f48201526a0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 --2- 172.21.15.44:0/1121162174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48200b29e0 0x7f48200ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.282+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 >> 172.21.15.44:0/1121162174 conn(0x7f48200d3700 msgr2=0x7f482015b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.283+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 shutdown_connections 2024-07-17T08:36:48.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.283+0000 7f4824a00640 1 -- 172.21.15.44:0/1121162174 wait complete. 2024-07-17T08:36:48.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:36:48.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 -- 172.21.15.44:0/3595548613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40b29e0 msgr2=0x7f9ce4166e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/3595548613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4166e40 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40cdce0 tx=0x7f9cdc02ab50 comp rx=0 tx=0).stop 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 -- 172.21.15.44:0/3595548613 shutdown_connections 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/3595548613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4166e40 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/3595548613 >> v2:172.21.15.175:3300/0 conn(0x7f9ce40b6790 0x7f9ce41648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 -- 172.21.15.44:0/3595548613 >> 172.21.15.44:0/3595548613 conn(0x7f9ce40d3700 msgr2=0x7f9ce40d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 -- 172.21.15.44:0/3595548613 shutdown_connections 2024-07-17T08:36:48.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.497+0000 7f9ce8a00640 1 -- 172.21.15.44:0/3595548613 wait complete. 2024-07-17T08:36:48.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.499+0000 7f9ce8a00640 1 Processor -- start 2024-07-17T08:36:48.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.499+0000 7f9ce8a00640 1 -- start start 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce40b6790 0x7f9ce41548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4167710 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4167590 con 0x7f9ce40b29e0 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce4167890 con 0x7f9ce40b6790 2024-07-17T08:36:48.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54382/0 (socket says 172.21.15.44:54382) 2024-07-17T08:36:48.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.500+0000 7f9ce2800640 1 -- 172.21.15.44:0/10503245 learned_addr learned my addr 172.21.15.44:0/10503245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:48.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9cdbe00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce40b6790 0x7f9ce41548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce40b6790 msgr2=0x7f9ce41548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce40b6790 0x7f9ce41548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 msgr2=0x7f9ce4159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce1e00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 -- 172.21.15.44:0/10503245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce4154fe0 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce1e00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.501+0000 7f9ce2800640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009d60 tx=0x7f9cdc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.502+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc044070 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.502+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cdc004250 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.502+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004680 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.502+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce415c1f0 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.502+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce4153c50 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.503+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce415c020 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.504+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 142) ==== 99050+0+0 (secure 0 0 0) 0x7f9cdc036400 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.505+0000 7f9cda000640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 0x7f9cc0079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:48.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.505+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7f9cdc0b96f0 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.505+0000 7f9ce1e00640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 msgr2=0x7f9cc0079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:48.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.505+0000 7f9ce1e00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 0x7f9cc0079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:48.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.509+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9cdc032030 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.707+0000 7f9ce1e00640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 msgr2=0x7f9cc0079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:48.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.707+0000 7f9ce1e00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 0x7f9cc0079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:48.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.716+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9ce40ce5b0 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.723+0000 7f9cda000640 1 -- 172.21.15.44:0/10503245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) ==== 133+0+0 (secure 0 0 0) 0x7f9cdc004820 con 0x7f9ce40ce1d0 2024-07-17T08:36:48.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 msgr2=0x7f9cc0079330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 0x7f9cc0079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 msgr2=0x7f9ce4153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009d60 tx=0x7f9cdc02c8e0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 shutdown_connections 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7f9cc0076e70 0x7f9cc0079330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce40ce1d0 0x7f9ce4153710 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce40b6790 0x7f9ce41548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 --2- 172.21.15.44:0/10503245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce40b29e0 0x7f9ce4159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.725+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 >> 172.21.15.44:0/10503245 conn(0x7f9ce40d3700 msgr2=0x7f9ce4157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.726+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 shutdown_connections 2024-07-17T08:36:48.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:48.726+0000 7f9ce8a00640 1 -- 172.21.15.44:0/10503245 wait complete. 2024-07-17T08:36:48.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:48.745 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2024-07-17T08:36:48.791 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2024-07-17T08:36:49.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 -- 172.21.15.44:0/1416480083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf80b1af0 msgr2=0x7feaf80ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 --2- 172.21.15.44:0/1416480083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf80b1af0 0x7feaf80ac190 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7feae0005fd0 tx=0x7feae002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 -- 172.21.15.44:0/1416480083 shutdown_connections 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 --2- 172.21.15.44:0/1416480083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf80b1af0 0x7feaf80ac190 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 --2- 172.21.15.44:0/1416480083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf815a280 0x7feaf80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 -- 172.21.15.44:0/1416480083 >> 172.21.15.44:0/1416480083 conn(0x7feaf80b0eb0 msgr2=0x7feaf80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 -- 172.21.15.44:0/1416480083 shutdown_connections 2024-07-17T08:36:49.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.044+0000 7feafe600640 1 -- 172.21.15.44:0/1416480083 wait complete. 2024-07-17T08:36:49.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.046+0000 7feafe600640 1 Processor -- start 2024-07-17T08:36:49.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.046+0000 7feafe600640 1 -- start start 2024-07-17T08:36:49.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf806efc0 0x7feaf8154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:49.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80b1af0 0x7feaf8158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:49.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf815a280 0x7feaf81e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:49.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80aca60 con 0x7feaf806efc0 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80ac8e0 con 0x7feaf815a280 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feafe600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feaf806d7c0 con 0x7feaf80b1af0 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feaf7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf806efc0 0x7feaf8154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feaf7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf806efc0 0x7feaf8154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34818/0 (socket says 172.21.15.44:34818) 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.047+0000 7feaf7400640 1 -- 172.21.15.44:0/1594414457 learned_addr learned my addr 172.21.15.44:0/1594414457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80b1af0 msgr2=0x7feaf8158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf80b1af0 0x7feaf8158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf815a280 msgr2=0x7feaf81e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:49.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf815a280 0x7feaf81e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf8157d40 con 0x7feaf806efc0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaf7400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf806efc0 0x7feaf8154bc0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7feae800d740 tx=0x7feae800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8017070 con 0x7feaf806efc0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feae8009d20 con 0x7feaf806efc0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feafe600640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf8158020 con 0x7feaf806efc0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80106c0 con 0x7feaf806efc0 2024-07-17T08:36:49.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.048+0000 7feafe600640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81585e0 con 0x7feaf806efc0 2024-07-17T08:36:49.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.050+0000 7feafe600640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf815b6d0 con 0x7feaf806efc0 2024-07-17T08:36:49.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.051+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 142) ==== 99050+0+0 (secure 0 0 0) 0x7feae801a070 con 0x7feaf806efc0 2024-07-17T08:36:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.052+0000 7feaef400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.052+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7541+0+0 (secure 0 0 0) 0x7feae8099130 con 0x7feaf806efc0 2024-07-17T08:36:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.052+0000 7feaefe00640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:49.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.052+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:49.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.056+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feae80619c0 con 0x7feaf806efc0 2024-07-17T08:36:49.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.254+0000 7feaefe00640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:49.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.254+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.289+0000 7feafe600640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) -- 0x7feaf80b3bd0 con 0x7fead4076e40 2024-07-17T08:36:49.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.656+0000 7feaefe00640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:49.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:49.656+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:36:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:50.457+0000 7feaefe00640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:50.457+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:36:52.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:52.060+0000 7feaefe00640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/189954344 2024-07-17T08:36:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:52.061+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:36:52.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:52.094+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 143) ==== 98557+0+0 (secure 0 0 0) 0x7feae805d550 con 0x7feaf806efc0 2024-07-17T08:36:52.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:52.095+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 msgr2=0x7fead4079300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:52.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:52.095+0000 7feaef400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/189954344,v1:172.21.15.175:6837/189954344] conn(0x7fead4076e40 0x7fead4079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.617 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.618 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.618 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.616+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1405a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:52.636 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:52.635+0000 7f1409600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:53.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.101+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 144) ==== 98684+0+0 (secure 0 0 0) 0x7feae80814a0 con 0x7feaf806efc0 2024-07-17T08:36:53.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.102+0000 7feaef400640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fead40820f0 0x7fead40844e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:53.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.102+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 --> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) -- 0x7fead400c9a0 con 0x7fead40820f0 2024-07-17T08:36:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.113+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fead40820f0 0x7fead40844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.116+0000 7feaefe00640 1 --2- 172.21.15.44:0/1594414457 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fead40820f0 0x7fead40844e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7feae0033000 tx=0x7feae00046c0 comp rx=0 tx=0).ready entity=mgr.8150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:53.123 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:53.123+0000 7f1412200640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2024-07-17T08:36:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.123+0000 7feaef400640 1 -- 172.21.15.44:0/1594414457 <== mgr.8150 v2:172.21.15.175:6834/1443688525 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) ==== 79+0+0 (secure 0 0 0) 0x7fead400c9a0 con 0x7fead40820f0 2024-07-17T08:36:53.124 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2024-07-17T08:36:53.141 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:36:53.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:53.142 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2024-07-17T08:36:53.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:36:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 -- 172.21.15.44:0/1879359903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fe0004020 msgr2=0x7f9fe0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/1879359903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fe0004020 0x7f9fe0004800 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800bbb0 tx=0x7f9fe0005830 comp rx=0 tx=0).stop 2024-07-17T08:36:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffb400640 1 -- 172.21.15.44:0/1879359903 reap_dead start 2024-07-17T08:36:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 -- 172.21.15.44:0/1879359903 shutdown_connections 2024-07-17T08:36:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 -- 172.21.15.44:0/1879359903 >> 172.21.15.44:0/1879359903 conn(0x7f9ffc06c120 msgr2=0x7f9ffc06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 -- 172.21.15.44:0/1879359903 shutdown_connections 2024-07-17T08:36:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.410+0000 7f9ffbe00640 1 -- 172.21.15.44:0/1879359903 wait complete. 2024-07-17T08:36:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.412+0000 7f9ffbe00640 1 Processor -- start 2024-07-17T08:36:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.412+0000 7f9ffbe00640 1 -- start start 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe0004020 0x7f9ffc0714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ffc14d370 0x7f9ffc14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc06f210 con 0x7f9ffc1553d0 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc06f090 con 0x7f9fe0004020 2024-07-17T08:36:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.413+0000 7f9ffbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc06f5a0 con 0x7f9ffc14d370 2024-07-17T08:36:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34828/0 (socket says 172.21.15.44:34828) 2024-07-17T08:36:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 -- 172.21.15.44:0/3110129458 learned_addr learned my addr 172.21.15.44:0/3110129458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffa000640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ffc14d370 0x7f9ffc14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffaa00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe0004020 0x7f9ffc0714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 -- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ffc14d370 msgr2=0x7f9ffc14ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ffc14d370 0x7f9ffc14ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 -- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe0004020 msgr2=0x7f9ffc0714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe0004020 0x7f9ffc0714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.414+0000 7f9ffb400640 1 -- 172.21.15.44:0/3110129458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc097380 con 0x7f9ffc1553d0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ffb400640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4005fd0 tx=0x7f9ff400a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff401b070 con 0x7f9ffc1553d0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff4003d80 con 0x7f9ffc1553d0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc1514d0 con 0x7f9ffc1553d0 2024-07-17T08:36:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4010970 con 0x7f9ffc1553d0 2024-07-17T08:36:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.415+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc14cfb0 con 0x7f9ffc1553d0 2024-07-17T08:36:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.416+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc0cefe0 con 0x7f9ffc1553d0 2024-07-17T08:36:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.418+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 144) ==== 98684+0+0 (secure 0 0 0) 0x7f9ff401e070 con 0x7f9ffc1553d0 2024-07-17T08:36:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.419+0000 7f9ff2a00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 0x7f9fdc078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.419+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f9ff4097fe0 con 0x7f9ffc1553d0 2024-07-17T08:36:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.419+0000 7f9ffaa00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 0x7f9fdc078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.420+0000 7f9ffaa00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 0x7f9fdc078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fec001eb0 tx=0x7f9fec001d20 comp rx=0 tx=0).ready entity=mgr.8150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:53.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.423+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9ff4011040 con 0x7f9ffc1553d0 2024-07-17T08:36:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:53.718+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f9ffc0b2d70 con 0x7f9ffc1553d0 2024-07-17T08:36:54.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.151+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v145) ==== 93+0+0 (secure 0 0 0) 0x7f9ff4060e60 con 0x7f9ffc1553d0 2024-07-17T08:36:54.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.155+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 msgr2=0x7f9fdc078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:54.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.155+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 0x7f9fdc078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fec001eb0 tx=0x7f9fec001d20 comp rx=0 tx=0).stop 2024-07-17T08:36:54.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.155+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 msgr2=0x7f9ffc072d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4005fd0 tx=0x7f9ff400a840 comp rx=0 tx=0).stop 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ff2a00640 1 -- 172.21.15.44:0/3110129458 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f9ff405c9f0 con 0x7f9ffc1553d0 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 shutdown_connections 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f9fdc076a50 0x7f9fdc078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ffc1553d0 0x7f9ffc072d80 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ffc14d370 0x7f9ffc14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 --2- 172.21.15.44:0/3110129458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe0004020 0x7f9ffc0714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.156+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 >> 172.21.15.44:0/3110129458 conn(0x7f9ffc06c120 msgr2=0x7f9ffc0ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.157+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 shutdown_connections 2024-07-17T08:36:54.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.158+0000 7f9ffbe00640 1 -- 172.21.15.44:0/3110129458 wait complete. 2024-07-17T08:36:54.181 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:36:54.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:36:54.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:36:54.381 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:36:54.381 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:36:54.381 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:54.381 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:54.428 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:54.428 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:54.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:36:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 -- 172.21.15.44:0/3041329499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89f0004d90 msgr2=0x7f89f0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 --2- 172.21.15.44:0/3041329499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89f0004d90 0x7f89f0005210 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000c610 tx=0x7f89f0005830 comp rx=0 tx=0).stop 2024-07-17T08:36:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a0e400640 1 -- 172.21.15.44:0/3041329499 reap_dead start 2024-07-17T08:36:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 -- 172.21.15.44:0/3041329499 shutdown_connections 2024-07-17T08:36:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 -- 172.21.15.44:0/3041329499 >> 172.21.15.44:0/3041329499 conn(0x7f8a08073d50 msgr2=0x7f8a0809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 -- 172.21.15.44:0/3041329499 shutdown_connections 2024-07-17T08:36:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.767+0000 7f8a10400640 1 -- 172.21.15.44:0/3041329499 wait complete. 2024-07-17T08:36:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.769+0000 7f8a10400640 1 Processor -- start 2024-07-17T08:36:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.769+0000 7f8a10400640 1 -- start start 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f0004d90 0x7f8a08153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815a260 0x7f8a0815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080b2290 con 0x7f8a08151b50 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080b2110 con 0x7f89f0004d90 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a10400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a080b2410 con 0x7f8a0815a260 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a0d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a0d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34860/0 (socket says 172.21.15.44:34860) 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.770+0000 7f8a0d000640 1 -- 172.21.15.44:0/2036483247 learned_addr learned my addr 172.21.15.44:0/2036483247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0da00640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f0004d90 0x7f8a08153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0e400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815a260 0x7f8a0815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815a260 msgr2=0x7f8a0815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815a260 0x7f8a0815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f0004d90 msgr2=0x7f8a08153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f0004d90 0x7f8a08153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 -- 172.21.15.44:0/2036483247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0806d050 con 0x7f8a08151b50 2024-07-17T08:36:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.771+0000 7f8a0d000640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f89f4004cb0 tx=0x7f89f40017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.772+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4015020 con 0x7f8a08151b50 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.772+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f4011760 con 0x7f8a08151b50 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.772+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0806d2f0 con 0x7f8a08151b50 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.772+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4011bb0 con 0x7f8a08151b50 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.772+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0806d8b0 con 0x7f8a08151b50 2024-07-17T08:36:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.773+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a080b4f10 con 0x7f8a08151b50 2024-07-17T08:36:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.774+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f89f4018020 con 0x7f8a08151b50 2024-07-17T08:36:54.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.776+0000 7f8a05600640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 0x7f89e80792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:54.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.776+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f89f4099ff0 con 0x7f8a08151b50 2024-07-17T08:36:54.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.776+0000 7f8a0da00640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 msgr2=0x7f89e80792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:54.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.776+0000 7f8a0da00640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 0x7f89e80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:54.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.779+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f89f401f020 con 0x7f8a08151b50 2024-07-17T08:36:54.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.977+0000 7f8a0da00640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 msgr2=0x7f89e80792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:54.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.977+0000 7f8a0da00640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 0x7f89e80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:54.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:54.999+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a0815e4d0 con 0x7f8a08151b50 2024-07-17T08:36:55.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.000+0000 7f8a05600640 1 -- 172.21.15.44:0/2036483247 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v248) ==== 81+0+592 (secure 0 0 0) 0x7f89f40627b0 con 0x7f8a08151b50 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3847370980 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1726477539 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/458180565 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4292218033 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2034094416 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/978474688 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/189954344 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.001 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1122698532 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.002 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/189954344 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.002 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1151690258 2024-07-18T08:36:52.077359+0000 2024-07-17T08:36:55.002 INFO:teuthology.orchestra.run.smithi044.stderr:listed 10 entries 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.002+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 msgr2=0x7f89e80792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.002+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 0x7f89e80792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.002+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 msgr2=0x7f8a081544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.002+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f89f4004cb0 tx=0x7f89f40017f0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 shutdown_connections 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f89e8076e20 0x7f89e80792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815a260 0x7f8a0815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a08151b50 0x7f8a081544e0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 --2- 172.21.15.44:0/2036483247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f0004d90 0x7f8a08153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 >> 172.21.15.44:0/2036483247 conn(0x7f8a08073d50 msgr2=0x7f8a080af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:55.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 shutdown_connections 2024-07-17T08:36:55.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.003+0000 7f8a10400640 1 -- 172.21.15.44:0/2036483247 wait complete. 2024-07-17T08:36:55.023 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:36:55.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:55.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:36:55.111 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:36:55.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:36:55.112 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:36:55.154 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:36:55.154 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:36:55.155 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:55.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:36:55.201 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:36:55.201 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:36:55.201 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:55.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:36:55.247 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:36:55.247 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:36:55.247 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:36:55.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:36:55.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 -- 172.21.15.44:0/1530415409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42700ac3e0 msgr2=0x7f42700ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 --2- 172.21.15.44:0/1530415409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42700ac3e0 0x7f42700ac870 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f425c005fd0 tx=0x7f425c02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 -- 172.21.15.44:0/1530415409 shutdown_connections 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 --2- 172.21.15.44:0/1530415409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 0x7f427006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 --2- 172.21.15.44:0/1530415409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42700ac3e0 0x7f42700ac870 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 -- 172.21.15.44:0/1530415409 >> 172.21.15.44:0/1530415409 conn(0x7f42700b0fb0 msgr2=0x7f42700b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 -- 172.21.15.44:0/1530415409 shutdown_connections 2024-07-17T08:36:55.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.503+0000 7f4275e00640 1 -- 172.21.15.44:0/1530415409 wait complete. 2024-07-17T08:36:55.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.504+0000 7f4275e00640 1 Processor -- start 2024-07-17T08:36:55.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 -- start start 2024-07-17T08:36:55.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 0x7f4270071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4270153a50 0x7f42701566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4270153fb0 con 0x7f42700ac3e0 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4270153e30 con 0x7f4270153a50 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.505+0000 7f4275e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f427006f710 con 0x7f427006e090 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34870/0 (socket says 172.21.15.44:34870) 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 -- 172.21.15.44:0/1499969894 learned_addr learned my addr 172.21.15.44:0/1499969894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426f400640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4270153a50 0x7f42701566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:55.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426ea00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 0x7f4270071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:55.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 msgr2=0x7f4270071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:55.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 0x7f4270071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4270153a50 msgr2=0x7f42701566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:55.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4270153a50 0x7f42701566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.506+0000 7f426e000640 1 -- 172.21.15.44:0/1499969894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270155680 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f426e000640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f425c0099f0 tx=0x7f425c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c03f070 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f425c005600 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c035030 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427013fb00 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.507+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4270155bd0 con 0x7f42700ac3e0 2024-07-17T08:36:55.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.508+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427006a1a0 con 0x7f42700ac3e0 2024-07-17T08:36:55.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.509+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f425c042070 con 0x7f42700ac3e0 2024-07-17T08:36:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.510+0000 7f4267400640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 0x7f42500792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.510+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f425c0b9b50 con 0x7f42700ac3e0 2024-07-17T08:36:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.510+0000 7f426ea00640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 msgr2=0x7f42500792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:55.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.511+0000 7f426ea00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 0x7f42500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:55.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.515+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f427006a1a0 con 0x7f42700ac3e0 2024-07-17T08:36:55.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.712+0000 7f426ea00640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 msgr2=0x7f42500792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:55.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.712+0000 7f426ea00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 0x7f42500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:55.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.770+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42700adba0 con 0x7f42700ac3e0 2024-07-17T08:36:55.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.771+0000 7f4267400640 1 -- 172.21.15.44:0/1499969894 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7f42700adba0 con 0x7f42700ac3e0 2024-07-17T08:36:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:55.772 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:55.772 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.773+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 msgr2=0x7f42500792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:55.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.773+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 0x7f42500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.773+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 msgr2=0x7f42701f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.773+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f425c0099f0 tx=0x7f425c003820 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 shutdown_connections 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f4250076df0 0x7f42500792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4270153a50 0x7f42701566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42700ac3e0 0x7f42701f1de0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 --2- 172.21.15.44:0/1499969894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f427006e090 0x7f4270071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 >> 172.21.15.44:0/1499969894 conn(0x7f42700b0fb0 msgr2=0x7f42700b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 shutdown_connections 2024-07-17T08:36:55.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.774+0000 7f4275e00640 1 -- 172.21.15.44:0/1499969894 wait complete. 2024-07-17T08:36:55.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:55.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.994+0000 7f43fdc00640 1 -- 172.21.15.44:0/2775568244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f80ad9e0 msgr2=0x7f43f80ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.994+0000 7f43fdc00640 1 --2- 172.21.15.44:0/2775568244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f80ad9e0 0x7f43f80ade70 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005fd0 tx=0x7f43f002a8b0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.994+0000 7f43fdc00640 1 -- 172.21.15.44:0/2775568244 shutdown_connections 2024-07-17T08:36:56.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.994+0000 7f43fdc00640 1 --2- 172.21.15.44:0/2775568244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f80ad9e0 0x7f43f80ade70 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.994+0000 7f43fdc00640 1 -- 172.21.15.44:0/2775568244 >> 172.21.15.44:0/2775568244 conn(0x7f43f80b2240 msgr2=0x7f43f80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:56.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.995+0000 7f43fdc00640 1 -- 172.21.15.44:0/2775568244 shutdown_connections 2024-07-17T08:36:56.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.995+0000 7f43fdc00640 1 -- 172.21.15.44:0/2775568244 wait complete. 2024-07-17T08:36:56.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.996+0000 7f43fdc00640 1 Processor -- start 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.996+0000 7f43fdc00640 1 -- start start 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43f806bf40 0x7f43f81542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43f80ad9e0 0x7f43f81e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8140880 con 0x7f43f8140b70 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8140700 con 0x7f43f806bf40 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43fdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43f80ae3b0 con 0x7f43f80ad9e0 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.997+0000 7f43f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34880/0 (socket says 172.21.15.44:34880) 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 -- 172.21.15.44:0/14010012 learned_addr learned my addr 172.21.15.44:0/14010012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f6000640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43f80ad9e0 0x7f43f81e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f6a00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43f806bf40 0x7f43f81542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43f80ad9e0 msgr2=0x7f43f81e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43f80ad9e0 0x7f43f81e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43f806bf40 msgr2=0x7f43f81542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43f806bf40 0x7f43f81542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 -- 172.21.15.44:0/14010012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f8130bc0 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.998+0000 7f43f7400640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006010 tx=0x7f43f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.999+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0040070 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.999+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43f0004330 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.999+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f8130ea0 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.999+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031640 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:55.999+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f8131460 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.000+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f80ade70 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.001+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f43f0039040 con 0x7f43f8140b70 2024-07-17T08:36:56.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.002+0000 7f43eea00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 0x7f43d80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.002+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f43f00ba890 con 0x7f43f8140b70 2024-07-17T08:36:56.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.002+0000 7f43f6a00640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 msgr2=0x7f43d80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:56.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.002+0000 7f43f6a00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 0x7f43d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:56.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.006+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f43f0083000 con 0x7f43f8140b70 2024-07-17T08:36:56.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.203+0000 7f43f6a00640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 msgr2=0x7f43d80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:56.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.203+0000 7f43f6a00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 0x7f43d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.261+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43f80b6610 con 0x7f43f8140b70 2024-07-17T08:36:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.263+0000 7f43eea00640 1 -- 172.21.15.44:0/14010012 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7f43f0082900 con 0x7f43f8140b70 2024-07-17T08:36:56.264 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:56.264 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:56.264 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 msgr2=0x7f43d80792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 0x7f43d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 msgr2=0x7f43f80b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006010 tx=0x7f43f0003ba0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 shutdown_connections 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f43d8076df0 0x7f43d80792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43f8140b70 0x7f43f80b0070 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43f80ad9e0 0x7f43f81e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 --2- 172.21.15.44:0/14010012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43f806bf40 0x7f43f81542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.265+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 >> 172.21.15.44:0/14010012 conn(0x7f43f80b2240 msgr2=0x7f43f806e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.266+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 shutdown_connections 2024-07-17T08:36:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.266+0000 7f43fdc00640 1 -- 172.21.15.44:0/14010012 wait complete. 2024-07-17T08:36:56.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:56.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 -- 172.21.15.44:0/932620216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900b9060 msgr2=0x7ff09015d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 --2- 172.21.15.44:0/932620216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900b9060 0x7ff09015d9d0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7ff088009be0 tx=0x7ff08802a8d0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 -- 172.21.15.44:0/932620216 shutdown_connections 2024-07-17T08:36:56.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 --2- 172.21.15.44:0/932620216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900b9060 0x7ff09015d9d0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 -- 172.21.15.44:0/932620216 >> 172.21.15.44:0/932620216 conn(0x7ff0900cfec0 msgr2=0x7ff0900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:56.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 -- 172.21.15.44:0/932620216 shutdown_connections 2024-07-17T08:36:56.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.477+0000 7ff096800640 1 -- 172.21.15.44:0/932620216 wait complete. 2024-07-17T08:36:56.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.479+0000 7ff096800640 1 Processor -- start 2024-07-17T08:36:56.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.479+0000 7ff096800640 1 -- start start 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900b8c80 0x7ff0900cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 0x7ff0900c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015e2a0 con 0x7ff0900ce5c0 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015e120 con 0x7ff0900b9060 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff096800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff09015e420 con 0x7ff0900b8c80 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff095e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff095e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34894/0 (socket says 172.21.15.44:34894) 2024-07-17T08:36:56.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.480+0000 7ff095e00640 1 -- 172.21.15.44:0/2234835050 learned_addr learned my addr 172.21.15.44:0/2234835050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095400640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900b8c80 0x7ff0900cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff094a00640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 0x7ff0900c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900b8c80 msgr2=0x7ff0900cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900b8c80 0x7ff0900cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 msgr2=0x7ff0900c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 0x7ff0900c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 -- 172.21.15.44:0/2234835050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900c7490 con 0x7ff0900ce5c0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff094a00640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 0x7ff0900c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff095e00640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7ff088006010 tx=0x7ff088003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.481+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088031d80 con 0x7ff0900ce5c0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.482+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0900c7730 con 0x7ff0900ce5c0 2024-07-17T08:36:56.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.482+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff088004040 con 0x7ff0900ce5c0 2024-07-17T08:36:56.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.482+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088004490 con 0x7ff0900ce5c0 2024-07-17T08:36:56.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.482+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0900c7cf0 con 0x7ff0900ce5c0 2024-07-17T08:36:56.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.483+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0900b4ec0 con 0x7ff0900ce5c0 2024-07-17T08:36:56.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.485+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7ff088039040 con 0x7ff0900ce5c0 2024-07-17T08:36:56.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.486+0000 7ff08d600640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 0x7ff0700792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.486+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7ff0880ba730 con 0x7ff0900ce5c0 2024-07-17T08:36:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.486+0000 7ff095400640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 msgr2=0x7ff0700792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:56.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.486+0000 7ff095400640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 0x7ff0700792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:56.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.490+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff088053080 con 0x7ff0900ce5c0 2024-07-17T08:36:56.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.688+0000 7ff095400640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 msgr2=0x7ff0700792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:56.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.688+0000 7ff095400640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 0x7ff0700792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.699+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff09015f5a0 con 0x7ff0900ce5c0 2024-07-17T08:36:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.700+0000 7ff08d600640 1 -- 172.21.15.44:0/2234835050 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v248) ==== 74+0+22174 (secure 0 0 0) 0x7ff088040030 con 0x7ff0900ce5c0 2024-07-17T08:36:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:56.702 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":248,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:52.077439+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3847370980":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1726477539":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/458180565":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/4292218033":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/2034094416":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/978474688":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1122698532":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6834/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1151690258":"2024-07-18T08:36:52.077359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 msgr2=0x7ff0700792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 0x7ff0700792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 msgr2=0x7ff0900c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7ff088006010 tx=0x7ff088003f10 comp rx=0 tx=0).stop 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 shutdown_connections 2024-07-17T08:36:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7ff070076df0 0x7ff0700792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0900ce5c0 0x7ff0900c8cb0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0900b9060 0x7ff0900c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 --2- 172.21.15.44:0/2234835050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0900b8c80 0x7ff0900cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.703+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 >> 172.21.15.44:0/2234835050 conn(0x7ff0900cfec0 msgr2=0x7ff09015e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.704+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 shutdown_connections 2024-07-17T08:36:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.704+0000 7ff096800640 1 -- 172.21.15.44:0/2234835050 wait complete. 2024-07-17T08:36:56.723 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:36:56.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 -- 172.21.15.44:0/913353051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eb8004dd0 msgr2=0x7f7eb8005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/913353051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eb8004dd0 0x7f7eb8005250 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c640 tx=0x7f7eb8005830 comp rx=0 tx=0).stop 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3400640 1 -- 172.21.15.44:0/913353051 reap_dead start 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 -- 172.21.15.44:0/913353051 shutdown_connections 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 -- 172.21.15.44:0/913353051 >> 172.21.15.44:0/913353051 conn(0x7f7ed40b5890 msgr2=0x7f7ed40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 -- 172.21.15.44:0/913353051 shutdown_connections 2024-07-17T08:36:56.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.918+0000 7f7ed3e00640 1 -- 172.21.15.44:0/913353051 wait complete. 2024-07-17T08:36:56.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.920+0000 7f7ed3e00640 1 Processor -- start 2024-07-17T08:36:56.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.920+0000 7f7ed3e00640 1 -- start start 2024-07-17T08:36:56.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 0x7f7ed40cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ed40c94d0 0x7f7ed414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40adee0 con 0x7f7ed41f4e60 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40add60 con 0x7f7ed40c94d0 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40ae060 con 0x7f7eb8004dd0 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7eca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7eca000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34910/0 (socket says 172.21.15.44:34910) 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7eca000640 1 -- 172.21.15.44:0/1289875327 learned_addr learned my addr 172.21.15.44:0/1289875327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed2a00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 0x7f7ed40cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.921+0000 7f7ed3400640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ed40c94d0 0x7f7ed414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 msgr2=0x7f7ed40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 0x7f7ed40cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ed40c94d0 msgr2=0x7f7ed414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ed40c94d0 0x7f7ed414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 -- 172.21.15.44:0/1289875327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed40cb690 con 0x7f7ed41f4e60 2024-07-17T08:36:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ed2a00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 0x7f7ed40cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7eca000640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc004db0 tx=0x7f7ebc001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0107d0 con 0x7f7ed41f4e60 2024-07-17T08:36:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ebc016070 con 0x7f7ed41f4e60 2024-07-17T08:36:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed40cb970 con 0x7f7ed41f4e60 2024-07-17T08:36:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc010d60 con 0x7f7ed41f4e60 2024-07-17T08:36:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.922+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed40cbf30 con 0x7f7ed41f4e60 2024-07-17T08:36:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.923+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed40b1030 con 0x7f7ed41f4e60 2024-07-17T08:36:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.925+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f7ebc01b460 con 0x7f7ed41f4e60 2024-07-17T08:36:56.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.926+0000 7f7ecb400640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 0x7f7eb00792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:56.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.926+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f7ebc099b40 con 0x7f7ed41f4e60 2024-07-17T08:36:56.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.926+0000 7f7ed2a00640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 msgr2=0x7f7eb00792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:56.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.926+0000 7f7ed2a00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 0x7f7eb00792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:56.934+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7ebc062220 con 0x7f7ed41f4e60 2024-07-17T08:36:57.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.128+0000 7f7ed2a00640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 msgr2=0x7f7eb00792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:57.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.128+0000 7f7ed2a00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 0x7f7eb00792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:57.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.184+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed40aeda0 con 0x7f7ed41f4e60 2024-07-17T08:36:57.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.185+0000 7f7ecb400640 1 -- 172.21.15.44:0/1289875327 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7f7ebc061b20 con 0x7f7ed41f4e60 2024-07-17T08:36:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:57.186 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:57.186 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:57.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 msgr2=0x7f7eb00792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 0x7f7eb00792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 msgr2=0x7f7ed40cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc004db0 tx=0x7f7ebc001800 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 shutdown_connections 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f7eb0076df0 0x7f7eb00792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ed40c94d0 0x7f7ed414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed41f4e60 0x7f7ed40cf1c0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 --2- 172.21.15.44:0/1289875327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eb8004dd0 0x7f7ed40cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 >> 172.21.15.44:0/1289875327 conn(0x7f7ed40b5890 msgr2=0x7f7ed40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 shutdown_connections 2024-07-17T08:36:57.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.188+0000 7f7ed3e00640 1 -- 172.21.15.44:0/1289875327 wait complete. 2024-07-17T08:36:57.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:36:57.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.390+0000 7f84fec00640 1 -- 172.21.15.44:0/1708988077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80b3db0 msgr2=0x7f84f80ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.390+0000 7f84fec00640 1 --2- 172.21.15.44:0/1708988077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80b3db0 0x7f84f80ad8e0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f84f80d00a0 tx=0x7f84e402a560 comp rx=0 tx=0).stop 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 -- 172.21.15.44:0/1708988077 shutdown_connections 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 --2- 172.21.15.44:0/1708988077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf920 0x7f84f80ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 --2- 172.21.15.44:0/1708988077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80b3db0 0x7f84f80ad8e0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 --2- 172.21.15.44:0/1708988077 >> v2:172.21.15.175:3300/0 conn(0x7f84f80cf540 0x7f84f80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 -- 172.21.15.44:0/1708988077 >> 172.21.15.44:0/1708988077 conn(0x7f84f80b0e80 msgr2=0x7f84f80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:57.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 -- 172.21.15.44:0/1708988077 shutdown_connections 2024-07-17T08:36:57.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.391+0000 7f84fec00640 1 -- 172.21.15.44:0/1708988077 wait complete. 2024-07-17T08:36:57.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.393+0000 7f84fec00640 1 Processor -- start 2024-07-17T08:36:57.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.393+0000 7f84fec00640 1 -- start start 2024-07-17T08:36:57.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f80b3db0 0x7f84f815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf540 0x7f84f81f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80ae6f0 con 0x7f84f80cf920 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80ae570 con 0x7f84f80b3db0 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80ae870 con 0x7f84f80cf540 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34934/0 (socket says 172.21.15.44:34934) 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 -- 172.21.15.44:0/2705438561 learned_addr learned my addr 172.21.15.44:0/2705438561 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fd800640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f80b3db0 0x7f84f815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fce00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf540 0x7f84f81f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf540 msgr2=0x7f84f81f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf540 0x7f84f81f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f80b3db0 msgr2=0x7f84f815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f80b3db0 0x7f84f815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.394+0000 7f84fe200640 1 -- 172.21.15.44:0/2705438561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f815d4b0 con 0x7f84f80cf920 2024-07-17T08:36:57.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84fe200640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f84f000cad0 tx=0x7f84f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:57.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000ae20 con 0x7f84f80cf920 2024-07-17T08:36:57.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84f000f450 con 0x7f84f80cf920 2024-07-17T08:36:57.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000f8c0 con 0x7f84f80cf920 2024-07-17T08:36:57.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f815d790 con 0x7f84f80cf920 2024-07-17T08:36:57.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.395+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f815bdd0 con 0x7f84f80cf920 2024-07-17T08:36:57.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.396+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f80b3870 con 0x7f84f80cf920 2024-07-17T08:36:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.397+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f84f000fa60 con 0x7f84f80cf920 2024-07-17T08:36:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.398+0000 7f84f5600640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 0x7f84d80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.398+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f84f0099bc0 con 0x7f84f80cf920 2024-07-17T08:36:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.399+0000 7f84fd800640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 msgr2=0x7f84d80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.399+0000 7f84fd800640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 0x7f84d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.403+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f84f0014030 con 0x7f84f80cf920 2024-07-17T08:36:57.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.600+0000 7f84fd800640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 msgr2=0x7f84d80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.601+0000 7f84fd800640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 0x7f84d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:57.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.659+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84f813be40 con 0x7f84f80cf920 2024-07-17T08:36:57.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.660+0000 7f84f5600640 1 -- 172.21.15.44:0/2705438561 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+6557 (secure 0 0 0) 0x7f84f0062330 con 0x7f84f80cf920 2024-07-17T08:36:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:57.662 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"btime":"2024-07-17T08:36:37:757628+0000","default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:22.075389+0000","modified":"2024-07-17T08:36:25.120303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8788},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8788":{"gid":8788,"name":"d","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6836/1736454513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1736454513},{"type":"v1","addr":"172.21.15.175:6836","nonce":1736454513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8788,"qdb_cluster":[8788]},"id":15},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:36:57.663 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 135 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 msgr2=0x7f84d80792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 0x7f84d80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 msgr2=0x7f84f815e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f84f000cad0 tx=0x7f84f000cf90 comp rx=0 tx=0).stop 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 shutdown_connections 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f84d8076df0 0x7f84d80792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84f80cf920 0x7f84f815e270 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84f80cf540 0x7f84f81f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 --2- 172.21.15.44:0/2705438561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f84f80b3db0 0x7f84f815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.663+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 >> 172.21.15.44:0/2705438561 conn(0x7f84f80b0e80 msgr2=0x7f84f813e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.664+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 shutdown_connections 2024-07-17T08:36:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.664+0000 7f84fec00640 1 -- 172.21.15.44:0/2705438561 wait complete. 2024-07-17T08:36:57.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 -- 172.21.15.44:0/1434576104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe264158e70 msgr2=0x7fe26415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 --2- 172.21.15.44:0/1434576104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe264158e70 0x7fe26415f9d0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fe254009830 tx=0x7fe25402a510 comp rx=0 tx=0).stop 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 -- 172.21.15.44:0/1434576104 shutdown_connections 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 --2- 172.21.15.44:0/1434576104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe2640d1240 0x7fe2640d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 --2- 172.21.15.44:0/1434576104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe264158e70 0x7fe26415f9d0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 -- 172.21.15.44:0/1434576104 >> 172.21.15.44:0/1434576104 conn(0x7fe26406bf40 msgr2=0x7fe26406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 -- 172.21.15.44:0/1434576104 shutdown_connections 2024-07-17T08:36:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.879+0000 7fe268e00640 1 -- 172.21.15.44:0/1434576104 wait complete. 2024-07-17T08:36:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.881+0000 7fe268e00640 1 Processor -- start 2024-07-17T08:36:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.881+0000 7fe268e00640 1 -- start start 2024-07-17T08:36:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 0x7fe2640b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 0x7fe26414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe26415bd60 con 0x7fe2640d1240 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe26415bbe0 con 0x7fe2640b56b0 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe268e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe26415bee0 con 0x7fe264158e70 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.882+0000 7fe262a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34946/0 (socket says 172.21.15.44:34946) 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 -- 172.21.15.44:0/3412328626 learned_addr learned my addr 172.21.15.44:0/3412328626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe263e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 0x7fe26414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe263400640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 0x7fe2640b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 msgr2=0x7fe26414aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 0x7fe26414aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 msgr2=0x7fe2640b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 0x7fe2640b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 -- 172.21.15.44:0/3412328626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26414e1c0 con 0x7fe2640d1240 2024-07-17T08:36:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe263e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 0x7fe26414aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe263400640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 0x7fe2640b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe262a00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c450 tx=0x7fe25000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.883+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250011790 con 0x7fe2640d1240 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.884+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe250017070 con 0x7fe2640d1240 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.884+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26414e4a0 con 0x7fe2640d1240 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.884+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250011cc0 con 0x7fe2640d1240 2024-07-17T08:36:57.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.884+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26414be20 con 0x7fe2640d1240 2024-07-17T08:36:57.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.885+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26415c390 con 0x7fe2640d1240 2024-07-17T08:36:57.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.886+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7fe25001c460 con 0x7fe2640d1240 2024-07-17T08:36:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.887+0000 7fe25b400640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 0x7fe2440792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.887+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7fe250098d80 con 0x7fe2640d1240 2024-07-17T08:36:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.887+0000 7fe263400640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 msgr2=0x7fe2440792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:57.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.887+0000 7fe263400640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 0x7fe2440792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:57.891+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe2500616d0 con 0x7fe2640d1240 2024-07-17T08:36:58.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.088+0000 7fe263400640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 msgr2=0x7fe2440792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:58.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.088+0000 7fe263400640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 0x7fe2440792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:36:58.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.098+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe26414bff0 con 0x7fe2640d1240 2024-07-17T08:36:58.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.101+0000 7fe25b400640 1 -- 172.21.15.44:0/3412328626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v248) ==== 74+0+22174 (secure 0 0 0) 0x7fe250060fd0 con 0x7fe2640d1240 2024-07-17T08:36:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:36:58.102 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":248,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:36:52.077439+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:36:20.029173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-07-17T08:36:21.060283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3847370980":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1726477539":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/458180565":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/4292218033":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/2034094416":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/978474688":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1122698532":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6834/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1151690258":"2024-07-18T08:36:52.077359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 msgr2=0x7fe2440792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 0x7fe2440792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 msgr2=0x7fe26415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c450 tx=0x7fe25000c910 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 shutdown_connections 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7fe244076e20 0x7fe2440792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe264158e70 0x7fe26414aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2640d1240 0x7fe26415de30 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 --2- 172.21.15.44:0/3412328626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2640b56b0 0x7fe2640b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 >> 172.21.15.44:0/3412328626 conn(0x7fe26406bf40 msgr2=0x7fe26406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:58.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 shutdown_connections 2024-07-17T08:36:58.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.104+0000 7fe268e00640 1 -- 172.21.15.44:0/3412328626 wait complete. 2024-07-17T08:36:58.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.309+0000 7f3415a00640 1 -- 172.21.15.44:0/747692491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f4004800 msgr2=0x7f33f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.309+0000 7f3415a00640 1 --2- 172.21.15.44:0/747692491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f4004800 0x7f33f4004c80 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009b50 tx=0x7f33f4005830 comp rx=0 tx=0).stop 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.309+0000 7f340f400640 1 -- 172.21.15.44:0/747692491 reap_dead start 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.310+0000 7f3415a00640 1 -- 172.21.15.44:0/747692491 shutdown_connections 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.310+0000 7f3415a00640 1 -- 172.21.15.44:0/747692491 >> 172.21.15.44:0/747692491 conn(0x7f34100b0eb0 msgr2=0x7f34100b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.310+0000 7f3415a00640 1 -- 172.21.15.44:0/747692491 shutdown_connections 2024-07-17T08:36:58.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.310+0000 7f3415a00640 1 -- 172.21.15.44:0/747692491 wait complete. 2024-07-17T08:36:58.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.312+0000 7f3415a00640 1 Processor -- start 2024-07-17T08:36:58.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.313+0000 7f3415a00640 1 -- start start 2024-07-17T08:36:58.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f3410073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410074220 0x7f34101f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34100cd2e0 con 0x7f3410072630 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f341015c7a0 con 0x7f3410074220 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f3415a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34100cd460 con 0x7f33f4004800 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.314+0000 7f340f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34976/0 (socket says 172.21.15.44:34976) 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 -- 172.21.15.44:0/2262699026 learned_addr learned my addr 172.21.15.44:0/2262699026 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340e000640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410074220 0x7f34101f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340ea00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f3410073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 -- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 msgr2=0x7f3410073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f3410073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 -- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410074220 msgr2=0x7f34101f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410074220 0x7f34101f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 -- 172.21.15.44:0/2262699026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34101f2c80 con 0x7f3410072630 2024-07-17T08:36:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.315+0000 7f340f400640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f341006f610 tx=0x7f340800e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:58.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.316+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004240 con 0x7f3410072630 2024-07-17T08:36:58.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.316+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3408005020 con 0x7f3410072630 2024-07-17T08:36:58.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.316+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408005470 con 0x7f3410072630 2024-07-17T08:36:58.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.316+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3410153ed0 con 0x7f3410072630 2024-07-17T08:36:58.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.316+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3410154490 con 0x7f3410072630 2024-07-17T08:36:58.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.317+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341006d620 con 0x7f3410072630 2024-07-17T08:36:58.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.320+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 145) ==== 99034+0+0 (secure 0 0 0) 0x7f3408019020 con 0x7f3410072630 2024-07-17T08:36:58.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.321+0000 7f3406a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 0x7f33ec0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.321+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7971+0+0 (secure 0 0 0) 0x7f3408098d40 con 0x7f3410072630 2024-07-17T08:36:58.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.321+0000 7f340ea00640 1 -- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 msgr2=0x7f33ec0792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1443688525 2024-07-17T08:36:58.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.321+0000 7f340ea00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 0x7f33ec0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:36:58.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.326+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34101f2c80 con 0x7f3410072630 2024-07-17T08:36:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.441+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 146) ==== 98540+0+0 (secure 0 0 0) 0x7f340805d0e0 con 0x7f3410072630 2024-07-17T08:36:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.441+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 msgr2=0x7f33ec0792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:36:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.441+0000 7f3406a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 0x7f33ec0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.518 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.517+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.519 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.518+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.519 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.518+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.519 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.518+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.519 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.518+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.520 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.519+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.520 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.519+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.520 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.519+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.520 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.519+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.520 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.519+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.533 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.532+0000 7fe20d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.533+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.533+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.533+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.534 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:36:58.533+0000 7fe211800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:36:58.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.588+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34100b3180 con 0x7f3410072630 2024-07-17T08:36:58.590 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:36:58.589+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:36:58.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.592+0000 7f3406a00640 1 -- 172.21.15.44:0/2262699026 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v136) ==== 161+0+0 (secure 0 0 0) 0x7f3408061550 con 0x7f3410072630 2024-07-17T08:36:58.593 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:36:58.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.596+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 msgr2=0x7f341014fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.596+0000 7f3415a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f341006f610 tx=0x7f340800e9b0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 shutdown_connections 2024-07-17T08:36:58.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:6834/1443688525,v1:172.21.15.175:6837/1443688525] conn(0x7f33ec076e20 0x7f33ec0792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410072630 0x7f341014fef0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3410074220 0x7f34101f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 --2- 172.21.15.44:0/2262699026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f4004800 0x7f3410073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 >> 172.21.15.44:0/2262699026 conn(0x7f34100b0eb0 msgr2=0x7f341015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.597+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 shutdown_connections 2024-07-17T08:36:58.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.598+0000 7f3415a00640 1 -- 172.21.15.44:0/2262699026 wait complete. 2024-07-17T08:36:58.612 INFO:tasks.ceph.mds.d.smithi175.stderr: -4529> 2024-07-17T08:36:24.547+0000 7f3a45e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:36:58.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:36:58.668 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:36:58.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7c5400640 1 -- 172.21.15.44:0/2062000830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 msgr2=0x7fc7a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7c5400640 1 --2- 172.21.15.44:0/2062000830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7a40051c0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c610 tx=0x7fc7a4005830 comp rx=0 tx=0).stop 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7bf200640 1 -- 172.21.15.44:0/2062000830 reap_dead start 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7c5400640 1 -- 172.21.15.44:0/2062000830 shutdown_connections 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7c5400640 1 -- 172.21.15.44:0/2062000830 >> 172.21.15.44:0/2062000830 conn(0x7fc7c00b0eb0 msgr2=0x7fc7c00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.825+0000 7fc7c5400640 1 -- 172.21.15.44:0/2062000830 shutdown_connections 2024-07-17T08:36:58.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.826+0000 7fc7c5400640 1 -- 172.21.15.44:0/2062000830 wait complete. 2024-07-17T08:36:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.827+0000 7fc7c5400640 1 Processor -- start 2024-07-17T08:36:58.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.827+0000 7fc7c5400640 1 -- start start 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c014e1b0 0x7fc7c01f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c01481a0 0x7fc7c0150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c006d9b0 con 0x7fc7a4004d40 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c006d830 con 0x7fc7c014e1b0 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7c5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c00cb830 con 0x7fc7c01481a0 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:58.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7be800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56988/0 (socket says 172.21.15.44:56988) 2024-07-17T08:36:58.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.828+0000 7fc7be800640 1 -- 172.21.15.44:0/3597390541 learned_addr learned my addr 172.21.15.44:0/3597390541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7bf200640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c01481a0 0x7fc7c0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 -- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c01481a0 msgr2=0x7fc7c0150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c01481a0 0x7fc7c0150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 -- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c014e1b0 msgr2=0x7fc7c01f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c014e1b0 0x7fc7c01f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 -- 172.21.15.44:0/3597390541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c01f57f0 con 0x7fc7a4004d40 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7be800640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000e990 tx=0x7fc7b000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000cbe0 con 0x7fc7a4004d40 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b0004030 con 0x7fc7a4004d40 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004460 con 0x7fc7a4004d40 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.829+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c01f5ad0 con 0x7fc7a4004d40 2024-07-17T08:36:58.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.830+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c00731f0 con 0x7fc7a4004d40 2024-07-17T08:36:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.831+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c015c2f0 con 0x7fc7a4004d40 2024-07-17T08:36:58.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.832+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 146) ==== 98540+0+0 (secure 0 0 0) 0x7fc7b00035f0 con 0x7fc7a4004d40 2024-07-17T08:36:58.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.833+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 8487+0+0 (secure 0 0 0) 0x7fc7b0099370 con 0x7fc7a4004d40 2024-07-17T08:36:58.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:58.837+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc7b0016070 con 0x7fc7a4004d40 2024-07-17T08:36:59.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.087+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc7c015f570 con 0x7fc7a4004d40 2024-07-17T08:36:59.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.444+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 147) ==== 98667+0+0 (secure 0 0 0) 0x7fc7b005d200 con 0x7fc7a4004d40 2024-07-17T08:36:59.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.446+0000 7fc7b6a00640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 0x7fc7a0081d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:59.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.447+0000 7fc7bde00640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 0x7fc7a0081d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:59.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.454+0000 7fc7bde00640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 0x7fc7a0081d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005d00 tx=0x7fc7ac005c50 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:59.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.596+0000 7fc7b6a00640 1 -- 172.21.15.44:0/3597390541 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v137) ==== 104+0+0 (secure 0 0 0) 0x7fc7b0062020 con 0x7fc7a4004d40 2024-07-17T08:36:59.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 msgr2=0x7fc7a0081d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:59.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 0x7fc7a0081d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005d00 tx=0x7fc7ac005c50 comp rx=0 tx=0).stop 2024-07-17T08:36:59.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 msgr2=0x7fc7c01f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:59.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000e990 tx=0x7fc7b000ee50 comp rx=0 tx=0).stop 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 shutdown_connections 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc7a007f950 0x7fc7a0081d40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c01481a0 0x7fc7c0150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c014e1b0 0x7fc7c01f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 --2- 172.21.15.44:0/3597390541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a4004d40 0x7fc7c01f5210 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.600+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 >> 172.21.15.44:0/3597390541 conn(0x7fc7c00b0eb0 msgr2=0x7fc7c00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.601+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 shutdown_connections 2024-07-17T08:36:59.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.601+0000 7fc7c5400640 1 -- 172.21.15.44:0/3597390541 wait complete. 2024-07-17T08:36:59.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:36:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.839+0000 7f0840a00640 1 -- 172.21.15.44:0/696803443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 msgr2=0x7f083c0ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.839+0000 7f0840a00640 1 --2- 172.21.15.44:0/696803443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c0ad250 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f082c009830 tx=0x7f082c02a510 comp rx=0 tx=0).stop 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.839+0000 7f0840a00640 1 -- 172.21.15.44:0/696803443 shutdown_connections 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.839+0000 7f0840a00640 1 --2- 172.21.15.44:0/696803443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c0ad250 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.839+0000 7f0840a00640 1 -- 172.21.15.44:0/696803443 >> 172.21.15.44:0/696803443 conn(0x7f083c0cf050 msgr2=0x7f083c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.840+0000 7f0840a00640 1 -- 172.21.15.44:0/696803443 shutdown_connections 2024-07-17T08:36:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.840+0000 7f0840a00640 1 -- 172.21.15.44:0/696803443 wait complete. 2024-07-17T08:36:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.842+0000 7f0840a00640 1 Processor -- start 2024-07-17T08:36:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.842+0000 7f0840a00640 1 -- start start 2024-07-17T08:36:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083c0ad790 0x7f083c1f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f083c0ae490 0x7f083c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f083c073450 con 0x7f083c155860 2024-07-17T08:36:59.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f083c0732d0 con 0x7f083c0ae490 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0840a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0735d0 con 0x7f083c0ad790 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f083be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f083be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57016/0 (socket says 172.21.15.44:57016) 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f083be00640 1 -- 172.21.15.44:0/3894742243 learned_addr learned my addr 172.21.15.44:0/3894742243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f083b400640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083c0ad790 0x7f083c1f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.844+0000 7f0833e00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f083c0ae490 0x7f083c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 -- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083c0ad790 msgr2=0x7f083c1f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083c0ad790 0x7f083c1f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 -- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f083c0ae490 msgr2=0x7f083c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f083c0ae490 0x7f083c1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:36:59.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 -- 172.21.15.44:0/3894742243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1529f0 con 0x7f083c155860 2024-07-17T08:36:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f083be00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f083400cbc0 tx=0x7f083400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834016020 con 0x7f083c155860 2024-07-17T08:36:59.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f083400ae30 con 0x7f083c155860 2024-07-17T08:36:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834012680 con 0x7f083c155860 2024-07-17T08:36:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.845+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c152cd0 con 0x7f083c155860 2024-07-17T08:36:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.846+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c153290 con 0x7f083c155860 2024-07-17T08:36:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.847+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c073ac0 con 0x7f083c155860 2024-07-17T08:36:59.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.848+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 147) ==== 98667+0+0 (secure 0 0 0) 0x7f0834019020 con 0x7f083c155860 2024-07-17T08:36:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.849+0000 7f0833400640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 0x7f0818078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:36:59.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.849+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 8487+0+0 (secure 0 0 0) 0x7f0834098e60 con 0x7f083c155860 2024-07-17T08:36:59.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.849+0000 7f083b400640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 0x7f0818078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:36:59.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.851+0000 7f083b400640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 0x7f0818078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f082c034000 tx=0x7f082c004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:36:59.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:36:59.856+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0834020030 con 0x7f083c155860 2024-07-17T08:37:00.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.066+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f083c154cc0 con 0x7f083c155860 2024-07-17T08:37:00.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:00.503+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:00.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.599+0000 7f0833400640 1 -- 172.21.15.44:0/3894742243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v251) ==== 181+0+0 (secure 0 0 0) 0x7f0834061ab0 con 0x7f083c155860 2024-07-17T08:37:00.600 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:37:00.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.605+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 msgr2=0x7f0818078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:00.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.605+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 0x7f0818078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f082c034000 tx=0x7f082c004730 comp rx=0 tx=0).stop 2024-07-17T08:37:00.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.605+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 msgr2=0x7f083c09c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:00.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.605+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f083400cbc0 tx=0x7f083400a430 comp rx=0 tx=0).stop 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 shutdown_connections 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f0818076a00 0x7f0818078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f083c155860 0x7f083c09c290 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f083c0ae490 0x7f083c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 --2- 172.21.15.44:0/3894742243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f083c0ad790 0x7f083c1f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 >> 172.21.15.44:0/3894742243 conn(0x7f083c0cf050 msgr2=0x7f083c15dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.606+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 shutdown_connections 2024-07-17T08:37:00.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.607+0000 7f0840a00640 1 -- 172.21.15.44:0/3894742243 wait complete. 2024-07-17T08:37:00.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.842+0000 7fabe0a00640 1 -- 172.21.15.44:0/1536545916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabc0004d40 msgr2=0x7fabc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.842+0000 7fabe0a00640 1 --2- 172.21.15.44:0/1536545916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabc0004d40 0x7fabc00051c0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c640 tx=0x7fabc0005830 comp rx=0 tx=0).stop 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.842+0000 7fabda800640 1 -- 172.21.15.44:0/1536545916 reap_dead start 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.843+0000 7fabe0a00640 1 -- 172.21.15.44:0/1536545916 shutdown_connections 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.843+0000 7fabe0a00640 1 -- 172.21.15.44:0/1536545916 >> 172.21.15.44:0/1536545916 conn(0x7fabdc0b0eb0 msgr2=0x7fabdc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.843+0000 7fabe0a00640 1 -- 172.21.15.44:0/1536545916 shutdown_connections 2024-07-17T08:37:00.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.843+0000 7fabe0a00640 1 -- 172.21.15.44:0/1536545916 wait complete. 2024-07-17T08:37:00.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.844+0000 7fabe0a00640 1 Processor -- start 2024-07-17T08:37:00.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.845+0000 7fabe0a00640 1 -- start start 2024-07-17T08:37:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabc0004d40 0x7fabdc149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0715e0 0x7fabdc14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc15b5c0 con 0x7fabdc0ac190 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc15b440 con 0x7fabc0004d40 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabe0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc06db60 con 0x7fabdc0715e0 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabda800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57034/0 (socket says 172.21.15.44:57034) 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabda800640 1 -- 172.21.15.44:0/892133033 learned_addr learned my addr 172.21.15.44:0/892133033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabd9e00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabc0004d40 0x7fabdc149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.846+0000 7fabd9400640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0715e0 0x7fabdc14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 -- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0715e0 msgr2=0x7fabdc14c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0715e0 0x7fabdc14c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 -- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabc0004d40 msgr2=0x7fabdc149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabc0004d40 0x7fabdc149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 -- 172.21.15.44:0/892133033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc1496d0 con 0x7fabdc0ac190 2024-07-17T08:37:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabda800640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fabd402b490 tx=0x7fabd4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4039070 con 0x7fabdc0ac190 2024-07-17T08:37:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd402bad0 con 0x7fabdc0ac190 2024-07-17T08:37:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4006180 con 0x7fabdc0ac190 2024-07-17T08:37:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc071b10 con 0x7fabdc0ac190 2024-07-17T08:37:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.847+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc0737b0 con 0x7fabdc0ac190 2024-07-17T08:37:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.849+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc0732c0 con 0x7fabdc0ac190 2024-07-17T08:37:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.850+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fabd403c020 con 0x7fabdc0ac190 2024-07-17T08:37:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.851+0000 7fabd2000640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 0x7fabbc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.851+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(251..251 src has 1..251) ==== 8049+0+0 (secure 0 0 0) 0x7fabd40b8e40 con 0x7fabdc0ac190 2024-07-17T08:37:00.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.852+0000 7fabd9e00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 0x7fabbc079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:00.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.853+0000 7fabd9e00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 0x7fabbc079220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009b20 tx=0x7fabcc005f90 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:00.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:00.856+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fabd4081630 con 0x7fabdc0ac190 2024-07-17T08:37:01.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.081+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fabdc06d620 con 0x7fabdc0ac190 2024-07-17T08:37:01.606 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:01.604+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:01.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.621+0000 7fabd2000640 1 -- 172.21.15.44:0/892133033 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v252) ==== 169+0+0 (secure 0 0 0) 0x7fabd4080f30 con 0x7fabdc0ac190 2024-07-17T08:37:01.622 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.627+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 msgr2=0x7fabbc079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.627+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 0x7fabbc079220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009b20 tx=0x7fabcc005f90 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.627+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 msgr2=0x7fabdc1f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.627+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fabd402b490 tx=0x7fabd4003ba0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 shutdown_connections 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fabbc076d60 0x7fabbc079220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabdc0ac190 0x7fabdc1f7280 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fabdc0715e0 0x7fabdc14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 --2- 172.21.15.44:0/892133033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabc0004d40 0x7fabdc149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 >> 172.21.15.44:0/892133033 conn(0x7fabdc0b0eb0 msgr2=0x7fabdc15d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 shutdown_connections 2024-07-17T08:37:01.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.628+0000 7fabe0a00640 1 -- 172.21.15.44:0/892133033 wait complete. 2024-07-17T08:37:01.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.848+0000 7f8f56400640 1 -- 172.21.15.44:0/1874228521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f500ac220 msgr2=0x7f8f500ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.848+0000 7f8f56400640 1 --2- 172.21.15.44:0/1874228521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f500ac220 0x7f8f500ac690 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f8f5015edd0 tx=0x7f8f3802a560 comp rx=0 tx=0).stop 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 -- 172.21.15.44:0/1874228521 shutdown_connections 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 --2- 172.21.15.44:0/1874228521 >> v2:172.21.15.175:3300/0 conn(0x7f8f500cd690 0x7f8f500cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 --2- 172.21.15.44:0/1874228521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f500ac220 0x7f8f500ac690 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 -- 172.21.15.44:0/1874228521 >> 172.21.15.44:0/1874228521 conn(0x7f8f500b0fb0 msgr2=0x7f8f500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:01.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 -- 172.21.15.44:0/1874228521 shutdown_connections 2024-07-17T08:37:01.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.849+0000 7f8f56400640 1 -- 172.21.15.44:0/1874228521 wait complete. 2024-07-17T08:37:01.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.851+0000 7f8f56400640 1 Processor -- start 2024-07-17T08:37:01.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.851+0000 7f8f56400640 1 -- start start 2024-07-17T08:37:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 0x7f8f5015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 0x7f8f5014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:01.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500ce020 con 0x7f8f501522c0 2024-07-17T08:37:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500adf80 con 0x7f8f500cd690 2024-07-17T08:37:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f56400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500ce1a0 con 0x7f8f500ac220 2024-07-17T08:37:01.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57050/0 (socket says 172.21.15.44:57050) 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 -- 172.21.15.44:0/463131404 learned_addr learned my addr 172.21.15.44:0/463131404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4f400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 0x7f8f5015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4ea00640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 0x7f8f5014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 -- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 msgr2=0x7f8f5015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 0x7f8f5015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 -- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 msgr2=0x7f8f5014a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 0x7f8f5014a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4fe00640 1 -- 172.21.15.44:0/463131404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5015dd20 con 0x7f8f501522c0 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4f400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 0x7f8f5015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:01.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.853+0000 7f8f4ea00640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 0x7f8f5014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f4fe00640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e960 tx=0x7f8f4800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48004d20 con 0x7f8f501522c0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4800cb20 con 0x7f8f501522c0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48005020 con 0x7f8f501522c0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5015e000 con 0x7f8f501522c0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.854+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f50073890 con 0x7f8f501522c0 2024-07-17T08:37:01.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.856+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f500ce6c0 con 0x7f8f501522c0 2024-07-17T08:37:01.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.857+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f8f480051c0 con 0x7f8f501522c0 2024-07-17T08:37:01.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.858+0000 7f8f47400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 0x7f8f2c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:01.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.858+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7f8f480998f0 con 0x7f8f501522c0 2024-07-17T08:37:01.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.858+0000 7f8f4f400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 0x7f8f2c079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:01.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.860+0000 7f8f4f400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 0x7f8f2c079220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000a830 tx=0x7f8f40008040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:01.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:01.862+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8f48016030 con 0x7f8f501522c0 2024-07-17T08:37:02.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.125+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f5014aa70 con 0x7f8f501522c0 2024-07-17T08:37:02.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.126+0000 7f8f47400640 1 -- 172.21.15.44:0/463131404 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+5239 (secure 0 0 0) 0x7f8f48062170 con 0x7f8f501522c0 2024-07-17T08:37:02.127 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:02.127 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":137,"btime":"2024-07-17T08:36:59:592606+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":9112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2268738226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2268738226},{"type":"v1","addr":"172.21.15.175:6836","nonce":2268738226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":137}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:37:02.127 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 137 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 msgr2=0x7f8f2c079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 0x7f8f2c079220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000a830 tx=0x7f8f40008040 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 msgr2=0x7f8f50073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800e960 tx=0x7f8f4800ee20 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 shutdown_connections 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f2c076d60 0x7f8f2c079220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f501522c0 0x7f8f50073db0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f500cd690 0x7f8f5014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 --2- 172.21.15.44:0/463131404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f500ac220 0x7f8f5015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 >> 172.21.15.44:0/463131404 conn(0x7f8f500b0fb0 msgr2=0x7f8f500cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:02.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 shutdown_connections 2024-07-17T08:37:02.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.129+0000 7f8f56400640 1 -- 172.21.15.44:0/463131404 wait complete. 2024-07-17T08:37:02.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:02.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 -- 172.21.15.44:0/484366696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 msgr2=0x7f04ac0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 --2- 172.21.15.44:0/484366696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac0b2b70 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f04ac06fae0 tx=0x7f049802a540 comp rx=0 tx=0).stop 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 -- 172.21.15.44:0/484366696 shutdown_connections 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 --2- 172.21.15.44:0/484366696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac0b2b70 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 --2- 172.21.15.44:0/484366696 >> v2:172.21.15.175:3300/0 conn(0x7f04ac159910 0x7f04ac06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 -- 172.21.15.44:0/484366696 >> 172.21.15.44:0/484366696 conn(0x7f04ac0b0fb0 msgr2=0x7f04ac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 -- 172.21.15.44:0/484366696 shutdown_connections 2024-07-17T08:37:02.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.367+0000 7f04b2a00640 1 -- 172.21.15.44:0/484366696 wait complete. 2024-07-17T08:37:02.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.368+0000 7f04b2a00640 1 Processor -- start 2024-07-17T08:37:02.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.369+0000 7f04b2a00640 1 -- start start 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac0b30b0 0x7f04ac152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04ac159910 0x7f04ac1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac0b3850 con 0x7f04ac0acc30 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac0b36d0 con 0x7f04ac0b30b0 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac06d680 con 0x7f04ac159910 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57074/0 (socket says 172.21.15.44:57074) 2024-07-17T08:37:02.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04abe00640 1 -- 172.21.15.44:0/3893708270 learned_addr learned my addr 172.21.15.44:0/3893708270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:02.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04ab400640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac0b30b0 0x7f04ac152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.370+0000 7f04b0a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04ac159910 0x7f04ac1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 -- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04ac159910 msgr2=0x7f04ac1f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04ac159910 0x7f04ac1f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 -- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac0b30b0 msgr2=0x7f04ac152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac0b30b0 0x7f04ac152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 -- 172.21.15.44:0/3893708270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac152b50 con 0x7f04ac0acc30 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04abe00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c9a0 tx=0x7f049c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c007bb0 con 0x7f04ac0acc30 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f049c007e40 con 0x7f04ac0acc30 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c005280 con 0x7f04ac0acc30 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac14d190 con 0x7f04ac0acc30 2024-07-17T08:37:02.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.371+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1f4160 con 0x7f04ac0acc30 2024-07-17T08:37:02.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.373+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac06db80 con 0x7f04ac0acc30 2024-07-17T08:37:02.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.374+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f049c005420 con 0x7f04ac0acc30 2024-07-17T08:37:02.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.375+0000 7f04a3e00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 0x7f048c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.375+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7f049c099310 con 0x7f04ac0acc30 2024-07-17T08:37:02.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.376+0000 7f04ab400640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 0x7f048c079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.377+0000 7f04ab400640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 0x7f048c079220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0498034000 tx=0x7f0498004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:02.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.379+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f049c014030 con 0x7f04ac0acc30 2024-07-17T08:37:02.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.603+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04ac0b27a0 con 0x7f04ac0acc30 2024-07-17T08:37:02.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.606+0000 7f04a3e00640 1 -- 172.21.15.44:0/3893708270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) ==== 74+0+19163 (secure 0 0 0) 0x7f049c08fe20 con 0x7f04ac0acc30 2024-07-17T08:37:02.608 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:02.608 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":252,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:01.617333+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6836/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:6835/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:0/2008194206":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4098747531":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4289249661":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2226805593":"2024-07-18T08:36:58.424347+0000","172.21.15.175:6834/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2343766834":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/210919030":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2618378282":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3172877096":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3847370980":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1726477539":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/458180565":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/4292218033":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/2034094416":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/978474688":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1122698532":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6834/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1151690258":"2024-07-18T08:36:52.077359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.609+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 msgr2=0x7f048c079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.609+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 0x7f048c079220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0498034000 tx=0x7f0498004730 comp rx=0 tx=0).stop 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.609+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 msgr2=0x7f04ac14cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.609+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f049c00c9a0 tx=0x7f049c00ce60 comp rx=0 tx=0).stop 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 shutdown_connections 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f048c076d60 0x7f048c079220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04ac159910 0x7f04ac1f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac0b30b0 0x7f04ac152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 --2- 172.21.15.44:0/3893708270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04ac0acc30 0x7f04ac14cb70 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 >> 172.21.15.44:0/3893708270 conn(0x7f04ac0b0fb0 msgr2=0x7f04ac15c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:02.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 shutdown_connections 2024-07-17T08:37:02.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.610+0000 7f04b2a00640 1 -- 172.21.15.44:0/3893708270 wait complete. 2024-07-17T08:37:02.630 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-17T08:37:02.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:02.669 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:02.668+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 -- 172.21.15.44:0/3767513305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f540b1160 msgr2=0x7f6f540738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3767513305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f540b1160 0x7f6f540738c0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6f54071dc0 tx=0x7f6f4c02aa30 comp rx=0 tx=0).stop 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 -- 172.21.15.44:0/3767513305 shutdown_connections 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3767513305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f540b1160 0x7f6f540738c0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3767513305 >> v2:172.21.15.175:3300/0 conn(0x7f6f54072a10 0x7f6f540b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3767513305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5415d620 0x7f6f5406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 -- 172.21.15.44:0/3767513305 >> 172.21.15.44:0/3767513305 conn(0x7f6f540b5890 msgr2=0x7f6f540b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 -- 172.21.15.44:0/3767513305 shutdown_connections 2024-07-17T08:37:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.852+0000 7f6f53e00640 1 -- 172.21.15.44:0/3767513305 wait complete. 2024-07-17T08:37:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.854+0000 7f6f53e00640 1 Processor -- start 2024-07-17T08:37:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.854+0000 7f6f53e00640 1 -- start start 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54072a10 0x7f6f541f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f540b1160 0x7f6f5414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f54073f80 con 0x7f6f5415d620 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f54073e00 con 0x7f6f540b1160 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f54074100 con 0x7f6f54072a10 2024-07-17T08:37:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57082/0 (socket says 172.21.15.44:57082) 2024-07-17T08:37:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.855+0000 7f6f53400640 1 -- 172.21.15.44:0/927248057 learned_addr learned my addr 172.21.15.44:0/927248057 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f52000640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f540b1160 0x7f6f5414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f52a00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54072a10 0x7f6f541f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 -- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54072a10 msgr2=0x7f6f541f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54072a10 0x7f6f541f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 -- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f540b1160 msgr2=0x7f6f5414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f540b1160 0x7f6f5414f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 -- 172.21.15.44:0/927248057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f541f8390 con 0x7f6f5415d620 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f53400640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d60 tx=0x7f6f4c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.856+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c047070 con 0x7f6f5415d620 2024-07-17T08:37:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.857+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f4c004250 con 0x7f6f5415d620 2024-07-17T08:37:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.857+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c033050 con 0x7f6f5415d620 2024-07-17T08:37:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.857+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f540c9f50 con 0x7f6f5415d620 2024-07-17T08:37:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.857+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f540ca4f0 con 0x7f6f5415d620 2024-07-17T08:37:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.858+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f540cae90 con 0x7f6f5415d620 2024-07-17T08:37:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.859+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f6f4c04a070 con 0x7f6f5415d620 2024-07-17T08:37:02.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.860+0000 7f6f4aa00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 0x7f6f34079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:02.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.860+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7f6f4c0ba260 con 0x7f6f5415d620 2024-07-17T08:37:02.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.861+0000 7f6f52a00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 0x7f6f34079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:02.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.862+0000 7f6f52a00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 0x7f6f34079220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b810 tx=0x7f6f44008040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:02.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:02.865+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6f540cae90 con 0x7f6f5415d620 2024-07-17T08:37:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.120+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f54050670 con 0x7f6f5415d620 2024-07-17T08:37:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.121+0000 7f6f4aa00640 1 -- 172.21.15.44:0/927248057 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+5239 (secure 0 0 0) 0x7f6f54050670 con 0x7f6f5415d620 2024-07-17T08:37:03.122 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:03.122 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":137,"btime":"2024-07-17T08:36:59:592606+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":9112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2268738226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2268738226},{"type":"v1","addr":"172.21.15.175:6836","nonce":2268738226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":137}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:37:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 137 2024-07-17T08:37:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.123+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 msgr2=0x7f6f34079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.123+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 0x7f6f34079220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b810 tx=0x7f6f44008040 comp rx=0 tx=0).stop 2024-07-17T08:37:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.123+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 msgr2=0x7f6f5406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.123+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d60 tx=0x7f6f4c003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 shutdown_connections 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f6f34076d60 0x7f6f34079220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5415d620 0x7f6f5406f5c0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f540b1160 0x7f6f5414f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 --2- 172.21.15.44:0/927248057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f54072a10 0x7f6f541f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 >> 172.21.15.44:0/927248057 conn(0x7f6f540b5890 msgr2=0x7f6f540d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 shutdown_connections 2024-07-17T08:37:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.124+0000 7f6f53e00640 1 -- 172.21.15.44:0/927248057 wait complete. 2024-07-17T08:37:03.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 -- 172.21.15.44:0/2774650592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8910003a50 msgr2=0x7f8910003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 --2- 172.21.15.44:0/2774650592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8910003a50 0x7f8910003e70 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f891c00baf0 tx=0x7f8910005830 comp rx=0 tx=0).stop 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f892b400640 1 -- 172.21.15.44:0/2774650592 reap_dead start 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 -- 172.21.15.44:0/2774650592 shutdown_connections 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 -- 172.21.15.44:0/2774650592 >> 172.21.15.44:0/2774650592 conn(0x7f892c0b0fb0 msgr2=0x7f892c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 -- 172.21.15.44:0/2774650592 shutdown_connections 2024-07-17T08:37:03.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.364+0000 7f8931e00640 1 -- 172.21.15.44:0/2774650592 wait complete. 2024-07-17T08:37:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.366+0000 7f8931e00640 1 Processor -- start 2024-07-17T08:37:03.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.366+0000 7f8931e00640 1 -- start start 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8910003a50 0x7f892c1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f892c1f7ba0 0x7f892c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f892c06f040 con 0x7f892c145eb0 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f892c06eec0 con 0x7f8910003a50 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f8931e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0b25d0 con 0x7f892c1f7ba0 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f892b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f892b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57090/0 (socket says 172.21.15.44:57090) 2024-07-17T08:37:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.367+0000 7f892b400640 1 -- 172.21.15.44:0/183777712 learned_addr learned my addr 172.21.15.44:0/183777712 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892aa00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8910003a50 0x7f892c1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892a000640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f892c1f7ba0 0x7f892c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 -- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f892c1f7ba0 msgr2=0x7f892c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f892c1f7ba0 0x7f892c09ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 -- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8910003a50 msgr2=0x7f892c1f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8910003a50 0x7f892c1f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 -- 172.21.15.44:0/183777712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c13ee10 con 0x7f892c145eb0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f892b400640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f892c15dad0 tx=0x7f892400fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400dc20 con 0x7f892c145eb0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8924011440 con 0x7f892c145eb0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c1f80a0 con 0x7f892c145eb0 2024-07-17T08:37:03.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.368+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924011890 con 0x7f892c145eb0 2024-07-17T08:37:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.369+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1f8660 con 0x7f892c145eb0 2024-07-17T08:37:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.370+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c0b3d50 con 0x7f892c145eb0 2024-07-17T08:37:03.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.371+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f8924011a30 con 0x7f892c145eb0 2024-07-17T08:37:03.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.372+0000 7f8923400640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 0x7f890c079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.372+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7f8924099ab0 con 0x7f892c145eb0 2024-07-17T08:37:03.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.373+0000 7f892aa00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 0x7f890c079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.374+0000 7f892aa00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 0x7f890c079250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f891c00baf0 tx=0x7f891c0046c0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:03.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.376+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f892c0b3d50 con 0x7f892c145eb0 2024-07-17T08:37:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.641+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f892c13efe0 con 0x7f892c145eb0 2024-07-17T08:37:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.642+0000 7f8923400640 1 -- 172.21.15.44:0/183777712 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+5239 (secure 0 0 0) 0x7f8924062460 con 0x7f892c145eb0 2024-07-17T08:37:03.643 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:03.643 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":137,"btime":"2024-07-17T08:36:59:592606+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":9112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2268738226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2268738226},{"type":"v1","addr":"172.21.15.175:6836","nonce":2268738226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":137}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:36:34.218363+0000","modified":"2024-07-17T08:36:37.261234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8785":{"gid":8785,"name":"b","rank":0,"incarnation":132,"state":"up:active","state_seq":6,"addr":"172.21.15.175:6801/3620068660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3620068660},{"type":"v1","addr":"172.21.15.175:6801","nonce":3620068660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8785,"qdb_cluster":[8785]},"id":16}]} 2024-07-17T08:37:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 137 2024-07-17T08:37:03.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 msgr2=0x7f890c079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 0x7f890c079250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f891c00baf0 tx=0x7f891c0046c0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 msgr2=0x7f892c146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f892c15dad0 tx=0x7f892400fd20 comp rx=0 tx=0).stop 2024-07-17T08:37:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 shutdown_connections 2024-07-17T08:37:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f890c076d90 0x7f890c079250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f892c145eb0 0x7f892c146310 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f892c1f7ba0 0x7f892c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 --2- 172.21.15.44:0/183777712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8910003a50 0x7f892c1f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 >> 172.21.15.44:0/183777712 conn(0x7f892c0b0fb0 msgr2=0x7f892c15edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.645+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 shutdown_connections 2024-07-17T08:37:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.646+0000 7f8931e00640 1 -- 172.21.15.44:0/183777712 wait complete. 2024-07-17T08:37:03.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:03.784 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:03.782+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:03.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 -- 172.21.15.44:0/2563492754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c81522c0 msgr2=0x7fa0c80ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 --2- 172.21.15.44:0/2563492754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c81522c0 0x7fa0c80ad790 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8005fd0 tx=0x7fa0b802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 -- 172.21.15.44:0/2563492754 shutdown_connections 2024-07-17T08:37:03.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 --2- 172.21.15.44:0/2563492754 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80adcd0 0x7fa0c80ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 --2- 172.21.15.44:0/2563492754 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c81522c0 0x7fa0c80ad790 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 -- 172.21.15.44:0/2563492754 >> 172.21.15.44:0/2563492754 conn(0x7fa0c80b0fb0 msgr2=0x7fa0c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:03.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.880+0000 7fa0cec00640 1 -- 172.21.15.44:0/2563492754 shutdown_connections 2024-07-17T08:37:03.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.881+0000 7fa0cec00640 1 -- 172.21.15.44:0/2563492754 wait complete. 2024-07-17T08:37:03.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.882+0000 7fa0cec00640 1 Processor -- start 2024-07-17T08:37:03.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.882+0000 7fa0cec00640 1 -- start start 2024-07-17T08:37:03.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80ac760 0x7fa0c8148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c80adcd0 0x7fa0c8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c815f5a0 con 0x7fa0c81522c0 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c80ce8b0 con 0x7fa0c80adcd0 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.883+0000 7fa0cec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c815f720 con 0x7fa0c80ac760 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57100/0 (socket says 172.21.15.44:57100) 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 -- 172.21.15.44:0/1615613255 learned_addr learned my addr 172.21.15.44:0/1615613255 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:03.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0c7400640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c80adcd0 0x7fa0c8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0c7e00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80ac760 0x7fa0c8148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 -- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80ac760 msgr2=0x7fa0c8148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80ac760 0x7fa0c8148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 -- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c80adcd0 msgr2=0x7fa0c8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c80adcd0 0x7fa0c8074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.884+0000 7fa0ccc00640 1 -- 172.21.15.44:0/1615613255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c81f2300 con 0x7fa0c81522c0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0ccc00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c730 tx=0x7fa0c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003b70 con 0x7fa0c81522c0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0c0003d10 con 0x7fa0c81522c0 2024-07-17T08:37:03.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000aab0 con 0x7fa0c81522c0 2024-07-17T08:37:03.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c81f25e0 con 0x7fa0c81522c0 2024-07-17T08:37:03.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.885+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81f2ba0 con 0x7fa0c81522c0 2024-07-17T08:37:03.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.886+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c806a1a0 con 0x7fa0c81522c0 2024-07-17T08:37:03.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.887+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fa0c000ac50 con 0x7fa0c81522c0 2024-07-17T08:37:03.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.888+0000 7fa0bfe00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 0x7fa0a8079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:03.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.889+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7fa0c0098d00 con 0x7fa0c81522c0 2024-07-17T08:37:03.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.889+0000 7fa0c7e00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 0x7fa0a8079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:03.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.890+0000 7fa0c7e00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 0x7fa0a8079220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8033000 tx=0x7fa0b8004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:03.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:03.893+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa0c806a1a0 con 0x7fa0c81522c0 2024-07-17T08:37:04.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.121+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0c815fc40 con 0x7fa0c81522c0 2024-07-17T08:37:04.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.123+0000 7fa0bfe00640 1 -- 172.21.15.44:0/1615613255 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) ==== 74+0+19163 (secure 0 0 0) 0x7fa0c0061720 con 0x7fa0c81522c0 2024-07-17T08:37:04.125 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:04.125 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":252,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:01.617333+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2024-07-17T08:36:32.650016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"246","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2024-07-17T08:36:33.174749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6836/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:6835/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:0/2008194206":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4098747531":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4289249661":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2226805593":"2024-07-18T08:36:58.424347+0000","172.21.15.175:6834/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2343766834":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/210919030":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2618378282":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3172877096":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3847370980":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1726477539":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/458180565":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/4292218033":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/2034094416":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/978474688":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1122698532":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6834/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1151690258":"2024-07-18T08:36:52.077359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:04.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.126+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 msgr2=0x7fa0a8079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.126+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 0x7fa0a8079220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8033000 tx=0x7fa0b8004730 comp rx=0 tx=0).stop 2024-07-17T08:37:04.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.126+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 msgr2=0x7fa0c815ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.126+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c730 tx=0x7fa0c000cbf0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 shutdown_connections 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa0a8076d60 0x7fa0a8079220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81522c0 0x7fa0c815ddc0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0c80adcd0 0x7fa0c8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 --2- 172.21.15.44:0/1615613255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0c80ac760 0x7fa0c8148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 >> 172.21.15.44:0/1615613255 conn(0x7fa0c80b0fb0 msgr2=0x7fa0c806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 shutdown_connections 2024-07-17T08:37:04.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.127+0000 7fa0cec00640 1 -- 172.21.15.44:0/1615613255 wait complete. 2024-07-17T08:37:04.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 -- 172.21.15.44:0/1035364774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f48152220 msgr2=0x7f5f4806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 --2- 172.21.15.44:0/1035364774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f48152220 0x7f5f4806d620 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009830 tx=0x7f5f3802a490 comp rx=0 tx=0).stop 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 -- 172.21.15.44:0/1035364774 shutdown_connections 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 --2- 172.21.15.44:0/1035364774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f4815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 --2- 172.21.15.44:0/1035364774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f48152220 0x7f5f4806d620 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 -- 172.21.15.44:0/1035364774 >> 172.21.15.44:0/1035364774 conn(0x7f5f480b0eb0 msgr2=0x7f5f480b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 -- 172.21.15.44:0/1035364774 shutdown_connections 2024-07-17T08:37:04.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.358+0000 7f5f4f200640 1 -- 172.21.15.44:0/1035364774 wait complete. 2024-07-17T08:37:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.360+0000 7f5f4f200640 1 Processor -- start 2024-07-17T08:37:04.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.360+0000 7f5f4f200640 1 -- start start 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f480b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f48152220 0x7f5f4815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f480cd780 con 0x7f5f4806ed30 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f480cd600 con 0x7f5f48152220 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f4f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f480cd900 con 0x7f5f480cbab0 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57116/0 (socket says 172.21.15.44:57116) 2024-07-17T08:37:04.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f47e00640 1 -- 172.21.15.44:0/2392352251 learned_addr learned my addr 172.21.15.44:0/2392352251 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:04.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.361+0000 7f5f47e00640 1 -- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 msgr2=0x7f5f480b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f3fe00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f480b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f47e00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f480b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f47e00640 1 -- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f48152220 msgr2=0x7f5f4815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f47e00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f48152220 0x7f5f4815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f47e00640 1 -- 172.21.15.44:0/2392352251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f48158730 con 0x7f5f4806ed30 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f3fe00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f480b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f47e00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f5f38000c00 tx=0x7f5f38033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38031d60 con 0x7f5f4806ed30 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f380330d0 con 0x7f5f4806ed30 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38033520 con 0x7f5f4806ed30 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f48158a10 con 0x7f5f4806ed30 2024-07-17T08:37:04.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.362+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f480b2960 con 0x7f5f4806ed30 2024-07-17T08:37:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.364+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f480cde20 con 0x7f5f4806ed30 2024-07-17T08:37:04.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.365+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f5f380035f0 con 0x7f5f4806ed30 2024-07-17T08:37:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.366+0000 7f5f44c00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 0x7f5f24079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.367+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7635+0+0 (secure 0 0 0) 0x7f5f380ba2b0 con 0x7f5f4806ed30 2024-07-17T08:37:04.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.367+0000 7f5f3fe00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 0x7f5f24079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.368+0000 7f5f3fe00640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 0x7f5f24079220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f30007fd0 tx=0x7f5f30009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:04.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.371+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f3803f030 con 0x7f5f4806ed30 2024-07-17T08:37:04.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.639+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7f5f4815bb10 con 0x7f5f4806ed30 2024-07-17T08:37:04.646 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:37:04.645+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:37:04.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.655+0000 7f5f44c00640 1 -- 172.21.15.44:0/2392352251 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v138) ==== 167+0+0 (secure 0 0 0) 0x7f5f38082c50 con 0x7f5f4806ed30 2024-07-17T08:37:04.656 INFO:teuthology.orchestra.run.smithi044.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:37:04.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 msgr2=0x7f5f24079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 0x7f5f24079220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f30007fd0 tx=0x7f5f30009040 comp rx=0 tx=0).stop 2024-07-17T08:37:04.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 msgr2=0x7f5f4815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f5f38000c00 tx=0x7f5f38033d60 comp rx=0 tx=0).stop 2024-07-17T08:37:04.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 shutdown_connections 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5f24076d60 0x7f5f24079220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f48152220 0x7f5f4815d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f480cbab0 0x7f5f480b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 --2- 172.21.15.44:0/2392352251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f4806ed30 0x7f5f4815f5e0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.659+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 >> 172.21.15.44:0/2392352251 conn(0x7f5f480b0eb0 msgr2=0x7f5f480cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.660+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 shutdown_connections 2024-07-17T08:37:04.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.660+0000 7f5f4f200640 1 -- 172.21.15.44:0/2392352251 wait complete. 2024-07-17T08:37:04.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-17T08:37:04.682 INFO:tasks.ceph.mds.b.smithi175.stderr: -1630> 2024-07-17T08:36:39.547+0000 7f6a35c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:37:04.752 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:37:04.869 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:04.868+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:04.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.894+0000 7f2329400640 1 -- 172.21.15.44:0/282925786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232415d970 msgr2=0x7f23240b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.894+0000 7f2329400640 1 --2- 172.21.15.44:0/282925786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232415d970 0x7f23240b37c0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f23240b08b0 tx=0x7f231002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 -- 172.21.15.44:0/282925786 shutdown_connections 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 --2- 172.21.15.44:0/282925786 >> v2:172.21.15.175:3300/0 conn(0x7f23240b0b10 0x7f23240b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 --2- 172.21.15.44:0/282925786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f232415d970 0x7f23240b37c0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 --2- 172.21.15.44:0/282925786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d590 0x7f23240ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 -- 172.21.15.44:0/282925786 >> 172.21.15.44:0/282925786 conn(0x7f232409aa30 msgr2=0x7f23240b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 -- 172.21.15.44:0/282925786 shutdown_connections 2024-07-17T08:37:04.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.895+0000 7f2329400640 1 -- 172.21.15.44:0/282925786 wait complete. 2024-07-17T08:37:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.896+0000 7f2329400640 1 Processor -- start 2024-07-17T08:37:04.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.897+0000 7f2329400640 1 -- start start 2024-07-17T08:37:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.897+0000 7f2329400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2329400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f232415d590 0x7f23241f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2329400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d970 0x7f23241f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2329400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23240cab10 con 0x7f23240b0b10 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2329400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2324161490 con 0x7f232415d590 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2329400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23240cac90 con 0x7f232415d970 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57126/0 (socket says 172.21.15.44:57126) 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 -- 172.21.15.44:0/4053085746 learned_addr learned my addr 172.21.15.44:0/4053085746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2321e00640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f232415d590 0x7f23241f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2323200640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d970 0x7f23241f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 -- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d970 msgr2=0x7f23241f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d970 0x7f23241f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 -- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f232415d590 msgr2=0x7f23241f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:04.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f232415d590 0x7f23241f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:04.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.898+0000 7f2322800640 1 -- 172.21.15.44:0/4053085746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23240b6320 con 0x7f23240b0b10 2024-07-17T08:37:04.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f2322800640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f231400d950 tx=0x7f231400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:04.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314017070 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2314009d20 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23140106c0 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23240b6600 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.899+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23240b6bc0 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.900+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23240ae390 con 0x7f23240b0b10 2024-07-17T08:37:04.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.902+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f2314010860 con 0x7f23240b0b10 2024-07-17T08:37:04.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.903+0000 7f231aa00640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 0x7f2304079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:04.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.903+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 7721+0+0 (secure 0 0 0) 0x7f2314099200 con 0x7f23240b0b10 2024-07-17T08:37:04.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.904+0000 7f2321e00640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 0x7f2304079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:04.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.905+0000 7f2321e00640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 0x7f2304079220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:04.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:04.908+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2314061b50 con 0x7f23240b0b10 2024-07-17T08:37:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.175+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f2324074300 con 0x7f23240b0b10 2024-07-17T08:37:05.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.659+0000 7f231aa00640 1 -- 172.21.15.44:0/4053085746 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v139) ==== 107+0+0 (secure 0 0 0) 0x7f2314061450 con 0x7f23240b0b10 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 msgr2=0x7f2304079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 0x7f2304079220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2310033000 tx=0x7f2310004730 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 msgr2=0x7f23241f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f231400d950 tx=0x7f231400de10 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 shutdown_connections 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2304076d60 0x7f2304079220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f232415d970 0x7f23241f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f232415d590 0x7f23241f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 --2- 172.21.15.44:0/4053085746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23240b0b10 0x7f23241f76e0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 >> 172.21.15.44:0/4053085746 conn(0x7f232409aa30 msgr2=0x7f23240b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 shutdown_connections 2024-07-17T08:37:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.663+0000 7f2329400640 1 -- 172.21.15.44:0/4053085746 wait complete. 2024-07-17T08:37:05.687 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-17T08:37:05.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.892+0000 7f464d600640 1 -- 172.21.15.44:0/250765233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 msgr2=0x7f46481543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.892+0000 7f464d600640 1 --2- 172.21.15.44:0/250765233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481543a0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f46480ad990 tx=0x7f464002a9e0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 -- 172.21.15.44:0/250765233 shutdown_connections 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 --2- 172.21.15.44:0/250765233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481543a0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 --2- 172.21.15.44:0/250765233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464815d930 0x7f4648153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 -- 172.21.15.44:0/250765233 >> 172.21.15.44:0/250765233 conn(0x7f46480cf050 msgr2=0x7f46480cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 -- 172.21.15.44:0/250765233 shutdown_connections 2024-07-17T08:37:05.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.893+0000 7f464d600640 1 -- 172.21.15.44:0/250765233 wait complete. 2024-07-17T08:37:05.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.894+0000 7f464d600640 1 Processor -- start 2024-07-17T08:37:05.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.895+0000 7f464d600640 1 -- start start 2024-07-17T08:37:05.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.895+0000 7f464d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4648074020 0x7f4648156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:05.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.895+0000 7f464d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:05.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f464d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464815d930 0x7f4648157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f464d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46480ae4b0 con 0x7f4648074400 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f464d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46480ae330 con 0x7f4648074020 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f464d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46480ae630 con 0x7f464815d930 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57144/0 (socket says 172.21.15.44:57144) 2024-07-17T08:37:05.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 -- 172.21.15.44:0/489251553 learned_addr learned my addr 172.21.15.44:0/489251553 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 -- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464815d930 msgr2=0x7f4648157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464815d930 0x7f4648157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 -- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4648074020 msgr2=0x7f4648156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4648074020 0x7f4648156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 -- 172.21.15.44:0/489251553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46481f4a80 con 0x7f4648074400 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.896+0000 7f463fe00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f463000c970 tx=0x7f463000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.897+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630007bb0 con 0x7f4648074400 2024-07-17T08:37:05.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.897+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4630007e40 con 0x7f4648074400 2024-07-17T08:37:05.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.897+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463000f690 con 0x7f4648074400 2024-07-17T08:37:05.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.897+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46481f4d60 con 0x7f4648074400 2024-07-17T08:37:05.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.897+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4648154c30 con 0x7f4648074400 2024-07-17T08:37:05.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.898+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464806eb10 con 0x7f4648074400 2024-07-17T08:37:05.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.899+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f463000f830 con 0x7f4648074400 2024-07-17T08:37:05.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.900+0000 7f4644c00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 0x7f4624079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:05.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.900+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 7721+0+0 (secure 0 0 0) 0x7f4630099670 con 0x7f4648074400 2024-07-17T08:37:05.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.901+0000 7f4647e00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 0x7f4624079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:05.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.902+0000 7f4647e00640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 0x7f4624079220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f464815a320 tx=0x7f4638007990 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:05.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:05.905+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f464806eb10 con 0x7f4648074400 2024-07-17T08:37:05.990 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:05.988+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:06.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.126+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46480ac0f0 con 0x7f4648074400 2024-07-17T08:37:06.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.670+0000 7f4644c00640 1 -- 172.21.15.44:0/489251553 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v254) ==== 190+0+0 (secure 0 0 0) 0x7f463006a030 con 0x7f4648074400 2024-07-17T08:37:06.673 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_metadata' removed 2024-07-17T08:37:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.675+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 msgr2=0x7f4624079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.675+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 0x7f4624079220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f464815a320 tx=0x7f4638007990 comp rx=0 tx=0).stop 2024-07-17T08:37:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.675+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 msgr2=0x7f46481f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.675+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f463000c970 tx=0x7f463000ce30 comp rx=0 tx=0).stop 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 shutdown_connections 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4624076d60 0x7f4624079220 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464815d930 0x7f4648157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4648074400 0x7f46481f41a0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 --2- 172.21.15.44:0/489251553 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4648074020 0x7f4648156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.676+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 >> 172.21.15.44:0/489251553 conn(0x7f46480cf050 msgr2=0x7f464815c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.682+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 shutdown_connections 2024-07-17T08:37:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.682+0000 7f464d600640 1 -- 172.21.15.44:0/489251553 wait complete. 2024-07-17T08:37:06.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.920+0000 7f378f200640 1 -- 172.21.15.44:0/3121669485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f378815b060 msgr2=0x7f378806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.920+0000 7f378f200640 1 --2- 172.21.15.44:0/3121669485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f378815b060 0x7f378806d620 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f3778009830 tx=0x7f377802a4d0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 -- 172.21.15.44:0/3121669485 shutdown_connections 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 --2- 172.21.15.44:0/3121669485 >> v2:172.21.15.175:3300/0 conn(0x7f37880cb880 0x7f378806db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 --2- 172.21.15.44:0/3121669485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f378815b060 0x7f378806d620 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 -- 172.21.15.44:0/3121669485 >> 172.21.15.44:0/3121669485 conn(0x7f37880b0eb0 msgr2=0x7f37880b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 -- 172.21.15.44:0/3121669485 shutdown_connections 2024-07-17T08:37:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.921+0000 7f378f200640 1 -- 172.21.15.44:0/3121669485 wait complete. 2024-07-17T08:37:06.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.923+0000 7f378f200640 1 Processor -- start 2024-07-17T08:37:06.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.923+0000 7f378f200640 1 -- start start 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f378815b060 0x7f3788155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f378815c220 0x7f37881454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f378815b5c0 con 0x7f37880cb880 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f378815b440 con 0x7f378815b060 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f378f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f378815f4a0 con 0x7f378815c220 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57160/0 (socket says 172.21.15.44:57160) 2024-07-17T08:37:06.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 -- 172.21.15.44:0/1227897970 learned_addr learned my addr 172.21.15.44:0/1227897970 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:06.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f377f400640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f378815b060 0x7f3788155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:06.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 -- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f378815c220 msgr2=0x7f37881454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:06.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f378815c220 0x7f37881454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.924+0000 7f3787e00640 1 -- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f378815b060 msgr2=0x7f3788155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3787e00640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f378815b060 0x7f3788155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3787e00640 1 -- 172.21.15.44:0/1227897970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3788159920 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3787e00640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3778000c00 tx=0x7f3778032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778040070 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3778003af0 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778045460 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3788159c00 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.925+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378815a1c0 con 0x7f37880cb880 2024-07-17T08:37:06.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.926+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378815f9c0 con 0x7f37880cb880 2024-07-17T08:37:06.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.928+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f3778039040 con 0x7f37880cb880 2024-07-17T08:37:06.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.928+0000 7f3784c00640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 0x7f3764079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:06.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.929+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 7277+0+0 (secure 0 0 0) 0x7f37780ba340 con 0x7f37880cb880 2024-07-17T08:37:06.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.929+0000 7f377f400640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 0x7f3764079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:06.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.931+0000 7f377f400640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 0x7f3764079220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37881f7290 tx=0x7f3770005c60 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:06.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:06.934+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3778082e50 con 0x7f37880cb880 2024-07-17T08:37:07.140 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:07.138+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:07.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.151+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37880b2240 con 0x7f37880cb880 2024-07-17T08:37:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.686+0000 7f3784c00640 1 -- 172.21.15.44:0/1227897970 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v255) ==== 178+0+0 (secure 0 0 0) 0x7f3778082750 con 0x7f37880cb880 2024-07-17T08:37:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'backup_fs_data' removed 2024-07-17T08:37:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 msgr2=0x7f3764079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 0x7f3764079220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37881f7290 tx=0x7f3770005c60 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 msgr2=0x7f37880c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f3778000c00 tx=0x7f3778032d50 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 shutdown_connections 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3764076d60 0x7f3764079220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f378815c220 0x7f37881454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f378815b060 0x7f3788155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 --2- 172.21.15.44:0/1227897970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37880cb880 0x7f37880c9c10 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 >> 172.21.15.44:0/1227897970 conn(0x7f37880b0eb0 msgr2=0x7f378815e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 shutdown_connections 2024-07-17T08:37:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.690+0000 7f378f200640 1 -- 172.21.15.44:0/1227897970 wait complete. 2024-07-17T08:37:07.719 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:37:07.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.933+0000 7f2c3c600640 1 -- 172.21.15.44:0/2923465582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 msgr2=0x7f2c1c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.933+0000 7f2c3c600640 1 --2- 172.21.15.44:0/2923465582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c1c003ec0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009b20 tx=0x7f2c1c005830 comp rx=0 tx=0).stop 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.934+0000 7f2c3c600640 1 -- 172.21.15.44:0/2923465582 shutdown_connections 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.934+0000 7f2c3c600640 1 --2- 172.21.15.44:0/2923465582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c1c003ec0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.934+0000 7f2c3c600640 1 -- 172.21.15.44:0/2923465582 >> 172.21.15.44:0/2923465582 conn(0x7f2c34073d50 msgr2=0x7f2c3409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.934+0000 7f2c3c600640 1 -- 172.21.15.44:0/2923465582 shutdown_connections 2024-07-17T08:37:07.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.934+0000 7f2c3c600640 1 -- 172.21.15.44:0/2923465582 wait complete. 2024-07-17T08:37:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.935+0000 7f2c3c600640 1 Processor -- start 2024-07-17T08:37:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 -- start start 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3415ff60 0x7f2c341603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3409b5f0 0x7f2c3409ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c340b5fb0 con 0x7f2c1c003aa0 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.936+0000 7f2c3c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406dae0 con 0x7f2c3409b5f0 2024-07-17T08:37:07.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c3c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c340b6130 con 0x7f2c3415ff60 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57164/0 (socket says 172.21.15.44:57164) 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 -- 172.21.15.44:0/3750331867 learned_addr learned my addr 172.21.15.44:0/3750331867 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39200640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3415ff60 0x7f2c341603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c3a600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3409b5f0 0x7f2c3409ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 -- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3415ff60 msgr2=0x7f2c341603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3415ff60 0x7f2c341603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 -- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3409b5f0 msgr2=0x7f2c3409ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3409b5f0 0x7f2c3409ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.937+0000 7f2c39c00640 1 -- 172.21.15.44:0/3750331867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34148d80 con 0x7f2c1c003aa0 2024-07-17T08:37:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c39c00640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f2c28036040 tx=0x7f2c280336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28038040 con 0x7f2c1c003aa0 2024-07-17T08:37:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c280338e0 con 0x7f2c1c003aa0 2024-07-17T08:37:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c34149060 con 0x7f2c1c003aa0 2024-07-17T08:37:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28033d30 con 0x7f2c1c003aa0 2024-07-17T08:37:07.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.938+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c34149620 con 0x7f2c1c003aa0 2024-07-17T08:37:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.939+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c34070740 con 0x7f2c1c003aa0 2024-07-17T08:37:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.940+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f2c28043020 con 0x7f2c1c003aa0 2024-07-17T08:37:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.941+0000 7f2c31600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 0x7f2c14079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.942+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f2c280b8980 con 0x7f2c1c003aa0 2024-07-17T08:37:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.942+0000 7f2c39200640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 0x7f2c14079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:07.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.943+0000 7f2c39200640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 0x7f2c14079250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c20001860 tx=0x7f2c20009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:07.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:07.946+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2c2804a020 con 0x7f2c1c003aa0 2024-07-17T08:37:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.185+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f2c34160900 con 0x7f2c1c003aa0 2024-07-17T08:37:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.193+0000 7f2c31600640 1 -- 172.21.15.44:0/3750331867 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v79) ==== 71+0+0 (secure 0 0 0) 0x7f2c34160900 con 0x7f2c1c003aa0 2024-07-17T08:37:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 msgr2=0x7f2c14079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 0x7f2c14079250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c20001860 tx=0x7f2c20009040 comp rx=0 tx=0).stop 2024-07-17T08:37:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 msgr2=0x7f2c34146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f2c28036040 tx=0x7f2c280336f0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 shutdown_connections 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f2c14076d90 0x7f2c14079250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c3409b5f0 0x7f2c3409ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3415ff60 0x7f2c341603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 --2- 172.21.15.44:0/3750331867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c003aa0 0x7f2c34146270 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.196+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 >> 172.21.15.44:0/3750331867 conn(0x7f2c34073d50 msgr2=0x7f2c3406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.197+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 shutdown_connections 2024-07-17T08:37:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.197+0000 7f2c3c600640 1 -- 172.21.15.44:0/3750331867 wait complete. 2024-07-17T08:37:08.217 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2024-07-17T08:37:08.296 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:08.294+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 -- 172.21.15.44:0/781851294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac0cf920 msgr2=0x7f64ac140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 --2- 172.21.15.44:0/781851294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac0cf920 0x7f64ac140030 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0b33c0 tx=0x7f649802a530 comp rx=0 tx=0).stop 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 -- 172.21.15.44:0/781851294 shutdown_connections 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 --2- 172.21.15.44:0/781851294 >> v2:172.21.15.175:3300/0 conn(0x7f64ac140570 0x7f64ac1409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 --2- 172.21.15.44:0/781851294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac0cf920 0x7f64ac140030 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 -- 172.21.15.44:0/781851294 >> 172.21.15.44:0/781851294 conn(0x7f64ac0b0e80 msgr2=0x7f64ac0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 -- 172.21.15.44:0/781851294 shutdown_connections 2024-07-17T08:37:08.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.429+0000 7f64b2400640 1 -- 172.21.15.44:0/781851294 wait complete. 2024-07-17T08:37:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.431+0000 7f64b2400640 1 Processor -- start 2024-07-17T08:37:08.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.431+0000 7f64b2400640 1 -- start start 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.432+0000 7f64b2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0cf540 0x7f64ac155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.432+0000 7f64b2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64ac0cf920 0x7f64ac1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.432+0000 7f64b2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0ac1e0 con 0x7f64ac140570 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac0ac060 con 0x7f64ac0cf540 2024-07-17T08:37:08.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac140f20 con 0x7f64ac0cf920 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57176/0 (socket says 172.21.15.44:57176) 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 -- 172.21.15.44:0/3387427435 learned_addr learned my addr 172.21.15.44:0/3387427435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64abe00640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64ac0cf920 0x7f64ac1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1000640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0cf540 0x7f64ac155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 -- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64ac0cf920 msgr2=0x7f64ac1f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64ac0cf920 0x7f64ac1f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 -- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0cf540 msgr2=0x7f64ac155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:08.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0cf540 0x7f64ac155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.433+0000 7f64b1a00640 1 -- 172.21.15.44:0/3387427435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac071880 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64b1a00640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f64a400cab0 tx=0x7f64a400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4007d90 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64a400f540 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac071b60 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a400f990 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.434+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac072120 con 0x7f64ac140570 2024-07-17T08:37:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.436+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac1413e0 con 0x7f64ac140570 2024-07-17T08:37:08.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.437+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f64a400fb30 con 0x7f64ac140570 2024-07-17T08:37:08.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.438+0000 7f64a8c00640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 0x7f648c079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:08.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.438+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f64a4099630 con 0x7f64ac140570 2024-07-17T08:37:08.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.438+0000 7f64b1000640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 0x7f648c079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:08.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.440+0000 7f64b1000640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 0x7f648c079250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f64a0000f60 tx=0x7f64a0009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:08.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.442+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f64a4014070 con 0x7f64ac140570 2024-07-17T08:37:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:08.758+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) -- 0x7f64ac158ae0 con 0x7f64ac140570 2024-07-17T08:37:09.441 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:09.439+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:09.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.692+0000 7f64a8c00640 1 -- 172.21.15.44:0/3387427435 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3369) ==== 187+0+0 (secure 0 0 0) 0x7f64a400fe20 con 0x7f64ac140570 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.696+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 msgr2=0x7f648c079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.696+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 0x7f648c079250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f64a0000f60 tx=0x7f64a0009040 comp rx=0 tx=0).stop 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 msgr2=0x7f64ac13cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f64a400cab0 tx=0x7f64a400cf70 comp rx=0 tx=0).stop 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 shutdown_connections 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f648c076d90 0x7f648c079250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64ac140570 0x7f64ac13cca0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f64ac0cf920 0x7f64ac1f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 --2- 172.21.15.44:0/3387427435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64ac0cf540 0x7f64ac155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 >> 172.21.15.44:0/3387427435 conn(0x7f64ac0b0e80 msgr2=0x7f64ac0af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 shutdown_connections 2024-07-17T08:37:09.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.697+0000 7f64b2400640 1 -- 172.21.15.44:0/3387427435 wait complete. 2024-07-17T08:37:09.722 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2024-07-17T08:37:09.722 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2024-07-17T08:37:09.722 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2024-07-17T08:37:09.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 -- 172.21.15.44:0/1235641163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ae810 msgr2=0x7fd88815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 --2- 172.21.15.44:0/1235641163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ae810 0x7fd88815e510 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802a580 comp rx=0 tx=0).stop 2024-07-17T08:37:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 -- 172.21.15.44:0/1235641163 shutdown_connections 2024-07-17T08:37:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 --2- 172.21.15.44:0/1235641163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ae810 0x7fd88815e510 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 -- 172.21.15.44:0/1235641163 >> 172.21.15.44:0/1235641163 conn(0x7fd8880b0fb0 msgr2=0x7fd8880b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 -- 172.21.15.44:0/1235641163 shutdown_connections 2024-07-17T08:37:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.936+0000 7fd88e000640 1 -- 172.21.15.44:0/1235641163 wait complete. 2024-07-17T08:37:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.938+0000 7fd88e000640 1 Processor -- start 2024-07-17T08:37:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.938+0000 7fd88e000640 1 -- start start 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 0x7fd88809be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880ae810 0x7fd8881f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd88806f040 con 0x7fd8880ac930 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd88806eec0 con 0x7fd88806ea50 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd88e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880b2ce0 con 0x7fd8880ae810 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.939+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47020/0 (socket says 172.21.15.44:47020) 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 -- 172.21.15.44:0/661497890 learned_addr learned my addr 172.21.15.44:0/661497890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd887400640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 0x7fd88809be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 -- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880ae810 msgr2=0x7fd8881f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880ae810 0x7fd8881f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 -- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 msgr2=0x7fd88809be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 0x7fd88809be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 -- 172.21.15.44:0/661497890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8881f49f0 con 0x7fd8880ac930 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd887400640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 0x7fd88809be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:09.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.940+0000 7fd87fe00640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c730 tx=0x7fd87000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.941+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870017070 con 0x7fd8880ac930 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.941+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd870005180 con 0x7fd8880ac930 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.941+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000a640 con 0x7fd8880ac930 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.941+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881f4cd0 con 0x7fd8880ac930 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.941+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881556a0 con 0x7fd8880ac930 2024-07-17T08:37:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.942+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8880b23b0 con 0x7fd8880ac930 2024-07-17T08:37:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.943+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fd87001a070 con 0x7fd8880ac930 2024-07-17T08:37:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.945+0000 7fd87f400640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 0x7fd864079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.945+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7fd870097f70 con 0x7fd8880ac930 2024-07-17T08:37:09.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.945+0000 7fd887400640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 0x7fd864079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.946+0000 7fd887400640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 0x7fd864079220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd878033000 tx=0x7fd878004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:09.949+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd88815ea50 con 0x7fd8880ac930 2024-07-17T08:37:10.130 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:37:10.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:37:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.172+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd888161ca0 con 0x7fd8880ac930 2024-07-17T08:37:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.174+0000 7fd87f400640 1 -- 172.21.15.44:0/661497890 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) ==== 74+0+15569 (secure 0 0 0) 0x7fd870060ca0 con 0x7fd8880ac930 2024-07-17T08:37:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:10.175 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":255,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:07.671359+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6801/3620068660":"2024-07-18T08:37:04.645697+0000","172.21.15.175:6800/3620068660":"2024-07-18T08:37:04.645697+0000","172.21.15.175:6836/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:6835/1736454513":"2024-07-18T08:36:58.589623+0000","172.21.15.175:0/2008194206":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4098747531":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/4289249661":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2226805593":"2024-07-18T08:36:58.424347+0000","172.21.15.175:6834/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2343766834":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/210919030":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/2618378282":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3172877096":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/3847370980":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1726477539":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/1443688525":"2024-07-18T08:36:58.424347+0000","172.21.15.175:0/458180565":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/4292218033":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/2034094416":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/978474688":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6837/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1122698532":"2024-07-18T08:36:52.077359+0000","172.21.15.175:6834/189954344":"2024-07-18T08:36:52.077359+0000","172.21.15.175:0/1151690258":"2024-07-18T08:36:52.077359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 msgr2=0x7fd864079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 0x7fd864079220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd878033000 tx=0x7fd878004730 comp rx=0 tx=0).stop 2024-07-17T08:37:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 msgr2=0x7fd8881f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c730 tx=0x7fd87000cbf0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 shutdown_connections 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd864076d60 0x7fd864079220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8880ae810 0x7fd8881f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8880ac930 0x7fd8881f4010 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 --2- 172.21.15.44:0/661497890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd88806ea50 0x7fd88809be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.176+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 >> 172.21.15.44:0/661497890 conn(0x7fd8880b0fb0 msgr2=0x7fd8880b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.177+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 shutdown_connections 2024-07-17T08:37:10.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.177+0000 7fd88e000640 1 -- 172.21.15.44:0/661497890 wait complete. 2024-07-17T08:37:10.197 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:37:10.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 -- 172.21.15.44:0/695617047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b8004800 msgr2=0x7f17b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 --2- 172.21.15.44:0/695617047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b8004800 0x7f17b8004c80 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c260 tx=0x7f17b8005830 comp rx=0 tx=0).stop 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d6800640 1 -- 172.21.15.44:0/695617047 reap_dead start 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 -- 172.21.15.44:0/695617047 shutdown_connections 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 -- 172.21.15.44:0/695617047 >> 172.21.15.44:0/695617047 conn(0x7f17d00b4ff0 msgr2=0x7f17d00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 -- 172.21.15.44:0/695617047 shutdown_connections 2024-07-17T08:37:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.452+0000 7f17d7200640 1 -- 172.21.15.44:0/695617047 wait complete. 2024-07-17T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.454+0000 7f17d7200640 1 Processor -- start 2024-07-17T08:37:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.454+0000 7f17d7200640 1 -- start start 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 0x7f17d00c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17d00b75b0 0x7f17d00b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0166400 con 0x7f17d01f4a20 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0166280 con 0x7f17b8004800 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f17d0166580 con 0x7f17d00b75b0 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47044/0 (socket says 172.21.15.44:47044) 2024-07-17T08:37:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.455+0000 7f17d5400640 1 -- 172.21.15.44:0/852098207 learned_addr learned my addr 172.21.15.44:0/852098207 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d6800640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17d00b75b0 0x7f17d00b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5e00640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 0x7f17d00c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 -- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17d00b75b0 msgr2=0x7f17d00b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17d00b75b0 0x7f17d00b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 -- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 msgr2=0x7f17d00c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 0x7f17d00c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 -- 172.21.15.44:0/852098207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00b6550 con 0x7f17d01f4a20 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5e00640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 0x7f17d00c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.456+0000 7f17d5400640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f17c002b0f0 tx=0x7f17c0001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.457+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0031030 con 0x7f17d01f4a20 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.457+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c00035f0 con 0x7f17d01f4a20 2024-07-17T08:37:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.457+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003a20 con 0x7f17d01f4a20 2024-07-17T08:37:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.457+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d00b67f0 con 0x7f17d01f4a20 2024-07-17T08:37:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.457+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d00b6db0 con 0x7f17d01f4a20 2024-07-17T08:37:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.458+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d00b2200 con 0x7f17d01f4a20 2024-07-17T08:37:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.460+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f17c0034070 con 0x7f17d01f4a20 2024-07-17T08:37:10.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.461+0000 7f17ca000640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 0x7f17b4079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.461+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f17c00b8950 con 0x7f17d01f4a20 2024-07-17T08:37:10.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.461+0000 7f17d5e00640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 0x7f17b4079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.462+0000 7f17d5e00640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 0x7f17b4079250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17d00cb060 tx=0x7f17c4006b70 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.467+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f17c0081680 con 0x7f17d01f4a20 2024-07-17T08:37:10.575 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:10.573+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:10.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.687+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17d00d0e10 con 0x7f17d01f4a20 2024-07-17T08:37:10.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.687+0000 7f17ca000640 1 -- 172.21.15.44:0/852098207 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v255) ==== 93+0+10 (secure 0 0 0) 0x7f17c0080f80 con 0x7f17d01f4a20 2024-07-17T08:37:10.688 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:37:10.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.690+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 msgr2=0x7f17b4079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.690+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 0x7f17b4079250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17d00cb060 tx=0x7f17c4006b70 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.690+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 msgr2=0x7f17d00cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.690+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f17c002b0f0 tx=0x7f17c0001d70 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 shutdown_connections 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f17b4076d90 0x7f17b4079250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17d00b75b0 0x7f17d00b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17d01f4a20 0x7f17d00cb0d0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 --2- 172.21.15.44:0/852098207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b8004800 0x7f17d00c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 >> 172.21.15.44:0/852098207 conn(0x7f17d00b4ff0 msgr2=0x7f17d015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:10.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 shutdown_connections 2024-07-17T08:37:10.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.691+0000 7f17d7200640 1 -- 172.21.15.44:0/852098207 wait complete. 2024-07-17T08:37:10.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-07-17T08:37:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 -- 172.21.15.44:0/3869533805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 msgr2=0x7f3bc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 --2- 172.21.15.44:0/3869533805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3bc8004c80 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00baf0 tx=0x7f3bc8005830 comp rx=0 tx=0).stop 2024-07-17T08:37:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 -- 172.21.15.44:0/3869533805 shutdown_connections 2024-07-17T08:37:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 --2- 172.21.15.44:0/3869533805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3bc8004c80 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 -- 172.21.15.44:0/3869533805 >> 172.21.15.44:0/3869533805 conn(0x7f3be40cfec0 msgr2=0x7f3be40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.913+0000 7f3be8c00640 1 -- 172.21.15.44:0/3869533805 shutdown_connections 2024-07-17T08:37:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.914+0000 7f3be8c00640 1 -- 172.21.15.44:0/3869533805 wait complete. 2024-07-17T08:37:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.915+0000 7f3be8c00640 1 Processor -- start 2024-07-17T08:37:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.915+0000 7f3be8c00640 1 -- start start 2024-07-17T08:37:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be40c9f00 0x7f3be41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4158e10 0x7f3be4159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40b54c0 con 0x7f3bc8004800 2024-07-17T08:37:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40b5340 con 0x7f3be4158e10 2024-07-17T08:37:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.916+0000 7f3be8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3be40b5640 con 0x7f3be40c9f00 2024-07-17T08:37:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47054/0 (socket says 172.21.15.44:47054) 2024-07-17T08:37:10.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 -- 172.21.15.44:0/2587249458 learned_addr learned my addr 172.21.15.44:0/2587249458 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be2a00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be40c9f00 0x7f3be41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 -- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be40c9f00 msgr2=0x7f3be41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be40c9f00 0x7f3be41566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 -- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4158e10 msgr2=0x7f3be4159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4158e10 0x7f3be4159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 -- 172.21.15.44:0/2587249458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be4158620 con 0x7f3bc8004800 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.917+0000 7f3be3400640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4001840 tx=0x7f3bd4001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.918+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4015020 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.918+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd4004030 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.918+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4004460 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.918+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be4157580 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.918+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be4157b40 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.919+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be406dca0 con 0x7f3bc8004800 2024-07-17T08:37:10.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.920+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f3bd4018020 con 0x7f3bc8004800 2024-07-17T08:37:10.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.921+0000 7f3bdb400640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 0x7f3bc0079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:10.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.921+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f3bd40996b0 con 0x7f3bc8004800 2024-07-17T08:37:10.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.922+0000 7f3be2a00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 0x7f3bc0079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:10.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.923+0000 7f3be2a00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 0x7f3bc0079220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc036000 tx=0x7f3bcc0046c0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:10.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:10.925+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bd401f020 con 0x7f3bc8004800 2024-07-17T08:37:11.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.222+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f3be40b5d70 con 0x7f3bc8004800 2024-07-17T08:37:11.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.705+0000 7f3bdb400640 1 -- 172.21.15.44:0/2587249458 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3371) ==== 158+0+0 (secure 0 0 0) 0x7f3bd40622d0 con 0x7f3bc8004800 2024-07-17T08:37:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 msgr2=0x7f3bc0079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 0x7f3bc0079220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc036000 tx=0x7f3bcc0046c0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 msgr2=0x7f3be415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4001840 tx=0x7f3bd4001d00 comp rx=0 tx=0).stop 2024-07-17T08:37:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 shutdown_connections 2024-07-17T08:37:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f3bc0076d60 0x7f3bc0079220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3be4158e10 0x7f3be4159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be40c9f00 0x7f3be41566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 --2- 172.21.15.44:0/2587249458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8004800 0x7f3be415d800 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 >> 172.21.15.44:0/2587249458 conn(0x7f3be40cfec0 msgr2=0x7f3be40b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.709+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 shutdown_connections 2024-07-17T08:37:11.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.710+0000 7f3be8c00640 1 -- 172.21.15.44:0/2587249458 wait complete. 2024-07-17T08:37:11.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:37:11.766 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:11.764+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 -- 172.21.15.44:0/1573334110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed24005e50 msgr2=0x7fed240062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 --2- 172.21.15.44:0/1573334110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed24005e50 0x7fed240062d0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fed34005fd0 tx=0x7fed24006940 comp rx=0 tx=0).stop 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed41800640 1 -- 172.21.15.44:0/1573334110 reap_dead start 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 -- 172.21.15.44:0/1573334110 shutdown_connections 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 -- 172.21.15.44:0/1573334110 >> 172.21.15.44:0/1573334110 conn(0x7fed3c06c120 msgr2=0x7fed3c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 -- 172.21.15.44:0/1573334110 shutdown_connections 2024-07-17T08:37:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.952+0000 7fed42200640 1 -- 172.21.15.44:0/1573334110 wait complete. 2024-07-17T08:37:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.954+0000 7fed42200640 1 Processor -- start 2024-07-17T08:37:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.954+0000 7fed42200640 1 -- start start 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 0x7fed3c1f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 0x7fed3c14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1503b0 con 0x7fed3c1f70d0 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c150230 con 0x7fed3c1f7550 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed42200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed3c150530 con 0x7fed24005e50 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47072/0 (socket says 172.21.15.44:47072) 2024-07-17T08:37:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed3be00640 1 -- 172.21.15.44:0/3378751820 learned_addr learned my addr 172.21.15.44:0/3378751820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.955+0000 7fed40e00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 0x7fed3c1f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed41800640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 0x7fed3c14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 -- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 msgr2=0x7fed3c1f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 0x7fed3c1f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 -- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 msgr2=0x7fed3c14e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 0x7fed3c14e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 -- 172.21.15.44:0/3378751820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1f8030 con 0x7fed3c1f70d0 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed40e00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 0x7fed3c1f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed41800640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 0x7fed3c14e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed3be00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fed2800fa30 tx=0x7fed2800fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2800bcb0 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed28011440 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1f7a70 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.956+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28011870 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.957+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1f3c60 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.958+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3c1f3a80 con 0x7fed3c1f70d0 2024-07-17T08:37:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.959+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fed28019020 con 0x7fed3c1f70d0 2024-07-17T08:37:11.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.960+0000 7fed38c00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 0x7fed1c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:11.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.961+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7fed28099500 con 0x7fed3c1f70d0 2024-07-17T08:37:11.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.961+0000 7fed40e00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 0x7fed1c079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:11.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.962+0000 7fed40e00640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 0x7fed1c079220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed30001860 tx=0x7fed30000ed0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:11.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:11.965+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fed3c1f3a80 con 0x7fed3c1f70d0 2024-07-17T08:37:12.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.182+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fed3c074fc0 con 0x7fed3c1f70d0 2024-07-17T08:37:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.183+0000 7fed38c00640 1 -- 172.21.15.44:0/3378751820 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v79) ==== 59+0+1998 (secure 0 0 0) 0x7fed280621b0 con 0x7fed3c1f70d0 2024-07-17T08:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:37:12.184 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:37:12.185 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:37:12.186 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 msgr2=0x7fed1c079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 0x7fed1c079220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed30001860 tx=0x7fed30000ed0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 msgr2=0x7fed3c14de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fed2800fa30 tx=0x7fed2800fef0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 shutdown_connections 2024-07-17T08:37:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fed1c076d60 0x7fed1c079220 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed3c1f7550 0x7fed3c14e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed3c1f70d0 0x7fed3c14de20 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 --2- 172.21.15.44:0/3378751820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed24005e50 0x7fed3c1f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 >> 172.21.15.44:0/3378751820 conn(0x7fed3c06c120 msgr2=0x7fed3c0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 shutdown_connections 2024-07-17T08:37:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.186+0000 7fed42200640 1 -- 172.21.15.44:0/3378751820 wait complete. 2024-07-17T08:37:12.207 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 -- 172.21.15.44:0/3992321925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb015a5a0 msgr2=0x7f4fb00b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/3992321925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb015a5a0 0x7f4fb00b3280 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00b1f20 tx=0x7f4f9802ade0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 -- 172.21.15.44:0/3992321925 shutdown_connections 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/3992321925 >> v2:172.21.15.175:3300/0 conn(0x7f4fb00ac3a0 0x7f4fb00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/3992321925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb015a5a0 0x7f4fb00b3280 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 -- 172.21.15.44:0/3992321925 >> 172.21.15.44:0/3992321925 conn(0x7f4fb00b0eb0 msgr2=0x7f4fb00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:12.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 -- 172.21.15.44:0/3992321925 shutdown_connections 2024-07-17T08:37:12.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.414+0000 7f4fb4e00640 1 -- 172.21.15.44:0/3992321925 wait complete. 2024-07-17T08:37:12.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.416+0000 7f4fb4e00640 1 Processor -- start 2024-07-17T08:37:12.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.416+0000 7f4fb4e00640 1 -- start start 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 0x7f4fb00715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 0x7f4fb014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb006f850 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb006f6d0 con 0x7f4fb006eb90 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fb4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb014fd60 con 0x7f4fb015a5a0 2024-07-17T08:37:12.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.417+0000 7f4fad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47082/0 (socket says 172.21.15.44:47082) 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 -- 172.21.15.44:0/240175542 learned_addr learned my addr 172.21.15.44:0/240175542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fade00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 0x7f4fb00715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fae800640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 0x7f4fb014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 -- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 msgr2=0x7f4fb014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 0x7f4fb014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 -- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 msgr2=0x7f4fb00715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 0x7f4fb00715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 -- 172.21.15.44:0/240175542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0074440 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fae800640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 0x7f4fb014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fade00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 0x7f4fb00715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:12.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.418+0000 7f4fad400640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f4f98035040 tx=0x7f4f98009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.419+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98040020 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.419+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f9802b930 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.419+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9802bd20 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.419+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb014da70 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.419+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb014e030 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.420+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb00ac7b0 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.422+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f4f98043020 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.422+0000 7f4fa6000640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 0x7f4f8c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:12.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.423+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f4f980b9150 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.423+0000 7f4fade00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 0x7f4f8c079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:12.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.424+0000 7f4fade00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 0x7f4f8c079220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0001dc0 tx=0x7f4fa0009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:12.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.427+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4f9804a030 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.648+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4fb00b4200 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.656+0000 7f4fa6000640 1 -- 172.21.15.44:0/240175542 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v80)=0 v80) ==== 122+0+0 (secure 0 0 0) 0x7f4f98081e80 con 0x7f4fb00ac3a0 2024-07-17T08:37:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 msgr2=0x7f4f8c079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 0x7f4f8c079220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0001dc0 tx=0x7f4fa0009040 comp rx=0 tx=0).stop 2024-07-17T08:37:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 msgr2=0x7f4fb0073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:12.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f4f98035040 tx=0x7f4f98009850 comp rx=0 tx=0).stop 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 shutdown_connections 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f4f8c076d60 0x7f4f8c079220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fb015a5a0 0x7f4fb014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb00ac3a0 0x7f4fb0073b90 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 --2- 172.21.15.44:0/240175542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fb006eb90 0x7f4fb00715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.659+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 >> 172.21.15.44:0/240175542 conn(0x7f4fb00b0eb0 msgr2=0x7f4fb015bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.660+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 shutdown_connections 2024-07-17T08:37:12.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:12.660+0000 7f4fb4e00640 1 -- 172.21.15.44:0/240175542 wait complete. 2024-07-17T08:37:12.681 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:37:12.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:12.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:12.907 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:12.906+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:12.910 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:37:12.910 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:12.910 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:12.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:37:12.968 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:12.968 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:13.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:13.023 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:13.023 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:13.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:37:13.085 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:13.085 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:13.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:13.137 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:37:13.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 -- 172.21.15.44:0/4153296154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc99c007430 msgr2=0x7fc99c0098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 --2- 172.21.15.44:0/4153296154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc99c007430 0x7fc99c0098a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000c310 tx=0x7fc99c003810 comp rx=0 tx=0).stop 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9ba400640 1 -- 172.21.15.44:0/4153296154 reap_dead start 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 -- 172.21.15.44:0/4153296154 shutdown_connections 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 -- 172.21.15.44:0/4153296154 >> 172.21.15.44:0/4153296154 conn(0x7fc9b40b0e80 msgr2=0x7fc9b40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:13.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 -- 172.21.15.44:0/4153296154 shutdown_connections 2024-07-17T08:37:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.353+0000 7fc9bae00640 1 -- 172.21.15.44:0/4153296154 wait complete. 2024-07-17T08:37:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.355+0000 7fc9bae00640 1 Processor -- start 2024-07-17T08:37:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.355+0000 7fc9bae00640 1 -- start start 2024-07-17T08:37:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c007430 0x7fc9b41f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b41f51e0 0x7fc9b41f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b415a2a0 con 0x7fc9b41585e0 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b415a120 con 0x7fc99c007430 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9bae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b40b1fa0 con 0x7fc9b41f51e0 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9ba400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9ba400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47100/0 (socket says 172.21.15.44:47100) 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.356+0000 7fc9ba400640 1 -- 172.21.15.44:0/746779660 learned_addr learned my addr 172.21.15.44:0/746779660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9b9000640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b41f51e0 0x7fc9b41f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9b9a00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c007430 0x7fc9b41f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 -- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b41f51e0 msgr2=0x7fc9b41f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b41f51e0 0x7fc9b41f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 -- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c007430 msgr2=0x7fc9b41f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c007430 0x7fc9b41f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 -- 172.21.15.44:0/746779660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b414bde0 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.357+0000 7fc9ba400640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4153e90 tx=0x7fc9ac00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.358+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac017070 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.358+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ac005e30 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.358+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b41f2360 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.358+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00a840 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.358+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41f2920 con 0x7fc9b41585e0 2024-07-17T08:37:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.359+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b40cf380 con 0x7fc9b41585e0 2024-07-17T08:37:13.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.360+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fc9ac01a070 con 0x7fc9b41585e0 2024-07-17T08:37:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.362+0000 7fc9b1600640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 0x7fc994079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.362+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7fc9ac097df0 con 0x7fc9b41585e0 2024-07-17T08:37:13.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.362+0000 7fc9b9a00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 0x7fc994079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.363+0000 7fc9b9a00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 0x7fc994079250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8005fd0 tx=0x7fc9a800c4e0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:13.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.366+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc9ac060bb0 con 0x7fc9b41585e0 2024-07-17T08:37:13.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.629+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9b40b3340 con 0x7fc9b41585e0 2024-07-17T08:37:13.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.631+0000 7fc9b1600640 1 -- 172.21.15.44:0/746779660 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) ==== 95+0+3919 (secure 0 0 0) 0x7fc9ac0604b0 con 0x7fc9b41585e0 2024-07-17T08:37:13.634 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:13.634 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":139,"btime":"2024-07-17T08:37:05:655859+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/366024869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":366024869},{"type":"v1","addr":"172.21.15.44:6834","nonce":366024869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":8318,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1411243894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1411243894},{"type":"v1","addr":"172.21.15.175:6801","nonce":1411243894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139},{"gid":8782,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3074625693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3074625693},{"type":"v1","addr":"172.21.15.44:6835","nonce":3074625693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125},{"gid":9112,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2268738226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2268738226},{"type":"v1","addr":"172.21.15.175:6836","nonce":2268738226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":137}],"filesystems":[]} 2024-07-17T08:37:13.634 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 139 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 msgr2=0x7fc994079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 0x7fc994079250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8005fd0 tx=0x7fc9a800c4e0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 msgr2=0x7fc9b4158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4153e90 tx=0x7fc9ac00cb90 comp rx=0 tx=0).stop 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 shutdown_connections 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc994076d90 0x7fc994079250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b41585e0 0x7fc9b4158a60 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b41f51e0 0x7fc9b41f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 --2- 172.21.15.44:0/746779660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc99c007430 0x7fc9b41f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.634+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 >> 172.21.15.44:0/746779660 conn(0x7fc9b40b0e80 msgr2=0x7fc9b40733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.635+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 shutdown_connections 2024-07-17T08:37:13.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.635+0000 7fc9bae00640 1 -- 172.21.15.44:0/746779660 wait complete. 2024-07-17T08:37:13.657 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:37:13.657 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:37:13.658 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:37:13.658 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:13.658 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:37:13.658 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:37:13.658 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:37:13.658 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:13.658 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:37:13.659 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:37:13.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:37:13.659 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:13.659 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:37:13.659 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:37:13.659 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:37:13.659 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:13.682 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:37:13.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:37:13.685 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:37:13.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:37:13.686 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:37:13.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:37:13.688 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:37:13.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:37:13.732 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:37:13.733 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:13.736 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:37:13.736 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:13.738 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:37:13.738 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:13.742 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:37:13.742 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:13.743 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:37:13.887 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:37:13.888 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:37:13.891 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:37:13.892 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.938+0000 7f846aa00640 1 -- 172.21.15.44:0/2740312534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8464161be0 msgr2=0x7f846415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.938+0000 7f846aa00640 1 --2- 172.21.15.44:0/2740312534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8464161be0 0x7f846415d510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8450009860 tx=0x7f845002a530 comp rx=0 tx=0).stop 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.938+0000 7f846aa00640 1 -- 172.21.15.44:0/2740312534 shutdown_connections 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.938+0000 7f846aa00640 1 --2- 172.21.15.44:0/2740312534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8464161be0 0x7f846415d510 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.938+0000 7f846aa00640 1 -- 172.21.15.44:0/2740312534 >> 172.21.15.44:0/2740312534 conn(0x7f84640b4ff0 msgr2=0x7f84640b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.939+0000 7f846aa00640 1 -- 172.21.15.44:0/2740312534 shutdown_connections 2024-07-17T08:37:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.939+0000 7f846aa00640 1 -- 172.21.15.44:0/2740312534 wait complete. 2024-07-17T08:37:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.940+0000 7f846aa00640 1 Processor -- start 2024-07-17T08:37:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.940+0000 7f846aa00640 1 -- start start 2024-07-17T08:37:13.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f846415d130 0x7f8464140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8464161be0 0x7f846415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8464168810 con 0x7f84640ce690 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8464168690 con 0x7f8464161be0 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f846aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8464168990 con 0x7f846415d130 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.941+0000 7f8469600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47138/0 (socket says 172.21.15.44:47138) 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 -- 172.21.15.44:0/1935617174 learned_addr learned my addr 172.21.15.44:0/1935617174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8468c00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f846415d130 0x7f8464140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f846a000640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8464161be0 0x7f846415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 -- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f846415d130 msgr2=0x7f8464140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f846415d130 0x7f8464140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 -- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8464161be0 msgr2=0x7f846415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8464161be0 0x7f846415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 -- 172.21.15.44:0/1935617174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641f7910 con 0x7f84640ce690 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.942+0000 7f8469600640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f845800c700 tx=0x7f845800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:13.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.943+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458016020 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.943+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f845800a930 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.943+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84641f7bf0 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.943+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845800ad40 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.943+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641f81b0 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.944+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846415d510 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.945+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f8458019020 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.947+0000 7f8461600640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 0x7f8444079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.947+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 6857+0+0 (secure 0 0 0) 0x7f84580989d0 con 0x7f84640ce690 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.948+0000 7f8468c00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 0x7f8444079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:13.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.948+0000 7f8468c00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 0x7f8444079220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f8450004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:13.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:13.951+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8458020030 con 0x7f84640ce690 2024-07-17T08:37:14.004 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:14.002+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:14.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.188+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f84640d0e10 con 0x7f84640ce690 2024-07-17T08:37:14.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.746+0000 7f8461600640 1 -- 172.21.15.44:0/1935617174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v257) ==== 97+0+0 (secure 0 0 0) 0x7f8458061680 con 0x7f84640ce690 2024-07-17T08:37:14.748 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:37:14.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 msgr2=0x7f8444079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 0x7f8444079220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f8450004730 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 msgr2=0x7f84640d3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f845800c700 tx=0x7f845800cbc0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 shutdown_connections 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8444076d60 0x7f8444079220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8464161be0 0x7f846415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f846415d130 0x7f8464140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 --2- 172.21.15.44:0/1935617174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84640ce690 0x7f84640d3df0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 >> 172.21.15.44:0/1935617174 conn(0x7f84640b4ff0 msgr2=0x7f8464157370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:14.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 shutdown_connections 2024-07-17T08:37:14.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.751+0000 7f846aa00640 1 -- 172.21.15.44:0/1935617174 wait complete. 2024-07-17T08:37:14.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:37:14.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9c00640 1 -- 172.21.15.44:0/783141800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc788004070 msgr2=0x7fc788004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:14.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/783141800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc788004070 0x7fc788004800 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fc78c001690 tx=0x7fc788005830 comp rx=0 tx=0).stop 2024-07-17T08:37:14.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9200640 1 -- 172.21.15.44:0/783141800 reap_dead start 2024-07-17T08:37:14.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9c00640 1 -- 172.21.15.44:0/783141800 shutdown_connections 2024-07-17T08:37:14.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9c00640 1 -- 172.21.15.44:0/783141800 >> 172.21.15.44:0/783141800 conn(0x7fc7a40b0e80 msgr2=0x7fc7a40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:14.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.990+0000 7fc7a9c00640 1 -- 172.21.15.44:0/783141800 shutdown_connections 2024-07-17T08:37:14.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.991+0000 7fc7a9c00640 1 -- 172.21.15.44:0/783141800 wait complete. 2024-07-17T08:37:14.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.992+0000 7fc7a9c00640 1 Processor -- start 2024-07-17T08:37:14.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.993+0000 7fc7a9c00640 1 -- start start 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc788004070 0x7fc7a41f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4148190 0x7fc7a40ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40cfce0 con 0x7fc7a415cd80 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40cfb60 con 0x7fc7a4148190 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40b26b0 con 0x7fc788004070 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47156/0 (socket says 172.21.15.44:47156) 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 -- 172.21.15.44:0/3917355962 learned_addr learned my addr 172.21.15.44:0/3917355962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:14.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a3e00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc788004070 0x7fc7a41f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc79b400640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4148190 0x7fc7a40ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 -- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc788004070 msgr2=0x7fc7a41f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc788004070 0x7fc7a41f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 -- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4148190 msgr2=0x7fc7a40ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4148190 0x7fc7a40ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:14.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.994+0000 7fc7a9200640 1 -- 172.21.15.44:0/3917355962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a41f5210 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a9200640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0015c0 tx=0x7fc79c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c015070 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc79c00caa0 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c011460 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a40ce5f0 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.995+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a41f5790 con 0x7fc7a415cd80 2024-07-17T08:37:14.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.996+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a40cb3e0 con 0x7fc7a415cd80 2024-07-17T08:37:14.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.997+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fc79c018070 con 0x7fc7a415cd80 2024-07-17T08:37:14.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.999+0000 7fc7a0c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 0x7fc780079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:15.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:14.999+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(259..259 src has 1..259) ==== 5825+0+0 (secure 0 0 0) 0x7fc79c098b70 con 0x7fc7a415cd80 2024-07-17T08:37:15.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.000+0000 7fc7a3e00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 0x7fc780079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:15.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.001+0000 7fc7a3e00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 0x7fc780079250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc79400ab70 tx=0x7fc794005f90 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:15.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.004+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc79c061c20 con 0x7fc7a415cd80 2024-07-17T08:37:15.190 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:15.188+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.312+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc7a40b2bc0 con 0x7fc7a415cd80 2024-07-17T08:37:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.312+0000 7fc7a0c00640 1 -- 172.21.15.44:0/3917355962 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v73) ==== 80+0+5304 (secure 0 0 0) 0x7fc79c061520 con 0x7fc7a415cd80 2024-07-17T08:37:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:15.313 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:37:15.313 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:37:15.314 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:15.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:37:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:37:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:37:15.327 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.315+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 msgr2=0x7fc780079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.315+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 0x7fc780079250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc79400ab70 tx=0x7fc794005f90 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.315+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 msgr2=0x7fc7a415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.315+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0015c0 tx=0x7fc79c00eb80 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 shutdown_connections 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fc780076d90 0x7fc780079250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7a415cd80 0x7fc7a415d200 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7a4148190 0x7fc7a40ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 --2- 172.21.15.44:0/3917355962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc788004070 0x7fc7a41f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 >> 172.21.15.44:0/3917355962 conn(0x7fc7a40b0e80 msgr2=0x7fc7a409b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:15.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 shutdown_connections 2024-07-17T08:37:15.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.316+0000 7fc7a9c00640 1 -- 172.21.15.44:0/3917355962 wait complete. 2024-07-17T08:37:15.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1138650815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 msgr2=0x7f7e7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1138650815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e7c00b9c0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f7e98074090 tx=0x7f7e7c005830 comp rx=0 tx=0).stop 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9cc00640 1 -- 172.21.15.44:0/1138650815 reap_dead start 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1138650815 shutdown_connections 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1138650815 >> 172.21.15.44:0/1138650815 conn(0x7f7e98072730 msgr2=0x7f7e98072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1138650815 shutdown_connections 2024-07-17T08:37:15.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.539+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1138650815 wait complete. 2024-07-17T08:37:15.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.541+0000 7f7e9ec00640 1 Processor -- start 2024-07-17T08:37:15.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.541+0000 7f7e9ec00640 1 -- start start 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e981557c0 0x7f7e981560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 0x7f7e981f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9815e8d0 con 0x7f7e7c004e60 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9815e750 con 0x7f7e981f7910 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e980adfb0 con 0x7f7e981557c0 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e97e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47170/0 (socket says 172.21.15.44:47170) 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e97e00640 1 -- 172.21.15.44:0/1344160812 learned_addr learned my addr 172.21.15.44:0/1344160812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e97400640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e981557c0 0x7f7e981560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:15.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e9cc00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 0x7f7e981f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.542+0000 7f7e97e00640 1 -- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e981557c0 msgr2=0x7f7e981560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e97e00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e981557c0 0x7f7e981560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e97e00640 1 -- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 msgr2=0x7f7e981f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e97e00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 0x7f7e981f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e97e00640 1 -- 172.21.15.44:0/1344160812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9814a2e0 con 0x7f7e7c004e60 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e9cc00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 0x7f7e981f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e97e00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e770 tx=0x7f7e8800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800cc00 con 0x7f7e7c004e60 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e8800ce90 con 0x7f7e7c004e60 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88005280 con 0x7f7e7c004e60 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e98157390 con 0x7f7e7c004e60 2024-07-17T08:37:15.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.543+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e98157930 con 0x7f7e7c004e60 2024-07-17T08:37:15.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.544+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e980b35f0 con 0x7f7e7c004e60 2024-07-17T08:37:15.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.546+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f7e88005420 con 0x7f7e7c004e60 2024-07-17T08:37:15.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.546+0000 7f7e8fe00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 0x7f7e78079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:15.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.547+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(259..259 src has 1..259) ==== 5825+0+0 (secure 0 0 0) 0x7f7e88098fe0 con 0x7f7e7c004e60 2024-07-17T08:37:15.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.547+0000 7f7e97400640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 0x7f7e78079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:15.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.548+0000 7f7e97400640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 0x7f7e78079250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e98153c10 tx=0x7f7e84009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:15.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.551+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7e980b35f0 con 0x7f7e7c004e60 2024-07-17T08:37:15.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.783+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7e980ae510 con 0x7f7e7c004e60 2024-07-17T08:37:15.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.784+0000 7f7e8fe00640 1 -- 172.21.15.44:0/1344160812 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v259) ==== 84+0+65 (secure 0 0 0) 0x7f7e88016030 con 0x7f7e7c004e60 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:15.785 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:37:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.787+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 msgr2=0x7f7e78079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.787+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 0x7f7e78079250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e98153c10 tx=0x7f7e84009040 comp rx=0 tx=0).stop 2024-07-17T08:37:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.787+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 msgr2=0x7f7e98149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.787+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e770 tx=0x7f7e8800ec30 comp rx=0 tx=0).stop 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 shutdown_connections 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f7e78076d90 0x7f7e78079250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e981f7910 0x7f7e981f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e981557c0 0x7f7e981560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 --2- 172.21.15.44:0/1344160812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e7c004e60 0x7f7e98149da0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 >> 172.21.15.44:0/1344160812 conn(0x7f7e98072730 msgr2=0x7f7e98072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 shutdown_connections 2024-07-17T08:37:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:15.788+0000 7f7e9ec00640 1 -- 172.21.15.44:0/1344160812 wait complete. 2024-07-17T08:37:15.808 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:37:15.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:37:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 -- 172.21.15.44:0/1762305724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40b4004d90 msgr2=0x7f40b4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 --2- 172.21.15.44:0/1762305724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40b4004d90 0x7f40b4005210 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f40c8006fa0 tx=0x7f40b4005830 comp rx=0 tx=0).stop 2024-07-17T08:37:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40cfe00640 1 -- 172.21.15.44:0/1762305724 reap_dead start 2024-07-17T08:37:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 -- 172.21.15.44:0/1762305724 shutdown_connections 2024-07-17T08:37:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 -- 172.21.15.44:0/1762305724 >> 172.21.15.44:0/1762305724 conn(0x7f40d00b0eb0 msgr2=0x7f40d00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 -- 172.21.15.44:0/1762305724 shutdown_connections 2024-07-17T08:37:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.020+0000 7f40d6800640 1 -- 172.21.15.44:0/1762305724 wait complete. 2024-07-17T08:37:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.022+0000 7f40d6800640 1 Processor -- start 2024-07-17T08:37:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.022+0000 7f40d6800640 1 -- start start 2024-07-17T08:37:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b4004d90 0x7f40d014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:16.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40d01605c0 0x7f40d0072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40d00b2540 con 0x7f40d0072ea0 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40d00b23c0 con 0x7f40b4004d90 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40d6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40d00b26c0 con 0x7f40d01605c0 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47192/0 (socket says 172.21.15.44:47192) 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40cfe00640 1 -- 172.21.15.44:0/118308039 learned_addr learned my addr 172.21.15.44:0/118308039 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40cea00640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40d01605c0 0x7f40d0072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.023+0000 7f40cf400640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b4004d90 0x7f40d014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 -- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40d01605c0 msgr2=0x7f40d0072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40d01605c0 0x7f40d0072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 -- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b4004d90 msgr2=0x7f40d014da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b4004d90 0x7f40d014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 -- 172.21.15.44:0/118308039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d014f400 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40cfe00640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f40c802b080 tx=0x7f40c8001180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8032030 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40c80038e0 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8001c70 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d00715e0 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.024+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d0071ba0 con 0x7f40d0072ea0 2024-07-17T08:37:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.025+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d00b2c50 con 0x7f40d0072ea0 2024-07-17T08:37:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.027+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f40c8035020 con 0x7f40d0072ea0 2024-07-17T08:37:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.028+0000 7f40c7400640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 0x7f40ac079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.028+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(259..259 src has 1..259) ==== 5825+0+0 (secure 0 0 0) 0x7f40c80b82d0 con 0x7f40d0072ea0 2024-07-17T08:37:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.028+0000 7f40cf400640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 0x7f40ac079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:16.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.030+0000 7f40cf400640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 0x7f40ac079250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f40d0074830 tx=0x7f40c0006cf0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:16.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.032+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f40c8081380 con 0x7f40d0072ea0 2024-07-17T08:37:16.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.263+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f40d006f780 con 0x7f40d0072ea0 2024-07-17T08:37:16.368 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:16.366+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.760+0000 7f40c7400640 1 -- 172.21.15.44:0/118308039 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v260) ==== 138+0+0 (secure 0 0 0) 0x7f40c8080c80 con 0x7f40d0072ea0 2024-07-17T08:37:16.761 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:37:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.764+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 msgr2=0x7f40ac079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.764+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 0x7f40ac079250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f40d0074830 tx=0x7f40c0006cf0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.764+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 msgr2=0x7f40d0150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:16.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.764+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f40c802b080 tx=0x7f40c8001180 comp rx=0 tx=0).stop 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 shutdown_connections 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f40ac076d90 0x7f40ac079250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40d0072ea0 0x7f40d0150930 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40d01605c0 0x7f40d0072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 --2- 172.21.15.44:0/118308039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b4004d90 0x7f40d014da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 >> 172.21.15.44:0/118308039 conn(0x7f40d00b0eb0 msgr2=0x7f40d00aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 shutdown_connections 2024-07-17T08:37:16.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:16.765+0000 7f40d6800640 1 -- 172.21.15.44:0/118308039 wait complete. 2024-07-17T08:37:16.796 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.009+0000 7f69eac00640 1 -- 172.21.15.44:0/4258694010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c80051c0 msgr2=0x7f69c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.009+0000 7f69eac00640 1 --2- 172.21.15.44:0/4258694010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c80051c0 0x7f69c800b9c0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f69dc001dc0 tx=0x7f69c8005830 comp rx=0 tx=0).stop 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.010+0000 7f69eac00640 1 -- 172.21.15.44:0/4258694010 shutdown_connections 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.010+0000 7f69eac00640 1 --2- 172.21.15.44:0/4258694010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69c80051c0 0x7f69c800b9c0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.010+0000 7f69eac00640 1 -- 172.21.15.44:0/4258694010 >> 172.21.15.44:0/4258694010 conn(0x7f69e40b2240 msgr2=0x7f69e40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.010+0000 7f69eac00640 1 -- 172.21.15.44:0/4258694010 shutdown_connections 2024-07-17T08:37:17.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.010+0000 7f69eac00640 1 -- 172.21.15.44:0/4258694010 wait complete. 2024-07-17T08:37:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.011+0000 7f69eac00640 1 Processor -- start 2024-07-17T08:37:17.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.012+0000 7f69eac00640 1 -- start start 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69c80051c0 0x7f69e415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e41f49b0 0x7f69e415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69e415b550 con 0x7f69e415d390 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e406fdc0 con 0x7f69e41f49b0 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69eac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69e4070710 con 0x7f69c80051c0 2024-07-17T08:37:17.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47214/0 (socket says 172.21.15.44:47214) 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 -- 172.21.15.44:0/1001635562 learned_addr learned my addr 172.21.15.44:0/1001635562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e3e00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69c80051c0 0x7f69e415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e3400640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e41f49b0 0x7f69e415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 -- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69c80051c0 msgr2=0x7f69e415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69c80051c0 0x7f69e415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 -- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e41f49b0 msgr2=0x7f69e415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e41f49b0 0x7f69e415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.013+0000 7f69e8c00640 1 -- 172.21.15.44:0/1001635562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e415d910 con 0x7f69e415d390 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69e8c00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02acc0 tx=0x7f69dc006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc003d80 con 0x7f69e415d390 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69dc0060d0 con 0x7f69e415d390 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0326d0 con 0x7f69e415d390 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e415bb20 con 0x7f69e415d390 2024-07-17T08:37:17.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.014+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e415c0e0 con 0x7f69e415d390 2024-07-17T08:37:17.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.016+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e406f6b0 con 0x7f69e415d390 2024-07-17T08:37:17.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.017+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f69dc032870 con 0x7f69e415d390 2024-07-17T08:37:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.018+0000 7f69dbe00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 0x7f69c4079250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:17.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.018+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(260..260 src has 1..260) ==== 6195+0+0 (secure 0 0 0) 0x7f69dc0b88d0 con 0x7f69e415d390 2024-07-17T08:37:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.019+0000 7f69e3e00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 0x7f69c4079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:17.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.020+0000 7f69e3e00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 0x7f69c4079250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69e40afa60 tx=0x7f69d40015e0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:17.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.022+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f69dc04c080 con 0x7f69e415d390 2024-07-17T08:37:17.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.237+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f69e415b8f0 con 0x7f69e415d390 2024-07-17T08:37:17.596 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:17.594+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:17.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.767+0000 7f69dbe00640 1 -- 172.21.15.44:0/1001635562 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v261) ==== 170+0+0 (secure 0 0 0) 0x7f69dc0819a0 con 0x7f69e415d390 2024-07-17T08:37:17.769 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:37:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.771+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 msgr2=0x7f69c4079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:17.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.771+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 0x7f69c4079250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69e40afa60 tx=0x7f69d40015e0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.771+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 msgr2=0x7f69e4152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.771+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02acc0 tx=0x7f69dc006990 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 shutdown_connections 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f69c4076d90 0x7f69c4079250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e415d390 0x7f69e4152c10 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e41f49b0 0x7f69e415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 --2- 172.21.15.44:0/1001635562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69c80051c0 0x7f69e415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.772+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 >> 172.21.15.44:0/1001635562 conn(0x7f69e40b2240 msgr2=0x7f69e4072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.773+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 shutdown_connections 2024-07-17T08:37:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:17.773+0000 7f69eac00640 1 -- 172.21.15.44:0/1001635562 wait complete. 2024-07-17T08:37:17.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:37:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 -- 172.21.15.44:0/3126713114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 msgr2=0x7f13a006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 --2- 172.21.15.44:0/3126713114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a006d620 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f13a006f610 tx=0x7f139402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 -- 172.21.15.44:0/3126713114 shutdown_connections 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 --2- 172.21.15.44:0/3126713114 >> v2:172.21.15.175:3300/0 conn(0x7f13a006ed30 0x7f13a015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 --2- 172.21.15.44:0/3126713114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a006d620 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 -- 172.21.15.44:0/3126713114 >> 172.21.15.44:0/3126713114 conn(0x7f13a00b0eb0 msgr2=0x7f13a00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 -- 172.21.15.44:0/3126713114 shutdown_connections 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.001+0000 7f13a8200640 1 -- 172.21.15.44:0/3126713114 wait complete. 2024-07-17T08:37:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.003+0000 7f13a8200640 1 Processor -- start 2024-07-17T08:37:18.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.003+0000 7f13a8200640 1 -- start start 2024-07-17T08:37:18.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a006ed30 0x7f13a00992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:18.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00cb3a0 0x7f13a0141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:18.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00cbc30 con 0x7f13a0152220 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00cbab0 con 0x7f13a006ed30 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a8200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13a015f570 con 0x7f13a00cb3a0 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47242/0 (socket says 172.21.15.44:47242) 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.004+0000 7f13a6200640 1 -- 172.21.15.44:0/3444020706 learned_addr learned my addr 172.21.15.44:0/3444020706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a5800640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a006ed30 0x7f13a00992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a4e00640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00cb3a0 0x7f13a0141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 -- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00cb3a0 msgr2=0x7f13a0141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00cb3a0 0x7f13a0141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 -- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a006ed30 msgr2=0x7f13a00992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a006ed30 0x7f13a00992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 -- 172.21.15.44:0/3444020706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a0142120 con 0x7f13a0152220 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.005+0000 7f13a6200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f139800c780 tx=0x7f139800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.006+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398017070 con 0x7f13a0152220 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.006+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1398005070 con 0x7f13a0152220 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.006+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398005500 con 0x7f13a0152220 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.006+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a014f210 con 0x7f13a0152220 2024-07-17T08:37:18.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.006+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a013f670 con 0x7f13a0152220 2024-07-17T08:37:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.007+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a015fb10 con 0x7f13a0152220 2024-07-17T08:37:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.009+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f13980056a0 con 0x7f13a0152220 2024-07-17T08:37:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.010+0000 7f139d600640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 0x7f1380079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.010+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 6577+0+0 (secure 0 0 0) 0x7f13980982e0 con 0x7f13a0152220 2024-07-17T08:37:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.010+0000 7f13a5800640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 0x7f1380079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.012+0000 7f13a5800640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 0x7f1380079220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1394033000 tx=0x7f1394004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.013+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1398020080 con 0x7f13a0152220 2024-07-17T08:37:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.286+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f13a0159190 con 0x7f13a0152220 2024-07-17T08:37:18.794 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:18.792+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:18.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.815+0000 7f139d600640 1 -- 172.21.15.44:0/3444020706 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 35 and data pool 36 v142) ==== 174+0+0 (secure 0 0 0) 0x7f13980611b0 con 0x7f13a0152220 2024-07-17T08:37:18.818 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 35 and data pool 36 2024-07-17T08:37:18.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.827+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 msgr2=0x7f1380079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:18.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.827+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 0x7f1380079220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1394033000 tx=0x7f1394004730 comp rx=0 tx=0).stop 2024-07-17T08:37:18.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.827+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 msgr2=0x7f13a013f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:18.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.827+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f139800c780 tx=0x7f139800cc40 comp rx=0 tx=0).stop 2024-07-17T08:37:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 shutdown_connections 2024-07-17T08:37:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1380076d60 0x7f1380079220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a0152220 0x7f13a013f130 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00cb3a0 0x7f13a0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 --2- 172.21.15.44:0/3444020706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13a006ed30 0x7f13a00992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 >> 172.21.15.44:0/3444020706 conn(0x7f13a00b0eb0 msgr2=0x7f13a00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.833+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 shutdown_connections 2024-07-17T08:37:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:18.834+0000 7f13a8200640 1 -- 172.21.15.44:0/3444020706 wait complete. 2024-07-17T08:37:18.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:19.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 -- 172.21.15.44:0/4147733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b9060 msgr2=0x7f0a0815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 --2- 172.21.15.44:0/4147733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b9060 0x7f0a0815d800 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005fd0 tx=0x7f0a0002aa50 comp rx=0 tx=0).stop 2024-07-17T08:37:19.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 -- 172.21.15.44:0/4147733766 shutdown_connections 2024-07-17T08:37:19.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 --2- 172.21.15.44:0/4147733766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b9060 0x7f0a0815d800 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 --2- 172.21.15.44:0/4147733766 >> v2:172.21.15.175:3300/0 conn(0x7f0a080ce5c0 0x7f0a0815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.052+0000 7f0a0d200640 1 -- 172.21.15.44:0/4147733766 >> 172.21.15.44:0/4147733766 conn(0x7f0a080cfec0 msgr2=0x7f0a080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:19.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.053+0000 7f0a0d200640 1 -- 172.21.15.44:0/4147733766 shutdown_connections 2024-07-17T08:37:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.053+0000 7f0a0d200640 1 -- 172.21.15.44:0/4147733766 wait complete. 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.054+0000 7f0a0d200640 1 Processor -- start 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.055+0000 7f0a0d200640 1 -- start start 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a080b9060 0x7f0a081f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 0x7f0a080ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0815e0d0 con 0x7f0a080b8c80 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0815df50 con 0x7f0a080b9060 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a0d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0815e250 con 0x7f0a080ce5c0 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a07400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34602/0 (socket says 172.21.15.44:34602) 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.056+0000 7f0a07400640 1 -- 172.21.15.44:0/3372375438 learned_addr learned my addr 172.21.15.44:0/3372375438 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 -- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 msgr2=0x7f0a080ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07e00640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 0x7f0a080ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a06a00640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a080b9060 0x7f0a081f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 0x7f0a080ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 -- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a080b9060 msgr2=0x7f0a081f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a080b9060 0x7f0a081f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 -- 172.21.15.44:0/3372375438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a080cb700 con 0x7f0a080b8c80 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07e00640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 0x7f0a080ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.057+0000 7f0a07400640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c410 tx=0x7f09f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.058+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8017070 con 0x7f0a080b8c80 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.058+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09f80035f0 con 0x7f0a080b8c80 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.058+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a080cb9e0 con 0x7f0a080b8c80 2024-07-17T08:37:19.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.058+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8003a80 con 0x7f0a080b8c80 2024-07-17T08:37:19.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.058+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a080cbfa0 con 0x7f0a080b8c80 2024-07-17T08:37:19.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.059+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a080cc170 con 0x7f0a080b8c80 2024-07-17T08:37:19.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.061+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f09f801a070 con 0x7f0a080b8c80 2024-07-17T08:37:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.062+0000 7f09ff400640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 0x7f09e8079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.063+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(262..262 src has 1..262) ==== 6677+0+0 (secure 0 0 0) 0x7f09f8098310 con 0x7f0a080b8c80 2024-07-17T08:37:19.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.063+0000 7f0a06a00640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 0x7f09e8079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.065+0000 7f0a06a00640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 0x7f09e8079220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a080cd0d0 tx=0x7f09f4008040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:19.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.066+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f09f8021020 con 0x7f0a080b8c80 2024-07-17T08:37:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.285+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a0815f900 con 0x7f0a080b8c80 2024-07-17T08:37:19.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.287+0000 7f09ff400640 1 -- 172.21.15.44:0/3372375438 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v262) ==== 74+0+17766 (secure 0 0 0) 0x7f09f8061170 con 0x7f0a080b8c80 2024-07-17T08:37:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:19.289 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":262,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:18.288402+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:16.290172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:17.260106+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:19.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.290+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 msgr2=0x7f09e8079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.290+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 0x7f09e8079220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a080cd0d0 tx=0x7f09f4008040 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.290+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 msgr2=0x7f0a0814d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.290+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c410 tx=0x7f09f800c8d0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.290+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 shutdown_connections 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f09e8076d60 0x7f09e8079220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a080ce5c0 0x7f0a080ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a080b9060 0x7f0a081f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 --2- 172.21.15.44:0/3372375438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a080b8c80 0x7f0a0814d7a0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 >> 172.21.15.44:0/3372375438 conn(0x7f0a080cfec0 msgr2=0x7f0a080b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 shutdown_connections 2024-07-17T08:37:19.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.291+0000 7f0a0d200640 1 -- 172.21.15.44:0/3372375438 wait complete. 2024-07-17T08:37:19.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 -- 172.21.15.44:0/542418762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ae810 msgr2=0x7f5d9415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 --2- 172.21.15.44:0/542418762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ae810 0x7f5d9415e510 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009860 tx=0x7f5d7c02a560 comp rx=0 tx=0).stop 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 -- 172.21.15.44:0/542418762 shutdown_connections 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 --2- 172.21.15.44:0/542418762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ae810 0x7f5d9415e510 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 -- 172.21.15.44:0/542418762 >> 172.21.15.44:0/542418762 conn(0x7f5d940b0fb0 msgr2=0x7f5d940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 -- 172.21.15.44:0/542418762 shutdown_connections 2024-07-17T08:37:19.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.528+0000 7f5d98c00640 1 -- 172.21.15.44:0/542418762 wait complete. 2024-07-17T08:37:19.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.530+0000 7f5d98c00640 1 Processor -- start 2024-07-17T08:37:19.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.530+0000 7f5d98c00640 1 -- start start 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 0x7f5d94074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d940ae810 0x7f5d941f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9406f040 con 0x7f5d940ac930 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9406eec0 con 0x7f5d9406ea50 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d98c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d940b2ce0 con 0x7f5d940ae810 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34620/0 (socket says 172.21.15.44:34620) 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 -- 172.21.15.44:0/1915574989 learned_addr learned my addr 172.21.15.44:0/1915574989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91e00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 0x7f5d94074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 -- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d940ae810 msgr2=0x7f5d941f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d940ae810 0x7f5d941f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 -- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 msgr2=0x7f5d94074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 0x7f5d94074200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91400640 1 -- 172.21.15.44:0/1915574989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d941f8190 con 0x7f5d940ac930 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.531+0000 7f5d91e00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 0x7f5d94074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d91400640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c730 tx=0x7f5d8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84016020 con 0x7f5d940ac930 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8400a930 con 0x7f5d940ac930 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400ad80 con 0x7f5d940ac930 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d941f8470 con 0x7f5d940ac930 2024-07-17T08:37:19.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.532+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941f46e0 con 0x7f5d940ac930 2024-07-17T08:37:19.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.533+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9406f390 con 0x7f5d940ac930 2024-07-17T08:37:19.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.535+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f5d84019020 con 0x7f5d940ac930 2024-07-17T08:37:19.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.536+0000 7f5d8a000640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 0x7f5d70079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.536+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(262..262 src has 1..262) ==== 6677+0+0 (secure 0 0 0) 0x7f5d84098b90 con 0x7f5d940ac930 2024-07-17T08:37:19.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.536+0000 7f5d91e00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 0x7f5d70079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.537+0000 7f5d91e00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 0x7f5d70079220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c034000 tx=0x7f5d7c004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:19.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.540+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5d9406f390 con 0x7f5d940ac930 2024-07-17T08:37:19.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.753+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d9406a1a0 con 0x7f5d940ac930 2024-07-17T08:37:19.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.755+0000 7f5d8a000640 1 -- 172.21.15.44:0/1915574989 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v262) ==== 74+0+17766 (secure 0 0 0) 0x7f5d9406a1a0 con 0x7f5d940ac930 2024-07-17T08:37:19.757 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:19.757 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":262,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:18.288402+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:16.290172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:17.260106+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.758+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 msgr2=0x7f5d70079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.758+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 0x7f5d70079220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c034000 tx=0x7f5d7c004730 comp rx=0 tx=0).stop 2024-07-17T08:37:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.758+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 msgr2=0x7f5d941f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.758+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c730 tx=0x7f5d8400cbf0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 shutdown_connections 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5d70076d60 0x7f5d70079220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d940ae810 0x7f5d941f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d940ac930 0x7f5d941f7a60 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 --2- 172.21.15.44:0/1915574989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d9406ea50 0x7f5d94074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:19.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 >> 172.21.15.44:0/1915574989 conn(0x7f5d940b0fb0 msgr2=0x7f5d9415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:19.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 shutdown_connections 2024-07-17T08:37:19.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.759+0000 7f5d98c00640 1 -- 172.21.15.44:0/1915574989 wait complete. 2024-07-17T08:37:19.781 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.992+0000 7fa330000640 1 -- 172.21.15.44:0/2816991085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa310004d40 msgr2=0x7fa3100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.992+0000 7fa330000640 1 --2- 172.21.15.44:0/2816991085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa310004d40 0x7fa3100051c0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa3280b3aa0 tx=0x7fa310005830 comp rx=0 tx=0).stop 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.993+0000 7fa32e000640 1 -- 172.21.15.44:0/2816991085 reap_dead start 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.993+0000 7fa330000640 1 -- 172.21.15.44:0/2816991085 shutdown_connections 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.993+0000 7fa330000640 1 -- 172.21.15.44:0/2816991085 >> 172.21.15.44:0/2816991085 conn(0x7fa3280b0eb0 msgr2=0x7fa3280b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.993+0000 7fa330000640 1 -- 172.21.15.44:0/2816991085 shutdown_connections 2024-07-17T08:37:19.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.993+0000 7fa330000640 1 -- 172.21.15.44:0/2816991085 wait complete. 2024-07-17T08:37:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.996+0000 7fa330000640 1 Processor -- start 2024-07-17T08:37:19.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.996+0000 7fa330000640 1 -- start start 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 0x7fa328152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa328151900 0x7fa32815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280ac850 con 0x7fa328160110 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280ac6d0 con 0x7fa310004d40 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.997+0000 7fa330000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280ac9d0 con 0x7fa328151900 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34636/0 (socket says 172.21.15.44:34636) 2024-07-17T08:37:19.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 -- 172.21.15.44:0/2341407200 learned_addr learned my addr 172.21.15.44:0/2341407200 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32d600640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 0x7fa328152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 -- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa328151900 msgr2=0x7fa32815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa328151900 0x7fa32815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 -- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 msgr2=0x7fa328152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 0x7fa328152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 -- 172.21.15.44:0/2341407200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3281534a0 con 0x7fa328160110 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32d600640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 0x7fa328152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:20.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.998+0000 7fa32e000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa320033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.999+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320038040 con 0x7fa328160110 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.999+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa32002b620 con 0x7fa328160110 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.999+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320004d80 con 0x7fa328160110 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.999+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa328151ec0 con 0x7fa328160110 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:19.999+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa328152480 con 0x7fa328160110 2024-07-17T08:37:20.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.000+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3280ac190 con 0x7fa328160110 2024-07-17T08:37:20.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.001+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fa320044030 con 0x7fa328160110 2024-07-17T08:37:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.002+0000 7fa325600640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 0x7fa308079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.002+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fa3200b8ae0 con 0x7fa328160110 2024-07-17T08:37:20.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.003+0000 7fa32d600640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 0x7fa308079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:20.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.004+0000 7fa32d600640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 0x7fa308079220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001ab0 tx=0x7fa31c009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:20.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.007+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa32004b020 con 0x7fa328160110 2024-07-17T08:37:20.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:20.031+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:20.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.276+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa328160e90 con 0x7fa328160110 2024-07-17T08:37:20.755 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-07-17T08:37:20.755+0000 7fd111a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:37:20.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.842+0000 7fa325600640 1 -- 172.21.15.44:0/2341407200 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v145) ==== 118+0+0 (secure 0 0 0) 0x7fa32002b7f0 con 0x7fa328160110 2024-07-17T08:37:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.846+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 msgr2=0x7fa308079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:20.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.846+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 0x7fa308079220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001ab0 tx=0x7fa31c009040 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.846+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 msgr2=0x7fa32814ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.846+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa320033670 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 shutdown_connections 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fa308076d60 0x7fa308079220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa328160110 0x7fa32814ff70 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa328151900 0x7fa32815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 --2- 172.21.15.44:0/2341407200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310004d40 0x7fa328152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 >> 172.21.15.44:0/2341407200 conn(0x7fa3280b0eb0 msgr2=0x7fa3280aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 shutdown_connections 2024-07-17T08:37:20.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:20.847+0000 7fa330000640 1 -- 172.21.15.44:0/2341407200 wait complete. 2024-07-17T08:37:20.874 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:37:20.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:37:21.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 -- 172.21.15.44:0/187084284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd964003ac0 msgr2=0x7fd964003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:21.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 --2- 172.21.15.44:0/187084284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd964003ac0 0x7fd964003ee0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fd970006f80 tx=0x7fd964005830 comp rx=0 tx=0).stop 2024-07-17T08:37:21.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97e800640 1 -- 172.21.15.44:0/187084284 reap_dead start 2024-07-17T08:37:21.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 -- 172.21.15.44:0/187084284 shutdown_connections 2024-07-17T08:37:21.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 -- 172.21.15.44:0/187084284 >> 172.21.15.44:0/187084284 conn(0x7fd9800cf0d0 msgr2=0x7fd9800cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:21.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 -- 172.21.15.44:0/187084284 shutdown_connections 2024-07-17T08:37:21.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.093+0000 7fd97f200640 1 -- 172.21.15.44:0/187084284 wait complete. 2024-07-17T08:37:21.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.095+0000 7fd97f200640 1 Processor -- start 2024-07-17T08:37:21.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.095+0000 7fd97f200640 1 -- start start 2024-07-17T08:37:21.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd964003ac0 0x7fd980072500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd98014e5d0 0x7fd98015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800cb200 con 0x7fd98014ea50 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800cb080 con 0x7fd98014e5d0 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.096+0000 7fd97f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9800cb380 con 0x7fd964003ac0 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:21.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34656/0 (socket says 172.21.15.44:34656) 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 -- 172.21.15.44:0/286461174 learned_addr learned my addr 172.21.15.44:0/286461174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97de00640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd964003ac0 0x7fd980072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97d400640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd98014e5d0 0x7fd98015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 -- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd964003ac0 msgr2=0x7fd980072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd964003ac0 0x7fd980072500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 -- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd98014e5d0 msgr2=0x7fd98015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd98014e5d0 0x7fd98015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 -- 172.21.15.44:0/286461174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd980153610 con 0x7fd98014ea50 2024-07-17T08:37:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.097+0000 7fd97e800640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fd98015c110 tx=0x7fd978009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.098+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978004b20 con 0x7fd98014ea50 2024-07-17T08:37:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.098+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd978004db0 con 0x7fd98014ea50 2024-07-17T08:37:21.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.098+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801538f0 con 0x7fd98014ea50 2024-07-17T08:37:21.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.098+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978011970 con 0x7fd98014ea50 2024-07-17T08:37:21.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.098+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98015d5f0 con 0x7fd98014ea50 2024-07-17T08:37:21.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.099+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98015d7f0 con 0x7fd98014ea50 2024-07-17T08:37:21.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.100+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fd97801b020 con 0x7fd98014ea50 2024-07-17T08:37:21.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.101+0000 7fd976000640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 0x7fd960079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:21.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.101+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fd978098ab0 con 0x7fd98014ea50 2024-07-17T08:37:21.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.102+0000 7fd97de00640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 0x7fd960079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:21.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.103+0000 7fd97de00640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 0x7fd960079220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd970033000 tx=0x7fd970001930 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:21.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.106+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd98015d7f0 con 0x7fd98014ea50 2024-07-17T08:37:21.286 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:21.284+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.375+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9800cb910 con 0x7fd98014ea50 2024-07-17T08:37:21.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.857+0000 7fd976000640 1 -- 172.21.15.44:0/286461174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v146) ==== 122+0+0 (secure 0 0 0) 0x7fd978061780 con 0x7fd98014ea50 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 msgr2=0x7fd960079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 0x7fd960079220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd970033000 tx=0x7fd970001930 comp rx=0 tx=0).stop 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 msgr2=0x7fd98015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fd98015c110 tx=0x7fd978009a80 comp rx=0 tx=0).stop 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 shutdown_connections 2024-07-17T08:37:21.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fd960076d60 0x7fd960079220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd98014ea50 0x7fd98015a6d0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd98014e5d0 0x7fd98015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 --2- 172.21.15.44:0/286461174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd964003ac0 0x7fd980072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 >> 172.21.15.44:0/286461174 conn(0x7fd9800cf0d0 msgr2=0x7fd9800b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.862+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 shutdown_connections 2024-07-17T08:37:21.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:21.863+0000 7fd97f200640 1 -- 172.21.15.44:0/286461174 wait complete. 2024-07-17T08:37:21.887 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:37:21.887 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 -- 172.21.15.44:0/2287726772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd580040c0 msgr2=0x7fdd58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 --2- 172.21.15.44:0/2287726772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd580040c0 0x7fdd58004800 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006f80 tx=0x7fdd58005830 comp rx=0 tx=0).stop 2024-07-17T08:37:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd72800640 1 -- 172.21.15.44:0/2287726772 reap_dead start 2024-07-17T08:37:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 -- 172.21.15.44:0/2287726772 shutdown_connections 2024-07-17T08:37:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 -- 172.21.15.44:0/2287726772 >> 172.21.15.44:0/2287726772 conn(0x7fdd740b0fb0 msgr2=0x7fdd740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 -- 172.21.15.44:0/2287726772 shutdown_connections 2024-07-17T08:37:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.101+0000 7fdd79200640 1 -- 172.21.15.44:0/2287726772 wait complete. 2024-07-17T08:37:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.103+0000 7fdd79200640 1 Processor -- start 2024-07-17T08:37:22.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.103+0000 7fdd79200640 1 -- start start 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd580040c0 0x7fdd740729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd740b3400 0x7fdd740b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd740cdda0 con 0x7fdd741528c0 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd740adf50 con 0x7fdd740b3400 2024-07-17T08:37:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.104+0000 7fdd79200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd740cdf20 con 0x7fdd580040c0 2024-07-17T08:37:22.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34680/0 (socket says 172.21.15.44:34680) 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 -- 172.21.15.44:0/2763115596 learned_addr learned my addr 172.21.15.44:0/2763115596 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71e00640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd580040c0 0x7fdd740729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd72800640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd740b3400 0x7fdd740b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 -- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd580040c0 msgr2=0x7fdd740729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd580040c0 0x7fdd740729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 -- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd740b3400 msgr2=0x7fdd740b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd740b3400 0x7fdd740b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 -- 172.21.15.44:0/2763115596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd74072ef0 con 0x7fdd741528c0 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.105+0000 7fdd71400640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006f80 tx=0x7fdd5c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.106+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c006bb0 con 0x7fdd741528c0 2024-07-17T08:37:22.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.106+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd5c0015a0 con 0x7fdd741528c0 2024-07-17T08:37:22.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.106+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd740716c0 con 0x7fdd741528c0 2024-07-17T08:37:22.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.106+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c02e680 con 0x7fdd741528c0 2024-07-17T08:37:22.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.106+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd74071c80 con 0x7fdd741528c0 2024-07-17T08:37:22.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.107+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd740ad940 con 0x7fdd741528c0 2024-07-17T08:37:22.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.108+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fdd5c035020 con 0x7fdd741528c0 2024-07-17T08:37:22.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.109+0000 7fdd6a000640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 0x7fdd50079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.109+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fdd5c0b8da0 con 0x7fdd741528c0 2024-07-17T08:37:22.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.110+0000 7fdd71e00640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 0x7fdd50079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.111+0000 7fdd71e00640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 0x7fdd50079220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006310 tx=0x7fdd64006f10 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:22.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.114+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdd740ad940 con 0x7fdd741528c0 2024-07-17T08:37:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.371+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd740cd690 con 0x7fdd741528c0 2024-07-17T08:37:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.372+0000 7fdd6a000640 1 -- 172.21.15.44:0/2763115596 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7fdd5c047030 con 0x7fdd741528c0 2024-07-17T08:37:22.373 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:22.373 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.374+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 msgr2=0x7fdd50079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.374+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 0x7fdd50079220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006310 tx=0x7fdd64006f10 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.374+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 msgr2=0x7fdd741f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.374+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006f80 tx=0x7fdd5c003de0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 shutdown_connections 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdd50076d60 0x7fdd50079220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd740b3400 0x7fdd740b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd741528c0 0x7fdd741f6ed0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 --2- 172.21.15.44:0/2763115596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd580040c0 0x7fdd740729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 >> 172.21.15.44:0/2763115596 conn(0x7fdd740b0fb0 msgr2=0x7fdd740ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 shutdown_connections 2024-07-17T08:37:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.375+0000 7fdd79200640 1 -- 172.21.15.44:0/2763115596 wait complete. 2024-07-17T08:37:22.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:22.533 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:22.530+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:22.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 -- 172.21.15.44:0/3811180565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b9060 msgr2=0x7fb57415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 --2- 172.21.15.44:0/3811180565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b9060 0x7fb57415d800 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009be0 tx=0x7fb56c02a8d0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 -- 172.21.15.44:0/3811180565 shutdown_connections 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 --2- 172.21.15.44:0/3811180565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b9060 0x7fb57415d800 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 --2- 172.21.15.44:0/3811180565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740ce5c0 0x7fb57415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 -- 172.21.15.44:0/3811180565 >> 172.21.15.44:0/3811180565 conn(0x7fb5740cfec0 msgr2=0x7fb5740d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 -- 172.21.15.44:0/3811180565 shutdown_connections 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.609+0000 7fb57a400640 1 -- 172.21.15.44:0/3811180565 wait complete. 2024-07-17T08:37:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.611+0000 7fb57a400640 1 Processor -- start 2024-07-17T08:37:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.611+0000 7fb57a400640 1 -- start start 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740b9060 0x7fb5741f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 0x7fb57414eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb57415e0d0 con 0x7fb5740b8c80 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb57415df50 con 0x7fb5740ce5c0 2024-07-17T08:37:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb57a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb57415e250 con 0x7fb5740b9060 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb579000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb579000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34698/0 (socket says 172.21.15.44:34698) 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb579000640 1 -- 172.21.15.44:0/3587561746 learned_addr learned my addr 172.21.15.44:0/3587561746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb573e00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740b9060 0x7fb5741f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.612+0000 7fb579a00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 0x7fb57414eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 -- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740b9060 msgr2=0x7fb5741f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740b9060 0x7fb5741f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 -- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 msgr2=0x7fb57414eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 0x7fb57414eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 -- 172.21.15.44:0/3587561746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5741f4d00 con 0x7fb5740b8c80 2024-07-17T08:37:22.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579a00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 0x7fb57414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:22.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb579000640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c530 tx=0x7fb56400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564017070 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5640035f0 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564003a40 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5741f8270 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.613+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5741f7940 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.615+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57415e700 con 0x7fb5740b8c80 2024-07-17T08:37:22.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.616+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fb56401a070 con 0x7fb5740b8c80 2024-07-17T08:37:22.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.617+0000 7fb570c00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 0x7fb554079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:22.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.617+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fb564097f70 con 0x7fb5740b8c80 2024-07-17T08:37:22.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.617+0000 7fb573e00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 0x7fb554079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:22.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.619+0000 7fb573e00640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 0x7fb554079220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb560005fd0 tx=0x7fb560005850 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:22.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.621+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb5741f4d00 con 0x7fb5740b8c80 2024-07-17T08:37:22.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.846+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5740b5d70 con 0x7fb5740b8c80 2024-07-17T08:37:22.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.847+0000 7fb570c00640 1 -- 172.21.15.44:0/3587561746 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) ==== 74+0+17753 (secure 0 0 0) 0x7fb564060de0 con 0x7fb5740b8c80 2024-07-17T08:37:22.849 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:22.849 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":263,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:19.814787+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:16.290172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:17.260106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"263","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:22.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.850+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 msgr2=0x7fb554079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.850+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 0x7fb554079220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb560005fd0 tx=0x7fb560005850 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.850+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 msgr2=0x7fb57414d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.850+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c530 tx=0x7fb56400c9f0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 shutdown_connections 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb554076d60 0x7fb554079220 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5740ce5c0 0x7fb57414eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5740b9060 0x7fb5741f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 --2- 172.21.15.44:0/3587561746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5740b8c80 0x7fb57414d500 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 >> 172.21.15.44:0/3587561746 conn(0x7fb5740cfec0 msgr2=0x7fb57415ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 shutdown_connections 2024-07-17T08:37:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:22.851+0000 7fb57a400640 1 -- 172.21.15.44:0/3587561746 wait complete. 2024-07-17T08:37:22.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:37:22.877 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 -- 172.21.15.44:0/4157690534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c815a230 msgr2=0x7ff9c80b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/4157690534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c815a230 0x7ff9c80b3150 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009830 tx=0x7ff9b802a380 comp rx=0 tx=0).stop 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 -- 172.21.15.44:0/4157690534 shutdown_connections 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/4157690534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c815a230 0x7ff9c80b3150 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 -- 172.21.15.44:0/4157690534 >> 172.21.15.44:0/4157690534 conn(0x7ff9c80b5890 msgr2=0x7ff9c80b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 -- 172.21.15.44:0/4157690534 shutdown_connections 2024-07-17T08:37:23.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.088+0000 7ff9c7e00640 1 -- 172.21.15.44:0/4157690534 wait complete. 2024-07-17T08:37:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.090+0000 7ff9c7e00640 1 Processor -- start 2024-07-17T08:37:23.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.090+0000 7ff9c7e00640 1 -- start start 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c815a230 0x7ff9c8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c815bac0 0x7ff9c81e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c80b1630 con 0x7ff9c80b0f60 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c80b14b0 con 0x7ff9c815bac0 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c80b17b0 con 0x7ff9c815a230 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34718/0 (socket says 172.21.15.44:34718) 2024-07-17T08:37:23.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 -- 172.21.15.44:0/3913570043 learned_addr learned my addr 172.21.15.44:0/3913570043 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 -- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c815a230 msgr2=0x7ff9c8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c815a230 0x7ff9c8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 -- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c815bac0 msgr2=0x7ff9c81e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c815bac0 0x7ff9c81e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.091+0000 7ff9c6a00640 1 -- 172.21.15.44:0/3913570043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c81f78a0 con 0x7ff9c80b0f60 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9c6a00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009780 tx=0x7ff9b8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b803f020 con 0x7ff9c80b0f60 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b80035f0 con 0x7ff9c80b0f60 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8037040 con 0x7ff9c80b0f60 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c8155b80 con 0x7ff9c80b0f60 2024-07-17T08:37:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.092+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81f7e20 con 0x7ff9c80b0f60 2024-07-17T08:37:23.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.094+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c8155f20 con 0x7ff9c80b0f60 2024-07-17T08:37:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.095+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7ff9b8042020 con 0x7ff9c80b0f60 2024-07-17T08:37:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.097+0000 7ff9bea00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 0x7ff9a8079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.097+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7ff9b80b8b80 con 0x7ff9c80b0f60 2024-07-17T08:37:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.098+0000 7ff9c6000640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 0x7ff9a8079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.099+0000 7ff9c6000640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 0x7ff9a8079220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000f60 tx=0x7ff9b40079b0 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.102+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff9b8049030 con 0x7ff9c80b0f60 2024-07-17T08:37:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.315+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 --> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9c8159de0 con 0x7ff9a8076d60 2024-07-17T08:37:23.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.331+0000 7ff9bea00640 1 -- 172.21.15.44:0/3913570043 <== mgr.9025 v2:172.21.15.175:6834/1251568930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7ff9c8159de0 con 0x7ff9a8076d60 2024-07-17T08:37:23.336 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:23.339 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:37:23.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 msgr2=0x7ff9a8079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 0x7ff9a8079220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000f60 tx=0x7ff9b40079b0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 msgr2=0x7ff9c81f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009780 tx=0x7ff9b8033c20 comp rx=0 tx=0).stop 2024-07-17T08:37:23.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 shutdown_connections 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff9a8076d60 0x7ff9a8079220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9c815bac0 0x7ff9c81e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c815a230 0x7ff9c8155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 --2- 172.21.15.44:0/3913570043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9c80b0f60 0x7ff9c81f8340 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 >> 172.21.15.44:0/3913570043 conn(0x7ff9c80b5890 msgr2=0x7ff9c80af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 shutdown_connections 2024-07-17T08:37:23.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.339+0000 7ff9c7e00640 1 -- 172.21.15.44:0/3913570043 wait complete. 2024-07-17T08:37:23.362 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-07-17T08:37:22.468101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":429,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4221,"ondisk_log_size":4221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444616,"kb_used_data":27144,"kb_used_omap":12,"kb_used_meta":417331,"kb_avail":754530104,"statfs":{"total":773094113280,"available":772638826496,"internally_reserved":0,"allocated":27795456,"data_stored":15736829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":427347504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703603"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836549+0000","last_change":"2024-07-17T08:37:18.832909+0000","last_active":"2024-07-17T08:37:18.836549+0000","last_peered":"2024-07-17T08:37:18.836549+0000","last_clean":"2024-07-17T08:37:18.836549+0000","last_became_active":"2024-07-17T08:37:18.832477+0000","last_became_peered":"2024-07-17T08:37:18.832477+0000","last_unstale":"2024-07-17T08:37:18.836549+0000","last_undegraded":"2024-07-17T08:37:18.836549+0000","last_fullsized":"2024-07-17T08:37:18.836549+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:29.907945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.830107+0000","last_change":"2024-07-17T08:37:18.826592+0000","last_active":"2024-07-17T08:37:18.830107+0000","last_peered":"2024-07-17T08:37:18.830107+0000","last_clean":"2024-07-17T08:37:18.830107+0000","last_became_active":"2024-07-17T08:37:18.824915+0000","last_became_peered":"2024-07-17T08:37:18.824915+0000","last_unstale":"2024-07-17T08:37:18.830107+0000","last_undegraded":"2024-07-17T08:37:18.830107+0000","last_fullsized":"2024-07-17T08:37:18.830107+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:47.421470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.849329+0000","last_change":"2024-07-17T08:37:18.846136+0000","last_active":"2024-07-17T08:37:18.849329+0000","last_peered":"2024-07-17T08:37:18.849329+0000","last_clean":"2024-07-17T08:37:18.849329+0000","last_became_active":"2024-07-17T08:37:18.844265+0000","last_became_peered":"2024-07-17T08:37:18.844265+0000","last_unstale":"2024-07-17T08:37:18.849329+0000","last_undegraded":"2024-07-17T08:37:18.849329+0000","last_fullsized":"2024-07-17T08:37:18.849329+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:44.438486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.827500+0000","last_change":"2024-07-17T08:37:18.825722+0000","last_active":"2024-07-17T08:37:18.827500+0000","last_peered":"2024-07-17T08:37:18.827500+0000","last_clean":"2024-07-17T08:37:18.827500+0000","last_became_active":"2024-07-17T08:37:18.824775+0000","last_became_peered":"2024-07-17T08:37:18.824775+0000","last_unstale":"2024-07-17T08:37:18.827500+0000","last_undegraded":"2024-07-17T08:37:18.827500+0000","last_fullsized":"2024-07-17T08:37:18.827500+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:37.746840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843770+0000","last_change":"2024-07-17T08:37:18.833819+0000","last_active":"2024-07-17T08:37:18.843770+0000","last_peered":"2024-07-17T08:37:18.843770+0000","last_clean":"2024-07-17T08:37:18.843770+0000","last_became_active":"2024-07-17T08:37:18.833495+0000","last_became_peered":"2024-07-17T08:37:18.833495+0000","last_unstale":"2024-07-17T08:37:18.843770+0000","last_undegraded":"2024-07-17T08:37:18.843770+0000","last_fullsized":"2024-07-17T08:37:18.843770+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:14.502180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854348+0000","last_change":"2024-07-17T08:37:18.853392+0000","last_active":"2024-07-17T08:37:18.854348+0000","last_peered":"2024-07-17T08:37:18.854348+0000","last_clean":"2024-07-17T08:37:18.854348+0000","last_became_active":"2024-07-17T08:37:18.852460+0000","last_became_peered":"2024-07-17T08:37:18.852460+0000","last_unstale":"2024-07-17T08:37:18.854348+0000","last_undegraded":"2024-07-17T08:37:18.854348+0000","last_fullsized":"2024-07-17T08:37:18.854348+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:08.693120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.830485+0000","last_change":"2024-07-17T08:37:18.828720+0000","last_active":"2024-07-17T08:37:18.830485+0000","last_peered":"2024-07-17T08:37:18.830485+0000","last_clean":"2024-07-17T08:37:18.830485+0000","last_became_active":"2024-07-17T08:37:18.828420+0000","last_became_peered":"2024-07-17T08:37:18.828420+0000","last_unstale":"2024-07-17T08:37:18.830485+0000","last_undegraded":"2024-07-17T08:37:18.830485+0000","last_fullsized":"2024-07-17T08:37:18.830485+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:45.033633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826107+0000","last_change":"2024-07-17T08:37:18.819167+0000","last_active":"2024-07-17T08:37:18.826107+0000","last_peered":"2024-07-17T08:37:18.826107+0000","last_clean":"2024-07-17T08:37:18.826107+0000","last_became_active":"2024-07-17T08:37:18.818604+0000","last_became_peered":"2024-07-17T08:37:18.818604+0000","last_unstale":"2024-07-17T08:37:18.826107+0000","last_undegraded":"2024-07-17T08:37:18.826107+0000","last_fullsized":"2024-07-17T08:37:18.826107+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:51.715080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847494+0000","last_change":"2024-07-17T08:37:18.846141+0000","last_active":"2024-07-17T08:37:18.847494+0000","last_peered":"2024-07-17T08:37:18.847494+0000","last_clean":"2024-07-17T08:37:18.847494+0000","last_became_active":"2024-07-17T08:37:18.845829+0000","last_became_peered":"2024-07-17T08:37:18.845829+0000","last_unstale":"2024-07-17T08:37:18.847494+0000","last_undegraded":"2024-07-17T08:37:18.847494+0000","last_fullsized":"2024-07-17T08:37:18.847494+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:20.541825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.832444+0000","last_change":"2024-07-17T08:37:18.831385+0000","last_active":"2024-07-17T08:37:18.832444+0000","last_peered":"2024-07-17T08:37:18.832444+0000","last_clean":"2024-07-17T08:37:18.832444+0000","last_became_active":"2024-07-17T08:37:18.830787+0000","last_became_peered":"2024-07-17T08:37:18.830787+0000","last_unstale":"2024-07-17T08:37:18.832444+0000","last_undegraded":"2024-07-17T08:37:18.832444+0000","last_fullsized":"2024-07-17T08:37:18.832444+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:33:08.351350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833225+0000","last_change":"2024-07-17T08:37:18.825713+0000","last_active":"2024-07-17T08:37:18.833225+0000","last_peered":"2024-07-17T08:37:18.833225+0000","last_clean":"2024-07-17T08:37:18.833225+0000","last_became_active":"2024-07-17T08:37:18.824869+0000","last_became_peered":"2024-07-17T08:37:18.824869+0000","last_unstale":"2024-07-17T08:37:18.833225+0000","last_undegraded":"2024-07-17T08:37:18.833225+0000","last_fullsized":"2024-07-17T08:37:18.833225+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:46.733734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838179+0000","last_change":"2024-07-17T08:37:18.834553+0000","last_active":"2024-07-17T08:37:18.838179+0000","last_peered":"2024-07-17T08:37:18.838179+0000","last_clean":"2024-07-17T08:37:18.838179+0000","last_became_active":"2024-07-17T08:37:18.832219+0000","last_became_peered":"2024-07-17T08:37:18.832219+0000","last_unstale":"2024-07-17T08:37:18.838179+0000","last_undegraded":"2024-07-17T08:37:18.838179+0000","last_fullsized":"2024-07-17T08:37:18.838179+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:40.688248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.825910+0000","last_change":"2024-07-17T08:37:18.822043+0000","last_active":"2024-07-17T08:37:18.825910+0000","last_peered":"2024-07-17T08:37:18.825910+0000","last_clean":"2024-07-17T08:37:18.825910+0000","last_became_active":"2024-07-17T08:37:18.819751+0000","last_became_peered":"2024-07-17T08:37:18.819751+0000","last_unstale":"2024-07-17T08:37:18.825910+0000","last_undegraded":"2024-07-17T08:37:18.825910+0000","last_fullsized":"2024-07-17T08:37:18.825910+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:48.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.834952+0000","last_change":"2024-07-17T08:37:18.832196+0000","last_active":"2024-07-17T08:37:18.834952+0000","last_peered":"2024-07-17T08:37:18.834952+0000","last_clean":"2024-07-17T08:37:18.834952+0000","last_became_active":"2024-07-17T08:37:18.828929+0000","last_became_peered":"2024-07-17T08:37:18.828929+0000","last_unstale":"2024-07-17T08:37:18.834952+0000","last_undegraded":"2024-07-17T08:37:18.834952+0000","last_fullsized":"2024-07-17T08:37:18.834952+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:18:47.759057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826155+0000","last_change":"2024-07-17T08:37:18.822833+0000","last_active":"2024-07-17T08:37:18.826155+0000","last_peered":"2024-07-17T08:37:18.826155+0000","last_clean":"2024-07-17T08:37:18.826155+0000","last_became_active":"2024-07-17T08:37:18.822533+0000","last_became_peered":"2024-07-17T08:37:18.822533+0000","last_unstale":"2024-07-17T08:37:18.826155+0000","last_undegraded":"2024-07-17T08:37:18.826155+0000","last_fullsized":"2024-07-17T08:37:18.826155+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:31.149684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838437+0000","last_change":"2024-07-17T08:37:18.832611+0000","last_active":"2024-07-17T08:37:18.838437+0000","last_peered":"2024-07-17T08:37:18.838437+0000","last_clean":"2024-07-17T08:37:18.838437+0000","last_became_active":"2024-07-17T08:37:18.831834+0000","last_became_peered":"2024-07-17T08:37:18.831834+0000","last_unstale":"2024-07-17T08:37:18.838437+0000","last_undegraded":"2024-07-17T08:37:18.838437+0000","last_fullsized":"2024-07-17T08:37:18.838437+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:50:01.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839904+0000","last_change":"2024-07-17T08:37:18.836977+0000","last_active":"2024-07-17T08:37:18.839904+0000","last_peered":"2024-07-17T08:37:18.839904+0000","last_clean":"2024-07-17T08:37:18.839904+0000","last_became_active":"2024-07-17T08:37:18.833773+0000","last_became_peered":"2024-07-17T08:37:18.833773+0000","last_unstale":"2024-07-17T08:37:18.839904+0000","last_undegraded":"2024-07-17T08:37:18.839904+0000","last_fullsized":"2024-07-17T08:37:18.839904+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:59.188997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.851761+0000","last_change":"2024-07-17T08:37:18.849713+0000","last_active":"2024-07-17T08:37:18.851761+0000","last_peered":"2024-07-17T08:37:18.851761+0000","last_clean":"2024-07-17T08:37:18.851761+0000","last_became_active":"2024-07-17T08:37:18.849379+0000","last_became_peered":"2024-07-17T08:37:18.849379+0000","last_unstale":"2024-07-17T08:37:18.851761+0000","last_undegraded":"2024-07-17T08:37:18.851761+0000","last_fullsized":"2024-07-17T08:37:18.851761+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:50:01.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.824046+0000","last_change":"2024-07-17T08:37:18.821331+0000","last_active":"2024-07-17T08:37:18.824046+0000","last_peered":"2024-07-17T08:37:18.824046+0000","last_clean":"2024-07-17T08:37:18.824046+0000","last_became_active":"2024-07-17T08:37:18.820940+0000","last_became_peered":"2024-07-17T08:37:18.820940+0000","last_unstale":"2024-07-17T08:37:18.824046+0000","last_undegraded":"2024-07-17T08:37:18.824046+0000","last_fullsized":"2024-07-17T08:37:18.824046+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:44.438486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842558+0000","last_change":"2024-07-17T08:37:18.840783+0000","last_active":"2024-07-17T08:37:18.842558+0000","last_peered":"2024-07-17T08:37:18.842558+0000","last_clean":"2024-07-17T08:37:18.842558+0000","last_became_active":"2024-07-17T08:37:18.840447+0000","last_became_peered":"2024-07-17T08:37:18.840447+0000","last_unstale":"2024-07-17T08:37:18.842558+0000","last_undegraded":"2024-07-17T08:37:18.842558+0000","last_fullsized":"2024-07-17T08:37:18.842558+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:23.650671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841862+0000","last_change":"2024-07-17T08:37:18.837910+0000","last_active":"2024-07-17T08:37:18.841862+0000","last_peered":"2024-07-17T08:37:18.841862+0000","last_clean":"2024-07-17T08:37:18.841862+0000","last_became_active":"2024-07-17T08:37:18.837616+0000","last_became_peered":"2024-07-17T08:37:18.837616+0000","last_unstale":"2024-07-17T08:37:18.841862+0000","last_undegraded":"2024-07-17T08:37:18.841862+0000","last_fullsized":"2024-07-17T08:37:18.841862+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:38.765630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852719+0000","last_change":"2024-07-17T08:37:18.849730+0000","last_active":"2024-07-17T08:37:18.852719+0000","last_peered":"2024-07-17T08:37:18.852719+0000","last_clean":"2024-07-17T08:37:18.852719+0000","last_became_active":"2024-07-17T08:37:18.849422+0000","last_became_peered":"2024-07-17T08:37:18.849422+0000","last_unstale":"2024-07-17T08:37:18.852719+0000","last_undegraded":"2024-07-17T08:37:18.852719+0000","last_fullsized":"2024-07-17T08:37:18.852719+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:27.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842887+0000","last_change":"2024-07-17T08:37:18.840358+0000","last_active":"2024-07-17T08:37:18.842887+0000","last_peered":"2024-07-17T08:37:18.842887+0000","last_clean":"2024-07-17T08:37:18.842887+0000","last_became_active":"2024-07-17T08:37:18.838902+0000","last_became_peered":"2024-07-17T08:37:18.838902+0000","last_unstale":"2024-07-17T08:37:18.842887+0000","last_undegraded":"2024-07-17T08:37:18.842887+0000","last_fullsized":"2024-07-17T08:37:18.842887+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:42:10.351738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.845086+0000","last_change":"2024-07-17T08:37:18.839303+0000","last_active":"2024-07-17T08:37:18.845086+0000","last_peered":"2024-07-17T08:37:18.845086+0000","last_clean":"2024-07-17T08:37:18.845086+0000","last_became_active":"2024-07-17T08:37:18.839043+0000","last_became_peered":"2024-07-17T08:37:18.839043+0000","last_unstale":"2024-07-17T08:37:18.845086+0000","last_undegraded":"2024-07-17T08:37:18.845086+0000","last_fullsized":"2024-07-17T08:37:18.845086+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:26.105486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833165+0000","last_change":"2024-07-17T08:37:18.831044+0000","last_active":"2024-07-17T08:37:18.833165+0000","last_peered":"2024-07-17T08:37:18.833165+0000","last_clean":"2024-07-17T08:37:18.833165+0000","last_became_active":"2024-07-17T08:37:18.830635+0000","last_became_peered":"2024-07-17T08:37:18.830635+0000","last_unstale":"2024-07-17T08:37:18.833165+0000","last_undegraded":"2024-07-17T08:37:18.833165+0000","last_fullsized":"2024-07-17T08:37:18.833165+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:31.780410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843680+0000","last_change":"2024-07-17T08:37:18.836001+0000","last_active":"2024-07-17T08:37:18.843680+0000","last_peered":"2024-07-17T08:37:18.843680+0000","last_clean":"2024-07-17T08:37:18.843680+0000","last_became_active":"2024-07-17T08:37:18.834701+0000","last_became_peered":"2024-07-17T08:37:18.834701+0000","last_unstale":"2024-07-17T08:37:18.843680+0000","last_undegraded":"2024-07-17T08:37:18.843680+0000","last_fullsized":"2024-07-17T08:37:18.843680+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:29.115550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843761+0000","last_change":"2024-07-17T08:37:18.837436+0000","last_active":"2024-07-17T08:37:18.843761+0000","last_peered":"2024-07-17T08:37:18.843761+0000","last_clean":"2024-07-17T08:37:18.843761+0000","last_became_active":"2024-07-17T08:37:18.837166+0000","last_became_peered":"2024-07-17T08:37:18.837166+0000","last_unstale":"2024-07-17T08:37:18.843761+0000","last_undegraded":"2024-07-17T08:37:18.843761+0000","last_fullsized":"2024-07-17T08:37:18.843761+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:01.072353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854980+0000","last_change":"2024-07-17T08:37:18.854070+0000","last_active":"2024-07-17T08:37:18.854980+0000","last_peered":"2024-07-17T08:37:18.854980+0000","last_clean":"2024-07-17T08:37:18.854980+0000","last_became_active":"2024-07-17T08:37:18.853810+0000","last_became_peered":"2024-07-17T08:37:18.853810+0000","last_unstale":"2024-07-17T08:37:18.854980+0000","last_undegraded":"2024-07-17T08:37:18.854980+0000","last_fullsized":"2024-07-17T08:37:18.854980+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:11.419771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838258+0000","last_change":"2024-07-17T08:37:18.834385+0000","last_active":"2024-07-17T08:37:18.838258+0000","last_peered":"2024-07-17T08:37:18.838258+0000","last_clean":"2024-07-17T08:37:18.838258+0000","last_became_active":"2024-07-17T08:37:18.834100+0000","last_became_peered":"2024-07-17T08:37:18.834100+0000","last_unstale":"2024-07-17T08:37:18.838258+0000","last_undegraded":"2024-07-17T08:37:18.838258+0000","last_fullsized":"2024-07-17T08:37:18.838258+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:03:36.920718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854973+0000","last_change":"2024-07-17T08:37:18.853236+0000","last_active":"2024-07-17T08:37:18.854973+0000","last_peered":"2024-07-17T08:37:18.854973+0000","last_clean":"2024-07-17T08:37:18.854973+0000","last_became_active":"2024-07-17T08:37:18.852890+0000","last_became_peered":"2024-07-17T08:37:18.852890+0000","last_unstale":"2024-07-17T08:37:18.854973+0000","last_undegraded":"2024-07-17T08:37:18.854973+0000","last_fullsized":"2024-07-17T08:37:18.854973+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:40.623338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838527+0000","last_change":"2024-07-17T08:37:18.837433+0000","last_active":"2024-07-17T08:37:18.838527+0000","last_peered":"2024-07-17T08:37:18.838527+0000","last_clean":"2024-07-17T08:37:18.838527+0000","last_became_active":"2024-07-17T08:37:18.836365+0000","last_became_peered":"2024-07-17T08:37:18.836365+0000","last_unstale":"2024-07-17T08:37:18.838527+0000","last_undegraded":"2024-07-17T08:37:18.838527+0000","last_fullsized":"2024-07-17T08:37:18.838527+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:51.645895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838877+0000","last_change":"2024-07-17T08:37:18.831124+0000","last_active":"2024-07-17T08:37:18.838877+0000","last_peered":"2024-07-17T08:37:18.838877+0000","last_clean":"2024-07-17T08:37:18.838877+0000","last_became_active":"2024-07-17T08:37:18.830831+0000","last_became_peered":"2024-07-17T08:37:18.830831+0000","last_unstale":"2024-07-17T08:37:18.838877+0000","last_undegraded":"2024-07-17T08:37:18.838877+0000","last_fullsized":"2024-07-17T08:37:18.838877+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:02:01.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"262'4","reported_seq":16,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852657+0000","last_change":"2024-07-17T08:37:17.804164+0000","last_active":"2024-07-17T08:37:18.852657+0000","last_peered":"2024-07-17T08:37:18.852657+0000","last_clean":"2024-07-17T08:37:18.852657+0000","last_became_active":"2024-07-17T08:37:17.802462+0000","last_became_peered":"2024-07-17T08:37:17.802462+0000","last_unstale":"2024-07-17T08:37:18.852657+0000","last_undegraded":"2024-07-17T08:37:18.852657+0000","last_fullsized":"2024-07-17T08:37:18.852657+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:41.962072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.828900+0000","last_change":"2024-07-17T08:37:18.826601+0000","last_active":"2024-07-17T08:37:18.828900+0000","last_peered":"2024-07-17T08:37:18.828900+0000","last_clean":"2024-07-17T08:37:18.828900+0000","last_became_active":"2024-07-17T08:37:18.826019+0000","last_became_peered":"2024-07-17T08:37:18.826019+0000","last_unstale":"2024-07-17T08:37:18.828900+0000","last_undegraded":"2024-07-17T08:37:18.828900+0000","last_fullsized":"2024-07-17T08:37:18.828900+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:29:26.884680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847593+0000","last_change":"2024-07-17T08:37:17.808715+0000","last_active":"2024-07-17T08:37:18.847593+0000","last_peered":"2024-07-17T08:37:18.847593+0000","last_clean":"2024-07-17T08:37:18.847593+0000","last_became_active":"2024-07-17T08:37:17.808441+0000","last_became_peered":"2024-07-17T08:37:17.808441+0000","last_unstale":"2024-07-17T08:37:18.847593+0000","last_undegraded":"2024-07-17T08:37:18.847593+0000","last_fullsized":"2024-07-17T08:37:18.847593+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:45:03.033848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836581+0000","last_change":"2024-07-17T08:37:18.834297+0000","last_active":"2024-07-17T08:37:18.836581+0000","last_peered":"2024-07-17T08:37:18.836581+0000","last_clean":"2024-07-17T08:37:18.836581+0000","last_became_active":"2024-07-17T08:37:18.832521+0000","last_became_peered":"2024-07-17T08:37:18.832521+0000","last_unstale":"2024-07-17T08:37:18.836581+0000","last_undegraded":"2024-07-17T08:37:18.836581+0000","last_fullsized":"2024-07-17T08:37:18.836581+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:53.339674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843272+0000","last_change":"2024-07-17T08:37:17.812708+0000","last_active":"2024-07-17T08:37:18.843272+0000","last_peered":"2024-07-17T08:37:18.843272+0000","last_clean":"2024-07-17T08:37:18.843272+0000","last_became_active":"2024-07-17T08:37:17.812402+0000","last_became_peered":"2024-07-17T08:37:17.812402+0000","last_unstale":"2024-07-17T08:37:18.843272+0000","last_undegraded":"2024-07-17T08:37:18.843272+0000","last_fullsized":"2024-07-17T08:37:18.843272+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:41:44.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847534+0000","last_change":"2024-07-17T08:37:18.845881+0000","last_active":"2024-07-17T08:37:18.847534+0000","last_peered":"2024-07-17T08:37:18.847534+0000","last_clean":"2024-07-17T08:37:18.847534+0000","last_became_active":"2024-07-17T08:37:18.845558+0000","last_became_peered":"2024-07-17T08:37:18.845558+0000","last_unstale":"2024-07-17T08:37:18.847534+0000","last_undegraded":"2024-07-17T08:37:18.847534+0000","last_fullsized":"2024-07-17T08:37:18.847534+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:39.564695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.846458+0000","last_change":"2024-07-17T08:37:17.804147+0000","last_active":"2024-07-17T08:37:18.846458+0000","last_peered":"2024-07-17T08:37:18.846458+0000","last_clean":"2024-07-17T08:37:18.846458+0000","last_became_active":"2024-07-17T08:37:17.803827+0000","last_became_peered":"2024-07-17T08:37:17.803827+0000","last_unstale":"2024-07-17T08:37:18.846458+0000","last_undegraded":"2024-07-17T08:37:18.846458+0000","last_fullsized":"2024-07-17T08:37:18.846458+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:12.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850667+0000","last_change":"2024-07-17T08:37:18.848036+0000","last_active":"2024-07-17T08:37:18.850667+0000","last_peered":"2024-07-17T08:37:18.850667+0000","last_clean":"2024-07-17T08:37:18.850667+0000","last_became_active":"2024-07-17T08:37:18.847708+0000","last_became_peered":"2024-07-17T08:37:18.847708+0000","last_unstale":"2024-07-17T08:37:18.850667+0000","last_undegraded":"2024-07-17T08:37:18.850667+0000","last_fullsized":"2024-07-17T08:37:18.850667+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:45.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850458+0000","last_change":"2024-07-17T08:37:17.801026+0000","last_active":"2024-07-17T08:37:18.850458+0000","last_peered":"2024-07-17T08:37:18.850458+0000","last_clean":"2024-07-17T08:37:18.850458+0000","last_became_active":"2024-07-17T08:37:17.800731+0000","last_became_peered":"2024-07-17T08:37:17.800731+0000","last_unstale":"2024-07-17T08:37:18.850458+0000","last_undegraded":"2024-07-17T08:37:18.850458+0000","last_fullsized":"2024-07-17T08:37:18.850458+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:35.363277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841058+0000","last_change":"2024-07-17T08:37:18.835823+0000","last_active":"2024-07-17T08:37:18.841058+0000","last_peered":"2024-07-17T08:37:18.841058+0000","last_clean":"2024-07-17T08:37:18.841058+0000","last_became_active":"2024-07-17T08:37:18.832770+0000","last_became_peered":"2024-07-17T08:37:18.832770+0000","last_unstale":"2024-07-17T08:37:18.841058+0000","last_undegraded":"2024-07-17T08:37:18.841058+0000","last_fullsized":"2024-07-17T08:37:18.841058+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:27.289210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850490+0000","last_change":"2024-07-17T08:37:17.799706+0000","last_active":"2024-07-17T08:37:18.850490+0000","last_peered":"2024-07-17T08:37:18.850490+0000","last_clean":"2024-07-17T08:37:18.850490+0000","last_became_active":"2024-07-17T08:37:17.799414+0000","last_became_peered":"2024-07-17T08:37:17.799414+0000","last_unstale":"2024-07-17T08:37:18.850490+0000","last_undegraded":"2024-07-17T08:37:18.850490+0000","last_fullsized":"2024-07-17T08:37:18.850490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:44:10.098693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854706+0000","last_change":"2024-07-17T08:37:18.852385+0000","last_active":"2024-07-17T08:37:18.854706+0000","last_peered":"2024-07-17T08:37:18.854706+0000","last_clean":"2024-07-17T08:37:18.854706+0000","last_became_active":"2024-07-17T08:37:18.852059+0000","last_became_peered":"2024-07-17T08:37:18.852059+0000","last_unstale":"2024-07-17T08:37:18.854706+0000","last_undegraded":"2024-07-17T08:37:18.854706+0000","last_fullsized":"2024-07-17T08:37:18.854706+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:02.423177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.816530+0000","last_change":"2024-07-17T08:37:17.806029+0000","last_active":"2024-07-17T08:37:18.816530+0000","last_peered":"2024-07-17T08:37:18.816530+0000","last_clean":"2024-07-17T08:37:18.816530+0000","last_became_active":"2024-07-17T08:37:17.805592+0000","last_became_peered":"2024-07-17T08:37:17.805592+0000","last_unstale":"2024-07-17T08:37:18.816530+0000","last_undegraded":"2024-07-17T08:37:18.816530+0000","last_fullsized":"2024-07-17T08:37:18.816530+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:17.318614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839882+0000","last_change":"2024-07-17T08:37:18.836745+0000","last_active":"2024-07-17T08:37:18.839882+0000","last_peered":"2024-07-17T08:37:18.839882+0000","last_clean":"2024-07-17T08:37:18.839882+0000","last_became_active":"2024-07-17T08:37:18.834613+0000","last_became_peered":"2024-07-17T08:37:18.834613+0000","last_unstale":"2024-07-17T08:37:18.839882+0000","last_undegraded":"2024-07-17T08:37:18.839882+0000","last_fullsized":"2024-07-17T08:37:18.839882+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:04.256365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.802981+0000","last_change":"2024-07-17T08:37:17.810344+0000","last_active":"2024-07-17T08:37:18.802981+0000","last_peered":"2024-07-17T08:37:18.802981+0000","last_clean":"2024-07-17T08:37:18.802981+0000","last_became_active":"2024-07-17T08:37:17.810021+0000","last_became_peered":"2024-07-17T08:37:17.810021+0000","last_unstale":"2024-07-17T08:37:18.802981+0000","last_undegraded":"2024-07-17T08:37:18.802981+0000","last_fullsized":"2024-07-17T08:37:18.802981+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:41.174971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826063+0000","last_change":"2024-07-17T08:37:18.818912+0000","last_active":"2024-07-17T08:37:18.826063+0000","last_peered":"2024-07-17T08:37:18.826063+0000","last_clean":"2024-07-17T08:37:18.826063+0000","last_became_active":"2024-07-17T08:37:18.818137+0000","last_became_peered":"2024-07-17T08:37:18.818137+0000","last_unstale":"2024-07-17T08:37:18.826063+0000","last_undegraded":"2024-07-17T08:37:18.826063+0000","last_fullsized":"2024-07-17T08:37:18.826063+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:58.127022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.815037+0000","last_change":"2024-07-17T08:37:17.812708+0000","last_active":"2024-07-17T08:37:18.815037+0000","last_peered":"2024-07-17T08:37:18.815037+0000","last_clean":"2024-07-17T08:37:18.815037+0000","last_became_active":"2024-07-17T08:37:17.812172+0000","last_became_peered":"2024-07-17T08:37:17.812172+0000","last_unstale":"2024-07-17T08:37:18.815037+0000","last_undegraded":"2024-07-17T08:37:18.815037+0000","last_fullsized":"2024-07-17T08:37:18.815037+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:10.299090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826296+0000","last_change":"2024-07-17T08:37:18.822752+0000","last_active":"2024-07-17T08:37:18.826296+0000","last_peered":"2024-07-17T08:37:18.826296+0000","last_clean":"2024-07-17T08:37:18.826296+0000","last_became_active":"2024-07-17T08:37:18.822334+0000","last_became_peered":"2024-07-17T08:37:18.822334+0000","last_unstale":"2024-07-17T08:37:18.826296+0000","last_undegraded":"2024-07-17T08:37:18.826296+0000","last_fullsized":"2024-07-17T08:37:18.826296+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:48.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.822647+0000","last_change":"2024-07-17T08:37:17.807022+0000","last_active":"2024-07-17T08:37:18.822647+0000","last_peered":"2024-07-17T08:37:18.822647+0000","last_clean":"2024-07-17T08:37:18.822647+0000","last_became_active":"2024-07-17T08:37:17.806415+0000","last_became_peered":"2024-07-17T08:37:17.806415+0000","last_unstale":"2024-07-17T08:37:18.822647+0000","last_undegraded":"2024-07-17T08:37:18.822647+0000","last_fullsized":"2024-07-17T08:37:18.822647+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:43.251800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826962+0000","last_change":"2024-07-17T08:37:18.825283+0000","last_active":"2024-07-17T08:37:18.826962+0000","last_peered":"2024-07-17T08:37:18.826962+0000","last_clean":"2024-07-17T08:37:18.826962+0000","last_became_active":"2024-07-17T08:37:18.824931+0000","last_became_peered":"2024-07-17T08:37:18.824931+0000","last_unstale":"2024-07-17T08:37:18.826962+0000","last_undegraded":"2024-07-17T08:37:18.826962+0000","last_fullsized":"2024-07-17T08:37:18.826962+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:04.616603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"262'3","reported_seq":15,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850978+0000","last_change":"2024-07-17T08:37:17.808985+0000","last_active":"2024-07-17T08:37:18.850978+0000","last_peered":"2024-07-17T08:37:18.850978+0000","last_clean":"2024-07-17T08:37:18.850978+0000","last_became_active":"2024-07-17T08:37:17.808641+0000","last_became_peered":"2024-07-17T08:37:17.808641+0000","last_unstale":"2024-07-17T08:37:18.850978+0000","last_undegraded":"2024-07-17T08:37:18.850978+0000","last_fullsized":"2024-07-17T08:37:18.850978+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:50.627655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843794+0000","last_change":"2024-07-17T08:37:18.837143+0000","last_active":"2024-07-17T08:37:18.843794+0000","last_peered":"2024-07-17T08:37:18.843794+0000","last_clean":"2024-07-17T08:37:18.843794+0000","last_became_active":"2024-07-17T08:37:18.836852+0000","last_became_peered":"2024-07-17T08:37:18.836852+0000","last_unstale":"2024-07-17T08:37:18.843794+0000","last_undegraded":"2024-07-17T08:37:18.843794+0000","last_fullsized":"2024-07-17T08:37:18.843794+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:09.651128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"262'6","reported_seq":18,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852977+0000","last_change":"2024-07-17T08:37:17.805610+0000","last_active":"2024-07-17T08:37:18.852977+0000","last_peered":"2024-07-17T08:37:18.852977+0000","last_clean":"2024-07-17T08:37:18.852977+0000","last_became_active":"2024-07-17T08:37:17.805108+0000","last_became_peered":"2024-07-17T08:37:17.805108+0000","last_unstale":"2024-07-17T08:37:18.852977+0000","last_undegraded":"2024-07-17T08:37:18.852977+0000","last_fullsized":"2024-07-17T08:37:18.852977+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:44.730802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.835358+0000","last_change":"2024-07-17T08:37:18.832037+0000","last_active":"2024-07-17T08:37:18.835358+0000","last_peered":"2024-07-17T08:37:18.835358+0000","last_clean":"2024-07-17T08:37:18.835358+0000","last_became_active":"2024-07-17T08:37:18.830707+0000","last_became_peered":"2024-07-17T08:37:18.830707+0000","last_unstale":"2024-07-17T08:37:18.835358+0000","last_undegraded":"2024-07-17T08:37:18.835358+0000","last_fullsized":"2024-07-17T08:37:18.835358+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:07.382219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.848983+0000","last_change":"2024-07-17T08:37:17.805988+0000","last_active":"2024-07-17T08:37:18.848983+0000","last_peered":"2024-07-17T08:37:18.848983+0000","last_clean":"2024-07-17T08:37:18.848983+0000","last_became_active":"2024-07-17T08:37:17.805550+0000","last_became_peered":"2024-07-17T08:37:17.805550+0000","last_unstale":"2024-07-17T08:37:18.848983+0000","last_undegraded":"2024-07-17T08:37:18.848983+0000","last_fullsized":"2024-07-17T08:37:18.848983+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:54.094039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.835243+0000","last_change":"2024-07-17T08:37:18.832302+0000","last_active":"2024-07-17T08:37:18.835243+0000","last_peered":"2024-07-17T08:37:18.835243+0000","last_clean":"2024-07-17T08:37:18.835243+0000","last_became_active":"2024-07-17T08:37:18.832024+0000","last_became_peered":"2024-07-17T08:37:18.832024+0000","last_unstale":"2024-07-17T08:37:18.835243+0000","last_undegraded":"2024-07-17T08:37:18.835243+0000","last_fullsized":"2024-07-17T08:37:18.835243+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:34.593534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.809762+0000","last_change":"2024-07-17T08:37:17.802128+0000","last_active":"2024-07-17T08:37:18.809762+0000","last_peered":"2024-07-17T08:37:18.809762+0000","last_clean":"2024-07-17T08:37:18.809762+0000","last_became_active":"2024-07-17T08:37:17.801392+0000","last_became_peered":"2024-07-17T08:37:17.801392+0000","last_unstale":"2024-07-17T08:37:18.809762+0000","last_undegraded":"2024-07-17T08:37:18.809762+0000","last_fullsized":"2024-07-17T08:37:18.809762+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:03.573199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839802+0000","last_change":"2024-07-17T08:37:18.833520+0000","last_active":"2024-07-17T08:37:18.839802+0000","last_peered":"2024-07-17T08:37:18.839802+0000","last_clean":"2024-07-17T08:37:18.839802+0000","last_became_active":"2024-07-17T08:37:18.833211+0000","last_became_peered":"2024-07-17T08:37:18.833211+0000","last_unstale":"2024-07-17T08:37:18.839802+0000","last_undegraded":"2024-07-17T08:37:18.839802+0000","last_fullsized":"2024-07-17T08:37:18.839802+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:46.218558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.813590+0000","last_change":"2024-07-17T08:37:17.798510+0000","last_active":"2024-07-17T08:37:18.813590+0000","last_peered":"2024-07-17T08:37:18.813590+0000","last_clean":"2024-07-17T08:37:18.813590+0000","last_became_active":"2024-07-17T08:37:17.798231+0000","last_became_peered":"2024-07-17T08:37:17.798231+0000","last_unstale":"2024-07-17T08:37:18.813590+0000","last_undegraded":"2024-07-17T08:37:18.813590+0000","last_fullsized":"2024-07-17T08:37:18.813590+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:10.410789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839193+0000","last_change":"2024-07-17T08:37:18.833412+0000","last_active":"2024-07-17T08:37:18.839193+0000","last_peered":"2024-07-17T08:37:18.839193+0000","last_clean":"2024-07-17T08:37:18.839193+0000","last_became_active":"2024-07-17T08:37:18.829375+0000","last_became_peered":"2024-07-17T08:37:18.829375+0000","last_unstale":"2024-07-17T08:37:18.839193+0000","last_undegraded":"2024-07-17T08:37:18.839193+0000","last_fullsized":"2024-07-17T08:37:18.839193+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:43:03.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"262'8","reported_seq":20,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847212+0000","last_change":"2024-07-17T08:37:17.809359+0000","last_active":"2024-07-17T08:37:18.847212+0000","last_peered":"2024-07-17T08:37:18.847212+0000","last_clean":"2024-07-17T08:37:18.847212+0000","last_became_active":"2024-07-17T08:37:17.809062+0000","last_became_peered":"2024-07-17T08:37:17.809062+0000","last_unstale":"2024-07-17T08:37:18.847212+0000","last_undegraded":"2024-07-17T08:37:18.847212+0000","last_fullsized":"2024-07-17T08:37:18.847212+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:57.278603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842735+0000","last_change":"2024-07-17T08:37:18.832363+0000","last_active":"2024-07-17T08:37:18.842735+0000","last_peered":"2024-07-17T08:37:18.842735+0000","last_clean":"2024-07-17T08:37:18.842735+0000","last_became_active":"2024-07-17T08:37:18.832050+0000","last_became_peered":"2024-07-17T08:37:18.832050+0000","last_unstale":"2024-07-17T08:37:18.842735+0000","last_undegraded":"2024-07-17T08:37:18.842735+0000","last_fullsized":"2024-07-17T08:37:18.842735+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:56.386723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.848240+0000","last_change":"2024-07-17T08:37:17.807706+0000","last_active":"2024-07-17T08:37:18.848240+0000","last_peered":"2024-07-17T08:37:18.848240+0000","last_clean":"2024-07-17T08:37:18.848240+0000","last_became_active":"2024-07-17T08:37:17.807400+0000","last_became_peered":"2024-07-17T08:37:17.807400+0000","last_unstale":"2024-07-17T08:37:18.848240+0000","last_undegraded":"2024-07-17T08:37:18.848240+0000","last_fullsized":"2024-07-17T08:37:18.848240+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:47.580344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.845142+0000","last_change":"2024-07-17T08:37:18.838688+0000","last_active":"2024-07-17T08:37:18.845142+0000","last_peered":"2024-07-17T08:37:18.845142+0000","last_clean":"2024-07-17T08:37:18.845142+0000","last_became_active":"2024-07-17T08:37:18.838357+0000","last_became_peered":"2024-07-17T08:37:18.838357+0000","last_unstale":"2024-07-17T08:37:18.845142+0000","last_undegraded":"2024-07-17T08:37:18.845142+0000","last_fullsized":"2024-07-17T08:37:18.845142+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:28:20.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.851943+0000","last_change":"2024-07-17T08:37:17.796761+0000","last_active":"2024-07-17T08:37:18.851943+0000","last_peered":"2024-07-17T08:37:18.851943+0000","last_clean":"2024-07-17T08:37:18.851943+0000","last_became_active":"2024-07-17T08:37:17.796435+0000","last_became_peered":"2024-07-17T08:37:17.796435+0000","last_unstale":"2024-07-17T08:37:18.851943+0000","last_undegraded":"2024-07-17T08:37:18.851943+0000","last_fullsized":"2024-07-17T08:37:18.851943+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:52:39.796928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850418+0000","last_change":"2024-07-17T08:37:18.839615+0000","last_active":"2024-07-17T08:37:18.850418+0000","last_peered":"2024-07-17T08:37:18.850418+0000","last_clean":"2024-07-17T08:37:18.850418+0000","last_became_active":"2024-07-17T08:37:18.839099+0000","last_became_peered":"2024-07-17T08:37:18.839099+0000","last_unstale":"2024-07-17T08:37:18.850418+0000","last_undegraded":"2024-07-17T08:37:18.850418+0000","last_fullsized":"2024-07-17T08:37:18.850418+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:19.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.846646+0000","last_change":"2024-07-17T08:37:17.790148+0000","last_active":"2024-07-17T08:37:18.846646+0000","last_peered":"2024-07-17T08:37:18.846646+0000","last_clean":"2024-07-17T08:37:18.846646+0000","last_became_active":"2024-07-17T08:37:17.789799+0000","last_became_peered":"2024-07-17T08:37:17.789799+0000","last_unstale":"2024-07-17T08:37:18.846646+0000","last_undegraded":"2024-07-17T08:37:18.846646+0000","last_fullsized":"2024-07-17T08:37:18.846646+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:37.370506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.834016+0000","last_change":"2024-07-17T08:37:18.830746+0000","last_active":"2024-07-17T08:37:18.834016+0000","last_peered":"2024-07-17T08:37:18.834016+0000","last_clean":"2024-07-17T08:37:18.834016+0000","last_became_active":"2024-07-17T08:37:18.830366+0000","last_became_peered":"2024-07-17T08:37:18.830366+0000","last_unstale":"2024-07-17T08:37:18.834016+0000","last_undegraded":"2024-07-17T08:37:18.834016+0000","last_fullsized":"2024-07-17T08:37:18.834016+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:16.848300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.817810+0000","last_change":"2024-07-17T08:37:17.813190+0000","last_active":"2024-07-17T08:37:18.817810+0000","last_peered":"2024-07-17T08:37:18.817810+0000","last_clean":"2024-07-17T08:37:18.817810+0000","last_became_active":"2024-07-17T08:37:17.812787+0000","last_became_peered":"2024-07-17T08:37:17.812787+0000","last_unstale":"2024-07-17T08:37:18.817810+0000","last_undegraded":"2024-07-17T08:37:18.817810+0000","last_fullsized":"2024-07-17T08:37:18.817810+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:00.425488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841962+0000","last_change":"2024-07-17T08:37:18.837433+0000","last_active":"2024-07-17T08:37:18.841962+0000","last_peered":"2024-07-17T08:37:18.841962+0000","last_clean":"2024-07-17T08:37:18.841962+0000","last_became_active":"2024-07-17T08:37:18.837050+0000","last_became_peered":"2024-07-17T08:37:18.837050+0000","last_unstale":"2024-07-17T08:37:18.841962+0000","last_undegraded":"2024-07-17T08:37:18.841962+0000","last_fullsized":"2024-07-17T08:37:18.841962+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:10.662238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854682+0000","last_change":"2024-07-17T08:37:17.809099+0000","last_active":"2024-07-17T08:37:18.854682+0000","last_peered":"2024-07-17T08:37:18.854682+0000","last_clean":"2024-07-17T08:37:18.854682+0000","last_became_active":"2024-07-17T08:37:17.808776+0000","last_became_peered":"2024-07-17T08:37:17.808776+0000","last_unstale":"2024-07-17T08:37:18.854682+0000","last_undegraded":"2024-07-17T08:37:18.854682+0000","last_fullsized":"2024-07-17T08:37:18.854682+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:25.959428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852672+0000","last_change":"2024-07-17T08:37:18.849632+0000","last_active":"2024-07-17T08:37:18.852672+0000","last_peered":"2024-07-17T08:37:18.852672+0000","last_clean":"2024-07-17T08:37:18.852672+0000","last_became_active":"2024-07-17T08:37:18.849307+0000","last_became_peered":"2024-07-17T08:37:18.849307+0000","last_unstale":"2024-07-17T08:37:18.852672+0000","last_undegraded":"2024-07-17T08:37:18.852672+0000","last_fullsized":"2024-07-17T08:37:18.852672+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:49.721269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.800088+0000","last_change":"2024-07-17T08:37:17.809016+0000","last_active":"2024-07-17T08:37:18.800088+0000","last_peered":"2024-07-17T08:37:18.800088+0000","last_clean":"2024-07-17T08:37:18.800088+0000","last_became_active":"2024-07-17T08:37:17.808718+0000","last_became_peered":"2024-07-17T08:37:17.808718+0000","last_unstale":"2024-07-17T08:37:18.800088+0000","last_undegraded":"2024-07-17T08:37:18.800088+0000","last_fullsized":"2024-07-17T08:37:18.800088+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:21.159778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839773+0000","last_change":"2024-07-17T08:37:18.834909+0000","last_active":"2024-07-17T08:37:18.839773+0000","last_peered":"2024-07-17T08:37:18.839773+0000","last_clean":"2024-07-17T08:37:18.839773+0000","last_became_active":"2024-07-17T08:37:18.834630+0000","last_became_peered":"2024-07-17T08:37:18.834630+0000","last_unstale":"2024-07-17T08:37:18.839773+0000","last_undegraded":"2024-07-17T08:37:18.839773+0000","last_fullsized":"2024-07-17T08:37:18.839773+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:45.701907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.827002+0000","last_change":"2024-07-17T08:37:17.809179+0000","last_active":"2024-07-17T08:37:18.827002+0000","last_peered":"2024-07-17T08:37:18.827002+0000","last_clean":"2024-07-17T08:37:18.827002+0000","last_became_active":"2024-07-17T08:37:17.808874+0000","last_became_peered":"2024-07-17T08:37:17.808874+0000","last_unstale":"2024-07-17T08:37:18.827002+0000","last_undegraded":"2024-07-17T08:37:18.827002+0000","last_fullsized":"2024-07-17T08:37:18.827002+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:43:53.512695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854616+0000","last_change":"2024-07-17T08:37:18.853410+0000","last_active":"2024-07-17T08:37:18.854616+0000","last_peered":"2024-07-17T08:37:18.854616+0000","last_clean":"2024-07-17T08:37:18.854616+0000","last_became_active":"2024-07-17T08:37:18.853127+0000","last_became_peered":"2024-07-17T08:37:18.853127+0000","last_unstale":"2024-07-17T08:37:18.854616+0000","last_undegraded":"2024-07-17T08:37:18.854616+0000","last_fullsized":"2024-07-17T08:37:18.854616+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:56.825447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.817914+0000","last_change":"2024-07-17T08:37:17.806994+0000","last_active":"2024-07-17T08:37:18.817914+0000","last_peered":"2024-07-17T08:37:18.817914+0000","last_clean":"2024-07-17T08:37:18.817914+0000","last_became_active":"2024-07-17T08:37:17.806665+0000","last_became_peered":"2024-07-17T08:37:17.806665+0000","last_unstale":"2024-07-17T08:37:18.817914+0000","last_undegraded":"2024-07-17T08:37:18.817914+0000","last_fullsized":"2024-07-17T08:37:18.817914+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:39.162444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826085+0000","last_change":"2024-07-17T08:37:18.818833+0000","last_active":"2024-07-17T08:37:18.826085+0000","last_peered":"2024-07-17T08:37:18.826085+0000","last_clean":"2024-07-17T08:37:18.826085+0000","last_became_active":"2024-07-17T08:37:18.817716+0000","last_became_peered":"2024-07-17T08:37:18.817716+0000","last_unstale":"2024-07-17T08:37:18.826085+0000","last_undegraded":"2024-07-17T08:37:18.826085+0000","last_fullsized":"2024-07-17T08:37:18.826085+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:16.713235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847014+0000","last_change":"2024-07-17T08:37:17.815629+0000","last_active":"2024-07-17T08:37:18.847014+0000","last_peered":"2024-07-17T08:37:18.847014+0000","last_clean":"2024-07-17T08:37:18.847014+0000","last_became_active":"2024-07-17T08:37:17.815305+0000","last_became_peered":"2024-07-17T08:37:17.815305+0000","last_unstale":"2024-07-17T08:37:18.847014+0000","last_undegraded":"2024-07-17T08:37:18.847014+0000","last_fullsized":"2024-07-17T08:37:18.847014+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:39.979339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833827+0000","last_change":"2024-07-17T08:37:18.830011+0000","last_active":"2024-07-17T08:37:18.833827+0000","last_peered":"2024-07-17T08:37:18.833827+0000","last_clean":"2024-07-17T08:37:18.833827+0000","last_became_active":"2024-07-17T08:37:18.829677+0000","last_became_peered":"2024-07-17T08:37:18.829677+0000","last_unstale":"2024-07-17T08:37:18.833827+0000","last_undegraded":"2024-07-17T08:37:18.833827+0000","last_fullsized":"2024-07-17T08:37:18.833827+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:45.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.813720+0000","last_change":"2024-07-17T08:37:17.803118+0000","last_active":"2024-07-17T08:37:18.813720+0000","last_peered":"2024-07-17T08:37:18.813720+0000","last_clean":"2024-07-17T08:37:18.813720+0000","last_became_active":"2024-07-17T08:37:17.802439+0000","last_became_peered":"2024-07-17T08:37:17.802439+0000","last_unstale":"2024-07-17T08:37:18.813720+0000","last_undegraded":"2024-07-17T08:37:18.813720+0000","last_fullsized":"2024-07-17T08:37:18.813720+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:03.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826676+0000","last_change":"2024-07-17T08:37:18.818293+0000","last_active":"2024-07-17T08:37:18.826676+0000","last_peered":"2024-07-17T08:37:18.826676+0000","last_clean":"2024-07-17T08:37:18.826676+0000","last_became_active":"2024-07-17T08:37:18.817958+0000","last_became_peered":"2024-07-17T08:37:18.817958+0000","last_unstale":"2024-07-17T08:37:18.826676+0000","last_undegraded":"2024-07-17T08:37:18.826676+0000","last_fullsized":"2024-07-17T08:37:18.826676+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:10.445048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.820343+0000","last_change":"2024-07-17T08:37:17.810382+0000","last_active":"2024-07-17T08:37:18.820343+0000","last_peered":"2024-07-17T08:37:18.820343+0000","last_clean":"2024-07-17T08:37:18.820343+0000","last_became_active":"2024-07-17T08:37:17.810080+0000","last_became_peered":"2024-07-17T08:37:17.810080+0000","last_unstale":"2024-07-17T08:37:18.820343+0000","last_undegraded":"2024-07-17T08:37:18.820343+0000","last_fullsized":"2024-07-17T08:37:18.820343+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:51:05.786598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843592+0000","last_change":"2024-07-17T08:37:18.840577+0000","last_active":"2024-07-17T08:37:18.843592+0000","last_peered":"2024-07-17T08:37:18.843592+0000","last_clean":"2024-07-17T08:37:18.843592+0000","last_became_active":"2024-07-17T08:37:18.840298+0000","last_became_peered":"2024-07-17T08:37:18.840298+0000","last_unstale":"2024-07-17T08:37:18.843592+0000","last_undegraded":"2024-07-17T08:37:18.843592+0000","last_fullsized":"2024-07-17T08:37:18.843592+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:29:47.038922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.810339+0000","last_change":"2024-07-17T08:37:17.809261+0000","last_active":"2024-07-17T08:37:18.810339+0000","last_peered":"2024-07-17T08:37:18.810339+0000","last_clean":"2024-07-17T08:37:18.810339+0000","last_became_active":"2024-07-17T08:37:17.808937+0000","last_became_peered":"2024-07-17T08:37:17.808937+0000","last_unstale":"2024-07-17T08:37:18.810339+0000","last_undegraded":"2024-07-17T08:37:18.810339+0000","last_fullsized":"2024-07-17T08:37:18.810339+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:11.890369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836518+0000","last_change":"2024-07-17T08:37:18.834100+0000","last_active":"2024-07-17T08:37:18.836518+0000","last_peered":"2024-07-17T08:37:18.836518+0000","last_clean":"2024-07-17T08:37:18.836518+0000","last_became_active":"2024-07-17T08:37:18.833809+0000","last_became_peered":"2024-07-17T08:37:18.833809+0000","last_unstale":"2024-07-17T08:37:18.836518+0000","last_undegraded":"2024-07-17T08:37:18.836518+0000","last_fullsized":"2024-07-17T08:37:18.836518+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:07.502846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.823129+0000","last_change":"2024-07-17T08:37:17.805773+0000","last_active":"2024-07-17T08:37:18.823129+0000","last_peered":"2024-07-17T08:37:18.823129+0000","last_clean":"2024-07-17T08:37:18.823129+0000","last_became_active":"2024-07-17T08:37:17.804458+0000","last_became_peered":"2024-07-17T08:37:17.804458+0000","last_unstale":"2024-07-17T08:37:18.823129+0000","last_undegraded":"2024-07-17T08:37:18.823129+0000","last_fullsized":"2024-07-17T08:37:18.823129+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:23.755011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.828911+0000","last_change":"2024-07-17T08:37:18.827315+0000","last_active":"2024-07-17T08:37:18.828911+0000","last_peered":"2024-07-17T08:37:18.828911+0000","last_clean":"2024-07-17T08:37:18.828911+0000","last_became_active":"2024-07-17T08:37:18.827038+0000","last_became_peered":"2024-07-17T08:37:18.827038+0000","last_unstale":"2024-07-17T08:37:18.828911+0000","last_undegraded":"2024-07-17T08:37:18.828911+0000","last_fullsized":"2024-07-17T08:37:18.828911+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:33.199619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.800591+0000","last_change":"2024-07-17T08:37:17.808953+0000","last_active":"2024-07-17T08:37:18.800591+0000","last_peered":"2024-07-17T08:37:18.800591+0000","last_clean":"2024-07-17T08:37:18.800591+0000","last_became_active":"2024-07-17T08:37:17.808653+0000","last_became_peered":"2024-07-17T08:37:17.808653+0000","last_unstale":"2024-07-17T08:37:18.800591+0000","last_undegraded":"2024-07-17T08:37:18.800591+0000","last_fullsized":"2024-07-17T08:37:18.800591+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:41.093679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838756+0000","last_change":"2024-07-17T08:37:18.828266+0000","last_active":"2024-07-17T08:37:18.838756+0000","last_peered":"2024-07-17T08:37:18.838756+0000","last_clean":"2024-07-17T08:37:18.838756+0000","last_became_active":"2024-07-17T08:37:18.827934+0000","last_became_peered":"2024-07-17T08:37:18.827934+0000","last_unstale":"2024-07-17T08:37:18.838756+0000","last_undegraded":"2024-07-17T08:37:18.838756+0000","last_fullsized":"2024-07-17T08:37:18.838756+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:10.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.814072+0000","last_change":"2024-07-17T08:37:17.813364+0000","last_active":"2024-07-17T08:37:18.814072+0000","last_peered":"2024-07-17T08:37:18.814072+0000","last_clean":"2024-07-17T08:37:18.814072+0000","last_became_active":"2024-07-17T08:37:17.812231+0000","last_became_peered":"2024-07-17T08:37:17.812231+0000","last_unstale":"2024-07-17T08:37:18.814072+0000","last_undegraded":"2024-07-17T08:37:18.814072+0000","last_fullsized":"2024-07-17T08:37:18.814072+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:33.575293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.849172+0000","last_change":"2024-07-17T08:37:18.844740+0000","last_active":"2024-07-17T08:37:18.849172+0000","last_peered":"2024-07-17T08:37:18.849172+0000","last_clean":"2024-07-17T08:37:18.849172+0000","last_became_active":"2024-07-17T08:37:18.844381+0000","last_became_peered":"2024-07-17T08:37:18.844381+0000","last_unstale":"2024-07-17T08:37:18.849172+0000","last_undegraded":"2024-07-17T08:37:18.849172+0000","last_fullsized":"2024-07-17T08:37:18.849172+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:11.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.818134+0000","last_change":"2024-07-17T08:37:17.812584+0000","last_active":"2024-07-17T08:37:18.818134+0000","last_peered":"2024-07-17T08:37:18.818134+0000","last_clean":"2024-07-17T08:37:18.818134+0000","last_became_active":"2024-07-17T08:37:17.812307+0000","last_became_peered":"2024-07-17T08:37:17.812307+0000","last_unstale":"2024-07-17T08:37:18.818134+0000","last_undegraded":"2024-07-17T08:37:18.818134+0000","last_fullsized":"2024-07-17T08:37:18.818134+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:12.466725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838543+0000","last_change":"2024-07-17T08:37:18.830703+0000","last_active":"2024-07-17T08:37:18.838543+0000","last_peered":"2024-07-17T08:37:18.838543+0000","last_clean":"2024-07-17T08:37:18.838543+0000","last_became_active":"2024-07-17T08:37:18.829187+0000","last_became_peered":"2024-07-17T08:37:18.829187+0000","last_unstale":"2024-07-17T08:37:18.838543+0000","last_undegraded":"2024-07-17T08:37:18.838543+0000","last_fullsized":"2024-07-17T08:37:18.838543+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:33.891781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"262'4183","reported_seq":5188,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:19.463908+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:37:19.463908+0000","last_peered":"2024-07-17T08:37:19.463908+0000","last_clean":"2024-07-17T08:37:19.463908+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:37:19.463908+0000","last_undegraded":"2024-07-17T08:37:19.463908+0000","last_fullsized":"2024-07-17T08:37:19.463908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":385,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":385,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4183,"ondisk_log_size":4183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543363,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71088,"kb_used_data":3812,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300752,"statfs":{"total":96636764160,"available":96563970048,"internally_reserved":0,"allocated":3903488,"data_stored":2402785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510658,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48320,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323520,"statfs":{"total":96636764160,"available":96587284480,"internally_reserved":0,"allocated":3321856,"data_stored":1820160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70532,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301308,"statfs":{"total":96636764160,"available":96564539392,"internally_reserved":0,"allocated":3330048,"data_stored":1820216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543364,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52492,"kb_used_data":3260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319348,"statfs":{"total":96636764160,"available":96583012352,"internally_reserved":0,"allocated":3338240,"data_stored":1821862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52484,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319356,"statfs":{"total":96636764160,"available":96583020544,"internally_reserved":0,"allocated":3330048,"data_stored":1820252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543362,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":3321856,"data_stored":1820160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510659,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48316,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":3325952,"data_stored":1820250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510658,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48904,"kb_used_data":3832,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322936,"statfs":{"total":96636764160,"available":96586686464,"internally_reserved":0,"allocated":3923968,"data_stored":2411144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:37:23.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:37:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 -- 172.21.15.44:0/138434660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 msgr2=0x7f1e0c153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 --2- 172.21.15.44:0/138434660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c153a50 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0d0340 tx=0x7f1dfc02b860 comp rx=0 tx=0).stop 2024-07-17T08:37:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 -- 172.21.15.44:0/138434660 shutdown_connections 2024-07-17T08:37:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 --2- 172.21.15.44:0/138434660 >> v2:172.21.15.175:3300/0 conn(0x7f1e0c0ac870 0x7f1e0c13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 --2- 172.21.15.44:0/138434660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c153a50 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 -- 172.21.15.44:0/138434660 >> 172.21.15.44:0/138434660 conn(0x7f1e0c0b0fb0 msgr2=0x7f1e0c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 -- 172.21.15.44:0/138434660 shutdown_connections 2024-07-17T08:37:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.620+0000 7f1e11200640 1 -- 172.21.15.44:0/138434660 wait complete. 2024-07-17T08:37:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.622+0000 7f1e11200640 1 Processor -- start 2024-07-17T08:37:23.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.622+0000 7f1e11200640 1 -- start start 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e0c0ac3e0 0x7f1e0c142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e0c0ac870 0x7f1e0c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0ae800 con 0x7f1e0c06f710 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c06f0a0 con 0x7f1e0c0ac3e0 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e11200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0ae980 con 0x7f1e0c0ac870 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e09e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e09e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34740/0 (socket says 172.21.15.44:34740) 2024-07-17T08:37:23.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.623+0000 7f1e09e00640 1 -- 172.21.15.44:0/106065701 learned_addr learned my addr 172.21.15.44:0/106065701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:23.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09400640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e0c0ac3e0 0x7f1e0c142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 -- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e0c0ac870 msgr2=0x7f1e0c13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e0c0ac870 0x7f1e0c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 -- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e0c0ac3e0 msgr2=0x7f1e0c142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e0c0ac3e0 0x7f1e0c142a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 -- 172.21.15.44:0/106065701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c140560 con 0x7f1e0c06f710 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e09e00640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc02be50 tx=0x7f1dfc031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc005c90 con 0x7f1e0c06f710 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dfc005310 con 0x7f1e0c06f710 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc031e90 con 0x7f1e0c06f710 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.624+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c140840 con 0x7f1e0c06f710 2024-07-17T08:37:23.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.625+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c140270 con 0x7f1e0c06f710 2024-07-17T08:37:23.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.626+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0c153da0 con 0x7f1e0c06f710 2024-07-17T08:37:23.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.627+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f1dfc0047a0 con 0x7f1e0c06f710 2024-07-17T08:37:23.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.628+0000 7f1e02000640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 0x7f1dec079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:23.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.628+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7f1dfc0b9f00 con 0x7f1e0c06f710 2024-07-17T08:37:23.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.629+0000 7f1e09400640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 0x7f1dec079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:23.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.630+0000 7f1e09400640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 0x7f1dec079220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c1f50a0 tx=0x7f1df8009040 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:23.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.633+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1dfc082ce0 con 0x7f1e0c06f710 2024-07-17T08:37:23.761 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:23.758+0000 7fe22a800640 -1 client.8255: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.8162:d254fb08-d311-499b-82eb-b6f2e9b4345e:v1:172.21.15.175:0/2008194206] 2024-07-17T08:37:23.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.851+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 --> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e0c142740 con 0x7f1dec076d60 2024-07-17T08:37:23.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.866+0000 7f1e02000640 1 -- 172.21.15.44:0/106065701 <== mgr.9025 v2:172.21.15.175:6834/1251568930 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7f1df800d450 con 0x7f1dec076d60 2024-07-17T08:37:23.869 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:23.876 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.874+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 msgr2=0x7f1dec079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.874+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 0x7f1dec079220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c1f50a0 tx=0x7f1df8009040 comp rx=0 tx=0).stop 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.874+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 msgr2=0x7f1e0c1f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.874+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc02be50 tx=0x7f1dfc031ca0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 shutdown_connections 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1dec076d60 0x7f1dec079220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e0c0ac870 0x7f1e0c13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e0c0ac3e0 0x7f1e0c142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 --2- 172.21.15.44:0/106065701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e0c06f710 0x7f1e0c1f4b60 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:23.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 >> 172.21.15.44:0/106065701 conn(0x7f1e0c0b0fb0 msgr2=0x7f1e0c15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:23.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 shutdown_connections 2024-07-17T08:37:23.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:23.875+0000 7f1e11200640 1 -- 172.21.15.44:0/106065701 wait complete. 2024-07-17T08:37:23.899 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-07-17T08:37:22.468101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":429,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4221,"ondisk_log_size":4221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444616,"kb_used_data":27144,"kb_used_omap":12,"kb_used_meta":417331,"kb_avail":754530104,"statfs":{"total":773094113280,"available":772638826496,"internally_reserved":0,"allocated":27795456,"data_stored":15736829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":427347504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703603"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836549+0000","last_change":"2024-07-17T08:37:18.832909+0000","last_active":"2024-07-17T08:37:18.836549+0000","last_peered":"2024-07-17T08:37:18.836549+0000","last_clean":"2024-07-17T08:37:18.836549+0000","last_became_active":"2024-07-17T08:37:18.832477+0000","last_became_peered":"2024-07-17T08:37:18.832477+0000","last_unstale":"2024-07-17T08:37:18.836549+0000","last_undegraded":"2024-07-17T08:37:18.836549+0000","last_fullsized":"2024-07-17T08:37:18.836549+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:29.907945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.830107+0000","last_change":"2024-07-17T08:37:18.826592+0000","last_active":"2024-07-17T08:37:18.830107+0000","last_peered":"2024-07-17T08:37:18.830107+0000","last_clean":"2024-07-17T08:37:18.830107+0000","last_became_active":"2024-07-17T08:37:18.824915+0000","last_became_peered":"2024-07-17T08:37:18.824915+0000","last_unstale":"2024-07-17T08:37:18.830107+0000","last_undegraded":"2024-07-17T08:37:18.830107+0000","last_fullsized":"2024-07-17T08:37:18.830107+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:47.421470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.849329+0000","last_change":"2024-07-17T08:37:18.846136+0000","last_active":"2024-07-17T08:37:18.849329+0000","last_peered":"2024-07-17T08:37:18.849329+0000","last_clean":"2024-07-17T08:37:18.849329+0000","last_became_active":"2024-07-17T08:37:18.844265+0000","last_became_peered":"2024-07-17T08:37:18.844265+0000","last_unstale":"2024-07-17T08:37:18.849329+0000","last_undegraded":"2024-07-17T08:37:18.849329+0000","last_fullsized":"2024-07-17T08:37:18.849329+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:44.438486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.827500+0000","last_change":"2024-07-17T08:37:18.825722+0000","last_active":"2024-07-17T08:37:18.827500+0000","last_peered":"2024-07-17T08:37:18.827500+0000","last_clean":"2024-07-17T08:37:18.827500+0000","last_became_active":"2024-07-17T08:37:18.824775+0000","last_became_peered":"2024-07-17T08:37:18.824775+0000","last_unstale":"2024-07-17T08:37:18.827500+0000","last_undegraded":"2024-07-17T08:37:18.827500+0000","last_fullsized":"2024-07-17T08:37:18.827500+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:37.746840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843770+0000","last_change":"2024-07-17T08:37:18.833819+0000","last_active":"2024-07-17T08:37:18.843770+0000","last_peered":"2024-07-17T08:37:18.843770+0000","last_clean":"2024-07-17T08:37:18.843770+0000","last_became_active":"2024-07-17T08:37:18.833495+0000","last_became_peered":"2024-07-17T08:37:18.833495+0000","last_unstale":"2024-07-17T08:37:18.843770+0000","last_undegraded":"2024-07-17T08:37:18.843770+0000","last_fullsized":"2024-07-17T08:37:18.843770+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:14.502180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854348+0000","last_change":"2024-07-17T08:37:18.853392+0000","last_active":"2024-07-17T08:37:18.854348+0000","last_peered":"2024-07-17T08:37:18.854348+0000","last_clean":"2024-07-17T08:37:18.854348+0000","last_became_active":"2024-07-17T08:37:18.852460+0000","last_became_peered":"2024-07-17T08:37:18.852460+0000","last_unstale":"2024-07-17T08:37:18.854348+0000","last_undegraded":"2024-07-17T08:37:18.854348+0000","last_fullsized":"2024-07-17T08:37:18.854348+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:08.693120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.830485+0000","last_change":"2024-07-17T08:37:18.828720+0000","last_active":"2024-07-17T08:37:18.830485+0000","last_peered":"2024-07-17T08:37:18.830485+0000","last_clean":"2024-07-17T08:37:18.830485+0000","last_became_active":"2024-07-17T08:37:18.828420+0000","last_became_peered":"2024-07-17T08:37:18.828420+0000","last_unstale":"2024-07-17T08:37:18.830485+0000","last_undegraded":"2024-07-17T08:37:18.830485+0000","last_fullsized":"2024-07-17T08:37:18.830485+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:45.033633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826107+0000","last_change":"2024-07-17T08:37:18.819167+0000","last_active":"2024-07-17T08:37:18.826107+0000","last_peered":"2024-07-17T08:37:18.826107+0000","last_clean":"2024-07-17T08:37:18.826107+0000","last_became_active":"2024-07-17T08:37:18.818604+0000","last_became_peered":"2024-07-17T08:37:18.818604+0000","last_unstale":"2024-07-17T08:37:18.826107+0000","last_undegraded":"2024-07-17T08:37:18.826107+0000","last_fullsized":"2024-07-17T08:37:18.826107+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:51.715080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847494+0000","last_change":"2024-07-17T08:37:18.846141+0000","last_active":"2024-07-17T08:37:18.847494+0000","last_peered":"2024-07-17T08:37:18.847494+0000","last_clean":"2024-07-17T08:37:18.847494+0000","last_became_active":"2024-07-17T08:37:18.845829+0000","last_became_peered":"2024-07-17T08:37:18.845829+0000","last_unstale":"2024-07-17T08:37:18.847494+0000","last_undegraded":"2024-07-17T08:37:18.847494+0000","last_fullsized":"2024-07-17T08:37:18.847494+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:20.541825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.832444+0000","last_change":"2024-07-17T08:37:18.831385+0000","last_active":"2024-07-17T08:37:18.832444+0000","last_peered":"2024-07-17T08:37:18.832444+0000","last_clean":"2024-07-17T08:37:18.832444+0000","last_became_active":"2024-07-17T08:37:18.830787+0000","last_became_peered":"2024-07-17T08:37:18.830787+0000","last_unstale":"2024-07-17T08:37:18.832444+0000","last_undegraded":"2024-07-17T08:37:18.832444+0000","last_fullsized":"2024-07-17T08:37:18.832444+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:33:08.351350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833225+0000","last_change":"2024-07-17T08:37:18.825713+0000","last_active":"2024-07-17T08:37:18.833225+0000","last_peered":"2024-07-17T08:37:18.833225+0000","last_clean":"2024-07-17T08:37:18.833225+0000","last_became_active":"2024-07-17T08:37:18.824869+0000","last_became_peered":"2024-07-17T08:37:18.824869+0000","last_unstale":"2024-07-17T08:37:18.833225+0000","last_undegraded":"2024-07-17T08:37:18.833225+0000","last_fullsized":"2024-07-17T08:37:18.833225+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:46.733734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838179+0000","last_change":"2024-07-17T08:37:18.834553+0000","last_active":"2024-07-17T08:37:18.838179+0000","last_peered":"2024-07-17T08:37:18.838179+0000","last_clean":"2024-07-17T08:37:18.838179+0000","last_became_active":"2024-07-17T08:37:18.832219+0000","last_became_peered":"2024-07-17T08:37:18.832219+0000","last_unstale":"2024-07-17T08:37:18.838179+0000","last_undegraded":"2024-07-17T08:37:18.838179+0000","last_fullsized":"2024-07-17T08:37:18.838179+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:20:40.688248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.825910+0000","last_change":"2024-07-17T08:37:18.822043+0000","last_active":"2024-07-17T08:37:18.825910+0000","last_peered":"2024-07-17T08:37:18.825910+0000","last_clean":"2024-07-17T08:37:18.825910+0000","last_became_active":"2024-07-17T08:37:18.819751+0000","last_became_peered":"2024-07-17T08:37:18.819751+0000","last_unstale":"2024-07-17T08:37:18.825910+0000","last_undegraded":"2024-07-17T08:37:18.825910+0000","last_fullsized":"2024-07-17T08:37:18.825910+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:48.996045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.834952+0000","last_change":"2024-07-17T08:37:18.832196+0000","last_active":"2024-07-17T08:37:18.834952+0000","last_peered":"2024-07-17T08:37:18.834952+0000","last_clean":"2024-07-17T08:37:18.834952+0000","last_became_active":"2024-07-17T08:37:18.828929+0000","last_became_peered":"2024-07-17T08:37:18.828929+0000","last_unstale":"2024-07-17T08:37:18.834952+0000","last_undegraded":"2024-07-17T08:37:18.834952+0000","last_fullsized":"2024-07-17T08:37:18.834952+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:18:47.759057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826155+0000","last_change":"2024-07-17T08:37:18.822833+0000","last_active":"2024-07-17T08:37:18.826155+0000","last_peered":"2024-07-17T08:37:18.826155+0000","last_clean":"2024-07-17T08:37:18.826155+0000","last_became_active":"2024-07-17T08:37:18.822533+0000","last_became_peered":"2024-07-17T08:37:18.822533+0000","last_unstale":"2024-07-17T08:37:18.826155+0000","last_undegraded":"2024-07-17T08:37:18.826155+0000","last_fullsized":"2024-07-17T08:37:18.826155+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:31.149684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838437+0000","last_change":"2024-07-17T08:37:18.832611+0000","last_active":"2024-07-17T08:37:18.838437+0000","last_peered":"2024-07-17T08:37:18.838437+0000","last_clean":"2024-07-17T08:37:18.838437+0000","last_became_active":"2024-07-17T08:37:18.831834+0000","last_became_peered":"2024-07-17T08:37:18.831834+0000","last_unstale":"2024-07-17T08:37:18.838437+0000","last_undegraded":"2024-07-17T08:37:18.838437+0000","last_fullsized":"2024-07-17T08:37:18.838437+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:50:01.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839904+0000","last_change":"2024-07-17T08:37:18.836977+0000","last_active":"2024-07-17T08:37:18.839904+0000","last_peered":"2024-07-17T08:37:18.839904+0000","last_clean":"2024-07-17T08:37:18.839904+0000","last_became_active":"2024-07-17T08:37:18.833773+0000","last_became_peered":"2024-07-17T08:37:18.833773+0000","last_unstale":"2024-07-17T08:37:18.839904+0000","last_undegraded":"2024-07-17T08:37:18.839904+0000","last_fullsized":"2024-07-17T08:37:18.839904+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:59.188997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.851761+0000","last_change":"2024-07-17T08:37:18.849713+0000","last_active":"2024-07-17T08:37:18.851761+0000","last_peered":"2024-07-17T08:37:18.851761+0000","last_clean":"2024-07-17T08:37:18.851761+0000","last_became_active":"2024-07-17T08:37:18.849379+0000","last_became_peered":"2024-07-17T08:37:18.849379+0000","last_unstale":"2024-07-17T08:37:18.851761+0000","last_undegraded":"2024-07-17T08:37:18.851761+0000","last_fullsized":"2024-07-17T08:37:18.851761+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:50:01.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.824046+0000","last_change":"2024-07-17T08:37:18.821331+0000","last_active":"2024-07-17T08:37:18.824046+0000","last_peered":"2024-07-17T08:37:18.824046+0000","last_clean":"2024-07-17T08:37:18.824046+0000","last_became_active":"2024-07-17T08:37:18.820940+0000","last_became_peered":"2024-07-17T08:37:18.820940+0000","last_unstale":"2024-07-17T08:37:18.824046+0000","last_undegraded":"2024-07-17T08:37:18.824046+0000","last_fullsized":"2024-07-17T08:37:18.824046+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:44.438486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842558+0000","last_change":"2024-07-17T08:37:18.840783+0000","last_active":"2024-07-17T08:37:18.842558+0000","last_peered":"2024-07-17T08:37:18.842558+0000","last_clean":"2024-07-17T08:37:18.842558+0000","last_became_active":"2024-07-17T08:37:18.840447+0000","last_became_peered":"2024-07-17T08:37:18.840447+0000","last_unstale":"2024-07-17T08:37:18.842558+0000","last_undegraded":"2024-07-17T08:37:18.842558+0000","last_fullsized":"2024-07-17T08:37:18.842558+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:23.650671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841862+0000","last_change":"2024-07-17T08:37:18.837910+0000","last_active":"2024-07-17T08:37:18.841862+0000","last_peered":"2024-07-17T08:37:18.841862+0000","last_clean":"2024-07-17T08:37:18.841862+0000","last_became_active":"2024-07-17T08:37:18.837616+0000","last_became_peered":"2024-07-17T08:37:18.837616+0000","last_unstale":"2024-07-17T08:37:18.841862+0000","last_undegraded":"2024-07-17T08:37:18.841862+0000","last_fullsized":"2024-07-17T08:37:18.841862+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:38.765630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852719+0000","last_change":"2024-07-17T08:37:18.849730+0000","last_active":"2024-07-17T08:37:18.852719+0000","last_peered":"2024-07-17T08:37:18.852719+0000","last_clean":"2024-07-17T08:37:18.852719+0000","last_became_active":"2024-07-17T08:37:18.849422+0000","last_became_peered":"2024-07-17T08:37:18.849422+0000","last_unstale":"2024-07-17T08:37:18.852719+0000","last_undegraded":"2024-07-17T08:37:18.852719+0000","last_fullsized":"2024-07-17T08:37:18.852719+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:27.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842887+0000","last_change":"2024-07-17T08:37:18.840358+0000","last_active":"2024-07-17T08:37:18.842887+0000","last_peered":"2024-07-17T08:37:18.842887+0000","last_clean":"2024-07-17T08:37:18.842887+0000","last_became_active":"2024-07-17T08:37:18.838902+0000","last_became_peered":"2024-07-17T08:37:18.838902+0000","last_unstale":"2024-07-17T08:37:18.842887+0000","last_undegraded":"2024-07-17T08:37:18.842887+0000","last_fullsized":"2024-07-17T08:37:18.842887+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:42:10.351738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.845086+0000","last_change":"2024-07-17T08:37:18.839303+0000","last_active":"2024-07-17T08:37:18.845086+0000","last_peered":"2024-07-17T08:37:18.845086+0000","last_clean":"2024-07-17T08:37:18.845086+0000","last_became_active":"2024-07-17T08:37:18.839043+0000","last_became_peered":"2024-07-17T08:37:18.839043+0000","last_unstale":"2024-07-17T08:37:18.845086+0000","last_undegraded":"2024-07-17T08:37:18.845086+0000","last_fullsized":"2024-07-17T08:37:18.845086+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:26.105486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833165+0000","last_change":"2024-07-17T08:37:18.831044+0000","last_active":"2024-07-17T08:37:18.833165+0000","last_peered":"2024-07-17T08:37:18.833165+0000","last_clean":"2024-07-17T08:37:18.833165+0000","last_became_active":"2024-07-17T08:37:18.830635+0000","last_became_peered":"2024-07-17T08:37:18.830635+0000","last_unstale":"2024-07-17T08:37:18.833165+0000","last_undegraded":"2024-07-17T08:37:18.833165+0000","last_fullsized":"2024-07-17T08:37:18.833165+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:31.780410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843680+0000","last_change":"2024-07-17T08:37:18.836001+0000","last_active":"2024-07-17T08:37:18.843680+0000","last_peered":"2024-07-17T08:37:18.843680+0000","last_clean":"2024-07-17T08:37:18.843680+0000","last_became_active":"2024-07-17T08:37:18.834701+0000","last_became_peered":"2024-07-17T08:37:18.834701+0000","last_unstale":"2024-07-17T08:37:18.843680+0000","last_undegraded":"2024-07-17T08:37:18.843680+0000","last_fullsized":"2024-07-17T08:37:18.843680+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:29.115550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843761+0000","last_change":"2024-07-17T08:37:18.837436+0000","last_active":"2024-07-17T08:37:18.843761+0000","last_peered":"2024-07-17T08:37:18.843761+0000","last_clean":"2024-07-17T08:37:18.843761+0000","last_became_active":"2024-07-17T08:37:18.837166+0000","last_became_peered":"2024-07-17T08:37:18.837166+0000","last_unstale":"2024-07-17T08:37:18.843761+0000","last_undegraded":"2024-07-17T08:37:18.843761+0000","last_fullsized":"2024-07-17T08:37:18.843761+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:01.072353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854980+0000","last_change":"2024-07-17T08:37:18.854070+0000","last_active":"2024-07-17T08:37:18.854980+0000","last_peered":"2024-07-17T08:37:18.854980+0000","last_clean":"2024-07-17T08:37:18.854980+0000","last_became_active":"2024-07-17T08:37:18.853810+0000","last_became_peered":"2024-07-17T08:37:18.853810+0000","last_unstale":"2024-07-17T08:37:18.854980+0000","last_undegraded":"2024-07-17T08:37:18.854980+0000","last_fullsized":"2024-07-17T08:37:18.854980+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:11.419771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838258+0000","last_change":"2024-07-17T08:37:18.834385+0000","last_active":"2024-07-17T08:37:18.838258+0000","last_peered":"2024-07-17T08:37:18.838258+0000","last_clean":"2024-07-17T08:37:18.838258+0000","last_became_active":"2024-07-17T08:37:18.834100+0000","last_became_peered":"2024-07-17T08:37:18.834100+0000","last_unstale":"2024-07-17T08:37:18.838258+0000","last_undegraded":"2024-07-17T08:37:18.838258+0000","last_fullsized":"2024-07-17T08:37:18.838258+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:03:36.920718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854973+0000","last_change":"2024-07-17T08:37:18.853236+0000","last_active":"2024-07-17T08:37:18.854973+0000","last_peered":"2024-07-17T08:37:18.854973+0000","last_clean":"2024-07-17T08:37:18.854973+0000","last_became_active":"2024-07-17T08:37:18.852890+0000","last_became_peered":"2024-07-17T08:37:18.852890+0000","last_unstale":"2024-07-17T08:37:18.854973+0000","last_undegraded":"2024-07-17T08:37:18.854973+0000","last_fullsized":"2024-07-17T08:37:18.854973+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:40.623338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838527+0000","last_change":"2024-07-17T08:37:18.837433+0000","last_active":"2024-07-17T08:37:18.838527+0000","last_peered":"2024-07-17T08:37:18.838527+0000","last_clean":"2024-07-17T08:37:18.838527+0000","last_became_active":"2024-07-17T08:37:18.836365+0000","last_became_peered":"2024-07-17T08:37:18.836365+0000","last_unstale":"2024-07-17T08:37:18.838527+0000","last_undegraded":"2024-07-17T08:37:18.838527+0000","last_fullsized":"2024-07-17T08:37:18.838527+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:51.645895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838877+0000","last_change":"2024-07-17T08:37:18.831124+0000","last_active":"2024-07-17T08:37:18.838877+0000","last_peered":"2024-07-17T08:37:18.838877+0000","last_clean":"2024-07-17T08:37:18.838877+0000","last_became_active":"2024-07-17T08:37:18.830831+0000","last_became_peered":"2024-07-17T08:37:18.830831+0000","last_unstale":"2024-07-17T08:37:18.838877+0000","last_undegraded":"2024-07-17T08:37:18.838877+0000","last_fullsized":"2024-07-17T08:37:18.838877+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:02:01.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"262'4","reported_seq":16,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852657+0000","last_change":"2024-07-17T08:37:17.804164+0000","last_active":"2024-07-17T08:37:18.852657+0000","last_peered":"2024-07-17T08:37:18.852657+0000","last_clean":"2024-07-17T08:37:18.852657+0000","last_became_active":"2024-07-17T08:37:17.802462+0000","last_became_peered":"2024-07-17T08:37:17.802462+0000","last_unstale":"2024-07-17T08:37:18.852657+0000","last_undegraded":"2024-07-17T08:37:18.852657+0000","last_fullsized":"2024-07-17T08:37:18.852657+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:41.962072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.828900+0000","last_change":"2024-07-17T08:37:18.826601+0000","last_active":"2024-07-17T08:37:18.828900+0000","last_peered":"2024-07-17T08:37:18.828900+0000","last_clean":"2024-07-17T08:37:18.828900+0000","last_became_active":"2024-07-17T08:37:18.826019+0000","last_became_peered":"2024-07-17T08:37:18.826019+0000","last_unstale":"2024-07-17T08:37:18.828900+0000","last_undegraded":"2024-07-17T08:37:18.828900+0000","last_fullsized":"2024-07-17T08:37:18.828900+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:29:26.884680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847593+0000","last_change":"2024-07-17T08:37:17.808715+0000","last_active":"2024-07-17T08:37:18.847593+0000","last_peered":"2024-07-17T08:37:18.847593+0000","last_clean":"2024-07-17T08:37:18.847593+0000","last_became_active":"2024-07-17T08:37:17.808441+0000","last_became_peered":"2024-07-17T08:37:17.808441+0000","last_unstale":"2024-07-17T08:37:18.847593+0000","last_undegraded":"2024-07-17T08:37:18.847593+0000","last_fullsized":"2024-07-17T08:37:18.847593+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:45:03.033848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836581+0000","last_change":"2024-07-17T08:37:18.834297+0000","last_active":"2024-07-17T08:37:18.836581+0000","last_peered":"2024-07-17T08:37:18.836581+0000","last_clean":"2024-07-17T08:37:18.836581+0000","last_became_active":"2024-07-17T08:37:18.832521+0000","last_became_peered":"2024-07-17T08:37:18.832521+0000","last_unstale":"2024-07-17T08:37:18.836581+0000","last_undegraded":"2024-07-17T08:37:18.836581+0000","last_fullsized":"2024-07-17T08:37:18.836581+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:53.339674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843272+0000","last_change":"2024-07-17T08:37:17.812708+0000","last_active":"2024-07-17T08:37:18.843272+0000","last_peered":"2024-07-17T08:37:18.843272+0000","last_clean":"2024-07-17T08:37:18.843272+0000","last_became_active":"2024-07-17T08:37:17.812402+0000","last_became_peered":"2024-07-17T08:37:17.812402+0000","last_unstale":"2024-07-17T08:37:18.843272+0000","last_undegraded":"2024-07-17T08:37:18.843272+0000","last_fullsized":"2024-07-17T08:37:18.843272+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:41:44.377488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847534+0000","last_change":"2024-07-17T08:37:18.845881+0000","last_active":"2024-07-17T08:37:18.847534+0000","last_peered":"2024-07-17T08:37:18.847534+0000","last_clean":"2024-07-17T08:37:18.847534+0000","last_became_active":"2024-07-17T08:37:18.845558+0000","last_became_peered":"2024-07-17T08:37:18.845558+0000","last_unstale":"2024-07-17T08:37:18.847534+0000","last_undegraded":"2024-07-17T08:37:18.847534+0000","last_fullsized":"2024-07-17T08:37:18.847534+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:55:39.564695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.846458+0000","last_change":"2024-07-17T08:37:17.804147+0000","last_active":"2024-07-17T08:37:18.846458+0000","last_peered":"2024-07-17T08:37:18.846458+0000","last_clean":"2024-07-17T08:37:18.846458+0000","last_became_active":"2024-07-17T08:37:17.803827+0000","last_became_peered":"2024-07-17T08:37:17.803827+0000","last_unstale":"2024-07-17T08:37:18.846458+0000","last_undegraded":"2024-07-17T08:37:18.846458+0000","last_fullsized":"2024-07-17T08:37:18.846458+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:15:12.884859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850667+0000","last_change":"2024-07-17T08:37:18.848036+0000","last_active":"2024-07-17T08:37:18.850667+0000","last_peered":"2024-07-17T08:37:18.850667+0000","last_clean":"2024-07-17T08:37:18.850667+0000","last_became_active":"2024-07-17T08:37:18.847708+0000","last_became_peered":"2024-07-17T08:37:18.847708+0000","last_unstale":"2024-07-17T08:37:18.850667+0000","last_undegraded":"2024-07-17T08:37:18.850667+0000","last_fullsized":"2024-07-17T08:37:18.850667+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:45.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850458+0000","last_change":"2024-07-17T08:37:17.801026+0000","last_active":"2024-07-17T08:37:18.850458+0000","last_peered":"2024-07-17T08:37:18.850458+0000","last_clean":"2024-07-17T08:37:18.850458+0000","last_became_active":"2024-07-17T08:37:17.800731+0000","last_became_peered":"2024-07-17T08:37:17.800731+0000","last_unstale":"2024-07-17T08:37:18.850458+0000","last_undegraded":"2024-07-17T08:37:18.850458+0000","last_fullsized":"2024-07-17T08:37:18.850458+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:35.363277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841058+0000","last_change":"2024-07-17T08:37:18.835823+0000","last_active":"2024-07-17T08:37:18.841058+0000","last_peered":"2024-07-17T08:37:18.841058+0000","last_clean":"2024-07-17T08:37:18.841058+0000","last_became_active":"2024-07-17T08:37:18.832770+0000","last_became_peered":"2024-07-17T08:37:18.832770+0000","last_unstale":"2024-07-17T08:37:18.841058+0000","last_undegraded":"2024-07-17T08:37:18.841058+0000","last_fullsized":"2024-07-17T08:37:18.841058+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:27.289210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850490+0000","last_change":"2024-07-17T08:37:17.799706+0000","last_active":"2024-07-17T08:37:18.850490+0000","last_peered":"2024-07-17T08:37:18.850490+0000","last_clean":"2024-07-17T08:37:18.850490+0000","last_became_active":"2024-07-17T08:37:17.799414+0000","last_became_peered":"2024-07-17T08:37:17.799414+0000","last_unstale":"2024-07-17T08:37:18.850490+0000","last_undegraded":"2024-07-17T08:37:18.850490+0000","last_fullsized":"2024-07-17T08:37:18.850490+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:44:10.098693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854706+0000","last_change":"2024-07-17T08:37:18.852385+0000","last_active":"2024-07-17T08:37:18.854706+0000","last_peered":"2024-07-17T08:37:18.854706+0000","last_clean":"2024-07-17T08:37:18.854706+0000","last_became_active":"2024-07-17T08:37:18.852059+0000","last_became_peered":"2024-07-17T08:37:18.852059+0000","last_unstale":"2024-07-17T08:37:18.854706+0000","last_undegraded":"2024-07-17T08:37:18.854706+0000","last_fullsized":"2024-07-17T08:37:18.854706+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:02.423177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.816530+0000","last_change":"2024-07-17T08:37:17.806029+0000","last_active":"2024-07-17T08:37:18.816530+0000","last_peered":"2024-07-17T08:37:18.816530+0000","last_clean":"2024-07-17T08:37:18.816530+0000","last_became_active":"2024-07-17T08:37:17.805592+0000","last_became_peered":"2024-07-17T08:37:17.805592+0000","last_unstale":"2024-07-17T08:37:18.816530+0000","last_undegraded":"2024-07-17T08:37:18.816530+0000","last_fullsized":"2024-07-17T08:37:18.816530+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:06:17.318614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839882+0000","last_change":"2024-07-17T08:37:18.836745+0000","last_active":"2024-07-17T08:37:18.839882+0000","last_peered":"2024-07-17T08:37:18.839882+0000","last_clean":"2024-07-17T08:37:18.839882+0000","last_became_active":"2024-07-17T08:37:18.834613+0000","last_became_peered":"2024-07-17T08:37:18.834613+0000","last_unstale":"2024-07-17T08:37:18.839882+0000","last_undegraded":"2024-07-17T08:37:18.839882+0000","last_fullsized":"2024-07-17T08:37:18.839882+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:04.256365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.802981+0000","last_change":"2024-07-17T08:37:17.810344+0000","last_active":"2024-07-17T08:37:18.802981+0000","last_peered":"2024-07-17T08:37:18.802981+0000","last_clean":"2024-07-17T08:37:18.802981+0000","last_became_active":"2024-07-17T08:37:17.810021+0000","last_became_peered":"2024-07-17T08:37:17.810021+0000","last_unstale":"2024-07-17T08:37:18.802981+0000","last_undegraded":"2024-07-17T08:37:18.802981+0000","last_fullsized":"2024-07-17T08:37:18.802981+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:41.174971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826063+0000","last_change":"2024-07-17T08:37:18.818912+0000","last_active":"2024-07-17T08:37:18.826063+0000","last_peered":"2024-07-17T08:37:18.826063+0000","last_clean":"2024-07-17T08:37:18.826063+0000","last_became_active":"2024-07-17T08:37:18.818137+0000","last_became_peered":"2024-07-17T08:37:18.818137+0000","last_unstale":"2024-07-17T08:37:18.826063+0000","last_undegraded":"2024-07-17T08:37:18.826063+0000","last_fullsized":"2024-07-17T08:37:18.826063+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:58.127022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.815037+0000","last_change":"2024-07-17T08:37:17.812708+0000","last_active":"2024-07-17T08:37:18.815037+0000","last_peered":"2024-07-17T08:37:18.815037+0000","last_clean":"2024-07-17T08:37:18.815037+0000","last_became_active":"2024-07-17T08:37:17.812172+0000","last_became_peered":"2024-07-17T08:37:17.812172+0000","last_unstale":"2024-07-17T08:37:18.815037+0000","last_undegraded":"2024-07-17T08:37:18.815037+0000","last_fullsized":"2024-07-17T08:37:18.815037+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:10.299090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826296+0000","last_change":"2024-07-17T08:37:18.822752+0000","last_active":"2024-07-17T08:37:18.826296+0000","last_peered":"2024-07-17T08:37:18.826296+0000","last_clean":"2024-07-17T08:37:18.826296+0000","last_became_active":"2024-07-17T08:37:18.822334+0000","last_became_peered":"2024-07-17T08:37:18.822334+0000","last_unstale":"2024-07-17T08:37:18.826296+0000","last_undegraded":"2024-07-17T08:37:18.826296+0000","last_fullsized":"2024-07-17T08:37:18.826296+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:48.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.822647+0000","last_change":"2024-07-17T08:37:17.807022+0000","last_active":"2024-07-17T08:37:18.822647+0000","last_peered":"2024-07-17T08:37:18.822647+0000","last_clean":"2024-07-17T08:37:18.822647+0000","last_became_active":"2024-07-17T08:37:17.806415+0000","last_became_peered":"2024-07-17T08:37:17.806415+0000","last_unstale":"2024-07-17T08:37:18.822647+0000","last_undegraded":"2024-07-17T08:37:18.822647+0000","last_fullsized":"2024-07-17T08:37:18.822647+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:50:43.251800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826962+0000","last_change":"2024-07-17T08:37:18.825283+0000","last_active":"2024-07-17T08:37:18.826962+0000","last_peered":"2024-07-17T08:37:18.826962+0000","last_clean":"2024-07-17T08:37:18.826962+0000","last_became_active":"2024-07-17T08:37:18.824931+0000","last_became_peered":"2024-07-17T08:37:18.824931+0000","last_unstale":"2024-07-17T08:37:18.826962+0000","last_undegraded":"2024-07-17T08:37:18.826962+0000","last_fullsized":"2024-07-17T08:37:18.826962+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:04.616603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"262'3","reported_seq":15,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850978+0000","last_change":"2024-07-17T08:37:17.808985+0000","last_active":"2024-07-17T08:37:18.850978+0000","last_peered":"2024-07-17T08:37:18.850978+0000","last_clean":"2024-07-17T08:37:18.850978+0000","last_became_active":"2024-07-17T08:37:17.808641+0000","last_became_peered":"2024-07-17T08:37:17.808641+0000","last_unstale":"2024-07-17T08:37:18.850978+0000","last_undegraded":"2024-07-17T08:37:18.850978+0000","last_fullsized":"2024-07-17T08:37:18.850978+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:50.627655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843794+0000","last_change":"2024-07-17T08:37:18.837143+0000","last_active":"2024-07-17T08:37:18.843794+0000","last_peered":"2024-07-17T08:37:18.843794+0000","last_clean":"2024-07-17T08:37:18.843794+0000","last_became_active":"2024-07-17T08:37:18.836852+0000","last_became_peered":"2024-07-17T08:37:18.836852+0000","last_unstale":"2024-07-17T08:37:18.843794+0000","last_undegraded":"2024-07-17T08:37:18.843794+0000","last_fullsized":"2024-07-17T08:37:18.843794+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:09.651128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"262'6","reported_seq":18,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852977+0000","last_change":"2024-07-17T08:37:17.805610+0000","last_active":"2024-07-17T08:37:18.852977+0000","last_peered":"2024-07-17T08:37:18.852977+0000","last_clean":"2024-07-17T08:37:18.852977+0000","last_became_active":"2024-07-17T08:37:17.805108+0000","last_became_peered":"2024-07-17T08:37:17.805108+0000","last_unstale":"2024-07-17T08:37:18.852977+0000","last_undegraded":"2024-07-17T08:37:18.852977+0000","last_fullsized":"2024-07-17T08:37:18.852977+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:44.730802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.835358+0000","last_change":"2024-07-17T08:37:18.832037+0000","last_active":"2024-07-17T08:37:18.835358+0000","last_peered":"2024-07-17T08:37:18.835358+0000","last_clean":"2024-07-17T08:37:18.835358+0000","last_became_active":"2024-07-17T08:37:18.830707+0000","last_became_peered":"2024-07-17T08:37:18.830707+0000","last_unstale":"2024-07-17T08:37:18.835358+0000","last_undegraded":"2024-07-17T08:37:18.835358+0000","last_fullsized":"2024-07-17T08:37:18.835358+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:53:07.382219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.848983+0000","last_change":"2024-07-17T08:37:17.805988+0000","last_active":"2024-07-17T08:37:18.848983+0000","last_peered":"2024-07-17T08:37:18.848983+0000","last_clean":"2024-07-17T08:37:18.848983+0000","last_became_active":"2024-07-17T08:37:17.805550+0000","last_became_peered":"2024-07-17T08:37:17.805550+0000","last_unstale":"2024-07-17T08:37:18.848983+0000","last_undegraded":"2024-07-17T08:37:18.848983+0000","last_fullsized":"2024-07-17T08:37:18.848983+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:34:54.094039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.835243+0000","last_change":"2024-07-17T08:37:18.832302+0000","last_active":"2024-07-17T08:37:18.835243+0000","last_peered":"2024-07-17T08:37:18.835243+0000","last_clean":"2024-07-17T08:37:18.835243+0000","last_became_active":"2024-07-17T08:37:18.832024+0000","last_became_peered":"2024-07-17T08:37:18.832024+0000","last_unstale":"2024-07-17T08:37:18.835243+0000","last_undegraded":"2024-07-17T08:37:18.835243+0000","last_fullsized":"2024-07-17T08:37:18.835243+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:34.593534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.809762+0000","last_change":"2024-07-17T08:37:17.802128+0000","last_active":"2024-07-17T08:37:18.809762+0000","last_peered":"2024-07-17T08:37:18.809762+0000","last_clean":"2024-07-17T08:37:18.809762+0000","last_became_active":"2024-07-17T08:37:17.801392+0000","last_became_peered":"2024-07-17T08:37:17.801392+0000","last_unstale":"2024-07-17T08:37:18.809762+0000","last_undegraded":"2024-07-17T08:37:18.809762+0000","last_fullsized":"2024-07-17T08:37:18.809762+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:19:03.573199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839802+0000","last_change":"2024-07-17T08:37:18.833520+0000","last_active":"2024-07-17T08:37:18.839802+0000","last_peered":"2024-07-17T08:37:18.839802+0000","last_clean":"2024-07-17T08:37:18.839802+0000","last_became_active":"2024-07-17T08:37:18.833211+0000","last_became_peered":"2024-07-17T08:37:18.833211+0000","last_unstale":"2024-07-17T08:37:18.839802+0000","last_undegraded":"2024-07-17T08:37:18.839802+0000","last_fullsized":"2024-07-17T08:37:18.839802+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:46.218558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.813590+0000","last_change":"2024-07-17T08:37:17.798510+0000","last_active":"2024-07-17T08:37:18.813590+0000","last_peered":"2024-07-17T08:37:18.813590+0000","last_clean":"2024-07-17T08:37:18.813590+0000","last_became_active":"2024-07-17T08:37:17.798231+0000","last_became_peered":"2024-07-17T08:37:17.798231+0000","last_unstale":"2024-07-17T08:37:18.813590+0000","last_undegraded":"2024-07-17T08:37:18.813590+0000","last_fullsized":"2024-07-17T08:37:18.813590+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:16:10.410789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839193+0000","last_change":"2024-07-17T08:37:18.833412+0000","last_active":"2024-07-17T08:37:18.839193+0000","last_peered":"2024-07-17T08:37:18.839193+0000","last_clean":"2024-07-17T08:37:18.839193+0000","last_became_active":"2024-07-17T08:37:18.829375+0000","last_became_peered":"2024-07-17T08:37:18.829375+0000","last_unstale":"2024-07-17T08:37:18.839193+0000","last_undegraded":"2024-07-17T08:37:18.839193+0000","last_fullsized":"2024-07-17T08:37:18.839193+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:43:03.086747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"262'8","reported_seq":20,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847212+0000","last_change":"2024-07-17T08:37:17.809359+0000","last_active":"2024-07-17T08:37:18.847212+0000","last_peered":"2024-07-17T08:37:18.847212+0000","last_clean":"2024-07-17T08:37:18.847212+0000","last_became_active":"2024-07-17T08:37:17.809062+0000","last_became_peered":"2024-07-17T08:37:17.809062+0000","last_unstale":"2024-07-17T08:37:18.847212+0000","last_undegraded":"2024-07-17T08:37:18.847212+0000","last_fullsized":"2024-07-17T08:37:18.847212+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:57.278603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.842735+0000","last_change":"2024-07-17T08:37:18.832363+0000","last_active":"2024-07-17T08:37:18.842735+0000","last_peered":"2024-07-17T08:37:18.842735+0000","last_clean":"2024-07-17T08:37:18.842735+0000","last_became_active":"2024-07-17T08:37:18.832050+0000","last_became_peered":"2024-07-17T08:37:18.832050+0000","last_unstale":"2024-07-17T08:37:18.842735+0000","last_undegraded":"2024-07-17T08:37:18.842735+0000","last_fullsized":"2024-07-17T08:37:18.842735+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:56.386723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.848240+0000","last_change":"2024-07-17T08:37:17.807706+0000","last_active":"2024-07-17T08:37:18.848240+0000","last_peered":"2024-07-17T08:37:18.848240+0000","last_clean":"2024-07-17T08:37:18.848240+0000","last_became_active":"2024-07-17T08:37:17.807400+0000","last_became_peered":"2024-07-17T08:37:17.807400+0000","last_unstale":"2024-07-17T08:37:18.848240+0000","last_undegraded":"2024-07-17T08:37:18.848240+0000","last_fullsized":"2024-07-17T08:37:18.848240+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:32:47.580344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.845142+0000","last_change":"2024-07-17T08:37:18.838688+0000","last_active":"2024-07-17T08:37:18.845142+0000","last_peered":"2024-07-17T08:37:18.845142+0000","last_clean":"2024-07-17T08:37:18.845142+0000","last_became_active":"2024-07-17T08:37:18.838357+0000","last_became_peered":"2024-07-17T08:37:18.838357+0000","last_unstale":"2024-07-17T08:37:18.845142+0000","last_undegraded":"2024-07-17T08:37:18.845142+0000","last_fullsized":"2024-07-17T08:37:18.845142+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:28:20.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.851943+0000","last_change":"2024-07-17T08:37:17.796761+0000","last_active":"2024-07-17T08:37:18.851943+0000","last_peered":"2024-07-17T08:37:18.851943+0000","last_clean":"2024-07-17T08:37:18.851943+0000","last_became_active":"2024-07-17T08:37:17.796435+0000","last_became_peered":"2024-07-17T08:37:17.796435+0000","last_unstale":"2024-07-17T08:37:18.851943+0000","last_undegraded":"2024-07-17T08:37:18.851943+0000","last_fullsized":"2024-07-17T08:37:18.851943+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:52:39.796928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.850418+0000","last_change":"2024-07-17T08:37:18.839615+0000","last_active":"2024-07-17T08:37:18.850418+0000","last_peered":"2024-07-17T08:37:18.850418+0000","last_clean":"2024-07-17T08:37:18.850418+0000","last_became_active":"2024-07-17T08:37:18.839099+0000","last_became_peered":"2024-07-17T08:37:18.839099+0000","last_unstale":"2024-07-17T08:37:18.850418+0000","last_undegraded":"2024-07-17T08:37:18.850418+0000","last_fullsized":"2024-07-17T08:37:18.850418+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:13:19.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"262'1","reported_seq":13,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.846646+0000","last_change":"2024-07-17T08:37:17.790148+0000","last_active":"2024-07-17T08:37:18.846646+0000","last_peered":"2024-07-17T08:37:18.846646+0000","last_clean":"2024-07-17T08:37:18.846646+0000","last_became_active":"2024-07-17T08:37:17.789799+0000","last_became_peered":"2024-07-17T08:37:17.789799+0000","last_unstale":"2024-07-17T08:37:18.846646+0000","last_undegraded":"2024-07-17T08:37:18.846646+0000","last_fullsized":"2024-07-17T08:37:18.846646+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:37.370506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.834016+0000","last_change":"2024-07-17T08:37:18.830746+0000","last_active":"2024-07-17T08:37:18.834016+0000","last_peered":"2024-07-17T08:37:18.834016+0000","last_clean":"2024-07-17T08:37:18.834016+0000","last_became_active":"2024-07-17T08:37:18.830366+0000","last_became_peered":"2024-07-17T08:37:18.830366+0000","last_unstale":"2024-07-17T08:37:18.834016+0000","last_undegraded":"2024-07-17T08:37:18.834016+0000","last_fullsized":"2024-07-17T08:37:18.834016+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:16.848300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.817810+0000","last_change":"2024-07-17T08:37:17.813190+0000","last_active":"2024-07-17T08:37:18.817810+0000","last_peered":"2024-07-17T08:37:18.817810+0000","last_clean":"2024-07-17T08:37:18.817810+0000","last_became_active":"2024-07-17T08:37:17.812787+0000","last_became_peered":"2024-07-17T08:37:17.812787+0000","last_unstale":"2024-07-17T08:37:18.817810+0000","last_undegraded":"2024-07-17T08:37:18.817810+0000","last_fullsized":"2024-07-17T08:37:18.817810+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:56:00.425488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.841962+0000","last_change":"2024-07-17T08:37:18.837433+0000","last_active":"2024-07-17T08:37:18.841962+0000","last_peered":"2024-07-17T08:37:18.841962+0000","last_clean":"2024-07-17T08:37:18.841962+0000","last_became_active":"2024-07-17T08:37:18.837050+0000","last_became_peered":"2024-07-17T08:37:18.837050+0000","last_unstale":"2024-07-17T08:37:18.841962+0000","last_undegraded":"2024-07-17T08:37:18.841962+0000","last_fullsized":"2024-07-17T08:37:18.841962+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:13:10.662238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854682+0000","last_change":"2024-07-17T08:37:17.809099+0000","last_active":"2024-07-17T08:37:18.854682+0000","last_peered":"2024-07-17T08:37:18.854682+0000","last_clean":"2024-07-17T08:37:18.854682+0000","last_became_active":"2024-07-17T08:37:17.808776+0000","last_became_peered":"2024-07-17T08:37:17.808776+0000","last_unstale":"2024-07-17T08:37:18.854682+0000","last_undegraded":"2024-07-17T08:37:18.854682+0000","last_fullsized":"2024-07-17T08:37:18.854682+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:25.959428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.852672+0000","last_change":"2024-07-17T08:37:18.849632+0000","last_active":"2024-07-17T08:37:18.852672+0000","last_peered":"2024-07-17T08:37:18.852672+0000","last_clean":"2024-07-17T08:37:18.852672+0000","last_became_active":"2024-07-17T08:37:18.849307+0000","last_became_peered":"2024-07-17T08:37:18.849307+0000","last_unstale":"2024-07-17T08:37:18.852672+0000","last_undegraded":"2024-07-17T08:37:18.852672+0000","last_fullsized":"2024-07-17T08:37:18.852672+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:25:49.721269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.800088+0000","last_change":"2024-07-17T08:37:17.809016+0000","last_active":"2024-07-17T08:37:18.800088+0000","last_peered":"2024-07-17T08:37:18.800088+0000","last_clean":"2024-07-17T08:37:18.800088+0000","last_became_active":"2024-07-17T08:37:17.808718+0000","last_became_peered":"2024-07-17T08:37:17.808718+0000","last_unstale":"2024-07-17T08:37:18.800088+0000","last_undegraded":"2024-07-17T08:37:18.800088+0000","last_fullsized":"2024-07-17T08:37:18.800088+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:21.159778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.839773+0000","last_change":"2024-07-17T08:37:18.834909+0000","last_active":"2024-07-17T08:37:18.839773+0000","last_peered":"2024-07-17T08:37:18.839773+0000","last_clean":"2024-07-17T08:37:18.839773+0000","last_became_active":"2024-07-17T08:37:18.834630+0000","last_became_peered":"2024-07-17T08:37:18.834630+0000","last_unstale":"2024-07-17T08:37:18.839773+0000","last_undegraded":"2024-07-17T08:37:18.839773+0000","last_fullsized":"2024-07-17T08:37:18.839773+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:45.701907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.827002+0000","last_change":"2024-07-17T08:37:17.809179+0000","last_active":"2024-07-17T08:37:18.827002+0000","last_peered":"2024-07-17T08:37:18.827002+0000","last_clean":"2024-07-17T08:37:18.827002+0000","last_became_active":"2024-07-17T08:37:17.808874+0000","last_became_peered":"2024-07-17T08:37:17.808874+0000","last_unstale":"2024-07-17T08:37:18.827002+0000","last_undegraded":"2024-07-17T08:37:18.827002+0000","last_fullsized":"2024-07-17T08:37:18.827002+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:43:53.512695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.854616+0000","last_change":"2024-07-17T08:37:18.853410+0000","last_active":"2024-07-17T08:37:18.854616+0000","last_peered":"2024-07-17T08:37:18.854616+0000","last_clean":"2024-07-17T08:37:18.854616+0000","last_became_active":"2024-07-17T08:37:18.853127+0000","last_became_peered":"2024-07-17T08:37:18.853127+0000","last_unstale":"2024-07-17T08:37:18.854616+0000","last_undegraded":"2024-07-17T08:37:18.854616+0000","last_fullsized":"2024-07-17T08:37:18.854616+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:56.825447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.817914+0000","last_change":"2024-07-17T08:37:17.806994+0000","last_active":"2024-07-17T08:37:18.817914+0000","last_peered":"2024-07-17T08:37:18.817914+0000","last_clean":"2024-07-17T08:37:18.817914+0000","last_became_active":"2024-07-17T08:37:17.806665+0000","last_became_peered":"2024-07-17T08:37:17.806665+0000","last_unstale":"2024-07-17T08:37:18.817914+0000","last_undegraded":"2024-07-17T08:37:18.817914+0000","last_fullsized":"2024-07-17T08:37:18.817914+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:39.162444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826085+0000","last_change":"2024-07-17T08:37:18.818833+0000","last_active":"2024-07-17T08:37:18.826085+0000","last_peered":"2024-07-17T08:37:18.826085+0000","last_clean":"2024-07-17T08:37:18.826085+0000","last_became_active":"2024-07-17T08:37:18.817716+0000","last_became_peered":"2024-07-17T08:37:18.817716+0000","last_unstale":"2024-07-17T08:37:18.826085+0000","last_undegraded":"2024-07-17T08:37:18.826085+0000","last_fullsized":"2024-07-17T08:37:18.826085+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:54:16.713235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"262'2","reported_seq":14,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.847014+0000","last_change":"2024-07-17T08:37:17.815629+0000","last_active":"2024-07-17T08:37:18.847014+0000","last_peered":"2024-07-17T08:37:18.847014+0000","last_clean":"2024-07-17T08:37:18.847014+0000","last_became_active":"2024-07-17T08:37:17.815305+0000","last_became_peered":"2024-07-17T08:37:17.815305+0000","last_unstale":"2024-07-17T08:37:18.847014+0000","last_undegraded":"2024-07-17T08:37:18.847014+0000","last_fullsized":"2024-07-17T08:37:18.847014+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:39.979339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.833827+0000","last_change":"2024-07-17T08:37:18.830011+0000","last_active":"2024-07-17T08:37:18.833827+0000","last_peered":"2024-07-17T08:37:18.833827+0000","last_clean":"2024-07-17T08:37:18.833827+0000","last_became_active":"2024-07-17T08:37:18.829677+0000","last_became_peered":"2024-07-17T08:37:18.829677+0000","last_unstale":"2024-07-17T08:37:18.833827+0000","last_undegraded":"2024-07-17T08:37:18.833827+0000","last_fullsized":"2024-07-17T08:37:18.833827+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:45.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.813720+0000","last_change":"2024-07-17T08:37:17.803118+0000","last_active":"2024-07-17T08:37:18.813720+0000","last_peered":"2024-07-17T08:37:18.813720+0000","last_clean":"2024-07-17T08:37:18.813720+0000","last_became_active":"2024-07-17T08:37:17.802439+0000","last_became_peered":"2024-07-17T08:37:17.802439+0000","last_unstale":"2024-07-17T08:37:18.813720+0000","last_undegraded":"2024-07-17T08:37:18.813720+0000","last_fullsized":"2024-07-17T08:37:18.813720+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:22:03.539221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.826676+0000","last_change":"2024-07-17T08:37:18.818293+0000","last_active":"2024-07-17T08:37:18.826676+0000","last_peered":"2024-07-17T08:37:18.826676+0000","last_clean":"2024-07-17T08:37:18.826676+0000","last_became_active":"2024-07-17T08:37:18.817958+0000","last_became_peered":"2024-07-17T08:37:18.817958+0000","last_unstale":"2024-07-17T08:37:18.826676+0000","last_undegraded":"2024-07-17T08:37:18.826676+0000","last_fullsized":"2024-07-17T08:37:18.826676+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:10.445048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.820343+0000","last_change":"2024-07-17T08:37:17.810382+0000","last_active":"2024-07-17T08:37:18.820343+0000","last_peered":"2024-07-17T08:37:18.820343+0000","last_clean":"2024-07-17T08:37:18.820343+0000","last_became_active":"2024-07-17T08:37:17.810080+0000","last_became_peered":"2024-07-17T08:37:17.810080+0000","last_unstale":"2024-07-17T08:37:18.820343+0000","last_undegraded":"2024-07-17T08:37:18.820343+0000","last_fullsized":"2024-07-17T08:37:18.820343+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:51:05.786598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.843592+0000","last_change":"2024-07-17T08:37:18.840577+0000","last_active":"2024-07-17T08:37:18.843592+0000","last_peered":"2024-07-17T08:37:18.843592+0000","last_clean":"2024-07-17T08:37:18.843592+0000","last_became_active":"2024-07-17T08:37:18.840298+0000","last_became_peered":"2024-07-17T08:37:18.840298+0000","last_unstale":"2024-07-17T08:37:18.843592+0000","last_undegraded":"2024-07-17T08:37:18.843592+0000","last_fullsized":"2024-07-17T08:37:18.843592+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:29:47.038922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.810339+0000","last_change":"2024-07-17T08:37:17.809261+0000","last_active":"2024-07-17T08:37:18.810339+0000","last_peered":"2024-07-17T08:37:18.810339+0000","last_clean":"2024-07-17T08:37:18.810339+0000","last_became_active":"2024-07-17T08:37:17.808937+0000","last_became_peered":"2024-07-17T08:37:17.808937+0000","last_unstale":"2024-07-17T08:37:18.810339+0000","last_undegraded":"2024-07-17T08:37:18.810339+0000","last_fullsized":"2024-07-17T08:37:18.810339+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:11.890369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.836518+0000","last_change":"2024-07-17T08:37:18.834100+0000","last_active":"2024-07-17T08:37:18.836518+0000","last_peered":"2024-07-17T08:37:18.836518+0000","last_clean":"2024-07-17T08:37:18.836518+0000","last_became_active":"2024-07-17T08:37:18.833809+0000","last_became_peered":"2024-07-17T08:37:18.833809+0000","last_unstale":"2024-07-17T08:37:18.836518+0000","last_undegraded":"2024-07-17T08:37:18.836518+0000","last_fullsized":"2024-07-17T08:37:18.836518+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:07.502846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.823129+0000","last_change":"2024-07-17T08:37:17.805773+0000","last_active":"2024-07-17T08:37:18.823129+0000","last_peered":"2024-07-17T08:37:18.823129+0000","last_clean":"2024-07-17T08:37:18.823129+0000","last_became_active":"2024-07-17T08:37:17.804458+0000","last_became_peered":"2024-07-17T08:37:17.804458+0000","last_unstale":"2024-07-17T08:37:18.823129+0000","last_undegraded":"2024-07-17T08:37:18.823129+0000","last_fullsized":"2024-07-17T08:37:18.823129+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:23.755011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.828911+0000","last_change":"2024-07-17T08:37:18.827315+0000","last_active":"2024-07-17T08:37:18.828911+0000","last_peered":"2024-07-17T08:37:18.828911+0000","last_clean":"2024-07-17T08:37:18.828911+0000","last_became_active":"2024-07-17T08:37:18.827038+0000","last_became_peered":"2024-07-17T08:37:18.827038+0000","last_unstale":"2024-07-17T08:37:18.828911+0000","last_undegraded":"2024-07-17T08:37:18.828911+0000","last_fullsized":"2024-07-17T08:37:18.828911+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:38:33.199619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.800591+0000","last_change":"2024-07-17T08:37:17.808953+0000","last_active":"2024-07-17T08:37:18.800591+0000","last_peered":"2024-07-17T08:37:18.800591+0000","last_clean":"2024-07-17T08:37:18.800591+0000","last_became_active":"2024-07-17T08:37:17.808653+0000","last_became_peered":"2024-07-17T08:37:17.808653+0000","last_unstale":"2024-07-17T08:37:18.800591+0000","last_undegraded":"2024-07-17T08:37:18.800591+0000","last_fullsized":"2024-07-17T08:37:18.800591+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:17:41.093679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838756+0000","last_change":"2024-07-17T08:37:18.828266+0000","last_active":"2024-07-17T08:37:18.838756+0000","last_peered":"2024-07-17T08:37:18.838756+0000","last_clean":"2024-07-17T08:37:18.838756+0000","last_became_active":"2024-07-17T08:37:18.827934+0000","last_became_peered":"2024-07-17T08:37:18.827934+0000","last_unstale":"2024-07-17T08:37:18.838756+0000","last_undegraded":"2024-07-17T08:37:18.838756+0000","last_fullsized":"2024-07-17T08:37:18.838756+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:13:10.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.814072+0000","last_change":"2024-07-17T08:37:17.813364+0000","last_active":"2024-07-17T08:37:18.814072+0000","last_peered":"2024-07-17T08:37:18.814072+0000","last_clean":"2024-07-17T08:37:18.814072+0000","last_became_active":"2024-07-17T08:37:17.812231+0000","last_became_peered":"2024-07-17T08:37:17.812231+0000","last_unstale":"2024-07-17T08:37:18.814072+0000","last_undegraded":"2024-07-17T08:37:18.814072+0000","last_fullsized":"2024-07-17T08:37:18.814072+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:57:33.575293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.849172+0000","last_change":"2024-07-17T08:37:18.844740+0000","last_active":"2024-07-17T08:37:18.849172+0000","last_peered":"2024-07-17T08:37:18.849172+0000","last_clean":"2024-07-17T08:37:18.849172+0000","last_became_active":"2024-07-17T08:37:18.844381+0000","last_became_peered":"2024-07-17T08:37:18.844381+0000","last_unstale":"2024-07-17T08:37:18.849172+0000","last_undegraded":"2024-07-17T08:37:18.849172+0000","last_fullsized":"2024-07-17T08:37:18.849172+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:11.317330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.818134+0000","last_change":"2024-07-17T08:37:17.812584+0000","last_active":"2024-07-17T08:37:18.818134+0000","last_peered":"2024-07-17T08:37:18.818134+0000","last_clean":"2024-07-17T08:37:18.818134+0000","last_became_active":"2024-07-17T08:37:17.812307+0000","last_became_peered":"2024-07-17T08:37:17.812307+0000","last_unstale":"2024-07-17T08:37:18.818134+0000","last_undegraded":"2024-07-17T08:37:18.818134+0000","last_fullsized":"2024-07-17T08:37:18.818134+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":260,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:16.742238+0000","last_clean_scrub_stamp":"2024-07-17T08:37:16.742238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:12.466725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":9,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:18.838543+0000","last_change":"2024-07-17T08:37:18.830703+0000","last_active":"2024-07-17T08:37:18.838543+0000","last_peered":"2024-07-17T08:37:18.838543+0000","last_clean":"2024-07-17T08:37:18.838543+0000","last_became_active":"2024-07-17T08:37:18.829187+0000","last_became_peered":"2024-07-17T08:37:18.829187+0000","last_unstale":"2024-07-17T08:37:18.838543+0000","last_undegraded":"2024-07-17T08:37:18.838543+0000","last_fullsized":"2024-07-17T08:37:18.838543+0000","mapping_epoch":261,"log_start":"0'0","ondisk_log_start":"0'0","created":261,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:17.760478+0000","last_clean_scrub_stamp":"2024-07-17T08:37:17.760478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:06:33.891781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"262'4183","reported_seq":5188,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-07-17T08:37:19.463908+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:37:19.463908+0000","last_peered":"2024-07-17T08:37:19.463908+0000","last_clean":"2024-07-17T08:37:19.463908+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:37:19.463908+0000","last_undegraded":"2024-07-17T08:37:19.463908+0000","last_fullsized":"2024-07-17T08:37:19.463908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":385,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1550,"num_read_kb":4325,"num_write":385,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4183,"ondisk_log_size":4183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543363,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71088,"kb_used_data":3812,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300752,"statfs":{"total":96636764160,"available":96563970048,"internally_reserved":0,"allocated":3903488,"data_stored":2402785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510658,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48320,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323520,"statfs":{"total":96636764160,"available":96587284480,"internally_reserved":0,"allocated":3321856,"data_stored":1820160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70532,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301308,"statfs":{"total":96636764160,"available":96564539392,"internally_reserved":0,"allocated":3330048,"data_stored":1820216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543364,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52492,"kb_used_data":3260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319348,"statfs":{"total":96636764160,"available":96583012352,"internally_reserved":0,"allocated":3338240,"data_stored":1821862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52484,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319356,"statfs":{"total":96636764160,"available":96583020544,"internally_reserved":0,"allocated":3330048,"data_stored":1820252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543362,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52480,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319360,"statfs":{"total":96636764160,"available":96583024640,"internally_reserved":0,"allocated":3321856,"data_stored":1820160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510659,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48316,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":3325952,"data_stored":1820250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510658,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48904,"kb_used_data":3832,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322936,"statfs":{"total":96636764160,"available":96586686464,"internally_reserved":0,"allocated":3923968,"data_stored":2411144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:37:23.905 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:37:23.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:37:24.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 -- 172.21.15.44:0/1285899220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06f4d0 msgr2=0x7fb41c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 --2- 172.21.15.44:0/1285899220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06f4d0 0x7fb41c15a450 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0cb730 tx=0x7fb40c02aa10 comp rx=0 tx=0).stop 2024-07-17T08:37:24.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 -- 172.21.15.44:0/1285899220 shutdown_connections 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 --2- 172.21.15.44:0/1285899220 >> v2:172.21.15.175:3300/0 conn(0x7fb41c1503c0 0x7fb41c0ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 --2- 172.21.15.44:0/1285899220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06f4d0 0x7fb41c15a450 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 -- 172.21.15.44:0/1285899220 >> 172.21.15.44:0/1285899220 conn(0x7fb41c0cf0d0 msgr2=0x7fb41c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 -- 172.21.15.44:0/1285899220 shutdown_connections 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.170+0000 7fb420a00640 1 -- 172.21.15.44:0/1285899220 wait complete. 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.172+0000 7fb420a00640 1 Processor -- start 2024-07-17T08:37:24.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.172+0000 7fb420a00640 1 -- start start 2024-07-17T08:37:24.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb41c06f4d0 0x7fb41c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 0x7fb41c09c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c06d850 con 0x7fb41c06d420 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0ac930 con 0x7fb41c06f4d0 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb420a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c06d9d0 con 0x7fb41c1503c0 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb41a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb41a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34756/0 (socket says 172.21.15.44:34756) 2024-07-17T08:37:24.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb41a800640 1 -- 172.21.15.44:0/3994249600 learned_addr learned my addr 172.21.15.44:0/3994249600 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:24.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb419e00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb41c06f4d0 0x7fb41c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.173+0000 7fb41b200640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 0x7fb41c09c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 -- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 msgr2=0x7fb41c09c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 0x7fb41c09c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 -- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb41c06f4d0 msgr2=0x7fb41c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb41c06f4d0 0x7fb41c0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 -- 172.21.15.44:0/3994249600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c1f1c20 con 0x7fb41c06d420 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41b200640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 0x7fb41c09c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb41a800640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009bf0 tx=0x7fb40c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:24.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c047070 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb40c004250 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c033050 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.174+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c1f21a0 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.175+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1f2a60 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.176+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c15a450 con 0x7fb41c06d420 2024-07-17T08:37:24.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.177+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fb40c04a070 con 0x7fb41c06d420 2024-07-17T08:37:24.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.178+0000 7fb412a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 0x7fb3f8079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.179+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fb40c0b9e30 con 0x7fb41c06d420 2024-07-17T08:37:24.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.179+0000 7fb419e00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 0x7fb3f8079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.180+0000 7fb419e00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 0x7fb3f8079220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb404005fd0 tx=0x7fb404005d80 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:24.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.183+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb40c082b90 con 0x7fb41c06d420 2024-07-17T08:37:24.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.477+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb41c0cffe0 con 0x7fb41c06d420 2024-07-17T08:37:24.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.481+0000 7fb412a00640 1 -- 172.21.15.44:0/3994249600 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v74) ==== 190+0+147 (secure 0 0 0) 0x7fb40c082490 con 0x7fb41c06d420 2024-07-17T08:37:24.482 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:37:24.482 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:37:24.483 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:37:24.483 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:37:24.483 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:37:24.483 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:37:24.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.484+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 msgr2=0x7fb3f8079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.484+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 0x7fb3f8079220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb404005fd0 tx=0x7fb404005d80 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.484+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 msgr2=0x7fb41c1f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.484+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009bf0 tx=0x7fb40c003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 shutdown_connections 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fb3f8076d60 0x7fb3f8079220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb41c1503c0 0x7fb41c09c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb41c06f4d0 0x7fb41c0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 --2- 172.21.15.44:0/3994249600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb41c06d420 0x7fb41c1f16e0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 >> 172.21.15.44:0/3994249600 conn(0x7fb41c0cf0d0 msgr2=0x7fb41c15aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 shutdown_connections 2024-07-17T08:37:24.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.485+0000 7fb420a00640 1 -- 172.21.15.44:0/3994249600 wait complete. 2024-07-17T08:37:24.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.705+0000 7fdf78a00640 1 -- 172.21.15.44:0/2145629128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 msgr2=0x7fdf7406d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.705+0000 7fdf78a00640 1 --2- 172.21.15.44:0/2145629128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf7406d9b0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009830 tx=0x7fdf5c02a580 comp rx=0 tx=0).stop 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 -- 172.21.15.44:0/2145629128 shutdown_connections 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 --2- 172.21.15.44:0/2145629128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf7406d9b0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 --2- 172.21.15.44:0/2145629128 >> v2:172.21.15.175:3300/0 conn(0x7fdf740b8c80 0x7fdf740b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 -- 172.21.15.44:0/2145629128 >> 172.21.15.44:0/2145629128 conn(0x7fdf740cfec0 msgr2=0x7fdf740d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 -- 172.21.15.44:0/2145629128 shutdown_connections 2024-07-17T08:37:24.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.706+0000 7fdf78a00640 1 -- 172.21.15.44:0/2145629128 wait complete. 2024-07-17T08:37:24.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.707+0000 7fdf78a00640 1 Processor -- start 2024-07-17T08:37:24.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 -- start start 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf740b5400 0x7fdf741f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf740b8c80 0x7fdf740cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7406e780 con 0x7fdf740ce5c0 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7406e600 con 0x7fdf740b8c80 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.708+0000 7fdf78a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7406e900 con 0x7fdf740b5400 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34780/0 (socket says 172.21.15.44:34780) 2024-07-17T08:37:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 -- 172.21.15.44:0/658573585 learned_addr learned my addr 172.21.15.44:0/658573585 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 -- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf740b5400 msgr2=0x7fdf741f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf740b5400 0x7fdf741f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 -- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf740b8c80 msgr2=0x7fdf740cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf740b8c80 0x7fdf740cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.709+0000 7fdf73e00640 1 -- 172.21.15.44:0/658573585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7415ad90 con 0x7fdf740ce5c0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf73e00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00cbc0 tx=0x7fdf6c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c016020 con 0x7fdf740ce5c0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf6c00ae30 con 0x7fdf740ce5c0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c012660 con 0x7fdf740ce5c0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf740c9770 con 0x7fdf740ce5c0 2024-07-17T08:37:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.710+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf740ca5b0 con 0x7fdf740ce5c0 2024-07-17T08:37:24.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.711+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7415e570 con 0x7fdf740ce5c0 2024-07-17T08:37:24.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.713+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7fdf6c019020 con 0x7fdf740ce5c0 2024-07-17T08:37:24.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.714+0000 7fdf6be00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 0x7fdf50079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:24.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.714+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6677+0+0 (secure 0 0 0) 0x7fdf6c099640 con 0x7fdf740ce5c0 2024-07-17T08:37:24.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.715+0000 7fdf73400640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 0x7fdf50079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:24.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.716+0000 7fdf73400640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 0x7fdf50079220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400a990 tx=0x7fdf64005c70 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:24.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.730+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdf6c020030 con 0x7fdf740ce5c0 2024-07-17T08:37:24.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.991+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf740cd850 con 0x7fdf740ce5c0 2024-07-17T08:37:24.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.992+0000 7fdf6be00640 1 -- 172.21.15.44:0/658573585 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7fdf6c0624a0 con 0x7fdf740ce5c0 2024-07-17T08:37:24.993 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:24.993 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:24.993 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:24.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 msgr2=0x7fdf50079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 0x7fdf50079220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400a990 tx=0x7fdf64005c70 comp rx=0 tx=0).stop 2024-07-17T08:37:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 msgr2=0x7fdf740c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00cbc0 tx=0x7fdf6c00a430 comp rx=0 tx=0).stop 2024-07-17T08:37:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 shutdown_connections 2024-07-17T08:37:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7fdf50076d60 0x7fdf50079220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf740ce5c0 0x7fdf740c6f50 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf740b8c80 0x7fdf740cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 --2- 172.21.15.44:0/658573585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf740b5400 0x7fdf741f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 >> 172.21.15.44:0/658573585 conn(0x7fdf740cfec0 msgr2=0x7fdf740b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 shutdown_connections 2024-07-17T08:37:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:24.995+0000 7fdf78a00640 1 -- 172.21.15.44:0/658573585 wait complete. 2024-07-17T08:37:25.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:37:18.288317+0000', 'modified': '2024-07-17T08:37:21.376632+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11325': {'gid': 11325, 'name': 'c', 'rank': 0, 'incarnation': 142, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/1990487283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 1990487283}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 1990487283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [36], 'metadata_pool': 35, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11325, 'qdb_cluster': [11325]} 2024-07-17T08:37:25.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:37:25.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 -- 172.21.15.44:0/3772685983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b8151f20 msgr2=0x7f52b815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 --2- 172.21.15.44:0/3772685983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b8151f20 0x7f52b815c140 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f52b80ccc20 tx=0x7f52a802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 -- 172.21.15.44:0/3772685983 shutdown_connections 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 --2- 172.21.15.44:0/3772685983 >> v2:172.21.15.175:3300/0 conn(0x7f52b806f380 0x7f52b80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 --2- 172.21.15.44:0/3772685983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b8151f20 0x7f52b815c140 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 -- 172.21.15.44:0/3772685983 >> 172.21.15.44:0/3772685983 conn(0x7f52b80cf0d0 msgr2=0x7f52b80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 -- 172.21.15.44:0/3772685983 shutdown_connections 2024-07-17T08:37:25.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.643+0000 7f52b7200640 1 -- 172.21.15.44:0/3772685983 wait complete. 2024-07-17T08:37:25.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.645+0000 7f52b7200640 1 Processor -- start 2024-07-17T08:37:25.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.645+0000 7f52b7200640 1 -- start start 2024-07-17T08:37:25.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b806f380 0x7f52b80989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 0x7f52b81e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80cb740 con 0x7f52b80cd0b0 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80cb5c0 con 0x7f52b8151f20 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80cb8c0 con 0x7f52b806f380 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34800/0 (socket says 172.21.15.44:34800) 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.646+0000 7f52b5400640 1 -- 172.21.15.44:0/3084137408 learned_addr learned my addr 172.21.15.44:0/3084137408 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b6800640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 0x7f52b81e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 -- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b806f380 msgr2=0x7f52b80989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5e00640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b806f380 0x7f52b80989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b806f380 0x7f52b80989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 -- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 msgr2=0x7f52b81e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 0x7f52b81e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 -- 172.21.15.44:0/3084137408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81409c0 con 0x7f52b80cd0b0 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b6800640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 0x7f52b81e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b5400640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c730 tx=0x7f52a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52ae000640 1 -- 172.21.15.44:0/3084137408 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4016020 con 0x7f52b80cd0b0 2024-07-17T08:37:25.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52ae000640 1 -- 172.21.15.44:0/3084137408 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a400a930 con 0x7f52b80cd0b0 2024-07-17T08:37:25.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52ae000640 1 -- 172.21.15.44:0/3084137408 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400ad80 con 0x7f52b80cd0b0 2024-07-17T08:37:25.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.647+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81f7040 con 0x7f52b80cd0b0 2024-07-17T08:37:25.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.648+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81f7600 con 0x7f52b80cd0b0 2024-07-17T08:37:25.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.650+0000 7f52ae000640 1 -- 172.21.15.44:0/3084137408 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7f52a4019020 con 0x7f52b80cd0b0 2024-07-17T08:37:25.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.651+0000 7f52ae000640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 0x7f5298079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.651+0000 7f52ae000640 1 -- 172.21.15.44:0/3084137408 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f52a4098b90 con 0x7f52b80cd0b0 2024-07-17T08:37:25.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.651+0000 7f52bf7e2740 1 Processor -- start 2024-07-17T08:37:25.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.652+0000 7f52bf7e2740 1 -- start start 2024-07-17T08:37:25.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.652+0000 7f52b5e00640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 0x7f5298079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6a8a280 0x557fa6aaf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6aafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557fa69fad60 con 0x557fa6ab2fc0 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557fa69ea260 con 0x557fa6aaf790 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x557fa69ee2a0 con 0x557fa6a8a280 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34810/0 (socket says 172.21.15.44:34810) 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 -- 172.21.15.44:0/1871888307 learned_addr learned my addr 172.21.15.44:0/1871888307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 0x7f5298079220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52a8034000 tx=0x7f52a8004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b6800640 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6aafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5400640 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6a8a280 0x557fa6aaf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 -- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6a8a280 msgr2=0x557fa6aaf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6a8a280 0x557fa6aaf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 -- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 msgr2=0x557fa6aafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6aafc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.653+0000 7f52b5e00640 1 -- 172.21.15.44:0/1871888307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa674e850 con 0x557fa6ab2fc0 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.654+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f52a802aa90 tx=0x7f52a803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60185d133ead4b4e server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.654+0000 7f52a2600640 1 -- 172.21.15.44:0/1871888307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8040070 con 0x557fa6ab2fc0 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.654+0000 7f52a2600640 1 -- 172.21.15.44:0/1871888307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a804c070 con 0x557fa6ab2fc0 2024-07-17T08:37:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.654+0000 7f52a2600640 1 -- 172.21.15.44:0/1871888307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8003da0 con 0x557fa6ab2fc0 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1871888307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 msgr2=0x557fa6ab33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f52a802aa90 tx=0x7f52a803b000 comp rx=0 tx=0).stop 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1871888307 shutdown_connections 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6aafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6a8a280 0x557fa6aaf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1871888307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6ab2fc0 0x557fa6ab33a0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1871888307 >> 172.21.15.44:0/1871888307 conn(0x557fa6a85460 msgr2=0x557fa6a88e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1871888307 shutdown_connections 2024-07-17T08:37:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.655+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1871888307 wait complete. 2024-07-17T08:37:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.656+0000 7f52bf7e2740 1 -- start start 2024-07-17T08:37:25.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.656+0000 7f52bf7e2740 1 Processor -- start 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6ae0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 0x557fa6a9c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557fa675d620 con 0x557fa6aa7540 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557fa6991f90 con 0x557fa6aaf790 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x557fa6a83b60 con 0x557fa6ab2fc0 2024-07-17T08:37:25.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52bf7e2740 10 client.0 _ll_register_callbacks cb 0x557fa69b7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52b6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34820/0 (socket says 172.21.15.44:34820) 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.657+0000 7f52b6800640 1 -- 172.21.15.44:0/2125515837 learned_addr learned my addr 172.21.15.44:0/2125515837 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b5400640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 0x557fa6a9c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b5e00640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6ae0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 -- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 msgr2=0x557fa6a9c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 0x557fa6a9c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 -- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 msgr2=0x557fa6ae0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6ae0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 -- 172.21.15.44:0/2125515837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa67c56b0 con 0x557fa6aa7540 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b5400640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 0x557fa6a9c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b5e00640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6ae0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52b6800640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f52b0010720 tx=0x7f52b0010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ea84472185aff server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f5296a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b001e060 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f52b000eb30 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.658+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fa69c1900 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f52b0024050 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 10 client.11457 fetch_fsmap learned FSMap version 146 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) -- 0x557fa6ac84c0 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557fa69c1900 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 146) ==== 3502+0+0 (secure 0 0 0) 0x7f52b0027020 con 0x557fa6aa7540 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 10 client.11457 fetch_fsmap finished waiting for FSMap version 146 2024-07-17T08:37:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 10 client.11457 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 20 client.11457 populate_metadata read hostname 'smithi044' 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6aab950 0x557fa6ac9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 4 client.11457 mds_command: new command op to 11325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.659+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 --> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa69fccb0 con 0x557fa6aab950 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.660+0000 7f52b5e00640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6aab950 0x557fa6ac9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.660+0000 7f52b5e00640 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6aab950 0x557fa6ac9390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62c039b2e956c19 server_cookie=2f5d2946dd0df768 in_seq=0 out_seq=0 2024-07-17T08:37:25.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.660+0000 7f5296a00640 10 client.11457 ms_handle_connect on v2:172.21.15.44:6834/1990487283 2024-07-17T08:37:25.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.661+0000 7f5296a00640 1 -- 172.21.15.44:0/2125515837 <== mds.0 v2:172.21.15.44:6834/1990487283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557fa69fccb0 con 0x557fa6aab950 2024-07-17T08:37:25.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.661+0000 7f5296a00640 10 client.11457 handle_command_reply: tid=0 2024-07-17T08:37:25.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.661+0000 7f52bf7e2740 1 client.11457 shutdown 2024-07-17T08:37:25.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.661+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 msgr2=0x557fa6a83f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.661+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f52b0010720 tx=0x7f52b0010be0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 shutdown_connections 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6aab950 0x557fa6ac9390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aa7540 0x557fa6a83f60 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6ab2fc0 0x557fa6a9c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2125515837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6aaf790 0x557fa6ae0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 >> 172.21.15.44:0/2125515837 conn(0x557fa6a8a280 msgr2=0x557fa6ab33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f5296a00640 0 client.11457 ms_handle_reset on v2:172.21.15.44:6834/1990487283 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 shutdown_connections 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2125515837 wait complete. 2024-07-17T08:37:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.662+0000 7f52bf7e2740 20 client.11457 trim_cache size 0 max 16384 2024-07-17T08:37:25.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.683+0000 7f52bf7e2740 1 Processor -- start 2024-07-17T08:37:25.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.683+0000 7f52bf7e2740 1 -- start start 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6ac7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6ac96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8200490 con 0x557fa6aaf790 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81f4590 con 0x557fa6ab2fc0 2024-07-17T08:37:25.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.684+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52b81ffe00 con 0x557fa6aa7540 2024-07-17T08:37:25.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34830/0 (socket says 172.21.15.44:34830) 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 -- 172.21.15.44:0/2820502444 learned_addr learned my addr 172.21.15.44:0/2820502444 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5e00640 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6ac7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b6800640 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6ac96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 -- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 msgr2=0x557fa6ac7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6ac7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 -- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 msgr2=0x557fa6ac96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6ac96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 -- 172.21.15.44:0/2820502444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa6a85290 con 0x557fa6aaf790 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.685+0000 7f52b5400640 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f52a4061200 tx=0x7f52a4060e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca3255e6348d9e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52a1c00640 1 -- 172.21.15.44:0/2820502444 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a407a070 con 0x557fa6aaf790 2024-07-17T08:37:25.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52a1c00640 1 -- 172.21.15.44:0/2820502444 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a4096db0 con 0x557fa6aaf790 2024-07-17T08:37:25.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52a1c00640 1 -- 172.21.15.44:0/2820502444 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a408fdb0 con 0x557fa6aaf790 2024-07-17T08:37:25.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2820502444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 msgr2=0x557fa6ac9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f52a4061200 tx=0x7f52a4060e40 comp rx=0 tx=0).stop 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2820502444 shutdown_connections 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6ac96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6ac9170 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/2820502444 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6ac7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.686+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2820502444 >> 172.21.15.44:0/2820502444 conn(0x557fa6a8a280 msgr2=0x557fa6a9bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.687+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2820502444 shutdown_connections 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.687+0000 7f52bf7e2740 1 -- 172.21.15.44:0/2820502444 wait complete. 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.687+0000 7f52bf7e2740 1 -- start start 2024-07-17T08:37:25.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.687+0000 7f52bf7e2740 1 Processor -- start 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6b46200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6a9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557fa6aab7d0 con 0x557fa6aaf790 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557fa6a9c780 con 0x557fa6ab2fc0 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x557fa6aabe20 con 0x557fa6aa7540 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.688+0000 7f52bf7e2740 10 client.0 _ll_register_callbacks cb 0x7f52b00095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34846/0 (socket says 172.21.15.44:34846) 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 -- 172.21.15.44:0/1896732290 learned_addr learned my addr 172.21.15.44:0/1896732290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6b46200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 -- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 msgr2=0x557fa6b46200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b6800640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6a9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6b46200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 -- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 msgr2=0x557fa6a9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6a9ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 -- 172.21.15.44:0/1896732290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa6ac8130 con 0x557fa6aaf790 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f52b5400640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f52a405d2a0 tx=0x7f52a405d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ea6cb4ff66c4c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.689+0000 7f5296a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4094060 con 0x557fa6aaf790 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f52a40606e0 con 0x557fa6aaf790 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4096db0 con 0x557fa6aaf790 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f52a407f050 con 0x557fa6aaf790 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 10 client.11463 fetch_fsmap learned FSMap version 146 2024-07-17T08:37:25.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) -- 0x557fa6a7e870 con 0x557fa6aaf790 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a4096db0 con 0x557fa6aaf790 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 146) ==== 3502+0+0 (secure 0 0 0) 0x7f52a408fb70 con 0x557fa6aaf790 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 10 client.11463 fetch_fsmap finished waiting for FSMap version 146 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 10 client.11463 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 20 client.11463 populate_metadata read hostname 'smithi044' 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6b4fc40 0x557fa6b50020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 4 client.11463 mds_command: new command op to 11325 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.690+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 --> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] -- command(tid 0: {"prefix": "status"}) -- 0x557fa69f8750 con 0x557fa6b4fc40 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.691+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6b4fc40 0x557fa6b50020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.691+0000 7f52b5e00640 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6b4fc40 0x557fa6b50020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd881859ee0893f server_cookie=3f7a99ec16354847 in_seq=0 out_seq=0 2024-07-17T08:37:25.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.691+0000 7f5296a00640 10 client.11463 ms_handle_connect on v2:172.21.15.44:6834/1990487283 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f5296a00640 1 -- 172.21.15.44:0/1896732290 <== mds.0 v2:172.21.15.44:6834/1990487283 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x557fa69f8750 con 0x557fa6b4fc40 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f5296a00640 10 client.11463 handle_command_reply: tid=0 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 client.11463 shutdown 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 msgr2=0x557fa6a9a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f52a405d2a0 tx=0x7f52a405d8c0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 shutdown_connections 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:6834/1990487283,v1:172.21.15.44:6835/1990487283] conn(0x557fa6b4fc40 0x557fa6b50020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557fa6ab2fc0 0x557fa6a9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fa6aaf790 0x557fa6a9a720 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 --2- 172.21.15.44:0/1896732290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x557fa6aa7540 0x557fa6b46200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f5296a00640 0 client.11463 ms_handle_reset on v2:172.21.15.44:6834/1990487283 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 >> 172.21.15.44:0/1896732290 conn(0x557fa6a8a280 msgr2=0x557fa6a83a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 shutdown_connections 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 1 -- 172.21.15.44:0/1896732290 wait complete. 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.692+0000 7f52bf7e2740 20 client.11463 trim_cache size 0 max 16384 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 11325, 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:37:25.697 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8732855449999999, 2024-07-17T08:37:25.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 146, 2024-07-17T08:37:25.698 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 264, 2024-07-17T08:37:25.698 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 263, 2024-07-17T08:37:25.698 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.794066781 2024-07-17T08:37:25.698 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 msgr2=0x7f5298079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 0x7f5298079220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52a8034000 tx=0x7f52a8004730 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 msgr2=0x7f52b81e7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c730 tx=0x7f52a400cbf0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 shutdown_connections 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5298076d60 0x7f5298079220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52b8151f20 0x7f52b81e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80cd0b0 0x7f52b81e7d60 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 --2- 172.21.15.44:0/3084137408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52b806f380 0x7f52b80989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 >> 172.21.15.44:0/3084137408 conn(0x7f52b80cf0d0 msgr2=0x7f52b80cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 shutdown_connections 2024-07-17T08:37:25.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.693+0000 7f52b7200640 1 -- 172.21.15.44:0/3084137408 wait complete. 2024-07-17T08:37:25.712 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:37:25.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 -- 172.21.15.44:0/5783405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 msgr2=0x7ff2200ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 --2- 172.21.15.44:0/5783405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff2200ade20 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7ff22006f770 tx=0x7ff21802aa30 comp rx=0 tx=0).stop 2024-07-17T08:37:25.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 -- 172.21.15.44:0/5783405 shutdown_connections 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 --2- 172.21.15.44:0/5783405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff2200ade20 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 --2- 172.21.15.44:0/5783405 >> v2:172.21.15.175:3300/0 conn(0x7ff2200b3db0 0x7ff2200ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 --2- 172.21.15.44:0/5783405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200cf540 0x7ff2200b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 -- 172.21.15.44:0/5783405 >> 172.21.15.44:0/5783405 conn(0x7ff2200b0e80 msgr2=0x7ff2200b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 -- 172.21.15.44:0/5783405 shutdown_connections 2024-07-17T08:37:25.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.917+0000 7ff226000640 1 -- 172.21.15.44:0/5783405 wait complete. 2024-07-17T08:37:25.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.919+0000 7ff226000640 1 Processor -- start 2024-07-17T08:37:25.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.919+0000 7ff226000640 1 -- start start 2024-07-17T08:37:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 0x7ff220072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 0x7ff22014b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200ae6f0 con 0x7ff2200cf920 2024-07-17T08:37:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200ae570 con 0x7ff2200cf540 2024-07-17T08:37:25.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.920+0000 7ff226000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2200ae870 con 0x7ff2200b3db0 2024-07-17T08:37:25.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34872/0 (socket says 172.21.15.44:34872) 2024-07-17T08:37:25.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 -- 172.21.15.44:0/1809907904 learned_addr learned my addr 172.21.15.44:0/1809907904 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff224c00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 0x7ff220072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff217e00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 0x7ff22014b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 -- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 msgr2=0x7ff220072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 0x7ff220072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 -- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 msgr2=0x7ff22014b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 0x7ff22014b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:25.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 -- 172.21.15.44:0/1809907904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22014bfb0 con 0x7ff2200cf920 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff224c00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 0x7ff220072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff217e00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 0x7ff22014b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.921+0000 7ff225600640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7ff218009d60 tx=0x7ff218003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.922+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218033050 con 0x7ff2200cf920 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.922+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff218004550 con 0x7ff2200cf920 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.922+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218038450 con 0x7ff2200cf920 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.922+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff220073450 con 0x7ff2200cf920 2024-07-17T08:37:25.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.922+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff220073a10 con 0x7ff2200cf920 2024-07-17T08:37:25.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.923+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2200b3870 con 0x7ff2200cf920 2024-07-17T08:37:25.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.924+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 148) ==== 98977+0+0 (secure 0 0 0) 0x7ff218036070 con 0x7ff2200cf920 2024-07-17T08:37:25.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.926+0000 7ff21d600640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 0x7ff1fc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:25.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.926+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7ff2180b9fa0 con 0x7ff2200cf920 2024-07-17T08:37:25.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.926+0000 7ff224c00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 0x7ff1fc079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:25.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.928+0000 7ff224c00640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 0x7ff1fc079220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2100058c0 tx=0x7ff210005d30 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:25.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:25.931+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff218082d00 con 0x7ff2200cf920 2024-07-17T08:37:26.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.181+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff220159060 con 0x7ff2200cf920 2024-07-17T08:37:26.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.182+0000 7ff21d600640 1 -- 172.21.15.44:0/1809907904 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v146) ==== 53+0+73 (secure 0 0 0) 0x7ff218082600 con 0x7ff2200cf920 2024-07-17T08:37:26.182 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:37:26.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 msgr2=0x7ff1fc079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 0x7ff1fc079220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2100058c0 tx=0x7ff210005d30 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 msgr2=0x7ff22014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7ff218009d60 tx=0x7ff218003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 shutdown_connections 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ff1fc076d60 0x7ff1fc079220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2200cf920 0x7ff22014b790 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2200cf540 0x7ff22014b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 --2- 172.21.15.44:0/1809907904 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2200b3db0 0x7ff220072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.184+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 >> 172.21.15.44:0/1809907904 conn(0x7ff2200b0e80 msgr2=0x7ff22006e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.185+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 shutdown_connections 2024-07-17T08:37:26.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26.185+0000 7ff226000640 1 -- 172.21.15.44:0/1809907904 wait complete. 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:37:26.204 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:37:26.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.205 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:37:26.255 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:37:26.255 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:37:26.255 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:37:26.256 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36411sec preferred_lft 36411sec 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.257 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:37:26.258 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:37:26.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.259 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:37:26.340 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:37:26.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.345 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:37:26.396 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:37:26.396 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:37:26.396 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:37:26.396 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:37:26.396 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:37:26.396 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:37:26.397 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:37:26.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:37:26.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:37:26.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:37:26.397 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:37:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:37:26.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:37:26.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.608 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:37:26.654 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:37:26.655 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:37:26.655 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:37:26.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:37:26.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:37:26.740 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:37:26.795 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:37:26.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:37:26.795 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:37:26.850 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:37:26.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:26.851 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:37:26.981 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:37:26.981 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:37:26.981 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:37:27.002 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:37:27.003 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:37:27.003 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:37:27.003 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:37:27.003 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:37:27.029 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:37:27.029 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:37:27.029 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:37:27.030 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:37:27.030 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:37:27.030 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:37:27.030 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:37:27.030 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:37:27.031 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:37:27.032 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:37:27.032 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:37:27.032 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:37:27.032 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:37:27.037 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:37:27.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:27.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:37:27.111 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:37:27.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:37:27.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:37:27.251 INFO:teuthology.orchestra.run.smithi044.stdout:client9256 2024-07-17T08:37:27.251 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:27.251 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:27.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.547+0000 7f91bf800640 1 -- 172.21.15.44:0/261514647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b806f6d0 msgr2=0x7f91b815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.547+0000 7f91bf800640 1 --2- 172.21.15.44:0/261514647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b806f6d0 0x7f91b815a5a0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009830 tx=0x7f91ac02a510 comp rx=0 tx=0).stop 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 -- 172.21.15.44:0/261514647 shutdown_connections 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 --2- 172.21.15.44:0/261514647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806eb90 0x7f91b80ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 --2- 172.21.15.44:0/261514647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b806f6d0 0x7f91b815a5a0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 -- 172.21.15.44:0/261514647 >> 172.21.15.44:0/261514647 conn(0x7f91b80b0eb0 msgr2=0x7f91b80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 -- 172.21.15.44:0/261514647 shutdown_connections 2024-07-17T08:37:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.548+0000 7f91bf800640 1 -- 172.21.15.44:0/261514647 wait complete. 2024-07-17T08:37:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.550+0000 7f91bf800640 1 Processor -- start 2024-07-17T08:37:27.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.550+0000 7f91bf800640 1 -- start start 2024-07-17T08:37:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b806eb90 0x7f91b81f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 0x7f91b8156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91b80acc70 con 0x7f91b80b3280 2024-07-17T08:37:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91b80acaf0 con 0x7f91b806eb90 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bf800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f91b814fd60 con 0x7f91b806f6d0 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bd800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34884/0 (socket says 172.21.15.44:34884) 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bd800640 1 -- 172.21.15.44:0/2801837984 learned_addr learned my addr 172.21.15.44:0/2801837984 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91bce00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b806eb90 0x7f91b81f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.551+0000 7f91b7e00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 0x7f91b8156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 -- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 msgr2=0x7f91b8156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 0x7f91b8156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 -- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b806eb90 msgr2=0x7f91b81f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b806eb90 0x7f91b81f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 -- 172.21.15.44:0/2801837984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b8152420 con 0x7f91b80b3280 2024-07-17T08:37:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91b7e00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 0x7f91b8156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bd800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f91b000eaa0 tx=0x7f91b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b000cdb0 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91b0004c10 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b81f4720 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0005020 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.552+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81f4ce0 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.554+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b81f5340 con 0x7f91b80b3280 2024-07-17T08:37:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.555+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 149) ==== 99033+0+0 (secure 0 0 0) 0x7f91b00051c0 con 0x7f91b80b3280 2024-07-17T08:37:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.557+0000 7f91b4c00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 0x7f91980792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.557+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f91b0099550 con 0x7f91b80b3280 2024-07-17T08:37:27.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.557+0000 7f91bce00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 0x7f91980792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:27.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.558+0000 7f91bce00640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 0x7f91980792e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f91ac034000 tx=0x7f91ac004730 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.561+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f91b0016030 con 0x7f91b80b3280 2024-07-17T08:37:27.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.797+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 --> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f91b81f5130 con 0x7f9198076e20 2024-07-17T08:37:27.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.802+0000 7f91b4c00640 1 -- 172.21.15.44:0/2801837984 <== mgr.9025 v2:172.21.15.175:6834/1251568930 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f91ac0043a0 con 0x7f9198076e20 2024-07-17T08:37:27.803 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:37:27.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:27.804 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:37:27.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:27.804 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.804+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 msgr2=0x7f91980792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 0x7f91980792e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f91ac034000 tx=0x7f91ac004730 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 msgr2=0x7f91b80b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f91b000eaa0 tx=0x7f91b000ef60 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 shutdown_connections 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f9198076e20 0x7f91980792e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91b80b3280 0x7f91b80b1290 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b806f6d0 0x7f91b8156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 --2- 172.21.15.44:0/2801837984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b806eb90 0x7f91b81f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 >> 172.21.15.44:0/2801837984 conn(0x7f91b80b0eb0 msgr2=0x7f91b80cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 shutdown_connections 2024-07-17T08:37:27.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:27.805+0000 7f91bf800640 1 -- 172.21.15.44:0/2801837984 wait complete. 2024-07-17T08:37:27.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.042+0000 7ffb77200640 1 -- 172.21.15.44:0/2013824147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 msgr2=0x7ffb7806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.042+0000 7ffb77200640 1 --2- 172.21.15.44:0/2013824147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb7806d610 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7ffb70005fd0 tx=0x7ffb7002b8e0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 -- 172.21.15.44:0/2013824147 shutdown_connections 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 --2- 172.21.15.44:0/2013824147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb7806d610 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 --2- 172.21.15.44:0/2013824147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7806f5a0 0x7ffb780b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 -- 172.21.15.44:0/2013824147 >> 172.21.15.44:0/2013824147 conn(0x7ffb7806c120 msgr2=0x7ffb7806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:28.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 -- 172.21.15.44:0/2013824147 shutdown_connections 2024-07-17T08:37:28.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.043+0000 7ffb77200640 1 -- 172.21.15.44:0/2013824147 wait complete. 2024-07-17T08:37:28.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.045+0000 7ffb77200640 1 Processor -- start 2024-07-17T08:37:28.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.045+0000 7ffb77200640 1 -- start start 2024-07-17T08:37:28.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7806f5a0 0x7ffb78158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:28.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:28.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 0x7ffb781f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:28.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780ae810 con 0x7ffb780cf6f0 2024-07-17T08:37:28.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb780ae690 con 0x7ffb7806f5a0 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.046+0000 7ffb77200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffb780ae990 con 0x7ffb78156d40 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34904/0 (socket says 172.21.15.44:34904) 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 -- 172.21.15.44:0/2370807194 learned_addr learned my addr 172.21.15.44:0/2370807194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb75e00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7806f5a0 0x7ffb78158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb76800640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 0x7ffb781f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 -- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 msgr2=0x7ffb781f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 0x7ffb781f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 -- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7806f5a0 msgr2=0x7ffb78158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7806f5a0 0x7ffb78158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb6fe00640 1 -- 172.21.15.44:0/2370807194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb78157ee0 con 0x7ffb780cf6f0 2024-07-17T08:37:28.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.047+0000 7ffb76800640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 0x7ffb781f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:37:28.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb6fe00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c430 tx=0x7ffb6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:28.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60017070 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb600035f0 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003a40 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb78158e90 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.048+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb78158150 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.049+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb781505b0 con 0x7ffb780cf6f0 2024-07-17T08:37:28.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.051+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 149) ==== 99033+0+0 (secure 0 0 0) 0x7ffb6001a070 con 0x7ffb780cf6f0 2024-07-17T08:37:28.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.052+0000 7ffb6e000640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 0x7ffb540792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:28.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.052+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7ffb60097e40 con 0x7ffb780cf6f0 2024-07-17T08:37:28.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.053+0000 7ffb75e00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 0x7ffb540792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.054+0000 7ffb75e00640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 0x7ffb540792b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb68005fd0 tx=0x7ffb68005850 comp rx=0 tx=0).ready entity=mgr.9025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:28.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.056+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffb781505b0 con 0x7ffb780cf6f0 2024-07-17T08:37:28.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.320+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7ffb7806d720 con 0x7ffb780cf6f0 2024-07-17T08:37:28.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.906+0000 7ffb6e000640 1 -- 172.21.15.44:0/2370807194 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v150) ==== 92+0+0 (secure 0 0 0) 0x7ffb60060600 con 0x7ffb780cf6f0 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 msgr2=0x7ffb540792b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 0x7ffb540792b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb68005fd0 tx=0x7ffb68005850 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 msgr2=0x7ffb781f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c430 tx=0x7ffb6000c8f0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 shutdown_connections 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7ffb54076df0 0x7ffb540792b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb78156d40 0x7ffb781f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb780cf6f0 0x7ffb781f0de0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 --2- 172.21.15.44:0/2370807194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7806f5a0 0x7ffb78158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:28.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.913+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 >> 172.21.15.44:0/2370807194 conn(0x7ffb7806c120 msgr2=0x7ffb78155290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:28.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.914+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 shutdown_connections 2024-07-17T08:37:28.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:28.914+0000 7ffb77200640 1 -- 172.21.15.44:0/2370807194 wait complete. 2024-07-17T08:37:28.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:37:29.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.172+0000 7f5e32200640 1 -- 172.21.15.44:0/865663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c0cbab0 msgr2=0x7f5e2c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.172+0000 7f5e32200640 1 --2- 172.21.15.44:0/865663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c06d620 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005fd0 tx=0x7f5e1402a540 comp rx=0 tx=0).stop 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.173+0000 7f5e32200640 1 -- 172.21.15.44:0/865663288 shutdown_connections 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.173+0000 7f5e32200640 1 --2- 172.21.15.44:0/865663288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c06d620 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.173+0000 7f5e32200640 1 -- 172.21.15.44:0/865663288 >> 172.21.15.44:0/865663288 conn(0x7f5e2c0b0eb0 msgr2=0x7f5e2c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.173+0000 7f5e32200640 1 -- 172.21.15.44:0/865663288 shutdown_connections 2024-07-17T08:37:29.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.173+0000 7f5e32200640 1 -- 172.21.15.44:0/865663288 wait complete. 2024-07-17T08:37:29.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.174+0000 7f5e32200640 1 Processor -- start 2024-07-17T08:37:29.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.175+0000 7f5e32200640 1 -- start start 2024-07-17T08:37:29.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e2c152220 0x7f5e2c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c0cd9f0 con 0x7f5e2c06ed30 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c15c800 con 0x7f5e2c0cbab0 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e32200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c0cdb70 con 0x7f5e2c152220 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51884/0 (socket says 172.21.15.44:51884) 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 -- 172.21.15.44:0/2952437571 learned_addr learned my addr 172.21.15.44:0/2952437571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:29.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e23e00640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:29.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2be00640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e2c152220 0x7f5e2c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:29.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e2c152220 msgr2=0x7f5e2c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e2c152220 0x7f5e2c14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2c0cbab0 msgr2=0x7f5e2c0ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c0ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.176+0000 7f5e2b400640 1 -- 172.21.15.44:0/2952437571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1f4690 con 0x7f5e2c06ed30 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e2b400640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c3a0 tx=0x7f5e1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c017070 con 0x7f5e2c06ed30 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e1c0053c0 con 0x7f5e2c06ed30 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c003850 con 0x7f5e2c06ed30 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1f4970 con 0x7f5e2c06ed30 2024-07-17T08:37:29.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.177+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1f4f30 con 0x7f5e2c06ed30 2024-07-17T08:37:29.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.178+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c1f5170 con 0x7f5e2c06ed30 2024-07-17T08:37:29.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.180+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 150) ==== 99050+0+0 (secure 0 0 0) 0x7f5e1c01a070 con 0x7f5e2c06ed30 2024-07-17T08:37:29.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.181+0000 7f5e23400640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 0x7f5e08079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.181+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f5e1c097eb0 con 0x7f5e2c06ed30 2024-07-17T08:37:29.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.181+0000 7f5e23e00640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 msgr2=0x7f5e08079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:29.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.181+0000 7f5e23e00640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 0x7f5e08079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:29.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.189+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5e2c1f5170 con 0x7f5e2c06ed30 2024-07-17T08:37:29.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.383+0000 7f5e23e00640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 msgr2=0x7f5e08079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:29.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.383+0000 7f5e23e00640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 0x7f5e08079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:29.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.403+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f5e2c0ce090 con 0x7f5e2c06ed30 2024-07-17T08:37:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.409+0000 7f5e23400640 1 -- 172.21.15.44:0/2952437571 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v81)=0 v81) ==== 138+0+0 (secure 0 0 0) 0x7f5e1c060630 con 0x7f5e2c06ed30 2024-07-17T08:37:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 msgr2=0x7f5e08079300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 0x7f5e08079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 msgr2=0x7f5e2c1476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c3a0 tx=0x7f5e1c00c860 comp rx=0 tx=0).stop 2024-07-17T08:37:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 shutdown_connections 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f5e08076e40 0x7f5e08079300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e2c152220 0x7f5e2c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e2c0cbab0 0x7f5e2c0ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 --2- 172.21.15.44:0/2952437571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e2c06ed30 0x7f5e2c1476c0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 >> 172.21.15.44:0/2952437571 conn(0x7f5e2c0b0eb0 msgr2=0x7f5e2c1581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.412+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 shutdown_connections 2024-07-17T08:37:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.413+0000 7f5e32200640 1 -- 172.21.15.44:0/2952437571 wait complete. 2024-07-17T08:37:29.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.632+0000 7f8f74e00640 1 -- 172.21.15.44:0/3025049174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 msgr2=0x7f8f540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.632+0000 7f8f74e00640 1 --2- 172.21.15.44:0/3025049174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f540051c0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005fd0 tx=0x7f8f54005830 comp rx=0 tx=0).stop 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.632+0000 7f8f6fe00640 1 -- 172.21.15.44:0/3025049174 reap_dead start 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.633+0000 7f8f74e00640 1 -- 172.21.15.44:0/3025049174 shutdown_connections 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.633+0000 7f8f74e00640 1 -- 172.21.15.44:0/3025049174 >> 172.21.15.44:0/3025049174 conn(0x7f8f700b4ff0 msgr2=0x7f8f700b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.633+0000 7f8f74e00640 1 -- 172.21.15.44:0/3025049174 shutdown_connections 2024-07-17T08:37:29.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.633+0000 7f8f74e00640 1 -- 172.21.15.44:0/3025049174 wait complete. 2024-07-17T08:37:29.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.636+0000 7f8f74e00640 1 Processor -- start 2024-07-17T08:37:29.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.636+0000 7f8f74e00640 1 -- start start 2024-07-17T08:37:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f700ccca0 0x7f8f70159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f701f4260 0x7f8f701f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f700d0850 con 0x7f8f54004d40 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f700d06d0 con 0x7f8f701f4260 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.637+0000 7f8f74e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f700d15c0 con 0x7f8f700ccca0 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51904/0 (socket says 172.21.15.44:51904) 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 -- 172.21.15.44:0/2870918171 learned_addr learned my addr 172.21.15.44:0/2870918171 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f700ccca0 msgr2=0x7f8f70159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f700ccca0 0x7f8f70159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f701f4260 msgr2=0x7f8f701f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6fe00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f701f4260 0x7f8f701f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f701f4260 0x7f8f701f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 -- 172.21.15.44:0/2870918171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f700cd160 con 0x7f8f54004d40 2024-07-17T08:37:29.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.638+0000 7f8f6f400640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011a50 tx=0x7f8f60011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.639+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6001f070 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.639+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f6000fdc0 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.639+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600126b0 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.639+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f700cd440 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.639+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f701f3d40 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.640+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f700cfdf0 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.641+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 150) ==== 99050+0+0 (secure 0 0 0) 0x7f8f60022070 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.642+0000 7f8f67400640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 0x7f8f4c079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.642+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f8f600995e0 con 0x7f8f54004d40 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.642+0000 7f8f67e00640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 msgr2=0x7f8f4c079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:29.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.642+0000 7f8f67e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 0x7f8f4c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:29.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.647+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8f60061d10 con 0x7f8f54004d40 2024-07-17T08:37:29.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.843+0000 7f8f67e00640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 msgr2=0x7f8f4c079330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:29.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.843+0000 7f8f67e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 0x7f8f4c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:29.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.862+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f8f700d0e10 con 0x7f8f54004d40 2024-07-17T08:37:29.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.869+0000 7f8f67400640 1 -- 172.21.15.44:0/2870918171 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v82)=0 v82) ==== 133+0+0 (secure 0 0 0) 0x7f8f60066260 con 0x7f8f54004d40 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.871+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 msgr2=0x7f8f4c079330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.871+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 0x7f8f4c079330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.871+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 msgr2=0x7f8f700ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.871+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011a50 tx=0x7f8f60011f10 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 shutdown_connections 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f8f4c076e70 0x7f8f4c079330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f701f4260 0x7f8f701f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f700ccca0 0x7f8f70159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 --2- 172.21.15.44:0/2870918171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f54004d40 0x7f8f700ca5b0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 >> 172.21.15.44:0/2870918171 conn(0x7f8f700b4ff0 msgr2=0x7f8f700b8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:29.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 shutdown_connections 2024-07-17T08:37:29.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:29.872+0000 7f8f74e00640 1 -- 172.21.15.44:0/2870918171 wait complete. 2024-07-17T08:37:29.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:29.891 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-07-17T08:37:29.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-07-17T08:37:30.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 -- 172.21.15.44:0/1610666813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1798004d40 msgr2=0x7f17980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 --2- 172.21.15.44:0/1610666813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1798004d40 0x7f17980051c0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009ed0 tx=0x7f1798005830 comp rx=0 tx=0).stop 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b6400640 1 -- 172.21.15.44:0/1610666813 reap_dead start 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 -- 172.21.15.44:0/1610666813 shutdown_connections 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 -- 172.21.15.44:0/1610666813 >> 172.21.15.44:0/1610666813 conn(0x7f17b00d3700 msgr2=0x7f17b00d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 -- 172.21.15.44:0/1610666813 shutdown_connections 2024-07-17T08:37:30.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.133+0000 7f17b8400640 1 -- 172.21.15.44:0/1610666813 wait complete. 2024-07-17T08:37:30.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.135+0000 7f17b8400640 1 Processor -- start 2024-07-17T08:37:30.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.135+0000 7f17b8400640 1 -- start start 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.136+0000 7f17b8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1798004d40 0x7f17b0155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.136+0000 7f17b8400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 0x7f17b0157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00b2ec0 0x7f17b00b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b8400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17b01663a0 con 0x7f17b0155b40 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0166220 con 0x7f17b00b2ec0 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f17b0166520 con 0x7f1798004d40 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 0x7f17b0157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 0x7f17b0157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51912/0 (socket says 172.21.15.44:51912) 2024-07-17T08:37:30.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 -- 172.21.15.44:0/1828866868 learned_addr learned my addr 172.21.15.44:0/1828866868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1798004d40 0x7f17b0155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b6400640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00b2ec0 0x7f17b00b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1798004d40 msgr2=0x7f17b0155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1798004d40 0x7f17b0155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00b2ec0 msgr2=0x7f17b00b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00b2ec0 0x7f17b00b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:30.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.137+0000 7f17b5000640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b0160920 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17b5000640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 0x7f17b0157c00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f179c004e50 tx=0x7f179c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c015020 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f179c011960 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c011d70 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b0158300 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.138+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b01f7610 con 0x7f17b0155b40 2024-07-17T08:37:30.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.139+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b00b3880 con 0x7f17b0155b40 2024-07-17T08:37:30.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.140+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 150) ==== 99050+0+0 (secure 0 0 0) 0x7f179c018020 con 0x7f17b0155b40 2024-07-17T08:37:30.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.141+0000 7f17ad600640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:30.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.142+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 6720+0+0 (secure 0 0 0) 0x7f179c0995f0 con 0x7f17b0155b40 2024-07-17T08:37:30.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.142+0000 7f17b5a00640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:30.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.142+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:30.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.146+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f179c01f020 con 0x7f17b0155b40 2024-07-17T08:37:30.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.343+0000 7f17b5a00640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:30.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.344+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:30.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.377+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f17b00ce5b0 con 0x7f1790076e40 2024-07-17T08:37:30.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.745+0000 7f17b5a00640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:30.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:30.745+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:37:31.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:31.547+0000 7f17b5a00640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:31.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:31.547+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:37:33.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:33.148+0000 7f17b5a00640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1251568930 2024-07-17T08:37:33.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:33.148+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:37:33.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:33.190+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 151) ==== 98557+0+0 (secure 0 0 0) 0x7f179c05ddf0 con 0x7f17b0155b40 2024-07-17T08:37:33.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:33.191+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 msgr2=0x7f1790079300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:33.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:33.191+0000 7f17ad600640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/1251568930,v1:172.21.15.175:6837/1251568930] conn(0x7f1790076e40 0x7f1790079300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:33.494 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.492+0000 7f4df5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.492+0000 7f4df5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.492+0000 7f4df5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.492+0000 7f4df5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.492+0000 7f4df5600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.493+0000 7f4df2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.493+0000 7f4df2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.493+0000 7f4df2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.493+0000 7f4df2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:33.495 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:33.493+0000 7f4df2400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.202+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 152) ==== 98684+0+0 (secure 0 0 0) 0x7f179c011d10 con 0x7f17b0155b40 2024-07-17T08:37:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.203+0000 7f17ad600640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f179003b630 0x7f1790081b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:34.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.203+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 --> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f179c00ae50 con 0x7f179003b630 2024-07-17T08:37:34.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.206+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f179003b630 0x7f1790081b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:34.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.217+0000 7f17b5a00640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f179003b630 0x7f1790081b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000fb0 tx=0x7f17a4001570 comp rx=0 tx=0).ready entity=mgr.8438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:34.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.404+0000 7f17ad600640 1 -- 172.21.15.44:0/1828866868 <== mgr.8438 v2:172.21.15.175:6834/697221060 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f179c00ae50 con 0x7f179003b630 2024-07-17T08:37:34.406 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path snap_test_dir1 2024-07-17T08:37:34.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.407+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f179003b630 msgr2=0x7f1790081b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.407+0000 7f17b8400640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f179003b630 0x7f1790081b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000fb0 tx=0x7f17a4001570 comp rx=0 tx=0).stop 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 msgr2=0x7f17b0157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 --2- 172.21.15.44:0/1828866868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0155b40 0x7f17b0157c00 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f179c004e50 tx=0x7f179c001630 comp rx=0 tx=0).stop 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b6400640 1 -- 172.21.15.44:0/1828866868 reap_dead start 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 shutdown_connections 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 >> 172.21.15.44:0/1828866868 conn(0x7f17b00d3700 msgr2=0x7f17b0159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 shutdown_connections 2024-07-17T08:37:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.408+0000 7f17b8400640 1 -- 172.21.15.44:0/1828866868 wait complete. 2024-07-17T08:37:34.431 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-07-17T08:37:34.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.648+0000 7fc356600640 1 -- 172.21.15.44:0/1911406173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b5910 msgr2=0x7fc3500cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.648+0000 7fc356600640 1 --2- 172.21.15.44:0/1911406173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b5910 0x7fc3500cdcf0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fc348005fd0 tx=0x7fc34802aa80 comp rx=0 tx=0).stop 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 -- 172.21.15.44:0/1911406173 shutdown_connections 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 --2- 172.21.15.44:0/1911406173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b5910 0x7fc3500cdcf0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 --2- 172.21.15.44:0/1911406173 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35015d600 0x7fc35006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 -- 172.21.15.44:0/1911406173 >> 172.21.15.44:0/1911406173 conn(0x7fc3500d00a0 msgr2=0x7fc3500d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 -- 172.21.15.44:0/1911406173 shutdown_connections 2024-07-17T08:37:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.649+0000 7fc356600640 1 -- 172.21.15.44:0/1911406173 wait complete. 2024-07-17T08:37:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.651+0000 7fc356600640 1 Processor -- start 2024-07-17T08:37:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.651+0000 7fc356600640 1 -- start start 2024-07-17T08:37:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3500b5910 0x7fc3500ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35015d600 0x7fc3500caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500ce680 con 0x7fc3500b2ad0 2024-07-17T08:37:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500ce500 con 0x7fc3500b5910 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc356600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500ce800 con 0x7fc35015d600 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51934/0 (socket says 172.21.15.44:51934) 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 -- 172.21.15.44:0/3471978358 learned_addr learned my addr 172.21.15.44:0/3471978358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 -- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35015d600 msgr2=0x7fc3500caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35015d600 0x7fc3500caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 -- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3500b5910 msgr2=0x7fc3500ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.652+0000 7fc34f400640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3500b5910 0x7fc3500ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc34f400640 1 -- 172.21.15.44:0/3471978358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc350146580 con 0x7fc3500b2ad0 2024-07-17T08:37:34.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc34f400640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fc34000b410 tx=0x7fc34000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340015020 con 0x7fc3500b2ad0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc340011940 con 0x7fc3500b2ad0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340011d90 con 0x7fc3500b2ad0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3500cd150 con 0x7fc3500b2ad0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.653+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3500cd710 con 0x7fc3500b2ad0 2024-07-17T08:37:34.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.655+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3500ced30 con 0x7fc3500b2ad0 2024-07-17T08:37:34.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.656+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 152) ==== 98684+0+0 (secure 0 0 0) 0x7fc340018020 con 0x7fc3500b2ad0 2024-07-17T08:37:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.658+0000 7fc347400640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 0x7fc32c078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:34.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.658+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7fc340098650 con 0x7fc3500b2ad0 2024-07-17T08:37:34.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.658+0000 7fc34ea00640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 0x7fc32c078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:34.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.659+0000 7fc34ea00640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 0x7fc32c078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3501549b0 tx=0x7fc3380079b0 comp rx=0 tx=0).ready entity=mgr.8438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:34.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.662+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc34001f020 con 0x7fc3500b2ad0 2024-07-17T08:37:34.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:34.904+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 --> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc35015aed0 con 0x7fc32c076a50 2024-07-17T08:37:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.071+0000 7fc347400640 1 -- 172.21.15.44:0/3471978358 <== mgr.8438 v2:172.21.15.175:6834/697221060 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+39 (secure 0 0 0) 0x7fc35015aed0 con 0x7fc32c076a50 2024-07-17T08:37:35.074 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path snap_test_dir1 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 msgr2=0x7fc32c078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 0x7fc32c078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3501549b0 tx=0x7fc3380079b0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 msgr2=0x7fc3500cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fc34000b410 tx=0x7fc34000b8d0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 shutdown_connections 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc32c076a50 0x7fc32c078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc35015d600 0x7fc3500caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3500b5910 0x7fc3500ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 --2- 172.21.15.44:0/3471978358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3500b2ad0 0x7fc3500cc700 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 >> 172.21.15.44:0/3471978358 conn(0x7fc3500d00a0 msgr2=0x7fc3500b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 shutdown_connections 2024-07-17T08:37:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.075+0000 7fc356600640 1 -- 172.21.15.44:0/3471978358 wait complete. 2024-07-17T08:37:35.098 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-07-17T08:37:35.098 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2024-07-17-08_38_00 in ~25s... 2024-07-17T08:37:35.098 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2024-07-17T08:37:35.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 -- 172.21.15.44:0/1889047886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dd4003a50 msgr2=0x7f2dd4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 --2- 172.21.15.44:0/1889047886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2dd4003a50 0x7f2dd4003e70 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006fb0 tx=0x7f2dd4005830 comp rx=0 tx=0).stop 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df4a00640 1 -- 172.21.15.44:0/1889047886 reap_dead start 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 -- 172.21.15.44:0/1889047886 shutdown_connections 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 -- 172.21.15.44:0/1889047886 >> 172.21.15.44:0/1889047886 conn(0x7f2df00b0eb0 msgr2=0x7f2df00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:35.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 -- 172.21.15.44:0/1889047886 shutdown_connections 2024-07-17T08:37:35.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.322+0000 7f2df6a00640 1 -- 172.21.15.44:0/1889047886 wait complete. 2024-07-17T08:37:35.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.324+0000 7f2df6a00640 1 Processor -- start 2024-07-17T08:37:35.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.324+0000 7f2df6a00640 1 -- start start 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd4003a50 0x7f2df0148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 0x7f2df00c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2df006d790 con 0x7f2df0154610 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2df006f1d0 con 0x7f2df00c9ac0 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2df6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2df015ac10 con 0x7f2dd4003a50 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2def400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2def400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51966/0 (socket says 172.21.15.44:51966) 2024-07-17T08:37:35.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.325+0000 7f2def400640 1 -- 172.21.15.44:0/3741836713 learned_addr learned my addr 172.21.15.44:0/3741836713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:35.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2df4a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 0x7f2df00c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 -- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd4003a50 msgr2=0x7f2df0148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd4003a50 0x7f2df0148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 -- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 msgr2=0x7f2df00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 0x7f2df00c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 -- 172.21.15.44:0/3741836713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df0074100 con 0x7f2df0154610 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2df4a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 0x7f2df00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2def400640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80109d0 tx=0x7f2dd8010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800ee00 con 0x7f2df0154610 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd8004030 con 0x7f2df0154610 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.326+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df00743e0 con 0x7f2df0154610 2024-07-17T08:37:35.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.327+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004460 con 0x7f2df0154610 2024-07-17T08:37:35.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.327+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01581e0 con 0x7f2df0154610 2024-07-17T08:37:35.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.328+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df00b21b0 con 0x7f2df0154610 2024-07-17T08:37:35.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.329+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 153) ==== 98949+0+0 (secure 0 0 0) 0x7f2dd80035f0 con 0x7f2df0154610 2024-07-17T08:37:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.330+0000 7f2de7e00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 0x7f2dcc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.330+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f2dd80999c0 con 0x7f2df0154610 2024-07-17T08:37:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.331+0000 7f2defe00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 0x7f2dcc0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.332+0000 7f2defe00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 0x7f2dcc0791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2de0033000 tx=0x7f2de0001930 comp rx=0 tx=0).ready entity=mgr.8438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:35.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.335+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2dd8062630 con 0x7f2df0154610 2024-07-17T08:37:35.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.570+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 --> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2df0155c60 con 0x7f2dcc076d20 2024-07-17T08:37:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.589+0000 7f2de7e00640 1 -- 172.21.15.44:0/3741836713 <== mgr.8438 v2:172.21.15.175:6834/697221060 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7f2df0155c60 con 0x7f2dcc076d20 2024-07-17T08:37:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:35.592 INFO:teuthology.orchestra.run.smithi044.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-07-17T08:37:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 msgr2=0x7f2dcc0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 0x7f2dcc0791e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2de0033000 tx=0x7f2de0001930 comp rx=0 tx=0).stop 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 msgr2=0x7f2df00c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80109d0 tx=0x7f2dd8010e90 comp rx=0 tx=0).stop 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 shutdown_connections 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f2dcc076d20 0x7f2dcc0791e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df00c9ac0 0x7f2df00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2df0154610 0x7f2df00c9580 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 --2- 172.21.15.44:0/3741836713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd4003a50 0x7f2df0148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 >> 172.21.15.44:0/3741836713 conn(0x7f2df00b0eb0 msgr2=0x7f2df006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 shutdown_connections 2024-07-17T08:37:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.592+0000 7f2df6a00640 1 -- 172.21.15.44:0/3741836713 wait complete. 2024-07-17T08:37:35.614 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-07-17T08:37:35.615 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2024-07-17T08:37:35.615 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-07-17T08:37:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.833+0000 7f0cf3e00640 1 -- 172.21.15.44:0/1143691672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 msgr2=0x7f0cd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.833+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/1143691672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cd80051c0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40aec50 tx=0x7f0cd8005830 comp rx=0 tx=0).stop 2024-07-17T08:37:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.833+0000 7f0cf3400640 1 -- 172.21.15.44:0/1143691672 reap_dead start 2024-07-17T08:37:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.833+0000 7f0cf3e00640 1 -- 172.21.15.44:0/1143691672 shutdown_connections 2024-07-17T08:37:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.833+0000 7f0cf3e00640 1 -- 172.21.15.44:0/1143691672 >> 172.21.15.44:0/1143691672 conn(0x7f0cf406c120 msgr2=0x7f0cf406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.834+0000 7f0cf3e00640 1 -- 172.21.15.44:0/1143691672 shutdown_connections 2024-07-17T08:37:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.834+0000 7f0cf3e00640 1 -- 172.21.15.44:0/1143691672 wait complete. 2024-07-17T08:37:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.835+0000 7f0cf3e00640 1 Processor -- start 2024-07-17T08:37:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.836+0000 7f0cf3e00640 1 -- start start 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf40735f0 0x7f0cf4148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 0x7f0cf4148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40ce670 con 0x7f0cd8004d40 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40b2e40 con 0x7f0cf40735f0 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf40ce7f0 con 0x7f0cf4073a70 2024-07-17T08:37:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51988/0 (socket says 172.21.15.44:51988) 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 -- 172.21.15.44:0/3870035304 learned_addr learned my addr 172.21.15.44:0/3870035304 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cea000640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf40735f0 0x7f0cf4148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3400640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 0x7f0cf4148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 -- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 msgr2=0x7f0cf4148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 0x7f0cf4148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 -- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf40735f0 msgr2=0x7f0cf4148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf40735f0 0x7f0cf4148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf2a00640 1 -- 172.21.15.44:0/3870035304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41f4cb0 con 0x7f0cd8004d40 2024-07-17T08:37:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.837+0000 7f0cf3400640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 0x7f0cf4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0cf2a00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400fa30 tx=0x7f0ce400fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400bcb0 con 0x7f0cd8004d40 2024-07-17T08:37:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ce4011440 con 0x7f0cd8004d40 2024-07-17T08:37:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4011890 con 0x7f0cd8004d40 2024-07-17T08:37:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf40af2d0 con 0x7f0cd8004d40 2024-07-17T08:37:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.838+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf40715c0 con 0x7f0cd8004d40 2024-07-17T08:37:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.840+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf40ce130 con 0x7f0cd8004d40 2024-07-17T08:37:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.841+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 153) ==== 98949+0+0 (secure 0 0 0) 0x7f0ce4019020 con 0x7f0cd8004d40 2024-07-17T08:37:35.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.842+0000 7f0ceb400640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 0x7f0cd00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:35.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.843+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f0ce4099480 con 0x7f0cd8004d40 2024-07-17T08:37:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.843+0000 7f0cea000640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 0x7f0cd00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.844+0000 7f0cea000640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 0x7f0cd00791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc001860 tx=0x7f0cdc001ce0 comp rx=0 tx=0).ready entity=mgr.8438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:35.847+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0ce4062220 con 0x7f0cd8004d40 2024-07-17T08:37:36.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.083+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 --> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f0cf40b4160 con 0x7f0cd0076d20 2024-07-17T08:37:36.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.267+0000 7f0ceb400640 1 -- 172.21.15.44:0/3870035304 <== mgr.8438 v2:172.21.15.175:6834/697221060 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0cf40b4160 con 0x7f0cd0076d20 2024-07-17T08:37:36.270 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path snap_test_dir1 2024-07-17T08:37:36.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 msgr2=0x7f0cd00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 0x7f0cd00791e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc001860 tx=0x7f0cdc001ce0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 msgr2=0x7f0cf40aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400fa30 tx=0x7f0ce400fef0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 shutdown_connections 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0cd0076d20 0x7f0cd00791e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cf4073a70 0x7f0cf4148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cf40735f0 0x7f0cf4148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 --2- 172.21.15.44:0/3870035304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cd8004d40 0x7f0cf40aed90 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 >> 172.21.15.44:0/3870035304 conn(0x7f0cf406c120 msgr2=0x7f0cf40adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 shutdown_connections 2024-07-17T08:37:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.272+0000 7f0cf3e00640 1 -- 172.21.15.44:0/3870035304 wait complete. 2024-07-17T08:37:36.299 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-07-17T08:37:36.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:36.299 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2024-07-17T08:37:36.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:36.355 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-07-17T08:37:36.408 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.617+0000 7f7b37200640 1 -- 172.21.15.44:0/3159534553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b3015d350 msgr2=0x7f7b30161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.617+0000 7f7b37200640 1 --2- 172.21.15.44:0/3159534553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b3015d350 0x7f7b30161770 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009900 tx=0x7f7b2402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 -- 172.21.15.44:0/3159534553 shutdown_connections 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 --2- 172.21.15.44:0/3159534553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b300ce3b0 0x7f7b300d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 --2- 172.21.15.44:0/3159534553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b3015d350 0x7f7b30161770 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 -- 172.21.15.44:0/3159534553 >> 172.21.15.44:0/3159534553 conn(0x7f7b300d04c0 msgr2=0x7f7b300d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 -- 172.21.15.44:0/3159534553 shutdown_connections 2024-07-17T08:37:36.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.618+0000 7f7b37200640 1 -- 172.21.15.44:0/3159534553 wait complete. 2024-07-17T08:37:36.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.620+0000 7f7b37200640 1 Processor -- start 2024-07-17T08:37:36.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.620+0000 7f7b37200640 1 -- start start 2024-07-17T08:37:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.621+0000 7f7b37200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.621+0000 7f7b37200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 0x7f7b30156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.621+0000 7f7b37200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b3015d350 0x7f7b301568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b37200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3006e3c0 con 0x7f7b300ce3b0 2024-07-17T08:37:36.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b37200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3006e240 con 0x7f7b300d1440 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b37200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3006e540 con 0x7f7b3015d350 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51998/0 (socket says 172.21.15.44:51998) 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 -- 172.21.15.44:0/1528581538 learned_addr learned my addr 172.21.15.44:0/1528581538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b36800640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b3015d350 0x7f7b301568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35400640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 0x7f7b30156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 -- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b3015d350 msgr2=0x7f7b301568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b3015d350 0x7f7b301568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 -- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 msgr2=0x7f7b30156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 0x7f7b30156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35e00640 1 -- 172.21.15.44:0/1528581538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3015c390 con 0x7f7b300ce3b0 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.622+0000 7f7b35400640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 0x7f7b30156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:36.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b35e00640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b24003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:36.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2403e030 con 0x7f7b300ce3b0 2024-07-17T08:37:36.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b24003e00 con 0x7f7b300ce3b0 2024-07-17T08:37:36.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3015c670 con 0x7f7b300ce3b0 2024-07-17T08:37:36.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24035040 con 0x7f7b300ce3b0 2024-07-17T08:37:36.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.623+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301571c0 con 0x7f7b300ce3b0 2024-07-17T08:37:36.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.624+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b300762e0 con 0x7f7b300ce3b0 2024-07-17T08:37:36.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.626+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 153) ==== 98949+0+0 (secure 0 0 0) 0x7f7b24041030 con 0x7f7b300ce3b0 2024-07-17T08:37:36.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.626+0000 7f7b2a000640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 0x7f7b140791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:36.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.627+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f7b240b9220 con 0x7f7b300ce3b0 2024-07-17T08:37:36.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.627+0000 7f7b35400640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 0x7f7b140791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:36.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.629+0000 7f7b35400640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 0x7f7b140791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b300c9ce0 tx=0x7f7b20009040 comp rx=0 tx=0).ready entity=mgr.8438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:36.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.631+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7b300762e0 con 0x7f7b300ce3b0 2024-07-17T08:37:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:36.895+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f7b3006eae0 con 0x7f7b300ce3b0 2024-07-17T08:37:37.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.282+0000 7f7b2a000640 1 -- 172.21.15.44:0/1528581538 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v154) ==== 93+0+0 (secure 0 0 0) 0x7f7b24081e90 con 0x7f7b300ce3b0 2024-07-17T08:37:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.286+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 msgr2=0x7f7b140791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.286+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 0x7f7b140791e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b300c9ce0 tx=0x7f7b20009040 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.286+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 msgr2=0x7f7b300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.286+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b24003910 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 shutdown_connections 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f7b14076d20 0x7f7b140791e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b3015d350 0x7f7b301568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b300d1440 0x7f7b30156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 --2- 172.21.15.44:0/1528581538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b300ce3b0 0x7f7b300b8b90 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 >> 172.21.15.44:0/1528581538 conn(0x7f7b300d04c0 msgr2=0x7f7b3015ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:37.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.287+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 shutdown_connections 2024-07-17T08:37:37.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.289+0000 7f7b37200640 1 -- 172.21.15.44:0/1528581538 wait complete. 2024-07-17T08:37:37.322 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:37:37.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:37.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:37.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:37:37.529 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:37:37.529 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:37.530 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:37.578 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:37.578 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:37.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:37:37.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 -- 172.21.15.44:0/632979144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 msgr2=0x7f5b5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:37.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 --2- 172.21.15.44:0/632979144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b5c004c80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5b640016c0 tx=0x7f5b5c005830 comp rx=0 tx=0).stop 2024-07-17T08:37:37.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 -- 172.21.15.44:0/632979144 shutdown_connections 2024-07-17T08:37:37.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 --2- 172.21.15.44:0/632979144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b5c004c80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 -- 172.21.15.44:0/632979144 >> 172.21.15.44:0/632979144 conn(0x7f5b780b5560 msgr2=0x7f5b780b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:37.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 -- 172.21.15.44:0/632979144 shutdown_connections 2024-07-17T08:37:37.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.915+0000 7f5b7da00640 1 -- 172.21.15.44:0/632979144 wait complete. 2024-07-17T08:37:37.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.917+0000 7f5b7da00640 1 Processor -- start 2024-07-17T08:37:37.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.917+0000 7f5b7da00640 1 -- start start 2024-07-17T08:37:37.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b78152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:37.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:37.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 0x7f5b78154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b780b3530 con 0x7f5b781f4830 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b780b33b0 con 0x7f5b5c004800 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.918+0000 7f5b7da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5b780b1cd0 con 0x7f5b78154b00 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52018/0 (socket says 172.21.15.44:52018) 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 -- 172.21.15.44:0/2488995809 learned_addr learned my addr 172.21.15.44:0/2488995809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77e00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b78152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b7d000640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 0x7f5b78154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 msgr2=0x7f5b78154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:37.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 0x7f5b78154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 msgr2=0x7f5b78152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b78152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 -- 172.21.15.44:0/2488995809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b781f51d0 con 0x7f5b781f4830 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b7d000640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 0x7f5b78154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.919+0000 7f5b77400640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f5b640017e0 tx=0x7f5b640063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.920+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64042070 con 0x7f5b781f4830 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.920+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b78149ee0 con 0x7f5b781f4830 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.920+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b64039070 con 0x7f5b781f4830 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.920+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64003d70 con 0x7f5b781f4830 2024-07-17T08:37:37.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.920+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b78155c40 con 0x7f5b781f4830 2024-07-17T08:37:37.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.921+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b780b21c0 con 0x7f5b781f4830 2024-07-17T08:37:37.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.922+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7f5b6403b070 con 0x7f5b781f4830 2024-07-17T08:37:37.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.923+0000 7f5b6fe00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 0x7f5b58079280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:37.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.923+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f5b640b8220 con 0x7f5b781f4830 2024-07-17T08:37:37.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.923+0000 7f5b77e00640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 msgr2=0x7f5b58079280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:37.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.923+0000 7f5b77e00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 0x7f5b58079280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:37.931+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5b64080e40 con 0x7f5b781f4830 2024-07-17T08:37:38.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.124+0000 7f5b77e00640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 msgr2=0x7f5b58079280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:38.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.124+0000 7f5b77e00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 0x7f5b58079280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.157+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b780b16e0 con 0x7f5b781f4830 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.158+0000 7f5b6fe00640 1 -- 172.21.15.44:0/2488995809 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v265) ==== 81+0+652 (secure 0 0 0) 0x7f5b64089080 con 0x7f5b781f4830 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2057041105 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1342905808 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/29215155 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3567462564 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/622582395 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.160 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3175300474 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2184874848 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/1251568930 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/1251568930 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1545286674 2024-07-18T08:37:33.175035+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2008194206 2024-07-17T09:37:24.171533+0000 2024-07-17T08:37:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:listed 11 entries 2024-07-17T08:37:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 msgr2=0x7f5b58079280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 0x7f5b58079280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 msgr2=0x7f5b781f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f5b640017e0 tx=0x7f5b640063a0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 shutdown_connections 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f5b58076dc0 0x7f5b58079280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b78154b00 0x7f5b78154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b781f4830 0x7f5b781f4c90 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 --2- 172.21.15.44:0/2488995809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b5c004800 0x7f5b78152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 >> 172.21.15.44:0/2488995809 conn(0x7f5b780b5560 msgr2=0x7f5b780b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.161+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 shutdown_connections 2024-07-17T08:37:38.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.162+0000 7f5b7da00640 1 -- 172.21.15.44:0/2488995809 wait complete. 2024-07-17T08:37:38.183 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:37:38.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:38.183 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:37:38.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:37:38.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:37:38.258 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:37:38.302 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:37:38.303 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:38.303 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:38.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:38.352 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:37:38.352 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:38.352 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:38.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:37:38.403 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:37:38.403 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:38.403 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:38.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:38.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.647+0000 7f40e7400640 1 -- 172.21.15.44:0/2880465362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 msgr2=0x7f40e00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.647+0000 7f40e7400640 1 --2- 172.21.15.44:0/2880465362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e00b25d0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009900 tx=0x7f40d802a580 comp rx=0 tx=0).stop 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.647+0000 7f40e7400640 1 -- 172.21.15.44:0/2880465362 shutdown_connections 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.647+0000 7f40e7400640 1 --2- 172.21.15.44:0/2880465362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e00b25d0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.647+0000 7f40e7400640 1 -- 172.21.15.44:0/2880465362 >> 172.21.15.44:0/2880465362 conn(0x7f40e00b0fb0 msgr2=0x7f40e00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.648+0000 7f40e7400640 1 -- 172.21.15.44:0/2880465362 shutdown_connections 2024-07-17T08:37:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.648+0000 7f40e7400640 1 -- 172.21.15.44:0/2880465362 wait complete. 2024-07-17T08:37:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.649+0000 7f40e7400640 1 Processor -- start 2024-07-17T08:37:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.649+0000 7f40e7400640 1 -- start start 2024-07-17T08:37:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e006ea50 0x7f40e009bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e00ae810 0x7f40e01f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40e00aed30 con 0x7f40e00ac930 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40e006f0d0 con 0x7f40e00ae810 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40e7400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40e00b2b10 con 0x7f40e006ea50 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.650+0000 7f40dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52036/0 (socket says 172.21.15.44:52036) 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 -- 172.21.15.44:0/2941171703 learned_addr learned my addr 172.21.15.44:0/2941171703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40e4a00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e006ea50 0x7f40e009bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40e5400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e00ae810 0x7f40e01f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e006ea50 msgr2=0x7f40e009bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e006ea50 0x7f40e009bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e00ae810 msgr2=0x7f40e01f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e00ae810 0x7f40e01f40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 -- 172.21.15.44:0/2941171703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e01577e0 con 0x7f40e00ac930 2024-07-17T08:37:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dfe00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c480 tx=0x7f40cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.651+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0052d0 con 0x7f40e00ac930 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.652+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40cc005470 con 0x7f40e00ac930 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.652+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc012870 con 0x7f40e00ac930 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.652+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e0157ac0 con 0x7f40e00ac930 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.652+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e01371a0 con 0x7f40e00ac930 2024-07-17T08:37:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.653+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e00b2f30 con 0x7f40e00ac930 2024-07-17T08:37:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.654+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7f40cc0035f0 con 0x7f40e00ac930 2024-07-17T08:37:38.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.655+0000 7f40dcc00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 0x7f40c0079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.655+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f40cc098800 con 0x7f40e00ac930 2024-07-17T08:37:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.655+0000 7f40e4a00640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 msgr2=0x7f40c0079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:38.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.655+0000 7f40e4a00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 0x7f40c0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:38.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.659+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f40cc0614a0 con 0x7f40e00ac930 2024-07-17T08:37:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.856+0000 7f40e4a00640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 msgr2=0x7f40c0079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.856+0000 7f40e4a00640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 0x7f40c0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:38.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.902+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40e0137370 con 0x7f40e00ac930 2024-07-17T08:37:38.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.903+0000 7f40dcc00640 1 -- 172.21.15.44:0/2941171703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7f40cc060da0 con 0x7f40e00ac930 2024-07-17T08:37:38.905 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:38.906 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:38.906 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:38.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 msgr2=0x7f40c0079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:38.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 0x7f40c0079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 msgr2=0x7f40e01f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c480 tx=0x7f40cc00c940 comp rx=0 tx=0).stop 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 shutdown_connections 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f40c0076d60 0x7f40c0079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40e00ae810 0x7f40e01f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e00ac930 0x7f40e01f3b70 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 --2- 172.21.15.44:0/2941171703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40e006ea50 0x7f40e009bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.906+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 >> 172.21.15.44:0/2941171703 conn(0x7f40e00b0fb0 msgr2=0x7f40e00b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.907+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 shutdown_connections 2024-07-17T08:37:38.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:38.907+0000 7f40e7400640 1 -- 172.21.15.44:0/2941171703 wait complete. 2024-07-17T08:37:38.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 -- 172.21.15.44:0/437556975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180b5400 msgr2=0x7fd61815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 --2- 172.21.15.44:0/437556975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180b5400 0x7fd61815e2d0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fd610009ca0 tx=0x7fd61002aa30 comp rx=0 tx=0).stop 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 -- 172.21.15.44:0/437556975 shutdown_connections 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 --2- 172.21.15.44:0/437556975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180b5400 0x7fd61815e2d0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 --2- 172.21.15.44:0/437556975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b8c80 0x7fd6180b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 -- 172.21.15.44:0/437556975 >> 172.21.15.44:0/437556975 conn(0x7fd6180cfec0 msgr2=0x7fd6180d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:39.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 -- 172.21.15.44:0/437556975 shutdown_connections 2024-07-17T08:37:39.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.119+0000 7fd617200640 1 -- 172.21.15.44:0/437556975 wait complete. 2024-07-17T08:37:39.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.121+0000 7fd617200640 1 Processor -- start 2024-07-17T08:37:39.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.121+0000 7fd617200640 1 -- start start 2024-07-17T08:37:39.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b5400 0x7fd61814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180b8c80 0x7fd61814c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd61806e0c0 con 0x7fd6180ce5c0 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6180b92f0 con 0x7fd6180b8c80 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd617200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd61806e240 con 0x7fd6180b5400 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd616800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd616800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38760/0 (socket says 172.21.15.44:38760) 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd616800640 1 -- 172.21.15.44:0/1695797046 learned_addr learned my addr 172.21.15.44:0/1695797046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd615e00640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b5400 0x7fd61814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.122+0000 7fd60fe00640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180b8c80 0x7fd61814c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b5400 msgr2=0x7fd61814e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b5400 0x7fd61814e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180b8c80 msgr2=0x7fd61814c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180b8c80 0x7fd61814c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 -- 172.21.15.44:0/1695797046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61814ef90 con 0x7fd6180ce5c0 2024-07-17T08:37:39.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd616800640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fd610009ca0 tx=0x7fd610003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:39.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610047070 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd610004250 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61814f270 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610033050 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.123+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6180cb700 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.125+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61814f790 con 0x7fd6180ce5c0 2024-07-17T08:37:39.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.126+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7fd61004a070 con 0x7fd6180ce5c0 2024-07-17T08:37:39.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.127+0000 7fd60e000640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 0x7fd5f4079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.127+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7fd6100b9e20 con 0x7fd6180ce5c0 2024-07-17T08:37:39.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.128+0000 7fd615e00640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 msgr2=0x7fd5f4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:39.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.128+0000 7fd615e00640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 0x7fd5f4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:39.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.132+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd61814f790 con 0x7fd6180ce5c0 2024-07-17T08:37:39.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.329+0000 7fd615e00640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 msgr2=0x7fd5f4079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:39.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.329+0000 7fd615e00640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 0x7fd5f4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:39.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.372+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd61806e6f0 con 0x7fd6180ce5c0 2024-07-17T08:37:39.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.373+0000 7fd60e000640 1 -- 172.21.15.44:0/1695797046 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7fd6100829b0 con 0x7fd6180ce5c0 2024-07-17T08:37:39.375 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:39.375 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:39.375 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:39.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.375+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 msgr2=0x7fd5f4079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.375+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 0x7fd5f4079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.375+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 msgr2=0x7fd6180ac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.375+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fd610009ca0 tx=0x7fd610003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 shutdown_connections 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fd5f4076d60 0x7fd5f4079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180ce5c0 0x7fd6180ac980 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6180b8c80 0x7fd61814c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 --2- 172.21.15.44:0/1695797046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6180b5400 0x7fd61814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 >> 172.21.15.44:0/1695797046 conn(0x7fd6180cfec0 msgr2=0x7fd6180b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 shutdown_connections 2024-07-17T08:37:39.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.376+0000 7fd617200640 1 -- 172.21.15.44:0/1695797046 wait complete. 2024-07-17T08:37:39.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:39.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.584+0000 7fe7e4600640 1 -- 172.21.15.44:0/3777427720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc0b5910 msgr2=0x7fe7dc0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.584+0000 7fe7e4600640 1 --2- 172.21.15.44:0/3777427720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc0b5910 0x7fe7dc0cdcf0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d402aa90 comp rx=0 tx=0).stop 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 -- 172.21.15.44:0/3777427720 shutdown_connections 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 --2- 172.21.15.44:0/3777427720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc0b5910 0x7fe7dc0cdcf0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 --2- 172.21.15.44:0/3777427720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc15d600 0x7fe7dc06e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 -- 172.21.15.44:0/3777427720 >> 172.21.15.44:0/3777427720 conn(0x7fe7dc0d00a0 msgr2=0x7fe7dc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 -- 172.21.15.44:0/3777427720 shutdown_connections 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.585+0000 7fe7e4600640 1 -- 172.21.15.44:0/3777427720 wait complete. 2024-07-17T08:37:39.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.586+0000 7fe7e4600640 1 Processor -- start 2024-07-17T08:37:39.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.587+0000 7fe7e4600640 1 -- start start 2024-07-17T08:37:39.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.587+0000 7fe7e4600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc0b2ad0 0x7fe7dc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.587+0000 7fe7e4600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc0b5910 0x7fe7dc0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.587+0000 7fe7e4600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e4600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc0ce710 con 0x7fe7dc15d600 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e4600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc0ce590 con 0x7fe7dc0b2ad0 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e4600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7dc0ce890 con 0x7fe7dc0b5910 2024-07-17T08:37:39.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38774/0 (socket says 172.21.15.44:38774) 2024-07-17T08:37:39.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 -- 172.21.15.44:0/369869262 learned_addr learned my addr 172.21.15.44:0/369869262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:39.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e1200640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc0b5910 0x7fe7dc0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e1c00640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc0b2ad0 0x7fe7dc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:39.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc0b5910 msgr2=0x7fe7dc0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc0b5910 0x7fe7dc0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc0b2ad0 msgr2=0x7fe7dc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc0b2ad0 0x7fe7dc0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.588+0000 7fe7e2600640 1 -- 172.21.15.44:0/369869262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc0cd120 con 0x7fe7dc15d600 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7e2600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009d90 tx=0x7fe7d4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:39.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4046070 con 0x7fe7dc15d600 2024-07-17T08:37:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7d4003e50 con 0x7fe7dc15d600 2024-07-17T08:37:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40044b0 con 0x7fe7dc15d600 2024-07-17T08:37:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc0cd400 con 0x7fe7dc15d600 2024-07-17T08:37:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.589+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc0caba0 con 0x7fe7dc15d600 2024-07-17T08:37:39.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.590+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc0cf540 con 0x7fe7dc15d600 2024-07-17T08:37:39.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.591+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7fe7d4049070 con 0x7fe7dc15d600 2024-07-17T08:37:39.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.593+0000 7fe7d9600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 0x7fe7bc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:39.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.593+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7fe7d40b9af0 con 0x7fe7dc15d600 2024-07-17T08:37:39.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.593+0000 7fe7e1c00640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 msgr2=0x7fe7bc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:39.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.593+0000 7fe7e1c00640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 0x7fe7bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:39.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.598+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7d4032030 con 0x7fe7dc15d600 2024-07-17T08:37:39.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.795+0000 7fe7e1c00640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 msgr2=0x7fe7bc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:39.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.795+0000 7fe7e1c00640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 0x7fe7bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:39.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.799+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7dc0b6120 con 0x7fe7dc15d600 2024-07-17T08:37:39.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.801+0000 7fe7d9600640 1 -- 172.21.15.44:0/369869262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) ==== 74+0+18448 (secure 0 0 0) 0x7fe7d4082690 con 0x7fe7dc15d600 2024-07-17T08:37:39.803 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:39.803 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":265,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:33.175126+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:16.290172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:17.260106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"263","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2057041105":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1342905808":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/29215155":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3567462564":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/622582395":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3175300474":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2184874848":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6837/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6834/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1545286674":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2008194206":"2024-07-17T09:37:24.171533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 msgr2=0x7fe7bc079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 0x7fe7bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 msgr2=0x7fe7dc1f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009d90 tx=0x7fe7d4003ce0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 shutdown_connections 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fe7bc076d60 0x7fe7bc079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7dc15d600 0x7fe7dc1f7300 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7dc0b5910 0x7fe7dc0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 --2- 172.21.15.44:0/369869262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7dc0b2ad0 0x7fe7dc0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 >> 172.21.15.44:0/369869262 conn(0x7fe7dc0d00a0 msgr2=0x7fe7dc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:39.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 shutdown_connections 2024-07-17T08:37:39.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:39.804+0000 7fe7e4600640 1 -- 172.21.15.44:0/369869262 wait complete. 2024-07-17T08:37:39.824 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:37:39.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 -- 172.21.15.44:0/398623980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41e40052e0 msgr2=0x7f41e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 --2- 172.21.15.44:0/398623980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41e40052e0 0x7f41e400b9c0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f41f4006fd0 tx=0x7f41e4005830 comp rx=0 tx=0).stop 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 -- 172.21.15.44:0/398623980 shutdown_connections 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 --2- 172.21.15.44:0/398623980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41e40052e0 0x7f41e400b9c0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 -- 172.21.15.44:0/398623980 >> 172.21.15.44:0/398623980 conn(0x7f41fc0b0fb0 msgr2=0x7f41fc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:40.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 -- 172.21.15.44:0/398623980 shutdown_connections 2024-07-17T08:37:40.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.017+0000 7f4204000640 1 -- 172.21.15.44:0/398623980 wait complete. 2024-07-17T08:37:40.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.019+0000 7f4204000640 1 Processor -- start 2024-07-17T08:37:40.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.019+0000 7f4204000640 1 -- start start 2024-07-17T08:37:40.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41e40052e0 0x7f41fc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41fc0ceef0 0x7f41fc0cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0adb20 con 0x7f41fc1f4880 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0ad9a0 con 0x7f41e40052e0 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4204000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc0ac220 con 0x7f41fc0ceef0 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38786/0 (socket says 172.21.15.44:38786) 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 -- 172.21.15.44:0/711581243 learned_addr learned my addr 172.21.15.44:0/711581243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41fc0ceef0 msgr2=0x7f41fc0cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41fc0ceef0 0x7f41fc0cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41e40052e0 msgr2=0x7f41fc071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41e40052e0 0x7f41fc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.020+0000 7f4200c00640 1 -- 172.21.15.44:0/711581243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc0725a0 con 0x7f41fc1f4880 2024-07-17T08:37:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f4200c00640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f41e8004f00 tx=0x7f41e8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8015020 con 0x7f41fc1f4880 2024-07-17T08:37:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e8011960 con 0x7f41fc1f4880 2024-07-17T08:37:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1f4d00 con 0x7f41fc1f4880 2024-07-17T08:37:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8011d90 con 0x7f41fc1f4880 2024-07-17T08:37:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.021+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc071fe0 con 0x7f41fc1f4880 2024-07-17T08:37:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.022+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc0cf8b0 con 0x7f41fc1f4880 2024-07-17T08:37:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.023+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7f41e8018020 con 0x7f41fc1f4880 2024-07-17T08:37:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.024+0000 7f41f9600640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 0x7f41dc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.024+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f41e8099720 con 0x7f41fc1f4880 2024-07-17T08:37:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.025+0000 7f4201600640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 msgr2=0x7f41dc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.025+0000 7f4201600640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 0x7f41dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.028+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f41fc0cf8b0 con 0x7f41fc1f4880 2024-07-17T08:37:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.226+0000 7f4201600640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 msgr2=0x7f41dc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:40.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.226+0000 7f4201600640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 0x7f41dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:40.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:37:40.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:37:40.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.280+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41fc15ee90 con 0x7f41fc1f4880 2024-07-17T08:37:40.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.281+0000 7f41f9600640 1 -- 172.21.15.44:0/711581243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7f41e801f020 con 0x7f41fc1f4880 2024-07-17T08:37:40.282 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:40.283 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:40.283 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 msgr2=0x7f41dc079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 0x7f41dc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 msgr2=0x7f41fc1465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f41e8004f00 tx=0x7f41e8001630 comp rx=0 tx=0).stop 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 shutdown_connections 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f41dc076d60 0x7f41dc079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.283+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41fc0ceef0 0x7f41fc0cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.284+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41fc1f4880 0x7f41fc1465e0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.284+0000 7f4204000640 1 --2- 172.21.15.44:0/711581243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41e40052e0 0x7f41fc071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.284+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 >> 172.21.15.44:0/711581243 conn(0x7f41fc0b0fb0 msgr2=0x7f41fc15a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.284+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 shutdown_connections 2024-07-17T08:37:40.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.284+0000 7f4204000640 1 -- 172.21.15.44:0/711581243 wait complete. 2024-07-17T08:37:40.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 -- 172.21.15.44:0/274074258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc0ac930 msgr2=0x7f91dc0cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 --2- 172.21.15.44:0/274074258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc0ac930 0x7f91dc0cd690 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009ca0 tx=0x7f91d402aac0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 -- 172.21.15.44:0/274074258 shutdown_connections 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 --2- 172.21.15.44:0/274074258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc0ac930 0x7f91dc0cd690 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 --2- 172.21.15.44:0/274074258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc1522c0 0x7f91dc0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 -- 172.21.15.44:0/274074258 >> 172.21.15.44:0/274074258 conn(0x7f91dc0b0fb0 msgr2=0x7f91dc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 -- 172.21.15.44:0/274074258 shutdown_connections 2024-07-17T08:37:40.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.532+0000 7f91e3400640 1 -- 172.21.15.44:0/274074258 wait complete. 2024-07-17T08:37:40.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.534+0000 7f91e3400640 1 Processor -- start 2024-07-17T08:37:40.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.535+0000 7f91e3400640 1 -- start start 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc0ac930 0x7f91dc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 0x7f91dc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0ce010 con 0x7f91dc1522c0 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cde90 con 0x7f91dc0adcd0 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.536+0000 7f91e3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0ce190 con 0x7f91dc0ac930 2024-07-17T08:37:40.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38798/0 (socket says 172.21.15.44:38798) 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 -- 172.21.15.44:0/3243224349 learned_addr learned my addr 172.21.15.44:0/3243224349 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e0a00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc0ac930 0x7f91dc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91dbe00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 0x7f91dc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc0ac930 msgr2=0x7f91dc1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc0ac930 0x7f91dc1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 msgr2=0x7f91dc15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 0x7f91dc15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91e1400640 1 -- 172.21.15.44:0/3243224349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1f2630 con 0x7f91dc1522c0 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.537+0000 7f91dbe00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 0x7f91dc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:40.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91e1400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f91d4006010 tx=0x7f91d4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4046070 con 0x7f91dc1522c0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91d4004250 con 0x7f91dc1522c0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004680 con 0x7f91dc1522c0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc1f2910 con 0x7f91dc1522c0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.538+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc153eb0 con 0x7f91dc1522c0 2024-07-17T08:37:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.539+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc1539e0 con 0x7f91dc1522c0 2024-07-17T08:37:40.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.540+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7f91d4049070 con 0x7f91dc1522c0 2024-07-17T08:37:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.541+0000 7f91d8c00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 0x7f91bc079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.541+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f91d40b9a70 con 0x7f91dc1522c0 2024-07-17T08:37:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.541+0000 7f91e0a00640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 msgr2=0x7f91bc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:40.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.541+0000 7f91e0a00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 0x7f91bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:40.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.545+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f91d4032030 con 0x7f91dc1522c0 2024-07-17T08:37:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.742+0000 7f91e0a00640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 msgr2=0x7f91bc079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:40.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.742+0000 7f91e0a00640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 0x7f91bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:40.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.793+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91dc0ce6b0 con 0x7f91dc1522c0 2024-07-17T08:37:40.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.794+0000 7f91d8c00640 1 -- 172.21.15.44:0/3243224349 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+5239 (secure 0 0 0) 0x7f91d4004820 con 0x7f91dc1522c0 2024-07-17T08:37:40.796 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:40.796 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":146,"btime":"2024-07-17T08:37:21:854315+0000","default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:18.288317+0000","modified":"2024-07-17T08:37:21.376632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":142,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/1990487283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1990487283},{"type":"v1","addr":"172.21.15.44:6835","nonce":1990487283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11325,"qdb_cluster":[11325]},"id":17}]} 2024-07-17T08:37:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 146 2024-07-17T08:37:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 msgr2=0x7f91bc079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 0x7f91bc079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 msgr2=0x7f91dc1f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:40.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f91d4006010 tx=0x7f91d4003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 shutdown_connections 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f91bc076d60 0x7f91bc079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91dc1522c0 0x7f91dc1f1b70 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91dc0adcd0 0x7f91dc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 --2- 172.21.15.44:0/3243224349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91dc0ac930 0x7f91dc1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 >> 172.21.15.44:0/3243224349 conn(0x7f91dc0b0fb0 msgr2=0x7f91dc0cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.797+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 shutdown_connections 2024-07-17T08:37:40.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:40.798+0000 7f91e3400640 1 -- 172.21.15.44:0/3243224349 wait complete. 2024-07-17T08:37:40.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:41.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.004+0000 7fc046e00640 1 -- 172.21.15.44:0/3618240140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 msgr2=0x7fc0400ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.004+0000 7fc046e00640 1 --2- 172.21.15.44:0/3618240140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc0400ae2d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc038009ca0 tx=0x7fc03802aac0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 -- 172.21.15.44:0/3618240140 shutdown_connections 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 --2- 172.21.15.44:0/3618240140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc0400ae2d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 --2- 172.21.15.44:0/3618240140 >> v2:172.21.15.175:3300/0 conn(0x7fc04006c020 0x7fc04006eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 -- 172.21.15.44:0/3618240140 >> 172.21.15.44:0/3618240140 conn(0x7fc0400b2240 msgr2=0x7fc0400b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 -- 172.21.15.44:0/3618240140 shutdown_connections 2024-07-17T08:37:41.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.005+0000 7fc046e00640 1 -- 172.21.15.44:0/3618240140 wait complete. 2024-07-17T08:37:41.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.007+0000 7fc046e00640 1 Processor -- start 2024-07-17T08:37:41.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.007+0000 7fc046e00640 1 -- start start 2024-07-17T08:37:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.008+0000 7fc046e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04006c020 0x7fc040160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.008+0000 7fc046e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc046e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc04015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc046e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400b4170 con 0x7fc040146cb0 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc046e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400b3ff0 con 0x7fc04006c020 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc046e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400b42f0 con 0x7fc040147090 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38810/0 (socket says 172.21.15.44:38810) 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 -- 172.21.15.44:0/3070090522 learned_addr learned my addr 172.21.15.44:0/3070090522 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc044e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc04015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc03fe00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04006c020 0x7fc040160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 msgr2=0x7fc04015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc04015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04006c020 msgr2=0x7fc040160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.009+0000 7fc037e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04006c020 0x7fc040160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc037e00640 1 -- 172.21.15.44:0/3070090522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04015dfb0 con 0x7fc040146cb0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc044e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc04015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc037e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fc02800e9f0 tx=0x7fc02800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02800cd80 con 0x7fc040146cb0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc028010430 con 0x7fc040146cb0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028010840 con 0x7fc040146cb0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04015e290 con 0x7fc040146cb0 2024-07-17T08:37:41.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.010+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04015d3d0 con 0x7fc040146cb0 2024-07-17T08:37:41.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.012+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0400b4810 con 0x7fc040146cb0 2024-07-17T08:37:41.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.013+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7fc0280109e0 con 0x7fc040146cb0 2024-07-17T08:37:41.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.014+0000 7fc03cc00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 0x7fc01c079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.014+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7fc0280997e0 con 0x7fc040146cb0 2024-07-17T08:37:41.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.015+0000 7fc03fe00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 msgr2=0x7fc01c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:41.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.015+0000 7fc03fe00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 0x7fc01c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:41.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.021+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc028062430 con 0x7fc040146cb0 2024-07-17T08:37:41.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.216+0000 7fc03fe00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 msgr2=0x7fc01c079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:41.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.217+0000 7fc03fe00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 0x7fc01c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:37:41.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.227+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc04006fd00 con 0x7fc040146cb0 2024-07-17T08:37:41.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.229+0000 7fc03cc00640 1 -- 172.21.15.44:0/3070090522 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) ==== 74+0+18448 (secure 0 0 0) 0x7fc028061d30 con 0x7fc040146cb0 2024-07-17T08:37:41.231 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:41.231 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":265,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:33.175126+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:16.290172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:17.260106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"263","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2057041105":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1342905808":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/29215155":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3567462564":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/622582395":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3175300474":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2184874848":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6837/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6834/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1545286674":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2008194206":"2024-07-17T09:37:24.171533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 msgr2=0x7fc01c079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 0x7fc01c079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 msgr2=0x7fc0401f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fc02800e9f0 tx=0x7fc02800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 shutdown_connections 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7fc01c076d60 0x7fc01c079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc040147090 0x7fc04015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040146cb0 0x7fc0401f51c0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 --2- 172.21.15.44:0/3070090522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04006c020 0x7fc040160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 >> 172.21.15.44:0/3070090522 conn(0x7fc0400b2240 msgr2=0x7fc04006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 shutdown_connections 2024-07-17T08:37:41.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.233+0000 7fc046e00640 1 -- 172.21.15.44:0/3070090522 wait complete. 2024-07-17T08:37:41.252 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:37:41.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 -- 172.21.15.44:0/1526120885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e980051c0 msgr2=0x7f0e9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/1526120885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e980051c0 0x7f0e9800b9c0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8005fd0 tx=0x7f0e98005830 comp rx=0 tx=0).stop 2024-07-17T08:37:41.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 -- 172.21.15.44:0/1526120885 shutdown_connections 2024-07-17T08:37:41.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/1526120885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e980051c0 0x7f0e9800b9c0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 -- 172.21.15.44:0/1526120885 >> 172.21.15.44:0/1526120885 conn(0x7f0eb00b0eb0 msgr2=0x7f0eb00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 -- 172.21.15.44:0/1526120885 shutdown_connections 2024-07-17T08:37:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.446+0000 7f0eb7e00640 1 -- 172.21.15.44:0/1526120885 wait complete. 2024-07-17T08:37:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.448+0000 7f0eb7e00640 1 Processor -- start 2024-07-17T08:37:41.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.448+0000 7f0eb7e00640 1 -- start start 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 0x7f0eb00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 0x7f0eb0072820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb00b2540 con 0x7f0eb01593d0 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb00b23c0 con 0x7f0eb01f7600 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb006d780 con 0x7f0e980051c0 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38834/0 (socket says 172.21.15.44:38834) 2024-07-17T08:37:41.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.449+0000 7f0eb5e00640 1 -- 172.21.15.44:0/3093492593 learned_addr learned my addr 172.21.15.44:0/3093492593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5400640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 0x7f0eb00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb4a00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 0x7f0eb0072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 -- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 msgr2=0x7f0eb00b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 0x7f0eb00b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 -- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 msgr2=0x7f0eb0072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 0x7f0eb0072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 -- 172.21.15.44:0/3093492593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb01f4010 con 0x7f0eb01593d0 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb4a00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 0x7f0eb0072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5400640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 0x7f0eb00b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:41.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.450+0000 7f0eb5e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009e20 tx=0x7f0ea8033a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:41.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.451+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8038040 con 0x7f0eb01593d0 2024-07-17T08:37:41.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.451+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ea8043070 con 0x7f0eb01593d0 2024-07-17T08:37:41.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.451+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea803d480 con 0x7f0eb01593d0 2024-07-17T08:37:41.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.451+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb01f7b00 con 0x7f0eb01593d0 2024-07-17T08:37:41.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.451+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb01f80c0 con 0x7f0eb01593d0 2024-07-17T08:37:41.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.452+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb00ac480 con 0x7f0eb01593d0 2024-07-17T08:37:41.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.453+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 154) ==== 98988+0+0 (secure 0 0 0) 0x7f0ea803b070 con 0x7f0eb01593d0 2024-07-17T08:37:41.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.454+0000 7f0ead600640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 0x7f0e90079220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.454+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7150+0+0 (secure 0 0 0) 0x7f0ea80b9720 con 0x7f0eb01593d0 2024-07-17T08:37:41.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.454+0000 7f0eb5400640 1 -- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 msgr2=0x7f0e90079220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/697221060 2024-07-17T08:37:41.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.454+0000 7f0eb5400640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 0x7f0e90079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:37:41.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.458+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0eb01f4010 con 0x7f0eb01593d0 2024-07-17T08:37:41.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.555+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 155) ==== 98540+0+0 (secure 0 0 0) 0x7f0ea807dec0 con 0x7f0eb01593d0 2024-07-17T08:37:41.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.556+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 msgr2=0x7f0e90079220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:37:41.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.556+0000 7f0ead600640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 0x7f0e90079220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.632+0000 7f3db0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.632+0000 7f3db0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.632+0000 7f3db0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.632+0000 7f3db0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.632+0000 7f3db0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.633+0000 7f3dada00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.633+0000 7f3dada00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.633+0000 7f3dada00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.633+0000 7f3dada00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.635 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:37:41.633+0000 7f3dada00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:37:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.720+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0eb00af960 con 0x7f0eb01593d0 2024-07-17T08:37:41.723 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:37:41.722+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:37:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.725+0000 7f0ead600640 1 -- 172.21.15.44:0/3093492593 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v147) ==== 161+0+0 (secure 0 0 0) 0x7f0ea8082330 con 0x7f0eb01593d0 2024-07-17T08:37:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:37:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 msgr2=0x7f0eb01553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009e20 tx=0x7f0ea8033a20 comp rx=0 tx=0).stop 2024-07-17T08:37:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 shutdown_connections 2024-07-17T08:37:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:6834/697221060,v1:172.21.15.175:6837/697221060] conn(0x7f0e90076d60 0x7f0e90079220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0eb01593d0 0x7f0eb01553f0 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0eb01f7600 0x7f0eb0072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 --2- 172.21.15.44:0/3093492593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e980051c0 0x7f0eb00b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.728+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 >> 172.21.15.44:0/3093492593 conn(0x7f0eb00b0eb0 msgr2=0x7f0eb006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.730+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 shutdown_connections 2024-07-17T08:37:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.730+0000 7f0eb7e00640 1 -- 172.21.15.44:0/3093492593 wait complete. 2024-07-17T08:37:41.743 INFO:tasks.ceph.mds.c.smithi044.stderr: -4093> 2024-07-17T08:37:20.755+0000 7fd111a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:37:41.750 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:37:41.804 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 -- 172.21.15.44:0/659726323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 msgr2=0x7f72e406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 --2- 172.21.15.44:0/659726323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e406d620 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f72e40b3a80 tx=0x7f72d402a5f0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 -- 172.21.15.44:0/659726323 shutdown_connections 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 --2- 172.21.15.44:0/659726323 >> v2:172.21.15.175:3300/0 conn(0x7f72e406db60 0x7f72e40ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 --2- 172.21.15.44:0/659726323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e406d620 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 -- 172.21.15.44:0/659726323 >> 172.21.15.44:0/659726323 conn(0x7f72e40b0eb0 msgr2=0x7f72e40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:41.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 -- 172.21.15.44:0/659726323 shutdown_connections 2024-07-17T08:37:41.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.934+0000 7f72ec200640 1 -- 172.21.15.44:0/659726323 wait complete. 2024-07-17T08:37:41.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.935+0000 7f72ec200640 1 Processor -- start 2024-07-17T08:37:41.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.936+0000 7f72ec200640 1 -- start start 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.936+0000 7f72ec200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e4154600 0x7f72e40715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72ec200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72ec200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e41f8a70 0x7f72e415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72ec200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72e415e0a0 con 0x7f72e4161120 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72ec200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72e415e220 con 0x7f72e41f8a70 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72ec200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72e415e3a0 con 0x7f72e4154600 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38856/0 (socket says 172.21.15.44:38856) 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 -- 172.21.15.44:0/2255956626 learned_addr learned my addr 172.21.15.44:0/2255956626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:41.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 -- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e4154600 msgr2=0x7f72e40715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e4154600 0x7f72e40715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 -- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e41f8a70 msgr2=0x7f72e415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:41.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e41f8a70 0x7f72e415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:41.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.937+0000 7f72e8e00640 1 -- 172.21.15.44:0/2255956626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e40748f0 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72e8e00640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f72d4034040 tx=0x7f72d40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d403e030 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72d40047a0 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4035040 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e41f7fd0 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.938+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e41f8270 con 0x7f72e4161120 2024-07-17T08:37:41.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.939+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e415e970 con 0x7f72e4161120 2024-07-17T08:37:41.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.940+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 155) ==== 98540+0+0 (secure 0 0 0) 0x7f72d4041030 con 0x7f72e4161120 2024-07-17T08:37:41.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.941+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(267..267 src has 1..267) ==== 7623+0+0 (secure 0 0 0) 0x7f72d40b8da0 con 0x7f72e4161120 2024-07-17T08:37:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:41.945+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f72d4081e50 con 0x7f72e4161120 2024-07-17T08:37:42.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.196+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f72e4071b20 con 0x7f72e4161120 2024-07-17T08:37:42.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.561+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 156) ==== 98667+0+0 (secure 0 0 0) 0x7f72d407d0c0 con 0x7f72e4161120 2024-07-17T08:37:42.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.562+0000 7f72e1600640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 0x7f72c807f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:42.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.571+0000 7f72e9800640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 0x7f72c807f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:42.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.576+0000 7f72e9800640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 0x7f72c807f660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000f60 tx=0x7f72d80079b0 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.729+0000 7f72e1600640 1 -- 172.21.15.44:0/2255956626 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v148) ==== 104+0+0 (secure 0 0 0) 0x7f72d40b2800 con 0x7f72e4161120 2024-07-17T08:37:42.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.733+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 msgr2=0x7f72c807f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.733+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 0x7f72c807f660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000f60 tx=0x7f72d80079b0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.733+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 msgr2=0x7f72e415b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.733+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f72d4034040 tx=0x7f72d40039e0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 shutdown_connections 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f72c807d270 0x7f72c807f660 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e41f8a70 0x7f72e415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e4161120 0x7f72e415b900 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 --2- 172.21.15.44:0/2255956626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e4154600 0x7f72e40715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 >> 172.21.15.44:0/2255956626 conn(0x7f72e40b0eb0 msgr2=0x7f72e41606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 shutdown_connections 2024-07-17T08:37:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.734+0000 7f72ec200640 1 -- 172.21.15.44:0/2255956626 wait complete. 2024-07-17T08:37:42.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 -- 172.21.15.44:0/3134187758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554015c360 msgr2=0x7f554006e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 --2- 172.21.15.44:0/3134187758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554015c360 0x7f554006e730 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f552802a670 comp rx=0 tx=0).stop 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 -- 172.21.15.44:0/3134187758 shutdown_connections 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 --2- 172.21.15.44:0/3134187758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554015c360 0x7f554006e730 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 -- 172.21.15.44:0/3134187758 >> 172.21.15.44:0/3134187758 conn(0x7f55400b2160 msgr2=0x7f55400b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 -- 172.21.15.44:0/3134187758 shutdown_connections 2024-07-17T08:37:42.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.966+0000 7f5546000640 1 -- 172.21.15.44:0/3134187758 wait complete. 2024-07-17T08:37:42.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.968+0000 7f5546000640 1 Processor -- start 2024-07-17T08:37:42.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.968+0000 7f5546000640 1 -- start start 2024-07-17T08:37:42.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55400cf630 0x7f55400b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f554015c360 0x7f55401405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5540144ab0 con 0x7f554006bf40 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f554015c780 con 0x7f554015c360 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f5546000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5540144c30 con 0x7f55400cf630 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38874/0 (socket says 172.21.15.44:38874) 2024-07-17T08:37:42.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 -- 172.21.15.44:0/1317768287 learned_addr learned my addr 172.21.15.44:0/1317768287 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:42.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553ea00640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55400cf630 0x7f55400b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553fe00640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f554015c360 0x7f55401405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 -- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55400cf630 msgr2=0x7f55400b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55400cf630 0x7f55400b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 -- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f554015c360 msgr2=0x7f55401405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f554015c360 0x7f55401405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.969+0000 7f553f400640 1 -- 172.21.15.44:0/1317768287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540140fd0 con 0x7f554006bf40 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f553f400640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f553000c870 tx=0x7f553000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:42.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530003b70 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5530003d10 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553000aab0 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55401412b0 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.970+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55400b3840 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.971+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554006e730 con 0x7f554006bf40 2024-07-17T08:37:42.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.972+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 156) ==== 98667+0+0 (secure 0 0 0) 0x7f553000ac50 con 0x7f554006bf40 2024-07-17T08:37:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.973+0000 7f5537400640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 0x7f551c078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.974+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(267..267 src has 1..267) ==== 7623+0+0 (secure 0 0 0) 0x7f5530098750 con 0x7f554006bf40 2024-07-17T08:37:42.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.974+0000 7f553ea00640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 0x7f551c078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:42.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.975+0000 7f553ea00640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 0x7f551c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5528033000 tx=0x7f5528004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:42.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:42.978+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55400cf170 con 0x7f554006bf40 2024-07-17T08:37:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.186+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f55400af250 con 0x7f554006bf40 2024-07-17T08:37:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.746+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 157) ==== 98875+0+0 (secure 0 0 0) 0x7f553005d220 con 0x7f554006bf40 2024-07-17T08:37:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.753+0000 7f5537400640 1 -- 172.21.15.44:0/1317768287 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v268) ==== 181+0+0 (secure 0 0 0) 0x7f5530061690 con 0x7f554006bf40 2024-07-17T08:37:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:37:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.757+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 msgr2=0x7f551c078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.757+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 0x7f551c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5528033000 tx=0x7f5528004730 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.757+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 msgr2=0x7f55400b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.757+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f553000c870 tx=0x7f553000cd30 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 shutdown_connections 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f551c076a00 0x7f551c078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f554015c360 0x7f55401405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55400cf630 0x7f55400b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 --2- 172.21.15.44:0/1317768287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554006bf40 0x7f55400b4040 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.758+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 >> 172.21.15.44:0/1317768287 conn(0x7f55400b2160 msgr2=0x7f554015d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:43.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.759+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 shutdown_connections 2024-07-17T08:37:43.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:43.760+0000 7f5546000640 1 -- 172.21.15.44:0/1317768287 wait complete. 2024-07-17T08:37:43.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:37:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.000+0000 7f2c96a00640 1 -- 172.21.15.44:0/1050416942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 msgr2=0x7f2c900ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.000+0000 7f2c96a00640 1 --2- 172.21.15.44:0/1050416942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c900ad250 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005fd0 tx=0x7f2c8402a520 comp rx=0 tx=0).stop 2024-07-17T08:37:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 -- 172.21.15.44:0/1050416942 shutdown_connections 2024-07-17T08:37:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 --2- 172.21.15.44:0/1050416942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ae490 0x7f2c900729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 --2- 172.21.15.44:0/1050416942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c900ad250 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 -- 172.21.15.44:0/1050416942 >> 172.21.15.44:0/1050416942 conn(0x7f2c900cf050 msgr2=0x7f2c900cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 -- 172.21.15.44:0/1050416942 shutdown_connections 2024-07-17T08:37:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.001+0000 7f2c96a00640 1 -- 172.21.15.44:0/1050416942 wait complete. 2024-07-17T08:37:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.002+0000 7f2c96a00640 1 Processor -- start 2024-07-17T08:37:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.003+0000 7f2c96a00640 1 -- start start 2024-07-17T08:37:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ad790 0x7f2c9015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c900ae490 0x7f2c901f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c900732c0 con 0x7f2c90155860 2024-07-17T08:37:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90073140 con 0x7f2c900ae490 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c90073440 con 0x7f2c900ad790 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38902/0 (socket says 172.21.15.44:38902) 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 -- 172.21.15.44:0/2359190391 learned_addr learned my addr 172.21.15.44:0/2359190391 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c95600640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ad790 0x7f2c9015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c94c00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c900ae490 0x7f2c901f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 -- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ad790 msgr2=0x7f2c9015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ad790 0x7f2c9015a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 -- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c900ae490 msgr2=0x7f2c901f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c900ae490 0x7f2c901f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.004+0000 7f2c96000640 1 -- 172.21.15.44:0/2359190391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c900b2310 con 0x7f2c90155860 2024-07-17T08:37:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c96000640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9e0 tx=0x7f2c8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88019070 con 0x7f2c90155860 2024-07-17T08:37:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8800cc30 con 0x7f2c90155860 2024-07-17T08:37:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880146a0 con 0x7f2c90155860 2024-07-17T08:37:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c900b25f0 con 0x7f2c90155860 2024-07-17T08:37:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.005+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c900b2bb0 con 0x7f2c90155860 2024-07-17T08:37:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.006+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c90072990 con 0x7f2c90155860 2024-07-17T08:37:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.007+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 157) ==== 98875+0+0 (secure 0 0 0) 0x7f2c8801c070 con 0x7f2c90155860 2024-07-17T08:37:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.008+0000 7f2c8d600640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.009+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(268..268 src has 1..268) ==== 7185+0+0 (secure 0 0 0) 0x7f2c88098790 con 0x7f2c90155860 2024-07-17T08:37:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.009+0000 7f2c95600640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.010+0000 7f2c95600640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 0x7f2c70079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c84033000 tx=0x7f2c84004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.013+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2c8800ce20 con 0x7f2c90155860 2024-07-17T08:37:44.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.226+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c90154d40 con 0x7f2c90155860 2024-07-17T08:37:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.757+0000 7f2c8d600640 1 -- 172.21.15.44:0/2359190391 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v269) ==== 169+0+0 (secure 0 0 0) 0x7f2c880615a0 con 0x7f2c90155860 2024-07-17T08:37:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 msgr2=0x7f2c70079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 0x7f2c70079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2c84033000 tx=0x7f2c84004730 comp rx=0 tx=0).stop 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 msgr2=0x7f2c901f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9e0 tx=0x7f2c8800bea0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 shutdown_connections 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2c70076c40 0x7f2c70079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c90155860 0x7f2c901f6ce0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c900ae490 0x7f2c901f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 --2- 172.21.15.44:0/2359190391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c900ad790 0x7f2c9015a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.761+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 >> 172.21.15.44:0/2359190391 conn(0x7f2c900cf050 msgr2=0x7f2c9015dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.764+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 shutdown_connections 2024-07-17T08:37:44.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.767+0000 7f2c96a00640 1 -- 172.21.15.44:0/2359190391 wait complete. 2024-07-17T08:37:44.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.986+0000 7ff889c00640 1 -- 172.21.15.44:0/2829000475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 msgr2=0x7ff8840ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 --2- 172.21.15.44:0/2829000475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840ac810 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7ff8840708b0 tx=0x7ff87c02aa10 comp rx=0 tx=0).stop 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 -- 172.21.15.44:0/2829000475 shutdown_connections 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 --2- 172.21.15.44:0/2829000475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 0x7ff8840ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 --2- 172.21.15.44:0/2829000475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840ac810 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 --2- 172.21.15.44:0/2829000475 >> v2:172.21.15.175:3300/0 conn(0x7ff88406eb90 0x7ff88415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 -- 172.21.15.44:0/2829000475 >> 172.21.15.44:0/2829000475 conn(0x7ff8840b0eb0 msgr2=0x7ff8840b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:44.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 -- 172.21.15.44:0/2829000475 shutdown_connections 2024-07-17T08:37:44.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.987+0000 7ff889c00640 1 -- 172.21.15.44:0/2829000475 wait complete. 2024-07-17T08:37:44.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.988+0000 7ff889c00640 1 Processor -- start 2024-07-17T08:37:44.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.989+0000 7ff889c00640 1 -- start start 2024-07-17T08:37:44.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.989+0000 7ff889c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff88406eb90 0x7ff884165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff889c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff889c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 0x7ff8841464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff889c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff88406f850 con 0x7ff8840ac3a0 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff889c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff88406f6d0 con 0x7ff88406eb90 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff889c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff88414fd60 con 0x7ff8840b3280 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38920/0 (socket says 172.21.15.44:38920) 2024-07-17T08:37:44.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 -- 172.21.15.44:0/920831311 learned_addr learned my addr 172.21.15.44:0/920831311 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff883400640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 0x7ff8841464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff882a00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff88406eb90 0x7ff884165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 -- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 msgr2=0x7ff8841464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 0x7ff8841464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 -- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff88406eb90 msgr2=0x7ff884165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:44.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff88406eb90 0x7ff884165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.990+0000 7ff87a000640 1 -- 172.21.15.44:0/920831311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff884148e80 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff87a000640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00b410 tx=0x7ff86c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c015020 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff86c011940 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c011d90 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff884149160 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.991+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff884149720 con 0x7ff8840ac3a0 2024-07-17T08:37:44.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.993+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8840ac810 con 0x7ff8840ac3a0 2024-07-17T08:37:44.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.994+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 157) ==== 98875+0+0 (secure 0 0 0) 0x7ff86c018020 con 0x7ff8840ac3a0 2024-07-17T08:37:44.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.995+0000 7ff87b400640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 0x7ff860079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:44.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.995+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7ff86c098700 con 0x7ff8840ac3a0 2024-07-17T08:37:44.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.996+0000 7ff882a00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 0x7ff860079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:44.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.997+0000 7ff882a00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 0x7ff860079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff874009890 tx=0x7ff874008040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:45.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:44.999+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff86c01f020 con 0x7ff8840ac3a0 2024-07-17T08:37:45.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.212+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ff8841498f0 con 0x7ff8840ac3a0 2024-07-17T08:37:45.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.219+0000 7ff87b400640 1 -- 172.21.15.44:0/920831311 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v83) ==== 71+0+0 (secure 0 0 0) 0x7ff86c0617b0 con 0x7ff8840ac3a0 2024-07-17T08:37:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 msgr2=0x7ff860079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 0x7ff860079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff874009890 tx=0x7ff874008040 comp rx=0 tx=0).stop 2024-07-17T08:37:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 msgr2=0x7ff8840cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00b410 tx=0x7ff86c00b8d0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 shutdown_connections 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ff860076c40 0x7ff860079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8840b3280 0x7ff8841464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8840ac3a0 0x7ff8840cf410 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 --2- 172.21.15.44:0/920831311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff88406eb90 0x7ff884165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 >> 172.21.15.44:0/920831311 conn(0x7ff8840b0eb0 msgr2=0x7ff88415bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.222+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 shutdown_connections 2024-07-17T08:37:45.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.223+0000 7ff889c00640 1 -- 172.21.15.44:0/920831311 wait complete. 2024-07-17T08:37:45.244 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-07-17T08:37:45.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 -- 172.21.15.44:0/2311472238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08140070 msgr2=0x7f6d0815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 --2- 172.21.15.44:0/2311472238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08140070 0x7f6d0815f2d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005fd0 tx=0x7f6cf802aa90 comp rx=0 tx=0).stop 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 -- 172.21.15.44:0/2311472238 shutdown_connections 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 --2- 172.21.15.44:0/2311472238 >> v2:172.21.15.175:3300/0 conn(0x7f6d080714f0 0x7f6d080cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 --2- 172.21.15.44:0/2311472238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08140070 0x7f6d0815f2d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 -- 172.21.15.44:0/2311472238 >> 172.21.15.44:0/2311472238 conn(0x7f6d080b3090 msgr2=0x7f6d080b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.454+0000 7f6d07200640 1 -- 172.21.15.44:0/2311472238 shutdown_connections 2024-07-17T08:37:45.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.455+0000 7f6d07200640 1 -- 172.21.15.44:0/2311472238 wait complete. 2024-07-17T08:37:45.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.457+0000 7f6d07200640 1 Processor -- start 2024-07-17T08:37:45.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.457+0000 7f6d07200640 1 -- start start 2024-07-17T08:37:45.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 0x7f6d081530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 0x7f6d08151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080d0800 con 0x7f6d0815f810 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d080d0680 con 0x7f6d080714f0 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.458+0000 7f6d07200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d080d0980 con 0x7f6d08155290 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38930/0 (socket says 172.21.15.44:38930) 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 -- 172.21.15.44:0/1738728509 learned_addr learned my addr 172.21.15.44:0/1738728509 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05e00640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 0x7f6d081530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:45.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d06800640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 0x7f6d08151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:45.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 -- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 msgr2=0x7f6d08151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:45.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 0x7f6d08151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 -- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 msgr2=0x7f6d081530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 0x7f6d081530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05400640 1 -- 172.21.15.44:0/1738728509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d08155740 con 0x7f6d0815f810 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d05e00640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 0x7f6d081530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.459+0000 7f6d06800640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 0x7f6d08151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6d05400640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c3a0 tx=0x7f6cf400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:45.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4017070 con 0x7f6d0815f810 2024-07-17T08:37:45.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cf40053c0 con 0x7f6d0815f810 2024-07-17T08:37:45.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4003850 con 0x7f6d0815f810 2024-07-17T08:37:45.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d08153750 con 0x7f6d0815f810 2024-07-17T08:37:45.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.460+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d08153d10 con 0x7f6d0815f810 2024-07-17T08:37:45.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.462+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d08070590 con 0x7f6d0815f810 2024-07-17T08:37:45.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.463+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 157) ==== 98875+0+0 (secure 0 0 0) 0x7f6cf401a070 con 0x7f6d0815f810 2024-07-17T08:37:45.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.464+0000 7f6cfe000640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 0x7f6ce8079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:45.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.464+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f6cf4097bf0 con 0x7f6d0815f810 2024-07-17T08:37:45.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.464+0000 7f6d05e00640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 0x7f6ce8079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:45.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.466+0000 7f6d05e00640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 0x7f6ce8079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8035000 tx=0x7f6cf8039040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:45.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.468+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6cf4060bb0 con 0x7f6d0815f810 2024-07-17T08:37:45.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:45.757+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f6d080d0ea0 con 0x7f6d0815f810 2024-07-17T08:37:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.241+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f6cf405c740 con 0x7f6d0815f810 2024-07-17T08:37:46.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.763+0000 7f6cfe000640 1 -- 172.21.15.44:0/1738728509 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3403) ==== 155+0+0 (secure 0 0 0) 0x7f6cf4067210 con 0x7f6d0815f810 2024-07-17T08:37:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 msgr2=0x7f6ce8079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 0x7f6ce8079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8035000 tx=0x7f6cf8039040 comp rx=0 tx=0).stop 2024-07-17T08:37:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 msgr2=0x7f6d081547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c3a0 tx=0x7f6cf400c860 comp rx=0 tx=0).stop 2024-07-17T08:37:46.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 shutdown_connections 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6ce8076c40 0x7f6ce8079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d08155290 0x7f6d08151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d0815f810 0x7f6d081547f0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 --2- 172.21.15.44:0/1738728509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d080714f0 0x7f6d081530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 >> 172.21.15.44:0/1738728509 conn(0x7f6d080b3090 msgr2=0x7f6d0815b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.767+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 shutdown_connections 2024-07-17T08:37:46.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:46.768+0000 7f6d07200640 1 -- 172.21.15.44:0/1738728509 wait complete. 2024-07-17T08:37:46.793 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2024-07-17T08:37:46.793 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_01 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule creation succeeds for default subvolgroup. 2024-07-17T08:37:46.793 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_01 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:37:46.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 -- 172.21.15.44:0/1207123432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 msgr2=0x7f55d4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1207123432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55d4003e10 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f55e0001e30 tx=0x7f55d4005830 comp rx=0 tx=0).stop 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55ef400640 1 -- 172.21.15.44:0/1207123432 reap_dead start 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 -- 172.21.15.44:0/1207123432 shutdown_connections 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 -- 172.21.15.44:0/1207123432 >> 172.21.15.44:0/1207123432 conn(0x7f55f00b0fb0 msgr2=0x7f55f00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 -- 172.21.15.44:0/1207123432 shutdown_connections 2024-07-17T08:37:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.011+0000 7f55f5a00640 1 -- 172.21.15.44:0/1207123432 wait complete. 2024-07-17T08:37:47.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.013+0000 7f55f5a00640 1 Processor -- start 2024-07-17T08:37:47.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.013+0000 7f55f5a00640 1 -- start start 2024-07-17T08:37:47.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b3cc0 0x7f55f01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 0x7f55f015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00cdda0 con 0x7f55d4003a30 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00adf50 con 0x7f55f014a700 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55f5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55f00cdf20 con 0x7f55f00b3cc0 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55eea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38946/0 (socket says 172.21.15.44:38946) 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.014+0000 7f55eea00640 1 -- 172.21.15.44:0/1047680203 learned_addr learned my addr 172.21.15.44:0/1047680203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55ef400640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 0x7f55f015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55ee000640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b3cc0 0x7f55f01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 -- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b3cc0 msgr2=0x7f55f01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b3cc0 0x7f55f01f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 -- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 msgr2=0x7f55f015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 0x7f55f015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 -- 172.21.15.44:0/1047680203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0152930 con 0x7f55d4003a30 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55ef400640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 0x7f55f015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55eea00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f55e0001e30 tx=0x7f55e0004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0040030 con 0x7f55d4003a30 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e00060c0 con 0x7f55d4003a30 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f01f7e40 con 0x7f55d4003a30 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.015+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0037040 con 0x7f55d4003a30 2024-07-17T08:37:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.016+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f01f8400 con 0x7f55d4003a30 2024-07-17T08:37:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.017+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f006a1a0 con 0x7f55d4003a30 2024-07-17T08:37:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.018+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f55e0043070 con 0x7f55d4003a30 2024-07-17T08:37:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.019+0000 7f55e6a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 0x7f55d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.019+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f55e00b83b0 con 0x7f55d4003a30 2024-07-17T08:37:47.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.020+0000 7f55ee000640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 0x7f55d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.021+0000 7f55ee000640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 0x7f55d0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000fd0 tx=0x7f55dc001540 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.023+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55f006a1a0 con 0x7f55d4003a30 2024-07-17T08:37:47.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.237+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55f00b0bf0 con 0x7f55d4003a30 2024-07-17T08:37:47.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.238+0000 7f55e6a00640 1 -- 172.21.15.44:0/1047680203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v269) ==== 74+0+15434 (secure 0 0 0) 0x7f55f00b0bf0 con 0x7f55d4003a30 2024-07-17T08:37:47.240 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:47.240 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":269,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:44.754781+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":261,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6834/1990487283":"2024-07-18T08:37:41.722976+0000","172.21.15.44:6835/1990487283":"2024-07-18T08:37:41.722976+0000","172.21.15.175:0/2616686791":"2024-07-18T08:37:41.535480+0000","172.21.15.175:6837/697221060":"2024-07-18T08:37:41.535480+0000","172.21.15.175:6834/697221060":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/3538471592":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/3203357073":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/1919018728":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/150520951":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/2057041105":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1342905808":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/29215155":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3567462564":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/622582395":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/3175300474":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/1970298676":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/2184874848":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6837/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:6834/1251568930":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2923266016":"2024-07-18T08:37:41.535480+0000","172.21.15.175:0/1545286674":"2024-07-18T08:37:33.175035+0000","172.21.15.175:0/2008194206":"2024-07-17T09:37:24.171533+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 msgr2=0x7f55d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 0x7f55d0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000fd0 tx=0x7f55dc001540 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 msgr2=0x7f55f01f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f55e0001e30 tx=0x7f55e0004ae0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 shutdown_connections 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f55d0076cd0 0x7f55d0079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55f014a700 0x7f55f015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55f00b3cc0 0x7f55f01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 --2- 172.21.15.44:0/1047680203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d4003a30 0x7f55f01f51c0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 >> 172.21.15.44:0/1047680203 conn(0x7f55f00b0fb0 msgr2=0x7f55f00af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:47.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 shutdown_connections 2024-07-17T08:37:47.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.241+0000 7f55f5a00640 1 -- 172.21.15.44:0/1047680203 wait complete. 2024-07-17T08:37:47.263 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:37:47.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:37:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.480+0000 7fc063600640 1 -- 172.21.15.44:0/2094545423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 msgr2=0x7fc040003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.480+0000 7fc063600640 1 --2- 172.21.15.44:0/2094545423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc040003e10 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009b50 tx=0x7fc040005830 comp rx=0 tx=0).stop 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.480+0000 7fc061600640 1 -- 172.21.15.44:0/2094545423 reap_dead start 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.480+0000 7fc063600640 1 -- 172.21.15.44:0/2094545423 shutdown_connections 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.480+0000 7fc063600640 1 -- 172.21.15.44:0/2094545423 >> 172.21.15.44:0/2094545423 conn(0x7fc05c073d50 msgr2=0x7fc05c09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.481+0000 7fc063600640 1 -- 172.21.15.44:0/2094545423 shutdown_connections 2024-07-17T08:37:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.481+0000 7fc063600640 1 -- 172.21.15.44:0/2094545423 wait complete. 2024-07-17T08:37:47.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.482+0000 7fc063600640 1 Processor -- start 2024-07-17T08:37:47.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.483+0000 7fc063600640 1 -- start start 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.483+0000 7fc063600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.483+0000 7fc063600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc05c13f7e0 0x7fc05c14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc063600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc05c06fe50 0x7fc05c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc063600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c0b6560 con 0x7fc040003a30 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc063600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c0b63e0 con 0x7fc05c06fe50 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc063600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc05c0aeb80 con 0x7fc05c13f7e0 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38956/0 (socket says 172.21.15.44:38956) 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 -- 172.21.15.44:0/419353817 learned_addr learned my addr 172.21.15.44:0/419353817 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc053e00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc05c13f7e0 0x7fc05c14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 -- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc05c13f7e0 msgr2=0x7fc05c14ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc05c13f7e0 0x7fc05c14ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 -- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc05c06fe50 msgr2=0x7fc05c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc05c06fe50 0x7fc05c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.484+0000 7fc060c00640 1 -- 172.21.15.44:0/419353817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c1f3ae0 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc060c00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02b3f0 tx=0x7fc04c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c043070 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc04c004140 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c1f3dc0 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c038040 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.485+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c1f4380 con 0x7fc040003a30 2024-07-17T08:37:47.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.486+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c0af110 con 0x7fc040003a30 2024-07-17T08:37:47.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.487+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fc04c046070 con 0x7fc040003a30 2024-07-17T08:37:47.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.488+0000 7fc059600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 0x7fc038079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.488+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7fc04c0b8c50 con 0x7fc040003a30 2024-07-17T08:37:47.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.489+0000 7fc053e00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 0x7fc038079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.490+0000 7fc053e00640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 0x7fc038079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0b1630 tx=0x7fc044006950 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.493+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc04c081b40 con 0x7fc040003a30 2024-07-17T08:37:47.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.708+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc05c157170 con 0x7fc040003a30 2024-07-17T08:37:47.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.709+0000 7fc059600640 1 -- 172.21.15.44:0/419353817 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v269) ==== 93+0+10 (secure 0 0 0) 0x7fc04c081440 con 0x7fc040003a30 2024-07-17T08:37:47.711 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:37:47.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 msgr2=0x7fc038079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 0x7fc038079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0b1630 tx=0x7fc044006950 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 msgr2=0x7fc05c151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02b3f0 tx=0x7fc04c033670 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 shutdown_connections 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc038076cd0 0x7fc038079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc05c06fe50 0x7fc05c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc05c13f7e0 0x7fc05c14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 --2- 172.21.15.44:0/419353817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc040003a30 0x7fc05c151a40 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.712+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 >> 172.21.15.44:0/419353817 conn(0x7fc05c073d50 msgr2=0x7fc05c159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.713+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 shutdown_connections 2024-07-17T08:37:47.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.713+0000 7fc063600640 1 -- 172.21.15.44:0/419353817 wait complete. 2024-07-17T08:37:47.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01' 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.939+0000 7fbf4a200640 1 -- 172.21.15.44:0/2078679057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf44140570 msgr2=0x7fbf44140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.939+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2078679057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf44140570 0x7fbf44140a00 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005fd0 tx=0x7fbf3c02a8b0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 -- 172.21.15.44:0/2078679057 shutdown_connections 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2078679057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf44140570 0x7fbf44140a00 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2078679057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf920 0x7fbf44140030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 -- 172.21.15.44:0/2078679057 >> 172.21.15.44:0/2078679057 conn(0x7fbf440b0e80 msgr2=0x7fbf440b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 -- 172.21.15.44:0/2078679057 shutdown_connections 2024-07-17T08:37:47.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.940+0000 7fbf4a200640 1 -- 172.21.15.44:0/2078679057 wait complete. 2024-07-17T08:37:47.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.942+0000 7fbf4a200640 1 Processor -- start 2024-07-17T08:37:47.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.942+0000 7fbf4a200640 1 -- start start 2024-07-17T08:37:47.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf540 0x7fbf4406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf44140570 0x7fbf44156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440aea10 con 0x7fbf440cf920 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440ac270 con 0x7fbf44140570 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf4a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf44140f40 con 0x7fbf440cf540 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38978/0 (socket says 172.21.15.44:38978) 2024-07-17T08:37:47.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf43e00640 1 -- 172.21.15.44:0/2728320007 learned_addr learned my addr 172.21.15.44:0/2728320007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf49800640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf44140570 0x7fbf44156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.943+0000 7fbf48e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf540 0x7fbf4406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 -- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf540 msgr2=0x7fbf4406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf540 0x7fbf4406d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 -- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf44140570 msgr2=0x7fbf44156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf44140570 0x7fbf44156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 -- 172.21.15.44:0/2728320007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4406db70 con 0x7fbf440cf920 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf43e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d950 tx=0x7fbf3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf440cf920 2024-07-17T08:37:47.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf30009d20 con 0x7fbf440cf920 2024-07-17T08:37:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf300106c0 con 0x7fbf440cf920 2024-07-17T08:37:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf44157100 con 0x7fbf440cf920 2024-07-17T08:37:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.944+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf44072940 con 0x7fbf440cf920 2024-07-17T08:37:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.945+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf441413f0 con 0x7fbf440cf920 2024-07-17T08:37:47.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.947+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fbf30010860 con 0x7fbf440cf920 2024-07-17T08:37:47.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.948+0000 7fbf40c00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 0x7fbf24079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.948+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7fbf30098d50 con 0x7fbf440cf920 2024-07-17T08:37:47.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.948+0000 7fbf48e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 0x7fbf24079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:47.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.950+0000 7fbf48e00640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 0x7fbf24079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf38007fd0 tx=0x7fbf38009040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:47.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:47.952+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbf30061bc0 con 0x7fbf440cf920 2024-07-17T08:37:48.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.239+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01"]} v 0) -- 0x7fbf440aefc0 con 0x7fbf440cf920 2024-07-17T08:37:48.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.778+0000 7fbf40c00640 1 -- 172.21.15.44:0/2728320007 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01"]}]=0 v3405) ==== 204+0+0 (secure 0 0 0) 0x7fbf300614c0 con 0x7fbf440cf920 2024-07-17T08:37:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 msgr2=0x7fbf24079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 0x7fbf24079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf38007fd0 tx=0x7fbf38009040 comp rx=0 tx=0).stop 2024-07-17T08:37:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 msgr2=0x7fbf440cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d950 tx=0x7fbf3000de10 comp rx=0 tx=0).stop 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 shutdown_connections 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fbf24076cd0 0x7fbf24079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf44140570 0x7fbf44156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf440cf920 0x7fbf440cd7f0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.781+0000 7fbf4a200640 1 --2- 172.21.15.44:0/2728320007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf440cf540 0x7fbf4406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.782+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 >> 172.21.15.44:0/2728320007 conn(0x7fbf440b0e80 msgr2=0x7fbf4409ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.782+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 shutdown_connections 2024-07-17T08:37:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:48.782+0000 7fbf4a200640 1 -- 172.21.15.44:0/2728320007 wait complete. 2024-07-17T08:37:48.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:37:49.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.016+0000 7f3a5f200640 1 -- 172.21.15.44:0/3579082177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 msgr2=0x7f3a6006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.016+0000 7f3a5f200640 1 --2- 172.21.15.44:0/3579082177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6006e570 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f3a6006f7c0 tx=0x7f3a4802aa10 comp rx=0 tx=0).stop 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 -- 172.21.15.44:0/3579082177 shutdown_connections 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 --2- 172.21.15.44:0/3579082177 >> v2:172.21.15.175:3300/0 conn(0x7f3a6006eab0 0x7f3a600d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 --2- 172.21.15.44:0/3579082177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6006e570 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 --2- 172.21.15.44:0/3579082177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a6015e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 -- 172.21.15.44:0/3579082177 >> 172.21.15.44:0/3579082177 conn(0x7f3a60073920 msgr2=0x7f3a60073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 -- 172.21.15.44:0/3579082177 shutdown_connections 2024-07-17T08:37:49.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.017+0000 7f3a5f200640 1 -- 172.21.15.44:0/3579082177 wait complete. 2024-07-17T08:37:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.018+0000 7f3a5f200640 1 Processor -- start 2024-07-17T08:37:49.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.019+0000 7f3a5f200640 1 -- start start 2024-07-17T08:37:49.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.019+0000 7f3a5f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a6006eab0 0x7f3a60151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a60152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a600d05b0 con 0x7f3a600cdcb0 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60070650 con 0x7f3a6006eab0 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a600d0730 con 0x7f3a60157d50 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49092/0 (socket says 172.21.15.44:49092) 2024-07-17T08:37:49.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 -- 172.21.15.44:0/2535117570 learned_addr learned my addr 172.21.15.44:0/2535117570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:49.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5de00640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a6006eab0 0x7f3a60151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5e800640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a60152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 -- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 msgr2=0x7f3a60152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a60152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 -- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a6006eab0 msgr2=0x7f3a60151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a6006eab0 0x7f3a60151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5d400640 1 -- 172.21.15.44:0/2535117570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a600cd4f0 con 0x7f3a600cdcb0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.020+0000 7f3a5e800640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a60152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a5d400640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009bf0 tx=0x7f3a48003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48047070 con 0x7f3a600cdcb0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a48004250 con 0x7f3a600cdcb0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a601f59d0 con 0x7f3a600cdcb0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48033050 con 0x7f3a600cdcb0 2024-07-17T08:37:49.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.021+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a601f5f90 con 0x7f3a600cdcb0 2024-07-17T08:37:49.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.022+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a600ce670 con 0x7f3a600cdcb0 2024-07-17T08:37:49.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.024+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f3a4804a070 con 0x7f3a600cdcb0 2024-07-17T08:37:49.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.025+0000 7f3a56000640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 0x7f3a3c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.025+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f3a480b9da0 con 0x7f3a600cdcb0 2024-07-17T08:37:49.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.026+0000 7f3a5de00640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 0x7f3a3c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.027+0000 7f3a5de00640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 0x7f3a3c0791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000b810 tx=0x7f3a50008040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:49.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.030+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3a600ce670 con 0x7f3a600cdcb0 2024-07-17T08:37:49.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.247+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f3a600d0cc0 con 0x7f3a600cdcb0 2024-07-17T08:37:49.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.248+0000 7f3a56000640 1 -- 172.21.15.44:0/2535117570 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v83) ==== 59+0+1998 (secure 0 0 0) 0x7f3a48082b90 con 0x7f3a600cdcb0 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:37:49.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:37:49.251 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:37:49.252 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 msgr2=0x7f3a3c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 0x7f3a3c0791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000b810 tx=0x7f3a50008040 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 msgr2=0x7f3a6014a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009bf0 tx=0x7f3a48003f10 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 shutdown_connections 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f3a3c076d00 0x7f3a3c0791c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a60157d50 0x7f3a60152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a600cdcb0 0x7f3a6014a730 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 --2- 172.21.15.44:0/2535117570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a6006eab0 0x7f3a60151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.250+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 >> 172.21.15.44:0/2535117570 conn(0x7f3a60073920 msgr2=0x7f3a600b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:49.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.251+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 shutdown_connections 2024-07-17T08:37:49.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.251+0000 7f3a5f200640 1 -- 172.21.15.44:0/2535117570 wait complete. 2024-07-17T08:37:49.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:37:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 -- 172.21.15.44:0/1656565654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09101512b0 msgr2=0x7f091015b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 --2- 172.21.15.44:0/1656565654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09101512b0 0x7f091015b040 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f0910154c30 tx=0x7f08f802a4f0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 -- 172.21.15.44:0/1656565654 shutdown_connections 2024-07-17T08:37:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 --2- 172.21.15.44:0/1656565654 >> v2:172.21.15.175:3300/0 conn(0x7f091006edc0 0x7f091006f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 --2- 172.21.15.44:0/1656565654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09101512b0 0x7f091015b040 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 --2- 172.21.15.44:0/1656565654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0910150ed0 0x7f09100ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 -- 172.21.15.44:0/1656565654 >> 172.21.15.44:0/1656565654 conn(0x7f09100cf0d0 msgr2=0x7f09100cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 -- 172.21.15.44:0/1656565654 shutdown_connections 2024-07-17T08:37:49.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.481+0000 7f090fe00640 1 -- 172.21.15.44:0/1656565654 wait complete. 2024-07-17T08:37:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.483+0000 7f090fe00640 1 Processor -- start 2024-07-17T08:37:49.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.483+0000 7f090fe00640 1 -- start start 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f091006edc0 0x7f09101505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09101512b0 0x7f091006dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f091015df20 con 0x7f0910150ed0 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f091015dda0 con 0x7f091006edc0 2024-07-17T08:37:49.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.484+0000 7f090fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f091015e0a0 con 0x7f09101512b0 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49114/0 (socket says 172.21.15.44:49114) 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 -- 172.21.15.44:0/2564662418 learned_addr learned my addr 172.21.15.44:0/2564662418 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090ea00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f091006edc0 0x7f09101505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090f400640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09101512b0 0x7f091006dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 -- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09101512b0 msgr2=0x7f091006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09101512b0 0x7f091006dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 -- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f091006edc0 msgr2=0x7f09101505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f091006edc0 0x7f09101505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.485+0000 7f090e000640 1 -- 172.21.15.44:0/2564662418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09100cc740 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f090e000640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f803f070 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08f8004890 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09100cca20 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8035030 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.486+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09100ccfe0 con 0x7f0910150ed0 2024-07-17T08:37:49.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.487+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09100ac290 con 0x7f0910150ed0 2024-07-17T08:37:49.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.489+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f08f8042070 con 0x7f0910150ed0 2024-07-17T08:37:49.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.490+0000 7f0906a00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 0x7f08ec079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:49.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.490+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f08f80b9840 con 0x7f0910150ed0 2024-07-17T08:37:49.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.490+0000 7f090ea00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 0x7f08ec079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:49.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.491+0000 7f090ea00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 0x7f08ec079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0900005c40 tx=0x7f090000a590 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:49.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.494+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f08f8081fe0 con 0x7f0910150ed0 2024-07-17T08:37:49.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.703+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f091006d520 con 0x7f0910150ed0 2024-07-17T08:37:49.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.709+0000 7f0906a00640 1 -- 172.21.15.44:0/2564662418 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v84)=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f08f8086530 con 0x7f0910150ed0 2024-07-17T08:37:49.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 msgr2=0x7f08ec079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 0x7f08ec079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0900005c40 tx=0x7f090000a590 comp rx=0 tx=0).stop 2024-07-17T08:37:49.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 msgr2=0x7f09100ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f80039e0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 shutdown_connections 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f08ec076cd0 0x7f08ec079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09101512b0 0x7f091006dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0910150ed0 0x7f09100ca8b0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 --2- 172.21.15.44:0/2564662418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f091006edc0 0x7f09101505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.712+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 >> 172.21.15.44:0/2564662418 conn(0x7f09100cf0d0 msgr2=0x7f09100ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.713+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 shutdown_connections 2024-07-17T08:37:49.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:49.713+0000 7f090fe00640 1 -- 172.21.15.44:0/2564662418 wait complete. 2024-07-17T08:37:49.736 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:37:49.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:49.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:37:49.979 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:37:49.980 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:49.980 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:50.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:37:50.037 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:50.038 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:50.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:50.092 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:37:50.092 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:50.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:37:50.149 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:37:50.149 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:37:50.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:37:50.205 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:37:50.205 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 -- 172.21.15.44:0/4172994032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6458153a50 msgr2=0x7f6458159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 --2- 172.21.15.44:0/4172994032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6458153a50 0x7f6458159b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6448005fd0 tx=0x7f644802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 -- 172.21.15.44:0/4172994032 shutdown_connections 2024-07-17T08:37:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 --2- 172.21.15.44:0/4172994032 >> v2:172.21.15.175:3300/0 conn(0x7f645815a040 0x7f64580ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 --2- 172.21.15.44:0/4172994032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6458153a50 0x7f6458159b00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 -- 172.21.15.44:0/4172994032 >> 172.21.15.44:0/4172994032 conn(0x7f64580b0fb0 msgr2=0x7f64580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 -- 172.21.15.44:0/4172994032 shutdown_connections 2024-07-17T08:37:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.428+0000 7f645ec00640 1 -- 172.21.15.44:0/4172994032 wait complete. 2024-07-17T08:37:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.431+0000 7f645ec00640 1 Processor -- start 2024-07-17T08:37:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.431+0000 7f645ec00640 1 -- start start 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64580ac920 0x7f6458157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645815a040 0x7f64581508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6458153fb0 con 0x7f6458153a50 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6458153e30 con 0x7f64580ac920 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f645ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f645806f710 con 0x7f645815a040 2024-07-17T08:37:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f644fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f644fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49122/0 (socket says 172.21.15.44:49122) 2024-07-17T08:37:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.432+0000 7f644fe00640 1 -- 172.21.15.44:0/2195733783 learned_addr learned my addr 172.21.15.44:0/2195733783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f6457e00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64580ac920 0x7f6458157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f645cc00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645815a040 0x7f64581508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 -- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645815a040 msgr2=0x7f64581508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645815a040 0x7f64581508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 -- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64580ac920 msgr2=0x7f6458157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64580ac920 0x7f6458157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 -- 172.21.15.44:0/2195733783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64581531b0 con 0x7f6458153a50 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.433+0000 7f644fe00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f644000ab50 tx=0x7f6440009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.434+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440018030 con 0x7f6458153a50 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.434+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6440004110 con 0x7f6458153a50 2024-07-17T08:37:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.434+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645813ee00 con 0x7f6458153a50 2024-07-17T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.434+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440014620 con 0x7f6458153a50 2024-07-17T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.434+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64580744e0 con 0x7f6458153a50 2024-07-17T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.435+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6458159b00 con 0x7f6458153a50 2024-07-17T08:37:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.436+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f644001b070 con 0x7f6458153a50 2024-07-17T08:37:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.437+0000 7f6454c00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 0x7f6434079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.437+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f6440098a80 con 0x7f6458153a50 2024-07-17T08:37:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.438+0000 7f6457e00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 0x7f6434079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.439+0000 7f6457e00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 0x7f6434079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6448033000 tx=0x7f6448004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.443+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f644009ae20 con 0x7f6458153a50 2024-07-17T08:37:50.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.697+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64581581d0 con 0x7f6458153a50 2024-07-17T08:37:50.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.698+0000 7f6454c00640 1 -- 172.21.15.44:0/2195733783 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) ==== 95+0+3918 (secure 0 0 0) 0x7f6440011070 con 0x7f6458153a50 2024-07-17T08:37:50.700 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:50.700 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":148,"btime":"2024-07-17T08:37:42:726955+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9193,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3830473222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3830473222},{"type":"v1","addr":"172.21.15.175:6801","nonce":3830473222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":140},{"gid":9199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/53209355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":53209355},{"type":"v1","addr":"172.21.15.175:6836","nonce":53209355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3173582645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3173582645},{"type":"v1","addr":"172.21.15.44:6833","nonce":3173582645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141},{"gid":11679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3832646180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":3832646180},{"type":"v1","addr":"172.21.15.44:6835","nonce":3832646180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[]} 2024-07-17T08:37:50.700 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 148 2024-07-17T08:37:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.701+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 msgr2=0x7f6434079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.701+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 0x7f6434079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6448033000 tx=0x7f6448004730 comp rx=0 tx=0).stop 2024-07-17T08:37:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.701+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 msgr2=0x7f64581f1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:50.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.701+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f644000ab50 tx=0x7f6440009a30 comp rx=0 tx=0).stop 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 shutdown_connections 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6434076cd0 0x7f6434079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f645815a040 0x7f64581508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6458153a50 0x7f64581f1320 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 --2- 172.21.15.44:0/2195733783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64580ac920 0x7f6458157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 >> 172.21.15.44:0/2195733783 conn(0x7f64580b0fb0 msgr2=0x7f64580b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 shutdown_connections 2024-07-17T08:37:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.702+0000 7f645ec00640 1 -- 172.21.15.44:0/2195733783 wait complete. 2024-07-17T08:37:50.726 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:37:50.726 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:37:50.726 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:37:50.726 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:50.727 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:37:50.727 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:37:50.727 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:37:50.727 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:50.727 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:37:50.727 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:37:50.728 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:37:50.728 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:50.728 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:37:50.728 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:37:50.728 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:37:50.728 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:37:50.745 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:37:50.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:37:50.747 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:37:50.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:37:50.750 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:37:50.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:37:50.765 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:37:50.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:37:50.794 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:37:50.795 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:50.800 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:37:50.801 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:50.810 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:37:50.810 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:50.816 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:37:50.816 DEBUG:teuthology.parallel:result is None 2024-07-17T08:37:50.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:37:50.941 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:37:50.943 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:37:50.958 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:37:50.959 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.997+0000 7f237b800640 1 -- 172.21.15.44:0/3542519017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 msgr2=0x7f237406bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.997+0000 7f237b800640 1 --2- 172.21.15.44:0/3542519017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237406bf40 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f2374144ed0 tx=0x7f236002a560 comp rx=0 tx=0).stop 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 -- 172.21.15.44:0/3542519017 shutdown_connections 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 --2- 172.21.15.44:0/3542519017 >> v2:172.21.15.175:3300/0 conn(0x7f237413e860 0x7f237406c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 --2- 172.21.15.44:0/3542519017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237406bf40 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 --2- 172.21.15.44:0/3542519017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 0x7f237415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 -- 172.21.15.44:0/3542519017 >> 172.21.15.44:0/3542519017 conn(0x7f237406d050 msgr2=0x7f237406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 -- 172.21.15.44:0/3542519017 shutdown_connections 2024-07-17T08:37:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.998+0000 7f237b800640 1 -- 172.21.15.44:0/3542519017 wait complete. 2024-07-17T08:37:51.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:50.999+0000 7f237b800640 1 Processor -- start 2024-07-17T08:37:51.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.000+0000 7f237b800640 1 -- start start 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.000+0000 7f237b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 0x7f23741e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f237b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f237413e860 0x7f23741e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f237b800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f237b800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2374140d60 con 0x7f237415f9f0 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f237b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2374140be0 con 0x7f237413e860 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f237b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2374140ee0 con 0x7f23740afa90 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:51.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49172/0 (socket says 172.21.15.44:49172) 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 -- 172.21.15.44:0/1907475635 learned_addr learned my addr 172.21.15.44:0/1907475635 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2378e00640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 0x7f23741e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2373e00640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f237413e860 0x7f23741e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 -- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 msgr2=0x7f23741e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 0x7f23741e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 -- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f237413e860 msgr2=0x7f23741e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f237413e860 0x7f23741e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.001+0000 7f2379800640 1 -- 172.21.15.44:0/1907475635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2374142400 con 0x7f237415f9f0 2024-07-17T08:37:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f2379800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f236c00eb40 tx=0x7f236c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c004b20 con 0x7f237415f9f0 2024-07-17T08:37:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f236c004db0 con 0x7f237415f9f0 2024-07-17T08:37:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23741f5a00 con 0x7f237415f9f0 2024-07-17T08:37:51.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c005280 con 0x7f237415f9f0 2024-07-17T08:37:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.002+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23741f5fc0 con 0x7f237415f9f0 2024-07-17T08:37:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.003+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237415f4b0 con 0x7f237415f9f0 2024-07-17T08:37:51.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.005+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f236c005420 con 0x7f237415f9f0 2024-07-17T08:37:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.006+0000 7f2370c00640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 0x7f2354079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.006+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6771+0+0 (secure 0 0 0) 0x7f236c0996f0 con 0x7f237415f9f0 2024-07-17T08:37:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.006+0000 7f2378e00640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 0x7f2354079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:51.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.007+0000 7f2378e00640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 0x7f2354079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2368009bd0 tx=0x7f2368005f90 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:51.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.010+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f236c061e80 con 0x7f237415f9f0 2024-07-17T08:37:51.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.231+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f237406ca70 con 0x7f237415f9f0 2024-07-17T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.819+0000 7f2370c00640 1 -- 172.21.15.44:0/1907475635 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v271) ==== 97+0+0 (secure 0 0 0) 0x7f236c0663d0 con 0x7f237415f9f0 2024-07-17T08:37:51.821 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 msgr2=0x7f2354079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 0x7f2354079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2368009bd0 tx=0x7f2368005f90 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 msgr2=0x7f237412caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f236c00eb40 tx=0x7f236c00c2d0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 shutdown_connections 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f2354076cd0 0x7f2354079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f237415f9f0 0x7f237412caa0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f237413e860 0x7f23741e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 --2- 172.21.15.44:0/1907475635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f23740afa90 0x7f23741e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 >> 172.21.15.44:0/1907475635 conn(0x7f237406d050 msgr2=0x7f237406e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:51.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 shutdown_connections 2024-07-17T08:37:51.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:51.823+0000 7f237b800640 1 -- 172.21.15.44:0/1907475635 wait complete. 2024-07-17T08:37:51.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:37:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 -- 172.21.15.44:0/4051107233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc06e610 msgr2=0x7fa2bc15daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 --2- 172.21.15.44:0/4051107233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc06e610 0x7fa2bc15daa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005fd0 tx=0x7fa2ac02aa90 comp rx=0 tx=0).stop 2024-07-17T08:37:52.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 -- 172.21.15.44:0/4051107233 shutdown_connections 2024-07-17T08:37:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 --2- 172.21.15.44:0/4051107233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc15ac30 0x7fa2bc15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 --2- 172.21.15.44:0/4051107233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc06e610 0x7fa2bc15daa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.056+0000 7fa2c2000640 1 -- 172.21.15.44:0/4051107233 >> 172.21.15.44:0/4051107233 conn(0x7fa2bc06ce80 msgr2=0x7fa2bc06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.057+0000 7fa2c2000640 1 -- 172.21.15.44:0/4051107233 shutdown_connections 2024-07-17T08:37:52.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.057+0000 7fa2c2000640 1 -- 172.21.15.44:0/4051107233 wait complete. 2024-07-17T08:37:52.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.058+0000 7fa2c2000640 1 Processor -- start 2024-07-17T08:37:52.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.058+0000 7fa2c2000640 1 -- start start 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06e610 0x7fa2bc12fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 0x7fa2bc141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc15c800 con 0x7fa2bc15ac30 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc15c680 con 0x7fa2bc15dfe0 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.059+0000 7fa2c2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc15c980 con 0x7fa2bc06e610 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49180/0 (socket says 172.21.15.44:49180) 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 -- 172.21.15.44:0/2180067969 learned_addr learned my addr 172.21.15.44:0/2180067969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2c0c00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06e610 0x7fa2bc12fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2c1600640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 0x7fa2bc141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 -- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06e610 msgr2=0x7fa2bc12fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06e610 0x7fa2bc12fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 -- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 msgr2=0x7fa2bc141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 0x7fa2bc141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2bbe00640 1 -- 172.21.15.44:0/2180067969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc132e30 con 0x7fa2bc15ac30 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.060+0000 7fa2c1600640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 0x7fa2bc141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:52.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2bbe00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c370 tx=0x7fa2a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40052d0 con 0x7fa2bc15ac30 2024-07-17T08:37:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2a4007020 con 0x7fa2bc15ac30 2024-07-17T08:37:52.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4007470 con 0x7fa2bc15ac30 2024-07-17T08:37:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc130480 con 0x7fa2bc15ac30 2024-07-17T08:37:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.061+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc130d20 con 0x7fa2bc15ac30 2024-07-17T08:37:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.062+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc15ce30 con 0x7fa2bc15ac30 2024-07-17T08:37:52.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.064+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fa2a40067d0 con 0x7fa2bc15ac30 2024-07-17T08:37:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.065+0000 7fa2b8c00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 0x7fa2980791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.065+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(273..273 src has 1..273) ==== 5825+0+0 (secure 0 0 0) 0x7fa2a4097fb0 con 0x7fa2bc15ac30 2024-07-17T08:37:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.066+0000 7fa2c0c00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 0x7fa2980791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.067+0000 7fa2c0c00640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 0x7fa2980791c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac035000 tx=0x7fa2ac039040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.068+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa2a4061250 con 0x7fa2bc15ac30 2024-07-17T08:37:52.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.358+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa2bc15dcb0 con 0x7fa2bc15ac30 2024-07-17T08:37:52.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.360+0000 7fa2b8c00640 1 -- 172.21.15.44:0/2180067969 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v74) ==== 80+0+5304 (secure 0 0 0) 0x7fa2a4060b50 con 0x7fa2bc15ac30 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:52.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:37:52.365 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.366 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:37:52.367 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.368 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:37:52.369 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.370 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.371 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:52.372 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:37:52.374 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 msgr2=0x7fa2980791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 0x7fa2980791c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac035000 tx=0x7fa2ac039040 comp rx=0 tx=0).stop 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 msgr2=0x7fa2bc1328f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c370 tx=0x7fa2a400c830 comp rx=0 tx=0).stop 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 shutdown_connections 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa298076d00 0x7fa2980791c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2bc15dfe0 0x7fa2bc141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2bc15ac30 0x7fa2bc1328f0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 --2- 172.21.15.44:0/2180067969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2bc06e610 0x7fa2bc12fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.362+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 >> 172.21.15.44:0/2180067969 conn(0x7fa2bc06ce80 msgr2=0x7fa2bc142e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.363+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 shutdown_connections 2024-07-17T08:37:52.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.363+0000 7fa2c2000640 1 -- 172.21.15.44:0/2180067969 wait complete. 2024-07-17T08:37:52.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:37:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 -- 172.21.15.44:0/2027160898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 msgr2=0x7fa7a40ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2027160898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a40ac760 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009830 tx=0x7fa79c02a510 comp rx=0 tx=0).stop 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 -- 172.21.15.44:0/2027160898 shutdown_connections 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2027160898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a40ac760 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2027160898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 0x7fa7a40ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 -- 172.21.15.44:0/2027160898 >> 172.21.15.44:0/2027160898 conn(0x7fa7a40b0fb0 msgr2=0x7fa7a40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 -- 172.21.15.44:0/2027160898 shutdown_connections 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.590+0000 7fa7abc00640 1 -- 172.21.15.44:0/2027160898 wait complete. 2024-07-17T08:37:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.592+0000 7fa7abc00640 1 Processor -- start 2024-07-17T08:37:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.592+0000 7fa7abc00640 1 -- start start 2024-07-17T08:37:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 0x7fa7a409c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a40ae810 0x7fa7a413be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a40b2750 con 0x7fa7a406ee90 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a40b25d0 con 0x7fa7a40ae810 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa7abc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a40b28d0 con 0x7fa7a406ea50 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.593+0000 7fa79be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49184/0 (socket says 172.21.15.44:49184) 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 -- 172.21.15.44:0/2068568443 learned_addr learned my addr 172.21.15.44:0/2068568443 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa7a9200640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 0x7fa7a409c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa7a9c00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a40ae810 0x7fa7a413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 -- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 msgr2=0x7fa7a409c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 0x7fa7a409c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 -- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a40ae810 msgr2=0x7fa7a413be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a40ae810 0x7fa7a413be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 -- 172.21.15.44:0/2068568443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a413c930 con 0x7fa7a406ee90 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.594+0000 7fa79be00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c920 tx=0x7fa78c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.595+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c007c70 con 0x7fa7a406ee90 2024-07-17T08:37:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.595+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa78c00f450 con 0x7fa7a406ee90 2024-07-17T08:37:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.595+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f8a0 con 0x7fa7a406ee90 2024-07-17T08:37:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.595+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a413cc10 con 0x7fa7a406ee90 2024-07-17T08:37:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.595+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a41570a0 con 0x7fa7a406ee90 2024-07-17T08:37:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.596+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a40cf020 con 0x7fa7a406ee90 2024-07-17T08:37:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.597+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fa78c00fa40 con 0x7fa7a406ee90 2024-07-17T08:37:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.598+0000 7fa7a1600640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 0x7fa780079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.598+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(273..273 src has 1..273) ==== 5825+0+0 (secure 0 0 0) 0x7fa78c098e40 con 0x7fa7a406ee90 2024-07-17T08:37:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.599+0000 7fa7a9200640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 0x7fa780079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:52.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.600+0000 7fa7a9200640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 0x7fa780079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa7940020f0 tx=0x7fa794009040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.603+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa7a40cf020 con 0x7fa7a406ee90 2024-07-17T08:37:52.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.813+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa7a4076330 con 0x7fa7a406ee90 2024-07-17T08:37:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.819+0000 7fa7a1600640 1 -- 172.21.15.44:0/2068568443 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v273) ==== 84+0+65 (secure 0 0 0) 0x7fa7a4076330 con 0x7fa7a406ee90 2024-07-17T08:37:52.821 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:52.821 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:37:52.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:37:52.822 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:37:52.822 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:37:52.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:37:52.823 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:37:52.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 msgr2=0x7fa780079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 0x7fa780079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa7940020f0 tx=0x7fa794009040 comp rx=0 tx=0).stop 2024-07-17T08:37:52.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 msgr2=0x7fa7a4154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:52.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c920 tx=0x7fa78c00cde0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 shutdown_connections 2024-07-17T08:37:52.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa780076cd0 0x7fa780079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7a40ae810 0x7fa7a413be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7a406ee90 0x7fa7a4154b50 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 --2- 172.21.15.44:0/2068568443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7a406ea50 0x7fa7a409c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:52.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 >> 172.21.15.44:0/2068568443 conn(0x7fa7a40b0fb0 msgr2=0x7fa7a41601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:52.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.822+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 shutdown_connections 2024-07-17T08:37:52.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:52.823+0000 7fa7abc00640 1 -- 172.21.15.44:0/2068568443 wait complete. 2024-07-17T08:37:52.844 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:37:52.844 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:37:53.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 -- 172.21.15.44:0/3188168602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 msgr2=0x7fc3ac0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 --2- 172.21.15.44:0/3188168602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac0b23a0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0b4c00 tx=0x7fc3a402a8b0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 -- 172.21.15.44:0/3188168602 shutdown_connections 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 --2- 172.21.15.44:0/3188168602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac0b23a0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 --2- 172.21.15.44:0/3188168602 >> v2:172.21.15.175:3300/0 conn(0x7fc3ac06edc0 0x7fc3ac06f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 -- 172.21.15.44:0/3188168602 >> 172.21.15.44:0/3188168602 conn(0x7fc3ac0cf050 msgr2=0x7fc3ac0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.052+0000 7fc3b2200640 1 -- 172.21.15.44:0/3188168602 shutdown_connections 2024-07-17T08:37:53.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.053+0000 7fc3b2200640 1 -- 172.21.15.44:0/3188168602 wait complete. 2024-07-17T08:37:53.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.054+0000 7fc3b2200640 1 Processor -- start 2024-07-17T08:37:53.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.054+0000 7fc3b2200640 1 -- start start 2024-07-17T08:37:53.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3ac06edc0 0x7fc3ac072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:53.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ac15a5a0 0x7fc3ac06dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac0ac8c0 con 0x7fc3ac0ac260 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac0b28e0 con 0x7fc3ac15a5a0 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3b2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac073550 con 0x7fc3ac06edc0 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.055+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49190/0 (socket says 172.21.15.44:49190) 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 -- 172.21.15.44:0/890432868 learned_addr learned my addr 172.21.15.44:0/890432868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 -- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3ac06edc0 msgr2=0x7fc3ac072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3ac06edc0 0x7fc3ac072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 -- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ac15a5a0 msgr2=0x7fc3ac06dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ac15a5a0 0x7fc3ac06dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 -- 172.21.15.44:0/890432868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1f3ce0 con 0x7fc3ac0ac260 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.056+0000 7fc3abe00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c430 tx=0x7fc39400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.057+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394011790 con 0x7fc3ac0ac260 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.057+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc394017070 con 0x7fc3ac0ac260 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.057+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1f3fc0 con 0x7fc3ac0ac260 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.057+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394011ce0 con 0x7fc3ac0ac260 2024-07-17T08:37:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.057+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac072c40 con 0x7fc3ac0ac260 2024-07-17T08:37:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.058+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac0b2e00 con 0x7fc3ac0ac260 2024-07-17T08:37:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.059+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fc39401c460 con 0x7fc3ac0ac260 2024-07-17T08:37:53.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.060+0000 7fc3a8c00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 0x7fc388079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:53.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.060+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(273..273 src has 1..273) ==== 5825+0+0 (secure 0 0 0) 0x7fc394098350 con 0x7fc3ac0ac260 2024-07-17T08:37:53.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.061+0000 7fc3b0e00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 0x7fc388079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:53.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.062+0000 7fc3b0e00640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 0x7fc388079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005fd0 tx=0x7fc39c005850 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:53.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.064+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc394061680 con 0x7fc3ac0ac260 2024-07-17T08:37:53.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.276+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3ac06d800 con 0x7fc3ac0ac260 2024-07-17T08:37:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.832+0000 7fc3a8c00640 1 -- 172.21.15.44:0/890432868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v274) ==== 138+0+0 (secure 0 0 0) 0x7fc394060f80 con 0x7fc3ac0ac260 2024-07-17T08:37:53.834 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:37:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 msgr2=0x7fc388079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 0x7fc388079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005fd0 tx=0x7fc39c005850 comp rx=0 tx=0).stop 2024-07-17T08:37:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 msgr2=0x7fc3ac14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:53.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c430 tx=0x7fc39400c8f0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 shutdown_connections 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc388076cd0 0x7fc388079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3ac15a5a0 0x7fc3ac06dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.837+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac0ac260 0x7fc3ac14fe20 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.838+0000 7fc3b2200640 1 --2- 172.21.15.44:0/890432868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc3ac06edc0 0x7fc3ac072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.838+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 >> 172.21.15.44:0/890432868 conn(0x7fc3ac0cf050 msgr2=0x7fc3ac0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.838+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 shutdown_connections 2024-07-17T08:37:53.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:53.838+0000 7fc3b2200640 1 -- 172.21.15.44:0/890432868 wait complete. 2024-07-17T08:37:53.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:37:54.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 -- 172.21.15.44:0/2726065746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c7815d130 msgr2=0x7f9c7815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/2726065746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c7815d130 0x7f9c7815d510 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c68005fd0 tx=0x7f9c6802a540 comp rx=0 tx=0).stop 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 -- 172.21.15.44:0/2726065746 shutdown_connections 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/2726065746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 0x7f9c780cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/2726065746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c7815d130 0x7f9c7815d510 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 -- 172.21.15.44:0/2726065746 >> 172.21.15.44:0/2726065746 conn(0x7f9c780b4ff0 msgr2=0x7f9c780b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 -- 172.21.15.44:0/2726065746 shutdown_connections 2024-07-17T08:37:54.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.076+0000 7f9c7ce00640 1 -- 172.21.15.44:0/2726065746 wait complete. 2024-07-17T08:37:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.078+0000 7f9c7ce00640 1 Processor -- start 2024-07-17T08:37:54.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.078+0000 7f9c7ce00640 1 -- start start 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 0x7f9c781f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c78161be0 0x7f9c781f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780d1a60 con 0x7f9c7815d130 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780d18e0 con 0x7f9c78161be0 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.079+0000 7f9c7ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780d1be0 con 0x7f9c780ce690 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49198/0 (socket says 172.21.15.44:49198) 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 -- 172.21.15.44:0/41908302 learned_addr learned my addr 172.21.15.44:0/41908302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:54.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c77e00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c78161be0 0x7f9c781f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c77400640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 0x7f9c781f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:54.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 -- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 msgr2=0x7f9c781f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 0x7f9c781f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 -- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c78161be0 msgr2=0x7f9c781f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c78161be0 0x7f9c781f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.080+0000 7f9c6fe00640 1 -- 172.21.15.44:0/41908302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c781f8bc0 con 0x7f9c7815d130 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c6fe00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c500 tx=0x7f9c6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600052d0 con 0x7f9c7815d130 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c60005470 con 0x7f9c7815d130 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60012870 con 0x7f9c7815d130 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c781f8ea0 con 0x7f9c7815d130 2024-07-17T08:37:54.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.081+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c780b23e0 con 0x7f9c7815d130 2024-07-17T08:37:54.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.083+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c780d2d70 con 0x7f9c7815d130 2024-07-17T08:37:54.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.084+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f9c600035f0 con 0x7f9c7815d130 2024-07-17T08:37:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.086+0000 7f9c6f400640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 0x7f9c540791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:54.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.086+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(274..274 src has 1..274) ==== 6195+0+0 (secure 0 0 0) 0x7f9c600982d0 con 0x7f9c7815d130 2024-07-17T08:37:54.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.086+0000 7f9c77400640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 0x7f9c540791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:54.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.088+0000 7f9c77400640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 0x7f9c540791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c68033000 tx=0x7f9c68004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:54.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.091+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9c780d2d70 con 0x7f9c7815d130 2024-07-17T08:37:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.305+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c780b82b0 con 0x7f9c7815d130 2024-07-17T08:37:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.839+0000 7f9c6f400640 1 -- 172.21.15.44:0/41908302 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v275) ==== 170+0+0 (secure 0 0 0) 0x7f9c780b82b0 con 0x7f9c7815d130 2024-07-17T08:37:54.841 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:37:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 msgr2=0x7f9c540791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 0x7f9c540791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c68033000 tx=0x7f9c68004730 comp rx=0 tx=0).stop 2024-07-17T08:37:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 msgr2=0x7f9c780c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:54.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c500 tx=0x7f9c6000c9c0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 shutdown_connections 2024-07-17T08:37:54.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f9c54076d00 0x7f9c540791c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c78161be0 0x7f9c781f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7815d130 0x7f9c780c87c0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.843+0000 7f9c7ce00640 1 --2- 172.21.15.44:0/41908302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c780ce690 0x7f9c781f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.844+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 >> 172.21.15.44:0/41908302 conn(0x7f9c780b4ff0 msgr2=0x7f9c780d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.844+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 shutdown_connections 2024-07-17T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:54.844+0000 7f9c7ce00640 1 -- 172.21.15.44:0/41908302 wait complete. 2024-07-17T08:37:54.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:37:55.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.068+0000 7fe0b7e00640 1 -- 172.21.15.44:0/4090645618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 msgr2=0x7fe09c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.068+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/4090645618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe09c003ec0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80016c0 tx=0x7fe09c005830 comp rx=0 tx=0).stop 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.068+0000 7fe0b7400640 1 -- 172.21.15.44:0/4090645618 reap_dead start 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.068+0000 7fe0b7e00640 1 -- 172.21.15.44:0/4090645618 shutdown_connections 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.068+0000 7fe0b7e00640 1 -- 172.21.15.44:0/4090645618 >> 172.21.15.44:0/4090645618 conn(0x7fe0b80b4ff0 msgr2=0x7fe0b80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.069+0000 7fe0b7e00640 1 -- 172.21.15.44:0/4090645618 shutdown_connections 2024-07-17T08:37:55.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.069+0000 7fe0b7e00640 1 -- 172.21.15.44:0/4090645618 wait complete. 2024-07-17T08:37:55.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.070+0000 7fe0b7e00640 1 Processor -- start 2024-07-17T08:37:55.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.071+0000 7fe0b7e00640 1 -- start start 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.071+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0b80cd090 0x7fe0b81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0b80b8120 0x7fe0b80b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8166400 con 0x7fe09c003aa0 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8166280 con 0x7fe0b80cd090 2024-07-17T08:37:55.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b8166580 con 0x7fe0b80b8120 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49212/0 (socket says 172.21.15.44:49212) 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 -- 172.21.15.44:0/1804983593 learned_addr learned my addr 172.21.15.44:0/1804983593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b7400640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0b80b8120 0x7fe0b80b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6000640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0b80cd090 0x7fe0b81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 -- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0b80b8120 msgr2=0x7fe0b80b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:55.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0b80b8120 0x7fe0b80b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 -- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0b80cd090 msgr2=0x7fe0b81567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:55.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0b80cd090 0x7fe0b81567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.072+0000 7fe0b6a00640 1 -- 172.21.15.44:0/1804983593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b8156d30 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0b6a00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fe0a802a930 tx=0x7fe0a8006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8041070 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0a8003af0 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8038070 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b815da60 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.073+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b815dd20 con 0x7fe09c003aa0 2024-07-17T08:37:55.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.074+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b80b2200 con 0x7fe09c003aa0 2024-07-17T08:37:55.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.076+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fe0a8030070 con 0x7fe09c003aa0 2024-07-17T08:37:55.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.077+0000 7fe0af400640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 0x7fe0940791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:55.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.077+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(275..275 src has 1..275) ==== 6577+0+0 (secure 0 0 0) 0x7fe0a80b7fd0 con 0x7fe09c003aa0 2024-07-17T08:37:55.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.078+0000 7fe0b6000640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 0x7fe0940791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:55.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.079+0000 7fe0b6000640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 0x7fe0940791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8160a90 tx=0x7fe0a00014f0 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.083+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0a804a080 con 0x7fe09c003aa0 2024-07-17T08:37:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.339+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe0b80d0e10 con 0x7fe09c003aa0 2024-07-17T08:37:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.903+0000 7fe0af400640 1 -- 172.21.15.44:0/1804983593 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 37 and data pool 38 v151) ==== 174+0+0 (secure 0 0 0) 0x7fe0a8081000 con 0x7fe09c003aa0 2024-07-17T08:37:55.905 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 37 and data pool 38 2024-07-17T08:37:55.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.909+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 msgr2=0x7fe0940791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:55.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.909+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 0x7fe0940791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8160a90 tx=0x7fe0a00014f0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.909+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 msgr2=0x7fe0b81f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:55.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.909+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fe0a802a930 tx=0x7fe0a8006310 comp rx=0 tx=0).stop 2024-07-17T08:37:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 shutdown_connections 2024-07-17T08:37:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fe094076d00 0x7fe0940791c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0b80b8120 0x7fe0b80b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0b80cd090 0x7fe0b81567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 --2- 172.21.15.44:0/1804983593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe09c003aa0 0x7fe0b81f4be0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.912+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 >> 172.21.15.44:0/1804983593 conn(0x7fe0b80b4ff0 msgr2=0x7fe0b80d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.913+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 shutdown_connections 2024-07-17T08:37:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:55.913+0000 7fe0b7e00640 1 -- 172.21.15.44:0/1804983593 wait complete. 2024-07-17T08:37:55.949 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 -- 172.21.15.44:0/3935136550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00cd690 msgr2=0x7f82e00cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 --2- 172.21.15.44:0/3935136550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00cd690 0x7f82e00cdb20 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005fd0 tx=0x7f82d802a9f0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 -- 172.21.15.44:0/3935136550 shutdown_connections 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 --2- 172.21.15.44:0/3935136550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00cd690 0x7f82e00cdb20 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 --2- 172.21.15.44:0/3935136550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00ac220 0x7f82e00ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 -- 172.21.15.44:0/3935136550 >> 172.21.15.44:0/3935136550 conn(0x7f82e00b0fb0 msgr2=0x7f82e00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 -- 172.21.15.44:0/3935136550 shutdown_connections 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.120+0000 7f82e7000640 1 -- 172.21.15.44:0/3935136550 wait complete. 2024-07-17T08:37:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.122+0000 7f82e7000640 1 Processor -- start 2024-07-17T08:37:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.122+0000 7f82e7000640 1 -- start start 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00cd690 0x7f82e01f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82e01522c0 0x7f82e00b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f82e00ce1e0 con 0x7f82e00ac220 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82e00ce060 con 0x7f82e01522c0 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82e7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82e00ce360 con 0x7f82e00cd690 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49232/0 (socket says 172.21.15.44:49232) 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.123+0000 7f82dfe00640 1 -- 172.21.15.44:0/3172661243 learned_addr learned my addr 172.21.15.44:0/3172661243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:56.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82d7400640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00cd690 0x7f82e01f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 -- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00cd690 msgr2=0x7f82e01f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00cd690 0x7f82e01f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 -- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82e01522c0 msgr2=0x7f82e00b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82e01522c0 0x7f82e00b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 -- 172.21.15.44:0/3172661243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e01f7ab0 con 0x7f82e00ac220 2024-07-17T08:37:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dfe00640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b3e0 tx=0x7f82d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0015020 con 0x7f82e00ac220 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.124+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82d0011940 con 0x7f82e00ac220 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.125+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0011d90 con 0x7f82e00ac220 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.125+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e01f7d90 con 0x7f82e00ac220 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.125+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e01f3f80 con 0x7f82e00ac220 2024-07-17T08:37:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.126+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e00ce880 con 0x7f82e00ac220 2024-07-17T08:37:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.127+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f82d0018020 con 0x7f82e00ac220 2024-07-17T08:37:56.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.128+0000 7f82dcc00640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 0x7f82bc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.128+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(276..276 src has 1..276) ==== 6677+0+0 (secure 0 0 0) 0x7f82d0098880 con 0x7f82e00ac220 2024-07-17T08:37:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.129+0000 7f82d7400640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 0x7f82bc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.130+0000 7f82d7400640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 0x7f82bc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82e0151430 tx=0x7f82c8005cc0 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.132+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f82d001f020 con 0x7f82e00ac220 2024-07-17T08:37:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.342+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82e01f4150 con 0x7f82e00ac220 2024-07-17T08:37:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.345+0000 7f82dcc00640 1 -- 172.21.15.44:0/3172661243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v276) ==== 74+0+17774 (secure 0 0 0) 0x7f82d0061860 con 0x7f82e00ac220 2024-07-17T08:37:56.347 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:56.347 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":276,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:55.342315+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:53.302243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":38,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:54.326826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.348+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 msgr2=0x7f82bc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.348+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 0x7f82bc079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82e0151430 tx=0x7f82c8005cc0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.348+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 msgr2=0x7f82e01f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.348+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f82d000b3e0 tx=0x7f82d000b8a0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 shutdown_connections 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f82bc076cd0 0x7f82bc079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82e01522c0 0x7f82e00b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82e00cd690 0x7f82e01f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 --2- 172.21.15.44:0/3172661243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e00ac220 0x7f82e01f44a0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 >> 172.21.15.44:0/3172661243 conn(0x7f82e00b0fb0 msgr2=0x7f82e006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 shutdown_connections 2024-07-17T08:37:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.349+0000 7f82e7000640 1 -- 172.21.15.44:0/3172661243 wait complete. 2024-07-17T08:37:56.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.571+0000 7fcc27c00640 1 -- 172.21.15.44:0/3752401077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 msgr2=0x7fcc2015ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.571+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3752401077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc2015ee90 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fcc20070bf0 tx=0x7fcc1802a560 comp rx=0 tx=0).stop 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 -- 172.21.15.44:0/3752401077 shutdown_connections 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3752401077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc200ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3752401077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc2015ee90 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3752401077 >> v2:172.21.15.175:3300/0 conn(0x7fcc201522c0 0x7fcc200ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 -- 172.21.15.44:0/3752401077 >> 172.21.15.44:0/3752401077 conn(0x7fcc200b0fb0 msgr2=0x7fcc200b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:56.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 -- 172.21.15.44:0/3752401077 shutdown_connections 2024-07-17T08:37:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.572+0000 7fcc27c00640 1 -- 172.21.15.44:0/3752401077 wait complete. 2024-07-17T08:37:56.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.574+0000 7fcc27c00640 1 Processor -- start 2024-07-17T08:37:56.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.574+0000 7fcc27c00640 1 -- start start 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc2014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 0x7fcc200c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015f760 con 0x7fcc200ac760 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015f5e0 con 0x7fcc201522c0 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.575+0000 7fcc27c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2015f8e0 con 0x7fcc200adcd0 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49258/0 (socket says 172.21.15.44:49258) 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 -- 172.21.15.44:0/3226245078 learned_addr learned my addr 172.21.15.44:0/3226245078 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:56.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc2014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 0x7fcc200c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 -- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 msgr2=0x7fcc2014c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc2014c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 -- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 msgr2=0x7fcc200c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 0x7fcc200c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25200640 1 -- 172.21.15.44:0/3226245078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2014ca00 con 0x7fcc200ac760 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc2014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.576+0000 7fcc25c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 0x7fcc200c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc25200640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000dac0 tx=0x7fcc1000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10017070 con 0x7fcc200ac760 2024-07-17T08:37:56.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc10003970 con 0x7fcc200ac760 2024-07-17T08:37:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000be60 con 0x7fcc200ac760 2024-07-17T08:37:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2014cce0 con 0x7fcc200ac760 2024-07-17T08:37:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.577+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2014d2a0 con 0x7fcc200ac760 2024-07-17T08:37:56.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.578+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200ac220 con 0x7fcc200ac760 2024-07-17T08:37:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.579+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fcc10004020 con 0x7fcc200ac760 2024-07-17T08:37:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.580+0000 7fcc1cc00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 0x7fcbfc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.581+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(276..276 src has 1..276) ==== 6677+0+0 (secure 0 0 0) 0x7fcc10098fe0 con 0x7fcc200ac760 2024-07-17T08:37:56.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.581+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 0x7fcbfc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:56.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.582+0000 7fcc1fe00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 0x7fcbfc079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc201f7650 tx=0x7fcc08005e50 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:56.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.585+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcc10020030 con 0x7fcc200ac760 2024-07-17T08:37:56.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.792+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc2015bde0 con 0x7fcc200ac760 2024-07-17T08:37:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.793+0000 7fcc1cc00640 1 -- 172.21.15.44:0/3226245078 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v276) ==== 74+0+17774 (secure 0 0 0) 0x7fcc10061e40 con 0x7fcc200ac760 2024-07-17T08:37:56.795 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:56.795 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":276,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:55.342315+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:53.302243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":38,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:54.326826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 msgr2=0x7fcbfc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 0x7fcbfc079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc201f7650 tx=0x7fcc08005e50 comp rx=0 tx=0).stop 2024-07-17T08:37:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 msgr2=0x7fcc201498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000dac0 tx=0x7fcc1000df80 comp rx=0 tx=0).stop 2024-07-17T08:37:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 shutdown_connections 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fcbfc076cd0 0x7fcbfc079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc201522c0 0x7fcc200c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc200adcd0 0x7fcc2014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 --2- 172.21.15.44:0/3226245078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc200ac760 0x7fcc201498b0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 >> 172.21.15.44:0/3226245078 conn(0x7fcc200b0fb0 msgr2=0x7fcc200cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 shutdown_connections 2024-07-17T08:37:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:56.796+0000 7fcc27c00640 1 -- 172.21.15.44:0/3226245078 wait complete. 2024-07-17T08:37:56.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:37:57.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 -- 172.21.15.44:0/2265683032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c15d810 msgr2=0x7fc21c0b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:57.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 --2- 172.21.15.44:0/2265683032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c15d810 0x7fc21c0b5fb0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0749d0 tx=0x7fc20c02a5f0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 -- 172.21.15.44:0/2265683032 shutdown_connections 2024-07-17T08:37:57.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 --2- 172.21.15.44:0/2265683032 >> v2:172.21.15.175:3300/0 conn(0x7fc21c0b1a10 0x7fc21c06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 --2- 172.21.15.44:0/2265683032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c15d810 0x7fc21c0b5fb0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 -- 172.21.15.44:0/2265683032 >> 172.21.15.44:0/2265683032 conn(0x7fc21c073d50 msgr2=0x7fc21c09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:57.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 -- 172.21.15.44:0/2265683032 shutdown_connections 2024-07-17T08:37:57.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.035+0000 7fc221000640 1 -- 172.21.15.44:0/2265683032 wait complete. 2024-07-17T08:37:57.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.037+0000 7fc221000640 1 Processor -- start 2024-07-17T08:37:57.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.037+0000 7fc221000640 1 -- start start 2024-07-17T08:37:57.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc21c06dbd0 0x7fc21c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc21c15d810 0x7fc21c15ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0af600 con 0x7fc21c0b1a10 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0af480 con 0x7fc21c15d810 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.038+0000 7fc221000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c0b5540 con 0x7fc21c06dbd0 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49276/0 (socket says 172.21.15.44:49276) 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 -- 172.21.15.44:0/3978316209 learned_addr learned my addr 172.21.15.44:0/3978316209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc219e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc21c06dbd0 0x7fc21c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:57.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc21a800640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc21c15d810 0x7fc21c15ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 -- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc21c06dbd0 msgr2=0x7fc21c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc21c06dbd0 0x7fc21c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 -- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc21c15d810 msgr2=0x7fc21c15ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc21c15d810 0x7fc21c15ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 -- 172.21.15.44:0/3978316209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c09b580 con 0x7fc21c0b1a10 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.039+0000 7fc213e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c920 tx=0x7fc20400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.040+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204007c70 con 0x7fc21c0b1a10 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.040+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc20400f450 con 0x7fc21c0b1a10 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.040+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20400f8a0 con 0x7fc21c0b1a10 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.040+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c1f8740 con 0x7fc21c0b1a10 2024-07-17T08:37:57.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.040+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c09bb00 con 0x7fc21c0b1a10 2024-07-17T08:37:57.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.041+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c06eac0 con 0x7fc21c0b1a10 2024-07-17T08:37:57.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.042+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fc20400fa40 con 0x7fc21c0b1a10 2024-07-17T08:37:57.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.043+0000 7fc212000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 0x7fc1f8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:57.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.044+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fc204099120 con 0x7fc21c0b1a10 2024-07-17T08:37:57.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.044+0000 7fc219e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 0x7fc1f8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:57.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.045+0000 7fc219e00640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 0x7fc1f8079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc20c033000 tx=0x7fc20c004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:57.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.048+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc21c06eac0 con 0x7fc21c0b1a10 2024-07-17T08:37:57.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.301+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc21c0b5a60 con 0x7fc21c0b1a10 2024-07-17T08:37:57.830 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:37:57.829+0000 7f4f7ae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:37:57.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.930+0000 7fc212000640 1 -- 172.21.15.44:0/3978316209 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v154) ==== 118+0+0 (secure 0 0 0) 0x7fc204014030 con 0x7fc21c0b1a10 2024-07-17T08:37:57.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 msgr2=0x7fc1f8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:57.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 0x7fc1f8079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc20c033000 tx=0x7fc20c004730 comp rx=0 tx=0).stop 2024-07-17T08:37:57.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 msgr2=0x7fc21c14bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:57.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c920 tx=0x7fc20400cde0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 shutdown_connections 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc1f8076cd0 0x7fc1f8079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc21c15d810 0x7fc21c15ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21c0b1a10 0x7fc21c14bdf0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 --2- 172.21.15.44:0/3978316209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc21c06dbd0 0x7fc21c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.934+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 >> 172.21.15.44:0/3978316209 conn(0x7fc21c073d50 msgr2=0x7fc21c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.935+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 shutdown_connections 2024-07-17T08:37:57.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:57.935+0000 7fc221000640 1 -- 172.21.15.44:0/3978316209 wait complete. 2024-07-17T08:37:57.957 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:37:57.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:37:58.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 -- 172.21.15.44:0/2627970135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 msgr2=0x7fa84006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:58.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 --2- 172.21.15.44:0/2627970135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84006d420 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fa83c005fd0 tx=0x7fa83c02a8b0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 -- 172.21.15.44:0/2627970135 shutdown_connections 2024-07-17T08:37:58.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 --2- 172.21.15.44:0/2627970135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84006d420 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 -- 172.21.15.44:0/2627970135 >> 172.21.15.44:0/2627970135 conn(0x7fa8400cf0d0 msgr2=0x7fa8400cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:58.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.163+0000 7fa847200640 1 -- 172.21.15.44:0/2627970135 shutdown_connections 2024-07-17T08:37:58.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.164+0000 7fa847200640 1 -- 172.21.15.44:0/2627970135 wait complete. 2024-07-17T08:37:58.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.165+0000 7fa847200640 1 Processor -- start 2024-07-17T08:37:58.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.165+0000 7fa847200640 1 -- start start 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006f4d0 0x7fa8401f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84015a450 0x7fa840151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa84006dcf0 con 0x7fa8401503c0 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa84006db70 con 0x7fa84015a450 2024-07-17T08:37:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.166+0000 7fa847200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8400ac010 con 0x7fa84006f4d0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49304/0 (socket says 172.21.15.44:49304) 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 -- 172.21.15.44:0/3239127124 learned_addr learned my addr 172.21.15.44:0/3239127124 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa846800640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84015a450 0x7fa840151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 -- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006f4d0 msgr2=0x7fa8401f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006f4d0 0x7fa8401f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 -- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84015a450 msgr2=0x7fa840151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84015a450 0x7fa840151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.167+0000 7fa845400640 1 -- 172.21.15.44:0/3239127124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8401f7cd0 con 0x7fa8401503c0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa845400640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b3f0 tx=0x7fa83000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830015020 con 0x7fa8401503c0 2024-07-17T08:37:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa830011920 con 0x7fa8401503c0 2024-07-17T08:37:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8401f7fb0 con 0x7fa8401503c0 2024-07-17T08:37:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830011d70 con 0x7fa8401503c0 2024-07-17T08:37:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.168+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401f8570 con 0x7fa8401503c0 2024-07-17T08:37:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.169+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa840158cf0 con 0x7fa8401503c0 2024-07-17T08:37:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.171+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fa830018020 con 0x7fa8401503c0 2024-07-17T08:37:58.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.172+0000 7fa83a000640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 0x7fa8240791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:58.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.172+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fa830098550 con 0x7fa8401503c0 2024-07-17T08:37:58.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.172+0000 7fa845e00640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 0x7fa8240791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.174+0000 7fa845e00640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 0x7fa8240791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa834007fd0 tx=0x7fa834007900 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.177+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa840158cf0 con 0x7fa8401503c0 2024-07-17T08:37:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.428+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa840050670 con 0x7fa8401503c0 2024-07-17T08:37:58.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.933+0000 7fa83a000640 1 -- 172.21.15.44:0/3239127124 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v155) ==== 122+0+0 (secure 0 0 0) 0x7fa840050670 con 0x7fa8401503c0 2024-07-17T08:37:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 msgr2=0x7fa8240791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 0x7fa8240791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa834007fd0 tx=0x7fa834007900 comp rx=0 tx=0).stop 2024-07-17T08:37:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 msgr2=0x7fa84009c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b3f0 tx=0x7fa83000b8b0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 shutdown_connections 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fa824076d00 0x7fa8240791c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84015a450 0x7fa840151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8401503c0 0x7fa84009c920 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 --2- 172.21.15.44:0/3239127124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006f4d0 0x7fa8401f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 >> 172.21.15.44:0/3239127124 conn(0x7fa8400cf0d0 msgr2=0x7fa8400ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.938+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 shutdown_connections 2024-07-17T08:37:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:58.939+0000 7fa847200640 1 -- 172.21.15.44:0/3239127124 wait complete. 2024-07-17T08:37:58.962 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:37:58.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.173+0000 7f0f0b200640 1 -- 172.21.15.44:0/2157195515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f0415b2e0 msgr2=0x7f0f040ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.173+0000 7f0f0b200640 1 --2- 172.21.15.44:0/2157195515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f0415b2e0 0x7f0f040ac650 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f0f040715d0 tx=0x7f0ef402a560 comp rx=0 tx=0).stop 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 -- 172.21.15.44:0/2157195515 shutdown_connections 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 --2- 172.21.15.44:0/2157195515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0406f560 0x7f0f04153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 --2- 172.21.15.44:0/2157195515 >> v2:172.21.15.175:3300/0 conn(0x7f0f040acb90 0x7f0f0406f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 --2- 172.21.15.44:0/2157195515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f0415b2e0 0x7f0f040ac650 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 -- 172.21.15.44:0/2157195515 >> 172.21.15.44:0/2157195515 conn(0x7f0f040b0fb0 msgr2=0x7f0f040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 -- 172.21.15.44:0/2157195515 shutdown_connections 2024-07-17T08:37:59.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.174+0000 7f0f0b200640 1 -- 172.21.15.44:0/2157195515 wait complete. 2024-07-17T08:37:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.176+0000 7f0f0b200640 1 Processor -- start 2024-07-17T08:37:59.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.176+0000 7f0f0b200640 1 -- start start 2024-07-17T08:37:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f0406f560 0x7f0f040cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 0x7f0f04071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04154710 con 0x7f0f040acb90 2024-07-17T08:37:59.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04154590 con 0x7f0f0406f560 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f0b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f04154890 con 0x7f0f0415b2e0 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0efb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0efb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49846/0 (socket says 172.21.15.44:49846) 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0efb400640 1 -- 172.21.15.44:0/3491229298 learned_addr learned my addr 172.21.15.44:0/3491229298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f03e00640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f0406f560 0x7f0f040cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.177+0000 7f0f09200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 0x7f0f04071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 -- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 msgr2=0x7f0f04071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 0x7f0f04071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 -- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f0406f560 msgr2=0x7f0f040cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f0406f560 0x7f0f040cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 -- 172.21.15.44:0/3491229298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f04072f80 con 0x7f0f040acb90 2024-07-17T08:37:59.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f09200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 0x7f0f04071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0efb400640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c920 tx=0x7f0eec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec007c70 con 0x7f0f040acb90 2024-07-17T08:37:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eec00f450 con 0x7f0f040acb90 2024-07-17T08:37:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00f8a0 con 0x7f0f040acb90 2024-07-17T08:37:59.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f04073200 con 0x7f0f040acb90 2024-07-17T08:37:59.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.178+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f040734a0 con 0x7f0f040acb90 2024-07-17T08:37:59.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.180+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0406f020 con 0x7f0f040acb90 2024-07-17T08:37:59.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.181+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f0eec00fa40 con 0x7f0f040acb90 2024-07-17T08:37:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.182+0000 7f0f00c00640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 0x7f0ee0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.182+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f0eec0990c0 con 0x7f0f040acb90 2024-07-17T08:37:59.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.183+0000 7f0f03e00640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 0x7f0ee0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.184+0000 7f0f03e00640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 0x7f0ee0079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4033000 tx=0x7f0ef4004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:59.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.188+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0f04072f80 con 0x7f0f040acb90 2024-07-17T08:37:59.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.436+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f0406dbf0 con 0x7f0f040acb90 2024-07-17T08:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.437+0000 7f0f00c00640 1 -- 172.21.15.44:0/3491229298 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7f0eec014030 con 0x7f0f040acb90 2024-07-17T08:37:59.438 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:59.438 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 msgr2=0x7f0ee0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 0x7f0ee0079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4033000 tx=0x7f0ef4004730 comp rx=0 tx=0).stop 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 msgr2=0x7f0f04145db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c920 tx=0x7f0eec00cde0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 shutdown_connections 2024-07-17T08:37:59.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f0ee0076cd0 0x7f0ee0079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f0415b2e0 0x7f0f04071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f040acb90 0x7f0f04145db0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 --2- 172.21.15.44:0/3491229298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f0406f560 0x7f0f040cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 >> 172.21.15.44:0/3491229298 conn(0x7f0f040b0fb0 msgr2=0x7f0f04159370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 shutdown_connections 2024-07-17T08:37:59.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.440+0000 7f0f0b200640 1 -- 172.21.15.44:0/3491229298 wait complete. 2024-07-17T08:37:59.461 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:37:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 -- 172.21.15.44:0/2000684318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ac220 msgr2=0x7ffadc0ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 --2- 172.21.15.44:0/2000684318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ac220 0x7ffadc0ac690 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7ffadc15edd0 tx=0x7ffac402a560 comp rx=0 tx=0).stop 2024-07-17T08:37:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 -- 172.21.15.44:0/2000684318 shutdown_connections 2024-07-17T08:37:59.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 --2- 172.21.15.44:0/2000684318 >> v2:172.21.15.175:3300/0 conn(0x7ffadc0cd690 0x7ffadc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 --2- 172.21.15.44:0/2000684318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc0ac220 0x7ffadc0ac690 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 -- 172.21.15.44:0/2000684318 >> 172.21.15.44:0/2000684318 conn(0x7ffadc0b0fb0 msgr2=0x7ffadc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 -- 172.21.15.44:0/2000684318 shutdown_connections 2024-07-17T08:37:59.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.667+0000 7ffae1400640 1 -- 172.21.15.44:0/2000684318 wait complete. 2024-07-17T08:37:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.669+0000 7ffae1400640 1 Processor -- start 2024-07-17T08:37:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.669+0000 7ffae1400640 1 -- start start 2024-07-17T08:37:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 0x7ffadc071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 0x7ffadc1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc0ce020 con 0x7ffadc1522c0 2024-07-17T08:37:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc0adf80 con 0x7ffadc0cd690 2024-07-17T08:37:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.670+0000 7ffae1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc0ce1a0 con 0x7ffadc0ac220 2024-07-17T08:37:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49870/0 (socket says 172.21.15.44:49870) 2024-07-17T08:37:59.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 -- 172.21.15.44:0/3105528404 learned_addr learned my addr 172.21.15.44:0/3105528404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffad9e00640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 0x7ffadc1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffada800640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 0x7ffadc071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 -- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 msgr2=0x7ffadc071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 0x7ffadc071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 -- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 msgr2=0x7ffadc1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 0x7ffadc1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffadb200640 1 -- 172.21.15.44:0/3105528404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc0725f0 con 0x7ffadc1522c0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffada800640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 0x7ffadc071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.671+0000 7ffad9e00640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 0x7ffadc1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffadb200640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7ffad400cab0 tx=0x7ffad400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad400adc0 con 0x7ffadc1522c0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad400f450 con 0x7ffadc1522c0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad400f8c0 con 0x7ffadc1522c0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc072b20 con 0x7ffadc1522c0 2024-07-17T08:37:59.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.672+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc073380 con 0x7ffadc1522c0 2024-07-17T08:37:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.673+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc06a1a0 con 0x7ffadc1522c0 2024-07-17T08:37:59.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.674+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7ffad400fa60 con 0x7ffadc1522c0 2024-07-17T08:37:59.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.675+0000 7ffad2a00640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 0x7ffab8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:37:59.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.675+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7ffad40993a0 con 0x7ffadc1522c0 2024-07-17T08:37:59.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.676+0000 7ffada800640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 0x7ffab8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:37:59.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.677+0000 7ffada800640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 0x7ffab8079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a810 tx=0x7ffacc008040 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:37:59.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.680+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffadc06a1a0 con 0x7ffadc1522c0 2024-07-17T08:37:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.894+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffadc0ce6c0 con 0x7ffadc1522c0 2024-07-17T08:37:59.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.896+0000 7ffad2a00640 1 -- 172.21.15.44:0/3105528404 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v277) ==== 74+0+17761 (secure 0 0 0) 0x7ffad4014030 con 0x7ffadc1522c0 2024-07-17T08:37:59.897 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:37:59.897 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":277,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:37:56.877319+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:53.302243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":38,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:54.326826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"277","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 msgr2=0x7ffab8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 0x7ffab8079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a810 tx=0x7ffacc008040 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 msgr2=0x7ffadc150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7ffad400cab0 tx=0x7ffad400cf70 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 shutdown_connections 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7ffab8076cd0 0x7ffab8079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc1522c0 0x7ffadc150e10 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffadc0cd690 0x7ffadc1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 --2- 172.21.15.44:0/3105528404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc0ac220 0x7ffadc071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:37:59.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 >> 172.21.15.44:0/3105528404 conn(0x7ffadc0b0fb0 msgr2=0x7ffadc0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:37:59.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 shutdown_connections 2024-07-17T08:37:59.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:37:59.899+0000 7ffae1400640 1 -- 172.21.15.44:0/3105528404 wait complete. 2024-07-17T08:37:59.922 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:37:59.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:38:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 -- 172.21.15.44:0/4098530366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd77815d7e0 msgr2=0x7fd778073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 --2- 172.21.15.44:0/4098530366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd77815d7e0 0x7fd778073c30 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fd76c005fd0 tx=0x7fd76c02a670 comp rx=0 tx=0).stop 2024-07-17T08:38:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 -- 172.21.15.44:0/4098530366 shutdown_connections 2024-07-17T08:38:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 --2- 172.21.15.44:0/4098530366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7780b28a0 0x7fd778074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 --2- 172.21.15.44:0/4098530366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd77815d7e0 0x7fd778073c30 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 -- 172.21.15.44:0/4098530366 >> 172.21.15.44:0/4098530366 conn(0x7fd7780d0350 msgr2=0x7fd7780d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 -- 172.21.15.44:0/4098530366 shutdown_connections 2024-07-17T08:38:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.137+0000 7fd780800640 1 -- 172.21.15.44:0/4098530366 wait complete. 2024-07-17T08:38:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.139+0000 7fd780800640 1 Processor -- start 2024-07-17T08:38:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.139+0000 7fd780800640 1 -- start start 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780b22e0 0x7fd7781600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815d7e0 0x7fd7780c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd7780d2bb0 con 0x7fd7780b28a0 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7780d2a30 con 0x7fd7780b22e0 2024-07-17T08:38:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd780800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7780d2d30 con 0x7fd77815d7e0 2024-07-17T08:38:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.140+0000 7fd77d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49878/0 (socket says 172.21.15.44:49878) 2024-07-17T08:38:00.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 -- 172.21.15.44:0/3040619763 learned_addr learned my addr 172.21.15.44:0/3040619763 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 -- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815d7e0 msgr2=0x7fd7780c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815d7e0 0x7fd7780c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 -- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780b22e0 msgr2=0x7fd7781600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77de00640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780b22e0 0x7fd7781600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780b22e0 0x7fd7781600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 -- 172.21.15.44:0/3040619763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7781f40e0 con 0x7fd7780b28a0 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.141+0000 7fd77d400640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd76800d7a0 tx=0x7fd76800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.142+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768017070 con 0x7fd7780b28a0 2024-07-17T08:38:00.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.142+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd768009d20 con 0x7fd7780b28a0 2024-07-17T08:38:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.142+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768004280 con 0x7fd7780b28a0 2024-07-17T08:38:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.142+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781f43a0 con 0x7fd7780b28a0 2024-07-17T08:38:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.142+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781f4c40 con 0x7fd7780b28a0 2024-07-17T08:38:00.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.143+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7780d32c0 con 0x7fd7780b28a0 2024-07-17T08:38:00.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.144+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fd7680035f0 con 0x7fd7780b28a0 2024-07-17T08:38:00.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.146+0000 7fd772000640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 0x7fd75c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.146+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fd768098a20 con 0x7fd7780b28a0 2024-07-17T08:38:00.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.146+0000 7fd77de00640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 0x7fd75c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.148+0000 7fd77de00640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 0x7fd75c079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd76c033000 tx=0x7fd76c004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:00.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.151+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd7680618f0 con 0x7fd7780b28a0 2024-07-17T08:38:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.359+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 --> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd778069450 con 0x7fd75c076cd0 2024-07-17T08:38:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.376+0000 7fd772000640 1 -- 172.21.15.44:0/3040619763 <== mgr.8516 v2:172.21.15.175:6834/2531126035 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252504 (secure 0 0 0) 0x7fd778069450 con 0x7fd75c076cd0 2024-07-17T08:38:00.381 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:38:00.382 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 msgr2=0x7fd75c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 0x7fd75c079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd76c033000 tx=0x7fd76c004730 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 msgr2=0x7fd7780ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd76800d7a0 tx=0x7fd76800dc60 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 shutdown_connections 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fd75c076cd0 0x7fd75c079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd77815d7e0 0x7fd7780c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7780b28a0 0x7fd7780ccef0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 --2- 172.21.15.44:0/3040619763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7780b22e0 0x7fd7781600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.382+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 >> 172.21.15.44:0/3040619763 conn(0x7fd7780d0350 msgr2=0x7fd7780b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.383+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 shutdown_connections 2024-07-17T08:38:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.383+0000 7fd780800640 1 -- 172.21.15.44:0/3040619763 wait complete. 2024-07-17T08:38:00.404 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:37:59.582630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":466,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4621,"ondisk_log_size":4621,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":454440,"kb_used_data":28644,"kb_used_omap":12,"kb_used_meta":425651,"kb_avail":754520280,"statfs":{"total":773094113280,"available":772628766720,"internally_reserved":0,"allocated":29331456,"data_stored":16590016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":435867184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745602"},"pg_stats":[{"pgid":"38.3f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.892521+0000","last_change":"2024-07-17T08:37:55.929216+0000","last_active":"2024-07-17T08:37:56.892521+0000","last_peered":"2024-07-17T08:37:56.892521+0000","last_clean":"2024-07-17T08:37:56.892521+0000","last_became_active":"2024-07-17T08:37:55.928086+0000","last_became_peered":"2024-07-17T08:37:55.928086+0000","last_unstale":"2024-07-17T08:37:56.892521+0000","last_undegraded":"2024-07-17T08:37:56.892521+0000","last_fullsized":"2024-07-17T08:37:56.892521+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:13.508054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.3e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895710+0000","last_change":"2024-07-17T08:37:55.940150+0000","last_active":"2024-07-17T08:37:56.895710+0000","last_peered":"2024-07-17T08:37:56.895710+0000","last_clean":"2024-07-17T08:37:56.895710+0000","last_became_active":"2024-07-17T08:37:55.933485+0000","last_became_peered":"2024-07-17T08:37:55.933485+0000","last_unstale":"2024-07-17T08:37:56.895710+0000","last_undegraded":"2024-07-17T08:37:56.895710+0000","last_fullsized":"2024-07-17T08:37:56.895710+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:51:53.920815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.3d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315070+0000","last_change":"2024-07-17T08:37:55.931742+0000","last_active":"2024-07-17T08:37:57.315070+0000","last_peered":"2024-07-17T08:37:57.315070+0000","last_clean":"2024-07-17T08:37:57.315070+0000","last_became_active":"2024-07-17T08:37:55.929961+0000","last_became_peered":"2024-07-17T08:37:55.929961+0000","last_unstale":"2024-07-17T08:37:57.315070+0000","last_undegraded":"2024-07-17T08:37:57.315070+0000","last_fullsized":"2024-07-17T08:37:57.315070+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:03.944749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.3c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316658+0000","last_change":"2024-07-17T08:37:55.941611+0000","last_active":"2024-07-17T08:37:57.316658+0000","last_peered":"2024-07-17T08:37:57.316658+0000","last_clean":"2024-07-17T08:37:57.316658+0000","last_became_active":"2024-07-17T08:37:55.941250+0000","last_became_peered":"2024-07-17T08:37:55.941250+0000","last_unstale":"2024-07-17T08:37:57.316658+0000","last_undegraded":"2024-07-17T08:37:57.316658+0000","last_fullsized":"2024-07-17T08:37:57.316658+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:12.734757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.3b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318164+0000","last_change":"2024-07-17T08:37:55.902280+0000","last_active":"2024-07-17T08:37:57.318164+0000","last_peered":"2024-07-17T08:37:57.318164+0000","last_clean":"2024-07-17T08:37:57.318164+0000","last_became_active":"2024-07-17T08:37:55.900915+0000","last_became_peered":"2024-07-17T08:37:55.900915+0000","last_unstale":"2024-07-17T08:37:57.318164+0000","last_undegraded":"2024-07-17T08:37:57.318164+0000","last_fullsized":"2024-07-17T08:37:57.318164+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:18.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.3a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072020+0000","last_change":"2024-07-17T08:37:55.933720+0000","last_active":"2024-07-17T08:37:57.072020+0000","last_peered":"2024-07-17T08:37:57.072020+0000","last_clean":"2024-07-17T08:37:57.072020+0000","last_became_active":"2024-07-17T08:37:55.932719+0000","last_became_peered":"2024-07-17T08:37:55.932719+0000","last_unstale":"2024-07-17T08:37:57.072020+0000","last_undegraded":"2024-07-17T08:37:57.072020+0000","last_fullsized":"2024-07-17T08:37:57.072020+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:04.233876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.39","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896957+0000","last_change":"2024-07-17T08:37:55.926485+0000","last_active":"2024-07-17T08:37:56.896957+0000","last_peered":"2024-07-17T08:37:56.896957+0000","last_clean":"2024-07-17T08:37:56.896957+0000","last_became_active":"2024-07-17T08:37:55.926199+0000","last_became_peered":"2024-07-17T08:37:55.926199+0000","last_unstale":"2024-07-17T08:37:56.896957+0000","last_undegraded":"2024-07-17T08:37:56.896957+0000","last_fullsized":"2024-07-17T08:37:56.896957+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:10:53.649798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.38","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891451+0000","last_change":"2024-07-17T08:37:55.897804+0000","last_active":"2024-07-17T08:37:56.891451+0000","last_peered":"2024-07-17T08:37:56.891451+0000","last_clean":"2024-07-17T08:37:56.891451+0000","last_became_active":"2024-07-17T08:37:55.896914+0000","last_became_peered":"2024-07-17T08:37:55.896914+0000","last_unstale":"2024-07-17T08:37:56.891451+0000","last_undegraded":"2024-07-17T08:37:56.891451+0000","last_fullsized":"2024-07-17T08:37:56.891451+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:11.494291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.37","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916712+0000","last_change":"2024-07-17T08:37:55.929321+0000","last_active":"2024-07-17T08:37:56.916712+0000","last_peered":"2024-07-17T08:37:56.916712+0000","last_clean":"2024-07-17T08:37:56.916712+0000","last_became_active":"2024-07-17T08:37:55.928740+0000","last_became_peered":"2024-07-17T08:37:55.928740+0000","last_unstale":"2024-07-17T08:37:56.916712+0000","last_undegraded":"2024-07-17T08:37:56.916712+0000","last_fullsized":"2024-07-17T08:37:56.916712+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:09.463661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.36","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072347+0000","last_change":"2024-07-17T08:37:55.913782+0000","last_active":"2024-07-17T08:37:57.072347+0000","last_peered":"2024-07-17T08:37:57.072347+0000","last_clean":"2024-07-17T08:37:57.072347+0000","last_became_active":"2024-07-17T08:37:55.913487+0000","last_became_peered":"2024-07-17T08:37:55.913487+0000","last_unstale":"2024-07-17T08:37:57.072347+0000","last_undegraded":"2024-07-17T08:37:57.072347+0000","last_fullsized":"2024-07-17T08:37:57.072347+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:48.492794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.35","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916500+0000","last_change":"2024-07-17T08:37:55.905536+0000","last_active":"2024-07-17T08:37:56.916500+0000","last_peered":"2024-07-17T08:37:56.916500+0000","last_clean":"2024-07-17T08:37:56.916500+0000","last_became_active":"2024-07-17T08:37:55.904486+0000","last_became_peered":"2024-07-17T08:37:55.904486+0000","last_unstale":"2024-07-17T08:37:56.916500+0000","last_undegraded":"2024-07-17T08:37:56.916500+0000","last_fullsized":"2024-07-17T08:37:56.916500+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:30.068667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.34","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072332+0000","last_change":"2024-07-17T08:37:55.932460+0000","last_active":"2024-07-17T08:37:57.072332+0000","last_peered":"2024-07-17T08:37:57.072332+0000","last_clean":"2024-07-17T08:37:57.072332+0000","last_became_active":"2024-07-17T08:37:55.931498+0000","last_became_peered":"2024-07-17T08:37:55.931498+0000","last_unstale":"2024-07-17T08:37:57.072332+0000","last_undegraded":"2024-07-17T08:37:57.072332+0000","last_fullsized":"2024-07-17T08:37:57.072332+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:17.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.33","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318532+0000","last_change":"2024-07-17T08:37:55.900782+0000","last_active":"2024-07-17T08:37:57.318532+0000","last_peered":"2024-07-17T08:37:57.318532+0000","last_clean":"2024-07-17T08:37:57.318532+0000","last_became_active":"2024-07-17T08:37:55.898066+0000","last_became_peered":"2024-07-17T08:37:55.898066+0000","last_unstale":"2024-07-17T08:37:57.318532+0000","last_undegraded":"2024-07-17T08:37:57.318532+0000","last_fullsized":"2024-07-17T08:37:57.318532+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:31.462872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.32","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894592+0000","last_change":"2024-07-17T08:37:55.926882+0000","last_active":"2024-07-17T08:37:56.894592+0000","last_peered":"2024-07-17T08:37:56.894592+0000","last_clean":"2024-07-17T08:37:56.894592+0000","last_became_active":"2024-07-17T08:37:55.926582+0000","last_became_peered":"2024-07-17T08:37:55.926582+0000","last_unstale":"2024-07-17T08:37:56.894592+0000","last_undegraded":"2024-07-17T08:37:56.894592+0000","last_fullsized":"2024-07-17T08:37:56.894592+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:22.105350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.31","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894923+0000","last_change":"2024-07-17T08:37:55.912540+0000","last_active":"2024-07-17T08:37:56.894923+0000","last_peered":"2024-07-17T08:37:56.894923+0000","last_clean":"2024-07-17T08:37:56.894923+0000","last_became_active":"2024-07-17T08:37:55.912175+0000","last_became_peered":"2024-07-17T08:37:55.912175+0000","last_unstale":"2024-07-17T08:37:56.894923+0000","last_undegraded":"2024-07-17T08:37:56.894923+0000","last_fullsized":"2024-07-17T08:37:56.894923+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:46.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.30","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072458+0000","last_change":"2024-07-17T08:37:55.929117+0000","last_active":"2024-07-17T08:37:57.072458+0000","last_peered":"2024-07-17T08:37:57.072458+0000","last_clean":"2024-07-17T08:37:57.072458+0000","last_became_active":"2024-07-17T08:37:55.927459+0000","last_became_peered":"2024-07-17T08:37:55.927459+0000","last_unstale":"2024-07-17T08:37:57.072458+0000","last_undegraded":"2024-07-17T08:37:57.072458+0000","last_fullsized":"2024-07-17T08:37:57.072458+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:32.852676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.2f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916910+0000","last_change":"2024-07-17T08:37:55.929033+0000","last_active":"2024-07-17T08:37:56.916910+0000","last_peered":"2024-07-17T08:37:56.916910+0000","last_clean":"2024-07-17T08:37:56.916910+0000","last_became_active":"2024-07-17T08:37:55.928344+0000","last_became_peered":"2024-07-17T08:37:55.928344+0000","last_unstale":"2024-07-17T08:37:56.916910+0000","last_undegraded":"2024-07-17T08:37:56.916910+0000","last_fullsized":"2024-07-17T08:37:56.916910+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:03.428501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.2e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317466+0000","last_change":"2024-07-17T08:37:55.935689+0000","last_active":"2024-07-17T08:37:57.317466+0000","last_peered":"2024-07-17T08:37:57.317466+0000","last_clean":"2024-07-17T08:37:57.317466+0000","last_became_active":"2024-07-17T08:37:55.933180+0000","last_became_peered":"2024-07-17T08:37:55.933180+0000","last_unstale":"2024-07-17T08:37:57.317466+0000","last_undegraded":"2024-07-17T08:37:57.317466+0000","last_fullsized":"2024-07-17T08:37:57.317466+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:07.158541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315265+0000","last_change":"2024-07-17T08:37:55.929067+0000","last_active":"2024-07-17T08:37:57.315265+0000","last_peered":"2024-07-17T08:37:57.315265+0000","last_clean":"2024-07-17T08:37:57.315265+0000","last_became_active":"2024-07-17T08:37:55.927231+0000","last_became_peered":"2024-07-17T08:37:55.927231+0000","last_unstale":"2024-07-17T08:37:57.315265+0000","last_undegraded":"2024-07-17T08:37:57.315265+0000","last_fullsized":"2024-07-17T08:37:57.315265+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:01.396291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.2c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317144+0000","last_change":"2024-07-17T08:37:55.940657+0000","last_active":"2024-07-17T08:37:57.317144+0000","last_peered":"2024-07-17T08:37:57.317144+0000","last_clean":"2024-07-17T08:37:57.317144+0000","last_became_active":"2024-07-17T08:37:55.939990+0000","last_became_peered":"2024-07-17T08:37:55.939990+0000","last_unstale":"2024-07-17T08:37:57.317144+0000","last_undegraded":"2024-07-17T08:37:57.317144+0000","last_fullsized":"2024-07-17T08:37:57.317144+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:54.281581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895387+0000","last_change":"2024-07-17T08:37:55.935810+0000","last_active":"2024-07-17T08:37:56.895387+0000","last_peered":"2024-07-17T08:37:56.895387+0000","last_clean":"2024-07-17T08:37:56.895387+0000","last_became_active":"2024-07-17T08:37:55.934713+0000","last_became_peered":"2024-07-17T08:37:55.934713+0000","last_unstale":"2024-07-17T08:37:56.895387+0000","last_undegraded":"2024-07-17T08:37:56.895387+0000","last_fullsized":"2024-07-17T08:37:56.895387+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:57.778195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316319+0000","last_change":"2024-07-17T08:37:55.924642+0000","last_active":"2024-07-17T08:37:57.316319+0000","last_peered":"2024-07-17T08:37:57.316319+0000","last_clean":"2024-07-17T08:37:57.316319+0000","last_became_active":"2024-07-17T08:37:55.924220+0000","last_became_peered":"2024-07-17T08:37:55.924220+0000","last_unstale":"2024-07-17T08:37:57.316319+0000","last_undegraded":"2024-07-17T08:37:57.316319+0000","last_fullsized":"2024-07-17T08:37:57.316319+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:29.111667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.29","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316450+0000","last_change":"2024-07-17T08:37:55.925568+0000","last_active":"2024-07-17T08:37:57.316450+0000","last_peered":"2024-07-17T08:37:57.316450+0000","last_clean":"2024-07-17T08:37:57.316450+0000","last_became_active":"2024-07-17T08:37:55.923641+0000","last_became_peered":"2024-07-17T08:37:55.923641+0000","last_unstale":"2024-07-17T08:37:57.316450+0000","last_undegraded":"2024-07-17T08:37:57.316450+0000","last_fullsized":"2024-07-17T08:37:57.316450+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:26:43.526804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.28","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314743+0000","last_change":"2024-07-17T08:37:55.925410+0000","last_active":"2024-07-17T08:37:57.314743+0000","last_peered":"2024-07-17T08:37:57.314743+0000","last_clean":"2024-07-17T08:37:57.314743+0000","last_became_active":"2024-07-17T08:37:55.925102+0000","last_became_peered":"2024-07-17T08:37:55.925102+0000","last_unstale":"2024-07-17T08:37:57.314743+0000","last_undegraded":"2024-07-17T08:37:57.314743+0000","last_fullsized":"2024-07-17T08:37:57.314743+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:42.566987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.26","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916758+0000","last_change":"2024-07-17T08:37:55.942207+0000","last_active":"2024-07-17T08:37:56.916758+0000","last_peered":"2024-07-17T08:37:56.916758+0000","last_clean":"2024-07-17T08:37:56.916758+0000","last_became_active":"2024-07-17T08:37:55.941877+0000","last_became_peered":"2024-07-17T08:37:55.941877+0000","last_unstale":"2024-07-17T08:37:56.916758+0000","last_undegraded":"2024-07-17T08:37:56.916758+0000","last_fullsized":"2024-07-17T08:37:56.916758+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:36.527739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.25","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316922+0000","last_change":"2024-07-17T08:37:55.931487+0000","last_active":"2024-07-17T08:37:57.316922+0000","last_peered":"2024-07-17T08:37:57.316922+0000","last_clean":"2024-07-17T08:37:57.316922+0000","last_became_active":"2024-07-17T08:37:55.930455+0000","last_became_peered":"2024-07-17T08:37:55.930455+0000","last_unstale":"2024-07-17T08:37:57.316922+0000","last_undegraded":"2024-07-17T08:37:57.316922+0000","last_fullsized":"2024-07-17T08:37:57.316922+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:31:58.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.24","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916497+0000","last_change":"2024-07-17T08:37:55.900125+0000","last_active":"2024-07-17T08:37:56.916497+0000","last_peered":"2024-07-17T08:37:56.916497+0000","last_clean":"2024-07-17T08:37:56.916497+0000","last_became_active":"2024-07-17T08:37:55.899269+0000","last_became_peered":"2024-07-17T08:37:55.899269+0000","last_unstale":"2024-07-17T08:37:56.916497+0000","last_undegraded":"2024-07-17T08:37:56.916497+0000","last_fullsized":"2024-07-17T08:37:56.916497+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:01.205902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.23","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897178+0000","last_change":"2024-07-17T08:37:55.913990+0000","last_active":"2024-07-17T08:37:56.897178+0000","last_peered":"2024-07-17T08:37:56.897178+0000","last_clean":"2024-07-17T08:37:56.897178+0000","last_became_active":"2024-07-17T08:37:55.913554+0000","last_became_peered":"2024-07-17T08:37:55.913554+0000","last_unstale":"2024-07-17T08:37:56.897178+0000","last_undegraded":"2024-07-17T08:37:56.897178+0000","last_fullsized":"2024-07-17T08:37:56.897178+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:33.892553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.22","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894639+0000","last_change":"2024-07-17T08:37:55.922274+0000","last_active":"2024-07-17T08:37:56.894639+0000","last_peered":"2024-07-17T08:37:56.894639+0000","last_clean":"2024-07-17T08:37:56.894639+0000","last_became_active":"2024-07-17T08:37:55.920167+0000","last_became_peered":"2024-07-17T08:37:55.920167+0000","last_unstale":"2024-07-17T08:37:56.894639+0000","last_undegraded":"2024-07-17T08:37:56.894639+0000","last_fullsized":"2024-07-17T08:37:56.894639+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:07.428691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.21","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896338+0000","last_change":"2024-07-17T08:37:55.900007+0000","last_active":"2024-07-17T08:37:56.896338+0000","last_peered":"2024-07-17T08:37:56.896338+0000","last_clean":"2024-07-17T08:37:56.896338+0000","last_became_active":"2024-07-17T08:37:55.899436+0000","last_became_peered":"2024-07-17T08:37:55.899436+0000","last_unstale":"2024-07-17T08:37:56.896338+0000","last_undegraded":"2024-07-17T08:37:56.896338+0000","last_fullsized":"2024-07-17T08:37:56.896338+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:13.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.20","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316053+0000","last_change":"2024-07-17T08:37:55.940851+0000","last_active":"2024-07-17T08:37:57.316053+0000","last_peered":"2024-07-17T08:37:57.316053+0000","last_clean":"2024-07-17T08:37:57.316053+0000","last_became_active":"2024-07-17T08:37:55.938048+0000","last_became_peered":"2024-07-17T08:37:55.938048+0000","last_unstale":"2024-07-17T08:37:57.316053+0000","last_undegraded":"2024-07-17T08:37:57.316053+0000","last_fullsized":"2024-07-17T08:37:57.316053+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:30.627098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317557+0000","last_change":"2024-07-17T08:37:55.940935+0000","last_active":"2024-07-17T08:37:57.317557+0000","last_peered":"2024-07-17T08:37:57.317557+0000","last_clean":"2024-07-17T08:37:57.317557+0000","last_became_active":"2024-07-17T08:37:55.940373+0000","last_became_peered":"2024-07-17T08:37:55.940373+0000","last_unstale":"2024-07-17T08:37:57.317557+0000","last_undegraded":"2024-07-17T08:37:57.317557+0000","last_fullsized":"2024-07-17T08:37:57.317557+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:40:18.331419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1f","version":"276'4","reported_seq":18,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.319061+0000","last_change":"2024-07-17T08:37:54.869318+0000","last_active":"2024-07-17T08:37:57.319061+0000","last_peered":"2024-07-17T08:37:57.319061+0000","last_clean":"2024-07-17T08:37:57.319061+0000","last_became_active":"2024-07-17T08:37:54.868856+0000","last_became_peered":"2024-07-17T08:37:54.868856+0000","last_unstale":"2024-07-17T08:37:57.319061+0000","last_undegraded":"2024-07-17T08:37:57.319061+0000","last_fullsized":"2024-07-17T08:37:57.319061+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:04.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.1d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916574+0000","last_change":"2024-07-17T08:37:55.904361+0000","last_active":"2024-07-17T08:37:56.916574+0000","last_peered":"2024-07-17T08:37:56.916574+0000","last_clean":"2024-07-17T08:37:56.916574+0000","last_became_active":"2024-07-17T08:37:55.902327+0000","last_became_peered":"2024-07-17T08:37:55.902327+0000","last_unstale":"2024-07-17T08:37:56.916574+0000","last_undegraded":"2024-07-17T08:37:56.916574+0000","last_fullsized":"2024-07-17T08:37:56.916574+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:23.542036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1e","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317617+0000","last_change":"2024-07-17T08:37:54.885204+0000","last_active":"2024-07-17T08:37:57.317617+0000","last_peered":"2024-07-17T08:37:57.317617+0000","last_clean":"2024-07-17T08:37:57.317617+0000","last_became_active":"2024-07-17T08:37:54.884919+0000","last_became_peered":"2024-07-17T08:37:54.884919+0000","last_unstale":"2024-07-17T08:37:57.317617+0000","last_undegraded":"2024-07-17T08:37:57.317617+0000","last_fullsized":"2024-07-17T08:37:57.317617+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:48.751228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895925+0000","last_change":"2024-07-17T08:37:55.940456+0000","last_active":"2024-07-17T08:37:56.895925+0000","last_peered":"2024-07-17T08:37:56.895925+0000","last_clean":"2024-07-17T08:37:56.895925+0000","last_became_active":"2024-07-17T08:37:55.934301+0000","last_became_peered":"2024-07-17T08:37:55.934301+0000","last_unstale":"2024-07-17T08:37:56.895925+0000","last_undegraded":"2024-07-17T08:37:56.895925+0000","last_fullsized":"2024-07-17T08:37:56.895925+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:18.656593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315229+0000","last_change":"2024-07-17T08:37:54.877444+0000","last_active":"2024-07-17T08:37:57.315229+0000","last_peered":"2024-07-17T08:37:57.315229+0000","last_clean":"2024-07-17T08:37:57.315229+0000","last_became_active":"2024-07-17T08:37:54.877116+0000","last_became_peered":"2024-07-17T08:37:54.877116+0000","last_unstale":"2024-07-17T08:37:57.315229+0000","last_undegraded":"2024-07-17T08:37:57.315229+0000","last_fullsized":"2024-07-17T08:37:57.315229+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:08:53.774393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895707+0000","last_change":"2024-07-17T08:37:55.935438+0000","last_active":"2024-07-17T08:37:56.895707+0000","last_peered":"2024-07-17T08:37:56.895707+0000","last_clean":"2024-07-17T08:37:56.895707+0000","last_became_active":"2024-07-17T08:37:55.934992+0000","last_became_peered":"2024-07-17T08:37:55.934992+0000","last_unstale":"2024-07-17T08:37:56.895707+0000","last_undegraded":"2024-07-17T08:37:56.895707+0000","last_fullsized":"2024-07-17T08:37:56.895707+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:58.012812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895349+0000","last_change":"2024-07-17T08:37:54.879541+0000","last_active":"2024-07-17T08:37:56.895349+0000","last_peered":"2024-07-17T08:37:56.895349+0000","last_clean":"2024-07-17T08:37:56.895349+0000","last_became_active":"2024-07-17T08:37:54.879232+0000","last_became_peered":"2024-07-17T08:37:54.879232+0000","last_unstale":"2024-07-17T08:37:56.895349+0000","last_undegraded":"2024-07-17T08:37:56.895349+0000","last_fullsized":"2024-07-17T08:37:56.895349+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:07.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.18","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891586+0000","last_change":"2024-07-17T08:37:55.924869+0000","last_active":"2024-07-17T08:37:56.891586+0000","last_peered":"2024-07-17T08:37:56.891586+0000","last_clean":"2024-07-17T08:37:56.891586+0000","last_became_active":"2024-07-17T08:37:55.924524+0000","last_became_peered":"2024-07-17T08:37:55.924524+0000","last_unstale":"2024-07-17T08:37:56.891586+0000","last_undegraded":"2024-07-17T08:37:56.891586+0000","last_fullsized":"2024-07-17T08:37:56.891586+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:04.668852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1b","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916155+0000","last_change":"2024-07-17T08:37:54.881921+0000","last_active":"2024-07-17T08:37:56.916155+0000","last_peered":"2024-07-17T08:37:56.916155+0000","last_clean":"2024-07-17T08:37:56.916155+0000","last_became_active":"2024-07-17T08:37:54.879385+0000","last_became_peered":"2024-07-17T08:37:54.879385+0000","last_unstale":"2024-07-17T08:37:56.916155+0000","last_undegraded":"2024-07-17T08:37:56.916155+0000","last_fullsized":"2024-07-17T08:37:56.916155+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:45.896894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.19","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916084+0000","last_change":"2024-07-17T08:37:55.904524+0000","last_active":"2024-07-17T08:37:56.916084+0000","last_peered":"2024-07-17T08:37:56.916084+0000","last_clean":"2024-07-17T08:37:56.916084+0000","last_became_active":"2024-07-17T08:37:55.904048+0000","last_became_peered":"2024-07-17T08:37:55.904048+0000","last_unstale":"2024-07-17T08:37:56.916084+0000","last_undegraded":"2024-07-17T08:37:56.916084+0000","last_fullsized":"2024-07-17T08:37:56.916084+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:24.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1a","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314819+0000","last_change":"2024-07-17T08:37:54.875033+0000","last_active":"2024-07-17T08:37:57.314819+0000","last_peered":"2024-07-17T08:37:57.314819+0000","last_clean":"2024-07-17T08:37:57.314819+0000","last_became_active":"2024-07-17T08:37:54.873796+0000","last_became_peered":"2024-07-17T08:37:54.873796+0000","last_unstale":"2024-07-17T08:37:57.314819+0000","last_undegraded":"2024-07-17T08:37:57.314819+0000","last_fullsized":"2024-07-17T08:37:57.314819+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:36.476351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318106+0000","last_change":"2024-07-17T08:37:55.927377+0000","last_active":"2024-07-17T08:37:57.318106+0000","last_peered":"2024-07-17T08:37:57.318106+0000","last_clean":"2024-07-17T08:37:57.318106+0000","last_became_active":"2024-07-17T08:37:55.927088+0000","last_became_peered":"2024-07-17T08:37:55.927088+0000","last_unstale":"2024-07-17T08:37:57.318106+0000","last_undegraded":"2024-07-17T08:37:57.318106+0000","last_fullsized":"2024-07-17T08:37:57.318106+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:37.748259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894900+0000","last_change":"2024-07-17T08:37:54.878471+0000","last_active":"2024-07-17T08:37:56.894900+0000","last_peered":"2024-07-17T08:37:56.894900+0000","last_clean":"2024-07-17T08:37:56.894900+0000","last_became_active":"2024-07-17T08:37:54.878132+0000","last_became_peered":"2024-07-17T08:37:54.878132+0000","last_unstale":"2024-07-17T08:37:56.894900+0000","last_undegraded":"2024-07-17T08:37:56.894900+0000","last_fullsized":"2024-07-17T08:37:56.894900+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:27.714894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.1b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895411+0000","last_change":"2024-07-17T08:37:55.926999+0000","last_active":"2024-07-17T08:37:56.895411+0000","last_peered":"2024-07-17T08:37:56.895411+0000","last_clean":"2024-07-17T08:37:56.895411+0000","last_became_active":"2024-07-17T08:37:55.926696+0000","last_became_peered":"2024-07-17T08:37:55.926696+0000","last_unstale":"2024-07-17T08:37:56.895411+0000","last_undegraded":"2024-07-17T08:37:56.895411+0000","last_fullsized":"2024-07-17T08:37:56.895411+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:51.021344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896093+0000","last_change":"2024-07-17T08:37:54.872505+0000","last_active":"2024-07-17T08:37:56.896093+0000","last_peered":"2024-07-17T08:37:56.896093+0000","last_clean":"2024-07-17T08:37:56.896093+0000","last_became_active":"2024-07-17T08:37:54.870852+0000","last_became_peered":"2024-07-17T08:37:54.870852+0000","last_unstale":"2024-07-17T08:37:56.896093+0000","last_undegraded":"2024-07-17T08:37:56.896093+0000","last_fullsized":"2024-07-17T08:37:56.896093+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:49.541607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.14","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317995+0000","last_change":"2024-07-17T08:37:55.927109+0000","last_active":"2024-07-17T08:37:57.317995+0000","last_peered":"2024-07-17T08:37:57.317995+0000","last_clean":"2024-07-17T08:37:57.317995+0000","last_became_active":"2024-07-17T08:37:55.926812+0000","last_became_peered":"2024-07-17T08:37:55.926812+0000","last_unstale":"2024-07-17T08:37:57.317995+0000","last_undegraded":"2024-07-17T08:37:57.317995+0000","last_fullsized":"2024-07-17T08:37:57.317995+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:05.938989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.319096+0000","last_change":"2024-07-17T08:37:54.880325+0000","last_active":"2024-07-17T08:37:57.319096+0000","last_peered":"2024-07-17T08:37:57.319096+0000","last_clean":"2024-07-17T08:37:57.319096+0000","last_became_active":"2024-07-17T08:37:54.880042+0000","last_became_peered":"2024-07-17T08:37:54.880042+0000","last_unstale":"2024-07-17T08:37:57.319096+0000","last_undegraded":"2024-07-17T08:37:57.319096+0000","last_fullsized":"2024-07-17T08:37:57.319096+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:23.869528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.15","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072099+0000","last_change":"2024-07-17T08:37:55.937311+0000","last_active":"2024-07-17T08:37:57.072099+0000","last_peered":"2024-07-17T08:37:57.072099+0000","last_clean":"2024-07-17T08:37:57.072099+0000","last_became_active":"2024-07-17T08:37:55.937007+0000","last_became_peered":"2024-07-17T08:37:55.937007+0000","last_unstale":"2024-07-17T08:37:57.072099+0000","last_undegraded":"2024-07-17T08:37:57.072099+0000","last_fullsized":"2024-07-17T08:37:57.072099+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:39.496200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896091+0000","last_change":"2024-07-17T08:37:54.879592+0000","last_active":"2024-07-17T08:37:56.896091+0000","last_peered":"2024-07-17T08:37:56.896091+0000","last_clean":"2024-07-17T08:37:56.896091+0000","last_became_active":"2024-07-17T08:37:54.879300+0000","last_became_peered":"2024-07-17T08:37:54.879300+0000","last_unstale":"2024-07-17T08:37:56.896091+0000","last_undegraded":"2024-07-17T08:37:56.896091+0000","last_fullsized":"2024-07-17T08:37:56.896091+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:02.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.16","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895958+0000","last_change":"2024-07-17T08:37:55.939472+0000","last_active":"2024-07-17T08:37:56.895958+0000","last_peered":"2024-07-17T08:37:56.895958+0000","last_clean":"2024-07-17T08:37:56.895958+0000","last_became_active":"2024-07-17T08:37:55.933941+0000","last_became_peered":"2024-07-17T08:37:55.933941+0000","last_unstale":"2024-07-17T08:37:56.895958+0000","last_undegraded":"2024-07-17T08:37:56.895958+0000","last_fullsized":"2024-07-17T08:37:56.895958+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:18.641060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.15","version":"276'3","reported_seq":17,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895744+0000","last_change":"2024-07-17T08:37:54.866083+0000","last_active":"2024-07-17T08:37:56.895744+0000","last_peered":"2024-07-17T08:37:56.895744+0000","last_clean":"2024-07-17T08:37:56.895744+0000","last_became_active":"2024-07-17T08:37:54.865734+0000","last_became_peered":"2024-07-17T08:37:54.865734+0000","last_unstale":"2024-07-17T08:37:56.895744+0000","last_undegraded":"2024-07-17T08:37:56.895744+0000","last_fullsized":"2024-07-17T08:37:56.895744+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:02.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.17","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317961+0000","last_change":"2024-07-17T08:37:55.926090+0000","last_active":"2024-07-17T08:37:57.317961+0000","last_peered":"2024-07-17T08:37:57.317961+0000","last_clean":"2024-07-17T08:37:57.317961+0000","last_became_active":"2024-07-17T08:37:55.925398+0000","last_became_peered":"2024-07-17T08:37:55.925398+0000","last_unstale":"2024-07-17T08:37:57.317961+0000","last_undegraded":"2024-07-17T08:37:57.317961+0000","last_fullsized":"2024-07-17T08:37:57.317961+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:42.118669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.14","version":"276'6","reported_seq":20,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.892443+0000","last_change":"2024-07-17T08:37:54.861182+0000","last_active":"2024-07-17T08:37:56.892443+0000","last_peered":"2024-07-17T08:37:56.892443+0000","last_clean":"2024-07-17T08:37:56.892443+0000","last_became_active":"2024-07-17T08:37:54.859764+0000","last_became_peered":"2024-07-17T08:37:54.859764+0000","last_unstale":"2024-07-17T08:37:56.892443+0000","last_undegraded":"2024-07-17T08:37:56.892443+0000","last_fullsized":"2024-07-17T08:37:56.892443+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:36.646268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.10","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916164+0000","last_change":"2024-07-17T08:37:55.924520+0000","last_active":"2024-07-17T08:37:56.916164+0000","last_peered":"2024-07-17T08:37:56.916164+0000","last_clean":"2024-07-17T08:37:56.916164+0000","last_became_active":"2024-07-17T08:37:55.923220+0000","last_became_peered":"2024-07-17T08:37:55.923220+0000","last_unstale":"2024-07-17T08:37:56.916164+0000","last_undegraded":"2024-07-17T08:37:56.916164+0000","last_fullsized":"2024-07-17T08:37:56.916164+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:17.623877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.13","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072187+0000","last_change":"2024-07-17T08:37:54.883948+0000","last_active":"2024-07-17T08:37:57.072187+0000","last_peered":"2024-07-17T08:37:57.072187+0000","last_clean":"2024-07-17T08:37:57.072187+0000","last_became_active":"2024-07-17T08:37:54.883653+0000","last_became_peered":"2024-07-17T08:37:54.883653+0000","last_unstale":"2024-07-17T08:37:57.072187+0000","last_undegraded":"2024-07-17T08:37:57.072187+0000","last_fullsized":"2024-07-17T08:37:57.072187+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:08.471335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.11","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317211+0000","last_change":"2024-07-17T08:37:55.915787+0000","last_active":"2024-07-17T08:37:57.317211+0000","last_peered":"2024-07-17T08:37:57.317211+0000","last_clean":"2024-07-17T08:37:57.317211+0000","last_became_active":"2024-07-17T08:37:55.915488+0000","last_became_peered":"2024-07-17T08:37:55.915488+0000","last_unstale":"2024-07-17T08:37:57.317211+0000","last_undegraded":"2024-07-17T08:37:57.317211+0000","last_fullsized":"2024-07-17T08:37:57.317211+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:40.269087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316912+0000","last_change":"2024-07-17T08:37:54.867477+0000","last_active":"2024-07-17T08:37:57.316912+0000","last_peered":"2024-07-17T08:37:57.316912+0000","last_clean":"2024-07-17T08:37:57.316912+0000","last_became_active":"2024-07-17T08:37:54.864482+0000","last_became_peered":"2024-07-17T08:37:54.864482+0000","last_unstale":"2024-07-17T08:37:57.316912+0000","last_undegraded":"2024-07-17T08:37:57.316912+0000","last_fullsized":"2024-07-17T08:37:57.316912+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:00.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.12","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317616+0000","last_change":"2024-07-17T08:37:55.932502+0000","last_active":"2024-07-17T08:37:57.317616+0000","last_peered":"2024-07-17T08:37:57.317616+0000","last_clean":"2024-07-17T08:37:57.317616+0000","last_became_active":"2024-07-17T08:37:55.932226+0000","last_became_peered":"2024-07-17T08:37:55.932226+0000","last_unstale":"2024-07-17T08:37:57.317616+0000","last_undegraded":"2024-07-17T08:37:57.317616+0000","last_fullsized":"2024-07-17T08:37:57.317616+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:33.405003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071565+0000","last_change":"2024-07-17T08:37:54.879115+0000","last_active":"2024-07-17T08:37:57.071565+0000","last_peered":"2024-07-17T08:37:57.071565+0000","last_clean":"2024-07-17T08:37:57.071565+0000","last_became_active":"2024-07-17T08:37:54.878826+0000","last_became_peered":"2024-07-17T08:37:54.878826+0000","last_unstale":"2024-07-17T08:37:57.071565+0000","last_undegraded":"2024-07-17T08:37:57.071565+0000","last_fullsized":"2024-07-17T08:37:57.071565+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:32.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.13","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071883+0000","last_change":"2024-07-17T08:37:55.917428+0000","last_active":"2024-07-17T08:37:57.071883+0000","last_peered":"2024-07-17T08:37:57.071883+0000","last_clean":"2024-07-17T08:37:57.071883+0000","last_became_active":"2024-07-17T08:37:55.917112+0000","last_became_peered":"2024-07-17T08:37:55.917112+0000","last_unstale":"2024-07-17T08:37:57.071883+0000","last_undegraded":"2024-07-17T08:37:57.071883+0000","last_fullsized":"2024-07-17T08:37:57.071883+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:03.178509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.10","version":"276'8","reported_seq":22,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316825+0000","last_change":"2024-07-17T08:37:54.879000+0000","last_active":"2024-07-17T08:37:57.316825+0000","last_peered":"2024-07-17T08:37:57.316825+0000","last_clean":"2024-07-17T08:37:57.316825+0000","last_became_active":"2024-07-17T08:37:54.878695+0000","last_became_peered":"2024-07-17T08:37:54.878695+0000","last_unstale":"2024-07-17T08:37:57.316825+0000","last_undegraded":"2024-07-17T08:37:57.316825+0000","last_fullsized":"2024-07-17T08:37:57.316825+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:37.901450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891916+0000","last_change":"2024-07-17T08:37:55.937775+0000","last_active":"2024-07-17T08:37:56.891916+0000","last_peered":"2024-07-17T08:37:56.891916+0000","last_clean":"2024-07-17T08:37:56.891916+0000","last_became_active":"2024-07-17T08:37:55.936752+0000","last_became_peered":"2024-07-17T08:37:55.936752+0000","last_unstale":"2024-07-17T08:37:56.891916+0000","last_undegraded":"2024-07-17T08:37:56.891916+0000","last_fullsized":"2024-07-17T08:37:56.891916+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:07.459108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.f","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318440+0000","last_change":"2024-07-17T08:37:54.883043+0000","last_active":"2024-07-17T08:37:57.318440+0000","last_peered":"2024-07-17T08:37:57.318440+0000","last_clean":"2024-07-17T08:37:57.318440+0000","last_became_active":"2024-07-17T08:37:54.882744+0000","last_became_peered":"2024-07-17T08:37:54.882744+0000","last_unstale":"2024-07-17T08:37:57.318440+0000","last_undegraded":"2024-07-17T08:37:57.318440+0000","last_fullsized":"2024-07-17T08:37:57.318440+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:08.697845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318925+0000","last_change":"2024-07-17T08:37:55.938139+0000","last_active":"2024-07-17T08:37:57.318925+0000","last_peered":"2024-07-17T08:37:57.318925+0000","last_clean":"2024-07-17T08:37:57.318925+0000","last_became_active":"2024-07-17T08:37:55.937880+0000","last_became_peered":"2024-07-17T08:37:55.937880+0000","last_unstale":"2024-07-17T08:37:57.318925+0000","last_undegraded":"2024-07-17T08:37:57.318925+0000","last_fullsized":"2024-07-17T08:37:57.318925+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:59:33.238969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.e","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891966+0000","last_change":"2024-07-17T08:37:54.871263+0000","last_active":"2024-07-17T08:37:56.891966+0000","last_peered":"2024-07-17T08:37:56.891966+0000","last_clean":"2024-07-17T08:37:56.891966+0000","last_became_active":"2024-07-17T08:37:54.870926+0000","last_became_peered":"2024-07-17T08:37:54.870926+0000","last_unstale":"2024-07-17T08:37:56.891966+0000","last_undegraded":"2024-07-17T08:37:56.891966+0000","last_fullsized":"2024-07-17T08:37:56.891966+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:00.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072534+0000","last_change":"2024-07-17T08:37:55.943600+0000","last_active":"2024-07-17T08:37:57.072534+0000","last_peered":"2024-07-17T08:37:57.072534+0000","last_clean":"2024-07-17T08:37:57.072534+0000","last_became_active":"2024-07-17T08:37:55.942922+0000","last_became_peered":"2024-07-17T08:37:55.942922+0000","last_unstale":"2024-07-17T08:37:57.072534+0000","last_undegraded":"2024-07-17T08:37:57.072534+0000","last_fullsized":"2024-07-17T08:37:57.072534+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:40.751051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.d","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318832+0000","last_change":"2024-07-17T08:37:54.858446+0000","last_active":"2024-07-17T08:37:57.318832+0000","last_peered":"2024-07-17T08:37:57.318832+0000","last_clean":"2024-07-17T08:37:57.318832+0000","last_became_active":"2024-07-17T08:37:54.857711+0000","last_became_peered":"2024-07-17T08:37:54.857711+0000","last_unstale":"2024-07-17T08:37:57.318832+0000","last_undegraded":"2024-07-17T08:37:57.318832+0000","last_fullsized":"2024-07-17T08:37:57.318832+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:46.780221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315275+0000","last_change":"2024-07-17T08:37:55.934835+0000","last_active":"2024-07-17T08:37:57.315275+0000","last_peered":"2024-07-17T08:37:57.315275+0000","last_clean":"2024-07-17T08:37:57.315275+0000","last_became_active":"2024-07-17T08:37:55.930163+0000","last_became_peered":"2024-07-17T08:37:55.930163+0000","last_unstale":"2024-07-17T08:37:57.315275+0000","last_undegraded":"2024-07-17T08:37:57.315275+0000","last_fullsized":"2024-07-17T08:37:57.315275+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:19.641428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895491+0000","last_change":"2024-07-17T08:37:54.862046+0000","last_active":"2024-07-17T08:37:56.895491+0000","last_peered":"2024-07-17T08:37:56.895491+0000","last_clean":"2024-07-17T08:37:56.895491+0000","last_became_active":"2024-07-17T08:37:54.861323+0000","last_became_peered":"2024-07-17T08:37:54.861323+0000","last_unstale":"2024-07-17T08:37:56.895491+0000","last_undegraded":"2024-07-17T08:37:56.895491+0000","last_fullsized":"2024-07-17T08:37:56.895491+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:37.685131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.8","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316526+0000","last_change":"2024-07-17T08:37:55.934657+0000","last_active":"2024-07-17T08:37:57.316526+0000","last_peered":"2024-07-17T08:37:57.316526+0000","last_clean":"2024-07-17T08:37:57.316526+0000","last_became_active":"2024-07-17T08:37:55.932040+0000","last_became_peered":"2024-07-17T08:37:55.932040+0000","last_unstale":"2024-07-17T08:37:57.316526+0000","last_undegraded":"2024-07-17T08:37:57.316526+0000","last_fullsized":"2024-07-17T08:37:57.316526+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:00.442580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.b","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916184+0000","last_change":"2024-07-17T08:37:54.871667+0000","last_active":"2024-07-17T08:37:56.916184+0000","last_peered":"2024-07-17T08:37:56.916184+0000","last_clean":"2024-07-17T08:37:56.916184+0000","last_became_active":"2024-07-17T08:37:54.869953+0000","last_became_peered":"2024-07-17T08:37:54.869953+0000","last_unstale":"2024-07-17T08:37:56.916184+0000","last_undegraded":"2024-07-17T08:37:56.916184+0000","last_fullsized":"2024-07-17T08:37:56.916184+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:56.761813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.9","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916028+0000","last_change":"2024-07-17T08:37:55.903688+0000","last_active":"2024-07-17T08:37:56.916028+0000","last_peered":"2024-07-17T08:37:56.916028+0000","last_clean":"2024-07-17T08:37:56.916028+0000","last_became_active":"2024-07-17T08:37:55.902821+0000","last_became_peered":"2024-07-17T08:37:55.902821+0000","last_unstale":"2024-07-17T08:37:56.916028+0000","last_undegraded":"2024-07-17T08:37:56.916028+0000","last_fullsized":"2024-07-17T08:37:56.916028+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:03.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318343+0000","last_change":"2024-07-17T08:37:54.880741+0000","last_active":"2024-07-17T08:37:57.318343+0000","last_peered":"2024-07-17T08:37:57.318343+0000","last_clean":"2024-07-17T08:37:57.318343+0000","last_became_active":"2024-07-17T08:37:54.879266+0000","last_became_peered":"2024-07-17T08:37:54.879266+0000","last_unstale":"2024-07-17T08:37:57.318343+0000","last_undegraded":"2024-07-17T08:37:57.318343+0000","last_fullsized":"2024-07-17T08:37:57.318343+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:58.160356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317141+0000","last_change":"2024-07-17T08:37:55.897290+0000","last_active":"2024-07-17T08:37:57.317141+0000","last_peered":"2024-07-17T08:37:57.317141+0000","last_clean":"2024-07-17T08:37:57.317141+0000","last_became_active":"2024-07-17T08:37:55.896720+0000","last_became_peered":"2024-07-17T08:37:55.896720+0000","last_unstale":"2024-07-17T08:37:57.317141+0000","last_undegraded":"2024-07-17T08:37:57.317141+0000","last_fullsized":"2024-07-17T08:37:57.317141+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:30.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314850+0000","last_change":"2024-07-17T08:37:54.882698+0000","last_active":"2024-07-17T08:37:57.314850+0000","last_peered":"2024-07-17T08:37:57.314850+0000","last_clean":"2024-07-17T08:37:57.314850+0000","last_became_active":"2024-07-17T08:37:54.882341+0000","last_became_peered":"2024-07-17T08:37:54.882341+0000","last_unstale":"2024-07-17T08:37:57.314850+0000","last_undegraded":"2024-07-17T08:37:57.314850+0000","last_fullsized":"2024-07-17T08:37:57.314850+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:32.449419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071934+0000","last_change":"2024-07-17T08:37:55.923100+0000","last_active":"2024-07-17T08:37:57.071934+0000","last_peered":"2024-07-17T08:37:57.071934+0000","last_clean":"2024-07-17T08:37:57.071934+0000","last_became_active":"2024-07-17T08:37:55.922562+0000","last_became_peered":"2024-07-17T08:37:55.922562+0000","last_unstale":"2024-07-17T08:37:57.071934+0000","last_undegraded":"2024-07-17T08:37:57.071934+0000","last_fullsized":"2024-07-17T08:37:57.071934+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:26.794293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314530+0000","last_change":"2024-07-17T08:37:54.879382+0000","last_active":"2024-07-17T08:37:57.314530+0000","last_peered":"2024-07-17T08:37:57.314530+0000","last_clean":"2024-07-17T08:37:57.314530+0000","last_became_active":"2024-07-17T08:37:54.879118+0000","last_became_peered":"2024-07-17T08:37:54.879118+0000","last_unstale":"2024-07-17T08:37:57.314530+0000","last_undegraded":"2024-07-17T08:37:57.314530+0000","last_fullsized":"2024-07-17T08:37:57.314530+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:47.831422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.4","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318303+0000","last_change":"2024-07-17T08:37:55.931683+0000","last_active":"2024-07-17T08:37:57.318303+0000","last_peered":"2024-07-17T08:37:57.318303+0000","last_clean":"2024-07-17T08:37:57.318303+0000","last_became_active":"2024-07-17T08:37:55.930847+0000","last_became_peered":"2024-07-17T08:37:55.930847+0000","last_unstale":"2024-07-17T08:37:57.318303+0000","last_undegraded":"2024-07-17T08:37:57.318303+0000","last_fullsized":"2024-07-17T08:37:57.318303+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:26.203019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.7","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896834+0000","last_change":"2024-07-17T08:37:54.878437+0000","last_active":"2024-07-17T08:37:56.896834+0000","last_peered":"2024-07-17T08:37:56.896834+0000","last_clean":"2024-07-17T08:37:56.896834+0000","last_became_active":"2024-07-17T08:37:54.878082+0000","last_became_peered":"2024-07-17T08:37:54.878082+0000","last_unstale":"2024-07-17T08:37:56.896834+0000","last_undegraded":"2024-07-17T08:37:56.896834+0000","last_fullsized":"2024-07-17T08:37:56.896834+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:31.168922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.5","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897355+0000","last_change":"2024-07-17T08:37:55.914637+0000","last_active":"2024-07-17T08:37:56.897355+0000","last_peered":"2024-07-17T08:37:56.897355+0000","last_clean":"2024-07-17T08:37:56.897355+0000","last_became_active":"2024-07-17T08:37:55.913572+0000","last_became_peered":"2024-07-17T08:37:55.913572+0000","last_unstale":"2024-07-17T08:37:56.897355+0000","last_undegraded":"2024-07-17T08:37:56.897355+0000","last_fullsized":"2024-07-17T08:37:56.897355+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:01.031020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.6","version":"277'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.919195+0000","last_change":"2024-07-17T08:37:54.883040+0000","last_active":"2024-07-17T08:37:56.919195+0000","last_peered":"2024-07-17T08:37:56.919195+0000","last_clean":"2024-07-17T08:37:56.919195+0000","last_became_active":"2024-07-17T08:37:54.882229+0000","last_became_peered":"2024-07-17T08:37:54.882229+0000","last_unstale":"2024-07-17T08:37:56.919195+0000","last_undegraded":"2024-07-17T08:37:56.919195+0000","last_fullsized":"2024-07-17T08:37:56.919195+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:06.412309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.6","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896137+0000","last_change":"2024-07-17T08:37:55.940430+0000","last_active":"2024-07-17T08:37:56.896137+0000","last_peered":"2024-07-17T08:37:56.896137+0000","last_clean":"2024-07-17T08:37:56.896137+0000","last_became_active":"2024-07-17T08:37:55.936192+0000","last_became_peered":"2024-07-17T08:37:55.936192+0000","last_unstale":"2024-07-17T08:37:56.896137+0000","last_undegraded":"2024-07-17T08:37:56.896137+0000","last_fullsized":"2024-07-17T08:37:56.896137+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:42.965155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072139+0000","last_change":"2024-07-17T08:37:54.880183+0000","last_active":"2024-07-17T08:37:57.072139+0000","last_peered":"2024-07-17T08:37:57.072139+0000","last_clean":"2024-07-17T08:37:57.072139+0000","last_became_active":"2024-07-17T08:37:54.879898+0000","last_became_peered":"2024-07-17T08:37:54.879898+0000","last_unstale":"2024-07-17T08:37:57.072139+0000","last_undegraded":"2024-07-17T08:37:57.072139+0000","last_fullsized":"2024-07-17T08:37:57.072139+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:09.811953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.7","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.917184+0000","last_change":"2024-07-17T08:37:55.925796+0000","last_active":"2024-07-17T08:37:56.917184+0000","last_peered":"2024-07-17T08:37:56.917184+0000","last_clean":"2024-07-17T08:37:56.917184+0000","last_became_active":"2024-07-17T08:37:55.925492+0000","last_became_peered":"2024-07-17T08:37:55.925492+0000","last_unstale":"2024-07-17T08:37:56.917184+0000","last_undegraded":"2024-07-17T08:37:56.917184+0000","last_fullsized":"2024-07-17T08:37:56.917184+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:08.611355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318356+0000","last_change":"2024-07-17T08:37:54.857247+0000","last_active":"2024-07-17T08:37:57.318356+0000","last_peered":"2024-07-17T08:37:57.318356+0000","last_clean":"2024-07-17T08:37:57.318356+0000","last_became_active":"2024-07-17T08:37:54.855852+0000","last_became_peered":"2024-07-17T08:37:54.855852+0000","last_unstale":"2024-07-17T08:37:57.318356+0000","last_undegraded":"2024-07-17T08:37:57.318356+0000","last_fullsized":"2024-07-17T08:37:57.318356+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:36.646268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.0","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317398+0000","last_change":"2024-07-17T08:37:55.926338+0000","last_active":"2024-07-17T08:37:57.317398+0000","last_peered":"2024-07-17T08:37:57.317398+0000","last_clean":"2024-07-17T08:37:57.317398+0000","last_became_active":"2024-07-17T08:37:55.925605+0000","last_became_peered":"2024-07-17T08:37:55.925605+0000","last_unstale":"2024-07-17T08:37:57.317398+0000","last_undegraded":"2024-07-17T08:37:57.317398+0000","last_fullsized":"2024-07-17T08:37:57.317398+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:47.120043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891594+0000","last_change":"2024-07-17T08:37:54.864208+0000","last_active":"2024-07-17T08:37:56.891594+0000","last_peered":"2024-07-17T08:37:56.891594+0000","last_clean":"2024-07-17T08:37:56.891594+0000","last_became_active":"2024-07-17T08:37:54.862456+0000","last_became_peered":"2024-07-17T08:37:54.862456+0000","last_unstale":"2024-07-17T08:37:56.891594+0000","last_undegraded":"2024-07-17T08:37:56.891594+0000","last_fullsized":"2024-07-17T08:37:56.891594+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:55.411469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.1","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894605+0000","last_change":"2024-07-17T08:37:55.920620+0000","last_active":"2024-07-17T08:37:56.894605+0000","last_peered":"2024-07-17T08:37:56.894605+0000","last_clean":"2024-07-17T08:37:56.894605+0000","last_became_active":"2024-07-17T08:37:55.920267+0000","last_became_peered":"2024-07-17T08:37:55.920267+0000","last_unstale":"2024-07-17T08:37:56.894605+0000","last_undegraded":"2024-07-17T08:37:56.894605+0000","last_fullsized":"2024-07-17T08:37:56.894605+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:42:38.406052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894992+0000","last_change":"2024-07-17T08:37:54.878296+0000","last_active":"2024-07-17T08:37:56.894992+0000","last_peered":"2024-07-17T08:37:56.894992+0000","last_clean":"2024-07-17T08:37:56.894992+0000","last_became_active":"2024-07-17T08:37:54.878006+0000","last_became_peered":"2024-07-17T08:37:54.878006+0000","last_unstale":"2024-07-17T08:37:56.894992+0000","last_undegraded":"2024-07-17T08:37:56.894992+0000","last_fullsized":"2024-07-17T08:37:56.894992+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:45.472248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916028+0000","last_change":"2024-07-17T08:37:55.913251+0000","last_active":"2024-07-17T08:37:56.916028+0000","last_peered":"2024-07-17T08:37:56.916028+0000","last_clean":"2024-07-17T08:37:56.916028+0000","last_became_active":"2024-07-17T08:37:55.912940+0000","last_became_peered":"2024-07-17T08:37:55.912940+0000","last_unstale":"2024-07-17T08:37:56.916028+0000","last_undegraded":"2024-07-17T08:37:56.916028+0000","last_fullsized":"2024-07-17T08:37:56.916028+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:52.905376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071567+0000","last_change":"2024-07-17T08:37:54.877902+0000","last_active":"2024-07-17T08:37:57.071567+0000","last_peered":"2024-07-17T08:37:57.071567+0000","last_clean":"2024-07-17T08:37:57.071567+0000","last_became_active":"2024-07-17T08:37:54.877308+0000","last_became_peered":"2024-07-17T08:37:54.877308+0000","last_unstale":"2024-07-17T08:37:57.071567+0000","last_undegraded":"2024-07-17T08:37:57.071567+0000","last_fullsized":"2024-07-17T08:37:57.071567+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:07.777174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.3","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897448+0000","last_change":"2024-07-17T08:37:55.895136+0000","last_active":"2024-07-17T08:37:56.897448+0000","last_peered":"2024-07-17T08:37:56.897448+0000","last_clean":"2024-07-17T08:37:56.897448+0000","last_became_active":"2024-07-17T08:37:55.894461+0000","last_became_peered":"2024-07-17T08:37:55.894461+0000","last_unstale":"2024-07-17T08:37:56.897448+0000","last_undegraded":"2024-07-17T08:37:56.897448+0000","last_fullsized":"2024-07-17T08:37:56.897448+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:31.151213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314655+0000","last_change":"2024-07-17T08:37:54.878627+0000","last_active":"2024-07-17T08:37:57.314655+0000","last_peered":"2024-07-17T08:37:57.314655+0000","last_clean":"2024-07-17T08:37:57.314655+0000","last_became_active":"2024-07-17T08:37:54.878270+0000","last_became_peered":"2024-07-17T08:37:54.878270+0000","last_unstale":"2024-07-17T08:37:57.314655+0000","last_undegraded":"2024-07-17T08:37:57.314655+0000","last_fullsized":"2024-07-17T08:37:57.314655+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:34.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.27","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072713+0000","last_change":"2024-07-17T08:37:55.933081+0000","last_active":"2024-07-17T08:37:57.072713+0000","last_peered":"2024-07-17T08:37:57.072713+0000","last_clean":"2024-07-17T08:37:57.072713+0000","last_became_active":"2024-07-17T08:37:55.932738+0000","last_became_peered":"2024-07-17T08:37:55.932738+0000","last_unstale":"2024-07-17T08:37:57.072713+0000","last_undegraded":"2024-07-17T08:37:57.072713+0000","last_fullsized":"2024-07-17T08:37:57.072713+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:41.575151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"267'4582","reported_seq":5670,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317095+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:37:57.317095+0000","last_peered":"2024-07-17T08:37:57.317095+0000","last_clean":"2024-07-17T08:37:57.317095+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:37:57.317095+0000","last_undegraded":"2024-07-17T08:37:57.317095+0000","last_fullsized":"2024-07-17T08:37:57.317095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4582,"log_dups_size":0,"ondisk_log_size":4582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":421,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":38,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":37,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":421,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4582,"ondisk_log_size":4582,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543369,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75452,"kb_used_data":4016,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296388,"statfs":{"total":96636764160,"available":96559501312,"internally_reserved":0,"allocated":4112384,"data_stored":2518673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48512,"kb_used_data":3440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323328,"statfs":{"total":96636764160,"available":96587087872,"internally_reserved":0,"allocated":3522560,"data_stored":1928887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510665,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70712,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301128,"statfs":{"total":96636764160,"available":96564355072,"internally_reserved":0,"allocated":3518464,"data_stored":1928317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543370,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3526656,"data_stored":1923369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510664,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52672,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319168,"statfs":{"total":96636764160,"available":96582828032,"internally_reserved":0,"allocated":3518464,"data_stored":1928317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543368,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3448,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3530752,"data_stored":1929965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510665,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52652,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319188,"statfs":{"total":96636764160,"available":96582848512,"internally_reserved":0,"allocated":3502080,"data_stored":1921015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510664,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49080,"kb_used_data":4004,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322760,"statfs":{"total":96636764160,"available":96586506240,"internally_reserved":0,"allocated":4100096,"data_stored":2511473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:38:00.408 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.661+0000 7fb97c400640 1 -- 172.21.15.44:0/27871232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9740acaf0 msgr2=0x7fb97406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.661+0000 7fb97c400640 1 --2- 172.21.15.44:0/27871232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9740acaf0 0x7fb97406e090 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fb974070a70 tx=0x7fb96c02a6f0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 -- 172.21.15.44:0/27871232 shutdown_connections 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 --2- 172.21.15.44:0/27871232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9740acaf0 0x7fb97406e090 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 --2- 172.21.15.44:0/27871232 >> v2:172.21.15.175:3300/0 conn(0x7fb974153a50 0x7fb974159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 -- 172.21.15.44:0/27871232 >> 172.21.15.44:0/27871232 conn(0x7fb9740b0fb0 msgr2=0x7fb9740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 -- 172.21.15.44:0/27871232 shutdown_connections 2024-07-17T08:38:00.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.662+0000 7fb97c400640 1 -- 172.21.15.44:0/27871232 wait complete. 2024-07-17T08:38:00.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.664+0000 7fb97c400640 1 Processor -- start 2024-07-17T08:38:00.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.664+0000 7fb97c400640 1 -- start start 2024-07-17T08:38:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 0x7fb9741f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 0x7fb9740970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb97406f710 con 0x7fb97415a040 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb974154040 con 0x7fb9740acaf0 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb97406f890 con 0x7fb974153a50 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49896/0 (socket says 172.21.15.44:49896) 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb97a400640 1 -- 172.21.15.44:0/1595894770 learned_addr learned my addr 172.21.15.44:0/1595894770 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.665+0000 7fb979a00640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 0x7fb9741f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 -- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 msgr2=0x7fb9740970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb979000640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 0x7fb9740970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 0x7fb9740970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 -- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 msgr2=0x7fb9741f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 0x7fb9741f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 -- 172.21.15.44:0/1595894770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740b0390 con 0x7fb97415a040 2024-07-17T08:38:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb979a00640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 0x7fb9741f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:38:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb979000640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 0x7fb9740970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.666+0000 7fb97a400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0098c0 tx=0x7fb96c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.667+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c004240 con 0x7fb97415a040 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.667+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb96c0043e0 con 0x7fb97415a040 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.667+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c004810 con 0x7fb97415a040 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.667+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97414dbc0 con 0x7fb97415a040 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.667+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97414e400 con 0x7fb97415a040 2024-07-17T08:38:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.668+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97406a1a0 con 0x7fb97415a040 2024-07-17T08:38:00.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.669+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fb96c039040 con 0x7fb97415a040 2024-07-17T08:38:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.671+0000 7fb971600640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 0x7fb954079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.671+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fb96c0b9d70 con 0x7fb97415a040 2024-07-17T08:38:00.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.671+0000 7fb979a00640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 0x7fb954079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:00.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.673+0000 7fb979a00640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 0x7fb954079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9640058c0 tx=0x7fb964005850 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:00.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.676+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb97406a1a0 con 0x7fb97415a040 2024-07-17T08:38:00.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.883+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 --> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb97406e0f0 con 0x7fb954076cd0 2024-07-17T08:38:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.896+0000 7fb971600640 1 -- 172.21.15.44:0/1595894770 <== mgr.8516 v2:172.21.15.175:6834/2531126035 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252504 (secure 0 0 0) 0x7fb97406e0f0 con 0x7fb954076cd0 2024-07-17T08:38:00.901 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:38:00.902 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:38:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.902+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 msgr2=0x7fb954079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.902+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 0x7fb954079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9640058c0 tx=0x7fb964005850 comp rx=0 tx=0).stop 2024-07-17T08:38:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.902+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 msgr2=0x7fb974097620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:00.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.902+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0098c0 tx=0x7fb96c003ba0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 shutdown_connections 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb954076cd0 0x7fb954079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb97415a040 0x7fb974097620 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb974153a50 0x7fb9740970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 --2- 172.21.15.44:0/1595894770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9740acaf0 0x7fb9741f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 >> 172.21.15.44:0/1595894770 conn(0x7fb9740b0fb0 msgr2=0x7fb9741e5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 shutdown_connections 2024-07-17T08:38:00.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:00.903+0000 7fb97c400640 1 -- 172.21.15.44:0/1595894770 wait complete. 2024-07-17T08:38:00.922 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:37:59.582630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":466,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4621,"ondisk_log_size":4621,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":454440,"kb_used_data":28644,"kb_used_omap":12,"kb_used_meta":425651,"kb_avail":754520280,"statfs":{"total":773094113280,"available":772628766720,"internally_reserved":0,"allocated":29331456,"data_stored":16590016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":435867184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745602"},"pg_stats":[{"pgid":"38.3f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.892521+0000","last_change":"2024-07-17T08:37:55.929216+0000","last_active":"2024-07-17T08:37:56.892521+0000","last_peered":"2024-07-17T08:37:56.892521+0000","last_clean":"2024-07-17T08:37:56.892521+0000","last_became_active":"2024-07-17T08:37:55.928086+0000","last_became_peered":"2024-07-17T08:37:55.928086+0000","last_unstale":"2024-07-17T08:37:56.892521+0000","last_undegraded":"2024-07-17T08:37:56.892521+0000","last_fullsized":"2024-07-17T08:37:56.892521+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:13.508054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.3e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895710+0000","last_change":"2024-07-17T08:37:55.940150+0000","last_active":"2024-07-17T08:37:56.895710+0000","last_peered":"2024-07-17T08:37:56.895710+0000","last_clean":"2024-07-17T08:37:56.895710+0000","last_became_active":"2024-07-17T08:37:55.933485+0000","last_became_peered":"2024-07-17T08:37:55.933485+0000","last_unstale":"2024-07-17T08:37:56.895710+0000","last_undegraded":"2024-07-17T08:37:56.895710+0000","last_fullsized":"2024-07-17T08:37:56.895710+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:51:53.920815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.3d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315070+0000","last_change":"2024-07-17T08:37:55.931742+0000","last_active":"2024-07-17T08:37:57.315070+0000","last_peered":"2024-07-17T08:37:57.315070+0000","last_clean":"2024-07-17T08:37:57.315070+0000","last_became_active":"2024-07-17T08:37:55.929961+0000","last_became_peered":"2024-07-17T08:37:55.929961+0000","last_unstale":"2024-07-17T08:37:57.315070+0000","last_undegraded":"2024-07-17T08:37:57.315070+0000","last_fullsized":"2024-07-17T08:37:57.315070+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:03.944749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.3c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316658+0000","last_change":"2024-07-17T08:37:55.941611+0000","last_active":"2024-07-17T08:37:57.316658+0000","last_peered":"2024-07-17T08:37:57.316658+0000","last_clean":"2024-07-17T08:37:57.316658+0000","last_became_active":"2024-07-17T08:37:55.941250+0000","last_became_peered":"2024-07-17T08:37:55.941250+0000","last_unstale":"2024-07-17T08:37:57.316658+0000","last_undegraded":"2024-07-17T08:37:57.316658+0000","last_fullsized":"2024-07-17T08:37:57.316658+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:12.734757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.3b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318164+0000","last_change":"2024-07-17T08:37:55.902280+0000","last_active":"2024-07-17T08:37:57.318164+0000","last_peered":"2024-07-17T08:37:57.318164+0000","last_clean":"2024-07-17T08:37:57.318164+0000","last_became_active":"2024-07-17T08:37:55.900915+0000","last_became_peered":"2024-07-17T08:37:55.900915+0000","last_unstale":"2024-07-17T08:37:57.318164+0000","last_undegraded":"2024-07-17T08:37:57.318164+0000","last_fullsized":"2024-07-17T08:37:57.318164+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:18.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.3a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072020+0000","last_change":"2024-07-17T08:37:55.933720+0000","last_active":"2024-07-17T08:37:57.072020+0000","last_peered":"2024-07-17T08:37:57.072020+0000","last_clean":"2024-07-17T08:37:57.072020+0000","last_became_active":"2024-07-17T08:37:55.932719+0000","last_became_peered":"2024-07-17T08:37:55.932719+0000","last_unstale":"2024-07-17T08:37:57.072020+0000","last_undegraded":"2024-07-17T08:37:57.072020+0000","last_fullsized":"2024-07-17T08:37:57.072020+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:04.233876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.39","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896957+0000","last_change":"2024-07-17T08:37:55.926485+0000","last_active":"2024-07-17T08:37:56.896957+0000","last_peered":"2024-07-17T08:37:56.896957+0000","last_clean":"2024-07-17T08:37:56.896957+0000","last_became_active":"2024-07-17T08:37:55.926199+0000","last_became_peered":"2024-07-17T08:37:55.926199+0000","last_unstale":"2024-07-17T08:37:56.896957+0000","last_undegraded":"2024-07-17T08:37:56.896957+0000","last_fullsized":"2024-07-17T08:37:56.896957+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:10:53.649798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.38","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891451+0000","last_change":"2024-07-17T08:37:55.897804+0000","last_active":"2024-07-17T08:37:56.891451+0000","last_peered":"2024-07-17T08:37:56.891451+0000","last_clean":"2024-07-17T08:37:56.891451+0000","last_became_active":"2024-07-17T08:37:55.896914+0000","last_became_peered":"2024-07-17T08:37:55.896914+0000","last_unstale":"2024-07-17T08:37:56.891451+0000","last_undegraded":"2024-07-17T08:37:56.891451+0000","last_fullsized":"2024-07-17T08:37:56.891451+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:11.494291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.37","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916712+0000","last_change":"2024-07-17T08:37:55.929321+0000","last_active":"2024-07-17T08:37:56.916712+0000","last_peered":"2024-07-17T08:37:56.916712+0000","last_clean":"2024-07-17T08:37:56.916712+0000","last_became_active":"2024-07-17T08:37:55.928740+0000","last_became_peered":"2024-07-17T08:37:55.928740+0000","last_unstale":"2024-07-17T08:37:56.916712+0000","last_undegraded":"2024-07-17T08:37:56.916712+0000","last_fullsized":"2024-07-17T08:37:56.916712+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:09.463661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.36","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072347+0000","last_change":"2024-07-17T08:37:55.913782+0000","last_active":"2024-07-17T08:37:57.072347+0000","last_peered":"2024-07-17T08:37:57.072347+0000","last_clean":"2024-07-17T08:37:57.072347+0000","last_became_active":"2024-07-17T08:37:55.913487+0000","last_became_peered":"2024-07-17T08:37:55.913487+0000","last_unstale":"2024-07-17T08:37:57.072347+0000","last_undegraded":"2024-07-17T08:37:57.072347+0000","last_fullsized":"2024-07-17T08:37:57.072347+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:05:48.492794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.35","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916500+0000","last_change":"2024-07-17T08:37:55.905536+0000","last_active":"2024-07-17T08:37:56.916500+0000","last_peered":"2024-07-17T08:37:56.916500+0000","last_clean":"2024-07-17T08:37:56.916500+0000","last_became_active":"2024-07-17T08:37:55.904486+0000","last_became_peered":"2024-07-17T08:37:55.904486+0000","last_unstale":"2024-07-17T08:37:56.916500+0000","last_undegraded":"2024-07-17T08:37:56.916500+0000","last_fullsized":"2024-07-17T08:37:56.916500+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:32:30.068667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.34","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072332+0000","last_change":"2024-07-17T08:37:55.932460+0000","last_active":"2024-07-17T08:37:57.072332+0000","last_peered":"2024-07-17T08:37:57.072332+0000","last_clean":"2024-07-17T08:37:57.072332+0000","last_became_active":"2024-07-17T08:37:55.931498+0000","last_became_peered":"2024-07-17T08:37:55.931498+0000","last_unstale":"2024-07-17T08:37:57.072332+0000","last_undegraded":"2024-07-17T08:37:57.072332+0000","last_fullsized":"2024-07-17T08:37:57.072332+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:17.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.33","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318532+0000","last_change":"2024-07-17T08:37:55.900782+0000","last_active":"2024-07-17T08:37:57.318532+0000","last_peered":"2024-07-17T08:37:57.318532+0000","last_clean":"2024-07-17T08:37:57.318532+0000","last_became_active":"2024-07-17T08:37:55.898066+0000","last_became_peered":"2024-07-17T08:37:55.898066+0000","last_unstale":"2024-07-17T08:37:57.318532+0000","last_undegraded":"2024-07-17T08:37:57.318532+0000","last_fullsized":"2024-07-17T08:37:57.318532+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:31.462872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.32","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894592+0000","last_change":"2024-07-17T08:37:55.926882+0000","last_active":"2024-07-17T08:37:56.894592+0000","last_peered":"2024-07-17T08:37:56.894592+0000","last_clean":"2024-07-17T08:37:56.894592+0000","last_became_active":"2024-07-17T08:37:55.926582+0000","last_became_peered":"2024-07-17T08:37:55.926582+0000","last_unstale":"2024-07-17T08:37:56.894592+0000","last_undegraded":"2024-07-17T08:37:56.894592+0000","last_fullsized":"2024-07-17T08:37:56.894592+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:22.105350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.31","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894923+0000","last_change":"2024-07-17T08:37:55.912540+0000","last_active":"2024-07-17T08:37:56.894923+0000","last_peered":"2024-07-17T08:37:56.894923+0000","last_clean":"2024-07-17T08:37:56.894923+0000","last_became_active":"2024-07-17T08:37:55.912175+0000","last_became_peered":"2024-07-17T08:37:55.912175+0000","last_unstale":"2024-07-17T08:37:56.894923+0000","last_undegraded":"2024-07-17T08:37:56.894923+0000","last_fullsized":"2024-07-17T08:37:56.894923+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:46.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.30","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072458+0000","last_change":"2024-07-17T08:37:55.929117+0000","last_active":"2024-07-17T08:37:57.072458+0000","last_peered":"2024-07-17T08:37:57.072458+0000","last_clean":"2024-07-17T08:37:57.072458+0000","last_became_active":"2024-07-17T08:37:55.927459+0000","last_became_peered":"2024-07-17T08:37:55.927459+0000","last_unstale":"2024-07-17T08:37:57.072458+0000","last_undegraded":"2024-07-17T08:37:57.072458+0000","last_fullsized":"2024-07-17T08:37:57.072458+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:32.852676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.2f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916910+0000","last_change":"2024-07-17T08:37:55.929033+0000","last_active":"2024-07-17T08:37:56.916910+0000","last_peered":"2024-07-17T08:37:56.916910+0000","last_clean":"2024-07-17T08:37:56.916910+0000","last_became_active":"2024-07-17T08:37:55.928344+0000","last_became_peered":"2024-07-17T08:37:55.928344+0000","last_unstale":"2024-07-17T08:37:56.916910+0000","last_undegraded":"2024-07-17T08:37:56.916910+0000","last_fullsized":"2024-07-17T08:37:56.916910+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:49:03.428501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.2e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317466+0000","last_change":"2024-07-17T08:37:55.935689+0000","last_active":"2024-07-17T08:37:57.317466+0000","last_peered":"2024-07-17T08:37:57.317466+0000","last_clean":"2024-07-17T08:37:57.317466+0000","last_became_active":"2024-07-17T08:37:55.933180+0000","last_became_peered":"2024-07-17T08:37:55.933180+0000","last_unstale":"2024-07-17T08:37:57.317466+0000","last_undegraded":"2024-07-17T08:37:57.317466+0000","last_fullsized":"2024-07-17T08:37:57.317466+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:53:07.158541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315265+0000","last_change":"2024-07-17T08:37:55.929067+0000","last_active":"2024-07-17T08:37:57.315265+0000","last_peered":"2024-07-17T08:37:57.315265+0000","last_clean":"2024-07-17T08:37:57.315265+0000","last_became_active":"2024-07-17T08:37:55.927231+0000","last_became_peered":"2024-07-17T08:37:55.927231+0000","last_unstale":"2024-07-17T08:37:57.315265+0000","last_undegraded":"2024-07-17T08:37:57.315265+0000","last_fullsized":"2024-07-17T08:37:57.315265+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:01.396291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.2c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317144+0000","last_change":"2024-07-17T08:37:55.940657+0000","last_active":"2024-07-17T08:37:57.317144+0000","last_peered":"2024-07-17T08:37:57.317144+0000","last_clean":"2024-07-17T08:37:57.317144+0000","last_became_active":"2024-07-17T08:37:55.939990+0000","last_became_peered":"2024-07-17T08:37:55.939990+0000","last_unstale":"2024-07-17T08:37:57.317144+0000","last_undegraded":"2024-07-17T08:37:57.317144+0000","last_fullsized":"2024-07-17T08:37:57.317144+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:54.281581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.2b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895387+0000","last_change":"2024-07-17T08:37:55.935810+0000","last_active":"2024-07-17T08:37:56.895387+0000","last_peered":"2024-07-17T08:37:56.895387+0000","last_clean":"2024-07-17T08:37:56.895387+0000","last_became_active":"2024-07-17T08:37:55.934713+0000","last_became_peered":"2024-07-17T08:37:55.934713+0000","last_unstale":"2024-07-17T08:37:56.895387+0000","last_undegraded":"2024-07-17T08:37:56.895387+0000","last_fullsized":"2024-07-17T08:37:56.895387+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:57.778195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316319+0000","last_change":"2024-07-17T08:37:55.924642+0000","last_active":"2024-07-17T08:37:57.316319+0000","last_peered":"2024-07-17T08:37:57.316319+0000","last_clean":"2024-07-17T08:37:57.316319+0000","last_became_active":"2024-07-17T08:37:55.924220+0000","last_became_peered":"2024-07-17T08:37:55.924220+0000","last_unstale":"2024-07-17T08:37:57.316319+0000","last_undegraded":"2024-07-17T08:37:57.316319+0000","last_fullsized":"2024-07-17T08:37:57.316319+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:18:29.111667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.29","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316450+0000","last_change":"2024-07-17T08:37:55.925568+0000","last_active":"2024-07-17T08:37:57.316450+0000","last_peered":"2024-07-17T08:37:57.316450+0000","last_clean":"2024-07-17T08:37:57.316450+0000","last_became_active":"2024-07-17T08:37:55.923641+0000","last_became_peered":"2024-07-17T08:37:55.923641+0000","last_unstale":"2024-07-17T08:37:57.316450+0000","last_undegraded":"2024-07-17T08:37:57.316450+0000","last_fullsized":"2024-07-17T08:37:57.316450+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:26:43.526804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.28","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314743+0000","last_change":"2024-07-17T08:37:55.925410+0000","last_active":"2024-07-17T08:37:57.314743+0000","last_peered":"2024-07-17T08:37:57.314743+0000","last_clean":"2024-07-17T08:37:57.314743+0000","last_became_active":"2024-07-17T08:37:55.925102+0000","last_became_peered":"2024-07-17T08:37:55.925102+0000","last_unstale":"2024-07-17T08:37:57.314743+0000","last_undegraded":"2024-07-17T08:37:57.314743+0000","last_fullsized":"2024-07-17T08:37:57.314743+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:04:42.566987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.26","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916758+0000","last_change":"2024-07-17T08:37:55.942207+0000","last_active":"2024-07-17T08:37:56.916758+0000","last_peered":"2024-07-17T08:37:56.916758+0000","last_clean":"2024-07-17T08:37:56.916758+0000","last_became_active":"2024-07-17T08:37:55.941877+0000","last_became_peered":"2024-07-17T08:37:55.941877+0000","last_unstale":"2024-07-17T08:37:56.916758+0000","last_undegraded":"2024-07-17T08:37:56.916758+0000","last_fullsized":"2024-07-17T08:37:56.916758+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:36.527739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.25","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316922+0000","last_change":"2024-07-17T08:37:55.931487+0000","last_active":"2024-07-17T08:37:57.316922+0000","last_peered":"2024-07-17T08:37:57.316922+0000","last_clean":"2024-07-17T08:37:57.316922+0000","last_became_active":"2024-07-17T08:37:55.930455+0000","last_became_peered":"2024-07-17T08:37:55.930455+0000","last_unstale":"2024-07-17T08:37:57.316922+0000","last_undegraded":"2024-07-17T08:37:57.316922+0000","last_fullsized":"2024-07-17T08:37:57.316922+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:31:58.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.24","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916497+0000","last_change":"2024-07-17T08:37:55.900125+0000","last_active":"2024-07-17T08:37:56.916497+0000","last_peered":"2024-07-17T08:37:56.916497+0000","last_clean":"2024-07-17T08:37:56.916497+0000","last_became_active":"2024-07-17T08:37:55.899269+0000","last_became_peered":"2024-07-17T08:37:55.899269+0000","last_unstale":"2024-07-17T08:37:56.916497+0000","last_undegraded":"2024-07-17T08:37:56.916497+0000","last_fullsized":"2024-07-17T08:37:56.916497+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:01.205902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.23","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897178+0000","last_change":"2024-07-17T08:37:55.913990+0000","last_active":"2024-07-17T08:37:56.897178+0000","last_peered":"2024-07-17T08:37:56.897178+0000","last_clean":"2024-07-17T08:37:56.897178+0000","last_became_active":"2024-07-17T08:37:55.913554+0000","last_became_peered":"2024-07-17T08:37:55.913554+0000","last_unstale":"2024-07-17T08:37:56.897178+0000","last_undegraded":"2024-07-17T08:37:56.897178+0000","last_fullsized":"2024-07-17T08:37:56.897178+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:33.892553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.22","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894639+0000","last_change":"2024-07-17T08:37:55.922274+0000","last_active":"2024-07-17T08:37:56.894639+0000","last_peered":"2024-07-17T08:37:56.894639+0000","last_clean":"2024-07-17T08:37:56.894639+0000","last_became_active":"2024-07-17T08:37:55.920167+0000","last_became_peered":"2024-07-17T08:37:55.920167+0000","last_unstale":"2024-07-17T08:37:56.894639+0000","last_undegraded":"2024-07-17T08:37:56.894639+0000","last_fullsized":"2024-07-17T08:37:56.894639+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:07.428691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.21","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896338+0000","last_change":"2024-07-17T08:37:55.900007+0000","last_active":"2024-07-17T08:37:56.896338+0000","last_peered":"2024-07-17T08:37:56.896338+0000","last_clean":"2024-07-17T08:37:56.896338+0000","last_became_active":"2024-07-17T08:37:55.899436+0000","last_became_peered":"2024-07-17T08:37:55.899436+0000","last_unstale":"2024-07-17T08:37:56.896338+0000","last_undegraded":"2024-07-17T08:37:56.896338+0000","last_fullsized":"2024-07-17T08:37:56.896338+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:13.993741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.20","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316053+0000","last_change":"2024-07-17T08:37:55.940851+0000","last_active":"2024-07-17T08:37:57.316053+0000","last_peered":"2024-07-17T08:37:57.316053+0000","last_clean":"2024-07-17T08:37:57.316053+0000","last_became_active":"2024-07-17T08:37:55.938048+0000","last_became_peered":"2024-07-17T08:37:55.938048+0000","last_unstale":"2024-07-17T08:37:57.316053+0000","last_undegraded":"2024-07-17T08:37:57.316053+0000","last_fullsized":"2024-07-17T08:37:57.316053+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:30.627098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317557+0000","last_change":"2024-07-17T08:37:55.940935+0000","last_active":"2024-07-17T08:37:57.317557+0000","last_peered":"2024-07-17T08:37:57.317557+0000","last_clean":"2024-07-17T08:37:57.317557+0000","last_became_active":"2024-07-17T08:37:55.940373+0000","last_became_peered":"2024-07-17T08:37:55.940373+0000","last_unstale":"2024-07-17T08:37:57.317557+0000","last_undegraded":"2024-07-17T08:37:57.317557+0000","last_fullsized":"2024-07-17T08:37:57.317557+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:40:18.331419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1f","version":"276'4","reported_seq":18,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.319061+0000","last_change":"2024-07-17T08:37:54.869318+0000","last_active":"2024-07-17T08:37:57.319061+0000","last_peered":"2024-07-17T08:37:57.319061+0000","last_clean":"2024-07-17T08:37:57.319061+0000","last_became_active":"2024-07-17T08:37:54.868856+0000","last_became_peered":"2024-07-17T08:37:54.868856+0000","last_unstale":"2024-07-17T08:37:57.319061+0000","last_undegraded":"2024-07-17T08:37:57.319061+0000","last_fullsized":"2024-07-17T08:37:57.319061+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:04.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.1d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916574+0000","last_change":"2024-07-17T08:37:55.904361+0000","last_active":"2024-07-17T08:37:56.916574+0000","last_peered":"2024-07-17T08:37:56.916574+0000","last_clean":"2024-07-17T08:37:56.916574+0000","last_became_active":"2024-07-17T08:37:55.902327+0000","last_became_peered":"2024-07-17T08:37:55.902327+0000","last_unstale":"2024-07-17T08:37:56.916574+0000","last_undegraded":"2024-07-17T08:37:56.916574+0000","last_fullsized":"2024-07-17T08:37:56.916574+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:50:23.542036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1e","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317617+0000","last_change":"2024-07-17T08:37:54.885204+0000","last_active":"2024-07-17T08:37:57.317617+0000","last_peered":"2024-07-17T08:37:57.317617+0000","last_clean":"2024-07-17T08:37:57.317617+0000","last_became_active":"2024-07-17T08:37:54.884919+0000","last_became_peered":"2024-07-17T08:37:54.884919+0000","last_unstale":"2024-07-17T08:37:57.317617+0000","last_undegraded":"2024-07-17T08:37:57.317617+0000","last_fullsized":"2024-07-17T08:37:57.317617+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:48.751228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.1e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895925+0000","last_change":"2024-07-17T08:37:55.940456+0000","last_active":"2024-07-17T08:37:56.895925+0000","last_peered":"2024-07-17T08:37:56.895925+0000","last_clean":"2024-07-17T08:37:56.895925+0000","last_became_active":"2024-07-17T08:37:55.934301+0000","last_became_peered":"2024-07-17T08:37:55.934301+0000","last_unstale":"2024-07-17T08:37:56.895925+0000","last_undegraded":"2024-07-17T08:37:56.895925+0000","last_fullsized":"2024-07-17T08:37:56.895925+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:14:18.656593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315229+0000","last_change":"2024-07-17T08:37:54.877444+0000","last_active":"2024-07-17T08:37:57.315229+0000","last_peered":"2024-07-17T08:37:57.315229+0000","last_clean":"2024-07-17T08:37:57.315229+0000","last_became_active":"2024-07-17T08:37:54.877116+0000","last_became_peered":"2024-07-17T08:37:54.877116+0000","last_unstale":"2024-07-17T08:37:57.315229+0000","last_undegraded":"2024-07-17T08:37:57.315229+0000","last_fullsized":"2024-07-17T08:37:57.315229+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:08:53.774393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895707+0000","last_change":"2024-07-17T08:37:55.935438+0000","last_active":"2024-07-17T08:37:56.895707+0000","last_peered":"2024-07-17T08:37:56.895707+0000","last_clean":"2024-07-17T08:37:56.895707+0000","last_became_active":"2024-07-17T08:37:55.934992+0000","last_became_peered":"2024-07-17T08:37:55.934992+0000","last_unstale":"2024-07-17T08:37:56.895707+0000","last_undegraded":"2024-07-17T08:37:56.895707+0000","last_fullsized":"2024-07-17T08:37:56.895707+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:50:58.012812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895349+0000","last_change":"2024-07-17T08:37:54.879541+0000","last_active":"2024-07-17T08:37:56.895349+0000","last_peered":"2024-07-17T08:37:56.895349+0000","last_clean":"2024-07-17T08:37:56.895349+0000","last_became_active":"2024-07-17T08:37:54.879232+0000","last_became_peered":"2024-07-17T08:37:54.879232+0000","last_unstale":"2024-07-17T08:37:56.895349+0000","last_undegraded":"2024-07-17T08:37:56.895349+0000","last_fullsized":"2024-07-17T08:37:56.895349+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:07.453094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.18","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891586+0000","last_change":"2024-07-17T08:37:55.924869+0000","last_active":"2024-07-17T08:37:56.891586+0000","last_peered":"2024-07-17T08:37:56.891586+0000","last_clean":"2024-07-17T08:37:56.891586+0000","last_became_active":"2024-07-17T08:37:55.924524+0000","last_became_peered":"2024-07-17T08:37:55.924524+0000","last_unstale":"2024-07-17T08:37:56.891586+0000","last_undegraded":"2024-07-17T08:37:56.891586+0000","last_fullsized":"2024-07-17T08:37:56.891586+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:04.668852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1b","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916155+0000","last_change":"2024-07-17T08:37:54.881921+0000","last_active":"2024-07-17T08:37:56.916155+0000","last_peered":"2024-07-17T08:37:56.916155+0000","last_clean":"2024-07-17T08:37:56.916155+0000","last_became_active":"2024-07-17T08:37:54.879385+0000","last_became_peered":"2024-07-17T08:37:54.879385+0000","last_unstale":"2024-07-17T08:37:56.916155+0000","last_undegraded":"2024-07-17T08:37:56.916155+0000","last_fullsized":"2024-07-17T08:37:56.916155+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:44:45.896894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.19","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916084+0000","last_change":"2024-07-17T08:37:55.904524+0000","last_active":"2024-07-17T08:37:56.916084+0000","last_peered":"2024-07-17T08:37:56.916084+0000","last_clean":"2024-07-17T08:37:56.916084+0000","last_became_active":"2024-07-17T08:37:55.904048+0000","last_became_peered":"2024-07-17T08:37:55.904048+0000","last_unstale":"2024-07-17T08:37:56.916084+0000","last_undegraded":"2024-07-17T08:37:56.916084+0000","last_fullsized":"2024-07-17T08:37:56.916084+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:24.270355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1a","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314819+0000","last_change":"2024-07-17T08:37:54.875033+0000","last_active":"2024-07-17T08:37:57.314819+0000","last_peered":"2024-07-17T08:37:57.314819+0000","last_clean":"2024-07-17T08:37:57.314819+0000","last_became_active":"2024-07-17T08:37:54.873796+0000","last_became_peered":"2024-07-17T08:37:54.873796+0000","last_unstale":"2024-07-17T08:37:57.314819+0000","last_undegraded":"2024-07-17T08:37:57.314819+0000","last_fullsized":"2024-07-17T08:37:57.314819+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:36.476351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.1a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318106+0000","last_change":"2024-07-17T08:37:55.927377+0000","last_active":"2024-07-17T08:37:57.318106+0000","last_peered":"2024-07-17T08:37:57.318106+0000","last_clean":"2024-07-17T08:37:57.318106+0000","last_became_active":"2024-07-17T08:37:55.927088+0000","last_became_peered":"2024-07-17T08:37:55.927088+0000","last_unstale":"2024-07-17T08:37:57.318106+0000","last_undegraded":"2024-07-17T08:37:57.318106+0000","last_fullsized":"2024-07-17T08:37:57.318106+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:37.748259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894900+0000","last_change":"2024-07-17T08:37:54.878471+0000","last_active":"2024-07-17T08:37:56.894900+0000","last_peered":"2024-07-17T08:37:56.894900+0000","last_clean":"2024-07-17T08:37:56.894900+0000","last_became_active":"2024-07-17T08:37:54.878132+0000","last_became_peered":"2024-07-17T08:37:54.878132+0000","last_unstale":"2024-07-17T08:37:56.894900+0000","last_undegraded":"2024-07-17T08:37:56.894900+0000","last_fullsized":"2024-07-17T08:37:56.894900+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:27.714894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.1b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895411+0000","last_change":"2024-07-17T08:37:55.926999+0000","last_active":"2024-07-17T08:37:56.895411+0000","last_peered":"2024-07-17T08:37:56.895411+0000","last_clean":"2024-07-17T08:37:56.895411+0000","last_became_active":"2024-07-17T08:37:55.926696+0000","last_became_peered":"2024-07-17T08:37:55.926696+0000","last_unstale":"2024-07-17T08:37:56.895411+0000","last_undegraded":"2024-07-17T08:37:56.895411+0000","last_fullsized":"2024-07-17T08:37:56.895411+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:51.021344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896093+0000","last_change":"2024-07-17T08:37:54.872505+0000","last_active":"2024-07-17T08:37:56.896093+0000","last_peered":"2024-07-17T08:37:56.896093+0000","last_clean":"2024-07-17T08:37:56.896093+0000","last_became_active":"2024-07-17T08:37:54.870852+0000","last_became_peered":"2024-07-17T08:37:54.870852+0000","last_unstale":"2024-07-17T08:37:56.896093+0000","last_undegraded":"2024-07-17T08:37:56.896093+0000","last_fullsized":"2024-07-17T08:37:56.896093+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:49.541607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.14","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317995+0000","last_change":"2024-07-17T08:37:55.927109+0000","last_active":"2024-07-17T08:37:57.317995+0000","last_peered":"2024-07-17T08:37:57.317995+0000","last_clean":"2024-07-17T08:37:57.317995+0000","last_became_active":"2024-07-17T08:37:55.926812+0000","last_became_peered":"2024-07-17T08:37:55.926812+0000","last_unstale":"2024-07-17T08:37:57.317995+0000","last_undegraded":"2024-07-17T08:37:57.317995+0000","last_fullsized":"2024-07-17T08:37:57.317995+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:05.938989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.319096+0000","last_change":"2024-07-17T08:37:54.880325+0000","last_active":"2024-07-17T08:37:57.319096+0000","last_peered":"2024-07-17T08:37:57.319096+0000","last_clean":"2024-07-17T08:37:57.319096+0000","last_became_active":"2024-07-17T08:37:54.880042+0000","last_became_peered":"2024-07-17T08:37:54.880042+0000","last_unstale":"2024-07-17T08:37:57.319096+0000","last_undegraded":"2024-07-17T08:37:57.319096+0000","last_fullsized":"2024-07-17T08:37:57.319096+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:23.869528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.15","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072099+0000","last_change":"2024-07-17T08:37:55.937311+0000","last_active":"2024-07-17T08:37:57.072099+0000","last_peered":"2024-07-17T08:37:57.072099+0000","last_clean":"2024-07-17T08:37:57.072099+0000","last_became_active":"2024-07-17T08:37:55.937007+0000","last_became_peered":"2024-07-17T08:37:55.937007+0000","last_unstale":"2024-07-17T08:37:57.072099+0000","last_undegraded":"2024-07-17T08:37:57.072099+0000","last_fullsized":"2024-07-17T08:37:57.072099+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:39.496200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896091+0000","last_change":"2024-07-17T08:37:54.879592+0000","last_active":"2024-07-17T08:37:56.896091+0000","last_peered":"2024-07-17T08:37:56.896091+0000","last_clean":"2024-07-17T08:37:56.896091+0000","last_became_active":"2024-07-17T08:37:54.879300+0000","last_became_peered":"2024-07-17T08:37:54.879300+0000","last_unstale":"2024-07-17T08:37:56.896091+0000","last_undegraded":"2024-07-17T08:37:56.896091+0000","last_fullsized":"2024-07-17T08:37:56.896091+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:29:02.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.16","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895958+0000","last_change":"2024-07-17T08:37:55.939472+0000","last_active":"2024-07-17T08:37:56.895958+0000","last_peered":"2024-07-17T08:37:56.895958+0000","last_clean":"2024-07-17T08:37:56.895958+0000","last_became_active":"2024-07-17T08:37:55.933941+0000","last_became_peered":"2024-07-17T08:37:55.933941+0000","last_unstale":"2024-07-17T08:37:56.895958+0000","last_undegraded":"2024-07-17T08:37:56.895958+0000","last_fullsized":"2024-07-17T08:37:56.895958+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:18.641060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.15","version":"276'3","reported_seq":17,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895744+0000","last_change":"2024-07-17T08:37:54.866083+0000","last_active":"2024-07-17T08:37:56.895744+0000","last_peered":"2024-07-17T08:37:56.895744+0000","last_clean":"2024-07-17T08:37:56.895744+0000","last_became_active":"2024-07-17T08:37:54.865734+0000","last_became_peered":"2024-07-17T08:37:54.865734+0000","last_unstale":"2024-07-17T08:37:56.895744+0000","last_undegraded":"2024-07-17T08:37:56.895744+0000","last_fullsized":"2024-07-17T08:37:56.895744+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:02.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.17","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317961+0000","last_change":"2024-07-17T08:37:55.926090+0000","last_active":"2024-07-17T08:37:57.317961+0000","last_peered":"2024-07-17T08:37:57.317961+0000","last_clean":"2024-07-17T08:37:57.317961+0000","last_became_active":"2024-07-17T08:37:55.925398+0000","last_became_peered":"2024-07-17T08:37:55.925398+0000","last_unstale":"2024-07-17T08:37:57.317961+0000","last_undegraded":"2024-07-17T08:37:57.317961+0000","last_fullsized":"2024-07-17T08:37:57.317961+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:02:42.118669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.14","version":"276'6","reported_seq":20,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.892443+0000","last_change":"2024-07-17T08:37:54.861182+0000","last_active":"2024-07-17T08:37:56.892443+0000","last_peered":"2024-07-17T08:37:56.892443+0000","last_clean":"2024-07-17T08:37:56.892443+0000","last_became_active":"2024-07-17T08:37:54.859764+0000","last_became_peered":"2024-07-17T08:37:54.859764+0000","last_unstale":"2024-07-17T08:37:56.892443+0000","last_undegraded":"2024-07-17T08:37:56.892443+0000","last_fullsized":"2024-07-17T08:37:56.892443+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:36.646268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.10","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916164+0000","last_change":"2024-07-17T08:37:55.924520+0000","last_active":"2024-07-17T08:37:56.916164+0000","last_peered":"2024-07-17T08:37:56.916164+0000","last_clean":"2024-07-17T08:37:56.916164+0000","last_became_active":"2024-07-17T08:37:55.923220+0000","last_became_peered":"2024-07-17T08:37:55.923220+0000","last_unstale":"2024-07-17T08:37:56.916164+0000","last_undegraded":"2024-07-17T08:37:56.916164+0000","last_fullsized":"2024-07-17T08:37:56.916164+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:53:17.623877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.13","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072187+0000","last_change":"2024-07-17T08:37:54.883948+0000","last_active":"2024-07-17T08:37:57.072187+0000","last_peered":"2024-07-17T08:37:57.072187+0000","last_clean":"2024-07-17T08:37:57.072187+0000","last_became_active":"2024-07-17T08:37:54.883653+0000","last_became_peered":"2024-07-17T08:37:54.883653+0000","last_unstale":"2024-07-17T08:37:57.072187+0000","last_undegraded":"2024-07-17T08:37:57.072187+0000","last_fullsized":"2024-07-17T08:37:57.072187+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:08.471335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.11","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317211+0000","last_change":"2024-07-17T08:37:55.915787+0000","last_active":"2024-07-17T08:37:57.317211+0000","last_peered":"2024-07-17T08:37:57.317211+0000","last_clean":"2024-07-17T08:37:57.317211+0000","last_became_active":"2024-07-17T08:37:55.915488+0000","last_became_peered":"2024-07-17T08:37:55.915488+0000","last_unstale":"2024-07-17T08:37:57.317211+0000","last_undegraded":"2024-07-17T08:37:57.317211+0000","last_fullsized":"2024-07-17T08:37:57.317211+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:40.269087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316912+0000","last_change":"2024-07-17T08:37:54.867477+0000","last_active":"2024-07-17T08:37:57.316912+0000","last_peered":"2024-07-17T08:37:57.316912+0000","last_clean":"2024-07-17T08:37:57.316912+0000","last_became_active":"2024-07-17T08:37:54.864482+0000","last_became_peered":"2024-07-17T08:37:54.864482+0000","last_unstale":"2024-07-17T08:37:57.316912+0000","last_undegraded":"2024-07-17T08:37:57.316912+0000","last_fullsized":"2024-07-17T08:37:57.316912+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:00.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.12","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317616+0000","last_change":"2024-07-17T08:37:55.932502+0000","last_active":"2024-07-17T08:37:57.317616+0000","last_peered":"2024-07-17T08:37:57.317616+0000","last_clean":"2024-07-17T08:37:57.317616+0000","last_became_active":"2024-07-17T08:37:55.932226+0000","last_became_peered":"2024-07-17T08:37:55.932226+0000","last_unstale":"2024-07-17T08:37:57.317616+0000","last_undegraded":"2024-07-17T08:37:57.317616+0000","last_fullsized":"2024-07-17T08:37:57.317616+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:33.405003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071565+0000","last_change":"2024-07-17T08:37:54.879115+0000","last_active":"2024-07-17T08:37:57.071565+0000","last_peered":"2024-07-17T08:37:57.071565+0000","last_clean":"2024-07-17T08:37:57.071565+0000","last_became_active":"2024-07-17T08:37:54.878826+0000","last_became_peered":"2024-07-17T08:37:54.878826+0000","last_unstale":"2024-07-17T08:37:57.071565+0000","last_undegraded":"2024-07-17T08:37:57.071565+0000","last_fullsized":"2024-07-17T08:37:57.071565+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:32.882089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.13","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071883+0000","last_change":"2024-07-17T08:37:55.917428+0000","last_active":"2024-07-17T08:37:57.071883+0000","last_peered":"2024-07-17T08:37:57.071883+0000","last_clean":"2024-07-17T08:37:57.071883+0000","last_became_active":"2024-07-17T08:37:55.917112+0000","last_became_peered":"2024-07-17T08:37:55.917112+0000","last_unstale":"2024-07-17T08:37:57.071883+0000","last_undegraded":"2024-07-17T08:37:57.071883+0000","last_fullsized":"2024-07-17T08:37:57.071883+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:03.178509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.10","version":"276'8","reported_seq":22,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316825+0000","last_change":"2024-07-17T08:37:54.879000+0000","last_active":"2024-07-17T08:37:57.316825+0000","last_peered":"2024-07-17T08:37:57.316825+0000","last_clean":"2024-07-17T08:37:57.316825+0000","last_became_active":"2024-07-17T08:37:54.878695+0000","last_became_peered":"2024-07-17T08:37:54.878695+0000","last_unstale":"2024-07-17T08:37:57.316825+0000","last_undegraded":"2024-07-17T08:37:57.316825+0000","last_fullsized":"2024-07-17T08:37:57.316825+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:39:37.901450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.c","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891916+0000","last_change":"2024-07-17T08:37:55.937775+0000","last_active":"2024-07-17T08:37:56.891916+0000","last_peered":"2024-07-17T08:37:56.891916+0000","last_clean":"2024-07-17T08:37:56.891916+0000","last_became_active":"2024-07-17T08:37:55.936752+0000","last_became_peered":"2024-07-17T08:37:55.936752+0000","last_unstale":"2024-07-17T08:37:56.891916+0000","last_undegraded":"2024-07-17T08:37:56.891916+0000","last_fullsized":"2024-07-17T08:37:56.891916+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:07.459108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.f","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318440+0000","last_change":"2024-07-17T08:37:54.883043+0000","last_active":"2024-07-17T08:37:57.318440+0000","last_peered":"2024-07-17T08:37:57.318440+0000","last_clean":"2024-07-17T08:37:57.318440+0000","last_became_active":"2024-07-17T08:37:54.882744+0000","last_became_peered":"2024-07-17T08:37:54.882744+0000","last_unstale":"2024-07-17T08:37:57.318440+0000","last_undegraded":"2024-07-17T08:37:57.318440+0000","last_fullsized":"2024-07-17T08:37:57.318440+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:08.697845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.d","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318925+0000","last_change":"2024-07-17T08:37:55.938139+0000","last_active":"2024-07-17T08:37:57.318925+0000","last_peered":"2024-07-17T08:37:57.318925+0000","last_clean":"2024-07-17T08:37:57.318925+0000","last_became_active":"2024-07-17T08:37:55.937880+0000","last_became_peered":"2024-07-17T08:37:55.937880+0000","last_unstale":"2024-07-17T08:37:57.318925+0000","last_undegraded":"2024-07-17T08:37:57.318925+0000","last_fullsized":"2024-07-17T08:37:57.318925+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:59:33.238969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.e","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891966+0000","last_change":"2024-07-17T08:37:54.871263+0000","last_active":"2024-07-17T08:37:56.891966+0000","last_peered":"2024-07-17T08:37:56.891966+0000","last_clean":"2024-07-17T08:37:56.891966+0000","last_became_active":"2024-07-17T08:37:54.870926+0000","last_became_peered":"2024-07-17T08:37:54.870926+0000","last_unstale":"2024-07-17T08:37:56.891966+0000","last_undegraded":"2024-07-17T08:37:56.891966+0000","last_fullsized":"2024-07-17T08:37:56.891966+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:00.284335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.e","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072534+0000","last_change":"2024-07-17T08:37:55.943600+0000","last_active":"2024-07-17T08:37:57.072534+0000","last_peered":"2024-07-17T08:37:57.072534+0000","last_clean":"2024-07-17T08:37:57.072534+0000","last_became_active":"2024-07-17T08:37:55.942922+0000","last_became_peered":"2024-07-17T08:37:55.942922+0000","last_unstale":"2024-07-17T08:37:57.072534+0000","last_undegraded":"2024-07-17T08:37:57.072534+0000","last_fullsized":"2024-07-17T08:37:57.072534+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:21:40.751051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.d","version":"276'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318832+0000","last_change":"2024-07-17T08:37:54.858446+0000","last_active":"2024-07-17T08:37:57.318832+0000","last_peered":"2024-07-17T08:37:57.318832+0000","last_clean":"2024-07-17T08:37:57.318832+0000","last_became_active":"2024-07-17T08:37:54.857711+0000","last_became_peered":"2024-07-17T08:37:54.857711+0000","last_unstale":"2024-07-17T08:37:57.318832+0000","last_undegraded":"2024-07-17T08:37:57.318832+0000","last_fullsized":"2024-07-17T08:37:57.318832+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:46.780221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.f","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.315275+0000","last_change":"2024-07-17T08:37:55.934835+0000","last_active":"2024-07-17T08:37:57.315275+0000","last_peered":"2024-07-17T08:37:57.315275+0000","last_clean":"2024-07-17T08:37:57.315275+0000","last_became_active":"2024-07-17T08:37:55.930163+0000","last_became_peered":"2024-07-17T08:37:55.930163+0000","last_unstale":"2024-07-17T08:37:57.315275+0000","last_undegraded":"2024-07-17T08:37:57.315275+0000","last_fullsized":"2024-07-17T08:37:57.315275+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:36:19.641428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.895491+0000","last_change":"2024-07-17T08:37:54.862046+0000","last_active":"2024-07-17T08:37:56.895491+0000","last_peered":"2024-07-17T08:37:56.895491+0000","last_clean":"2024-07-17T08:37:56.895491+0000","last_became_active":"2024-07-17T08:37:54.861323+0000","last_became_peered":"2024-07-17T08:37:54.861323+0000","last_unstale":"2024-07-17T08:37:56.895491+0000","last_undegraded":"2024-07-17T08:37:56.895491+0000","last_fullsized":"2024-07-17T08:37:56.895491+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:37.685131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.8","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.316526+0000","last_change":"2024-07-17T08:37:55.934657+0000","last_active":"2024-07-17T08:37:57.316526+0000","last_peered":"2024-07-17T08:37:57.316526+0000","last_clean":"2024-07-17T08:37:57.316526+0000","last_became_active":"2024-07-17T08:37:55.932040+0000","last_became_peered":"2024-07-17T08:37:55.932040+0000","last_unstale":"2024-07-17T08:37:57.316526+0000","last_undegraded":"2024-07-17T08:37:57.316526+0000","last_fullsized":"2024-07-17T08:37:57.316526+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:01:00.442580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.b","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916184+0000","last_change":"2024-07-17T08:37:54.871667+0000","last_active":"2024-07-17T08:37:56.916184+0000","last_peered":"2024-07-17T08:37:56.916184+0000","last_clean":"2024-07-17T08:37:56.916184+0000","last_became_active":"2024-07-17T08:37:54.869953+0000","last_became_peered":"2024-07-17T08:37:54.869953+0000","last_unstale":"2024-07-17T08:37:56.916184+0000","last_undegraded":"2024-07-17T08:37:56.916184+0000","last_fullsized":"2024-07-17T08:37:56.916184+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:13:56.761813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.9","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916028+0000","last_change":"2024-07-17T08:37:55.903688+0000","last_active":"2024-07-17T08:37:56.916028+0000","last_peered":"2024-07-17T08:37:56.916028+0000","last_clean":"2024-07-17T08:37:56.916028+0000","last_became_active":"2024-07-17T08:37:55.902821+0000","last_became_peered":"2024-07-17T08:37:55.902821+0000","last_unstale":"2024-07-17T08:37:56.916028+0000","last_undegraded":"2024-07-17T08:37:56.916028+0000","last_fullsized":"2024-07-17T08:37:56.916028+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:03.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318343+0000","last_change":"2024-07-17T08:37:54.880741+0000","last_active":"2024-07-17T08:37:57.318343+0000","last_peered":"2024-07-17T08:37:57.318343+0000","last_clean":"2024-07-17T08:37:57.318343+0000","last_became_active":"2024-07-17T08:37:54.879266+0000","last_became_peered":"2024-07-17T08:37:54.879266+0000","last_unstale":"2024-07-17T08:37:57.318343+0000","last_undegraded":"2024-07-17T08:37:57.318343+0000","last_fullsized":"2024-07-17T08:37:57.318343+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:58.160356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"38.a","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317141+0000","last_change":"2024-07-17T08:37:55.897290+0000","last_active":"2024-07-17T08:37:57.317141+0000","last_peered":"2024-07-17T08:37:57.317141+0000","last_clean":"2024-07-17T08:37:57.317141+0000","last_became_active":"2024-07-17T08:37:55.896720+0000","last_became_peered":"2024-07-17T08:37:55.896720+0000","last_unstale":"2024-07-17T08:37:57.317141+0000","last_undegraded":"2024-07-17T08:37:57.317141+0000","last_fullsized":"2024-07-17T08:37:57.317141+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:30.562905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314850+0000","last_change":"2024-07-17T08:37:54.882698+0000","last_active":"2024-07-17T08:37:57.314850+0000","last_peered":"2024-07-17T08:37:57.314850+0000","last_clean":"2024-07-17T08:37:57.314850+0000","last_became_active":"2024-07-17T08:37:54.882341+0000","last_became_peered":"2024-07-17T08:37:54.882341+0000","last_unstale":"2024-07-17T08:37:57.314850+0000","last_undegraded":"2024-07-17T08:37:57.314850+0000","last_fullsized":"2024-07-17T08:37:57.314850+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:32.449419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.b","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071934+0000","last_change":"2024-07-17T08:37:55.923100+0000","last_active":"2024-07-17T08:37:57.071934+0000","last_peered":"2024-07-17T08:37:57.071934+0000","last_clean":"2024-07-17T08:37:57.071934+0000","last_became_active":"2024-07-17T08:37:55.922562+0000","last_became_peered":"2024-07-17T08:37:55.922562+0000","last_unstale":"2024-07-17T08:37:57.071934+0000","last_undegraded":"2024-07-17T08:37:57.071934+0000","last_fullsized":"2024-07-17T08:37:57.071934+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:26:26.794293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314530+0000","last_change":"2024-07-17T08:37:54.879382+0000","last_active":"2024-07-17T08:37:57.314530+0000","last_peered":"2024-07-17T08:37:57.314530+0000","last_clean":"2024-07-17T08:37:57.314530+0000","last_became_active":"2024-07-17T08:37:54.879118+0000","last_became_peered":"2024-07-17T08:37:54.879118+0000","last_unstale":"2024-07-17T08:37:57.314530+0000","last_undegraded":"2024-07-17T08:37:57.314530+0000","last_fullsized":"2024-07-17T08:37:57.314530+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:47.831422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.4","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318303+0000","last_change":"2024-07-17T08:37:55.931683+0000","last_active":"2024-07-17T08:37:57.318303+0000","last_peered":"2024-07-17T08:37:57.318303+0000","last_clean":"2024-07-17T08:37:57.318303+0000","last_became_active":"2024-07-17T08:37:55.930847+0000","last_became_peered":"2024-07-17T08:37:55.930847+0000","last_unstale":"2024-07-17T08:37:57.318303+0000","last_undegraded":"2024-07-17T08:37:57.318303+0000","last_fullsized":"2024-07-17T08:37:57.318303+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:26.203019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.7","version":"276'2","reported_seq":16,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896834+0000","last_change":"2024-07-17T08:37:54.878437+0000","last_active":"2024-07-17T08:37:56.896834+0000","last_peered":"2024-07-17T08:37:56.896834+0000","last_clean":"2024-07-17T08:37:56.896834+0000","last_became_active":"2024-07-17T08:37:54.878082+0000","last_became_peered":"2024-07-17T08:37:54.878082+0000","last_unstale":"2024-07-17T08:37:56.896834+0000","last_undegraded":"2024-07-17T08:37:56.896834+0000","last_fullsized":"2024-07-17T08:37:56.896834+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:31.168922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"38.5","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897355+0000","last_change":"2024-07-17T08:37:55.914637+0000","last_active":"2024-07-17T08:37:56.897355+0000","last_peered":"2024-07-17T08:37:56.897355+0000","last_clean":"2024-07-17T08:37:56.897355+0000","last_became_active":"2024-07-17T08:37:55.913572+0000","last_became_peered":"2024-07-17T08:37:55.913572+0000","last_unstale":"2024-07-17T08:37:56.897355+0000","last_undegraded":"2024-07-17T08:37:56.897355+0000","last_fullsized":"2024-07-17T08:37:56.897355+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:01.031020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.6","version":"277'1","reported_seq":15,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.919195+0000","last_change":"2024-07-17T08:37:54.883040+0000","last_active":"2024-07-17T08:37:56.919195+0000","last_peered":"2024-07-17T08:37:56.919195+0000","last_clean":"2024-07-17T08:37:56.919195+0000","last_became_active":"2024-07-17T08:37:54.882229+0000","last_became_peered":"2024-07-17T08:37:54.882229+0000","last_unstale":"2024-07-17T08:37:56.919195+0000","last_undegraded":"2024-07-17T08:37:56.919195+0000","last_fullsized":"2024-07-17T08:37:56.919195+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:30:06.412309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"38.6","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.896137+0000","last_change":"2024-07-17T08:37:55.940430+0000","last_active":"2024-07-17T08:37:56.896137+0000","last_peered":"2024-07-17T08:37:56.896137+0000","last_clean":"2024-07-17T08:37:56.896137+0000","last_became_active":"2024-07-17T08:37:55.936192+0000","last_became_peered":"2024-07-17T08:37:55.936192+0000","last_unstale":"2024-07-17T08:37:56.896137+0000","last_undegraded":"2024-07-17T08:37:56.896137+0000","last_fullsized":"2024-07-17T08:37:56.896137+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:42.965155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072139+0000","last_change":"2024-07-17T08:37:54.880183+0000","last_active":"2024-07-17T08:37:57.072139+0000","last_peered":"2024-07-17T08:37:57.072139+0000","last_clean":"2024-07-17T08:37:57.072139+0000","last_became_active":"2024-07-17T08:37:54.879898+0000","last_became_peered":"2024-07-17T08:37:54.879898+0000","last_unstale":"2024-07-17T08:37:57.072139+0000","last_undegraded":"2024-07-17T08:37:57.072139+0000","last_fullsized":"2024-07-17T08:37:57.072139+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:09.811953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.7","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.917184+0000","last_change":"2024-07-17T08:37:55.925796+0000","last_active":"2024-07-17T08:37:56.917184+0000","last_peered":"2024-07-17T08:37:56.917184+0000","last_clean":"2024-07-17T08:37:56.917184+0000","last_became_active":"2024-07-17T08:37:55.925492+0000","last_became_peered":"2024-07-17T08:37:55.925492+0000","last_unstale":"2024-07-17T08:37:56.917184+0000","last_undegraded":"2024-07-17T08:37:56.917184+0000","last_fullsized":"2024-07-17T08:37:56.917184+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:08.611355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.318356+0000","last_change":"2024-07-17T08:37:54.857247+0000","last_active":"2024-07-17T08:37:57.318356+0000","last_peered":"2024-07-17T08:37:57.318356+0000","last_clean":"2024-07-17T08:37:57.318356+0000","last_became_active":"2024-07-17T08:37:54.855852+0000","last_became_peered":"2024-07-17T08:37:54.855852+0000","last_unstale":"2024-07-17T08:37:57.318356+0000","last_undegraded":"2024-07-17T08:37:57.318356+0000","last_fullsized":"2024-07-17T08:37:57.318356+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:36.646268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"38.0","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317398+0000","last_change":"2024-07-17T08:37:55.926338+0000","last_active":"2024-07-17T08:37:57.317398+0000","last_peered":"2024-07-17T08:37:57.317398+0000","last_clean":"2024-07-17T08:37:57.317398+0000","last_became_active":"2024-07-17T08:37:55.925605+0000","last_became_peered":"2024-07-17T08:37:55.925605+0000","last_unstale":"2024-07-17T08:37:57.317398+0000","last_undegraded":"2024-07-17T08:37:57.317398+0000","last_fullsized":"2024-07-17T08:37:57.317398+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:47.120043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.891594+0000","last_change":"2024-07-17T08:37:54.864208+0000","last_active":"2024-07-17T08:37:56.891594+0000","last_peered":"2024-07-17T08:37:56.891594+0000","last_clean":"2024-07-17T08:37:56.891594+0000","last_became_active":"2024-07-17T08:37:54.862456+0000","last_became_peered":"2024-07-17T08:37:54.862456+0000","last_unstale":"2024-07-17T08:37:56.891594+0000","last_undegraded":"2024-07-17T08:37:56.891594+0000","last_fullsized":"2024-07-17T08:37:56.891594+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:35:55.411469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"38.1","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894605+0000","last_change":"2024-07-17T08:37:55.920620+0000","last_active":"2024-07-17T08:37:56.894605+0000","last_peered":"2024-07-17T08:37:56.894605+0000","last_clean":"2024-07-17T08:37:56.894605+0000","last_became_active":"2024-07-17T08:37:55.920267+0000","last_became_peered":"2024-07-17T08:37:55.920267+0000","last_unstale":"2024-07-17T08:37:56.894605+0000","last_undegraded":"2024-07-17T08:37:56.894605+0000","last_fullsized":"2024-07-17T08:37:56.894605+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:42:38.406052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.894992+0000","last_change":"2024-07-17T08:37:54.878296+0000","last_active":"2024-07-17T08:37:56.894992+0000","last_peered":"2024-07-17T08:37:56.894992+0000","last_clean":"2024-07-17T08:37:56.894992+0000","last_became_active":"2024-07-17T08:37:54.878006+0000","last_became_peered":"2024-07-17T08:37:54.878006+0000","last_unstale":"2024-07-17T08:37:56.894992+0000","last_undegraded":"2024-07-17T08:37:56.894992+0000","last_fullsized":"2024-07-17T08:37:56.894992+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:45.472248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"38.2","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.916028+0000","last_change":"2024-07-17T08:37:55.913251+0000","last_active":"2024-07-17T08:37:56.916028+0000","last_peered":"2024-07-17T08:37:56.916028+0000","last_clean":"2024-07-17T08:37:56.916028+0000","last_became_active":"2024-07-17T08:37:55.912940+0000","last_became_peered":"2024-07-17T08:37:55.912940+0000","last_unstale":"2024-07-17T08:37:56.916028+0000","last_undegraded":"2024-07-17T08:37:56.916028+0000","last_fullsized":"2024-07-17T08:37:56.916028+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:42:52.905376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.071567+0000","last_change":"2024-07-17T08:37:54.877902+0000","last_active":"2024-07-17T08:37:57.071567+0000","last_peered":"2024-07-17T08:37:57.071567+0000","last_clean":"2024-07-17T08:37:57.071567+0000","last_became_active":"2024-07-17T08:37:54.877308+0000","last_became_peered":"2024-07-17T08:37:54.877308+0000","last_unstale":"2024-07-17T08:37:57.071567+0000","last_undegraded":"2024-07-17T08:37:57.071567+0000","last_fullsized":"2024-07-17T08:37:57.071567+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:07.777174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"38.3","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:56.897448+0000","last_change":"2024-07-17T08:37:55.895136+0000","last_active":"2024-07-17T08:37:56.897448+0000","last_peered":"2024-07-17T08:37:56.897448+0000","last_clean":"2024-07-17T08:37:56.897448+0000","last_became_active":"2024-07-17T08:37:55.894461+0000","last_became_peered":"2024-07-17T08:37:55.894461+0000","last_unstale":"2024-07-17T08:37:56.897448+0000","last_undegraded":"2024-07-17T08:37:56.897448+0000","last_fullsized":"2024-07-17T08:37:56.897448+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:31.151213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":14,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.314655+0000","last_change":"2024-07-17T08:37:54.878627+0000","last_active":"2024-07-17T08:37:57.314655+0000","last_peered":"2024-07-17T08:37:57.314655+0000","last_clean":"2024-07-17T08:37:57.314655+0000","last_became_active":"2024-07-17T08:37:54.878270+0000","last_became_peered":"2024-07-17T08:37:54.878270+0000","last_unstale":"2024-07-17T08:37:57.314655+0000","last_undegraded":"2024-07-17T08:37:57.314655+0000","last_fullsized":"2024-07-17T08:37:57.314655+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:53.817120+0000","last_clean_scrub_stamp":"2024-07-17T08:37:53.817120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:24:34.431885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"38.27","version":"0'0","reported_seq":12,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.072713+0000","last_change":"2024-07-17T08:37:55.933081+0000","last_active":"2024-07-17T08:37:57.072713+0000","last_peered":"2024-07-17T08:37:57.072713+0000","last_clean":"2024-07-17T08:37:57.072713+0000","last_became_active":"2024-07-17T08:37:55.932738+0000","last_became_peered":"2024-07-17T08:37:55.932738+0000","last_unstale":"2024-07-17T08:37:57.072713+0000","last_undegraded":"2024-07-17T08:37:57.072713+0000","last_fullsized":"2024-07-17T08:37:57.072713+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":275,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:37:54.833502+0000","last_clean_scrub_stamp":"2024-07-17T08:37:54.833502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:41.575151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"267'4582","reported_seq":5670,"reported_epoch":277,"state":"active+clean","last_fresh":"2024-07-17T08:37:57.317095+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:37:57.317095+0000","last_peered":"2024-07-17T08:37:57.317095+0000","last_clean":"2024-07-17T08:37:57.317095+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:37:57.317095+0000","last_undegraded":"2024-07-17T08:37:57.317095+0000","last_fullsized":"2024-07-17T08:37:57.317095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4582,"log_dups_size":0,"ondisk_log_size":4582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":421,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":38,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":37,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1729,"num_read_kb":4854,"num_write":421,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4582,"ondisk_log_size":4582,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543369,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75452,"kb_used_data":4016,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296388,"statfs":{"total":96636764160,"available":96559501312,"internally_reserved":0,"allocated":4112384,"data_stored":2518673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48512,"kb_used_data":3440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323328,"statfs":{"total":96636764160,"available":96587087872,"internally_reserved":0,"allocated":3522560,"data_stored":1928887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510665,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70712,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301128,"statfs":{"total":96636764160,"available":96564355072,"internally_reserved":0,"allocated":3518464,"data_stored":1928317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543370,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3526656,"data_stored":1923369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510664,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52672,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319168,"statfs":{"total":96636764160,"available":96582828032,"internally_reserved":0,"allocated":3518464,"data_stored":1928317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543368,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3448,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3530752,"data_stored":1929965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510665,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52652,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319188,"statfs":{"total":96636764160,"available":96582848512,"internally_reserved":0,"allocated":3502080,"data_stored":1921015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510664,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49080,"kb_used_data":4004,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322760,"statfs":{"total":96636764160,"available":96586506240,"internally_reserved":0,"allocated":4100096,"data_stored":2511473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":38,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:38:00.926 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:38:00.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:38:01.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 -- 172.21.15.44:0/3704272499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580ce5c0 msgr2=0x7f4b5806d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 --2- 172.21.15.44:0/3704272499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580ce5c0 0x7f4b5806d9b0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009900 tx=0x7f4b4402a560 comp rx=0 tx=0).stop 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 -- 172.21.15.44:0/3704272499 shutdown_connections 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 --2- 172.21.15.44:0/3704272499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580ce5c0 0x7f4b5806d9b0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 --2- 172.21.15.44:0/3704272499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 0x7f4b580b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 -- 172.21.15.44:0/3704272499 >> 172.21.15.44:0/3704272499 conn(0x7f4b580cfec0 msgr2=0x7f4b580d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.185+0000 7f4b5f000640 1 -- 172.21.15.44:0/3704272499 shutdown_connections 2024-07-17T08:38:01.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.186+0000 7f4b5f000640 1 -- 172.21.15.44:0/3704272499 wait complete. 2024-07-17T08:38:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.187+0000 7f4b5f000640 1 Processor -- start 2024-07-17T08:38:01.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.187+0000 7f4b5f000640 1 -- start start 2024-07-17T08:38:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 0x7f4b58072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 0x7f4b5815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5806e780 con 0x7f4b580b5400 2024-07-17T08:38:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5806e600 con 0x7f4b580ce5c0 2024-07-17T08:38:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.188+0000 7f4b5f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5806e900 con 0x7f4b580b8c80 2024-07-17T08:38:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49918/0 (socket says 172.21.15.44:49918) 2024-07-17T08:38:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 -- 172.21.15.44:0/1655744405 learned_addr learned my addr 172.21.15.44:0/1655744405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5d200640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 0x7f4b58072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5e600640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 0x7f4b5815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 -- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 msgr2=0x7f4b58072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 0x7f4b58072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 -- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 msgr2=0x7f4b5815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 0x7f4b5815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5dc00640 1 -- 172.21.15.44:0/1655744405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580cc920 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.189+0000 7f4b5e600640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 0x7f4b5815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b5dc00640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cce0 tx=0x7f4b4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48004b20 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b48004db0 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b580ccc00 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48005260 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.190+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b580cd1c0 con 0x7f4b580b5400 2024-07-17T08:38:01.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.191+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5815e520 con 0x7f4b580b5400 2024-07-17T08:38:01.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.193+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f4b48005400 con 0x7f4b580b5400 2024-07-17T08:38:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.194+0000 7f4b55600640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 0x7f4b380791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.194+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f4b480991d0 con 0x7f4b580b5400 2024-07-17T08:38:01.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.194+0000 7f4b5d200640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 0x7f4b380791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.195+0000 7f4b5d200640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 0x7f4b380791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b44033000 tx=0x7f4b44004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:01.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.198+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4b48014030 con 0x7f4b580b5400 2024-07-17T08:38:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.497+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4b580b5d70 con 0x7f4b580b5400 2024-07-17T08:38:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.502+0000 7f4b55600640 1 -- 172.21.15.44:0/1655744405 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v75) ==== 190+0+147 (secure 0 0 0) 0x7f4b480620a0 con 0x7f4b580b5400 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:38:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:38:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 msgr2=0x7f4b380791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 0x7f4b380791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b44033000 tx=0x7f4b44004730 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 msgr2=0x7f4b581f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cce0 tx=0x7f4b4800a620 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 shutdown_connections 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f4b38076d00 0x7f4b380791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b580ce5c0 0x7f4b5815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b580b8c80 0x7f4b58072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 --2- 172.21.15.44:0/1655744405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b580b5400 0x7f4b581f8fd0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 >> 172.21.15.44:0/1655744405 conn(0x7f4b580cfec0 msgr2=0x7f4b580c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.505+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 shutdown_connections 2024-07-17T08:38:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.506+0000 7f4b5f000640 1 -- 172.21.15.44:0/1655744405 wait complete. 2024-07-17T08:38:01.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 -- 172.21.15.44:0/3050254096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6954153a50 msgr2=0x7f6954159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 --2- 172.21.15.44:0/3050254096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6954153a50 0x7f6954159b00 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f69540aeda0 tx=0x7f694402b940 comp rx=0 tx=0).stop 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 -- 172.21.15.44:0/3050254096 shutdown_connections 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 --2- 172.21.15.44:0/3050254096 >> v2:172.21.15.175:3300/0 conn(0x7f69540acaf0 0x7f695406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 --2- 172.21.15.44:0/3050254096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6954153a50 0x7f6954159b00 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 -- 172.21.15.44:0/3050254096 >> 172.21.15.44:0/3050254096 conn(0x7f69540b0fb0 msgr2=0x7f69540b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 -- 172.21.15.44:0/3050254096 shutdown_connections 2024-07-17T08:38:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.733+0000 7f695a600640 1 -- 172.21.15.44:0/3050254096 wait complete. 2024-07-17T08:38:01.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.735+0000 7f695a600640 1 Processor -- start 2024-07-17T08:38:01.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.735+0000 7f695a600640 1 -- start start 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540acaf0 0x7f69541f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 0x7f69540b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f695406f710 con 0x7f695415a040 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6954154040 con 0x7f69540acaf0 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f695a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f695406f890 con 0x7f6954153a50 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f6953e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f6953e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49928/0 (socket says 172.21.15.44:49928) 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f6953e00640 1 -- 172.21.15.44:0/3291602587 learned_addr learned my addr 172.21.15.44:0/3291602587 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f6953400640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540acaf0 0x7f69541f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.737+0000 7f6952a00640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 0x7f69540b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 -- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 msgr2=0x7f69540b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 0x7f69540b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 -- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540acaf0 msgr2=0x7f69541f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540acaf0 0x7f69541f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 -- 172.21.15.44:0/3291602587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6954142fb0 con 0x7f695415a040 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6952a00640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 0x7f69540b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f6953e00640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f694c00cab0 tx=0x7f694c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00ae20 con 0x7f695415a040 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f694c00f450 con 0x7f695415a040 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00f8a0 con 0x7f695415a040 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69540cb430 con 0x7f695415a040 2024-07-17T08:38:01.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.738+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6954073ab0 con 0x7f695415a040 2024-07-17T08:38:01.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.739+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695406e090 con 0x7f695415a040 2024-07-17T08:38:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.741+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f694c004b20 con 0x7f695415a040 2024-07-17T08:38:01.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.742+0000 7f694b400640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 0x7f69300791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.742+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f694c099520 con 0x7f695415a040 2024-07-17T08:38:01.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.743+0000 7f6953400640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 0x7f69300791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:01.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.744+0000 7f6953400640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 0x7f69300791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6944035000 tx=0x7f6944004730 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:01.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:01.747+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f694c014030 con 0x7f695415a040 2024-07-17T08:38:02.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.004+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6954073c80 con 0x7f695415a040 2024-07-17T08:38:02.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.005+0000 7f694b400640 1 -- 172.21.15.44:0/3291602587 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7f694c062470 con 0x7f695415a040 2024-07-17T08:38:02.006 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:38:02.007 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:38:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:38:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 msgr2=0x7f69300791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 0x7f69300791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6944035000 tx=0x7f6944004730 comp rx=0 tx=0).stop 2024-07-17T08:38:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 msgr2=0x7f6954073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f694c00cab0 tx=0x7f694c00cf70 comp rx=0 tx=0).stop 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 shutdown_connections 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6930076d00 0x7f69300791c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f695415a040 0x7f6954073fd0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6954153a50 0x7f69540b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 --2- 172.21.15.44:0/3291602587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540acaf0 0x7f69541f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.008+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 >> 172.21.15.44:0/3291602587 conn(0x7f69540b0fb0 msgr2=0x7f69540af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.009+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 shutdown_connections 2024-07-17T08:38:02.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.009+0000 7f695a600640 1 -- 172.21.15.44:0/3291602587 wait complete. 2024-07-17T08:38:02.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 155, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:37:55.342248+0000', 'modified': '2024-07-17T08:37:58.430195+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11763}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11763': {'gid': 11763, 'name': 'a', 'rank': 0, 'incarnation': 151, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/4265556511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 4265556511}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 4265556511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [38], 'metadata_pool': 37, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11763, 'qdb_cluster': [11763]} 2024-07-17T08:38:02.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:38:02.432 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:38:02.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 -- 172.21.15.44:0/1552894638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780806d420 msgr2=0x7f780806d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 --2- 172.21.15.44:0/1552894638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780806d420 0x7f780806d8b0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f7808070830 tx=0x7f780002aa10 comp rx=0 tx=0).stop 2024-07-17T08:38:02.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 -- 172.21.15.44:0/1552894638 shutdown_connections 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 --2- 172.21.15.44:0/1552894638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78081503c0 0x7f78080ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 --2- 172.21.15.44:0/1552894638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f780806d420 0x7f780806d8b0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 --2- 172.21.15.44:0/1552894638 >> v2:172.21.15.175:3300/0 conn(0x7f780806f4d0 0x7f780815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 -- 172.21.15.44:0/1552894638 >> 172.21.15.44:0/1552894638 conn(0x7f78080cf0d0 msgr2=0x7f78080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 -- 172.21.15.44:0/1552894638 shutdown_connections 2024-07-17T08:38:02.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.655+0000 7f780ec00640 1 -- 172.21.15.44:0/1552894638 wait complete. 2024-07-17T08:38:02.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.657+0000 7f780ec00640 1 Processor -- start 2024-07-17T08:38:02.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.657+0000 7f780ec00640 1 -- start start 2024-07-17T08:38:02.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780806d420 0x7f780814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f780806f4d0 0x7f780809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78080b3240 con 0x7f78081503c0 2024-07-17T08:38:02.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78080b30c0 con 0x7f780806d420 2024-07-17T08:38:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.658+0000 7f780ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78080b33c0 con 0x7f780806f4d0 2024-07-17T08:38:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49940/0 (socket says 172.21.15.44:49940) 2024-07-17T08:38:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 -- 172.21.15.44:0/4228977166 learned_addr learned my addr 172.21.15.44:0/4228977166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780d800640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780806d420 0x7f780814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780ce00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f780806f4d0 0x7f780809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 -- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f780806f4d0 msgr2=0x7f780809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f780806f4d0 0x7f780809cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 -- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780806d420 msgr2=0x7f780814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780806d420 0x7f780814f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.659+0000 7f780e200640 1 -- 172.21.15.44:0/4228977166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780814fc00 con 0x7f78081503c0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f780e200640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7800009d60 tx=0x7f7800003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f7805600640 1 -- 172.21.15.44:0/4228977166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800047070 con 0x7f78081503c0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f7805600640 1 -- 172.21.15.44:0/4228977166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7800004250 con 0x7f78081503c0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f7805600640 1 -- 172.21.15.44:0/4228977166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800033050 con 0x7f78081503c0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780814e1a0 con 0x7f78081503c0 2024-07-17T08:38:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.660+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780814e760 con 0x7f78081503c0 2024-07-17T08:38:02.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.663+0000 7f7805600640 1 -- 172.21.15.44:0/4228977166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f780004a070 con 0x7f78081503c0 2024-07-17T08:38:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.664+0000 7f7805600640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 0x7f77e8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.664+0000 7f7805600640 1 -- 172.21.15.44:0/4228977166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f78000b9bf0 con 0x7f78081503c0 2024-07-17T08:38:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.664+0000 7f7812f9e740 1 Processor -- start 2024-07-17T08:38:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.664+0000 7f7812f9e740 1 -- start start 2024-07-17T08:38:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.664+0000 7f780d800640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 0x7f77e8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x563825917e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x563825963020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56382588ed60 con 0x56382591e0d0 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56382587e260 con 0x56382593fcb0 2024-07-17T08:38:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.665+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5638258822a0 con 0x563825962bb0 2024-07-17T08:38:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780d800640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 0x7f77e8079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0058c0 tx=0x7f77fc005c60 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49952/0 (socket says 172.21.15.44:49952) 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 -- 172.21.15.44:0/932903343 learned_addr learned my addr 172.21.15.44:0/932903343 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780d800640 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x563825917e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780e200640 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x563825963020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 -- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 msgr2=0x563825963020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x563825963020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 -- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 msgr2=0x563825917e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x563825917e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 -- 172.21.15.44:0/932903343 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638255e2850 con 0x56382591e0d0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.666+0000 7f780ce00640 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a7f0 tx=0x7f77f400acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80f477aa601b69c server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.667+0000 7f77fa000640 1 -- 172.21.15.44:0/932903343 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4015070 con 0x56382591e0d0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.667+0000 7f77fa000640 1 -- 172.21.15.44:0/932903343 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f400cc30 con 0x56382591e0d0 2024-07-17T08:38:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.667+0000 7f77fa000640 1 -- 172.21.15.44:0/932903343 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4011650 con 0x56382591e0d0 2024-07-17T08:38:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.667+0000 7f7812f9e740 1 -- 172.21.15.44:0/932903343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 msgr2=0x563825940090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.667+0000 7f7812f9e740 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a7f0 tx=0x7f77f400acb0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 -- 172.21.15.44:0/932903343 shutdown_connections 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x563825963020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382591e0d0 0x563825940090 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 --2- 172.21.15.44:0/932903343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x563825917e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 -- 172.21.15.44:0/932903343 >> 172.21.15.44:0/932903343 conn(0x563825919460 msgr2=0x56382591ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 -- 172.21.15.44:0/932903343 shutdown_connections 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 -- 172.21.15.44:0/932903343 wait complete. 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.668+0000 7f7812f9e740 1 -- start start 2024-07-17T08:38:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.669+0000 7f7812f9e740 1 Processor -- start 2024-07-17T08:38:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x56382593e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382595f4f0 0x563825961060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638255f1620 con 0x563825962bb0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563825825f90 con 0x56382593fcb0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563825918390 con 0x56382595f4f0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f7812f9e740 10 client.0 _ll_register_callbacks cb 0x56382584bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49954/0 (socket says 172.21.15.44:49954) 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 -- 172.21.15.44:0/3123253865 learned_addr learned my addr 172.21.15.44:0/3123253865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780d800640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x56382593e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 -- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382595f4f0 msgr2=0x563825961060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382595f4f0 0x563825961060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 -- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 msgr2=0x56382593e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x56382593e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.670+0000 7f780ce00640 1 -- 172.21.15.44:0/3123253865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638256596b0 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f780ce00640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c520 tx=0x7f77f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a2af8dff147a2 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f77eea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4020070 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f77f400a300 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f400cc30 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) ==== 24+0+0 (secure 0 0 0) 0x7f77f402e050 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f7812f9e740 10 client.11898 fetch_fsmap learned FSMap version 155 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.671+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=2+}) -- 0x56382593ecc0 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f400cc30 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 155) ==== 3502+0+0 (secure 0 0 0) 0x7f77f4031020 con 0x563825962bb0 2024-07-17T08:38:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 10 client.11898 fetch_fsmap finished waiting for FSMap version 155 2024-07-17T08:38:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 10 client.11898 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:38:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 20 client.11898 populate_metadata read hostname 'smithi044' 2024-07-17T08:38:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x563825945010 0x563825984930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 4 client.11898 mds_command: new command op to 11763 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:38:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 --> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563825890cb0 con 0x563825945010 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.672+0000 7f780d800640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x563825945010 0x563825984930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.673+0000 7f780d800640 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x563825945010 0x563825984930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68da566a89572e15 server_cookie=7680844284d60485 in_seq=0 out_seq=0 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.673+0000 7f77eea00640 10 client.11898 ms_handle_connect on v2:172.21.15.44:6833/4265556511 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f77eea00640 1 -- 172.21.15.44:0/3123253865 <== mds.0 v2:172.21.15.44:6833/4265556511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563825890cb0 con 0x563825945010 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f77eea00640 10 client.11898 handle_command_reply: tid=0 2024-07-17T08:38:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 client.11898 shutdown 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 msgr2=0x56382593f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c520 tx=0x7f77f4011ad0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 shutdown_connections 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x563825945010 0x563825984930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382595f4f0 0x563825961060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563825962bb0 0x56382593f4c0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 --2- 172.21.15.44:0/3123253865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x56382593e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 >> 172.21.15.44:0/3123253865 conn(0x56382591e0d0 msgr2=0x563825949db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f77eea00640 0 client.11898 ms_handle_reset on v2:172.21.15.44:6833/4265556511 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 shutdown_connections 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.674+0000 7f7812f9e740 1 -- 172.21.15.44:0/3123253865 wait complete. 2024-07-17T08:38:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.675+0000 7f7812f9e740 20 client.11898 trim_cache size 0 max 16384 2024-07-17T08:38:02.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.696+0000 7f7812f9e740 1 Processor -- start 2024-07-17T08:38:02.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.696+0000 7f7812f9e740 1 -- start start 2024-07-17T08:38:02.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382593fcb0 0x56382591d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563825962bb0 0x56382591b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78081fcf50 con 0x56382595f4f0 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78081670e0 con 0x563825962bb0 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7808166a90 con 0x56382593fcb0 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49956/0 (socket says 172.21.15.44:49956) 2024-07-17T08:38:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 -- 172.21.15.44:0/1489047657 learned_addr learned my addr 172.21.15.44:0/1489047657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 -- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382593fcb0 msgr2=0x56382591d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382593fcb0 0x56382591d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 -- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563825962bb0 msgr2=0x56382591b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780e200640 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563825962bb0 0x56382591b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563825962bb0 0x56382591b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.697+0000 7f780ce00640 1 -- 172.21.15.44:0/1489047657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638259636c0 con 0x56382595f4f0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.698+0000 7f780ce00640 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c700 tx=0x7f77f4011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2b77235689f823 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.698+0000 7f77f9600640 1 -- 172.21.15.44:0/1489047657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4037070 con 0x56382595f4f0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.698+0000 7f77f9600640 1 -- 172.21.15.44:0/1489047657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f4029250 con 0x56382595f4f0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.698+0000 7f77f9600640 1 -- 172.21.15.44:0/1489047657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4020070 con 0x56382595f4f0 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 -- 172.21.15.44:0/1489047657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 msgr2=0x56382591acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c700 tx=0x7f77f4011dc0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 -- 172.21.15.44:0/1489047657 shutdown_connections 2024-07-17T08:38:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563825962bb0 0x56382591b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591acc0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1489047657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56382593fcb0 0x56382591d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 -- 172.21.15.44:0/1489047657 >> 172.21.15.44:0/1489047657 conn(0x56382591e0d0 msgr2=0x56382593f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 -- 172.21.15.44:0/1489047657 shutdown_connections 2024-07-17T08:38:02.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.699+0000 7f7812f9e740 1 -- 172.21.15.44:0/1489047657 wait complete. 2024-07-17T08:38:02.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.700+0000 7f7812f9e740 1 -- start start 2024-07-17T08:38:02.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.700+0000 7f7812f9e740 1 Processor -- start 2024-07-17T08:38:02.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x5638259de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x56382591b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5638259615a0 con 0x56382595f4f0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56382593fa00 con 0x56382593fcb0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56382595e940 con 0x563825962bb0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f7812f9e740 10 client.0 _ll_register_callbacks cb 0x7f77f400a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f780ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49968/0 (socket says 172.21.15.44:49968) 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.701+0000 7f780ce00640 1 -- 172.21.15.44:0/1577356190 learned_addr learned my addr 172.21.15.44:0/1577356190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780d800640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x5638259de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780e200640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x56382591b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 -- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 msgr2=0x56382591b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x56382591b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 -- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 msgr2=0x5638259de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:38:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x5638259de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 -- 172.21.15.44:0/1577356190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56382593f010 con 0x56382595f4f0 2024-07-17T08:38:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780e200640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x56382591b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f780ce00640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a710 tx=0x7f77f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b73099eed48c1f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f77eea00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.702+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4037070 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f77f400a300 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f4029250 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=155) ==== 24+0+0 (secure 0 0 0) 0x7f77f4020050 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 10 client.11904 fetch_fsmap learned FSMap version 155 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=155,monmap=2+}) -- 0x56382591c780 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f4029250 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 155) ==== 3502+0+0 (secure 0 0 0) 0x7f77f4031030 con 0x56382595f4f0 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 10 client.11904 fetch_fsmap finished waiting for FSMap version 155 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 10 client.11904 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 20 client.11904 populate_metadata read hostname 'smithi044' 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x5638259e83f0 0x5638259e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 4 client.11904 mds_command: new command op to 11763 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:38:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.703+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 --> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] -- command(tid 0: {"prefix": "status"}) -- 0x56382591df20 con 0x5638259e83f0 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.704+0000 7f780d800640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x5638259e83f0 0x5638259e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.704+0000 7f780d800640 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x5638259e83f0 0x5638259e87d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49b4f5b0e83bab server_cookie=92abe96f5e27eab9 in_seq=0 out_seq=0 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.704+0000 7f77eea00640 10 client.11904 ms_handle_connect on v2:172.21.15.44:6833/4265556511 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f77eea00640 1 -- 172.21.15.44:0/1577356190 <== mds.0 v2:172.21.15.44:6833/4265556511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x56382591df20 con 0x5638259e83f0 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f77eea00640 10 client.11904 handle_command_reply: tid=0 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 client.11904 shutdown 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 msgr2=0x56382591aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a710 tx=0x7f77f4005f90 comp rx=0 tx=0).stop 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 shutdown_connections 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:6833/4265556511,v1:172.21.15.44:6835/4265556511] conn(0x5638259e83f0 0x5638259e87d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563825962bb0 0x56382591b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56382595f4f0 0x56382591aba0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f77eea00640 0 client.11904 ms_handle_reset on v2:172.21.15.44:6833/4265556511 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 --2- 172.21.15.44:0/1577356190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56382593fcb0 0x5638259de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 >> 172.21.15.44:0/1577356190 conn(0x56382591e0d0 msgr2=0x56382591d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 shutdown_connections 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 1 -- 172.21.15.44:0/1577356190 wait complete. 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.705+0000 7f7812f9e740 20 client.11904 trim_cache size 0 max 16384 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 11763, 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:38:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8002077549999997, 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 155, 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 277, 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 277, 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.757194291999999 2024-07-17T08:38:02.710 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 msgr2=0x7f77e8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 0x7f77e8079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0058c0 tx=0x7f77fc005c60 comp rx=0 tx=0).stop 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 msgr2=0x7f78080733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7800009d60 tx=0x7f7800003f10 comp rx=0 tx=0).stop 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 shutdown_connections 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f77e8076cd0 0x7f77e8079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78081503c0 0x7f78080733f0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f780806f4d0 0x7f780809cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 --2- 172.21.15.44:0/4228977166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f780806d420 0x7f780814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 >> 172.21.15.44:0/4228977166 conn(0x7f78080cf0d0 msgr2=0x7f78080aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 shutdown_connections 2024-07-17T08:38:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.706+0000 7f780ec00640 1 -- 172.21.15.44:0/4228977166 wait complete. 2024-07-17T08:38:02.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.931+0000 7f5879200640 1 -- 172.21.15.44:0/3288948652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58741594b0 msgr2=0x7f58740abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.931+0000 7f5879200640 1 --2- 172.21.15.44:0/3288948652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58741594b0 0x7f58740abf50 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f58740ceb50 tx=0x7f586402a560 comp rx=0 tx=0).stop 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 -- 172.21.15.44:0/3288948652 shutdown_connections 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 --2- 172.21.15.44:0/3288948652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5874159890 0x7f587409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 --2- 172.21.15.44:0/3288948652 >> v2:172.21.15.175:3300/0 conn(0x7f587409b5f0 0x7f58740b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 --2- 172.21.15.44:0/3288948652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58741594b0 0x7f58740abf50 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 -- 172.21.15.44:0/3288948652 >> 172.21.15.44:0/3288948652 conn(0x7f58740b0e80 msgr2=0x7f58740b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:02.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 -- 172.21.15.44:0/3288948652 shutdown_connections 2024-07-17T08:38:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.932+0000 7f5879200640 1 -- 172.21.15.44:0/3288948652 wait complete. 2024-07-17T08:38:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.934+0000 7f5879200640 1 Processor -- start 2024-07-17T08:38:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.934+0000 7f5879200640 1 -- start start 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58741594b0 0x7f58741e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5874159890 0x7f587415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58740cf6e0 con 0x7f587409b5f0 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58740cf560 con 0x7f5874159890 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5879200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58740cf860 con 0x7f58741594b0 2024-07-17T08:38:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49974/0 (socket says 172.21.15.44:49974) 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 -- 172.21.15.44:0/3362361695 learned_addr learned my addr 172.21.15.44:0/3362361695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 -- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58741594b0 msgr2=0x7f58741e1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58741594b0 0x7f58741e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 -- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5874159890 msgr2=0x7f587415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5874159890 0x7f587415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.936+0000 7f5873400640 1 -- 172.21.15.44:0/3362361695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58741f30f0 con 0x7f587409b5f0 2024-07-17T08:38:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f5873400640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f5864009850 tx=0x7f5864003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586403f070 con 0x7f587409b5f0 2024-07-17T08:38:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58640047a0 con 0x7f587409b5f0 2024-07-17T08:38:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58741e25b0 con 0x7f587409b5f0 2024-07-17T08:38:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864035030 con 0x7f587409b5f0 2024-07-17T08:38:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.937+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741f2630 con 0x7f587409b5f0 2024-07-17T08:38:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.939+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587409c5f0 con 0x7f587409b5f0 2024-07-17T08:38:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.940+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f5864042070 con 0x7f587409b5f0 2024-07-17T08:38:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.941+0000 7f586b400640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 0x7f5850079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.941+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f58640b9860 con 0x7f587409b5f0 2024-07-17T08:38:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.942+0000 7f5872a00640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 0x7f5850079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.943+0000 7f5872a00640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 0x7f5850079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f585c0058c0 tx=0x7f585c005850 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:02.945+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f587409c5f0 con 0x7f587409b5f0 2024-07-17T08:38:03.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.195+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58740abf50 con 0x7f587409b5f0 2024-07-17T08:38:03.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.196+0000 7f586b400640 1 -- 172.21.15.44:0/3362361695 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v155) ==== 53+0+73 (secure 0 0 0) 0x7f58640826a0 con 0x7f587409b5f0 2024-07-17T08:38:03.197 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 msgr2=0x7f5850079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 0x7f5850079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f585c0058c0 tx=0x7f585c005850 comp rx=0 tx=0).stop 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 msgr2=0x7f58741e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f5864009850 tx=0x7f5864003910 comp rx=0 tx=0).stop 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 shutdown_connections 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f5850076cd0 0x7f5850079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:03.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5874159890 0x7f587415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58741594b0 0x7f58741e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 --2- 172.21.15.44:0/3362361695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587409b5f0 0x7f58741e49a0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 >> 172.21.15.44:0/3362361695 conn(0x7f58740b0e80 msgr2=0x7f58740ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.199+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 shutdown_connections 2024-07-17T08:38:03.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03.200+0000 7f5879200640 1 -- 172.21.15.44:0/3362361695 wait complete. 2024-07-17T08:38:03.220 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:38:03.221 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:38:03.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.222 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:38:03.277 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:38:03.277 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:38:03.277 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:38:03.278 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36374sec preferred_lft 36374sec 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:38:03.279 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:38:03.280 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:38:03.281 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:38:03.281 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:38:03.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.282 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:38:03.362 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:38:03.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.367 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:38:03.421 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:38:03.421 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:38:03.421 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:38:03.421 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:38:03.421 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:38:03.422 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:38:03.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:38:03.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:38:03.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.635 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:38:03.685 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:38:03.685 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:38:03.686 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:38:03.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:38:03.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:38:03.775 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:38:03.826 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:38:03.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:38:03.827 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:38:03.872 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:38:03.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:03.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:38:04.037 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:38:04.037 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:38:04.037 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:38:04.058 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:38:04.058 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:38:04.058 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:38:04.059 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:38:04.059 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:38:04.085 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:38:04.086 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:38:04.087 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:38:04.087 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:38:04.087 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:38:04.087 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:38:04.087 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:38:04.091 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:38:04.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:38:04.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:38:04.165 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:38:04.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:38:04.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:38:04.301 INFO:teuthology.orchestra.run.smithi044.stdout:client8681 2024-07-17T08:38:04.301 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:38:04.301 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:38:04.356 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:38:04.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.562+0000 7f6def200640 1 -- 172.21.15.44:0/674407693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 msgr2=0x7f6dd4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.562+0000 7f6def200640 1 --2- 172.21.15.44:0/674407693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6dd4004800 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009b50 tx=0x7f6dd4005830 comp rx=0 tx=0).stop 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.562+0000 7f6dee800640 1 -- 172.21.15.44:0/674407693 reap_dead start 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.562+0000 7f6def200640 1 -- 172.21.15.44:0/674407693 shutdown_connections 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.563+0000 7f6def200640 1 -- 172.21.15.44:0/674407693 >> 172.21.15.44:0/674407693 conn(0x7f6df00cf0d0 msgr2=0x7f6df00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.563+0000 7f6def200640 1 -- 172.21.15.44:0/674407693 shutdown_connections 2024-07-17T08:38:04.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.563+0000 7f6def200640 1 -- 172.21.15.44:0/674407693 wait complete. 2024-07-17T08:38:04.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.564+0000 7f6def200640 1 Processor -- start 2024-07-17T08:38:04.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.565+0000 7f6def200640 1 -- start start 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df0157f60 0x7f6df01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df015d440 0x7f6df015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6df01503c0 con 0x7f6dd4004070 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6df00b3690 con 0x7f6df015d440 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6def200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df0150540 con 0x7f6df0157f60 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:04.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49996/0 (socket says 172.21.15.44:49996) 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 -- 172.21.15.44:0/1248975520 learned_addr learned my addr 172.21.15.44:0/1248975520 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6ded400640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df0157f60 0x7f6df01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dee800640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df015d440 0x7f6df015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 -- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df0157f60 msgr2=0x7f6df01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df0157f60 0x7f6df01583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 -- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df015d440 msgr2=0x7f6df015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df015d440 0x7f6df015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.566+0000 7f6dede00640 1 -- 172.21.15.44:0/1248975520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df0158a60 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6dede00640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f6de000e660 tx=0x7f6de000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0015020 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6de000cad0 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0011460 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df0158d40 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.567+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df009b350 con 0x7f6dd4004070 2024-07-17T08:38:04.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.568+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df00ae340 con 0x7f6dd4004070 2024-07-17T08:38:04.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.570+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7f6de0018020 con 0x7f6dd4004070 2024-07-17T08:38:04.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.571+0000 7f6de6000640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 0x7f6dd00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:04.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.571+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f6de0098fc0 con 0x7f6dd4004070 2024-07-17T08:38:04.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.572+0000 7f6ded400640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 0x7f6dd00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:04.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.573+0000 7f6ded400640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 0x7f6dd00791c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc035000 tx=0x7f6ddc0046c0 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:04.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.576+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6de001f020 con 0x7f6dd4004070 2024-07-17T08:38:04.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.811+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 --> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6df00cffe0 con 0x7f6dd0076d00 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.815+0000 7f6de6000640 1 -- 172.21.15.44:0/1248975520 <== mgr.8516 v2:172.21.15.175:6834/2531126035 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6df00cffe0 con 0x7f6dd0076d00 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:38:04.816 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:38:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 msgr2=0x7f6dd00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 0x7f6dd00791c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc035000 tx=0x7f6ddc0046c0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 msgr2=0x7f6df014e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f6de000e660 tx=0x7f6de000eb20 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 shutdown_connections 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6dd0076d00 0x7f6dd00791c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df015d440 0x7f6df015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df0157f60 0x7f6df01583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 --2- 172.21.15.44:0/1248975520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dd4004070 0x7f6df014e760 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.818+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 >> 172.21.15.44:0/1248975520 conn(0x7f6df00cf0d0 msgr2=0x7f6df00b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.819+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 shutdown_connections 2024-07-17T08:38:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:04.819+0000 7f6def200640 1 -- 172.21.15.44:0/1248975520 wait complete. 2024-07-17T08:38:04.839 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:38:05.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.054+0000 7fc5f5600640 1 -- 172.21.15.44:0/2739777017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 msgr2=0x7fc5d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.054+0000 7fc5f5600640 1 --2- 172.21.15.44:0/2739777017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5d0004c80 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009b50 tx=0x7fc5d0005830 comp rx=0 tx=0).stop 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.054+0000 7fc5f5600640 1 -- 172.21.15.44:0/2739777017 shutdown_connections 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.054+0000 7fc5f5600640 1 --2- 172.21.15.44:0/2739777017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5d0004c80 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.054+0000 7fc5f5600640 1 -- 172.21.15.44:0/2739777017 >> 172.21.15.44:0/2739777017 conn(0x7fc5f00d04c0 msgr2=0x7fc5f00d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.055+0000 7fc5f5600640 1 -- 172.21.15.44:0/2739777017 shutdown_connections 2024-07-17T08:38:05.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.055+0000 7fc5f5600640 1 -- 172.21.15.44:0/2739777017 wait complete. 2024-07-17T08:38:05.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.056+0000 7fc5f5600640 1 Processor -- start 2024-07-17T08:38:05.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.057+0000 7fc5f5600640 1 -- start start 2024-07-17T08:38:05.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.057+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:05.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.057+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 0x7fc5f015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:05.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.057+0000 7fc5f5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 0x7fc5f015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5f5600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f00d1bb0 con 0x7fc5d0004800 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5f5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f00d1a30 con 0x7fc5f00cb5b0 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5f5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f00ce130 con 0x7fc5f00cb130 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50000/0 (socket says 172.21.15.44:50000) 2024-07-17T08:38:05.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 -- 172.21.15.44:0/347079290 learned_addr learned my addr 172.21.15.44:0/347079290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:05.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5ef400640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 0x7fc5f015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:05.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5f4c00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 0x7fc5f015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:05.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 -- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 msgr2=0x7fc5f015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:05.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 0x7fc5f015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 -- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 msgr2=0x7fc5f015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 0x7fc5f015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5efe00640 1 -- 172.21.15.44:0/347079290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f00c9a20 con 0x7fc5d0004800 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5ef400640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 0x7fc5f015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.058+0000 7fc5f4c00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 0x7fc5f015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5efe00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000d4a0 tx=0x7fc5e000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0005020 con 0x7fc5d0004800 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5e00051e0 con 0x7fc5d0004800 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0005430 con 0x7fc5d0004800 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f00cbb80 con 0x7fc5d0004800 2024-07-17T08:38:05.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.059+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f015d8f0 con 0x7fc5d0004800 2024-07-17T08:38:05.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.060+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0161770 con 0x7fc5d0004800 2024-07-17T08:38:05.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.062+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 158) ==== 98931+0+0 (secure 0 0 0) 0x7fc5e0019020 con 0x7fc5d0004800 2024-07-17T08:38:05.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.062+0000 7fc5e7e00640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 0x7fc5cc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:05.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.063+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fc5e0097ef0 con 0x7fc5d0004800 2024-07-17T08:38:05.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.063+0000 7fc5ef400640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 0x7fc5cc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:05.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.064+0000 7fc5ef400640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 0x7fc5cc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8035000 tx=0x7fc5d80046c0 comp rx=0 tx=0).ready entity=mgr.8516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:05.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.069+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc5e0060e40 con 0x7fc5d0004800 2024-07-17T08:38:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.334+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc5f00b29f0 con 0x7fc5d0004800 2024-07-17T08:38:05.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.983+0000 7fc5e7e00640 1 -- 172.21.15.44:0/347079290 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v159) ==== 92+0+0 (secure 0 0 0) 0x7fc5e0069080 con 0x7fc5d0004800 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.986+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 msgr2=0x7fc5cc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.986+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 0x7fc5cc079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8035000 tx=0x7fc5d80046c0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 msgr2=0x7fc5f00c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000d4a0 tx=0x7fc5e000d960 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 shutdown_connections 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fc5cc076cd0 0x7fc5cc079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f00cb5b0 0x7fc5f015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5f00cb130 0x7fc5f015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 --2- 172.21.15.44:0/347079290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d0004800 0x7fc5f00c94e0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:05.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 >> 172.21.15.44:0/347079290 conn(0x7fc5f00d04c0 msgr2=0x7fc5f00d3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 shutdown_connections 2024-07-17T08:38:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:05.987+0000 7fc5f5600640 1 -- 172.21.15.44:0/347079290 wait complete. 2024-07-17T08:38:06.015 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:38:06.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.218+0000 7f32db400640 1 -- 172.21.15.44:0/1386298318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d40b5ae0 msgr2=0x7f32d4072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.218+0000 7f32db400640 1 --2- 172.21.15.44:0/1386298318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d40b5ae0 0x7f32d4072e90 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f32d40640a0 tx=0x7f32c002a560 comp rx=0 tx=0).stop 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.218+0000 7f32db400640 1 -- 172.21.15.44:0/1386298318 shutdown_connections 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.219+0000 7f32db400640 1 --2- 172.21.15.44:0/1386298318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d40b5ae0 0x7f32d4072e90 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.219+0000 7f32db400640 1 --2- 172.21.15.44:0/1386298318 >> v2:172.21.15.175:3300/0 conn(0x7f32d4159990 0x7f32d40b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.219+0000 7f32db400640 1 -- 172.21.15.44:0/1386298318 >> 172.21.15.44:0/1386298318 conn(0x7f32d406ce80 msgr2=0x7f32d406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.219+0000 7f32db400640 1 -- 172.21.15.44:0/1386298318 shutdown_connections 2024-07-17T08:38:06.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.219+0000 7f32db400640 1 -- 172.21.15.44:0/1386298318 wait complete. 2024-07-17T08:38:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.220+0000 7f32db400640 1 Processor -- start 2024-07-17T08:38:06.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.221+0000 7f32db400640 1 -- start start 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.221+0000 7f32db400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32d40733d0 0x7f32d40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.221+0000 7f32db400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 0x7f32d4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.221+0000 7f32db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32db400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4068bf0 con 0x7f32d4159990 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32db400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4068a70 con 0x7f32d40b5ae0 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32db400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f32d4068d70 con 0x7f32d40733d0 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50028/0 (socket says 172.21.15.44:50028) 2024-07-17T08:38:06.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 -- 172.21.15.44:0/4121373500 learned_addr learned my addr 172.21.15.44:0/4121373500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d3e00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 0x7f32d4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d8a00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32d40733d0 0x7f32d40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32d40733d0 msgr2=0x7f32d40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32d40733d0 0x7f32d40ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 msgr2=0x7f32d4157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 0x7f32d4157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d9400640 1 -- 172.21.15.44:0/4121373500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d414b910 con 0x7f32d4159990 2024-07-17T08:38:06.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.222+0000 7f32d3e00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 0x7f32d4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32d9400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c780 tx=0x7f32cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc017070 con 0x7f32d4159990 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32cc005070 con 0x7f32d4159990 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d414bed0 con 0x7f32d4159990 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc005500 con 0x7f32d4159990 2024-07-17T08:38:06.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.223+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d414c490 con 0x7f32d4159990 2024-07-17T08:38:06.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.225+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d4072e90 con 0x7f32d4159990 2024-07-17T08:38:06.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.226+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 159) ==== 98948+0+0 (secure 0 0 0) 0x7f32cc01a070 con 0x7f32d4159990 2024-07-17T08:38:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.227+0000 7f32d0c00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 0x7f32b40791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.227+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f32cc098080 con 0x7f32d4159990 2024-07-17T08:38:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.227+0000 7f32d8a00640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 msgr2=0x7f32b40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:06.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.227+0000 7f32d8a00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 0x7f32b40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:38:06.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.231+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f32cc060fd0 con 0x7f32d4159990 2024-07-17T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.429+0000 7f32d8a00640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 msgr2=0x7f32b40791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:06.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.429+0000 7f32d8a00640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 0x7f32b40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:38:06.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.445+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f32d40b3970 con 0x7f32d4159990 2024-07-17T08:38:06.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.452+0000 7f32d0c00640 1 -- 172.21.15.44:0/4121373500 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v85)=0 v85) ==== 138+0+0 (secure 0 0 0) 0x7f32cc069210 con 0x7f32d4159990 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.454+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 msgr2=0x7f32b40791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.454+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 0x7f32b40791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.454+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 msgr2=0x7f32d414c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.454+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c780 tx=0x7f32cc00cc40 comp rx=0 tx=0).stop 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 shutdown_connections 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f32b4076d20 0x7f32b40791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32d4159990 0x7f32d414c9b0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32d40b5ae0 0x7f32d4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 --2- 172.21.15.44:0/4121373500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32d40733d0 0x7f32d40ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 >> 172.21.15.44:0/4121373500 conn(0x7f32d406ce80 msgr2=0x7f32d40afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:06.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 shutdown_connections 2024-07-17T08:38:06.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.455+0000 7f32db400640 1 -- 172.21.15.44:0/4121373500 wait complete. 2024-07-17T08:38:06.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:38:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 -- 172.21.15.44:0/2858044061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac0b23e0 msgr2=0x7f6aac0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/2858044061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac0b23e0 0x7f6aac0ae720 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4005fd0 tx=0x7f6aa402aaa0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 -- 172.21.15.44:0/2858044061 shutdown_connections 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/2858044061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac0b23e0 0x7f6aac0ae720 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/2858044061 >> v2:172.21.15.175:3300/0 conn(0x7f6aac15b3a0 0x7f6aac0d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 -- 172.21.15.44:0/2858044061 >> 172.21.15.44:0/2858044061 conn(0x7f6aac0b5560 msgr2=0x7f6aac0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 -- 172.21.15.44:0/2858044061 shutdown_connections 2024-07-17T08:38:06.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.674+0000 7f6ab0e00640 1 -- 172.21.15.44:0/2858044061 wait complete. 2024-07-17T08:38:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.676+0000 7f6ab0e00640 1 Processor -- start 2024-07-17T08:38:06.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.676+0000 7f6ab0e00640 1 -- start start 2024-07-17T08:38:06.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 0x7f6aac0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac15b3a0 0x7f6aac06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0aeff0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0aee70 con 0x7f6aac0b3040 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6ab0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0af170 con 0x7f6aac15b3a0 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6aab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.677+0000 7f6aab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50034/0 (socket says 172.21.15.44:50034) 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 -- 172.21.15.44:0/3300864722 learned_addr learned my addr 172.21.15.44:0/3300864722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aabe00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac15b3a0 0x7f6aac06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 0x7f6aac0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac15b3a0 msgr2=0x7f6aac06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac15b3a0 0x7f6aac06ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 msgr2=0x7f6aac0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 0x7f6aac0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 -- 172.21.15.44:0/3300864722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac15b820 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 0x7f6aac0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aab400640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ea20 tx=0x7f6a9c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.678+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00cde0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.679+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac15bb00 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.679+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c004020 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.679+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004470 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.679+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac15c0c0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.680+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac070560 con 0x7f6aac0b23e0 2024-07-17T08:38:06.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.682+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 159) ==== 98948+0+0 (secure 0 0 0) 0x7f6a9c0035f0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.683+0000 7f6aa3400640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 0x7f6a880791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:06.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.683+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7f6a9c099450 con 0x7f6aac0b23e0 2024-07-17T08:38:06.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.683+0000 7f6aa3e00640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 msgr2=0x7f6a880791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:06.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.683+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 0x7f6a880791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:38:06.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.687+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6a9c0622b0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.885+0000 7f6aa3e00640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 msgr2=0x7f6a880791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:06.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.885+0000 7f6aa3e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 0x7f6a880791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:38:06.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.894+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f6aac15c4c0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.900+0000 7f6aa3400640 1 -- 172.21.15.44:0/3300864722 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v86)=0 v86) ==== 133+0+0 (secure 0 0 0) 0x7f6a9c061bb0 con 0x7f6aac0b23e0 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.902+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 msgr2=0x7f6a880791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.902+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 0x7f6a880791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.902+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 msgr2=0x7f6aac1f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.902+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ea20 tx=0x7f6a9c00eee0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 shutdown_connections 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7f6a88076d20 0x7f6a880791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6aac15b3a0 0x7f6aac06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6aac0b3040 0x7f6aac0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 --2- 172.21.15.44:0/3300864722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6aac0b23e0 0x7f6aac1f3990 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 >> 172.21.15.44:0/3300864722 conn(0x7f6aac0b5560 msgr2=0x7f6aac0cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:06.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 shutdown_connections 2024-07-17T08:38:06.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:06.903+0000 7f6ab0e00640 1 -- 172.21.15.44:0/3300864722 wait complete. 2024-07-17T08:38:06.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv01 2024-07-17T08:38:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 -- 172.21.15.44:0/3678888471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc15b720 msgr2=0x7fb0bc0ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 --2- 172.21.15.44:0/3678888471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc15b720 0x7fb0bc0ae390 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005fd0 tx=0x7fb0a802a670 comp rx=0 tx=0).stop 2024-07-17T08:38:07.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 -- 172.21.15.44:0/3678888471 shutdown_connections 2024-07-17T08:38:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 --2- 172.21.15.44:0/3678888471 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc0ae8d0 0x7fb0bc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 --2- 172.21.15.44:0/3678888471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc15b720 0x7fb0bc0ae390 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 -- 172.21.15.44:0/3678888471 >> 172.21.15.44:0/3678888471 conn(0x7fb0bc09aa30 msgr2=0x7fb0bc0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 -- 172.21.15.44:0/3678888471 shutdown_connections 2024-07-17T08:38:07.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.122+0000 7fb0c3800640 1 -- 172.21.15.44:0/3678888471 wait complete. 2024-07-17T08:38:07.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.124+0000 7fb0c3800640 1 Processor -- start 2024-07-17T08:38:07.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.124+0000 7fb0c3800640 1 -- start start 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 0x7fb0bc09b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc15b340 0x7fb0bc1f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc15b720 0x7fb0bc147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0cf2e0 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0cd7d0 con 0x7fb0bc15b720 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc0cf460 con 0x7fb0bc15b340 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 0x7fb0bc09b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 0x7fb0bc09b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50056/0 (socket says 172.21.15.44:50056) 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 -- 172.21.15.44:0/2111852379 learned_addr learned my addr 172.21.15.44:0/2111852379 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc15b340 0x7fb0bc1f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:07.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c1800640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc15b720 0x7fb0bc147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc15b340 msgr2=0x7fb0bc1f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0bc15b340 0x7fb0bc1f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc15b720 msgr2=0x7fb0bc147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0bc15b720 0x7fb0bc147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.126+0000 7fb0c0e00640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0b3730 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0c0e00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 0x7fb0bc09b510 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c530 tx=0x7fb0b000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003a40 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc09ba50 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.127+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc0b2970 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.128+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc0b3370 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.130+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 159) ==== 98948+0+0 (secure 0 0 0) 0x7fb0b001a070 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.131+0000 7fb0b8c00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.131+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(277..277 src has 1..277) ==== 6677+0+0 (secure 0 0 0) 0x7fb0b0097d60 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.131+0000 7fb0bbe00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.131+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:38:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.136+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb0b0060d40 con 0x7fb0bc0ae8d0 2024-07-17T08:38:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.333+0000 7fb0bbe00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.333+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:38:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.366+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv01", "target": ["mon-mgr", ""]}) -- 0x7fb0bc0cf980 con 0x7fb09c076d20 2024-07-17T08:38:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.734+0000 7fb0bbe00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:07.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:07.734+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:38:08.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:08.535+0000 7fb0bbe00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:08.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:08.536+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:38:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:10.138+0000 7fb0bbe00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2531126035 2024-07-17T08:38:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:10.138+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:38:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:10.198+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 160) ==== 98557+0+0 (secure 0 0 0) 0x7fb0b005c8d0 con 0x7fb0bc0ae8d0 2024-07-17T08:38:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:10.199+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 msgr2=0x7fb09c0791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:38:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:10.199+0000 7fb0b8c00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/2531126035,v1:172.21.15.175:6837/2531126035] conn(0x7fb09c076d20 0x7fb09c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:10.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:38:10.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:38:10.448 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.447+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.449 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.447+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.449 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.447+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.449 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.447+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.449 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.447+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.450 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.448+0000 7f104f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.450 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.448+0000 7f104f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.450 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.448+0000 7f104f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.450 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.448+0000 7f104f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:10.450 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:10.448+0000 7f104f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.207+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 161) ==== 98684+0+0 (secure 0 0 0) 0x7fb0b006ca60 con 0x7fb0bc0ae8d0 2024-07-17T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.208+0000 7fb0b8c00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb09c07f900 0x7fb09c081cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:11.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.208+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv01", "target": ["mon-mgr", ""]}) -- 0x7fb09c00c880 con 0x7fb09c07f900 2024-07-17T08:38:11.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.210+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb09c07f900 0x7fb09c081cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:11.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.221+0000 7fb0bbe00640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb09c07f900 0x7fb09c081cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8033000 tx=0x7fb0a8004730 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:11.239 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:11.238+0000 7f1047a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.239 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:11.238+0000 7f1047a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.239 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:11.238+0000 7f1047a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.239 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:11.238+0000 7f1047a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.239 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:38:11.238+0000 7f1047a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:38:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.321+0000 7fb0b8c00640 1 -- 172.21.15.44:0/2111852379 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb09c00c880 con 0x7fb09c07f900 2024-07-17T08:38:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb09c07f900 msgr2=0x7fb09c081cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb09c07f900 0x7fb09c081cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8033000 tx=0x7fb0a8004730 comp rx=0 tx=0).stop 2024-07-17T08:38:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 msgr2=0x7fb0bc09b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 --2- 172.21.15.44:0/2111852379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0bc0ae8d0 0x7fb0bc09b510 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c530 tx=0x7fb0b000c9f0 comp rx=0 tx=0).stop 2024-07-17T08:38:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c1800640 1 -- 172.21.15.44:0/2111852379 reap_dead start 2024-07-17T08:38:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 shutdown_connections 2024-07-17T08:38:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.324+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 >> 172.21.15.44:0/2111852379 conn(0x7fb0bc09aa30 msgr2=0x7fb0bc0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.325+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 shutdown_connections 2024-07-17T08:38:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.325+0000 7fb0c3800640 1 -- 172.21.15.44:0/2111852379 wait complete. 2024-07-17T08:38:11.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv01 . 1m 2024-07-17T08:38:11.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 -- 172.21.15.44:0/547190838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 msgr2=0x7fb148004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:11.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 --2- 172.21.15.44:0/547190838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb148004c80 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00a690 tx=0x7fb148005830 comp rx=0 tx=0).stop 2024-07-17T08:38:11.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 -- 172.21.15.44:0/547190838 shutdown_connections 2024-07-17T08:38:11.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 --2- 172.21.15.44:0/547190838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb148004c80 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:11.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 -- 172.21.15.44:0/547190838 >> 172.21.15.44:0/547190838 conn(0x7fb1640b0eb0 msgr2=0x7fb1640b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:11.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 -- 172.21.15.44:0/547190838 shutdown_connections 2024-07-17T08:38:11.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.567+0000 7fb169c00640 1 -- 172.21.15.44:0/547190838 wait complete. 2024-07-17T08:38:11.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.569+0000 7fb169c00640 1 Processor -- start 2024-07-17T08:38:11.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.569+0000 7fb169c00640 1 -- start start 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb164151b90 0x7fb164140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb164151640 0x7fb1641412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640ac3a0 con 0x7fb148004800 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb16406f8e0 con 0x7fb164151b90 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb169c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640ac520 con 0x7fb164151640 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb162a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb162a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55760/0 (socket says 172.21.15.44:55760) 2024-07-17T08:38:11.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.571+0000 7fb162a00640 1 -- 172.21.15.44:0/3521376052 learned_addr learned my addr 172.21.15.44:0/3521376052 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162000640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb164151b90 0x7fb164140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 -- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb164151640 msgr2=0x7fb1641412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb164151640 0x7fb1641412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 -- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb164151b90 msgr2=0x7fb164140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb164151b90 0x7fb164140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:11.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 -- 172.21.15.44:0/3521376052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1641f7c00 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb162a00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d4d0 tx=0x7fb15400d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154016020 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb154012630 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154012a60 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1641f7ee0 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.572+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1640cc760 con 0x7fb148004800 2024-07-17T08:38:11.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.574+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1640ae6b0 con 0x7fb148004800 2024-07-17T08:38:11.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.575+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 161) ==== 98684+0+0 (secure 0 0 0) 0x7fb154019020 con 0x7fb148004800 2024-07-17T08:38:11.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.576+0000 7fb15aa00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 0x7fb140078f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:38:11.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.576+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(278..278 src has 1..278) ==== 7021+0+0 (secure 0 0 0) 0x7fb1540983c0 con 0x7fb148004800 2024-07-17T08:38:11.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.577+0000 7fb162000640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 0x7fb140078f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:38:11.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.578+0000 7fb162000640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 0x7fb140078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb14c034000 tx=0x7fb14c004730 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:38:11.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.581+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb1640ae6b0 con 0x7fb148004800 2024-07-17T08:38:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:11.813+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv01", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb1641f79b0 con 0x7fb140076a80 2024-07-17T08:38:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.034+0000 7fb15aa00640 1 -- 172.21.15.44:0/3521376052 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+85 (secure 0 0 0) 0x7fb1641f79b0 con 0x7fb140076a80 2024-07-17T08:38:12.036 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/.. 2024-07-17T08:38:12.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 msgr2=0x7fb140078f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:12.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 0x7fb140078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb14c034000 tx=0x7fb14c004730 comp rx=0 tx=0).stop 2024-07-17T08:38:12.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 msgr2=0x7fb1641ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:38:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d4d0 tx=0x7fb15400d990 comp rx=0 tx=0).stop 2024-07-17T08:38:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 shutdown_connections 2024-07-17T08:38:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb140076a80 0x7fb140078f40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb164151640 0x7fb1641412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:12.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb164151b90 0x7fb164140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 --2- 172.21.15.44:0/3521376052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb148004800 0x7fb1641ee3e0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:38:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 >> 172.21.15.44:0/3521376052 conn(0x7fb1640b0eb0 msgr2=0x7fb1640b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:38:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 shutdown_connections 2024-07-17T08:38:12.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:38:12.038+0000 7fb169c00640 1 -- 172.21.15.44:0/3521376052 wait complete. 2024-07-17T08:38:12.062 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/.. 2024-07-17T08:38:40.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:38:40.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:39:00.058 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:00.057+0000 7f1034e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:00.058 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:00.057+0000 7f1034e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:00.058 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:00.057+0000 7f1034e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:00.058 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:00.057+0000 7f1034e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:00.058 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:00.057+0000 7f1034e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:10.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:39:10.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:39:27.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv01 2024-07-17T08:39:27.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 -- 172.21.15.44:0/2920024546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca80051c0 msgr2=0x7f7ca800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/2920024546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ca80051c0 0x7f7ca800b9c0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00d3380 tx=0x7f7ca8005830 comp rx=0 tx=0).stop 2024-07-17T08:39:27.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6400640 1 -- 172.21.15.44:0/2920024546 reap_dead start 2024-07-17T08:39:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 -- 172.21.15.44:0/2920024546 shutdown_connections 2024-07-17T08:39:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 -- 172.21.15.44:0/2920024546 >> 172.21.15.44:0/2920024546 conn(0x7f7cc00cfec0 msgr2=0x7f7cc00d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 -- 172.21.15.44:0/2920024546 shutdown_connections 2024-07-17T08:39:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.291+0000 7f7cc6e00640 1 -- 172.21.15.44:0/2920024546 wait complete. 2024-07-17T08:39:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.293+0000 7f7cc6e00640 1 Processor -- start 2024-07-17T08:39:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.293+0000 7f7cc6e00640 1 -- start start 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca80051c0 0x7f7cc0152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 0x7f7cc00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00d1220 con 0x7f7cc00ccc20 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00d1e30 con 0x7f7cc015f410 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc00cd090 con 0x7f7ca80051c0 2024-07-17T08:39:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41064/0 (socket says 172.21.15.44:41064) 2024-07-17T08:39:27.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.294+0000 7f7cc6400640 1 -- 172.21.15.44:0/3406074592 learned_addr learned my addr 172.21.15.44:0/3406074592 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc5000640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 0x7f7cc00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 -- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca80051c0 msgr2=0x7f7cc0152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca80051c0 0x7f7cc0152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 -- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 msgr2=0x7f7cc00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 0x7f7cc00cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 -- 172.21.15.44:0/3406074592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc01f7830 con 0x7f7cc00ccc20 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc5000640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 0x7f7cc00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cc6400640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009e10 tx=0x7f7cb802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.295+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80099a0 con 0x7f7cc00ccc20 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.296+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cb8036760 con 0x7f7cc00ccc20 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.296+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8036bb0 con 0x7f7cc00ccc20 2024-07-17T08:39:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.296+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc01f7b10 con 0x7f7cc00ccc20 2024-07-17T08:39:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.296+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01f80d0 con 0x7f7cc00ccc20 2024-07-17T08:39:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.297+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc00d18f0 con 0x7f7cc00ccc20 2024-07-17T08:39:27.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.298+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 164) ==== 99113+0+0 (secure 0 0 0) 0x7f7cb802b430 con 0x7f7cc00ccc20 2024-07-17T08:39:27.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.299+0000 7f7cbd600640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 0x7f7ca00793c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.299+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7f7cb80b8630 con 0x7f7cc00ccc20 2024-07-17T08:39:27.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.300+0000 7f7cc5a00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 0x7f7ca00793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.301+0000 7f7cc5a00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 0x7f7ca00793c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00ca440 tx=0x7f7cb4001bc0 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:27.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.304+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7cb8081170 con 0x7f7cc00ccc20 2024-07-17T08:39:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.548+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv01", "target": ["mon-mgr", ""]}) -- 0x7f7cc00b93c0 con 0x7f7ca0076f00 2024-07-17T08:39:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.565+0000 7f7cbd600640 1 -- 172.21.15.44:0/3406074592 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+60 (secure 0 0 0) 0x7f7cc00b93c0 con 0x7f7ca0076f00 2024-07-17T08:39:27.566 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577 2024-07-17T08:39:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 msgr2=0x7f7ca00793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 0x7f7ca00793c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00ca440 tx=0x7f7cb4001bc0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 msgr2=0x7f7cc0153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009e10 tx=0x7f7cb802cbf0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 shutdown_connections 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f7ca0076f00 0x7f7ca00793c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cc00ccc20 0x7f7cc0153c00 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7cc015f410 0x7f7cc00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 --2- 172.21.15.44:0/3406074592 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ca80051c0 0x7f7cc0152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.567+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 >> 172.21.15.44:0/3406074592 conn(0x7f7cc00cfec0 msgr2=0x7f7cc006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.568+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 shutdown_connections 2024-07-17T08:39:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.568+0000 7f7cc6e00640 1 -- 172.21.15.44:0/3406074592 wait complete. 2024-07-17T08:39:27.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:27.588 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/../.snap) 2024-07-17T08:39:27.637 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_39_00_UTC 2024-07-17T08:39:27.638 DEBUG:tasks.cephfs.test_snap_schedules:snaps:['scheduled-2024-07-17-08_39_00_UTC'] 2024-07-17T08:39:27.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv01 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.840+0000 7fab13200640 1 -- 172.21.15.44:0/4234715491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14150ed0 msgr2=0x7fab140ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.840+0000 7fab13200640 1 --2- 172.21.15.44:0/4234715491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14150ed0 0x7fab140ac290 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fab04009830 tx=0x7fab0402a480 comp rx=0 tx=0).stop 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.841+0000 7fab13200640 1 -- 172.21.15.44:0/4234715491 shutdown_connections 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.841+0000 7fab13200640 1 --2- 172.21.15.44:0/4234715491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab14150ed0 0x7fab140ac290 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.841+0000 7fab13200640 1 -- 172.21.15.44:0/4234715491 >> 172.21.15.44:0/4234715491 conn(0x7fab140cf0d0 msgr2=0x7fab140cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.841+0000 7fab13200640 1 -- 172.21.15.44:0/4234715491 shutdown_connections 2024-07-17T08:39:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.841+0000 7fab13200640 1 -- 172.21.15.44:0/4234715491 wait complete. 2024-07-17T08:39:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.842+0000 7fab13200640 1 Processor -- start 2024-07-17T08:39:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.843+0000 7fab13200640 1 -- start start 2024-07-17T08:39:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.843+0000 7fab13200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab1406f4d0 0x7fab14099190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab13200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab14150ed0 0x7fab141f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab13200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab13200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fab140b2930 con 0x7fab141512b0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab13200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab140b27b0 con 0x7fab1406f4d0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab13200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fab140b2ab0 con 0x7fab14150ed0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41082/0 (socket says 172.21.15.44:41082) 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab12800640 1 -- 172.21.15.44:0/1914842827 learned_addr learned my addr 172.21.15.44:0/1914842827 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab11e00640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab1406f4d0 0x7fab14099190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.844+0000 7fab0be00640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab14150ed0 0x7fab141f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 -- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab14150ed0 msgr2=0x7fab141f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab14150ed0 0x7fab141f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 -- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab1406f4d0 msgr2=0x7fab14099190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab1406f4d0 0x7fab14099190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 -- 172.21.15.44:0/1914842827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab140999a0 con 0x7fab141512b0 2024-07-17T08:39:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab12800640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c760 tx=0x7fab0c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017070 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab0c00ce10 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0052b0 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab140996d0 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.845+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1414c2e0 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.847+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab140cdeb0 con 0x7fab141512b0 2024-07-17T08:39:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.848+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 164) ==== 99113+0+0 (secure 0 0 0) 0x7fab0c01a070 con 0x7fab141512b0 2024-07-17T08:39:27.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.849+0000 7fab0a000640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 0x7faaf0079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:27.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.849+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7fab0c098380 con 0x7fab141512b0 2024-07-17T08:39:27.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.849+0000 7fab11e00640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 0x7faaf0079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:27.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.851+0000 7fab11e00640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 0x7faaf0079390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab04034000 tx=0x7fab04004730 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:27.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:27.853+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fab140999a0 con 0x7fab141512b0 2024-07-17T08:39:28.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.097+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv01", "target": ["mon-mgr", ""]}) -- 0x7fab1406d520 con 0x7faaf0076ed0 2024-07-17T08:39:28.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.109+0000 7fab0a000640 1 -- 172.21.15.44:0/1914842827 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+60 (secure 0 0 0) 0x7fab1406d520 con 0x7faaf0076ed0 2024-07-17T08:39:28.110 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577 2024-07-17T08:39:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 msgr2=0x7faaf0079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 0x7faaf0079390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab04034000 tx=0x7fab04004730 comp rx=0 tx=0).stop 2024-07-17T08:39:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 msgr2=0x7fab141f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c760 tx=0x7fab0c00cc20 comp rx=0 tx=0).stop 2024-07-17T08:39:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 shutdown_connections 2024-07-17T08:39:28.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7faaf0076ed0 0x7faaf0079390 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab141512b0 0x7fab141f7f00 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fab14150ed0 0x7fab141f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 --2- 172.21.15.44:0/1914842827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab1406f4d0 0x7fab14099190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.112+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 >> 172.21.15.44:0/1914842827 conn(0x7fab140cf0d0 msgr2=0x7fab1415c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:28.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.113+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 shutdown_connections 2024-07-17T08:39:28.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.113+0000 7fab13200640 1 -- 172.21.15.44:0/1914842827 wait complete. 2024-07-17T08:39:28.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:28.135 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/../.snap) 2024-07-17T08:39:28.187 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_39_00_UTC 2024-07-17T08:39:28.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/../.snap/scheduled-2024-07-17-08_39_00_UTC 2024-07-17T08:39:28.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:28.189 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/../.snap/scheduled-2024-07-17-08_39_00_UTC) 2024-07-17T08:39:28.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --subvol sv01 . 1m 2024-07-17T08:39:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 -- 172.21.15.44:0/2414437031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ad960 msgr2=0x7fb3540add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 --2- 172.21.15.44:0/2414437031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ad960 0x7fb3540add80 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fb340005fd0 tx=0x7fb34002a520 comp rx=0 tx=0).stop 2024-07-17T08:39:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 -- 172.21.15.44:0/2414437031 shutdown_connections 2024-07-17T08:39:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 --2- 172.21.15.44:0/2414437031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ae2c0 0x7fb3540729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 --2- 172.21.15.44:0/2414437031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ad960 0x7fb3540add80 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 -- 172.21.15.44:0/2414437031 >> 172.21.15.44:0/2414437031 conn(0x7fb3540cf050 msgr2=0x7fb3540cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:28.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 -- 172.21.15.44:0/2414437031 shutdown_connections 2024-07-17T08:39:28.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.511+0000 7fb35a000640 1 -- 172.21.15.44:0/2414437031 wait complete. 2024-07-17T08:39:28.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.513+0000 7fb35a000640 1 Processor -- start 2024-07-17T08:39:28.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.513+0000 7fb35a000640 1 -- start start 2024-07-17T08:39:28.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ad960 0x7fb3541f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:28.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:28.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 0x7fb354158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3540732c0 con 0x7fb3540ae2c0 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb354073140 con 0x7fb354155860 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb35a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb354073440 con 0x7fb3540ad960 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb353e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb353e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41090/0 (socket says 172.21.15.44:41090) 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb353e00640 1 -- 172.21.15.44:0/370056978 learned_addr learned my addr 172.21.15.44:0/370056978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb359600640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 0x7fb354158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb358c00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ad960 0x7fb3541f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.514+0000 7fb353e00640 1 -- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ad960 msgr2=0x7fb3541f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb353e00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ad960 0x7fb3541f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb353e00640 1 -- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 msgr2=0x7fb354158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb353e00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 0x7fb354158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb353e00640 1 -- 172.21.15.44:0/370056978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb354154020 con 0x7fb3540ae2c0 2024-07-17T08:39:28.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb359600640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 0x7fb354158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:28.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb353e00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fb340009850 tx=0x7fb3400039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:28.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34003f070 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb340004890 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340035030 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb354154300 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.515+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3541f41d0 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.517+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3540add80 con 0x7fb3540ae2c0 2024-07-17T08:39:28.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.518+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 164) ==== 99113+0+0 (secure 0 0 0) 0x7fb340042070 con 0x7fb3540ae2c0 2024-07-17T08:39:28.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.519+0000 7fb350c00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 0x7fb334079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:28.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.519+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7fb3400ba220 con 0x7fb3540ae2c0 2024-07-17T08:39:28.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.520+0000 7fb358c00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 0x7fb334079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:28.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.521+0000 7fb358c00640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 0x7fb334079390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb34800a750 tx=0x7fb348008040 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:28.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.523+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb340082d00 con 0x7fb3540ae2c0 2024-07-17T08:39:28.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.763+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "subvol": "sv01", "path": ".", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb3540cfc50 con 0x7fb334076ed0 2024-07-17T08:39:28.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.994+0000 7fb350c00640 1 -- 172.21.15.44:0/370056978 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+89 (secure 0 0 0) 0x7fb3540cfc50 con 0x7fb334076ed0 2024-07-17T08:39:28.995 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/.. 2024-07-17T08:39:28.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 msgr2=0x7fb334079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 0x7fb334079390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb34800a750 tx=0x7fb348008040 comp rx=0 tx=0).stop 2024-07-17T08:39:28.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 msgr2=0x7fb3541f7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fb340009850 tx=0x7fb3400039e0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 shutdown_connections 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fb334076ed0 0x7fb334079390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb354155860 0x7fb354158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3540ae2c0 0x7fb3541f7b30 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 --2- 172.21.15.44:0/370056978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3540ad960 0x7fb3541f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 >> 172.21.15.44:0/370056978 conn(0x7fb3540cf050 msgr2=0x7fb35406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.997+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 shutdown_connections 2024-07-17T08:39:28.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:28.998+0000 7fb35a000640 1 -- 172.21.15.44:0/370056978 wait complete. 2024-07-17T08:39:29.020 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /volumes/_nogroup/sv01/ce3ca4e1-4993-431a-b965-fbf206e10577/.. 2024-07-17T08:39:29.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv01 2024-07-17T08:39:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 -- 172.21.15.44:0/834814396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f0c0052c0 msgr2=0x7f9f0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 --2- 172.21.15.44:0/834814396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f0c0052c0 0x7f9f0c00b9c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f9f20005fd0 tx=0x7f9f0c005830 comp rx=0 tx=0).stop 2024-07-17T08:39:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 -- 172.21.15.44:0/834814396 shutdown_connections 2024-07-17T08:39:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 --2- 172.21.15.44:0/834814396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f0c0052c0 0x7f9f0c00b9c0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 -- 172.21.15.44:0/834814396 >> 172.21.15.44:0/834814396 conn(0x7f9f280cf0d0 msgr2=0x7f9f280cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:29.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 -- 172.21.15.44:0/834814396 shutdown_connections 2024-07-17T08:39:29.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.237+0000 7f9f27e00640 1 -- 172.21.15.44:0/834814396 wait complete. 2024-07-17T08:39:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.239+0000 7f9f27e00640 1 Processor -- start 2024-07-17T08:39:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.239+0000 7f9f27e00640 1 -- start start 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 0x7f9f28148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 0x7f9f2809b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2806f000 con 0x7f9f2809ceb0 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2806ee80 con 0x7f9f280c4dd0 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f27e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2806f180 con 0x7f9f0c0052c0 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.240+0000 7f9f26000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57664/0 (socket says 172.21.15.44:57664) 2024-07-17T08:39:29.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 -- 172.21.15.44:0/4095308741 learned_addr learned my addr 172.21.15.44:0/4095308741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26a00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 0x7f9f28148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 -- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 msgr2=0x7f9f28148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 0x7f9f28148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 -- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 msgr2=0x7f9f2809b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f27400640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 0x7f9f2809b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 0x7f9f2809b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 -- 172.21.15.44:0/4095308741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2809be10 con 0x7f9f2809ceb0 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26a00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 0x7f9f28148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f27400640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 0x7f9f2809b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:29.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.241+0000 7f9f26000640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000d5b0 tx=0x7f9f1000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.242+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10016020 con 0x7f9f2809ceb0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.242+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f10012630 con 0x7f9f2809ceb0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.242+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10012a60 con 0x7f9f2809ceb0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.242+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2809c0f0 con 0x7f9f2809ceb0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.242+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2809c6b0 con 0x7f9f2809ceb0 2024-07-17T08:39:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.243+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f280c51e0 con 0x7f9f2809ceb0 2024-07-17T08:39:29.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.244+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 164) ==== 99113+0+0 (secure 0 0 0) 0x7f9f10019020 con 0x7f9f2809ceb0 2024-07-17T08:39:29.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.245+0000 7f9f1ea00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 0x7f9f040793c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.245+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7f9f10098c30 con 0x7f9f2809ceb0 2024-07-17T08:39:29.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.246+0000 7f9f26a00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 0x7f9f040793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.247+0000 7f9f26a00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 0x7f9f040793c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f18001240 tx=0x7f9f18001bb0 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:29.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.249+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9f280c51e0 con 0x7f9f2809ceb0 2024-07-17T08:39:29.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.490+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 --> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv01", "target": ["mon-mgr", ""]}) -- 0x7f9f2815c3c0 con 0x7f9f04076f00 2024-07-17T08:39:29.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.531+0000 7f9f1ea00640 1 -- 172.21.15.44:0/4095308741 <== mgr.9556 v2:172.21.15.175:6834/3098340808 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f2815c3c0 con 0x7f9f04076f00 2024-07-17T08:39:29.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 msgr2=0x7f9f040793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 0x7f9f040793c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f18001240 tx=0x7f9f18001bb0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 msgr2=0x7f9f280c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000d5b0 tx=0x7f9f1000da70 comp rx=0 tx=0).stop 2024-07-17T08:39:29.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 shutdown_connections 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f9f04076f00 0x7f9f040793c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f280c4dd0 0x7f9f2809b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f2809ceb0 0x7f9f280c4890 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 --2- 172.21.15.44:0/4095308741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f0c0052c0 0x7f9f28148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.534+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 >> 172.21.15.44:0/4095308741 conn(0x7f9f280cf0d0 msgr2=0x7f9f2806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.535+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 shutdown_connections 2024-07-17T08:39:29.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.535+0000 7f9f27e00640 1 -- 172.21.15.44:0/4095308741 wait complete. 2024-07-17T08:39:29.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:39:29.560 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:29.558+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:29.560 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:29.558+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:29.560 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:29.558+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:29.560 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:29.558+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:29.560 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:29.558+0000 7f104ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 -- 172.21.15.44:0/2952295783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b14003a00 msgr2=0x7f5b14003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 --2- 172.21.15.44:0/2952295783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b14003a00 0x7f5b14003e20 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f5b20001eb0 tx=0x7f5b14005830 comp rx=0 tx=0).stop 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2e800640 1 -- 172.21.15.44:0/2952295783 reap_dead start 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 -- 172.21.15.44:0/2952295783 shutdown_connections 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 -- 172.21.15.44:0/2952295783 >> 172.21.15.44:0/2952295783 conn(0x7f5b3006c120 msgr2=0x7f5b3006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 -- 172.21.15.44:0/2952295783 shutdown_connections 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.761+0000 7f5b2f200640 1 -- 172.21.15.44:0/2952295783 wait complete. 2024-07-17T08:39:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.763+0000 7f5b2f200640 1 Processor -- start 2024-07-17T08:39:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.763+0000 7f5b2f200640 1 -- start start 2024-07-17T08:39:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b14003a00 0x7f5b30158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b301f7d60 0x7f5b30151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3006f210 con 0x7f5b30152340 2024-07-17T08:39:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3006f090 con 0x7f5b14003a00 2024-07-17T08:39:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5b300cf420 con 0x7f5b301f7d60 2024-07-17T08:39:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.764+0000 7f5b2e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57676/0 (socket says 172.21.15.44:57676) 2024-07-17T08:39:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 -- 172.21.15.44:0/3758694230 learned_addr learned my addr 172.21.15.44:0/3758694230 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2d400640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b301f7d60 0x7f5b30151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 -- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b301f7d60 msgr2=0x7f5b30151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b301f7d60 0x7f5b30151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 -- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b14003a00 msgr2=0x7f5b30158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b14003a00 0x7f5b30158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 -- 172.21.15.44:0/3758694230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301f81e0 con 0x7f5b30152340 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b2e800640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b28009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.765+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004b20 con 0x7f5b30152340 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.766+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b28004db0 con 0x7f5b30152340 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.766+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28011970 con 0x7f5b30152340 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.766+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b30072c90 con 0x7f5b30152340 2024-07-17T08:39:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.766+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b30157880 con 0x7f5b30152340 2024-07-17T08:39:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.767+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b30156d40 con 0x7f5b30152340 2024-07-17T08:39:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.768+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 164) ==== 99113+0+0 (secure 0 0 0) 0x7f5b2801b020 con 0x7f5b30152340 2024-07-17T08:39:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.769+0000 7f5b26000640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 0x7f5b0c0793c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.769+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7f5b28099300 con 0x7f5b30152340 2024-07-17T08:39:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.770+0000 7f5b2de00640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 0x7f5b0c0793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:29.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.771+0000 7f5b2de00640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 0x7f5b0c0793c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b20033000 tx=0x7f5b20001a30 comp rx=0 tx=0).ready entity=mgr.9556 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:29.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:29.772+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5b28061e30 con 0x7f5b30152340 2024-07-17T08:39:30.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.038+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f5b300b1cc0 con 0x7f5b30152340 2024-07-17T08:39:30.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.629+0000 7f5b26000640 1 -- 172.21.15.44:0/3758694230 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v165) ==== 93+0+0 (secure 0 0 0) 0x7f5b2806a070 con 0x7f5b30152340 2024-07-17T08:39:30.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.632+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 msgr2=0x7f5b0c0793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.632+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 0x7f5b0c0793c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b20033000 tx=0x7f5b20001a30 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.632+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 msgr2=0x7f5b300714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.632+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b28009a80 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 shutdown_connections 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f5b0c076f00 0x7f5b0c0793c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b30152340 0x7f5b300714b0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b301f7d60 0x7f5b30151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 --2- 172.21.15.44:0/3758694230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b14003a00 0x7f5b30158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.633+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 >> 172.21.15.44:0/3758694230 conn(0x7f5b3006c120 msgr2=0x7f5b30153200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:30.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.634+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 shutdown_connections 2024-07-17T08:39:30.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:30.634+0000 7f5b2f200640 1 -- 172.21.15.44:0/3758694230 wait complete. 2024-07-17T08:39:30.658 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:39:30.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:39:30.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:39:30.856 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:39:30.856 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:39:30.856 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:30.856 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:30.909 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:30.909 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:30.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:39:31.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 -- 172.21.15.44:0/8365407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154154600 msgr2=0x7f21540b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 --2- 172.21.15.44:0/8365407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154154600 0x7f21540b3ce0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f21540b43f0 tx=0x7f214402a540 comp rx=0 tx=0).stop 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 -- 172.21.15.44:0/8365407 shutdown_connections 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 --2- 172.21.15.44:0/8365407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215406db60 0x7f21540ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 --2- 172.21.15.44:0/8365407 >> v2:172.21.15.175:3300/0 conn(0x7f2154161120 0x7f215406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 --2- 172.21.15.44:0/8365407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154154600 0x7f21540b3ce0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 -- 172.21.15.44:0/8365407 >> 172.21.15.44:0/8365407 conn(0x7f21540b0eb0 msgr2=0x7f21540b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 -- 172.21.15.44:0/8365407 shutdown_connections 2024-07-17T08:39:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.199+0000 7f215b200640 1 -- 172.21.15.44:0/8365407 wait complete. 2024-07-17T08:39:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.201+0000 7f215b200640 1 Processor -- start 2024-07-17T08:39:31.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.201+0000 7f215b200640 1 -- start start 2024-07-17T08:39:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154154600 0x7f2154151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 0x7f2154152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21540aca60 con 0x7f2154161120 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21540ac8e0 con 0x7f2154154600 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.202+0000 7f215b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f215415e0a0 con 0x7f215414e3a0 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57700/0 (socket says 172.21.15.44:57700) 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 -- 172.21.15.44:0/1112172460 learned_addr learned my addr 172.21.15.44:0/1112172460 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f2159200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 0x7f2154152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f2153e00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154154600 0x7f2154151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 msgr2=0x7f2154152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 0x7f2154152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154154600 msgr2=0x7f2154151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154154600 0x7f2154151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f214be00640 1 -- 172.21.15.44:0/1112172460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2154155ba0 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.203+0000 7f2159200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 0x7f2154152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f214be00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f213c00d6d0 tx=0x7f213c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c017070 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f213c009b10 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c010460 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2154154de0 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.204+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541553a0 con 0x7f2154161120 2024-07-17T08:39:31.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.205+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21540b3ce0 con 0x7f2154161120 2024-07-17T08:39:31.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.206+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f213c01a070 con 0x7f2154161120 2024-07-17T08:39:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.207+0000 7f2150c00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 0x7f21300793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.208+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(280..280 src has 1..280) ==== 7108+0+0 (secure 0 0 0) 0x7f213c099270 con 0x7f2154161120 2024-07-17T08:39:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.208+0000 7f2153e00640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 msgr2=0x7f21300793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:31.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.208+0000 7f2153e00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 0x7f21300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:31.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.212+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f213c061bf0 con 0x7f2154161120 2024-07-17T08:39:31.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.410+0000 7f2153e00640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 msgr2=0x7f21300793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:31.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.410+0000 7f2153e00640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 0x7f21300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.432+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2154160e90 con 0x7f2154161120 2024-07-17T08:39:31.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.432+0000 7f2150c00640 1 -- 172.21.15.44:0/1112172460 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v280) ==== 80+0+478 (secure 0 0 0) 0x7f213c0614f0 con 0x7f2154161120 2024-07-17T08:39:31.433 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1260211667 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.433 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3073710198 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2738299478 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3444992690 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3343515448 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3797609750 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/2531126035 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/2531126035 2024-07-18T08:38:10.178682+0000 2024-07-17T08:39:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:39:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 msgr2=0x7f21300793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 0x7f21300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 msgr2=0x7f2154145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f213c00d6d0 tx=0x7f213c00db90 comp rx=0 tx=0).stop 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 shutdown_connections 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f2130076f10 0x7f21300793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f215414e3a0 0x7f2154152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154161120 0x7f2154145370 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 --2- 172.21.15.44:0/1112172460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2154154600 0x7f2154151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.435+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 >> 172.21.15.44:0/1112172460 conn(0x7f21540b0eb0 msgr2=0x7f21540b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.436+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 shutdown_connections 2024-07-17T08:39:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.436+0000 7f215b200640 1 -- 172.21.15.44:0/1112172460 wait complete. 2024-07-17T08:39:31.455 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:39:31.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:31.456 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:39:31.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:39:31.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:31.537 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:39:31.579 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:39:31.579 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:39:31.579 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:31.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:39:31.623 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:39:31.623 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:31.623 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:31.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:39:31.670 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:39:31.670 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:39:31.670 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:31.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:39:31.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:31.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 -- 172.21.15.44:0/4023429047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80cf6b0 msgr2=0x7f6eb815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 --2- 172.21.15.44:0/4023429047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80cf6b0 0x7f6eb815b600 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b5dd0 tx=0x7f6ea402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 -- 172.21.15.44:0/4023429047 shutdown_connections 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 --2- 172.21.15.44:0/4023429047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 0x7f6eb8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 --2- 172.21.15.44:0/4023429047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80cf6b0 0x7f6eb815b600 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 --2- 172.21.15.44:0/4023429047 >> v2:172.21.15.175:3300/0 conn(0x7f6eb80ccd20 0x7f6eb80cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.911+0000 7f6ebe400640 1 -- 172.21.15.44:0/4023429047 >> 172.21.15.44:0/4023429047 conn(0x7f6eb806ce80 msgr2=0x7f6eb806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.912+0000 7f6ebe400640 1 -- 172.21.15.44:0/4023429047 shutdown_connections 2024-07-17T08:39:31.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.912+0000 7f6ebe400640 1 -- 172.21.15.44:0/4023429047 wait complete. 2024-07-17T08:39:31.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.913+0000 7f6ebe400640 1 Processor -- start 2024-07-17T08:39:31.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.913+0000 7f6ebe400640 1 -- start start 2024-07-17T08:39:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6eb80ccd20 0x7f6eb814e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 0x7f6eb81f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb80733a0 con 0x7f6eb81505c0 2024-07-17T08:39:31.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb80ce750 con 0x7f6eb80ccd20 2024-07-17T08:39:31.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.914+0000 7f6ebe400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb8073520 con 0x7f6eb815bb40 2024-07-17T08:39:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6eb80ccd20 0x7f6eb814e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57708/0 (socket says 172.21.15.44:57708) 2024-07-17T08:39:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 -- 172.21.15.44:0/783550635 learned_addr learned my addr 172.21.15.44:0/783550635 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:31.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6eb7e00640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 0x7f6eb81f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 msgr2=0x7f6eb81f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 0x7f6eb81f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6eb80ccd20 msgr2=0x7f6eb814e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6eb80ccd20 0x7f6eb814e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.915+0000 7f6ebda00640 1 -- 172.21.15.44:0/783550635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb81f83a0 con 0x7f6eb81505c0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6ebda00640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e990 tx=0x7f6eb000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0004d20 con 0x7f6eb81505c0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eb000cb00 con 0x7f6eb81505c0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb814eed0 con 0x7f6eb81505c0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0005020 con 0x7f6eb81505c0 2024-07-17T08:39:31.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.916+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb81540b0 con 0x7f6eb81505c0 2024-07-17T08:39:31.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.918+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb815b600 con 0x7f6eb81505c0 2024-07-17T08:39:31.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.920+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f6eb00051c0 con 0x7f6eb81505c0 2024-07-17T08:39:31.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.921+0000 7f6eb4c00640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 0x7f6e980793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.921+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f6eb00997d0 con 0x7f6eb81505c0 2024-07-17T08:39:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.921+0000 7f6ebd000640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 msgr2=0x7f6e980793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:31.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.921+0000 7f6ebd000640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 0x7f6e980793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:31.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:31.925+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6eb0016030 con 0x7f6eb81505c0 2024-07-17T08:39:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.123+0000 7f6ebd000640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 msgr2=0x7f6e980793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:32.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.123+0000 7f6ebd000640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 0x7f6e980793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.172+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eb80702a0 con 0x7f6eb81505c0 2024-07-17T08:39:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.173+0000 7f6eb4c00640 1 -- 172.21.15.44:0/783550635 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7f6eb006a250 con 0x7f6eb81505c0 2024-07-17T08:39:32.174 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:32.175 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:39:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.175+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 msgr2=0x7f6e980793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.175+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 0x7f6e980793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.175+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 msgr2=0x7f6eb814bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.175+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e990 tx=0x7f6eb000ee50 comp rx=0 tx=0).stop 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 shutdown_connections 2024-07-17T08:39:32.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6e98076f10 0x7f6e980793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb81505c0 0x7f6eb814bd80 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eb815bb40 0x7f6eb81f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 --2- 172.21.15.44:0/783550635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6eb80ccd20 0x7f6eb814e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 >> 172.21.15.44:0/783550635 conn(0x7f6eb806ce80 msgr2=0x7f6eb815df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 shutdown_connections 2024-07-17T08:39:32.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.176+0000 7f6ebe400640 1 -- 172.21.15.44:0/783550635 wait complete. 2024-07-17T08:39:32.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.384+0000 7fe8cea00640 1 -- 172.21.15.44:0/712477780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c806bf40 msgr2=0x7fe8c80cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 --2- 172.21.15.44:0/712477780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c806bf40 0x7fe8c80cfd00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005fd0 tx=0x7fe8b802a5c0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 -- 172.21.15.44:0/712477780 shutdown_connections 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 --2- 172.21.15.44:0/712477780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c815b7f0 0x7fe8c806e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 --2- 172.21.15.44:0/712477780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c806bf40 0x7fe8c80cfd00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 -- 172.21.15.44:0/712477780 >> 172.21.15.44:0/712477780 conn(0x7fe8c80b2240 msgr2=0x7fe8c80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 -- 172.21.15.44:0/712477780 shutdown_connections 2024-07-17T08:39:32.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.385+0000 7fe8cea00640 1 -- 172.21.15.44:0/712477780 wait complete. 2024-07-17T08:39:32.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.387+0000 7fe8cea00640 1 Processor -- start 2024-07-17T08:39:32.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.387+0000 7fe8cea00640 1 -- start start 2024-07-17T08:39:32.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c806bf40 0x7fe8c81f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 0x7fe8c8150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c815be70 con 0x7fe8c815ef10 2024-07-17T08:39:32.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c806ecd0 con 0x7fe8c815b7f0 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c81f4690 con 0x7fe8c806bf40 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57714/0 (socket says 172.21.15.44:57714) 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8cca00640 1 -- 172.21.15.44:0/2008385751 learned_addr learned my addr 172.21.15.44:0/2008385751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8c7e00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c806bf40 0x7fe8c81f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.388+0000 7fe8c7400640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 0x7fe8c8150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c806bf40 msgr2=0x7fe8c81f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c806bf40 0x7fe8c81f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 msgr2=0x7fe8c8150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 0x7fe8c8150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 -- 172.21.15.44:0/2008385751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8155230 con 0x7fe8c815ef10 2024-07-17T08:39:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8c7400640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 0x7fe8c8150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cca00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000eb10 tx=0x7fe8c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004b20 con 0x7fe8c815ef10 2024-07-17T08:39:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8c0004ce0 con 0x7fe8c815ef10 2024-07-17T08:39:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0005280 con 0x7fe8c815ef10 2024-07-17T08:39:32.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c8155510 con 0x7fe8c815ef10 2024-07-17T08:39:32.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.389+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c8154ee0 con 0x7fe8c815ef10 2024-07-17T08:39:32.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.391+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c806f220 con 0x7fe8c815ef10 2024-07-17T08:39:32.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.392+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7fe8c0005420 con 0x7fe8c815ef10 2024-07-17T08:39:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.393+0000 7fe8bfe00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 0x7fe8a8079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.393+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7fe8c0099cd0 con 0x7fe8c815ef10 2024-07-17T08:39:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.393+0000 7fe8c7e00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 msgr2=0x7fe8a8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.393+0000 7fe8c7e00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 0x7fe8a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:32.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.397+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe8c0062640 con 0x7fe8c815ef10 2024-07-17T08:39:32.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.595+0000 7fe8c7e00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 msgr2=0x7fe8a8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:32.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.595+0000 7fe8c7e00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 0x7fe8a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:32.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.650+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8c815fcb0 con 0x7fe8c815ef10 2024-07-17T08:39:32.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.652+0000 7fe8bfe00640 1 -- 172.21.15.44:0/2008385751 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7fe8c0061f40 con 0x7fe8c815ef10 2024-07-17T08:39:32.653 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:32.653 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:39:32.653 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 msgr2=0x7fe8a8079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 0x7fe8a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 msgr2=0x7fe8c8153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000eb10 tx=0x7fe8c000c2d0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 shutdown_connections 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7fe8a8076e80 0x7fe8a8079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8c815ef10 0x7fe8c8153f40 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8c815b7f0 0x7fe8c8150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 --2- 172.21.15.44:0/2008385751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8c806bf40 0x7fe8c81f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 >> 172.21.15.44:0/2008385751 conn(0x7fe8c80b2240 msgr2=0x7fe8c8068040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 shutdown_connections 2024-07-17T08:39:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.655+0000 7fe8cea00640 1 -- 172.21.15.44:0/2008385751 wait complete. 2024-07-17T08:39:32.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:32.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.863+0000 7f316b000640 1 -- 172.21.15.44:0/2045627591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640704f0 msgr2=0x7f3164165f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.863+0000 7f316b000640 1 --2- 172.21.15.44:0/2045627591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640704f0 0x7f3164165f80 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f315c005fd0 tx=0x7f315c02a820 comp rx=0 tx=0).stop 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.863+0000 7f316b000640 1 -- 172.21.15.44:0/2045627591 shutdown_connections 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.863+0000 7f316b000640 1 --2- 172.21.15.44:0/2045627591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640704f0 0x7f3164165f80 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.863+0000 7f316b000640 1 -- 172.21.15.44:0/2045627591 >> 172.21.15.44:0/2045627591 conn(0x7f31640b3020 msgr2=0x7f31640b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.864+0000 7f316b000640 1 -- 172.21.15.44:0/2045627591 shutdown_connections 2024-07-17T08:39:32.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.864+0000 7f316b000640 1 -- 172.21.15.44:0/2045627591 wait complete. 2024-07-17T08:39:32.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.865+0000 7f316b000640 1 Processor -- start 2024-07-17T08:39:32.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.865+0000 7f316b000640 1 -- start start 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 0x7f31641f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316415b560 0x7f31641449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31641666d0 con 0x7f31640cfc70 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3164166850 con 0x7f31640704f0 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f316b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31641669d0 con 0x7f316415b560 2024-07-17T08:39:32.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.866+0000 7f3163400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57730/0 (socket says 172.21.15.44:57730) 2024-07-17T08:39:32.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 -- 172.21.15.44:0/3124530129 learned_addr learned my addr 172.21.15.44:0/3124530129 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316415b560 msgr2=0x7f31641449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3169000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316415b560 0x7f31641449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163e00640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 0x7f31641f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316415b560 0x7f31641449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 msgr2=0x7f31641f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 0x7f31641f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 -- 172.21.15.44:0/3124530129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641f85e0 con 0x7f31640cfc70 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163e00640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 0x7f31641f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.867+0000 7f3163400640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f314c00c460 tx=0x7f314c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.868+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c016020 con 0x7f31640cfc70 2024-07-17T08:39:32.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.868+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f314c012960 con 0x7f31640cfc70 2024-07-17T08:39:32.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.868+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c012db0 con 0x7f31640cfc70 2024-07-17T08:39:32.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.868+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3164154b10 con 0x7f31640cfc70 2024-07-17T08:39:32.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.868+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31640af530 con 0x7f31640cfc70 2024-07-17T08:39:32.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.869+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31640afd00 con 0x7f31640cfc70 2024-07-17T08:39:32.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.870+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f314c019020 con 0x7f31640cfc70 2024-07-17T08:39:32.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.871+0000 7f315be00640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 0x7f31400793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:32.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.871+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f314c098ba0 con 0x7f31640cfc70 2024-07-17T08:39:32.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.871+0000 7f3163e00640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 msgr2=0x7f31400793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:32.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.871+0000 7f3163e00640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 0x7f31400793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:32.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:32.875+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f314c020030 con 0x7f31640cfc70 2024-07-17T08:39:33.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.073+0000 7f3163e00640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 msgr2=0x7f31400793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:33.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.073+0000 7f3163e00640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 0x7f31400793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:33.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.083+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f316406fde0 con 0x7f31640cfc70 2024-07-17T08:39:33.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.084+0000 7f315be00640 1 -- 172.21.15.44:0/3124530129 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v281) ==== 74+0+18584 (secure 0 0 0) 0x7f31641f85e0 con 0x7f31640cfc70 2024-07-17T08:39:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:33.088 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":281,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:31.629248+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:53.302243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":38,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:54.326826+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":281,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"38.11","mappings":[{"from":7,"to":2}]},{"pgid":"38.1a","mappings":[{"from":7,"to":4}]},{"pgid":"38.2b","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1260211667":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3073710198":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/2738299478":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3444992690":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3343515448":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3797609750":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6837/2531126035":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6834/2531126035":"2024-07-18T08:38:10.178682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.087+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 msgr2=0x7f31400793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.087+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 0x7f31400793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.087+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 msgr2=0x7f3164155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.087+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f314c00c460 tx=0x7f314c00c920 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 shutdown_connections 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3140076f10 0x7f31400793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316415b560 0x7f31641449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31640cfc70 0x7f3164155e90 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 --2- 172.21.15.44:0/3124530129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31640704f0 0x7f31641f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 >> 172.21.15.44:0/3124530129 conn(0x7f31640b3020 msgr2=0x7f31640b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:33.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 shutdown_connections 2024-07-17T08:39:33.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.088+0000 7f316b000640 1 -- 172.21.15.44:0/3124530129 wait complete. 2024-07-17T08:39:33.107 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:39:33.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:33.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 -- 172.21.15.44:0/2336774164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 msgr2=0x7f6938003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 --2- 172.21.15.44:0/2336774164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f6938003e50 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f694400c290 tx=0x7f6938005830 comp rx=0 tx=0).stop 2024-07-17T08:39:33.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f6958c00640 1 -- 172.21.15.44:0/2336774164 reap_dead start 2024-07-17T08:39:33.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 -- 172.21.15.44:0/2336774164 shutdown_connections 2024-07-17T08:39:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 -- 172.21.15.44:0/2336774164 >> 172.21.15.44:0/2336774164 conn(0x7f69540b3020 msgr2=0x7f69540b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 -- 172.21.15.44:0/2336774164 shutdown_connections 2024-07-17T08:39:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.303+0000 7f695ac00640 1 -- 172.21.15.44:0/2336774164 wait complete. 2024-07-17T08:39:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.305+0000 7f695ac00640 1 Processor -- start 2024-07-17T08:39:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.305+0000 7f695ac00640 1 -- start start 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69540b21b0 0x7f69540b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540b6330 0x7f69540b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6954071230 con 0x7f6938003a30 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69540710b0 con 0x7f69540b6330 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f695ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69540c7810 con 0x7f69540b21b0 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57750/0 (socket says 172.21.15.44:57750) 2024-07-17T08:39:33.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 -- 172.21.15.44:0/1759719935 learned_addr learned my addr 172.21.15.44:0/1759719935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69540b21b0 msgr2=0x7f69540b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69540b21b0 0x7f69540b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540b6330 msgr2=0x7f69540b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540b6330 0x7f69540b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.306+0000 7f6953e00640 1 -- 172.21.15.44:0/1759719935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540b7090 con 0x7f6938003a30 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f6953e00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f694402b110 tx=0x7f6944001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944031070 con 0x7f6938003a30 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6944003800 con 0x7f6938003a30 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944003c10 con 0x7f6938003a30 2024-07-17T08:39:33.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541f73b0 con 0x7f6938003a30 2024-07-17T08:39:33.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.307+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541f7970 con 0x7f6938003a30 2024-07-17T08:39:33.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.308+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69540c7da0 con 0x7f6938003a30 2024-07-17T08:39:33.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.309+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f694403e030 con 0x7f6938003a30 2024-07-17T08:39:33.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.311+0000 7f6950c00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 0x7f6930079400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.311+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f69440b8ec0 con 0x7f6938003a30 2024-07-17T08:39:33.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.311+0000 7f694b400640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 msgr2=0x7f6930079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:33.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.311+0000 7f694b400640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 0x7f6930079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:33.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.315+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6944081780 con 0x7f6938003a30 2024-07-17T08:39:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.512+0000 7f694b400640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 msgr2=0x7f6930079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:33.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.512+0000 7f694b400640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 0x7f6930079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:33.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.562+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69540b1c60 con 0x7f6938003a30 2024-07-17T08:39:33.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.563+0000 7f6950c00640 1 -- 172.21.15.44:0/1759719935 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7f6944081080 con 0x7f6938003a30 2024-07-17T08:39:33.565 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:33.565 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:39:33.566 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 msgr2=0x7f6930079400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 0x7f6930079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 msgr2=0x7f69541543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f694402b110 tx=0x7f6944001dd0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 shutdown_connections 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f6930076f40 0x7f6930079400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69540b6330 0x7f69540b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69540b21b0 0x7f69540b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 --2- 172.21.15.44:0/1759719935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6938003a30 0x7f69541543e0 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 >> 172.21.15.44:0/1759719935 conn(0x7f69540b3020 msgr2=0x7f69540b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:33.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 shutdown_connections 2024-07-17T08:39:33.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.566+0000 7f695ac00640 1 -- 172.21.15.44:0/1759719935 wait complete. 2024-07-17T08:39:33.588 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 -- 172.21.15.44:0/3593860843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 msgr2=0x7f240815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 --2- 172.21.15.44:0/3593860843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815d2c0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009900 tx=0x7f23f002a600 comp rx=0 tx=0).stop 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 -- 172.21.15.44:0/3593860843 shutdown_connections 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 --2- 172.21.15.44:0/3593860843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815d2c0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 -- 172.21.15.44:0/3593860843 >> 172.21.15.44:0/3593860843 conn(0x7f24080cfec0 msgr2=0x7f24080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 -- 172.21.15.44:0/3593860843 shutdown_connections 2024-07-17T08:39:33.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.776+0000 7f240ca00640 1 -- 172.21.15.44:0/3593860843 wait complete. 2024-07-17T08:39:33.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.778+0000 7f240ca00640 1 Processor -- start 2024-07-17T08:39:33.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.778+0000 7f240ca00640 1 -- start start 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24080b8c80 0x7f24081f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24080b9060 0x7f24080c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f240815e2a0 con 0x7f24080ce5c0 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f240815e120 con 0x7f24080b8c80 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f240ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f240815e420 con 0x7f24080b9060 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f2407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f2407e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57766/0 (socket says 172.21.15.44:57766) 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f2407e00640 1 -- 172.21.15.44:0/4005161999 learned_addr learned my addr 172.21.15.44:0/4005161999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.779+0000 7f2407400640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24080b8c80 0x7f24081f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:33.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f23ffe00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24080b9060 0x7f24080c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24080b9060 msgr2=0x7f24080c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24080b9060 0x7f24080c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24080b8c80 msgr2=0x7f24081f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24080b8c80 0x7f24081f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 -- 172.21.15.44:0/4005161999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24080c6ff0 con 0x7f24080ce5c0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f2407e00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f240000c780 tx=0x7f240000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400004b60 con 0x7f24080ce5c0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2400004df0 con 0x7f24080ce5c0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.780+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240000aa70 con 0x7f24080ce5c0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.781+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24080c9ec0 con 0x7f24080ce5c0 2024-07-17T08:39:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.781+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24080c7540 con 0x7f24080ce5c0 2024-07-17T08:39:33.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.782+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24080b4ec0 con 0x7f24080ce5c0 2024-07-17T08:39:33.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.784+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f240000ac10 con 0x7f24080ce5c0 2024-07-17T08:39:33.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.785+0000 7f23ff400640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 0x7f23e4079400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:33.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.785+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f24000991f0 con 0x7f24080ce5c0 2024-07-17T08:39:33.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.785+0000 7f2407400640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 msgr2=0x7f23e4079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:33.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.785+0000 7f2407400640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 0x7f23e4079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:33.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.789+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2400020070 con 0x7f24080ce5c0 2024-07-17T08:39:33.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.986+0000 7f2407400640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 msgr2=0x7f23e4079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:33.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:33.986+0000 7f2407400640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 0x7f23e4079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.033+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24080b5d70 con 0x7f24080ce5c0 2024-07-17T08:39:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.034+0000 7f23ff400640 1 -- 172.21.15.44:0/4005161999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5242 (secure 0 0 0) 0x7f2400061bc0 con 0x7f24080ce5c0 2024-07-17T08:39:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:34.036 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":155,"btime":"2024-07-17T08:37:58:930166+0000","default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:37:55.342248+0000","modified":"2024-07-17T08:37:58.430195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11763":{"gid":11763,"name":"a","rank":0,"incarnation":151,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4265556511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4265556511},{"type":"v1","addr":"172.21.15.44:6835","nonce":4265556511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[38],"metadata_pool":37,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11763,"qdb_cluster":[11763]},"id":18}]} 2024-07-17T08:39:34.036 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 155 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 msgr2=0x7f23e4079400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 0x7f23e4079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 msgr2=0x7f240815c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f240000c780 tx=0x7f240000cc40 comp rx=0 tx=0).stop 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 shutdown_connections 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f23e4076f40 0x7f23e4079400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24080ce5c0 0x7f240815c740 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24080b9060 0x7f24080c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 --2- 172.21.15.44:0/4005161999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24080b8c80 0x7f24081f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 >> 172.21.15.44:0/4005161999 conn(0x7f24080cfec0 msgr2=0x7f24080d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:34.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 shutdown_connections 2024-07-17T08:39:34.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.037+0000 7f240ca00640 1 -- 172.21.15.44:0/4005161999 wait complete. 2024-07-17T08:39:34.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:34.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 -- 172.21.15.44:0/3863107072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 msgr2=0x7f1e240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 --2- 172.21.15.44:0/3863107072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e240051c0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c640 tx=0x7f1e24005830 comp rx=0 tx=0).stop 2024-07-17T08:39:34.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e3fe00640 1 -- 172.21.15.44:0/3863107072 reap_dead start 2024-07-17T08:39:34.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 -- 172.21.15.44:0/3863107072 shutdown_connections 2024-07-17T08:39:34.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 -- 172.21.15.44:0/3863107072 >> 172.21.15.44:0/3863107072 conn(0x7f1e400d00a0 msgr2=0x7f1e400d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:34.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 -- 172.21.15.44:0/3863107072 shutdown_connections 2024-07-17T08:39:34.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.256+0000 7f1e46200640 1 -- 172.21.15.44:0/3863107072 wait complete. 2024-07-17T08:39:34.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.258+0000 7f1e46200640 1 Processor -- start 2024-07-17T08:39:34.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.258+0000 7f1e46200640 1 -- start start 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 0x7f1e40142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e401429b0 0x7f1e400b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4006e600 con 0x7f1e24004d40 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4006e480 con 0x7f1e4013fbc0 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e46200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4006e780 con 0x7f1e401429b0 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57784/0 (socket says 172.21.15.44:57784) 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.259+0000 7f1e3f400640 1 -- 172.21.15.44:0/3844103401 learned_addr learned my addr 172.21.15.44:0/3844103401 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3ea00640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 0x7f1e40142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e401429b0 msgr2=0x7f1e400b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e401429b0 0x7f1e400b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 msgr2=0x7f1e40142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 0x7f1e40142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 -- 172.21.15.44:0/3844103401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40142f10 con 0x7f1e24004d40 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3ea00640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 0x7f1e40142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e3f400640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f1e300119d0 tx=0x7f1e30011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3001f070 con 0x7f1e24004d40 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e401431f0 con 0x7f1e24004d40 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3000fe40 con 0x7f1e24004d40 2024-07-17T08:39:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30012670 con 0x7f1e24004d40 2024-07-17T08:39:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.260+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e401f7a10 con 0x7f1e24004d40 2024-07-17T08:39:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.261+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e401553e0 con 0x7f1e24004d40 2024-07-17T08:39:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.263+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f1e30022070 con 0x7f1e24004d40 2024-07-17T08:39:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.264+0000 7f1e37400640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 0x7f1e200793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.264+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f1e30099940 con 0x7f1e24004d40 2024-07-17T08:39:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.264+0000 7f1e3ea00640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 msgr2=0x7f1e200793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.264+0000 7f1e3ea00640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 0x7f1e200793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:34.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.268+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e30061cd0 con 0x7f1e24004d40 2024-07-17T08:39:34.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.466+0000 7f1e3ea00640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 msgr2=0x7f1e200793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:34.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.466+0000 7f1e3ea00640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 0x7f1e200793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:34.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.479+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e401433c0 con 0x7f1e24004d40 2024-07-17T08:39:34.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.481+0000 7f1e37400640 1 -- 172.21.15.44:0/3844103401 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v281) ==== 74+0+18584 (secure 0 0 0) 0x7f1e401433c0 con 0x7f1e24004d40 2024-07-17T08:39:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:34.484 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":281,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:31.629248+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:37:53.302243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":38,"pool_name":"cephfs_data","create_time":"2024-07-17T08:37:54.326826+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"281","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":281,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"38.11","mappings":[{"from":7,"to":2}]},{"pgid":"38.1a","mappings":[{"from":7,"to":4}]},{"pgid":"38.2b","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1260211667":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3073710198":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/2738299478":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3444992690":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3343515448":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3797609750":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6837/2531126035":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6834/2531126035":"2024-07-18T08:38:10.178682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 msgr2=0x7f1e200793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 0x7f1e200793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 msgr2=0x7f1e401f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f1e300119d0 tx=0x7f1e30011e90 comp rx=0 tx=0).stop 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 shutdown_connections 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f1e20076f10 0x7f1e200793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e401429b0 0x7f1e400b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e4013fbc0 0x7f1e40142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 --2- 172.21.15.44:0/3844103401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e24004d40 0x7f1e401f7f30 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 >> 172.21.15.44:0/3844103401 conn(0x7f1e400d00a0 msgr2=0x7f1e400cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:34.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.484+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 shutdown_connections 2024-07-17T08:39:34.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.485+0000 7f1e46200640 1 -- 172.21.15.44:0/3844103401 wait complete. 2024-07-17T08:39:34.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 -- 172.21.15.44:0/1496127131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8003aa0 msgr2=0x7f3bc8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 --2- 172.21.15.44:0/1496127131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8003aa0 0x7f3bc8003ec0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009b20 tx=0x7f3bc8005830 comp rx=0 tx=0).stop 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 -- 172.21.15.44:0/1496127131 shutdown_connections 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 --2- 172.21.15.44:0/1496127131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bc8003aa0 0x7f3bc8003ec0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 -- 172.21.15.44:0/1496127131 >> 172.21.15.44:0/1496127131 conn(0x7f3be4073d50 msgr2=0x7f3be409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 -- 172.21.15.44:0/1496127131 shutdown_connections 2024-07-17T08:39:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.707+0000 7f3bea400640 1 -- 172.21.15.44:0/1496127131 wait complete. 2024-07-17T08:39:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.708+0000 7f3bea400640 1 Processor -- start 2024-07-17T08:39:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.709+0000 7f3bea400640 1 -- start start 2024-07-17T08:39:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.709+0000 7f3bea400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 0x7f3be4144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3bea400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3bea400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 0x7f3be41f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3bea400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40b1f00 con 0x7f3be41484f0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3bea400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40b5390 con 0x7f3bc8003aa0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3bea400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3be40b2080 con 0x7f3be415a1a0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57806/0 (socket says 172.21.15.44:57806) 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 -- 172.21.15.44:0/1137964161 learned_addr learned my addr 172.21.15.44:0/1137964161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be3400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 0x7f3be4144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be3e00640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 0x7f3be41f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 msgr2=0x7f3be41f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 0x7f3be41f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 msgr2=0x7f3be4144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 0x7f3be4144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be2a00640 1 -- 172.21.15.44:0/1137964161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41f7bf0 con 0x7f3be41484f0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be3400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 0x7f3be4144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.710+0000 7f3be3e00640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 0x7f3be41f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3be2a00640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000d7d0 tx=0x7f3bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0018070 con 0x7f3be41484f0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd000de80 con 0x7f3be41484f0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000b660 con 0x7f3be41484f0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be4147a10 con 0x7f3be41484f0 2024-07-17T08:39:34.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.711+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be4147fd0 con 0x7f3be41484f0 2024-07-17T08:39:34.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.712+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be40af960 con 0x7f3be41484f0 2024-07-17T08:39:34.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.714+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 165) ==== 99147+0+0 (secure 0 0 0) 0x7f3bd001b070 con 0x7f3be41484f0 2024-07-17T08:39:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.716+0000 7f3bdb400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 0x7f3bc40793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.716+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(281..281 src has 1..281) ==== 7164+0+0 (secure 0 0 0) 0x7f3bd00985f0 con 0x7f3be41484f0 2024-07-17T08:39:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.716+0000 7f3be3400640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 msgr2=0x7f3bc40793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.716+0000 7f3be3400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 0x7f3bc40793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:39:34.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.720+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bd0060970 con 0x7f3be41484f0 2024-07-17T08:39:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.919+0000 7f3be3400640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 msgr2=0x7f3bc40793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3098340808 2024-07-17T08:39:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.919+0000 7f3be3400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 0x7f3bc40793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:39:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.919+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 166) ==== 98540+0+0 (secure 0 0 0) 0x7f3bd005cc00 con 0x7f3be41484f0 2024-07-17T08:39:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.919+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 msgr2=0x7f3bc40793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:39:34.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:34.919+0000 7f3bdb400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 0x7f3bc40793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.028+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3be415a580 con 0x7f3be41484f0 2024-07-17T08:39:35.031 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:39:35.029+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:39:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.031+0000 7f3bdb400640 1 -- 172.21.15.44:0/1137964161 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v156) ==== 161+0+0 (secure 0 0 0) 0x7f3bd006c730 con 0x7f3be41484f0 2024-07-17T08:39:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:39:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.033+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 msgr2=0x7f3be4159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.033+0000 7f3bea400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000d7d0 tx=0x7f3bd000dc90 comp rx=0 tx=0).stop 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 shutdown_connections 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:6834/3098340808,v1:172.21.15.175:6837/3098340808] conn(0x7f3bc4076f10 0x7f3bc40793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3be415a1a0 0x7f3be41f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3be41484f0 0x7f3be4159c60 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 --2- 172.21.15.44:0/1137964161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bc8003aa0 0x7f3be4144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 >> 172.21.15.44:0/1137964161 conn(0x7f3be4073d50 msgr2=0x7f3be415cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 shutdown_connections 2024-07-17T08:39:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.034+0000 7f3bea400640 1 -- 172.21.15.44:0/1137964161 wait complete. 2024-07-17T08:39:35.041 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f0701e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f0701e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f0701e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f0701e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f0701e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f06ff600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f06ff600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f06ff600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f06ff600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.042 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:39:35.040+0000 7f06ff600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:39:35.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:39:35.125 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.227+0000 7f44c3200640 1 -- 172.21.15.44:0/3058761142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 msgr2=0x7f44bc15ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.227+0000 7f44c3200640 1 --2- 172.21.15.44:0/3058761142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc15ac10 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005fd0 tx=0x7f44b402aaa0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 -- 172.21.15.44:0/3058761142 shutdown_connections 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 --2- 172.21.15.44:0/3058761142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc15ac10 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 --2- 172.21.15.44:0/3058761142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc15a280 0x7f44bc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 -- 172.21.15.44:0/3058761142 >> 172.21.15.44:0/3058761142 conn(0x7f44bc0b0eb0 msgr2=0x7f44bc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:35.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 -- 172.21.15.44:0/3058761142 shutdown_connections 2024-07-17T08:39:35.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.228+0000 7f44c3200640 1 -- 172.21.15.44:0/3058761142 wait complete. 2024-07-17T08:39:35.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.229+0000 7f44c3200640 1 Processor -- start 2024-07-17T08:39:35.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.229+0000 7f44c3200640 1 -- start start 2024-07-17T08:39:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0b1af0 0x7f44bc15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc15a280 0x7f44bc1486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:35.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc15b150 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc06f1d0 con 0x7f44bc15a280 2024-07-17T08:39:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.230+0000 7f44c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f44bc15b2d0 con 0x7f44bc0b1af0 2024-07-17T08:39:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57824/0 (socket says 172.21.15.44:57824) 2024-07-17T08:39:35.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 -- 172.21.15.44:0/2575604875 learned_addr learned my addr 172.21.15.44:0/2575604875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:35.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 -- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0b1af0 msgr2=0x7f44bc15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44b3e00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0b1af0 0x7f44bc15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44c1200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc15a280 0x7f44bc1486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0b1af0 0x7f44bc15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 -- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc15a280 msgr2=0x7f44bc1486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc15a280 0x7f44bc1486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:35.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.231+0000 7f44bbe00640 1 -- 172.21.15.44:0/2575604875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc15c1f0 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44bbe00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c970 tx=0x7f44ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac007bb0 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44ac007e40 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc158150 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac005280 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.232+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc15c750 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.233+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc15ac10 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.234+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 166) ==== 98540+0+0 (secure 0 0 0) 0x7f44ac005420 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.236+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(283..283 src has 1..283) ==== 7738+0+0 (secure 0 0 0) 0x7f44ac098a60 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.240+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f44bc15c1f0 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.498+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f44bc0ac330 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.928+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 167) ==== 98667+0+0 (secure 0 0 0) 0x7f44ac05cdd0 con 0x7f44bc0ac8a0 2024-07-17T08:39:35.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.929+0000 7f44b8c00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 0x7f449807f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:35.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.938+0000 7f44b3e00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 0x7f449807f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:35.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:35.947+0000 7f44b3e00640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 0x7f449807f870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f44bc1f73f0 tx=0x7f44a4009040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:36.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.035+0000 7f44b8c00640 1 -- 172.21.15.44:0/2575604875 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v157) ==== 104+0+0 (secure 0 0 0) 0x7f44ac014030 con 0x7f44bc0ac8a0 2024-07-17T08:39:36.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 msgr2=0x7f449807f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 0x7f449807f870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f44bc1f73f0 tx=0x7f44a4009040 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 msgr2=0x7f44bc156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00c970 tx=0x7f44ac00ce30 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 shutdown_connections 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f449807d480 0x7f449807f870 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44bc15a280 0x7f44bc1486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44bc0b1af0 0x7f44bc15d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 --2- 172.21.15.44:0/2575604875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44bc0ac8a0 0x7f44bc156740 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 >> 172.21.15.44:0/2575604875 conn(0x7f44bc0b0eb0 msgr2=0x7f44bc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:36.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 shutdown_connections 2024-07-17T08:39:36.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.039+0000 7f44c3200640 1 -- 172.21.15.44:0/2575604875 wait complete. 2024-07-17T08:39:36.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:39:36.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 -- 172.21.15.44:0/447287249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4240ae110 msgr2=0x7fd4240ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 --2- 172.21.15.44:0/447287249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4240ae110 0x7fd4240ae570 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fd410005fd0 tx=0x7fd41002b860 comp rx=0 tx=0).stop 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 -- 172.21.15.44:0/447287249 shutdown_connections 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 --2- 172.21.15.44:0/447287249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4240ae110 0x7fd4240ae570 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 --2- 172.21.15.44:0/447287249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd42406f460 0x7fd4241537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.267+0000 7fd42a200640 1 -- 172.21.15.44:0/447287249 >> 172.21.15.44:0/447287249 conn(0x7fd4240cf050 msgr2=0x7fd4240cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.268+0000 7fd42a200640 1 -- 172.21.15.44:0/447287249 shutdown_connections 2024-07-17T08:39:36.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.268+0000 7fd42a200640 1 -- 172.21.15.44:0/447287249 wait complete. 2024-07-17T08:39:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.269+0000 7fd42a200640 1 Processor -- start 2024-07-17T08:39:36.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.270+0000 7fd42a200640 1 -- start start 2024-07-17T08:39:36.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:36.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4240abfa0 0x7fd42414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:36.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 0x7fd42414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:36.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240ac3f0 con 0x7fd42406f460 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240b2fe0 con 0x7fd4240abfa0 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd42a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240ac570 con 0x7fd4240ae110 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57838/0 (socket says 172.21.15.44:57838) 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 -- 172.21.15.44:0/298005730 learned_addr learned my addr 172.21.15.44:0/298005730 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd423e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4240abfa0 0x7fd42414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd429800640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 0x7fd42414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:36.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 -- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 msgr2=0x7fd42414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 0x7fd42414f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 -- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4240abfa0 msgr2=0x7fd42414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4240abfa0 0x7fd42414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd428e00640 1 -- 172.21.15.44:0/298005730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240cc1f0 con 0x7fd42406f460 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.271+0000 7fd429800640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 0x7fd42414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:36.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd428e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fd41800d950 tx=0x7fd41800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418017070 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd418009d20 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180106c0 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4240cc4d0 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.272+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4240cca90 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.273+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4241537d0 con 0x7fd42406f460 2024-07-17T08:39:36.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.275+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 167) ==== 98667+0+0 (secure 0 0 0) 0x7fd418010860 con 0x7fd42406f460 2024-07-17T08:39:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.276+0000 7fd420c00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 0x7fd404078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.276+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(283..283 src has 1..283) ==== 7738+0+0 (secure 0 0 0) 0x7fd418098b60 con 0x7fd42406f460 2024-07-17T08:39:36.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.277+0000 7fd423e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 0x7fd404078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:36.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.278+0000 7fd423e00640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 0x7fd404078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd410035000 tx=0x7fd410004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:36.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.281+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd418061ab0 con 0x7fd42406f460 2024-07-17T08:39:36.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:36.481+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd4240cfca0 con 0x7fd42406f460 2024-07-17T08:39:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.039+0000 7fd420c00640 1 -- 172.21.15.44:0/298005730 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v284) ==== 181+0+0 (secure 0 0 0) 0x7fd4180613b0 con 0x7fd42406f460 2024-07-17T08:39:37.041 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:39:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.042+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 msgr2=0x7fd404078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.042+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 0x7fd404078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd410035000 tx=0x7fd410004730 comp rx=0 tx=0).stop 2024-07-17T08:39:37.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.042+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 msgr2=0x7fd4241f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.042+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fd41800d950 tx=0x7fd41800de10 comp rx=0 tx=0).stop 2024-07-17T08:39:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 shutdown_connections 2024-07-17T08:39:37.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd404076a00 0x7fd404078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4240ae110 0x7fd42414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4240abfa0 0x7fd42414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 --2- 172.21.15.44:0/298005730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42406f460 0x7fd4241f6eb0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 >> 172.21.15.44:0/298005730 conn(0x7fd4240cf050 msgr2=0x7fd42415b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 shutdown_connections 2024-07-17T08:39:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.043+0000 7fd42a200640 1 -- 172.21.15.44:0/298005730 wait complete. 2024-07-17T08:39:37.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:39:37.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 -- 172.21.15.44:0/2278155501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b015a5d0 msgr2=0x7ff8b00ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:37.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/2278155501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b015a5d0 0x7ff8b00ac3a0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7ff8b015cef0 tx=0x7ff8a002adc0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 -- 172.21.15.44:0/2278155501 shutdown_connections 2024-07-17T08:39:37.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/2278155501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b00ac8e0 0x7ff8b00b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/2278155501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b015a5d0 0x7ff8b00ac3a0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 -- 172.21.15.44:0/2278155501 >> 172.21.15.44:0/2278155501 conn(0x7ff8b00b0eb0 msgr2=0x7ff8b00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:37.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 -- 172.21.15.44:0/2278155501 shutdown_connections 2024-07-17T08:39:37.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.261+0000 7ff8b7a00640 1 -- 172.21.15.44:0/2278155501 wait complete. 2024-07-17T08:39:37.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.263+0000 7ff8b7a00640 1 Processor -- start 2024-07-17T08:39:37.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.264+0000 7ff8b7a00640 1 -- start start 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8b00ac8e0 0x7ff8b00cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 0x7ff8b00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b006f060 con 0x7ff8b01539f0 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b006f8e0 con 0x7ff8b00ac8e0 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8b7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b00b26f0 con 0x7ff8b015a5d0 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57858/0 (socket says 172.21.15.44:57858) 2024-07-17T08:39:37.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.265+0000 7ff8afe00640 1 -- 172.21.15.44:0/116511912 learned_addr learned my addr 172.21.15.44:0/116511912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8b5000640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8b00ac8e0 0x7ff8b00cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8b5a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 0x7ff8b00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 -- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 msgr2=0x7ff8b00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 0x7ff8b00ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 -- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8b00ac8e0 msgr2=0x7ff8b00cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8b00ac8e0 0x7ff8b00cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 -- 172.21.15.44:0/116511912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b00b3f90 con 0x7ff8b01539f0 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8b5a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 0x7ff8b00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:37.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8afe00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000b3c0 tx=0x7ff8a0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0040020 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8a002ba00 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a002bdf0 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.266+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b01f5a50 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.267+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b00cc960 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.268+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b00b21b0 con 0x7ff8b01539f0 2024-07-17T08:39:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.269+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 168) ==== 98773+0+0 (secure 0 0 0) 0x7ff8a0043020 con 0x7ff8b01539f0 2024-07-17T08:39:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.270+0000 7ff8acc00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 0x7ff894078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.270+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(284..284 src has 1..284) ==== 7300+0+0 (secure 0 0 0) 0x7ff8a00b8ef0 con 0x7ff8b01539f0 2024-07-17T08:39:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.271+0000 7ff8b5000640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 0x7ff894078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.272+0000 7ff8b5000640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 0x7ff894078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4007fd0 tx=0x7ff8a4009040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:37.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.274+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff8a004a030 con 0x7ff8b01539f0 2024-07-17T08:39:37.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:37.478+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8b0075550 con 0x7ff8b01539f0 2024-07-17T08:39:38.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.058+0000 7ff8acc00640 1 -- 172.21.15.44:0/116511912 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v285) ==== 169+0+0 (secure 0 0 0) 0x7ff8a0081d30 con 0x7ff8b01539f0 2024-07-17T08:39:38.060 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 msgr2=0x7ff894078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 0x7ff894078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4007fd0 tx=0x7ff8a4009040 comp rx=0 tx=0).stop 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 msgr2=0x7ff8b00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000b3c0 tx=0x7ff8a0009850 comp rx=0 tx=0).stop 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 shutdown_connections 2024-07-17T08:39:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ff894076b20 0x7ff894078fe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8b015a5d0 0x7ff8b00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8b01539f0 0x7ff8b00cb5e0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 --2- 172.21.15.44:0/116511912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8b00ac8e0 0x7ff8b00cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 >> 172.21.15.44:0/116511912 conn(0x7ff8b00b0eb0 msgr2=0x7ff8b00737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 shutdown_connections 2024-07-17T08:39:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.062+0000 7ff8b7a00640 1 -- 172.21.15.44:0/116511912 wait complete. 2024-07-17T08:39:38.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 -- 172.21.15.44:0/209567366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a415d130 msgr2=0x7f09a415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 --2- 172.21.15.44:0/209567366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a415d130 0x7f09a415d510 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f098c005fd0 tx=0x7f098c02a4b0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 -- 172.21.15.44:0/209567366 shutdown_connections 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 --2- 172.21.15.44:0/209567366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a4161be0 0x7f09a40b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 --2- 172.21.15.44:0/209567366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a415d130 0x7f09a415d510 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 -- 172.21.15.44:0/209567366 >> 172.21.15.44:0/209567366 conn(0x7f09a40b4ff0 msgr2=0x7f09a40b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 -- 172.21.15.44:0/209567366 shutdown_connections 2024-07-17T08:39:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.285+0000 7f09a3e00640 1 -- 172.21.15.44:0/209567366 wait complete. 2024-07-17T08:39:38.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.287+0000 7f09a3e00640 1 Processor -- start 2024-07-17T08:39:38.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.287+0000 7f09a3e00640 1 -- start start 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 0x7f09a41f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a415d130 0x7f09a4159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40d1a10 con 0x7f09a4161be0 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40d1890 con 0x7f09a415d130 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09a40d1b90 con 0x7f09a40ce690 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.288+0000 7f09a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57864/0 (socket says 172.21.15.44:57864) 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 -- 172.21.15.44:0/3953900696 learned_addr learned my addr 172.21.15.44:0/3953900696 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a2a00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 0x7f09a41f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a2000640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a415d130 0x7f09a4159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 -- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 msgr2=0x7f09a41f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 0x7f09a41f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 -- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a415d130 msgr2=0x7f09a4159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a415d130 0x7f09a4159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 -- 172.21.15.44:0/3953900696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41f8630 con 0x7f09a4161be0 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a2a00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 0x7f09a41f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:38.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.289+0000 7f09a3400640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b440 tx=0x7f099c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.290+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c015020 con 0x7f09a4161be0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.290+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f099c011920 con 0x7f09a4161be0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.290+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c011d70 con 0x7f09a4161be0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.290+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41f4780 con 0x7f09a4161be0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.290+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41f4d40 con 0x7f09a4161be0 2024-07-17T08:39:38.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.291+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a40b84e0 con 0x7f09a4161be0 2024-07-17T08:39:38.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.292+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 168) ==== 98773+0+0 (secure 0 0 0) 0x7f099c018020 con 0x7f09a4161be0 2024-07-17T08:39:38.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.293+0000 7f099b400640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 0x7f0980078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.294+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f099c098310 con 0x7f09a4161be0 2024-07-17T08:39:38.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.294+0000 7f09a2a00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 0x7f0980078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.296+0000 7f09a2a00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 0x7f0980078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f098c033000 tx=0x7f098c004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:38.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.298+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f09a40b84e0 con 0x7f09a4161be0 2024-07-17T08:39:38.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.522+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f09a40d0e10 con 0x7f09a4161be0 2024-07-17T08:39:38.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.530+0000 7f099b400640 1 -- 172.21.15.44:0/3953900696 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v87) ==== 71+0+0 (secure 0 0 0) 0x7f099c01f030 con 0x7f09a4161be0 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 msgr2=0x7f0980078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 0x7f0980078fe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f098c033000 tx=0x7f098c004730 comp rx=0 tx=0).stop 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 msgr2=0x7f09a41f80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b440 tx=0x7f099c00b900 comp rx=0 tx=0).stop 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 shutdown_connections 2024-07-17T08:39:38.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f0980076b20 0x7f0980078fe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09a4161be0 0x7f09a41f80f0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a415d130 0x7f09a4159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 --2- 172.21.15.44:0/3953900696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09a40ce690 0x7f09a41f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 >> 172.21.15.44:0/3953900696 conn(0x7f09a40b4ff0 msgr2=0x7f09a40b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.533+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 shutdown_connections 2024-07-17T08:39:38.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.534+0000 7f09a3e00640 1 -- 172.21.15.44:0/3953900696 wait complete. 2024-07-17T08:39:38.554 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01' 2024-07-17T08:39:38.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 -- 172.21.15.44:0/3630379283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b2ad0 msgr2=0x7f57040b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 --2- 172.21.15.44:0/3630379283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b2ad0 0x7f57040b5200 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f5704069c00 tx=0x7f56f002aac0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 -- 172.21.15.44:0/3630379283 shutdown_connections 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 --2- 172.21.15.44:0/3630379283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b2ad0 0x7f57040b5200 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 --2- 172.21.15.44:0/3630379283 >> v2:172.21.15.175:3300/0 conn(0x7f570415d600 0x7f570406e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 -- 172.21.15.44:0/3630379283 >> 172.21.15.44:0/3630379283 conn(0x7f57040d00a0 msgr2=0x7f57040d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 -- 172.21.15.44:0/3630379283 shutdown_connections 2024-07-17T08:39:38.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.758+0000 7f5709000640 1 -- 172.21.15.44:0/3630379283 wait complete. 2024-07-17T08:39:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.761+0000 7f5709000640 1 Processor -- start 2024-07-17T08:39:38.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.761+0000 7f5709000640 1 -- start start 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57040b2ad0 0x7f57041f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570415d600 0x7f570415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5704161730 con 0x7f57040b5740 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57040cef10 con 0x7f57040b2ad0 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.762+0000 7f5709000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57041618b0 con 0x7f570415d600 2024-07-17T08:39:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57878/0 (socket says 172.21.15.44:57878) 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 -- 172.21.15.44:0/1571321432 learned_addr learned my addr 172.21.15.44:0/1571321432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701e00640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57040b2ad0 0x7f57041f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5702800640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570415d600 0x7f570415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 -- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570415d600 msgr2=0x7f570415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570415d600 0x7f570415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 -- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57040b2ad0 msgr2=0x7f57041f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57040b2ad0 0x7f57041f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.763+0000 7f5701400640 1 -- 172.21.15.44:0/1571321432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57041f5170 con 0x7f57040b5740 2024-07-17T08:39:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f5701400640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009bf0 tx=0x7f56f0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0044070 con 0x7f57040b5740 2024-07-17T08:39:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f0004250 con 0x7f57040b5740 2024-07-17T08:39:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0004680 con 0x7f57040b5740 2024-07-17T08:39:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57040cb350 con 0x7f57040b5740 2024-07-17T08:39:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.764+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57040cc350 con 0x7f57040b5740 2024-07-17T08:39:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.765+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5704155a80 con 0x7f57040b5740 2024-07-17T08:39:38.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.766+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 168) ==== 98773+0+0 (secure 0 0 0) 0x7f56f002c8c0 con 0x7f57040b5740 2024-07-17T08:39:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.767+0000 7f56fa000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 0x7f56e4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.767+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f56f00b8c20 con 0x7f57040b5740 2024-07-17T08:39:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.768+0000 7f5701e00640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 0x7f56e4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.769+0000 7f5701e00640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 0x7f56e4079040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005fd0 tx=0x7f56f4005850 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:38.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:38.772+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5704155a80 con 0x7f57040b5740 2024-07-17T08:39:39.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:39.067+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01"]} v 0) -- 0x7f5704161df0 con 0x7f57040b5740 2024-07-17T08:39:39.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:39.552+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f56f007d910 con 0x7f57040b5740 2024-07-17T08:39:40.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.065+0000 7f56fa000640 1 -- 172.21.15.44:0/1571321432 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_01"]}]=0 v3479) ==== 201+0+0 (secure 0 0 0) 0x7f56f0032030 con 0x7f57040b5740 2024-07-17T08:39:40.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.068+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 msgr2=0x7f56e4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.068+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 0x7f56e4079040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005fd0 tx=0x7f56f4005850 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.068+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 msgr2=0x7f57040b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.068+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009bf0 tx=0x7f56f0003f10 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 shutdown_connections 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f56e4076b80 0x7f56e4079040 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f570415d600 0x7f570415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57040b5740 0x7f57040b6570 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 --2- 172.21.15.44:0/1571321432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57040b2ad0 0x7f57041f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 >> 172.21.15.44:0/1571321432 conn(0x7f57040d00a0 msgr2=0x7f570406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 shutdown_connections 2024-07-17T08:39:40.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.069+0000 7f5709000640 1 -- 172.21.15.44:0/1571321432 wait complete. 2024-07-17T08:39:40.091 INFO:tasks.cephfs_test_runner:Test subvol schedule creation succeeds for default subvolgroup. ... ok 2024-07-17T08:39:40.091 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_02 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule creation fails for non-default subvolgroup. 2024-07-17T08:39:40.091 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_02 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:39:40.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:40.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 -- 172.21.15.44:0/1613121187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc004070 msgr2=0x7ffadc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 --2- 172.21.15.44:0/1613121187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffadc004070 0x7ffadc004800 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c260 tx=0x7ffadc005830 comp rx=0 tx=0).stop 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffaf7e00640 1 -- 172.21.15.44:0/1613121187 reap_dead start 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 -- 172.21.15.44:0/1613121187 shutdown_connections 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 -- 172.21.15.44:0/1613121187 >> 172.21.15.44:0/1613121187 conn(0x7ffaf80b0e80 msgr2=0x7ffaf80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 -- 172.21.15.44:0/1613121187 shutdown_connections 2024-07-17T08:39:40.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.304+0000 7ffafd000640 1 -- 172.21.15.44:0/1613121187 wait complete. 2024-07-17T08:39:40.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.306+0000 7ffafd000640 1 Processor -- start 2024-07-17T08:39:40.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.306+0000 7ffafd000640 1 -- start start 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.307+0000 7ffafd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 0x7ffaf8152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.307+0000 7ffafd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf80ccf40 0x7ffaf80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffafd000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffafd000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf815a2a0 con 0x7ffaf81582d0 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffafd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf815a120 con 0x7ffaf80ccf40 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffafd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf80b1fa0 con 0x7ffadc004070 2024-07-17T08:39:40.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59022/0 (socket says 172.21.15.44:59022) 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 -- 172.21.15.44:0/4243327786 learned_addr learned my addr 172.21.15.44:0/4243327786 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaefe00640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf80ccf40 0x7ffaf80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7400640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 0x7ffaf8152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 -- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 msgr2=0x7ffaf8152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 0x7ffaf8152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 -- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf80ccf40 msgr2=0x7ffaf80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf80ccf40 0x7ffaf80cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7e00640 1 -- 172.21.15.44:0/4243327786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf80cdab0 con 0x7ffaf81582d0 2024-07-17T08:39:40.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.308+0000 7ffaf7400640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 0x7ffaf8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffaf7e00640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ffaf809c0e0 tx=0x7ffaf00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0011870 con 0x7ffaf81582d0 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf0017070 con 0x7ffaf81582d0 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf81f4010 con 0x7ffaf81582d0 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0011de0 con 0x7ffaf81582d0 2024-07-17T08:39:40.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.309+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81f45d0 con 0x7ffaf81582d0 2024-07-17T08:39:40.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.310+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf80cf380 con 0x7ffaf81582d0 2024-07-17T08:39:40.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.311+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7ffaf001c460 con 0x7ffaf81582d0 2024-07-17T08:39:40.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.312+0000 7ffaef400640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 0x7ffad40790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.312+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7ffaf0098ee0 con 0x7ffaf81582d0 2024-07-17T08:39:40.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.313+0000 7ffaf7400640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 0x7ffad40790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.314+0000 7ffaf7400640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 0x7ffad40790a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009a90 tx=0x7ffae8005f90 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:40.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.317+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffaf0061ed0 con 0x7ffaf81582d0 2024-07-17T08:39:40.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.534+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffaf80b1400 con 0x7ffaf81582d0 2024-07-17T08:39:40.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.535+0000 7ffaef400640 1 -- 172.21.15.44:0/4243327786 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v285) ==== 74+0+15485 (secure 0 0 0) 0x7ffaf006a110 con 0x7ffaf81582d0 2024-07-17T08:39:40.537 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:40.538 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":285,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:38.056316+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":38,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":279,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":275,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/4265556511":"2024-07-18T08:39:35.030833+0000","172.21.15.44:6833/4265556511":"2024-07-18T08:39:35.030833+0000","172.21.15.175:0/2218756844":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/3296654801":"2024-07-18T08:39:34.905797+0000","172.21.15.175:6834/3098340808":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/4201645279":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/1523107768":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/3868933550":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/2984331372":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/3831547450":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/1265605150":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/3233777945":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/1260211667":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3073710198":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/2738299478":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6837/3098340808":"2024-07-18T08:39:34.905797+0000","172.21.15.175:0/3444992690":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3343515448":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/3797609750":"2024-07-18T08:38:10.178682+0000","172.21.15.175:0/2038197500":"2024-07-18T08:39:34.905797+0000","172.21.15.175:6837/2531126035":"2024-07-18T08:38:10.178682+0000","172.21.15.175:6834/2531126035":"2024-07-18T08:38:10.178682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.538+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 msgr2=0x7ffad40790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.538+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 0x7ffad40790a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009a90 tx=0x7ffae8005f90 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.538+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 msgr2=0x7ffaf8158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.538+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ffaf809c0e0 tx=0x7ffaf00019b0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 shutdown_connections 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7ffad4076be0 0x7ffad40790a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffaf81582d0 0x7ffaf8158750 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffaf80ccf40 0x7ffaf80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 --2- 172.21.15.44:0/4243327786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffadc004070 0x7ffaf8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 >> 172.21.15.44:0/4243327786 conn(0x7ffaf80b0e80 msgr2=0x7ffaf80b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 shutdown_connections 2024-07-17T08:39:40.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.539+0000 7ffafd000640 1 -- 172.21.15.44:0/4243327786 wait complete. 2024-07-17T08:39:40.561 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:39:40.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 -- 172.21.15.44:0/1767917050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c0cd0b0 msgr2=0x7f3a9c06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 --2- 172.21.15.44:0/1767917050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c0cd0b0 0x7f3a9c06ec70 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005fd0 tx=0x7f3a8402a5a0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 -- 172.21.15.44:0/1767917050 shutdown_connections 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 --2- 172.21.15.44:0/1767917050 >> v2:172.21.15.175:3300/0 conn(0x7f3a9c152300 0x7f3a9c06f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 --2- 172.21.15.44:0/1767917050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c0cd0b0 0x7f3a9c06ec70 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 -- 172.21.15.44:0/1767917050 >> 172.21.15.44:0/1767917050 conn(0x7f3a9c0cf0d0 msgr2=0x7f3a9c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 -- 172.21.15.44:0/1767917050 shutdown_connections 2024-07-17T08:39:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.776+0000 7f3aa1600640 1 -- 172.21.15.44:0/1767917050 wait complete. 2024-07-17T08:39:40.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.778+0000 7f3aa1600640 1 Processor -- start 2024-07-17T08:39:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.778+0000 7f3aa1600640 1 -- start start 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a9c0cd0b0 0x7f3a9c072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c152300 0x7f3a9c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c0cb6e0 con 0x7f3a9c151f20 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c0cb560 con 0x7f3a9c152300 2024-07-17T08:39:40.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3aa1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9c0cb860 con 0x7f3a9c0cd0b0 2024-07-17T08:39:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3a93400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3a93400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59044/0 (socket says 172.21.15.44:59044) 2024-07-17T08:39:40.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.779+0000 7f3a93400640 1 -- 172.21.15.44:0/3485987546 learned_addr learned my addr 172.21.15.44:0/3485987546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 -- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a9c0cd0b0 msgr2=0x7f3a9c072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a9c0cd0b0 0x7f3a9c072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3aa0c00640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c152300 0x7f3a9c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 -- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c152300 msgr2=0x7f3a9c14bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c152300 0x7f3a9c14bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 -- 172.21.15.44:0/3485987546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c1f4e00 con 0x7f3a9c151f20 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a93400640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006010 tx=0x7f3a840039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8403f070 con 0x7f3a9c151f20 2024-07-17T08:39:40.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a84004890 con 0x7f3a9c151f20 2024-07-17T08:39:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84035030 con 0x7f3a9c151f20 2024-07-17T08:39:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.780+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c0730d0 con 0x7f3a9c151f20 2024-07-17T08:39:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.781+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c073690 con 0x7f3a9c151f20 2024-07-17T08:39:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.782+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9c15c140 con 0x7f3a9c151f20 2024-07-17T08:39:40.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.783+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f3a84042070 con 0x7f3a9c151f20 2024-07-17T08:39:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.784+0000 7f3a98c00640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 0x7f3a78079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:40.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.784+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f3a840b9610 con 0x7f3a9c151f20 2024-07-17T08:39:40.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.785+0000 7f3a9be00640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 0x7f3a78079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:40.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.786+0000 7f3a9be00640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 0x7f3a78079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0012e0 tx=0x7f3a8c007d30 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:40.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:40.788+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3a84082600 con 0x7f3a9c151f20 2024-07-17T08:39:40.829 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:39:40.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:39:41.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.000+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a9c0008d0 con 0x7f3a9c151f20 2024-07-17T08:39:41.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.001+0000 7f3a98c00640 1 -- 172.21.15.44:0/3485987546 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v285) ==== 93+0+10 (secure 0 0 0) 0x7f3a84081f00 con 0x7f3a9c151f20 2024-07-17T08:39:41.002 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:39:41.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 msgr2=0x7f3a78079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 0x7f3a78079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0012e0 tx=0x7f3a8c007d30 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 msgr2=0x7f3a9c072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006010 tx=0x7f3a840039e0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 shutdown_connections 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f3a78076bb0 0x7f3a78079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c152300 0x7f3a9c14bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a9c151f20 0x7f3a9c072aa0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 --2- 172.21.15.44:0/3485987546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a9c0cd0b0 0x7f3a9c072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.003+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 >> 172.21.15.44:0/3485987546 conn(0x7f3a9c0cf0d0 msgr2=0x7f3a9c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.004+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 shutdown_connections 2024-07-17T08:39:41.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.004+0000 7f3aa1600640 1 -- 172.21.15.44:0/3485987546 wait complete. 2024-07-17T08:39:41.025 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02' 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 -- 172.21.15.44:0/1644651061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40acaf0 msgr2=0x7fe7c406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/1644651061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40acaf0 0x7fe7c406e090 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4070a70 tx=0x7fe7bc02b880 comp rx=0 tx=0).stop 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 -- 172.21.15.44:0/1644651061 shutdown_connections 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/1644651061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c40acaf0 0x7fe7c406e090 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/1644651061 >> v2:172.21.15.175:3300/0 conn(0x7fe7c4153a50 0x7fe7c4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 -- 172.21.15.44:0/1644651061 >> 172.21.15.44:0/1644651061 conn(0x7fe7c40b0fb0 msgr2=0x7fe7c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 -- 172.21.15.44:0/1644651061 shutdown_connections 2024-07-17T08:39:41.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.237+0000 7fe7c9e00640 1 -- 172.21.15.44:0/1644651061 wait complete. 2024-07-17T08:39:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.239+0000 7fe7c9e00640 1 Processor -- start 2024-07-17T08:39:41.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.239+0000 7fe7c9e00640 1 -- start start 2024-07-17T08:39:41.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c40acaf0 0x7fe7c41f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4153a50 0x7fe7c4152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c406f710 con 0x7fe7c415a040 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4154040 con 0x7fe7c40acaf0 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c406f890 con 0x7fe7c4153a50 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59062/0 (socket says 172.21.15.44:59062) 2024-07-17T08:39:41.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c3400640 1 -- 172.21.15.44:0/621170436 learned_addr learned my addr 172.21.15.44:0/621170436 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c3400640 1 -- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4153a50 msgr2=0x7fe7c4152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c2a00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c40acaf0 0x7fe7c41f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.240+0000 7fe7c2000640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4153a50 0x7fe7c4152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c3400640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4153a50 0x7fe7c4152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c3400640 1 -- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c40acaf0 msgr2=0x7fe7c41f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c3400640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c40acaf0 0x7fe7c41f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c3400640 1 -- 172.21.15.44:0/621170436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c41553b0 con 0x7fe7c415a040 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c3400640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc034040 tx=0x7fe7bc003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc040030 con 0x7fe7c415a040 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7bc003c80 con 0x7fe7c415a040 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c4152af0 con 0x7fe7c415a040 2024-07-17T08:39:41.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0047a0 con 0x7fe7c415a040 2024-07-17T08:39:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.241+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c41530b0 con 0x7fe7c415a040 2024-07-17T08:39:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.243+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c406e090 con 0x7fe7c415a040 2024-07-17T08:39:41.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.244+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fe7bc043030 con 0x7fe7c415a040 2024-07-17T08:39:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.245+0000 7fe7bb400640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 0x7fe7a0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:41.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.245+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7fe7bc0b94d0 con 0x7fe7c415a040 2024-07-17T08:39:41.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.246+0000 7fe7c2a00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 0x7fe7a0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:41.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.247+0000 7fe7c2a00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 0x7fe7a0079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4005c40 tx=0x7fe7b4005bd0 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.249+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe7c41553b0 con 0x7fe7c415a040 2024-07-17T08:39:41.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:41.544+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02"]} v 0) -- 0x7fe7c40af150 con 0x7fe7c415a040 2024-07-17T08:39:42.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.077+0000 7fe7bb400640 1 -- 172.21.15.44:0/621170436 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02"]}]=0 v3481) ==== 204+0+0 (secure 0 0 0) 0x7fe7bc02f050 con 0x7fe7c415a040 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 msgr2=0x7fe7a0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 0x7fe7a0079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4005c40 tx=0x7fe7b4005bd0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 msgr2=0x7fe7c40b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc034040 tx=0x7fe7bc003880 comp rx=0 tx=0).stop 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 shutdown_connections 2024-07-17T08:39:42.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe7a0076bb0 0x7fe7a0079070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7c415a040 0x7fe7c40b39d0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7c4153a50 0x7fe7c4152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 --2- 172.21.15.44:0/621170436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7c40acaf0 0x7fe7c41f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 >> 172.21.15.44:0/621170436 conn(0x7fe7c40b0fb0 msgr2=0x7fe7c415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 shutdown_connections 2024-07-17T08:39:42.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.081+0000 7fe7c9e00640 1 -- 172.21.15.44:0/621170436 wait complete. 2024-07-17T08:39:42.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 -- 172.21.15.44:0/542687760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd25815cb20 msgr2=0x7fd2580cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 --2- 172.21.15.44:0/542687760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd25815cb20 0x7fd2580cf090 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fd244005fd0 tx=0x7fd24402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 -- 172.21.15.44:0/542687760 shutdown_connections 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 --2- 172.21.15.44:0/542687760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd25815cb20 0x7fd2580cf090 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 -- 172.21.15.44:0/542687760 >> 172.21.15.44:0/542687760 conn(0x7fd25806ce80 msgr2=0x7fd25806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:42.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 -- 172.21.15.44:0/542687760 shutdown_connections 2024-07-17T08:39:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.322+0000 7fd25e600640 1 -- 172.21.15.44:0/542687760 wait complete. 2024-07-17T08:39:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.324+0000 7fd25e600640 1 Processor -- start 2024-07-17T08:39:42.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.324+0000 7fd25e600640 1 -- start start 2024-07-17T08:39:42.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2580cf5d0 0x7fd2581f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25815cb20 0x7fd2580cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd258152940 con 0x7fd258152420 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd258070470 con 0x7fd2580cf5d0 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd25e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580ce660 con 0x7fd25815cb20 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd256a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59080/0 (socket says 172.21.15.44:59080) 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.325+0000 7fd256a00640 1 -- 172.21.15.44:0/2651906558 learned_addr learned my addr 172.21.15.44:0/2651906558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd257400640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2580cf5d0 0x7fd2581f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd257e00640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25815cb20 0x7fd2580cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 -- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25815cb20 msgr2=0x7fd2580cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25815cb20 0x7fd2580cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 -- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2580cf5d0 msgr2=0x7fd2581f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2580cf5d0 0x7fd2581f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 -- 172.21.15.44:0/2651906558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2581f8a30 con 0x7fd258152420 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd256a00640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fd244006010 tx=0x7fd2440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:42.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24403e030 con 0x7fd258152420 2024-07-17T08:39:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd244003ca0 con 0x7fd258152420 2024-07-17T08:39:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244035040 con 0x7fd258152420 2024-07-17T08:39:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.326+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25806e310 con 0x7fd258152420 2024-07-17T08:39:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.327+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25806e8d0 con 0x7fd258152420 2024-07-17T08:39:42.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.328+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2580ceb00 con 0x7fd258152420 2024-07-17T08:39:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.329+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fd244041030 con 0x7fd258152420 2024-07-17T08:39:42.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.330+0000 7fd24f400640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 0x7fd238079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.330+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7fd2440b8c00 con 0x7fd258152420 2024-07-17T08:39:42.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.331+0000 7fd257400640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 0x7fd238079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.332+0000 7fd257400640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 0x7fd238079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd248001680 tx=0x7fd248009040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:42.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.334+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd2581f8a30 con 0x7fd258152420 2024-07-17T08:39:42.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.545+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd258068e70 con 0x7fd258152420 2024-07-17T08:39:42.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.546+0000 7fd24f400640 1 -- 172.21.15.44:0/2651906558 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v87) ==== 59+0+1998 (secure 0 0 0) 0x7fd244081c80 con 0x7fd258152420 2024-07-17T08:39:42.549 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:39:42.549 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:39:42.550 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:39:42.551 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:39:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.549+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 msgr2=0x7fd238079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.549+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 0x7fd238079070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd248001680 tx=0x7fd248009040 comp rx=0 tx=0).stop 2024-07-17T08:39:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.549+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 msgr2=0x7fd25806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.549+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fd244006010 tx=0x7fd2440038a0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 shutdown_connections 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fd238076bb0 0x7fd238079070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25815cb20 0x7fd2580cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd258152420 0x7fd25806f3b0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 --2- 172.21.15.44:0/2651906558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2580cf5d0 0x7fd2581f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 >> 172.21.15.44:0/2651906558 conn(0x7fd25806ce80 msgr2=0x7fd25815abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 shutdown_connections 2024-07-17T08:39:42.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.550+0000 7fd25e600640 1 -- 172.21.15.44:0/2651906558 wait complete. 2024-07-17T08:39:42.570 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 -- 172.21.15.44:0/1578121736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 msgr2=0x7f664c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 --2- 172.21.15.44:0/1578121736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c0b4ec0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f6638009830 tx=0x7f663802a530 comp rx=0 tx=0).stop 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 -- 172.21.15.44:0/1578121736 shutdown_connections 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 --2- 172.21.15.44:0/1578121736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c0b4ec0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 -- 172.21.15.44:0/1578121736 >> 172.21.15.44:0/1578121736 conn(0x7f664c0cfec0 msgr2=0x7f664c0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 -- 172.21.15.44:0/1578121736 shutdown_connections 2024-07-17T08:39:42.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.780+0000 7f6652e00640 1 -- 172.21.15.44:0/1578121736 wait complete. 2024-07-17T08:39:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.782+0000 7f6652e00640 1 Processor -- start 2024-07-17T08:39:42.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.782+0000 7f6652e00640 1 -- start start 2024-07-17T08:39:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f664c0d2900 0x7f664c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 0x7f664c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f664c0d1190 con 0x7f664c15d300 2024-07-17T08:39:42.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f664c0d1010 con 0x7f664c0d2fa0 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f664c0d1310 con 0x7f664c0d2900 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59102/0 (socket says 172.21.15.44:59102) 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6652400640 1 -- 172.21.15.44:0/1082391703 learned_addr learned my addr 172.21.15.44:0/1082391703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.783+0000 7f6651000640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 0x7f664c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 -- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f664c0d2900 msgr2=0x7f664c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f664c0d2900 0x7f664c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 -- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 msgr2=0x7f664c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 0x7f664c0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 -- 172.21.15.44:0/1082391703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c160c20 con 0x7f664c15d300 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6651000640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 0x7f664c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:42.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652400640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f664400cbf0 tx=0x7f664400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:42.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644016020 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f664400acf0 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644012680 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c160380 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.784+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c160940 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.786+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664c0b56a0 con 0x7f664c15d300 2024-07-17T08:39:42.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.787+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f6644019020 con 0x7f664c15d300 2024-07-17T08:39:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.788+0000 7f6649600640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 0x7f662c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.788+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f6644098b50 con 0x7f664c15d300 2024-07-17T08:39:42.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.789+0000 7f6651a00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 0x7f662c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:42.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.790+0000 7f6651a00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 0x7f662c079070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6638034000 tx=0x7f66380046c0 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:42.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:42.793+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6644020030 con 0x7f664c15d300 2024-07-17T08:39:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.002+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f664c0d2500 con 0x7f664c15d300 2024-07-17T08:39:43.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.010+0000 7f6649600640 1 -- 172.21.15.44:0/1082391703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v88)=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7f6644061bd0 con 0x7f664c15d300 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.012+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 msgr2=0x7f662c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.012+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 0x7f662c079070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6638034000 tx=0x7f66380046c0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.012+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 msgr2=0x7f664c1f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.012+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f664400cbf0 tx=0x7f664400a430 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 shutdown_connections 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f662c076bb0 0x7f662c079070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f664c15d300 0x7f664c1f7b00 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f664c0d2fa0 0x7f664c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 --2- 172.21.15.44:0/1082391703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f664c0d2900 0x7f664c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 >> 172.21.15.44:0/1082391703 conn(0x7f664c0cfec0 msgr2=0x7f664c0b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 shutdown_connections 2024-07-17T08:39:43.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.013+0000 7f6652e00640 1 -- 172.21.15.44:0/1082391703 wait complete. 2024-07-17T08:39:43.035 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:39:43.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:39:43.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:39:43.272 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:39:43.272 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:43.272 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:43.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:39:43.332 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:39:43.332 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:43.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:39:43.387 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:43.387 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:43.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:39:43.445 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:39:43.445 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:43.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:39:43.502 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:39:43.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 -- 172.21.15.44:0/1895298437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf540 msgr2=0x7f37d00b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 --2- 172.21.15.44:0/1895298437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf540 0x7f37d00b3870 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005fd0 tx=0x7f37c002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 -- 172.21.15.44:0/1895298437 shutdown_connections 2024-07-17T08:39:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 --2- 172.21.15.44:0/1895298437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 0x7f37d00ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 --2- 172.21.15.44:0/1895298437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf540 0x7f37d00b3870 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 -- 172.21.15.44:0/1895298437 >> 172.21.15.44:0/1895298437 conn(0x7f37d00b0e80 msgr2=0x7f37d00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 -- 172.21.15.44:0/1895298437 shutdown_connections 2024-07-17T08:39:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.722+0000 7f37cf200640 1 -- 172.21.15.44:0/1895298437 wait complete. 2024-07-17T08:39:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.724+0000 7f37cf200640 1 Processor -- start 2024-07-17T08:39:43.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.724+0000 7f37cf200640 1 -- start start 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 0x7f37d01e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cf540 0x7f37d01e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00ae940 con 0x7f37d00cf920 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00ae7c0 con 0x7f37d00cf540 2024-07-17T08:39:43.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37cf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37d0141040 con 0x7f37d00b3db0 2024-07-17T08:39:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.725+0000 7f37ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59134/0 (socket says 172.21.15.44:59134) 2024-07-17T08:39:43.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 -- 172.21.15.44:0/3937345430 learned_addr learned my addr 172.21.15.44:0/3937345430 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37cde00640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 0x7f37d01e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37cd400640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cf540 0x7f37d01e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 -- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 msgr2=0x7f37d01e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 0x7f37d01e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 -- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cf540 msgr2=0x7f37d01e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cf540 0x7f37d01e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 -- 172.21.15.44:0/3937345430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d015c620 con 0x7f37d00cf920 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.726+0000 7f37ce800640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c780 tx=0x7f37c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:43.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.727+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8017070 con 0x7f37d00cf920 2024-07-17T08:39:43.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.727+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c800ce30 con 0x7f37d00cf920 2024-07-17T08:39:43.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.727+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c80052d0 con 0x7f37d00cf920 2024-07-17T08:39:43.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.727+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d015c900 con 0x7f37d00cf920 2024-07-17T08:39:43.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.727+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d015cec0 con 0x7f37d00cf920 2024-07-17T08:39:43.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.728+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d01414c0 con 0x7f37d00cf920 2024-07-17T08:39:43.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.729+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f37c801a070 con 0x7f37d00cf920 2024-07-17T08:39:43.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.731+0000 7f37c6000640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 0x7f37b0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.731+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f37c8097e40 con 0x7f37d00cf920 2024-07-17T08:39:43.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.732+0000 7f37cde00640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 0x7f37b0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:43.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.733+0000 7f37cde00640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 0x7f37b0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f37c0033000 tx=0x7f37c0004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:43.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.736+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f37c8060e30 con 0x7f37d00cf920 2024-07-17T08:39:43.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.987+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37d006ec40 con 0x7f37d00cf920 2024-07-17T08:39:43.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.988+0000 7f37c6000640 1 -- 172.21.15.44:0/3937345430 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) ==== 95+0+3918 (secure 0 0 0) 0x7f37c8069070 con 0x7f37d00cf920 2024-07-17T08:39:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:43.990 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":157,"btime":"2024-07-17T08:39:36:033010+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1145386380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1145386380},{"type":"v1","addr":"172.21.15.175:6801","nonce":1145386380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":9484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/4191794586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":4191794586},{"type":"v1","addr":"172.21.15.175:6836","nonce":4191794586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":150},{"gid":11760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/608013107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":608013107},{"type":"v1","addr":"172.21.15.44:6834","nonce":608013107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":149},{"gid":12123,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/697481020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":697481020},{"type":"v1","addr":"172.21.15.44:6835","nonce":697481020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":157}],"filesystems":[]} 2024-07-17T08:39:43.990 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 157 2024-07-17T08:39:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 msgr2=0x7f37b0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 0x7f37b0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f37c0033000 tx=0x7f37c0004730 comp rx=0 tx=0).stop 2024-07-17T08:39:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 msgr2=0x7f37d0154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c780 tx=0x7f37c800cc40 comp rx=0 tx=0).stop 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 shutdown_connections 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37b0076bb0 0x7f37b0079070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37d00cf920 0x7f37d0154e50 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cf540 0x7f37d01e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 --2- 172.21.15.44:0/3937345430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00b3db0 0x7f37d01e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 >> 172.21.15.44:0/3937345430 conn(0x7f37d00b0e80 msgr2=0x7f37d00b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 shutdown_connections 2024-07-17T08:39:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:43.991+0000 7f37cf200640 1 -- 172.21.15.44:0/3937345430 wait complete. 2024-07-17T08:39:44.013 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:39:44.013 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:39:44.013 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:39:44.013 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:39:44.014 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:39:44.014 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:39:44.014 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:39:44.014 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:39:44.014 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:39:44.014 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:39:44.015 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:39:44.015 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:39:44.015 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:39:44.015 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:39:44.015 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:39:44.015 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:39:44.032 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:39:44.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:39:44.035 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:39:44.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:39:44.036 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:39:44.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:39:44.039 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:39:44.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:39:44.082 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:39:44.083 DEBUG:teuthology.parallel:result is None 2024-07-17T08:39:44.086 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:39:44.087 DEBUG:teuthology.parallel:result is None 2024-07-17T08:39:44.088 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:39:44.088 DEBUG:teuthology.parallel:result is None 2024-07-17T08:39:44.092 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:39:44.093 DEBUG:teuthology.parallel:result is None 2024-07-17T08:39:44.093 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:39:44.236 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:39:44.236 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:39:44.239 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:39:44.239 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:39:44.239 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T08:39:44.237+0000 7fe48a400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 -- 172.21.15.44:0/1442698033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f657806ed30 msgr2=0x7f657815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 --2- 172.21.15.44:0/1442698033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f657806ed30 0x7f657815c2f0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f6560005fd0 tx=0x7f656002a440 comp rx=0 tx=0).stop 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 -- 172.21.15.44:0/1442698033 shutdown_connections 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 --2- 172.21.15.44:0/1442698033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f657806ed30 0x7f657815c2f0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 --2- 172.21.15.44:0/1442698033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6578152220 0x7f657806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 -- 172.21.15.44:0/1442698033 >> 172.21.15.44:0/1442698033 conn(0x7f65780b0eb0 msgr2=0x7f65780b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 -- 172.21.15.44:0/1442698033 shutdown_connections 2024-07-17T08:39:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.275+0000 7f657e000640 1 -- 172.21.15.44:0/1442698033 wait complete. 2024-07-17T08:39:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.277+0000 7f657e000640 1 Processor -- start 2024-07-17T08:39:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.277+0000 7f657e000640 1 -- start start 2024-07-17T08:39:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657806ed30 0x7f65781f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:44.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 0x7f65780ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65780cbbf0 con 0x7f6578152220 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65780cba70 con 0x7f65780cb3a0 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f657e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f657815f570 con 0x7f657806ed30 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f6577e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:44.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f6577e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59184/0 (socket says 172.21.15.44:59184) 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f6577e00640 1 -- 172.21.15.44:0/3792173569 learned_addr learned my addr 172.21.15.44:0/3792173569 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.278+0000 7f656ea00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 0x7f65780ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577400640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657806ed30 0x7f65781f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 -- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657806ed30 msgr2=0x7f65781f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657806ed30 0x7f65781f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 -- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 msgr2=0x7f65780ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:44.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 0x7f65780ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 -- 172.21.15.44:0/3792173569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65781f8a90 con 0x7f6578152220 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f656ea00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 0x7f65780ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f6577e00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f657000caa0 tx=0x7f657000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570017070 con 0x7f6578152220 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6570005070 con 0x7f6578152220 2024-07-17T08:39:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700054c0 con 0x7f6578152220 2024-07-17T08:39:44.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.279+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781f4c10 con 0x7f6578152220 2024-07-17T08:39:44.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.280+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781f51d0 con 0x7f6578152220 2024-07-17T08:39:44.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.281+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657815fac0 con 0x7f6578152220 2024-07-17T08:39:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.282+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f6570005660 con 0x7f6578152220 2024-07-17T08:39:44.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.283+0000 7f656fe00640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 0x7f65540790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.284+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(285..285 src has 1..285) ==== 6799+0+0 (secure 0 0 0) 0x7f6570097e30 con 0x7f6578152220 2024-07-17T08:39:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.284+0000 7f6577400640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 0x7f65540790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.285+0000 7f6577400640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 0x7f65540790a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6568009a00 tx=0x7f6568005f90 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:44.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.288+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f65781f8a90 con 0x7f6578152220 2024-07-17T08:39:44.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:44.507+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65780ac910 con 0x7f6578152220 2024-07-17T08:39:45.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.122+0000 7f656fe00640 1 -- 172.21.15.44:0/3792173569 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v287) ==== 97+0+0 (secure 0 0 0) 0x7f6570060e30 con 0x7f6578152220 2024-07-17T08:39:45.124 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 msgr2=0x7f65540790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 0x7f65540790a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6568009a00 tx=0x7f6568005f90 comp rx=0 tx=0).stop 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 msgr2=0x7f6578152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f657000caa0 tx=0x7f657000cf60 comp rx=0 tx=0).stop 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 shutdown_connections 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6554076be0 0x7f65540790a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6578152220 0x7f6578152900 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65780cb3a0 0x7f65780ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 --2- 172.21.15.44:0/3792173569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f657806ed30 0x7f65781f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 >> 172.21.15.44:0/3792173569 conn(0x7f65780b0eb0 msgr2=0x7f65780b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 shutdown_connections 2024-07-17T08:39:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.126+0000 7f657e000640 1 -- 172.21.15.44:0/3792173569 wait complete. 2024-07-17T08:39:45.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:39:45.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 -- 172.21.15.44:0/1755772538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94159890 msgr2=0x7f1c940abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 --2- 172.21.15.44:0/1755772538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94159890 0x7f1c940abf50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005fd0 tx=0x7f1c8c02a820 comp rx=0 tx=0).stop 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 -- 172.21.15.44:0/1755772538 shutdown_connections 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 --2- 172.21.15.44:0/1755772538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c94159890 0x7f1c940abf50 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 -- 172.21.15.44:0/1755772538 >> 172.21.15.44:0/1755772538 conn(0x7f1c940b0e80 msgr2=0x7f1c940b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 -- 172.21.15.44:0/1755772538 shutdown_connections 2024-07-17T08:39:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.361+0000 7f1c99000640 1 -- 172.21.15.44:0/1755772538 wait complete. 2024-07-17T08:39:45.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.363+0000 7f1c99000640 1 Processor -- start 2024-07-17T08:39:45.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.363+0000 7f1c99000640 1 -- start start 2024-07-17T08:39:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c9409b7c0 0x7f1c941f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 0x7f1c94155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c940cf9f0 con 0x7f1c941594b0 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c940cf870 con 0x7f1c94159890 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c99000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9406e7c0 con 0x7f1c9409b7c0 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.364+0000 7f1c92a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59198/0 (socket says 172.21.15.44:59198) 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 -- 172.21.15.44:0/1595686196 learned_addr learned my addr 172.21.15.44:0/1595686196 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c93e00640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 0x7f1c94155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c93400640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c9409b7c0 0x7f1c941f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 -- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c9409b7c0 msgr2=0x7f1c941f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c9409b7c0 0x7f1c941f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 -- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 msgr2=0x7f1c94155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 0x7f1c94155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 -- 172.21.15.44:0/1595686196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9415ad60 con 0x7f1c941594b0 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c93e00640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 0x7f1c94155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c92a00640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c430 tx=0x7f1c8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.365+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80011790 con 0x7f1c941594b0 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.366+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c80017070 con 0x7f1c941594b0 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.366+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80011d00 con 0x7f1c941594b0 2024-07-17T08:39:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.366+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c941f7290 con 0x7f1c941594b0 2024-07-17T08:39:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.366+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c941f7840 con 0x7f1c941594b0 2024-07-17T08:39:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.367+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c940abf50 con 0x7f1c941594b0 2024-07-17T08:39:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.368+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f1c8001c460 con 0x7f1c941594b0 2024-07-17T08:39:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.370+0000 7f1c8b400640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 0x7f1c74079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.370+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(289..289 src has 1..289) ==== 5853+0+0 (secure 0 0 0) 0x7f1c800982f0 con 0x7f1c941594b0 2024-07-17T08:39:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.370+0000 7f1c93400640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 0x7f1c74079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.372+0000 7f1c93400640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 0x7f1c74079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c941f6cb0 tx=0x7f1c84009040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:45.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.375+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1c80061720 con 0x7f1c941594b0 2024-07-17T08:39:45.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.658+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1c94146fd0 con 0x7f1c941594b0 2024-07-17T08:39:45.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.659+0000 7f1c8b400640 1 -- 172.21.15.44:0/1595686196 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v76) ==== 80+0+5304 (secure 0 0 0) 0x7f1c80061020 con 0x7f1c941594b0 2024-07-17T08:39:45.660 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:45.660 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:39:45.662 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:39:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.665 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:39:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:39:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:39:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:39:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:39:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:39:45.674 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:39:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 msgr2=0x7f1c74079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 0x7f1c74079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c941f6cb0 tx=0x7f1c84009040 comp rx=0 tx=0).stop 2024-07-17T08:39:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 msgr2=0x7f1c94146a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c430 tx=0x7f1c8000c8f0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 shutdown_connections 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f1c74076bb0 0x7f1c74079070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c94159890 0x7f1c94155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c941594b0 0x7f1c94146a90 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 --2- 172.21.15.44:0/1595686196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c9409b7c0 0x7f1c941f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 >> 172.21.15.44:0/1595686196 conn(0x7f1c940b0e80 msgr2=0x7f1c940b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.661+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 shutdown_connections 2024-07-17T08:39:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.662+0000 7f1c99000640 1 -- 172.21.15.44:0/1595686196 wait complete. 2024-07-17T08:39:45.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 -- 172.21.15.44:0/3755877018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 msgr2=0x7fb9ac0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 --2- 172.21.15.44:0/3755877018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac0b3870 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0cfdb0 tx=0x7fb99c02aa10 comp rx=0 tx=0).stop 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 -- 172.21.15.44:0/3755877018 shutdown_connections 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 --2- 172.21.15.44:0/3755877018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 0x7fb9ac0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 --2- 172.21.15.44:0/3755877018 >> v2:172.21.15.175:3300/0 conn(0x7fb9ac0cf920 0x7fb9ac0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 --2- 172.21.15.44:0/3755877018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac0b3870 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 -- 172.21.15.44:0/3755877018 >> 172.21.15.44:0/3755877018 conn(0x7fb9ac0b0e80 msgr2=0x7fb9ac0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 -- 172.21.15.44:0/3755877018 shutdown_connections 2024-07-17T08:39:45.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.879+0000 7fb9ab200640 1 -- 172.21.15.44:0/3755877018 wait complete. 2024-07-17T08:39:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.881+0000 7fb9ab200640 1 Processor -- start 2024-07-17T08:39:45.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.882+0000 7fb9ab200640 1 -- start start 2024-07-17T08:39:45.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 0x7fb9ac073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 0x7fb9ac154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac0ae6f0 con 0x7fb9ac0cf540 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac0ae570 con 0x7fb9ac0cf920 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9ab200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac0ae870 con 0x7fb9ac0b3db0 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59208/0 (socket says 172.21.15.44:59208) 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.883+0000 7fb9a9400640 1 -- 172.21.15.44:0/2766795016 learned_addr learned my addr 172.21.15.44:0/2766795016 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9e00640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 0x7fb9ac073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 -- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 msgr2=0x7fb9ac073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 0x7fb9ac073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 -- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 msgr2=0x7fb9ac154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9aa800640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 0x7fb9ac154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 0x7fb9ac154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 -- 172.21.15.44:0/2766795016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac071560 con 0x7fb9ac0cf540 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9aa800640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 0x7fb9ac154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a9400640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b4a0 tx=0x7fb99800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998014070 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb998010700 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.884+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998010b50 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.885+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac13d190 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.885+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac156f70 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.886+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ac0b3870 con 0x7fb9ac0cf540 2024-07-17T08:39:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.887+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fb998017070 con 0x7fb9ac0cf540 2024-07-17T08:39:45.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.888+0000 7fb9a2000640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 0x7fb98c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.888+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(289..289 src has 1..289) ==== 5853+0+0 (secure 0 0 0) 0x7fb998097f20 con 0x7fb9ac0cf540 2024-07-17T08:39:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.889+0000 7fb9a9e00640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 0x7fb98c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.890+0000 7fb9a9e00640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 0x7fb98c079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb99c035000 tx=0x7fb99c03a040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:45.892+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb998061350 con 0x7fb9ac0cf540 2024-07-17T08:39:46.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.106+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb9ac071730 con 0x7fb9ac0cf540 2024-07-17T08:39:46.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.107+0000 7fb9a2000640 1 -- 172.21.15.44:0/2766795016 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v289) ==== 84+0+65 (secure 0 0 0) 0x7fb998060c50 con 0x7fb9ac0cf540 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:39:46.109 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:39:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 msgr2=0x7fb98c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 0x7fb98c079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb99c035000 tx=0x7fb99c03a040 comp rx=0 tx=0).stop 2024-07-17T08:39:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 msgr2=0x7fb9ac157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b4a0 tx=0x7fb99800b960 comp rx=0 tx=0).stop 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 shutdown_connections 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fb98c076bb0 0x7fb98c079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9ac0cf920 0x7fb9ac154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9ac0cf540 0x7fb9ac157a20 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 --2- 172.21.15.44:0/2766795016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9ac0b3db0 0x7fb9ac073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.109+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 >> 172.21.15.44:0/2766795016 conn(0x7fb9ac0b0e80 msgr2=0x7fb9ac13e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.110+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 shutdown_connections 2024-07-17T08:39:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.110+0000 7fb9ab200640 1 -- 172.21.15.44:0/2766795016 wait complete. 2024-07-17T08:39:46.133 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:39:46.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:39:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 -- 172.21.15.44:0/2760177735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a8004d40 msgr2=0x7f76a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 --2- 172.21.15.44:0/2760177735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a8004d40 0x7f76a80051c0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f76c40713b0 tx=0x7f76a8005830 comp rx=0 tx=0).stop 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c3e00640 1 -- 172.21.15.44:0/2760177735 reap_dead start 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 -- 172.21.15.44:0/2760177735 shutdown_connections 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 -- 172.21.15.44:0/2760177735 >> 172.21.15.44:0/2760177735 conn(0x7f76c40cf050 msgr2=0x7f76c40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 -- 172.21.15.44:0/2760177735 shutdown_connections 2024-07-17T08:39:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.330+0000 7f76c9200640 1 -- 172.21.15.44:0/2760177735 wait complete. 2024-07-17T08:39:46.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.332+0000 7f76c9200640 1 Processor -- start 2024-07-17T08:39:46.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.332+0000 7f76c9200640 1 -- start start 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76a8004d40 0x7f76c415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c4163950 0x7f76c406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4164000 con 0x7f76c409cae0 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4163e80 con 0x7f76a8004d40 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76c4164180 con 0x7f76c4163950 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59224/0 (socket says 172.21.15.44:59224) 2024-07-17T08:39:46.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.333+0000 7f76c2a00640 1 -- 172.21.15.44:0/1032928451 learned_addr learned my addr 172.21.15.44:0/1032928451 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 -- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c4163950 msgr2=0x7f76c406e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c4163950 0x7f76c406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 -- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76a8004d40 msgr2=0x7f76c415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76a8004d40 0x7f76c415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 -- 172.21.15.44:0/1032928451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c4162bd0 con 0x7f76c409cae0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c2a00640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d810 tx=0x7f76ac00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac016020 con 0x7f76c409cae0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76ac00b660 con 0x7f76c409cae0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00ba90 con 0x7f76c409cae0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c409d080 con 0x7f76c409cae0 2024-07-17T08:39:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.334+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c4163150 con 0x7f76c409cae0 2024-07-17T08:39:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.336+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c40add80 con 0x7f76c409cae0 2024-07-17T08:39:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.337+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f76ac019020 con 0x7f76c409cae0 2024-07-17T08:39:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.338+0000 7f76bb400640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 0x7f76a0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.338+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(289..289 src has 1..289) ==== 5853+0+0 (secure 0 0 0) 0x7f76ac097f40 con 0x7f76c409cae0 2024-07-17T08:39:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.339+0000 7f76c3400640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 0x7f76a0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.340+0000 7f76c3400640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 0x7f76a0079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76b4002230 tx=0x7f76b4001910 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.342+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f76ac020030 con 0x7f76c409cae0 2024-07-17T08:39:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:46.552+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f76c40ac700 con 0x7f76c409cae0 2024-07-17T08:39:47.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.126+0000 7f76bb400640 1 -- 172.21.15.44:0/1032928451 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v290) ==== 138+0+0 (secure 0 0 0) 0x7f76ac061430 con 0x7f76c409cae0 2024-07-17T08:39:47.128 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:39:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 msgr2=0x7f76a0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 0x7f76a0079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76b4002230 tx=0x7f76b4001910 comp rx=0 tx=0).stop 2024-07-17T08:39:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 msgr2=0x7f76c414add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:47.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d810 tx=0x7f76ac00dcd0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 shutdown_connections 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f76a0076bb0 0x7f76a0079070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c4163950 0x7f76c406e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c409cae0 0x7f76c414add0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 --2- 172.21.15.44:0/1032928451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76a8004d40 0x7f76c415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.130+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 >> 172.21.15.44:0/1032928451 conn(0x7f76c40cf050 msgr2=0x7f76c41543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.131+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 shutdown_connections 2024-07-17T08:39:47.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.131+0000 7f76c9200640 1 -- 172.21.15.44:0/1032928451 wait complete. 2024-07-17T08:39:47.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:39:47.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 -- 172.21.15.44:0/2475454929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739c003a80 msgr2=0x7f739c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:47.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 --2- 172.21.15.44:0/2475454929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f739c003a80 0x7f739c003ea0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f73a80016f0 tx=0x7f739c005830 comp rx=0 tx=0).stop 2024-07-17T08:39:47.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bdc00640 1 -- 172.21.15.44:0/2475454929 reap_dead start 2024-07-17T08:39:47.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 -- 172.21.15.44:0/2475454929 shutdown_connections 2024-07-17T08:39:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 -- 172.21.15.44:0/2475454929 >> 172.21.15.44:0/2475454929 conn(0x7f73b80d00f0 msgr2=0x7f73b80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 -- 172.21.15.44:0/2475454929 shutdown_connections 2024-07-17T08:39:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.363+0000 7f73bfc00640 1 -- 172.21.15.44:0/2475454929 wait complete. 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.365+0000 7f73bfc00640 1 Processor -- start 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.365+0000 7f73bfc00640 1 -- start start 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f739c003a80 0x7f73b8160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80ad970 0x7f73b80addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80ca4f0 con 0x7f73b814c8a0 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80ca370 con 0x7f739c003a80 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73bfc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73b80b9030 con 0x7f73b80ad970 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73afe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59242/0 (socket says 172.21.15.44:59242) 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.366+0000 7f73afe00640 1 -- 172.21.15.44:0/1868652102 learned_addr learned my addr 172.21.15.44:0/1868652102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73bdc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80ad970 0x7f73b80addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73bd200640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f739c003a80 0x7f73b8160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 -- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80ad970 msgr2=0x7f73b80addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80ad970 0x7f73b80addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 -- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f739c003a80 msgr2=0x7f73b8160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f739c003a80 0x7f73b8160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 -- 172.21.15.44:0/1868652102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b81f5040 con 0x7f73b814c8a0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73afe00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f73a000d780 tx=0x7f73a000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018070 con 0x7f73b814c8a0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73a00050d0 con 0x7f73b814c8a0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a000b6c0 con 0x7f73b814c8a0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.367+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b81f1a20 con 0x7f73b814c8a0 2024-07-17T08:39:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.368+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81f55c0 con 0x7f73b814c8a0 2024-07-17T08:39:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.369+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b80ca850 con 0x7f73b814c8a0 2024-07-17T08:39:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.370+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f73a001b070 con 0x7f73b814c8a0 2024-07-17T08:39:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.371+0000 7f73b5600640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 0x7f7394079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.371+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(290..290 src has 1..290) ==== 6223+0+0 (secure 0 0 0) 0x7f73a0097ca0 con 0x7f73b814c8a0 2024-07-17T08:39:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.372+0000 7f73bd200640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 0x7f7394079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.373+0000 7f73bd200640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 0x7f7394079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73a8032000 tx=0x7f73a8001190 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.376+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f73a0060880 con 0x7f73b814c8a0 2024-07-17T08:39:47.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:47.593+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f73b80c8570 con 0x7f73b814c8a0 2024-07-17T08:39:48.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.134+0000 7f73b5600640 1 -- 172.21.15.44:0/1868652102 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v291) ==== 170+0+0 (secure 0 0 0) 0x7f73a0069110 con 0x7f73b814c8a0 2024-07-17T08:39:48.136 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:39:48.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.139+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 msgr2=0x7f7394079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:48.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.139+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 0x7f7394079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f73a8032000 tx=0x7f73a8001190 comp rx=0 tx=0).stop 2024-07-17T08:39:48.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.139+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 msgr2=0x7f73b814cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:48.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.139+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f73a000d780 tx=0x7f73a000dc40 comp rx=0 tx=0).stop 2024-07-17T08:39:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 shutdown_connections 2024-07-17T08:39:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7394076bb0 0x7f7394079070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73b80ad970 0x7f73b80addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73b814c8a0 0x7f73b814cd00 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 --2- 172.21.15.44:0/1868652102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f739c003a80 0x7f73b8160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 >> 172.21.15.44:0/1868652102 conn(0x7f73b80d00f0 msgr2=0x7f73b80c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:48.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 shutdown_connections 2024-07-17T08:39:48.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.140+0000 7f73bfc00640 1 -- 172.21.15.44:0/1868652102 wait complete. 2024-07-17T08:39:48.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:39:48.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 -- 172.21.15.44:0/1332151610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381006ed30 msgr2=0x7f381006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:48.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 --2- 172.21.15.44:0/1332151610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381006ed30 0x7f381006d620 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f3808009830 tx=0x7f380802a510 comp rx=0 tx=0).stop 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 -- 172.21.15.44:0/1332151610 shutdown_connections 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 --2- 172.21.15.44:0/1332151610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381006ed30 0x7f381006d620 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 -- 172.21.15.44:0/1332151610 >> 172.21.15.44:0/1332151610 conn(0x7f38100b0eb0 msgr2=0x7f38100b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 -- 172.21.15.44:0/1332151610 shutdown_connections 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.361+0000 7f3815a00640 1 -- 172.21.15.44:0/1332151610 wait complete. 2024-07-17T08:39:48.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.363+0000 7f3815a00640 1 Processor -- start 2024-07-17T08:39:48.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.363+0000 7f3815a00640 1 -- start start 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381006ed30 0x7f3810150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38100cbab0 0x7f38101550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38100cd9f0 con 0x7f3810152220 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f381015c7d0 con 0x7f38100cbab0 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f3815a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38100cdb70 con 0x7f381006ed30 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f380f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f380f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59264/0 (socket says 172.21.15.44:59264) 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.364+0000 7f380f400640 1 -- 172.21.15.44:0/3621160702 learned_addr learned my addr 172.21.15.44:0/3621160702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380e000640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38100cbab0 0x7f38101550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:48.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 -- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381006ed30 msgr2=0x7f3810150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:48.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381006ed30 0x7f3810150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 -- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38100cbab0 msgr2=0x7f38101550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:48.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38100cbab0 0x7f38101550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:48.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 -- 172.21.15.44:0/3621160702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101528e0 con 0x7f3810152220 2024-07-17T08:39:48.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f380f400640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f3808009950 tx=0x7f3808032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.365+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808040070 con 0x7f3810152220 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.366+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808005310 con 0x7f3810152220 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.366+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080047a0 con 0x7f3810152220 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.366+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381015d600 con 0x7f3810152220 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.366+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381015dbc0 con 0x7f3810152220 2024-07-17T08:39:48.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.367+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38100ce040 con 0x7f3810152220 2024-07-17T08:39:48.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.369+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f3808039040 con 0x7f3810152220 2024-07-17T08:39:48.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.369+0000 7f3806a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 0x7f37ec079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:48.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.370+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(291..291 src has 1..291) ==== 6605+0+0 (secure 0 0 0) 0x7f38080b97b0 con 0x7f3810152220 2024-07-17T08:39:48.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.370+0000 7f380ea00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 0x7f37ec079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:48.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.371+0000 7f380ea00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 0x7f37ec079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8007900 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:48.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.374+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3808082970 con 0x7f3810152220 2024-07-17T08:39:48.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:48.628+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3810152ab0 con 0x7f3810152220 2024-07-17T08:39:49.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.179+0000 7f3806a00640 1 -- 172.21.15.44:0/3621160702 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 39 and data pool 40 v160) ==== 174+0+0 (secure 0 0 0) 0x7f3808082270 con 0x7f3810152220 2024-07-17T08:39:49.181 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 39 and data pool 40 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.188+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 msgr2=0x7f37ec079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.188+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 0x7f37ec079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8007900 comp rx=0 tx=0).stop 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.188+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 msgr2=0x7f381015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.188+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f3808009950 tx=0x7f3808032b90 comp rx=0 tx=0).stop 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 shutdown_connections 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f37ec076bb0 0x7f37ec079070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3810152220 0x7f381015ea80 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38100cbab0 0x7f38101550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 --2- 172.21.15.44:0/3621160702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381006ed30 0x7f3810150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.189+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 >> 172.21.15.44:0/3621160702 conn(0x7f38100b0eb0 msgr2=0x7f38100b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:49.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.191+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 shutdown_connections 2024-07-17T08:39:49.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.192+0000 7f3815a00640 1 -- 172.21.15.44:0/3621160702 wait complete. 2024-07-17T08:39:49.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 -- 172.21.15.44:0/4077895648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 msgr2=0x7f9010153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 --2- 172.21.15.44:0/4077895648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f9010153a50 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f90101576d0 tx=0x7f900402a560 comp rx=0 tx=0).stop 2024-07-17T08:39:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 -- 172.21.15.44:0/4077895648 shutdown_connections 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 --2- 172.21.15.44:0/4077895648 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90100ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 --2- 172.21.15.44:0/4077895648 >> v2:172.21.15.175:3300/0 conn(0x7f9010159b00 0x7f9010159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 --2- 172.21.15.44:0/4077895648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f9010153a50 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 -- 172.21.15.44:0/4077895648 >> 172.21.15.44:0/4077895648 conn(0x7f90100b0fb0 msgr2=0x7f90100b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 -- 172.21.15.44:0/4077895648 shutdown_connections 2024-07-17T08:39:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.408+0000 7f9018000640 1 -- 172.21.15.44:0/4077895648 wait complete. 2024-07-17T08:39:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.410+0000 7f9018000640 1 Processor -- start 2024-07-17T08:39:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.410+0000 7f9018000640 1 -- start start 2024-07-17T08:39:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90101520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 0x7f9010152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90100ae980 con 0x7f901006f710 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90100ae800 con 0x7f9010159b00 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9018000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f901013ed60 con 0x7f90100ac3e0 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.411+0000 7f9015600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41610/0 (socket says 172.21.15.44:41610) 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 -- 172.21.15.44:0/2385203618 learned_addr learned my addr 172.21.15.44:0/2385203618 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9014c00640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90101520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9016000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 0x7f9010152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 -- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 msgr2=0x7f90101520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90101520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 -- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 msgr2=0x7f9010152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 0x7f9010152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 -- 172.21.15.44:0/2385203618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101f5470 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9014c00640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90101520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9016000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 0x7f9010152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.412+0000 7f9015600640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f9004009850 tx=0x7f90040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.413+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900403f070 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.413+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9004004890 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.413+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004035030 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.413+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90101f5750 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.413+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101f5d10 con 0x7f901006f710 2024-07-17T08:39:49.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.414+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90100ac7f0 con 0x7f901006f710 2024-07-17T08:39:49.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.415+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f9004042070 con 0x7f901006f710 2024-07-17T08:39:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.416+0000 7f900d600640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 0x7f8ff0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.416+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(292..292 src has 1..292) ==== 6705+0+0 (secure 0 0 0) 0x7f90040b96a0 con 0x7f901006f710 2024-07-17T08:39:49.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.417+0000 7f9014c00640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 0x7f8ff0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.418+0000 7f9014c00640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 0x7f8ff0079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9010157690 tx=0x7f8ffc008040 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.421+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f90040826e0 con 0x7f901006f710 2024-07-17T08:39:49.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.635+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f901015cb80 con 0x7f901006f710 2024-07-17T08:39:49.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.639+0000 7f900d600640 1 -- 172.21.15.44:0/2385203618 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v292) ==== 74+0+17807 (secure 0 0 0) 0x7f9004081fe0 con 0x7f901006f710 2024-07-17T08:39:49.641 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:49.641 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":292,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:48.630594+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:39:46.575840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":40,"pool_name":"cephfs_data","create_time":"2024-07-17T08:39:47.617938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.641+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 msgr2=0x7f8ff0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.641+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 0x7f8ff0079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9010157690 tx=0x7f8ffc008040 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.641+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 msgr2=0x7f90101f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.641+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f9004009850 tx=0x7f90040039e0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 shutdown_connections 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8ff0076bb0 0x7f8ff0079070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9010159b00 0x7f9010152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90100ac3e0 0x7f90101520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 --2- 172.21.15.44:0/2385203618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f901006f710 0x7f90101f1520 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 >> 172.21.15.44:0/2385203618 conn(0x7f90100b0fb0 msgr2=0x7f90100b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 shutdown_connections 2024-07-17T08:39:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.642+0000 7f9018000640 1 -- 172.21.15.44:0/2385203618 wait complete. 2024-07-17T08:39:49.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 -- 172.21.15.44:0/1209918724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc0cfa90 msgr2=0x7f05fc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 --2- 172.21.15.44:0/1209918724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc0cfa90 0x7f05fc06e060 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f05e4007350 tx=0x7f05e402b1a0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 -- 172.21.15.44:0/1209918724 shutdown_connections 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 --2- 172.21.15.44:0/1209918724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 0x7f05fc159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 --2- 172.21.15.44:0/1209918724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc0cfa90 0x7f05fc06e060 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 -- 172.21.15.44:0/1209918724 >> 172.21.15.44:0/1209918724 conn(0x7f05fc0b0e80 msgr2=0x7f05fc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 -- 172.21.15.44:0/1209918724 shutdown_connections 2024-07-17T08:39:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.869+0000 7f0601c00640 1 -- 172.21.15.44:0/1209918724 wait complete. 2024-07-17T08:39:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.871+0000 7f0601c00640 1 Processor -- start 2024-07-17T08:39:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.871+0000 7f0601c00640 1 -- start start 2024-07-17T08:39:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 0x7f05fc0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc0cfa90 0x7f05fc073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b2120 con 0x7f05fc09b300 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc0b1fa0 con 0x7f05fc0cfa90 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0b22a0 con 0x7f05fc09b6e0 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41622/0 (socket says 172.21.15.44:41622) 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fbe00640 1 -- 172.21.15.44:0/1121454970 learned_addr learned my addr 172.21.15.44:0/1121454970 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fb400640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 0x7f05fc0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f0601200640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc0cfa90 0x7f05fc073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fbe00640 1 -- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 msgr2=0x7f05fc0733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.872+0000 7f05fbe00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 0x7f05fc0733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05fbe00640 1 -- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc0cfa90 msgr2=0x7f05fc073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05fbe00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc0cfa90 0x7f05fc073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05fbe00640 1 -- 172.21.15.44:0/1121454970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1473f0 con 0x7f05fc09b300 2024-07-17T08:39:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05fbe00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c450 tx=0x7f05ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec016020 con 0x7f05fc09b300 2024-07-17T08:39:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05ec012920 con 0x7f05fc09b300 2024-07-17T08:39:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec012d70 con 0x7f05fc09b300 2024-07-17T08:39:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1f4f50 con 0x7f05fc09b300 2024-07-17T08:39:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.873+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1f5510 con 0x7f05fc09b300 2024-07-17T08:39:49.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.875+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc159800 con 0x7f05fc09b300 2024-07-17T08:39:49.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.876+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f05ec019020 con 0x7f05fc09b300 2024-07-17T08:39:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.878+0000 7f05f3e00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 0x7f05d8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:49.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.878+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(292..292 src has 1..292) ==== 6705+0+0 (secure 0 0 0) 0x7f05ec098cd0 con 0x7f05fc09b300 2024-07-17T08:39:49.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.879+0000 7f05fb400640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 0x7f05d8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:49.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.880+0000 7f05fb400640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 0x7f05d8079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e4004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:49.881+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f05ec020020 con 0x7f05fc09b300 2024-07-17T08:39:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.092+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05fc0cf3f0 con 0x7f05fc09b300 2024-07-17T08:39:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.093+0000 7f05f3e00640 1 -- 172.21.15.44:0/1121454970 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v292) ==== 74+0+17807 (secure 0 0 0) 0x7f05ec061e20 con 0x7f05fc09b300 2024-07-17T08:39:50.095 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:50.096 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":292,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:48.630594+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:39:46.575840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":40,"pool_name":"cephfs_data","create_time":"2024-07-17T08:39:47.617938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 msgr2=0x7f05d8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 0x7f05d8079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e4004730 comp rx=0 tx=0).stop 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 msgr2=0x7f05fc146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c450 tx=0x7f05ec00c910 comp rx=0 tx=0).stop 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 shutdown_connections 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f05d8076bb0 0x7f05d8079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc0cfa90 0x7f05fc073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05fc09b6e0 0x7f05fc0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 --2- 172.21.15.44:0/1121454970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05fc09b300 0x7f05fc146eb0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 >> 172.21.15.44:0/1121454970 conn(0x7f05fc0b0e80 msgr2=0x7f05fc0abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:50.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 shutdown_connections 2024-07-17T08:39:50.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.096+0000 7f0601c00640 1 -- 172.21.15.44:0/1121454970 wait complete. 2024-07-17T08:39:50.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 -- 172.21.15.44:0/339742352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 msgr2=0x7f167806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 --2- 172.21.15.44:0/339742352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f167806d620 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1668009900 tx=0x7f166802a610 comp rx=0 tx=0).stop 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 -- 172.21.15.44:0/339742352 shutdown_connections 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 --2- 172.21.15.44:0/339742352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f167806d620 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 --2- 172.21.15.44:0/339742352 >> v2:172.21.15.175:3300/0 conn(0x7f1678154600 0x7f16780b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 -- 172.21.15.44:0/339742352 >> 172.21.15.44:0/339742352 conn(0x7f16780b0eb0 msgr2=0x7f16780b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:50.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 -- 172.21.15.44:0/339742352 shutdown_connections 2024-07-17T08:39:50.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.321+0000 7f1680800640 1 -- 172.21.15.44:0/339742352 wait complete. 2024-07-17T08:39:50.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.323+0000 7f1680800640 1 Processor -- start 2024-07-17T08:39:50.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.323+0000 7f1680800640 1 -- start start 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f1678159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678159bd0 0x7f1678071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f167815e220 con 0x7f1678154600 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f167815e0a0 con 0x7f1678159bd0 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f1680800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f167815e3a0 con 0x7f1678161120 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41634/0 (socket says 172.21.15.44:41634) 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 -- 172.21.15.44:0/2824894918 learned_addr learned my addr 172.21.15.44:0/2824894918 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:50.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167d400640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f1678159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:50.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 -- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 msgr2=0x7f1678159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f1678159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 -- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678159bd0 msgr2=0x7f1678071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678159bd0 0x7f1678071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.324+0000 7f167de00640 1 -- 172.21.15.44:0/2824894918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16781f4820 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f167de00640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f166c00ea20 tx=0x7f166c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c00cd80 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f166c010430 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c010840 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1678153850 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.325+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16781f4da0 con 0x7f1678154600 2024-07-17T08:39:50.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.326+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167815e8c0 con 0x7f1678154600 2024-07-17T08:39:50.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.327+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f166c0109e0 con 0x7f1678154600 2024-07-17T08:39:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.328+0000 7f1672000640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 0x7f165c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.328+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f166c099320 con 0x7f1678154600 2024-07-17T08:39:50.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.329+0000 7f167d400640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 0x7f165c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:50.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.330+0000 7f167d400640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 0x7f165c079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1668033000 tx=0x7f1668004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:50.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.332+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f166c062360 con 0x7f1678154600 2024-07-17T08:39:50.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:50.585+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f16780b0c60 con 0x7f1678154600 2024-07-17T08:39:51.130 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:39:51.128+0000 7fea88200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:39:51.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.211+0000 7f1672000640 1 -- 172.21.15.44:0/2824894918 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v163) ==== 118+0+0 (secure 0 0 0) 0x7f166c061c60 con 0x7f1678154600 2024-07-17T08:39:51.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 msgr2=0x7f165c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 0x7f165c079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1668033000 tx=0x7f1668004730 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 msgr2=0x7f167814a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f166c00ea20 tx=0x7f166c00eee0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 shutdown_connections 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f165c076bb0 0x7f165c079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1678159bd0 0x7f1678071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1678161120 0x7f1678159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 --2- 172.21.15.44:0/2824894918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1678154600 0x7f167814a540 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 >> 172.21.15.44:0/2824894918 conn(0x7f16780b0eb0 msgr2=0x7f167806df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 shutdown_connections 2024-07-17T08:39:51.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.215+0000 7f1680800640 1 -- 172.21.15.44:0/2824894918 wait complete. 2024-07-17T08:39:51.239 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:39:51.239 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:39:51.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.455+0000 7fa2ab000640 1 -- 172.21.15.44:0/551033237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 msgr2=0x7fa2a40ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.455+0000 7fa2ab000640 1 --2- 172.21.15.44:0/551033237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40ae560 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fa290005fd0 tx=0x7fa29002a520 comp rx=0 tx=0).stop 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 -- 172.21.15.44:0/551033237 shutdown_connections 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 --2- 172.21.15.44:0/551033237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40ae560 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 --2- 172.21.15.44:0/551033237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a406eb90 0x7fa2a415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 -- 172.21.15.44:0/551033237 >> 172.21.15.44:0/551033237 conn(0x7fa2a40b0eb0 msgr2=0x7fa2a40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 -- 172.21.15.44:0/551033237 shutdown_connections 2024-07-17T08:39:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.456+0000 7fa2ab000640 1 -- 172.21.15.44:0/551033237 wait complete. 2024-07-17T08:39:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.457+0000 7fa2ab000640 1 Processor -- start 2024-07-17T08:39:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.457+0000 7fa2ab000640 1 -- start start 2024-07-17T08:39:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.458+0000 7fa2ab000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2a406eb90 0x7fa2a41f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.458+0000 7fa2ab000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a40ac3a0 0x7fa2a414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.458+0000 7fa2ab000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.458+0000 7fa2ab000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a406f850 con 0x7fa2a40b3280 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.458+0000 7fa2ab000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a406f6d0 con 0x7fa2a406eb90 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2ab000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a4150470 con 0x7fa2a40ac3a0 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41646/0 (socket says 172.21.15.44:41646) 2024-07-17T08:39:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 -- 172.21.15.44:0/2254697139 learned_addr learned my addr 172.21.15.44:0/2254697139 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 -- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a40ac3a0 msgr2=0x7fa2a414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a3400640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a40ac3a0 0x7fa2a414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a3e00640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2a406eb90 0x7fa2a41f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a40ac3a0 0x7fa2a414caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 -- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2a406eb90 msgr2=0x7fa2a41f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2a406eb90 0x7fa2a41f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.459+0000 7fa2a9000640 1 -- 172.21.15.44:0/2254697139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a414d030 con 0x7fa2a40b3280 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa2a9000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c780 tx=0x7fa29c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004b60 con 0x7fa2a40b3280 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa29c004df0 con 0x7fa2a40b3280 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00aa70 con 0x7fa2a40b3280 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a414d3e0 con 0x7fa2a40b3280 2024-07-17T08:39:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.460+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a414d6c0 con 0x7fa2a40b3280 2024-07-17T08:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.461+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a41f7890 con 0x7fa2a40b3280 2024-07-17T08:39:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.462+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fa29c00ac10 con 0x7fa2a40b3280 2024-07-17T08:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.463+0000 7fa29be00640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 0x7fa284079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.463+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fa29c098aa0 con 0x7fa2a40b3280 2024-07-17T08:39:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.464+0000 7fa2a3e00640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 0x7fa284079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.465+0000 7fa2a3e00640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 0x7fa284079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2940058c0 tx=0x7fa294005c60 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.468+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa29c020070 con 0x7fa2a40b3280 2024-07-17T08:39:51.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:51.715+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa2a40b4200 con 0x7fa2a40b3280 2024-07-17T08:39:52.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.215+0000 7fa29be00640 1 -- 172.21.15.44:0/2254697139 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v164) ==== 122+0+0 (secure 0 0 0) 0x7fa29c061b70 con 0x7fa2a40b3280 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.218+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 msgr2=0x7fa284079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 0x7fa284079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2940058c0 tx=0x7fa294005c60 comp rx=0 tx=0).stop 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 msgr2=0x7fa2a40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c780 tx=0x7fa29c00cc40 comp rx=0 tx=0).stop 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 shutdown_connections 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fa284076bb0 0x7fa284079070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a40b3280 0x7fa2a40affc0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2a40ac3a0 0x7fa2a414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 --2- 172.21.15.44:0/2254697139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2a406eb90 0x7fa2a41f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 >> 172.21.15.44:0/2254697139 conn(0x7fa2a40b0eb0 msgr2=0x7fa2a40cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 shutdown_connections 2024-07-17T08:39:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.219+0000 7fa2ab000640 1 -- 172.21.15.44:0/2254697139 wait complete. 2024-07-17T08:39:52.242 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:39:52.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:52.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 -- 172.21.15.44:0/1200056013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 msgr2=0x7f4c540b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/1200056013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540b30f0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f4c540cfd20 tx=0x7f4c4c02aa10 comp rx=0 tx=0).stop 2024-07-17T08:39:52.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 -- 172.21.15.44:0/1200056013 shutdown_connections 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/1200056013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540b30f0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/1200056013 >> v2:172.21.15.175:3300/0 conn(0x7f4c54159890 0x7f4c5409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/1200056013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c541594b0 0x7f4c540abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 -- 172.21.15.44:0/1200056013 >> 172.21.15.44:0/1200056013 conn(0x7f4c540b0e80 msgr2=0x7f4c540b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 -- 172.21.15.44:0/1200056013 shutdown_connections 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.441+0000 7f4c5ac00640 1 -- 172.21.15.44:0/1200056013 wait complete. 2024-07-17T08:39:52.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.443+0000 7f4c5ac00640 1 Processor -- start 2024-07-17T08:39:52.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.443+0000 7f4c5ac00640 1 -- start start 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c541594b0 0x7f4c5414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c54159890 0x7f4c5415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540cf6e0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540cf560 con 0x7f4c54159890 2024-07-17T08:39:52.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c5ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c540cf860 con 0x7f4c541594b0 2024-07-17T08:39:52.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c59800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.444+0000 7f4c59800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41654/0 (socket says 172.21.15.44:41654) 2024-07-17T08:39:52.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 -- 172.21.15.44:0/3911255025 learned_addr learned my addr 172.21.15.44:0/3911255025 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 -- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c541594b0 msgr2=0x7f4c5414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c541594b0 0x7f4c5414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 -- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c54159890 msgr2=0x7f4c5415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c54159890 0x7f4c5415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 -- 172.21.15.44:0/3911255025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c54159ee0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.445+0000 7f4c59800640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f4c48005940 tx=0x7f4c4800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.446+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48007bb0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.446+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c48007e40 con 0x7f4c5409b5f0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.446+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800f690 con 0x7f4c5409b5f0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.446+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5415a1c0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.446+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5415a780 con 0x7f4c5409b5f0 2024-07-17T08:39:52.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.447+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c540abf50 con 0x7f4c5409b5f0 2024-07-17T08:39:52.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.448+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f4c4800f830 con 0x7f4c5409b5f0 2024-07-17T08:39:52.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.450+0000 7f4c51600640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 0x7f4c34079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.450+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f4c48098ee0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.450+0000 7f4c58e00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 0x7f4c34079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.452+0000 7f4c58e00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 0x7f4c34079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c540addb0 tx=0x7f4c40005bd0 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:52.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.454+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4c48014030 con 0x7f4c5409b5f0 2024-07-17T08:39:52.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.705+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c54074ee0 con 0x7f4c5409b5f0 2024-07-17T08:39:52.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.706+0000 7f4c51600640 1 -- 172.21.15.44:0/3911255025 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f4c48061f20 con 0x7f4c5409b5f0 2024-07-17T08:39:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:52.709 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:39:52.709 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.709+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 msgr2=0x7f4c34079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.709+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 0x7f4c34079070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c540addb0 tx=0x7f4c40005bd0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.709+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 msgr2=0x7f4c540ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.709+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f4c48005940 tx=0x7f4c4800cd60 comp rx=0 tx=0).stop 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 shutdown_connections 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f4c34076bb0 0x7f4c34079070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c54159890 0x7f4c5415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c541594b0 0x7f4c5414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 --2- 172.21.15.44:0/3911255025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c5409b5f0 0x7f4c540ce1c0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 >> 172.21.15.44:0/3911255025 conn(0x7f4c540b0e80 msgr2=0x7f4c540b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:52.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 shutdown_connections 2024-07-17T08:39:52.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.710+0000 7f4c5ac00640 1 -- 172.21.15.44:0/3911255025 wait complete. 2024-07-17T08:39:52.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:39:52.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 -- 172.21.15.44:0/937549521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 msgr2=0x7f5848004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 --2- 172.21.15.44:0/937549521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f5848004c80 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f584c001dc0 tx=0x7f5848005830 comp rx=0 tx=0).stop 2024-07-17T08:39:52.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5863200640 1 -- 172.21.15.44:0/937549521 reap_dead start 2024-07-17T08:39:52.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 -- 172.21.15.44:0/937549521 shutdown_connections 2024-07-17T08:39:52.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 -- 172.21.15.44:0/937549521 >> 172.21.15.44:0/937549521 conn(0x7f58640b0fb0 msgr2=0x7f58640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:52.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 -- 172.21.15.44:0/937549521 shutdown_connections 2024-07-17T08:39:52.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.934+0000 7f5869400640 1 -- 172.21.15.44:0/937549521 wait complete. 2024-07-17T08:39:52.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.936+0000 7f5869400640 1 Processor -- start 2024-07-17T08:39:52.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.936+0000 7f5869400640 1 -- start start 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58641f5400 0x7f58641f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 0x7f5864074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f586406f040 con 0x7f5848004800 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f586406eec0 con 0x7f58641f5400 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5869400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58640b25d0 con 0x7f5864073d10 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5862800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5862800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41670/0 (socket says 172.21.15.44:41670) 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.937+0000 7f5862800640 1 -- 172.21.15.44:0/715006512 learned_addr learned my addr 172.21.15.44:0/715006512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5861e00640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58641f5400 0x7f58641f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5863200640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 0x7f5864074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 -- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 msgr2=0x7f5864074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 0x7f5864074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 -- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58641f5400 msgr2=0x7f58641f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58641f5400 0x7f58641f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 -- 172.21.15.44:0/715006512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58641f8780 con 0x7f5848004800 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5863200640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 0x7f5864074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:52.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f5862800640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f585400e5c0 tx=0x7f585400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.938+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854015070 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.939+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f585400ca60 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.939+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854011460 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.939+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58641f4630 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.939+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58641f4bf0 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.940+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58640746b0 con 0x7f5848004800 2024-07-17T08:39:52.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.941+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f5854018070 con 0x7f5848004800 2024-07-17T08:39:52.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.942+0000 7f585aa00640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 0x7f58400790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:52.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.942+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f58540989b0 con 0x7f5848004800 2024-07-17T08:39:52.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.943+0000 7f5861e00640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 0x7f58400790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:52.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.944+0000 7f5861e00640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 0x7f58400790a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f584c032000 tx=0x7f584c001510 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:52.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:52.946+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f58640746b0 con 0x7f5848004800 2024-07-17T08:39:53.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.161+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58640b2b60 con 0x7f5848004800 2024-07-17T08:39:53.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.163+0000 7f585aa00640 1 -- 172.21.15.44:0/715006512 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v293) ==== 74+0+17794 (secure 0 0 0) 0x7f58641f8780 con 0x7f5848004800 2024-07-17T08:39:53.165 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:53.165 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":293,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:39:50.179279+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:39:46.575840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":40,"pool_name":"cephfs_data","create_time":"2024-07-17T08:39:47.617938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"293","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 msgr2=0x7f58400790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 0x7f58400790a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f584c032000 tx=0x7f584c001510 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 msgr2=0x7f58641f8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f585400e5c0 tx=0x7f585400ea80 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 shutdown_connections 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f5840076be0 0x7f58400790a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5864073d10 0x7f5864074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58641f5400 0x7f58641f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 --2- 172.21.15.44:0/715006512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5848004800 0x7f58641f8240 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 >> 172.21.15.44:0/715006512 conn(0x7f58640b0fb0 msgr2=0x7f58640aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 shutdown_connections 2024-07-17T08:39:53.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.166+0000 7f5869400640 1 -- 172.21.15.44:0/715006512 wait complete. 2024-07-17T08:39:53.187 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:39:53.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:39:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 -- 172.21.15.44:0/457638131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0cd690 msgr2=0x7fc98c0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 --2- 172.21.15.44:0/457638131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0cd690 0x7fc98c0cdb20 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fc984005fd0 tx=0x7fc98402a8a0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 -- 172.21.15.44:0/457638131 shutdown_connections 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 --2- 172.21.15.44:0/457638131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0cd690 0x7fc98c0cdb20 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 --2- 172.21.15.44:0/457638131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc98c0ac220 0x7fc98c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 -- 172.21.15.44:0/457638131 >> 172.21.15.44:0/457638131 conn(0x7fc98c0b0fb0 msgr2=0x7fc98c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 -- 172.21.15.44:0/457638131 shutdown_connections 2024-07-17T08:39:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.401+0000 7fc992000640 1 -- 172.21.15.44:0/457638131 wait complete. 2024-07-17T08:39:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.403+0000 7fc992000640 1 Processor -- start 2024-07-17T08:39:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.403+0000 7fc992000640 1 -- start start 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc98c0cd690 0x7fc98c152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98c1522c0 0x7fc98c1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c0ce1e0 con 0x7fc98c0ac220 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c0ce060 con 0x7fc98c1522c0 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.404+0000 7fc992000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc98c0ce360 con 0x7fc98c0cd690 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41696/0 (socket says 172.21.15.44:41696) 2024-07-17T08:39:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 -- 172.21.15.44:0/4105510240 learned_addr learned my addr 172.21.15.44:0/4105510240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 -- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc98c0cd690 msgr2=0x7fc98c152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc98c0cd690 0x7fc98c152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98be00640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98c1522c0 0x7fc98c1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 -- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98c1522c0 msgr2=0x7fc98c1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98c1522c0 0x7fc98c1f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.405+0000 7fc98b400640 1 -- 172.21.15.44:0/4105510240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c14cbc0 con 0x7fc98c0ac220 2024-07-17T08:39:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc98b400640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00c3a0 tx=0x7fc97c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c011790 con 0x7fc98c0ac220 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc97c017070 con 0x7fc98c0ac220 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c011cc0 con 0x7fc98c0ac220 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98c150a60 con 0x7fc98c0ac220 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.406+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98c0b2690 con 0x7fc98c0ac220 2024-07-17T08:39:53.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.407+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98c0ce880 con 0x7fc98c0ac220 2024-07-17T08:39:53.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.408+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fc97c01c460 con 0x7fc98c0ac220 2024-07-17T08:39:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.409+0000 7fc983400640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 0x7fc9680790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.409+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fc97c0984d0 con 0x7fc98c0ac220 2024-07-17T08:39:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.410+0000 7fc983e00640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 0x7fc9680790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.411+0000 7fc983e00640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 0x7fc9680790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc974005fd0 tx=0x7fc974005850 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.414+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc98c14cbc0 con 0x7fc98c0ac220 2024-07-17T08:39:53.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.622+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 --> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc98c152e50 con 0x7fc968076be0 2024-07-17T08:39:53.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.638+0000 7fc983400640 1 -- 172.21.15.44:0/4105510240 <== mgr.8795 v2:172.21.15.175:6834/1518538177 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7fc98c152e50 con 0x7fc968076be0 2024-07-17T08:39:53.644 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:53.646 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:39:53.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 msgr2=0x7fc9680790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 0x7fc9680790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc974005fd0 tx=0x7fc974005850 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 msgr2=0x7fc98c0740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00c3a0 tx=0x7fc97c00c860 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 shutdown_connections 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc968076be0 0x7fc9680790a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc98c1522c0 0x7fc98c1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc98c0cd690 0x7fc98c152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 --2- 172.21.15.44:0/4105510240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc98c0ac220 0x7fc98c0740d0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.644+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 >> 172.21.15.44:0/4105510240 conn(0x7fc98c0b0fb0 msgr2=0x7fc98c0ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.645+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 shutdown_connections 2024-07-17T08:39:53.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.645+0000 7fc992000640 1 -- 172.21.15.44:0/4105510240 wait complete. 2024-07-17T08:39:53.667 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:39:52.961386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":490,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4635,"ondisk_log_size":4635,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":468524,"kb_used_data":30256,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754506196,"statfs":{"total":773094113280,"available":772614344704,"internally_reserved":0,"allocated":30982144,"data_stored":17532876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.831037"},"pg_stats":[{"pgid":"40.3f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751161+0000","last_change":"2024-07-17T08:39:49.210184+0000","last_active":"2024-07-17T08:39:50.751161+0000","last_peered":"2024-07-17T08:39:50.751161+0000","last_clean":"2024-07-17T08:39:50.751161+0000","last_became_active":"2024-07-17T08:39:49.209173+0000","last_became_peered":"2024-07-17T08:39:49.209173+0000","last_unstale":"2024-07-17T08:39:50.751161+0000","last_undegraded":"2024-07-17T08:39:50.751161+0000","last_fullsized":"2024-07-17T08:39:50.751161+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:57.075030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.3e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749350+0000","last_change":"2024-07-17T08:39:49.206209+0000","last_active":"2024-07-17T08:39:50.749350+0000","last_peered":"2024-07-17T08:39:50.749350+0000","last_clean":"2024-07-17T08:39:50.749350+0000","last_became_active":"2024-07-17T08:39:49.205939+0000","last_became_peered":"2024-07-17T08:39:49.205939+0000","last_unstale":"2024-07-17T08:39:50.749350+0000","last_undegraded":"2024-07-17T08:39:50.749350+0000","last_fullsized":"2024-07-17T08:39:50.749350+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:17.701023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.3d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192415+0000","last_change":"2024-07-17T08:39:49.201095+0000","last_active":"2024-07-17T08:39:50.192415+0000","last_peered":"2024-07-17T08:39:50.192415+0000","last_clean":"2024-07-17T08:39:50.192415+0000","last_became_active":"2024-07-17T08:39:49.200755+0000","last_became_peered":"2024-07-17T08:39:49.200755+0000","last_unstale":"2024-07-17T08:39:50.192415+0000","last_undegraded":"2024-07-17T08:39:50.192415+0000","last_fullsized":"2024-07-17T08:39:50.192415+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:40.850194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.3c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186922+0000","last_change":"2024-07-17T08:39:49.196806+0000","last_active":"2024-07-17T08:39:50.186922+0000","last_peered":"2024-07-17T08:39:50.186922+0000","last_clean":"2024-07-17T08:39:50.186922+0000","last_became_active":"2024-07-17T08:39:49.196472+0000","last_became_peered":"2024-07-17T08:39:49.196472+0000","last_unstale":"2024-07-17T08:39:50.186922+0000","last_undegraded":"2024-07-17T08:39:50.186922+0000","last_fullsized":"2024-07-17T08:39:50.186922+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:42:52.560875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.3b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747536+0000","last_change":"2024-07-17T08:39:49.196707+0000","last_active":"2024-07-17T08:39:50.747536+0000","last_peered":"2024-07-17T08:39:50.747536+0000","last_clean":"2024-07-17T08:39:50.747536+0000","last_became_active":"2024-07-17T08:39:49.196386+0000","last_became_peered":"2024-07-17T08:39:49.196386+0000","last_unstale":"2024-07-17T08:39:50.747536+0000","last_undegraded":"2024-07-17T08:39:50.747536+0000","last_fullsized":"2024-07-17T08:39:50.747536+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:29:03.443911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205875+0000","last_change":"2024-07-17T08:39:49.189067+0000","last_active":"2024-07-17T08:39:50.205875+0000","last_peered":"2024-07-17T08:39:50.205875+0000","last_clean":"2024-07-17T08:39:50.205875+0000","last_became_active":"2024-07-17T08:39:49.187660+0000","last_became_peered":"2024-07-17T08:39:49.187660+0000","last_unstale":"2024-07-17T08:39:50.205875+0000","last_undegraded":"2024-07-17T08:39:50.205875+0000","last_fullsized":"2024-07-17T08:39:50.205875+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:17.564331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.39","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205091+0000","last_change":"2024-07-17T08:39:49.202104+0000","last_active":"2024-07-17T08:39:50.205091+0000","last_peered":"2024-07-17T08:39:50.205091+0000","last_clean":"2024-07-17T08:39:50.205091+0000","last_became_active":"2024-07-17T08:39:49.201658+0000","last_became_peered":"2024-07-17T08:39:49.201658+0000","last_unstale":"2024-07-17T08:39:50.205091+0000","last_undegraded":"2024-07-17T08:39:50.205091+0000","last_fullsized":"2024-07-17T08:39:50.205091+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:46.199352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.38","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745462+0000","last_change":"2024-07-17T08:39:49.197656+0000","last_active":"2024-07-17T08:39:50.745462+0000","last_peered":"2024-07-17T08:39:50.745462+0000","last_clean":"2024-07-17T08:39:50.745462+0000","last_became_active":"2024-07-17T08:39:49.197162+0000","last_became_peered":"2024-07-17T08:39:49.197162+0000","last_unstale":"2024-07-17T08:39:50.745462+0000","last_undegraded":"2024-07-17T08:39:50.745462+0000","last_fullsized":"2024-07-17T08:39:50.745462+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:01.542812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.37","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747624+0000","last_change":"2024-07-17T08:39:49.204101+0000","last_active":"2024-07-17T08:39:50.747624+0000","last_peered":"2024-07-17T08:39:50.747624+0000","last_clean":"2024-07-17T08:39:50.747624+0000","last_became_active":"2024-07-17T08:39:49.203758+0000","last_became_peered":"2024-07-17T08:39:49.203758+0000","last_unstale":"2024-07-17T08:39:50.747624+0000","last_undegraded":"2024-07-17T08:39:50.747624+0000","last_fullsized":"2024-07-17T08:39:50.747624+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:44:30.948595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.36","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192075+0000","last_change":"2024-07-17T08:39:49.197205+0000","last_active":"2024-07-17T08:39:50.192075+0000","last_peered":"2024-07-17T08:39:50.192075+0000","last_clean":"2024-07-17T08:39:50.192075+0000","last_became_active":"2024-07-17T08:39:49.196869+0000","last_became_peered":"2024-07-17T08:39:49.196869+0000","last_unstale":"2024-07-17T08:39:50.192075+0000","last_undegraded":"2024-07-17T08:39:50.192075+0000","last_fullsized":"2024-07-17T08:39:50.192075+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:01.763452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.35","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.206834+0000","last_change":"2024-07-17T08:39:49.194824+0000","last_active":"2024-07-17T08:39:50.206834+0000","last_peered":"2024-07-17T08:39:50.206834+0000","last_clean":"2024-07-17T08:39:50.206834+0000","last_became_active":"2024-07-17T08:39:49.194511+0000","last_became_peered":"2024-07-17T08:39:49.194511+0000","last_unstale":"2024-07-17T08:39:50.206834+0000","last_undegraded":"2024-07-17T08:39:50.206834+0000","last_fullsized":"2024-07-17T08:39:50.206834+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:56:02.757637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.34","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750164+0000","last_change":"2024-07-17T08:39:49.209397+0000","last_active":"2024-07-17T08:39:50.750164+0000","last_peered":"2024-07-17T08:39:50.750164+0000","last_clean":"2024-07-17T08:39:50.750164+0000","last_became_active":"2024-07-17T08:39:49.209102+0000","last_became_peered":"2024-07-17T08:39:49.209102+0000","last_unstale":"2024-07-17T08:39:50.750164+0000","last_undegraded":"2024-07-17T08:39:50.750164+0000","last_fullsized":"2024-07-17T08:39:50.750164+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:51.611046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.33","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186382+0000","last_change":"2024-07-17T08:39:49.196339+0000","last_active":"2024-07-17T08:39:50.186382+0000","last_peered":"2024-07-17T08:39:50.186382+0000","last_clean":"2024-07-17T08:39:50.186382+0000","last_became_active":"2024-07-17T08:39:49.194882+0000","last_became_peered":"2024-07-17T08:39:49.194882+0000","last_unstale":"2024-07-17T08:39:50.186382+0000","last_undegraded":"2024-07-17T08:39:50.186382+0000","last_fullsized":"2024-07-17T08:39:50.186382+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:41.631295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.32","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746125+0000","last_change":"2024-07-17T08:39:49.203741+0000","last_active":"2024-07-17T08:39:50.746125+0000","last_peered":"2024-07-17T08:39:50.746125+0000","last_clean":"2024-07-17T08:39:50.746125+0000","last_became_active":"2024-07-17T08:39:49.200081+0000","last_became_peered":"2024-07-17T08:39:49.200081+0000","last_unstale":"2024-07-17T08:39:50.746125+0000","last_undegraded":"2024-07-17T08:39:50.746125+0000","last_fullsized":"2024-07-17T08:39:50.746125+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:26.807248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.31","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.206412+0000","last_change":"2024-07-17T08:39:49.181430+0000","last_active":"2024-07-17T08:39:50.206412+0000","last_peered":"2024-07-17T08:39:50.206412+0000","last_clean":"2024-07-17T08:39:50.206412+0000","last_became_active":"2024-07-17T08:39:49.180103+0000","last_became_peered":"2024-07-17T08:39:49.180103+0000","last_unstale":"2024-07-17T08:39:50.206412+0000","last_undegraded":"2024-07-17T08:39:50.206412+0000","last_fullsized":"2024-07-17T08:39:50.206412+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:16.836012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.30","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189558+0000","last_change":"2024-07-17T08:39:49.206233+0000","last_active":"2024-07-17T08:39:50.189558+0000","last_peered":"2024-07-17T08:39:50.189558+0000","last_clean":"2024-07-17T08:39:50.189558+0000","last_became_active":"2024-07-17T08:39:49.205157+0000","last_became_peered":"2024-07-17T08:39:49.205157+0000","last_unstale":"2024-07-17T08:39:50.189558+0000","last_undegraded":"2024-07-17T08:39:50.189558+0000","last_fullsized":"2024-07-17T08:39:50.189558+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:31:13.393851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.2f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748289+0000","last_change":"2024-07-17T08:39:49.203688+0000","last_active":"2024-07-17T08:39:50.748289+0000","last_peered":"2024-07-17T08:39:50.748289+0000","last_clean":"2024-07-17T08:39:50.748289+0000","last_became_active":"2024-07-17T08:39:49.203366+0000","last_became_peered":"2024-07-17T08:39:49.203366+0000","last_unstale":"2024-07-17T08:39:50.748289+0000","last_undegraded":"2024-07-17T08:39:50.748289+0000","last_fullsized":"2024-07-17T08:39:50.748289+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:26:28.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192266+0000","last_change":"2024-07-17T08:39:49.183932+0000","last_active":"2024-07-17T08:39:50.192266+0000","last_peered":"2024-07-17T08:39:50.192266+0000","last_clean":"2024-07-17T08:39:50.192266+0000","last_became_active":"2024-07-17T08:39:49.183324+0000","last_became_peered":"2024-07-17T08:39:49.183324+0000","last_unstale":"2024-07-17T08:39:50.192266+0000","last_undegraded":"2024-07-17T08:39:50.192266+0000","last_fullsized":"2024-07-17T08:39:50.192266+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:31.995488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.2d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192602+0000","last_change":"2024-07-17T08:39:49.205250+0000","last_active":"2024-07-17T08:39:50.192602+0000","last_peered":"2024-07-17T08:39:50.192602+0000","last_clean":"2024-07-17T08:39:50.192602+0000","last_became_active":"2024-07-17T08:39:49.204061+0000","last_became_peered":"2024-07-17T08:39:49.204061+0000","last_unstale":"2024-07-17T08:39:50.192602+0000","last_undegraded":"2024-07-17T08:39:50.192602+0000","last_fullsized":"2024-07-17T08:39:50.192602+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:09.809269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.2c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205768+0000","last_change":"2024-07-17T08:39:49.190402+0000","last_active":"2024-07-17T08:39:50.205768+0000","last_peered":"2024-07-17T08:39:50.205768+0000","last_clean":"2024-07-17T08:39:50.205768+0000","last_became_active":"2024-07-17T08:39:49.188205+0000","last_became_peered":"2024-07-17T08:39:49.188205+0000","last_unstale":"2024-07-17T08:39:50.205768+0000","last_undegraded":"2024-07-17T08:39:50.205768+0000","last_fullsized":"2024-07-17T08:39:50.205768+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:10.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747010+0000","last_change":"2024-07-17T08:39:49.213098+0000","last_active":"2024-07-17T08:39:50.747010+0000","last_peered":"2024-07-17T08:39:50.747010+0000","last_clean":"2024-07-17T08:39:50.747010+0000","last_became_active":"2024-07-17T08:39:49.205645+0000","last_became_peered":"2024-07-17T08:39:49.205645+0000","last_unstale":"2024-07-17T08:39:50.747010+0000","last_undegraded":"2024-07-17T08:39:50.747010+0000","last_fullsized":"2024-07-17T08:39:50.747010+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:20.532920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745462+0000","last_change":"2024-07-17T08:39:49.200730+0000","last_active":"2024-07-17T08:39:50.745462+0000","last_peered":"2024-07-17T08:39:50.745462+0000","last_clean":"2024-07-17T08:39:50.745462+0000","last_became_active":"2024-07-17T08:39:49.199251+0000","last_became_peered":"2024-07-17T08:39:49.199251+0000","last_unstale":"2024-07-17T08:39:50.745462+0000","last_undegraded":"2024-07-17T08:39:50.745462+0000","last_fullsized":"2024-07-17T08:39:50.745462+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:39.657107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.28","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747509+0000","last_change":"2024-07-17T08:39:49.204724+0000","last_active":"2024-07-17T08:39:50.747509+0000","last_peered":"2024-07-17T08:39:50.747509+0000","last_clean":"2024-07-17T08:39:50.747509+0000","last_became_active":"2024-07-17T08:39:49.204382+0000","last_became_peered":"2024-07-17T08:39:49.204382+0000","last_unstale":"2024-07-17T08:39:50.747509+0000","last_undegraded":"2024-07-17T08:39:50.747509+0000","last_fullsized":"2024-07-17T08:39:50.747509+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:31:40.398161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.27","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748271+0000","last_change":"2024-07-17T08:39:49.196244+0000","last_active":"2024-07-17T08:39:50.748271+0000","last_peered":"2024-07-17T08:39:50.748271+0000","last_clean":"2024-07-17T08:39:50.748271+0000","last_became_active":"2024-07-17T08:39:49.192099+0000","last_became_peered":"2024-07-17T08:39:49.192099+0000","last_unstale":"2024-07-17T08:39:50.748271+0000","last_undegraded":"2024-07-17T08:39:50.748271+0000","last_fullsized":"2024-07-17T08:39:50.748271+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:23:36.119712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.26","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750504+0000","last_change":"2024-07-17T08:39:49.206225+0000","last_active":"2024-07-17T08:39:50.750504+0000","last_peered":"2024-07-17T08:39:50.750504+0000","last_clean":"2024-07-17T08:39:50.750504+0000","last_became_active":"2024-07-17T08:39:49.204799+0000","last_became_peered":"2024-07-17T08:39:49.204799+0000","last_unstale":"2024-07-17T08:39:50.750504+0000","last_undegraded":"2024-07-17T08:39:50.750504+0000","last_fullsized":"2024-07-17T08:39:50.750504+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:17.454112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.25","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192650+0000","last_change":"2024-07-17T08:39:49.197823+0000","last_active":"2024-07-17T08:39:50.192650+0000","last_peered":"2024-07-17T08:39:50.192650+0000","last_clean":"2024-07-17T08:39:50.192650+0000","last_became_active":"2024-07-17T08:39:49.196073+0000","last_became_peered":"2024-07-17T08:39:49.196073+0000","last_unstale":"2024-07-17T08:39:50.192650+0000","last_undegraded":"2024-07-17T08:39:50.192650+0000","last_fullsized":"2024-07-17T08:39:50.192650+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:17.942979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.24","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748373+0000","last_change":"2024-07-17T08:39:49.184470+0000","last_active":"2024-07-17T08:39:50.748373+0000","last_peered":"2024-07-17T08:39:50.748373+0000","last_clean":"2024-07-17T08:39:50.748373+0000","last_became_active":"2024-07-17T08:39:49.183734+0000","last_became_peered":"2024-07-17T08:39:49.183734+0000","last_unstale":"2024-07-17T08:39:50.748373+0000","last_undegraded":"2024-07-17T08:39:50.748373+0000","last_fullsized":"2024-07-17T08:39:50.748373+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:23.219278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.23","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747986+0000","last_change":"2024-07-17T08:39:49.192970+0000","last_active":"2024-07-17T08:39:50.747986+0000","last_peered":"2024-07-17T08:39:50.747986+0000","last_clean":"2024-07-17T08:39:50.747986+0000","last_became_active":"2024-07-17T08:39:49.192683+0000","last_became_peered":"2024-07-17T08:39:49.192683+0000","last_unstale":"2024-07-17T08:39:50.747986+0000","last_undegraded":"2024-07-17T08:39:50.747986+0000","last_fullsized":"2024-07-17T08:39:50.747986+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:54.904157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.22","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746188+0000","last_change":"2024-07-17T08:39:49.200039+0000","last_active":"2024-07-17T08:39:50.746188+0000","last_peered":"2024-07-17T08:39:50.746188+0000","last_clean":"2024-07-17T08:39:50.746188+0000","last_became_active":"2024-07-17T08:39:49.199749+0000","last_became_peered":"2024-07-17T08:39:49.199749+0000","last_unstale":"2024-07-17T08:39:50.746188+0000","last_undegraded":"2024-07-17T08:39:50.746188+0000","last_fullsized":"2024-07-17T08:39:50.746188+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:20.010704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.21","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745968+0000","last_change":"2024-07-17T08:39:49.213304+0000","last_active":"2024-07-17T08:39:50.745968+0000","last_peered":"2024-07-17T08:39:50.745968+0000","last_clean":"2024-07-17T08:39:50.745968+0000","last_became_active":"2024-07-17T08:39:49.205101+0000","last_became_peered":"2024-07-17T08:39:49.205101+0000","last_unstale":"2024-07-17T08:39:50.745968+0000","last_undegraded":"2024-07-17T08:39:50.745968+0000","last_fullsized":"2024-07-17T08:39:50.745968+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:30:07.937731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.20","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189194+0000","last_change":"2024-07-17T08:39:49.193733+0000","last_active":"2024-07-17T08:39:50.189194+0000","last_peered":"2024-07-17T08:39:50.189194+0000","last_clean":"2024-07-17T08:39:50.189194+0000","last_became_active":"2024-07-17T08:39:49.191631+0000","last_became_peered":"2024-07-17T08:39:49.191631+0000","last_unstale":"2024-07-17T08:39:50.189194+0000","last_undegraded":"2024-07-17T08:39:50.189194+0000","last_fullsized":"2024-07-17T08:39:50.189194+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:18.965583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.10","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.204891+0000","last_change":"2024-07-17T08:39:49.201691+0000","last_active":"2024-07-17T08:39:50.204891+0000","last_peered":"2024-07-17T08:39:50.204891+0000","last_clean":"2024-07-17T08:39:50.204891+0000","last_became_active":"2024-07-17T08:39:49.196734+0000","last_became_peered":"2024-07-17T08:39:49.196734+0000","last_unstale":"2024-07-17T08:39:50.204891+0000","last_undegraded":"2024-07-17T08:39:50.204891+0000","last_fullsized":"2024-07-17T08:39:50.204891+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:56.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.1f","version":"292'4","reported_seq":18,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.187263+0000","last_change":"2024-07-17T08:39:48.165633+0000","last_active":"2024-07-17T08:39:50.187263+0000","last_peered":"2024-07-17T08:39:50.187263+0000","last_clean":"2024-07-17T08:39:50.187263+0000","last_became_active":"2024-07-17T08:39:48.165317+0000","last_became_peered":"2024-07-17T08:39:48.165317+0000","last_unstale":"2024-07-17T08:39:50.187263+0000","last_undegraded":"2024-07-17T08:39:50.187263+0000","last_fullsized":"2024-07-17T08:39:50.187263+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:55:11.776255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.11","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186482+0000","last_change":"2024-07-17T08:39:49.203969+0000","last_active":"2024-07-17T08:39:50.186482+0000","last_peered":"2024-07-17T08:39:50.186482+0000","last_clean":"2024-07-17T08:39:50.186482+0000","last_became_active":"2024-07-17T08:39:49.203689+0000","last_became_peered":"2024-07-17T08:39:49.203689+0000","last_unstale":"2024-07-17T08:39:50.186482+0000","last_undegraded":"2024-07-17T08:39:50.186482+0000","last_fullsized":"2024-07-17T08:39:50.186482+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:12.150998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.1e","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193497+0000","last_change":"2024-07-17T08:39:48.162361+0000","last_active":"2024-07-17T08:39:50.193497+0000","last_peered":"2024-07-17T08:39:50.193497+0000","last_clean":"2024-07-17T08:39:50.193497+0000","last_became_active":"2024-07-17T08:39:48.161265+0000","last_became_peered":"2024-07-17T08:39:48.161265+0000","last_unstale":"2024-07-17T08:39:50.193497+0000","last_undegraded":"2024-07-17T08:39:50.193497+0000","last_fullsized":"2024-07-17T08:39:50.193497+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:56.228522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.12","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746835+0000","last_change":"2024-07-17T08:39:49.207688+0000","last_active":"2024-07-17T08:39:50.746835+0000","last_peered":"2024-07-17T08:39:50.746835+0000","last_clean":"2024-07-17T08:39:50.746835+0000","last_became_active":"2024-07-17T08:39:49.204861+0000","last_became_peered":"2024-07-17T08:39:49.204861+0000","last_unstale":"2024-07-17T08:39:50.746835+0000","last_undegraded":"2024-07-17T08:39:50.746835+0000","last_fullsized":"2024-07-17T08:39:50.746835+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:14.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.1d","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191840+0000","last_change":"2024-07-17T08:39:48.161567+0000","last_active":"2024-07-17T08:39:50.191840+0000","last_peered":"2024-07-17T08:39:50.191840+0000","last_clean":"2024-07-17T08:39:50.191840+0000","last_became_active":"2024-07-17T08:39:48.161292+0000","last_became_peered":"2024-07-17T08:39:48.161292+0000","last_unstale":"2024-07-17T08:39:50.191840+0000","last_undegraded":"2024-07-17T08:39:50.191840+0000","last_fullsized":"2024-07-17T08:39:50.191840+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:34.850760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.13","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748421+0000","last_change":"2024-07-17T08:39:49.185040+0000","last_active":"2024-07-17T08:39:50.748421+0000","last_peered":"2024-07-17T08:39:50.748421+0000","last_clean":"2024-07-17T08:39:50.748421+0000","last_became_active":"2024-07-17T08:39:49.184366+0000","last_became_peered":"2024-07-17T08:39:49.184366+0000","last_unstale":"2024-07-17T08:39:50.748421+0000","last_undegraded":"2024-07-17T08:39:50.748421+0000","last_fullsized":"2024-07-17T08:39:50.748421+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:23.719237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.1c","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747513+0000","last_change":"2024-07-17T08:39:48.166086+0000","last_active":"2024-07-17T08:39:50.747513+0000","last_peered":"2024-07-17T08:39:50.747513+0000","last_clean":"2024-07-17T08:39:50.747513+0000","last_became_active":"2024-07-17T08:39:48.165790+0000","last_became_peered":"2024-07-17T08:39:48.165790+0000","last_unstale":"2024-07-17T08:39:50.747513+0000","last_undegraded":"2024-07-17T08:39:50.747513+0000","last_fullsized":"2024-07-17T08:39:50.747513+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:45.189150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.14","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190925+0000","last_change":"2024-07-17T08:39:49.200346+0000","last_active":"2024-07-17T08:39:50.190925+0000","last_peered":"2024-07-17T08:39:50.190925+0000","last_clean":"2024-07-17T08:39:50.190925+0000","last_became_active":"2024-07-17T08:39:49.200055+0000","last_became_peered":"2024-07-17T08:39:49.200055+0000","last_unstale":"2024-07-17T08:39:50.190925+0000","last_undegraded":"2024-07-17T08:39:50.190925+0000","last_fullsized":"2024-07-17T08:39:50.190925+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:17.100733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.1b","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748733+0000","last_change":"2024-07-17T08:39:48.171457+0000","last_active":"2024-07-17T08:39:50.748733+0000","last_peered":"2024-07-17T08:39:50.748733+0000","last_clean":"2024-07-17T08:39:50.748733+0000","last_became_active":"2024-07-17T08:39:48.171135+0000","last_became_peered":"2024-07-17T08:39:48.171135+0000","last_unstale":"2024-07-17T08:39:50.748733+0000","last_undegraded":"2024-07-17T08:39:50.748733+0000","last_fullsized":"2024-07-17T08:39:50.748733+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:48.924683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.15","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750331+0000","last_change":"2024-07-17T08:39:49.206120+0000","last_active":"2024-07-17T08:39:50.750331+0000","last_peered":"2024-07-17T08:39:50.750331+0000","last_clean":"2024-07-17T08:39:50.750331+0000","last_became_active":"2024-07-17T08:39:49.205405+0000","last_became_peered":"2024-07-17T08:39:49.205405+0000","last_unstale":"2024-07-17T08:39:50.750331+0000","last_undegraded":"2024-07-17T08:39:50.750331+0000","last_fullsized":"2024-07-17T08:39:50.750331+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:33.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.1a","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205176+0000","last_change":"2024-07-17T08:39:48.171134+0000","last_active":"2024-07-17T08:39:50.205176+0000","last_peered":"2024-07-17T08:39:50.205176+0000","last_clean":"2024-07-17T08:39:50.205176+0000","last_became_active":"2024-07-17T08:39:48.170586+0000","last_became_peered":"2024-07-17T08:39:48.170586+0000","last_unstale":"2024-07-17T08:39:50.205176+0000","last_undegraded":"2024-07-17T08:39:50.205176+0000","last_fullsized":"2024-07-17T08:39:50.205176+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:46.204589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.16","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751076+0000","last_change":"2024-07-17T08:39:49.204903+0000","last_active":"2024-07-17T08:39:50.751076+0000","last_peered":"2024-07-17T08:39:50.751076+0000","last_clean":"2024-07-17T08:39:50.751076+0000","last_became_active":"2024-07-17T08:39:49.204565+0000","last_became_peered":"2024-07-17T08:39:49.204565+0000","last_unstale":"2024-07-17T08:39:50.751076+0000","last_undegraded":"2024-07-17T08:39:50.751076+0000","last_fullsized":"2024-07-17T08:39:50.751076+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:32.556141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.19","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189168+0000","last_change":"2024-07-17T08:39:48.167801+0000","last_active":"2024-07-17T08:39:50.189168+0000","last_peered":"2024-07-17T08:39:50.189168+0000","last_clean":"2024-07-17T08:39:50.189168+0000","last_became_active":"2024-07-17T08:39:48.167501+0000","last_became_peered":"2024-07-17T08:39:48.167501+0000","last_unstale":"2024-07-17T08:39:50.189168+0000","last_undegraded":"2024-07-17T08:39:50.189168+0000","last_fullsized":"2024-07-17T08:39:50.189168+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:54.002002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.17","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193022+0000","last_change":"2024-07-17T08:39:49.195083+0000","last_active":"2024-07-17T08:39:50.193022+0000","last_peered":"2024-07-17T08:39:50.193022+0000","last_clean":"2024-07-17T08:39:50.193022+0000","last_became_active":"2024-07-17T08:39:49.194800+0000","last_became_peered":"2024-07-17T08:39:49.194800+0000","last_unstale":"2024-07-17T08:39:50.193022+0000","last_undegraded":"2024-07-17T08:39:50.193022+0000","last_fullsized":"2024-07-17T08:39:50.193022+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:31.750567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.18","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745771+0000","last_change":"2024-07-17T08:39:48.164429+0000","last_active":"2024-07-17T08:39:50.745771+0000","last_peered":"2024-07-17T08:39:50.745771+0000","last_clean":"2024-07-17T08:39:50.745771+0000","last_became_active":"2024-07-17T08:39:48.163887+0000","last_became_peered":"2024-07-17T08:39:48.163887+0000","last_unstale":"2024-07-17T08:39:50.745771+0000","last_undegraded":"2024-07-17T08:39:50.745771+0000","last_fullsized":"2024-07-17T08:39:50.745771+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:20.004633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.18","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745943+0000","last_change":"2024-07-17T08:39:49.205703+0000","last_active":"2024-07-17T08:39:50.745943+0000","last_peered":"2024-07-17T08:39:50.745943+0000","last_clean":"2024-07-17T08:39:50.745943+0000","last_became_active":"2024-07-17T08:39:49.205390+0000","last_became_peered":"2024-07-17T08:39:49.205390+0000","last_unstale":"2024-07-17T08:39:50.745943+0000","last_undegraded":"2024-07-17T08:39:50.745943+0000","last_fullsized":"2024-07-17T08:39:50.745943+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:40:35.218069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.17","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748703+0000","last_change":"2024-07-17T08:39:48.172031+0000","last_active":"2024-07-17T08:39:50.748703+0000","last_peered":"2024-07-17T08:39:50.748703+0000","last_clean":"2024-07-17T08:39:50.748703+0000","last_became_active":"2024-07-17T08:39:48.171580+0000","last_became_peered":"2024-07-17T08:39:48.171580+0000","last_unstale":"2024-07-17T08:39:50.748703+0000","last_undegraded":"2024-07-17T08:39:50.748703+0000","last_fullsized":"2024-07-17T08:39:50.748703+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:56.064828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.19","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205920+0000","last_change":"2024-07-17T08:39:49.196620+0000","last_active":"2024-07-17T08:39:50.205920+0000","last_peered":"2024-07-17T08:39:50.205920+0000","last_clean":"2024-07-17T08:39:50.205920+0000","last_became_active":"2024-07-17T08:39:49.195232+0000","last_became_peered":"2024-07-17T08:39:49.195232+0000","last_unstale":"2024-07-17T08:39:50.205920+0000","last_undegraded":"2024-07-17T08:39:50.205920+0000","last_fullsized":"2024-07-17T08:39:50.205920+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:23.362643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.16","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186943+0000","last_change":"2024-07-17T08:39:48.165774+0000","last_active":"2024-07-17T08:39:50.186943+0000","last_peered":"2024-07-17T08:39:50.186943+0000","last_clean":"2024-07-17T08:39:50.186943+0000","last_became_active":"2024-07-17T08:39:48.165270+0000","last_became_peered":"2024-07-17T08:39:48.165270+0000","last_unstale":"2024-07-17T08:39:50.186943+0000","last_undegraded":"2024-07-17T08:39:50.186943+0000","last_fullsized":"2024-07-17T08:39:50.186943+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:01.710783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.1a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749753+0000","last_change":"2024-07-17T08:39:49.207881+0000","last_active":"2024-07-17T08:39:50.749753+0000","last_peered":"2024-07-17T08:39:50.749753+0000","last_clean":"2024-07-17T08:39:50.749753+0000","last_became_active":"2024-07-17T08:39:49.207607+0000","last_became_peered":"2024-07-17T08:39:49.207607+0000","last_unstale":"2024-07-17T08:39:50.749753+0000","last_undegraded":"2024-07-17T08:39:50.749753+0000","last_fullsized":"2024-07-17T08:39:50.749753+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:27:16.942577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.15","version":"292'3","reported_seq":17,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193007+0000","last_change":"2024-07-17T08:39:48.173854+0000","last_active":"2024-07-17T08:39:50.193007+0000","last_peered":"2024-07-17T08:39:50.193007+0000","last_clean":"2024-07-17T08:39:50.193007+0000","last_became_active":"2024-07-17T08:39:48.173346+0000","last_became_peered":"2024-07-17T08:39:48.173346+0000","last_unstale":"2024-07-17T08:39:50.193007+0000","last_undegraded":"2024-07-17T08:39:50.193007+0000","last_fullsized":"2024-07-17T08:39:50.193007+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:14.390430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.1b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191442+0000","last_change":"2024-07-17T08:39:49.204963+0000","last_active":"2024-07-17T08:39:50.191442+0000","last_peered":"2024-07-17T08:39:50.191442+0000","last_clean":"2024-07-17T08:39:50.191442+0000","last_became_active":"2024-07-17T08:39:49.204614+0000","last_became_peered":"2024-07-17T08:39:49.204614+0000","last_unstale":"2024-07-17T08:39:50.191442+0000","last_undegraded":"2024-07-17T08:39:50.191442+0000","last_fullsized":"2024-07-17T08:39:50.191442+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:16.887020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.14","version":"292'6","reported_seq":20,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750074+0000","last_change":"2024-07-17T08:39:48.166353+0000","last_active":"2024-07-17T08:39:50.750074+0000","last_peered":"2024-07-17T08:39:50.750074+0000","last_clean":"2024-07-17T08:39:50.750074+0000","last_became_active":"2024-07-17T08:39:48.163682+0000","last_became_peered":"2024-07-17T08:39:48.163682+0000","last_unstale":"2024-07-17T08:39:50.750074+0000","last_undegraded":"2024-07-17T08:39:50.750074+0000","last_fullsized":"2024-07-17T08:39:50.750074+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.953888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745646+0000","last_change":"2024-07-17T08:39:49.204829+0000","last_active":"2024-07-17T08:39:50.745646+0000","last_peered":"2024-07-17T08:39:50.745646+0000","last_clean":"2024-07-17T08:39:50.745646+0000","last_became_active":"2024-07-17T08:39:49.204499+0000","last_became_peered":"2024-07-17T08:39:49.204499+0000","last_unstale":"2024-07-17T08:39:50.745646+0000","last_undegraded":"2024-07-17T08:39:50.745646+0000","last_fullsized":"2024-07-17T08:39:50.745646+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:08.773614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"39.13","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746069+0000","last_change":"2024-07-17T08:39:48.174660+0000","last_active":"2024-07-17T08:39:50.746069+0000","last_peered":"2024-07-17T08:39:50.746069+0000","last_clean":"2024-07-17T08:39:50.746069+0000","last_became_active":"2024-07-17T08:39:48.174361+0000","last_became_peered":"2024-07-17T08:39:48.174361+0000","last_unstale":"2024-07-17T08:39:50.746069+0000","last_undegraded":"2024-07-17T08:39:50.746069+0000","last_fullsized":"2024-07-17T08:39:50.746069+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:01:34.979710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.1d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193155+0000","last_change":"2024-07-17T08:39:49.202811+0000","last_active":"2024-07-17T08:39:50.193155+0000","last_peered":"2024-07-17T08:39:50.193155+0000","last_clean":"2024-07-17T08:39:50.193155+0000","last_became_active":"2024-07-17T08:39:49.202534+0000","last_became_peered":"2024-07-17T08:39:49.202534+0000","last_unstale":"2024-07-17T08:39:50.193155+0000","last_undegraded":"2024-07-17T08:39:50.193155+0000","last_fullsized":"2024-07-17T08:39:50.193155+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:45:03.313106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.12","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749489+0000","last_change":"2024-07-17T08:39:48.173852+0000","last_active":"2024-07-17T08:39:50.749489+0000","last_peered":"2024-07-17T08:39:50.749489+0000","last_clean":"2024-07-17T08:39:50.749489+0000","last_became_active":"2024-07-17T08:39:48.173429+0000","last_became_peered":"2024-07-17T08:39:48.173429+0000","last_unstale":"2024-07-17T08:39:50.749489+0000","last_undegraded":"2024-07-17T08:39:50.749489+0000","last_fullsized":"2024-07-17T08:39:50.749489+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:34.317499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192477+0000","last_change":"2024-07-17T08:39:49.184195+0000","last_active":"2024-07-17T08:39:50.192477+0000","last_peered":"2024-07-17T08:39:50.192477+0000","last_clean":"2024-07-17T08:39:50.192477+0000","last_became_active":"2024-07-17T08:39:49.183551+0000","last_became_peered":"2024-07-17T08:39:49.183551+0000","last_unstale":"2024-07-17T08:39:50.192477+0000","last_undegraded":"2024-07-17T08:39:50.192477+0000","last_fullsized":"2024-07-17T08:39:50.192477+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:22.025251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.11","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745874+0000","last_change":"2024-07-17T08:39:48.172319+0000","last_active":"2024-07-17T08:39:50.745874+0000","last_peered":"2024-07-17T08:39:50.745874+0000","last_clean":"2024-07-17T08:39:50.745874+0000","last_became_active":"2024-07-17T08:39:48.171643+0000","last_became_peered":"2024-07-17T08:39:48.171643+0000","last_unstale":"2024-07-17T08:39:50.745874+0000","last_undegraded":"2024-07-17T08:39:50.745874+0000","last_fullsized":"2024-07-17T08:39:50.745874+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:50.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.1f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751087+0000","last_change":"2024-07-17T08:39:49.209453+0000","last_active":"2024-07-17T08:39:50.751087+0000","last_peered":"2024-07-17T08:39:50.751087+0000","last_clean":"2024-07-17T08:39:50.751087+0000","last_became_active":"2024-07-17T08:39:49.208731+0000","last_became_peered":"2024-07-17T08:39:49.208731+0000","last_unstale":"2024-07-17T08:39:50.751087+0000","last_undegraded":"2024-07-17T08:39:50.751087+0000","last_fullsized":"2024-07-17T08:39:50.751087+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:27.449975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.10","version":"292'8","reported_seq":22,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191835+0000","last_change":"2024-07-17T08:39:48.173488+0000","last_active":"2024-07-17T08:39:50.191835+0000","last_peered":"2024-07-17T08:39:50.191835+0000","last_clean":"2024-07-17T08:39:50.191835+0000","last_became_active":"2024-07-17T08:39:48.171757+0000","last_became_peered":"2024-07-17T08:39:48.171757+0000","last_unstale":"2024-07-17T08:39:50.191835+0000","last_undegraded":"2024-07-17T08:39:50.191835+0000","last_fullsized":"2024-07-17T08:39:50.191835+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:56.241476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.0","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745766+0000","last_change":"2024-07-17T08:39:49.207321+0000","last_active":"2024-07-17T08:39:50.745766+0000","last_peered":"2024-07-17T08:39:50.745766+0000","last_clean":"2024-07-17T08:39:50.745766+0000","last_became_active":"2024-07-17T08:39:49.203599+0000","last_became_peered":"2024-07-17T08:39:49.203599+0000","last_unstale":"2024-07-17T08:39:50.745766+0000","last_undegraded":"2024-07-17T08:39:50.745766+0000","last_fullsized":"2024-07-17T08:39:50.745766+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:31.183780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.f","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751294+0000","last_change":"2024-07-17T08:39:48.172822+0000","last_active":"2024-07-17T08:39:50.751294+0000","last_peered":"2024-07-17T08:39:50.751294+0000","last_clean":"2024-07-17T08:39:50.751294+0000","last_became_active":"2024-07-17T08:39:48.172493+0000","last_became_peered":"2024-07-17T08:39:48.172493+0000","last_unstale":"2024-07-17T08:39:50.751294+0000","last_undegraded":"2024-07-17T08:39:50.751294+0000","last_fullsized":"2024-07-17T08:39:50.751294+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:06:03.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746379+0000","last_change":"2024-07-17T08:39:49.213791+0000","last_active":"2024-07-17T08:39:50.746379+0000","last_peered":"2024-07-17T08:39:50.746379+0000","last_clean":"2024-07-17T08:39:50.746379+0000","last_became_active":"2024-07-17T08:39:49.205472+0000","last_became_peered":"2024-07-17T08:39:49.205472+0000","last_unstale":"2024-07-17T08:39:50.746379+0000","last_undegraded":"2024-07-17T08:39:50.746379+0000","last_fullsized":"2024-07-17T08:39:50.746379+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:36.631906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.e","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747325+0000","last_change":"2024-07-17T08:39:48.170718+0000","last_active":"2024-07-17T08:39:50.747325+0000","last_peered":"2024-07-17T08:39:50.747325+0000","last_clean":"2024-07-17T08:39:50.747325+0000","last_became_active":"2024-07-17T08:39:48.170401+0000","last_became_peered":"2024-07-17T08:39:48.170401+0000","last_unstale":"2024-07-17T08:39:50.747325+0000","last_undegraded":"2024-07-17T08:39:50.747325+0000","last_fullsized":"2024-07-17T08:39:50.747325+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:26.439772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.2","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747304+0000","last_change":"2024-07-17T08:39:49.202315+0000","last_active":"2024-07-17T08:39:50.747304+0000","last_peered":"2024-07-17T08:39:50.747304+0000","last_clean":"2024-07-17T08:39:50.747304+0000","last_became_active":"2024-07-17T08:39:49.201687+0000","last_became_peered":"2024-07-17T08:39:49.201687+0000","last_unstale":"2024-07-17T08:39:50.747304+0000","last_undegraded":"2024-07-17T08:39:50.747304+0000","last_fullsized":"2024-07-17T08:39:50.747304+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:07.813566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.d","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748844+0000","last_change":"2024-07-17T08:39:48.170778+0000","last_active":"2024-07-17T08:39:50.748844+0000","last_peered":"2024-07-17T08:39:50.748844+0000","last_clean":"2024-07-17T08:39:50.748844+0000","last_became_active":"2024-07-17T08:39:48.170153+0000","last_became_peered":"2024-07-17T08:39:48.170153+0000","last_unstale":"2024-07-17T08:39:50.748844+0000","last_undegraded":"2024-07-17T08:39:50.748844+0000","last_fullsized":"2024-07-17T08:39:50.748844+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:31.545939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749788+0000","last_change":"2024-07-17T08:39:49.202817+0000","last_active":"2024-07-17T08:39:50.749788+0000","last_peered":"2024-07-17T08:39:50.749788+0000","last_clean":"2024-07-17T08:39:50.749788+0000","last_became_active":"2024-07-17T08:39:49.202518+0000","last_became_peered":"2024-07-17T08:39:49.202518+0000","last_unstale":"2024-07-17T08:39:50.749788+0000","last_undegraded":"2024-07-17T08:39:50.749788+0000","last_fullsized":"2024-07-17T08:39:50.749788+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:21.014835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.c","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747702+0000","last_change":"2024-07-17T08:39:48.159704+0000","last_active":"2024-07-17T08:39:50.747702+0000","last_peered":"2024-07-17T08:39:50.747702+0000","last_clean":"2024-07-17T08:39:50.747702+0000","last_became_active":"2024-07-17T08:39:48.159419+0000","last_became_peered":"2024-07-17T08:39:48.159419+0000","last_unstale":"2024-07-17T08:39:50.747702+0000","last_undegraded":"2024-07-17T08:39:50.747702+0000","last_fullsized":"2024-07-17T08:39:50.747702+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:09.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.4","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191031+0000","last_change":"2024-07-17T08:39:49.199289+0000","last_active":"2024-07-17T08:39:50.191031+0000","last_peered":"2024-07-17T08:39:50.191031+0000","last_clean":"2024-07-17T08:39:50.191031+0000","last_became_active":"2024-07-17T08:39:49.199012+0000","last_became_peered":"2024-07-17T08:39:49.199012+0000","last_unstale":"2024-07-17T08:39:50.191031+0000","last_undegraded":"2024-07-17T08:39:50.191031+0000","last_fullsized":"2024-07-17T08:39:50.191031+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:09.132629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.b","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191772+0000","last_change":"2024-07-17T08:39:48.173469+0000","last_active":"2024-07-17T08:39:50.191772+0000","last_peered":"2024-07-17T08:39:50.191772+0000","last_clean":"2024-07-17T08:39:50.191772+0000","last_became_active":"2024-07-17T08:39:48.172535+0000","last_became_peered":"2024-07-17T08:39:48.172535+0000","last_unstale":"2024-07-17T08:39:50.191772+0000","last_undegraded":"2024-07-17T08:39:50.191772+0000","last_fullsized":"2024-07-17T08:39:50.191772+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:15.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.5","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191538+0000","last_change":"2024-07-17T08:39:49.194931+0000","last_active":"2024-07-17T08:39:50.191538+0000","last_peered":"2024-07-17T08:39:50.191538+0000","last_clean":"2024-07-17T08:39:50.191538+0000","last_became_active":"2024-07-17T08:39:49.194618+0000","last_became_peered":"2024-07-17T08:39:49.194618+0000","last_unstale":"2024-07-17T08:39:50.191538+0000","last_undegraded":"2024-07-17T08:39:50.191538+0000","last_fullsized":"2024-07-17T08:39:50.191538+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:28.365958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.a","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746854+0000","last_change":"2024-07-17T08:39:48.168665+0000","last_active":"2024-07-17T08:39:50.746854+0000","last_peered":"2024-07-17T08:39:50.746854+0000","last_clean":"2024-07-17T08:39:50.746854+0000","last_became_active":"2024-07-17T08:39:48.168343+0000","last_became_peered":"2024-07-17T08:39:48.168343+0000","last_unstale":"2024-07-17T08:39:50.746854+0000","last_undegraded":"2024-07-17T08:39:50.746854+0000","last_fullsized":"2024-07-17T08:39:50.746854+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:32.892048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.6","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192128+0000","last_change":"2024-07-17T08:39:49.206859+0000","last_active":"2024-07-17T08:39:50.192128+0000","last_peered":"2024-07-17T08:39:50.192128+0000","last_clean":"2024-07-17T08:39:50.192128+0000","last_became_active":"2024-07-17T08:39:49.205573+0000","last_became_peered":"2024-07-17T08:39:49.205573+0000","last_unstale":"2024-07-17T08:39:50.192128+0000","last_undegraded":"2024-07-17T08:39:50.192128+0000","last_fullsized":"2024-07-17T08:39:50.192128+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:41:58.519570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.9","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745408+0000","last_change":"2024-07-17T08:39:48.174570+0000","last_active":"2024-07-17T08:39:50.745408+0000","last_peered":"2024-07-17T08:39:50.745408+0000","last_clean":"2024-07-17T08:39:50.745408+0000","last_became_active":"2024-07-17T08:39:48.174272+0000","last_became_peered":"2024-07-17T08:39:48.174272+0000","last_unstale":"2024-07-17T08:39:50.745408+0000","last_undegraded":"2024-07-17T08:39:50.745408+0000","last_fullsized":"2024-07-17T08:39:50.745408+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:48.470494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.7","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748712+0000","last_change":"2024-07-17T08:39:49.184120+0000","last_active":"2024-07-17T08:39:50.748712+0000","last_peered":"2024-07-17T08:39:50.748712+0000","last_clean":"2024-07-17T08:39:50.748712+0000","last_became_active":"2024-07-17T08:39:49.183336+0000","last_became_peered":"2024-07-17T08:39:49.183336+0000","last_unstale":"2024-07-17T08:39:50.748712+0000","last_undegraded":"2024-07-17T08:39:50.748712+0000","last_fullsized":"2024-07-17T08:39:50.748712+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:09.007050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.8","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189368+0000","last_change":"2024-07-17T08:39:48.169791+0000","last_active":"2024-07-17T08:39:50.189368+0000","last_peered":"2024-07-17T08:39:50.189368+0000","last_clean":"2024-07-17T08:39:50.189368+0000","last_became_active":"2024-07-17T08:39:48.169455+0000","last_became_peered":"2024-07-17T08:39:48.169455+0000","last_unstale":"2024-07-17T08:39:50.189368+0000","last_undegraded":"2024-07-17T08:39:50.189368+0000","last_fullsized":"2024-07-17T08:39:50.189368+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:41.692113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.8","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746338+0000","last_change":"2024-07-17T08:39:49.193363+0000","last_active":"2024-07-17T08:39:50.746338+0000","last_peered":"2024-07-17T08:39:50.746338+0000","last_clean":"2024-07-17T08:39:50.746338+0000","last_became_active":"2024-07-17T08:39:49.192822+0000","last_became_peered":"2024-07-17T08:39:49.192822+0000","last_unstale":"2024-07-17T08:39:50.746338+0000","last_undegraded":"2024-07-17T08:39:50.746338+0000","last_fullsized":"2024-07-17T08:39:50.746338+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:20.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.7","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748594+0000","last_change":"2024-07-17T08:39:48.158819+0000","last_active":"2024-07-17T08:39:50.748594+0000","last_peered":"2024-07-17T08:39:50.748594+0000","last_clean":"2024-07-17T08:39:50.748594+0000","last_became_active":"2024-07-17T08:39:48.158426+0000","last_became_peered":"2024-07-17T08:39:48.158426+0000","last_unstale":"2024-07-17T08:39:50.748594+0000","last_undegraded":"2024-07-17T08:39:50.748594+0000","last_fullsized":"2024-07-17T08:39:50.748594+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:16.554128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.9","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190976+0000","last_change":"2024-07-17T08:39:49.215412+0000","last_active":"2024-07-17T08:39:50.190976+0000","last_peered":"2024-07-17T08:39:50.190976+0000","last_clean":"2024-07-17T08:39:50.190976+0000","last_became_active":"2024-07-17T08:39:49.215119+0000","last_became_peered":"2024-07-17T08:39:49.215119+0000","last_unstale":"2024-07-17T08:39:50.190976+0000","last_undegraded":"2024-07-17T08:39:50.190976+0000","last_fullsized":"2024-07-17T08:39:50.190976+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:32.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.6","version":"293'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.203296+0000","last_change":"2024-07-17T08:39:48.169017+0000","last_active":"2024-07-17T08:39:50.203296+0000","last_peered":"2024-07-17T08:39:50.203296+0000","last_clean":"2024-07-17T08:39:50.203296+0000","last_became_active":"2024-07-17T08:39:48.168720+0000","last_became_peered":"2024-07-17T08:39:48.168720+0000","last_unstale":"2024-07-17T08:39:50.203296+0000","last_undegraded":"2024-07-17T08:39:50.203296+0000","last_fullsized":"2024-07-17T08:39:50.203296+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:50.610464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746517+0000","last_change":"2024-07-17T08:39:49.202553+0000","last_active":"2024-07-17T08:39:50.746517+0000","last_peered":"2024-07-17T08:39:50.746517+0000","last_clean":"2024-07-17T08:39:50.746517+0000","last_became_active":"2024-07-17T08:39:49.202009+0000","last_became_peered":"2024-07-17T08:39:49.202009+0000","last_unstale":"2024-07-17T08:39:50.746517+0000","last_undegraded":"2024-07-17T08:39:50.746517+0000","last_fullsized":"2024-07-17T08:39:50.746517+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:43.518457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.5","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745807+0000","last_change":"2024-07-17T08:39:48.175168+0000","last_active":"2024-07-17T08:39:50.745807+0000","last_peered":"2024-07-17T08:39:50.745807+0000","last_clean":"2024-07-17T08:39:50.745807+0000","last_became_active":"2024-07-17T08:39:48.174897+0000","last_became_peered":"2024-07-17T08:39:48.174897+0000","last_unstale":"2024-07-17T08:39:50.745807+0000","last_undegraded":"2024-07-17T08:39:50.745807+0000","last_fullsized":"2024-07-17T08:39:50.745807+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:01.403849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186611+0000","last_change":"2024-07-17T08:39:49.195151+0000","last_active":"2024-07-17T08:39:50.186611+0000","last_peered":"2024-07-17T08:39:50.186611+0000","last_clean":"2024-07-17T08:39:50.186611+0000","last_became_active":"2024-07-17T08:39:49.194482+0000","last_became_peered":"2024-07-17T08:39:49.194482+0000","last_unstale":"2024-07-17T08:39:50.186611+0000","last_undegraded":"2024-07-17T08:39:50.186611+0000","last_fullsized":"2024-07-17T08:39:50.186611+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:58.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.4","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750323+0000","last_change":"2024-07-17T08:39:48.165774+0000","last_active":"2024-07-17T08:39:50.750323+0000","last_peered":"2024-07-17T08:39:50.750323+0000","last_clean":"2024-07-17T08:39:50.750323+0000","last_became_active":"2024-07-17T08:39:48.163275+0000","last_became_peered":"2024-07-17T08:39:48.163275+0000","last_unstale":"2024-07-17T08:39:50.750323+0000","last_undegraded":"2024-07-17T08:39:50.750323+0000","last_fullsized":"2024-07-17T08:39:50.750323+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:36.092376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205874+0000","last_change":"2024-07-17T08:39:49.180920+0000","last_active":"2024-07-17T08:39:50.205874+0000","last_peered":"2024-07-17T08:39:50.205874+0000","last_clean":"2024-07-17T08:39:50.205874+0000","last_became_active":"2024-07-17T08:39:49.180449+0000","last_became_peered":"2024-07-17T08:39:49.180449+0000","last_unstale":"2024-07-17T08:39:50.205874+0000","last_undegraded":"2024-07-17T08:39:50.205874+0000","last_fullsized":"2024-07-17T08:39:50.205874+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:54.499878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.3","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192044+0000","last_change":"2024-07-17T08:39:48.171352+0000","last_active":"2024-07-17T08:39:50.192044+0000","last_peered":"2024-07-17T08:39:50.192044+0000","last_clean":"2024-07-17T08:39:50.192044+0000","last_became_active":"2024-07-17T08:39:48.163520+0000","last_became_peered":"2024-07-17T08:39:48.163520+0000","last_unstale":"2024-07-17T08:39:50.192044+0000","last_undegraded":"2024-07-17T08:39:50.192044+0000","last_fullsized":"2024-07-17T08:39:50.192044+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:12.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750472+0000","last_change":"2024-07-17T08:39:49.209807+0000","last_active":"2024-07-17T08:39:50.750472+0000","last_peered":"2024-07-17T08:39:50.750472+0000","last_clean":"2024-07-17T08:39:50.750472+0000","last_became_active":"2024-07-17T08:39:49.205828+0000","last_became_peered":"2024-07-17T08:39:49.205828+0000","last_unstale":"2024-07-17T08:39:50.750472+0000","last_undegraded":"2024-07-17T08:39:50.750472+0000","last_fullsized":"2024-07-17T08:39:50.750472+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:49.934885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.2","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747126+0000","last_change":"2024-07-17T08:39:48.161149+0000","last_active":"2024-07-17T08:39:50.747126+0000","last_peered":"2024-07-17T08:39:50.747126+0000","last_clean":"2024-07-17T08:39:50.747126+0000","last_became_active":"2024-07-17T08:39:48.160845+0000","last_became_peered":"2024-07-17T08:39:48.160845+0000","last_unstale":"2024-07-17T08:39:50.747126+0000","last_undegraded":"2024-07-17T08:39:50.747126+0000","last_fullsized":"2024-07-17T08:39:50.747126+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:16.443909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191439+0000","last_change":"2024-07-17T08:39:49.209881+0000","last_active":"2024-07-17T08:39:50.191439+0000","last_peered":"2024-07-17T08:39:50.191439+0000","last_clean":"2024-07-17T08:39:50.191439+0000","last_became_active":"2024-07-17T08:39:49.209502+0000","last_became_peered":"2024-07-17T08:39:49.209502+0000","last_unstale":"2024-07-17T08:39:50.191439+0000","last_undegraded":"2024-07-17T08:39:50.191439+0000","last_fullsized":"2024-07-17T08:39:50.191439+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:15.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.1","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745728+0000","last_change":"2024-07-17T08:39:48.176448+0000","last_active":"2024-07-17T08:39:50.745728+0000","last_peered":"2024-07-17T08:39:50.745728+0000","last_clean":"2024-07-17T08:39:50.745728+0000","last_became_active":"2024-07-17T08:39:48.176150+0000","last_became_peered":"2024-07-17T08:39:48.176150+0000","last_unstale":"2024-07-17T08:39:50.745728+0000","last_undegraded":"2024-07-17T08:39:50.745728+0000","last_fullsized":"2024-07-17T08:39:50.745728+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:06.043287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193390+0000","last_change":"2024-07-17T08:39:49.198952+0000","last_active":"2024-07-17T08:39:50.193390+0000","last_peered":"2024-07-17T08:39:50.193390+0000","last_clean":"2024-07-17T08:39:50.193390+0000","last_became_active":"2024-07-17T08:39:49.198402+0000","last_became_peered":"2024-07-17T08:39:49.198402+0000","last_unstale":"2024-07-17T08:39:50.193390+0000","last_undegraded":"2024-07-17T08:39:50.193390+0000","last_fullsized":"2024-07-17T08:39:50.193390+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:39.782605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.0","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.204736+0000","last_change":"2024-07-17T08:39:48.170086+0000","last_active":"2024-07-17T08:39:50.204736+0000","last_peered":"2024-07-17T08:39:50.204736+0000","last_clean":"2024-07-17T08:39:50.204736+0000","last_became_active":"2024-07-17T08:39:48.169527+0000","last_became_peered":"2024-07-17T08:39:48.169527+0000","last_unstale":"2024-07-17T08:39:50.204736+0000","last_undegraded":"2024-07-17T08:39:50.204736+0000","last_fullsized":"2024-07-17T08:39:50.204736+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:50.296585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.29","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190371+0000","last_change":"2024-07-17T08:39:49.206500+0000","last_active":"2024-07-17T08:39:50.190371+0000","last_peered":"2024-07-17T08:39:50.190371+0000","last_clean":"2024-07-17T08:39:50.190371+0000","last_became_active":"2024-07-17T08:39:49.206230+0000","last_became_peered":"2024-07-17T08:39:49.206230+0000","last_unstale":"2024-07-17T08:39:50.190371+0000","last_undegraded":"2024-07-17T08:39:50.190371+0000","last_fullsized":"2024-07-17T08:39:50.190371+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:51.095125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"284'4596","reported_seq":5750,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745623+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:39:50.745623+0000","last_peered":"2024-07-17T08:39:50.745623+0000","last_clean":"2024-07-17T08:39:50.745623+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:39:50.745623+0000","last_undegraded":"2024-07-17T08:39:50.745623+0000","last_fullsized":"2024-07-17T08:39:50.745623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4596,"log_dups_size":0,"ondisk_log_size":4596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":445,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":39,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":40,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":445,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4596,"ondisk_log_size":4596,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543390,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75656,"kb_used_data":4216,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296184,"statfs":{"total":96636764160,"available":96559292416,"internally_reserved":0,"allocated":4317184,"data_stored":2635641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510685,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52860,"kb_used_data":3632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318980,"statfs":{"total":96636764160,"available":96582635520,"internally_reserved":0,"allocated":3719168,"data_stored":2045239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510686,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70928,"kb_used_data":3644,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300912,"statfs":{"total":96636764160,"available":96564133888,"internally_reserved":0,"allocated":3731456,"data_stored":2045981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543391,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52872,"kb_used_data":3640,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318968,"statfs":{"total":96636764160,"available":96582623232,"internally_reserved":0,"allocated":3727360,"data_stored":2045351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510685,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52856,"kb_used_data":3628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318984,"statfs":{"total":96636764160,"available":96582639616,"internally_reserved":0,"allocated":3715072,"data_stored":2038047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543389,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57044,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314796,"statfs":{"total":96636764160,"available":96578351104,"internally_reserved":0,"allocated":3735552,"data_stored":2047433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52860,"kb_used_data":3632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318980,"statfs":{"total":96636764160,"available":96582635520,"internally_reserved":0,"allocated":3719168,"data_stored":2045239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510685,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53448,"kb_used_data":4216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318392,"statfs":{"total":96636764160,"available":96582033408,"internally_reserved":0,"allocated":4317184,"data_stored":2629945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:39:53.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.924+0000 7f993b400640 1 -- 172.21.15.44:0/3470843858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99340acaf0 msgr2=0x7f993406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.924+0000 7f993b400640 1 --2- 172.21.15.44:0/3470843858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99340acaf0 0x7f993406e090 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f9934070a70 tx=0x7f992c02b1e0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 -- 172.21.15.44:0/3470843858 shutdown_connections 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 --2- 172.21.15.44:0/3470843858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99340acaf0 0x7f993406e090 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 --2- 172.21.15.44:0/3470843858 >> v2:172.21.15.175:3300/0 conn(0x7f9934153a50 0x7f9934159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 -- 172.21.15.44:0/3470843858 >> 172.21.15.44:0/3470843858 conn(0x7f99340b0fb0 msgr2=0x7f99340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 -- 172.21.15.44:0/3470843858 shutdown_connections 2024-07-17T08:39:53.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.925+0000 7f993b400640 1 -- 172.21.15.44:0/3470843858 wait complete. 2024-07-17T08:39:53.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.927+0000 7f993b400640 1 Processor -- start 2024-07-17T08:39:53.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.927+0000 7f993b400640 1 -- start start 2024-07-17T08:39:53.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99340acaf0 0x7f9934072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9934153a50 0x7f9934159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f993406f710 con 0x7f993415a040 2024-07-17T08:39:53.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9934154040 con 0x7f99340acaf0 2024-07-17T08:39:53.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.928+0000 7f993b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f993406f890 con 0x7f9934153a50 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.929+0000 7f9939400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.929+0000 7f9939400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41724/0 (socket says 172.21.15.44:41724) 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.929+0000 7f9939400640 1 -- 172.21.15.44:0/3270740015 learned_addr learned my addr 172.21.15.44:0/3270740015 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.929+0000 7f9938a00640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99340acaf0 0x7f9934072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.929+0000 7f9933e00640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9934153a50 0x7f9934159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 -- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9934153a50 msgr2=0x7f9934159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9934153a50 0x7f9934159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 -- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99340acaf0 msgr2=0x7f9934072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99340acaf0 0x7f9934072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 -- 172.21.15.44:0/3270740015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341f2c50 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9939400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f992c02b7d0 tx=0x7f992c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c041020 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f992c003ca0 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9934071830 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c005d50 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.930+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9934071df0 con 0x7f993415a040 2024-07-17T08:39:53.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.932+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9934071ff0 con 0x7f993415a040 2024-07-17T08:39:53.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.933+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f992c007cb0 con 0x7f993415a040 2024-07-17T08:39:53.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.934+0000 7f9930c00640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 0x7f9910079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:53.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.934+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f992c0b8710 con 0x7f993415a040 2024-07-17T08:39:53.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.934+0000 7f9938a00640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 0x7f9910079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:53.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.936+0000 7f9938a00640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 0x7f9910079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009870 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:53.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:53.938+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9934071ff0 con 0x7f993415a040 2024-07-17T08:39:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.145+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 --> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9934071620 con 0x7f9910076bb0 2024-07-17T08:39:54.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.159+0000 7f9930c00640 1 -- 172.21.15.44:0/3270740015 <== mgr.8795 v2:172.21.15.175:6834/1518538177 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f9934071620 con 0x7f9910076bb0 2024-07-17T08:39:54.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:54.168 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:39:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.165+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 msgr2=0x7f9910079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.165+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 0x7f9910079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009870 comp rx=0 tx=0).stop 2024-07-17T08:39:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.165+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 msgr2=0x7f99340982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.165+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f992c02b7d0 tx=0x7f992c0038a0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 shutdown_connections 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f9910076bb0 0x7f9910079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f993415a040 0x7f99340982f0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9934153a50 0x7f9934159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 --2- 172.21.15.44:0/3270740015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99340acaf0 0x7f9934072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 >> 172.21.15.44:0/3270740015 conn(0x7f99340b0fb0 msgr2=0x7f99340aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:54.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 shutdown_connections 2024-07-17T08:39:54.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.166+0000 7f993b400640 1 -- 172.21.15.44:0/3270740015 wait complete. 2024-07-17T08:39:54.188 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:39:52.961386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":490,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4635,"ondisk_log_size":4635,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":468524,"kb_used_data":30256,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754506196,"statfs":{"total":773094113280,"available":772614344704,"internally_reserved":0,"allocated":30982144,"data_stored":17532876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.831037"},"pg_stats":[{"pgid":"40.3f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751161+0000","last_change":"2024-07-17T08:39:49.210184+0000","last_active":"2024-07-17T08:39:50.751161+0000","last_peered":"2024-07-17T08:39:50.751161+0000","last_clean":"2024-07-17T08:39:50.751161+0000","last_became_active":"2024-07-17T08:39:49.209173+0000","last_became_peered":"2024-07-17T08:39:49.209173+0000","last_unstale":"2024-07-17T08:39:50.751161+0000","last_undegraded":"2024-07-17T08:39:50.751161+0000","last_fullsized":"2024-07-17T08:39:50.751161+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:57.075030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.3e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749350+0000","last_change":"2024-07-17T08:39:49.206209+0000","last_active":"2024-07-17T08:39:50.749350+0000","last_peered":"2024-07-17T08:39:50.749350+0000","last_clean":"2024-07-17T08:39:50.749350+0000","last_became_active":"2024-07-17T08:39:49.205939+0000","last_became_peered":"2024-07-17T08:39:49.205939+0000","last_unstale":"2024-07-17T08:39:50.749350+0000","last_undegraded":"2024-07-17T08:39:50.749350+0000","last_fullsized":"2024-07-17T08:39:50.749350+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:17.701023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.3d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192415+0000","last_change":"2024-07-17T08:39:49.201095+0000","last_active":"2024-07-17T08:39:50.192415+0000","last_peered":"2024-07-17T08:39:50.192415+0000","last_clean":"2024-07-17T08:39:50.192415+0000","last_became_active":"2024-07-17T08:39:49.200755+0000","last_became_peered":"2024-07-17T08:39:49.200755+0000","last_unstale":"2024-07-17T08:39:50.192415+0000","last_undegraded":"2024-07-17T08:39:50.192415+0000","last_fullsized":"2024-07-17T08:39:50.192415+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:38:40.850194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.3c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186922+0000","last_change":"2024-07-17T08:39:49.196806+0000","last_active":"2024-07-17T08:39:50.186922+0000","last_peered":"2024-07-17T08:39:50.186922+0000","last_clean":"2024-07-17T08:39:50.186922+0000","last_became_active":"2024-07-17T08:39:49.196472+0000","last_became_peered":"2024-07-17T08:39:49.196472+0000","last_unstale":"2024-07-17T08:39:50.186922+0000","last_undegraded":"2024-07-17T08:39:50.186922+0000","last_fullsized":"2024-07-17T08:39:50.186922+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:42:52.560875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.3b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747536+0000","last_change":"2024-07-17T08:39:49.196707+0000","last_active":"2024-07-17T08:39:50.747536+0000","last_peered":"2024-07-17T08:39:50.747536+0000","last_clean":"2024-07-17T08:39:50.747536+0000","last_became_active":"2024-07-17T08:39:49.196386+0000","last_became_peered":"2024-07-17T08:39:49.196386+0000","last_unstale":"2024-07-17T08:39:50.747536+0000","last_undegraded":"2024-07-17T08:39:50.747536+0000","last_fullsized":"2024-07-17T08:39:50.747536+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:29:03.443911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205875+0000","last_change":"2024-07-17T08:39:49.189067+0000","last_active":"2024-07-17T08:39:50.205875+0000","last_peered":"2024-07-17T08:39:50.205875+0000","last_clean":"2024-07-17T08:39:50.205875+0000","last_became_active":"2024-07-17T08:39:49.187660+0000","last_became_peered":"2024-07-17T08:39:49.187660+0000","last_unstale":"2024-07-17T08:39:50.205875+0000","last_undegraded":"2024-07-17T08:39:50.205875+0000","last_fullsized":"2024-07-17T08:39:50.205875+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:17.564331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.39","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205091+0000","last_change":"2024-07-17T08:39:49.202104+0000","last_active":"2024-07-17T08:39:50.205091+0000","last_peered":"2024-07-17T08:39:50.205091+0000","last_clean":"2024-07-17T08:39:50.205091+0000","last_became_active":"2024-07-17T08:39:49.201658+0000","last_became_peered":"2024-07-17T08:39:49.201658+0000","last_unstale":"2024-07-17T08:39:50.205091+0000","last_undegraded":"2024-07-17T08:39:50.205091+0000","last_fullsized":"2024-07-17T08:39:50.205091+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:46.199352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.38","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745462+0000","last_change":"2024-07-17T08:39:49.197656+0000","last_active":"2024-07-17T08:39:50.745462+0000","last_peered":"2024-07-17T08:39:50.745462+0000","last_clean":"2024-07-17T08:39:50.745462+0000","last_became_active":"2024-07-17T08:39:49.197162+0000","last_became_peered":"2024-07-17T08:39:49.197162+0000","last_unstale":"2024-07-17T08:39:50.745462+0000","last_undegraded":"2024-07-17T08:39:50.745462+0000","last_fullsized":"2024-07-17T08:39:50.745462+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:01.542812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.37","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747624+0000","last_change":"2024-07-17T08:39:49.204101+0000","last_active":"2024-07-17T08:39:50.747624+0000","last_peered":"2024-07-17T08:39:50.747624+0000","last_clean":"2024-07-17T08:39:50.747624+0000","last_became_active":"2024-07-17T08:39:49.203758+0000","last_became_peered":"2024-07-17T08:39:49.203758+0000","last_unstale":"2024-07-17T08:39:50.747624+0000","last_undegraded":"2024-07-17T08:39:50.747624+0000","last_fullsized":"2024-07-17T08:39:50.747624+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:44:30.948595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.36","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192075+0000","last_change":"2024-07-17T08:39:49.197205+0000","last_active":"2024-07-17T08:39:50.192075+0000","last_peered":"2024-07-17T08:39:50.192075+0000","last_clean":"2024-07-17T08:39:50.192075+0000","last_became_active":"2024-07-17T08:39:49.196869+0000","last_became_peered":"2024-07-17T08:39:49.196869+0000","last_unstale":"2024-07-17T08:39:50.192075+0000","last_undegraded":"2024-07-17T08:39:50.192075+0000","last_fullsized":"2024-07-17T08:39:50.192075+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:01.763452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.35","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.206834+0000","last_change":"2024-07-17T08:39:49.194824+0000","last_active":"2024-07-17T08:39:50.206834+0000","last_peered":"2024-07-17T08:39:50.206834+0000","last_clean":"2024-07-17T08:39:50.206834+0000","last_became_active":"2024-07-17T08:39:49.194511+0000","last_became_peered":"2024-07-17T08:39:49.194511+0000","last_unstale":"2024-07-17T08:39:50.206834+0000","last_undegraded":"2024-07-17T08:39:50.206834+0000","last_fullsized":"2024-07-17T08:39:50.206834+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:56:02.757637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.34","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750164+0000","last_change":"2024-07-17T08:39:49.209397+0000","last_active":"2024-07-17T08:39:50.750164+0000","last_peered":"2024-07-17T08:39:50.750164+0000","last_clean":"2024-07-17T08:39:50.750164+0000","last_became_active":"2024-07-17T08:39:49.209102+0000","last_became_peered":"2024-07-17T08:39:49.209102+0000","last_unstale":"2024-07-17T08:39:50.750164+0000","last_undegraded":"2024-07-17T08:39:50.750164+0000","last_fullsized":"2024-07-17T08:39:50.750164+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:51.611046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.33","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186382+0000","last_change":"2024-07-17T08:39:49.196339+0000","last_active":"2024-07-17T08:39:50.186382+0000","last_peered":"2024-07-17T08:39:50.186382+0000","last_clean":"2024-07-17T08:39:50.186382+0000","last_became_active":"2024-07-17T08:39:49.194882+0000","last_became_peered":"2024-07-17T08:39:49.194882+0000","last_unstale":"2024-07-17T08:39:50.186382+0000","last_undegraded":"2024-07-17T08:39:50.186382+0000","last_fullsized":"2024-07-17T08:39:50.186382+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:41.631295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.32","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746125+0000","last_change":"2024-07-17T08:39:49.203741+0000","last_active":"2024-07-17T08:39:50.746125+0000","last_peered":"2024-07-17T08:39:50.746125+0000","last_clean":"2024-07-17T08:39:50.746125+0000","last_became_active":"2024-07-17T08:39:49.200081+0000","last_became_peered":"2024-07-17T08:39:49.200081+0000","last_unstale":"2024-07-17T08:39:50.746125+0000","last_undegraded":"2024-07-17T08:39:50.746125+0000","last_fullsized":"2024-07-17T08:39:50.746125+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:59:26.807248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.31","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.206412+0000","last_change":"2024-07-17T08:39:49.181430+0000","last_active":"2024-07-17T08:39:50.206412+0000","last_peered":"2024-07-17T08:39:50.206412+0000","last_clean":"2024-07-17T08:39:50.206412+0000","last_became_active":"2024-07-17T08:39:49.180103+0000","last_became_peered":"2024-07-17T08:39:49.180103+0000","last_unstale":"2024-07-17T08:39:50.206412+0000","last_undegraded":"2024-07-17T08:39:50.206412+0000","last_fullsized":"2024-07-17T08:39:50.206412+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:16.836012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.30","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189558+0000","last_change":"2024-07-17T08:39:49.206233+0000","last_active":"2024-07-17T08:39:50.189558+0000","last_peered":"2024-07-17T08:39:50.189558+0000","last_clean":"2024-07-17T08:39:50.189558+0000","last_became_active":"2024-07-17T08:39:49.205157+0000","last_became_peered":"2024-07-17T08:39:49.205157+0000","last_unstale":"2024-07-17T08:39:50.189558+0000","last_undegraded":"2024-07-17T08:39:50.189558+0000","last_fullsized":"2024-07-17T08:39:50.189558+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:31:13.393851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.2f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748289+0000","last_change":"2024-07-17T08:39:49.203688+0000","last_active":"2024-07-17T08:39:50.748289+0000","last_peered":"2024-07-17T08:39:50.748289+0000","last_clean":"2024-07-17T08:39:50.748289+0000","last_became_active":"2024-07-17T08:39:49.203366+0000","last_became_peered":"2024-07-17T08:39:49.203366+0000","last_unstale":"2024-07-17T08:39:50.748289+0000","last_undegraded":"2024-07-17T08:39:50.748289+0000","last_fullsized":"2024-07-17T08:39:50.748289+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:26:28.711680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192266+0000","last_change":"2024-07-17T08:39:49.183932+0000","last_active":"2024-07-17T08:39:50.192266+0000","last_peered":"2024-07-17T08:39:50.192266+0000","last_clean":"2024-07-17T08:39:50.192266+0000","last_became_active":"2024-07-17T08:39:49.183324+0000","last_became_peered":"2024-07-17T08:39:49.183324+0000","last_unstale":"2024-07-17T08:39:50.192266+0000","last_undegraded":"2024-07-17T08:39:50.192266+0000","last_fullsized":"2024-07-17T08:39:50.192266+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:31.995488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.2d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192602+0000","last_change":"2024-07-17T08:39:49.205250+0000","last_active":"2024-07-17T08:39:50.192602+0000","last_peered":"2024-07-17T08:39:50.192602+0000","last_clean":"2024-07-17T08:39:50.192602+0000","last_became_active":"2024-07-17T08:39:49.204061+0000","last_became_peered":"2024-07-17T08:39:49.204061+0000","last_unstale":"2024-07-17T08:39:50.192602+0000","last_undegraded":"2024-07-17T08:39:50.192602+0000","last_fullsized":"2024-07-17T08:39:50.192602+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:09.809269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.2c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205768+0000","last_change":"2024-07-17T08:39:49.190402+0000","last_active":"2024-07-17T08:39:50.205768+0000","last_peered":"2024-07-17T08:39:50.205768+0000","last_clean":"2024-07-17T08:39:50.205768+0000","last_became_active":"2024-07-17T08:39:49.188205+0000","last_became_peered":"2024-07-17T08:39:49.188205+0000","last_unstale":"2024-07-17T08:39:50.205768+0000","last_undegraded":"2024-07-17T08:39:50.205768+0000","last_fullsized":"2024-07-17T08:39:50.205768+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:10.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747010+0000","last_change":"2024-07-17T08:39:49.213098+0000","last_active":"2024-07-17T08:39:50.747010+0000","last_peered":"2024-07-17T08:39:50.747010+0000","last_clean":"2024-07-17T08:39:50.747010+0000","last_became_active":"2024-07-17T08:39:49.205645+0000","last_became_peered":"2024-07-17T08:39:49.205645+0000","last_unstale":"2024-07-17T08:39:50.747010+0000","last_undegraded":"2024-07-17T08:39:50.747010+0000","last_fullsized":"2024-07-17T08:39:50.747010+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:20.532920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.2a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745462+0000","last_change":"2024-07-17T08:39:49.200730+0000","last_active":"2024-07-17T08:39:50.745462+0000","last_peered":"2024-07-17T08:39:50.745462+0000","last_clean":"2024-07-17T08:39:50.745462+0000","last_became_active":"2024-07-17T08:39:49.199251+0000","last_became_peered":"2024-07-17T08:39:49.199251+0000","last_unstale":"2024-07-17T08:39:50.745462+0000","last_undegraded":"2024-07-17T08:39:50.745462+0000","last_fullsized":"2024-07-17T08:39:50.745462+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:19:39.657107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.28","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747509+0000","last_change":"2024-07-17T08:39:49.204724+0000","last_active":"2024-07-17T08:39:50.747509+0000","last_peered":"2024-07-17T08:39:50.747509+0000","last_clean":"2024-07-17T08:39:50.747509+0000","last_became_active":"2024-07-17T08:39:49.204382+0000","last_became_peered":"2024-07-17T08:39:49.204382+0000","last_unstale":"2024-07-17T08:39:50.747509+0000","last_undegraded":"2024-07-17T08:39:50.747509+0000","last_fullsized":"2024-07-17T08:39:50.747509+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:31:40.398161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.27","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748271+0000","last_change":"2024-07-17T08:39:49.196244+0000","last_active":"2024-07-17T08:39:50.748271+0000","last_peered":"2024-07-17T08:39:50.748271+0000","last_clean":"2024-07-17T08:39:50.748271+0000","last_became_active":"2024-07-17T08:39:49.192099+0000","last_became_peered":"2024-07-17T08:39:49.192099+0000","last_unstale":"2024-07-17T08:39:50.748271+0000","last_undegraded":"2024-07-17T08:39:50.748271+0000","last_fullsized":"2024-07-17T08:39:50.748271+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:23:36.119712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.26","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750504+0000","last_change":"2024-07-17T08:39:49.206225+0000","last_active":"2024-07-17T08:39:50.750504+0000","last_peered":"2024-07-17T08:39:50.750504+0000","last_clean":"2024-07-17T08:39:50.750504+0000","last_became_active":"2024-07-17T08:39:49.204799+0000","last_became_peered":"2024-07-17T08:39:49.204799+0000","last_unstale":"2024-07-17T08:39:50.750504+0000","last_undegraded":"2024-07-17T08:39:50.750504+0000","last_fullsized":"2024-07-17T08:39:50.750504+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:17.454112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.25","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192650+0000","last_change":"2024-07-17T08:39:49.197823+0000","last_active":"2024-07-17T08:39:50.192650+0000","last_peered":"2024-07-17T08:39:50.192650+0000","last_clean":"2024-07-17T08:39:50.192650+0000","last_became_active":"2024-07-17T08:39:49.196073+0000","last_became_peered":"2024-07-17T08:39:49.196073+0000","last_unstale":"2024-07-17T08:39:50.192650+0000","last_undegraded":"2024-07-17T08:39:50.192650+0000","last_fullsized":"2024-07-17T08:39:50.192650+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:17.942979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.24","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748373+0000","last_change":"2024-07-17T08:39:49.184470+0000","last_active":"2024-07-17T08:39:50.748373+0000","last_peered":"2024-07-17T08:39:50.748373+0000","last_clean":"2024-07-17T08:39:50.748373+0000","last_became_active":"2024-07-17T08:39:49.183734+0000","last_became_peered":"2024-07-17T08:39:49.183734+0000","last_unstale":"2024-07-17T08:39:50.748373+0000","last_undegraded":"2024-07-17T08:39:50.748373+0000","last_fullsized":"2024-07-17T08:39:50.748373+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:23.219278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.23","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747986+0000","last_change":"2024-07-17T08:39:49.192970+0000","last_active":"2024-07-17T08:39:50.747986+0000","last_peered":"2024-07-17T08:39:50.747986+0000","last_clean":"2024-07-17T08:39:50.747986+0000","last_became_active":"2024-07-17T08:39:49.192683+0000","last_became_peered":"2024-07-17T08:39:49.192683+0000","last_unstale":"2024-07-17T08:39:50.747986+0000","last_undegraded":"2024-07-17T08:39:50.747986+0000","last_fullsized":"2024-07-17T08:39:50.747986+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:54.904157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.22","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746188+0000","last_change":"2024-07-17T08:39:49.200039+0000","last_active":"2024-07-17T08:39:50.746188+0000","last_peered":"2024-07-17T08:39:50.746188+0000","last_clean":"2024-07-17T08:39:50.746188+0000","last_became_active":"2024-07-17T08:39:49.199749+0000","last_became_peered":"2024-07-17T08:39:49.199749+0000","last_unstale":"2024-07-17T08:39:50.746188+0000","last_undegraded":"2024-07-17T08:39:50.746188+0000","last_fullsized":"2024-07-17T08:39:50.746188+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:03:20.010704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.21","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745968+0000","last_change":"2024-07-17T08:39:49.213304+0000","last_active":"2024-07-17T08:39:50.745968+0000","last_peered":"2024-07-17T08:39:50.745968+0000","last_clean":"2024-07-17T08:39:50.745968+0000","last_became_active":"2024-07-17T08:39:49.205101+0000","last_became_peered":"2024-07-17T08:39:49.205101+0000","last_unstale":"2024-07-17T08:39:50.745968+0000","last_undegraded":"2024-07-17T08:39:50.745968+0000","last_fullsized":"2024-07-17T08:39:50.745968+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:30:07.937731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.20","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189194+0000","last_change":"2024-07-17T08:39:49.193733+0000","last_active":"2024-07-17T08:39:50.189194+0000","last_peered":"2024-07-17T08:39:50.189194+0000","last_clean":"2024-07-17T08:39:50.189194+0000","last_became_active":"2024-07-17T08:39:49.191631+0000","last_became_peered":"2024-07-17T08:39:49.191631+0000","last_unstale":"2024-07-17T08:39:50.189194+0000","last_undegraded":"2024-07-17T08:39:50.189194+0000","last_fullsized":"2024-07-17T08:39:50.189194+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:18.965583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.10","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.204891+0000","last_change":"2024-07-17T08:39:49.201691+0000","last_active":"2024-07-17T08:39:50.204891+0000","last_peered":"2024-07-17T08:39:50.204891+0000","last_clean":"2024-07-17T08:39:50.204891+0000","last_became_active":"2024-07-17T08:39:49.196734+0000","last_became_peered":"2024-07-17T08:39:49.196734+0000","last_unstale":"2024-07-17T08:39:50.204891+0000","last_undegraded":"2024-07-17T08:39:50.204891+0000","last_fullsized":"2024-07-17T08:39:50.204891+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:56.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.1f","version":"292'4","reported_seq":18,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.187263+0000","last_change":"2024-07-17T08:39:48.165633+0000","last_active":"2024-07-17T08:39:50.187263+0000","last_peered":"2024-07-17T08:39:50.187263+0000","last_clean":"2024-07-17T08:39:50.187263+0000","last_became_active":"2024-07-17T08:39:48.165317+0000","last_became_peered":"2024-07-17T08:39:48.165317+0000","last_unstale":"2024-07-17T08:39:50.187263+0000","last_undegraded":"2024-07-17T08:39:50.187263+0000","last_fullsized":"2024-07-17T08:39:50.187263+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:55:11.776255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.11","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186482+0000","last_change":"2024-07-17T08:39:49.203969+0000","last_active":"2024-07-17T08:39:50.186482+0000","last_peered":"2024-07-17T08:39:50.186482+0000","last_clean":"2024-07-17T08:39:50.186482+0000","last_became_active":"2024-07-17T08:39:49.203689+0000","last_became_peered":"2024-07-17T08:39:49.203689+0000","last_unstale":"2024-07-17T08:39:50.186482+0000","last_undegraded":"2024-07-17T08:39:50.186482+0000","last_fullsized":"2024-07-17T08:39:50.186482+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:12.150998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.1e","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193497+0000","last_change":"2024-07-17T08:39:48.162361+0000","last_active":"2024-07-17T08:39:50.193497+0000","last_peered":"2024-07-17T08:39:50.193497+0000","last_clean":"2024-07-17T08:39:50.193497+0000","last_became_active":"2024-07-17T08:39:48.161265+0000","last_became_peered":"2024-07-17T08:39:48.161265+0000","last_unstale":"2024-07-17T08:39:50.193497+0000","last_undegraded":"2024-07-17T08:39:50.193497+0000","last_fullsized":"2024-07-17T08:39:50.193497+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:56.228522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.12","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746835+0000","last_change":"2024-07-17T08:39:49.207688+0000","last_active":"2024-07-17T08:39:50.746835+0000","last_peered":"2024-07-17T08:39:50.746835+0000","last_clean":"2024-07-17T08:39:50.746835+0000","last_became_active":"2024-07-17T08:39:49.204861+0000","last_became_peered":"2024-07-17T08:39:49.204861+0000","last_unstale":"2024-07-17T08:39:50.746835+0000","last_undegraded":"2024-07-17T08:39:50.746835+0000","last_fullsized":"2024-07-17T08:39:50.746835+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:22:14.756330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.1d","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191840+0000","last_change":"2024-07-17T08:39:48.161567+0000","last_active":"2024-07-17T08:39:50.191840+0000","last_peered":"2024-07-17T08:39:50.191840+0000","last_clean":"2024-07-17T08:39:50.191840+0000","last_became_active":"2024-07-17T08:39:48.161292+0000","last_became_peered":"2024-07-17T08:39:48.161292+0000","last_unstale":"2024-07-17T08:39:50.191840+0000","last_undegraded":"2024-07-17T08:39:50.191840+0000","last_fullsized":"2024-07-17T08:39:50.191840+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:34.850760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.13","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748421+0000","last_change":"2024-07-17T08:39:49.185040+0000","last_active":"2024-07-17T08:39:50.748421+0000","last_peered":"2024-07-17T08:39:50.748421+0000","last_clean":"2024-07-17T08:39:50.748421+0000","last_became_active":"2024-07-17T08:39:49.184366+0000","last_became_peered":"2024-07-17T08:39:49.184366+0000","last_unstale":"2024-07-17T08:39:50.748421+0000","last_undegraded":"2024-07-17T08:39:50.748421+0000","last_fullsized":"2024-07-17T08:39:50.748421+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:23.719237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.1c","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747513+0000","last_change":"2024-07-17T08:39:48.166086+0000","last_active":"2024-07-17T08:39:50.747513+0000","last_peered":"2024-07-17T08:39:50.747513+0000","last_clean":"2024-07-17T08:39:50.747513+0000","last_became_active":"2024-07-17T08:39:48.165790+0000","last_became_peered":"2024-07-17T08:39:48.165790+0000","last_unstale":"2024-07-17T08:39:50.747513+0000","last_undegraded":"2024-07-17T08:39:50.747513+0000","last_fullsized":"2024-07-17T08:39:50.747513+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:44:45.189150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.14","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190925+0000","last_change":"2024-07-17T08:39:49.200346+0000","last_active":"2024-07-17T08:39:50.190925+0000","last_peered":"2024-07-17T08:39:50.190925+0000","last_clean":"2024-07-17T08:39:50.190925+0000","last_became_active":"2024-07-17T08:39:49.200055+0000","last_became_peered":"2024-07-17T08:39:49.200055+0000","last_unstale":"2024-07-17T08:39:50.190925+0000","last_undegraded":"2024-07-17T08:39:50.190925+0000","last_fullsized":"2024-07-17T08:39:50.190925+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:41:17.100733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.1b","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748733+0000","last_change":"2024-07-17T08:39:48.171457+0000","last_active":"2024-07-17T08:39:50.748733+0000","last_peered":"2024-07-17T08:39:50.748733+0000","last_clean":"2024-07-17T08:39:50.748733+0000","last_became_active":"2024-07-17T08:39:48.171135+0000","last_became_peered":"2024-07-17T08:39:48.171135+0000","last_unstale":"2024-07-17T08:39:50.748733+0000","last_undegraded":"2024-07-17T08:39:50.748733+0000","last_fullsized":"2024-07-17T08:39:50.748733+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:48.924683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.15","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750331+0000","last_change":"2024-07-17T08:39:49.206120+0000","last_active":"2024-07-17T08:39:50.750331+0000","last_peered":"2024-07-17T08:39:50.750331+0000","last_clean":"2024-07-17T08:39:50.750331+0000","last_became_active":"2024-07-17T08:39:49.205405+0000","last_became_peered":"2024-07-17T08:39:49.205405+0000","last_unstale":"2024-07-17T08:39:50.750331+0000","last_undegraded":"2024-07-17T08:39:50.750331+0000","last_fullsized":"2024-07-17T08:39:50.750331+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:33.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.1a","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205176+0000","last_change":"2024-07-17T08:39:48.171134+0000","last_active":"2024-07-17T08:39:50.205176+0000","last_peered":"2024-07-17T08:39:50.205176+0000","last_clean":"2024-07-17T08:39:50.205176+0000","last_became_active":"2024-07-17T08:39:48.170586+0000","last_became_peered":"2024-07-17T08:39:48.170586+0000","last_unstale":"2024-07-17T08:39:50.205176+0000","last_undegraded":"2024-07-17T08:39:50.205176+0000","last_fullsized":"2024-07-17T08:39:50.205176+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:46.204589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.16","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751076+0000","last_change":"2024-07-17T08:39:49.204903+0000","last_active":"2024-07-17T08:39:50.751076+0000","last_peered":"2024-07-17T08:39:50.751076+0000","last_clean":"2024-07-17T08:39:50.751076+0000","last_became_active":"2024-07-17T08:39:49.204565+0000","last_became_peered":"2024-07-17T08:39:49.204565+0000","last_unstale":"2024-07-17T08:39:50.751076+0000","last_undegraded":"2024-07-17T08:39:50.751076+0000","last_fullsized":"2024-07-17T08:39:50.751076+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:32.556141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.19","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189168+0000","last_change":"2024-07-17T08:39:48.167801+0000","last_active":"2024-07-17T08:39:50.189168+0000","last_peered":"2024-07-17T08:39:50.189168+0000","last_clean":"2024-07-17T08:39:50.189168+0000","last_became_active":"2024-07-17T08:39:48.167501+0000","last_became_peered":"2024-07-17T08:39:48.167501+0000","last_unstale":"2024-07-17T08:39:50.189168+0000","last_undegraded":"2024-07-17T08:39:50.189168+0000","last_fullsized":"2024-07-17T08:39:50.189168+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:53:54.002002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.17","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193022+0000","last_change":"2024-07-17T08:39:49.195083+0000","last_active":"2024-07-17T08:39:50.193022+0000","last_peered":"2024-07-17T08:39:50.193022+0000","last_clean":"2024-07-17T08:39:50.193022+0000","last_became_active":"2024-07-17T08:39:49.194800+0000","last_became_peered":"2024-07-17T08:39:49.194800+0000","last_unstale":"2024-07-17T08:39:50.193022+0000","last_undegraded":"2024-07-17T08:39:50.193022+0000","last_fullsized":"2024-07-17T08:39:50.193022+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:31.750567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.18","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745771+0000","last_change":"2024-07-17T08:39:48.164429+0000","last_active":"2024-07-17T08:39:50.745771+0000","last_peered":"2024-07-17T08:39:50.745771+0000","last_clean":"2024-07-17T08:39:50.745771+0000","last_became_active":"2024-07-17T08:39:48.163887+0000","last_became_peered":"2024-07-17T08:39:48.163887+0000","last_unstale":"2024-07-17T08:39:50.745771+0000","last_undegraded":"2024-07-17T08:39:50.745771+0000","last_fullsized":"2024-07-17T08:39:50.745771+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:20.004633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.18","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745943+0000","last_change":"2024-07-17T08:39:49.205703+0000","last_active":"2024-07-17T08:39:50.745943+0000","last_peered":"2024-07-17T08:39:50.745943+0000","last_clean":"2024-07-17T08:39:50.745943+0000","last_became_active":"2024-07-17T08:39:49.205390+0000","last_became_peered":"2024-07-17T08:39:49.205390+0000","last_unstale":"2024-07-17T08:39:50.745943+0000","last_undegraded":"2024-07-17T08:39:50.745943+0000","last_fullsized":"2024-07-17T08:39:50.745943+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:40:35.218069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.17","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748703+0000","last_change":"2024-07-17T08:39:48.172031+0000","last_active":"2024-07-17T08:39:50.748703+0000","last_peered":"2024-07-17T08:39:50.748703+0000","last_clean":"2024-07-17T08:39:50.748703+0000","last_became_active":"2024-07-17T08:39:48.171580+0000","last_became_peered":"2024-07-17T08:39:48.171580+0000","last_unstale":"2024-07-17T08:39:50.748703+0000","last_undegraded":"2024-07-17T08:39:50.748703+0000","last_fullsized":"2024-07-17T08:39:50.748703+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:39:56.064828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.19","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205920+0000","last_change":"2024-07-17T08:39:49.196620+0000","last_active":"2024-07-17T08:39:50.205920+0000","last_peered":"2024-07-17T08:39:50.205920+0000","last_clean":"2024-07-17T08:39:50.205920+0000","last_became_active":"2024-07-17T08:39:49.195232+0000","last_became_peered":"2024-07-17T08:39:49.195232+0000","last_unstale":"2024-07-17T08:39:50.205920+0000","last_undegraded":"2024-07-17T08:39:50.205920+0000","last_fullsized":"2024-07-17T08:39:50.205920+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:34:23.362643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.16","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186943+0000","last_change":"2024-07-17T08:39:48.165774+0000","last_active":"2024-07-17T08:39:50.186943+0000","last_peered":"2024-07-17T08:39:50.186943+0000","last_clean":"2024-07-17T08:39:50.186943+0000","last_became_active":"2024-07-17T08:39:48.165270+0000","last_became_peered":"2024-07-17T08:39:48.165270+0000","last_unstale":"2024-07-17T08:39:50.186943+0000","last_undegraded":"2024-07-17T08:39:50.186943+0000","last_fullsized":"2024-07-17T08:39:50.186943+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:01.710783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.1a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749753+0000","last_change":"2024-07-17T08:39:49.207881+0000","last_active":"2024-07-17T08:39:50.749753+0000","last_peered":"2024-07-17T08:39:50.749753+0000","last_clean":"2024-07-17T08:39:50.749753+0000","last_became_active":"2024-07-17T08:39:49.207607+0000","last_became_peered":"2024-07-17T08:39:49.207607+0000","last_unstale":"2024-07-17T08:39:50.749753+0000","last_undegraded":"2024-07-17T08:39:50.749753+0000","last_fullsized":"2024-07-17T08:39:50.749753+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:27:16.942577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.15","version":"292'3","reported_seq":17,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193007+0000","last_change":"2024-07-17T08:39:48.173854+0000","last_active":"2024-07-17T08:39:50.193007+0000","last_peered":"2024-07-17T08:39:50.193007+0000","last_clean":"2024-07-17T08:39:50.193007+0000","last_became_active":"2024-07-17T08:39:48.173346+0000","last_became_peered":"2024-07-17T08:39:48.173346+0000","last_unstale":"2024-07-17T08:39:50.193007+0000","last_undegraded":"2024-07-17T08:39:50.193007+0000","last_fullsized":"2024-07-17T08:39:50.193007+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:14.390430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.1b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191442+0000","last_change":"2024-07-17T08:39:49.204963+0000","last_active":"2024-07-17T08:39:50.191442+0000","last_peered":"2024-07-17T08:39:50.191442+0000","last_clean":"2024-07-17T08:39:50.191442+0000","last_became_active":"2024-07-17T08:39:49.204614+0000","last_became_peered":"2024-07-17T08:39:49.204614+0000","last_unstale":"2024-07-17T08:39:50.191442+0000","last_undegraded":"2024-07-17T08:39:50.191442+0000","last_fullsized":"2024-07-17T08:39:50.191442+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:16.887020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.14","version":"292'6","reported_seq":20,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750074+0000","last_change":"2024-07-17T08:39:48.166353+0000","last_active":"2024-07-17T08:39:50.750074+0000","last_peered":"2024-07-17T08:39:50.750074+0000","last_clean":"2024-07-17T08:39:50.750074+0000","last_became_active":"2024-07-17T08:39:48.163682+0000","last_became_peered":"2024-07-17T08:39:48.163682+0000","last_unstale":"2024-07-17T08:39:50.750074+0000","last_undegraded":"2024-07-17T08:39:50.750074+0000","last_fullsized":"2024-07-17T08:39:50.750074+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:18.953888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745646+0000","last_change":"2024-07-17T08:39:49.204829+0000","last_active":"2024-07-17T08:39:50.745646+0000","last_peered":"2024-07-17T08:39:50.745646+0000","last_clean":"2024-07-17T08:39:50.745646+0000","last_became_active":"2024-07-17T08:39:49.204499+0000","last_became_peered":"2024-07-17T08:39:49.204499+0000","last_unstale":"2024-07-17T08:39:50.745646+0000","last_undegraded":"2024-07-17T08:39:50.745646+0000","last_fullsized":"2024-07-17T08:39:50.745646+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:59:08.773614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"39.13","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746069+0000","last_change":"2024-07-17T08:39:48.174660+0000","last_active":"2024-07-17T08:39:50.746069+0000","last_peered":"2024-07-17T08:39:50.746069+0000","last_clean":"2024-07-17T08:39:50.746069+0000","last_became_active":"2024-07-17T08:39:48.174361+0000","last_became_peered":"2024-07-17T08:39:48.174361+0000","last_unstale":"2024-07-17T08:39:50.746069+0000","last_undegraded":"2024-07-17T08:39:50.746069+0000","last_fullsized":"2024-07-17T08:39:50.746069+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:01:34.979710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.1d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193155+0000","last_change":"2024-07-17T08:39:49.202811+0000","last_active":"2024-07-17T08:39:50.193155+0000","last_peered":"2024-07-17T08:39:50.193155+0000","last_clean":"2024-07-17T08:39:50.193155+0000","last_became_active":"2024-07-17T08:39:49.202534+0000","last_became_peered":"2024-07-17T08:39:49.202534+0000","last_unstale":"2024-07-17T08:39:50.193155+0000","last_undegraded":"2024-07-17T08:39:50.193155+0000","last_fullsized":"2024-07-17T08:39:50.193155+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:45:03.313106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.12","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749489+0000","last_change":"2024-07-17T08:39:48.173852+0000","last_active":"2024-07-17T08:39:50.749489+0000","last_peered":"2024-07-17T08:39:50.749489+0000","last_clean":"2024-07-17T08:39:50.749489+0000","last_became_active":"2024-07-17T08:39:48.173429+0000","last_became_peered":"2024-07-17T08:39:48.173429+0000","last_unstale":"2024-07-17T08:39:50.749489+0000","last_undegraded":"2024-07-17T08:39:50.749489+0000","last_fullsized":"2024-07-17T08:39:50.749489+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:34.317499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192477+0000","last_change":"2024-07-17T08:39:49.184195+0000","last_active":"2024-07-17T08:39:50.192477+0000","last_peered":"2024-07-17T08:39:50.192477+0000","last_clean":"2024-07-17T08:39:50.192477+0000","last_became_active":"2024-07-17T08:39:49.183551+0000","last_became_peered":"2024-07-17T08:39:49.183551+0000","last_unstale":"2024-07-17T08:39:50.192477+0000","last_undegraded":"2024-07-17T08:39:50.192477+0000","last_fullsized":"2024-07-17T08:39:50.192477+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:22.025251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.11","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745874+0000","last_change":"2024-07-17T08:39:48.172319+0000","last_active":"2024-07-17T08:39:50.745874+0000","last_peered":"2024-07-17T08:39:50.745874+0000","last_clean":"2024-07-17T08:39:50.745874+0000","last_became_active":"2024-07-17T08:39:48.171643+0000","last_became_peered":"2024-07-17T08:39:48.171643+0000","last_unstale":"2024-07-17T08:39:50.745874+0000","last_undegraded":"2024-07-17T08:39:50.745874+0000","last_fullsized":"2024-07-17T08:39:50.745874+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:50.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.1f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751087+0000","last_change":"2024-07-17T08:39:49.209453+0000","last_active":"2024-07-17T08:39:50.751087+0000","last_peered":"2024-07-17T08:39:50.751087+0000","last_clean":"2024-07-17T08:39:50.751087+0000","last_became_active":"2024-07-17T08:39:49.208731+0000","last_became_peered":"2024-07-17T08:39:49.208731+0000","last_unstale":"2024-07-17T08:39:50.751087+0000","last_undegraded":"2024-07-17T08:39:50.751087+0000","last_fullsized":"2024-07-17T08:39:50.751087+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:27.449975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.10","version":"292'8","reported_seq":22,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191835+0000","last_change":"2024-07-17T08:39:48.173488+0000","last_active":"2024-07-17T08:39:50.191835+0000","last_peered":"2024-07-17T08:39:50.191835+0000","last_clean":"2024-07-17T08:39:50.191835+0000","last_became_active":"2024-07-17T08:39:48.171757+0000","last_became_peered":"2024-07-17T08:39:48.171757+0000","last_unstale":"2024-07-17T08:39:50.191835+0000","last_undegraded":"2024-07-17T08:39:50.191835+0000","last_fullsized":"2024-07-17T08:39:50.191835+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:56.241476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.0","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745766+0000","last_change":"2024-07-17T08:39:49.207321+0000","last_active":"2024-07-17T08:39:50.745766+0000","last_peered":"2024-07-17T08:39:50.745766+0000","last_clean":"2024-07-17T08:39:50.745766+0000","last_became_active":"2024-07-17T08:39:49.203599+0000","last_became_peered":"2024-07-17T08:39:49.203599+0000","last_unstale":"2024-07-17T08:39:50.745766+0000","last_undegraded":"2024-07-17T08:39:50.745766+0000","last_fullsized":"2024-07-17T08:39:50.745766+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:31.183780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.f","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.751294+0000","last_change":"2024-07-17T08:39:48.172822+0000","last_active":"2024-07-17T08:39:50.751294+0000","last_peered":"2024-07-17T08:39:50.751294+0000","last_clean":"2024-07-17T08:39:50.751294+0000","last_became_active":"2024-07-17T08:39:48.172493+0000","last_became_peered":"2024-07-17T08:39:48.172493+0000","last_unstale":"2024-07-17T08:39:50.751294+0000","last_undegraded":"2024-07-17T08:39:50.751294+0000","last_fullsized":"2024-07-17T08:39:50.751294+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:06:03.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.1","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746379+0000","last_change":"2024-07-17T08:39:49.213791+0000","last_active":"2024-07-17T08:39:50.746379+0000","last_peered":"2024-07-17T08:39:50.746379+0000","last_clean":"2024-07-17T08:39:50.746379+0000","last_became_active":"2024-07-17T08:39:49.205472+0000","last_became_peered":"2024-07-17T08:39:49.205472+0000","last_unstale":"2024-07-17T08:39:50.746379+0000","last_undegraded":"2024-07-17T08:39:50.746379+0000","last_fullsized":"2024-07-17T08:39:50.746379+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:19:36.631906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"39.e","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747325+0000","last_change":"2024-07-17T08:39:48.170718+0000","last_active":"2024-07-17T08:39:50.747325+0000","last_peered":"2024-07-17T08:39:50.747325+0000","last_clean":"2024-07-17T08:39:50.747325+0000","last_became_active":"2024-07-17T08:39:48.170401+0000","last_became_peered":"2024-07-17T08:39:48.170401+0000","last_unstale":"2024-07-17T08:39:50.747325+0000","last_undegraded":"2024-07-17T08:39:50.747325+0000","last_fullsized":"2024-07-17T08:39:50.747325+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:22:26.439772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.2","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747304+0000","last_change":"2024-07-17T08:39:49.202315+0000","last_active":"2024-07-17T08:39:50.747304+0000","last_peered":"2024-07-17T08:39:50.747304+0000","last_clean":"2024-07-17T08:39:50.747304+0000","last_became_active":"2024-07-17T08:39:49.201687+0000","last_became_peered":"2024-07-17T08:39:49.201687+0000","last_unstale":"2024-07-17T08:39:50.747304+0000","last_undegraded":"2024-07-17T08:39:50.747304+0000","last_fullsized":"2024-07-17T08:39:50.747304+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:49:07.813566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.d","version":"292'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748844+0000","last_change":"2024-07-17T08:39:48.170778+0000","last_active":"2024-07-17T08:39:50.748844+0000","last_peered":"2024-07-17T08:39:50.748844+0000","last_clean":"2024-07-17T08:39:50.748844+0000","last_became_active":"2024-07-17T08:39:48.170153+0000","last_became_peered":"2024-07-17T08:39:48.170153+0000","last_unstale":"2024-07-17T08:39:50.748844+0000","last_undegraded":"2024-07-17T08:39:50.748844+0000","last_fullsized":"2024-07-17T08:39:50.748844+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:31.545939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.749788+0000","last_change":"2024-07-17T08:39:49.202817+0000","last_active":"2024-07-17T08:39:50.749788+0000","last_peered":"2024-07-17T08:39:50.749788+0000","last_clean":"2024-07-17T08:39:50.749788+0000","last_became_active":"2024-07-17T08:39:49.202518+0000","last_became_peered":"2024-07-17T08:39:49.202518+0000","last_unstale":"2024-07-17T08:39:50.749788+0000","last_undegraded":"2024-07-17T08:39:50.749788+0000","last_fullsized":"2024-07-17T08:39:50.749788+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:21.014835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.c","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747702+0000","last_change":"2024-07-17T08:39:48.159704+0000","last_active":"2024-07-17T08:39:50.747702+0000","last_peered":"2024-07-17T08:39:50.747702+0000","last_clean":"2024-07-17T08:39:50.747702+0000","last_became_active":"2024-07-17T08:39:48.159419+0000","last_became_peered":"2024-07-17T08:39:48.159419+0000","last_unstale":"2024-07-17T08:39:50.747702+0000","last_undegraded":"2024-07-17T08:39:50.747702+0000","last_fullsized":"2024-07-17T08:39:50.747702+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:09.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.4","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191031+0000","last_change":"2024-07-17T08:39:49.199289+0000","last_active":"2024-07-17T08:39:50.191031+0000","last_peered":"2024-07-17T08:39:50.191031+0000","last_clean":"2024-07-17T08:39:50.191031+0000","last_became_active":"2024-07-17T08:39:49.199012+0000","last_became_peered":"2024-07-17T08:39:49.199012+0000","last_unstale":"2024-07-17T08:39:50.191031+0000","last_undegraded":"2024-07-17T08:39:50.191031+0000","last_fullsized":"2024-07-17T08:39:50.191031+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:09.132629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.b","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191772+0000","last_change":"2024-07-17T08:39:48.173469+0000","last_active":"2024-07-17T08:39:50.191772+0000","last_peered":"2024-07-17T08:39:50.191772+0000","last_clean":"2024-07-17T08:39:50.191772+0000","last_became_active":"2024-07-17T08:39:48.172535+0000","last_became_peered":"2024-07-17T08:39:48.172535+0000","last_unstale":"2024-07-17T08:39:50.191772+0000","last_undegraded":"2024-07-17T08:39:50.191772+0000","last_fullsized":"2024-07-17T08:39:50.191772+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:15.105839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.5","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191538+0000","last_change":"2024-07-17T08:39:49.194931+0000","last_active":"2024-07-17T08:39:50.191538+0000","last_peered":"2024-07-17T08:39:50.191538+0000","last_clean":"2024-07-17T08:39:50.191538+0000","last_became_active":"2024-07-17T08:39:49.194618+0000","last_became_peered":"2024-07-17T08:39:49.194618+0000","last_unstale":"2024-07-17T08:39:50.191538+0000","last_undegraded":"2024-07-17T08:39:50.191538+0000","last_fullsized":"2024-07-17T08:39:50.191538+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:09:28.365958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.a","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746854+0000","last_change":"2024-07-17T08:39:48.168665+0000","last_active":"2024-07-17T08:39:50.746854+0000","last_peered":"2024-07-17T08:39:50.746854+0000","last_clean":"2024-07-17T08:39:50.746854+0000","last_became_active":"2024-07-17T08:39:48.168343+0000","last_became_peered":"2024-07-17T08:39:48.168343+0000","last_unstale":"2024-07-17T08:39:50.746854+0000","last_undegraded":"2024-07-17T08:39:50.746854+0000","last_fullsized":"2024-07-17T08:39:50.746854+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:32.892048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.6","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192128+0000","last_change":"2024-07-17T08:39:49.206859+0000","last_active":"2024-07-17T08:39:50.192128+0000","last_peered":"2024-07-17T08:39:50.192128+0000","last_clean":"2024-07-17T08:39:50.192128+0000","last_became_active":"2024-07-17T08:39:49.205573+0000","last_became_peered":"2024-07-17T08:39:49.205573+0000","last_unstale":"2024-07-17T08:39:50.192128+0000","last_undegraded":"2024-07-17T08:39:50.192128+0000","last_fullsized":"2024-07-17T08:39:50.192128+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:41:58.519570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.9","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745408+0000","last_change":"2024-07-17T08:39:48.174570+0000","last_active":"2024-07-17T08:39:50.745408+0000","last_peered":"2024-07-17T08:39:50.745408+0000","last_clean":"2024-07-17T08:39:50.745408+0000","last_became_active":"2024-07-17T08:39:48.174272+0000","last_became_peered":"2024-07-17T08:39:48.174272+0000","last_unstale":"2024-07-17T08:39:50.745408+0000","last_undegraded":"2024-07-17T08:39:50.745408+0000","last_fullsized":"2024-07-17T08:39:50.745408+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:48.470494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.7","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748712+0000","last_change":"2024-07-17T08:39:49.184120+0000","last_active":"2024-07-17T08:39:50.748712+0000","last_peered":"2024-07-17T08:39:50.748712+0000","last_clean":"2024-07-17T08:39:50.748712+0000","last_became_active":"2024-07-17T08:39:49.183336+0000","last_became_peered":"2024-07-17T08:39:49.183336+0000","last_unstale":"2024-07-17T08:39:50.748712+0000","last_undegraded":"2024-07-17T08:39:50.748712+0000","last_fullsized":"2024-07-17T08:39:50.748712+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:09.007050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.8","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.189368+0000","last_change":"2024-07-17T08:39:48.169791+0000","last_active":"2024-07-17T08:39:50.189368+0000","last_peered":"2024-07-17T08:39:50.189368+0000","last_clean":"2024-07-17T08:39:50.189368+0000","last_became_active":"2024-07-17T08:39:48.169455+0000","last_became_peered":"2024-07-17T08:39:48.169455+0000","last_unstale":"2024-07-17T08:39:50.189368+0000","last_undegraded":"2024-07-17T08:39:50.189368+0000","last_fullsized":"2024-07-17T08:39:50.189368+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:41.692113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.8","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746338+0000","last_change":"2024-07-17T08:39:49.193363+0000","last_active":"2024-07-17T08:39:50.746338+0000","last_peered":"2024-07-17T08:39:50.746338+0000","last_clean":"2024-07-17T08:39:50.746338+0000","last_became_active":"2024-07-17T08:39:49.192822+0000","last_became_peered":"2024-07-17T08:39:49.192822+0000","last_unstale":"2024-07-17T08:39:50.746338+0000","last_undegraded":"2024-07-17T08:39:50.746338+0000","last_fullsized":"2024-07-17T08:39:50.746338+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:20.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.7","version":"292'2","reported_seq":16,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.748594+0000","last_change":"2024-07-17T08:39:48.158819+0000","last_active":"2024-07-17T08:39:50.748594+0000","last_peered":"2024-07-17T08:39:50.748594+0000","last_clean":"2024-07-17T08:39:50.748594+0000","last_became_active":"2024-07-17T08:39:48.158426+0000","last_became_peered":"2024-07-17T08:39:48.158426+0000","last_unstale":"2024-07-17T08:39:50.748594+0000","last_undegraded":"2024-07-17T08:39:50.748594+0000","last_fullsized":"2024-07-17T08:39:50.748594+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:16.554128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.9","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190976+0000","last_change":"2024-07-17T08:39:49.215412+0000","last_active":"2024-07-17T08:39:50.190976+0000","last_peered":"2024-07-17T08:39:50.190976+0000","last_clean":"2024-07-17T08:39:50.190976+0000","last_became_active":"2024-07-17T08:39:49.215119+0000","last_became_peered":"2024-07-17T08:39:49.215119+0000","last_unstale":"2024-07-17T08:39:50.190976+0000","last_undegraded":"2024-07-17T08:39:50.190976+0000","last_fullsized":"2024-07-17T08:39:50.190976+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:32.925696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.6","version":"293'1","reported_seq":15,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.203296+0000","last_change":"2024-07-17T08:39:48.169017+0000","last_active":"2024-07-17T08:39:50.203296+0000","last_peered":"2024-07-17T08:39:50.203296+0000","last_clean":"2024-07-17T08:39:50.203296+0000","last_became_active":"2024-07-17T08:39:48.168720+0000","last_became_peered":"2024-07-17T08:39:48.168720+0000","last_unstale":"2024-07-17T08:39:50.203296+0000","last_undegraded":"2024-07-17T08:39:50.203296+0000","last_fullsized":"2024-07-17T08:39:50.203296+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:03:50.610464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.a","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.746517+0000","last_change":"2024-07-17T08:39:49.202553+0000","last_active":"2024-07-17T08:39:50.746517+0000","last_peered":"2024-07-17T08:39:50.746517+0000","last_clean":"2024-07-17T08:39:50.746517+0000","last_became_active":"2024-07-17T08:39:49.202009+0000","last_became_peered":"2024-07-17T08:39:49.202009+0000","last_unstale":"2024-07-17T08:39:50.746517+0000","last_undegraded":"2024-07-17T08:39:50.746517+0000","last_fullsized":"2024-07-17T08:39:50.746517+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:43.518457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"39.5","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745807+0000","last_change":"2024-07-17T08:39:48.175168+0000","last_active":"2024-07-17T08:39:50.745807+0000","last_peered":"2024-07-17T08:39:50.745807+0000","last_clean":"2024-07-17T08:39:50.745807+0000","last_became_active":"2024-07-17T08:39:48.174897+0000","last_became_peered":"2024-07-17T08:39:48.174897+0000","last_unstale":"2024-07-17T08:39:50.745807+0000","last_undegraded":"2024-07-17T08:39:50.745807+0000","last_fullsized":"2024-07-17T08:39:50.745807+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:01.403849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.b","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.186611+0000","last_change":"2024-07-17T08:39:49.195151+0000","last_active":"2024-07-17T08:39:50.186611+0000","last_peered":"2024-07-17T08:39:50.186611+0000","last_clean":"2024-07-17T08:39:50.186611+0000","last_became_active":"2024-07-17T08:39:49.194482+0000","last_became_peered":"2024-07-17T08:39:49.194482+0000","last_unstale":"2024-07-17T08:39:50.186611+0000","last_undegraded":"2024-07-17T08:39:50.186611+0000","last_fullsized":"2024-07-17T08:39:50.186611+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:58.928172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"39.4","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750323+0000","last_change":"2024-07-17T08:39:48.165774+0000","last_active":"2024-07-17T08:39:50.750323+0000","last_peered":"2024-07-17T08:39:50.750323+0000","last_clean":"2024-07-17T08:39:50.750323+0000","last_became_active":"2024-07-17T08:39:48.163275+0000","last_became_peered":"2024-07-17T08:39:48.163275+0000","last_unstale":"2024-07-17T08:39:50.750323+0000","last_undegraded":"2024-07-17T08:39:50.750323+0000","last_fullsized":"2024-07-17T08:39:50.750323+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:08:36.092376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.c","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.205874+0000","last_change":"2024-07-17T08:39:49.180920+0000","last_active":"2024-07-17T08:39:50.205874+0000","last_peered":"2024-07-17T08:39:50.205874+0000","last_clean":"2024-07-17T08:39:50.205874+0000","last_became_active":"2024-07-17T08:39:49.180449+0000","last_became_peered":"2024-07-17T08:39:49.180449+0000","last_unstale":"2024-07-17T08:39:50.205874+0000","last_undegraded":"2024-07-17T08:39:50.205874+0000","last_fullsized":"2024-07-17T08:39:50.205874+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:54.499878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"39.3","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.192044+0000","last_change":"2024-07-17T08:39:48.171352+0000","last_active":"2024-07-17T08:39:50.192044+0000","last_peered":"2024-07-17T08:39:50.192044+0000","last_clean":"2024-07-17T08:39:50.192044+0000","last_became_active":"2024-07-17T08:39:48.163520+0000","last_became_peered":"2024-07-17T08:39:48.163520+0000","last_unstale":"2024-07-17T08:39:50.192044+0000","last_undegraded":"2024-07-17T08:39:50.192044+0000","last_fullsized":"2024-07-17T08:39:50.192044+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:42:12.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.d","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.750472+0000","last_change":"2024-07-17T08:39:49.209807+0000","last_active":"2024-07-17T08:39:50.750472+0000","last_peered":"2024-07-17T08:39:50.750472+0000","last_clean":"2024-07-17T08:39:50.750472+0000","last_became_active":"2024-07-17T08:39:49.205828+0000","last_became_peered":"2024-07-17T08:39:49.205828+0000","last_unstale":"2024-07-17T08:39:50.750472+0000","last_undegraded":"2024-07-17T08:39:50.750472+0000","last_fullsized":"2024-07-17T08:39:50.750472+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:49.934885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"39.2","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.747126+0000","last_change":"2024-07-17T08:39:48.161149+0000","last_active":"2024-07-17T08:39:50.747126+0000","last_peered":"2024-07-17T08:39:50.747126+0000","last_clean":"2024-07-17T08:39:50.747126+0000","last_became_active":"2024-07-17T08:39:48.160845+0000","last_became_peered":"2024-07-17T08:39:48.160845+0000","last_unstale":"2024-07-17T08:39:50.747126+0000","last_undegraded":"2024-07-17T08:39:50.747126+0000","last_fullsized":"2024-07-17T08:39:50.747126+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:16.443909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.e","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.191439+0000","last_change":"2024-07-17T08:39:49.209881+0000","last_active":"2024-07-17T08:39:50.191439+0000","last_peered":"2024-07-17T08:39:50.191439+0000","last_clean":"2024-07-17T08:39:50.191439+0000","last_became_active":"2024-07-17T08:39:49.209502+0000","last_became_peered":"2024-07-17T08:39:49.209502+0000","last_unstale":"2024-07-17T08:39:50.191439+0000","last_undegraded":"2024-07-17T08:39:50.191439+0000","last_fullsized":"2024-07-17T08:39:50.191439+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:16:15.754084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"39.1","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745728+0000","last_change":"2024-07-17T08:39:48.176448+0000","last_active":"2024-07-17T08:39:50.745728+0000","last_peered":"2024-07-17T08:39:50.745728+0000","last_clean":"2024-07-17T08:39:50.745728+0000","last_became_active":"2024-07-17T08:39:48.176150+0000","last_became_peered":"2024-07-17T08:39:48.176150+0000","last_unstale":"2024-07-17T08:39:50.745728+0000","last_undegraded":"2024-07-17T08:39:50.745728+0000","last_fullsized":"2024-07-17T08:39:50.745728+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:06.043287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.f","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.193390+0000","last_change":"2024-07-17T08:39:49.198952+0000","last_active":"2024-07-17T08:39:50.193390+0000","last_peered":"2024-07-17T08:39:50.193390+0000","last_clean":"2024-07-17T08:39:50.193390+0000","last_became_active":"2024-07-17T08:39:49.198402+0000","last_became_peered":"2024-07-17T08:39:49.198402+0000","last_unstale":"2024-07-17T08:39:50.193390+0000","last_undegraded":"2024-07-17T08:39:50.193390+0000","last_fullsized":"2024-07-17T08:39:50.193390+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:23:39.782605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"39.0","version":"0'0","reported_seq":14,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.204736+0000","last_change":"2024-07-17T08:39:48.170086+0000","last_active":"2024-07-17T08:39:50.204736+0000","last_peered":"2024-07-17T08:39:50.204736+0000","last_clean":"2024-07-17T08:39:50.204736+0000","last_became_active":"2024-07-17T08:39:48.169527+0000","last_became_peered":"2024-07-17T08:39:48.169527+0000","last_unstale":"2024-07-17T08:39:50.204736+0000","last_undegraded":"2024-07-17T08:39:50.204736+0000","last_fullsized":"2024-07-17T08:39:50.204736+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":290,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:47.117275+0000","last_clean_scrub_stamp":"2024-07-17T08:39:47.117275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:50.296585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.29","version":"0'0","reported_seq":12,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.190371+0000","last_change":"2024-07-17T08:39:49.206500+0000","last_active":"2024-07-17T08:39:50.190371+0000","last_peered":"2024-07-17T08:39:50.190371+0000","last_clean":"2024-07-17T08:39:50.190371+0000","last_became_active":"2024-07-17T08:39:49.206230+0000","last_became_peered":"2024-07-17T08:39:49.206230+0000","last_unstale":"2024-07-17T08:39:50.190371+0000","last_undegraded":"2024-07-17T08:39:50.190371+0000","last_fullsized":"2024-07-17T08:39:50.190371+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":291,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:39:48.127478+0000","last_clean_scrub_stamp":"2024-07-17T08:39:48.127478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:00:51.095125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"284'4596","reported_seq":5750,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-07-17T08:39:50.745623+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:39:50.745623+0000","last_peered":"2024-07-17T08:39:50.745623+0000","last_clean":"2024-07-17T08:39:50.745623+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:39:50.745623+0000","last_undegraded":"2024-07-17T08:39:50.745623+0000","last_fullsized":"2024-07-17T08:39:50.745623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4596,"log_dups_size":0,"ondisk_log_size":4596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":445,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":39,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":40,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1849,"num_read_kb":5206,"num_write":445,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4596,"ondisk_log_size":4596,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543390,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75656,"kb_used_data":4216,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296184,"statfs":{"total":96636764160,"available":96559292416,"internally_reserved":0,"allocated":4317184,"data_stored":2635641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510685,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52860,"kb_used_data":3632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318980,"statfs":{"total":96636764160,"available":96582635520,"internally_reserved":0,"allocated":3719168,"data_stored":2045239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510686,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70928,"kb_used_data":3644,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300912,"statfs":{"total":96636764160,"available":96564133888,"internally_reserved":0,"allocated":3731456,"data_stored":2045981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543391,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52872,"kb_used_data":3640,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318968,"statfs":{"total":96636764160,"available":96582623232,"internally_reserved":0,"allocated":3727360,"data_stored":2045351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510685,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52856,"kb_used_data":3628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318984,"statfs":{"total":96636764160,"available":96582639616,"internally_reserved":0,"allocated":3715072,"data_stored":2038047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543389,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57044,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314796,"statfs":{"total":96636764160,"available":96578351104,"internally_reserved":0,"allocated":3735552,"data_stored":2047433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52860,"kb_used_data":3632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318980,"statfs":{"total":96636764160,"available":96582635520,"internally_reserved":0,"allocated":3719168,"data_stored":2045239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510685,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53448,"kb_used_data":4216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318392,"statfs":{"total":96636764160,"available":96582033408,"internally_reserved":0,"allocated":4317184,"data_stored":2629945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":39,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:39:54.192 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:39:54.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 -- 172.21.15.44:0/1519153621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 msgr2=0x7f8b6006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 --2- 172.21.15.44:0/1519153621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b6006d620 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f8b6006f610 tx=0x7f8b5002a4d0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 -- 172.21.15.44:0/1519153621 shutdown_connections 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 --2- 172.21.15.44:0/1519153621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 0x7f8b600cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 --2- 172.21.15.44:0/1519153621 >> v2:172.21.15.175:3300/0 conn(0x7f8b6006ed30 0x7f8b6015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 --2- 172.21.15.44:0/1519153621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b6006d620 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 -- 172.21.15.44:0/1519153621 >> 172.21.15.44:0/1519153621 conn(0x7f8b600b0eb0 msgr2=0x7f8b600b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 -- 172.21.15.44:0/1519153621 shutdown_connections 2024-07-17T08:39:54.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.456+0000 7f8b66c00640 1 -- 172.21.15.44:0/1519153621 wait complete. 2024-07-17T08:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.458+0000 7f8b66c00640 1 Processor -- start 2024-07-17T08:39:54.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.458+0000 7f8b66c00640 1 -- start start 2024-07-17T08:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6006ed30 0x7f8b6014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 0x7f8b600adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6015f1f0 con 0x7f8b60152220 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6015f070 con 0x7f8b6006ed30 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b66c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6015f370 con 0x7f8b600cb3a0 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.459+0000 7f8b64c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41732/0 (socket says 172.21.15.44:41732) 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 -- 172.21.15.44:0/3609881123 learned_addr learned my addr 172.21.15.44:0/3609881123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b5fe00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6006ed30 0x7f8b6014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b5f400640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 0x7f8b600adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 -- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 msgr2=0x7f8b600adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 0x7f8b600adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 -- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6006ed30 msgr2=0x7f8b6014db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6006ed30 0x7f8b6014db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 -- 172.21.15.44:0/3609881123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600ac370 con 0x7f8b60152220 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.460+0000 7f8b64c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800cb00 tx=0x7f8b5800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.461+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004b20 con 0x7f8b60152220 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.461+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b58004db0 con 0x7f8b60152220 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.461+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b600ac650 con 0x7f8b60152220 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.461+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58005280 con 0x7f8b60152220 2024-07-17T08:39:54.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.461+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b600acc10 con 0x7f8b60152220 2024-07-17T08:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.462+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b600acde0 con 0x7f8b60152220 2024-07-17T08:39:54.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.463+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f8b58005420 con 0x7f8b60152220 2024-07-17T08:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.464+0000 7f8b57e00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 0x7f8b40079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.464+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f8b58098ee0 con 0x7f8b60152220 2024-07-17T08:39:54.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.465+0000 7f8b5fe00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 0x7f8b40079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.466+0000 7f8b5fe00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 0x7f8b40079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b50033000 tx=0x7f8b50004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:54.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.469+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8b58014030 con 0x7f8b60152220 2024-07-17T08:39:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.753+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8b600cb7d0 con 0x7f8b60152220 2024-07-17T08:39:54.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.757+0000 7f8b57e00640 1 -- 172.21.15.44:0/3609881123 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v77) ==== 190+0+147 (secure 0 0 0) 0x7f8b58061f20 con 0x7f8b60152220 2024-07-17T08:39:54.759 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:39:54.759 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:39:54.759 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:39:54.760 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:39:54.760 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:39:54.760 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.760+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 msgr2=0x7f8b40079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.760+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 0x7f8b40079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b50033000 tx=0x7f8b50004730 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.760+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 msgr2=0x7f8b60155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.760+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800cb00 tx=0x7f8b5800a250 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 shutdown_connections 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f8b40076bb0 0x7f8b40079070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b60152220 0x7f8b60155ea0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b600cb3a0 0x7f8b600adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 --2- 172.21.15.44:0/3609881123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b6006ed30 0x7f8b6014db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 >> 172.21.15.44:0/3609881123 conn(0x7f8b600b0eb0 msgr2=0x7f8b6015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 shutdown_connections 2024-07-17T08:39:54.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.761+0000 7f8b66c00640 1 -- 172.21.15.44:0/3609881123 wait complete. 2024-07-17T08:39:54.783 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:39:54.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.989+0000 7f6aca400640 1 -- 172.21.15.44:0/759111951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 msgr2=0x7f6ac4159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.989+0000 7f6aca400640 1 --2- 172.21.15.44:0/759111951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac4159b00 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4005fd0 tx=0x7f6ab402a530 comp rx=0 tx=0).stop 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 -- 172.21.15.44:0/759111951 shutdown_connections 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 --2- 172.21.15.44:0/759111951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 0x7f6ac406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 --2- 172.21.15.44:0/759111951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac4159b00 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 -- 172.21.15.44:0/759111951 >> 172.21.15.44:0/759111951 conn(0x7f6ac40b0fb0 msgr2=0x7f6ac40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 -- 172.21.15.44:0/759111951 shutdown_connections 2024-07-17T08:39:54.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.990+0000 7f6aca400640 1 -- 172.21.15.44:0/759111951 wait complete. 2024-07-17T08:39:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.992+0000 7f6aca400640 1 Processor -- start 2024-07-17T08:39:54.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.992+0000 7f6aca400640 1 -- start start 2024-07-17T08:39:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 0x7f6ac414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac415a040 0x7f6ac4156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac406f890 con 0x7f6ac4153a50 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac406f710 con 0x7f6ac415a040 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6aca400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac406e5d0 con 0x7f6ac40acaf0 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6ac2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6ac2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41740/0 (socket says 172.21.15.44:41740) 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6ac2a00640 1 -- 172.21.15.44:0/3195594690 learned_addr learned my addr 172.21.15.44:0/3195594690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6ac3400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 0x7f6ac414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.993+0000 7f6ac3e00640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac415a040 0x7f6ac4156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 -- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 msgr2=0x7f6ac414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 0x7f6ac414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 -- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac415a040 msgr2=0x7f6ac4156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac415a040 0x7f6ac4156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 -- 172.21.15.44:0/3195594690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac414f3c0 con 0x7f6ac4153a50 2024-07-17T08:39:54.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6ac2a00640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b410 tx=0x7f6ab000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:54.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0015020 con 0x7f6ac4153a50 2024-07-17T08:39:54.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab0011920 con 0x7f6ac4153a50 2024-07-17T08:39:54.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0011d70 con 0x7f6ac4153a50 2024-07-17T08:39:54.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac40cb640 con 0x7f6ac4153a50 2024-07-17T08:39:54.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.994+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41424c0 con 0x7f6ac4153a50 2024-07-17T08:39:54.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.995+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac4142000 con 0x7f6ac4153a50 2024-07-17T08:39:54.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.997+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f6ab0018020 con 0x7f6ac4153a50 2024-07-17T08:39:54.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.998+0000 7f6abb400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 0x7f6aa4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:54.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.998+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f6ab0098640 con 0x7f6ac4153a50 2024-07-17T08:39:55.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:54.998+0000 7f6ac3400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 0x7f6aa4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.000+0000 7f6ac3400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 0x7f6aa4079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4033000 tx=0x7f6ab4004730 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.002+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6ac4142000 con 0x7f6ac4153a50 2024-07-17T08:39:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.254+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ac406e090 con 0x7f6ac4153a50 2024-07-17T08:39:55.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.255+0000 7f6abb400640 1 -- 172.21.15.44:0/3195594690 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f6ab001f020 con 0x7f6ac4153a50 2024-07-17T08:39:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:39:55.257 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:39:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:39:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.257+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 msgr2=0x7f6aa4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.257+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 0x7f6aa4079070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4033000 tx=0x7f6ab4004730 comp rx=0 tx=0).stop 2024-07-17T08:39:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.257+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 msgr2=0x7f6ac41566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.257+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b410 tx=0x7f6ab000b8d0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 shutdown_connections 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6aa4076bb0 0x7f6aa4079070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ac415a040 0x7f6ac4156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ac4153a50 0x7f6ac41566f0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 --2- 172.21.15.44:0/3195594690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ac40acaf0 0x7f6ac414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 >> 172.21.15.44:0/3195594690 conn(0x7f6ac40b0fb0 msgr2=0x7f6ac40b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 shutdown_connections 2024-07-17T08:39:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.258+0000 7f6aca400640 1 -- 172.21.15.44:0/3195594690 wait complete. 2024-07-17T08:39:55.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 164, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:39:48.630457+0000', 'modified': '2024-07-17T08:39:51.717689+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12228}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12228': {'gid': 12228, 'name': 'a', 'rank': 0, 'incarnation': 160, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/789460381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 789460381}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 789460381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [40], 'metadata_pool': 39, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12228, 'qdb_cluster': [12228]} 2024-07-17T08:39:55.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:39:55.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:39:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.900+0000 7fbedbe00640 1 -- 172.21.15.44:0/192025600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc004800 msgr2=0x7fbebc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.900+0000 7fbedbe00640 1 --2- 172.21.15.44:0/192025600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c710 tx=0x7fbebc005830 comp rx=0 tx=0).stop 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.900+0000 7fbedbe00640 1 -- 172.21.15.44:0/192025600 shutdown_connections 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.900+0000 7fbedbe00640 1 --2- 172.21.15.44:0/192025600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.900+0000 7fbedbe00640 1 -- 172.21.15.44:0/192025600 >> 172.21.15.44:0/192025600 conn(0x7fbedc0cf0d0 msgr2=0x7fbedc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.901+0000 7fbedbe00640 1 -- 172.21.15.44:0/192025600 shutdown_connections 2024-07-17T08:39:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.901+0000 7fbedbe00640 1 -- 172.21.15.44:0/192025600 wait complete. 2024-07-17T08:39:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.902+0000 7fbedbe00640 1 Processor -- start 2024-07-17T08:39:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.902+0000 7fbedbe00640 1 -- start start 2024-07-17T08:39:55.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc004800 0x7fbedc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbedc0c9880 0x7fbedc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc06dc20 con 0x7fbedc1f4090 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc06daa0 con 0x7fbebc004800 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.903+0000 7fbedbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbedc06dda0 con 0x7fbedc0c9880 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41762/0 (socket says 172.21.15.44:41762) 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 -- 172.21.15.44:0/1673362351 learned_addr learned my addr 172.21.15.44:0/1673362351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedaa00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc004800 0x7fbedc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbed3e00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbedc0c9880 0x7fbedc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 -- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbedc0c9880 msgr2=0x7fbedc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbedc0c9880 0x7fbedc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 -- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc004800 msgr2=0x7fbedc0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc004800 0x7fbedc0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.904+0000 7fbedb400640 1 -- 172.21.15.44:0/1673362351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc0ca240 con 0x7fbedc1f4090 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbedb400640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fbedc15ce40 tx=0x7fbed40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbed2a00640 1 -- 172.21.15.44:0/1673362351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4015070 con 0x7fbedc1f4090 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbed2a00640 1 -- 172.21.15.44:0/1673362351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbed4010990 con 0x7fbedc1f4090 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbed2a00640 1 -- 172.21.15.44:0/1673362351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4010dc0 con 0x7fbedc1f4090 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbedc072e60 con 0x7fbedc1f4090 2024-07-17T08:39:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.905+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbedc073400 con 0x7fbedc1f4090 2024-07-17T08:39:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.908+0000 7fbed2a00640 1 -- 172.21.15.44:0/1673362351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7fbed4018070 con 0x7fbedc1f4090 2024-07-17T08:39:55.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.909+0000 7fbed2a00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 0x7fbeb8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.909+0000 7fbed2a00640 1 -- 172.21.15.44:0/1673362351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fbed4099130 con 0x7fbedc1f4090 2024-07-17T08:39:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.910+0000 7fbee4581740 1 Processor -- start 2024-07-17T08:39:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.910+0000 7fbedaa00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 0x7fbeb8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.910+0000 7fbee4581740 1 -- start start 2024-07-17T08:39:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 0x564fad47b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad477c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564fad3c2d60 con 0x564fad452280 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fad3b2260 con 0x564fad477790 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564fad3b62a0 con 0x564fad47afc0 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbedaa00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 0x7fbeb8079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0ca870 tx=0x7fbecc001630 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41764/0 (socket says 172.21.15.44:41764) 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.911+0000 7fbed3e00640 1 -- 172.21.15.44:0/2078800491 learned_addr learned my addr 172.21.15.44:0/2078800491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbedaa00640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 0x564fad47b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 -- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 msgr2=0x564fad47b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 0x564fad47b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 -- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 msgr2=0x564fad477c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbedb400640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad477c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad477c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 -- 172.21.15.44:0/2078800491 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fad116850 con 0x564fad452280 2024-07-17T08:39:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbedaa00640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 0x564fad47b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:39:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.912+0000 7fbed3e00640 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c630 tx=0x7fbec40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e7912b363bbc3 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbecb000640 1 -- 172.21.15.44:0/2078800491 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4042070 con 0x564fad452280 2024-07-17T08:39:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbecb000640 1 -- 172.21.15.44:0/2078800491 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbec4039040 con 0x564fad452280 2024-07-17T08:39:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbecb000640 1 -- 172.21.15.44:0/2078800491 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4006920 con 0x564fad452280 2024-07-17T08:39:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 -- 172.21.15.44:0/2078800491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 msgr2=0x564fad477250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c630 tx=0x7fbec40036f0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 -- 172.21.15.44:0/2078800491 shutdown_connections 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad477c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad452280 0x564fad477250 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 --2- 172.21.15.44:0/2078800491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad47afc0 0x564fad47b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 -- 172.21.15.44:0/2078800491 >> 172.21.15.44:0/2078800491 conn(0x564fad44d460 msgr2=0x564fad450e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 -- 172.21.15.44:0/2078800491 shutdown_connections 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.913+0000 7fbee4581740 1 -- 172.21.15.44:0/2078800491 wait complete. 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.914+0000 7fbee4581740 1 -- start start 2024-07-17T08:39:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.914+0000 7fbee4581740 1 Processor -- start 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad4a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad48f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564fad125620 con 0x564fad47afc0 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fad359f90 con 0x564fad477790 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564fad44bb60 con 0x564fad462680 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.915+0000 7fbee4581740 10 client.0 _ll_register_callbacks cb 0x564fad37ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41770/0 (socket says 172.21.15.44:41770) 2024-07-17T08:39:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 -- 172.21.15.44:0/3107770561 learned_addr learned my addr 172.21.15.44:0/3107770561 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 -- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 msgr2=0x564fad48f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad48f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 -- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 msgr2=0x564fad4a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad4a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 -- 172.21.15.44:0/3107770561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fad18d6b0 con 0x564fad47afc0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbed3e00640 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c830 tx=0x7fbec400c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d1aee21939a65 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.916+0000 7fbec3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4042040 con 0x564fad47afc0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fbec4004620 con 0x564fad47afc0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fad389900 con 0x564fad47afc0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) ==== 24+0+0 (secure 0 0 0) 0x7fbec4050050 con 0x564fad47afc0 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 10 client.12345 fetch_fsmap learned FSMap version 164 2024-07-17T08:39:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=2+}) -- 0x564fad475880 con 0x564fad47afc0 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564fad389900 con 0x564fad47afc0 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 164) ==== 3502+0+0 (secure 0 0 0) 0x7fbec4053020 con 0x564fad47afc0 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 10 client.12345 fetch_fsmap finished waiting for FSMap version 164 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 10 client.12345 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 20 client.12345 populate_metadata read hostname 'smithi044' 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.917+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad476030 0x564fad476410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.918+0000 7fbee4581740 4 client.12345 mds_command: new command op to 12228 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.918+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 --> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fad3c4cb0 con 0x564fad476030 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.918+0000 7fbedaa00640 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad476030 0x564fad476410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.918+0000 7fbedaa00640 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad476030 0x564fad476410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fdcad71e28d445a server_cookie=d95c738423d9a140 in_seq=0 out_seq=0 2024-07-17T08:39:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.918+0000 7fbec3400640 10 client.12345 ms_handle_connect on v2:172.21.15.44:6833/789460381 2024-07-17T08:39:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.919+0000 7fbec3400640 1 -- 172.21.15.44:0/3107770561 <== mds.0 v2:172.21.15.44:6833/789460381 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fad3c4cb0 con 0x564fad476030 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.919+0000 7fbec3400640 10 client.12345 handle_command_reply: tid=0 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.919+0000 7fbee4581740 1 client.12345 shutdown 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 msgr2=0x564fad4738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c830 tx=0x7fbec400c370 comp rx=0 tx=0).stop 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 shutdown_connections 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad476030 0x564fad476410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad48f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad4738c0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 --2- 172.21.15.44:0/3107770561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad4a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbec3400640 0 client.12345 ms_handle_reset on v2:172.21.15.44:6833/789460381 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 >> 172.21.15.44:0/3107770561 conn(0x564fad452280 msgr2=0x564fad450b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 shutdown_connections 2024-07-17T08:39:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 1 -- 172.21.15.44:0/3107770561 wait complete. 2024-07-17T08:39:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.920+0000 7fbee4581740 20 client.12345 trim_cache size 0 max 16384 2024-07-17T08:39:55.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 Processor -- start 2024-07-17T08:39:55.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 -- start start 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad477790 0x564fad44a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad47afc0 0x564fad4b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc200b50 con 0x564fad462680 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.941+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc0c8e50 con 0x564fad47afc0 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbedc0c94a0 con 0x564fad477790 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41772/0 (socket says 172.21.15.44:41772) 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 -- 172.21.15.44:0/3361814077 learned_addr learned my addr 172.21.15.44:0/3361814077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 -- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad477790 msgr2=0x564fad44a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad477790 0x564fad44a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 -- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad47afc0 msgr2=0x564fad4b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad47afc0 0x564fad4b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 -- 172.21.15.44:0/3361814077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fad48c2a0 con 0x564fad462680 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.942+0000 7fbedaa00640 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fbedc15ce70 tx=0x7fbecc008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c210c1652bd1edbf server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.943+0000 7fbeca600640 1 -- 172.21.15.44:0/3361814077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc01a040 con 0x564fad462680 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.943+0000 7fbeca600640 1 -- 172.21.15.44:0/3361814077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc01b040 con 0x564fad462680 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.943+0000 7fbeca600640 1 -- 172.21.15.44:0/3361814077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc003a70 con 0x564fad462680 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.943+0000 7fbee4581740 1 -- 172.21.15.44:0/3361814077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 msgr2=0x564fad44cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.943+0000 7fbee4581740 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fbedc15ce70 tx=0x7fbecc008070 comp rx=0 tx=0).stop 2024-07-17T08:39:55.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 -- 172.21.15.44:0/3361814077 shutdown_connections 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad47afc0 0x564fad4b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad477790 0x564fad44a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 --2- 172.21.15.44:0/3361814077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad462680 0x564fad44cb70 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 -- 172.21.15.44:0/3361814077 >> 172.21.15.44:0/3361814077 conn(0x564fad452280 msgr2=0x564fad4747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 -- 172.21.15.44:0/3361814077 shutdown_connections 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 -- 172.21.15.44:0/3361814077 wait complete. 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 -- start start 2024-07-17T08:39:55.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.944+0000 7fbee4581740 1 Processor -- start 2024-07-17T08:39:55.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad44c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad498f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564fad125620 con 0x564fad47afc0 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564fad359f90 con 0x564fad477790 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.945+0000 7fbee4581740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564fad479d30 con 0x564fad462680 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbee4581740 10 client.0 _ll_register_callbacks cb 0x7fbec4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41782/0 (socket says 172.21.15.44:41782) 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 -- 172.21.15.44:0/500167694 learned_addr learned my addr 172.21.15.44:0/500167694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedaa00640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad44c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 -- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 msgr2=0x564fad44c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad44c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 -- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 msgr2=0x564fad498f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad498f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 -- 172.21.15.44:0/500167694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fad517710 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbedb400640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fbed4061a00 tx=0x7fbed405da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432fa8274ed3dbc9 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.946+0000 7fbec3400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed401f070 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fbed4068270 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbecc01b040 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) ==== 24+0+0 (secure 0 0 0) 0x7fbed4062170 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 10 client.12351 fetch_fsmap learned FSMap version 164 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=2+}) -- 0x564fad4a8370 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc01b040 con 0x564fad47afc0 2024-07-17T08:39:55.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 164) ==== 3502+0+0 (secure 0 0 0) 0x7fbed400f040 con 0x564fad47afc0 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 10 client.12351 fetch_fsmap finished waiting for FSMap version 164 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 10 client.12351 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 20 client.12351 populate_metadata read hostname 'smithi044' 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad514ce0 0x564fad476410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 4 client.12351 mds_command: new command op to 12228 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.947+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 --> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] -- command(tid 0: {"prefix": "status"}) -- 0x564fad3c0750 con 0x564fad514ce0 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbedaa00640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad514ce0 0x564fad476410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbedaa00640 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad514ce0 0x564fad476410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979035ea1b48baa6 server_cookie=a8860fe2d0e43906 in_seq=0 out_seq=0 2024-07-17T08:39:55.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbec3400640 10 client.12351 ms_handle_connect on v2:172.21.15.44:6833/789460381 2024-07-17T08:39:55.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbec3400640 1 -- 172.21.15.44:0/500167694 <== mds.0 v2:172.21.15.44:6833/789460381 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x564fad3c0750 con 0x564fad514ce0 2024-07-17T08:39:55.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbec3400640 10 client.12351 handle_command_reply: tid=0 2024-07-17T08:39:55.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.948+0000 7fbee4581740 1 client.12351 shutdown 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 msgr2=0x564fad499470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fbed4061a00 tx=0x7fbed405da40 comp rx=0 tx=0).stop 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 shutdown_connections 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:6833/789460381,v1:172.21.15.44:6835/789460381] conn(0x564fad514ce0 0x564fad476410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564fad47afc0 0x564fad499470 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564fad477790 0x564fad498f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 --2- 172.21.15.44:0/500167694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564fad462680 0x564fad44c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbec3400640 0 client.12351 ms_handle_reset on v2:172.21.15.44:6833/789460381 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 >> 172.21.15.44:0/500167694 conn(0x564fad452280 msgr2=0x564fad450f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 shutdown_connections 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 1 -- 172.21.15.44:0/500167694 wait complete. 2024-07-17T08:39:55.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.949+0000 7fbee4581740 20 client.12351 trim_cache size 0 max 16384 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 12228, 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7678771539999998, 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 164, 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 293, 2024-07-17T08:39:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 293, 2024-07-17T08:39:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.709372395000001 2024-07-17T08:39:55.956 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:39:55.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 msgr2=0x7fbeb8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 0x7fbeb8079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0ca870 tx=0x7fbecc001630 comp rx=0 tx=0).stop 2024-07-17T08:39:55.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 msgr2=0x7fbedc1f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fbedc15ce40 tx=0x7fbed40098d0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 shutdown_connections 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbeb8076bb0 0x7fbeb8079070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbedc1f4090 0x7fbedc1f4540 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbedc0c9880 0x7fbedc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 --2- 172.21.15.44:0/1673362351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbebc004800 0x7fbedc0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 >> 172.21.15.44:0/1673362351 conn(0x7fbedc0cf0d0 msgr2=0x7fbedc0ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 shutdown_connections 2024-07-17T08:39:55.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:55.950+0000 7fbedbe00640 1 -- 172.21.15.44:0/1673362351 wait complete. 2024-07-17T08:39:55.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 -- 172.21.15.44:0/366656392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b2ad0 msgr2=0x7f67dc0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 --2- 172.21.15.44:0/366656392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b2ad0 0x7f67dc0b5200 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0d0000 tx=0x7f67c802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 -- 172.21.15.44:0/366656392 shutdown_connections 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 --2- 172.21.15.44:0/366656392 >> v2:172.21.15.175:3300/0 conn(0x7f67dc0b5740 0x7f67dc0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 --2- 172.21.15.44:0/366656392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b2ad0 0x7f67dc0b5200 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 -- 172.21.15.44:0/366656392 >> 172.21.15.44:0/366656392 conn(0x7f67dc0d00a0 msgr2=0x7f67dc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:56.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 -- 172.21.15.44:0/366656392 shutdown_connections 2024-07-17T08:39:56.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.174+0000 7f67e1800640 1 -- 172.21.15.44:0/366656392 wait complete. 2024-07-17T08:39:56.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.176+0000 7f67e1800640 1 Processor -- start 2024-07-17T08:39:56.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.176+0000 7f67e1800640 1 -- start start 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0b2ad0 0x7f67dc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15d600 0x7f67dc1f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0ce670 con 0x7f67dc0b5740 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0ce4f0 con 0x7f67dc15d600 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67e1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67dc0ce7f0 con 0x7f67dc0b2ad0 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67da000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:56.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67da000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41796/0 (socket says 172.21.15.44:41796) 2024-07-17T08:39:56.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.177+0000 7f67da000640 1 -- 172.21.15.44:0/3079495402 learned_addr learned my addr 172.21.15.44:0/3079495402 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 -- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0b2ad0 msgr2=0x7f67dc150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0b2ad0 0x7f67dc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 -- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15d600 msgr2=0x7f67dc1f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15d600 0x7f67dc1f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 -- 172.21.15.44:0/3079495402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc0b9030 con 0x7f67dc0b5740 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67da000640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009780 tx=0x7f67c80329e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8040070 con 0x7f67dc0b5740 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c8005110 con 0x7f67dc0b5740 2024-07-17T08:39:56.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8005560 con 0x7f67dc0b5740 2024-07-17T08:39:56.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.178+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc153370 con 0x7f67dc0b5740 2024-07-17T08:39:56.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.179+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc15ec60 con 0x7f67dc0b5740 2024-07-17T08:39:56.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.180+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc153760 con 0x7f67dc0b5740 2024-07-17T08:39:56.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.181+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f67c8039040 con 0x7f67dc0b5740 2024-07-17T08:39:56.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.182+0000 7f67d2a00640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 0x7f67bc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:56.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.182+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f67c80b95a0 con 0x7f67dc0b5740 2024-07-17T08:39:56.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.183+0000 7f67daa00640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 0x7f67bc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:56.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.184+0000 7f67daa00640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 0x7f67bc079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67cc001510 tx=0x7f67cc007930 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:56.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.187+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f67dc0b9030 con 0x7f67dc0b5740 2024-07-17T08:39:56.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.431+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67dc0ced10 con 0x7f67dc0b5740 2024-07-17T08:39:56.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.432+0000 7f67d2a00640 1 -- 172.21.15.44:0/3079495402 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v164) ==== 53+0+73 (secure 0 0 0) 0x7f67c8082710 con 0x7f67dc0b5740 2024-07-17T08:39:56.434 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 msgr2=0x7f67bc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 0x7f67bc079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67cc001510 tx=0x7f67cc007930 comp rx=0 tx=0).stop 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 msgr2=0x7f67dc152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009780 tx=0x7f67c80329e0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 shutdown_connections 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f67bc076bb0 0x7f67bc079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67dc15d600 0x7f67dc1f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc0b5740 0x7f67dc152930 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 --2- 172.21.15.44:0/3079495402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67dc0b2ad0 0x7f67dc150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 >> 172.21.15.44:0/3079495402 conn(0x7f67dc0d00a0 msgr2=0x7f67dc15a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 shutdown_connections 2024-07-17T08:39:56.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56.435+0000 7f67e1800640 1 -- 172.21.15.44:0/3079495402 wait complete. 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:39:56.456 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:39:56.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:56.457 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:39:56.507 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36261sec preferred_lft 36261sec 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.508 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:39:56.509 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.510 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:39:56.510 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:39:56.510 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:39:56.510 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:39:56.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:56.511 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:39:56.590 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:39:56.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:56.595 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:39:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:39:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:39:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:39:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:39:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:39:56.649 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:39:56.649 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:39:56.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:39:56.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:39:56.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:39:56.650 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:39:56.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:39:56.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:39:56.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:56.873 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:39:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:39:56.923 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:39:56.924 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:39:56.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:56.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:39:57.005 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:39:57.005 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:39:57.062 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:39:57.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:39:57.062 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:39:57.112 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:39:57.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:57.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:39:57.247 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:39:57.247 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:39:57.247 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:39:57.268 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:39:57.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:39:57.268 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:39:57.268 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:39:57.269 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:39:57.293 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:39:57.296 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:39:57.296 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:39:57.296 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:39:57.296 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:39:57.297 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:39:57.300 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:39:57.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:39:57.301 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:39:57.369 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:39:57.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:39:57.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:39:57.508 INFO:teuthology.orchestra.run.smithi044.stdout:client12363 2024-07-17T08:39:57.508 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:39:57.508 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:39:57.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:39:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.808+0000 7f6e01400640 1 -- 172.21.15.44:0/3191977893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de0003ac0 msgr2=0x7f6de0003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.808+0000 7f6e01400640 1 --2- 172.21.15.44:0/3191977893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de0003ac0 0x7f6de0003ee0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f6dec001690 tx=0x7f6de0005830 comp rx=0 tx=0).stop 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.809+0000 7f6e01400640 1 -- 172.21.15.44:0/3191977893 shutdown_connections 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.809+0000 7f6e01400640 1 --2- 172.21.15.44:0/3191977893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de0003ac0 0x7f6de0003ee0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.809+0000 7f6e01400640 1 -- 172.21.15.44:0/3191977893 >> 172.21.15.44:0/3191977893 conn(0x7f6dfc0cf0d0 msgr2=0x7f6dfc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.809+0000 7f6e01400640 1 -- 172.21.15.44:0/3191977893 shutdown_connections 2024-07-17T08:39:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.809+0000 7f6e01400640 1 -- 172.21.15.44:0/3191977893 wait complete. 2024-07-17T08:39:57.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.811+0000 7f6e01400640 1 Processor -- start 2024-07-17T08:39:57.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.811+0000 7f6e01400640 1 -- start start 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6de0003ac0 0x7f6dfc1ef240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc099190 0x7f6dfc154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc06d5a0 con 0x7f6dfc099610 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc06d420 con 0x7f6dfc099190 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.812+0000 7f6e01400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc06d720 con 0x7f6de0003ac0 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41802/0 (socket says 172.21.15.44:41802) 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 -- 172.21.15.44:0/27643130 learned_addr learned my addr 172.21.15.44:0/27643130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6df3e00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc099190 0x7f6dfc154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6dfbe00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6de0003ac0 0x7f6dfc1ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 -- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6de0003ac0 msgr2=0x7f6dfc1ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6de0003ac0 0x7f6dfc1ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 -- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc099190 msgr2=0x7f6dfc154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:39:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc099190 0x7f6dfc154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:39:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.813+0000 7f6e00a00640 1 -- 172.21.15.44:0/27643130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc099be0 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6e00a00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc15ce70 tx=0x7f6df400a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4004d20 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6df400f790 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df400fbe0 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc099ec0 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.814+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc09a480 con 0x7f6dfc099610 2024-07-17T08:39:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.815+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc0ae420 con 0x7f6dfc099610 2024-07-17T08:39:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.816+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f6df4004fb0 con 0x7f6dfc099610 2024-07-17T08:39:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.818+0000 7f6df8c00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 0x7f6dd80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:39:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.818+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f6df4098b40 con 0x7f6dfc099610 2024-07-17T08:39:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.819+0000 7f6dfbe00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 0x7f6dd80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:39:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.820+0000 7f6dfbe00640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 0x7f6dd80790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6dec032000 tx=0x7f6dec001190 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:39:57.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:57.822+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6df4061c10 con 0x7f6dfc099610 2024-07-17T08:39:58.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:39:58.058+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 --> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6dfc0cffe0 con 0x7f6dd8076be0 2024-07-17T08:40:05.063 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.062+0000 7f0703200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.063 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.062+0000 7f0703200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.063 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.062+0000 7f0703200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.064 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.062+0000 7f0703200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.064 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.062+0000 7f0703200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.065+0000 7f06fec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.065+0000 7f06fec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.065+0000 7f06fec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.065+0000 7f06fec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.065 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:05.065+0000 7f06fec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:05.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.080+0000 7f6df8c00640 1 -- 172.21.15.44:0/27643130 <== mgr.8795 v2:172.21.15.175:6834/1518538177 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6dfc0cffe0 con 0x7f6dd8076be0 2024-07-17T08:40:05.081 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:40:05.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:40:05.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:40:05.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:40:05.081 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:40:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 msgr2=0x7f6dd80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 0x7f6dd80790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6dec032000 tx=0x7f6dec001190 comp rx=0 tx=0).stop 2024-07-17T08:40:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 msgr2=0x7f6dfc154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:05.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc15ce70 tx=0x7f6df400a960 comp rx=0 tx=0).stop 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 shutdown_connections 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f6dd8076be0 0x7f6dd80790a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc099610 0x7f6dfc154ea0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc099190 0x7f6dfc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 --2- 172.21.15.44:0/27643130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6de0003ac0 0x7f6dfc1ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.084+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 >> 172.21.15.44:0/27643130 conn(0x7f6dfc0cf0d0 msgr2=0x7f6dfc0ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.085+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 shutdown_connections 2024-07-17T08:40:05.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.085+0000 7f6e01400640 1 -- 172.21.15.44:0/27643130 wait complete. 2024-07-17T08:40:05.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:40:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.328+0000 7f7df6800640 1 -- 172.21.15.44:0/3083282313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df006f5a0 msgr2=0x7f7df00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.328+0000 7f7df6800640 1 --2- 172.21.15.44:0/3083282313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df006f5a0 0x7f7df00b2140 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005fd0 tx=0x7f7de402b910 comp rx=0 tx=0).stop 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.329+0000 7f7df6800640 1 -- 172.21.15.44:0/3083282313 shutdown_connections 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.329+0000 7f7df6800640 1 --2- 172.21.15.44:0/3083282313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df006f5a0 0x7f7df00b2140 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.329+0000 7f7df6800640 1 -- 172.21.15.44:0/3083282313 >> 172.21.15.44:0/3083282313 conn(0x7f7df006c120 msgr2=0x7f7df006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.329+0000 7f7df6800640 1 -- 172.21.15.44:0/3083282313 shutdown_connections 2024-07-17T08:40:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.329+0000 7f7df6800640 1 -- 172.21.15.44:0/3083282313 wait complete. 2024-07-17T08:40:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.330+0000 7f7df6800640 1 Processor -- start 2024-07-17T08:40:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.331+0000 7f7df6800640 1 -- start start 2024-07-17T08:40:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df006f5a0 0x7f7df01e2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df00cf6f0 0x7f7df014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7df006e0b0 con 0x7f7df0156d40 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7df006df30 con 0x7f7df00cf6f0 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7df006e230 con 0x7f7df006f5a0 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38092/0 (socket says 172.21.15.44:38092) 2024-07-17T08:40:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 -- 172.21.15.44:0/474494549 learned_addr learned my addr 172.21.15.44:0/474494549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:40:05.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df4a00640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df00cf6f0 0x7f7df014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:05.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 -- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df006f5a0 msgr2=0x7f7df01e2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df006f5a0 0x7f7df01e2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 -- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df00cf6f0 msgr2=0x7f7df014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df00cf6f0 0x7f7df014f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.332+0000 7f7df5e00640 1 -- 172.21.15.44:0/474494549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df014fbb0 con 0x7f7df0156d40 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7df5e00640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c7d0 tx=0x7f7de800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8003b70 con 0x7f7df0156d40 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7de8003d10 con 0x7f7df0156d40 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800aab0 con 0x7f7df0156d40 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df014fe90 con 0x7f7df0156d40 2024-07-17T08:40:05.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.333+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df0157710 con 0x7f7df0156d40 2024-07-17T08:40:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.334+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00b2140 con 0x7f7df0156d40 2024-07-17T08:40:05.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.335+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 169) ==== 98829+0+0 (secure 0 0 0) 0x7f7de800ac50 con 0x7f7df0156d40 2024-07-17T08:40:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.336+0000 7f7ded600640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 0x7f7dd0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.337+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7f7de8098650 con 0x7f7df0156d40 2024-07-17T08:40:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.337+0000 7f7df5400640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 0x7f7dd0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.338+0000 7f7df5400640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 0x7f7dd0079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7de4035000 tx=0x7f7de40046c0 comp rx=0 tx=0).ready entity=mgr.8795 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.341+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7de8061690 con 0x7f7df0156d40 2024-07-17T08:40:05.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:05.611+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7df01e4890 con 0x7f7df0156d40 2024-07-17T08:40:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.294+0000 7f7ded600640 1 -- 172.21.15.44:0/474494549 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v170) ==== 92+0+0 (secure 0 0 0) 0x7f7de8060f90 con 0x7f7df0156d40 2024-07-17T08:40:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.298+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 msgr2=0x7f7dd0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.298+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 0x7f7dd0079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7de4035000 tx=0x7f7de40046c0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.298+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 msgr2=0x7f7df01e2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.298+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c7d0 tx=0x7f7de800cc90 comp rx=0 tx=0).stop 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 shutdown_connections 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7f7dd0076bb0 0x7f7dd0079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df0156d40 0x7f7df01e2b90 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df00cf6f0 0x7f7df014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 --2- 172.21.15.44:0/474494549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7df006f5a0 0x7f7df01e2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.299+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 >> 172.21.15.44:0/474494549 conn(0x7f7df006c120 msgr2=0x7f7df0152860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:06.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.300+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 shutdown_connections 2024-07-17T08:40:06.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.301+0000 7f7df6800640 1 -- 172.21.15.44:0/474494549 wait complete. 2024-07-17T08:40:06.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:40:06.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 -- 172.21.15.44:0/2330263020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc45815b340 msgr2=0x7fc4580ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 --2- 172.21.15.44:0/2330263020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc45815b340 0x7fc4580ce4b0 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fc458073410 tx=0x7fc44002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 -- 172.21.15.44:0/2330263020 shutdown_connections 2024-07-17T08:40:06.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 --2- 172.21.15.44:0/2330263020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4580ae8d0 0x7fc4580ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 --2- 172.21.15.44:0/2330263020 >> v2:172.21.15.175:3300/0 conn(0x7fc45815b720 0x7fc4580ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 --2- 172.21.15.44:0/2330263020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc45815b340 0x7fc4580ce4b0 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 -- 172.21.15.44:0/2330263020 >> 172.21.15.44:0/2330263020 conn(0x7fc45809aa30 msgr2=0x7fc4580b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 -- 172.21.15.44:0/2330263020 shutdown_connections 2024-07-17T08:40:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.532+0000 7fc45ea00640 1 -- 172.21.15.44:0/2330263020 wait complete. 2024-07-17T08:40:06.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.534+0000 7fc45ea00640 1 Processor -- start 2024-07-17T08:40:06.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.534+0000 7fc45ea00640 1 -- start start 2024-07-17T08:40:06.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc45815b340 0x7fc4580b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc45815b720 0x7fc4580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580cd800 con 0x7fc4580ae8d0 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580cd680 con 0x7fc45815b720 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.535+0000 7fc45ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4580cf2e0 con 0x7fc45815b340 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38138/0 (socket says 172.21.15.44:38138) 2024-07-17T08:40:06.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 -- 172.21.15.44:0/961635978 learned_addr learned my addr 172.21.15.44:0/961635978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457400640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc45815b340 0x7fc4580b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc45815b340 msgr2=0x7fc4580b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc45815b340 0x7fc4580b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc45815b720 msgr2=0x7fc4580b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc45815b720 0x7fc4580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 -- 172.21.15.44:0/961635978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4581f50d0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.536+0000 7fc457e00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fc440009850 tx=0x7fc440003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.537+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44003f070 con 0x7fc4580ae8d0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.537+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4400047a0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.537+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4581f4310 con 0x7fc4580ae8d0 2024-07-17T08:40:06.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.537+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440035030 con 0x7fc4580ae8d0 2024-07-17T08:40:06.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.537+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4581f48d0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.538+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4580ce4b0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.539+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 170) ==== 98846+0+0 (secure 0 0 0) 0x7fc440042070 con 0x7fc4580ae8d0 2024-07-17T08:40:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.540+0000 7fc44fe00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 0x7fc4340790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.540+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fc4400b95f0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.541+0000 7fc457400640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 msgr2=0x7fc4340790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.541+0000 7fc457400640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 0x7fc4340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:40:06.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.545+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc440082630 con 0x7fc4580ae8d0 2024-07-17T08:40:06.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.742+0000 7fc457400640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 msgr2=0x7fc4340790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:06.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.742+0000 7fc457400640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 0x7fc4340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:40:06.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.757+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc4580b06e0 con 0x7fc4580ae8d0 2024-07-17T08:40:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.765+0000 7fc44fe00640 1 -- 172.21.15.44:0/961635978 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v89)=0 v89) ==== 138+0+0 (secure 0 0 0) 0x7fc440081f30 con 0x7fc4580ae8d0 2024-07-17T08:40:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.767+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 msgr2=0x7fc4340790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:40:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.767+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 0x7fc4340790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.767+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 msgr2=0x7fc4580b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.767+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fc440009850 tx=0x7fc440003910 comp rx=0 tx=0).stop 2024-07-17T08:40:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 shutdown_connections 2024-07-17T08:40:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fc434076c00 0x7fc4340790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc45815b720 0x7fc4580b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc45815b340 0x7fc4580b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 --2- 172.21.15.44:0/961635978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4580ae8d0 0x7fc4580b6830 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 >> 172.21.15.44:0/961635978 conn(0x7fc45809aa30 msgr2=0x7fc458070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 shutdown_connections 2024-07-17T08:40:06.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.768+0000 7fc45ea00640 1 -- 172.21.15.44:0/961635978 wait complete. 2024-07-17T08:40:06.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 -- 172.21.15.44:0/2613955663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 msgr2=0x7fe410166e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 --2- 172.21.15.44:0/2613955663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe410166e40 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fe408005fd0 tx=0x7fe40802a940 comp rx=0 tx=0).stop 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 -- 172.21.15.44:0/2613955663 shutdown_connections 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 --2- 172.21.15.44:0/2613955663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe410166e40 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 --2- 172.21.15.44:0/2613955663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 0x7fe4101648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 -- 172.21.15.44:0/2613955663 >> 172.21.15.44:0/2613955663 conn(0x7fe4100d3700 msgr2=0x7fe4100d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 -- 172.21.15.44:0/2613955663 shutdown_connections 2024-07-17T08:40:06.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.980+0000 7fe417000640 1 -- 172.21.15.44:0/2613955663 wait complete. 2024-07-17T08:40:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.982+0000 7fe417000640 1 Processor -- start 2024-07-17T08:40:06.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.982+0000 7fe417000640 1 -- start start 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 0x7fe4100cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4100ce1d0 0x7fe410156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe410167710 con 0x7fe4100b29e0 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe410167590 con 0x7fe4100ce1d0 2024-07-17T08:40:06.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe417000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe410167890 con 0x7fe4100b6790 2024-07-17T08:40:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.983+0000 7fe40fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38154/0 (socket says 172.21.15.44:38154) 2024-07-17T08:40:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 -- 172.21.15.44:0/1455630406 learned_addr learned my addr 172.21.15.44:0/1455630406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:40:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe407e00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 0x7fe4100cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:06.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe415000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4100ce1d0 0x7fe410156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 msgr2=0x7fe4100cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 0x7fe4100cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4100ce1d0 msgr2=0x7fe410156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4100ce1d0 0x7fe410156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 -- 172.21.15.44:0/1455630406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe410159780 con 0x7fe4100b29e0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.984+0000 7fe40fe00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c3d0 tx=0x7fe40000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.985+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400011790 con 0x7fe4100b29e0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.985+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe400017070 con 0x7fe4100b29e0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.985+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400011cc0 con 0x7fe4100b29e0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.985+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4100c7360 con 0x7fe4100b29e0 2024-07-17T08:40:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.985+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4100c7920 con 0x7fe4100b29e0 2024-07-17T08:40:06.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.986+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4100ce5b0 con 0x7fe4100b29e0 2024-07-17T08:40:06.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.988+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 170) ==== 98846+0+0 (secure 0 0 0) 0x7fe40001c460 con 0x7fe4100b29e0 2024-07-17T08:40:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.989+0000 7fe40cc00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 0x7fe3ec0790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:06.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.989+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fe400098590 con 0x7fe4100b29e0 2024-07-17T08:40:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.989+0000 7fe407e00640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 msgr2=0x7fe3ec0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.989+0000 7fe407e00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 0x7fe3ec0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:40:06.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:06.997+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe400061660 con 0x7fe4100b29e0 2024-07-17T08:40:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.190+0000 7fe407e00640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 msgr2=0x7fe3ec0790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:07.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.190+0000 7fe407e00640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 0x7fe3ec0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:40:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.205+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe4100c7af0 con 0x7fe4100b29e0 2024-07-17T08:40:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.209+0000 7fe40cc00640 1 -- 172.21.15.44:0/1455630406 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v90)=0 v90) ==== 133+0+0 (secure 0 0 0) 0x7fe400060f60 con 0x7fe4100b29e0 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 msgr2=0x7fe3ec0790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 0x7fe3ec0790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 msgr2=0x7fe4100c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c3d0 tx=0x7fe40000c890 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 shutdown_connections 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fe3ec076c00 0x7fe3ec0790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4100ce1d0 0x7fe410156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4100b6790 0x7fe4100cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 --2- 172.21.15.44:0/1455630406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4100b29e0 0x7fe4100c7e40 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 >> 172.21.15.44:0/1455630406 conn(0x7fe4100d3700 msgr2=0x7fe410158230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 shutdown_connections 2024-07-17T08:40:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.212+0000 7fe417000640 1 -- 172.21.15.44:0/1455630406 wait complete. 2024-07-17T08:40:07.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv02 2024-07-17T08:40:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 -- 172.21.15.44:0/4247591083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b2210 msgr2=0x7fbe7806bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 --2- 172.21.15.44:0/4247591083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b2210 0x7fbe7806bf40 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fbe780b5510 tx=0x7fbe7002aa30 comp rx=0 tx=0).stop 2024-07-17T08:40:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 -- 172.21.15.44:0/4247591083 shutdown_connections 2024-07-17T08:40:07.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 --2- 172.21.15.44:0/4247591083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe780b3040 0x7fbe780b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 --2- 172.21.15.44:0/4247591083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b2210 0x7fbe7806bf40 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 --2- 172.21.15.44:0/4247591083 >> v2:172.21.15.175:3300/0 conn(0x7fbe7815b3a0 0x7fbe780d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 -- 172.21.15.44:0/4247591083 >> 172.21.15.44:0/4247591083 conn(0x7fbe780b5560 msgr2=0x7fbe780b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 -- 172.21.15.44:0/4247591083 shutdown_connections 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.416+0000 7fbe77e00640 1 -- 172.21.15.44:0/4247591083 wait complete. 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.418+0000 7fbe77e00640 1 Processor -- start 2024-07-17T08:40:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.418+0000 7fbe77e00640 1 -- start start 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780b2210 0x7fbe7814b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 0x7fbe780ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe7815b3a0 0x7fbe780cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7806c810 con 0x7fbe780b3040 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7806c690 con 0x7fbe780b2210 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe77e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7806c990 con 0x7fbe7815b3a0 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe76000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 0x7fbe780ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe76000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 0x7fbe780ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38172/0 (socket says 172.21.15.44:38172) 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.419+0000 7fbe76000640 1 -- 172.21.15.44:0/3245506174 learned_addr learned my addr 172.21.15.44:0/3245506174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe77400640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe7815b3a0 0x7fbe780cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:07.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780b2210 0x7fbe7814b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe7815b3a0 msgr2=0x7fbe780cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe7815b3a0 0x7fbe780cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780b2210 msgr2=0x7fbe7814b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780b2210 0x7fbe7814b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7815bb20 con 0x7fbe780b3040 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe780b2210 0x7fbe7814b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:40:07.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe76000640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 0x7fbe780ce6e0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400d750 tx=0x7fbe6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.420+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64017070 con 0x7fbe780b3040 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.421+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe64009cc0 con 0x7fbe780b3040 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.421+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6400de60 con 0x7fbe780b3040 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.421+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7815a980 con 0x7fbe780b3040 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.421+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7814b990 con 0x7fbe780b3040 2024-07-17T08:40:07.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.422+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe780b1cd0 con 0x7fbe780b3040 2024-07-17T08:40:07.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.423+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 170) ==== 98846+0+0 (secure 0 0 0) 0x7fbe64005280 con 0x7fbe780b3040 2024-07-17T08:40:07.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.425+0000 7fbe6ea00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:07.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.425+0000 7fbe76a00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.425+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(293..293 src has 1..293) ==== 6705+0+0 (secure 0 0 0) 0x7fbe640987e0 con 0x7fbe780b3040 2024-07-17T08:40:07.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.425+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:40:07.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.429+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbe640618b0 con 0x7fbe780b3040 2024-07-17T08:40:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.627+0000 7fbe76a00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:07.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.627+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:40:07.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:07.671+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv02", "target": ["mon-mgr", ""]}) -- 0x7fbe7815b950 con 0x7fbe58076c00 2024-07-17T08:40:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:08.029+0000 7fbe76a00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:08.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:08.029+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:40:08.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:08.830+0000 7fbe76a00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:08.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:08.830+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:40:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:10.432+0000 7fbe76a00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1518538177 2024-07-17T08:40:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:10.432+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:40:10.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:10.515+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 171) ==== 98557+0+0 (secure 0 0 0) 0x7fbe6405d440 con 0x7fbe780b3040 2024-07-17T08:40:10.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:10.516+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 msgr2=0x7fbe580790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:40:10.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:10.516+0000 7fbe6ea00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1518538177,v1:172.21.15.175:6837/1518538177] conn(0x7fbe58076c00 0x7fbe580790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91be600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91be600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91be600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91be600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91be600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91c2200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91c2200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91c2200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91c2200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.817 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:10.816+0000 7f91c2200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:10.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:40:10.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:40:11.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:11.528+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 172) ==== 98684+0+0 (secure 0 0 0) 0x7fbe6406ae50 con 0x7fbe780b3040 2024-07-17T08:40:11.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:11.529+0000 7fbe6ea00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fbe5803bb50 0x7fbe58081870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:40:11.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:11.529+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 --> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv02", "target": ["mon-mgr", ""]}) -- 0x7fbe6401ee70 con 0x7fbe5803bb50 2024-07-17T08:40:11.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:11.531+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fbe5803bb50 0x7fbe58081870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:40:11.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:11.547+0000 7fbe76a00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fbe5803bb50 0x7fbe58081870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005c90 tx=0x7fbe680093d0 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:40:11.566 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:11.566+0000 7f91baa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:11.566 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:11.566+0000 7f91baa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:11.567 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:11.566+0000 7f91baa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:11.567 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:11.566+0000 7f91baa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:11.567 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:40:11.566+0000 7f91baa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:40:12.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:12.581+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 9 ==== mgrmap(e 173) ==== 99000+0+0 (secure 0 0 0) 0x7fbe64013cd0 con 0x7fbe780b3040 2024-07-17T08:40:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:40:14.597+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mon.0 v2:172.21.15.44:3300/0 10 ==== mgrmap(e 174) ==== 99056+0+0 (secure 0 0 0) 0x7fbe6405dd10 con 0x7fbe780b3040 2024-07-17T08:40:41.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:40:41.223 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:41:09.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.322+0000 7fbe6ea00640 1 -- 172.21.15.44:0/3245506174 <== mgr.9871 v2:172.21.15.175:6834/1750219097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe6401ee70 con 0x7fbe5803bb50 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fbe5803bb50 msgr2=0x7fbe58081870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fbe5803bb50 0x7fbe58081870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005c90 tx=0x7fbe680093d0 comp rx=0 tx=0).stop 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 msgr2=0x7fbe780ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 --2- 172.21.15.44:0/3245506174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe780b3040 0x7fbe780ce6e0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400d750 tx=0x7fbe6400dc10 comp rx=0 tx=0).stop 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77400640 1 -- 172.21.15.44:0/3245506174 reap_dead start 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 shutdown_connections 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 >> 172.21.15.44:0/3245506174 conn(0x7fbe780b5560 msgr2=0x7fbe78157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 shutdown_connections 2024-07-17T08:41:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.326+0000 7fbe77e00640 1 -- 172.21.15.44:0/3245506174 wait complete. 2024-07-17T08:41:09.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv02 --group mygrp02 . 1m 2024-07-17T08:41:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 -- 172.21.15.44:0/3614738942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c151f20 msgr2=0x7fc76c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 --2- 172.21.15.44:0/3614738942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c151f20 0x7fc76c15c140 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fc760005fd0 tx=0x7fc76002a540 comp rx=0 tx=0).stop 2024-07-17T08:41:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 -- 172.21.15.44:0/3614738942 shutdown_connections 2024-07-17T08:41:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 --2- 172.21.15.44:0/3614738942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc76c0cd0b0 0x7fc76c06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 --2- 172.21.15.44:0/3614738942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c151f20 0x7fc76c15c140 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 -- 172.21.15.44:0/3614738942 >> 172.21.15.44:0/3614738942 conn(0x7fc76c0cf0d0 msgr2=0x7fc76c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 -- 172.21.15.44:0/3614738942 shutdown_connections 2024-07-17T08:41:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.574+0000 7fc772400640 1 -- 172.21.15.44:0/3614738942 wait complete. 2024-07-17T08:41:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.576+0000 7fc772400640 1 Processor -- start 2024-07-17T08:41:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.576+0000 7fc772400640 1 -- start start 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c0cd0b0 0x7fc76c1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc76c151f20 0x7fc76c098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc76c152300 0x7fc76c1e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c0cb790 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c06f610 con 0x7fc76c152300 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc772400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c0cb910 con 0x7fc76c151f20 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc771000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c0cd0b0 0x7fc76c1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc771000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c0cd0b0 0x7fc76c1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34432/0 (socket says 172.21.15.44:34432) 2024-07-17T08:41:09.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.578+0000 7fc771000640 1 -- 172.21.15.44:0/3602215335 learned_addr learned my addr 172.21.15.44:0/3602215335 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771a00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc76c152300 0x7fc76c1e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc76be00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc76c151f20 0x7fc76c098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 -- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc76c151f20 msgr2=0x7fc76c098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc76c151f20 0x7fc76c098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 -- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc76c152300 msgr2=0x7fc76c1e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc76c152300 0x7fc76c1e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 -- 172.21.15.44:0/3602215335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c14dc20 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771a00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc76c152300 0x7fc76c1e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.579+0000 7fc771000640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc76c0cd0b0 0x7fc76c1527e0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc7600039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.580+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76003f070 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.580+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc760004890 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.580+0000 7fc772400640 1 -- 172.21.15.44:0/3602215335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c14f1b0 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.580+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760035030 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.580+0000 7fc772400640 1 -- 172.21.15.44:0/3602215335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c14f450 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.581+0000 7fc772400640 1 -- 172.21.15.44:0/3602215335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c0cc3f0 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.582+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 174) ==== 99056+0+0 (secure 0 0 0) 0x7fc760042070 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.583+0000 7fc768c00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc74c076e40 0x7fc74c079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.583+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(294..294 src has 1..294) ==== 6963+0+0 (secure 0 0 0) 0x7fc7600b9bb0 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.584+0000 7fc76be00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc74c076e40 0x7fc74c079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.585+0000 7fc76be00640 1 --2- 172.21.15.44:0/3602215335 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc74c076e40 0x7fc74c079300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc76c15afd0 tx=0x7fc758005cc0 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.588+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc76c0cc3f0 con 0x7fc76c0cd0b0 2024-07-17T08:41:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.830+0000 7fc772400640 1 -- 172.21.15.44:0/3602215335 --> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv02", "group": "mygrp02", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc76c06ec70 con 0x7fc74c076e40 2024-07-17T08:41:09.837 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:09.836+0000 7f91cae00640 -1 mgr.server reply reply (22) Invalid argument No such subvol: mygrp02::sv02 2024-07-17T08:41:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:09.837+0000 7fc768c00640 1 -- 172.21.15.44:0/3602215335 <== mgr.9871 v2:172.21.15.175:6834/1750219097 1 ==== mgr_command_reply(tid 0: -22 No such subvol: mygrp02::sv02) ==== 37+0+0 (secure 0 0 0) 0x7fc76c06ec70 con 0x7fc74c076e40 2024-07-17T08:41:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: No such subvol: mygrp02::sv02 2024-07-17T08:41:09.859 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:41:09.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv02 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 -- 172.21.15.44:0/2902161580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 msgr2=0x7fb47c06eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 --2- 172.21.15.44:0/2902161580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c06eb90 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7fb47c06f250 tx=0x7fb46402b930 comp rx=0 tx=0).stop 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 -- 172.21.15.44:0/2902161580 shutdown_connections 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 --2- 172.21.15.44:0/2902161580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c06eb90 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 --2- 172.21.15.44:0/2902161580 >> v2:172.21.15.175:3300/0 conn(0x7fb47c06f6d0 0x7fb47c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 -- 172.21.15.44:0/2902161580 >> 172.21.15.44:0/2902161580 conn(0x7fb47c0b0eb0 msgr2=0x7fb47c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 -- 172.21.15.44:0/2902161580 shutdown_connections 2024-07-17T08:41:10.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.080+0000 7fb483200640 1 -- 172.21.15.44:0/2902161580 wait complete. 2024-07-17T08:41:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.082+0000 7fb483200640 1 Processor -- start 2024-07-17T08:41:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.082+0000 7fb483200640 1 -- start start 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 0x7fb47c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 0x7fb47c1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c0aea90 con 0x7fb47c0b3280 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c0ae910 con 0x7fb47c0ac3a0 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb483200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c0aec10 con 0x7fb47c06f6d0 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb481200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb481200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34436/0 (socket says 172.21.15.44:34436) 2024-07-17T08:41:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.083+0000 7fb481200640 1 -- 172.21.15.44:0/3641113102 learned_addr learned my addr 172.21.15.44:0/3641113102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb473e00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 0x7fb47c1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb47be00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 0x7fb47c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 -- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 msgr2=0x7fb47c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 0x7fb47c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 -- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 msgr2=0x7fb47c1f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 0x7fb47c1f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 -- 172.21.15.44:0/3641113102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c0728e0 con 0x7fb47c0b3280 2024-07-17T08:41:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb473e00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 0x7fb47c1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb47be00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 0x7fb47c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.084+0000 7fb481200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c7a0 tx=0x7fb47400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.085+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474003b70 con 0x7fb47c0b3280 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.085+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb474003d30 con 0x7fb47c0b3280 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.085+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400aa50 con 0x7fb47c0b3280 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.085+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c072bc0 con 0x7fb47c0b3280 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.085+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c073180 con 0x7fb47c0b3280 2024-07-17T08:41:10.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.086+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c15a5a0 con 0x7fb47c0b3280 2024-07-17T08:41:10.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.087+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 174) ==== 99056+0+0 (secure 0 0 0) 0x7fb47400abf0 con 0x7fb47c0b3280 2024-07-17T08:41:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.088+0000 7fb478c00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 0x7fb458079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.088+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(294..294 src has 1..294) ==== 6963+0+0 (secure 0 0 0) 0x7fb474098a70 con 0x7fb47c0b3280 2024-07-17T08:41:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.089+0000 7fb47be00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 0x7fb458079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.090+0000 7fb47be00640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 0x7fb458079330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009b40 tx=0x7fb46c005e50 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:10.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.093+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb4740616e0 con 0x7fb47c0b3280 2024-07-17T08:41:10.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.335+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 --> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv02", "target": ["mon-mgr", ""]}) -- 0x7fb47c06ee00 con 0x7fb458076e70 2024-07-17T08:41:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.370+0000 7fb478c00640 1 -- 172.21.15.44:0/3641113102 <== mgr.9871 v2:172.21.15.175:6834/1750219097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb47c06ee00 con 0x7fb458076e70 2024-07-17T08:41:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 msgr2=0x7fb458079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 0x7fb458079330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009b40 tx=0x7fb46c005e50 comp rx=0 tx=0).stop 2024-07-17T08:41:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 msgr2=0x7fb47c073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c7a0 tx=0x7fb47400cc60 comp rx=0 tx=0).stop 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 shutdown_connections 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fb458076e70 0x7fb458079330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb47c0b3280 0x7fb47c073980 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb47c0ac3a0 0x7fb47c1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 --2- 172.21.15.44:0/3641113102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb47c06f6d0 0x7fb47c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.373+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 >> 172.21.15.44:0/3641113102 conn(0x7fb47c0b0eb0 msgr2=0x7fb47c15c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.374+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 shutdown_connections 2024-07-17T08:41:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.374+0000 7fb483200640 1 -- 172.21.15.44:0/3641113102 wait complete. 2024-07-17T08:41:10.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:41:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:10.396+0000 7f91bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:10.396+0000 7f91bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:10.396+0000 7f91bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:10.396+0000 7f91bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:10.396+0000 7f91bd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.601+0000 7fa69d000640 1 -- 172.21.15.44:0/1596723062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 msgr2=0x7fa6980b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.601+0000 7fa69d000640 1 --2- 172.21.15.44:0/1596723062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa6980b1f00 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa68802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.602+0000 7fa69d000640 1 -- 172.21.15.44:0/1596723062 shutdown_connections 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.602+0000 7fa69d000640 1 --2- 172.21.15.44:0/1596723062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa6980b1f00 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.602+0000 7fa69d000640 1 -- 172.21.15.44:0/1596723062 >> 172.21.15.44:0/1596723062 conn(0x7fa698073d50 msgr2=0x7fa69809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.602+0000 7fa69d000640 1 -- 172.21.15.44:0/1596723062 shutdown_connections 2024-07-17T08:41:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.602+0000 7fa69d000640 1 -- 172.21.15.44:0/1596723062 wait complete. 2024-07-17T08:41:10.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.604+0000 7fa69d000640 1 Processor -- start 2024-07-17T08:41:10.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.604+0000 7fa69d000640 1 -- start start 2024-07-17T08:41:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6980b0fe0 0x7fa6981f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 0x7fa6981465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981666b0 con 0x7fa69815b5d0 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6980b04a0 con 0x7fa6980b0fe0 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.605+0000 7fa69d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa698166830 con 0x7fa6980b5620 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34454/0 (socket says 172.21.15.44:34454) 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 -- 172.21.15.44:0/4167671783 learned_addr learned my addr 172.21.15.44:0/4167671783 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa695e00640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6980b0fe0 0x7fa6981f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa695400640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 0x7fa6981465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 -- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 msgr2=0x7fa6981465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 0x7fa6981465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 -- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6980b0fe0 msgr2=0x7fa6981f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6980b0fe0 0x7fa6981f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 -- 172.21.15.44:0/4167671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa698070950 con 0x7fa69815b5d0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa695400640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 0x7fa6981465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.606+0000 7fa696800640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c780 tx=0x7fa69000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.607+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa69815b5d0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.607+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa690005070 con 0x7fa69815b5d0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.607+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa698070c30 con 0x7fa69815b5d0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.607+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690005500 con 0x7fa69815b5d0 2024-07-17T08:41:10.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.607+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa698146b20 con 0x7fa69815b5d0 2024-07-17T08:41:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.608+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa698070e70 con 0x7fa69815b5d0 2024-07-17T08:41:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.610+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 174) ==== 99056+0+0 (secure 0 0 0) 0x7fa69001a070 con 0x7fa69815b5d0 2024-07-17T08:41:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.610+0000 7fa68e000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 0x7fa674079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.611+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(294..294 src has 1..294) ==== 6963+0+0 (secure 0 0 0) 0x7fa6900982d0 con 0x7fa69815b5d0 2024-07-17T08:41:10.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.611+0000 7fa695e00640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 0x7fa674079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:10.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.612+0000 7fa695e00640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 0x7fa674079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa688033000 tx=0x7fa688004730 comp rx=0 tx=0).ready entity=mgr.9871 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:10.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.616+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa698070e70 con 0x7fa69815b5d0 2024-07-17T08:41:10.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.655+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 175) ==== 99107+0+0 (secure 0 0 0) 0x7fa6900056a0 con 0x7fa69815b5d0 2024-07-17T08:41:10.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:10.881+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fa6980b5d60 con 0x7fa69815b5d0 2024-07-17T08:41:11.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:41:11.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:41:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.662+0000 7fa68e000640 1 -- 172.21.15.44:0/4167671783 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v176) ==== 93+0+0 (secure 0 0 0) 0x7fa6980b5d60 con 0x7fa69815b5d0 2024-07-17T08:41:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.666+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 msgr2=0x7fa674079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.666+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 0x7fa674079300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa688033000 tx=0x7fa688004730 comp rx=0 tx=0).stop 2024-07-17T08:41:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.666+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 msgr2=0x7fa69806d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.666+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c780 tx=0x7fa69000cc40 comp rx=0 tx=0).stop 2024-07-17T08:41:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 shutdown_connections 2024-07-17T08:41:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fa674076e40 0x7fa674079300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa69815b5d0 0x7fa69806d050 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6980b5620 0x7fa6981465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 --2- 172.21.15.44:0/4167671783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6980b0fe0 0x7fa6981f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 >> 172.21.15.44:0/4167671783 conn(0x7fa698073d50 msgr2=0x7fa6980710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 shutdown_connections 2024-07-17T08:41:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:11.668+0000 7fa69d000640 1 -- 172.21.15.44:0/4167671783 wait complete. 2024-07-17T08:41:11.695 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:41:11.696 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:11.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:11.884 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:41:11.884 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:11.884 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:11.884 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:11.936 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:11.936 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:11.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:41:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 -- 172.21.15.44:0/287208631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 msgr2=0x7fe590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 --2- 172.21.15.44:0/287208631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe590004c80 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fe59800baf0 tx=0x7fe590005830 comp rx=0 tx=0).stop 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 -- 172.21.15.44:0/287208631 shutdown_connections 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 --2- 172.21.15.44:0/287208631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe590004c80 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 -- 172.21.15.44:0/287208631 >> 172.21.15.44:0/287208631 conn(0x7fe5ac0b4ff0 msgr2=0x7fe5ac0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 -- 172.21.15.44:0/287208631 shutdown_connections 2024-07-17T08:41:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.236+0000 7fe5b2400640 1 -- 172.21.15.44:0/287208631 wait complete. 2024-07-17T08:41:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.237+0000 7fe5b2400640 1 Processor -- start 2024-07-17T08:41:12.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.238+0000 7fe5b2400640 1 -- start start 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.238+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac1f8010 0x7fe5ac0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5ac0b7ab0 0x7fe5ac0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac0b2860 con 0x7fe590004800 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac0b26e0 con 0x7fe5ac1f8010 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac0cfdf0 con 0x7fe5ac0b7ab0 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34500/0 (socket says 172.21.15.44:34500) 2024-07-17T08:41:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 -- 172.21.15.44:0/1353596472 learned_addr learned my addr 172.21.15.44:0/1353596472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5ac0b7ab0 msgr2=0x7fe5ac0aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5ac0b7ab0 0x7fe5ac0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac1f8010 msgr2=0x7fe5ac0b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5abe00640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac1f8010 0x7fe5ac0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac1f8010 0x7fe5ac0b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.239+0000 7fe5b1000640 1 -- 172.21.15.44:0/1353596472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac1f4ca0 con 0x7fe590004800 2024-07-17T08:41:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5b1000640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00dbd0 tx=0x7fe59c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c018070 con 0x7fe590004800 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe59c00bbc0 con 0x7fe590004800 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c003970 con 0x7fe590004800 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac1f3ee0 con 0x7fe590004800 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.240+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac1f44a0 con 0x7fe590004800 2024-07-17T08:41:12.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.241+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac0d02c0 con 0x7fe590004800 2024-07-17T08:41:12.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.242+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7fe59c01b070 con 0x7fe590004800 2024-07-17T08:41:12.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.243+0000 7fe5a8c00640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 0x7fe58c079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.243+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7fe59c098680 con 0x7fe590004800 2024-07-17T08:41:12.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.243+0000 7fe5abe00640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 msgr2=0x7fe58c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:12.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.243+0000 7fe5abe00640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 0x7fe58c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:12.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.247+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe59c060be0 con 0x7fe590004800 2024-07-17T08:41:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.444+0000 7fe5abe00640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 msgr2=0x7fe58c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:12.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.444+0000 7fe5abe00640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 0x7fe58c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:12.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.471+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5ac0d0e10 con 0x7fe590004800 2024-07-17T08:41:12.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.472+0000 7fe5a8c00640 1 -- 172.21.15.44:0/1353596472 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v296) ==== 80+0+359 (secure 0 0 0) 0x7fe59c065130 con 0x7fe590004800 2024-07-17T08:41:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/1518538177 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/1518538177 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2395145048 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.473 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3806999519 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.474 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1608213311 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.474 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/676716344 2024-07-18T08:40:10.498668+0000 2024-07-17T08:41:12.474 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T08:41:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 msgr2=0x7fe58c079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 0x7fe58c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 msgr2=0x7fe5ac1f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00dbd0 tx=0x7fe59c00b4b0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 shutdown_connections 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fe58c076e80 0x7fe58c079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5ac0b7ab0 0x7fe5ac0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5ac1f8010 0x7fe5ac0b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 --2- 172.21.15.44:0/1353596472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe590004800 0x7fe5ac1f7ad0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.475+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 >> 172.21.15.44:0/1353596472 conn(0x7fe5ac0b4ff0 msgr2=0x7fe5ac0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.476+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 shutdown_connections 2024-07-17T08:41:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.476+0000 7fe5b2400640 1 -- 172.21.15.44:0/1353596472 wait complete. 2024-07-17T08:41:12.496 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:41:12.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:12.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:12.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:41:12.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:12.587 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:41:12.630 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:12.630 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:12.630 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:12.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:12.673 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:12.673 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:12.673 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:12.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:41:12.714 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:12.714 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:12.714 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:12.758 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:12.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:12.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 -- 172.21.15.44:0/1174408150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fdc0052e0 msgr2=0x7f3fdc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1174408150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fdc0052e0 0x7f3fdc00b9c0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005fd0 tx=0x7f3fdc005830 comp rx=0 tx=0).stop 2024-07-17T08:41:12.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 -- 172.21.15.44:0/1174408150 shutdown_connections 2024-07-17T08:41:12.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1174408150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fdc0052e0 0x7f3fdc00b9c0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 -- 172.21.15.44:0/1174408150 >> 172.21.15.44:0/1174408150 conn(0x7f3ff80d00a0 msgr2=0x7f3ff80d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:12.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 -- 172.21.15.44:0/1174408150 shutdown_connections 2024-07-17T08:41:12.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.958+0000 7f3ffea00640 1 -- 172.21.15.44:0/1174408150 wait complete. 2024-07-17T08:41:12.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.960+0000 7f3ffea00640 1 Processor -- start 2024-07-17T08:41:12.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.960+0000 7f3ffea00640 1 -- start start 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc0052e0 0x7f3ff814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ff81e88e0 0x7f3ff81f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff806e480 con 0x7f3ff815bf90 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80b56f0 con 0x7f3fdc0052e0 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ffea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff806e600 con 0x7f3ff81e88e0 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ff7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34516/0 (socket says 172.21.15.44:34516) 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.961+0000 7f3ff7400640 1 -- 172.21.15.44:0/1738166463 learned_addr learned my addr 172.21.15.44:0/1738166463 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7e00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc0052e0 0x7f3ff814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ffca00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ff81e88e0 0x7f3ff81f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ff81e88e0 msgr2=0x7f3ff81f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ff81e88e0 0x7f3ff81f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc0052e0 msgr2=0x7f3ff814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc0052e0 0x7f3ff814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 -- 172.21.15.44:0/1738166463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff815b1d0 con 0x7f3ff815bf90 2024-07-17T08:41:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3ff7400640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e570 tx=0x7f3fe400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.962+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4015020 con 0x7f3ff815bf90 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.963+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fe400c9e0 con 0x7f3ff815bf90 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.963+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400ce30 con 0x7f3ff815bf90 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.963+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff815b4b0 con 0x7f3ff815bf90 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.963+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff8150520 con 0x7f3ff815bf90 2024-07-17T08:41:12.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.964+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff81e8cf0 con 0x7f3ff815bf90 2024-07-17T08:41:12.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.965+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7f3fe4018020 con 0x7f3ff815bf90 2024-07-17T08:41:12.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.966+0000 7f3fefe00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 0x7f3fd8079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:12.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.966+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7f3fe40991b0 con 0x7f3ff815bf90 2024-07-17T08:41:12.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.966+0000 7f3ff7e00640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 msgr2=0x7f3fd8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:12.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.966+0000 7f3ff7e00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 0x7f3fd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:12.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:12.970+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3ff81e8cf0 con 0x7f3ff815bf90 2024-07-17T08:41:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.168+0000 7f3ff7e00640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 msgr2=0x7f3fd8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.168+0000 7f3ff7e00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 0x7f3fd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:13.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.218+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff8161020 con 0x7f3ff815bf90 2024-07-17T08:41:13.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.220+0000 7f3fefe00640 1 -- 172.21.15.44:0/1738166463 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f3fe401f020 con 0x7f3ff815bf90 2024-07-17T08:41:13.221 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:13.221 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:41:13.221 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 msgr2=0x7f3fd8079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 0x7f3fd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 msgr2=0x7f3ff81e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e570 tx=0x7f3fe400ea30 comp rx=0 tx=0).stop 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 shutdown_connections 2024-07-17T08:41:13.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3fd8076eb0 0x7f3fd8079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ff81e88e0 0x7f3ff81f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ff815bf90 0x7f3ff81e83a0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 --2- 172.21.15.44:0/1738166463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc0052e0 0x7f3ff814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 >> 172.21.15.44:0/1738166463 conn(0x7f3ff80d00a0 msgr2=0x7f3ff80b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 shutdown_connections 2024-07-17T08:41:13.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.223+0000 7f3ffea00640 1 -- 172.21.15.44:0/1738166463 wait complete. 2024-07-17T08:41:13.244 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:13.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 -- 172.21.15.44:0/3465395372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 msgr2=0x7f4094003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 --2- 172.21.15.44:0/3465395372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f4094003e50 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fc0 tx=0x7f4094005830 comp rx=0 tx=0).stop 2024-07-17T08:41:13.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b2200640 1 -- 172.21.15.44:0/3465395372 reap_dead start 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 -- 172.21.15.44:0/3465395372 shutdown_connections 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 -- 172.21.15.44:0/3465395372 >> 172.21.15.44:0/3465395372 conn(0x7f40ac0b0fb0 msgr2=0x7f40ac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 -- 172.21.15.44:0/3465395372 shutdown_connections 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.433+0000 7f40b4200640 1 -- 172.21.15.44:0/3465395372 wait complete. 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.435+0000 7f40b4200640 1 Processor -- start 2024-07-17T08:41:13.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.435+0000 7f40b4200640 1 -- start start 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40ac0cf3a0 0x7f40ac0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ac1f36a0 0x7f40ac1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac0ac7e0 con 0x7f4094003a30 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac0ac660 con 0x7f40ac1f36a0 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b4200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac0ac960 con 0x7f40ac0cf3a0 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34532/0 (socket says 172.21.15.44:34532) 2024-07-17T08:41:13.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.436+0000 7f40b1800640 1 -- 172.21.15.44:0/2848467543 learned_addr learned my addr 172.21.15.44:0/2848467543 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b0e00640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40ac0cf3a0 0x7f40ac0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b2200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ac1f36a0 0x7f40ac1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40ac0cf3a0 msgr2=0x7f40ac0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40ac0cf3a0 0x7f40ac0cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ac1f36a0 msgr2=0x7f40ac1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ac1f36a0 0x7f40ac1f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 -- 172.21.15.44:0/2848467543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac1f50e0 con 0x7f4094003a30 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b1800640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c290 tx=0x7f40a0001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0032030 con 0x7f4094003a30 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40a0003af0 con 0x7f4094003a30 2024-07-17T08:41:13.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac1f4500 con 0x7f4094003a30 2024-07-17T08:41:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.437+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0042480 con 0x7f4094003a30 2024-07-17T08:41:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.438+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac1f4ac0 con 0x7f4094003a30 2024-07-17T08:41:13.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.439+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac0cdb20 con 0x7f4094003a30 2024-07-17T08:41:13.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.440+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7f40a0035020 con 0x7f4094003a30 2024-07-17T08:41:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.441+0000 7f40a9600640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 0x7f408c079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.441+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7f40a00b99a0 con 0x7f4094003a30 2024-07-17T08:41:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.441+0000 7f40b0e00640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 msgr2=0x7f408c079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:13.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.441+0000 7f40b0e00640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 0x7f408c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.445+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f40a0047060 con 0x7f4094003a30 2024-07-17T08:41:13.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.643+0000 7f40b0e00640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 msgr2=0x7f408c079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:13.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.643+0000 7f40b0e00640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 0x7f408c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:13.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.691+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40ac15da70 con 0x7f4094003a30 2024-07-17T08:41:13.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.693+0000 7f40a9600640 1 -- 172.21.15.44:0/2848467543 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f40a0082490 con 0x7f4094003a30 2024-07-17T08:41:13.694 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:13.694 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:41:13.694 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 msgr2=0x7f408c079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 0x7f408c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 msgr2=0x7f40ac073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c290 tx=0x7f40a0001da0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 shutdown_connections 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f408c076eb0 0x7f408c079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ac1f36a0 0x7f40ac1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40ac0cf3a0 0x7f40ac0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 --2- 172.21.15.44:0/2848467543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4094003a30 0x7f40ac073fe0 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 >> 172.21.15.44:0/2848467543 conn(0x7f40ac0b0fb0 msgr2=0x7f40ac15df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:13.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 shutdown_connections 2024-07-17T08:41:13.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.696+0000 7f40b4200640 1 -- 172.21.15.44:0/2848467543 wait complete. 2024-07-17T08:41:13.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 -- 172.21.15.44:0/993564067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb680b2ad0 msgr2=0x7fcb680b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 --2- 172.21.15.44:0/993564067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb680b2ad0 0x7fcb680b5200 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fcb68069c00 tx=0x7fcb5002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 -- 172.21.15.44:0/993564067 shutdown_connections 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 --2- 172.21.15.44:0/993564067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 0x7fcb68161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 --2- 172.21.15.44:0/993564067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb680b2ad0 0x7fcb680b5200 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 --2- 172.21.15.44:0/993564067 >> v2:172.21.15.175:3300/0 conn(0x7fcb6815d600 0x7fcb6806e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 -- 172.21.15.44:0/993564067 >> 172.21.15.44:0/993564067 conn(0x7fcb680d00a0 msgr2=0x7fcb680d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 -- 172.21.15.44:0/993564067 shutdown_connections 2024-07-17T08:41:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.909+0000 7fcb6f600640 1 -- 172.21.15.44:0/993564067 wait complete. 2024-07-17T08:41:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.910+0000 7fcb6f600640 1 Processor -- start 2024-07-17T08:41:13.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.911+0000 7fcb6f600640 1 -- start start 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb680b2ad0 0x7fcb6815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 0x7fcb68147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb68161770 con 0x7fcb6815d600 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6806eb60 con 0x7fcb680b2ad0 2024-07-17T08:41:13.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb681618f0 con 0x7fcb680b5740 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34544/0 (socket says 172.21.15.44:34544) 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 -- 172.21.15.44:0/357544606 learned_addr learned my addr 172.21.15.44:0/357544606 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb5fe00640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 0x7fcb68147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6cc00640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb680b2ad0 0x7fcb6815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 msgr2=0x7fcb68147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 0x7fcb68147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb680b2ad0 msgr2=0x7fcb6815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb680b2ad0 0x7fcb6815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.912+0000 7fcb6d600640 1 -- 172.21.15.44:0/357544606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6815f860 con 0x7fcb6815d600 2024-07-17T08:41:13.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb6d600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000ead0 tx=0x7fcb6000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000cde0 con 0x7fcb6815d600 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb60004c10 con 0x7fcb6815d600 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60005020 con 0x7fcb6815d600 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6815fb40 con 0x7fcb6815d600 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.913+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb68160100 con 0x7fcb6815d600 2024-07-17T08:41:13.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.914+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb68161da0 con 0x7fcb6815d600 2024-07-17T08:41:13.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.916+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7fcb600051c0 con 0x7fcb6815d600 2024-07-17T08:41:13.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.917+0000 7fcb65600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 0x7fcb44079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:13.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.917+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7fcb600996f0 con 0x7fcb6815d600 2024-07-17T08:41:13.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.917+0000 7fcb6cc00640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 msgr2=0x7fcb44079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:13.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.917+0000 7fcb6cc00640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 0x7fcb44079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:13.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:13.925+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcb60016030 con 0x7fcb6815d600 2024-07-17T08:41:14.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.119+0000 7fcb6cc00640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 msgr2=0x7fcb44079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:14.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.119+0000 7fcb6cc00640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 0x7fcb44079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:14.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.136+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb680b6120 con 0x7fcb6815d600 2024-07-17T08:41:14.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.138+0000 7fcb65600640 1 -- 172.21.15.44:0/357544606 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v296) ==== 74+0+18322 (secure 0 0 0) 0x7fcb600621e0 con 0x7fcb6815d600 2024-07-17T08:41:14.139 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:14.139 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":296,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:11.646055+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:39:46.575840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":40,"pool_name":"cephfs_data","create_time":"2024-07-17T08:39:47.617938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"293","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"40.13","mappings":[{"from":5,"to":1}]},{"pgid":"40.18","mappings":[{"from":5,"to":1}]},{"pgid":"40.31","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:6834/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/2395145048":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/3806999519":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/1608213311":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/676716344":"2024-07-18T08:40:10.498668+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:14.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 msgr2=0x7fcb44079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:14.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 0x7fcb44079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 msgr2=0x7fcb68144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000ead0 tx=0x7fcb6000ef90 comp rx=0 tx=0).stop 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 shutdown_connections 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fcb44076e80 0x7fcb44079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb6815d600 0x7fcb68144770 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb680b5740 0x7fcb68147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 --2- 172.21.15.44:0/357544606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb680b2ad0 0x7fcb6815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 >> 172.21.15.44:0/357544606 conn(0x7fcb680d00a0 msgr2=0x7fcb680cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.141+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 shutdown_connections 2024-07-17T08:41:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.142+0000 7fcb6f600640 1 -- 172.21.15.44:0/357544606 wait complete. 2024-07-17T08:41:14.161 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:41:14.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 -- 172.21.15.44:0/2332168989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 msgr2=0x7f48c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 --2- 172.21.15.44:0/2332168989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48c000b9c0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f48d0001d90 tx=0x7f48c0005830 comp rx=0 tx=0).stop 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 -- 172.21.15.44:0/2332168989 shutdown_connections 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 --2- 172.21.15.44:0/2332168989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48c000b9c0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 -- 172.21.15.44:0/2332168989 >> 172.21.15.44:0/2332168989 conn(0x7f48d80b2240 msgr2=0x7f48d80b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 -- 172.21.15.44:0/2332168989 shutdown_connections 2024-07-17T08:41:14.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.355+0000 7f48e0600640 1 -- 172.21.15.44:0/2332168989 wait complete. 2024-07-17T08:41:14.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.357+0000 7f48e0600640 1 Processor -- start 2024-07-17T08:41:14.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.357+0000 7f48e0600640 1 -- start start 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d815fbf0 0x7f48d80b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d80b0d60 0x7f48d81409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48d806ef00 con 0x7f48c00052e0 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48d806ed80 con 0x7f48d80b0d60 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48e0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48d806f080 con 0x7f48d815fbf0 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48ddc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34564/0 (socket says 172.21.15.44:34564) 2024-07-17T08:41:14.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.358+0000 7f48ddc00640 1 -- 172.21.15.44:0/3127494483 learned_addr learned my addr 172.21.15.44:0/3127494483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48dd200640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d815fbf0 0x7f48d80b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d815fbf0 msgr2=0x7f48d80b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48de600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d80b0d60 0x7f48d81409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d815fbf0 0x7f48d80b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d80b0d60 msgr2=0x7f48d81409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d80b0d60 0x7f48d81409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 -- 172.21.15.44:0/3127494483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d815eb50 con 0x7f48c00052e0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48ddc00640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e5c0 tx=0x7f48cc00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc015070 con 0x7f48c00052e0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48cc0109a0 con 0x7f48c00052e0 2024-07-17T08:41:14.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc010df0 con 0x7f48c00052e0 2024-07-17T08:41:14.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.359+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d815ee30 con 0x7f48c00052e0 2024-07-17T08:41:14.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.360+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d815f3f0 con 0x7f48c00052e0 2024-07-17T08:41:14.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.361+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d80ae3f0 con 0x7f48c00052e0 2024-07-17T08:41:14.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.362+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7f48cc018070 con 0x7f48c00052e0 2024-07-17T08:41:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.363+0000 7f48d5600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 0x7f48b8079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.363+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7f48cc0990b0 con 0x7f48c00052e0 2024-07-17T08:41:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.363+0000 7f48dd200640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 msgr2=0x7f48b8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:14.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.363+0000 7f48dd200640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 0x7f48b8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:14.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.367+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48cc061ba0 con 0x7f48c00052e0 2024-07-17T08:41:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.564+0000 7f48dd200640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 msgr2=0x7f48b8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.564+0000 7f48dd200640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 0x7f48b8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.619+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48d8073770 con 0x7f48c00052e0 2024-07-17T08:41:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.620+0000 7f48d5600640 1 -- 172.21.15.44:0/3127494483 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f48cc0614a0 con 0x7f48c00052e0 2024-07-17T08:41:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:14.621 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:41:14.621 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:41:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.622+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 msgr2=0x7f48b8079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.622+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 0x7f48b8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.622+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 msgr2=0x7f48d81f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.622+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e5c0 tx=0x7f48cc00ea80 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 shutdown_connections 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f48b8076eb0 0x7f48b8079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48d80b0d60 0x7f48d81409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d815fbf0 0x7f48d80b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 --2- 172.21.15.44:0/3127494483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00052e0 0x7f48d81f4a60 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 >> 172.21.15.44:0/3127494483 conn(0x7f48d80b2240 msgr2=0x7f48d8073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 shutdown_connections 2024-07-17T08:41:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.623+0000 7f48e0600640 1 -- 172.21.15.44:0/3127494483 wait complete. 2024-07-17T08:41:14.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:14.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 -- 172.21.15.44:0/1199552633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0004d40 msgr2=0x7f76b00051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 --2- 172.21.15.44:0/1199552633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0004d40 0x7f76b00051a0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f76c0001dc0 tx=0x7f76b0005830 comp rx=0 tx=0).stop 2024-07-17T08:41:14.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cde00640 1 -- 172.21.15.44:0/1199552633 reap_dead start 2024-07-17T08:41:14.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 -- 172.21.15.44:0/1199552633 shutdown_connections 2024-07-17T08:41:14.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 -- 172.21.15.44:0/1199552633 >> 172.21.15.44:0/1199552633 conn(0x7f76c80b0fb0 msgr2=0x7f76c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:14.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 -- 172.21.15.44:0/1199552633 shutdown_connections 2024-07-17T08:41:14.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.833+0000 7f76cfe00640 1 -- 172.21.15.44:0/1199552633 wait complete. 2024-07-17T08:41:14.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.835+0000 7f76cfe00640 1 Processor -- start 2024-07-17T08:41:14.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.835+0000 7f76cfe00640 1 -- start start 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 0x7f76c809b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c8150620 0x7f76c81f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80ac7e0 con 0x7f76c8152930 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80ac660 con 0x7f76b0004d40 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80ac960 con 0x7f76c8150620 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34582/0 (socket says 172.21.15.44:34582) 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.836+0000 7f76cca00640 1 -- 172.21.15.44:0/984896556 learned_addr learned my addr 172.21.15.44:0/984896556 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cde00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c8150620 0x7f76c81f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cd400640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 0x7f76c809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:14.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c8150620 msgr2=0x7f76c81f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c8150620 0x7f76c81f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 msgr2=0x7f76c809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 0x7f76c809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 -- 172.21.15.44:0/984896556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8099440 con 0x7f76c8152930 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cd400640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 0x7f76c809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:14.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76cca00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f76b4011a50 tx=0x7f76b4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b401f070 con 0x7f76c8152930 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76b400fdc0 con 0x7f76c8152930 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.837+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b40126b0 con 0x7f76c8152930 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.838+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c8155760 con 0x7f76c8152930 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.838+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81f2ef0 con 0x7f76c8152930 2024-07-17T08:41:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.839+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c80cdb20 con 0x7f76c8152930 2024-07-17T08:41:14.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.841+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7f76b4022070 con 0x7f76c8152930 2024-07-17T08:41:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.842+0000 7f76c5600640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 0x7f76a8079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.842+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7f76b4099710 con 0x7f76c8152930 2024-07-17T08:41:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.842+0000 7f76cd400640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 msgr2=0x7f76a8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.842+0000 7f76cd400640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 0x7f76a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:14.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:14.846+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f76b4061cc0 con 0x7f76c8152930 2024-07-17T08:41:15.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.043+0000 7f76cd400640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 msgr2=0x7f76a8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:15.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.043+0000 7f76cd400640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 0x7f76a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:15.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.092+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76c815c470 con 0x7f76c8152930 2024-07-17T08:41:15.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.094+0000 7f76c5600640 1 -- 172.21.15.44:0/984896556 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5241 (secure 0 0 0) 0x7f76b4066210 con 0x7f76c8152930 2024-07-17T08:41:15.095 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:15.095 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":164,"btime":"2024-07-17T08:39:52:212695+0000","default_fscid":19,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:39:48.630457+0000","modified":"2024-07-17T08:39:51.717689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12228},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12228":{"gid":12228,"name":"a","rank":0,"incarnation":160,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/789460381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":789460381},{"type":"v1","addr":"172.21.15.44:6835","nonce":789460381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[40],"metadata_pool":39,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12228,"qdb_cluster":[12228]},"id":19}]} 2024-07-17T08:41:15.095 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 164 2024-07-17T08:41:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.096+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 msgr2=0x7f76a8079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.096+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 0x7f76a8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.096+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 msgr2=0x7f76c81f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.096+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f76b4011a50 tx=0x7f76b4011f10 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 shutdown_connections 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f76a8076e80 0x7f76a8079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c8150620 0x7f76c81f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76c8152930 0x7f76c81f55c0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 --2- 172.21.15.44:0/984896556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76b0004d40 0x7f76c809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 >> 172.21.15.44:0/984896556 conn(0x7f76c80b0fb0 msgr2=0x7f76c80b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 shutdown_connections 2024-07-17T08:41:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.097+0000 7f76cfe00640 1 -- 172.21.15.44:0/984896556 wait complete. 2024-07-17T08:41:15.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.311+0000 7f36a2a00640 1 -- 172.21.15.44:0/3139454010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 msgr2=0x7f3680004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.311+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3139454010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f3680004c80 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f36840016c0 tx=0x7f3680005830 comp rx=0 tx=0).stop 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.312+0000 7f36a2a00640 1 -- 172.21.15.44:0/3139454010 shutdown_connections 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.312+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3139454010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f3680004c80 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.312+0000 7f36a2a00640 1 -- 172.21.15.44:0/3139454010 >> 172.21.15.44:0/3139454010 conn(0x7f369c0d00a0 msgr2=0x7f369c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.312+0000 7f36a2a00640 1 -- 172.21.15.44:0/3139454010 shutdown_connections 2024-07-17T08:41:15.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.312+0000 7f36a2a00640 1 -- 172.21.15.44:0/3139454010 wait complete. 2024-07-17T08:41:15.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.313+0000 7f36a2a00640 1 Processor -- start 2024-07-17T08:41:15.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.314+0000 7f36a2a00640 1 -- start start 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369c147cc0 0x7f369c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c1f3010 0x7f369c1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f369c06e480 con 0x7f3680004800 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0b5790 con 0x7f369c1f3010 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f369c06e600 con 0x7f369c147cc0 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34602/0 (socket says 172.21.15.44:34602) 2024-07-17T08:41:15.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 -- 172.21.15.44:0/3597626112 learned_addr learned my addr 172.21.15.44:0/3597626112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f3693e00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369c147cc0 0x7f369c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f36a0a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c1f3010 0x7f369c1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369c147cc0 msgr2=0x7f369c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369c147cc0 0x7f369c15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c1f3010 msgr2=0x7f369c1f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c1f3010 0x7f369c1f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.315+0000 7f369be00640 1 -- 172.21.15.44:0/3597626112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1602b0 con 0x7f3680004800 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f369be00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e690 tx=0x7f368c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00ca50 con 0x7f3680004800 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f368c00cce0 con 0x7f3680004800 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0116d0 con 0x7f3680004800 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c160590 con 0x7f3680004800 2024-07-17T08:41:15.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.316+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c160b50 con 0x7f3680004800 2024-07-17T08:41:15.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.318+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c160d50 con 0x7f3680004800 2024-07-17T08:41:15.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.319+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7f368c011870 con 0x7f3680004800 2024-07-17T08:41:15.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.320+0000 7f3698c00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 0x7f3678079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.320+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7f368c0997d0 con 0x7f3680004800 2024-07-17T08:41:15.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.320+0000 7f3693e00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 msgr2=0x7f3678079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:15.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.320+0000 7f3693e00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 0x7f3678079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.324+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f368c016030 con 0x7f3680004800 2024-07-17T08:41:15.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.522+0000 7f3693e00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 msgr2=0x7f3678079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:15.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.522+0000 7f3693e00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 0x7f3678079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:15.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.536+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f369c15ab70 con 0x7f3680004800 2024-07-17T08:41:15.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.539+0000 7f3698c00640 1 -- 172.21.15.44:0/3597626112 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v296) ==== 74+0+18322 (secure 0 0 0) 0x7f368c062350 con 0x7f3680004800 2024-07-17T08:41:15.540 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:15.540 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":296,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:11.646055+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":39,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:39:46.575840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"292","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":40,"pool_name":"cephfs_data","create_time":"2024-07-17T08:39:47.617938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"293","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"40.13","mappings":[{"from":5,"to":1}]},{"pgid":"40.18","mappings":[{"from":5,"to":1}]},{"pgid":"40.31","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:6834/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/2395145048":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/3806999519":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/1608213311":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/676716344":"2024-07-18T08:40:10.498668+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:15.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 msgr2=0x7f3678079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 0x7f3678079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 msgr2=0x7f369c143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e690 tx=0x7f368c00eb50 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 shutdown_connections 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7f3678076eb0 0x7f3678079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f369c1f3010 0x7f369c1f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369c147cc0 0x7f369c15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 --2- 172.21.15.44:0/3597626112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680004800 0x7f369c143e00 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.541+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 >> 172.21.15.44:0/3597626112 conn(0x7f369c0d00a0 msgr2=0x7f369c0b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.542+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 shutdown_connections 2024-07-17T08:41:15.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.542+0000 7f36a2a00640 1 -- 172.21.15.44:0/3597626112 wait complete. 2024-07-17T08:41:15.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.746+0000 7fc67ae00640 1 -- 172.21.15.44:0/334110086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6740ce690 msgr2=0x7fc6740cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.746+0000 7fc67ae00640 1 --2- 172.21.15.44:0/334110086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6740ce690 0x7fc6740cfdf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005fd0 tx=0x7fc66c02a820 comp rx=0 tx=0).stop 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 -- 172.21.15.44:0/334110086 shutdown_connections 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 --2- 172.21.15.44:0/334110086 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6740ce690 0x7fc6740cfdf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 --2- 172.21.15.44:0/334110086 >> v2:172.21.15.175:3300/0 conn(0x7fc674161be0 0x7fc67415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 -- 172.21.15.44:0/334110086 >> 172.21.15.44:0/334110086 conn(0x7fc6740b4ff0 msgr2=0x7fc6740b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 -- 172.21.15.44:0/334110086 shutdown_connections 2024-07-17T08:41:15.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.747+0000 7fc67ae00640 1 -- 172.21.15.44:0/334110086 wait complete. 2024-07-17T08:41:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.748+0000 7fc67ae00640 1 Processor -- start 2024-07-17T08:41:15.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.748+0000 7fc67ae00640 1 -- start start 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6740ce690 0x7fc6741f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc67415d130 0x7fc67415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740d1a60 con 0x7fc674161be0 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740d18e0 con 0x7fc6740ce690 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.749+0000 7fc67ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6740d1be0 con 0x7fc67415d130 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34604/0 (socket says 172.21.15.44:34604) 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 -- 172.21.15.44:0/3536923456 learned_addr learned my addr 172.21.15.44:0/3536923456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc679a00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6740ce690 0x7fc6741f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc679000640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc67415d130 0x7fc67415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 -- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc67415d130 msgr2=0x7fc67415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc67415d130 0x7fc67415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 -- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6740ce690 msgr2=0x7fc6741f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6740ce690 0x7fc6741f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.750+0000 7fc67a400640 1 -- 172.21.15.44:0/3536923456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6741f52b0 con 0x7fc674161be0 2024-07-17T08:41:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc67a400640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006010 tx=0x7fc66c032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c037040 con 0x7fc674161be0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc66c042070 con 0x7fc674161be0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67415b250 con 0x7fc674161be0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c004040 con 0x7fc674161be0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.751+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6740ca930 con 0x7fc674161be0 2024-07-17T08:41:15.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.753+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6740d2170 con 0x7fc674161be0 2024-07-17T08:41:15.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.754+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 176) ==== 99090+0+0 (secure 0 0 0) 0x7fc66c0041e0 con 0x7fc674161be0 2024-07-17T08:41:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.756+0000 7fc671600640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 0x7fc658079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:15.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.756+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(296..296 src has 1..296) ==== 7050+0+0 (secure 0 0 0) 0x7fc66c0ba420 con 0x7fc674161be0 2024-07-17T08:41:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.756+0000 7fc679a00640 1 -- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 msgr2=0x7fc658079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1750219097 2024-07-17T08:41:15.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.756+0000 7fc679a00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 0x7fc658079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:15.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.759+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc66c082f90 con 0x7fc674161be0 2024-07-17T08:41:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.915+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 177) ==== 98540+0+0 (secure 0 0 0) 0x7fc66c07eb20 con 0x7fc674161be0 2024-07-17T08:41:15.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.916+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 msgr2=0x7fc658079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:15.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:15.916+0000 7fc671600640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 0x7fc658079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bbcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bbcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bbcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bbcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bbcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bb9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.033 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bb9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.034 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bb9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.034 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bb9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.034 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:16.032+0000 7f9bb9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:16.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.060+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc6740b91b0 con 0x7fc674161be0 2024-07-17T08:41:16.062 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:41:16.061+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:41:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.063+0000 7fc671600640 1 -- 172.21.15.44:0/3536923456 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v165) ==== 161+0+0 (secure 0 0 0) 0x7fc66c09d990 con 0x7fc674161be0 2024-07-17T08:41:16.064 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 msgr2=0x7fc67415f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006010 tx=0x7fc66c032be0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 shutdown_connections 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:6834/1750219097,v1:172.21.15.175:6837/1750219097] conn(0x7fc658076e80 0x7fc658079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc674161be0 0x7fc67415f470 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc67415d130 0x7fc67415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 --2- 172.21.15.44:0/3536923456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6740ce690 0x7fc6741f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 >> 172.21.15.44:0/3536923456 conn(0x7fc6740b4ff0 msgr2=0x7fc6740b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 shutdown_connections 2024-07-17T08:41:16.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.067+0000 7fc67ae00640 1 -- 172.21.15.44:0/3536923456 wait complete. 2024-07-17T08:41:16.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:41:16.136 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.265+0000 7f7118400640 1 -- 172.21.15.44:0/1251514684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 msgr2=0x7f7110166e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.265+0000 7f7118400640 1 --2- 172.21.15.44:0/1251514684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f7110166e40 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f7108005fd0 tx=0x7f710802aa90 comp rx=0 tx=0).stop 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 -- 172.21.15.44:0/1251514684 shutdown_connections 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 --2- 172.21.15.44:0/1251514684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f7110166e40 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 --2- 172.21.15.44:0/1251514684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100b6790 0x7f71101648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 -- 172.21.15.44:0/1251514684 >> 172.21.15.44:0/1251514684 conn(0x7f71100d3700 msgr2=0x7f71100d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 -- 172.21.15.44:0/1251514684 shutdown_connections 2024-07-17T08:41:16.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.266+0000 7f7118400640 1 -- 172.21.15.44:0/1251514684 wait complete. 2024-07-17T08:41:16.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.267+0000 7f7118400640 1 Processor -- start 2024-07-17T08:41:16.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.267+0000 7f7118400640 1 -- start start 2024-07-17T08:41:16.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100b6790 0x7f71101549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100ce1d0 0x7f71101563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7110167710 con 0x7f71100b29e0 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7110167590 con 0x7f71100b6790 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.268+0000 7f7118400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7110167890 con 0x7f71100ce1d0 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34632/0 (socket says 172.21.15.44:34632) 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 -- 172.21.15.44:0/601359583 learned_addr learned my addr 172.21.15.44:0/601359583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115000640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100b6790 0x7f71101549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 -- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100ce1d0 msgr2=0x7f71101563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7116400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100ce1d0 0x7f71101563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100ce1d0 0x7f71101563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 -- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100b6790 msgr2=0x7f71101549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100b6790 0x7f71101549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 -- 172.21.15.44:0/601359583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7110153f00 con 0x7f71100b29e0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.269+0000 7f7115a00640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f710400b3f0 tx=0x7f710400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.270+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104015020 con 0x7f71100b29e0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.270+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7104011920 con 0x7f71100b29e0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.270+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104011d70 con 0x7f71100b29e0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.270+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71100cb540 con 0x7f71100b29e0 2024-07-17T08:41:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.270+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7110154480 con 0x7f71100b29e0 2024-07-17T08:41:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.271+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71100ce5b0 con 0x7f71100b29e0 2024-07-17T08:41:16.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.272+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 177) ==== 98540+0+0 (secure 0 0 0) 0x7f7104018020 con 0x7f71100b29e0 2024-07-17T08:41:16.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.273+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(298..298 src has 1..298) ==== 7609+0+0 (secure 0 0 0) 0x7f7104098280 con 0x7f71100b29e0 2024-07-17T08:41:16.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.277+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f710401f030 con 0x7f71100b29e0 2024-07-17T08:41:16.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.521+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f71100d26e0 con 0x7f71100b29e0 2024-07-17T08:41:16.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.921+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 178) ==== 98667+0+0 (secure 0 0 0) 0x7f710405c5b0 con 0x7f71100b29e0 2024-07-17T08:41:16.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.922+0000 7f710d600640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 0x7f70f007f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.923+0000 7f7115000640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 0x7f70f007f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:16.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:16.933+0000 7f7115000640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 0x7f70f007f780 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71100c6ea0 tx=0x7f70fc005cc0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.067+0000 7f710d600640 1 -- 172.21.15.44:0/601359583 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v166) ==== 104+0+0 (secure 0 0 0) 0x7f7104061340 con 0x7f71100b29e0 2024-07-17T08:41:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 msgr2=0x7f70f007f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 0x7f70f007f780 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71100c6ea0 tx=0x7f70fc005cc0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 msgr2=0x7f71100cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:17.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f710400b3f0 tx=0x7f710400b8b0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 shutdown_connections 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f70f007d390 0x7f70f007f780 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71100ce1d0 0x7f71101563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71100b6790 0x7f71101549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 --2- 172.21.15.44:0/601359583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71100b29e0 0x7f71100cd1d0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.072+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 >> 172.21.15.44:0/601359583 conn(0x7f71100d3700 msgr2=0x7f71100d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.073+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 shutdown_connections 2024-07-17T08:41:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.073+0000 7f7118400640 1 -- 172.21.15.44:0/601359583 wait complete. 2024-07-17T08:41:17.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:41:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 -- 172.21.15.44:0/286956790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c161b50 msgr2=0x7fda8c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 --2- 172.21.15.44:0/286956790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c161b50 0x7fda8c0d0430 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fda78009830 tx=0x7fda7802a3f0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 -- 172.21.15.44:0/286956790 shutdown_connections 2024-07-17T08:41:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 --2- 172.21.15.44:0/286956790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c161b50 0x7fda8c0d0430 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 -- 172.21.15.44:0/286956790 >> 172.21.15.44:0/286956790 conn(0x7fda8c073a60 msgr2=0x7fda8c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:17.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 -- 172.21.15.44:0/286956790 shutdown_connections 2024-07-17T08:41:17.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.309+0000 7fda8be00640 1 -- 172.21.15.44:0/286956790 wait complete. 2024-07-17T08:41:17.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.311+0000 7fda8be00640 1 Processor -- start 2024-07-17T08:41:17.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.311+0000 7fda8be00640 1 -- start start 2024-07-17T08:41:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c15e370 0x7fda8c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 0x7fda8c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:17.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0b8f00 con 0x7fda8c0d0970 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0b8d80 con 0x7fda8c161b50 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c0b9080 con 0x7fda8c15e370 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34644/0 (socket says 172.21.15.44:34644) 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 -- 172.21.15.44:0/446567656 learned_addr learned my addr 172.21.15.44:0/446567656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8a000640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c15e370 0x7fda8c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8b400640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 0x7fda8c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 -- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c15e370 msgr2=0x7fda8c0cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c15e370 0x7fda8c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 -- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 msgr2=0x7fda8c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 0x7fda8c15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.312+0000 7fda8aa00640 1 -- 172.21.15.44:0/446567656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c1f7500 con 0x7fda8c0d0970 2024-07-17T08:41:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda8b400640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 0x7fda8c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:17.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda8aa00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c760 tx=0x7fda7c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c017070 con 0x7fda8c0d0970 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda7c005180 con 0x7fda8c0d0970 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00a640 con 0x7fda8c0d0970 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1f4590 con 0x7fda8c0d0970 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.313+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1f4b50 con 0x7fda8c0d0970 2024-07-17T08:41:17.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.314+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c0d0430 con 0x7fda8c0d0970 2024-07-17T08:41:17.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.316+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 178) ==== 98667+0+0 (secure 0 0 0) 0x7fda7c01a070 con 0x7fda8c0d0970 2024-07-17T08:41:17.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.317+0000 7fda83400640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 0x7fda6c078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:17.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.317+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(298..298 src has 1..298) ==== 7609+0+0 (secure 0 0 0) 0x7fda7c097bb0 con 0x7fda8c0d0970 2024-07-17T08:41:17.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.318+0000 7fda8a000640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 0x7fda6c078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:17.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.319+0000 7fda8a000640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 0x7fda6c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fda78034000 tx=0x7fda78004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:17.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.322+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fda8c0d0d80 con 0x7fda8c0d0970 2024-07-17T08:41:17.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:17.522+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda8c069460 con 0x7fda8c0d0970 2024-07-17T08:41:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.070+0000 7fda83400640 1 -- 172.21.15.44:0/446567656 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v299) ==== 181+0+0 (secure 0 0 0) 0x7fda7c060b80 con 0x7fda8c0d0970 2024-07-17T08:41:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:41:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 msgr2=0x7fda6c078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 0x7fda6c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fda78034000 tx=0x7fda78004730 comp rx=0 tx=0).stop 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 msgr2=0x7fda8c1f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c760 tx=0x7fda7c00cc20 comp rx=0 tx=0).stop 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 shutdown_connections 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fda6c076a00 0x7fda6c078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c161b50 0x7fda8c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c15e370 0x7fda8c0cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 --2- 172.21.15.44:0/446567656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0d0970 0x7fda8c1f6fc0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.079+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 >> 172.21.15.44:0/446567656 conn(0x7fda8c073a60 msgr2=0x7fda8c073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.081+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 shutdown_connections 2024-07-17T08:41:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.081+0000 7fda8be00640 1 -- 172.21.15.44:0/446567656 wait complete. 2024-07-17T08:41:18.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:41:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 -- 172.21.15.44:0/1333149574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 msgr2=0x7f2b6c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1333149574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c06d620 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009830 tx=0x7f2b5c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 -- 172.21.15.44:0/1333149574 shutdown_connections 2024-07-17T08:41:18.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1333149574 >> v2:172.21.15.175:3300/0 conn(0x7f2b6c0cb880 0x7f2b6c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1333149574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c06d620 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 -- 172.21.15.44:0/1333149574 >> 172.21.15.44:0/1333149574 conn(0x7f2b6c0b0eb0 msgr2=0x7f2b6c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 -- 172.21.15.44:0/1333149574 shutdown_connections 2024-07-17T08:41:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.302+0000 7f2b71a00640 1 -- 172.21.15.44:0/1333149574 wait complete. 2024-07-17T08:41:18.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.304+0000 7f2b71a00640 1 Processor -- start 2024-07-17T08:41:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.304+0000 7f2b71a00640 1 -- start start 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b6c0cb880 0x7f2b6c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6c15c220 0x7f2b6c144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15b5c0 con 0x7f2b6c15b060 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15b440 con 0x7f2b6c0cb880 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b71a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c15f4a0 con 0x7f2b6c15c220 2024-07-17T08:41:18.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34662/0 (socket says 172.21.15.44:34662) 2024-07-17T08:41:18.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.305+0000 7f2b6a000640 1 -- 172.21.15.44:0/1804719651 learned_addr learned my addr 172.21.15.44:0/1804719651 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 -- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6c15c220 msgr2=0x7f2b6c144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6b400640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6c15c220 0x7f2b6c144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b6c0cb880 0x7f2b6c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6c15c220 0x7f2b6c144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 -- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b6c0cb880 msgr2=0x7f2b6c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b6c0cb880 0x7f2b6c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 -- 172.21.15.44:0/1804719651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c0b0c90 con 0x7f2b6c15b060 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.306+0000 7f2b6a000640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c8c0 tx=0x7f2b5400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.307+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54004220 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.307+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b540079b0 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.307+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54007de0 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.307+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1f4050 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.307+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c0b06d0 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.308+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c15f9c0 con 0x7f2b6c15b060 2024-07-17T08:41:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.309+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 179) ==== 98875+0+0 (secure 0 0 0) 0x7f2b54005020 con 0x7f2b6c15b060 2024-07-17T08:41:18.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.310+0000 7f2b62a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 0x7f2b48079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:18.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.310+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(299..299 src has 1..299) ==== 7171+0+0 (secure 0 0 0) 0x7f2b54098f40 con 0x7f2b6c15b060 2024-07-17T08:41:18.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.311+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 0x7f2b48079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:18.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.312+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 0x7f2b48079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c034000 tx=0x7f2b5c004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:18.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.315+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2b54014030 con 0x7f2b6c15b060 2024-07-17T08:41:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:18.529+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2b6c157290 con 0x7f2b6c15b060 2024-07-17T08:41:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.090+0000 7f2b62a00640 1 -- 172.21.15.44:0/1804719651 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v300) ==== 169+0+0 (secure 0 0 0) 0x7f2b54061c50 con 0x7f2b6c15b060 2024-07-17T08:41:19.091 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:41:19.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.095+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 msgr2=0x7f2b48079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.095+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 0x7f2b48079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c034000 tx=0x7f2b5c004730 comp rx=0 tx=0).stop 2024-07-17T08:41:19.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.095+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 msgr2=0x7f2b6c158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.095+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c8c0 tx=0x7f2b5400cd80 comp rx=0 tx=0).stop 2024-07-17T08:41:19.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 shutdown_connections 2024-07-17T08:41:19.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b48076c40 0x7f2b48079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b6c15c220 0x7f2b6c144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b6c15b060 0x7f2b6c158570 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 --2- 172.21.15.44:0/1804719651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b6c0cb880 0x7f2b6c159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.096+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 >> 172.21.15.44:0/1804719651 conn(0x7f2b6c0b0eb0 msgr2=0x7f2b6c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.097+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 shutdown_connections 2024-07-17T08:41:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.097+0000 7f2b71a00640 1 -- 172.21.15.44:0/1804719651 wait complete. 2024-07-17T08:41:19.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 -- 172.21.15.44:0/2678028969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc10004090 msgr2=0x7fcc10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 --2- 172.21.15.44:0/2678028969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc10004090 0x7fcc10004800 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009b50 tx=0x7fcc10005830 comp rx=0 tx=0).stop 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc2be00640 1 -- 172.21.15.44:0/2678028969 reap_dead start 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 -- 172.21.15.44:0/2678028969 shutdown_connections 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 -- 172.21.15.44:0/2678028969 >> 172.21.15.44:0/2678028969 conn(0x7fcc2c0d1350 msgr2=0x7fcc2c0d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 -- 172.21.15.44:0/2678028969 shutdown_connections 2024-07-17T08:41:19.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.329+0000 7fcc31200640 1 -- 172.21.15.44:0/2678028969 wait complete. 2024-07-17T08:41:19.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.331+0000 7fcc31200640 1 Processor -- start 2024-07-17T08:41:19.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.331+0000 7fcc31200640 1 -- start start 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc10004090 0x7fcc2c1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc2c0cca70 0x7fcc2c1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c0739a0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c073820 con 0x7fcc2c0cca70 2024-07-17T08:41:19.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc31200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2c0b6760 con 0x7fcc10004090 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52414/0 (socket says 172.21.15.44:52414) 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.332+0000 7fcc23e00640 1 -- 172.21.15.44:0/1507665996 learned_addr learned my addr 172.21.15.44:0/1507665996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc2be00640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc2c0cca70 0x7fcc2c1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 -- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc10004090 msgr2=0x7fcc2c1f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc10004090 0x7fcc2c1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 -- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc2c0cca70 msgr2=0x7fcc2c1f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc2c0cca70 0x7fcc2c1f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 -- 172.21.15.44:0/1507665996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c0cff00 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23e00640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fcc14036040 tx=0x7fcc140336d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14038040 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc14041070 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1403c430 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.333+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c0d01e0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.334+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c0cfc10 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.335+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2c0b4400 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.336+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 179) ==== 98875+0+0 (secure 0 0 0) 0x7fcc1402b8c0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.337+0000 7fcc23400640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 0x7fcc08079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.338+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7fcc140b8fe0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.338+0000 7fcc2b400640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 0x7fcc08079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.339+0000 7fcc2b400640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 0x7fcc08079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004820 tx=0x7fcc1c000f50 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:19.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.342+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcc140820b0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.550+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fcc2c15a6e0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.556+0000 7fcc23400640 1 -- 172.21.15.44:0/1507665996 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v91) ==== 71+0+0 (secure 0 0 0) 0x7fcc140819b0 con 0x7fcc2c1f7e70 2024-07-17T08:41:19.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 msgr2=0x7fcc08079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 0x7fcc08079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004820 tx=0x7fcc1c000f50 comp rx=0 tx=0).stop 2024-07-17T08:41:19.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 msgr2=0x7fcc2c0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fcc14036040 tx=0x7fcc140336d0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 shutdown_connections 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcc08076c70 0x7fcc08079130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc2c0cca70 0x7fcc2c1f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc2c1f7e70 0x7fcc2c0cf6d0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 --2- 172.21.15.44:0/1507665996 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc10004090 0x7fcc2c1f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.559+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 >> 172.21.15.44:0/1507665996 conn(0x7fcc2c0d1350 msgr2=0x7fcc2c073b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.560+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 shutdown_connections 2024-07-17T08:41:19.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.560+0000 7fcc31200640 1 -- 172.21.15.44:0/1507665996 wait complete. 2024-07-17T08:41:19.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02' 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 -- 172.21.15.44:0/1209575281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 msgr2=0x7fb0d80ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 --2- 172.21.15.44:0/1209575281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d80ac810 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009900 tx=0x7fb0d002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 -- 172.21.15.44:0/1209575281 shutdown_connections 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 --2- 172.21.15.44:0/1209575281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d80ac810 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 --2- 172.21.15.44:0/1209575281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 0x7fb0d8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.785+0000 7fb0de000640 1 -- 172.21.15.44:0/1209575281 >> 172.21.15.44:0/1209575281 conn(0x7fb0d80b0fb0 msgr2=0x7fb0d80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.786+0000 7fb0de000640 1 -- 172.21.15.44:0/1209575281 shutdown_connections 2024-07-17T08:41:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.786+0000 7fb0de000640 1 -- 172.21.15.44:0/1209575281 wait complete. 2024-07-17T08:41:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.787+0000 7fb0de000640 1 Processor -- start 2024-07-17T08:41:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.788+0000 7fb0de000640 1 -- start start 2024-07-17T08:41:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0d806f710 0x7fb0d81637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 0x7fb0d815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81643f0 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8164270 con 0x7fb0d806f710 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0de000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d8164570 con 0x7fb0d8159b00 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52434/0 (socket says 172.21.15.44:52434) 2024-07-17T08:41:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 -- 172.21.15.44:0/2354251251 learned_addr learned my addr 172.21.15.44:0/2354251251 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:19.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d7400640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0d806f710 0x7fb0d81637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d7e00640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 0x7fb0d815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 -- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 msgr2=0x7fb0d815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 0x7fb0d815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 -- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0d806f710 msgr2=0x7fb0d81637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0d806f710 0x7fb0d81637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.789+0000 7fb0d6a00640 1 -- 172.21.15.44:0/2354251251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d815dfa0 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0d6a00640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c970 tx=0x7fb0c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0007bb0 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c0007e40 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d815d9e0 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0005280 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.790+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d815af40 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.791+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d80ac810 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.792+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 179) ==== 98875+0+0 (secure 0 0 0) 0x7fb0c0005420 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.793+0000 7fb0cf400640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 0x7fb0b4079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.793+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7fb0c0098c80 con 0x7fb0d80ac3e0 2024-07-17T08:41:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.794+0000 7fb0d7400640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 0x7fb0b4079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.795+0000 7fb0d7400640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 0x7fb0b4079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80012e0 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:19.798+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb0c0014030 con 0x7fb0d80ac3e0 2024-07-17T08:41:20.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:20.094+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02"]} v 0) -- 0x7fb0d81508d0 con 0x7fb0d80ac3e0 2024-07-17T08:41:20.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:20.576+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7fb0c005d7e0 con 0x7fb0d80ac3e0 2024-07-17T08:41:21.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.096+0000 7fb0cf400640 1 -- 172.21.15.44:0/2354251251 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_02"]}]=0 v3546) ==== 201+0+0 (secure 0 0 0) 0x7fb0c008b2a0 con 0x7fb0d80ac3e0 2024-07-17T08:41:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 msgr2=0x7fb0b4079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 0x7fb0b4079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80012e0 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2024-07-17T08:41:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 msgr2=0x7fb0d815b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c970 tx=0x7fb0c000ce30 comp rx=0 tx=0).stop 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 shutdown_connections 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fb0b4076c40 0x7fb0b4079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0d8159b00 0x7fb0d815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d80ac3e0 0x7fb0d815b890 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 --2- 172.21.15.44:0/2354251251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0d806f710 0x7fb0d81637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.100+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 >> 172.21.15.44:0/2354251251 conn(0x7fb0d80b0fb0 msgr2=0x7fb0d80b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.101+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 shutdown_connections 2024-07-17T08:41:21.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.101+0000 7fb0de000640 1 -- 172.21.15.44:0/2354251251 wait complete. 2024-07-17T08:41:21.122 INFO:tasks.cephfs_test_runner:Test subvol schedule creation fails for non-default subvolgroup. ... ok 2024-07-17T08:41:21.122 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_03 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule creation fails when subvol exists only under default group. 2024-07-17T08:41:21.122 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_03 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:41:21.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 -- 172.21.15.44:0/2134969795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc0ae810 msgr2=0x7ff8dc15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2134969795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc0ae810 0x7ff8dc15e510 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009860 tx=0x7ff8d002a560 comp rx=0 tx=0).stop 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 -- 172.21.15.44:0/2134969795 shutdown_connections 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2134969795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc0ae810 0x7ff8dc15e510 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 -- 172.21.15.44:0/2134969795 >> 172.21.15.44:0/2134969795 conn(0x7ff8dc0b0fb0 msgr2=0x7ff8dc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 -- 172.21.15.44:0/2134969795 shutdown_connections 2024-07-17T08:41:21.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.328+0000 7ff8e4200640 1 -- 172.21.15.44:0/2134969795 wait complete. 2024-07-17T08:41:21.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.330+0000 7ff8e4200640 1 Processor -- start 2024-07-17T08:41:21.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.330+0000 7ff8e4200640 1 -- start start 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8dc0ac930 0x7ff8dc1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8dc0ae810 0x7ff8dc1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc06f040 con 0x7ff8dc06ea50 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc06eec0 con 0x7ff8dc0ac930 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e4200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc0b2ce0 con 0x7ff8dc0ae810 2024-07-17T08:41:21.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52460/0 (socket says 172.21.15.44:52460) 2024-07-17T08:41:21.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.331+0000 7ff8e1800640 1 -- 172.21.15.44:0/2778859113 learned_addr learned my addr 172.21.15.44:0/2778859113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 -- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8dc0ae810 msgr2=0x7ff8dc1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8dc0ae810 0x7ff8dc1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 -- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8dc0ac930 msgr2=0x7ff8dc1f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8dc0ac930 0x7ff8dc1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 -- 172.21.15.44:0/2778859113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc1f8190 con 0x7ff8dc06ea50 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.332+0000 7ff8e1800640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009780 tx=0x7ff8d0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.333+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0031b50 con 0x7ff8dc06ea50 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.333+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8d0031e00 con 0x7ff8dc06ea50 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.333+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc1f8470 con 0x7ff8dc06ea50 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.333+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0005280 con 0x7ff8dc06ea50 2024-07-17T08:41:21.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.333+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc1f46e0 con 0x7ff8dc06ea50 2024-07-17T08:41:21.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.334+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc06f390 con 0x7ff8dc06ea50 2024-07-17T08:41:21.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.335+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7ff8d0005420 con 0x7ff8dc06ea50 2024-07-17T08:41:21.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.337+0000 7ff8d9600640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 0x7ff8c0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.337+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7ff8d00b9970 con 0x7ff8dc06ea50 2024-07-17T08:41:21.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.338+0000 7ff8e0e00640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 0x7ff8c0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.339+0000 7ff8e0e00640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 0x7ff8c0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0046b0 tx=0x7ff8cc0012b0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:21.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.343+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff8dc06f390 con 0x7ff8dc06ea50 2024-07-17T08:41:21.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.548+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8dc06a1a0 con 0x7ff8dc06ea50 2024-07-17T08:41:21.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.549+0000 7ff8d9600640 1 -- 172.21.15.44:0/2778859113 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v300) ==== 74+0+15293 (secure 0 0 0) 0x7ff8dc06a1a0 con 0x7ff8dc06ea50 2024-07-17T08:41:21.550 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:21.551 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":300,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:19.086452+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":40,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":295,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":291,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/789460381":"2024-07-18T08:41:16.061764+0000","172.21.15.44:6833/789460381":"2024-07-18T08:41:16.061764+0000","172.21.15.175:0/1542167015":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/3276089562":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/3923591867":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/3893359478":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/3631900278":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/1988906330":"2024-07-18T08:41:15.896480+0000","172.21.15.175:6837/1750219097":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/2860338237":"2024-07-18T08:41:15.896480+0000","172.21.15.175:6837/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/1002609044":"2024-07-18T08:41:15.896480+0000","172.21.15.175:6834/1750219097":"2024-07-18T08:41:15.896480+0000","172.21.15.175:0/2425673728":"2024-07-18T08:41:15.896480+0000","172.21.15.175:6834/1518538177":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/2395145048":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/3806999519":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/1608213311":"2024-07-18T08:40:10.498668+0000","172.21.15.175:0/676716344":"2024-07-18T08:40:10.498668+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:21.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 msgr2=0x7ff8c0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 0x7ff8c0079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0046b0 tx=0x7ff8cc0012b0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 msgr2=0x7ff8dc074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009780 tx=0x7ff8d0033ba0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 shutdown_connections 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff8c0076cd0 0x7ff8c0079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8dc0ae810 0x7ff8dc1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8dc0ac930 0x7ff8dc1f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 --2- 172.21.15.44:0/2778859113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8dc06ea50 0x7ff8dc074200 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.552+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 >> 172.21.15.44:0/2778859113 conn(0x7ff8dc0b0fb0 msgr2=0x7ff8dc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.553+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 shutdown_connections 2024-07-17T08:41:21.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.553+0000 7ff8e4200640 1 -- 172.21.15.44:0/2778859113 wait complete. 2024-07-17T08:41:21.570 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:41:21.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:41:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 -- 172.21.15.44:0/98251656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000051c0 msgr2=0x7fe10000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 --2- 172.21.15.44:0/98251656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000051c0 0x7fe10000b9c0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c610 tx=0x7fe100005830 comp rx=0 tx=0).stop 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 -- 172.21.15.44:0/98251656 shutdown_connections 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 --2- 172.21.15.44:0/98251656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000051c0 0x7fe10000b9c0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 -- 172.21.15.44:0/98251656 >> 172.21.15.44:0/98251656 conn(0x7fe1200b0eb0 msgr2=0x7fe1200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 -- 172.21.15.44:0/98251656 shutdown_connections 2024-07-17T08:41:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.763+0000 7fe124c00640 1 -- 172.21.15.44:0/98251656 wait complete. 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.765+0000 7fe124c00640 1 Processor -- start 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.765+0000 7fe124c00640 1 -- start start 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1000051c0 0x7fe120073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1200715e0 0x7fe120158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200b3660 con 0x7fe120158960 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200b34e0 con 0x7fe1000051c0 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.766+0000 7fe124c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1200ae560 con 0x7fe1200715e0 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52470/0 (socket says 172.21.15.44:52470) 2024-07-17T08:41:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 -- 172.21.15.44:0/928722786 learned_addr learned my addr 172.21.15.44:0/928722786 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe117e00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1200715e0 0x7fe120158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11de00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1000051c0 0x7fe120073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 -- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1200715e0 msgr2=0x7fe120158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1200715e0 0x7fe120158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 -- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1000051c0 msgr2=0x7fe120073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1000051c0 0x7fe120073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 -- 172.21.15.44:0/928722786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201f75f0 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.767+0000 7fe11e800640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fe118001330 tx=0x7fe11800c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.768+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118039070 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.768+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe11802bd20 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.768+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201f78d0 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.768+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118006290 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.768+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201f7e90 con 0x7fe120158960 2024-07-17T08:41:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.769+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1200ac3a0 con 0x7fe120158960 2024-07-17T08:41:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.771+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7fe118006430 con 0x7fe120158960 2024-07-17T08:41:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.772+0000 7fe116000640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 0x7fe0fc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.773+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7fe1180b8470 con 0x7fe120158960 2024-07-17T08:41:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.773+0000 7fe11de00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 0x7fe0fc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:21.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.774+0000 7fe11de00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 0x7fe0fc0791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1100016d0 tx=0x7fe110009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.775+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe118081450 con 0x7fe120158960 2024-07-17T08:41:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.987+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1200acd10 con 0x7fe120158960 2024-07-17T08:41:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.987+0000 7fe116000640 1 -- 172.21.15.44:0/928722786 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v300) ==== 93+0+10 (secure 0 0 0) 0x7fe118080d50 con 0x7fe120158960 2024-07-17T08:41:21.988 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:41:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 msgr2=0x7fe0fc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 0x7fe0fc0791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1100016d0 tx=0x7fe110009040 comp rx=0 tx=0).stop 2024-07-17T08:41:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 msgr2=0x7fe12014f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fe118001330 tx=0x7fe11800c260 comp rx=0 tx=0).stop 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 shutdown_connections 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fe0fc076d00 0x7fe0fc0791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe120158960 0x7fe12014f8f0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1200715e0 0x7fe120158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 --2- 172.21.15.44:0/928722786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1000051c0 0x7fe120073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.990+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 >> 172.21.15.44:0/928722786 conn(0x7fe1200b0eb0 msgr2=0x7fe1200aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.991+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 shutdown_connections 2024-07-17T08:41:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:21.991+0000 7fe124c00640 1 -- 172.21.15.44:0/928722786 wait complete. 2024-07-17T08:41:22.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03' 2024-07-17T08:41:22.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 -- 172.21.15.44:0/3724410501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c06f4b0 msgr2=0x7f805c159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 --2- 172.21.15.44:0/3724410501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c06f4b0 0x7f805c159800 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f8054005fd0 tx=0x7f805402aa00 comp rx=0 tx=0).stop 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 -- 172.21.15.44:0/3724410501 shutdown_connections 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 --2- 172.21.15.44:0/3724410501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c06f4b0 0x7f805c159800 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 --2- 172.21.15.44:0/3724410501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 0x7f805c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.211+0000 7f805b200640 1 -- 172.21.15.44:0/3724410501 >> 172.21.15.44:0/3724410501 conn(0x7f805c0b0e80 msgr2=0x7f805c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.212+0000 7f805b200640 1 -- 172.21.15.44:0/3724410501 shutdown_connections 2024-07-17T08:41:22.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.212+0000 7f805b200640 1 -- 172.21.15.44:0/3724410501 wait complete. 2024-07-17T08:41:22.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.213+0000 7f805b200640 1 Processor -- start 2024-07-17T08:41:22.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.213+0000 7f805b200640 1 -- start start 2024-07-17T08:41:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 0x7f805c1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 0x7f805c147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:22.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f805c0b2120 con 0x7f805c09b300 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f805c0b1fa0 con 0x7f805c06f4b0 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f805b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f805c0b22a0 con 0x7f805c0cfa90 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.214+0000 7f8059400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52496/0 (socket says 172.21.15.44:52496) 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 -- 172.21.15.44:0/3231930615 learned_addr learned my addr 172.21.15.44:0/3231930615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059e00640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 0x7f805c1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:22.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f805a800640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 0x7f805c147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 -- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 msgr2=0x7f805c147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 0x7f805c147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 -- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 msgr2=0x7f805c1f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 0x7f805c1f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059400640 1 -- 172.21.15.44:0/3231930615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c1f43f0 con 0x7f805c09b300 2024-07-17T08:41:22.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.215+0000 7f8059e00640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 0x7f805c1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f8059400640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f804800b410 tx=0x7f804800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048015020 con 0x7f805c09b300 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8048011940 con 0x7f805c09b300 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048011d90 con 0x7f805c09b300 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c1f7f10 con 0x7f805c09b300 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.216+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c074120 con 0x7f805c09b300 2024-07-17T08:41:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.217+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c159800 con 0x7f805c09b300 2024-07-17T08:41:22.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.218+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f8048018020 con 0x7f805c09b300 2024-07-17T08:41:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.219+0000 7f8052000640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 0x7f803c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.219+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7f8048098900 con 0x7f805c09b300 2024-07-17T08:41:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.220+0000 7f8059e00640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 0x7f803c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.221+0000 7f8059e00640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 0x7f803c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f805c0734d0 tx=0x7f804c009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:22.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.225+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f804801f020 con 0x7f805c09b300 2024-07-17T08:41:22.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:22.511+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03"]} v 0) -- 0x7f805c0cee70 con 0x7f805c09b300 2024-07-17T08:41:23.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.111+0000 7f8052000640 1 -- 172.21.15.44:0/3231930615 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03"]}]=0 v3548) ==== 204+0+0 (secure 0 0 0) 0x7f805c1f7f10 con 0x7f805c09b300 2024-07-17T08:41:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 msgr2=0x7f803c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 0x7f803c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f805c0734d0 tx=0x7f804c009040 comp rx=0 tx=0).stop 2024-07-17T08:41:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 msgr2=0x7f805c073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f804800b410 tx=0x7f804800b8d0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 shutdown_connections 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f803c076cd0 0x7f803c079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f805c0cfa90 0x7f805c147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f805c09b300 0x7f805c073be0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 --2- 172.21.15.44:0/3231930615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f805c06f4b0 0x7f805c1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 >> 172.21.15.44:0/3231930615 conn(0x7f805c0b0e80 msgr2=0x7f805c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 shutdown_connections 2024-07-17T08:41:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.115+0000 7f805b200640 1 -- 172.21.15.44:0/3231930615 wait complete. 2024-07-17T08:41:23.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:41:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 -- 172.21.15.44:0/2758102981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c80ce2f0 msgr2=0x7f59c80ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 --2- 172.21.15.44:0/2758102981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c80ce2f0 0x7f59c80ce730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f59c80b42f0 tx=0x7f59c002aa90 comp rx=0 tx=0).stop 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 -- 172.21.15.44:0/2758102981 shutdown_connections 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 --2- 172.21.15.44:0/2758102981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c80ce2f0 0x7f59c80ce730 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 --2- 172.21.15.44:0/2758102981 >> v2:172.21.15.175:3300/0 conn(0x7f59c8074510 0x7f59c81521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 -- 172.21.15.44:0/2758102981 >> 172.21.15.44:0/2758102981 conn(0x7f59c806c120 msgr2=0x7f59c806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 -- 172.21.15.44:0/2758102981 shutdown_connections 2024-07-17T08:41:23.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.351+0000 7f59cca00640 1 -- 172.21.15.44:0/2758102981 wait complete. 2024-07-17T08:41:23.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.354+0000 7f59cca00640 1 Processor -- start 2024-07-17T08:41:23.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.354+0000 7f59cca00640 1 -- start start 2024-07-17T08:41:23.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 0x7f59c81f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c8151dd0 0x7f59c81f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cb520 con 0x7f59c80ce2f0 2024-07-17T08:41:23.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cb3a0 con 0x7f59c8151dd0 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59cca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80cb6a0 con 0x7f59c8074510 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59bde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59bde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52512/0 (socket says 172.21.15.44:52512) 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.355+0000 7f59bde00640 1 -- 172.21.15.44:0/2874988061 learned_addr learned my addr 172.21.15.44:0/2874988061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59c6800640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 0x7f59c81f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 -- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 msgr2=0x7f59c81f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59c7200640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c8151dd0 0x7f59c81f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 0x7f59c81f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 -- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c8151dd0 msgr2=0x7f59c81f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c8151dd0 0x7f59c81f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 -- 172.21.15.44:0/2874988061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c8150330 con 0x7f59c80ce2f0 2024-07-17T08:41:23.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59c6800640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 0x7f59c81f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bde00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f59b000e790 tx=0x7f59b000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b000eea0 con 0x7f59c80ce2f0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59b000cdb0 con 0x7f59c80ce2f0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.356+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81f8650 con 0x7f59c80ce2f0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.357+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0010690 con 0x7f59c80ce2f0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.357+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81f4ab0 con 0x7f59c80ce2f0 2024-07-17T08:41:23.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.358+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c80cbbc0 con 0x7f59c80ce2f0 2024-07-17T08:41:23.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.359+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f59b0010830 con 0x7f59c80ce2f0 2024-07-17T08:41:23.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.360+0000 7f59bf400640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 0x7f59a4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.360+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7f59b0099340 con 0x7f59c80ce2f0 2024-07-17T08:41:23.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.360+0000 7f59c6800640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 0x7f59a4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.362+0000 7f59c6800640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 0x7f59a4079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59c809c700 tx=0x7f59b8009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:23.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.364+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f59c81f8650 con 0x7f59c80ce2f0 2024-07-17T08:41:23.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.566+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f59c80b1ca0 con 0x7f59c80ce2f0 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.567+0000 7f59bf400640 1 -- 172.21.15.44:0/2874988061 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v91) ==== 59+0+1998 (secure 0 0 0) 0x7f59b0062240 con 0x7f59c80ce2f0 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:41:23.568 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:41:23.569 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:41:23.570 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:41:23.570 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:41:23.570 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:41:23.570 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.569+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 msgr2=0x7f59a4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.569+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 0x7f59a4079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f59c809c700 tx=0x7f59b8009040 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.569+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 msgr2=0x7f59c814fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.569+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f59b000e790 tx=0x7f59b000ec50 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 shutdown_connections 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59a4076cd0 0x7f59a4079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59c8151dd0 0x7f59c81f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59c80ce2f0 0x7f59c814fdf0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 --2- 172.21.15.44:0/2874988061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59c8074510 0x7f59c81f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 >> 172.21.15.44:0/2874988061 conn(0x7f59c806c120 msgr2=0x7f59c80cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:23.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 shutdown_connections 2024-07-17T08:41:23.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.570+0000 7f59cca00640 1 -- 172.21.15.44:0/2874988061 wait complete. 2024-07-17T08:41:23.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 -- 172.21.15.44:0/1812031142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 msgr2=0x7f048c06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 --2- 172.21.15.44:0/1812031142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c06ec70 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f0478005fd0 tx=0x7f047802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 -- 172.21.15.44:0/1812031142 shutdown_connections 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 --2- 172.21.15.44:0/1812031142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c06ec70 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 --2- 172.21.15.44:0/1812031142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c151f20 0x7f048c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 -- 172.21.15.44:0/1812031142 >> 172.21.15.44:0/1812031142 conn(0x7f048c0cf0d0 msgr2=0x7f048c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 -- 172.21.15.44:0/1812031142 shutdown_connections 2024-07-17T08:41:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.798+0000 7f0492000640 1 -- 172.21.15.44:0/1812031142 wait complete. 2024-07-17T08:41:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.800+0000 7f0492000640 1 Processor -- start 2024-07-17T08:41:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.800+0000 7f0492000640 1 -- start start 2024-07-17T08:41:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c151f20 0x7f048c1f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 0x7f048c14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f048c0cb790 con 0x7f048c0cd0b0 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f048c06f530 con 0x7f048c151f20 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0492000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f048c0cb910 con 0x7f048c152300 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52520/0 (socket says 172.21.15.44:52520) 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 -- 172.21.15.44:0/2935896544 learned_addr learned my addr 172.21.15.44:0/2935896544 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f048be00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c151f20 0x7f048c1f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0491600640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 0x7f048c14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 -- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 msgr2=0x7f048c14f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 0x7f048c14f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 -- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c151f20 msgr2=0x7f048c1f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.801+0000 7f0490c00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c151f20 0x7f048c1f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0490c00640 1 -- 172.21.15.44:0/2935896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c0b03d0 con 0x7f048c0cd0b0 2024-07-17T08:41:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0491600640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 0x7f048c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0490c00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f047c00eab0 tx=0x7f047c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ce40 con 0x7f048c0cd0b0 2024-07-17T08:41:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f047c011560 con 0x7f048c0cd0b0 2024-07-17T08:41:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1553a0 con 0x7f048c0cd0b0 2024-07-17T08:41:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c011930 con 0x7f048c0cd0b0 2024-07-17T08:41:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.802+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c155960 con 0x7f048c0cd0b0 2024-07-17T08:41:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.804+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c06ec70 con 0x7f048c0cd0b0 2024-07-17T08:41:23.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.805+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f047c011ad0 con 0x7f048c0cd0b0 2024-07-17T08:41:23.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.806+0000 7f0488c00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 0x7f046c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:23.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.806+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7f047c0996e0 con 0x7f048c0cd0b0 2024-07-17T08:41:23.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.807+0000 7f048be00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 0x7f046c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:23.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.808+0000 7f048be00640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 0x7f046c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0478033000 tx=0x7f0478004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:23.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:23.811+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f047c016030 con 0x7f048c0cd0b0 2024-07-17T08:41:24.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.026+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f048c06da10 con 0x7f048c0cd0b0 2024-07-17T08:41:24.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.034+0000 7f0488c00640 1 -- 172.21.15.44:0/2935896544 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v92)=0 v92) ==== 122+0+0 (secure 0 0 0) 0x7f047c0624b0 con 0x7f048c0cd0b0 2024-07-17T08:41:24.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 msgr2=0x7f046c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:24.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 0x7f046c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0478033000 tx=0x7f0478004730 comp rx=0 tx=0).stop 2024-07-17T08:41:24.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 msgr2=0x7f048c0afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:24.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f047c00eab0 tx=0x7f047c00ef70 comp rx=0 tx=0).stop 2024-07-17T08:41:24.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 shutdown_connections 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f046c076cd0 0x7f046c079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f048c152300 0x7f048c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f048c151f20 0x7f048c1f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 --2- 172.21.15.44:0/2935896544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f048c0cd0b0 0x7f048c0afd40 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.037+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 >> 172.21.15.44:0/2935896544 conn(0x7f048c0cf0d0 msgr2=0x7f048c15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.038+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 shutdown_connections 2024-07-17T08:41:24.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.038+0000 7f0492000640 1 -- 172.21.15.44:0/2935896544 wait complete. 2024-07-17T08:41:24.057 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:41:24.058 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:24.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:24.298 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:41:24.298 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:24.298 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:24.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:41:24.358 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:24.358 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:24.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:24.414 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:24.414 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:24.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:41:24.473 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:24.473 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:24.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:24.528 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:41:24.528 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 -- 172.21.15.44:0/2552444322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 msgr2=0x7f57c815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 --2- 172.21.15.44:0/2552444322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c815f9d0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f57bc005fd0 tx=0x7f57bc02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 -- 172.21.15.44:0/2552444322 shutdown_connections 2024-07-17T08:41:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 --2- 172.21.15.44:0/2552444322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80b56b0 0x7f57c80b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 --2- 172.21.15.44:0/2552444322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c815f9d0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 -- 172.21.15.44:0/2552444322 >> 172.21.15.44:0/2552444322 conn(0x7f57c806bf40 msgr2=0x7f57c806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 -- 172.21.15.44:0/2552444322 shutdown_connections 2024-07-17T08:41:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.754+0000 7f57cec00640 1 -- 172.21.15.44:0/2552444322 wait complete. 2024-07-17T08:41:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.756+0000 7f57cec00640 1 Processor -- start 2024-07-17T08:41:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.756+0000 7f57cec00640 1 -- start start 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80b56b0 0x7f57c80b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80d1240 0x7f57c81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57c815c260 con 0x7f57c8158e70 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57c815c0e0 con 0x7f57c80b56b0 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.757+0000 7f57cec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57c815c3e0 con 0x7f57c80d1240 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52534/0 (socket says 172.21.15.44:52534) 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 -- 172.21.15.44:0/4290746626 learned_addr learned my addr 172.21.15.44:0/4290746626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57cd800640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80b56b0 0x7f57c80b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57cce00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80d1240 0x7f57c81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 -- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80d1240 msgr2=0x7f57c81f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80d1240 0x7f57c81f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 -- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80b56b0 msgr2=0x7f57c80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80b56b0 0x7f57c80b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.758+0000 7f57ce200640 1 -- 172.21.15.44:0/4290746626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c80b3e40 con 0x7f57c8158e70 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57ce200640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c7a0 tx=0x7f57c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003b70 con 0x7f57c8158e70 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57c0003d10 con 0x7f57c8158e70 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000aab0 con 0x7f57c8158e70 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c80b2860 con 0x7f57c8158e70 2024-07-17T08:41:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.759+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c80b3100 con 0x7f57c8158e70 2024-07-17T08:41:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.760+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c80b2c50 con 0x7f57c8158e70 2024-07-17T08:41:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.761+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f57c000ac50 con 0x7f57c8158e70 2024-07-17T08:41:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.762+0000 7f57c5600640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 0x7f57a8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.762+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7f57c0098750 con 0x7f57c8158e70 2024-07-17T08:41:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.763+0000 7f57cd800640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 0x7f57a8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.764+0000 7f57cd800640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 0x7f57a8079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f57bc033000 tx=0x7f57bc004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:24.767+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f57c0061740 con 0x7f57c8158e70 2024-07-17T08:41:25.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.017+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57c80d1b50 con 0x7f57c8158e70 2024-07-17T08:41:25.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.018+0000 7f57c5600640 1 -- 172.21.15.44:0/4290746626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+3923 (secure 0 0 0) 0x7f57c0061040 con 0x7f57c8158e70 2024-07-17T08:41:25.019 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:25.019 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":166,"btime":"2024-07-17T08:41:17:063866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3225190375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3225190375},{"type":"v1","addr":"172.21.15.175:6801","nonce":3225190375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":158},{"gid":9781,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/2658093512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2658093512},{"type":"v1","addr":"172.21.15.44:6834","nonce":2658093512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":9787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2621687329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2621687329},{"type":"v1","addr":"172.21.15.175:6836","nonce":2621687329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159},{"gid":12555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4284381669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4284381669},{"type":"v1","addr":"172.21.15.44:6835","nonce":4284381669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":166}],"filesystems":[]} 2024-07-17T08:41:25.019 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 166 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 msgr2=0x7f57a8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 0x7f57a8079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f57bc033000 tx=0x7f57bc004730 comp rx=0 tx=0).stop 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 msgr2=0x7f57c8145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c7a0 tx=0x7f57c000cc60 comp rx=0 tx=0).stop 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 shutdown_connections 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f57a8076cd0 0x7f57a8079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57c8158e70 0x7f57c8145610 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80d1240 0x7f57c81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 --2- 172.21.15.44:0/4290746626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80b56b0 0x7f57c80b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 >> 172.21.15.44:0/4290746626 conn(0x7f57c806bf40 msgr2=0x7f57c815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 shutdown_connections 2024-07-17T08:41:25.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.021+0000 7f57cec00640 1 -- 172.21.15.44:0/4290746626 wait complete. 2024-07-17T08:41:25.042 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:41:25.042 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:41:25.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:41:25.042 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:41:25.043 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:41:25.043 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:41:25.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:41:25.043 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:41:25.043 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:41:25.043 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:41:25.044 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:41:25.044 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:41:25.044 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:41:25.044 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:41:25.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:41:25.044 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:41:25.062 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:41:25.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:41:25.063 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:41:25.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:41:25.067 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:41:25.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:41:25.069 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:41:25.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:41:25.107 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:41:25.108 DEBUG:teuthology.parallel:result is None 2024-07-17T08:41:25.110 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:41:25.110 DEBUG:teuthology.parallel:result is None 2024-07-17T08:41:25.116 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:41:25.116 DEBUG:teuthology.parallel:result is None 2024-07-17T08:41:25.122 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:41:25.122 DEBUG:teuthology.parallel:result is None 2024-07-17T08:41:25.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:41:25.255 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:41:25.256 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:41:25.262 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:41:25.277 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 -- 172.21.15.44:0/1955429461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370006efc0 msgr2=0x7f37000ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 --2- 172.21.15.44:0/1955429461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370006efc0 0x7f37000ac6d0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f3700070b60 tx=0x7f36f802aa10 comp rx=0 tx=0).stop 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 -- 172.21.15.44:0/1955429461 shutdown_connections 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 --2- 172.21.15.44:0/1955429461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370006efc0 0x7f37000ac6d0 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 --2- 172.21.15.44:0/1955429461 >> v2:172.21.15.175:3300/0 conn(0x7f37000b1af0 0x7f37000ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 --2- 172.21.15.44:0/1955429461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370015a280 0x7f37000b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 -- 172.21.15.44:0/1955429461 >> 172.21.15.44:0/1955429461 conn(0x7f37000b0eb0 msgr2=0x7f37000b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 -- 172.21.15.44:0/1955429461 shutdown_connections 2024-07-17T08:41:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.302+0000 7f3707600640 1 -- 172.21.15.44:0/1955429461 wait complete. 2024-07-17T08:41:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.304+0000 7f3707600640 1 Processor -- start 2024-07-17T08:41:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.304+0000 7f3707600640 1 -- start start 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 0x7f37001482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37000b1af0 0x7f3700074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f370015ad90 con 0x7f370015a280 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f370015ac10 con 0x7f370006efc0 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3707600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f370015af10 con 0x7f37000b1af0 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.305+0000 7f3705600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52576/0 (socket says 172.21.15.44:52576) 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 -- 172.21.15.44:0/757013636 learned_addr learned my addr 172.21.15.44:0/757013636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3704c00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 0x7f37001482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f36ffe00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37000b1af0 0x7f3700074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 -- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37000b1af0 msgr2=0x7f3700074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37000b1af0 0x7f3700074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 -- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 msgr2=0x7f37001482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 0x7f37001482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 -- 172.21.15.44:0/757013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370015d4f0 con 0x7f370015a280 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3704c00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 0x7f37001482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:41:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.306+0000 7f3705600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d60 tx=0x7f36f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.307+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8047070 con 0x7f370015a280 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.307+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f802f670 con 0x7f370015a280 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.307+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8033050 con 0x7f370015a280 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.307+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3700145bc0 con 0x7f370015a280 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.307+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370015b9c0 con 0x7f370015a280 2024-07-17T08:41:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.308+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37000ce300 con 0x7f370015a280 2024-07-17T08:41:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.309+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f36f804a070 con 0x7f370015a280 2024-07-17T08:41:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.310+0000 7f36fcc00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 0x7f36e0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.310+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(300..300 src has 1..300) ==== 6670+0+0 (secure 0 0 0) 0x7f36f80b9a30 con 0x7f370015a280 2024-07-17T08:41:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.311+0000 7f3704c00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 0x7f36e0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.312+0000 7f3704c00640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 0x7f36e0079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009b80 tx=0x7f36f4005f90 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:25.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.314+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f37000ce300 con 0x7f370015a280 2024-07-17T08:41:25.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:25.535+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f37000b21b0 con 0x7f370015a280 2024-07-17T08:41:26.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.150+0000 7f36fcc00640 1 -- 172.21.15.44:0/757013636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v302) ==== 97+0+0 (secure 0 0 0) 0x7f36f8082230 con 0x7f370015a280 2024-07-17T08:41:26.151 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:41:26.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 msgr2=0x7f36e0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 0x7f36e0079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009b80 tx=0x7f36f4005f90 comp rx=0 tx=0).stop 2024-07-17T08:41:26.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 msgr2=0x7f3700145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d60 tx=0x7f36f8003f10 comp rx=0 tx=0).stop 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 shutdown_connections 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f36e0076cd0 0x7f36e0079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f370015a280 0x7f3700145680 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37000b1af0 0x7f3700074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 --2- 172.21.15.44:0/757013636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370006efc0 0x7f37001482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.154+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 >> 172.21.15.44:0/757013636 conn(0x7f37000b0eb0 msgr2=0x7f370014f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.155+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 shutdown_connections 2024-07-17T08:41:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.155+0000 7f3707600640 1 -- 172.21.15.44:0/757013636 wait complete. 2024-07-17T08:41:26.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:41:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 -- 172.21.15.44:0/1454683494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc004800 msgr2=0x7f59cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 --2- 172.21.15.44:0/1454683494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc004800 0x7f59cc004c80 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009b50 tx=0x7f59cc005830 comp rx=0 tx=0).stop 2024-07-17T08:41:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 -- 172.21.15.44:0/1454683494 shutdown_connections 2024-07-17T08:41:26.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 --2- 172.21.15.44:0/1454683494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc004800 0x7f59cc004c80 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 -- 172.21.15.44:0/1454683494 >> 172.21.15.44:0/1454683494 conn(0x7f59e40b0e80 msgr2=0x7f59e40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:26.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.380+0000 7f59eb000640 1 -- 172.21.15.44:0/1454683494 shutdown_connections 2024-07-17T08:41:26.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.381+0000 7f59eb000640 1 -- 172.21.15.44:0/1454683494 wait complete. 2024-07-17T08:41:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.383+0000 7f59eb000640 1 Processor -- start 2024-07-17T08:41:26.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.383+0000 7f59eb000640 1 -- start start 2024-07-17T08:41:26.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59cc004800 0x7f59e4139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59e40c84e0 0x7f59e40c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41403c0 con 0x7f59e406e710 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4140240 con 0x7f59e40c84e0 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.384+0000 7f59eb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59e40b3870 con 0x7f59cc004800 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52592/0 (socket says 172.21.15.44:52592) 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 -- 172.21.15.44:0/452815009 learned_addr learned my addr 172.21.15.44:0/452815009 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:26.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59e9c00640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59cc004800 0x7f59e4139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59e9200640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59e40c84e0 0x7f59e40c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 -- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59cc004800 msgr2=0x7f59e4139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59cc004800 0x7f59e4139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 -- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59e40c84e0 msgr2=0x7f59e40c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59e40c84e0 0x7f59e40c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 -- 172.21.15.44:0/452815009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e4137330 con 0x7f59e406e710 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.385+0000 7f59ea600640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f59e40cfdb0 tx=0x7f59dc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.386+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00cea0 con 0x7f59e406e710 2024-07-17T08:41:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.386+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59dc003a60 con 0x7f59e406e710 2024-07-17T08:41:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.386+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc005070 con 0x7f59e406e710 2024-07-17T08:41:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.386+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e4136720 con 0x7f59e406e710 2024-07-17T08:41:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.386+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e4136ce0 con 0x7f59e406e710 2024-07-17T08:41:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.387+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e40b3dd0 con 0x7f59e406e710 2024-07-17T08:41:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.388+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f59dc011070 con 0x7f59e406e710 2024-07-17T08:41:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.389+0000 7f59e1600640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 0x7f59c40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.389+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(304..304 src has 1..304) ==== 5853+0+0 (secure 0 0 0) 0x7f59dc097a20 con 0x7f59e406e710 2024-07-17T08:41:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.390+0000 7f59e9c00640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 0x7f59c40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.391+0000 7f59e9c00640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 0x7f59c40791c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59d8007fd0 tx=0x7f59d8009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.394+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f59dc060d40 con 0x7f59e406e710 2024-07-17T08:41:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.673+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f59e406df10 con 0x7f59e406e710 2024-07-17T08:41:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.674+0000 7f59e1600640 1 -- 172.21.15.44:0/452815009 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v77) ==== 80+0+5304 (secure 0 0 0) 0x7f59dc060640 con 0x7f59e406e710 2024-07-17T08:41:26.675 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:26.675 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:41:26.675 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:41:26.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:41:26.676 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:41:26.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:41:26.679 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.682 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.682 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:41:26.683 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:41:26.683 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:41:26.686 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:41:26.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:41:26.689 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:41:26.690 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:41:26.693 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:41:26.694 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:41:26.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 msgr2=0x7f59c40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 0x7f59c40791c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59d8007fd0 tx=0x7f59d8009040 comp rx=0 tx=0).stop 2024-07-17T08:41:26.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 msgr2=0x7f59e406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f59e40cfdb0 tx=0x7f59dc00cb90 comp rx=0 tx=0).stop 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 shutdown_connections 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f59c4076d00 0x7f59c40791c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59e406e710 0x7f59e406eb90 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59e40c84e0 0x7f59e40c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 --2- 172.21.15.44:0/452815009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59cc004800 0x7f59e4139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 >> 172.21.15.44:0/452815009 conn(0x7f59e40b0e80 msgr2=0x7f59e409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:26.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.677+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 shutdown_connections 2024-07-17T08:41:26.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.678+0000 7f59eb000640 1 -- 172.21.15.44:0/452815009 wait complete. 2024-07-17T08:41:26.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:41:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 -- 172.21.15.44:0/4011181927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415bac0 msgr2=0x7f13940b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 --2- 172.21.15.44:0/4011181927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415bac0 0x7f13940b3690 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f13940725f0 tx=0x7f138c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 -- 172.21.15.44:0/4011181927 shutdown_connections 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 --2- 172.21.15.44:0/4011181927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415bac0 0x7f13940b3690 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 --2- 172.21.15.44:0/4011181927 >> v2:172.21.15.175:3300/0 conn(0x7f139415a230 0x7f13940b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.906+0000 7f139a800640 1 -- 172.21.15.44:0/4011181927 >> 172.21.15.44:0/4011181927 conn(0x7f13940b5890 msgr2=0x7f13940b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.907+0000 7f139a800640 1 -- 172.21.15.44:0/4011181927 shutdown_connections 2024-07-17T08:41:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.907+0000 7f139a800640 1 -- 172.21.15.44:0/4011181927 wait complete. 2024-07-17T08:41:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.908+0000 7f139a800640 1 Processor -- start 2024-07-17T08:41:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 -- start start 2024-07-17T08:41:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13940b0f60 0x7f13941582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 0x7f1394146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b1630 con 0x7f139415a230 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.909+0000 7f139a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13940b14b0 con 0x7f13940b0f60 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f139a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13940b17b0 con 0x7f139415bac0 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52610/0 (socket says 172.21.15.44:52610) 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 -- 172.21.15.44:0/1323985220 learned_addr learned my addr 172.21.15.44:0/1323985220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1399e00640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 0x7f1394146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1399400640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13940b0f60 0x7f13941582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 -- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 msgr2=0x7f1394146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 0x7f1394146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 -- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13940b0f60 msgr2=0x7f13941582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.910+0000 7f1398a00640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13940b0f60 0x7f13941582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1398a00640 1 -- 172.21.15.44:0/1323985220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13941e7090 con 0x7f139415a230 2024-07-17T08:41:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1399e00640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 0x7f1394146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:41:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1398a00640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f138400c7c0 tx=0x7f138400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384016020 con 0x7f139415a230 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f138400ab40 con 0x7f139415a230 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13941e7370 con 0x7f139415a230 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384012420 con 0x7f139415a230 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.911+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13941e7930 con 0x7f139415a230 2024-07-17T08:41:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.912+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139415e670 con 0x7f139415a230 2024-07-17T08:41:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.914+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f1384019020 con 0x7f139415a230 2024-07-17T08:41:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.915+0000 7f1391600640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 0x7f1378079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.915+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(304..304 src has 1..304) ==== 5853+0+0 (secure 0 0 0) 0x7f1384098300 con 0x7f139415a230 2024-07-17T08:41:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.916+0000 7f1399400640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 0x7f1378079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.917+0000 7f1399400640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 0x7f1378079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1388005fd0 tx=0x7f1388005850 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:26.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:26.919+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1384020070 con 0x7f139415a230 2024-07-17T08:41:27.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.134+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f139406c220 con 0x7f139415a230 2024-07-17T08:41:27.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.138+0000 7f1391600640 1 -- 172.21.15.44:0/1323985220 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v304) ==== 84+0+65 (secure 0 0 0) 0x7f13840616c0 con 0x7f139415a230 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:27.139 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.140+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 msgr2=0x7f1378079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 0x7f1378079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1388005fd0 tx=0x7f1388005850 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 msgr2=0x7f1394146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f138400c7c0 tx=0x7f138400cc80 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 shutdown_connections 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1378076cd0 0x7f1378079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139415bac0 0x7f1394146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f139415a230 0x7f1394146220 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 --2- 172.21.15.44:0/1323985220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13940b0f60 0x7f13941582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 >> 172.21.15.44:0/1323985220 conn(0x7f13940b5890 msgr2=0x7f13941595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:27.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 shutdown_connections 2024-07-17T08:41:27.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.141+0000 7f139a800640 1 -- 172.21.15.44:0/1323985220 wait complete. 2024-07-17T08:41:27.161 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:41:27.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:41:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 -- 172.21.15.44:0/1688231135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c15a5d0 msgr2=0x7f3f7c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 --2- 172.21.15.44:0/1688231135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c1539f0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0b1ef0 tx=0x7f3f6402a560 comp rx=0 tx=0).stop 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 -- 172.21.15.44:0/1688231135 shutdown_connections 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 --2- 172.21.15.44:0/1688231135 >> v2:172.21.15.175:3300/0 conn(0x7f3f7c0b21b0 0x7f3f7c0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 --2- 172.21.15.44:0/1688231135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c1539f0 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 -- 172.21.15.44:0/1688231135 >> 172.21.15.44:0/1688231135 conn(0x7f3f7c0b0eb0 msgr2=0x7f3f7c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 -- 172.21.15.44:0/1688231135 shutdown_connections 2024-07-17T08:41:27.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.371+0000 7f3f82c00640 1 -- 172.21.15.44:0/1688231135 wait complete. 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.373+0000 7f3f82c00640 1 Processor -- start 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.373+0000 7f3f82c00640 1 -- start start 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f7c0b21b0 0x7f3f7c09c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0b2ef0 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0b2d70 con 0x7f3f7c15a5d0 2024-07-17T08:41:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f82c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c0b3070 con 0x7f3f7c0b21b0 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f7be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52634/0 (socket says 172.21.15.44:52634) 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.374+0000 7f3f7be00640 1 -- 172.21.15.44:0/2411381751 learned_addr learned my addr 172.21.15.44:0/2411381751 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f73e00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f7c0b21b0 0x7f3f7c09c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f80c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 -- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f7c0b21b0 msgr2=0x7f3f7c09c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f7c0b21b0 0x7f3f7c09c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 -- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 msgr2=0x7f3f7c09b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:27.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c09b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 -- 172.21.15.44:0/2411381751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c153010 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f80c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f7be00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c430 tx=0x7f3f6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.375+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c011790 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.376+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f6c017070 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.376+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c011d00 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.376+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c152b70 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.376+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c0b0820 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.377+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c0b2620 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.378+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f3f6c01c460 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.379+0000 7f3f78c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 0x7f3f58079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.379+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(304..304 src has 1..304) ==== 5853+0+0 (secure 0 0 0) 0x7f3f6c0984c0 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.380+0000 7f3f73e00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 0x7f3f58079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.381+0000 7f3f73e00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 0x7f3f58079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3f64033000 tx=0x7f3f64004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.384+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3f6c0617e0 con 0x7f3f7c06f6d0 2024-07-17T08:41:27.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:27.597+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3f7c152d40 con 0x7f3f7c06f6d0 2024-07-17T08:41:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.158+0000 7f3f78c00640 1 -- 172.21.15.44:0/2411381751 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v305) ==== 138+0+0 (secure 0 0 0) 0x7f3f6c0610e0 con 0x7f3f7c06f6d0 2024-07-17T08:41:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:41:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 msgr2=0x7f3f58079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 0x7f3f58079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3f64033000 tx=0x7f3f64004730 comp rx=0 tx=0).stop 2024-07-17T08:41:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 msgr2=0x7f3f7c09c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c430 tx=0x7f3f6c00c8f0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 shutdown_connections 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.163+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f3f58076cd0 0x7f3f58079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f7c15a5d0 0x7f3f7c09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f7c0b21b0 0x7f3f7c09c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 --2- 172.21.15.44:0/2411381751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f7c06f6d0 0x7f3f7c09c330 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 >> 172.21.15.44:0/2411381751 conn(0x7f3f7c0b0eb0 msgr2=0x7f3f7c0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 shutdown_connections 2024-07-17T08:41:28.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.164+0000 7f3f82c00640 1 -- 172.21.15.44:0/2411381751 wait complete. 2024-07-17T08:41:28.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 -- 172.21.15.44:0/1418539410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f3c003aa0 msgr2=0x7f2f3c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 --2- 172.21.15.44:0/1418539410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f3c003aa0 0x7f2f3c003ec0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f2f48005fd0 tx=0x7f2f3c005830 comp rx=0 tx=0).stop 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5dc00640 1 -- 172.21.15.44:0/1418539410 reap_dead start 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 -- 172.21.15.44:0/1418539410 shutdown_connections 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 -- 172.21.15.44:0/1418539410 >> 172.21.15.44:0/1418539410 conn(0x7f2f580b4ff0 msgr2=0x7f2f580b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 -- 172.21.15.44:0/1418539410 shutdown_connections 2024-07-17T08:41:28.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.403+0000 7f2f5e600640 1 -- 172.21.15.44:0/1418539410 wait complete. 2024-07-17T08:41:28.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.405+0000 7f2f5e600640 1 Processor -- start 2024-07-17T08:41:28.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.405+0000 7f2f5e600640 1 -- start start 2024-07-17T08:41:28.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f3c003aa0 0x7f2f580ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 0x7f2f581f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f581623e0 con 0x7f2f581f81a0 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f58162260 con 0x7f2f3c003aa0 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f58162560 con 0x7f2f580cd480 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52650/0 (socket says 172.21.15.44:52650) 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.406+0000 7f2f5dc00640 1 -- 172.21.15.44:0/2970719617 learned_addr learned my addr 172.21.15.44:0/2970719617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5d200640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f3c003aa0 0x7f2f580ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f57e00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 0x7f2f581f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 -- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 msgr2=0x7f2f581f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 0x7f2f581f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 -- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f3c003aa0 msgr2=0x7f2f580ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:28.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f3c003aa0 0x7f2f580ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:28.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 -- 172.21.15.44:0/2970719617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f58159040 con 0x7f2f581f81a0 2024-07-17T08:41:28.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f57e00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 0x7f2f581f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:28.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f5dc00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f2f50001d80 tx=0x7f2f5000e810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:28.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f500107d0 con 0x7f2f581f81a0 2024-07-17T08:41:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f50016070 con 0x7f2f581f81a0 2024-07-17T08:41:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.407+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50010d00 con 0x7f2f581f81a0 2024-07-17T08:41:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.408+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f58159320 con 0x7f2f581f81a0 2024-07-17T08:41:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.408+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5815edc0 con 0x7f2f581f81a0 2024-07-17T08:41:28.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.409+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f580cfdf0 con 0x7f2f581f81a0 2024-07-17T08:41:28.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.411+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f2f5001b460 con 0x7f2f581f81a0 2024-07-17T08:41:28.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.412+0000 7f2f54c00640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 0x7f2f34079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:28.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.412+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(305..305 src has 1..305) ==== 6223+0+0 (secure 0 0 0) 0x7f2f50099650 con 0x7f2f581f81a0 2024-07-17T08:41:28.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.413+0000 7f2f5d200640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 0x7f2f34079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:28.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.414+0000 7f2f5d200640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 0x7f2f34079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035000 tx=0x7f2f48004a40 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:28.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.416+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2f50062770 con 0x7f2f581f81a0 2024-07-17T08:41:28.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:28.640+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2f580d0e10 con 0x7f2f581f81a0 2024-07-17T08:41:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.166+0000 7f2f54c00640 1 -- 172.21.15.44:0/2970719617 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v306) ==== 170+0+0 (secure 0 0 0) 0x7f2f50062070 con 0x7f2f581f81a0 2024-07-17T08:41:29.167 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:41:29.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.169+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 msgr2=0x7f2f34079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:29.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.169+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 0x7f2f34079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035000 tx=0x7f2f48004a40 comp rx=0 tx=0).stop 2024-07-17T08:41:29.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.169+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 msgr2=0x7f2f58158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:29.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.169+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f2f50001d80 tx=0x7f2f5000e810 comp rx=0 tx=0).stop 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 shutdown_connections 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2f34076cd0 0x7f2f34079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f581f81a0 0x7f2f58158310 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f580cd480 0x7f2f581f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 --2- 172.21.15.44:0/2970719617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f3c003aa0 0x7f2f580ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.170+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 >> 172.21.15.44:0/2970719617 conn(0x7f2f580b4ff0 msgr2=0x7f2f580b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:29.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.171+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 shutdown_connections 2024-07-17T08:41:29.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.172+0000 7f2f5e600640 1 -- 172.21.15.44:0/2970719617 wait complete. 2024-07-17T08:41:29.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.393+0000 7f1c39600640 1 -- 172.21.15.44:0/4206959068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c34154490 msgr2=0x7f1c3415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.393+0000 7f1c39600640 1 --2- 172.21.15.44:0/4206959068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c34154490 0x7f1c3415d830 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005fd0 tx=0x7f1c2402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 -- 172.21.15.44:0/4206959068 shutdown_connections 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 --2- 172.21.15.44:0/4206959068 >> v2:172.21.15.175:3300/0 conn(0x7f1c3406f4f0 0x7f1c340ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 --2- 172.21.15.44:0/4206959068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c34154490 0x7f1c3415d830 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 -- 172.21.15.44:0/4206959068 >> 172.21.15.44:0/4206959068 conn(0x7f1c340cf0d0 msgr2=0x7f1c340cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 -- 172.21.15.44:0/4206959068 shutdown_connections 2024-07-17T08:41:29.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.394+0000 7f1c39600640 1 -- 172.21.15.44:0/4206959068 wait complete. 2024-07-17T08:41:29.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.396+0000 7f1c39600640 1 Processor -- start 2024-07-17T08:41:29.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.396+0000 7f1c39600640 1 -- start start 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.397+0000 7f1c39600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.397+0000 7f1c39600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c34154490 0x7f1c34156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.397+0000 7f1c39600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c3415dd70 0x7f1c34156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c39600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340b2220 con 0x7f1c3406f4f0 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c39600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340b20a0 con 0x7f1c34154490 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c39600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340b23a0 con 0x7f1c3415dd70 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55116/0 (socket says 172.21.15.44:55116) 2024-07-17T08:41:29.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 -- 172.21.15.44:0/2035323908 learned_addr learned my addr 172.21.15.44:0/2035323908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c2be00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c34154490 0x7f1c34156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c38c00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c3415dd70 0x7f1c34156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 -- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c3415dd70 msgr2=0x7f1c34156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c3415dd70 0x7f1c34156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 -- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c34154490 msgr2=0x7f1c34156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c34154490 0x7f1c34156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:29.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.398+0000 7f1c33e00640 1 -- 172.21.15.44:0/2035323908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c34095cd0 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c33e00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009850 tx=0x7f1c240035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2403f070 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c24003ce0 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24035030 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c34157230 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.399+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c34095f40 con 0x7f1c3406f4f0 2024-07-17T08:41:29.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.401+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c340b2930 con 0x7f1c3406f4f0 2024-07-17T08:41:29.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.402+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f1c24042070 con 0x7f1c3406f4f0 2024-07-17T08:41:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.403+0000 7f1c30c00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 0x7f1c100791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.403+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(306..306 src has 1..306) ==== 6605+0+0 (secure 0 0 0) 0x7f1c240b98c0 con 0x7f1c3406f4f0 2024-07-17T08:41:29.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.404+0000 7f1c2be00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 0x7f1c100791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:29.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.405+0000 7f1c2be00640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 0x7f1c100791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b42e0 tx=0x7f1c1c009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.407+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1c24082860 con 0x7f1c3406f4f0 2024-07-17T08:41:29.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:29.656+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c341606e0 con 0x7f1c3406f4f0 2024-07-17T08:41:30.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.226+0000 7f1c30c00640 1 -- 172.21.15.44:0/2035323908 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 41 and data pool 42 v169) ==== 174+0+0 (secure 0 0 0) 0x7f1c24082160 con 0x7f1c3406f4f0 2024-07-17T08:41:30.230 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 41 and data pool 42 2024-07-17T08:41:30.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 msgr2=0x7f1c100791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 0x7f1c100791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b42e0 tx=0x7f1c1c009040 comp rx=0 tx=0).stop 2024-07-17T08:41:30.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 msgr2=0x7f1c341fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009850 tx=0x7f1c240035f0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 shutdown_connections 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1c10076d00 0x7f1c100791c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c3415dd70 0x7f1c34156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c34154490 0x7f1c34156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 --2- 172.21.15.44:0/2035323908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c3406f4f0 0x7f1c341fb240 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.235+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 >> 172.21.15.44:0/2035323908 conn(0x7f1c340cf0d0 msgr2=0x7f1c340adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.236+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 shutdown_connections 2024-07-17T08:41:30.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.236+0000 7f1c39600640 1 -- 172.21.15.44:0/2035323908 wait complete. 2024-07-17T08:41:30.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:30.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 -- 172.21.15.44:0/60538150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0480ae810 msgr2=0x7ff04815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 --2- 172.21.15.44:0/60538150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0480ae810 0x7ff04815e510 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff038005fd0 tx=0x7ff03802aa90 comp rx=0 tx=0).stop 2024-07-17T08:41:30.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 -- 172.21.15.44:0/60538150 shutdown_connections 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 --2- 172.21.15.44:0/60538150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ea50 0x7ff0480ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 --2- 172.21.15.44:0/60538150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0480ae810 0x7ff04815e510 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 -- 172.21.15.44:0/60538150 >> 172.21.15.44:0/60538150 conn(0x7ff0480b0fb0 msgr2=0x7ff0480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 -- 172.21.15.44:0/60538150 shutdown_connections 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.444+0000 7ff04ec00640 1 -- 172.21.15.44:0/60538150 wait complete. 2024-07-17T08:41:30.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.446+0000 7ff04ec00640 1 Processor -- start 2024-07-17T08:41:30.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.446+0000 7ff04ec00640 1 -- start start 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04806ea50 0x7ff048141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ee90 0x7ff0480989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0480b2ce0 con 0x7ff0480ae810 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff04806f830 con 0x7ff04806ea50 2024-07-17T08:41:30.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0480b2e60 con 0x7ff04806ee90 2024-07-17T08:41:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.447+0000 7ff04cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55132/0 (socket says 172.21.15.44:55132) 2024-07-17T08:41:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 -- 172.21.15.44:0/1626432694 learned_addr learned my addr 172.21.15.44:0/1626432694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:30.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff047400640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ee90 0x7ff0480989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff047e00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04806ea50 0x7ff048141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 -- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ee90 msgr2=0x7ff0480989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ee90 0x7ff0480989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 -- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04806ea50 msgr2=0x7ff048141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04806ea50 0x7ff048141760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 -- 172.21.15.44:0/1626432694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff048074650 con 0x7ff0480ae810 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.448+0000 7ff04cc00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c370 tx=0x7ff04000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:30.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.449+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040017070 con 0x7ff0480ae810 2024-07-17T08:41:30.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.449+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0481f46d0 con 0x7ff0480ae810 2024-07-17T08:41:30.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.449+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0481f4c90 con 0x7ff0480ae810 2024-07-17T08:41:30.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.449+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0400052d0 con 0x7ff0480ae810 2024-07-17T08:41:30.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.449+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040003810 con 0x7ff0480ae810 2024-07-17T08:41:30.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.450+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff020005290 con 0x7ff0480ae810 2024-07-17T08:41:30.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.452+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7ff04001a070 con 0x7ff0480ae810 2024-07-17T08:41:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.452+0000 7ff03fe00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 0x7ff028079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.453+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(307..307 src has 1..307) ==== 6705+0+0 (secure 0 0 0) 0x7ff040097eb0 con 0x7ff0480ae810 2024-07-17T08:41:30.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.453+0000 7ff047e00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 0x7ff028079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.455+0000 7ff047e00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 0x7ff028079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff038035000 tx=0x7ff038004ab0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.474+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff048074650 con 0x7ff0480ae810 2024-07-17T08:41:30.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.676+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff020005850 con 0x7ff0480ae810 2024-07-17T08:41:30.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.679+0000 7ff03fe00640 1 -- 172.21.15.44:0/1626432694 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v307) ==== 74+0+17879 (secure 0 0 0) 0x7ff04815ea50 con 0x7ff0480ae810 2024-07-17T08:41:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:30.681 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":307,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:29.658305+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:41:27.625746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":42,"pool_name":"cephfs_data","create_time":"2024-07-17T08:41:28.667439+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:30.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 msgr2=0x7ff028079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 0x7ff028079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff038035000 tx=0x7ff038004ab0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 msgr2=0x7ff048074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c370 tx=0x7ff04000c830 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 shutdown_connections 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7ff028076cd0 0x7ff028079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0480ae810 0x7ff048074110 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff04806ee90 0x7ff0480989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 --2- 172.21.15.44:0/1626432694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff04806ea50 0x7ff048141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 >> 172.21.15.44:0/1626432694 conn(0x7ff0480b0fb0 msgr2=0x7ff0480b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 shutdown_connections 2024-07-17T08:41:30.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.682+0000 7ff04ec00640 1 -- 172.21.15.44:0/1626432694 wait complete. 2024-07-17T08:41:30.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 -- 172.21.15.44:0/221338921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 msgr2=0x7f1b8415de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 --2- 172.21.15.44:0/221338921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b8415de20 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f1b84069c30 tx=0x7f1b7c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 -- 172.21.15.44:0/221338921 shutdown_connections 2024-07-17T08:41:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 --2- 172.21.15.44:0/221338921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b8415de20 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 --2- 172.21.15.44:0/221338921 >> v2:172.21.15.175:3300/0 conn(0x7f1b84154580 0x7f1b84072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 -- 172.21.15.44:0/221338921 >> 172.21.15.44:0/221338921 conn(0x7f1b8406bf40 msgr2=0x7f1b8406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 -- 172.21.15.44:0/221338921 shutdown_connections 2024-07-17T08:41:30.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.894+0000 7f1b89000640 1 -- 172.21.15.44:0/221338921 wait complete. 2024-07-17T08:41:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.897+0000 7f1b89000640 1 Processor -- start 2024-07-17T08:41:30.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.897+0000 7f1b89000640 1 -- start start 2024-07-17T08:41:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b84154580 0x7f1b84156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8415d9f0 0x7f1b841f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b840adc30 con 0x7f1b84073e70 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b840adab0 con 0x7f1b8415d9f0 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b89000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b840addb0 con 0x7f1b84154580 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b81e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55148/0 (socket says 172.21.15.44:55148) 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.898+0000 7f1b81e00640 1 -- 172.21.15.44:0/2459487244 learned_addr learned my addr 172.21.15.44:0/2459487244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 -- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b84154580 msgr2=0x7f1b84156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b84154580 0x7f1b84156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 -- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8415d9f0 msgr2=0x7f1b841f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8415d9f0 0x7f1b841f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 -- 172.21.15.44:0/2459487244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b841f5080 con 0x7f1b84073e70 2024-07-17T08:41:30.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b81e00640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400eab0 tx=0x7f1b7400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400ce40 con 0x7f1b84073e70 2024-07-17T08:41:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b74004c10 con 0x7f1b84073e70 2024-07-17T08:41:30.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b841f5360 con 0x7f1b84073e70 2024-07-17T08:41:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.899+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74005020 con 0x7f1b84073e70 2024-07-17T08:41:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.900+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b84152d30 con 0x7f1b84073e70 2024-07-17T08:41:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.901+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b840ae210 con 0x7f1b84073e70 2024-07-17T08:41:30.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.902+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f1b740051c0 con 0x7f1b84073e70 2024-07-17T08:41:30.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.903+0000 7f1b7aa00640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 0x7f1b60079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:30.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.903+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(307..307 src has 1..307) ==== 6705+0+0 (secure 0 0 0) 0x7f1b74099290 con 0x7f1b84073e70 2024-07-17T08:41:30.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.903+0000 7f1b79400640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 0x7f1b60079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:30.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.905+0000 7f1b79400640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 0x7f1b60079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006fd0 tx=0x7f1b6c006bc0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:30.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:30.907+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1b74016030 con 0x7f1b84073e70 2024-07-17T08:41:31.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.120+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b84068b70 con 0x7f1b84073e70 2024-07-17T08:41:31.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.121+0000 7f1b7aa00640 1 -- 172.21.15.44:0/2459487244 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v307) ==== 74+0+17879 (secure 0 0 0) 0x7f1b740621e0 con 0x7f1b84073e70 2024-07-17T08:41:31.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:31.123 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":307,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:29.658305+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:41:27.625746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":42,"pool_name":"cephfs_data","create_time":"2024-07-17T08:41:28.667439+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.124+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 msgr2=0x7f1b60079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.124+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 0x7f1b60079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006fd0 tx=0x7f1b6c006bc0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.124+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 msgr2=0x7f1b84158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.124+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400eab0 tx=0x7f1b7400ef70 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 shutdown_connections 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1b60076cd0 0x7f1b60079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b8415d9f0 0x7f1b841f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b84154580 0x7f1b84156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 --2- 172.21.15.44:0/2459487244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b84073e70 0x7f1b84158a70 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 >> 172.21.15.44:0/2459487244 conn(0x7f1b8406bf40 msgr2=0x7f1b840afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:31.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 shutdown_connections 2024-07-17T08:41:31.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.125+0000 7f1b89000640 1 -- 172.21.15.44:0/2459487244 wait complete. 2024-07-17T08:41:31.144 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:41:31.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 -- 172.21.15.44:0/1952426944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 msgr2=0x7f7d300b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 --2- 172.21.15.44:0/1952426944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300b2200 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f7d3008c800 tx=0x7f7d1c02a560 comp rx=0 tx=0).stop 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 -- 172.21.15.44:0/1952426944 shutdown_connections 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 --2- 172.21.15.44:0/1952426944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300b2200 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 --2- 172.21.15.44:0/1952426944 >> v2:172.21.15.175:3300/0 conn(0x7f7d3015d130 0x7f7d3015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.355+0000 7f7d36a00640 1 -- 172.21.15.44:0/1952426944 >> 172.21.15.44:0/1952426944 conn(0x7f7d300b4ff0 msgr2=0x7f7d300b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.356+0000 7f7d36a00640 1 -- 172.21.15.44:0/1952426944 shutdown_connections 2024-07-17T08:41:31.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.356+0000 7f7d36a00640 1 -- 172.21.15.44:0/1952426944 wait complete. 2024-07-17T08:41:31.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.357+0000 7f7d36a00640 1 Processor -- start 2024-07-17T08:41:31.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.357+0000 7f7d36a00640 1 -- start start 2024-07-17T08:41:31.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d300ce690 0x7f7d300cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:31.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d3015d130 0x7f7d300b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300d1a00 con 0x7f7d30161be0 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300d1880 con 0x7f7d3015d130 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.358+0000 7f7d36a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300d1b80 con 0x7f7d300ce690 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55162/0 (socket says 172.21.15.44:55162) 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 -- 172.21.15.44:0/3328067433 learned_addr learned my addr 172.21.15.44:0/3328067433 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d34c00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d3015d130 0x7f7d300b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:31.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d35600640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d300ce690 0x7f7d300cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:31.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 -- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d300ce690 msgr2=0x7f7d300cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:31.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d300ce690 0x7f7d300cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 -- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d3015d130 msgr2=0x7f7d300b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:31.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d3015d130 0x7f7d300b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.359+0000 7f7d36000640 1 -- 172.21.15.44:0/3328067433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d300ca320 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d36000640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c6b0 tx=0x7f7d2800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28003b70 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d28003d30 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28005330 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301f3c60 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.360+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301f4200 con 0x7f7d30161be0 2024-07-17T08:41:31.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.361+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d300d0e10 con 0x7f7d30161be0 2024-07-17T08:41:31.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.362+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f7d280054d0 con 0x7f7d30161be0 2024-07-17T08:41:31.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.363+0000 7f7d2d600640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 0x7f7d10079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:31.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.363+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f7d28098700 con 0x7f7d30161be0 2024-07-17T08:41:31.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.364+0000 7f7d35600640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 0x7f7d10079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:31.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.365+0000 7f7d35600640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 0x7f7d10079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005fd0 tx=0x7f7d24005850 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:31.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.368+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7d300ca320 con 0x7f7d30161be0 2024-07-17T08:41:31.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:31.623+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7d300b4ba0 con 0x7f7d30161be0 2024-07-17T08:41:32.159 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-07-17T08:41:32.158+0000 7fc237800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:41:32.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.224+0000 7f7d2d600640 1 -- 172.21.15.44:0/3328067433 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v171) ==== 118+0+0 (secure 0 0 0) 0x7f7d280616e0 con 0x7f7d30161be0 2024-07-17T08:41:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 msgr2=0x7f7d10079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 0x7f7d10079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005fd0 tx=0x7f7d24005850 comp rx=0 tx=0).stop 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 msgr2=0x7f7d300a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c6b0 tx=0x7f7d2800cb70 comp rx=0 tx=0).stop 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 shutdown_connections 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f7d10076cd0 0x7f7d10079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d30161be0 0x7f7d300a9f70 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d3015d130 0x7f7d300b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 --2- 172.21.15.44:0/3328067433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d300ce690 0x7f7d300cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 >> 172.21.15.44:0/3328067433 conn(0x7f7d300b4ff0 msgr2=0x7f7d300d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 shutdown_connections 2024-07-17T08:41:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.228+0000 7f7d36a00640 1 -- 172.21.15.44:0/3328067433 wait complete. 2024-07-17T08:41:32.252 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:41:32.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 -- 172.21.15.44:0/1517135970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 msgr2=0x7f1f800b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 --2- 172.21.15.44:0/1517135970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f800b21b0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f1f80071570 tx=0x7f1f7002a560 comp rx=0 tx=0).stop 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 -- 172.21.15.44:0/1517135970 shutdown_connections 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 --2- 172.21.15.44:0/1517135970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f8006efc0 0x7f1f800ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 --2- 172.21.15.44:0/1517135970 >> v2:172.21.15.175:3300/0 conn(0x7f1f800b1af0 0x7f1f800ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 --2- 172.21.15.44:0/1517135970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f800b21b0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 -- 172.21.15.44:0/1517135970 >> 172.21.15.44:0/1517135970 conn(0x7f1f800b0eb0 msgr2=0x7f1f800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 -- 172.21.15.44:0/1517135970 shutdown_connections 2024-07-17T08:41:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.465+0000 7f1f87000640 1 -- 172.21.15.44:0/1517135970 wait complete. 2024-07-17T08:41:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.467+0000 7f1f87000640 1 Processor -- start 2024-07-17T08:41:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.467+0000 7f1f87000640 1 -- start start 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f8006efc0 0x7f1f8006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f800b1af0 0x7f1f800ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8015ad90 con 0x7f1f8015a280 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8015ac10 con 0x7f1f8006efc0 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f87000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8015af10 con 0x7f1f800b1af0 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f85000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f85000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55182/0 (socket says 172.21.15.44:55182) 2024-07-17T08:41:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f85000640 1 -- 172.21.15.44:0/1819025503 learned_addr learned my addr 172.21.15.44:0/1819025503 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f7f400640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f800b1af0 0x7f1f800ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f8006efc0 0x7f1f8006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.468+0000 7f1f85000640 1 -- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f800b1af0 msgr2=0x7f1f800ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f85000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f800b1af0 0x7f1f800ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f85000640 1 -- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f8006efc0 msgr2=0x7f1f8006e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f85000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f8006efc0 0x7f1f8006e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f85000640 1 -- 172.21.15.44:0/1819025503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f800c86b0 con 0x7f1f8015a280 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f85000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800ca70 tx=0x7f1f7800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800adc0 con 0x7f1f8015a280 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f78004c10 con 0x7f1f8015a280 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78005280 con 0x7f1f8015a280 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f800c8970 con 0x7f1f8015a280 2024-07-17T08:41:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.469+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8006ebb0 con 0x7f1f8015a280 2024-07-17T08:41:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.470+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f800b21b0 con 0x7f1f8015a280 2024-07-17T08:41:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.472+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f1f78005420 con 0x7f1f8015a280 2024-07-17T08:41:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.473+0000 7f1f77e00640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 0x7f1f5c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.473+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f1f780990f0 con 0x7f1f8015a280 2024-07-17T08:41:32.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.473+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 0x7f1f5c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:32.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.475+0000 7f1f7fe00640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 0x7f1f5c0791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f70033000 tx=0x7f1f70004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:32.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.477+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1f78014030 con 0x7f1f8015a280 2024-07-17T08:41:32.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:32.728+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1f800b3340 con 0x7f1f8015a280 2024-07-17T08:41:33.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.228+0000 7f1f77e00640 1 -- 172.21.15.44:0/1819025503 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v172) ==== 122+0+0 (secure 0 0 0) 0x7f1f78061f10 con 0x7f1f8015a280 2024-07-17T08:41:33.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.231+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 msgr2=0x7f1f5c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.231+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 0x7f1f5c0791c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f70033000 tx=0x7f1f70004730 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.231+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 msgr2=0x7f1f801f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.231+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800ca70 tx=0x7f1f7800cf30 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 shutdown_connections 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f1f5c076d00 0x7f1f5c0791c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8015a280 0x7f1f801f73c0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f800b1af0 0x7f1f800ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 --2- 172.21.15.44:0/1819025503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f8006efc0 0x7f1f8006e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 >> 172.21.15.44:0/1819025503 conn(0x7f1f800b0eb0 msgr2=0x7f1f800b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 shutdown_connections 2024-07-17T08:41:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.232+0000 7f1f87000640 1 -- 172.21.15.44:0/1819025503 wait complete. 2024-07-17T08:41:33.253 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:41:33.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.458+0000 7f2bb1600640 1 -- 172.21.15.44:0/3604679208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 msgr2=0x7f2bac0b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.458+0000 7f2bb1600640 1 --2- 172.21.15.44:0/3604679208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac0b56b0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005fd0 tx=0x7f2b9802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.459+0000 7f2bb1600640 1 -- 172.21.15.44:0/3604679208 shutdown_connections 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.459+0000 7f2bb1600640 1 --2- 172.21.15.44:0/3604679208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac0b56b0 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.459+0000 7f2bb1600640 1 -- 172.21.15.44:0/3604679208 >> 172.21.15.44:0/3604679208 conn(0x7f2bac06bf40 msgr2=0x7f2bac06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.459+0000 7f2bb1600640 1 -- 172.21.15.44:0/3604679208 shutdown_connections 2024-07-17T08:41:33.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.459+0000 7f2bb1600640 1 -- 172.21.15.44:0/3604679208 wait complete. 2024-07-17T08:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.460+0000 7f2bb1600640 1 Processor -- start 2024-07-17T08:41:33.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.461+0000 7f2bb1600640 1 -- start start 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 0x7f2bac0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bac158e70 0x7f2bac15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac0d1a80 con 0x7f2bac15f9d0 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac0d1900 con 0x7f2bac158e70 2024-07-17T08:41:33.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bac15c0e0 con 0x7f2bac0d1240 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55196/0 (socket says 172.21.15.44:55196) 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 -- 172.21.15.44:0/2895142778 learned_addr learned my addr 172.21.15.44:0/2895142778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2babe00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 0x7f2bac0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bab400640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bac158e70 0x7f2bac15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 -- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 msgr2=0x7f2bac0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 0x7f2bac0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 -- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bac158e70 msgr2=0x7f2bac15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bac158e70 0x7f2bac15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2bb0c00640 1 -- 172.21.15.44:0/2895142778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac147080 con 0x7f2bac15f9d0 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.462+0000 7f2babe00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 0x7f2bac0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:33.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2bb0c00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c390 tx=0x7f2ba400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:33.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40052d0 con 0x7f2bac15f9d0 2024-07-17T08:41:33.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ba4007020 con 0x7f2bac15f9d0 2024-07-17T08:41:33.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4007470 con 0x7f2bac15f9d0 2024-07-17T08:41:33.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac147360 con 0x7f2bac15f9d0 2024-07-17T08:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.463+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac147920 con 0x7f2bac15f9d0 2024-07-17T08:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.464+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac0d23d0 con 0x7f2bac15f9d0 2024-07-17T08:41:33.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.466+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f2ba40067d0 con 0x7f2bac15f9d0 2024-07-17T08:41:33.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.467+0000 7f2ba3e00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.467+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f2ba4097db0 con 0x7f2bac15f9d0 2024-07-17T08:41:33.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.468+0000 7f2babe00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.469+0000 7f2babe00640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 0x7f2b8c079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c004520 tx=0x7f2b9c009040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:33.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.472+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2ba4060d80 con 0x7f2bac15f9d0 2024-07-17T08:41:33.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.727+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bac0b56b0 con 0x7f2bac15f9d0 2024-07-17T08:41:33.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.728+0000 7f2ba3e00640 1 -- 172.21.15.44:0/2895142778 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7f2ba4060680 con 0x7f2bac15f9d0 2024-07-17T08:41:33.729 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:33.729 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:33.729 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:33.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 msgr2=0x7f2b8c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 0x7f2b8c079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c004520 tx=0x7f2b9c009040 comp rx=0 tx=0).stop 2024-07-17T08:41:33.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 msgr2=0x7f2bac15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c390 tx=0x7f2ba400c850 comp rx=0 tx=0).stop 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 shutdown_connections 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bac15f9d0 0x7f2bac15d8b0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bac158e70 0x7f2bac15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.731+0000 7f2bb1600640 1 --2- 172.21.15.44:0/2895142778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bac0d1240 0x7f2bac0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.732+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 >> 172.21.15.44:0/2895142778 conn(0x7f2bac06bf40 msgr2=0x7f2bac0cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.732+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 shutdown_connections 2024-07-17T08:41:33.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.732+0000 7f2bb1600640 1 -- 172.21.15.44:0/2895142778 wait complete. 2024-07-17T08:41:33.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:33.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.967+0000 7f0711600640 1 -- 172.21.15.44:0/1389684856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00051c0 msgr2=0x7f06f000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.967+0000 7f0711600640 1 --2- 172.21.15.44:0/1389684856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00051c0 0x7f06f000dbd0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f0704005fd0 tx=0x7f06f0005830 comp rx=0 tx=0).stop 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.967+0000 7f0711600640 1 -- 172.21.15.44:0/1389684856 shutdown_connections 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.967+0000 7f0711600640 1 --2- 172.21.15.44:0/1389684856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06f00051c0 0x7f06f000dbd0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.967+0000 7f0711600640 1 -- 172.21.15.44:0/1389684856 >> 172.21.15.44:0/1389684856 conn(0x7f070c0b0fb0 msgr2=0x7f070c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.968+0000 7f0711600640 1 -- 172.21.15.44:0/1389684856 shutdown_connections 2024-07-17T08:41:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.968+0000 7f0711600640 1 -- 172.21.15.44:0/1389684856 wait complete. 2024-07-17T08:41:33.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.969+0000 7f0711600640 1 Processor -- start 2024-07-17T08:41:33.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.970+0000 7f0711600640 1 -- start start 2024-07-17T08:41:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f00051c0 0x7f070c142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f070c1f8150 0x7f070c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f070c06db20 con 0x7f070c146eb0 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f070c06d9a0 con 0x7f070c1f8150 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0711600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f070c06dca0 con 0x7f06f00051c0 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0703e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0703e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55212/0 (socket says 172.21.15.44:55212) 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f0703e00640 1 -- 172.21.15.44:0/167935590 learned_addr learned my addr 172.21.15.44:0/167935590 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f070b400640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f070c1f8150 0x7f070c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.971+0000 7f070aa00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f00051c0 0x7f070c142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 -- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f00051c0 msgr2=0x7f070c142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f00051c0 0x7f070c142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 -- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f070c1f8150 msgr2=0x7f070c1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f070c1f8150 0x7f070c1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 -- 172.21.15.44:0/167935590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1436d0 con 0x7f070c146eb0 2024-07-17T08:41:33.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0703e00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c500 tx=0x7f06f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:33.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f40052d0 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06f4005490 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c1439b0 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.972+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4012870 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.973+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1f7c30 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.974+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0af460 con 0x7f070c146eb0 2024-07-17T08:41:33.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.975+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f06f40035f0 con 0x7f070c146eb0 2024-07-17T08:41:33.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.976+0000 7f0702a00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 0x7f06e80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:33.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.976+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f06f40985c0 con 0x7f070c146eb0 2024-07-17T08:41:33.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.976+0000 7f070aa00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 0x7f06e80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.978+0000 7f070aa00640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 0x7f06e80791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0069d0 tx=0x7f06fc001590 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:33.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:33.980+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f070c0af460 con 0x7f070c146eb0 2024-07-17T08:41:34.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.190+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f070c143b80 con 0x7f070c146eb0 2024-07-17T08:41:34.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.193+0000 7f0702a00640 1 -- 172.21.15.44:0/167935590 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v308) ==== 74+0+17866 (secure 0 0 0) 0x7f06f40615c0 con 0x7f070c146eb0 2024-07-17T08:41:34.194 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:34.194 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":308,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:31.198052+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:41:27.625746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":42,"pool_name":"cephfs_data","create_time":"2024-07-17T08:41:28.667439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"308","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:34.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 msgr2=0x7f06e80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 0x7f06e80791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0069d0 tx=0x7f06fc001590 comp rx=0 tx=0).stop 2024-07-17T08:41:34.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 msgr2=0x7f070c0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c500 tx=0x7f06f400c9c0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 shutdown_connections 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f06e8076d00 0x7f06e80791c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f070c1f8150 0x7f070c1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f070c146eb0 0x7f070c0b3d40 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.195+0000 7f0711600640 1 --2- 172.21.15.44:0/167935590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f00051c0 0x7f070c142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.196+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 >> 172.21.15.44:0/167935590 conn(0x7f070c0b0fb0 msgr2=0x7f070c15b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.196+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 shutdown_connections 2024-07-17T08:41:34.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.196+0000 7f0711600640 1 -- 172.21.15.44:0/167935590 wait complete. 2024-07-17T08:41:34.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:41:34.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:41:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 -- 172.21.15.44:0/2439709380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c0cf380 msgr2=0x7f9a8c0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 --2- 172.21.15.44:0/2439709380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c0cf380 0x7f9a8c0cf810 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005fd0 tx=0x7f9a7802b950 comp rx=0 tx=0).stop 2024-07-17T08:41:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 -- 172.21.15.44:0/2439709380 shutdown_connections 2024-07-17T08:41:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 --2- 172.21.15.44:0/2439709380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c0cf380 0x7f9a8c0cf810 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 --2- 172.21.15.44:0/2439709380 >> v2:172.21.15.175:3300/0 conn(0x7f9a8c06f4b0 0x7f9a8c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 -- 172.21.15.44:0/2439709380 >> 172.21.15.44:0/2439709380 conn(0x7f9a8c0b0e80 msgr2=0x7f9a8c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:34.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 -- 172.21.15.44:0/2439709380 shutdown_connections 2024-07-17T08:41:34.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.409+0000 7f9a91e00640 1 -- 172.21.15.44:0/2439709380 wait complete. 2024-07-17T08:41:34.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.411+0000 7f9a91e00640 1 Processor -- start 2024-07-17T08:41:34.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.411+0000 7f9a91e00640 1 -- start start 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a8c0cf380 0x7f9a8c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 0x7f9a8c0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0b2120 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0b1fa0 con 0x7f9a8c159800 2024-07-17T08:41:34.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a91e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c0b22a0 con 0x7f9a8c0cf380 2024-07-17T08:41:34.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a90a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a90a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55232/0 (socket says 172.21.15.44:55232) 2024-07-17T08:41:34.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.412+0000 7f9a90a00640 1 -- 172.21.15.44:0/3968351146 learned_addr learned my addr 172.21.15.44:0/3968351146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:34.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 -- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a8c0cf380 msgr2=0x7f9a8c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a91400640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 0x7f9a8c0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a8c0cf380 0x7f9a8c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 -- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 msgr2=0x7f9a8c0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 0x7f9a8c0b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 -- 172.21.15.44:0/3968351146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c1f73a0 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a91400640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 0x7f9a8c0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a90a00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000d980 tx=0x7f9a8000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80017070 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a80009d20 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800106c0 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.413+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c1f7680 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.414+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1f7c40 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.415+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c0cecb0 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.416+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f9a80010860 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.417+0000 7f9a88c00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 0x7f9a6c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.417+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f9a80098d60 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.417+0000 7f9a8be00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 0x7f9a6c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.419+0000 7f9a8be00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 0x7f9a6c079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a78035000 tx=0x7f9a780046c0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:34.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.421+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9a8c0cecb0 con 0x7f9a8c06f4b0 2024-07-17T08:41:34.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.617+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 --> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a8c06e060 con 0x7f9a6c076cd0 2024-07-17T08:41:34.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.632+0000 7f9a88c00640 1 -- 172.21.15.44:0/3968351146 <== mgr.9029 v2:172.21.15.175:6834/3743840884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f9a8c06e060 con 0x7f9a6c076cd0 2024-07-17T08:41:34.636 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:34.638 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:41:34.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 msgr2=0x7f9a6c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 0x7f9a6c079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a78035000 tx=0x7f9a780046c0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 msgr2=0x7f9a8c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000d980 tx=0x7f9a8000de40 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 shutdown_connections 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9a6c076cd0 0x7f9a6c079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a8c159800 0x7f9a8c0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a8c0cf380 0x7f9a8c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 --2- 172.21.15.44:0/3968351146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a8c06f4b0 0x7f9a8c149bc0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 >> 172.21.15.44:0/3968351146 conn(0x7f9a8c0b0e80 msgr2=0x7f9a8c0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 shutdown_connections 2024-07-17T08:41:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.637+0000 7f9a91e00640 1 -- 172.21.15.44:0/3968351146 wait complete. 2024-07-17T08:41:34.656 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:41:33.949938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":514,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4649,"ondisk_log_size":4649,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":474196,"kb_used_data":31772,"kb_used_omap":12,"kb_used_meta":442291,"kb_avail":754500524,"statfs":{"total":773094113280,"available":772608536576,"internally_reserved":0,"allocated":32534528,"data_stored":18405196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452906544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.788172"},"pg_stats":[{"pgid":"42.3f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545897+0000","last_change":"2024-07-17T08:41:30.218044+0000","last_active":"2024-07-17T08:41:31.545897+0000","last_peered":"2024-07-17T08:41:31.545897+0000","last_clean":"2024-07-17T08:41:31.545897+0000","last_became_active":"2024-07-17T08:41:30.217230+0000","last_became_peered":"2024-07-17T08:41:30.217230+0000","last_unstale":"2024-07-17T08:41:31.545897+0000","last_undegraded":"2024-07-17T08:41:31.545897+0000","last_fullsized":"2024-07-17T08:41:31.545897+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:17:04.952677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546345+0000","last_change":"2024-07-17T08:41:30.253962+0000","last_active":"2024-07-17T08:41:31.546345+0000","last_peered":"2024-07-17T08:41:31.546345+0000","last_clean":"2024-07-17T08:41:31.546345+0000","last_became_active":"2024-07-17T08:41:30.252198+0000","last_became_peered":"2024-07-17T08:41:30.252198+0000","last_unstale":"2024-07-17T08:41:31.546345+0000","last_undegraded":"2024-07-17T08:41:31.546345+0000","last_fullsized":"2024-07-17T08:41:31.546345+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:41.865153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.3d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548065+0000","last_change":"2024-07-17T08:41:30.242648+0000","last_active":"2024-07-17T08:41:31.548065+0000","last_peered":"2024-07-17T08:41:31.548065+0000","last_clean":"2024-07-17T08:41:31.548065+0000","last_became_active":"2024-07-17T08:41:30.242327+0000","last_became_peered":"2024-07-17T08:41:30.242327+0000","last_unstale":"2024-07-17T08:41:31.548065+0000","last_undegraded":"2024-07-17T08:41:31.548065+0000","last_fullsized":"2024-07-17T08:41:31.548065+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:26.342934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547194+0000","last_change":"2024-07-17T08:41:30.240939+0000","last_active":"2024-07-17T08:41:31.547194+0000","last_peered":"2024-07-17T08:41:31.547194+0000","last_clean":"2024-07-17T08:41:31.547194+0000","last_became_active":"2024-07-17T08:41:30.240665+0000","last_became_peered":"2024-07-17T08:41:30.240665+0000","last_unstale":"2024-07-17T08:41:31.547194+0000","last_undegraded":"2024-07-17T08:41:31.547194+0000","last_fullsized":"2024-07-17T08:41:31.547194+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:56.109775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218771+0000","last_change":"2024-07-17T08:41:30.237397+0000","last_active":"2024-07-17T08:41:31.218771+0000","last_peered":"2024-07-17T08:41:31.218771+0000","last_clean":"2024-07-17T08:41:31.218771+0000","last_became_active":"2024-07-17T08:41:30.236120+0000","last_became_peered":"2024-07-17T08:41:30.236120+0000","last_unstale":"2024-07-17T08:41:31.218771+0000","last_undegraded":"2024-07-17T08:41:31.218771+0000","last_fullsized":"2024-07-17T08:41:31.218771+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:55:27.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.3a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545907+0000","last_change":"2024-07-17T08:41:30.249579+0000","last_active":"2024-07-17T08:41:31.545907+0000","last_peered":"2024-07-17T08:41:31.545907+0000","last_clean":"2024-07-17T08:41:31.545907+0000","last_became_active":"2024-07-17T08:41:30.246124+0000","last_became_peered":"2024-07-17T08:41:30.246124+0000","last_unstale":"2024-07-17T08:41:31.545907+0000","last_undegraded":"2024-07-17T08:41:31.545907+0000","last_fullsized":"2024-07-17T08:41:31.545907+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:33.474235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.39","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217520+0000","last_change":"2024-07-17T08:41:30.238672+0000","last_active":"2024-07-17T08:41:31.217520+0000","last_peered":"2024-07-17T08:41:31.217520+0000","last_clean":"2024-07-17T08:41:31.217520+0000","last_became_active":"2024-07-17T08:41:30.238372+0000","last_became_peered":"2024-07-17T08:41:30.238372+0000","last_unstale":"2024-07-17T08:41:31.217520+0000","last_undegraded":"2024-07-17T08:41:31.217520+0000","last_fullsized":"2024-07-17T08:41:31.217520+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.485715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.38","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545822+0000","last_change":"2024-07-17T08:41:30.248116+0000","last_active":"2024-07-17T08:41:31.545822+0000","last_peered":"2024-07-17T08:41:31.545822+0000","last_clean":"2024-07-17T08:41:31.545822+0000","last_became_active":"2024-07-17T08:41:30.246503+0000","last_became_peered":"2024-07-17T08:41:30.246503+0000","last_unstale":"2024-07-17T08:41:31.545822+0000","last_undegraded":"2024-07-17T08:41:31.545822+0000","last_fullsized":"2024-07-17T08:41:31.545822+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:17.471148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.37","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546896+0000","last_change":"2024-07-17T08:41:30.241429+0000","last_active":"2024-07-17T08:41:31.546896+0000","last_peered":"2024-07-17T08:41:31.546896+0000","last_clean":"2024-07-17T08:41:31.546896+0000","last_became_active":"2024-07-17T08:41:30.241103+0000","last_became_peered":"2024-07-17T08:41:30.241103+0000","last_unstale":"2024-07-17T08:41:31.546896+0000","last_undegraded":"2024-07-17T08:41:31.546896+0000","last_fullsized":"2024-07-17T08:41:31.546896+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:21.575456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.36","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217591+0000","last_change":"2024-07-17T08:41:30.237779+0000","last_active":"2024-07-17T08:41:31.217591+0000","last_peered":"2024-07-17T08:41:31.217591+0000","last_clean":"2024-07-17T08:41:31.217591+0000","last_became_active":"2024-07-17T08:41:30.237382+0000","last_became_peered":"2024-07-17T08:41:30.237382+0000","last_unstale":"2024-07-17T08:41:31.217591+0000","last_undegraded":"2024-07-17T08:41:31.217591+0000","last_fullsized":"2024-07-17T08:41:31.217591+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:08.481577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.35","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545795+0000","last_change":"2024-07-17T08:41:30.251748+0000","last_active":"2024-07-17T08:41:31.545795+0000","last_peered":"2024-07-17T08:41:31.545795+0000","last_clean":"2024-07-17T08:41:31.545795+0000","last_became_active":"2024-07-17T08:41:30.251444+0000","last_became_peered":"2024-07-17T08:41:30.251444+0000","last_unstale":"2024-07-17T08:41:31.545795+0000","last_undegraded":"2024-07-17T08:41:31.545795+0000","last_fullsized":"2024-07-17T08:41:31.545795+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:38.106633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.34","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548219+0000","last_change":"2024-07-17T08:41:30.238575+0000","last_active":"2024-07-17T08:41:31.548219+0000","last_peered":"2024-07-17T08:41:31.548219+0000","last_clean":"2024-07-17T08:41:31.548219+0000","last_became_active":"2024-07-17T08:41:30.234809+0000","last_became_peered":"2024-07-17T08:41:30.234809+0000","last_unstale":"2024-07-17T08:41:31.548219+0000","last_undegraded":"2024-07-17T08:41:31.548219+0000","last_fullsized":"2024-07-17T08:41:31.548219+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:45.951593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.33","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256608+0000","last_change":"2024-07-17T08:41:30.237984+0000","last_active":"2024-07-17T08:41:31.256608+0000","last_peered":"2024-07-17T08:41:31.256608+0000","last_clean":"2024-07-17T08:41:31.256608+0000","last_became_active":"2024-07-17T08:41:30.237648+0000","last_became_peered":"2024-07-17T08:41:30.237648+0000","last_unstale":"2024-07-17T08:41:31.256608+0000","last_undegraded":"2024-07-17T08:41:31.256608+0000","last_fullsized":"2024-07-17T08:41:31.256608+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:29:48.974758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.32","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546357+0000","last_change":"2024-07-17T08:41:30.241210+0000","last_active":"2024-07-17T08:41:31.546357+0000","last_peered":"2024-07-17T08:41:31.546357+0000","last_clean":"2024-07-17T08:41:31.546357+0000","last_became_active":"2024-07-17T08:41:30.240916+0000","last_became_peered":"2024-07-17T08:41:30.240916+0000","last_unstale":"2024-07-17T08:41:31.546357+0000","last_undegraded":"2024-07-17T08:41:31.546357+0000","last_fullsized":"2024-07-17T08:41:31.546357+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:37.537804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.31","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545300+0000","last_change":"2024-07-17T08:41:30.249610+0000","last_active":"2024-07-17T08:41:31.545300+0000","last_peered":"2024-07-17T08:41:31.545300+0000","last_clean":"2024-07-17T08:41:31.545300+0000","last_became_active":"2024-07-17T08:41:30.249320+0000","last_became_peered":"2024-07-17T08:41:30.249320+0000","last_unstale":"2024-07-17T08:41:31.545300+0000","last_undegraded":"2024-07-17T08:41:31.545300+0000","last_fullsized":"2024-07-17T08:41:31.545300+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:08.481577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.30","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546372+0000","last_change":"2024-07-17T08:41:30.247253+0000","last_active":"2024-07-17T08:41:31.546372+0000","last_peered":"2024-07-17T08:41:31.546372+0000","last_clean":"2024-07-17T08:41:31.546372+0000","last_became_active":"2024-07-17T08:41:30.246986+0000","last_became_peered":"2024-07-17T08:41:30.246986+0000","last_unstale":"2024-07-17T08:41:31.546372+0000","last_undegraded":"2024-07-17T08:41:31.546372+0000","last_fullsized":"2024-07-17T08:41:31.546372+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:16.253173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.2f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212873+0000","last_change":"2024-07-17T08:41:30.232574+0000","last_active":"2024-07-17T08:41:31.212873+0000","last_peered":"2024-07-17T08:41:31.212873+0000","last_clean":"2024-07-17T08:41:31.212873+0000","last_became_active":"2024-07-17T08:41:30.231793+0000","last_became_peered":"2024-07-17T08:41:30.231793+0000","last_unstale":"2024-07-17T08:41:31.212873+0000","last_undegraded":"2024-07-17T08:41:31.212873+0000","last_fullsized":"2024-07-17T08:41:31.212873+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:16.973257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.2e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217522+0000","last_change":"2024-07-17T08:41:30.245658+0000","last_active":"2024-07-17T08:41:31.217522+0000","last_peered":"2024-07-17T08:41:31.217522+0000","last_clean":"2024-07-17T08:41:31.217522+0000","last_became_active":"2024-07-17T08:41:30.245350+0000","last_became_peered":"2024-07-17T08:41:30.245350+0000","last_unstale":"2024-07-17T08:41:31.217522+0000","last_undegraded":"2024-07-17T08:41:31.217522+0000","last_fullsized":"2024-07-17T08:41:31.217522+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.587744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547621+0000","last_change":"2024-07-17T08:41:30.250071+0000","last_active":"2024-07-17T08:41:31.547621+0000","last_peered":"2024-07-17T08:41:31.547621+0000","last_clean":"2024-07-17T08:41:31.547621+0000","last_became_active":"2024-07-17T08:41:30.249731+0000","last_became_peered":"2024-07-17T08:41:30.249731+0000","last_unstale":"2024-07-17T08:41:31.547621+0000","last_undegraded":"2024-07-17T08:41:31.547621+0000","last_fullsized":"2024-07-17T08:41:31.547621+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:04.911004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.2c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217305+0000","last_change":"2024-07-17T08:41:30.240025+0000","last_active":"2024-07-17T08:41:31.217305+0000","last_peered":"2024-07-17T08:41:31.217305+0000","last_clean":"2024-07-17T08:41:31.217305+0000","last_became_active":"2024-07-17T08:41:30.239462+0000","last_became_peered":"2024-07-17T08:41:30.239462+0000","last_unstale":"2024-07-17T08:41:31.217305+0000","last_undegraded":"2024-07-17T08:41:31.217305+0000","last_fullsized":"2024-07-17T08:41:31.217305+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:26:32.642649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547043+0000","last_change":"2024-07-17T08:41:30.239426+0000","last_active":"2024-07-17T08:41:31.547043+0000","last_peered":"2024-07-17T08:41:31.547043+0000","last_clean":"2024-07-17T08:41:31.547043+0000","last_became_active":"2024-07-17T08:41:30.239096+0000","last_became_peered":"2024-07-17T08:41:30.239096+0000","last_unstale":"2024-07-17T08:41:31.547043+0000","last_undegraded":"2024-07-17T08:41:31.547043+0000","last_fullsized":"2024-07-17T08:41:31.547043+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:05.463382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.29","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545267+0000","last_change":"2024-07-17T08:41:30.229883+0000","last_active":"2024-07-17T08:41:31.545267+0000","last_peered":"2024-07-17T08:41:31.545267+0000","last_clean":"2024-07-17T08:41:31.545267+0000","last_became_active":"2024-07-17T08:41:30.229500+0000","last_became_peered":"2024-07-17T08:41:30.229500+0000","last_unstale":"2024-07-17T08:41:31.545267+0000","last_undegraded":"2024-07-17T08:41:31.545267+0000","last_fullsized":"2024-07-17T08:41:31.545267+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:54.560584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.28","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217404+0000","last_change":"2024-07-17T08:41:30.239944+0000","last_active":"2024-07-17T08:41:31.217404+0000","last_peered":"2024-07-17T08:41:31.217404+0000","last_clean":"2024-07-17T08:41:31.217404+0000","last_became_active":"2024-07-17T08:41:30.238820+0000","last_became_peered":"2024-07-17T08:41:30.238820+0000","last_unstale":"2024-07-17T08:41:31.217404+0000","last_undegraded":"2024-07-17T08:41:31.217404+0000","last_fullsized":"2024-07-17T08:41:31.217404+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:18.363390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.27","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546469+0000","last_change":"2024-07-17T08:41:30.241126+0000","last_active":"2024-07-17T08:41:31.546469+0000","last_peered":"2024-07-17T08:41:31.546469+0000","last_clean":"2024-07-17T08:41:31.546469+0000","last_became_active":"2024-07-17T08:41:30.240798+0000","last_became_peered":"2024-07-17T08:41:30.240798+0000","last_unstale":"2024-07-17T08:41:31.546469+0000","last_undegraded":"2024-07-17T08:41:31.546469+0000","last_fullsized":"2024-07-17T08:41:31.546469+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:17.852383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.26","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219417+0000","last_change":"2024-07-17T08:41:30.253177+0000","last_active":"2024-07-17T08:41:31.219417+0000","last_peered":"2024-07-17T08:41:31.219417+0000","last_clean":"2024-07-17T08:41:31.219417+0000","last_became_active":"2024-07-17T08:41:30.252744+0000","last_became_peered":"2024-07-17T08:41:30.252744+0000","last_unstale":"2024-07-17T08:41:31.219417+0000","last_undegraded":"2024-07-17T08:41:31.219417+0000","last_fullsized":"2024-07-17T08:41:31.219417+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:23.752543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.25","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256351+0000","last_change":"2024-07-17T08:41:30.240173+0000","last_active":"2024-07-17T08:41:31.256351+0000","last_peered":"2024-07-17T08:41:31.256351+0000","last_clean":"2024-07-17T08:41:31.256351+0000","last_became_active":"2024-07-17T08:41:30.236929+0000","last_became_peered":"2024-07-17T08:41:30.236929+0000","last_unstale":"2024-07-17T08:41:31.256351+0000","last_undegraded":"2024-07-17T08:41:31.256351+0000","last_fullsized":"2024-07-17T08:41:31.256351+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:41.257244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.24","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217334+0000","last_change":"2024-07-17T08:41:30.239790+0000","last_active":"2024-07-17T08:41:31.217334+0000","last_peered":"2024-07-17T08:41:31.217334+0000","last_clean":"2024-07-17T08:41:31.217334+0000","last_became_active":"2024-07-17T08:41:30.238970+0000","last_became_peered":"2024-07-17T08:41:30.238970+0000","last_unstale":"2024-07-17T08:41:31.217334+0000","last_undegraded":"2024-07-17T08:41:31.217334+0000","last_fullsized":"2024-07-17T08:41:31.217334+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:30.966488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.23","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546855+0000","last_change":"2024-07-17T08:41:30.253077+0000","last_active":"2024-07-17T08:41:31.546855+0000","last_peered":"2024-07-17T08:41:31.546855+0000","last_clean":"2024-07-17T08:41:31.546855+0000","last_became_active":"2024-07-17T08:41:30.252566+0000","last_became_peered":"2024-07-17T08:41:30.252566+0000","last_unstale":"2024-07-17T08:41:31.546855+0000","last_undegraded":"2024-07-17T08:41:31.546855+0000","last_fullsized":"2024-07-17T08:41:31.546855+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:33.026942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.22","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545347+0000","last_change":"2024-07-17T08:41:30.220777+0000","last_active":"2024-07-17T08:41:31.545347+0000","last_peered":"2024-07-17T08:41:31.545347+0000","last_clean":"2024-07-17T08:41:31.545347+0000","last_became_active":"2024-07-17T08:41:30.219962+0000","last_became_peered":"2024-07-17T08:41:30.219962+0000","last_unstale":"2024-07-17T08:41:31.545347+0000","last_undegraded":"2024-07-17T08:41:31.545347+0000","last_fullsized":"2024-07-17T08:41:31.545347+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:57.752364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.21","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212200+0000","last_change":"2024-07-17T08:41:30.237902+0000","last_active":"2024-07-17T08:41:31.212200+0000","last_peered":"2024-07-17T08:41:31.212200+0000","last_clean":"2024-07-17T08:41:31.212200+0000","last_became_active":"2024-07-17T08:41:30.237535+0000","last_became_peered":"2024-07-17T08:41:30.237535+0000","last_unstale":"2024-07-17T08:41:31.212200+0000","last_undegraded":"2024-07-17T08:41:31.212200+0000","last_fullsized":"2024-07-17T08:41:31.212200+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:16.197454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.20","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217595+0000","last_change":"2024-07-17T08:41:30.240551+0000","last_active":"2024-07-17T08:41:31.217595+0000","last_peered":"2024-07-17T08:41:31.217595+0000","last_clean":"2024-07-17T08:41:31.217595+0000","last_became_active":"2024-07-17T08:41:30.240274+0000","last_became_peered":"2024-07-17T08:41:30.240274+0000","last_unstale":"2024-07-17T08:41:31.217595+0000","last_undegraded":"2024-07-17T08:41:31.217595+0000","last_fullsized":"2024-07-17T08:41:31.217595+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:52.281898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217597+0000","last_change":"2024-07-17T08:41:30.239383+0000","last_active":"2024-07-17T08:41:31.217597+0000","last_peered":"2024-07-17T08:41:31.217597+0000","last_clean":"2024-07-17T08:41:31.217597+0000","last_became_active":"2024-07-17T08:41:30.238602+0000","last_became_peered":"2024-07-17T08:41:30.238602+0000","last_unstale":"2024-07-17T08:41:31.217597+0000","last_undegraded":"2024-07-17T08:41:31.217597+0000","last_fullsized":"2024-07-17T08:41:31.217597+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:38.106633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1f","version":"307'4","reported_seq":18,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217277+0000","last_change":"2024-07-17T08:41:29.207594+0000","last_active":"2024-07-17T08:41:31.217277+0000","last_peered":"2024-07-17T08:41:31.217277+0000","last_clean":"2024-07-17T08:41:31.217277+0000","last_became_active":"2024-07-17T08:41:29.206945+0000","last_became_peered":"2024-07-17T08:41:29.206945+0000","last_unstale":"2024-07-17T08:41:31.217277+0000","last_undegraded":"2024-07-17T08:41:31.217277+0000","last_fullsized":"2024-07-17T08:41:31.217277+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:08.458737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.1d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212773+0000","last_change":"2024-07-17T08:41:30.234126+0000","last_active":"2024-07-17T08:41:31.212773+0000","last_peered":"2024-07-17T08:41:31.212773+0000","last_clean":"2024-07-17T08:41:31.212773+0000","last_became_active":"2024-07-17T08:41:30.233774+0000","last_became_peered":"2024-07-17T08:41:30.233774+0000","last_unstale":"2024-07-17T08:41:31.212773+0000","last_undegraded":"2024-07-17T08:41:31.212773+0000","last_fullsized":"2024-07-17T08:41:31.212773+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:28.414468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.1e","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545518+0000","last_change":"2024-07-17T08:41:29.185379+0000","last_active":"2024-07-17T08:41:31.545518+0000","last_peered":"2024-07-17T08:41:31.545518+0000","last_clean":"2024-07-17T08:41:31.545518+0000","last_became_active":"2024-07-17T08:41:29.184582+0000","last_became_peered":"2024-07-17T08:41:29.184582+0000","last_unstale":"2024-07-17T08:41:31.545518+0000","last_undegraded":"2024-07-17T08:41:31.545518+0000","last_fullsized":"2024-07-17T08:41:31.545518+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:09.843310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545509+0000","last_change":"2024-07-17T08:41:30.219523+0000","last_active":"2024-07-17T08:41:31.545509+0000","last_peered":"2024-07-17T08:41:31.545509+0000","last_clean":"2024-07-17T08:41:31.545509+0000","last_became_active":"2024-07-17T08:41:30.218230+0000","last_became_peered":"2024-07-17T08:41:30.218230+0000","last_unstale":"2024-07-17T08:41:31.545509+0000","last_undegraded":"2024-07-17T08:41:31.545509+0000","last_fullsized":"2024-07-17T08:41:31.545509+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:10.714530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1d","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546299+0000","last_change":"2024-07-17T08:41:29.212291+0000","last_active":"2024-07-17T08:41:31.546299+0000","last_peered":"2024-07-17T08:41:31.546299+0000","last_clean":"2024-07-17T08:41:31.546299+0000","last_became_active":"2024-07-17T08:41:29.210999+0000","last_became_peered":"2024-07-17T08:41:29.210999+0000","last_unstale":"2024-07-17T08:41:31.546299+0000","last_undegraded":"2024-07-17T08:41:31.546299+0000","last_fullsized":"2024-07-17T08:41:31.546299+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:08.458737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.1f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547618+0000","last_change":"2024-07-17T08:41:30.251821+0000","last_active":"2024-07-17T08:41:31.547618+0000","last_peered":"2024-07-17T08:41:31.547618+0000","last_clean":"2024-07-17T08:41:31.547618+0000","last_became_active":"2024-07-17T08:41:30.251285+0000","last_became_peered":"2024-07-17T08:41:30.251285+0000","last_unstale":"2024-07-17T08:41:31.547618+0000","last_undegraded":"2024-07-17T08:41:31.547618+0000","last_fullsized":"2024-07-17T08:41:31.547618+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:18.461569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1c","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256219+0000","last_change":"2024-07-17T08:41:29.205750+0000","last_active":"2024-07-17T08:41:31.256219+0000","last_peered":"2024-07-17T08:41:31.256219+0000","last_clean":"2024-07-17T08:41:31.256219+0000","last_became_active":"2024-07-17T08:41:29.205451+0000","last_became_peered":"2024-07-17T08:41:29.205451+0000","last_unstale":"2024-07-17T08:41:31.256219+0000","last_undegraded":"2024-07-17T08:41:31.256219+0000","last_fullsized":"2024-07-17T08:41:31.256219+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:52:54.135306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.18","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217230+0000","last_change":"2024-07-17T08:41:30.247721+0000","last_active":"2024-07-17T08:41:31.217230+0000","last_peered":"2024-07-17T08:41:31.217230+0000","last_clean":"2024-07-17T08:41:31.217230+0000","last_became_active":"2024-07-17T08:41:30.247417+0000","last_became_peered":"2024-07-17T08:41:30.247417+0000","last_unstale":"2024-07-17T08:41:31.217230+0000","last_undegraded":"2024-07-17T08:41:31.217230+0000","last_fullsized":"2024-07-17T08:41:31.217230+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:00:02.046438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1b","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547132+0000","last_change":"2024-07-17T08:41:29.202319+0000","last_active":"2024-07-17T08:41:31.547132+0000","last_peered":"2024-07-17T08:41:31.547132+0000","last_clean":"2024-07-17T08:41:31.547132+0000","last_became_active":"2024-07-17T08:41:29.202026+0000","last_became_peered":"2024-07-17T08:41:29.202026+0000","last_unstale":"2024-07-17T08:41:31.547132+0000","last_undegraded":"2024-07-17T08:41:31.547132+0000","last_fullsized":"2024-07-17T08:41:31.547132+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:33.592690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.19","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545025+0000","last_change":"2024-07-17T08:41:30.241976+0000","last_active":"2024-07-17T08:41:31.545025+0000","last_peered":"2024-07-17T08:41:31.545025+0000","last_clean":"2024-07-17T08:41:31.545025+0000","last_became_active":"2024-07-17T08:41:30.241566+0000","last_became_peered":"2024-07-17T08:41:30.241566+0000","last_unstale":"2024-07-17T08:41:31.545025+0000","last_undegraded":"2024-07-17T08:41:31.545025+0000","last_fullsized":"2024-07-17T08:41:31.545025+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.485715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1a","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.255828+0000","last_change":"2024-07-17T08:41:29.202871+0000","last_active":"2024-07-17T08:41:31.255828+0000","last_peered":"2024-07-17T08:41:31.255828+0000","last_clean":"2024-07-17T08:41:31.255828+0000","last_became_active":"2024-07-17T08:41:29.201742+0000","last_became_peered":"2024-07-17T08:41:29.201742+0000","last_unstale":"2024-07-17T08:41:31.255828+0000","last_undegraded":"2024-07-17T08:41:31.255828+0000","last_fullsized":"2024-07-17T08:41:31.255828+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:59.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.1a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546025+0000","last_change":"2024-07-17T08:41:30.220746+0000","last_active":"2024-07-17T08:41:31.546025+0000","last_peered":"2024-07-17T08:41:31.546025+0000","last_clean":"2024-07-17T08:41:31.546025+0000","last_became_active":"2024-07-17T08:41:30.218128+0000","last_became_peered":"2024-07-17T08:41:30.218128+0000","last_unstale":"2024-07-17T08:41:31.546025+0000","last_undegraded":"2024-07-17T08:41:31.546025+0000","last_fullsized":"2024-07-17T08:41:31.546025+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:22:03.437246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.19","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546339+0000","last_change":"2024-07-17T08:41:29.193729+0000","last_active":"2024-07-17T08:41:31.546339+0000","last_peered":"2024-07-17T08:41:31.546339+0000","last_clean":"2024-07-17T08:41:31.546339+0000","last_became_active":"2024-07-17T08:41:29.193439+0000","last_became_peered":"2024-07-17T08:41:29.193439+0000","last_unstale":"2024-07-17T08:41:31.546339+0000","last_undegraded":"2024-07-17T08:41:31.546339+0000","last_fullsized":"2024-07-17T08:41:31.546339+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:14.237905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.1b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545128+0000","last_change":"2024-07-17T08:41:30.216823+0000","last_active":"2024-07-17T08:41:31.545128+0000","last_peered":"2024-07-17T08:41:31.545128+0000","last_clean":"2024-07-17T08:41:31.545128+0000","last_became_active":"2024-07-17T08:41:30.215858+0000","last_became_peered":"2024-07-17T08:41:30.215858+0000","last_unstale":"2024-07-17T08:41:31.545128+0000","last_undegraded":"2024-07-17T08:41:31.545128+0000","last_fullsized":"2024-07-17T08:41:31.545128+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:28.543985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.18","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546628+0000","last_change":"2024-07-17T08:41:29.208196+0000","last_active":"2024-07-17T08:41:31.546628+0000","last_peered":"2024-07-17T08:41:31.546628+0000","last_clean":"2024-07-17T08:41:31.546628+0000","last_became_active":"2024-07-17T08:41:29.207232+0000","last_became_peered":"2024-07-17T08:41:29.207232+0000","last_unstale":"2024-07-17T08:41:31.546628+0000","last_undegraded":"2024-07-17T08:41:31.546628+0000","last_fullsized":"2024-07-17T08:41:31.546628+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:44.922862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.14","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548364+0000","last_change":"2024-07-17T08:41:30.236896+0000","last_active":"2024-07-17T08:41:31.548364+0000","last_peered":"2024-07-17T08:41:31.548364+0000","last_clean":"2024-07-17T08:41:31.548364+0000","last_became_active":"2024-07-17T08:41:30.234420+0000","last_became_peered":"2024-07-17T08:41:30.234420+0000","last_unstale":"2024-07-17T08:41:31.548364+0000","last_undegraded":"2024-07-17T08:41:31.548364+0000","last_fullsized":"2024-07-17T08:41:31.548364+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:16.236674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.17","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256633+0000","last_change":"2024-07-17T08:41:29.202037+0000","last_active":"2024-07-17T08:41:31.256633+0000","last_peered":"2024-07-17T08:41:31.256633+0000","last_clean":"2024-07-17T08:41:31.256633+0000","last_became_active":"2024-07-17T08:41:29.201706+0000","last_became_peered":"2024-07-17T08:41:29.201706+0000","last_unstale":"2024-07-17T08:41:31.256633+0000","last_undegraded":"2024-07-17T08:41:31.256633+0000","last_fullsized":"2024-07-17T08:41:31.256633+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:13.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.15","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219170+0000","last_change":"2024-07-17T08:41:30.237164+0000","last_active":"2024-07-17T08:41:31.219170+0000","last_peered":"2024-07-17T08:41:31.219170+0000","last_clean":"2024-07-17T08:41:31.219170+0000","last_became_active":"2024-07-17T08:41:30.236862+0000","last_became_peered":"2024-07-17T08:41:30.236862+0000","last_unstale":"2024-07-17T08:41:31.219170+0000","last_undegraded":"2024-07-17T08:41:31.219170+0000","last_fullsized":"2024-07-17T08:41:31.219170+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:15.629782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.16","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256020+0000","last_change":"2024-07-17T08:41:29.193104+0000","last_active":"2024-07-17T08:41:31.256020+0000","last_peered":"2024-07-17T08:41:31.256020+0000","last_clean":"2024-07-17T08:41:31.256020+0000","last_became_active":"2024-07-17T08:41:29.192603+0000","last_became_peered":"2024-07-17T08:41:29.192603+0000","last_unstale":"2024-07-17T08:41:31.256020+0000","last_undegraded":"2024-07-17T08:41:31.256020+0000","last_fullsized":"2024-07-17T08:41:31.256020+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:39.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.16","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217795+0000","last_change":"2024-07-17T08:41:30.244392+0000","last_active":"2024-07-17T08:41:31.217795+0000","last_peered":"2024-07-17T08:41:31.217795+0000","last_clean":"2024-07-17T08:41:31.217795+0000","last_became_active":"2024-07-17T08:41:30.243216+0000","last_became_peered":"2024-07-17T08:41:30.243216+0000","last_unstale":"2024-07-17T08:41:31.217795+0000","last_undegraded":"2024-07-17T08:41:31.217795+0000","last_fullsized":"2024-07-17T08:41:31.217795+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:29.052969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.15","version":"307'3","reported_seq":17,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547522+0000","last_change":"2024-07-17T08:41:29.205092+0000","last_active":"2024-07-17T08:41:31.547522+0000","last_peered":"2024-07-17T08:41:31.547522+0000","last_clean":"2024-07-17T08:41:31.547522+0000","last_became_active":"2024-07-17T08:41:29.204782+0000","last_became_peered":"2024-07-17T08:41:29.204782+0000","last_unstale":"2024-07-17T08:41:31.547522+0000","last_undegraded":"2024-07-17T08:41:31.547522+0000","last_fullsized":"2024-07-17T08:41:31.547522+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:06:52.646670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.17","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547386+0000","last_change":"2024-07-17T08:41:30.251580+0000","last_active":"2024-07-17T08:41:31.547386+0000","last_peered":"2024-07-17T08:41:31.547386+0000","last_clean":"2024-07-17T08:41:31.547386+0000","last_became_active":"2024-07-17T08:41:30.250887+0000","last_became_peered":"2024-07-17T08:41:30.250887+0000","last_unstale":"2024-07-17T08:41:31.547386+0000","last_undegraded":"2024-07-17T08:41:31.547386+0000","last_fullsized":"2024-07-17T08:41:31.547386+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:03.578868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.14","version":"307'6","reported_seq":20,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545464+0000","last_change":"2024-07-17T08:41:29.209823+0000","last_active":"2024-07-17T08:41:31.545464+0000","last_peered":"2024-07-17T08:41:31.545464+0000","last_clean":"2024-07-17T08:41:31.545464+0000","last_became_active":"2024-07-17T08:41:29.209523+0000","last_became_peered":"2024-07-17T08:41:29.209523+0000","last_unstale":"2024-07-17T08:41:31.545464+0000","last_undegraded":"2024-07-17T08:41:31.545464+0000","last_fullsized":"2024-07-17T08:41:31.545464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:00.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.10","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.544735+0000","last_change":"2024-07-17T08:41:30.221069+0000","last_active":"2024-07-17T08:41:31.544735+0000","last_peered":"2024-07-17T08:41:31.544735+0000","last_clean":"2024-07-17T08:41:31.544735+0000","last_became_active":"2024-07-17T08:41:30.220224+0000","last_became_peered":"2024-07-17T08:41:30.220224+0000","last_unstale":"2024-07-17T08:41:31.544735+0000","last_undegraded":"2024-07-17T08:41:31.544735+0000","last_fullsized":"2024-07-17T08:41:31.544735+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:07:41.955008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.13","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547415+0000","last_change":"2024-07-17T08:41:29.207619+0000","last_active":"2024-07-17T08:41:31.547415+0000","last_peered":"2024-07-17T08:41:31.547415+0000","last_clean":"2024-07-17T08:41:31.547415+0000","last_became_active":"2024-07-17T08:41:29.206624+0000","last_became_peered":"2024-07-17T08:41:29.206624+0000","last_unstale":"2024-07-17T08:41:31.547415+0000","last_undegraded":"2024-07-17T08:41:31.547415+0000","last_fullsized":"2024-07-17T08:41:31.547415+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:13.583701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.11","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546479+0000","last_change":"2024-07-17T08:41:30.235697+0000","last_active":"2024-07-17T08:41:31.546479+0000","last_peered":"2024-07-17T08:41:31.546479+0000","last_clean":"2024-07-17T08:41:31.546479+0000","last_became_active":"2024-07-17T08:41:30.234429+0000","last_became_peered":"2024-07-17T08:41:30.234429+0000","last_unstale":"2024-07-17T08:41:31.546479+0000","last_undegraded":"2024-07-17T08:41:31.546479+0000","last_fullsized":"2024-07-17T08:41:31.546479+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:21.017995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"41.12","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545621+0000","last_change":"2024-07-17T08:41:29.209987+0000","last_active":"2024-07-17T08:41:31.545621+0000","last_peered":"2024-07-17T08:41:31.545621+0000","last_clean":"2024-07-17T08:41:31.545621+0000","last_became_active":"2024-07-17T08:41:29.208171+0000","last_became_peered":"2024-07-17T08:41:29.208171+0000","last_unstale":"2024-07-17T08:41:31.545621+0000","last_undegraded":"2024-07-17T08:41:31.545621+0000","last_fullsized":"2024-07-17T08:41:31.545621+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:12:49.381673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.12","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218287+0000","last_change":"2024-07-17T08:41:30.236119+0000","last_active":"2024-07-17T08:41:31.218287+0000","last_peered":"2024-07-17T08:41:31.218287+0000","last_clean":"2024-07-17T08:41:31.218287+0000","last_became_active":"2024-07-17T08:41:30.235828+0000","last_became_peered":"2024-07-17T08:41:30.235828+0000","last_unstale":"2024-07-17T08:41:31.218287+0000","last_undegraded":"2024-07-17T08:41:31.218287+0000","last_fullsized":"2024-07-17T08:41:31.218287+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:06.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.11","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545452+0000","last_change":"2024-07-17T08:41:29.210820+0000","last_active":"2024-07-17T08:41:31.545452+0000","last_peered":"2024-07-17T08:41:31.545452+0000","last_clean":"2024-07-17T08:41:31.545452+0000","last_became_active":"2024-07-17T08:41:29.210276+0000","last_became_peered":"2024-07-17T08:41:29.210276+0000","last_unstale":"2024-07-17T08:41:31.545452+0000","last_undegraded":"2024-07-17T08:41:31.545452+0000","last_fullsized":"2024-07-17T08:41:31.545452+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:59.985686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.13","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547288+0000","last_change":"2024-07-17T08:41:30.250985+0000","last_active":"2024-07-17T08:41:31.547288+0000","last_peered":"2024-07-17T08:41:31.547288+0000","last_clean":"2024-07-17T08:41:31.547288+0000","last_became_active":"2024-07-17T08:41:30.249950+0000","last_became_peered":"2024-07-17T08:41:30.249950+0000","last_unstale":"2024-07-17T08:41:31.547288+0000","last_undegraded":"2024-07-17T08:41:31.547288+0000","last_fullsized":"2024-07-17T08:41:31.547288+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:09.804315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.10","version":"307'8","reported_seq":22,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217242+0000","last_change":"2024-07-17T08:41:29.211608+0000","last_active":"2024-07-17T08:41:31.217242+0000","last_peered":"2024-07-17T08:41:31.217242+0000","last_clean":"2024-07-17T08:41:31.217242+0000","last_became_active":"2024-07-17T08:41:29.211144+0000","last_became_peered":"2024-07-17T08:41:29.211144+0000","last_unstale":"2024-07-17T08:41:31.217242+0000","last_undegraded":"2024-07-17T08:41:31.217242+0000","last_fullsized":"2024-07-17T08:41:31.217242+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:13.583701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217669+0000","last_change":"2024-07-17T08:41:30.238323+0000","last_active":"2024-07-17T08:41:31.217669+0000","last_peered":"2024-07-17T08:41:31.217669+0000","last_clean":"2024-07-17T08:41:31.217669+0000","last_became_active":"2024-07-17T08:41:30.236904+0000","last_became_peered":"2024-07-17T08:41:30.236904+0000","last_unstale":"2024-07-17T08:41:31.217669+0000","last_undegraded":"2024-07-17T08:41:31.217669+0000","last_fullsized":"2024-07-17T08:41:31.217669+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:33.474235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.f","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546759+0000","last_change":"2024-07-17T08:41:29.198973+0000","last_active":"2024-07-17T08:41:31.546759+0000","last_peered":"2024-07-17T08:41:31.546759+0000","last_clean":"2024-07-17T08:41:31.546759+0000","last_became_active":"2024-07-17T08:41:29.198583+0000","last_became_peered":"2024-07-17T08:41:29.198583+0000","last_unstale":"2024-07-17T08:41:31.546759+0000","last_undegraded":"2024-07-17T08:41:31.546759+0000","last_fullsized":"2024-07-17T08:41:31.546759+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:24:42.232001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545997+0000","last_change":"2024-07-17T08:41:30.246129+0000","last_active":"2024-07-17T08:41:31.545997+0000","last_peered":"2024-07-17T08:41:31.545997+0000","last_clean":"2024-07-17T08:41:31.545997+0000","last_became_active":"2024-07-17T08:41:30.245848+0000","last_became_peered":"2024-07-17T08:41:30.245848+0000","last_unstale":"2024-07-17T08:41:31.545997+0000","last_undegraded":"2024-07-17T08:41:31.545997+0000","last_fullsized":"2024-07-17T08:41:31.545997+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:57.819939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.e","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547042+0000","last_change":"2024-07-17T08:41:29.208552+0000","last_active":"2024-07-17T08:41:31.547042+0000","last_peered":"2024-07-17T08:41:31.547042+0000","last_clean":"2024-07-17T08:41:31.547042+0000","last_became_active":"2024-07-17T08:41:29.208144+0000","last_became_peered":"2024-07-17T08:41:29.208144+0000","last_unstale":"2024-07-17T08:41:31.547042+0000","last_undegraded":"2024-07-17T08:41:31.547042+0000","last_fullsized":"2024-07-17T08:41:31.547042+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:17.124174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212931+0000","last_change":"2024-07-17T08:41:30.228596+0000","last_active":"2024-07-17T08:41:31.212931+0000","last_peered":"2024-07-17T08:41:31.212931+0000","last_clean":"2024-07-17T08:41:31.212931+0000","last_became_active":"2024-07-17T08:41:30.227986+0000","last_became_peered":"2024-07-17T08:41:30.227986+0000","last_unstale":"2024-07-17T08:41:31.212931+0000","last_undegraded":"2024-07-17T08:41:31.212931+0000","last_fullsized":"2024-07-17T08:41:31.212931+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:35.777773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.d","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547919+0000","last_change":"2024-07-17T08:41:29.201902+0000","last_active":"2024-07-17T08:41:31.547919+0000","last_peered":"2024-07-17T08:41:31.547919+0000","last_clean":"2024-07-17T08:41:31.547919+0000","last_became_active":"2024-07-17T08:41:29.201278+0000","last_became_peered":"2024-07-17T08:41:29.201278+0000","last_unstale":"2024-07-17T08:41:31.547919+0000","last_undegraded":"2024-07-17T08:41:31.547919+0000","last_fullsized":"2024-07-17T08:41:31.547919+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:21.014391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219478+0000","last_change":"2024-07-17T08:41:30.237892+0000","last_active":"2024-07-17T08:41:31.219478+0000","last_peered":"2024-07-17T08:41:31.219478+0000","last_clean":"2024-07-17T08:41:31.219478+0000","last_became_active":"2024-07-17T08:41:30.237106+0000","last_became_peered":"2024-07-17T08:41:30.237106+0000","last_unstale":"2024-07-17T08:41:31.219478+0000","last_undegraded":"2024-07-17T08:41:31.219478+0000","last_fullsized":"2024-07-17T08:41:31.219478+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:03.757560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.c","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217749+0000","last_change":"2024-07-17T08:41:29.206513+0000","last_active":"2024-07-17T08:41:31.217749+0000","last_peered":"2024-07-17T08:41:31.217749+0000","last_clean":"2024-07-17T08:41:31.217749+0000","last_became_active":"2024-07-17T08:41:29.206230+0000","last_became_peered":"2024-07-17T08:41:29.206230+0000","last_unstale":"2024-07-17T08:41:31.217749+0000","last_undegraded":"2024-07-17T08:41:31.217749+0000","last_fullsized":"2024-07-17T08:41:31.217749+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:20.995346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.8","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217811+0000","last_change":"2024-07-17T08:41:30.251934+0000","last_active":"2024-07-17T08:41:31.217811+0000","last_peered":"2024-07-17T08:41:31.217811+0000","last_clean":"2024-07-17T08:41:31.217811+0000","last_became_active":"2024-07-17T08:41:30.251635+0000","last_became_peered":"2024-07-17T08:41:30.251635+0000","last_unstale":"2024-07-17T08:41:31.217811+0000","last_undegraded":"2024-07-17T08:41:31.217811+0000","last_fullsized":"2024-07-17T08:41:31.217811+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:10.283195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.b","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212377+0000","last_change":"2024-07-17T08:41:29.199653+0000","last_active":"2024-07-17T08:41:31.212377+0000","last_peered":"2024-07-17T08:41:31.212377+0000","last_clean":"2024-07-17T08:41:31.212377+0000","last_became_active":"2024-07-17T08:41:29.198482+0000","last_became_peered":"2024-07-17T08:41:29.198482+0000","last_unstale":"2024-07-17T08:41:31.212377+0000","last_undegraded":"2024-07-17T08:41:31.212377+0000","last_fullsized":"2024-07-17T08:41:31.212377+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:37:23.654997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.9","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545918+0000","last_change":"2024-07-17T08:41:30.221439+0000","last_active":"2024-07-17T08:41:31.545918+0000","last_peered":"2024-07-17T08:41:31.545918+0000","last_clean":"2024-07-17T08:41:31.545918+0000","last_became_active":"2024-07-17T08:41:30.219926+0000","last_became_peered":"2024-07-17T08:41:30.219926+0000","last_unstale":"2024-07-17T08:41:31.545918+0000","last_undegraded":"2024-07-17T08:41:31.545918+0000","last_fullsized":"2024-07-17T08:41:31.545918+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:12.922573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.a","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256098+0000","last_change":"2024-07-17T08:41:29.195413+0000","last_active":"2024-07-17T08:41:31.256098+0000","last_peered":"2024-07-17T08:41:31.256098+0000","last_clean":"2024-07-17T08:41:31.256098+0000","last_became_active":"2024-07-17T08:41:29.192565+0000","last_became_peered":"2024-07-17T08:41:29.192565+0000","last_unstale":"2024-07-17T08:41:31.256098+0000","last_undegraded":"2024-07-17T08:41:31.256098+0000","last_fullsized":"2024-07-17T08:41:31.256098+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:05.997918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546768+0000","last_change":"2024-07-17T08:41:30.244895+0000","last_active":"2024-07-17T08:41:31.546768+0000","last_peered":"2024-07-17T08:41:31.546768+0000","last_clean":"2024-07-17T08:41:31.546768+0000","last_became_active":"2024-07-17T08:41:30.244599+0000","last_became_peered":"2024-07-17T08:41:30.244599+0000","last_unstale":"2024-07-17T08:41:31.546768+0000","last_undegraded":"2024-07-17T08:41:31.546768+0000","last_fullsized":"2024-07-17T08:41:31.546768+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:54:30.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.9","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546653+0000","last_change":"2024-07-17T08:41:29.198505+0000","last_active":"2024-07-17T08:41:31.546653+0000","last_peered":"2024-07-17T08:41:31.546653+0000","last_clean":"2024-07-17T08:41:31.546653+0000","last_became_active":"2024-07-17T08:41:29.198173+0000","last_became_peered":"2024-07-17T08:41:29.198173+0000","last_unstale":"2024-07-17T08:41:31.546653+0000","last_undegraded":"2024-07-17T08:41:31.546653+0000","last_fullsized":"2024-07-17T08:41:31.546653+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:48:27.066458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256744+0000","last_change":"2024-07-17T08:41:30.219305+0000","last_active":"2024-07-17T08:41:31.256744+0000","last_peered":"2024-07-17T08:41:31.256744+0000","last_clean":"2024-07-17T08:41:31.256744+0000","last_became_active":"2024-07-17T08:41:30.218957+0000","last_became_peered":"2024-07-17T08:41:30.218957+0000","last_unstale":"2024-07-17T08:41:31.256744+0000","last_undegraded":"2024-07-17T08:41:31.256744+0000","last_fullsized":"2024-07-17T08:41:31.256744+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:26.521135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.8","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546940+0000","last_change":"2024-07-17T08:41:29.207133+0000","last_active":"2024-07-17T08:41:31.546940+0000","last_peered":"2024-07-17T08:41:31.546940+0000","last_clean":"2024-07-17T08:41:31.546940+0000","last_became_active":"2024-07-17T08:41:29.206797+0000","last_became_peered":"2024-07-17T08:41:29.206797+0000","last_unstale":"2024-07-17T08:41:31.546940+0000","last_undegraded":"2024-07-17T08:41:31.546940+0000","last_fullsized":"2024-07-17T08:41:31.546940+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:28.578970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.4","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256233+0000","last_change":"2024-07-17T08:41:30.224810+0000","last_active":"2024-07-17T08:41:31.256233+0000","last_peered":"2024-07-17T08:41:31.256233+0000","last_clean":"2024-07-17T08:41:31.256233+0000","last_became_active":"2024-07-17T08:41:30.224211+0000","last_became_peered":"2024-07-17T08:41:30.224211+0000","last_unstale":"2024-07-17T08:41:31.256233+0000","last_undegraded":"2024-07-17T08:41:31.256233+0000","last_fullsized":"2024-07-17T08:41:31.256233+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:13.725228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.7","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547165+0000","last_change":"2024-07-17T08:41:29.205986+0000","last_active":"2024-07-17T08:41:31.547165+0000","last_peered":"2024-07-17T08:41:31.547165+0000","last_clean":"2024-07-17T08:41:31.547165+0000","last_became_active":"2024-07-17T08:41:29.205626+0000","last_became_peered":"2024-07-17T08:41:29.205626+0000","last_unstale":"2024-07-17T08:41:31.547165+0000","last_undegraded":"2024-07-17T08:41:31.547165+0000","last_fullsized":"2024-07-17T08:41:31.547165+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:36.744072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.5","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546114+0000","last_change":"2024-07-17T08:41:30.252458+0000","last_active":"2024-07-17T08:41:31.546114+0000","last_peered":"2024-07-17T08:41:31.546114+0000","last_clean":"2024-07-17T08:41:31.546114+0000","last_became_active":"2024-07-17T08:41:30.252176+0000","last_became_peered":"2024-07-17T08:41:30.252176+0000","last_unstale":"2024-07-17T08:41:31.546114+0000","last_undegraded":"2024-07-17T08:41:31.546114+0000","last_fullsized":"2024-07-17T08:41:31.546114+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:29.052969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.6","version":"308'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.260019+0000","last_change":"2024-07-17T08:41:29.207891+0000","last_active":"2024-07-17T08:41:31.260019+0000","last_peered":"2024-07-17T08:41:31.260019+0000","last_clean":"2024-07-17T08:41:31.260019+0000","last_became_active":"2024-07-17T08:41:29.206951+0000","last_became_peered":"2024-07-17T08:41:29.206951+0000","last_unstale":"2024-07-17T08:41:31.260019+0000","last_undegraded":"2024-07-17T08:41:31.260019+0000","last_fullsized":"2024-07-17T08:41:31.260019+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:16:11.063831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.6","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546764+0000","last_change":"2024-07-17T08:41:30.245714+0000","last_active":"2024-07-17T08:41:31.546764+0000","last_peered":"2024-07-17T08:41:31.546764+0000","last_clean":"2024-07-17T08:41:31.546764+0000","last_became_active":"2024-07-17T08:41:30.244299+0000","last_became_peered":"2024-07-17T08:41:30.244299+0000","last_unstale":"2024-07-17T08:41:31.546764+0000","last_undegraded":"2024-07-17T08:41:31.546764+0000","last_fullsized":"2024-07-17T08:41:31.546764+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:09.374417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.5","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546345+0000","last_change":"2024-07-17T08:41:29.195626+0000","last_active":"2024-07-17T08:41:31.546345+0000","last_peered":"2024-07-17T08:41:31.546345+0000","last_clean":"2024-07-17T08:41:31.546345+0000","last_became_active":"2024-07-17T08:41:29.195338+0000","last_became_peered":"2024-07-17T08:41:29.195338+0000","last_unstale":"2024-07-17T08:41:31.546345+0000","last_undegraded":"2024-07-17T08:41:31.546345+0000","last_fullsized":"2024-07-17T08:41:31.546345+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:23.083181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.7","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219664+0000","last_change":"2024-07-17T08:41:30.238364+0000","last_active":"2024-07-17T08:41:31.219664+0000","last_peered":"2024-07-17T08:41:31.219664+0000","last_clean":"2024-07-17T08:41:31.219664+0000","last_became_active":"2024-07-17T08:41:30.237609+0000","last_became_peered":"2024-07-17T08:41:30.237609+0000","last_unstale":"2024-07-17T08:41:31.219664+0000","last_undegraded":"2024-07-17T08:41:31.219664+0000","last_fullsized":"2024-07-17T08:41:31.219664+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:41.124346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.4","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548612+0000","last_change":"2024-07-17T08:41:29.182690+0000","last_active":"2024-07-17T08:41:31.548612+0000","last_peered":"2024-07-17T08:41:31.548612+0000","last_clean":"2024-07-17T08:41:31.548612+0000","last_became_active":"2024-07-17T08:41:29.181887+0000","last_became_peered":"2024-07-17T08:41:29.181887+0000","last_unstale":"2024-07-17T08:41:31.548612+0000","last_undegraded":"2024-07-17T08:41:31.548612+0000","last_fullsized":"2024-07-17T08:41:31.548612+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:41.926926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.0","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545128+0000","last_change":"2024-07-17T08:41:30.244387+0000","last_active":"2024-07-17T08:41:31.545128+0000","last_peered":"2024-07-17T08:41:31.545128+0000","last_clean":"2024-07-17T08:41:31.545128+0000","last_became_active":"2024-07-17T08:41:30.243646+0000","last_became_peered":"2024-07-17T08:41:30.243646+0000","last_unstale":"2024-07-17T08:41:31.545128+0000","last_undegraded":"2024-07-17T08:41:31.545128+0000","last_fullsized":"2024-07-17T08:41:31.545128+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.587744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.3","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212474+0000","last_change":"2024-07-17T08:41:29.199331+0000","last_active":"2024-07-17T08:41:31.212474+0000","last_peered":"2024-07-17T08:41:31.212474+0000","last_clean":"2024-07-17T08:41:31.212474+0000","last_became_active":"2024-07-17T08:41:29.198876+0000","last_became_peered":"2024-07-17T08:41:29.198876+0000","last_unstale":"2024-07-17T08:41:31.212474+0000","last_undegraded":"2024-07-17T08:41:31.212474+0000","last_fullsized":"2024-07-17T08:41:31.212474+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:03.989498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.1","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217311+0000","last_change":"2024-07-17T08:41:30.245521+0000","last_active":"2024-07-17T08:41:31.217311+0000","last_peered":"2024-07-17T08:41:31.217311+0000","last_clean":"2024-07-17T08:41:31.217311+0000","last_became_active":"2024-07-17T08:41:30.245253+0000","last_became_peered":"2024-07-17T08:41:30.245253+0000","last_unstale":"2024-07-17T08:41:31.217311+0000","last_undegraded":"2024-07-17T08:41:31.217311+0000","last_fullsized":"2024-07-17T08:41:31.217311+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:14.933853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.2","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218285+0000","last_change":"2024-07-17T08:41:29.212127+0000","last_active":"2024-07-17T08:41:31.218285+0000","last_peered":"2024-07-17T08:41:31.218285+0000","last_clean":"2024-07-17T08:41:31.218285+0000","last_became_active":"2024-07-17T08:41:29.211801+0000","last_became_peered":"2024-07-17T08:41:29.211801+0000","last_unstale":"2024-07-17T08:41:31.218285+0000","last_undegraded":"2024-07-17T08:41:31.218285+0000","last_fullsized":"2024-07-17T08:41:31.218285+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:45.284157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545840+0000","last_change":"2024-07-17T08:41:30.245885+0000","last_active":"2024-07-17T08:41:31.545840+0000","last_peered":"2024-07-17T08:41:31.545840+0000","last_clean":"2024-07-17T08:41:31.545840+0000","last_became_active":"2024-07-17T08:41:30.241917+0000","last_became_peered":"2024-07-17T08:41:30.241917+0000","last_unstale":"2024-07-17T08:41:31.545840+0000","last_undegraded":"2024-07-17T08:41:31.545840+0000","last_fullsized":"2024-07-17T08:41:31.545840+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:22:56.170592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212456+0000","last_change":"2024-07-17T08:41:29.195963+0000","last_active":"2024-07-17T08:41:31.212456+0000","last_peered":"2024-07-17T08:41:31.212456+0000","last_clean":"2024-07-17T08:41:31.212456+0000","last_became_active":"2024-07-17T08:41:29.195222+0000","last_became_peered":"2024-07-17T08:41:29.195222+0000","last_unstale":"2024-07-17T08:41:31.212456+0000","last_undegraded":"2024-07-17T08:41:31.212456+0000","last_fullsized":"2024-07-17T08:41:31.212456+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:51.932727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.3","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547213+0000","last_change":"2024-07-17T08:41:30.240156+0000","last_active":"2024-07-17T08:41:31.547213+0000","last_peered":"2024-07-17T08:41:31.547213+0000","last_clean":"2024-07-17T08:41:31.547213+0000","last_became_active":"2024-07-17T08:41:30.238063+0000","last_became_peered":"2024-07-17T08:41:30.238063+0000","last_unstale":"2024-07-17T08:41:31.547213+0000","last_undegraded":"2024-07-17T08:41:31.547213+0000","last_fullsized":"2024-07-17T08:41:31.547213+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:36.730131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.0","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.255940+0000","last_change":"2024-07-17T08:41:29.209700+0000","last_active":"2024-07-17T08:41:31.255940+0000","last_peered":"2024-07-17T08:41:31.255940+0000","last_clean":"2024-07-17T08:41:31.255940+0000","last_became_active":"2024-07-17T08:41:29.209333+0000","last_became_peered":"2024-07-17T08:41:29.209333+0000","last_unstale":"2024-07-17T08:41:31.255940+0000","last_undegraded":"2024-07-17T08:41:31.255940+0000","last_fullsized":"2024-07-17T08:41:31.255940+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:58.758378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.2b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546923+0000","last_change":"2024-07-17T08:41:30.247457+0000","last_active":"2024-07-17T08:41:31.546923+0000","last_peered":"2024-07-17T08:41:31.546923+0000","last_clean":"2024-07-17T08:41:31.546923+0000","last_became_active":"2024-07-17T08:41:30.247176+0000","last_became_peered":"2024-07-17T08:41:30.247176+0000","last_unstale":"2024-07-17T08:41:31.546923+0000","last_undegraded":"2024-07-17T08:41:31.546923+0000","last_fullsized":"2024-07-17T08:41:31.546923+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:29.319225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"299'4610","reported_seq":5829,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217762+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:41:31.217762+0000","last_peered":"2024-07-17T08:41:31.217762+0000","last_clean":"2024-07-17T08:41:31.217762+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:41:31.217762+0000","last_undegraded":"2024-07-17T08:41:31.217762+0000","last_fullsized":"2024-07-17T08:41:31.217762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4610,"log_dups_size":0,"ondisk_log_size":4610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":469,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":42,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":41,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":469,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4610,"ondisk_log_size":4610,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75840,"kb_used_data":4404,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296000,"statfs":{"total":96636764160,"available":96559104000,"internally_reserved":0,"allocated":4509696,"data_stored":2743825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510705,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53036,"kb_used_data":3808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318804,"statfs":{"total":96636764160,"available":96582455296,"internally_reserved":0,"allocated":3899392,"data_stored":2146071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510706,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71120,"kb_used_data":3836,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300720,"statfs":{"total":96636764160,"available":96563937280,"internally_reserved":0,"allocated":3928064,"data_stored":2155013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543411,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53052,"kb_used_data":3824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318788,"statfs":{"total":96636764160,"available":96582438912,"internally_reserved":0,"allocated":3915776,"data_stored":2153401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510705,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57212,"kb_used_data":3820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314628,"statfs":{"total":96636764160,"available":96578179072,"internally_reserved":0,"allocated":3911680,"data_stored":2153367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543409,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57224,"kb_used_data":3836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314616,"statfs":{"total":96636764160,"available":96578166784,"internally_reserved":0,"allocated":3928064,"data_stored":2155001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510706,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53068,"kb_used_data":3832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318772,"statfs":{"total":96636764160,"available":96582422528,"internally_reserved":0,"allocated":3923968,"data_stored":2154041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510705,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53644,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318196,"statfs":{"total":96636764160,"available":96581832704,"internally_reserved":0,"allocated":4517888,"data_stored":2744477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:41:34.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 -- 172.21.15.44:0/946039431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f502815a040 msgr2=0x7f50280ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 --2- 172.21.15.44:0/946039431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f502815a040 0x7f50280ac3e0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f5028070a70 tx=0x7f501002a560 comp rx=0 tx=0).stop 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 -- 172.21.15.44:0/946039431 shutdown_connections 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 --2- 172.21.15.44:0/946039431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f502815a040 0x7f50280ac3e0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 --2- 172.21.15.44:0/946039431 >> v2:172.21.15.175:3300/0 conn(0x7f5028153a50 0x7f5028159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 -- 172.21.15.44:0/946039431 >> 172.21.15.44:0/946039431 conn(0x7f50280b0fb0 msgr2=0x7f50280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:34.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 -- 172.21.15.44:0/946039431 shutdown_connections 2024-07-17T08:41:34.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.908+0000 7f502e800640 1 -- 172.21.15.44:0/946039431 wait complete. 2024-07-17T08:41:34.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.910+0000 7f502e800640 1 Processor -- start 2024-07-17T08:41:34.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.910+0000 7f502e800640 1 -- start start 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5028153a50 0x7f5028096f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f502815a040 0x7f5028152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5028153e30 con 0x7f50280ac920 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f502806f1b0 con 0x7f5028153a50 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f502e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5028153fb0 con 0x7f502815a040 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f5027400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f5027400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55242/0 (socket says 172.21.15.44:55242) 2024-07-17T08:41:34.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f5027400640 1 -- 172.21.15.44:0/3251092005 learned_addr learned my addr 172.21.15.44:0/3251092005 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:34.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.912+0000 7f501ea00640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5028153a50 0x7f5028096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027e00640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f502815a040 0x7f5028152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 -- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f502815a040 msgr2=0x7f5028152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f502815a040 0x7f5028152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 -- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5028153a50 msgr2=0x7f5028096f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5028153a50 0x7f5028096f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 -- 172.21.15.44:0/3251092005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5028097490 con 0x7f50280ac920 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f5027400640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f501800de30 tx=0x7f501800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018004d20 con 0x7f50280ac920 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f501800e660 con 0x7f50280ac920 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800ea90 con 0x7f50280ac920 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.913+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5028097730 con 0x7f50280ac920 2024-07-17T08:41:34.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.914+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5028071b50 con 0x7f50280ac920 2024-07-17T08:41:34.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.915+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5028159b00 con 0x7f50280ac920 2024-07-17T08:41:34.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.916+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f501800ec30 con 0x7f50280ac920 2024-07-17T08:41:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.917+0000 7f501fe00640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 0x7f50040791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:34.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.917+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f50180995c0 con 0x7f50280ac920 2024-07-17T08:41:34.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.918+0000 7f501ea00640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 0x7f50040791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:34.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.919+0000 7f501ea00640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 0x7f50040791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5010033000 tx=0x7f5010004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:34.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:34.921+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f50180624f0 con 0x7f50280ac920 2024-07-17T08:41:35.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.135+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 --> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f502809d590 con 0x7f5004076d00 2024-07-17T08:41:35.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.148+0000 7f501fe00640 1 -- 172.21.15.44:0/3251092005 <== mgr.9029 v2:172.21.15.175:6834/3743840884 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f502809d590 con 0x7f5004076d00 2024-07-17T08:41:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:35.155 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 msgr2=0x7f50040791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 0x7f50040791c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5010033000 tx=0x7f5010004730 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 msgr2=0x7f50281f39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f501800de30 tx=0x7f501800b730 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 shutdown_connections 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f5004076d00 0x7f50040791c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f502815a040 0x7f5028152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5028153a50 0x7f5028096f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 --2- 172.21.15.44:0/3251092005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50280ac920 0x7f50281f39b0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 >> 172.21.15.44:0/3251092005 conn(0x7f50280b0fb0 msgr2=0x7f502815d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.156+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 shutdown_connections 2024-07-17T08:41:35.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.157+0000 7f502e800640 1 -- 172.21.15.44:0/3251092005 wait complete. 2024-07-17T08:41:35.177 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:41:33.949938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":514,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4649,"ondisk_log_size":4649,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":474196,"kb_used_data":31772,"kb_used_omap":12,"kb_used_meta":442291,"kb_avail":754500524,"statfs":{"total":773094113280,"available":772608536576,"internally_reserved":0,"allocated":32534528,"data_stored":18405196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452906544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.788172"},"pg_stats":[{"pgid":"42.3f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545897+0000","last_change":"2024-07-17T08:41:30.218044+0000","last_active":"2024-07-17T08:41:31.545897+0000","last_peered":"2024-07-17T08:41:31.545897+0000","last_clean":"2024-07-17T08:41:31.545897+0000","last_became_active":"2024-07-17T08:41:30.217230+0000","last_became_peered":"2024-07-17T08:41:30.217230+0000","last_unstale":"2024-07-17T08:41:31.545897+0000","last_undegraded":"2024-07-17T08:41:31.545897+0000","last_fullsized":"2024-07-17T08:41:31.545897+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:17:04.952677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.3e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546345+0000","last_change":"2024-07-17T08:41:30.253962+0000","last_active":"2024-07-17T08:41:31.546345+0000","last_peered":"2024-07-17T08:41:31.546345+0000","last_clean":"2024-07-17T08:41:31.546345+0000","last_became_active":"2024-07-17T08:41:30.252198+0000","last_became_peered":"2024-07-17T08:41:30.252198+0000","last_unstale":"2024-07-17T08:41:31.546345+0000","last_undegraded":"2024-07-17T08:41:31.546345+0000","last_fullsized":"2024-07-17T08:41:31.546345+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:41.865153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.3d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548065+0000","last_change":"2024-07-17T08:41:30.242648+0000","last_active":"2024-07-17T08:41:31.548065+0000","last_peered":"2024-07-17T08:41:31.548065+0000","last_clean":"2024-07-17T08:41:31.548065+0000","last_became_active":"2024-07-17T08:41:30.242327+0000","last_became_peered":"2024-07-17T08:41:30.242327+0000","last_unstale":"2024-07-17T08:41:31.548065+0000","last_undegraded":"2024-07-17T08:41:31.548065+0000","last_fullsized":"2024-07-17T08:41:31.548065+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:26.342934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547194+0000","last_change":"2024-07-17T08:41:30.240939+0000","last_active":"2024-07-17T08:41:31.547194+0000","last_peered":"2024-07-17T08:41:31.547194+0000","last_clean":"2024-07-17T08:41:31.547194+0000","last_became_active":"2024-07-17T08:41:30.240665+0000","last_became_peered":"2024-07-17T08:41:30.240665+0000","last_unstale":"2024-07-17T08:41:31.547194+0000","last_undegraded":"2024-07-17T08:41:31.547194+0000","last_fullsized":"2024-07-17T08:41:31.547194+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:56.109775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.3b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218771+0000","last_change":"2024-07-17T08:41:30.237397+0000","last_active":"2024-07-17T08:41:31.218771+0000","last_peered":"2024-07-17T08:41:31.218771+0000","last_clean":"2024-07-17T08:41:31.218771+0000","last_became_active":"2024-07-17T08:41:30.236120+0000","last_became_peered":"2024-07-17T08:41:30.236120+0000","last_unstale":"2024-07-17T08:41:31.218771+0000","last_undegraded":"2024-07-17T08:41:31.218771+0000","last_fullsized":"2024-07-17T08:41:31.218771+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:55:27.725747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.3a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545907+0000","last_change":"2024-07-17T08:41:30.249579+0000","last_active":"2024-07-17T08:41:31.545907+0000","last_peered":"2024-07-17T08:41:31.545907+0000","last_clean":"2024-07-17T08:41:31.545907+0000","last_became_active":"2024-07-17T08:41:30.246124+0000","last_became_peered":"2024-07-17T08:41:30.246124+0000","last_unstale":"2024-07-17T08:41:31.545907+0000","last_undegraded":"2024-07-17T08:41:31.545907+0000","last_fullsized":"2024-07-17T08:41:31.545907+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:33.474235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.39","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217520+0000","last_change":"2024-07-17T08:41:30.238672+0000","last_active":"2024-07-17T08:41:31.217520+0000","last_peered":"2024-07-17T08:41:31.217520+0000","last_clean":"2024-07-17T08:41:31.217520+0000","last_became_active":"2024-07-17T08:41:30.238372+0000","last_became_peered":"2024-07-17T08:41:30.238372+0000","last_unstale":"2024-07-17T08:41:31.217520+0000","last_undegraded":"2024-07-17T08:41:31.217520+0000","last_fullsized":"2024-07-17T08:41:31.217520+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.485715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.38","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545822+0000","last_change":"2024-07-17T08:41:30.248116+0000","last_active":"2024-07-17T08:41:31.545822+0000","last_peered":"2024-07-17T08:41:31.545822+0000","last_clean":"2024-07-17T08:41:31.545822+0000","last_became_active":"2024-07-17T08:41:30.246503+0000","last_became_peered":"2024-07-17T08:41:30.246503+0000","last_unstale":"2024-07-17T08:41:31.545822+0000","last_undegraded":"2024-07-17T08:41:31.545822+0000","last_fullsized":"2024-07-17T08:41:31.545822+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:17.471148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.37","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546896+0000","last_change":"2024-07-17T08:41:30.241429+0000","last_active":"2024-07-17T08:41:31.546896+0000","last_peered":"2024-07-17T08:41:31.546896+0000","last_clean":"2024-07-17T08:41:31.546896+0000","last_became_active":"2024-07-17T08:41:30.241103+0000","last_became_peered":"2024-07-17T08:41:30.241103+0000","last_unstale":"2024-07-17T08:41:31.546896+0000","last_undegraded":"2024-07-17T08:41:31.546896+0000","last_fullsized":"2024-07-17T08:41:31.546896+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:21.575456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.36","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217591+0000","last_change":"2024-07-17T08:41:30.237779+0000","last_active":"2024-07-17T08:41:31.217591+0000","last_peered":"2024-07-17T08:41:31.217591+0000","last_clean":"2024-07-17T08:41:31.217591+0000","last_became_active":"2024-07-17T08:41:30.237382+0000","last_became_peered":"2024-07-17T08:41:30.237382+0000","last_unstale":"2024-07-17T08:41:31.217591+0000","last_undegraded":"2024-07-17T08:41:31.217591+0000","last_fullsized":"2024-07-17T08:41:31.217591+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:08.481577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.35","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545795+0000","last_change":"2024-07-17T08:41:30.251748+0000","last_active":"2024-07-17T08:41:31.545795+0000","last_peered":"2024-07-17T08:41:31.545795+0000","last_clean":"2024-07-17T08:41:31.545795+0000","last_became_active":"2024-07-17T08:41:30.251444+0000","last_became_peered":"2024-07-17T08:41:30.251444+0000","last_unstale":"2024-07-17T08:41:31.545795+0000","last_undegraded":"2024-07-17T08:41:31.545795+0000","last_fullsized":"2024-07-17T08:41:31.545795+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:38.106633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.34","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548219+0000","last_change":"2024-07-17T08:41:30.238575+0000","last_active":"2024-07-17T08:41:31.548219+0000","last_peered":"2024-07-17T08:41:31.548219+0000","last_clean":"2024-07-17T08:41:31.548219+0000","last_became_active":"2024-07-17T08:41:30.234809+0000","last_became_peered":"2024-07-17T08:41:30.234809+0000","last_unstale":"2024-07-17T08:41:31.548219+0000","last_undegraded":"2024-07-17T08:41:31.548219+0000","last_fullsized":"2024-07-17T08:41:31.548219+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:41:45.951593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.33","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256608+0000","last_change":"2024-07-17T08:41:30.237984+0000","last_active":"2024-07-17T08:41:31.256608+0000","last_peered":"2024-07-17T08:41:31.256608+0000","last_clean":"2024-07-17T08:41:31.256608+0000","last_became_active":"2024-07-17T08:41:30.237648+0000","last_became_peered":"2024-07-17T08:41:30.237648+0000","last_unstale":"2024-07-17T08:41:31.256608+0000","last_undegraded":"2024-07-17T08:41:31.256608+0000","last_fullsized":"2024-07-17T08:41:31.256608+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:29:48.974758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.32","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546357+0000","last_change":"2024-07-17T08:41:30.241210+0000","last_active":"2024-07-17T08:41:31.546357+0000","last_peered":"2024-07-17T08:41:31.546357+0000","last_clean":"2024-07-17T08:41:31.546357+0000","last_became_active":"2024-07-17T08:41:30.240916+0000","last_became_peered":"2024-07-17T08:41:30.240916+0000","last_unstale":"2024-07-17T08:41:31.546357+0000","last_undegraded":"2024-07-17T08:41:31.546357+0000","last_fullsized":"2024-07-17T08:41:31.546357+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:37.537804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.31","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545300+0000","last_change":"2024-07-17T08:41:30.249610+0000","last_active":"2024-07-17T08:41:31.545300+0000","last_peered":"2024-07-17T08:41:31.545300+0000","last_clean":"2024-07-17T08:41:31.545300+0000","last_became_active":"2024-07-17T08:41:30.249320+0000","last_became_peered":"2024-07-17T08:41:30.249320+0000","last_unstale":"2024-07-17T08:41:31.545300+0000","last_undegraded":"2024-07-17T08:41:31.545300+0000","last_fullsized":"2024-07-17T08:41:31.545300+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:24:08.481577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.30","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546372+0000","last_change":"2024-07-17T08:41:30.247253+0000","last_active":"2024-07-17T08:41:31.546372+0000","last_peered":"2024-07-17T08:41:31.546372+0000","last_clean":"2024-07-17T08:41:31.546372+0000","last_became_active":"2024-07-17T08:41:30.246986+0000","last_became_peered":"2024-07-17T08:41:30.246986+0000","last_unstale":"2024-07-17T08:41:31.546372+0000","last_undegraded":"2024-07-17T08:41:31.546372+0000","last_fullsized":"2024-07-17T08:41:31.546372+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:16.253173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.2f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212873+0000","last_change":"2024-07-17T08:41:30.232574+0000","last_active":"2024-07-17T08:41:31.212873+0000","last_peered":"2024-07-17T08:41:31.212873+0000","last_clean":"2024-07-17T08:41:31.212873+0000","last_became_active":"2024-07-17T08:41:30.231793+0000","last_became_peered":"2024-07-17T08:41:30.231793+0000","last_unstale":"2024-07-17T08:41:31.212873+0000","last_undegraded":"2024-07-17T08:41:31.212873+0000","last_fullsized":"2024-07-17T08:41:31.212873+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:03:16.973257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.2e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217522+0000","last_change":"2024-07-17T08:41:30.245658+0000","last_active":"2024-07-17T08:41:31.217522+0000","last_peered":"2024-07-17T08:41:31.217522+0000","last_clean":"2024-07-17T08:41:31.217522+0000","last_became_active":"2024-07-17T08:41:30.245350+0000","last_became_peered":"2024-07-17T08:41:30.245350+0000","last_unstale":"2024-07-17T08:41:31.217522+0000","last_undegraded":"2024-07-17T08:41:31.217522+0000","last_fullsized":"2024-07-17T08:41:31.217522+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.587744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547621+0000","last_change":"2024-07-17T08:41:30.250071+0000","last_active":"2024-07-17T08:41:31.547621+0000","last_peered":"2024-07-17T08:41:31.547621+0000","last_clean":"2024-07-17T08:41:31.547621+0000","last_became_active":"2024-07-17T08:41:30.249731+0000","last_became_peered":"2024-07-17T08:41:30.249731+0000","last_unstale":"2024-07-17T08:41:31.547621+0000","last_undegraded":"2024-07-17T08:41:31.547621+0000","last_fullsized":"2024-07-17T08:41:31.547621+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:04.911004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.2c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217305+0000","last_change":"2024-07-17T08:41:30.240025+0000","last_active":"2024-07-17T08:41:31.217305+0000","last_peered":"2024-07-17T08:41:31.217305+0000","last_clean":"2024-07-17T08:41:31.217305+0000","last_became_active":"2024-07-17T08:41:30.239462+0000","last_became_peered":"2024-07-17T08:41:30.239462+0000","last_unstale":"2024-07-17T08:41:31.217305+0000","last_undegraded":"2024-07-17T08:41:31.217305+0000","last_fullsized":"2024-07-17T08:41:31.217305+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:26:32.642649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.2a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547043+0000","last_change":"2024-07-17T08:41:30.239426+0000","last_active":"2024-07-17T08:41:31.547043+0000","last_peered":"2024-07-17T08:41:31.547043+0000","last_clean":"2024-07-17T08:41:31.547043+0000","last_became_active":"2024-07-17T08:41:30.239096+0000","last_became_peered":"2024-07-17T08:41:30.239096+0000","last_unstale":"2024-07-17T08:41:31.547043+0000","last_undegraded":"2024-07-17T08:41:31.547043+0000","last_fullsized":"2024-07-17T08:41:31.547043+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:19:05.463382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.29","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545267+0000","last_change":"2024-07-17T08:41:30.229883+0000","last_active":"2024-07-17T08:41:31.545267+0000","last_peered":"2024-07-17T08:41:31.545267+0000","last_clean":"2024-07-17T08:41:31.545267+0000","last_became_active":"2024-07-17T08:41:30.229500+0000","last_became_peered":"2024-07-17T08:41:30.229500+0000","last_unstale":"2024-07-17T08:41:31.545267+0000","last_undegraded":"2024-07-17T08:41:31.545267+0000","last_fullsized":"2024-07-17T08:41:31.545267+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:54.560584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.28","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217404+0000","last_change":"2024-07-17T08:41:30.239944+0000","last_active":"2024-07-17T08:41:31.217404+0000","last_peered":"2024-07-17T08:41:31.217404+0000","last_clean":"2024-07-17T08:41:31.217404+0000","last_became_active":"2024-07-17T08:41:30.238820+0000","last_became_peered":"2024-07-17T08:41:30.238820+0000","last_unstale":"2024-07-17T08:41:31.217404+0000","last_undegraded":"2024-07-17T08:41:31.217404+0000","last_fullsized":"2024-07-17T08:41:31.217404+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:33:18.363390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.27","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546469+0000","last_change":"2024-07-17T08:41:30.241126+0000","last_active":"2024-07-17T08:41:31.546469+0000","last_peered":"2024-07-17T08:41:31.546469+0000","last_clean":"2024-07-17T08:41:31.546469+0000","last_became_active":"2024-07-17T08:41:30.240798+0000","last_became_peered":"2024-07-17T08:41:30.240798+0000","last_unstale":"2024-07-17T08:41:31.546469+0000","last_undegraded":"2024-07-17T08:41:31.546469+0000","last_fullsized":"2024-07-17T08:41:31.546469+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:17.852383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.26","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219417+0000","last_change":"2024-07-17T08:41:30.253177+0000","last_active":"2024-07-17T08:41:31.219417+0000","last_peered":"2024-07-17T08:41:31.219417+0000","last_clean":"2024-07-17T08:41:31.219417+0000","last_became_active":"2024-07-17T08:41:30.252744+0000","last_became_peered":"2024-07-17T08:41:30.252744+0000","last_unstale":"2024-07-17T08:41:31.219417+0000","last_undegraded":"2024-07-17T08:41:31.219417+0000","last_fullsized":"2024-07-17T08:41:31.219417+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:23.752543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.25","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256351+0000","last_change":"2024-07-17T08:41:30.240173+0000","last_active":"2024-07-17T08:41:31.256351+0000","last_peered":"2024-07-17T08:41:31.256351+0000","last_clean":"2024-07-17T08:41:31.256351+0000","last_became_active":"2024-07-17T08:41:30.236929+0000","last_became_peered":"2024-07-17T08:41:30.236929+0000","last_unstale":"2024-07-17T08:41:31.256351+0000","last_undegraded":"2024-07-17T08:41:31.256351+0000","last_fullsized":"2024-07-17T08:41:31.256351+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:41.257244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.24","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217334+0000","last_change":"2024-07-17T08:41:30.239790+0000","last_active":"2024-07-17T08:41:31.217334+0000","last_peered":"2024-07-17T08:41:31.217334+0000","last_clean":"2024-07-17T08:41:31.217334+0000","last_became_active":"2024-07-17T08:41:30.238970+0000","last_became_peered":"2024-07-17T08:41:30.238970+0000","last_unstale":"2024-07-17T08:41:31.217334+0000","last_undegraded":"2024-07-17T08:41:31.217334+0000","last_fullsized":"2024-07-17T08:41:31.217334+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:30.966488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.23","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546855+0000","last_change":"2024-07-17T08:41:30.253077+0000","last_active":"2024-07-17T08:41:31.546855+0000","last_peered":"2024-07-17T08:41:31.546855+0000","last_clean":"2024-07-17T08:41:31.546855+0000","last_became_active":"2024-07-17T08:41:30.252566+0000","last_became_peered":"2024-07-17T08:41:30.252566+0000","last_unstale":"2024-07-17T08:41:31.546855+0000","last_undegraded":"2024-07-17T08:41:31.546855+0000","last_fullsized":"2024-07-17T08:41:31.546855+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:33.026942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.22","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545347+0000","last_change":"2024-07-17T08:41:30.220777+0000","last_active":"2024-07-17T08:41:31.545347+0000","last_peered":"2024-07-17T08:41:31.545347+0000","last_clean":"2024-07-17T08:41:31.545347+0000","last_became_active":"2024-07-17T08:41:30.219962+0000","last_became_peered":"2024-07-17T08:41:30.219962+0000","last_unstale":"2024-07-17T08:41:31.545347+0000","last_undegraded":"2024-07-17T08:41:31.545347+0000","last_fullsized":"2024-07-17T08:41:31.545347+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:57.752364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.21","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212200+0000","last_change":"2024-07-17T08:41:30.237902+0000","last_active":"2024-07-17T08:41:31.212200+0000","last_peered":"2024-07-17T08:41:31.212200+0000","last_clean":"2024-07-17T08:41:31.212200+0000","last_became_active":"2024-07-17T08:41:30.237535+0000","last_became_peered":"2024-07-17T08:41:30.237535+0000","last_unstale":"2024-07-17T08:41:31.212200+0000","last_undegraded":"2024-07-17T08:41:31.212200+0000","last_fullsized":"2024-07-17T08:41:31.212200+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:16.197454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.20","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217595+0000","last_change":"2024-07-17T08:41:30.240551+0000","last_active":"2024-07-17T08:41:31.217595+0000","last_peered":"2024-07-17T08:41:31.217595+0000","last_clean":"2024-07-17T08:41:31.217595+0000","last_became_active":"2024-07-17T08:41:30.240274+0000","last_became_peered":"2024-07-17T08:41:30.240274+0000","last_unstale":"2024-07-17T08:41:31.217595+0000","last_undegraded":"2024-07-17T08:41:31.217595+0000","last_fullsized":"2024-07-17T08:41:31.217595+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:52.281898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.1c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217597+0000","last_change":"2024-07-17T08:41:30.239383+0000","last_active":"2024-07-17T08:41:31.217597+0000","last_peered":"2024-07-17T08:41:31.217597+0000","last_clean":"2024-07-17T08:41:31.217597+0000","last_became_active":"2024-07-17T08:41:30.238602+0000","last_became_peered":"2024-07-17T08:41:30.238602+0000","last_unstale":"2024-07-17T08:41:31.217597+0000","last_undegraded":"2024-07-17T08:41:31.217597+0000","last_fullsized":"2024-07-17T08:41:31.217597+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:38.106633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1f","version":"307'4","reported_seq":18,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217277+0000","last_change":"2024-07-17T08:41:29.207594+0000","last_active":"2024-07-17T08:41:31.217277+0000","last_peered":"2024-07-17T08:41:31.217277+0000","last_clean":"2024-07-17T08:41:31.217277+0000","last_became_active":"2024-07-17T08:41:29.206945+0000","last_became_peered":"2024-07-17T08:41:29.206945+0000","last_unstale":"2024-07-17T08:41:31.217277+0000","last_undegraded":"2024-07-17T08:41:31.217277+0000","last_fullsized":"2024-07-17T08:41:31.217277+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:08.458737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.1d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212773+0000","last_change":"2024-07-17T08:41:30.234126+0000","last_active":"2024-07-17T08:41:31.212773+0000","last_peered":"2024-07-17T08:41:31.212773+0000","last_clean":"2024-07-17T08:41:31.212773+0000","last_became_active":"2024-07-17T08:41:30.233774+0000","last_became_peered":"2024-07-17T08:41:30.233774+0000","last_unstale":"2024-07-17T08:41:31.212773+0000","last_undegraded":"2024-07-17T08:41:31.212773+0000","last_fullsized":"2024-07-17T08:41:31.212773+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:28.414468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.1e","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545518+0000","last_change":"2024-07-17T08:41:29.185379+0000","last_active":"2024-07-17T08:41:31.545518+0000","last_peered":"2024-07-17T08:41:31.545518+0000","last_clean":"2024-07-17T08:41:31.545518+0000","last_became_active":"2024-07-17T08:41:29.184582+0000","last_became_peered":"2024-07-17T08:41:29.184582+0000","last_unstale":"2024-07-17T08:41:31.545518+0000","last_undegraded":"2024-07-17T08:41:31.545518+0000","last_fullsized":"2024-07-17T08:41:31.545518+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:31:09.843310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.1e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545509+0000","last_change":"2024-07-17T08:41:30.219523+0000","last_active":"2024-07-17T08:41:31.545509+0000","last_peered":"2024-07-17T08:41:31.545509+0000","last_clean":"2024-07-17T08:41:31.545509+0000","last_became_active":"2024-07-17T08:41:30.218230+0000","last_became_peered":"2024-07-17T08:41:30.218230+0000","last_unstale":"2024-07-17T08:41:31.545509+0000","last_undegraded":"2024-07-17T08:41:31.545509+0000","last_fullsized":"2024-07-17T08:41:31.545509+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:10.714530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1d","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546299+0000","last_change":"2024-07-17T08:41:29.212291+0000","last_active":"2024-07-17T08:41:31.546299+0000","last_peered":"2024-07-17T08:41:31.546299+0000","last_clean":"2024-07-17T08:41:31.546299+0000","last_became_active":"2024-07-17T08:41:29.210999+0000","last_became_peered":"2024-07-17T08:41:29.210999+0000","last_unstale":"2024-07-17T08:41:31.546299+0000","last_undegraded":"2024-07-17T08:41:31.546299+0000","last_fullsized":"2024-07-17T08:41:31.546299+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:08.458737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.1f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547618+0000","last_change":"2024-07-17T08:41:30.251821+0000","last_active":"2024-07-17T08:41:31.547618+0000","last_peered":"2024-07-17T08:41:31.547618+0000","last_clean":"2024-07-17T08:41:31.547618+0000","last_became_active":"2024-07-17T08:41:30.251285+0000","last_became_peered":"2024-07-17T08:41:30.251285+0000","last_unstale":"2024-07-17T08:41:31.547618+0000","last_undegraded":"2024-07-17T08:41:31.547618+0000","last_fullsized":"2024-07-17T08:41:31.547618+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:18.461569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1c","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256219+0000","last_change":"2024-07-17T08:41:29.205750+0000","last_active":"2024-07-17T08:41:31.256219+0000","last_peered":"2024-07-17T08:41:31.256219+0000","last_clean":"2024-07-17T08:41:31.256219+0000","last_became_active":"2024-07-17T08:41:29.205451+0000","last_became_peered":"2024-07-17T08:41:29.205451+0000","last_unstale":"2024-07-17T08:41:31.256219+0000","last_undegraded":"2024-07-17T08:41:31.256219+0000","last_fullsized":"2024-07-17T08:41:31.256219+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:52:54.135306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.18","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217230+0000","last_change":"2024-07-17T08:41:30.247721+0000","last_active":"2024-07-17T08:41:31.217230+0000","last_peered":"2024-07-17T08:41:31.217230+0000","last_clean":"2024-07-17T08:41:31.217230+0000","last_became_active":"2024-07-17T08:41:30.247417+0000","last_became_peered":"2024-07-17T08:41:30.247417+0000","last_unstale":"2024-07-17T08:41:31.217230+0000","last_undegraded":"2024-07-17T08:41:31.217230+0000","last_fullsized":"2024-07-17T08:41:31.217230+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:00:02.046438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.1b","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547132+0000","last_change":"2024-07-17T08:41:29.202319+0000","last_active":"2024-07-17T08:41:31.547132+0000","last_peered":"2024-07-17T08:41:31.547132+0000","last_clean":"2024-07-17T08:41:31.547132+0000","last_became_active":"2024-07-17T08:41:29.202026+0000","last_became_peered":"2024-07-17T08:41:29.202026+0000","last_unstale":"2024-07-17T08:41:31.547132+0000","last_undegraded":"2024-07-17T08:41:31.547132+0000","last_fullsized":"2024-07-17T08:41:31.547132+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:33.592690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.19","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545025+0000","last_change":"2024-07-17T08:41:30.241976+0000","last_active":"2024-07-17T08:41:31.545025+0000","last_peered":"2024-07-17T08:41:31.545025+0000","last_clean":"2024-07-17T08:41:31.545025+0000","last_became_active":"2024-07-17T08:41:30.241566+0000","last_became_peered":"2024-07-17T08:41:30.241566+0000","last_unstale":"2024-07-17T08:41:31.545025+0000","last_undegraded":"2024-07-17T08:41:31.545025+0000","last_fullsized":"2024-07-17T08:41:31.545025+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:35:58.485715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.1a","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.255828+0000","last_change":"2024-07-17T08:41:29.202871+0000","last_active":"2024-07-17T08:41:31.255828+0000","last_peered":"2024-07-17T08:41:31.255828+0000","last_clean":"2024-07-17T08:41:31.255828+0000","last_became_active":"2024-07-17T08:41:29.201742+0000","last_became_peered":"2024-07-17T08:41:29.201742+0000","last_unstale":"2024-07-17T08:41:31.255828+0000","last_undegraded":"2024-07-17T08:41:31.255828+0000","last_fullsized":"2024-07-17T08:41:31.255828+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:07:59.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.1a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546025+0000","last_change":"2024-07-17T08:41:30.220746+0000","last_active":"2024-07-17T08:41:31.546025+0000","last_peered":"2024-07-17T08:41:31.546025+0000","last_clean":"2024-07-17T08:41:31.546025+0000","last_became_active":"2024-07-17T08:41:30.218128+0000","last_became_peered":"2024-07-17T08:41:30.218128+0000","last_unstale":"2024-07-17T08:41:31.546025+0000","last_undegraded":"2024-07-17T08:41:31.546025+0000","last_fullsized":"2024-07-17T08:41:31.546025+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:22:03.437246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.19","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546339+0000","last_change":"2024-07-17T08:41:29.193729+0000","last_active":"2024-07-17T08:41:31.546339+0000","last_peered":"2024-07-17T08:41:31.546339+0000","last_clean":"2024-07-17T08:41:31.546339+0000","last_became_active":"2024-07-17T08:41:29.193439+0000","last_became_peered":"2024-07-17T08:41:29.193439+0000","last_unstale":"2024-07-17T08:41:31.546339+0000","last_undegraded":"2024-07-17T08:41:31.546339+0000","last_fullsized":"2024-07-17T08:41:31.546339+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:14.237905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.1b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545128+0000","last_change":"2024-07-17T08:41:30.216823+0000","last_active":"2024-07-17T08:41:31.545128+0000","last_peered":"2024-07-17T08:41:31.545128+0000","last_clean":"2024-07-17T08:41:31.545128+0000","last_became_active":"2024-07-17T08:41:30.215858+0000","last_became_peered":"2024-07-17T08:41:30.215858+0000","last_unstale":"2024-07-17T08:41:31.545128+0000","last_undegraded":"2024-07-17T08:41:31.545128+0000","last_fullsized":"2024-07-17T08:41:31.545128+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:46:28.543985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.18","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546628+0000","last_change":"2024-07-17T08:41:29.208196+0000","last_active":"2024-07-17T08:41:31.546628+0000","last_peered":"2024-07-17T08:41:31.546628+0000","last_clean":"2024-07-17T08:41:31.546628+0000","last_became_active":"2024-07-17T08:41:29.207232+0000","last_became_peered":"2024-07-17T08:41:29.207232+0000","last_unstale":"2024-07-17T08:41:31.546628+0000","last_undegraded":"2024-07-17T08:41:31.546628+0000","last_fullsized":"2024-07-17T08:41:31.546628+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:44.922862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.14","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548364+0000","last_change":"2024-07-17T08:41:30.236896+0000","last_active":"2024-07-17T08:41:31.548364+0000","last_peered":"2024-07-17T08:41:31.548364+0000","last_clean":"2024-07-17T08:41:31.548364+0000","last_became_active":"2024-07-17T08:41:30.234420+0000","last_became_peered":"2024-07-17T08:41:30.234420+0000","last_unstale":"2024-07-17T08:41:31.548364+0000","last_undegraded":"2024-07-17T08:41:31.548364+0000","last_fullsized":"2024-07-17T08:41:31.548364+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:04:16.236674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.17","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256633+0000","last_change":"2024-07-17T08:41:29.202037+0000","last_active":"2024-07-17T08:41:31.256633+0000","last_peered":"2024-07-17T08:41:31.256633+0000","last_clean":"2024-07-17T08:41:31.256633+0000","last_became_active":"2024-07-17T08:41:29.201706+0000","last_became_peered":"2024-07-17T08:41:29.201706+0000","last_unstale":"2024-07-17T08:41:31.256633+0000","last_undegraded":"2024-07-17T08:41:31.256633+0000","last_fullsized":"2024-07-17T08:41:31.256633+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:13.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.15","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219170+0000","last_change":"2024-07-17T08:41:30.237164+0000","last_active":"2024-07-17T08:41:31.219170+0000","last_peered":"2024-07-17T08:41:31.219170+0000","last_clean":"2024-07-17T08:41:31.219170+0000","last_became_active":"2024-07-17T08:41:30.236862+0000","last_became_peered":"2024-07-17T08:41:30.236862+0000","last_unstale":"2024-07-17T08:41:31.219170+0000","last_undegraded":"2024-07-17T08:41:31.219170+0000","last_fullsized":"2024-07-17T08:41:31.219170+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:15.629782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.16","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256020+0000","last_change":"2024-07-17T08:41:29.193104+0000","last_active":"2024-07-17T08:41:31.256020+0000","last_peered":"2024-07-17T08:41:31.256020+0000","last_clean":"2024-07-17T08:41:31.256020+0000","last_became_active":"2024-07-17T08:41:29.192603+0000","last_became_peered":"2024-07-17T08:41:29.192603+0000","last_unstale":"2024-07-17T08:41:31.256020+0000","last_undegraded":"2024-07-17T08:41:31.256020+0000","last_fullsized":"2024-07-17T08:41:31.256020+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:39.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.16","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217795+0000","last_change":"2024-07-17T08:41:30.244392+0000","last_active":"2024-07-17T08:41:31.217795+0000","last_peered":"2024-07-17T08:41:31.217795+0000","last_clean":"2024-07-17T08:41:31.217795+0000","last_became_active":"2024-07-17T08:41:30.243216+0000","last_became_peered":"2024-07-17T08:41:30.243216+0000","last_unstale":"2024-07-17T08:41:31.217795+0000","last_undegraded":"2024-07-17T08:41:31.217795+0000","last_fullsized":"2024-07-17T08:41:31.217795+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:29.052969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.15","version":"307'3","reported_seq":17,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547522+0000","last_change":"2024-07-17T08:41:29.205092+0000","last_active":"2024-07-17T08:41:31.547522+0000","last_peered":"2024-07-17T08:41:31.547522+0000","last_clean":"2024-07-17T08:41:31.547522+0000","last_became_active":"2024-07-17T08:41:29.204782+0000","last_became_peered":"2024-07-17T08:41:29.204782+0000","last_unstale":"2024-07-17T08:41:31.547522+0000","last_undegraded":"2024-07-17T08:41:31.547522+0000","last_fullsized":"2024-07-17T08:41:31.547522+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:06:52.646670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.17","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547386+0000","last_change":"2024-07-17T08:41:30.251580+0000","last_active":"2024-07-17T08:41:31.547386+0000","last_peered":"2024-07-17T08:41:31.547386+0000","last_clean":"2024-07-17T08:41:31.547386+0000","last_became_active":"2024-07-17T08:41:30.250887+0000","last_became_peered":"2024-07-17T08:41:30.250887+0000","last_unstale":"2024-07-17T08:41:31.547386+0000","last_undegraded":"2024-07-17T08:41:31.547386+0000","last_fullsized":"2024-07-17T08:41:31.547386+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:03.578868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.14","version":"307'6","reported_seq":20,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545464+0000","last_change":"2024-07-17T08:41:29.209823+0000","last_active":"2024-07-17T08:41:31.545464+0000","last_peered":"2024-07-17T08:41:31.545464+0000","last_clean":"2024-07-17T08:41:31.545464+0000","last_became_active":"2024-07-17T08:41:29.209523+0000","last_became_peered":"2024-07-17T08:41:29.209523+0000","last_unstale":"2024-07-17T08:41:31.545464+0000","last_undegraded":"2024-07-17T08:41:31.545464+0000","last_fullsized":"2024-07-17T08:41:31.545464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:00.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.10","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.544735+0000","last_change":"2024-07-17T08:41:30.221069+0000","last_active":"2024-07-17T08:41:31.544735+0000","last_peered":"2024-07-17T08:41:31.544735+0000","last_clean":"2024-07-17T08:41:31.544735+0000","last_became_active":"2024-07-17T08:41:30.220224+0000","last_became_peered":"2024-07-17T08:41:30.220224+0000","last_unstale":"2024-07-17T08:41:31.544735+0000","last_undegraded":"2024-07-17T08:41:31.544735+0000","last_fullsized":"2024-07-17T08:41:31.544735+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:07:41.955008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.13","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547415+0000","last_change":"2024-07-17T08:41:29.207619+0000","last_active":"2024-07-17T08:41:31.547415+0000","last_peered":"2024-07-17T08:41:31.547415+0000","last_clean":"2024-07-17T08:41:31.547415+0000","last_became_active":"2024-07-17T08:41:29.206624+0000","last_became_peered":"2024-07-17T08:41:29.206624+0000","last_unstale":"2024-07-17T08:41:31.547415+0000","last_undegraded":"2024-07-17T08:41:31.547415+0000","last_fullsized":"2024-07-17T08:41:31.547415+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:13.583701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.11","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546479+0000","last_change":"2024-07-17T08:41:30.235697+0000","last_active":"2024-07-17T08:41:31.546479+0000","last_peered":"2024-07-17T08:41:31.546479+0000","last_clean":"2024-07-17T08:41:31.546479+0000","last_became_active":"2024-07-17T08:41:30.234429+0000","last_became_peered":"2024-07-17T08:41:30.234429+0000","last_unstale":"2024-07-17T08:41:31.546479+0000","last_undegraded":"2024-07-17T08:41:31.546479+0000","last_fullsized":"2024-07-17T08:41:31.546479+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:21.017995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"41.12","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545621+0000","last_change":"2024-07-17T08:41:29.209987+0000","last_active":"2024-07-17T08:41:31.545621+0000","last_peered":"2024-07-17T08:41:31.545621+0000","last_clean":"2024-07-17T08:41:31.545621+0000","last_became_active":"2024-07-17T08:41:29.208171+0000","last_became_peered":"2024-07-17T08:41:29.208171+0000","last_unstale":"2024-07-17T08:41:31.545621+0000","last_undegraded":"2024-07-17T08:41:31.545621+0000","last_fullsized":"2024-07-17T08:41:31.545621+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:12:49.381673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.12","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218287+0000","last_change":"2024-07-17T08:41:30.236119+0000","last_active":"2024-07-17T08:41:31.218287+0000","last_peered":"2024-07-17T08:41:31.218287+0000","last_clean":"2024-07-17T08:41:31.218287+0000","last_became_active":"2024-07-17T08:41:30.235828+0000","last_became_peered":"2024-07-17T08:41:30.235828+0000","last_unstale":"2024-07-17T08:41:31.218287+0000","last_undegraded":"2024-07-17T08:41:31.218287+0000","last_fullsized":"2024-07-17T08:41:31.218287+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:06.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.11","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545452+0000","last_change":"2024-07-17T08:41:29.210820+0000","last_active":"2024-07-17T08:41:31.545452+0000","last_peered":"2024-07-17T08:41:31.545452+0000","last_clean":"2024-07-17T08:41:31.545452+0000","last_became_active":"2024-07-17T08:41:29.210276+0000","last_became_peered":"2024-07-17T08:41:29.210276+0000","last_unstale":"2024-07-17T08:41:31.545452+0000","last_undegraded":"2024-07-17T08:41:31.545452+0000","last_fullsized":"2024-07-17T08:41:31.545452+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:59.985686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.13","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547288+0000","last_change":"2024-07-17T08:41:30.250985+0000","last_active":"2024-07-17T08:41:31.547288+0000","last_peered":"2024-07-17T08:41:31.547288+0000","last_clean":"2024-07-17T08:41:31.547288+0000","last_became_active":"2024-07-17T08:41:30.249950+0000","last_became_peered":"2024-07-17T08:41:30.249950+0000","last_unstale":"2024-07-17T08:41:31.547288+0000","last_undegraded":"2024-07-17T08:41:31.547288+0000","last_fullsized":"2024-07-17T08:41:31.547288+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:09:09.804315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.10","version":"307'8","reported_seq":22,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217242+0000","last_change":"2024-07-17T08:41:29.211608+0000","last_active":"2024-07-17T08:41:31.217242+0000","last_peered":"2024-07-17T08:41:31.217242+0000","last_clean":"2024-07-17T08:41:31.217242+0000","last_became_active":"2024-07-17T08:41:29.211144+0000","last_became_peered":"2024-07-17T08:41:29.211144+0000","last_unstale":"2024-07-17T08:41:31.217242+0000","last_undegraded":"2024-07-17T08:41:31.217242+0000","last_fullsized":"2024-07-17T08:41:31.217242+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:13.583701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.c","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217669+0000","last_change":"2024-07-17T08:41:30.238323+0000","last_active":"2024-07-17T08:41:31.217669+0000","last_peered":"2024-07-17T08:41:31.217669+0000","last_clean":"2024-07-17T08:41:31.217669+0000","last_became_active":"2024-07-17T08:41:30.236904+0000","last_became_peered":"2024-07-17T08:41:30.236904+0000","last_unstale":"2024-07-17T08:41:31.217669+0000","last_undegraded":"2024-07-17T08:41:31.217669+0000","last_fullsized":"2024-07-17T08:41:31.217669+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:43:33.474235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.f","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546759+0000","last_change":"2024-07-17T08:41:29.198973+0000","last_active":"2024-07-17T08:41:31.546759+0000","last_peered":"2024-07-17T08:41:31.546759+0000","last_clean":"2024-07-17T08:41:31.546759+0000","last_became_active":"2024-07-17T08:41:29.198583+0000","last_became_peered":"2024-07-17T08:41:29.198583+0000","last_unstale":"2024-07-17T08:41:31.546759+0000","last_undegraded":"2024-07-17T08:41:31.546759+0000","last_fullsized":"2024-07-17T08:41:31.546759+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:24:42.232001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.d","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545997+0000","last_change":"2024-07-17T08:41:30.246129+0000","last_active":"2024-07-17T08:41:31.545997+0000","last_peered":"2024-07-17T08:41:31.545997+0000","last_clean":"2024-07-17T08:41:31.545997+0000","last_became_active":"2024-07-17T08:41:30.245848+0000","last_became_peered":"2024-07-17T08:41:30.245848+0000","last_unstale":"2024-07-17T08:41:31.545997+0000","last_undegraded":"2024-07-17T08:41:31.545997+0000","last_fullsized":"2024-07-17T08:41:31.545997+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:57.819939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.e","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547042+0000","last_change":"2024-07-17T08:41:29.208552+0000","last_active":"2024-07-17T08:41:31.547042+0000","last_peered":"2024-07-17T08:41:31.547042+0000","last_clean":"2024-07-17T08:41:31.547042+0000","last_became_active":"2024-07-17T08:41:29.208144+0000","last_became_peered":"2024-07-17T08:41:29.208144+0000","last_unstale":"2024-07-17T08:41:31.547042+0000","last_undegraded":"2024-07-17T08:41:31.547042+0000","last_fullsized":"2024-07-17T08:41:31.547042+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:17.124174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.e","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212931+0000","last_change":"2024-07-17T08:41:30.228596+0000","last_active":"2024-07-17T08:41:31.212931+0000","last_peered":"2024-07-17T08:41:31.212931+0000","last_clean":"2024-07-17T08:41:31.212931+0000","last_became_active":"2024-07-17T08:41:30.227986+0000","last_became_peered":"2024-07-17T08:41:30.227986+0000","last_unstale":"2024-07-17T08:41:31.212931+0000","last_undegraded":"2024-07-17T08:41:31.212931+0000","last_fullsized":"2024-07-17T08:41:31.212931+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:05:35.777773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"41.d","version":"307'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547919+0000","last_change":"2024-07-17T08:41:29.201902+0000","last_active":"2024-07-17T08:41:31.547919+0000","last_peered":"2024-07-17T08:41:31.547919+0000","last_clean":"2024-07-17T08:41:31.547919+0000","last_became_active":"2024-07-17T08:41:29.201278+0000","last_became_peered":"2024-07-17T08:41:29.201278+0000","last_unstale":"2024-07-17T08:41:31.547919+0000","last_undegraded":"2024-07-17T08:41:31.547919+0000","last_fullsized":"2024-07-17T08:41:31.547919+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:21.014391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.f","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219478+0000","last_change":"2024-07-17T08:41:30.237892+0000","last_active":"2024-07-17T08:41:31.219478+0000","last_peered":"2024-07-17T08:41:31.219478+0000","last_clean":"2024-07-17T08:41:31.219478+0000","last_became_active":"2024-07-17T08:41:30.237106+0000","last_became_peered":"2024-07-17T08:41:30.237106+0000","last_unstale":"2024-07-17T08:41:31.219478+0000","last_undegraded":"2024-07-17T08:41:31.219478+0000","last_fullsized":"2024-07-17T08:41:31.219478+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:48:03.757560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.c","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217749+0000","last_change":"2024-07-17T08:41:29.206513+0000","last_active":"2024-07-17T08:41:31.217749+0000","last_peered":"2024-07-17T08:41:31.217749+0000","last_clean":"2024-07-17T08:41:31.217749+0000","last_became_active":"2024-07-17T08:41:29.206230+0000","last_became_peered":"2024-07-17T08:41:29.206230+0000","last_unstale":"2024-07-17T08:41:31.217749+0000","last_undegraded":"2024-07-17T08:41:31.217749+0000","last_fullsized":"2024-07-17T08:41:31.217749+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:20.995346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"42.8","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217811+0000","last_change":"2024-07-17T08:41:30.251934+0000","last_active":"2024-07-17T08:41:31.217811+0000","last_peered":"2024-07-17T08:41:31.217811+0000","last_clean":"2024-07-17T08:41:31.217811+0000","last_became_active":"2024-07-17T08:41:30.251635+0000","last_became_peered":"2024-07-17T08:41:30.251635+0000","last_unstale":"2024-07-17T08:41:31.217811+0000","last_undegraded":"2024-07-17T08:41:31.217811+0000","last_fullsized":"2024-07-17T08:41:31.217811+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:19:10.283195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.b","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212377+0000","last_change":"2024-07-17T08:41:29.199653+0000","last_active":"2024-07-17T08:41:31.212377+0000","last_peered":"2024-07-17T08:41:31.212377+0000","last_clean":"2024-07-17T08:41:31.212377+0000","last_became_active":"2024-07-17T08:41:29.198482+0000","last_became_peered":"2024-07-17T08:41:29.198482+0000","last_unstale":"2024-07-17T08:41:31.212377+0000","last_undegraded":"2024-07-17T08:41:31.212377+0000","last_fullsized":"2024-07-17T08:41:31.212377+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:37:23.654997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.9","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545918+0000","last_change":"2024-07-17T08:41:30.221439+0000","last_active":"2024-07-17T08:41:31.545918+0000","last_peered":"2024-07-17T08:41:31.545918+0000","last_clean":"2024-07-17T08:41:31.545918+0000","last_became_active":"2024-07-17T08:41:30.219926+0000","last_became_peered":"2024-07-17T08:41:30.219926+0000","last_unstale":"2024-07-17T08:41:31.545918+0000","last_undegraded":"2024-07-17T08:41:31.545918+0000","last_fullsized":"2024-07-17T08:41:31.545918+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:41:12.922573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.a","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256098+0000","last_change":"2024-07-17T08:41:29.195413+0000","last_active":"2024-07-17T08:41:31.256098+0000","last_peered":"2024-07-17T08:41:31.256098+0000","last_clean":"2024-07-17T08:41:31.256098+0000","last_became_active":"2024-07-17T08:41:29.192565+0000","last_became_peered":"2024-07-17T08:41:29.192565+0000","last_unstale":"2024-07-17T08:41:31.256098+0000","last_undegraded":"2024-07-17T08:41:31.256098+0000","last_fullsized":"2024-07-17T08:41:31.256098+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:05.997918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.a","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546768+0000","last_change":"2024-07-17T08:41:30.244895+0000","last_active":"2024-07-17T08:41:31.546768+0000","last_peered":"2024-07-17T08:41:31.546768+0000","last_clean":"2024-07-17T08:41:31.546768+0000","last_became_active":"2024-07-17T08:41:30.244599+0000","last_became_peered":"2024-07-17T08:41:30.244599+0000","last_unstale":"2024-07-17T08:41:31.546768+0000","last_undegraded":"2024-07-17T08:41:31.546768+0000","last_fullsized":"2024-07-17T08:41:31.546768+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:54:30.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.9","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546653+0000","last_change":"2024-07-17T08:41:29.198505+0000","last_active":"2024-07-17T08:41:31.546653+0000","last_peered":"2024-07-17T08:41:31.546653+0000","last_clean":"2024-07-17T08:41:31.546653+0000","last_became_active":"2024-07-17T08:41:29.198173+0000","last_became_peered":"2024-07-17T08:41:29.198173+0000","last_unstale":"2024-07-17T08:41:31.546653+0000","last_undegraded":"2024-07-17T08:41:31.546653+0000","last_fullsized":"2024-07-17T08:41:31.546653+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:48:27.066458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"42.b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256744+0000","last_change":"2024-07-17T08:41:30.219305+0000","last_active":"2024-07-17T08:41:31.256744+0000","last_peered":"2024-07-17T08:41:31.256744+0000","last_clean":"2024-07-17T08:41:31.256744+0000","last_became_active":"2024-07-17T08:41:30.218957+0000","last_became_peered":"2024-07-17T08:41:30.218957+0000","last_unstale":"2024-07-17T08:41:31.256744+0000","last_undegraded":"2024-07-17T08:41:31.256744+0000","last_fullsized":"2024-07-17T08:41:31.256744+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:26.521135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.8","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546940+0000","last_change":"2024-07-17T08:41:29.207133+0000","last_active":"2024-07-17T08:41:31.546940+0000","last_peered":"2024-07-17T08:41:31.546940+0000","last_clean":"2024-07-17T08:41:31.546940+0000","last_became_active":"2024-07-17T08:41:29.206797+0000","last_became_peered":"2024-07-17T08:41:29.206797+0000","last_unstale":"2024-07-17T08:41:31.546940+0000","last_undegraded":"2024-07-17T08:41:31.546940+0000","last_fullsized":"2024-07-17T08:41:31.546940+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:28.578970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"42.4","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.256233+0000","last_change":"2024-07-17T08:41:30.224810+0000","last_active":"2024-07-17T08:41:31.256233+0000","last_peered":"2024-07-17T08:41:31.256233+0000","last_clean":"2024-07-17T08:41:31.256233+0000","last_became_active":"2024-07-17T08:41:30.224211+0000","last_became_peered":"2024-07-17T08:41:30.224211+0000","last_unstale":"2024-07-17T08:41:31.256233+0000","last_undegraded":"2024-07-17T08:41:31.256233+0000","last_fullsized":"2024-07-17T08:41:31.256233+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:13.725228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"41.7","version":"307'2","reported_seq":16,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547165+0000","last_change":"2024-07-17T08:41:29.205986+0000","last_active":"2024-07-17T08:41:31.547165+0000","last_peered":"2024-07-17T08:41:31.547165+0000","last_clean":"2024-07-17T08:41:31.547165+0000","last_became_active":"2024-07-17T08:41:29.205626+0000","last_became_peered":"2024-07-17T08:41:29.205626+0000","last_unstale":"2024-07-17T08:41:31.547165+0000","last_undegraded":"2024-07-17T08:41:31.547165+0000","last_fullsized":"2024-07-17T08:41:31.547165+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:52:36.744072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.5","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546114+0000","last_change":"2024-07-17T08:41:30.252458+0000","last_active":"2024-07-17T08:41:31.546114+0000","last_peered":"2024-07-17T08:41:31.546114+0000","last_clean":"2024-07-17T08:41:31.546114+0000","last_became_active":"2024-07-17T08:41:30.252176+0000","last_became_peered":"2024-07-17T08:41:30.252176+0000","last_unstale":"2024-07-17T08:41:31.546114+0000","last_undegraded":"2024-07-17T08:41:31.546114+0000","last_fullsized":"2024-07-17T08:41:31.546114+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:49:29.052969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.6","version":"308'1","reported_seq":15,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.260019+0000","last_change":"2024-07-17T08:41:29.207891+0000","last_active":"2024-07-17T08:41:31.260019+0000","last_peered":"2024-07-17T08:41:31.260019+0000","last_clean":"2024-07-17T08:41:31.260019+0000","last_became_active":"2024-07-17T08:41:29.206951+0000","last_became_peered":"2024-07-17T08:41:29.206951+0000","last_unstale":"2024-07-17T08:41:31.260019+0000","last_undegraded":"2024-07-17T08:41:31.260019+0000","last_fullsized":"2024-07-17T08:41:31.260019+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:16:11.063831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.6","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546764+0000","last_change":"2024-07-17T08:41:30.245714+0000","last_active":"2024-07-17T08:41:31.546764+0000","last_peered":"2024-07-17T08:41:31.546764+0000","last_clean":"2024-07-17T08:41:31.546764+0000","last_became_active":"2024-07-17T08:41:30.244299+0000","last_became_peered":"2024-07-17T08:41:30.244299+0000","last_unstale":"2024-07-17T08:41:31.546764+0000","last_undegraded":"2024-07-17T08:41:31.546764+0000","last_fullsized":"2024-07-17T08:41:31.546764+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:09.374417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.5","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546345+0000","last_change":"2024-07-17T08:41:29.195626+0000","last_active":"2024-07-17T08:41:31.546345+0000","last_peered":"2024-07-17T08:41:31.546345+0000","last_clean":"2024-07-17T08:41:31.546345+0000","last_became_active":"2024-07-17T08:41:29.195338+0000","last_became_peered":"2024-07-17T08:41:29.195338+0000","last_unstale":"2024-07-17T08:41:31.546345+0000","last_undegraded":"2024-07-17T08:41:31.546345+0000","last_fullsized":"2024-07-17T08:41:31.546345+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:23.083181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"42.7","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.219664+0000","last_change":"2024-07-17T08:41:30.238364+0000","last_active":"2024-07-17T08:41:31.219664+0000","last_peered":"2024-07-17T08:41:31.219664+0000","last_clean":"2024-07-17T08:41:31.219664+0000","last_became_active":"2024-07-17T08:41:30.237609+0000","last_became_peered":"2024-07-17T08:41:30.237609+0000","last_unstale":"2024-07-17T08:41:31.219664+0000","last_undegraded":"2024-07-17T08:41:31.219664+0000","last_fullsized":"2024-07-17T08:41:31.219664+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:49:41.124346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"41.4","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.548612+0000","last_change":"2024-07-17T08:41:29.182690+0000","last_active":"2024-07-17T08:41:31.548612+0000","last_peered":"2024-07-17T08:41:31.548612+0000","last_clean":"2024-07-17T08:41:31.548612+0000","last_became_active":"2024-07-17T08:41:29.181887+0000","last_became_peered":"2024-07-17T08:41:29.181887+0000","last_unstale":"2024-07-17T08:41:31.548612+0000","last_undegraded":"2024-07-17T08:41:31.548612+0000","last_fullsized":"2024-07-17T08:41:31.548612+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:41.926926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"42.0","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545128+0000","last_change":"2024-07-17T08:41:30.244387+0000","last_active":"2024-07-17T08:41:31.545128+0000","last_peered":"2024-07-17T08:41:31.545128+0000","last_clean":"2024-07-17T08:41:31.545128+0000","last_became_active":"2024-07-17T08:41:30.243646+0000","last_became_peered":"2024-07-17T08:41:30.243646+0000","last_unstale":"2024-07-17T08:41:31.545128+0000","last_undegraded":"2024-07-17T08:41:31.545128+0000","last_fullsized":"2024-07-17T08:41:31.545128+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:41:13.587744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"41.3","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212474+0000","last_change":"2024-07-17T08:41:29.199331+0000","last_active":"2024-07-17T08:41:31.212474+0000","last_peered":"2024-07-17T08:41:31.212474+0000","last_clean":"2024-07-17T08:41:31.212474+0000","last_became_active":"2024-07-17T08:41:29.198876+0000","last_became_peered":"2024-07-17T08:41:29.198876+0000","last_unstale":"2024-07-17T08:41:31.212474+0000","last_undegraded":"2024-07-17T08:41:31.212474+0000","last_fullsized":"2024-07-17T08:41:31.212474+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:38:03.989498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.1","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217311+0000","last_change":"2024-07-17T08:41:30.245521+0000","last_active":"2024-07-17T08:41:31.217311+0000","last_peered":"2024-07-17T08:41:31.217311+0000","last_clean":"2024-07-17T08:41:31.217311+0000","last_became_active":"2024-07-17T08:41:30.245253+0000","last_became_peered":"2024-07-17T08:41:30.245253+0000","last_unstale":"2024-07-17T08:41:31.217311+0000","last_undegraded":"2024-07-17T08:41:31.217311+0000","last_fullsized":"2024-07-17T08:41:31.217311+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:10:14.933853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"41.2","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.218285+0000","last_change":"2024-07-17T08:41:29.212127+0000","last_active":"2024-07-17T08:41:31.218285+0000","last_peered":"2024-07-17T08:41:31.218285+0000","last_clean":"2024-07-17T08:41:31.218285+0000","last_became_active":"2024-07-17T08:41:29.211801+0000","last_became_peered":"2024-07-17T08:41:29.211801+0000","last_unstale":"2024-07-17T08:41:31.218285+0000","last_undegraded":"2024-07-17T08:41:31.218285+0000","last_fullsized":"2024-07-17T08:41:31.218285+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:45.284157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"42.2","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.545840+0000","last_change":"2024-07-17T08:41:30.245885+0000","last_active":"2024-07-17T08:41:31.545840+0000","last_peered":"2024-07-17T08:41:31.545840+0000","last_clean":"2024-07-17T08:41:31.545840+0000","last_became_active":"2024-07-17T08:41:30.241917+0000","last_became_peered":"2024-07-17T08:41:30.241917+0000","last_unstale":"2024-07-17T08:41:31.545840+0000","last_undegraded":"2024-07-17T08:41:31.545840+0000","last_fullsized":"2024-07-17T08:41:31.545840+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:22:56.170592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"41.1","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.212456+0000","last_change":"2024-07-17T08:41:29.195963+0000","last_active":"2024-07-17T08:41:31.212456+0000","last_peered":"2024-07-17T08:41:31.212456+0000","last_clean":"2024-07-17T08:41:31.212456+0000","last_became_active":"2024-07-17T08:41:29.195222+0000","last_became_peered":"2024-07-17T08:41:29.195222+0000","last_unstale":"2024-07-17T08:41:31.212456+0000","last_undegraded":"2024-07-17T08:41:31.212456+0000","last_fullsized":"2024-07-17T08:41:31.212456+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:51.932727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"42.3","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.547213+0000","last_change":"2024-07-17T08:41:30.240156+0000","last_active":"2024-07-17T08:41:31.547213+0000","last_peered":"2024-07-17T08:41:31.547213+0000","last_clean":"2024-07-17T08:41:31.547213+0000","last_became_active":"2024-07-17T08:41:30.238063+0000","last_became_peered":"2024-07-17T08:41:30.238063+0000","last_unstale":"2024-07-17T08:41:31.547213+0000","last_undegraded":"2024-07-17T08:41:31.547213+0000","last_fullsized":"2024-07-17T08:41:31.547213+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:36.730131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"41.0","version":"0'0","reported_seq":14,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.255940+0000","last_change":"2024-07-17T08:41:29.209700+0000","last_active":"2024-07-17T08:41:31.255940+0000","last_peered":"2024-07-17T08:41:31.255940+0000","last_clean":"2024-07-17T08:41:31.255940+0000","last_became_active":"2024-07-17T08:41:29.209333+0000","last_became_peered":"2024-07-17T08:41:29.209333+0000","last_unstale":"2024-07-17T08:41:31.255940+0000","last_undegraded":"2024-07-17T08:41:31.255940+0000","last_fullsized":"2024-07-17T08:41:31.255940+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:28.149073+0000","last_clean_scrub_stamp":"2024-07-17T08:41:28.149073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:58.758378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"42.2b","version":"0'0","reported_seq":12,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.546923+0000","last_change":"2024-07-17T08:41:30.247457+0000","last_active":"2024-07-17T08:41:31.546923+0000","last_peered":"2024-07-17T08:41:31.546923+0000","last_clean":"2024-07-17T08:41:31.546923+0000","last_became_active":"2024-07-17T08:41:30.247176+0000","last_became_peered":"2024-07-17T08:41:30.247176+0000","last_unstale":"2024-07-17T08:41:31.546923+0000","last_undegraded":"2024-07-17T08:41:31.546923+0000","last_fullsized":"2024-07-17T08:41:31.546923+0000","mapping_epoch":306,"log_start":"0'0","ondisk_log_start":"0'0","created":306,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:41:29.159080+0000","last_clean_scrub_stamp":"2024-07-17T08:41:29.159080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:07:29.319225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"299'4610","reported_seq":5829,"reported_epoch":308,"state":"active+clean","last_fresh":"2024-07-17T08:41:31.217762+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:41:31.217762+0000","last_peered":"2024-07-17T08:41:31.217762+0000","last_clean":"2024-07-17T08:41:31.217762+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:41:31.217762+0000","last_undegraded":"2024-07-17T08:41:31.217762+0000","last_fullsized":"2024-07-17T08:41:31.217762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4610,"log_dups_size":0,"ondisk_log_size":4610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":469,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":42,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":41,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1969,"num_read_kb":5558,"num_write":469,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4610,"ondisk_log_size":4610,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75840,"kb_used_data":4404,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296000,"statfs":{"total":96636764160,"available":96559104000,"internally_reserved":0,"allocated":4509696,"data_stored":2743825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510705,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53036,"kb_used_data":3808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318804,"statfs":{"total":96636764160,"available":96582455296,"internally_reserved":0,"allocated":3899392,"data_stored":2146071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510706,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71120,"kb_used_data":3836,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300720,"statfs":{"total":96636764160,"available":96563937280,"internally_reserved":0,"allocated":3928064,"data_stored":2155013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543411,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53052,"kb_used_data":3824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318788,"statfs":{"total":96636764160,"available":96582438912,"internally_reserved":0,"allocated":3915776,"data_stored":2153401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510705,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57212,"kb_used_data":3820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314628,"statfs":{"total":96636764160,"available":96578179072,"internally_reserved":0,"allocated":3911680,"data_stored":2153367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543409,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57224,"kb_used_data":3836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314616,"statfs":{"total":96636764160,"available":96578166784,"internally_reserved":0,"allocated":3928064,"data_stored":2155001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510706,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53068,"kb_used_data":3832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318772,"statfs":{"total":96636764160,"available":96582422528,"internally_reserved":0,"allocated":3923968,"data_stored":2154041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510705,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53644,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318196,"statfs":{"total":96636764160,"available":96581832704,"internally_reserved":0,"allocated":4517888,"data_stored":2744477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":41,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":42,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:41:35.181 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:41:35.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:41:35.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 -- 172.21.15.44:0/3044736833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 msgr2=0x7f10c40b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 --2- 172.21.15.44:0/3044736833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c40b32e0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f10c4070830 tx=0x7f10bc02aac0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 -- 172.21.15.44:0/3044736833 shutdown_connections 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 --2- 172.21.15.44:0/3044736833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c40b32e0 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 --2- 172.21.15.44:0/3044736833 >> v2:172.21.15.175:3300/0 conn(0x7f10c406f4d0 0x7f10c415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 -- 172.21.15.44:0/3044736833 >> 172.21.15.44:0/3044736833 conn(0x7f10c40cf0d0 msgr2=0x7f10c40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 -- 172.21.15.44:0/3044736833 shutdown_connections 2024-07-17T08:41:35.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.443+0000 7f10c3e00640 1 -- 172.21.15.44:0/3044736833 wait complete. 2024-07-17T08:41:35.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.445+0000 7f10c3e00640 1 Processor -- start 2024-07-17T08:41:35.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.445+0000 7f10c3e00640 1 -- start start 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10c406f4d0 0x7f10c41f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 0x7f10c40cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10c406daf0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10c406d970 con 0x7f10c41503c0 2024-07-17T08:41:35.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.446+0000 7f10c3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10c406dc70 con 0x7f10c406f4d0 2024-07-17T08:41:35.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55256/0 (socket says 172.21.15.44:55256) 2024-07-17T08:41:35.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 -- 172.21.15.44:0/1450901787 learned_addr learned my addr 172.21.15.44:0/1450901787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:35.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c3400640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 0x7f10c40cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 -- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10c406f4d0 msgr2=0x7f10c41f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2a00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10c406f4d0 0x7f10c41f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10c406f4d0 0x7f10c41f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 -- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 msgr2=0x7f10c40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 0x7f10c40cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 -- 172.21.15.44:0/1450901787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c41431b0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c3400640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 0x7f10c40cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.447+0000 7f10c2000640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c450 tx=0x7f10b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.448+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b00052d0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.448+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b0005470 con 0x7f10c40b2eb0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.448+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0012870 con 0x7f10c40b2eb0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.448+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c4143450 con 0x7f10c40b2eb0 2024-07-17T08:41:35.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.448+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c4143a10 con 0x7f10c40b2eb0 2024-07-17T08:41:35.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.449+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c415a450 con 0x7f10c40b2eb0 2024-07-17T08:41:35.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.450+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f10b00035f0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.451+0000 7f10bb400640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 0x7f10a40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.451+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f10b0098480 con 0x7f10c40b2eb0 2024-07-17T08:41:35.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.452+0000 7f10c2a00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 0x7f10a40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.453+0000 7f10c2a00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 0x7f10a40791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005fd0 tx=0x7f10b4005bc0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:35.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.455+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f10b00613b0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.743+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f10c415a6f0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.748+0000 7f10bb400640 1 -- 172.21.15.44:0/1450901787 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v78) ==== 190+0+147 (secure 0 0 0) 0x7f10b0060cb0 con 0x7f10c40b2eb0 2024-07-17T08:41:35.749 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:41:35.749 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:41:35.749 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:41:35.749 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:41:35.750 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:41:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:41:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 msgr2=0x7f10a40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 0x7f10a40791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005fd0 tx=0x7f10b4005bc0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 msgr2=0x7f10c41f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c450 tx=0x7f10b000c910 comp rx=0 tx=0).stop 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 shutdown_connections 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10a4076d00 0x7f10a40791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10c41503c0 0x7f10c40cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10c40b2eb0 0x7f10c41f29f0 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 --2- 172.21.15.44:0/1450901787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10c406f4d0 0x7f10c41f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.751+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 >> 172.21.15.44:0/1450901787 conn(0x7f10c40cf0d0 msgr2=0x7f10c41535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.752+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 shutdown_connections 2024-07-17T08:41:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.752+0000 7f10c3e00640 1 -- 172.21.15.44:0/1450901787 wait complete. 2024-07-17T08:41:35.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:35.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 -- 172.21.15.44:0/4179277030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 msgr2=0x7f10f80ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 --2- 172.21.15.44:0/4179277030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f80ac760 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005fd0 tx=0x7f10f002a9c0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 -- 172.21.15.44:0/4179277030 shutdown_connections 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 --2- 172.21.15.44:0/4179277030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f80ac760 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 --2- 172.21.15.44:0/4179277030 >> v2:172.21.15.175:3300/0 conn(0x7f10f806ea50 0x7f10f80ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 -- 172.21.15.44:0/4179277030 >> 172.21.15.44:0/4179277030 conn(0x7f10f80b0fb0 msgr2=0x7f10f80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 -- 172.21.15.44:0/4179277030 shutdown_connections 2024-07-17T08:41:35.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.978+0000 7f10ff200640 1 -- 172.21.15.44:0/4179277030 wait complete. 2024-07-17T08:41:35.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.980+0000 7f10ff200640 1 Processor -- start 2024-07-17T08:41:35.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.981+0000 7f10ff200640 1 -- start start 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f806ea50 0x7f10f81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 0x7f10f809b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80b2750 con 0x7f10f806eec0 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80b25d0 con 0x7f10f806ea50 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10f80b28d0 con 0x7f10f80ae810 2024-07-17T08:41:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55278/0 (socket says 172.21.15.44:55278) 2024-07-17T08:41:35.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.982+0000 7f10f7400640 1 -- 172.21.15.44:0/3282719917 learned_addr learned my addr 172.21.15.44:0/3282719917 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7e00640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f806ea50 0x7f10f81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10fd200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 0x7f10f809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 -- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 msgr2=0x7f10f809b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 0x7f10f809b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 -- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f806ea50 msgr2=0x7f10f81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f806ea50 0x7f10f81577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 -- 172.21.15.44:0/3282719917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f8156ae0 con 0x7f10f806eec0 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10fd200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 0x7f10f809b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.983+0000 7f10f7400640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d7a0 tx=0x7f10e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.984+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4017070 con 0x7f10f806eec0 2024-07-17T08:41:35.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.984+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10e4009d20 con 0x7f10f806eec0 2024-07-17T08:41:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.984+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4005280 con 0x7f10f806eec0 2024-07-17T08:41:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.984+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f8156dc0 con 0x7f10f806eec0 2024-07-17T08:41:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.984+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f8157380 con 0x7f10f806eec0 2024-07-17T08:41:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.985+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f8157580 con 0x7f10f806eec0 2024-07-17T08:41:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.986+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f10e4005420 con 0x7f10f806eec0 2024-07-17T08:41:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.987+0000 7f10efe00640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 0x7f10d8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.988+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f10e4098b20 con 0x7f10f806eec0 2024-07-17T08:41:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.988+0000 7f10f7e00640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 0x7f10d8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:35.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.989+0000 7f10f7e00640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 0x7f10d8079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8008040 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:35.992+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f10e4061940 con 0x7f10f806eec0 2024-07-17T08:41:36.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.240+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10f80b2e40 con 0x7f10f806eec0 2024-07-17T08:41:36.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.242+0000 7f10efe00640 1 -- 172.21.15.44:0/3282719917 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7f10e4061240 con 0x7f10f806eec0 2024-07-17T08:41:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:36.243 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:36.243 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.244+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 msgr2=0x7f10d8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.244+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 0x7f10d8079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8008040 comp rx=0 tx=0).stop 2024-07-17T08:41:36.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.244+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 msgr2=0x7f10f81f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.244+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d7a0 tx=0x7f10e400dc60 comp rx=0 tx=0).stop 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 shutdown_connections 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f10d8076cd0 0x7f10d8079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10f80ae810 0x7f10f809b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10f806eec0 0x7f10f81f3880 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 --2- 172.21.15.44:0/3282719917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10f806ea50 0x7f10f81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 >> 172.21.15.44:0/3282719917 conn(0x7f10f80b0fb0 msgr2=0x7f10f80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 shutdown_connections 2024-07-17T08:41:36.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.245+0000 7f10ff200640 1 -- 172.21.15.44:0/3282719917 wait complete. 2024-07-17T08:41:36.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:41:29.658222+0000', 'modified': '2024-07-17T08:41:32.729647+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12654}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12654': {'gid': 12654, 'name': 'd', 'rank': 0, 'incarnation': 169, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6801/1168128599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1168128599}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 1168128599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [42], 'metadata_pool': 41, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12654, 'qdb_cluster': [12654]} 2024-07-17T08:41:36.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:41:36.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.895+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2168125285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c900050e0 msgr2=0x7f9c90005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.895+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2168125285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c900050e0 0x7f9c90005910 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009b50 tx=0x7f9c90006940 comp rx=0 tx=0).stop 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.895+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2168125285 reap_dead start 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.896+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2168125285 shutdown_connections 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.896+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2168125285 >> 172.21.15.44:0/2168125285 conn(0x7f9cac0b0eb0 msgr2=0x7f9cac0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.896+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2168125285 shutdown_connections 2024-07-17T08:41:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.896+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2168125285 wait complete. 2024-07-17T08:41:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.898+0000 7f9cb2e00640 1 Processor -- start 2024-07-17T08:41:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.898+0000 7f9cb2e00640 1 -- start start 2024-07-17T08:41:36.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c900050e0 0x7f9cac1e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 0x7f9cac0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac0ae560 con 0x7f9cac071e80 2024-07-17T08:41:36.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac15a8f0 con 0x7f9c900050e0 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac0ae6e0 con 0x7f9cac1f7bc0 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55296/0 (socket says 172.21.15.44:55296) 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2668003591 learned_addr learned my addr 172.21.15.44:0/2668003591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cabe00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c900050e0 0x7f9cac1e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cab400640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 0x7f9cac0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 msgr2=0x7f9cac0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 0x7f9cac0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c900050e0 msgr2=0x7f9cac1e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c900050e0 0x7f9cac1e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2668003591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac1f8080 con 0x7f9cac071e80 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.899+0000 7f9cab400640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 0x7f9cac0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:36.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9ca3e00640 1 -- 172.21.15.44:0/2668003591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4004b20 con 0x7f9cac071e80 2024-07-17T08:41:36.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9ca3e00640 1 -- 172.21.15.44:0/2668003591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca4004db0 con 0x7f9cac071e80 2024-07-17T08:41:36.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9ca3e00640 1 -- 172.21.15.44:0/2668003591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400cd60 con 0x7f9cac071e80 2024-07-17T08:41:36.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac1f8360 con 0x7f9cac071e80 2024-07-17T08:41:36.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.900+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac1f8920 con 0x7f9cac071e80 2024-07-17T08:41:36.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.904+0000 7f9ca3e00640 1 -- 172.21.15.44:0/2668003591 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f9ca401b030 con 0x7f9cac071e80 2024-07-17T08:41:36.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.905+0000 7f9ca3e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 0x7f9c880791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.905+0000 7f9cb5d80740 1 Processor -- start 2024-07-17T08:41:36.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.905+0000 7f9ca3e00640 1 -- 172.21.15.44:0/2668003591 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f9ca4098390 con 0x7f9cac071e80 2024-07-17T08:41:36.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.905+0000 7f9cb5d80740 1 -- start start 2024-07-17T08:41:36.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.905+0000 7f9cabe00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 0x7f9c880791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d5370290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d5376420 0x5566d53707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566d52e7d60 con 0x5566d539d140 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566d52d7260 con 0x5566d53b1320 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5566d52db2a0 con 0x5566d5376420 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55298/0 (socket says 172.21.15.44:55298) 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.906+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2950069835 learned_addr learned my addr 172.21.15.44:0/2950069835 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d5376420 msgr2=0x5566d53707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d5376420 0x5566d53707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cabe00640 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d5370290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 msgr2=0x5566d5370290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d5370290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 -- 172.21.15.44:0/2950069835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d503b850 con 0x5566d539d140 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f9ca405d130 tx=0x7f9ca40602b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477c13c2a7fb8da5 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9cabe00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 0x7f9c880791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001eb0 tx=0x7f9c9c007db0 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9ca0c00640 1 -- 172.21.15.44:0/2950069835 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4060b40 con 0x5566d539d140 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9ca0c00640 1 -- 172.21.15.44:0/2950069835 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca4065140 con 0x5566d539d140 2024-07-17T08:41:36.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.907+0000 7f9ca0c00640 1 -- 172.21.15.44:0/2950069835 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4069b60 con 0x5566d539d140 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 -- 172.21.15.44:0/2950069835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 msgr2=0x5566d5370d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f9ca405d130 tx=0x7f9ca40602b0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 -- 172.21.15.44:0/2950069835 shutdown_connections 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d5370d10 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d5376420 0x5566d53707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/2950069835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d5370290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 -- 172.21.15.44:0/2950069835 >> 172.21.15.44:0/2950069835 conn(0x5566d5372280 msgr2=0x5566d5375ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 -- 172.21.15.44:0/2950069835 shutdown_connections 2024-07-17T08:41:36.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.908+0000 7f9cb5d80740 1 -- 172.21.15.44:0/2950069835 wait complete. 2024-07-17T08:41:36.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.909+0000 7f9cb5d80740 1 -- start start 2024-07-17T08:41:36.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.909+0000 7f9cb5d80740 1 Processor -- start 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d53b1320 0x5566d539e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539eec0 0x5566d53d63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566d504a620 con 0x5566d539d140 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566d527ef90 con 0x5566d539eec0 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5566d5373b70 con 0x5566d53b1320 2024-07-17T08:41:36.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cb5d80740 10 client.0 _ll_register_callbacks cb 0x5566d52a4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55310/0 (socket says 172.21.15.44:55310) 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.910+0000 7f9cabe00640 1 -- 172.21.15.44:0/1897916532 learned_addr learned my addr 172.21.15.44:0/1897916532 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cab400640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d53b1320 0x5566d539e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 -- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d53b1320 msgr2=0x5566d539e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d53b1320 0x5566d539e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539eec0 0x5566d53d63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 -- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539eec0 msgr2=0x5566d53d63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539eec0 0x5566d53d63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 -- 172.21.15.44:0/1897916532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d50b26b0 con 0x5566d539d140 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0075b0 tx=0x7f9c9c001ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e570299837b88b6b server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9c99600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:41:36.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00e760 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.911+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f9c9c020d10 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566d52ae900 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c00e900 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 10 client.12777 fetch_fsmap learned FSMap version 172 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) -- 0x5566d53d4b00 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5566d52ae900 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 172) ==== 3502+0+0 (secure 0 0 0) 0x7f9c9c02d070 con 0x5566d539d140 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 10 client.12777 fetch_fsmap finished waiting for FSMap version 172 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 10 client.12777 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 20 client.12777 populate_metadata read hostname 'smithi044' 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d539de70 0x5566d53d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 4 client.12777 mds_command: new command op to 12654 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:41:36.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.912+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 --> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566d52e9cb0 con 0x5566d539de70 2024-07-17T08:41:36.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.913+0000 7f9cab400640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d539de70 0x5566d53d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.914+0000 7f9cab400640 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d539de70 0x5566d53d5ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439ee8bc5d08c64d server_cookie=9acda3a77a326f40 in_seq=0 out_seq=0 2024-07-17T08:41:36.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.914+0000 7f9c99600640 10 client.12777 ms_handle_connect on v2:172.21.15.175:6800/1168128599 2024-07-17T08:41:36.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.917+0000 7f9c99600640 1 -- 172.21.15.44:0/1897916532 <== mds.0 v2:172.21.15.175:6800/1168128599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566d52e9cb0 con 0x5566d539de70 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.917+0000 7f9c99600640 10 client.12777 handle_command_reply: tid=0 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.917+0000 7f9cb5d80740 1 client.12777 shutdown 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 msgr2=0x5566d53a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0075b0 tx=0x7f9c9c001ce0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 shutdown_connections 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d539de70 0x5566d53d5ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539eec0 0x5566d53d63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d53b1320 0x5566d539e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1897916532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53a3620 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9c99600640 0 client.12777 ms_handle_reset on v2:172.21.15.175:6800/1168128599 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 >> 172.21.15.44:0/1897916532 conn(0x5566d5376420 msgr2=0x5566d53bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 shutdown_connections 2024-07-17T08:41:36.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1897916532 wait complete. 2024-07-17T08:41:36.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.918+0000 7f9cb5d80740 20 client.12777 trim_cache size 0 max 16384 2024-07-17T08:41:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.938+0000 7f9cb5d80740 1 Processor -- start 2024-07-17T08:41:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.938+0000 7f9cb5d80740 1 -- start start 2024-07-17T08:41:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539d140 0x5566d543b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1560a0 con 0x5566d53b1320 2024-07-17T08:41:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1553e0 con 0x5566d539d140 2024-07-17T08:41:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac155a30 con 0x5566d539eec0 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55316/0 (socket says 172.21.15.44:55316) 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cb0e00640 1 -- 172.21.15.44:0/1131703249 learned_addr learned my addr 172.21.15.44:0/1131703249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.939+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539d140 0x5566d543b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 -- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 msgr2=0x5566d53cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 -- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539d140 msgr2=0x5566d543b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539d140 0x5566d543b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 -- 172.21.15.44:0/1131703249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d539f9c0 con 0x5566d53b1320 2024-07-17T08:41:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009a10 tx=0x7f9ca4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1b4c4a37f0a91e server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9c9be00640 1 -- 172.21.15.44:0/1131703249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4069b60 con 0x5566d53b1320 2024-07-17T08:41:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9c9be00640 1 -- 172.21.15.44:0/1131703249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca4060b40 con 0x5566d53b1320 2024-07-17T08:41:36.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.940+0000 7f9c9be00640 1 -- 172.21.15.44:0/1131703249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4065140 con 0x5566d53b1320 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1131703249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 msgr2=0x5566d53d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009a10 tx=0x7f9ca4004ab0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1131703249 shutdown_connections 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d53b1320 0x5566d53d0180 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1131703249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d539d140 0x5566d543b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1131703249 >> 172.21.15.44:0/1131703249 conn(0x5566d5376420 msgr2=0x5566d53759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1131703249 shutdown_connections 2024-07-17T08:41:36.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.941+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1131703249 wait complete. 2024-07-17T08:41:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.942+0000 7f9cb5d80740 1 -- start start 2024-07-17T08:41:36.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.942+0000 7f9cb5d80740 1 Processor -- start 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d53a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5566d539dcf0 con 0x5566d539d140 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566d539a400 con 0x5566d53b1320 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5566d539e2b0 con 0x5566d539eec0 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cb5d80740 10 client.0 _ll_register_callbacks cb 0x7f9c9c001b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55332/0 (socket says 172.21.15.44:55332) 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.943+0000 7f9cabe00640 1 -- 172.21.15.44:0/1671069502 learned_addr learned my addr 172.21.15.44:0/1671069502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:36.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 -- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 msgr2=0x5566d53bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cb0e00640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d53a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 -- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 msgr2=0x5566d53a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d53a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 -- 172.21.15.44:0/1671069502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d5373930 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cabe00640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00f010 tx=0x7f9c9c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55653c4cfd203373 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9c99600640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c02f490 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f9c9c01f070 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.944+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca4060b40 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c03c050 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 10 client.12783 fetch_fsmap learned FSMap version 172 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) -- 0x5566d53dccd0 con 0x5566d539d140 2024-07-17T08:41:36.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca4060b40 con 0x5566d539d140 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 172) ==== 3502+0+0 (secure 0 0 0) 0x7f9c9c03f020 con 0x5566d539d140 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 10 client.12783 fetch_fsmap finished waiting for FSMap version 172 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 10 client.12783 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 20 client.12783 populate_metadata read hostname 'smithi044' 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d536f630 0x5566d536fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 4 client.12783 mds_command: new command op to 12654 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:41:36.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.945+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 --> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] -- command(tid 0: {"prefix": "status"}) -- 0x5566d52e5750 con 0x5566d536f630 2024-07-17T08:41:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.946+0000 7f9cab400640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d536f630 0x5566d536fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.947+0000 7f9cab400640 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d536f630 0x5566d536fa10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fd281864b4db1d server_cookie=f67bfdb732b00c83 in_seq=0 out_seq=0 2024-07-17T08:41:36.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.947+0000 7f9c99600640 10 client.12783 ms_handle_connect on v2:172.21.15.175:6800/1168128599 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.947+0000 7f9c99600640 1 -- 172.21.15.44:0/1671069502 <== mds.0 v2:172.21.15.175:6800/1168128599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x5566d52e5750 con 0x5566d536f630 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.947+0000 7f9c99600640 10 client.12783 handle_command_reply: tid=0 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.947+0000 7f9cb5d80740 1 client.12783 shutdown 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 msgr2=0x5566d53bc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00f010 tx=0x7f9c9c004830 comp rx=0 tx=0).stop 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 shutdown_connections 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:6800/1168128599,v1:172.21.15.175:6801/1168128599] conn(0x5566d536f630 0x5566d536fa10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d53b1320 0x5566d53a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5566d539eec0 0x5566d53bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 --2- 172.21.15.44:0/1671069502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5566d539d140 0x5566d53bc750 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9c99600640 0 client.12783 ms_handle_reset on v2:172.21.15.175:6800/1168128599 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 >> 172.21.15.44:0/1671069502 conn(0x5566d5376420 msgr2=0x5566d53759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 shutdown_connections 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 1 -- 172.21.15.44:0/1671069502 wait complete. 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.948+0000 7f9cb5d80740 20 client.12783 trim_cache size 0 max 16384 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 12654, 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:41:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7191307269999996, 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 172, 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 308, 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 308, 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.684764269 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 msgr2=0x7f9c880791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 0x7f9c880791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001eb0 tx=0x7f9c9c007db0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 msgr2=0x7f9cac0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005fd0 tx=0x7f9ca400ee10 comp rx=0 tx=0).stop 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 shutdown_connections 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f9c88076d00 0x7f9c880791c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac071e80 0x7f9cac0cfb70 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cac1f7bc0 0x7f9cac0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 --2- 172.21.15.44:0/2668003591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c900050e0 0x7f9cac1e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 >> 172.21.15.44:0/2668003591 conn(0x7f9cac0b0eb0 msgr2=0x7f9cac15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 shutdown_connections 2024-07-17T08:41:36.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:36.949+0000 7f9cb2e00640 1 -- 172.21.15.44:0/2668003591 wait complete. 2024-07-17T08:41:36.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.166+0000 7f48c4e00640 1 -- 172.21.15.44:0/924926189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c00b1fa0 msgr2=0x7f48c00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.166+0000 7f48c4e00640 1 --2- 172.21.15.44:0/924926189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c00b1fa0 0x7f48c00cf2d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005fd0 tx=0x7f48a802a540 comp rx=0 tx=0).stop 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 -- 172.21.15.44:0/924926189 shutdown_connections 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 --2- 172.21.15.44:0/924926189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c00b1fa0 0x7f48c00cf2d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 --2- 172.21.15.44:0/924926189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 -- 172.21.15.44:0/924926189 >> 172.21.15.44:0/924926189 conn(0x7f48c00b0e80 msgr2=0x7f48c00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 -- 172.21.15.44:0/924926189 shutdown_connections 2024-07-17T08:41:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.167+0000 7f48c4e00640 1 -- 172.21.15.44:0/924926189 wait complete. 2024-07-17T08:41:37.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.169+0000 7f48c4e00640 1 Processor -- start 2024-07-17T08:41:37.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.169+0000 7f48c4e00640 1 -- start start 2024-07-17T08:41:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 0x7f48c015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48c01614f0 con 0x7f48c00cf810 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00ad780 con 0x7f48c0152300 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.170+0000 7f48c4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48c0161670 con 0x7f48c015d240 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55346/0 (socket says 172.21.15.44:55346) 2024-07-17T08:41:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 -- 172.21.15.44:0/3295868907 learned_addr learned my addr 172.21.15.44:0/3295868907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bea00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bfe00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 0x7f48c015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 -- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 msgr2=0x7f48c014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 -- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 msgr2=0x7f48c015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:37.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 0x7f48c015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bf400640 1 -- 172.21.15.44:0/3295868907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c0152c30 con 0x7f48c00cf810 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bea00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.171+0000 7f48bfe00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 0x7f48c015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48bf400640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e7c0 tx=0x7f48b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b000ccc0 con 0x7f48c00cf810 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b0010430 con 0x7f48c00cf810 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c01f42f0 con 0x7f48c00cf810 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0010860 con 0x7f48c00cf810 2024-07-17T08:41:37.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.172+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c01f48b0 con 0x7f48c00cf810 2024-07-17T08:41:37.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.173+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c0161b90 con 0x7f48c00cf810 2024-07-17T08:41:37.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.174+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f48b0010a00 con 0x7f48c00cf810 2024-07-17T08:41:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.175+0000 7f48b7400640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 0x7f489c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.175+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f48b0099360 con 0x7f48c00cf810 2024-07-17T08:41:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.176+0000 7f48bea00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 0x7f489c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:37.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.177+0000 7f48bea00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 0x7f489c079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48a8033000 tx=0x7f48a8004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:37.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.179+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48b0062220 con 0x7f48c00cf810 2024-07-17T08:41:37.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.435+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48c00cf480 con 0x7f48c00cf810 2024-07-17T08:41:37.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.436+0000 7f48b7400640 1 -- 172.21.15.44:0/3295868907 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v172) ==== 53+0+73 (secure 0 0 0) 0x7f48b0061b20 con 0x7f48c00cf810 2024-07-17T08:41:37.437 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 msgr2=0x7f489c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 0x7f489c079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48a8033000 tx=0x7f48a8004730 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 msgr2=0x7f48c014eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e7c0 tx=0x7f48b000ec80 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 shutdown_connections 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f489c076cd0 0x7f489c079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48c0152300 0x7f48c015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48c015d240 0x7f48c014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 --2- 172.21.15.44:0/3295868907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48c00cf810 0x7f48c014eb30 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:37.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 >> 172.21.15.44:0/3295868907 conn(0x7f48c00b0e80 msgr2=0x7f48c009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:37.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 shutdown_connections 2024-07-17T08:41:37.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37.439+0000 7f48c4e00640 1 -- 172.21.15.44:0/3295868907 wait complete. 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:41:37.460 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:41:37.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:37.461 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:41:37.511 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:41:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:41:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:41:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:41:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.512 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:41:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36160sec preferred_lft 36160sec 2024-07-17T08:41:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:41:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:41:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:41:37.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:37.516 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:41:37.594 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:41:37.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:37.599 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:41:37.651 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:41:37.651 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:41:37.651 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:41:37.651 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:41:37.651 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:41:37.652 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:41:37.652 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:41:37.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:41:37.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:41:37.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:41:37.653 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:41:37.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:41:37.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:41:37.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:37.871 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:41:37.919 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:41:37.919 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:41:37.919 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:41:37.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:37.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:41:37.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:41:37.995 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:38.048 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:41:38.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:41:38.048 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:38.100 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:41:38.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:38.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:41:38.234 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:41:38.234 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:41:38.234 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:41:38.257 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:41:38.257 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:41:38.258 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:41:38.258 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:41:38.258 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:41:38.285 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:41:38.287 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:41:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:41:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:41:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:41:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:41:38.291 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:41:38.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:38.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:38.361 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:41:38.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:41:38.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:41:38.502 INFO:teuthology.orchestra.run.smithi044.stdout:client9173 2024-07-17T08:41:38.503 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:38.503 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:38.554 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:41:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 -- 172.21.15.44:0/2004558240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415d130 msgr2=0x7f2a7415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2004558240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415d130 0x7f2a7415d510 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a6402a4a0 comp rx=0 tx=0).stop 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 -- 172.21.15.44:0/2004558240 shutdown_connections 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2004558240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a740ce690 0x7f2a740cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2004558240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7415d130 0x7f2a7415d510 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.769+0000 7f2a78a00640 1 -- 172.21.15.44:0/2004558240 >> 172.21.15.44:0/2004558240 conn(0x7f2a740b4ff0 msgr2=0x7f2a740b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.770+0000 7f2a78a00640 1 -- 172.21.15.44:0/2004558240 shutdown_connections 2024-07-17T08:41:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.770+0000 7f2a78a00640 1 -- 172.21.15.44:0/2004558240 wait complete. 2024-07-17T08:41:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.772+0000 7f2a78a00640 1 Processor -- start 2024-07-17T08:41:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.772+0000 7f2a78a00640 1 -- start start 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a740ce690 0x7f2a741f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a7415d130 0x7f2a7414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740d1a60 con 0x7f2a74161be0 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740d18e0 con 0x7f2a740ce690 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.773+0000 7f2a78a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a740d1be0 con 0x7f2a7415d130 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55362/0 (socket says 172.21.15.44:55362) 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 -- 172.21.15.44:0/2610125147 learned_addr learned my addr 172.21.15.44:0/2610125147 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73400640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a740ce690 0x7f2a741f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a6be00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a7415d130 0x7f2a7414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 -- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a7415d130 msgr2=0x7f2a7414f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a7415d130 0x7f2a7414f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 -- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a740ce690 msgr2=0x7f2a741f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a740ce690 0x7f2a741f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 -- 172.21.15.44:0/2610125147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a740ccea0 con 0x7f2a74161be0 2024-07-17T08:41:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.774+0000 7f2a73e00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b970 tx=0x7f2a6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.775+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c019070 con 0x7f2a74161be0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.775+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a6c00cc50 con 0x7f2a74161be0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.775+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0146a0 con 0x7f2a74161be0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.775+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a740c76e0 con 0x7f2a74161be0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.775+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a740c7980 con 0x7f2a74161be0 2024-07-17T08:41:38.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.776+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a740cfdf0 con 0x7f2a74161be0 2024-07-17T08:41:38.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.777+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f2a6c01c070 con 0x7f2a74161be0 2024-07-17T08:41:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.778+0000 7f2a6b400640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 0x7f2a50079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.779+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f2a6c098620 con 0x7f2a74161be0 2024-07-17T08:41:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.779+0000 7f2a73400640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 0x7f2a50079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:38.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.780+0000 7f2a73400640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 0x7f2a50079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a64033000 tx=0x7f2a64004730 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:38.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:38.783+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2a6c061680 con 0x7f2a74161be0 2024-07-17T08:41:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.022+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 --> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2a74160900 con 0x7f2a50076cd0 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.026+0000 7f2a6b400640 1 -- 172.21.15.44:0/2610125147 <== mgr.9029 v2:172.21.15.175:6834/3743840884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2a74160900 con 0x7f2a50076cd0 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:41:39.027 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:41:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 msgr2=0x7f2a50079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 0x7f2a50079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a64033000 tx=0x7f2a64004730 comp rx=0 tx=0).stop 2024-07-17T08:41:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 msgr2=0x7f2a740c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b970 tx=0x7f2a6c00be30 comp rx=0 tx=0).stop 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 shutdown_connections 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f2a50076cd0 0x7f2a50079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a74161be0 0x7f2a740c8180 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a7415d130 0x7f2a7414f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 --2- 172.21.15.44:0/2610125147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a740ce690 0x7f2a741f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.029+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 >> 172.21.15.44:0/2610125147 conn(0x7f2a740b4ff0 msgr2=0x7f2a740b66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.030+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 shutdown_connections 2024-07-17T08:41:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.030+0000 7f2a78a00640 1 -- 172.21.15.44:0/2610125147 wait complete. 2024-07-17T08:41:39.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.266+0000 7f8dcce00640 1 -- 172.21.15.44:0/766976392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80acab0 msgr2=0x7f8dc806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.266+0000 7f8dcce00640 1 --2- 172.21.15.44:0/766976392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80acab0 0x7f8dc806eb90 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f8dc806f250 tx=0x7f8dc002b8f0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 -- 172.21.15.44:0/766976392 shutdown_connections 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 --2- 172.21.15.44:0/766976392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80acab0 0x7f8dc806eb90 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 --2- 172.21.15.44:0/766976392 >> v2:172.21.15.175:3300/0 conn(0x7f8dc806f6d0 0x7f8dc815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 -- 172.21.15.44:0/766976392 >> 172.21.15.44:0/766976392 conn(0x7f8dc80b0eb0 msgr2=0x7f8dc80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 -- 172.21.15.44:0/766976392 shutdown_connections 2024-07-17T08:41:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.267+0000 7f8dcce00640 1 -- 172.21.15.44:0/766976392 wait complete. 2024-07-17T08:41:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.269+0000 7f8dcce00640 1 Processor -- start 2024-07-17T08:41:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.269+0000 7f8dcce00640 1 -- start start 2024-07-17T08:41:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc806f6d0 0x7f8dc8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc80acab0 0x7f8dc8155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80aea90 con 0x7f8dc80b3280 2024-07-17T08:41:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80ae910 con 0x7f8dc806f6d0 2024-07-17T08:41:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.270+0000 7f8dcce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc80aec10 con 0x7f8dc80acab0 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dbd400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc80acab0 0x7f8dc8155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40964/0 (socket says 172.21.15.44:40964) 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 -- 172.21.15.44:0/2179948229 learned_addr learned my addr 172.21.15.44:0/2179948229 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc5e00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc806f6d0 0x7f8dc8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 -- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc80acab0 msgr2=0x7f8dc8155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc80acab0 0x7f8dc8155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 -- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc806f6d0 msgr2=0x7f8dc8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc806f6d0 0x7f8dc8159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.271+0000 7f8dc6800640 1 -- 172.21.15.44:0/2179948229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc81f8350 con 0x7f8dc80b3280 2024-07-17T08:41:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dc6800640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000ad00 tx=0x7f8dc0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0004040 con 0x7f8dc80b3280 2024-07-17T08:41:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc002e070 con 0x7f8dc80b3280 2024-07-17T08:41:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0004440 con 0x7f8dc80b3280 2024-07-17T08:41:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc81f53b0 con 0x7f8dc80b3280 2024-07-17T08:41:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.272+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81f5970 con 0x7f8dc80b3280 2024-07-17T08:41:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.273+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc815a5a0 con 0x7f8dc80b3280 2024-07-17T08:41:39.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.275+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 180) ==== 98931+0+0 (secure 0 0 0) 0x7f8dc00045e0 con 0x7f8dc80b3280 2024-07-17T08:41:39.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.276+0000 7f8dbea00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 0x7f8da4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:39.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.276+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f8dc00ba200 con 0x7f8dc80b3280 2024-07-17T08:41:39.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.276+0000 7f8dc5e00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 0x7f8da4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:39.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.277+0000 7f8dc5e00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 0x7f8da4079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8db80058c0 tx=0x7f8db800a250 comp rx=0 tx=0).ready entity=mgr.9029 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:39.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.280+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8dc0082ae0 con 0x7f8dc80b3280 2024-07-17T08:41:39.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.540+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f8dc81f51a0 con 0x7f8dc80b3280 2024-07-17T08:41:39.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.772+0000 7f8dbea00640 1 -- 172.21.15.44:0/2179948229 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v181) ==== 92+0+0 (secure 0 0 0) 0x7f8dc0087030 con 0x7f8dc80b3280 2024-07-17T08:41:39.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.775+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 msgr2=0x7f8da4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.775+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 0x7f8da4079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8db80058c0 tx=0x7f8db800a250 comp rx=0 tx=0).stop 2024-07-17T08:41:39.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.775+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 msgr2=0x7f8dc814f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:39.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.775+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000ad00 tx=0x7f8dc0003c10 comp rx=0 tx=0).stop 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 shutdown_connections 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8da4076cd0 0x7f8da4079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8dc80b3280 0x7f8dc814f210 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8dc80acab0 0x7f8dc8155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 --2- 172.21.15.44:0/2179948229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8dc806f6d0 0x7f8dc8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 >> 172.21.15.44:0/2179948229 conn(0x7f8dc80b0eb0 msgr2=0x7f8dc815bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 shutdown_connections 2024-07-17T08:41:39.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:39.776+0000 7f8dcce00640 1 -- 172.21.15.44:0/2179948229 wait complete. 2024-07-17T08:41:39.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:41:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 -- 172.21.15.44:0/1927394760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c0b33e0 msgr2=0x7f599c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 --2- 172.21.15.44:0/1927394760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c0b33e0 0x7f599c072d10 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f5988009830 tx=0x7f598802a3f0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 -- 172.21.15.44:0/1927394760 shutdown_connections 2024-07-17T08:41:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 --2- 172.21.15.44:0/1927394760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c0b33e0 0x7f599c072d10 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 -- 172.21.15.44:0/1927394760 >> 172.21.15.44:0/1927394760 conn(0x7f599c06bf40 msgr2=0x7f599c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 -- 172.21.15.44:0/1927394760 shutdown_connections 2024-07-17T08:41:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.000+0000 7f59a2000640 1 -- 172.21.15.44:0/1927394760 wait complete. 2024-07-17T08:41:40.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.003+0000 7f59a2000640 1 Processor -- start 2024-07-17T08:41:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.003+0000 7f59a2000640 1 -- start start 2024-07-17T08:41:40.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 0x7f599c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1544f0 0x7f599c146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f599c0adc30 con 0x7f599c15dac0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f599c0adab0 con 0x7f599c1544f0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f59a2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f599c0addb0 con 0x7f599c0b33e0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f599be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f599be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41006/0 (socket says 172.21.15.44:41006) 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f599be00640 1 -- 172.21.15.44:0/676801368 learned_addr learned my addr 172.21.15.44:0/676801368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f599b400640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 0x7f599c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.005+0000 7f599aa00640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1544f0 0x7f599c146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 msgr2=0x7f599c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 0x7f599c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1544f0 msgr2=0x7f599c146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1544f0 0x7f599c146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 -- 172.21.15.44:0/676801368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c153100 con 0x7f599c15dac0 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599b400640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 0x7f599c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:41:40.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f599be00640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f599400c6f0 tx=0x7f599400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.006+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994017070 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.007+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5994003c90 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.007+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994003e60 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.007+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c15b310 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.007+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c1583d0 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.008+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599c0ae2a0 con 0x7f599c15dac0 2024-07-17T08:41:40.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.009+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 181) ==== 98948+0+0 (secure 0 0 0) 0x7f599401a070 con 0x7f599c15dac0 2024-07-17T08:41:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.011+0000 7f5993400640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 0x7f597c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.011+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f5994097f90 con 0x7f599c15dac0 2024-07-17T08:41:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.011+0000 7f599b400640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 msgr2=0x7f597c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:40.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.011+0000 7f599b400640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 0x7f597c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:40.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.014+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f599c0b51e0 con 0x7f599c15dac0 2024-07-17T08:41:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.213+0000 7f599b400640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 msgr2=0x7f597c0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.213+0000 7f599b400640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 0x7f597c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:40.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.244+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f599c0afb30 con 0x7f599c15dac0 2024-07-17T08:41:40.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.249+0000 7f5993400640 1 -- 172.21.15.44:0/676801368 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v93)=0 v93) ==== 138+0+0 (secure 0 0 0) 0x7f5994060f50 con 0x7f599c15dac0 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 msgr2=0x7f597c0791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 0x7f597c0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 msgr2=0x7f599c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f599400c6f0 tx=0x7f599400cbb0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 shutdown_connections 2024-07-17T08:41:40.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f597c076d20 0x7f597c0791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f599c15dac0 0x7f599c06ef60 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1544f0 0x7f599c146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 --2- 172.21.15.44:0/676801368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c0b33e0 0x7f599c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 >> 172.21.15.44:0/676801368 conn(0x7f599c06bf40 msgr2=0x7f599c15ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 shutdown_connections 2024-07-17T08:41:40.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.252+0000 7f59a2000640 1 -- 172.21.15.44:0/676801368 wait complete. 2024-07-17T08:41:40.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 -- 172.21.15.44:0/3805465452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04003ac0 msgr2=0x7fcc04003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 --2- 172.21.15.44:0/3805465452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04003ac0 0x7fcc04003ee0 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009b50 tx=0x7fcc04005830 comp rx=0 tx=0).stop 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 -- 172.21.15.44:0/3805465452 shutdown_connections 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 --2- 172.21.15.44:0/3805465452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc04003ac0 0x7fcc04003ee0 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 -- 172.21.15.44:0/3805465452 >> 172.21.15.44:0/3805465452 conn(0x7fcc1c0b4ff0 msgr2=0x7fcc1c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 -- 172.21.15.44:0/3805465452 shutdown_connections 2024-07-17T08:41:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.466+0000 7fcc22e00640 1 -- 172.21.15.44:0/3805465452 wait complete. 2024-07-17T08:41:40.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.468+0000 7fcc22e00640 1 Processor -- start 2024-07-17T08:41:40.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.468+0000 7fcc22e00640 1 -- start start 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04003ac0 0x7fcc1c0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c0b8640 0x7fcc1c0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0b2860 con 0x7fcc1c1540d0 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0b26e0 con 0x7fcc1c0b8640 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc22e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1c0cfdf0 con 0x7fcc04003ac0 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc21000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc21000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41024/0 (socket says 172.21.15.44:41024) 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.469+0000 7fcc21000640 1 -- 172.21.15.44:0/4113902601 learned_addr learned my addr 172.21.15.44:0/4113902601 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc22400640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c0b8640 0x7fcc1c0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21a00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04003ac0 0x7fcc1c0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04003ac0 msgr2=0x7fcc1c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04003ac0 0x7fcc1c0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c0b8640 msgr2=0x7fcc1c0b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c0b8640 0x7fcc1c0b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 -- 172.21.15.44:0/4113902601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c1f42f0 con 0x7fcc1c1540d0 2024-07-17T08:41:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.470+0000 7fcc21000640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d540 tx=0x7fcc0800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.471+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08016020 con 0x7fcc1c1540d0 2024-07-17T08:41:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.471+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc08012630 con 0x7fcc1c1540d0 2024-07-17T08:41:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.471+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08012a80 con 0x7fcc1c1540d0 2024-07-17T08:41:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.471+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1f45d0 con 0x7fcc1c1540d0 2024-07-17T08:41:40.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.471+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1f50b0 con 0x7fcc1c1540d0 2024-07-17T08:41:40.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.472+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c0b9000 con 0x7fcc1c1540d0 2024-07-17T08:41:40.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.473+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 181) ==== 98948+0+0 (secure 0 0 0) 0x7fcc08019020 con 0x7fcc1c1540d0 2024-07-17T08:41:40.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.474+0000 7fcc19600640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 0x7fcbfc079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.474+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7fcc08098730 con 0x7fcc1c1540d0 2024-07-17T08:41:40.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.474+0000 7fcc21a00640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 msgr2=0x7fcbfc079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:40.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.474+0000 7fcc21a00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 0x7fcbfc079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:40.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.479+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcc08020020 con 0x7fcc1c1540d0 2024-07-17T08:41:40.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.674+0000 7fcc21a00640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 msgr2=0x7fcbfc079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:40.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.674+0000 7fcc21a00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 0x7fcbfc079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:40.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.688+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fcc1c0d0e10 con 0x7fcc1c1540d0 2024-07-17T08:41:40.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.696+0000 7fcc19600640 1 -- 172.21.15.44:0/4113902601 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v94)=0 v94) ==== 133+0+0 (secure 0 0 0) 0x7fcc08061660 con 0x7fcc1c1540d0 2024-07-17T08:41:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.699+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 msgr2=0x7fcbfc079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.699+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 0x7fcbfc079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.699+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 msgr2=0x7fcc1c154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.699+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d540 tx=0x7fcc0800da00 comp rx=0 tx=0).stop 2024-07-17T08:41:40.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 shutdown_connections 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7fcbfc076d50 0x7fcbfc079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c0b8640 0x7fcc1c0b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c1540d0 0x7fcc1c154550 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 --2- 172.21.15.44:0/4113902601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc04003ac0 0x7fcc1c0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 >> 172.21.15.44:0/4113902601 conn(0x7fcc1c0b4ff0 msgr2=0x7fcc1c0b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 shutdown_connections 2024-07-17T08:41:40.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.700+0000 7fcc22e00640 1 -- 172.21.15.44:0/4113902601 wait complete. 2024-07-17T08:41:40.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs mygrp03 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 -- 172.21.15.44:0/3567063818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c88003a80 msgr2=0x7f8c88003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 --2- 172.21.15.44:0/3567063818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c88003a80 0x7f8c88003ea0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f8c94001dc0 tx=0x7f8c88005830 comp rx=0 tx=0).stop 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca6000640 1 -- 172.21.15.44:0/3567063818 reap_dead start 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 -- 172.21.15.44:0/3567063818 shutdown_connections 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 -- 172.21.15.44:0/3567063818 >> 172.21.15.44:0/3567063818 conn(0x7f8ca00b0eb0 msgr2=0x7f8ca00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 -- 172.21.15.44:0/3567063818 shutdown_connections 2024-07-17T08:41:40.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.899+0000 7f8ca8000640 1 -- 172.21.15.44:0/3567063818 wait complete. 2024-07-17T08:41:40.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.900+0000 7f8ca8000640 1 Processor -- start 2024-07-17T08:41:40.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.901+0000 7f8ca8000640 1 -- start start 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88003a80 0x7f8ca015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ca0072cd0 0x7f8ca00715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 0x7f8ca014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca015c470 con 0x7f8ca0071b20 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca015c2f0 con 0x7f8c88003a80 2024-07-17T08:41:40.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca015c5f0 con 0x7f8ca0072cd0 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 0x7f8ca014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 0x7f8ca014a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41044/0 (socket says 172.21.15.44:41044) 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 -- 172.21.15.44:0/431425404 learned_addr learned my addr 172.21.15.44:0/431425404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88003a80 0x7f8ca015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca4c00640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ca0072cd0 0x7f8ca00715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ca0072cd0 msgr2=0x7f8ca00715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ca0072cd0 0x7f8ca00715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88003a80 msgr2=0x7f8ca015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c88003a80 0x7f8ca015ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.902+0000 7f8ca6000640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca0152900 con 0x7f8ca0071b20 2024-07-17T08:41:40.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8ca6000640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 0x7f8ca014a260 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0070b60 tx=0x7f8c9800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:40.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98006440 con 0x7f8ca0071b20 2024-07-17T08:41:40.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca0152be0 con 0x7f8ca0071b20 2024-07-17T08:41:40.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c980036e0 con 0x7f8ca0071b20 2024-07-17T08:41:40.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98003b10 con 0x7f8ca0071b20 2024-07-17T08:41:40.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.903+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01531a0 con 0x7f8ca0071b20 2024-07-17T08:41:40.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.904+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca00cd2e0 con 0x7f8ca0071b20 2024-07-17T08:41:40.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.907+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 181) ==== 98948+0+0 (secure 0 0 0) 0x7f8c98003cb0 con 0x7f8ca0071b20 2024-07-17T08:41:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.908+0000 7f8c9d600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.908+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(308..308 src has 1..308) ==== 6705+0+0 (secure 0 0 0) 0x7f8c98098d10 con 0x7f8ca0071b20 2024-07-17T08:41:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.908+0000 7f8ca5600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:40.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.908+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:40.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:40.912+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8c9809e020 con 0x7f8ca0071b20 2024-07-17T08:41:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:41.110+0000 7f8ca5600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:41.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:41.110+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:41.154+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp03", "target": ["mon-mgr", ""]}) -- 0x7f8ca0159220 con 0x7f8c80076d50 2024-07-17T08:41:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:41.512+0000 7f8ca5600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:41.512+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:41:41.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:41:41.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:41:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:42.313+0000 7f8ca5600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:42.313+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:41:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:43.915+0000 7f8ca5600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3743840884 2024-07-17T08:41:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:43.916+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:41:44.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:44.027+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 182) ==== 98557+0+0 (secure 0 0 0) 0x7f8c9805d750 con 0x7f8ca0071b20 2024-07-17T08:41:44.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:44.027+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 msgr2=0x7f8c80079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:44.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:44.027+0000 7f8c9d600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/3743840884,v1:172.21.15.175:6837/3743840884] conn(0x7f8c80076d50 0x7f8c80079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:44.315 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74a7e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74a7e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74a7e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74a7e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74a7e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74aba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74aba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.317 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74aba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.317 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74aba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:44.317 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:44.313+0000 7f74aba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.036+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 183) ==== 98684+0+0 (secure 0 0 0) 0x7f8c9806cc80 con 0x7f8ca0071b20 2024-07-17T08:41:45.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.037+0000 7f8c9d600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8c8007f9b0 0x7f8c80081da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.037+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 --> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp03", "target": ["mon-mgr", ""]}) -- 0x7f8c98004e50 con 0x7f8c8007f9b0 2024-07-17T08:41:45.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.041+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8c8007f9b0 0x7f8c80081da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.055+0000 7f8ca5600640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8c8007f9b0 0x7f8c80081da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c94032000 tx=0x7f8c94001510 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:45.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:45.076+0000 7f74a4200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:45.076+0000 7f74a4200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:45.076+0000 7f74a4200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:45.076+0000 7f74a4200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.078 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:45.076+0000 7f74a4200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:45.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.098+0000 7f8c9d600640 1 -- 172.21.15.44:0/431425404 <== mgr.10189 v2:172.21.15.175:6834/2231139025 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c98004e50 con 0x7f8c8007f9b0 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8c8007f9b0 msgr2=0x7f8c80081da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8c8007f9b0 0x7f8c80081da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c94032000 tx=0x7f8c94001510 comp rx=0 tx=0).stop 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 msgr2=0x7f8ca014a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 --2- 172.21.15.44:0/431425404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ca0071b20 0x7f8ca014a260 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0070b60 tx=0x7f8c9800da60 comp rx=0 tx=0).stop 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca6000640 1 -- 172.21.15.44:0/431425404 reap_dead start 2024-07-17T08:41:45.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 shutdown_connections 2024-07-17T08:41:45.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.102+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 >> 172.21.15.44:0/431425404 conn(0x7f8ca00b0eb0 msgr2=0x7f8ca015e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:45.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.103+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 shutdown_connections 2024-07-17T08:41:45.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.103+0000 7f8ca8000640 1 -- 172.21.15.44:0/431425404 wait complete. 2024-07-17T08:41:45.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv03 --group_name mygrp03 2024-07-17T08:41:45.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 -- 172.21.15.44:0/3066263502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580cf3f0 msgr2=0x7fdc580ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 --2- 172.21.15.44:0/3066263502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580cf3f0 0x7fdc580ad090 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005fd0 tx=0x7fdc4802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 -- 172.21.15.44:0/3066263502 shutdown_connections 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 --2- 172.21.15.44:0/3066263502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 0x7fdc580b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 --2- 172.21.15.44:0/3066263502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580cf3f0 0x7fdc580ad090 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.334+0000 7fdc5d200640 1 -- 172.21.15.44:0/3066263502 >> 172.21.15.44:0/3066263502 conn(0x7fdc580b55d0 msgr2=0x7fdc580b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.335+0000 7fdc5d200640 1 -- 172.21.15.44:0/3066263502 shutdown_connections 2024-07-17T08:41:45.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.335+0000 7fdc5d200640 1 -- 172.21.15.44:0/3066263502 wait complete. 2024-07-17T08:41:45.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.336+0000 7fdc5d200640 1 Processor -- start 2024-07-17T08:41:45.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.336+0000 7fdc5d200640 1 -- start start 2024-07-17T08:41:45.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 0x7fdc5809a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 0x7fdc581f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580d06f0 con 0x7fdc580b1d40 2024-07-17T08:41:45.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc580d0570 con 0x7fdc580cf3f0 2024-07-17T08:41:45.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.337+0000 7fdc5d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc580d0870 con 0x7fdc580cf7d0 2024-07-17T08:41:45.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41056/0 (socket says 172.21.15.44:41056) 2024-07-17T08:41:45.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 -- 172.21.15.44:0/596707972 learned_addr learned my addr 172.21.15.44:0/596707972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc56a00640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 0x7fdc5809a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57e00640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 0x7fdc581f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 -- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 msgr2=0x7fdc581f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 0x7fdc581f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 -- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 msgr2=0x7fdc5809a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 0x7fdc5809a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc57400640 1 -- 172.21.15.44:0/596707972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5809c080 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.338+0000 7fdc56a00640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 0x7fdc5809a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc57400640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009850 tx=0x7fdc480038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4803f070 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc480047a0 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48035030 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5809c360 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.339+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc58151ad0 con 0x7fdc580b1d40 2024-07-17T08:41:45.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.340+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5809c750 con 0x7fdc580b1d40 2024-07-17T08:41:45.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.341+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 183) ==== 98684+0+0 (secure 0 0 0) 0x7fdc48042070 con 0x7fdc580b1d40 2024-07-17T08:41:45.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.342+0000 7fdc4f400640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 0x7fdc34078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.343+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7fdc480b91d0 con 0x7fdc580b1d40 2024-07-17T08:41:45.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.343+0000 7fdc56a00640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 0x7fdc34078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.344+0000 7fdc56a00640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 0x7fdc34078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc581f7fa0 tx=0x7fdc40005d10 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:45.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.347+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdc480823d0 con 0x7fdc580b1d40 2024-07-17T08:41:45.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.587+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 --> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv03", "group_name": "mygrp03", "target": ["mon-mgr", ""]}) -- 0x7fdc58074570 con 0x7fdc34076a50 2024-07-17T08:41:45.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.651+0000 7fdc4f400640 1 -- 172.21.15.44:0/596707972 <== mgr.10189 v2:172.21.15.175:6834/2231139025 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc58074570 con 0x7fdc34076a50 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 msgr2=0x7fdc34078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 0x7fdc34078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc581f7fa0 tx=0x7fdc40005d10 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 msgr2=0x7fdc581f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009850 tx=0x7fdc480038a0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 shutdown_connections 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fdc34076a50 0x7fdc34078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc580cf7d0 0x7fdc581f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc580cf3f0 0x7fdc5809a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 --2- 172.21.15.44:0/596707972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc580b1d40 0x7fdc581f7a40 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 >> 172.21.15.44:0/596707972 conn(0x7fdc580b55d0 msgr2=0x7fdc58159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 shutdown_connections 2024-07-17T08:41:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.654+0000 7fdc5d200640 1 -- 172.21.15.44:0/596707972 wait complete. 2024-07-17T08:41:45.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv03 . 1m 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 -- 172.21.15.44:0/2076142497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280015d930 msgr2=0x7f2800153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 --2- 172.21.15.44:0/2076142497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280015d930 0x7f2800153e60 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009830 tx=0x7f27ec02a510 comp rx=0 tx=0).stop 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 -- 172.21.15.44:0/2076142497 shutdown_connections 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 --2- 172.21.15.44:0/2076142497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2800074400 0x7f28001543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 --2- 172.21.15.44:0/2076142497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280015d930 0x7f2800153e60 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 -- 172.21.15.44:0/2076142497 >> 172.21.15.44:0/2076142497 conn(0x7f28000cf050 msgr2=0x7f28000cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 -- 172.21.15.44:0/2076142497 shutdown_connections 2024-07-17T08:41:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.896+0000 7f2806c00640 1 -- 172.21.15.44:0/2076142497 wait complete. 2024-07-17T08:41:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.898+0000 7f2806c00640 1 Processor -- start 2024-07-17T08:41:45.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.898+0000 7f2806c00640 1 -- start start 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2800074020 0x7f28001549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2800074400 0x7f28000726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280015d930 0x7f28001f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28000ae490 con 0x7f2800074400 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28000ae310 con 0x7f2800074020 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2806c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28000ae610 con 0x7f280015d930 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2804e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2800074400 0x7f28000726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2804e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2800074400 0x7f28000726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41068/0 (socket says 172.21.15.44:41068) 2024-07-17T08:41:45.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.899+0000 7f2804e00640 1 -- 172.21.15.44:0/2365817979 learned_addr learned my addr 172.21.15.44:0/2365817979 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:45.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 -- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280015d930 msgr2=0x7f28001f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280015d930 0x7f28001f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 -- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2800074020 msgr2=0x7f28001549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2800074020 0x7f28001549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 -- 172.21.15.44:0/2365817979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2800159480 con 0x7f2800074400 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2804e00640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2800074400 0x7f28000726c0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02ab00 tx=0x7f27ec033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec005c90 con 0x7f2800074400 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ec031e50 con 0x7f2800074400 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec005260 con 0x7f2800074400 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2806c00640 1 -- 172.21.15.44:0/2365817979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2800159760 con 0x7f2800074400 2024-07-17T08:41:45.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.900+0000 7f2806c00640 1 -- 172.21.15.44:0/2365817979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2800159d20 con 0x7f2800074400 2024-07-17T08:41:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.901+0000 7f2806c00640 1 -- 172.21.15.44:0/2365817979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280015a150 con 0x7f2800074400 2024-07-17T08:41:45.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.903+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 183) ==== 98684+0+0 (secure 0 0 0) 0x7f27ec005400 con 0x7f2800074400 2024-07-17T08:41:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.904+0000 7f27fd600640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f27e0076a50 0x7f27e0078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:45.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.904+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f27ec0b9420 con 0x7f2800074400 2024-07-17T08:41:45.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.904+0000 7f2805800640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f27e0076a50 0x7f27e0078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:45.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.905+0000 7f2805800640 1 --2- 172.21.15.44:0/2365817979 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f27e0076a50 0x7f27e0078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27f4004a30 tx=0x7f27f40016f0 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:45.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:45.908+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f27ec03f030 con 0x7f2800074400 2024-07-17T08:41:46.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.096+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 184) ==== 99000+0+0 (secure 0 0 0) 0x7f27ec07e280 con 0x7f2800074400 2024-07-17T08:41:46.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.149+0000 7f2806c00640 1 -- 172.21.15.44:0/2365817979 --> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv03", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f28000ac0f0 con 0x7f27e0076a50 2024-07-17T08:41:46.154 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.153+0000 7f74b4600640 -1 mgr.server reply reply (22) Invalid argument No such subvol: None::sv03 2024-07-17T08:41:46.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.153+0000 7f27fd600640 1 -- 172.21.15.44:0/2365817979 <== mgr.10189 v2:172.21.15.175:6834/2231139025 1 ==== mgr_command_reply(tid 0: -22 No such subvol: None::sv03) ==== 34+0+0 (secure 0 0 0) 0x7f28000ac0f0 con 0x7f27e0076a50 2024-07-17T08:41:46.155 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: No such subvol: None::sv03 2024-07-17T08:41:46.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:41:46.170 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv03 --group_name mygrp03 2024-07-17T08:41:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 -- 172.21.15.44:0/1496738175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c0d0b40 msgr2=0x7f8e1c0b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 --2- 172.21.15.44:0/1496738175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c0b8670 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005fd0 tx=0x7f8e1402a780 comp rx=0 tx=0).stop 2024-07-17T08:41:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 -- 172.21.15.44:0/1496738175 shutdown_connections 2024-07-17T08:41:46.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 --2- 172.21.15.44:0/1496738175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c0b8670 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 -- 172.21.15.44:0/1496738175 >> 172.21.15.44:0/1496738175 conn(0x7f8e1c073a60 msgr2=0x7f8e1c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 -- 172.21.15.44:0/1496738175 shutdown_connections 2024-07-17T08:41:46.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.383+0000 7f8e21000640 1 -- 172.21.15.44:0/1496738175 wait complete. 2024-07-17T08:41:46.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.385+0000 7f8e21000640 1 Processor -- start 2024-07-17T08:41:46.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.385+0000 7f8e21000640 1 -- start start 2024-07-17T08:41:46.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1c161b50 0x7f8e1c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c0b8d30 con 0x7f8e1c15e370 2024-07-17T08:41:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c0b8bb0 con 0x7f8e1c0d0b40 2024-07-17T08:41:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.386+0000 7f8e21000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c0b8eb0 con 0x7f8e1c161b50 2024-07-17T08:41:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41072/0 (socket says 172.21.15.44:41072) 2024-07-17T08:41:46.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 -- 172.21.15.44:0/1921377729 learned_addr learned my addr 172.21.15.44:0/1921377729 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1b400640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 -- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1c161b50 msgr2=0x7f8e1c1f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1c161b50 0x7f8e1c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 -- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e1c0d0b40 msgr2=0x7f8e1c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.387+0000 7f8e1aa00640 1 -- 172.21.15.44:0/1921377729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c0c7050 con 0x7f8e1c15e370 2024-07-17T08:41:46.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e1aa00640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400b420 tx=0x7f8e0400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04011830 con 0x7f8e1c15e370 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e04016070 con 0x7f8e1c15e370 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04011d60 con 0x7f8e1c15e370 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c0c7310 con 0x7f8e1c15e370 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.388+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c0c78d0 con 0x7f8e1c15e370 2024-07-17T08:41:46.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.389+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c0b9330 con 0x7f8e1c15e370 2024-07-17T08:41:46.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.390+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 184) ==== 99000+0+0 (secure 0 0 0) 0x7f8e0401b460 con 0x7f8e1c15e370 2024-07-17T08:41:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.391+0000 7f8e13400640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 0x7f8df8079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.391+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f8e04098820 con 0x7f8e1c15e370 2024-07-17T08:41:46.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.392+0000 7f8e1b400640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 0x7f8df8079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:46.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.393+0000 7f8e1b400640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 0x7f8df8079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0016b0 tx=0x7f8e0c009040 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:46.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.395+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8e04020020 con 0x7f8e1c15e370 2024-07-17T08:41:46.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.633+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 --> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv03", "group_name": "mygrp03", "target": ["mon-mgr", ""]}) -- 0x7f8e1c15dee0 con 0x7f8df8076db0 2024-07-17T08:41:46.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.675+0000 7f8e13400640 1 -- 172.21.15.44:0/1921377729 <== mgr.10189 v2:172.21.15.175:6834/2231139025 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8e1c15dee0 con 0x7f8df8076db0 2024-07-17T08:41:46.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 msgr2=0x7f8df8079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 0x7f8df8079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0016b0 tx=0x7f8e0c009040 comp rx=0 tx=0).stop 2024-07-17T08:41:46.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 msgr2=0x7f8e1c0b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400b420 tx=0x7f8e0400b8e0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 shutdown_connections 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8df8076db0 0x7f8df8079270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e1c161b50 0x7f8e1c1f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e1c15e370 0x7f8e1c0b77d0 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 --2- 172.21.15.44:0/1921377729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e1c0d0b40 0x7f8e1c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.678+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 >> 172.21.15.44:0/1921377729 conn(0x7f8e1c073a60 msgr2=0x7f8e1c15fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.679+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 shutdown_connections 2024-07-17T08:41:46.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.679+0000 7f8e21000640 1 -- 172.21.15.44:0/1921377729 wait complete. 2024-07-17T08:41:46.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs mygrp03 2024-07-17T08:41:46.705 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.703+0000 7f74a6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:46.705 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.703+0000 7f74a6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:46.705 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.703+0000 7f74a6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:46.705 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.703+0000 7f74a6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:46.705 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:46.703+0000 7f74a6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 -- 172.21.15.44:0/3308408231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c06f380 msgr2=0x7fda8c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 --2- 172.21.15.44:0/3308408231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c06f380 0x7fda8c0cb080 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fda84005fd0 tx=0x7fda8402aaa0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 -- 172.21.15.44:0/3308408231 shutdown_connections 2024-07-17T08:41:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 --2- 172.21.15.44:0/3308408231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c06f380 0x7fda8c0cb080 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 --2- 172.21.15.44:0/3308408231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c151f20 0x7fda8c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 -- 172.21.15.44:0/3308408231 >> 172.21.15.44:0/3308408231 conn(0x7fda8c0cf0d0 msgr2=0x7fda8c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 -- 172.21.15.44:0/3308408231 shutdown_connections 2024-07-17T08:41:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.917+0000 7fda91600640 1 -- 172.21.15.44:0/3308408231 wait complete. 2024-07-17T08:41:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.919+0000 7fda91600640 1 Processor -- start 2024-07-17T08:41:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.919+0000 7fda91600640 1 -- start start 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c06f380 0x7fda8c150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c151f20 0x7fda8c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0cb740 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0cb5c0 con 0x7fda8c151f20 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda91600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c0cb8c0 con 0x7fda8c06f380 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda83e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda83e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41098/0 (socket says 172.21.15.44:41098) 2024-07-17T08:41:46.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.920+0000 7fda83e00640 1 -- 172.21.15.44:0/151695991 learned_addr learned my addr 172.21.15.44:0/151695991 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:46.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 -- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c06f380 msgr2=0x7fda8c150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c06f380 0x7fda8c150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 -- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c151f20 msgr2=0x7fda8c0af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c151f20 0x7fda8c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 -- 172.21.15.44:0/151695991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c1f73b0 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda83e00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c530 tx=0x7fda7400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740052d0 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda74005470 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1f7690 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.921+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74012870 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.922+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1f7c50 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.923+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c0cb080 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.924+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 184) ==== 99000+0+0 (secure 0 0 0) 0x7fda740035f0 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.925+0000 7fda88c00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 0x7fda68079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:46.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.925+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7fda74098910 con 0x7fda8c0cd0b0 2024-07-17T08:41:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.926+0000 7fda8be00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 0x7fda68079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:46.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.927+0000 7fda8be00640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 0x7fda68079270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00a8f0 tx=0x7fda7c005bd0 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:46.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:46.929+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fda8c15c680 con 0x7fda8c0cd0b0 2024-07-17T08:41:47.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.175+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 --> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "mygrp03", "target": ["mon-mgr", ""]}) -- 0x7fda8c0cf830 con 0x7fda68076db0 2024-07-17T08:41:47.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.193+0000 7fda88c00640 1 -- 172.21.15.44:0/151695991 <== mgr.10189 v2:172.21.15.175:6834/2231139025 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda8c0cf830 con 0x7fda68076db0 2024-07-17T08:41:47.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 msgr2=0x7fda68079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:47.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 0x7fda68079270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00a8f0 tx=0x7fda7c005bd0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 msgr2=0x7fda8c1f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c530 tx=0x7fda7400c9f0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 shutdown_connections 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fda68076db0 0x7fda68079270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda8c151f20 0x7fda8c0af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c0cd0b0 0x7fda8c1f4af0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 --2- 172.21.15.44:0/151695991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda8c06f380 0x7fda8c150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.195+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 >> 172.21.15.44:0/151695991 conn(0x7fda8c0cf0d0 msgr2=0x7fda8c15c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.196+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 shutdown_connections 2024-07-17T08:41:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.196+0000 7fda91600640 1 -- 172.21.15.44:0/151695991 wait complete. 2024-07-17T08:41:47.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.426+0000 7f7e12c00640 1 -- 172.21.15.44:0/1231367858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 msgr2=0x7f7df4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12c00640 1 --2- 172.21.15.44:0/1231367858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7df4003e50 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000bb20 tx=0x7f7df4005830 comp rx=0 tx=0).stop 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12200640 1 -- 172.21.15.44:0/1231367858 reap_dead start 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12c00640 1 -- 172.21.15.44:0/1231367858 shutdown_connections 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12c00640 1 -- 172.21.15.44:0/1231367858 >> 172.21.15.44:0/1231367858 conn(0x7f7e0c06c120 msgr2=0x7f7e0c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12c00640 1 -- 172.21.15.44:0/1231367858 shutdown_connections 2024-07-17T08:41:47.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.427+0000 7f7e12c00640 1 -- 172.21.15.44:0/1231367858 wait complete. 2024-07-17T08:41:47.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.428+0000 7f7e12c00640 1 Processor -- start 2024-07-17T08:41:47.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.429+0000 7f7e12c00640 1 -- start start 2024-07-17T08:41:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:47.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c15bf40 0x7f7e0c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0c8c50 0x7f7e0c0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ce9c0 con 0x7f7df4003a30 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ce840 con 0x7f7e0c15bf40 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ceb40 con 0x7f7e0c0c8c50 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41130/0 (socket says 172.21.15.44:41130) 2024-07-17T08:41:47.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 -- 172.21.15.44:0/3379038208 learned_addr learned my addr 172.21.15.44:0/3379038208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:47.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e12200640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0c8c50 0x7f7e0c0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:47.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e10e00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c15bf40 0x7f7e0c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 -- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0c8c50 msgr2=0x7f7e0c0c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0c8c50 0x7f7e0c0c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 -- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c15bf40 msgr2=0x7f7e0c15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c15bf40 0x7f7e0c15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.430+0000 7f7e11800640 1 -- 172.21.15.44:0/3379038208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c0c99a0 con 0x7f7df4003a30 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e11800640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000bb20 tx=0x7f7e0000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00040070 con 0x7f7df4003a30 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e0003a920 con 0x7f7df4003a30 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00035040 con 0x7f7df4003a30 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c0c9c80 con 0x7f7df4003a30 2024-07-17T08:41:47.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.431+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c0c9610 con 0x7f7df4003a30 2024-07-17T08:41:47.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.433+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0c0cf0d0 con 0x7f7df4003a30 2024-07-17T08:41:47.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.434+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 184) ==== 99000+0+0 (secure 0 0 0) 0x7f7e00038020 con 0x7f7df4003a30 2024-07-17T08:41:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.436+0000 7f7e09600640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 0x7f7dec0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.436+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f7e000b9940 con 0x7f7df4003a30 2024-07-17T08:41:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.437+0000 7f7e10e00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 0x7f7dec0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.438+0000 7f7e10e00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 0x7f7dec0792a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c153520 tx=0x7f7df8009040 comp rx=0 tx=0).ready entity=mgr.10189 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.440+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7e000825c0 con 0x7f7df4003a30 2024-07-17T08:41:47.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:47.704+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f7e0c0b4160 con 0x7f7df4003a30 2024-07-17T08:41:48.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.104+0000 7f7e09600640 1 -- 172.21.15.44:0/3379038208 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v185) ==== 93+0+0 (secure 0 0 0) 0x7f7e00081ec0 con 0x7f7df4003a30 2024-07-17T08:41:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.108+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 msgr2=0x7f7dec0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.108+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 0x7f7dec0792a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c153520 tx=0x7f7df8009040 comp rx=0 tx=0).stop 2024-07-17T08:41:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.108+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 msgr2=0x7f7e0c14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.108+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000bb20 tx=0x7f7e0000b790 comp rx=0 tx=0).stop 2024-07-17T08:41:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 shutdown_connections 2024-07-17T08:41:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f7dec076de0 0x7f7dec0792a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0c8c50 0x7f7e0c0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c15bf40 0x7f7e0c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 --2- 172.21.15.44:0/3379038208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7df4003a30 0x7f7e0c14e770 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 >> 172.21.15.44:0/3379038208 conn(0x7f7e0c06c120 msgr2=0x7f7e0c1584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.109+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 shutdown_connections 2024-07-17T08:41:48.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.110+0000 7f7e12c00640 1 -- 172.21.15.44:0/3379038208 wait complete. 2024-07-17T08:41:48.136 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:41:48.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:48.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:41:48.335 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:41:48.335 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:48.335 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:48.335 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:48.385 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:48.385 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:48.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:41:48.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 -- 172.21.15.44:0/998268532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00004800 msgr2=0x7f5f00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 --2- 172.21.15.44:0/998268532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00004800 0x7f5f00004c80 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f5f080016c0 tx=0x7f5f00005830 comp rx=0 tx=0).stop 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f1be00640 1 -- 172.21.15.44:0/998268532 reap_dead start 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 -- 172.21.15.44:0/998268532 shutdown_connections 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 -- 172.21.15.44:0/998268532 >> 172.21.15.44:0/998268532 conn(0x7f5f1c0d1350 msgr2=0x7f5f1c0d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 -- 172.21.15.44:0/998268532 shutdown_connections 2024-07-17T08:41:48.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.671+0000 7f5f21000640 1 -- 172.21.15.44:0/998268532 wait complete. 2024-07-17T08:41:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.674+0000 7f5f21000640 1 Processor -- start 2024-07-17T08:41:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.674+0000 7f5f21000640 1 -- start start 2024-07-17T08:41:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f00004800 0x7f5f1c1f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f1c0b35d0 0x7f5f1c1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:48.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c070e50 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c073960 con 0x7f5f00004800 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f21000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c070fd0 con 0x7f5f1c0b35d0 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f1b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f00004800 0x7f5f1c1f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41182/0 (socket says 172.21.15.44:41182) 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f1aa00640 1 -- 172.21.15.44:0/2679656294 learned_addr learned my addr 172.21.15.44:0/2679656294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.676+0000 7f5f1be00640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f1c0b35d0 0x7f5f1c1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f1c0b35d0 msgr2=0x7f5f1c1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f1c0b35d0 0x7f5f1c1f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f00004800 msgr2=0x7f5f1c1f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f00004800 0x7f5f1c1f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 -- 172.21.15.44:0/2679656294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c146ac0 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f1aa00640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802a930 tx=0x7f5f08006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08041070 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f080038e0 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c133800 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.677+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08038070 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.678+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c0b2810 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.679+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c0b4400 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.680+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f5f08044070 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.681+0000 7f5f13400640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 0x7f5efc079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.681+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f5f080b8270 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.681+0000 7f5f1b400640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 msgr2=0x7f5efc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.681+0000 7f5f1b400640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 0x7f5efc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:48.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.689+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5f080808f0 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.883+0000 7f5f1b400640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 msgr2=0x7f5efc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:48.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.883+0000 7f5f1b400640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 0x7f5efc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:48.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.911+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5f1c146d60 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.912+0000 7f5f13400640 1 -- 172.21.15.44:0/2679656294 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v309) ==== 80+0+474 (secure 0 0 0) 0x7f5f08084e40 con 0x7f5f1c0b1fc0 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1315508424 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/530094003 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/666245803 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3453853590 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/3743840884 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/3743840884 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/97927255 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2166198562 2024-07-18T08:41:44.012240+0000 2024-07-17T08:41:48.914 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:41:48.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 msgr2=0x7f5efc079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 0x7f5efc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 msgr2=0x7f5f1c1332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802a930 tx=0x7f5f08006290 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 shutdown_connections 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f5efc076e80 0x7f5efc079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f1c0b35d0 0x7f5f1c1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1c0b1fc0 0x7f5f1c1332c0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 --2- 172.21.15.44:0/2679656294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f00004800 0x7f5f1c1f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 >> 172.21.15.44:0/2679656294 conn(0x7f5f1c0d1350 msgr2=0x7f5f1c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 shutdown_connections 2024-07-17T08:41:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:48.915+0000 7f5f21000640 1 -- 172.21.15.44:0/2679656294 wait complete. 2024-07-17T08:41:48.935 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:41:48.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:48.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:41:49.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:41:49.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:41:49.017 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:41:49.059 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:49.059 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:49.059 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:49.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:49.100 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:49.101 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:41:49.101 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:49.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:41:49.146 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:41:49.146 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:41:49.146 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:41:49.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:41:49.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:49.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 -- 172.21.15.44:0/2907968232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c15ac90 msgr2=0x7f3b3c072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 --2- 172.21.15.44:0/2907968232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c15ac90 0x7f3b3c072f90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3b24005fd0 tx=0x7f3b2402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 -- 172.21.15.44:0/2907968232 shutdown_connections 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 --2- 172.21.15.44:0/2907968232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c15ac90 0x7f3b3c072f90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 --2- 172.21.15.44:0/2907968232 >> v2:172.21.15.175:3300/0 conn(0x7f3b3c06efd0 0x7f3b3c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 -- 172.21.15.44:0/2907968232 >> 172.21.15.44:0/2907968232 conn(0x7f3b3c06bf40 msgr2=0x7f3b3c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 -- 172.21.15.44:0/2907968232 shutdown_connections 2024-07-17T08:41:49.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.392+0000 7f3b42000640 1 -- 172.21.15.44:0/2907968232 wait complete. 2024-07-17T08:41:49.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.394+0000 7f3b42000640 1 Processor -- start 2024-07-17T08:41:49.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.394+0000 7f3b42000640 1 -- start start 2024-07-17T08:41:49.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c0734d0 0x7f3b3c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3c15ac90 0x7f3b3c0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c15e3e0 con 0x7f3b3c06efd0 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c15d480 con 0x7f3b3c15ac90 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b42000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3c15e560 con 0x7f3b3c0734d0 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b3b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33624/0 (socket says 172.21.15.44:33624) 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b3b400640 1 -- 172.21.15.44:0/681241761 learned_addr learned my addr 172.21.15.44:0/681241761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b3be00640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3c15ac90 0x7f3b3c0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.395+0000 7f3b3aa00640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c0734d0 0x7f3b3c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c0734d0 msgr2=0x7f3b3c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c0734d0 0x7f3b3c149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3c15ac90 msgr2=0x7f3b3c0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3c15ac90 0x7f3b3c0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 -- 172.21.15.44:0/681241761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c0cf520 con 0x7f3b3c06efd0 2024-07-17T08:41:49.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b3b400640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400ebc0 tx=0x7f3b3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:49.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.396+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34010630 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.397+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b340108c0 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.397+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c0ae450 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.397+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34010d10 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.397+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c0aea10 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.398+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3c15eab0 con 0x7f3b3c06efd0 2024-07-17T08:41:49.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.399+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f3b34018420 con 0x7f3b3c06efd0 2024-07-17T08:41:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.400+0000 7f3b33400640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 0x7f3b18079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.401+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f3b34099a20 con 0x7f3b3c06efd0 2024-07-17T08:41:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.401+0000 7f3b3aa00640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 msgr2=0x7f3b18079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:49.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.401+0000 7f3b3aa00640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 0x7f3b18079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.405+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3b3c0cf520 con 0x7f3b3c06efd0 2024-07-17T08:41:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.602+0000 7f3b3aa00640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 msgr2=0x7f3b18079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.602+0000 7f3b3aa00640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 0x7f3b18079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.645+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b3c06d0a0 con 0x7f3b3c06efd0 2024-07-17T08:41:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.646+0000 7f3b33400640 1 -- 172.21.15.44:0/681241761 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7f3b34061f50 con 0x7f3b3c06efd0 2024-07-17T08:41:49.648 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:49.648 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:49.649 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:49.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 msgr2=0x7f3b18079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:49.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 0x7f3b18079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 msgr2=0x7f3b3c0cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400ebc0 tx=0x7f3b3400c6a0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 shutdown_connections 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f3b18076e80 0x7f3b18079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b3c15ac90 0x7f3b3c0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.649+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b3c0734d0 0x7f3b3c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.650+0000 7f3b42000640 1 --2- 172.21.15.44:0/681241761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b3c06efd0 0x7f3b3c0cc330 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.650+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 >> 172.21.15.44:0/681241761 conn(0x7f3b3c06bf40 msgr2=0x7f3b3c072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.650+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 shutdown_connections 2024-07-17T08:41:49.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.650+0000 7f3b42000640 1 -- 172.21.15.44:0/681241761 wait complete. 2024-07-17T08:41:49.671 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 -- 172.21.15.44:0/1445886951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf04003aa0 msgr2=0x7fcf04003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/1445886951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf04003aa0 0x7fcf04003e80 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fcf10001e40 tx=0x7fcf04005830 comp rx=0 tx=0).stop 2024-07-17T08:41:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 -- 172.21.15.44:0/1445886951 shutdown_connections 2024-07-17T08:41:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/1445886951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf04003aa0 0x7fcf04003e80 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 -- 172.21.15.44:0/1445886951 >> 172.21.15.44:0/1445886951 conn(0x7fcf200cfec0 msgr2=0x7fcf200d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 -- 172.21.15.44:0/1445886951 shutdown_connections 2024-07-17T08:41:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.867+0000 7fcf1fe00640 1 -- 172.21.15.44:0/1445886951 wait complete. 2024-07-17T08:41:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.869+0000 7fcf1fe00640 1 Processor -- start 2024-07-17T08:41:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.869+0000 7fcf1fe00640 1 -- start start 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf04003aa0 0x7fcf2015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf2015d270 0x7fcf200cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200b54c0 con 0x7fcf200ca510 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200b5340 con 0x7fcf04003aa0 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.870+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf200b5640 con 0x7fcf2015d270 2024-07-17T08:41:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33630/0 (socket says 172.21.15.44:33630) 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 -- 172.21.15.44:0/3629081182 learned_addr learned my addr 172.21.15.44:0/3629081182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf04003aa0 0x7fcf2015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1e000640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf2015d270 0x7fcf200cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf2015d270 msgr2=0x7fcf200cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf2015d270 0x7fcf200cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf04003aa0 msgr2=0x7fcf2015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf04003aa0 0x7fcf2015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.871+0000 7fcf1f400640 1 -- 172.21.15.44:0/3629081182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf200cccd0 con 0x7fcf200ca510 2024-07-17T08:41:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf1f400640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fcf200b2e70 tx=0x7fcf1800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18006440 con 0x7fcf200ca510 2024-07-17T08:41:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf18015070 con 0x7fcf200ca510 2024-07-17T08:41:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18003920 con 0x7fcf200ca510 2024-07-17T08:41:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf200ccf70 con 0x7fcf200ca510 2024-07-17T08:41:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.872+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf200cd530 con 0x7fcf200ca510 2024-07-17T08:41:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.873+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2006dca0 con 0x7fcf200ca510 2024-07-17T08:41:49.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.875+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7fcf18003ac0 con 0x7fcf200ca510 2024-07-17T08:41:49.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.875+0000 7fcf16a00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 0x7fcefc079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:49.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.876+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7fcf18098a60 con 0x7fcf200ca510 2024-07-17T08:41:49.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.876+0000 7fcf1ea00640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 msgr2=0x7fcefc079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:49.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.876+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 0x7fcefc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:49.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:49.880+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcf18060f90 con 0x7fcf200ca510 2024-07-17T08:41:50.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.077+0000 7fcf1ea00640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 msgr2=0x7fcefc079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:50.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.077+0000 7fcf1ea00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 0x7fcefc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.132+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf200b5d70 con 0x7fcf200ca510 2024-07-17T08:41:50.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.134+0000 7fcf16a00640 1 -- 172.21.15.44:0/3629081182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7fcf180654e0 con 0x7fcf200ca510 2024-07-17T08:41:50.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:50.136 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:50.136 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 msgr2=0x7fcefc079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 0x7fcefc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 msgr2=0x7fcf2015b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fcf200b2e70 tx=0x7fcf1800da10 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 shutdown_connections 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7fcefc076eb0 0x7fcefc079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf200ca510 0x7fcf2015b9e0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf2015d270 0x7fcf200cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 --2- 172.21.15.44:0/3629081182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf04003aa0 0x7fcf2015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 >> 172.21.15.44:0/3629081182 conn(0x7fcf200cfec0 msgr2=0x7fcf200d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 shutdown_connections 2024-07-17T08:41:50.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.136+0000 7fcf1fe00640 1 -- 172.21.15.44:0/3629081182 wait complete. 2024-07-17T08:41:50.156 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 -- 172.21.15.44:0/385239631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 msgr2=0x7f15cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 --2- 172.21.15.44:0/385239631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15cc003e20 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009f00 tx=0x7f15cc005830 comp rx=0 tx=0).stop 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ea200640 1 -- 172.21.15.44:0/385239631 reap_dead start 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 -- 172.21.15.44:0/385239631 shutdown_connections 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 -- 172.21.15.44:0/385239631 >> 172.21.15.44:0/385239631 conn(0x7f15e406bf40 msgr2=0x7f15e406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 -- 172.21.15.44:0/385239631 shutdown_connections 2024-07-17T08:41:50.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.357+0000 7f15ec200640 1 -- 172.21.15.44:0/385239631 wait complete. 2024-07-17T08:41:50.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.360+0000 7f15ec200640 1 Processor -- start 2024-07-17T08:41:50.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.360+0000 7f15ec200640 1 -- start start 2024-07-17T08:41:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.361+0000 7f15ec200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.361+0000 7f15ec200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15e414e0c0 0x7f15e4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15ec200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e4153080 0x7f15e414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15ec200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15e415aa90 con 0x7f15cc003a00 2024-07-17T08:41:50.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15ec200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15e415a910 con 0x7f15e4153080 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15ec200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15e415ac10 con 0x7f15e414e0c0 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33656/0 (socket says 172.21.15.44:33656) 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 -- 172.21.15.44:0/1607274246 learned_addr learned my addr 172.21.15.44:0/1607274246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15ea200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e4153080 0x7f15e414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e8e00640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15e414e0c0 0x7f15e4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15e414e0c0 msgr2=0x7f15e4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15e414e0c0 0x7f15e4152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e4153080 msgr2=0x7f15e414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e4153080 0x7f15e414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.362+0000 7f15e9800640 1 -- 172.21.15.44:0/1607274246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e414be40 con 0x7f15cc003a00 2024-07-17T08:41:50.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15e9800640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009f00 tx=0x7f15d8033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:50.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8043070 con 0x7f15cc003a00 2024-07-17T08:41:50.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d8003b70 con 0x7f15cc003a00 2024-07-17T08:41:50.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8038040 con 0x7f15cc003a00 2024-07-17T08:41:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41f7ce0 con 0x7f15cc003a00 2024-07-17T08:41:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.363+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41f82a0 con 0x7f15cc003a00 2024-07-17T08:41:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.364+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e4068ae0 con 0x7f15cc003a00 2024-07-17T08:41:50.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.366+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f15d8046070 con 0x7f15cc003a00 2024-07-17T08:41:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.367+0000 7f15e1600640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 0x7f15c4079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.367+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f15d80b9bf0 con 0x7f15cc003a00 2024-07-17T08:41:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.367+0000 7f15e8e00640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 msgr2=0x7f15c4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:50.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.367+0000 7f15e8e00640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 0x7f15c4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:50.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.370+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f15d804d020 con 0x7f15cc003a00 2024-07-17T08:41:50.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.569+0000 7f15e8e00640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 msgr2=0x7f15c4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:50.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.569+0000 7f15e8e00640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 0x7f15c4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:50.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.578+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15e40cc4c0 con 0x7f15cc003a00 2024-07-17T08:41:50.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.581+0000 7f15e1600640 1 -- 172.21.15.44:0/1607274246 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v309) ==== 74+0+18371 (secure 0 0 0) 0x7f15d80822e0 con 0x7f15cc003a00 2024-07-17T08:41:50.583 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:50.583 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":309,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:44.012319+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:41:27.625746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":42,"pool_name":"cephfs_data","create_time":"2024-07-17T08:41:28.667439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"308","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1315508424":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/530094003":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/666245803":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/3453853590":"2024-07-18T08:41:44.012240+0000","172.21.15.175:6837/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:6834/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/97927255":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/2166198562":"2024-07-18T08:41:44.012240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:50.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.583+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 msgr2=0x7f15c4079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.583+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 0x7f15c4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.583+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 msgr2=0x7f15e41589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.583+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009f00 tx=0x7f15d8033dd0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 shutdown_connections 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f15c4076e80 0x7f15c4079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15e4153080 0x7f15e414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15e414e0c0 0x7f15e4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 --2- 172.21.15.44:0/1607274246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15cc003a00 0x7f15e41589e0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 >> 172.21.15.44:0/1607274246 conn(0x7f15e406bf40 msgr2=0x7f15e414c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:50.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 shutdown_connections 2024-07-17T08:41:50.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.584+0000 7f15ec200640 1 -- 172.21.15.44:0/1607274246 wait complete. 2024-07-17T08:41:50.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:41:50.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.800+0000 7f1594e00640 1 -- 172.21.15.44:0/2176956309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 msgr2=0x7f1590140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.800+0000 7f1594e00640 1 --2- 172.21.15.44:0/2176956309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f1590140030 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1578009830 tx=0x7f157802a510 comp rx=0 tx=0).stop 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 -- 172.21.15.44:0/2176956309 shutdown_connections 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 --2- 172.21.15.44:0/2176956309 >> v2:172.21.15.175:3300/0 conn(0x7f1590140570 0x7f1590140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 --2- 172.21.15.44:0/2176956309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f1590140030 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 -- 172.21.15.44:0/2176956309 >> 172.21.15.44:0/2176956309 conn(0x7f15900b0e80 msgr2=0x7f15900b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 -- 172.21.15.44:0/2176956309 shutdown_connections 2024-07-17T08:41:50.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.801+0000 7f1594e00640 1 -- 172.21.15.44:0/2176956309 wait complete. 2024-07-17T08:41:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.803+0000 7f1594e00640 1 Processor -- start 2024-07-17T08:41:50.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.803+0000 7f1594e00640 1 -- start start 2024-07-17T08:41:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f159013c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1590140570 0x7f1590071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15900aea10 con 0x7f15900cf540 2024-07-17T08:41:50.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15900ac270 con 0x7f1590140570 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1594e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1590140f40 con 0x7f15900cf920 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33660/0 (socket says 172.21.15.44:33660) 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 -- 172.21.15.44:0/3624911182 learned_addr learned my addr 172.21.15.44:0/3624911182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158fe00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1590140570 0x7f1590071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f1587e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f159013c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 msgr2=0x7f159013c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f159013c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1590140570 msgr2=0x7f1590071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1590140570 0x7f1590071e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.804+0000 7f158f400640 1 -- 172.21.15.44:0/3624911182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159013ce90 con 0x7f15900cf540 2024-07-17T08:41:50.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f158f400640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f158000c420 tx=0x7f158000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580017070 con 0x7f15900cf540 2024-07-17T08:41:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15800053c0 con 0x7f15900cf540 2024-07-17T08:41:50.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580003850 con 0x7f15900cf540 2024-07-17T08:41:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159009c790 con 0x7f15900cf540 2024-07-17T08:41:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.805+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159009ca30 con 0x7f15900cf540 2024-07-17T08:41:50.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.806+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15901413f0 con 0x7f15900cf540 2024-07-17T08:41:50.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.809+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f158001a070 con 0x7f15900cf540 2024-07-17T08:41:50.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.810+0000 7f1587400640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 0x7f156c079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.810+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f15800980e0 con 0x7f15900cf540 2024-07-17T08:41:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.810+0000 7f1587e00640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 msgr2=0x7f156c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:50.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.810+0000 7f1587e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 0x7f156c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:50.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:50.814+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1580060d70 con 0x7f15900cf540 2024-07-17T08:41:51.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.012+0000 7f1587e00640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 msgr2=0x7f156c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:51.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.012+0000 7f1587e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 0x7f156c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:51.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.066+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1590140ca0 con 0x7f15900cf540 2024-07-17T08:41:51.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.066+0000 7f1587400640 1 -- 172.21.15.44:0/3624911182 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7f1580060670 con 0x7f15900cf540 2024-07-17T08:41:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:51.069 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:51.069 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 msgr2=0x7f156c079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 0x7f156c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 msgr2=0x7f1590073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f158000c420 tx=0x7f158000c8e0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 shutdown_connections 2024-07-17T08:41:51.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f156c076e80 0x7f156c079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1590140570 0x7f1590071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15900cf920 0x7f159013c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 --2- 172.21.15.44:0/3624911182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15900cf540 0x7f1590073e70 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 >> 172.21.15.44:0/3624911182 conn(0x7f15900b0e80 msgr2=0x7f15900b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 shutdown_connections 2024-07-17T08:41:51.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.070+0000 7f1594e00640 1 -- 172.21.15.44:0/3624911182 wait complete. 2024-07-17T08:41:51.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:41:51.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.287+0000 7f815a800640 1 -- 172.21.15.44:0/3790726748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540b8c80 msgr2=0x7f81540b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.287+0000 7f815a800640 1 --2- 172.21.15.44:0/3790726748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540b8c80 0x7f81540b4ec0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f8148009860 tx=0x7f814802a540 comp rx=0 tx=0).stop 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 -- 172.21.15.44:0/3790726748 shutdown_connections 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 --2- 172.21.15.44:0/3790726748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540ce5c0 0x7f815406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 --2- 172.21.15.44:0/3790726748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540b8c80 0x7f81540b4ec0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 -- 172.21.15.44:0/3790726748 >> 172.21.15.44:0/3790726748 conn(0x7f81540cfec0 msgr2=0x7f81540d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 -- 172.21.15.44:0/3790726748 shutdown_connections 2024-07-17T08:41:51.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.288+0000 7f815a800640 1 -- 172.21.15.44:0/3790726748 wait complete. 2024-07-17T08:41:51.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.290+0000 7f815a800640 1 Processor -- start 2024-07-17T08:41:51.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.290+0000 7f815a800640 1 -- start start 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81540b5400 0x7f81540ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540b8c80 0x7f81540b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f815406e780 con 0x7f81540ce5c0 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f815406e600 con 0x7f81540b5400 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f815a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f815406e900 con 0x7f81540b8c80 2024-07-17T08:41:51.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f8159e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f8159e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33674/0 (socket says 172.21.15.44:33674) 2024-07-17T08:41:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.292+0000 7f8159e00640 1 -- 172.21.15.44:0/3240669999 learned_addr learned my addr 172.21.15.44:0/3240669999 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159400640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81540b5400 0x7f81540ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8158a00640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540b8c80 0x7f81540b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540b8c80 msgr2=0x7f81540b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540b8c80 0x7f81540b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81540b5400 msgr2=0x7f81540ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81540b5400 0x7f81540ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 -- 172.21.15.44:0/3240669999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81540cb1d0 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8159e00640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f814c00cbc0 tx=0x7f814c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c016020 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.293+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f814c00add0 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.294+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c012680 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.294+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81540aa870 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.294+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81540aae30 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.295+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815415e520 con 0x7f81540ce5c0 2024-07-17T08:41:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.296+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f814c019020 con 0x7f81540ce5c0 2024-07-17T08:41:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.297+0000 7f8151600640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 0x7f8134079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.298+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f814c099120 con 0x7f81540ce5c0 2024-07-17T08:41:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.298+0000 7f8159400640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 msgr2=0x7f8134079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:51.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.298+0000 7f8159400640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 0x7f8134079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:51.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.301+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f814c020030 con 0x7f81540ce5c0 2024-07-17T08:41:51.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.499+0000 7f8159400640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 msgr2=0x7f8134079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:51.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.499+0000 7f8159400640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 0x7f8134079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:51.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.547+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81540b5d70 con 0x7f81540ce5c0 2024-07-17T08:41:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.548+0000 7f8151600640 1 -- 172.21.15.44:0/3240669999 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5246 (secure 0 0 0) 0x7f814c061cd0 con 0x7f81540ce5c0 2024-07-17T08:41:51.550 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:51.550 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":172,"btime":"2024-07-17T08:41:33:225041+0000","default_fscid":20,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:41:29.658222+0000","modified":"2024-07-17T08:41:32.729647+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12654},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12654":{"gid":12654,"name":"d","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1168128599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1168128599},{"type":"v1","addr":"172.21.15.175:6801","nonce":1168128599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[42],"metadata_pool":41,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12654,"qdb_cluster":[12654]},"id":20}]} 2024-07-17T08:41:51.550 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 172 2024-07-17T08:41:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 msgr2=0x7f8134079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 0x7f8134079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 msgr2=0x7f815415a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f814c00cbc0 tx=0x7f814c00a430 comp rx=0 tx=0).stop 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 shutdown_connections 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f8134076e80 0x7f8134079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81540ce5c0 0x7f815415a440 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81540b8c80 0x7f81540b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 --2- 172.21.15.44:0/3240669999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81540b5400 0x7f81540ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 >> 172.21.15.44:0/3240669999 conn(0x7f81540cfec0 msgr2=0x7f81540d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.551+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 shutdown_connections 2024-07-17T08:41:51.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.552+0000 7f815a800640 1 -- 172.21.15.44:0/3240669999 wait complete. 2024-07-17T08:41:51.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 -- 172.21.15.44:0/3077217895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80b9060 msgr2=0x7f38c815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 --2- 172.21.15.44:0/3077217895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80b9060 0x7f38c815d800 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009be0 tx=0x7f38c002a8e0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 -- 172.21.15.44:0/3077217895 shutdown_connections 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 --2- 172.21.15.44:0/3077217895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80b9060 0x7f38c815d800 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 --2- 172.21.15.44:0/3077217895 >> v2:172.21.15.175:3300/0 conn(0x7f38c80ce5c0 0x7f38c815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 -- 172.21.15.44:0/3077217895 >> 172.21.15.44:0/3077217895 conn(0x7f38c80cfec0 msgr2=0x7f38c80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:51.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 -- 172.21.15.44:0/3077217895 shutdown_connections 2024-07-17T08:41:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.767+0000 7f38c7200640 1 -- 172.21.15.44:0/3077217895 wait complete. 2024-07-17T08:41:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.768+0000 7f38c7200640 1 Processor -- start 2024-07-17T08:41:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.769+0000 7f38c7200640 1 -- start start 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.769+0000 7f38c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80b8c80 0x7f38c80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.769+0000 7f38c7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.769+0000 7f38c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80ce5c0 0x7f38c8159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.769+0000 7f38c7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38c815e0d0 con 0x7f38c80b9060 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38c815df50 con 0x7f38c80b8c80 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38c815e250 con 0x7f38c80ce5c0 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33680/0 (socket says 172.21.15.44:33680) 2024-07-17T08:41:51.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 -- 172.21.15.44:0/775527400 learned_addr learned my addr 172.21.15.44:0/775527400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38c5e00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80b8c80 0x7f38c80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38c6800640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80ce5c0 0x7f38c8159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80ce5c0 msgr2=0x7f38c8159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80ce5c0 0x7f38c8159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80b8c80 msgr2=0x7f38c80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80b8c80 0x7f38c80cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.770+0000 7f38bfe00640 1 -- 172.21.15.44:0/775527400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c815aa50 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38bfe00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f38b000e9f0 tx=0x7f38b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b000cd80 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b0004020 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c81f8040 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0004470 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.771+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c81f8600 con 0x7f38c80b9060 2024-07-17T08:41:51.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.772+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c80cdd00 con 0x7f38c80b9060 2024-07-17T08:41:51.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.774+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f38b00035f0 con 0x7f38c80b9060 2024-07-17T08:41:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.775+0000 7f38be000640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 0x7f38a4079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.775+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f38b0099610 con 0x7f38c80b9060 2024-07-17T08:41:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.775+0000 7f38c5e00640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 msgr2=0x7f38a4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:51.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.775+0000 7f38c5e00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 0x7f38a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:51.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.779+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f38c80cdd00 con 0x7f38c80b9060 2024-07-17T08:41:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.976+0000 7f38c5e00640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 msgr2=0x7f38a4079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:51.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.976+0000 7f38c5e00640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 0x7f38a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:41:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.989+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38c815e700 con 0x7f38c80b9060 2024-07-17T08:41:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.990+0000 7f38be000640 1 -- 172.21.15.44:0/775527400 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v309) ==== 74+0+18371 (secure 0 0 0) 0x7f38b00621d0 con 0x7f38c80b9060 2024-07-17T08:41:51.992 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:51.992 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":309,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:44.012319+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":41,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:41:27.625746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":42,"pool_name":"cephfs_data","create_time":"2024-07-17T08:41:28.667439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"308","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1315508424":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/530094003":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/666245803":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/3453853590":"2024-07-18T08:41:44.012240+0000","172.21.15.175:6837/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:6834/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/97927255":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/2166198562":"2024-07-18T08:41:44.012240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:51.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 msgr2=0x7f38a4079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:51.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 0x7f38a4079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 msgr2=0x7f38c81f79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f38b000e9f0 tx=0x7f38b000eeb0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 shutdown_connections 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f38a4076e80 0x7f38a4079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c80ce5c0 0x7f38c8159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38c80b9060 0x7f38c81f79b0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 --2- 172.21.15.44:0/775527400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80b8c80 0x7f38c80cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.993+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 >> 172.21.15.44:0/775527400 conn(0x7f38c80cfec0 msgr2=0x7f38c80b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.994+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 shutdown_connections 2024-07-17T08:41:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:51.994+0000 7f38c7200640 1 -- 172.21.15.44:0/775527400 wait complete. 2024-07-17T08:41:52.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 -- 172.21.15.44:0/2315519495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0b1a10 msgr2=0x7f98bc06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 --2- 172.21.15.44:0/2315519495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0b1a10 0x7f98bc06d690 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009830 tx=0x7f98a802a540 comp rx=0 tx=0).stop 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 -- 172.21.15.44:0/2315519495 shutdown_connections 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 --2- 172.21.15.44:0/2315519495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0b1a10 0x7f98bc06d690 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 -- 172.21.15.44:0/2315519495 >> 172.21.15.44:0/2315519495 conn(0x7f98bc073d50 msgr2=0x7f98bc09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 -- 172.21.15.44:0/2315519495 shutdown_connections 2024-07-17T08:41:52.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.213+0000 7f98c1c00640 1 -- 172.21.15.44:0/2315519495 wait complete. 2024-07-17T08:41:52.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.215+0000 7f98c1c00640 1 Processor -- start 2024-07-17T08:41:52.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.215+0000 7f98c1c00640 1 -- start start 2024-07-17T08:41:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc06dbd0 0x7f98bc1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc15d810 0x7f98bc15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc0aed90 con 0x7f98bc0b1a10 2024-07-17T08:41:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc0b2ba0 con 0x7f98bc06dbd0 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.216+0000 7f98c1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc0aef10 con 0x7f98bc15d810 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33696/0 (socket says 172.21.15.44:33696) 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 -- 172.21.15.44:0/3868776977 learned_addr learned my addr 172.21.15.44:0/3868776977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98baa00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc06dbd0 0x7f98bc1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98bb400640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc15d810 0x7f98bc15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 -- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc15d810 msgr2=0x7f98bc15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc15d810 0x7f98bc15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 -- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc06dbd0 msgr2=0x7f98bc1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc06dbd0 0x7f98bc1442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.217+0000 7f98ba000640 1 -- 172.21.15.44:0/3868776977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc146ab0 con 0x7f98bc0b1a10 2024-07-17T08:41:52.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98ba000640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009950 tx=0x7f98a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005c90 con 0x7f98bc0b1a10 2024-07-17T08:41:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a8005020 con 0x7f98bc0b1a10 2024-07-17T08:41:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005450 con 0x7f98bc0b1a10 2024-07-17T08:41:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc146d90 con 0x7f98bc0b1a10 2024-07-17T08:41:52.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.218+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc147350 con 0x7f98bc0b1a10 2024-07-17T08:41:52.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.220+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc0af400 con 0x7f98bc0b1a10 2024-07-17T08:41:52.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.222+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 185) ==== 99090+0+0 (secure 0 0 0) 0x7f98a8039040 con 0x7f98bc0b1a10 2024-07-17T08:41:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.223+0000 7f98b2a00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 0x7f989c079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.223+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(309..309 src has 1..309) ==== 7049+0+0 (secure 0 0 0) 0x7f98a80b9c20 con 0x7f98bc0b1a10 2024-07-17T08:41:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.223+0000 7f98baa00640 1 -- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 msgr2=0x7f989c079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2231139025 2024-07-17T08:41:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.223+0000 7f98baa00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 0x7f989c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:41:52.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.227+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98a8040030 con 0x7f98bc0b1a10 2024-07-17T08:41:52.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.341+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 186) ==== 98540+0+0 (secure 0 0 0) 0x7f98a807e420 con 0x7f98bc0b1a10 2024-07-17T08:41:52.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.341+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 msgr2=0x7f989c079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:41:52.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.341+0000 7f98b2a00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 0x7f989c079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.467+0000 7feaf7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.467+0000 7feaf7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.467+0000 7feaf7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.467+0000 7feaf7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.468 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.467+0000 7feaf7400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.468+0000 7feafa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.468+0000 7feafa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.468+0000 7feafa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.468+0000 7feafa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.469 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:41:52.468+0000 7feafa600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:41:52.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.517+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f98bc1583d0 con 0x7f98bc0b1a10 2024-07-17T08:41:52.520 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:41:52.518+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:41:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.520+0000 7f98b2a00640 1 -- 172.21.15.44:0/3868776977 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v173) ==== 161+0+0 (secure 0 0 0) 0x7f98a80b21a0 con 0x7f98bc0b1a10 2024-07-17T08:41:52.522 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:41:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 msgr2=0x7f98bc15c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009950 tx=0x7f98a8003c10 comp rx=0 tx=0).stop 2024-07-17T08:41:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 shutdown_connections 2024-07-17T08:41:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:6834/2231139025,v1:172.21.15.175:6837/2231139025] conn(0x7f989c076eb0 0x7f989c079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc15d810 0x7f98bc15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98bc0b1a10 0x7f98bc15c270 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 --2- 172.21.15.44:0/3868776977 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc06dbd0 0x7f98bc1442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 >> 172.21.15.44:0/3868776977 conn(0x7f98bc073d50 msgr2=0x7f98bc0afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.524+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 shutdown_connections 2024-07-17T08:41:52.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.525+0000 7f98c1c00640 1 -- 172.21.15.44:0/3868776977 wait complete. 2024-07-17T08:41:52.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:41:52.588 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:41:52.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.727+0000 7f2457200640 1 -- 172.21.15.44:0/1225430480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f243c004d40 msgr2=0x7f243c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.727+0000 7f2457200640 1 --2- 172.21.15.44:0/1225430480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f243c004d40 0x7f243c0051c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f243c005830 comp rx=0 tx=0).stop 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.727+0000 7f2456800640 1 -- 172.21.15.44:0/1225430480 reap_dead start 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.728+0000 7f2457200640 1 -- 172.21.15.44:0/1225430480 shutdown_connections 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.728+0000 7f2457200640 1 -- 172.21.15.44:0/1225430480 >> 172.21.15.44:0/1225430480 conn(0x7f245806c120 msgr2=0x7f245806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.728+0000 7f2457200640 1 -- 172.21.15.44:0/1225430480 shutdown_connections 2024-07-17T08:41:52.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.728+0000 7f2457200640 1 -- 172.21.15.44:0/1225430480 wait complete. 2024-07-17T08:41:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.729+0000 7f2457200640 1 Processor -- start 2024-07-17T08:41:52.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 -- start start 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24580b2c70 0x7f24581f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24581f7b30 0x7f24580714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f245806f7b0 con 0x7f245814d760 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f245806f230 con 0x7f24580b2c70 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.730+0000 7f2457200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24580ae8a0 con 0x7f24581f7b30 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33712/0 (socket says 172.21.15.44:33712) 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 -- 172.21.15.44:0/2339768725 learned_addr learned my addr 172.21.15.44:0/2339768725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455400640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24580b2c70 0x7f24581f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2456800640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24581f7b30 0x7f24580714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 -- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24581f7b30 msgr2=0x7f24580714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24581f7b30 0x7f24580714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 -- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24580b2c70 msgr2=0x7f24581f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24580b2c70 0x7f24581f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:52.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.731+0000 7f2455e00640 1 -- 172.21.15.44:0/2339768725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24581f4080 con 0x7f245814d760 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f2455e00640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f2448010a70 tx=0x7f2448010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244800ecc0 con 0x7f245814d760 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f244800ee60 con 0x7f245814d760 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448005280 con 0x7f245814d760 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581f4360 con 0x7f245814d760 2024-07-17T08:41:52.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.732+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581f3d20 con 0x7f245814d760 2024-07-17T08:41:52.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.733+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24580b2140 con 0x7f245814d760 2024-07-17T08:41:52.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.734+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 186) ==== 98540+0+0 (secure 0 0 0) 0x7f2448005420 con 0x7f245814d760 2024-07-17T08:41:52.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.735+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(311..311 src has 1..311) ==== 7608+0+0 (secure 0 0 0) 0x7f2448099370 con 0x7f245814d760 2024-07-17T08:41:52.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.740+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f24480623a0 con 0x7f245814d760 2024-07-17T08:41:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:52.994+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2458151f70 con 0x7f245814d760 2024-07-17T08:41:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.350+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 187) ==== 98667+0+0 (secure 0 0 0) 0x7f244805d610 con 0x7f245814d760 2024-07-17T08:41:53.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.351+0000 7f244e000640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 0x7f243807f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:53.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.353+0000 7f2455400640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 0x7f243807f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:53.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.364+0000 7f2455400640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 0x7f243807f700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f24440018e0 tx=0x7f2444009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:53.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.527+0000 7f244e000640 1 -- 172.21.15.44:0/2339768725 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v174) ==== 104+0+0 (secure 0 0 0) 0x7f24480553e0 con 0x7f245814d760 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 msgr2=0x7f243807f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 0x7f243807f700 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f24440018e0 tx=0x7f2444009040 comp rx=0 tx=0).stop 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 msgr2=0x7f24580b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f2448010a70 tx=0x7f2448010f30 comp rx=0 tx=0).stop 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 shutdown_connections 2024-07-17T08:41:53.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f243807d310 0x7f243807f700 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24581f7b30 0x7f24580714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24580b2c70 0x7f24581f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 --2- 172.21.15.44:0/2339768725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245814d760 0x7f24580b2730 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.531+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 >> 172.21.15.44:0/2339768725 conn(0x7f245806c120 msgr2=0x7f2458157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.532+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 shutdown_connections 2024-07-17T08:41:53.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.532+0000 7f2457200640 1 -- 172.21.15.44:0/2339768725 wait complete. 2024-07-17T08:41:53.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:41:53.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 -- 172.21.15.44:0/3960296836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 msgr2=0x7f562c0abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:53.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 --2- 172.21.15.44:0/3960296836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c0abfa0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f5624005fd0 tx=0x7f562402a640 comp rx=0 tx=0).stop 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 -- 172.21.15.44:0/3960296836 shutdown_connections 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 --2- 172.21.15.44:0/3960296836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c0abfa0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 --2- 172.21.15.44:0/3960296836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 0x7f562c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 -- 172.21.15.44:0/3960296836 >> 172.21.15.44:0/3960296836 conn(0x7f562c0cf050 msgr2=0x7f562c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 -- 172.21.15.44:0/3960296836 shutdown_connections 2024-07-17T08:41:53.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.764+0000 7f562b200640 1 -- 172.21.15.44:0/3960296836 wait complete. 2024-07-17T08:41:53.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.766+0000 7f562b200640 1 Processor -- start 2024-07-17T08:41:53.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.766+0000 7f562b200640 1 -- start start 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f562c1537d0 0x7f562c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 0x7f562c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0ac660 con 0x7f562c0ae650 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0ac4e0 con 0x7f562c1537d0 2024-07-17T08:41:53.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.767+0000 7f562b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f562c0ac7e0 con 0x7f562c153bb0 2024-07-17T08:41:53.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:53.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33740/0 (socket says 172.21.15.44:33740) 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 -- 172.21.15.44:0/3048719898 learned_addr learned my addr 172.21.15.44:0/3048719898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629400640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f562c1537d0 0x7f562c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f562a800640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 0x7f562c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 -- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 msgr2=0x7f562c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 0x7f562c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 -- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f562c1537d0 msgr2=0x7f562c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f562c1537d0 0x7f562c158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.768+0000 7f5629e00640 1 -- 172.21.15.44:0/3048719898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1f80d0 con 0x7f562c0ae650 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f5629e00640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c430 tx=0x7f561c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c011790 con 0x7f562c0ae650 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f561c017070 con 0x7f562c0ae650 2024-07-17T08:41:53.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1443a0 con 0x7f562c0ae650 2024-07-17T08:41:53.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c011d00 con 0x7f562c0ae650 2024-07-17T08:41:53.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.769+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c144960 con 0x7f562c0ae650 2024-07-17T08:41:53.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.770+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c0abfa0 con 0x7f562c0ae650 2024-07-17T08:41:53.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.771+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 187) ==== 98667+0+0 (secure 0 0 0) 0x7f561c01c460 con 0x7f562c0ae650 2024-07-17T08:41:53.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.772+0000 7f5622000640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 0x7f5608078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:53.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.772+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(311..311 src has 1..311) ==== 7608+0+0 (secure 0 0 0) 0x7f561c0985e0 con 0x7f562c0ae650 2024-07-17T08:41:53.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.773+0000 7f5629400640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 0x7f5608078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:53.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.774+0000 7f5629400640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 0x7f5608078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614005850 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:53.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.776+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f561c061680 con 0x7f562c0ae650 2024-07-17T08:41:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:53.984+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f562c144b30 con 0x7f562c0ae650 2024-07-17T08:41:54.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.540+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 188) ==== 98875+0+0 (secure 0 0 0) 0x7f561c05d210 con 0x7f562c0ae650 2024-07-17T08:41:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.551+0000 7f5622000640 1 -- 172.21.15.44:0/3048719898 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v312) ==== 181+0+0 (secure 0 0 0) 0x7f561c065580 con 0x7f562c0ae650 2024-07-17T08:41:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:41:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.556+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 msgr2=0x7f5608078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.556+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 0x7f5608078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614005850 comp rx=0 tx=0).stop 2024-07-17T08:41:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.556+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 msgr2=0x7f562c071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.556+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c430 tx=0x7f561c00c8f0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 shutdown_connections 2024-07-17T08:41:54.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5608076a00 0x7f5608078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f562c153bb0 0x7f562c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f562c1537d0 0x7f562c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 --2- 172.21.15.44:0/3048719898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f562c0ae650 0x7f562c071630 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.557+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 >> 172.21.15.44:0/3048719898 conn(0x7f562c0cf050 msgr2=0x7f562c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:54.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.558+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 shutdown_connections 2024-07-17T08:41:54.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.558+0000 7f562b200640 1 -- 172.21.15.44:0/3048719898 wait complete. 2024-07-17T08:41:54.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:41:54.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 -- 172.21.15.44:0/4261932118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 msgr2=0x7f656c0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:54.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 --2- 172.21.15.44:0/4261932118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c0b5200 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f655402a940 comp rx=0 tx=0).stop 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 -- 172.21.15.44:0/4261932118 shutdown_connections 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 --2- 172.21.15.44:0/4261932118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c0b5200 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 --2- 172.21.15.44:0/4261932118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c15d600 0x7f656c06e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 -- 172.21.15.44:0/4261932118 >> 172.21.15.44:0/4261932118 conn(0x7f656c0d00a0 msgr2=0x7f656c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.782+0000 7f6573000640 1 -- 172.21.15.44:0/4261932118 shutdown_connections 2024-07-17T08:41:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.783+0000 7f6573000640 1 -- 172.21.15.44:0/4261932118 wait complete. 2024-07-17T08:41:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.784+0000 7f6573000640 1 Processor -- start 2024-07-17T08:41:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.784+0000 7f6573000640 1 -- start start 2024-07-17T08:41:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 0x7f656c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f656c15d600 0x7f656c1f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f656c161730 con 0x7f656c0b2ad0 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f656c0cef10 con 0x7f656c15d600 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.785+0000 7f6573000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f656c1618b0 con 0x7f656c0b5740 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33760/0 (socket says 172.21.15.44:33760) 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 -- 172.21.15.44:0/44623437 learned_addr learned my addr 172.21.15.44:0/44623437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656b400640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 0x7f656c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f6571000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f656c15d600 0x7f656c1f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 -- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 msgr2=0x7f656c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 0x7f656c149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 -- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f656c15d600 msgr2=0x7f656c1f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f656c15d600 0x7f656c1f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656be00640 1 -- 172.21.15.44:0/44623437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c1495c0 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.786+0000 7f656b400640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 0x7f656c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f656be00640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f655c00d740 tx=0x7f655c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c017070 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f655c009d20 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0106c0 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c1f7fb0 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.787+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1f8570 con 0x7f656c0b2ad0 2024-07-17T08:41:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.788+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656c0b6150 con 0x7f656c0b2ad0 2024-07-17T08:41:54.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.789+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 188) ==== 98875+0+0 (secure 0 0 0) 0x7f655c01a070 con 0x7f656c0b2ad0 2024-07-17T08:41:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.790+0000 7f6563e00640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 0x7f6548079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.791+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(312..312 src has 1..312) ==== 7170+0+0 (secure 0 0 0) 0x7f655c098a20 con 0x7f656c0b2ad0 2024-07-17T08:41:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.791+0000 7f656b400640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 0x7f6548079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.793+0000 7f656b400640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 0x7f6548079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6554034000 tx=0x7f6554004ab0 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:54.795+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f656c0b6150 con 0x7f656c0b2ad0 2024-07-17T08:41:55.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.006+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f656c161d60 con 0x7f656c0b2ad0 2024-07-17T08:41:55.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.555+0000 7f6563e00640 1 -- 172.21.15.44:0/44623437 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v313) ==== 169+0+0 (secure 0 0 0) 0x7f655c0618c0 con 0x7f656c0b2ad0 2024-07-17T08:41:55.557 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:41:55.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.560+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 msgr2=0x7f6548079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.560+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 0x7f6548079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6554034000 tx=0x7f6554004ab0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.560+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 msgr2=0x7f656c15dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.560+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f655c00d740 tx=0x7f655c00dc00 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 shutdown_connections 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6548076c40 0x7f6548079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f656c15d600 0x7f656c1f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f656c0b5740 0x7f656c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 --2- 172.21.15.44:0/44623437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f656c0b2ad0 0x7f656c15dae0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 >> 172.21.15.44:0/44623437 conn(0x7f656c0d00a0 msgr2=0x7f656c0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:55.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.561+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 shutdown_connections 2024-07-17T08:41:55.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.562+0000 7f6573000640 1 -- 172.21.15.44:0/44623437 wait complete. 2024-07-17T08:41:55.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:41:55.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 -- 172.21.15.44:0/2679181072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20440051c0 msgr2=0x7f204400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:55.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 --2- 172.21.15.44:0/2679181072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20440051c0 0x7f204400b9c0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f2058001dc0 tx=0x7f2044005830 comp rx=0 tx=0).stop 2024-07-17T08:41:55.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 -- 172.21.15.44:0/2679181072 shutdown_connections 2024-07-17T08:41:55.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 --2- 172.21.15.44:0/2679181072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20440051c0 0x7f204400b9c0 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 -- 172.21.15.44:0/2679181072 >> 172.21.15.44:0/2679181072 conn(0x7f20600d3700 msgr2=0x7f20600d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:55.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 -- 172.21.15.44:0/2679181072 shutdown_connections 2024-07-17T08:41:55.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.787+0000 7f2066200640 1 -- 172.21.15.44:0/2679181072 wait complete. 2024-07-17T08:41:55.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.789+0000 7f2066200640 1 Processor -- start 2024-07-17T08:41:55.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.789+0000 7f2066200640 1 -- start start 2024-07-17T08:41:55.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20440051c0 0x7f20600c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20601f3e40 0x7f20600b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f20600ce940 con 0x7f20601f7430 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20600ce7c0 con 0x7f20601f3e40 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f2066200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20600ceac0 con 0x7f20440051c0 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f205ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f205ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33780/0 (socket says 172.21.15.44:33780) 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.790+0000 7f205ea00640 1 -- 172.21.15.44:0/1799929023 learned_addr learned my addr 172.21.15.44:0/1799929023 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205f400640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20440051c0 0x7f20600c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205fe00640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20601f3e40 0x7f20600b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 -- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20440051c0 msgr2=0x7f20600c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20440051c0 0x7f20600c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 -- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20601f3e40 msgr2=0x7f20600b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:55.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20601f3e40 0x7f20600b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:55.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 -- 172.21.15.44:0/1799929023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206015fac0 con 0x7f20601f7430 2024-07-17T08:41:55.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f205ea00640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d7d0 tx=0x7f204c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:55.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.791+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c019070 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.792+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f204c0050d0 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.792+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20601f7930 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.792+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c00b660 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.792+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20601f7ef0 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.793+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2060161730 con 0x7f20601f7430 2024-07-17T08:41:55.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.794+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 188) ==== 98875+0+0 (secure 0 0 0) 0x7f204c01c070 con 0x7f20601f7430 2024-07-17T08:41:55.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.795+0000 7f2057400640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 0x7f2040079130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:55.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.795+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f204c097ef0 con 0x7f20601f7430 2024-07-17T08:41:55.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.796+0000 7f205f400640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 0x7f2040079130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:55.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.797+0000 7f205f400640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 0x7f2040079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2050009a20 tx=0x7f2050005f90 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:55.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:55.800+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f204c060eb0 con 0x7f20601f7430 2024-07-17T08:41:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.012+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f20601ea210 con 0x7f20601f7430 2024-07-17T08:41:56.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.021+0000 7f2057400640 1 -- 172.21.15.44:0/1799929023 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v95) ==== 71+0+0 (secure 0 0 0) 0x7f204c0607b0 con 0x7f20601f7430 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 msgr2=0x7f2040079130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 0x7f2040079130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2050009a20 tx=0x7f2050005f90 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 msgr2=0x7f206015f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d7d0 tx=0x7f204c00dc90 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 shutdown_connections 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2040076c70 0x7f2040079130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20601f3e40 0x7f20600b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20601f7430 0x7f206015f1d0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 --2- 172.21.15.44:0/1799929023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20440051c0 0x7f20600c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 >> 172.21.15.44:0/1799929023 conn(0x7f20600d3700 msgr2=0x7f206015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:56.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 shutdown_connections 2024-07-17T08:41:56.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.024+0000 7f2066200640 1 -- 172.21.15.44:0/1799929023 wait complete. 2024-07-17T08:41:56.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03' 2024-07-17T08:41:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.259+0000 7fac46800640 1 -- 172.21.15.44:0/829034271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 msgr2=0x7fac400b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.259+0000 7fac46800640 1 --2- 172.21.15.44:0/829034271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac400b25d0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fac38009ca0 tx=0x7fac3802aa30 comp rx=0 tx=0).stop 2024-07-17T08:41:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.259+0000 7fac46800640 1 -- 172.21.15.44:0/829034271 shutdown_connections 2024-07-17T08:41:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.259+0000 7fac46800640 1 --2- 172.21.15.44:0/829034271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac400b25d0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.259+0000 7fac46800640 1 -- 172.21.15.44:0/829034271 >> 172.21.15.44:0/829034271 conn(0x7fac400b0fb0 msgr2=0x7fac400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.260+0000 7fac46800640 1 -- 172.21.15.44:0/829034271 shutdown_connections 2024-07-17T08:41:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.260+0000 7fac46800640 1 -- 172.21.15.44:0/829034271 wait complete. 2024-07-17T08:41:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.261+0000 7fac46800640 1 Processor -- start 2024-07-17T08:41:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.261+0000 7fac46800640 1 -- start start 2024-07-17T08:41:56.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac4006ea50 0x7fac40072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:56.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:56.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac400ae810 0x7fac401f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:56.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac400aed30 con 0x7fac400ac930 2024-07-17T08:41:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fac4006f0d0 con 0x7fac400ae810 2024-07-17T08:41:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.262+0000 7fac46800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fac400b2b10 con 0x7fac4006ea50 2024-07-17T08:41:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33800/0 (socket says 172.21.15.44:33800) 2024-07-17T08:41:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 -- 172.21.15.44:0/2609508669 learned_addr learned my addr 172.21.15.44:0/2609508669 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac3fe00640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac400ae810 0x7fac401f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac3f400640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac4006ea50 0x7fac40072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 -- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac4006ea50 msgr2=0x7fac40072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac4006ea50 0x7fac40072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 -- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac400ae810 msgr2=0x7fac401f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac400ae810 0x7fac401f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.263+0000 7fac37e00640 1 -- 172.21.15.44:0/2609508669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac40073580 con 0x7fac400ac930 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac37e00640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c440 tx=0x7fac2800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28017070 con 0x7fac400ac930 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac280035f0 con 0x7fac400ac930 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28003a40 con 0x7fac400ac930 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac40073860 con 0x7fac400ac930 2024-07-17T08:41:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.264+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac40073e20 con 0x7fac400ac930 2024-07-17T08:41:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.265+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac400b2f30 con 0x7fac400ac930 2024-07-17T08:41:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.267+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 188) ==== 98875+0+0 (secure 0 0 0) 0x7fac28003be0 con 0x7fac400ac930 2024-07-17T08:41:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.268+0000 7fac37400640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 0x7fac1c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.268+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7fac28097b10 con 0x7fac400ac930 2024-07-17T08:41:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.269+0000 7fac3f400640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 0x7fac1c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:56.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.270+0000 7fac3f400640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 0x7fac1c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac300015a0 tx=0x7fac30009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:56.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.273+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fac28060bd0 con 0x7fac400ac930 2024-07-17T08:41:56.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:56.567+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03"]} v 0) -- 0x7fac400b23b0 con 0x7fac400ac930 2024-07-17T08:41:57.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.039+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fac2805c760 con 0x7fac400ac930 2024-07-17T08:41:57.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.561+0000 7fac37400640 1 -- 172.21.15.44:0/2609508669 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_03"]}]=0 v3578) ==== 201+0+0 (secure 0 0 0) 0x7fac280010c0 con 0x7fac400ac930 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 msgr2=0x7fac1c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 0x7fac1c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac300015a0 tx=0x7fac30009040 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 msgr2=0x7fac401f82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c440 tx=0x7fac2800c900 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 shutdown_connections 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fac1c076c40 0x7fac1c079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac400ae810 0x7fac401f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac400ac930 0x7fac401f82e0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 --2- 172.21.15.44:0/2609508669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fac4006ea50 0x7fac40072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 >> 172.21.15.44:0/2609508669 conn(0x7fac400b0fb0 msgr2=0x7fac400b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:57.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 shutdown_connections 2024-07-17T08:41:57.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.565+0000 7fac46800640 1 -- 172.21.15.44:0/2609508669 wait complete. 2024-07-17T08:41:57.591 INFO:tasks.cephfs_test_runner:Test subvol schedule creation fails when subvol exists only under default group. ... ok 2024-07-17T08:41:57.591 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_04 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule creation fails without subvol argument. 2024-07-17T08:41:57.592 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_04 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:41:57.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:41:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 -- 172.21.15.44:0/741560334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 msgr2=0x7f34f0073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 --2- 172.21.15.44:0/741560334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f0073c30 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005fd0 tx=0x7f34e402a670 comp rx=0 tx=0).stop 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 -- 172.21.15.44:0/741560334 shutdown_connections 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 --2- 172.21.15.44:0/741560334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f0074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 --2- 172.21.15.44:0/741560334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f0073c30 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 -- 172.21.15.44:0/741560334 >> 172.21.15.44:0/741560334 conn(0x7f34f00d0350 msgr2=0x7f34f00d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.808+0000 7f34f7600640 1 -- 172.21.15.44:0/741560334 shutdown_connections 2024-07-17T08:41:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.809+0000 7f34f7600640 1 -- 172.21.15.44:0/741560334 wait complete. 2024-07-17T08:41:57.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.810+0000 7f34f7600640 1 Processor -- start 2024-07-17T08:41:57.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.810+0000 7f34f7600640 1 -- start start 2024-07-17T08:41:57.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 0x7f34f015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:57.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:57.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:57.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00d2a30 con 0x7f34f015d7e0 2024-07-17T08:41:57.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00b2580 con 0x7f34f00b28a0 2024-07-17T08:41:57.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.811+0000 7f34f7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34f00d2bb0 con 0x7f34f00b6d50 2024-07-17T08:41:57.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:57.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33824/0 (socket says 172.21.15.44:33824) 2024-07-17T08:41:57.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 -- 172.21.15.44:0/3313724537 learned_addr learned my addr 172.21.15.44:0/3313724537 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f4c00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 0x7f34f015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 -- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 msgr2=0x7f34f015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34efe00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 -- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 msgr2=0x7f34f015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 0x7f34f015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f5600640 1 -- 172.21.15.44:0/3313724537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f01f5230 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34f4c00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 0x7f34f015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.812+0000 7f34efe00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34f5600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009fd0 tx=0x7f34e800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8004d20 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e800cb20 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8005020 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f01f46b0 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.813+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f01f4c70 con 0x7f34f015d7e0 2024-07-17T08:41:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.814+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f00d3140 con 0x7f34f015d7e0 2024-07-17T08:41:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.815+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f34e80051c0 con 0x7f34f015d7e0 2024-07-17T08:41:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.816+0000 7f34ecc00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 0x7f34d4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.817+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f34e8099290 con 0x7f34f015d7e0 2024-07-17T08:41:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.817+0000 7f34f4c00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 0x7f34d4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.818+0000 7f34f4c00640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 0x7f34d4079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f34e4033000 tx=0x7f34e4004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:57.821+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34e8016030 con 0x7f34f015d7e0 2024-07-17T08:41:58.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.031+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34f0069450 con 0x7f34f015d7e0 2024-07-17T08:41:58.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.033+0000 7f34ecc00640 1 -- 172.21.15.44:0/3313724537 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v313) ==== 74+0+15416 (secure 0 0 0) 0x7f34e8062110 con 0x7f34f015d7e0 2024-07-17T08:41:58.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:41:58.035 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":313,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:41:55.552588+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":42,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":306,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/3428137178":"2024-07-18T08:41:52.328434+0000","172.21.15.175:6834/2231139025":"2024-07-18T08:41:52.328434+0000","172.21.15.175:6800/1168128599":"2024-07-18T08:41:52.519607+0000","172.21.15.175:6837/2231139025":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/1046030030":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/776083098":"2024-07-18T08:41:52.328434+0000","172.21.15.175:6801/1168128599":"2024-07-18T08:41:52.519607+0000","172.21.15.175:0/1805444746":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/2669996804":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/1740702979":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/1315508424":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/530094003":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/130847171":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/666245803":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/3453853590":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/654849825":"2024-07-18T08:41:52.328434+0000","172.21.15.175:6837/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:6834/3743840884":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/303853874":"2024-07-18T08:41:52.328434+0000","172.21.15.175:0/97927255":"2024-07-18T08:41:44.012240+0000","172.21.15.175:0/2166198562":"2024-07-18T08:41:44.012240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:41:58.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 msgr2=0x7f34d4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 0x7f34d4079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f34e4033000 tx=0x7f34e4004730 comp rx=0 tx=0).stop 2024-07-17T08:41:58.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 msgr2=0x7f34f015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009fd0 tx=0x7f34e800ee60 comp rx=0 tx=0).stop 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 shutdown_connections 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f34d4076cd0 0x7f34d4079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34f015d7e0 0x7f34f015b980 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34f00b6d50 0x7f34f015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 --2- 172.21.15.44:0/3313724537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34f00b28a0 0x7f34f015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.036+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 >> 172.21.15.44:0/3313724537 conn(0x7f34f00d0350 msgr2=0x7f34f00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.037+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 shutdown_connections 2024-07-17T08:41:58.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.037+0000 7f34f7600640 1 -- 172.21.15.44:0/3313724537 wait complete. 2024-07-17T08:41:58.060 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:41:58.060 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:41:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 -- 172.21.15.44:0/610289218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc06f380 msgr2=0x7fa6cc0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/610289218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc06f380 0x7fa6cc0cb080 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0b1e60 tx=0x7fa6c402a4a0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 -- 172.21.15.44:0/610289218 shutdown_connections 2024-07-17T08:41:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/610289218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc06f380 0x7fa6cc0cb080 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/610289218 >> v2:172.21.15.175:3300/0 conn(0x7fa6cc151f20 0x7fa6cc15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 -- 172.21.15.44:0/610289218 >> 172.21.15.44:0/610289218 conn(0x7fa6cc0cf0d0 msgr2=0x7fa6cc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 -- 172.21.15.44:0/610289218 shutdown_connections 2024-07-17T08:41:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.275+0000 7fa6d2a00640 1 -- 172.21.15.44:0/610289218 wait complete. 2024-07-17T08:41:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.277+0000 7fa6d2a00640 1 Processor -- start 2024-07-17T08:41:58.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.277+0000 7fa6d2a00640 1 -- start start 2024-07-17T08:41:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6cc06f380 0x7fa6cc1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6cc0cd0b0 0x7fa6cc14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc0cb740 con 0x7fa6cc151f20 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc0cb5c0 con 0x7fa6cc0cd0b0 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6cc0cb8c0 con 0x7fa6cc06f380 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33850/0 (socket says 172.21.15.44:33850) 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.278+0000 7fa6d2000640 1 -- 172.21.15.44:0/3051613485 learned_addr learned my addr 172.21.15.44:0/3051613485 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d0c00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6cc0cd0b0 0x7fa6cc14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 -- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6cc06f380 msgr2=0x7fa6cc1f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6cc06f380 0x7fa6cc1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 -- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6cc0cd0b0 msgr2=0x7fa6cc14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6cc0cd0b0 0x7fa6cc14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 -- 172.21.15.44:0/3051613485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc09d030 con 0x7fa6cc151f20 2024-07-17T08:41:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6d2000640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009920 tx=0x7fa6c4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4040070 con 0x7fa6cc151f20 2024-07-17T08:41:58.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6c4005330 con 0x7fa6cc151f20 2024-07-17T08:41:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.279+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40047a0 con 0x7fa6cc151f20 2024-07-17T08:41:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.280+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc06e470 con 0x7fa6cc151f20 2024-07-17T08:41:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.280+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc06ea30 con 0x7fa6cc151f20 2024-07-17T08:41:58.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.281+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc09b1e0 con 0x7fa6cc151f20 2024-07-17T08:41:58.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.282+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fa6c4039040 con 0x7fa6cc151f20 2024-07-17T08:41:58.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.283+0000 7fa6c9600640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 0x7fa6ac0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.283+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7fa6c40b9950 con 0x7fa6cc151f20 2024-07-17T08:41:58.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.284+0000 7fa6d1600640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 0x7fa6ac0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.285+0000 7fa6d1600640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 0x7fa6ac0791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005c40 tx=0x7fa6c0005bd0 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:58.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.287+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa6c40827d0 con 0x7fa6cc151f20 2024-07-17T08:41:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.505+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6cc0cb080 con 0x7fa6cc151f20 2024-07-17T08:41:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.506+0000 7fa6c9600640 1 -- 172.21.15.44:0/3051613485 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v313) ==== 93+0+10 (secure 0 0 0) 0x7fa6c40820d0 con 0x7fa6cc151f20 2024-07-17T08:41:58.508 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:41:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 msgr2=0x7fa6ac0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 0x7fa6ac0791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005c40 tx=0x7fa6c0005bd0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 msgr2=0x7fa6cc09b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009920 tx=0x7fa6c4032b90 comp rx=0 tx=0).stop 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 shutdown_connections 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fa6ac076d00 0x7fa6ac0791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6cc151f20 0x7fa6cc09b5b0 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6cc0cd0b0 0x7fa6cc14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 --2- 172.21.15.44:0/3051613485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6cc06f380 0x7fa6cc1f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.509+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 >> 172.21.15.44:0/3051613485 conn(0x7fa6cc0cf0d0 msgr2=0x7fa6cc15d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.510+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 shutdown_connections 2024-07-17T08:41:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.510+0000 7fa6d2a00640 1 -- 172.21.15.44:0/3051613485 wait complete. 2024-07-17T08:41:58.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04' 2024-07-17T08:41:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 -- 172.21.15.44:0/395973159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15d300 msgr2=0x7f124c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 --2- 172.21.15.44:0/395973159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15d300 0x7f124c06e3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1240005fd0 tx=0x7f124002a4b0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 -- 172.21.15.44:0/395973159 shutdown_connections 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 --2- 172.21.15.44:0/395973159 >> v2:172.21.15.175:3300/0 conn(0x7f124c15ddf0 0x7f124c0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 --2- 172.21.15.44:0/395973159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15d300 0x7f124c06e3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 -- 172.21.15.44:0/395973159 >> 172.21.15.44:0/395973159 conn(0x7f124c0d00a0 msgr2=0x7f124c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.744+0000 7f1251e00640 1 -- 172.21.15.44:0/395973159 shutdown_connections 2024-07-17T08:41:58.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.745+0000 7f1251e00640 1 -- 172.21.15.44:0/395973159 wait complete. 2024-07-17T08:41:58.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.746+0000 7f1251e00640 1 Processor -- start 2024-07-17T08:41:58.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.746+0000 7f1251e00640 1 -- start start 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 0x7f124c0b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15ddf0 0x7f124c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0d1660 con 0x7f124c0b4ec0 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0d14e0 con 0x7f124c15d300 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.747+0000 7f1251e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f124c0d17e0 con 0x7f124c15ddf0 2024-07-17T08:41:58.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33866/0 (socket says 172.21.15.44:33866) 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 -- 172.21.15.44:0/1210989509 learned_addr learned my addr 172.21.15.44:0/1210989509 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f124be00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 0x7f124c0b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1251400640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15ddf0 0x7f124c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 -- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15ddf0 msgr2=0x7f124c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15ddf0 0x7f124c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 -- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 msgr2=0x7f124c0b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 0x7f124c0b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f1250a00640 1 -- 172.21.15.44:0/1210989509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c0cc810 con 0x7f124c0b4ec0 2024-07-17T08:41:58.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.748+0000 7f124be00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 0x7f124c0b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1250a00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f1240009850 tx=0x7f12400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003f070 con 0x7f124c0b4ec0 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12400047a0 con 0x7f124c0b4ec0 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240035030 con 0x7f124c0b4ec0 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c0ccaf0 con 0x7f124c0b4ec0 2024-07-17T08:41:58.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.749+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c0ca990 con 0x7f124c0b4ec0 2024-07-17T08:41:58.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.750+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c0ccd30 con 0x7f124c0b4ec0 2024-07-17T08:41:58.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.752+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f1240042070 con 0x7f124c0b4ec0 2024-07-17T08:41:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.753+0000 7f1248c00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 0x7f122c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:58.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.753+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f12400b9b20 con 0x7f124c0b4ec0 2024-07-17T08:41:58.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.753+0000 7f124be00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 0x7f122c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:58.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.754+0000 7f124be00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 0x7f122c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1238007fd0 tx=0x7f1238009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:58.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:58.757+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f124c0ccd30 con 0x7f124c0b4ec0 2024-07-17T08:41:59.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.054+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04"]} v 0) -- 0x7f124c0cf3a0 con 0x7f124c0b4ec0 2024-07-17T08:41:59.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.577+0000 7f1248c00640 1 -- 172.21.15.44:0/1210989509 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04"]}]=0 v3580) ==== 204+0+0 (secure 0 0 0) 0x7f124c0cf3a0 con 0x7f124c0b4ec0 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 msgr2=0x7f122c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 0x7f122c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1238007fd0 tx=0x7f1238009040 comp rx=0 tx=0).stop 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 msgr2=0x7f124c0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f1240009850 tx=0x7f12400038a0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 shutdown_connections 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f122c076cd0 0x7f122c079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f124c15ddf0 0x7f124c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124c15d300 0x7f124c0b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 --2- 172.21.15.44:0/1210989509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c0b4ec0 0x7f124c0caeb0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 >> 172.21.15.44:0/1210989509 conn(0x7f124c0d00a0 msgr2=0x7f124c15f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 shutdown_connections 2024-07-17T08:41:59.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.581+0000 7f1251e00640 1 -- 172.21.15.44:0/1210989509 wait complete. 2024-07-17T08:41:59.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:41:59.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 -- 172.21.15.44:0/3705157532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 msgr2=0x7f6a840ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 --2- 172.21.15.44:0/3705157532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a840ce080 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005fd0 tx=0x7f6a6c02a540 comp rx=0 tx=0).stop 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 -- 172.21.15.44:0/3705157532 shutdown_connections 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 --2- 172.21.15.44:0/3705157532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a840ce080 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 -- 172.21.15.44:0/3705157532 >> 172.21.15.44:0/3705157532 conn(0x7f6a840d3400 msgr2=0x7f6a840d3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 -- 172.21.15.44:0/3705157532 shutdown_connections 2024-07-17T08:41:59.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.813+0000 7f6a89800640 1 -- 172.21.15.44:0/3705157532 wait complete. 2024-07-17T08:41:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.815+0000 7f6a89800640 1 Processor -- start 2024-07-17T08:41:59.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.815+0000 7f6a89800640 1 -- start start 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a840ce5c0 0x7f6a841f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 0x7f6a841f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8415dae0 con 0x7f6a840b64d0 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8415d960 con 0x7f6a8415d580 2024-07-17T08:41:59.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a89800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a840d2b40 con 0x7f6a840ce5c0 2024-07-17T08:41:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.816+0000 7f6a83e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45784/0 (socket says 172.21.15.44:45784) 2024-07-17T08:41:59.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 -- 172.21.15.44:0/3137423961 learned_addr learned my addr 172.21.15.44:0/3137423961 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a88e00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 0x7f6a841f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a7b400640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a840ce5c0 0x7f6a841f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 -- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a840ce5c0 msgr2=0x7f6a841f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a840ce5c0 0x7f6a841f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 -- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 msgr2=0x7f6a841f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 0x7f6a841f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 -- 172.21.15.44:0/3137423961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a841f4e30 con 0x7f6a840b64d0 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a88e00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 0x7f6a841f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:41:59.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.817+0000 7f6a83e00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400b3f0 tx=0x7f6a7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.818+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74015020 con 0x7f6a840b64d0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.818+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a74011920 con 0x7f6a840b64d0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.818+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74011d70 con 0x7f6a840b64d0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.818+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a841f56e0 con 0x7f6a840b64d0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.818+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a841f5f20 con 0x7f6a840b64d0 2024-07-17T08:41:59.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.819+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a840d3060 con 0x7f6a840b64d0 2024-07-17T08:41:59.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.820+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f6a74018020 con 0x7f6a840b64d0 2024-07-17T08:41:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.821+0000 7f6a80c00640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 0x7f6a60079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:41:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.822+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f6a74098610 con 0x7f6a840b64d0 2024-07-17T08:41:59.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.822+0000 7f6a7b400640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 0x7f6a60079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:41:59.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.823+0000 7f6a7b400640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 0x7f6a60079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c033000 tx=0x7f6a6c004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:41:59.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:41:59.826+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6a7401f030 con 0x7f6a840b64d0 2024-07-17T08:42:00.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.040+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f6a840d3f80 con 0x7f6a840b64d0 2024-07-17T08:42:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.041+0000 7f6a80c00640 1 -- 172.21.15.44:0/3137423961 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v95) ==== 59+0+1998 (secure 0 0 0) 0x7f6a841f56e0 con 0x7f6a840b64d0 2024-07-17T08:42:00.043 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:42:00.043 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:42:00.043 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:42:00.043 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:42:00.044 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:42:00.045 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:42:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 msgr2=0x7f6a60079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 0x7f6a60079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c033000 tx=0x7f6a6c004730 comp rx=0 tx=0).stop 2024-07-17T08:42:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 msgr2=0x7f6a841f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400b3f0 tx=0x7f6a7400b8b0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 shutdown_connections 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6a60076cd0 0x7f6a60079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a8415d580 0x7f6a841f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a840ce5c0 0x7f6a841f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 --2- 172.21.15.44:0/3137423961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a840b64d0 0x7f6a841f5030 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 >> 172.21.15.44:0/3137423961 conn(0x7f6a840d3400 msgr2=0x7f6a840b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 shutdown_connections 2024-07-17T08:42:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.044+0000 7f6a89800640 1 -- 172.21.15.44:0/3137423961 wait complete. 2024-07-17T08:42:00.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:42:00.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 -- 172.21.15.44:0/3538852590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c151f20 msgr2=0x7fd43c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 --2- 172.21.15.44:0/3538852590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c151f20 0x7fd43c15c140 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c02a540 comp rx=0 tx=0).stop 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 -- 172.21.15.44:0/3538852590 shutdown_connections 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 --2- 172.21.15.44:0/3538852590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c0cd0b0 0x7fd43c06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 --2- 172.21.15.44:0/3538852590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c151f20 0x7fd43c15c140 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 -- 172.21.15.44:0/3538852590 >> 172.21.15.44:0/3538852590 conn(0x7fd43c0cf0d0 msgr2=0x7fd43c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.282+0000 7fd441000640 1 -- 172.21.15.44:0/3538852590 shutdown_connections 2024-07-17T08:42:00.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.283+0000 7fd441000640 1 -- 172.21.15.44:0/3538852590 wait complete. 2024-07-17T08:42:00.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.284+0000 7fd441000640 1 Processor -- start 2024-07-17T08:42:00.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.284+0000 7fd441000640 1 -- start start 2024-07-17T08:42:00.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd43c0cd0b0 0x7fd43c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:00.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 0x7fd43c14a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:00.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:00.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0cb790 con 0x7fd43c152300 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c06f610 con 0x7fd43c0cd0b0 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.285+0000 7fd441000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c0cb910 con 0x7fd43c151f20 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45800/0 (socket says 172.21.15.44:45800) 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 -- 172.21.15.44:0/286899502 learned_addr learned my addr 172.21.15.44:0/286899502 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43b400640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd43c0cd0b0 0x7fd43c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43aa00640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 0x7fd43c14a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 -- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 msgr2=0x7fd43c14a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 0x7fd43c14a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 -- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd43c0cd0b0 msgr2=0x7fd43c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd43c0cd0b0 0x7fd43c1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43be00640 1 -- 172.21.15.44:0/286899502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c155160 con 0x7fd43c152300 2024-07-17T08:42:00.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.286+0000 7fd43aa00640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 0x7fd43c14a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:00.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd43be00640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fd43400b410 tx=0x7fd43400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:00.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434015030 con 0x7fd43c152300 2024-07-17T08:42:00.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd434004020 con 0x7fd43c152300 2024-07-17T08:42:00.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434004470 con 0x7fd43c152300 2024-07-17T08:42:00.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c14a810 con 0x7fd43c152300 2024-07-17T08:42:00.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.287+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c0728e0 con 0x7fd43c152300 2024-07-17T08:42:00.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.288+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c06ec70 con 0x7fd43c152300 2024-07-17T08:42:00.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.290+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fd434018070 con 0x7fd43c152300 2024-07-17T08:42:00.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.290+0000 7fd433400640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 0x7fd41c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:00.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.291+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7fd4340984f0 con 0x7fd43c152300 2024-07-17T08:42:00.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.291+0000 7fd43b400640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 0x7fd41c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:00.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.292+0000 7fd43b400640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 0x7fd41c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd42c033000 tx=0x7fd42c004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:00.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.295+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd43c14a810 con 0x7fd43c152300 2024-07-17T08:42:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.500+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd43c074ee0 con 0x7fd43c152300 2024-07-17T08:42:00.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.507+0000 7fd433400640 1 -- 172.21.15.44:0/286899502 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v96)=0 v96) ==== 122+0+0 (secure 0 0 0) 0x7fd434061480 con 0x7fd43c152300 2024-07-17T08:42:00.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 msgr2=0x7fd41c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 0x7fd41c079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd42c033000 tx=0x7fd42c004730 comp rx=0 tx=0).stop 2024-07-17T08:42:00.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 msgr2=0x7fd43c154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:00.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fd43400b410 tx=0x7fd43400b8d0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 shutdown_connections 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd41c076cd0 0x7fd41c079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd43c152300 0x7fd43c154390 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd43c151f20 0x7fd43c14a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 --2- 172.21.15.44:0/286899502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd43c0cd0b0 0x7fd43c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.510+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 >> 172.21.15.44:0/286899502 conn(0x7fd43c0cf0d0 msgr2=0x7fd43c15a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.511+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 shutdown_connections 2024-07-17T08:42:00.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:00.511+0000 7fd441000640 1 -- 172.21.15.44:0/286899502 wait complete. 2024-07-17T08:42:00.532 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:42:00.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:00.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:00.783 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:42:00.783 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:00.784 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:00.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:42:00.844 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:00.844 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:00.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:00.898 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:00.898 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:00.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:42:00.957 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:00.957 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:01.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:01.012 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:42:01.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:01.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.235+0000 7f1fcfe00640 1 -- 172.21.15.44:0/1378136975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4004b60 msgr2=0x7f1fb4004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.235+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/1378136975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4004b60 0x7f1fb4004f40 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0001740 tx=0x7f1fb4006940 comp rx=0 tx=0).stop 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.235+0000 7f1fcf400640 1 -- 172.21.15.44:0/1378136975 reap_dead start 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.235+0000 7f1fcfe00640 1 -- 172.21.15.44:0/1378136975 shutdown_connections 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.235+0000 7f1fcfe00640 1 -- 172.21.15.44:0/1378136975 >> 172.21.15.44:0/1378136975 conn(0x7f1fd00b0e80 msgr2=0x7f1fd00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.236+0000 7f1fcfe00640 1 -- 172.21.15.44:0/1378136975 shutdown_connections 2024-07-17T08:42:01.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.236+0000 7f1fcfe00640 1 -- 172.21.15.44:0/1378136975 wait complete. 2024-07-17T08:42:01.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.237+0000 7f1fcfe00640 1 Processor -- start 2024-07-17T08:42:01.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.237+0000 7f1fcfe00640 1 -- start start 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.238+0000 7f1fcfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4004b60 0x7f1fd01f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.238+0000 7f1fcfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fd01569c0 0x7f1fd0150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00cf4d0 con 0x7f1fd01412e0 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00cf350 con 0x7f1fd01569c0 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd00cf650 con 0x7f1fb4004b60 2024-07-17T08:42:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45828/0 (socket says 172.21.15.44:45828) 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 -- 172.21.15.44:0/431417155 learned_addr learned my addr 172.21.15.44:0/431417155 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcea00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4004b60 0x7f1fd01f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fce000640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fd01569c0 0x7f1fd0150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 -- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4004b60 msgr2=0x7f1fd01f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4004b60 0x7f1fd01f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 -- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fd01569c0 msgr2=0x7f1fd0150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fd01569c0 0x7f1fd0150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.239+0000 7f1fcf400640 1 -- 172.21.15.44:0/431417155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd0157130 con 0x7f1fd01412e0 2024-07-17T08:42:01.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fcf400640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f1fd015d1b0 tx=0x7f1fc800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:01.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8018070 con 0x7f1fd01412e0 2024-07-17T08:42:01.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fc800a780 con 0x7f1fd01412e0 2024-07-17T08:42:01.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800abd0 con 0x7f1fd01412e0 2024-07-17T08:42:01.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd0157410 con 0x7f1fd01412e0 2024-07-17T08:42:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.240+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd01579d0 con 0x7f1fd01412e0 2024-07-17T08:42:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.241+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd009b0b0 con 0x7f1fd01412e0 2024-07-17T08:42:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.243+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f1fc8004030 con 0x7f1fd01412e0 2024-07-17T08:42:01.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.244+0000 7f1fc7400640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 0x7f1fb0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.244+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f1fc80982c0 con 0x7f1fd01412e0 2024-07-17T08:42:01.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.244+0000 7f1fcea00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 0x7f1fb0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.246+0000 7f1fcea00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 0x7f1fb0079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0001710 tx=0x7f1fc0001270 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:01.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.248+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1fc80611c0 con 0x7f1fd01412e0 2024-07-17T08:42:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.498+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fd00af300 con 0x7f1fd01412e0 2024-07-17T08:42:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.499+0000 7f1fc7400640 1 -- 172.21.15.44:0/431417155 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+3924 (secure 0 0 0) 0x7f1fc8060ac0 con 0x7f1fd01412e0 2024-07-17T08:42:01.501 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:01.501 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":174,"btime":"2024-07-17T08:41:53:521938+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1783227889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1783227889},{"type":"v1","addr":"172.21.15.44:6834","nonce":1783227889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":9344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/1570006981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1570006981},{"type":"v1","addr":"172.21.15.175:6801","nonce":1570006981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":174},{"gid":10075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3171301306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":3171301306},{"type":"v1","addr":"172.21.15.44:6835","nonce":3171301306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":10084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1726892891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1726892891},{"type":"v1","addr":"172.21.15.175:6836","nonce":1726892891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[]} 2024-07-17T08:42:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 174 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 msgr2=0x7f1fb0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 0x7f1fb0079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0001710 tx=0x7f1fc0001270 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 msgr2=0x7f1fd0141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f1fd015d1b0 tx=0x7f1fc800a060 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 shutdown_connections 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f1fb0076cd0 0x7f1fb0079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fd01412e0 0x7f1fd0141770 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fd01569c0 0x7f1fd0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.502+0000 7f1fcfe00640 1 --2- 172.21.15.44:0/431417155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fb4004b60 0x7f1fd01f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.503+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 >> 172.21.15.44:0/431417155 conn(0x7f1fd00b0e80 msgr2=0x7f1fd015a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.503+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 shutdown_connections 2024-07-17T08:42:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.503+0000 7f1fcfe00640 1 -- 172.21.15.44:0/431417155 wait complete. 2024-07-17T08:42:01.527 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:42:01.527 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:42:01.527 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:42:01.528 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:01.528 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:42:01.528 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:42:01.528 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:42:01.529 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:01.529 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:42:01.529 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:42:01.529 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:42:01.530 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:01.530 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:42:01.530 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:42:01.530 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:42:01.531 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:01.548 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:42:01.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:42:01.549 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:42:01.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:42:01.551 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:42:01.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:42:01.554 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:42:01.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:42:01.597 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:42:01.597 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:01.605 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:42:01.607 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:01.607 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:42:01.608 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:01.611 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:42:01.611 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:01.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:42:01.748 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:42:01.748 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:42:01.756 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:42:01.759 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:42:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 -- 172.21.15.44:0/2879287389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce80b3f20 msgr2=0x7f2ce8161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 --2- 172.21.15.44:0/2879287389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce80b3f20 0x7f2ce8161020 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005fd0 tx=0x7f2cd002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 -- 172.21.15.44:0/2879287389 shutdown_connections 2024-07-17T08:42:01.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 --2- 172.21.15.44:0/2879287389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce80b3f20 0x7f2ce8161020 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 --2- 172.21.15.44:0/2879287389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce806bf40 0x7f2ce806c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 -- 172.21.15.44:0/2879287389 >> 172.21.15.44:0/2879287389 conn(0x7f2ce806d050 msgr2=0x7f2ce806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.791+0000 7f2ceca00640 1 -- 172.21.15.44:0/2879287389 shutdown_connections 2024-07-17T08:42:01.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.792+0000 7f2ceca00640 1 -- 172.21.15.44:0/2879287389 wait complete. 2024-07-17T08:42:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.793+0000 7f2ceca00640 1 Processor -- start 2024-07-17T08:42:01.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.793+0000 7f2ceca00640 1 -- start start 2024-07-17T08:42:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce80b3f20 0x7f2ce80ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 0x7f2ce80adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8161c70 con 0x7f2ce806bf40 2024-07-17T08:42:01.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8161df0 con 0x7f2ce80b3f20 2024-07-17T08:42:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.794+0000 7f2ceca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce806f8f0 con 0x7f2ce80b4360 2024-07-17T08:42:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45870/0 (socket says 172.21.15.44:45870) 2024-07-17T08:42:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 -- 172.21.15.44:0/1740230336 learned_addr learned my addr 172.21.15.44:0/1740230336 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce80b3f20 0x7f2ce80ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce6800640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 0x7f2ce80adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 -- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 msgr2=0x7f2ce80adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 0x7f2ce80adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 -- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce80b3f20 msgr2=0x7f2ce80ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce80b3f20 0x7f2ce80ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce5e00640 1 -- 172.21.15.44:0/1740230336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce8140110 con 0x7f2ce806bf40 2024-07-17T08:42:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.795+0000 7f2ce6800640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 0x7f2ce80adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2ce5e00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800ea20 tx=0x7f2cd800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:01.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800cd80 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cd8010430 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8010840 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce80ae580 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.796+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce8130a70 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.797+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce806c320 con 0x7f2ce806bf40 2024-07-17T08:42:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.798+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f2cd80109e0 con 0x7f2ce806bf40 2024-07-17T08:42:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.799+0000 7f2cde000640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 0x7f2cc4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.799+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6756+0+0 (secure 0 0 0) 0x7f2cd8099560 con 0x7f2ce806bf40 2024-07-17T08:42:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.800+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 0x7f2cc4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.801+0000 7f2cdfe00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 0x7f2cc4079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0033000 tx=0x7f2cd0004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:01.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:01.804+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2cd8062350 con 0x7f2ce806bf40 2024-07-17T08:42:02.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.026+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2ce806ce40 con 0x7f2ce806bf40 2024-07-17T08:42:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.617+0000 7f2cde000640 1 -- 172.21.15.44:0/1740230336 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v315) ==== 97+0+0 (secure 0 0 0) 0x7f2cd8061c50 con 0x7f2ce806bf40 2024-07-17T08:42:02.623 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:42:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 msgr2=0x7f2cc4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 0x7f2cc4079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0033000 tx=0x7f2cd0004730 comp rx=0 tx=0).stop 2024-07-17T08:42:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 msgr2=0x7f2ce80cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800ea20 tx=0x7f2cd800eee0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 shutdown_connections 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cc4076cd0 0x7f2cc4079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce80b4360 0x7f2ce80adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce80b3f20 0x7f2ce80ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 --2- 172.21.15.44:0/1740230336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ce806bf40 0x7f2ce80cc5e0 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.621+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 >> 172.21.15.44:0/1740230336 conn(0x7f2ce806d050 msgr2=0x7f2ce8145250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.622+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 shutdown_connections 2024-07-17T08:42:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.622+0000 7f2ceca00640 1 -- 172.21.15.44:0/1740230336 wait complete. 2024-07-17T08:42:02.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:42:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 -- 172.21.15.44:0/3794017107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99900acaf0 msgr2=0x7f9990159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 --2- 172.21.15.44:0/3794017107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99900acaf0 0x7f9990159f60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9988005fd0 tx=0x7f998802aa90 comp rx=0 tx=0).stop 2024-07-17T08:42:02.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 -- 172.21.15.44:0/3794017107 shutdown_connections 2024-07-17T08:42:02.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 --2- 172.21.15.44:0/3794017107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99900acaf0 0x7f9990159f60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 --2- 172.21.15.44:0/3794017107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 0x7f9990153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 -- 172.21.15.44:0/3794017107 >> 172.21.15.44:0/3794017107 conn(0x7f99900b0fb0 msgr2=0x7f99900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:02.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 -- 172.21.15.44:0/3794017107 shutdown_connections 2024-07-17T08:42:02.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.852+0000 7f9995c00640 1 -- 172.21.15.44:0/3794017107 wait complete. 2024-07-17T08:42:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.854+0000 7f9995c00640 1 Processor -- start 2024-07-17T08:42:02.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.854+0000 7f9995c00640 1 -- start start 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 0x7f999009cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 0x7f99901636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99901643f0 con 0x7f99900acaf0 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9990164270 con 0x7f9990159b00 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f9995c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9990164570 con 0x7f999006f710 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.855+0000 7f998e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:02.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45880/0 (socket says 172.21.15.44:45880) 2024-07-17T08:42:02.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 -- 172.21.15.44:0/2082475763 learned_addr learned my addr 172.21.15.44:0/2082475763 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998ea00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 0x7f999009cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998f400640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 0x7f99901636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 -- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 msgr2=0x7f999009cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 0x7f999009cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 -- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 msgr2=0x7f99901636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 0x7f99901636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 -- 172.21.15.44:0/2082475763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9990150800 con 0x7f99900acaf0 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998f400640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 0x7f99901636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f998e000640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f997c00c3d0 tx=0x7f997c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:02.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.856+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c017070 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.857+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f997c0053c0 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.857+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c003850 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.857+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999009c510 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.857+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99901504b0 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.858+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999009cbd0 con 0x7f99900acaf0 2024-07-17T08:42:02.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.859+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f997c01a070 con 0x7f99900acaf0 2024-07-17T08:42:02.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.860+0000 7f9986a00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 0x7f9970079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:02.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.860+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(317..317 src has 1..317) ==== 5853+0+0 (secure 0 0 0) 0x7f997c097970 con 0x7f99900acaf0 2024-07-17T08:42:02.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.861+0000 7f998ea00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 0x7f9970079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:02.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.862+0000 7f998ea00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 0x7f9970079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9980009aa0 tx=0x7f9980005f90 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:02.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:02.865+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f999015acc0 con 0x7f99900acaf0 2024-07-17T08:42:03.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.149+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f999009c300 con 0x7f99900acaf0 2024-07-17T08:42:03.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.150+0000 7f9986a00640 1 -- 172.21.15.44:0/2082475763 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v78) ==== 80+0+5304 (secure 0 0 0) 0x7f997c060c90 con 0x7f99900acaf0 2024-07-17T08:42:03.152 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:03.152 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:42:03.152 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:42:03.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:03.154 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.155 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:42:03.157 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.158 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:42:03.159 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:42:03.162 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:03.163 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:42:03.164 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:03.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:42:03.166 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:42:03.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 msgr2=0x7f9970079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 0x7f9970079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9980009aa0 tx=0x7f9980005f90 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 msgr2=0x7f9990099610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f997c00c3d0 tx=0x7f997c00c890 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 shutdown_connections 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f9970076cd0 0x7f9970079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990159b00 0x7f99901636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99900acaf0 0x7f9990099610 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 --2- 172.21.15.44:0/2082475763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f999006f710 0x7f999009cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 >> 172.21.15.44:0/2082475763 conn(0x7f99900b0fb0 msgr2=0x7f99900b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:03.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 shutdown_connections 2024-07-17T08:42:03.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.153+0000 7f9995c00640 1 -- 172.21.15.44:0/2082475763 wait complete. 2024-07-17T08:42:03.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:42:03.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.379+0000 7fcedbe00640 1 -- 172.21.15.44:0/2426650681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0004800 msgr2=0x7fcec0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.379+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2426650681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcec0004800 0x7fcec0004c80 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009b50 tx=0x7fcec0005830 comp rx=0 tx=0).stop 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.380+0000 7fcedb400640 1 -- 172.21.15.44:0/2426650681 reap_dead start 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.380+0000 7fcedbe00640 1 -- 172.21.15.44:0/2426650681 shutdown_connections 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.380+0000 7fcedbe00640 1 -- 172.21.15.44:0/2426650681 >> 172.21.15.44:0/2426650681 conn(0x7fcedc0b0e80 msgr2=0x7fcedc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.380+0000 7fcedbe00640 1 -- 172.21.15.44:0/2426650681 shutdown_connections 2024-07-17T08:42:03.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.380+0000 7fcedbe00640 1 -- 172.21.15.44:0/2426650681 wait complete. 2024-07-17T08:42:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.381+0000 7fcedbe00640 1 Processor -- start 2024-07-17T08:42:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.382+0000 7fcedbe00640 1 -- start start 2024-07-17T08:42:03.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcedc139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc154ac0 0x7fcedc1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0ac1e0 con 0x7fcedc13a490 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0ac060 con 0x7fcec0004800 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fcedbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc140030 con 0x7fcedc154ac0 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:03.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45894/0 (socket says 172.21.15.44:45894) 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 -- 172.21.15.44:0/2969034749 learned_addr learned my addr 172.21.15.44:0/2969034749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 -- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc154ac0 msgr2=0x7fcedc1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc154ac0 0x7fcedc1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 -- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 msgr2=0x7fcedc139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcedc139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.383+0000 7fced3e00640 1 -- 172.21.15.44:0/2969034749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc155350 con 0x7fcedc13a490 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fced3e00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fcec4036040 tx=0x7fcec4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4038040 con 0x7fcedc13a490 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec4004020 con 0x7fcedc13a490 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4004470 con 0x7fcedc13a490 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc1555f0 con 0x7fcedc13a490 2024-07-17T08:42:03.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.384+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc139110 con 0x7fcedc13a490 2024-07-17T08:42:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.385+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc0b3870 con 0x7fcedc13a490 2024-07-17T08:42:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.387+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fcec4045020 con 0x7fcedc13a490 2024-07-17T08:42:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.388+0000 7fced2a00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 0x7fceb80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.388+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(317..317 src has 1..317) ==== 5853+0+0 (secure 0 0 0) 0x7fcec40b8bb0 con 0x7fcedc13a490 2024-07-17T08:42:03.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.389+0000 7fcedaa00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 0x7fceb80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:03.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.390+0000 7fcedaa00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 0x7fceb80791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001eb0 tx=0x7fcecc001b20 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:03.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.392+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcec4081f70 con 0x7fcedc13a490 2024-07-17T08:42:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.604+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcedc0aff00 con 0x7fcedc13a490 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.605+0000 7fced2a00640 1 -- 172.21.15.44:0/2969034749 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v317) ==== 84+0+65 (secure 0 0 0) 0x7fcec408a1b0 con 0x7fcedc13a490 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:42:03.607 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:42:03.608 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:03.608 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 msgr2=0x7fceb80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 0x7fceb80791c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001eb0 tx=0x7fcecc001b20 comp rx=0 tx=0).stop 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 msgr2=0x7fcedc0734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fcec4036040 tx=0x7fcec4033670 comp rx=0 tx=0).stop 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 shutdown_connections 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fceb8076d00 0x7fceb80791c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc154ac0 0x7fcedc1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcedc13a490 0x7fcedc0734d0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 --2- 172.21.15.44:0/2969034749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcedc139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 >> 172.21.15.44:0/2969034749 conn(0x7fcedc0b0e80 msgr2=0x7fcedc06ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 shutdown_connections 2024-07-17T08:42:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.608+0000 7fcedbe00640 1 -- 172.21.15.44:0/2969034749 wait complete. 2024-07-17T08:42:03.631 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:42:03.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:42:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 -- 172.21.15.44:0/2374791364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f763015e370 msgr2=0x7f76300ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 --2- 172.21.15.44:0/2374791364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f763015e370 0x7f76300ce500 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f7620005fd0 tx=0x7f762002a4b0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 -- 172.21.15.44:0/2374791364 shutdown_connections 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 --2- 172.21.15.44:0/2374791364 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76300d0b40 0x7f76300b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 --2- 172.21.15.44:0/2374791364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f763015e370 0x7f76300ce500 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 -- 172.21.15.44:0/2374791364 >> 172.21.15.44:0/2374791364 conn(0x7f7630073a60 msgr2=0x7f7630073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 -- 172.21.15.44:0/2374791364 shutdown_connections 2024-07-17T08:42:03.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.838+0000 7f762f200640 1 -- 172.21.15.44:0/2374791364 wait complete. 2024-07-17T08:42:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.840+0000 7f762f200640 1 Processor -- start 2024-07-17T08:42:03.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.841+0000 7f762f200640 1 -- start start 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 0x7f7630152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7630161b50 0x7f7630159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76300b8d30 con 0x7f76300d0b40 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76300b8bb0 con 0x7f763015e370 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76300b8eb0 con 0x7f7630161b50 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45910/0 (socket says 172.21.15.44:45910) 2024-07-17T08:42:03.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.842+0000 7f762de00640 1 -- 172.21.15.44:0/397604906 learned_addr learned my addr 172.21.15.44:0/397604906 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:03.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f7625400640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 0x7f7630152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 -- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7630161b50 msgr2=0x7f7630159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7630161b50 0x7f7630159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 -- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 msgr2=0x7f7630152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 0x7f7630152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 -- 172.21.15.44:0/397604906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7630141760 con 0x7f76300d0b40 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f7625400640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 0x7f7630152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f762de00640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f7620009850 tx=0x7f76200039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762003f070 con 0x7f76300d0b40 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7620004890 con 0x7f76300d0b40 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.843+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620035030 con 0x7f76300d0b40 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.844+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7630141a40 con 0x7f76300d0b40 2024-07-17T08:42:03.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.844+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76301422e0 con 0x7f76300d0b40 2024-07-17T08:42:03.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.845+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76300b9310 con 0x7f76300d0b40 2024-07-17T08:42:03.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.846+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f7620042070 con 0x7f76300d0b40 2024-07-17T08:42:03.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.847+0000 7f7626a00640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:03.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.847+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(317..317 src has 1..317) ==== 5853+0+0 (secure 0 0 0) 0x7f76200b9490 con 0x7f76300d0b40 2024-07-17T08:42:03.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.847+0000 7f7625400640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:03.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.849+0000 7f7625400640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 0x7f760c079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f763015ebe0 tx=0x7f7618009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:03.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:03.851+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f76200826a0 con 0x7f76300d0b40 2024-07-17T08:42:04.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.065+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7630141c10 con 0x7f76300d0b40 2024-07-17T08:42:04.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.621+0000 7f7626a00640 1 -- 172.21.15.44:0/397604906 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v318) ==== 138+0+0 (secure 0 0 0) 0x7f7620081fa0 con 0x7f76300d0b40 2024-07-17T08:42:04.624 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:42:04.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.625+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 msgr2=0x7f760c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:04.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.625+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 0x7f760c079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f763015ebe0 tx=0x7f7618009040 comp rx=0 tx=0).stop 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 msgr2=0x7f76300c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f7620009850 tx=0x7f76200039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 shutdown_connections 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f760c076cd0 0x7f760c079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7630161b50 0x7f7630159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f763015e370 0x7f7630152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 --2- 172.21.15.44:0/397604906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76300d0b40 0x7f76300c8a50 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.626+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 >> 172.21.15.44:0/397604906 conn(0x7f7630073a60 msgr2=0x7f763015b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:04.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.627+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 shutdown_connections 2024-07-17T08:42:04.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.627+0000 7f762f200640 1 -- 172.21.15.44:0/397604906 wait complete. 2024-07-17T08:42:04.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 -- 172.21.15.44:0/503716561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 msgr2=0x7f74a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 --2- 172.21.15.44:0/503716561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74a80051c0 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f74c000bbb0 tx=0x7f74a8005830 comp rx=0 tx=0).stop 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74c6800640 1 -- 172.21.15.44:0/503716561 reap_dead start 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 -- 172.21.15.44:0/503716561 shutdown_connections 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 -- 172.21.15.44:0/503716561 >> 172.21.15.44:0/503716561 conn(0x7f74c80d00f0 msgr2=0x7f74c80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 -- 172.21.15.44:0/503716561 shutdown_connections 2024-07-17T08:42:04.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.866+0000 7f74ccc00640 1 -- 172.21.15.44:0/503716561 wait complete. 2024-07-17T08:42:04.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.868+0000 7f74ccc00640 1 Processor -- start 2024-07-17T08:42:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.868+0000 7f74ccc00640 1 -- start start 2024-07-17T08:42:04.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:04.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c80cd4d0 0x7f74c814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 0x7f74c81483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74c815dca0 con 0x7f74a8004d40 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74c815db20 con 0x7f74c80cd4d0 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74ccc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74c80ca370 con 0x7f74c814b6d0 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45924/0 (socket says 172.21.15.44:45924) 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.869+0000 7f74c5e00640 1 -- 172.21.15.44:0/3894087703 learned_addr learned my addr 172.21.15.44:0/3894087703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c6800640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 0x7f74c81483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5400640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c80cd4d0 0x7f74c814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:04.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 -- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 msgr2=0x7f74c81483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 0x7f74c81483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 -- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c80cd4d0 msgr2=0x7f74c814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c80cd4d0 0x7f74c814b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 -- 172.21.15.44:0/3894087703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c815f400 con 0x7f74a8004d40 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c6800640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 0x7f74c81483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.870+0000 7f74c5e00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f74b80117d0 tx=0x7f74b8011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.871+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004460 con 0x7f74a8004d40 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.871+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74b8016070 con 0x7f74a8004d40 2024-07-17T08:42:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.871+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b800fb00 con 0x7f74a8004d40 2024-07-17T08:42:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.871+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c815f6e0 con 0x7f74a8004d40 2024-07-17T08:42:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.871+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c815fca0 con 0x7f74a8004d40 2024-07-17T08:42:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.872+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c8161020 con 0x7f74a8004d40 2024-07-17T08:42:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.874+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f74b8018030 con 0x7f74a8004d40 2024-07-17T08:42:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.875+0000 7f74be000640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 0x7f74a40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.875+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(318..318 src has 1..318) ==== 6223+0+0 (secure 0 0 0) 0x7f74b8098ea0 con 0x7f74a8004d40 2024-07-17T08:42:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.875+0000 7f74c5400640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 0x7f74a40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.877+0000 7f74c5400640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 0x7f74a40791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74b0002230 tx=0x7f74b0001250 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:04.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:04.880+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f74c8161020 con 0x7f74a8004d40 2024-07-17T08:42:05.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.092+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f74c814bab0 con 0x7f74a8004d40 2024-07-17T08:42:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.628+0000 7f74be000640 1 -- 172.21.15.44:0/3894087703 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v319) ==== 170+0+0 (secure 0 0 0) 0x7f74b8062040 con 0x7f74a8004d40 2024-07-17T08:42:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:42:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.632+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 msgr2=0x7f74a40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 0x7f74a40791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74b0002230 tx=0x7f74b0001250 comp rx=0 tx=0).stop 2024-07-17T08:42:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 msgr2=0x7f74c80aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f74b80117d0 tx=0x7f74b8011c90 comp rx=0 tx=0).stop 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 shutdown_connections 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f74a4076d00 0x7f74a40791c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c814b6d0 0x7f74c81483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c80cd4d0 0x7f74c814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 --2- 172.21.15.44:0/3894087703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a8004d40 0x7f74c80aa260 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.633+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 >> 172.21.15.44:0/3894087703 conn(0x7f74c80d00f0 msgr2=0x7f74c815c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.634+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 shutdown_connections 2024-07-17T08:42:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.635+0000 7f74ccc00640 1 -- 172.21.15.44:0/3894087703 wait complete. 2024-07-17T08:42:05.671 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:42:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 -- 172.21.15.44:0/53130800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 msgr2=0x7f4c50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 --2- 172.21.15.44:0/53130800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c50004c80 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c161550 tx=0x7f4c50005830 comp rx=0 tx=0).stop 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 -- 172.21.15.44:0/53130800 shutdown_connections 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 --2- 172.21.15.44:0/53130800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c50004c80 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 -- 172.21.15.44:0/53130800 >> 172.21.15.44:0/53130800 conn(0x7f4c6c0b0eb0 msgr2=0x7f4c6c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 -- 172.21.15.44:0/53130800 shutdown_connections 2024-07-17T08:42:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.852+0000 7f4c72800640 1 -- 172.21.15.44:0/53130800 wait complete. 2024-07-17T08:42:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.854+0000 7f4c72800640 1 Processor -- start 2024-07-17T08:42:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.854+0000 7f4c72800640 1 -- start start 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6c1f7a90 0x7f4c6c1f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6c15e010 0x7f4c6c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c0b3e60 con 0x7f4c50004800 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c0b3ce0 con 0x7f4c6c15e010 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c72800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c0b3fe0 con 0x7f4c6c1f7a90 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45932/0 (socket says 172.21.15.44:45932) 2024-07-17T08:42:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.855+0000 7f4c6b400640 1 -- 172.21.15.44:0/1018880633 learned_addr learned my addr 172.21.15.44:0/1018880633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:05.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 -- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6c1f7a90 msgr2=0x7f4c6c1f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6aa00640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6c1f7a90 0x7f4c6c1f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6c1f7a90 0x7f4c6c1f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 -- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6c15e010 msgr2=0x7f4c6c15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6c15e010 0x7f4c6c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 -- 172.21.15.44:0/1018880633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c141150 con 0x7f4c50004800 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c6b400640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0117e0 tx=0x7f4c5c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00fd50 con 0x7f4c50004800 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c5c016070 con 0x7f4c50004800 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c004290 con 0x7f4c50004800 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.856+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c09cce0 con 0x7f4c50004800 2024-07-17T08:42:05.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.857+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c1413f0 con 0x7f4c50004800 2024-07-17T08:42:05.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.858+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6c0ac940 con 0x7f4c50004800 2024-07-17T08:42:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.860+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f4c5c018070 con 0x7f4c50004800 2024-07-17T08:42:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.861+0000 7f4c63400640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 0x7f4c4c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.861+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(319..319 src has 1..319) ==== 6605+0+0 (secure 0 0 0) 0x7f4c5c0994c0 con 0x7f4c50004800 2024-07-17T08:42:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.861+0000 7f4c6aa00640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 0x7f4c4c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.863+0000 7f4c6aa00640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 0x7f4c4c0791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4c58001050 tx=0x7f4c58001bc0 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:05.864+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4c5c062460 con 0x7f4c50004800 2024-07-17T08:42:06.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.118+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4c6c160e90 con 0x7f4c50004800 2024-07-17T08:42:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.649+0000 7f4c63400640 1 -- 172.21.15.44:0/1018880633 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 43 and data pool 44 v177) ==== 174+0+0 (secure 0 0 0) 0x7f4c5c061d60 con 0x7f4c50004800 2024-07-17T08:42:06.651 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 43 and data pool 44 2024-07-17T08:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.652+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 msgr2=0x7f4c4c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.652+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 0x7f4c4c0791c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4c58001050 tx=0x7f4c58001bc0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.652+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 msgr2=0x7f4c6c153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.652+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0117e0 tx=0x7f4c5c011ca0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 shutdown_connections 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f4c4c076d00 0x7f4c4c0791c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c6c15e010 0x7f4c6c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c6c1f7a90 0x7f4c6c1f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 --2- 172.21.15.44:0/1018880633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c50004800 0x7f4c6c153240 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 >> 172.21.15.44:0/1018880633 conn(0x7f4c6c0b0eb0 msgr2=0x7f4c6c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.653+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 shutdown_connections 2024-07-17T08:42:06.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.654+0000 7f4c72800640 1 -- 172.21.15.44:0/1018880633 wait complete. 2024-07-17T08:42:06.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 -- 172.21.15.44:0/3991184824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c1522c0 msgr2=0x7f6f5c0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 --2- 172.21.15.44:0/3991184824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0ad790 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c005fd0 tx=0x7f6f4c02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 -- 172.21.15.44:0/3991184824 shutdown_connections 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 --2- 172.21.15.44:0/3991184824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0ac930 0x7f6f5c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 --2- 172.21.15.44:0/3991184824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0ad790 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 -- 172.21.15.44:0/3991184824 >> 172.21.15.44:0/3991184824 conn(0x7f6f5c0b0fb0 msgr2=0x7f6f5c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 -- 172.21.15.44:0/3991184824 shutdown_connections 2024-07-17T08:42:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.873+0000 7f6f62600640 1 -- 172.21.15.44:0/3991184824 wait complete. 2024-07-17T08:42:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.875+0000 7f6f62600640 1 Processor -- start 2024-07-17T08:42:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.875+0000 7f6f62600640 1 -- start start 2024-07-17T08:42:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 0x7f6f5c0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c0cdf60 con 0x7f6f5c0ac930 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c0cdde0 con 0x7f6f5c1522c0 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f62600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f5c0ce0e0 con 0x7f6f5c0adcd0 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f5b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45944/0 (socket says 172.21.15.44:45944) 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.876+0000 7f6f5b400640 1 -- 172.21.15.44:0/3985784846 learned_addr learned my addr 172.21.15.44:0/3985784846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 0x7f6f5c0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 -- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 msgr2=0x7f6f5c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 0x7f6f5c0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 -- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f5c1522c0 msgr2=0x7f6f5c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5be00640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 -- 172.21.15.44:0/3985784846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c0b28f0 con 0x7f6f5c0ac930 2024-07-17T08:42:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 0x7f6f5c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f5b400640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000c00 tx=0x7f6f4c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.877+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c03e030 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.878+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f4c003ca0 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.878+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c035040 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.878+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c0b2b90 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.878+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c0b3150 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.879+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c0ce630 con 0x7f6f5c0ac930 2024-07-17T08:42:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.880+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f6f4c041030 con 0x7f6f5c0ac930 2024-07-17T08:42:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.881+0000 7f6f53400640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 0x7f6f38079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.881+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(320..320 src has 1..320) ==== 6705+0+0 (secure 0 0 0) 0x7f6f4c0b8d90 con 0x7f6f5c0ac930 2024-07-17T08:42:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.882+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 0x7f6f38079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.883+0000 7f6f53e00640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 0x7f6f38079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c073db0 tx=0x7f6f44009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:06.886+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6f4c081c40 con 0x7f6f5c0ac930 2024-07-17T08:42:07.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.092+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f5c0cd930 con 0x7f6f5c0ac930 2024-07-17T08:42:07.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.094+0000 7f6f53400640 1 -- 172.21.15.44:0/3985784846 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v320) ==== 74+0+17819 (secure 0 0 0) 0x7f6f4c081540 con 0x7f6f5c0ac930 2024-07-17T08:42:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:07.097 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":320,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:06.120517+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":43,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:04.096300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":44,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:05.114416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 msgr2=0x7f6f38079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 0x7f6f38079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c073db0 tx=0x7f6f44009040 comp rx=0 tx=0).stop 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 msgr2=0x7f6f5c1f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000c00 tx=0x7f6f4c0038a0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 shutdown_connections 2024-07-17T08:42:07.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6f38076cd0 0x7f6f38079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f5c1522c0 0x7f6f5c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f5c0adcd0 0x7f6f5c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 --2- 172.21.15.44:0/3985784846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f5c0ac930 0x7f6f5c1f3d10 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 >> 172.21.15.44:0/3985784846 conn(0x7f6f5c0b0fb0 msgr2=0x7f6f5c0af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 shutdown_connections 2024-07-17T08:42:07.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.097+0000 7f6f62600640 1 -- 172.21.15.44:0/3985784846 wait complete. 2024-07-17T08:42:07.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 -- 172.21.15.44:0/1341638871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80cf920 msgr2=0x7fceb8141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 --2- 172.21.15.44:0/1341638871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80cf920 0x7fceb8141040 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fceb0005fd0 tx=0x7fceb002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 -- 172.21.15.44:0/1341638871 shutdown_connections 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 --2- 172.21.15.44:0/1341638871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80cf920 0x7fceb8141040 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 --2- 172.21.15.44:0/1341638871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb80cf540 0x7fceb80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 -- 172.21.15.44:0/1341638871 >> 172.21.15.44:0/1341638871 conn(0x7fceb80b0e80 msgr2=0x7fceb80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 -- 172.21.15.44:0/1341638871 shutdown_connections 2024-07-17T08:42:07.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.320+0000 7fcebe600640 1 -- 172.21.15.44:0/1341638871 wait complete. 2024-07-17T08:42:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.322+0000 7fcebe600640 1 Processor -- start 2024-07-17T08:42:07.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.322+0000 7fcebe600640 1 -- start start 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb80cf540 0x7fceb809c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cf920 0x7fceb806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80ae380 con 0x7fceb80b3db0 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fceb80ae200 con 0x7fceb80cf920 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebe600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fceb80ae500 con 0x7fceb80cf540 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebd200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45966/0 (socket says 172.21.15.44:45966) 2024-07-17T08:42:07.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.323+0000 7fcebd200640 1 -- 172.21.15.44:0/1841904646 learned_addr learned my addr 172.21.15.44:0/1841904646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:07.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 -- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb80cf540 msgr2=0x7fceb809c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb80cf540 0x7fceb809c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 -- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cf920 msgr2=0x7fceb806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cf920 0x7fceb806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 -- 172.21.15.44:0/1841904646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb8071610 con 0x7fceb80b3db0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebd200640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fcea800ebc0 tx=0x7fcea800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8010630 con 0x7fceb80b3db0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea80108c0 con 0x7fceb80b3db0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8010d10 con 0x7fceb80b3db0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb80718f0 con 0x7fceb80b3db0 2024-07-17T08:42:07.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.324+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb8071eb0 con 0x7fceb80b3db0 2024-07-17T08:42:07.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.326+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb80b3870 con 0x7fceb80b3db0 2024-07-17T08:42:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.327+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fcea8018420 con 0x7fceb80b3db0 2024-07-17T08:42:07.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.328+0000 7fceb5600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 0x7fce940791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.328+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(320..320 src has 1..320) ==== 6705+0+0 (secure 0 0 0) 0x7fcea80996b0 con 0x7fceb80b3db0 2024-07-17T08:42:07.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.329+0000 7fceafe00640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 0x7fce940791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.330+0000 7fceafe00640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 0x7fce940791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000f60 tx=0x7fcea00079b0 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:07.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.333+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcea8062560 con 0x7fceb80b3db0 2024-07-17T08:42:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.533+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fceb813e020 con 0x7fceb80b3db0 2024-07-17T08:42:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.536+0000 7fceb5600640 1 -- 172.21.15.44:0/1841904646 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v320) ==== 74+0+17819 (secure 0 0 0) 0x7fcea8061e60 con 0x7fceb80b3db0 2024-07-17T08:42:07.538 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:07.538 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":320,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:06.120517+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":43,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:04.096300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":44,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:05.114416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.538+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 msgr2=0x7fce940791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.538+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 0x7fce940791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000f60 tx=0x7fcea00079b0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 msgr2=0x7fceb806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fcea800ebc0 tx=0x7fcea800c6a0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 shutdown_connections 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fce94076d00 0x7fce940791c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fceb80cf920 0x7fceb806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fceb80cf540 0x7fceb809c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 --2- 172.21.15.44:0/1841904646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fceb80b3db0 0x7fceb806d420 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 >> 172.21.15.44:0/1841904646 conn(0x7fceb80b0e80 msgr2=0x7fceb81590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 shutdown_connections 2024-07-17T08:42:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.539+0000 7fcebe600640 1 -- 172.21.15.44:0/1841904646 wait complete. 2024-07-17T08:42:07.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:42:07.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.771+0000 7fe0f6800640 1 -- 172.21.15.44:0/4224689670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0d4003a00 msgr2=0x7fe0d4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.771+0000 7fe0f6800640 1 --2- 172.21.15.44:0/4224689670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0d4003a00 0x7fe0d4003e20 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c2c0 tx=0x7fe0d4005830 comp rx=0 tx=0).stop 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.771+0000 7fe0efe00640 1 -- 172.21.15.44:0/4224689670 reap_dead start 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.771+0000 7fe0f6800640 1 -- 172.21.15.44:0/4224689670 shutdown_connections 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.771+0000 7fe0f6800640 1 -- 172.21.15.44:0/4224689670 >> 172.21.15.44:0/4224689670 conn(0x7fe0f00b0eb0 msgr2=0x7fe0f00b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.772+0000 7fe0f6800640 1 -- 172.21.15.44:0/4224689670 shutdown_connections 2024-07-17T08:42:07.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.772+0000 7fe0f6800640 1 -- 172.21.15.44:0/4224689670 wait complete. 2024-07-17T08:42:07.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.773+0000 7fe0f6800640 1 Processor -- start 2024-07-17T08:42:07.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.774+0000 7fe0f6800640 1 -- start start 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.774+0000 7fe0f6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d4003a00 0x7fe0f0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0f6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f01563d0 0x7fe0f00cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0f6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0f6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006f850 con 0x7fe0f00ce380 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0f6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006f6d0 con 0x7fe0d4003a00 2024-07-17T08:42:07.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0f6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f00ac7f0 con 0x7fe0f01563d0 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45986/0 (socket says 172.21.15.44:45986) 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 -- 172.21.15.44:0/2745686467 learned_addr learned my addr 172.21.15.44:0/2745686467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0eea00640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f01563d0 0x7fe0f00cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0ef400640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d4003a00 0x7fe0f0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 -- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f01563d0 msgr2=0x7fe0f00cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f01563d0 0x7fe0f00cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 -- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d4003a00 msgr2=0x7fe0f0155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d4003a00 0x7fe0f0155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.775+0000 7fe0efe00640 1 -- 172.21.15.44:0/2745686467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01550d0 con 0x7fe0f00ce380 2024-07-17T08:42:07.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0efe00640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009460 tx=0x7fe0e800ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:07.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8005220 con 0x7fe0f00ce380 2024-07-17T08:42:07.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0e80054b0 con 0x7fe0f00ce380 2024-07-17T08:42:07.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004a00 con 0x7fe0f00ce380 2024-07-17T08:42:07.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01553b0 con 0x7fe0f00ce380 2024-07-17T08:42:07.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.776+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f0155970 con 0x7fe0f00ce380 2024-07-17T08:42:07.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.777+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f00b3280 con 0x7fe0f00ce380 2024-07-17T08:42:07.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.779+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fe0e8004ba0 con 0x7fe0f00ce380 2024-07-17T08:42:07.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.780+0000 7fe0e7400640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 0x7fe0d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:07.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.780+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fe0e8098b90 con 0x7fe0f00ce380 2024-07-17T08:42:07.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.780+0000 7fe0ef400640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 0x7fe0d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:07.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.781+0000 7fe0ef400640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 0x7fe0d0079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0033000 tx=0x7fe0e002d000 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:07.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:07.784+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0e8061a40 con 0x7fe0f00ce380 2024-07-17T08:42:08.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.044+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe0f0153d00 con 0x7fe0f00ce380 2024-07-17T08:42:08.626 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-07-17T08:42:08.625+0000 7f055ac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:42:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.701+0000 7fe0e7400640 1 -- 172.21.15.44:0/2745686467 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v180) ==== 118+0+0 (secure 0 0 0) 0x7fe0e8061340 con 0x7fe0f00ce380 2024-07-17T08:42:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 msgr2=0x7fe0d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 0x7fe0d0079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0033000 tx=0x7fe0e002d000 comp rx=0 tx=0).stop 2024-07-17T08:42:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 msgr2=0x7fe0f00c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009460 tx=0x7fe0e800ea80 comp rx=0 tx=0).stop 2024-07-17T08:42:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 shutdown_connections 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fe0d0076cd0 0x7fe0d0079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0f00ce380 0x7fe0f00c97c0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0f01563d0 0x7fe0f00cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 --2- 172.21.15.44:0/2745686467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d4003a00 0x7fe0f0155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.705+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 >> 172.21.15.44:0/2745686467 conn(0x7fe0f00b0eb0 msgr2=0x7fe0f015be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.706+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 shutdown_connections 2024-07-17T08:42:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.706+0000 7fe0f6800640 1 -- 172.21.15.44:0/2745686467 wait complete. 2024-07-17T08:42:08.730 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:42:08.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 -- 172.21.15.44:0/1024295041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56406f4d0 msgr2=0x7fd56415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 --2- 172.21.15.44:0/1024295041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56406f4d0 0x7fd56415a450 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fd554005fd0 tx=0x7fd55402b860 comp rx=0 tx=0).stop 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 -- 172.21.15.44:0/1024295041 shutdown_connections 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 --2- 172.21.15.44:0/1024295041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5641503c0 0x7fd5640ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 --2- 172.21.15.44:0/1024295041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd56406f4d0 0x7fd56415a450 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 -- 172.21.15.44:0/1024295041 >> 172.21.15.44:0/1024295041 conn(0x7fd5640cf0d0 msgr2=0x7fd5640cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 -- 172.21.15.44:0/1024295041 shutdown_connections 2024-07-17T08:42:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.948+0000 7fd569c00640 1 -- 172.21.15.44:0/1024295041 wait complete. 2024-07-17T08:42:08.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.950+0000 7fd569c00640 1 Processor -- start 2024-07-17T08:42:08.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.950+0000 7fd569c00640 1 -- start start 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd56406d420 0x7fd56409cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd56406f4d0 0x7fd5641f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd56406da30 con 0x7fd5641503c0 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd56406d8b0 con 0x7fd56406f4d0 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd56406dbb0 con 0x7fd56406d420 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58252/0 (socket says 172.21.15.44:58252) 2024-07-17T08:42:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 -- 172.21.15.44:0/2503796776 learned_addr learned my addr 172.21.15.44:0/2503796776 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:08.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd563e00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd56406d420 0x7fd56409cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd563400640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd56406f4d0 0x7fd5641f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 -- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd56406d420 msgr2=0x7fd56409cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd56406d420 0x7fd56409cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 -- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd56406f4d0 msgr2=0x7fd5641f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd56406f4d0 0x7fd5641f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 -- 172.21.15.44:0/2503796776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5641f5de0 con 0x7fd5641503c0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569200640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c7b0 tx=0x7fd55c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c017070 con 0x7fd5641503c0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd55c005070 con 0x7fd5641503c0 2024-07-17T08:42:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00ce60 con 0x7fd5641503c0 2024-07-17T08:42:08.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5641f60c0 con 0x7fd5641503c0 2024-07-17T08:42:08.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.953+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641f2300 con 0x7fd5641503c0 2024-07-17T08:42:08.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.955+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5641f20f0 con 0x7fd5641503c0 2024-07-17T08:42:08.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.956+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fd55c0054c0 con 0x7fd5641503c0 2024-07-17T08:42:08.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.958+0000 7fd55be00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 0x7fd544079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:08.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.958+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fd55c097ef0 con 0x7fd5641503c0 2024-07-17T08:42:08.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.958+0000 7fd563e00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 0x7fd544079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:08.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.959+0000 7fd563e00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 0x7fd544079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd554035000 tx=0x7fd554004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:08.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:08.962+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd55c0057b0 con 0x7fd5641503c0 2024-07-17T08:42:09.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.220+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd56415a450 con 0x7fd5641503c0 2024-07-17T08:42:09.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.706+0000 7fd55be00640 1 -- 172.21.15.44:0/2503796776 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v181) ==== 122+0+0 (secure 0 0 0) 0x7fd55c060eb0 con 0x7fd5641503c0 2024-07-17T08:42:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.709+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 msgr2=0x7fd544079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.709+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 0x7fd544079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd554035000 tx=0x7fd554004730 comp rx=0 tx=0).stop 2024-07-17T08:42:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.709+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 msgr2=0x7fd56414f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.709+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c7b0 tx=0x7fd55c00cc70 comp rx=0 tx=0).stop 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.709+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 shutdown_connections 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd544076cd0 0x7fd544079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5641503c0 0x7fd56414f600 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd56406f4d0 0x7fd5641f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 --2- 172.21.15.44:0/2503796776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd56406d420 0x7fd56409cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 >> 172.21.15.44:0/2503796776 conn(0x7fd5640cf0d0 msgr2=0x7fd5640b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 shutdown_connections 2024-07-17T08:42:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.710+0000 7fd569c00640 1 -- 172.21.15.44:0/2503796776 wait complete. 2024-07-17T08:42:09.732 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:42:09.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.946+0000 7f2cdca00640 1 -- 172.21.15.44:0/3793275682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cbc004d40 msgr2=0x7f2cbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.946+0000 7f2cdca00640 1 --2- 172.21.15.44:0/3793275682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cbc004d40 0x7f2cbc0051c0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cbc005830 comp rx=0 tx=0).stop 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.947+0000 7f2cd7e00640 1 -- 172.21.15.44:0/3793275682 reap_dead start 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.947+0000 7f2cdca00640 1 -- 172.21.15.44:0/3793275682 shutdown_connections 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.947+0000 7f2cdca00640 1 -- 172.21.15.44:0/3793275682 >> 172.21.15.44:0/3793275682 conn(0x7f2cd80b0e80 msgr2=0x7f2cd80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.947+0000 7f2cdca00640 1 -- 172.21.15.44:0/3793275682 shutdown_connections 2024-07-17T08:42:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.947+0000 7f2cdca00640 1 -- 172.21.15.44:0/3793275682 wait complete. 2024-07-17T08:42:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.948+0000 7f2cdca00640 1 Processor -- start 2024-07-17T08:42:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.949+0000 7f2cdca00640 1 -- start start 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 0x7f2cd81f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cd80cd3c0 0x7f2cd80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd809b850 con 0x7f2cd8153e90 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd809b6d0 con 0x7f2cbc004d40 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cdca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd809b9d0 con 0x7f2cd80cd3c0 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58268/0 (socket says 172.21.15.44:58268) 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 -- 172.21.15.44:0/775206721 learned_addr learned my addr 172.21.15.44:0/775206721 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cd7e00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cd80cd3c0 0x7f2cd80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cd7400640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 0x7f2cd81f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 -- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cd80cd3c0 msgr2=0x7f2cd80cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cd80cd3c0 0x7f2cd80cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 -- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 msgr2=0x7f2cd81f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 0x7f2cd81f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2ccea00640 1 -- 172.21.15.44:0/775206721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd80cb600 con 0x7f2cd8153e90 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.950+0000 7f2cd7400640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 0x7f2cd81f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2ccea00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000fc40 tx=0x7f2cc000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0005220 con 0x7f2cd8153e90 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc00053c0 con 0x7f2cd8153e90 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00047a0 con 0x7f2cd8153e90 2024-07-17T08:42:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd80cb8e0 con 0x7f2cd8153e90 2024-07-17T08:42:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.951+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd80cbea0 con 0x7f2cd8153e90 2024-07-17T08:42:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.952+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd80cf380 con 0x7f2cd8153e90 2024-07-17T08:42:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.954+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f2cc0004940 con 0x7f2cd8153e90 2024-07-17T08:42:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.955+0000 7f2ccfe00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 0x7f2cb4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.955+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f2cc00999b0 con 0x7f2cd8153e90 2024-07-17T08:42:09.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.955+0000 7f2cd7400640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 0x7f2cb4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.957+0000 7f2cd7400640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 0x7f2cb4079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80017d0 tx=0x7f2cc8009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:09.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:09.960+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2cc00627d0 con 0x7f2cd8153e90 2024-07-17T08:42:10.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.215+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd81536c0 con 0x7f2cd8153e90 2024-07-17T08:42:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.216+0000 7f2ccfe00640 1 -- 172.21.15.44:0/775206721 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7f2cc00620d0 con 0x7f2cd8153e90 2024-07-17T08:42:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:10.217 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:10.218 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 msgr2=0x7f2cb4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 0x7f2cb4079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80017d0 tx=0x7f2cc8009040 comp rx=0 tx=0).stop 2024-07-17T08:42:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 msgr2=0x7f2cd81542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000fc40 tx=0x7f2cc000d7b0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 shutdown_connections 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f2cb4076cd0 0x7f2cb4079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cd80cd3c0 0x7f2cd80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cd8153e90 0x7f2cd81542f0 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 --2- 172.21.15.44:0/775206721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2cbc004d40 0x7f2cd81f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.219+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 >> 172.21.15.44:0/775206721 conn(0x7f2cd80b0e80 msgr2=0x7f2cd80b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.220+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 shutdown_connections 2024-07-17T08:42:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.220+0000 7f2cdca00640 1 -- 172.21.15.44:0/775206721 wait complete. 2024-07-17T08:42:10.243 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 -- 172.21.15.44:0/2776427253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 msgr2=0x7f3d1c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 --2- 172.21.15.44:0/2776427253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c15d800 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009860 tx=0x7f3d1402a540 comp rx=0 tx=0).stop 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 -- 172.21.15.44:0/2776427253 shutdown_connections 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 --2- 172.21.15.44:0/2776427253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c15d800 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 --2- 172.21.15.44:0/2776427253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c0ce5c0 0x7f3d1c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.453+0000 7f3d21200640 1 -- 172.21.15.44:0/2776427253 >> 172.21.15.44:0/2776427253 conn(0x7f3d1c0cfec0 msgr2=0x7f3d1c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.454+0000 7f3d21200640 1 -- 172.21.15.44:0/2776427253 shutdown_connections 2024-07-17T08:42:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.454+0000 7f3d21200640 1 -- 172.21.15.44:0/2776427253 wait complete. 2024-07-17T08:42:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.455+0000 7f3d21200640 1 Processor -- start 2024-07-17T08:42:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.455+0000 7f3d21200640 1 -- start start 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c0b8c80 0x7f3d1c0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d1c0ce5c0 0x7f3d1c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c15e0d0 con 0x7f3d1c0b9060 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c15df50 con 0x7f3d1c0ce5c0 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.456+0000 7f3d21200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c15e250 con 0x7f3d1c0b8c80 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58284/0 (socket says 172.21.15.44:58284) 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 -- 172.21.15.44:0/1546308921 learned_addr learned my addr 172.21.15.44:0/1546308921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 -- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c0b8c80 msgr2=0x7f3d1c0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c0b8c80 0x7f3d1c0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 -- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d1c0ce5c0 msgr2=0x7f3d1c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d1c0ce5c0 0x7f3d1c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 -- 172.21.15.44:0/1546308921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c0caa50 con 0x7f3d1c0b9060 2024-07-17T08:42:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.457+0000 7f3d1aa00640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400ea20 tx=0x7f3d0400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.458+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400cde0 con 0x7f3d1c0b9060 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.458+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d04011560 con 0x7f3d1c0b9060 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.458+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04011990 con 0x7f3d1c0b9060 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.458+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c0aa0f0 con 0x7f3d1c0b9060 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.458+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c0cc930 con 0x7f3d1c0b9060 2024-07-17T08:42:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.459+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1c15e700 con 0x7f3d1c0b9060 2024-07-17T08:42:10.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.460+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f3d04011b30 con 0x7f3d1c0b9060 2024-07-17T08:42:10.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.461+0000 7f3d13400640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 0x7f3cf80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.461+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f3d040995c0 con 0x7f3d1c0b9060 2024-07-17T08:42:10.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.462+0000 7f3d1b400640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 0x7f3cf80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.463+0000 7f3d1b400640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 0x7f3cf80791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0015a0 tx=0x7f3d0c007a30 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.466+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3d04016030 con 0x7f3d1c0b9060 2024-07-17T08:42:10.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.679+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d1c0b5d70 con 0x7f3d1c0b9060 2024-07-17T08:42:10.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.682+0000 7f3d13400640 1 -- 172.21.15.44:0/1546308921 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v321) ==== 74+0+17806 (secure 0 0 0) 0x7f3d04011e20 con 0x7f3d1c0b9060 2024-07-17T08:42:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:10.683 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":321,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:07.650234+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":43,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:04.096300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":44,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:05.114416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"321","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 msgr2=0x7f3cf80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 0x7f3cf80791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0015a0 tx=0x7f3d0c007a30 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 msgr2=0x7f3d1c1f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400ea20 tx=0x7f3d0400eee0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 shutdown_connections 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3cf8076d00 0x7f3cf80791c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d1c0ce5c0 0x7f3d1c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d1c0b9060 0x7f3d1c1f4620 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 --2- 172.21.15.44:0/1546308921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d1c0b8c80 0x7f3d1c0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 >> 172.21.15.44:0/1546308921 conn(0x7f3d1c0cfec0 msgr2=0x7f3d1c0b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:10.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 shutdown_connections 2024-07-17T08:42:10.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.685+0000 7f3d21200640 1 -- 172.21.15.44:0/1546308921 wait complete. 2024-07-17T08:42:10.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:42:10.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 -- 172.21.15.44:0/2186958882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40ac930 msgr2=0x7f33f40cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2186958882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40ac930 0x7f33f40cd690 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009ca0 tx=0x7f33ec02aac0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 -- 172.21.15.44:0/2186958882 shutdown_connections 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2186958882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40ac930 0x7f33f40cd690 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 --2- 172.21.15.44:0/2186958882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 0x7f33f40ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.907+0000 7f33fbe00640 1 -- 172.21.15.44:0/2186958882 >> 172.21.15.44:0/2186958882 conn(0x7f33f40b0fb0 msgr2=0x7f33f40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.908+0000 7f33fbe00640 1 -- 172.21.15.44:0/2186958882 shutdown_connections 2024-07-17T08:42:10.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.908+0000 7f33fbe00640 1 -- 172.21.15.44:0/2186958882 wait complete. 2024-07-17T08:42:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.910+0000 7f33fbe00640 1 Processor -- start 2024-07-17T08:42:10.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.910+0000 7f33fbe00640 1 -- start start 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40ac930 0x7f33f40740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 0x7f33f4072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33f40ce010 con 0x7f33f40adcd0 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33f40cde90 con 0x7f33f40ac930 2024-07-17T08:42:10.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33fbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33f40ce190 con 0x7f33f41522c0 2024-07-17T08:42:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.911+0000 7f33f8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58306/0 (socket says 172.21.15.44:58306) 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 -- 172.21.15.44:0/225156000 learned_addr learned my addr 172.21.15.44:0/225156000 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f9e00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 0x7f33f4072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f9400640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40ac930 0x7f33f40740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 -- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 msgr2=0x7f33f4072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 0x7f33f4072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 -- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40ac930 msgr2=0x7f33f40740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40ac930 0x7f33f40740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 -- 172.21.15.44:0/225156000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f40b2450 con 0x7f33f40adcd0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.912+0000 7f33f8a00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c420 tx=0x7f33e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.913+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0017070 con 0x7f33f40adcd0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.913+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33e00053c0 con 0x7f33f40adcd0 2024-07-17T08:42:10.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.913+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f414ec00 con 0x7f33f40adcd0 2024-07-17T08:42:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.913+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0003850 con 0x7f33f40adcd0 2024-07-17T08:42:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.913+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f414eee0 con 0x7f33f40adcd0 2024-07-17T08:42:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.914+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f40ce6b0 con 0x7f33f40adcd0 2024-07-17T08:42:10.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.915+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f33e001a070 con 0x7f33f40adcd0 2024-07-17T08:42:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.916+0000 7f33f1600640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 0x7f33d40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.916+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f33e0097cc0 con 0x7f33f40adcd0 2024-07-17T08:42:10.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.917+0000 7f33f9400640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 0x7f33d40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:10.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.918+0000 7f33f9400640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 0x7f33d40791c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33e80058c0 tx=0x7f33e800c440 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:10.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:10.922+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f33e009e080 con 0x7f33f40adcd0 2024-07-17T08:42:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.121+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 --> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33f415a810 con 0x7f33d4076d00 2024-07-17T08:42:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.137+0000 7f33f1600640 1 -- 172.21.15.44:0/225156000 <== mgr.10276 v2:172.21.15.175:6834/3850741755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7f33f415a810 con 0x7f33d4076d00 2024-07-17T08:42:11.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:11.143 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 msgr2=0x7f33d40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 0x7f33d40791c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33e80058c0 tx=0x7f33e800c440 comp rx=0 tx=0).stop 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 msgr2=0x7f33f41f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c420 tx=0x7f33e000c8e0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 shutdown_connections 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f33d4076d00 0x7f33d40791c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33f41522c0 0x7f33f4072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33f40adcd0 0x7f33f41f3b80 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 --2- 172.21.15.44:0/225156000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33f40ac930 0x7f33f40740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.143+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 >> 172.21.15.44:0/225156000 conn(0x7f33f40b0fb0 msgr2=0x7f33f415ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.144+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 shutdown_connections 2024-07-17T08:42:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.144+0000 7f33fbe00640 1 -- 172.21.15.44:0/225156000 wait complete. 2024-07-17T08:42:11.162 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:42:10.381901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":537,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4662,"ondisk_log_size":4662,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479672,"kb_used_data":33072,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754495048,"statfs":{"total":773094113280,"available":772602929152,"internally_reserved":0,"allocated":33865728,"data_stored":19148852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756082"},"pg_stats":[{"pgid":"44.3f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661472+0000","last_change":"2024-07-17T08:42:06.685173+0000","last_active":"2024-07-17T08:42:07.661472+0000","last_peered":"2024-07-17T08:42:07.661472+0000","last_clean":"2024-07-17T08:42:07.661472+0000","last_became_active":"2024-07-17T08:42:06.684915+0000","last_became_peered":"2024-07-17T08:42:06.684915+0000","last_unstale":"2024-07-17T08:42:07.661472+0000","last_undegraded":"2024-07-17T08:42:07.661472+0000","last_fullsized":"2024-07-17T08:42:07.661472+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:51.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.3e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985408+0000","last_change":"2024-07-17T08:42:06.682009+0000","last_active":"2024-07-17T08:42:07.985408+0000","last_peered":"2024-07-17T08:42:07.985408+0000","last_clean":"2024-07-17T08:42:07.985408+0000","last_became_active":"2024-07-17T08:42:06.681739+0000","last_became_peered":"2024-07-17T08:42:06.681739+0000","last_unstale":"2024-07-17T08:42:07.985408+0000","last_undegraded":"2024-07-17T08:42:07.985408+0000","last_fullsized":"2024-07-17T08:42:07.985408+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:14:43.533749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.3d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982803+0000","last_change":"2024-07-17T08:42:06.671934+0000","last_active":"2024-07-17T08:42:07.982803+0000","last_peered":"2024-07-17T08:42:07.982803+0000","last_clean":"2024-07-17T08:42:07.982803+0000","last_became_active":"2024-07-17T08:42:06.671670+0000","last_became_peered":"2024-07-17T08:42:06.671670+0000","last_unstale":"2024-07-17T08:42:07.982803+0000","last_undegraded":"2024-07-17T08:42:07.982803+0000","last_fullsized":"2024-07-17T08:42:07.982803+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:33.549416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.3c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985067+0000","last_change":"2024-07-17T08:42:06.684073+0000","last_active":"2024-07-17T08:42:07.985067+0000","last_peered":"2024-07-17T08:42:07.985067+0000","last_clean":"2024-07-17T08:42:07.985067+0000","last_became_active":"2024-07-17T08:42:06.683768+0000","last_became_peered":"2024-07-17T08:42:06.683768+0000","last_unstale":"2024-07-17T08:42:07.985067+0000","last_undegraded":"2024-07-17T08:42:07.985067+0000","last_fullsized":"2024-07-17T08:42:07.985067+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:01.851954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.3b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982149+0000","last_change":"2024-07-17T08:42:06.683682+0000","last_active":"2024-07-17T08:42:07.982149+0000","last_peered":"2024-07-17T08:42:07.982149+0000","last_clean":"2024-07-17T08:42:07.982149+0000","last_became_active":"2024-07-17T08:42:06.683378+0000","last_became_peered":"2024-07-17T08:42:06.683378+0000","last_unstale":"2024-07-17T08:42:07.982149+0000","last_undegraded":"2024-07-17T08:42:07.982149+0000","last_fullsized":"2024-07-17T08:42:07.982149+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:50.840183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.3a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982275+0000","last_change":"2024-07-17T08:42:06.696413+0000","last_active":"2024-07-17T08:42:07.982275+0000","last_peered":"2024-07-17T08:42:07.982275+0000","last_clean":"2024-07-17T08:42:07.982275+0000","last_became_active":"2024-07-17T08:42:06.696128+0000","last_became_peered":"2024-07-17T08:42:06.696128+0000","last_unstale":"2024-07-17T08:42:07.982275+0000","last_undegraded":"2024-07-17T08:42:07.982275+0000","last_fullsized":"2024-07-17T08:42:07.982275+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:51.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.39","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657883+0000","last_change":"2024-07-17T08:42:06.692670+0000","last_active":"2024-07-17T08:42:07.657883+0000","last_peered":"2024-07-17T08:42:07.657883+0000","last_clean":"2024-07-17T08:42:07.657883+0000","last_became_active":"2024-07-17T08:42:06.692082+0000","last_became_peered":"2024-07-17T08:42:06.692082+0000","last_unstale":"2024-07-17T08:42:07.657883+0000","last_undegraded":"2024-07-17T08:42:07.657883+0000","last_fullsized":"2024-07-17T08:42:07.657883+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:33.257603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.38","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660178+0000","last_change":"2024-07-17T08:42:06.688736+0000","last_active":"2024-07-17T08:42:07.660178+0000","last_peered":"2024-07-17T08:42:07.660178+0000","last_clean":"2024-07-17T08:42:07.660178+0000","last_became_active":"2024-07-17T08:42:06.688425+0000","last_became_peered":"2024-07-17T08:42:06.688425+0000","last_unstale":"2024-07-17T08:42:07.660178+0000","last_undegraded":"2024-07-17T08:42:07.660178+0000","last_fullsized":"2024-07-17T08:42:07.660178+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:54.620438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.37","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661323+0000","last_change":"2024-07-17T08:42:06.689965+0000","last_active":"2024-07-17T08:42:07.661323+0000","last_peered":"2024-07-17T08:42:07.661323+0000","last_clean":"2024-07-17T08:42:07.661323+0000","last_became_active":"2024-07-17T08:42:06.689628+0000","last_became_peered":"2024-07-17T08:42:06.689628+0000","last_unstale":"2024-07-17T08:42:07.661323+0000","last_undegraded":"2024-07-17T08:42:07.661323+0000","last_fullsized":"2024-07-17T08:42:07.661323+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:07:30.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.36","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661740+0000","last_change":"2024-07-17T08:42:06.689771+0000","last_active":"2024-07-17T08:42:07.661740+0000","last_peered":"2024-07-17T08:42:07.661740+0000","last_clean":"2024-07-17T08:42:07.661740+0000","last_became_active":"2024-07-17T08:42:06.688293+0000","last_became_peered":"2024-07-17T08:42:06.688293+0000","last_unstale":"2024-07-17T08:42:07.661740+0000","last_undegraded":"2024-07-17T08:42:07.661740+0000","last_fullsized":"2024-07-17T08:42:07.661740+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:47.790788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.35","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661073+0000","last_change":"2024-07-17T08:42:06.688653+0000","last_active":"2024-07-17T08:42:07.661073+0000","last_peered":"2024-07-17T08:42:07.661073+0000","last_clean":"2024-07-17T08:42:07.661073+0000","last_became_active":"2024-07-17T08:42:06.688351+0000","last_became_peered":"2024-07-17T08:42:06.688351+0000","last_unstale":"2024-07-17T08:42:07.661073+0000","last_undegraded":"2024-07-17T08:42:07.661073+0000","last_fullsized":"2024-07-17T08:42:07.661073+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:45.546848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.34","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658826+0000","last_change":"2024-07-17T08:42:06.691748+0000","last_active":"2024-07-17T08:42:07.658826+0000","last_peered":"2024-07-17T08:42:07.658826+0000","last_clean":"2024-07-17T08:42:07.658826+0000","last_became_active":"2024-07-17T08:42:06.687996+0000","last_became_peered":"2024-07-17T08:42:06.687996+0000","last_unstale":"2024-07-17T08:42:07.658826+0000","last_undegraded":"2024-07-17T08:42:07.658826+0000","last_fullsized":"2024-07-17T08:42:07.658826+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:01.082911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.33","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984798+0000","last_change":"2024-07-17T08:42:06.694341+0000","last_active":"2024-07-17T08:42:07.984798+0000","last_peered":"2024-07-17T08:42:07.984798+0000","last_clean":"2024-07-17T08:42:07.984798+0000","last_became_active":"2024-07-17T08:42:06.694005+0000","last_became_peered":"2024-07-17T08:42:06.694005+0000","last_unstale":"2024-07-17T08:42:07.984798+0000","last_undegraded":"2024-07-17T08:42:07.984798+0000","last_fullsized":"2024-07-17T08:42:07.984798+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:19:58.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.32","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982720+0000","last_change":"2024-07-17T08:42:06.678319+0000","last_active":"2024-07-17T08:42:07.982720+0000","last_peered":"2024-07-17T08:42:07.982720+0000","last_clean":"2024-07-17T08:42:07.982720+0000","last_became_active":"2024-07-17T08:42:06.678001+0000","last_became_peered":"2024-07-17T08:42:06.678001+0000","last_unstale":"2024-07-17T08:42:07.982720+0000","last_undegraded":"2024-07-17T08:42:07.982720+0000","last_fullsized":"2024-07-17T08:42:07.982720+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:04.540386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.31","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660209+0000","last_change":"2024-07-17T08:42:06.683786+0000","last_active":"2024-07-17T08:42:07.660209+0000","last_peered":"2024-07-17T08:42:07.660209+0000","last_clean":"2024-07-17T08:42:07.660209+0000","last_became_active":"2024-07-17T08:42:06.683511+0000","last_became_peered":"2024-07-17T08:42:06.683511+0000","last_unstale":"2024-07-17T08:42:07.660209+0000","last_undegraded":"2024-07-17T08:42:07.660209+0000","last_fullsized":"2024-07-17T08:42:07.660209+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:45.932665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.30","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659816+0000","last_change":"2024-07-17T08:42:06.684355+0000","last_active":"2024-07-17T08:42:07.659816+0000","last_peered":"2024-07-17T08:42:07.659816+0000","last_clean":"2024-07-17T08:42:07.659816+0000","last_became_active":"2024-07-17T08:42:06.684061+0000","last_became_peered":"2024-07-17T08:42:06.684061+0000","last_unstale":"2024-07-17T08:42:07.659816+0000","last_undegraded":"2024-07-17T08:42:07.659816+0000","last_fullsized":"2024-07-17T08:42:07.659816+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:30:07.526346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.2f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658418+0000","last_change":"2024-07-17T08:42:06.692345+0000","last_active":"2024-07-17T08:42:07.658418+0000","last_peered":"2024-07-17T08:42:07.658418+0000","last_clean":"2024-07-17T08:42:07.658418+0000","last_became_active":"2024-07-17T08:42:06.691974+0000","last_became_peered":"2024-07-17T08:42:06.691974+0000","last_unstale":"2024-07-17T08:42:07.658418+0000","last_undegraded":"2024-07-17T08:42:07.658418+0000","last_fullsized":"2024-07-17T08:42:07.658418+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:10.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.2e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982820+0000","last_change":"2024-07-17T08:42:06.694280+0000","last_active":"2024-07-17T08:42:07.982820+0000","last_peered":"2024-07-17T08:42:07.982820+0000","last_clean":"2024-07-17T08:42:07.982820+0000","last_became_active":"2024-07-17T08:42:06.693680+0000","last_became_peered":"2024-07-17T08:42:06.693680+0000","last_unstale":"2024-07-17T08:42:07.982820+0000","last_undegraded":"2024-07-17T08:42:07.982820+0000","last_fullsized":"2024-07-17T08:42:07.982820+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:14.156933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.2c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985308+0000","last_change":"2024-07-17T08:42:06.690221+0000","last_active":"2024-07-17T08:42:07.985308+0000","last_peered":"2024-07-17T08:42:07.985308+0000","last_clean":"2024-07-17T08:42:07.985308+0000","last_became_active":"2024-07-17T08:42:06.689914+0000","last_became_peered":"2024-07-17T08:42:06.689914+0000","last_unstale":"2024-07-17T08:42:07.985308+0000","last_undegraded":"2024-07-17T08:42:07.985308+0000","last_fullsized":"2024-07-17T08:42:07.985308+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:03.691858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.2b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983322+0000","last_change":"2024-07-17T08:42:06.682270+0000","last_active":"2024-07-17T08:42:07.983322+0000","last_peered":"2024-07-17T08:42:07.983322+0000","last_clean":"2024-07-17T08:42:07.983322+0000","last_became_active":"2024-07-17T08:42:06.681027+0000","last_became_peered":"2024-07-17T08:42:06.681027+0000","last_unstale":"2024-07-17T08:42:07.983322+0000","last_undegraded":"2024-07-17T08:42:07.983322+0000","last_fullsized":"2024-07-17T08:42:07.983322+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:27.221763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.2a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982152+0000","last_change":"2024-07-17T08:42:06.667352+0000","last_active":"2024-07-17T08:42:07.982152+0000","last_peered":"2024-07-17T08:42:07.982152+0000","last_clean":"2024-07-17T08:42:07.982152+0000","last_became_active":"2024-07-17T08:42:06.666995+0000","last_became_peered":"2024-07-17T08:42:06.666995+0000","last_unstale":"2024-07-17T08:42:07.982152+0000","last_undegraded":"2024-07-17T08:42:07.982152+0000","last_fullsized":"2024-07-17T08:42:07.982152+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:45.237328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.29","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981533+0000","last_change":"2024-07-17T08:42:06.682171+0000","last_active":"2024-07-17T08:42:07.981533+0000","last_peered":"2024-07-17T08:42:07.981533+0000","last_clean":"2024-07-17T08:42:07.981533+0000","last_became_active":"2024-07-17T08:42:06.681632+0000","last_became_peered":"2024-07-17T08:42:06.681632+0000","last_unstale":"2024-07-17T08:42:07.981533+0000","last_undegraded":"2024-07-17T08:42:07.981533+0000","last_fullsized":"2024-07-17T08:42:07.981533+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:58.488319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.28","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983027+0000","last_change":"2024-07-17T08:42:06.697767+0000","last_active":"2024-07-17T08:42:07.983027+0000","last_peered":"2024-07-17T08:42:07.983027+0000","last_clean":"2024-07-17T08:42:07.983027+0000","last_became_active":"2024-07-17T08:42:06.697309+0000","last_became_peered":"2024-07-17T08:42:06.697309+0000","last_unstale":"2024-07-17T08:42:07.983027+0000","last_undegraded":"2024-07-17T08:42:07.983027+0000","last_fullsized":"2024-07-17T08:42:07.983027+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:47.790788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.27","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982386+0000","last_change":"2024-07-17T08:42:06.686378+0000","last_active":"2024-07-17T08:42:07.982386+0000","last_peered":"2024-07-17T08:42:07.982386+0000","last_clean":"2024-07-17T08:42:07.982386+0000","last_became_active":"2024-07-17T08:42:06.686071+0000","last_became_peered":"2024-07-17T08:42:06.686071+0000","last_unstale":"2024-07-17T08:42:07.982386+0000","last_undegraded":"2024-07-17T08:42:07.982386+0000","last_fullsized":"2024-07-17T08:42:07.982386+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:06.210985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.26","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983054+0000","last_change":"2024-07-17T08:42:06.670944+0000","last_active":"2024-07-17T08:42:07.983054+0000","last_peered":"2024-07-17T08:42:07.983054+0000","last_clean":"2024-07-17T08:42:07.983054+0000","last_became_active":"2024-07-17T08:42:06.667772+0000","last_became_peered":"2024-07-17T08:42:06.667772+0000","last_unstale":"2024-07-17T08:42:07.983054+0000","last_undegraded":"2024-07-17T08:42:07.983054+0000","last_fullsized":"2024-07-17T08:42:07.983054+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:59:19.421877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.25","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661467+0000","last_change":"2024-07-17T08:42:06.686850+0000","last_active":"2024-07-17T08:42:07.661467+0000","last_peered":"2024-07-17T08:42:07.661467+0000","last_clean":"2024-07-17T08:42:07.661467+0000","last_became_active":"2024-07-17T08:42:06.686513+0000","last_became_peered":"2024-07-17T08:42:06.686513+0000","last_unstale":"2024-07-17T08:42:07.661467+0000","last_undegraded":"2024-07-17T08:42:07.661467+0000","last_fullsized":"2024-07-17T08:42:07.661467+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:46.284529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.24","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658600+0000","last_change":"2024-07-17T08:42:06.683668+0000","last_active":"2024-07-17T08:42:07.658600+0000","last_peered":"2024-07-17T08:42:07.658600+0000","last_clean":"2024-07-17T08:42:07.658600+0000","last_became_active":"2024-07-17T08:42:06.682943+0000","last_became_peered":"2024-07-17T08:42:06.682943+0000","last_unstale":"2024-07-17T08:42:07.658600+0000","last_undegraded":"2024-07-17T08:42:07.658600+0000","last_fullsized":"2024-07-17T08:42:07.658600+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:42:37.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.23","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981988+0000","last_change":"2024-07-17T08:42:06.675986+0000","last_active":"2024-07-17T08:42:07.981988+0000","last_peered":"2024-07-17T08:42:07.981988+0000","last_clean":"2024-07-17T08:42:07.981988+0000","last_became_active":"2024-07-17T08:42:06.667919+0000","last_became_peered":"2024-07-17T08:42:06.667919+0000","last_unstale":"2024-07-17T08:42:07.981988+0000","last_undegraded":"2024-07-17T08:42:07.981988+0000","last_fullsized":"2024-07-17T08:42:07.981988+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:10:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.22","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981614+0000","last_change":"2024-07-17T08:42:06.682603+0000","last_active":"2024-07-17T08:42:07.981614+0000","last_peered":"2024-07-17T08:42:07.981614+0000","last_clean":"2024-07-17T08:42:07.981614+0000","last_became_active":"2024-07-17T08:42:06.682307+0000","last_became_peered":"2024-07-17T08:42:06.682307+0000","last_unstale":"2024-07-17T08:42:07.981614+0000","last_undegraded":"2024-07-17T08:42:07.981614+0000","last_fullsized":"2024-07-17T08:42:07.981614+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:24.179304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.21","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984252+0000","last_change":"2024-07-17T08:42:06.691890+0000","last_active":"2024-07-17T08:42:07.984252+0000","last_peered":"2024-07-17T08:42:07.984252+0000","last_clean":"2024-07-17T08:42:07.984252+0000","last_became_active":"2024-07-17T08:42:06.687855+0000","last_became_peered":"2024-07-17T08:42:06.687855+0000","last_unstale":"2024-07-17T08:42:07.984252+0000","last_undegraded":"2024-07-17T08:42:07.984252+0000","last_fullsized":"2024-07-17T08:42:07.984252+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:19.090582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.20","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660019+0000","last_change":"2024-07-17T08:42:06.688833+0000","last_active":"2024-07-17T08:42:07.660019+0000","last_peered":"2024-07-17T08:42:07.660019+0000","last_clean":"2024-07-17T08:42:07.660019+0000","last_became_active":"2024-07-17T08:42:06.688476+0000","last_became_peered":"2024-07-17T08:42:06.688476+0000","last_unstale":"2024-07-17T08:42:07.660019+0000","last_undegraded":"2024-07-17T08:42:07.660019+0000","last_fullsized":"2024-07-17T08:42:07.660019+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:30.851281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.18","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659788+0000","last_change":"2024-07-17T08:42:06.662588+0000","last_active":"2024-07-17T08:42:07.659788+0000","last_peered":"2024-07-17T08:42:07.659788+0000","last_clean":"2024-07-17T08:42:07.659788+0000","last_became_active":"2024-07-17T08:42:06.662190+0000","last_became_peered":"2024-07-17T08:42:06.662190+0000","last_unstale":"2024-07-17T08:42:07.659788+0000","last_undegraded":"2024-07-17T08:42:07.659788+0000","last_fullsized":"2024-07-17T08:42:07.659788+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:51.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.1f","version":"320'4","reported_seq":18,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985711+0000","last_change":"2024-07-17T08:42:05.668629+0000","last_active":"2024-07-17T08:42:07.985711+0000","last_peered":"2024-07-17T08:42:07.985711+0000","last_clean":"2024-07-17T08:42:07.985711+0000","last_became_active":"2024-07-17T08:42:05.668320+0000","last_became_peered":"2024-07-17T08:42:05.668320+0000","last_unstale":"2024-07-17T08:42:07.985711+0000","last_undegraded":"2024-07-17T08:42:07.985711+0000","last_fullsized":"2024-07-17T08:42:07.985711+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:53.013732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.19","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984191+0000","last_change":"2024-07-17T08:42:06.691597+0000","last_active":"2024-07-17T08:42:07.984191+0000","last_peered":"2024-07-17T08:42:07.984191+0000","last_clean":"2024-07-17T08:42:07.984191+0000","last_became_active":"2024-07-17T08:42:06.685587+0000","last_became_peered":"2024-07-17T08:42:06.685587+0000","last_unstale":"2024-07-17T08:42:07.984191+0000","last_undegraded":"2024-07-17T08:42:07.984191+0000","last_fullsized":"2024-07-17T08:42:07.984191+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:30.281884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.1e","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983132+0000","last_change":"2024-07-17T08:42:05.665763+0000","last_active":"2024-07-17T08:42:07.983132+0000","last_peered":"2024-07-17T08:42:07.983132+0000","last_clean":"2024-07-17T08:42:07.983132+0000","last_became_active":"2024-07-17T08:42:05.665453+0000","last_became_peered":"2024-07-17T08:42:05.665453+0000","last_unstale":"2024-07-17T08:42:07.983132+0000","last_undegraded":"2024-07-17T08:42:07.983132+0000","last_fullsized":"2024-07-17T08:42:07.983132+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:36.270152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982084+0000","last_change":"2024-07-17T08:42:06.693507+0000","last_active":"2024-07-17T08:42:07.982084+0000","last_peered":"2024-07-17T08:42:07.982084+0000","last_clean":"2024-07-17T08:42:07.982084+0000","last_became_active":"2024-07-17T08:42:06.693203+0000","last_became_peered":"2024-07-17T08:42:06.693203+0000","last_unstale":"2024-07-17T08:42:07.982084+0000","last_undegraded":"2024-07-17T08:42:07.982084+0000","last_fullsized":"2024-07-17T08:42:07.982084+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:46.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.1d","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981782+0000","last_change":"2024-07-17T08:42:05.666848+0000","last_active":"2024-07-17T08:42:07.981782+0000","last_peered":"2024-07-17T08:42:07.981782+0000","last_clean":"2024-07-17T08:42:07.981782+0000","last_became_active":"2024-07-17T08:42:05.666498+0000","last_became_peered":"2024-07-17T08:42:05.666498+0000","last_unstale":"2024-07-17T08:42:07.981782+0000","last_undegraded":"2024-07-17T08:42:07.981782+0000","last_fullsized":"2024-07-17T08:42:07.981782+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:45:10.283939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658412+0000","last_change":"2024-07-17T08:42:06.689322+0000","last_active":"2024-07-17T08:42:07.658412+0000","last_peered":"2024-07-17T08:42:07.658412+0000","last_clean":"2024-07-17T08:42:07.658412+0000","last_became_active":"2024-07-17T08:42:06.687983+0000","last_became_peered":"2024-07-17T08:42:06.687983+0000","last_unstale":"2024-07-17T08:42:07.658412+0000","last_undegraded":"2024-07-17T08:42:07.658412+0000","last_fullsized":"2024-07-17T08:42:07.658412+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.561117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.1c","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660708+0000","last_change":"2024-07-17T08:42:05.655590+0000","last_active":"2024-07-17T08:42:07.660708+0000","last_peered":"2024-07-17T08:42:07.660708+0000","last_clean":"2024-07-17T08:42:07.660708+0000","last_became_active":"2024-07-17T08:42:05.655247+0000","last_became_peered":"2024-07-17T08:42:05.655247+0000","last_unstale":"2024-07-17T08:42:07.660708+0000","last_undegraded":"2024-07-17T08:42:07.660708+0000","last_fullsized":"2024-07-17T08:42:07.660708+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:25.185151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.1c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657870+0000","last_change":"2024-07-17T08:42:06.693864+0000","last_active":"2024-07-17T08:42:07.657870+0000","last_peered":"2024-07-17T08:42:07.657870+0000","last_clean":"2024-07-17T08:42:07.657870+0000","last_became_active":"2024-07-17T08:42:06.693571+0000","last_became_peered":"2024-07-17T08:42:06.693571+0000","last_unstale":"2024-07-17T08:42:07.657870+0000","last_undegraded":"2024-07-17T08:42:07.657870+0000","last_fullsized":"2024-07-17T08:42:07.657870+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:55.220579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.1b","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658920+0000","last_change":"2024-07-17T08:42:05.659152+0000","last_active":"2024-07-17T08:42:07.658920+0000","last_peered":"2024-07-17T08:42:07.658920+0000","last_clean":"2024-07-17T08:42:07.658920+0000","last_became_active":"2024-07-17T08:42:05.658842+0000","last_became_peered":"2024-07-17T08:42:05.658842+0000","last_unstale":"2024-07-17T08:42:07.658920+0000","last_undegraded":"2024-07-17T08:42:07.658920+0000","last_fullsized":"2024-07-17T08:42:07.658920+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:08.477038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985220+0000","last_change":"2024-07-17T08:42:06.691295+0000","last_active":"2024-07-17T08:42:07.985220+0000","last_peered":"2024-07-17T08:42:07.985220+0000","last_clean":"2024-07-17T08:42:07.985220+0000","last_became_active":"2024-07-17T08:42:06.689645+0000","last_became_peered":"2024-07-17T08:42:06.689645+0000","last_unstale":"2024-07-17T08:42:07.985220+0000","last_undegraded":"2024-07-17T08:42:07.985220+0000","last_fullsized":"2024-07-17T08:42:07.985220+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:37.304601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.1a","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982364+0000","last_change":"2024-07-17T08:42:05.663162+0000","last_active":"2024-07-17T08:42:07.982364+0000","last_peered":"2024-07-17T08:42:07.982364+0000","last_clean":"2024-07-17T08:42:07.982364+0000","last_became_active":"2024-07-17T08:42:05.662384+0000","last_became_peered":"2024-07-17T08:42:05.662384+0000","last_unstale":"2024-07-17T08:42:07.982364+0000","last_undegraded":"2024-07-17T08:42:07.982364+0000","last_fullsized":"2024-07-17T08:42:07.982364+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:27:35.320892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983591+0000","last_change":"2024-07-17T08:42:06.688724+0000","last_active":"2024-07-17T08:42:07.983591+0000","last_peered":"2024-07-17T08:42:07.983591+0000","last_clean":"2024-07-17T08:42:07.983591+0000","last_became_active":"2024-07-17T08:42:06.688377+0000","last_became_peered":"2024-07-17T08:42:06.688377+0000","last_unstale":"2024-07-17T08:42:07.983591+0000","last_undegraded":"2024-07-17T08:42:07.983591+0000","last_fullsized":"2024-07-17T08:42:07.983591+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:29:19.269818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.19","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982649+0000","last_change":"2024-07-17T08:42:05.656321+0000","last_active":"2024-07-17T08:42:07.982649+0000","last_peered":"2024-07-17T08:42:07.982649+0000","last_clean":"2024-07-17T08:42:07.982649+0000","last_became_active":"2024-07-17T08:42:05.655307+0000","last_became_peered":"2024-07-17T08:42:05.655307+0000","last_unstale":"2024-07-17T08:42:07.982649+0000","last_undegraded":"2024-07-17T08:42:07.982649+0000","last_fullsized":"2024-07-17T08:42:07.982649+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:09.253237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661524+0000","last_change":"2024-07-17T08:42:06.684595+0000","last_active":"2024-07-17T08:42:07.661524+0000","last_peered":"2024-07-17T08:42:07.661524+0000","last_clean":"2024-07-17T08:42:07.661524+0000","last_became_active":"2024-07-17T08:42:06.684034+0000","last_became_peered":"2024-07-17T08:42:06.684034+0000","last_unstale":"2024-07-17T08:42:07.661524+0000","last_undegraded":"2024-07-17T08:42:07.661524+0000","last_fullsized":"2024-07-17T08:42:07.661524+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:04.540386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.18","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.980960+0000","last_change":"2024-07-17T08:42:05.657449+0000","last_active":"2024-07-17T08:42:07.980960+0000","last_peered":"2024-07-17T08:42:07.980960+0000","last_clean":"2024-07-17T08:42:07.980960+0000","last_became_active":"2024-07-17T08:42:05.657153+0000","last_became_peered":"2024-07-17T08:42:05.657153+0000","last_unstale":"2024-07-17T08:42:07.980960+0000","last_undegraded":"2024-07-17T08:42:07.980960+0000","last_fullsized":"2024-07-17T08:42:07.980960+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:35.223085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.10","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981649+0000","last_change":"2024-07-17T08:42:06.666120+0000","last_active":"2024-07-17T08:42:07.981649+0000","last_peered":"2024-07-17T08:42:07.981649+0000","last_clean":"2024-07-17T08:42:07.981649+0000","last_became_active":"2024-07-17T08:42:06.665720+0000","last_became_peered":"2024-07-17T08:42:06.665720+0000","last_unstale":"2024-07-17T08:42:07.981649+0000","last_undegraded":"2024-07-17T08:42:07.981649+0000","last_fullsized":"2024-07-17T08:42:07.981649+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:18.525206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.17","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985872+0000","last_change":"2024-07-17T08:42:05.669438+0000","last_active":"2024-07-17T08:42:07.985872+0000","last_peered":"2024-07-17T08:42:07.985872+0000","last_clean":"2024-07-17T08:42:07.985872+0000","last_became_active":"2024-07-17T08:42:05.668992+0000","last_became_peered":"2024-07-17T08:42:05.668992+0000","last_unstale":"2024-07-17T08:42:07.985872+0000","last_undegraded":"2024-07-17T08:42:07.985872+0000","last_fullsized":"2024-07-17T08:42:07.985872+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:14.146779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.11","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657513+0000","last_change":"2024-07-17T08:42:06.690575+0000","last_active":"2024-07-17T08:42:07.657513+0000","last_peered":"2024-07-17T08:42:07.657513+0000","last_clean":"2024-07-17T08:42:07.657513+0000","last_became_active":"2024-07-17T08:42:06.690197+0000","last_became_peered":"2024-07-17T08:42:06.690197+0000","last_unstale":"2024-07-17T08:42:07.657513+0000","last_undegraded":"2024-07-17T08:42:07.657513+0000","last_fullsized":"2024-07-17T08:42:07.657513+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:05.331208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.16","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658474+0000","last_change":"2024-07-17T08:42:05.649750+0000","last_active":"2024-07-17T08:42:07.658474+0000","last_peered":"2024-07-17T08:42:07.658474+0000","last_clean":"2024-07-17T08:42:07.658474+0000","last_became_active":"2024-07-17T08:42:05.649334+0000","last_became_peered":"2024-07-17T08:42:05.649334+0000","last_unstale":"2024-07-17T08:42:07.658474+0000","last_undegraded":"2024-07-17T08:42:07.658474+0000","last_fullsized":"2024-07-17T08:42:07.658474+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:24.344179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.12","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981819+0000","last_change":"2024-07-17T08:42:06.680051+0000","last_active":"2024-07-17T08:42:07.981819+0000","last_peered":"2024-07-17T08:42:07.981819+0000","last_clean":"2024-07-17T08:42:07.981819+0000","last_became_active":"2024-07-17T08:42:06.679019+0000","last_became_peered":"2024-07-17T08:42:06.679019+0000","last_unstale":"2024-07-17T08:42:07.981819+0000","last_undegraded":"2024-07-17T08:42:07.981819+0000","last_fullsized":"2024-07-17T08:42:07.981819+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:02.985055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.15","version":"320'3","reported_seq":17,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982129+0000","last_change":"2024-07-17T08:42:05.659058+0000","last_active":"2024-07-17T08:42:07.982129+0000","last_peered":"2024-07-17T08:42:07.982129+0000","last_clean":"2024-07-17T08:42:07.982129+0000","last_became_active":"2024-07-17T08:42:05.658788+0000","last_became_peered":"2024-07-17T08:42:05.658788+0000","last_unstale":"2024-07-17T08:42:07.982129+0000","last_undegraded":"2024-07-17T08:42:07.982129+0000","last_fullsized":"2024-07-17T08:42:07.982129+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:52.167975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.13","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982329+0000","last_change":"2024-07-17T08:42:06.676994+0000","last_active":"2024-07-17T08:42:07.982329+0000","last_peered":"2024-07-17T08:42:07.982329+0000","last_clean":"2024-07-17T08:42:07.982329+0000","last_became_active":"2024-07-17T08:42:06.676384+0000","last_became_peered":"2024-07-17T08:42:06.676384+0000","last_unstale":"2024-07-17T08:42:07.982329+0000","last_undegraded":"2024-07-17T08:42:07.982329+0000","last_fullsized":"2024-07-17T08:42:07.982329+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:27.562983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.14","version":"320'6","reported_seq":20,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658026+0000","last_change":"2024-07-17T08:42:05.660141+0000","last_active":"2024-07-17T08:42:07.658026+0000","last_peered":"2024-07-17T08:42:07.658026+0000","last_clean":"2024-07-17T08:42:07.658026+0000","last_became_active":"2024-07-17T08:42:05.659866+0000","last_became_peered":"2024-07-17T08:42:05.659866+0000","last_unstale":"2024-07-17T08:42:07.658026+0000","last_undegraded":"2024-07-17T08:42:07.658026+0000","last_fullsized":"2024-07-17T08:42:07.658026+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:11.563107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.14","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981995+0000","last_change":"2024-07-17T08:42:06.680810+0000","last_active":"2024-07-17T08:42:07.981995+0000","last_peered":"2024-07-17T08:42:07.981995+0000","last_clean":"2024-07-17T08:42:07.981995+0000","last_became_active":"2024-07-17T08:42:06.680423+0000","last_became_peered":"2024-07-17T08:42:06.680423+0000","last_unstale":"2024-07-17T08:42:07.981995+0000","last_undegraded":"2024-07-17T08:42:07.981995+0000","last_fullsized":"2024-07-17T08:42:07.981995+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:17.106070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.13","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660221+0000","last_change":"2024-07-17T08:42:05.662142+0000","last_active":"2024-07-17T08:42:07.660221+0000","last_peered":"2024-07-17T08:42:07.660221+0000","last_clean":"2024-07-17T08:42:07.660221+0000","last_became_active":"2024-07-17T08:42:05.660670+0000","last_became_peered":"2024-07-17T08:42:05.660670+0000","last_unstale":"2024-07-17T08:42:07.660221+0000","last_undegraded":"2024-07-17T08:42:07.660221+0000","last_fullsized":"2024-07-17T08:42:07.660221+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:19.930213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.15","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983597+0000","last_change":"2024-07-17T08:42:06.686334+0000","last_active":"2024-07-17T08:42:07.983597+0000","last_peered":"2024-07-17T08:42:07.983597+0000","last_clean":"2024-07-17T08:42:07.983597+0000","last_became_active":"2024-07-17T08:42:06.686056+0000","last_became_peered":"2024-07-17T08:42:06.686056+0000","last_unstale":"2024-07-17T08:42:07.983597+0000","last_undegraded":"2024-07-17T08:42:07.983597+0000","last_fullsized":"2024-07-17T08:42:07.983597+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:44.117561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.12","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660069+0000","last_change":"2024-07-17T08:42:05.661673+0000","last_active":"2024-07-17T08:42:07.660069+0000","last_peered":"2024-07-17T08:42:07.660069+0000","last_clean":"2024-07-17T08:42:07.660069+0000","last_became_active":"2024-07-17T08:42:05.661105+0000","last_became_peered":"2024-07-17T08:42:05.661105+0000","last_unstale":"2024-07-17T08:42:07.660069+0000","last_undegraded":"2024-07-17T08:42:07.660069+0000","last_fullsized":"2024-07-17T08:42:07.660069+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:11.390459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.16","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985837+0000","last_change":"2024-07-17T08:42:06.693700+0000","last_active":"2024-07-17T08:42:07.985837+0000","last_peered":"2024-07-17T08:42:07.985837+0000","last_clean":"2024-07-17T08:42:07.985837+0000","last_became_active":"2024-07-17T08:42:06.689297+0000","last_became_peered":"2024-07-17T08:42:06.689297+0000","last_unstale":"2024-07-17T08:42:07.985837+0000","last_undegraded":"2024-07-17T08:42:07.985837+0000","last_fullsized":"2024-07-17T08:42:07.985837+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:14.580419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.11","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981828+0000","last_change":"2024-07-17T08:42:05.662092+0000","last_active":"2024-07-17T08:42:07.981828+0000","last_peered":"2024-07-17T08:42:07.981828+0000","last_clean":"2024-07-17T08:42:07.981828+0000","last_became_active":"2024-07-17T08:42:05.661447+0000","last_became_peered":"2024-07-17T08:42:05.661447+0000","last_unstale":"2024-07-17T08:42:07.981828+0000","last_undegraded":"2024-07-17T08:42:07.981828+0000","last_fullsized":"2024-07-17T08:42:07.981828+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:27.300759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.17","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982808+0000","last_change":"2024-07-17T08:42:06.690592+0000","last_active":"2024-07-17T08:42:07.982808+0000","last_peered":"2024-07-17T08:42:07.982808+0000","last_clean":"2024-07-17T08:42:07.982808+0000","last_became_active":"2024-07-17T08:42:06.690294+0000","last_became_peered":"2024-07-17T08:42:06.690294+0000","last_unstale":"2024-07-17T08:42:07.982808+0000","last_undegraded":"2024-07-17T08:42:07.982808+0000","last_fullsized":"2024-07-17T08:42:07.982808+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:25.770977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.10","version":"320'8","reported_seq":22,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657203+0000","last_change":"2024-07-17T08:42:05.655757+0000","last_active":"2024-07-17T08:42:07.657203+0000","last_peered":"2024-07-17T08:42:07.657203+0000","last_clean":"2024-07-17T08:42:07.657203+0000","last_became_active":"2024-07-17T08:42:05.654293+0000","last_became_peered":"2024-07-17T08:42:05.654293+0000","last_unstale":"2024-07-17T08:42:07.657203+0000","last_undegraded":"2024-07-17T08:42:07.657203+0000","last_fullsized":"2024-07-17T08:42:07.657203+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:34.595127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.8","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659796+0000","last_change":"2024-07-17T08:42:06.674013+0000","last_active":"2024-07-17T08:42:07.659796+0000","last_peered":"2024-07-17T08:42:07.659796+0000","last_clean":"2024-07-17T08:42:07.659796+0000","last_became_active":"2024-07-17T08:42:06.673720+0000","last_became_peered":"2024-07-17T08:42:06.673720+0000","last_unstale":"2024-07-17T08:42:07.659796+0000","last_undegraded":"2024-07-17T08:42:07.659796+0000","last_fullsized":"2024-07-17T08:42:07.659796+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:58.469274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.f","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661747+0000","last_change":"2024-07-17T08:42:05.670380+0000","last_active":"2024-07-17T08:42:07.661747+0000","last_peered":"2024-07-17T08:42:07.661747+0000","last_clean":"2024-07-17T08:42:07.661747+0000","last_became_active":"2024-07-17T08:42:05.669956+0000","last_became_peered":"2024-07-17T08:42:05.669956+0000","last_unstale":"2024-07-17T08:42:07.661747+0000","last_undegraded":"2024-07-17T08:42:07.661747+0000","last_fullsized":"2024-07-17T08:42:07.661747+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:52.606014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.9","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657854+0000","last_change":"2024-07-17T08:42:06.692716+0000","last_active":"2024-07-17T08:42:07.657854+0000","last_peered":"2024-07-17T08:42:07.657854+0000","last_clean":"2024-07-17T08:42:07.657854+0000","last_became_active":"2024-07-17T08:42:06.692432+0000","last_became_peered":"2024-07-17T08:42:06.692432+0000","last_unstale":"2024-07-17T08:42:07.657854+0000","last_undegraded":"2024-07-17T08:42:07.657854+0000","last_fullsized":"2024-07-17T08:42:07.657854+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:54:19.762262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.e","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661104+0000","last_change":"2024-07-17T08:42:05.657820+0000","last_active":"2024-07-17T08:42:07.661104+0000","last_peered":"2024-07-17T08:42:07.661104+0000","last_clean":"2024-07-17T08:42:07.661104+0000","last_became_active":"2024-07-17T08:42:05.657530+0000","last_became_peered":"2024-07-17T08:42:05.657530+0000","last_unstale":"2024-07-17T08:42:07.661104+0000","last_undegraded":"2024-07-17T08:42:07.661104+0000","last_fullsized":"2024-07-17T08:42:07.661104+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:47:44.234698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983628+0000","last_change":"2024-07-17T08:42:06.685191+0000","last_active":"2024-07-17T08:42:07.983628+0000","last_peered":"2024-07-17T08:42:07.983628+0000","last_clean":"2024-07-17T08:42:07.983628+0000","last_became_active":"2024-07-17T08:42:06.684878+0000","last_became_peered":"2024-07-17T08:42:06.684878+0000","last_unstale":"2024-07-17T08:42:07.983628+0000","last_undegraded":"2024-07-17T08:42:07.983628+0000","last_fullsized":"2024-07-17T08:42:07.983628+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:44.386511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.d","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985521+0000","last_change":"2024-07-17T08:42:05.670466+0000","last_active":"2024-07-17T08:42:07.985521+0000","last_peered":"2024-07-17T08:42:07.985521+0000","last_clean":"2024-07-17T08:42:07.985521+0000","last_became_active":"2024-07-17T08:42:05.670171+0000","last_became_peered":"2024-07-17T08:42:05.670171+0000","last_unstale":"2024-07-17T08:42:07.985521+0000","last_undegraded":"2024-07-17T08:42:07.985521+0000","last_fullsized":"2024-07-17T08:42:07.985521+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:31.486090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660724+0000","last_change":"2024-07-17T08:42:06.684917+0000","last_active":"2024-07-17T08:42:07.660724+0000","last_peered":"2024-07-17T08:42:07.660724+0000","last_clean":"2024-07-17T08:42:07.660724+0000","last_became_active":"2024-07-17T08:42:06.684558+0000","last_became_peered":"2024-07-17T08:42:06.684558+0000","last_unstale":"2024-07-17T08:42:07.660724+0000","last_undegraded":"2024-07-17T08:42:07.660724+0000","last_fullsized":"2024-07-17T08:42:07.660724+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:41:07.850142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.c","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985360+0000","last_change":"2024-07-17T08:42:05.667966+0000","last_active":"2024-07-17T08:42:07.985360+0000","last_peered":"2024-07-17T08:42:07.985360+0000","last_clean":"2024-07-17T08:42:07.985360+0000","last_became_active":"2024-07-17T08:42:05.667650+0000","last_became_peered":"2024-07-17T08:42:05.667650+0000","last_unstale":"2024-07-17T08:42:07.985360+0000","last_undegraded":"2024-07-17T08:42:07.985360+0000","last_fullsized":"2024-07-17T08:42:07.985360+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:54.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981888+0000","last_change":"2024-07-17T08:42:06.681536+0000","last_active":"2024-07-17T08:42:07.981888+0000","last_peered":"2024-07-17T08:42:07.981888+0000","last_clean":"2024-07-17T08:42:07.981888+0000","last_became_active":"2024-07-17T08:42:06.681183+0000","last_became_peered":"2024-07-17T08:42:06.681183+0000","last_unstale":"2024-07-17T08:42:07.981888+0000","last_undegraded":"2024-07-17T08:42:07.981888+0000","last_fullsized":"2024-07-17T08:42:07.981888+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:52:20.993804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.b","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660585+0000","last_change":"2024-07-17T08:42:05.653735+0000","last_active":"2024-07-17T08:42:07.660585+0000","last_peered":"2024-07-17T08:42:07.660585+0000","last_clean":"2024-07-17T08:42:07.660585+0000","last_became_active":"2024-07-17T08:42:05.653409+0000","last_became_peered":"2024-07-17T08:42:05.653409+0000","last_unstale":"2024-07-17T08:42:07.660585+0000","last_undegraded":"2024-07-17T08:42:07.660585+0000","last_fullsized":"2024-07-17T08:42:07.660585+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:44:24.500271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661583+0000","last_change":"2024-07-17T08:42:06.677145+0000","last_active":"2024-07-17T08:42:07.661583+0000","last_peered":"2024-07-17T08:42:07.661583+0000","last_clean":"2024-07-17T08:42:07.661583+0000","last_became_active":"2024-07-17T08:42:06.676900+0000","last_became_peered":"2024-07-17T08:42:06.676900+0000","last_unstale":"2024-07-17T08:42:07.661583+0000","last_undegraded":"2024-07-17T08:42:07.661583+0000","last_fullsized":"2024-07-17T08:42:07.661583+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:33.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.a","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984124+0000","last_change":"2024-07-17T08:42:05.667393+0000","last_active":"2024-07-17T08:42:07.984124+0000","last_peered":"2024-07-17T08:42:07.984124+0000","last_clean":"2024-07-17T08:42:07.984124+0000","last_became_active":"2024-07-17T08:42:05.666967+0000","last_became_peered":"2024-07-17T08:42:05.666967+0000","last_unstale":"2024-07-17T08:42:07.984124+0000","last_undegraded":"2024-07-17T08:42:07.984124+0000","last_fullsized":"2024-07-17T08:42:07.984124+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:24.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661436+0000","last_change":"2024-07-17T08:42:06.688484+0000","last_active":"2024-07-17T08:42:07.661436+0000","last_peered":"2024-07-17T08:42:07.661436+0000","last_clean":"2024-07-17T08:42:07.661436+0000","last_became_active":"2024-07-17T08:42:06.687865+0000","last_became_peered":"2024-07-17T08:42:06.687865+0000","last_unstale":"2024-07-17T08:42:07.661436+0000","last_undegraded":"2024-07-17T08:42:07.661436+0000","last_fullsized":"2024-07-17T08:42:07.661436+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:58.488319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.9","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982067+0000","last_change":"2024-07-17T08:42:05.662923+0000","last_active":"2024-07-17T08:42:07.982067+0000","last_peered":"2024-07-17T08:42:07.982067+0000","last_clean":"2024-07-17T08:42:07.982067+0000","last_became_active":"2024-07-17T08:42:05.661824+0000","last_became_peered":"2024-07-17T08:42:05.661824+0000","last_unstale":"2024-07-17T08:42:07.982067+0000","last_undegraded":"2024-07-17T08:42:07.982067+0000","last_fullsized":"2024-07-17T08:42:07.982067+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:12.515784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983414+0000","last_change":"2024-07-17T08:42:06.680072+0000","last_active":"2024-07-17T08:42:07.983414+0000","last_peered":"2024-07-17T08:42:07.983414+0000","last_clean":"2024-07-17T08:42:07.983414+0000","last_became_active":"2024-07-17T08:42:06.679674+0000","last_became_peered":"2024-07-17T08:42:06.679674+0000","last_unstale":"2024-07-17T08:42:07.983414+0000","last_undegraded":"2024-07-17T08:42:07.983414+0000","last_fullsized":"2024-07-17T08:42:07.983414+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:20.678205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.8","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981003+0000","last_change":"2024-07-17T08:42:05.669408+0000","last_active":"2024-07-17T08:42:07.981003+0000","last_peered":"2024-07-17T08:42:07.981003+0000","last_clean":"2024-07-17T08:42:07.981003+0000","last_became_active":"2024-07-17T08:42:05.668987+0000","last_became_peered":"2024-07-17T08:42:05.668987+0000","last_unstale":"2024-07-17T08:42:07.981003+0000","last_undegraded":"2024-07-17T08:42:07.981003+0000","last_fullsized":"2024-07-17T08:42:07.981003+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:36.005753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.0","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981441+0000","last_change":"2024-07-17T08:42:06.678892+0000","last_active":"2024-07-17T08:42:07.981441+0000","last_peered":"2024-07-17T08:42:07.981441+0000","last_clean":"2024-07-17T08:42:07.981441+0000","last_became_active":"2024-07-17T08:42:06.678545+0000","last_became_peered":"2024-07-17T08:42:06.678545+0000","last_unstale":"2024-07-17T08:42:07.981441+0000","last_undegraded":"2024-07-17T08:42:07.981441+0000","last_fullsized":"2024-07-17T08:42:07.981441+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:49.491439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.7","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982465+0000","last_change":"2024-07-17T08:42:05.666706+0000","last_active":"2024-07-17T08:42:07.982465+0000","last_peered":"2024-07-17T08:42:07.982465+0000","last_clean":"2024-07-17T08:42:07.982465+0000","last_became_active":"2024-07-17T08:42:05.665981+0000","last_became_peered":"2024-07-17T08:42:05.665981+0000","last_unstale":"2024-07-17T08:42:07.982465+0000","last_undegraded":"2024-07-17T08:42:07.982465+0000","last_fullsized":"2024-07-17T08:42:07.982465+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:50.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.1","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982373+0000","last_change":"2024-07-17T08:42:06.683511+0000","last_active":"2024-07-17T08:42:07.982373+0000","last_peered":"2024-07-17T08:42:07.982373+0000","last_clean":"2024-07-17T08:42:07.982373+0000","last_became_active":"2024-07-17T08:42:06.683224+0000","last_became_peered":"2024-07-17T08:42:06.683224+0000","last_unstale":"2024-07-17T08:42:07.982373+0000","last_undegraded":"2024-07-17T08:42:07.982373+0000","last_fullsized":"2024-07-17T08:42:07.982373+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:24:35.662352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.6","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983874+0000","last_change":"2024-07-17T08:42:05.662691+0000","last_active":"2024-07-17T08:42:07.983874+0000","last_peered":"2024-07-17T08:42:07.983874+0000","last_clean":"2024-07-17T08:42:07.983874+0000","last_became_active":"2024-07-17T08:42:05.662376+0000","last_became_peered":"2024-07-17T08:42:05.662376+0000","last_unstale":"2024-07-17T08:42:07.983874+0000","last_undegraded":"2024-07-17T08:42:07.983874+0000","last_fullsized":"2024-07-17T08:42:07.983874+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:21.463356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.2","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660155+0000","last_change":"2024-07-17T08:42:06.685897+0000","last_active":"2024-07-17T08:42:07.660155+0000","last_peered":"2024-07-17T08:42:07.660155+0000","last_clean":"2024-07-17T08:42:07.660155+0000","last_became_active":"2024-07-17T08:42:06.685583+0000","last_became_peered":"2024-07-17T08:42:06.685583+0000","last_unstale":"2024-07-17T08:42:07.660155+0000","last_undegraded":"2024-07-17T08:42:07.660155+0000","last_fullsized":"2024-07-17T08:42:07.660155+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.439424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.5","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661267+0000","last_change":"2024-07-17T08:42:05.669462+0000","last_active":"2024-07-17T08:42:07.661267+0000","last_peered":"2024-07-17T08:42:07.661267+0000","last_clean":"2024-07-17T08:42:07.661267+0000","last_became_active":"2024-07-17T08:42:05.668937+0000","last_became_peered":"2024-07-17T08:42:05.668937+0000","last_unstale":"2024-07-17T08:42:07.661267+0000","last_undegraded":"2024-07-17T08:42:07.661267+0000","last_fullsized":"2024-07-17T08:42:07.661267+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:47.434897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.3","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982593+0000","last_change":"2024-07-17T08:42:06.675237+0000","last_active":"2024-07-17T08:42:07.982593+0000","last_peered":"2024-07-17T08:42:07.982593+0000","last_clean":"2024-07-17T08:42:07.982593+0000","last_became_active":"2024-07-17T08:42:06.674960+0000","last_became_peered":"2024-07-17T08:42:06.674960+0000","last_unstale":"2024-07-17T08:42:07.982593+0000","last_undegraded":"2024-07-17T08:42:07.982593+0000","last_fullsized":"2024-07-17T08:42:07.982593+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:44.540505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.4","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658735+0000","last_change":"2024-07-17T08:42:05.665881+0000","last_active":"2024-07-17T08:42:07.658735+0000","last_peered":"2024-07-17T08:42:07.658735+0000","last_clean":"2024-07-17T08:42:07.658735+0000","last_became_active":"2024-07-17T08:42:05.665565+0000","last_became_peered":"2024-07-17T08:42:05.665565+0000","last_unstale":"2024-07-17T08:42:07.658735+0000","last_undegraded":"2024-07-17T08:42:07.658735+0000","last_fullsized":"2024-07-17T08:42:07.658735+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:30.101051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.4","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982717+0000","last_change":"2024-07-17T08:42:06.687005+0000","last_active":"2024-07-17T08:42:07.982717+0000","last_peered":"2024-07-17T08:42:07.982717+0000","last_clean":"2024-07-17T08:42:07.982717+0000","last_became_active":"2024-07-17T08:42:06.682854+0000","last_became_peered":"2024-07-17T08:42:06.682854+0000","last_unstale":"2024-07-17T08:42:07.982717+0000","last_undegraded":"2024-07-17T08:42:07.982717+0000","last_fullsized":"2024-07-17T08:42:07.982717+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:20.518080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.3","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982849+0000","last_change":"2024-07-17T08:42:05.648439+0000","last_active":"2024-07-17T08:42:07.982849+0000","last_peered":"2024-07-17T08:42:07.982849+0000","last_clean":"2024-07-17T08:42:07.982849+0000","last_became_active":"2024-07-17T08:42:05.648059+0000","last_became_peered":"2024-07-17T08:42:05.648059+0000","last_unstale":"2024-07-17T08:42:07.982849+0000","last_undegraded":"2024-07-17T08:42:07.982849+0000","last_fullsized":"2024-07-17T08:42:07.982849+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:33.427557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.5","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983470+0000","last_change":"2024-07-17T08:42:06.688632+0000","last_active":"2024-07-17T08:42:07.983470+0000","last_peered":"2024-07-17T08:42:07.983470+0000","last_clean":"2024-07-17T08:42:07.983470+0000","last_became_active":"2024-07-17T08:42:06.688348+0000","last_became_peered":"2024-07-17T08:42:06.688348+0000","last_unstale":"2024-07-17T08:42:07.983470+0000","last_undegraded":"2024-07-17T08:42:07.983470+0000","last_fullsized":"2024-07-17T08:42:07.983470+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:25.203321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.2","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981581+0000","last_change":"2024-07-17T08:42:05.662065+0000","last_active":"2024-07-17T08:42:07.981581+0000","last_peered":"2024-07-17T08:42:07.981581+0000","last_clean":"2024-07-17T08:42:07.981581+0000","last_became_active":"2024-07-17T08:42:05.661496+0000","last_became_peered":"2024-07-17T08:42:05.661496+0000","last_unstale":"2024-07-17T08:42:07.981581+0000","last_undegraded":"2024-07-17T08:42:07.981581+0000","last_fullsized":"2024-07-17T08:42:07.981581+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:29.987993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.6","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983650+0000","last_change":"2024-07-17T08:42:06.694032+0000","last_active":"2024-07-17T08:42:07.983650+0000","last_peered":"2024-07-17T08:42:07.983650+0000","last_clean":"2024-07-17T08:42:07.983650+0000","last_became_active":"2024-07-17T08:42:06.693748+0000","last_became_peered":"2024-07-17T08:42:06.693748+0000","last_unstale":"2024-07-17T08:42:07.983650+0000","last_undegraded":"2024-07-17T08:42:07.983650+0000","last_fullsized":"2024-07-17T08:42:07.983650+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:39.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.1","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982300+0000","last_change":"2024-07-17T08:42:05.660712+0000","last_active":"2024-07-17T08:42:07.982300+0000","last_peered":"2024-07-17T08:42:07.982300+0000","last_clean":"2024-07-17T08:42:07.982300+0000","last_became_active":"2024-07-17T08:42:05.660426+0000","last_became_peered":"2024-07-17T08:42:05.660426+0000","last_unstale":"2024-07-17T08:42:07.982300+0000","last_undegraded":"2024-07-17T08:42:07.982300+0000","last_fullsized":"2024-07-17T08:42:07.982300+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:10.408996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.7","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658626+0000","last_change":"2024-07-17T08:42:06.697677+0000","last_active":"2024-07-17T08:42:07.658626+0000","last_peered":"2024-07-17T08:42:07.658626+0000","last_clean":"2024-07-17T08:42:07.658626+0000","last_became_active":"2024-07-17T08:42:06.696346+0000","last_became_peered":"2024-07-17T08:42:06.696346+0000","last_unstale":"2024-07-17T08:42:07.658626+0000","last_undegraded":"2024-07-17T08:42:07.658626+0000","last_fullsized":"2024-07-17T08:42:07.658626+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:14.310708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.0","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660069+0000","last_change":"2024-07-17T08:42:05.653041+0000","last_active":"2024-07-17T08:42:07.660069+0000","last_peered":"2024-07-17T08:42:07.660069+0000","last_clean":"2024-07-17T08:42:07.660069+0000","last_became_active":"2024-07-17T08:42:05.652757+0000","last_became_peered":"2024-07-17T08:42:05.652757+0000","last_unstale":"2024-07-17T08:42:07.660069+0000","last_undegraded":"2024-07-17T08:42:07.660069+0000","last_fullsized":"2024-07-17T08:42:07.660069+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:38:04.044951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.2d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660861+0000","last_change":"2024-07-17T08:42:06.697214+0000","last_active":"2024-07-17T08:42:07.660861+0000","last_peered":"2024-07-17T08:42:07.660861+0000","last_clean":"2024-07-17T08:42:07.660861+0000","last_became_active":"2024-07-17T08:42:06.696910+0000","last_became_peered":"2024-07-17T08:42:06.696910+0000","last_unstale":"2024-07-17T08:42:07.660861+0000","last_undegraded":"2024-07-17T08:42:07.660861+0000","last_fullsized":"2024-07-17T08:42:07.660861+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:49:11.066618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"311'4624","reported_seq":5902,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981140+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:42:07.981140+0000","last_peered":"2024-07-17T08:42:07.981140+0000","last_clean":"2024-07-17T08:42:07.981140+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:42:07.981140+0000","last_undegraded":"2024-07-17T08:42:07.981140+0000","last_fullsized":"2024-07-17T08:42:07.981140+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":493,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":44,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":43,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":493,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4624,"ondisk_log_size":4624,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543416,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80168,"kb_used_data":4564,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291672,"statfs":{"total":96636764160,"available":96554672128,"internally_reserved":0,"allocated":4673536,"data_stored":2830982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510711,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53220,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318620,"statfs":{"total":96636764160,"available":96582266880,"internally_reserved":0,"allocated":4079616,"data_stored":2247236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71284,"kb_used_data":4000,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300556,"statfs":{"total":96636764160,"available":96563769344,"internally_reserved":0,"allocated":4096000,"data_stored":2249430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543417,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53216,"kb_used_data":3988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318624,"statfs":{"total":96636764160,"available":96582270976,"internally_reserved":0,"allocated":4083712,"data_stored":2247326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57396,"kb_used_data":4000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314444,"statfs":{"total":96636764160,"available":96577990656,"internally_reserved":0,"allocated":4096000,"data_stored":2248938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543415,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57364,"kb_used_data":3976,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314476,"statfs":{"total":96636764160,"available":96578023424,"internally_reserved":0,"allocated":4071424,"data_stored":2239974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510712,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53228,"kb_used_data":3996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318612,"statfs":{"total":96636764160,"available":96582258688,"internally_reserved":0,"allocated":4091904,"data_stored":2247362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510711,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53796,"kb_used_data":4564,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318044,"statfs":{"total":96636764160,"available":96581677056,"internally_reserved":0,"allocated":4673536,"data_stored":2837604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:42:11.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 -- 172.21.15.44:0/144825554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0b8c80 msgr2=0x7fd5ac0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/144825554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0b8c80 0x7fd5ac0b4ec0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005fd0 tx=0x7fd59c02a620 comp rx=0 tx=0).stop 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 -- 172.21.15.44:0/144825554 shutdown_connections 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/144825554 >> v2:172.21.15.175:3300/0 conn(0x7fd5ac0ce5c0 0x7fd5ac06d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/144825554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0b8c80 0x7fd5ac0b4ec0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 -- 172.21.15.44:0/144825554 >> 172.21.15.44:0/144825554 conn(0x7fd5ac0cfec0 msgr2=0x7fd5ac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 -- 172.21.15.44:0/144825554 shutdown_connections 2024-07-17T08:42:11.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.416+0000 7fd5b0e00640 1 -- 172.21.15.44:0/144825554 wait complete. 2024-07-17T08:42:11.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.418+0000 7fd5b0e00640 1 Processor -- start 2024-07-17T08:42:11.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.418+0000 7fd5b0e00640 1 -- start start 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac0b5400 0x7fd5ac0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac0b8c80 0x7fd5ac158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac06e780 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac06e600 con 0x7fd5ac0b5400 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5b0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac06e900 con 0x7fd5ac0b8c80 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58324/0 (socket says 172.21.15.44:58324) 2024-07-17T08:42:11.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.419+0000 7fd5abe00640 1 -- 172.21.15.44:0/1892005131 learned_addr learned my addr 172.21.15.44:0/1892005131 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 -- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac0b8c80 msgr2=0x7fd5ac158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac0b8c80 0x7fd5ac158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 -- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac0b5400 msgr2=0x7fd5ac0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac0b5400 0x7fd5ac0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 -- 172.21.15.44:0/1892005131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac0c6ff0 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5abe00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c7a0 tx=0x7fd5a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4003b70 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5a4003d30 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac0c72d0 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.420+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400aab0 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.421+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac0c7890 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.422+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac0ce220 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.423+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fd5a400ac50 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.424+0000 7fd5a3400640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 0x7fd58c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.424+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fd5a4098790 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.425+0000 7fd5ab400640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 0x7fd58c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.426+0000 7fd5ab400640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 0x7fd58c079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd59c033000 tx=0x7fd59c0046c0 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:11.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.428+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd5ac0ce220 con 0x7fd5ac0ce5c0 2024-07-17T08:42:11.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.636+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 --> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5ac15e570 con 0x7fd58c076cd0 2024-07-17T08:42:11.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.649+0000 7fd5a3400640 1 -- 172.21.15.44:0/1892005131 <== mgr.10276 v2:172.21.15.175:6834/3850741755 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7fd5ac15e570 con 0x7fd58c076cd0 2024-07-17T08:42:11.652 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:42:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.655+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 msgr2=0x7fd58c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.655+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 0x7fd58c079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd59c033000 tx=0x7fd59c0046c0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.655+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 msgr2=0x7fd5ac0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.655+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400c7a0 tx=0x7fd5a400cc60 comp rx=0 tx=0).stop 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 shutdown_connections 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd58c076cd0 0x7fd58c079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5ac0ce5c0 0x7fd5ac0c9b90 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac0b8c80 0x7fd5ac158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 --2- 172.21.15.44:0/1892005131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac0b5400 0x7fd5ac0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 >> 172.21.15.44:0/1892005131 conn(0x7fd5ac0cfec0 msgr2=0x7fd5ac0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 shutdown_connections 2024-07-17T08:42:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.656+0000 7fd5b0e00640 1 -- 172.21.15.44:0/1892005131 wait complete. 2024-07-17T08:42:11.676 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:42:10.381901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":537,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4662,"ondisk_log_size":4662,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":479672,"kb_used_data":33072,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754495048,"statfs":{"total":773094113280,"available":772602929152,"internally_reserved":0,"allocated":33865728,"data_stored":19148852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756082"},"pg_stats":[{"pgid":"44.3f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661472+0000","last_change":"2024-07-17T08:42:06.685173+0000","last_active":"2024-07-17T08:42:07.661472+0000","last_peered":"2024-07-17T08:42:07.661472+0000","last_clean":"2024-07-17T08:42:07.661472+0000","last_became_active":"2024-07-17T08:42:06.684915+0000","last_became_peered":"2024-07-17T08:42:06.684915+0000","last_unstale":"2024-07-17T08:42:07.661472+0000","last_undegraded":"2024-07-17T08:42:07.661472+0000","last_fullsized":"2024-07-17T08:42:07.661472+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:51.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.3e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985408+0000","last_change":"2024-07-17T08:42:06.682009+0000","last_active":"2024-07-17T08:42:07.985408+0000","last_peered":"2024-07-17T08:42:07.985408+0000","last_clean":"2024-07-17T08:42:07.985408+0000","last_became_active":"2024-07-17T08:42:06.681739+0000","last_became_peered":"2024-07-17T08:42:06.681739+0000","last_unstale":"2024-07-17T08:42:07.985408+0000","last_undegraded":"2024-07-17T08:42:07.985408+0000","last_fullsized":"2024-07-17T08:42:07.985408+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:14:43.533749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.3d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982803+0000","last_change":"2024-07-17T08:42:06.671934+0000","last_active":"2024-07-17T08:42:07.982803+0000","last_peered":"2024-07-17T08:42:07.982803+0000","last_clean":"2024-07-17T08:42:07.982803+0000","last_became_active":"2024-07-17T08:42:06.671670+0000","last_became_peered":"2024-07-17T08:42:06.671670+0000","last_unstale":"2024-07-17T08:42:07.982803+0000","last_undegraded":"2024-07-17T08:42:07.982803+0000","last_fullsized":"2024-07-17T08:42:07.982803+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:04:33.549416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.3c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985067+0000","last_change":"2024-07-17T08:42:06.684073+0000","last_active":"2024-07-17T08:42:07.985067+0000","last_peered":"2024-07-17T08:42:07.985067+0000","last_clean":"2024-07-17T08:42:07.985067+0000","last_became_active":"2024-07-17T08:42:06.683768+0000","last_became_peered":"2024-07-17T08:42:06.683768+0000","last_unstale":"2024-07-17T08:42:07.985067+0000","last_undegraded":"2024-07-17T08:42:07.985067+0000","last_fullsized":"2024-07-17T08:42:07.985067+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:17:01.851954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.3b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982149+0000","last_change":"2024-07-17T08:42:06.683682+0000","last_active":"2024-07-17T08:42:07.982149+0000","last_peered":"2024-07-17T08:42:07.982149+0000","last_clean":"2024-07-17T08:42:07.982149+0000","last_became_active":"2024-07-17T08:42:06.683378+0000","last_became_peered":"2024-07-17T08:42:06.683378+0000","last_unstale":"2024-07-17T08:42:07.982149+0000","last_undegraded":"2024-07-17T08:42:07.982149+0000","last_fullsized":"2024-07-17T08:42:07.982149+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:50.840183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.3a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982275+0000","last_change":"2024-07-17T08:42:06.696413+0000","last_active":"2024-07-17T08:42:07.982275+0000","last_peered":"2024-07-17T08:42:07.982275+0000","last_clean":"2024-07-17T08:42:07.982275+0000","last_became_active":"2024-07-17T08:42:06.696128+0000","last_became_peered":"2024-07-17T08:42:06.696128+0000","last_unstale":"2024-07-17T08:42:07.982275+0000","last_undegraded":"2024-07-17T08:42:07.982275+0000","last_fullsized":"2024-07-17T08:42:07.982275+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:51.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.39","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657883+0000","last_change":"2024-07-17T08:42:06.692670+0000","last_active":"2024-07-17T08:42:07.657883+0000","last_peered":"2024-07-17T08:42:07.657883+0000","last_clean":"2024-07-17T08:42:07.657883+0000","last_became_active":"2024-07-17T08:42:06.692082+0000","last_became_peered":"2024-07-17T08:42:06.692082+0000","last_unstale":"2024-07-17T08:42:07.657883+0000","last_undegraded":"2024-07-17T08:42:07.657883+0000","last_fullsized":"2024-07-17T08:42:07.657883+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:33.257603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.38","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660178+0000","last_change":"2024-07-17T08:42:06.688736+0000","last_active":"2024-07-17T08:42:07.660178+0000","last_peered":"2024-07-17T08:42:07.660178+0000","last_clean":"2024-07-17T08:42:07.660178+0000","last_became_active":"2024-07-17T08:42:06.688425+0000","last_became_peered":"2024-07-17T08:42:06.688425+0000","last_unstale":"2024-07-17T08:42:07.660178+0000","last_undegraded":"2024-07-17T08:42:07.660178+0000","last_fullsized":"2024-07-17T08:42:07.660178+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:54.620438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.37","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661323+0000","last_change":"2024-07-17T08:42:06.689965+0000","last_active":"2024-07-17T08:42:07.661323+0000","last_peered":"2024-07-17T08:42:07.661323+0000","last_clean":"2024-07-17T08:42:07.661323+0000","last_became_active":"2024-07-17T08:42:06.689628+0000","last_became_peered":"2024-07-17T08:42:06.689628+0000","last_unstale":"2024-07-17T08:42:07.661323+0000","last_undegraded":"2024-07-17T08:42:07.661323+0000","last_fullsized":"2024-07-17T08:42:07.661323+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:07:30.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.36","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661740+0000","last_change":"2024-07-17T08:42:06.689771+0000","last_active":"2024-07-17T08:42:07.661740+0000","last_peered":"2024-07-17T08:42:07.661740+0000","last_clean":"2024-07-17T08:42:07.661740+0000","last_became_active":"2024-07-17T08:42:06.688293+0000","last_became_peered":"2024-07-17T08:42:06.688293+0000","last_unstale":"2024-07-17T08:42:07.661740+0000","last_undegraded":"2024-07-17T08:42:07.661740+0000","last_fullsized":"2024-07-17T08:42:07.661740+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:47.790788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.35","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661073+0000","last_change":"2024-07-17T08:42:06.688653+0000","last_active":"2024-07-17T08:42:07.661073+0000","last_peered":"2024-07-17T08:42:07.661073+0000","last_clean":"2024-07-17T08:42:07.661073+0000","last_became_active":"2024-07-17T08:42:06.688351+0000","last_became_peered":"2024-07-17T08:42:06.688351+0000","last_unstale":"2024-07-17T08:42:07.661073+0000","last_undegraded":"2024-07-17T08:42:07.661073+0000","last_fullsized":"2024-07-17T08:42:07.661073+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:45.546848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.34","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658826+0000","last_change":"2024-07-17T08:42:06.691748+0000","last_active":"2024-07-17T08:42:07.658826+0000","last_peered":"2024-07-17T08:42:07.658826+0000","last_clean":"2024-07-17T08:42:07.658826+0000","last_became_active":"2024-07-17T08:42:06.687996+0000","last_became_peered":"2024-07-17T08:42:06.687996+0000","last_unstale":"2024-07-17T08:42:07.658826+0000","last_undegraded":"2024-07-17T08:42:07.658826+0000","last_fullsized":"2024-07-17T08:42:07.658826+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:45:01.082911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.33","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984798+0000","last_change":"2024-07-17T08:42:06.694341+0000","last_active":"2024-07-17T08:42:07.984798+0000","last_peered":"2024-07-17T08:42:07.984798+0000","last_clean":"2024-07-17T08:42:07.984798+0000","last_became_active":"2024-07-17T08:42:06.694005+0000","last_became_peered":"2024-07-17T08:42:06.694005+0000","last_unstale":"2024-07-17T08:42:07.984798+0000","last_undegraded":"2024-07-17T08:42:07.984798+0000","last_fullsized":"2024-07-17T08:42:07.984798+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:19:58.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.32","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982720+0000","last_change":"2024-07-17T08:42:06.678319+0000","last_active":"2024-07-17T08:42:07.982720+0000","last_peered":"2024-07-17T08:42:07.982720+0000","last_clean":"2024-07-17T08:42:07.982720+0000","last_became_active":"2024-07-17T08:42:06.678001+0000","last_became_peered":"2024-07-17T08:42:06.678001+0000","last_unstale":"2024-07-17T08:42:07.982720+0000","last_undegraded":"2024-07-17T08:42:07.982720+0000","last_fullsized":"2024-07-17T08:42:07.982720+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:04.540386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.31","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660209+0000","last_change":"2024-07-17T08:42:06.683786+0000","last_active":"2024-07-17T08:42:07.660209+0000","last_peered":"2024-07-17T08:42:07.660209+0000","last_clean":"2024-07-17T08:42:07.660209+0000","last_became_active":"2024-07-17T08:42:06.683511+0000","last_became_peered":"2024-07-17T08:42:06.683511+0000","last_unstale":"2024-07-17T08:42:07.660209+0000","last_undegraded":"2024-07-17T08:42:07.660209+0000","last_fullsized":"2024-07-17T08:42:07.660209+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:45.932665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.30","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659816+0000","last_change":"2024-07-17T08:42:06.684355+0000","last_active":"2024-07-17T08:42:07.659816+0000","last_peered":"2024-07-17T08:42:07.659816+0000","last_clean":"2024-07-17T08:42:07.659816+0000","last_became_active":"2024-07-17T08:42:06.684061+0000","last_became_peered":"2024-07-17T08:42:06.684061+0000","last_unstale":"2024-07-17T08:42:07.659816+0000","last_undegraded":"2024-07-17T08:42:07.659816+0000","last_fullsized":"2024-07-17T08:42:07.659816+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:30:07.526346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.2f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658418+0000","last_change":"2024-07-17T08:42:06.692345+0000","last_active":"2024-07-17T08:42:07.658418+0000","last_peered":"2024-07-17T08:42:07.658418+0000","last_clean":"2024-07-17T08:42:07.658418+0000","last_became_active":"2024-07-17T08:42:06.691974+0000","last_became_peered":"2024-07-17T08:42:06.691974+0000","last_unstale":"2024-07-17T08:42:07.658418+0000","last_undegraded":"2024-07-17T08:42:07.658418+0000","last_fullsized":"2024-07-17T08:42:07.658418+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:10.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.2e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982820+0000","last_change":"2024-07-17T08:42:06.694280+0000","last_active":"2024-07-17T08:42:07.982820+0000","last_peered":"2024-07-17T08:42:07.982820+0000","last_clean":"2024-07-17T08:42:07.982820+0000","last_became_active":"2024-07-17T08:42:06.693680+0000","last_became_peered":"2024-07-17T08:42:06.693680+0000","last_unstale":"2024-07-17T08:42:07.982820+0000","last_undegraded":"2024-07-17T08:42:07.982820+0000","last_fullsized":"2024-07-17T08:42:07.982820+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:14.156933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.2c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985308+0000","last_change":"2024-07-17T08:42:06.690221+0000","last_active":"2024-07-17T08:42:07.985308+0000","last_peered":"2024-07-17T08:42:07.985308+0000","last_clean":"2024-07-17T08:42:07.985308+0000","last_became_active":"2024-07-17T08:42:06.689914+0000","last_became_peered":"2024-07-17T08:42:06.689914+0000","last_unstale":"2024-07-17T08:42:07.985308+0000","last_undegraded":"2024-07-17T08:42:07.985308+0000","last_fullsized":"2024-07-17T08:42:07.985308+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:04:03.691858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.2b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983322+0000","last_change":"2024-07-17T08:42:06.682270+0000","last_active":"2024-07-17T08:42:07.983322+0000","last_peered":"2024-07-17T08:42:07.983322+0000","last_clean":"2024-07-17T08:42:07.983322+0000","last_became_active":"2024-07-17T08:42:06.681027+0000","last_became_peered":"2024-07-17T08:42:06.681027+0000","last_unstale":"2024-07-17T08:42:07.983322+0000","last_undegraded":"2024-07-17T08:42:07.983322+0000","last_fullsized":"2024-07-17T08:42:07.983322+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:27.221763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.2a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982152+0000","last_change":"2024-07-17T08:42:06.667352+0000","last_active":"2024-07-17T08:42:07.982152+0000","last_peered":"2024-07-17T08:42:07.982152+0000","last_clean":"2024-07-17T08:42:07.982152+0000","last_became_active":"2024-07-17T08:42:06.666995+0000","last_became_peered":"2024-07-17T08:42:06.666995+0000","last_unstale":"2024-07-17T08:42:07.982152+0000","last_undegraded":"2024-07-17T08:42:07.982152+0000","last_fullsized":"2024-07-17T08:42:07.982152+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:10:45.237328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.29","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981533+0000","last_change":"2024-07-17T08:42:06.682171+0000","last_active":"2024-07-17T08:42:07.981533+0000","last_peered":"2024-07-17T08:42:07.981533+0000","last_clean":"2024-07-17T08:42:07.981533+0000","last_became_active":"2024-07-17T08:42:06.681632+0000","last_became_peered":"2024-07-17T08:42:06.681632+0000","last_unstale":"2024-07-17T08:42:07.981533+0000","last_undegraded":"2024-07-17T08:42:07.981533+0000","last_fullsized":"2024-07-17T08:42:07.981533+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:58.488319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.28","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983027+0000","last_change":"2024-07-17T08:42:06.697767+0000","last_active":"2024-07-17T08:42:07.983027+0000","last_peered":"2024-07-17T08:42:07.983027+0000","last_clean":"2024-07-17T08:42:07.983027+0000","last_became_active":"2024-07-17T08:42:06.697309+0000","last_became_peered":"2024-07-17T08:42:06.697309+0000","last_unstale":"2024-07-17T08:42:07.983027+0000","last_undegraded":"2024-07-17T08:42:07.983027+0000","last_fullsized":"2024-07-17T08:42:07.983027+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:47.790788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.27","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982386+0000","last_change":"2024-07-17T08:42:06.686378+0000","last_active":"2024-07-17T08:42:07.982386+0000","last_peered":"2024-07-17T08:42:07.982386+0000","last_clean":"2024-07-17T08:42:07.982386+0000","last_became_active":"2024-07-17T08:42:06.686071+0000","last_became_peered":"2024-07-17T08:42:06.686071+0000","last_unstale":"2024-07-17T08:42:07.982386+0000","last_undegraded":"2024-07-17T08:42:07.982386+0000","last_fullsized":"2024-07-17T08:42:07.982386+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:06.210985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.26","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983054+0000","last_change":"2024-07-17T08:42:06.670944+0000","last_active":"2024-07-17T08:42:07.983054+0000","last_peered":"2024-07-17T08:42:07.983054+0000","last_clean":"2024-07-17T08:42:07.983054+0000","last_became_active":"2024-07-17T08:42:06.667772+0000","last_became_peered":"2024-07-17T08:42:06.667772+0000","last_unstale":"2024-07-17T08:42:07.983054+0000","last_undegraded":"2024-07-17T08:42:07.983054+0000","last_fullsized":"2024-07-17T08:42:07.983054+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:59:19.421877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.25","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661467+0000","last_change":"2024-07-17T08:42:06.686850+0000","last_active":"2024-07-17T08:42:07.661467+0000","last_peered":"2024-07-17T08:42:07.661467+0000","last_clean":"2024-07-17T08:42:07.661467+0000","last_became_active":"2024-07-17T08:42:06.686513+0000","last_became_peered":"2024-07-17T08:42:06.686513+0000","last_unstale":"2024-07-17T08:42:07.661467+0000","last_undegraded":"2024-07-17T08:42:07.661467+0000","last_fullsized":"2024-07-17T08:42:07.661467+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:46.284529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.24","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658600+0000","last_change":"2024-07-17T08:42:06.683668+0000","last_active":"2024-07-17T08:42:07.658600+0000","last_peered":"2024-07-17T08:42:07.658600+0000","last_clean":"2024-07-17T08:42:07.658600+0000","last_became_active":"2024-07-17T08:42:06.682943+0000","last_became_peered":"2024-07-17T08:42:06.682943+0000","last_unstale":"2024-07-17T08:42:07.658600+0000","last_undegraded":"2024-07-17T08:42:07.658600+0000","last_fullsized":"2024-07-17T08:42:07.658600+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:42:37.488300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.23","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981988+0000","last_change":"2024-07-17T08:42:06.675986+0000","last_active":"2024-07-17T08:42:07.981988+0000","last_peered":"2024-07-17T08:42:07.981988+0000","last_clean":"2024-07-17T08:42:07.981988+0000","last_became_active":"2024-07-17T08:42:06.667919+0000","last_became_peered":"2024-07-17T08:42:06.667919+0000","last_unstale":"2024-07-17T08:42:07.981988+0000","last_undegraded":"2024-07-17T08:42:07.981988+0000","last_fullsized":"2024-07-17T08:42:07.981988+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:10:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.22","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981614+0000","last_change":"2024-07-17T08:42:06.682603+0000","last_active":"2024-07-17T08:42:07.981614+0000","last_peered":"2024-07-17T08:42:07.981614+0000","last_clean":"2024-07-17T08:42:07.981614+0000","last_became_active":"2024-07-17T08:42:06.682307+0000","last_became_peered":"2024-07-17T08:42:06.682307+0000","last_unstale":"2024-07-17T08:42:07.981614+0000","last_undegraded":"2024-07-17T08:42:07.981614+0000","last_fullsized":"2024-07-17T08:42:07.981614+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:35:24.179304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.21","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984252+0000","last_change":"2024-07-17T08:42:06.691890+0000","last_active":"2024-07-17T08:42:07.984252+0000","last_peered":"2024-07-17T08:42:07.984252+0000","last_clean":"2024-07-17T08:42:07.984252+0000","last_became_active":"2024-07-17T08:42:06.687855+0000","last_became_peered":"2024-07-17T08:42:06.687855+0000","last_unstale":"2024-07-17T08:42:07.984252+0000","last_undegraded":"2024-07-17T08:42:07.984252+0000","last_fullsized":"2024-07-17T08:42:07.984252+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:19.090582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.20","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660019+0000","last_change":"2024-07-17T08:42:06.688833+0000","last_active":"2024-07-17T08:42:07.660019+0000","last_peered":"2024-07-17T08:42:07.660019+0000","last_clean":"2024-07-17T08:42:07.660019+0000","last_became_active":"2024-07-17T08:42:06.688476+0000","last_became_peered":"2024-07-17T08:42:06.688476+0000","last_unstale":"2024-07-17T08:42:07.660019+0000","last_undegraded":"2024-07-17T08:42:07.660019+0000","last_fullsized":"2024-07-17T08:42:07.660019+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:30.851281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.18","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659788+0000","last_change":"2024-07-17T08:42:06.662588+0000","last_active":"2024-07-17T08:42:07.659788+0000","last_peered":"2024-07-17T08:42:07.659788+0000","last_clean":"2024-07-17T08:42:07.659788+0000","last_became_active":"2024-07-17T08:42:06.662190+0000","last_became_peered":"2024-07-17T08:42:06.662190+0000","last_unstale":"2024-07-17T08:42:07.659788+0000","last_undegraded":"2024-07-17T08:42:07.659788+0000","last_fullsized":"2024-07-17T08:42:07.659788+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:40:51.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.1f","version":"320'4","reported_seq":18,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985711+0000","last_change":"2024-07-17T08:42:05.668629+0000","last_active":"2024-07-17T08:42:07.985711+0000","last_peered":"2024-07-17T08:42:07.985711+0000","last_clean":"2024-07-17T08:42:07.985711+0000","last_became_active":"2024-07-17T08:42:05.668320+0000","last_became_peered":"2024-07-17T08:42:05.668320+0000","last_unstale":"2024-07-17T08:42:07.985711+0000","last_undegraded":"2024-07-17T08:42:07.985711+0000","last_fullsized":"2024-07-17T08:42:07.985711+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:24:53.013732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.19","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984191+0000","last_change":"2024-07-17T08:42:06.691597+0000","last_active":"2024-07-17T08:42:07.984191+0000","last_peered":"2024-07-17T08:42:07.984191+0000","last_clean":"2024-07-17T08:42:07.984191+0000","last_became_active":"2024-07-17T08:42:06.685587+0000","last_became_peered":"2024-07-17T08:42:06.685587+0000","last_unstale":"2024-07-17T08:42:07.984191+0000","last_undegraded":"2024-07-17T08:42:07.984191+0000","last_fullsized":"2024-07-17T08:42:07.984191+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:30.281884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.1e","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983132+0000","last_change":"2024-07-17T08:42:05.665763+0000","last_active":"2024-07-17T08:42:07.983132+0000","last_peered":"2024-07-17T08:42:07.983132+0000","last_clean":"2024-07-17T08:42:07.983132+0000","last_became_active":"2024-07-17T08:42:05.665453+0000","last_became_peered":"2024-07-17T08:42:05.665453+0000","last_unstale":"2024-07-17T08:42:07.983132+0000","last_undegraded":"2024-07-17T08:42:07.983132+0000","last_fullsized":"2024-07-17T08:42:07.983132+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:56:36.270152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982084+0000","last_change":"2024-07-17T08:42:06.693507+0000","last_active":"2024-07-17T08:42:07.982084+0000","last_peered":"2024-07-17T08:42:07.982084+0000","last_clean":"2024-07-17T08:42:07.982084+0000","last_became_active":"2024-07-17T08:42:06.693203+0000","last_became_peered":"2024-07-17T08:42:06.693203+0000","last_unstale":"2024-07-17T08:42:07.982084+0000","last_undegraded":"2024-07-17T08:42:07.982084+0000","last_fullsized":"2024-07-17T08:42:07.982084+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:19:46.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.1d","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981782+0000","last_change":"2024-07-17T08:42:05.666848+0000","last_active":"2024-07-17T08:42:07.981782+0000","last_peered":"2024-07-17T08:42:07.981782+0000","last_clean":"2024-07-17T08:42:07.981782+0000","last_became_active":"2024-07-17T08:42:05.666498+0000","last_became_peered":"2024-07-17T08:42:05.666498+0000","last_unstale":"2024-07-17T08:42:07.981782+0000","last_undegraded":"2024-07-17T08:42:07.981782+0000","last_fullsized":"2024-07-17T08:42:07.981782+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:45:10.283939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658412+0000","last_change":"2024-07-17T08:42:06.689322+0000","last_active":"2024-07-17T08:42:07.658412+0000","last_peered":"2024-07-17T08:42:07.658412+0000","last_clean":"2024-07-17T08:42:07.658412+0000","last_became_active":"2024-07-17T08:42:06.687983+0000","last_became_peered":"2024-07-17T08:42:06.687983+0000","last_unstale":"2024-07-17T08:42:07.658412+0000","last_undegraded":"2024-07-17T08:42:07.658412+0000","last_fullsized":"2024-07-17T08:42:07.658412+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:11:33.561117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.1c","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660708+0000","last_change":"2024-07-17T08:42:05.655590+0000","last_active":"2024-07-17T08:42:07.660708+0000","last_peered":"2024-07-17T08:42:07.660708+0000","last_clean":"2024-07-17T08:42:07.660708+0000","last_became_active":"2024-07-17T08:42:05.655247+0000","last_became_peered":"2024-07-17T08:42:05.655247+0000","last_unstale":"2024-07-17T08:42:07.660708+0000","last_undegraded":"2024-07-17T08:42:07.660708+0000","last_fullsized":"2024-07-17T08:42:07.660708+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:07:25.185151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.1c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657870+0000","last_change":"2024-07-17T08:42:06.693864+0000","last_active":"2024-07-17T08:42:07.657870+0000","last_peered":"2024-07-17T08:42:07.657870+0000","last_clean":"2024-07-17T08:42:07.657870+0000","last_became_active":"2024-07-17T08:42:06.693571+0000","last_became_peered":"2024-07-17T08:42:06.693571+0000","last_unstale":"2024-07-17T08:42:07.657870+0000","last_undegraded":"2024-07-17T08:42:07.657870+0000","last_fullsized":"2024-07-17T08:42:07.657870+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:55.220579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.1b","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658920+0000","last_change":"2024-07-17T08:42:05.659152+0000","last_active":"2024-07-17T08:42:07.658920+0000","last_peered":"2024-07-17T08:42:07.658920+0000","last_clean":"2024-07-17T08:42:07.658920+0000","last_became_active":"2024-07-17T08:42:05.658842+0000","last_became_peered":"2024-07-17T08:42:05.658842+0000","last_unstale":"2024-07-17T08:42:07.658920+0000","last_undegraded":"2024-07-17T08:42:07.658920+0000","last_fullsized":"2024-07-17T08:42:07.658920+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:06:08.477038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985220+0000","last_change":"2024-07-17T08:42:06.691295+0000","last_active":"2024-07-17T08:42:07.985220+0000","last_peered":"2024-07-17T08:42:07.985220+0000","last_clean":"2024-07-17T08:42:07.985220+0000","last_became_active":"2024-07-17T08:42:06.689645+0000","last_became_peered":"2024-07-17T08:42:06.689645+0000","last_unstale":"2024-07-17T08:42:07.985220+0000","last_undegraded":"2024-07-17T08:42:07.985220+0000","last_fullsized":"2024-07-17T08:42:07.985220+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:05:37.304601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.1a","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982364+0000","last_change":"2024-07-17T08:42:05.663162+0000","last_active":"2024-07-17T08:42:07.982364+0000","last_peered":"2024-07-17T08:42:07.982364+0000","last_clean":"2024-07-17T08:42:07.982364+0000","last_became_active":"2024-07-17T08:42:05.662384+0000","last_became_peered":"2024-07-17T08:42:05.662384+0000","last_unstale":"2024-07-17T08:42:07.982364+0000","last_undegraded":"2024-07-17T08:42:07.982364+0000","last_fullsized":"2024-07-17T08:42:07.982364+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:27:35.320892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983591+0000","last_change":"2024-07-17T08:42:06.688724+0000","last_active":"2024-07-17T08:42:07.983591+0000","last_peered":"2024-07-17T08:42:07.983591+0000","last_clean":"2024-07-17T08:42:07.983591+0000","last_became_active":"2024-07-17T08:42:06.688377+0000","last_became_peered":"2024-07-17T08:42:06.688377+0000","last_unstale":"2024-07-17T08:42:07.983591+0000","last_undegraded":"2024-07-17T08:42:07.983591+0000","last_fullsized":"2024-07-17T08:42:07.983591+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:29:19.269818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.19","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982649+0000","last_change":"2024-07-17T08:42:05.656321+0000","last_active":"2024-07-17T08:42:07.982649+0000","last_peered":"2024-07-17T08:42:07.982649+0000","last_clean":"2024-07-17T08:42:07.982649+0000","last_became_active":"2024-07-17T08:42:05.655307+0000","last_became_peered":"2024-07-17T08:42:05.655307+0000","last_unstale":"2024-07-17T08:42:07.982649+0000","last_undegraded":"2024-07-17T08:42:07.982649+0000","last_fullsized":"2024-07-17T08:42:07.982649+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:09.253237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.1f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661524+0000","last_change":"2024-07-17T08:42:06.684595+0000","last_active":"2024-07-17T08:42:07.661524+0000","last_peered":"2024-07-17T08:42:07.661524+0000","last_clean":"2024-07-17T08:42:07.661524+0000","last_became_active":"2024-07-17T08:42:06.684034+0000","last_became_peered":"2024-07-17T08:42:06.684034+0000","last_unstale":"2024-07-17T08:42:07.661524+0000","last_undegraded":"2024-07-17T08:42:07.661524+0000","last_fullsized":"2024-07-17T08:42:07.661524+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:49:04.540386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.18","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.980960+0000","last_change":"2024-07-17T08:42:05.657449+0000","last_active":"2024-07-17T08:42:07.980960+0000","last_peered":"2024-07-17T08:42:07.980960+0000","last_clean":"2024-07-17T08:42:07.980960+0000","last_became_active":"2024-07-17T08:42:05.657153+0000","last_became_peered":"2024-07-17T08:42:05.657153+0000","last_unstale":"2024-07-17T08:42:07.980960+0000","last_undegraded":"2024-07-17T08:42:07.980960+0000","last_fullsized":"2024-07-17T08:42:07.980960+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:35.223085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.10","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981649+0000","last_change":"2024-07-17T08:42:06.666120+0000","last_active":"2024-07-17T08:42:07.981649+0000","last_peered":"2024-07-17T08:42:07.981649+0000","last_clean":"2024-07-17T08:42:07.981649+0000","last_became_active":"2024-07-17T08:42:06.665720+0000","last_became_peered":"2024-07-17T08:42:06.665720+0000","last_unstale":"2024-07-17T08:42:07.981649+0000","last_undegraded":"2024-07-17T08:42:07.981649+0000","last_fullsized":"2024-07-17T08:42:07.981649+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:58:18.525206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.17","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985872+0000","last_change":"2024-07-17T08:42:05.669438+0000","last_active":"2024-07-17T08:42:07.985872+0000","last_peered":"2024-07-17T08:42:07.985872+0000","last_clean":"2024-07-17T08:42:07.985872+0000","last_became_active":"2024-07-17T08:42:05.668992+0000","last_became_peered":"2024-07-17T08:42:05.668992+0000","last_unstale":"2024-07-17T08:42:07.985872+0000","last_undegraded":"2024-07-17T08:42:07.985872+0000","last_fullsized":"2024-07-17T08:42:07.985872+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:14.146779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.11","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657513+0000","last_change":"2024-07-17T08:42:06.690575+0000","last_active":"2024-07-17T08:42:07.657513+0000","last_peered":"2024-07-17T08:42:07.657513+0000","last_clean":"2024-07-17T08:42:07.657513+0000","last_became_active":"2024-07-17T08:42:06.690197+0000","last_became_peered":"2024-07-17T08:42:06.690197+0000","last_unstale":"2024-07-17T08:42:07.657513+0000","last_undegraded":"2024-07-17T08:42:07.657513+0000","last_fullsized":"2024-07-17T08:42:07.657513+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:05.331208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.16","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658474+0000","last_change":"2024-07-17T08:42:05.649750+0000","last_active":"2024-07-17T08:42:07.658474+0000","last_peered":"2024-07-17T08:42:07.658474+0000","last_clean":"2024-07-17T08:42:07.658474+0000","last_became_active":"2024-07-17T08:42:05.649334+0000","last_became_peered":"2024-07-17T08:42:05.649334+0000","last_unstale":"2024-07-17T08:42:07.658474+0000","last_undegraded":"2024-07-17T08:42:07.658474+0000","last_fullsized":"2024-07-17T08:42:07.658474+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:24.344179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.12","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981819+0000","last_change":"2024-07-17T08:42:06.680051+0000","last_active":"2024-07-17T08:42:07.981819+0000","last_peered":"2024-07-17T08:42:07.981819+0000","last_clean":"2024-07-17T08:42:07.981819+0000","last_became_active":"2024-07-17T08:42:06.679019+0000","last_became_peered":"2024-07-17T08:42:06.679019+0000","last_unstale":"2024-07-17T08:42:07.981819+0000","last_undegraded":"2024-07-17T08:42:07.981819+0000","last_fullsized":"2024-07-17T08:42:07.981819+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:49:02.985055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.15","version":"320'3","reported_seq":17,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982129+0000","last_change":"2024-07-17T08:42:05.659058+0000","last_active":"2024-07-17T08:42:07.982129+0000","last_peered":"2024-07-17T08:42:07.982129+0000","last_clean":"2024-07-17T08:42:07.982129+0000","last_became_active":"2024-07-17T08:42:05.658788+0000","last_became_peered":"2024-07-17T08:42:05.658788+0000","last_unstale":"2024-07-17T08:42:07.982129+0000","last_undegraded":"2024-07-17T08:42:07.982129+0000","last_fullsized":"2024-07-17T08:42:07.982129+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:52.167975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.13","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982329+0000","last_change":"2024-07-17T08:42:06.676994+0000","last_active":"2024-07-17T08:42:07.982329+0000","last_peered":"2024-07-17T08:42:07.982329+0000","last_clean":"2024-07-17T08:42:07.982329+0000","last_became_active":"2024-07-17T08:42:06.676384+0000","last_became_peered":"2024-07-17T08:42:06.676384+0000","last_unstale":"2024-07-17T08:42:07.982329+0000","last_undegraded":"2024-07-17T08:42:07.982329+0000","last_fullsized":"2024-07-17T08:42:07.982329+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:27.562983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.14","version":"320'6","reported_seq":20,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658026+0000","last_change":"2024-07-17T08:42:05.660141+0000","last_active":"2024-07-17T08:42:07.658026+0000","last_peered":"2024-07-17T08:42:07.658026+0000","last_clean":"2024-07-17T08:42:07.658026+0000","last_became_active":"2024-07-17T08:42:05.659866+0000","last_became_peered":"2024-07-17T08:42:05.659866+0000","last_unstale":"2024-07-17T08:42:07.658026+0000","last_undegraded":"2024-07-17T08:42:07.658026+0000","last_fullsized":"2024-07-17T08:42:07.658026+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:22:11.563107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.14","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981995+0000","last_change":"2024-07-17T08:42:06.680810+0000","last_active":"2024-07-17T08:42:07.981995+0000","last_peered":"2024-07-17T08:42:07.981995+0000","last_clean":"2024-07-17T08:42:07.981995+0000","last_became_active":"2024-07-17T08:42:06.680423+0000","last_became_peered":"2024-07-17T08:42:06.680423+0000","last_unstale":"2024-07-17T08:42:07.981995+0000","last_undegraded":"2024-07-17T08:42:07.981995+0000","last_fullsized":"2024-07-17T08:42:07.981995+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:17.106070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.13","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660221+0000","last_change":"2024-07-17T08:42:05.662142+0000","last_active":"2024-07-17T08:42:07.660221+0000","last_peered":"2024-07-17T08:42:07.660221+0000","last_clean":"2024-07-17T08:42:07.660221+0000","last_became_active":"2024-07-17T08:42:05.660670+0000","last_became_peered":"2024-07-17T08:42:05.660670+0000","last_unstale":"2024-07-17T08:42:07.660221+0000","last_undegraded":"2024-07-17T08:42:07.660221+0000","last_fullsized":"2024-07-17T08:42:07.660221+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:31:19.930213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.15","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983597+0000","last_change":"2024-07-17T08:42:06.686334+0000","last_active":"2024-07-17T08:42:07.983597+0000","last_peered":"2024-07-17T08:42:07.983597+0000","last_clean":"2024-07-17T08:42:07.983597+0000","last_became_active":"2024-07-17T08:42:06.686056+0000","last_became_peered":"2024-07-17T08:42:06.686056+0000","last_unstale":"2024-07-17T08:42:07.983597+0000","last_undegraded":"2024-07-17T08:42:07.983597+0000","last_fullsized":"2024-07-17T08:42:07.983597+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:44.117561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.12","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660069+0000","last_change":"2024-07-17T08:42:05.661673+0000","last_active":"2024-07-17T08:42:07.660069+0000","last_peered":"2024-07-17T08:42:07.660069+0000","last_clean":"2024-07-17T08:42:07.660069+0000","last_became_active":"2024-07-17T08:42:05.661105+0000","last_became_peered":"2024-07-17T08:42:05.661105+0000","last_unstale":"2024-07-17T08:42:07.660069+0000","last_undegraded":"2024-07-17T08:42:07.660069+0000","last_fullsized":"2024-07-17T08:42:07.660069+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:11.390459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.16","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985837+0000","last_change":"2024-07-17T08:42:06.693700+0000","last_active":"2024-07-17T08:42:07.985837+0000","last_peered":"2024-07-17T08:42:07.985837+0000","last_clean":"2024-07-17T08:42:07.985837+0000","last_became_active":"2024-07-17T08:42:06.689297+0000","last_became_peered":"2024-07-17T08:42:06.689297+0000","last_unstale":"2024-07-17T08:42:07.985837+0000","last_undegraded":"2024-07-17T08:42:07.985837+0000","last_fullsized":"2024-07-17T08:42:07.985837+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:57:14.580419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.11","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981828+0000","last_change":"2024-07-17T08:42:05.662092+0000","last_active":"2024-07-17T08:42:07.981828+0000","last_peered":"2024-07-17T08:42:07.981828+0000","last_clean":"2024-07-17T08:42:07.981828+0000","last_became_active":"2024-07-17T08:42:05.661447+0000","last_became_peered":"2024-07-17T08:42:05.661447+0000","last_unstale":"2024-07-17T08:42:07.981828+0000","last_undegraded":"2024-07-17T08:42:07.981828+0000","last_fullsized":"2024-07-17T08:42:07.981828+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:39:27.300759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.17","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982808+0000","last_change":"2024-07-17T08:42:06.690592+0000","last_active":"2024-07-17T08:42:07.982808+0000","last_peered":"2024-07-17T08:42:07.982808+0000","last_clean":"2024-07-17T08:42:07.982808+0000","last_became_active":"2024-07-17T08:42:06.690294+0000","last_became_peered":"2024-07-17T08:42:06.690294+0000","last_unstale":"2024-07-17T08:42:07.982808+0000","last_undegraded":"2024-07-17T08:42:07.982808+0000","last_fullsized":"2024-07-17T08:42:07.982808+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:25.770977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.10","version":"320'8","reported_seq":22,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657203+0000","last_change":"2024-07-17T08:42:05.655757+0000","last_active":"2024-07-17T08:42:07.657203+0000","last_peered":"2024-07-17T08:42:07.657203+0000","last_clean":"2024-07-17T08:42:07.657203+0000","last_became_active":"2024-07-17T08:42:05.654293+0000","last_became_peered":"2024-07-17T08:42:05.654293+0000","last_unstale":"2024-07-17T08:42:07.657203+0000","last_undegraded":"2024-07-17T08:42:07.657203+0000","last_fullsized":"2024-07-17T08:42:07.657203+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:34.595127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.8","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.659796+0000","last_change":"2024-07-17T08:42:06.674013+0000","last_active":"2024-07-17T08:42:07.659796+0000","last_peered":"2024-07-17T08:42:07.659796+0000","last_clean":"2024-07-17T08:42:07.659796+0000","last_became_active":"2024-07-17T08:42:06.673720+0000","last_became_peered":"2024-07-17T08:42:06.673720+0000","last_unstale":"2024-07-17T08:42:07.659796+0000","last_undegraded":"2024-07-17T08:42:07.659796+0000","last_fullsized":"2024-07-17T08:42:07.659796+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:25:58.469274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.f","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661747+0000","last_change":"2024-07-17T08:42:05.670380+0000","last_active":"2024-07-17T08:42:07.661747+0000","last_peered":"2024-07-17T08:42:07.661747+0000","last_clean":"2024-07-17T08:42:07.661747+0000","last_became_active":"2024-07-17T08:42:05.669956+0000","last_became_peered":"2024-07-17T08:42:05.669956+0000","last_unstale":"2024-07-17T08:42:07.661747+0000","last_undegraded":"2024-07-17T08:42:07.661747+0000","last_fullsized":"2024-07-17T08:42:07.661747+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:25:52.606014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.9","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.657854+0000","last_change":"2024-07-17T08:42:06.692716+0000","last_active":"2024-07-17T08:42:07.657854+0000","last_peered":"2024-07-17T08:42:07.657854+0000","last_clean":"2024-07-17T08:42:07.657854+0000","last_became_active":"2024-07-17T08:42:06.692432+0000","last_became_peered":"2024-07-17T08:42:06.692432+0000","last_unstale":"2024-07-17T08:42:07.657854+0000","last_undegraded":"2024-07-17T08:42:07.657854+0000","last_fullsized":"2024-07-17T08:42:07.657854+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:54:19.762262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.e","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661104+0000","last_change":"2024-07-17T08:42:05.657820+0000","last_active":"2024-07-17T08:42:07.661104+0000","last_peered":"2024-07-17T08:42:07.661104+0000","last_clean":"2024-07-17T08:42:07.661104+0000","last_became_active":"2024-07-17T08:42:05.657530+0000","last_became_peered":"2024-07-17T08:42:05.657530+0000","last_unstale":"2024-07-17T08:42:07.661104+0000","last_undegraded":"2024-07-17T08:42:07.661104+0000","last_fullsized":"2024-07-17T08:42:07.661104+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:47:44.234698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.a","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983628+0000","last_change":"2024-07-17T08:42:06.685191+0000","last_active":"2024-07-17T08:42:07.983628+0000","last_peered":"2024-07-17T08:42:07.983628+0000","last_clean":"2024-07-17T08:42:07.983628+0000","last_became_active":"2024-07-17T08:42:06.684878+0000","last_became_peered":"2024-07-17T08:42:06.684878+0000","last_unstale":"2024-07-17T08:42:07.983628+0000","last_undegraded":"2024-07-17T08:42:07.983628+0000","last_fullsized":"2024-07-17T08:42:07.983628+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:44.386511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"43.d","version":"320'1","reported_seq":15,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985521+0000","last_change":"2024-07-17T08:42:05.670466+0000","last_active":"2024-07-17T08:42:07.985521+0000","last_peered":"2024-07-17T08:42:07.985521+0000","last_clean":"2024-07-17T08:42:07.985521+0000","last_became_active":"2024-07-17T08:42:05.670171+0000","last_became_peered":"2024-07-17T08:42:05.670171+0000","last_unstale":"2024-07-17T08:42:07.985521+0000","last_undegraded":"2024-07-17T08:42:07.985521+0000","last_fullsized":"2024-07-17T08:42:07.985521+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:11:31.486090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.b","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660724+0000","last_change":"2024-07-17T08:42:06.684917+0000","last_active":"2024-07-17T08:42:07.660724+0000","last_peered":"2024-07-17T08:42:07.660724+0000","last_clean":"2024-07-17T08:42:07.660724+0000","last_became_active":"2024-07-17T08:42:06.684558+0000","last_became_peered":"2024-07-17T08:42:06.684558+0000","last_unstale":"2024-07-17T08:42:07.660724+0000","last_undegraded":"2024-07-17T08:42:07.660724+0000","last_fullsized":"2024-07-17T08:42:07.660724+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:41:07.850142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.c","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.985360+0000","last_change":"2024-07-17T08:42:05.667966+0000","last_active":"2024-07-17T08:42:07.985360+0000","last_peered":"2024-07-17T08:42:07.985360+0000","last_clean":"2024-07-17T08:42:07.985360+0000","last_became_active":"2024-07-17T08:42:05.667650+0000","last_became_peered":"2024-07-17T08:42:05.667650+0000","last_unstale":"2024-07-17T08:42:07.985360+0000","last_undegraded":"2024-07-17T08:42:07.985360+0000","last_fullsized":"2024-07-17T08:42:07.985360+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:54.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.c","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981888+0000","last_change":"2024-07-17T08:42:06.681536+0000","last_active":"2024-07-17T08:42:07.981888+0000","last_peered":"2024-07-17T08:42:07.981888+0000","last_clean":"2024-07-17T08:42:07.981888+0000","last_became_active":"2024-07-17T08:42:06.681183+0000","last_became_peered":"2024-07-17T08:42:06.681183+0000","last_unstale":"2024-07-17T08:42:07.981888+0000","last_undegraded":"2024-07-17T08:42:07.981888+0000","last_fullsized":"2024-07-17T08:42:07.981888+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:52:20.993804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.b","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660585+0000","last_change":"2024-07-17T08:42:05.653735+0000","last_active":"2024-07-17T08:42:07.660585+0000","last_peered":"2024-07-17T08:42:07.660585+0000","last_clean":"2024-07-17T08:42:07.660585+0000","last_became_active":"2024-07-17T08:42:05.653409+0000","last_became_peered":"2024-07-17T08:42:05.653409+0000","last_unstale":"2024-07-17T08:42:07.660585+0000","last_undegraded":"2024-07-17T08:42:07.660585+0000","last_fullsized":"2024-07-17T08:42:07.660585+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:44:24.500271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661583+0000","last_change":"2024-07-17T08:42:06.677145+0000","last_active":"2024-07-17T08:42:07.661583+0000","last_peered":"2024-07-17T08:42:07.661583+0000","last_clean":"2024-07-17T08:42:07.661583+0000","last_became_active":"2024-07-17T08:42:06.676900+0000","last_became_peered":"2024-07-17T08:42:06.676900+0000","last_unstale":"2024-07-17T08:42:07.661583+0000","last_undegraded":"2024-07-17T08:42:07.661583+0000","last_fullsized":"2024-07-17T08:42:07.661583+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:33.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"43.a","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.984124+0000","last_change":"2024-07-17T08:42:05.667393+0000","last_active":"2024-07-17T08:42:07.984124+0000","last_peered":"2024-07-17T08:42:07.984124+0000","last_clean":"2024-07-17T08:42:07.984124+0000","last_became_active":"2024-07-17T08:42:05.666967+0000","last_became_peered":"2024-07-17T08:42:05.666967+0000","last_unstale":"2024-07-17T08:42:07.984124+0000","last_undegraded":"2024-07-17T08:42:07.984124+0000","last_fullsized":"2024-07-17T08:42:07.984124+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:24.942490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.e","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661436+0000","last_change":"2024-07-17T08:42:06.688484+0000","last_active":"2024-07-17T08:42:07.661436+0000","last_peered":"2024-07-17T08:42:07.661436+0000","last_clean":"2024-07-17T08:42:07.661436+0000","last_became_active":"2024-07-17T08:42:06.687865+0000","last_became_peered":"2024-07-17T08:42:06.687865+0000","last_unstale":"2024-07-17T08:42:07.661436+0000","last_undegraded":"2024-07-17T08:42:07.661436+0000","last_fullsized":"2024-07-17T08:42:07.661436+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:27:58.488319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.9","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982067+0000","last_change":"2024-07-17T08:42:05.662923+0000","last_active":"2024-07-17T08:42:07.982067+0000","last_peered":"2024-07-17T08:42:07.982067+0000","last_clean":"2024-07-17T08:42:07.982067+0000","last_became_active":"2024-07-17T08:42:05.661824+0000","last_became_peered":"2024-07-17T08:42:05.661824+0000","last_unstale":"2024-07-17T08:42:07.982067+0000","last_undegraded":"2024-07-17T08:42:07.982067+0000","last_fullsized":"2024-07-17T08:42:07.982067+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:18:12.515784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.f","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983414+0000","last_change":"2024-07-17T08:42:06.680072+0000","last_active":"2024-07-17T08:42:07.983414+0000","last_peered":"2024-07-17T08:42:07.983414+0000","last_clean":"2024-07-17T08:42:07.983414+0000","last_became_active":"2024-07-17T08:42:06.679674+0000","last_became_peered":"2024-07-17T08:42:06.679674+0000","last_unstale":"2024-07-17T08:42:07.983414+0000","last_undegraded":"2024-07-17T08:42:07.983414+0000","last_fullsized":"2024-07-17T08:42:07.983414+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:52:20.678205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.8","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981003+0000","last_change":"2024-07-17T08:42:05.669408+0000","last_active":"2024-07-17T08:42:07.981003+0000","last_peered":"2024-07-17T08:42:07.981003+0000","last_clean":"2024-07-17T08:42:07.981003+0000","last_became_active":"2024-07-17T08:42:05.668987+0000","last_became_peered":"2024-07-17T08:42:05.668987+0000","last_unstale":"2024-07-17T08:42:07.981003+0000","last_undegraded":"2024-07-17T08:42:07.981003+0000","last_fullsized":"2024-07-17T08:42:07.981003+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:11:36.005753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.0","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981441+0000","last_change":"2024-07-17T08:42:06.678892+0000","last_active":"2024-07-17T08:42:07.981441+0000","last_peered":"2024-07-17T08:42:07.981441+0000","last_clean":"2024-07-17T08:42:07.981441+0000","last_became_active":"2024-07-17T08:42:06.678545+0000","last_became_peered":"2024-07-17T08:42:06.678545+0000","last_unstale":"2024-07-17T08:42:07.981441+0000","last_undegraded":"2024-07-17T08:42:07.981441+0000","last_fullsized":"2024-07-17T08:42:07.981441+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:12:49.491439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.7","version":"320'2","reported_seq":16,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982465+0000","last_change":"2024-07-17T08:42:05.666706+0000","last_active":"2024-07-17T08:42:07.982465+0000","last_peered":"2024-07-17T08:42:07.982465+0000","last_clean":"2024-07-17T08:42:07.982465+0000","last_became_active":"2024-07-17T08:42:05.665981+0000","last_became_peered":"2024-07-17T08:42:05.665981+0000","last_unstale":"2024-07-17T08:42:07.982465+0000","last_undegraded":"2024-07-17T08:42:07.982465+0000","last_fullsized":"2024-07-17T08:42:07.982465+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:51:50.066205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.1","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982373+0000","last_change":"2024-07-17T08:42:06.683511+0000","last_active":"2024-07-17T08:42:07.982373+0000","last_peered":"2024-07-17T08:42:07.982373+0000","last_clean":"2024-07-17T08:42:07.982373+0000","last_became_active":"2024-07-17T08:42:06.683224+0000","last_became_peered":"2024-07-17T08:42:06.683224+0000","last_unstale":"2024-07-17T08:42:07.982373+0000","last_undegraded":"2024-07-17T08:42:07.982373+0000","last_fullsized":"2024-07-17T08:42:07.982373+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:24:35.662352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"43.6","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983874+0000","last_change":"2024-07-17T08:42:05.662691+0000","last_active":"2024-07-17T08:42:07.983874+0000","last_peered":"2024-07-17T08:42:07.983874+0000","last_clean":"2024-07-17T08:42:07.983874+0000","last_became_active":"2024-07-17T08:42:05.662376+0000","last_became_peered":"2024-07-17T08:42:05.662376+0000","last_unstale":"2024-07-17T08:42:07.983874+0000","last_undegraded":"2024-07-17T08:42:07.983874+0000","last_fullsized":"2024-07-17T08:42:07.983874+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:18:21.463356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.2","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660155+0000","last_change":"2024-07-17T08:42:06.685897+0000","last_active":"2024-07-17T08:42:07.660155+0000","last_peered":"2024-07-17T08:42:07.660155+0000","last_clean":"2024-07-17T08:42:07.660155+0000","last_became_active":"2024-07-17T08:42:06.685583+0000","last_became_peered":"2024-07-17T08:42:06.685583+0000","last_unstale":"2024-07-17T08:42:07.660155+0000","last_undegraded":"2024-07-17T08:42:07.660155+0000","last_fullsized":"2024-07-17T08:42:07.660155+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:54:28.439424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"43.5","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.661267+0000","last_change":"2024-07-17T08:42:05.669462+0000","last_active":"2024-07-17T08:42:07.661267+0000","last_peered":"2024-07-17T08:42:07.661267+0000","last_clean":"2024-07-17T08:42:07.661267+0000","last_became_active":"2024-07-17T08:42:05.668937+0000","last_became_peered":"2024-07-17T08:42:05.668937+0000","last_unstale":"2024-07-17T08:42:07.661267+0000","last_undegraded":"2024-07-17T08:42:07.661267+0000","last_fullsized":"2024-07-17T08:42:07.661267+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:46:47.434897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.3","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982593+0000","last_change":"2024-07-17T08:42:06.675237+0000","last_active":"2024-07-17T08:42:07.982593+0000","last_peered":"2024-07-17T08:42:07.982593+0000","last_clean":"2024-07-17T08:42:07.982593+0000","last_became_active":"2024-07-17T08:42:06.674960+0000","last_became_peered":"2024-07-17T08:42:06.674960+0000","last_unstale":"2024-07-17T08:42:07.982593+0000","last_undegraded":"2024-07-17T08:42:07.982593+0000","last_fullsized":"2024-07-17T08:42:07.982593+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:44.540505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"43.4","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658735+0000","last_change":"2024-07-17T08:42:05.665881+0000","last_active":"2024-07-17T08:42:07.658735+0000","last_peered":"2024-07-17T08:42:07.658735+0000","last_clean":"2024-07-17T08:42:07.658735+0000","last_became_active":"2024-07-17T08:42:05.665565+0000","last_became_peered":"2024-07-17T08:42:05.665565+0000","last_unstale":"2024-07-17T08:42:07.658735+0000","last_undegraded":"2024-07-17T08:42:07.658735+0000","last_fullsized":"2024-07-17T08:42:07.658735+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:46:30.101051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.4","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982717+0000","last_change":"2024-07-17T08:42:06.687005+0000","last_active":"2024-07-17T08:42:07.982717+0000","last_peered":"2024-07-17T08:42:07.982717+0000","last_clean":"2024-07-17T08:42:07.982717+0000","last_became_active":"2024-07-17T08:42:06.682854+0000","last_became_peered":"2024-07-17T08:42:06.682854+0000","last_unstale":"2024-07-17T08:42:07.982717+0000","last_undegraded":"2024-07-17T08:42:07.982717+0000","last_fullsized":"2024-07-17T08:42:07.982717+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:20.518080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"43.3","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982849+0000","last_change":"2024-07-17T08:42:05.648439+0000","last_active":"2024-07-17T08:42:07.982849+0000","last_peered":"2024-07-17T08:42:07.982849+0000","last_clean":"2024-07-17T08:42:07.982849+0000","last_became_active":"2024-07-17T08:42:05.648059+0000","last_became_peered":"2024-07-17T08:42:05.648059+0000","last_unstale":"2024-07-17T08:42:07.982849+0000","last_undegraded":"2024-07-17T08:42:07.982849+0000","last_fullsized":"2024-07-17T08:42:07.982849+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:33.427557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.5","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983470+0000","last_change":"2024-07-17T08:42:06.688632+0000","last_active":"2024-07-17T08:42:07.983470+0000","last_peered":"2024-07-17T08:42:07.983470+0000","last_clean":"2024-07-17T08:42:07.983470+0000","last_became_active":"2024-07-17T08:42:06.688348+0000","last_became_peered":"2024-07-17T08:42:06.688348+0000","last_unstale":"2024-07-17T08:42:07.983470+0000","last_undegraded":"2024-07-17T08:42:07.983470+0000","last_fullsized":"2024-07-17T08:42:07.983470+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:25.203321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.2","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981581+0000","last_change":"2024-07-17T08:42:05.662065+0000","last_active":"2024-07-17T08:42:07.981581+0000","last_peered":"2024-07-17T08:42:07.981581+0000","last_clean":"2024-07-17T08:42:07.981581+0000","last_became_active":"2024-07-17T08:42:05.661496+0000","last_became_peered":"2024-07-17T08:42:05.661496+0000","last_unstale":"2024-07-17T08:42:07.981581+0000","last_undegraded":"2024-07-17T08:42:07.981581+0000","last_fullsized":"2024-07-17T08:42:07.981581+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:21:29.987993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.6","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.983650+0000","last_change":"2024-07-17T08:42:06.694032+0000","last_active":"2024-07-17T08:42:07.983650+0000","last_peered":"2024-07-17T08:42:07.983650+0000","last_clean":"2024-07-17T08:42:07.983650+0000","last_became_active":"2024-07-17T08:42:06.693748+0000","last_became_peered":"2024-07-17T08:42:06.693748+0000","last_unstale":"2024-07-17T08:42:07.983650+0000","last_undegraded":"2024-07-17T08:42:07.983650+0000","last_fullsized":"2024-07-17T08:42:07.983650+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:39.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"43.1","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.982300+0000","last_change":"2024-07-17T08:42:05.660712+0000","last_active":"2024-07-17T08:42:07.982300+0000","last_peered":"2024-07-17T08:42:07.982300+0000","last_clean":"2024-07-17T08:42:07.982300+0000","last_became_active":"2024-07-17T08:42:05.660426+0000","last_became_peered":"2024-07-17T08:42:05.660426+0000","last_unstale":"2024-07-17T08:42:07.982300+0000","last_undegraded":"2024-07-17T08:42:07.982300+0000","last_fullsized":"2024-07-17T08:42:07.982300+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:08:10.408996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.7","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.658626+0000","last_change":"2024-07-17T08:42:06.697677+0000","last_active":"2024-07-17T08:42:07.658626+0000","last_peered":"2024-07-17T08:42:07.658626+0000","last_clean":"2024-07-17T08:42:07.658626+0000","last_became_active":"2024-07-17T08:42:06.696346+0000","last_became_peered":"2024-07-17T08:42:06.696346+0000","last_unstale":"2024-07-17T08:42:07.658626+0000","last_undegraded":"2024-07-17T08:42:07.658626+0000","last_fullsized":"2024-07-17T08:42:07.658626+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:14.310708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"43.0","version":"0'0","reported_seq":14,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660069+0000","last_change":"2024-07-17T08:42:05.653041+0000","last_active":"2024-07-17T08:42:07.660069+0000","last_peered":"2024-07-17T08:42:07.660069+0000","last_clean":"2024-07-17T08:42:07.660069+0000","last_became_active":"2024-07-17T08:42:05.652757+0000","last_became_peered":"2024-07-17T08:42:05.652757+0000","last_unstale":"2024-07-17T08:42:07.660069+0000","last_undegraded":"2024-07-17T08:42:07.660069+0000","last_fullsized":"2024-07-17T08:42:07.660069+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":318,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:04.613780+0000","last_clean_scrub_stamp":"2024-07-17T08:42:04.613780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:38:04.044951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.2d","version":"0'0","reported_seq":12,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.660861+0000","last_change":"2024-07-17T08:42:06.697214+0000","last_active":"2024-07-17T08:42:07.660861+0000","last_peered":"2024-07-17T08:42:07.660861+0000","last_clean":"2024-07-17T08:42:07.660861+0000","last_became_active":"2024-07-17T08:42:06.696910+0000","last_became_peered":"2024-07-17T08:42:06.696910+0000","last_unstale":"2024-07-17T08:42:07.660861+0000","last_undegraded":"2024-07-17T08:42:07.660861+0000","last_fullsized":"2024-07-17T08:42:07.660861+0000","mapping_epoch":319,"log_start":"0'0","ondisk_log_start":"0'0","created":319,"last_epoch_clean":320,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:05.623001+0000","last_clean_scrub_stamp":"2024-07-17T08:42:05.623001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:49:11.066618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"311'4624","reported_seq":5902,"reported_epoch":321,"state":"active+clean","last_fresh":"2024-07-17T08:42:07.981140+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:42:07.981140+0000","last_peered":"2024-07-17T08:42:07.981140+0000","last_clean":"2024-07-17T08:42:07.981140+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:42:07.981140+0000","last_undegraded":"2024-07-17T08:42:07.981140+0000","last_fullsized":"2024-07-17T08:42:07.981140+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":493,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":44,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":43,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2089,"num_read_kb":5910,"num_write":493,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4624,"ondisk_log_size":4624,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543416,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80168,"kb_used_data":4564,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291672,"statfs":{"total":96636764160,"available":96554672128,"internally_reserved":0,"allocated":4673536,"data_stored":2830982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510711,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53220,"kb_used_data":3984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318620,"statfs":{"total":96636764160,"available":96582266880,"internally_reserved":0,"allocated":4079616,"data_stored":2247236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71284,"kb_used_data":4000,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300556,"statfs":{"total":96636764160,"available":96563769344,"internally_reserved":0,"allocated":4096000,"data_stored":2249430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543417,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53216,"kb_used_data":3988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318624,"statfs":{"total":96636764160,"available":96582270976,"internally_reserved":0,"allocated":4083712,"data_stored":2247326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57396,"kb_used_data":4000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314444,"statfs":{"total":96636764160,"available":96577990656,"internally_reserved":0,"allocated":4096000,"data_stored":2248938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543415,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57364,"kb_used_data":3976,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314476,"statfs":{"total":96636764160,"available":96578023424,"internally_reserved":0,"allocated":4071424,"data_stored":2239974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510712,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53228,"kb_used_data":3996,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318612,"statfs":{"total":96636764160,"available":96582258688,"internally_reserved":0,"allocated":4091904,"data_stored":2247362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510711,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53796,"kb_used_data":4564,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318044,"statfs":{"total":96636764160,"available":96581677056,"internally_reserved":0,"allocated":4673536,"data_stored":2837604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":43,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:42:11.680 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:42:11.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:42:11.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:42:11.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 -- 172.21.15.44:0/1153703171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c1503c0 msgr2=0x7fd35c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 --2- 172.21.15.44:0/1153703171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c1503c0 0x7fd35c15a450 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0cffb0 tx=0x7fd35002a560 comp rx=0 tx=0).stop 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 -- 172.21.15.44:0/1153703171 shutdown_connections 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 --2- 172.21.15.44:0/1153703171 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 --2- 172.21.15.44:0/1153703171 >> v2:172.21.15.175:3300/0 conn(0x7fd35c0ae340 0x7fd35c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 --2- 172.21.15.44:0/1153703171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c1503c0 0x7fd35c15a450 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 -- 172.21.15.44:0/1153703171 >> 172.21.15.44:0/1153703171 conn(0x7fd35c0cf0d0 msgr2=0x7fd35c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 -- 172.21.15.44:0/1153703171 shutdown_connections 2024-07-17T08:42:11.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.917+0000 7fd362a00640 1 -- 172.21.15.44:0/1153703171 wait complete. 2024-07-17T08:42:11.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.919+0000 7fd362a00640 1 Processor -- start 2024-07-17T08:42:11.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.920+0000 7fd362a00640 1 -- start start 2024-07-17T08:42:11.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c1f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35c1503c0 0x7fd35c072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c06f650 con 0x7fd35c0ae340 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c06f4d0 con 0x7fd35c1503c0 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd362a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c06d960 con 0x7fd35c0b33f0 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd361600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd361600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58336/0 (socket says 172.21.15.44:58336) 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.921+0000 7fd361600640 1 -- 172.21.15.44:0/945834725 learned_addr learned my addr 172.21.15.44:0/945834725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd360c00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c1f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 -- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 msgr2=0x7fd35c1f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c1f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 -- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35c1503c0 msgr2=0x7fd35c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:11.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35c1503c0 0x7fd35c072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:11.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 -- 172.21.15.44:0/945834725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c0cd300 con 0x7fd35c0ae340 2024-07-17T08:42:11.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd360c00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c1f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:11.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd361600640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fd350034040 tx=0x7fd3500039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:11.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.922+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35003e030 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.923+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3500047a0 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.923+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350035040 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.923+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c0cd5e0 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.923+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c0cdba0 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.924+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c0aec80 con 0x7fd35c0ae340 2024-07-17T08:42:11.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.925+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fd350041030 con 0x7fd35c0ae340 2024-07-17T08:42:11.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.927+0000 7fd359600640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 0x7fd33c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:11.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.927+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fd3500b90c0 con 0x7fd35c0ae340 2024-07-17T08:42:11.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.928+0000 7fd360c00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 0x7fd33c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:11.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.929+0000 7fd360c00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 0x7fd33c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd348005940 tx=0x7fd348005c90 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:11.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:11.931+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd350081ee0 con 0x7fd35c0ae340 2024-07-17T08:42:12.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.220+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd35c0cffe0 con 0x7fd35c0ae340 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.226+0000 7fd359600640 1 -- 172.21.15.44:0/945834725 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v79) ==== 190+0+147 (secure 0 0 0) 0x7fd3500817e0 con 0x7fd35c0ae340 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:42:12.227 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 msgr2=0x7fd33c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 0x7fd33c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd348005940 tx=0x7fd348005c90 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 msgr2=0x7fd35c0ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fd350034040 tx=0x7fd3500039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 shutdown_connections 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fd33c076cd0 0x7fd33c079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd35c1503c0 0x7fd35c072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd35c0b33f0 0x7fd35c1f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 --2- 172.21.15.44:0/945834725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd35c0ae340 0x7fd35c0ccc70 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 >> 172.21.15.44:0/945834725 conn(0x7fd35c0cf0d0 msgr2=0x7fd35c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 shutdown_connections 2024-07-17T08:42:12.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.229+0000 7fd362a00640 1 -- 172.21.15.44:0/945834725 wait complete. 2024-07-17T08:42:12.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 -- 172.21.15.44:0/3066992700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980cf540 msgr2=0x7fc5980b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 --2- 172.21.15.44:0/3066992700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980cf540 0x7fc5980b3870 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fc59813e5e0 tx=0x7fc58802a4d0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 -- 172.21.15.44:0/3066992700 shutdown_connections 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 --2- 172.21.15.44:0/3066992700 >> v2:172.21.15.175:3300/0 conn(0x7fc5980b3db0 0x7fc5980ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 --2- 172.21.15.44:0/3066992700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980cf540 0x7fc5980b3870 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 -- 172.21.15.44:0/3066992700 >> 172.21.15.44:0/3066992700 conn(0x7fc5980b0e80 msgr2=0x7fc5980b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 -- 172.21.15.44:0/3066992700 shutdown_connections 2024-07-17T08:42:12.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.451+0000 7fc59ce00640 1 -- 172.21.15.44:0/3066992700 wait complete. 2024-07-17T08:42:12.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.453+0000 7fc59ce00640 1 Processor -- start 2024-07-17T08:42:12.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.453+0000 7fc59ce00640 1 -- start start 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5980cf540 0x7fc598157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5980cf920 0x7fc5980cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980ae8c0 con 0x7fc5980b3db0 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980ae740 con 0x7fc5980cf920 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc59ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980aea40 con 0x7fc5980cf540 2024-07-17T08:42:12.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc597400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:12.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc597400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58348/0 (socket says 172.21.15.44:58348) 2024-07-17T08:42:12.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.454+0000 7fc597400640 1 -- 172.21.15.44:0/2118490307 learned_addr learned my addr 172.21.15.44:0/2118490307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 -- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5980cf540 msgr2=0x7fc598157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5980cf540 0x7fc598157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 -- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5980cf920 msgr2=0x7fc5980cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5980cf920 0x7fc5980cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 -- 172.21.15.44:0/2118490307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5981f5180 con 0x7fc5980b3db0 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc597400640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fc588009850 tx=0x7fc588003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58803f070 con 0x7fc5980b3db0 2024-07-17T08:42:12.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5880047a0 con 0x7fc5980b3db0 2024-07-17T08:42:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.455+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588035030 con 0x7fc5980b3db0 2024-07-17T08:42:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.456+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59806e5b0 con 0x7fc5980b3db0 2024-07-17T08:42:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.456+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc598072820 con 0x7fc5980b3db0 2024-07-17T08:42:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.457+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5981412e0 con 0x7fc5980b3db0 2024-07-17T08:42:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.458+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7fc588042070 con 0x7fc5980b3db0 2024-07-17T08:42:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.459+0000 7fc58f400640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 0x7fc5740791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.459+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fc5880b9730 con 0x7fc5980b3db0 2024-07-17T08:42:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.460+0000 7fc58fe00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 0x7fc5740791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:12.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.461+0000 7fc58fe00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 0x7fc5740791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc580005fd0 tx=0x7fc580005850 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:12.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.464+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc5981f5180 con 0x7fc5980b3db0 2024-07-17T08:42:12.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.713+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5980ae0c0 con 0x7fc5980b3db0 2024-07-17T08:42:12.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.714+0000 7fc58f400640 1 -- 172.21.15.44:0/2118490307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7fc5880825e0 con 0x7fc5980b3db0 2024-07-17T08:42:12.715 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:12.715 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:12.716 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 msgr2=0x7fc5740791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 0x7fc5740791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc580005fd0 tx=0x7fc580005850 comp rx=0 tx=0).stop 2024-07-17T08:42:12.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 msgr2=0x7fc5981f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fc588009850 tx=0x7fc588003910 comp rx=0 tx=0).stop 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 shutdown_connections 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fc574076d00 0x7fc5740791c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5980cf920 0x7fc5980cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5980cf540 0x7fc598157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 --2- 172.21.15.44:0/2118490307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5980b3db0 0x7fc5981f4c40 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.717+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 >> 172.21.15.44:0/2118490307 conn(0x7fc5980b0e80 msgr2=0x7fc5981592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.718+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 shutdown_connections 2024-07-17T08:42:12.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:12.718+0000 7fc59ce00640 1 -- 172.21.15.44:0/2118490307 wait complete. 2024-07-17T08:42:13.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 181, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:42:06.120454+0000', 'modified': '2024-07-17T08:42:09.223219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10432': {'gid': 10432, 'name': 'd', 'rank': 0, 'incarnation': 177, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6801/1754084600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1754084600}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 1754084600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [44], 'metadata_pool': 43, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10432, 'qdb_cluster': [10432]} 2024-07-17T08:42:13.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:42:13.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-17T08:42:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 -- 172.21.15.44:0/2089592470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6670152220 msgr2=0x7f667006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 --2- 172.21.15.44:0/2089592470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6670152220 0x7f667006d620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6664005fd0 tx=0x7f666402a540 comp rx=0 tx=0).stop 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 -- 172.21.15.44:0/2089592470 shutdown_connections 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 --2- 172.21.15.44:0/2089592470 >> v2:172.21.15.175:3300/0 conn(0x7f667006ed30 0x7f667015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 --2- 172.21.15.44:0/2089592470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6670152220 0x7f667006d620 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.363+0000 7f6678200640 1 -- 172.21.15.44:0/2089592470 >> 172.21.15.44:0/2089592470 conn(0x7f66700b0eb0 msgr2=0x7f66700b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.364+0000 7f6678200640 1 -- 172.21.15.44:0/2089592470 shutdown_connections 2024-07-17T08:42:13.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.364+0000 7f6678200640 1 -- 172.21.15.44:0/2089592470 wait complete. 2024-07-17T08:42:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.366+0000 7f6678200640 1 Processor -- start 2024-07-17T08:42:13.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.366+0000 7f6678200640 1 -- start start 2024-07-17T08:42:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 0x7f66701f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66700cb3a0 0x7f6670150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66700cbbf0 con 0x7f6670152220 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66700cba70 con 0x7f66700cb3a0 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.367+0000 7f6678200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f667015f570 con 0x7f667006ed30 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58358/0 (socket says 172.21.15.44:58358) 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 -- 172.21.15.44:0/698756716 learned_addr learned my addr 172.21.15.44:0/698756716 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6675800640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 0x7f66701f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6674e00640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66700cb3a0 0x7f6670150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 -- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 msgr2=0x7f66701f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 0x7f66701f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 -- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66700cb3a0 msgr2=0x7f6670150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66700cb3a0 0x7f6670150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6676200640 1 -- 172.21.15.44:0/698756716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6670152af0 con 0x7f6670152220 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.368+0000 7f6675800640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 0x7f66701f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f6676200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f666800c7d0 tx=0x7f666800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f666d600640 1 -- 172.21.15.44:0/698756716 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668003b70 con 0x7f6670152220 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f666d600640 1 -- 172.21.15.44:0/698756716 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6668003d10 con 0x7f6670152220 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f666d600640 1 -- 172.21.15.44:0/698756716 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800aab0 con 0x7f6670152220 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6670152dd0 con 0x7f6670152220 2024-07-17T08:42:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.369+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6670153390 con 0x7f6670152220 2024-07-17T08:42:13.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.371+0000 7f666d600640 1 -- 172.21.15.44:0/698756716 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f666800ac50 con 0x7f6670152220 2024-07-17T08:42:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.372+0000 7f666d600640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 0x7f6654079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.373+0000 7f666d600640 1 -- 172.21.15.44:0/698756716 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f6668098830 con 0x7f6670152220 2024-07-17T08:42:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.373+0000 7f667b028740 1 Processor -- start 2024-07-17T08:42:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.373+0000 7f667b028740 1 -- start start 2024-07-17T08:42:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.373+0000 7f6675800640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 0x7f6654079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e8662cdf00 0x55e8662efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e86623ed60 con 0x55e8663129e0 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e86622e260 con 0x55e8662efae0 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e8662322a0 con 0x55e8662cdf00 2024-07-17T08:42:13.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f6676200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f6676200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58374/0 (socket says 172.21.15.44:58374) 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f6676200640 1 -- 172.21.15.44:0/1108875813 learned_addr learned my addr 172.21.15.44:0/1108875813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.374+0000 7f6675800640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 0x7f6654079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6664033000 tx=0x7f6664004730 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 -- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e8662cdf00 msgr2=0x55e8662efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e8662cdf00 0x55e8662efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 -- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 msgr2=0x55e8662c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 -- 172.21.15.44:0/1108875813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e865f92850 con 0x55e8663129e0 2024-07-17T08:42:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f6676200640 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f666800a430 tx=0x7f66680607d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99a0544d73b550d server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f665e000640 1 -- 172.21.15.44:0/1108875813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668061060 con 0x55e8663129e0 2024-07-17T08:42:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f665e000640 1 -- 172.21.15.44:0/1108875813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6668065660 con 0x55e8663129e0 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.375+0000 7f665e000640 1 -- 172.21.15.44:0/1108875813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668060520 con 0x55e8663129e0 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 -- 172.21.15.44:0/1108875813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 msgr2=0x55e866312e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f666800a430 tx=0x7f66680607d0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 -- 172.21.15.44:0/1108875813 shutdown_connections 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e866312e50 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e8662cdf00 0x55e8662efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 --2- 172.21.15.44:0/1108875813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 -- 172.21.15.44:0/1108875813 >> 172.21.15.44:0/1108875813 conn(0x55e8662c9290 msgr2=0x55e8662cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 -- 172.21.15.44:0/1108875813 shutdown_connections 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.376+0000 7f667b028740 1 -- 172.21.15.44:0/1108875813 wait complete. 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.377+0000 7f667b028740 1 -- start start 2024-07-17T08:42:13.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.377+0000 7f667b028740 1 Processor -- start 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e865fa1620 con 0x55e8663129e0 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8661d5f90 con 0x55e8662efae0 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e8662c81c0 con 0x55e866328fa0 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f667b028740 10 client.0 _ll_register_callbacks cb 0x55e8661fbf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f6674e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f6674e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58380/0 (socket says 172.21.15.44:58380) 2024-07-17T08:42:13.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.378+0000 7f6674e00640 1 -- 172.21.15.44:0/811039943 learned_addr learned my addr 172.21.15.44:0/811039943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6675800640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 -- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 msgr2=0x55e8662c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 -- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 msgr2=0x55e8662e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 -- 172.21.15.44:0/811039943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8660096b0 con 0x55e8663129e0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6674e00640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f6660011490 tx=0x7f6660011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addc70fc8f8c9e04 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6652a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660004460 con 0x55e8663129e0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f66600035f0 con 0x55e8663129e0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.379+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6668065660 con 0x55e8663129e0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7f66600037f0 con 0x55e8663129e0 2024-07-17T08:42:13.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 10 client.13194 fetch_fsmap learned FSMap version 181 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) -- 0x55e866329d20 con 0x55e8663129e0 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6668065660 con 0x55e8663129e0 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 181) ==== 3502+0+0 (secure 0 0 0) 0x7f6660003a80 con 0x55e8663129e0 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 10 client.13194 fetch_fsmap finished waiting for FSMap version 181 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 10 client.13194 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 20 client.13194 populate_metadata read hostname 'smithi044' 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f8f70 0x55e8662f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 4 client.13194 mds_command: new command op to 10432 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:42:13.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.380+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 --> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e866240cb0 con 0x55e8662f8f70 2024-07-17T08:42:13.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.381+0000 7f6675800640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f8f70 0x55e8662f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.382+0000 7f6675800640 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f8f70 0x55e8662f8220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a80dc20b9e5f862 server_cookie=ca69fa4b3dba1acb in_seq=0 out_seq=0 2024-07-17T08:42:13.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.382+0000 7f6652a00640 10 client.13194 ms_handle_connect on v2:172.21.15.175:6800/1754084600 2024-07-17T08:42:13.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f6652a00640 1 -- 172.21.15.44:0/811039943 <== mds.0 v2:172.21.15.175:6800/1754084600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e866240cb0 con 0x55e8662f8f70 2024-07-17T08:42:13.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f6652a00640 10 client.13194 handle_command_reply: tid=0 2024-07-17T08:42:13.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 client.13194 shutdown 2024-07-17T08:42:13.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 msgr2=0x55e8662e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f6660011490 tx=0x7f6660011950 comp rx=0 tx=0).stop 2024-07-17T08:42:13.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 shutdown_connections 2024-07-17T08:42:13.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e84b0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f8f70 0x55e8662f8220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 --2- 172.21.15.44:0/811039943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 >> 172.21.15.44:0/811039943 conn(0x55e8662cdf00 msgr2=0x55e866314d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f6652a00640 0 client.13194 ms_handle_reset on v2:172.21.15.175:6800/1754084600 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 shutdown_connections 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 1 -- 172.21.15.44:0/811039943 wait complete. 2024-07-17T08:42:13.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.385+0000 7f667b028740 20 client.13194 trim_cache size 0 max 16384 2024-07-17T08:42:13.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.406+0000 7f667b028740 1 Processor -- start 2024-07-17T08:42:13.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.406+0000 7f667b028740 1 -- start start 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66701fc3a0 con 0x55e8663129e0 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66700ad820 con 0x55e8662efae0 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66700ade70 con 0x55e866328fa0 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f6674e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f6674e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58392/0 (socket says 172.21.15.44:58392) 2024-07-17T08:42:13.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.407+0000 7f6674e00640 1 -- 172.21.15.44:0/1477363663 learned_addr learned my addr 172.21.15.44:0/1477363663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 -- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 msgr2=0x55e8662f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 -- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 msgr2=0x55e8662e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 -- 172.21.15.44:0/1477363663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8662b9050 con 0x55e8663129e0 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f6674e00640 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f6660011ed0 tx=0x7f6660003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf65b878e0d00f7e server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f665d600640 1 -- 172.21.15.44:0/1477363663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660004260 con 0x55e8663129e0 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f665d600640 1 -- 172.21.15.44:0/1477363663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f666002c070 con 0x55e8663129e0 2024-07-17T08:42:13.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.408+0000 7f665d600640 1 -- 172.21.15.44:0/1477363663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660004a00 con 0x55e8663129e0 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 -- 172.21.15.44:0/1477363663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 msgr2=0x55e8662e9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f6660011ed0 tx=0x7f6660003780 comp rx=0 tx=0).stop 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 -- 172.21.15.44:0/1477363663 shutdown_connections 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e8662f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8663129e0 0x55e8662e9060 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 --2- 172.21.15.44:0/1477363663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8662efae0 0x55e8662e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 -- 172.21.15.44:0/1477363663 >> 172.21.15.44:0/1477363663 conn(0x55e8662cdf00 msgr2=0x55e8662c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 -- 172.21.15.44:0/1477363663 shutdown_connections 2024-07-17T08:42:13.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.409+0000 7f667b028740 1 -- 172.21.15.44:0/1477363663 wait complete. 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.410+0000 7f667b028740 1 -- start start 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.410+0000 7f667b028740 1 Processor -- start 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8663129e0 0x55e86638ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e866338450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e8662f8df0 con 0x55e8662efae0 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8662e7cb0 con 0x55e8663129e0 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55e8662f9410 con 0x55e866328fa0 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f667b028740 10 client.0 _ll_register_callbacks cb 0x7f6660009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f6675800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f6675800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58406/0 (socket says 172.21.15.44:58406) 2024-07-17T08:42:13.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.411+0000 7f6675800640 1 -- 172.21.15.44:0/3424651930 learned_addr learned my addr 172.21.15.44:0/3424651930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 -- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 msgr2=0x55e866338450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e866338450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 -- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8663129e0 msgr2=0x55e86638ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8663129e0 0x55e86638ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 -- 172.21.15.44:0/3424651930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8662f48b0 con 0x55e8662efae0 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6675800640 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f6664034040 tx=0x7f6664005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2b07dc9f771ce0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6652a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664044040 con 0x55e8662efae0 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f6664003d40 con 0x55e8662efae0 2024-07-17T08:42:13.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.412+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666002c070 con 0x55e8662efae0 2024-07-17T08:42:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7f6664035040 con 0x55e8662efae0 2024-07-17T08:42:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 10 client.13200 fetch_fsmap learned FSMap version 181 2024-07-17T08:42:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) -- 0x55e8662e8320 con 0x55e8662efae0 2024-07-17T08:42:13.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f666002c070 con 0x55e8662efae0 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 181) ==== 3502+0+0 (secure 0 0 0) 0x7f666405b050 con 0x55e8662efae0 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 10 client.13200 fetch_fsmap finished waiting for FSMap version 181 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 10 client.13200 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 20 client.13200 populate_metadata read hostname 'smithi044' 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f9af0 0x55e8662e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 4 client.13200 mds_command: new command op to 10432 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.413+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 --> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] -- command(tid 0: {"prefix": "status"}) -- 0x55e86623c750 con 0x55e8662f9af0 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.414+0000 7f6674e00640 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f9af0 0x55e8662e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f6674e00640 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f9af0 0x55e8662e6470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7274e316e62a775 server_cookie=d024de600876cdbe in_seq=0 out_seq=0 2024-07-17T08:42:13.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f6652a00640 10 client.13200 ms_handle_connect on v2:172.21.15.175:6800/1754084600 2024-07-17T08:42:13.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f6652a00640 1 -- 172.21.15.44:0/3424651930 <== mds.0 v2:172.21.15.175:6800/1754084600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55e86623c750 con 0x55e8662f9af0 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f6652a00640 10 client.13200 handle_command_reply: tid=0 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 client.13200 shutdown 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 msgr2=0x55e866398270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f6664034040 tx=0x7f6664005c00 comp rx=0 tx=0).stop 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 shutdown_connections 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55e866328fa0 0x55e866338450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:6800/1754084600,v1:172.21.15.175:6801/1754084600] conn(0x55e8662f9af0 0x55e8662e6470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8663129e0 0x55e86638ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 --2- 172.21.15.44:0/3424651930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e8662efae0 0x55e866398270 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f6652a00640 0 client.13200 ms_handle_reset on v2:172.21.15.175:6800/1754084600 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.415+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 >> 172.21.15.44:0/3424651930 conn(0x55e8662cdf00 msgr2=0x55e8662e8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 shutdown_connections 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f667b028740 1 -- 172.21.15.44:0/3424651930 wait complete. 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f667b028740 20 client.13200 trim_cache size 0 max 16384 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 10432, 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:42:13.419 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7622334070000001, 2024-07-17T08:42:13.420 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 181, 2024-07-17T08:42:13.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 321, 2024-07-17T08:42:13.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 321, 2024-07-17T08:42:13.420 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.658868424 2024-07-17T08:42:13.420 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 msgr2=0x7f6654079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 0x7f6654079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6664033000 tx=0x7f6664004730 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 msgr2=0x7f6670150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.416+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f666800c7d0 tx=0x7f666800cc90 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 shutdown_connections 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f6654076cd0 0x7f6654079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6670152220 0x7f6670150aa0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66700cb3a0 0x7f6670150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 --2- 172.21.15.44:0/698756716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f667006ed30 0x7f66701f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 >> 172.21.15.44:0/698756716 conn(0x7f66700b0eb0 msgr2=0x7f667015e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 shutdown_connections 2024-07-17T08:42:13.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.417+0000 7f6678200640 1 -- 172.21.15.44:0/698756716 wait complete. 2024-07-17T08:42:13.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:42:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 -- 172.21.15.44:0/2506897388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 msgr2=0x7f58840ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 --2- 172.21.15.44:0/2506897388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58840ac810 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f58840b4670 tx=0x7f587c02a560 comp rx=0 tx=0).stop 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 -- 172.21.15.44:0/2506897388 shutdown_connections 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 --2- 172.21.15.44:0/2506897388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58840ac810 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 --2- 172.21.15.44:0/2506897388 >> v2:172.21.15.175:3300/0 conn(0x7f5884159b00 0x7f5884159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 --2- 172.21.15.44:0/2506897388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588406f710 0x7f5884153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 -- 172.21.15.44:0/2506897388 >> 172.21.15.44:0/2506897388 conn(0x7f58840b0fb0 msgr2=0x7f58840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 -- 172.21.15.44:0/2506897388 shutdown_connections 2024-07-17T08:42:13.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.616+0000 7f5889800640 1 -- 172.21.15.44:0/2506897388 wait complete. 2024-07-17T08:42:13.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.618+0000 7f5889800640 1 Processor -- start 2024-07-17T08:42:13.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.618+0000 7f5889800640 1 -- start start 2024-07-17T08:42:13.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588406f710 0x7f5884152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5884159b00 0x7f588414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f58840ae980 con 0x7f58840ac3e0 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58840ae800 con 0x7f5884159b00 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.619+0000 7f5889800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f588413ed60 con 0x7f588406f710 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58414/0 (socket says 172.21.15.44:58414) 2024-07-17T08:42:13.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 -- 172.21.15.44:0/77299303 learned_addr learned my addr 172.21.15.44:0/77299303 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 -- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588406f710 msgr2=0x7f5884152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588406f710 0x7f5884152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 -- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5884159b00 msgr2=0x7f588414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f5883400640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5884159b00 0x7f588414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5884159b00 0x7f588414d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 -- 172.21.15.44:0/77299303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588414bfd0 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.620+0000 7f587a000640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f586c00d770 tx=0x7f586c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.621+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c017070 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.621+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f586c009d20 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.621+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588414c2b0 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.621+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0106c0 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.621+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588414c870 con 0x7f58840ac3e0 2024-07-17T08:42:13.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.622+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58840ac810 con 0x7f58840ac3e0 2024-07-17T08:42:13.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.623+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f586c01a070 con 0x7f58840ac3e0 2024-07-17T08:42:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.624+0000 7f587b400640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 0x7f58600791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.624+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f586c098aa0 con 0x7f58840ac3e0 2024-07-17T08:42:13.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.625+0000 7f5882a00640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 0x7f58600791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:13.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.626+0000 7f5882a00640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 0x7f58600791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874009710 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:13.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.628+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f586c0618c0 con 0x7f58840ac3e0 2024-07-17T08:42:13.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.882+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58840cb8e0 con 0x7f58840ac3e0 2024-07-17T08:42:13.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.883+0000 7f587b400640 1 -- 172.21.15.44:0/77299303 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v181) ==== 53+0+73 (secure 0 0 0) 0x7f586c0611c0 con 0x7f58840ac3e0 2024-07-17T08:42:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:42:13.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 msgr2=0x7f58600791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 0x7f58600791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5874005fd0 tx=0x7f5874009710 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 msgr2=0x7f58841f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f586c00d770 tx=0x7f586c00dc30 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 shutdown_connections 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f5860076d00 0x7f58600791c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5884159b00 0x7f588414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58840ac3e0 0x7f58841f2290 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 --2- 172.21.15.44:0/77299303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f588406f710 0x7f5884152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 >> 172.21.15.44:0/77299303 conn(0x7f58840b0fb0 msgr2=0x7f588415c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.885+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 shutdown_connections 2024-07-17T08:42:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:13.886+0000 7f5889800640 1 -- 172.21.15.44:0/77299303 wait complete. 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:42:13.906 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:13.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:13.907 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:13.957 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36124sec preferred_lft 36124sec 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:13.958 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:42:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:13.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:13.961 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:42:14.037 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:42:14.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:14.041 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:42:14.089 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:42:14.089 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:42:14.089 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:42:14.089 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:42:14.089 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:42:14.090 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:42:14.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:42:14.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:42:14.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:14.350 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:42:14.396 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:42:14.396 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:42:14.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:14.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:14.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:42:14.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:42:14.532 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:14.590 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:14.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:42:14.591 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:14.642 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:42:14.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:14.643 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:14.771 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:14.771 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:14.771 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:14.792 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:42:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:42:14.825 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:42:14.828 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:42:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:42:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:42:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:42:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:42:14.833 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:42:14.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:14.833 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:14.914 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:42:15.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:42:15.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:42:15.068 INFO:teuthology.orchestra.run.smithi044.stdout:client10492 2024-07-17T08:42:15.068 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:15.068 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:15.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 -- 172.21.15.44:0/847741504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 msgr2=0x7f3eb40b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 --2- 172.21.15.44:0/847741504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb40b3280 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3eac005fd0 tx=0x7f3eac02a920 comp rx=0 tx=0).stop 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 -- 172.21.15.44:0/847741504 shutdown_connections 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 --2- 172.21.15.44:0/847741504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb40b3280 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 -- 172.21.15.44:0/847741504 >> 172.21.15.44:0/847741504 conn(0x7f3eb40b0eb0 msgr2=0x7f3eb40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 -- 172.21.15.44:0/847741504 shutdown_connections 2024-07-17T08:42:15.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.338+0000 7f3ebac00640 1 -- 172.21.15.44:0/847741504 wait complete. 2024-07-17T08:42:15.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.340+0000 7f3ebac00640 1 Processor -- start 2024-07-17T08:42:15.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.340+0000 7f3ebac00640 1 -- start start 2024-07-17T08:42:15.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb406eb90 0x7f3eb414a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb414ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb414fd60 con 0x7f3eb415a5a0 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb406f8e0 con 0x7f3eb406eb90 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3ebac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb414fee0 con 0x7f3eb40acab0 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3eb8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3eb8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58424/0 (socket says 172.21.15.44:58424) 2024-07-17T08:42:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3eb8c00640 1 -- 172.21.15.44:0/3199740849 learned_addr learned my addr 172.21.15.44:0/3199740849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:15.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3eb3e00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb406eb90 0x7f3eb414a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.342+0000 7f3eb3400640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb414ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 -- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 msgr2=0x7f3eb414ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb414ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 -- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb406eb90 msgr2=0x7f3eb414a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb406eb90 0x7f3eb414a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 -- 172.21.15.44:0/3199740849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb414acf0 con 0x7f3eb415a5a0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eb8c00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f3eac02af10 tx=0x7f3eac030660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac030c60 con 0x7f3eb415a5a0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3eac003970 con 0x7f3eb415a5a0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac003da0 con 0x7f3eb415a5a0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.343+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb4148cf0 con 0x7f3eb415a5a0 2024-07-17T08:42:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.344+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb4148730 con 0x7f3eb415a5a0 2024-07-17T08:42:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.345+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb40ade60 con 0x7f3eb415a5a0 2024-07-17T08:42:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.346+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f3eac003f40 con 0x7f3eb415a5a0 2024-07-17T08:42:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.347+0000 7f3eabe00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.347+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f3eac0ba4e0 con 0x7f3eb415a5a0 2024-07-17T08:42:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.348+0000 7f3eb3e00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.349+0000 7f3eb3e00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 0x7f3e94079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40016b0 tx=0x7f3ea4009040 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.351+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3eb40ade60 con 0x7f3eb415a5a0 2024-07-17T08:42:15.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.589+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 --> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3eb40cd5c0 con 0x7f3e94076cd0 2024-07-17T08:42:15.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.593+0000 7f3eabe00640 1 -- 172.21.15.44:0/3199740849 <== mgr.10276 v2:172.21.15.175:6834/3850741755 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3eb40cd5c0 con 0x7f3e94076cd0 2024-07-17T08:42:15.594 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:42:15.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:15.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:42:15.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:15.594 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:42:15.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 msgr2=0x7f3e94079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 0x7f3e94079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40016b0 tx=0x7f3ea4009040 comp rx=0 tx=0).stop 2024-07-17T08:42:15.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 msgr2=0x7f3eb40975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f3eac02af10 tx=0x7f3eac030660 comp rx=0 tx=0).stop 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 shutdown_connections 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f3e94076cd0 0x7f3e94079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb415a5a0 0x7f3eb40975c0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3eb40acab0 0x7f3eb414ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 --2- 172.21.15.44:0/3199740849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3eb406eb90 0x7f3eb414a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 >> 172.21.15.44:0/3199740849 conn(0x7f3eb40b0eb0 msgr2=0x7f3eb414ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 shutdown_connections 2024-07-17T08:42:15.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.596+0000 7f3ebac00640 1 -- 172.21.15.44:0/3199740849 wait complete. 2024-07-17T08:42:15.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:42:15.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 -- 172.21.15.44:0/3751738369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 msgr2=0x7f54f40ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 --2- 172.21.15.44:0/3751738369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f40ce130 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f54f40cf640 tx=0x7f54ec02a560 comp rx=0 tx=0).stop 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 -- 172.21.15.44:0/3751738369 shutdown_connections 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 --2- 172.21.15.44:0/3751738369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f40ce130 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 --2- 172.21.15.44:0/3751738369 >> v2:172.21.15.175:3300/0 conn(0x7f54f40d1440 0x7f54f40d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 -- 172.21.15.44:0/3751738369 >> 172.21.15.44:0/3751738369 conn(0x7f54f40d04c0 msgr2=0x7f54f40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.825+0000 7f54fa400640 1 -- 172.21.15.44:0/3751738369 shutdown_connections 2024-07-17T08:42:15.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.826+0000 7f54fa400640 1 -- 172.21.15.44:0/3751738369 wait complete. 2024-07-17T08:42:15.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.827+0000 7f54fa400640 1 Processor -- start 2024-07-17T08:42:15.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.827+0000 7f54fa400640 1 -- start start 2024-07-17T08:42:15.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.828+0000 7f54fa400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.828+0000 7f54fa400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40d1440 0x7f54f40d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.828+0000 7f54fa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f415d350 0x7f54f40d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54fa400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54f406e3c0 con 0x7f54f40cdca0 2024-07-17T08:42:15.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54fa400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f406e240 con 0x7f54f40d1440 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54fa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54f406e540 con 0x7f54f415d350 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58448/0 (socket says 172.21.15.44:58448) 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 -- 172.21.15.44:0/3910764722 learned_addr learned my addr 172.21.15.44:0/3910764722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 -- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f415d350 msgr2=0x7f54f40d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9a00640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f415d350 0x7f54f40d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f415d350 0x7f54f40d33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 -- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40d1440 msgr2=0x7f54f40d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40d1440 0x7f54f40d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.829+0000 7f54f9000640 1 -- 172.21.15.44:0/3910764722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f41f4350 con 0x7f54f40cdca0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54f9000640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c730 tx=0x7f54e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4016020 con 0x7f54f40cdca0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e400ce40 con 0x7f54f40cdca0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4006e60 con 0x7f54f40cdca0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f41f4630 con 0x7f54f40cdca0 2024-07-17T08:42:15.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.830+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f41f4bf0 con 0x7f54f40cdca0 2024-07-17T08:42:15.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.831+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f406ea40 con 0x7f54f40cdca0 2024-07-17T08:42:15.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.833+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 189) ==== 98931+0+0 (secure 0 0 0) 0x7f54e4003bf0 con 0x7f54f40cdca0 2024-07-17T08:42:15.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.833+0000 7f54f1600640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 0x7f54d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:15.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.834+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f54e40985f0 con 0x7f54f40cdca0 2024-07-17T08:42:15.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.834+0000 7f54ebe00640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 0x7f54d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:15.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.835+0000 7f54ebe00640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 0x7f54d0079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005fd0 tx=0x7f54dc005850 comp rx=0 tx=0).ready entity=mgr.10276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:15.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:15.838+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f54e4020020 con 0x7f54f40cdca0 2024-07-17T08:42:16.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.110+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f54f40b29f0 con 0x7f54f40cdca0 2024-07-17T08:42:16.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.242+0000 7f54f1600640 1 -- 172.21.15.44:0/3910764722 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v190) ==== 92+0+0 (secure 0 0 0) 0x7f54e40615b0 con 0x7f54f40cdca0 2024-07-17T08:42:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.245+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 msgr2=0x7f54d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.245+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 0x7f54d0079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005fd0 tx=0x7f54dc005850 comp rx=0 tx=0).stop 2024-07-17T08:42:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.245+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 msgr2=0x7f54f4152bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.245+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c730 tx=0x7f54e400cbf0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 shutdown_connections 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f54d0076cd0 0x7f54d0079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f415d350 0x7f54f40d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40d1440 0x7f54f40d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 --2- 172.21.15.44:0/3910764722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54f40cdca0 0x7f54f4152bd0 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 >> 172.21.15.44:0/3910764722 conn(0x7f54f40d04c0 msgr2=0x7f54f415b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 shutdown_connections 2024-07-17T08:42:16.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.246+0000 7f54fa400640 1 -- 172.21.15.44:0/3910764722 wait complete. 2024-07-17T08:42:16.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:42:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 -- 172.21.15.44:0/173562845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 msgr2=0x7fb11c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 --2- 172.21.15.44:0/173562845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb11c00b9c0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fb130005fd0 tx=0x7fb11c005830 comp rx=0 tx=0).stop 2024-07-17T08:42:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 -- 172.21.15.44:0/173562845 shutdown_connections 2024-07-17T08:42:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 --2- 172.21.15.44:0/173562845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb11c00b9c0 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 -- 172.21.15.44:0/173562845 >> 172.21.15.44:0/173562845 conn(0x7fb13809aa30 msgr2=0x7fb1380b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 -- 172.21.15.44:0/173562845 shutdown_connections 2024-07-17T08:42:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.473+0000 7fb13dc00640 1 -- 172.21.15.44:0/173562845 wait complete. 2024-07-17T08:42:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.475+0000 7fb13dc00640 1 Processor -- start 2024-07-17T08:42:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.475+0000 7fb13dc00640 1 -- start start 2024-07-17T08:42:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13814e4a0 0x7fb13806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 0x7fb1380ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1380cd750 con 0x7fb11c005210 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1380cd5d0 con 0x7fb13814e4a0 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.476+0000 7fb13dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1380cd8d0 con 0x7fb13806ede0 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58474/0 (socket says 172.21.15.44:58474) 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 -- 172.21.15.44:0/1421003447 learned_addr learned my addr 172.21.15.44:0/1421003447 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb12fe00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13814e4a0 0x7fb13806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb137400640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 0x7fb1380ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 msgr2=0x7fb1380ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 0x7fb1380ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13814e4a0 msgr2=0x7fb13806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13814e4a0 0x7fb13806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb136a00640 1 -- 172.21.15.44:0/1421003447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb138158c00 con 0x7fb11c005210 2024-07-17T08:42:16.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.477+0000 7fb137400640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 0x7fb1380ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb136a00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fb12800d640 tx=0x7fb12800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1280062e0 con 0x7fb11c005210 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb128015020 con 0x7fb11c005210 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128003870 con 0x7fb11c005210 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb138158ee0 con 0x7fb11c005210 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.478+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1381594a0 con 0x7fb11c005210 2024-07-17T08:42:16.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.479+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1380ce890 con 0x7fb11c005210 2024-07-17T08:42:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.480+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 190) ==== 98948+0+0 (secure 0 0 0) 0x7fb128003a10 con 0x7fb11c005210 2024-07-17T08:42:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.481+0000 7fb12ea00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 0x7fb1140791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.482+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7fb1280990c0 con 0x7fb11c005210 2024-07-17T08:42:16.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.482+0000 7fb12fe00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 msgr2=0x7fb1140791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:16.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.482+0000 7fb12fe00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 0x7fb1140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.486+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb12801f020 con 0x7fb11c005210 2024-07-17T08:42:16.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.683+0000 7fb12fe00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 msgr2=0x7fb1140791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:16.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.683+0000 7fb12fe00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 0x7fb1140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:16.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.699+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb138073a80 con 0x7fb11c005210 2024-07-17T08:42:16.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.706+0000 7fb12ea00640 1 -- 172.21.15.44:0/1421003447 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v97)=0 v97) ==== 138+0+0 (secure 0 0 0) 0x7fb128061ee0 con 0x7fb11c005210 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.708+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 msgr2=0x7fb1140791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 0x7fb1140791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 msgr2=0x7fb138159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fb12800d640 tx=0x7fb12800db00 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 shutdown_connections 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7fb114076d20 0x7fb1140791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13806ede0 0x7fb1380ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13814e4a0 0x7fb13806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 --2- 172.21.15.44:0/1421003447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c005210 0x7fb138159ca0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 >> 172.21.15.44:0/1421003447 conn(0x7fb13809aa30 msgr2=0x7fb1380aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 shutdown_connections 2024-07-17T08:42:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.709+0000 7fb13dc00640 1 -- 172.21.15.44:0/1421003447 wait complete. 2024-07-17T08:42:16.731 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:42:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 -- 172.21.15.44:0/670069125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c815b5d0 msgr2=0x7f14c80b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 --2- 172.21.15.44:0/670069125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c815b5d0 0x7f14c80b1f00 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f14c80ae680 tx=0x7f14b802aa30 comp rx=0 tx=0).stop 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 -- 172.21.15.44:0/670069125 shutdown_connections 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 --2- 172.21.15.44:0/670069125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 0x7f14c80b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 --2- 172.21.15.44:0/670069125 >> v2:172.21.15.175:3300/0 conn(0x7f14c80b5450 0x7f14c80af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 --2- 172.21.15.44:0/670069125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c815b5d0 0x7f14c80b1f00 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.935+0000 7f14cca00640 1 -- 172.21.15.44:0/670069125 >> 172.21.15.44:0/670069125 conn(0x7f14c8073d50 msgr2=0x7f14c809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.936+0000 7f14cca00640 1 -- 172.21.15.44:0/670069125 shutdown_connections 2024-07-17T08:42:16.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.936+0000 7f14cca00640 1 -- 172.21.15.44:0/670069125 wait complete. 2024-07-17T08:42:16.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.937+0000 7f14cca00640 1 Processor -- start 2024-07-17T08:42:16.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.937+0000 7f14cca00640 1 -- start start 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 0x7f14c813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14c815b5d0 0x7f14c8070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80afa70 con 0x7f14c80b5450 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14c80af8f0 con 0x7f14c815b5d0 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.938+0000 7f14cca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f14c80afbf0 con 0x7f14c80b0fe0 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58480/0 (socket says 172.21.15.44:58480) 2024-07-17T08:42:16.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 -- 172.21.15.44:0/1191718277 learned_addr learned my addr 172.21.15.44:0/1191718277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5e00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 0x7f14c813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c6800640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14c815b5d0 0x7f14c8070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 msgr2=0x7f14c813f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 0x7f14c813f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14c815b5d0 msgr2=0x7f14c8070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14c815b5d0 0x7f14c8070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 -- 172.21.15.44:0/1191718277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c813f590 con 0x7f14c80b5450 2024-07-17T08:42:16.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.939+0000 7f14c5400640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c940 tx=0x7f14b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.940+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0007bb0 con 0x7f14c80b5450 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.940+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b0005020 con 0x7f14c80b5450 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.940+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81f4c40 con 0x7f14c80b5450 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.940+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0007e60 con 0x7f14c80b5450 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.940+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c8141c90 con 0x7f14c80b5450 2024-07-17T08:42:16.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.941+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c80b0180 con 0x7f14c80b5450 2024-07-17T08:42:16.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.942+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 190) ==== 98948+0+0 (secure 0 0 0) 0x7f14b0005470 con 0x7f14c80b5450 2024-07-17T08:42:16.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.943+0000 7f14be000640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 0x7f14a4079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:16.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.943+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f14b0098ea0 con 0x7f14c80b5450 2024-07-17T08:42:16.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.944+0000 7f14c5e00640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 msgr2=0x7f14a4079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:16.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.944+0000 7f14c5e00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 0x7f14a4079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:16.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:16.948+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f14b0014030 con 0x7f14c80b5450 2024-07-17T08:42:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.145+0000 7f14c5e00640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 msgr2=0x7f14a4079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:17.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.145+0000 7f14c5e00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 0x7f14a4079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:17.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.156+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f14c815bde0 con 0x7f14c80b5450 2024-07-17T08:42:17.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.164+0000 7f14be000640 1 -- 172.21.15.44:0/1191718277 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v98)=0 v98) ==== 133+0+0 (secure 0 0 0) 0x7f14b0061cc0 con 0x7f14c80b5450 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.166+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 msgr2=0x7f14a4079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 0x7f14a4079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 msgr2=0x7f14c8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c940 tx=0x7f14b000ce00 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 shutdown_connections 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f14a4076d50 0x7f14a4079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14c815b5d0 0x7f14c8070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c80b5450 0x7f14c8070690 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 --2- 172.21.15.44:0/1191718277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14c80b0fe0 0x7f14c813f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 >> 172.21.15.44:0/1191718277 conn(0x7f14c8073d50 msgr2=0x7f14c815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:17.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 shutdown_connections 2024-07-17T08:42:17.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.167+0000 7f14cca00640 1 -- 172.21.15.44:0/1191718277 wait complete. 2024-07-17T08:42:17.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs mygrp04 2024-07-17T08:42:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 -- 172.21.15.44:0/2401453288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15ef10 msgr2=0x7f891c06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 --2- 172.21.15.44:0/2401453288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15ef10 0x7f891c06c320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f890c009830 tx=0x7f890c02a480 comp rx=0 tx=0).stop 2024-07-17T08:42:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 -- 172.21.15.44:0/2401453288 shutdown_connections 2024-07-17T08:42:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 --2- 172.21.15.44:0/2401453288 >> v2:172.21.15.175:3300/0 conn(0x7f891c15b840 0x7f891c06e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 --2- 172.21.15.44:0/2401453288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15ef10 0x7f891c06c320 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 -- 172.21.15.44:0/2401453288 >> 172.21.15.44:0/2401453288 conn(0x7f891c0b2240 msgr2=0x7f891c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 -- 172.21.15.44:0/2401453288 shutdown_connections 2024-07-17T08:42:17.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.382+0000 7f8923a00640 1 -- 172.21.15.44:0/2401453288 wait complete. 2024-07-17T08:42:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.384+0000 7f8923a00640 1 Processor -- start 2024-07-17T08:42:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.384+0000 7f8923a00640 1 -- start start 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f891c06bf40 0x7f891c14c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15b840 0x7f891c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 0x7f891c1f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f891c06ee50 con 0x7f891c15ef10 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f891c06ecd0 con 0x7f891c06bf40 2024-07-17T08:42:17.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8923a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f891c06efd0 con 0x7f891c15b840 2024-07-17T08:42:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.385+0000 7f8921a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 0x7f891c1f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 0x7f891c1f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58490/0 (socket says 172.21.15.44:58490) 2024-07-17T08:42:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 -- 172.21.15.44:0/2466913166 learned_addr learned my addr 172.21.15.44:0/2466913166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:17.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f891be00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15b840 0x7f891c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15b840 msgr2=0x7f891c149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15b840 0x7f891c149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f891c06bf40 msgr2=0x7f891c14c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f891c06bf40 0x7f891c14c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c149e70 con 0x7f891c15ef10 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f891be00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f891c15b840 0x7f891c149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.386+0000 7f8921a00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 0x7f891c1f1950 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f891400eb10 tx=0x7f891400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.387+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914004b20 con 0x7f891c15ef10 2024-07-17T08:42:17.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.387+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8914004ce0 con 0x7f891c15ef10 2024-07-17T08:42:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.387+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c157ef0 con 0x7f891c15ef10 2024-07-17T08:42:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.387+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914005280 con 0x7f891c15ef10 2024-07-17T08:42:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.387+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1584b0 con 0x7f891c15ef10 2024-07-17T08:42:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.388+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c06cd20 con 0x7f891c15ef10 2024-07-17T08:42:17.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.389+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 190) ==== 98948+0+0 (secure 0 0 0) 0x7f8914005420 con 0x7f891c15ef10 2024-07-17T08:42:17.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.391+0000 7f8918c00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.391+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(321..321 src has 1..321) ==== 6705+0+0 (secure 0 0 0) 0x7f8914099770 con 0x7f891c15ef10 2024-07-17T08:42:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.391+0000 7f8921000640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:17.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.391+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.395+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8914062590 con 0x7f891c15ef10 2024-07-17T08:42:17.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.593+0000 7f8921000640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:17.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.593+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:17.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.634+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp04", "target": ["mon-mgr", ""]}) -- 0x7f891c0b2df0 con 0x7f88f8076d20 2024-07-17T08:42:17.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.995+0000 7f8921000640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:17.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:17.995+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:42:18.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:18.796+0000 7f8921000640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:18.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:18.796+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:42:20.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:20.399+0000 7f8921000640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3850741755 2024-07-17T08:42:20.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:20.399+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:42:20.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:20.505+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 191) ==== 98557+0+0 (secure 0 0 0) 0x7f891405e120 con 0x7f891c15ef10 2024-07-17T08:42:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:20.506+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 msgr2=0x7f88f80791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:20.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:20.506+0000 7f8918c00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/3850741755,v1:172.21.15.175:6837/3850741755] conn(0x7f88f8076d20 0x7f88f80791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0954c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0954c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0954c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0954c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0954c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0951000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0951000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0951000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0951000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:20.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:20.745+0000 7f0951000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.513+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 192) ==== 98684+0+0 (secure 0 0 0) 0x7f89140049d0 con 0x7f891c15ef10 2024-07-17T08:42:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.514+0000 7f8918c00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f88f807f980 0x7f88f8081d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.514+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 --> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp04", "target": ["mon-mgr", ""]}) -- 0x7f88f800c880 con 0x7f88f807f980 2024-07-17T08:42:21.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.516+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f88f807f980 0x7f88f8081d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:21.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.520+0000 7f8921000640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f88f807f980 0x7f88f8081d70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8904000f00 tx=0x7f8904007ab0 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:21.549 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:21.548+0000 7f094d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.549 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:21.548+0000 7f094d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.549 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:21.548+0000 7f094d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.549 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:21.548+0000 7f094d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.549 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:21.548+0000 7f094d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:21.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.575+0000 7f8918c00640 1 -- 172.21.15.44:0/2466913166 <== mgr.9464 v2:172.21.15.175:6834/2701447474 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88f800c880 con 0x7f88f807f980 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f88f807f980 msgr2=0x7f88f8081d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f88f807f980 0x7f88f8081d70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8904000f00 tx=0x7f8904007ab0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 msgr2=0x7f891c1f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 --2- 172.21.15.44:0/2466913166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c15ef10 0x7f891c1f1950 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f891400eb10 tx=0x7f891400c2d0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8921a00640 1 -- 172.21.15.44:0/2466913166 reap_dead start 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 shutdown_connections 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 >> 172.21.15.44:0/2466913166 conn(0x7f891c0b2240 msgr2=0x7f891c070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.579+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 shutdown_connections 2024-07-17T08:42:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.580+0000 7f8923a00640 1 -- 172.21.15.44:0/2466913166 wait complete. 2024-07-17T08:42:21.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv04 --group_name mygrp04 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.816+0000 7f7d52c00640 1 -- 172.21.15.44:0/827496283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 msgr2=0x7f7d4c0b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.816+0000 7f7d52c00640 1 --2- 172.21.15.44:0/827496283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c0b8670 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c08c210 tx=0x7f7d4402a8f0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 -- 172.21.15.44:0/827496283 shutdown_connections 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 --2- 172.21.15.44:0/827496283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c0b8670 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 --2- 172.21.15.44:0/827496283 >> v2:172.21.15.175:3300/0 conn(0x7f7d4c161b50 0x7f7d4c0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 -- 172.21.15.44:0/827496283 >> 172.21.15.44:0/827496283 conn(0x7f7d4c073a60 msgr2=0x7f7d4c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 -- 172.21.15.44:0/827496283 shutdown_connections 2024-07-17T08:42:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.817+0000 7f7d52c00640 1 -- 172.21.15.44:0/827496283 wait complete. 2024-07-17T08:42:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.818+0000 7f7d52c00640 1 Processor -- start 2024-07-17T08:42:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.819+0000 7f7d52c00640 1 -- start start 2024-07-17T08:42:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 0x7f7d4c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d4c161b50 0x7f7d4c1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c0b8d30 con 0x7f7d4c0d0970 2024-07-17T08:42:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c0b8bb0 con 0x7f7d4c15e370 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4c0b8eb0 con 0x7f7d4c161b50 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44106/0 (socket says 172.21.15.44:44106) 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 -- 172.21.15.44:0/4290190145 learned_addr learned my addr 172.21.15.44:0/4290190145 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d50e00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 0x7f7d4c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d52200640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d4c161b50 0x7f7d4c1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 -- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d4c161b50 msgr2=0x7f7d4c1f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d4c161b50 0x7f7d4c1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 -- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 msgr2=0x7f7d4c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 0x7f7d4c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.820+0000 7f7d51800640 1 -- 172.21.15.44:0/4290190145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c0b42c0 con 0x7f7d4c0d0970 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d50e00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 0x7f7d4c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d51800640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00c430 tx=0x7f7d3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c011790 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d3c017070 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c011d00 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4c0b45a0 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.821+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4c0b4b60 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.822+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4c0b9360 con 0x7f7d4c0d0970 2024-07-17T08:42:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.824+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 192) ==== 98684+0+0 (secure 0 0 0) 0x7f7d3c01c460 con 0x7f7d4c0d0970 2024-07-17T08:42:21.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.824+0000 7f7d49600640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 0x7f7d2c078f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:21.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.825+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f7d3c098480 con 0x7f7d4c0d0970 2024-07-17T08:42:21.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.825+0000 7f7d50e00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 0x7f7d2c078f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:21.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.826+0000 7f7d50e00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 0x7f7d2c078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0ca160 tx=0x7f7d38005db0 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:21.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:21.829+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7d3c061680 con 0x7f7d4c0d0970 2024-07-17T08:42:22.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.069+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 --> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv04", "group_name": "mygrp04", "target": ["mon-mgr", ""]}) -- 0x7f7d4c0c6d90 con 0x7f7d2c076a80 2024-07-17T08:42:22.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.133+0000 7f7d49600640 1 -- 172.21.15.44:0/4290190145 <== mgr.9464 v2:172.21.15.175:6834/2701447474 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d4c0c6d90 con 0x7f7d2c076a80 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 msgr2=0x7f7d2c078f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 0x7f7d2c078f40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0ca160 tx=0x7f7d38005db0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 msgr2=0x7f7d4c1f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00c430 tx=0x7f7d3c00c8f0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 shutdown_connections 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d4c161b50 0x7f7d4c1f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d4c15e370 0x7f7d4c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f7d2c076a80 0x7f7d2c078f40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 --2- 172.21.15.44:0/4290190145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d4c0d0970 0x7f7d4c1f4e10 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 >> 172.21.15.44:0/4290190145 conn(0x7f7d4c073a60 msgr2=0x7f7d4c15f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 shutdown_connections 2024-07-17T08:42:22.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.136+0000 7f7d52c00640 1 -- 172.21.15.44:0/4290190145 wait complete. 2024-07-17T08:42:22.155 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --group mygrp04 . 1m 2024-07-17T08:42:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 -- 172.21.15.44:0/4059145786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 msgr2=0x7fae740add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 --2- 172.21.15.44:0/4059145786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae740add80 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fae740aebe0 tx=0x7fae5c02a530 comp rx=0 tx=0).stop 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 -- 172.21.15.44:0/4059145786 shutdown_connections 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 --2- 172.21.15.44:0/4059145786 >> v2:172.21.15.175:3300/0 conn(0x7fae740ae2c0 0x7fae740729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 --2- 172.21.15.44:0/4059145786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae740add80 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 -- 172.21.15.44:0/4059145786 >> 172.21.15.44:0/4059145786 conn(0x7fae740cf050 msgr2=0x7fae740cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 -- 172.21.15.44:0/4059145786 shutdown_connections 2024-07-17T08:42:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.372+0000 7fae73200640 1 -- 172.21.15.44:0/4059145786 wait complete. 2024-07-17T08:42:22.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.374+0000 7fae73200640 1 Processor -- start 2024-07-17T08:42:22.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.374+0000 7fae73200640 1 -- start start 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae74152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae740ae2c0 0x7fae7414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae74155860 0x7fae7414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae74073280 con 0x7fae740ad960 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae74073100 con 0x7fae740ae2c0 2024-07-17T08:42:22.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.375+0000 7fae73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae74073400 con 0x7fae74155860 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae74152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae74152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44132/0 (socket says 172.21.15.44:44132) 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 -- 172.21.15.44:0/2958064763 learned_addr learned my addr 172.21.15.44:0/2958064763 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae69400640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae740ae2c0 0x7fae7414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae72800640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae74155860 0x7fae7414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 -- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae74155860 msgr2=0x7fae7414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae74155860 0x7fae7414f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 -- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae740ae2c0 msgr2=0x7fae7414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae740ae2c0 0x7fae7414f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.376+0000 7fae71e00640 1 -- 172.21.15.44:0/2958064763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae74151530 con 0x7fae740ad960 2024-07-17T08:42:22.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae71e00640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae740ad960 0x7fae74152280 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fae6400c3c0 tx=0x7fae6400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64017070 con 0x7fae740ad960 2024-07-17T08:42:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae640035f0 con 0x7fae740ad960 2024-07-17T08:42:22.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64003a80 con 0x7fae740ad960 2024-07-17T08:42:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae73200640 1 -- 172.21.15.44:0/2958064763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae741517a0 con 0x7fae740ad960 2024-07-17T08:42:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.377+0000 7fae73200640 1 -- 172.21.15.44:0/2958064763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741f2d50 con 0x7fae740ad960 2024-07-17T08:42:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.378+0000 7fae73200640 1 -- 172.21.15.44:0/2958064763 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae740add80 con 0x7fae740ad960 2024-07-17T08:42:22.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.379+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 192) ==== 98684+0+0 (secure 0 0 0) 0x7fae6401a070 con 0x7fae740ad960 2024-07-17T08:42:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.380+0000 7fae6aa00640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fae50076a50 0x7fae50078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.381+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7fae64097910 con 0x7fae740ad960 2024-07-17T08:42:22.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.381+0000 7fae69400640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fae50076a50 0x7fae50078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.382+0000 7fae69400640 1 --2- 172.21.15.44:0/2958064763 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fae50076a50 0x7fae50078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c004730 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:22.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.385+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fae64060b10 con 0x7fae740ad960 2024-07-17T08:42:22.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.568+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 193) ==== 99000+0+0 (secure 0 0 0) 0x7fae6405c6a0 con 0x7fae740ad960 2024-07-17T08:42:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.627+0000 7fae73200640 1 -- 172.21.15.44:0/2958064763 --> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "group": "mygrp04", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fae740cfc50 con 0x7fae50076a50 2024-07-17T08:42:22.634 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:22.634+0000 7f095d800640 -1 mgr.server reply reply (22) Invalid argument No such subvol: mygrp04::None 2024-07-17T08:42:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.634+0000 7fae6aa00640 1 -- 172.21.15.44:0/2958064763 <== mgr.9464 v2:172.21.15.175:6834/2701447474 1 ==== mgr_command_reply(tid 0: -22 No such subvol: mygrp04::None) ==== 37+0+0 (secure 0 0 0) 0x7fae740cfc50 con 0x7fae50076a50 2024-07-17T08:42:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: No such subvol: mygrp04::None 2024-07-17T08:42:22.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:42:22.647 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv04 --group_name mygrp04 2024-07-17T08:42:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 -- 172.21.15.44:0/189588941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 msgr2=0x7f38d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 --2- 172.21.15.44:0/189588941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38d000b9c0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009e20 tx=0x7f38d0005830 comp rx=0 tx=0).stop 2024-07-17T08:42:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 -- 172.21.15.44:0/189588941 shutdown_connections 2024-07-17T08:42:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 --2- 172.21.15.44:0/189588941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38d000b9c0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 -- 172.21.15.44:0/189588941 >> 172.21.15.44:0/189588941 conn(0x7f38ec0d00a0 msgr2=0x7f38ec0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 -- 172.21.15.44:0/189588941 shutdown_connections 2024-07-17T08:42:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.870+0000 7f38f2800640 1 -- 172.21.15.44:0/189588941 wait complete. 2024-07-17T08:42:22.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.872+0000 7f38f2800640 1 Processor -- start 2024-07-17T08:42:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.872+0000 7f38f2800640 1 -- start start 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 0x7f38ec153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38ec0b75c0 0x7f38ec0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec06e480 con 0x7f38d0005210 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec0b56f0 con 0x7f38ec153f70 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38f2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec06e600 con 0x7f38ec0b75c0 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38eb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38eb400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44162/0 (socket says 172.21.15.44:44162) 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.873+0000 7f38eb400640 1 -- 172.21.15.44:0/2728261234 learned_addr learned my addr 172.21.15.44:0/2728261234 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:22.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38ebe00640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38ec0b75c0 0x7f38ec0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 -- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38ec0b75c0 msgr2=0x7f38ec0b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eaa00640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 0x7f38ec153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38ec0b75c0 0x7f38ec0b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 -- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 msgr2=0x7f38ec153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 0x7f38ec153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 -- 172.21.15.44:0/2728261234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec154400 con 0x7f38d0005210 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eaa00640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 0x7f38ec153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:22.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.874+0000 7f38eb400640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c6b0 tx=0x7f38dc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.875+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc017070 con 0x7f38d0005210 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.875+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38dc005380 con 0x7f38d0005210 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.875+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec156a30 con 0x7f38d0005210 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.875+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00a8a0 con 0x7f38d0005210 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.875+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec152e50 con 0x7f38d0005210 2024-07-17T08:42:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.876+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec161020 con 0x7f38d0005210 2024-07-17T08:42:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.877+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 193) ==== 99000+0+0 (secure 0 0 0) 0x7f38dc01a070 con 0x7f38d0005210 2024-07-17T08:42:22.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.878+0000 7f38e3400640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 0x7f38cc079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.878+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f38dc098170 con 0x7f38d0005210 2024-07-17T08:42:22.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.879+0000 7f38eaa00640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 0x7f38cc079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:22.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.880+0000 7f38eaa00640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 0x7f38cc079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ec1f86a0 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:22.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:22.882+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f38dc060790 con 0x7f38d0005210 2024-07-17T08:42:23.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.124+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 --> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv04", "group_name": "mygrp04", "target": ["mon-mgr", ""]}) -- 0x7f38ec0b4dc0 con 0x7f38cc076db0 2024-07-17T08:42:23.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.162+0000 7f38e3400640 1 -- 172.21.15.44:0/2728261234 <== mgr.9464 v2:172.21.15.175:6834/2701447474 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38ec0b4dc0 con 0x7f38cc076db0 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 msgr2=0x7f38cc079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 0x7f38cc079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ec1f86a0 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 msgr2=0x7f38ec143630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00c6b0 tx=0x7f38dc00cb70 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 shutdown_connections 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38ec0b75c0 0x7f38ec0b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38ec153f70 0x7f38ec153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f38cc076db0 0x7f38cc079270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 --2- 172.21.15.44:0/2728261234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38d0005210 0x7f38ec143630 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 >> 172.21.15.44:0/2728261234 conn(0x7f38ec0d00a0 msgr2=0x7f38ec0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:23.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 shutdown_connections 2024-07-17T08:42:23.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.166+0000 7f38f2800640 1 -- 172.21.15.44:0/2728261234 wait complete. 2024-07-17T08:42:23.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs mygrp04 2024-07-17T08:42:23.189 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:23.188+0000 7f094fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:23.189 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:23.188+0000 7f094fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:23.190 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:23.188+0000 7f094fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:23.190 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:23.188+0000 7f094fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:23.190 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:23.188+0000 7f094fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1203071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba4152300 msgr2=0x7f1ba406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1203071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba4152300 0x7f1ba406f1b0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005fd0 tx=0x7f1b9c02aa00 comp rx=0 tx=0).stop 2024-07-17T08:42:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1203071397 shutdown_connections 2024-07-17T08:42:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1203071397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba4152300 0x7f1ba406f1b0 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1203071397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba40cd0b0 0x7f1ba406ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1203071397 >> 172.21.15.44:0/1203071397 conn(0x7f1ba40cf0d0 msgr2=0x7f1ba40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:23.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1203071397 shutdown_connections 2024-07-17T08:42:23.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.405+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1203071397 wait complete. 2024-07-17T08:42:23.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.407+0000 7f1ba8e00640 1 Processor -- start 2024-07-17T08:42:23.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.407+0000 7f1ba8e00640 1 -- start start 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 0x7f1ba41f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba4152300 0x7f1ba415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40cb6b0 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40cb530 con 0x7f1ba4151f20 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.408+0000 7f1ba8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40cb830 con 0x7f1ba4152300 2024-07-17T08:42:23.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44174/0 (socket says 172.21.15.44:44174) 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 -- 172.21.15.44:0/1846660766 learned_addr learned my addr 172.21.15.44:0/1846660766 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba2a00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 0x7f1ba41f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba4152300 0x7f1ba415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 -- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba4152300 msgr2=0x7f1ba415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba4152300 0x7f1ba415daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 -- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 msgr2=0x7f1ba41f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 0x7f1ba41f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 -- 172.21.15.44:0/1846660766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41f46e0 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba2a00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 0x7f1ba41f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.409+0000 7f1ba3400640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400c430 tx=0x7f1b9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.410+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94011790 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.410+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b94017070 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.410+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94011d00 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.410+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41f49c0 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.410+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba4154770 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.411+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4154560 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.412+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 193) ==== 99000+0+0 (secure 0 0 0) 0x7f1b9401c460 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.413+0000 7f1b9b400640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 0x7f1b84079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.414+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f1b94098c00 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.414+0000 7f1ba2a00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 0x7f1b84079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.415+0000 7f1ba2a00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 0x7f1b84079270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4155c00 tx=0x7f1b90005d90 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:23.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.418+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1b94061870 con 0x7f1ba40cd0b0 2024-07-17T08:42:23.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.661+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 --> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "mygrp04", "target": ["mon-mgr", ""]}) -- 0x7f1ba41f4480 con 0x7f1b84076db0 2024-07-17T08:42:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.674+0000 7f1b9b400640 1 -- 172.21.15.44:0/1846660766 <== mgr.9464 v2:172.21.15.175:6834/2701447474 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1b9000b6b0 con 0x7f1b84076db0 2024-07-17T08:42:23.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 msgr2=0x7f1b84079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 0x7f1b84079270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4155c00 tx=0x7f1b90005d90 comp rx=0 tx=0).stop 2024-07-17T08:42:23.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 msgr2=0x7f1ba41f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400c430 tx=0x7f1b9400c8f0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 shutdown_connections 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ba4152300 0x7f1ba415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ba4151f20 0x7f1ba41f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b84076db0 0x7f1b84079270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 --2- 172.21.15.44:0/1846660766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba40cd0b0 0x7f1ba41f7f40 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.676+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 >> 172.21.15.44:0/1846660766 conn(0x7f1ba40cf0d0 msgr2=0x7f1ba406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.677+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 shutdown_connections 2024-07-17T08:42:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.677+0000 7f1ba8e00640 1 -- 172.21.15.44:0/1846660766 wait complete. 2024-07-17T08:42:23.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:42:23.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.910+0000 7f54d3e00640 1 -- 172.21.15.44:0/297270817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d40cf690 msgr2=0x7f54d406e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.910+0000 7f54d3e00640 1 --2- 172.21.15.44:0/297270817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d40cf690 0x7f54d406e190 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f54d40cea30 tx=0x7f54bc02a590 comp rx=0 tx=0).stop 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 -- 172.21.15.44:0/297270817 shutdown_connections 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 --2- 172.21.15.44:0/297270817 >> v2:172.21.15.175:3300/0 conn(0x7f54d4154560 0x7f54d406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 --2- 172.21.15.44:0/297270817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d40cf690 0x7f54d406e190 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 -- 172.21.15.44:0/297270817 >> 172.21.15.44:0/297270817 conn(0x7f54d40b1ce0 msgr2=0x7f54d40b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 -- 172.21.15.44:0/297270817 shutdown_connections 2024-07-17T08:42:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.911+0000 7f54d3e00640 1 -- 172.21.15.44:0/297270817 wait complete. 2024-07-17T08:42:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.912+0000 7f54d3e00640 1 Processor -- start 2024-07-17T08:42:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.913+0000 7f54d3e00640 1 -- start start 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.913+0000 7f54d3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54d40cf690 0x7f54d41f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54d4154560 0x7f54d4072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f54d415dfe0 con 0x7f54d4154180 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54d415de60 con 0x7f54d40cf690 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54d415e160 con 0x7f54d4154560 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44206/0 (socket says 172.21.15.44:44206) 2024-07-17T08:42:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 -- 172.21.15.44:0/1413082607 learned_addr learned my addr 172.21.15.44:0/1413082607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2a00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54d40cf690 0x7f54d41f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d3400640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54d4154560 0x7f54d4072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 -- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54d4154560 msgr2=0x7f54d4072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54d4154560 0x7f54d4072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 -- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54d40cf690 msgr2=0x7f54d41f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54d40cf690 0x7f54d41f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.914+0000 7f54d2000640 1 -- 172.21.15.44:0/1413082607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d41f4450 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54d2000640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009850 tx=0x7f54bc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc03f070 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54bc004890 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc035030 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41f4730 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.915+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d41f4cf0 con 0x7f54d4154180 2024-07-17T08:42:23.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.916+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d415e610 con 0x7f54d4154180 2024-07-17T08:42:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.918+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 193) ==== 99000+0+0 (secure 0 0 0) 0x7f54bc042070 con 0x7f54d4154180 2024-07-17T08:42:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.918+0000 7f54cb400640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 0x7f54b00792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.919+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f54bc0b9c20 con 0x7f54d4154180 2024-07-17T08:42:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.919+0000 7f54d2a00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 0x7f54b00792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:23.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.920+0000 7f54d2a00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 0x7f54b00792a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f54d414a680 tx=0x7f54c4009040 comp rx=0 tx=0).ready entity=mgr.9464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:23.923+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f54bc082780 con 0x7f54d4154180 2024-07-17T08:42:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.186+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f54d406c6a0 con 0x7f54d4154180 2024-07-17T08:42:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.578+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v194) ==== 93+0+0 (secure 0 0 0) 0x7f54bc082080 con 0x7f54d4154180 2024-07-17T08:42:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.584+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 msgr2=0x7f54b00792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.584+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 0x7f54b00792a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f54d414a680 tx=0x7f54c4009040 comp rx=0 tx=0).stop 2024-07-17T08:42:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.584+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 msgr2=0x7f54d41f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.584+0000 7f54cb400640 1 -- 172.21.15.44:0/1413082607 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f54bc07e310 con 0x7f54d4154180 2024-07-17T08:42:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009850 tx=0x7f54bc0039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 shutdown_connections 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54d4154560 0x7f54d4072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f54b0076de0 0x7f54b00792a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54d4154180 0x7f54d41f7e00 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 --2- 172.21.15.44:0/1413082607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54d40cf690 0x7f54d41f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:24.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.585+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 >> 172.21.15.44:0/1413082607 conn(0x7f54d40b1ce0 msgr2=0x7f54d406ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.586+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 shutdown_connections 2024-07-17T08:42:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:24.586+0000 7f54d3e00640 1 -- 172.21.15.44:0/1413082607 wait complete. 2024-07-17T08:42:24.609 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:42:24.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:24.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:24.794 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:42:24.794 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:42:24.794 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:24.794 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:24.843 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:24.843 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:24.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:42:25.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.174+0000 7f1a43e00640 1 -- 172.21.15.44:0/940446722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 msgr2=0x7f1a24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.174+0000 7f1a43e00640 1 --2- 172.21.15.44:0/940446722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a24004c80 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f1a28001690 tx=0x7f1a24005830 comp rx=0 tx=0).stop 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.174+0000 7f1a43e00640 1 -- 172.21.15.44:0/940446722 shutdown_connections 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.174+0000 7f1a43e00640 1 --2- 172.21.15.44:0/940446722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a24004c80 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.174+0000 7f1a43e00640 1 -- 172.21.15.44:0/940446722 >> 172.21.15.44:0/940446722 conn(0x7f1a3c0d3700 msgr2=0x7f1a3c0d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.175+0000 7f1a43e00640 1 -- 172.21.15.44:0/940446722 shutdown_connections 2024-07-17T08:42:25.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.175+0000 7f1a43e00640 1 -- 172.21.15.44:0/940446722 wait complete. 2024-07-17T08:42:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.176+0000 7f1a43e00640 1 Processor -- start 2024-07-17T08:42:25.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.176+0000 7f1a43e00640 1 -- start start 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a3c15e0e0 0x7f1a3c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a3c1592b0 0x7f1a3c159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c0ce940 con 0x7f1a24004800 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c0ce7c0 con 0x7f1a3c1592b0 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.177+0000 7f1a43e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c0ceac0 con 0x7f1a3c15e0e0 2024-07-17T08:42:25.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44244/0 (socket says 172.21.15.44:44244) 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 -- 172.21.15.44:0/1826307948 learned_addr learned my addr 172.21.15.44:0/1826307948 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a40a00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a3c15e0e0 0x7f1a3c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a3c1592b0 0x7f1a3c159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a3c15e0e0 msgr2=0x7f1a3c15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a3c15e0e0 0x7f1a3c15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a3c1592b0 msgr2=0x7f1a3c159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a3c1592b0 0x7f1a3c159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.178+0000 7f1a41400640 1 -- 172.21.15.44:0/1826307948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c153c40 con 0x7f1a24004800 2024-07-17T08:42:25.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a41400640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e940 tx=0x7f1a3000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000cc60 con 0x7f1a24004800 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a30011470 con 0x7f1a24004800 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30011880 con 0x7f1a24004800 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c153960 con 0x7f1a24004800 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.179+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c158ab0 con 0x7f1a24004800 2024-07-17T08:42:25.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.180+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c161ef0 con 0x7f1a24004800 2024-07-17T08:42:25.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.181+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f1a30011a20 con 0x7f1a24004800 2024-07-17T08:42:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.182+0000 7f1a39600640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 0x7f1a1c079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.182+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f1a30099860 con 0x7f1a24004800 2024-07-17T08:42:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.183+0000 7f1a40a00640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 msgr2=0x7f1a1c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:25.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.183+0000 7f1a40a00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 0x7f1a1c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.186+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1a30016030 con 0x7f1a24004800 2024-07-17T08:42:25.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.384+0000 7f1a40a00640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 msgr2=0x7f1a1c079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:25.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.384+0000 7f1a40a00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 0x7f1a1c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.406+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a3c0b3d20 con 0x7f1a24004800 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.407+0000 7f1a39600640 1 -- 172.21.15.44:0/1826307948 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v322) ==== 80+0+478 (secure 0 0 0) 0x7f1a300623e0 con 0x7f1a24004800 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/3850741755 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2545797614 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3424032169 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/3850741755 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3407108431 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2271900493 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3634187181 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.409 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3817206034 2024-07-18T08:42:20.486767+0000 2024-07-17T08:42:25.409 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 msgr2=0x7f1a1c079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 0x7f1a1c079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 msgr2=0x7f1a3c152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e940 tx=0x7f1a3000ee00 comp rx=0 tx=0).stop 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 shutdown_connections 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a3c1592b0 0x7f1a3c159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a3c15e0e0 0x7f1a3c15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1a1c076e80 0x7f1a1c079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 --2- 172.21.15.44:0/1826307948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a24004800 0x7f1a3c152260 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 >> 172.21.15.44:0/1826307948 conn(0x7f1a3c0d3700 msgr2=0x7f1a3c0b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.410+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 shutdown_connections 2024-07-17T08:42:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.411+0000 7f1a43e00640 1 -- 172.21.15.44:0/1826307948 wait complete. 2024-07-17T08:42:25.431 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:42:25.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:25.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:25.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:42:25.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:25.523 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:42:25.562 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:42:25.563 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:25.563 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:25.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:25.608 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:42:25.608 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:25.608 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:25.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:42:25.658 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:42:25.658 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:25.658 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:25.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:25.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 -- 172.21.15.44:0/2886071611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 msgr2=0x7f1e4806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 --2- 172.21.15.44:0/2886071611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e4806f480 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005fd0 tx=0x7f1e3402a530 comp rx=0 tx=0).stop 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 -- 172.21.15.44:0/2886071611 shutdown_connections 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 --2- 172.21.15.44:0/2886071611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e48153e40 0x7f1e481542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 --2- 172.21.15.44:0/2886071611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e4806f480 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 -- 172.21.15.44:0/2886071611 >> 172.21.15.44:0/2886071611 conn(0x7f1e480b0fb0 msgr2=0x7f1e480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 -- 172.21.15.44:0/2886071611 shutdown_connections 2024-07-17T08:42:25.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.905+0000 7f1e50000640 1 -- 172.21.15.44:0/2886071611 wait complete. 2024-07-17T08:42:25.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.908+0000 7f1e50000640 1 Processor -- start 2024-07-17T08:42:25.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.908+0000 7f1e50000640 1 -- start start 2024-07-17T08:42:25.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 0x7f1e48156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 0x7f1e481574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48154810 con 0x7f1e4806f020 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4815b8d0 con 0x7f1e48153e40 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e50000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e48154990 con 0x7f1e4815b2e0 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44264/0 (socket says 172.21.15.44:44264) 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e4d600640 1 -- 172.21.15.44:0/4190901068 learned_addr learned my addr 172.21.15.44:0/4190901068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.910+0000 7f1e4cc00640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 0x7f1e48156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4e000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 0x7f1e481574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 msgr2=0x7f1e481574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 0x7f1e481574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 msgr2=0x7f1e48156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 0x7f1e48156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 -- 172.21.15.44:0/4190901068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481f2f00 con 0x7f1e4806f020 2024-07-17T08:42:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4e000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 0x7f1e481574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4cc00640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 0x7f1e48156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:42:25.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e4d600640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00ba50 tx=0x7f1e3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c019070 con 0x7f1e4806f020 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3c00cd10 con 0x7f1e4806f020 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e480cde20 con 0x7f1e4806f020 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0146a0 con 0x7f1e4806f020 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.911+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e480ce3e0 con 0x7f1e4806f020 2024-07-17T08:42:25.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.913+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48153640 con 0x7f1e4806f020 2024-07-17T08:42:25.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.915+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f1e3c01c070 con 0x7f1e4806f020 2024-07-17T08:42:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.916+0000 7f1e45600640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 0x7f1e28079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.916+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f1e3c098920 con 0x7f1e4806f020 2024-07-17T08:42:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.916+0000 7f1e4cc00640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 msgr2=0x7f1e28079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:25.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.916+0000 7f1e4cc00640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 0x7f1e28079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:25.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:25.920+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e48153640 con 0x7f1e4806f020 2024-07-17T08:42:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.118+0000 7f1e4cc00640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 msgr2=0x7f1e28079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.118+0000 7f1e4cc00640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 0x7f1e28079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:26.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.173+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e4806f480 con 0x7f1e4806f020 2024-07-17T08:42:26.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.175+0000 7f1e45600640 1 -- 172.21.15.44:0/4190901068 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7f1e3c060f60 con 0x7f1e4806f020 2024-07-17T08:42:26.177 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:26.177 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:26.177 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:26.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 msgr2=0x7f1e28079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 0x7f1e28079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 msgr2=0x7f1e481f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00ba50 tx=0x7f1e3c00bf10 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 shutdown_connections 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e4815b2e0 0x7f1e481574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e48153e40 0x7f1e48156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1e28076e80 0x7f1e28079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 --2- 172.21.15.44:0/4190901068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e4806f020 0x7f1e481f29c0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.178+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 >> 172.21.15.44:0/4190901068 conn(0x7f1e480b0fb0 msgr2=0x7f1e4814aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.179+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 shutdown_connections 2024-07-17T08:42:26.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.179+0000 7f1e50000640 1 -- 172.21.15.44:0/4190901068 wait complete. 2024-07-17T08:42:26.197 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 -- 172.21.15.44:0/1950121089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0ce5c0 msgr2=0x7f140c06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 --2- 172.21.15.44:0/1950121089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0ce5c0 0x7f140c06d9b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009900 tx=0x7f13f402a610 comp rx=0 tx=0).stop 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 -- 172.21.15.44:0/1950121089 shutdown_connections 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 --2- 172.21.15.44:0/1950121089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0ce5c0 0x7f140c06d9b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 --2- 172.21.15.44:0/1950121089 >> v2:172.21.15.175:3300/0 conn(0x7f140c0b8c80 0x7f140c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 -- 172.21.15.44:0/1950121089 >> 172.21.15.44:0/1950121089 conn(0x7f140c0cfec0 msgr2=0x7f140c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 -- 172.21.15.44:0/1950121089 shutdown_connections 2024-07-17T08:42:26.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.388+0000 7f1411000640 1 -- 172.21.15.44:0/1950121089 wait complete. 2024-07-17T08:42:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.389+0000 7f1411000640 1 Processor -- start 2024-07-17T08:42:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.390+0000 7f1411000640 1 -- start start 2024-07-17T08:42:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.390+0000 7f1411000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.390+0000 7f1411000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0b8c80 0x7f140c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f1411000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c0ce5c0 0x7f140c0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f1411000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f140c06e780 con 0x7f140c0b5400 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f1411000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f140c06e600 con 0x7f140c0ce5c0 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f1411000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f140c06e900 con 0x7f140c0b8c80 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44270/0 (socket says 172.21.15.44:44270) 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 -- 172.21.15.44:0/3090431346 learned_addr learned my addr 172.21.15.44:0/3090431346 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f1403e00640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0b8c80 0x7f140c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140be00640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c0ce5c0 0x7f140c0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0b8c80 msgr2=0x7f140c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0b8c80 0x7f140c159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c0ce5c0 msgr2=0x7f140c0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c0ce5c0 0x7f140c0cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.391+0000 7f140b400640 1 -- 172.21.15.44:0/3090431346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c1f3f30 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f140b400640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00ea90 tx=0x7f13fc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00ce40 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13fc004020 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c1f41f0 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc004470 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.392+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c1f47b0 con 0x7f140c0b5400 2024-07-17T08:42:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.394+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140c0b5d70 con 0x7f140c0b5400 2024-07-17T08:42:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.395+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f13fc0035f0 con 0x7f140c0b5400 2024-07-17T08:42:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.396+0000 7f1403400640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 0x7f13e8079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.396+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f13fc0996f0 con 0x7f140c0b5400 2024-07-17T08:42:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.396+0000 7f1403e00640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 msgr2=0x7f13e8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.397+0000 7f1403e00640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 0x7f13e8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:26.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.401+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f13fc0621e0 con 0x7f140c0b5400 2024-07-17T08:42:26.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.598+0000 7f1403e00640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 msgr2=0x7f13e8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:26.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.598+0000 7f1403e00640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 0x7f13e8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:26.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.665+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f140c0d04f0 con 0x7f140c0b5400 2024-07-17T08:42:26.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.666+0000 7f1403400640 1 -- 172.21.15.44:0/3090431346 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7f13fc061ae0 con 0x7f140c0b5400 2024-07-17T08:42:26.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:26.668 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:26.668 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 msgr2=0x7f13e8079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 0x7f13e8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 msgr2=0x7f140c15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00ea90 tx=0x7f13fc00ef50 comp rx=0 tx=0).stop 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 shutdown_connections 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f140c0ce5c0 0x7f140c0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f140c0b8c80 0x7f140c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f13e8076e80 0x7f13e8079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 --2- 172.21.15.44:0/3090431346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f140c0b5400 0x7f140c15aa30 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.669+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 >> 172.21.15.44:0/3090431346 conn(0x7f140c0cfec0 msgr2=0x7f140c15f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:26.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.670+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 shutdown_connections 2024-07-17T08:42:26.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.670+0000 7f1411000640 1 -- 172.21.15.44:0/3090431346 wait complete. 2024-07-17T08:42:26.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 -- 172.21.15.44:0/1862688408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f40b1330 msgr2=0x7f59f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 --2- 172.21.15.44:0/1862688408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f40b1330 0x7f59f406c920 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f59f40b1db0 tx=0x7f59dc02a540 comp rx=0 tx=0).stop 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 -- 172.21.15.44:0/1862688408 shutdown_connections 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 --2- 172.21.15.44:0/1862688408 >> v2:172.21.15.175:3300/0 conn(0x7f59f4072a10 0x7f59f40af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 --2- 172.21.15.44:0/1862688408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f40b1330 0x7f59f406c920 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 -- 172.21.15.44:0/1862688408 >> 172.21.15.44:0/1862688408 conn(0x7f59f40b5890 msgr2=0x7f59f40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 -- 172.21.15.44:0/1862688408 shutdown_connections 2024-07-17T08:42:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.877+0000 7f59f9a00640 1 -- 172.21.15.44:0/1862688408 wait complete. 2024-07-17T08:42:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.878+0000 7f59f9a00640 1 Processor -- start 2024-07-17T08:42:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.879+0000 7f59f9a00640 1 -- start start 2024-07-17T08:42:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.879+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f4072a10 0x7f59f415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.879+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40b1330 0x7f59f414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.879+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f59f40afcc0 con 0x7f59f415d620 2024-07-17T08:42:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59f40afb40 con 0x7f59f4072a10 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59f40afe40 con 0x7f59f40b1330 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44290/0 (socket says 172.21.15.44:44290) 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 -- 172.21.15.44:0/200205832 learned_addr learned my addr 172.21.15.44:0/200205832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f3e00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f4072a10 0x7f59f415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59eb400640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40b1330 0x7f59f414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40b1330 msgr2=0x7f59f414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40b1330 0x7f59f414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f4072a10 msgr2=0x7f59f415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f4072a10 0x7f59f415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.880+0000 7f59f9000640 1 -- 172.21.15.44:0/200205832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f414d8d0 con 0x7f59f415d620 2024-07-17T08:42:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f9000640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00cb00 tx=0x7f59ec00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec004b20 con 0x7f59f415d620 2024-07-17T08:42:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59ec004db0 con 0x7f59f415d620 2024-07-17T08:42:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec005280 con 0x7f59f415d620 2024-07-17T08:42:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f414dbb0 con 0x7f59f415d620 2024-07-17T08:42:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.881+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f414e170 con 0x7f59f415d620 2024-07-17T08:42:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.882+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f40b03d0 con 0x7f59f415d620 2024-07-17T08:42:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.884+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f59ec005420 con 0x7f59f415d620 2024-07-17T08:42:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.885+0000 7f59f0c00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 0x7f59d0079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.885+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f59ec0995c0 con 0x7f59f415d620 2024-07-17T08:42:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.886+0000 7f59f3e00640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 msgr2=0x7f59d0079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.886+0000 7f59f3e00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 0x7f59d0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:26.890+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f59ec014030 con 0x7f59f415d620 2024-07-17T08:42:27.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.087+0000 7f59f3e00640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 msgr2=0x7f59d0079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:27.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.087+0000 7f59f3e00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 0x7f59d0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:27.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.095+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59f4074ee0 con 0x7f59f415d620 2024-07-17T08:42:27.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.097+0000 7f59f0c00640 1 -- 172.21.15.44:0/200205832 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v322) ==== 74+0+18315 (secure 0 0 0) 0x7f59ec0620c0 con 0x7f59f415d620 2024-07-17T08:42:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:27.099 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":322,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:20.486850+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":43,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:04.096300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":44,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:05.114416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"321","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6834/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2545797614":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3424032169":"2024-07-18T08:42:20.486767+0000","172.21.15.175:6837/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3407108431":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2271900493":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3634187181":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3817206034":"2024-07-18T08:42:20.486767+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 msgr2=0x7f59d0079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 0x7f59d0079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 msgr2=0x7f59f414b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00cb00 tx=0x7f59ec00a250 comp rx=0 tx=0).stop 2024-07-17T08:42:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 shutdown_connections 2024-07-17T08:42:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f59d0076e80 0x7f59d0079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59f415d620 0x7f59f414b7d0 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40b1330 0x7f59f414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 --2- 172.21.15.44:0/200205832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f4072a10 0x7f59f415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.100+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 >> 172.21.15.44:0/200205832 conn(0x7f59f40b5890 msgr2=0x7f59f406c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.101+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 shutdown_connections 2024-07-17T08:42:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.101+0000 7f59f9a00640 1 -- 172.21.15.44:0/200205832 wait complete. 2024-07-17T08:42:27.121 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:42:27.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 -- 172.21.15.44:0/3143571376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b5400 msgr2=0x7f8f6815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 --2- 172.21.15.44:0/3143571376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b5400 0x7f8f6815e2d0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009860 tx=0x7f8f6002a540 comp rx=0 tx=0).stop 2024-07-17T08:42:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 -- 172.21.15.44:0/3143571376 shutdown_connections 2024-07-17T08:42:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 --2- 172.21.15.44:0/3143571376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b5400 0x7f8f6815e2d0 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 --2- 172.21.15.44:0/3143571376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680b8c80 0x7f8f680b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 -- 172.21.15.44:0/3143571376 >> 172.21.15.44:0/3143571376 conn(0x7f8f680cfec0 msgr2=0x7f8f680d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:27.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 -- 172.21.15.44:0/3143571376 shutdown_connections 2024-07-17T08:42:27.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.314+0000 7f8f6e000640 1 -- 172.21.15.44:0/3143571376 wait complete. 2024-07-17T08:42:27.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.316+0000 7f8f6e000640 1 Processor -- start 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.316+0000 7f8f6e000640 1 -- start start 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f680b5400 0x7f8f680b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680ce5c0 0x7f8f680b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6806e0c0 con 0x7f8f680b8c80 2024-07-17T08:42:27.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680b92f0 con 0x7f8f680b5400 2024-07-17T08:42:27.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f6e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6806e240 con 0x7f8f680ce5c0 2024-07-17T08:42:27.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44294/0 (socket says 172.21.15.44:44294) 2024-07-17T08:42:27.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.317+0000 7f8f67e00640 1 -- 172.21.15.44:0/726493887 learned_addr learned my addr 172.21.15.44:0/726493887 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:27.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f6d600640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680ce5c0 0x7f8f680b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f6cc00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f680b5400 0x7f8f680b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680ce5c0 msgr2=0x7f8f680b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680ce5c0 0x7f8f680b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f680b5400 msgr2=0x7f8f680b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f680b5400 0x7f8f680b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 -- 172.21.15.44:0/726493887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f680b1d10 con 0x7f8f680b8c80 2024-07-17T08:42:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f67e00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400cb40 tx=0x7f8f5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54016020 con 0x7f8f680b8c80 2024-07-17T08:42:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f5400adb0 con 0x7f8f680b8c80 2024-07-17T08:42:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54012680 con 0x7f8f680b8c80 2024-07-17T08:42:27.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.318+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f681f4310 con 0x7f8f680b8c80 2024-07-17T08:42:27.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.319+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681f4c00 con 0x7f8f680b8c80 2024-07-17T08:42:27.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.320+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f681f4830 con 0x7f8f680b8c80 2024-07-17T08:42:27.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.321+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f8f54019020 con 0x7f8f680b8c80 2024-07-17T08:42:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.322+0000 7f8f64c00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 0x7f8f48079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.322+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f8f540994a0 con 0x7f8f680b8c80 2024-07-17T08:42:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.322+0000 7f8f6cc00640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 msgr2=0x7f8f48079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.322+0000 7f8f6cc00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 0x7f8f48079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:27.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.326+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8f681f4830 con 0x7f8f680b8c80 2024-07-17T08:42:27.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.524+0000 7f8f6cc00640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 msgr2=0x7f8f48079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:27.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.524+0000 7f8f6cc00640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 0x7f8f48079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.563+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f6806e6f0 con 0x7f8f680b8c80 2024-07-17T08:42:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.564+0000 7f8f64c00640 1 -- 172.21.15.44:0/726493887 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7f8f54020020 con 0x7f8f680b8c80 2024-07-17T08:42:27.565 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:27.565 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 msgr2=0x7f8f48079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 0x7f8f48079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 msgr2=0x7f8f681f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400cb40 tx=0x7f8f5400a060 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 shutdown_connections 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f680ce5c0 0x7f8f680b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f8f48076e80 0x7f8f48079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f680b8c80 0x7f8f681f38e0 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 --2- 172.21.15.44:0/726493887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f680b5400 0x7f8f680b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 >> 172.21.15.44:0/726493887 conn(0x7f8f680cfec0 msgr2=0x7f8f680b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 shutdown_connections 2024-07-17T08:42:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.567+0000 7f8f6e000640 1 -- 172.21.15.44:0/726493887 wait complete. 2024-07-17T08:42:27.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 -- 172.21.15.44:0/1886257650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b8c80 msgr2=0x7fdd280b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 --2- 172.21.15.44:0/1886257650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b8c80 0x7fdd280b4ec0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009900 tx=0x7fdd1802a610 comp rx=0 tx=0).stop 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 -- 172.21.15.44:0/1886257650 shutdown_connections 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 --2- 172.21.15.44:0/1886257650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b8c80 0x7fdd280b4ec0 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 -- 172.21.15.44:0/1886257650 >> 172.21.15.44:0/1886257650 conn(0x7fdd280cfec0 msgr2=0x7fdd280d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 -- 172.21.15.44:0/1886257650 shutdown_connections 2024-07-17T08:42:27.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.770+0000 7fdd27e00640 1 -- 172.21.15.44:0/1886257650 wait complete. 2024-07-17T08:42:27.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.772+0000 7fdd27e00640 1 Processor -- start 2024-07-17T08:42:27.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.772+0000 7fdd27e00640 1 -- start start 2024-07-17T08:42:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 0x7fdd280cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd280ce5c0 0x7fdd280c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2815e2d0 con 0x7fdd280b5400 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280b9270 con 0x7fdd280b8c80 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2815e450 con 0x7fdd280ce5c0 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44302/0 (socket says 172.21.15.44:44302) 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 -- 172.21.15.44:0/3897775158 learned_addr learned my addr 172.21.15.44:0/3897775158 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd1fe00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 0x7fdd280cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd27400640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd280ce5c0 0x7fdd280c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd280ce5c0 msgr2=0x7fdd280c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd280ce5c0 0x7fdd280c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 msgr2=0x7fdd280cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.773+0000 7fdd26a00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 0x7fdd280cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd26a00640 1 -- 172.21.15.44:0/3897775158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd280ca510 con 0x7fdd280b5400 2024-07-17T08:42:27.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd1fe00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 0x7fdd280cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:42:27.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd26a00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7fdd18006010 tx=0x7fdd180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:27.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1803f070 con 0x7fdd280b5400 2024-07-17T08:42:27.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd180047a0 con 0x7fdd280b5400 2024-07-17T08:42:27.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18035030 con 0x7fdd280b5400 2024-07-17T08:42:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd280ca7d0 con 0x7fdd280b5400 2024-07-17T08:42:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.774+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd280c70f0 con 0x7fdd280b5400 2024-07-17T08:42:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.776+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd280b4ec0 con 0x7fdd280b5400 2024-07-17T08:42:27.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.777+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7fdd18042070 con 0x7fdd280b5400 2024-07-17T08:42:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.778+0000 7fdd1ea00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 0x7fdd04079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:27.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.778+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7fdd180b9a50 con 0x7fdd280b5400 2024-07-17T08:42:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.778+0000 7fdd1fe00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 msgr2=0x7fdd04079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:27.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.778+0000 7fdd1fe00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 0x7fdd04079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:27.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.782+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdd18082580 con 0x7fdd280b5400 2024-07-17T08:42:27.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.980+0000 7fdd1fe00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 msgr2=0x7fdd04079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:27.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:27.980+0000 7fdd1fe00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 0x7fdd04079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:28.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.031+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd280b5d70 con 0x7fdd280b5400 2024-07-17T08:42:28.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.032+0000 7fdd1ea00640 1 -- 172.21.15.44:0/3897775158 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5242 (secure 0 0 0) 0x7fdd18081e80 con 0x7fdd280b5400 2024-07-17T08:42:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:28.033 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":181,"btime":"2024-07-17T08:42:09:702854+0000","default_fscid":21,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:06.120454+0000","modified":"2024-07-17T08:42:09.223219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10432":{"gid":10432,"name":"d","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1754084600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1754084600},{"type":"v1","addr":"172.21.15.175:6801","nonce":1754084600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[44],"metadata_pool":43,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10432,"qdb_cluster":[10432]},"id":21}]} 2024-07-17T08:42:28.033 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 181 2024-07-17T08:42:28.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 msgr2=0x7fdd04079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:28.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 0x7fdd04079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 msgr2=0x7fdd280c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7fdd18006010 tx=0x7fdd180038a0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 shutdown_connections 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd280ce5c0 0x7fdd280c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd280b8c80 0x7fdd280cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fdd04076eb0 0x7fdd04079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 --2- 172.21.15.44:0/3897775158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd280b5400 0x7fdd280c9630 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 >> 172.21.15.44:0/3897775158 conn(0x7fdd280cfec0 msgr2=0x7fdd2815ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 shutdown_connections 2024-07-17T08:42:28.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.035+0000 7fdd27e00640 1 -- 172.21.15.44:0/3897775158 wait complete. 2024-07-17T08:42:28.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:28.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 -- 172.21.15.44:0/1137518980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f0005210 msgr2=0x7fc5f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 --2- 172.21.15.44:0/1137518980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f0005210 0x7fc5f000b9c0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc5f0005830 comp rx=0 tx=0).stop 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 -- 172.21.15.44:0/1137518980 shutdown_connections 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 --2- 172.21.15.44:0/1137518980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5f0005210 0x7fc5f000b9c0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 -- 172.21.15.44:0/1137518980 >> 172.21.15.44:0/1137518980 conn(0x7fc60806bf40 msgr2=0x7fc60806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.255+0000 7fc610800640 1 -- 172.21.15.44:0/1137518980 shutdown_connections 2024-07-17T08:42:28.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.256+0000 7fc610800640 1 -- 172.21.15.44:0/1137518980 wait complete. 2024-07-17T08:42:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.258+0000 7fc610800640 1 Processor -- start 2024-07-17T08:42:28.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.258+0000 7fc610800640 1 -- start start 2024-07-17T08:42:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 0x7fc608157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 0x7fc6081488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080cf140 con 0x7fc6080b3870 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080cefc0 con 0x7fc5f0005210 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc610800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6080cf2c0 con 0x7fc608148470 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc60e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc60e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44314/0 (socket says 172.21.15.44:44314) 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.260+0000 7fc60e800640 1 -- 172.21.15.44:0/1830157778 learned_addr learned my addr 172.21.15.44:0/1830157778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60de00640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 0x7fc608157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60d400640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 0x7fc6081488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 msgr2=0x7fc6081488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 0x7fc6081488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 msgr2=0x7fc608157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 0x7fc608157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 -- 172.21.15.44:0/1830157778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc608148e30 con 0x7fc6080b3870 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60d400640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 0x7fc6081488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60de00640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 0x7fc608157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc60e800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7fc60402b350 tx=0x7fc604001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604039070 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.261+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc604006680 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.262+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6080b4230 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.262+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604006d60 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.262+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6080b47d0 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.263+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60815efc0 con 0x7fc6080b3870 2024-07-17T08:42:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.264+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7fc60403c020 con 0x7fc6080b3870 2024-07-17T08:42:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.265+0000 7fc602000640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 0x7fc5ec079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.266+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7fc6040b8b60 con 0x7fc6080b3870 2024-07-17T08:42:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.266+0000 7fc60de00640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 msgr2=0x7fc5ec079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.266+0000 7fc60de00640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 0x7fc5ec079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:28.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.269+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc604081760 con 0x7fc6080b3870 2024-07-17T08:42:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.467+0000 7fc60de00640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 msgr2=0x7fc5ec079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.467+0000 7fc60de00640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 0x7fc5ec079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:28.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.478+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc608073c70 con 0x7fc6080b3870 2024-07-17T08:42:28.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.481+0000 7fc602000640 1 -- 172.21.15.44:0/1830157778 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v322) ==== 74+0+18315 (secure 0 0 0) 0x7fc604081060 con 0x7fc6080b3870 2024-07-17T08:42:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:28.482 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":322,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:20.486850+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":43,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:04.096300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"320","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":44,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:05.114416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"321","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6834/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2545797614":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3424032169":"2024-07-18T08:42:20.486767+0000","172.21.15.175:6837/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3407108431":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2271900493":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3634187181":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3817206034":"2024-07-18T08:42:20.486767+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 msgr2=0x7fc5ec079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:28.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 0x7fc5ec079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 msgr2=0x7fc6080b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7fc60402b350 tx=0x7fc604001b20 comp rx=0 tx=0).stop 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 shutdown_connections 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7fc5ec076e80 0x7fc5ec079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6080b3870 0x7fc6080b3cf0 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc608148470 0x7fc6081488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 --2- 172.21.15.44:0/1830157778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5f0005210 0x7fc608157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.483+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 >> 172.21.15.44:0/1830157778 conn(0x7fc60806bf40 msgr2=0x7fc6080b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.484+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 shutdown_connections 2024-07-17T08:42:28.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.484+0000 7fc610800640 1 -- 172.21.15.44:0/1830157778 wait complete. 2024-07-17T08:42:28.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:42:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 -- 172.21.15.44:0/784070645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b6815b720 msgr2=0x7f1b680ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 --2- 172.21.15.44:0/784070645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b6815b720 0x7f1b680ae390 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009830 tx=0x7f1b5402a5a0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 -- 172.21.15.44:0/784070645 shutdown_connections 2024-07-17T08:42:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 --2- 172.21.15.44:0/784070645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b6815b720 0x7f1b680ae390 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 --2- 172.21.15.44:0/784070645 >> v2:172.21.15.175:3300/0 conn(0x7f1b6815b340 0x7f1b680ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 -- 172.21.15.44:0/784070645 >> 172.21.15.44:0/784070645 conn(0x7f1b6809aa30 msgr2=0x7f1b680b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 -- 172.21.15.44:0/784070645 shutdown_connections 2024-07-17T08:42:28.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.708+0000 7f1b6d400640 1 -- 172.21.15.44:0/784070645 wait complete. 2024-07-17T08:42:28.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.710+0000 7f1b6d400640 1 Processor -- start 2024-07-17T08:42:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.710+0000 7f1b6d400640 1 -- start start 2024-07-17T08:42:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 0x7f1b681f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 0x7f1b68157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680cf2e0 con 0x7f1b6815b720 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680cd850 con 0x7f1b6815b340 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b6d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b680cf460 con 0x7f1b680ae8d0 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b67200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b67200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44318/0 (socket says 172.21.15.44:44318) 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b67200640 1 -- 172.21.15.44:0/3255635320 learned_addr learned my addr 172.21.15.44:0/3255635320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:28.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.711+0000 7f1b66800640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 0x7f1b681f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b65e00640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 0x7f1b68157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 -- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 msgr2=0x7f1b681f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 0x7f1b681f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 -- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 msgr2=0x7f1b68157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 0x7f1b68157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:28.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 -- 172.21.15.44:0/3255635320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b680b7100 con 0x7f1b6815b720 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b66800640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 0x7f1b681f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b65e00640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 0x7f1b68157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b67200640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c770 tx=0x7f1b6000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003b70 con 0x7f1b6815b720 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b60003d10 con 0x7f1b6815b720 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b68152890 con 0x7f1b6815b720 2024-07-17T08:42:28.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000aab0 con 0x7f1b6815b720 2024-07-17T08:42:28.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.712+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b681f8430 con 0x7f1b6815b720 2024-07-17T08:42:28.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.714+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b680cf910 con 0x7f1b6815b720 2024-07-17T08:42:28.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.715+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 194) ==== 99090+0+0 (secure 0 0 0) 0x7f1b6000ac50 con 0x7f1b6815b720 2024-07-17T08:42:28.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.716+0000 7f1b5ea00640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 0x7f1b48079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:28.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.716+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(322..322 src has 1..322) ==== 7049+0+0 (secure 0 0 0) 0x7f1b60098b50 con 0x7f1b6815b720 2024-07-17T08:42:28.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.716+0000 7f1b66800640 1 -- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 msgr2=0x7f1b48079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2701447474 2024-07-17T08:42:28.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.716+0000 7f1b66800640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 0x7f1b48079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:28.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.721+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1b680b7100 con 0x7f1b6815b720 2024-07-17T08:42:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.891+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 195) ==== 98540+0+0 (secure 0 0 0) 0x7f1b6005d2e0 con 0x7f1b6815b720 2024-07-17T08:42:28.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.891+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 msgr2=0x7f1b48079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:28.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:28.891+0000 7f1b5ea00640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 0x7f1b48079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.008+0000 7f8e64a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.008+0000 7f8e64a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.008+0000 7f8e64a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.008+0000 7f8e64a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.008+0000 7f8e64a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.009+0000 7f8e67c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.010 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.009+0000 7f8e67c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.011 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.009+0000 7f8e67c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.011 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.009+0000 7f8e67c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.011 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:42:29.009+0000 7f8e67c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:42:29.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.021+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1b680af4f0 con 0x7f1b6815b720 2024-07-17T08:42:29.022 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:42:29.022+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:42:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.024+0000 7f1b5ea00640 1 -- 172.21.15.44:0/3255635320 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v182) ==== 161+0+0 (secure 0 0 0) 0x7f1b60061750 con 0x7f1b6815b720 2024-07-17T08:42:29.024 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.026+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 msgr2=0x7f1b680b6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.026+0000 7f1b6d400640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c770 tx=0x7f1b6000cc30 comp rx=0 tx=0).stop 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 shutdown_connections 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:6834/2701447474,v1:172.21.15.175:6837/2701447474] conn(0x7f1b48076e80 0x7f1b48079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b6815b720 0x7f1b680b6bc0 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6815b340 0x7f1b68157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 --2- 172.21.15.44:0/3255635320 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b680ae8d0 0x7f1b681f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 >> 172.21.15.44:0/3255635320 conn(0x7f1b6809aa30 msgr2=0x7f1b680aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 shutdown_connections 2024-07-17T08:42:29.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.027+0000 7f1b6d400640 1 -- 172.21.15.44:0/3255635320 wait complete. 2024-07-17T08:42:29.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:42:29.089 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:42:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 -- 172.21.15.44:0/1334726253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 msgr2=0x7ff58006eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 --2- 172.21.15.44:0/1334726253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff58006eb90 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7ff580157540 tx=0x7ff56802a560 comp rx=0 tx=0).stop 2024-07-17T08:42:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 -- 172.21.15.44:0/1334726253 shutdown_connections 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 --2- 172.21.15.44:0/1334726253 >> v2:172.21.15.175:3300/0 conn(0x7ff5800ac3a0 0x7ff5800ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 --2- 172.21.15.44:0/1334726253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff58006eb90 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 --2- 172.21.15.44:0/1334726253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff58015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 -- 172.21.15.44:0/1334726253 >> 172.21.15.44:0/1334726253 conn(0x7ff5800b0eb0 msgr2=0x7ff5800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 -- 172.21.15.44:0/1334726253 shutdown_connections 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.223+0000 7ff585c00640 1 -- 172.21.15.44:0/1334726253 wait complete. 2024-07-17T08:42:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.225+0000 7ff585c00640 1 Processor -- start 2024-07-17T08:42:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.225+0000 7ff585c00640 1 -- start start 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff5800cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5800ac3a0 0x7ff5800cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800ae8f0 con 0x7ff5800b3280 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800ae770 con 0x7ff5800ac3a0 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff585c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff5800aea70 con 0x7ff58006f6d0 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff57f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff57f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33722/0 (socket says 172.21.15.44:33722) 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff57f400640 1 -- 172.21.15.44:0/3688090358 learned_addr learned my addr 172.21.15.44:0/3688090358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.226+0000 7ff57ea00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff5800cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff577e00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5800ac3a0 0x7ff5800cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 -- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 msgr2=0x7ff5800cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff5800cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 -- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5800ac3a0 msgr2=0x7ff5800cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5800ac3a0 0x7ff5800cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 -- 172.21.15.44:0/3688090358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5800ce050 con 0x7ff5800b3280 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57ea00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff5800cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff57f400640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7ff57800cd10 tx=0x7ff57800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800f450 con 0x7ff5800b3280 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff57800f6e0 con 0x7ff5800b3280 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800fb30 con 0x7ff5800b3280 2024-07-17T08:42:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.227+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5800ce330 con 0x7ff5800b3280 2024-07-17T08:42:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.228+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5800ce8f0 con 0x7ff5800b3280 2024-07-17T08:42:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.229+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5800ceb30 con 0x7ff5800b3280 2024-07-17T08:42:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.230+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 195) ==== 98540+0+0 (secure 0 0 0) 0x7ff57800fcd0 con 0x7ff5800b3280 2024-07-17T08:42:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.231+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(324..324 src has 1..324) ==== 7608+0+0 (secure 0 0 0) 0x7ff5780990a0 con 0x7ff5800b3280 2024-07-17T08:42:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.235+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff5800ceb30 con 0x7ff5800b3280 2024-07-17T08:42:29.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.484+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff58006fef0 con 0x7ff5800b3280 2024-07-17T08:42:29.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.899+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 196) ==== 98667+0+0 (secure 0 0 0) 0x7ff57805d3d0 con 0x7ff5800b3280 2024-07-17T08:42:29.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.900+0000 7ff576a00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 0x7ff55c07f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:29.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.901+0000 7ff57ea00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 0x7ff55c07f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:29.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:29.904+0000 7ff57ea00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 0x7ff55c07f690 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff570009b40 tx=0x7ff570005e50 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:30.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.027+0000 7ff576a00640 1 -- 172.21.15.44:0/3688090358 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v183) ==== 104+0+0 (secure 0 0 0) 0x7ff58006fef0 con 0x7ff5800b3280 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.031+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 msgr2=0x7ff55c07f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.031+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 0x7ff55c07f690 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff570009b40 tx=0x7ff570005e50 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.031+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 msgr2=0x7ff5800719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.031+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7ff57800cd10 tx=0x7ff57800a620 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 shutdown_connections 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff55c07d2a0 0x7ff55c07f690 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5800b3280 0x7ff5800719c0 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5800ac3a0 0x7ff5800cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 --2- 172.21.15.44:0/3688090358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff58006f6d0 0x7ff5800cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 >> 172.21.15.44:0/3688090358 conn(0x7ff5800b0eb0 msgr2=0x7ff5800afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 shutdown_connections 2024-07-17T08:42:30.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.032+0000 7ff585c00640 1 -- 172.21.15.44:0/3688090358 wait complete. 2024-07-17T08:42:30.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:42:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.272+0000 7f2c42600640 1 -- 172.21.15.44:0/878502704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c200052e0 msgr2=0x7f2c2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:30.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.272+0000 7f2c42600640 1 --2- 172.21.15.44:0/878502704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c200052e0 0x7f2c2000b9c0 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fd0 tx=0x7f2c20005830 comp rx=0 tx=0).stop 2024-07-17T08:42:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.272+0000 7f2c42600640 1 -- 172.21.15.44:0/878502704 shutdown_connections 2024-07-17T08:42:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.272+0000 7f2c42600640 1 --2- 172.21.15.44:0/878502704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c200052e0 0x7f2c2000b9c0 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.272+0000 7f2c42600640 1 -- 172.21.15.44:0/878502704 >> 172.21.15.44:0/878502704 conn(0x7f2c3c0b0eb0 msgr2=0x7f2c3c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.273+0000 7f2c42600640 1 -- 172.21.15.44:0/878502704 shutdown_connections 2024-07-17T08:42:30.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.273+0000 7f2c42600640 1 -- 172.21.15.44:0/878502704 wait complete. 2024-07-17T08:42:30.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.275+0000 7f2c42600640 1 Processor -- start 2024-07-17T08:42:30.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.275+0000 7f2c42600640 1 -- start start 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c200052e0 0x7f2c3c09cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3c1f7fa0 0x7f2c3c09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0ac3a0 con 0x7f2c3c142400 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0ac520 con 0x7f2c200052e0 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.276+0000 7f2c42600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c0ac6a0 con 0x7f2c3c1f7fa0 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33734/0 (socket says 172.21.15.44:33734) 2024-07-17T08:42:30.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 -- 172.21.15.44:0/2515768446 learned_addr learned my addr 172.21.15.44:0/2515768446 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3b400640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c200052e0 0x7f2c3c09cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3aa00640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3c1f7fa0 0x7f2c3c09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 -- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3c1f7fa0 msgr2=0x7f2c3c09c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3c1f7fa0 0x7f2c3c09c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 -- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c200052e0 msgr2=0x7f2c3c09cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c200052e0 0x7f2c3c09cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:30.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.277+0000 7f2c3be00640 1 -- 172.21.15.44:0/2515768446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c09a6b0 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c3be00640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fd0 tx=0x7f2c34004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34044070 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c340052f0 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340071c0 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c154170 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.278+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c09a970 con 0x7f2c3c142400 2024-07-17T08:42:30.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.280+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c06fef0 con 0x7f2c3c142400 2024-07-17T08:42:30.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.282+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 196) ==== 98667+0+0 (secure 0 0 0) 0x7f2c34005700 con 0x7f2c3c142400 2024-07-17T08:42:30.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.284+0000 7f2c33400640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 0x7f2c1c078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:30.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.284+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(324..324 src has 1..324) ==== 7608+0+0 (secure 0 0 0) 0x7f2c340b8450 con 0x7f2c3c142400 2024-07-17T08:42:30.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.284+0000 7f2c3b400640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 0x7f2c1c078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:30.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.285+0000 7f2c3b400640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 0x7f2c1c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001860 tx=0x7f2c2c000ed0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:30.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.289+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2c3c06fef0 con 0x7f2c3c142400 2024-07-17T08:42:30.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:30.507+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c3c06d790 con 0x7f2c3c142400 2024-07-17T08:42:31.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.043+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 197) ==== 98875+0+0 (secure 0 0 0) 0x7f2c3407cfb0 con 0x7f2c3c142400 2024-07-17T08:42:31.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.049+0000 7f2c33400640 1 -- 172.21.15.44:0/2515768446 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v325) ==== 181+0+0 (secure 0 0 0) 0x7f2c340059f0 con 0x7f2c3c142400 2024-07-17T08:42:31.050 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:42:31.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.054+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 msgr2=0x7f2c1c078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:31.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.054+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 0x7f2c1c078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001860 tx=0x7f2c2c000ed0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.054+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 msgr2=0x7f2c3c1428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:31.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.054+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fd0 tx=0x7f2c34004f90 comp rx=0 tx=0).stop 2024-07-17T08:42:31.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 shutdown_connections 2024-07-17T08:42:31.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2c1c076a00 0x7f2c1c078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c3c142400 0x7f2c3c1428b0 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c3c1f7fa0 0x7f2c3c09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 --2- 172.21.15.44:0/2515768446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c200052e0 0x7f2c3c09cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.056+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 >> 172.21.15.44:0/2515768446 conn(0x7f2c3c0b0eb0 msgr2=0x7f2c3c0715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:31.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.057+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 shutdown_connections 2024-07-17T08:42:31.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.058+0000 7f2c42600640 1 -- 172.21.15.44:0/2515768446 wait complete. 2024-07-17T08:42:31.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.273+0000 7f01c0a00640 1 -- 172.21.15.44:0/372070120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01a0003a80 msgr2=0x7f01a0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.273+0000 7f01c0a00640 1 --2- 172.21.15.44:0/372070120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01a0003a80 0x7f01a0003ea0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0016c0 tx=0x7f01a0005830 comp rx=0 tx=0).stop 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.274+0000 7f01ba800640 1 -- 172.21.15.44:0/372070120 reap_dead start 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.274+0000 7f01c0a00640 1 -- 172.21.15.44:0/372070120 shutdown_connections 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.274+0000 7f01c0a00640 1 -- 172.21.15.44:0/372070120 >> 172.21.15.44:0/372070120 conn(0x7f01bc0b0eb0 msgr2=0x7f01bc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.274+0000 7f01c0a00640 1 -- 172.21.15.44:0/372070120 shutdown_connections 2024-07-17T08:42:31.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.274+0000 7f01c0a00640 1 -- 172.21.15.44:0/372070120 wait complete. 2024-07-17T08:42:31.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.275+0000 7f01c0a00640 1 Processor -- start 2024-07-17T08:42:31.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.276+0000 7f01c0a00640 1 -- start start 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a0003a80 0x7f01bc156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01bc15f4a0 0x7f01bc1486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0b23c0 con 0x7f01bc09c660 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc15de90 con 0x7f01bc15f4a0 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01c0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc0b2540 con 0x7f01a0003a80 2024-07-17T08:42:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33758/0 (socket says 172.21.15.44:33758) 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 -- 172.21.15.44:0/169383829 learned_addr learned my addr 172.21.15.44:0/169383829 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01b9e00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a0003a80 0x7f01bc156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01b3e00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01bc15f4a0 0x7f01bc1486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 -- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a0003a80 msgr2=0x7f01bc156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a0003a80 0x7f01bc156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 -- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01bc15f4a0 msgr2=0x7f01bc1486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.277+0000 7f01ba800640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01bc15f4a0 0x7f01bc1486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:31.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01ba800640 1 -- 172.21.15.44:0/169383829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc155990 con 0x7f01bc09c660 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01ba800640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0709f0 tx=0x7f01b40064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4015070 con 0x7f01bc09c660 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01b40109a0 con 0x7f01bc09c660 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc155c70 con 0x7f01bc09c660 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4010df0 con 0x7f01bc09c660 2024-07-17T08:42:31.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.278+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc156230 con 0x7f01bc09c660 2024-07-17T08:42:31.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.280+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc0b2ad0 con 0x7f01bc09c660 2024-07-17T08:42:31.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.281+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 197) ==== 98875+0+0 (secure 0 0 0) 0x7f01b4011460 con 0x7f01bc09c660 2024-07-17T08:42:31.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.282+0000 7f01b2000640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 0x7f0198079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:31.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.282+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(325..325 src has 1..325) ==== 7170+0+0 (secure 0 0 0) 0x7f01b4098ff0 con 0x7f01bc09c660 2024-07-17T08:42:31.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.282+0000 7f01b9e00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 0x7f0198079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:31.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.284+0000 7f01b9e00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 0x7f0198079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01ac032000 tx=0x7f01ac001190 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:31.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.286+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f01b4061df0 con 0x7f01bc09c660 2024-07-17T08:42:31.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:31.497+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01bc06f700 con 0x7f01bc09c660 2024-07-17T08:42:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.052+0000 7f01b2000640 1 -- 172.21.15.44:0/169383829 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v326) ==== 169+0+0 (secure 0 0 0) 0x7f01b40616f0 con 0x7f01bc09c660 2024-07-17T08:42:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:42:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 msgr2=0x7f0198079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 0x7f0198079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01ac032000 tx=0x7f01ac001190 comp rx=0 tx=0).stop 2024-07-17T08:42:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 msgr2=0x7f01bc09cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0709f0 tx=0x7f01b40064e0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 shutdown_connections 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0198076c40 0x7f0198079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01bc09c660 0x7f01bc09cae0 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01bc15f4a0 0x7f01bc1486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 --2- 172.21.15.44:0/169383829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a0003a80 0x7f01bc156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.056+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 >> 172.21.15.44:0/169383829 conn(0x7f01bc0b0eb0 msgr2=0x7f01bc0735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.057+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 shutdown_connections 2024-07-17T08:42:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.057+0000 7f01c0a00640 1 -- 172.21.15.44:0/169383829 wait complete. 2024-07-17T08:42:32.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 -- 172.21.15.44:0/2249986246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7640004d40 msgr2=0x7f76400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 --2- 172.21.15.44:0/2249986246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7640004d40 0x7f76400051c0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f7650005fd0 tx=0x7f7640005830 comp rx=0 tx=0).stop 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765de00640 1 -- 172.21.15.44:0/2249986246 reap_dead start 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 -- 172.21.15.44:0/2249986246 shutdown_connections 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 -- 172.21.15.44:0/2249986246 >> 172.21.15.44:0/2249986246 conn(0x7f76580b0eb0 msgr2=0x7f76580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 -- 172.21.15.44:0/2249986246 shutdown_connections 2024-07-17T08:42:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.276+0000 7f765fe00640 1 -- 172.21.15.44:0/2249986246 wait complete. 2024-07-17T08:42:32.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.278+0000 7f765fe00640 1 Processor -- start 2024-07-17T08:42:32.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.278+0000 7f765fe00640 1 -- start start 2024-07-17T08:42:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 0x7f76580ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 0x7f7658073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76580cd2e0 con 0x7f76580735d0 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f765806d8e0 con 0x7f7640004d40 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76580cd460 con 0x7f76581f39e0 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33778/0 (socket says 172.21.15.44:33778) 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.279+0000 7f765de00640 1 -- 172.21.15.44:0/1366567903 learned_addr learned my addr 172.21.15.44:0/1366567903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 -- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 msgr2=0x7f7658073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765ca00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 0x7f7658073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765d400640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 0x7f76580ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 0x7f7658073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 -- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 msgr2=0x7f76580ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 0x7f76580ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 -- 172.21.15.44:0/1366567903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765815dc20 con 0x7f76580735d0 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765ca00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 0x7f7658073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765d400640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 0x7f76580ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765de00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f7650005fd0 tx=0x7f7650003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650031050 con 0x7f76580735d0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7650004c10 con 0x7f76580735d0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.280+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765815dec0 con 0x7f76580735d0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.281+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650035610 con 0x7f76580735d0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.281+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765815e480 con 0x7f76580735d0 2024-07-17T08:42:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.282+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765815c2f0 con 0x7f76580735d0 2024-07-17T08:42:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.283+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 197) ==== 98875+0+0 (secure 0 0 0) 0x7f7650046020 con 0x7f76580735d0 2024-07-17T08:42:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.284+0000 7f7655600640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 0x7f7638079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.284+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f76500b8750 con 0x7f76580735d0 2024-07-17T08:42:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.285+0000 7f765d400640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 0x7f7638079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.286+0000 7f765d400640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 0x7f7638079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ea60 tx=0x7f764c00e460 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:32.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.288+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7650081710 con 0x7f76580735d0 2024-07-17T08:42:32.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.500+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f7658159530 con 0x7f76580735d0 2024-07-17T08:42:32.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.507+0000 7f7655600640 1 -- 172.21.15.44:0/1366567903 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v99) ==== 71+0+0 (secure 0 0 0) 0x7f7650081010 con 0x7f76580735d0 2024-07-17T08:42:32.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 msgr2=0x7f7638079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 0x7f7638079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ea60 tx=0x7f764c00e460 comp rx=0 tx=0).stop 2024-07-17T08:42:32.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 msgr2=0x7f765815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f7650005fd0 tx=0x7f7650003ba0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 shutdown_connections 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7638076c40 0x7f7638079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76580735d0 0x7f765815ec80 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76581f39e0 0x7f7658073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 --2- 172.21.15.44:0/1366567903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7640004d40 0x7f76580ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.510+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 >> 172.21.15.44:0/1366567903 conn(0x7f76580b0eb0 msgr2=0x7f765815ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.511+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 shutdown_connections 2024-07-17T08:42:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.511+0000 7f765fe00640 1 -- 172.21.15.44:0/1366567903 wait complete. 2024-07-17T08:42:32.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04' 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 -- 172.21.15.44:0/342144595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0adcd0 msgr2=0x7f236c0ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 --2- 172.21.15.44:0/342144595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0adcd0 0x7f236c0ac220 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f236c070bf0 tx=0x7f235802a560 comp rx=0 tx=0).stop 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 -- 172.21.15.44:0/342144595 shutdown_connections 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 --2- 172.21.15.44:0/342144595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0adcd0 0x7f236c0ac220 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 --2- 172.21.15.44:0/342144595 >> v2:172.21.15.175:3300/0 conn(0x7f236c1522c0 0x7f236c0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 -- 172.21.15.44:0/342144595 >> 172.21.15.44:0/342144595 conn(0x7f236c0b0fb0 msgr2=0x7f236c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:32.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 -- 172.21.15.44:0/342144595 shutdown_connections 2024-07-17T08:42:32.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.746+0000 7f2374400640 1 -- 172.21.15.44:0/342144595 wait complete. 2024-07-17T08:42:32.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.748+0000 7f2374400640 1 Processor -- start 2024-07-17T08:42:32.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.748+0000 7f2374400640 1 -- start start 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f236c0adcd0 0x7f236c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f236c1522c0 0x7f236c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0ce820 con 0x7f236c0ac760 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0ce6a0 con 0x7f236c0adcd0 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2374400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f236c0ce9a0 con 0x7f236c1522c0 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2371a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2371a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33804/0 (socket says 172.21.15.44:33804) 2024-07-17T08:42:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.749+0000 7f2371a00640 1 -- 172.21.15.44:0/2976865316 learned_addr learned my addr 172.21.15.44:0/2976865316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 -- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f236c1522c0 msgr2=0x7f236c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f236c1522c0 0x7f236c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 -- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f236c0adcd0 msgr2=0x7f236c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f236c0adcd0 0x7f236c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 -- 172.21.15.44:0/2976865316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c0735d0 con 0x7f236c0ac760 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2371a00640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f236000ba50 tx=0x7f236000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360019070 con 0x7f236c0ac760 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f236000ccf0 con 0x7f236c0ac760 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600146a0 con 0x7f236c0ac760 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.750+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c0717a0 con 0x7f236c0ac760 2024-07-17T08:42:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.751+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c071d60 con 0x7f236c0ac760 2024-07-17T08:42:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.752+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c0ac220 con 0x7f236c0ac760 2024-07-17T08:42:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.753+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 197) ==== 98875+0+0 (secure 0 0 0) 0x7f236001c070 con 0x7f236c0ac760 2024-07-17T08:42:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.754+0000 7f2369600640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 0x7f234c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.754+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f23600985f0 con 0x7f236c0ac760 2024-07-17T08:42:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.755+0000 7f2371000640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 0x7f234c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.756+0000 7f2371000640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 0x7f234c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2358033000 tx=0x7f23580046c0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:32.759+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2360061640 con 0x7f236c0ac760 2024-07-17T08:42:33.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.048+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04"]} v 0) -- 0x7f236c15d6a0 con 0x7f236c0ac760 2024-07-17T08:42:33.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.056+0000 7f2369600640 1 -- 172.21.15.44:0/2976865316 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_04"]}]=0 v3609) ==== 201+0+0 (secure 0 0 0) 0x7f2360060f40 con 0x7f236c0ac760 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 msgr2=0x7f234c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 0x7f234c079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2358033000 tx=0x7f23580046c0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 msgr2=0x7f236c072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f236000ba50 tx=0x7f236000bf10 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 shutdown_connections 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f234c076c40 0x7f234c079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f236c1522c0 0x7f236c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f236c0adcd0 0x7f236c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 --2- 172.21.15.44:0/2976865316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f236c0ac760 0x7f236c072280 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 >> 172.21.15.44:0/2976865316 conn(0x7f236c0b0fb0 msgr2=0x7f236c15dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 shutdown_connections 2024-07-17T08:42:33.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.060+0000 7f2374400640 1 -- 172.21.15.44:0/2976865316 wait complete. 2024-07-17T08:42:33.079 INFO:tasks.cephfs_test_runner:Test subvol schedule creation fails without subvol argument. ... ok 2024-07-17T08:42:33.079 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_05 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule creation succeeds for a subvol under a subvolgroup. 2024-07-17T08:42:33.079 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_05 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:42:33.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 -- 172.21.15.44:0/703163894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 msgr2=0x7fcd480ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 --2- 172.21.15.44:0/703163894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd480ac220 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fcd30005fd0 tx=0x7fcd3002a5a0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 -- 172.21.15.44:0/703163894 shutdown_connections 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 --2- 172.21.15.44:0/703163894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd480ac220 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 --2- 172.21.15.44:0/703163894 >> v2:172.21.15.175:3300/0 conn(0x7fcd480ae810 0x7fcd4815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 -- 172.21.15.44:0/703163894 >> 172.21.15.44:0/703163894 conn(0x7fcd480b0fb0 msgr2=0x7fcd480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:33.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 -- 172.21.15.44:0/703163894 shutdown_connections 2024-07-17T08:42:33.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.289+0000 7fcd4d600640 1 -- 172.21.15.44:0/703163894 wait complete. 2024-07-17T08:42:33.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.291+0000 7fcd4d600640 1 Processor -- start 2024-07-17T08:42:33.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.291+0000 7fcd4d600640 1 -- start start 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4806eec0 0x7fcd48145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd480ae810 0x7fcd481461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480b2e60 con 0x7fcd4806ea50 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480b2ce0 con 0x7fcd4806eec0 2024-07-17T08:42:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.292+0000 7fcd4d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd480b2fe0 con 0x7fcd480ae810 2024-07-17T08:42:33.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33820/0 (socket says 172.21.15.44:33820) 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 -- 172.21.15.44:0/2465638210 learned_addr learned my addr 172.21.15.44:0/2465638210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd47400640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd480ae810 0x7fcd481461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46000640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4806eec0 0x7fcd48145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 -- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd480ae810 msgr2=0x7fcd481461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd480ae810 0x7fcd481461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 -- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4806eec0 msgr2=0x7fcd48145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4806eec0 0x7fcd48145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.293+0000 7fcd46a00640 1 -- 172.21.15.44:0/2465638210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd48073a20 con 0x7fcd4806ea50 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd46a00640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ea20 tx=0x7fcd3800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800cde0 con 0x7fcd4806ea50 2024-07-17T08:42:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd38011560 con 0x7fcd4806ea50 2024-07-17T08:42:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011990 con 0x7fcd4806ea50 2024-07-17T08:42:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd48142330 con 0x7fcd4806ea50 2024-07-17T08:42:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.294+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd481428f0 con 0x7fcd4806ea50 2024-07-17T08:42:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.295+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd48146de0 con 0x7fcd4806ea50 2024-07-17T08:42:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.297+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 197) ==== 98875+0+0 (secure 0 0 0) 0x7fcd38011b30 con 0x7fcd4806ea50 2024-07-17T08:42:33.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.298+0000 7fcd3ea00640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 0x7fcd24079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.298+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7fcd38099520 con 0x7fcd4806ea50 2024-07-17T08:42:33.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.299+0000 7fcd46000640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 0x7fcd24079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.300+0000 7fcd46000640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 0x7fcd24079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd30033000 tx=0x7fcd30004730 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:33.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.303+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcd48146de0 con 0x7fcd4806ea50 2024-07-17T08:42:33.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.508+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd480ac3b0 con 0x7fcd4806ea50 2024-07-17T08:42:33.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.515+0000 7fcd3ea00640 1 -- 172.21.15.44:0/2465638210 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v326) ==== 74+0+15423 (secure 0 0 0) 0x7fcd480ac3b0 con 0x7fcd4806ea50 2024-07-17T08:42:33.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:33.516 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":326,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:32.049627+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":44,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":319,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6801/1754084600":"2024-07-18T08:42:29.022285+0000","172.21.15.175:6800/1754084600":"2024-07-18T08:42:29.022285+0000","172.21.15.175:0/2600563189":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/2846510060":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/2753934550":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/2865226962":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/1191177389":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/2560609937":"2024-07-18T08:42:28.876337+0000","172.21.15.175:6837/2701447474":"2024-07-18T08:42:28.876337+0000","172.21.15.175:6834/2701447474":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/3829872441":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/422671409":"2024-07-18T08:42:28.876337+0000","172.21.15.175:0/1949052193":"2024-07-18T08:42:28.876337+0000","172.21.15.175:6834/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2545797614":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3424032169":"2024-07-18T08:42:20.486767+0000","172.21.15.175:6837/3850741755":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3407108431":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/2271900493":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3634187181":"2024-07-18T08:42:20.486767+0000","172.21.15.175:0/3817206034":"2024-07-18T08:42:20.486767+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.517+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 msgr2=0x7fcd24079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.517+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 0x7fcd24079100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd30033000 tx=0x7fcd30004730 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.517+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 msgr2=0x7fcd481f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.517+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ea20 tx=0x7fcd3800eee0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 shutdown_connections 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fcd24076c40 0x7fcd24079100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd480ae810 0x7fcd481461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd4806eec0 0x7fcd48145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 --2- 172.21.15.44:0/2465638210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd4806ea50 0x7fcd481f3c90 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 >> 172.21.15.44:0/2465638210 conn(0x7fcd480b0fb0 msgr2=0x7fcd480b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:33.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 shutdown_connections 2024-07-17T08:42:33.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.518+0000 7fcd4d600640 1 -- 172.21.15.44:0/2465638210 wait complete. 2024-07-17T08:42:33.538 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:42:33.539 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:42:33.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.747+0000 7f9d4ee00640 1 -- 172.21.15.44:0/250716001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d2c004d40 msgr2=0x7f9d2c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.747+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/250716001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009f00 tx=0x7f9d2c005830 comp rx=0 tx=0).stop 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.747+0000 7f9d4ce00640 1 -- 172.21.15.44:0/250716001 reap_dead start 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.748+0000 7f9d4ee00640 1 -- 172.21.15.44:0/250716001 shutdown_connections 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.748+0000 7f9d4ee00640 1 -- 172.21.15.44:0/250716001 >> 172.21.15.44:0/250716001 conn(0x7f9d480b0eb0 msgr2=0x7f9d480b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.748+0000 7f9d4ee00640 1 -- 172.21.15.44:0/250716001 shutdown_connections 2024-07-17T08:42:33.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.748+0000 7f9d4ee00640 1 -- 172.21.15.44:0/250716001 wait complete. 2024-07-17T08:42:33.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.749+0000 7f9d4ee00640 1 Processor -- start 2024-07-17T08:42:33.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.750+0000 7f9d4ee00640 1 -- start start 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d2c004d40 0x7f9d481f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48073b10 0x7f9d48072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480b3e60 con 0x7f9d481508a0 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480b3ce0 con 0x7f9d48073b10 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480b3fe0 con 0x7f9d2c004d40 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33826/0 (socket says 172.21.15.44:33826) 2024-07-17T08:42:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 -- 172.21.15.44:0/4063438681 learned_addr learned my addr 172.21.15.44:0/4063438681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d47e00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d2c004d40 0x7f9d481f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d3f400640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48073b10 0x7f9d48072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 -- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d2c004d40 msgr2=0x7f9d481f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d2c004d40 0x7f9d481f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 -- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48073b10 msgr2=0x7f9d48072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48073b10 0x7f9d48072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.751+0000 7f9d4ce00640 1 -- 172.21.15.44:0/4063438681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480731b0 con 0x7f9d481508a0 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d4ce00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009f00 tx=0x7f9d40034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40038040 con 0x7f9d481508a0 2024-07-17T08:42:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d40003c60 con 0x7f9d481508a0 2024-07-17T08:42:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40009200 con 0x7f9d481508a0 2024-07-17T08:42:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4814f030 con 0x7f9d481508a0 2024-07-17T08:42:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.752+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4814f5f0 con 0x7f9d481508a0 2024-07-17T08:42:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.754+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4806f380 con 0x7f9d481508a0 2024-07-17T08:42:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.755+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f9d40004290 con 0x7f9d481508a0 2024-07-17T08:42:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.756+0000 7f9d44c00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 0x7f9d24079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.756+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f9d400b9430 con 0x7f9d481508a0 2024-07-17T08:42:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.756+0000 7f9d47e00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 0x7f9d24079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:33.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.757+0000 7f9d47e00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 0x7f9d24079210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009aa0 tx=0x7f9d38005f90 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:33.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.760+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9d4806f380 con 0x7f9d481508a0 2024-07-17T08:42:33.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.973+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d480ac640 con 0x7f9d481508a0 2024-07-17T08:42:33.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.974+0000 7f9d44c00640 1 -- 172.21.15.44:0/4063438681 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v326) ==== 93+0+10 (secure 0 0 0) 0x7f9d40082540 con 0x7f9d481508a0 2024-07-17T08:42:33.975 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 msgr2=0x7f9d24079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 0x7f9d24079210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009aa0 tx=0x7f9d38005f90 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 msgr2=0x7f9d48150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009f00 tx=0x7f9d40034d40 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 shutdown_connections 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9d24076d50 0x7f9d24079210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d481508a0 0x7f9d48150d50 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48073b10 0x7f9d48072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 --2- 172.21.15.44:0/4063438681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d2c004d40 0x7f9d481f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 >> 172.21.15.44:0/4063438681 conn(0x7f9d480b0eb0 msgr2=0x7f9d480b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 shutdown_connections 2024-07-17T08:42:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:33.977+0000 7f9d4ee00640 1 -- 172.21.15.44:0/4063438681 wait complete. 2024-07-17T08:42:33.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05' 2024-07-17T08:42:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 -- 172.21.15.44:0/3694334284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 msgr2=0x7f9c540cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 --2- 172.21.15.44:0/3694334284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c540cc620 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005fd0 tx=0x7f9c4c02a970 comp rx=0 tx=0).stop 2024-07-17T08:42:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 -- 172.21.15.44:0/3694334284 shutdown_connections 2024-07-17T08:42:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 --2- 172.21.15.44:0/3694334284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c540cc620 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:34.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 --2- 172.21.15.44:0/3694334284 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c5406fec0 0x7f9c540702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 -- 172.21.15.44:0/3694334284 >> 172.21.15.44:0/3694334284 conn(0x7f9c540b3120 msgr2=0x7f9c540b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 -- 172.21.15.44:0/3694334284 shutdown_connections 2024-07-17T08:42:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.210+0000 7f9c5a600640 1 -- 172.21.15.44:0/3694334284 wait complete. 2024-07-17T08:42:34.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.212+0000 7f9c5a600640 1 Processor -- start 2024-07-17T08:42:34.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.212+0000 7f9c5a600640 1 -- start start 2024-07-17T08:42:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5406fec0 0x7f9c54149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:34.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c540ce530 0x7f9c540af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c540ccb60 con 0x7f9c540cd610 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c540cd200 con 0x7f9c5406fec0 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.213+0000 7f9c5a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c540ccce0 con 0x7f9c540ce530 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33844/0 (socket says 172.21.15.44:33844) 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 -- 172.21.15.44:0/2009091425 learned_addr learned my addr 172.21.15.44:0/2009091425 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c53400640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5406fec0 0x7f9c54149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c53e00640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c540ce530 0x7f9c540af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 -- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c540ce530 msgr2=0x7f9c540af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c540ce530 0x7f9c540af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 -- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5406fec0 msgr2=0x7f9c54149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:34.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5406fec0 0x7f9c54149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 -- 172.21.15.44:0/2009091425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c541f7aa0 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.214+0000 7f9c52a00640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c3f0 tx=0x7f9c3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.215+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c007020 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.215+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c3c0072b0 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.215+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0067d0 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.215+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c541f7d80 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.215+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541f8340 con 0x7f9c540cd610 2024-07-17T08:42:34.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.216+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54071fe0 con 0x7f9c540cd610 2024-07-17T08:42:34.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.217+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f9c3c019020 con 0x7f9c540cd610 2024-07-17T08:42:34.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.218+0000 7f9c4b400640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 0x7f9c30079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:34.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.218+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f9c3c098180 con 0x7f9c540cd610 2024-07-17T08:42:34.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.219+0000 7f9c53400640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 0x7f9c30079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:34.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.220+0000 7f9c53400640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 0x7f9c30079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c440058c0 tx=0x7f9c44005d80 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:34.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.223+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9c54071fe0 con 0x7f9c540cd610 2024-07-17T08:42:34.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:34.509+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05"]} v 0) -- 0x7f9c540702a0 con 0x7f9c540cd610 2024-07-17T08:42:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.070+0000 7f9c4b400640 1 -- 172.21.15.44:0/2009091425 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05"]}]=0 v3611) ==== 204+0+0 (secure 0 0 0) 0x7f9c3c061080 con 0x7f9c540cd610 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.073+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 msgr2=0x7f9c30079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.073+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 0x7f9c30079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c440058c0 tx=0x7f9c44005d80 comp rx=0 tx=0).stop 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.073+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 msgr2=0x7f9c5406ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.073+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c3f0 tx=0x7f9c3c00c8b0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 shutdown_connections 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f9c30076cd0 0x7f9c30079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c540ce530 0x7f9c540af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c540cd610 0x7f9c5406ea30 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 --2- 172.21.15.44:0/2009091425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5406fec0 0x7f9c54149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 >> 172.21.15.44:0/2009091425 conn(0x7f9c540b3120 msgr2=0x7f9c5414f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 shutdown_connections 2024-07-17T08:42:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.074+0000 7f9c5a600640 1 -- 172.21.15.44:0/2009091425 wait complete. 2024-07-17T08:42:35.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:42:35.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 -- 172.21.15.44:0/3279993250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b80ce660 msgr2=0x7fe6b815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 --2- 172.21.15.44:0/3279993250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b80ce660 0x7fe6b815b640 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009900 tx=0x7fe6a802a560 comp rx=0 tx=0).stop 2024-07-17T08:42:35.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 -- 172.21.15.44:0/3279993250 shutdown_connections 2024-07-17T08:42:35.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 --2- 172.21.15.44:0/3279993250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b81522b0 0x7fe6b80cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 --2- 172.21.15.44:0/3279993250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b80ce660 0x7fe6b815b640 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 -- 172.21.15.44:0/3279993250 >> 172.21.15.44:0/3279993250 conn(0x7fe6b806c120 msgr2=0x7fe6b806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:35.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 -- 172.21.15.44:0/3279993250 shutdown_connections 2024-07-17T08:42:35.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.312+0000 7fe6bca00640 1 -- 172.21.15.44:0/3279993250 wait complete. 2024-07-17T08:42:35.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.314+0000 7fe6bca00640 1 Processor -- start 2024-07-17T08:42:35.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.314+0000 7fe6bca00640 1 -- start start 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 0x7fe6b80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b815bb80 0x7fe6b815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b80cb6d0 con 0x7fe6b81522b0 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b80cec50 con 0x7fe6b815bb80 2024-07-17T08:42:35.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6bca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b80cb850 con 0x7fe6b80ce660 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6b6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33852/0 (socket says 172.21.15.44:33852) 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.315+0000 7fe6b6a00640 1 -- 172.21.15.44:0/2447821401 learned_addr learned my addr 172.21.15.44:0/2447821401 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b7400640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 0x7fe6b80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 -- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 msgr2=0x7fe6b80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 0x7fe6b80b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b7e00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b815bb80 0x7fe6b815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 -- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b815bb80 msgr2=0x7fe6b815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b815bb80 0x7fe6b815a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 -- 172.21.15.44:0/2447821401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b81f4fc0 con 0x7fe6b81522b0 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b7400640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 0x7fe6b80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.316+0000 7fe6b6a00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000b3e0 tx=0x7fe6a000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:35.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.317+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0015020 con 0x7fe6b81522b0 2024-07-17T08:42:35.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.317+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a0011940 con 0x7fe6b81522b0 2024-07-17T08:42:35.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.317+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0011d90 con 0x7fe6b81522b0 2024-07-17T08:42:35.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.317+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b81f7a60 con 0x7fe6b81522b0 2024-07-17T08:42:35.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.317+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b81f8020 con 0x7fe6b81522b0 2024-07-17T08:42:35.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.318+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b80b4060 con 0x7fe6b81522b0 2024-07-17T08:42:35.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.319+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fe6a0018020 con 0x7fe6b81522b0 2024-07-17T08:42:35.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.320+0000 7fe6af400640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 0x7fe694079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.320+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7fe6a00988c0 con 0x7fe6b81522b0 2024-07-17T08:42:35.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.321+0000 7fe6b7400640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 0x7fe694079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.322+0000 7fe6b7400640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 0x7fe694079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8033000 tx=0x7fe6a8004730 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:35.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.325+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe6b80b4060 con 0x7fe6b81522b0 2024-07-17T08:42:35.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.540+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe6b815b640 con 0x7fe6b81522b0 2024-07-17T08:42:35.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.542+0000 7fe6af400640 1 -- 172.21.15.44:0/2447821401 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v99) ==== 59+0+1998 (secure 0 0 0) 0x7fe6a001f020 con 0x7fe6b81522b0 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:42:35.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:42:35.544 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:42:35.545 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:42:35.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 msgr2=0x7fe694079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 0x7fe694079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8033000 tx=0x7fe6a8004730 comp rx=0 tx=0).stop 2024-07-17T08:42:35.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 msgr2=0x7fe6b80b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000b3e0 tx=0x7fe6a000b8a0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 shutdown_connections 2024-07-17T08:42:35.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe694076cd0 0x7fe694079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6b815bb80 0x7fe6b815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6b81522b0 0x7fe6b80b3290 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 --2- 172.21.15.44:0/2447821401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6b80ce660 0x7fe6b80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 >> 172.21.15.44:0/2447821401 conn(0x7fe6b806c120 msgr2=0x7fe6b80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 shutdown_connections 2024-07-17T08:42:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.545+0000 7fe6bca00640 1 -- 172.21.15.44:0/2447821401 wait complete. 2024-07-17T08:42:35.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:42:35.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 -- 172.21.15.44:0/2975358812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a4152220 msgr2=0x7f18a406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 --2- 172.21.15.44:0/2975358812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a4152220 0x7f18a406d620 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7f18a406f610 tx=0x7f189402a560 comp rx=0 tx=0).stop 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 -- 172.21.15.44:0/2975358812 shutdown_connections 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 --2- 172.21.15.44:0/2975358812 >> v2:172.21.15.175:3300/0 conn(0x7f18a406ed30 0x7f18a415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 --2- 172.21.15.44:0/2975358812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a4152220 0x7f18a406d620 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 -- 172.21.15.44:0/2975358812 >> 172.21.15.44:0/2975358812 conn(0x7f18a40b0eb0 msgr2=0x7f18a40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.783+0000 7f18aba00640 1 -- 172.21.15.44:0/2975358812 shutdown_connections 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.784+0000 7f18aba00640 1 -- 172.21.15.44:0/2975358812 wait complete. 2024-07-17T08:42:35.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.785+0000 7f18aba00640 1 Processor -- start 2024-07-17T08:42:35.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.785+0000 7f18aba00640 1 -- start start 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a40cb3a0 0x7f18a41f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a4152220 0x7f18a41f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40cbc30 con 0x7f18a406ed30 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40cbab0 con 0x7f18a40cb3a0 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.786+0000 7f18aba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18a415f570 con 0x7f18a4152220 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33864/0 (socket says 172.21.15.44:33864) 2024-07-17T08:42:35.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 -- 172.21.15.44:0/3264766419 learned_addr learned my addr 172.21.15.44:0/3264766419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 -- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a4152220 msgr2=0x7f18a41f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a4152220 0x7f18a41f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 -- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a40cb3a0 msgr2=0x7f18a41f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a40cb3a0 0x7f18a41f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:35.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 -- 172.21.15.44:0/3264766419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a41f2a40 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.787+0000 7f18a9000640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f1894034040 tx=0x7f18940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.788+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189403e030 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.788+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18940047a0 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.788+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035040 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.788+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a4073f60 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.788+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a41f18b0 con 0x7f18a406ed30 2024-07-17T08:42:35.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.789+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a415fa90 con 0x7f18a406ed30 2024-07-17T08:42:35.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.790+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f1894041030 con 0x7f18a406ed30 2024-07-17T08:42:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.791+0000 7f18a0c00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 0x7f1880079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:35.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.791+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f18940b90c0 con 0x7f18a406ed30 2024-07-17T08:42:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.792+0000 7f18a3e00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 0x7f1880079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:35.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.793+0000 7f18a3e00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 0x7f1880079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f18a406d5d0 tx=0x7f188c005c70 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:35.796+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f18a41f2a40 con 0x7f18a406ed30 2024-07-17T08:42:36.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.010+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f18a4074480 con 0x7f18a406ed30 2024-07-17T08:42:36.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.018+0000 7f18a0c00640 1 -- 172.21.15.44:0/3264766419 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v100)=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f1894081f40 con 0x7f18a406ed30 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 msgr2=0x7f1880079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 0x7f1880079190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f18a406d5d0 tx=0x7f188c005c70 comp rx=0 tx=0).stop 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 msgr2=0x7f18a41f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f1894034040 tx=0x7f18940039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 shutdown_connections 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1880076cd0 0x7f1880079190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18a4152220 0x7f18a41f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18a40cb3a0 0x7f18a41f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 --2- 172.21.15.44:0/3264766419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18a406ed30 0x7f18a41f2500 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 >> 172.21.15.44:0/3264766419 conn(0x7f18a40b0eb0 msgr2=0x7f18a40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:36.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 shutdown_connections 2024-07-17T08:42:36.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.021+0000 7f18aba00640 1 -- 172.21.15.44:0/3264766419 wait complete. 2024-07-17T08:42:36.041 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:42:36.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:36.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:42:36.271 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:42:36.271 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:36.271 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:36.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:42:36.332 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:36.332 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:36.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:36.388 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:36.388 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:36.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:42:36.447 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:42:36.447 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:36.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:42:36.503 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:42:36.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.726+0000 7f0160e00640 1 -- 172.21.15.44:0/3543277900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c15d600 msgr2=0x7f015c06e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.726+0000 7f0160e00640 1 --2- 172.21.15.44:0/3543277900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c15d600 0x7f015c06e480 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f014c005fd0 tx=0x7f014c02a660 comp rx=0 tx=0).stop 2024-07-17T08:42:36.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.727+0000 7f0160e00640 1 -- 172.21.15.44:0/3543277900 shutdown_connections 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.727+0000 7f0160e00640 1 --2- 172.21.15.44:0/3543277900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c15d600 0x7f015c06e480 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.727+0000 7f0160e00640 1 -- 172.21.15.44:0/3543277900 >> 172.21.15.44:0/3543277900 conn(0x7f015c0d00a0 msgr2=0x7f015c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.727+0000 7f0160e00640 1 -- 172.21.15.44:0/3543277900 shutdown_connections 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.727+0000 7f0160e00640 1 -- 172.21.15.44:0/3543277900 wait complete. 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.728+0000 7f0160e00640 1 Processor -- start 2024-07-17T08:42:36.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.729+0000 7f0160e00640 1 -- start start 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 0x7f015c1477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 0x7f015c1f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0ce850 con 0x7f015c0b2ad0 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0ce6d0 con 0x7f015c15d600 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0160e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f015c0ce9d0 con 0x7f015c0b5910 2024-07-17T08:42:36.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33872/0 (socket says 172.21.15.44:33872) 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 -- 172.21.15.44:0/2028045294 learned_addr learned my addr 172.21.15.44:0/2028045294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f015a800640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 0x7f015c1f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159400640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 0x7f015c1477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 -- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 msgr2=0x7f015c1477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 0x7f015c1477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 -- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 msgr2=0x7f015c1f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 0x7f015c1f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159e00640 1 -- 172.21.15.44:0/2028045294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c0cc460 con 0x7f015c0b2ad0 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f0159400640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 0x7f015c1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:36.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.730+0000 7f015a800640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 0x7f015c1f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0159e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f014c034040 tx=0x7f014c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c03e030 con 0x7f015c0b2ad0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f014c0047a0 con 0x7f015c0b2ad0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c035040 con 0x7f015c0b2ad0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c0cad20 con 0x7f015c0b2ad0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.731+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c0cc9e0 con 0x7f015c0b2ad0 2024-07-17T08:42:36.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.733+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c0cc270 con 0x7f015c0b2ad0 2024-07-17T08:42:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.734+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f014c041030 con 0x7f015c0b2ad0 2024-07-17T08:42:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.735+0000 7f0152000640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 0x7f01380791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:36.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.735+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 6756+0+0 (secure 0 0 0) 0x7f014c0b91a0 con 0x7f015c0b2ad0 2024-07-17T08:42:36.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.736+0000 7f0159400640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 0x7f01380791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:36.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.737+0000 7f0159400640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 0x7f01380791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b2760 tx=0x7f0144009040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:36.742+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f014c081f90 con 0x7f015c0b2ad0 2024-07-17T08:42:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.001+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f015c0cee80 con 0x7f015c0b2ad0 2024-07-17T08:42:37.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.002+0000 7f0152000640 1 -- 172.21.15.44:0/2028045294 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) ==== 95+0+3921 (secure 0 0 0) 0x7f014c08a1d0 con 0x7f015c0b2ad0 2024-07-17T08:42:37.003 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:37.004 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":183,"btime":"2024-07-17T08:42:30:024527+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/666145926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":666145926},{"type":"v1","addr":"172.21.15.44:6835","nonce":666145926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1685709364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1685709364},{"type":"v1","addr":"172.21.15.175:6836","nonce":1685709364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":10423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3238551453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3238551453},{"type":"v1","addr":"172.21.15.44:6834","nonce":3238551453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":10675,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2878892982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2878892982},{"type":"v1","addr":"172.21.15.175:6801","nonce":2878892982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":183}],"filesystems":[]} 2024-07-17T08:42:37.004 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 183 2024-07-17T08:42:37.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.005+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 msgr2=0x7f01380791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 0x7f01380791c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f015c0b2760 tx=0x7f0144009040 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 msgr2=0x7f015c155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f014c034040 tx=0x7f014c0039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 shutdown_connections 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f0138076d00 0x7f01380791c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015c15d600 0x7f015c1f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f015c0b5910 0x7f015c1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 --2- 172.21.15.44:0/2028045294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f015c0b2ad0 0x7f015c155370 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 >> 172.21.15.44:0/2028045294 conn(0x7f015c0d00a0 msgr2=0x7f015c157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 shutdown_connections 2024-07-17T08:42:37.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:37.006+0000 7f0160e00640 1 -- 172.21.15.44:0/2028045294 wait complete. 2024-07-17T08:42:37.027 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:42:37.027 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:42:37.027 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:42:37.027 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:37.027 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:42:37.027 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:42:37.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:42:37.028 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:37.028 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:42:37.028 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:42:37.028 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:42:37.028 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:37.028 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:42:37.029 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:42:37.029 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:42:37.029 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:42:37.046 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:42:37.046 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:42:37.050 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:42:37.050 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:42:37.051 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:42:37.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:42:37.093 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:42:37.093 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:37.098 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:42:37.098 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:37.104 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:42:37.104 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:37.246 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:42:37.246 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:42:37.264 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:42:41.781 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-07-17T08:42:42.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:42:42.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:42:43.045 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:42:43.046 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:42:43.054 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:42:43.054 DEBUG:teuthology.parallel:result is None 2024-07-17T08:42:43.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:42:43.215 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.249+0000 7f42fa200640 1 -- 172.21.15.44:0/743074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 msgr2=0x7f42f406d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.249+0000 7f42fa200640 1 --2- 172.21.15.44:0/743074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f406d690 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005fd0 tx=0x7f42dc02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.250+0000 7f42fa200640 1 -- 172.21.15.44:0/743074498 shutdown_connections 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.250+0000 7f42fa200640 1 --2- 172.21.15.44:0/743074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f406d690 unknown :-1 s=CLOSED pgs=3004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.250+0000 7f42fa200640 1 -- 172.21.15.44:0/743074498 >> 172.21.15.44:0/743074498 conn(0x7f42f4073d50 msgr2=0x7f42f409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.250+0000 7f42fa200640 1 -- 172.21.15.44:0/743074498 shutdown_connections 2024-07-17T08:42:43.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.250+0000 7f42fa200640 1 -- 172.21.15.44:0/743074498 wait complete. 2024-07-17T08:42:43.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.251+0000 7f42fa200640 1 Processor -- start 2024-07-17T08:42:43.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.251+0000 7f42fa200640 1 -- start start 2024-07-17T08:42:43.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 0x7f42f414eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:43.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:43.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 0x7f42f41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:43.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40aed90 con 0x7f42f40b1a10 2024-07-17T08:42:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40b2ba0 con 0x7f42f415d810 2024-07-17T08:42:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.252+0000 7f42fa200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40aef10 con 0x7f42f406dbd0 2024-07-17T08:42:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41818/0 (socket says 172.21.15.44:41818) 2024-07-17T08:42:43.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 -- 172.21.15.44:0/2072304390 learned_addr learned my addr 172.21.15.44:0/2072304390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f3e00640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 0x7f42f41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f3400640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 0x7f42f414eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 -- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 msgr2=0x7f42f414eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 0x7f42f414eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 -- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 msgr2=0x7f42f41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 0x7f42f41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f2a00640 1 -- 172.21.15.44:0/2072304390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f41f3f00 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f3e00640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 0x7f42f41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.253+0000 7f42f3400640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 0x7f42f414eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42f2a00640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc03f070 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42dc0047a0 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f41f41e0 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc035030 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.254+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f41f47a0 con 0x7f42f40b1a10 2024-07-17T08:42:43.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.255+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f40af420 con 0x7f42f40b1a10 2024-07-17T08:42:43.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.256+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f42dc042070 con 0x7f42f40b1a10 2024-07-17T08:42:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.257+0000 7f42eb400640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 0x7f42d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.257+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(329..329 src has 1..329) ==== 6756+0+0 (secure 0 0 0) 0x7f42dc0b96b0 con 0x7f42f40b1a10 2024-07-17T08:42:43.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.258+0000 7f42f3400640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 0x7f42d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:43.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.259+0000 7f42f3400640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 0x7f42d0079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00a830 tx=0x7f42ec008040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:43.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.262+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f42dc082530 con 0x7f42f40b1a10 2024-07-17T08:42:43.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:43.488+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f42f40681e0 con 0x7f42f40b1a10 2024-07-17T08:42:45.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.123+0000 7f42eb400640 1 -- 172.21.15.44:0/2072304390 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v331) ==== 97+0+0 (secure 0 0 0) 0x7f42dc081e30 con 0x7f42f40b1a10 2024-07-17T08:42:45.126 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:42:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 msgr2=0x7f42d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 0x7f42d0079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00a830 tx=0x7f42ec008040 comp rx=0 tx=0).stop 2024-07-17T08:42:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 msgr2=0x7f42f41560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc0038a0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 shutdown_connections 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f42d0076cd0 0x7f42d0079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.127+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42f415d810 0x7f42f41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.128+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42f40b1a10 0x7f42f41560f0 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.128+0000 7f42fa200640 1 --2- 172.21.15.44:0/2072304390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42f406dbd0 0x7f42f414eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.128+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 >> 172.21.15.44:0/2072304390 conn(0x7f42f4073d50 msgr2=0x7f42f415ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.128+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 shutdown_connections 2024-07-17T08:42:45.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.128+0000 7f42fa200640 1 -- 172.21.15.44:0/2072304390 wait complete. 2024-07-17T08:42:45.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.361+0000 7f55cb200640 1 -- 172.21.15.44:0/1133159805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 msgr2=0x7f55c40ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.361+0000 7f55cb200640 1 --2- 172.21.15.44:0/1133159805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c40ac220 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f55ac005fd0 tx=0x7f55ac02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.361+0000 7f55cb200640 1 -- 172.21.15.44:0/1133159805 shutdown_connections 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.361+0000 7f55cb200640 1 --2- 172.21.15.44:0/1133159805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c40ac220 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.362+0000 7f55cb200640 1 --2- 172.21.15.44:0/1133159805 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c40ae810 0x7f55c415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.362+0000 7f55cb200640 1 -- 172.21.15.44:0/1133159805 >> 172.21.15.44:0/1133159805 conn(0x7f55c40b0fb0 msgr2=0x7f55c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.362+0000 7f55cb200640 1 -- 172.21.15.44:0/1133159805 shutdown_connections 2024-07-17T08:42:45.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.362+0000 7f55cb200640 1 -- 172.21.15.44:0/1133159805 wait complete. 2024-07-17T08:42:45.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.363+0000 7f55cb200640 1 Processor -- start 2024-07-17T08:42:45.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 -- start start 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c414d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 0x7f55c41f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55c40b2e60 con 0x7f55c40ae810 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55c40b2ce0 con 0x7f55c406ea50 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.364+0000 7f55cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55c40b2fe0 con 0x7f55c406ee90 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41828/0 (socket says 172.21.15.44:41828) 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 -- 172.21.15.44:0/2701664722 learned_addr learned my addr 172.21.15.44:0/2701664722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c3e00640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c414d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 -- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 msgr2=0x7f55c41f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c3400640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 0x7f55c41f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 0x7f55c41f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 -- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 msgr2=0x7f55c414d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c414d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c9200640 1 -- 172.21.15.44:0/2701664722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c4154b80 con 0x7f55c40ae810 2024-07-17T08:42:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.365+0000 7f55c3400640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 0x7f55c41f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55c9200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c340 tx=0x7f55bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc017070 con 0x7f55c40ae810 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55bc0053c0 con 0x7f55c40ae810 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41f4870 con 0x7f55c40ae810 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003850 con 0x7f55c40ae810 2024-07-17T08:42:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.366+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41f4e30 con 0x7f55c40ae810 2024-07-17T08:42:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.367+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c40ac920 con 0x7f55c40ae810 2024-07-17T08:42:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.369+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f55bc01a070 con 0x7f55c40ae810 2024-07-17T08:42:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.370+0000 7f55bbe00640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 0x7f55a00791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.370+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(331..331 src has 1..331) ==== 5853+0+0 (secure 0 0 0) 0x7f55bc097900 con 0x7f55c40ae810 2024-07-17T08:42:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.370+0000 7f55c3e00640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 0x7f55a00791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.371+0000 7f55c3e00640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 0x7f55a00791c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b4009500 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:45.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.374+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f55c415ea50 con 0x7f55c40ae810 2024-07-17T08:42:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.665+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f55c40b23b0 con 0x7f55c40ae810 2024-07-17T08:42:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.665+0000 7f55bbe00640 1 -- 172.21.15.44:0/2701664722 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v80) ==== 80+0+5304 (secure 0 0 0) 0x7f55bc060ba0 con 0x7f55c40ae810 2024-07-17T08:42:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:45.667 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:42:45.667 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:42:45.668 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:42:45.672 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:45.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.679 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:45.681 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:42:45.682 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 msgr2=0x7f55a00791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 0x7f55a00791c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b4009500 comp rx=0 tx=0).stop 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 msgr2=0x7f55c4155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c340 tx=0x7f55bc00c800 comp rx=0 tx=0).stop 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 shutdown_connections 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f55a0076d00 0x7f55a00791c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55c40ae810 0x7f55c4155360 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55c406ee90 0x7f55c41f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 --2- 172.21.15.44:0/2701664722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55c406ea50 0x7f55c414d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.668+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 >> 172.21.15.44:0/2701664722 conn(0x7f55c40b0fb0 msgr2=0x7f55c40aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.669+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 shutdown_connections 2024-07-17T08:42:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.669+0000 7f55cb200640 1 -- 172.21.15.44:0/2701664722 wait complete. 2024-07-17T08:42:45.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:42:45.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.884+0000 7fade7e00640 1 -- 172.21.15.44:0/1274991899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc004800 msgr2=0x7fadcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.884+0000 7fade7e00640 1 --2- 172.21.15.44:0/1274991899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc004800 0x7fadcc004c80 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fadd400baf0 tx=0x7fadcc005830 comp rx=0 tx=0).stop 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.884+0000 7fade7400640 1 -- 172.21.15.44:0/1274991899 reap_dead start 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.885+0000 7fade7e00640 1 -- 172.21.15.44:0/1274991899 shutdown_connections 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.885+0000 7fade7e00640 1 -- 172.21.15.44:0/1274991899 >> 172.21.15.44:0/1274991899 conn(0x7fade80cfec0 msgr2=0x7fade80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.885+0000 7fade7e00640 1 -- 172.21.15.44:0/1274991899 shutdown_connections 2024-07-17T08:42:45.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.885+0000 7fade7e00640 1 -- 172.21.15.44:0/1274991899 wait complete. 2024-07-17T08:42:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.886+0000 7fade7e00640 1 Processor -- start 2024-07-17T08:42:45.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.887+0000 7fade7e00640 1 -- start start 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.887+0000 7fade7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade8148f50 0x7fade81f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.887+0000 7fade7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade81f7850 0x7fade81e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fade815d980 con 0x7fade8149330 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fade815d800 con 0x7fade81f7850 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fade815db00 con 0x7fade8148f50 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41846/0 (socket says 172.21.15.44:41846) 2024-07-17T08:42:45.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 -- 172.21.15.44:0/2327655480 learned_addr learned my addr 172.21.15.44:0/2327655480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6a00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade8148f50 0x7fade81f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 -- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade8148f50 msgr2=0x7fade81f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade8148f50 0x7fade81f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 -- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade81f7850 msgr2=0x7fade81e72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade81f7850 0x7fade81e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:45.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.888+0000 7fade6000640 1 -- 172.21.15.44:0/2327655480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade81f51e0 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7fade6000640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fadd402b000 tx=0x7fadd400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4040070 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadd4004230 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4035040 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade81f4940 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.889+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81f4f00 con 0x7fade8149330 2024-07-17T08:42:45.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.890+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade80d0fe0 con 0x7fade8149330 2024-07-17T08:42:45.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.892+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fadd4038020 con 0x7fade8149330 2024-07-17T08:42:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.893+0000 7faddf400640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 0x7fadc80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:45.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.893+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(331..331 src has 1..331) ==== 5853+0+0 (secure 0 0 0) 0x7fadd40b9260 con 0x7fade8149330 2024-07-17T08:42:45.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.893+0000 7fade6a00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 0x7fadc80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:45.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.894+0000 7fade6a00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 0x7fadc80791c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000fb0 tx=0x7fadd8009040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:45.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:45.896+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fade80d0fe0 con 0x7fade8149330 2024-07-17T08:42:46.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.108+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fade815d2c0 con 0x7fade8149330 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.109+0000 7faddf400640 1 -- 172.21.15.44:0/2327655480 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v331) ==== 84+0+65 (secure 0 0 0) 0x7fadd4082500 con 0x7fade8149330 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:46.111 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 msgr2=0x7fadc80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 0x7fadc80791c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000fb0 tx=0x7fadd8009040 comp rx=0 tx=0).stop 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 msgr2=0x7fade81e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fadd402b000 tx=0x7fadd400b6f0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 shutdown_connections 2024-07-17T08:42:46.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fadc8076d00 0x7fadc80791c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fade81f7850 0x7fade81e72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fade8149330 0x7fade81e6d60 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 --2- 172.21.15.44:0/2327655480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade8148f50 0x7fade81f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 >> 172.21.15.44:0/2327655480 conn(0x7fade80cfec0 msgr2=0x7fade8160000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 shutdown_connections 2024-07-17T08:42:46.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.112+0000 7fade7e00640 1 -- 172.21.15.44:0/2327655480 wait complete. 2024-07-17T08:42:46.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:42:46.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:42:46.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.350+0000 7f79a0000640 1 -- 172.21.15.44:0/3788495950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 msgr2=0x7f79980b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.350+0000 7f79a0000640 1 --2- 172.21.15.44:0/3788495950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79980b5200 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f7984009900 tx=0x7f798402a6a0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.350+0000 7f79a0000640 1 -- 172.21.15.44:0/3788495950 shutdown_connections 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.350+0000 7f79a0000640 1 --2- 172.21.15.44:0/3788495950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79980b5740 0x7f79980cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.351+0000 7f79a0000640 1 --2- 172.21.15.44:0/3788495950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79980b5200 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.351+0000 7f79a0000640 1 -- 172.21.15.44:0/3788495950 >> 172.21.15.44:0/3788495950 conn(0x7f79980d00a0 msgr2=0x7f79980d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.351+0000 7f79a0000640 1 -- 172.21.15.44:0/3788495950 shutdown_connections 2024-07-17T08:42:46.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.351+0000 7f79a0000640 1 -- 172.21.15.44:0/3788495950 wait complete. 2024-07-17T08:42:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.352+0000 7f79a0000640 1 Processor -- start 2024-07-17T08:42:46.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.352+0000 7f79a0000640 1 -- start start 2024-07-17T08:42:46.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:46.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79980b5740 0x7f79980ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:46.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f799815d600 0x7f79980cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:46.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79980ce760 con 0x7f79980b2ad0 2024-07-17T08:42:46.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79980ce5e0 con 0x7f79980b5740 2024-07-17T08:42:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.353+0000 7f79a0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79980ce8e0 con 0x7f799815d600 2024-07-17T08:42:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41854/0 (socket says 172.21.15.44:41854) 2024-07-17T08:42:46.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 -- 172.21.15.44:0/3436507832 learned_addr learned my addr 172.21.15.44:0/3436507832 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799e000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f799815d600 0x7f79980cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 -- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f799815d600 msgr2=0x7f79980cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f799815d600 0x7f79980cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 -- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79980b5740 msgr2=0x7f79980ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799cc00640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79980b5740 0x7f79980ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79980b5740 0x7f79980ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 -- 172.21.15.44:0/3436507832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799815f990 con 0x7f79980b2ad0 2024-07-17T08:42:46.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.354+0000 7f799d600640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c3c0 tx=0x7f798c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.355+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c017070 con 0x7f79980b2ad0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.355+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f798c0035f0 con 0x7f79980b2ad0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.355+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799815fc70 con 0x7f79980b2ad0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.355+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c003a40 con 0x7f79980b2ad0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.355+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7998160230 con 0x7f79980b2ad0 2024-07-17T08:42:46.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.356+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79980d2790 con 0x7f79980b2ad0 2024-07-17T08:42:46.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.357+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f798c01a070 con 0x7f79980b2ad0 2024-07-17T08:42:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.358+0000 7f7995600640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 0x7f7978079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.358+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(331..331 src has 1..331) ==== 5853+0+0 (secure 0 0 0) 0x7f798c097880 con 0x7f79980b2ad0 2024-07-17T08:42:46.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.359+0000 7f799cc00640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 0x7f7978079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:46.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.360+0000 7f799cc00640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 0x7f7978079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7984033000 tx=0x7f7984004730 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:46.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.363+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f79980d2790 con 0x7f79980b2ad0 2024-07-17T08:42:46.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:46.578+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f79980ced90 con 0x7f79980b2ad0 2024-07-17T08:42:47.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.157+0000 7f7995600640 1 -- 172.21.15.44:0/3436507832 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v332) ==== 138+0+0 (secure 0 0 0) 0x7f798c060ba0 con 0x7f79980b2ad0 2024-07-17T08:42:47.159 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 msgr2=0x7f7978079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 0x7f7978079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7984033000 tx=0x7f7984004730 comp rx=0 tx=0).stop 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 msgr2=0x7f79981542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c3c0 tx=0x7f798c00c880 comp rx=0 tx=0).stop 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 shutdown_connections 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7978076cd0 0x7f7978079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f799815d600 0x7f79980cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79980b5740 0x7f79980ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 --2- 172.21.15.44:0/3436507832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79980b2ad0 0x7f79981542e0 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 >> 172.21.15.44:0/3436507832 conn(0x7f79980d00a0 msgr2=0x7f799815b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 shutdown_connections 2024-07-17T08:42:47.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.161+0000 7f79a0000640 1 -- 172.21.15.44:0/3436507832 wait complete. 2024-07-17T08:42:47.190 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:42:47.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 -- 172.21.15.44:0/2452938125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc20015d130 msgr2=0x7fc20015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 --2- 172.21.15.44:0/2452938125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc20015d130 0x7fc20015d510 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fc2000cf340 tx=0x7fc1f002a820 comp rx=0 tx=0).stop 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 -- 172.21.15.44:0/2452938125 shutdown_connections 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 --2- 172.21.15.44:0/2452938125 >> v2:172.21.15.175:3300/0 conn(0x7fc2000ce690 0x7fc2000cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 --2- 172.21.15.44:0/2452938125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc20015d130 0x7fc20015d510 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 -- 172.21.15.44:0/2452938125 >> 172.21.15.44:0/2452938125 conn(0x7fc2000b4ff0 msgr2=0x7fc2000b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 -- 172.21.15.44:0/2452938125 shutdown_connections 2024-07-17T08:42:47.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.391+0000 7fc206600640 1 -- 172.21.15.44:0/2452938125 wait complete. 2024-07-17T08:42:47.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.393+0000 7fc206600640 1 Processor -- start 2024-07-17T08:42:47.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.393+0000 7fc206600640 1 -- start start 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc20015d130 0x7fc20015f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc200161be0 0x7fc20015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2000d1a60 con 0x7fc2000ce690 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2000d18e0 con 0x7fc20015d130 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.394+0000 7fc206600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2000d1be0 con 0x7fc200161be0 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41870/0 (socket says 172.21.15.44:41870) 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 -- 172.21.15.44:0/1570827931 learned_addr learned my addr 172.21.15.44:0/1570827931 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205c00640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc200161be0 0x7fc20015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc20015d130 0x7fc20015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:47.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 -- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc200161be0 msgr2=0x7fc20015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:47.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc200161be0 0x7fc20015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 -- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc20015d130 msgr2=0x7fc20015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:47.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc20015d130 0x7fc20015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:47.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.395+0000 7fc205200640 1 -- 172.21.15.44:0/1570827931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20014c510 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc205200640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000f80 tx=0x7fc1f0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f003f020 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f0004040 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0037040 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20014c7f0 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.396+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20015fc10 con 0x7fc2000ce690 2024-07-17T08:42:47.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.397+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2000d2930 con 0x7fc2000ce690 2024-07-17T08:42:47.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.399+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fc1f0042020 con 0x7fc2000ce690 2024-07-17T08:42:47.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.400+0000 7fc1fcc00640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 0x7fc1dc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:47.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.401+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(332..332 src has 1..332) ==== 6223+0+0 (secure 0 0 0) 0x7fc1f00b8dd0 con 0x7fc2000ce690 2024-07-17T08:42:47.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.401+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 0x7fc1dc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.402+0000 7fc1ffe00640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 0x7fc1dc079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2000abbd0 tx=0x7fc1e8009040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:47.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.406+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc1f0049030 con 0x7fc2000ce690 2024-07-17T08:42:47.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:47.621+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc2000d0e10 con 0x7fc2000ce690 2024-07-17T08:42:48.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.165+0000 7fc1fcc00640 1 -- 172.21.15.44:0/1570827931 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v333) ==== 170+0+0 (secure 0 0 0) 0x7fc1f0081e70 con 0x7fc2000ce690 2024-07-17T08:42:48.167 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:42:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 msgr2=0x7fc1dc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 0x7fc1dc079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2000abbd0 tx=0x7fc1e8009040 comp rx=0 tx=0).stop 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 msgr2=0x7fc2000aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000f80 tx=0x7fc1f0032d10 comp rx=0 tx=0).stop 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 shutdown_connections 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fc1dc076cd0 0x7fc1dc079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc200161be0 0x7fc20015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc20015d130 0x7fc20015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 --2- 172.21.15.44:0/1570827931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2000ce690 0x7fc2000aa1e0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.169+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 >> 172.21.15.44:0/1570827931 conn(0x7fc2000b4ff0 msgr2=0x7fc2000b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.172+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 shutdown_connections 2024-07-17T08:42:48.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.172+0000 7fc206600640 1 -- 172.21.15.44:0/1570827931 wait complete. 2024-07-17T08:42:48.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:42:48.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 -- 172.21.15.44:0/1862517405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd058004800 msgr2=0x7fd058004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:48.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 --2- 172.21.15.44:0/1862517405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd058004800 0x7fd058004c80 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009b20 tx=0x7fd058005830 comp rx=0 tx=0).stop 2024-07-17T08:42:48.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd075e00640 1 -- 172.21.15.44:0/1862517405 reap_dead start 2024-07-17T08:42:48.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 -- 172.21.15.44:0/1862517405 shutdown_connections 2024-07-17T08:42:48.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 -- 172.21.15.44:0/1862517405 >> 172.21.15.44:0/1862517405 conn(0x7fd0700cf0d0 msgr2=0x7fd0700cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:48.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 -- 172.21.15.44:0/1862517405 shutdown_connections 2024-07-17T08:42:48.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.401+0000 7fd076800640 1 -- 172.21.15.44:0/1862517405 wait complete. 2024-07-17T08:42:48.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.403+0000 7fd076800640 1 Processor -- start 2024-07-17T08:42:48.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.403+0000 7fd076800640 1 -- start start 2024-07-17T08:42:48.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 0x7fd0701f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd07015a190 0x7fd0700ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd07006ec70 con 0x7fd0700aca20 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700cbb90 con 0x7fd058004800 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.404+0000 7fd076800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd07006edf0 con 0x7fd07015a190 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41902/0 (socket says 172.21.15.44:41902) 2024-07-17T08:42:48.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 -- 172.21.15.44:0/2980001963 learned_addr learned my addr 172.21.15.44:0/2980001963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd075e00640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd07015a190 0x7fd0700ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd075400640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 0x7fd0701f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 -- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd07015a190 msgr2=0x7fd0700ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd07015a190 0x7fd0700ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 -- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 msgr2=0x7fd0701f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 0x7fd0701f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd074a00640 1 -- 172.21.15.44:0/2980001963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0701f3e50 con 0x7fd0700aca20 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.405+0000 7fd075400640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 0x7fd0701f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:48.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd074a00640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009a70 tx=0x7fd05c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:48.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c043070 con 0x7fd0700aca20 2024-07-17T08:42:48.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd05c004140 con 0x7fd0700aca20 2024-07-17T08:42:48.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c038040 con 0x7fd0700aca20 2024-07-17T08:42:48.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0701f7940 con 0x7fd0700aca20 2024-07-17T08:42:48.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.406+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701f7f00 con 0x7fd0700aca20 2024-07-17T08:42:48.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.407+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0700b2e00 con 0x7fd0700aca20 2024-07-17T08:42:48.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.410+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fd05c033860 con 0x7fd0700aca20 2024-07-17T08:42:48.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.410+0000 7fd06d600640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 0x7fd050079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:48.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.411+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(333..333 src has 1..333) ==== 6605+0+0 (secure 0 0 0) 0x7fd05c0b8e40 con 0x7fd0700aca20 2024-07-17T08:42:48.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.411+0000 7fd075400640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 0x7fd050079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.412+0000 7fd075400640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 0x7fd050079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0701f7870 tx=0x7fd064009500 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:48.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.414+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd0700b2e00 con 0x7fd0700aca20 2024-07-17T08:42:48.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-07-17T08:42:48.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:48.666+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd07006f380 con 0x7fd0700aca20 2024-07-17T08:42:49.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.196+0000 7fd06d600640 1 -- 172.21.15.44:0/2980001963 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 45 and data pool 46 v187) ==== 174+0+0 (secure 0 0 0) 0x7fd05c04c020 con 0x7fd0700aca20 2024-07-17T08:42:49.198 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 45 and data pool 46 2024-07-17T08:42:49.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.201+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 msgr2=0x7fd050079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.201+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 0x7fd050079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0701f7870 tx=0x7fd064009500 comp rx=0 tx=0).stop 2024-07-17T08:42:49.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.201+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 msgr2=0x7fd0701f10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.201+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009a70 tx=0x7fd05c033670 comp rx=0 tx=0).stop 2024-07-17T08:42:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 shutdown_connections 2024-07-17T08:42:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fd050076cd0 0x7fd050079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd07015a190 0x7fd0700ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0700aca20 0x7fd0701f10e0 unknown :-1 s=CLOSED pgs=3018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 --2- 172.21.15.44:0/2980001963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd058004800 0x7fd0701f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.203+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 >> 172.21.15.44:0/2980001963 conn(0x7fd0700cf0d0 msgr2=0x7fd070154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:49.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.204+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 shutdown_connections 2024-07-17T08:42:49.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.204+0000 7fd076800640 1 -- 172.21.15.44:0/2980001963 wait complete. 2024-07-17T08:42:49.246 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:49.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 -- 172.21.15.44:0/215561503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 msgr2=0x7fe0bc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 --2- 172.21.15.44:0/215561503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc0b25d0 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005fd0 tx=0x7fe0b402a870 comp rx=0 tx=0).stop 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 -- 172.21.15.44:0/215561503 shutdown_connections 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 --2- 172.21.15.44:0/215561503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc0b25d0 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 --2- 172.21.15.44:0/215561503 >> v2:172.21.15.175:3300/0 conn(0x7fe0bc0ae810 0x7fe0bc15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 -- 172.21.15.44:0/215561503 >> 172.21.15.44:0/215561503 conn(0x7fe0bc0b0fb0 msgr2=0x7fe0bc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.419+0000 7fe0c3600640 1 -- 172.21.15.44:0/215561503 shutdown_connections 2024-07-17T08:42:49.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.420+0000 7fe0c3600640 1 -- 172.21.15.44:0/215561503 wait complete. 2024-07-17T08:42:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.421+0000 7fe0c3600640 1 Processor -- start 2024-07-17T08:42:49.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.421+0000 7fe0c3600640 1 -- start start 2024-07-17T08:42:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 0x7fe0bc153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0bc0ae810 0x7fe0bc1460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc06f040 con 0x7fe0bc0ac930 2024-07-17T08:42:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc06eec0 con 0x7fe0bc0ae810 2024-07-17T08:42:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.422+0000 7fe0c3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc0b2b10 con 0x7fe0bc06ea50 2024-07-17T08:42:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40194/0 (socket says 172.21.15.44:40194) 2024-07-17T08:42:49.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 -- 172.21.15.44:0/1116156650 learned_addr learned my addr 172.21.15.44:0/1116156650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0c1600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0bc0ae810 0x7fe0bc1460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0c0c00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 0x7fe0bc153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 -- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 msgr2=0x7fe0bc153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 0x7fe0bc153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 -- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0bc0ae810 msgr2=0x7fe0bc1460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0bc0ae810 0x7fe0bc1460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0bbe00640 1 -- 172.21.15.44:0/1116156650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc144f70 con 0x7fe0bc0ac930 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.423+0000 7fe0c0c00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 0x7fe0bc153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0bbe00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c760 tx=0x7fe0b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0005070 con 0x7fe0bc0ac930 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0b0005230 con 0x7fe0bc0ac930 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000a640 con 0x7fe0bc0ac930 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc155ca0 con 0x7fe0bc0ac930 2024-07-17T08:42:49.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.424+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc156260 con 0x7fe0bc0ac930 2024-07-17T08:42:49.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.425+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc06a1a0 con 0x7fe0bc0ac930 2024-07-17T08:42:49.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.426+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fe0b000a7e0 con 0x7fe0bc0ac930 2024-07-17T08:42:49.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.428+0000 7fe0b8c00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 0x7fe0a0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.428+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(334..334 src has 1..334) ==== 6705+0+0 (secure 0 0 0) 0x7fe0b00983d0 con 0x7fe0bc0ac930 2024-07-17T08:42:49.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.429+0000 7fe0c0c00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 0x7fe0a0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.430+0000 7fe0c0c00640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 0x7fe0a0079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc1f7720 tx=0x7fe0ac005dd0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:49.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.433+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe0bc15ea50 con 0x7fe0bc0ac930 2024-07-17T08:42:49.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.639+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0bc0b3030 con 0x7fe0bc0ac930 2024-07-17T08:42:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.642+0000 7fe0b8c00640 1 -- 172.21.15.44:0/1116156650 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v334) ==== 74+0+17867 (secure 0 0 0) 0x7fe0b0061390 con 0x7fe0bc0ac930 2024-07-17T08:42:49.645 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:49.645 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":334,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:48.668677+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":45,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:46.606384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":46,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:47.649642+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 msgr2=0x7fe0a0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 0x7fe0a0079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc1f7720 tx=0x7fe0ac005dd0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 msgr2=0x7fe0bc145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c760 tx=0x7fe0b000cc20 comp rx=0 tx=0).stop 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 shutdown_connections 2024-07-17T08:42:49.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fe0a0076cd0 0x7fe0a0079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0bc0ae810 0x7fe0bc1460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0bc0ac930 0x7fe0bc145b80 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 --2- 172.21.15.44:0/1116156650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0bc06ea50 0x7fe0bc153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.645+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 >> 172.21.15.44:0/1116156650 conn(0x7fe0bc0b0fb0 msgr2=0x7fe0bc0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.646+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 shutdown_connections 2024-07-17T08:42:49.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.646+0000 7fe0c3600640 1 -- 172.21.15.44:0/1116156650 wait complete. 2024-07-17T08:42:49.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 -- 172.21.15.44:0/3236607656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc004d40 msgr2=0x7f16cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 --2- 172.21.15.44:0/3236607656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc004d40 0x7f16cc0051c0 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005fd0 tx=0x7f16cc005830 comp rx=0 tx=0).stop 2024-07-17T08:42:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16ea400640 1 -- 172.21.15.44:0/3236607656 reap_dead start 2024-07-17T08:42:49.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 -- 172.21.15.44:0/3236607656 shutdown_connections 2024-07-17T08:42:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 -- 172.21.15.44:0/3236607656 >> 172.21.15.44:0/3236607656 conn(0x7f16e40b0e80 msgr2=0x7f16e40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 -- 172.21.15.44:0/3236607656 shutdown_connections 2024-07-17T08:42:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.880+0000 7f16eae00640 1 -- 172.21.15.44:0/3236607656 wait complete. 2024-07-17T08:42:49.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.882+0000 7f16eae00640 1 Processor -- start 2024-07-17T08:42:49.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.882+0000 7f16eae00640 1 -- start start 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 0x7f16e4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e4139d70 0x7f16e41f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40ac1e0 con 0x7f16e406c1b0 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40ac060 con 0x7f16cc004d40 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16eae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16e4140a00 con 0x7f16e4139d70 2024-07-17T08:42:49.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.883+0000 7f16ea400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40222/0 (socket says 172.21.15.44:40222) 2024-07-17T08:42:49.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 -- 172.21.15.44:0/471545808 learned_addr learned my addr 172.21.15.44:0/471545808 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16e9a00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 0x7f16e4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 -- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e4139d70 msgr2=0x7f16e41f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e4139d70 0x7f16e41f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 -- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 msgr2=0x7f16e4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 0x7f16e4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 -- 172.21.15.44:0/471545808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4157b30 con 0x7f16e406c1b0 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16e9a00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 0x7f16e4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.884+0000 7f16ea400640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f16dc02aea0 tx=0x7f16dc033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.885+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc042070 con 0x7f16e406c1b0 2024-07-17T08:42:49.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.885+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16dc003970 con 0x7f16e406c1b0 2024-07-17T08:42:49.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.885+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc038040 con 0x7f16e406c1b0 2024-07-17T08:42:49.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.885+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e4157e10 con 0x7f16e406c1b0 2024-07-17T08:42:49.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.885+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41f7560 con 0x7f16e406c1b0 2024-07-17T08:42:49.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.886+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4158050 con 0x7f16e406c1b0 2024-07-17T08:42:49.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.887+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f16dc004030 con 0x7f16e406c1b0 2024-07-17T08:42:49.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.889+0000 7f16e1600640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 0x7f16c8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:49.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.889+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(334..334 src has 1..334) ==== 6705+0+0 (secure 0 0 0) 0x7f16dc0b8b90 con 0x7f16e406c1b0 2024-07-17T08:42:49.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.889+0000 7f16e9a00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 0x7f16c8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:49.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.890+0000 7f16e9a00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 0x7f16c8079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16d8001860 tx=0x7f16d8001e40 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:49.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:49.893+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f16e4158050 con 0x7f16e406c1b0 2024-07-17T08:42:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.103+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16e41f7760 con 0x7f16e406c1b0 2024-07-17T08:42:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.106+0000 7f16e1600640 1 -- 172.21.15.44:0/471545808 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v334) ==== 74+0+17867 (secure 0 0 0) 0x7f16e41f7760 con 0x7f16e406c1b0 2024-07-17T08:42:50.108 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:50.108 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":334,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:48.668677+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":45,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:46.606384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":46,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:47.649642+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 msgr2=0x7f16c8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 0x7f16c8079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16d8001860 tx=0x7f16d8001e40 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 msgr2=0x7f16e413ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f16dc02aea0 tx=0x7f16dc033bf0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 shutdown_connections 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f16c8076cd0 0x7f16c8079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16e406c1b0 0x7f16e413ccb0 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16e4139d70 0x7f16e41f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 --2- 172.21.15.44:0/471545808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16cc004d40 0x7f16e4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 >> 172.21.15.44:0/471545808 conn(0x7f16e40b0e80 msgr2=0x7f16e40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 shutdown_connections 2024-07-17T08:42:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.109+0000 7f16eae00640 1 -- 172.21.15.44:0/471545808 wait complete. 2024-07-17T08:42:50.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.343+0000 7f839ec00640 1 -- 172.21.15.44:0/2163339099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 msgr2=0x7f839815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.343+0000 7f839ec00640 1 --2- 172.21.15.44:0/2163339099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f839815c140 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f8388005fd0 tx=0x7f838802a980 comp rx=0 tx=0).stop 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 -- 172.21.15.44:0/2163339099 shutdown_connections 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 --2- 172.21.15.44:0/2163339099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 0x7f83980cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 --2- 172.21.15.44:0/2163339099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f839815c140 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 -- 172.21.15.44:0/2163339099 >> 172.21.15.44:0/2163339099 conn(0x7f83980cf0d0 msgr2=0x7f83980cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 -- 172.21.15.44:0/2163339099 shutdown_connections 2024-07-17T08:42:50.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.344+0000 7f839ec00640 1 -- 172.21.15.44:0/2163339099 wait complete. 2024-07-17T08:42:50.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.347+0000 7f839ec00640 1 Processor -- start 2024-07-17T08:42:50.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.347+0000 7f839ec00640 1 -- start start 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 0x7f8398142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83980cd0b0 0x7f83980713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83980cb740 con 0x7f8398151f20 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83980cb5c0 con 0x7f83980cd0b0 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83980cb8c0 con 0x7f839806f380 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:50.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40246/0 (socket says 172.21.15.44:40246) 2024-07-17T08:42:50.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.348+0000 7f839e200640 1 -- 172.21.15.44:0/775144247 learned_addr learned my addr 172.21.15.44:0/775144247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839d800640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 0x7f8398142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839ce00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83980cd0b0 0x7f83980713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 -- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 msgr2=0x7f8398142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 0x7f8398142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 -- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83980cd0b0 msgr2=0x7f83980713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83980cd0b0 0x7f83980713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 -- 172.21.15.44:0/775144247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8398073e80 con 0x7f8398151f20 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f839e200640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f839000c9a0 tx=0x7f839000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.349+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390007bb0 con 0x7f8398151f20 2024-07-17T08:42:50.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.350+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8390007e40 con 0x7f8398151f20 2024-07-17T08:42:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.350+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390005280 con 0x7f8398151f20 2024-07-17T08:42:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.350+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8398155590 con 0x7f8398151f20 2024-07-17T08:42:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.350+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8398155870 con 0x7f8398151f20 2024-07-17T08:42:50.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.351+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83980cb080 con 0x7f8398151f20 2024-07-17T08:42:50.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.352+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f8390005420 con 0x7f8398151f20 2024-07-17T08:42:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.353+0000 7f8395600640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 0x7f8378079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.353+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f8390098b10 con 0x7f8398151f20 2024-07-17T08:42:50.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.354+0000 7f839d800640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 0x7f8378079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:50.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.355+0000 7f839d800640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 0x7f8378079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8388035000 tx=0x7f8388004ab0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:50.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.358+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8390014030 con 0x7f8398151f20 2024-07-17T08:42:50.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:50.611+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f839815afc0 con 0x7f8398151f20 2024-07-17T08:42:51.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.255+0000 7f8395600640 1 -- 172.21.15.44:0/775144247 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v190) ==== 118+0+0 (secure 0 0 0) 0x7f8390061930 con 0x7f8398151f20 2024-07-17T08:42:51.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.258+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 msgr2=0x7f8378079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:51.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.258+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 0x7f8378079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8388035000 tx=0x7f8388004ab0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 msgr2=0x7f8398141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:51.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f839000c9a0 tx=0x7f839000ce60 comp rx=0 tx=0).stop 2024-07-17T08:42:51.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 shutdown_connections 2024-07-17T08:42:51.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f8378076cd0 0x7f8378079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8398151f20 0x7f8398141de0 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83980cd0b0 0x7f83980713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 --2- 172.21.15.44:0/775144247 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f839806f380 0x7f8398142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 >> 172.21.15.44:0/775144247 conn(0x7f83980cf0d0 msgr2=0x7f83980cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:51.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 shutdown_connections 2024-07-17T08:42:51.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.259+0000 7f839ec00640 1 -- 172.21.15.44:0/775144247 wait complete. 2024-07-17T08:42:51.285 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:42:51.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:42:51.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 -- 172.21.15.44:0/67628644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006ee80 msgr2=0x7ff4200b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:51.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 --2- 172.21.15.44:0/67628644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006ee80 0x7ff4200b23a0 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7ff4200b1d10 tx=0x7ff40c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 -- 172.21.15.44:0/67628644 shutdown_connections 2024-07-17T08:42:51.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 --2- 172.21.15.44:0/67628644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006ee80 0x7ff4200b23a0 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 --2- 172.21.15.44:0/67628644 >> v2:172.21.15.175:3300/0 conn(0x7ff420150710 0x7ff4200ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 -- 172.21.15.44:0/67628644 >> 172.21.15.44:0/67628644 conn(0x7ff4200cf0d0 msgr2=0x7ff4200cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:51.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 -- 172.21.15.44:0/67628644 shutdown_connections 2024-07-17T08:42:51.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.501+0000 7ff41f200640 1 -- 172.21.15.44:0/67628644 wait complete. 2024-07-17T08:42:51.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.503+0000 7ff41f200640 1 Processor -- start 2024-07-17T08:42:51.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.503+0000 7ff41f200640 1 -- start start 2024-07-17T08:42:51.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff42006ee80 0x7ff42015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:51.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:51.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 0x7ff4200cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4200b2c70 con 0x7ff42006f2f0 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4200b2af0 con 0x7ff420150710 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff42015da90 con 0x7ff42006ee80 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40268/0 (socket says 172.21.15.44:40268) 2024-07-17T08:42:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.504+0000 7ff41d400640 1 -- 172.21.15.44:0/1108883282 learned_addr learned my addr 172.21.15.44:0/1108883282 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:51.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41de00640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff42006ee80 0x7ff42015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:51.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41e800640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 0x7ff4200cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:51.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 -- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff42006ee80 msgr2=0x7ff42015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:51.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff42006ee80 0x7ff42015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 -- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 msgr2=0x7ff4200cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:51.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 0x7ff4200cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:51.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 -- 172.21.15.44:0/1108883282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4201f66a0 con 0x7ff42006f2f0 2024-07-17T08:42:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41e800640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 0x7ff4200cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.505+0000 7ff41d400640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0099c0 tx=0x7ff40c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.506+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c03f070 con 0x7ff42006f2f0 2024-07-17T08:42:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.506+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff40c004890 con 0x7ff42006f2f0 2024-07-17T08:42:51.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.506+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c035030 con 0x7ff42006f2f0 2024-07-17T08:42:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.506+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42014bf70 con 0x7ff42006f2f0 2024-07-17T08:42:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.506+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42014c530 con 0x7ff42006f2f0 2024-07-17T08:42:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.507+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4200ae070 con 0x7ff42006f2f0 2024-07-17T08:42:51.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.508+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7ff40c042070 con 0x7ff42006f2f0 2024-07-17T08:42:51.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.509+0000 7ff416000640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 0x7ff400079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:51.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.509+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7ff40c0b9830 con 0x7ff42006f2f0 2024-07-17T08:42:51.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.510+0000 7ff41de00640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 0x7ff400079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:51.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.511+0000 7ff41de00640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 0x7ff400079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff410005d70 tx=0x7ff410005d00 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:51.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.514+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4200ae070 con 0x7ff42006f2f0 2024-07-17T08:42:51.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:51.769+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff4200ae900 con 0x7ff42006f2f0 2024-07-17T08:42:52.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.258+0000 7ff416000640 1 -- 172.21.15.44:0/1108883282 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v191) ==== 122+0+0 (secure 0 0 0) 0x7ff4200ae900 con 0x7ff42006f2f0 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 msgr2=0x7ff400079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 0x7ff400079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff410005d70 tx=0x7ff410005d00 comp rx=0 tx=0).stop 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 msgr2=0x7ff420145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0099c0 tx=0x7ff40c0039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 shutdown_connections 2024-07-17T08:42:52.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7ff400076cd0 0x7ff400079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff420150710 0x7ff4200cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff42006f2f0 0x7ff420145460 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 --2- 172.21.15.44:0/1108883282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff42006ee80 0x7ff42015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 >> 172.21.15.44:0/1108883282 conn(0x7ff4200cf0d0 msgr2=0x7ff42015b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.262+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 shutdown_connections 2024-07-17T08:42:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.263+0000 7ff41f200640 1 -- 172.21.15.44:0/1108883282 wait complete. 2024-07-17T08:42:52.287 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:42:52.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 -- 172.21.15.44:0/1754076241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cec004800 msgr2=0x7f2cec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/1754076241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2cec004800 0x7f2cec004c80 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009b50 tx=0x7f2cec005830 comp rx=0 tx=0).stop 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d07400640 1 -- 172.21.15.44:0/1754076241 reap_dead start 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 -- 172.21.15.44:0/1754076241 shutdown_connections 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 -- 172.21.15.44:0/1754076241 >> 172.21.15.44:0/1754076241 conn(0x7f2d080b0fb0 msgr2=0x7f2d080b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:52.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 -- 172.21.15.44:0/1754076241 shutdown_connections 2024-07-17T08:42:52.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.503+0000 7f2d0dc00640 1 -- 172.21.15.44:0/1754076241 wait complete. 2024-07-17T08:42:52.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.505+0000 7f2d0dc00640 1 Processor -- start 2024-07-17T08:42:52.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.505+0000 7f2d0dc00640 1 -- start start 2024-07-17T08:42:52.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 0x7f2d081f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 0x7f2d0815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080ad790 con 0x7f2d0814d790 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d080ce8b0 con 0x7f2d081f7750 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d0dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d080ad910 con 0x7f2cec004800 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.506+0000 7f2d06000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40288/0 (socket says 172.21.15.44:40288) 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 -- 172.21.15.44:0/3558344429 learned_addr learned my addr 172.21.15.44:0/3558344429 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d07400640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 0x7f2d0815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06a00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 0x7f2d081f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 -- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 msgr2=0x7f2d081f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 0x7f2d081f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 -- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 msgr2=0x7f2d0815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 0x7f2d0815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 -- 172.21.15.44:0/3558344429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d081f8030 con 0x7f2d0814d790 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d07400640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 0x7f2d0815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06a00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 0x7f2d081f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.507+0000 7f2d06000640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4036040 tx=0x7f2cf4033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.508+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4038040 con 0x7f2d0814d790 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.508+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cf4005020 con 0x7f2d0814d790 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.508+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4005470 con 0x7f2d0814d790 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.508+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d081f8310 con 0x7f2d0814d790 2024-07-17T08:42:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.508+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d081f5040 con 0x7f2d0814d790 2024-07-17T08:42:52.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.509+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d080ac220 con 0x7f2d0814d790 2024-07-17T08:42:52.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.510+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f2cf4045020 con 0x7f2d0814d790 2024-07-17T08:42:52.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.511+0000 7f2cfea00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 0x7f2ce80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:52.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.511+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f2cf40b90e0 con 0x7f2d0814d790 2024-07-17T08:42:52.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.512+0000 7f2d06a00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 0x7f2ce80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:52.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.513+0000 7f2d06a00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 0x7f2ce80791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005b60 tx=0x7f2cf8005f90 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:52.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.516+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2cf40820b0 con 0x7f2d0814d790 2024-07-17T08:42:52.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.766+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d080cfc40 con 0x7f2d0814d790 2024-07-17T08:42:52.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.768+0000 7f2cfea00640 1 -- 172.21.15.44:0/3558344429 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7f2cf408a2f0 con 0x7f2d0814d790 2024-07-17T08:42:52.770 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:52.770 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:42:52.770 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 msgr2=0x7f2ce80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 0x7f2ce80791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005b60 tx=0x7f2cf8005f90 comp rx=0 tx=0).stop 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 msgr2=0x7f2d08147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4036040 tx=0x7f2cf4033650 comp rx=0 tx=0).stop 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 shutdown_connections 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f2ce8076d00 0x7f2ce80791c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d081f7750 0x7f2d0815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d0814d790 0x7f2d08147760 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 --2- 172.21.15.44:0/3558344429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2cec004800 0x7f2d081f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 >> 172.21.15.44:0/3558344429 conn(0x7f2d080b0fb0 msgr2=0x7f2d080b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 shutdown_connections 2024-07-17T08:42:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:52.771+0000 7f2d0dc00640 1 -- 172.21.15.44:0/3558344429 wait complete. 2024-07-17T08:42:52.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 -- 172.21.15.44:0/2184695232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040cfa90 msgr2=0x7f490406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 --2- 172.21.15.44:0/2184695232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040cfa90 0x7f490406e060 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f48ec005fd0 tx=0x7f48ec02aa00 comp rx=0 tx=0).stop 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 -- 172.21.15.44:0/2184695232 shutdown_connections 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 --2- 172.21.15.44:0/2184695232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f490406f4b0 0x7f4904159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 --2- 172.21.15.44:0/2184695232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49040cfa90 0x7f490406e060 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 -- 172.21.15.44:0/2184695232 >> 172.21.15.44:0/2184695232 conn(0x7f49040b0e80 msgr2=0x7f49040b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:53.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 -- 172.21.15.44:0/2184695232 shutdown_connections 2024-07-17T08:42:53.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.009+0000 7f4908e00640 1 -- 172.21.15.44:0/2184695232 wait complete. 2024-07-17T08:42:53.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.011+0000 7f4908e00640 1 Processor -- start 2024-07-17T08:42:53.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.011+0000 7f4908e00640 1 -- start start 2024-07-17T08:42:53.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 0x7f490415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49040cfa90 0x7f49040ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f49040b2120 con 0x7f490406f4b0 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49040b1fa0 con 0x7f490409b300 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4908e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49040b22a0 con 0x7f49040cfa90 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4903400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4903400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40296/0 (socket says 172.21.15.44:40296) 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.012+0000 7f4903400640 1 -- 172.21.15.44:0/2455807953 learned_addr learned my addr 172.21.15.44:0/2455807953 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4902a00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 0x7f490415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49040cfa90 0x7f49040ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 -- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49040cfa90 msgr2=0x7f49040ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49040cfa90 0x7f49040ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 -- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 msgr2=0x7f490415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 0x7f490415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 -- 172.21.15.44:0/2455807953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49040cbd80 con 0x7f490406f4b0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4902a00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 0x7f490415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4903400640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c430 tx=0x7f48f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4011790 con 0x7f490406f4b0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f4017070 con 0x7f490406f4b0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4011d00 con 0x7f490406f4b0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.013+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49040cc040 con 0x7f490406f4b0 2024-07-17T08:42:53.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.014+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49040cb480 con 0x7f490406f4b0 2024-07-17T08:42:53.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.015+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904159800 con 0x7f490406f4b0 2024-07-17T08:42:53.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.016+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f48f401c460 con 0x7f490406f4b0 2024-07-17T08:42:53.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.017+0000 7f48fb400640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 0x7f48e0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.018+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f48f40987d0 con 0x7f490406f4b0 2024-07-17T08:42:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.018+0000 7f4902a00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 0x7f48e0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.019+0000 7f4902a00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 0x7f48e0079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48ec035000 tx=0x7f48ec004ab0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:53.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.022+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48f4061790 con 0x7f490406f4b0 2024-07-17T08:42:53.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.236+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49040b06f0 con 0x7f490406f4b0 2024-07-17T08:42:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.239+0000 7f48fb400640 1 -- 172.21.15.44:0/2455807953 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v335) ==== 74+0+17854 (secure 0 0 0) 0x7f48f4061090 con 0x7f490406f4b0 2024-07-17T08:42:53.241 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:53.241 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":335,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:42:50.197474+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":45,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:46.606384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":46,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:47.649642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"335","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:42:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 msgr2=0x7f48e0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 0x7f48e0079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48ec035000 tx=0x7f48ec004ab0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 msgr2=0x7f49040ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c430 tx=0x7f48f400c8f0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 shutdown_connections 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f48e0076cd0 0x7f48e0079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49040cfa90 0x7f49040ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f490409b300 0x7f490415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 --2- 172.21.15.44:0/2455807953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f490406f4b0 0x7f49040ce3c0 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.242+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 >> 172.21.15.44:0/2455807953 conn(0x7f49040b0e80 msgr2=0x7f49040b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.243+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 shutdown_connections 2024-07-17T08:42:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.243+0000 7f4908e00640 1 -- 172.21.15.44:0/2455807953 wait complete. 2024-07-17T08:42:53.266 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:42:53.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 -- 172.21.15.44:0/497835352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 msgr2=0x7f1a4809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 --2- 172.21.15.44:0/497835352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a4809ca20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005fd0 tx=0x7f1a3802a540 comp rx=0 tx=0).stop 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 -- 172.21.15.44:0/497835352 shutdown_connections 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 --2- 172.21.15.44:0/497835352 >> v2:172.21.15.175:3300/0 conn(0x7f1a480cf9e0 0x7f1a480ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 --2- 172.21.15.44:0/497835352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a4809ca20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 -- 172.21.15.44:0/497835352 >> 172.21.15.44:0/497835352 conn(0x7f1a480b0e80 msgr2=0x7f1a480b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:53.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 -- 172.21.15.44:0/497835352 shutdown_connections 2024-07-17T08:42:53.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.475+0000 7f1a4e200640 1 -- 172.21.15.44:0/497835352 wait complete. 2024-07-17T08:42:53.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.477+0000 7f1a4e200640 1 Processor -- start 2024-07-17T08:42:53.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.477+0000 7f1a4e200640 1 -- start start 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a480cf9e0 0x7f1a4815f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a48159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a480acee0 con 0x7f1a480b1fa0 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a480acd60 con 0x7f1a480cf9e0 2024-07-17T08:42:53.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.478+0000 7f1a4e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a480ad060 con 0x7f1a4815d240 2024-07-17T08:42:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40302/0 (socket says 172.21.15.44:40302) 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 -- 172.21.15.44:0/3801070055 learned_addr learned my addr 172.21.15.44:0/3801070055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a47e00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a480cf9e0 0x7f1a4815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4d800640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a48159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 -- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 msgr2=0x7f1a48159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a48159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 -- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a480cf9e0 msgr2=0x7f1a4815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a480cf9e0 0x7f1a4815f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 -- 172.21.15.44:0/3801070055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a481602a0 con 0x7f1a480b1fa0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.479+0000 7f1a4ce00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034040 tx=0x7f1a380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.480+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3803e030 con 0x7f1a480b1fa0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.480+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a38003d90 con 0x7f1a480b1fa0 2024-07-17T08:42:53.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.480+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38035040 con 0x7f1a480b1fa0 2024-07-17T08:42:53.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.480+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a48155490 con 0x7f1a480b1fa0 2024-07-17T08:42:53.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.480+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a48155a50 con 0x7f1a480b1fa0 2024-07-17T08:42:53.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.481+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a48050670 con 0x7f1a480b1fa0 2024-07-17T08:42:53.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.482+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f1a38041030 con 0x7f1a480b1fa0 2024-07-17T08:42:53.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.484+0000 7f1a44c00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 0x7f1a28079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.484+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f1a380b8f80 con 0x7f1a480b1fa0 2024-07-17T08:42:53.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.485+0000 7f1a47e00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 0x7f1a28079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:53.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.486+0000 7f1a47e00640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 0x7f1a28079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a481f7770 tx=0x7f1a34009040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:53.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.489+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1a48050670 con 0x7f1a480b1fa0 2024-07-17T08:42:53.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.702+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 --> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a480ad510 con 0x7f1a28076cd0 2024-07-17T08:42:53.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.717+0000 7f1a44c00640 1 -- 172.21.15.44:0/3801070055 <== mgr.9539 v2:172.21.15.175:6834/1233420839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7f1a480ad510 con 0x7f1a28076cd0 2024-07-17T08:42:53.726 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:53.727 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:42:53.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 msgr2=0x7f1a28079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 0x7f1a28079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a481f7770 tx=0x7f1a34009040 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 msgr2=0x7f1a48150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034040 tx=0x7f1a380038a0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 shutdown_connections 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f1a28076cd0 0x7f1a28079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a4815d240 0x7f1a48159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a480cf9e0 0x7f1a4815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 --2- 172.21.15.44:0/3801070055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480b1fa0 0x7f1a48150770 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 >> 172.21.15.44:0/3801070055 conn(0x7f1a480b0e80 msgr2=0x7f1a4806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 shutdown_connections 2024-07-17T08:42:53.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:53.724+0000 7f1a4e200640 1 -- 172.21.15.44:0/3801070055 wait complete. 2024-07-17T08:42:53.746 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-07-17T08:42:52.939845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":561,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4676,"ondisk_log_size":4676,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":480944,"kb_used_data":34360,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754493776,"statfs":{"total":773094113280,"available":772601626624,"internally_reserved":0,"allocated":35184640,"data_stored":19902133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.779038"},"pg_stats":[{"pgid":"46.3f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247536+0000","last_change":"2024-07-17T08:42:49.242624+0000","last_active":"2024-07-17T08:42:49.247536+0000","last_peered":"2024-07-17T08:42:49.247536+0000","last_clean":"2024-07-17T08:42:49.247536+0000","last_became_active":"2024-07-17T08:42:49.242347+0000","last_became_peered":"2024-07-17T08:42:49.242347+0000","last_unstale":"2024-07-17T08:42:49.247536+0000","last_undegraded":"2024-07-17T08:42:49.247536+0000","last_fullsized":"2024-07-17T08:42:49.247536+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:20.190911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.3e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238774+0000","last_change":"2024-07-17T08:42:49.233209+0000","last_active":"2024-07-17T08:42:49.238774+0000","last_peered":"2024-07-17T08:42:49.238774+0000","last_clean":"2024-07-17T08:42:49.238774+0000","last_became_active":"2024-07-17T08:42:49.230388+0000","last_became_peered":"2024-07-17T08:42:49.230388+0000","last_unstale":"2024-07-17T08:42:49.238774+0000","last_undegraded":"2024-07-17T08:42:49.238774+0000","last_fullsized":"2024-07-17T08:42:49.238774+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:49.107565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.3d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243485+0000","last_change":"2024-07-17T08:42:49.239344+0000","last_active":"2024-07-17T08:42:49.243485+0000","last_peered":"2024-07-17T08:42:49.243485+0000","last_clean":"2024-07-17T08:42:49.243485+0000","last_became_active":"2024-07-17T08:42:49.238220+0000","last_became_peered":"2024-07-17T08:42:49.238220+0000","last_unstale":"2024-07-17T08:42:49.243485+0000","last_undegraded":"2024-07-17T08:42:49.243485+0000","last_fullsized":"2024-07-17T08:42:49.243485+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:23.961958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.3c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249879+0000","last_change":"2024-07-17T08:42:49.246947+0000","last_active":"2024-07-17T08:42:49.249879+0000","last_peered":"2024-07-17T08:42:49.249879+0000","last_clean":"2024-07-17T08:42:49.249879+0000","last_became_active":"2024-07-17T08:42:49.246223+0000","last_became_peered":"2024-07-17T08:42:49.246223+0000","last_unstale":"2024-07-17T08:42:49.249879+0000","last_undegraded":"2024-07-17T08:42:49.249879+0000","last_fullsized":"2024-07-17T08:42:49.249879+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:53.804393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.3b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213991+0000","last_change":"2024-07-17T08:42:49.211782+0000","last_active":"2024-07-17T08:42:49.213991+0000","last_peered":"2024-07-17T08:42:49.213991+0000","last_clean":"2024-07-17T08:42:49.213991+0000","last_became_active":"2024-07-17T08:42:49.211170+0000","last_became_peered":"2024-07-17T08:42:49.211170+0000","last_unstale":"2024-07-17T08:42:49.213991+0000","last_undegraded":"2024-07-17T08:42:49.213991+0000","last_fullsized":"2024-07-17T08:42:49.213991+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:25.288474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.3a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235059+0000","last_change":"2024-07-17T08:42:49.233445+0000","last_active":"2024-07-17T08:42:49.235059+0000","last_peered":"2024-07-17T08:42:49.235059+0000","last_clean":"2024-07-17T08:42:49.235059+0000","last_became_active":"2024-07-17T08:42:49.233179+0000","last_became_peered":"2024-07-17T08:42:49.233179+0000","last_unstale":"2024-07-17T08:42:49.235059+0000","last_undegraded":"2024-07-17T08:42:49.235059+0000","last_fullsized":"2024-07-17T08:42:49.235059+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:59.303095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.39","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.225472+0000","last_change":"2024-07-17T08:42:49.224432+0000","last_active":"2024-07-17T08:42:49.225472+0000","last_peered":"2024-07-17T08:42:49.225472+0000","last_clean":"2024-07-17T08:42:49.225472+0000","last_became_active":"2024-07-17T08:42:49.224031+0000","last_became_peered":"2024-07-17T08:42:49.224031+0000","last_unstale":"2024-07-17T08:42:49.225472+0000","last_undegraded":"2024-07-17T08:42:49.225472+0000","last_fullsized":"2024-07-17T08:42:49.225472+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:55.436467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.38","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249804+0000","last_change":"2024-07-17T08:42:49.240760+0000","last_active":"2024-07-17T08:42:49.249804+0000","last_peered":"2024-07-17T08:42:49.249804+0000","last_clean":"2024-07-17T08:42:49.249804+0000","last_became_active":"2024-07-17T08:42:49.237356+0000","last_became_peered":"2024-07-17T08:42:49.237356+0000","last_unstale":"2024-07-17T08:42:49.249804+0000","last_undegraded":"2024-07-17T08:42:49.249804+0000","last_fullsized":"2024-07-17T08:42:49.249804+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:10:40.357282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.37","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248616+0000","last_change":"2024-07-17T08:42:49.245877+0000","last_active":"2024-07-17T08:42:49.248616+0000","last_peered":"2024-07-17T08:42:49.248616+0000","last_clean":"2024-07-17T08:42:49.248616+0000","last_became_active":"2024-07-17T08:42:49.244328+0000","last_became_peered":"2024-07-17T08:42:49.244328+0000","last_unstale":"2024-07-17T08:42:49.248616+0000","last_undegraded":"2024-07-17T08:42:49.248616+0000","last_fullsized":"2024-07-17T08:42:49.248616+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:54.776716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.36","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251419+0000","last_change":"2024-07-17T08:42:49.250121+0000","last_active":"2024-07-17T08:42:49.251419+0000","last_peered":"2024-07-17T08:42:49.251419+0000","last_clean":"2024-07-17T08:42:49.251419+0000","last_became_active":"2024-07-17T08:42:49.249198+0000","last_became_peered":"2024-07-17T08:42:49.249198+0000","last_unstale":"2024-07-17T08:42:49.251419+0000","last_undegraded":"2024-07-17T08:42:49.251419+0000","last_fullsized":"2024-07-17T08:42:49.251419+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:32:36.376662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.35","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.217190+0000","last_change":"2024-07-17T08:42:49.215764+0000","last_active":"2024-07-17T08:42:49.217190+0000","last_peered":"2024-07-17T08:42:49.217190+0000","last_clean":"2024-07-17T08:42:49.217190+0000","last_became_active":"2024-07-17T08:42:49.215215+0000","last_became_peered":"2024-07-17T08:42:49.215215+0000","last_unstale":"2024-07-17T08:42:49.217190+0000","last_undegraded":"2024-07-17T08:42:49.217190+0000","last_fullsized":"2024-07-17T08:42:49.217190+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:32.026461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.34","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252063+0000","last_change":"2024-07-17T08:42:49.246619+0000","last_active":"2024-07-17T08:42:49.252063+0000","last_peered":"2024-07-17T08:42:49.252063+0000","last_clean":"2024-07-17T08:42:49.252063+0000","last_became_active":"2024-07-17T08:42:49.244842+0000","last_became_peered":"2024-07-17T08:42:49.244842+0000","last_unstale":"2024-07-17T08:42:49.252063+0000","last_undegraded":"2024-07-17T08:42:49.252063+0000","last_fullsized":"2024-07-17T08:42:49.252063+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:55:02.297284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.33","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238816+0000","last_change":"2024-07-17T08:42:49.233160+0000","last_active":"2024-07-17T08:42:49.238816+0000","last_peered":"2024-07-17T08:42:49.238816+0000","last_clean":"2024-07-17T08:42:49.238816+0000","last_became_active":"2024-07-17T08:42:49.232863+0000","last_became_peered":"2024-07-17T08:42:49.232863+0000","last_unstale":"2024-07-17T08:42:49.238816+0000","last_undegraded":"2024-07-17T08:42:49.238816+0000","last_fullsized":"2024-07-17T08:42:49.238816+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:00.939094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.32","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.233689+0000","last_change":"2024-07-17T08:42:49.231262+0000","last_active":"2024-07-17T08:42:49.233689+0000","last_peered":"2024-07-17T08:42:49.233689+0000","last_clean":"2024-07-17T08:42:49.233689+0000","last_became_active":"2024-07-17T08:42:49.230973+0000","last_became_peered":"2024-07-17T08:42:49.230973+0000","last_unstale":"2024-07-17T08:42:49.233689+0000","last_undegraded":"2024-07-17T08:42:49.233689+0000","last_fullsized":"2024-07-17T08:42:49.233689+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:26:36.150258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.31","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229358+0000","last_change":"2024-07-17T08:42:49.226547+0000","last_active":"2024-07-17T08:42:49.229358+0000","last_peered":"2024-07-17T08:42:49.229358+0000","last_clean":"2024-07-17T08:42:49.229358+0000","last_became_active":"2024-07-17T08:42:49.226196+0000","last_became_peered":"2024-07-17T08:42:49.226196+0000","last_unstale":"2024-07-17T08:42:49.229358+0000","last_undegraded":"2024-07-17T08:42:49.229358+0000","last_fullsized":"2024-07-17T08:42:49.229358+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:07.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.30","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234848+0000","last_change":"2024-07-17T08:42:49.230841+0000","last_active":"2024-07-17T08:42:49.234848+0000","last_peered":"2024-07-17T08:42:49.234848+0000","last_clean":"2024-07-17T08:42:49.234848+0000","last_became_active":"2024-07-17T08:42:49.230560+0000","last_became_peered":"2024-07-17T08:42:49.230560+0000","last_unstale":"2024-07-17T08:42:49.234848+0000","last_undegraded":"2024-07-17T08:42:49.234848+0000","last_fullsized":"2024-07-17T08:42:49.234848+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:23.998448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.2e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243682+0000","last_change":"2024-07-17T08:42:49.240627+0000","last_active":"2024-07-17T08:42:49.243682+0000","last_peered":"2024-07-17T08:42:49.243682+0000","last_clean":"2024-07-17T08:42:49.243682+0000","last_became_active":"2024-07-17T08:42:49.240334+0000","last_became_peered":"2024-07-17T08:42:49.240334+0000","last_unstale":"2024-07-17T08:42:49.243682+0000","last_undegraded":"2024-07-17T08:42:49.243682+0000","last_fullsized":"2024-07-17T08:42:49.243682+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:54:08.252838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.2d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.246618+0000","last_change":"2024-07-17T08:42:49.242068+0000","last_active":"2024-07-17T08:42:49.246618+0000","last_peered":"2024-07-17T08:42:49.246618+0000","last_clean":"2024-07-17T08:42:49.246618+0000","last_became_active":"2024-07-17T08:42:49.237514+0000","last_became_peered":"2024-07-17T08:42:49.237514+0000","last_unstale":"2024-07-17T08:42:49.246618+0000","last_undegraded":"2024-07-17T08:42:49.246618+0000","last_fullsized":"2024-07-17T08:42:49.246618+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:21.501649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.2c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.223019+0000","last_change":"2024-07-17T08:42:49.215499+0000","last_active":"2024-07-17T08:42:49.223019+0000","last_peered":"2024-07-17T08:42:49.223019+0000","last_clean":"2024-07-17T08:42:49.223019+0000","last_became_active":"2024-07-17T08:42:49.214503+0000","last_became_peered":"2024-07-17T08:42:49.214503+0000","last_unstale":"2024-07-17T08:42:49.223019+0000","last_undegraded":"2024-07-17T08:42:49.223019+0000","last_fullsized":"2024-07-17T08:42:49.223019+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:29.103470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.2b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245544+0000","last_change":"2024-07-17T08:42:49.241313+0000","last_active":"2024-07-17T08:42:49.245544+0000","last_peered":"2024-07-17T08:42:49.245544+0000","last_clean":"2024-07-17T08:42:49.245544+0000","last_became_active":"2024-07-17T08:42:49.241017+0000","last_became_peered":"2024-07-17T08:42:49.241017+0000","last_unstale":"2024-07-17T08:42:49.245544+0000","last_undegraded":"2024-07-17T08:42:49.245544+0000","last_fullsized":"2024-07-17T08:42:49.245544+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:24.397193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.2a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.216211+0000","last_change":"2024-07-17T08:42:49.214125+0000","last_active":"2024-07-17T08:42:49.216211+0000","last_peered":"2024-07-17T08:42:49.216211+0000","last_clean":"2024-07-17T08:42:49.216211+0000","last_became_active":"2024-07-17T08:42:49.213847+0000","last_became_peered":"2024-07-17T08:42:49.213847+0000","last_unstale":"2024-07-17T08:42:49.216211+0000","last_undegraded":"2024-07-17T08:42:49.216211+0000","last_fullsized":"2024-07-17T08:42:49.216211+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:43.713168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.29","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.226765+0000","last_change":"2024-07-17T08:42:49.222751+0000","last_active":"2024-07-17T08:42:49.226765+0000","last_peered":"2024-07-17T08:42:49.226765+0000","last_clean":"2024-07-17T08:42:49.226765+0000","last_became_active":"2024-07-17T08:42:49.221500+0000","last_became_peered":"2024-07-17T08:42:49.221500+0000","last_unstale":"2024-07-17T08:42:49.226765+0000","last_undegraded":"2024-07-17T08:42:49.226765+0000","last_fullsized":"2024-07-17T08:42:49.226765+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:43.549003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.28","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243612+0000","last_change":"2024-07-17T08:42:49.239433+0000","last_active":"2024-07-17T08:42:49.243612+0000","last_peered":"2024-07-17T08:42:49.243612+0000","last_clean":"2024-07-17T08:42:49.243612+0000","last_became_active":"2024-07-17T08:42:49.239147+0000","last_became_peered":"2024-07-17T08:42:49.239147+0000","last_unstale":"2024-07-17T08:42:49.243612+0000","last_undegraded":"2024-07-17T08:42:49.243612+0000","last_fullsized":"2024-07-17T08:42:49.243612+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:47.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.27","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247620+0000","last_change":"2024-07-17T08:42:49.244353+0000","last_active":"2024-07-17T08:42:49.247620+0000","last_peered":"2024-07-17T08:42:49.247620+0000","last_clean":"2024-07-17T08:42:49.247620+0000","last_became_active":"2024-07-17T08:42:49.244031+0000","last_became_peered":"2024-07-17T08:42:49.244031+0000","last_unstale":"2024-07-17T08:42:49.247620+0000","last_undegraded":"2024-07-17T08:42:49.247620+0000","last_fullsized":"2024-07-17T08:42:49.247620+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:48.496046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.26","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235487+0000","last_change":"2024-07-17T08:42:49.233478+0000","last_active":"2024-07-17T08:42:49.235487+0000","last_peered":"2024-07-17T08:42:49.235487+0000","last_clean":"2024-07-17T08:42:49.235487+0000","last_became_active":"2024-07-17T08:42:49.232250+0000","last_became_peered":"2024-07-17T08:42:49.232250+0000","last_unstale":"2024-07-17T08:42:49.235487+0000","last_undegraded":"2024-07-17T08:42:49.235487+0000","last_fullsized":"2024-07-17T08:42:49.235487+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:34.252723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.25","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.246592+0000","last_change":"2024-07-17T08:42:49.237799+0000","last_active":"2024-07-17T08:42:49.246592+0000","last_peered":"2024-07-17T08:42:49.246592+0000","last_clean":"2024-07-17T08:42:49.246592+0000","last_became_active":"2024-07-17T08:42:49.236754+0000","last_became_peered":"2024-07-17T08:42:49.236754+0000","last_unstale":"2024-07-17T08:42:49.246592+0000","last_undegraded":"2024-07-17T08:42:49.246592+0000","last_fullsized":"2024-07-17T08:42:49.246592+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:09.756785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.24","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241945+0000","last_change":"2024-07-17T08:42:49.228957+0000","last_active":"2024-07-17T08:42:49.241945+0000","last_peered":"2024-07-17T08:42:49.241945+0000","last_clean":"2024-07-17T08:42:49.241945+0000","last_became_active":"2024-07-17T08:42:49.228695+0000","last_became_peered":"2024-07-17T08:42:49.228695+0000","last_unstale":"2024-07-17T08:42:49.241945+0000","last_undegraded":"2024-07-17T08:42:49.241945+0000","last_fullsized":"2024-07-17T08:42:49.241945+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:48.746007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.23","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.242773+0000","last_change":"2024-07-17T08:42:49.239430+0000","last_active":"2024-07-17T08:42:49.242773+0000","last_peered":"2024-07-17T08:42:49.242773+0000","last_clean":"2024-07-17T08:42:49.242773+0000","last_became_active":"2024-07-17T08:42:49.239140+0000","last_became_peered":"2024-07-17T08:42:49.239140+0000","last_unstale":"2024-07-17T08:42:49.242773+0000","last_undegraded":"2024-07-17T08:42:49.242773+0000","last_fullsized":"2024-07-17T08:42:49.242773+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:26.733393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.22","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248104+0000","last_change":"2024-07-17T08:42:49.243780+0000","last_active":"2024-07-17T08:42:49.248104+0000","last_peered":"2024-07-17T08:42:49.248104+0000","last_clean":"2024-07-17T08:42:49.248104+0000","last_became_active":"2024-07-17T08:42:49.243496+0000","last_became_peered":"2024-07-17T08:42:49.243496+0000","last_unstale":"2024-07-17T08:42:49.248104+0000","last_undegraded":"2024-07-17T08:42:49.248104+0000","last_fullsized":"2024-07-17T08:42:49.248104+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:19.401904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.21","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.242207+0000","last_change":"2024-07-17T08:42:49.236123+0000","last_active":"2024-07-17T08:42:49.242207+0000","last_peered":"2024-07-17T08:42:49.242207+0000","last_clean":"2024-07-17T08:42:49.242207+0000","last_became_active":"2024-07-17T08:42:49.235826+0000","last_became_peered":"2024-07-17T08:42:49.235826+0000","last_unstale":"2024-07-17T08:42:49.242207+0000","last_undegraded":"2024-07-17T08:42:49.242207+0000","last_fullsized":"2024-07-17T08:42:49.242207+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:28.405741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.20","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.236936+0000","last_change":"2024-07-17T08:42:49.228470+0000","last_active":"2024-07-17T08:42:49.236936+0000","last_peered":"2024-07-17T08:42:49.236936+0000","last_clean":"2024-07-17T08:42:49.236936+0000","last_became_active":"2024-07-17T08:42:49.228204+0000","last_became_peered":"2024-07-17T08:42:49.228204+0000","last_unstale":"2024-07-17T08:42:49.236936+0000","last_undegraded":"2024-07-17T08:42:49.236936+0000","last_fullsized":"2024-07-17T08:42:49.236936+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:18.197375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.1c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234878+0000","last_change":"2024-07-17T08:42:49.232819+0000","last_active":"2024-07-17T08:42:49.234878+0000","last_peered":"2024-07-17T08:42:49.234878+0000","last_clean":"2024-07-17T08:42:49.234878+0000","last_became_active":"2024-07-17T08:42:49.232455+0000","last_became_peered":"2024-07-17T08:42:49.232455+0000","last_unstale":"2024-07-17T08:42:49.234878+0000","last_undegraded":"2024-07-17T08:42:49.234878+0000","last_fullsized":"2024-07-17T08:42:49.234878+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:52.788077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.1f","version":"334'4","reported_seq":16,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249531+0000","last_change":"2024-07-17T08:42:48.200065+0000","last_active":"2024-07-17T08:42:49.249531+0000","last_peered":"2024-07-17T08:42:49.249531+0000","last_clean":"2024-07-17T08:42:49.249531+0000","last_became_active":"2024-07-17T08:42:48.199156+0000","last_became_peered":"2024-07-17T08:42:48.199156+0000","last_unstale":"2024-07-17T08:42:49.249531+0000","last_undegraded":"2024-07-17T08:42:49.249531+0000","last_fullsized":"2024-07-17T08:42:49.249531+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:03.485720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.1d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239162+0000","last_change":"2024-07-17T08:42:49.235456+0000","last_active":"2024-07-17T08:42:49.239162+0000","last_peered":"2024-07-17T08:42:49.239162+0000","last_clean":"2024-07-17T08:42:49.239162+0000","last_became_active":"2024-07-17T08:42:49.235167+0000","last_became_peered":"2024-07-17T08:42:49.235167+0000","last_unstale":"2024-07-17T08:42:49.239162+0000","last_undegraded":"2024-07-17T08:42:49.239162+0000","last_fullsized":"2024-07-17T08:42:49.239162+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:37.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.1e","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251597+0000","last_change":"2024-07-17T08:42:48.207167+0000","last_active":"2024-07-17T08:42:49.251597+0000","last_peered":"2024-07-17T08:42:49.251597+0000","last_clean":"2024-07-17T08:42:49.251597+0000","last_became_active":"2024-07-17T08:42:48.206785+0000","last_became_peered":"2024-07-17T08:42:48.206785+0000","last_unstale":"2024-07-17T08:42:49.251597+0000","last_undegraded":"2024-07-17T08:42:49.251597+0000","last_fullsized":"2024-07-17T08:42:49.251597+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:04.823813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.1e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.237466+0000","last_change":"2024-07-17T08:42:49.231754+0000","last_active":"2024-07-17T08:42:49.237466+0000","last_peered":"2024-07-17T08:42:49.237466+0000","last_clean":"2024-07-17T08:42:49.237466+0000","last_became_active":"2024-07-17T08:42:49.231448+0000","last_became_peered":"2024-07-17T08:42:49.231448+0000","last_unstale":"2024-07-17T08:42:49.237466+0000","last_undegraded":"2024-07-17T08:42:49.237466+0000","last_fullsized":"2024-07-17T08:42:49.237466+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:08.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.1d","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248688+0000","last_change":"2024-07-17T08:42:48.206235+0000","last_active":"2024-07-17T08:42:49.248688+0000","last_peered":"2024-07-17T08:42:49.248688+0000","last_clean":"2024-07-17T08:42:49.248688+0000","last_became_active":"2024-07-17T08:42:48.205952+0000","last_became_peered":"2024-07-17T08:42:48.205952+0000","last_unstale":"2024-07-17T08:42:49.248688+0000","last_undegraded":"2024-07-17T08:42:49.248688+0000","last_fullsized":"2024-07-17T08:42:49.248688+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:55.318098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.1f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239897+0000","last_change":"2024-07-17T08:42:49.237762+0000","last_active":"2024-07-17T08:42:49.239897+0000","last_peered":"2024-07-17T08:42:49.239897+0000","last_clean":"2024-07-17T08:42:49.239897+0000","last_became_active":"2024-07-17T08:42:49.237466+0000","last_became_peered":"2024-07-17T08:42:49.237466+0000","last_unstale":"2024-07-17T08:42:49.239897+0000","last_undegraded":"2024-07-17T08:42:49.239897+0000","last_fullsized":"2024-07-17T08:42:49.239897+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:47.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.1c","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248200+0000","last_change":"2024-07-17T08:42:48.203047+0000","last_active":"2024-07-17T08:42:49.248200+0000","last_peered":"2024-07-17T08:42:49.248200+0000","last_clean":"2024-07-17T08:42:49.248200+0000","last_became_active":"2024-07-17T08:42:48.202780+0000","last_became_peered":"2024-07-17T08:42:48.202780+0000","last_unstale":"2024-07-17T08:42:49.248200+0000","last_undegraded":"2024-07-17T08:42:49.248200+0000","last_fullsized":"2024-07-17T08:42:49.248200+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:42.648872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.18","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229340+0000","last_change":"2024-07-17T08:42:49.227333+0000","last_active":"2024-07-17T08:42:49.229340+0000","last_peered":"2024-07-17T08:42:49.229340+0000","last_clean":"2024-07-17T08:42:49.229340+0000","last_became_active":"2024-07-17T08:42:49.226092+0000","last_became_peered":"2024-07-17T08:42:49.226092+0000","last_unstale":"2024-07-17T08:42:49.229340+0000","last_undegraded":"2024-07-17T08:42:49.229340+0000","last_fullsized":"2024-07-17T08:42:49.229340+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:03.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.1b","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251221+0000","last_change":"2024-07-17T08:42:48.192186+0000","last_active":"2024-07-17T08:42:49.251221+0000","last_peered":"2024-07-17T08:42:49.251221+0000","last_clean":"2024-07-17T08:42:49.251221+0000","last_became_active":"2024-07-17T08:42:48.191891+0000","last_became_peered":"2024-07-17T08:42:48.191891+0000","last_unstale":"2024-07-17T08:42:49.251221+0000","last_undegraded":"2024-07-17T08:42:49.251221+0000","last_fullsized":"2024-07-17T08:42:49.251221+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:46.445238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.19","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.228306+0000","last_change":"2024-07-17T08:42:49.226762+0000","last_active":"2024-07-17T08:42:49.228306+0000","last_peered":"2024-07-17T08:42:49.228306+0000","last_clean":"2024-07-17T08:42:49.228306+0000","last_became_active":"2024-07-17T08:42:49.226442+0000","last_became_peered":"2024-07-17T08:42:49.226442+0000","last_unstale":"2024-07-17T08:42:49.228306+0000","last_undegraded":"2024-07-17T08:42:49.228306+0000","last_fullsized":"2024-07-17T08:42:49.228306+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:05.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.1a","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.244666+0000","last_change":"2024-07-17T08:42:48.203897+0000","last_active":"2024-07-17T08:42:49.244666+0000","last_peered":"2024-07-17T08:42:49.244666+0000","last_clean":"2024-07-17T08:42:49.244666+0000","last_became_active":"2024-07-17T08:42:48.203377+0000","last_became_peered":"2024-07-17T08:42:48.203377+0000","last_unstale":"2024-07-17T08:42:49.244666+0000","last_undegraded":"2024-07-17T08:42:49.244666+0000","last_fullsized":"2024-07-17T08:42:49.244666+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:00.166950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.1a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231034+0000","last_change":"2024-07-17T08:42:49.223355+0000","last_active":"2024-07-17T08:42:49.231034+0000","last_peered":"2024-07-17T08:42:49.231034+0000","last_clean":"2024-07-17T08:42:49.231034+0000","last_became_active":"2024-07-17T08:42:49.223056+0000","last_became_peered":"2024-07-17T08:42:49.223056+0000","last_unstale":"2024-07-17T08:42:49.231034+0000","last_undegraded":"2024-07-17T08:42:49.231034+0000","last_fullsized":"2024-07-17T08:42:49.231034+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:06.714326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.19","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.225604+0000","last_change":"2024-07-17T08:42:48.203245+0000","last_active":"2024-07-17T08:42:49.225604+0000","last_peered":"2024-07-17T08:42:49.225604+0000","last_clean":"2024-07-17T08:42:49.225604+0000","last_became_active":"2024-07-17T08:42:48.202883+0000","last_became_peered":"2024-07-17T08:42:48.202883+0000","last_unstale":"2024-07-17T08:42:49.225604+0000","last_undegraded":"2024-07-17T08:42:49.225604+0000","last_fullsized":"2024-07-17T08:42:49.225604+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:10.926602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.1b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229367+0000","last_change":"2024-07-17T08:42:49.222635+0000","last_active":"2024-07-17T08:42:49.229367+0000","last_peered":"2024-07-17T08:42:49.229367+0000","last_clean":"2024-07-17T08:42:49.229367+0000","last_became_active":"2024-07-17T08:42:49.222222+0000","last_became_peered":"2024-07-17T08:42:49.222222+0000","last_unstale":"2024-07-17T08:42:49.229367+0000","last_undegraded":"2024-07-17T08:42:49.229367+0000","last_fullsized":"2024-07-17T08:42:49.229367+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:01.573358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.18","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.202896+0000","last_change":"2024-07-17T08:42:48.189906+0000","last_active":"2024-07-17T08:42:49.202896+0000","last_peered":"2024-07-17T08:42:49.202896+0000","last_clean":"2024-07-17T08:42:49.202896+0000","last_became_active":"2024-07-17T08:42:48.188419+0000","last_became_peered":"2024-07-17T08:42:48.188419+0000","last_unstale":"2024-07-17T08:42:49.202896+0000","last_undegraded":"2024-07-17T08:42:49.202896+0000","last_fullsized":"2024-07-17T08:42:49.202896+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:49.408645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.14","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241618+0000","last_change":"2024-07-17T08:42:49.237791+0000","last_active":"2024-07-17T08:42:49.241618+0000","last_peered":"2024-07-17T08:42:49.241618+0000","last_clean":"2024-07-17T08:42:49.241618+0000","last_became_active":"2024-07-17T08:42:49.237484+0000","last_became_peered":"2024-07-17T08:42:49.237484+0000","last_unstale":"2024-07-17T08:42:49.241618+0000","last_undegraded":"2024-07-17T08:42:49.241618+0000","last_fullsized":"2024-07-17T08:42:49.241618+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:23.749782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.17","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212890+0000","last_change":"2024-07-17T08:42:48.191283+0000","last_active":"2024-07-17T08:42:49.212890+0000","last_peered":"2024-07-17T08:42:49.212890+0000","last_clean":"2024-07-17T08:42:49.212890+0000","last_became_active":"2024-07-17T08:42:48.190826+0000","last_became_peered":"2024-07-17T08:42:48.190826+0000","last_unstale":"2024-07-17T08:42:49.212890+0000","last_undegraded":"2024-07-17T08:42:49.212890+0000","last_fullsized":"2024-07-17T08:42:49.212890+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:54:20.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.15","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.236557+0000","last_change":"2024-07-17T08:42:49.232776+0000","last_active":"2024-07-17T08:42:49.236557+0000","last_peered":"2024-07-17T08:42:49.236557+0000","last_clean":"2024-07-17T08:42:49.236557+0000","last_became_active":"2024-07-17T08:42:49.232453+0000","last_became_peered":"2024-07-17T08:42:49.232453+0000","last_unstale":"2024-07-17T08:42:49.236557+0000","last_undegraded":"2024-07-17T08:42:49.236557+0000","last_fullsized":"2024-07-17T08:42:49.236557+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:00.939094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"45.16","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.214730+0000","last_change":"2024-07-17T08:42:48.209763+0000","last_active":"2024-07-17T08:42:49.214730+0000","last_peered":"2024-07-17T08:42:49.214730+0000","last_clean":"2024-07-17T08:42:49.214730+0000","last_became_active":"2024-07-17T08:42:48.209420+0000","last_became_peered":"2024-07-17T08:42:48.209420+0000","last_unstale":"2024-07-17T08:42:49.214730+0000","last_undegraded":"2024-07-17T08:42:49.214730+0000","last_fullsized":"2024-07-17T08:42:49.214730+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:36.992656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.16","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243668+0000","last_change":"2024-07-17T08:42:49.240996+0000","last_active":"2024-07-17T08:42:49.243668+0000","last_peered":"2024-07-17T08:42:49.243668+0000","last_clean":"2024-07-17T08:42:49.243668+0000","last_became_active":"2024-07-17T08:42:49.240471+0000","last_became_peered":"2024-07-17T08:42:49.240471+0000","last_unstale":"2024-07-17T08:42:49.243668+0000","last_undegraded":"2024-07-17T08:42:49.243668+0000","last_fullsized":"2024-07-17T08:42:49.243668+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:35.972200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.15","version":"334'3","reported_seq":15,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250059+0000","last_change":"2024-07-17T08:42:48.207171+0000","last_active":"2024-07-17T08:42:49.250059+0000","last_peered":"2024-07-17T08:42:49.250059+0000","last_clean":"2024-07-17T08:42:49.250059+0000","last_became_active":"2024-07-17T08:42:48.206909+0000","last_became_peered":"2024-07-17T08:42:48.206909+0000","last_unstale":"2024-07-17T08:42:49.250059+0000","last_undegraded":"2024-07-17T08:42:49.250059+0000","last_fullsized":"2024-07-17T08:42:49.250059+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:55:48.824122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.17","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231588+0000","last_change":"2024-07-17T08:42:49.228869+0000","last_active":"2024-07-17T08:42:49.231588+0000","last_peered":"2024-07-17T08:42:49.231588+0000","last_clean":"2024-07-17T08:42:49.231588+0000","last_became_active":"2024-07-17T08:42:49.228529+0000","last_became_peered":"2024-07-17T08:42:49.228529+0000","last_unstale":"2024-07-17T08:42:49.231588+0000","last_undegraded":"2024-07-17T08:42:49.231588+0000","last_fullsized":"2024-07-17T08:42:49.231588+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:12.675925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"45.14","version":"334'6","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.253239+0000","last_change":"2024-07-17T08:42:48.199744+0000","last_active":"2024-07-17T08:42:49.253239+0000","last_peered":"2024-07-17T08:42:49.253239+0000","last_clean":"2024-07-17T08:42:49.253239+0000","last_became_active":"2024-07-17T08:42:48.199437+0000","last_became_peered":"2024-07-17T08:42:48.199437+0000","last_unstale":"2024-07-17T08:42:49.253239+0000","last_undegraded":"2024-07-17T08:42:49.253239+0000","last_fullsized":"2024-07-17T08:42:49.253239+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:42.648872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.10","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249829+0000","last_change":"2024-07-17T08:42:49.241942+0000","last_active":"2024-07-17T08:42:49.249829+0000","last_peered":"2024-07-17T08:42:49.249829+0000","last_clean":"2024-07-17T08:42:49.249829+0000","last_became_active":"2024-07-17T08:42:49.236994+0000","last_became_peered":"2024-07-17T08:42:49.236994+0000","last_unstale":"2024-07-17T08:42:49.249829+0000","last_undegraded":"2024-07-17T08:42:49.249829+0000","last_fullsized":"2024-07-17T08:42:49.249829+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:17:22.860542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.13","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250105+0000","last_change":"2024-07-17T08:42:48.193918+0000","last_active":"2024-07-17T08:42:49.250105+0000","last_peered":"2024-07-17T08:42:49.250105+0000","last_clean":"2024-07-17T08:42:49.250105+0000","last_became_active":"2024-07-17T08:42:48.193512+0000","last_became_peered":"2024-07-17T08:42:48.193512+0000","last_unstale":"2024-07-17T08:42:49.250105+0000","last_undegraded":"2024-07-17T08:42:49.250105+0000","last_fullsized":"2024-07-17T08:42:49.250105+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:53.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.11","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239224+0000","last_change":"2024-07-17T08:42:49.237616+0000","last_active":"2024-07-17T08:42:49.239224+0000","last_peered":"2024-07-17T08:42:49.239224+0000","last_clean":"2024-07-17T08:42:49.239224+0000","last_became_active":"2024-07-17T08:42:49.237101+0000","last_became_peered":"2024-07-17T08:42:49.237101+0000","last_unstale":"2024-07-17T08:42:49.239224+0000","last_undegraded":"2024-07-17T08:42:49.239224+0000","last_fullsized":"2024-07-17T08:42:49.239224+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:35.196396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.12","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213914+0000","last_change":"2024-07-17T08:42:48.191342+0000","last_active":"2024-07-17T08:42:49.213914+0000","last_peered":"2024-07-17T08:42:49.213914+0000","last_clean":"2024-07-17T08:42:49.213914+0000","last_became_active":"2024-07-17T08:42:48.190955+0000","last_became_peered":"2024-07-17T08:42:48.190955+0000","last_unstale":"2024-07-17T08:42:49.213914+0000","last_undegraded":"2024-07-17T08:42:49.213914+0000","last_fullsized":"2024-07-17T08:42:49.213914+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:38.672577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.12","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241257+0000","last_change":"2024-07-17T08:42:49.237435+0000","last_active":"2024-07-17T08:42:49.241257+0000","last_peered":"2024-07-17T08:42:49.241257+0000","last_clean":"2024-07-17T08:42:49.241257+0000","last_became_active":"2024-07-17T08:42:49.237096+0000","last_became_peered":"2024-07-17T08:42:49.237096+0000","last_unstale":"2024-07-17T08:42:49.241257+0000","last_undegraded":"2024-07-17T08:42:49.241257+0000","last_fullsized":"2024-07-17T08:42:49.241257+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:39:23.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.11","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.218694+0000","last_change":"2024-07-17T08:42:48.203957+0000","last_active":"2024-07-17T08:42:49.218694+0000","last_peered":"2024-07-17T08:42:49.218694+0000","last_clean":"2024-07-17T08:42:49.218694+0000","last_became_active":"2024-07-17T08:42:48.203578+0000","last_became_peered":"2024-07-17T08:42:48.203578+0000","last_unstale":"2024-07-17T08:42:49.218694+0000","last_undegraded":"2024-07-17T08:42:49.218694+0000","last_fullsized":"2024-07-17T08:42:49.218694+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:00.618114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.13","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231186+0000","last_change":"2024-07-17T08:42:49.224064+0000","last_active":"2024-07-17T08:42:49.231186+0000","last_peered":"2024-07-17T08:42:49.231186+0000","last_clean":"2024-07-17T08:42:49.231186+0000","last_became_active":"2024-07-17T08:42:49.223788+0000","last_became_peered":"2024-07-17T08:42:49.223788+0000","last_unstale":"2024-07-17T08:42:49.231186+0000","last_undegraded":"2024-07-17T08:42:49.231186+0000","last_fullsized":"2024-07-17T08:42:49.231186+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:03.528826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.10","version":"334'8","reported_seq":20,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250888+0000","last_change":"2024-07-17T08:42:48.203718+0000","last_active":"2024-07-17T08:42:49.250888+0000","last_peered":"2024-07-17T08:42:49.250888+0000","last_clean":"2024-07-17T08:42:49.250888+0000","last_became_active":"2024-07-17T08:42:48.203373+0000","last_became_peered":"2024-07-17T08:42:48.203373+0000","last_unstale":"2024-07-17T08:42:49.250888+0000","last_undegraded":"2024-07-17T08:42:49.250888+0000","last_fullsized":"2024-07-17T08:42:49.250888+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:51.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229431+0000","last_change":"2024-07-17T08:42:49.228442+0000","last_active":"2024-07-17T08:42:49.229431+0000","last_peered":"2024-07-17T08:42:49.229431+0000","last_clean":"2024-07-17T08:42:49.229431+0000","last_became_active":"2024-07-17T08:42:49.226346+0000","last_became_peered":"2024-07-17T08:42:49.226346+0000","last_unstale":"2024-07-17T08:42:49.229431+0000","last_undegraded":"2024-07-17T08:42:49.229431+0000","last_fullsized":"2024-07-17T08:42:49.229431+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:54.934703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.f","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252663+0000","last_change":"2024-07-17T08:42:48.191313+0000","last_active":"2024-07-17T08:42:49.252663+0000","last_peered":"2024-07-17T08:42:49.252663+0000","last_clean":"2024-07-17T08:42:49.252663+0000","last_became_active":"2024-07-17T08:42:48.191037+0000","last_became_peered":"2024-07-17T08:42:48.191037+0000","last_unstale":"2024-07-17T08:42:49.252663+0000","last_undegraded":"2024-07-17T08:42:49.252663+0000","last_fullsized":"2024-07-17T08:42:49.252663+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:00.333893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245456+0000","last_change":"2024-07-17T08:42:49.233170+0000","last_active":"2024-07-17T08:42:49.245456+0000","last_peered":"2024-07-17T08:42:49.245456+0000","last_clean":"2024-07-17T08:42:49.245456+0000","last_became_active":"2024-07-17T08:42:49.232742+0000","last_became_peered":"2024-07-17T08:42:49.232742+0000","last_unstale":"2024-07-17T08:42:49.245456+0000","last_undegraded":"2024-07-17T08:42:49.245456+0000","last_fullsized":"2024-07-17T08:42:49.245456+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:52:08.251882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.e","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241780+0000","last_change":"2024-07-17T08:42:48.209845+0000","last_active":"2024-07-17T08:42:49.241780+0000","last_peered":"2024-07-17T08:42:49.241780+0000","last_clean":"2024-07-17T08:42:49.241780+0000","last_became_active":"2024-07-17T08:42:48.209408+0000","last_became_peered":"2024-07-17T08:42:48.209408+0000","last_unstale":"2024-07-17T08:42:49.241780+0000","last_undegraded":"2024-07-17T08:42:49.241780+0000","last_fullsized":"2024-07-17T08:42:49.241780+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:56.440721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.237490+0000","last_change":"2024-07-17T08:42:49.234587+0000","last_active":"2024-07-17T08:42:49.237490+0000","last_peered":"2024-07-17T08:42:49.237490+0000","last_clean":"2024-07-17T08:42:49.237490+0000","last_became_active":"2024-07-17T08:42:49.234262+0000","last_became_peered":"2024-07-17T08:42:49.234262+0000","last_unstale":"2024-07-17T08:42:49.237490+0000","last_undegraded":"2024-07-17T08:42:49.237490+0000","last_fullsized":"2024-07-17T08:42:49.237490+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:56.691955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.d","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252016+0000","last_change":"2024-07-17T08:42:48.208416+0000","last_active":"2024-07-17T08:42:49.252016+0000","last_peered":"2024-07-17T08:42:49.252016+0000","last_clean":"2024-07-17T08:42:49.252016+0000","last_became_active":"2024-07-17T08:42:48.208125+0000","last_became_peered":"2024-07-17T08:42:48.208125+0000","last_unstale":"2024-07-17T08:42:49.252016+0000","last_undegraded":"2024-07-17T08:42:49.252016+0000","last_fullsized":"2024-07-17T08:42:49.252016+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:56:16.472678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229434+0000","last_change":"2024-07-17T08:42:49.227907+0000","last_active":"2024-07-17T08:42:49.229434+0000","last_peered":"2024-07-17T08:42:49.229434+0000","last_clean":"2024-07-17T08:42:49.229434+0000","last_became_active":"2024-07-17T08:42:49.227624+0000","last_became_peered":"2024-07-17T08:42:49.227624+0000","last_unstale":"2024-07-17T08:42:49.229434+0000","last_undegraded":"2024-07-17T08:42:49.229434+0000","last_fullsized":"2024-07-17T08:42:49.229434+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:47:30.979141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.c","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212686+0000","last_change":"2024-07-17T08:42:48.192166+0000","last_active":"2024-07-17T08:42:49.212686+0000","last_peered":"2024-07-17T08:42:49.212686+0000","last_clean":"2024-07-17T08:42:49.212686+0000","last_became_active":"2024-07-17T08:42:48.191763+0000","last_became_peered":"2024-07-17T08:42:48.191763+0000","last_unstale":"2024-07-17T08:42:49.212686+0000","last_undegraded":"2024-07-17T08:42:49.212686+0000","last_fullsized":"2024-07-17T08:42:49.212686+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:23:36.445437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.8","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238733+0000","last_change":"2024-07-17T08:42:49.234007+0000","last_active":"2024-07-17T08:42:49.238733+0000","last_peered":"2024-07-17T08:42:49.238733+0000","last_clean":"2024-07-17T08:42:49.238733+0000","last_became_active":"2024-07-17T08:42:49.232237+0000","last_became_peered":"2024-07-17T08:42:49.232237+0000","last_unstale":"2024-07-17T08:42:49.238733+0000","last_undegraded":"2024-07-17T08:42:49.238733+0000","last_fullsized":"2024-07-17T08:42:49.238733+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:48.496046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.b","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247980+0000","last_change":"2024-07-17T08:42:48.199031+0000","last_active":"2024-07-17T08:42:49.247980+0000","last_peered":"2024-07-17T08:42:49.247980+0000","last_clean":"2024-07-17T08:42:49.247980+0000","last_became_active":"2024-07-17T08:42:48.198707+0000","last_became_peered":"2024-07-17T08:42:48.198707+0000","last_unstale":"2024-07-17T08:42:49.247980+0000","last_undegraded":"2024-07-17T08:42:49.247980+0000","last_fullsized":"2024-07-17T08:42:49.247980+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:22.983373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.9","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252044+0000","last_change":"2024-07-17T08:42:49.247212+0000","last_active":"2024-07-17T08:42:49.252044+0000","last_peered":"2024-07-17T08:42:49.252044+0000","last_clean":"2024-07-17T08:42:49.252044+0000","last_became_active":"2024-07-17T08:42:49.246161+0000","last_became_peered":"2024-07-17T08:42:49.246161+0000","last_unstale":"2024-07-17T08:42:49.252044+0000","last_undegraded":"2024-07-17T08:42:49.252044+0000","last_fullsized":"2024-07-17T08:42:49.252044+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:01:37.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.a","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213905+0000","last_change":"2024-07-17T08:42:48.195854+0000","last_active":"2024-07-17T08:42:49.213905+0000","last_peered":"2024-07-17T08:42:49.213905+0000","last_clean":"2024-07-17T08:42:49.213905+0000","last_became_active":"2024-07-17T08:42:48.195346+0000","last_became_peered":"2024-07-17T08:42:48.195346+0000","last_unstale":"2024-07-17T08:42:49.213905+0000","last_undegraded":"2024-07-17T08:42:49.213905+0000","last_fullsized":"2024-07-17T08:42:49.213905+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:23:01.872400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235333+0000","last_change":"2024-07-17T08:42:49.234127+0000","last_active":"2024-07-17T08:42:49.235333+0000","last_peered":"2024-07-17T08:42:49.235333+0000","last_clean":"2024-07-17T08:42:49.235333+0000","last_became_active":"2024-07-17T08:42:49.233833+0000","last_became_peered":"2024-07-17T08:42:49.233833+0000","last_unstale":"2024-07-17T08:42:49.235333+0000","last_undegraded":"2024-07-17T08:42:49.235333+0000","last_fullsized":"2024-07-17T08:42:49.235333+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:41:31.551567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.9","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212005+0000","last_change":"2024-07-17T08:42:48.206101+0000","last_active":"2024-07-17T08:42:49.212005+0000","last_peered":"2024-07-17T08:42:49.212005+0000","last_clean":"2024-07-17T08:42:49.212005+0000","last_became_active":"2024-07-17T08:42:48.205793+0000","last_became_peered":"2024-07-17T08:42:48.205793+0000","last_unstale":"2024-07-17T08:42:49.212005+0000","last_undegraded":"2024-07-17T08:42:49.212005+0000","last_fullsized":"2024-07-17T08:42:49.212005+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:27.788182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252301+0000","last_change":"2024-07-17T08:42:49.249103+0000","last_active":"2024-07-17T08:42:49.252301+0000","last_peered":"2024-07-17T08:42:49.252301+0000","last_clean":"2024-07-17T08:42:49.252301+0000","last_became_active":"2024-07-17T08:42:49.243780+0000","last_became_peered":"2024-07-17T08:42:49.243780+0000","last_unstale":"2024-07-17T08:42:49.252301+0000","last_undegraded":"2024-07-17T08:42:49.252301+0000","last_fullsized":"2024-07-17T08:42:49.252301+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:15.792625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.8","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229693+0000","last_change":"2024-07-17T08:42:48.201389+0000","last_active":"2024-07-17T08:42:49.229693+0000","last_peered":"2024-07-17T08:42:49.229693+0000","last_clean":"2024-07-17T08:42:49.229693+0000","last_became_active":"2024-07-17T08:42:48.201094+0000","last_became_peered":"2024-07-17T08:42:48.201094+0000","last_unstale":"2024-07-17T08:42:49.229693+0000","last_undegraded":"2024-07-17T08:42:49.229693+0000","last_fullsized":"2024-07-17T08:42:49.229693+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:10.926602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.4","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248812+0000","last_change":"2024-07-17T08:42:49.246191+0000","last_active":"2024-07-17T08:42:49.248812+0000","last_peered":"2024-07-17T08:42:49.248812+0000","last_clean":"2024-07-17T08:42:49.248812+0000","last_became_active":"2024-07-17T08:42:49.245912+0000","last_became_peered":"2024-07-17T08:42:49.245912+0000","last_unstale":"2024-07-17T08:42:49.248812+0000","last_undegraded":"2024-07-17T08:42:49.248812+0000","last_fullsized":"2024-07-17T08:42:49.248812+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:22.466050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.7","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249005+0000","last_change":"2024-07-17T08:42:48.208183+0000","last_active":"2024-07-17T08:42:49.249005+0000","last_peered":"2024-07-17T08:42:49.249005+0000","last_clean":"2024-07-17T08:42:49.249005+0000","last_became_active":"2024-07-17T08:42:48.207624+0000","last_became_peered":"2024-07-17T08:42:48.207624+0000","last_unstale":"2024-07-17T08:42:49.249005+0000","last_undegraded":"2024-07-17T08:42:49.249005+0000","last_fullsized":"2024-07-17T08:42:49.249005+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:24:21.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.5","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239138+0000","last_change":"2024-07-17T08:42:49.234749+0000","last_active":"2024-07-17T08:42:49.239138+0000","last_peered":"2024-07-17T08:42:49.239138+0000","last_clean":"2024-07-17T08:42:49.239138+0000","last_became_active":"2024-07-17T08:42:49.234444+0000","last_became_peered":"2024-07-17T08:42:49.234444+0000","last_unstale":"2024-07-17T08:42:49.239138+0000","last_undegraded":"2024-07-17T08:42:49.239138+0000","last_fullsized":"2024-07-17T08:42:49.239138+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.6","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.215200+0000","last_change":"2024-07-17T08:42:48.198045+0000","last_active":"2024-07-17T08:42:49.215200+0000","last_peered":"2024-07-17T08:42:49.215200+0000","last_clean":"2024-07-17T08:42:49.215200+0000","last_became_active":"2024-07-17T08:42:48.197729+0000","last_became_peered":"2024-07-17T08:42:48.197729+0000","last_unstale":"2024-07-17T08:42:49.215200+0000","last_undegraded":"2024-07-17T08:42:49.215200+0000","last_fullsized":"2024-07-17T08:42:49.215200+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:30.611918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.6","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245517+0000","last_change":"2024-07-17T08:42:49.242614+0000","last_active":"2024-07-17T08:42:49.245517+0000","last_peered":"2024-07-17T08:42:49.245517+0000","last_clean":"2024-07-17T08:42:49.245517+0000","last_became_active":"2024-07-17T08:42:49.242342+0000","last_became_peered":"2024-07-17T08:42:49.242342+0000","last_unstale":"2024-07-17T08:42:49.245517+0000","last_undegraded":"2024-07-17T08:42:49.245517+0000","last_fullsized":"2024-07-17T08:42:49.245517+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:49.107565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"45.5","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.219198+0000","last_change":"2024-07-17T08:42:48.203135+0000","last_active":"2024-07-17T08:42:49.219198+0000","last_peered":"2024-07-17T08:42:49.219198+0000","last_clean":"2024-07-17T08:42:49.219198+0000","last_became_active":"2024-07-17T08:42:48.202840+0000","last_became_peered":"2024-07-17T08:42:48.202840+0000","last_unstale":"2024-07-17T08:42:49.219198+0000","last_undegraded":"2024-07-17T08:42:49.219198+0000","last_fullsized":"2024-07-17T08:42:49.219198+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:26.440022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.7","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247178+0000","last_change":"2024-07-17T08:42:49.242996+0000","last_active":"2024-07-17T08:42:49.247178+0000","last_peered":"2024-07-17T08:42:49.247178+0000","last_clean":"2024-07-17T08:42:49.247178+0000","last_became_active":"2024-07-17T08:42:49.242663+0000","last_became_peered":"2024-07-17T08:42:49.242663+0000","last_unstale":"2024-07-17T08:42:49.247178+0000","last_undegraded":"2024-07-17T08:42:49.247178+0000","last_fullsized":"2024-07-17T08:42:49.247178+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:23.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.4","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.204198+0000","last_change":"2024-07-17T08:42:48.200791+0000","last_active":"2024-07-17T08:42:49.204198+0000","last_peered":"2024-07-17T08:42:49.204198+0000","last_clean":"2024-07-17T08:42:49.204198+0000","last_became_active":"2024-07-17T08:42:48.200357+0000","last_became_peered":"2024-07-17T08:42:48.200357+0000","last_unstale":"2024-07-17T08:42:49.204198+0000","last_undegraded":"2024-07-17T08:42:49.204198+0000","last_fullsized":"2024-07-17T08:42:49.204198+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:37.920200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.0","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234013+0000","last_change":"2024-07-17T08:42:49.225973+0000","last_active":"2024-07-17T08:42:49.234013+0000","last_peered":"2024-07-17T08:42:49.234013+0000","last_clean":"2024-07-17T08:42:49.234013+0000","last_became_active":"2024-07-17T08:42:49.225676+0000","last_became_peered":"2024-07-17T08:42:49.225676+0000","last_unstale":"2024-07-17T08:42:49.234013+0000","last_undegraded":"2024-07-17T08:42:49.234013+0000","last_fullsized":"2024-07-17T08:42:49.234013+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:29.388495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.3","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213265+0000","last_change":"2024-07-17T08:42:48.205853+0000","last_active":"2024-07-17T08:42:49.213265+0000","last_peered":"2024-07-17T08:42:49.213265+0000","last_clean":"2024-07-17T08:42:49.213265+0000","last_became_active":"2024-07-17T08:42:48.205053+0000","last_became_peered":"2024-07-17T08:42:48.205053+0000","last_unstale":"2024-07-17T08:42:49.213265+0000","last_undegraded":"2024-07-17T08:42:49.213265+0000","last_fullsized":"2024-07-17T08:42:49.213265+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:37.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.1","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241431+0000","last_change":"2024-07-17T08:42:49.237873+0000","last_active":"2024-07-17T08:42:49.241431+0000","last_peered":"2024-07-17T08:42:49.241431+0000","last_clean":"2024-07-17T08:42:49.241431+0000","last_became_active":"2024-07-17T08:42:49.237587+0000","last_became_peered":"2024-07-17T08:42:49.237587+0000","last_unstale":"2024-07-17T08:42:49.241431+0000","last_undegraded":"2024-07-17T08:42:49.241431+0000","last_fullsized":"2024-07-17T08:42:49.241431+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:09.037595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.2","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229967+0000","last_change":"2024-07-17T08:42:48.209443+0000","last_active":"2024-07-17T08:42:49.229967+0000","last_peered":"2024-07-17T08:42:49.229967+0000","last_clean":"2024-07-17T08:42:49.229967+0000","last_became_active":"2024-07-17T08:42:48.209097+0000","last_became_peered":"2024-07-17T08:42:48.209097+0000","last_unstale":"2024-07-17T08:42:49.229967+0000","last_undegraded":"2024-07-17T08:42:49.229967+0000","last_fullsized":"2024-07-17T08:42:49.229967+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:37.848317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.2","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252110+0000","last_change":"2024-07-17T08:42:49.245611+0000","last_active":"2024-07-17T08:42:49.252110+0000","last_peered":"2024-07-17T08:42:49.252110+0000","last_clean":"2024-07-17T08:42:49.252110+0000","last_became_active":"2024-07-17T08:42:49.242427+0000","last_became_peered":"2024-07-17T08:42:49.242427+0000","last_unstale":"2024-07-17T08:42:49.252110+0000","last_undegraded":"2024-07-17T08:42:49.252110+0000","last_fullsized":"2024-07-17T08:42:49.252110+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:47.866230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.1","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.215804+0000","last_change":"2024-07-17T08:42:48.205039+0000","last_active":"2024-07-17T08:42:49.215804+0000","last_peered":"2024-07-17T08:42:49.215804+0000","last_clean":"2024-07-17T08:42:49.215804+0000","last_became_active":"2024-07-17T08:42:48.204758+0000","last_became_peered":"2024-07-17T08:42:48.204758+0000","last_unstale":"2024-07-17T08:42:49.215804+0000","last_undegraded":"2024-07-17T08:42:49.215804+0000","last_fullsized":"2024-07-17T08:42:49.215804+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:09.803427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.3","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.240155+0000","last_change":"2024-07-17T08:42:49.236126+0000","last_active":"2024-07-17T08:42:49.240155+0000","last_peered":"2024-07-17T08:42:49.240155+0000","last_clean":"2024-07-17T08:42:49.240155+0000","last_became_active":"2024-07-17T08:42:49.235845+0000","last_became_peered":"2024-07-17T08:42:49.235845+0000","last_unstale":"2024-07-17T08:42:49.240155+0000","last_undegraded":"2024-07-17T08:42:49.240155+0000","last_fullsized":"2024-07-17T08:42:49.240155+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:53.804393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.0","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212917+0000","last_change":"2024-07-17T08:42:48.205131+0000","last_active":"2024-07-17T08:42:49.212917+0000","last_peered":"2024-07-17T08:42:49.212917+0000","last_clean":"2024-07-17T08:42:49.212917+0000","last_became_active":"2024-07-17T08:42:48.204523+0000","last_became_peered":"2024-07-17T08:42:48.204523+0000","last_unstale":"2024-07-17T08:42:49.212917+0000","last_undegraded":"2024-07-17T08:42:49.212917+0000","last_fullsized":"2024-07-17T08:42:49.212917+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:35:30.121314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.2f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238449+0000","last_change":"2024-07-17T08:42:49.234277+0000","last_active":"2024-07-17T08:42:49.238449+0000","last_peered":"2024-07-17T08:42:49.238449+0000","last_clean":"2024-07-17T08:42:49.238449+0000","last_became_active":"2024-07-17T08:42:49.234000+0000","last_became_peered":"2024-07-17T08:42:49.234000+0000","last_unstale":"2024-07-17T08:42:49.238449+0000","last_undegraded":"2024-07-17T08:42:49.238449+0000","last_fullsized":"2024-07-17T08:42:49.238449+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:59.565110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"324'4638","reported_seq":5976,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.224075+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:42:49.224075+0000","last_peered":"2024-07-17T08:42:49.224075+0000","last_clean":"2024-07-17T08:42:49.224075+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:42:49.224075+0000","last_undegraded":"2024-07-17T08:42:49.224075+0000","last_fullsized":"2024-07-17T08:42:49.224075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":517,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":46,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":45,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":517,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4638,"ondisk_log_size":4638,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2925176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53364,"kb_used_data":4136,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318476,"statfs":{"total":96636764160,"available":96582119424,"internally_reserved":0,"allocated":4235264,"data_stored":2334808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510719,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71440,"kb_used_data":4160,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300400,"statfs":{"total":96636764160,"available":96563609600,"internally_reserved":0,"allocated":4259840,"data_stored":2344191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543424,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53388,"kb_used_data":4152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318452,"statfs":{"total":96636764160,"available":96582094848,"internally_reserved":0,"allocated":4251648,"data_stored":2342669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510718,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57524,"kb_used_data":4136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314316,"statfs":{"total":96636764160,"available":96577859584,"internally_reserved":0,"allocated":4235264,"data_stored":2334808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543422,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57560,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314280,"statfs":{"total":96636764160,"available":96577822720,"internally_reserved":0,"allocated":4272128,"data_stored":2344829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510719,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53384,"kb_used_data":4152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318456,"statfs":{"total":96636764160,"available":96582098944,"internally_reserved":0,"allocated":4251648,"data_stored":2342613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53964,"kb_used_data":4736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317876,"statfs":{"total":96636764160,"available":96581505024,"internally_reserved":0,"allocated":4849664,"data_stored":2933039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:42:53.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 -- 172.21.15.44:0/1966339851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 msgr2=0x7f73440ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 --2- 172.21.15.44:0/1966339851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73440ac690 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7f734415edd0 tx=0x7f733002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 -- 172.21.15.44:0/1966339851 shutdown_connections 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 --2- 172.21.15.44:0/1966339851 >> v2:172.21.15.175:3300/0 conn(0x7f73440cd690 0x7f73440cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 --2- 172.21.15.44:0/1966339851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73440ac690 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 -- 172.21.15.44:0/1966339851 >> 172.21.15.44:0/1966339851 conn(0x7f73440b0fb0 msgr2=0x7f73440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 -- 172.21.15.44:0/1966339851 shutdown_connections 2024-07-17T08:42:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.005+0000 7f734b600640 1 -- 172.21.15.44:0/1966339851 wait complete. 2024-07-17T08:42:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.007+0000 7f734b600640 1 Processor -- start 2024-07-17T08:42:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.007+0000 7f734b600640 1 -- start start 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73440cd690 0x7f7344150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73441522c0 0x7f73441f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73440ce020 con 0x7f73440ac220 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73440adf80 con 0x7f73440cd690 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f734b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73440ce1a0 con 0x7f73441522c0 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40318/0 (socket says 172.21.15.44:40318) 2024-07-17T08:42:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 -- 172.21.15.44:0/2418403092 learned_addr learned my addr 172.21.15.44:0/2418403092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 -- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73441522c0 msgr2=0x7f73441f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73441522c0 0x7f73441f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7348c00640 1 -- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73440cd690 msgr2=0x7f7344150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.008+0000 7f7343e00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73440cd690 0x7f7344150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7348c00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73440cd690 0x7f7344150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7348c00640 1 -- 172.21.15.44:0/2418403092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734414f000 con 0x7f73440ac220 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7348c00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f733800c3d0 tx=0x7f733800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338017070 con 0x7f73440ac220 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73380053c0 con 0x7f73440ac220 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734414f2c0 con 0x7f73440ac220 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338003850 con 0x7f73440ac220 2024-07-17T08:42:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.009+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734414f880 con 0x7f73440ac220 2024-07-17T08:42:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.010+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73440ce6c0 con 0x7f73440ac220 2024-07-17T08:42:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.012+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f733801a070 con 0x7f73440ac220 2024-07-17T08:42:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.012+0000 7f7340c00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 0x7f7324079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.013+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f7338097c30 con 0x7f73440ac220 2024-07-17T08:42:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.013+0000 7f7343e00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 0x7f7324079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.014+0000 7f7343e00640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 0x7f7324079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7330033000 tx=0x7f7330004730 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:54.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.017+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7338060bf0 con 0x7f73440ac220 2024-07-17T08:42:54.147 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-07-17T08:42:54.145+0000 7f1626000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:42:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.232+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 --> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7344075550 con 0x7f7324076cd0 2024-07-17T08:42:54.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.244+0000 7f7340c00640 1 -- 172.21.15.44:0/2418403092 <== mgr.9539 v2:172.21.15.175:6834/1233420839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7f7344075550 con 0x7f7324076cd0 2024-07-17T08:42:54.249 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:54.252 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:42:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 msgr2=0x7f7324079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 0x7f7324079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7330033000 tx=0x7f7330004730 comp rx=0 tx=0).stop 2024-07-17T08:42:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 msgr2=0x7f73441f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f733800c3d0 tx=0x7f733800c890 comp rx=0 tx=0).stop 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 shutdown_connections 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7324076cd0 0x7f7324079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73441522c0 0x7f73441f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73440cd690 0x7f7344150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 --2- 172.21.15.44:0/2418403092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73440ac220 0x7f73441f35a0 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.252+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 >> 172.21.15.44:0/2418403092 conn(0x7f73440b0fb0 msgr2=0x7f73440cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.253+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 shutdown_connections 2024-07-17T08:42:54.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.253+0000 7f734b600640 1 -- 172.21.15.44:0/2418403092 wait complete. 2024-07-17T08:42:54.276 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-07-17T08:42:52.939845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":561,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4676,"ondisk_log_size":4676,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":480944,"kb_used_data":34360,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754493776,"statfs":{"total":773094113280,"available":772601626624,"internally_reserved":0,"allocated":35184640,"data_stored":19902133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.779038"},"pg_stats":[{"pgid":"46.3f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247536+0000","last_change":"2024-07-17T08:42:49.242624+0000","last_active":"2024-07-17T08:42:49.247536+0000","last_peered":"2024-07-17T08:42:49.247536+0000","last_clean":"2024-07-17T08:42:49.247536+0000","last_became_active":"2024-07-17T08:42:49.242347+0000","last_became_peered":"2024-07-17T08:42:49.242347+0000","last_unstale":"2024-07-17T08:42:49.247536+0000","last_undegraded":"2024-07-17T08:42:49.247536+0000","last_fullsized":"2024-07-17T08:42:49.247536+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:20.190911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.3e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238774+0000","last_change":"2024-07-17T08:42:49.233209+0000","last_active":"2024-07-17T08:42:49.238774+0000","last_peered":"2024-07-17T08:42:49.238774+0000","last_clean":"2024-07-17T08:42:49.238774+0000","last_became_active":"2024-07-17T08:42:49.230388+0000","last_became_peered":"2024-07-17T08:42:49.230388+0000","last_unstale":"2024-07-17T08:42:49.238774+0000","last_undegraded":"2024-07-17T08:42:49.238774+0000","last_fullsized":"2024-07-17T08:42:49.238774+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:49.107565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.3d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243485+0000","last_change":"2024-07-17T08:42:49.239344+0000","last_active":"2024-07-17T08:42:49.243485+0000","last_peered":"2024-07-17T08:42:49.243485+0000","last_clean":"2024-07-17T08:42:49.243485+0000","last_became_active":"2024-07-17T08:42:49.238220+0000","last_became_peered":"2024-07-17T08:42:49.238220+0000","last_unstale":"2024-07-17T08:42:49.243485+0000","last_undegraded":"2024-07-17T08:42:49.243485+0000","last_fullsized":"2024-07-17T08:42:49.243485+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:17:23.961958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.3c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249879+0000","last_change":"2024-07-17T08:42:49.246947+0000","last_active":"2024-07-17T08:42:49.249879+0000","last_peered":"2024-07-17T08:42:49.249879+0000","last_clean":"2024-07-17T08:42:49.249879+0000","last_became_active":"2024-07-17T08:42:49.246223+0000","last_became_peered":"2024-07-17T08:42:49.246223+0000","last_unstale":"2024-07-17T08:42:49.249879+0000","last_undegraded":"2024-07-17T08:42:49.249879+0000","last_fullsized":"2024-07-17T08:42:49.249879+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:53.804393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.3b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213991+0000","last_change":"2024-07-17T08:42:49.211782+0000","last_active":"2024-07-17T08:42:49.213991+0000","last_peered":"2024-07-17T08:42:49.213991+0000","last_clean":"2024-07-17T08:42:49.213991+0000","last_became_active":"2024-07-17T08:42:49.211170+0000","last_became_peered":"2024-07-17T08:42:49.211170+0000","last_unstale":"2024-07-17T08:42:49.213991+0000","last_undegraded":"2024-07-17T08:42:49.213991+0000","last_fullsized":"2024-07-17T08:42:49.213991+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:28:25.288474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.3a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235059+0000","last_change":"2024-07-17T08:42:49.233445+0000","last_active":"2024-07-17T08:42:49.235059+0000","last_peered":"2024-07-17T08:42:49.235059+0000","last_clean":"2024-07-17T08:42:49.235059+0000","last_became_active":"2024-07-17T08:42:49.233179+0000","last_became_peered":"2024-07-17T08:42:49.233179+0000","last_unstale":"2024-07-17T08:42:49.235059+0000","last_undegraded":"2024-07-17T08:42:49.235059+0000","last_fullsized":"2024-07-17T08:42:49.235059+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:48:59.303095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.39","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.225472+0000","last_change":"2024-07-17T08:42:49.224432+0000","last_active":"2024-07-17T08:42:49.225472+0000","last_peered":"2024-07-17T08:42:49.225472+0000","last_clean":"2024-07-17T08:42:49.225472+0000","last_became_active":"2024-07-17T08:42:49.224031+0000","last_became_peered":"2024-07-17T08:42:49.224031+0000","last_unstale":"2024-07-17T08:42:49.225472+0000","last_undegraded":"2024-07-17T08:42:49.225472+0000","last_fullsized":"2024-07-17T08:42:49.225472+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:55.436467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.38","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249804+0000","last_change":"2024-07-17T08:42:49.240760+0000","last_active":"2024-07-17T08:42:49.249804+0000","last_peered":"2024-07-17T08:42:49.249804+0000","last_clean":"2024-07-17T08:42:49.249804+0000","last_became_active":"2024-07-17T08:42:49.237356+0000","last_became_peered":"2024-07-17T08:42:49.237356+0000","last_unstale":"2024-07-17T08:42:49.249804+0000","last_undegraded":"2024-07-17T08:42:49.249804+0000","last_fullsized":"2024-07-17T08:42:49.249804+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:10:40.357282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.37","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248616+0000","last_change":"2024-07-17T08:42:49.245877+0000","last_active":"2024-07-17T08:42:49.248616+0000","last_peered":"2024-07-17T08:42:49.248616+0000","last_clean":"2024-07-17T08:42:49.248616+0000","last_became_active":"2024-07-17T08:42:49.244328+0000","last_became_peered":"2024-07-17T08:42:49.244328+0000","last_unstale":"2024-07-17T08:42:49.248616+0000","last_undegraded":"2024-07-17T08:42:49.248616+0000","last_fullsized":"2024-07-17T08:42:49.248616+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:54.776716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.36","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251419+0000","last_change":"2024-07-17T08:42:49.250121+0000","last_active":"2024-07-17T08:42:49.251419+0000","last_peered":"2024-07-17T08:42:49.251419+0000","last_clean":"2024-07-17T08:42:49.251419+0000","last_became_active":"2024-07-17T08:42:49.249198+0000","last_became_peered":"2024-07-17T08:42:49.249198+0000","last_unstale":"2024-07-17T08:42:49.251419+0000","last_undegraded":"2024-07-17T08:42:49.251419+0000","last_fullsized":"2024-07-17T08:42:49.251419+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:32:36.376662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.35","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.217190+0000","last_change":"2024-07-17T08:42:49.215764+0000","last_active":"2024-07-17T08:42:49.217190+0000","last_peered":"2024-07-17T08:42:49.217190+0000","last_clean":"2024-07-17T08:42:49.217190+0000","last_became_active":"2024-07-17T08:42:49.215215+0000","last_became_peered":"2024-07-17T08:42:49.215215+0000","last_unstale":"2024-07-17T08:42:49.217190+0000","last_undegraded":"2024-07-17T08:42:49.217190+0000","last_fullsized":"2024-07-17T08:42:49.217190+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:32.026461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.34","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252063+0000","last_change":"2024-07-17T08:42:49.246619+0000","last_active":"2024-07-17T08:42:49.252063+0000","last_peered":"2024-07-17T08:42:49.252063+0000","last_clean":"2024-07-17T08:42:49.252063+0000","last_became_active":"2024-07-17T08:42:49.244842+0000","last_became_peered":"2024-07-17T08:42:49.244842+0000","last_unstale":"2024-07-17T08:42:49.252063+0000","last_undegraded":"2024-07-17T08:42:49.252063+0000","last_fullsized":"2024-07-17T08:42:49.252063+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:55:02.297284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.33","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238816+0000","last_change":"2024-07-17T08:42:49.233160+0000","last_active":"2024-07-17T08:42:49.238816+0000","last_peered":"2024-07-17T08:42:49.238816+0000","last_clean":"2024-07-17T08:42:49.238816+0000","last_became_active":"2024-07-17T08:42:49.232863+0000","last_became_peered":"2024-07-17T08:42:49.232863+0000","last_unstale":"2024-07-17T08:42:49.238816+0000","last_undegraded":"2024-07-17T08:42:49.238816+0000","last_fullsized":"2024-07-17T08:42:49.238816+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:00.939094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.32","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.233689+0000","last_change":"2024-07-17T08:42:49.231262+0000","last_active":"2024-07-17T08:42:49.233689+0000","last_peered":"2024-07-17T08:42:49.233689+0000","last_clean":"2024-07-17T08:42:49.233689+0000","last_became_active":"2024-07-17T08:42:49.230973+0000","last_became_peered":"2024-07-17T08:42:49.230973+0000","last_unstale":"2024-07-17T08:42:49.233689+0000","last_undegraded":"2024-07-17T08:42:49.233689+0000","last_fullsized":"2024-07-17T08:42:49.233689+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:26:36.150258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.31","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229358+0000","last_change":"2024-07-17T08:42:49.226547+0000","last_active":"2024-07-17T08:42:49.229358+0000","last_peered":"2024-07-17T08:42:49.229358+0000","last_clean":"2024-07-17T08:42:49.229358+0000","last_became_active":"2024-07-17T08:42:49.226196+0000","last_became_peered":"2024-07-17T08:42:49.226196+0000","last_unstale":"2024-07-17T08:42:49.229358+0000","last_undegraded":"2024-07-17T08:42:49.229358+0000","last_fullsized":"2024-07-17T08:42:49.229358+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:07.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.30","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234848+0000","last_change":"2024-07-17T08:42:49.230841+0000","last_active":"2024-07-17T08:42:49.234848+0000","last_peered":"2024-07-17T08:42:49.234848+0000","last_clean":"2024-07-17T08:42:49.234848+0000","last_became_active":"2024-07-17T08:42:49.230560+0000","last_became_peered":"2024-07-17T08:42:49.230560+0000","last_unstale":"2024-07-17T08:42:49.234848+0000","last_undegraded":"2024-07-17T08:42:49.234848+0000","last_fullsized":"2024-07-17T08:42:49.234848+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:23.998448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.2e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243682+0000","last_change":"2024-07-17T08:42:49.240627+0000","last_active":"2024-07-17T08:42:49.243682+0000","last_peered":"2024-07-17T08:42:49.243682+0000","last_clean":"2024-07-17T08:42:49.243682+0000","last_became_active":"2024-07-17T08:42:49.240334+0000","last_became_peered":"2024-07-17T08:42:49.240334+0000","last_unstale":"2024-07-17T08:42:49.243682+0000","last_undegraded":"2024-07-17T08:42:49.243682+0000","last_fullsized":"2024-07-17T08:42:49.243682+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:54:08.252838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.2d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.246618+0000","last_change":"2024-07-17T08:42:49.242068+0000","last_active":"2024-07-17T08:42:49.246618+0000","last_peered":"2024-07-17T08:42:49.246618+0000","last_clean":"2024-07-17T08:42:49.246618+0000","last_became_active":"2024-07-17T08:42:49.237514+0000","last_became_peered":"2024-07-17T08:42:49.237514+0000","last_unstale":"2024-07-17T08:42:49.246618+0000","last_undegraded":"2024-07-17T08:42:49.246618+0000","last_fullsized":"2024-07-17T08:42:49.246618+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:38:21.501649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.2c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.223019+0000","last_change":"2024-07-17T08:42:49.215499+0000","last_active":"2024-07-17T08:42:49.223019+0000","last_peered":"2024-07-17T08:42:49.223019+0000","last_clean":"2024-07-17T08:42:49.223019+0000","last_became_active":"2024-07-17T08:42:49.214503+0000","last_became_peered":"2024-07-17T08:42:49.214503+0000","last_unstale":"2024-07-17T08:42:49.223019+0000","last_undegraded":"2024-07-17T08:42:49.223019+0000","last_fullsized":"2024-07-17T08:42:49.223019+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:46:29.103470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.2b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245544+0000","last_change":"2024-07-17T08:42:49.241313+0000","last_active":"2024-07-17T08:42:49.245544+0000","last_peered":"2024-07-17T08:42:49.245544+0000","last_clean":"2024-07-17T08:42:49.245544+0000","last_became_active":"2024-07-17T08:42:49.241017+0000","last_became_peered":"2024-07-17T08:42:49.241017+0000","last_unstale":"2024-07-17T08:42:49.245544+0000","last_undegraded":"2024-07-17T08:42:49.245544+0000","last_fullsized":"2024-07-17T08:42:49.245544+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:24.397193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.2a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.216211+0000","last_change":"2024-07-17T08:42:49.214125+0000","last_active":"2024-07-17T08:42:49.216211+0000","last_peered":"2024-07-17T08:42:49.216211+0000","last_clean":"2024-07-17T08:42:49.216211+0000","last_became_active":"2024-07-17T08:42:49.213847+0000","last_became_peered":"2024-07-17T08:42:49.213847+0000","last_unstale":"2024-07-17T08:42:49.216211+0000","last_undegraded":"2024-07-17T08:42:49.216211+0000","last_fullsized":"2024-07-17T08:42:49.216211+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:43.713168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.29","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.226765+0000","last_change":"2024-07-17T08:42:49.222751+0000","last_active":"2024-07-17T08:42:49.226765+0000","last_peered":"2024-07-17T08:42:49.226765+0000","last_clean":"2024-07-17T08:42:49.226765+0000","last_became_active":"2024-07-17T08:42:49.221500+0000","last_became_peered":"2024-07-17T08:42:49.221500+0000","last_unstale":"2024-07-17T08:42:49.226765+0000","last_undegraded":"2024-07-17T08:42:49.226765+0000","last_fullsized":"2024-07-17T08:42:49.226765+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:23:43.549003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.28","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243612+0000","last_change":"2024-07-17T08:42:49.239433+0000","last_active":"2024-07-17T08:42:49.243612+0000","last_peered":"2024-07-17T08:42:49.243612+0000","last_clean":"2024-07-17T08:42:49.243612+0000","last_became_active":"2024-07-17T08:42:49.239147+0000","last_became_peered":"2024-07-17T08:42:49.239147+0000","last_unstale":"2024-07-17T08:42:49.243612+0000","last_undegraded":"2024-07-17T08:42:49.243612+0000","last_fullsized":"2024-07-17T08:42:49.243612+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:47.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.27","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247620+0000","last_change":"2024-07-17T08:42:49.244353+0000","last_active":"2024-07-17T08:42:49.247620+0000","last_peered":"2024-07-17T08:42:49.247620+0000","last_clean":"2024-07-17T08:42:49.247620+0000","last_became_active":"2024-07-17T08:42:49.244031+0000","last_became_peered":"2024-07-17T08:42:49.244031+0000","last_unstale":"2024-07-17T08:42:49.247620+0000","last_undegraded":"2024-07-17T08:42:49.247620+0000","last_fullsized":"2024-07-17T08:42:49.247620+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:48.496046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.26","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235487+0000","last_change":"2024-07-17T08:42:49.233478+0000","last_active":"2024-07-17T08:42:49.235487+0000","last_peered":"2024-07-17T08:42:49.235487+0000","last_clean":"2024-07-17T08:42:49.235487+0000","last_became_active":"2024-07-17T08:42:49.232250+0000","last_became_peered":"2024-07-17T08:42:49.232250+0000","last_unstale":"2024-07-17T08:42:49.235487+0000","last_undegraded":"2024-07-17T08:42:49.235487+0000","last_fullsized":"2024-07-17T08:42:49.235487+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:34.252723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.25","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.246592+0000","last_change":"2024-07-17T08:42:49.237799+0000","last_active":"2024-07-17T08:42:49.246592+0000","last_peered":"2024-07-17T08:42:49.246592+0000","last_clean":"2024-07-17T08:42:49.246592+0000","last_became_active":"2024-07-17T08:42:49.236754+0000","last_became_peered":"2024-07-17T08:42:49.236754+0000","last_unstale":"2024-07-17T08:42:49.246592+0000","last_undegraded":"2024-07-17T08:42:49.246592+0000","last_fullsized":"2024-07-17T08:42:49.246592+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:09.756785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.24","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241945+0000","last_change":"2024-07-17T08:42:49.228957+0000","last_active":"2024-07-17T08:42:49.241945+0000","last_peered":"2024-07-17T08:42:49.241945+0000","last_clean":"2024-07-17T08:42:49.241945+0000","last_became_active":"2024-07-17T08:42:49.228695+0000","last_became_peered":"2024-07-17T08:42:49.228695+0000","last_unstale":"2024-07-17T08:42:49.241945+0000","last_undegraded":"2024-07-17T08:42:49.241945+0000","last_fullsized":"2024-07-17T08:42:49.241945+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:14:48.746007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.23","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.242773+0000","last_change":"2024-07-17T08:42:49.239430+0000","last_active":"2024-07-17T08:42:49.242773+0000","last_peered":"2024-07-17T08:42:49.242773+0000","last_clean":"2024-07-17T08:42:49.242773+0000","last_became_active":"2024-07-17T08:42:49.239140+0000","last_became_peered":"2024-07-17T08:42:49.239140+0000","last_unstale":"2024-07-17T08:42:49.242773+0000","last_undegraded":"2024-07-17T08:42:49.242773+0000","last_fullsized":"2024-07-17T08:42:49.242773+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:26.733393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.22","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248104+0000","last_change":"2024-07-17T08:42:49.243780+0000","last_active":"2024-07-17T08:42:49.248104+0000","last_peered":"2024-07-17T08:42:49.248104+0000","last_clean":"2024-07-17T08:42:49.248104+0000","last_became_active":"2024-07-17T08:42:49.243496+0000","last_became_peered":"2024-07-17T08:42:49.243496+0000","last_unstale":"2024-07-17T08:42:49.248104+0000","last_undegraded":"2024-07-17T08:42:49.248104+0000","last_fullsized":"2024-07-17T08:42:49.248104+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:19.401904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.21","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.242207+0000","last_change":"2024-07-17T08:42:49.236123+0000","last_active":"2024-07-17T08:42:49.242207+0000","last_peered":"2024-07-17T08:42:49.242207+0000","last_clean":"2024-07-17T08:42:49.242207+0000","last_became_active":"2024-07-17T08:42:49.235826+0000","last_became_peered":"2024-07-17T08:42:49.235826+0000","last_unstale":"2024-07-17T08:42:49.242207+0000","last_undegraded":"2024-07-17T08:42:49.242207+0000","last_fullsized":"2024-07-17T08:42:49.242207+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:28.405741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.20","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.236936+0000","last_change":"2024-07-17T08:42:49.228470+0000","last_active":"2024-07-17T08:42:49.236936+0000","last_peered":"2024-07-17T08:42:49.236936+0000","last_clean":"2024-07-17T08:42:49.236936+0000","last_became_active":"2024-07-17T08:42:49.228204+0000","last_became_peered":"2024-07-17T08:42:49.228204+0000","last_unstale":"2024-07-17T08:42:49.236936+0000","last_undegraded":"2024-07-17T08:42:49.236936+0000","last_fullsized":"2024-07-17T08:42:49.236936+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:25:18.197375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.1c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234878+0000","last_change":"2024-07-17T08:42:49.232819+0000","last_active":"2024-07-17T08:42:49.234878+0000","last_peered":"2024-07-17T08:42:49.234878+0000","last_clean":"2024-07-17T08:42:49.234878+0000","last_became_active":"2024-07-17T08:42:49.232455+0000","last_became_peered":"2024-07-17T08:42:49.232455+0000","last_unstale":"2024-07-17T08:42:49.234878+0000","last_undegraded":"2024-07-17T08:42:49.234878+0000","last_fullsized":"2024-07-17T08:42:49.234878+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:52.788077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.1f","version":"334'4","reported_seq":16,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249531+0000","last_change":"2024-07-17T08:42:48.200065+0000","last_active":"2024-07-17T08:42:49.249531+0000","last_peered":"2024-07-17T08:42:49.249531+0000","last_clean":"2024-07-17T08:42:49.249531+0000","last_became_active":"2024-07-17T08:42:48.199156+0000","last_became_peered":"2024-07-17T08:42:48.199156+0000","last_unstale":"2024-07-17T08:42:49.249531+0000","last_undegraded":"2024-07-17T08:42:49.249531+0000","last_fullsized":"2024-07-17T08:42:49.249531+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:50:03.485720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.1d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239162+0000","last_change":"2024-07-17T08:42:49.235456+0000","last_active":"2024-07-17T08:42:49.239162+0000","last_peered":"2024-07-17T08:42:49.239162+0000","last_clean":"2024-07-17T08:42:49.239162+0000","last_became_active":"2024-07-17T08:42:49.235167+0000","last_became_peered":"2024-07-17T08:42:49.235167+0000","last_unstale":"2024-07-17T08:42:49.239162+0000","last_undegraded":"2024-07-17T08:42:49.239162+0000","last_fullsized":"2024-07-17T08:42:49.239162+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:35:37.507611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.1e","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251597+0000","last_change":"2024-07-17T08:42:48.207167+0000","last_active":"2024-07-17T08:42:49.251597+0000","last_peered":"2024-07-17T08:42:49.251597+0000","last_clean":"2024-07-17T08:42:49.251597+0000","last_became_active":"2024-07-17T08:42:48.206785+0000","last_became_peered":"2024-07-17T08:42:48.206785+0000","last_unstale":"2024-07-17T08:42:49.251597+0000","last_undegraded":"2024-07-17T08:42:49.251597+0000","last_fullsized":"2024-07-17T08:42:49.251597+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:04.823813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.1e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.237466+0000","last_change":"2024-07-17T08:42:49.231754+0000","last_active":"2024-07-17T08:42:49.237466+0000","last_peered":"2024-07-17T08:42:49.237466+0000","last_clean":"2024-07-17T08:42:49.237466+0000","last_became_active":"2024-07-17T08:42:49.231448+0000","last_became_peered":"2024-07-17T08:42:49.231448+0000","last_unstale":"2024-07-17T08:42:49.237466+0000","last_undegraded":"2024-07-17T08:42:49.237466+0000","last_fullsized":"2024-07-17T08:42:49.237466+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:09:08.305999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.1d","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248688+0000","last_change":"2024-07-17T08:42:48.206235+0000","last_active":"2024-07-17T08:42:49.248688+0000","last_peered":"2024-07-17T08:42:49.248688+0000","last_clean":"2024-07-17T08:42:49.248688+0000","last_became_active":"2024-07-17T08:42:48.205952+0000","last_became_peered":"2024-07-17T08:42:48.205952+0000","last_unstale":"2024-07-17T08:42:49.248688+0000","last_undegraded":"2024-07-17T08:42:49.248688+0000","last_fullsized":"2024-07-17T08:42:49.248688+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:55.318098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.1f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239897+0000","last_change":"2024-07-17T08:42:49.237762+0000","last_active":"2024-07-17T08:42:49.239897+0000","last_peered":"2024-07-17T08:42:49.239897+0000","last_clean":"2024-07-17T08:42:49.239897+0000","last_became_active":"2024-07-17T08:42:49.237466+0000","last_became_peered":"2024-07-17T08:42:49.237466+0000","last_unstale":"2024-07-17T08:42:49.239897+0000","last_undegraded":"2024-07-17T08:42:49.239897+0000","last_fullsized":"2024-07-17T08:42:49.239897+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:11:47.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.1c","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248200+0000","last_change":"2024-07-17T08:42:48.203047+0000","last_active":"2024-07-17T08:42:49.248200+0000","last_peered":"2024-07-17T08:42:49.248200+0000","last_clean":"2024-07-17T08:42:49.248200+0000","last_became_active":"2024-07-17T08:42:48.202780+0000","last_became_peered":"2024-07-17T08:42:48.202780+0000","last_unstale":"2024-07-17T08:42:49.248200+0000","last_undegraded":"2024-07-17T08:42:49.248200+0000","last_fullsized":"2024-07-17T08:42:49.248200+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:42.648872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.18","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229340+0000","last_change":"2024-07-17T08:42:49.227333+0000","last_active":"2024-07-17T08:42:49.229340+0000","last_peered":"2024-07-17T08:42:49.229340+0000","last_clean":"2024-07-17T08:42:49.229340+0000","last_became_active":"2024-07-17T08:42:49.226092+0000","last_became_peered":"2024-07-17T08:42:49.226092+0000","last_unstale":"2024-07-17T08:42:49.229340+0000","last_undegraded":"2024-07-17T08:42:49.229340+0000","last_fullsized":"2024-07-17T08:42:49.229340+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:03.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.1b","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.251221+0000","last_change":"2024-07-17T08:42:48.192186+0000","last_active":"2024-07-17T08:42:49.251221+0000","last_peered":"2024-07-17T08:42:49.251221+0000","last_clean":"2024-07-17T08:42:49.251221+0000","last_became_active":"2024-07-17T08:42:48.191891+0000","last_became_peered":"2024-07-17T08:42:48.191891+0000","last_unstale":"2024-07-17T08:42:49.251221+0000","last_undegraded":"2024-07-17T08:42:49.251221+0000","last_fullsized":"2024-07-17T08:42:49.251221+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:32:46.445238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.19","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.228306+0000","last_change":"2024-07-17T08:42:49.226762+0000","last_active":"2024-07-17T08:42:49.228306+0000","last_peered":"2024-07-17T08:42:49.228306+0000","last_clean":"2024-07-17T08:42:49.228306+0000","last_became_active":"2024-07-17T08:42:49.226442+0000","last_became_peered":"2024-07-17T08:42:49.226442+0000","last_unstale":"2024-07-17T08:42:49.228306+0000","last_undegraded":"2024-07-17T08:42:49.228306+0000","last_fullsized":"2024-07-17T08:42:49.228306+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:35:05.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.1a","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.244666+0000","last_change":"2024-07-17T08:42:48.203897+0000","last_active":"2024-07-17T08:42:49.244666+0000","last_peered":"2024-07-17T08:42:49.244666+0000","last_clean":"2024-07-17T08:42:49.244666+0000","last_became_active":"2024-07-17T08:42:48.203377+0000","last_became_peered":"2024-07-17T08:42:48.203377+0000","last_unstale":"2024-07-17T08:42:49.244666+0000","last_undegraded":"2024-07-17T08:42:49.244666+0000","last_fullsized":"2024-07-17T08:42:49.244666+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:27:00.166950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.1a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231034+0000","last_change":"2024-07-17T08:42:49.223355+0000","last_active":"2024-07-17T08:42:49.231034+0000","last_peered":"2024-07-17T08:42:49.231034+0000","last_clean":"2024-07-17T08:42:49.231034+0000","last_became_active":"2024-07-17T08:42:49.223056+0000","last_became_peered":"2024-07-17T08:42:49.223056+0000","last_unstale":"2024-07-17T08:42:49.231034+0000","last_undegraded":"2024-07-17T08:42:49.231034+0000","last_fullsized":"2024-07-17T08:42:49.231034+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:36:06.714326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.19","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.225604+0000","last_change":"2024-07-17T08:42:48.203245+0000","last_active":"2024-07-17T08:42:49.225604+0000","last_peered":"2024-07-17T08:42:49.225604+0000","last_clean":"2024-07-17T08:42:49.225604+0000","last_became_active":"2024-07-17T08:42:48.202883+0000","last_became_peered":"2024-07-17T08:42:48.202883+0000","last_unstale":"2024-07-17T08:42:49.225604+0000","last_undegraded":"2024-07-17T08:42:49.225604+0000","last_fullsized":"2024-07-17T08:42:49.225604+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:10.926602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.1b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229367+0000","last_change":"2024-07-17T08:42:49.222635+0000","last_active":"2024-07-17T08:42:49.229367+0000","last_peered":"2024-07-17T08:42:49.229367+0000","last_clean":"2024-07-17T08:42:49.229367+0000","last_became_active":"2024-07-17T08:42:49.222222+0000","last_became_peered":"2024-07-17T08:42:49.222222+0000","last_unstale":"2024-07-17T08:42:49.229367+0000","last_undegraded":"2024-07-17T08:42:49.229367+0000","last_fullsized":"2024-07-17T08:42:49.229367+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:01:01.573358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.18","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.202896+0000","last_change":"2024-07-17T08:42:48.189906+0000","last_active":"2024-07-17T08:42:49.202896+0000","last_peered":"2024-07-17T08:42:49.202896+0000","last_clean":"2024-07-17T08:42:49.202896+0000","last_became_active":"2024-07-17T08:42:48.188419+0000","last_became_peered":"2024-07-17T08:42:48.188419+0000","last_unstale":"2024-07-17T08:42:49.202896+0000","last_undegraded":"2024-07-17T08:42:49.202896+0000","last_fullsized":"2024-07-17T08:42:49.202896+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:26:49.408645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.14","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241618+0000","last_change":"2024-07-17T08:42:49.237791+0000","last_active":"2024-07-17T08:42:49.241618+0000","last_peered":"2024-07-17T08:42:49.241618+0000","last_clean":"2024-07-17T08:42:49.241618+0000","last_became_active":"2024-07-17T08:42:49.237484+0000","last_became_peered":"2024-07-17T08:42:49.237484+0000","last_unstale":"2024-07-17T08:42:49.241618+0000","last_undegraded":"2024-07-17T08:42:49.241618+0000","last_fullsized":"2024-07-17T08:42:49.241618+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:40:23.749782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.17","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212890+0000","last_change":"2024-07-17T08:42:48.191283+0000","last_active":"2024-07-17T08:42:49.212890+0000","last_peered":"2024-07-17T08:42:49.212890+0000","last_clean":"2024-07-17T08:42:49.212890+0000","last_became_active":"2024-07-17T08:42:48.190826+0000","last_became_peered":"2024-07-17T08:42:48.190826+0000","last_unstale":"2024-07-17T08:42:49.212890+0000","last_undegraded":"2024-07-17T08:42:49.212890+0000","last_fullsized":"2024-07-17T08:42:49.212890+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:54:20.634436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.15","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.236557+0000","last_change":"2024-07-17T08:42:49.232776+0000","last_active":"2024-07-17T08:42:49.236557+0000","last_peered":"2024-07-17T08:42:49.236557+0000","last_clean":"2024-07-17T08:42:49.236557+0000","last_became_active":"2024-07-17T08:42:49.232453+0000","last_became_peered":"2024-07-17T08:42:49.232453+0000","last_unstale":"2024-07-17T08:42:49.236557+0000","last_undegraded":"2024-07-17T08:42:49.236557+0000","last_fullsized":"2024-07-17T08:42:49.236557+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:52:00.939094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"45.16","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.214730+0000","last_change":"2024-07-17T08:42:48.209763+0000","last_active":"2024-07-17T08:42:49.214730+0000","last_peered":"2024-07-17T08:42:49.214730+0000","last_clean":"2024-07-17T08:42:49.214730+0000","last_became_active":"2024-07-17T08:42:48.209420+0000","last_became_peered":"2024-07-17T08:42:48.209420+0000","last_unstale":"2024-07-17T08:42:49.214730+0000","last_undegraded":"2024-07-17T08:42:49.214730+0000","last_fullsized":"2024-07-17T08:42:49.214730+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:34:36.992656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.16","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.243668+0000","last_change":"2024-07-17T08:42:49.240996+0000","last_active":"2024-07-17T08:42:49.243668+0000","last_peered":"2024-07-17T08:42:49.243668+0000","last_clean":"2024-07-17T08:42:49.243668+0000","last_became_active":"2024-07-17T08:42:49.240471+0000","last_became_peered":"2024-07-17T08:42:49.240471+0000","last_unstale":"2024-07-17T08:42:49.243668+0000","last_undegraded":"2024-07-17T08:42:49.243668+0000","last_fullsized":"2024-07-17T08:42:49.243668+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:04:35.972200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.15","version":"334'3","reported_seq":15,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250059+0000","last_change":"2024-07-17T08:42:48.207171+0000","last_active":"2024-07-17T08:42:49.250059+0000","last_peered":"2024-07-17T08:42:49.250059+0000","last_clean":"2024-07-17T08:42:49.250059+0000","last_became_active":"2024-07-17T08:42:48.206909+0000","last_became_peered":"2024-07-17T08:42:48.206909+0000","last_unstale":"2024-07-17T08:42:49.250059+0000","last_undegraded":"2024-07-17T08:42:49.250059+0000","last_fullsized":"2024-07-17T08:42:49.250059+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:55:48.824122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.17","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231588+0000","last_change":"2024-07-17T08:42:49.228869+0000","last_active":"2024-07-17T08:42:49.231588+0000","last_peered":"2024-07-17T08:42:49.231588+0000","last_clean":"2024-07-17T08:42:49.231588+0000","last_became_active":"2024-07-17T08:42:49.228529+0000","last_became_peered":"2024-07-17T08:42:49.228529+0000","last_unstale":"2024-07-17T08:42:49.231588+0000","last_undegraded":"2024-07-17T08:42:49.231588+0000","last_fullsized":"2024-07-17T08:42:49.231588+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:12.675925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"45.14","version":"334'6","reported_seq":18,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.253239+0000","last_change":"2024-07-17T08:42:48.199744+0000","last_active":"2024-07-17T08:42:49.253239+0000","last_peered":"2024-07-17T08:42:49.253239+0000","last_clean":"2024-07-17T08:42:49.253239+0000","last_became_active":"2024-07-17T08:42:48.199437+0000","last_became_peered":"2024-07-17T08:42:48.199437+0000","last_unstale":"2024-07-17T08:42:49.253239+0000","last_undegraded":"2024-07-17T08:42:49.253239+0000","last_fullsized":"2024-07-17T08:42:49.253239+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:42.648872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"46.10","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249829+0000","last_change":"2024-07-17T08:42:49.241942+0000","last_active":"2024-07-17T08:42:49.249829+0000","last_peered":"2024-07-17T08:42:49.249829+0000","last_clean":"2024-07-17T08:42:49.249829+0000","last_became_active":"2024-07-17T08:42:49.236994+0000","last_became_peered":"2024-07-17T08:42:49.236994+0000","last_unstale":"2024-07-17T08:42:49.249829+0000","last_undegraded":"2024-07-17T08:42:49.249829+0000","last_fullsized":"2024-07-17T08:42:49.249829+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:17:22.860542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.13","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250105+0000","last_change":"2024-07-17T08:42:48.193918+0000","last_active":"2024-07-17T08:42:49.250105+0000","last_peered":"2024-07-17T08:42:49.250105+0000","last_clean":"2024-07-17T08:42:49.250105+0000","last_became_active":"2024-07-17T08:42:48.193512+0000","last_became_peered":"2024-07-17T08:42:48.193512+0000","last_unstale":"2024-07-17T08:42:49.250105+0000","last_undegraded":"2024-07-17T08:42:49.250105+0000","last_fullsized":"2024-07-17T08:42:49.250105+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:51:53.601087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.11","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239224+0000","last_change":"2024-07-17T08:42:49.237616+0000","last_active":"2024-07-17T08:42:49.239224+0000","last_peered":"2024-07-17T08:42:49.239224+0000","last_clean":"2024-07-17T08:42:49.239224+0000","last_became_active":"2024-07-17T08:42:49.237101+0000","last_became_peered":"2024-07-17T08:42:49.237101+0000","last_unstale":"2024-07-17T08:42:49.239224+0000","last_undegraded":"2024-07-17T08:42:49.239224+0000","last_fullsized":"2024-07-17T08:42:49.239224+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:20:35.196396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.12","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213914+0000","last_change":"2024-07-17T08:42:48.191342+0000","last_active":"2024-07-17T08:42:49.213914+0000","last_peered":"2024-07-17T08:42:49.213914+0000","last_clean":"2024-07-17T08:42:49.213914+0000","last_became_active":"2024-07-17T08:42:48.190955+0000","last_became_peered":"2024-07-17T08:42:48.190955+0000","last_unstale":"2024-07-17T08:42:49.213914+0000","last_undegraded":"2024-07-17T08:42:49.213914+0000","last_fullsized":"2024-07-17T08:42:49.213914+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:22:38.672577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.12","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241257+0000","last_change":"2024-07-17T08:42:49.237435+0000","last_active":"2024-07-17T08:42:49.241257+0000","last_peered":"2024-07-17T08:42:49.241257+0000","last_clean":"2024-07-17T08:42:49.241257+0000","last_became_active":"2024-07-17T08:42:49.237096+0000","last_became_peered":"2024-07-17T08:42:49.237096+0000","last_unstale":"2024-07-17T08:42:49.241257+0000","last_undegraded":"2024-07-17T08:42:49.241257+0000","last_fullsized":"2024-07-17T08:42:49.241257+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:39:23.249824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.11","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.218694+0000","last_change":"2024-07-17T08:42:48.203957+0000","last_active":"2024-07-17T08:42:49.218694+0000","last_peered":"2024-07-17T08:42:49.218694+0000","last_clean":"2024-07-17T08:42:49.218694+0000","last_became_active":"2024-07-17T08:42:48.203578+0000","last_became_peered":"2024-07-17T08:42:48.203578+0000","last_unstale":"2024-07-17T08:42:49.218694+0000","last_undegraded":"2024-07-17T08:42:49.218694+0000","last_fullsized":"2024-07-17T08:42:49.218694+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:00.618114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.13","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.231186+0000","last_change":"2024-07-17T08:42:49.224064+0000","last_active":"2024-07-17T08:42:49.231186+0000","last_peered":"2024-07-17T08:42:49.231186+0000","last_clean":"2024-07-17T08:42:49.231186+0000","last_became_active":"2024-07-17T08:42:49.223788+0000","last_became_peered":"2024-07-17T08:42:49.223788+0000","last_unstale":"2024-07-17T08:42:49.231186+0000","last_undegraded":"2024-07-17T08:42:49.231186+0000","last_fullsized":"2024-07-17T08:42:49.231186+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:03.528826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.10","version":"334'8","reported_seq":20,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.250888+0000","last_change":"2024-07-17T08:42:48.203718+0000","last_active":"2024-07-17T08:42:49.250888+0000","last_peered":"2024-07-17T08:42:49.250888+0000","last_clean":"2024-07-17T08:42:49.250888+0000","last_became_active":"2024-07-17T08:42:48.203373+0000","last_became_peered":"2024-07-17T08:42:48.203373+0000","last_unstale":"2024-07-17T08:42:49.250888+0000","last_undegraded":"2024-07-17T08:42:49.250888+0000","last_fullsized":"2024-07-17T08:42:49.250888+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:24:51.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.c","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229431+0000","last_change":"2024-07-17T08:42:49.228442+0000","last_active":"2024-07-17T08:42:49.229431+0000","last_peered":"2024-07-17T08:42:49.229431+0000","last_clean":"2024-07-17T08:42:49.229431+0000","last_became_active":"2024-07-17T08:42:49.226346+0000","last_became_peered":"2024-07-17T08:42:49.226346+0000","last_unstale":"2024-07-17T08:42:49.229431+0000","last_undegraded":"2024-07-17T08:42:49.229431+0000","last_fullsized":"2024-07-17T08:42:49.229431+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:54.934703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.f","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252663+0000","last_change":"2024-07-17T08:42:48.191313+0000","last_active":"2024-07-17T08:42:49.252663+0000","last_peered":"2024-07-17T08:42:49.252663+0000","last_clean":"2024-07-17T08:42:49.252663+0000","last_became_active":"2024-07-17T08:42:48.191037+0000","last_became_peered":"2024-07-17T08:42:48.191037+0000","last_unstale":"2024-07-17T08:42:49.252663+0000","last_undegraded":"2024-07-17T08:42:49.252663+0000","last_fullsized":"2024-07-17T08:42:49.252663+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:49:00.333893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.d","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245456+0000","last_change":"2024-07-17T08:42:49.233170+0000","last_active":"2024-07-17T08:42:49.245456+0000","last_peered":"2024-07-17T08:42:49.245456+0000","last_clean":"2024-07-17T08:42:49.245456+0000","last_became_active":"2024-07-17T08:42:49.232742+0000","last_became_peered":"2024-07-17T08:42:49.232742+0000","last_unstale":"2024-07-17T08:42:49.245456+0000","last_undegraded":"2024-07-17T08:42:49.245456+0000","last_fullsized":"2024-07-17T08:42:49.245456+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:52:08.251882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.e","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241780+0000","last_change":"2024-07-17T08:42:48.209845+0000","last_active":"2024-07-17T08:42:49.241780+0000","last_peered":"2024-07-17T08:42:49.241780+0000","last_clean":"2024-07-17T08:42:49.241780+0000","last_became_active":"2024-07-17T08:42:48.209408+0000","last_became_peered":"2024-07-17T08:42:48.209408+0000","last_unstale":"2024-07-17T08:42:49.241780+0000","last_undegraded":"2024-07-17T08:42:49.241780+0000","last_fullsized":"2024-07-17T08:42:49.241780+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:56.440721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.e","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.237490+0000","last_change":"2024-07-17T08:42:49.234587+0000","last_active":"2024-07-17T08:42:49.237490+0000","last_peered":"2024-07-17T08:42:49.237490+0000","last_clean":"2024-07-17T08:42:49.237490+0000","last_became_active":"2024-07-17T08:42:49.234262+0000","last_became_peered":"2024-07-17T08:42:49.234262+0000","last_unstale":"2024-07-17T08:42:49.237490+0000","last_undegraded":"2024-07-17T08:42:49.237490+0000","last_fullsized":"2024-07-17T08:42:49.237490+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:55:56.691955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.d","version":"334'1","reported_seq":13,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252016+0000","last_change":"2024-07-17T08:42:48.208416+0000","last_active":"2024-07-17T08:42:49.252016+0000","last_peered":"2024-07-17T08:42:49.252016+0000","last_clean":"2024-07-17T08:42:49.252016+0000","last_became_active":"2024-07-17T08:42:48.208125+0000","last_became_peered":"2024-07-17T08:42:48.208125+0000","last_unstale":"2024-07-17T08:42:49.252016+0000","last_undegraded":"2024-07-17T08:42:49.252016+0000","last_fullsized":"2024-07-17T08:42:49.252016+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:56:16.472678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229434+0000","last_change":"2024-07-17T08:42:49.227907+0000","last_active":"2024-07-17T08:42:49.229434+0000","last_peered":"2024-07-17T08:42:49.229434+0000","last_clean":"2024-07-17T08:42:49.229434+0000","last_became_active":"2024-07-17T08:42:49.227624+0000","last_became_peered":"2024-07-17T08:42:49.227624+0000","last_unstale":"2024-07-17T08:42:49.229434+0000","last_undegraded":"2024-07-17T08:42:49.229434+0000","last_fullsized":"2024-07-17T08:42:49.229434+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:47:30.979141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.c","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212686+0000","last_change":"2024-07-17T08:42:48.192166+0000","last_active":"2024-07-17T08:42:49.212686+0000","last_peered":"2024-07-17T08:42:49.212686+0000","last_clean":"2024-07-17T08:42:49.212686+0000","last_became_active":"2024-07-17T08:42:48.191763+0000","last_became_peered":"2024-07-17T08:42:48.191763+0000","last_unstale":"2024-07-17T08:42:49.212686+0000","last_undegraded":"2024-07-17T08:42:49.212686+0000","last_fullsized":"2024-07-17T08:42:49.212686+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:23:36.445437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.8","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238733+0000","last_change":"2024-07-17T08:42:49.234007+0000","last_active":"2024-07-17T08:42:49.238733+0000","last_peered":"2024-07-17T08:42:49.238733+0000","last_clean":"2024-07-17T08:42:49.238733+0000","last_became_active":"2024-07-17T08:42:49.232237+0000","last_became_peered":"2024-07-17T08:42:49.232237+0000","last_unstale":"2024-07-17T08:42:49.238733+0000","last_undegraded":"2024-07-17T08:42:49.238733+0000","last_fullsized":"2024-07-17T08:42:49.238733+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:04:48.496046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.b","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247980+0000","last_change":"2024-07-17T08:42:48.199031+0000","last_active":"2024-07-17T08:42:49.247980+0000","last_peered":"2024-07-17T08:42:49.247980+0000","last_clean":"2024-07-17T08:42:49.247980+0000","last_became_active":"2024-07-17T08:42:48.198707+0000","last_became_peered":"2024-07-17T08:42:48.198707+0000","last_unstale":"2024-07-17T08:42:49.247980+0000","last_undegraded":"2024-07-17T08:42:49.247980+0000","last_fullsized":"2024-07-17T08:42:49.247980+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:39:22.983373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.9","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252044+0000","last_change":"2024-07-17T08:42:49.247212+0000","last_active":"2024-07-17T08:42:49.252044+0000","last_peered":"2024-07-17T08:42:49.252044+0000","last_clean":"2024-07-17T08:42:49.252044+0000","last_became_active":"2024-07-17T08:42:49.246161+0000","last_became_peered":"2024-07-17T08:42:49.246161+0000","last_unstale":"2024-07-17T08:42:49.252044+0000","last_undegraded":"2024-07-17T08:42:49.252044+0000","last_fullsized":"2024-07-17T08:42:49.252044+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:01:37.755601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.a","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213905+0000","last_change":"2024-07-17T08:42:48.195854+0000","last_active":"2024-07-17T08:42:49.213905+0000","last_peered":"2024-07-17T08:42:49.213905+0000","last_clean":"2024-07-17T08:42:49.213905+0000","last_became_active":"2024-07-17T08:42:48.195346+0000","last_became_peered":"2024-07-17T08:42:48.195346+0000","last_unstale":"2024-07-17T08:42:49.213905+0000","last_undegraded":"2024-07-17T08:42:49.213905+0000","last_fullsized":"2024-07-17T08:42:49.213905+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:23:01.872400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.a","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.235333+0000","last_change":"2024-07-17T08:42:49.234127+0000","last_active":"2024-07-17T08:42:49.235333+0000","last_peered":"2024-07-17T08:42:49.235333+0000","last_clean":"2024-07-17T08:42:49.235333+0000","last_became_active":"2024-07-17T08:42:49.233833+0000","last_became_peered":"2024-07-17T08:42:49.233833+0000","last_unstale":"2024-07-17T08:42:49.235333+0000","last_undegraded":"2024-07-17T08:42:49.235333+0000","last_fullsized":"2024-07-17T08:42:49.235333+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:41:31.551567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.9","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212005+0000","last_change":"2024-07-17T08:42:48.206101+0000","last_active":"2024-07-17T08:42:49.212005+0000","last_peered":"2024-07-17T08:42:49.212005+0000","last_clean":"2024-07-17T08:42:49.212005+0000","last_became_active":"2024-07-17T08:42:48.205793+0000","last_became_peered":"2024-07-17T08:42:48.205793+0000","last_unstale":"2024-07-17T08:42:49.212005+0000","last_undegraded":"2024-07-17T08:42:49.212005+0000","last_fullsized":"2024-07-17T08:42:49.212005+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:27.788182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.b","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252301+0000","last_change":"2024-07-17T08:42:49.249103+0000","last_active":"2024-07-17T08:42:49.252301+0000","last_peered":"2024-07-17T08:42:49.252301+0000","last_clean":"2024-07-17T08:42:49.252301+0000","last_became_active":"2024-07-17T08:42:49.243780+0000","last_became_peered":"2024-07-17T08:42:49.243780+0000","last_unstale":"2024-07-17T08:42:49.252301+0000","last_undegraded":"2024-07-17T08:42:49.252301+0000","last_fullsized":"2024-07-17T08:42:49.252301+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:43:15.792625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.8","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229693+0000","last_change":"2024-07-17T08:42:48.201389+0000","last_active":"2024-07-17T08:42:49.229693+0000","last_peered":"2024-07-17T08:42:49.229693+0000","last_clean":"2024-07-17T08:42:49.229693+0000","last_became_active":"2024-07-17T08:42:48.201094+0000","last_became_peered":"2024-07-17T08:42:48.201094+0000","last_unstale":"2024-07-17T08:42:49.229693+0000","last_undegraded":"2024-07-17T08:42:49.229693+0000","last_fullsized":"2024-07-17T08:42:49.229693+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:10.926602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"46.4","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.248812+0000","last_change":"2024-07-17T08:42:49.246191+0000","last_active":"2024-07-17T08:42:49.248812+0000","last_peered":"2024-07-17T08:42:49.248812+0000","last_clean":"2024-07-17T08:42:49.248812+0000","last_became_active":"2024-07-17T08:42:49.245912+0000","last_became_peered":"2024-07-17T08:42:49.245912+0000","last_unstale":"2024-07-17T08:42:49.248812+0000","last_undegraded":"2024-07-17T08:42:49.248812+0000","last_fullsized":"2024-07-17T08:42:49.248812+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:41:22.466050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"45.7","version":"334'2","reported_seq":14,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.249005+0000","last_change":"2024-07-17T08:42:48.208183+0000","last_active":"2024-07-17T08:42:49.249005+0000","last_peered":"2024-07-17T08:42:49.249005+0000","last_clean":"2024-07-17T08:42:49.249005+0000","last_became_active":"2024-07-17T08:42:48.207624+0000","last_became_peered":"2024-07-17T08:42:48.207624+0000","last_unstale":"2024-07-17T08:42:49.249005+0000","last_undegraded":"2024-07-17T08:42:49.249005+0000","last_fullsized":"2024-07-17T08:42:49.249005+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:24:21.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"46.5","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.239138+0000","last_change":"2024-07-17T08:42:49.234749+0000","last_active":"2024-07-17T08:42:49.239138+0000","last_peered":"2024-07-17T08:42:49.239138+0000","last_clean":"2024-07-17T08:42:49.239138+0000","last_became_active":"2024-07-17T08:42:49.234444+0000","last_became_peered":"2024-07-17T08:42:49.234444+0000","last_unstale":"2024-07-17T08:42:49.239138+0000","last_undegraded":"2024-07-17T08:42:49.239138+0000","last_fullsized":"2024-07-17T08:42:49.239138+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:00.089249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"45.6","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.215200+0000","last_change":"2024-07-17T08:42:48.198045+0000","last_active":"2024-07-17T08:42:49.215200+0000","last_peered":"2024-07-17T08:42:49.215200+0000","last_clean":"2024-07-17T08:42:49.215200+0000","last_became_active":"2024-07-17T08:42:48.197729+0000","last_became_peered":"2024-07-17T08:42:48.197729+0000","last_unstale":"2024-07-17T08:42:49.215200+0000","last_undegraded":"2024-07-17T08:42:49.215200+0000","last_fullsized":"2024-07-17T08:42:49.215200+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:08:30.611918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.6","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.245517+0000","last_change":"2024-07-17T08:42:49.242614+0000","last_active":"2024-07-17T08:42:49.245517+0000","last_peered":"2024-07-17T08:42:49.245517+0000","last_clean":"2024-07-17T08:42:49.245517+0000","last_became_active":"2024-07-17T08:42:49.242342+0000","last_became_peered":"2024-07-17T08:42:49.242342+0000","last_unstale":"2024-07-17T08:42:49.245517+0000","last_undegraded":"2024-07-17T08:42:49.245517+0000","last_fullsized":"2024-07-17T08:42:49.245517+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:49.107565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"45.5","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.219198+0000","last_change":"2024-07-17T08:42:48.203135+0000","last_active":"2024-07-17T08:42:49.219198+0000","last_peered":"2024-07-17T08:42:49.219198+0000","last_clean":"2024-07-17T08:42:49.219198+0000","last_became_active":"2024-07-17T08:42:48.202840+0000","last_became_peered":"2024-07-17T08:42:48.202840+0000","last_unstale":"2024-07-17T08:42:49.219198+0000","last_undegraded":"2024-07-17T08:42:49.219198+0000","last_fullsized":"2024-07-17T08:42:49.219198+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:26.440022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"46.7","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.247178+0000","last_change":"2024-07-17T08:42:49.242996+0000","last_active":"2024-07-17T08:42:49.247178+0000","last_peered":"2024-07-17T08:42:49.247178+0000","last_clean":"2024-07-17T08:42:49.247178+0000","last_became_active":"2024-07-17T08:42:49.242663+0000","last_became_peered":"2024-07-17T08:42:49.242663+0000","last_unstale":"2024-07-17T08:42:49.247178+0000","last_undegraded":"2024-07-17T08:42:49.247178+0000","last_fullsized":"2024-07-17T08:42:49.247178+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:15:23.879797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.4","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.204198+0000","last_change":"2024-07-17T08:42:48.200791+0000","last_active":"2024-07-17T08:42:49.204198+0000","last_peered":"2024-07-17T08:42:49.204198+0000","last_clean":"2024-07-17T08:42:49.204198+0000","last_became_active":"2024-07-17T08:42:48.200357+0000","last_became_peered":"2024-07-17T08:42:48.200357+0000","last_unstale":"2024-07-17T08:42:49.204198+0000","last_undegraded":"2024-07-17T08:42:49.204198+0000","last_fullsized":"2024-07-17T08:42:49.204198+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:51:37.920200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"46.0","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.234013+0000","last_change":"2024-07-17T08:42:49.225973+0000","last_active":"2024-07-17T08:42:49.234013+0000","last_peered":"2024-07-17T08:42:49.234013+0000","last_clean":"2024-07-17T08:42:49.234013+0000","last_became_active":"2024-07-17T08:42:49.225676+0000","last_became_peered":"2024-07-17T08:42:49.225676+0000","last_unstale":"2024-07-17T08:42:49.234013+0000","last_undegraded":"2024-07-17T08:42:49.234013+0000","last_fullsized":"2024-07-17T08:42:49.234013+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:29.388495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"45.3","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.213265+0000","last_change":"2024-07-17T08:42:48.205853+0000","last_active":"2024-07-17T08:42:49.213265+0000","last_peered":"2024-07-17T08:42:49.213265+0000","last_clean":"2024-07-17T08:42:49.213265+0000","last_became_active":"2024-07-17T08:42:48.205053+0000","last_became_peered":"2024-07-17T08:42:48.205053+0000","last_unstale":"2024-07-17T08:42:49.213265+0000","last_undegraded":"2024-07-17T08:42:49.213265+0000","last_fullsized":"2024-07-17T08:42:49.213265+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:37.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.1","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.241431+0000","last_change":"2024-07-17T08:42:49.237873+0000","last_active":"2024-07-17T08:42:49.241431+0000","last_peered":"2024-07-17T08:42:49.241431+0000","last_clean":"2024-07-17T08:42:49.241431+0000","last_became_active":"2024-07-17T08:42:49.237587+0000","last_became_peered":"2024-07-17T08:42:49.237587+0000","last_unstale":"2024-07-17T08:42:49.241431+0000","last_undegraded":"2024-07-17T08:42:49.241431+0000","last_fullsized":"2024-07-17T08:42:49.241431+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:09.037595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"45.2","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.229967+0000","last_change":"2024-07-17T08:42:48.209443+0000","last_active":"2024-07-17T08:42:49.229967+0000","last_peered":"2024-07-17T08:42:49.229967+0000","last_clean":"2024-07-17T08:42:49.229967+0000","last_became_active":"2024-07-17T08:42:48.209097+0000","last_became_peered":"2024-07-17T08:42:48.209097+0000","last_unstale":"2024-07-17T08:42:49.229967+0000","last_undegraded":"2024-07-17T08:42:49.229967+0000","last_fullsized":"2024-07-17T08:42:49.229967+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:37.848317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"46.2","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.252110+0000","last_change":"2024-07-17T08:42:49.245611+0000","last_active":"2024-07-17T08:42:49.252110+0000","last_peered":"2024-07-17T08:42:49.252110+0000","last_clean":"2024-07-17T08:42:49.252110+0000","last_became_active":"2024-07-17T08:42:49.242427+0000","last_became_peered":"2024-07-17T08:42:49.242427+0000","last_unstale":"2024-07-17T08:42:49.252110+0000","last_undegraded":"2024-07-17T08:42:49.252110+0000","last_fullsized":"2024-07-17T08:42:49.252110+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:20:47.866230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"45.1","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.215804+0000","last_change":"2024-07-17T08:42:48.205039+0000","last_active":"2024-07-17T08:42:49.215804+0000","last_peered":"2024-07-17T08:42:49.215804+0000","last_clean":"2024-07-17T08:42:49.215804+0000","last_became_active":"2024-07-17T08:42:48.204758+0000","last_became_peered":"2024-07-17T08:42:48.204758+0000","last_unstale":"2024-07-17T08:42:49.215804+0000","last_undegraded":"2024-07-17T08:42:49.215804+0000","last_fullsized":"2024-07-17T08:42:49.215804+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:09.803427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"46.3","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.240155+0000","last_change":"2024-07-17T08:42:49.236126+0000","last_active":"2024-07-17T08:42:49.240155+0000","last_peered":"2024-07-17T08:42:49.240155+0000","last_clean":"2024-07-17T08:42:49.240155+0000","last_became_active":"2024-07-17T08:42:49.235845+0000","last_became_peered":"2024-07-17T08:42:49.235845+0000","last_unstale":"2024-07-17T08:42:49.240155+0000","last_undegraded":"2024-07-17T08:42:49.240155+0000","last_fullsized":"2024-07-17T08:42:49.240155+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:53.804393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"45.0","version":"0'0","reported_seq":12,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.212917+0000","last_change":"2024-07-17T08:42:48.205131+0000","last_active":"2024-07-17T08:42:49.212917+0000","last_peered":"2024-07-17T08:42:49.212917+0000","last_clean":"2024-07-17T08:42:49.212917+0000","last_became_active":"2024-07-17T08:42:48.204523+0000","last_became_peered":"2024-07-17T08:42:48.204523+0000","last_unstale":"2024-07-17T08:42:49.212917+0000","last_undegraded":"2024-07-17T08:42:49.212917+0000","last_fullsized":"2024-07-17T08:42:49.212917+0000","mapping_epoch":332,"log_start":"0'0","ondisk_log_start":"0'0","created":332,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:47.142948+0000","last_clean_scrub_stamp":"2024-07-17T08:42:47.142948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:35:30.121314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"46.2f","version":"0'0","reported_seq":9,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.238449+0000","last_change":"2024-07-17T08:42:49.234277+0000","last_active":"2024-07-17T08:42:49.238449+0000","last_peered":"2024-07-17T08:42:49.238449+0000","last_clean":"2024-07-17T08:42:49.238449+0000","last_became_active":"2024-07-17T08:42:49.234000+0000","last_became_peered":"2024-07-17T08:42:49.234000+0000","last_unstale":"2024-07-17T08:42:49.238449+0000","last_undegraded":"2024-07-17T08:42:49.238449+0000","last_fullsized":"2024-07-17T08:42:49.238449+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":333,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:42:48.158023+0000","last_clean_scrub_stamp":"2024-07-17T08:42:48.158023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:37:59.565110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"324'4638","reported_seq":5976,"reported_epoch":334,"state":"active+clean","last_fresh":"2024-07-17T08:42:49.224075+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:42:49.224075+0000","last_peered":"2024-07-17T08:42:49.224075+0000","last_clean":"2024-07-17T08:42:49.224075+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:42:49.224075+0000","last_undegraded":"2024-07-17T08:42:49.224075+0000","last_fullsized":"2024-07-17T08:42:49.224075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":517,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":46,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":45,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2209,"num_read_kb":6262,"num_write":517,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4638,"ondisk_log_size":4638,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2925176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53364,"kb_used_data":4136,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318476,"statfs":{"total":96636764160,"available":96582119424,"internally_reserved":0,"allocated":4235264,"data_stored":2334808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510719,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71440,"kb_used_data":4160,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300400,"statfs":{"total":96636764160,"available":96563609600,"internally_reserved":0,"allocated":4259840,"data_stored":2344191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543424,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53388,"kb_used_data":4152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318452,"statfs":{"total":96636764160,"available":96582094848,"internally_reserved":0,"allocated":4251648,"data_stored":2342669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510718,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57524,"kb_used_data":4136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314316,"statfs":{"total":96636764160,"available":96577859584,"internally_reserved":0,"allocated":4235264,"data_stored":2334808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543422,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57560,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314280,"statfs":{"total":96636764160,"available":96577822720,"internally_reserved":0,"allocated":4272128,"data_stored":2344829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510719,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53384,"kb_used_data":4152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318456,"statfs":{"total":96636764160,"available":96582098944,"internally_reserved":0,"allocated":4251648,"data_stored":2342613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53964,"kb_used_data":4736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317876,"statfs":{"total":96636764160,"available":96581505024,"internally_reserved":0,"allocated":4849664,"data_stored":2933039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":45,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":46,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:42:54.280 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:42:54.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:42:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 -- 172.21.15.44:0/2375357071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 msgr2=0x7efe280cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 --2- 172.21.15.44:0/2375357071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280cfdf0 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7efe2808c800 tx=0x7efe2002a820 comp rx=0 tx=0).stop 2024-07-17T08:42:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 -- 172.21.15.44:0/2375357071 shutdown_connections 2024-07-17T08:42:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 --2- 172.21.15.44:0/2375357071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280cfdf0 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 --2- 172.21.15.44:0/2375357071 >> v2:172.21.15.175:3300/0 conn(0x7efe2815d130 0x7efe2815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 -- 172.21.15.44:0/2375357071 >> 172.21.15.44:0/2375357071 conn(0x7efe280b4ff0 msgr2=0x7efe280b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 -- 172.21.15.44:0/2375357071 shutdown_connections 2024-07-17T08:42:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.545+0000 7efe2e800640 1 -- 172.21.15.44:0/2375357071 wait complete. 2024-07-17T08:42:54.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.547+0000 7efe2e800640 1 Processor -- start 2024-07-17T08:42:54.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.547+0000 7efe2e800640 1 -- start start 2024-07-17T08:42:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe2815d130 0x7efe2814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe28161be0 0x7efe2814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe280d1a60 con 0x7efe280ce690 2024-07-17T08:42:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe280d18e0 con 0x7efe2815d130 2024-07-17T08:42:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe280d1be0 con 0x7efe28161be0 2024-07-17T08:42:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40336/0 (socket says 172.21.15.44:40336) 2024-07-17T08:42:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.548+0000 7efe2d400640 1 -- 172.21.15.44:0/1491190001 learned_addr learned my addr 172.21.15.44:0/1491190001 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2ca00640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe2815d130 0x7efe2814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 -- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe28161be0 msgr2=0x7efe2814c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe28161be0 0x7efe2814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 -- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe2815d130 msgr2=0x7efe2814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe2815d130 0x7efe2814f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 -- 172.21.15.44:0/1491190001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe280cc120 con 0x7efe280ce690 2024-07-17T08:42:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe2d400640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e9f0 tx=0x7efe1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.549+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1800cd80 con 0x7efe280ce690 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.550+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe18010430 con 0x7efe280ce690 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.550+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18010840 con 0x7efe280ce690 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.550+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe280cc400 con 0x7efe280ce690 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.550+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe280cc9c0 con 0x7efe280ce690 2024-07-17T08:42:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.551+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe280d2170 con 0x7efe280ce690 2024-07-17T08:42:54.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.552+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7efe180109e0 con 0x7efe280ce690 2024-07-17T08:42:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.553+0000 7efe25600640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 0x7efe080791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.553+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7efe18099620 con 0x7efe280ce690 2024-07-17T08:42:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.554+0000 7efe2ca00640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 0x7efe080791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.555+0000 7efe2ca00640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 0x7efe080791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe14007fd0 tx=0x7efe14009040 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.556+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efe18062610 con 0x7efe280ce690 2024-07-17T08:42:54.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.846+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7efe280d0e10 con 0x7efe280ce690 2024-07-17T08:42:54.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.852+0000 7efe25600640 1 -- 172.21.15.44:0/1491190001 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v81) ==== 190+0+147 (secure 0 0 0) 0x7efe18061f10 con 0x7efe280ce690 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:42:54.854 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 msgr2=0x7efe080791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 0x7efe080791c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe14007fd0 tx=0x7efe14009040 comp rx=0 tx=0).stop 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 msgr2=0x7efe280a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e9f0 tx=0x7efe1800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 shutdown_connections 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7efe08076d00 0x7efe080791c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe28161be0 0x7efe2814c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe2815d130 0x7efe2814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 --2- 172.21.15.44:0/1491190001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe280ce690 0x7efe280a9f70 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:54.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 >> 172.21.15.44:0/1491190001 conn(0x7efe280b4ff0 msgr2=0x7efe280c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:54.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 shutdown_connections 2024-07-17T08:42:54.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:54.855+0000 7efe2e800640 1 -- 172.21.15.44:0/1491190001 wait complete. 2024-07-17T08:42:54.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:42:55.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 -- 172.21.15.44:0/1302204340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489415b2e0 msgr2=0x7f48940ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:55.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 --2- 172.21.15.44:0/1302204340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489415b2e0 0x7f48940ac650 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f48940715d0 tx=0x7f488402a560 comp rx=0 tx=0).stop 2024-07-17T08:42:55.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 -- 172.21.15.44:0/1302204340 shutdown_connections 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 --2- 172.21.15.44:0/1302204340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f489406f560 0x7f4894153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 --2- 172.21.15.44:0/1302204340 >> v2:172.21.15.175:3300/0 conn(0x7f48940acb90 0x7f489406f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 --2- 172.21.15.44:0/1302204340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489415b2e0 0x7f48940ac650 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 -- 172.21.15.44:0/1302204340 >> 172.21.15.44:0/1302204340 conn(0x7f48940b0fb0 msgr2=0x7f48940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 -- 172.21.15.44:0/1302204340 shutdown_connections 2024-07-17T08:42:55.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.081+0000 7f4899000640 1 -- 172.21.15.44:0/1302204340 wait complete. 2024-07-17T08:42:55.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.083+0000 7f4899000640 1 Processor -- start 2024-07-17T08:42:55.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.083+0000 7f4899000640 1 -- start start 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 0x7f489414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f489415b2e0 0x7f489414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4894154710 con 0x7f489406f560 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4894154590 con 0x7f489415b2e0 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4899000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4894154890 con 0x7f48940acb90 2024-07-17T08:42:55.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4891e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:55.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4891e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40344/0 (socket says 172.21.15.44:40344) 2024-07-17T08:42:55.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.084+0000 7f4891e00640 1 -- 172.21.15.44:0/764157720 learned_addr learned my addr 172.21.15.44:0/764157720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4892800640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f489415b2e0 0x7f489414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f488be00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 0x7f489414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 -- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 msgr2=0x7f489414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 0x7f489414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 -- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f489415b2e0 msgr2=0x7f489414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f489415b2e0 0x7f489414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 -- 172.21.15.44:0/764157720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48940b0580 con 0x7f489406f560 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f488be00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 0x7f489414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.085+0000 7f4891e00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f4884034040 tx=0x7f48840039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.086+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488403e030 con 0x7f489406f560 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.086+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48840047a0 con 0x7f489406f560 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.086+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884035040 con 0x7f489406f560 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.086+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48940affc0 con 0x7f489406f560 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.086+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48940734c0 con 0x7f489406f560 2024-07-17T08:42:55.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.087+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4894153e40 con 0x7f489406f560 2024-07-17T08:42:55.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.088+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f4884041030 con 0x7f489406f560 2024-07-17T08:42:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.089+0000 7f488a000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 0x7f48700791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.089+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f48840b8ff0 con 0x7f489406f560 2024-07-17T08:42:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.090+0000 7f488be00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 0x7f48700791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:55.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.091+0000 7f488be00640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 0x7f48700791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48940b0c80 tx=0x7f487c005d20 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.094+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4884081f00 con 0x7f489406f560 2024-07-17T08:42:55.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.344+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48940b0d90 con 0x7f489406f560 2024-07-17T08:42:55.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.346+0000 7f488a000640 1 -- 172.21.15.44:0/764157720 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7f488408a140 con 0x7f489406f560 2024-07-17T08:42:55.348 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:42:55.348 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:42:55.348 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:42:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 msgr2=0x7f48700791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 0x7f48700791c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48940b0c80 tx=0x7f487c005d20 comp rx=0 tx=0).stop 2024-07-17T08:42:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 msgr2=0x7f48940b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:55.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f4884034040 tx=0x7f48840039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 shutdown_connections 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4870076d00 0x7f48700791c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f489415b2e0 0x7f489414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48940acb90 0x7f489414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 --2- 172.21.15.44:0/764157720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f489406f560 0x7f48940b3c00 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 >> 172.21.15.44:0/764157720 conn(0x7f48940b0fb0 msgr2=0x7f4894158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.349+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 shutdown_connections 2024-07-17T08:42:55.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:55.350+0000 7f4899000640 1 -- 172.21.15.44:0/764157720 wait complete. 2024-07-17T08:42:55.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 191, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:42:48.668598+0000', 'modified': '2024-07-17T08:42:51.771429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13491': {'gid': 13491, 'name': 'c', 'rank': 0, 'incarnation': 187, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/4048233303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 4048233303}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 4048233303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [46], 'metadata_pool': 45, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 13491, 'qdb_cluster': [13491]} 2024-07-17T08:42:55.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:42:55.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-17T08:42:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 -- 172.21.15.44:0/554721606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c1503c0 msgr2=0x7f7e0c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 --2- 172.21.15.44:0/554721606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c15a450 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0cffb0 tx=0x7f7e0002a5f0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 -- 172.21.15.44:0/554721606 shutdown_connections 2024-07-17T08:42:56.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 --2- 172.21.15.44:0/554721606 >> v2:172.21.15.175:3300/0 conn(0x7f7e0c0ae340 0x7f7e0c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 --2- 172.21.15.44:0/554721606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c15a450 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 -- 172.21.15.44:0/554721606 >> 172.21.15.44:0/554721606 conn(0x7f7e0c0cf0d0 msgr2=0x7f7e0c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 -- 172.21.15.44:0/554721606 shutdown_connections 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.002+0000 7f7e13200640 1 -- 172.21.15.44:0/554721606 wait complete. 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.004+0000 7f7e13200640 1 Processor -- start 2024-07-17T08:42:56.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.004+0000 7f7e13200640 1 -- start start 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0b33f0 0x7f7e0c149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c06dcb0 con 0x7f7e0c0ae340 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c06db30 con 0x7f7e0c1503c0 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e13200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c06f4d0 con 0x7f7e0c0b33f0 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40356/0 (socket says 172.21.15.44:40356) 2024-07-17T08:42:56.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e11e00640 1 -- 172.21.15.44:0/3212842663 learned_addr learned my addr 172.21.15.44:0/3212842663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e11e00640 1 -- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0b33f0 msgr2=0x7f7e0c149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.005+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0b33f0 0x7f7e0c149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e11e00640 1 -- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c1503c0 msgr2=0x7f7e0c1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e12800640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c1f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e11e00640 1 -- 172.21.15.44:0/3212842663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c1f4510 con 0x7f7e0c0ae340 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f7e00034040 tx=0x7f7e000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e06000640 1 -- 172.21.15.44:0/3212842663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0003e030 con 0x7f7e0c0ae340 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e06000640 1 -- 172.21.15.44:0/3212842663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e000047a0 con 0x7f7e0c0ae340 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e06000640 1 -- 172.21.15.44:0/3212842663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00035040 con 0x7f7e0c0ae340 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c1f47f0 con 0x7f7e0c0ae340 2024-07-17T08:42:56.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.006+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c151380 con 0x7f7e0c0ae340 2024-07-17T08:42:56.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.009+0000 7f7e06000640 1 -- 172.21.15.44:0/3212842663 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f7e00041030 con 0x7f7e0c0ae340 2024-07-17T08:42:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.010+0000 7f7e06000640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 0x7f7dec079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.010+0000 7f7e06000640 1 -- 172.21.15.44:0/3212842663 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f7e000b9130 con 0x7f7e0c0ae340 2024-07-17T08:42:56.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.010+0000 7f7e17584740 1 Processor -- start 2024-07-17T08:42:56.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.010+0000 7f7e17584740 1 -- start start 2024-07-17T08:42:56.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.011+0000 7f7e11400640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 0x7f7dec079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021527fc0 0x5630215283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5630214ff280 0x563021524250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56302146fd60 con 0x563021524790 2024-07-17T08:42:56.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56302145f260 con 0x563021527fc0 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5630214632a0 con 0x5630214ff280 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e11400640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 0x7f7dec079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c06cbd0 tx=0x7f7df8005c70 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40372/0 (socket says 172.21.15.44:40372) 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 -- 172.21.15.44:0/762475390 learned_addr learned my addr 172.21.15.44:0/762475390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e11e00640 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021527fc0 0x5630215283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e11400640 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5630214ff280 0x563021524250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 -- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5630214ff280 msgr2=0x563021524250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5630214ff280 0x563021524250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 -- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021527fc0 msgr2=0x5630215283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021527fc0 0x5630215283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.012+0000 7f7e12800640 1 -- 172.21.15.44:0/762475390 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630211c3850 con 0x563021524790 2024-07-17T08:42:56.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.013+0000 7f7e12800640 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f7e08001d40 tx=0x7f7e0800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437b431094f42016 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.013+0000 7f7dfe600640 1 -- 172.21.15.44:0/762475390 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08018070 con 0x563021524790 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.013+0000 7f7dfe600640 1 -- 172.21.15.44:0/762475390 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e0800d920 con 0x563021524790 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.013+0000 7f7dfe600640 1 -- 172.21.15.44:0/762475390 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800dd10 con 0x563021524790 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 -- 172.21.15.44:0/762475390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 msgr2=0x563021524c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f7e08001d40 tx=0x7f7e0800d150 comp rx=0 tx=0).stop 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 -- 172.21.15.44:0/762475390 shutdown_connections 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x563021524c40 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5630214ff280 0x563021524250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 --2- 172.21.15.44:0/762475390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021527fc0 0x5630215283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 -- 172.21.15.44:0/762475390 >> 172.21.15.44:0/762475390 conn(0x5630214fa460 msgr2=0x5630214fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 -- 172.21.15.44:0/762475390 shutdown_connections 2024-07-17T08:42:56.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.014+0000 7f7e17584740 1 -- 172.21.15.44:0/762475390 wait complete. 2024-07-17T08:42:56.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.015+0000 7f7e17584740 1 -- start start 2024-07-17T08:42:56.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.015+0000 7f7e17584740 1 Processor -- start 2024-07-17T08:42:56.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021524790 0x563021538b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x56302151eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5630211d2620 con 0x56302151f070 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563021406f90 con 0x563021524790 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5630214f8b60 con 0x563021527fc0 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e17584740 10 client.0 _ll_register_callbacks cb 0x56302142cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e12800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40376/0 (socket says 172.21.15.44:40376) 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.016+0000 7f7e12800640 1 -- 172.21.15.44:0/3086999938 learned_addr learned my addr 172.21.15.44:0/3086999938 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021524790 0x563021538b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 -- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 msgr2=0x56302151eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x56302151eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 -- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021524790 msgr2=0x563021538b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021524790 0x563021538b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 -- 172.21.15.44:0/3086999938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302123a6b0 con 0x56302151f070 2024-07-17T08:42:56.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e12800640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009010 tx=0x7f7e0800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0443b2fbe1e3d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7df2a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08021020 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f7e0800ddc0 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.017+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563021436900 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=191) ==== 24+0+0 (secure 0 0 0) 0x7f7e08018040 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 10 client.13614 fetch_fsmap learned FSMap version 191 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=191,monmap=2+}) -- 0x56302151e0e0 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563021436900 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 191) ==== 3502+0+0 (secure 0 0 0) 0x7f7e0802e020 con 0x56302151f070 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 10 client.13614 fetch_fsmap finished waiting for FSMap version 191 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 10 client.13614 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:42:56.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 20 client.13614 populate_metadata read hostname 'smithi044' 2024-07-17T08:42:56.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x563021521130 0x563021556660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 4 client.13614 mds_command: new command op to 13491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:42:56.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.018+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 --> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563021471cb0 con 0x563021521130 2024-07-17T08:42:56.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.019+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x563021521130 0x563021556660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.019+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x563021521130 0x563021556660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6cf445047fa05b server_cookie=72f6d9508c90c0f0 in_seq=0 out_seq=0 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.019+0000 7f7df2a00640 10 client.13614 ms_handle_connect on v2:172.21.15.44:6834/4048233303 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7df2a00640 1 -- 172.21.15.44:0/3086999938 <== mds.0 v2:172.21.15.44:6834/4048233303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563021471cb0 con 0x563021521130 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7df2a00640 10 client.13614 handle_command_reply: tid=0 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 client.13614 shutdown 2024-07-17T08:42:56.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 msgr2=0x56302153c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009010 tx=0x7f7e0800dbd0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 shutdown_connections 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x563021521130 0x563021556660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56302151f070 0x56302153c130 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x56302151eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 --2- 172.21.15.44:0/3086999938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563021524790 0x563021538b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7df2a00640 0 client.13614 ms_handle_reset on v2:172.21.15.44:6834/4048233303 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 >> 172.21.15.44:0/3086999938 conn(0x5630214ff280 msgr2=0x5630214f7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.020+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 shutdown_connections 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.021+0000 7f7e17584740 1 -- 172.21.15.44:0/3086999938 wait complete. 2024-07-17T08:42:56.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.021+0000 7f7e17584740 20 client.13614 trim_cache size 0 max 16384 2024-07-17T08:42:56.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.041+0000 7f7e17584740 1 Processor -- start 2024-07-17T08:42:56.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.041+0000 7f7e17584740 1 -- start start 2024-07-17T08:42:56.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x563021510f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c201260 con 0x563021524790 2024-07-17T08:42:56.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c200280 con 0x56302151f070 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c1ff6c0 con 0x563021527fc0 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40380/0 (socket says 172.21.15.44:40380) 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.042+0000 7f7e11400640 1 -- 172.21.15.44:0/3128623946 learned_addr learned my addr 172.21.15.44:0/3128623946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11e00640 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x563021510f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 -- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 msgr2=0x5630214f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 -- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 msgr2=0x563021510f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x563021510f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 -- 172.21.15.44:0/3128623946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563021527c00 con 0x563021524790 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7e11400640 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f7df800ce10 tx=0x7f7df8005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840816064da1183 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7dfdc00640 1 -- 172.21.15.44:0/3128623946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8021070 con 0x563021524790 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7dfdc00640 1 -- 172.21.15.44:0/3128623946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7df8015040 con 0x563021524790 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.043+0000 7f7dfdc00640 1 -- 172.21.15.44:0/3128623946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8005590 con 0x563021524790 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 -- 172.21.15.44:0/3128623946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 msgr2=0x56302155c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f7df800ce10 tx=0x7f7df8005b10 comp rx=0 tx=0).stop 2024-07-17T08:42:56.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 -- 172.21.15.44:0/3128623946 shutdown_connections 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x56302155c690 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 --2- 172.21.15.44:0/3128623946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x563021510f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 -- 172.21.15.44:0/3128623946 >> 172.21.15.44:0/3128623946 conn(0x5630214ff280 msgr2=0x563021546500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 -- 172.21.15.44:0/3128623946 shutdown_connections 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.044+0000 7f7e17584740 1 -- 172.21.15.44:0/3128623946 wait complete. 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.045+0000 7f7e17584740 1 -- start start 2024-07-17T08:42:56.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.045+0000 7f7e17584740 1 Processor -- start 2024-07-17T08:42:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x5630215515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563021521c00 con 0x563021524790 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1ffc30 con 0x56302151f070 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563021521570 con 0x563021527fc0 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e17584740 10 client.0 _ll_register_callbacks cb 0x7f7e08016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e11400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40394/0 (socket says 172.21.15.44:40394) 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.046+0000 7f7e11400640 1 -- 172.21.15.44:0/2600160974 learned_addr learned my addr 172.21.15.44:0/2600160974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11e00640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x5630215515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 -- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 msgr2=0x5630214fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 -- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 msgr2=0x5630215515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x5630215515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 -- 172.21.15.44:0/2600160974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302151d5d0 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e11400640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f7df800cde0 tx=0x7f7df800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a63b337c59c674 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7df2a00640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8021040 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f7df8004220 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.047+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df8015040 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=191) ==== 24+0+0 (secure 0 0 0) 0x7f7df8036050 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 10 client.13620 fetch_fsmap learned FSMap version 191 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=191,monmap=2+}) -- 0x5630214fcfd0 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7df8015040 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 191) ==== 3502+0+0 (secure 0 0 0) 0x7f7df8039020 con 0x563021524790 2024-07-17T08:42:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 10 client.13620 fetch_fsmap finished waiting for FSMap version 191 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 10 client.13620 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 20 client.13620 populate_metadata read hostname 'smithi044' 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x5630215c0eb0 0x5630215c1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 4 client.13620 mds_command: new command op to 13491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 --> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] -- command(tid 0: {"prefix": "status"}) -- 0x56302146d750 con 0x5630215c0eb0 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.048+0000 7f7e11e00640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x5630215c0eb0 0x5630215c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e11e00640 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x5630215c0eb0 0x5630215c1290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa020948af598408 server_cookie=5e751680385daa22 in_seq=0 out_seq=0 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7df2a00640 10 client.13620 ms_handle_connect on v2:172.21.15.44:6834/4048233303 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7df2a00640 1 -- 172.21.15.44:0/2600160974 <== mds.0 v2:172.21.15.44:6834/4048233303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x56302146d750 con 0x5630215c0eb0 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7df2a00640 10 client.13620 handle_command_reply: tid=0 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 client.13620 shutdown 2024-07-17T08:42:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 msgr2=0x5630214fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f7df800cde0 tx=0x7f7df800a650 comp rx=0 tx=0).stop 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 shutdown_connections 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:6834/4048233303,v1:172.21.15.44:6835/4048233303] conn(0x5630215c0eb0 0x5630215c1290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563021527fc0 0x5630214fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563021524790 0x5630214fb8e0 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 --2- 172.21.15.44:0/2600160974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56302151f070 0x5630215515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7df2a00640 0 client.13620 ms_handle_reset on v2:172.21.15.44:6834/4048233303 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.049+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 >> 172.21.15.44:0/2600160974 conn(0x5630214ff280 msgr2=0x56302155c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 shutdown_connections 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e17584740 1 -- 172.21.15.44:0/2600160974 wait complete. 2024-07-17T08:42:56.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e17584740 20 client.13620 trim_cache size 0 max 16384 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 13491, 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8514219719999998, 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 191, 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 335, 2024-07-17T08:42:56.056 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 335, 2024-07-17T08:42:56.057 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 12.831173685 2024-07-17T08:42:56.057 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:42:56.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 msgr2=0x7f7dec079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 0x7f7dec079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c06cbd0 tx=0x7f7df8005c70 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 msgr2=0x7f7e0c148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f7e00034040 tx=0x7f7e000039e0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 shutdown_connections 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f7dec076cd0 0x7f7dec079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e0c1503c0 0x7f7e0c1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e0c0b33f0 0x7f7e0c149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 --2- 172.21.15.44:0/3212842663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e0c0ae340 0x7f7e0c148260 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.050+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 >> 172.21.15.44:0/3212842663 conn(0x7f7e0c0cf0d0 msgr2=0x7f7e0c14b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.051+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 shutdown_connections 2024-07-17T08:42:56.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.051+0000 7f7e13200640 1 -- 172.21.15.44:0/3212842663 wait complete. 2024-07-17T08:42:56.071 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.267+0000 7f3bd4e00640 1 -- 172.21.15.44:0/3609363926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 msgr2=0x7f3bd006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.267+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/3609363926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd006e060 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005fd0 tx=0x7f3bc002b8e0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.267+0000 7f3bd4e00640 1 -- 172.21.15.44:0/3609363926 shutdown_connections 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.267+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/3609363926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd00cf380 0x7f3bd00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.267+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/3609363926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd006e060 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.268+0000 7f3bd4e00640 1 -- 172.21.15.44:0/3609363926 >> 172.21.15.44:0/3609363926 conn(0x7f3bd00b0e80 msgr2=0x7f3bd00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.268+0000 7f3bd4e00640 1 -- 172.21.15.44:0/3609363926 shutdown_connections 2024-07-17T08:42:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.268+0000 7f3bd4e00640 1 -- 172.21.15.44:0/3609363926 wait complete. 2024-07-17T08:42:56.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.270+0000 7f3bd4e00640 1 Processor -- start 2024-07-17T08:42:56.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.270+0000 7f3bd4e00640 1 -- start start 2024-07-17T08:42:56.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd00cf380 0x7f3bd00ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 0x7f3bd01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00b2120 con 0x7f3bd006f4b0 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00b1fa0 con 0x7f3bd00cf380 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bd4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd00b22a0 con 0x7f3bd0159800 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40410/0 (socket says 172.21.15.44:40410) 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 -- 172.21.15.44:0/2008160952 learned_addr learned my addr 172.21.15.44:0/2008160952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcea00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd00cf380 0x7f3bd00ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcfe00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 0x7f3bd01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 -- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 msgr2=0x7f3bd01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 0x7f3bd01531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 -- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd00cf380 msgr2=0x7f3bd00ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.272+0000 7f3bcf400640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd00cf380 0x7f3bd00ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bcf400640 1 -- 172.21.15.44:0/2008160952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd00acca0 con 0x7f3bd006f4b0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bcfe00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 0x7f3bd01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bcf400640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc0031ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0040070 con 0x7f3bd006f4b0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bc0004340 con 0x7f3bd006f4b0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd00acf80 con 0x7f3bd006f4b0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0004730 con 0x7f3bd006f4b0 2024-07-17T08:42:56.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.273+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd0072de0 con 0x7f3bd006f4b0 2024-07-17T08:42:56.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.275+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd006e060 con 0x7f3bd006f4b0 2024-07-17T08:42:56.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.276+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f3bc0032470 con 0x7f3bd006f4b0 2024-07-17T08:42:56.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.277+0000 7f3bc7400640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 0x7f3bb0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:56.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.277+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f3bc00ba5e0 con 0x7f3bd006f4b0 2024-07-17T08:42:56.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.277+0000 7f3bcea00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 0x7f3bb0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:56.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.279+0000 7f3bcea00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 0x7f3bb0079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc005e10 tx=0x7f3bbc005da0 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:56.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.282+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3bc0083490 con 0x7f3bd006f4b0 2024-07-17T08:42:56.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.542+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3bd00ad310 con 0x7f3bd006f4b0 2024-07-17T08:42:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.543+0000 7f3bc7400640 1 -- 172.21.15.44:0/2008160952 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v191) ==== 53+0+73 (secure 0 0 0) 0x7f3bc0082d90 con 0x7f3bd006f4b0 2024-07-17T08:42:56.545 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 msgr2=0x7f3bb0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 0x7f3bb0079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc005e10 tx=0x7f3bbc005da0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 msgr2=0x7f3bd0152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc0031ed0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 shutdown_connections 2024-07-17T08:42:56.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f3bb0076cd0 0x7f3bb0079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3bd0159800 0x7f3bd01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3bd00cf380 0x7f3bd00ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 --2- 172.21.15.44:0/2008160952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bd006f4b0 0x7f3bd0152920 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 >> 172.21.15.44:0/2008160952 conn(0x7f3bd00b0e80 msgr2=0x7f3bd00af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 shutdown_connections 2024-07-17T08:42:56.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56.546+0000 7f3bd4e00640 1 -- 172.21.15.44:0/2008160952 wait complete. 2024-07-17T08:42:56.567 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:42:56.568 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:56.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:56.569 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 36081sec preferred_lft 36081sec 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.622 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.623 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:42:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:42:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:42:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:42:56.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:56.625 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:42:56.702 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:42:56.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:56.706 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:42:56.759 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:42:56.759 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:42:56.759 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:42:56.759 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:42:56.759 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:42:56.761 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:42:56.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:42:56.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:42:56.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:56.984 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:42:57.035 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:42:57.035 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:42:57.036 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:57.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:57.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:42:57.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:42:57.116 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:57.169 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:42:57.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:42:57.169 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:57.220 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:42:57.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:57.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:57.349 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:57.350 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:57.350 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:57.371 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:57.371 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:57.371 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:57.371 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:42:57.371 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:42:57.399 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:42:57.400 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:42:57.400 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:42:57.401 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:42:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:42:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:42:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:42:57.401 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:42:57.405 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:42:57.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:42:57.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:42:57.476 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:42:57.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:42:57.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:42:57.621 INFO:teuthology.orchestra.run.smithi044.stdout:client10792 2024-07-17T08:42:57.622 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:42:57.622 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:42:57.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.888+0000 7fbd02200640 1 -- 172.21.15.44:0/479553476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce00051c0 msgr2=0x7fbce000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.888+0000 7fbd02200640 1 --2- 172.21.15.44:0/479553476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce00051c0 0x7fbce000b9c0 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0b1a80 tx=0x7fbce0005830 comp rx=0 tx=0).stop 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.888+0000 7fbcfbe00640 1 -- 172.21.15.44:0/479553476 reap_dead start 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.889+0000 7fbd02200640 1 -- 172.21.15.44:0/479553476 shutdown_connections 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.889+0000 7fbd02200640 1 -- 172.21.15.44:0/479553476 >> 172.21.15.44:0/479553476 conn(0x7fbcfc0b0eb0 msgr2=0x7fbcfc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.889+0000 7fbd02200640 1 -- 172.21.15.44:0/479553476 shutdown_connections 2024-07-17T08:42:57.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.889+0000 7fbd02200640 1 -- 172.21.15.44:0/479553476 wait complete. 2024-07-17T08:42:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.890+0000 7fbd02200640 1 Processor -- start 2024-07-17T08:42:57.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.891+0000 7fbd02200640 1 -- start start 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.891+0000 7fbd02200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbce00051c0 0x7fbcfc0c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.891+0000 7fbd02200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbd02200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc15c210 0x7fbcfc15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbd02200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0b21b0 con 0x7fbcfc1f8010 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbd02200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc06f1d0 con 0x7fbcfc15c210 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbd02200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc0b2330 con 0x7fbce00051c0 2024-07-17T08:42:57.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40426/0 (socket says 172.21.15.44:40426) 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 -- 172.21.15.44:0/2644663677 learned_addr learned my addr 172.21.15.44:0/2644663677 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfb400640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbce00051c0 0x7fbcfc0c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfbe00640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc15c210 0x7fbcfc15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 -- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbce00051c0 msgr2=0x7fbcfc0c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbce00051c0 0x7fbcfc0c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 -- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc15c210 msgr2=0x7fbcfc15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc15c210 0x7fbcfc15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.892+0000 7fbcfaa00640 1 -- 172.21.15.44:0/2644663677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0c8a70 con 0x7fbcfc1f8010 2024-07-17T08:42:57.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbcfaa00640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fbce800ea80 tx=0x7fbce800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce800ce40 con 0x7fbcfc1f8010 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce8004b20 con 0x7fbcfc1f8010 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc151d90 con 0x7fbcfc1f8010 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8005020 con 0x7fbcfc1f8010 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.893+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc1508b0 con 0x7fbcfc1f8010 2024-07-17T08:42:57.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.894+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc0c9f00 con 0x7fbcfc1f8010 2024-07-17T08:42:57.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.895+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7fbce80051c0 con 0x7fbcfc1f8010 2024-07-17T08:42:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.897+0000 7fbcf3400640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 0x7fbcdc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.897+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7fbce80998f0 con 0x7fbcfc1f8010 2024-07-17T08:42:57.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.898+0000 7fbcfb400640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 0x7fbcdc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:57.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.899+0000 7fbcfb400640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 0x7fbcdc079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcec004a80 tx=0x7fbcec001d30 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:57.901+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbcfc0c9f00 con 0x7fbcfc1f8010 2024-07-17T08:42:58.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.136+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 --> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbcfc15cbb0 con 0x7fbcdc076cd0 2024-07-17T08:42:58.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.141+0000 7fbcf3400640 1 -- 172.21.15.44:0/2644663677 <== mgr.9539 v2:172.21.15.175:6834/1233420839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbcfc15cbb0 con 0x7fbcdc076cd0 2024-07-17T08:42:58.142 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:42:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:42:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:42:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:42:58.143 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:42:58.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 msgr2=0x7fbcdc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:58.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 0x7fbcdc079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcec004a80 tx=0x7fbcec001d30 comp rx=0 tx=0).stop 2024-07-17T08:42:58.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 msgr2=0x7fbcfc1f7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:58.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fbce800ea80 tx=0x7fbce800ef40 comp rx=0 tx=0).stop 2024-07-17T08:42:58.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 shutdown_connections 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fbcdc076cd0 0x7fbcdc079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcfc15c210 0x7fbcfc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcfc1f8010 0x7fbcfc1f7670 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 --2- 172.21.15.44:0/2644663677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbce00051c0 0x7fbcfc0c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.143+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 >> 172.21.15.44:0/2644663677 conn(0x7fbcfc0b0eb0 msgr2=0x7fbcfc0af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.144+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 shutdown_connections 2024-07-17T08:42:58.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.144+0000 7fbd02200640 1 -- 172.21.15.44:0/2644663677 wait complete. 2024-07-17T08:42:58.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:42:58.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 -- 172.21.15.44:0/4160780824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48680acab0 msgr2=0x7f486815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:58.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 --2- 172.21.15.44:0/4160780824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48680acab0 0x7f486815a5a0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7f4850005fd0 tx=0x7f485002b860 comp rx=0 tx=0).stop 2024-07-17T08:42:58.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 -- 172.21.15.44:0/4160780824 shutdown_connections 2024-07-17T08:42:58.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 --2- 172.21.15.44:0/4160780824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486806eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 --2- 172.21.15.44:0/4160780824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48680acab0 0x7f486815a5a0 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 -- 172.21.15.44:0/4160780824 >> 172.21.15.44:0/4160780824 conn(0x7f48680b0eb0 msgr2=0x7f48680b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:58.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 -- 172.21.15.44:0/4160780824 shutdown_connections 2024-07-17T08:42:58.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.381+0000 7f486ce00640 1 -- 172.21.15.44:0/4160780824 wait complete. 2024-07-17T08:42:58.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.383+0000 7f486ce00640 1 Processor -- start 2024-07-17T08:42:58.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.383+0000 7f486ce00640 1 -- start start 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680acab0 0x7f486814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f48680aea80 con 0x7f486806f6d0 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48680ae900 con 0x7f48680acab0 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.384+0000 7f486ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48680aec00 con 0x7f48680b3280 2024-07-17T08:42:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40450/0 (socket says 172.21.15.44:40450) 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 -- 172.21.15.44:0/1744183192 learned_addr learned my addr 172.21.15.44:0/1744183192 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4866800640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 -- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 msgr2=0x7f486814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f485fe00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680acab0 0x7f486814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 -- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680acab0 msgr2=0x7f486814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680acab0 0x7f486814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4865e00640 1 -- 172.21.15.44:0/1744183192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48681f83e0 con 0x7f486806f6d0 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.385+0000 7f4866800640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:42:58.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f4865e00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f485800c430 tx=0x7f485800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858011790 con 0x7f486806f6d0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4858017070 con 0x7f486806f6d0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48681f42f0 con 0x7f486806f6d0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858011d00 con 0x7f486806f6d0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.386+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48681f48b0 con 0x7f486806f6d0 2024-07-17T08:42:58.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.387+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486815a5a0 con 0x7f486806f6d0 2024-07-17T08:42:58.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.389+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 198) ==== 98931+0+0 (secure 0 0 0) 0x7f485801c460 con 0x7f486806f6d0 2024-07-17T08:42:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.390+0000 7f485e000640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 0x7f4844079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:58.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.390+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f4858098750 con 0x7f486806f6d0 2024-07-17T08:42:58.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.390+0000 7f485fe00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 0x7f4844079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:58.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.391+0000 7f485fe00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 0x7f4844079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4850035000 tx=0x7f4850004730 comp rx=0 tx=0).ready entity=mgr.9539 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:58.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.394+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f48580617b0 con 0x7f486806f6d0 2024-07-17T08:42:58.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:58.657+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f48680b4200 con 0x7f486806f6d0 2024-07-17T08:42:59.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.312+0000 7f485e000640 1 -- 172.21.15.44:0/1744183192 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v199) ==== 92+0+0 (secure 0 0 0) 0x7f48580610b0 con 0x7f486806f6d0 2024-07-17T08:42:59.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.317+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 msgr2=0x7f4844079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.317+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 0x7f4844079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4850035000 tx=0x7f4850004730 comp rx=0 tx=0).stop 2024-07-17T08:42:59.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.317+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 msgr2=0x7f4868152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.317+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f485800c430 tx=0x7f485800c8f0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 shutdown_connections 2024-07-17T08:42:59.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f4844076cd0 0x7f4844079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48680b3280 0x7f486814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48680acab0 0x7f486814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 --2- 172.21.15.44:0/1744183192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486806f6d0 0x7f4868152b20 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.319+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 >> 172.21.15.44:0/1744183192 conn(0x7f48680b0eb0 msgr2=0x7f486815b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:59.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.320+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 shutdown_connections 2024-07-17T08:42:59.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.321+0000 7f486ce00640 1 -- 172.21.15.44:0/1744183192 wait complete. 2024-07-17T08:42:59.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:42:59.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.550+0000 7fed8f200640 1 -- 172.21.15.44:0/1608213016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 msgr2=0x7fed88161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.550+0000 7fed8f200640 1 --2- 172.21.15.44:0/1608213016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed88161b90 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7fed88075790 tx=0x7fed8002aaa0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 -- 172.21.15.44:0/1608213016 shutdown_connections 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 --2- 172.21.15.44:0/1608213016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880b6790 0x7fed88161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 --2- 172.21.15.44:0/1608213016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed88161b90 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 --2- 172.21.15.44:0/1608213016 >> v2:172.21.15.175:3300/0 conn(0x7fed880ce1d0 0x7fed880ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 -- 172.21.15.44:0/1608213016 >> 172.21.15.44:0/1608213016 conn(0x7fed880d3700 msgr2=0x7fed880d3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 -- 172.21.15.44:0/1608213016 shutdown_connections 2024-07-17T08:42:59.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.551+0000 7fed8f200640 1 -- 172.21.15.44:0/1608213016 wait complete. 2024-07-17T08:42:59.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.552+0000 7fed8f200640 1 Processor -- start 2024-07-17T08:42:59.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.553+0000 7fed8f200640 1 -- start start 2024-07-17T08:42:59.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.553+0000 7fed8f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:59.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.553+0000 7fed8f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed880b6790 0x7fed8815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:59.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.553+0000 7fed8f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880ce1d0 0x7fed8815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed8f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed88162250 con 0x7fed880b29e0 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed8f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed881620d0 con 0x7fed880b6790 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed8f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed881623d0 con 0x7fed880ce1d0 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54556/0 (socket says 172.21.15.44:54556) 2024-07-17T08:42:59.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 -- 172.21.15.44:0/622892536 learned_addr learned my addr 172.21.15.44:0/622892536 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed8d200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880ce1d0 0x7fed8815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87400640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed880b6790 0x7fed8815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880ce1d0 msgr2=0x7fed8815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880ce1d0 0x7fed8815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed880b6790 msgr2=0x7fed8815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed880b6790 0x7fed8815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.554+0000 7fed87e00640 1 -- 172.21.15.44:0/622892536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8814fa70 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed87e00640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b410 tx=0x7fed7800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78015020 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed78011940 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78011d90 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8815f7e0 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.555+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8815fda0 con 0x7fed880b29e0 2024-07-17T08:42:59.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.557+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed880ce5b0 con 0x7fed880b29e0 2024-07-17T08:42:59.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.558+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 199) ==== 98948+0+0 (secure 0 0 0) 0x7fed78018020 con 0x7fed880b29e0 2024-07-17T08:42:59.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.559+0000 7fed7fe00640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 0x7fed68079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:42:59.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.559+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7fed78098860 con 0x7fed880b29e0 2024-07-17T08:42:59.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.559+0000 7fed87400640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 msgr2=0x7fed68079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:42:59.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.559+0000 7fed87400640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 0x7fed68079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:42:59.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.564+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fed7801f020 con 0x7fed880b29e0 2024-07-17T08:42:59.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.761+0000 7fed87400640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 msgr2=0x7fed68079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:42:59.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.761+0000 7fed87400640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 0x7fed68079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:42:59.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.770+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fed881613c0 con 0x7fed880b29e0 2024-07-17T08:42:59.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.776+0000 7fed7fe00640 1 -- 172.21.15.44:0/622892536 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v101)=0 v101) ==== 138+0+0 (secure 0 0 0) 0x7fed78061820 con 0x7fed880b29e0 2024-07-17T08:42:59.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 msgr2=0x7fed68079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:42:59.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 0x7fed68079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 msgr2=0x7fed880c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b410 tx=0x7fed7800b8d0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 shutdown_connections 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7fed68076d50 0x7fed68079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed880ce1d0 0x7fed8815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed880b6790 0x7fed8815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 --2- 172.21.15.44:0/622892536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed880b29e0 0x7fed880c9d40 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.780+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 >> 172.21.15.44:0/622892536 conn(0x7fed880d3700 msgr2=0x7fed88158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.781+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 shutdown_connections 2024-07-17T08:42:59.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:42:59.781+0000 7fed8f200640 1 -- 172.21.15.44:0/622892536 wait complete. 2024-07-17T08:42:59.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 -- 172.21.15.44:0/1465785386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 msgr2=0x7f051c0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 --2- 172.21.15.44:0/1465785386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c0ae720 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f051c0d0580 tx=0x7f051402a9e0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 -- 172.21.15.44:0/1465785386 shutdown_connections 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 --2- 172.21.15.44:0/1465785386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c0ae720 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 --2- 172.21.15.44:0/1465785386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c15b3a0 0x7f051c0d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 -- 172.21.15.44:0/1465785386 >> 172.21.15.44:0/1465785386 conn(0x7f051c0b5560 msgr2=0x7f051c0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 -- 172.21.15.44:0/1465785386 shutdown_connections 2024-07-17T08:43:00.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.015+0000 7f051be00640 1 -- 172.21.15.44:0/1465785386 wait complete. 2024-07-17T08:43:00.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.018+0000 7f051be00640 1 Processor -- start 2024-07-17T08:43:00.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.018+0000 7f051be00640 1 -- start start 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 0x7f051c147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 0x7f051c0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0af0a0 con 0x7f051c0b23e0 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0aef20 con 0x7f051c15b3a0 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f051c0af220 con 0x7f051c0b3040 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54576/0 (socket says 172.21.15.44:54576) 2024-07-17T08:43:00.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.019+0000 7f051aa00640 1 -- 172.21.15.44:0/695074356 learned_addr learned my addr 172.21.15.44:0/695074356 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051a000640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 0x7f051c147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 msgr2=0x7f051c147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 0x7f051c147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 msgr2=0x7f051c0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051b400640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 0x7f051c0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 0x7f051c0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 -- 172.21.15.44:0/695074356 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c15bde0 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051a000640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 0x7f051c147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051b400640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 0x7f051c0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.020+0000 7f051aa00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f050c00d700 tx=0x7f050c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.021+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c017070 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.021+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f050c005310 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.021+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c009820 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.021+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c0b0e10 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.021+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c0b13d0 con 0x7f051c0b23e0 2024-07-17T08:43:00.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.022+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c0af760 con 0x7f051c0b23e0 2024-07-17T08:43:00.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.023+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 199) ==== 98948+0+0 (secure 0 0 0) 0x7f050c0099c0 con 0x7f051c0b23e0 2024-07-17T08:43:00.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.024+0000 7f0512a00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 0x7f04fc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.024+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f050c098af0 con 0x7f051c0b23e0 2024-07-17T08:43:00.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.025+0000 7f051a000640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 msgr2=0x7f04fc0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:00.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.025+0000 7f051a000640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 0x7f04fc0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:43:00.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.031+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f050c0619a0 con 0x7f051c0b23e0 2024-07-17T08:43:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.226+0000 7f051a000640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 msgr2=0x7f04fc0791e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:00.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.226+0000 7f051a000640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 0x7f04fc0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:43:00.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.235+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f051c06f3c0 con 0x7f051c0b23e0 2024-07-17T08:43:00.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.243+0000 7f0512a00640 1 -- 172.21.15.44:0/695074356 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v102)=0 v102) ==== 133+0+0 (secure 0 0 0) 0x7f050c0612a0 con 0x7f051c0b23e0 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 msgr2=0x7f04fc0791e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 0x7f04fc0791e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 msgr2=0x7f051c1f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f050c00d700 tx=0x7f050c00dbc0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 shutdown_connections 2024-07-17T08:43:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f04fc076d20 0x7f04fc0791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051c15b3a0 0x7f051c0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f051c0b3040 0x7f051c147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 --2- 172.21.15.44:0/695074356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f051c0b23e0 0x7f051c1f3920 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.245+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 >> 172.21.15.44:0/695074356 conn(0x7f051c0b5560 msgr2=0x7f051c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.246+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 shutdown_connections 2024-07-17T08:43:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.246+0000 7f051be00640 1 -- 172.21.15.44:0/695074356 wait complete. 2024-07-17T08:43:00.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs mygrp05 2024-07-17T08:43:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 -- 172.21.15.44:0/1189836470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69980b5620 msgr2=0x7f6998165fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 --2- 172.21.15.44:0/1189836470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69980b5620 0x7f6998165fa0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f6990005fd0 tx=0x7f699002aa90 comp rx=0 tx=0).stop 2024-07-17T08:43:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 -- 172.21.15.44:0/1189836470 shutdown_connections 2024-07-17T08:43:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 --2- 172.21.15.44:0/1189836470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69980b5620 0x7f6998165fa0 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 -- 172.21.15.44:0/1189836470 >> 172.21.15.44:0/1189836470 conn(0x7f6998073d50 msgr2=0x7f699809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 -- 172.21.15.44:0/1189836470 shutdown_connections 2024-07-17T08:43:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.449+0000 7f699fa00640 1 -- 172.21.15.44:0/1189836470 wait complete. 2024-07-17T08:43:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.451+0000 7f699fa00640 1 Processor -- start 2024-07-17T08:43:00.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.451+0000 7f699fa00640 1 -- start start 2024-07-17T08:43:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69980b01b0 0x7f6998149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69980b5620 0x7f6998149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 0x7f699814dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69981666f0 con 0x7f699815b5d0 2024-07-17T08:43:00.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69980b1310 con 0x7f69980b5620 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6998166870 con 0x7f69980b01b0 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 0x7f699814dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 0x7f699814dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54590/0 (socket says 172.21.15.44:54590) 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 -- 172.21.15.44:0/1493094868 learned_addr learned my addr 172.21.15.44:0/1493094868 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69980b01b0 0x7f6998149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f6997e00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69980b5620 0x7f6998149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69980b01b0 msgr2=0x7f6998149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69980b01b0 0x7f6998149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69980b5620 msgr2=0x7f6998149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69980b5620 0x7f6998149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f699da00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699814c540 con 0x7f699815b5d0 2024-07-17T08:43:00.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.452+0000 7f6997e00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69980b5620 0x7f6998149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:43:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f699da00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 0x7f699814dc30 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f6990009d90 tx=0x7f6990003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990046070 con 0x7f699815b5d0 2024-07-17T08:43:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6990004250 con 0x7f699815b5d0 2024-07-17T08:43:00.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990004680 con 0x7f699815b5d0 2024-07-17T08:43:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6998163750 con 0x7f699815b5d0 2024-07-17T08:43:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.453+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6998149850 con 0x7f699815b5d0 2024-07-17T08:43:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.454+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69980762e0 con 0x7f699815b5d0 2024-07-17T08:43:00.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.456+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 199) ==== 98948+0+0 (secure 0 0 0) 0x7f6990049070 con 0x7f699815b5d0 2024-07-17T08:43:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.456+0000 7f6994c00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.457+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6705+0+0 (secure 0 0 0) 0x7f69900b99b0 con 0x7f699815b5d0 2024-07-17T08:43:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.457+0000 7f699d000640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:00.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.457+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:43:00.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.461+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f69980762e0 con 0x7f699815b5d0 2024-07-17T08:43:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.658+0000 7f699d000640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.659+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:43:00.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:00.690+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7f69980b4f10 con 0x7f6978076d50 2024-07-17T08:43:01.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:01.061+0000 7f699d000640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:01.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:01.061+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:43:01.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:01.862+0000 7f699d000640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:01.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:01.862+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:43:03.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:03.464+0000 7f699d000640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/1233420839 2024-07-17T08:43:03.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:03.464+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:43:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:03.598+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 200) ==== 98557+0+0 (secure 0 0 0) 0x7f699007e360 con 0x7f699815b5d0 2024-07-17T08:43:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:03.598+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 msgr2=0x7f6978079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:43:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:03.598+0000 7f6994c00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/1233420839,v1:172.21.15.175:6837/1233420839] conn(0x7f6978076d50 0x7f6978079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:03.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:03.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:03.949+0000 7f43d1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.614+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 201) ==== 98684+0+0 (secure 0 0 0) 0x7f69900b2d20 con 0x7f699815b5d0 2024-07-17T08:43:04.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.615+0000 7f6994c00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f697807f9a0 0x7f6978081d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:04.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.615+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7f6990032ec0 con 0x7f697807f9a0 2024-07-17T08:43:04.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.617+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f697807f9a0 0x7f6978081d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:04.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.622+0000 7f699d000640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f697807f9a0 0x7f6978081d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f698c006d40 tx=0x7f698c006cd0 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:04.648 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:04.647+0000 7f43ce000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.648 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:04.647+0000 7f43ce000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.648 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:04.647+0000 7f43ce000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.648 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:04.647+0000 7f43ce000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.648 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:43:04.647+0000 7f43ce000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:43:04.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.682+0000 7f6994c00640 1 -- 172.21.15.44:0/1493094868 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6990032ec0 con 0x7f697807f9a0 2024-07-17T08:43:04.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f697807f9a0 msgr2=0x7f6978081d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:04.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f697807f9a0 0x7f6978081d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f698c006d40 tx=0x7f698c006cd0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 msgr2=0x7f699814dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:04.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 --2- 172.21.15.44:0/1493094868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f699815b5d0 0x7f699814dc30 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f6990009d90 tx=0x7f6990003f10 comp rx=0 tx=0).stop 2024-07-17T08:43:04.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699da00640 1 -- 172.21.15.44:0/1493094868 reap_dead start 2024-07-17T08:43:04.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 shutdown_connections 2024-07-17T08:43:04.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.686+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 >> 172.21.15.44:0/1493094868 conn(0x7f6998073d50 msgr2=0x7f69980b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:04.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.687+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 shutdown_connections 2024-07-17T08:43:04.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.687+0000 7f699fa00640 1 -- 172.21.15.44:0/1493094868 wait complete. 2024-07-17T08:43:04.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv05 --group_name mygrp05 2024-07-17T08:43:04.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 -- 172.21.15.44:0/469678409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 msgr2=0x7fbae41539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:04.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 --2- 172.21.15.44:0/469678409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41539f0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7fbae40b1ef0 tx=0x7fbacc02aac0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 -- 172.21.15.44:0/469678409 shutdown_connections 2024-07-17T08:43:04.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 --2- 172.21.15.44:0/469678409 >> v2:172.21.15.175:3300/0 conn(0x7fbae40b21b0 0x7fbae40b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 --2- 172.21.15.44:0/469678409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41539f0 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 -- 172.21.15.44:0/469678409 >> 172.21.15.44:0/469678409 conn(0x7fbae40b0eb0 msgr2=0x7fbae40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:04.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 -- 172.21.15.44:0/469678409 shutdown_connections 2024-07-17T08:43:04.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.924+0000 7fbae9400640 1 -- 172.21.15.44:0/469678409 wait complete. 2024-07-17T08:43:04.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.926+0000 7fbae9400640 1 Processor -- start 2024-07-17T08:43:04.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.926+0000 7fbae9400640 1 -- start start 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae406f6d0 0x7fbae41ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 0x7fbae41f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40b3090 con 0x7fbae415a5d0 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40b2f10 con 0x7fbae40b21b0 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbae40b3210 con 0x7fbae406f6d0 2024-07-17T08:43:04.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:04.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae3200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54608/0 (socket says 172.21.15.44:54608) 2024-07-17T08:43:04.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.927+0000 7fbae3200640 1 -- 172.21.15.44:0/2108874232 learned_addr learned my addr 172.21.15.44:0/2108874232 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae1e00640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 0x7fbae41f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae2800640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae406f6d0 0x7fbae41ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 -- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae406f6d0 msgr2=0x7fbae41ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae406f6d0 0x7fbae41ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 -- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 msgr2=0x7fbae41f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 0x7fbae41f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 -- 172.21.15.44:0/2108874232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae4159730 con 0x7fbae415a5d0 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae1e00640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 0x7fbae41f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbae3200640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00db10 tx=0x7fbadc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:04.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.928+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc00ae60 con 0x7fbae415a5d0 2024-07-17T08:43:04.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.929+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbadc005110 con 0x7fbae415a5d0 2024-07-17T08:43:04.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.929+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc005560 con 0x7fbae415a5d0 2024-07-17T08:43:04.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.929+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae40cd420 con 0x7fbae415a5d0 2024-07-17T08:43:04.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.929+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae40cd9e0 con 0x7fbae415a5d0 2024-07-17T08:43:04.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.930+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae40cb3a0 con 0x7fbae415a5d0 2024-07-17T08:43:04.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.931+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 201) ==== 98684+0+0 (secure 0 0 0) 0x7fbadc0047a0 con 0x7fbae415a5d0 2024-07-17T08:43:04.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.932+0000 7fbadaa00640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 0x7fbac0078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:04.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.932+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(336..336 src has 1..336) ==== 7049+0+0 (secure 0 0 0) 0x7fbadc098b80 con 0x7fbae415a5d0 2024-07-17T08:43:04.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.933+0000 7fbae2800640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 0x7fbac0078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:04.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.934+0000 7fbae2800640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 0x7fbac0078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbad4001220 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:04.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:04.936+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbae40cb3a0 con 0x7fbae415a5d0 2024-07-17T08:43:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.180+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv05", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7fbae41ef700 con 0x7fbac0076a50 2024-07-17T08:43:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.242+0000 7fbadaa00640 1 -- 172.21.15.44:0/2108874232 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbad40050d0 con 0x7fbac0076a50 2024-07-17T08:43:05.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.245+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 msgr2=0x7fbac0078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.245+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 0x7fbac0078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbad4001220 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.245+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 msgr2=0x7fbae41f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.245+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00db10 tx=0x7fbadc00a250 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 shutdown_connections 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fbac0076a50 0x7fbac0078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbae415a5d0 0x7fbae41f5860 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbae40b21b0 0x7fbae41f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 --2- 172.21.15.44:0/2108874232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbae406f6d0 0x7fbae41ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 >> 172.21.15.44:0/2108874232 conn(0x7fbae40b0eb0 msgr2=0x7fbae4141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 shutdown_connections 2024-07-17T08:43:05.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.246+0000 7fbae9400640 1 -- 172.21.15.44:0/2108874232 wait complete. 2024-07-17T08:43:05.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv05 --group mygrp05 --fs cephfs . 1m 2024-07-17T08:43:05.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 -- 172.21.15.44:0/673127282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2550153a50 msgr2=0x7f2550159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 --2- 172.21.15.44:0/673127282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2550153a50 0x7f2550159b00 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7f25500cbb00 tx=0x7f254002a540 comp rx=0 tx=0).stop 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 -- 172.21.15.44:0/673127282 shutdown_connections 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 --2- 172.21.15.44:0/673127282 >> v2:172.21.15.175:3300/0 conn(0x7f255015a040 0x7f25500ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 --2- 172.21.15.44:0/673127282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2550153a50 0x7f2550159b00 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.479+0000 7f2556e00640 1 -- 172.21.15.44:0/673127282 >> 172.21.15.44:0/673127282 conn(0x7f25500b0fb0 msgr2=0x7f25500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.480+0000 7f2556e00640 1 -- 172.21.15.44:0/673127282 shutdown_connections 2024-07-17T08:43:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.480+0000 7f2556e00640 1 -- 172.21.15.44:0/673127282 wait complete. 2024-07-17T08:43:05.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.481+0000 7f2556e00640 1 Processor -- start 2024-07-17T08:43:05.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.481+0000 7f2556e00640 1 -- start start 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.482+0000 7f2556e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500ac920 0x7f2550074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.482+0000 7f2556e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550153a50 0x7f255014ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.482+0000 7f2556e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2556e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2550153e30 con 0x7f255015a040 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2556e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f255006f1b0 con 0x7f25500ac920 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2556e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2550153fb0 con 0x7f2550153a50 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54626/0 (socket says 172.21.15.44:54626) 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 -- 172.21.15.44:0/1083705584 learned_addr learned my addr 172.21.15.44:0/1083705584 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:43:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f254fe00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500ac920 0x7f2550074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 -- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550153a50 msgr2=0x7f255014ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550153a50 0x7f255014ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 -- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500ac920 msgr2=0x7f2550074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500ac920 0x7f2550074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.483+0000 7f2554e00640 1 -- 172.21.15.44:0/1083705584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2550073390 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2554e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f254800ca70 tx=0x7f254800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548016020 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f254800acc0 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548012680 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2550073670 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.484+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2550073c30 con 0x7f255015a040 2024-07-17T08:43:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.485+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2550159b00 con 0x7f255015a040 2024-07-17T08:43:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.487+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 201) ==== 98684+0+0 (secure 0 0 0) 0x7f2548019020 con 0x7f255015a040 2024-07-17T08:43:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.488+0000 7f2547e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 0x7f2530078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:43:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.488+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(336..336 src has 1..336) ==== 7049+0+0 (secure 0 0 0) 0x7f2548098c10 con 0x7f255015a040 2024-07-17T08:43:05.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.488+0000 7f254fe00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 0x7f2530078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:43:05.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.489+0000 7f254fe00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 0x7f2530078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f2540004730 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:43:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.492+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2548020020 con 0x7f255015a040 2024-07-17T08:43:05.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.677+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 202) ==== 99000+0+0 (secure 0 0 0) 0x7f254805d9a0 con 0x7f255015a040 2024-07-17T08:43:05.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.736+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv05", "group": "mygrp05", "fs": "cephfs", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f255014f2f0 con 0x7f2530076a50 2024-07-17T08:43:05.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.951+0000 7f2547e00640 1 -- 172.21.15.44:0/1083705584 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+84 (secure 0 0 0) 0x7f255014f2f0 con 0x7f2530076a50 2024-07-17T08:43:05.954 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/.. 2024-07-17T08:43:05.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.955+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 msgr2=0x7f2530078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.955+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 0x7f2530078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f2540004730 comp rx=0 tx=0).stop 2024-07-17T08:43:05.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.955+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 msgr2=0x7f25501f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.955+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f254800ca70 tx=0x7f254800cf30 comp rx=0 tx=0).stop 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 shutdown_connections 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2530076a50 0x7f2530078f10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f255015a040 0x7f25501f3c30 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550153a50 0x7f255014ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 --2- 172.21.15.44:0/1083705584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25500ac920 0x7f2550074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 >> 172.21.15.44:0/1083705584 conn(0x7f25500b0fb0 msgr2=0x7f25500b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 shutdown_connections 2024-07-17T08:43:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:43:05.956+0000 7f2556e00640 1 -- 172.21.15.44:0/1083705584 wait complete. 2024-07-17T08:43:05.979 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/.. 2024-07-17T08:43:12.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:43:12.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:43:42.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:43:42.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:44:00.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:00.977+0000 7f43b6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:00.979 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:00.977+0000 7f43b6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:00.979 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:00.977+0000 7f43b6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:00.979 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:00.977+0000 7f43b6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:00.979 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:00.977+0000 7f43b6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:12.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:44:12.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:44:20.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv05 --group_name mygrp05 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 -- 172.21.15.44:0/3947145200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 msgr2=0x7fdc9c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 --2- 172.21.15.44:0/3947145200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdc9c0051a0 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0001690 tx=0x7fdc9c005830 comp rx=0 tx=0).stop 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbd600640 1 -- 172.21.15.44:0/3947145200 reap_dead start 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 -- 172.21.15.44:0/3947145200 shutdown_connections 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 -- 172.21.15.44:0/3947145200 >> 172.21.15.44:0/3947145200 conn(0x7fdcb80b0eb0 msgr2=0x7fdcb80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 -- 172.21.15.44:0/3947145200 shutdown_connections 2024-07-17T08:44:21.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.216+0000 7fdcbf600640 1 -- 172.21.15.44:0/3947145200 wait complete. 2024-07-17T08:44:21.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.218+0000 7fdcbf600640 1 Processor -- start 2024-07-17T08:44:21.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.218+0000 7fdcbf600640 1 -- start start 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb81f3fb0 0x7fdcb80adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 0x7fdcb81e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb815c430 con 0x7fdc9c004d40 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb815b690 con 0x7fdcb81f3fb0 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbf600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb80cd300 con 0x7fdcb8153060 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbcc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51240/0 (socket says 172.21.15.44:51240) 2024-07-17T08:44:21.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbcc00640 1 -- 172.21.15.44:0/2165231637 learned_addr learned my addr 172.21.15.44:0/2165231637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbd600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 0x7fdcb81e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcbcc00640 1 -- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 msgr2=0x7fdcb81e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.219+0000 7fdcb7e00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb81f3fb0 0x7fdcb80adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbcc00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 0x7fdcb81e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbcc00640 1 -- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb81f3fb0 msgr2=0x7fdcb80adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbcc00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb81f3fb0 0x7fdcb80adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbcc00640 1 -- 172.21.15.44:0/2165231637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb8150a90 con 0x7fdc9c004d40 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbd600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 0x7fdcb81e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbcc00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7fdca800dc60 tx=0x7fdca800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003b70 con 0x7fdc9c004d40 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdca8003e00 con 0x7fdc9c004d40 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8006370 con 0x7fdc9c004d40 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb81f4470 con 0x7fdc9c004d40 2024-07-17T08:44:21.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.220+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81f4a30 con 0x7fdc9c004d40 2024-07-17T08:44:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.222+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb806d620 con 0x7fdc9c004d40 2024-07-17T08:44:21.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.223+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 204) ==== 99113+0+0 (secure 0 0 0) 0x7fdca8006510 con 0x7fdc9c004d40 2024-07-17T08:44:21.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.224+0000 7fdcb4c00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 0x7fdc94079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.224+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7fdca80997f0 con 0x7fdc9c004d40 2024-07-17T08:44:21.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.225+0000 7fdcb7e00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 0x7fdc94079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.226+0000 7fdcb7e00640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 0x7fdc94079390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdca00058c0 tx=0x7fdca0005850 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:21.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.228+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdca8006800 con 0x7fdc9c004d40 2024-07-17T08:44:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.467+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv05", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7fdcb80ac190 con 0x7fdc94076ed0 2024-07-17T08:44:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.483+0000 7fdcb4c00640 1 -- 172.21.15.44:0/2165231637 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fdcb80ac190 con 0x7fdc94076ed0 2024-07-17T08:44:21.484 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a 2024-07-17T08:44:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.485+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 msgr2=0x7fdc94079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.485+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 0x7fdc94079390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdca00058c0 tx=0x7fdca0005850 comp rx=0 tx=0).stop 2024-07-17T08:44:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.485+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 msgr2=0x7fdcb81540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7fdca800dc60 tx=0x7fdca800b620 comp rx=0 tx=0).stop 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 shutdown_connections 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fdc94076ed0 0x7fdc94079390 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcb8153060 0x7fdcb81e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcb81f3fb0 0x7fdcb80adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 --2- 172.21.15.44:0/2165231637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdc9c004d40 0x7fdcb81540b0 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 >> 172.21.15.44:0/2165231637 conn(0x7fdcb80b0eb0 msgr2=0x7fdcb80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 shutdown_connections 2024-07-17T08:44:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.486+0000 7fdcbf600640 1 -- 172.21.15.44:0/2165231637 wait complete. 2024-07-17T08:44:21.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:21.505 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/../.snap) 2024-07-17T08:44:21.559 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_44_00_UTC 2024-07-17T08:44:21.560 DEBUG:tasks.cephfs.test_snap_schedules:snaps:['scheduled-2024-07-17-08_44_00_UTC'] 2024-07-17T08:44:21.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --subvol sv05 --group mygrp05 --fs cephfs . 1m 2024-07-17T08:44:21.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.777+0000 7f7c03000640 1 -- 172.21.15.44:0/2819139437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cfe10 msgr2=0x7f7bfc0b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.777+0000 7f7c03000640 1 --2- 172.21.15.44:0/2819139437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cfe10 0x7f7bfc0b4660 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc08d450 tx=0x7f7bf002a540 comp rx=0 tx=0).stop 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 -- 172.21.15.44:0/2819139437 shutdown_connections 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 --2- 172.21.15.44:0/2819139437 >> v2:172.21.15.175:3300/0 conn(0x7f7bfc0d01f0 0x7f7bfc0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 --2- 172.21.15.44:0/2819139437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0cfe10 0x7f7bfc0b4660 unknown :-1 s=CLOSED pgs=3079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 -- 172.21.15.44:0/2819139437 >> 172.21.15.44:0/2819139437 conn(0x7f7bfc08ca20 msgr2=0x7f7bfc0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 -- 172.21.15.44:0/2819139437 shutdown_connections 2024-07-17T08:44:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.778+0000 7f7c03000640 1 -- 172.21.15.44:0/2819139437 wait complete. 2024-07-17T08:44:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.780+0000 7f7c03000640 1 Processor -- start 2024-07-17T08:44:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.780+0000 7f7c03000640 1 -- start start 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc0cfe10 0x7f7bfc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc0d1790 0x7f7bfc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0b4ba0 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0b4d20 con 0x7f7bfc0d1790 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c03000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc0d1c00 con 0x7f7bfc0cfe10 2024-07-17T08:44:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c01200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c01200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51248/0 (socket says 172.21.15.44:51248) 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c01200640 1 -- 172.21.15.44:0/634476275 learned_addr learned my addr 172.21.15.44:0/634476275 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c01c00640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc0cfe10 0x7f7bfc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.782+0000 7f7c02600640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc0d1790 0x7f7bfc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 -- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc0cfe10 msgr2=0x7f7bfc0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc0cfe10 0x7f7bfc0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 -- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc0d1790 msgr2=0x7f7bfc1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc0d1790 0x7f7bfc1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 -- 172.21.15.44:0/634476275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc15da10 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c01200640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f7be800d770 tx=0x7f7be800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8017070 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7be8009d20 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8005280 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc0cc960 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.783+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc0ccf20 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.785+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc0cd120 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.786+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 204) ==== 99113+0+0 (secure 0 0 0) 0x7f7be8005420 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.787+0000 7f7bf9600640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 0x7f7bdc079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.787+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f7be8098fd0 con 0x7f7bfc0d01f0 2024-07-17T08:44:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.787+0000 7f7c01c00640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 0x7f7bdc079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.789+0000 7f7c01c00640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 0x7f7bdc079390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0033000 tx=0x7f7bf0004730 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:21.791+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7bfc0cd120 con 0x7f7bfc0d01f0 2024-07-17T08:44:22.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.039+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "subvol": "sv05", "group": "mygrp05", "fs": "cephfs", "path": ".", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7bfc0b4660 con 0x7f7bdc076ed0 2024-07-17T08:44:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.276+0000 7f7bf9600640 1 -- 172.21.15.44:0/634476275 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+88 (secure 0 0 0) 0x7f7bfc0b4660 con 0x7f7bdc076ed0 2024-07-17T08:44:22.277 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/.. 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.279+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 msgr2=0x7f7bdc079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.279+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 0x7f7bdc079390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0033000 tx=0x7f7bf0004730 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 msgr2=0x7f7bfc0b86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f7be800d770 tx=0x7f7be800dc30 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 shutdown_connections 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f7bdc076ed0 0x7f7bdc079390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bfc0d1790 0x7f7bfc1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bfc0d01f0 0x7f7bfc0b86e0 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 --2- 172.21.15.44:0/634476275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bfc0cfe10 0x7f7bfc0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 >> 172.21.15.44:0/634476275 conn(0x7f7bfc08ca20 msgr2=0x7f7bfc0cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 shutdown_connections 2024-07-17T08:44:22.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.280+0000 7f7c03000640 1 -- 172.21.15.44:0/634476275 wait complete. 2024-07-17T08:44:22.301 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/.. 2024-07-17T08:44:22.301 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv05 --group_name mygrp05 2024-07-17T08:44:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 -- 172.21.15.44:0/1896268555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 msgr2=0x7f98a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 --2- 172.21.15.44:0/1896268555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98a0003e00 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c6e0 tx=0x7f98a0005830 comp rx=0 tx=0).stop 2024-07-17T08:44:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 -- 172.21.15.44:0/1896268555 shutdown_connections 2024-07-17T08:44:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 --2- 172.21.15.44:0/1896268555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98a0003e00 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 -- 172.21.15.44:0/1896268555 >> 172.21.15.44:0/1896268555 conn(0x7f98bc0cf050 msgr2=0x7f98bc0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:22.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 -- 172.21.15.44:0/1896268555 shutdown_connections 2024-07-17T08:44:22.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.519+0000 7f98bbe00640 1 -- 172.21.15.44:0/1896268555 wait complete. 2024-07-17T08:44:22.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.521+0000 7f98bbe00640 1 Processor -- start 2024-07-17T08:44:22.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.521+0000 7f98bbe00640 1 -- start start 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 0x7f98bc1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc1f5550 0x7f98bc152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc0ac5b0 con 0x7f98a0003a00 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc0ac430 con 0x7f98bc0ce890 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98bbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc0ac730 con 0x7f98bc1f5550 2024-07-17T08:44:22.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:22.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.522+0000 7f98baa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51264/0 (socket says 172.21.15.44:51264) 2024-07-17T08:44:22.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 -- 172.21.15.44:0/2824980260 learned_addr learned my addr 172.21.15.44:0/2824980260 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98b3e00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 0x7f98bc1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98bb400640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc1f5550 0x7f98bc152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 -- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc1f5550 msgr2=0x7f98bc152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc1f5550 0x7f98bc152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 -- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 msgr2=0x7f98bc1f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 0x7f98bc1f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 -- 172.21.15.44:0/2824980260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc14ef30 con 0x7f98a0003a00 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98b3e00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 0x7f98bc1f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.523+0000 7f98baa00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02a7f0 tx=0x7f98ac006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:22.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.524+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004830 con 0x7f98a0003a00 2024-07-17T08:44:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.524+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98ac039070 con 0x7f98a0003a00 2024-07-17T08:44:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.524+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc151c80 con 0x7f98a0003a00 2024-07-17T08:44:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.524+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004d80 con 0x7f98a0003a00 2024-07-17T08:44:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.524+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1505c0 con 0x7f98a0003a00 2024-07-17T08:44:22.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.525+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc1538b0 con 0x7f98a0003a00 2024-07-17T08:44:22.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.526+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 204) ==== 99113+0+0 (secure 0 0 0) 0x7f98ac043030 con 0x7f98a0003a00 2024-07-17T08:44:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.527+0000 7f98b2a00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 0x7f9898079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.527+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f98ac0b8600 con 0x7f98a0003a00 2024-07-17T08:44:22.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.528+0000 7f98b3e00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 0x7f9898079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:22.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.529+0000 7f98b3e00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 0x7f9898079390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005fd0 tx=0x7f98a4005ad0 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:22.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.532+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98ac04a060 con 0x7f98a0003a00 2024-07-17T08:44:22.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.777+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv05", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7f98bc06c2c0 con 0x7f9898076ed0 2024-07-17T08:44:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.788+0000 7f98b2a00640 1 -- 172.21.15.44:0/2824980260 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f98bc06c2c0 con 0x7f9898076ed0 2024-07-17T08:44:22.789 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 msgr2=0x7f9898079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 0x7f9898079390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005fd0 tx=0x7f98a4005ad0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 msgr2=0x7f98bc0738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02a7f0 tx=0x7f98ac006380 comp rx=0 tx=0).stop 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 shutdown_connections 2024-07-17T08:44:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9898076ed0 0x7f9898079390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98bc1f5550 0x7f98bc152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98bc0ce890 0x7f98bc1f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 --2- 172.21.15.44:0/2824980260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a0003a00 0x7f98bc0738c0 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.791+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 >> 172.21.15.44:0/2824980260 conn(0x7f98bc0cf050 msgr2=0x7f98bc0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.792+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 shutdown_connections 2024-07-17T08:44:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:22.792+0000 7f98bbe00640 1 -- 172.21.15.44:0/2824980260 wait complete. 2024-07-17T08:44:22.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:22.812 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/../.snap) 2024-07-17T08:44:22.867 INFO:teuthology.orchestra.run.smithi044.stdout:scheduled-2024-07-17-08_44_00_UTC 2024-07-17T08:44:22.869 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/../.snap/scheduled-2024-07-17-08_44_00_UTC 2024-07-17T08:44:22.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:22.869 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir volumes/mygrp05/sv05/4ca45e6e-81f1-45f0-b4de-d707bdf7706a/../.snap/scheduled-2024-07-17-08_44_00_UTC) 2024-07-17T08:44:22.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv05 --group_name mygrp05 2024-07-17T08:44:23.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 -- 172.21.15.44:0/2374091911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46b80051c0 msgr2=0x7f46b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 --2- 172.21.15.44:0/2374091911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46b80051c0 0x7f46b800b9c0 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 crypto rx=0x7f46cc001d90 tx=0x7f46b8005830 comp rx=0 tx=0).stop 2024-07-17T08:44:23.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 -- 172.21.15.44:0/2374091911 shutdown_connections 2024-07-17T08:44:23.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 --2- 172.21.15.44:0/2374091911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46b80051c0 0x7f46b800b9c0 unknown :-1 s=CLOSED pgs=3085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 -- 172.21.15.44:0/2374091911 >> 172.21.15.44:0/2374091911 conn(0x7f46d406ce80 msgr2=0x7f46d406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:23.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.195+0000 7f46d9e00640 1 -- 172.21.15.44:0/2374091911 shutdown_connections 2024-07-17T08:44:23.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.196+0000 7f46d9e00640 1 -- 172.21.15.44:0/2374091911 wait complete. 2024-07-17T08:44:23.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.197+0000 7f46d9e00640 1 Processor -- start 2024-07-17T08:44:23.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.197+0000 7f46d9e00640 1 -- start start 2024-07-17T08:44:23.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b80051c0 0x7f46d4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 0x7f46d4155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4073270 con 0x7f46d4150c90 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46d406f6a0 con 0x7f46b80051c0 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.198+0000 7f46d9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46d40733f0 con 0x7f46d414fd00 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51294/0 (socket says 172.21.15.44:51294) 2024-07-17T08:44:23.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 -- 172.21.15.44:0/43204389 learned_addr learned my addr 172.21.15.44:0/43204389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:23.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d2000640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 0x7f46d4155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d2a00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b80051c0 0x7f46d4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 -- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 msgr2=0x7f46d4155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 0x7f46d4155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 -- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b80051c0 msgr2=0x7f46d4152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b80051c0 0x7f46d4152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d3400640 1 -- 172.21.15.44:0/43204389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d4151190 con 0x7f46d4150c90 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.199+0000 7f46d2000640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 0x7f46d4155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46d3400640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f46cc001eb0 tx=0x7f46cc006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc003d80 con 0x7f46d4150c90 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46cc005b00 con 0x7f46d4150c90 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc005f50 con 0x7f46d4150c90 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d4151470 con 0x7f46d4150c90 2024-07-17T08:44:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.200+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d4151a30 con 0x7f46d4150c90 2024-07-17T08:44:23.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.201+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d40aea20 con 0x7f46d4150c90 2024-07-17T08:44:23.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.203+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 204) ==== 99113+0+0 (secure 0 0 0) 0x7f46cc0060f0 con 0x7f46d4150c90 2024-07-17T08:44:23.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.204+0000 7f46cb400640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 0x7f46b4079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.204+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f46cc0b88b0 con 0x7f46d4150c90 2024-07-17T08:44:23.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.204+0000 7f46d2a00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 0x7f46b4079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.206+0000 7f46d2a00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 0x7f46b4079390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001eb0 tx=0x7f46c4001110 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.208+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f46cc045030 con 0x7f46d4150c90 2024-07-17T08:44:23.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.451+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv05", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7f46d4152770 con 0x7f46b4076ed0 2024-07-17T08:44:23.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.495+0000 7f46cb400640 1 -- 172.21.15.44:0/43204389 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f46c4004e10 con 0x7f46b4076ed0 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 msgr2=0x7f46b4079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 0x7f46b4079390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001eb0 tx=0x7f46c4001110 comp rx=0 tx=0).stop 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 msgr2=0x7f46d4155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f46cc001eb0 tx=0x7f46cc006ca0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 shutdown_connections 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f46b4076ed0 0x7f46b4079390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46d4150c90 0x7f46d4155b80 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46d414fd00 0x7f46d4155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 --2- 172.21.15.44:0/43204389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b80051c0 0x7f46d4152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.498+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 >> 172.21.15.44:0/43204389 conn(0x7f46d406ce80 msgr2=0x7f46d41537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.499+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 shutdown_connections 2024-07-17T08:44:23.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.499+0000 7f46d9e00640 1 -- 172.21.15.44:0/43204389 wait complete. 2024-07-17T08:44:23.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs mygrp05 2024-07-17T08:44:23.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:23.527+0000 7f43cfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:23.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:23.527+0000 7f43cfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:23.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:23.527+0000 7f43cfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:23.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:23.527+0000 7f43cfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:23.528 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:23.527+0000 7f43cfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 -- 172.21.15.44:0/2195843965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d806f380 msgr2=0x7f68d80cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 --2- 172.21.15.44:0/2195843965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d806f380 0x7f68d80cb080 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005fd0 tx=0x7f68d002a540 comp rx=0 tx=0).stop 2024-07-17T08:44:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 -- 172.21.15.44:0/2195843965 shutdown_connections 2024-07-17T08:44:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 --2- 172.21.15.44:0/2195843965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d806f380 0x7f68d80cb080 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 -- 172.21.15.44:0/2195843965 >> 172.21.15.44:0/2195843965 conn(0x7f68d80cf0d0 msgr2=0x7f68d80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 -- 172.21.15.44:0/2195843965 shutdown_connections 2024-07-17T08:44:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.725+0000 7f68d7e00640 1 -- 172.21.15.44:0/2195843965 wait complete. 2024-07-17T08:44:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.727+0000 7f68d7e00640 1 Processor -- start 2024-07-17T08:44:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.727+0000 7f68d7e00640 1 -- start start 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d806f380 0x7f68d81f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d80cd0b0 0x7f68d81506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f68d80cb5c0 con 0x7f68d8151f20 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8152510 con 0x7f68d80cd0b0 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.728+0000 7f68d7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68d80cb740 con 0x7f68d806f380 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51320/0 (socket says 172.21.15.44:51320) 2024-07-17T08:44:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 -- 172.21.15.44:0/852611784 learned_addr learned my addr 172.21.15.44:0/852611784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d6a00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d806f380 0x7f68d81f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d6000640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d80cd0b0 0x7f68d81506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 -- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d806f380 msgr2=0x7f68d81f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d806f380 0x7f68d81f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 -- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d80cd0b0 msgr2=0x7f68d81506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d80cd0b0 0x7f68d81506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.729+0000 7f68d7400640 1 -- 172.21.15.44:0/852611784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d809d0e0 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68d7400640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009230 tx=0x7f68d0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d003f070 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68d0004890 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d809cb20 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0035030 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.730+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81527c0 con 0x7f68d8151f20 2024-07-17T08:44:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.731+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d80cb080 con 0x7f68d8151f20 2024-07-17T08:44:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.733+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 204) ==== 99113+0+0 (secure 0 0 0) 0x7f68d0042070 con 0x7f68d8151f20 2024-07-17T08:44:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.734+0000 7f68cea00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 0x7f68b4079390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.734+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f68d00b9c90 con 0x7f68d8151f20 2024-07-17T08:44:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.734+0000 7f68d6a00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 0x7f68b4079390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:23.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.735+0000 7f68d6a00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 0x7f68b4079390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f20 tx=0x7f68c80094d0 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:23.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.738+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f68d00827a0 con 0x7f68d8151f20 2024-07-17T08:44:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.977+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 --> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "mygrp05", "target": ["mon-mgr", ""]}) -- 0x7f68d80008d0 con 0x7f68b4076ed0 2024-07-17T08:44:23.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.988+0000 7f68cea00640 1 -- 172.21.15.44:0/852611784 <== mgr.9695 v2:172.21.15.175:6834/64945748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68d80008d0 con 0x7f68b4076ed0 2024-07-17T08:44:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 msgr2=0x7f68b4079390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 0x7f68b4079390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f20 tx=0x7f68c80094d0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 msgr2=0x7f68d8154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009230 tx=0x7f68d0003960 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 shutdown_connections 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f68b4076ed0 0x7f68b4079390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f68d8151f20 0x7f68d8154d90 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68d80cd0b0 0x7f68d81506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 --2- 172.21.15.44:0/852611784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68d806f380 0x7f68d81f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.991+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 >> 172.21.15.44:0/852611784 conn(0x7f68d80cf0d0 msgr2=0x7f68d80af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.992+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 shutdown_connections 2024-07-17T08:44:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:23.992+0000 7f68d7e00640 1 -- 172.21.15.44:0/852611784 wait complete. 2024-07-17T08:44:24.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:44:24.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 -- 172.21.15.44:0/2752373544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 msgr2=0x7f2f10003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:24.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 --2- 172.21.15.44:0/2752373544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f10003ea0 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c006f80 tx=0x7f2f10005830 comp rx=0 tx=0).stop 2024-07-17T08:44:24.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f2be00640 1 -- 172.21.15.44:0/2752373544 reap_dead start 2024-07-17T08:44:24.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 -- 172.21.15.44:0/2752373544 shutdown_connections 2024-07-17T08:44:24.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 -- 172.21.15.44:0/2752373544 >> 172.21.15.44:0/2752373544 conn(0x7f2f2c0d3460 msgr2=0x7f2f2c0d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:24.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 -- 172.21.15.44:0/2752373544 shutdown_connections 2024-07-17T08:44:24.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.231+0000 7f2f31200640 1 -- 172.21.15.44:0/2752373544 wait complete. 2024-07-17T08:44:24.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.233+0000 7f2f31200640 1 Processor -- start 2024-07-17T08:44:24.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.233+0000 7f2f31200640 1 -- start start 2024-07-17T08:44:24.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:24.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 0x7f2f2c0ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:24.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 0x7f2f2c0abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0b62b0 con 0x7f2f10003a80 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0b8ed0 con 0x7f2f2c160000 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.234+0000 7f2f31200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2c0b6430 con 0x7f2f2c160440 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51334/0 (socket says 172.21.15.44:51334) 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 -- 172.21.15.44:0/2632746692 learned_addr learned my addr 172.21.15.44:0/2632746692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2be00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 0x7f2f2c0abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2aa00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 0x7f2f2c0ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 -- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 msgr2=0x7f2f2c0abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 0x7f2f2c0abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 -- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 msgr2=0x7f2f2c0ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 0x7f2f2c0ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:24.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 -- 172.21.15.44:0/2632746692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c15d810 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2aa00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 0x7f2f2c0ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2be00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 0x7f2f2c0abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.235+0000 7f2f2b400640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c006f80 tx=0x7f2f1c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.236+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c006bb0 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.236+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f1c0015e0 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.236+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c037690 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.236+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c1f41d0 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.236+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1f4790 con 0x7f2f10003a80 2024-07-17T08:44:24.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.237+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c0d2950 con 0x7f2f10003a80 2024-07-17T08:44:24.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.238+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 205) ==== 99164+0+0 (secure 0 0 0) 0x7f2f1c035020 con 0x7f2f10003a80 2024-07-17T08:44:24.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.239+0000 7f2f23400640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 0x7f2f0c079420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:24.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.239+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f2f1c0b96c0 con 0x7f2f10003a80 2024-07-17T08:44:24.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.240+0000 7f2f2aa00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 0x7f2f0c079420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.241+0000 7f2f2aa00640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 0x7f2f0c079420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0b7d40 tx=0x7f2f18005f90 comp rx=0 tx=0).ready entity=mgr.9695 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.244+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2f1c0820c0 con 0x7f2f10003a80 2024-07-17T08:44:24.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:24.509+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f2f2c0b2ca0 con 0x7f2f10003a80 2024-07-17T08:44:25.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.049+0000 7f2f23400640 1 -- 172.21.15.44:0/2632746692 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v206) ==== 93+0+0 (secure 0 0 0) 0x7f2f1c006260 con 0x7f2f10003a80 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.052+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 msgr2=0x7f2f0c079420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.052+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 0x7f2f0c079420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0b7d40 tx=0x7f2f18005f90 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.052+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 msgr2=0x7f2f2c0b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.052+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c006f80 tx=0x7f2f1c003de0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 shutdown_connections 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f2f0c076f60 0x7f2f0c079420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f2c160440 0x7f2f2c0abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f2c160000 0x7f2f2c0ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 --2- 172.21.15.44:0/2632746692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f10003a80 0x7f2f2c0b15a0 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 >> 172.21.15.44:0/2632746692 conn(0x7f2f2c0d3460 msgr2=0x7f2f2c0d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:25.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 shutdown_connections 2024-07-17T08:44:25.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.053+0000 7f2f31200640 1 -- 172.21.15.44:0/2632746692 wait complete. 2024-07-17T08:44:25.081 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:44:25.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:44:25.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:44:25.274 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:44:25.274 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:44:25.274 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:25.274 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:25.326 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:25.326 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:25.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.660+0000 7fcf3f400640 1 -- 172.21.15.44:0/3837270032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 msgr2=0x7fcf380cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.660+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3837270032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf380cb810 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7fcf380d0260 tx=0x7fcf3002a980 comp rx=0 tx=0).stop 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.660+0000 7fcf3f400640 1 -- 172.21.15.44:0/3837270032 shutdown_connections 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.660+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3837270032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 0x7fcf3815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.660+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3837270032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf380cb810 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.661+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3837270032 >> v2:172.21.15.175:3300/0 conn(0x7fcf38152220 0x7fcf3806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.661+0000 7fcf3f400640 1 -- 172.21.15.44:0/3837270032 >> 172.21.15.44:0/3837270032 conn(0x7fcf380b0eb0 msgr2=0x7fcf380b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:25.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.661+0000 7fcf3f400640 1 -- 172.21.15.44:0/3837270032 shutdown_connections 2024-07-17T08:44:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.661+0000 7fcf3f400640 1 -- 172.21.15.44:0/3837270032 wait complete. 2024-07-17T08:44:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.663+0000 7fcf3f400640 1 Processor -- start 2024-07-17T08:44:25.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.663+0000 7fcf3f400640 1 -- start start 2024-07-17T08:44:25.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 0x7fcf381430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf38152220 0x7fcf381454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3815f1f0 con 0x7fcf380cb3a0 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3815f070 con 0x7fcf38152220 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3815f370 con 0x7fcf3806ed30 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf37e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51354/0 (socket says 172.21.15.44:51354) 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf37e00640 1 -- 172.21.15.44:0/3841891861 learned_addr learned my addr 172.21.15.44:0/3841891861 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3ca00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 0x7fcf381430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.664+0000 7fcf3d400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf38152220 0x7fcf381454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 msgr2=0x7fcf381430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 0x7fcf381430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf38152220 msgr2=0x7fcf381454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf38152220 0x7fcf381454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 -- 172.21.15.44:0/3841891861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf38074270 con 0x7fcf380cb3a0 2024-07-17T08:44:25.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf37e00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400d770 tx=0x7fcf2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24017070 con 0x7fcf380cb3a0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf24009d20 con 0x7fcf380cb3a0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24005280 con 0x7fcf380cb3a0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.665+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf380ac330 con 0x7fcf380cb3a0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.666+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf380ac8f0 con 0x7fcf380cb3a0 2024-07-17T08:44:25.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.667+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf380cb810 con 0x7fcf380cb3a0 2024-07-17T08:44:25.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.668+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7fcf24005420 con 0x7fcf380cb3a0 2024-07-17T08:44:25.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.669+0000 7fcf34c00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 0x7fcf18079400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:25.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.669+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7fcf240995a0 con 0x7fcf380cb3a0 2024-07-17T08:44:25.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.670+0000 7fcf3ca00640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 msgr2=0x7fcf18079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:25.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.670+0000 7fcf3ca00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 0x7fcf18079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:25.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.673+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcf24061ea0 con 0x7fcf380cb3a0 2024-07-17T08:44:25.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.871+0000 7fcf3ca00640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 msgr2=0x7fcf18079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:25.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.871+0000 7fcf3ca00640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 0x7fcf18079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.895+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf380cde20 con 0x7fcf380cb3a0 2024-07-17T08:44:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.896+0000 7fcf34c00640 1 -- 172.21.15.44:0/3841891861 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v338) ==== 80+0+475 (secure 0 0 0) 0x7fcf2406a0e0 con 0x7fcf380cb3a0 2024-07-17T08:44:25.898 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/19909624 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.898 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4141702117 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.898 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/723525579 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.899 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2709336751 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.899 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1419861374 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.899 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1265901122 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.899 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/1233420839 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.899 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/1233420839 2024-07-18T08:43:03.584378+0000 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.898+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 msgr2=0x7fcf18079400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.898+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 0x7fcf18079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.898+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 msgr2=0x7fcf38144f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.898+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400d770 tx=0x7fcf2400dc30 comp rx=0 tx=0).stop 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 shutdown_connections 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcf18076f40 0x7fcf18079400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf38152220 0x7fcf381454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf380cb3a0 0x7fcf38144f60 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 --2- 172.21.15.44:0/3841891861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf3806ed30 0x7fcf381430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 >> 172.21.15.44:0/3841891861 conn(0x7fcf380b0eb0 msgr2=0x7fcf380b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 shutdown_connections 2024-07-17T08:44:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:25.899+0000 7fcf3f400640 1 -- 172.21.15.44:0/3841891861 wait complete. 2024-07-17T08:44:25.919 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:44:25.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:25.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:44:26.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:44:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:26.008 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:44:26.049 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:44:26.049 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:44:26.049 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:26.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:44:26.097 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:44:26.098 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:26.098 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:26.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:44:26.145 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:44:26.145 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:44:26.145 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:26.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:44:26.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:26.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 -- 172.21.15.44:0/3584697121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 msgr2=0x7f9c980b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 --2- 172.21.15.44:0/3584697121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980b5700 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009830 tx=0x7f9c8402a5a0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 -- 172.21.15.44:0/3584697121 shutdown_connections 2024-07-17T08:44:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 --2- 172.21.15.44:0/3584697121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980b5700 unknown :-1 s=CLOSED pgs=3098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 -- 172.21.15.44:0/3584697121 >> 172.21.15.44:0/3584697121 conn(0x7f9c98072730 msgr2=0x7f9c98072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 -- 172.21.15.44:0/3584697121 shutdown_connections 2024-07-17T08:44:26.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.404+0000 7f9c9f400640 1 -- 172.21.15.44:0/3584697121 wait complete. 2024-07-17T08:44:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.406+0000 7f9c9f400640 1 Processor -- start 2024-07-17T08:44:26.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.407+0000 7f9c9f400640 1 -- start start 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c980afb50 0x7f9c981f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c98154840 0x7f9c980ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98073a50 con 0x7f9c9815b1a0 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c980700c0 con 0x7f9c98154840 2024-07-17T08:44:26.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9c980b5c40 con 0x7f9c980afb50 2024-07-17T08:44:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51378/0 (socket says 172.21.15.44:51378) 2024-07-17T08:44:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.408+0000 7f9c9d400640 1 -- 172.21.15.44:0/37686512 learned_addr learned my addr 172.21.15.44:0/37686512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9ca00640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c980afb50 0x7f9c981f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c97e00640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c98154840 0x7f9c980ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c980afb50 msgr2=0x7f9c981f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c980afb50 0x7f9c981f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c98154840 msgr2=0x7f9c980ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c98154840 0x7f9c980ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 -- 172.21.15.44:0/37686512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c981f1f20 con 0x7f9c9815b1a0 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.409+0000 7f9c9d400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c440 tx=0x7f9c9000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:26.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.410+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90011790 con 0x7f9c9815b1a0 2024-07-17T08:44:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.410+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c90017070 con 0x7f9c9815b1a0 2024-07-17T08:44:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.410+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90011cc0 con 0x7f9c9815b1a0 2024-07-17T08:44:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.410+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c981f2200 con 0x7f9c9815b1a0 2024-07-17T08:44:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.410+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981f27c0 con 0x7f9c9815b1a0 2024-07-17T08:44:26.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.411+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c980b5700 con 0x7f9c9815b1a0 2024-07-17T08:44:26.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.412+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7f9c9001c460 con 0x7f9c9815b1a0 2024-07-17T08:44:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.413+0000 7f9c94c00640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 0x7f9c780793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.413+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f9c90098b70 con 0x7f9c9815b1a0 2024-07-17T08:44:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.413+0000 7f9c9ca00640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 msgr2=0x7f9c780793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:26.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.413+0000 7f9c9ca00640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 0x7f9c780793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:26.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.417+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9c90060ff0 con 0x7f9c9815b1a0 2024-07-17T08:44:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.614+0000 7f9c9ca00640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 msgr2=0x7f9c780793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.614+0000 7f9c9ca00640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 0x7f9c780793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:26.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.669+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c98073560 con 0x7f9c9815b1a0 2024-07-17T08:44:26.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.670+0000 7f9c94c00640 1 -- 172.21.15.44:0/37686512 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7f9c90065540 con 0x7f9c9815b1a0 2024-07-17T08:44:26.670 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:26.671 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:44:26.671 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.672+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 msgr2=0x7f9c780793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.672+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 0x7f9c780793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.672+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 msgr2=0x7f9c980ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.672+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c440 tx=0x7f9c9000c900 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 shutdown_connections 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f9c78076f10 0x7f9c780793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c9815b1a0 0x7f9c980ce5d0 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c98154840 0x7f9c980ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 --2- 172.21.15.44:0/37686512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9c980afb50 0x7f9c981f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 >> 172.21.15.44:0/37686512 conn(0x7f9c98072730 msgr2=0x7f9c9806f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 shutdown_connections 2024-07-17T08:44:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.673+0000 7f9c9f400640 1 -- 172.21.15.44:0/37686512 wait complete. 2024-07-17T08:44:26.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.890+0000 7fde01200640 1 -- 172.21.15.44:0/3252379524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc1522c0 msgr2=0x7fddfc0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.890+0000 7fde01200640 1 --2- 172.21.15.44:0/3252379524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc1522c0 0x7fddfc0ad790 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b1f40 tx=0x7fddec02a560 comp rx=0 tx=0).stop 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 -- 172.21.15.44:0/3252379524 shutdown_connections 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 --2- 172.21.15.44:0/3252379524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc0adcd0 0x7fddfc0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 --2- 172.21.15.44:0/3252379524 >> v2:172.21.15.175:3300/0 conn(0x7fddfc0ac760 0x7fddfc15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 --2- 172.21.15.44:0/3252379524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc1522c0 0x7fddfc0ad790 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 -- 172.21.15.44:0/3252379524 >> 172.21.15.44:0/3252379524 conn(0x7fddfc0b0fb0 msgr2=0x7fddfc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 -- 172.21.15.44:0/3252379524 shutdown_connections 2024-07-17T08:44:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.891+0000 7fde01200640 1 -- 172.21.15.44:0/3252379524 wait complete. 2024-07-17T08:44:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.892+0000 7fde01200640 1 Processor -- start 2024-07-17T08:44:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 -- start start 2024-07-17T08:44:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 0x7fddfc153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc1522c0 0x7fddfc153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15f760 con 0x7fddfc0ac760 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15f5e0 con 0x7fddfc0adcd0 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.893+0000 7fde01200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc15f8e0 con 0x7fddfc1522c0 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51386/0 (socket says 172.21.15.44:51386) 2024-07-17T08:44:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 -- 172.21.15.44:0/3910332060 learned_addr learned my addr 172.21.15.44:0/3910332060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddfa800640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc1522c0 0x7fddfc153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf3e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 0x7fddfc153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc1522c0 msgr2=0x7fddfc153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc1522c0 0x7fddfc153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 msgr2=0x7fddfc153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 0x7fddfc153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf9e00640 1 -- 172.21.15.44:0/3910332060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc14f370 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.894+0000 7fddf3e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 0x7fddfc153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fddf9e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7fddec034040 tx=0x7fddec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec03e030 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddec0047a0 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec035040 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc14f650 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.895+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc14fc10 con 0x7fddfc0ac760 2024-07-17T08:44:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.896+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc0ac220 con 0x7fddfc0ac760 2024-07-17T08:44:26.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.898+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7fddec041030 con 0x7fddfc0ac760 2024-07-17T08:44:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.898+0000 7fddf2000640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 0x7fddd80793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.899+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7fddec0b9680 con 0x7fddfc0ac760 2024-07-17T08:44:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.899+0000 7fddf3e00640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 msgr2=0x7fddd80793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:26.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.899+0000 7fddf3e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 0x7fddd80793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:26.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:26.904+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fddec082040 con 0x7fddfc0ac760 2024-07-17T08:44:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.101+0000 7fddf3e00640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 msgr2=0x7fddd80793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:27.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.101+0000 7fddf3e00640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 0x7fddd80793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:27.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.156+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddfc15b560 con 0x7fddfc0ac760 2024-07-17T08:44:27.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.157+0000 7fddf2000640 1 -- 172.21.15.44:0/3910332060 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7fddec08a280 con 0x7fddfc0ac760 2024-07-17T08:44:27.158 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:27.158 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:44:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 msgr2=0x7fddd80793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 0x7fddd80793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 msgr2=0x7fddfc1503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7fddec034040 tx=0x7fddec0039e0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 shutdown_connections 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fddd8076f10 0x7fddd80793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddfc1522c0 0x7fddfc153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddfc0adcd0 0x7fddfc153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 --2- 172.21.15.44:0/3910332060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddfc0ac760 0x7fddfc1503f0 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 >> 172.21.15.44:0/3910332060 conn(0x7fddfc0b0fb0 msgr2=0x7fddfc06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 shutdown_connections 2024-07-17T08:44:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.160+0000 7fde01200640 1 -- 172.21.15.44:0/3910332060 wait complete. 2024-07-17T08:44:27.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 -- 172.21.15.44:0/2494005698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 msgr2=0x7f888415ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 --2- 172.21.15.44:0/2494005698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415ee90 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7f8884070e20 tx=0x7f887c02aa30 comp rx=0 tx=0).stop 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 -- 172.21.15.44:0/2494005698 shutdown_connections 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 --2- 172.21.15.44:0/2494005698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415ee90 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 --2- 172.21.15.44:0/2494005698 >> v2:172.21.15.175:3300/0 conn(0x7f88840adcd0 0x7f88840ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 --2- 172.21.15.44:0/2494005698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88841522c0 0x7f88840ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 -- 172.21.15.44:0/2494005698 >> 172.21.15.44:0/2494005698 conn(0x7f88840b0fb0 msgr2=0x7f88840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 -- 172.21.15.44:0/2494005698 shutdown_connections 2024-07-17T08:44:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.380+0000 7f888a800640 1 -- 172.21.15.44:0/2494005698 wait complete. 2024-07-17T08:44:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.382+0000 7f888a800640 1 Processor -- start 2024-07-17T08:44:27.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.382+0000 7f888a800640 1 -- start start 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88840adcd0 0x7f88841f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88841522c0 0x7f88841f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f888415f760 con 0x7f88840ac760 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f888415f5e0 con 0x7f88841522c0 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f888a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f888415f8e0 con 0x7f88840adcd0 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f8883400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f8883400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51398/0 (socket says 172.21.15.44:51398) 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f8883400640 1 -- 172.21.15.44:0/1560625269 learned_addr learned my addr 172.21.15.44:0/1560625269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:27.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f887be00640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88840adcd0 0x7f88841f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.383+0000 7f8883e00640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88841522c0 0x7f88841f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88840adcd0 msgr2=0x7f88841f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88840adcd0 0x7f88841f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88841522c0 msgr2=0x7f88841f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88841522c0 0x7f88841f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 -- 172.21.15.44:0/1560625269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8884152930 con 0x7f88840ac760 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f8883400640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f887400c9a0 tx=0x7f887400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874007bb0 con 0x7f88840ac760 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8874007e40 con 0x7f88840ac760 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874005280 con 0x7f88840ac760 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8884152c10 con 0x7f88840ac760 2024-07-17T08:44:27.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.384+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8884155560 con 0x7f88840ac760 2024-07-17T08:44:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.386+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88840cbcb0 con 0x7f88840ac760 2024-07-17T08:44:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.387+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7f8874005420 con 0x7f88840ac760 2024-07-17T08:44:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.388+0000 7f887b400640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 0x7f88600793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.388+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f8874099370 con 0x7f88840ac760 2024-07-17T08:44:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.388+0000 7f887be00640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 msgr2=0x7f88600793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.388+0000 7f887be00640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 0x7f88600793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.392+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8874014030 con 0x7f88840ac760 2024-07-17T08:44:27.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.590+0000 7f887be00640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 msgr2=0x7f88600793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:27.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.590+0000 7f887be00640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 0x7f88600793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.604+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88840b0c00 con 0x7f88840ac760 2024-07-17T08:44:27.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.607+0000 7f887b400640 1 -- 172.21.15.44:0/1560625269 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v338) ==== 74+0+18492 (secure 0 0 0) 0x7f8874061d80 con 0x7f88840ac760 2024-07-17T08:44:27.608 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:27.608 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":338,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:04.885991+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":45,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:46.606384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":46,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:47.649642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"335","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"46.4","mappings":[{"from":7,"to":4}]},{"pgid":"46.c","mappings":[{"from":6,"to":4}]},{"pgid":"46.22","mappings":[{"from":6,"to":5}]},{"pgid":"46.25","mappings":[{"from":2,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/19909624":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/4141702117":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/723525579":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/2709336751":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/1419861374":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/1265901122":"2024-07-18T08:43:03.584378+0000","172.21.15.175:6837/1233420839":"2024-07-18T08:43:03.584378+0000","172.21.15.175:6834/1233420839":"2024-07-18T08:43:03.584378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 msgr2=0x7f88600793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 0x7f88600793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 msgr2=0x7f888415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f887400c9a0 tx=0x7f887400ce60 comp rx=0 tx=0).stop 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 shutdown_connections 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f8860076f10 0x7f88600793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88841522c0 0x7f88841f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88840adcd0 0x7f88841f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.609+0000 7f888a800640 1 --2- 172.21.15.44:0/1560625269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88840ac760 0x7f888415a480 unknown :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.610+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 >> 172.21.15.44:0/1560625269 conn(0x7f88840b0fb0 msgr2=0x7f88840cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.610+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 shutdown_connections 2024-07-17T08:44:27.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.610+0000 7f888a800640 1 -- 172.21.15.44:0/1560625269 wait complete. 2024-07-17T08:44:27.628 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:44:27.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 -- 172.21.15.44:0/1090318575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a5006ea50 msgr2=0x7f3a500ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 --2- 172.21.15.44:0/1090318575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a5006ea50 0x7f3a500ac220 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005fd0 tx=0x7f3a3c02a4f0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 -- 172.21.15.44:0/1090318575 shutdown_connections 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 --2- 172.21.15.44:0/1090318575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a5006ea50 0x7f3a500ac220 unknown :-1 s=CLOSED pgs=3104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 --2- 172.21.15.44:0/1090318575 >> v2:172.21.15.175:3300/0 conn(0x7f3a500ae810 0x7f3a5015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 -- 172.21.15.44:0/1090318575 >> 172.21.15.44:0/1090318575 conn(0x7f3a500b0fb0 msgr2=0x7f3a500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 -- 172.21.15.44:0/1090318575 shutdown_connections 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.823+0000 7f3a57600640 1 -- 172.21.15.44:0/1090318575 wait complete. 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.825+0000 7f3a57600640 1 Processor -- start 2024-07-17T08:44:27.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.825+0000 7f3a57600640 1 -- start start 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a5006ea50 0x7f3a50153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a5006eec0 0x7f3a5009a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500b2e60 con 0x7f3a500ae810 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500b2ce0 con 0x7f3a5006eec0 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a57600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a500b2fe0 con 0x7f3a5006ea50 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a55600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a55600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51414/0 (socket says 172.21.15.44:51414) 2024-07-17T08:44:27.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.826+0000 7f3a55600640 1 -- 172.21.15.44:0/1341044629 learned_addr learned my addr 172.21.15.44:0/1341044629 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a54c00640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a5006ea50 0x7f3a50153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a4fe00640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a5006eec0 0x7f3a5009a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a5006ea50 msgr2=0x7f3a50153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a5006ea50 0x7f3a50153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a5006eec0 msgr2=0x7f3a5009a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a5006eec0 0x7f3a5009a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 -- 172.21.15.44:0/1341044629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a500994d0 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a55600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c7b0 tx=0x7f3a4800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48017070 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a48005070 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a500997b0 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800ce60 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.827+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a50099d70 con 0x7f3a500ae810 2024-07-17T08:44:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.828+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a500ac940 con 0x7f3a500ae810 2024-07-17T08:44:27.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.830+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7f3a4801a070 con 0x7f3a500ae810 2024-07-17T08:44:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.831+0000 7f3a4cc00640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 0x7f3a300793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.831+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f3a480987f0 con 0x7f3a500ae810 2024-07-17T08:44:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.831+0000 7f3a54c00640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 msgr2=0x7f3a300793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.831+0000 7f3a54c00640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 0x7f3a300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:27.835+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3a48021020 con 0x7f3a500ae810 2024-07-17T08:44:28.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.032+0000 7f3a54c00640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 msgr2=0x7f3a300793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:28.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.032+0000 7f3a54c00640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 0x7f3a300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.081+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a500b23b0 con 0x7f3a500ae810 2024-07-17T08:44:28.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.082+0000 7f3a4cc00640 1 -- 172.21.15.44:0/1341044629 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7f3a480613d0 con 0x7f3a500ae810 2024-07-17T08:44:28.083 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:28.083 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:44:28.083 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.084+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 msgr2=0x7f3a300793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 0x7f3a300793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 msgr2=0x7f3a50072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c7b0 tx=0x7f3a4800cc70 comp rx=0 tx=0).stop 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 shutdown_connections 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f3a30076f10 0x7f3a300793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a500ae810 0x7f3a50072c10 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a5006eec0 0x7f3a5009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 --2- 172.21.15.44:0/1341044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a5006ea50 0x7f3a50153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 >> 172.21.15.44:0/1341044629 conn(0x7f3a500b0fb0 msgr2=0x7f3a50161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 shutdown_connections 2024-07-17T08:44:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.085+0000 7f3a57600640 1 -- 172.21.15.44:0/1341044629 wait complete. 2024-07-17T08:44:28.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 -- 172.21.15.44:0/3006269413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 msgr2=0x7fc19c15daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 --2- 172.21.15.44:0/3006269413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c15daa0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0cf9a0 tx=0x7fc18c02a560 comp rx=0 tx=0).stop 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 -- 172.21.15.44:0/3006269413 shutdown_connections 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 --2- 172.21.15.44:0/3006269413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 0x7fc19c15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 --2- 172.21.15.44:0/3006269413 >> v2:172.21.15.175:3300/0 conn(0x7fc19c15dfe0 0x7fc19c15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 --2- 172.21.15.44:0/3006269413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c15daa0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.300+0000 7fc1a1400640 1 -- 172.21.15.44:0/3006269413 >> 172.21.15.44:0/3006269413 conn(0x7fc19c06ce80 msgr2=0x7fc19c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.301+0000 7fc1a1400640 1 -- 172.21.15.44:0/3006269413 shutdown_connections 2024-07-17T08:44:28.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.301+0000 7fc1a1400640 1 -- 172.21.15.44:0/3006269413 wait complete. 2024-07-17T08:44:28.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.302+0000 7fc1a1400640 1 Processor -- start 2024-07-17T08:44:28.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.302+0000 7fc1a1400640 1 -- start start 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 0x7fc19c137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c15dfe0 0x7fc19c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c06f140 con 0x7fc19c06e610 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c06efc0 con 0x7fc19c15dfe0 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.303+0000 7fc1a1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c06f2c0 con 0x7fc19c15aa60 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51438/0 (socket says 172.21.15.44:51438) 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 -- 172.21.15.44:0/1892591253 learned_addr learned my addr 172.21.15.44:0/1892591253 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:28.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19b400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 0x7fc19c137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc1a0a00640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c15dfe0 0x7fc19c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 msgr2=0x7fc19c137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 0x7fc19c137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c15dfe0 msgr2=0x7fc19c0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c15dfe0 0x7fc19c0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.304+0000 7fc19be00640 1 -- 172.21.15.44:0/1892591253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c1f7460 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc19be00640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c03f070 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc18c0047a0 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c035030 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c1f7740 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.305+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c0af4e0 con 0x7fc19c06e610 2024-07-17T08:44:28.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.307+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c0cca20 con 0x7fc19c06e610 2024-07-17T08:44:28.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.307+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7fc18c042070 con 0x7fc19c06e610 2024-07-17T08:44:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.308+0000 7fc193e00640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 0x7fc17c0793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.309+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7fc18c0b9d50 con 0x7fc19c06e610 2024-07-17T08:44:28.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.309+0000 7fc19b400640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 msgr2=0x7fc17c0793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:28.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.309+0000 7fc19b400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 0x7fc17c0793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:28.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.313+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc19c0cca20 con 0x7fc19c06e610 2024-07-17T08:44:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.511+0000 7fc19b400640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 msgr2=0x7fc17c0793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:28.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.511+0000 7fc19b400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 0x7fc17c0793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:28.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.562+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19c06f770 con 0x7fc19c06e610 2024-07-17T08:44:28.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.564+0000 7fc193e00640 1 -- 172.21.15.44:0/1892591253 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+5246 (secure 0 0 0) 0x7fc18c082730 con 0x7fc19c06e610 2024-07-17T08:44:28.565 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:28.565 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":191,"btime":"2024-07-17T08:42:52:256212+0000","default_fscid":22,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:42:48.668598+0000","modified":"2024-07-17T08:42:51.771429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13491":{"gid":13491,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/4048233303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":4048233303},{"type":"v1","addr":"172.21.15.44:6835","nonce":4048233303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[46],"metadata_pool":45,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13491,"qdb_cluster":[13491]},"id":22}]} 2024-07-17T08:44:28.565 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 191 2024-07-17T08:44:28.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.566+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 msgr2=0x7fc17c0793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.566+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 0x7fc17c0793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.566+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 msgr2=0x7fc19c0aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.566+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c003910 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 shutdown_connections 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fc17c076f10 0x7fc17c0793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc19c15dfe0 0x7fc19c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc19c15aa60 0x7fc19c137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 --2- 172.21.15.44:0/1892591253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc19c06e610 0x7fc19c0aee00 unknown :-1 s=CLOSED pgs=3107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 >> 172.21.15.44:0/1892591253 conn(0x7fc19c06ce80 msgr2=0x7fc19c1433f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 shutdown_connections 2024-07-17T08:44:28.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.567+0000 7fc1a1400640 1 -- 172.21.15.44:0/1892591253 wait complete. 2024-07-17T08:44:28.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 -- 172.21.15.44:0/2236006935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 msgr2=0x7f61b406c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 --2- 172.21.15.44:0/2236006935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b406c320 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005fd0 tx=0x7f61a802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 -- 172.21.15.44:0/2236006935 shutdown_connections 2024-07-17T08:44:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 --2- 172.21.15.44:0/2236006935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 0x7f61b40ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 --2- 172.21.15.44:0/2236006935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b406c320 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 -- 172.21.15.44:0/2236006935 >> 172.21.15.44:0/2236006935 conn(0x7f61b40b2240 msgr2=0x7f61b40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 -- 172.21.15.44:0/2236006935 shutdown_connections 2024-07-17T08:44:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.781+0000 7f61bb600640 1 -- 172.21.15.44:0/2236006935 wait complete. 2024-07-17T08:44:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.783+0000 7f61bb600640 1 Processor -- start 2024-07-17T08:44:28.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.783+0000 7f61bb600640 1 -- start start 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 0x7f61b41f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 0x7f61b41f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4140880 con 0x7f61b406bf40 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4140700 con 0x7f61b4140b70 2024-07-17T08:44:28.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61bb600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61b40ae3b0 con 0x7f61b40ad9e0 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51452/0 (socket says 172.21.15.44:51452) 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 -- 172.21.15.44:0/2757772212 learned_addr learned my addr 172.21.15.44:0/2757772212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b3e00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 0x7f61b41f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b9600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 0x7f61b41f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 msgr2=0x7f61b41f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 0x7f61b41f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 msgr2=0x7f61b41f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 0x7f61b41f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b8c00640 1 -- 172.21.15.44:0/2757772212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b413e640 con 0x7f61b406bf40 2024-07-17T08:44:28.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.784+0000 7f61b9600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 0x7f61b41f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61b8c00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009850 tx=0x7f61a80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a803f070 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61a80047a0 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8035030 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b41f76d0 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.785+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b41f7c90 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.787+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b406c320 con 0x7f61b406bf40 2024-07-17T08:44:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.787+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7f61a8042070 con 0x7f61b406bf40 2024-07-17T08:44:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.789+0000 7f61b0c00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 0x7f61940793d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.789+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(338..338 src has 1..338) ==== 7165+0+0 (secure 0 0 0) 0x7f61a80b9c50 con 0x7f61b406bf40 2024-07-17T08:44:28.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.789+0000 7f61b3e00640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 msgr2=0x7f61940793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:28.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.789+0000 7f61b3e00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 0x7f61940793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:28.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.793+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f61a8082630 con 0x7f61b406bf40 2024-07-17T08:44:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.991+0000 7f61b3e00640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 msgr2=0x7f61940793d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:28.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:28.991+0000 7f61b3e00640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 0x7f61940793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:29.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.011+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61b40b6610 con 0x7f61b406bf40 2024-07-17T08:44:29.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.014+0000 7f61b0c00640 1 -- 172.21.15.44:0/2757772212 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v338) ==== 74+0+18492 (secure 0 0 0) 0x7f61a8081f30 con 0x7f61b406bf40 2024-07-17T08:44:29.015 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:29.016 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":338,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:04.885991+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":45,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:42:46.606384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"334","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":46,"pool_name":"cephfs_data","create_time":"2024-07-17T08:42:47.649642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"335","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"46.4","mappings":[{"from":7,"to":4}]},{"pgid":"46.c","mappings":[{"from":6,"to":4}]},{"pgid":"46.22","mappings":[{"from":6,"to":5}]},{"pgid":"46.25","mappings":[{"from":2,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/19909624":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/4141702117":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/723525579":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/2709336751":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/1419861374":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/1265901122":"2024-07-18T08:43:03.584378+0000","172.21.15.175:6837/1233420839":"2024-07-18T08:43:03.584378+0000","172.21.15.175:6834/1233420839":"2024-07-18T08:43:03.584378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 msgr2=0x7f61940793d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 0x7f61940793d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 msgr2=0x7f61b41f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009850 tx=0x7f61a80038a0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 shutdown_connections 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7f6194076f10 0x7f61940793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61b4140b70 0x7f61b41f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61b40ad9e0 0x7f61b41f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 --2- 172.21.15.44:0/2757772212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61b406bf40 0x7f61b41f4e30 unknown :-1 s=CLOSED pgs=3110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 >> 172.21.15.44:0/2757772212 conn(0x7f61b40b2240 msgr2=0x7f61b415f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 shutdown_connections 2024-07-17T08:44:29.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.017+0000 7f61bb600640 1 -- 172.21.15.44:0/2757772212 wait complete. 2024-07-17T08:44:29.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 -- 172.21.15.44:0/839075112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 msgr2=0x7fcc1c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 --2- 172.21.15.44:0/839075112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15d510 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005fd0 tx=0x7fcc0c02a540 comp rx=0 tx=0).stop 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 -- 172.21.15.44:0/839075112 shutdown_connections 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 --2- 172.21.15.44:0/839075112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 0x7fcc1c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 --2- 172.21.15.44:0/839075112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15d510 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 -- 172.21.15.44:0/839075112 >> 172.21.15.44:0/839075112 conn(0x7fcc1c0b4ff0 msgr2=0x7fcc1c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 -- 172.21.15.44:0/839075112 shutdown_connections 2024-07-17T08:44:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.231+0000 7fcc1be00640 1 -- 172.21.15.44:0/839075112 wait complete. 2024-07-17T08:44:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.232+0000 7fcc1be00640 1 Processor -- start 2024-07-17T08:44:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 -- start start 2024-07-17T08:44:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 0x7fcc1c1f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d1a10 con 0x7fcc1c0ce690 2024-07-17T08:44:29.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d1890 con 0x7fcc1c15d130 2024-07-17T08:44:29.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.233+0000 7fcc1be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d1b90 con 0x7fcc1c161be0 2024-07-17T08:44:29.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:29.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45280/0 (socket says 172.21.15.44:45280) 2024-07-17T08:44:29.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3324617557 learned_addr learned my addr 172.21.15.44:0/3324617557 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc13e00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1b400640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 0x7fcc1c1f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 msgr2=0x7fcc1c1f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 0x7fcc1c1f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 msgr2=0x7fcc1c15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.234+0000 7fcc1aa00640 1 -- 172.21.15.44:0/3324617557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c159e70 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc1aa00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009850 tx=0x7fcc0c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c03f070 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc0c0047a0 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c035030 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1f44b0 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.235+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1f4a70 con 0x7fcc1c0ce690 2024-07-17T08:44:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.236+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c1f42c0 con 0x7fcc1c0ce690 2024-07-17T08:44:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.237+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 206) ==== 99147+0+0 (secure 0 0 0) 0x7fcc0c042070 con 0x7fcc1c0ce690 2024-07-17T08:44:29.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.239+0000 7fcc12a00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 0x7fcbf8079400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.239+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(339..339 src has 1..339) ==== 7221+0+0 (secure 0 0 0) 0x7fcc0c0b9c50 con 0x7fcc1c0ce690 2024-07-17T08:44:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.239+0000 7fcc13e00640 1 -- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 msgr2=0x7fcbf8079400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/64945748 2024-07-17T08:44:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.239+0000 7fcc13e00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 0x7fcbf8079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:29.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.243+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcc0c082660 con 0x7fcc1c0ce690 2024-07-17T08:44:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.319+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 207) ==== 98540+0+0 (secure 0 0 0) 0x7fcc0c07e1f0 con 0x7fcc1c0ce690 2024-07-17T08:44:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.320+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 msgr2=0x7fcbf8079400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.320+0000 7fcc12a00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 0x7fcbf8079400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.434 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.434 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.434 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.434 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.434 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:29.432+0000 7fe97b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:29.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.546+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcc1c0b2200 con 0x7fcc1c0ce690 2024-07-17T08:44:29.547 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:44:29.547+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:44:29.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.551+0000 7fcc12a00640 1 -- 172.21.15.44:0/3324617557 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v192) ==== 161+0+0 (secure 0 0 0) 0x7fcc0c0b2020 con 0x7fcc1c0ce690 2024-07-17T08:44:29.552 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:44:29.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 msgr2=0x7fcc1c15f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009850 tx=0x7fcc0c0038a0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 shutdown_connections 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:6834/64945748,v1:172.21.15.175:6837/64945748] conn(0x7fcbf8076f40 0x7fcbf8079400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc1c161be0 0x7fcc1c1f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc1c15d130 0x7fcc1c15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 --2- 172.21.15.44:0/3324617557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c15f040 secure :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009850 tx=0x7fcc0c0038a0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.555+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 >> 172.21.15.44:0/3324617557 conn(0x7fcc1c0b4ff0 msgr2=0x7fcc1c0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.556+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 shutdown_connections 2024-07-17T08:44:29.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.556+0000 7fcc1be00640 1 -- 172.21.15.44:0/3324617557 wait complete. 2024-07-17T08:44:29.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:44:29.632 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:44:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 -- 172.21.15.44:0/1648993226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8960152220 msgr2=0x7f896006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 --2- 172.21.15.44:0/1648993226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8960152220 0x7f896006d620 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f8950009830 tx=0x7f895002a480 comp rx=0 tx=0).stop 2024-07-17T08:44:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 -- 172.21.15.44:0/1648993226 shutdown_connections 2024-07-17T08:44:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 --2- 172.21.15.44:0/1648993226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8960152220 0x7f896006d620 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 -- 172.21.15.44:0/1648993226 >> 172.21.15.44:0/1648993226 conn(0x7f89600b0eb0 msgr2=0x7f89600b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 -- 172.21.15.44:0/1648993226 shutdown_connections 2024-07-17T08:44:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.763+0000 7f8967200640 1 -- 172.21.15.44:0/1648993226 wait complete. 2024-07-17T08:44:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.765+0000 7f8967200640 1 Processor -- start 2024-07-17T08:44:29.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.765+0000 7f8967200640 1 -- start start 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f896006ed30 0x7f8960155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8960152220 0x7f8960147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f89600cd9f0 con 0x7f89600cbab0 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f896015c800 con 0x7f8960152220 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8967200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89600cdb70 con 0x7f896006ed30 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45296/0 (socket says 172.21.15.44:45296) 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 -- 172.21.15.44:0/1111320006 learned_addr learned my addr 172.21.15.44:0/1111320006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:29.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 -- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f896006ed30 msgr2=0x7f8960155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f896006ed30 0x7f8960155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 -- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8960152220 msgr2=0x7f8960147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8960152220 0x7f8960147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.766+0000 7f8957e00640 1 -- 172.21.15.44:0/1111320006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89600ad0e0 con 0x7f89600cbab0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f8957e00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f894800c500 tx=0x7f894800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948018070 con 0x7f89600cbab0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89480035f0 con 0x7f89600cbab0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948003ac0 con 0x7f89600cbab0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89600ad380 con 0x7f89600cbab0 2024-07-17T08:44:29.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.767+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89600ad940 con 0x7f89600cbab0 2024-07-17T08:44:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.768+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89600ce090 con 0x7f89600cbab0 2024-07-17T08:44:29.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.769+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 207) ==== 98540+0+0 (secure 0 0 0) 0x7f8948003c60 con 0x7f89600cbab0 2024-07-17T08:44:29.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.771+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(341..341 src has 1..341) ==== 7795+0+0 (secure 0 0 0) 0x7f8948097ba0 con 0x7f89600cbab0 2024-07-17T08:44:29.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:29.775+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8948060b20 con 0x7f89600cbab0 2024-07-17T08:44:30.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.016+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f89600b3140 con 0x7f89600cbab0 2024-07-17T08:44:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.323+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 208) ==== 98667+0+0 (secure 0 0 0) 0x7f8948067d10 con 0x7f89600cbab0 2024-07-17T08:44:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.324+0000 7f895cc00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 0x7f893c07f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.326+0000 7f895fe00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 0x7f893c07f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:30.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.334+0000 7f895fe00640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 0x7f893c07f970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8950034000 tx=0x7f89500046c0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:30.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.554+0000 7f895cc00640 1 -- 172.21.15.44:0/1111320006 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v193) ==== 104+0+0 (secure 0 0 0) 0x7f894801b360 con 0x7f89600cbab0 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 msgr2=0x7f893c07f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 0x7f893c07f970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8950034000 tx=0x7f89500046c0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 msgr2=0x7f89600acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f894800c500 tx=0x7f894800c9c0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 shutdown_connections 2024-07-17T08:44:30.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f893c07d580 0x7f893c07f970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8960152220 0x7f8960147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f89600cbab0 0x7f89600acba0 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 --2- 172.21.15.44:0/1111320006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f896006ed30 0x7f8960155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 >> 172.21.15.44:0/1111320006 conn(0x7f89600b0eb0 msgr2=0x7f89600b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.558+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 shutdown_connections 2024-07-17T08:44:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.559+0000 7f8967200640 1 -- 172.21.15.44:0/1111320006 wait complete. 2024-07-17T08:44:30.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:44:30.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 -- 172.21.15.44:0/153976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4024161b50 msgr2=0x7f40240d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:30.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 --2- 172.21.15.44:0/153976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4024161b50 0x7f40240d0430 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7f4010009830 tx=0x7f401002a3f0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 -- 172.21.15.44:0/153976181 shutdown_connections 2024-07-17T08:44:30.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 --2- 172.21.15.44:0/153976181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4024161b50 0x7f40240d0430 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 -- 172.21.15.44:0/153976181 >> 172.21.15.44:0/153976181 conn(0x7f4024073a60 msgr2=0x7f4024073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:30.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 -- 172.21.15.44:0/153976181 shutdown_connections 2024-07-17T08:44:30.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.839+0000 7f4028e00640 1 -- 172.21.15.44:0/153976181 wait complete. 2024-07-17T08:44:30.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.841+0000 7f4028e00640 1 Processor -- start 2024-07-17T08:44:30.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.841+0000 7f4028e00640 1 -- start start 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402415e370 0x7f40240c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4024161b50 0x7f40240c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40240b8f00 con 0x7f40240d0970 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40240b8d80 con 0x7f4024161b50 2024-07-17T08:44:30.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4028e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40240b9080 con 0x7f402415e370 2024-07-17T08:44:30.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.842+0000 7f4023400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:30.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45314/0 (socket says 172.21.15.44:45314) 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 -- 172.21.15.44:0/641695038 learned_addr learned my addr 172.21.15.44:0/641695038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4024161b50 0x7f40240c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4022a00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402415e370 0x7f40240c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 -- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402415e370 msgr2=0x7f40240c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402415e370 0x7f40240c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 -- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4024161b50 msgr2=0x7f40240c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4024161b50 0x7f40240c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 -- 172.21.15.44:0/641695038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40240c8530 con 0x7f40240d0970 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.843+0000 7f4023400640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f401400c460 tx=0x7f401400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.844+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014016020 con 0x7f40240d0970 2024-07-17T08:44:30.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.844+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4014012960 con 0x7f40240d0970 2024-07-17T08:44:30.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.844+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014012db0 con 0x7f40240d0970 2024-07-17T08:44:30.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.844+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40240b5ba0 con 0x7f40240d0970 2024-07-17T08:44:30.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.844+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40240c8250 con 0x7f40240d0970 2024-07-17T08:44:30.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.845+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4024050ec0 con 0x7f40240d0970 2024-07-17T08:44:30.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.846+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 208) ==== 98667+0+0 (secure 0 0 0) 0x7f4014019020 con 0x7f40240d0970 2024-07-17T08:44:30.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.848+0000 7f401b400640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 0x7f4004078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:30.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.848+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(341..341 src has 1..341) ==== 7795+0+0 (secure 0 0 0) 0x7f4014098350 con 0x7f40240d0970 2024-07-17T08:44:30.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.848+0000 7f4022a00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 0x7f4004078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:30.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.850+0000 7f4022a00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 0x7f4004078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4010034000 tx=0x7f4010004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:30.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:30.852+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4024050ec0 con 0x7f40240d0970 2024-07-17T08:44:31.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.051+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f40240d0430 con 0x7f40240d0970 2024-07-17T08:44:31.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.573+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 209) ==== 98875+0+0 (secure 0 0 0) 0x7f401405cdf0 con 0x7f40240d0970 2024-07-17T08:44:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.581+0000 7f401b400640 1 -- 172.21.15.44:0/641695038 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v342) ==== 181+0+0 (secure 0 0 0) 0x7f4014020070 con 0x7f40240d0970 2024-07-17T08:44:31.582 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:44:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 msgr2=0x7f4004078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 0x7f4004078ec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4010034000 tx=0x7f4010004730 comp rx=0 tx=0).stop 2024-07-17T08:44:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 msgr2=0x7f4024157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f401400c460 tx=0x7f401400c920 comp rx=0 tx=0).stop 2024-07-17T08:44:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 shutdown_connections 2024-07-17T08:44:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4004076a00 0x7f4004078ec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4024161b50 0x7f40240c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f402415e370 0x7f40240c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.586+0000 7f4028e00640 1 --2- 172.21.15.44:0/641695038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40240d0970 0x7f4024157c30 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.587+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 >> 172.21.15.44:0/641695038 conn(0x7f4024073a60 msgr2=0x7f402415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:31.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.588+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 shutdown_connections 2024-07-17T08:44:31.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.589+0000 7f4028e00640 1 -- 172.21.15.44:0/641695038 wait complete. 2024-07-17T08:44:31.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:44:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 -- 172.21.15.44:0/1421470718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612015a5d0 msgr2=0x7f61200ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 --2- 172.21.15.44:0/1421470718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612015a5d0 0x7f61200ac3a0 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7f6120070850 tx=0x7f610802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 -- 172.21.15.44:0/1421470718 shutdown_connections 2024-07-17T08:44:31.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 --2- 172.21.15.44:0/1421470718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f612015a5d0 0x7f61200ac3a0 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 --2- 172.21.15.44:0/1421470718 >> v2:172.21.15.175:3300/0 conn(0x7f61201539f0 0x7f612006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 -- 172.21.15.44:0/1421470718 >> 172.21.15.44:0/1421470718 conn(0x7f61200b0eb0 msgr2=0x7f61200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:31.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 -- 172.21.15.44:0/1421470718 shutdown_connections 2024-07-17T08:44:31.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.824+0000 7f6126200640 1 -- 172.21.15.44:0/1421470718 wait complete. 2024-07-17T08:44:31.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.826+0000 7f6126200640 1 Processor -- start 2024-07-17T08:44:31.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.826+0000 7f6126200640 1 -- start start 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.827+0000 7f6126200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ac8e0 0x7f6120150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.827+0000 7f6126200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f6126200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f612015a5d0 0x7f61201413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f6126200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f612006dce0 con 0x7f61201539f0 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f6126200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f612006db60 con 0x7f61200ac8e0 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f6126200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f612006f6d0 con 0x7f612015a5d0 2024-07-17T08:44:31.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45342/0 (socket says 172.21.15.44:45342) 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 -- 172.21.15.44:0/3288045257 learned_addr learned my addr 172.21.15.44:0/3288045257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611f400640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ac8e0 0x7f6120150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611fe00640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f612015a5d0 0x7f61201413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 -- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f612015a5d0 msgr2=0x7f61201413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f612015a5d0 0x7f61201413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 -- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ac8e0 msgr2=0x7f6120150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.828+0000 7f611ea00640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ac8e0 0x7f6120150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f611ea00640 1 -- 172.21.15.44:0/3288045257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61201f4b10 con 0x7f61201539f0 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f611ea00640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f6108009b30 tx=0x7f6108003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:31.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108031da0 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6108003e10 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6120074470 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108033310 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.829+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61200cf830 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.831+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61200cfa60 con 0x7f61201539f0 2024-07-17T08:44:31.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.832+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 209) ==== 98875+0+0 (secure 0 0 0) 0x7f61080334b0 con 0x7f61201539f0 2024-07-17T08:44:31.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.833+0000 7f6117400640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 0x7f60fc079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:31.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.833+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(342..342 src has 1..342) ==== 7357+0+0 (secure 0 0 0) 0x7f61080ba0f0 con 0x7f61201539f0 2024-07-17T08:44:31.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.834+0000 7f611f400640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 0x7f60fc079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:31.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.835+0000 7f611f400640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 0x7f60fc079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6110005fd0 tx=0x7f6110005e40 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:31.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:31.837+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f610803f030 con 0x7f61201539f0 2024-07-17T08:44:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.058+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f612006d620 con 0x7f61201539f0 2024-07-17T08:44:32.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.585+0000 7f6117400640 1 -- 172.21.15.44:0/3288045257 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v343) ==== 169+0+0 (secure 0 0 0) 0x7f6108082de0 con 0x7f61201539f0 2024-07-17T08:44:32.586 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:44:32.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.588+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 msgr2=0x7f60fc079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:32.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.588+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 0x7f60fc079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6110005fd0 tx=0x7f6110005e40 comp rx=0 tx=0).stop 2024-07-17T08:44:32.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.588+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 msgr2=0x7f61201f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:32.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.588+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f6108009b30 tx=0x7f6108003c20 comp rx=0 tx=0).stop 2024-07-17T08:44:32.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 shutdown_connections 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f60fc076c40 0x7f60fc079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f612015a5d0 0x7f61201413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61201539f0 0x7f61201f43f0 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 --2- 172.21.15.44:0/3288045257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ac8e0 0x7f6120150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 >> 172.21.15.44:0/3288045257 conn(0x7f61200b0eb0 msgr2=0x7f6120158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 shutdown_connections 2024-07-17T08:44:32.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.589+0000 7f6126200640 1 -- 172.21.15.44:0/3288045257 wait complete. 2024-07-17T08:44:32.623 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.811+0000 7f9331000640 1 -- 172.21.15.44:0/2475073090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310004800 msgr2=0x7f9310004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f9331000640 1 --2- 172.21.15.44:0/2475073090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310004800 0x7f9310004c80 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f9318009b50 tx=0x7f9310005830 comp rx=0 tx=0).stop 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f932a800640 1 -- 172.21.15.44:0/2475073090 reap_dead start 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f9331000640 1 -- 172.21.15.44:0/2475073090 shutdown_connections 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f9331000640 1 -- 172.21.15.44:0/2475073090 >> 172.21.15.44:0/2475073090 conn(0x7f932c0b0eb0 msgr2=0x7f932c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f9331000640 1 -- 172.21.15.44:0/2475073090 shutdown_connections 2024-07-17T08:44:32.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.812+0000 7f9331000640 1 -- 172.21.15.44:0/2475073090 wait complete. 2024-07-17T08:44:32.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.814+0000 7f9331000640 1 Processor -- start 2024-07-17T08:44:32.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.814+0000 7f9331000640 1 -- start start 2024-07-17T08:44:32.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9310004800 0x7f932c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:32.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c15df50 0x7f932c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15c7f0 con 0x7f932c1f5310 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15c670 con 0x7f932c15df50 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9331000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0cb810 con 0x7f9310004800 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f932a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f932a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45370/0 (socket says 172.21.15.44:45370) 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f932a800640 1 -- 172.21.15.44:0/60613758 learned_addr learned my addr 172.21.15.44:0/60613758 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9329e00640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9310004800 0x7f932c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.815+0000 7f9329400640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c15df50 0x7f932c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 -- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9310004800 msgr2=0x7f932c158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9310004800 0x7f932c158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 -- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c15df50 msgr2=0x7f932c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c15df50 0x7f932c15e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 -- 172.21.15.44:0/60613758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1544e0 con 0x7f932c1f5310 2024-07-17T08:44:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f932a800640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f9324001330 tx=0x7f932400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324017070 con 0x7f932c1f5310 2024-07-17T08:44:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9324007070 con 0x7f932c1f5310 2024-07-17T08:44:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93240074a0 con 0x7f932c1f5310 2024-07-17T08:44:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.816+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1547c0 con 0x7f932c1f5310 2024-07-17T08:44:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.817+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c154d80 con 0x7f932c1f5310 2024-07-17T08:44:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.818+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c06d620 con 0x7f932c1f5310 2024-07-17T08:44:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.819+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 209) ==== 98875+0+0 (secure 0 0 0) 0x7f932401a070 con 0x7f932c1f5310 2024-07-17T08:44:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.820+0000 7f9322000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 0x7f930c079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.820+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7f9324098400 con 0x7f932c1f5310 2024-07-17T08:44:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.821+0000 7f9329e00640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 0x7f930c079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:32.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.822+0000 7f9329e00640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 0x7f930c079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f931c009b60 tx=0x7f931c005740 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:32.824+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9324061410 con 0x7f932c1f5310 2024-07-17T08:44:33.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.037+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f932c15d150 con 0x7f932c1f5310 2024-07-17T08:44:33.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.046+0000 7f9322000640 1 -- 172.21.15.44:0/60613758 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v103) ==== 71+0+0 (secure 0 0 0) 0x7f9324060d10 con 0x7f932c1f5310 2024-07-17T08:44:33.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.049+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 msgr2=0x7f930c079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.049+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 0x7f930c079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f931c009b60 tx=0x7f931c005740 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.049+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 msgr2=0x7f932c1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.049+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f9324001330 tx=0x7f932400a430 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 shutdown_connections 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f930c076c40 0x7f930c079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f932c1f5310 0x7f932c1f5790 unknown :-1 s=CLOSED pgs=3128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f932c15df50 0x7f932c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 --2- 172.21.15.44:0/60613758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9310004800 0x7f932c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 >> 172.21.15.44:0/60613758 conn(0x7f932c0b0eb0 msgr2=0x7f932c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 shutdown_connections 2024-07-17T08:44:33.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.050+0000 7f9331000640 1 -- 172.21.15.44:0/60613758 wait complete. 2024-07-17T08:44:33.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05' 2024-07-17T08:44:33.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 -- 172.21.15.44:0/1632549982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae480ac920 msgr2=0x7fae4813ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:33.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 --2- 172.21.15.44:0/1632549982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae480ac920 0x7fae4813ed60 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7fae48070a70 tx=0x7fae4002aa10 comp rx=0 tx=0).stop 2024-07-17T08:44:33.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 -- 172.21.15.44:0/1632549982 shutdown_connections 2024-07-17T08:44:33.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 --2- 172.21.15.44:0/1632549982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae4815a040 0x7fae480ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 --2- 172.21.15.44:0/1632549982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae480ac920 0x7fae4813ed60 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 --2- 172.21.15.44:0/1632549982 >> v2:172.21.15.175:3300/0 conn(0x7fae48153a50 0x7fae48159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 -- 172.21.15.44:0/1632549982 >> 172.21.15.44:0/1632549982 conn(0x7fae480b0fb0 msgr2=0x7fae480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 -- 172.21.15.44:0/1632549982 shutdown_connections 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.276+0000 7fae4e000640 1 -- 172.21.15.44:0/1632549982 wait complete. 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.278+0000 7fae4e000640 1 Processor -- start 2024-07-17T08:44:33.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.278+0000 7fae4e000640 1 -- start start 2024-07-17T08:44:33.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae480ac920 0x7fae48071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:33.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48153a50 0x7fae481f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:33.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae48153e30 con 0x7fae4815a040 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae48153fb0 con 0x7fae48153a50 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.279+0000 7fae4e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae4806f710 con 0x7fae480ac920 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45386/0 (socket says 172.21.15.44:45386) 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 -- 172.21.15.44:0/3896664628 learned_addr learned my addr 172.21.15.44:0/3896664628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae46a00640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48153a50 0x7fae481f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:33.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47400640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae480ac920 0x7fae48071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 -- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae480ac920 msgr2=0x7fae48071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae480ac920 0x7fae48071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 -- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48153a50 msgr2=0x7fae481f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48153a50 0x7fae481f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.280+0000 7fae47e00640 1 -- 172.21.15.44:0/3896664628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae481e4bc0 con 0x7fae4815a040 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae47e00640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fae4002af30 tx=0x7fae40039a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:33.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40046070 con 0x7fae4815a040 2024-07-17T08:44:33.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae40004050 con 0x7fae4815a040 2024-07-17T08:44:33.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae481f1cc0 con 0x7fae4815a040 2024-07-17T08:44:33.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40033040 con 0x7fae4815a040 2024-07-17T08:44:33.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.281+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae480afe80 con 0x7fae4815a040 2024-07-17T08:44:33.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.282+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae48159b00 con 0x7fae4815a040 2024-07-17T08:44:33.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.283+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 209) ==== 98875+0+0 (secure 0 0 0) 0x7fae40049070 con 0x7fae4815a040 2024-07-17T08:44:33.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.284+0000 7fae3f400640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 0x7fae28079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:33.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.284+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7fae400b9e20 con 0x7fae4815a040 2024-07-17T08:44:33.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.285+0000 7fae47400640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 0x7fae28079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:33.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.286+0000 7fae47400640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 0x7fae28079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae380058c0 tx=0x7fae3800a4c0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:33.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.289+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fae4003e070 con 0x7fae4815a040 2024-07-17T08:44:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:33.587+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05"]} v 0) -- 0x7fae4815be80 con 0x7fae4815a040 2024-07-17T08:44:34.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.064+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fae400bf020 con 0x7fae4815a040 2024-07-17T08:44:34.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.591+0000 7fae3f400640 1 -- 172.21.15.44:0/3896664628 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_05"]}]=0 v3690) ==== 201+0+0 (secure 0 0 0) 0x7fae40082c90 con 0x7fae4815a040 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.594+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 msgr2=0x7fae28079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 0x7fae28079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae380058c0 tx=0x7fae3800a4c0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 msgr2=0x7fae480b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fae4002af30 tx=0x7fae40039a80 comp rx=0 tx=0).stop 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 shutdown_connections 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fae28076c40 0x7fae28079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae4815a040 0x7fae480b3300 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48153a50 0x7fae481f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 --2- 172.21.15.44:0/3896664628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae480ac920 0x7fae48071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 >> 172.21.15.44:0/3896664628 conn(0x7fae480b0fb0 msgr2=0x7fae4806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 shutdown_connections 2024-07-17T08:44:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.595+0000 7fae4e000640 1 -- 172.21.15.44:0/3896664628 wait complete. 2024-07-17T08:44:34.619 INFO:tasks.cephfs_test_runner:Test subvol schedule creation succeeds for a subvol under a subvolgroup. ... ok 2024-07-17T08:44:34.620 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_06 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule listing fails without a subvolgroup argument. 2024-07-17T08:44:34.620 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_06 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:44:34.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.831+0000 7f1df1a00640 1 -- 172.21.15.44:0/3488645649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec1522c0 msgr2=0x7f1dec0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.831+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3488645649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec1522c0 0x7f1dec0ad790 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc005fd0 tx=0x7f1ddc02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 -- 172.21.15.44:0/3488645649 shutdown_connections 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3488645649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec0adcd0 0x7f1dec0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3488645649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec1522c0 0x7f1dec0ad790 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 -- 172.21.15.44:0/3488645649 >> 172.21.15.44:0/3488645649 conn(0x7f1dec0b0fb0 msgr2=0x7f1dec0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 -- 172.21.15.44:0/3488645649 shutdown_connections 2024-07-17T08:44:34.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.832+0000 7f1df1a00640 1 -- 172.21.15.44:0/3488645649 wait complete. 2024-07-17T08:44:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.833+0000 7f1df1a00640 1 Processor -- start 2024-07-17T08:44:34.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.834+0000 7f1df1a00640 1 -- start start 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 0x7f1dec14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 0x7f1dec1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec15f5a0 con 0x7f1dec0adcd0 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0ce8b0 con 0x7f1dec0ac760 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1df1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1dec15f720 con 0x7f1dec1522c0 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1de3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45388/0 (socket says 172.21.15.44:45388) 2024-07-17T08:44:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.835+0000 7f1de3e00640 1 -- 172.21.15.44:0/3370866290 learned_addr learned my addr 172.21.15.44:0/3370866290 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1deaa00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 0x7f1dec14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1deb400640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 0x7f1dec1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 -- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 msgr2=0x7f1dec1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 0x7f1dec1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 -- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 msgr2=0x7f1dec14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 0x7f1dec14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 -- 172.21.15.44:0/3370866290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec14d2a0 con 0x7f1dec0adcd0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1deb400640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 0x7f1dec1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1deaa00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 0x7f1dec14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de3e00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e9c0 tx=0x7f1dd400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400cd80 con 0x7f1dec0adcd0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd4004020 con 0x7f1dec0adcd0 2024-07-17T08:44:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec0716c0 con 0x7f1dec0adcd0 2024-07-17T08:44:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.836+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4004450 con 0x7f1dec0adcd0 2024-07-17T08:44:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.837+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec071c80 con 0x7f1dec0adcd0 2024-07-17T08:44:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.838+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec15fc40 con 0x7f1dec0adcd0 2024-07-17T08:44:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.839+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f1dd40035f0 con 0x7f1dec0adcd0 2024-07-17T08:44:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.840+0000 7f1de2a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 0x7f1dc8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.840+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7f1dd40994b0 con 0x7f1dec0adcd0 2024-07-17T08:44:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.840+0000 7f1deaa00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 0x7f1dc8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:34.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.842+0000 7f1deaa00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 0x7f1dc8079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc033000 tx=0x7f1ddc004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:34.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:34.844+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1dd4062260 con 0x7f1dec0adcd0 2024-07-17T08:44:35.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.056+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dec0ad780 con 0x7f1dec0adcd0 2024-07-17T08:44:35.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.058+0000 7f1de2a00640 1 -- 172.21.15.44:0/3370866290 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v343) ==== 74+0+15522 (secure 0 0 0) 0x7f1dd4061b60 con 0x7f1dec0adcd0 2024-07-17T08:44:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:35.059 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":343,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:32.581406+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":46,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":333,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":337,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/4048233303":"2024-07-18T08:44:29.547430+0000","172.21.15.44:6834/4048233303":"2024-07-18T08:44:29.547430+0000","172.21.15.175:0/4269616882":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/3068960738":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/716769195":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/3501439643":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/2231609231":"2024-07-18T08:44:29.298357+0000","172.21.15.175:6837/64945748":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/3500941592":"2024-07-18T08:44:29.298357+0000","172.21.15.175:6834/64945748":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/19909624":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/4141702117":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/3232784591":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/723525579":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/2709336751":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/2785128662":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/2646609442":"2024-07-18T08:44:29.298357+0000","172.21.15.175:0/1419861374":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/1265901122":"2024-07-18T08:43:03.584378+0000","172.21.15.175:6837/1233420839":"2024-07-18T08:43:03.584378+0000","172.21.15.175:0/3103573968":"2024-07-18T08:44:29.298357+0000","172.21.15.175:6834/1233420839":"2024-07-18T08:43:03.584378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 msgr2=0x7f1dc8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 0x7f1dc8079190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc033000 tx=0x7f1ddc004730 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 msgr2=0x7f1dec14c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e9c0 tx=0x7f1dd400ee80 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 shutdown_connections 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1dc8076cd0 0x7f1dc8079190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1dec1522c0 0x7f1dec1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dec0adcd0 0x7f1dec14c540 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 --2- 172.21.15.44:0/3370866290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dec0ac760 0x7f1dec14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 >> 172.21.15.44:0/3370866290 conn(0x7f1dec0b0fb0 msgr2=0x7f1dec06e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 shutdown_connections 2024-07-17T08:44:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.061+0000 7f1df1a00640 1 -- 172.21.15.44:0/3370866290 wait complete. 2024-07-17T08:44:35.079 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:44:35.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:44:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 -- 172.21.15.44:0/3888898881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c152220 msgr2=0x7feb7c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 --2- 172.21.15.44:0/3888898881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c152220 0x7feb7c06d620 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c02a540 comp rx=0 tx=0).stop 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 -- 172.21.15.44:0/3888898881 shutdown_connections 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 --2- 172.21.15.44:0/3888898881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c06ed30 0x7feb7c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 --2- 172.21.15.44:0/3888898881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c152220 0x7feb7c06d620 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 -- 172.21.15.44:0/3888898881 >> 172.21.15.44:0/3888898881 conn(0x7feb7c0b0eb0 msgr2=0x7feb7c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 -- 172.21.15.44:0/3888898881 shutdown_connections 2024-07-17T08:44:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.283+0000 7feb81a00640 1 -- 172.21.15.44:0/3888898881 wait complete. 2024-07-17T08:44:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.285+0000 7feb81a00640 1 Processor -- start 2024-07-17T08:44:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.285+0000 7feb81a00640 1 -- start start 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0cb3a0 0x7feb7c0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c152220 0x7feb7c0acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c0cbbd0 con 0x7feb7c06ed30 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c0cba50 con 0x7feb7c152220 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.286+0000 7feb81a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c0cbd50 con 0x7feb7c0cb3a0 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45402/0 (socket says 172.21.15.44:45402) 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 -- 172.21.15.44:0/1969385548 learned_addr learned my addr 172.21.15.44:0/1969385548 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7a000640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0cb3a0 0x7feb7c0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 -- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0cb3a0 msgr2=0x7feb7c0ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0cb3a0 0x7feb7c0ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 -- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c152220 msgr2=0x7feb7c0acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c152220 0x7feb7c0acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.287+0000 7feb7aa00640 1 -- 172.21.15.44:0/1969385548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c153130 con 0x7feb7c06ed30 2024-07-17T08:44:35.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb7aa00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7feb6c034040 tx=0x7feb6c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c03e030 con 0x7feb7c06ed30 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb6c003d90 con 0x7feb7c06ed30 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c035040 con 0x7feb7c06ed30 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c153410 con 0x7feb7c06ed30 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.288+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1539d0 con 0x7feb7c06ed30 2024-07-17T08:44:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.289+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c15f940 con 0x7feb7c06ed30 2024-07-17T08:44:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.291+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7feb6c041030 con 0x7feb7c06ed30 2024-07-17T08:44:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.292+0000 7feb72a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 0x7feb5c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.292+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7feb6c0b9400 con 0x7feb7c06ed30 2024-07-17T08:44:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.293+0000 7feb7a000640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 0x7feb5c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.294+0000 7feb7a000640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 0x7feb5c079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb68005fd0 tx=0x7feb68005850 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:35.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.297+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7feb6c082240 con 0x7feb7c06ed30 2024-07-17T08:44:35.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.509+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb7c068640 con 0x7feb7c06ed30 2024-07-17T08:44:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.510+0000 7feb72a00640 1 -- 172.21.15.44:0/1969385548 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v343) ==== 93+0+10 (secure 0 0 0) 0x7feb6c081b40 con 0x7feb7c06ed30 2024-07-17T08:44:35.511 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:44:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 msgr2=0x7feb5c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 0x7feb5c079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb68005fd0 tx=0x7feb68005850 comp rx=0 tx=0).stop 2024-07-17T08:44:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 msgr2=0x7feb7c071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7feb6c034040 tx=0x7feb6c0038a0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 shutdown_connections 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7feb5c076cd0 0x7feb5c079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c152220 0x7feb7c0acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0cb3a0 0x7feb7c0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 --2- 172.21.15.44:0/1969385548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb7c06ed30 0x7feb7c071d30 unknown :-1 s=CLOSED pgs=3134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.513+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 >> 172.21.15.44:0/1969385548 conn(0x7feb7c0b0eb0 msgr2=0x7feb7c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.514+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 shutdown_connections 2024-07-17T08:44:35.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.514+0000 7feb81a00640 1 -- 172.21.15.44:0/1969385548 wait complete. 2024-07-17T08:44:35.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06' 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.748+0000 7faa44c00640 1 -- 172.21.15.44:0/584532274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4009b300 msgr2=0x7faa4006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.748+0000 7faa44c00640 1 --2- 172.21.15.44:0/584532274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4009b300 0x7faa4006e060 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7faa4009c6a0 tx=0x7faa2802a560 comp rx=0 tx=0).stop 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 -- 172.21.15.44:0/584532274 shutdown_connections 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 --2- 172.21.15.44:0/584532274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b6e0 0x7faa400cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 --2- 172.21.15.44:0/584532274 >> v2:172.21.15.175:3300/0 conn(0x7faa4006f4b0 0x7faa400cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 --2- 172.21.15.44:0/584532274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4009b300 0x7faa4006e060 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 -- 172.21.15.44:0/584532274 >> 172.21.15.44:0/584532274 conn(0x7faa400b0e80 msgr2=0x7faa400b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 -- 172.21.15.44:0/584532274 shutdown_connections 2024-07-17T08:44:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.749+0000 7faa44c00640 1 -- 172.21.15.44:0/584532274 wait complete. 2024-07-17T08:44:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.751+0000 7faa44c00640 1 Processor -- start 2024-07-17T08:44:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.751+0000 7faa44c00640 1 -- start start 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b300 0x7faa401f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4009b6e0 0x7faa401e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faa4015a1f0 con 0x7faa4006f4b0 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa400b1fa0 con 0x7faa4009b6e0 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.752+0000 7faa44c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faa401f41e0 con 0x7faa4009b300 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45426/0 (socket says 172.21.15.44:45426) 2024-07-17T08:44:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 -- 172.21.15.44:0/1016962627 learned_addr learned my addr 172.21.15.44:0/1016962627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3ea00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b300 0x7faa401f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3fe00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4009b6e0 0x7faa401e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 -- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b300 msgr2=0x7faa401f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b300 0x7faa401f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 -- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4009b6e0 msgr2=0x7faa401e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4009b6e0 0x7faa401e83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.753+0000 7faa3f400640 1 -- 172.21.15.44:0/1016962627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa401f7ef0 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa3f400640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7faa2802ab50 tx=0x7faa280039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2803f070 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa28004890 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28035030 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa401f81d0 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.754+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401f84b0 con 0x7faa4006f4b0 2024-07-17T08:44:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.755+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4006e060 con 0x7faa4006f4b0 2024-07-17T08:44:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.757+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7faa28042070 con 0x7faa4006f4b0 2024-07-17T08:44:35.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.759+0000 7faa37400640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 0x7faa1c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:35.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.759+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7faa280b9900 con 0x7faa4006f4b0 2024-07-17T08:44:35.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.759+0000 7faa3ea00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 0x7faa1c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:35.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.761+0000 7faa3ea00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 0x7faa1c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa401f1c30 tx=0x7faa300093c0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:35.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:35.765+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faa280826b0 con 0x7faa4006f4b0 2024-07-17T08:44:36.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.055+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06"]} v 0) -- 0x7faa400af5b0 con 0x7faa4006f4b0 2024-07-17T08:44:36.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.604+0000 7faa37400640 1 -- 172.21.15.44:0/1016962627 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06"]}]=0 v3692) ==== 204+0+0 (secure 0 0 0) 0x7faa28081fb0 con 0x7faa4006f4b0 2024-07-17T08:44:36.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 msgr2=0x7faa1c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:36.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 0x7faa1c079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa401f1c30 tx=0x7faa300093c0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 msgr2=0x7faa401f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7faa2802ab50 tx=0x7faa280039e0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 shutdown_connections 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faa1c076cd0 0x7faa1c079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa4009b6e0 0x7faa401e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa4009b300 0x7faa401f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 --2- 172.21.15.44:0/1016962627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faa4006f4b0 0x7faa401f76c0 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.609+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 >> 172.21.15.44:0/1016962627 conn(0x7faa400b0e80 msgr2=0x7faa40158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.610+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 shutdown_connections 2024-07-17T08:44:36.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.610+0000 7faa44c00640 1 -- 172.21.15.44:0/1016962627 wait complete. 2024-07-17T08:44:36.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:44:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 -- 172.21.15.44:0/230687247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a0161450 msgr2=0x7fd9a00d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 --2- 172.21.15.44:0/230687247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a0161450 0x7fd9a00d1280 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7fd998005fd0 tx=0x7fd99802aa90 comp rx=0 tx=0).stop 2024-07-17T08:44:36.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 -- 172.21.15.44:0/230687247 shutdown_connections 2024-07-17T08:44:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 --2- 172.21.15.44:0/230687247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a0161450 0x7fd9a00d1280 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 -- 172.21.15.44:0/230687247 >> 172.21.15.44:0/230687247 conn(0x7fd9a006c390 msgr2=0x7fd9a006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 -- 172.21.15.44:0/230687247 shutdown_connections 2024-07-17T08:44:36.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.845+0000 7fd99f200640 1 -- 172.21.15.44:0/230687247 wait complete. 2024-07-17T08:44:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.847+0000 7fd99f200640 1 Processor -- start 2024-07-17T08:44:36.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.847+0000 7fd99f200640 1 -- start start 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9a01579d0 0x7fd9a00b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a0161450 0x7fd9a0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00d1b50 con 0x7fd9a00ce410 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00d19d0 con 0x7fd9a0161450 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a00d1cd0 con 0x7fd9a01579d0 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45446/0 (socket says 172.21.15.44:45446) 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 -- 172.21.15.44:0/1659684950 learned_addr learned my addr 172.21.15.44:0/1659684950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:36.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 -- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9a01579d0 msgr2=0x7fd9a00b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9a01579d0 0x7fd9a00b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 -- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a0161450 msgr2=0x7fd9a0158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a0161450 0x7fd9a0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.848+0000 7fd99de00640 1 -- 172.21.15.44:0/1659684950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a00b8ac0 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd99de00640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c410 tx=0x7fd99000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990017070 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9900035f0 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990003a80 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a01f76d0 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.849+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a01f7c90 con 0x7fd9a00ce410 2024-07-17T08:44:36.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.850+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a00d1280 con 0x7fd9a00ce410 2024-07-17T08:44:36.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.852+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fd990003c20 con 0x7fd9a00ce410 2024-07-17T08:44:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.853+0000 7fd996000640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 0x7fd980079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:36.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.853+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7fd990097b50 con 0x7fd9a00ce410 2024-07-17T08:44:36.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.853+0000 7fd99d400640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 0x7fd980079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:36.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.854+0000 7fd99d400640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 0x7fd980079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd98c000f60 tx=0x7fd98c0079b0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:36.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:36.857+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd990060b40 con 0x7fd9a00ce410 2024-07-17T08:44:37.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.058+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd9a006cb80 con 0x7fd9a00ce410 2024-07-17T08:44:37.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.059+0000 7fd996000640 1 -- 172.21.15.44:0/1659684950 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v103) ==== 59+0+1998 (secure 0 0 0) 0x7fd990060440 con 0x7fd9a00ce410 2024-07-17T08:44:37.060 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:44:37.061 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:44:37.062 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:44:37.063 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:44:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:44:37.065 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:44:37.065 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:44:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 msgr2=0x7fd980079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 0x7fd980079190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd98c000f60 tx=0x7fd98c0079b0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 msgr2=0x7fd9a01569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c410 tx=0x7fd99000c8d0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 shutdown_connections 2024-07-17T08:44:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fd980076cd0 0x7fd980079190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9a0161450 0x7fd9a0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9a01579d0 0x7fd9a00b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 --2- 172.21.15.44:0/1659684950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9a00ce410 0x7fd9a01569c0 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.062+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 >> 172.21.15.44:0/1659684950 conn(0x7fd9a006c390 msgr2=0x7fd9a0160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.063+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 shutdown_connections 2024-07-17T08:44:37.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.063+0000 7fd99f200640 1 -- 172.21.15.44:0/1659684950 wait complete. 2024-07-17T08:44:37.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 -- 172.21.15.44:0/80151636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640cd0b0 msgr2=0x7fc36406ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 --2- 172.21.15.44:0/80151636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640cd0b0 0x7fc36406ec70 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fc36415ec10 tx=0x7fc34c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 -- 172.21.15.44:0/80151636 shutdown_connections 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 --2- 172.21.15.44:0/80151636 >> v2:172.21.15.175:3300/0 conn(0x7fc364152300 0x7fc36406f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 --2- 172.21.15.44:0/80151636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3640cd0b0 0x7fc36406ec70 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 --2- 172.21.15.44:0/80151636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 0x7fc36415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 -- 172.21.15.44:0/80151636 >> 172.21.15.44:0/80151636 conn(0x7fc3640cf0d0 msgr2=0x7fc3640cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 -- 172.21.15.44:0/80151636 shutdown_connections 2024-07-17T08:44:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.292+0000 7fc363200640 1 -- 172.21.15.44:0/80151636 wait complete. 2024-07-17T08:44:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.294+0000 7fc363200640 1 Processor -- start 2024-07-17T08:44:37.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.295+0000 7fc363200640 1 -- start start 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3640cd0b0 0x7fc3641527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 0x7fc364073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640cb450 con 0x7fc364152300 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640cb2d0 con 0x7fc3640cd0b0 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc363200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc3640cb5d0 con 0x7fc364151f20 2024-07-17T08:44:37.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.296+0000 7fc362800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45460/0 (socket says 172.21.15.44:45460) 2024-07-17T08:44:37.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 -- 172.21.15.44:0/3143401432 learned_addr learned my addr 172.21.15.44:0/3143401432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc35be00640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 0x7fc364073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc361e00640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3640cd0b0 0x7fc3641527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 -- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 msgr2=0x7fc364073df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 0x7fc364073df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 -- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3640cd0b0 msgr2=0x7fc3641527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3640cd0b0 0x7fc3641527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 -- 172.21.15.44:0/3143401432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc364155580 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.297+0000 7fc362800640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00cb00 tx=0x7fc35c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.298+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00f450 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.298+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc35c00f700 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.298+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00fb50 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.298+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc364155b00 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.298+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc364152e30 con 0x7fc364152300 2024-07-17T08:44:37.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.299+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36415e2b0 con 0x7fc364152300 2024-07-17T08:44:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.300+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fc35c00fcf0 con 0x7fc364152300 2024-07-17T08:44:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.301+0000 7fc35a000640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 0x7fc3400791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.302+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7fc35c099500 con 0x7fc364152300 2024-07-17T08:44:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.302+0000 7fc361e00640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 0x7fc3400791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.303+0000 7fc361e00640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 0x7fc3400791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc35400a4e0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.305+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc36415e2b0 con 0x7fc364152300 2024-07-17T08:44:37.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.516+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc36415cc10 con 0x7fc364152300 2024-07-17T08:44:37.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.524+0000 7fc35a000640 1 -- 172.21.15.44:0/3143401432 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v104)=0 v104) ==== 122+0+0 (secure 0 0 0) 0x7fc35c014030 con 0x7fc364152300 2024-07-17T08:44:37.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 msgr2=0x7fc3400791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 0x7fc3400791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc35400a4e0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 msgr2=0x7fc36413f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00cb00 tx=0x7fc35c00a250 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 shutdown_connections 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc340076d00 0x7fc3400791c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc364152300 0x7fc36413f2b0 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc364151f20 0x7fc364073df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 --2- 172.21.15.44:0/3143401432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc3640cd0b0 0x7fc3641527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:37.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 >> 172.21.15.44:0/3143401432 conn(0x7fc3640cf0d0 msgr2=0x7fc3640aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:37.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 shutdown_connections 2024-07-17T08:44:37.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:37.527+0000 7fc363200640 1 -- 172.21.15.44:0/3143401432 wait complete. 2024-07-17T08:44:37.549 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:44:37.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:44:37.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:44:37.790 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:44:37.791 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:37.791 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:37.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:44:37.849 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:44:37.849 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:37.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:44:37.905 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:37.905 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:37.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:44:37.965 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:44:37.965 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:38.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:44:38.019 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:44:38.019 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 -- 172.21.15.44:0/1757385412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 msgr2=0x7fa9a0003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 --2- 172.21.15.44:0/1757385412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9a0003f60 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006f90 tx=0x7fa9a0005830 comp rx=0 tx=0).stop 2024-07-17T08:44:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 -- 172.21.15.44:0/1757385412 shutdown_connections 2024-07-17T08:44:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 --2- 172.21.15.44:0/1757385412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9a0003f60 unknown :-1 s=CLOSED pgs=3143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 -- 172.21.15.44:0/1757385412 >> 172.21.15.44:0/1757385412 conn(0x7fa9bc0cfec0 msgr2=0x7fa9bc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.240+0000 7fa9c2400640 1 -- 172.21.15.44:0/1757385412 shutdown_connections 2024-07-17T08:44:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.241+0000 7fa9c2400640 1 -- 172.21.15.44:0/1757385412 wait complete. 2024-07-17T08:44:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.242+0000 7fa9c2400640 1 Processor -- start 2024-07-17T08:44:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.242+0000 7fa9c2400640 1 -- start start 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9bc15d530 0x7fa9bc15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9bc0c8d50 0x7fa9bc0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc0b5460 con 0x7fa9a0003b40 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc0b52e0 con 0x7fa9bc15d530 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.243+0000 7fa9c2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9bc0b55e0 con 0x7fa9bc0c8d50 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45480/0 (socket says 172.21.15.44:45480) 2024-07-17T08:44:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 -- 172.21.15.44:0/308062269 learned_addr learned my addr 172.21.15.44:0/308062269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9bbe00640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9bc15d530 0x7fa9bc15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1a00640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9bc0c8d50 0x7fa9bc0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 -- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9bc0c8d50 msgr2=0x7fa9bc0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9bc0c8d50 0x7fa9bc0c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 -- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9bc15d530 msgr2=0x7fa9bc15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9bc15d530 0x7fa9bc15d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.244+0000 7fa9c1000640 1 -- 172.21.15.44:0/308062269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc1f42d0 con 0x7fa9a0003b40 2024-07-17T08:44:38.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9c1000640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4031040 tx=0x7fa9a4003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:38.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4006bb0 con 0x7fa9a0003b40 2024-07-17T08:44:38.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a40014f0 con 0x7fa9a0003b40 2024-07-17T08:44:38.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40426a0 con 0x7fa9a0003b40 2024-07-17T08:44:38.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc1f45b0 con 0x7fa9a0003b40 2024-07-17T08:44:38.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.245+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1f4b70 con 0x7fa9a0003b40 2024-07-17T08:44:38.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.246+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc06dc30 con 0x7fa9a0003b40 2024-07-17T08:44:38.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.247+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fa9a4035020 con 0x7fa9a0003b40 2024-07-17T08:44:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.248+0000 7fa9b8c00640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 0x7fa998079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.248+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7fa9a40b9270 con 0x7fa9a0003b40 2024-07-17T08:44:38.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.249+0000 7fa9bbe00640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 0x7fa998079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.250+0000 7fa9bbe00640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 0x7fa998079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc1f7680 tx=0x7fa9ac009500 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:38.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.253+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa9a4081f90 con 0x7fa9a0003b40 2024-07-17T08:44:38.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.507+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9bc0b5d70 con 0x7fa9a0003b40 2024-07-17T08:44:38.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.507+0000 7fa9b8c00640 1 -- 172.21.15.44:0/308062269 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) ==== 95+0+3925 (secure 0 0 0) 0x7fa9a408a260 con 0x7fa9a0003b40 2024-07-17T08:44:38.508 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:38.509 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":193,"btime":"2024-07-17T08:44:30:551506+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9629,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1947856170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1947856170},{"type":"v1","addr":"172.21.15.175:6836","nonce":1947856170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":10717,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/3752628290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3752628290},{"type":"v1","addr":"172.21.15.175:6801","nonce":3752628290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184},{"gid":13485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/1075600455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1075600455},{"type":"v1","addr":"172.21.15.44:6833","nonce":1075600455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":13869,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/3429809948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":3429809948},{"type":"v1","addr":"172.21.15.44:6835","nonce":3429809948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":193}],"filesystems":[]} 2024-07-17T08:44:38.509 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 193 2024-07-17T08:44:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.510+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 msgr2=0x7fa998079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.510+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 0x7fa998079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc1f7680 tx=0x7fa9ac009500 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.510+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 msgr2=0x7fa9bc1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.510+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4031040 tx=0x7fa9a4003bd0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 shutdown_connections 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa998076cd0 0x7fa998079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9bc0c8d50 0x7fa9bc0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9bc15d530 0x7fa9bc15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 --2- 172.21.15.44:0/308062269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0003b40 0x7fa9bc1564c0 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 >> 172.21.15.44:0/308062269 conn(0x7fa9bc0cfec0 msgr2=0x7fa9bc0b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 shutdown_connections 2024-07-17T08:44:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.511+0000 7fa9c2400640 1 -- 172.21.15.44:0/308062269 wait complete. 2024-07-17T08:44:38.532 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:44:38.532 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:44:38.532 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:44:38.532 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:44:38.532 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:44:38.532 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:44:38.533 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:44:38.533 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:44:38.533 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:44:38.533 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:44:38.533 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:44:38.533 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:44:38.533 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:44:38.534 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:44:38.534 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:44:38.534 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:44:38.554 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:44:38.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:44:38.556 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:44:38.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:44:38.563 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:44:38.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:44:38.568 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:44:38.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:44:38.603 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:44:38.604 DEBUG:teuthology.parallel:result is None 2024-07-17T08:44:38.609 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:44:38.609 DEBUG:teuthology.parallel:result is None 2024-07-17T08:44:38.615 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:44:38.616 DEBUG:teuthology.parallel:result is None 2024-07-17T08:44:38.622 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:44:38.626 DEBUG:teuthology.parallel:result is None 2024-07-17T08:44:38.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:44:38.747 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:44:38.748 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:44:38.765 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:44:38.773 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:44:38.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.801+0000 7fafcdc00640 1 -- 172.21.15.44:0/2096875294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 msgr2=0x7fafc806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.801+0000 7fafcdc00640 1 --2- 172.21.15.44:0/2096875294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc806eb90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005fd0 tx=0x7fafc002bbc0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 -- 172.21.15.44:0/2096875294 shutdown_connections 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 --2- 172.21.15.44:0/2096875294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc806eb90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 --2- 172.21.15.44:0/2096875294 >> v2:172.21.15.44:3300/0 conn(0x7fafc806f6d0 0x7fafc815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 -- 172.21.15.44:0/2096875294 >> 172.21.15.44:0/2096875294 conn(0x7fafc80b0eb0 msgr2=0x7fafc80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 -- 172.21.15.44:0/2096875294 shutdown_connections 2024-07-17T08:44:38.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.802+0000 7fafcdc00640 1 -- 172.21.15.44:0/2096875294 wait complete. 2024-07-17T08:44:38.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.803+0000 7fafcdc00640 1 Processor -- start 2024-07-17T08:44:38.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.803+0000 7fafcdc00640 1 -- start start 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc8073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 0x7fafc813f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80aea90 con 0x7fafc806f6d0 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80ae910 con 0x7fafc80b3280 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.804+0000 7fafcdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80aec10 con 0x7fafc80acab0 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45530/0 (socket says 172.21.15.44:45530) 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 -- 172.21.15.44:0/1175104155 learned_addr learned my addr 172.21.15.44:0/1175104155 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafbe000640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc8073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc7400640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 0x7fafc813f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 -- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 msgr2=0x7fafc8073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc8073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 -- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 msgr2=0x7fafc813f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 0x7fafc813f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 -- 172.21.15.44:0/1175104155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc8151920 con 0x7fafc806f6d0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc7400640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 0x7fafc813f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.805+0000 7fafc6a00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e9f0 tx=0x7fafb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.806+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb800cd80 con 0x7fafc806f6d0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.806+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb8010430 con 0x7fafc806f6d0 2024-07-17T08:44:38.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.806+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8010840 con 0x7fafc806f6d0 2024-07-17T08:44:38.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.806+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc8151c00 con 0x7fafc806f6d0 2024-07-17T08:44:38.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.806+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81521c0 con 0x7fafc806f6d0 2024-07-17T08:44:38.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.807+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc815a5a0 con 0x7fafc806f6d0 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.808+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fafb80109e0 con 0x7fafc806f6d0 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.809+0000 7fafbf400640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 0x7fafa4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.809+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(343..343 src has 1..343) ==== 6827+0+0 (secure 0 0 0) 0x7fafb8099640 con 0x7fafc806f6d0 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.809+0000 7fafbe000640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 0x7fafa4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.810+0000 7fafbe000640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 0x7fafa4079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafb0007fd0 tx=0x7fafb0009040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:38.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:38.813+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fafb8062420 con 0x7fafc806f6d0 2024-07-17T08:44:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.032+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fafc80b4200 con 0x7fafc806f6d0 2024-07-17T08:44:39.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.640+0000 7fafbf400640 1 -- 172.21.15.44:0/1175104155 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v345) ==== 97+0+0 (secure 0 0 0) 0x7fafb8061d20 con 0x7fafc806f6d0 2024-07-17T08:44:39.642 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 msgr2=0x7fafa4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 0x7fafa4079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafb0007fd0 tx=0x7fafb0009040 comp rx=0 tx=0).stop 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 msgr2=0x7fafc81510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7fafb800e9f0 tx=0x7fafb800eeb0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 shutdown_connections 2024-07-17T08:44:39.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fafa4076cd0 0x7fafa4079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafc80b3280 0x7fafc813f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fafc80acab0 0x7fafc8073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 --2- 172.21.15.44:0/1175104155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafc806f6d0 0x7fafc81510d0 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 >> 172.21.15.44:0/1175104155 conn(0x7fafc80b0eb0 msgr2=0x7fafc81537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 shutdown_connections 2024-07-17T08:44:39.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.645+0000 7fafcdc00640 1 -- 172.21.15.44:0/1175104155 wait complete. 2024-07-17T08:44:39.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:44:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 -- 172.21.15.44:0/3851690889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 msgr2=0x7fa6900ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 --2- 172.21.15.44:0/3851690889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa6900ac690 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7fa69015edd0 tx=0x7fa67802a540 comp rx=0 tx=0).stop 2024-07-17T08:44:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 -- 172.21.15.44:0/3851690889 shutdown_connections 2024-07-17T08:44:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 --2- 172.21.15.44:0/3851690889 >> v2:172.21.15.175:3300/0 conn(0x7fa6900cd690 0x7fa6900cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 --2- 172.21.15.44:0/3851690889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa6900ac690 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 -- 172.21.15.44:0/3851690889 >> 172.21.15.44:0/3851690889 conn(0x7fa6900b0fb0 msgr2=0x7fa6900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 -- 172.21.15.44:0/3851690889 shutdown_connections 2024-07-17T08:44:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.887+0000 7fa696800640 1 -- 172.21.15.44:0/3851690889 wait complete. 2024-07-17T08:44:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.889+0000 7fa696800640 1 Processor -- start 2024-07-17T08:44:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.889+0000 7fa696800640 1 -- start start 2024-07-17T08:44:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6900cd690 0x7fa690154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6901522c0 0x7fa6901f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900ce020 con 0x7fa6900ac220 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900adf80 con 0x7fa6900cd690 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa696800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6900ce1a0 con 0x7fa6901522c0 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa68f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa68f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49938/0 (socket says 172.21.15.44:49938) 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.890+0000 7fa68f400640 1 -- 172.21.15.44:0/4105350936 learned_addr learned my addr 172.21.15.44:0/4105350936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 -- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6901522c0 msgr2=0x7fa6901f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6901522c0 0x7fa6901f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa686a00640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6900cd690 0x7fa690154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 -- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6900cd690 msgr2=0x7fa690154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6900cd690 0x7fa690154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 -- 172.21.15.44:0/4105350936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6901f7440 con 0x7fa6900ac220 2024-07-17T08:44:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa68f400640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c430 tx=0x7fa68000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011790 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa680017070 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.891+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011ce0 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.892+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6901f7720 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.892+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6901f7ce0 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.893+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6900ce6c0 con 0x7fa6900ac220 2024-07-17T08:44:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.894+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fa68001c460 con 0x7fa6900ac220 2024-07-17T08:44:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.895+0000 7fa687e00640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 0x7fa66c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.895+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(347..347 src has 1..347) ==== 5881+0+0 (secure 0 0 0) 0x7fa680098380 con 0x7fa6900ac220 2024-07-17T08:44:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.896+0000 7fa686a00640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 0x7fa66c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.897+0000 7fa686a00640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 0x7fa66c079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa678034000 tx=0x7fa678004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:39.899+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa680061680 con 0x7fa6900ac220 2024-07-17T08:44:40.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.196+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa6900cfec0 con 0x7fa6900ac220 2024-07-17T08:44:40.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.197+0000 7fa687e00640 1 -- 172.21.15.44:0/4105350936 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v82) ==== 80+0+5304 (secure 0 0 0) 0x7fa680060f80 con 0x7fa6900ac220 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:44:40.198 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:44:40.201 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:44:40.203 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:44:40.205 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:44:40.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.207 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:44:40.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.211 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:44:40.212 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.199+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 msgr2=0x7fa66c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.199+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 0x7fa66c079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa678034000 tx=0x7fa678004730 comp rx=0 tx=0).stop 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.199+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 msgr2=0x7fa690074040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.199+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c430 tx=0x7fa68000c8f0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 shutdown_connections 2024-07-17T08:44:40.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa66c076cd0 0x7fa66c079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6901522c0 0x7fa6901f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6900cd690 0x7fa690154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 --2- 172.21.15.44:0/4105350936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6900ac220 0x7fa690074040 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 >> 172.21.15.44:0/4105350936 conn(0x7fa6900b0fb0 msgr2=0x7fa69015a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 shutdown_connections 2024-07-17T08:44:40.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.200+0000 7fa696800640 1 -- 172.21.15.44:0/4105350936 wait complete. 2024-07-17T08:44:40.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 -- 172.21.15.44:0/2487700313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 msgr2=0x7f479c0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 --2- 172.21.15.44:0/2487700313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0b3870 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 crypto rx=0x7f478c009830 tx=0x7f478c02a510 comp rx=0 tx=0).stop 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 -- 172.21.15.44:0/2487700313 shutdown_connections 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 --2- 172.21.15.44:0/2487700313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0cf920 0x7f479c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 --2- 172.21.15.44:0/2487700313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0b3870 unknown :-1 s=CLOSED pgs=3155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 -- 172.21.15.44:0/2487700313 >> 172.21.15.44:0/2487700313 conn(0x7f479c0b0e80 msgr2=0x7f479c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 -- 172.21.15.44:0/2487700313 shutdown_connections 2024-07-17T08:44:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.428+0000 7f47a1000640 1 -- 172.21.15.44:0/2487700313 wait complete. 2024-07-17T08:44:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.430+0000 7f47a1000640 1 Processor -- start 2024-07-17T08:44:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.430+0000 7f47a1000640 1 -- start start 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0b3db0 0x7f479c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f479c0cf920 0x7f479c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0ae170 con 0x7f479c0cf540 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0adff0 con 0x7f479c0cf920 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f47a1000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f479c0ae2f0 con 0x7f479c0b3db0 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f479aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f479aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49964/0 (socket says 172.21.15.44:49964) 2024-07-17T08:44:40.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.431+0000 7f479aa00640 1 -- 172.21.15.44:0/3300742550 learned_addr learned my addr 172.21.15.44:0/3300742550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479b400640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0b3db0 0x7f479c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479be00640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f479c0cf920 0x7f479c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 -- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0b3db0 msgr2=0x7f479c06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0b3db0 0x7f479c06f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 -- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f479c0cf920 msgr2=0x7f479c155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f479c0cf920 0x7f479c155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 -- 172.21.15.44:0/3300742550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c06e5e0 con 0x7f479c0cf540 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f479aa00640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f478800c400 tx=0x7f478800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788017070 con 0x7f479c0cf540 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47880035f0 con 0x7f479c0cf540 2024-07-17T08:44:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788003a40 con 0x7f479c0cf540 2024-07-17T08:44:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.432+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c06e020 con 0x7f479c0cf540 2024-07-17T08:44:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.433+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c072ec0 con 0x7f479c0cf540 2024-07-17T08:44:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.434+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c0ae880 con 0x7f479c0cf540 2024-07-17T08:44:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.435+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f478801a070 con 0x7f479c0cf540 2024-07-17T08:44:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.436+0000 7f4793400640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 0x7f477c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.436+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(347..347 src has 1..347) ==== 5881+0+0 (secure 0 0 0) 0x7f4788097850 con 0x7f479c0cf540 2024-07-17T08:44:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.437+0000 7f479b400640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 0x7f477c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.438+0000 7f479b400640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 0x7f477c079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f478c034000 tx=0x7f478c004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.441+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4788060bf0 con 0x7f479c0cf540 2024-07-17T08:44:40.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.651+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f479c09ca40 con 0x7f479c0cf540 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.652+0000 7f4793400640 1 -- 172.21.15.44:0/3300742550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v347) ==== 84+0+65 (secure 0 0 0) 0x7f47880604f0 con 0x7f479c0cf540 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:44:40.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:40.654 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 msgr2=0x7f477c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 0x7f477c079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f478c034000 tx=0x7f478c004730 comp rx=0 tx=0).stop 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 msgr2=0x7f479c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f478800c400 tx=0x7f478800c8c0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 shutdown_connections 2024-07-17T08:44:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f477c076cd0 0x7f477c079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f479c0cf920 0x7f479c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f479c0cf540 0x7f479c0736c0 unknown :-1 s=CLOSED pgs=3156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 --2- 172.21.15.44:0/3300742550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f479c0b3db0 0x7f479c06f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 >> 172.21.15.44:0/3300742550 conn(0x7f479c0b0e80 msgr2=0x7f479c1592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 shutdown_connections 2024-07-17T08:44:40.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.655+0000 7f47a1000640 1 -- 172.21.15.44:0/3300742550 wait complete. 2024-07-17T08:44:40.676 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:44:40.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:44:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.886+0000 7f0fe7e00640 1 -- 172.21.15.44:0/768906120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 msgr2=0x7f0fcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.886+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/768906120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fcc00b9c0 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0001690 tx=0x7f0fcc005830 comp rx=0 tx=0).stop 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.886+0000 7f0fe7e00640 1 -- 172.21.15.44:0/768906120 shutdown_connections 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.886+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/768906120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fcc00b9c0 unknown :-1 s=CLOSED pgs=3158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.886+0000 7f0fe7e00640 1 -- 172.21.15.44:0/768906120 >> 172.21.15.44:0/768906120 conn(0x7f0fe80b55d0 msgr2=0x7f0fe80b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.887+0000 7f0fe7e00640 1 -- 172.21.15.44:0/768906120 shutdown_connections 2024-07-17T08:44:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.887+0000 7f0fe7e00640 1 -- 172.21.15.44:0/768906120 wait complete. 2024-07-17T08:44:40.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.888+0000 7f0fe7e00640 1 Processor -- start 2024-07-17T08:44:40.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.888+0000 7f0fe7e00640 1 -- start start 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fe8150400 0x7f0fe80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fe81e6740 0x7f0fe81e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80b4050 con 0x7f0fcc0052e0 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80b3ed0 con 0x7f0fe8150400 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.889+0000 7f0fe7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe80b41d0 con 0x7f0fe81e6740 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49982/0 (socket says 172.21.15.44:49982) 2024-07-17T08:44:40.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 -- 172.21.15.44:0/3548371457 learned_addr learned my addr 172.21.15.44:0/3548371457 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe7400640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fe81e6740 0x7f0fe81e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 -- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fe81e6740 msgr2=0x7f0fe81e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fe81e6740 0x7f0fe81e6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6000640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fe8150400 0x7f0fe80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 -- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fe8150400 msgr2=0x7f0fe80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fe8150400 0x7f0fe80cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 -- 172.21.15.44:0/3548371457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe8150880 con 0x7f0fcc0052e0 2024-07-17T08:44:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.890+0000 7f0fe6a00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9a0 tx=0x7f0fd800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.891+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8015020 con 0x7f0fcc0052e0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.891+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fd800ce20 con 0x7f0fcc0052e0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.891+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd80116c0 con 0x7f0fcc0052e0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.891+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe80ccba0 con 0x7f0fcc0052e0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.891+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe80cd160 con 0x7f0fcc0052e0 2024-07-17T08:44:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.892+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe8069a20 con 0x7f0fcc0052e0 2024-07-17T08:44:40.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.893+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f0fd8018020 con 0x7f0fcc0052e0 2024-07-17T08:44:40.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.894+0000 7f0fdea00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 0x7f0fc4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:40.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.894+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(347..347 src has 1..347) ==== 5881+0+0 (secure 0 0 0) 0x7f0fd8098f70 con 0x7f0fcc0052e0 2024-07-17T08:44:40.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.895+0000 7f0fe6000640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 0x7f0fc4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:40.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.896+0000 7f0fe6000640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 0x7f0fc4079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80af910 tx=0x7f0fd0005f90 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:40.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:40.899+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0fd801f020 con 0x7f0fcc0052e0 2024-07-17T08:44:41.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.110+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0fe80b3220 con 0x7f0fcc0052e0 2024-07-17T08:44:41.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.651+0000 7f0fdea00640 1 -- 172.21.15.44:0/3548371457 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v348) ==== 138+0+0 (secure 0 0 0) 0x7f0fd8062160 con 0x7f0fcc0052e0 2024-07-17T08:44:41.653 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:44:41.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.655+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 msgr2=0x7f0fc4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:41.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.655+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 0x7f0fc4079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80af910 tx=0x7f0fd0005f90 comp rx=0 tx=0).stop 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.655+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 msgr2=0x7f0fe80cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.655+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9a0 tx=0x7f0fd800ee60 comp rx=0 tx=0).stop 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 shutdown_connections 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f0fc4076cd0 0x7f0fc4079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0fe81e6740 0x7f0fe81e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0fe8150400 0x7f0fe80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 --2- 172.21.15.44:0/3548371457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fcc0052e0 0x7f0fe80cafb0 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 >> 172.21.15.44:0/3548371457 conn(0x7f0fe80b55d0 msgr2=0x7f0fe815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:41.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.656+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 shutdown_connections 2024-07-17T08:44:41.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.657+0000 7f0fe7e00640 1 -- 172.21.15.44:0/3548371457 wait complete. 2024-07-17T08:44:41.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 -- 172.21.15.44:0/2512154885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 msgr2=0x7faf78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 --2- 172.21.15.44:0/2512154885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf78004c80 secure :-1 s=READY pgs=3161 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00bb20 tx=0x7faf78005830 comp rx=0 tx=0).stop 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf92800640 1 -- 172.21.15.44:0/2512154885 reap_dead start 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 -- 172.21.15.44:0/2512154885 shutdown_connections 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 -- 172.21.15.44:0/2512154885 >> 172.21.15.44:0/2512154885 conn(0x7faf940cf0d0 msgr2=0x7faf940cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:41.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 -- 172.21.15.44:0/2512154885 shutdown_connections 2024-07-17T08:44:41.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.895+0000 7faf93200640 1 -- 172.21.15.44:0/2512154885 wait complete. 2024-07-17T08:44:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.897+0000 7faf93200640 1 Processor -- start 2024-07-17T08:44:41.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.897+0000 7faf93200640 1 -- start start 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf941f3c80 0x7faf941531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf94153730 0x7faf94153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf940cbb20 con 0x7faf78004800 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf940cb9a0 con 0x7faf941f3c80 2024-07-17T08:44:41.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.898+0000 7faf93200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf940713e0 con 0x7faf94153730 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49992/0 (socket says 172.21.15.44:49992) 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 -- 172.21.15.44:0/2088627784 learned_addr learned my addr 172.21.15.44:0/2088627784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf92800640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf94153730 0x7faf94153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf8be00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf941f3c80 0x7faf941531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 -- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf94153730 msgr2=0x7faf94153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf94153730 0x7faf94153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 -- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf941f3c80 msgr2=0x7faf941531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf941f3c80 0x7faf941531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:41.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.899+0000 7faf91e00640 1 -- 172.21.15.44:0/2088627784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf94073f60 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf91e00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7faf84006630 tx=0x7faf840039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84015070 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf84011900 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84011d50 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf94074240 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.900+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9414a910 con 0x7faf78004800 2024-07-17T08:44:41.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.901+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf940cb080 con 0x7faf78004800 2024-07-17T08:44:41.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.903+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7faf84018070 con 0x7faf78004800 2024-07-17T08:44:41.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.905+0000 7faf8a000640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 0x7faf70079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:41.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.905+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(348..348 src has 1..348) ==== 6251+0+0 (secure 0 0 0) 0x7faf840996f0 con 0x7faf78004800 2024-07-17T08:44:41.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.906+0000 7faf8be00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 0x7faf70079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:41.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.907+0000 7faf8be00640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 0x7faf70079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf7c036000 tx=0x7faf7c0046c0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:41.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:41.909+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7faf84062770 con 0x7faf78004800 2024-07-17T08:44:42.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.122+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faf940af510 con 0x7faf78004800 2024-07-17T08:44:42.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:44:42.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:44:42.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.659+0000 7faf8a000640 1 -- 172.21.15.44:0/2088627784 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v349) ==== 170+0+0 (secure 0 0 0) 0x7faf84062070 con 0x7faf78004800 2024-07-17T08:44:42.660 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:44:42.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.662+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 msgr2=0x7faf70079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:42.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.662+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 0x7faf70079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf7c036000 tx=0x7faf7c0046c0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.662+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 msgr2=0x7faf9414a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:42.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.662+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7faf84006630 tx=0x7faf840039c0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 shutdown_connections 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7faf70076cd0 0x7faf70079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf94153730 0x7faf94153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf941f3c80 0x7faf941531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 --2- 172.21.15.44:0/2088627784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf78004800 0x7faf9414a3d0 unknown :-1 s=CLOSED pgs=3162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.663+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 >> 172.21.15.44:0/2088627784 conn(0x7faf940cf0d0 msgr2=0x7faf940ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.664+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 shutdown_connections 2024-07-17T08:44:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.664+0000 7faf93200640 1 -- 172.21.15.44:0/2088627784 wait complete. 2024-07-17T08:44:42.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:44:42.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 -- 172.21.15.44:0/792941403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 msgr2=0x7f3d4006c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:42.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 --2- 172.21.15.44:0/792941403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4006c410 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 crypto rx=0x7f3d4008c9f0 tx=0x7f3d3002a440 comp rx=0 tx=0).stop 2024-07-17T08:44:42.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 -- 172.21.15.44:0/792941403 shutdown_connections 2024-07-17T08:44:42.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 --2- 172.21.15.44:0/792941403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 0x7f3d400d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 --2- 172.21.15.44:0/792941403 >> v2:172.21.15.175:3300/0 conn(0x7f3d400ce710 0x7f3d4006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 --2- 172.21.15.44:0/792941403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4006c410 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 -- 172.21.15.44:0/792941403 >> 172.21.15.44:0/792941403 conn(0x7f3d400d00f0 msgr2=0x7f3d400d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:42.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 -- 172.21.15.44:0/792941403 shutdown_connections 2024-07-17T08:44:42.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.940+0000 7f3d45600640 1 -- 172.21.15.44:0/792941403 wait complete. 2024-07-17T08:44:42.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.942+0000 7f3d45600640 1 Processor -- start 2024-07-17T08:44:42.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.942+0000 7f3d45600640 1 -- start start 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d400ce710 0x7f3d400b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 0x7f3d4015fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4015dca0 con 0x7f3d4015d740 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4015db20 con 0x7f3d400ce710 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d45600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d400ca370 con 0x7f3d400d11b0 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50006/0 (socket says 172.21.15.44:50006) 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d3f400640 1 -- 172.21.15.44:0/1063323930 learned_addr learned my addr 172.21.15.44:0/1063323930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:42.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d3ea00640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d400ce710 0x7f3d400b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:42.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.944+0000 7f3d37e00640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 0x7f3d4015fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 -- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 msgr2=0x7f3d4015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 0x7f3d4015fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 -- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d400ce710 msgr2=0x7f3d400b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d400ce710 0x7f3d400b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 -- 172.21.15.44:0/1063323930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d401f74b0 con 0x7f3d4015d740 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d3f400640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800cce0 tx=0x7f3d3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800f450 con 0x7f3d4015d740 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d3800f700 con 0x7f3d4015d740 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d401f7790 con 0x7f3d4015d740 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.945+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800fb50 con 0x7f3d4015d740 2024-07-17T08:44:42.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.946+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d401f7d50 con 0x7f3d4015d740 2024-07-17T08:44:42.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.947+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d400b6950 con 0x7f3d4015d740 2024-07-17T08:44:42.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.948+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f3d3800fcf0 con 0x7f3d4015d740 2024-07-17T08:44:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.949+0000 7f3d36a00640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 0x7f3d1c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.949+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(349..349 src has 1..349) ==== 6633+0+0 (secure 0 0 0) 0x7f3d38099680 con 0x7f3d4015d740 2024-07-17T08:44:42.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.950+0000 7f3d3ea00640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 0x7f3d1c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:42.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.951+0000 7f3d3ea00640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 0x7f3d1c079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d30033000 tx=0x7f3d30004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:42.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:42.954+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3d38014030 con 0x7f3d4015d740 2024-07-17T08:44:43.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.208+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3d400d07b0 con 0x7f3d4015d740 2024-07-17T08:44:43.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.687+0000 7f3d36a00640 1 -- 172.21.15.44:0/1063323930 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 47 and data pool 48 v196) ==== 174+0+0 (secure 0 0 0) 0x7f3d38062600 con 0x7f3d4015d740 2024-07-17T08:44:43.689 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 47 and data pool 48 2024-07-17T08:44:43.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.692+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 msgr2=0x7f3d1c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:43.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.692+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 0x7f3d1c079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d30033000 tx=0x7f3d30004730 comp rx=0 tx=0).stop 2024-07-17T08:44:43.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.692+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 msgr2=0x7f3d4015ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:43.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.692+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800cce0 tx=0x7f3d3800a620 comp rx=0 tx=0).stop 2024-07-17T08:44:43.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 shutdown_connections 2024-07-17T08:44:43.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3d1c076cd0 0x7f3d1c079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d4015d740 0x7f3d4015ff40 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d400d11b0 0x7f3d4015fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 --2- 172.21.15.44:0/1063323930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d400ce710 0x7f3d400b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 >> 172.21.15.44:0/1063323930 conn(0x7f3d400d00f0 msgr2=0x7f3d40160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.693+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 shutdown_connections 2024-07-17T08:44:43.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.694+0000 7f3d45600640 1 -- 172.21.15.44:0/1063323930 wait complete. 2024-07-17T08:44:43.740 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 -- 172.21.15.44:0/3732708253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 msgr2=0x7f9e1406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 --2- 172.21.15.44:0/3732708253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e1406e060 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009830 tx=0x7f9e0402a510 comp rx=0 tx=0).stop 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 -- 172.21.15.44:0/3732708253 shutdown_connections 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 --2- 172.21.15.44:0/3732708253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 --2- 172.21.15.44:0/3732708253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e1406e060 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 -- 172.21.15.44:0/3732708253 >> 172.21.15.44:0/3732708253 conn(0x7f9e140b0e80 msgr2=0x7f9e140b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 -- 172.21.15.44:0/3732708253 shutdown_connections 2024-07-17T08:44:43.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.915+0000 7f9e13200640 1 -- 172.21.15.44:0/3732708253 wait complete. 2024-07-17T08:44:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.917+0000 7f9e13200640 1 Processor -- start 2024-07-17T08:44:43.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.917+0000 7f9e13200640 1 -- start start 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 0x7f9e141f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140b2120 con 0x7f9e1406f4b0 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140b1fa0 con 0x7f9e14159800 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e13200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140b22a0 con 0x7f9e140cf380 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:43.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e11e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50016/0 (socket says 172.21.15.44:50016) 2024-07-17T08:44:43.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.918+0000 7f9e11e00640 1 -- 172.21.15.44:0/1791097510 learned_addr learned my addr 172.21.15.44:0/1791097510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e12800640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 0x7f9e141f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e09400640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 -- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 msgr2=0x7f9e140adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 -- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 msgr2=0x7f9e141f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 0x7f9e141f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 -- 172.21.15.44:0/1791097510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140cd140 con 0x7f9e1406f4b0 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e12800640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 0x7f9e141f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:43.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e09400640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.919+0000 7f9e11e00640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 crypto rx=0x7f9e0402ab00 tx=0x7f9e04032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.920+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04040070 con 0x7f9e1406f4b0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.920+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e04005310 con 0x7f9e1406f4b0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.920+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04005c90 con 0x7f9e1406f4b0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.920+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e140cd3b0 con 0x7f9e1406f4b0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.920+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e140cdc50 con 0x7f9e1406f4b0 2024-07-17T08:44:43.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.921+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e140cd7a0 con 0x7f9e1406f4b0 2024-07-17T08:44:43.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.922+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f9e04039040 con 0x7f9e1406f4b0 2024-07-17T08:44:43.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.923+0000 7f9e0aa00640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 0x7f9df0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:43.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.923+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(350..350 src has 1..350) ==== 6733+0+0 (secure 0 0 0) 0x7f9e040b9ca0 con 0x7f9e1406f4b0 2024-07-17T08:44:43.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.924+0000 7f9e09400640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 0x7f9df0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:43.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.925+0000 7f9e09400640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 0x7f9df0079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e141f4f70 tx=0x7f9dfc005e50 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:43.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:43.928+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9e04082ab0 con 0x7f9e1406f4b0 2024-07-17T08:44:44.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.134+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e1406e060 con 0x7f9e1406f4b0 2024-07-17T08:44:44.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.135+0000 7f9e0aa00640 1 -- 172.21.15.44:0/1791097510 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v350) ==== 74+0+17924 (secure 0 0 0) 0x7f9e040823b0 con 0x7f9e1406f4b0 2024-07-17T08:44:44.137 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:44.137 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":350,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:43.210185+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":47,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:44:41.136788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":48,"pool_name":"cephfs_data","create_time":"2024-07-17T08:44:42.144654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 msgr2=0x7f9df0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 0x7f9df0079190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9e141f4f70 tx=0x7f9dfc005e50 comp rx=0 tx=0).stop 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 msgr2=0x7f9e140ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 crypto rx=0x7f9e0402ab00 tx=0x7f9e04032c10 comp rx=0 tx=0).stop 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 shutdown_connections 2024-07-17T08:44:44.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9df0076cd0 0x7f9df0079190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e14159800 0x7f9e141f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e140cf380 0x7f9e140adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 --2- 172.21.15.44:0/1791097510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e1406f4b0 0x7f9e140ce170 unknown :-1 s=CLOSED pgs=3167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 >> 172.21.15.44:0/1791097510 conn(0x7f9e140b0e80 msgr2=0x7f9e14153620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 shutdown_connections 2024-07-17T08:44:44.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.138+0000 7f9e13200640 1 -- 172.21.15.44:0/1791097510 wait complete. 2024-07-17T08:44:44.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 -- 172.21.15.44:0/3626306194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 msgr2=0x7ff4c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/3626306194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4c4004c80 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009b20 tx=0x7ff4c4005830 comp rx=0 tx=0).stop 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5200640 1 -- 172.21.15.44:0/3626306194 reap_dead start 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 -- 172.21.15.44:0/3626306194 shutdown_connections 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 -- 172.21.15.44:0/3626306194 >> 172.21.15.44:0/3626306194 conn(0x7ff4e00b0e80 msgr2=0x7ff4e00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 -- 172.21.15.44:0/3626306194 shutdown_connections 2024-07-17T08:44:44.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.348+0000 7ff4e5c00640 1 -- 172.21.15.44:0/3626306194 wait complete. 2024-07-17T08:44:44.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.350+0000 7ff4e5c00640 1 Processor -- start 2024-07-17T08:44:44.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.350+0000 7ff4e5c00640 1 -- start start 2024-07-17T08:44:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e01f3550 0x7ff4e015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e01f39d0 0x7ff4e015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0140030 con 0x7ff4c4004800 2024-07-17T08:44:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e00ac270 con 0x7ff4e01f39d0 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e01401b0 con 0x7ff4e01f3550 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4dfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50028/0 (socket says 172.21.15.44:50028) 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4dfe00640 1 -- 172.21.15.44:0/1682673972 learned_addr learned my addr 172.21.15.44:0/1682673972 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4e5200640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e01f39d0 0x7ff4e015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4dfe00640 1 -- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e01f3550 msgr2=0x7ff4e015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4d7e00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e01f3550 0x7ff4e015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.351+0000 7ff4dfe00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e01f3550 0x7ff4e015a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dfe00640 1 -- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e01f39d0 msgr2=0x7ff4e015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dfe00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e01f39d0 0x7ff4e015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dfe00640 1 -- 172.21.15.44:0/1682673972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e0156a20 con 0x7ff4c4004800 2024-07-17T08:44:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dfe00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0004df0 tx=0x7ff4d0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0015020 con 0x7ff4c4004800 2024-07-17T08:44:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4d0011960 con 0x7ff4c4004800 2024-07-17T08:44:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0001d40 con 0x7ff4c4004800 2024-07-17T08:44:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e0156d00 con 0x7ff4c4004800 2024-07-17T08:44:44.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.352+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e00716a0 con 0x7ff4c4004800 2024-07-17T08:44:44.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.353+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e01411f0 con 0x7ff4c4004800 2024-07-17T08:44:44.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.355+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7ff4d0018020 con 0x7ff4c4004800 2024-07-17T08:44:44.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.356+0000 7ff4dcc00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 0x7ff4bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.356+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(350..350 src has 1..350) ==== 6733+0+0 (secure 0 0 0) 0x7ff4d00992f0 con 0x7ff4c4004800 2024-07-17T08:44:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.356+0000 7ff4d7e00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 0x7ff4bc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.358+0000 7ff4d7e00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 0x7ff4bc0791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8035000 tx=0x7ff4c802d040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.360+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ff4d001f020 con 0x7ff4c4004800 2024-07-17T08:44:44.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.570+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4e00aed80 con 0x7ff4c4004800 2024-07-17T08:44:44.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.573+0000 7ff4dcc00640 1 -- 172.21.15.44:0/1682673972 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v350) ==== 74+0+17924 (secure 0 0 0) 0x7ff4d00621a0 con 0x7ff4c4004800 2024-07-17T08:44:44.574 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:44.574 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":350,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:43.210185+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":47,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:44:41.136788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":48,"pool_name":"cephfs_data","create_time":"2024-07-17T08:44:42.144654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 msgr2=0x7ff4bc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 0x7ff4bc0791c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8035000 tx=0x7ff4c802d040 comp rx=0 tx=0).stop 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 msgr2=0x7ff4e0071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0004df0 tx=0x7ff4d0001630 comp rx=0 tx=0).stop 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 shutdown_connections 2024-07-17T08:44:44.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7ff4bc076d00 0x7ff4bc0791c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e01f39d0 0x7ff4e015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e01f3550 0x7ff4e015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 --2- 172.21.15.44:0/1682673972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c4004800 0x7ff4e0071ea0 unknown :-1 s=CLOSED pgs=3170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 >> 172.21.15.44:0/1682673972 conn(0x7ff4e00b0e80 msgr2=0x7ff4e00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 shutdown_connections 2024-07-17T08:44:44.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.576+0000 7ff4e5c00640 1 -- 172.21.15.44:0/1682673972 wait complete. 2024-07-17T08:44:44.598 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 -- 172.21.15.44:0/476582841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 msgr2=0x7f94940b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 --2- 172.21.15.44:0/476582841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f94940b37c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f948c009830 tx=0x7f948c02a5a0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 -- 172.21.15.44:0/476582841 shutdown_connections 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 --2- 172.21.15.44:0/476582841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f94940b37c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 --2- 172.21.15.44:0/476582841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 0x7f94940ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 -- 172.21.15.44:0/476582841 >> 172.21.15.44:0/476582841 conn(0x7f949409aa30 msgr2=0x7f94940b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:44.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 -- 172.21.15.44:0/476582841 shutdown_connections 2024-07-17T08:44:44.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.817+0000 7f949bc00640 1 -- 172.21.15.44:0/476582841 wait complete. 2024-07-17T08:44:44.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.819+0000 7f949bc00640 1 Processor -- start 2024-07-17T08:44:44.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.819+0000 7f949bc00640 1 -- start start 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f949415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 0x7f94941f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94940caea0 con 0x7f949415d970 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94940cad20 con 0x7f94940b1220 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f949bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94940cb020 con 0x7f949415d590 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f9499c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f9499c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50038/0 (socket says 172.21.15.44:50038) 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f9499c00640 1 -- 172.21.15.44:0/861161712 learned_addr learned my addr 172.21.15.44:0/861161712 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f9499200640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f949415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.821+0000 7f9493e00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 0x7f94941f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 -- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 msgr2=0x7f94941f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 0x7f94941f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 -- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 msgr2=0x7f949415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f949415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 -- 172.21.15.44:0/861161712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949414faf0 con 0x7f949415d970 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9499c00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7f948c009950 tx=0x7f948c032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c040070 con 0x7f949415d970 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f948c005310 con 0x7f949415d970 2024-07-17T08:44:44.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0047a0 con 0x7f949415d970 2024-07-17T08:44:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.822+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941602c0 con 0x7f949415d970 2024-07-17T08:44:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.823+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9494160880 con 0x7f949415d970 2024-07-17T08:44:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.824+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940cb4d0 con 0x7f949415d970 2024-07-17T08:44:44.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.825+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f948c039040 con 0x7f949415d970 2024-07-17T08:44:44.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.826+0000 7f9490c00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 0x7f9470079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.826+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f948c0b9a00 con 0x7f949415d970 2024-07-17T08:44:44.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.827+0000 7f9499200640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 0x7f9470079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:44.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.828+0000 7f9499200640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 0x7f9470079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9484005fd0 tx=0x7f94840095c0 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:44.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:44.830+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f948c0828a0 con 0x7f949415d970 2024-07-17T08:44:45.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.093+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f94940b50b0 con 0x7f949415d970 2024-07-17T08:44:45.649 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:44:45.648+0000 7f8dbda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:44:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.739+0000 7f9490c00640 1 -- 172.21.15.44:0/861161712 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v199) ==== 118+0+0 (secure 0 0 0) 0x7f948c0821a0 con 0x7f949415d970 2024-07-17T08:44:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 msgr2=0x7f9470079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 0x7f9470079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9484005fd0 tx=0x7f94840095c0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 msgr2=0x7f949415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7f948c009950 tx=0x7f948c032b90 comp rx=0 tx=0).stop 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 shutdown_connections 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f9470076cd0 0x7f9470079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f949415d970 0x7f949415dd80 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f949415d590 0x7f94941f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 --2- 172.21.15.44:0/861161712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94940b1220 0x7f949415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 >> 172.21.15.44:0/861161712 conn(0x7f949409aa30 msgr2=0x7f94940d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 shutdown_connections 2024-07-17T08:44:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.743+0000 7f949bc00640 1 -- 172.21.15.44:0/861161712 wait complete. 2024-07-17T08:44:45.766 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:44:45.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:44:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 -- 172.21.15.44:0/1260188820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 msgr2=0x7f3f1c06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1260188820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c06d620 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c06f610 tx=0x7f3f0c02a4b0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 -- 172.21.15.44:0/1260188820 shutdown_connections 2024-07-17T08:44:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1260188820 >> v2:172.21.15.175:3300/0 conn(0x7f3f1c06ed30 0x7f3f1c15c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1260188820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c06d620 unknown :-1 s=CLOSED pgs=3173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 -- 172.21.15.44:0/1260188820 >> 172.21.15.44:0/1260188820 conn(0x7f3f1c0b0eb0 msgr2=0x7f3f1c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:45.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 -- 172.21.15.44:0/1260188820 shutdown_connections 2024-07-17T08:44:45.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.980+0000 7f3f21e00640 1 -- 172.21.15.44:0/1260188820 wait complete. 2024-07-17T08:44:45.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.982+0000 7f3f21e00640 1 Processor -- start 2024-07-17T08:44:45.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.982+0000 7f3f21e00640 1 -- start start 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 0x7f3f1c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1c0cb3a0 0x7f3f1c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c0cbc30 con 0x7f3f1c152220 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c0cbab0 con 0x7f3f1c0cb3a0 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.983+0000 7f3f21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c15f570 con 0x7f3f1c06ed30 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50060/0 (socket says 172.21.15.44:50060) 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 -- 172.21.15.44:0/1828513801 learned_addr learned my addr 172.21.15.44:0/1828513801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:45.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1a000640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1c0cb3a0 0x7f3f1c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1aa00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 0x7f3f1c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 -- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 msgr2=0x7f3f1c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 0x7f3f1c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 -- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1c0cb3a0 msgr2=0x7f3f1c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1c0cb3a0 0x7f3f1c152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 -- 172.21.15.44:0/1828513801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c0ac310 con 0x7f3f1c152220 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1aa00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 0x7f3f1c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:45.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.984+0000 7f3f1b400640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400cbc0 tx=0x7f3f1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.985+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14016020 con 0x7f3f1c152220 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.985+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f1400add0 con 0x7f3f1c152220 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.985+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1f5090 con 0x7f3f1c152220 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.985+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14012680 con 0x7f3f1c152220 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.985+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c0ac5d0 con 0x7f3f1c152220 2024-07-17T08:44:45.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.986+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c15fa90 con 0x7f3f1c152220 2024-07-17T08:44:45.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.987+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f3f14019020 con 0x7f3f1c152220 2024-07-17T08:44:45.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.988+0000 7f3f13400640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 0x7f3ef8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:45.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.988+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f3f14098cf0 con 0x7f3f1c152220 2024-07-17T08:44:45.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.989+0000 7f3f1aa00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 0x7f3ef8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:45.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.990+0000 7f3f1aa00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 0x7f3ef8079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c034000 tx=0x7f3f0c004730 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:45.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:45.993+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3f14020030 con 0x7f3f1c152220 2024-07-17T08:44:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.245+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3f1c06d830 con 0x7f3f1c152220 2024-07-17T08:44:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.742+0000 7f3f13400640 1 -- 172.21.15.44:0/1828513801 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v200) ==== 122+0+0 (secure 0 0 0) 0x7f3f14061c10 con 0x7f3f1c152220 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.745+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 msgr2=0x7f3ef8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.745+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 0x7f3ef8079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c034000 tx=0x7f3f0c004730 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.745+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 msgr2=0x7f3f1c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.745+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400cbc0 tx=0x7f3f1400a430 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 shutdown_connections 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f3ef8076cd0 0x7f3ef8079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f1c152220 0x7f3f1c14fd00 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1c0cb3a0 0x7f3f1c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 --2- 172.21.15.44:0/1828513801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1c06ed30 0x7f3f1c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 >> 172.21.15.44:0/1828513801 conn(0x7f3f1c0b0eb0 msgr2=0x7f3f1c0b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:46.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 shutdown_connections 2024-07-17T08:44:46.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.746+0000 7f3f21e00640 1 -- 172.21.15.44:0/1828513801 wait complete. 2024-07-17T08:44:46.767 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:44:46.767 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 -- 172.21.15.44:0/811044557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0cded0 msgr2=0x7fa15c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:46.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 --2- 172.21.15.44:0/811044557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0cded0 0x7fa15c0ce340 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7fa15c160fb0 tx=0x7fa15402a820 comp rx=0 tx=0).stop 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 -- 172.21.15.44:0/811044557 shutdown_connections 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 --2- 172.21.15.44:0/811044557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0cded0 0x7fa15c0ce340 unknown :-1 s=CLOSED pgs=3176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 --2- 172.21.15.44:0/811044557 >> v2:172.21.15.175:3300/0 conn(0x7fa15c0d3f10 0x7fa15c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 -- 172.21.15.44:0/811044557 >> 172.21.15.44:0/811044557 conn(0x7fa15c08ca20 msgr2=0x7fa15c08ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 -- 172.21.15.44:0/811044557 shutdown_connections 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.978+0000 7fa161c00640 1 -- 172.21.15.44:0/811044557 wait complete. 2024-07-17T08:44:46.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.980+0000 7fa161c00640 1 Processor -- start 2024-07-17T08:44:46.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.980+0000 7fa161c00640 1 -- start start 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 0x7fa15c160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15c0d3420 0x7fa15c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0ce880 con 0x7fa15c0d3f10 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0cea00 con 0x7fa15c0cded0 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c15afb0 con 0x7fa15c0d3420 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:46.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50082/0 (socket says 172.21.15.44:50082) 2024-07-17T08:44:46.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.981+0000 7fa161200640 1 -- 172.21.15.44:0/1110841190 learned_addr learned my addr 172.21.15.44:0/1110841190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa15be00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 0x7fa15c160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa15b400640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15c0d3420 0x7fa15c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 -- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15c0d3420 msgr2=0x7fa15c15a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15c0d3420 0x7fa15c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 -- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 msgr2=0x7fa15c160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 0x7fa15c160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 -- 172.21.15.44:0/1110841190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1f5180 con 0x7fa15c0d3f10 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa15be00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 0x7fa15c160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.982+0000 7fa161200640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7fa154009b30 tx=0x7fa154032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.983+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154037040 con 0x7fa15c0d3f10 2024-07-17T08:44:46.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.983+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa154042070 con 0x7fa15c0d3f10 2024-07-17T08:44:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.983+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154004280 con 0x7fa15c0d3f10 2024-07-17T08:44:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.983+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c0ca660 con 0x7fa15c0d3f10 2024-07-17T08:44:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.983+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c0caf00 con 0x7fa15c0d3f10 2024-07-17T08:44:46.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.984+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c0cef90 con 0x7fa15c0d3f10 2024-07-17T08:44:46.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.985+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fa154004420 con 0x7fa15c0d3f10 2024-07-17T08:44:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.986+0000 7fa153e00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 0x7fa13c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.986+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fa1540b97d0 con 0x7fa15c0d3f10 2024-07-17T08:44:46.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.987+0000 7fa15be00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 0x7fa13c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:46.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.988+0000 7fa15be00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 0x7fa13c079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c009040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:46.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:46.991+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa1540825e0 con 0x7fa15c0d3f10 2024-07-17T08:44:47.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.245+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa15c0b4440 con 0x7fa15c0d3f10 2024-07-17T08:44:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.247+0000 7fa153e00640 1 -- 172.21.15.44:0/1110841190 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7fa154081ee0 con 0x7fa15c0d3f10 2024-07-17T08:44:47.248 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:47.248 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:44:47.248 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 msgr2=0x7fa13c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 0x7fa13c079190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c009040 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 msgr2=0x7fa15c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7fa154009b30 tx=0x7fa154032d10 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 shutdown_connections 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa13c076cd0 0x7fa13c079190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa15c0d3f10 0x7fa15c15aa70 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15c0d3420 0x7fa15c15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 --2- 172.21.15.44:0/1110841190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa15c0cded0 0x7fa15c160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 >> 172.21.15.44:0/1110841190 conn(0x7fa15c08ca20 msgr2=0x7fa15c0b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 shutdown_connections 2024-07-17T08:44:47.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.250+0000 7fa161c00640 1 -- 172.21.15.44:0/1110841190 wait complete. 2024-07-17T08:44:47.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:44:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 -- 172.21.15.44:0/1042595446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc858003ac0 msgr2=0x7fc858003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 --2- 172.21.15.44:0/1042595446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc858003ac0 0x7fc858003ee0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7fc8640016c0 tx=0x7fc858005830 comp rx=0 tx=0).stop 2024-07-17T08:44:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 -- 172.21.15.44:0/1042595446 shutdown_connections 2024-07-17T08:44:47.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 --2- 172.21.15.44:0/1042595446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc858003ac0 0x7fc858003ee0 unknown :-1 s=CLOSED pgs=3179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 -- 172.21.15.44:0/1042595446 >> 172.21.15.44:0/1042595446 conn(0x7fc8740b0e80 msgr2=0x7fc8740b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 -- 172.21.15.44:0/1042595446 shutdown_connections 2024-07-17T08:44:47.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.473+0000 7fc878a00640 1 -- 172.21.15.44:0/1042595446 wait complete. 2024-07-17T08:44:47.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.475+0000 7fc878a00640 1 Processor -- start 2024-07-17T08:44:47.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.475+0000 7fc878a00640 1 -- start start 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc858003ac0 0x7fc8741e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8740ad510 0x7fc8740ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc8740cf710 con 0x7fc87414d5d0 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8740cf590 con 0x7fc8740ad510 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.476+0000 7fc878a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8740cf890 con 0x7fc858003ac0 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50092/0 (socket says 172.21.15.44:50092) 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 -- 172.21.15.44:0/2300030545 learned_addr learned my addr 172.21.15.44:0/2300030545 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:47.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc873e00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8740ad510 0x7fc8740ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc873400640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc858003ac0 0x7fc8741e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 -- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc858003ac0 msgr2=0x7fc8741e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc858003ac0 0x7fc8741e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 -- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8740ad510 msgr2=0x7fc8740ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8740ad510 0x7fc8740ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 -- 172.21.15.44:0/2300030545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8741f4430 con 0x7fc87414d5d0 2024-07-17T08:44:47.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.477+0000 7fc86be00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d920 tx=0x7fc85c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.478+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0016a0 con 0x7fc87414d5d0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.478+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc85c00b830 con 0x7fc87414d5d0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.478+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00bc60 con 0x7fc87414d5d0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.478+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8741f4710 con 0x7fc87414d5d0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.478+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87414df70 con 0x7fc87414d5d0 2024-07-17T08:44:47.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.479+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8741f4e30 con 0x7fc87414d5d0 2024-07-17T08:44:47.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.480+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fc85c0035f0 con 0x7fc87414d5d0 2024-07-17T08:44:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.481+0000 7fc86b400640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 0x7fc8500791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.481+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fc85c098ce0 con 0x7fc87414d5d0 2024-07-17T08:44:47.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.482+0000 7fc873400640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 0x7fc8500791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.483+0000 7fc873400640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 0x7fc8500791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc864032000 tx=0x7fc864001190 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:47.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.485+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc85c061c90 con 0x7fc87414d5d0 2024-07-17T08:44:47.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.697+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc874159af0 con 0x7fc87414d5d0 2024-07-17T08:44:47.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.700+0000 7fc86b400640 1 -- 172.21.15.44:0/2300030545 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v351) ==== 74+0+17911 (secure 0 0 0) 0x7fc85c061590 con 0x7fc87414d5d0 2024-07-17T08:44:47.701 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:47.701 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":351,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:44.687779+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":47,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:44:41.136788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":48,"pool_name":"cephfs_data","create_time":"2024-07-17T08:44:42.144654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"351","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:44:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 msgr2=0x7fc8500791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 0x7fc8500791c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc864032000 tx=0x7fc864001190 comp rx=0 tx=0).stop 2024-07-17T08:44:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 msgr2=0x7fc87414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d920 tx=0x7fc85c00dde0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 shutdown_connections 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc850076d00 0x7fc8500791c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8740ad510 0x7fc8740ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87414d5d0 0x7fc87414da30 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 --2- 172.21.15.44:0/2300030545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc858003ac0 0x7fc8741e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.703+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 >> 172.21.15.44:0/2300030545 conn(0x7fc8740b0e80 msgr2=0x7fc8740b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.704+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 shutdown_connections 2024-07-17T08:44:47.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.704+0000 7fc878a00640 1 -- 172.21.15.44:0/2300030545 wait complete. 2024-07-17T08:44:47.723 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:44:47.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:44:47.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 -- 172.21.15.44:0/2330880813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c163ce0 msgr2=0x7f360c1660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 --2- 172.21.15.44:0/2330880813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c163ce0 0x7f360c1660d0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7f3604005fd0 tx=0x7f360402a8b0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 -- 172.21.15.44:0/2330880813 shutdown_connections 2024-07-17T08:44:47.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 --2- 172.21.15.44:0/2330880813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c163ce0 0x7f360c1660d0 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 -- 172.21.15.44:0/2330880813 >> 172.21.15.44:0/2330880813 conn(0x7f360c0b0e80 msgr2=0x7f360c0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:47.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 -- 172.21.15.44:0/2330880813 shutdown_connections 2024-07-17T08:44:47.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.925+0000 7f360b200640 1 -- 172.21.15.44:0/2330880813 wait complete. 2024-07-17T08:44:47.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.927+0000 7f360b200640 1 Processor -- start 2024-07-17T08:44:47.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.927+0000 7f360b200640 1 -- start start 2024-07-17T08:44:47.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f360c0cf920 0x7f360c06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f360c163ce0 0x7f360c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0ac270 con 0x7f360c0cf540 2024-07-17T08:44:47.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0aea90 con 0x7f360c163ce0 2024-07-17T08:44:47.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f360b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f360c166610 con 0x7f360c0cf920 2024-07-17T08:44:47.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f3609e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f3609e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50112/0 (socket says 172.21.15.44:50112) 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.928+0000 7f3609e00640 1 -- 172.21.15.44:0/3040512017 learned_addr learned my addr 172.21.15.44:0/3040512017 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f360a800640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f360c163ce0 0x7f360c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609400640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f360c0cf920 0x7f360c06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 -- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f360c0cf920 msgr2=0x7f360c06bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f360c0cf920 0x7f360c06bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 -- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f360c163ce0 msgr2=0x7f360c06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f360c163ce0 0x7f360c06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 -- 172.21.15.44:0/3040512017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c1648e0 con 0x7f360c0cf540 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3609e00640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c420 tx=0x7f35fc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc006890 con 0x7f360c0cf540 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35fc006b20 con 0x7f360c0cf540 2024-07-17T08:44:47.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c164bc0 con 0x7f360c0cf540 2024-07-17T08:44:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.929+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc007020 con 0x7f360c0cf540 2024-07-17T08:44:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.930+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c165180 con 0x7f360c0cf540 2024-07-17T08:44:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.931+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c0b3870 con 0x7f360c0cf540 2024-07-17T08:44:47.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.932+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f35fc019020 con 0x7f360c0cf540 2024-07-17T08:44:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.933+0000 7f3602000640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 0x7f35ec079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:47.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.933+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f35fc098080 con 0x7f360c0cf540 2024-07-17T08:44:47.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.934+0000 7f3609400640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 0x7f35ec079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:47.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.935+0000 7f3609400640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 0x7f35ec079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35f8007fd0 tx=0x7f35f8009040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:47.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:47.937+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f35fc061040 con 0x7f360c0cf540 2024-07-17T08:44:48.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.146+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 --> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f360c1373f0 con 0x7f35ec076cd0 2024-07-17T08:44:48.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.164+0000 7f3602000640 1 -- 172.21.15.44:0/3040512017 <== mgr.9797 v2:172.21.15.175:6834/3157277076 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252504 (secure 0 0 0) 0x7f360c1373f0 con 0x7f35ec076cd0 2024-07-17T08:44:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:48.170 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 msgr2=0x7f35ec079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 0x7f35ec079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35f8007fd0 tx=0x7f35f8009040 comp rx=0 tx=0).stop 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 msgr2=0x7f360c164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c420 tx=0x7f35fc00c8e0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 shutdown_connections 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f35ec076cd0 0x7f35ec079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f360c163ce0 0x7f360c06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f360c0cf920 0x7f360c06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 --2- 172.21.15.44:0/3040512017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f360c0cf540 0x7f360c164170 unknown :-1 s=CLOSED pgs=3183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.170+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 >> 172.21.15.44:0/3040512017 conn(0x7f360c0b0e80 msgr2=0x7f360c13a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.171+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 shutdown_connections 2024-07-17T08:44:48.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.171+0000 7f360b200640 1 -- 172.21.15.44:0/3040512017 wait complete. 2024-07-17T08:44:48.191 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:44:47.355563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":585,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4690,"ondisk_log_size":4690,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":509036,"kb_used_data":36080,"kb_used_omap":12,"kb_used_meta":472819,"kb_avail":754465684,"statfs":{"total":773094113280,"available":772572860416,"internally_reserved":0,"allocated":36945920,"data_stored":20910900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":484167216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.701009"},"pg_stats":[{"pgid":"48.3f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087078+0000","last_change":"2024-07-17T08:44:43.719558+0000","last_active":"2024-07-17T08:44:45.087078+0000","last_peered":"2024-07-17T08:44:45.087078+0000","last_clean":"2024-07-17T08:44:45.087078+0000","last_became_active":"2024-07-17T08:44:43.719273+0000","last_became_peered":"2024-07-17T08:44:43.719273+0000","last_unstale":"2024-07-17T08:44:45.087078+0000","last_undegraded":"2024-07-17T08:44:45.087078+0000","last_fullsized":"2024-07-17T08:44:45.087078+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:27:27.460894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.3e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699179+0000","last_change":"2024-07-17T08:44:43.728200+0000","last_active":"2024-07-17T08:44:44.699179+0000","last_peered":"2024-07-17T08:44:44.699179+0000","last_clean":"2024-07-17T08:44:44.699179+0000","last_became_active":"2024-07-17T08:44:43.727916+0000","last_became_peered":"2024-07-17T08:44:43.727916+0000","last_unstale":"2024-07-17T08:44:44.699179+0000","last_undegraded":"2024-07-17T08:44:44.699179+0000","last_fullsized":"2024-07-17T08:44:44.699179+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:20.430634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.3d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872831+0000","last_change":"2024-07-17T08:44:43.738368+0000","last_active":"2024-07-17T08:44:44.872831+0000","last_peered":"2024-07-17T08:44:44.872831+0000","last_clean":"2024-07-17T08:44:44.872831+0000","last_became_active":"2024-07-17T08:44:43.736733+0000","last_became_peered":"2024-07-17T08:44:43.736733+0000","last_unstale":"2024-07-17T08:44:44.872831+0000","last_undegraded":"2024-07-17T08:44:44.872831+0000","last_fullsized":"2024-07-17T08:44:44.872831+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:25.772020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.3c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694795+0000","last_change":"2024-07-17T08:44:43.725846+0000","last_active":"2024-07-17T08:44:44.694795+0000","last_peered":"2024-07-17T08:44:44.694795+0000","last_clean":"2024-07-17T08:44:44.694795+0000","last_became_active":"2024-07-17T08:44:43.725537+0000","last_became_peered":"2024-07-17T08:44:43.725537+0000","last_unstale":"2024-07-17T08:44:44.694795+0000","last_undegraded":"2024-07-17T08:44:44.694795+0000","last_fullsized":"2024-07-17T08:44:44.694795+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:14.221997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"48.3b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084969+0000","last_change":"2024-07-17T08:44:43.720084+0000","last_active":"2024-07-17T08:44:45.084969+0000","last_peered":"2024-07-17T08:44:45.084969+0000","last_clean":"2024-07-17T08:44:45.084969+0000","last_became_active":"2024-07-17T08:44:43.716569+0000","last_became_peered":"2024-07-17T08:44:43.716569+0000","last_unstale":"2024-07-17T08:44:45.084969+0000","last_undegraded":"2024-07-17T08:44:45.084969+0000","last_fullsized":"2024-07-17T08:44:45.084969+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:17.025559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.3a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083183+0000","last_change":"2024-07-17T08:44:43.730424+0000","last_active":"2024-07-17T08:44:45.083183+0000","last_peered":"2024-07-17T08:44:45.083183+0000","last_clean":"2024-07-17T08:44:45.083183+0000","last_became_active":"2024-07-17T08:44:43.729158+0000","last_became_peered":"2024-07-17T08:44:43.729158+0000","last_unstale":"2024-07-17T08:44:45.083183+0000","last_undegraded":"2024-07-17T08:44:45.083183+0000","last_fullsized":"2024-07-17T08:44:45.083183+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:59.846577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.39","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083472+0000","last_change":"2024-07-17T08:44:43.730916+0000","last_active":"2024-07-17T08:44:45.083472+0000","last_peered":"2024-07-17T08:44:45.083472+0000","last_clean":"2024-07-17T08:44:45.083472+0000","last_became_active":"2024-07-17T08:44:43.730400+0000","last_became_peered":"2024-07-17T08:44:43.730400+0000","last_unstale":"2024-07-17T08:44:45.083472+0000","last_undegraded":"2024-07-17T08:44:45.083472+0000","last_fullsized":"2024-07-17T08:44:45.083472+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:35.284971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.38","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084256+0000","last_change":"2024-07-17T08:44:43.736588+0000","last_active":"2024-07-17T08:44:45.084256+0000","last_peered":"2024-07-17T08:44:45.084256+0000","last_clean":"2024-07-17T08:44:45.084256+0000","last_became_active":"2024-07-17T08:44:43.736226+0000","last_became_peered":"2024-07-17T08:44:43.736226+0000","last_unstale":"2024-07-17T08:44:45.084256+0000","last_undegraded":"2024-07-17T08:44:45.084256+0000","last_fullsized":"2024-07-17T08:44:45.084256+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:25.630273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.37","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699644+0000","last_change":"2024-07-17T08:44:43.727410+0000","last_active":"2024-07-17T08:44:44.699644+0000","last_peered":"2024-07-17T08:44:44.699644+0000","last_clean":"2024-07-17T08:44:44.699644+0000","last_became_active":"2024-07-17T08:44:43.725512+0000","last_became_peered":"2024-07-17T08:44:43.725512+0000","last_unstale":"2024-07-17T08:44:44.699644+0000","last_undegraded":"2024-07-17T08:44:44.699644+0000","last_fullsized":"2024-07-17T08:44:44.699644+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:10.590023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.36","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085319+0000","last_change":"2024-07-17T08:44:43.727190+0000","last_active":"2024-07-17T08:44:45.085319+0000","last_peered":"2024-07-17T08:44:45.085319+0000","last_clean":"2024-07-17T08:44:45.085319+0000","last_became_active":"2024-07-17T08:44:43.726841+0000","last_became_peered":"2024-07-17T08:44:43.726841+0000","last_unstale":"2024-07-17T08:44:45.085319+0000","last_undegraded":"2024-07-17T08:44:45.085319+0000","last_fullsized":"2024-07-17T08:44:45.085319+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:32.501614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.35","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086425+0000","last_change":"2024-07-17T08:44:43.722529+0000","last_active":"2024-07-17T08:44:45.086425+0000","last_peered":"2024-07-17T08:44:45.086425+0000","last_clean":"2024-07-17T08:44:45.086425+0000","last_became_active":"2024-07-17T08:44:43.722232+0000","last_became_peered":"2024-07-17T08:44:43.722232+0000","last_unstale":"2024-07-17T08:44:45.086425+0000","last_undegraded":"2024-07-17T08:44:45.086425+0000","last_fullsized":"2024-07-17T08:44:45.086425+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:52.184906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.34","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085850+0000","last_change":"2024-07-17T08:44:43.721076+0000","last_active":"2024-07-17T08:44:45.085850+0000","last_peered":"2024-07-17T08:44:45.085850+0000","last_clean":"2024-07-17T08:44:45.085850+0000","last_became_active":"2024-07-17T08:44:43.716996+0000","last_became_peered":"2024-07-17T08:44:43.716996+0000","last_unstale":"2024-07-17T08:44:45.085850+0000","last_undegraded":"2024-07-17T08:44:45.085850+0000","last_fullsized":"2024-07-17T08:44:45.085850+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:01:22.257737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.33","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698968+0000","last_change":"2024-07-17T08:44:43.729061+0000","last_active":"2024-07-17T08:44:44.698968+0000","last_peered":"2024-07-17T08:44:44.698968+0000","last_clean":"2024-07-17T08:44:44.698968+0000","last_became_active":"2024-07-17T08:44:43.728420+0000","last_became_peered":"2024-07-17T08:44:43.728420+0000","last_unstale":"2024-07-17T08:44:44.698968+0000","last_undegraded":"2024-07-17T08:44:44.698968+0000","last_fullsized":"2024-07-17T08:44:44.698968+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:17.354426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.32","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085511+0000","last_change":"2024-07-17T08:44:43.719897+0000","last_active":"2024-07-17T08:44:45.085511+0000","last_peered":"2024-07-17T08:44:45.085511+0000","last_clean":"2024-07-17T08:44:45.085511+0000","last_became_active":"2024-07-17T08:44:43.719602+0000","last_became_peered":"2024-07-17T08:44:43.719602+0000","last_unstale":"2024-07-17T08:44:45.085511+0000","last_undegraded":"2024-07-17T08:44:45.085511+0000","last_fullsized":"2024-07-17T08:44:45.085511+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:39:02.980616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.30","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872118+0000","last_change":"2024-07-17T08:44:43.719267+0000","last_active":"2024-07-17T08:44:44.872118+0000","last_peered":"2024-07-17T08:44:44.872118+0000","last_clean":"2024-07-17T08:44:44.872118+0000","last_became_active":"2024-07-17T08:44:43.718968+0000","last_became_peered":"2024-07-17T08:44:43.718968+0000","last_unstale":"2024-07-17T08:44:44.872118+0000","last_undegraded":"2024-07-17T08:44:44.872118+0000","last_fullsized":"2024-07-17T08:44:44.872118+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:57:25.299985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.2f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699115+0000","last_change":"2024-07-17T08:44:43.720002+0000","last_active":"2024-07-17T08:44:44.699115+0000","last_peered":"2024-07-17T08:44:44.699115+0000","last_clean":"2024-07-17T08:44:44.699115+0000","last_became_active":"2024-07-17T08:44:43.719727+0000","last_became_peered":"2024-07-17T08:44:43.719727+0000","last_unstale":"2024-07-17T08:44:44.699115+0000","last_undegraded":"2024-07-17T08:44:44.699115+0000","last_fullsized":"2024-07-17T08:44:44.699115+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:14.492434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.2e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086089+0000","last_change":"2024-07-17T08:44:43.733583+0000","last_active":"2024-07-17T08:44:45.086089+0000","last_peered":"2024-07-17T08:44:45.086089+0000","last_clean":"2024-07-17T08:44:45.086089+0000","last_became_active":"2024-07-17T08:44:43.733298+0000","last_became_peered":"2024-07-17T08:44:43.733298+0000","last_unstale":"2024-07-17T08:44:45.086089+0000","last_undegraded":"2024-07-17T08:44:45.086089+0000","last_fullsized":"2024-07-17T08:44:45.086089+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:28:07.161262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.2d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084889+0000","last_change":"2024-07-17T08:44:43.727539+0000","last_active":"2024-07-17T08:44:45.084889+0000","last_peered":"2024-07-17T08:44:45.084889+0000","last_clean":"2024-07-17T08:44:45.084889+0000","last_became_active":"2024-07-17T08:44:43.726648+0000","last_became_peered":"2024-07-17T08:44:43.726648+0000","last_unstale":"2024-07-17T08:44:45.084889+0000","last_undegraded":"2024-07-17T08:44:45.084889+0000","last_fullsized":"2024-07-17T08:44:45.084889+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:20.178735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.2c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873182+0000","last_change":"2024-07-17T08:44:43.727798+0000","last_active":"2024-07-17T08:44:44.873182+0000","last_peered":"2024-07-17T08:44:44.873182+0000","last_clean":"2024-07-17T08:44:44.873182+0000","last_became_active":"2024-07-17T08:44:43.727493+0000","last_became_peered":"2024-07-17T08:44:43.727493+0000","last_unstale":"2024-07-17T08:44:44.873182+0000","last_undegraded":"2024-07-17T08:44:44.873182+0000","last_fullsized":"2024-07-17T08:44:44.873182+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:21.021834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.2b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699028+0000","last_change":"2024-07-17T08:44:43.729340+0000","last_active":"2024-07-17T08:44:44.699028+0000","last_peered":"2024-07-17T08:44:44.699028+0000","last_clean":"2024-07-17T08:44:44.699028+0000","last_became_active":"2024-07-17T08:44:43.728795+0000","last_became_peered":"2024-07-17T08:44:43.728795+0000","last_unstale":"2024-07-17T08:44:44.699028+0000","last_undegraded":"2024-07-17T08:44:44.699028+0000","last_fullsized":"2024-07-17T08:44:44.699028+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:42.360113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.2a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872751+0000","last_change":"2024-07-17T08:44:43.736059+0000","last_active":"2024-07-17T08:44:44.872751+0000","last_peered":"2024-07-17T08:44:44.872751+0000","last_clean":"2024-07-17T08:44:44.872751+0000","last_became_active":"2024-07-17T08:44:43.734414+0000","last_became_peered":"2024-07-17T08:44:43.734414+0000","last_unstale":"2024-07-17T08:44:44.872751+0000","last_undegraded":"2024-07-17T08:44:44.872751+0000","last_fullsized":"2024-07-17T08:44:44.872751+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:31.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.29","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084200+0000","last_change":"2024-07-17T08:44:43.713679+0000","last_active":"2024-07-17T08:44:45.084200+0000","last_peered":"2024-07-17T08:44:45.084200+0000","last_clean":"2024-07-17T08:44:45.084200+0000","last_became_active":"2024-07-17T08:44:43.713253+0000","last_became_peered":"2024-07-17T08:44:43.713253+0000","last_unstale":"2024-07-17T08:44:45.084200+0000","last_undegraded":"2024-07-17T08:44:45.084200+0000","last_fullsized":"2024-07-17T08:44:45.084200+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:37.577503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.28","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698099+0000","last_change":"2024-07-17T08:44:43.733632+0000","last_active":"2024-07-17T08:44:44.698099+0000","last_peered":"2024-07-17T08:44:44.698099+0000","last_clean":"2024-07-17T08:44:44.698099+0000","last_became_active":"2024-07-17T08:44:43.732935+0000","last_became_peered":"2024-07-17T08:44:43.732935+0000","last_unstale":"2024-07-17T08:44:44.698099+0000","last_undegraded":"2024-07-17T08:44:44.698099+0000","last_fullsized":"2024-07-17T08:44:44.698099+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:02.745766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.27","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085553+0000","last_change":"2024-07-17T08:44:43.738381+0000","last_active":"2024-07-17T08:44:45.085553+0000","last_peered":"2024-07-17T08:44:45.085553+0000","last_clean":"2024-07-17T08:44:45.085553+0000","last_became_active":"2024-07-17T08:44:43.737969+0000","last_became_peered":"2024-07-17T08:44:43.737969+0000","last_unstale":"2024-07-17T08:44:45.085553+0000","last_undegraded":"2024-07-17T08:44:45.085553+0000","last_fullsized":"2024-07-17T08:44:45.085553+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:49.930350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.26","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085373+0000","last_change":"2024-07-17T08:44:43.726173+0000","last_active":"2024-07-17T08:44:45.085373+0000","last_peered":"2024-07-17T08:44:45.085373+0000","last_clean":"2024-07-17T08:44:45.085373+0000","last_became_active":"2024-07-17T08:44:43.725860+0000","last_became_peered":"2024-07-17T08:44:43.725860+0000","last_unstale":"2024-07-17T08:44:45.085373+0000","last_undegraded":"2024-07-17T08:44:45.085373+0000","last_fullsized":"2024-07-17T08:44:45.085373+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:26.120877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.25","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699803+0000","last_change":"2024-07-17T08:44:43.725207+0000","last_active":"2024-07-17T08:44:44.699803+0000","last_peered":"2024-07-17T08:44:44.699803+0000","last_clean":"2024-07-17T08:44:44.699803+0000","last_became_active":"2024-07-17T08:44:43.724906+0000","last_became_peered":"2024-07-17T08:44:43.724906+0000","last_unstale":"2024-07-17T08:44:44.699803+0000","last_undegraded":"2024-07-17T08:44:44.699803+0000","last_fullsized":"2024-07-17T08:44:44.699803+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:45:14.517205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.24","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086267+0000","last_change":"2024-07-17T08:44:43.716638+0000","last_active":"2024-07-17T08:44:45.086267+0000","last_peered":"2024-07-17T08:44:45.086267+0000","last_clean":"2024-07-17T08:44:45.086267+0000","last_became_active":"2024-07-17T08:44:43.716309+0000","last_became_peered":"2024-07-17T08:44:43.716309+0000","last_unstale":"2024-07-17T08:44:45.086267+0000","last_undegraded":"2024-07-17T08:44:45.086267+0000","last_fullsized":"2024-07-17T08:44:45.086267+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:26.668369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.23","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085596+0000","last_change":"2024-07-17T08:44:43.716521+0000","last_active":"2024-07-17T08:44:45.085596+0000","last_peered":"2024-07-17T08:44:45.085596+0000","last_clean":"2024-07-17T08:44:45.085596+0000","last_became_active":"2024-07-17T08:44:43.716203+0000","last_became_peered":"2024-07-17T08:44:43.716203+0000","last_unstale":"2024-07-17T08:44:45.085596+0000","last_undegraded":"2024-07-17T08:44:45.085596+0000","last_fullsized":"2024-07-17T08:44:45.085596+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:45.741312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.22","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085931+0000","last_change":"2024-07-17T08:44:43.732786+0000","last_active":"2024-07-17T08:44:45.085931+0000","last_peered":"2024-07-17T08:44:45.085931+0000","last_clean":"2024-07-17T08:44:45.085931+0000","last_became_active":"2024-07-17T08:44:43.732484+0000","last_became_peered":"2024-07-17T08:44:43.732484+0000","last_unstale":"2024-07-17T08:44:45.085931+0000","last_undegraded":"2024-07-17T08:44:45.085931+0000","last_fullsized":"2024-07-17T08:44:45.085931+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:09.524217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.21","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872344+0000","last_change":"2024-07-17T08:44:43.735714+0000","last_active":"2024-07-17T08:44:44.872344+0000","last_peered":"2024-07-17T08:44:44.872344+0000","last_clean":"2024-07-17T08:44:44.872344+0000","last_became_active":"2024-07-17T08:44:43.733396+0000","last_became_peered":"2024-07-17T08:44:43.733396+0000","last_unstale":"2024-07-17T08:44:44.872344+0000","last_undegraded":"2024-07-17T08:44:44.872344+0000","last_fullsized":"2024-07-17T08:44:44.872344+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:49.203341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.20","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083401+0000","last_change":"2024-07-17T08:44:43.702459+0000","last_active":"2024-07-17T08:44:45.083401+0000","last_peered":"2024-07-17T08:44:45.083401+0000","last_clean":"2024-07-17T08:44:45.083401+0000","last_became_active":"2024-07-17T08:44:43.702133+0000","last_became_peered":"2024-07-17T08:44:43.702133+0000","last_unstale":"2024-07-17T08:44:45.083401+0000","last_undegraded":"2024-07-17T08:44:45.083401+0000","last_fullsized":"2024-07-17T08:44:45.083401+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:21.003815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.0","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694390+0000","last_change":"2024-07-17T08:44:43.732424+0000","last_active":"2024-07-17T08:44:44.694390+0000","last_peered":"2024-07-17T08:44:44.694390+0000","last_clean":"2024-07-17T08:44:44.694390+0000","last_became_active":"2024-07-17T08:44:43.728310+0000","last_became_peered":"2024-07-17T08:44:43.728310+0000","last_unstale":"2024-07-17T08:44:44.694390+0000","last_undegraded":"2024-07-17T08:44:44.694390+0000","last_fullsized":"2024-07-17T08:44:44.694390+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:00.730177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.1f","version":"350'4","reported_seq":18,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872757+0000","last_change":"2024-07-17T08:44:42.692992+0000","last_active":"2024-07-17T08:44:44.872757+0000","last_peered":"2024-07-17T08:44:44.872757+0000","last_clean":"2024-07-17T08:44:44.872757+0000","last_became_active":"2024-07-17T08:44:42.692639+0000","last_became_peered":"2024-07-17T08:44:42.692639+0000","last_unstale":"2024-07-17T08:44:44.872757+0000","last_undegraded":"2024-07-17T08:44:44.872757+0000","last_fullsized":"2024-07-17T08:44:44.872757+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:21:04.240811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.1","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872983+0000","last_change":"2024-07-17T08:44:43.730428+0000","last_active":"2024-07-17T08:44:44.872983+0000","last_peered":"2024-07-17T08:44:44.872983+0000","last_clean":"2024-07-17T08:44:44.872983+0000","last_became_active":"2024-07-17T08:44:43.730114+0000","last_became_peered":"2024-07-17T08:44:43.730114+0000","last_unstale":"2024-07-17T08:44:44.872983+0000","last_undegraded":"2024-07-17T08:44:44.872983+0000","last_fullsized":"2024-07-17T08:44:44.872983+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:39:20.063168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1e","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698399+0000","last_change":"2024-07-17T08:44:42.701350+0000","last_active":"2024-07-17T08:44:44.698399+0000","last_peered":"2024-07-17T08:44:44.698399+0000","last_clean":"2024-07-17T08:44:44.698399+0000","last_became_active":"2024-07-17T08:44:42.701061+0000","last_became_peered":"2024-07-17T08:44:42.701061+0000","last_unstale":"2024-07-17T08:44:44.698399+0000","last_undegraded":"2024-07-17T08:44:44.698399+0000","last_fullsized":"2024-07-17T08:44:44.698399+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:32.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.2","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698466+0000","last_change":"2024-07-17T08:44:43.717561+0000","last_active":"2024-07-17T08:44:44.698466+0000","last_peered":"2024-07-17T08:44:44.698466+0000","last_clean":"2024-07-17T08:44:44.698466+0000","last_became_active":"2024-07-17T08:44:43.714594+0000","last_became_peered":"2024-07-17T08:44:43.714594+0000","last_unstale":"2024-07-17T08:44:44.698466+0000","last_undegraded":"2024-07-17T08:44:44.698466+0000","last_fullsized":"2024-07-17T08:44:44.698466+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:34.851165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.1d","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086374+0000","last_change":"2024-07-17T08:44:42.699132+0000","last_active":"2024-07-17T08:44:45.086374+0000","last_peered":"2024-07-17T08:44:45.086374+0000","last_clean":"2024-07-17T08:44:45.086374+0000","last_became_active":"2024-07-17T08:44:42.698812+0000","last_became_peered":"2024-07-17T08:44:42.698812+0000","last_unstale":"2024-07-17T08:44:45.086374+0000","last_undegraded":"2024-07-17T08:44:45.086374+0000","last_fullsized":"2024-07-17T08:44:45.086374+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:42:23.317048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.3","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698322+0000","last_change":"2024-07-17T08:44:43.727218+0000","last_active":"2024-07-17T08:44:44.698322+0000","last_peered":"2024-07-17T08:44:44.698322+0000","last_clean":"2024-07-17T08:44:44.698322+0000","last_became_active":"2024-07-17T08:44:43.726093+0000","last_became_peered":"2024-07-17T08:44:43.726093+0000","last_unstale":"2024-07-17T08:44:44.698322+0000","last_undegraded":"2024-07-17T08:44:44.698322+0000","last_fullsized":"2024-07-17T08:44:44.698322+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:03.235964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.1c","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086227+0000","last_change":"2024-07-17T08:44:42.699144+0000","last_active":"2024-07-17T08:44:45.086227+0000","last_peered":"2024-07-17T08:44:45.086227+0000","last_clean":"2024-07-17T08:44:45.086227+0000","last_became_active":"2024-07-17T08:44:42.698803+0000","last_became_peered":"2024-07-17T08:44:42.698803+0000","last_unstale":"2024-07-17T08:44:45.086227+0000","last_undegraded":"2024-07-17T08:44:45.086227+0000","last_fullsized":"2024-07-17T08:44:45.086227+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:59.316118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.4","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694900+0000","last_change":"2024-07-17T08:44:43.707267+0000","last_active":"2024-07-17T08:44:44.694900+0000","last_peered":"2024-07-17T08:44:44.694900+0000","last_clean":"2024-07-17T08:44:44.694900+0000","last_became_active":"2024-07-17T08:44:43.704746+0000","last_became_peered":"2024-07-17T08:44:43.704746+0000","last_unstale":"2024-07-17T08:44:44.694900+0000","last_undegraded":"2024-07-17T08:44:44.694900+0000","last_fullsized":"2024-07-17T08:44:44.694900+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:00.865925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.1b","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085895+0000","last_change":"2024-07-17T08:44:42.698333+0000","last_active":"2024-07-17T08:44:45.085895+0000","last_peered":"2024-07-17T08:44:45.085895+0000","last_clean":"2024-07-17T08:44:45.085895+0000","last_became_active":"2024-07-17T08:44:42.698045+0000","last_became_peered":"2024-07-17T08:44:42.698045+0000","last_unstale":"2024-07-17T08:44:45.085895+0000","last_undegraded":"2024-07-17T08:44:45.085895+0000","last_fullsized":"2024-07-17T08:44:45.085895+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:25.104223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.5","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699379+0000","last_change":"2024-07-17T08:44:43.718594+0000","last_active":"2024-07-17T08:44:44.699379+0000","last_peered":"2024-07-17T08:44:44.699379+0000","last_clean":"2024-07-17T08:44:44.699379+0000","last_became_active":"2024-07-17T08:44:43.717801+0000","last_became_peered":"2024-07-17T08:44:43.717801+0000","last_unstale":"2024-07-17T08:44:44.699379+0000","last_undegraded":"2024-07-17T08:44:44.699379+0000","last_fullsized":"2024-07-17T08:44:44.699379+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:56.791167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.1a","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084286+0000","last_change":"2024-07-17T08:44:42.682008+0000","last_active":"2024-07-17T08:44:45.084286+0000","last_peered":"2024-07-17T08:44:45.084286+0000","last_clean":"2024-07-17T08:44:45.084286+0000","last_became_active":"2024-07-17T08:44:42.681711+0000","last_became_peered":"2024-07-17T08:44:42.681711+0000","last_unstale":"2024-07-17T08:44:45.084286+0000","last_undegraded":"2024-07-17T08:44:45.084286+0000","last_fullsized":"2024-07-17T08:44:45.084286+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:52:06.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.6","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087123+0000","last_change":"2024-07-17T08:44:43.732190+0000","last_active":"2024-07-17T08:44:45.087123+0000","last_peered":"2024-07-17T08:44:45.087123+0000","last_clean":"2024-07-17T08:44:45.087123+0000","last_became_active":"2024-07-17T08:44:43.729838+0000","last_became_peered":"2024-07-17T08:44:43.729838+0000","last_unstale":"2024-07-17T08:44:45.087123+0000","last_undegraded":"2024-07-17T08:44:45.087123+0000","last_fullsized":"2024-07-17T08:44:45.087123+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:16.201991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.19","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.697866+0000","last_change":"2024-07-17T08:44:42.701239+0000","last_active":"2024-07-17T08:44:44.697866+0000","last_peered":"2024-07-17T08:44:44.697866+0000","last_clean":"2024-07-17T08:44:44.697866+0000","last_became_active":"2024-07-17T08:44:42.700968+0000","last_became_peered":"2024-07-17T08:44:42.700968+0000","last_unstale":"2024-07-17T08:44:44.697866+0000","last_undegraded":"2024-07-17T08:44:44.697866+0000","last_fullsized":"2024-07-17T08:44:44.697866+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:24.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.7","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699754+0000","last_change":"2024-07-17T08:44:43.732561+0000","last_active":"2024-07-17T08:44:44.699754+0000","last_peered":"2024-07-17T08:44:44.699754+0000","last_clean":"2024-07-17T08:44:44.699754+0000","last_became_active":"2024-07-17T08:44:43.726639+0000","last_became_peered":"2024-07-17T08:44:43.726639+0000","last_unstale":"2024-07-17T08:44:44.699754+0000","last_undegraded":"2024-07-17T08:44:44.699754+0000","last_fullsized":"2024-07-17T08:44:44.699754+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:47.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.18","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872529+0000","last_change":"2024-07-17T08:44:42.683806+0000","last_active":"2024-07-17T08:44:44.872529+0000","last_peered":"2024-07-17T08:44:44.872529+0000","last_clean":"2024-07-17T08:44:44.872529+0000","last_became_active":"2024-07-17T08:44:42.683445+0000","last_became_peered":"2024-07-17T08:44:42.683445+0000","last_unstale":"2024-07-17T08:44:44.872529+0000","last_undegraded":"2024-07-17T08:44:44.872529+0000","last_fullsized":"2024-07-17T08:44:44.872529+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:34.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.8","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084805+0000","last_change":"2024-07-17T08:44:43.729487+0000","last_active":"2024-07-17T08:44:45.084805+0000","last_peered":"2024-07-17T08:44:45.084805+0000","last_clean":"2024-07-17T08:44:45.084805+0000","last_became_active":"2024-07-17T08:44:43.729062+0000","last_became_peered":"2024-07-17T08:44:43.729062+0000","last_unstale":"2024-07-17T08:44:45.084805+0000","last_undegraded":"2024-07-17T08:44:45.084805+0000","last_fullsized":"2024-07-17T08:44:45.084805+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:11.981637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.17","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699262+0000","last_change":"2024-07-17T08:44:42.686378+0000","last_active":"2024-07-17T08:44:44.699262+0000","last_peered":"2024-07-17T08:44:44.699262+0000","last_clean":"2024-07-17T08:44:44.699262+0000","last_became_active":"2024-07-17T08:44:42.685988+0000","last_became_peered":"2024-07-17T08:44:42.685988+0000","last_unstale":"2024-07-17T08:44:44.699262+0000","last_undegraded":"2024-07-17T08:44:44.699262+0000","last_fullsized":"2024-07-17T08:44:44.699262+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:56:03.376791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.9","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084446+0000","last_change":"2024-07-17T08:44:43.730676+0000","last_active":"2024-07-17T08:44:45.084446+0000","last_peered":"2024-07-17T08:44:45.084446+0000","last_clean":"2024-07-17T08:44:45.084446+0000","last_became_active":"2024-07-17T08:44:43.730015+0000","last_became_peered":"2024-07-17T08:44:43.730015+0000","last_unstale":"2024-07-17T08:44:45.084446+0000","last_undegraded":"2024-07-17T08:44:45.084446+0000","last_fullsized":"2024-07-17T08:44:45.084446+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:17.330433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.16","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086885+0000","last_change":"2024-07-17T08:44:42.702785+0000","last_active":"2024-07-17T08:44:45.086885+0000","last_peered":"2024-07-17T08:44:45.086885+0000","last_clean":"2024-07-17T08:44:45.086885+0000","last_became_active":"2024-07-17T08:44:42.702509+0000","last_became_peered":"2024-07-17T08:44:42.702509+0000","last_unstale":"2024-07-17T08:44:45.086885+0000","last_undegraded":"2024-07-17T08:44:45.086885+0000","last_fullsized":"2024-07-17T08:44:45.086885+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:59.849271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694686+0000","last_change":"2024-07-17T08:44:43.707940+0000","last_active":"2024-07-17T08:44:44.694686+0000","last_peered":"2024-07-17T08:44:44.694686+0000","last_clean":"2024-07-17T08:44:44.694686+0000","last_became_active":"2024-07-17T08:44:43.707650+0000","last_became_peered":"2024-07-17T08:44:43.707650+0000","last_unstale":"2024-07-17T08:44:44.694686+0000","last_undegraded":"2024-07-17T08:44:44.694686+0000","last_fullsized":"2024-07-17T08:44:44.694686+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:56.734043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.15","version":"350'3","reported_seq":17,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872754+0000","last_change":"2024-07-17T08:44:42.696442+0000","last_active":"2024-07-17T08:44:44.872754+0000","last_peered":"2024-07-17T08:44:44.872754+0000","last_clean":"2024-07-17T08:44:44.872754+0000","last_became_active":"2024-07-17T08:44:42.696141+0000","last_became_peered":"2024-07-17T08:44:42.696141+0000","last_unstale":"2024-07-17T08:44:44.872754+0000","last_undegraded":"2024-07-17T08:44:44.872754+0000","last_fullsized":"2024-07-17T08:44:44.872754+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:54:15.632379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873142+0000","last_change":"2024-07-17T08:44:43.733058+0000","last_active":"2024-07-17T08:44:44.873142+0000","last_peered":"2024-07-17T08:44:44.873142+0000","last_clean":"2024-07-17T08:44:44.873142+0000","last_became_active":"2024-07-17T08:44:43.731982+0000","last_became_peered":"2024-07-17T08:44:43.731982+0000","last_unstale":"2024-07-17T08:44:44.873142+0000","last_undegraded":"2024-07-17T08:44:44.873142+0000","last_fullsized":"2024-07-17T08:44:44.873142+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:30.056544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.14","version":"350'6","reported_seq":20,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084897+0000","last_change":"2024-07-17T08:44:42.682483+0000","last_active":"2024-07-17T08:44:45.084897+0000","last_peered":"2024-07-17T08:44:45.084897+0000","last_clean":"2024-07-17T08:44:45.084897+0000","last_became_active":"2024-07-17T08:44:42.681967+0000","last_became_peered":"2024-07-17T08:44:42.681967+0000","last_unstale":"2024-07-17T08:44:45.084897+0000","last_undegraded":"2024-07-17T08:44:45.084897+0000","last_fullsized":"2024-07-17T08:44:45.084897+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:36.609958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084956+0000","last_change":"2024-07-17T08:44:43.719781+0000","last_active":"2024-07-17T08:44:45.084956+0000","last_peered":"2024-07-17T08:44:45.084956+0000","last_clean":"2024-07-17T08:44:45.084956+0000","last_became_active":"2024-07-17T08:44:43.719496+0000","last_became_peered":"2024-07-17T08:44:43.719496+0000","last_unstale":"2024-07-17T08:44:45.084956+0000","last_undegraded":"2024-07-17T08:44:45.084956+0000","last_fullsized":"2024-07-17T08:44:45.084956+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:09.458890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.13","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873209+0000","last_change":"2024-07-17T08:44:42.684393+0000","last_active":"2024-07-17T08:44:44.873209+0000","last_peered":"2024-07-17T08:44:44.873209+0000","last_clean":"2024-07-17T08:44:44.873209+0000","last_became_active":"2024-07-17T08:44:42.684095+0000","last_became_peered":"2024-07-17T08:44:42.684095+0000","last_unstale":"2024-07-17T08:44:44.873209+0000","last_undegraded":"2024-07-17T08:44:44.873209+0000","last_fullsized":"2024-07-17T08:44:44.873209+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:21.220207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699456+0000","last_change":"2024-07-17T08:44:43.720369+0000","last_active":"2024-07-17T08:44:44.699456+0000","last_peered":"2024-07-17T08:44:44.699456+0000","last_clean":"2024-07-17T08:44:44.699456+0000","last_became_active":"2024-07-17T08:44:43.719971+0000","last_became_peered":"2024-07-17T08:44:43.719971+0000","last_unstale":"2024-07-17T08:44:44.699456+0000","last_undegraded":"2024-07-17T08:44:44.699456+0000","last_fullsized":"2024-07-17T08:44:44.699456+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:32.249485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.12","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694453+0000","last_change":"2024-07-17T08:44:42.697185+0000","last_active":"2024-07-17T08:44:44.694453+0000","last_peered":"2024-07-17T08:44:44.694453+0000","last_clean":"2024-07-17T08:44:44.694453+0000","last_became_active":"2024-07-17T08:44:42.696755+0000","last_became_peered":"2024-07-17T08:44:42.696755+0000","last_unstale":"2024-07-17T08:44:44.694453+0000","last_undegraded":"2024-07-17T08:44:44.694453+0000","last_fullsized":"2024-07-17T08:44:44.694453+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:16.233733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"48.e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.695102+0000","last_change":"2024-07-17T08:44:43.730594+0000","last_active":"2024-07-17T08:44:44.695102+0000","last_peered":"2024-07-17T08:44:44.695102+0000","last_clean":"2024-07-17T08:44:44.695102+0000","last_became_active":"2024-07-17T08:44:43.730318+0000","last_became_peered":"2024-07-17T08:44:43.730318+0000","last_unstale":"2024-07-17T08:44:44.695102+0000","last_undegraded":"2024-07-17T08:44:44.695102+0000","last_fullsized":"2024-07-17T08:44:44.695102+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:28:47.875138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.11","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085070+0000","last_change":"2024-07-17T08:44:42.694983+0000","last_active":"2024-07-17T08:44:45.085070+0000","last_peered":"2024-07-17T08:44:45.085070+0000","last_clean":"2024-07-17T08:44:45.085070+0000","last_became_active":"2024-07-17T08:44:42.694615+0000","last_became_peered":"2024-07-17T08:44:42.694615+0000","last_unstale":"2024-07-17T08:44:45.085070+0000","last_undegraded":"2024-07-17T08:44:45.085070+0000","last_fullsized":"2024-07-17T08:44:45.085070+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:55.717389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087191+0000","last_change":"2024-07-17T08:44:43.718453+0000","last_active":"2024-07-17T08:44:45.087191+0000","last_peered":"2024-07-17T08:44:45.087191+0000","last_clean":"2024-07-17T08:44:45.087191+0000","last_became_active":"2024-07-17T08:44:43.718162+0000","last_became_peered":"2024-07-17T08:44:43.718162+0000","last_unstale":"2024-07-17T08:44:45.087191+0000","last_undegraded":"2024-07-17T08:44:45.087191+0000","last_fullsized":"2024-07-17T08:44:45.087191+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:44.830658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.10","version":"350'8","reported_seq":22,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083579+0000","last_change":"2024-07-17T08:44:42.693727+0000","last_active":"2024-07-17T08:44:45.083579+0000","last_peered":"2024-07-17T08:44:45.083579+0000","last_clean":"2024-07-17T08:44:45.083579+0000","last_became_active":"2024-07-17T08:44:42.693437+0000","last_became_peered":"2024-07-17T08:44:42.693437+0000","last_unstale":"2024-07-17T08:44:45.083579+0000","last_undegraded":"2024-07-17T08:44:45.083579+0000","last_fullsized":"2024-07-17T08:44:45.083579+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:24.613619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.10","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872325+0000","last_change":"2024-07-17T08:44:43.728538+0000","last_active":"2024-07-17T08:44:44.872325+0000","last_peered":"2024-07-17T08:44:44.872325+0000","last_clean":"2024-07-17T08:44:44.872325+0000","last_became_active":"2024-07-17T08:44:43.722026+0000","last_became_peered":"2024-07-17T08:44:43.722026+0000","last_unstale":"2024-07-17T08:44:44.872325+0000","last_undegraded":"2024-07-17T08:44:44.872325+0000","last_fullsized":"2024-07-17T08:44:44.872325+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:27.823998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.f","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699981+0000","last_change":"2024-07-17T08:44:42.696853+0000","last_active":"2024-07-17T08:44:44.699981+0000","last_peered":"2024-07-17T08:44:44.699981+0000","last_clean":"2024-07-17T08:44:44.699981+0000","last_became_active":"2024-07-17T08:44:42.695919+0000","last_became_peered":"2024-07-17T08:44:42.695919+0000","last_unstale":"2024-07-17T08:44:44.699981+0000","last_undegraded":"2024-07-17T08:44:44.699981+0000","last_fullsized":"2024-07-17T08:44:44.699981+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:45.498511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.11","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083963+0000","last_change":"2024-07-17T08:44:43.725035+0000","last_active":"2024-07-17T08:44:45.083963+0000","last_peered":"2024-07-17T08:44:45.083963+0000","last_clean":"2024-07-17T08:44:45.083963+0000","last_became_active":"2024-07-17T08:44:43.723021+0000","last_became_peered":"2024-07-17T08:44:43.723021+0000","last_unstale":"2024-07-17T08:44:45.083963+0000","last_undegraded":"2024-07-17T08:44:45.083963+0000","last_fullsized":"2024-07-17T08:44:45.083963+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:29.932048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.e","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872763+0000","last_change":"2024-07-17T08:44:42.689712+0000","last_active":"2024-07-17T08:44:44.872763+0000","last_peered":"2024-07-17T08:44:44.872763+0000","last_clean":"2024-07-17T08:44:44.872763+0000","last_became_active":"2024-07-17T08:44:42.688637+0000","last_became_peered":"2024-07-17T08:44:42.688637+0000","last_unstale":"2024-07-17T08:44:44.872763+0000","last_undegraded":"2024-07-17T08:44:44.872763+0000","last_fullsized":"2024-07-17T08:44:44.872763+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:46.193763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.12","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083311+0000","last_change":"2024-07-17T08:44:43.726109+0000","last_active":"2024-07-17T08:44:45.083311+0000","last_peered":"2024-07-17T08:44:45.083311+0000","last_clean":"2024-07-17T08:44:45.083311+0000","last_became_active":"2024-07-17T08:44:43.724087+0000","last_became_peered":"2024-07-17T08:44:43.724087+0000","last_unstale":"2024-07-17T08:44:45.083311+0000","last_undegraded":"2024-07-17T08:44:45.083311+0000","last_fullsized":"2024-07-17T08:44:45.083311+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:52.377147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.d","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086643+0000","last_change":"2024-07-17T08:44:42.699947+0000","last_active":"2024-07-17T08:44:45.086643+0000","last_peered":"2024-07-17T08:44:45.086643+0000","last_clean":"2024-07-17T08:44:45.086643+0000","last_became_active":"2024-07-17T08:44:42.699657+0000","last_became_peered":"2024-07-17T08:44:42.699657+0000","last_unstale":"2024-07-17T08:44:45.086643+0000","last_undegraded":"2024-07-17T08:44:45.086643+0000","last_fullsized":"2024-07-17T08:44:45.086643+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:28:46.858484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.13","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698083+0000","last_change":"2024-07-17T08:44:43.733556+0000","last_active":"2024-07-17T08:44:44.698083+0000","last_peered":"2024-07-17T08:44:44.698083+0000","last_clean":"2024-07-17T08:44:44.698083+0000","last_became_active":"2024-07-17T08:44:43.730184+0000","last_became_peered":"2024-07-17T08:44:43.730184+0000","last_unstale":"2024-07-17T08:44:44.698083+0000","last_undegraded":"2024-07-17T08:44:44.698083+0000","last_fullsized":"2024-07-17T08:44:44.698083+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:29:23.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.c","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086529+0000","last_change":"2024-07-17T08:44:42.698959+0000","last_active":"2024-07-17T08:44:45.086529+0000","last_peered":"2024-07-17T08:44:45.086529+0000","last_clean":"2024-07-17T08:44:45.086529+0000","last_became_active":"2024-07-17T08:44:42.698669+0000","last_became_peered":"2024-07-17T08:44:42.698669+0000","last_unstale":"2024-07-17T08:44:45.086529+0000","last_undegraded":"2024-07-17T08:44:45.086529+0000","last_fullsized":"2024-07-17T08:44:45.086529+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:31.484961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.14","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698348+0000","last_change":"2024-07-17T08:44:43.718712+0000","last_active":"2024-07-17T08:44:44.698348+0000","last_peered":"2024-07-17T08:44:44.698348+0000","last_clean":"2024-07-17T08:44:44.698348+0000","last_became_active":"2024-07-17T08:44:43.718246+0000","last_became_peered":"2024-07-17T08:44:43.718246+0000","last_unstale":"2024-07-17T08:44:44.698348+0000","last_undegraded":"2024-07-17T08:44:44.698348+0000","last_fullsized":"2024-07-17T08:44:44.698348+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:28:37.855550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.b","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873436+0000","last_change":"2024-07-17T08:44:42.685470+0000","last_active":"2024-07-17T08:44:44.873436+0000","last_peered":"2024-07-17T08:44:44.873436+0000","last_clean":"2024-07-17T08:44:44.873436+0000","last_became_active":"2024-07-17T08:44:42.684659+0000","last_became_peered":"2024-07-17T08:44:42.684659+0000","last_unstale":"2024-07-17T08:44:44.873436+0000","last_undegraded":"2024-07-17T08:44:44.873436+0000","last_fullsized":"2024-07-17T08:44:44.873436+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:17:13.434838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.15","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085110+0000","last_change":"2024-07-17T08:44:43.726114+0000","last_active":"2024-07-17T08:44:45.085110+0000","last_peered":"2024-07-17T08:44:45.085110+0000","last_clean":"2024-07-17T08:44:45.085110+0000","last_became_active":"2024-07-17T08:44:43.725817+0000","last_became_peered":"2024-07-17T08:44:43.725817+0000","last_unstale":"2024-07-17T08:44:45.085110+0000","last_undegraded":"2024-07-17T08:44:45.085110+0000","last_fullsized":"2024-07-17T08:44:45.085110+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:00.332772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.a","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084380+0000","last_change":"2024-07-17T08:44:42.698996+0000","last_active":"2024-07-17T08:44:45.084380+0000","last_peered":"2024-07-17T08:44:45.084380+0000","last_clean":"2024-07-17T08:44:45.084380+0000","last_became_active":"2024-07-17T08:44:42.698649+0000","last_became_peered":"2024-07-17T08:44:42.698649+0000","last_unstale":"2024-07-17T08:44:45.084380+0000","last_undegraded":"2024-07-17T08:44:45.084380+0000","last_fullsized":"2024-07-17T08:44:45.084380+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:11.682915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.16","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086578+0000","last_change":"2024-07-17T08:44:43.722432+0000","last_active":"2024-07-17T08:44:45.086578+0000","last_peered":"2024-07-17T08:44:45.086578+0000","last_clean":"2024-07-17T08:44:45.086578+0000","last_became_active":"2024-07-17T08:44:43.722139+0000","last_became_peered":"2024-07-17T08:44:43.722139+0000","last_unstale":"2024-07-17T08:44:45.086578+0000","last_undegraded":"2024-07-17T08:44:45.086578+0000","last_fullsized":"2024-07-17T08:44:45.086578+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:32.647055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.9","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698148+0000","last_change":"2024-07-17T08:44:42.701245+0000","last_active":"2024-07-17T08:44:44.698148+0000","last_peered":"2024-07-17T08:44:44.698148+0000","last_clean":"2024-07-17T08:44:44.698148+0000","last_became_active":"2024-07-17T08:44:42.700948+0000","last_became_peered":"2024-07-17T08:44:42.700948+0000","last_unstale":"2024-07-17T08:44:44.698148+0000","last_undegraded":"2024-07-17T08:44:44.698148+0000","last_fullsized":"2024-07-17T08:44:44.698148+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:53:11.616600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.17","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699480+0000","last_change":"2024-07-17T08:44:43.730549+0000","last_active":"2024-07-17T08:44:44.699480+0000","last_peered":"2024-07-17T08:44:44.699480+0000","last_clean":"2024-07-17T08:44:44.699480+0000","last_became_active":"2024-07-17T08:44:43.730275+0000","last_became_peered":"2024-07-17T08:44:43.730275+0000","last_unstale":"2024-07-17T08:44:44.699480+0000","last_undegraded":"2024-07-17T08:44:44.699480+0000","last_fullsized":"2024-07-17T08:44:44.699480+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:10.286508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.8","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084581+0000","last_change":"2024-07-17T08:44:42.697403+0000","last_active":"2024-07-17T08:44:45.084581+0000","last_peered":"2024-07-17T08:44:45.084581+0000","last_clean":"2024-07-17T08:44:45.084581+0000","last_became_active":"2024-07-17T08:44:42.697045+0000","last_became_peered":"2024-07-17T08:44:42.697045+0000","last_unstale":"2024-07-17T08:44:45.084581+0000","last_undegraded":"2024-07-17T08:44:45.084581+0000","last_fullsized":"2024-07-17T08:44:45.084581+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:13.205343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.18","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084262+0000","last_change":"2024-07-17T08:44:43.730348+0000","last_active":"2024-07-17T08:44:45.084262+0000","last_peered":"2024-07-17T08:44:45.084262+0000","last_clean":"2024-07-17T08:44:45.084262+0000","last_became_active":"2024-07-17T08:44:43.720122+0000","last_became_peered":"2024-07-17T08:44:43.720122+0000","last_unstale":"2024-07-17T08:44:45.084262+0000","last_undegraded":"2024-07-17T08:44:45.084262+0000","last_fullsized":"2024-07-17T08:44:45.084262+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:06.474426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.7","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086927+0000","last_change":"2024-07-17T08:44:42.696788+0000","last_active":"2024-07-17T08:44:45.086927+0000","last_peered":"2024-07-17T08:44:45.086927+0000","last_clean":"2024-07-17T08:44:45.086927+0000","last_became_active":"2024-07-17T08:44:42.696431+0000","last_became_peered":"2024-07-17T08:44:42.696431+0000","last_unstale":"2024-07-17T08:44:45.086927+0000","last_undegraded":"2024-07-17T08:44:45.086927+0000","last_fullsized":"2024-07-17T08:44:45.086927+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:10.964983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.19","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084011+0000","last_change":"2024-07-17T08:44:43.731620+0000","last_active":"2024-07-17T08:44:45.084011+0000","last_peered":"2024-07-17T08:44:45.084011+0000","last_clean":"2024-07-17T08:44:45.084011+0000","last_became_active":"2024-07-17T08:44:43.728927+0000","last_became_peered":"2024-07-17T08:44:43.728927+0000","last_unstale":"2024-07-17T08:44:45.084011+0000","last_undegraded":"2024-07-17T08:44:45.084011+0000","last_fullsized":"2024-07-17T08:44:45.084011+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:07.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.6","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698732+0000","last_change":"2024-07-17T08:44:42.700701+0000","last_active":"2024-07-17T08:44:44.698732+0000","last_peered":"2024-07-17T08:44:44.698732+0000","last_clean":"2024-07-17T08:44:44.698732+0000","last_became_active":"2024-07-17T08:44:42.700370+0000","last_became_peered":"2024-07-17T08:44:42.700370+0000","last_unstale":"2024-07-17T08:44:44.698732+0000","last_undegraded":"2024-07-17T08:44:44.698732+0000","last_fullsized":"2024-07-17T08:44:44.698732+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:56:50.278858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.1a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872315+0000","last_change":"2024-07-17T08:44:43.739617+0000","last_active":"2024-07-17T08:44:44.872315+0000","last_peered":"2024-07-17T08:44:44.872315+0000","last_clean":"2024-07-17T08:44:44.872315+0000","last_became_active":"2024-07-17T08:44:43.739273+0000","last_became_peered":"2024-07-17T08:44:43.739273+0000","last_unstale":"2024-07-17T08:44:44.872315+0000","last_undegraded":"2024-07-17T08:44:44.872315+0000","last_fullsized":"2024-07-17T08:44:44.872315+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:13.895269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.5","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085367+0000","last_change":"2024-07-17T08:44:42.695536+0000","last_active":"2024-07-17T08:44:45.085367+0000","last_peered":"2024-07-17T08:44:45.085367+0000","last_clean":"2024-07-17T08:44:45.085367+0000","last_became_active":"2024-07-17T08:44:42.694458+0000","last_became_peered":"2024-07-17T08:44:42.694458+0000","last_unstale":"2024-07-17T08:44:45.085367+0000","last_undegraded":"2024-07-17T08:44:45.085367+0000","last_fullsized":"2024-07-17T08:44:45.085367+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:39.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.1b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086117+0000","last_change":"2024-07-17T08:44:43.718290+0000","last_active":"2024-07-17T08:44:45.086117+0000","last_peered":"2024-07-17T08:44:45.086117+0000","last_clean":"2024-07-17T08:44:45.086117+0000","last_became_active":"2024-07-17T08:44:43.717658+0000","last_became_peered":"2024-07-17T08:44:43.717658+0000","last_unstale":"2024-07-17T08:44:45.086117+0000","last_undegraded":"2024-07-17T08:44:45.086117+0000","last_fullsized":"2024-07-17T08:44:45.086117+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:27:31.051859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.4","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085182+0000","last_change":"2024-07-17T08:44:42.694414+0000","last_active":"2024-07-17T08:44:45.085182+0000","last_peered":"2024-07-17T08:44:45.085182+0000","last_clean":"2024-07-17T08:44:45.085182+0000","last_became_active":"2024-07-17T08:44:42.694121+0000","last_became_peered":"2024-07-17T08:44:42.694121+0000","last_unstale":"2024-07-17T08:44:45.085182+0000","last_undegraded":"2024-07-17T08:44:45.085182+0000","last_fullsized":"2024-07-17T08:44:45.085182+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:51.021421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.1c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084325+0000","last_change":"2024-07-17T08:44:43.724107+0000","last_active":"2024-07-17T08:44:45.084325+0000","last_peered":"2024-07-17T08:44:45.084325+0000","last_clean":"2024-07-17T08:44:45.084325+0000","last_became_active":"2024-07-17T08:44:43.723093+0000","last_became_peered":"2024-07-17T08:44:43.723093+0000","last_unstale":"2024-07-17T08:44:45.084325+0000","last_undegraded":"2024-07-17T08:44:45.084325+0000","last_fullsized":"2024-07-17T08:44:45.084325+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:32.793428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.3","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698458+0000","last_change":"2024-07-17T08:44:42.685736+0000","last_active":"2024-07-17T08:44:44.698458+0000","last_peered":"2024-07-17T08:44:44.698458+0000","last_clean":"2024-07-17T08:44:44.698458+0000","last_became_active":"2024-07-17T08:44:42.685414+0000","last_became_peered":"2024-07-17T08:44:42.685414+0000","last_unstale":"2024-07-17T08:44:44.698458+0000","last_undegraded":"2024-07-17T08:44:44.698458+0000","last_fullsized":"2024-07-17T08:44:44.698458+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:09:56.098368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.1d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086147+0000","last_change":"2024-07-17T08:44:43.720583+0000","last_active":"2024-07-17T08:44:45.086147+0000","last_peered":"2024-07-17T08:44:45.086147+0000","last_clean":"2024-07-17T08:44:45.086147+0000","last_became_active":"2024-07-17T08:44:43.720030+0000","last_became_peered":"2024-07-17T08:44:43.720030+0000","last_unstale":"2024-07-17T08:44:45.086147+0000","last_undegraded":"2024-07-17T08:44:45.086147+0000","last_fullsized":"2024-07-17T08:44:45.086147+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:03.532983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.2","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084980+0000","last_change":"2024-07-17T08:44:42.695210+0000","last_active":"2024-07-17T08:44:45.084980+0000","last_peered":"2024-07-17T08:44:45.084980+0000","last_clean":"2024-07-17T08:44:45.084980+0000","last_became_active":"2024-07-17T08:44:42.694935+0000","last_became_peered":"2024-07-17T08:44:42.694935+0000","last_unstale":"2024-07-17T08:44:45.084980+0000","last_undegraded":"2024-07-17T08:44:45.084980+0000","last_fullsized":"2024-07-17T08:44:45.084980+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:18.765704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.1e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085720+0000","last_change":"2024-07-17T08:44:43.735113+0000","last_active":"2024-07-17T08:44:45.085720+0000","last_peered":"2024-07-17T08:44:45.085720+0000","last_clean":"2024-07-17T08:44:45.085720+0000","last_became_active":"2024-07-17T08:44:43.734850+0000","last_became_peered":"2024-07-17T08:44:43.734850+0000","last_unstale":"2024-07-17T08:44:45.085720+0000","last_undegraded":"2024-07-17T08:44:45.085720+0000","last_fullsized":"2024-07-17T08:44:45.085720+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:00.136216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.1","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084358+0000","last_change":"2024-07-17T08:44:42.680499+0000","last_active":"2024-07-17T08:44:45.084358+0000","last_peered":"2024-07-17T08:44:45.084358+0000","last_clean":"2024-07-17T08:44:45.084358+0000","last_became_active":"2024-07-17T08:44:42.679909+0000","last_became_peered":"2024-07-17T08:44:42.679909+0000","last_unstale":"2024-07-17T08:44:45.084358+0000","last_undegraded":"2024-07-17T08:44:45.084358+0000","last_fullsized":"2024-07-17T08:44:45.084358+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:57:26.315507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.1f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086895+0000","last_change":"2024-07-17T08:44:43.730072+0000","last_active":"2024-07-17T08:44:45.086895+0000","last_peered":"2024-07-17T08:44:45.086895+0000","last_clean":"2024-07-17T08:44:45.086895+0000","last_became_active":"2024-07-17T08:44:43.726565+0000","last_became_peered":"2024-07-17T08:44:43.726565+0000","last_unstale":"2024-07-17T08:44:45.086895+0000","last_undegraded":"2024-07-17T08:44:45.086895+0000","last_fullsized":"2024-07-17T08:44:45.086895+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:23.733758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.0","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084515+0000","last_change":"2024-07-17T08:44:42.693085+0000","last_active":"2024-07-17T08:44:45.084515+0000","last_peered":"2024-07-17T08:44:45.084515+0000","last_clean":"2024-07-17T08:44:45.084515+0000","last_became_active":"2024-07-17T08:44:42.692782+0000","last_became_peered":"2024-07-17T08:44:42.692782+0000","last_unstale":"2024-07-17T08:44:45.084515+0000","last_undegraded":"2024-07-17T08:44:45.084515+0000","last_fullsized":"2024-07-17T08:44:45.084515+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:19.723260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.31","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084661+0000","last_change":"2024-07-17T08:44:43.719221+0000","last_active":"2024-07-17T08:44:45.084661+0000","last_peered":"2024-07-17T08:44:45.084661+0000","last_clean":"2024-07-17T08:44:45.084661+0000","last_became_active":"2024-07-17T08:44:43.718938+0000","last_became_peered":"2024-07-17T08:44:43.718938+0000","last_unstale":"2024-07-17T08:44:45.084661+0000","last_undegraded":"2024-07-17T08:44:45.084661+0000","last_fullsized":"2024-07-17T08:44:45.084661+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:14.043880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"341'4652","reported_seq":6058,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872477+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:44:44.872477+0000","last_peered":"2024-07-17T08:44:44.872477+0000","last_clean":"2024-07-17T08:44:44.872477+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:44:44.872477+0000","last_undegraded":"2024-07-17T08:44:44.872477+0000","last_fullsized":"2024-07-17T08:44:44.872477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":541,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":48,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":47,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":541,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4652,"ondisk_log_size":4652,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80556,"kb_used_data":4956,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291284,"statfs":{"total":96636764160,"available":96554274816,"internally_reserved":0,"allocated":5074944,"data_stored":3058595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510740,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53604,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318236,"statfs":{"total":96636764160,"available":96581873664,"internally_reserved":0,"allocated":4472832,"data_stored":2467591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510741,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75808,"kb_used_data":4372,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296032,"statfs":{"total":96636764160,"available":96559136768,"internally_reserved":0,"allocated":4476928,"data_stored":2469111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57744,"kb_used_data":4356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314096,"statfs":{"total":96636764160,"available":96577634304,"internally_reserved":0,"allocated":4460544,"data_stored":2460335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57764,"kb_used_data":4372,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314076,"statfs":{"total":96636764160,"available":96577613824,"internally_reserved":0,"allocated":4476928,"data_stored":2469111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543444,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75804,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296036,"statfs":{"total":96636764160,"available":96559140864,"internally_reserved":0,"allocated":4468736,"data_stored":2467533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53580,"kb_used_data":4348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318260,"statfs":{"total":96636764160,"available":96581898240,"internally_reserved":0,"allocated":4452352,"data_stored":2460175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510740,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54176,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317664,"statfs":{"total":96636764160,"available":96581287936,"internally_reserved":0,"allocated":5062656,"data_stored":3058449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:44:48.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:44:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 -- 172.21.15.44:0/3684398289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8838003a50 msgr2=0x7f8838003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 --2- 172.21.15.44:0/3684398289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8838003a50 0x7f8838003e70 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 crypto rx=0x7f8844005fd0 tx=0x7f8838005830 comp rx=0 tx=0).stop 2024-07-17T08:44:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 -- 172.21.15.44:0/3684398289 shutdown_connections 2024-07-17T08:44:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 --2- 172.21.15.44:0/3684398289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8838003a50 0x7f8838003e70 unknown :-1 s=CLOSED pgs=3185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 -- 172.21.15.44:0/3684398289 >> 172.21.15.44:0/3684398289 conn(0x7f88540b0fb0 msgr2=0x7f88540b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.445+0000 7f885a800640 1 -- 172.21.15.44:0/3684398289 shutdown_connections 2024-07-17T08:44:48.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.446+0000 7f885a800640 1 -- 172.21.15.44:0/3684398289 wait complete. 2024-07-17T08:44:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.447+0000 7f885a800640 1 Processor -- start 2024-07-17T08:44:48.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.448+0000 7f885a800640 1 -- start start 2024-07-17T08:44:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.448+0000 7f885a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8838003a50 0x7f88540937f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f885a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f885a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88541568d0 0x7f8854156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f885a800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f885406f710 con 0x7f8854158c50 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f885a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8854154040 con 0x7f88541568d0 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f885a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f885406f890 con 0x7f8838003a50 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50132/0 (socket says 172.21.15.44:50132) 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 -- 172.21.15.44:0/488085251 learned_addr learned my addr 172.21.15.44:0/488085251 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8853400640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8838003a50 0x7f88540937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8853e00640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88541568d0 0x7f8854156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 -- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8838003a50 msgr2=0x7f88540937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8838003a50 0x7f88540937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 -- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88541568d0 msgr2=0x7f8854156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.449+0000 7f8852a00640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88541568d0 0x7f8854156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f8852a00640 1 -- 172.21.15.44:0/488085251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8854142d90 con 0x7f8854158c50 2024-07-17T08:44:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f8852a00640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 crypto rx=0x7f884000c790 tx=0x7f884000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840017070 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8840005380 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884000a8a0 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885413fff0 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.450+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8854141aa0 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.451+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8854159df0 con 0x7f8854158c50 2024-07-17T08:44:48.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.453+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f884001a070 con 0x7f8854158c50 2024-07-17T08:44:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.454+0000 7f884b400640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 0x7f8834079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.454+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f8840097d00 con 0x7f8854158c50 2024-07-17T08:44:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.455+0000 7f8853400640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 0x7f8834079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.456+0000 7f8853400640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 0x7f8834079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8844036000 tx=0x7f884400d040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:48.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.459+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f885415acc0 con 0x7f8854158c50 2024-07-17T08:44:48.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.662+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 --> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f885413fe30 con 0x7f8834076cd0 2024-07-17T08:44:48.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.675+0000 7f884b400640 1 -- 172.21.15.44:0/488085251 <== mgr.9797 v2:172.21.15.175:6834/3157277076 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252504 (secure 0 0 0) 0x7f884402e040 con 0x7f8834076cd0 2024-07-17T08:44:48.678 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 msgr2=0x7f8834079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 0x7f8834079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8844036000 tx=0x7f884400d040 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 msgr2=0x7f8854156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 crypto rx=0x7f884000c790 tx=0x7f884000cc50 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 shutdown_connections 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f8834076cd0 0x7f8834079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88541568d0 0x7f8854156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8854158c50 0x7f8854156390 unknown :-1 s=CLOSED pgs=3186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 --2- 172.21.15.44:0/488085251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8838003a50 0x7f88540937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.681+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 >> 172.21.15.44:0/488085251 conn(0x7f88540b0fb0 msgr2=0x7f885415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:48.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.682+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 shutdown_connections 2024-07-17T08:44:48.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.682+0000 7f885a800640 1 -- 172.21.15.44:0/488085251 wait complete. 2024-07-17T08:44:48.701 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:44:47.355563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":585,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4690,"ondisk_log_size":4690,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":509036,"kb_used_data":36080,"kb_used_omap":12,"kb_used_meta":472819,"kb_avail":754465684,"statfs":{"total":773094113280,"available":772572860416,"internally_reserved":0,"allocated":36945920,"data_stored":20910900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":484167216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.701009"},"pg_stats":[{"pgid":"48.3f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087078+0000","last_change":"2024-07-17T08:44:43.719558+0000","last_active":"2024-07-17T08:44:45.087078+0000","last_peered":"2024-07-17T08:44:45.087078+0000","last_clean":"2024-07-17T08:44:45.087078+0000","last_became_active":"2024-07-17T08:44:43.719273+0000","last_became_peered":"2024-07-17T08:44:43.719273+0000","last_unstale":"2024-07-17T08:44:45.087078+0000","last_undegraded":"2024-07-17T08:44:45.087078+0000","last_fullsized":"2024-07-17T08:44:45.087078+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:27:27.460894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.3e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699179+0000","last_change":"2024-07-17T08:44:43.728200+0000","last_active":"2024-07-17T08:44:44.699179+0000","last_peered":"2024-07-17T08:44:44.699179+0000","last_clean":"2024-07-17T08:44:44.699179+0000","last_became_active":"2024-07-17T08:44:43.727916+0000","last_became_peered":"2024-07-17T08:44:43.727916+0000","last_unstale":"2024-07-17T08:44:44.699179+0000","last_undegraded":"2024-07-17T08:44:44.699179+0000","last_fullsized":"2024-07-17T08:44:44.699179+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:20.430634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.3d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872831+0000","last_change":"2024-07-17T08:44:43.738368+0000","last_active":"2024-07-17T08:44:44.872831+0000","last_peered":"2024-07-17T08:44:44.872831+0000","last_clean":"2024-07-17T08:44:44.872831+0000","last_became_active":"2024-07-17T08:44:43.736733+0000","last_became_peered":"2024-07-17T08:44:43.736733+0000","last_unstale":"2024-07-17T08:44:44.872831+0000","last_undegraded":"2024-07-17T08:44:44.872831+0000","last_fullsized":"2024-07-17T08:44:44.872831+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:03:25.772020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.3c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694795+0000","last_change":"2024-07-17T08:44:43.725846+0000","last_active":"2024-07-17T08:44:44.694795+0000","last_peered":"2024-07-17T08:44:44.694795+0000","last_clean":"2024-07-17T08:44:44.694795+0000","last_became_active":"2024-07-17T08:44:43.725537+0000","last_became_peered":"2024-07-17T08:44:43.725537+0000","last_unstale":"2024-07-17T08:44:44.694795+0000","last_undegraded":"2024-07-17T08:44:44.694795+0000","last_fullsized":"2024-07-17T08:44:44.694795+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:14.221997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"48.3b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084969+0000","last_change":"2024-07-17T08:44:43.720084+0000","last_active":"2024-07-17T08:44:45.084969+0000","last_peered":"2024-07-17T08:44:45.084969+0000","last_clean":"2024-07-17T08:44:45.084969+0000","last_became_active":"2024-07-17T08:44:43.716569+0000","last_became_peered":"2024-07-17T08:44:43.716569+0000","last_unstale":"2024-07-17T08:44:45.084969+0000","last_undegraded":"2024-07-17T08:44:45.084969+0000","last_fullsized":"2024-07-17T08:44:45.084969+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:17.025559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.3a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083183+0000","last_change":"2024-07-17T08:44:43.730424+0000","last_active":"2024-07-17T08:44:45.083183+0000","last_peered":"2024-07-17T08:44:45.083183+0000","last_clean":"2024-07-17T08:44:45.083183+0000","last_became_active":"2024-07-17T08:44:43.729158+0000","last_became_peered":"2024-07-17T08:44:43.729158+0000","last_unstale":"2024-07-17T08:44:45.083183+0000","last_undegraded":"2024-07-17T08:44:45.083183+0000","last_fullsized":"2024-07-17T08:44:45.083183+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:59.846577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.39","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083472+0000","last_change":"2024-07-17T08:44:43.730916+0000","last_active":"2024-07-17T08:44:45.083472+0000","last_peered":"2024-07-17T08:44:45.083472+0000","last_clean":"2024-07-17T08:44:45.083472+0000","last_became_active":"2024-07-17T08:44:43.730400+0000","last_became_peered":"2024-07-17T08:44:43.730400+0000","last_unstale":"2024-07-17T08:44:45.083472+0000","last_undegraded":"2024-07-17T08:44:45.083472+0000","last_fullsized":"2024-07-17T08:44:45.083472+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:35.284971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.38","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084256+0000","last_change":"2024-07-17T08:44:43.736588+0000","last_active":"2024-07-17T08:44:45.084256+0000","last_peered":"2024-07-17T08:44:45.084256+0000","last_clean":"2024-07-17T08:44:45.084256+0000","last_became_active":"2024-07-17T08:44:43.736226+0000","last_became_peered":"2024-07-17T08:44:43.736226+0000","last_unstale":"2024-07-17T08:44:45.084256+0000","last_undegraded":"2024-07-17T08:44:45.084256+0000","last_fullsized":"2024-07-17T08:44:45.084256+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:25.630273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.37","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699644+0000","last_change":"2024-07-17T08:44:43.727410+0000","last_active":"2024-07-17T08:44:44.699644+0000","last_peered":"2024-07-17T08:44:44.699644+0000","last_clean":"2024-07-17T08:44:44.699644+0000","last_became_active":"2024-07-17T08:44:43.725512+0000","last_became_peered":"2024-07-17T08:44:43.725512+0000","last_unstale":"2024-07-17T08:44:44.699644+0000","last_undegraded":"2024-07-17T08:44:44.699644+0000","last_fullsized":"2024-07-17T08:44:44.699644+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:14:10.590023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.36","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085319+0000","last_change":"2024-07-17T08:44:43.727190+0000","last_active":"2024-07-17T08:44:45.085319+0000","last_peered":"2024-07-17T08:44:45.085319+0000","last_clean":"2024-07-17T08:44:45.085319+0000","last_became_active":"2024-07-17T08:44:43.726841+0000","last_became_peered":"2024-07-17T08:44:43.726841+0000","last_unstale":"2024-07-17T08:44:45.085319+0000","last_undegraded":"2024-07-17T08:44:45.085319+0000","last_fullsized":"2024-07-17T08:44:45.085319+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:32.501614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.35","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086425+0000","last_change":"2024-07-17T08:44:43.722529+0000","last_active":"2024-07-17T08:44:45.086425+0000","last_peered":"2024-07-17T08:44:45.086425+0000","last_clean":"2024-07-17T08:44:45.086425+0000","last_became_active":"2024-07-17T08:44:43.722232+0000","last_became_peered":"2024-07-17T08:44:43.722232+0000","last_unstale":"2024-07-17T08:44:45.086425+0000","last_undegraded":"2024-07-17T08:44:45.086425+0000","last_fullsized":"2024-07-17T08:44:45.086425+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:30:52.184906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.34","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085850+0000","last_change":"2024-07-17T08:44:43.721076+0000","last_active":"2024-07-17T08:44:45.085850+0000","last_peered":"2024-07-17T08:44:45.085850+0000","last_clean":"2024-07-17T08:44:45.085850+0000","last_became_active":"2024-07-17T08:44:43.716996+0000","last_became_peered":"2024-07-17T08:44:43.716996+0000","last_unstale":"2024-07-17T08:44:45.085850+0000","last_undegraded":"2024-07-17T08:44:45.085850+0000","last_fullsized":"2024-07-17T08:44:45.085850+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:01:22.257737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.33","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698968+0000","last_change":"2024-07-17T08:44:43.729061+0000","last_active":"2024-07-17T08:44:44.698968+0000","last_peered":"2024-07-17T08:44:44.698968+0000","last_clean":"2024-07-17T08:44:44.698968+0000","last_became_active":"2024-07-17T08:44:43.728420+0000","last_became_peered":"2024-07-17T08:44:43.728420+0000","last_unstale":"2024-07-17T08:44:44.698968+0000","last_undegraded":"2024-07-17T08:44:44.698968+0000","last_fullsized":"2024-07-17T08:44:44.698968+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:17.354426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.32","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085511+0000","last_change":"2024-07-17T08:44:43.719897+0000","last_active":"2024-07-17T08:44:45.085511+0000","last_peered":"2024-07-17T08:44:45.085511+0000","last_clean":"2024-07-17T08:44:45.085511+0000","last_became_active":"2024-07-17T08:44:43.719602+0000","last_became_peered":"2024-07-17T08:44:43.719602+0000","last_unstale":"2024-07-17T08:44:45.085511+0000","last_undegraded":"2024-07-17T08:44:45.085511+0000","last_fullsized":"2024-07-17T08:44:45.085511+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:39:02.980616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.30","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872118+0000","last_change":"2024-07-17T08:44:43.719267+0000","last_active":"2024-07-17T08:44:44.872118+0000","last_peered":"2024-07-17T08:44:44.872118+0000","last_clean":"2024-07-17T08:44:44.872118+0000","last_became_active":"2024-07-17T08:44:43.718968+0000","last_became_peered":"2024-07-17T08:44:43.718968+0000","last_unstale":"2024-07-17T08:44:44.872118+0000","last_undegraded":"2024-07-17T08:44:44.872118+0000","last_fullsized":"2024-07-17T08:44:44.872118+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:57:25.299985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.2f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699115+0000","last_change":"2024-07-17T08:44:43.720002+0000","last_active":"2024-07-17T08:44:44.699115+0000","last_peered":"2024-07-17T08:44:44.699115+0000","last_clean":"2024-07-17T08:44:44.699115+0000","last_became_active":"2024-07-17T08:44:43.719727+0000","last_became_peered":"2024-07-17T08:44:43.719727+0000","last_unstale":"2024-07-17T08:44:44.699115+0000","last_undegraded":"2024-07-17T08:44:44.699115+0000","last_fullsized":"2024-07-17T08:44:44.699115+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:14.492434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.2e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086089+0000","last_change":"2024-07-17T08:44:43.733583+0000","last_active":"2024-07-17T08:44:45.086089+0000","last_peered":"2024-07-17T08:44:45.086089+0000","last_clean":"2024-07-17T08:44:45.086089+0000","last_became_active":"2024-07-17T08:44:43.733298+0000","last_became_peered":"2024-07-17T08:44:43.733298+0000","last_unstale":"2024-07-17T08:44:45.086089+0000","last_undegraded":"2024-07-17T08:44:45.086089+0000","last_fullsized":"2024-07-17T08:44:45.086089+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:28:07.161262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.2d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084889+0000","last_change":"2024-07-17T08:44:43.727539+0000","last_active":"2024-07-17T08:44:45.084889+0000","last_peered":"2024-07-17T08:44:45.084889+0000","last_clean":"2024-07-17T08:44:45.084889+0000","last_became_active":"2024-07-17T08:44:43.726648+0000","last_became_peered":"2024-07-17T08:44:43.726648+0000","last_unstale":"2024-07-17T08:44:45.084889+0000","last_undegraded":"2024-07-17T08:44:45.084889+0000","last_fullsized":"2024-07-17T08:44:45.084889+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:46:20.178735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.2c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873182+0000","last_change":"2024-07-17T08:44:43.727798+0000","last_active":"2024-07-17T08:44:44.873182+0000","last_peered":"2024-07-17T08:44:44.873182+0000","last_clean":"2024-07-17T08:44:44.873182+0000","last_became_active":"2024-07-17T08:44:43.727493+0000","last_became_peered":"2024-07-17T08:44:43.727493+0000","last_unstale":"2024-07-17T08:44:44.873182+0000","last_undegraded":"2024-07-17T08:44:44.873182+0000","last_fullsized":"2024-07-17T08:44:44.873182+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:21.021834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.2b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699028+0000","last_change":"2024-07-17T08:44:43.729340+0000","last_active":"2024-07-17T08:44:44.699028+0000","last_peered":"2024-07-17T08:44:44.699028+0000","last_clean":"2024-07-17T08:44:44.699028+0000","last_became_active":"2024-07-17T08:44:43.728795+0000","last_became_peered":"2024-07-17T08:44:43.728795+0000","last_unstale":"2024-07-17T08:44:44.699028+0000","last_undegraded":"2024-07-17T08:44:44.699028+0000","last_fullsized":"2024-07-17T08:44:44.699028+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:42.360113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.2a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872751+0000","last_change":"2024-07-17T08:44:43.736059+0000","last_active":"2024-07-17T08:44:44.872751+0000","last_peered":"2024-07-17T08:44:44.872751+0000","last_clean":"2024-07-17T08:44:44.872751+0000","last_became_active":"2024-07-17T08:44:43.734414+0000","last_became_peered":"2024-07-17T08:44:43.734414+0000","last_unstale":"2024-07-17T08:44:44.872751+0000","last_undegraded":"2024-07-17T08:44:44.872751+0000","last_fullsized":"2024-07-17T08:44:44.872751+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:31.957556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.29","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084200+0000","last_change":"2024-07-17T08:44:43.713679+0000","last_active":"2024-07-17T08:44:45.084200+0000","last_peered":"2024-07-17T08:44:45.084200+0000","last_clean":"2024-07-17T08:44:45.084200+0000","last_became_active":"2024-07-17T08:44:43.713253+0000","last_became_peered":"2024-07-17T08:44:43.713253+0000","last_unstale":"2024-07-17T08:44:45.084200+0000","last_undegraded":"2024-07-17T08:44:45.084200+0000","last_fullsized":"2024-07-17T08:44:45.084200+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:48:37.577503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.28","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698099+0000","last_change":"2024-07-17T08:44:43.733632+0000","last_active":"2024-07-17T08:44:44.698099+0000","last_peered":"2024-07-17T08:44:44.698099+0000","last_clean":"2024-07-17T08:44:44.698099+0000","last_became_active":"2024-07-17T08:44:43.732935+0000","last_became_peered":"2024-07-17T08:44:43.732935+0000","last_unstale":"2024-07-17T08:44:44.698099+0000","last_undegraded":"2024-07-17T08:44:44.698099+0000","last_fullsized":"2024-07-17T08:44:44.698099+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:54:02.745766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.27","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085553+0000","last_change":"2024-07-17T08:44:43.738381+0000","last_active":"2024-07-17T08:44:45.085553+0000","last_peered":"2024-07-17T08:44:45.085553+0000","last_clean":"2024-07-17T08:44:45.085553+0000","last_became_active":"2024-07-17T08:44:43.737969+0000","last_became_peered":"2024-07-17T08:44:43.737969+0000","last_unstale":"2024-07-17T08:44:45.085553+0000","last_undegraded":"2024-07-17T08:44:45.085553+0000","last_fullsized":"2024-07-17T08:44:45.085553+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:49.930350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.26","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085373+0000","last_change":"2024-07-17T08:44:43.726173+0000","last_active":"2024-07-17T08:44:45.085373+0000","last_peered":"2024-07-17T08:44:45.085373+0000","last_clean":"2024-07-17T08:44:45.085373+0000","last_became_active":"2024-07-17T08:44:43.725860+0000","last_became_peered":"2024-07-17T08:44:43.725860+0000","last_unstale":"2024-07-17T08:44:45.085373+0000","last_undegraded":"2024-07-17T08:44:45.085373+0000","last_fullsized":"2024-07-17T08:44:45.085373+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:26.120877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.25","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699803+0000","last_change":"2024-07-17T08:44:43.725207+0000","last_active":"2024-07-17T08:44:44.699803+0000","last_peered":"2024-07-17T08:44:44.699803+0000","last_clean":"2024-07-17T08:44:44.699803+0000","last_became_active":"2024-07-17T08:44:43.724906+0000","last_became_peered":"2024-07-17T08:44:43.724906+0000","last_unstale":"2024-07-17T08:44:44.699803+0000","last_undegraded":"2024-07-17T08:44:44.699803+0000","last_fullsized":"2024-07-17T08:44:44.699803+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:45:14.517205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.24","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086267+0000","last_change":"2024-07-17T08:44:43.716638+0000","last_active":"2024-07-17T08:44:45.086267+0000","last_peered":"2024-07-17T08:44:45.086267+0000","last_clean":"2024-07-17T08:44:45.086267+0000","last_became_active":"2024-07-17T08:44:43.716309+0000","last_became_peered":"2024-07-17T08:44:43.716309+0000","last_unstale":"2024-07-17T08:44:45.086267+0000","last_undegraded":"2024-07-17T08:44:45.086267+0000","last_fullsized":"2024-07-17T08:44:45.086267+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:26.668369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.23","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085596+0000","last_change":"2024-07-17T08:44:43.716521+0000","last_active":"2024-07-17T08:44:45.085596+0000","last_peered":"2024-07-17T08:44:45.085596+0000","last_clean":"2024-07-17T08:44:45.085596+0000","last_became_active":"2024-07-17T08:44:43.716203+0000","last_became_peered":"2024-07-17T08:44:43.716203+0000","last_unstale":"2024-07-17T08:44:45.085596+0000","last_undegraded":"2024-07-17T08:44:45.085596+0000","last_fullsized":"2024-07-17T08:44:45.085596+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:45.741312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.22","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085931+0000","last_change":"2024-07-17T08:44:43.732786+0000","last_active":"2024-07-17T08:44:45.085931+0000","last_peered":"2024-07-17T08:44:45.085931+0000","last_clean":"2024-07-17T08:44:45.085931+0000","last_became_active":"2024-07-17T08:44:43.732484+0000","last_became_peered":"2024-07-17T08:44:43.732484+0000","last_unstale":"2024-07-17T08:44:45.085931+0000","last_undegraded":"2024-07-17T08:44:45.085931+0000","last_fullsized":"2024-07-17T08:44:45.085931+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:14:09.524217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.21","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872344+0000","last_change":"2024-07-17T08:44:43.735714+0000","last_active":"2024-07-17T08:44:44.872344+0000","last_peered":"2024-07-17T08:44:44.872344+0000","last_clean":"2024-07-17T08:44:44.872344+0000","last_became_active":"2024-07-17T08:44:43.733396+0000","last_became_peered":"2024-07-17T08:44:43.733396+0000","last_unstale":"2024-07-17T08:44:44.872344+0000","last_undegraded":"2024-07-17T08:44:44.872344+0000","last_fullsized":"2024-07-17T08:44:44.872344+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:55:49.203341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.20","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083401+0000","last_change":"2024-07-17T08:44:43.702459+0000","last_active":"2024-07-17T08:44:45.083401+0000","last_peered":"2024-07-17T08:44:45.083401+0000","last_clean":"2024-07-17T08:44:45.083401+0000","last_became_active":"2024-07-17T08:44:43.702133+0000","last_became_peered":"2024-07-17T08:44:43.702133+0000","last_unstale":"2024-07-17T08:44:45.083401+0000","last_undegraded":"2024-07-17T08:44:45.083401+0000","last_fullsized":"2024-07-17T08:44:45.083401+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:58:21.003815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.0","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694390+0000","last_change":"2024-07-17T08:44:43.732424+0000","last_active":"2024-07-17T08:44:44.694390+0000","last_peered":"2024-07-17T08:44:44.694390+0000","last_clean":"2024-07-17T08:44:44.694390+0000","last_became_active":"2024-07-17T08:44:43.728310+0000","last_became_peered":"2024-07-17T08:44:43.728310+0000","last_unstale":"2024-07-17T08:44:44.694390+0000","last_undegraded":"2024-07-17T08:44:44.694390+0000","last_fullsized":"2024-07-17T08:44:44.694390+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:00.730177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.1f","version":"350'4","reported_seq":18,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872757+0000","last_change":"2024-07-17T08:44:42.692992+0000","last_active":"2024-07-17T08:44:44.872757+0000","last_peered":"2024-07-17T08:44:44.872757+0000","last_clean":"2024-07-17T08:44:44.872757+0000","last_became_active":"2024-07-17T08:44:42.692639+0000","last_became_peered":"2024-07-17T08:44:42.692639+0000","last_unstale":"2024-07-17T08:44:44.872757+0000","last_undegraded":"2024-07-17T08:44:44.872757+0000","last_fullsized":"2024-07-17T08:44:44.872757+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:21:04.240811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.1","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872983+0000","last_change":"2024-07-17T08:44:43.730428+0000","last_active":"2024-07-17T08:44:44.872983+0000","last_peered":"2024-07-17T08:44:44.872983+0000","last_clean":"2024-07-17T08:44:44.872983+0000","last_became_active":"2024-07-17T08:44:43.730114+0000","last_became_peered":"2024-07-17T08:44:43.730114+0000","last_unstale":"2024-07-17T08:44:44.872983+0000","last_undegraded":"2024-07-17T08:44:44.872983+0000","last_fullsized":"2024-07-17T08:44:44.872983+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:39:20.063168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.1e","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698399+0000","last_change":"2024-07-17T08:44:42.701350+0000","last_active":"2024-07-17T08:44:44.698399+0000","last_peered":"2024-07-17T08:44:44.698399+0000","last_clean":"2024-07-17T08:44:44.698399+0000","last_became_active":"2024-07-17T08:44:42.701061+0000","last_became_peered":"2024-07-17T08:44:42.701061+0000","last_unstale":"2024-07-17T08:44:44.698399+0000","last_undegraded":"2024-07-17T08:44:44.698399+0000","last_fullsized":"2024-07-17T08:44:44.698399+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:32.755995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.2","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698466+0000","last_change":"2024-07-17T08:44:43.717561+0000","last_active":"2024-07-17T08:44:44.698466+0000","last_peered":"2024-07-17T08:44:44.698466+0000","last_clean":"2024-07-17T08:44:44.698466+0000","last_became_active":"2024-07-17T08:44:43.714594+0000","last_became_peered":"2024-07-17T08:44:43.714594+0000","last_unstale":"2024-07-17T08:44:44.698466+0000","last_undegraded":"2024-07-17T08:44:44.698466+0000","last_fullsized":"2024-07-17T08:44:44.698466+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:12:34.851165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.1d","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086374+0000","last_change":"2024-07-17T08:44:42.699132+0000","last_active":"2024-07-17T08:44:45.086374+0000","last_peered":"2024-07-17T08:44:45.086374+0000","last_clean":"2024-07-17T08:44:45.086374+0000","last_became_active":"2024-07-17T08:44:42.698812+0000","last_became_peered":"2024-07-17T08:44:42.698812+0000","last_unstale":"2024-07-17T08:44:45.086374+0000","last_undegraded":"2024-07-17T08:44:45.086374+0000","last_fullsized":"2024-07-17T08:44:45.086374+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:42:23.317048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.3","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698322+0000","last_change":"2024-07-17T08:44:43.727218+0000","last_active":"2024-07-17T08:44:44.698322+0000","last_peered":"2024-07-17T08:44:44.698322+0000","last_clean":"2024-07-17T08:44:44.698322+0000","last_became_active":"2024-07-17T08:44:43.726093+0000","last_became_peered":"2024-07-17T08:44:43.726093+0000","last_unstale":"2024-07-17T08:44:44.698322+0000","last_undegraded":"2024-07-17T08:44:44.698322+0000","last_fullsized":"2024-07-17T08:44:44.698322+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:03.235964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.1c","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086227+0000","last_change":"2024-07-17T08:44:42.699144+0000","last_active":"2024-07-17T08:44:45.086227+0000","last_peered":"2024-07-17T08:44:45.086227+0000","last_clean":"2024-07-17T08:44:45.086227+0000","last_became_active":"2024-07-17T08:44:42.698803+0000","last_became_peered":"2024-07-17T08:44:42.698803+0000","last_unstale":"2024-07-17T08:44:45.086227+0000","last_undegraded":"2024-07-17T08:44:45.086227+0000","last_fullsized":"2024-07-17T08:44:45.086227+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:11:59.316118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.4","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694900+0000","last_change":"2024-07-17T08:44:43.707267+0000","last_active":"2024-07-17T08:44:44.694900+0000","last_peered":"2024-07-17T08:44:44.694900+0000","last_clean":"2024-07-17T08:44:44.694900+0000","last_became_active":"2024-07-17T08:44:43.704746+0000","last_became_peered":"2024-07-17T08:44:43.704746+0000","last_unstale":"2024-07-17T08:44:44.694900+0000","last_undegraded":"2024-07-17T08:44:44.694900+0000","last_fullsized":"2024-07-17T08:44:44.694900+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:16:00.865925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.1b","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085895+0000","last_change":"2024-07-17T08:44:42.698333+0000","last_active":"2024-07-17T08:44:45.085895+0000","last_peered":"2024-07-17T08:44:45.085895+0000","last_clean":"2024-07-17T08:44:45.085895+0000","last_became_active":"2024-07-17T08:44:42.698045+0000","last_became_peered":"2024-07-17T08:44:42.698045+0000","last_unstale":"2024-07-17T08:44:45.085895+0000","last_undegraded":"2024-07-17T08:44:45.085895+0000","last_fullsized":"2024-07-17T08:44:45.085895+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:10:25.104223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.5","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699379+0000","last_change":"2024-07-17T08:44:43.718594+0000","last_active":"2024-07-17T08:44:44.699379+0000","last_peered":"2024-07-17T08:44:44.699379+0000","last_clean":"2024-07-17T08:44:44.699379+0000","last_became_active":"2024-07-17T08:44:43.717801+0000","last_became_peered":"2024-07-17T08:44:43.717801+0000","last_unstale":"2024-07-17T08:44:44.699379+0000","last_undegraded":"2024-07-17T08:44:44.699379+0000","last_fullsized":"2024-07-17T08:44:44.699379+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:56.791167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.1a","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084286+0000","last_change":"2024-07-17T08:44:42.682008+0000","last_active":"2024-07-17T08:44:45.084286+0000","last_peered":"2024-07-17T08:44:45.084286+0000","last_clean":"2024-07-17T08:44:45.084286+0000","last_became_active":"2024-07-17T08:44:42.681711+0000","last_became_peered":"2024-07-17T08:44:42.681711+0000","last_unstale":"2024-07-17T08:44:45.084286+0000","last_undegraded":"2024-07-17T08:44:45.084286+0000","last_fullsized":"2024-07-17T08:44:45.084286+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:52:06.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.6","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087123+0000","last_change":"2024-07-17T08:44:43.732190+0000","last_active":"2024-07-17T08:44:45.087123+0000","last_peered":"2024-07-17T08:44:45.087123+0000","last_clean":"2024-07-17T08:44:45.087123+0000","last_became_active":"2024-07-17T08:44:43.729838+0000","last_became_peered":"2024-07-17T08:44:43.729838+0000","last_unstale":"2024-07-17T08:44:45.087123+0000","last_undegraded":"2024-07-17T08:44:45.087123+0000","last_fullsized":"2024-07-17T08:44:45.087123+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:19:16.201991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.19","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.697866+0000","last_change":"2024-07-17T08:44:42.701239+0000","last_active":"2024-07-17T08:44:44.697866+0000","last_peered":"2024-07-17T08:44:44.697866+0000","last_clean":"2024-07-17T08:44:44.697866+0000","last_became_active":"2024-07-17T08:44:42.700968+0000","last_became_peered":"2024-07-17T08:44:42.700968+0000","last_unstale":"2024-07-17T08:44:44.697866+0000","last_undegraded":"2024-07-17T08:44:44.697866+0000","last_fullsized":"2024-07-17T08:44:44.697866+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:24.800946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.7","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699754+0000","last_change":"2024-07-17T08:44:43.732561+0000","last_active":"2024-07-17T08:44:44.699754+0000","last_peered":"2024-07-17T08:44:44.699754+0000","last_clean":"2024-07-17T08:44:44.699754+0000","last_became_active":"2024-07-17T08:44:43.726639+0000","last_became_peered":"2024-07-17T08:44:43.726639+0000","last_unstale":"2024-07-17T08:44:44.699754+0000","last_undegraded":"2024-07-17T08:44:44.699754+0000","last_fullsized":"2024-07-17T08:44:44.699754+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:47.304709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.18","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872529+0000","last_change":"2024-07-17T08:44:42.683806+0000","last_active":"2024-07-17T08:44:44.872529+0000","last_peered":"2024-07-17T08:44:44.872529+0000","last_clean":"2024-07-17T08:44:44.872529+0000","last_became_active":"2024-07-17T08:44:42.683445+0000","last_became_peered":"2024-07-17T08:44:42.683445+0000","last_unstale":"2024-07-17T08:44:44.872529+0000","last_undegraded":"2024-07-17T08:44:44.872529+0000","last_fullsized":"2024-07-17T08:44:44.872529+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:34.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.8","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084805+0000","last_change":"2024-07-17T08:44:43.729487+0000","last_active":"2024-07-17T08:44:45.084805+0000","last_peered":"2024-07-17T08:44:45.084805+0000","last_clean":"2024-07-17T08:44:45.084805+0000","last_became_active":"2024-07-17T08:44:43.729062+0000","last_became_peered":"2024-07-17T08:44:43.729062+0000","last_unstale":"2024-07-17T08:44:45.084805+0000","last_undegraded":"2024-07-17T08:44:45.084805+0000","last_fullsized":"2024-07-17T08:44:45.084805+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:11.981637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.17","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699262+0000","last_change":"2024-07-17T08:44:42.686378+0000","last_active":"2024-07-17T08:44:44.699262+0000","last_peered":"2024-07-17T08:44:44.699262+0000","last_clean":"2024-07-17T08:44:44.699262+0000","last_became_active":"2024-07-17T08:44:42.685988+0000","last_became_peered":"2024-07-17T08:44:42.685988+0000","last_unstale":"2024-07-17T08:44:44.699262+0000","last_undegraded":"2024-07-17T08:44:44.699262+0000","last_fullsized":"2024-07-17T08:44:44.699262+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:56:03.376791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.9","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084446+0000","last_change":"2024-07-17T08:44:43.730676+0000","last_active":"2024-07-17T08:44:45.084446+0000","last_peered":"2024-07-17T08:44:45.084446+0000","last_clean":"2024-07-17T08:44:45.084446+0000","last_became_active":"2024-07-17T08:44:43.730015+0000","last_became_peered":"2024-07-17T08:44:43.730015+0000","last_unstale":"2024-07-17T08:44:45.084446+0000","last_undegraded":"2024-07-17T08:44:45.084446+0000","last_fullsized":"2024-07-17T08:44:45.084446+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:17.330433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.16","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086885+0000","last_change":"2024-07-17T08:44:42.702785+0000","last_active":"2024-07-17T08:44:45.086885+0000","last_peered":"2024-07-17T08:44:45.086885+0000","last_clean":"2024-07-17T08:44:45.086885+0000","last_became_active":"2024-07-17T08:44:42.702509+0000","last_became_peered":"2024-07-17T08:44:42.702509+0000","last_unstale":"2024-07-17T08:44:45.086885+0000","last_undegraded":"2024-07-17T08:44:45.086885+0000","last_fullsized":"2024-07-17T08:44:45.086885+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:15:59.849271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694686+0000","last_change":"2024-07-17T08:44:43.707940+0000","last_active":"2024-07-17T08:44:44.694686+0000","last_peered":"2024-07-17T08:44:44.694686+0000","last_clean":"2024-07-17T08:44:44.694686+0000","last_became_active":"2024-07-17T08:44:43.707650+0000","last_became_peered":"2024-07-17T08:44:43.707650+0000","last_unstale":"2024-07-17T08:44:44.694686+0000","last_undegraded":"2024-07-17T08:44:44.694686+0000","last_fullsized":"2024-07-17T08:44:44.694686+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:56.734043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.15","version":"350'3","reported_seq":17,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872754+0000","last_change":"2024-07-17T08:44:42.696442+0000","last_active":"2024-07-17T08:44:44.872754+0000","last_peered":"2024-07-17T08:44:44.872754+0000","last_clean":"2024-07-17T08:44:44.872754+0000","last_became_active":"2024-07-17T08:44:42.696141+0000","last_became_peered":"2024-07-17T08:44:42.696141+0000","last_unstale":"2024-07-17T08:44:44.872754+0000","last_undegraded":"2024-07-17T08:44:44.872754+0000","last_fullsized":"2024-07-17T08:44:44.872754+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:54:15.632379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873142+0000","last_change":"2024-07-17T08:44:43.733058+0000","last_active":"2024-07-17T08:44:44.873142+0000","last_peered":"2024-07-17T08:44:44.873142+0000","last_clean":"2024-07-17T08:44:44.873142+0000","last_became_active":"2024-07-17T08:44:43.731982+0000","last_became_peered":"2024-07-17T08:44:43.731982+0000","last_unstale":"2024-07-17T08:44:44.873142+0000","last_undegraded":"2024-07-17T08:44:44.873142+0000","last_fullsized":"2024-07-17T08:44:44.873142+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:37:30.056544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.14","version":"350'6","reported_seq":20,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084897+0000","last_change":"2024-07-17T08:44:42.682483+0000","last_active":"2024-07-17T08:44:45.084897+0000","last_peered":"2024-07-17T08:44:45.084897+0000","last_clean":"2024-07-17T08:44:45.084897+0000","last_became_active":"2024-07-17T08:44:42.681967+0000","last_became_peered":"2024-07-17T08:44:42.681967+0000","last_unstale":"2024-07-17T08:44:45.084897+0000","last_undegraded":"2024-07-17T08:44:45.084897+0000","last_fullsized":"2024-07-17T08:44:45.084897+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:54:36.609958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084956+0000","last_change":"2024-07-17T08:44:43.719781+0000","last_active":"2024-07-17T08:44:45.084956+0000","last_peered":"2024-07-17T08:44:45.084956+0000","last_clean":"2024-07-17T08:44:45.084956+0000","last_became_active":"2024-07-17T08:44:43.719496+0000","last_became_peered":"2024-07-17T08:44:43.719496+0000","last_unstale":"2024-07-17T08:44:45.084956+0000","last_undegraded":"2024-07-17T08:44:45.084956+0000","last_fullsized":"2024-07-17T08:44:45.084956+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:31:09.458890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.13","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873209+0000","last_change":"2024-07-17T08:44:42.684393+0000","last_active":"2024-07-17T08:44:44.873209+0000","last_peered":"2024-07-17T08:44:44.873209+0000","last_clean":"2024-07-17T08:44:44.873209+0000","last_became_active":"2024-07-17T08:44:42.684095+0000","last_became_peered":"2024-07-17T08:44:42.684095+0000","last_unstale":"2024-07-17T08:44:44.873209+0000","last_undegraded":"2024-07-17T08:44:44.873209+0000","last_fullsized":"2024-07-17T08:44:44.873209+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:21.220207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699456+0000","last_change":"2024-07-17T08:44:43.720369+0000","last_active":"2024-07-17T08:44:44.699456+0000","last_peered":"2024-07-17T08:44:44.699456+0000","last_clean":"2024-07-17T08:44:44.699456+0000","last_became_active":"2024-07-17T08:44:43.719971+0000","last_became_peered":"2024-07-17T08:44:43.719971+0000","last_unstale":"2024-07-17T08:44:44.699456+0000","last_undegraded":"2024-07-17T08:44:44.699456+0000","last_fullsized":"2024-07-17T08:44:44.699456+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:03:32.249485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.12","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.694453+0000","last_change":"2024-07-17T08:44:42.697185+0000","last_active":"2024-07-17T08:44:44.694453+0000","last_peered":"2024-07-17T08:44:44.694453+0000","last_clean":"2024-07-17T08:44:44.694453+0000","last_became_active":"2024-07-17T08:44:42.696755+0000","last_became_peered":"2024-07-17T08:44:42.696755+0000","last_unstale":"2024-07-17T08:44:44.694453+0000","last_undegraded":"2024-07-17T08:44:44.694453+0000","last_fullsized":"2024-07-17T08:44:44.694453+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:16.233733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"48.e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.695102+0000","last_change":"2024-07-17T08:44:43.730594+0000","last_active":"2024-07-17T08:44:44.695102+0000","last_peered":"2024-07-17T08:44:44.695102+0000","last_clean":"2024-07-17T08:44:44.695102+0000","last_became_active":"2024-07-17T08:44:43.730318+0000","last_became_peered":"2024-07-17T08:44:43.730318+0000","last_unstale":"2024-07-17T08:44:44.695102+0000","last_undegraded":"2024-07-17T08:44:44.695102+0000","last_fullsized":"2024-07-17T08:44:44.695102+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:28:47.875138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"47.11","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085070+0000","last_change":"2024-07-17T08:44:42.694983+0000","last_active":"2024-07-17T08:44:45.085070+0000","last_peered":"2024-07-17T08:44:45.085070+0000","last_clean":"2024-07-17T08:44:45.085070+0000","last_became_active":"2024-07-17T08:44:42.694615+0000","last_became_peered":"2024-07-17T08:44:42.694615+0000","last_unstale":"2024-07-17T08:44:45.085070+0000","last_undegraded":"2024-07-17T08:44:45.085070+0000","last_fullsized":"2024-07-17T08:44:45.085070+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:41:55.717389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.087191+0000","last_change":"2024-07-17T08:44:43.718453+0000","last_active":"2024-07-17T08:44:45.087191+0000","last_peered":"2024-07-17T08:44:45.087191+0000","last_clean":"2024-07-17T08:44:45.087191+0000","last_became_active":"2024-07-17T08:44:43.718162+0000","last_became_peered":"2024-07-17T08:44:43.718162+0000","last_unstale":"2024-07-17T08:44:45.087191+0000","last_undegraded":"2024-07-17T08:44:45.087191+0000","last_fullsized":"2024-07-17T08:44:45.087191+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:47:44.830658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.10","version":"350'8","reported_seq":22,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083579+0000","last_change":"2024-07-17T08:44:42.693727+0000","last_active":"2024-07-17T08:44:45.083579+0000","last_peered":"2024-07-17T08:44:45.083579+0000","last_clean":"2024-07-17T08:44:45.083579+0000","last_became_active":"2024-07-17T08:44:42.693437+0000","last_became_peered":"2024-07-17T08:44:42.693437+0000","last_unstale":"2024-07-17T08:44:45.083579+0000","last_undegraded":"2024-07-17T08:44:45.083579+0000","last_fullsized":"2024-07-17T08:44:45.083579+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:24.613619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.10","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872325+0000","last_change":"2024-07-17T08:44:43.728538+0000","last_active":"2024-07-17T08:44:44.872325+0000","last_peered":"2024-07-17T08:44:44.872325+0000","last_clean":"2024-07-17T08:44:44.872325+0000","last_became_active":"2024-07-17T08:44:43.722026+0000","last_became_peered":"2024-07-17T08:44:43.722026+0000","last_unstale":"2024-07-17T08:44:44.872325+0000","last_undegraded":"2024-07-17T08:44:44.872325+0000","last_fullsized":"2024-07-17T08:44:44.872325+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:20:27.823998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.f","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699981+0000","last_change":"2024-07-17T08:44:42.696853+0000","last_active":"2024-07-17T08:44:44.699981+0000","last_peered":"2024-07-17T08:44:44.699981+0000","last_clean":"2024-07-17T08:44:44.699981+0000","last_became_active":"2024-07-17T08:44:42.695919+0000","last_became_peered":"2024-07-17T08:44:42.695919+0000","last_unstale":"2024-07-17T08:44:44.699981+0000","last_undegraded":"2024-07-17T08:44:44.699981+0000","last_fullsized":"2024-07-17T08:44:44.699981+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:08:45.498511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.11","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083963+0000","last_change":"2024-07-17T08:44:43.725035+0000","last_active":"2024-07-17T08:44:45.083963+0000","last_peered":"2024-07-17T08:44:45.083963+0000","last_clean":"2024-07-17T08:44:45.083963+0000","last_became_active":"2024-07-17T08:44:43.723021+0000","last_became_peered":"2024-07-17T08:44:43.723021+0000","last_unstale":"2024-07-17T08:44:45.083963+0000","last_undegraded":"2024-07-17T08:44:45.083963+0000","last_fullsized":"2024-07-17T08:44:45.083963+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:29.932048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"47.e","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872763+0000","last_change":"2024-07-17T08:44:42.689712+0000","last_active":"2024-07-17T08:44:44.872763+0000","last_peered":"2024-07-17T08:44:44.872763+0000","last_clean":"2024-07-17T08:44:44.872763+0000","last_became_active":"2024-07-17T08:44:42.688637+0000","last_became_peered":"2024-07-17T08:44:42.688637+0000","last_unstale":"2024-07-17T08:44:44.872763+0000","last_undegraded":"2024-07-17T08:44:44.872763+0000","last_fullsized":"2024-07-17T08:44:44.872763+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:48:46.193763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.12","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.083311+0000","last_change":"2024-07-17T08:44:43.726109+0000","last_active":"2024-07-17T08:44:45.083311+0000","last_peered":"2024-07-17T08:44:45.083311+0000","last_clean":"2024-07-17T08:44:45.083311+0000","last_became_active":"2024-07-17T08:44:43.724087+0000","last_became_peered":"2024-07-17T08:44:43.724087+0000","last_unstale":"2024-07-17T08:44:45.083311+0000","last_undegraded":"2024-07-17T08:44:45.083311+0000","last_fullsized":"2024-07-17T08:44:45.083311+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:08:52.377147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.d","version":"350'1","reported_seq":15,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086643+0000","last_change":"2024-07-17T08:44:42.699947+0000","last_active":"2024-07-17T08:44:45.086643+0000","last_peered":"2024-07-17T08:44:45.086643+0000","last_clean":"2024-07-17T08:44:45.086643+0000","last_became_active":"2024-07-17T08:44:42.699657+0000","last_became_peered":"2024-07-17T08:44:42.699657+0000","last_unstale":"2024-07-17T08:44:45.086643+0000","last_undegraded":"2024-07-17T08:44:45.086643+0000","last_fullsized":"2024-07-17T08:44:45.086643+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:28:46.858484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.13","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698083+0000","last_change":"2024-07-17T08:44:43.733556+0000","last_active":"2024-07-17T08:44:44.698083+0000","last_peered":"2024-07-17T08:44:44.698083+0000","last_clean":"2024-07-17T08:44:44.698083+0000","last_became_active":"2024-07-17T08:44:43.730184+0000","last_became_peered":"2024-07-17T08:44:43.730184+0000","last_unstale":"2024-07-17T08:44:44.698083+0000","last_undegraded":"2024-07-17T08:44:44.698083+0000","last_fullsized":"2024-07-17T08:44:44.698083+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:29:23.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.c","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086529+0000","last_change":"2024-07-17T08:44:42.698959+0000","last_active":"2024-07-17T08:44:45.086529+0000","last_peered":"2024-07-17T08:44:45.086529+0000","last_clean":"2024-07-17T08:44:45.086529+0000","last_became_active":"2024-07-17T08:44:42.698669+0000","last_became_peered":"2024-07-17T08:44:42.698669+0000","last_unstale":"2024-07-17T08:44:45.086529+0000","last_undegraded":"2024-07-17T08:44:45.086529+0000","last_fullsized":"2024-07-17T08:44:45.086529+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:31.484961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.14","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698348+0000","last_change":"2024-07-17T08:44:43.718712+0000","last_active":"2024-07-17T08:44:44.698348+0000","last_peered":"2024-07-17T08:44:44.698348+0000","last_clean":"2024-07-17T08:44:44.698348+0000","last_became_active":"2024-07-17T08:44:43.718246+0000","last_became_peered":"2024-07-17T08:44:43.718246+0000","last_unstale":"2024-07-17T08:44:44.698348+0000","last_undegraded":"2024-07-17T08:44:44.698348+0000","last_fullsized":"2024-07-17T08:44:44.698348+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:28:37.855550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"47.b","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.873436+0000","last_change":"2024-07-17T08:44:42.685470+0000","last_active":"2024-07-17T08:44:44.873436+0000","last_peered":"2024-07-17T08:44:44.873436+0000","last_clean":"2024-07-17T08:44:44.873436+0000","last_became_active":"2024-07-17T08:44:42.684659+0000","last_became_peered":"2024-07-17T08:44:42.684659+0000","last_unstale":"2024-07-17T08:44:44.873436+0000","last_undegraded":"2024-07-17T08:44:44.873436+0000","last_fullsized":"2024-07-17T08:44:44.873436+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:17:13.434838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"48.15","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085110+0000","last_change":"2024-07-17T08:44:43.726114+0000","last_active":"2024-07-17T08:44:45.085110+0000","last_peered":"2024-07-17T08:44:45.085110+0000","last_clean":"2024-07-17T08:44:45.085110+0000","last_became_active":"2024-07-17T08:44:43.725817+0000","last_became_peered":"2024-07-17T08:44:43.725817+0000","last_unstale":"2024-07-17T08:44:45.085110+0000","last_undegraded":"2024-07-17T08:44:45.085110+0000","last_fullsized":"2024-07-17T08:44:45.085110+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:12:00.332772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.a","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084380+0000","last_change":"2024-07-17T08:44:42.698996+0000","last_active":"2024-07-17T08:44:45.084380+0000","last_peered":"2024-07-17T08:44:45.084380+0000","last_clean":"2024-07-17T08:44:45.084380+0000","last_became_active":"2024-07-17T08:44:42.698649+0000","last_became_peered":"2024-07-17T08:44:42.698649+0000","last_unstale":"2024-07-17T08:44:45.084380+0000","last_undegraded":"2024-07-17T08:44:45.084380+0000","last_fullsized":"2024-07-17T08:44:45.084380+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:11.682915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.16","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086578+0000","last_change":"2024-07-17T08:44:43.722432+0000","last_active":"2024-07-17T08:44:45.086578+0000","last_peered":"2024-07-17T08:44:45.086578+0000","last_clean":"2024-07-17T08:44:45.086578+0000","last_became_active":"2024-07-17T08:44:43.722139+0000","last_became_peered":"2024-07-17T08:44:43.722139+0000","last_unstale":"2024-07-17T08:44:45.086578+0000","last_undegraded":"2024-07-17T08:44:45.086578+0000","last_fullsized":"2024-07-17T08:44:45.086578+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:07:32.647055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.9","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698148+0000","last_change":"2024-07-17T08:44:42.701245+0000","last_active":"2024-07-17T08:44:44.698148+0000","last_peered":"2024-07-17T08:44:44.698148+0000","last_clean":"2024-07-17T08:44:44.698148+0000","last_became_active":"2024-07-17T08:44:42.700948+0000","last_became_peered":"2024-07-17T08:44:42.700948+0000","last_unstale":"2024-07-17T08:44:44.698148+0000","last_undegraded":"2024-07-17T08:44:44.698148+0000","last_fullsized":"2024-07-17T08:44:44.698148+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:53:11.616600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"48.17","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.699480+0000","last_change":"2024-07-17T08:44:43.730549+0000","last_active":"2024-07-17T08:44:44.699480+0000","last_peered":"2024-07-17T08:44:44.699480+0000","last_clean":"2024-07-17T08:44:44.699480+0000","last_became_active":"2024-07-17T08:44:43.730275+0000","last_became_peered":"2024-07-17T08:44:43.730275+0000","last_unstale":"2024-07-17T08:44:44.699480+0000","last_undegraded":"2024-07-17T08:44:44.699480+0000","last_fullsized":"2024-07-17T08:44:44.699480+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:45:10.286508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"47.8","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084581+0000","last_change":"2024-07-17T08:44:42.697403+0000","last_active":"2024-07-17T08:44:45.084581+0000","last_peered":"2024-07-17T08:44:45.084581+0000","last_clean":"2024-07-17T08:44:45.084581+0000","last_became_active":"2024-07-17T08:44:42.697045+0000","last_became_peered":"2024-07-17T08:44:42.697045+0000","last_unstale":"2024-07-17T08:44:45.084581+0000","last_undegraded":"2024-07-17T08:44:45.084581+0000","last_fullsized":"2024-07-17T08:44:45.084581+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:13.205343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"48.18","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084262+0000","last_change":"2024-07-17T08:44:43.730348+0000","last_active":"2024-07-17T08:44:45.084262+0000","last_peered":"2024-07-17T08:44:45.084262+0000","last_clean":"2024-07-17T08:44:45.084262+0000","last_became_active":"2024-07-17T08:44:43.720122+0000","last_became_peered":"2024-07-17T08:44:43.720122+0000","last_unstale":"2024-07-17T08:44:45.084262+0000","last_undegraded":"2024-07-17T08:44:45.084262+0000","last_fullsized":"2024-07-17T08:44:45.084262+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:31:06.474426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.7","version":"350'2","reported_seq":16,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086927+0000","last_change":"2024-07-17T08:44:42.696788+0000","last_active":"2024-07-17T08:44:45.086927+0000","last_peered":"2024-07-17T08:44:45.086927+0000","last_clean":"2024-07-17T08:44:45.086927+0000","last_became_active":"2024-07-17T08:44:42.696431+0000","last_became_peered":"2024-07-17T08:44:42.696431+0000","last_unstale":"2024-07-17T08:44:45.086927+0000","last_undegraded":"2024-07-17T08:44:45.086927+0000","last_fullsized":"2024-07-17T08:44:45.086927+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:10.964983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.19","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084011+0000","last_change":"2024-07-17T08:44:43.731620+0000","last_active":"2024-07-17T08:44:45.084011+0000","last_peered":"2024-07-17T08:44:45.084011+0000","last_clean":"2024-07-17T08:44:45.084011+0000","last_became_active":"2024-07-17T08:44:43.728927+0000","last_became_peered":"2024-07-17T08:44:43.728927+0000","last_unstale":"2024-07-17T08:44:45.084011+0000","last_undegraded":"2024-07-17T08:44:45.084011+0000","last_fullsized":"2024-07-17T08:44:45.084011+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:07.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.6","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698732+0000","last_change":"2024-07-17T08:44:42.700701+0000","last_active":"2024-07-17T08:44:44.698732+0000","last_peered":"2024-07-17T08:44:44.698732+0000","last_clean":"2024-07-17T08:44:44.698732+0000","last_became_active":"2024-07-17T08:44:42.700370+0000","last_became_peered":"2024-07-17T08:44:42.700370+0000","last_unstale":"2024-07-17T08:44:44.698732+0000","last_undegraded":"2024-07-17T08:44:44.698732+0000","last_fullsized":"2024-07-17T08:44:44.698732+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:56:50.278858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.1a","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872315+0000","last_change":"2024-07-17T08:44:43.739617+0000","last_active":"2024-07-17T08:44:44.872315+0000","last_peered":"2024-07-17T08:44:44.872315+0000","last_clean":"2024-07-17T08:44:44.872315+0000","last_became_active":"2024-07-17T08:44:43.739273+0000","last_became_peered":"2024-07-17T08:44:43.739273+0000","last_unstale":"2024-07-17T08:44:44.872315+0000","last_undegraded":"2024-07-17T08:44:44.872315+0000","last_fullsized":"2024-07-17T08:44:44.872315+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:13.895269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"47.5","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085367+0000","last_change":"2024-07-17T08:44:42.695536+0000","last_active":"2024-07-17T08:44:45.085367+0000","last_peered":"2024-07-17T08:44:45.085367+0000","last_clean":"2024-07-17T08:44:45.085367+0000","last_became_active":"2024-07-17T08:44:42.694458+0000","last_became_peered":"2024-07-17T08:44:42.694458+0000","last_unstale":"2024-07-17T08:44:45.085367+0000","last_undegraded":"2024-07-17T08:44:45.085367+0000","last_fullsized":"2024-07-17T08:44:45.085367+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:17:39.489239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.1b","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086117+0000","last_change":"2024-07-17T08:44:43.718290+0000","last_active":"2024-07-17T08:44:45.086117+0000","last_peered":"2024-07-17T08:44:45.086117+0000","last_clean":"2024-07-17T08:44:45.086117+0000","last_became_active":"2024-07-17T08:44:43.717658+0000","last_became_peered":"2024-07-17T08:44:43.717658+0000","last_unstale":"2024-07-17T08:44:45.086117+0000","last_undegraded":"2024-07-17T08:44:45.086117+0000","last_fullsized":"2024-07-17T08:44:45.086117+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:27:31.051859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"47.4","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085182+0000","last_change":"2024-07-17T08:44:42.694414+0000","last_active":"2024-07-17T08:44:45.085182+0000","last_peered":"2024-07-17T08:44:45.085182+0000","last_clean":"2024-07-17T08:44:45.085182+0000","last_became_active":"2024-07-17T08:44:42.694121+0000","last_became_peered":"2024-07-17T08:44:42.694121+0000","last_unstale":"2024-07-17T08:44:45.085182+0000","last_undegraded":"2024-07-17T08:44:45.085182+0000","last_fullsized":"2024-07-17T08:44:45.085182+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:51.021421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"48.1c","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084325+0000","last_change":"2024-07-17T08:44:43.724107+0000","last_active":"2024-07-17T08:44:45.084325+0000","last_peered":"2024-07-17T08:44:45.084325+0000","last_clean":"2024-07-17T08:44:45.084325+0000","last_became_active":"2024-07-17T08:44:43.723093+0000","last_became_peered":"2024-07-17T08:44:43.723093+0000","last_unstale":"2024-07-17T08:44:45.084325+0000","last_undegraded":"2024-07-17T08:44:45.084325+0000","last_fullsized":"2024-07-17T08:44:45.084325+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:58:32.793428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.3","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.698458+0000","last_change":"2024-07-17T08:44:42.685736+0000","last_active":"2024-07-17T08:44:44.698458+0000","last_peered":"2024-07-17T08:44:44.698458+0000","last_clean":"2024-07-17T08:44:44.698458+0000","last_became_active":"2024-07-17T08:44:42.685414+0000","last_became_peered":"2024-07-17T08:44:42.685414+0000","last_unstale":"2024-07-17T08:44:44.698458+0000","last_undegraded":"2024-07-17T08:44:44.698458+0000","last_fullsized":"2024-07-17T08:44:44.698458+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:09:56.098368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"48.1d","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086147+0000","last_change":"2024-07-17T08:44:43.720583+0000","last_active":"2024-07-17T08:44:45.086147+0000","last_peered":"2024-07-17T08:44:45.086147+0000","last_clean":"2024-07-17T08:44:45.086147+0000","last_became_active":"2024-07-17T08:44:43.720030+0000","last_became_peered":"2024-07-17T08:44:43.720030+0000","last_unstale":"2024-07-17T08:44:45.086147+0000","last_undegraded":"2024-07-17T08:44:45.086147+0000","last_fullsized":"2024-07-17T08:44:45.086147+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:58:03.532983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.2","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084980+0000","last_change":"2024-07-17T08:44:42.695210+0000","last_active":"2024-07-17T08:44:45.084980+0000","last_peered":"2024-07-17T08:44:45.084980+0000","last_clean":"2024-07-17T08:44:45.084980+0000","last_became_active":"2024-07-17T08:44:42.694935+0000","last_became_peered":"2024-07-17T08:44:42.694935+0000","last_unstale":"2024-07-17T08:44:45.084980+0000","last_undegraded":"2024-07-17T08:44:45.084980+0000","last_fullsized":"2024-07-17T08:44:45.084980+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:18.765704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"48.1e","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.085720+0000","last_change":"2024-07-17T08:44:43.735113+0000","last_active":"2024-07-17T08:44:45.085720+0000","last_peered":"2024-07-17T08:44:45.085720+0000","last_clean":"2024-07-17T08:44:45.085720+0000","last_became_active":"2024-07-17T08:44:43.734850+0000","last_became_peered":"2024-07-17T08:44:43.734850+0000","last_unstale":"2024-07-17T08:44:45.085720+0000","last_undegraded":"2024-07-17T08:44:45.085720+0000","last_fullsized":"2024-07-17T08:44:45.085720+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:00.136216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"47.1","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084358+0000","last_change":"2024-07-17T08:44:42.680499+0000","last_active":"2024-07-17T08:44:45.084358+0000","last_peered":"2024-07-17T08:44:45.084358+0000","last_clean":"2024-07-17T08:44:45.084358+0000","last_became_active":"2024-07-17T08:44:42.679909+0000","last_became_peered":"2024-07-17T08:44:42.679909+0000","last_unstale":"2024-07-17T08:44:45.084358+0000","last_undegraded":"2024-07-17T08:44:45.084358+0000","last_fullsized":"2024-07-17T08:44:45.084358+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:57:26.315507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.1f","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.086895+0000","last_change":"2024-07-17T08:44:43.730072+0000","last_active":"2024-07-17T08:44:45.086895+0000","last_peered":"2024-07-17T08:44:45.086895+0000","last_clean":"2024-07-17T08:44:45.086895+0000","last_became_active":"2024-07-17T08:44:43.726565+0000","last_became_peered":"2024-07-17T08:44:43.726565+0000","last_unstale":"2024-07-17T08:44:45.086895+0000","last_undegraded":"2024-07-17T08:44:45.086895+0000","last_fullsized":"2024-07-17T08:44:45.086895+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:47:23.733758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"47.0","version":"0'0","reported_seq":14,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084515+0000","last_change":"2024-07-17T08:44:42.693085+0000","last_active":"2024-07-17T08:44:45.084515+0000","last_peered":"2024-07-17T08:44:45.084515+0000","last_clean":"2024-07-17T08:44:45.084515+0000","last_became_active":"2024-07-17T08:44:42.692782+0000","last_became_peered":"2024-07-17T08:44:42.692782+0000","last_unstale":"2024-07-17T08:44:45.084515+0000","last_undegraded":"2024-07-17T08:44:45.084515+0000","last_fullsized":"2024-07-17T08:44:45.084515+0000","mapping_epoch":348,"log_start":"0'0","ondisk_log_start":"0'0","created":348,"last_epoch_clean":349,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:41.635253+0000","last_clean_scrub_stamp":"2024-07-17T08:44:41.635253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:15:19.723260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"48.31","version":"0'0","reported_seq":12,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:45.084661+0000","last_change":"2024-07-17T08:44:43.719221+0000","last_active":"2024-07-17T08:44:45.084661+0000","last_peered":"2024-07-17T08:44:45.084661+0000","last_clean":"2024-07-17T08:44:45.084661+0000","last_became_active":"2024-07-17T08:44:43.718938+0000","last_became_peered":"2024-07-17T08:44:43.718938+0000","last_unstale":"2024-07-17T08:44:45.084661+0000","last_undegraded":"2024-07-17T08:44:45.084661+0000","last_fullsized":"2024-07-17T08:44:45.084661+0000","mapping_epoch":349,"log_start":"0'0","ondisk_log_start":"0'0","created":349,"last_epoch_clean":350,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:44:42.651907+0000","last_clean_scrub_stamp":"2024-07-17T08:44:42.651907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:14:14.043880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"341'4652","reported_seq":6058,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-07-17T08:44:44.872477+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:44:44.872477+0000","last_peered":"2024-07-17T08:44:44.872477+0000","last_clean":"2024-07-17T08:44:44.872477+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:44:44.872477+0000","last_undegraded":"2024-07-17T08:44:44.872477+0000","last_fullsized":"2024-07-17T08:44:44.872477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":541,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":48,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":47,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2329,"num_read_kb":6614,"num_write":541,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4652,"ondisk_log_size":4652,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80556,"kb_used_data":4956,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291284,"statfs":{"total":96636764160,"available":96554274816,"internally_reserved":0,"allocated":5074944,"data_stored":3058595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510740,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53604,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318236,"statfs":{"total":96636764160,"available":96581873664,"internally_reserved":0,"allocated":4472832,"data_stored":2467591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510741,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75808,"kb_used_data":4372,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296032,"statfs":{"total":96636764160,"available":96559136768,"internally_reserved":0,"allocated":4476928,"data_stored":2469111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57744,"kb_used_data":4356,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314096,"statfs":{"total":96636764160,"available":96577634304,"internally_reserved":0,"allocated":4460544,"data_stored":2460335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57764,"kb_used_data":4372,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314076,"statfs":{"total":96636764160,"available":96577613824,"internally_reserved":0,"allocated":4476928,"data_stored":2469111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543444,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75804,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296036,"statfs":{"total":96636764160,"available":96559140864,"internally_reserved":0,"allocated":4468736,"data_stored":2467533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510741,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53580,"kb_used_data":4348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318260,"statfs":{"total":96636764160,"available":96581898240,"internally_reserved":0,"allocated":4452352,"data_stored":2460175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510740,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54176,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317664,"statfs":{"total":96636764160,"available":96581287936,"internally_reserved":0,"allocated":5062656,"data_stored":3058449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":47,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":48,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:44:48.705 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:44:48.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 -- 172.21.15.44:0/3934785787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5b0003ac0 msgr2=0x7fa5b0003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/3934785787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5b0003ac0 0x7fa5b0003ee0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0016c0 tx=0x7fa5b0005830 comp rx=0 tx=0).stop 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 -- 172.21.15.44:0/3934785787 shutdown_connections 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/3934785787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5b0003ac0 0x7fa5b0003ee0 unknown :-1 s=CLOSED pgs=3188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 -- 172.21.15.44:0/3934785787 >> 172.21.15.44:0/3934785787 conn(0x7fa5cc0cf0d0 msgr2=0x7fa5cc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 -- 172.21.15.44:0/3934785787 shutdown_connections 2024-07-17T08:44:48.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.964+0000 7fa5d1a00640 1 -- 172.21.15.44:0/3934785787 wait complete. 2024-07-17T08:44:48.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.966+0000 7fa5d1a00640 1 Processor -- start 2024-07-17T08:44:48.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.966+0000 7fa5d1a00640 1 -- start start 2024-07-17T08:44:48.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5b0003ac0 0x7fa5cc153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5cc09a900 0x7fa5cc13d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc0b2eb0 con 0x7fa5cc13d710 2024-07-17T08:44:48.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc0ae830 con 0x7fa5cc09a900 2024-07-17T08:44:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5cc0b3030 con 0x7fa5b0003ac0 2024-07-17T08:44:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36028/0 (socket says 172.21.15.44:36028) 2024-07-17T08:44:48.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.967+0000 7fa5d1000640 1 -- 172.21.15.44:0/2239640423 learned_addr learned my addr 172.21.15.44:0/2239640423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 -- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5b0003ac0 msgr2=0x7fa5cc153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5b0003ac0 0x7fa5cc153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 -- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5cc09a900 msgr2=0x7fa5cc13d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5cc09a900 0x7fa5cc13d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:48.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 -- 172.21.15.44:0/2239640423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc13b1e0 con 0x7fa5cc13d710 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1000640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005fd0 tx=0x7fa5c400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003da0 con 0x7fa5cc13d710 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5c4018070 con 0x7fa5cc13d710 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400cc40 con 0x7fa5cc13d710 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.968+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc13b4c0 con 0x7fa5cc13d710 2024-07-17T08:44:48.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.969+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc153cc0 con 0x7fa5cc13d710 2024-07-17T08:44:48.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.970+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc0b3590 con 0x7fa5cc13d710 2024-07-17T08:44:48.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.971+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fa5c4004030 con 0x7fa5cc13d710 2024-07-17T08:44:48.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.972+0000 7fa5c8c00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 0x7fa5a8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:48.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.972+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fa5c4098e10 con 0x7fa5cc13d710 2024-07-17T08:44:48.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.973+0000 7fa5cbe00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 0x7fa5a8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:48.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.974+0000 7fa5cbe00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 0x7fa5a8079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc032000 tx=0x7fa5bc001120 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:48.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:48.976+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa5c409c030 con 0x7fa5cc13d710 2024-07-17T08:44:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.274+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5cc0cffe0 con 0x7fa5cc13d710 2024-07-17T08:44:49.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.279+0000 7fa5c8c00640 1 -- 172.21.15.44:0/2239640423 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v83) ==== 190+0+147 (secure 0 0 0) 0x7fa5c4061c20 con 0x7fa5cc13d710 2024-07-17T08:44:49.280 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:44:49.280 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:44:49.281 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:44:49.281 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:44:49.281 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:44:49.281 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:44:49.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 msgr2=0x7fa5a8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 0x7fa5a8079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc032000 tx=0x7fa5bc001120 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 msgr2=0x7fa5cc0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005fd0 tx=0x7fa5c400ee10 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 shutdown_connections 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fa5a8076cd0 0x7fa5a8079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5cc13d710 0x7fa5cc0cd630 unknown :-1 s=CLOSED pgs=3189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5cc09a900 0x7fa5cc13d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.282+0000 7fa5d1a00640 1 --2- 172.21.15.44:0/2239640423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5b0003ac0 0x7fa5cc153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.283+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 >> 172.21.15.44:0/2239640423 conn(0x7fa5cc0cf0d0 msgr2=0x7fa5cc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.283+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 shutdown_connections 2024-07-17T08:44:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.283+0000 7fa5d1a00640 1 -- 172.21.15.44:0/2239640423 wait complete. 2024-07-17T08:44:49.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:44:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.513+0000 7f0327c00640 1 -- 172.21.15.44:0/187497498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 msgr2=0x7f0300003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.513+0000 7f0327c00640 1 --2- 172.21.15.44:0/187497498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f0300003ea0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f03100016c0 tx=0x7f0300005830 comp rx=0 tx=0).stop 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.513+0000 7f0325c00640 1 -- 172.21.15.44:0/187497498 reap_dead start 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.514+0000 7f0327c00640 1 -- 172.21.15.44:0/187497498 shutdown_connections 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.514+0000 7f0327c00640 1 -- 172.21.15.44:0/187497498 >> 172.21.15.44:0/187497498 conn(0x7f03200af940 msgr2=0x7f03200afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.514+0000 7f0327c00640 1 -- 172.21.15.44:0/187497498 shutdown_connections 2024-07-17T08:44:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.514+0000 7f0327c00640 1 -- 172.21.15.44:0/187497498 wait complete. 2024-07-17T08:44:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.515+0000 7f0327c00640 1 Processor -- start 2024-07-17T08:44:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.515+0000 7f0327c00640 1 -- start start 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 0x7f032014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 0x7f03200aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f032015b210 con 0x7f0300003a80 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f032015b090 con 0x7f032014c550 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.516+0000 7f0327c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03200b58c0 con 0x7f03200ae9e0 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36052/0 (socket says 172.21.15.44:36052) 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 -- 172.21.15.44:0/3455357002 learned_addr learned my addr 172.21.15.44:0/3455357002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f031fe00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 0x7f032014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 0x7f03200aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 -- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 msgr2=0x7f03200aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 0x7f03200aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 -- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 msgr2=0x7f032014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 0x7f032014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325200640 1 -- 172.21.15.44:0/3455357002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200ad940 con 0x7f0300003a80 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f031fe00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 0x7f032014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.517+0000 7f0325c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 0x7f03200aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f0325200640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f03100017b0 tx=0x7f03100063e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310038070 con 0x7f0300003a80 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0310003af0 con 0x7f0300003a80 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03200adc20 con 0x7f0300003a80 2024-07-17T08:44:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03100012e0 con 0x7f0300003a80 2024-07-17T08:44:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.518+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03200ae1e0 con 0x7f0300003a80 2024-07-17T08:44:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.519+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03200cf340 con 0x7f0300003a80 2024-07-17T08:44:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.520+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f0310043020 con 0x7f0300003a80 2024-07-17T08:44:49.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.521+0000 7f031cc00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 0x7f02fc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:49.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.522+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f03100b8070 con 0x7f0300003a80 2024-07-17T08:44:49.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.522+0000 7f031fe00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 0x7f02fc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:49.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.523+0000 7f031fe00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 0x7f02fc079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f032014fdf0 tx=0x7f0308005f90 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.526+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f0310080f90 con 0x7f0300003a80 2024-07-17T08:44:49.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.771+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03200b3430 con 0x7f0300003a80 2024-07-17T08:44:49.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.772+0000 7f031cc00640 1 -- 172.21.15.44:0/3455357002 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7f0310080890 con 0x7f0300003a80 2024-07-17T08:44:49.773 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:44:49.773 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:44:49.774 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 msgr2=0x7f02fc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 0x7f02fc079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f032014fdf0 tx=0x7f0308005f90 comp rx=0 tx=0).stop 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 msgr2=0x7f03201f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f03100017b0 tx=0x7f03100063e0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 shutdown_connections 2024-07-17T08:44:49.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f02fc076cd0 0x7f02fc079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03200ae9e0 0x7f03200aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f032014c550 0x7f032014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 --2- 172.21.15.44:0/3455357002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0300003a80 0x7f03201f4850 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 >> 172.21.15.44:0/3455357002 conn(0x7f03200af940 msgr2=0x7f03200afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.775+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 shutdown_connections 2024-07-17T08:44:49.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:49.776+0000 7f0327c00640 1 -- 172.21.15.44:0/3455357002 wait complete. 2024-07-17T08:44:50.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 200, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:44:43.210144+0000', 'modified': '2024-07-17T08:44:46.246155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13956}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13956': {'gid': 13956, 'name': 'a', 'rank': 0, 'incarnation': 196, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6835/2592823249', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6833', 'nonce': 2592823249}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2592823249}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [48], 'metadata_pool': 47, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 13956, 'qdb_cluster': [13956]} 2024-07-17T08:44:50.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:44:50.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:44:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.418+0000 7fb8e1600640 1 -- 172.21.15.44:0/4267867931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc06db60 msgr2=0x7fb8dc0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.418+0000 7fb8e1600640 1 --2- 172.21.15.44:0/4267867931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc06db60 0x7fb8dc0ac190 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0b43f0 tx=0x7fb8d402aa30 comp rx=0 tx=0).stop 2024-07-17T08:44:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 -- 172.21.15.44:0/4267867931 shutdown_connections 2024-07-17T08:44:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 --2- 172.21.15.44:0/4267867931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc06db60 0x7fb8dc0ac190 unknown :-1 s=CLOSED pgs=3193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 --2- 172.21.15.44:0/4267867931 >> v2:172.21.15.175:3300/0 conn(0x7fb8dc161120 0x7fb8dc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 -- 172.21.15.44:0/4267867931 >> 172.21.15.44:0/4267867931 conn(0x7fb8dc0b0eb0 msgr2=0x7fb8dc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 -- 172.21.15.44:0/4267867931 shutdown_connections 2024-07-17T08:44:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.419+0000 7fb8e1600640 1 -- 172.21.15.44:0/4267867931 wait complete. 2024-07-17T08:44:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.420+0000 7fb8e1600640 1 Processor -- start 2024-07-17T08:44:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.420+0000 7fb8e1600640 1 -- start start 2024-07-17T08:44:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 0x7fb8dc1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 0x7fb8dc1f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc15e0a0 con 0x7fb8dc161120 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc15e220 con 0x7fb8dc1f4110 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.421+0000 7fb8e1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc15e3a0 con 0x7fb8dc154600 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36078/0 (socket says 172.21.15.44:36078) 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 -- 172.21.15.44:0/3913335554 learned_addr learned my addr 172.21.15.44:0/3913335554 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8db400640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 0x7fb8dc1f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 -- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 msgr2=0x7fb8dc1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 0x7fb8dc1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 0x7fb8dc1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 -- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 msgr2=0x7fb8dc1f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 0x7fb8dc1f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 -- 172.21.15.44:0/3913335554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc158de0 con 0x7fb8dc161120 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8db400640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 0x7fb8dc1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 0x7fb8dc1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.422+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c970 tx=0x7fb8c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.423+0000 7fb8d3400640 1 -- 172.21.15.44:0/3913335554 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4007bb0 con 0x7fb8dc161120 2024-07-17T08:44:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.423+0000 7fb8d3400640 1 -- 172.21.15.44:0/3913335554 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c4007d50 con 0x7fb8dc161120 2024-07-17T08:44:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.423+0000 7fb8d3400640 1 -- 172.21.15.44:0/3913335554 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4005280 con 0x7fb8dc161120 2024-07-17T08:44:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.423+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc1424b0 con 0x7fb8dc161120 2024-07-17T08:44:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.423+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1f4fa0 con 0x7fb8dc161120 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.426+0000 7fb8d3400640 1 -- 172.21.15.44:0/3913335554 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fb8c4005420 con 0x7fb8dc161120 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.427+0000 7fb8d3400640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 0x7fb8b8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.427+0000 7fb8d3400640 1 -- 172.21.15.44:0/3913335554 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fb8c4098df0 con 0x7fb8dc161120 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.427+0000 7fb8e445a740 1 Processor -- start 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.427+0000 7fb8e445a740 1 -- start start 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.427+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 0x7fb8b8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f109da0 0x55a64f0e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f129de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a64f055d60 con 0x55a64f0df1e0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a64f045260 con 0x55a64f1299e0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.428+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a64f0492a0 con 0x55a64f109da0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36084/0 (socket says 172.21.15.44:36084) 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 -- 172.21.15.44:0/742633040 learned_addr learned my addr 172.21.15.44:0/742633040 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 0x7fb8b8079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc15b550 tx=0x7fb8cc009040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8d2000640 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f129de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 -- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f109da0 msgr2=0x55a64f0e1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f109da0 0x55a64f0e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 -- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 msgr2=0x55a64f129de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f129de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 -- 172.21.15.44:0/742633040 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64eda9850 con 0x55a64f0df1e0 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.429+0000 7fb8db400640 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb8d402b020 tx=0x7fb8d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49301c5ec66de07c server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8cb000640 1 -- 172.21.15.44:0/742633040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4046070 con 0x55a64f0df1e0 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8cb000640 1 -- 172.21.15.44:0/742633040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d4003b20 con 0x55a64f0df1e0 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8cb000640 1 -- 172.21.15.44:0/742633040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4033050 con 0x55a64f0df1e0 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 -- 172.21.15.44:0/742633040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 msgr2=0x55a64f0df630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb8d402b020 tx=0x7fb8d4004a40 comp rx=0 tx=0).stop 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 -- 172.21.15.44:0/742633040 shutdown_connections 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f0df1e0 0x55a64f0df630 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f129de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.430+0000 7fb8e445a740 1 --2- 172.21.15.44:0/742633040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f109da0 0x55a64f0e1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.431+0000 7fb8e445a740 1 -- 172.21.15.44:0/742633040 >> 172.21.15.44:0/742633040 conn(0x55a64f0e0290 msgr2=0x55a64f0e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.431+0000 7fb8e445a740 1 -- 172.21.15.44:0/742633040 shutdown_connections 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.431+0000 7fb8e445a740 1 -- 172.21.15.44:0/742633040 wait complete. 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.431+0000 7fb8e445a740 1 -- start start 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 Processor -- start 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f10ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f1057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a64edb8620 con 0x55a64f109da0 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a64efecf90 con 0x55a64f1299e0 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.432+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a64f0de4b0 con 0x55a64f10cfc0 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8e445a740 10 client.0 _ll_register_callbacks cb 0x55a64f012f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36088/0 (socket says 172.21.15.44:36088) 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 -- 172.21.15.44:0/3664190537 learned_addr learned my addr 172.21.15.44:0/3664190537 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f10ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8db400640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f1057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 -- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 msgr2=0x55a64f1057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f1057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 -- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 msgr2=0x55a64f10ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f10ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8daa00640 1 -- 172.21.15.44:0/3664190537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64ee206b0 con 0x55a64f109da0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.433+0000 7fb8db400640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8daa00640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00ddb0 tx=0x7fb8cc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293538772547e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8bf400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc021660 con 0x55a64f109da0 2024-07-17T08:44:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fb8cc01edb0 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d4003b20 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc027070 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8e445a740 10 client.14091 fetch_fsmap learned FSMap version 200 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.434+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) -- 0x55a64f13a220 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d4003b20 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 200) ==== 3502+0+0 (secure 0 0 0) 0x7fb8cc02a070 con 0x55a64f109da0 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 10 client.14091 fetch_fsmap finished waiting for FSMap version 200 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 10 client.14091 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 20 client.14091 populate_metadata read hostname 'smithi044' 2024-07-17T08:44:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f13b1a0 0x55a64f13b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 4 client.14091 mds_command: new command op to 13956 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 --> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a64f057cb0 con 0x55a64f13b1a0 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.435+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f13b1a0 0x55a64f13b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.436+0000 7fb8d2000640 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f13b1a0 0x55a64f13b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0ec1bdbd69a200 server_cookie=747d4f24cbba8f6e in_seq=0 out_seq=0 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.436+0000 7fb8bf400640 10 client.14091 ms_handle_connect on v2:172.21.15.44:6833/2592823249 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8bf400640 1 -- 172.21.15.44:0/3664190537 <== mds.0 v2:172.21.15.44:6833/2592823249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a64f057cb0 con 0x55a64f13b1a0 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8bf400640 10 client.14091 handle_command_reply: tid=0 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 client.14091 shutdown 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 msgr2=0x55a64f101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00ddb0 tx=0x7fb8cc0039c0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 shutdown_connections 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f1299e0 0x55a64f10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f13b1a0 0x55a64f13b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 --2- 172.21.15.44:0/3664190537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f109da0 0x55a64f101ef0 unknown :-1 s=CLOSED pgs=3196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 >> 172.21.15.44:0/3664190537 conn(0x55a64f0df1e0 msgr2=0x55a64f12a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8bf400640 0 client.14091 ms_handle_reset on v2:172.21.15.44:6833/2592823249 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 shutdown_connections 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 1 -- 172.21.15.44:0/3664190537 wait complete. 2024-07-17T08:44:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.437+0000 7fb8e445a740 20 client.14091 trim_cache size 0 max 16384 2024-07-17T08:44:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.458+0000 7fb8e445a740 1 Processor -- start 2024-07-17T08:44:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.458+0000 7fb8e445a740 1 -- start start 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f10a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f10a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc200620 con 0x55a64f1299e0 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1533f0 con 0x55a64f109da0 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc1fff50 con 0x55a64f10cfc0 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.459+0000 7fb8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36094/0 (socket says 172.21.15.44:36094) 2024-07-17T08:44:50.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 -- 172.21.15.44:0/2061148338 learned_addr learned my addr 172.21.15.44:0/2061148338 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 -- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 msgr2=0x55a64f10a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8daa00640 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f10a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8d2000640 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f10a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f10a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 -- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 msgr2=0x55a64f10a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f10a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 -- 172.21.15.44:0/2061148338 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a64f139600 con 0x55a64f1299e0 2024-07-17T08:44:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8db400640 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000f80 tx=0x7fb8d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e09c9f1512ba6d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.460+0000 7fb8ca600640 1 -- 172.21.15.44:0/2061148338 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d404f070 con 0x55a64f1299e0 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8ca600640 1 -- 172.21.15.44:0/2061148338 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d402f920 con 0x55a64f1299e0 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8ca600640 1 -- 172.21.15.44:0/2061148338 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4046070 con 0x55a64f1299e0 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 -- 172.21.15.44:0/2061148338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 msgr2=0x55a64f0dd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000f80 tx=0x7fb8d40095c0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 -- 172.21.15.44:0/2061148338 shutdown_connections 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f1299e0 0x55a64f0dd430 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f10cfc0 0x55a64f10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2061148338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 -- 172.21.15.44:0/2061148338 >> 172.21.15.44:0/2061148338 conn(0x55a64f0df1e0 msgr2=0x55a64f10b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 -- 172.21.15.44:0/2061148338 shutdown_connections 2024-07-17T08:44:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.461+0000 7fb8e445a740 1 -- 172.21.15.44:0/2061148338 wait complete. 2024-07-17T08:44:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.462+0000 7fb8e445a740 1 -- start start 2024-07-17T08:44:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.462+0000 7fb8e445a740 1 Processor -- start 2024-07-17T08:44:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f12ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f1299e0 0x55a64f10a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a64f13c2a0 con 0x55a64f10cfc0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a64f13a500 con 0x55a64f109da0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a64f13c8f0 con 0x55a64f1299e0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8e445a740 10 client.0 _ll_register_callbacks cb 0x7fb8cc00da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.463+0000 7fb8d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36098/0 (socket says 172.21.15.44:36098) 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 -- 172.21.15.44:0/2680533412 learned_addr learned my addr 172.21.15.44:0/2680533412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 -- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f1299e0 msgr2=0x55a64f10a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f1299e0 0x55a64f10a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 -- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 msgr2=0x55a64f12ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f12ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 -- 172.21.15.44:0/2680533412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a64f1acb30 con 0x55a64f10cfc0 2024-07-17T08:44:50.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8d2000640 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4061200 tx=0x7fb8c40076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9d21d403bcf9e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8bf400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c401d040 con 0x55a64f10cfc0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fb8c4096e70 con 0x55a64f10cfc0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.464+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d402f920 con 0x55a64f10cfc0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) ==== 24+0+0 (secure 0 0 0) 0x7fb8c4065b00 con 0x55a64f10cfc0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 10 client.14097 fetch_fsmap learned FSMap version 200 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) -- 0x55a64f109110 con 0x55a64f10cfc0 2024-07-17T08:44:50.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d402f920 con 0x55a64f10cfc0 2024-07-17T08:44:50.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 200) ==== 3502+0+0 (secure 0 0 0) 0x7fb8c4060760 con 0x55a64f10cfc0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 10 client.14097 fetch_fsmap finished waiting for FSMap version 200 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 10 client.14097 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 20 client.14097 populate_metadata read hostname 'smithi044' 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f1afca0 0x55a64f1b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 4 client.14097 mds_command: new command op to 13956 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 --> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] -- command(tid 0: {"prefix": "status"}) -- 0x55a64f053750 con 0x55a64f1afca0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.465+0000 7fb8daa00640 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f1afca0 0x55a64f1b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8daa00640 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f1afca0 0x55a64f1b26e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf889cb2e5c6db3 server_cookie=2ca8464d849eac02 in_seq=0 out_seq=0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8bf400640 10 client.14097 ms_handle_connect on v2:172.21.15.44:6833/2592823249 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8bf400640 1 -- 172.21.15.44:0/2680533412 <== mds.0 v2:172.21.15.44:6833/2592823249 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55a64f053750 con 0x55a64f1afca0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8bf400640 10 client.14097 handle_command_reply: tid=0 2024-07-17T08:44:50.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8e445a740 1 client.14097 shutdown 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 msgr2=0x55a64f1ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4061200 tx=0x7fb8c40076c0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 shutdown_connections 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.466+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a64f1299e0 0x55a64f10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:6833/2592823249,v1:172.21.15.44:6835/2592823249] conn(0x55a64f1afca0 0x55a64f1b26e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a64f10cfc0 0x55a64f1ac5f0 unknown :-1 s=CLOSED pgs=3198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 --2- 172.21.15.44:0/2680533412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a64f109da0 0x55a64f12ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8bf400640 0 client.14097 ms_handle_reset on v2:172.21.15.44:6833/2592823249 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 >> 172.21.15.44:0/2680533412 conn(0x55a64f0df1e0 msgr2=0x55a64f10b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 shutdown_connections 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 1 -- 172.21.15.44:0/2680533412 wait complete. 2024-07-17T08:44:50.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e445a740 20 client.14097 trim_cache size 0 max 16384 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 13956, 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.7771398410000003, 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 200, 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 351, 2024-07-17T08:44:50.471 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 351, 2024-07-17T08:44:50.472 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.713083946999999 2024-07-17T08:44:50.472 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:44:50.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 msgr2=0x7fb8b8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 0x7fb8b8079190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc15b550 tx=0x7fb8cc009040 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 msgr2=0x7fb8dc141f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c970 tx=0x7fb8c400ce30 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.467+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 shutdown_connections 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb8b8076cd0 0x7fb8b8079190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8dc1f4110 0x7fb8dc1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8dc161120 0x7fb8dc141f70 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 --2- 172.21.15.44:0/3913335554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8dc154600 0x7fb8dc1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 >> 172.21.15.44:0/3913335554 conn(0x7fb8dc0b0eb0 msgr2=0x7fb8dc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 shutdown_connections 2024-07-17T08:44:50.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.468+0000 7fb8e1600640 1 -- 172.21.15.44:0/3913335554 wait complete. 2024-07-17T08:44:50.485 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:44:50.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 -- 172.21.15.44:0/1372799348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc218003ac0 msgr2=0x7fc218003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 --2- 172.21.15.44:0/1372799348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc218003ac0 0x7fc218003ee0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7fc224007540 tx=0x7fc218005830 comp rx=0 tx=0).stop 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 -- 172.21.15.44:0/1372799348 shutdown_connections 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 --2- 172.21.15.44:0/1372799348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc218003ac0 0x7fc218003ee0 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 -- 172.21.15.44:0/1372799348 >> 172.21.15.44:0/1372799348 conn(0x7fc2300b0e80 msgr2=0x7fc2300b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 -- 172.21.15.44:0/1372799348 shutdown_connections 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.678+0000 7fc236000640 1 -- 172.21.15.44:0/1372799348 wait complete. 2024-07-17T08:44:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.680+0000 7fc236000640 1 Processor -- start 2024-07-17T08:44:50.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.680+0000 7fc236000640 1 -- start start 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ac0 0x7fc2301f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 0x7fc2300b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2300cf710 con 0x7fc23014f430 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2300cf590 con 0x7fc2301f4be0 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc236000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2300cf890 con 0x7fc218003ac0 2024-07-17T08:44:50.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc22fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc22fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36124/0 (socket says 172.21.15.44:36124) 2024-07-17T08:44:50.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.681+0000 7fc22fe00640 1 -- 172.21.15.44:0/3061917527 learned_addr learned my addr 172.21.15.44:0/3061917527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc234c00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ac0 0x7fc2301f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc235600640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 0x7fc2300b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 -- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ac0 msgr2=0x7fc2301f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ac0 0x7fc2301f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 -- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 msgr2=0x7fc2300b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 0x7fc2300b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 -- 172.21.15.44:0/3061917527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2301f4470 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc235600640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 0x7fc2300b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22fe00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c450 tx=0x7fc22000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.682+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200052d0 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.683+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc220005490 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.683+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301f8180 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.683+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220012890 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.683+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301f46e0 con 0x7fc23014f430 2024-07-17T08:44:50.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.684+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2300b17c0 con 0x7fc23014f430 2024-07-17T08:44:50.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.685+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fc2200035f0 con 0x7fc23014f430 2024-07-17T08:44:50.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.687+0000 7fc22cc00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 0x7fc2140791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:50.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.687+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fc2200983f0 con 0x7fc23014f430 2024-07-17T08:44:50.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.688+0000 7fc234c00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 0x7fc2140791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:50.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.689+0000 7fc234c00640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 0x7fc2140791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc224034000 tx=0x7fc22400f040 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:50.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.692+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc2300b17c0 con 0x7fc23014f430 2024-07-17T08:44:50.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.949+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc230159b10 con 0x7fc23014f430 2024-07-17T08:44:50.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.950+0000 7fc22cc00640 1 -- 172.21.15.44:0/3061917527 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v200) ==== 53+0+73 (secure 0 0 0) 0x7fc220061390 con 0x7fc23014f430 2024-07-17T08:44:50.951 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:44:50.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 msgr2=0x7fc2140791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 0x7fc2140791c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc224034000 tx=0x7fc22400f040 comp rx=0 tx=0).stop 2024-07-17T08:44:50.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 msgr2=0x7fc230073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:50.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c450 tx=0x7fc22000c910 comp rx=0 tx=0).stop 2024-07-17T08:44:50.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 shutdown_connections 2024-07-17T08:44:50.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc214076d00 0x7fc2140791c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2301f4be0 0x7fc2300b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc23014f430 0x7fc230073930 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.953+0000 7fc236000640 1 --2- 172.21.15.44:0/3061917527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc218003ac0 0x7fc2301f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:50.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.954+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 >> 172.21.15.44:0/3061917527 conn(0x7fc2300b0e80 msgr2=0x7fc23015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:50.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.954+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 shutdown_connections 2024-07-17T08:44:50.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:50.954+0000 7fc236000640 1 -- 172.21.15.44:0/3061917527 wait complete. 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:44:50.975 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:44:50.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:44:50.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:44:50.976 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:44:50.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:50.976 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:44:51.026 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:44:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:44:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:44:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:44:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 35967sec preferred_lft 35967sec 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:44:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:44:51.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.030 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:44:51.103 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:44:51.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.107 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:44:51.160 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:44:51.161 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:44:51.161 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:44:51.161 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:44:51.161 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:44:51.161 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:44:51.161 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:44:51.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:44:51.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:44:51.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:44:51.162 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:44:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:44:51.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:44:51.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.385 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:44:51.434 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:44:51.434 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:44:51.434 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:44:51.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:44:51.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:44:51.518 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:44:51.574 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:44:51.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:44:51.575 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:44:51.624 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:44:51.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:44:51.755 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:44:51.755 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:44:51.756 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:44:51.779 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:44:51.779 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:44:51.779 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:44:51.779 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:44:51.779 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:44:51.804 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:44:51.804 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:44:51.804 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:44:51.805 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:44:51.805 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:44:51.805 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:44:51.805 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:44:51.805 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:44:51.806 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:44:51.806 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:44:51.806 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:44:51.806 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:44:51.806 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:44:51.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:44:51.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:44:51.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:44:51.874 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:44:51.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:44:51.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:44:52.004 INFO:teuthology.orchestra.run.smithi044.stdout:client11116 2024-07-17T08:44:52.004 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:44:52.004 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:44:52.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:44:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 -- 172.21.15.44:0/3231315004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b4003af0 msgr2=0x7fc9b4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 --2- 172.21.15.44:0/3231315004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9b4003af0 0x7fc9b4003f10 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00016c0 tx=0x7fc9b4005830 comp rx=0 tx=0).stop 2024-07-17T08:44:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d1e00640 1 -- 172.21.15.44:0/3231315004 reap_dead start 2024-07-17T08:44:52.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 -- 172.21.15.44:0/3231315004 shutdown_connections 2024-07-17T08:44:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 -- 172.21.15.44:0/3231315004 >> 172.21.15.44:0/3231315004 conn(0x7fc9cc0b55d0 msgr2=0x7fc9cc0b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 -- 172.21.15.44:0/3231315004 shutdown_connections 2024-07-17T08:44:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.251+0000 7fc9d2800640 1 -- 172.21.15.44:0/3231315004 wait complete. 2024-07-17T08:44:52.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.253+0000 7fc9d2800640 1 Processor -- start 2024-07-17T08:44:52.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.253+0000 7fc9d2800640 1 -- start start 2024-07-17T08:44:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b4003af0 0x7fc9cc1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc14c550 0x7fc9cc14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc162cd0 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc162e50 con 0x7fc9cc14c550 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc9cc162fd0 con 0x7fc9b4003af0 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.254+0000 7fc9d0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36142/0 (socket says 172.21.15.44:36142) 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 -- 172.21.15.44:0/284736087 learned_addr learned my addr 172.21.15.44:0/284736087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d1400640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b4003af0 0x7fc9cc1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 -- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b4003af0 msgr2=0x7fc9cc1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b4003af0 0x7fc9cc1f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 -- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc14c550 msgr2=0x7fc9cc14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc14c550 0x7fc9cc14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 -- 172.21.15.44:0/284736087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc1f5290 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9d0a00640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800d920 tx=0x7fc9b800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.255+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800b7b0 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.256+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9b800ba40 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.256+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b80035f0 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.256+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc14cf40 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.256+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc150f60 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.257+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9cc06fc40 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.258+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7fc9b8003790 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.259+0000 7fc9c9600640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 0x7fc9ac079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.259+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fc9b8098e60 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.260+0000 7fc9d1400640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 0x7fc9ac079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.261+0000 7fc9d1400640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 0x7fc9ac079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0032000 tx=0x7fc9c0001190 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:52.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.263+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc9b8061e10 con 0x7fc9cc1f4e10 2024-07-17T08:44:52.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.500+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 --> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc9cc0d08e0 con 0x7fc9ac076cd0 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.505+0000 7fc9c9600640 1 -- 172.21.15.44:0/284736087 <== mgr.9797 v2:172.21.15.175:6834/3157277076 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc9cc0d08e0 con 0x7fc9ac076cd0 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:44:52.506 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 msgr2=0x7fc9ac079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 0x7fc9ac079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0032000 tx=0x7fc9c0001190 comp rx=0 tx=0).stop 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 msgr2=0x7fc9cc149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800d920 tx=0x7fc9b800dde0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 shutdown_connections 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fc9ac076cd0 0x7fc9ac079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc14c550 0x7fc9cc14ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9cc1f4e10 0x7fc9cc149120 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 --2- 172.21.15.44:0/284736087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc9b4003af0 0x7fc9cc1f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 >> 172.21.15.44:0/284736087 conn(0x7fc9cc0b55d0 msgr2=0x7fc9cc0b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 shutdown_connections 2024-07-17T08:44:52.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.508+0000 7fc9d2800640 1 -- 172.21.15.44:0/284736087 wait complete. 2024-07-17T08:44:52.530 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:44:52.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.743+0000 7f1c88600640 1 -- 172.21.15.44:0/58086359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c64003a50 msgr2=0x7f1c64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.743+0000 7f1c88600640 1 --2- 172.21.15.44:0/58086359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c64003a50 0x7f1c64003e70 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009f00 tx=0x7f1c64005830 comp rx=0 tx=0).stop 2024-07-17T08:44:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.743+0000 7f1c86600640 1 -- 172.21.15.44:0/58086359 reap_dead start 2024-07-17T08:44:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.743+0000 7f1c88600640 1 -- 172.21.15.44:0/58086359 shutdown_connections 2024-07-17T08:44:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.743+0000 7f1c88600640 1 -- 172.21.15.44:0/58086359 >> 172.21.15.44:0/58086359 conn(0x7f1c800b0eb0 msgr2=0x7f1c800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.744+0000 7f1c88600640 1 -- 172.21.15.44:0/58086359 shutdown_connections 2024-07-17T08:44:52.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.744+0000 7f1c88600640 1 -- 172.21.15.44:0/58086359 wait complete. 2024-07-17T08:44:52.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.745+0000 7f1c88600640 1 Processor -- start 2024-07-17T08:44:52.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.745+0000 7f1c88600640 1 -- start start 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c64003a50 0x7f1c801f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c8009b270 0x7f1c8009b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c80150470 con 0x7f1c8009a6e0 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8015a8f0 con 0x7f1c64003a50 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.746+0000 7f1c88600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c801505f0 con 0x7f1c8009b270 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36162/0 (socket says 172.21.15.44:36162) 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 -- 172.21.15.44:0/492428221 learned_addr learned my addr 172.21.15.44:0/492428221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85c00640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c64003a50 0x7f1c801f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c86600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c8009b270 0x7f1c8009b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 -- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c8009b270 msgr2=0x7f1c8009b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c8009b270 0x7f1c8009b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 -- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c64003a50 msgr2=0x7f1c801f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c64003a50 0x7f1c801f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.747+0000 7f1c85200640 1 -- 172.21.15.44:0/492428221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c801f2b90 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c85200640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00fe70 tx=0x7f1c6c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c01c070 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c6c00ae10 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c80099640 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00c820 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.748+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c80099c00 con 0x7f1c8009a6e0 2024-07-17T08:44:52.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.749+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8006eb90 con 0x7f1c8009a6e0 2024-07-17T08:44:52.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.750+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 210) ==== 98931+0+0 (secure 0 0 0) 0x7f1c6c01f070 con 0x7f1c8009a6e0 2024-07-17T08:44:52.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.751+0000 7f1c7d600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 0x7f1c60079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:52.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.751+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f1c6c098fb0 con 0x7f1c8009a6e0 2024-07-17T08:44:52.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.752+0000 7f1c85c00640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 0x7f1c60079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:52.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.753+0000 7f1c85c00640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 0x7f1c60079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c78035000 tx=0x7f1c78004a40 comp rx=0 tx=0).ready entity=mgr.9797 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:52.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:52.756+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1c6c061770 con 0x7f1c8009a6e0 2024-07-17T08:44:53.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.021+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1c800b4200 con 0x7f1c8009a6e0 2024-07-17T08:44:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.788+0000 7f1c7d600640 1 -- 172.21.15.44:0/492428221 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v211) ==== 92+0+0 (secure 0 0 0) 0x7f1c6c065cc0 con 0x7f1c8009a6e0 2024-07-17T08:44:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.793+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 msgr2=0x7f1c60079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.793+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 0x7f1c60079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c78035000 tx=0x7f1c78004a40 comp rx=0 tx=0).stop 2024-07-17T08:44:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.793+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 msgr2=0x7f1c80158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.793+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00fe70 tx=0x7f1c6c00d7b0 comp rx=0 tx=0).stop 2024-07-17T08:44:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 shutdown_connections 2024-07-17T08:44:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f1c60076cd0 0x7f1c60079190 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c78035000 tx=0x7f1c78004a40 comp rx=0 tx=0).stop 2024-07-17T08:44:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c8009b270 0x7f1c8009b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c8009a6e0 0x7f1c80158860 unknown :-1 s=CLOSED pgs=3207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 --2- 172.21.15.44:0/492428221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c64003a50 0x7f1c801f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.794+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 >> 172.21.15.44:0/492428221 conn(0x7f1c800b0eb0 msgr2=0x7f1c800b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.796+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 shutdown_connections 2024-07-17T08:44:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:53.796+0000 7f1c88600640 1 -- 172.21.15.44:0/492428221 wait complete. 2024-07-17T08:44:53.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:44:54.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.018+0000 7fb147c00640 1 -- 172.21.15.44:0/2929243318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb128003a50 msgr2=0x7fb128003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.018+0000 7fb147c00640 1 --2- 172.21.15.44:0/2929243318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb128003a50 0x7fb128003e90 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 crypto rx=0x7fb134009b50 tx=0x7fb128005830 comp rx=0 tx=0).stop 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.019+0000 7fb147c00640 1 -- 172.21.15.44:0/2929243318 shutdown_connections 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.019+0000 7fb147c00640 1 --2- 172.21.15.44:0/2929243318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb128003a50 0x7fb128003e90 unknown :-1 s=CLOSED pgs=3211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.019+0000 7fb147c00640 1 -- 172.21.15.44:0/2929243318 >> 172.21.15.44:0/2929243318 conn(0x7fb1400d00f0 msgr2=0x7fb1400d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.019+0000 7fb147c00640 1 -- 172.21.15.44:0/2929243318 shutdown_connections 2024-07-17T08:44:54.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.019+0000 7fb147c00640 1 -- 172.21.15.44:0/2929243318 wait complete. 2024-07-17T08:44:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.020+0000 7fb147c00640 1 Processor -- start 2024-07-17T08:44:54.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.021+0000 7fb147c00640 1 -- start start 2024-07-17T08:44:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.021+0000 7fb147c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 0x7fb140151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb147c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401517c0 0x7fb1400cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb147c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb147c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb14015dca0 con 0x7fb1400ccdc0 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb147c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb14015db20 con 0x7fb128003a50 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb147c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1400ca370 con 0x7fb1401517c0 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36206/0 (socket says 172.21.15.44:36206) 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 -- 172.21.15.44:0/2281952398 learned_addr learned my addr 172.21.15.44:0/2281952398 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145200640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 0x7fb140151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb13fe00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401517c0 0x7fb1400cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401517c0 msgr2=0x7fb1400cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401517c0 0x7fb1400cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 msgr2=0x7fb140151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 0x7fb140151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145c00640 1 -- 172.21.15.44:0/2281952398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb140151cd0 con 0x7fb1400ccdc0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.022+0000 7fb145200640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 0x7fb140151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb145c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7fb1400d4640 tx=0x7fb1380019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138011870 con 0x7fb1400ccdc0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb138017070 con 0x7fb1400ccdc0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138011d60 con 0x7fb1400ccdc0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb140151fb0 con 0x7fb1400ccdc0 2024-07-17T08:44:54.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.023+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb140152570 con 0x7fb1400ccdc0 2024-07-17T08:44:54.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.024+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140150360 con 0x7fb1400ccdc0 2024-07-17T08:44:54.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.025+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 211) ==== 98948+0+0 (secure 0 0 0) 0x7fb13801c460 con 0x7fb1400ccdc0 2024-07-17T08:44:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.026+0000 7fb13cc00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 0x7fb124079260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.026+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fb138099180 con 0x7fb1400ccdc0 2024-07-17T08:44:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.027+0000 7fb145200640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 msgr2=0x7fb124079260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:54.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.027+0000 7fb145200640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 0x7fb124079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:54.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.031+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb140150360 con 0x7fb1400ccdc0 2024-07-17T08:44:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.228+0000 7fb145200640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 msgr2=0x7fb124079260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.228+0000 7fb145200640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 0x7fb124079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:54.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.241+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb14006d420 con 0x7fb1400ccdc0 2024-07-17T08:44:54.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.249+0000 7fb13cc00640 1 -- 172.21.15.44:0/2281952398 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v105)=0 v105) ==== 138+0+0 (secure 0 0 0) 0x7fb138062010 con 0x7fb1400ccdc0 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 msgr2=0x7fb124079260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 0x7fb124079260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 msgr2=0x7fb1401f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7fb1400d4640 tx=0x7fb1380019b0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 shutdown_connections 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fb124076da0 0x7fb124079260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1400ccdc0 0x7fb1401f7580 unknown :-1 s=CLOSED pgs=3212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401517c0 0x7fb1400cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 --2- 172.21.15.44:0/2281952398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb128003a50 0x7fb140151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 >> 172.21.15.44:0/2281952398 conn(0x7fb1400d00f0 msgr2=0x7fb1400b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 shutdown_connections 2024-07-17T08:44:54.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.252+0000 7fb147c00640 1 -- 172.21.15.44:0/2281952398 wait complete. 2024-07-17T08:44:54.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:44:54.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 -- 172.21.15.44:0/2727889039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb380052c0 msgr2=0x7fcb3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 --2- 172.21.15.44:0/2727889039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb380052c0 0x7fcb3800b9c0 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c640 tx=0x7fcb38005830 comp rx=0 tx=0).stop 2024-07-17T08:44:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 -- 172.21.15.44:0/2727889039 shutdown_connections 2024-07-17T08:44:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 --2- 172.21.15.44:0/2727889039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb380052c0 0x7fcb3800b9c0 unknown :-1 s=CLOSED pgs=3214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 -- 172.21.15.44:0/2727889039 >> 172.21.15.44:0/2727889039 conn(0x7fcb5006f710 msgr2=0x7fcb5006faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.471+0000 7fcb56000640 1 -- 172.21.15.44:0/2727889039 shutdown_connections 2024-07-17T08:44:54.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.472+0000 7fcb56000640 1 -- 172.21.15.44:0/2727889039 wait complete. 2024-07-17T08:44:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.473+0000 7fcb56000640 1 Processor -- start 2024-07-17T08:44:54.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.474+0000 7fcb56000640 1 -- start start 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb380052c0 0x7fcb5014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb501f76f0 0x7fcb501587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500b6500 con 0x7fcb5006d810 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500b6380 con 0x7fcb501f76f0 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb56000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5015c290 con 0x7fcb380052c0 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36222/0 (socket says 172.21.15.44:36222) 2024-07-17T08:44:54.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 -- 172.21.15.44:0/3634106323 learned_addr learned my addr 172.21.15.44:0/3634106323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb4fe00640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb501f76f0 0x7fcb501587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb54c00640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb380052c0 0x7fcb5014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb380052c0 msgr2=0x7fcb5014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb380052c0 0x7fcb5014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb501f76f0 msgr2=0x7fcb501587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb501f76f0 0x7fcb501587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.475+0000 7fcb55600640 1 -- 172.21.15.44:0/3634106323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5014b500 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb55600640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c730 tx=0x7fcb4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48039070 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb48003c60 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb480062d0 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb501f7bd0 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.476+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501f8190 con 0x7fcb5006d810 2024-07-17T08:44:54.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.477+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb500b4400 con 0x7fcb5006d810 2024-07-17T08:44:54.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.479+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 211) ==== 98948+0+0 (secure 0 0 0) 0x7fcb48006470 con 0x7fcb5006d810 2024-07-17T08:44:54.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.480+0000 7fcb4cc00640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 0x7fcb34079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.480+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7fcb480b8930 con 0x7fcb5006d810 2024-07-17T08:44:54.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.480+0000 7fcb54c00640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 msgr2=0x7fcb34079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:54.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.480+0000 7fcb54c00640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 0x7fcb34079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:54.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.488+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcb5014b500 con 0x7fcb5006d810 2024-07-17T08:44:54.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.681+0000 7fcb54c00640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 msgr2=0x7fcb34079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:54.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.682+0000 7fcb54c00640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 0x7fcb34079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.696+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fcb50159720 con 0x7fcb5006d810 2024-07-17T08:44:54.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.703+0000 7fcb4cc00640 1 -- 172.21.15.44:0/3634106323 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v106)=0 v106) ==== 133+0+0 (secure 0 0 0) 0x7fcb48081860 con 0x7fcb5006d810 2024-07-17T08:44:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 msgr2=0x7fcb34079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 0x7fcb34079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 msgr2=0x7fcb5006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c730 tx=0x7fcb4802bed0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 shutdown_connections 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7fcb34076d50 0x7fcb34079210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb5006d810 0x7fcb5006dc90 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb501f76f0 0x7fcb501587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 --2- 172.21.15.44:0/3634106323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb380052c0 0x7fcb5014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 >> 172.21.15.44:0/3634106323 conn(0x7fcb5006f710 msgr2=0x7fcb50070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 shutdown_connections 2024-07-17T08:44:54.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.706+0000 7fcb56000640 1 -- 172.21.15.44:0/3634106323 wait complete. 2024-07-17T08:44:54.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs mygrp06 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.908+0000 7f4b26000640 1 -- 172.21.15.44:0/3871623131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 msgr2=0x7f4b2006c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.908+0000 7f4b26000640 1 --2- 172.21.15.44:0/3871623131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 0x7f4b2006c410 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4b10005fd0 tx=0x7f4b1002a410 comp rx=0 tx=0).stop 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 -- 172.21.15.44:0/3871623131 shutdown_connections 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 --2- 172.21.15.44:0/3871623131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b2006c950 0x7f4b200d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 --2- 172.21.15.44:0/3871623131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 0x7f4b2006c410 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 -- 172.21.15.44:0/3871623131 >> 172.21.15.44:0/3871623131 conn(0x7f4b200d00f0 msgr2=0x7f4b200d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 -- 172.21.15.44:0/3871623131 shutdown_connections 2024-07-17T08:44:54.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.909+0000 7f4b26000640 1 -- 172.21.15.44:0/3871623131 wait complete. 2024-07-17T08:44:54.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.911+0000 7f4b26000640 1 Processor -- start 2024-07-17T08:44:54.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.911+0000 7f4b26000640 1 -- start start 2024-07-17T08:44:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b2006c950 0x7f4b200cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 0x7f4b201f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 0x7f4b200ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2015dca0 con 0x7f4b200ce710 2024-07-17T08:44:54.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2015db20 con 0x7f4b2015d740 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b26000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200ca370 con 0x7f4b2006c950 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 0x7f4b201f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.912+0000 7f4b1ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 0x7f4b201f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36226/0 (socket says 172.21.15.44:36226) 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 -- 172.21.15.44:0/3829750202 learned_addr learned my addr 172.21.15.44:0/3829750202 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b2006c950 0x7f4b200cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1fe00640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 0x7f4b200ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b2006c950 msgr2=0x7f4b200cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b2006c950 0x7f4b200cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 msgr2=0x7f4b200ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b2015d740 0x7f4b200ac980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200cc4d0 con 0x7f4b200ce710 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.913+0000 7f4b1ea00640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 0x7f4b201f1530 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00c460 tx=0x7f4b0c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:54.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.914+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c016020 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.914+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b0c012960 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.914+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c012db0 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.914+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b200cc7b0 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.914+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b200ccd70 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.915+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b200d2f80 con 0x7f4b200ce710 2024-07-17T08:44:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.916+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 211) ==== 98948+0+0 (secure 0 0 0) 0x7f4b0c019020 con 0x7f4b200ce710 2024-07-17T08:44:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.918+0000 7f4b17400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.918+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(351..351 src has 1..351) ==== 6733+0+0 (secure 0 0 0) 0x7f4b0c098570 con 0x7f4b200ce710 2024-07-17T08:44:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.918+0000 7f4b1f400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.918+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:44:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:54.922+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4b0c020030 con 0x7f4b200ce710 2024-07-17T08:44:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:55.120+0000 7f4b1f400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:55.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:55.120+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:44:55.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:55.159+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7f4b200b8860 con 0x7f4b00076d50 2024-07-17T08:44:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:55.522+0000 7f4b1f400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:55.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:55.522+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:44:56.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:56.323+0000 7f4b1f400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:56.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:56.323+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:44:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:57.925+0000 7f4b1f400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3157277076 2024-07-17T08:44:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:57.925+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:44:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:58.009+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 212) ==== 98557+0+0 (secure 0 0 0) 0x7f4b0c05d0d0 con 0x7f4b200ce710 2024-07-17T08:44:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:58.009+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 msgr2=0x7f4b00079210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:44:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:58.009+0000 7f4b17400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/3157277076,v1:172.21.15.175:6837/3157277076] conn(0x7f4b00076d50 0x7f4b00079210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:58.324 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.323+0000 7f2df0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.324 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.323+0000 7f2df0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.324 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.323+0000 7f2df0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.325 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.323+0000 7f2df0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.325 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.323+0000 7f2df0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.325 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.325+0000 7f2df2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.326 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.325+0000 7f2df2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.326 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.325+0000 7f2df2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.326 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.325+0000 7f2df2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:58.326 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:58.325+0000 7f2df2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.018+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 213) ==== 98684+0+0 (secure 0 0 0) 0x7f4b0c001bb0 con 0x7f4b200ce710 2024-07-17T08:44:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.019+0000 7f4b17400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4b0007fa30 0x7f4b00081e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.019+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7f4b0c007650 con 0x7f4b0007fa30 2024-07-17T08:44:59.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.021+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4b0007fa30 0x7f4b00081e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.037+0000 7f4b1f400640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4b0007fa30 0x7f4b00081e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b10033000 tx=0x7f4b10004730 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:59.053 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:59.052+0000 7f2decc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.053 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:59.052+0000 7f2decc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.053 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:59.052+0000 7f2decc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.053 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:59.052+0000 7f2decc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.053 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:44:59.052+0000 7f2decc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:44:59.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.092+0000 7f4b17400640 1 -- 172.21.15.44:0/3829750202 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4b0c007650 con 0x7f4b0007fa30 2024-07-17T08:44:59.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4b0007fa30 msgr2=0x7f4b00081e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4b0007fa30 0x7f4b00081e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b10033000 tx=0x7f4b10004730 comp rx=0 tx=0).stop 2024-07-17T08:44:59.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 msgr2=0x7f4b201f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 --2- 172.21.15.44:0/3829750202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b200ce710 0x7f4b201f1530 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00c460 tx=0x7f4b0c00c920 comp rx=0 tx=0).stop 2024-07-17T08:44:59.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b1fe00640 1 -- 172.21.15.44:0/3829750202 reap_dead start 2024-07-17T08:44:59.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 shutdown_connections 2024-07-17T08:44:59.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.096+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 >> 172.21.15.44:0/3829750202 conn(0x7f4b200d00f0 msgr2=0x7f4b2015e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:59.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.097+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 shutdown_connections 2024-07-17T08:44:59.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.097+0000 7f4b26000640 1 -- 172.21.15.44:0/3829750202 wait complete. 2024-07-17T08:44:59.120 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv06 --group_name mygrp06 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 -- 172.21.15.44:0/1849621690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33a4004020 msgr2=0x7f33a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 --2- 172.21.15.44:0/1849621690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33a4004020 0x7f33a4004800 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fc0 tx=0x7f33a4005830 comp rx=0 tx=0).stop 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c2600640 1 -- 172.21.15.44:0/1849621690 reap_dead start 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 -- 172.21.15.44:0/1849621690 shutdown_connections 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 -- 172.21.15.44:0/1849621690 >> 172.21.15.44:0/1849621690 conn(0x7f33bc073a60 msgr2=0x7f33bc073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:59.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 -- 172.21.15.44:0/1849621690 shutdown_connections 2024-07-17T08:44:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.335+0000 7f33c3000640 1 -- 172.21.15.44:0/1849621690 wait complete. 2024-07-17T08:44:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.337+0000 7f33c3000640 1 Processor -- start 2024-07-17T08:44:59.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.337+0000 7f33c3000640 1 -- start start 2024-07-17T08:44:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33a4004020 0x7f33bc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33bc0ca9c0 0x7f33bc0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0ce890 con 0x7f33bc1f9020 2024-07-17T08:44:59.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0ce710 con 0x7f33a4004020 2024-07-17T08:44:59.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.338+0000 7f33c3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc0cea10 con 0x7f33bc0ca9c0 2024-07-17T08:44:59.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44206/0 (socket says 172.21.15.44:44206) 2024-07-17T08:44:59.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 -- 172.21.15.44:0/1074088050 learned_addr learned my addr 172.21.15.44:0/1074088050 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:59.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c1c00640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33a4004020 0x7f33bc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c1200640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33bc0ca9c0 0x7f33bc0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 -- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33bc0ca9c0 msgr2=0x7f33bc0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33bc0ca9c0 0x7f33bc0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 -- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33a4004020 msgr2=0x7f33bc0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33a4004020 0x7f33bc0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.339+0000 7f33c2600640 1 -- 172.21.15.44:0/1074088050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1f8580 con 0x7f33bc1f9020 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33c2600640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7f33bc08c230 tx=0x7f33b4010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4004220 con 0x7f33bc1f9020 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b4005020 con 0x7f33bc1f9020 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4005470 con 0x7f33bc1f9020 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc0cbf90 con 0x7f33bc1f9020 2024-07-17T08:44:59.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.340+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1f8820 con 0x7f33bc1f9020 2024-07-17T08:44:59.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.341+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc0b8e30 con 0x7f33bc1f9020 2024-07-17T08:44:59.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.342+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 213) ==== 98684+0+0 (secure 0 0 0) 0x7f33b400e850 con 0x7f33bc1f9020 2024-07-17T08:44:59.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.344+0000 7f33b9600640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 0x7f339c078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.344+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f33b4098b00 con 0x7f33bc1f9020 2024-07-17T08:44:59.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.345+0000 7f33c1c00640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 0x7f339c078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.346+0000 7f33c1c00640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 0x7f339c078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005b40 tx=0x7f33b0005bd0 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:59.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.348+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f33b4061db0 con 0x7f33bc1f9020 2024-07-17T08:44:59.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.587+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv06", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7f33bc069460 con 0x7f339c076a50 2024-07-17T08:44:59.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.655+0000 7f33b9600640 1 -- 172.21.15.44:0/1074088050 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33bc069460 con 0x7f339c076a50 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.658+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 msgr2=0x7f339c078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.658+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 0x7f339c078f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005b40 tx=0x7f33b0005bd0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.658+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 msgr2=0x7f33bc159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.658+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7f33bc08c230 tx=0x7f33b4010b50 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 shutdown_connections 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f339c076a50 0x7f339c078f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc1f9020 0x7f33bc159850 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33bc0ca9c0 0x7f33bc0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 --2- 172.21.15.44:0/1074088050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33a4004020 0x7f33bc0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 >> 172.21.15.44:0/1074088050 conn(0x7f33bc073a60 msgr2=0x7f33bc0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 shutdown_connections 2024-07-17T08:44:59.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.659+0000 7f33c3000640 1 -- 172.21.15.44:0/1074088050 wait complete. 2024-07-17T08:44:59.682 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv06 --group mygrp06 --fs cephfs . 1m 2024-07-17T08:44:59.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 -- 172.21.15.44:0/1379012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3420 msgr2=0x7fc2f80b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/1379012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3420 0x7fc2f80b4a30 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8160fb0 tx=0x7fc2e802a4b0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 -- 172.21.15.44:0/1379012257 shutdown_connections 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/1379012257 >> v2:172.21.15.175:3300/0 conn(0x7fc2f80d3800 0x7fc2f80d3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/1379012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3420 0x7fc2f80b4a30 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.905+0000 7fc2f7e00640 1 -- 172.21.15.44:0/1379012257 >> 172.21.15.44:0/1379012257 conn(0x7fc2f808ca20 msgr2=0x7fc2f808ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.906+0000 7fc2f7e00640 1 -- 172.21.15.44:0/1379012257 shutdown_connections 2024-07-17T08:44:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.906+0000 7fc2f7e00640 1 -- 172.21.15.44:0/1379012257 wait complete. 2024-07-17T08:44:59.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.908+0000 7fc2f7e00640 1 Processor -- start 2024-07-17T08:44:59.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.908+0000 7fc2f7e00640 1 -- start start 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2f80b3420 0x7fc2f80ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2f80d3420 0x7fc2f80cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f815d980 con 0x7fc2f80d3800 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f815d800 con 0x7fc2f80b3420 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.909+0000 7fc2f7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80d2190 con 0x7fc2f80d3420 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44230/0 (socket says 172.21.15.44:44230) 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 -- 172.21.15.44:0/2559237749 learned_addr learned my addr 172.21.15.44:0/2559237749 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f6a00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2f80b3420 0x7fc2f80ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f6000640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2f80d3420 0x7fc2f80cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 -- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2f80d3420 msgr2=0x7fc2f80cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2f80d3420 0x7fc2f80cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 -- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2f80b3420 msgr2=0x7fc2f80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2f80b3420 0x7fc2f80ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:44:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.910+0000 7fc2f7400640 1 -- 172.21.15.44:0/2559237749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f80caba0 con 0x7fc2f80d3800 2024-07-17T08:44:59.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2f7400640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000cbc0 tx=0x7fc2f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:59.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0016020 con 0x7fc2f80d3800 2024-07-17T08:44:59.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2f000add0 con 0x7fc2f80d3800 2024-07-17T08:44:59.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f80cae80 con 0x7fc2f80d3800 2024-07-17T08:44:59.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0012680 con 0x7fc2f80d3800 2024-07-17T08:44:59.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.911+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f80cb440 con 0x7fc2f80d3800 2024-07-17T08:44:59.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.912+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f80b4110 con 0x7fc2f80d3800 2024-07-17T08:44:59.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.914+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 213) ==== 98684+0+0 (secure 0 0 0) 0x7fc2f0019020 con 0x7fc2f80d3800 2024-07-17T08:44:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.915+0000 7fc2eea00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 0x7fc2d8078f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:44:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.915+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7fc2f0098c80 con 0x7fc2f80d3800 2024-07-17T08:44:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.916+0000 7fc2f6a00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 0x7fc2d8078f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:44:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.917+0000 7fc2f6a00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 0x7fc2d8078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e8004730 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:44:59.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:44:59.921+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc2f009f080 con 0x7fc2f80d3800 2024-07-17T08:45:00.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.072+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 214) ==== 99000+0+0 (secure 0 0 0) 0x7fc2f005d9f0 con 0x7fc2f80d3800 2024-07-17T08:45:00.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.164+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv06", "group": "mygrp06", "fs": "cephfs", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc2f80cfa20 con 0x7fc2d8076a50 2024-07-17T08:45:00.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.378+0000 7fc2eea00640 1 -- 172.21.15.44:0/2559237749 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+84 (secure 0 0 0) 0x7fc2f80cfa20 con 0x7fc2d8076a50 2024-07-17T08:45:00.381 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247/.. 2024-07-17T08:45:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 msgr2=0x7fc2d8078f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 0x7fc2d8078f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e8004730 comp rx=0 tx=0).stop 2024-07-17T08:45:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 msgr2=0x7fc2f80b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:00.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000cbc0 tx=0x7fc2f000a430 comp rx=0 tx=0).stop 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 shutdown_connections 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc2d8076a50 0x7fc2d8078f10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2f80d3800 0x7fc2f80b01a0 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2f80d3420 0x7fc2f80cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.382+0000 7fc2f7e00640 1 --2- 172.21.15.44:0/2559237749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2f80b3420 0x7fc2f80ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.383+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 >> 172.21.15.44:0/2559237749 conn(0x7fc2f808ca20 msgr2=0x7fc2f80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.383+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 shutdown_connections 2024-07-17T08:45:00.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.383+0000 7fc2f7e00640 1 -- 172.21.15.44:0/2559237749 wait complete. 2024-07-17T08:45:00.406 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247/.. 2024-07-17T08:45:00.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --subvol sv06 --fs cephfs . 2024-07-17T08:45:00.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.623+0000 7f327ec00640 1 -- 172.21.15.44:0/2248871123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 msgr2=0x7f3260004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.623+0000 7f327ec00640 1 --2- 172.21.15.44:0/2248871123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 0x7f3260004c80 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f3264009d20 tx=0x7f3260005830 comp rx=0 tx=0).stop 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.623+0000 7f327e200640 1 -- 172.21.15.44:0/2248871123 reap_dead start 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.624+0000 7f327ec00640 1 -- 172.21.15.44:0/2248871123 shutdown_connections 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.624+0000 7f327ec00640 1 -- 172.21.15.44:0/2248871123 >> 172.21.15.44:0/2248871123 conn(0x7f32780cf0d0 msgr2=0x7f32780cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.624+0000 7f327ec00640 1 -- 172.21.15.44:0/2248871123 shutdown_connections 2024-07-17T08:45:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.624+0000 7f327ec00640 1 -- 172.21.15.44:0/2248871123 wait complete. 2024-07-17T08:45:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.625+0000 7f327ec00640 1 Processor -- start 2024-07-17T08:45:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.626+0000 7f327ec00640 1 -- start start 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.626+0000 7f327ec00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 0x7f32780cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32780c8a90 0x7f32781e5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327814f340 0x7f32781e7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ec00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f327806f650 con 0x7f3260004800 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f327806f4d0 con 0x7f32780c8a90 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f327815a450 con 0x7f327814f340 2024-07-17T08:45:00.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 0x7f32780cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 0x7f32780cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44256/0 (socket says 172.21.15.44:44256) 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 -- 172.21.15.44:0/2486491131 learned_addr learned my addr 172.21.15.44:0/2486491131 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ce00640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32780c8a90 0x7f32781e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 -- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327814f340 msgr2=0x7f32781e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327e200640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327814f340 0x7f32781e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327814f340 0x7f32781e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 -- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32780c8a90 msgr2=0x7f32781e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32780c8a90 0x7f32781e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327d800640 1 -- 172.21.15.44:0/2486491131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32781e5a20 con 0x7f3260004800 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327ce00640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32780c8a90 0x7f32781e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.627+0000 7f327e200640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f327814f340 0x7f32781e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f327d800640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3260004800 0x7f32780cab90 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7f326c0078d0 tx=0x7f326c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c019070 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f326c010ac0 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f327ec00640 1 -- 172.21.15.44:0/2486491131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32781e5d00 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c015450 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.628+0000 7f327ec00640 1 -- 172.21.15.44:0/2486491131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32780cec40 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.629+0000 7f327ec00640 1 -- 172.21.15.44:0/2486491131 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32781e5f40 con 0x7f3260004800 2024-07-17T08:45:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.631+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 214) ==== 99000+0+0 (secure 0 0 0) 0x7f326c01c070 con 0x7f3260004800 2024-07-17T08:45:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.631+0000 7f3275600640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3258076db0 0x7f3258079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.632+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f326c099300 con 0x7f3260004800 2024-07-17T08:45:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.632+0000 7f327ce00640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3258076db0 0x7f3258079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.633+0000 7f327ce00640 1 --2- 172.21.15.44:0/2486491131 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3258076db0 0x7f3258079270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f32640046c0 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:00.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.636+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f32781e5f40 con 0x7f3260004800 2024-07-17T08:45:00.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.868+0000 7f327ec00640 1 -- 172.21.15.44:0/2486491131 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "subvol": "sv06", "fs": "cephfs", "path": ".", "target": ["mon-mgr", ""]}) -- 0x7f32780b2eb0 con 0x7f3258076db0 2024-07-17T08:45:00.875 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:00.874+0000 7f2dfd000640 -1 mgr.server reply reply (22) Invalid argument No such subvol: None::sv06 2024-07-17T08:45:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:00.874+0000 7f3275600640 1 -- 172.21.15.44:0/2486491131 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: -22 No such subvol: None::sv06) ==== 34+0+0 (secure 0 0 0) 0x7f32780b2eb0 con 0x7f3258076db0 2024-07-17T08:45:00.875 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: No such subvol: None::sv06 2024-07-17T08:45:00.890 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:45:00.890 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --subvol sv06 --group mygrp06 --fs cephfs . 1m 2024-07-17T08:45:01.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 -- 172.21.15.44:0/1486265203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb814004800 msgr2=0x7fb814004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 --2- 172.21.15.44:0/1486265203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb814004800 0x7fb814004c80 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7fb81800bb20 tx=0x7fb814005830 comp rx=0 tx=0).stop 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb82f400640 1 -- 172.21.15.44:0/1486265203 reap_dead start 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 -- 172.21.15.44:0/1486265203 shutdown_connections 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 -- 172.21.15.44:0/1486265203 >> 172.21.15.44:0/1486265203 conn(0x7fb8300b0fb0 msgr2=0x7fb8300b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 -- 172.21.15.44:0/1486265203 shutdown_connections 2024-07-17T08:45:01.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.117+0000 7fb835600640 1 -- 172.21.15.44:0/1486265203 wait complete. 2024-07-17T08:45:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.119+0000 7fb835600640 1 Processor -- start 2024-07-17T08:45:01.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.119+0000 7fb835600640 1 -- start start 2024-07-17T08:45:01.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 0x7fb830153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8300733f0 0x7fb8300b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300b2e60 con 0x7fb830098c20 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300b2ce0 con 0x7fb814004800 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb835600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8300b2fe0 con 0x7fb8300733f0 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb826000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb826000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44268/0 (socket says 172.21.15.44:44268) 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.120+0000 7fb826000640 1 -- 172.21.15.44:0/1770549044 learned_addr learned my addr 172.21.15.44:0/1770549044 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb82ea00640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 0x7fb830153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb82f400640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8300733f0 0x7fb8300b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 -- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8300733f0 msgr2=0x7fb8300b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8300733f0 0x7fb8300b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 -- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 msgr2=0x7fb830153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 0x7fb830153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 -- 172.21.15.44:0/1770549044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb830153880 con 0x7fb830098c20 2024-07-17T08:45:01.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb82ea00640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 0x7fb830153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.121+0000 7fb826000640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fb81802b000 tx=0x7fb818031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.122+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818040070 con 0x7fb830098c20 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.122+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb818004030 con 0x7fb830098c20 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.122+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818035040 con 0x7fb830098c20 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.122+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8300af2c0 con 0x7fb830098c20 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.122+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83015fd40 con 0x7fb830098c20 2024-07-17T08:45:01.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.123+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830154180 con 0x7fb830098c20 2024-07-17T08:45:01.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.124+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 214) ==== 99000+0+0 (secure 0 0 0) 0x7fb818038020 con 0x7fb830098c20 2024-07-17T08:45:01.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.125+0000 7fb827400640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 0x7fb80c0792a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.126+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7fb8180b9800 con 0x7fb830098c20 2024-07-17T08:45:01.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.126+0000 7fb82ea00640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 0x7fb80c0792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.127+0000 7fb82ea00640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 0x7fb80c0792a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb820000c00 tx=0x7fb820006ae0 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:01.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.130+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb8180824f0 con 0x7fb830098c20 2024-07-17T08:45:01.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.368+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "subvol": "sv06", "group": "mygrp06", "fs": "cephfs", "path": ".", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb830143520 con 0x7fb80c076de0 2024-07-17T08:45:01.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.593+0000 7fb827400640 1 -- 172.21.15.44:0/1770549044 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+88 (secure 0 0 0) 0x7fb830143520 con 0x7fb80c076de0 2024-07-17T08:45:01.595 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247/.. 2024-07-17T08:45:01.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.596+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 msgr2=0x7fb80c0792a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.596+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 0x7fb80c0792a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb820000c00 tx=0x7fb820006ae0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.596+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 msgr2=0x7fb830160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.596+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fb81802b000 tx=0x7fb818031f90 comp rx=0 tx=0).stop 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 shutdown_connections 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fb80c076de0 0x7fb80c0792a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8300733f0 0x7fb8300b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb830098c20 0x7fb830160260 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 --2- 172.21.15.44:0/1770549044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb814004800 0x7fb830153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 >> 172.21.15.44:0/1770549044 conn(0x7fb8300b0fb0 msgr2=0x7fb8300724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 shutdown_connections 2024-07-17T08:45:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.597+0000 7fb835600640 1 -- 172.21.15.44:0/1770549044 wait complete. 2024-07-17T08:45:01.618 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247/.. 2024-07-17T08:45:01.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv06 --group_name mygrp06 2024-07-17T08:45:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 -- 172.21.15.44:0/2028225879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc1537d0 msgr2=0x7f2afc06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 --2- 172.21.15.44:0/2028225879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc1537d0 0x7f2afc06e6d0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0ad3e0 tx=0x7f2af002b890 comp rx=0 tx=0).stop 2024-07-17T08:45:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 -- 172.21.15.44:0/2028225879 shutdown_connections 2024-07-17T08:45:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 --2- 172.21.15.44:0/2028225879 >> v2:172.21.15.175:3300/0 conn(0x7f2afc0ae820 0x7f2afc0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 --2- 172.21.15.44:0/2028225879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc1537d0 0x7f2afc06e6d0 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 -- 172.21.15.44:0/2028225879 >> 172.21.15.44:0/2028225879 conn(0x7f2afc0cf050 msgr2=0x7f2afc0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 -- 172.21.15.44:0/2028225879 shutdown_connections 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.823+0000 7f2b02800640 1 -- 172.21.15.44:0/2028225879 wait complete. 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.825+0000 7f2b02800640 1 Processor -- start 2024-07-17T08:45:01.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.825+0000 7f2b02800640 1 -- start start 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc06ec10 0x7f2afc072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc1537d0 0x7f2afc1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0ac660 con 0x7f2afc0ae820 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0ac4e0 con 0x7f2afc06ec10 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b02800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc0ac7e0 con 0x7f2afc1537d0 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b00a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44276/0 (socket says 172.21.15.44:44276) 2024-07-17T08:45:01.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.826+0000 7f2b00a00640 1 -- 172.21.15.44:0/2689653576 learned_addr learned my addr 172.21.15.44:0/2689653576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b01400640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc06ec10 0x7f2afc072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 -- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc1537d0 msgr2=0x7f2afc1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc1537d0 0x7f2afc1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 -- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc06ec10 msgr2=0x7f2afc072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc06ec10 0x7f2afc072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 -- 172.21.15.44:0/2689653576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc071780 con 0x7f2afc0ae820 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2b00a00640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c3c0 tx=0x7f2ae800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8017070 con 0x7f2afc0ae820 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ae80035f0 con 0x7f2afc0ae820 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.827+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003a80 con 0x7f2afc0ae820 2024-07-17T08:45:01.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.828+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc071a60 con 0x7f2afc0ae820 2024-07-17T08:45:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.828+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc072020 con 0x7f2afc0ae820 2024-07-17T08:45:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.829+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0abfa0 con 0x7f2afc0ae820 2024-07-17T08:45:01.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.830+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 214) ==== 99000+0+0 (secure 0 0 0) 0x7f2ae801a070 con 0x7f2afc0ae820 2024-07-17T08:45:01.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.831+0000 7f2af9600640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 0x7f2adc079270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:01.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.831+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f2ae8097f50 con 0x7f2afc0ae820 2024-07-17T08:45:01.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.832+0000 7f2b01400640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 0x7f2adc079270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:01.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.833+0000 7f2b01400640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 0x7f2adc079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2af0035000 tx=0x7f2af0004730 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:01.836+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2ae8060cf0 con 0x7f2afc0ae820 2024-07-17T08:45:02.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.080+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv06", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7f2afc0af8b0 con 0x7f2adc076db0 2024-07-17T08:45:02.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.091+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 215) ==== 99056+0+0 (secure 0 0 0) 0x7f2ae805c880 con 0x7f2afc0ae820 2024-07-17T08:45:02.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.101+0000 7f2af9600640 1 -- 172.21.15.44:0/2689653576 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2afc0af8b0 con 0x7f2adc076db0 2024-07-17T08:45:02.103 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247 2024-07-17T08:45:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 msgr2=0x7f2adc079270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 0x7f2adc079270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2af0035000 tx=0x7f2af0004730 comp rx=0 tx=0).stop 2024-07-17T08:45:02.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 msgr2=0x7f2afc158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c3c0 tx=0x7f2ae800c880 comp rx=0 tx=0).stop 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 shutdown_connections 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2adc076db0 0x7f2adc079270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc1537d0 0x7f2afc1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2afc0ae820 0x7f2afc158390 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 --2- 172.21.15.44:0/2689653576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc06ec10 0x7f2afc072cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 >> 172.21.15.44:0/2689653576 conn(0x7f2afc0cf050 msgr2=0x7f2afc15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 shutdown_connections 2024-07-17T08:45:02.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.104+0000 7f2b02800640 1 -- 172.21.15.44:0/2689653576 wait complete. 2024-07-17T08:45:02.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:02.126 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/mygrp06/sv06/e9d0c53f-79d7-4afd-b753-ed2ab009e247/../.snap) 2024-07-17T08:45:02.184 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv06 --group_name mygrp06 2024-07-17T08:45:02.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 -- 172.21.15.44:0/72084737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a2c004800 msgr2=0x7f3a2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 --2- 172.21.15.44:0/72084737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a2c004800 0x7f3a2c004c80 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c710 tx=0x7f3a2c005830 comp rx=0 tx=0).stop 2024-07-17T08:45:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4ca00640 1 -- 172.21.15.44:0/72084737 reap_dead start 2024-07-17T08:45:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 -- 172.21.15.44:0/72084737 shutdown_connections 2024-07-17T08:45:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 -- 172.21.15.44:0/72084737 >> 172.21.15.44:0/72084737 conn(0x7f3a48073a60 msgr2=0x7f3a48073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 -- 172.21.15.44:0/72084737 shutdown_connections 2024-07-17T08:45:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.400+0000 7f3a4d400640 1 -- 172.21.15.44:0/72084737 wait complete. 2024-07-17T08:45:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.402+0000 7f3a4d400640 1 Processor -- start 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.402+0000 7f3a4d400640 1 -- start start 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 0x7f3a480b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a481f7270 0x7f3a481f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a480b87f0 con 0x7f3a48156a40 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a480b8670 con 0x7f3a481f7270 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a4d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a480b8970 con 0x7f3a2c004800 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44288/0 (socket says 172.21.15.44:44288) 2024-07-17T08:45:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 -- 172.21.15.44:0/843096282 learned_addr learned my addr 172.21.15.44:0/843096282 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 -- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 msgr2=0x7f3a480b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47e00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 0x7f3a480b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 0x7f3a480b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.403+0000 7f3a47400640 1 -- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a481f7270 msgr2=0x7f3a481f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a4ca00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a481f7270 0x7f3a481f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a47400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a481f7270 0x7f3a481f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a47400640 1 -- 172.21.15.44:0/843096282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a48155860 con 0x7f3a48156a40 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a47e00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 0x7f3a480b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a47400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c630 tx=0x7f3a30006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30004830 con 0x7f3a48156a40 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a30039070 con 0x7f3a48156a40 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a48155b40 con 0x7f3a48156a40 2024-07-17T08:45:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30004d80 con 0x7f3a48156a40 2024-07-17T08:45:02.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.404+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a48156100 con 0x7f3a48156a40 2024-07-17T08:45:02.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.406+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a48156530 con 0x7f3a48156a40 2024-07-17T08:45:02.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.407+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 215) ==== 99056+0+0 (secure 0 0 0) 0x7f3a30043030 con 0x7f3a48156a40 2024-07-17T08:45:02.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.408+0000 7f3a3fe00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 0x7f3a24079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.408+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f3a300b86b0 con 0x7f3a48156a40 2024-07-17T08:45:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.408+0000 7f3a47e00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 0x7f3a24079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.409+0000 7f3a47e00640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 0x7f3a24079330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a481569d0 tx=0x7f3a38009040 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.414+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3a3004a020 con 0x7f3a48156a40 2024-07-17T08:45:02.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.655+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv06", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7f3a480ce500 con 0x7f3a24076e70 2024-07-17T08:45:02.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.700+0000 7f3a3fe00640 1 -- 172.21.15.44:0/843096282 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a480ce500 con 0x7f3a24076e70 2024-07-17T08:45:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 msgr2=0x7f3a24079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 0x7f3a24079330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a481569d0 tx=0x7f3a38009040 comp rx=0 tx=0).stop 2024-07-17T08:45:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 msgr2=0x7f3a48156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c630 tx=0x7f3a30006380 comp rx=0 tx=0).stop 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 shutdown_connections 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3a24076e70 0x7f3a24079330 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a481f7270 0x7f3a481f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a48156a40 0x7f3a48156ec0 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 --2- 172.21.15.44:0/843096282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a2c004800 0x7f3a480b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.703+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 >> 172.21.15.44:0/843096282 conn(0x7f3a48073a60 msgr2=0x7f3a4815edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.704+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 shutdown_connections 2024-07-17T08:45:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.704+0000 7f3a4d400640 1 -- 172.21.15.44:0/843096282 wait complete. 2024-07-17T08:45:02.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs mygrp06 2024-07-17T08:45:02.729 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:02.729+0000 7f2defe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:02.729 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:02.729+0000 7f2defe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:02.729 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:02.729+0000 7f2defe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:02.729 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:02.729+0000 7f2defe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:02.730 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:02.729+0000 7f2defe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 -- 172.21.15.44:0/838916867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c003af0 msgr2=0x7fc12c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 --2- 172.21.15.44:0/838916867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc12c003af0 0x7fc12c003f10 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7fc1380016c0 tx=0x7fc12c005830 comp rx=0 tx=0).stop 2024-07-17T08:45:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14d200640 1 -- 172.21.15.44:0/838916867 reap_dead start 2024-07-17T08:45:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 -- 172.21.15.44:0/838916867 shutdown_connections 2024-07-17T08:45:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 -- 172.21.15.44:0/838916867 >> 172.21.15.44:0/838916867 conn(0x7fc1480cf0d0 msgr2=0x7fc1480cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 -- 172.21.15.44:0/838916867 shutdown_connections 2024-07-17T08:45:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.938+0000 7fc14dc00640 1 -- 172.21.15.44:0/838916867 wait complete. 2024-07-17T08:45:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.940+0000 7fc14dc00640 1 Processor -- start 2024-07-17T08:45:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.940+0000 7fc14dc00640 1 -- start start 2024-07-17T08:45:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 0x7fc14809cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc14815aac0 0x7fc1481f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b3030 con 0x7fc1481f3750 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b2eb0 con 0x7fc14815aac0 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480b31b0 con 0x7fc12c003af0 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.941+0000 7fc14d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44312/0 (socket says 172.21.15.44:44312) 2024-07-17T08:45:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 -- 172.21.15.44:0/12688045 learned_addr learned my addr 172.21.15.44:0/12688045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc147e00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 0x7fc14809cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc147400640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc14815aac0 0x7fc1481f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 -- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 msgr2=0x7fc14809cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 0x7fc14809cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 -- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc14815aac0 msgr2=0x7fc1481f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc14815aac0 0x7fc1481f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 -- 172.21.15.44:0/12688045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14815af50 con 0x7fc1481f3750 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc147e00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 0x7fc14809cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.942+0000 7fc14d200640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ae130 tx=0x7fc14000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.943+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140004d20 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.943+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc140003970 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.943+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14815b230 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.943+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140003dc0 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.943+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1481f2a10 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.944+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1481f2ef0 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.945+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 215) ==== 99056+0+0 (secure 0 0 0) 0x7fc140003f60 con 0x7fc1481f3750 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.946+0000 7fc13fe00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 0x7fc128079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.946+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7fc140099570 con 0x7fc1481f3750 2024-07-17T08:45:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.947+0000 7fc147e00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 0x7fc128079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.948+0000 7fc147e00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 0x7fc128079300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc138032000 tx=0x7fc138001190 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:02.951+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc1481f2ef0 con 0x7fc1481f3750 2024-07-17T08:45:03.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.193+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 --> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "mygrp06", "target": ["mon-mgr", ""]}) -- 0x7fc1480ae340 con 0x7fc128076e40 2024-07-17T08:45:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.207+0000 7fc13fe00640 1 -- 172.21.15.44:0/12688045 <== mgr.9971 v2:172.21.15.175:6834/2989936540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1480ae340 con 0x7fc128076e40 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.210+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 msgr2=0x7fc128079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.210+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 0x7fc128079300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc138032000 tx=0x7fc138001190 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 msgr2=0x7fc14806bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ae130 tx=0x7fc14000d7b0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 shutdown_connections 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fc128076e40 0x7fc128079300 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1481f3750 0x7fc14806bf40 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc14815aac0 0x7fc1481f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 --2- 172.21.15.44:0/12688045 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc12c003af0 0x7fc14809cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 >> 172.21.15.44:0/12688045 conn(0x7fc1480cf0d0 msgr2=0x7fc1480733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 shutdown_connections 2024-07-17T08:45:03.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.211+0000 7fc14dc00640 1 -- 172.21.15.44:0/12688045 wait complete. 2024-07-17T08:45:03.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:45:03.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 -- 172.21.15.44:0/3429754750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 msgr2=0x7fda74005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 --2- 172.21.15.44:0/3429754750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda74005d90 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7fda780016c0 tx=0x7fda74006940 comp rx=0 tx=0).stop 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 -- 172.21.15.44:0/3429754750 shutdown_connections 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 --2- 172.21.15.44:0/3429754750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda74005d90 unknown :-1 s=CLOSED pgs=3239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 -- 172.21.15.44:0/3429754750 >> 172.21.15.44:0/3429754750 conn(0x7fda9006c120 msgr2=0x7fda9006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 -- 172.21.15.44:0/3429754750 shutdown_connections 2024-07-17T08:45:03.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.451+0000 7fda94a00640 1 -- 172.21.15.44:0/3429754750 wait complete. 2024-07-17T08:45:03.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.452+0000 7fda94a00640 1 Processor -- start 2024-07-17T08:45:03.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.453+0000 7fda94a00640 1 -- start start 2024-07-17T08:45:03.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:03.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 0x7fda90158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:03.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 0x7fda900b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:03.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda900b24d0 con 0x7fda74005910 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda900b2350 con 0x7fda900c9f10 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda94a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda900cefe0 con 0x7fda901f7db0 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44338/0 (socket says 172.21.15.44:44338) 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 -- 172.21.15.44:0/2985139080 learned_addr learned my addr 172.21.15.44:0/2985139080 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8de00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 0x7fda90158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 -- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 msgr2=0x7fda900b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8f200640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 0x7fda900b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 0x7fda900b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 -- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 msgr2=0x7fda90158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 0x7fda90158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8e800640 1 -- 172.21.15.44:0/2985139080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda901581b0 con 0x7fda74005910 2024-07-17T08:45:03.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.454+0000 7fda8de00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 0x7fda90158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:03.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda8f200640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 0x7fda900b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:03.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda8e800640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7fda8000ea80 tx=0x7fda8000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8000cea0 con 0x7fda74005910 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda80004c10 con 0x7fda74005910 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80005020 con 0x7fda74005910 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda90158490 con 0x7fda74005910 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.455+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda90158a50 con 0x7fda74005910 2024-07-17T08:45:03.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.457+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda900ae480 con 0x7fda74005910 2024-07-17T08:45:03.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.458+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 215) ==== 99056+0+0 (secure 0 0 0) 0x7fda800051c0 con 0x7fda74005910 2024-07-17T08:45:03.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.459+0000 7fda86a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 0x7fda6c079360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:03.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.459+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7fda80099b20 con 0x7fda74005910 2024-07-17T08:45:03.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.459+0000 7fda8de00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 0x7fda6c079360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:03.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.461+0000 7fda8de00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 0x7fda6c079360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda78032000 tx=0x7fda78001190 comp rx=0 tx=0).ready entity=mgr.9971 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.463+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fda80062610 con 0x7fda74005910 2024-07-17T08:45:03.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:03.729+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7fda901564c0 con 0x7fda74005910 2024-07-17T08:45:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.101+0000 7fda86a00640 1 -- 172.21.15.44:0/2985139080 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v216) ==== 93+0+0 (secure 0 0 0) 0x7fda80061f10 con 0x7fda74005910 2024-07-17T08:45:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 msgr2=0x7fda6c079360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 0x7fda6c079360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda78032000 tx=0x7fda78001190 comp rx=0 tx=0).stop 2024-07-17T08:45:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 msgr2=0x7fda900ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7fda8000ea80 tx=0x7fda8000ef40 comp rx=0 tx=0).stop 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 shutdown_connections 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7fda6c076ea0 0x7fda6c079360 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda901f7db0 0x7fda900b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda900c9f10 0x7fda90158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 --2- 172.21.15.44:0/2985139080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda74005910 0x7fda900ce470 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.106+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 >> 172.21.15.44:0/2985139080 conn(0x7fda9006c120 msgr2=0x7fda9015bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.107+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 shutdown_connections 2024-07-17T08:45:04.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.107+0000 7fda94a00640 1 -- 172.21.15.44:0/2985139080 wait complete. 2024-07-17T08:45:04.138 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:45:04.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:45:04.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:45:04.332 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:45:04.332 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:45:04.332 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:04.332 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:04.380 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:04.380 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:04.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:45:04.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7e00640 1 -- 172.21.15.44:0/1526860472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc003aa0 msgr2=0x7f1bdc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/1526860472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bdc003aa0 0x7f1bdc003ec0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c310 tx=0x7f1bdc005830 comp rx=0 tx=0).stop 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7400640 1 -- 172.21.15.44:0/1526860472 reap_dead start 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7e00640 1 -- 172.21.15.44:0/1526860472 shutdown_connections 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7e00640 1 -- 172.21.15.44:0/1526860472 >> 172.21.15.44:0/1526860472 conn(0x7f1bf80b4ff0 msgr2=0x7f1bf80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.664+0000 7f1bf7e00640 1 -- 172.21.15.44:0/1526860472 shutdown_connections 2024-07-17T08:45:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.665+0000 7f1bf7e00640 1 -- 172.21.15.44:0/1526860472 wait complete. 2024-07-17T08:45:04.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.666+0000 7f1bf7e00640 1 Processor -- start 2024-07-17T08:45:04.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.666+0000 7f1bf7e00640 1 -- start start 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc003aa0 0x7f1bf814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 0x7f1bf815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf8166400 con 0x7f1bf81606a0 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf8166280 con 0x7f1bf814d710 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.667+0000 7f1bf7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf8166580 con 0x7f1bdc003aa0 2024-07-17T08:45:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44372/0 (socket says 172.21.15.44:44372) 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 -- 172.21.15.44:0/770492615 learned_addr learned my addr 172.21.15.44:0/770492615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6a00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc003aa0 0x7f1bf814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf7400640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 0x7f1bf815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc003aa0 msgr2=0x7f1bf814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc003aa0 0x7f1bf814d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 msgr2=0x7f1bf815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 0x7f1bf815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf6000640 1 -- 172.21.15.44:0/770492615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf815e4e0 con 0x7f1bf81606a0 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.668+0000 7f1bf7400640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 0x7f1bf815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1bf6000640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f1be4004d90 tx=0x7f1be4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40107d0 con 0x7f1bf81606a0 2024-07-17T08:45:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be4016070 con 0x7f1bf81606a0 2024-07-17T08:45:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4010d00 con 0x7f1bf81606a0 2024-07-17T08:45:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf80b8f00 con 0x7f1bf81606a0 2024-07-17T08:45:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.669+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf815e7a0 con 0x7f1bf81606a0 2024-07-17T08:45:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.670+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf80b2200 con 0x7f1bf81606a0 2024-07-17T08:45:04.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.673+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f1be401b460 con 0x7f1bf81606a0 2024-07-17T08:45:04.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.673+0000 7f1beea00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 0x7f1bd8079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:04.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.674+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f1be4099450 con 0x7f1bf81606a0 2024-07-17T08:45:04.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.674+0000 7f1bf6a00640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 msgr2=0x7f1bd8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:04.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.674+0000 7f1bf6a00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 0x7f1bd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:04.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.682+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1be4061ff0 con 0x7f1bf81606a0 2024-07-17T08:45:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.876+0000 7f1bf6a00640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 msgr2=0x7f1bd8079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:04.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.876+0000 7f1bf6a00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 0x7f1bd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:04.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.907+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1bf815bf70 con 0x7f1bf81606a0 2024-07-17T08:45:04.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.908+0000 7f1beea00640 1 -- 172.21.15.44:0/770492615 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v352) ==== 80+0+478 (secure 0 0 0) 0x7f1be40618f0 con 0x7f1bf81606a0 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2938461940 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1993189342 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2478355158 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2460909214 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/3157277076 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3314250175 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/3157277076 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.910 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3191294602 2024-07-18T08:44:57.990054+0000 2024-07-17T08:45:04.911 INFO:teuthology.orchestra.run.smithi044.stderr:listed 8 entries 2024-07-17T08:45:04.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.910+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 msgr2=0x7f1bd8079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:04.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.910+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 0x7f1bd8079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.910+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 msgr2=0x7f1bf815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:04.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.910+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f1be4004d90 tx=0x7f1be4001630 comp rx=0 tx=0).stop 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 shutdown_connections 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f1bd8076eb0 0x7f1bd8079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bf814d710 0x7f1bf815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bf81606a0 0x7f1bf815da60 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 --2- 172.21.15.44:0/770492615 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc003aa0 0x7f1bf814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 >> 172.21.15.44:0/770492615 conn(0x7f1bf80b4ff0 msgr2=0x7f1bf80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 shutdown_connections 2024-07-17T08:45:04.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:04.911+0000 7f1bf7e00640 1 -- 172.21.15.44:0/770492615 wait complete. 2024-07-17T08:45:04.930 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:45:04.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:04.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:45:05.010 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:45:05.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:05.011 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:45:05.059 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:45:05.060 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:45:05.060 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:05.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:45:05.101 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:45:05.102 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:05.102 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:05.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:45:05.146 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:45:05.146 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:45:05.147 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:05.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:45:05.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:05.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 -- 172.21.15.44:0/3641164573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352815eff0 msgr2=0x7f352806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 --2- 172.21.15.44:0/3641164573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352815eff0 0x7f352806f7a0 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f35280cdf30 tx=0x7f352002a820 comp rx=0 tx=0).stop 2024-07-17T08:45:05.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 -- 172.21.15.44:0/3641164573 shutdown_connections 2024-07-17T08:45:05.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 --2- 172.21.15.44:0/3641164573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352815eff0 0x7f352806f7a0 unknown :-1 s=CLOSED pgs=3249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 --2- 172.21.15.44:0/3641164573 >> v2:172.21.15.175:3300/0 conn(0x7f35281444d0 0x7f35281448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 -- 172.21.15.44:0/3641164573 >> 172.21.15.44:0/3641164573 conn(0x7f3528072730 msgr2=0x7f3528072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:05.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 -- 172.21.15.44:0/3641164573 shutdown_connections 2024-07-17T08:45:05.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.394+0000 7f352f600640 1 -- 172.21.15.44:0/3641164573 wait complete. 2024-07-17T08:45:05.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.396+0000 7f352f600640 1 Processor -- start 2024-07-17T08:45:05.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.396+0000 7f352f600640 1 -- start start 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 0x7f35280aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 0x7f3528142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f35280700d0 con 0x7f35281444d0 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f352806ff50 con 0x7f35280cd8e0 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35280b5850 con 0x7f352815eff0 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f3527e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f3527e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44390/0 (socket says 172.21.15.44:44390) 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f3527e00640 1 -- 172.21.15.44:0/1794426481 learned_addr learned my addr 172.21.15.44:0/1794426481 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:05.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.397+0000 7f352cc00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 0x7f35280aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f352d600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 0x7f3528142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 msgr2=0x7f3528142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 0x7f3528142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 msgr2=0x7f35280aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 0x7f35280aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 -- 172.21.15.44:0/1794426481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3528141d10 con 0x7f35281444d0 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f352cc00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 0x7f35280aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f352d600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 0x7f3528142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3527e00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f351400c3f0 tx=0x7f351400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514017070 con 0x7f35281444d0 2024-07-17T08:45:05.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35140035f0 con 0x7f35281444d0 2024-07-17T08:45:05.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.398+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514003a40 con 0x7f35281444d0 2024-07-17T08:45:05.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.399+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3528141ff0 con 0x7f35281444d0 2024-07-17T08:45:05.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.399+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281425b0 con 0x7f35281444d0 2024-07-17T08:45:05.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.400+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35281448b0 con 0x7f35281444d0 2024-07-17T08:45:05.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.401+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f351401a070 con 0x7f35281444d0 2024-07-17T08:45:05.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.402+0000 7f3524c00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 0x7f3508079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.402+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f35140980d0 con 0x7f35281444d0 2024-07-17T08:45:05.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.402+0000 7f352cc00640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 msgr2=0x7f3508079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:05.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.402+0000 7f352cc00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 0x7f3508079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:05.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.411+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3514060cb0 con 0x7f35281444d0 2024-07-17T08:45:05.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.604+0000 7f352cc00640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 msgr2=0x7f3508079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:05.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.604+0000 7f352cc00640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 0x7f3508079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:05.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.655+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3528075550 con 0x7f35281444d0 2024-07-17T08:45:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.656+0000 7f3524c00640 1 -- 172.21.15.44:0/1794426481 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7f35140605b0 con 0x7f35281444d0 2024-07-17T08:45:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:05.658 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:45:05.658 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:45:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 msgr2=0x7f3508079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 0x7f3508079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 msgr2=0x7f35281f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f351400c3f0 tx=0x7f351400c8b0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 shutdown_connections 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3508076e80 0x7f3508079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f352815eff0 0x7f3528142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35281444d0 0x7f35281f4310 unknown :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 --2- 172.21.15.44:0/1794426481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35280cd8e0 0x7f35280aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 >> 172.21.15.44:0/1794426481 conn(0x7f3528072730 msgr2=0x7f3528072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 shutdown_connections 2024-07-17T08:45:05.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.659+0000 7f352f600640 1 -- 172.21.15.44:0/1794426481 wait complete. 2024-07-17T08:45:05.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:05.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 -- 172.21.15.44:0/2923333018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e180704f0 msgr2=0x7f7e180ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 --2- 172.21.15.44:0/2923333018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e180704f0 0x7f7e180ae300 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f7e18072100 tx=0x7f7e1002aa30 comp rx=0 tx=0).stop 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 -- 172.21.15.44:0/2923333018 shutdown_connections 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 --2- 172.21.15.44:0/2923333018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e180704f0 0x7f7e180ae300 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 --2- 172.21.15.44:0/2923333018 >> v2:172.21.15.175:3300/0 conn(0x7f7e1815b560 0x7f7e180cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 -- 172.21.15.44:0/2923333018 >> 172.21.15.44:0/2923333018 conn(0x7f7e180b3020 msgr2=0x7f7e180b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 -- 172.21.15.44:0/2923333018 shutdown_connections 2024-07-17T08:45:05.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.876+0000 7f7e1d400640 1 -- 172.21.15.44:0/2923333018 wait complete. 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.878+0000 7f7e1d400640 1 Processor -- start 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.878+0000 7f7e1d400640 1 -- start start 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 0x7f7e180b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e180cfaa0 0x7f7e1814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180ae9c0 con 0x7f7e1815b560 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180ae840 con 0x7f7e180cfaa0 2024-07-17T08:45:05.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e1d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e180aeb40 con 0x7f7e180704f0 2024-07-17T08:45:05.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e17200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e17200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44398/0 (socket says 172.21.15.44:44398) 2024-07-17T08:45:05.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.879+0000 7f7e17200640 1 -- 172.21.15.44:0/3516741406 learned_addr learned my addr 172.21.15.44:0/3516741406 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e15e00640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e180cfaa0 0x7f7e1814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e16800640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 0x7f7e180b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 msgr2=0x7f7e180b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 0x7f7e180b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e180cfaa0 msgr2=0x7f7e1814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e180cfaa0 0x7f7e1814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:05.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 -- 172.21.15.44:0/3516741406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e180b67b0 con 0x7f7e1815b560 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e16800640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 0x7f7e180b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.880+0000 7f7e17200640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009d60 tx=0x7f7e10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.881+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10047070 con 0x7f7e1815b560 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.881+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e10004250 con 0x7f7e1815b560 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.881+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1806d410 con 0x7f7e1815b560 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.881+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10033050 con 0x7f7e1815b560 2024-07-17T08:45:05.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.881+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181f82e0 con 0x7f7e1815b560 2024-07-17T08:45:05.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.882+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e181580f0 con 0x7f7e1815b560 2024-07-17T08:45:05.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.883+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f7e1004a070 con 0x7f7e1815b560 2024-07-17T08:45:05.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.885+0000 7f7e0ea00640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 0x7f7df4079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:05.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.885+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f7e100ba060 con 0x7f7e1815b560 2024-07-17T08:45:05.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.885+0000 7f7e16800640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 msgr2=0x7f7df4079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:05.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.885+0000 7f7e16800640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 0x7f7df4079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:05.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:05.889+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7e180b67b0 con 0x7f7e1815b560 2024-07-17T08:45:06.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.087+0000 7f7e16800640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 msgr2=0x7f7df4079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:06.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.087+0000 7f7e16800640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 0x7f7df4079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:06.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.136+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e18157e10 con 0x7f7e1815b560 2024-07-17T08:45:06.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.137+0000 7f7e0ea00640 1 -- 172.21.15.44:0/3516741406 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7f7e10082b70 con 0x7f7e1815b560 2024-07-17T08:45:06.138 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:06.139 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:45:06.139 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:45:06.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.139+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 msgr2=0x7f7df4079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:06.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.139+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 0x7f7df4079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.139+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 msgr2=0x7f7e181f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.139+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009d60 tx=0x7f7e10003f10 comp rx=0 tx=0).stop 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 shutdown_connections 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f7df4076eb0 0x7f7df4079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e1815b560 0x7f7e181f7890 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e180cfaa0 0x7f7e1814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 --2- 172.21.15.44:0/3516741406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e180704f0 0x7f7e180b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 >> 172.21.15.44:0/3516741406 conn(0x7f7e180b3020 msgr2=0x7f7e1815da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 shutdown_connections 2024-07-17T08:45:06.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.140+0000 7f7e1d400640 1 -- 172.21.15.44:0/3516741406 wait complete. 2024-07-17T08:45:06.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 -- 172.21.15.44:0/1690936413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e4004d40 msgr2=0x7f00e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 --2- 172.21.15.44:0/1690936413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00e4004d40 0x7f00e40051c0 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 crypto rx=0x7f00fc15b5d0 tx=0x7f00e4005830 comp rx=0 tx=0).stop 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0101c00640 1 -- 172.21.15.44:0/1690936413 reap_dead start 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 -- 172.21.15.44:0/1690936413 shutdown_connections 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 -- 172.21.15.44:0/1690936413 >> 172.21.15.44:0/1690936413 conn(0x7f00fc072730 msgr2=0x7f00fc072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 -- 172.21.15.44:0/1690936413 shutdown_connections 2024-07-17T08:45:06.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.345+0000 7f0103c00640 1 -- 172.21.15.44:0/1690936413 wait complete. 2024-07-17T08:45:06.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.347+0000 7f0103c00640 1 Processor -- start 2024-07-17T08:45:06.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.347+0000 7f0103c00640 1 -- start start 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e4004d40 0x7f00fc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc1f1ef0 0x7f00fc0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0afcd0 con 0x7f00fc1f6170 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0afb50 con 0x7f00e4004d40 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f0103c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc0afe50 con 0x7f00fc1f1ef0 2024-07-17T08:45:06.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:06.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f00fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44420/0 (socket says 172.21.15.44:44420) 2024-07-17T08:45:06.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.348+0000 7f00fbe00640 1 -- 172.21.15.44:0/2409989148 learned_addr learned my addr 172.21.15.44:0/2409989148 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f0101200640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e4004d40 0x7f00fc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc1f1ef0 msgr2=0x7f00fc0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc1f1ef0 0x7f00fc0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e4004d40 msgr2=0x7f00fc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e4004d40 0x7f00fc157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 -- 172.21.15.44:0/2409989148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1f2430 con 0x7f00fc1f6170 2024-07-17T08:45:06.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00fbe00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 crypto rx=0x7f00e800fa60 tx=0x7f00e800ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e800bcf0 con 0x7f00fc1f6170 2024-07-17T08:45:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00e8011440 con 0x7f00fc1f6170 2024-07-17T08:45:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8011890 con 0x7f00fc1f6170 2024-07-17T08:45:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.349+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc1f2710 con 0x7f00fc1f6170 2024-07-17T08:45:06.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.350+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1f2cd0 con 0x7f00fc1f6170 2024-07-17T08:45:06.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.351+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc0b3850 con 0x7f00fc1f6170 2024-07-17T08:45:06.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.352+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f00e8019020 con 0x7f00fc1f6170 2024-07-17T08:45:06.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.353+0000 7f00f8c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 0x7f00dc079370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.353+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f00e80997e0 con 0x7f00fc1f6170 2024-07-17T08:45:06.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.353+0000 7f0101200640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 msgr2=0x7f00dc079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:06.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.353+0000 7f0101200640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 0x7f00dc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:06.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.357+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f00fc0b3850 con 0x7f00fc1f6170 2024-07-17T08:45:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.555+0000 7f0101200640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 msgr2=0x7f00dc079370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.555+0000 7f0101200640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 0x7f00dc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.567+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00fc050790 con 0x7f00fc1f6170 2024-07-17T08:45:06.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.570+0000 7f00f8c00640 1 -- 172.21.15.44:0/2409989148 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v352) ==== 74+0+18420 (secure 0 0 0) 0x7f00fc050790 con 0x7f00fc1f6170 2024-07-17T08:45:06.572 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:06.572 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":352,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:57.990183+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":47,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:44:41.136788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":48,"pool_name":"cephfs_data","create_time":"2024-07-17T08:44:42.144654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"351","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2938461940":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/1993189342":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2478355158":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2460909214":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6837/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3314250175":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6834/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3191294602":"2024-07-18T08:44:57.990054+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:06.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 msgr2=0x7f00dc079370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:06.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 0x7f00dc079370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 msgr2=0x7f00fc1f19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 crypto rx=0x7f00e800fa60 tx=0x7f00e800ff20 comp rx=0 tx=0).stop 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 shutdown_connections 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f00dc076eb0 0x7f00dc079370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc1f1ef0 0x7f00fc0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00fc1f6170 0x7f00fc1f19b0 unknown :-1 s=CLOSED pgs=3255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 --2- 172.21.15.44:0/2409989148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e4004d40 0x7f00fc157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.572+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 >> 172.21.15.44:0/2409989148 conn(0x7f00fc072730 msgr2=0x7f00fc158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.573+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 shutdown_connections 2024-07-17T08:45:06.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.573+0000 7f0103c00640 1 -- 172.21.15.44:0/2409989148 wait complete. 2024-07-17T08:45:06.591 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:45:06.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:06.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 -- 172.21.15.44:0/1911857214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 msgr2=0x7f4e340b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1911857214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e340b4ec0 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005fd0 tx=0x7f4e2402a580 comp rx=0 tx=0).stop 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 -- 172.21.15.44:0/1911857214 shutdown_connections 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1911857214 >> v2:172.21.15.175:3300/0 conn(0x7f4e340b5400 0x7f4e3415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1911857214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e340b4ec0 unknown :-1 s=CLOSED pgs=3256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.775+0000 7f4e33e00640 1 -- 172.21.15.44:0/1911857214 >> 172.21.15.44:0/1911857214 conn(0x7f4e340cfec0 msgr2=0x7f4e340d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.776+0000 7f4e33e00640 1 -- 172.21.15.44:0/1911857214 shutdown_connections 2024-07-17T08:45:06.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.776+0000 7f4e33e00640 1 -- 172.21.15.44:0/1911857214 wait complete. 2024-07-17T08:45:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.777+0000 7f4e33e00640 1 Processor -- start 2024-07-17T08:45:06.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.777+0000 7f4e33e00640 1 -- start start 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e340b5400 0x7f4e3415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e340ce5c0 0x7f4e341f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3406e0c0 con 0x7f4e340b8c80 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340b9320 con 0x7f4e340b5400 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.778+0000 7f4e33e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3406e240 con 0x7f4e340ce5c0 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44426/0 (socket says 172.21.15.44:44426) 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 -- 172.21.15.44:0/1654775935 learned_addr learned my addr 172.21.15.44:0/1654775935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32a00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e340b5400 0x7f4e3415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:06.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e33400640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e340ce5c0 0x7f4e341f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e340ce5c0 msgr2=0x7f4e341f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e340ce5c0 0x7f4e341f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e340b5400 msgr2=0x7f4e3415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e340b5400 0x7f4e3415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.779+0000 7f4e32000640 1 -- 172.21.15.44:0/1654775935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e340c9b40 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e32000640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b3c0 tx=0x7f4e2000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011830 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e20016070 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011d80 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e340c9e20 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.780+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3415dfb0 con 0x7f4e340b8c80 2024-07-17T08:45:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.781+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3406e750 con 0x7f4e340b8c80 2024-07-17T08:45:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.782+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f4e2001b460 con 0x7f4e340b8c80 2024-07-17T08:45:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.783+0000 7f4e2aa00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 0x7f4e14079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.784+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f4e20098a00 con 0x7f4e340b8c80 2024-07-17T08:45:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.784+0000 7f4e32a00640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 msgr2=0x7f4e14079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.784+0000 7f4e32a00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 0x7f4e14079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:06.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.788+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4e20061670 con 0x7f4e340b8c80 2024-07-17T08:45:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.985+0000 7f4e32a00640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 msgr2=0x7f4e14079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:06.985+0000 7f4e32a00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 0x7f4e14079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:07.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.037+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e340c9ff0 con 0x7f4e340b8c80 2024-07-17T08:45:07.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.038+0000 7f4e2aa00640 1 -- 172.21.15.44:0/1654775935 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7f4e20060f70 con 0x7f4e340b8c80 2024-07-17T08:45:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:07.040 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:45:07.040 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 msgr2=0x7f4e14079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 0x7f4e14079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 msgr2=0x7f4e341e75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b3c0 tx=0x7f4e2000b880 comp rx=0 tx=0).stop 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 shutdown_connections 2024-07-17T08:45:07.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f4e14076e80 0x7f4e14079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e340ce5c0 0x7f4e341f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e340b8c80 0x7f4e341e75e0 unknown :-1 s=CLOSED pgs=3257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 --2- 172.21.15.44:0/1654775935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e340b5400 0x7f4e3415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 >> 172.21.15.44:0/1654775935 conn(0x7f4e340cfec0 msgr2=0x7f4e340d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 shutdown_connections 2024-07-17T08:45:07.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.041+0000 7f4e33e00640 1 -- 172.21.15.44:0/1654775935 wait complete. 2024-07-17T08:45:07.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 -- 172.21.15.44:0/1162236432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 msgr2=0x7f294415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 --2- 172.21.15.44:0/1162236432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f294415d2c0 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 crypto rx=0x7f292c009900 tx=0x7f292c02a610 comp rx=0 tx=0).stop 2024-07-17T08:45:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 -- 172.21.15.44:0/1162236432 shutdown_connections 2024-07-17T08:45:07.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 --2- 172.21.15.44:0/1162236432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 0x7f294415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 --2- 172.21.15.44:0/1162236432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f294415d2c0 unknown :-1 s=CLOSED pgs=3258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 -- 172.21.15.44:0/1162236432 >> 172.21.15.44:0/1162236432 conn(0x7f29440cfec0 msgr2=0x7f29440d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 -- 172.21.15.44:0/1162236432 shutdown_connections 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.248+0000 7f2949800640 1 -- 172.21.15.44:0/1162236432 wait complete. 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.250+0000 7f2949800640 1 Processor -- start 2024-07-17T08:45:07.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.250+0000 7f2949800640 1 -- start start 2024-07-17T08:45:07.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 0x7f294414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 0x7f29440cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f294415e0d0 con 0x7f29440ce5c0 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f294415df50 con 0x7f29440b8c80 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2949800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f294415e250 con 0x7f29440b9060 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2948e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2948e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44452/0 (socket says 172.21.15.44:44452) 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2948e00640 1 -- 172.21.15.44:0/192610643 learned_addr learned my addr 172.21.15.44:0/192610643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2943e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 0x7f294414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.251+0000 7f2943400640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 0x7f29440cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 msgr2=0x7f29440cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 0x7f29440cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 msgr2=0x7f294414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 0x7f294414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 -- 172.21.15.44:0/192610643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29441f4420 con 0x7f29440ce5c0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2943e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 0x7f294414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2948e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7f293c00c970 tx=0x7f293c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c007bb0 con 0x7f29440ce5c0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f293c00f450 con 0x7f29440ce5c0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c007e60 con 0x7f29440ce5c0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.252+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29441f4700 con 0x7f29440ce5c0 2024-07-17T08:45:07.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.253+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441f4cc0 con 0x7f29440ce5c0 2024-07-17T08:45:07.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.254+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294415e700 con 0x7f29440ce5c0 2024-07-17T08:45:07.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.255+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f293c00f8a0 con 0x7f29440ce5c0 2024-07-17T08:45:07.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.256+0000 7f293be00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 0x7f2920079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.256+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f293c099550 con 0x7f29440ce5c0 2024-07-17T08:45:07.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.256+0000 7f2943e00640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 msgr2=0x7f2920079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:07.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.256+0000 7f2943e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 0x7f2920079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:07.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.260+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f293c014030 con 0x7f29440ce5c0 2024-07-17T08:45:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.457+0000 7f2943e00640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 msgr2=0x7f2920079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.457+0000 7f2943e00640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 0x7f2920079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:07.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.503+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29440b5d70 con 0x7f29440ce5c0 2024-07-17T08:45:07.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.504+0000 7f293be00640 1 -- 172.21.15.44:0/192610643 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5244 (secure 0 0 0) 0x7f293c0620b0 con 0x7f29440ce5c0 2024-07-17T08:45:07.506 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:07.507 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":200,"btime":"2024-07-17T08:44:46:739031+0000","default_fscid":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:44:43.210144+0000","modified":"2024-07-17T08:44:46.246155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":13956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13956":{"gid":13956,"name":"a","rank":0,"incarnation":196,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6835/2592823249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":2592823249},{"type":"v1","addr":"172.21.15.44:6835","nonce":2592823249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[48],"metadata_pool":47,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":13956,"qdb_cluster":[13956]},"id":23}]} 2024-07-17T08:45:07.507 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 200 2024-07-17T08:45:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 msgr2=0x7f2920079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:07.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 0x7f2920079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 msgr2=0x7f29440cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7f293c00c970 tx=0x7f293c00ce30 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 shutdown_connections 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f2920076e80 0x7f2920079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29440ce5c0 0x7f29440cc030 unknown :-1 s=CLOSED pgs=3259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29440b9060 0x7f29440cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.507+0000 7f2949800640 1 --2- 172.21.15.44:0/192610643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29440b8c80 0x7f294414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.508+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 >> 172.21.15.44:0/192610643 conn(0x7f29440cfec0 msgr2=0x7f29440b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.508+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 shutdown_connections 2024-07-17T08:45:07.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.508+0000 7f2949800640 1 -- 172.21.15.44:0/192610643 wait complete. 2024-07-17T08:45:07.528 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 -- 172.21.15.44:0/2159765294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400006ea50 msgr2=0x7f40000ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 --2- 172.21.15.44:0/2159765294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400006ea50 0x7f40000ac220 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009860 tx=0x7f3fe802a4b0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 -- 172.21.15.44:0/2159765294 shutdown_connections 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 --2- 172.21.15.44:0/2159765294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f400006ea50 0x7f40000ac220 unknown :-1 s=CLOSED pgs=3261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 -- 172.21.15.44:0/2159765294 >> 172.21.15.44:0/2159765294 conn(0x7f40000b0fb0 msgr2=0x7f40000b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 -- 172.21.15.44:0/2159765294 shutdown_connections 2024-07-17T08:45:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.711+0000 7f4005400640 1 -- 172.21.15.44:0/2159765294 wait complete. 2024-07-17T08:45:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.713+0000 7f4005400640 1 Processor -- start 2024-07-17T08:45:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.713+0000 7f4005400640 1 -- start start 2024-07-17T08:45:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400006ea50 0x7f4000141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400006eec0 0x7f40001f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f40000b3070 con 0x7f40000ae810 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40000b2ef0 con 0x7f400006eec0 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f4005400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40000aed30 con 0x7f400006ea50 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f3fff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f3fff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44472/0 (socket says 172.21.15.44:44472) 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f3fff200640 1 -- 172.21.15.44:0/1179577149 learned_addr learned my addr 172.21.15.44:0/1179577149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f3ff7e00640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400006eec0 0x7f40001f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.714+0000 7f3ffe800640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400006ea50 0x7f4000141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400006ea50 msgr2=0x7f4000141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400006ea50 0x7f4000141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400006eec0 msgr2=0x7f40001f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400006eec0 0x7f40001f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 -- 172.21.15.44:0/1179577149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001450e0 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3fff200640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c7e0 tx=0x7f3ff800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8017070 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.715+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ff8005070 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.716+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4000148e00 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.716+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80054c0 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.716+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4000144d00 con 0x7f40000ae810 2024-07-17T08:45:07.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.717+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40000aca90 con 0x7f40000ae810 2024-07-17T08:45:07.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.719+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f3ff801a070 con 0x7f40000ae810 2024-07-17T08:45:07.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.720+0000 7f3ff6a00640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 0x7f3fdc079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:07.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.720+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f3ff8098330 con 0x7f40000ae810 2024-07-17T08:45:07.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.720+0000 7f3ffe800640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 msgr2=0x7f3fdc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:07.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.720+0000 7f3ffe800640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 0x7f3fdc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:07.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.724+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3ff8060f20 con 0x7f40000ae810 2024-07-17T08:45:07.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.921+0000 7f3ffe800640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 msgr2=0x7f3fdc079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:07.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.922+0000 7f3ffe800640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 0x7f3fdc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:07.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.934+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40000b23b0 con 0x7f40000ae810 2024-07-17T08:45:07.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.937+0000 7f3ff6a00640 1 -- 172.21.15.44:0/1179577149 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v352) ==== 74+0+18420 (secure 0 0 0) 0x7f3ff8069160 con 0x7f40000ae810 2024-07-17T08:45:07.940 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:07.940 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":352,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:44:57.990183+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":47,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:44:41.136788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"350","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":48,"pool_name":"cephfs_data","create_time":"2024-07-17T08:44:42.144654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"351","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/2938461940":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/1993189342":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2478355158":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2460909214":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6837/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3314250175":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6834/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3191294602":"2024-07-18T08:44:57.990054+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 msgr2=0x7f3fdc079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 0x7f3fdc079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 msgr2=0x7f4000071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c7e0 tx=0x7f3ff800cca0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 shutdown_connections 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f3fdc076e80 0x7f3fdc079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40000ae810 0x7f4000071620 unknown :-1 s=CLOSED pgs=3262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400006eec0 0x7f40001f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 --2- 172.21.15.44:0/1179577149 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400006ea50 0x7f4000141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 >> 172.21.15.44:0/1179577149 conn(0x7f40000b0fb0 msgr2=0x7f40000b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 shutdown_connections 2024-07-17T08:45:07.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:07.940+0000 7f4005400640 1 -- 172.21.15.44:0/1179577149 wait complete. 2024-07-17T08:45:07.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 -- 172.21.15.44:0/2171613263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc06efc0 msgr2=0x7f06fc0ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 --2- 172.21.15.44:0/2171613263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc06efc0 0x7f06fc0ac6d0 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0d0260 tx=0x7f06f402aa30 comp rx=0 tx=0).stop 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 -- 172.21.15.44:0/2171613263 shutdown_connections 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 --2- 172.21.15.44:0/2171613263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc0b1af0 0x7f06fc0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 --2- 172.21.15.44:0/2171613263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc06efc0 0x7f06fc0ac6d0 unknown :-1 s=CLOSED pgs=3263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 --2- 172.21.15.44:0/2171613263 >> v2:172.21.15.175:3300/0 conn(0x7f06fc15a280 0x7f06fc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 -- 172.21.15.44:0/2171613263 >> 172.21.15.44:0/2171613263 conn(0x7f06fc0b0eb0 msgr2=0x7f06fc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:08.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 -- 172.21.15.44:0/2171613263 shutdown_connections 2024-07-17T08:45:08.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.154+0000 7f0700e00640 1 -- 172.21.15.44:0/2171613263 wait complete. 2024-07-17T08:45:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.157+0000 7f0700e00640 1 Processor -- start 2024-07-17T08:45:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.157+0000 7f0700e00640 1 -- start start 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc06efc0 0x7f06fc155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc0b1af0 0x7f06fc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc15ad90 con 0x7f06fc15a280 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc15ac10 con 0x7f06fc0b1af0 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f0700e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc15af10 con 0x7f06fc06efc0 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f06fa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f06fa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44488/0 (socket says 172.21.15.44:44488) 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f06fa800640 1 -- 172.21.15.44:0/4210064148 learned_addr learned my addr 172.21.15.44:0/4210064148 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f06f3e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc0b1af0 0x7f06fc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.159+0000 7f06f9e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc06efc0 0x7f06fc155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 -- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc06efc0 msgr2=0x7f06fc155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc06efc0 0x7f06fc155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 -- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc0b1af0 msgr2=0x7f06fc1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc0b1af0 0x7f06fc1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 -- 172.21.15.44:0/4210064148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc154270 con 0x7f06fc15a280 2024-07-17T08:45:08.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06fa800640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7f06f402b020 tx=0x7f06f4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4047070 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06f4004250 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4033050 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.160+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc154550 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.161+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc155570 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.162+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc0adab0 con 0x7f06fc15a280 2024-07-17T08:45:08.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.163+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 216) ==== 99090+0+0 (secure 0 0 0) 0x7f06f404a070 con 0x7f06fc15a280 2024-07-17T08:45:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.164+0000 7f06f2000640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 0x7f06d8079340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.165+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(352..352 src has 1..352) ==== 7077+0+0 (secure 0 0 0) 0x7f06f40b9f90 con 0x7f06fc15a280 2024-07-17T08:45:08.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.165+0000 7f06f9e00640 1 -- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 msgr2=0x7f06d8079340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/2989936540 2024-07-17T08:45:08.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.165+0000 7f06f9e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 0x7f06d8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:08.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.168+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f06fc0adab0 con 0x7f06fc15a280 2024-07-17T08:45:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.359+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 217) ==== 98540+0+0 (secure 0 0 0) 0x7f06f4036070 con 0x7f06fc15a280 2024-07-17T08:45:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.360+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 msgr2=0x7f06d8079340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.360+0000 7f06f2000640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 0x7f06d8079340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.473+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f06fc0b21b0 con 0x7f06fc15a280 2024-07-17T08:45:08.475 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:45:08.473+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:45:08.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.475+0000 7f06f2000640 1 -- 172.21.15.44:0/4210064148 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v201) ==== 161+0+0 (secure 0 0 0) 0x7f06f40b46a0 con 0x7f06fc15a280 2024-07-17T08:45:08.477 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72fd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72fd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72fd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72fd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72fd200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72ffa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72ffa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72ffa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.478 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72ffa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.479 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:08.477+0000 7f72ffa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 msgr2=0x7f06fc1f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7f06f402b020 tx=0x7f06f4003f10 comp rx=0 tx=0).stop 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 shutdown_connections 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:6834/2989936540,v1:172.21.15.175:6837/2989936540] conn(0x7f06d8076e80 0x7f06d8079340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc15a280 0x7f06fc1f7530 unknown :-1 s=CLOSED pgs=3264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc0b1af0 0x7f06fc1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 --2- 172.21.15.44:0/4210064148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc06efc0 0x7f06fc155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 >> 172.21.15.44:0/4210064148 conn(0x7f06fc0b0eb0 msgr2=0x7f06fc0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 shutdown_connections 2024-07-17T08:45:08.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.479+0000 7f0700e00640 1 -- 172.21.15.44:0/4210064148 wait complete. 2024-07-17T08:45:08.511 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:45:08.565 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 -- 172.21.15.44:0/2969321940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f247c003a00 msgr2=0x7f247c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 --2- 172.21.15.44:0/2969321940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f247c003a00 0x7f247c003e00 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7f248800c310 tx=0x7f247c005830 comp rx=0 tx=0).stop 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 -- 172.21.15.44:0/2969321940 shutdown_connections 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 --2- 172.21.15.44:0/2969321940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f247c003a00 0x7f247c003e00 unknown :-1 s=CLOSED pgs=3269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 -- 172.21.15.44:0/2969321940 >> 172.21.15.44:0/2969321940 conn(0x7f24980b0eb0 msgr2=0x7f24980b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 -- 172.21.15.44:0/2969321940 shutdown_connections 2024-07-17T08:45:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.691+0000 7f249d600640 1 -- 172.21.15.44:0/2969321940 wait complete. 2024-07-17T08:45:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.693+0000 7f249d600640 1 Processor -- start 2024-07-17T08:45:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.693+0000 7f249d600640 1 -- start start 2024-07-17T08:45:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f247c003a00 0x7f24981f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24981f7220 0x7f24981f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24980b21b0 con 0x7f2498140ae0 2024-07-17T08:45:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f249806f1d0 con 0x7f247c003a00 2024-07-17T08:45:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f249d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f24980b2330 con 0x7f24981f7220 2024-07-17T08:45:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f248e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f248e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44524/0 (socket says 172.21.15.44:44524) 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f248e000640 1 -- 172.21.15.44:0/779195501 learned_addr learned my addr 172.21.15.44:0/779195501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f2496a00640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f247c003a00 0x7f24981f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.694+0000 7f2497400640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24981f7220 0x7f24981f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 -- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24981f7220 msgr2=0x7f24981f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24981f7220 0x7f24981f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 -- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f247c003a00 msgr2=0x7f24981f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f247c003a00 0x7f24981f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 -- 172.21.15.44:0/779195501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249815c5a0 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248e000640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 crypto rx=0x7f248000d2d0 tx=0x7f248000d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480014020 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2480010970 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480010dc0 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.695+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249815c880 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.696+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24980c88b0 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.697+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2498155f30 con 0x7f2498140ae0 2024-07-17T08:45:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.698+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 217) ==== 98540+0+0 (secure 0 0 0) 0x7f2480017020 con 0x7f2498140ae0 2024-07-17T08:45:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.699+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(354..354 src has 1..354) ==== 7636+0+0 (secure 0 0 0) 0x7f2480098a40 con 0x7f2498140ae0 2024-07-17T08:45:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.703+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2498155f30 con 0x7f2498140ae0 2024-07-17T08:45:08.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:08.956+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f24980ac190 con 0x7f2498140ae0 2024-07-17T08:45:09.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.363+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 218) ==== 98667+0+0 (secure 0 0 0) 0x7f248005cd50 con 0x7f2498140ae0 2024-07-17T08:45:09.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.364+0000 7f248f400640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 0x7f247407f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.372+0000 7f2496a00640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 0x7f247407f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:09.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.382+0000 7f2496a00640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 0x7f247407f710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2488033000 tx=0x7f2488001aa0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:09.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.479+0000 7f248f400640 1 -- 172.21.15.44:0/779195501 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v202) ==== 104+0+0 (secure 0 0 0) 0x7f248001e020 con 0x7f2498140ae0 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 msgr2=0x7f247407f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 0x7f247407f710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2488033000 tx=0x7f2488001aa0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 msgr2=0x7f24981f33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 crypto rx=0x7f248000d2d0 tx=0x7f248000d790 comp rx=0 tx=0).stop 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 shutdown_connections 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f247407d320 0x7f247407f710 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24981f7220 0x7f24981f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2498140ae0 0x7f24981f33a0 unknown :-1 s=CLOSED pgs=3270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 --2- 172.21.15.44:0/779195501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f247c003a00 0x7f24981f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.484+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 >> 172.21.15.44:0/779195501 conn(0x7f24980b0eb0 msgr2=0x7f24980cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.485+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 shutdown_connections 2024-07-17T08:45:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.485+0000 7f249d600640 1 -- 172.21.15.44:0/779195501 wait complete. 2024-07-17T08:45:09.510 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.711+0000 7f7584c00640 1 -- 172.21.15.44:0/3419565330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 msgr2=0x7f75800729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.711+0000 7f7584c00640 1 --2- 172.21.15.44:0/3419565330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f75800729b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7578005fd0 tx=0x7f757802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 -- 172.21.15.44:0/3419565330 shutdown_connections 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 --2- 172.21.15.44:0/3419565330 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f75800729b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 --2- 172.21.15.44:0/3419565330 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 0x7f75800ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 -- 172.21.15.44:0/3419565330 >> 172.21.15.44:0/3419565330 conn(0x7f75800cf050 msgr2=0x7f75800cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 -- 172.21.15.44:0/3419565330 shutdown_connections 2024-07-17T08:45:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.712+0000 7f7584c00640 1 -- 172.21.15.44:0/3419565330 wait complete. 2024-07-17T08:45:09.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.714+0000 7f7584c00640 1 Processor -- start 2024-07-17T08:45:09.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.714+0000 7f7584c00640 1 -- start start 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f7580153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 0x7f758006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75800733f0 con 0x7f75800ad790 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7580073270 con 0x7f75800ae490 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.715+0000 7f7584c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7580073570 con 0x7f7580155860 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51112/0 (socket says 172.21.15.44:51112) 2024-07-17T08:45:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 -- 172.21.15.44:0/3452155748 learned_addr learned my addr 172.21.15.44:0/3452155748 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757ea00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f7580153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757fe00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 0x7f758006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 -- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 msgr2=0x7f758006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 0x7f758006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 -- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 msgr2=0x7f7580153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f7580153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.716+0000 7f757f400640 1 -- 172.21.15.44:0/3452155748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580154540 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f757f400640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f757000e790 tx=0x7f757000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757000eea0 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f757000cdb0 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570010690 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758015b2f0 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.717+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801f78c0 con 0x7f75800ad790 2024-07-17T08:45:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.718+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75800739d0 con 0x7f75800ad790 2024-07-17T08:45:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.720+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 218) ==== 98667+0+0 (secure 0 0 0) 0x7f7570010830 con 0x7f75800ad790 2024-07-17T08:45:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.720+0000 7f7577400640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 0x7f7560078ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.721+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(354..354 src has 1..354) ==== 7636+0+0 (secure 0 0 0) 0x7f75700990e0 con 0x7f75800ad790 2024-07-17T08:45:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.721+0000 7f757ea00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 0x7f7560078ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.722+0000 7f757ea00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 0x7f7560078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75800b0c10 tx=0x7f756c009040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:09.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.725+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7570062090 con 0x7f75800ad790 2024-07-17T08:45:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:09.936+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75800ae020 con 0x7f75800ad790 2024-07-17T08:45:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.494+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 219) ==== 98773+0+0 (secure 0 0 0) 0x7f757005dc20 con 0x7f75800ad790 2024-07-17T08:45:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.503+0000 7f7577400640 1 -- 172.21.15.44:0/3452155748 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v355) ==== 181+0+0 (secure 0 0 0) 0x7f7570082f80 con 0x7f75800ad790 2024-07-17T08:45:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:45:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.506+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 msgr2=0x7f7560078ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.506+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 0x7f7560078ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75800b0c10 tx=0x7f756c009040 comp rx=0 tx=0).stop 2024-07-17T08:45:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.506+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 msgr2=0x7f758009cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.506+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f757000e790 tx=0x7f757000ec50 comp rx=0 tx=0).stop 2024-07-17T08:45:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 shutdown_connections 2024-07-17T08:45:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f7560076a00 0x7f7560078ec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7580155860 0x7f758006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ae490 0x7f7580153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 --2- 172.21.15.44:0/3452155748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75800ad790 0x7f758009cd30 unknown :-1 s=CLOSED pgs=3271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 >> 172.21.15.44:0/3452155748 conn(0x7f75800cf050 msgr2=0x7f75800b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 shutdown_connections 2024-07-17T08:45:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.508+0000 7f7584c00640 1 -- 172.21.15.44:0/3452155748 wait complete. 2024-07-17T08:45:10.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:45:10.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 -- 172.21.15.44:0/2994737533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 msgr2=0x7fbb900b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:10.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 --2- 172.21.15.44:0/2994737533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb900b2620 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005fd0 tx=0x7fbb8802a8b0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 -- 172.21.15.44:0/2994737533 shutdown_connections 2024-07-17T08:45:10.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 --2- 172.21.15.44:0/2994737533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb900b2620 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 --2- 172.21.15.44:0/2994737533 >> v2:172.21.15.175:3300/0 conn(0x7fbb9006f6d0 0x7fbb901539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 -- 172.21.15.44:0/2994737533 >> 172.21.15.44:0/2994737533 conn(0x7fbb900b0eb0 msgr2=0x7fbb900b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:10.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 -- 172.21.15.44:0/2994737533 shutdown_connections 2024-07-17T08:45:10.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.728+0000 7fbb96200640 1 -- 172.21.15.44:0/2994737533 wait complete. 2024-07-17T08:45:10.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.730+0000 7fbb96200640 1 Processor -- start 2024-07-17T08:45:10.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.730+0000 7fbb96200640 1 -- start start 2024-07-17T08:45:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb901f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb9015a5d0 0x7fbb90141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900b30e0 con 0x7fbb9006f6d0 2024-07-17T08:45:10.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900b2f60 con 0x7fbb9015a5d0 2024-07-17T08:45:10.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.731+0000 7fbb96200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900b3260 con 0x7fbb900b21b0 2024-07-17T08:45:10.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:10.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51122/0 (socket says 172.21.15.44:51122) 2024-07-17T08:45:10.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 -- 172.21.15.44:0/382132099 learned_addr learned my addr 172.21.15.44:0/382132099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8fe00640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb9015a5d0 0x7fbb90141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb86a00640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb901f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 -- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 msgr2=0x7fbb901f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb901f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 -- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb9015a5d0 msgr2=0x7fbb90141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb9015a5d0 0x7fbb90141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 -- 172.21.15.44:0/382132099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb901f4c70 con 0x7fbb9006f6d0 2024-07-17T08:45:10.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.732+0000 7fbb8f400640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000ea20 tx=0x7fbb8000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.733+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000cd40 con 0x7fbb9006f6d0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.733+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb80016410 con 0x7fbb9006f6d0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.733+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb90158960 con 0x7fbb9006f6d0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.733+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80016860 con 0x7fbb9006f6d0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.733+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb90158f20 con 0x7fbb9006f6d0 2024-07-17T08:45:10.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.734+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb901539f0 con 0x7fbb9006f6d0 2024-07-17T08:45:10.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.735+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 219) ==== 98773+0+0 (secure 0 0 0) 0x7fbb80016a00 con 0x7fbb9006f6d0 2024-07-17T08:45:10.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.736+0000 7fbb87e00640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 0x7fbb6c078fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:10.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.736+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(355..355 src has 1..355) ==== 7198+0+0 (secure 0 0 0) 0x7fbb80098d10 con 0x7fbb9006f6d0 2024-07-17T08:45:10.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.737+0000 7fbb86a00640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 0x7fbb6c078fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:10.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.738+0000 7fbb86a00640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 0x7fbb6c078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007fd0 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:10.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.741+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbb8009f080 con 0x7fbb9006f6d0 2024-07-17T08:45:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:10.950+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb900af2d0 con 0x7fbb9006f6d0 2024-07-17T08:45:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.506+0000 7fbb87e00640 1 -- 172.21.15.44:0/382132099 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v356) ==== 169+0+0 (secure 0 0 0) 0x7fbb80061c50 con 0x7fbb9006f6d0 2024-07-17T08:45:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:45:11.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.510+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 msgr2=0x7fbb6c078fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.510+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 0x7fbb6c078fe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb78007fd0 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.510+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 msgr2=0x7fbb90073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.510+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000ea20 tx=0x7fbb8000eee0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 shutdown_connections 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fbb6c076b20 0x7fbb6c078fe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb9015a5d0 0x7fbb90141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb900b21b0 0x7fbb901f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 --2- 172.21.15.44:0/382132099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb9006f6d0 0x7fbb90073280 unknown :-1 s=CLOSED pgs=3273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 >> 172.21.15.44:0/382132099 conn(0x7fbb900b0eb0 msgr2=0x7fbb900739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.511+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 shutdown_connections 2024-07-17T08:45:11.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.514+0000 7fbb96200640 1 -- 172.21.15.44:0/382132099 wait complete. 2024-07-17T08:45:11.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:45:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.744+0000 7f3e4be00640 1 -- 172.21.15.44:0/360324796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e30004d90 msgr2=0x7f3e30005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.744+0000 7f3e4be00640 1 --2- 172.21.15.44:0/360324796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e30004d90 0x7f3e30005210 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009ed0 tx=0x7f3e30005830 comp rx=0 tx=0).stop 2024-07-17T08:45:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.744+0000 7f3e4b400640 1 -- 172.21.15.44:0/360324796 reap_dead start 2024-07-17T08:45:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.745+0000 7f3e4be00640 1 -- 172.21.15.44:0/360324796 shutdown_connections 2024-07-17T08:45:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.745+0000 7f3e4be00640 1 -- 172.21.15.44:0/360324796 >> 172.21.15.44:0/360324796 conn(0x7f3e4c0cf0d0 msgr2=0x7f3e4c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.745+0000 7f3e4be00640 1 -- 172.21.15.44:0/360324796 shutdown_connections 2024-07-17T08:45:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.745+0000 7f3e4be00640 1 -- 172.21.15.44:0/360324796 wait complete. 2024-07-17T08:45:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.747+0000 7f3e4be00640 1 Processor -- start 2024-07-17T08:45:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.748+0000 7f3e4be00640 1 -- start start 2024-07-17T08:45:11.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e4c074160 0x7f3e4c1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:11.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:11.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e4c072090 0x7f3e4c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:11.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c0ac7f0 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c0ac670 con 0x7f3e4c072090 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4c0b20a0 con 0x7f3e4c074160 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51148/0 (socket says 172.21.15.44:51148) 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 -- 172.21.15.44:0/2598561214 learned_addr learned my addr 172.21.15.44:0/2598561214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4aa00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e4c074160 0x7f3e4c1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 -- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e4c074160 msgr2=0x7f3e4c1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e4c074160 0x7f3e4c1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 -- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e4c072090 msgr2=0x7f3e4c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e4c072090 0x7f3e4c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.749+0000 7f3e4a000640 1 -- 172.21.15.44:0/2598561214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1f44b0 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e4a000640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006690 tx=0x7f3e380016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38015020 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e38011840 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c1f4790 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38011c90 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.750+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c071630 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.751+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c0b2600 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.752+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 219) ==== 98773+0+0 (secure 0 0 0) 0x7f3e38018020 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.753+0000 7f3e42a00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 0x7f3e2c079010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:11.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.753+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7f3e38099080 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.754+0000 7f3e4aa00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 0x7f3e2c079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:11.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.755+0000 7f3e4aa00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 0x7f3e2c079010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c003670 tx=0x7f3e3c001c10 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:11.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.757+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f3e3801f020 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.969+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f3e4c152510 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.976+0000 7f3e42a00640 1 -- 172.21.15.44:0/2598561214 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v107) ==== 71+0+0 (secure 0 0 0) 0x7f3e38061b30 con 0x7f3e4c1f3cd0 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 msgr2=0x7f3e2c079010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 0x7f3e2c079010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c003670 tx=0x7f3e3c001c10 comp rx=0 tx=0).stop 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 msgr2=0x7f3e4c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006690 tx=0x7f3e380016f0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 shutdown_connections 2024-07-17T08:45:11.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3e2c076b50 0x7f3e2c079010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e4c072090 0x7f3e4c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e4c1f3cd0 0x7f3e4c071b50 unknown :-1 s=CLOSED pgs=3276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 --2- 172.21.15.44:0/2598561214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e4c074160 0x7f3e4c1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 >> 172.21.15.44:0/2598561214 conn(0x7f3e4c0cf0d0 msgr2=0x7f3e4c152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 shutdown_connections 2024-07-17T08:45:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:11.979+0000 7f3e4be00640 1 -- 172.21.15.44:0/2598561214 wait complete. 2024-07-17T08:45:12.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06' 2024-07-17T08:45:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 -- 172.21.15.44:0/2698154425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 msgr2=0x7f03bc06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:12.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2698154425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc06d9b0 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009900 tx=0x7f03a802a560 comp rx=0 tx=0).stop 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 -- 172.21.15.44:0/2698154425 shutdown_connections 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2698154425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc06d9b0 unknown :-1 s=CLOSED pgs=3277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2698154425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b8c80 0x7f03bc0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.209+0000 7f03c2e00640 1 -- 172.21.15.44:0/2698154425 >> 172.21.15.44:0/2698154425 conn(0x7f03bc0cfec0 msgr2=0x7f03bc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.210+0000 7f03c2e00640 1 -- 172.21.15.44:0/2698154425 shutdown_connections 2024-07-17T08:45:12.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.210+0000 7f03c2e00640 1 -- 172.21.15.44:0/2698154425 wait complete. 2024-07-17T08:45:12.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.212+0000 7f03c2e00640 1 Processor -- start 2024-07-17T08:45:12.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.212+0000 7f03c2e00640 1 -- start start 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 0x7f03bc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 0x7f03bc1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc06e780 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc06e600 con 0x7f03bc0b8c80 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03bc06e900 con 0x7f03bc0b5400 2024-07-17T08:45:12.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.213+0000 7f03c2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51156/0 (socket says 172.21.15.44:51156) 2024-07-17T08:45:12.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 -- 172.21.15.44:0/2839877630 learned_addr learned my addr 172.21.15.44:0/2839877630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c1a00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 0x7f03bc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c1000640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 0x7f03bc1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 -- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 msgr2=0x7f03bc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 0x7f03bc0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 -- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 msgr2=0x7f03bc1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:12.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 0x7f03bc1578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 -- 172.21.15.44:0/2839877630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc158170 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c1a00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 0x7f03bc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c1000640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 0x7f03bc1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.214+0000 7f03c2400640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c410 tx=0x7f03b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.215+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4011790 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.215+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03b4017070 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.215+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4011cc0 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.215+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc0cd020 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.215+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1583e0 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.216+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc15e520 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.217+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 219) ==== 98773+0+0 (secure 0 0 0) 0x7f03b401c460 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.218+0000 7f03b9600640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 0x7f039c079010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:12.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.218+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7f03b4098230 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.219+0000 7f03c1a00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 0x7f039c079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:12.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.220+0000 7f03c1a00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 0x7f039c079010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005c40 tx=0x7f03b00082f0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:12.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.223+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f03b4021020 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.522+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06"]} v 0) -- 0x7f03bc0b5d70 con 0x7f03bc0ce5c0 2024-07-17T08:45:12.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:45:12.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:45:12.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:12.996+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f03b405cf50 con 0x7f03bc0ce5c0 2024-07-17T08:45:13.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.513+0000 7f03b9600640 1 -- 172.21.15.44:0/2839877630 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_06"]}]=0 v3726) ==== 201+0+0 (secure 0 0 0) 0x7f03b40613c0 con 0x7f03bc0ce5c0 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 msgr2=0x7f039c079010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 0x7f039c079010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03b0005c40 tx=0x7f03b00082f0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 msgr2=0x7f03bc0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c410 tx=0x7f03b400c8d0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 shutdown_connections 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f039c076b50 0x7f039c079010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03bc0ce5c0 0x7f03bc0cc6b0 unknown :-1 s=CLOSED pgs=3278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03bc0b8c80 0x7f03bc1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 --2- 172.21.15.44:0/2839877630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0b5400 0x7f03bc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 >> 172.21.15.44:0/2839877630 conn(0x7f03bc0cfec0 msgr2=0x7f03bc0d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:13.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 shutdown_connections 2024-07-17T08:45:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.517+0000 7f03c2e00640 1 -- 172.21.15.44:0/2839877630 wait complete. 2024-07-17T08:45:13.543 INFO:tasks.cephfs_test_runner:Test subvol schedule listing fails without a subvolgroup argument. ... ok 2024-07-17T08:45:13.543 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_07 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule listing fails without a subvol argument. 2024-07-17T08:45:13.543 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_07 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:45:13.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:13.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 -- 172.21.15.44:0/977130152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3048003aa0 msgr2=0x7f3048003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 --2- 172.21.15.44:0/977130152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3048003aa0 0x7f3048003ec0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 crypto rx=0x7f305400c4a0 tx=0x7f3048005830 comp rx=0 tx=0).stop 2024-07-17T08:45:13.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 -- 172.21.15.44:0/977130152 shutdown_connections 2024-07-17T08:45:13.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 --2- 172.21.15.44:0/977130152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3048003aa0 0x7f3048003ec0 unknown :-1 s=CLOSED pgs=3280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 -- 172.21.15.44:0/977130152 >> 172.21.15.44:0/977130152 conn(0x7f30640b0fb0 msgr2=0x7f30640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:13.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 -- 172.21.15.44:0/977130152 shutdown_connections 2024-07-17T08:45:13.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.761+0000 7f306ae00640 1 -- 172.21.15.44:0/977130152 wait complete. 2024-07-17T08:45:13.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.763+0000 7f306ae00640 1 Processor -- start 2024-07-17T08:45:13.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.763+0000 7f306ae00640 1 -- start start 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3048003aa0 0x7f3064142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 0x7f30641f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30640ae910 con 0x7f306413fe30 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3064154040 con 0x7f30641f5080 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f306ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f306406f710 con 0x7f3048003aa0 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f3063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.764+0000 7f3063400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51166/0 (socket says 172.21.15.44:51166) 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 -- 172.21.15.44:0/1754990534 learned_addr learned my addr 172.21.15.44:0/1754990534 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:13.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063e00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3048003aa0 0x7f3064142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3068e00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 0x7f30641f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 -- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3048003aa0 msgr2=0x7f3064142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3048003aa0 0x7f3064142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 -- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 msgr2=0x7f30641f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 0x7f30641f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 -- 172.21.15.44:0/1754990534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3064158960 con 0x7f306413fe30 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3068e00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 0x7f30641f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:13.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.765+0000 7f3063400640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f304c004eb0 tx=0x7f304c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.766+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c015020 con 0x7f306413fe30 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.766+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f304c011960 con 0x7f306413fe30 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.766+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c011d90 con 0x7f306413fe30 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.766+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3064158c40 con 0x7f306413fe30 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.766+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641424d0 con 0x7f306413fe30 2024-07-17T08:45:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.767+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3064159b50 con 0x7f306413fe30 2024-07-17T08:45:13.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.768+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f304c018020 con 0x7f306413fe30 2024-07-17T08:45:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.769+0000 7f305be00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 0x7f3040079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.769+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7f304c099160 con 0x7f306413fe30 2024-07-17T08:45:13.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.770+0000 7f3063e00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 0x7f3040079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:13.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.771+0000 7f3063e00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 0x7f3040079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054001b00 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:13.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.774+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f304c01f020 con 0x7f306413fe30 2024-07-17T08:45:13.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.983+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f306415ce10 con 0x7f306413fe30 2024-07-17T08:45:13.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.986+0000 7f305be00640 1 -- 172.21.15.44:0/1754990534 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v356) ==== 74+0+15466 (secure 0 0 0) 0x7f304c062180 con 0x7f306413fe30 2024-07-17T08:45:13.988 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:13.988 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":356,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:45:11.504202+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":48,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":349,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6835/2592823249":"2024-07-18T08:45:08.475116+0000","172.21.15.175:0/3200676085":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/3992536299":"2024-07-18T08:45:08.341142+0000","172.21.15.44:6833/2592823249":"2024-07-18T08:45:08.475116+0000","172.21.15.175:0/479270347":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/2156582823":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/1469519005":"2024-07-18T08:45:08.341142+0000","172.21.15.175:6834/2989936540":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/2556679271":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/2024341270":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/2391398102":"2024-07-18T08:45:08.341142+0000","172.21.15.175:6837/2989936540":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/3088750347":"2024-07-18T08:45:08.341142+0000","172.21.15.175:0/2938461940":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/1993189342":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2478355158":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/2460909214":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6837/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3314250175":"2024-07-18T08:44:57.990054+0000","172.21.15.175:6834/3157277076":"2024-07-18T08:44:57.990054+0000","172.21.15.175:0/3191294602":"2024-07-18T08:44:57.990054+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 msgr2=0x7f3040079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 0x7f3040079070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054001b00 comp rx=0 tx=0).stop 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 msgr2=0x7f3064098a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f304c004eb0 tx=0x7f304c001630 comp rx=0 tx=0).stop 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 shutdown_connections 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f3040076bb0 0x7f3040079070 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30641f5080 0x7f30641f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f306413fe30 0x7f3064098a70 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 --2- 172.21.15.44:0/1754990534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3048003aa0 0x7f3064142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 >> 172.21.15.44:0/1754990534 conn(0x7f30640b0fb0 msgr2=0x7f306415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 shutdown_connections 2024-07-17T08:45:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:13.989+0000 7f306ae00640 1 -- 172.21.15.44:0/1754990534 wait complete. 2024-07-17T08:45:14.012 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:45:14.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:45:14.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 -- 172.21.15.44:0/3784461181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa12006db60 msgr2=0x7fa1200ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 --2- 172.21.15.44:0/3784461181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa12006db60 0x7fa1200ac190 secure :-1 s=READY pgs=3282 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005fd0 tx=0x7fa11c02aa50 comp rx=0 tx=0).stop 2024-07-17T08:45:14.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 -- 172.21.15.44:0/3784461181 shutdown_connections 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 --2- 172.21.15.44:0/3784461181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120161120 0x7fa12006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 --2- 172.21.15.44:0/3784461181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa12006db60 0x7fa1200ac190 unknown :-1 s=CLOSED pgs=3282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 --2- 172.21.15.44:0/3784461181 >> v2:172.21.15.175:3300/0 conn(0x7fa120154600 0x7fa1200b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 -- 172.21.15.44:0/3784461181 >> 172.21.15.44:0/3784461181 conn(0x7fa1200b0eb0 msgr2=0x7fa1200b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 -- 172.21.15.44:0/3784461181 shutdown_connections 2024-07-17T08:45:14.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.228+0000 7fa128800640 1 -- 172.21.15.44:0/3784461181 wait complete. 2024-07-17T08:45:14.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.230+0000 7fa128800640 1 Processor -- start 2024-07-17T08:45:14.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.230+0000 7fa128800640 1 -- start start 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa120154600 0x7fa12015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 0x7fa120159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200aca60 con 0x7fa120161120 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200ac8e0 con 0x7fa120154600 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa128800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa12015e0a0 con 0x7fa120073900 2024-07-17T08:45:14.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.231+0000 7fa125400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51188/0 (socket says 172.21.15.44:51188) 2024-07-17T08:45:14.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 -- 172.21.15.44:0/1535158116 learned_addr learned my addr 172.21.15.44:0/1535158116 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa126800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 0x7fa120159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125e00640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa120154600 0x7fa12015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 -- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 msgr2=0x7fa120159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 0x7fa120159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 -- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa120154600 msgr2=0x7fa12015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa120154600 0x7fa12015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 -- 172.21.15.44:0/1535158116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1201f4100 con 0x7fa120161120 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa126800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 0x7fa120159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:14.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.232+0000 7fa125400640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b410 tx=0x7fa11000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.233+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110015020 con 0x7fa120161120 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.233+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa110011940 con 0x7fa120161120 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.233+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1201f43a0 con 0x7fa120161120 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.233+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110011d90 con 0x7fa120161120 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.233+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201f4c40 con 0x7fa120161120 2024-07-17T08:45:14.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.234+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1200b3ce0 con 0x7fa120161120 2024-07-17T08:45:14.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.235+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fa110018020 con 0x7fa120161120 2024-07-17T08:45:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.237+0000 7fa11a000640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 0x7fa104079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.237+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7fa110098740 con 0x7fa120161120 2024-07-17T08:45:14.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.237+0000 7fa125e00640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 0x7fa104079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.239+0000 7fa125e00640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 0x7fa104079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114005850 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:14.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.240+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa11001f020 con 0x7fa120161120 2024-07-17T08:45:14.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.452+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa120160e90 con 0x7fa120161120 2024-07-17T08:45:14.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.453+0000 7fa11a000640 1 -- 172.21.15.44:0/1535158116 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v356) ==== 93+0+10 (secure 0 0 0) 0x7fa110061760 con 0x7fa120161120 2024-07-17T08:45:14.455 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:45:14.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 msgr2=0x7fa104079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 0x7fa104079070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114005850 comp rx=0 tx=0).stop 2024-07-17T08:45:14.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 msgr2=0x7fa1200733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b410 tx=0x7fa11000b8d0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 shutdown_connections 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa104076bb0 0x7fa104079070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa120073900 0x7fa120159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa120161120 0x7fa1200733c0 unknown :-1 s=CLOSED pgs=3283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 --2- 172.21.15.44:0/1535158116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa120154600 0x7fa12015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.456+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 >> 172.21.15.44:0/1535158116 conn(0x7fa1200b0eb0 msgr2=0x7fa1201605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.457+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 shutdown_connections 2024-07-17T08:45:14.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.457+0000 7fa128800640 1 -- 172.21.15.44:0/1535158116 wait complete. 2024-07-17T08:45:14.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07' 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.692+0000 7facb7e00640 1 -- 172.21.15.44:0/3292044993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 msgr2=0x7facb00ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.692+0000 7facb7e00640 1 --2- 172.21.15.44:0/3292044993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb00ac3e0 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7facb0070a70 tx=0x7fac9c02ade0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 -- 172.21.15.44:0/3292044993 shutdown_connections 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 --2- 172.21.15.44:0/3292044993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb00ac3e0 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 --2- 172.21.15.44:0/3292044993 >> v2:172.21.15.175:3300/0 conn(0x7facb0153a50 0x7facb0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 -- 172.21.15.44:0/3292044993 >> 172.21.15.44:0/3292044993 conn(0x7facb00b0fb0 msgr2=0x7facb00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 -- 172.21.15.44:0/3292044993 shutdown_connections 2024-07-17T08:45:14.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.693+0000 7facb7e00640 1 -- 172.21.15.44:0/3292044993 wait complete. 2024-07-17T08:45:14.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.695+0000 7facb7e00640 1 Processor -- start 2024-07-17T08:45:14.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.695+0000 7facb7e00640 1 -- start start 2024-07-17T08:45:14.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 0x7facb01ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 0x7facb01f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7facb0153e30 con 0x7facb015a040 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facb006f1b0 con 0x7facb0153a50 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facb0153fb0 con 0x7facb00ac920 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51206/0 (socket says 172.21.15.44:51206) 2024-07-17T08:45:14.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb5e00640 1 -- 172.21.15.44:0/654235970 learned_addr learned my addr 172.21.15.44:0/654235970 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb4a00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 0x7facb01f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.696+0000 7facb5400640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 0x7facb01ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 -- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 msgr2=0x7facb01ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 0x7facb01ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 -- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 msgr2=0x7facb01f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 0x7facb01f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 -- 172.21.15.44:0/654235970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb0151d50 con 0x7facb015a040 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb4a00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 0x7facb01f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5400640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 0x7facb01ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:14.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb5e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7faca800c7b0 tx=0x7faca800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800ce60 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faca8004c00 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800a850 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb0152030 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.697+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb01525f0 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.699+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb00afd80 con 0x7facb015a040 2024-07-17T08:45:14.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.700+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7faca800a9f0 con 0x7facb015a040 2024-07-17T08:45:14.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.701+0000 7facad600640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 0x7fac90079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:14.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.701+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7faca8098a90 con 0x7facb015a040 2024-07-17T08:45:14.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.702+0000 7facb5400640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 0x7fac90079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:14.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.703+0000 7facb5400640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 0x7fac90079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faca00058c0 tx=0x7faca0005d30 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:14.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.706+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7facb00afd80 con 0x7facb015a040 2024-07-17T08:45:14.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:14.997+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07"]} v 0) -- 0x7facb0159b00 con 0x7facb015a040 2024-07-17T08:45:15.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.527+0000 7facad600640 1 -- 172.21.15.44:0/654235970 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07"]}]=0 v3728) ==== 204+0+0 (secure 0 0 0) 0x7faca8020070 con 0x7facb015a040 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 msgr2=0x7fac90079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 0x7fac90079070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faca00058c0 tx=0x7faca0005d30 comp rx=0 tx=0).stop 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 msgr2=0x7facb01f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7faca800c7b0 tx=0x7faca800cc70 comp rx=0 tx=0).stop 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 shutdown_connections 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fac90076bb0 0x7fac90079070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facb015a040 0x7facb01f2a30 unknown :-1 s=CLOSED pgs=3285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facb0153a50 0x7facb01f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 --2- 172.21.15.44:0/654235970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facb00ac920 0x7facb01ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 >> 172.21.15.44:0/654235970 conn(0x7facb00b0fb0 msgr2=0x7facb01576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 shutdown_connections 2024-07-17T08:45:15.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.531+0000 7facb7e00640 1 -- 172.21.15.44:0/654235970 wait complete. 2024-07-17T08:45:15.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.778+0000 7fd4ff200640 1 -- 172.21.15.44:0/3240064177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 msgr2=0x7fd5000b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.778+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3240064177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd5000b3b80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0005fd0 tx=0x7fd4f002b910 comp rx=0 tx=0).stop 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 -- 172.21.15.44:0/3240064177 shutdown_connections 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3240064177 >> v2:172.21.15.175:3300/0 conn(0x7fd5000adb70 0x7fd50014fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3240064177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd5000b3b80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 -- 172.21.15.44:0/3240064177 >> 172.21.15.44:0/3240064177 conn(0x7fd50006c120 msgr2=0x7fd50006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 -- 172.21.15.44:0/3240064177 shutdown_connections 2024-07-17T08:45:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.779+0000 7fd4ff200640 1 -- 172.21.15.44:0/3240064177 wait complete. 2024-07-17T08:45:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.780+0000 7fd4ff200640 1 Processor -- start 2024-07-17T08:45:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.781+0000 7fd4ff200640 1 -- start start 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd500148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd50015d6d0 0x7fd50015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5001500c0 con 0x7fd5000adb70 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd50015dd00 con 0x7fd50015d6d0 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4ff200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd500150240 con 0x7fd50015d2f0 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51216/0 (socket says 172.21.15.44:51216) 2024-07-17T08:45:15.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 -- 172.21.15.44:0/3485753838 learned_addr learned my addr 172.21.15.44:0/3485753838 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4f5400640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd500148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fe800640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd50015d6d0 0x7fd50015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 -- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 msgr2=0x7fd500148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd500148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 -- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd50015d6d0 msgr2=0x7fd50015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd50015d6d0 0x7fd50015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.782+0000 7fd4fde00640 1 -- 172.21.15.44:0/3485753838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd500156240 con 0x7fd5000adb70 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4fde00640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004da0 con 0x7fd5000adb70 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4f0031700 con 0x7fd5000adb70 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0031b50 con 0x7fd5000adb70 2024-07-17T08:45:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd500156520 con 0x7fd5000adb70 2024-07-17T08:45:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.783+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd500156ae0 con 0x7fd5000adb70 2024-07-17T08:45:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.785+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd500153ce0 con 0x7fd5000adb70 2024-07-17T08:45:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.786+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fd4f0038070 con 0x7fd5000adb70 2024-07-17T08:45:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.787+0000 7fd4f6a00640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 0x7fd4dc0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:15.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.787+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7fd4f00b9850 con 0x7fd5000adb70 2024-07-17T08:45:15.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.788+0000 7fd4f5400640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 0x7fd4dc0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:15.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.789+0000 7fd4f5400640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 0x7fd4dc0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5000c8a10 tx=0x7fd4e8009040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:15.791+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd500153ce0 con 0x7fd5000adb70 2024-07-17T08:45:16.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.007+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd5001506f0 con 0x7fd5000adb70 2024-07-17T08:45:16.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.008+0000 7fd4f6a00640 1 -- 172.21.15.44:0/3485753838 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v107) ==== 59+0+1998 (secure 0 0 0) 0x7fd4f003f030 con 0x7fd5000adb70 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:45:16.010 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:45:16.011 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:45:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:45:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 msgr2=0x7fd4dc0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 0x7fd4dc0790a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5000c8a10 tx=0x7fd4e8009040 comp rx=0 tx=0).stop 2024-07-17T08:45:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 msgr2=0x7fd5001f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00047a0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 shutdown_connections 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.011+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fd4dc076be0 0x7fd4dc0790a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd50015d6d0 0x7fd50015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd50015d2f0 0x7fd500148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 --2- 172.21.15.44:0/3485753838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5000adb70 0x7fd5001f3550 unknown :-1 s=CLOSED pgs=3286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 >> 172.21.15.44:0/3485753838 conn(0x7fd50006c120 msgr2=0x7fd50006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 shutdown_connections 2024-07-17T08:45:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.012+0000 7fd4ff200640 1 -- 172.21.15.44:0/3485753838 wait complete. 2024-07-17T08:45:16.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:45:16.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 -- 172.21.15.44:0/2177052528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc161120 msgr2=0x7f06fc06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 --2- 172.21.15.44:0/2177052528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc161120 0x7f06fc06d620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009900 tx=0x7f06e402a610 comp rx=0 tx=0).stop 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 -- 172.21.15.44:0/2177052528 shutdown_connections 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 --2- 172.21.15.44:0/2177052528 >> v2:172.21.15.175:3300/0 conn(0x7f06fc06db60 0x7f06fc0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 --2- 172.21.15.44:0/2177052528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc161120 0x7f06fc06d620 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.249+0000 7f0703000640 1 -- 172.21.15.44:0/2177052528 >> 172.21.15.44:0/2177052528 conn(0x7f06fc0b0eb0 msgr2=0x7f06fc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.250+0000 7f0703000640 1 -- 172.21.15.44:0/2177052528 shutdown_connections 2024-07-17T08:45:16.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.250+0000 7f0703000640 1 -- 172.21.15.44:0/2177052528 wait complete. 2024-07-17T08:45:16.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.251+0000 7f0703000640 1 Processor -- start 2024-07-17T08:45:16.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.252+0000 7f0703000640 1 -- start start 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc161120 0x7f06fc1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc14e640 0x7f06fc072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc15e220 con 0x7f06fc154600 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc15e0a0 con 0x7f06fc161120 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0703000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc15e3a0 con 0x7f06fc14e640 2024-07-17T08:45:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51222/0 (socket says 172.21.15.44:51222) 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 -- 172.21.15.44:0/1166712958 learned_addr learned my addr 172.21.15.44:0/1166712958 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06f3e00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc161120 0x7f06fc1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f0701000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc14e640 0x7f06fc072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 -- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc14e640 msgr2=0x7f06fc072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc14e640 0x7f06fc072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 -- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc161120 msgr2=0x7f06fc1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc161120 0x7f06fc1f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.253+0000 7f06fbe00640 1 -- 172.21.15.44:0/1166712958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc145a10 con 0x7f06fc154600 2024-07-17T08:45:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f06fbe00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c480 tx=0x7f06ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0052d0 con 0x7f06fc154600 2024-07-17T08:45:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06ec005470 con 0x7f06fc154600 2024-07-17T08:45:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec012870 con 0x7f06fc154600 2024-07-17T08:45:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc0715e0 con 0x7f06fc154600 2024-07-17T08:45:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.254+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc071b60 con 0x7f06fc154600 2024-07-17T08:45:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.256+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc15e8c0 con 0x7f06fc154600 2024-07-17T08:45:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.257+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f06ec0035f0 con 0x7f06fc154600 2024-07-17T08:45:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.258+0000 7f06f8c00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 0x7f06d80790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.258+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7f06ec098440 con 0x7f06fc154600 2024-07-17T08:45:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.259+0000 7f06f3e00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 0x7f06d80790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.260+0000 7f06f3e00640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 0x7f06d80790a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f06e4033000 tx=0x7f06e4004730 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:16.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.263+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f06fc1549e0 con 0x7f06fc154600 2024-07-17T08:45:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.471+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f06fc160e90 con 0x7f06fc154600 2024-07-17T08:45:16.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.479+0000 7f06f8c00640 1 -- 172.21.15.44:0/1166712958 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v108)=0 v108) ==== 122+0+0 (secure 0 0 0) 0x7f06ec061460 con 0x7f06fc154600 2024-07-17T08:45:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 msgr2=0x7f06d80790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 0x7f06d80790a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f06e4033000 tx=0x7f06e4004730 comp rx=0 tx=0).stop 2024-07-17T08:45:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 msgr2=0x7f06fc15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c480 tx=0x7f06ec00c940 comp rx=0 tx=0).stop 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 shutdown_connections 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f06d8076be0 0x7f06d80790a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06fc14e640 0x7f06fc072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06fc161120 0x7f06fc1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 --2- 172.21.15.44:0/1166712958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06fc154600 0x7f06fc15a620 unknown :-1 s=CLOSED pgs=3287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.482+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 >> 172.21.15.44:0/1166712958 conn(0x7f06fc0b0eb0 msgr2=0x7f06fc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.483+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 shutdown_connections 2024-07-17T08:45:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:16.483+0000 7f0703000640 1 -- 172.21.15.44:0/1166712958 wait complete. 2024-07-17T08:45:16.506 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:45:16.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:45:16.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:45:16.739 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:45:16.740 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:16.740 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:16.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:45:16.800 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:45:16.800 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:16.856 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:45:16.857 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:16.857 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:16.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:45:16.915 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:45:16.915 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:16.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:45:16.972 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:45:16.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 -- 172.21.15.44:0/3691056931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 msgr2=0x7fedc00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 --2- 172.21.15.44:0/3691056931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00b25d0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009830 tx=0x7fedb802a4d0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 -- 172.21.15.44:0/3691056931 shutdown_connections 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 --2- 172.21.15.44:0/3691056931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00b25d0 unknown :-1 s=CLOSED pgs=3289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 --2- 172.21.15.44:0/3691056931 >> v2:172.21.15.175:3300/0 conn(0x7fedc00ae810 0x7fedc015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 -- 172.21.15.44:0/3691056931 >> 172.21.15.44:0/3691056931 conn(0x7fedc00b0fb0 msgr2=0x7fedc00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 -- 172.21.15.44:0/3691056931 shutdown_connections 2024-07-17T08:45:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.195+0000 7fedc8000640 1 -- 172.21.15.44:0/3691056931 wait complete. 2024-07-17T08:45:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.197+0000 7fedc8000640 1 Processor -- start 2024-07-17T08:45:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.197+0000 7fedc8000640 1 -- start start 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedc006ea50 0x7fedc01f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedc00ae810 0x7fedc01f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fedc006f040 con 0x7fedc00ac930 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fedc006eec0 con 0x7fedc00ae810 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fedc00b2b10 con 0x7fedc006ea50 2024-07-17T08:45:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51242/0 (socket says 172.21.15.44:51242) 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 -- 172.21.15.44:0/1071122702 learned_addr learned my addr 172.21.15.44:0/1071122702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc5600640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedc006ea50 0x7fedc01f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc6000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedc00ae810 0x7fedc01f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 -- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedc006ea50 msgr2=0x7fedc01f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedc006ea50 0x7fedc01f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 -- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedc00ae810 msgr2=0x7fedc01f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedc00ae810 0x7fedc01f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.198+0000 7fedc4c00640 1 -- 172.21.15.44:0/1071122702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00b1940 con 0x7fedc00ac930 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedc4c00640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c730 tx=0x7fedac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac017070 con 0x7fedc00ac930 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedac005180 con 0x7fedc00ac930 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00a640 con 0x7fedc00ac930 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc00983d0 con 0x7fedc00ac930 2024-07-17T08:45:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.199+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc0097e10 con 0x7fedc00ac930 2024-07-17T08:45:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.201+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc006a1a0 con 0x7fedc00ac930 2024-07-17T08:45:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.202+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fedac01a070 con 0x7fedc00ac930 2024-07-17T08:45:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.203+0000 7fedbd600640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 0x7feda0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.203+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7fedac097b40 con 0x7fedc00ac930 2024-07-17T08:45:17.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.204+0000 7fedc5600640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 0x7feda0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.205+0000 7fedc5600640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 0x7feda0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005c40 tx=0x7fedb4005bd0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:17.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.207+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fedc015ea50 con 0x7fedc00ac930 2024-07-17T08:45:17.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.462+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedc00b4690 con 0x7fedc00ac930 2024-07-17T08:45:17.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.464+0000 7fedbd600640 1 -- 172.21.15.44:0/1071122702 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) ==== 95+0+3923 (secure 0 0 0) 0x7fedac060c70 con 0x7fedc00ac930 2024-07-17T08:45:17.466 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:17.466 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":202,"btime":"2024-07-17T08:45:09:477238+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/4192135110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":4192135110},{"type":"v1","addr":"172.21.15.175:6801","nonce":4192135110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":11050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/831107770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":831107770},{"type":"v1","addr":"172.21.15.175:6836","nonce":831107770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":195},{"gid":13953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/1379961159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1379961159},{"type":"v1","addr":"172.21.15.44:6834","nonce":1379961159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":14334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1559344831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":1559344831},{"type":"v1","addr":"172.21.15.44:6835","nonce":1559344831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":202}],"filesystems":[]} 2024-07-17T08:45:17.466 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 202 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 msgr2=0x7feda0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 0x7feda0079070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fedb4005c40 tx=0x7fedb4005bd0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 msgr2=0x7fedc00974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c730 tx=0x7fedac00cbf0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 shutdown_connections 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7feda0076bb0 0x7feda0079070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedc00ae810 0x7fedc01f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedc00ac930 0x7fedc00974a0 unknown :-1 s=CLOSED pgs=3290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 --2- 172.21.15.44:0/1071122702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedc006ea50 0x7fedc01f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 >> 172.21.15.44:0/1071122702 conn(0x7fedc00b0fb0 msgr2=0x7fedc00af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 shutdown_connections 2024-07-17T08:45:17.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.467+0000 7fedc8000640 1 -- 172.21.15.44:0/1071122702 wait complete. 2024-07-17T08:45:17.487 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:45:17.487 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:45:17.488 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:45:17.488 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:45:17.488 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:45:17.488 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:45:17.488 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:45:17.488 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:45:17.489 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:45:17.489 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:45:17.489 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:45:17.489 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:45:17.489 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:45:17.489 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:45:17.489 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:45:17.490 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:45:17.514 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:45:17.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:45:17.518 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:45:17.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:45:17.519 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:45:17.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:45:17.521 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:45:17.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:45:17.566 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:45:17.567 DEBUG:teuthology.parallel:result is None 2024-07-17T08:45:17.568 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:45:17.569 DEBUG:teuthology.parallel:result is None 2024-07-17T08:45:17.571 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:45:17.571 DEBUG:teuthology.parallel:result is None 2024-07-17T08:45:17.574 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:45:17.574 DEBUG:teuthology.parallel:result is None 2024-07-17T08:45:17.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:45:17.713 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:45:17.714 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:45:17.722 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:45:17.722 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:45:17.723 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T08:45:17.721+0000 7f4e37800640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 -- 172.21.15.44:0/973862742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 msgr2=0x7fe93c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 --2- 172.21.15.44:0/973862742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c15d510 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0cf340 tx=0x7fe92402a4b0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 -- 172.21.15.44:0/973862742 shutdown_connections 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 --2- 172.21.15.44:0/973862742 >> v2:172.21.15.175:3300/0 conn(0x7fe93c0ce690 0x7fe93c0cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 --2- 172.21.15.44:0/973862742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c15d510 unknown :-1 s=CLOSED pgs=3293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 -- 172.21.15.44:0/973862742 >> 172.21.15.44:0/973862742 conn(0x7fe93c0b4ff0 msgr2=0x7fe93c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 -- 172.21.15.44:0/973862742 shutdown_connections 2024-07-17T08:45:17.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.751+0000 7fe941c00640 1 -- 172.21.15.44:0/973862742 wait complete. 2024-07-17T08:45:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.752+0000 7fe941c00640 1 Processor -- start 2024-07-17T08:45:17.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.753+0000 7fe941c00640 1 -- start start 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.753+0000 7fe941c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe93c0ce690 0x7fe93c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.753+0000 7fe941c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 0x7fe93c160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0d19d0 con 0x7fe93c161be0 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0d1850 con 0x7fe93c15d130 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c0d1b50 con 0x7fe93c0ce690 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51270/0 (socket says 172.21.15.44:51270) 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 -- 172.21.15.44:0/353522866 learned_addr learned my addr 172.21.15.44:0/353522866 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe933400640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 0x7fe93c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe93be00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe93c0ce690 0x7fe93c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 -- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe93c0ce690 msgr2=0x7fe93c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe93c0ce690 0x7fe93c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 -- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 msgr2=0x7fe93c160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 0x7fe93c160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe941200640 1 -- 172.21.15.44:0/353522866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c15d720 con 0x7fe93c161be0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.754+0000 7fe933400640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 0x7fe93c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe941200640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e990 tx=0x7fe93400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934004d20 con 0x7fe93c161be0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe93400cb00 con 0x7fe93c161be0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c15da00 con 0x7fe93c161be0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934005020 con 0x7fe93c161be0 2024-07-17T08:45:17.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.755+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c15eb80 con 0x7fe93c161be0 2024-07-17T08:45:17.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.756+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c1f3c60 con 0x7fe93c161be0 2024-07-17T08:45:17.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.757+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fe9340051c0 con 0x7fe93c161be0 2024-07-17T08:45:17.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.758+0000 7fe938c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 0x7fe918079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:17.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.758+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(356..356 src has 1..356) ==== 6784+0+0 (secure 0 0 0) 0x7fe934098f80 con 0x7fe93c161be0 2024-07-17T08:45:17.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.759+0000 7fe93be00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 0x7fe918079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:17.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.760+0000 7fe93be00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 0x7fe918079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0017c0 tx=0x7fe92c007e90 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:17.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.763+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe934016030 con 0x7fe93c161be0 2024-07-17T08:45:17.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:17.988+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe93c0cfdf0 con 0x7fe93c161be0 2024-07-17T08:45:18.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.567+0000 7fe938c00640 1 -- 172.21.15.44:0/353522866 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v358) ==== 97+0+0 (secure 0 0 0) 0x7fe93406a260 con 0x7fe93c161be0 2024-07-17T08:45:18.570 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:45:18.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 msgr2=0x7fe918079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:18.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 0x7fe918079070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0017c0 tx=0x7fe92c007e90 comp rx=0 tx=0).stop 2024-07-17T08:45:18.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 msgr2=0x7fe93c1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:18.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7fe93400e990 tx=0x7fe93400ee50 comp rx=0 tx=0).stop 2024-07-17T08:45:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 shutdown_connections 2024-07-17T08:45:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fe918076bb0 0x7fe918079070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe93c161be0 0x7fe93c1f4390 unknown :-1 s=CLOSED pgs=3294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe93c15d130 0x7fe93c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 --2- 172.21.15.44:0/353522866 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe93c0ce690 0x7fe93c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.572+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 >> 172.21.15.44:0/353522866 conn(0x7fe93c0b4ff0 msgr2=0x7fe93c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.573+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 shutdown_connections 2024-07-17T08:45:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.573+0000 7fe941c00640 1 -- 172.21.15.44:0/353522866 wait complete. 2024-07-17T08:45:18.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:45:18.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 -- 172.21.15.44:0/3453938356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 msgr2=0x7f1754004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:18.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 --2- 172.21.15.44:0/3453938356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f1754004c80 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7f175800bb20 tx=0x7f1754005830 comp rx=0 tx=0).stop 2024-07-17T08:45:18.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1775600640 1 -- 172.21.15.44:0/3453938356 reap_dead start 2024-07-17T08:45:18.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 -- 172.21.15.44:0/3453938356 shutdown_connections 2024-07-17T08:45:18.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 -- 172.21.15.44:0/3453938356 >> 172.21.15.44:0/3453938356 conn(0x7f17700b0e80 msgr2=0x7f17700b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:18.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 -- 172.21.15.44:0/3453938356 shutdown_connections 2024-07-17T08:45:18.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.806+0000 7f1776000640 1 -- 172.21.15.44:0/3453938356 wait complete. 2024-07-17T08:45:18.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.808+0000 7f1776000640 1 Processor -- start 2024-07-17T08:45:18.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.808+0000 7f1776000640 1 -- start start 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1770152ba0 0x7f17701f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1770158540 0x7f177014e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f177009b690 con 0x7f1754004800 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f177009b510 con 0x7f1770152ba0 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.809+0000 7f1776000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f177009b810 con 0x7f1770158540 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51286/0 (socket says 172.21.15.44:51286) 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 -- 172.21.15.44:0/33296387 learned_addr learned my addr 172.21.15.44:0/33296387 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1775600640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1770158540 0x7f177014e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:18.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 -- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1770158540 msgr2=0x7f177014e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1770158540 0x7f177014e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1767e00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1770152ba0 0x7f17701f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 -- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1770152ba0 msgr2=0x7f17701f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1770152ba0 0x7f17701f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 -- 172.21.15.44:0/33296387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17700ac1f0 con 0x7f1754004800 2024-07-17T08:45:18.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.810+0000 7f1774c00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 secure :-1 s=READY pgs=3297 cs=0 l=1 rev1=1 crypto rx=0x7f176000d8f0 tx=0x7f176000ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.811+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176000b830 con 0x7f1754004800 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.811+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f176000bac0 con 0x7f1754004800 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.811+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600035f0 con 0x7f1754004800 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.811+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1770153080 con 0x7f1754004800 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.811+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17700adf30 con 0x7f1754004800 2024-07-17T08:45:18.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.812+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177006e060 con 0x7f1754004800 2024-07-17T08:45:18.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.813+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f1760003790 con 0x7f1754004800 2024-07-17T08:45:18.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.814+0000 7f176d600640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 0x7f174c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:18.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.814+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(360..360 src has 1..360) ==== 5881+0+0 (secure 0 0 0) 0x7f1760098ac0 con 0x7f1754004800 2024-07-17T08:45:18.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.815+0000 7f1767e00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 0x7f174c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:18.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.816+0000 7f1767e00640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 0x7f174c079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f175800bb20 tx=0x7f17580046c0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:18.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:18.819+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1760061e40 con 0x7f1754004800 2024-07-17T08:45:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.104+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f17701ecf90 con 0x7f1754004800 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.105+0000 7f176d600640 1 -- 172.21.15.44:0/33296387 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v83) ==== 80+0+5304 (secure 0 0 0) 0x7f1760061740 con 0x7f1754004800 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.107 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.109 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.110 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.111 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.112 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.118 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:45:19.120 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.107+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 msgr2=0x7f174c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.107+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 0x7f174c079070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f175800bb20 tx=0x7f17580046c0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 msgr2=0x7f17701f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 secure :-1 s=READY pgs=3297 cs=0 l=1 rev1=1 crypto rx=0x7f176000d8f0 tx=0x7f176000ddb0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 shutdown_connections 2024-07-17T08:45:19.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f174c076bb0 0x7f174c079070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1770158540 0x7f177014e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1770152ba0 0x7f17701f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 --2- 172.21.15.44:0/33296387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1754004800 0x7f17701f7970 unknown :-1 s=CLOSED pgs=3297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 >> 172.21.15.44:0/33296387 conn(0x7f17700b0e80 msgr2=0x7f17700b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 shutdown_connections 2024-07-17T08:45:19.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.108+0000 7f1776000640 1 -- 172.21.15.44:0/33296387 wait complete. 2024-07-17T08:45:19.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:45:19.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 -- 172.21.15.44:0/3759955495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac0cfa90 msgr2=0x7fb5ac06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 --2- 172.21.15.44:0/3759955495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac0cfa90 0x7fb5ac06e060 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7fb598005fd0 tx=0x7fb59802b860 comp rx=0 tx=0).stop 2024-07-17T08:45:19.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 -- 172.21.15.44:0/3759955495 shutdown_connections 2024-07-17T08:45:19.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 --2- 172.21.15.44:0/3759955495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac159800 0x7fb5ac159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 --2- 172.21.15.44:0/3759955495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac0cfa90 0x7fb5ac06e060 unknown :-1 s=CLOSED pgs=3298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 -- 172.21.15.44:0/3759955495 >> 172.21.15.44:0/3759955495 conn(0x7fb5ac0b0e80 msgr2=0x7fb5ac0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:19.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 -- 172.21.15.44:0/3759955495 shutdown_connections 2024-07-17T08:45:19.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.341+0000 7fb5abe00640 1 -- 172.21.15.44:0/3759955495 wait complete. 2024-07-17T08:45:19.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.344+0000 7fb5abe00640 1 Processor -- start 2024-07-17T08:45:19.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.344+0000 7fb5abe00640 1 -- start start 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac0cfa90 0x7fb5ac151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ac159800 0x7fb5ac0afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac0b2120 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac0b1fa0 con 0x7fb5ac159800 2024-07-17T08:45:19.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5abe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac0b22a0 con 0x7fb5ac0cfa90 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35722/0 (socket says 172.21.15.44:35722) 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 -- 172.21.15.44:0/1873350236 learned_addr learned my addr 172.21.15.44:0/1873350236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aa000640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac0cfa90 0x7fb5ac151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5ab400640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ac159800 0x7fb5ac0afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 -- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac0cfa90 msgr2=0x7fb5ac151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac0cfa90 0x7fb5ac151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 -- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ac159800 msgr2=0x7fb5ac0afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ac159800 0x7fb5ac0afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.345+0000 7fb5aaa00640 1 -- 172.21.15.44:0/1873350236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac0cb8d0 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5aaa00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c460 tx=0x7fb59c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c016020 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb59c012960 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ac0cbbb0 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c012db0 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.346+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ac0cc170 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.347+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ac06e060 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.348+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fb59c019020 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.349+0000 7fb5a2a00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 0x7fb58c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.349+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(360..360 src has 1..360) ==== 5881+0+0 (secure 0 0 0) 0x7fb59c098160 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.350+0000 7fb5aa000640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 0x7fb58c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.351+0000 7fb5aa000640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 0x7fb58c079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb598035000 tx=0x7fb598004730 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:19.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.353+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb59c020030 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.566+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb5ac153a60 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.567+0000 7fb5a2a00640 1 -- 172.21.15.44:0/1873350236 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v360) ==== 84+0+65 (secure 0 0 0) 0x7fb59c061570 con 0x7fb5ac06f4b0 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:45:19.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:19.570 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 msgr2=0x7fb58c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 0x7fb58c079070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb598035000 tx=0x7fb598004730 comp rx=0 tx=0).stop 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 msgr2=0x7fb5ac0c40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c460 tx=0x7fb59c00c920 comp rx=0 tx=0).stop 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 shutdown_connections 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb58c076bb0 0x7fb58c079070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5ac159800 0x7fb5ac0afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5ac0cfa90 0x7fb5ac151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 --2- 172.21.15.44:0/1873350236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5ac06f4b0 0x7fb5ac0c40a0 unknown :-1 s=CLOSED pgs=3299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 >> 172.21.15.44:0/1873350236 conn(0x7fb5ac0b0e80 msgr2=0x7fb5ac06c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:19.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 shutdown_connections 2024-07-17T08:45:19.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.570+0000 7fb5abe00640 1 -- 172.21.15.44:0/1873350236 wait complete. 2024-07-17T08:45:19.591 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:45:19.591 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:45:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 -- 172.21.15.44:0/508547997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb815c340 msgr2=0x7f1bb8073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 --2- 172.21.15.44:0/508547997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb815c340 0x7f1bb8073fb0 secure :-1 s=READY pgs=3301 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80b4440 tx=0x7f1ba802a540 comp rx=0 tx=0).stop 2024-07-17T08:45:19.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 -- 172.21.15.44:0/508547997 shutdown_connections 2024-07-17T08:45:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 --2- 172.21.15.44:0/508547997 >> v2:172.21.15.175:3300/0 conn(0x7f1bb806f670 0x7f1bb806fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 --2- 172.21.15.44:0/508547997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb815c340 0x7f1bb8073fb0 unknown :-1 s=CLOSED pgs=3301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 -- 172.21.15.44:0/508547997 >> 172.21.15.44:0/508547997 conn(0x7f1bb806bf40 msgr2=0x7f1bb806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 -- 172.21.15.44:0/508547997 shutdown_connections 2024-07-17T08:45:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.786+0000 7f1bb7200640 1 -- 172.21.15.44:0/508547997 wait complete. 2024-07-17T08:45:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.788+0000 7f1bb7200640 1 Processor -- start 2024-07-17T08:45:19.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.788+0000 7f1bb7200640 1 -- start start 2024-07-17T08:45:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bb806f670 0x7f1bb81591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 0x7f1bb815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb80b4a30 con 0x7f1bb80cfe40 2024-07-17T08:45:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb80b48b0 con 0x7f1bb815c340 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb80b4bb0 con 0x7f1bb806f670 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35738/0 (socket says 172.21.15.44:35738) 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 -- 172.21.15.44:0/3465207848 learned_addr learned my addr 172.21.15.44:0/3465207848 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5e00640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bb806f670 0x7f1bb81591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb6800640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 0x7f1bb815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 -- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bb806f670 msgr2=0x7f1bb81591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bb806f670 0x7f1bb81591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 -- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 msgr2=0x7f1bb815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 0x7f1bb815d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb5400640 1 -- 172.21.15.44:0/3465207848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb806d4e0 con 0x7f1bb80cfe40 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.790+0000 7f1bb6800640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 0x7f1bb815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bb5400640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 secure :-1 s=READY pgs=3302 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400d950 tx=0x7f1ba400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4017070 con 0x7f1bb80cfe40 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba4009d20 con 0x7f1bb80cfe40 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb806d7c0 con 0x7f1bb80cfe40 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40106c0 con 0x7f1bb80cfe40 2024-07-17T08:45:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.791+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb806dd80 con 0x7f1bb80cfe40 2024-07-17T08:45:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.792+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb80b5820 con 0x7f1bb80cfe40 2024-07-17T08:45:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.793+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f1ba4010860 con 0x7f1bb80cfe40 2024-07-17T08:45:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.795+0000 7f1bae000640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 0x7f1b98079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.795+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(360..360 src has 1..360) ==== 5881+0+0 (secure 0 0 0) 0x7f1ba4098740 con 0x7f1bb80cfe40 2024-07-17T08:45:19.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.795+0000 7f1bb5e00640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 0x7f1b98079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:19.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.798+0000 7f1bb5e00640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 0x7f1b98079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8034000 tx=0x7f1ba8004730 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:19.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:19.800+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1ba4061b50 con 0x7f1bb80cfe40 2024-07-17T08:45:20.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.007+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1bb8074d50 con 0x7f1bb80cfe40 2024-07-17T08:45:20.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.580+0000 7f1bae000640 1 -- 172.21.15.44:0/3465207848 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v361) ==== 138+0+0 (secure 0 0 0) 0x7f1ba4061450 con 0x7f1bb80cfe40 2024-07-17T08:45:20.581 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:45:20.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 msgr2=0x7f1b98079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:20.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 0x7f1b98079070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8034000 tx=0x7f1ba8004730 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 msgr2=0x7f1bb814bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 secure :-1 s=READY pgs=3302 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400d950 tx=0x7f1ba400de10 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 shutdown_connections 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1b98076bb0 0x7f1b98079070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bb815c340 0x7f1bb815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1bb80cfe40 0x7f1bb814bab0 unknown :-1 s=CLOSED pgs=3302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 --2- 172.21.15.44:0/3465207848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bb806f670 0x7f1bb81591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.584+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 >> 172.21.15.44:0/3465207848 conn(0x7f1bb806bf40 msgr2=0x7f1bb80b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.585+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 shutdown_connections 2024-07-17T08:45:20.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.585+0000 7f1bb7200640 1 -- 172.21.15.44:0/3465207848 wait complete. 2024-07-17T08:45:20.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:45:20.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2be2000640 1 -- 172.21.15.44:0/628149163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc0004d40 msgr2=0x7f2bc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2be2000640 1 --2- 172.21.15.44:0/628149163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bc0004d40 0x7f2bc00051c0 secure :-1 s=READY pgs=3304 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c630 tx=0x7f2bc0005830 comp rx=0 tx=0).stop 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2bdbe00640 1 -- 172.21.15.44:0/628149163 reap_dead start 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2be2000640 1 -- 172.21.15.44:0/628149163 shutdown_connections 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2be2000640 1 -- 172.21.15.44:0/628149163 >> 172.21.15.44:0/628149163 conn(0x7f2bdc0b0eb0 msgr2=0x7f2bdc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.816+0000 7f2be2000640 1 -- 172.21.15.44:0/628149163 shutdown_connections 2024-07-17T08:45:20.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.817+0000 7f2be2000640 1 -- 172.21.15.44:0/628149163 wait complete. 2024-07-17T08:45:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.818+0000 7f2be2000640 1 Processor -- start 2024-07-17T08:45:20.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.818+0000 7f2be2000640 1 -- start start 2024-07-17T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.819+0000 7f2be2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc0004d40 0x7f2bdc1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.819+0000 7f2be2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.819+0000 7f2be2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc1f7c30 0x7f2bdc1f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:20.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.819+0000 7f2be2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc15c470 con 0x7f2bdc158540 2024-07-17T08:45:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.819+0000 7f2be2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc15c2f0 con 0x7f2bc0004d40 2024-07-17T08:45:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2be2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc15c5f0 con 0x7f2bdc1f7c30 2024-07-17T08:45:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35758/0 (socket says 172.21.15.44:35758) 2024-07-17T08:45:20.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 -- 172.21.15.44:0/4027356790 learned_addr learned my addr 172.21.15.44:0/4027356790 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bdbe00640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc1f7c30 0x7f2bdc1f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bdb400640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc0004d40 0x7f2bdc1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 -- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc1f7c30 msgr2=0x7f2bdc1f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc1f7c30 0x7f2bdc1f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 -- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc0004d40 msgr2=0x7f2bdc1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc0004d40 0x7f2bdc1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:20.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.820+0000 7f2bd2a00640 1 -- 172.21.15.44:0/4027356790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc158f00 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2bd2a00640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4010a10 tx=0x7f2bc4010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400eda0 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc4004030 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc15d480 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004440 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.821+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc15da40 con 0x7f2bdc158540 2024-07-17T08:45:20.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.822+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc0cd2e0 con 0x7f2bdc158540 2024-07-17T08:45:20.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.824+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f2bc40035f0 con 0x7f2bdc158540 2024-07-17T08:45:20.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.825+0000 7f2bd3e00640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 0x7f2bb8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:20.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.826+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(361..361 src has 1..361) ==== 6251+0+0 (secure 0 0 0) 0x7f2bc40997a0 con 0x7f2bdc158540 2024-07-17T08:45:20.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.826+0000 7f2bdb400640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 0x7f2bb8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:20.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.827+0000 7f2bdb400640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 0x7f2bb8079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc006b20 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:20.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:20.829+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2bdc158f00 con 0x7f2bdc158540 2024-07-17T08:45:21.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.040+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2bdc0ae1a0 con 0x7f2bdc158540 2024-07-17T08:45:21.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.602+0000 7f2bd3e00640 1 -- 172.21.15.44:0/4027356790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v362) ==== 170+0+0 (secure 0 0 0) 0x7f2bc4062a30 con 0x7f2bdc158540 2024-07-17T08:45:21.603 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 msgr2=0x7f2bb8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 0x7f2bb8079070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc006b20 comp rx=0 tx=0).stop 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 msgr2=0x7f2bdc1589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4010a10 tx=0x7f2bc4010ed0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 shutdown_connections 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2bb8076bb0 0x7f2bb8079070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc1f7c30 0x7f2bdc1f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bdc158540 0x7f2bdc1589c0 unknown :-1 s=CLOSED pgs=3305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 --2- 172.21.15.44:0/4027356790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc0004d40 0x7f2bdc1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.606+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 >> 172.21.15.44:0/4027356790 conn(0x7f2bdc0b0eb0 msgr2=0x7f2bdc159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:21.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.608+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 shutdown_connections 2024-07-17T08:45:21.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.611+0000 7f2be2000640 1 -- 172.21.15.44:0/4027356790 wait complete. 2024-07-17T08:45:21.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:45:21.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.833+0000 7f41ca800640 1 -- 172.21.15.44:0/3070093252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41a80051c0 msgr2=0x7f41a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.833+0000 7f41ca800640 1 --2- 172.21.15.44:0/3070093252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41a80051c0 0x7f41a800b9c0 secure :-1 s=READY pgs=3307 cs=0 l=1 rev1=1 crypto rx=0x7f41c40ad090 tx=0x7f41a8005830 comp rx=0 tx=0).stop 2024-07-17T08:45:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.834+0000 7f41c3e00640 1 -- 172.21.15.44:0/3070093252 reap_dead start 2024-07-17T08:45:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.834+0000 7f41ca800640 1 -- 172.21.15.44:0/3070093252 shutdown_connections 2024-07-17T08:45:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.834+0000 7f41ca800640 1 -- 172.21.15.44:0/3070093252 >> 172.21.15.44:0/3070093252 conn(0x7f41c40b0eb0 msgr2=0x7f41c40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:21.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.834+0000 7f41ca800640 1 -- 172.21.15.44:0/3070093252 shutdown_connections 2024-07-17T08:45:21.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.834+0000 7f41ca800640 1 -- 172.21.15.44:0/3070093252 wait complete. 2024-07-17T08:45:21.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.836+0000 7f41ca800640 1 Processor -- start 2024-07-17T08:45:21.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.836+0000 7f41ca800640 1 -- start start 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 0x7f41c4155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c41f5490 0x7f41c40af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41c406d790 con 0x7f41c41f26f0 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41c406f1d0 con 0x7f41a80051c0 2024-07-17T08:45:21.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41ca800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41c415ac10 con 0x7f41c41f5490 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35778/0 (socket says 172.21.15.44:35778) 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 -- 172.21.15.44:0/719507307 learned_addr learned my addr 172.21.15.44:0/719507307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41bbe00640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c41f5490 0x7f41c40af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3400640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 0x7f41c4155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:21.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 -- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c41f5490 msgr2=0x7f41c40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c41f5490 0x7f41c40af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 -- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 msgr2=0x7f41c4155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 0x7f41c4155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3e00640 1 -- 172.21.15.44:0/719507307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c41f5c70 con 0x7f41c41f26f0 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.837+0000 7f41c3400640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 0x7f41c4155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41c3e00640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e410 tx=0x7f41bc033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc009c30 con 0x7f41c41f26f0 2024-07-17T08:45:21.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41bc043070 con 0x7f41c41f26f0 2024-07-17T08:45:21.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c41f5f50 con 0x7f41c41f26f0 2024-07-17T08:45:21.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc03d6c0 con 0x7f41c41f26f0 2024-07-17T08:45:21.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.838+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41f21d0 con 0x7f41c41f26f0 2024-07-17T08:45:21.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.839+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c40b21b0 con 0x7f41c41f26f0 2024-07-17T08:45:21.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.841+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f41bc03d860 con 0x7f41c41f26f0 2024-07-17T08:45:21.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.843+0000 7f41bb400640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 0x7f41a0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:21.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.843+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(362..362 src has 1..362) ==== 6633+0+0 (secure 0 0 0) 0x7f41bc0ba690 con 0x7f41c41f26f0 2024-07-17T08:45:21.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.844+0000 7f41c3400640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 0x7f41a0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:21.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.845+0000 7f41c3400640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 0x7f41a0079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009af0 tx=0x7f41b4005f90 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:21.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:21.847+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f41bc0837a0 con 0x7f41c41f26f0 2024-07-17T08:45:22.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.105+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f41c41f23a0 con 0x7f41c41f26f0 2024-07-17T08:45:22.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.606+0000 7f41bb400640 1 -- 172.21.15.44:0/719507307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 49 and data pool 50 v205) ==== 174+0+0 (secure 0 0 0) 0x7f41bc0830a0 con 0x7f41c41f26f0 2024-07-17T08:45:22.607 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 49 and data pool 50 2024-07-17T08:45:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.610+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 msgr2=0x7f41a0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.610+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 0x7f41a0079070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009af0 tx=0x7f41b4005f90 comp rx=0 tx=0).stop 2024-07-17T08:45:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.610+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 msgr2=0x7f41c4150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:22.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.610+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e410 tx=0x7f41bc033c10 comp rx=0 tx=0).stop 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 shutdown_connections 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f41a0076bb0 0x7f41a0079070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41c41f26f0 0x7f41c4150630 unknown :-1 s=CLOSED pgs=3308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c41f5490 0x7f41c40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 --2- 172.21.15.44:0/719507307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41a80051c0 0x7f41c4155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.611+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 >> 172.21.15.44:0/719507307 conn(0x7f41c40b0eb0 msgr2=0x7f41c40ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.613+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 shutdown_connections 2024-07-17T08:45:22.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.613+0000 7f41ca800640 1 -- 172.21.15.44:0/719507307 wait complete. 2024-07-17T08:45:22.644 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:22.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 -- 172.21.15.44:0/3561642518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 msgr2=0x7f85f0153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:22.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 --2- 172.21.15.44:0/3561642518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0153a50 secure :-1 s=READY pgs=3310 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009830 tx=0x7f85e002a510 comp rx=0 tx=0).stop 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 -- 172.21.15.44:0/3561642518 shutdown_connections 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 --2- 172.21.15.44:0/3561642518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 0x7f85f00ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 --2- 172.21.15.44:0/3561642518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0153a50 unknown :-1 s=CLOSED pgs=3310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 -- 172.21.15.44:0/3561642518 >> 172.21.15.44:0/3561642518 conn(0x7f85f00b0fb0 msgr2=0x7f85f00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 -- 172.21.15.44:0/3561642518 shutdown_connections 2024-07-17T08:45:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.828+0000 7f85f7200640 1 -- 172.21.15.44:0/3561642518 wait complete. 2024-07-17T08:45:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.830+0000 7f85f7200640 1 Processor -- start 2024-07-17T08:45:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.830+0000 7f85f7200640 1 -- start start 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 0x7f85f01e4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 0x7f85f01595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85f00ae800 con 0x7f85f006f710 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85f006f0a0 con 0x7f85f013ed60 2024-07-17T08:45:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.831+0000 7f85f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85f00ae980 con 0x7f85f00ac3e0 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35790/0 (socket says 172.21.15.44:35790) 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 -- 172.21.15.44:0/670179941 learned_addr learned my addr 172.21.15.44:0/670179941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85ef400640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 0x7f85f01e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 -- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 msgr2=0x7f85f01e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 0x7f85f01e4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 -- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 msgr2=0x7f85f01595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85f5200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 0x7f85f01595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 0x7f85f01595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85efe00640 1 -- 172.21.15.44:0/670179941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f0098eb0 con 0x7f85f006f710 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.832+0000 7f85f5200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 0x7f85f01595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85efe00640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009950 tx=0x7f85e0032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0040070 con 0x7f85f006f710 2024-07-17T08:45:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85e0005310 con 0x7f85f006f710 2024-07-17T08:45:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0005c90 con 0x7f85f006f710 2024-07-17T08:45:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f01e4f10 con 0x7f85f006f710 2024-07-17T08:45:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.833+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01e54d0 con 0x7f85f006f710 2024-07-17T08:45:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.834+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f013fad0 con 0x7f85f006f710 2024-07-17T08:45:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.835+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f85e0039040 con 0x7f85f006f710 2024-07-17T08:45:22.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.836+0000 7f85e7e00640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 0x7f85d0079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:22.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.836+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(363..363 src has 1..363) ==== 6733+0+0 (secure 0 0 0) 0x7f85e00b9a10 con 0x7f85f006f710 2024-07-17T08:45:22.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.837+0000 7f85ef400640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 0x7f85d0079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:22.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.838+0000 7f85ef400640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 0x7f85d0079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0058c0 tx=0x7f85dc005850 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:22.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:22.841+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f85f0098eb0 con 0x7f85f006f710 2024-07-17T08:45:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.052+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85f00727b0 con 0x7f85f006f710 2024-07-17T08:45:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.054+0000 7f85e7e00640 1 -- 172.21.15.44:0/670179941 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v363) ==== 74+0+17920 (secure 0 0 0) 0x7f85e0082a40 con 0x7f85f006f710 2024-07-17T08:45:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:23.056 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":363,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:45:22.106789+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":49,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:45:20.033990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":50,"pool_name":"cephfs_data","create_time":"2024-07-17T08:45:21.067621+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 msgr2=0x7f85d0079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 0x7f85d0079070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0058c0 tx=0x7f85dc005850 comp rx=0 tx=0).stop 2024-07-17T08:45:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 msgr2=0x7f85f0098970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009950 tx=0x7f85e0032c10 comp rx=0 tx=0).stop 2024-07-17T08:45:23.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 shutdown_connections 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f85d0076bb0 0x7f85d0079070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f013ed60 0x7f85f01595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f00ac3e0 0x7f85f01e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 --2- 172.21.15.44:0/670179941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85f006f710 0x7f85f0098970 unknown :-1 s=CLOSED pgs=3311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 >> 172.21.15.44:0/670179941 conn(0x7f85f00b0fb0 msgr2=0x7f85f015bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 shutdown_connections 2024-07-17T08:45:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.057+0000 7f85f7200640 1 -- 172.21.15.44:0/670179941 wait complete. 2024-07-17T08:45:23.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 -- 172.21.15.44:0/3991794048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 msgr2=0x7fb5c80ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 --2- 172.21.15.44:0/3991794048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c80ad8e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4005fd0 tx=0x7fb5b402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 -- 172.21.15.44:0/3991794048 shutdown_connections 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 --2- 172.21.15.44:0/3991794048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c80ad8e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 --2- 172.21.15.44:0/3991794048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf540 0x7fb5c80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.285+0000 7fb5ce200640 1 -- 172.21.15.44:0/3991794048 >> 172.21.15.44:0/3991794048 conn(0x7fb5c80b0e80 msgr2=0x7fb5c80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.286+0000 7fb5ce200640 1 -- 172.21.15.44:0/3991794048 shutdown_connections 2024-07-17T08:45:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.286+0000 7fb5ce200640 1 -- 172.21.15.44:0/3991794048 wait complete. 2024-07-17T08:45:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.288+0000 7fb5ce200640 1 Processor -- start 2024-07-17T08:45:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.288+0000 7fb5ce200640 1 -- start start 2024-07-17T08:45:23.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c8071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf920 0x7fb5c809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80ae940 con 0x7fb5c80cf540 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80ae7c0 con 0x7fb5c80b3db0 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.289+0000 7fb5ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c8141040 con 0x7fb5c80cf920 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35800/0 (socket says 172.21.15.44:35800) 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 -- 172.21.15.44:0/1520401391 learned_addr learned my addr 172.21.15.44:0/1520401391 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5cce00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c8071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5cd800640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf920 0x7fb5c809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 -- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf920 msgr2=0x7fb5c809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf920 0x7fb5c809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 -- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 msgr2=0x7fb5c8071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c8071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.290+0000 7fb5c7e00640 1 -- 172.21.15.44:0/1520401391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c81e70c0 con 0x7fb5c80cf540 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5c7e00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b403f070 con 0x7fb5c80cf540 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b4005600 con 0x7fb5c80cf540 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4035030 con 0x7fb5c80cf540 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c81e73a0 con 0x7fb5c80cf540 2024-07-17T08:45:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.291+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c81e7960 con 0x7fb5c80cf540 2024-07-17T08:45:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.292+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c81414c0 con 0x7fb5c80cf540 2024-07-17T08:45:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.294+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fb5b4042070 con 0x7fb5c80cf540 2024-07-17T08:45:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.295+0000 7fb5c4c00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 0x7fb5a8079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.296+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(363..363 src has 1..363) ==== 6733+0+0 (secure 0 0 0) 0x7fb5b40b91d0 con 0x7fb5c80cf540 2024-07-17T08:45:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.296+0000 7fb5cce00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 0x7fb5a8079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.297+0000 7fb5cce00640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 0x7fb5a8079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009b60 tx=0x7fb5bc005f90 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.300+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb5b4082330 con 0x7fb5c80cf540 2024-07-17T08:45:23.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.514+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5c81e6580 con 0x7fb5c80cf540 2024-07-17T08:45:23.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.515+0000 7fb5c4c00640 1 -- 172.21.15.44:0/1520401391 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v363) ==== 74+0+17920 (secure 0 0 0) 0x7fb5b4081c30 con 0x7fb5c80cf540 2024-07-17T08:45:23.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:23.517 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":363,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:45:22.106789+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":49,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:45:20.033990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":50,"pool_name":"cephfs_data","create_time":"2024-07-17T08:45:21.067621+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.518+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 msgr2=0x7fb5a8079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.518+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 0x7fb5a8079070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009b60 tx=0x7fb5bc005f90 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.518+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 msgr2=0x7fb5c81549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.518+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4003820 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 shutdown_connections 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fb5a8076bb0 0x7fb5a8079070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5c80cf920 0x7fb5c809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5c80cf540 0x7fb5c81549e0 unknown :-1 s=CLOSED pgs=3312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 --2- 172.21.15.44:0/1520401391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5c80b3db0 0x7fb5c8071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 >> 172.21.15.44:0/1520401391 conn(0x7fb5c80b0e80 msgr2=0x7fb5c80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:23.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 shutdown_connections 2024-07-17T08:45:23.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.519+0000 7fb5ce200640 1 -- 172.21.15.44:0/1520401391 wait complete. 2024-07-17T08:45:23.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 -- 172.21.15.44:0/2741233279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc161be0 msgr2=0x7f81fc0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 --2- 172.21.15.44:0/2741233279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc161be0 0x7f81fc0b2200 secure :-1 s=READY pgs=3313 cs=0 l=1 rev1=1 crypto rx=0x7f81e8005fd0 tx=0x7f81e802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 -- 172.21.15.44:0/2741233279 shutdown_connections 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 --2- 172.21.15.44:0/2741233279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc161be0 0x7f81fc0b2200 unknown :-1 s=CLOSED pgs=3313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 --2- 172.21.15.44:0/2741233279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81fc15d130 0x7f81fc15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 -- 172.21.15.44:0/2741233279 >> 172.21.15.44:0/2741233279 conn(0x7f81fc0b4ff0 msgr2=0x7f81fc0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 -- 172.21.15.44:0/2741233279 shutdown_connections 2024-07-17T08:45:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.750+0000 7f8203000640 1 -- 172.21.15.44:0/2741233279 wait complete. 2024-07-17T08:45:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.752+0000 7f8203000640 1 Processor -- start 2024-07-17T08:45:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.752+0000 7f8203000640 1 -- start start 2024-07-17T08:45:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81fc0ce690 0x7f81fc1f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81fc161be0 0x7f81fc159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d1a10 con 0x7f81fc15d130 2024-07-17T08:45:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d1890 con 0x7f81fc0ce690 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8203000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0d1b90 con 0x7f81fc161be0 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8201200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8201200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35810/0 (socket says 172.21.15.44:35810) 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.753+0000 7f8201200640 1 -- 172.21.15.44:0/3605375656 learned_addr learned my addr 172.21.15.44:0/3605375656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201c00640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81fc0ce690 0x7f81fc1f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 -- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81fc161be0 msgr2=0x7f81fc159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81fc161be0 0x7f81fc159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 -- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81fc0ce690 msgr2=0x7f81fc1f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81fc0ce690 0x7f81fc1f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 -- 172.21.15.44:0/3605375656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0aa1b0 con 0x7f81fc15d130 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8201200640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 secure :-1 s=READY pgs=3314 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006010 tx=0x7f81e80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e803f070 con 0x7f81fc15d130 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81e8004890 con 0x7f81fc15d130 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8035030 con 0x7f81fc15d130 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.754+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0aa490 con 0x7f81fc15d130 2024-07-17T08:45:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.755+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0aaa50 con 0x7f81fc15d130 2024-07-17T08:45:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.756+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc0d3830 con 0x7f81fc15d130 2024-07-17T08:45:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.757+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f81e8042070 con 0x7f81fc15d130 2024-07-17T08:45:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.758+0000 7f81f9600640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 0x7f81dc079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.758+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f81e80b9760 con 0x7f81fc15d130 2024-07-17T08:45:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.758+0000 7f8201c00640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 0x7f81dc079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.760+0000 7f8201c00640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 0x7f81dc079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0058c0 tx=0x7f81ec005c60 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:23.762+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f81fc0d3830 con 0x7f81fc15d130 2024-07-17T08:45:24.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.012+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f81fc0cf410 con 0x7f81fc15d130 2024-07-17T08:45:24.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.651+0000 7f81f9600640 1 -- 172.21.15.44:0/3605375656 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v208) ==== 118+0+0 (secure 0 0 0) 0x7f81fc0cf410 con 0x7f81fc15d130 2024-07-17T08:45:24.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.654+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 msgr2=0x7f81dc079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:24.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.654+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 0x7f81dc079070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0058c0 tx=0x7f81ec005c60 comp rx=0 tx=0).stop 2024-07-17T08:45:24.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.654+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 msgr2=0x7f81fc15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:24.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.654+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 secure :-1 s=READY pgs=3314 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006010 tx=0x7f81e80039e0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 shutdown_connections 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81dc076bb0 0x7f81dc079070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81fc161be0 0x7f81fc159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81fc15d130 0x7f81fc15f560 unknown :-1 s=CLOSED pgs=3314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 --2- 172.21.15.44:0/3605375656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81fc0ce690 0x7f81fc1f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 >> 172.21.15.44:0/3605375656 conn(0x7f81fc0b4ff0 msgr2=0x7f81fc0d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 shutdown_connections 2024-07-17T08:45:24.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.655+0000 7f8203000640 1 -- 172.21.15.44:0/3605375656 wait complete. 2024-07-17T08:45:24.676 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:45:24.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:45:24.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 -- 172.21.15.44:0/1269904435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5618151f20 msgr2=0x7f561815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:24.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 --2- 172.21.15.44:0/1269904435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5618151f20 0x7f561815c140 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5608005fd0 tx=0x7f560802a5d0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 -- 172.21.15.44:0/1269904435 shutdown_connections 2024-07-17T08:45:24.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 --2- 172.21.15.44:0/1269904435 >> v2:172.21.15.175:3300/0 conn(0x7f561806f380 0x7f56180cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 --2- 172.21.15.44:0/1269904435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5618151f20 0x7f561815c140 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 -- 172.21.15.44:0/1269904435 >> 172.21.15.44:0/1269904435 conn(0x7f56180cf0d0 msgr2=0x7f56180cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:24.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 -- 172.21.15.44:0/1269904435 shutdown_connections 2024-07-17T08:45:24.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.889+0000 7f561d000640 1 -- 172.21.15.44:0/1269904435 wait complete. 2024-07-17T08:45:24.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.891+0000 7f561d000640 1 Processor -- start 2024-07-17T08:45:24.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.892+0000 7f561d000640 1 -- start start 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 0x7f56181f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56180cd0b0 0x7f56181f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f56180cb740 con 0x7f5618151f20 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56180cb5c0 con 0x7f561806f380 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f561d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56180cb8c0 con 0x7f56180cd0b0 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f5617e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:24.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f5617e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35824/0 (socket says 172.21.15.44:35824) 2024-07-17T08:45:24.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f5617e00640 1 -- 172.21.15.44:0/497023707 learned_addr learned my addr 172.21.15.44:0/497023707 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:24.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.893+0000 7f5617400640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 0x7f56181f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:24.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f560fe00640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56180cd0b0 0x7f56181f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 -- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56180cd0b0 msgr2=0x7f56181f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56180cd0b0 0x7f56181f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 -- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 msgr2=0x7f56181f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 0x7f56181f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 -- 172.21.15.44:0/497023707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561814d6b0 con 0x7f5618151f20 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617400640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 0x7f56181f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f5617e00640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f561000eb40 tx=0x7f561000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610004b20 con 0x7f5618151f20 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5610004db0 con 0x7f5618151f20 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561814d990 con 0x7f5618151f20 2024-07-17T08:45:24.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.894+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610005280 con 0x7f5618151f20 2024-07-17T08:45:24.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.895+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5618073590 con 0x7f5618151f20 2024-07-17T08:45:24.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.896+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56180cb080 con 0x7f5618151f20 2024-07-17T08:45:24.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.897+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f5610005420 con 0x7f5618151f20 2024-07-17T08:45:24.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.898+0000 7f560f400640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 0x7f55f4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:24.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.898+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f5610099680 con 0x7f5618151f20 2024-07-17T08:45:24.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.899+0000 7f5617400640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 0x7f55f4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:24.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.900+0000 7f5617400640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 0x7f55f4079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5608033000 tx=0x7f5608004730 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:24.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:24.903+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5610062630 con 0x7f5618151f20 2024-07-17T08:45:25.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.154+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f56181f3ec0 con 0x7f5618151f20 2024-07-17T08:45:25.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.655+0000 7f560f400640 1 -- 172.21.15.44:0/497023707 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v209) ==== 122+0+0 (secure 0 0 0) 0x7f5610061f30 con 0x7f5618151f20 2024-07-17T08:45:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 msgr2=0x7f55f4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 0x7f55f4079070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5608033000 tx=0x7f5608004730 comp rx=0 tx=0).stop 2024-07-17T08:45:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 msgr2=0x7f561814cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:25.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f561000eb40 tx=0x7f561000c2d0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 shutdown_connections 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f55f4076bb0 0x7f55f4079070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5618151f20 0x7f561814cfa0 unknown :-1 s=CLOSED pgs=3315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56180cd0b0 0x7f56181f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 --2- 172.21.15.44:0/497023707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f561806f380 0x7f56181f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 >> 172.21.15.44:0/497023707 conn(0x7f56180cf0d0 msgr2=0x7f561815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.658+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 shutdown_connections 2024-07-17T08:45:25.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.659+0000 7f561d000640 1 -- 172.21.15.44:0/497023707 wait complete. 2024-07-17T08:45:25.683 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:45:25.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 -- 172.21.15.44:0/347526463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 msgr2=0x7f82080ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 --2- 172.21.15.44:0/347526463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f82080ac870 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f81f0005fd0 tx=0x7f81f002b910 comp rx=0 tx=0).stop 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 -- 172.21.15.44:0/347526463 shutdown_connections 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 --2- 172.21.15.44:0/347526463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 0x7f820806e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 --2- 172.21.15.44:0/347526463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f82080ac870 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 -- 172.21.15.44:0/347526463 >> 172.21.15.44:0/347526463 conn(0x7f82080b0fb0 msgr2=0x7f82080b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 -- 172.21.15.44:0/347526463 shutdown_connections 2024-07-17T08:45:25.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.892+0000 7f820f200640 1 -- 172.21.15.44:0/347526463 wait complete. 2024-07-17T08:45:25.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.894+0000 7f820f200640 1 Processor -- start 2024-07-17T08:45:25.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.894+0000 7f820f200640 1 -- start start 2024-07-17T08:45:25.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 0x7f82080b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f8208156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8208159c80 con 0x7f820806f710 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8208159b00 con 0x7f82080ac3e0 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f820f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8208159e00 con 0x7f820806e090 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.895+0000 7f81ff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35836/0 (socket says 172.21.15.44:35836) 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 -- 172.21.15.44:0/2675694613 learned_addr learned my addr 172.21.15.44:0/2675694613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f8207e00640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 0x7f82080b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f820d200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f8208156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 -- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 msgr2=0x7f82080b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:25.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 0x7f82080b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 -- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 msgr2=0x7f8208156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f8208156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 -- 172.21.15.44:0/2675694613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8208141390 con 0x7f820806f710 2024-07-17T08:45:25.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f820d200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f8208156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.896+0000 7f81ff400640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.897+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f003e070 con 0x7f820806f710 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.897+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81f0031610 con 0x7f820806f710 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.897+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8208141600 con 0x7f820806f710 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.897+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0031a00 con 0x7f820806f710 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.897+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82080b43a0 con 0x7f820806f710 2024-07-17T08:45:25.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.898+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080adb70 con 0x7f820806f710 2024-07-17T08:45:25.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.899+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f81f0038070 con 0x7f820806f710 2024-07-17T08:45:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.900+0000 7f8204c00640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 0x7f81e40790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.901+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f81f00b95d0 con 0x7f820806f710 2024-07-17T08:45:25.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.901+0000 7f8207e00640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 0x7f81e40790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:25.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.902+0000 7f8207e00640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 0x7f81e40790a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82081e60c0 tx=0x7f81f8009040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:25.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:25.905+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f82080adb70 con 0x7f820806f710 2024-07-17T08:45:26.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.156+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8208159760 con 0x7f820806f710 2024-07-17T08:45:26.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.158+0000 7f8204c00640 1 -- 172.21.15.44:0/2675694613 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7f8208159760 con 0x7f820806f710 2024-07-17T08:45:26.158 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:26.159 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:45:26.159 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 msgr2=0x7f81e40790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 0x7f81e40790a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82081e60c0 tx=0x7f81f8009040 comp rx=0 tx=0).stop 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 msgr2=0x7f8208156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f00049d0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 shutdown_connections 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f81e4076be0 0x7f81e40790a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82080ac3e0 0x7f8208156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f820806f710 0x7f8208156840 unknown :-1 s=CLOSED pgs=3316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 --2- 172.21.15.44:0/2675694613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f820806e090 0x7f82080b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 >> 172.21.15.44:0/2675694613 conn(0x7f82080b0fb0 msgr2=0x7f82081587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 shutdown_connections 2024-07-17T08:45:26.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.161+0000 7f820f200640 1 -- 172.21.15.44:0/2675694613 wait complete. 2024-07-17T08:45:26.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:45:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 -- 172.21.15.44:0/1338262890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1308004e60 msgr2=0x7f130800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 --2- 172.21.15.44:0/1338262890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1308004e60 0x7f130800b9c0 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 crypto rx=0x7f131c001dc0 tx=0x7f1308005830 comp rx=0 tx=0).stop 2024-07-17T08:45:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1322800640 1 -- 172.21.15.44:0/1338262890 reap_dead start 2024-07-17T08:45:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 -- 172.21.15.44:0/1338262890 shutdown_connections 2024-07-17T08:45:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 -- 172.21.15.44:0/1338262890 >> 172.21.15.44:0/1338262890 conn(0x7f13240b3020 msgr2=0x7f13240b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 -- 172.21.15.44:0/1338262890 shutdown_connections 2024-07-17T08:45:26.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.389+0000 7f1329000640 1 -- 172.21.15.44:0/1338262890 wait complete. 2024-07-17T08:45:26.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.391+0000 7f1329000640 1 Processor -- start 2024-07-17T08:45:26.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.391+0000 7f1329000640 1 -- start start 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1308004e60 0x7f132414e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13241f5010 0x7f132406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13240cf8f0 con 0x7f13240b6730 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13240cf770 con 0x7f13241f5010 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.392+0000 7f1329000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13240aea10 con 0x7f1308004e60 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35852/0 (socket says 172.21.15.44:35852) 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 -- 172.21.15.44:0/3964311678 learned_addr learned my addr 172.21.15.44:0/3964311678 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f131be00640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13241f5010 0x7f132406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1321e00640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1308004e60 0x7f132414e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 -- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1308004e60 msgr2=0x7f132414e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1308004e60 0x7f132414e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 -- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13241f5010 msgr2=0x7f132406eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13241f5010 0x7f132406eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 -- 172.21.15.44:0/3964311678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13240b7060 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.393+0000 7f1322800640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 secure :-1 s=READY pgs=3319 cs=0 l=1 rev1=1 crypto rx=0x7f131c001eb0 tx=0x7f131c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.394+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0062b0 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.394+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f131c006450 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.394+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c006880 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.394+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13241f4530 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.394+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13240b6bb0 con 0x7f13240b6730 2024-07-17T08:45:26.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.395+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1324159bb0 con 0x7f13240b6730 2024-07-17T08:45:26.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.396+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f131c003a30 con 0x7f13240b6730 2024-07-17T08:45:26.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.398+0000 7f131a000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 0x7f1300079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.398+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f131c0b82e0 con 0x7f13240b6730 2024-07-17T08:45:26.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.399+0000 7f1321e00640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 0x7f1300079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.400+0000 7f1321e00640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 0x7f1300079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1314005fd0 tx=0x7f131400c500 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:26.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.402+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1324159bb0 con 0x7f13240b6730 2024-07-17T08:45:26.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.615+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13241f4340 con 0x7f13240b6730 2024-07-17T08:45:26.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.617+0000 7f131a000640 1 -- 172.21.15.44:0/3964311678 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v364) ==== 74+0+17907 (secure 0 0 0) 0x7f131c081310 con 0x7f13240b6730 2024-07-17T08:45:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:26.619 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":364,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:45:23.606030+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":49,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:45:20.033990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":50,"pool_name":"cephfs_data","create_time":"2024-07-17T08:45:21.067621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"364","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:45:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 msgr2=0x7f1300079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 0x7f1300079070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1314005fd0 tx=0x7f131400c500 comp rx=0 tx=0).stop 2024-07-17T08:45:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 msgr2=0x7f13240b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 secure :-1 s=READY pgs=3319 cs=0 l=1 rev1=1 crypto rx=0x7f131c001eb0 tx=0x7f131c001b10 comp rx=0 tx=0).stop 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 shutdown_connections 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1300076bb0 0x7f1300079070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13240b6730 0x7f13240b2060 unknown :-1 s=CLOSED pgs=3319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13241f5010 0x7f132406eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 --2- 172.21.15.44:0/3964311678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1308004e60 0x7f132414e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.620+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 >> 172.21.15.44:0/3964311678 conn(0x7f13240b3020 msgr2=0x7f13240ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.621+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 shutdown_connections 2024-07-17T08:45:26.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.621+0000 7f1329000640 1 -- 172.21.15.44:0/3964311678 wait complete. 2024-07-17T08:45:26.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:45:26.641 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:45:26.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 -- 172.21.15.44:0/2835883327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4388003a80 msgr2=0x7f4388003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 --2- 172.21.15.44:0/2835883327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4388003a80 0x7f4388003ea0 secure :-1 s=READY pgs=3321 cs=0 l=1 rev1=1 crypto rx=0x7f4394006f90 tx=0x7f4388005830 comp rx=0 tx=0).stop 2024-07-17T08:45:26.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43a8a00640 1 -- 172.21.15.44:0/2835883327 reap_dead start 2024-07-17T08:45:26.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 -- 172.21.15.44:0/2835883327 shutdown_connections 2024-07-17T08:45:26.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 -- 172.21.15.44:0/2835883327 >> 172.21.15.44:0/2835883327 conn(0x7f43a40b0fb0 msgr2=0x7f43a40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:26.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 -- 172.21.15.44:0/2835883327 shutdown_connections 2024-07-17T08:45:26.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.854+0000 7f43aaa00640 1 -- 172.21.15.44:0/2835883327 wait complete. 2024-07-17T08:45:26.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.856+0000 7f43aaa00640 1 Processor -- start 2024-07-17T08:45:26.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.856+0000 7f43aaa00640 1 -- start start 2024-07-17T08:45:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4388003a80 0x7f43a4145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43a4158200 0x7f43a4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406d800 con 0x7f43a409cc70 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406d680 con 0x7f43a4158200 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.857+0000 7f43aaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43a406d980 con 0x7f4388003a80 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35878/0 (socket says 172.21.15.44:35878) 2024-07-17T08:45:26.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 -- 172.21.15.44:0/3733490110 learned_addr learned my addr 172.21.15.44:0/3733490110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f439b400640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43a4158200 0x7f43a4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a3e00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4388003a80 0x7f43a4145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 -- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4388003a80 msgr2=0x7f43a4145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4388003a80 0x7f43a4145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 -- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43a4158200 msgr2=0x7f43a4156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43a4158200 0x7f43a4156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.858+0000 7f43a8a00640 1 -- 172.21.15.44:0/3733490110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a4157160 con 0x7f43a409cc70 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43a8a00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 secure :-1 s=READY pgs=3322 cs=0 l=1 rev1=1 crypto rx=0x7f43a4070ff0 tx=0x7f439c00a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:26.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c004d20 con 0x7f43a409cc70 2024-07-17T08:45:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f439c00f6d0 con 0x7f43a409cc70 2024-07-17T08:45:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c00fb20 con 0x7f43a409cc70 2024-07-17T08:45:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a4157440 con 0x7f43a409cc70 2024-07-17T08:45:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.859+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a4157a00 con 0x7f43a409cc70 2024-07-17T08:45:26.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.860+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a40b2b70 con 0x7f43a409cc70 2024-07-17T08:45:26.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.861+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f439c00fcc0 con 0x7f43a409cc70 2024-07-17T08:45:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.862+0000 7f43a0c00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 0x7f43800790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.863+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f439c0989b0 con 0x7f43a409cc70 2024-07-17T08:45:26.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.863+0000 7f43a3e00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 0x7f43800790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:26.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.864+0000 7f43a3e00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 0x7f43800790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4394033000 tx=0x7f43a406eb90 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:26.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:26.868+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f439c061a70 con 0x7f43a409cc70 2024-07-17T08:45:27.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.069+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 --> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43a40b27a0 con 0x7f4380076be0 2024-07-17T08:45:27.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.084+0000 7f43a0c00640 1 -- 172.21.15.44:0/3733490110 <== mgr.10067 v2:172.21.15.175:6834/945953093 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252505 (secure 0 0 0) 0x7f43a40b27a0 con 0x7f4380076be0 2024-07-17T08:45:27.089 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:27.091 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 msgr2=0x7f43800790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 0x7f43800790a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4394033000 tx=0x7f43a406eb90 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 msgr2=0x7f43a40993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 secure :-1 s=READY pgs=3322 cs=0 l=1 rev1=1 crypto rx=0x7f43a4070ff0 tx=0x7f439c00a8a0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 shutdown_connections 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4380076be0 0x7f43800790a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43a409cc70 0x7f43a40993a0 unknown :-1 s=CLOSED pgs=3322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43a4158200 0x7f43a4156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.091+0000 7f43aaa00640 1 --2- 172.21.15.44:0/3733490110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4388003a80 0x7f43a4145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.092+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 >> 172.21.15.44:0/3733490110 conn(0x7f43a40b0fb0 msgr2=0x7f43a40b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.092+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 shutdown_connections 2024-07-17T08:45:27.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.092+0000 7f43aaa00640 1 -- 172.21.15.44:0/3733490110 wait complete. 2024-07-17T08:45:27.113 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:45:26.404151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":609,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4704,"ondisk_log_size":4704,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":528416,"kb_used_data":37404,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":754446304,"statfs":{"total":773094113280,"available":772553015296,"internally_reserved":0,"allocated":38301696,"data_stored":21672152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502648368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.820628"},"pg_stats":[{"pgid":"50.3f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654785+0000","last_change":"2024-07-17T08:45:22.631528+0000","last_active":"2024-07-17T08:45:23.654785+0000","last_peered":"2024-07-17T08:45:23.654785+0000","last_clean":"2024-07-17T08:45:23.654785+0000","last_became_active":"2024-07-17T08:45:22.631235+0000","last_became_peered":"2024-07-17T08:45:22.631235+0000","last_unstale":"2024-07-17T08:45:23.654785+0000","last_undegraded":"2024-07-17T08:45:23.654785+0000","last_fullsized":"2024-07-17T08:45:23.654785+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:06.475474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.3e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618622+0000","last_change":"2024-07-17T08:45:22.639458+0000","last_active":"2024-07-17T08:45:23.618622+0000","last_peered":"2024-07-17T08:45:23.618622+0000","last_clean":"2024-07-17T08:45:23.618622+0000","last_became_active":"2024-07-17T08:45:22.639057+0000","last_became_peered":"2024-07-17T08:45:22.639057+0000","last_unstale":"2024-07-17T08:45:23.618622+0000","last_undegraded":"2024-07-17T08:45:23.618622+0000","last_fullsized":"2024-07-17T08:45:23.618622+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:16.507327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.3d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654059+0000","last_change":"2024-07-17T08:45:22.664068+0000","last_active":"2024-07-17T08:45:23.654059+0000","last_peered":"2024-07-17T08:45:23.654059+0000","last_clean":"2024-07-17T08:45:23.654059+0000","last_became_active":"2024-07-17T08:45:22.663446+0000","last_became_peered":"2024-07-17T08:45:22.663446+0000","last_unstale":"2024-07-17T08:45:23.654059+0000","last_undegraded":"2024-07-17T08:45:23.654059+0000","last_fullsized":"2024-07-17T08:45:23.654059+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:08:33.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.3c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656493+0000","last_change":"2024-07-17T08:45:22.657155+0000","last_active":"2024-07-17T08:45:23.656493+0000","last_peered":"2024-07-17T08:45:23.656493+0000","last_clean":"2024-07-17T08:45:23.656493+0000","last_became_active":"2024-07-17T08:45:22.656865+0000","last_became_peered":"2024-07-17T08:45:22.656865+0000","last_unstale":"2024-07-17T08:45:23.656493+0000","last_undegraded":"2024-07-17T08:45:23.656493+0000","last_fullsized":"2024-07-17T08:45:23.656493+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:30.501722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.3b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616486+0000","last_change":"2024-07-17T08:45:22.642391+0000","last_active":"2024-07-17T08:45:23.616486+0000","last_peered":"2024-07-17T08:45:23.616486+0000","last_clean":"2024-07-17T08:45:23.616486+0000","last_became_active":"2024-07-17T08:45:22.642114+0000","last_became_peered":"2024-07-17T08:45:22.642114+0000","last_unstale":"2024-07-17T08:45:23.616486+0000","last_undegraded":"2024-07-17T08:45:23.616486+0000","last_fullsized":"2024-07-17T08:45:23.616486+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:16:38.776401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.3a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615921+0000","last_change":"2024-07-17T08:45:22.655267+0000","last_active":"2024-07-17T08:45:23.615921+0000","last_peered":"2024-07-17T08:45:23.615921+0000","last_clean":"2024-07-17T08:45:23.615921+0000","last_became_active":"2024-07-17T08:45:22.654978+0000","last_became_peered":"2024-07-17T08:45:22.654978+0000","last_unstale":"2024-07-17T08:45:23.615921+0000","last_undegraded":"2024-07-17T08:45:23.615921+0000","last_fullsized":"2024-07-17T08:45:23.615921+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:44.211168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.39","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655195+0000","last_change":"2024-07-17T08:45:22.657425+0000","last_active":"2024-07-17T08:45:23.655195+0000","last_peered":"2024-07-17T08:45:23.655195+0000","last_clean":"2024-07-17T08:45:23.655195+0000","last_became_active":"2024-07-17T08:45:22.657104+0000","last_became_peered":"2024-07-17T08:45:22.657104+0000","last_unstale":"2024-07-17T08:45:23.655195+0000","last_undegraded":"2024-07-17T08:45:23.655195+0000","last_fullsized":"2024-07-17T08:45:23.655195+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:39.917237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.38","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652792+0000","last_change":"2024-07-17T08:45:22.639042+0000","last_active":"2024-07-17T08:45:23.652792+0000","last_peered":"2024-07-17T08:45:23.652792+0000","last_clean":"2024-07-17T08:45:23.652792+0000","last_became_active":"2024-07-17T08:45:22.638208+0000","last_became_peered":"2024-07-17T08:45:22.638208+0000","last_unstale":"2024-07-17T08:45:23.652792+0000","last_undegraded":"2024-07-17T08:45:23.652792+0000","last_fullsized":"2024-07-17T08:45:23.652792+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:11.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.37","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619848+0000","last_change":"2024-07-17T08:45:22.633731+0000","last_active":"2024-07-17T08:45:23.619848+0000","last_peered":"2024-07-17T08:45:23.619848+0000","last_clean":"2024-07-17T08:45:23.619848+0000","last_became_active":"2024-07-17T08:45:22.632976+0000","last_became_peered":"2024-07-17T08:45:22.632976+0000","last_unstale":"2024-07-17T08:45:23.619848+0000","last_undegraded":"2024-07-17T08:45:23.619848+0000","last_fullsized":"2024-07-17T08:45:23.619848+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.676431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.36","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656932+0000","last_change":"2024-07-17T08:45:22.658087+0000","last_active":"2024-07-17T08:45:23.656932+0000","last_peered":"2024-07-17T08:45:23.656932+0000","last_clean":"2024-07-17T08:45:23.656932+0000","last_became_active":"2024-07-17T08:45:22.657816+0000","last_became_peered":"2024-07-17T08:45:22.657816+0000","last_unstale":"2024-07-17T08:45:23.656932+0000","last_undegraded":"2024-07-17T08:45:23.656932+0000","last_fullsized":"2024-07-17T08:45:23.656932+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:16.793058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.35","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619593+0000","last_change":"2024-07-17T08:45:22.629027+0000","last_active":"2024-07-17T08:45:23.619593+0000","last_peered":"2024-07-17T08:45:23.619593+0000","last_clean":"2024-07-17T08:45:23.619593+0000","last_became_active":"2024-07-17T08:45:22.628653+0000","last_became_peered":"2024-07-17T08:45:22.628653+0000","last_unstale":"2024-07-17T08:45:23.619593+0000","last_undegraded":"2024-07-17T08:45:23.619593+0000","last_fullsized":"2024-07-17T08:45:23.619593+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:26.211785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.34","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652038+0000","last_change":"2024-07-17T08:45:22.656268+0000","last_active":"2024-07-17T08:45:23.652038+0000","last_peered":"2024-07-17T08:45:23.652038+0000","last_clean":"2024-07-17T08:45:23.652038+0000","last_became_active":"2024-07-17T08:45:22.655888+0000","last_became_peered":"2024-07-17T08:45:22.655888+0000","last_unstale":"2024-07-17T08:45:23.652038+0000","last_undegraded":"2024-07-17T08:45:23.652038+0000","last_fullsized":"2024-07-17T08:45:23.652038+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:50.822411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.32","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653520+0000","last_change":"2024-07-17T08:45:22.660586+0000","last_active":"2024-07-17T08:45:23.653520+0000","last_peered":"2024-07-17T08:45:23.653520+0000","last_clean":"2024-07-17T08:45:23.653520+0000","last_became_active":"2024-07-17T08:45:22.660248+0000","last_became_peered":"2024-07-17T08:45:22.660248+0000","last_unstale":"2024-07-17T08:45:23.653520+0000","last_undegraded":"2024-07-17T08:45:23.653520+0000","last_fullsized":"2024-07-17T08:45:23.653520+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:59.360458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.31","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655323+0000","last_change":"2024-07-17T08:45:22.659885+0000","last_active":"2024-07-17T08:45:23.655323+0000","last_peered":"2024-07-17T08:45:23.655323+0000","last_clean":"2024-07-17T08:45:23.655323+0000","last_became_active":"2024-07-17T08:45:22.657416+0000","last_became_peered":"2024-07-17T08:45:22.657416+0000","last_unstale":"2024-07-17T08:45:23.655323+0000","last_undegraded":"2024-07-17T08:45:23.655323+0000","last_fullsized":"2024-07-17T08:45:23.655323+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:27:45.840422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.30","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652895+0000","last_change":"2024-07-17T08:45:22.664448+0000","last_active":"2024-07-17T08:45:23.652895+0000","last_peered":"2024-07-17T08:45:23.652895+0000","last_clean":"2024-07-17T08:45:23.652895+0000","last_became_active":"2024-07-17T08:45:22.663662+0000","last_became_peered":"2024-07-17T08:45:22.663662+0000","last_unstale":"2024-07-17T08:45:23.652895+0000","last_undegraded":"2024-07-17T08:45:23.652895+0000","last_fullsized":"2024-07-17T08:45:23.652895+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:01.727218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.2f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619845+0000","last_change":"2024-07-17T08:45:22.633061+0000","last_active":"2024-07-17T08:45:23.619845+0000","last_peered":"2024-07-17T08:45:23.619845+0000","last_clean":"2024-07-17T08:45:23.619845+0000","last_became_active":"2024-07-17T08:45:22.632784+0000","last_became_peered":"2024-07-17T08:45:22.632784+0000","last_unstale":"2024-07-17T08:45:23.619845+0000","last_undegraded":"2024-07-17T08:45:23.619845+0000","last_fullsized":"2024-07-17T08:45:23.619845+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:51:32.726803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.2e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654468+0000","last_change":"2024-07-17T08:45:22.633958+0000","last_active":"2024-07-17T08:45:23.654468+0000","last_peered":"2024-07-17T08:45:23.654468+0000","last_clean":"2024-07-17T08:45:23.654468+0000","last_became_active":"2024-07-17T08:45:22.633334+0000","last_became_peered":"2024-07-17T08:45:22.633334+0000","last_unstale":"2024-07-17T08:45:23.654468+0000","last_undegraded":"2024-07-17T08:45:23.654468+0000","last_fullsized":"2024-07-17T08:45:23.654468+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:17.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.2d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656316+0000","last_change":"2024-07-17T08:45:22.657642+0000","last_active":"2024-07-17T08:45:23.656316+0000","last_peered":"2024-07-17T08:45:23.656316+0000","last_clean":"2024-07-17T08:45:23.656316+0000","last_became_active":"2024-07-17T08:45:22.656772+0000","last_became_peered":"2024-07-17T08:45:22.656772+0000","last_unstale":"2024-07-17T08:45:23.656316+0000","last_undegraded":"2024-07-17T08:45:23.656316+0000","last_fullsized":"2024-07-17T08:45:23.656316+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:47.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.2c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618633+0000","last_change":"2024-07-17T08:45:22.640936+0000","last_active":"2024-07-17T08:45:23.618633+0000","last_peered":"2024-07-17T08:45:23.618633+0000","last_clean":"2024-07-17T08:45:23.618633+0000","last_became_active":"2024-07-17T08:45:22.640539+0000","last_became_peered":"2024-07-17T08:45:22.640539+0000","last_unstale":"2024-07-17T08:45:23.618633+0000","last_undegraded":"2024-07-17T08:45:23.618633+0000","last_fullsized":"2024-07-17T08:45:23.618633+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:30.879504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.2b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656178+0000","last_change":"2024-07-17T08:45:22.657827+0000","last_active":"2024-07-17T08:45:23.656178+0000","last_peered":"2024-07-17T08:45:23.656178+0000","last_clean":"2024-07-17T08:45:23.656178+0000","last_became_active":"2024-07-17T08:45:22.656566+0000","last_became_peered":"2024-07-17T08:45:22.656566+0000","last_unstale":"2024-07-17T08:45:23.656178+0000","last_undegraded":"2024-07-17T08:45:23.656178+0000","last_fullsized":"2024-07-17T08:45:23.656178+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:42:32.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.2a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615902+0000","last_change":"2024-07-17T08:45:22.643825+0000","last_active":"2024-07-17T08:45:23.615902+0000","last_peered":"2024-07-17T08:45:23.615902+0000","last_clean":"2024-07-17T08:45:23.615902+0000","last_became_active":"2024-07-17T08:45:22.643547+0000","last_became_peered":"2024-07-17T08:45:22.643547+0000","last_unstale":"2024-07-17T08:45:23.615902+0000","last_undegraded":"2024-07-17T08:45:23.615902+0000","last_fullsized":"2024-07-17T08:45:23.615902+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:03:28.972866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.29","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653020+0000","last_change":"2024-07-17T08:45:22.619996+0000","last_active":"2024-07-17T08:45:23.653020+0000","last_peered":"2024-07-17T08:45:23.653020+0000","last_clean":"2024-07-17T08:45:23.653020+0000","last_became_active":"2024-07-17T08:45:22.619259+0000","last_became_peered":"2024-07-17T08:45:22.619259+0000","last_unstale":"2024-07-17T08:45:23.653020+0000","last_undegraded":"2024-07-17T08:45:23.653020+0000","last_fullsized":"2024-07-17T08:45:23.653020+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:57.335076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.28","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652905+0000","last_change":"2024-07-17T08:45:22.654143+0000","last_active":"2024-07-17T08:45:23.652905+0000","last_peered":"2024-07-17T08:45:23.652905+0000","last_clean":"2024-07-17T08:45:23.652905+0000","last_became_active":"2024-07-17T08:45:22.651222+0000","last_became_peered":"2024-07-17T08:45:22.651222+0000","last_unstale":"2024-07-17T08:45:23.652905+0000","last_undegraded":"2024-07-17T08:45:23.652905+0000","last_fullsized":"2024-07-17T08:45:23.652905+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:30:02.645610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.27","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655019+0000","last_change":"2024-07-17T08:45:22.633593+0000","last_active":"2024-07-17T08:45:23.655019+0000","last_peered":"2024-07-17T08:45:23.655019+0000","last_clean":"2024-07-17T08:45:23.655019+0000","last_became_active":"2024-07-17T08:45:22.633247+0000","last_became_peered":"2024-07-17T08:45:22.633247+0000","last_unstale":"2024-07-17T08:45:23.655019+0000","last_undegraded":"2024-07-17T08:45:23.655019+0000","last_fullsized":"2024-07-17T08:45:23.655019+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:01.108593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.26","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653798+0000","last_change":"2024-07-17T08:45:22.652224+0000","last_active":"2024-07-17T08:45:23.653798+0000","last_peered":"2024-07-17T08:45:23.653798+0000","last_clean":"2024-07-17T08:45:23.653798+0000","last_became_active":"2024-07-17T08:45:22.651919+0000","last_became_peered":"2024-07-17T08:45:22.651919+0000","last_unstale":"2024-07-17T08:45:23.653798+0000","last_undegraded":"2024-07-17T08:45:23.653798+0000","last_fullsized":"2024-07-17T08:45:23.653798+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:54.286035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.25","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656303+0000","last_change":"2024-07-17T08:45:22.660985+0000","last_active":"2024-07-17T08:45:23.656303+0000","last_peered":"2024-07-17T08:45:23.656303+0000","last_clean":"2024-07-17T08:45:23.656303+0000","last_became_active":"2024-07-17T08:45:22.660006+0000","last_became_peered":"2024-07-17T08:45:22.660006+0000","last_unstale":"2024-07-17T08:45:23.656303+0000","last_undegraded":"2024-07-17T08:45:23.656303+0000","last_fullsized":"2024-07-17T08:45:23.656303+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:15.051353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.24","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653249+0000","last_change":"2024-07-17T08:45:22.636920+0000","last_active":"2024-07-17T08:45:23.653249+0000","last_peered":"2024-07-17T08:45:23.653249+0000","last_clean":"2024-07-17T08:45:23.653249+0000","last_became_active":"2024-07-17T08:45:22.636394+0000","last_became_peered":"2024-07-17T08:45:22.636394+0000","last_unstale":"2024-07-17T08:45:23.653249+0000","last_undegraded":"2024-07-17T08:45:23.653249+0000","last_fullsized":"2024-07-17T08:45:23.653249+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:56:10.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.23","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653735+0000","last_change":"2024-07-17T08:45:22.653165+0000","last_active":"2024-07-17T08:45:23.653735+0000","last_peered":"2024-07-17T08:45:23.653735+0000","last_clean":"2024-07-17T08:45:23.653735+0000","last_became_active":"2024-07-17T08:45:22.652145+0000","last_became_peered":"2024-07-17T08:45:22.652145+0000","last_unstale":"2024-07-17T08:45:23.653735+0000","last_undegraded":"2024-07-17T08:45:23.653735+0000","last_fullsized":"2024-07-17T08:45:23.653735+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:18.423637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.22","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.657182+0000","last_change":"2024-07-17T08:45:22.658584+0000","last_active":"2024-07-17T08:45:23.657182+0000","last_peered":"2024-07-17T08:45:23.657182+0000","last_clean":"2024-07-17T08:45:23.657182+0000","last_became_active":"2024-07-17T08:45:22.658324+0000","last_became_peered":"2024-07-17T08:45:22.658324+0000","last_unstale":"2024-07-17T08:45:23.657182+0000","last_undegraded":"2024-07-17T08:45:23.657182+0000","last_fullsized":"2024-07-17T08:45:23.657182+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:53.075185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.21","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653075+0000","last_change":"2024-07-17T08:45:22.650222+0000","last_active":"2024-07-17T08:45:23.653075+0000","last_peered":"2024-07-17T08:45:23.653075+0000","last_clean":"2024-07-17T08:45:23.653075+0000","last_became_active":"2024-07-17T08:45:22.649935+0000","last_became_peered":"2024-07-17T08:45:22.649935+0000","last_unstale":"2024-07-17T08:45:23.653075+0000","last_undegraded":"2024-07-17T08:45:23.653075+0000","last_fullsized":"2024-07-17T08:45:23.653075+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:42.165788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.20","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.651582+0000","last_change":"2024-07-17T08:45:22.637969+0000","last_active":"2024-07-17T08:45:23.651582+0000","last_peered":"2024-07-17T08:45:23.651582+0000","last_clean":"2024-07-17T08:45:23.651582+0000","last_became_active":"2024-07-17T08:45:22.637665+0000","last_became_peered":"2024-07-17T08:45:22.637665+0000","last_unstale":"2024-07-17T08:45:23.651582+0000","last_undegraded":"2024-07-17T08:45:23.651582+0000","last_fullsized":"2024-07-17T08:45:23.651582+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:18.601816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.1c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652087+0000","last_change":"2024-07-17T08:45:22.645115+0000","last_active":"2024-07-17T08:45:23.652087+0000","last_peered":"2024-07-17T08:45:23.652087+0000","last_clean":"2024-07-17T08:45:23.652087+0000","last_became_active":"2024-07-17T08:45:22.644710+0000","last_became_peered":"2024-07-17T08:45:22.644710+0000","last_unstale":"2024-07-17T08:45:23.652087+0000","last_undegraded":"2024-07-17T08:45:23.652087+0000","last_fullsized":"2024-07-17T08:45:23.652087+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:16.556436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"49.1f","version":"363'4","reported_seq":18,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656776+0000","last_change":"2024-07-17T08:45:21.641994+0000","last_active":"2024-07-17T08:45:23.656776+0000","last_peered":"2024-07-17T08:45:23.656776+0000","last_clean":"2024-07-17T08:45:23.656776+0000","last_became_active":"2024-07-17T08:45:21.641706+0000","last_became_peered":"2024-07-17T08:45:21.641706+0000","last_unstale":"2024-07-17T08:45:23.656776+0000","last_undegraded":"2024-07-17T08:45:23.656776+0000","last_fullsized":"2024-07-17T08:45:23.656776+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:22.752042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.1d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614206+0000","last_change":"2024-07-17T08:45:22.638516+0000","last_active":"2024-07-17T08:45:23.614206+0000","last_peered":"2024-07-17T08:45:23.614206+0000","last_clean":"2024-07-17T08:45:23.614206+0000","last_became_active":"2024-07-17T08:45:22.637323+0000","last_became_peered":"2024-07-17T08:45:22.637323+0000","last_unstale":"2024-07-17T08:45:23.614206+0000","last_undegraded":"2024-07-17T08:45:23.614206+0000","last_fullsized":"2024-07-17T08:45:23.614206+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:53.263330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.1e","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614628+0000","last_change":"2024-07-17T08:45:21.643031+0000","last_active":"2024-07-17T08:45:23.614628+0000","last_peered":"2024-07-17T08:45:23.614628+0000","last_clean":"2024-07-17T08:45:23.614628+0000","last_became_active":"2024-07-17T08:45:21.642689+0000","last_became_peered":"2024-07-17T08:45:21.642689+0000","last_unstale":"2024-07-17T08:45:23.614628+0000","last_undegraded":"2024-07-17T08:45:23.614628+0000","last_fullsized":"2024-07-17T08:45:23.614628+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:01.569671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.1e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.657209+0000","last_change":"2024-07-17T08:45:22.662086+0000","last_active":"2024-07-17T08:45:23.657209+0000","last_peered":"2024-07-17T08:45:23.657209+0000","last_clean":"2024-07-17T08:45:23.657209+0000","last_became_active":"2024-07-17T08:45:22.661700+0000","last_became_peered":"2024-07-17T08:45:22.661700+0000","last_unstale":"2024-07-17T08:45:23.657209+0000","last_undegraded":"2024-07-17T08:45:23.657209+0000","last_fullsized":"2024-07-17T08:45:23.657209+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:02:12.536975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1d","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617125+0000","last_change":"2024-07-17T08:45:21.637434+0000","last_active":"2024-07-17T08:45:23.617125+0000","last_peered":"2024-07-17T08:45:23.617125+0000","last_clean":"2024-07-17T08:45:23.617125+0000","last_became_active":"2024-07-17T08:45:21.637145+0000","last_became_peered":"2024-07-17T08:45:21.637145+0000","last_unstale":"2024-07-17T08:45:23.617125+0000","last_undegraded":"2024-07-17T08:45:23.617125+0000","last_fullsized":"2024-07-17T08:45:23.617125+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:50.611660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.1f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656215+0000","last_change":"2024-07-17T08:45:22.663558+0000","last_active":"2024-07-17T08:45:23.656215+0000","last_peered":"2024-07-17T08:45:23.656215+0000","last_clean":"2024-07-17T08:45:23.656215+0000","last_became_active":"2024-07-17T08:45:22.662162+0000","last_became_peered":"2024-07-17T08:45:22.662162+0000","last_unstale":"2024-07-17T08:45:23.656215+0000","last_undegraded":"2024-07-17T08:45:23.656215+0000","last_fullsized":"2024-07-17T08:45:23.656215+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:05.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1c","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654354+0000","last_change":"2024-07-17T08:45:21.640083+0000","last_active":"2024-07-17T08:45:23.654354+0000","last_peered":"2024-07-17T08:45:23.654354+0000","last_clean":"2024-07-17T08:45:23.654354+0000","last_became_active":"2024-07-17T08:45:21.639747+0000","last_became_peered":"2024-07-17T08:45:21.639747+0000","last_unstale":"2024-07-17T08:45:23.654354+0000","last_undegraded":"2024-07-17T08:45:23.654354+0000","last_fullsized":"2024-07-17T08:45:23.654354+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:17.405904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.18","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655190+0000","last_change":"2024-07-17T08:45:22.652834+0000","last_active":"2024-07-17T08:45:23.655190+0000","last_peered":"2024-07-17T08:45:23.655190+0000","last_clean":"2024-07-17T08:45:23.655190+0000","last_became_active":"2024-07-17T08:45:22.652545+0000","last_became_peered":"2024-07-17T08:45:22.652545+0000","last_unstale":"2024-07-17T08:45:23.655190+0000","last_undegraded":"2024-07-17T08:45:23.655190+0000","last_fullsized":"2024-07-17T08:45:23.655190+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:41.406681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1b","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616903+0000","last_change":"2024-07-17T08:45:21.638119+0000","last_active":"2024-07-17T08:45:23.616903+0000","last_peered":"2024-07-17T08:45:23.616903+0000","last_clean":"2024-07-17T08:45:23.616903+0000","last_became_active":"2024-07-17T08:45:21.636966+0000","last_became_peered":"2024-07-17T08:45:21.636966+0000","last_unstale":"2024-07-17T08:45:23.616903+0000","last_undegraded":"2024-07-17T08:45:23.616903+0000","last_fullsized":"2024-07-17T08:45:23.616903+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:50.351419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.19","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615472+0000","last_change":"2024-07-17T08:45:22.656481+0000","last_active":"2024-07-17T08:45:23.615472+0000","last_peered":"2024-07-17T08:45:23.615472+0000","last_clean":"2024-07-17T08:45:23.615472+0000","last_became_active":"2024-07-17T08:45:22.656182+0000","last_became_peered":"2024-07-17T08:45:22.656182+0000","last_unstale":"2024-07-17T08:45:23.615472+0000","last_undegraded":"2024-07-17T08:45:23.615472+0000","last_fullsized":"2024-07-17T08:45:23.615472+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:41.676545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.1a","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618783+0000","last_change":"2024-07-17T08:45:21.641567+0000","last_active":"2024-07-17T08:45:23.618783+0000","last_peered":"2024-07-17T08:45:23.618783+0000","last_clean":"2024-07-17T08:45:23.618783+0000","last_became_active":"2024-07-17T08:45:21.639194+0000","last_became_peered":"2024-07-17T08:45:21.639194+0000","last_unstale":"2024-07-17T08:45:23.618783+0000","last_undegraded":"2024-07-17T08:45:23.618783+0000","last_fullsized":"2024-07-17T08:45:23.618783+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:50:32.582220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.1a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618639+0000","last_change":"2024-07-17T08:45:22.640834+0000","last_active":"2024-07-17T08:45:23.618639+0000","last_peered":"2024-07-17T08:45:23.618639+0000","last_clean":"2024-07-17T08:45:23.618639+0000","last_became_active":"2024-07-17T08:45:22.640492+0000","last_became_peered":"2024-07-17T08:45:22.640492+0000","last_unstale":"2024-07-17T08:45:23.618639+0000","last_undegraded":"2024-07-17T08:45:23.618639+0000","last_fullsized":"2024-07-17T08:45:23.618639+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:54:58.251751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.19","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613569+0000","last_change":"2024-07-17T08:45:21.614877+0000","last_active":"2024-07-17T08:45:23.613569+0000","last_peered":"2024-07-17T08:45:23.613569+0000","last_clean":"2024-07-17T08:45:23.613569+0000","last_became_active":"2024-07-17T08:45:21.614455+0000","last_became_peered":"2024-07-17T08:45:21.614455+0000","last_unstale":"2024-07-17T08:45:23.613569+0000","last_undegraded":"2024-07-17T08:45:23.613569+0000","last_fullsized":"2024-07-17T08:45:23.613569+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:06.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.1b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616614+0000","last_change":"2024-07-17T08:45:22.637834+0000","last_active":"2024-07-17T08:45:23.616614+0000","last_peered":"2024-07-17T08:45:23.616614+0000","last_clean":"2024-07-17T08:45:23.616614+0000","last_became_active":"2024-07-17T08:45:22.637542+0000","last_became_peered":"2024-07-17T08:45:22.637542+0000","last_unstale":"2024-07-17T08:45:23.616614+0000","last_undegraded":"2024-07-17T08:45:23.616614+0000","last_fullsized":"2024-07-17T08:45:23.616614+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:31.306972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.18","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655489+0000","last_change":"2024-07-17T08:45:21.645361+0000","last_active":"2024-07-17T08:45:23.655489+0000","last_peered":"2024-07-17T08:45:23.655489+0000","last_clean":"2024-07-17T08:45:23.655489+0000","last_became_active":"2024-07-17T08:45:21.645097+0000","last_became_peered":"2024-07-17T08:45:21.645097+0000","last_unstale":"2024-07-17T08:45:23.655489+0000","last_undegraded":"2024-07-17T08:45:23.655489+0000","last_fullsized":"2024-07-17T08:45:23.655489+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:43.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.14","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653012+0000","last_change":"2024-07-17T08:45:22.628784+0000","last_active":"2024-07-17T08:45:23.653012+0000","last_peered":"2024-07-17T08:45:23.653012+0000","last_clean":"2024-07-17T08:45:23.653012+0000","last_became_active":"2024-07-17T08:45:22.628476+0000","last_became_peered":"2024-07-17T08:45:22.628476+0000","last_unstale":"2024-07-17T08:45:23.653012+0000","last_undegraded":"2024-07-17T08:45:23.653012+0000","last_fullsized":"2024-07-17T08:45:23.653012+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:49.607712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.17","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653052+0000","last_change":"2024-07-17T08:45:21.637980+0000","last_active":"2024-07-17T08:45:23.653052+0000","last_peered":"2024-07-17T08:45:23.653052+0000","last_clean":"2024-07-17T08:45:23.653052+0000","last_became_active":"2024-07-17T08:45:21.637562+0000","last_became_peered":"2024-07-17T08:45:21.637562+0000","last_unstale":"2024-07-17T08:45:23.653052+0000","last_undegraded":"2024-07-17T08:45:23.653052+0000","last_fullsized":"2024-07-17T08:45:23.653052+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:06.756223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.15","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653051+0000","last_change":"2024-07-17T08:45:22.630194+0000","last_active":"2024-07-17T08:45:23.653051+0000","last_peered":"2024-07-17T08:45:23.653051+0000","last_clean":"2024-07-17T08:45:23.653051+0000","last_became_active":"2024-07-17T08:45:22.629883+0000","last_became_peered":"2024-07-17T08:45:22.629883+0000","last_unstale":"2024-07-17T08:45:23.653051+0000","last_undegraded":"2024-07-17T08:45:23.653051+0000","last_fullsized":"2024-07-17T08:45:23.653051+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:14.130370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.16","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619095+0000","last_change":"2024-07-17T08:45:21.635754+0000","last_active":"2024-07-17T08:45:23.619095+0000","last_peered":"2024-07-17T08:45:23.619095+0000","last_clean":"2024-07-17T08:45:23.619095+0000","last_became_active":"2024-07-17T08:45:21.635246+0000","last_became_peered":"2024-07-17T08:45:21.635246+0000","last_unstale":"2024-07-17T08:45:23.619095+0000","last_undegraded":"2024-07-17T08:45:23.619095+0000","last_fullsized":"2024-07-17T08:45:23.619095+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:57.694449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.16","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614424+0000","last_change":"2024-07-17T08:45:22.650147+0000","last_active":"2024-07-17T08:45:23.614424+0000","last_peered":"2024-07-17T08:45:23.614424+0000","last_clean":"2024-07-17T08:45:23.614424+0000","last_became_active":"2024-07-17T08:45:22.649867+0000","last_became_peered":"2024-07-17T08:45:22.649867+0000","last_unstale":"2024-07-17T08:45:23.614424+0000","last_undegraded":"2024-07-17T08:45:23.614424+0000","last_fullsized":"2024-07-17T08:45:23.614424+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:17:59.492479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.15","version":"363'3","reported_seq":17,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619477+0000","last_change":"2024-07-17T08:45:21.642749+0000","last_active":"2024-07-17T08:45:23.619477+0000","last_peered":"2024-07-17T08:45:23.619477+0000","last_clean":"2024-07-17T08:45:23.619477+0000","last_became_active":"2024-07-17T08:45:21.642422+0000","last_became_peered":"2024-07-17T08:45:21.642422+0000","last_unstale":"2024-07-17T08:45:23.619477+0000","last_undegraded":"2024-07-17T08:45:23.619477+0000","last_fullsized":"2024-07-17T08:45:23.619477+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:21:51.923298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.17","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.620110+0000","last_change":"2024-07-17T08:45:22.654122+0000","last_active":"2024-07-17T08:45:23.620110+0000","last_peered":"2024-07-17T08:45:23.620110+0000","last_clean":"2024-07-17T08:45:23.620110+0000","last_became_active":"2024-07-17T08:45:22.653830+0000","last_became_peered":"2024-07-17T08:45:22.653830+0000","last_unstale":"2024-07-17T08:45:23.620110+0000","last_undegraded":"2024-07-17T08:45:23.620110+0000","last_fullsized":"2024-07-17T08:45:23.620110+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:04.975275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.14","version":"363'6","reported_seq":20,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613920+0000","last_change":"2024-07-17T08:45:21.634132+0000","last_active":"2024-07-17T08:45:23.613920+0000","last_peered":"2024-07-17T08:45:23.613920+0000","last_clean":"2024-07-17T08:45:23.613920+0000","last_became_active":"2024-07-17T08:45:21.633787+0000","last_became_peered":"2024-07-17T08:45:21.633787+0000","last_unstale":"2024-07-17T08:45:23.613920+0000","last_undegraded":"2024-07-17T08:45:23.613920+0000","last_fullsized":"2024-07-17T08:45:23.613920+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:45:36.558767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.10","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652675+0000","last_change":"2024-07-17T08:45:22.633685+0000","last_active":"2024-07-17T08:45:23.652675+0000","last_peered":"2024-07-17T08:45:23.652675+0000","last_clean":"2024-07-17T08:45:23.652675+0000","last_became_active":"2024-07-17T08:45:22.633382+0000","last_became_peered":"2024-07-17T08:45:22.633382+0000","last_unstale":"2024-07-17T08:45:23.652675+0000","last_undegraded":"2024-07-17T08:45:23.652675+0000","last_fullsized":"2024-07-17T08:45:23.652675+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:33:18.763337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.13","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654166+0000","last_change":"2024-07-17T08:45:21.647706+0000","last_active":"2024-07-17T08:45:23.654166+0000","last_peered":"2024-07-17T08:45:23.654166+0000","last_clean":"2024-07-17T08:45:23.654166+0000","last_became_active":"2024-07-17T08:45:21.647406+0000","last_became_peered":"2024-07-17T08:45:21.647406+0000","last_unstale":"2024-07-17T08:45:23.654166+0000","last_undegraded":"2024-07-17T08:45:23.654166+0000","last_fullsized":"2024-07-17T08:45:23.654166+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:04.580461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.11","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615932+0000","last_change":"2024-07-17T08:45:22.641880+0000","last_active":"2024-07-17T08:45:23.615932+0000","last_peered":"2024-07-17T08:45:23.615932+0000","last_clean":"2024-07-17T08:45:23.615932+0000","last_became_active":"2024-07-17T08:45:22.640870+0000","last_became_peered":"2024-07-17T08:45:22.640870+0000","last_unstale":"2024-07-17T08:45:23.615932+0000","last_undegraded":"2024-07-17T08:45:23.615932+0000","last_fullsized":"2024-07-17T08:45:23.615932+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:36.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.12","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619041+0000","last_change":"2024-07-17T08:45:21.642676+0000","last_active":"2024-07-17T08:45:23.619041+0000","last_peered":"2024-07-17T08:45:23.619041+0000","last_clean":"2024-07-17T08:45:23.619041+0000","last_became_active":"2024-07-17T08:45:21.641889+0000","last_became_peered":"2024-07-17T08:45:21.641889+0000","last_unstale":"2024-07-17T08:45:23.619041+0000","last_undegraded":"2024-07-17T08:45:23.619041+0000","last_fullsized":"2024-07-17T08:45:23.619041+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:42.679901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.12","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654040+0000","last_change":"2024-07-17T08:45:22.651843+0000","last_active":"2024-07-17T08:45:23.654040+0000","last_peered":"2024-07-17T08:45:23.654040+0000","last_clean":"2024-07-17T08:45:23.654040+0000","last_became_active":"2024-07-17T08:45:22.651509+0000","last_became_peered":"2024-07-17T08:45:22.651509+0000","last_unstale":"2024-07-17T08:45:23.654040+0000","last_undegraded":"2024-07-17T08:45:23.654040+0000","last_fullsized":"2024-07-17T08:45:23.654040+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:42.394140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"49.11","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655459+0000","last_change":"2024-07-17T08:45:21.642778+0000","last_active":"2024-07-17T08:45:23.655459+0000","last_peered":"2024-07-17T08:45:23.655459+0000","last_clean":"2024-07-17T08:45:23.655459+0000","last_became_active":"2024-07-17T08:45:21.640468+0000","last_became_peered":"2024-07-17T08:45:21.640468+0000","last_unstale":"2024-07-17T08:45:23.655459+0000","last_undegraded":"2024-07-17T08:45:23.655459+0000","last_fullsized":"2024-07-17T08:45:23.655459+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:08:16.200628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.13","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653395+0000","last_change":"2024-07-17T08:45:22.653465+0000","last_active":"2024-07-17T08:45:23.653395+0000","last_peered":"2024-07-17T08:45:23.653395+0000","last_clean":"2024-07-17T08:45:23.653395+0000","last_became_active":"2024-07-17T08:45:22.652529+0000","last_became_peered":"2024-07-17T08:45:22.652529+0000","last_unstale":"2024-07-17T08:45:23.653395+0000","last_undegraded":"2024-07-17T08:45:23.653395+0000","last_fullsized":"2024-07-17T08:45:23.653395+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:53.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.10","version":"363'8","reported_seq":22,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617714+0000","last_change":"2024-07-17T08:45:21.645972+0000","last_active":"2024-07-17T08:45:23.617714+0000","last_peered":"2024-07-17T08:45:23.617714+0000","last_clean":"2024-07-17T08:45:23.617714+0000","last_became_active":"2024-07-17T08:45:21.641849+0000","last_became_peered":"2024-07-17T08:45:21.641849+0000","last_unstale":"2024-07-17T08:45:23.617714+0000","last_undegraded":"2024-07-17T08:45:23.617714+0000","last_fullsized":"2024-07-17T08:45:23.617714+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:32.339202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617573+0000","last_change":"2024-07-17T08:45:22.643172+0000","last_active":"2024-07-17T08:45:23.617573+0000","last_peered":"2024-07-17T08:45:23.617573+0000","last_clean":"2024-07-17T08:45:23.617573+0000","last_became_active":"2024-07-17T08:45:22.642906+0000","last_became_peered":"2024-07-17T08:45:22.642906+0000","last_unstale":"2024-07-17T08:45:23.617573+0000","last_undegraded":"2024-07-17T08:45:23.617573+0000","last_fullsized":"2024-07-17T08:45:23.617573+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:46.583129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.f","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654865+0000","last_change":"2024-07-17T08:45:21.632440+0000","last_active":"2024-07-17T08:45:23.654865+0000","last_peered":"2024-07-17T08:45:23.654865+0000","last_clean":"2024-07-17T08:45:23.654865+0000","last_became_active":"2024-07-17T08:45:21.631723+0000","last_became_peered":"2024-07-17T08:45:21.631723+0000","last_unstale":"2024-07-17T08:45:23.654865+0000","last_undegraded":"2024-07-17T08:45:23.654865+0000","last_fullsized":"2024-07-17T08:45:23.654865+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:15.767642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654048+0000","last_change":"2024-07-17T08:45:22.652927+0000","last_active":"2024-07-17T08:45:23.654048+0000","last_peered":"2024-07-17T08:45:23.654048+0000","last_clean":"2024-07-17T08:45:23.654048+0000","last_became_active":"2024-07-17T08:45:22.652330+0000","last_became_peered":"2024-07-17T08:45:22.652330+0000","last_unstale":"2024-07-17T08:45:23.654048+0000","last_undegraded":"2024-07-17T08:45:23.654048+0000","last_fullsized":"2024-07-17T08:45:23.654048+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:21.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.e","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654604+0000","last_change":"2024-07-17T08:45:21.641066+0000","last_active":"2024-07-17T08:45:23.654604+0000","last_peered":"2024-07-17T08:45:23.654604+0000","last_clean":"2024-07-17T08:45:23.654604+0000","last_became_active":"2024-07-17T08:45:21.640750+0000","last_became_peered":"2024-07-17T08:45:21.640750+0000","last_unstale":"2024-07-17T08:45:23.654604+0000","last_undegraded":"2024-07-17T08:45:23.654604+0000","last_fullsized":"2024-07-17T08:45:23.654604+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:52.404526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619050+0000","last_change":"2024-07-17T08:45:22.630493+0000","last_active":"2024-07-17T08:45:23.619050+0000","last_peered":"2024-07-17T08:45:23.619050+0000","last_clean":"2024-07-17T08:45:23.619050+0000","last_became_active":"2024-07-17T08:45:22.630073+0000","last_became_peered":"2024-07-17T08:45:22.630073+0000","last_unstale":"2024-07-17T08:45:23.619050+0000","last_undegraded":"2024-07-17T08:45:23.619050+0000","last_fullsized":"2024-07-17T08:45:23.619050+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:55.955383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.d","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654435+0000","last_change":"2024-07-17T08:45:21.634147+0000","last_active":"2024-07-17T08:45:23.654435+0000","last_peered":"2024-07-17T08:45:23.654435+0000","last_clean":"2024-07-17T08:45:23.654435+0000","last_became_active":"2024-07-17T08:45:21.633836+0000","last_became_peered":"2024-07-17T08:45:21.633836+0000","last_unstale":"2024-07-17T08:45:23.654435+0000","last_undegraded":"2024-07-17T08:45:23.654435+0000","last_fullsized":"2024-07-17T08:45:23.654435+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:39.278452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656453+0000","last_change":"2024-07-17T08:45:22.664374+0000","last_active":"2024-07-17T08:45:23.656453+0000","last_peered":"2024-07-17T08:45:23.656453+0000","last_clean":"2024-07-17T08:45:23.656453+0000","last_became_active":"2024-07-17T08:45:22.663284+0000","last_became_peered":"2024-07-17T08:45:22.663284+0000","last_unstale":"2024-07-17T08:45:23.656453+0000","last_undegraded":"2024-07-17T08:45:23.656453+0000","last_fullsized":"2024-07-17T08:45:23.656453+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:03.804506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.c","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653180+0000","last_change":"2024-07-17T08:45:21.638821+0000","last_active":"2024-07-17T08:45:23.653180+0000","last_peered":"2024-07-17T08:45:23.653180+0000","last_clean":"2024-07-17T08:45:23.653180+0000","last_became_active":"2024-07-17T08:45:21.638522+0000","last_became_peered":"2024-07-17T08:45:21.638522+0000","last_unstale":"2024-07-17T08:45:23.653180+0000","last_undegraded":"2024-07-17T08:45:23.653180+0000","last_fullsized":"2024-07-17T08:45:23.653180+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:54.121648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.8","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613206+0000","last_change":"2024-07-17T08:45:22.630047+0000","last_active":"2024-07-17T08:45:23.613206+0000","last_peered":"2024-07-17T08:45:23.613206+0000","last_clean":"2024-07-17T08:45:23.613206+0000","last_became_active":"2024-07-17T08:45:22.629784+0000","last_became_peered":"2024-07-17T08:45:22.629784+0000","last_unstale":"2024-07-17T08:45:23.613206+0000","last_undegraded":"2024-07-17T08:45:23.613206+0000","last_fullsized":"2024-07-17T08:45:23.613206+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:35.049312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.b","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654214+0000","last_change":"2024-07-17T08:45:21.642914+0000","last_active":"2024-07-17T08:45:23.654214+0000","last_peered":"2024-07-17T08:45:23.654214+0000","last_clean":"2024-07-17T08:45:23.654214+0000","last_became_active":"2024-07-17T08:45:21.642626+0000","last_became_peered":"2024-07-17T08:45:21.642626+0000","last_unstale":"2024-07-17T08:45:23.654214+0000","last_undegraded":"2024-07-17T08:45:23.654214+0000","last_fullsized":"2024-07-17T08:45:23.654214+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:01.645849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.9","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616311+0000","last_change":"2024-07-17T08:45:22.641250+0000","last_active":"2024-07-17T08:45:23.616311+0000","last_peered":"2024-07-17T08:45:23.616311+0000","last_clean":"2024-07-17T08:45:23.616311+0000","last_became_active":"2024-07-17T08:45:22.640949+0000","last_became_peered":"2024-07-17T08:45:22.640949+0000","last_unstale":"2024-07-17T08:45:23.616311+0000","last_undegraded":"2024-07-17T08:45:23.616311+0000","last_fullsized":"2024-07-17T08:45:23.616311+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:12.358380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.a","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655973+0000","last_change":"2024-07-17T08:45:21.639279+0000","last_active":"2024-07-17T08:45:23.655973+0000","last_peered":"2024-07-17T08:45:23.655973+0000","last_clean":"2024-07-17T08:45:23.655973+0000","last_became_active":"2024-07-17T08:45:21.638974+0000","last_became_peered":"2024-07-17T08:45:21.638974+0000","last_unstale":"2024-07-17T08:45:23.655973+0000","last_undegraded":"2024-07-17T08:45:23.655973+0000","last_fullsized":"2024-07-17T08:45:23.655973+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:03.435617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618569+0000","last_change":"2024-07-17T08:45:22.643387+0000","last_active":"2024-07-17T08:45:23.618569+0000","last_peered":"2024-07-17T08:45:23.618569+0000","last_clean":"2024-07-17T08:45:23.618569+0000","last_became_active":"2024-07-17T08:45:22.643061+0000","last_became_peered":"2024-07-17T08:45:22.643061+0000","last_unstale":"2024-07-17T08:45:23.618569+0000","last_undegraded":"2024-07-17T08:45:23.618569+0000","last_fullsized":"2024-07-17T08:45:23.618569+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:49.126846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.9","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655604+0000","last_change":"2024-07-17T08:45:21.643685+0000","last_active":"2024-07-17T08:45:23.655604+0000","last_peered":"2024-07-17T08:45:23.655604+0000","last_clean":"2024-07-17T08:45:23.655604+0000","last_became_active":"2024-07-17T08:45:21.640386+0000","last_became_peered":"2024-07-17T08:45:21.640386+0000","last_unstale":"2024-07-17T08:45:23.655604+0000","last_undegraded":"2024-07-17T08:45:23.655604+0000","last_fullsized":"2024-07-17T08:45:23.655604+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:14.380824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653867+0000","last_change":"2024-07-17T08:45:22.650106+0000","last_active":"2024-07-17T08:45:23.653867+0000","last_peered":"2024-07-17T08:45:23.653867+0000","last_clean":"2024-07-17T08:45:23.653867+0000","last_became_active":"2024-07-17T08:45:22.649662+0000","last_became_peered":"2024-07-17T08:45:22.649662+0000","last_unstale":"2024-07-17T08:45:23.653867+0000","last_undegraded":"2024-07-17T08:45:23.653867+0000","last_fullsized":"2024-07-17T08:45:23.653867+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:00.884268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.8","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613234+0000","last_change":"2024-07-17T08:45:21.639063+0000","last_active":"2024-07-17T08:45:23.613234+0000","last_peered":"2024-07-17T08:45:23.613234+0000","last_clean":"2024-07-17T08:45:23.613234+0000","last_became_active":"2024-07-17T08:45:21.638757+0000","last_became_peered":"2024-07-17T08:45:21.638757+0000","last_unstale":"2024-07-17T08:45:23.613234+0000","last_undegraded":"2024-07-17T08:45:23.613234+0000","last_fullsized":"2024-07-17T08:45:23.613234+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:50.088045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.4","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653597+0000","last_change":"2024-07-17T08:45:22.648714+0000","last_active":"2024-07-17T08:45:23.653597+0000","last_peered":"2024-07-17T08:45:23.653597+0000","last_clean":"2024-07-17T08:45:23.653597+0000","last_became_active":"2024-07-17T08:45:22.648325+0000","last_became_peered":"2024-07-17T08:45:22.648325+0000","last_unstale":"2024-07-17T08:45:23.653597+0000","last_undegraded":"2024-07-17T08:45:23.653597+0000","last_fullsized":"2024-07-17T08:45:23.653597+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:26.510615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.7","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654942+0000","last_change":"2024-07-17T08:45:21.640091+0000","last_active":"2024-07-17T08:45:23.654942+0000","last_peered":"2024-07-17T08:45:23.654942+0000","last_clean":"2024-07-17T08:45:23.654942+0000","last_became_active":"2024-07-17T08:45:21.639759+0000","last_became_peered":"2024-07-17T08:45:21.639759+0000","last_unstale":"2024-07-17T08:45:23.654942+0000","last_undegraded":"2024-07-17T08:45:23.654942+0000","last_fullsized":"2024-07-17T08:45:23.654942+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:58.342726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.5","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654445+0000","last_change":"2024-07-17T08:45:22.652657+0000","last_active":"2024-07-17T08:45:23.654445+0000","last_peered":"2024-07-17T08:45:23.654445+0000","last_clean":"2024-07-17T08:45:23.654445+0000","last_became_active":"2024-07-17T08:45:22.651996+0000","last_became_peered":"2024-07-17T08:45:22.651996+0000","last_unstale":"2024-07-17T08:45:23.654445+0000","last_undegraded":"2024-07-17T08:45:23.654445+0000","last_fullsized":"2024-07-17T08:45:23.654445+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:25.393623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.6","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653352+0000","last_change":"2024-07-17T08:45:21.632954+0000","last_active":"2024-07-17T08:45:23.653352+0000","last_peered":"2024-07-17T08:45:23.653352+0000","last_clean":"2024-07-17T08:45:23.653352+0000","last_became_active":"2024-07-17T08:45:21.632276+0000","last_became_peered":"2024-07-17T08:45:21.632276+0000","last_unstale":"2024-07-17T08:45:23.653352+0000","last_undegraded":"2024-07-17T08:45:23.653352+0000","last_fullsized":"2024-07-17T08:45:23.653352+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:14.035627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.6","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614362+0000","last_change":"2024-07-17T08:45:22.650231+0000","last_active":"2024-07-17T08:45:23.614362+0000","last_peered":"2024-07-17T08:45:23.614362+0000","last_clean":"2024-07-17T08:45:23.614362+0000","last_became_active":"2024-07-17T08:45:22.649904+0000","last_became_peered":"2024-07-17T08:45:22.649904+0000","last_unstale":"2024-07-17T08:45:23.614362+0000","last_undegraded":"2024-07-17T08:45:23.614362+0000","last_fullsized":"2024-07-17T08:45:23.614362+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:19.650588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.5","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656592+0000","last_change":"2024-07-17T08:45:21.637158+0000","last_active":"2024-07-17T08:45:23.656592+0000","last_peered":"2024-07-17T08:45:23.656592+0000","last_clean":"2024-07-17T08:45:23.656592+0000","last_became_active":"2024-07-17T08:45:21.636855+0000","last_became_peered":"2024-07-17T08:45:21.636855+0000","last_unstale":"2024-07-17T08:45:23.656592+0000","last_undegraded":"2024-07-17T08:45:23.656592+0000","last_fullsized":"2024-07-17T08:45:23.656592+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:48.607708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.7","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656631+0000","last_change":"2024-07-17T08:45:22.664554+0000","last_active":"2024-07-17T08:45:23.656631+0000","last_peered":"2024-07-17T08:45:23.656631+0000","last_clean":"2024-07-17T08:45:23.656631+0000","last_became_active":"2024-07-17T08:45:22.662937+0000","last_became_peered":"2024-07-17T08:45:22.662937+0000","last_unstale":"2024-07-17T08:45:23.656631+0000","last_undegraded":"2024-07-17T08:45:23.656631+0000","last_fullsized":"2024-07-17T08:45:23.656631+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:29.665356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.4","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654348+0000","last_change":"2024-07-17T08:45:21.640911+0000","last_active":"2024-07-17T08:45:23.654348+0000","last_peered":"2024-07-17T08:45:23.654348+0000","last_clean":"2024-07-17T08:45:23.654348+0000","last_became_active":"2024-07-17T08:45:21.640584+0000","last_became_peered":"2024-07-17T08:45:21.640584+0000","last_unstale":"2024-07-17T08:45:23.654348+0000","last_undegraded":"2024-07-17T08:45:23.654348+0000","last_fullsized":"2024-07-17T08:45:23.654348+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:25.492883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.0","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653355+0000","last_change":"2024-07-17T08:45:22.649110+0000","last_active":"2024-07-17T08:45:23.653355+0000","last_peered":"2024-07-17T08:45:23.653355+0000","last_clean":"2024-07-17T08:45:23.653355+0000","last_became_active":"2024-07-17T08:45:22.647077+0000","last_became_peered":"2024-07-17T08:45:22.647077+0000","last_unstale":"2024-07-17T08:45:23.653355+0000","last_undegraded":"2024-07-17T08:45:23.653355+0000","last_fullsized":"2024-07-17T08:45:23.653355+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:16.785374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.3","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.651902+0000","last_change":"2024-07-17T08:45:21.622862+0000","last_active":"2024-07-17T08:45:23.651902+0000","last_peered":"2024-07-17T08:45:23.651902+0000","last_clean":"2024-07-17T08:45:23.651902+0000","last_became_active":"2024-07-17T08:45:21.621981+0000","last_became_peered":"2024-07-17T08:45:21.621981+0000","last_unstale":"2024-07-17T08:45:23.651902+0000","last_undegraded":"2024-07-17T08:45:23.651902+0000","last_fullsized":"2024-07-17T08:45:23.651902+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:21.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.1","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652055+0000","last_change":"2024-07-17T08:45:22.650698+0000","last_active":"2024-07-17T08:45:23.652055+0000","last_peered":"2024-07-17T08:45:23.652055+0000","last_clean":"2024-07-17T08:45:23.652055+0000","last_became_active":"2024-07-17T08:45:22.650366+0000","last_became_peered":"2024-07-17T08:45:22.650366+0000","last_unstale":"2024-07-17T08:45:23.652055+0000","last_undegraded":"2024-07-17T08:45:23.652055+0000","last_fullsized":"2024-07-17T08:45:23.652055+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:38.371838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"49.2","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613294+0000","last_change":"2024-07-17T08:45:21.630581+0000","last_active":"2024-07-17T08:45:23.613294+0000","last_peered":"2024-07-17T08:45:23.613294+0000","last_clean":"2024-07-17T08:45:23.613294+0000","last_became_active":"2024-07-17T08:45:21.630061+0000","last_became_peered":"2024-07-17T08:45:21.630061+0000","last_unstale":"2024-07-17T08:45:23.613294+0000","last_undegraded":"2024-07-17T08:45:23.613294+0000","last_fullsized":"2024-07-17T08:45:23.613294+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:09.125793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.2","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619447+0000","last_change":"2024-07-17T08:45:22.630652+0000","last_active":"2024-07-17T08:45:23.619447+0000","last_peered":"2024-07-17T08:45:23.619447+0000","last_clean":"2024-07-17T08:45:23.619447+0000","last_became_active":"2024-07-17T08:45:22.629645+0000","last_became_peered":"2024-07-17T08:45:22.629645+0000","last_unstale":"2024-07-17T08:45:23.619447+0000","last_undegraded":"2024-07-17T08:45:23.619447+0000","last_fullsized":"2024-07-17T08:45:23.619447+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:12.287100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.1","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617443+0000","last_change":"2024-07-17T08:45:21.631992+0000","last_active":"2024-07-17T08:45:23.617443+0000","last_peered":"2024-07-17T08:45:23.617443+0000","last_clean":"2024-07-17T08:45:23.617443+0000","last_became_active":"2024-07-17T08:45:21.631695+0000","last_became_peered":"2024-07-17T08:45:21.631695+0000","last_unstale":"2024-07-17T08:45:23.617443+0000","last_undegraded":"2024-07-17T08:45:23.617443+0000","last_fullsized":"2024-07-17T08:45:23.617443+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:18.857381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.3","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613959+0000","last_change":"2024-07-17T08:45:22.618882+0000","last_active":"2024-07-17T08:45:23.613959+0000","last_peered":"2024-07-17T08:45:23.613959+0000","last_clean":"2024-07-17T08:45:23.613959+0000","last_became_active":"2024-07-17T08:45:22.618230+0000","last_became_peered":"2024-07-17T08:45:22.618230+0000","last_unstale":"2024-07-17T08:45:23.613959+0000","last_undegraded":"2024-07-17T08:45:23.613959+0000","last_fullsized":"2024-07-17T08:45:23.613959+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:17.810684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.0","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615603+0000","last_change":"2024-07-17T08:45:21.632273+0000","last_active":"2024-07-17T08:45:23.615603+0000","last_peered":"2024-07-17T08:45:23.615603+0000","last_clean":"2024-07-17T08:45:23.615603+0000","last_became_active":"2024-07-17T08:45:21.631974+0000","last_became_peered":"2024-07-17T08:45:21.631974+0000","last_unstale":"2024-07-17T08:45:23.615603+0000","last_undegraded":"2024-07-17T08:45:23.615603+0000","last_fullsized":"2024-07-17T08:45:23.615603+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:18.397395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.33","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613407+0000","last_change":"2024-07-17T08:45:22.627003+0000","last_active":"2024-07-17T08:45:23.613407+0000","last_peered":"2024-07-17T08:45:23.613407+0000","last_clean":"2024-07-17T08:45:23.613407+0000","last_became_active":"2024-07-17T08:45:22.626150+0000","last_became_peered":"2024-07-17T08:45:22.626150+0000","last_unstale":"2024-07-17T08:45:23.613407+0000","last_undegraded":"2024-07-17T08:45:23.613407+0000","last_fullsized":"2024-07-17T08:45:23.613407+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:14.427357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"355'4666","reported_seq":6131,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655446+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:45:23.655446+0000","last_peered":"2024-07-17T08:45:23.655446+0000","last_clean":"2024-07-17T08:45:23.655446+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:45:23.655446+0000","last_undegraded":"2024-07-17T08:45:23.655446+0000","last_fullsized":"2024-07-17T08:45:23.655446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4666,"log_dups_size":0,"ondisk_log_size":4666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":565,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":50,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":49,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":565,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4666,"ondisk_log_size":4666,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80708,"kb_used_data":5108,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291132,"statfs":{"total":96636764160,"available":96554119168,"internally_reserved":0,"allocated":5230592,"data_stored":3152690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510747,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53756,"kb_used_data":4524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318084,"statfs":{"total":96636764160,"available":96581718016,"internally_reserved":0,"allocated":4632576,"data_stored":2561740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510748,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75976,"kb_used_data":4532,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295864,"statfs":{"total":96636764160,"available":96558964736,"internally_reserved":0,"allocated":4640768,"data_stored":2561844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57936,"kb_used_data":4544,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313904,"statfs":{"total":96636764160,"available":96577437696,"internally_reserved":0,"allocated":4653056,"data_stored":2563476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510747,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75968,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295872,"statfs":{"total":96636764160,"available":96558972928,"internally_reserved":0,"allocated":4636672,"data_stored":2561810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543451,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75980,"kb_used_data":4540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295860,"statfs":{"total":96636764160,"available":96558960640,"internally_reserved":0,"allocated":4648960,"data_stored":2563374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53748,"kb_used_data":4516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318092,"statfs":{"total":96636764160,"available":96581726208,"internally_reserved":0,"allocated":4624384,"data_stored":2554438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510747,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":5112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":5234688,"data_stored":3152780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:45:27.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:45:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 -- 172.21.15.44:0/514975810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7400cd690 msgr2=0x7fc7400cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 --2- 172.21.15.44:0/514975810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7400cd690 0x7fc7400cdae0 secure :-1 s=READY pgs=3323 cs=0 l=1 rev1=1 crypto rx=0x7fc74015c950 tx=0x7fc73802a560 comp rx=0 tx=0).stop 2024-07-17T08:45:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 -- 172.21.15.44:0/514975810 shutdown_connections 2024-07-17T08:45:27.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 --2- 172.21.15.44:0/514975810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7400cd690 0x7fc7400cdae0 unknown :-1 s=CLOSED pgs=3323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 --2- 172.21.15.44:0/514975810 >> v2:172.21.15.175:3300/0 conn(0x7fc7400ac220 0x7fc7400ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 -- 172.21.15.44:0/514975810 >> 172.21.15.44:0/514975810 conn(0x7fc7400b0fb0 msgr2=0x7fc7400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 -- 172.21.15.44:0/514975810 shutdown_connections 2024-07-17T08:45:27.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.363+0000 7fc746400640 1 -- 172.21.15.44:0/514975810 wait complete. 2024-07-17T08:45:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.365+0000 7fc746400640 1 Processor -- start 2024-07-17T08:45:27.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.365+0000 7fc746400640 1 -- start start 2024-07-17T08:45:27.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7400ac220 0x7fc740072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7400cd690 0x7fc740073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc7400ce020 con 0x7fc7401522c0 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7400adf80 con 0x7fc7400ac220 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.366+0000 7fc746400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7400ce1a0 con 0x7fc7400cd690 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35902/0 (socket says 172.21.15.44:35902) 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 -- 172.21.15.44:0/2848610368 learned_addr learned my addr 172.21.15.44:0/2848610368 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73ea00640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7400cd690 0x7fc740073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73f400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7400ac220 0x7fc740072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 -- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7400cd690 msgr2=0x7fc740073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7400cd690 0x7fc740073520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 -- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7400ac220 msgr2=0x7fc740072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7400ac220 0x7fc740072fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 -- 172.21.15.44:0/2848610368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74014f870 con 0x7fc7401522c0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.367+0000 7fc73fe00640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fc738009230 tx=0x7fc738003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.368+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73803f070 con 0x7fc7401522c0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.368+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc738004890 con 0x7fc7401522c0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.368+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7401f2310 con 0x7fc7401522c0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.368+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738035030 con 0x7fc7401522c0 2024-07-17T08:45:27.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.368+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401f1fe0 con 0x7fc7401522c0 2024-07-17T08:45:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.369+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7401f2ac0 con 0x7fc7401522c0 2024-07-17T08:45:27.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.370+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fc738042070 con 0x7fc7401522c0 2024-07-17T08:45:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.371+0000 7fc737400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 0x7fc71c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.371+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7fc7380b9550 con 0x7fc7401522c0 2024-07-17T08:45:27.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.372+0000 7fc73f400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 0x7fc71c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.373+0000 7fc73f400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 0x7fc71c079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730008040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:27.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.375+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc738082640 con 0x7fc7401522c0 2024-07-17T08:45:27.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.583+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 --> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7400ce6c0 con 0x7fc71c076bb0 2024-07-17T08:45:27.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.597+0000 7fc737400640 1 -- 172.21.15.44:0/2848610368 <== mgr.10067 v2:172.21.15.175:6834/945953093 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252505 (secure 0 0 0) 0x7fc7400ce6c0 con 0x7fc71c076bb0 2024-07-17T08:45:27.600 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:27.602 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 msgr2=0x7fc71c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 0x7fc71c079070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc730006fd0 tx=0x7fc730008040 comp rx=0 tx=0).stop 2024-07-17T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 msgr2=0x7fc7401f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fc738009230 tx=0x7fc738003960 comp rx=0 tx=0).stop 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 shutdown_connections 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc71c076bb0 0x7fc71c079070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7401522c0 0x7fc7401f58a0 unknown :-1 s=CLOSED pgs=3324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7400cd690 0x7fc740073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 --2- 172.21.15.44:0/2848610368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7400ac220 0x7fc740072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.603+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 >> 172.21.15.44:0/2848610368 conn(0x7fc7400b0fb0 msgr2=0x7fc7400afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.604+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 shutdown_connections 2024-07-17T08:45:27.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.604+0000 7fc746400640 1 -- 172.21.15.44:0/2848610368 wait complete. 2024-07-17T08:45:27.625 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:45:26.404151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":609,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4704,"ondisk_log_size":4704,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":528416,"kb_used_data":37404,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":754446304,"statfs":{"total":773094113280,"available":772553015296,"internally_reserved":0,"allocated":38301696,"data_stored":21672152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502648368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.820628"},"pg_stats":[{"pgid":"50.3f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654785+0000","last_change":"2024-07-17T08:45:22.631528+0000","last_active":"2024-07-17T08:45:23.654785+0000","last_peered":"2024-07-17T08:45:23.654785+0000","last_clean":"2024-07-17T08:45:23.654785+0000","last_became_active":"2024-07-17T08:45:22.631235+0000","last_became_peered":"2024-07-17T08:45:22.631235+0000","last_unstale":"2024-07-17T08:45:23.654785+0000","last_undegraded":"2024-07-17T08:45:23.654785+0000","last_fullsized":"2024-07-17T08:45:23.654785+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:11:06.475474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.3e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618622+0000","last_change":"2024-07-17T08:45:22.639458+0000","last_active":"2024-07-17T08:45:23.618622+0000","last_peered":"2024-07-17T08:45:23.618622+0000","last_clean":"2024-07-17T08:45:23.618622+0000","last_became_active":"2024-07-17T08:45:22.639057+0000","last_became_peered":"2024-07-17T08:45:22.639057+0000","last_unstale":"2024-07-17T08:45:23.618622+0000","last_undegraded":"2024-07-17T08:45:23.618622+0000","last_fullsized":"2024-07-17T08:45:23.618622+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:49:16.507327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.3d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654059+0000","last_change":"2024-07-17T08:45:22.664068+0000","last_active":"2024-07-17T08:45:23.654059+0000","last_peered":"2024-07-17T08:45:23.654059+0000","last_clean":"2024-07-17T08:45:23.654059+0000","last_became_active":"2024-07-17T08:45:22.663446+0000","last_became_peered":"2024-07-17T08:45:22.663446+0000","last_unstale":"2024-07-17T08:45:23.654059+0000","last_undegraded":"2024-07-17T08:45:23.654059+0000","last_fullsized":"2024-07-17T08:45:23.654059+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:08:33.891865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.3c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656493+0000","last_change":"2024-07-17T08:45:22.657155+0000","last_active":"2024-07-17T08:45:23.656493+0000","last_peered":"2024-07-17T08:45:23.656493+0000","last_clean":"2024-07-17T08:45:23.656493+0000","last_became_active":"2024-07-17T08:45:22.656865+0000","last_became_peered":"2024-07-17T08:45:22.656865+0000","last_unstale":"2024-07-17T08:45:23.656493+0000","last_undegraded":"2024-07-17T08:45:23.656493+0000","last_fullsized":"2024-07-17T08:45:23.656493+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:30.501722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.3b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616486+0000","last_change":"2024-07-17T08:45:22.642391+0000","last_active":"2024-07-17T08:45:23.616486+0000","last_peered":"2024-07-17T08:45:23.616486+0000","last_clean":"2024-07-17T08:45:23.616486+0000","last_became_active":"2024-07-17T08:45:22.642114+0000","last_became_peered":"2024-07-17T08:45:22.642114+0000","last_unstale":"2024-07-17T08:45:23.616486+0000","last_undegraded":"2024-07-17T08:45:23.616486+0000","last_fullsized":"2024-07-17T08:45:23.616486+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:16:38.776401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.3a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615921+0000","last_change":"2024-07-17T08:45:22.655267+0000","last_active":"2024-07-17T08:45:23.615921+0000","last_peered":"2024-07-17T08:45:23.615921+0000","last_clean":"2024-07-17T08:45:23.615921+0000","last_became_active":"2024-07-17T08:45:22.654978+0000","last_became_peered":"2024-07-17T08:45:22.654978+0000","last_unstale":"2024-07-17T08:45:23.615921+0000","last_undegraded":"2024-07-17T08:45:23.615921+0000","last_fullsized":"2024-07-17T08:45:23.615921+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:44.211168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.39","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655195+0000","last_change":"2024-07-17T08:45:22.657425+0000","last_active":"2024-07-17T08:45:23.655195+0000","last_peered":"2024-07-17T08:45:23.655195+0000","last_clean":"2024-07-17T08:45:23.655195+0000","last_became_active":"2024-07-17T08:45:22.657104+0000","last_became_peered":"2024-07-17T08:45:22.657104+0000","last_unstale":"2024-07-17T08:45:23.655195+0000","last_undegraded":"2024-07-17T08:45:23.655195+0000","last_fullsized":"2024-07-17T08:45:23.655195+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:33:39.917237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.38","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652792+0000","last_change":"2024-07-17T08:45:22.639042+0000","last_active":"2024-07-17T08:45:23.652792+0000","last_peered":"2024-07-17T08:45:23.652792+0000","last_clean":"2024-07-17T08:45:23.652792+0000","last_became_active":"2024-07-17T08:45:22.638208+0000","last_became_peered":"2024-07-17T08:45:22.638208+0000","last_unstale":"2024-07-17T08:45:23.652792+0000","last_undegraded":"2024-07-17T08:45:23.652792+0000","last_fullsized":"2024-07-17T08:45:23.652792+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:11.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.37","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619848+0000","last_change":"2024-07-17T08:45:22.633731+0000","last_active":"2024-07-17T08:45:23.619848+0000","last_peered":"2024-07-17T08:45:23.619848+0000","last_clean":"2024-07-17T08:45:23.619848+0000","last_became_active":"2024-07-17T08:45:22.632976+0000","last_became_peered":"2024-07-17T08:45:22.632976+0000","last_unstale":"2024-07-17T08:45:23.619848+0000","last_undegraded":"2024-07-17T08:45:23.619848+0000","last_fullsized":"2024-07-17T08:45:23.619848+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:07.676431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.36","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656932+0000","last_change":"2024-07-17T08:45:22.658087+0000","last_active":"2024-07-17T08:45:23.656932+0000","last_peered":"2024-07-17T08:45:23.656932+0000","last_clean":"2024-07-17T08:45:23.656932+0000","last_became_active":"2024-07-17T08:45:22.657816+0000","last_became_peered":"2024-07-17T08:45:22.657816+0000","last_unstale":"2024-07-17T08:45:23.656932+0000","last_undegraded":"2024-07-17T08:45:23.656932+0000","last_fullsized":"2024-07-17T08:45:23.656932+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:00:16.793058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.35","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619593+0000","last_change":"2024-07-17T08:45:22.629027+0000","last_active":"2024-07-17T08:45:23.619593+0000","last_peered":"2024-07-17T08:45:23.619593+0000","last_clean":"2024-07-17T08:45:23.619593+0000","last_became_active":"2024-07-17T08:45:22.628653+0000","last_became_peered":"2024-07-17T08:45:22.628653+0000","last_unstale":"2024-07-17T08:45:23.619593+0000","last_undegraded":"2024-07-17T08:45:23.619593+0000","last_fullsized":"2024-07-17T08:45:23.619593+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:03:26.211785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.34","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652038+0000","last_change":"2024-07-17T08:45:22.656268+0000","last_active":"2024-07-17T08:45:23.652038+0000","last_peered":"2024-07-17T08:45:23.652038+0000","last_clean":"2024-07-17T08:45:23.652038+0000","last_became_active":"2024-07-17T08:45:22.655888+0000","last_became_peered":"2024-07-17T08:45:22.655888+0000","last_unstale":"2024-07-17T08:45:23.652038+0000","last_undegraded":"2024-07-17T08:45:23.652038+0000","last_fullsized":"2024-07-17T08:45:23.652038+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:07:50.822411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.32","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653520+0000","last_change":"2024-07-17T08:45:22.660586+0000","last_active":"2024-07-17T08:45:23.653520+0000","last_peered":"2024-07-17T08:45:23.653520+0000","last_clean":"2024-07-17T08:45:23.653520+0000","last_became_active":"2024-07-17T08:45:22.660248+0000","last_became_peered":"2024-07-17T08:45:22.660248+0000","last_unstale":"2024-07-17T08:45:23.653520+0000","last_undegraded":"2024-07-17T08:45:23.653520+0000","last_fullsized":"2024-07-17T08:45:23.653520+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:59.360458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.31","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655323+0000","last_change":"2024-07-17T08:45:22.659885+0000","last_active":"2024-07-17T08:45:23.655323+0000","last_peered":"2024-07-17T08:45:23.655323+0000","last_clean":"2024-07-17T08:45:23.655323+0000","last_became_active":"2024-07-17T08:45:22.657416+0000","last_became_peered":"2024-07-17T08:45:22.657416+0000","last_unstale":"2024-07-17T08:45:23.655323+0000","last_undegraded":"2024-07-17T08:45:23.655323+0000","last_fullsized":"2024-07-17T08:45:23.655323+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:27:45.840422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.30","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652895+0000","last_change":"2024-07-17T08:45:22.664448+0000","last_active":"2024-07-17T08:45:23.652895+0000","last_peered":"2024-07-17T08:45:23.652895+0000","last_clean":"2024-07-17T08:45:23.652895+0000","last_became_active":"2024-07-17T08:45:22.663662+0000","last_became_peered":"2024-07-17T08:45:22.663662+0000","last_unstale":"2024-07-17T08:45:23.652895+0000","last_undegraded":"2024-07-17T08:45:23.652895+0000","last_fullsized":"2024-07-17T08:45:23.652895+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:04:01.727218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.2f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619845+0000","last_change":"2024-07-17T08:45:22.633061+0000","last_active":"2024-07-17T08:45:23.619845+0000","last_peered":"2024-07-17T08:45:23.619845+0000","last_clean":"2024-07-17T08:45:23.619845+0000","last_became_active":"2024-07-17T08:45:22.632784+0000","last_became_peered":"2024-07-17T08:45:22.632784+0000","last_unstale":"2024-07-17T08:45:23.619845+0000","last_undegraded":"2024-07-17T08:45:23.619845+0000","last_fullsized":"2024-07-17T08:45:23.619845+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:51:32.726803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.2e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654468+0000","last_change":"2024-07-17T08:45:22.633958+0000","last_active":"2024-07-17T08:45:23.654468+0000","last_peered":"2024-07-17T08:45:23.654468+0000","last_clean":"2024-07-17T08:45:23.654468+0000","last_became_active":"2024-07-17T08:45:22.633334+0000","last_became_peered":"2024-07-17T08:45:22.633334+0000","last_unstale":"2024-07-17T08:45:23.654468+0000","last_undegraded":"2024-07-17T08:45:23.654468+0000","last_fullsized":"2024-07-17T08:45:23.654468+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:27:17.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.2d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656316+0000","last_change":"2024-07-17T08:45:22.657642+0000","last_active":"2024-07-17T08:45:23.656316+0000","last_peered":"2024-07-17T08:45:23.656316+0000","last_clean":"2024-07-17T08:45:23.656316+0000","last_became_active":"2024-07-17T08:45:22.656772+0000","last_became_peered":"2024-07-17T08:45:22.656772+0000","last_unstale":"2024-07-17T08:45:23.656316+0000","last_undegraded":"2024-07-17T08:45:23.656316+0000","last_fullsized":"2024-07-17T08:45:23.656316+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:47.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.2c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618633+0000","last_change":"2024-07-17T08:45:22.640936+0000","last_active":"2024-07-17T08:45:23.618633+0000","last_peered":"2024-07-17T08:45:23.618633+0000","last_clean":"2024-07-17T08:45:23.618633+0000","last_became_active":"2024-07-17T08:45:22.640539+0000","last_became_peered":"2024-07-17T08:45:22.640539+0000","last_unstale":"2024-07-17T08:45:23.618633+0000","last_undegraded":"2024-07-17T08:45:23.618633+0000","last_fullsized":"2024-07-17T08:45:23.618633+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:40:30.879504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.2b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656178+0000","last_change":"2024-07-17T08:45:22.657827+0000","last_active":"2024-07-17T08:45:23.656178+0000","last_peered":"2024-07-17T08:45:23.656178+0000","last_clean":"2024-07-17T08:45:23.656178+0000","last_became_active":"2024-07-17T08:45:22.656566+0000","last_became_peered":"2024-07-17T08:45:22.656566+0000","last_unstale":"2024-07-17T08:45:23.656178+0000","last_undegraded":"2024-07-17T08:45:23.656178+0000","last_fullsized":"2024-07-17T08:45:23.656178+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:42:32.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.2a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615902+0000","last_change":"2024-07-17T08:45:22.643825+0000","last_active":"2024-07-17T08:45:23.615902+0000","last_peered":"2024-07-17T08:45:23.615902+0000","last_clean":"2024-07-17T08:45:23.615902+0000","last_became_active":"2024-07-17T08:45:22.643547+0000","last_became_peered":"2024-07-17T08:45:22.643547+0000","last_unstale":"2024-07-17T08:45:23.615902+0000","last_undegraded":"2024-07-17T08:45:23.615902+0000","last_fullsized":"2024-07-17T08:45:23.615902+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:03:28.972866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.29","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653020+0000","last_change":"2024-07-17T08:45:22.619996+0000","last_active":"2024-07-17T08:45:23.653020+0000","last_peered":"2024-07-17T08:45:23.653020+0000","last_clean":"2024-07-17T08:45:23.653020+0000","last_became_active":"2024-07-17T08:45:22.619259+0000","last_became_peered":"2024-07-17T08:45:22.619259+0000","last_unstale":"2024-07-17T08:45:23.653020+0000","last_undegraded":"2024-07-17T08:45:23.653020+0000","last_fullsized":"2024-07-17T08:45:23.653020+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:17:57.335076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.28","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652905+0000","last_change":"2024-07-17T08:45:22.654143+0000","last_active":"2024-07-17T08:45:23.652905+0000","last_peered":"2024-07-17T08:45:23.652905+0000","last_clean":"2024-07-17T08:45:23.652905+0000","last_became_active":"2024-07-17T08:45:22.651222+0000","last_became_peered":"2024-07-17T08:45:22.651222+0000","last_unstale":"2024-07-17T08:45:23.652905+0000","last_undegraded":"2024-07-17T08:45:23.652905+0000","last_fullsized":"2024-07-17T08:45:23.652905+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:30:02.645610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.27","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655019+0000","last_change":"2024-07-17T08:45:22.633593+0000","last_active":"2024-07-17T08:45:23.655019+0000","last_peered":"2024-07-17T08:45:23.655019+0000","last_clean":"2024-07-17T08:45:23.655019+0000","last_became_active":"2024-07-17T08:45:22.633247+0000","last_became_peered":"2024-07-17T08:45:22.633247+0000","last_unstale":"2024-07-17T08:45:23.655019+0000","last_undegraded":"2024-07-17T08:45:23.655019+0000","last_fullsized":"2024-07-17T08:45:23.655019+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:01.108593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.26","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653798+0000","last_change":"2024-07-17T08:45:22.652224+0000","last_active":"2024-07-17T08:45:23.653798+0000","last_peered":"2024-07-17T08:45:23.653798+0000","last_clean":"2024-07-17T08:45:23.653798+0000","last_became_active":"2024-07-17T08:45:22.651919+0000","last_became_peered":"2024-07-17T08:45:22.651919+0000","last_unstale":"2024-07-17T08:45:23.653798+0000","last_undegraded":"2024-07-17T08:45:23.653798+0000","last_fullsized":"2024-07-17T08:45:23.653798+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:51:54.286035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.25","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656303+0000","last_change":"2024-07-17T08:45:22.660985+0000","last_active":"2024-07-17T08:45:23.656303+0000","last_peered":"2024-07-17T08:45:23.656303+0000","last_clean":"2024-07-17T08:45:23.656303+0000","last_became_active":"2024-07-17T08:45:22.660006+0000","last_became_peered":"2024-07-17T08:45:22.660006+0000","last_unstale":"2024-07-17T08:45:23.656303+0000","last_undegraded":"2024-07-17T08:45:23.656303+0000","last_fullsized":"2024-07-17T08:45:23.656303+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:28:15.051353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.24","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653249+0000","last_change":"2024-07-17T08:45:22.636920+0000","last_active":"2024-07-17T08:45:23.653249+0000","last_peered":"2024-07-17T08:45:23.653249+0000","last_clean":"2024-07-17T08:45:23.653249+0000","last_became_active":"2024-07-17T08:45:22.636394+0000","last_became_peered":"2024-07-17T08:45:22.636394+0000","last_unstale":"2024-07-17T08:45:23.653249+0000","last_undegraded":"2024-07-17T08:45:23.653249+0000","last_fullsized":"2024-07-17T08:45:23.653249+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:56:10.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.23","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653735+0000","last_change":"2024-07-17T08:45:22.653165+0000","last_active":"2024-07-17T08:45:23.653735+0000","last_peered":"2024-07-17T08:45:23.653735+0000","last_clean":"2024-07-17T08:45:23.653735+0000","last_became_active":"2024-07-17T08:45:22.652145+0000","last_became_peered":"2024-07-17T08:45:22.652145+0000","last_unstale":"2024-07-17T08:45:23.653735+0000","last_undegraded":"2024-07-17T08:45:23.653735+0000","last_fullsized":"2024-07-17T08:45:23.653735+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:18.423637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.22","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.657182+0000","last_change":"2024-07-17T08:45:22.658584+0000","last_active":"2024-07-17T08:45:23.657182+0000","last_peered":"2024-07-17T08:45:23.657182+0000","last_clean":"2024-07-17T08:45:23.657182+0000","last_became_active":"2024-07-17T08:45:22.658324+0000","last_became_peered":"2024-07-17T08:45:22.658324+0000","last_unstale":"2024-07-17T08:45:23.657182+0000","last_undegraded":"2024-07-17T08:45:23.657182+0000","last_fullsized":"2024-07-17T08:45:23.657182+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:53.075185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.21","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653075+0000","last_change":"2024-07-17T08:45:22.650222+0000","last_active":"2024-07-17T08:45:23.653075+0000","last_peered":"2024-07-17T08:45:23.653075+0000","last_clean":"2024-07-17T08:45:23.653075+0000","last_became_active":"2024-07-17T08:45:22.649935+0000","last_became_peered":"2024-07-17T08:45:22.649935+0000","last_unstale":"2024-07-17T08:45:23.653075+0000","last_undegraded":"2024-07-17T08:45:23.653075+0000","last_fullsized":"2024-07-17T08:45:23.653075+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:16:42.165788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.20","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.651582+0000","last_change":"2024-07-17T08:45:22.637969+0000","last_active":"2024-07-17T08:45:23.651582+0000","last_peered":"2024-07-17T08:45:23.651582+0000","last_clean":"2024-07-17T08:45:23.651582+0000","last_became_active":"2024-07-17T08:45:22.637665+0000","last_became_peered":"2024-07-17T08:45:22.637665+0000","last_unstale":"2024-07-17T08:45:23.651582+0000","last_undegraded":"2024-07-17T08:45:23.651582+0000","last_fullsized":"2024-07-17T08:45:23.651582+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:18.601816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.1c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652087+0000","last_change":"2024-07-17T08:45:22.645115+0000","last_active":"2024-07-17T08:45:23.652087+0000","last_peered":"2024-07-17T08:45:23.652087+0000","last_clean":"2024-07-17T08:45:23.652087+0000","last_became_active":"2024-07-17T08:45:22.644710+0000","last_became_peered":"2024-07-17T08:45:22.644710+0000","last_unstale":"2024-07-17T08:45:23.652087+0000","last_undegraded":"2024-07-17T08:45:23.652087+0000","last_fullsized":"2024-07-17T08:45:23.652087+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:55:16.556436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"49.1f","version":"363'4","reported_seq":18,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656776+0000","last_change":"2024-07-17T08:45:21.641994+0000","last_active":"2024-07-17T08:45:23.656776+0000","last_peered":"2024-07-17T08:45:23.656776+0000","last_clean":"2024-07-17T08:45:23.656776+0000","last_became_active":"2024-07-17T08:45:21.641706+0000","last_became_peered":"2024-07-17T08:45:21.641706+0000","last_unstale":"2024-07-17T08:45:23.656776+0000","last_undegraded":"2024-07-17T08:45:23.656776+0000","last_fullsized":"2024-07-17T08:45:23.656776+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:19:22.752042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.1d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614206+0000","last_change":"2024-07-17T08:45:22.638516+0000","last_active":"2024-07-17T08:45:23.614206+0000","last_peered":"2024-07-17T08:45:23.614206+0000","last_clean":"2024-07-17T08:45:23.614206+0000","last_became_active":"2024-07-17T08:45:22.637323+0000","last_became_peered":"2024-07-17T08:45:22.637323+0000","last_unstale":"2024-07-17T08:45:23.614206+0000","last_undegraded":"2024-07-17T08:45:23.614206+0000","last_fullsized":"2024-07-17T08:45:23.614206+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:08:53.263330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.1e","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614628+0000","last_change":"2024-07-17T08:45:21.643031+0000","last_active":"2024-07-17T08:45:23.614628+0000","last_peered":"2024-07-17T08:45:23.614628+0000","last_clean":"2024-07-17T08:45:23.614628+0000","last_became_active":"2024-07-17T08:45:21.642689+0000","last_became_peered":"2024-07-17T08:45:21.642689+0000","last_unstale":"2024-07-17T08:45:23.614628+0000","last_undegraded":"2024-07-17T08:45:23.614628+0000","last_fullsized":"2024-07-17T08:45:23.614628+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:01.569671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.1e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.657209+0000","last_change":"2024-07-17T08:45:22.662086+0000","last_active":"2024-07-17T08:45:23.657209+0000","last_peered":"2024-07-17T08:45:23.657209+0000","last_clean":"2024-07-17T08:45:23.657209+0000","last_became_active":"2024-07-17T08:45:22.661700+0000","last_became_peered":"2024-07-17T08:45:22.661700+0000","last_unstale":"2024-07-17T08:45:23.657209+0000","last_undegraded":"2024-07-17T08:45:23.657209+0000","last_fullsized":"2024-07-17T08:45:23.657209+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:02:12.536975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1d","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617125+0000","last_change":"2024-07-17T08:45:21.637434+0000","last_active":"2024-07-17T08:45:23.617125+0000","last_peered":"2024-07-17T08:45:23.617125+0000","last_clean":"2024-07-17T08:45:23.617125+0000","last_became_active":"2024-07-17T08:45:21.637145+0000","last_became_peered":"2024-07-17T08:45:21.637145+0000","last_unstale":"2024-07-17T08:45:23.617125+0000","last_undegraded":"2024-07-17T08:45:23.617125+0000","last_fullsized":"2024-07-17T08:45:23.617125+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:49:50.611660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.1f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656215+0000","last_change":"2024-07-17T08:45:22.663558+0000","last_active":"2024-07-17T08:45:23.656215+0000","last_peered":"2024-07-17T08:45:23.656215+0000","last_clean":"2024-07-17T08:45:23.656215+0000","last_became_active":"2024-07-17T08:45:22.662162+0000","last_became_peered":"2024-07-17T08:45:22.662162+0000","last_unstale":"2024-07-17T08:45:23.656215+0000","last_undegraded":"2024-07-17T08:45:23.656215+0000","last_fullsized":"2024-07-17T08:45:23.656215+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:33:05.372036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1c","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654354+0000","last_change":"2024-07-17T08:45:21.640083+0000","last_active":"2024-07-17T08:45:23.654354+0000","last_peered":"2024-07-17T08:45:23.654354+0000","last_clean":"2024-07-17T08:45:23.654354+0000","last_became_active":"2024-07-17T08:45:21.639747+0000","last_became_peered":"2024-07-17T08:45:21.639747+0000","last_unstale":"2024-07-17T08:45:23.654354+0000","last_undegraded":"2024-07-17T08:45:23.654354+0000","last_fullsized":"2024-07-17T08:45:23.654354+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:06:17.405904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.18","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655190+0000","last_change":"2024-07-17T08:45:22.652834+0000","last_active":"2024-07-17T08:45:23.655190+0000","last_peered":"2024-07-17T08:45:23.655190+0000","last_clean":"2024-07-17T08:45:23.655190+0000","last_became_active":"2024-07-17T08:45:22.652545+0000","last_became_peered":"2024-07-17T08:45:22.652545+0000","last_unstale":"2024-07-17T08:45:23.655190+0000","last_undegraded":"2024-07-17T08:45:23.655190+0000","last_fullsized":"2024-07-17T08:45:23.655190+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:14:41.406681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.1b","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616903+0000","last_change":"2024-07-17T08:45:21.638119+0000","last_active":"2024-07-17T08:45:23.616903+0000","last_peered":"2024-07-17T08:45:23.616903+0000","last_clean":"2024-07-17T08:45:23.616903+0000","last_became_active":"2024-07-17T08:45:21.636966+0000","last_became_peered":"2024-07-17T08:45:21.636966+0000","last_unstale":"2024-07-17T08:45:23.616903+0000","last_undegraded":"2024-07-17T08:45:23.616903+0000","last_fullsized":"2024-07-17T08:45:23.616903+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:31:50.351419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.19","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615472+0000","last_change":"2024-07-17T08:45:22.656481+0000","last_active":"2024-07-17T08:45:23.615472+0000","last_peered":"2024-07-17T08:45:23.615472+0000","last_clean":"2024-07-17T08:45:23.615472+0000","last_became_active":"2024-07-17T08:45:22.656182+0000","last_became_peered":"2024-07-17T08:45:22.656182+0000","last_unstale":"2024-07-17T08:45:23.615472+0000","last_undegraded":"2024-07-17T08:45:23.615472+0000","last_fullsized":"2024-07-17T08:45:23.615472+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:41.676545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.1a","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618783+0000","last_change":"2024-07-17T08:45:21.641567+0000","last_active":"2024-07-17T08:45:23.618783+0000","last_peered":"2024-07-17T08:45:23.618783+0000","last_clean":"2024-07-17T08:45:23.618783+0000","last_became_active":"2024-07-17T08:45:21.639194+0000","last_became_peered":"2024-07-17T08:45:21.639194+0000","last_unstale":"2024-07-17T08:45:23.618783+0000","last_undegraded":"2024-07-17T08:45:23.618783+0000","last_fullsized":"2024-07-17T08:45:23.618783+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:50:32.582220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.1a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618639+0000","last_change":"2024-07-17T08:45:22.640834+0000","last_active":"2024-07-17T08:45:23.618639+0000","last_peered":"2024-07-17T08:45:23.618639+0000","last_clean":"2024-07-17T08:45:23.618639+0000","last_became_active":"2024-07-17T08:45:22.640492+0000","last_became_peered":"2024-07-17T08:45:22.640492+0000","last_unstale":"2024-07-17T08:45:23.618639+0000","last_undegraded":"2024-07-17T08:45:23.618639+0000","last_fullsized":"2024-07-17T08:45:23.618639+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:54:58.251751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.19","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613569+0000","last_change":"2024-07-17T08:45:21.614877+0000","last_active":"2024-07-17T08:45:23.613569+0000","last_peered":"2024-07-17T08:45:23.613569+0000","last_clean":"2024-07-17T08:45:23.613569+0000","last_became_active":"2024-07-17T08:45:21.614455+0000","last_became_peered":"2024-07-17T08:45:21.614455+0000","last_unstale":"2024-07-17T08:45:23.613569+0000","last_undegraded":"2024-07-17T08:45:23.613569+0000","last_fullsized":"2024-07-17T08:45:23.613569+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:58:06.976208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.1b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616614+0000","last_change":"2024-07-17T08:45:22.637834+0000","last_active":"2024-07-17T08:45:23.616614+0000","last_peered":"2024-07-17T08:45:23.616614+0000","last_clean":"2024-07-17T08:45:23.616614+0000","last_became_active":"2024-07-17T08:45:22.637542+0000","last_became_peered":"2024-07-17T08:45:22.637542+0000","last_unstale":"2024-07-17T08:45:23.616614+0000","last_undegraded":"2024-07-17T08:45:23.616614+0000","last_fullsized":"2024-07-17T08:45:23.616614+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:09:31.306972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.18","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655489+0000","last_change":"2024-07-17T08:45:21.645361+0000","last_active":"2024-07-17T08:45:23.655489+0000","last_peered":"2024-07-17T08:45:23.655489+0000","last_clean":"2024-07-17T08:45:23.655489+0000","last_became_active":"2024-07-17T08:45:21.645097+0000","last_became_peered":"2024-07-17T08:45:21.645097+0000","last_unstale":"2024-07-17T08:45:23.655489+0000","last_undegraded":"2024-07-17T08:45:23.655489+0000","last_fullsized":"2024-07-17T08:45:23.655489+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:43.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.14","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653012+0000","last_change":"2024-07-17T08:45:22.628784+0000","last_active":"2024-07-17T08:45:23.653012+0000","last_peered":"2024-07-17T08:45:23.653012+0000","last_clean":"2024-07-17T08:45:23.653012+0000","last_became_active":"2024-07-17T08:45:22.628476+0000","last_became_peered":"2024-07-17T08:45:22.628476+0000","last_unstale":"2024-07-17T08:45:23.653012+0000","last_undegraded":"2024-07-17T08:45:23.653012+0000","last_fullsized":"2024-07-17T08:45:23.653012+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:01:49.607712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.17","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653052+0000","last_change":"2024-07-17T08:45:21.637980+0000","last_active":"2024-07-17T08:45:23.653052+0000","last_peered":"2024-07-17T08:45:23.653052+0000","last_clean":"2024-07-17T08:45:23.653052+0000","last_became_active":"2024-07-17T08:45:21.637562+0000","last_became_peered":"2024-07-17T08:45:21.637562+0000","last_unstale":"2024-07-17T08:45:23.653052+0000","last_undegraded":"2024-07-17T08:45:23.653052+0000","last_fullsized":"2024-07-17T08:45:23.653052+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:06.756223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.15","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653051+0000","last_change":"2024-07-17T08:45:22.630194+0000","last_active":"2024-07-17T08:45:23.653051+0000","last_peered":"2024-07-17T08:45:23.653051+0000","last_clean":"2024-07-17T08:45:23.653051+0000","last_became_active":"2024-07-17T08:45:22.629883+0000","last_became_peered":"2024-07-17T08:45:22.629883+0000","last_unstale":"2024-07-17T08:45:23.653051+0000","last_undegraded":"2024-07-17T08:45:23.653051+0000","last_fullsized":"2024-07-17T08:45:23.653051+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:37:14.130370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.16","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619095+0000","last_change":"2024-07-17T08:45:21.635754+0000","last_active":"2024-07-17T08:45:23.619095+0000","last_peered":"2024-07-17T08:45:23.619095+0000","last_clean":"2024-07-17T08:45:23.619095+0000","last_became_active":"2024-07-17T08:45:21.635246+0000","last_became_peered":"2024-07-17T08:45:21.635246+0000","last_unstale":"2024-07-17T08:45:23.619095+0000","last_undegraded":"2024-07-17T08:45:23.619095+0000","last_fullsized":"2024-07-17T08:45:23.619095+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:57.694449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.16","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614424+0000","last_change":"2024-07-17T08:45:22.650147+0000","last_active":"2024-07-17T08:45:23.614424+0000","last_peered":"2024-07-17T08:45:23.614424+0000","last_clean":"2024-07-17T08:45:23.614424+0000","last_became_active":"2024-07-17T08:45:22.649867+0000","last_became_peered":"2024-07-17T08:45:22.649867+0000","last_unstale":"2024-07-17T08:45:23.614424+0000","last_undegraded":"2024-07-17T08:45:23.614424+0000","last_fullsized":"2024-07-17T08:45:23.614424+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:17:59.492479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.15","version":"363'3","reported_seq":17,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619477+0000","last_change":"2024-07-17T08:45:21.642749+0000","last_active":"2024-07-17T08:45:23.619477+0000","last_peered":"2024-07-17T08:45:23.619477+0000","last_clean":"2024-07-17T08:45:23.619477+0000","last_became_active":"2024-07-17T08:45:21.642422+0000","last_became_peered":"2024-07-17T08:45:21.642422+0000","last_unstale":"2024-07-17T08:45:23.619477+0000","last_undegraded":"2024-07-17T08:45:23.619477+0000","last_fullsized":"2024-07-17T08:45:23.619477+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:21:51.923298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.17","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.620110+0000","last_change":"2024-07-17T08:45:22.654122+0000","last_active":"2024-07-17T08:45:23.620110+0000","last_peered":"2024-07-17T08:45:23.620110+0000","last_clean":"2024-07-17T08:45:23.620110+0000","last_became_active":"2024-07-17T08:45:22.653830+0000","last_became_peered":"2024-07-17T08:45:22.653830+0000","last_unstale":"2024-07-17T08:45:23.620110+0000","last_undegraded":"2024-07-17T08:45:23.620110+0000","last_fullsized":"2024-07-17T08:45:23.620110+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:44:04.975275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.14","version":"363'6","reported_seq":20,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613920+0000","last_change":"2024-07-17T08:45:21.634132+0000","last_active":"2024-07-17T08:45:23.613920+0000","last_peered":"2024-07-17T08:45:23.613920+0000","last_clean":"2024-07-17T08:45:23.613920+0000","last_became_active":"2024-07-17T08:45:21.633787+0000","last_became_peered":"2024-07-17T08:45:21.633787+0000","last_unstale":"2024-07-17T08:45:23.613920+0000","last_undegraded":"2024-07-17T08:45:23.613920+0000","last_fullsized":"2024-07-17T08:45:23.613920+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:45:36.558767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.10","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652675+0000","last_change":"2024-07-17T08:45:22.633685+0000","last_active":"2024-07-17T08:45:23.652675+0000","last_peered":"2024-07-17T08:45:23.652675+0000","last_clean":"2024-07-17T08:45:23.652675+0000","last_became_active":"2024-07-17T08:45:22.633382+0000","last_became_peered":"2024-07-17T08:45:22.633382+0000","last_unstale":"2024-07-17T08:45:23.652675+0000","last_undegraded":"2024-07-17T08:45:23.652675+0000","last_fullsized":"2024-07-17T08:45:23.652675+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:33:18.763337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"49.13","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654166+0000","last_change":"2024-07-17T08:45:21.647706+0000","last_active":"2024-07-17T08:45:23.654166+0000","last_peered":"2024-07-17T08:45:23.654166+0000","last_clean":"2024-07-17T08:45:23.654166+0000","last_became_active":"2024-07-17T08:45:21.647406+0000","last_became_peered":"2024-07-17T08:45:21.647406+0000","last_unstale":"2024-07-17T08:45:23.654166+0000","last_undegraded":"2024-07-17T08:45:23.654166+0000","last_fullsized":"2024-07-17T08:45:23.654166+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:04:04.580461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.11","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615932+0000","last_change":"2024-07-17T08:45:22.641880+0000","last_active":"2024-07-17T08:45:23.615932+0000","last_peered":"2024-07-17T08:45:23.615932+0000","last_clean":"2024-07-17T08:45:23.615932+0000","last_became_active":"2024-07-17T08:45:22.640870+0000","last_became_peered":"2024-07-17T08:45:22.640870+0000","last_unstale":"2024-07-17T08:45:23.615932+0000","last_undegraded":"2024-07-17T08:45:23.615932+0000","last_fullsized":"2024-07-17T08:45:23.615932+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:10:36.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.12","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619041+0000","last_change":"2024-07-17T08:45:21.642676+0000","last_active":"2024-07-17T08:45:23.619041+0000","last_peered":"2024-07-17T08:45:23.619041+0000","last_clean":"2024-07-17T08:45:23.619041+0000","last_became_active":"2024-07-17T08:45:21.641889+0000","last_became_peered":"2024-07-17T08:45:21.641889+0000","last_unstale":"2024-07-17T08:45:23.619041+0000","last_undegraded":"2024-07-17T08:45:23.619041+0000","last_fullsized":"2024-07-17T08:45:23.619041+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:23:42.679901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.12","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654040+0000","last_change":"2024-07-17T08:45:22.651843+0000","last_active":"2024-07-17T08:45:23.654040+0000","last_peered":"2024-07-17T08:45:23.654040+0000","last_clean":"2024-07-17T08:45:23.654040+0000","last_became_active":"2024-07-17T08:45:22.651509+0000","last_became_peered":"2024-07-17T08:45:22.651509+0000","last_unstale":"2024-07-17T08:45:23.654040+0000","last_undegraded":"2024-07-17T08:45:23.654040+0000","last_fullsized":"2024-07-17T08:45:23.654040+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:41:42.394140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"49.11","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655459+0000","last_change":"2024-07-17T08:45:21.642778+0000","last_active":"2024-07-17T08:45:23.655459+0000","last_peered":"2024-07-17T08:45:23.655459+0000","last_clean":"2024-07-17T08:45:23.655459+0000","last_became_active":"2024-07-17T08:45:21.640468+0000","last_became_peered":"2024-07-17T08:45:21.640468+0000","last_unstale":"2024-07-17T08:45:23.655459+0000","last_undegraded":"2024-07-17T08:45:23.655459+0000","last_fullsized":"2024-07-17T08:45:23.655459+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:08:16.200628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.13","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653395+0000","last_change":"2024-07-17T08:45:22.653465+0000","last_active":"2024-07-17T08:45:23.653395+0000","last_peered":"2024-07-17T08:45:23.653395+0000","last_clean":"2024-07-17T08:45:23.653395+0000","last_became_active":"2024-07-17T08:45:22.652529+0000","last_became_peered":"2024-07-17T08:45:22.652529+0000","last_unstale":"2024-07-17T08:45:23.653395+0000","last_undegraded":"2024-07-17T08:45:23.653395+0000","last_fullsized":"2024-07-17T08:45:23.653395+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:53.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.10","version":"363'8","reported_seq":22,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617714+0000","last_change":"2024-07-17T08:45:21.645972+0000","last_active":"2024-07-17T08:45:23.617714+0000","last_peered":"2024-07-17T08:45:23.617714+0000","last_clean":"2024-07-17T08:45:23.617714+0000","last_became_active":"2024-07-17T08:45:21.641849+0000","last_became_peered":"2024-07-17T08:45:21.641849+0000","last_unstale":"2024-07-17T08:45:23.617714+0000","last_undegraded":"2024-07-17T08:45:23.617714+0000","last_fullsized":"2024-07-17T08:45:23.617714+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:32.339202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.c","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617573+0000","last_change":"2024-07-17T08:45:22.643172+0000","last_active":"2024-07-17T08:45:23.617573+0000","last_peered":"2024-07-17T08:45:23.617573+0000","last_clean":"2024-07-17T08:45:23.617573+0000","last_became_active":"2024-07-17T08:45:22.642906+0000","last_became_peered":"2024-07-17T08:45:22.642906+0000","last_unstale":"2024-07-17T08:45:23.617573+0000","last_undegraded":"2024-07-17T08:45:23.617573+0000","last_fullsized":"2024-07-17T08:45:23.617573+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:36:46.583129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.f","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654865+0000","last_change":"2024-07-17T08:45:21.632440+0000","last_active":"2024-07-17T08:45:23.654865+0000","last_peered":"2024-07-17T08:45:23.654865+0000","last_clean":"2024-07-17T08:45:23.654865+0000","last_became_active":"2024-07-17T08:45:21.631723+0000","last_became_peered":"2024-07-17T08:45:21.631723+0000","last_unstale":"2024-07-17T08:45:23.654865+0000","last_undegraded":"2024-07-17T08:45:23.654865+0000","last_fullsized":"2024-07-17T08:45:23.654865+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:15.767642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.d","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654048+0000","last_change":"2024-07-17T08:45:22.652927+0000","last_active":"2024-07-17T08:45:23.654048+0000","last_peered":"2024-07-17T08:45:23.654048+0000","last_clean":"2024-07-17T08:45:23.654048+0000","last_became_active":"2024-07-17T08:45:22.652330+0000","last_became_peered":"2024-07-17T08:45:22.652330+0000","last_unstale":"2024-07-17T08:45:23.654048+0000","last_undegraded":"2024-07-17T08:45:23.654048+0000","last_fullsized":"2024-07-17T08:45:23.654048+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:28:21.513043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.e","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654604+0000","last_change":"2024-07-17T08:45:21.641066+0000","last_active":"2024-07-17T08:45:23.654604+0000","last_peered":"2024-07-17T08:45:23.654604+0000","last_clean":"2024-07-17T08:45:23.654604+0000","last_became_active":"2024-07-17T08:45:21.640750+0000","last_became_peered":"2024-07-17T08:45:21.640750+0000","last_unstale":"2024-07-17T08:45:23.654604+0000","last_undegraded":"2024-07-17T08:45:23.654604+0000","last_fullsized":"2024-07-17T08:45:23.654604+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:52.404526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.e","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619050+0000","last_change":"2024-07-17T08:45:22.630493+0000","last_active":"2024-07-17T08:45:23.619050+0000","last_peered":"2024-07-17T08:45:23.619050+0000","last_clean":"2024-07-17T08:45:23.619050+0000","last_became_active":"2024-07-17T08:45:22.630073+0000","last_became_peered":"2024-07-17T08:45:22.630073+0000","last_unstale":"2024-07-17T08:45:23.619050+0000","last_undegraded":"2024-07-17T08:45:23.619050+0000","last_fullsized":"2024-07-17T08:45:23.619050+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:16:55.955383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.d","version":"363'1","reported_seq":15,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654435+0000","last_change":"2024-07-17T08:45:21.634147+0000","last_active":"2024-07-17T08:45:23.654435+0000","last_peered":"2024-07-17T08:45:23.654435+0000","last_clean":"2024-07-17T08:45:23.654435+0000","last_became_active":"2024-07-17T08:45:21.633836+0000","last_became_peered":"2024-07-17T08:45:21.633836+0000","last_unstale":"2024-07-17T08:45:23.654435+0000","last_undegraded":"2024-07-17T08:45:23.654435+0000","last_fullsized":"2024-07-17T08:45:23.654435+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:32:39.278452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.f","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656453+0000","last_change":"2024-07-17T08:45:22.664374+0000","last_active":"2024-07-17T08:45:23.656453+0000","last_peered":"2024-07-17T08:45:23.656453+0000","last_clean":"2024-07-17T08:45:23.656453+0000","last_became_active":"2024-07-17T08:45:22.663284+0000","last_became_peered":"2024-07-17T08:45:22.663284+0000","last_unstale":"2024-07-17T08:45:23.656453+0000","last_undegraded":"2024-07-17T08:45:23.656453+0000","last_fullsized":"2024-07-17T08:45:23.656453+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:03.804506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.c","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653180+0000","last_change":"2024-07-17T08:45:21.638821+0000","last_active":"2024-07-17T08:45:23.653180+0000","last_peered":"2024-07-17T08:45:23.653180+0000","last_clean":"2024-07-17T08:45:23.653180+0000","last_became_active":"2024-07-17T08:45:21.638522+0000","last_became_peered":"2024-07-17T08:45:21.638522+0000","last_unstale":"2024-07-17T08:45:23.653180+0000","last_undegraded":"2024-07-17T08:45:23.653180+0000","last_fullsized":"2024-07-17T08:45:23.653180+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:38:54.121648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.8","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613206+0000","last_change":"2024-07-17T08:45:22.630047+0000","last_active":"2024-07-17T08:45:23.613206+0000","last_peered":"2024-07-17T08:45:23.613206+0000","last_clean":"2024-07-17T08:45:23.613206+0000","last_became_active":"2024-07-17T08:45:22.629784+0000","last_became_peered":"2024-07-17T08:45:22.629784+0000","last_unstale":"2024-07-17T08:45:23.613206+0000","last_undegraded":"2024-07-17T08:45:23.613206+0000","last_fullsized":"2024-07-17T08:45:23.613206+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:37:35.049312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.b","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654214+0000","last_change":"2024-07-17T08:45:21.642914+0000","last_active":"2024-07-17T08:45:23.654214+0000","last_peered":"2024-07-17T08:45:23.654214+0000","last_clean":"2024-07-17T08:45:23.654214+0000","last_became_active":"2024-07-17T08:45:21.642626+0000","last_became_peered":"2024-07-17T08:45:21.642626+0000","last_unstale":"2024-07-17T08:45:23.654214+0000","last_undegraded":"2024-07-17T08:45:23.654214+0000","last_fullsized":"2024-07-17T08:45:23.654214+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:48:01.645849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"50.9","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.616311+0000","last_change":"2024-07-17T08:45:22.641250+0000","last_active":"2024-07-17T08:45:23.616311+0000","last_peered":"2024-07-17T08:45:23.616311+0000","last_clean":"2024-07-17T08:45:23.616311+0000","last_became_active":"2024-07-17T08:45:22.640949+0000","last_became_peered":"2024-07-17T08:45:22.640949+0000","last_unstale":"2024-07-17T08:45:23.616311+0000","last_undegraded":"2024-07-17T08:45:23.616311+0000","last_fullsized":"2024-07-17T08:45:23.616311+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:12.358380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"49.a","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655973+0000","last_change":"2024-07-17T08:45:21.639279+0000","last_active":"2024-07-17T08:45:23.655973+0000","last_peered":"2024-07-17T08:45:23.655973+0000","last_clean":"2024-07-17T08:45:23.655973+0000","last_became_active":"2024-07-17T08:45:21.638974+0000","last_became_peered":"2024-07-17T08:45:21.638974+0000","last_unstale":"2024-07-17T08:45:23.655973+0000","last_undegraded":"2024-07-17T08:45:23.655973+0000","last_fullsized":"2024-07-17T08:45:23.655973+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:03.435617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.a","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.618569+0000","last_change":"2024-07-17T08:45:22.643387+0000","last_active":"2024-07-17T08:45:23.618569+0000","last_peered":"2024-07-17T08:45:23.618569+0000","last_clean":"2024-07-17T08:45:23.618569+0000","last_became_active":"2024-07-17T08:45:22.643061+0000","last_became_peered":"2024-07-17T08:45:22.643061+0000","last_unstale":"2024-07-17T08:45:23.618569+0000","last_undegraded":"2024-07-17T08:45:23.618569+0000","last_fullsized":"2024-07-17T08:45:23.618569+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:52:49.126846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.9","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655604+0000","last_change":"2024-07-17T08:45:21.643685+0000","last_active":"2024-07-17T08:45:23.655604+0000","last_peered":"2024-07-17T08:45:23.655604+0000","last_clean":"2024-07-17T08:45:23.655604+0000","last_became_active":"2024-07-17T08:45:21.640386+0000","last_became_peered":"2024-07-17T08:45:21.640386+0000","last_unstale":"2024-07-17T08:45:23.655604+0000","last_undegraded":"2024-07-17T08:45:23.655604+0000","last_fullsized":"2024-07-17T08:45:23.655604+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:13:14.380824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.b","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653867+0000","last_change":"2024-07-17T08:45:22.650106+0000","last_active":"2024-07-17T08:45:23.653867+0000","last_peered":"2024-07-17T08:45:23.653867+0000","last_clean":"2024-07-17T08:45:23.653867+0000","last_became_active":"2024-07-17T08:45:22.649662+0000","last_became_peered":"2024-07-17T08:45:22.649662+0000","last_unstale":"2024-07-17T08:45:23.653867+0000","last_undegraded":"2024-07-17T08:45:23.653867+0000","last_fullsized":"2024-07-17T08:45:23.653867+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:00.884268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.8","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613234+0000","last_change":"2024-07-17T08:45:21.639063+0000","last_active":"2024-07-17T08:45:23.613234+0000","last_peered":"2024-07-17T08:45:23.613234+0000","last_clean":"2024-07-17T08:45:23.613234+0000","last_became_active":"2024-07-17T08:45:21.638757+0000","last_became_peered":"2024-07-17T08:45:21.638757+0000","last_unstale":"2024-07-17T08:45:23.613234+0000","last_undegraded":"2024-07-17T08:45:23.613234+0000","last_fullsized":"2024-07-17T08:45:23.613234+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:05:50.088045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.4","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653597+0000","last_change":"2024-07-17T08:45:22.648714+0000","last_active":"2024-07-17T08:45:23.653597+0000","last_peered":"2024-07-17T08:45:23.653597+0000","last_clean":"2024-07-17T08:45:23.653597+0000","last_became_active":"2024-07-17T08:45:22.648325+0000","last_became_peered":"2024-07-17T08:45:22.648325+0000","last_unstale":"2024-07-17T08:45:23.653597+0000","last_undegraded":"2024-07-17T08:45:23.653597+0000","last_fullsized":"2024-07-17T08:45:23.653597+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:26.510615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.7","version":"363'2","reported_seq":16,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654942+0000","last_change":"2024-07-17T08:45:21.640091+0000","last_active":"2024-07-17T08:45:23.654942+0000","last_peered":"2024-07-17T08:45:23.654942+0000","last_clean":"2024-07-17T08:45:23.654942+0000","last_became_active":"2024-07-17T08:45:21.639759+0000","last_became_peered":"2024-07-17T08:45:21.639759+0000","last_unstale":"2024-07-17T08:45:23.654942+0000","last_undegraded":"2024-07-17T08:45:23.654942+0000","last_fullsized":"2024-07-17T08:45:23.654942+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:47:58.342726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.5","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654445+0000","last_change":"2024-07-17T08:45:22.652657+0000","last_active":"2024-07-17T08:45:23.654445+0000","last_peered":"2024-07-17T08:45:23.654445+0000","last_clean":"2024-07-17T08:45:23.654445+0000","last_became_active":"2024-07-17T08:45:22.651996+0000","last_became_peered":"2024-07-17T08:45:22.651996+0000","last_unstale":"2024-07-17T08:45:23.654445+0000","last_undegraded":"2024-07-17T08:45:23.654445+0000","last_fullsized":"2024-07-17T08:45:23.654445+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:08:25.393623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.6","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653352+0000","last_change":"2024-07-17T08:45:21.632954+0000","last_active":"2024-07-17T08:45:23.653352+0000","last_peered":"2024-07-17T08:45:23.653352+0000","last_clean":"2024-07-17T08:45:23.653352+0000","last_became_active":"2024-07-17T08:45:21.632276+0000","last_became_peered":"2024-07-17T08:45:21.632276+0000","last_unstale":"2024-07-17T08:45:23.653352+0000","last_undegraded":"2024-07-17T08:45:23.653352+0000","last_fullsized":"2024-07-17T08:45:23.653352+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:21:14.035627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"50.6","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.614362+0000","last_change":"2024-07-17T08:45:22.650231+0000","last_active":"2024-07-17T08:45:23.614362+0000","last_peered":"2024-07-17T08:45:23.614362+0000","last_clean":"2024-07-17T08:45:23.614362+0000","last_became_active":"2024-07-17T08:45:22.649904+0000","last_became_peered":"2024-07-17T08:45:22.649904+0000","last_unstale":"2024-07-17T08:45:23.614362+0000","last_undegraded":"2024-07-17T08:45:23.614362+0000","last_fullsized":"2024-07-17T08:45:23.614362+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:07:19.650588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.5","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656592+0000","last_change":"2024-07-17T08:45:21.637158+0000","last_active":"2024-07-17T08:45:23.656592+0000","last_peered":"2024-07-17T08:45:23.656592+0000","last_clean":"2024-07-17T08:45:23.656592+0000","last_became_active":"2024-07-17T08:45:21.636855+0000","last_became_peered":"2024-07-17T08:45:21.636855+0000","last_unstale":"2024-07-17T08:45:23.656592+0000","last_undegraded":"2024-07-17T08:45:23.656592+0000","last_fullsized":"2024-07-17T08:45:23.656592+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:48:48.607708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"50.7","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.656631+0000","last_change":"2024-07-17T08:45:22.664554+0000","last_active":"2024-07-17T08:45:23.656631+0000","last_peered":"2024-07-17T08:45:23.656631+0000","last_clean":"2024-07-17T08:45:23.656631+0000","last_became_active":"2024-07-17T08:45:22.662937+0000","last_became_peered":"2024-07-17T08:45:22.662937+0000","last_unstale":"2024-07-17T08:45:23.656631+0000","last_undegraded":"2024-07-17T08:45:23.656631+0000","last_fullsized":"2024-07-17T08:45:23.656631+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:34:29.665356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"49.4","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.654348+0000","last_change":"2024-07-17T08:45:21.640911+0000","last_active":"2024-07-17T08:45:23.654348+0000","last_peered":"2024-07-17T08:45:23.654348+0000","last_clean":"2024-07-17T08:45:23.654348+0000","last_became_active":"2024-07-17T08:45:21.640584+0000","last_became_peered":"2024-07-17T08:45:21.640584+0000","last_unstale":"2024-07-17T08:45:23.654348+0000","last_undegraded":"2024-07-17T08:45:23.654348+0000","last_fullsized":"2024-07-17T08:45:23.654348+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:53:25.492883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"50.0","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.653355+0000","last_change":"2024-07-17T08:45:22.649110+0000","last_active":"2024-07-17T08:45:23.653355+0000","last_peered":"2024-07-17T08:45:23.653355+0000","last_clean":"2024-07-17T08:45:23.653355+0000","last_became_active":"2024-07-17T08:45:22.647077+0000","last_became_peered":"2024-07-17T08:45:22.647077+0000","last_unstale":"2024-07-17T08:45:23.653355+0000","last_undegraded":"2024-07-17T08:45:23.653355+0000","last_fullsized":"2024-07-17T08:45:23.653355+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:29:16.785374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"49.3","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.651902+0000","last_change":"2024-07-17T08:45:21.622862+0000","last_active":"2024-07-17T08:45:23.651902+0000","last_peered":"2024-07-17T08:45:23.651902+0000","last_clean":"2024-07-17T08:45:23.651902+0000","last_became_active":"2024-07-17T08:45:21.621981+0000","last_became_peered":"2024-07-17T08:45:21.621981+0000","last_unstale":"2024-07-17T08:45:23.651902+0000","last_undegraded":"2024-07-17T08:45:23.651902+0000","last_fullsized":"2024-07-17T08:45:23.651902+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:48:21.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"50.1","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.652055+0000","last_change":"2024-07-17T08:45:22.650698+0000","last_active":"2024-07-17T08:45:23.652055+0000","last_peered":"2024-07-17T08:45:23.652055+0000","last_clean":"2024-07-17T08:45:23.652055+0000","last_became_active":"2024-07-17T08:45:22.650366+0000","last_became_peered":"2024-07-17T08:45:22.650366+0000","last_unstale":"2024-07-17T08:45:23.652055+0000","last_undegraded":"2024-07-17T08:45:23.652055+0000","last_fullsized":"2024-07-17T08:45:23.652055+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:38.371838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"49.2","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613294+0000","last_change":"2024-07-17T08:45:21.630581+0000","last_active":"2024-07-17T08:45:23.613294+0000","last_peered":"2024-07-17T08:45:23.613294+0000","last_clean":"2024-07-17T08:45:23.613294+0000","last_became_active":"2024-07-17T08:45:21.630061+0000","last_became_peered":"2024-07-17T08:45:21.630061+0000","last_unstale":"2024-07-17T08:45:23.613294+0000","last_undegraded":"2024-07-17T08:45:23.613294+0000","last_fullsized":"2024-07-17T08:45:23.613294+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:09.125793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"50.2","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.619447+0000","last_change":"2024-07-17T08:45:22.630652+0000","last_active":"2024-07-17T08:45:23.619447+0000","last_peered":"2024-07-17T08:45:23.619447+0000","last_clean":"2024-07-17T08:45:23.619447+0000","last_became_active":"2024-07-17T08:45:22.629645+0000","last_became_peered":"2024-07-17T08:45:22.629645+0000","last_unstale":"2024-07-17T08:45:23.619447+0000","last_undegraded":"2024-07-17T08:45:23.619447+0000","last_fullsized":"2024-07-17T08:45:23.619447+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:12.287100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"49.1","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.617443+0000","last_change":"2024-07-17T08:45:21.631992+0000","last_active":"2024-07-17T08:45:23.617443+0000","last_peered":"2024-07-17T08:45:23.617443+0000","last_clean":"2024-07-17T08:45:23.617443+0000","last_became_active":"2024-07-17T08:45:21.631695+0000","last_became_peered":"2024-07-17T08:45:21.631695+0000","last_unstale":"2024-07-17T08:45:23.617443+0000","last_undegraded":"2024-07-17T08:45:23.617443+0000","last_fullsized":"2024-07-17T08:45:23.617443+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:37:18.857381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"50.3","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613959+0000","last_change":"2024-07-17T08:45:22.618882+0000","last_active":"2024-07-17T08:45:23.613959+0000","last_peered":"2024-07-17T08:45:23.613959+0000","last_clean":"2024-07-17T08:45:23.613959+0000","last_became_active":"2024-07-17T08:45:22.618230+0000","last_became_peered":"2024-07-17T08:45:22.618230+0000","last_unstale":"2024-07-17T08:45:23.613959+0000","last_undegraded":"2024-07-17T08:45:23.613959+0000","last_fullsized":"2024-07-17T08:45:23.613959+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:20:17.810684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"49.0","version":"0'0","reported_seq":14,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.615603+0000","last_change":"2024-07-17T08:45:21.632273+0000","last_active":"2024-07-17T08:45:23.615603+0000","last_peered":"2024-07-17T08:45:23.615603+0000","last_clean":"2024-07-17T08:45:23.615603+0000","last_became_active":"2024-07-17T08:45:21.631974+0000","last_became_peered":"2024-07-17T08:45:21.631974+0000","last_unstale":"2024-07-17T08:45:23.615603+0000","last_undegraded":"2024-07-17T08:45:23.615603+0000","last_fullsized":"2024-07-17T08:45:23.615603+0000","mapping_epoch":361,"log_start":"0'0","ondisk_log_start":"0'0","created":361,"last_epoch_clean":362,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:20.563998+0000","last_clean_scrub_stamp":"2024-07-17T08:45:20.563998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:19:18.397395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"50.33","version":"0'0","reported_seq":12,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.613407+0000","last_change":"2024-07-17T08:45:22.627003+0000","last_active":"2024-07-17T08:45:23.613407+0000","last_peered":"2024-07-17T08:45:23.613407+0000","last_clean":"2024-07-17T08:45:23.613407+0000","last_became_active":"2024-07-17T08:45:22.626150+0000","last_became_peered":"2024-07-17T08:45:22.626150+0000","last_unstale":"2024-07-17T08:45:23.613407+0000","last_undegraded":"2024-07-17T08:45:23.613407+0000","last_fullsized":"2024-07-17T08:45:23.613407+0000","mapping_epoch":362,"log_start":"0'0","ondisk_log_start":"0'0","created":362,"last_epoch_clean":363,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:45:21.581731+0000","last_clean_scrub_stamp":"2024-07-17T08:45:21.581731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:23:14.427357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"355'4666","reported_seq":6131,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-07-17T08:45:23.655446+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:45:23.655446+0000","last_peered":"2024-07-17T08:45:23.655446+0000","last_clean":"2024-07-17T08:45:23.655446+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:45:23.655446+0000","last_undegraded":"2024-07-17T08:45:23.655446+0000","last_fullsized":"2024-07-17T08:45:23.655446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4666,"log_dups_size":0,"ondisk_log_size":4666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":565,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":50,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":49,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2449,"num_read_kb":6966,"num_write":565,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4666,"ondisk_log_size":4666,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80708,"kb_used_data":5108,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291132,"statfs":{"total":96636764160,"available":96554119168,"internally_reserved":0,"allocated":5230592,"data_stored":3152690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510747,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53756,"kb_used_data":4524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318084,"statfs":{"total":96636764160,"available":96581718016,"internally_reserved":0,"allocated":4632576,"data_stored":2561740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510748,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75976,"kb_used_data":4532,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295864,"statfs":{"total":96636764160,"available":96558964736,"internally_reserved":0,"allocated":4640768,"data_stored":2561844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57936,"kb_used_data":4544,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313904,"statfs":{"total":96636764160,"available":96577437696,"internally_reserved":0,"allocated":4653056,"data_stored":2563476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510747,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75968,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295872,"statfs":{"total":96636764160,"available":96558972928,"internally_reserved":0,"allocated":4636672,"data_stored":2561810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543451,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75980,"kb_used_data":4540,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295860,"statfs":{"total":96636764160,"available":96558960640,"internally_reserved":0,"allocated":4648960,"data_stored":2563374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53748,"kb_used_data":4516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318092,"statfs":{"total":96636764160,"available":96581726208,"internally_reserved":0,"allocated":4624384,"data_stored":2554438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510747,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":5112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":5234688,"data_stored":3152780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":49,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":50,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:45:27.628 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:45:27.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 -- 172.21.15.44:0/4157951976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e70004120 msgr2=0x7f1e70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 --2- 172.21.15.44:0/4157951976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e70004120 0x7f1e70004800 secure :-1 s=READY pgs=3326 cs=0 l=1 rev1=1 crypto rx=0x7f1e780016c0 tx=0x7f1e70005830 comp rx=0 tx=0).stop 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8a800640 1 -- 172.21.15.44:0/4157951976 reap_dead start 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 -- 172.21.15.44:0/4157951976 shutdown_connections 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 -- 172.21.15.44:0/4157951976 >> 172.21.15.44:0/4157951976 conn(0x7f1e8c0b5560 msgr2=0x7f1e8c0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 -- 172.21.15.44:0/4157951976 shutdown_connections 2024-07-17T08:45:27.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.895+0000 7f1e8b200640 1 -- 172.21.15.44:0/4157951976 wait complete. 2024-07-17T08:45:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.897+0000 7f1e8b200640 1 Processor -- start 2024-07-17T08:45:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.897+0000 7f1e8b200640 1 -- start start 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e70004120 0x7f1e8c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 0x7f1e8c1f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c166260 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1660e0 con 0x7f1e8c0b2320 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c1663e0 con 0x7f1e70004120 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35926/0 (socket says 172.21.15.44:35926) 2024-07-17T08:45:27.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.898+0000 7f1e8a800640 1 -- 172.21.15.44:0/446070600 learned_addr learned my addr 172.21.15.44:0/446070600 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e89e00640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e70004120 0x7f1e8c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e89400640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 0x7f1e8c1f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 -- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e70004120 msgr2=0x7f1e8c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e70004120 0x7f1e8c0b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 -- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 msgr2=0x7f1e8c1f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 0x7f1e8c1f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 -- 172.21.15.44:0/446070600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c0afbb0 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e89400640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 0x7f1e8c1f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:27.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e8a800640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c072820 tx=0x7f1e8400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84005220 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.899+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e84015070 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.900+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e840047a0 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.900+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1f87f0 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.900+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c0b0150 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.901+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c0af910 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.902+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f1e84004940 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.903+0000 7f1e82000640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 0x7f1e6c079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.903+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f1e84098fc0 con 0x7f1e8c1f7fe0 2024-07-17T08:45:27.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.904+0000 7f1e89e00640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 0x7f1e6c079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:27.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.906+0000 7f1e89e00640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 0x7f1e6c079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c072110 tx=0x7f1e7c006b50 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:27.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:27.907+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e8c0af910 con 0x7f1e8c1f7fe0 2024-07-17T08:45:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.191+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1e8c06c690 con 0x7f1e8c1f7fe0 2024-07-17T08:45:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.195+0000 7f1e82000640 1 -- 172.21.15.44:0/446070600 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v84) ==== 190+0+147 (secure 0 0 0) 0x7f1e8c06c690 con 0x7f1e8c1f7fe0 2024-07-17T08:45:28.196 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:45:28.197 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:45:28.197 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:45:28.197 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:45:28.197 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:45:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 msgr2=0x7f1e6c079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 0x7f1e6c079070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c072110 tx=0x7f1e7c006b50 comp rx=0 tx=0).stop 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 msgr2=0x7f1e8c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c072820 tx=0x7f1e8400b980 comp rx=0 tx=0).stop 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 shutdown_connections 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1e6c076bb0 0x7f1e6c079070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e8c1f7fe0 0x7f1e8c0b0950 unknown :-1 s=CLOSED pgs=3327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e8c0b2320 0x7f1e8c1f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 --2- 172.21.15.44:0/446070600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e70004120 0x7f1e8c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 >> 172.21.15.44:0/446070600 conn(0x7f1e8c0b5560 msgr2=0x7f1e8c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.199+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 shutdown_connections 2024-07-17T08:45:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.200+0000 7f1e8b200640 1 -- 172.21.15.44:0/446070600 wait complete. 2024-07-17T08:45:28.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:45:28.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 -- 172.21.15.44:0/2810459101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa138003a50 msgr2=0x7fa138003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 --2- 172.21.15.44:0/2810459101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa138003a50 0x7fa138003e50 secure :-1 s=READY pgs=3329 cs=0 l=1 rev1=1 crypto rx=0x7fa14400baf0 tx=0x7fa138005830 comp rx=0 tx=0).stop 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 -- 172.21.15.44:0/2810459101 shutdown_connections 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 --2- 172.21.15.44:0/2810459101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa138003a50 0x7fa138003e50 unknown :-1 s=CLOSED pgs=3329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 -- 172.21.15.44:0/2810459101 >> 172.21.15.44:0/2810459101 conn(0x7fa1540b0fb0 msgr2=0x7fa1540b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 -- 172.21.15.44:0/2810459101 shutdown_connections 2024-07-17T08:45:28.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.422+0000 7fa159000640 1 -- 172.21.15.44:0/2810459101 wait complete. 2024-07-17T08:45:28.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.424+0000 7fa159000640 1 Processor -- start 2024-07-17T08:45:28.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.424+0000 7fa159000640 1 -- start start 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa138003a50 0x7fa154155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1541f3f50 0x7fa1541f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa15406eec0 con 0x7fa154156fd0 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa15406f040 con 0x7fa1541f3f50 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.425+0000 7fa159000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa15415e510 con 0x7fa138003a50 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35948/0 (socket says 172.21.15.44:35948) 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 -- 172.21.15.44:0/232250601 learned_addr learned my addr 172.21.15.44:0/232250601 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa151e00640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa138003a50 0x7fa154155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:28.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 -- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa138003a50 msgr2=0x7fa154155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa138003a50 0x7fa154155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 -- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1541f3f50 msgr2=0x7fa1541f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1541f3f50 0x7fa1541f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.426+0000 7fa152800640 1 -- 172.21.15.44:0/232250601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154157990 con 0x7fa154156fd0 2024-07-17T08:45:28.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa152800640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7fa1540ad120 tx=0x7fa14c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c017070 con 0x7fa154156fd0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa14c0035f0 con 0x7fa154156fd0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c003a40 con 0x7fa154156fd0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1541f4a20 con 0x7fa154156fd0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.427+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541f4fe0 con 0x7fa154156fd0 2024-07-17T08:45:28.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.428+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540ac220 con 0x7fa154156fd0 2024-07-17T08:45:28.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.430+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fa14c01a070 con 0x7fa154156fd0 2024-07-17T08:45:28.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.431+0000 7fa14a000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 0x7fa1340790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:28.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.431+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7fa14c0978a0 con 0x7fa154156fd0 2024-07-17T08:45:28.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.432+0000 7fa151e00640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 0x7fa1340790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:28.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.433+0000 7fa151e00640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 0x7fa1340790a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa144036000 tx=0x7fa1440046c0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.435+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa154157990 con 0x7fa154156fd0 2024-07-17T08:45:28.579 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-07-17T08:45:28.578+0000 7f0648a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:45:28.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.677+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1540b23b0 con 0x7fa154156fd0 2024-07-17T08:45:28.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.678+0000 7fa14a000640 1 -- 172.21.15.44:0/232250601 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7fa15415ea50 con 0x7fa154156fd0 2024-07-17T08:45:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:45:28.679 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:45:28.679 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:45:28.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 msgr2=0x7fa1340790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 0x7fa1340790a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa144036000 tx=0x7fa1440046c0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 msgr2=0x7fa154157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7fa1540ad120 tx=0x7fa14c00c8b0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 shutdown_connections 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fa134076be0 0x7fa1340790a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa154156fd0 0x7fa154157450 unknown :-1 s=CLOSED pgs=3330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1541f3f50 0x7fa1541f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 --2- 172.21.15.44:0/232250601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa138003a50 0x7fa154155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 >> 172.21.15.44:0/232250601 conn(0x7fa1540b0fb0 msgr2=0x7fa154160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 shutdown_connections 2024-07-17T08:45:28.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:28.681+0000 7fa159000640 1 -- 172.21.15.44:0/232250601 wait complete. 2024-07-17T08:45:29.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 209, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:45:22.106720+0000', 'modified': '2024-07-17T08:45:25.156126+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14406}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14406': {'gid': 14406, 'name': 'a', 'rank': 0, 'incarnation': 205, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6834/2831509556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 2831509556}, {'type': 'v1', 'addr': '172.21.15.44:6834', 'nonce': 2831509556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [50], 'metadata_pool': 49, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 14406, 'qdb_cluster': [14406]} 2024-07-17T08:45:29.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:45:29.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-17T08:45:29.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.327+0000 7f4f07200640 1 -- 172.21.15.44:0/2570156029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 msgr2=0x7f4f0806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.327+0000 7f4f07200640 1 --2- 172.21.15.44:0/2570156029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f0806f1b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005fd0 tx=0x7f4f0002aaa0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 -- 172.21.15.44:0/2570156029 shutdown_connections 2024-07-17T08:45:29.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 --2- 172.21.15.44:0/2570156029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f0806f1b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 --2- 172.21.15.44:0/2570156029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f0806ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 -- 172.21.15.44:0/2570156029 >> 172.21.15.44:0/2570156029 conn(0x7f4f080cf0d0 msgr2=0x7f4f080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 -- 172.21.15.44:0/2570156029 shutdown_connections 2024-07-17T08:45:29.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.328+0000 7f4f07200640 1 -- 172.21.15.44:0/2570156029 wait complete. 2024-07-17T08:45:29.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.329+0000 7f4f07200640 1 Processor -- start 2024-07-17T08:45:29.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.330+0000 7f4f07200640 1 -- start start 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f081f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f080ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f080cb740 con 0x7f4f08151f20 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f080cb5c0 con 0x7f4f08152300 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f07200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f080cb8c0 con 0x7f4f080cd0b0 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58444/0 (socket says 172.21.15.44:58444) 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 -- 172.21.15.44:0/2302947762 learned_addr learned my addr 172.21.15.44:0/2302947762 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f06800640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f080ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f05e00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f081f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 -- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 msgr2=0x7f4f081f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f081f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 -- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 msgr2=0x7f4f080ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f080ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4effe00640 1 -- 172.21.15.44:0/2302947762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f08073a80 con 0x7f4f08151f20 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f05e00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f081f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:29.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.331+0000 7f4f06800640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f080ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:29.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4effe00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9f0 tx=0x7f4ef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4efe000640 1 -- 172.21.15.44:0/2302947762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000cd20 con 0x7f4f08151f20 2024-07-17T08:45:29.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4efe000640 1 -- 172.21.15.44:0/2302947762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef0010430 con 0x7f4f08151f20 2024-07-17T08:45:29.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4efe000640 1 -- 172.21.15.44:0/2302947762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0010840 con 0x7f4f08151f20 2024-07-17T08:45:29.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f08073d60 con 0x7f4f08151f20 2024-07-17T08:45:29.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.332+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f08074320 con 0x7f4f08151f20 2024-07-17T08:45:29.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.335+0000 7f4efe000640 1 -- 172.21.15.44:0/2302947762 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f4ef00109e0 con 0x7f4f08151f20 2024-07-17T08:45:29.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.337+0000 7f4efe000640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 0x7f4ee4079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.337+0000 7f4efe000640 1 -- 172.21.15.44:0/2302947762 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f4ef0099230 con 0x7f4f08151f20 2024-07-17T08:45:29.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.337+0000 7f4f0fd6b740 1 Processor -- start 2024-07-17T08:45:29.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.337+0000 7f4f0fd6b740 1 -- start start 2024-07-17T08:45:29.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.337+0000 7f4f05e00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 0x7f4ee4079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1853c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc18538c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc18483d60 con 0x55cc18513280 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc18473260 con 0x55cc18538790 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55cc184772a0 con 0x55cc1853bfc0 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.338+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58456/0 (socket says 172.21.15.44:58456) 2024-07-17T08:45:29.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 -- 172.21.15.44:0/744434604 learned_addr learned my addr 172.21.15.44:0/744434604 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4f05e00640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 0x7f4ee4079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8001740 tx=0x7f4ef8009040 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 -- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 msgr2=0x55cc1853c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4f06800640 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc18538c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1853c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 -- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 msgr2=0x55cc18538c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc18538c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 -- 172.21.15.44:0/744434604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc181d7850 con 0x55cc18513280 2024-07-17T08:45:29.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4effe00640 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 crypto rx=0x7f4ef005dd30 tx=0x7f4ef00617c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eebe3a745b03ef1 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4ef6a00640 1 -- 172.21.15.44:0/744434604 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0061cb0 con 0x55cc18513280 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4ef6a00640 1 -- 172.21.15.44:0/744434604 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef00662b0 con 0x55cc18513280 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.339+0000 7f4ef6a00640 1 -- 172.21.15.44:0/744434604 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef006acd0 con 0x55cc18513280 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/744434604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 msgr2=0x55cc18538250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 crypto rx=0x7f4ef005dd30 tx=0x7f4ef00617c0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/744434604 shutdown_connections 2024-07-17T08:45:29.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc18538c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18513280 0x55cc18538250 unknown :-1 s=CLOSED pgs=3332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/744434604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1853c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/744434604 >> 172.21.15.44:0/744434604 conn(0x55cc1850e460 msgr2=0x55cc18511e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/744434604 shutdown_connections 2024-07-17T08:45:29.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.340+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/744434604 wait complete. 2024-07-17T08:45:29.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.341+0000 7f4f0fd6b740 1 -- start start 2024-07-17T08:45:29.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.341+0000 7f4f0fd6b740 1 Processor -- start 2024-07-17T08:45:29.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc1854d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1852fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc181e6620 con 0x55cc1852dc70 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cc1841af90 con 0x55cc18538790 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55cc1850cb60 con 0x55cc1853bfc0 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.342+0000 7f4f0fd6b740 10 client.0 _ll_register_callbacks cb 0x55cc18440f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58464/0 (socket says 172.21.15.44:58464) 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 -- 172.21.15.44:0/494087598 learned_addr learned my addr 172.21.15.44:0/494087598 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f05e00640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc1854d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4effe00640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1852fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 -- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 msgr2=0x55cc1852fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1852fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 -- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 msgr2=0x55cc1854d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc1854d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 -- 172.21.15.44:0/494087598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1824e6b0 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f05e00640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc1854d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4f06800640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009a30 tx=0x7f4f00005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8055f0dd1352288 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.343+0000 7f4eeb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0002e870 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f4f0002ea10 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc1844a900 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7f4f0002ec10 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 10 client.14520 fetch_fsmap learned FSMap version 209 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) -- 0x55cc18531780 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55cc1844a900 con 0x55cc1852dc70 2024-07-17T08:45:29.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 209) ==== 3502+0+0 (secure 0 0 0) 0x7f4f0004a020 con 0x55cc1852dc70 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 10 client.14520 fetch_fsmap finished waiting for FSMap version 209 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 10 client.14520 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 20 client.14520 populate_metadata read hostname 'smithi044' 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1856a7d0 0x55cc18575480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 4 client.14520 mds_command: new command op to 14406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.344+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 --> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc18485cb0 con 0x55cc1856a7d0 2024-07-17T08:45:29.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.345+0000 7f4f05e00640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1856a7d0 0x55cc18575480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.345+0000 7f4f05e00640 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1856a7d0 0x55cc18575480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c577e52e6ef265fc server_cookie=1a2e5abd49d6967 in_seq=0 out_seq=0 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.345+0000 7f4eeb400640 10 client.14520 ms_handle_connect on v2:172.21.15.44:6832/2831509556 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.346+0000 7f4eeb400640 1 -- 172.21.15.44:0/494087598 <== mds.0 v2:172.21.15.44:6832/2831509556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cc18485cb0 con 0x55cc1856a7d0 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.346+0000 7f4eeb400640 10 client.14520 handle_command_reply: tid=0 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.346+0000 7f4f0fd6b740 1 client.14520 shutdown 2024-07-17T08:45:29.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 msgr2=0x55cc18530190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009a30 tx=0x7f4f00005f70 comp rx=0 tx=0).stop 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 shutdown_connections 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1856a7d0 0x55cc18575480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc1852dc70 0x55cc18530190 unknown :-1 s=CLOSED pgs=3333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1853bfc0 0x55cc1852fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/494087598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc18538790 0x55cc1854d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4eeb400640 0 client.14520 ms_handle_reset on v2:172.21.15.44:6832/2831509556 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 >> 172.21.15.44:0/494087598 conn(0x55cc18513280 msgr2=0x55cc18511b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 shutdown_connections 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/494087598 wait complete. 2024-07-17T08:45:29.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.347+0000 7f4f0fd6b740 20 client.14520 trim_cache size 0 max 16384 2024-07-17T08:45:29.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.367+0000 7f4f0fd6b740 1 Processor -- start 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 -- start start 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc18535350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc185502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08201620 con 0x55cc18538790 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08200600 con 0x55cc1853bfc0 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.368+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f081ff920 con 0x55cc1852dc70 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58478/0 (socket says 172.21.15.44:58478) 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 -- 172.21.15.44:0/359345115 learned_addr learned my addr 172.21.15.44:0/359345115 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4f06800640 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc185502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 -- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 msgr2=0x55cc18535350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc18535350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 -- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 msgr2=0x55cc185502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc185502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 -- 172.21.15.44:0/359345115 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1852daa0 con 0x55cc18538790 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4f06800640 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc185502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.369+0000 7f4effe00640 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7f4ef005dd30 tx=0x7f4ef0097220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6e2b3c8bfbd939 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4ef6000640 1 -- 172.21.15.44:0/359345115 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0080070 con 0x55cc18538790 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4ef6000640 1 -- 172.21.15.44:0/359345115 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef006acd0 con 0x55cc18538790 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4ef6000640 1 -- 172.21.15.44:0/359345115 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0061cb0 con 0x55cc18538790 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/359345115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 msgr2=0x55cc1854fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7f4ef005dd30 tx=0x7f4ef0097220 comp rx=0 tx=0).stop 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/359345115 shutdown_connections 2024-07-17T08:45:29.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc185502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc1854fd90 unknown :-1 s=CLOSED pgs=3334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/359345115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc18535350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/359345115 >> 172.21.15.44:0/359345115 conn(0x55cc18513280 msgr2=0x55cc18546fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/359345115 shutdown_connections 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.370+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/359345115 wait complete. 2024-07-17T08:45:29.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.371+0000 7f4f0fd6b740 1 -- start start 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.371+0000 7f4f0fd6b740 1 Processor -- start 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc185d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc1853a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc18531a60 con 0x55cc18538790 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081fff70 con 0x55cc1853bfc0 2024-07-17T08:45:29.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55cc185312a0 con 0x55cc1852dc70 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.372+0000 7f4f0fd6b740 10 client.0 _ll_register_callbacks cb 0x7f4ef0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58490/0 (socket says 172.21.15.44:58490) 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 -- 172.21.15.44:0/1583483420 learned_addr learned my addr 172.21.15.44:0/1583483420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 -- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 msgr2=0x55cc185d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc185d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 -- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 msgr2=0x55cc1853a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc1853a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 -- 172.21.15.44:0/1583483420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc18511c20 con 0x55cc18538790 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4effe00640 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000cfd0 tx=0x7f4ef0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a3b99ba1520372 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.373+0000 7f4eeb400640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:45:29.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0080070 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f4ef0061120 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef006acd0 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0061cb0 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 10 client.14526 fetch_fsmap learned FSMap version 209 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=2+}) -- 0x55cc18512640 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef006acd0 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 209) ==== 3502+0+0 (secure 0 0 0) 0x7f4ef0083020 con 0x55cc18538790 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 10 client.14526 fetch_fsmap finished waiting for FSMap version 209 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 10 client.14526 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 20 client.14526 populate_metadata read hostname 'smithi044' 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1850f950 0x55cc1850fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 4 client.14526 mds_command: new command op to 14406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:45:29.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.374+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 --> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] -- command(tid 0: {"prefix": "status"}) -- 0x55cc18481750 con 0x55cc1850f950 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4f05e00640 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1850f950 0x55cc1850fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4f05e00640 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1850f950 0x55cc1850fd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ca19674f6d0b62 server_cookie=c494c4207360ad07 in_seq=0 out_seq=0 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4eeb400640 10 client.14526 ms_handle_connect on v2:172.21.15.44:6832/2831509556 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4eeb400640 1 -- 172.21.15.44:0/1583483420 <== mds.0 v2:172.21.15.44:6832/2831509556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55cc18481750 con 0x55cc1850f950 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4eeb400640 10 client.14526 handle_command_reply: tid=0 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4f0fd6b740 1 client.14526 shutdown 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 msgr2=0x55cc18512100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.375+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000cfd0 tx=0x7f4ef0005c50 comp rx=0 tx=0).stop 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 shutdown_connections 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:6832/2831509556,v1:172.21.15.44:6834/2831509556] conn(0x55cc1850f950 0x55cc1850fd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cc1853bfc0 0x55cc1853a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc18538790 0x55cc18512100 unknown :-1 s=CLOSED pgs=3335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4eeb400640 0 client.14526 ms_handle_reset on v2:172.21.15.44:6832/2831509556 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 --2- 172.21.15.44:0/1583483420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55cc1852dc70 0x55cc185d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 >> 172.21.15.44:0/1583483420 conn(0x55cc18513280 msgr2=0x55cc1852fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 shutdown_connections 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 1 -- 172.21.15.44:0/1583483420 wait complete. 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.376+0000 7f4f0fd6b740 20 client.14526 trim_cache size 0 max 16384 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:45:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 14406, 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.768167021, 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 209, 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 364, 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 364, 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.657962023 2024-07-17T08:45:29.382 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 msgr2=0x7f4ee4079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 0x7f4ee4079070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8001740 tx=0x7f4ef8009040 comp rx=0 tx=0).stop 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 msgr2=0x7f4f080ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9f0 tx=0x7f4ef000eeb0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 shutdown_connections 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4ee4076bb0 0x7f4ee4079070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f08152300 0x7f4f080ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f08151f20 0x7f4f080ad060 unknown :-1 s=CLOSED pgs=3331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 --2- 172.21.15.44:0/2302947762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f080cd0b0 0x7f4f081f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 >> 172.21.15.44:0/2302947762 conn(0x7f4f080cf0d0 msgr2=0x7f4f0806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 shutdown_connections 2024-07-17T08:45:29.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.377+0000 7f4f07200640 1 -- 172.21.15.44:0/2302947762 wait complete. 2024-07-17T08:45:29.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:45:29.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 -- 172.21.15.44:0/1461926778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 msgr2=0x7f4e4c0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 --2- 172.21.15.44:0/1461926778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0d2a50 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c08ca10 tx=0x7f4e4002a540 comp rx=0 tx=0).stop 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 -- 172.21.15.44:0/1461926778 shutdown_connections 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 --2- 172.21.15.44:0/1461926778 >> v2:172.21.15.175:3300/0 conn(0x7f4e4c0b6390 0x7f4e4c0b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 --2- 172.21.15.44:0/1461926778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0d2a50 unknown :-1 s=CLOSED pgs=3337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.605+0000 7f4e53200640 1 -- 172.21.15.44:0/1461926778 >> 172.21.15.44:0/1461926778 conn(0x7f4e4c0cfec0 msgr2=0x7f4e4c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.606+0000 7f4e53200640 1 -- 172.21.15.44:0/1461926778 shutdown_connections 2024-07-17T08:45:29.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.606+0000 7f4e53200640 1 -- 172.21.15.44:0/1461926778 wait complete. 2024-07-17T08:45:29.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.607+0000 7f4e53200640 1 Processor -- start 2024-07-17T08:45:29.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.607+0000 7f4e53200640 1 -- start start 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e4c0b6390 0x7f4e4c14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4c0cdca0 0x7f4e4c0a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c15da90 con 0x7f4e4c15d530 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c15d910 con 0x7f4e4c0cdca0 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.608+0000 7f4e53200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c0b4ec0 con 0x7f4e4c0b6390 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58500/0 (socket says 172.21.15.44:58500) 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 -- 172.21.15.44:0/1607174898 learned_addr learned my addr 172.21.15.44:0/1607174898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e51400640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4c0cdca0 0x7f4e4c0a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e51e00640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e4c0b6390 0x7f4e4c14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 -- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e4c0b6390 msgr2=0x7f4e4c14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e4c0b6390 0x7f4e4c14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 -- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4c0cdca0 msgr2=0x7f4e4c0a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4c0cdca0 0x7f4e4c0a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.609+0000 7f4e52800640 1 -- 172.21.15.44:0/1607174898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1f3b10 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e52800640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e960 tx=0x7f4e4800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48004d20 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e4800cb20 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1f3df0 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48005020 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.610+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1f43b0 con 0x7f4e4c15d530 2024-07-17T08:45:29.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.611+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c0b67c0 con 0x7f4e4c15d530 2024-07-17T08:45:29.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.612+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f4e480051c0 con 0x7f4e4c15d530 2024-07-17T08:45:29.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.614+0000 7f4e46000640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 0x7f4e30079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:29.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.614+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f4e48099040 con 0x7f4e4c15d530 2024-07-17T08:45:29.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.615+0000 7f4e51e00640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 0x7f4e30079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:29.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.616+0000 7f4e51e00640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 0x7f4e30079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e40034000 tx=0x7f4e40004730 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:29.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.619+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4e48016030 con 0x7f4e4c15d530 2024-07-17T08:45:29.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.880+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e4c160a30 con 0x7f4e4c15d530 2024-07-17T08:45:29.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.881+0000 7f4e46000640 1 -- 172.21.15.44:0/1607174898 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v209) ==== 53+0+73 (secure 0 0 0) 0x7f4e48062070 con 0x7f4e4c15d530 2024-07-17T08:45:29.882 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 msgr2=0x7f4e30079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 0x7f4e30079070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e40034000 tx=0x7f4e40004730 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 msgr2=0x7f4e4c0aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e960 tx=0x7f4e4800ee20 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 shutdown_connections 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f4e30076bb0 0x7f4e30079070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e4c15d530 0x7f4e4c0aa520 unknown :-1 s=CLOSED pgs=3338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e4c0cdca0 0x7f4e4c0a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 --2- 172.21.15.44:0/1607174898 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e4c0b6390 0x7f4e4c14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:29.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 >> 172.21.15.44:0/1607174898 conn(0x7f4e4c0cfec0 msgr2=0x7f4e4c0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:29.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 shutdown_connections 2024-07-17T08:45:29.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:29.884+0000 7f4e53200640 1 -- 172.21.15.44:0/1607174898 wait complete. 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:45:29.903 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:45:29.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:29.904 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:45:29.946 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:45:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 35928sec preferred_lft 35928sec 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:45:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:45:29.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:29.950 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:45:30.076 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:45:30.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:30.081 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:45:30.135 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:45:30.135 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:45:30.135 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:45:30.136 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:45:30.136 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:45:30.137 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:45:30.137 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:45:30.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:45:30.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:45:30.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:45:30.138 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:45:30.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:45:30.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:45:30.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:30.361 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:45:30.408 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:45:30.408 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:45:30.409 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:45:30.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:30.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:45:30.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:45:30.490 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:45:30.544 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:45:30.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:45:30.544 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:45:30.595 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:45:30.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:30.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:45:30.732 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:45:30.733 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:45:30.733 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:45:30.754 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:45:30.755 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:45:30.755 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:45:30.755 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:45:30.755 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:45:30.780 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:45:30.782 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:45:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:45:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:45:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:45:30.782 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:45:30.786 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:45:30.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:45:30.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:45:30.864 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:45:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:45:30.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:45:31.005 INFO:teuthology.orchestra.run.smithi044.stdout:client14538 2024-07-17T08:45:31.005 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:45:31.005 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:45:31.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:45:31.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.328+0000 7f669f200640 1 -- 172.21.15.44:0/2100504846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6684004bd0 msgr2=0x7f6684004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.328+0000 7f669f200640 1 --2- 172.21.15.44:0/2100504846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6684004bd0 0x7f6684004ff0 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7f66a00b1d50 tx=0x7f6684006940 comp rx=0 tx=0).stop 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.328+0000 7f669e800640 1 -- 172.21.15.44:0/2100504846 reap_dead start 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.329+0000 7f669f200640 1 -- 172.21.15.44:0/2100504846 shutdown_connections 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.329+0000 7f669f200640 1 -- 172.21.15.44:0/2100504846 >> 172.21.15.44:0/2100504846 conn(0x7f66a00cf0d0 msgr2=0x7f66a00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.329+0000 7f669f200640 1 -- 172.21.15.44:0/2100504846 shutdown_connections 2024-07-17T08:45:31.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.329+0000 7f669f200640 1 -- 172.21.15.44:0/2100504846 wait complete. 2024-07-17T08:45:31.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.330+0000 7f669f200640 1 Processor -- start 2024-07-17T08:45:31.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.331+0000 7f669f200640 1 -- start start 2024-07-17T08:45:31.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6684004bd0 0x7f66a00af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:31.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:31.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a00ca110 0x7f66a0149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:31.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66a00b2eb0 con 0x7f66a014e550 2024-07-17T08:45:31.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66a015a880 con 0x7f66a00ca110 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66a00b3030 con 0x7f6684004bd0 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58524/0 (socket says 172.21.15.44:58524) 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 -- 172.21.15.44:0/763611136 learned_addr learned my addr 172.21.15.44:0/763611136 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669de00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6684004bd0 0x7f66a00af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 -- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6684004bd0 msgr2=0x7f66a00af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6684004bd0 0x7f66a00af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 -- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a00ca110 msgr2=0x7f66a0149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f669e800640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a00ca110 0x7f66a0149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a00ca110 0x7f66a0149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:31.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.332+0000 7f6697e00640 1 -- 172.21.15.44:0/763611136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a00c8630 con 0x7f66a014e550 2024-07-17T08:45:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f6697e00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7f668800e610 tx=0x7f668800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688015020 con 0x7f66a014e550 2024-07-17T08:45:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6688010980 con 0x7f66a014e550 2024-07-17T08:45:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688010dd0 con 0x7f66a014e550 2024-07-17T08:45:31.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a00c8910 con 0x7f66a014e550 2024-07-17T08:45:31.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.333+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a00c8ed0 con 0x7f66a014e550 2024-07-17T08:45:31.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.335+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a00762e0 con 0x7f66a014e550 2024-07-17T08:45:31.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.336+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7f6688018020 con 0x7f66a014e550 2024-07-17T08:45:31.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.337+0000 7f6696000640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 0x7f667c0790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:31.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.337+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f6688098e10 con 0x7f66a014e550 2024-07-17T08:45:31.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.337+0000 7f669de00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 0x7f667c0790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:31.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.339+0000 7f669de00640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 0x7f667c0790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6690036000 tx=0x7f66900046c0 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:31.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.341+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f66a00762e0 con 0x7f66a014e550 2024-07-17T08:45:31.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:31.583+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 --> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f66a00ca6d0 con 0x7f667c076be0 2024-07-17T08:45:38.505 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.503+0000 7f72fb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.503+0000 7f72fb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.503+0000 7f72fb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.503+0000 7f72fb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.506 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.503+0000 7f72fb400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.506+0000 7f7300400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.506+0000 7f7300400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.506+0000 7f7300400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.506+0000 7f7300400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.508 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:38.506+0000 7f7300400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:38.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.520+0000 7f6696000640 1 -- 172.21.15.44:0/763611136 <== mgr.10067 v2:172.21.15.175:6834/945953093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6690034710 con 0x7f667c076be0 2024-07-17T08:45:38.522 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:45:38.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:45:38.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephfs" 2024-07-17T08:45:38.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:45:38.523 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:45:38.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 msgr2=0x7f667c0790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:38.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 0x7f667c0790a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6690036000 tx=0x7f66900046c0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 msgr2=0x7f66a01490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:38.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7f668800e610 tx=0x7f668800ead0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 shutdown_connections 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f667c076be0 0x7f667c0790a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66a00ca110 0x7f66a0149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66a014e550 0x7f66a01490e0 unknown :-1 s=CLOSED pgs=3342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 --2- 172.21.15.44:0/763611136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6684004bd0 0x7f66a00af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 >> 172.21.15.44:0/763611136 conn(0x7f66a00cf0d0 msgr2=0x7f66a0073f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 shutdown_connections 2024-07-17T08:45:38.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.525+0000 7f669f200640 1 -- 172.21.15.44:0/763611136 wait complete. 2024-07-17T08:45:38.550 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 -- 172.21.15.44:0/1230897101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c06ed30 msgr2=0x7fc89c15c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/1230897101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c06ed30 0x7fc89c15c2f0 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0d0260 tx=0x7fc88402a4d0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 -- 172.21.15.44:0/1230897101 shutdown_connections 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/1230897101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 0x7fc89c0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/1230897101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c06ed30 0x7fc89c15c2f0 unknown :-1 s=CLOSED pgs=3343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/1230897101 >> v2:172.21.15.175:3300/0 conn(0x7fc89c152220 0x7fc89c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 -- 172.21.15.44:0/1230897101 >> 172.21.15.44:0/1230897101 conn(0x7fc89c0b0eb0 msgr2=0x7fc89c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 -- 172.21.15.44:0/1230897101 shutdown_connections 2024-07-17T08:45:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.765+0000 7fc8a2c00640 1 -- 172.21.15.44:0/1230897101 wait complete. 2024-07-17T08:45:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.767+0000 7fc8a2c00640 1 Processor -- start 2024-07-17T08:45:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.767+0000 7fc8a2c00640 1 -- start start 2024-07-17T08:45:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c06ed30 0x7fc89c1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 0x7fc89c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:38.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c15f1f0 con 0x7fc89c152220 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c15f070 con 0x7fc89c06ed30 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c15f370 con 0x7fc89c0cb3a0 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a0c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58554/0 (socket says 172.21.15.44:58554) 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.768+0000 7fc8a0c00640 1 -- 172.21.15.44:0/804407979 learned_addr learned my addr 172.21.15.44:0/804407979 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc89be00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c06ed30 0x7fc89c1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc893e00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 0x7fc89c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 -- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 msgr2=0x7fc89c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 0x7fc89c14ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 -- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c06ed30 msgr2=0x7fc89c1f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:38.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c06ed30 0x7fc89c1f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 -- 172.21.15.44:0/804407979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c1f4640 con 0x7fc89c152220 2024-07-17T08:45:38.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a0c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7fc89400eaa0 tx=0x7fc89400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89400cd50 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc894004b20 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894005020 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.769+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c1f4920 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.770+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c0c9210 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.771+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c0cb7f0 con 0x7fc89c152220 2024-07-17T08:45:38.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.772+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 220) ==== 98829+0+0 (secure 0 0 0) 0x7fc8940051c0 con 0x7fc89c152220 2024-07-17T08:45:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.774+0000 7fc898c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 0x7fc878079070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:38.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.774+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7fc894098fb0 con 0x7fc89c152220 2024-07-17T08:45:38.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.774+0000 7fc89be00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 0x7fc878079070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:38.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.775+0000 7fc89be00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 0x7fc878079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009960 tx=0x7fc88c005c90 comp rx=0 tx=0).ready entity=mgr.10067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:38.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:38.778+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc894016030 con 0x7fc89c152220 2024-07-17T08:45:39.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.045+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc89c0afc40 con 0x7fc89c152220 2024-07-17T08:45:39.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.733+0000 7fc898c00640 1 -- 172.21.15.44:0/804407979 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v221) ==== 92+0+0 (secure 0 0 0) 0x7fc894062070 con 0x7fc89c152220 2024-07-17T08:45:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.737+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 msgr2=0x7fc878079070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.737+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 0x7fc878079070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009960 tx=0x7fc88c005c90 comp rx=0 tx=0).stop 2024-07-17T08:45:39.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.737+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 msgr2=0x7fc89c0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:39.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.737+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7fc89400eaa0 tx=0x7fc89400ef60 comp rx=0 tx=0).stop 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 shutdown_connections 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7fc878076bb0 0x7fc878079070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc89c152220 0x7fc89c0c9730 unknown :-1 s=CLOSED pgs=3344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc89c0cb3a0 0x7fc89c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 --2- 172.21.15.44:0/804407979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc89c06ed30 0x7fc89c1f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:39.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.738+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 >> 172.21.15.44:0/804407979 conn(0x7fc89c0b0eb0 msgr2=0x7fc89c0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.739+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 shutdown_connections 2024-07-17T08:45:39.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.739+0000 7fc8a2c00640 1 -- 172.21.15.44:0/804407979 wait complete. 2024-07-17T08:45:39.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 -- 172.21.15.44:0/1247854475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 msgr2=0x7f1aa8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 --2- 172.21.15.44:0/1247854475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1aa8003e70 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009b50 tx=0x7f1aa8005830 comp rx=0 tx=0).stop 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 -- 172.21.15.44:0/1247854475 shutdown_connections 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 --2- 172.21.15.44:0/1247854475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1aa8003e70 unknown :-1 s=CLOSED pgs=3349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 -- 172.21.15.44:0/1247854475 >> 172.21.15.44:0/1247854475 conn(0x7f1ac40b2240 msgr2=0x7f1ac40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:39.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 -- 172.21.15.44:0/1247854475 shutdown_connections 2024-07-17T08:45:39.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.994+0000 7f1ac9800640 1 -- 172.21.15.44:0/1247854475 wait complete. 2024-07-17T08:45:39.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.997+0000 7f1ac9800640 1 Processor -- start 2024-07-17T08:45:39.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.997+0000 7f1ac9800640 1 -- start start 2024-07-17T08:45:40.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ac415db10 0x7f1ac4142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 0x7f1ac415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac406eb70 con 0x7f1aa8003a50 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40aeb70 con 0x7f1ac415f490 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac406ecf0 con 0x7f1ac415db10 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52272/0 (socket says 172.21.15.44:52272) 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1ac2a00640 1 -- 172.21.15.44:0/3001018017 learned_addr learned my addr 172.21.15.44:0/3001018017 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:39.999+0000 7f1abbe00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ac415db10 0x7f1ac4142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac3400640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 0x7f1ac415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ac415db10 msgr2=0x7f1ac4142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ac415db10 0x7f1ac4142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 msgr2=0x7f1ac415f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 0x7f1ac415f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 -- 172.21.15.44:0/3001018017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac4143870 con 0x7f1aa8003a50 2024-07-17T08:45:40.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac3400640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 0x7f1ac415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:40.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac2a00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009a70 tx=0x7f1ab40336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:40.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4038040 con 0x7f1aa8003a50 2024-07-17T08:45:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ab40338e0 con 0x7f1aa8003a50 2024-07-17T08:45:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4033d30 con 0x7f1aa8003a50 2024-07-17T08:45:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.000+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41f5e40 con 0x7f1aa8003a50 2024-07-17T08:45:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.001+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac415e2b0 con 0x7f1aa8003a50 2024-07-17T08:45:40.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.002+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac41610c0 con 0x7f1aa8003a50 2024-07-17T08:45:40.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.004+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 221) ==== 98846+0+0 (secure 0 0 0) 0x7f1ab4043020 con 0x7f1aa8003a50 2024-07-17T08:45:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.006+0000 7f1abaa00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 0x7f1aa00790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.006+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f1ab40b8880 con 0x7f1aa8003a50 2024-07-17T08:45:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.006+0000 7f1abbe00640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 msgr2=0x7f1aa00790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:40.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.006+0000 7f1abbe00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 0x7f1aa00790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:40.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.010+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1ab404a020 con 0x7f1aa8003a50 2024-07-17T08:45:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.208+0000 7f1abbe00640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 msgr2=0x7f1aa00790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:40.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.208+0000 7f1abbe00640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 0x7f1aa00790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:40.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.223+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1ac4074160 con 0x7f1aa8003a50 2024-07-17T08:45:40.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.230+0000 7f1abaa00640 1 -- 172.21.15.44:0/3001018017 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v109)=0 v109) ==== 138+0+0 (secure 0 0 0) 0x7f1ac41f5e40 con 0x7f1aa8003a50 2024-07-17T08:45:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 msgr2=0x7f1aa00790f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:40.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 0x7f1aa00790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 msgr2=0x7f1ac41f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009a70 tx=0x7f1ab40336f0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 shutdown_connections 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f1aa0076c30 0x7f1aa00790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ac415f490 0x7f1ac415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ac415db10 0x7f1ac4142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 --2- 172.21.15.44:0/3001018017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1aa8003a50 0x7f1ac41f5900 unknown :-1 s=CLOSED pgs=3350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.233+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 >> 172.21.15.44:0/3001018017 conn(0x7f1ac40b2240 msgr2=0x7f1ac40b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.234+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 shutdown_connections 2024-07-17T08:45:40.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.234+0000 7f1ac9800640 1 -- 172.21.15.44:0/3001018017 wait complete. 2024-07-17T08:45:40.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-17T08:45:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 -- 172.21.15.44:0/2818601909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 msgr2=0x7efee015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 --2- 172.21.15.44:0/2818601909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee015d510 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7efed0005fd0 tx=0x7efed002a540 comp rx=0 tx=0).stop 2024-07-17T08:45:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 -- 172.21.15.44:0/2818601909 shutdown_connections 2024-07-17T08:45:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 --2- 172.21.15.44:0/2818601909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee0161be0 0x7efee00b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 --2- 172.21.15.44:0/2818601909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee015d510 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 -- 172.21.15.44:0/2818601909 >> 172.21.15.44:0/2818601909 conn(0x7efee00b4ff0 msgr2=0x7efee00b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.443+0000 7efee5000640 1 -- 172.21.15.44:0/2818601909 shutdown_connections 2024-07-17T08:45:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.444+0000 7efee5000640 1 -- 172.21.15.44:0/2818601909 wait complete. 2024-07-17T08:45:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.445+0000 7efee5000640 1 Processor -- start 2024-07-17T08:45:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.445+0000 7efee5000640 1 -- start start 2024-07-17T08:45:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 0x7efee0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efee00d1a10 con 0x7efee0161be0 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efee00d1890 con 0x7efee015d130 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.446+0000 7efee5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efee00d1b90 con 0x7efee00ce690 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52280/0 (socket says 172.21.15.44:52280) 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 -- 172.21.15.44:0/71785020 learned_addr learned my addr 172.21.15.44:0/71785020 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedf400640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 0x7efee0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedea00640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 msgr2=0x7efee0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 0x7efee0152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 msgr2=0x7efee01540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee01540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedfe00640 1 -- 172.21.15.44:0/71785020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01551d0 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.447+0000 7efedf400640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 0x7efee0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efedfe00640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 crypto rx=0x7efed800c6b0 tx=0x7efed800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8003b70 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed8003d30 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01554b0 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8005330 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.448+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee0155d80 con 0x7efee0161be0 2024-07-17T08:45:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.449+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee00b2200 con 0x7efee0161be0 2024-07-17T08:45:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.452+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 221) ==== 98846+0+0 (secure 0 0 0) 0x7efed80054d0 con 0x7efee0161be0 2024-07-17T08:45:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.453+0000 7efed7400640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 0x7efec00790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.453+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7efed8098580 con 0x7efee0161be0 2024-07-17T08:45:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.453+0000 7efedf400640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 msgr2=0x7efec00790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.453+0000 7efedf400640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 0x7efec00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:40.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.461+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7efed80615b0 con 0x7efee0161be0 2024-07-17T08:45:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.655+0000 7efedf400640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 msgr2=0x7efec00790c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:40.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.655+0000 7efedf400640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 0x7efec00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:40.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.669+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7efee00b3bc0 con 0x7efee0161be0 2024-07-17T08:45:40.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.674+0000 7efed7400640 1 -- 172.21.15.44:0/71785020 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v110)=0 v110) ==== 133+0+0 (secure 0 0 0) 0x7efed8060eb0 con 0x7efee0161be0 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 msgr2=0x7efec00790c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 0x7efec00790c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 msgr2=0x7efee01562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 crypto rx=0x7efed800c6b0 tx=0x7efed800cb70 comp rx=0 tx=0).stop 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 shutdown_connections 2024-07-17T08:45:40.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7efec0076c00 0x7efec00790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee0161be0 0x7efee01562a0 unknown :-1 s=CLOSED pgs=3351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee015d130 0x7efee01540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 --2- 172.21.15.44:0/71785020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee00ce690 0x7efee0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 >> 172.21.15.44:0/71785020 conn(0x7efee00b4ff0 msgr2=0x7efee00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.677+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 shutdown_connections 2024-07-17T08:45:40.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.678+0000 7efee5000640 1 -- 172.21.15.44:0/71785020 wait complete. 2024-07-17T08:45:40.697 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs mygrp07 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1166092818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 msgr2=0x7f2b080b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 --2- 172.21.15.44:0/1166092818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 0x7f2b080b3ce0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc02a610 comp rx=0 tx=0).stop 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1166092818 shutdown_connections 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 --2- 172.21.15.44:0/1166092818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b08161120 0x7f2b0806d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 --2- 172.21.15.44:0/1166092818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 0x7f2b080b3ce0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1166092818 >> 172.21.15.44:0/1166092818 conn(0x7f2b080b0eb0 msgr2=0x7f2b080b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:45:40.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1166092818 shutdown_connections 2024-07-17T08:45:40.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.886+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1166092818 wait complete. 2024-07-17T08:45:40.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.888+0000 7f2b0fc00640 1 Processor -- start 2024-07-17T08:45:40.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.888+0000 7f2b0fc00640 1 -- start start 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 0x7f2b0814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 0x7f2b0814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0814e140 0x7f2b080afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0815e220 con 0x7f2b08161120 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0815e0a0 con 0x7f2b08154600 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.889+0000 7f2b0fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0815e3a0 con 0x7f2b0814e140 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 0x7f2b0814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 0x7f2b0814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52288/0 (socket says 172.21.15.44:52288) 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 -- 172.21.15.44:0/1928764638 learned_addr learned my addr 172.21.15.44:0/1928764638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:45:40.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 0x7f2b0814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b0dc00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0814e140 0x7f2b080afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0814e140 msgr2=0x7f2b080afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0814e140 0x7f2b080afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 msgr2=0x7f2b0814f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b08154600 0x7f2b0814f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b07e00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b081595f0 con 0x7f2b08161120 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.890+0000 7f2b0dc00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b0814e140 0x7f2b080afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b07e00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 0x7f2b0814d9b0 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e7c0 tx=0x7f2af400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400ccc0 con 0x7f2b08161120 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af4010430 con 0x7f2b08161120 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4010860 con 0x7f2b08161120 2024-07-17T08:45:40.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0814def0 con 0x7f2b08161120 2024-07-17T08:45:40.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.891+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b08159890 con 0x7f2b08161120 2024-07-17T08:45:40.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.893+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0815e8c0 con 0x7f2b08161120 2024-07-17T08:45:40.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.894+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 221) ==== 98846+0+0 (secure 0 0 0) 0x7f2af4010a00 con 0x7f2b08161120 2024-07-17T08:45:40.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.895+0000 7f2b04c00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:40.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.896+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(364..364 src has 1..364) ==== 6733+0+0 (secure 0 0 0) 0x7f2af4099120 con 0x7f2b08161120 2024-07-17T08:45:40.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.896+0000 7f2b0d200640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:40.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.896+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:45:40.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:40.900+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2af40621e0 con 0x7f2b08161120 2024-07-17T08:45:41.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:41.097+0000 7f2b0d200640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:41.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:41.098+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:45:41.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:41.142+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7f2b081560f0 con 0x7f2ae8076c30 2024-07-17T08:45:41.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:41.500+0000 7f2b0d200640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:41.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:41.500+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-17T08:45:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:42.301+0000 7f2b0d200640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:42.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:42.301+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-17T08:45:43.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:45:43.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:45:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:43.903+0000 7f2b0d200640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/945953093 2024-07-17T08:45:43.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:43.903+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-17T08:45:43.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:43.992+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 222) ==== 98557+0+0 (secure 0 0 0) 0x7f2af4010cf0 con 0x7f2b08161120 2024-07-17T08:45:43.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:43.993+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 msgr2=0x7f2ae80790f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:45:43.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:43.993+0000 7f2b04c00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/945953093,v1:172.21.15.175:6837/945953093] conn(0x7f2ae8076c30 0x7f2ae80790f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:45:44.299 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.298+0000 7febdf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.300 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.298+0000 7febdf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.300 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.298+0000 7febdf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.300 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.298+0000 7febdf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.300 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.298+0000 7febdf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.301 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.299+0000 7febdb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.301 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.299+0000 7febdb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.301 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.299+0000 7febdb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.301 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.299+0000 7febdb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:44.301 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:44.299+0000 7febdb800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:45.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:45.000+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mgrmap(e 223) ==== 98684+0+0 (secure 0 0 0) 0x7f2af405dd70 con 0x7f2b08161120 2024-07-17T08:45:45.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:45.001+0000 7f2b04c00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2ae803bb80 0x7f2ae8081920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:45:45.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:45.001+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7f2b081560f0 con 0x7f2ae803bb80 2024-07-17T08:45:45.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:45.008+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2ae803bb80 0x7f2ae8081920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:45:45.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:45.017+0000 7f2b0d200640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2ae803bb80 0x7f2ae8081920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2afc033000 tx=0x7f2afc004730 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:45:45.035 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:45.033+0000 7febd8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:45.035 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:45.033+0000 7febd8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:45.035 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:45.033+0000 7febd8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:45.036 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:45.033+0000 7febd8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:45.036 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:45:45.033+0000 7febd8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:45:46.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:46.059+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 9 ==== mgrmap(e 224) ==== 99000+0+0 (secure 0 0 0) 0x7f2af405e640 con 0x7f2b08161120 2024-07-17T08:45:48.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:45:48.075+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mon.0 v2:172.21.15.44:3300/0 10 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f2af409ec90 con 0x7f2b08161120 2024-07-17T08:46:13.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:46:13.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:46:42.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.753+0000 7f2b04c00640 1 -- 172.21.15.44:0/1928764638 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b081560f0 con 0x7f2ae803bb80 2024-07-17T08:46:42.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2ae803bb80 msgr2=0x7f2ae8081920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:42.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2ae803bb80 0x7f2ae8081920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2afc033000 tx=0x7f2afc004730 comp rx=0 tx=0).stop 2024-07-17T08:46:42.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 msgr2=0x7f2b0814d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:42.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 --2- 172.21.15.44:0/1928764638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b08161120 0x7f2b0814d9b0 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e7c0 tx=0x7f2af400ec80 comp rx=0 tx=0).stop 2024-07-17T08:46:42.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0dc00640 1 -- 172.21.15.44:0/1928764638 reap_dead start 2024-07-17T08:46:42.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 shutdown_connections 2024-07-17T08:46:42.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 >> 172.21.15.44:0/1928764638 conn(0x7f2b080b0eb0 msgr2=0x7f2b080ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:42.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 shutdown_connections 2024-07-17T08:46:42.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.757+0000 7f2b0fc00640 1 -- 172.21.15.44:0/1928764638 wait complete. 2024-07-17T08:46:42.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv07 --group_name mygrp07 2024-07-17T08:46:42.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 -- 172.21.15.44:0/1569023637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 msgr2=0x7f8bcc0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:42.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1569023637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc0cc220 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0b2330 tx=0x7f8bb402a510 comp rx=0 tx=0).stop 2024-07-17T08:46:42.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 -- 172.21.15.44:0/1569023637 shutdown_connections 2024-07-17T08:46:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1569023637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc0cc220 unknown :-1 s=CLOSED pgs=3354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 -- 172.21.15.44:0/1569023637 >> 172.21.15.44:0/1569023637 conn(0x7f8bcc0b0eb0 msgr2=0x7f8bcc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 -- 172.21.15.44:0/1569023637 shutdown_connections 2024-07-17T08:46:43.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:42.998+0000 7f8bd3400640 1 -- 172.21.15.44:0/1569023637 wait complete. 2024-07-17T08:46:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.000+0000 7f8bd3400640 1 Processor -- start 2024-07-17T08:46:43.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.000+0000 7f8bd3400640 1 -- start start 2024-07-17T08:46:43.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bcc0ac630 0x7f8bcc1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bcc15d5d0 0x7f8bcc152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b3bb0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0acf20 con 0x7f8bcc15d5d0 2024-07-17T08:46:43.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.001+0000 7f8bd3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b3d30 con 0x7f8bcc0ac630 2024-07-17T08:46:43.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40162/0 (socket says 172.21.15.44:40162) 2024-07-17T08:46:43.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 -- 172.21.15.44:0/1074153908 learned_addr learned my addr 172.21.15.44:0/1074153908 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd1400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bcc15d5d0 0x7f8bcc152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 -- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bcc0ac630 msgr2=0x7f8bcc1526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bcc0ac630 0x7f8bcc1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 -- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bcc15d5d0 msgr2=0x7f8bcc152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bcc15d5d0 0x7f8bcc152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.002+0000 7f8bd0a00640 1 -- 172.21.15.44:0/1074153908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc155fb0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bd0a00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c760 tx=0x7f8bbc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003b70 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bbc003d30 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc156350 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00aa50 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.003+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc156910 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.004+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc15a9c0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.005+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f8bbc00abf0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.006+0000 7f8bc8c00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 0x7f8ba8079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.006+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(365..365 src has 1..365) ==== 6991+0+0 (secure 0 0 0) 0x7f8bbc098aa0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.007+0000 7f8bcbe00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 0x7f8ba8079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.008+0000 7f8bcbe00640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 0x7f8ba8079300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4034000 tx=0x7f8bb40046c0 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:43.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.011+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f8bcc15a9c0 con 0x7f8bcc06f6d0 2024-07-17T08:46:43.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.254+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv07", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7f8bcc0b41e0 con 0x7f8ba8076e40 2024-07-17T08:46:43.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:46:43.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:46:43.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.476+0000 7f8bc8c00640 1 -- 172.21.15.44:0/1074153908 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8bcc0b41e0 con 0x7f8ba8076e40 2024-07-17T08:46:43.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.479+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 msgr2=0x7f8ba8079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.479+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 0x7f8ba8079300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4034000 tx=0x7f8bb40046c0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.479+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 msgr2=0x7f8bcc1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.479+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c760 tx=0x7f8bbc00cc20 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.479+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 shutdown_connections 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bcc15d5d0 0x7f8bcc152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f8ba8076e40 0x7f8ba8079300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bcc0ac630 0x7f8bcc1526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 --2- 172.21.15.44:0/1074153908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8bcc06f6d0 0x7f8bcc1582c0 unknown :-1 s=CLOSED pgs=3355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 >> 172.21.15.44:0/1074153908 conn(0x7f8bcc0b0eb0 msgr2=0x7f8bcc15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 shutdown_connections 2024-07-17T08:46:43.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.480+0000 7f8bd3400640 1 -- 172.21.15.44:0/1074153908 wait complete. 2024-07-17T08:46:43.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --subvol sv07 --group mygrp07 --fs cephfs . 1m 2024-07-17T08:46:43.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.712+0000 7fdb88000640 1 -- 172.21.15.44:0/3229077925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800ac220 msgr2=0x7fdb800ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.712+0000 7fdb88000640 1 --2- 172.21.15.44:0/3229077925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800ac220 0x7fdb800ac690 secure :-1 s=READY pgs=3357 cs=0 l=1 rev1=1 crypto rx=0x7fdb70005fd0 tx=0x7fdb7002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.713+0000 7fdb88000640 1 -- 172.21.15.44:0/3229077925 shutdown_connections 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.713+0000 7fdb88000640 1 --2- 172.21.15.44:0/3229077925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800ac220 0x7fdb800ac690 unknown :-1 s=CLOSED pgs=3357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.713+0000 7fdb88000640 1 -- 172.21.15.44:0/3229077925 >> 172.21.15.44:0/3229077925 conn(0x7fdb800b0fb0 msgr2=0x7fdb800b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.713+0000 7fdb88000640 1 -- 172.21.15.44:0/3229077925 shutdown_connections 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.713+0000 7fdb88000640 1 -- 172.21.15.44:0/3229077925 wait complete. 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.715+0000 7fdb88000640 1 Processor -- start 2024-07-17T08:46:43.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.715+0000 7fdb88000640 1 -- start start 2024-07-17T08:46:43.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 0x7fdb801f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb801522c0 0x7fdb80155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800ce370 con 0x7fdb800cd690 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800ce1f0 con 0x7fdb800ac220 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb88000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800ce4f0 con 0x7fdb801522c0 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb84c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb84c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40184/0 (socket says 172.21.15.44:40184) 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb84c00640 1 -- 172.21.15.44:0/1338434401 learned_addr learned my addr 172.21.15.44:0/1338434401 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb85600640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 0x7fdb801f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.716+0000 7fdb86000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb801522c0 0x7fdb80155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 -- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb801522c0 msgr2=0x7fdb80155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb801522c0 0x7fdb80155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 -- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 msgr2=0x7fdb801f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:43.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 0x7fdb801f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:43.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 -- 172.21.15.44:0/1338434401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801f4cc0 con 0x7fdb800cd690 2024-07-17T08:46:43.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb85600640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 0x7fdb801f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:43.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb84c00640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009850 tx=0x7fdb700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7003f070 con 0x7fdb800cd690 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb700047a0 con 0x7fdb800cd690 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70035030 con 0x7fdb800cd690 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.717+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801f43c0 con 0x7fdb800cd690 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.718+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801f4980 con 0x7fdb800cd690 2024-07-17T08:46:43.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.719+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb800ac690 con 0x7fdb800cd690 2024-07-17T08:46:43.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.720+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7fdb70042070 con 0x7fdb800cd690 2024-07-17T08:46:43.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.721+0000 7fdb7d600640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 0x7fdb64079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:43.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.721+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(365..365 src has 1..365) ==== 6991+0+0 (secure 0 0 0) 0x7fdb700b9930 con 0x7fdb800cd690 2024-07-17T08:46:43.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.722+0000 7fdb85600640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 0x7fdb64079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.723+0000 7fdb85600640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 0x7fdb64079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdb740016a0 tx=0x7fdb74009040 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:43.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.725+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdb70082550 con 0x7fdb800cd690 2024-07-17T08:46:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:43.959+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "subvol": "sv07", "group": "mygrp07", "fs": "cephfs", "path": ".", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdb801f4e90 con 0x7fdb64076e40 2024-07-17T08:46:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.194+0000 7fdb7d600640 1 -- 172.21.15.44:0/1338434401 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+84 (secure 0 0 0) 0x7fdb801f4e90 con 0x7fdb64076e40 2024-07-17T08:46:44.195 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path /volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30/.. 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.198+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 msgr2=0x7fdb64079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.198+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 0x7fdb64079300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdb740016a0 tx=0x7fdb74009040 comp rx=0 tx=0).stop 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 msgr2=0x7fdb8014c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009850 tx=0x7fdb700038a0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 shutdown_connections 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fdb64076e40 0x7fdb64079300 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb801522c0 0x7fdb80155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb800cd690 0x7fdb8014c640 unknown :-1 s=CLOSED pgs=3358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 --2- 172.21.15.44:0/1338434401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb800ac220 0x7fdb801f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 >> 172.21.15.44:0/1338434401 conn(0x7fdb800b0fb0 msgr2=0x7fdb800afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:44.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 shutdown_connections 2024-07-17T08:46:44.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.199+0000 7fdb88000640 1 -- 172.21.15.44:0/1338434401 wait complete. 2024-07-17T08:46:44.223 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30/.. 2024-07-17T08:46:44.223 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --group mygrp07 --fs cephfs . 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.443+0000 7f74d2a00640 1 -- 172.21.15.44:0/1562509444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74b4004140 msgr2=0x7f74b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.443+0000 7f74d2a00640 1 --2- 172.21.15.44:0/1562509444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74b4004140 0x7f74b4004800 secure :-1 s=READY pgs=3360 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0cf0a0 tx=0x7f74b4005830 comp rx=0 tx=0).stop 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.444+0000 7f74d2000640 1 -- 172.21.15.44:0/1562509444 reap_dead start 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.444+0000 7f74d2a00640 1 -- 172.21.15.44:0/1562509444 shutdown_connections 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.444+0000 7f74d2a00640 1 -- 172.21.15.44:0/1562509444 >> 172.21.15.44:0/1562509444 conn(0x7f74cc0cf0d0 msgr2=0x7f74cc0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.444+0000 7f74d2a00640 1 -- 172.21.15.44:0/1562509444 shutdown_connections 2024-07-17T08:46:44.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.444+0000 7f74d2a00640 1 -- 172.21.15.44:0/1562509444 wait complete. 2024-07-17T08:46:44.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.445+0000 7f74d2a00640 1 Processor -- start 2024-07-17T08:46:44.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.446+0000 7f74d2a00640 1 -- start start 2024-07-17T08:46:44.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74b4004140 0x7f74cc148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74cc099a10 0x7f74cc099e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc072720 0x7f74cc072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0ac6d0 con 0x7f74cc072720 2024-07-17T08:46:44.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0ac550 con 0x7f74cc099a10 2024-07-17T08:46:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74cc0ac850 con 0x7f74b4004140 2024-07-17T08:46:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc072720 0x7f74cc072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc072720 0x7f74cc072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40198/0 (socket says 172.21.15.44:40198) 2024-07-17T08:46:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 -- 172.21.15.44:0/2337268849 learned_addr learned my addr 172.21.15.44:0/2337268849 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:44.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d1600640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74b4004140 0x7f74cc148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d0c00640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74cc099a10 0x7f74cc099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 -- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74b4004140 msgr2=0x7f74cc148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74b4004140 0x7f74cc148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 -- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74cc099a10 msgr2=0x7f74cc099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.447+0000 7f74d2000640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74cc099a10 0x7f74cc099e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74d2000640 1 -- 172.21.15.44:0/2337268849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc0713e0 con 0x7f74cc072720 2024-07-17T08:46:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74d2000640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74cc072720 0x7f74cc072ba0 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005fd0 tx=0x7f74c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400cc60 con 0x7f74cc072720 2024-07-17T08:46:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c4019070 con 0x7f74cc072720 2024-07-17T08:46:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74d2a00640 1 -- 172.21.15.44:0/2337268849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc071680 con 0x7f74cc072720 2024-07-17T08:46:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40116d0 con 0x7f74cc072720 2024-07-17T08:46:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.448+0000 7f74d2a00640 1 -- 172.21.15.44:0/2337268849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc071c40 con 0x7f74cc072720 2024-07-17T08:46:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.450+0000 7f74d2a00640 1 -- 172.21.15.44:0/2337268849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc0ac010 con 0x7f74cc072720 2024-07-17T08:46:44.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.451+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f74c4011870 con 0x7f74cc072720 2024-07-17T08:46:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.452+0000 7f74c9600640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f74ac076ea0 0x7f74ac079360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.452+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(365..365 src has 1..365) ==== 6991+0+0 (secure 0 0 0) 0x7f74c4099000 con 0x7f74cc072720 2024-07-17T08:46:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.452+0000 7f74d1600640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f74ac076ea0 0x7f74ac079360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.453+0000 7f74d1600640 1 --2- 172.21.15.44:0/2337268849 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f74ac076ea0 0x7f74ac079360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005bc0 tx=0x7f74c0005a70 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:44.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.456+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f74c4023020 con 0x7f74cc072720 2024-07-17T08:46:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.706+0000 7f74d2a00640 1 -- 172.21.15.44:0/2337268849 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "group": "mygrp07", "fs": "cephfs", "path": ".", "target": ["mon-mgr", ""]}) -- 0x7f74cc152d70 con 0x7f74ac076ea0 2024-07-17T08:46:44.717 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:44.715+0000 7febe8a00640 -1 mgr.server reply reply (22) Invalid argument No such subvol: mygrp07::None 2024-07-17T08:46:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.717+0000 7f74c9600640 1 -- 172.21.15.44:0/2337268849 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: -22 No such subvol: mygrp07::None) ==== 37+0+0 (secure 0 0 0) 0x7f74cc152d70 con 0x7f74ac076ea0 2024-07-17T08:46:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:Error EINVAL: No such subvol: mygrp07::None 2024-07-17T08:46:44.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-17T08:46:44.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --subvol sv07 --group mygrp07 --fs cephfs . 1m 2024-07-17T08:46:44.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.959+0000 7f34e9600640 1 -- 172.21.15.44:0/1805354008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 msgr2=0x7f34c8003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.959+0000 7f34e9600640 1 --2- 172.21.15.44:0/1805354008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34c8003ee0 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009b50 tx=0x7f34c8005830 comp rx=0 tx=0).stop 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.959+0000 7f34e9600640 1 -- 172.21.15.44:0/1805354008 shutdown_connections 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.959+0000 7f34e9600640 1 --2- 172.21.15.44:0/1805354008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34c8003ee0 unknown :-1 s=CLOSED pgs=3363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.959+0000 7f34e9600640 1 -- 172.21.15.44:0/1805354008 >> 172.21.15.44:0/1805354008 conn(0x7f34e406bf40 msgr2=0x7f34e406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.960+0000 7f34e9600640 1 -- 172.21.15.44:0/1805354008 shutdown_connections 2024-07-17T08:46:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.960+0000 7f34e9600640 1 -- 172.21.15.44:0/1805354008 wait complete. 2024-07-17T08:46:44.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.961+0000 7f34e9600640 1 Processor -- start 2024-07-17T08:46:44.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.961+0000 7f34e9600640 1 -- start start 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34e41f1ea0 0x7f34e41f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34e41f3440 0x7f34e40b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4073a40 con 0x7f34c8003ac0 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34e40738c0 con 0x7f34e41f3440 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.962+0000 7f34e9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34e40719f0 con 0x7f34e41f1ea0 2024-07-17T08:46:44.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40214/0 (socket says 172.21.15.44:40214) 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 -- 172.21.15.44:0/1340284560 learned_addr learned my addr 172.21.15.44:0/1340284560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3400640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34e41f1ea0 0x7f34e41f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 -- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34e41f1ea0 msgr2=0x7f34e41f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34e41f1ea0 0x7f34e41f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 -- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34e41f3440 msgr2=0x7f34e40b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e8c00640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34e41f3440 0x7f34e40b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34e41f3440 0x7f34e40b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.963+0000 7f34e3e00640 1 -- 172.21.15.44:0/1340284560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e41f2380 con 0x7f34c8003ac0 2024-07-17T08:46:44.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34e3e00640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009a70 tx=0x7f34d40336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4040020 con 0x7f34c8003ac0 2024-07-17T08:46:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34d4004310 con 0x7f34c8003ac0 2024-07-17T08:46:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4038040 con 0x7f34c8003ac0 2024-07-17T08:46:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e41f2660 con 0x7f34c8003ac0 2024-07-17T08:46:44.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.964+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41f2c20 con 0x7f34c8003ac0 2024-07-17T08:46:44.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.965+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e40cddf0 con 0x7f34c8003ac0 2024-07-17T08:46:44.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.966+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f34d402b8a0 con 0x7f34c8003ac0 2024-07-17T08:46:44.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.967+0000 7f34dbe00640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 0x7f34c4079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:44.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.967+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(366..366 src has 1..366) ==== 7136+0+0 (secure 0 0 0) 0x7f34d40b91a0 con 0x7f34c8003ac0 2024-07-17T08:46:44.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.968+0000 7f34e3400640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 0x7f34c4079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:44.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.969+0000 7f34e3400640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 0x7f34c4079300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f34e4140340 tx=0x7f34d0006a40 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:44.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:44.972+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34d404a020 con 0x7f34c8003ac0 2024-07-17T08:46:45.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.204+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "subvol": "sv07", "group": "mygrp07", "fs": "cephfs", "path": ".", "repeat": "1m", "target": ["mon-mgr", ""]}) -- 0x7f34e4152200 con 0x7f34c4076e40 2024-07-17T08:46:45.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.425+0000 7f34dbe00640 1 -- 172.21.15.44:0/1340284560 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+88 (secure 0 0 0) 0x7f34e4152200 con 0x7f34c4076e40 2024-07-17T08:46:45.426 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule removed for path /volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30/.. 2024-07-17T08:46:45.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.428+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 msgr2=0x7f34c4079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.428+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 0x7f34c4079300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f34e4140340 tx=0x7f34d0006a40 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.428+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 msgr2=0x7f34e41f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.428+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009a70 tx=0x7f34d40336f0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 shutdown_connections 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34e41f3440 0x7f34e40b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34c4076e40 0x7f34c4079300 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34e41f1ea0 0x7f34e41f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 --2- 172.21.15.44:0/1340284560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c8003ac0 0x7f34e41f5eb0 unknown :-1 s=CLOSED pgs=3364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 >> 172.21.15.44:0/1340284560 conn(0x7f34e406bf40 msgr2=0x7f34e415cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:45.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 shutdown_connections 2024-07-17T08:46:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.429+0000 7f34e9600640 1 -- 172.21.15.44:0/1340284560 wait complete. 2024-07-17T08:46:45.451 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30/.. 2024-07-17T08:46:45.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv07 --group_name mygrp07 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 -- 172.21.15.44:0/2595725699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e6006e690 msgr2=0x7f6e6015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 --2- 172.21.15.44:0/2595725699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e6006e690 0x7f6e6015dad0 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005fd0 tx=0x7f6e4c02a4a0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 -- 172.21.15.44:0/2595725699 shutdown_connections 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 --2- 172.21.15.44:0/2595725699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e600ae8b0 0x7f6e6015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 --2- 172.21.15.44:0/2595725699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e6006e690 0x7f6e6015dad0 unknown :-1 s=CLOSED pgs=3365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 -- 172.21.15.44:0/2595725699 >> 172.21.15.44:0/2595725699 conn(0x7f6e600b0eb0 msgr2=0x7f6e600b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 -- 172.21.15.44:0/2595725699 shutdown_connections 2024-07-17T08:46:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.670+0000 7f6e68000640 1 -- 172.21.15.44:0/2595725699 wait complete. 2024-07-17T08:46:45.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.672+0000 7f6e68000640 1 Processor -- start 2024-07-17T08:46:45.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.672+0000 7f6e68000640 1 -- start start 2024-07-17T08:46:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e6006e690 0x7f6e601604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e600ae8b0 0x7f6e60140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:45.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600b26e0 con 0x7f6e60154560 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600b2560 con 0x7f6e600ae8b0 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e68000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600b2860 con 0x7f6e6006e690 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e66000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e66000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40234/0 (socket says 172.21.15.44:40234) 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.673+0000 7f6e66000640 1 -- 172.21.15.44:0/1693812715 learned_addr learned my addr 172.21.15.44:0/1693812715 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e64c00640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e600ae8b0 0x7f6e60140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e65600640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e6006e690 0x7f6e601604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 -- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e6006e690 msgr2=0x7f6e601604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e6006e690 0x7f6e601604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 -- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e600ae8b0 msgr2=0x7f6e60140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e600ae8b0 0x7f6e60140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 -- 172.21.15.44:0/1693812715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e60144a10 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e66000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b9a0 tx=0x7f6e5800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.674+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58019070 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.675+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5800cc50 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.675+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580146a0 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.675+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e60143c50 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.675+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e60144210 con 0x7f6e60154560 2024-07-17T08:46:45.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.676+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e600b2e00 con 0x7f6e60154560 2024-07-17T08:46:45.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.677+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f6e5801c070 con 0x7f6e60154560 2024-07-17T08:46:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.679+0000 7f6e5d600640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 0x7f6e40079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.679+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f6e58098a10 con 0x7f6e60154560 2024-07-17T08:46:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.679+0000 7f6e65600640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 0x7f6e40079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.681+0000 7f6e65600640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 0x7f6e40079330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:45.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.683+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f6e58061600 con 0x7f6e60154560 2024-07-17T08:46:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.930+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv07", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7f6e601416f0 con 0x7f6e40076e70 2024-07-17T08:46:45.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.942+0000 7f6e5d600640 1 -- 172.21.15.44:0/1693812715 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6e601416f0 con 0x7f6e40076e70 2024-07-17T08:46:45.942 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30 2024-07-17T08:46:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.944+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 msgr2=0x7f6e40079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.944+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 0x7f6e40079330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e54006fd0 tx=0x7f6e54008040 comp rx=0 tx=0).stop 2024-07-17T08:46:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.944+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 msgr2=0x7f6e601411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:45.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.944+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b9a0 tx=0x7f6e5800be60 comp rx=0 tx=0).stop 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 shutdown_connections 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e60154560 0x7f6e601411b0 unknown :-1 s=CLOSED pgs=3366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e600ae8b0 0x7f6e60140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f6e40076e70 0x7f6e40079330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 --2- 172.21.15.44:0/1693812715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e6006e690 0x7f6e601604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:45.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 >> 172.21.15.44:0/1693812715 conn(0x7f6e600b0eb0 msgr2=0x7f6e6006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 shutdown_connections 2024-07-17T08:46:45.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:45.945+0000 7f6e68000640 1 -- 172.21.15.44:0/1693812715 wait complete. 2024-07-17T08:46:45.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:46:45.965 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls volumes/mygrp07/sv07/8678920d-d9af-46a7-bf12-f722f5eb7f30/../.snap) 2024-07-17T08:46:46.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv07 --group_name mygrp07 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 -- 172.21.15.44:0/1430250016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed815af70 msgr2=0x7fbed806e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 --2- 172.21.15.44:0/1430250016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed815af70 0x7fbed806e570 secure :-1 s=READY pgs=3367 cs=0 l=1 rev1=1 crypto rx=0x7fbed80b3de0 tx=0x7fbecc02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 -- 172.21.15.44:0/1430250016 shutdown_connections 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 --2- 172.21.15.44:0/1430250016 >> v2:172.21.15.175:3300/0 conn(0x7fbed8072a50 0x7fbed806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 --2- 172.21.15.44:0/1430250016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed815af70 0x7fbed806e570 unknown :-1 s=CLOSED pgs=3367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 -- 172.21.15.44:0/1430250016 >> 172.21.15.44:0/1430250016 conn(0x7fbed806dad0 msgr2=0x7fbed8073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 -- 172.21.15.44:0/1430250016 shutdown_connections 2024-07-17T08:46:46.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.237+0000 7fbede600640 1 -- 172.21.15.44:0/1430250016 wait complete. 2024-07-17T08:46:46.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.239+0000 7fbede600640 1 Processor -- start 2024-07-17T08:46:46.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.239+0000 7fbede600640 1 -- start start 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbed806c480 0x7fbed8146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 0x7fbed8147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbed80b4170 con 0x7fbed8072a50 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbed80b3ff0 con 0x7fbed815af70 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbede600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbed81616d0 con 0x7fbed806c480 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbed7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbed7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40256/0 (socket says 172.21.15.44:40256) 2024-07-17T08:46:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.240+0000 7fbed7e00640 1 -- 172.21.15.44:0/3919268467 learned_addr learned my addr 172.21.15.44:0/3919268467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 -- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbed806c480 msgr2=0x7fbed8146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbed806c480 0x7fbed8146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbeddc00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 0x7fbed8147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 -- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 msgr2=0x7fbed8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 0x7fbed8147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 -- 172.21.15.44:0/3919268467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed809c5c0 con 0x7fbed8072a50 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbeddc00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 0x7fbed8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed7e00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c970 tx=0x7fbec400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007bb0 con 0x7fbed8072a50 2024-07-17T08:46:46.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbec4005020 con 0x7fbed8072a50 2024-07-17T08:46:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007e60 con 0x7fbed8072a50 2024-07-17T08:46:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.241+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed809c8a0 con 0x7fbed8072a50 2024-07-17T08:46:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.242+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed8146eb0 con 0x7fbed8072a50 2024-07-17T08:46:46.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.243+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed8161ac0 con 0x7fbed8072a50 2024-07-17T08:46:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.244+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7fbec4005470 con 0x7fbed8072a50 2024-07-17T08:46:46.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.245+0000 7fbed4c00640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 0x7fbeb8079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.245+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7fbec40992c0 con 0x7fbed8072a50 2024-07-17T08:46:46.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.245+0000 7fbedd200640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 0x7fbeb8079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.246+0000 7fbedd200640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 0x7fbeb8079300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbecc033000 tx=0x7fbecc0046c0 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:46.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.249+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fbec4014030 con 0x7fbed8072a50 2024-07-17T08:46:46.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.493+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv07", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7fbed815e320 con 0x7fbeb8076e40 2024-07-17T08:46:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.533+0000 7fbed4c00640 1 -- 172.21.15.44:0/3919268467 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbed815e320 con 0x7fbeb8076e40 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 msgr2=0x7fbeb8079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 0x7fbeb8079300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbecc033000 tx=0x7fbecc0046c0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 msgr2=0x7fbed80b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c970 tx=0x7fbec400ce30 comp rx=0 tx=0).stop 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 shutdown_connections 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbed815af70 0x7fbed8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbed8072a50 0x7fbed80b1c60 unknown :-1 s=CLOSED pgs=3368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fbeb8076e40 0x7fbeb8079300 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 --2- 172.21.15.44:0/3919268467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbed806c480 0x7fbed8146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 >> 172.21.15.44:0/3919268467 conn(0x7fbed806dad0 msgr2=0x7fbed8160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 shutdown_connections 2024-07-17T08:46:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.537+0000 7fbede600640 1 -- 172.21.15.44:0/3919268467 wait complete. 2024-07-17T08:46:46.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs mygrp07 2024-07-17T08:46:46.561 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:46.559+0000 7febdae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:46.561 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:46.559+0000 7febdae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:46.561 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:46.559+0000 7febdae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:46.561 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:46.559+0000 7febdae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:46.561 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:46.559+0000 7febdae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 -- 172.21.15.44:0/484444974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 msgr2=0x7f34d00ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 --2- 172.21.15.44:0/484444974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d00ae340 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7f34d0070830 tx=0x7f34bc02a5f0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 -- 172.21.15.44:0/484444974 shutdown_connections 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 --2- 172.21.15.44:0/484444974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d00ae340 unknown :-1 s=CLOSED pgs=3371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 --2- 172.21.15.44:0/484444974 >> v2:172.21.15.175:3300/0 conn(0x7f34d006f4d0 0x7f34d015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 -- 172.21.15.44:0/484444974 >> 172.21.15.44:0/484444974 conn(0x7f34d00cf0d0 msgr2=0x7f34d00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:46.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 -- 172.21.15.44:0/484444974 shutdown_connections 2024-07-17T08:46:46.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.755+0000 7f34d6800640 1 -- 172.21.15.44:0/484444974 wait complete. 2024-07-17T08:46:46.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.757+0000 7f34d6800640 1 Processor -- start 2024-07-17T08:46:46.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.757+0000 7f34d6800640 1 -- start start 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 0x7f34d014b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34d006f4d0 0x7f34d00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34d006d9d0 con 0x7f34d01503c0 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34d006d850 con 0x7f34d006d420 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34d006db50 con 0x7f34d006f4d0 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d5e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40270/0 (socket says 172.21.15.44:40270) 2024-07-17T08:46:46.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.758+0000 7f34d5e00640 1 -- 172.21.15.44:0/2954502084 learned_addr learned my addr 172.21.15.44:0/2954502084 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 -- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34d006f4d0 msgr2=0x7f34d00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34d006f4d0 0x7f34d00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5400640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 0x7f34d014b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 -- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 msgr2=0x7f34d014b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 0x7f34d014b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 -- 172.21.15.44:0/2954502084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d0071480 con 0x7f34d01503c0 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5400640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 0x7f34d014b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34d5e00640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c7a0 tx=0x7f34c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.759+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003b70 con 0x7f34d01503c0 2024-07-17T08:46:46.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.760+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c8003d30 con 0x7f34d01503c0 2024-07-17T08:46:46.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.760+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c800aab0 con 0x7f34d01503c0 2024-07-17T08:46:46.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.760+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d0071760 con 0x7f34d01503c0 2024-07-17T08:46:46.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.760+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d014bc00 con 0x7f34d01503c0 2024-07-17T08:46:46.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.761+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d0071f10 con 0x7f34d01503c0 2024-07-17T08:46:46.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.762+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f34c800ac50 con 0x7f34d01503c0 2024-07-17T08:46:46.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.763+0000 7f34cd600640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 0x7f34b0079330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:46.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.763+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f34c8098bf0 con 0x7f34d01503c0 2024-07-17T08:46:46.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.764+0000 7f34d5400640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 0x7f34b0079330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:46.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.765+0000 7f34d5400640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 0x7f34b0079330 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006fd0 tx=0x7f34c4008040 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:46.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:46.768+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f34c8061750 con 0x7f34d01503c0 2024-07-17T08:46:47.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.012+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 --> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "mygrp07", "target": ["mon-mgr", ""]}) -- 0x7f34d015a450 con 0x7f34b0076e70 2024-07-17T08:46:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.027+0000 7f34cd600640 1 -- 172.21.15.44:0/2954502084 <== mgr.11518 v2:172.21.15.175:6834/3626582469 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34d015a450 con 0x7f34b0076e70 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.029+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 msgr2=0x7f34b0079330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.029+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 0x7f34b0079330 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006fd0 tx=0x7f34c4008040 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.029+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 msgr2=0x7f34d015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.029+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c7a0 tx=0x7f34c800cc60 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 shutdown_connections 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34d01503c0 0x7f34d015aa80 unknown :-1 s=CLOSED pgs=3372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f34b0076e70 0x7f34b0079330 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34d006f4d0 0x7f34d00cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 --2- 172.21.15.44:0/2954502084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34d006d420 0x7f34d014b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 >> 172.21.15.44:0/2954502084 conn(0x7f34d00cf0d0 msgr2=0x7f34d00c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 shutdown_connections 2024-07-17T08:46:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.030+0000 7f34d6800640 1 -- 172.21.15.44:0/2954502084 wait complete. 2024-07-17T08:46:47.049 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-07-17T08:46:47.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 -- 172.21.15.44:0/2484511899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 msgr2=0x7f2d980051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 --2- 172.21.15.44:0/2484511899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2d980051a0 secure :-1 s=READY pgs=3374 cs=0 l=1 rev1=1 crypto rx=0x7f2db40724e0 tx=0x7f2d98005830 comp rx=0 tx=0).stop 2024-07-17T08:46:47.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9200640 1 -- 172.21.15.44:0/2484511899 reap_dead start 2024-07-17T08:46:47.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 -- 172.21.15.44:0/2484511899 shutdown_connections 2024-07-17T08:46:47.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 -- 172.21.15.44:0/2484511899 >> 172.21.15.44:0/2484511899 conn(0x7f2db4073920 msgr2=0x7f2db4073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:47.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 -- 172.21.15.44:0/2484511899 shutdown_connections 2024-07-17T08:46:47.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.264+0000 7f2db9c00640 1 -- 172.21.15.44:0/2484511899 wait complete. 2024-07-17T08:46:47.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.267+0000 7f2db9c00640 1 Processor -- start 2024-07-17T08:46:47.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.267+0000 7f2db9c00640 1 -- start start 2024-07-17T08:46:47.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:47.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 0x7f2db41f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:47.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 0x7f2db41574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:47.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4154610 con 0x7f2d98004d40 2024-07-17T08:46:47.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4154790 con 0x7f2db4151730 2024-07-17T08:46:47.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2db4154910 con 0x7f2db41f4050 2024-07-17T08:46:47.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:47.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40290/0 (socket says 172.21.15.44:40290) 2024-07-17T08:46:47.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.268+0000 7f2db3e00640 1 -- 172.21.15.44:0/1352120791 learned_addr learned my addr 172.21.15.44:0/1352120791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3400640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 0x7f2db41f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db9200640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 0x7f2db41574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 -- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 msgr2=0x7f2db41574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 0x7f2db41574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 -- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 msgr2=0x7f2db41f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 0x7f2db41f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 -- 172.21.15.44:0/1352120791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4155b20 con 0x7f2d98004d40 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db9200640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 0x7f2db41574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3400640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 0x7f2db41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2db3e00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7f2da4011a50 tx=0x7f2da4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.269+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da401f070 con 0x7f2d98004d40 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.270+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da400fdc0 con 0x7f2d98004d40 2024-07-17T08:46:47.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.270+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40126b0 con 0x7f2d98004d40 2024-07-17T08:46:47.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.270+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db4155dc0 con 0x7f2d98004d40 2024-07-17T08:46:47.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.270+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41f4850 con 0x7f2d98004d40 2024-07-17T08:46:47.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.271+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db415b200 con 0x7f2d98004d40 2024-07-17T08:46:47.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.272+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 225) ==== 99056+0+0 (secure 0 0 0) 0x7f2da4022070 con 0x7f2d98004d40 2024-07-17T08:46:47.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.273+0000 7f2dabe00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 0x7f2d94079300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:47.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.273+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f2da40997c0 con 0x7f2d98004d40 2024-07-17T08:46:47.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.274+0000 7f2db3400640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 0x7f2d94079300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:47.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.275+0000 7f2db3400640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 0x7f2d94079300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2da0006fd0 tx=0x7f2da0006b10 comp rx=0 tx=0).ready entity=mgr.11518 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.279+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2da4061d60 con 0x7f2d98004d40 2024-07-17T08:46:47.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.542+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f2db415aec0 con 0x7f2d98004d40 2024-07-17T08:46:47.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.607+0000 7f2dabe00640 1 -- 172.21.15.44:0/1352120791 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v226) ==== 93+0+0 (secure 0 0 0) 0x7f2da40662b0 con 0x7f2d98004d40 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 msgr2=0x7f2d94079300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 0x7f2d94079300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2da0006fd0 tx=0x7f2da0006b10 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 msgr2=0x7f2db40b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7f2da4011a50 tx=0x7f2da4011f10 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 shutdown_connections 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2d94076e40 0x7f2d94079300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2db41f4050 0x7f2db41574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2db4151730 0x7f2db41f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 --2- 172.21.15.44:0/1352120791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98004d40 0x7f2db40b30b0 unknown :-1 s=CLOSED pgs=3375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:47.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.610+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 >> 172.21.15.44:0/1352120791 conn(0x7f2db4073920 msgr2=0x7f2db415c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:47.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.611+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 shutdown_connections 2024-07-17T08:46:47.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:47.611+0000 7f2db9c00640 1 -- 172.21.15.44:0/1352120791 wait complete. 2024-07-17T08:46:47.636 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:46:47.636 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:46:47.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:46:47.828 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:46:47.828 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:46:47.829 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:46:47.829 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:46:47.880 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:46:47.880 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:46:47.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 -- 172.21.15.44:0/1325862624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 msgr2=0x7f2634003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 --2- 172.21.15.44:0/1325862624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f2634003ec0 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f20 tx=0x7f2634005830 comp rx=0 tx=0).stop 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 -- 172.21.15.44:0/1325862624 shutdown_connections 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 --2- 172.21.15.44:0/1325862624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f2634003ec0 unknown :-1 s=CLOSED pgs=3381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 -- 172.21.15.44:0/1325862624 >> 172.21.15.44:0/1325862624 conn(0x7f26500b4ff0 msgr2=0x7f26500b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:48.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 -- 172.21.15.44:0/1325862624 shutdown_connections 2024-07-17T08:46:48.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.214+0000 7f2655c00640 1 -- 172.21.15.44:0/1325862624 wait complete. 2024-07-17T08:46:48.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.216+0000 7f2655c00640 1 Processor -- start 2024-07-17T08:46:48.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.216+0000 7f2655c00640 1 -- start start 2024-07-17T08:46:48.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 0x7f26500ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26500caaf0 0x7f26501f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26500b2800 con 0x7f2634003ac0 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26500b2680 con 0x7f26500cd2e0 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26500cfdf0 con 0x7f26500caaf0 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f264fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f264fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40332/0 (socket says 172.21.15.44:40332) 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f264fe00640 1 -- 172.21.15.44:0/1466004516 learned_addr learned my addr 172.21.15.44:0/1466004516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.217+0000 7f2655200640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26500caaf0 0x7f26501f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f2647400640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 0x7f26500ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26500caaf0 msgr2=0x7f26501f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26500caaf0 0x7f26501f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 msgr2=0x7f26500ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 0x7f26500ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 -- 172.21.15.44:0/1466004516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500cc560 con 0x7f2634003ac0 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f2647400640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 0x7f26500ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:48.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264fe00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f264000d3d0 tx=0x7f2640003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640031040 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2640044070 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400043d0 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.218+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26501f4a50 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.219+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26500ccae0 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.220+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26500d0350 con 0x7f2634003ac0 2024-07-17T08:46:48.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.221+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7f264003c070 con 0x7f2634003ac0 2024-07-17T08:46:48.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.222+0000 7f264cc00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 0x7f262c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.222+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f26400b96a0 con 0x7f2634003ac0 2024-07-17T08:46:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.222+0000 7f2647400640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 msgr2=0x7f262c0792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:48.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.222+0000 7f2647400640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 0x7f262c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:48.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.226+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2640082200 con 0x7f2634003ac0 2024-07-17T08:46:48.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.424+0000 7f2647400640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 msgr2=0x7f262c0792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:48.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.424+0000 7f2647400640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 0x7f262c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.450+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26500caed0 con 0x7f2634003ac0 2024-07-17T08:46:48.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.451+0000 7f264cc00640 1 -- 172.21.15.44:0/1466004516 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v367) ==== 80+0+358 (secure 0 0 0) 0x7f2640081b00 con 0x7f2634003ac0 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/1629162363 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/3973742197 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/4233314912 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6837/945953093 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:6834/945953093 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.175:0/2555277869 2024-07-18T08:45:43.974493+0000 2024-07-17T08:46:48.452 INFO:teuthology.orchestra.run.smithi044.stderr:listed 6 entries 2024-07-17T08:46:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 msgr2=0x7f262c0792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:48.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 0x7f262c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 msgr2=0x7f26500c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f264000d3d0 tx=0x7f2640003c20 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 shutdown_connections 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f262c076e20 0x7f262c0792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.453+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26500caaf0 0x7f26501f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.454+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26500cd2e0 0x7f26500ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.454+0000 7f2655c00640 1 --2- 172.21.15.44:0/1466004516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2634003ac0 0x7f26500c9c00 unknown :-1 s=CLOSED pgs=3382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.454+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 >> 172.21.15.44:0/1466004516 conn(0x7f26500b4ff0 msgr2=0x7f265006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.454+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 shutdown_connections 2024-07-17T08:46:48.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.454+0000 7f2655c00640 1 -- 172.21.15.44:0/1466004516 wait complete. 2024-07-17T08:46:48.472 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:46:48.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:46:48.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:46:48.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:46:48.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:46:48.555 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:46:48.594 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:46:48.595 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:46:48.595 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:46:48.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:46:48.639 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:46:48.639 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:46:48.639 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:46:48.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:46:48.680 INFO:tasks.cephfs.mount:Terminating background process 2024-07-17T08:46:48.680 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:46:48.680 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:46:48.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:46:48.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:46:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.923+0000 7fa13cc00640 1 -- 172.21.15.44:0/2143936623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380b8c80 msgr2=0x7fa1380b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.923+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2143936623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380b8c80 0x7fa1380b4ec0 secure :-1 s=READY pgs=3383 cs=0 l=1 rev1=1 crypto rx=0x7fa128009900 tx=0x7fa12802a560 comp rx=0 tx=0).stop 2024-07-17T08:46:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 -- 172.21.15.44:0/2143936623 shutdown_connections 2024-07-17T08:46:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2143936623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 0x7fa13815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2143936623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380b8c80 0x7fa1380b4ec0 unknown :-1 s=CLOSED pgs=3383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 -- 172.21.15.44:0/2143936623 >> 172.21.15.44:0/2143936623 conn(0x7fa1380cfec0 msgr2=0x7fa1380d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 -- 172.21.15.44:0/2143936623 shutdown_connections 2024-07-17T08:46:48.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.924+0000 7fa13cc00640 1 -- 172.21.15.44:0/2143936623 wait complete. 2024-07-17T08:46:48.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.925+0000 7fa13cc00640 1 Processor -- start 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 -- start start 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 0x7fa1380cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1380b8c80 0x7fa138156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa13806e0c0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.926+0000 7fa13cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1380b92f0 con 0x7fa1380b8c80 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa13cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa13806e240 con 0x7fa1380b5400 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33492/0 (socket says 172.21.15.44:33492) 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 -- 172.21.15.44:0/2741125108 learned_addr learned my addr 172.21.15.44:0/2741125108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:48.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137400640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 0x7fa1380cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa136a00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1380b8c80 0x7fa138156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 msgr2=0x7fa1380cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 0x7fa1380cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1380b8c80 msgr2=0x7fa138156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1380b8c80 0x7fa138156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.927+0000 7fa137e00640 1 -- 172.21.15.44:0/2741125108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1380cbbf0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa137e00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7fa13000eaa0 tx=0x7fa13000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13000cdb0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa130004c10 con 0x7fa1380ce5c0 2024-07-17T08:46:48.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130005020 con 0x7fa1380ce5c0 2024-07-17T08:46:48.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1380cbed0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.928+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1380cc490 con 0x7fa1380ce5c0 2024-07-17T08:46:48.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.929+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13806e710 con 0x7fa1380ce5c0 2024-07-17T08:46:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.931+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7fa1300051c0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.931+0000 7fa12f400640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 0x7fa1180792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.932+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7fa1300996c0 con 0x7fa1380ce5c0 2024-07-17T08:46:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.932+0000 7fa137400640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 msgr2=0x7fa1180792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:48.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.932+0000 7fa137400640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 0x7fa1180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:48.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:48.936+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa130016030 con 0x7fa1380ce5c0 2024-07-17T08:46:49.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.134+0000 7fa137400640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 msgr2=0x7fa1180792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:49.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.134+0000 7fa137400640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 0x7fa1180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:49.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.181+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13815e5f0 con 0x7fa1380ce5c0 2024-07-17T08:46:49.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.182+0000 7fa12f400640 1 -- 172.21.15.44:0/2741125108 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7fa13006a350 con 0x7fa1380ce5c0 2024-07-17T08:46:49.183 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:49.184 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:46:49.184 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:46:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 msgr2=0x7fa1180792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 0x7fa1180792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 msgr2=0x7fa138155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7fa13000eaa0 tx=0x7fa13000ef60 comp rx=0 tx=0).stop 2024-07-17T08:46:49.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 shutdown_connections 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1380ce5c0 0x7fa138155870 unknown :-1 s=CLOSED pgs=3384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1380b8c80 0x7fa138156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fa118076df0 0x7fa1180792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 --2- 172.21.15.44:0/2741125108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1380b5400 0x7fa1380cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.185+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 >> 172.21.15.44:0/2741125108 conn(0x7fa1380cfec0 msgr2=0x7fa1380d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.186+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 shutdown_connections 2024-07-17T08:46:49.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.186+0000 7fa13cc00640 1 -- 172.21.15.44:0/2741125108 wait complete. 2024-07-17T08:46:49.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:46:49.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 -- 172.21.15.44:0/2453237314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f758006e090 msgr2=0x7f758006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 --2- 172.21.15.44:0/2453237314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f758006e090 0x7f758006e4a0 secure :-1 s=READY pgs=3386 cs=0 l=1 rev1=1 crypto rx=0x7f75800cbb00 tx=0x7f757802a530 comp rx=0 tx=0).stop 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 -- 172.21.15.44:0/2453237314 shutdown_connections 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 --2- 172.21.15.44:0/2453237314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f758006e090 0x7f758006e4a0 unknown :-1 s=CLOSED pgs=3386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 --2- 172.21.15.44:0/2453237314 >> v2:172.21.15.175:3300/0 conn(0x7f75800ac3e0 0x7f75800ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.400+0000 7f7587c00640 1 -- 172.21.15.44:0/2453237314 >> 172.21.15.44:0/2453237314 conn(0x7f75800b0fb0 msgr2=0x7f75800b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.401+0000 7f7587c00640 1 -- 172.21.15.44:0/2453237314 shutdown_connections 2024-07-17T08:46:49.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.401+0000 7f7587c00640 1 -- 172.21.15.44:0/2453237314 wait complete. 2024-07-17T08:46:49.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.403+0000 7f7587c00640 1 Processor -- start 2024-07-17T08:46:49.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.403+0000 7f7587c00640 1 -- start start 2024-07-17T08:46:49.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f758006e090 0x7f75800b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ac3e0 0x7f75800b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7580153fb0 con 0x7f7580153a50 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7580153e30 con 0x7f75800ac3e0 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7587c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f758006f710 con 0x7f758006e090 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7585c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7585c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33510/0 (socket says 172.21.15.44:33510) 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.405+0000 7f7585c00640 1 -- 172.21.15.44:0/1619609311 learned_addr learned my addr 172.21.15.44:0/1619609311 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7577e00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ac3e0 0x7f75800b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585200640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f758006e090 0x7f75800b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f758006e090 msgr2=0x7f75800b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f758006e090 0x7f75800b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ac3e0 msgr2=0x7f75800b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ac3e0 0x7f75800b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 -- 172.21.15.44:0/1619609311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75800b3d70 con 0x7f7580153a50 2024-07-17T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.406+0000 7f7585c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7f757802ab20 tx=0x7f7578032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.407+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578040070 con 0x7f7580153a50 2024-07-17T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.407+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7578005310 con 0x7f7580153a50 2024-07-17T08:46:49.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.407+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75800b4050 con 0x7f7580153a50 2024-07-17T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.407+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578005c90 con 0x7f7580153a50 2024-07-17T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.407+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75800b3190 con 0x7f7580153a50 2024-07-17T08:46:49.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.408+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75800b4290 con 0x7f7580153a50 2024-07-17T08:46:49.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.409+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7f7578039040 con 0x7f7580153a50 2024-07-17T08:46:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.411+0000 7f757d600640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 0x7f755c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.411+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f75780b9c80 con 0x7f7580153a50 2024-07-17T08:46:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.411+0000 7f7585200640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 msgr2=0x7f755c0792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:49.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.411+0000 7f7585200640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 0x7f755c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:49.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.414+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f75800b4290 con 0x7f7580153a50 2024-07-17T08:46:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.613+0000 7f7585200640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 msgr2=0x7f755c0792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:49.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.613+0000 7f7585200640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 0x7f755c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.660+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75800ac870 con 0x7f7580153a50 2024-07-17T08:46:49.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.661+0000 7f757d600640 1 -- 172.21.15.44:0/1619609311 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7f75780bb090 con 0x7f7580153a50 2024-07-17T08:46:49.662 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:49.662 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:46:49.662 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:46:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 msgr2=0x7f755c0792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 0x7f755c0792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 msgr2=0x7f75800c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7f757802ab20 tx=0x7f7578032c10 comp rx=0 tx=0).stop 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 shutdown_connections 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7580153a50 0x7f75800c8630 unknown :-1 s=CLOSED pgs=3387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75800ac3e0 0x7f75800b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f755c076e20 0x7f755c0792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 --2- 172.21.15.44:0/1619609311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f758006e090 0x7f75800b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.664+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 >> 172.21.15.44:0/1619609311 conn(0x7f75800b0fb0 msgr2=0x7f758014ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.665+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 shutdown_connections 2024-07-17T08:46:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.665+0000 7f7587c00640 1 -- 172.21.15.44:0/1619609311 wait complete. 2024-07-17T08:46:49.684 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 -- 172.21.15.44:0/536678598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 msgr2=0x7fe46c0cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 --2- 172.21.15.44:0/536678598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c0cf340 secure :-1 s=READY pgs=3388 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0cef70 tx=0x7fe45c02a560 comp rx=0 tx=0).stop 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 -- 172.21.15.44:0/536678598 shutdown_connections 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 --2- 172.21.15.44:0/536678598 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c06efd0 0x7fe46c15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 --2- 172.21.15.44:0/536678598 >> v2:172.21.15.175:3300/0 conn(0x7fe46c15b3c0 0x7fe46c0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 --2- 172.21.15.44:0/536678598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c0cf340 unknown :-1 s=CLOSED pgs=3388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 -- 172.21.15.44:0/536678598 >> 172.21.15.44:0/536678598 conn(0x7fe46c0af940 msgr2=0x7fe46c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 -- 172.21.15.44:0/536678598 shutdown_connections 2024-07-17T08:46:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.865+0000 7fe471200640 1 -- 172.21.15.44:0/536678598 wait complete. 2024-07-17T08:46:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.867+0000 7fe471200640 1 Processor -- start 2024-07-17T08:46:49.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.867+0000 7fe471200640 1 -- start start 2024-07-17T08:46:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 0x7fe46c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c15b3c0 0x7fe46c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c0b5840 con 0x7fe46c154650 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c0b56c0 con 0x7fe46c06efd0 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe471200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe46c0cd860 con 0x7fe46c15b3c0 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe463e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe463e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33528/0 (socket says 172.21.15.44:33528) 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe463e00640 1 -- 172.21.15.44:0/2435460194 learned_addr learned my addr 172.21.15.44:0/2435460194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.868+0000 7fe469e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 0x7fe46c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe46a800640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c15b3c0 0x7fe46c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:49.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c15b3c0 msgr2=0x7fe46c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c15b3c0 0x7fe46c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 msgr2=0x7fe46c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 0x7fe46c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 -- 172.21.15.44:0/2435460194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c0ab670 con 0x7fe46c154650 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe469e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 0x7fe46c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:46:49.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe463e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7fe454005940 tx=0x7fe45400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454004220 con 0x7fe46c154650 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4540079b0 con 0x7fe46c154650 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c0ab8e0 con 0x7fe46c154650 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.869+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454007e00 con 0x7fe46c154650 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.870+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c0abea0 con 0x7fe46c154650 2024-07-17T08:46:49.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.871+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c0cf340 con 0x7fe46c154650 2024-07-17T08:46:49.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.872+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7fe454005020 con 0x7fe46c154650 2024-07-17T08:46:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.873+0000 7fe462000640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 0x7fe4480792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.873+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7fe4540991d0 con 0x7fe46c154650 2024-07-17T08:46:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.874+0000 7fe469e00640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 msgr2=0x7fe4480792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.874+0000 7fe469e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 0x7fe4480792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:49.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:49.877+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe454014030 con 0x7fe46c154650 2024-07-17T08:46:50.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.075+0000 7fe469e00640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 msgr2=0x7fe4480792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:50.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.076+0000 7fe469e00640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 0x7fe4480792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:50.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.086+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe46c0ad760 con 0x7fe46c154650 2024-07-17T08:46:50.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.089+0000 7fe462000640 1 -- 172.21.15.44:0/2435460194 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v367) ==== 74+0+18530 (secure 0 0 0) 0x7fe45408fe20 con 0x7fe46c154650 2024-07-17T08:46:50.090 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:50.090 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":367,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:46:45.572100+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":49,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:45:20.033990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":50,"pool_name":"cephfs_data","create_time":"2024-07-17T08:45:21.067621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"364","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"50.1d","mappings":[{"from":7,"to":1}]},{"pgid":"50.1f","mappings":[{"from":7,"to":0}]},{"pgid":"50.2a","mappings":[{"from":7,"to":1}]},{"pgid":"50.30","mappings":[{"from":7,"to":0}]},{"pgid":"50.39","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1629162363":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/3973742197":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/4233314912":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6837/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6834/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/2555277869":"2024-07-18T08:45:43.974493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:46:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 msgr2=0x7fe4480792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 0x7fe4480792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 msgr2=0x7fe46c159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7fe454005940 tx=0x7fe45400cdd0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 shutdown_connections 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe448076df0 0x7fe4480792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe46c15b3c0 0x7fe46c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c154650 0x7fe46c159730 unknown :-1 s=CLOSED pgs=3389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 --2- 172.21.15.44:0/2435460194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe46c06efd0 0x7fe46c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 >> 172.21.15.44:0/2435460194 conn(0x7fe46c0af940 msgr2=0x7fe46c0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.091+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 shutdown_connections 2024-07-17T08:46:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.092+0000 7fe471200640 1 -- 172.21.15.44:0/2435460194 wait complete. 2024-07-17T08:46:50.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:46:50.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1249260490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 msgr2=0x7fe3e815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1249260490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e815d2c0 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009860 tx=0x7fe3d402a540 comp rx=0 tx=0).stop 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1249260490 shutdown_connections 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1249260490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e815d2c0 unknown :-1 s=CLOSED pgs=3391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1249260490 >> 172.21.15.44:0/1249260490 conn(0x7fe3e80cfec0 msgr2=0x7fe3e80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:50.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1249260490 shutdown_connections 2024-07-17T08:46:50.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.304+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1249260490 wait complete. 2024-07-17T08:46:50.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.306+0000 7fe3e7e00640 1 Processor -- start 2024-07-17T08:46:50.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.306+0000 7fe3e7e00640 1 -- start start 2024-07-17T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e80b8c80 0x7fe3e80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3e80b9060 0x7fe3e80b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e815e2a0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e815e120 con 0x7fe3e80b8c80 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e815e420 con 0x7fe3e80b9060 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33542/0 (socket says 172.21.15.44:33542) 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 -- 172.21.15.44:0/1865882890 learned_addr learned my addr 172.21.15.44:0/1865882890 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e6000640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3e80b9060 0x7fe3e80b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e6a00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e80b8c80 0x7fe3e80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3e80b9060 msgr2=0x7fe3e80b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3e80b9060 0x7fe3e80b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e80b8c80 msgr2=0x7fe3e80c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.307+0000 7fe3e7400640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e80b8c80 0x7fe3e80c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3e7400640 1 -- 172.21.15.44:0/1865882890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81f4b10 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3e7400640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c440 tx=0x7fe3e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:50.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0011790 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3e0017070 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0011cc0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e81f4df0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.308+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e80c9310 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.309+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e80b4ec0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.310+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7fe3e001c460 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.311+0000 7fe3dea00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 0x7fe3c80792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.312+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7fe3e0098ab0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.312+0000 7fe3e6a00640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 msgr2=0x7fe3c80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:50.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.312+0000 7fe3e6a00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 0x7fe3c80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:50.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.316+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe3e00616e0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.514+0000 7fe3e6a00640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 msgr2=0x7fe3c80792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:50.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.514+0000 7fe3e6a00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 0x7fe3c80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.560+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3e80b5d70 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.561+0000 7fe3dea00640 1 -- 172.21.15.44:0/1865882890 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7fe3e0060fe0 con 0x7fe3e80ce5c0 2024-07-17T08:46:50.562 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:50.562 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:46:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 msgr2=0x7fe3c80792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 0x7fe3c80792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 msgr2=0x7fe3e80c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c440 tx=0x7fe3e000c900 comp rx=0 tx=0).stop 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 shutdown_connections 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3e80ce5c0 0x7fe3e80c6f50 unknown :-1 s=CLOSED pgs=3392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7fe3c8076df0 0x7fe3c80792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3e80b9060 0x7fe3e80b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 --2- 172.21.15.44:0/1865882890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3e80b8c80 0x7fe3e80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 >> 172.21.15.44:0/1865882890 conn(0x7fe3e80cfec0 msgr2=0x7fe3e815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:50.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 shutdown_connections 2024-07-17T08:46:50.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.564+0000 7fe3e7e00640 1 -- 172.21.15.44:0/1865882890 wait complete. 2024-07-17T08:46:50.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:46:50.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 -- 172.21.15.44:0/4144196260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 msgr2=0x7f2b740cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 --2- 172.21.15.44:0/4144196260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b740cd690 secure :-1 s=READY pgs=3393 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009ca0 tx=0x7f2b6c02aa30 comp rx=0 tx=0).stop 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 -- 172.21.15.44:0/4144196260 shutdown_connections 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 --2- 172.21.15.44:0/4144196260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b740cd690 unknown :-1 s=CLOSED pgs=3393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 --2- 172.21.15.44:0/4144196260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 0x7f2b740ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 -- 172.21.15.44:0/4144196260 >> 172.21.15.44:0/4144196260 conn(0x7f2b740b0fb0 msgr2=0x7f2b740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 -- 172.21.15.44:0/4144196260 shutdown_connections 2024-07-17T08:46:50.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.776+0000 7f2b79400640 1 -- 172.21.15.44:0/4144196260 wait complete. 2024-07-17T08:46:50.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.778+0000 7f2b79400640 1 Processor -- start 2024-07-17T08:46:50.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.778+0000 7f2b79400640 1 -- start start 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 0x7f2b74154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 0x7f2b7415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cdf90 con 0x7f2b740ac930 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740cde10 con 0x7f2b740adcd0 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b79400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740ce110 con 0x7f2b741522c0 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b72800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b72800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33556/0 (socket says 172.21.15.44:33556) 2024-07-17T08:46:50.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.779+0000 7f2b72800640 1 -- 172.21.15.44:0/2549205983 learned_addr learned my addr 172.21.15.44:0/2549205983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:50.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b71e00640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 0x7f2b74154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b73200640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 0x7f2b7415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 msgr2=0x7f2b7415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 0x7f2b7415a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 msgr2=0x7f2b74154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 0x7f2b74154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 -- 172.21.15.44:0/2549205983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b740b2600 con 0x7f2b740ac930 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b71e00640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 0x7f2b74154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b72800640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400d6d0 tx=0x7f2b6400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64017070 con 0x7f2b740ac930 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b64005330 con 0x7f2b740ac930 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64009820 con 0x7f2b740ac930 2024-07-17T08:46:50.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.780+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b740b28e0 con 0x7f2b740ac930 2024-07-17T08:46:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.781+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7414bbe0 con 0x7f2b740ac930 2024-07-17T08:46:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.782+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b740cd930 con 0x7f2b740ac930 2024-07-17T08:46:50.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.783+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7f2b64009bd0 con 0x7f2b740ac930 2024-07-17T08:46:50.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.784+0000 7f2b6aa00640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 0x7f2b500792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.784+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f2b64098fe0 con 0x7f2b740ac930 2024-07-17T08:46:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.784+0000 7f2b71e00640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 msgr2=0x7f2b500792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:50.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.784+0000 7f2b71e00640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 0x7f2b500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:50.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.788+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2b740cd930 con 0x7f2b740ac930 2024-07-17T08:46:50.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.986+0000 7f2b71e00640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 msgr2=0x7f2b500792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:50.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:50.986+0000 7f2b71e00640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 0x7f2b500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:51.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.034+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b740ce680 con 0x7f2b740ac930 2024-07-17T08:46:51.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.035+0000 7f2b6aa00640 1 -- 172.21.15.44:0/2549205983 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5244 (secure 0 0 0) 0x7f2b64061ae0 con 0x7f2b740ac930 2024-07-17T08:46:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:51.036 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":209,"btime":"2024-07-17T08:45:25:651363+0000","default_fscid":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:45:22.106720+0000","modified":"2024-07-17T08:45:25.156126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14406":{"gid":14406,"name":"a","rank":0,"incarnation":205,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6834/2831509556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2831509556},{"type":"v1","addr":"172.21.15.44:6834","nonce":2831509556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[50],"metadata_pool":49,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14406,"qdb_cluster":[14406]},"id":24}]} 2024-07-17T08:46:51.036 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 209 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 msgr2=0x7f2b500792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 0x7f2b500792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 msgr2=0x7f2b7414b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 secure :-1 s=READY pgs=3394 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400d6d0 tx=0x7f2b6400db90 comp rx=0 tx=0).stop 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 shutdown_connections 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f2b50076df0 0x7f2b500792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b741522c0 0x7f2b7415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b740adcd0 0x7f2b74154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 --2- 172.21.15.44:0/2549205983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b740ac930 0x7f2b7414b6a0 unknown :-1 s=CLOSED pgs=3394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 >> 172.21.15.44:0/2549205983 conn(0x7f2b740b0fb0 msgr2=0x7f2b74073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 shutdown_connections 2024-07-17T08:46:51.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.038+0000 7f2b79400640 1 -- 172.21.15.44:0/2549205983 wait complete. 2024-07-17T08:46:51.058 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:46:51.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 -- 172.21.15.44:0/2058045943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 msgr2=0x7f74a40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 --2- 172.21.15.44:0/2058045943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40b4ec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7498009be0 tx=0x7f749802a8d0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 -- 172.21.15.44:0/2058045943 shutdown_connections 2024-07-17T08:46:51.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 --2- 172.21.15.44:0/2058045943 >> v2:172.21.15.175:3300/0 conn(0x7f74a40b5400 0x7f74a415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 --2- 172.21.15.44:0/2058045943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40b4ec0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 -- 172.21.15.44:0/2058045943 >> 172.21.15.44:0/2058045943 conn(0x7f74a40cfec0 msgr2=0x7f74a40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:51.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 -- 172.21.15.44:0/2058045943 shutdown_connections 2024-07-17T08:46:51.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.246+0000 7f74aa800640 1 -- 172.21.15.44:0/2058045943 wait complete. 2024-07-17T08:46:51.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.248+0000 7f74aa800640 1 Processor -- start 2024-07-17T08:46:51.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.248+0000 7f74aa800640 1 -- start start 2024-07-17T08:46:51.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a40b5400 0x7f74a40c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74a406e0c0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74a40b92f0 con 0x7f74a40b5400 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74aa800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74a406e240 con 0x7f74a40b8c80 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74a9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74a9e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33564/0 (socket says 172.21.15.44:33564) 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74a9e00640 1 -- 172.21.15.44:0/48800316 learned_addr learned my addr 172.21.15.44:0/48800316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74a8a00640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.249+0000 7f74a9400640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a40b5400 0x7f74a40c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 msgr2=0x7f74a40cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a40b5400 msgr2=0x7f74a40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a40b5400 0x7f74a40c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 -- 172.21.15.44:0/48800316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a40cbd90 con 0x7f74a40ce5c0 2024-07-17T08:46:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a9e00640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 secure :-1 s=READY pgs=3395 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e970 tx=0x7f749c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c00ccc0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f749c005020 con 0x7f74a40ce5c0 2024-07-17T08:46:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c00ce60 con 0x7f74a40ce5c0 2024-07-17T08:46:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a40cc070 con 0x7f74a40ce5c0 2024-07-17T08:46:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.250+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a40cc630 con 0x7f74a40ce5c0 2024-07-17T08:46:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.252+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a406e6f0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.253+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7f749c005470 con 0x7f74a40ce5c0 2024-07-17T08:46:51.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.254+0000 7f74a1600640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 0x7f74840792b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.254+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f749c0997c0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.254+0000 7f74a9400640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 msgr2=0x7f74840792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:51.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.254+0000 7f74a9400640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 0x7f74840792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:51.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.258+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f749c0622d0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.455+0000 7f74a9400640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 msgr2=0x7f74840792b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:51.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.455+0000 7f74a9400640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 0x7f74840792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-17T08:46:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.462+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74a40b5d70 con 0x7f74a40ce5c0 2024-07-17T08:46:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.464+0000 7f74a1600640 1 -- 172.21.15.44:0/48800316 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v367) ==== 74+0+18530 (secure 0 0 0) 0x7f749c061bd0 con 0x7f74a40ce5c0 2024-07-17T08:46:51.466 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:51.466 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":367,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:46:45.572100+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":49,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:45:20.033990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"363","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":50,"pool_name":"cephfs_data","create_time":"2024-07-17T08:45:21.067621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"364","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"50.1d","mappings":[{"from":7,"to":1}]},{"pgid":"50.1f","mappings":[{"from":7,"to":0}]},{"pgid":"50.2a","mappings":[{"from":7,"to":1}]},{"pgid":"50.30","mappings":[{"from":7,"to":0}]},{"pgid":"50.39","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/1629162363":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/3973742197":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/4233314912":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6837/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6834/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/2555277869":"2024-07-18T08:45:43.974493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 msgr2=0x7f74840792b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 0x7f74840792b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 msgr2=0x7f74a41f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 secure :-1 s=READY pgs=3395 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e970 tx=0x7f749c00ee30 comp rx=0 tx=0).stop 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 shutdown_connections 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a40ce5c0 0x7f74a41f7460 unknown :-1 s=CLOSED pgs=3395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7484076df0 0x7f74840792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74a40b8c80 0x7f74a40cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 --2- 172.21.15.44:0/48800316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74a40b5400 0x7f74a40c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 >> 172.21.15.44:0/48800316 conn(0x7f74a40cfec0 msgr2=0x7f74a40b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.467+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 shutdown_connections 2024-07-17T08:46:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.468+0000 7f74aa800640 1 -- 172.21.15.44:0/48800316 wait complete. 2024-07-17T08:46:51.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-17T08:46:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 -- 172.21.15.44:0/2970312603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a015ae80 msgr2=0x7f78a015e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 --2- 172.21.15.44:0/2970312603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a015ae80 0x7f78a015e360 secure :-1 s=READY pgs=3396 cs=0 l=1 rev1=1 crypto rx=0x7f78a0070780 tx=0x7f789402a560 comp rx=0 tx=0).stop 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 -- 172.21.15.44:0/2970312603 shutdown_connections 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 --2- 172.21.15.44:0/2970312603 >> v2:172.21.15.175:3300/0 conn(0x7f78a00ad8a0 0x7f78a00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 --2- 172.21.15.44:0/2970312603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a015ae80 0x7f78a015e360 unknown :-1 s=CLOSED pgs=3396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.675+0000 7f78a7200640 1 -- 172.21.15.44:0/2970312603 >> 172.21.15.44:0/2970312603 conn(0x7f78a00cd490 msgr2=0x7f78a006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.676+0000 7f78a7200640 1 -- 172.21.15.44:0/2970312603 shutdown_connections 2024-07-17T08:46:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.676+0000 7f78a7200640 1 -- 172.21.15.44:0/2970312603 wait complete. 2024-07-17T08:46:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.677+0000 7f78a7200640 1 Processor -- start 2024-07-17T08:46:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.678+0000 7f78a7200640 1 -- start start 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.678+0000 7f78a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0071bd0 0x7f78a01f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.678+0000 7f78a7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78a015ae80 0x7f78a00c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a7200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a7200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78a00723b0 con 0x7f78a00c7010 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0072230 con 0x7f78a0071bd0 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78a0072530 con 0x7f78a015ae80 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33590/0 (socket says 172.21.15.44:33590) 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 -- 172.21.15.44:0/4059645464 learned_addr learned my addr 172.21.15.44:0/4059645464 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:51.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a5e00640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0071bd0 0x7f78a01f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a5400640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78a015ae80 0x7f78a00c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 -- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78a015ae80 msgr2=0x7f78a00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78a015ae80 0x7f78a00c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 -- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0071bd0 msgr2=0x7f78a01f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.679+0000 7f78a6800640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0071bd0 0x7f78a01f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f78a6800640 1 -- 172.21.15.44:0/4059645464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a01f49d0 con 0x7f78a00c7010 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f78a6800640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c780 tx=0x7f789c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c004b60 con 0x7f78a00c7010 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f789c004df0 con 0x7f78a00c7010 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a00c6940 con 0x7f78a00c7010 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c00aa70 con 0x7f78a00c7010 2024-07-17T08:46:51.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.680+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a00c4f20 con 0x7f78a00c7010 2024-07-17T08:46:51.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.682+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a015e360 con 0x7f78a00c7010 2024-07-17T08:46:51.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.682+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 226) ==== 99039+0+0 (secure 0 0 0) 0x7f789c00ac10 con 0x7f78a00c7010 2024-07-17T08:46:51.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.683+0000 7f789a000640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 0x7f78840792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:51.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.684+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(367..367 src has 1..367) ==== 7136+0+0 (secure 0 0 0) 0x7f789c098f90 con 0x7f78a00c7010 2024-07-17T08:46:51.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.684+0000 7f78a5e00640 1 -- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 msgr2=0x7f78840792e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6834/3626582469 2024-07-17T08:46:51.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.684+0000 7f78a5e00640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 0x7f78840792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-17T08:46:51.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.687+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f789c020070 con 0x7f78a00c7010 2024-07-17T08:46:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.854+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 227) ==== 98540+0+0 (secure 0 0 0) 0x7f789c05d680 con 0x7f78a00c7010 2024-07-17T08:46:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.854+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 msgr2=0x7f78840792e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-17T08:46:51.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.854+0000 7f789a000640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 0x7f78840792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.977 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.975+0000 7f603ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.977 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.975+0000 7f603ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.977 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.975+0000 7f603ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.977 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.975+0000 7f603ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.977 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.975+0000 7f603ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.976+0000 7f6037e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.976+0000 7f6037e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.976+0000 7f6037e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.976+0000 7f6037e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.978 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-07-17T08:46:51.976+0000 7f6037e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-17T08:46:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.983+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f78a006cc10 con 0x7f78a00c7010 2024-07-17T08:46:51.985 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-07-17T08:46:51.984+0000 7fd200a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-17T08:46:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.986+0000 7f789a000640 1 -- 172.21.15.44:0/4059645464 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) ==== 161+0+0 (secure 0 0 0) 0x7f78a01f49d0 con 0x7f78a00c7010 2024-07-17T08:46:51.987 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-17T08:46:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 msgr2=0x7f78a01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c780 tx=0x7f789c00cc40 comp rx=0 tx=0).stop 2024-07-17T08:46:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 shutdown_connections 2024-07-17T08:46:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a00c7010 0x7f78a01609c0 unknown :-1 s=CLOSED pgs=3397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:6834/3626582469,v1:172.21.15.175:6837/3626582469] conn(0x7f7884076e20 0x7f78840792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78a015ae80 0x7f78a00c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 --2- 172.21.15.44:0/4059645464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78a0071bd0 0x7f78a01f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.989+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 >> 172.21.15.44:0/4059645464 conn(0x7f78a00cd490 msgr2=0x7f78a006ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.990+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 shutdown_connections 2024-07-17T08:46:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:51.990+0000 7f78a7200640 1 -- 172.21.15.44:0/4059645464 wait complete. 2024-07-17T08:46:52.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-17T08:46:52.059 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 -- 172.21.15.44:0/3714220870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa28003a80 msgr2=0x7ffa28003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 --2- 172.21.15.44:0/3714220870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa28003a80 0x7ffa28003ea0 secure :-1 s=READY pgs=3402 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009b50 tx=0x7ffa28005830 comp rx=0 tx=0).stop 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43400640 1 -- 172.21.15.44:0/3714220870 reap_dead start 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 -- 172.21.15.44:0/3714220870 shutdown_connections 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 -- 172.21.15.44:0/3714220870 >> 172.21.15.44:0/3714220870 conn(0x7ffa44073920 msgr2=0x7ffa44073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:52.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 -- 172.21.15.44:0/3714220870 shutdown_connections 2024-07-17T08:46:52.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.185+0000 7ffa43e00640 1 -- 172.21.15.44:0/3714220870 wait complete. 2024-07-17T08:46:52.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.187+0000 7ffa43e00640 1 Processor -- start 2024-07-17T08:46:52.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.187+0000 7ffa43e00640 1 -- start start 2024-07-17T08:46:52.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28003a80 0x7ffa4409d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:52.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:52.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa440ad4a0 0x7ffa440ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:52.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4415cec0 con 0x7ffa441584b0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4406c5b0 con 0x7ffa440ad4a0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa43e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4415d040 con 0x7ffa28003a80 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa42000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33622/0 (socket says 172.21.15.44:33622) 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.188+0000 7ffa42000640 1 -- 172.21.15.44:0/1864462661 learned_addr learned my addr 172.21.15.44:0/1864462661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42a00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28003a80 0x7ffa4409d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 -- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28003a80 msgr2=0x7ffa4409d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28003a80 0x7ffa4409d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 -- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa440ad4a0 msgr2=0x7ffa440ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa43400640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa440ad4a0 0x7ffa440ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa440ad4a0 0x7ffa440ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 -- 172.21.15.44:0/1864462661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441f5d50 con 0x7ffa441584b0 2024-07-17T08:46:52.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa42000640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000d800 tx=0x7ffa3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30018070 con 0x7ffa441584b0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa300052e0 con 0x7ffa441584b0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000b8a0 con 0x7ffa441584b0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.189+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441f1860 con 0x7ffa441584b0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.190+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441f1df0 con 0x7ffa441584b0 2024-07-17T08:46:52.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.191+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa44159760 con 0x7ffa441584b0 2024-07-17T08:46:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.192+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 227) ==== 98540+0+0 (secure 0 0 0) 0x7ffa3001b070 con 0x7ffa441584b0 2024-07-17T08:46:52.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.193+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(369..369 src has 1..369) ==== 7652+0+0 (secure 0 0 0) 0x7ffa30097d10 con 0x7ffa441584b0 2024-07-17T08:46:52.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.197+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7ffa3009d020 con 0x7ffa441584b0 2024-07-17T08:46:52.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.449+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ffa440ce570 con 0x7ffa441584b0 2024-07-17T08:46:52.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.859+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 228) ==== 98667+0+0 (secure 0 0 0) 0x7ffa3005c0c0 con 0x7ffa441584b0 2024-07-17T08:46:52.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.861+0000 7ffa3aa00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 0x7ffa2407f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.862+0000 7ffa42a00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 0x7ffa2407f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:52.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.873+0000 7ffa42a00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 0x7ffa2407f8b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009b50 tx=0x7ffa3402d040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:52.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.990+0000 7ffa3aa00640 1 -- 172.21.15.44:0/1864462661 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v211) ==== 104+0+0 (secure 0 0 0) 0x7ffa3006c810 con 0x7ffa441584b0 2024-07-17T08:46:52.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 msgr2=0x7ffa2407f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 0x7ffa2407f8b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009b50 tx=0x7ffa3402d040 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 msgr2=0x7ffa44158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000d800 tx=0x7ffa3000dcc0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 shutdown_connections 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffa2407d4c0 0x7ffa2407f8b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa440ad4a0 0x7ffa440ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa441584b0 0x7ffa44158910 unknown :-1 s=CLOSED pgs=3403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 --2- 172.21.15.44:0/1864462661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa28003a80 0x7ffa4409d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.994+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 >> 172.21.15.44:0/1864462661 conn(0x7ffa44073920 msgr2=0x7ffa4415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.995+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 shutdown_connections 2024-07-17T08:46:52.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:52.995+0000 7ffa43e00640 1 -- 172.21.15.44:0/1864462661 wait complete. 2024-07-17T08:46:53.017 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-17T08:46:53.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 -- 172.21.15.44:0/1119017548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 msgr2=0x7fdf1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:53.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 --2- 172.21.15.44:0/1119017548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 secure :-1 s=READY pgs=3405 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c310 tx=0x7fdf10005830 comp rx=0 tx=0).stop 2024-07-17T08:46:53.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 -- 172.21.15.44:0/1119017548 shutdown_connections 2024-07-17T08:46:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 --2- 172.21.15.44:0/1119017548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 unknown :-1 s=CLOSED pgs=3405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 -- 172.21.15.44:0/1119017548 >> 172.21.15.44:0/1119017548 conn(0x7fdf2c0cf050 msgr2=0x7fdf2c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 -- 172.21.15.44:0/1119017548 shutdown_connections 2024-07-17T08:46:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.234+0000 7fdf31800640 1 -- 172.21.15.44:0/1119017548 wait complete. 2024-07-17T08:46:53.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.236+0000 7fdf31800640 1 Processor -- start 2024-07-17T08:46:53.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.236+0000 7fdf31800640 1 -- start start 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 0x7fdf2c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf2c072e20 0x7fdf2c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0ac8c0 con 0x7fdf100051c0 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1514e0 con 0x7fdf2c072e20 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf31800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c06f1a0 con 0x7fdf2c0af550 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.237+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33640/0 (socket says 172.21.15.44:33640) 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 -- 172.21.15.44:0/2309762278 learned_addr learned my addr 172.21.15.44:0/2309762278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf30e00640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf2c072e20 0x7fdf2c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2b400640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 0x7fdf2c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 -- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 msgr2=0x7fdf2c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 0x7fdf2c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 -- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf2c072e20 msgr2=0x7fdf2c1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf2c072e20 0x7fdf2c1f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 -- 172.21.15.44:0/2309762278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c0af9d0 con 0x7fdf100051c0 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2b400640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 0x7fdf2c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.238+0000 7fdf2be00640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00fb80 tx=0x7fdf1c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.239+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c011640 con 0x7fdf100051c0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.239+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c0117e0 con 0x7fdf100051c0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.239+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c011c30 con 0x7fdf100051c0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.239+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1f4b70 con 0x7fdf100051c0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.239+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c0aed50 con 0x7fdf100051c0 2024-07-17T08:46:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.240+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c0b32e0 con 0x7fdf100051c0 2024-07-17T08:46:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.241+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 228) ==== 98667+0+0 (secure 0 0 0) 0x7fdf1c024440 con 0x7fdf100051c0 2024-07-17T08:46:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.242+0000 7fdf23e00640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 0x7fdf08078ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.242+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(369..369 src has 1..369) ==== 7652+0+0 (secure 0 0 0) 0x7fdf1c0995a0 con 0x7fdf100051c0 2024-07-17T08:46:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.243+0000 7fdf2b400640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 0x7fdf08078ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.244+0000 7fdf2b400640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 0x7fdf08078ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000f60 tx=0x7fdf14006a70 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:53.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.247+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fdf1c062540 con 0x7fdf100051c0 2024-07-17T08:46:53.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.462+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf2c06d800 con 0x7fdf100051c0 2024-07-17T08:46:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.994+0000 7fdf23e00640 1 -- 172.21.15.44:0/2309762278 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v370) ==== 181+0+0 (secure 0 0 0) 0x7fdf1c061e40 con 0x7fdf100051c0 2024-07-17T08:46:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-07-17T08:46:54.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 msgr2=0x7fdf08078ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 0x7fdf08078ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000f60 tx=0x7fdf14006a70 comp rx=0 tx=0).stop 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 msgr2=0x7fdf2c0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00fb80 tx=0x7fdf1c00d6b0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 shutdown_connections 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fdf08076a30 0x7fdf08078ef0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000f60 tx=0x7fdf14006a70 comp rx=0 tx=0).stop 2024-07-17T08:46:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf2c072e20 0x7fdf2c1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf2c0af550 0x7fdf2c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:53.999+0000 7fdf31800640 1 --2- 172.21.15.44:0/2309762278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf100051c0 0x7fdf2c0b0420 unknown :-1 s=CLOSED pgs=3406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.000+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 >> 172.21.15.44:0/2309762278 conn(0x7fdf2c0cf050 msgr2=0x7fdf2c159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.000+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 shutdown_connections 2024-07-17T08:46:54.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.000+0000 7fdf31800640 1 -- 172.21.15.44:0/2309762278 wait complete. 2024-07-17T08:46:54.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-17T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 -- 172.21.15.44:0/4131939744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36740ae490 msgr2=0x7f36740729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 --2- 172.21.15.44:0/4131939744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36740ae490 0x7f36740729b0 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 crypto rx=0x7f36740cfe90 tx=0x7f366c02a8b0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 -- 172.21.15.44:0/4131939744 shutdown_connections 2024-07-17T08:46:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 --2- 172.21.15.44:0/4131939744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36740ae490 0x7f36740729b0 unknown :-1 s=CLOSED pgs=3409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 --2- 172.21.15.44:0/4131939744 >> v2:172.21.15.175:3300/0 conn(0x7f3674155860 0x7f36740ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 -- 172.21.15.44:0/4131939744 >> 172.21.15.44:0/4131939744 conn(0x7f36740cf050 msgr2=0x7f36740cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 -- 172.21.15.44:0/4131939744 shutdown_connections 2024-07-17T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.231+0000 7f3673200640 1 -- 172.21.15.44:0/4131939744 wait complete. 2024-07-17T08:46:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.233+0000 7f3673200640 1 Processor -- start 2024-07-17T08:46:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.233+0000 7f3673200640 1 -- start start 2024-07-17T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 0x7f367415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 0x7f367415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3674073280 con 0x7f3674155860 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3674073100 con 0x7f36740ae490 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3673200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3674073400 con 0x7f36740ad790 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3672800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3672800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33658/0 (socket says 172.21.15.44:33658) 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.234+0000 7f3672800640 1 -- 172.21.15.44:0/4293957741 learned_addr learned my addr 172.21.15.44:0/4293957741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3671e00640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 0x7f367415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3671400640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 0x7f367415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 -- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 msgr2=0x7f367415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 0x7f367415a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 -- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 msgr2=0x7f367415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 0x7f367415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 -- 172.21.15.44:0/4293957741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741f7980 con 0x7f3674155860 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3671e00640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 0x7f367415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3671400640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 0x7f367415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.235+0000 7f3672800640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f366c02aea0 tx=0x7f366c031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.236+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c040070 con 0x7f3674155860 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.236+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f366c004040 con 0x7f3674155860 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.236+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c004470 con 0x7f3674155860 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.236+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36741f7c20 con 0x7f3674155860 2024-07-17T08:46:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.236+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367415ae20 con 0x7f3674155860 2024-07-17T08:46:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.237+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36740729b0 con 0x7f3674155860 2024-07-17T08:46:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.238+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 229) ==== 98875+0+0 (secure 0 0 0) 0x7f366c039040 con 0x7f3674155860 2024-07-17T08:46:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.239+0000 7f366a000640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 0x7f3654079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:54.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.239+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(370..370 src has 1..370) ==== 7214+0+0 (secure 0 0 0) 0x7f366c0b9ff0 con 0x7f3674155860 2024-07-17T08:46:54.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.240+0000 7f3671e00640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 0x7f3654079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:54.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.241+0000 7f3671e00640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 0x7f3654079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3664009aa0 tx=0x7f3664005db0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:54.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.243+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f366c082d90 con 0x7f3674155860 2024-07-17T08:46:54.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:54.452+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36741f7df0 con 0x7f3674155860 2024-07-17T08:46:55.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.009+0000 7f366a000640 1 -- 172.21.15.44:0/4293957741 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v371) ==== 169+0+0 (secure 0 0 0) 0x7f366c087e20 con 0x7f3674155860 2024-07-17T08:46:55.010 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-07-17T08:46:55.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.015+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 msgr2=0x7f3654079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.015+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 0x7f3654079100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3664009aa0 tx=0x7f3664005db0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.015+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 msgr2=0x7f36741f3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.015+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f366c02aea0 tx=0x7f366c031720 comp rx=0 tx=0).stop 2024-07-17T08:46:55.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 shutdown_connections 2024-07-17T08:46:55.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3654076c40 0x7f3654079100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3674155860 0x7f36741f3a30 unknown :-1 s=CLOSED pgs=3410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36740ae490 0x7f367415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 --2- 172.21.15.44:0/4293957741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36740ad790 0x7f367415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 >> 172.21.15.44:0/4293957741 conn(0x7f36740cf050 msgr2=0x7f36740abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:55.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 shutdown_connections 2024-07-17T08:46:55.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.016+0000 7f3673200640 1 -- 172.21.15.44:0/4293957741 wait complete. 2024-07-17T08:46:55.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 -- 172.21.15.44:0/572340690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838154600 msgr2=0x7f78380b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 --2- 172.21.15.44:0/572340690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838154600 0x7f78380b3ce0 secure :-1 s=READY pgs=3412 cs=0 l=1 rev1=1 crypto rx=0x7f7828009830 tx=0x7f782802a510 comp rx=0 tx=0).stop 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 -- 172.21.15.44:0/572340690 shutdown_connections 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 --2- 172.21.15.44:0/572340690 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 0x7f78380ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 --2- 172.21.15.44:0/572340690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838154600 0x7f78380b3ce0 unknown :-1 s=CLOSED pgs=3412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 -- 172.21.15.44:0/572340690 >> 172.21.15.44:0/572340690 conn(0x7f78380b0eb0 msgr2=0x7f78380b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 -- 172.21.15.44:0/572340690 shutdown_connections 2024-07-17T08:46:55.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.253+0000 7f783ee00640 1 -- 172.21.15.44:0/572340690 wait complete. 2024-07-17T08:46:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.255+0000 7f783ee00640 1 Processor -- start 2024-07-17T08:46:55.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.255+0000 7f783ee00640 1 -- start start 2024-07-17T08:46:55.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 0x7f7838073350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7838154600 0x7f78381f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f783815e220 con 0x7f7838161120 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f783815e0a0 con 0x7f7838154600 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f783815e3a0 con 0x7f78380ac190 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33680/0 (socket says 172.21.15.44:33680) 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f783ce00640 1 -- 172.21.15.44:0/3591300760 learned_addr learned my addr 172.21.15.44:0/3591300760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f7837400640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7838154600 0x7f78381f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.256+0000 7f7837e00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 0x7f7838073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 -- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 msgr2=0x7f7838073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 0x7f7838073350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 -- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7838154600 msgr2=0x7f78381f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7838154600 0x7f78381f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 -- 172.21.15.44:0/3591300760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7838157940 con 0x7f7838161120 2024-07-17T08:46:55.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ce00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7f783000b9a0 tx=0x7f783000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830019070 con 0x7f7838161120 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f783000ccb0 con 0x7f7838161120 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78380739e0 con 0x7f7838161120 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78300146a0 con 0x7f7838161120 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.257+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7838155040 con 0x7f7838161120 2024-07-17T08:46:55.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.258+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783815e950 con 0x7f7838161120 2024-07-17T08:46:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.260+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 229) ==== 98875+0+0 (secure 0 0 0) 0x7f783001c070 con 0x7f7838161120 2024-07-17T08:46:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.261+0000 7f782fe00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 0x7f7818079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.261+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f7830098500 con 0x7f7838161120 2024-07-17T08:46:55.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.261+0000 7f7837e00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 0x7f7818079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.262+0000 7f7837e00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 0x7f7818079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7828034000 tx=0x7f7828004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:55.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.265+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f78300615c0 con 0x7f7838161120 2024-07-17T08:46:55.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.476+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f7838160e90 con 0x7f7838161120 2024-07-17T08:46:55.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.485+0000 7f782fe00640 1 -- 172.21.15.44:0/3591300760 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v111) ==== 71+0+0 (secure 0 0 0) 0x7f7830060ec0 con 0x7f7838161120 2024-07-17T08:46:55.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.487+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 msgr2=0x7f7818079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.487+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 0x7f7818079100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7828034000 tx=0x7f7828004730 comp rx=0 tx=0).stop 2024-07-17T08:46:55.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 msgr2=0x7f78381f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7f783000b9a0 tx=0x7f783000be60 comp rx=0 tx=0).stop 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 shutdown_connections 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7818076c40 0x7f7818079100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7838161120 0x7f78381f6f90 unknown :-1 s=CLOSED pgs=3413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7838154600 0x7f78381f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 --2- 172.21.15.44:0/3591300760 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78380ac190 0x7f7838073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 >> 172.21.15.44:0/3591300760 conn(0x7f78380b0eb0 msgr2=0x7f78380ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:55.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 shutdown_connections 2024-07-17T08:46:55.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.488+0000 7f783ee00640 1 -- 172.21.15.44:0/3591300760 wait complete. 2024-07-17T08:46:55.509 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07' 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.722+0000 7fb431000640 1 -- 172.21.15.44:0/1458091125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c0acaf0 msgr2=0x7fb42c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.722+0000 7fb431000640 1 --2- 172.21.15.44:0/1458091125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c0acaf0 0x7fb42c06e090 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb424005fd0 tx=0x7fb42402b980 comp rx=0 tx=0).stop 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 -- 172.21.15.44:0/1458091125 shutdown_connections 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 --2- 172.21.15.44:0/1458091125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c0acaf0 0x7fb42c06e090 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 --2- 172.21.15.44:0/1458091125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c153a50 0x7fb42c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 -- 172.21.15.44:0/1458091125 >> 172.21.15.44:0/1458091125 conn(0x7fb42c0b0fb0 msgr2=0x7fb42c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 -- 172.21.15.44:0/1458091125 shutdown_connections 2024-07-17T08:46:55.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.723+0000 7fb431000640 1 -- 172.21.15.44:0/1458091125 wait complete. 2024-07-17T08:46:55.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.725+0000 7fb431000640 1 Processor -- start 2024-07-17T08:46:55.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.725+0000 7fb431000640 1 -- start start 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c153a50 0x7fb42c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c15a040 0x7fb42c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c06f710 con 0x7fb42c0acaf0 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c154040 con 0x7fb42c153a50 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb431000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c06f890 con 0x7fb42c15a040 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb429e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb429e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33686/0 (socket says 172.21.15.44:33686) 2024-07-17T08:46:55.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb429e00640 1 -- 172.21.15.44:0/4211781599 learned_addr learned my addr 172.21.15.44:0/4211781599 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:55.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb42a800640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c15a040 0x7fb42c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.726+0000 7fb423e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c153a50 0x7fb42c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 -- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c15a040 msgr2=0x7fb42c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c15a040 0x7fb42c15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 -- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c153a50 msgr2=0x7fb42c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c153a50 0x7fb42c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 -- 172.21.15.44:0/4211781599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c1f8360 con 0x7fb42c0acaf0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb429e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00ea20 tx=0x7fb41c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00cd80 con 0x7fb42c0acaf0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb41c010430 con 0x7fb42c0acaf0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c010840 con 0x7fb42c0acaf0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.727+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c156e60 con 0x7fb42c0acaf0 2024-07-17T08:46:55.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.728+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c156b50 con 0x7fb42c0acaf0 2024-07-17T08:46:55.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.729+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42c0b3cf0 con 0x7fb42c0acaf0 2024-07-17T08:46:55.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.730+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 229) ==== 98875+0+0 (secure 0 0 0) 0x7fb41c0109e0 con 0x7fb42c0acaf0 2024-07-17T08:46:55.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.731+0000 7fb422000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 0x7fb408079100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:55.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.731+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7fb41c099360 con 0x7fb42c0acaf0 2024-07-17T08:46:55.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.732+0000 7fb423e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 0x7fb408079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:55.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.733+0000 7fb423e00640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 0x7fb408079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0b2b80 tx=0x7fb414009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:55.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:55.736+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb42c0b3cf0 con 0x7fb42c0acaf0 2024-07-17T08:46:56.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:56.021+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07"]} v 0) -- 0x7fb42c06e090 con 0x7fb42c0acaf0 2024-07-17T08:46:56.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:56.504+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb41c010cd0 con 0x7fb42c0acaf0 2024-07-17T08:46:57.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.015+0000 7fb422000640 1 -- 172.21.15.44:0/4211781599 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_07"]}]=0 v3795) ==== 201+0+0 (secure 0 0 0) 0x7fb41c062310 con 0x7fb42c0acaf0 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 msgr2=0x7fb408079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 0x7fb408079100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0b2b80 tx=0x7fb414009040 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 msgr2=0x7fb42c1e48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00ea20 tx=0x7fb41c00eee0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 shutdown_connections 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb408076c40 0x7fb408079100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb42c15a040 0x7fb42c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb42c153a50 0x7fb42c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 --2- 172.21.15.44:0/4211781599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb42c0acaf0 0x7fb42c1e48b0 unknown :-1 s=CLOSED pgs=3414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 >> 172.21.15.44:0/4211781599 conn(0x7fb42c0b0fb0 msgr2=0x7fb42c152900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:57.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 shutdown_connections 2024-07-17T08:46:57.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.019+0000 7fb431000640 1 -- 172.21.15.44:0/4211781599 wait complete. 2024-07-17T08:46:57.043 INFO:tasks.cephfs_test_runner:Test subvol schedule listing fails without a subvol argument. ... ok 2024-07-17T08:46:57.043 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol schedule listing succeeds with a subvol and a subvolgroup argument. 2024-07-17T08:46:57.043 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:46:57.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:46:57.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.260+0000 7fc04ba00640 1 -- 172.21.15.44:0/1141965679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ae810 msgr2=0x7fc04415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.260+0000 7fc04ba00640 1 --2- 172.21.15.44:0/1141965679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ae810 0x7fc04415e510 secure :-1 s=READY pgs=3416 cs=0 l=1 rev1=1 crypto rx=0x7fc034005fd0 tx=0x7fc03402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 -- 172.21.15.44:0/1141965679 shutdown_connections 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 --2- 172.21.15.44:0/1141965679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ac930 0x7fc0440b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 --2- 172.21.15.44:0/1141965679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ae810 0x7fc04415e510 unknown :-1 s=CLOSED pgs=3416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 -- 172.21.15.44:0/1141965679 >> 172.21.15.44:0/1141965679 conn(0x7fc0440b0fb0 msgr2=0x7fc0440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 -- 172.21.15.44:0/1141965679 shutdown_connections 2024-07-17T08:46:57.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.261+0000 7fc04ba00640 1 -- 172.21.15.44:0/1141965679 wait complete. 2024-07-17T08:46:57.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.262+0000 7fc04ba00640 1 Processor -- start 2024-07-17T08:46:57.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.263+0000 7fc04ba00640 1 -- start start 2024-07-17T08:46:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 0x7fc0440742a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ae810 0x7fc0440998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc04406f040 con 0x7fc0440ac930 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc04406eec0 con 0x7fc04406ea50 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc04ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc0440b2b10 con 0x7fc0440ae810 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33710/0 (socket says 172.21.15.44:33710) 2024-07-17T08:46:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 -- 172.21.15.44:0/2754862802 learned_addr learned my addr 172.21.15.44:0/2754862802 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc049000640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 0x7fc0440742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc049a00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ae810 0x7fc0440998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 -- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ae810 msgr2=0x7fc0440998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ae810 0x7fc0440998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 -- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 msgr2=0x7fc0440742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 0x7fc0440742a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc043e00640 1 -- 172.21.15.44:0/2754862802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0440747e0 con 0x7fc0440ac930 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.264+0000 7fc049000640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 0x7fc0440742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc043e00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c760 tx=0x7fc03000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030017070 con 0x7fc0440ac930 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc030005180 con 0x7fc0440ac930 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04409c330 con 0x7fc0440ac930 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03000a640 con 0x7fc0440ac930 2024-07-17T08:46:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.265+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04409c8f0 con 0x7fc0440ac930 2024-07-17T08:46:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.266+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0440ad7e0 con 0x7fc0440ac930 2024-07-17T08:46:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.268+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fc03001a070 con 0x7fc0440ac930 2024-07-17T08:46:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.268+0000 7fc040c00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 0x7fc0240791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.269+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7fc030097c60 con 0x7fc0440ac930 2024-07-17T08:46:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.269+0000 7fc049000640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 0x7fc0240791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.270+0000 7fc049000640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 0x7fc0240791c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc034033000 tx=0x7fc034004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.273+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc0440ad7e0 con 0x7fc0440ac930 2024-07-17T08:46:57.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.486+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc04409cfa0 con 0x7fc0440ac930 2024-07-17T08:46:57.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.488+0000 7fc040c00640 1 -- 172.21.15.44:0/2754862802 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v371) ==== 74+0+15274 (secure 0 0 0) 0x7fc0440747e0 con 0x7fc0440ac930 2024-07-17T08:46:57.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:46:57.489 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":371,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:46:55.005678+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":50,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":366,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":362,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/3626582469":"2024-07-18T08:46:51.835382+0000","172.21.15.44:6834/2831509556":"2024-07-18T08:46:51.984621+0000","172.21.15.175:6834/3626582469":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/135552899":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/1035355254":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/3811232586":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/2277642556":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/3659467291":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/863435010":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/3915956881":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/1629162363":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/3973742197":"2024-07-18T08:45:43.974493+0000","172.21.15.175:0/2249913934":"2024-07-18T08:46:51.835382+0000","172.21.15.175:0/4233314912":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6837/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.175:6834/945953093":"2024-07-18T08:45:43.974493+0000","172.21.15.44:6832/2831509556":"2024-07-18T08:46:51.984621+0000","172.21.15.175:0/2555277869":"2024-07-18T08:45:43.974493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:46:57.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 msgr2=0x7fc0240791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 0x7fc0240791c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc034033000 tx=0x7fc034004730 comp rx=0 tx=0).stop 2024-07-17T08:46:57.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 msgr2=0x7fc0440993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c760 tx=0x7fc03000cc20 comp rx=0 tx=0).stop 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 shutdown_connections 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc024076d00 0x7fc0240791c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0440ae810 0x7fc0440998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0440ac930 0x7fc0440993a0 unknown :-1 s=CLOSED pgs=3417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 --2- 172.21.15.44:0/2754862802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc04406ea50 0x7fc0440742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.491+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 >> 172.21.15.44:0/2754862802 conn(0x7fc0440b0fb0 msgr2=0x7fc044160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.492+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 shutdown_connections 2024-07-17T08:46:57.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.492+0000 7fc04ba00640 1 -- 172.21.15.44:0/2754862802 wait complete. 2024-07-17T08:46:57.514 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-17T06:54:41.754444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-17T08:46:57.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-17T08:46:57.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 -- 172.21.15.44:0/871269830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 msgr2=0x7f5bf815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 --2- 172.21.15.44:0/871269830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf815d510 secure :-1 s=READY pgs=3418 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005fd0 tx=0x7f5be802a4a0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 -- 172.21.15.44:0/871269830 shutdown_connections 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 --2- 172.21.15.44:0/871269830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf80ce690 0x7f5bf80cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 --2- 172.21.15.44:0/871269830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf815d510 unknown :-1 s=CLOSED pgs=3418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.729+0000 7f5bfe200640 1 -- 172.21.15.44:0/871269830 >> 172.21.15.44:0/871269830 conn(0x7f5bf80b4ff0 msgr2=0x7f5bf80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.730+0000 7f5bfe200640 1 -- 172.21.15.44:0/871269830 shutdown_connections 2024-07-17T08:46:57.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.730+0000 7f5bfe200640 1 -- 172.21.15.44:0/871269830 wait complete. 2024-07-17T08:46:57.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.731+0000 7f5bfe200640 1 Processor -- start 2024-07-17T08:46:57.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.731+0000 7f5bfe200640 1 -- start start 2024-07-17T08:46:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.732+0000 7f5bfe200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf80ce690 0x7f5bf81f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.732+0000 7f5bfe200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.732+0000 7f5bfe200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf8161be0 0x7f5bf81494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.732+0000 7f5bfe200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d1a60 con 0x7f5bf815d130 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.732+0000 7f5bfe200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d18e0 con 0x7f5bf80ce690 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bfe200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf80d1be0 con 0x7f5bf8161be0 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33738/0 (socket says 172.21.15.44:33738) 2024-07-17T08:46:57.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 -- 172.21.15.44:0/176802113 learned_addr learned my addr 172.21.15.44:0/176802113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bfd800640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf8161be0 0x7f5bf81494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bfce00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf80ce690 0x7f5bf81f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 -- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf8161be0 msgr2=0x7f5bf81494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf8161be0 0x7f5bf81494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 -- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf80ce690 msgr2=0x7f5bf81f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf80ce690 0x7f5bf81f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.733+0000 7f5bf7e00640 1 -- 172.21.15.44:0/176802113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80cc120 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bf7e00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c7c0 tx=0x7f5be400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4016020 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be400ab40 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4012420 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80cc400 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.734+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf81e7190 con 0x7f5bf815d130 2024-07-17T08:46:57.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.735+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf80cfdf0 con 0x7f5bf815d130 2024-07-17T08:46:57.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.737+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f5be4019020 con 0x7f5bf815d130 2024-07-17T08:46:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.738+0000 7f5bf4c00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 0x7f5bd8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.738+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f5be4098750 con 0x7f5bf815d130 2024-07-17T08:46:57.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.739+0000 7f5bfce00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 0x7f5bd8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:57.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.740+0000 7f5bfce00640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 0x7f5bd8079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5be8033000 tx=0x7f5be8004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:57.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.742+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f5be4020070 con 0x7f5bf815d130 2024-07-17T08:46:57.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.944+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bf80c9dd0 con 0x7f5bf815d130 2024-07-17T08:46:57.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.945+0000 7f5bf4c00640 1 -- 172.21.15.44:0/176802113 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v371) ==== 93+0+10 (secure 0 0 0) 0x7f5be40616c0 con 0x7f5bf815d130 2024-07-17T08:46:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-07-17T08:46:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 msgr2=0x7f5bd8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 0x7f5bd8079190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5be8033000 tx=0x7f5be8004730 comp rx=0 tx=0).stop 2024-07-17T08:46:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 msgr2=0x7f5bf81f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c7c0 tx=0x7f5be400cc80 comp rx=0 tx=0).stop 2024-07-17T08:46:57.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 shutdown_connections 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f5bd8076cd0 0x7f5bd8079190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bf8161be0 0x7f5bf81494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bf815d130 0x7f5bf81f7a10 unknown :-1 s=CLOSED pgs=3419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 --2- 172.21.15.44:0/176802113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bf80ce690 0x7f5bf81f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 >> 172.21.15.44:0/176802113 conn(0x7f5bf80b4ff0 msgr2=0x7f5bf80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 shutdown_connections 2024-07-17T08:46:57.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:57.948+0000 7f5bfe200640 1 -- 172.21.15.44:0/176802113 wait complete. 2024-07-17T08:46:57.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_08' 2024-07-17T08:46:58.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 -- 172.21.15.44:0/3450076151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd0159910 msgr2=0x7f4bd006f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:58.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 --2- 172.21.15.44:0/3450076151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd0159910 0x7f4bd006f4a0 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005fd0 tx=0x7f4bc402b910 comp rx=0 tx=0).stop 2024-07-17T08:46:58.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 -- 172.21.15.44:0/3450076151 shutdown_connections 2024-07-17T08:46:58.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 --2- 172.21.15.44:0/3450076151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd00acc30 0x7f4bd00b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 --2- 172.21.15.44:0/3450076151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd0159910 0x7f4bd006f4a0 unknown :-1 s=CLOSED pgs=3421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 -- 172.21.15.44:0/3450076151 >> 172.21.15.44:0/3450076151 conn(0x7f4bd00b0fb0 msgr2=0x7f4bd00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 -- 172.21.15.44:0/3450076151 shutdown_connections 2024-07-17T08:46:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.172+0000 7f4bd8800640 1 -- 172.21.15.44:0/3450076151 wait complete. 2024-07-17T08:46:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.174+0000 7f4bd8800640 1 Processor -- start 2024-07-17T08:46:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.174+0000 7f4bd8800640 1 -- start start 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 0x7f4bd014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd0159910 0x7f4bd0071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd006d680 con 0x7f4bd00b30b0 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00b37e0 con 0x7f4bd00acc30 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.175+0000 7f4bd8800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd006d800 con 0x7f4bd0159910 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33756/0 (socket says 172.21.15.44:33756) 2024-07-17T08:46:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 -- 172.21.15.44:0/2944497859 learned_addr learned my addr 172.21.15.44:0/2944497859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd6800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd0159910 0x7f4bd0071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5e00640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 0x7f4bd014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 -- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd0159910 msgr2=0x7f4bd0071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd0159910 0x7f4bd0071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 -- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 msgr2=0x7f4bd014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 0x7f4bd014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 -- 172.21.15.44:0/2944497859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd0073510 con 0x7f4bd00b30b0 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5e00640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 0x7f4bd014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.176+0000 7f4bd5400640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d720 tx=0x7f4bc000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.177+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0009c30 con 0x7f4bd00b30b0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.177+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bc000de30 con 0x7f4bd00b30b0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.177+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd00737f0 con 0x7f4bd00b30b0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.177+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0004280 con 0x7f4bd00b30b0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.177+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd0152680 con 0x7f4bd00b30b0 2024-07-17T08:46:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.178+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd006dcb0 con 0x7f4bd00b30b0 2024-07-17T08:46:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.179+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f4bc00035f0 con 0x7f4bd00b30b0 2024-07-17T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.180+0000 7f4bca000640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 0x7f4bb4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.180+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f4bc0098e00 con 0x7f4bd00b30b0 2024-07-17T08:46:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.181+0000 7f4bd5e00640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 0x7f4bb4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:58.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.182+0000 7f4bd5e00640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 0x7f4bb4079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4035000 tx=0x7f4bc4004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:58.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.184+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4bc0061cf0 con 0x7f4bd00b30b0 2024-07-17T08:46:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:58.460+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_08"]} v 0) -- 0x7f4bd00b27a0 con 0x7f4bd00b30b0 2024-07-17T08:46:59.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.027+0000 7f4bca000640 1 -- 172.21.15.44:0/2944497859 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments_08"]}]=0 v3797) ==== 204+0+0 (secure 0 0 0) 0x7f4bc00615f0 con 0x7f4bd00b30b0 2024-07-17T08:46:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.030+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 msgr2=0x7f4bb4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 0x7f4bb4079190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4035000 tx=0x7f4bc4004730 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 msgr2=0x7f4bd0154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000d720 tx=0x7f4bc000dbe0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 shutdown_connections 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4bb4076cd0 0x7f4bb4079190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4bd0159910 0x7f4bd0071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00b30b0 0x7f4bd0154bd0 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 --2- 172.21.15.44:0/2944497859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bd00acc30 0x7f4bd014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 >> 172.21.15.44:0/2944497859 conn(0x7f4bd00b0fb0 msgr2=0x7f4bd00b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 shutdown_connections 2024-07-17T08:46:59.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.031+0000 7f4bd8800640 1 -- 172.21.15.44:0/2944497859 wait complete. 2024-07-17T08:46:59.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-17T08:46:59.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 -- 172.21.15.44:0/2787123250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98ac003a80 msgr2=0x7f98ac003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 --2- 172.21.15.44:0/2787123250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98ac003a80 0x7f98ac003ea0 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7f98b800baf0 tx=0x7f98ac005830 comp rx=0 tx=0).stop 2024-07-17T08:46:59.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cd400640 1 -- 172.21.15.44:0/2787123250 reap_dead start 2024-07-17T08:46:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 -- 172.21.15.44:0/2787123250 shutdown_connections 2024-07-17T08:46:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 -- 172.21.15.44:0/2787123250 >> 172.21.15.44:0/2787123250 conn(0x7f98c806bf40 msgr2=0x7f98c806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 -- 172.21.15.44:0/2787123250 shutdown_connections 2024-07-17T08:46:59.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.272+0000 7f98cf400640 1 -- 172.21.15.44:0/2787123250 wait complete. 2024-07-17T08:46:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.274+0000 7f98cf400640 1 Processor -- start 2024-07-17T08:46:59.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.274+0000 7f98cf400640 1 -- start start 2024-07-17T08:46:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac003a80 0x7f98c80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c80c89e0 0x7f98c80c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80cef00 con 0x7f98c80cadb0 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80ced80 con 0x7f98c80c89e0 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.275+0000 7f98cf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98c80cf080 con 0x7f98ac003a80 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38652/0 (socket says 172.21.15.44:38652) 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 -- 172.21.15.44:0/1074506341 learned_addr learned my addr 172.21.15.44:0/1074506341 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98cd400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c80c89e0 0x7f98c80c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98cca00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac003a80 0x7f98c80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 -- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac003a80 msgr2=0x7f98c80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac003a80 0x7f98c80ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 -- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c80c89e0 msgr2=0x7f98c80c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c80c89e0 0x7f98c80c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.276+0000 7f98bfe00640 1 -- 172.21.15.44:0/1074506341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c8158130 con 0x7f98c80cadb0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98bfe00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e4f0 tx=0x7f98b000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0015070 con 0x7f98c80cadb0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98b0010980 con 0x7f98c80cadb0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c8158410 con 0x7f98c80cadb0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0010dd0 con 0x7f98c80cadb0 2024-07-17T08:46:59.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.277+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81589d0 con 0x7f98c80cadb0 2024-07-17T08:46:59.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.278+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c8071d40 con 0x7f98c80cadb0 2024-07-17T08:46:59.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.279+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f98b0018070 con 0x7f98c80cadb0 2024-07-17T08:46:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.280+0000 7f98c5600640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 0x7f98a40791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.280+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f98b0098cb0 con 0x7f98c80cadb0 2024-07-17T08:46:59.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.281+0000 7f98cca00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 0x7f98a40791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.282+0000 7f98cca00640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 0x7f98a40791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98b800baf0 tx=0x7f98b80046c0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.285+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f98c8071d40 con 0x7f98c80cadb0 2024-07-17T08:46:59.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.498+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f98c815a8a0 con 0x7f98c80cadb0 2024-07-17T08:46:59.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.499+0000 7f98c5600640 1 -- 172.21.15.44:0/1074506341 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v111) ==== 59+0+1998 (secure 0 0 0) 0x7f98c815a8a0 con 0x7f98c80cadb0 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:mon advanced mon_op_complaint_time 120 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4187.112377 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3933.278847 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4982.550022 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4534.829173 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3537.105260 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5597.758780 2024-07-17T08:46:59.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5507.207495 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3797.325118 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds 20/20 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_mds_balancer 20/20 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced debug_ms 1/1 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_frag true 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_debug_scatterstat true 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds dev mds_verify_scatter true 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_mon_op_timeout 900 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:mds advanced rados_osd_op_timeout 900 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced client_mount_timeout 600 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_client 20/20 2024-07-17T08:46:59.502 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced debug_ms 1/1 2024-07-17T08:46:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_mon_op_timeout 900 2024-07-17T08:46:59.503 INFO:teuthology.orchestra.run.smithi044.stdout:client advanced rados_osd_op_timeout 900 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.501+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 msgr2=0x7f98a40791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.501+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 0x7f98a40791c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98b800baf0 tx=0x7f98b80046c0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.501+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 msgr2=0x7f98c81f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e4f0 tx=0x7f98b000e9b0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 shutdown_connections 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f98a4076d00 0x7f98a40791c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c80c89e0 0x7f98c80c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98c80cadb0 0x7f98c81f72e0 unknown :-1 s=CLOSED pgs=3425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 --2- 172.21.15.44:0/1074506341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac003a80 0x7f98c80ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 >> 172.21.15.44:0/1074506341 conn(0x7f98c806bf40 msgr2=0x7f98c815c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:59.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 shutdown_connections 2024-07-17T08:46:59.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.502+0000 7f98cf400640 1 -- 172.21.15.44:0/1074506341 wait complete. 2024-07-17T08:46:59.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-17T08:46:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 -- 172.21.15.44:0/476089548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31180b3280 msgr2=0x7f311806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 --2- 172.21.15.44:0/476089548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31180b3280 0x7f311806eb90 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f3110005fd0 tx=0x7f311002b980 comp rx=0 tx=0).stop 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 -- 172.21.15.44:0/476089548 shutdown_connections 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 --2- 172.21.15.44:0/476089548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31180b3280 0x7f311806eb90 unknown :-1 s=CLOSED pgs=3426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 --2- 172.21.15.44:0/476089548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 0x7f311815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 -- 172.21.15.44:0/476089548 >> 172.21.15.44:0/476089548 conn(0x7f31180b0eb0 msgr2=0x7f31180b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.740+0000 7f311dc00640 1 -- 172.21.15.44:0/476089548 shutdown_connections 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.741+0000 7f311dc00640 1 -- 172.21.15.44:0/476089548 wait complete. 2024-07-17T08:46:59.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.742+0000 7f311dc00640 1 Processor -- start 2024-07-17T08:46:59.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.742+0000 7f311dc00640 1 -- start start 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 0x7f311814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 0x7f31180735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31180aea90 con 0x7f311806f6d0 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31180ae910 con 0x7f31180b3280 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.743+0000 7f311dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31180aec10 con 0x7f31180acab0 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38664/0 (socket says 172.21.15.44:38664) 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 -- 172.21.15.44:0/2840679030 learned_addr learned my addr 172.21.15.44:0/2840679030 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116000640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 0x7f311814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3117400640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 0x7f31180735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 -- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 msgr2=0x7f311814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 0x7f311814ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 -- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 msgr2=0x7f31180735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 0x7f31180735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3116a00640 1 -- 172.21.15.44:0/2840679030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31181f5600 con 0x7f311806f6d0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.744+0000 7f3117400640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 0x7f31180735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f3116a00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7f310800c450 tx=0x7f310800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080052d0 con 0x7f311806f6d0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3108005470 con 0x7f311806f6d0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3118149df0 con 0x7f311806f6d0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108012870 con 0x7f311806f6d0 2024-07-17T08:46:59.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.745+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311815bf70 con 0x7f311806f6d0 2024-07-17T08:46:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.746+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f311815a5a0 con 0x7f311806f6d0 2024-07-17T08:46:59.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.747+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f31080035f0 con 0x7f311806f6d0 2024-07-17T08:46:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.748+0000 7f310ea00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 0x7f30f80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:46:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.748+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f31080984b0 con 0x7f311806f6d0 2024-07-17T08:46:59.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.749+0000 7f3116000640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 0x7f30f80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:46:59.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.750+0000 7f3116000640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 0x7f30f80791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f3104005850 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:46:59.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.753+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f31181f5600 con 0x7f311806f6d0 2024-07-17T08:46:59.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.963+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f31180ac820 con 0x7f311806f6d0 2024-07-17T08:46:59.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.972+0000 7f310ea00640 1 -- 172.21.15.44:0/2840679030 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v112)=0 v112) ==== 122+0+0 (secure 0 0 0) 0x7f3108061420 con 0x7f311806f6d0 2024-07-17T08:46:59.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 msgr2=0x7f30f80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 0x7f30f80791c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f3104005850 comp rx=0 tx=0).stop 2024-07-17T08:46:59.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 msgr2=0x7f31181f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:46:59.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7f310800c450 tx=0x7f310800c910 comp rx=0 tx=0).stop 2024-07-17T08:46:59.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 shutdown_connections 2024-07-17T08:46:59.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f30f8076d00 0x7f30f80791c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31180b3280 0x7f31180735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31180acab0 0x7f311814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 --2- 172.21.15.44:0/2840679030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f311806f6d0 0x7f31181f50c0 unknown :-1 s=CLOSED pgs=3427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 >> 172.21.15.44:0/2840679030 conn(0x7f31180b0eb0 msgr2=0x7f3118152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 shutdown_connections 2024-07-17T08:46:59.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:46:59.975+0000 7f311dc00640 1 -- 172.21.15.44:0/2840679030 wait complete. 2024-07-17T08:46:59.999 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-17T08:46:59.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:47:00.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-17T08:47:00.249 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-17T08:47:00.249 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:47:00.250 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:47:00.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:47:00.311 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:47:00.311 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:47:00.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:47:00.368 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:47:00.368 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:47:00.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:47:00.427 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:47:00.428 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:47:00.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:47:00.482 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-07-17T08:47:00.482 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.708+0000 7fb1d6200640 1 -- 172.21.15.44:0/2050093921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1b4004800 msgr2=0x7fb1b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.708+0000 7fb1d6200640 1 --2- 172.21.15.44:0/2050093921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1b4004800 0x7fb1b4004c80 secure :-1 s=READY pgs=3429 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00d0030 tx=0x7fb1b4005830 comp rx=0 tx=0).stop 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.708+0000 7fb1d5800640 1 -- 172.21.15.44:0/2050093921 reap_dead start 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.709+0000 7fb1d6200640 1 -- 172.21.15.44:0/2050093921 shutdown_connections 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.709+0000 7fb1d6200640 1 -- 172.21.15.44:0/2050093921 >> 172.21.15.44:0/2050093921 conn(0x7fb1d00b0e80 msgr2=0x7fb1d00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.709+0000 7fb1d6200640 1 -- 172.21.15.44:0/2050093921 shutdown_connections 2024-07-17T08:47:00.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.709+0000 7fb1d6200640 1 -- 172.21.15.44:0/2050093921 wait complete. 2024-07-17T08:47:00.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.710+0000 7fb1d6200640 1 Processor -- start 2024-07-17T08:47:00.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.711+0000 7fb1d6200640 1 -- start start 2024-07-17T08:47:00.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.711+0000 7fb1d6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b4004800 0x7fb1d00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:00.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.711+0000 7fb1d6200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:00.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.711+0000 7fb1d6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d01f7160 0x7fb1d006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1d6200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00adff0 con 0x7fb1d009c910 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1d6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00b3b00 con 0x7fb1d01f7160 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1d6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d00ae170 con 0x7fb1b4004800 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38672/0 (socket says 172.21.15.44:38672) 2024-07-17T08:47:00.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 -- 172.21.15.44:0/897748549 learned_addr learned my addr 172.21.15.44:0/897748549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:00.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1d5800640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d01f7160 0x7fb1d006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:00.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1d4e00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b4004800 0x7fb1d00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 -- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b4004800 msgr2=0x7fb1d00713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b4004800 0x7fb1d00713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 -- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d01f7160 msgr2=0x7fb1d006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d01f7160 0x7fb1d006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.712+0000 7fb1cfe00640 1 -- 172.21.15.44:0/897748549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d013a240 con 0x7fb1d009c910 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1cfe00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006310 tx=0x7fb1bc006340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc041070 con 0x7fb1d009c910 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1bc003af0 con 0x7fb1d009c910 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc038070 con 0x7fb1d009c910 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d013a4e0 con 0x7fb1d009c910 2024-07-17T08:47:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.713+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d0138730 con 0x7fb1d009c910 2024-07-17T08:47:00.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.715+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d00ae670 con 0x7fb1d009c910 2024-07-17T08:47:00.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.716+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb1bc030070 con 0x7fb1d009c910 2024-07-17T08:47:00.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.716+0000 7fb1ccc00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 0x7fb1b0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:00.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.717+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7fb1bc0b7aa0 con 0x7fb1d009c910 2024-07-17T08:47:00.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.717+0000 7fb1d4e00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 0x7fb1b0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:00.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.718+0000 7fb1d4e00640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 0x7fb1b0079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005fd0 tx=0x7fb1c0003a50 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:00.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.721+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb1bc080b40 con 0x7fb1d009c910 2024-07-17T08:47:00.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.979+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d0140dc0 con 0x7fb1d009c910 2024-07-17T08:47:00.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.980+0000 7fb1ccc00640 1 -- 172.21.15.44:0/897748549 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) ==== 95+0+3923 (secure 0 0 0) 0x7fb1bc080440 con 0x7fb1d009c910 2024-07-17T08:47:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:00.982 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":211,"btime":"2024-07-17T08:46:52:986779+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/785062576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":785062576},{"type":"v1","addr":"172.21.15.44:6835","nonce":785062576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2824577450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2824577450},{"type":"v1","addr":"172.21.15.175:6835","nonce":2824577450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":11410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1421471670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1421471670},{"type":"v1","addr":"172.21.15.175:6836","nonce":1421471670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":204},{"gid":14772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3513115319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3513115319},{"type":"v1","addr":"172.21.15.44:6834","nonce":3513115319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":211}],"filesystems":[]} 2024-07-17T08:47:00.983 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 211 2024-07-17T08:47:00.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 msgr2=0x7fb1b0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 0x7fb1b0079190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005fd0 tx=0x7fb1c0003a50 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 msgr2=0x7fb1d0154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006310 tx=0x7fb1bc006340 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 shutdown_connections 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb1b0076cd0 0x7fb1b0079190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1d01f7160 0x7fb1d006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1d009c910 0x7fb1d0154e10 unknown :-1 s=CLOSED pgs=3430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.983+0000 7fb1d6200640 1 --2- 172.21.15.44:0/897748549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1b4004800 0x7fb1d00713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.984+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 >> 172.21.15.44:0/897748549 conn(0x7fb1d00b0e80 msgr2=0x7fb1d013d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:00.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.984+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 shutdown_connections 2024-07-17T08:47:00.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:00.984+0000 7fb1d6200640 1 -- 172.21.15.44:0/897748549 wait complete. 2024-07-17T08:47:01.006 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-17T08:47:01.006 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-17T08:47:01.006 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:47:01.006 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:47:01.006 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-17T08:47:01.007 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-17T08:47:01.007 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:47:01.007 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:47:01.007 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-17T08:47:01.007 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-17T08:47:01.007 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:47:01.007 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:47:01.008 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-17T08:47:01.008 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-17T08:47:01.008 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:47:01.008 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:47:01.023 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:47:01.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-17T08:47:01.027 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:47:01.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-17T08:47:01.036 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:47:01.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-17T08:47:01.041 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:47:01.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-17T08:47:01.072 INFO:tasks.ceph.mds.a:Started 2024-07-17T08:47:01.072 DEBUG:teuthology.parallel:result is None 2024-07-17T08:47:01.076 INFO:tasks.ceph.mds.c:Started 2024-07-17T08:47:01.076 DEBUG:teuthology.parallel:result is None 2024-07-17T08:47:01.086 INFO:tasks.ceph.mds.b:Started 2024-07-17T08:47:01.086 DEBUG:teuthology.parallel:result is None 2024-07-17T08:47:01.090 INFO:tasks.ceph.mds.d:Started 2024-07-17T08:47:01.090 DEBUG:teuthology.parallel:result is None 2024-07-17T08:47:01.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-17T08:47:01.222 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-07-17T08:47:01.223 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-07-17T08:47:01.235 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-07-17T08:47:01.239 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-07-17T08:47:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 -- 172.21.15.44:0/706111321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c0ce690 msgr2=0x7f181c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 --2- 172.21.15.44:0/706111321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c0ce690 0x7f181c15d510 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 crypto rx=0x7f1814005fd0 tx=0x7f181402aaa0 comp rx=0 tx=0).stop 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 -- 172.21.15.44:0/706111321 shutdown_connections 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 --2- 172.21.15.44:0/706111321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c0ce690 0x7f181c15d510 unknown :-1 s=CLOSED pgs=3436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 -- 172.21.15.44:0/706111321 >> 172.21.15.44:0/706111321 conn(0x7f181c0b4ff0 msgr2=0x7f181c0b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.273+0000 7f1821e00640 1 -- 172.21.15.44:0/706111321 shutdown_connections 2024-07-17T08:47:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.274+0000 7f1821e00640 1 -- 172.21.15.44:0/706111321 wait complete. 2024-07-17T08:47:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.275+0000 7f1821e00640 1 Processor -- start 2024-07-17T08:47:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.275+0000 7f1821e00640 1 -- start start 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 0x7f181c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c15d130 0x7f181c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f181c168920 con 0x7f181c161be0 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1687a0 con 0x7f181c15d130 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f181c168aa0 con 0x7f181c0ce690 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.276+0000 7f1821400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:01.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38698/0 (socket says 172.21.15.44:38698) 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 -- 172.21.15.44:0/3026486583 learned_addr learned my addr 172.21.15.44:0/3026486583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1820a00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 0x7f181c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f181be00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c15d130 0x7f181c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 -- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 msgr2=0x7f181c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 0x7f181c1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 -- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c15d130 msgr2=0x7f181c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c15d130 0x7f181c14e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 -- 172.21.15.44:0/3026486583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c0cd460 con 0x7f181c161be0 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1820a00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 0x7f181c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:47:01.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.277+0000 7f1821400640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 secure :-1 s=READY pgs=3437 cs=0 l=1 rev1=1 crypto rx=0x7f1814006010 tx=0x7f1814003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:01.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.278+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814044070 con 0x7f181c161be0 2024-07-17T08:47:01.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.278+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1814004250 con 0x7f181c161be0 2024-07-17T08:47:01.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.278+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814004680 con 0x7f181c161be0 2024-07-17T08:47:01.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.278+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c15c2c0 con 0x7f181c161be0 2024-07-17T08:47:01.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.278+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c15c880 con 0x7f181c161be0 2024-07-17T08:47:01.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.279+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c15d510 con 0x7f181c161be0 2024-07-17T08:47:01.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.280+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f1814047070 con 0x7f181c161be0 2024-07-17T08:47:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.282+0000 7f1818c00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 0x7f17fc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:01.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.282+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(371..371 src has 1..371) ==== 6655+0+0 (secure 0 0 0) 0x7f18140b99e0 con 0x7f181c161be0 2024-07-17T08:47:01.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.283+0000 7f1820a00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 0x7f17fc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:01.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.284+0000 7f1820a00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 0x7f17fc079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1810007fd0 tx=0x7f1810009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:01.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.286+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1814032030 con 0x7f181c161be0 2024-07-17T08:47:01.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:01.511+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f181c0b9460 con 0x7f181c161be0 2024-07-17T08:47:02.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.063+0000 7f1818c00640 1 -- 172.21.15.44:0/3026486583 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v373) ==== 97+0+0 (secure 0 0 0) 0x7f1814004820 con 0x7f181c161be0 2024-07-17T08:47:02.065 INFO:teuthology.orchestra.run.smithi044.stderr: removed all blocklist entries 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 msgr2=0x7f17fc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 0x7f17fc079190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1810007fd0 tx=0x7f1810009040 comp rx=0 tx=0).stop 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 msgr2=0x7f181c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 secure :-1 s=READY pgs=3437 cs=0 l=1 rev1=1 crypto rx=0x7f1814006010 tx=0x7f1814003f10 comp rx=0 tx=0).stop 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 shutdown_connections 2024-07-17T08:47:02.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f17fc076cd0 0x7f17fc079190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f181c161be0 0x7f181c159fe0 unknown :-1 s=CLOSED pgs=3437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f181c15d130 0x7f181c14e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 --2- 172.21.15.44:0/3026486583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f181c0ce690 0x7f181c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.067+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 >> 172.21.15.44:0/3026486583 conn(0x7f181c0b4ff0 msgr2=0x7f181c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.068+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 shutdown_connections 2024-07-17T08:47:02.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.068+0000 7f1821e00640 1 -- 172.21.15.44:0/3026486583 wait complete. 2024-07-17T08:47:02.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.302+0000 7f3765400640 1 -- 172.21.15.44:0/2783734241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37440051c0 msgr2=0x7f374400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.302+0000 7f3765400640 1 --2- 172.21.15.44:0/2783734241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37440051c0 0x7f374400b9c0 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 crypto rx=0x7f37600b5b20 tx=0x7f3744005830 comp rx=0 tx=0).stop 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.303+0000 7f375f200640 1 -- 172.21.15.44:0/2783734241 reap_dead start 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.303+0000 7f3765400640 1 -- 172.21.15.44:0/2783734241 shutdown_connections 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.303+0000 7f3765400640 1 -- 172.21.15.44:0/2783734241 >> 172.21.15.44:0/2783734241 conn(0x7f37600d00a0 msgr2=0x7f37600d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.303+0000 7f3765400640 1 -- 172.21.15.44:0/2783734241 shutdown_connections 2024-07-17T08:47:02.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.303+0000 7f3765400640 1 -- 172.21.15.44:0/2783734241 wait complete. 2024-07-17T08:47:02.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.305+0000 7f3765400640 1 Processor -- start 2024-07-17T08:47:02.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.305+0000 7f3765400640 1 -- start start 2024-07-17T08:47:02.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f376014dbd0 0x7f37601f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 0x7f3760158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37600ce100 con 0x7f376015cbb0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37600cdf80 con 0x7f376014e4f0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f3765400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37600ce280 con 0x7f376014dbd0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f375de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f375de00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38718/0 (socket says 172.21.15.44:38718) 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f375de00640 1 -- 172.21.15.44:0/2158422488 learned_addr learned my addr 172.21.15.44:0/2158422488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f375e800640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f376014dbd0 0x7f37601f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.306+0000 7f375f200640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 0x7f3760158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 -- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f376014dbd0 msgr2=0x7f37601f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f376014dbd0 0x7f37601f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 -- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 msgr2=0x7f3760158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 0x7f3760158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 -- 172.21.15.44:0/2158422488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37600aa280 con 0x7f376015cbb0 2024-07-17T08:47:02.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375f200640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 0x7f3760158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:02.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f375de00640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f3748010c70 tx=0x7f374800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748017070 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3748011640 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748011a90 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37600aa560 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.307+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37601f7a80 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.308+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37600b5200 con 0x7f376015cbb0 2024-07-17T08:47:02.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.310+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f3748011c30 con 0x7f376015cbb0 2024-07-17T08:47:02.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.311+0000 7f3756a00640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 0x7f373c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.311+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(375..375 src has 1..375) ==== 5881+0+0 (secure 0 0 0) 0x7f37480994b0 con 0x7f376015cbb0 2024-07-17T08:47:02.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.312+0000 7f375e800640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 0x7f373c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.313+0000 7f375e800640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 0x7f373c079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3750001860 tx=0x7f37500035f0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:02.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.316+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f37480627b0 con 0x7f376015cbb0 2024-07-17T08:47:02.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.606+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3760157d10 con 0x7f376015cbb0 2024-07-17T08:47:02.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.608+0000 7f3756a00640 1 -- 172.21.15.44:0/2158422488 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v85) ==== 80+0+5304 (secure 0 0 0) 0x7f37480620b0 con 0x7f376015cbb0 2024-07-17T08:47:02.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_dump": [ 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.a", 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmzytDJxAAmrnA3Zx98sBAKV8v4ecabw==", 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.b", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmOg1vCRAAAm0NQ7fPa4ciNqMs1jFm4Q==", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.c", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdm6lGQNhAAjUx7vjEiJhMH9GcZskk5Gg==", 2024-07-17T08:47:02.611 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mds.d", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdm38G/FxAAI+Fn9FRIzDLJVgkLr42dGQ==", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:47:02.612 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.0", 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB8apdmcHkjHBAAUNRdDz0ngUO9LqEkaa6n2Q==", 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.1", 2024-07-17T08:47:02.613 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB/apdmitB/NxAAS8G1ab/2+gLEfApgTToClQ==", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.2", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCDapdmEVDJBxAAggoJp8LJPw/KgvBnANgkeQ==", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.3", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCGapdmBP3nIBAAgijMr34GkY90rEzaAyX85w==", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.4", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCMapdmlmyiLxAAl8X0KPoXqq18RByiCL7fgw==", 2024-07-17T08:47:02.615 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.5", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCQapdmoXphDxAA/8MwSk6YibeSb1zxmiJYzQ==", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.6", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCTapdmebVwGhAAKMLANVtZkL4yfelziYh7sw==", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "osd.7", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQCWapdmzKPENRAAvqGE8wRrEQEIvjmyuV+mQg==", 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.617 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow profile osd", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile osd", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.0", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg==", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow r", 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.1", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmU8+tNxAA8kRQH7mlqBfnxoSgfDC5AA==", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.2", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB4apdmzHcALhAAByIb+YyOgsRyLCDV9AD+xA==", 2024-07-17T08:47:02.619 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.3", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB5apdm9DpZBRAAkKHJqAxicIFOEZae4izpyQ==", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow r", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow rw", 2024-07-17T08:47:02.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow rwx" 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "client.admin", 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB2apdmy1ChHxAA1RLSPJF1fkoexd5OrIDFaw==", 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": "allow *", 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow *", 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.x", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmyiY8HBAAsbsV2Vs2PLlWax7GwNJFYQ==", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "entity": "mgr.y", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "AQB3apdmuLjbERAAFAb2MnI9ShplUhIFTeflPg==", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": { 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": "allow *", 2024-07-17T08:47:02.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": "allow profile mgr", 2024-07-17T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": "allow *" 2024-07-17T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-07-17T08:47:02.623 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 msgr2=0x7f373c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 0x7f373c079190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3750001860 tx=0x7f37500035f0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 msgr2=0x7f376014dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f3748010c70 tx=0x7f374800e720 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 shutdown_connections 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f373c076cd0 0x7f373c079190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f376014e4f0 0x7f3760158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f376015cbb0 0x7f376014dfb0 unknown :-1 s=CLOSED pgs=3440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 --2- 172.21.15.44:0/2158422488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f376014dbd0 0x7f37601f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 >> 172.21.15.44:0/2158422488 conn(0x7f37600d00a0 msgr2=0x7f37600cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.610+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 shutdown_connections 2024-07-17T08:47:02.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.611+0000 7f3765400640 1 -- 172.21.15.44:0/2158422488 wait complete. 2024-07-17T08:47:02.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-17T08:47:02.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 -- 172.21.15.44:0/4281876439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 msgr2=0x7f789009b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 --2- 172.21.15.44:0/4281876439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f789009b9a0 secure :-1 s=READY pgs=3441 cs=0 l=1 rev1=1 crypto rx=0x7f789006dc90 tx=0x7f787802a560 comp rx=0 tx=0).stop 2024-07-17T08:47:02.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 -- 172.21.15.44:0/4281876439 shutdown_connections 2024-07-17T08:47:02.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 --2- 172.21.15.44:0/4281876439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f789009b9a0 unknown :-1 s=CLOSED pgs=3441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 --2- 172.21.15.44:0/4281876439 >> v2:172.21.15.175:3300/0 conn(0x7f7890157ba0 0x7f78900cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 -- 172.21.15.44:0/4281876439 >> 172.21.15.44:0/4281876439 conn(0x7f78900b0e80 msgr2=0x7f78900b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:02.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 -- 172.21.15.44:0/4281876439 shutdown_connections 2024-07-17T08:47:02.837 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.834+0000 7f788fe00640 1 -- 172.21.15.44:0/4281876439 wait complete. 2024-07-17T08:47:02.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.836+0000 7f788fe00640 1 Processor -- start 2024-07-17T08:47:02.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.836+0000 7f788fe00640 1 -- start start 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 0x7f78901f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 0x7f78901f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78900ac930 con 0x7f78900cf8e0 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78900ac7b0 con 0x7f789009bee0 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.837+0000 7f788fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78900b38c0 con 0x7f7890157ba0 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38732/0 (socket says 172.21.15.44:38732) 2024-07-17T08:47:02.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 -- 172.21.15.44:0/4248276699 learned_addr learned my addr 172.21.15.44:0/4248276699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788ea00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 0x7f78901f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 -- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 msgr2=0x7f78901f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788f400640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 0x7f78901f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 0x7f78901f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 -- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 msgr2=0x7f78901f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 0x7f78901f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 -- 172.21.15.44:0/4248276699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901f38b0 con 0x7f78900cf8e0 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788ea00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 0x7f78901f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788f400640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 0x7f78901f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:02.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.838+0000 7f788e000640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 crypto rx=0x7f7878009850 tx=0x7f78780039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.839+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787803f070 con 0x7f78900cf8e0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.839+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7878004890 con 0x7f78900cf8e0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.839+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878035030 con 0x7f78900cf8e0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.839+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7890146eb0 con 0x7f78900cf8e0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.839+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7890147470 con 0x7f78900cf8e0 2024-07-17T08:47:02.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.840+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78900b3e40 con 0x7f78900cf8e0 2024-07-17T08:47:02.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.841+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7878042070 con 0x7f78900cf8e0 2024-07-17T08:47:02.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.842+0000 7f7886a00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 0x7f786c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:02.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.842+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(375..375 src has 1..375) ==== 5881+0+0 (secure 0 0 0) 0x7f78780b95c0 con 0x7f78900cf8e0 2024-07-17T08:47:02.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.843+0000 7f788ea00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 0x7f786c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:02.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.844+0000 7f788ea00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 0x7f786c079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f788000a870 tx=0x7f7880008040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:02.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:02.847+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7878082720 con 0x7f78900cf8e0 2024-07-17T08:47:03.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.057+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f78901f3ab0 con 0x7f78900cf8e0 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.063+0000 7f7886a00640 1 -- 172.21.15.44:0/4248276699 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v375) ==== 84+0+65 (secure 0 0 0) 0x7f7878082020 con 0x7f78900cf8e0 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-07-17T08:47:03.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-07-17T08:47:03.066 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-07-17T08:47:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.066+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 msgr2=0x7f786c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.066+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 0x7f786c079190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f788000a870 tx=0x7f7880008040 comp rx=0 tx=0).stop 2024-07-17T08:47:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.066+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 msgr2=0x7f78900cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.066+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 crypto rx=0x7f7878009850 tx=0x7f78780039e0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 shutdown_connections 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f786c076cd0 0x7f786c079190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7890157ba0 0x7f78901f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78900cf8e0 0x7f78900cd800 unknown :-1 s=CLOSED pgs=3442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 --2- 172.21.15.44:0/4248276699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789009bee0 0x7f78901f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 >> 172.21.15.44:0/4248276699 conn(0x7f78900b0e80 msgr2=0x7f789006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 shutdown_connections 2024-07-17T08:47:03.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.067+0000 7f788fe00640 1 -- 172.21.15.44:0/4248276699 wait complete. 2024-07-17T08:47:03.089 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-17T08:47:03.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 -- 172.21.15.44:0/3776225858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 msgr2=0x7f728c0ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 --2- 172.21.15.44:0/3776225858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c0ad250 secure :-1 s=READY pgs=3444 cs=0 l=1 rev1=1 crypto rx=0x7f727c009830 tx=0x7f727c02a510 comp rx=0 tx=0).stop 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 -- 172.21.15.44:0/3776225858 shutdown_connections 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 --2- 172.21.15.44:0/3776225858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c0ad250 unknown :-1 s=CLOSED pgs=3444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 -- 172.21.15.44:0/3776225858 >> 172.21.15.44:0/3776225858 conn(0x7f728c0cf050 msgr2=0x7f728c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 -- 172.21.15.44:0/3776225858 shutdown_connections 2024-07-17T08:47:03.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.306+0000 7f728be00640 1 -- 172.21.15.44:0/3776225858 wait complete. 2024-07-17T08:47:03.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.308+0000 7f728be00640 1 Processor -- start 2024-07-17T08:47:03.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.308+0000 7f728be00640 1 -- start start 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 0x7f728c1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae490 0x7f728c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f728c073450 con 0x7f728c155860 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0732d0 con 0x7f728c0ad790 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0735d0 con 0x7f728c0ae490 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728b400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38742/0 (socket says 172.21.15.44:38742) 2024-07-17T08:47:03.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728b400640 1 -- 172.21.15.44:0/55019913 learned_addr learned my addr 172.21.15.44:0/55019913 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:03.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.309+0000 7f728aa00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 0x7f728c1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:03.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 -- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae490 msgr2=0x7f728c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:03.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f7283e00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae490 0x7f728c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae490 0x7f728c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 -- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 msgr2=0x7f728c1f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 0x7f728c1f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 -- 172.21.15.44:0/55019913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0b0940 con 0x7f728c155860 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728aa00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 0x7f728c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728b400640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 crypto rx=0x7f728400cbc0 tx=0x7f728400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284016020 con 0x7f728c155860 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f728400ad90 con 0x7f728c155860 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284012680 con 0x7f728c155860 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.310+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c099190 con 0x7f728c155860 2024-07-17T08:47:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.311+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c099750 con 0x7f728c155860 2024-07-17T08:47:03.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.312+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c073ac0 con 0x7f728c155860 2024-07-17T08:47:03.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.313+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7284019020 con 0x7f728c155860 2024-07-17T08:47:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.314+0000 7f7283400640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 0x7f7268079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:03.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.314+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(375..375 src has 1..375) ==== 5881+0+0 (secure 0 0 0) 0x7f7284098c10 con 0x7f728c155860 2024-07-17T08:47:03.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.314+0000 7f728aa00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 0x7f7268079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:03.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.316+0000 7f728aa00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 0x7f7268079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f727c034000 tx=0x7f727c004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:03.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.318+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7284020020 con 0x7f728c155860 2024-07-17T08:47:03.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:03.529+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f728c06cd20 con 0x7f728c155860 2024-07-17T08:47:04.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.077+0000 7f7283400640 1 -- 172.21.15.44:0/55019913 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v376) ==== 138+0+0 (secure 0 0 0) 0x7f7284061f10 con 0x7f728c155860 2024-07-17T08:47:04.079 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.080+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 msgr2=0x7f7268079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.080+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 0x7f7268079190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f727c034000 tx=0x7f727c004730 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.080+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 msgr2=0x7f728c15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.080+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 crypto rx=0x7f728400cbc0 tx=0x7f728400a060 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 shutdown_connections 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7268076cd0 0x7f7268079190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f728c155860 0x7f728c15a730 unknown :-1 s=CLOSED pgs=3445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f728c0ae490 0x7f728c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 --2- 172.21.15.44:0/55019913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f728c0ad790 0x7f728c1f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 >> 172.21.15.44:0/55019913 conn(0x7f728c0cf050 msgr2=0x7f728c15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:04.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 shutdown_connections 2024-07-17T08:47:04.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.081+0000 7f728be00640 1 -- 172.21.15.44:0/55019913 wait complete. 2024-07-17T08:47:04.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 -- 172.21.15.44:0/1698391997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca40152220 msgr2=0x7fca4006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 --2- 172.21.15.44:0/1698391997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca40152220 0x7fca4006d620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fca34005fd0 tx=0x7fca3402a540 comp rx=0 tx=0).stop 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 -- 172.21.15.44:0/1698391997 shutdown_connections 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 --2- 172.21.15.44:0/1698391997 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca400cbab0 0x7fca4015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 --2- 172.21.15.44:0/1698391997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca40152220 0x7fca4006d620 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 -- 172.21.15.44:0/1698391997 >> 172.21.15.44:0/1698391997 conn(0x7fca400b0eb0 msgr2=0x7fca400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 -- 172.21.15.44:0/1698391997 shutdown_connections 2024-07-17T08:47:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.314+0000 7fca48000640 1 -- 172.21.15.44:0/1698391997 wait complete. 2024-07-17T08:47:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.316+0000 7fca48000640 1 Processor -- start 2024-07-17T08:47:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.316+0000 7fca48000640 1 -- start start 2024-07-17T08:47:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4006ed30 0x7fca401476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca40152220 0x7fca401f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fca400cd750 con 0x7fca400cbab0 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca400cd5d0 con 0x7fca4006ed30 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca48000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca400cd8d0 con 0x7fca40152220 2024-07-17T08:47:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca44c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca44c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38756/0 (socket says 172.21.15.44:38756) 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca44c00640 1 -- 172.21.15.44:0/3281733821 learned_addr learned my addr 172.21.15.44:0/3281733821 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca45600640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4006ed30 0x7fca401476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.318+0000 7fca46000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca40152220 0x7fca401f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 -- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca40152220 msgr2=0x7fca401f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca40152220 0x7fca401f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 -- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4006ed30 msgr2=0x7fca401476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4006ed30 0x7fca401476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 -- 172.21.15.44:0/3281733821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40153da0 con 0x7fca400cbab0 2024-07-17T08:47:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca44c00640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 secure :-1 s=READY pgs=3446 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c450 tx=0x7fca2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0052d0 con 0x7fca400cbab0 2024-07-17T08:47:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca2c005470 con 0x7fca400cbab0 2024-07-17T08:47:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c012870 con 0x7fca400cbab0 2024-07-17T08:47:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.319+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca40154080 con 0x7fca400cbab0 2024-07-17T08:47:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.320+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca40154640 con 0x7fca400cbab0 2024-07-17T08:47:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.321+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca400cddf0 con 0x7fca400cbab0 2024-07-17T08:47:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.322+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fca2c0035f0 con 0x7fca400cbab0 2024-07-17T08:47:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.323+0000 7fca3d600640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 0x7fca20079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.323+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(376..376 src has 1..376) ==== 6251+0+0 (secure 0 0 0) 0x7fca2c0982e0 con 0x7fca400cbab0 2024-07-17T08:47:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.324+0000 7fca45600640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 0x7fca20079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:04.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.325+0000 7fca45600640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 0x7fca20079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca34033000 tx=0x7fca34004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:04.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.328+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fca2c061470 con 0x7fca400cbab0 2024-07-17T08:47:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:04.539+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fca40154810 con 0x7fca400cbab0 2024-07-17T08:47:05.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.096+0000 7fca3d600640 1 -- 172.21.15.44:0/3281733821 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v377) ==== 170+0+0 (secure 0 0 0) 0x7fca2c060d70 con 0x7fca400cbab0 2024-07-17T08:47:05.099 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-07-17T08:47:05.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 msgr2=0x7fca20079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:05.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 0x7fca20079190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca34033000 tx=0x7fca34004730 comp rx=0 tx=0).stop 2024-07-17T08:47:05.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 msgr2=0x7fca401f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:05.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 secure :-1 s=READY pgs=3446 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c450 tx=0x7fca2c00c910 comp rx=0 tx=0).stop 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 shutdown_connections 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fca20076cd0 0x7fca20079190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca40152220 0x7fca401f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca400cbab0 0x7fca401f2760 unknown :-1 s=CLOSED pgs=3446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 --2- 172.21.15.44:0/3281733821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4006ed30 0x7fca401476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.100+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 >> 172.21.15.44:0/3281733821 conn(0x7fca400b0eb0 msgr2=0x7fca400b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.101+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 shutdown_connections 2024-07-17T08:47:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.101+0000 7fca48000640 1 -- 172.21.15.44:0/3281733821 wait complete. 2024-07-17T08:47:05.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 -- 172.21.15.44:0/2235450301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac0b3280 msgr2=0x7f7eac06eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2235450301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac0b3280 0x7f7eac06eb90 secure :-1 s=READY pgs=3447 cs=0 l=1 rev1=1 crypto rx=0x7f7e94005fd0 tx=0x7f7e9402a5d0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 -- 172.21.15.44:0/2235450301 shutdown_connections 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2235450301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac0b3280 0x7f7eac06eb90 unknown :-1 s=CLOSED pgs=3447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2235450301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac06f6d0 0x7f7eac15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 -- 172.21.15.44:0/2235450301 >> 172.21.15.44:0/2235450301 conn(0x7f7eac0b0eb0 msgr2=0x7f7eac0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:05.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 -- 172.21.15.44:0/2235450301 shutdown_connections 2024-07-17T08:47:05.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.317+0000 7f7eb3400640 1 -- 172.21.15.44:0/2235450301 wait complete. 2024-07-17T08:47:05.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.319+0000 7f7eb3400640 1 Processor -- start 2024-07-17T08:47:05.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.319+0000 7f7eb3400640 1 -- start start 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac0ae560 0x7f7eac0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7eac0b3280 0x7f7eac071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0accd0 con 0x7f7eac06f6d0 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0aeba0 con 0x7f7eac0b3280 2024-07-17T08:47:05.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.320+0000 7f7eb3400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac14fd60 con 0x7f7eac0ae560 2024-07-17T08:47:05.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:05.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38778/0 (socket says 172.21.15.44:38778) 2024-07-17T08:47:05.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 -- 172.21.15.44:0/2953933626 learned_addr learned my addr 172.21.15.44:0/2953933626 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb1400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7eac0b3280 0x7f7eac071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eabe00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac0ae560 0x7f7eac0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 -- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac0ae560 msgr2=0x7f7eac0715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac0ae560 0x7f7eac0715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 -- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7eac0b3280 msgr2=0x7f7eac071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7eac0b3280 0x7f7eac071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.321+0000 7f7eb0a00640 1 -- 172.21.15.44:0/2953933626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac142490 con 0x7f7eac06f6d0 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7eb0a00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ea80 tx=0x7f7e9c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:05.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00ce40 con 0x7f7eac06f6d0 2024-07-17T08:47:05.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e9c004c10 con 0x7f7eac06f6d0 2024-07-17T08:47:05.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c005020 con 0x7f7eac06f6d0 2024-07-17T08:47:05.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac142770 con 0x7f7eac06f6d0 2024-07-17T08:47:05.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.322+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac142d30 con 0x7f7eac06f6d0 2024-07-17T08:47:05.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.323+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac15a5a0 con 0x7f7eac06f6d0 2024-07-17T08:47:05.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.326+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7e9c0051e0 con 0x7f7eac06f6d0 2024-07-17T08:47:05.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.327+0000 7f7ea8c00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 0x7f7e88079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:05.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.327+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(377..377 src has 1..377) ==== 6633+0+0 (secure 0 0 0) 0x7f7e9c0997c0 con 0x7f7eac06f6d0 2024-07-17T08:47:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.327+0000 7f7eabe00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 0x7f7e88079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.329+0000 7f7eabe00640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 0x7f7e88079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e94033000 tx=0x7f7e94004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.330+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7e9c016030 con 0x7f7eac06f6d0 2024-07-17T08:47:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:05.576+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7eac0cb2b0 con 0x7f7eac06f6d0 2024-07-17T08:47:06.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.100+0000 7f7ea8c00640 1 -- 172.21.15.44:0/2953933626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 51 and data pool 52 v214) ==== 174+0+0 (secure 0 0 0) 0x7f7e9c0626b0 con 0x7f7eac06f6d0 2024-07-17T08:47:06.103 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 51 and data pool 52 2024-07-17T08:47:06.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 msgr2=0x7f7e88079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 0x7f7e88079190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e94033000 tx=0x7f7e94004730 comp rx=0 tx=0).stop 2024-07-17T08:47:06.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 msgr2=0x7f7eac0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ea80 tx=0x7f7e9c00ef40 comp rx=0 tx=0).stop 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 shutdown_connections 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7e88076cd0 0x7f7e88079190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7eac0b3280 0x7f7eac071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7eac0ae560 0x7f7eac0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 --2- 172.21.15.44:0/2953933626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7eac06f6d0 0x7f7eac0caca0 unknown :-1 s=CLOSED pgs=3448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.105+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 >> 172.21.15.44:0/2953933626 conn(0x7f7eac0b0eb0 msgr2=0x7f7eac15c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.106+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 shutdown_connections 2024-07-17T08:47:06.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.107+0000 7f7eb3400640 1 -- 172.21.15.44:0/2953933626 wait complete. 2024-07-17T08:47:06.150 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:47:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 -- 172.21.15.44:0/2327447203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1c4004d40 msgr2=0x7fa1c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 --2- 172.21.15.44:0/2327447203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1c4004d40 0x7fa1c40051c0 secure :-1 s=READY pgs=3450 cs=0 l=1 rev1=1 crypto rx=0x7fa1e015edf0 tx=0x7fa1c4005830 comp rx=0 tx=0).stop 2024-07-17T08:47:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e5600640 1 -- 172.21.15.44:0/2327447203 reap_dead start 2024-07-17T08:47:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 -- 172.21.15.44:0/2327447203 shutdown_connections 2024-07-17T08:47:06.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 -- 172.21.15.44:0/2327447203 >> 172.21.15.44:0/2327447203 conn(0x7fa1e00b0fb0 msgr2=0x7fa1e00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:06.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 -- 172.21.15.44:0/2327447203 shutdown_connections 2024-07-17T08:47:06.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.319+0000 7fa1e7600640 1 -- 172.21.15.44:0/2327447203 wait complete. 2024-07-17T08:47:06.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.321+0000 7fa1e7600640 1 Processor -- start 2024-07-17T08:47:06.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.321+0000 7fa1e7600640 1 -- start start 2024-07-17T08:47:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 0x7fa1e01f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e00740c0 0x7fa1e00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00adb50 con 0x7fa1e0150350 2024-07-17T08:47:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00ad9d0 con 0x7fa1c4004d40 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00cdda0 con 0x7fa1e00740c0 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.322+0000 7fa1e5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38784/0 (socket says 172.21.15.44:38784) 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 -- 172.21.15.44:0/817469871 learned_addr learned my addr 172.21.15.44:0/817469871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e4c00640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 0x7fa1e01f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1d7e00640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e00740c0 0x7fa1e00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 -- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e00740c0 msgr2=0x7fa1e00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e00740c0 0x7fa1e00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 -- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 msgr2=0x7fa1e01f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 0x7fa1e01f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 -- 172.21.15.44:0/817469871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00cb620 con 0x7fa1e0150350 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e4c00640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 0x7fa1e01f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.323+0000 7fa1e5600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 secure :-1 s=READY pgs=3451 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802aba0 tx=0x7fa1d8006d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.324+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8042070 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.324+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1d8006070 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.324+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80062a0 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.324+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e01f4100 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.324+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e01f46c0 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.325+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e00cd690 con 0x7fa1e0150350 2024-07-17T08:47:06.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.326+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fa1d8045070 con 0x7fa1e0150350 2024-07-17T08:47:06.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.327+0000 7fa1dd600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 0x7fa1bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.327+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(378..378 src has 1..378) ==== 6733+0+0 (secure 0 0 0) 0x7fa1d80b7f40 con 0x7fa1e0150350 2024-07-17T08:47:06.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.328+0000 7fa1e4c00640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 0x7fa1bc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.329+0000 7fa1e4c00640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 0x7fa1bc0791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005fd0 tx=0x7fa1d000c2d0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:06.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.332+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa1e00cb620 con 0x7fa1e0150350 2024-07-17T08:47:06.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.534+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1e0074550 con 0x7fa1e0150350 2024-07-17T08:47:06.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.536+0000 7fa1dd600640 1 -- 172.21.15.44:0/817469871 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v378) ==== 74+0+17924 (secure 0 0 0) 0x7fa1d8080700 con 0x7fa1e0150350 2024-07-17T08:47:06.539 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:06.539 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":378,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:47:05.578404+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":52,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":51,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:47:03.554223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":52,"pool_name":"cephfs_data","create_time":"2024-07-17T08:47:04.562957+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 msgr2=0x7fa1bc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 0x7fa1bc0791c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005fd0 tx=0x7fa1d000c2d0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 msgr2=0x7fa1e01507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 secure :-1 s=READY pgs=3451 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802aba0 tx=0x7fa1d8006d30 comp rx=0 tx=0).stop 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 shutdown_connections 2024-07-17T08:47:06.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa1bc076d00 0x7fa1bc0791c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e0150350 0x7fa1e01507d0 unknown :-1 s=CLOSED pgs=3451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e00740c0 0x7fa1e00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 --2- 172.21.15.44:0/817469871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1c4004d40 0x7fa1e01f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 >> 172.21.15.44:0/817469871 conn(0x7fa1e00b0fb0 msgr2=0x7fa1e00cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.539+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 shutdown_connections 2024-07-17T08:47:06.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.540+0000 7fa1e7600640 1 -- 172.21.15.44:0/817469871 wait complete. 2024-07-17T08:47:06.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 -- 172.21.15.44:0/4233208139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a40ac3e0 msgr2=0x7f71a40ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 --2- 172.21.15.44:0/4233208139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a40ac3e0 0x7f71a40ac840 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c02b860 comp rx=0 tx=0).stop 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 -- 172.21.15.44:0/4233208139 shutdown_connections 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 --2- 172.21.15.44:0/4233208139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a40ac3e0 0x7f71a40ac840 unknown :-1 s=CLOSED pgs=3452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 --2- 172.21.15.44:0/4233208139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71a406f710 0x7f71a4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 -- 172.21.15.44:0/4233208139 >> 172.21.15.44:0/4233208139 conn(0x7f71a40b0fb0 msgr2=0x7f71a40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 -- 172.21.15.44:0/4233208139 shutdown_connections 2024-07-17T08:47:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.762+0000 7f71ab400640 1 -- 172.21.15.44:0/4233208139 wait complete. 2024-07-17T08:47:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.764+0000 7f71ab400640 1 Processor -- start 2024-07-17T08:47:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.764+0000 7f71ab400640 1 -- start start 2024-07-17T08:47:06.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a40ac3e0 0x7f71a41f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71a413ed60 0x7f71a41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71a40ae800 con 0x7f71a406f710 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71a406f0a0 con 0x7f71a40ac3e0 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71ab400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71a40ae980 con 0x7f71a413ed60 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38794/0 (socket says 172.21.15.44:38794) 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 -- 172.21.15.44:0/1595342902 learned_addr learned my addr 172.21.15.44:0/1595342902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f719be00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a40ac3e0 0x7f71a41f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 -- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71a413ed60 msgr2=0x7f71a41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71a413ed60 0x7f71a41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 -- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a40ac3e0 msgr2=0x7f71a41f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.765+0000 7f71a8a00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a40ac3e0 0x7f71a41f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71a8a00640 1 -- 172.21.15.44:0/1595342902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a415c500 con 0x7f71a406f710 2024-07-17T08:47:06.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71a8a00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f719400c890 tx=0x7f719400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194004220 con 0x7f71a406f710 2024-07-17T08:47:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71940079b0 con 0x7f71a406f710 2024-07-17T08:47:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a415c7e0 con 0x7f71a406f710 2024-07-17T08:47:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194007de0 con 0x7f71a406f710 2024-07-17T08:47:06.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.766+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a40723b0 con 0x7f71a406f710 2024-07-17T08:47:06.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.767+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a413fad0 con 0x7f71a406f710 2024-07-17T08:47:06.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.769+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7194005020 con 0x7f71a406f710 2024-07-17T08:47:06.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.771+0000 7f71a1600640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 0x7f7180079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:06.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.771+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(378..378 src has 1..378) ==== 6733+0+0 (secure 0 0 0) 0x7f71940992d0 con 0x7f71a406f710 2024-07-17T08:47:06.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.771+0000 7f719be00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 0x7f7180079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:06.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.773+0000 7f719be00640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 0x7f7180079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f718c035000 tx=0x7f718c004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:06.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.774+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7194014030 con 0x7f71a406f710 2024-07-17T08:47:06.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.983+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a4159a40 con 0x7f71a406f710 2024-07-17T08:47:06.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.986+0000 7f71a1600640 1 -- 172.21.15.44:0/1595342902 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v378) ==== 74+0+17924 (secure 0 0 0) 0x7f7194062160 con 0x7f71a406f710 2024-07-17T08:47:06.988 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:06.988 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":378,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:47:05.578404+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":52,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":51,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:47:03.554223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":52,"pool_name":"cephfs_data","create_time":"2024-07-17T08:47:04.562957+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 msgr2=0x7f7180079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 0x7f7180079190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f718c035000 tx=0x7f718c004730 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 msgr2=0x7f71a4158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f719400c890 tx=0x7f719400cd50 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 shutdown_connections 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7180076cd0 0x7f7180079190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71a413ed60 0x7f71a41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a40ac3e0 0x7f71a41f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 --2- 172.21.15.44:0/1595342902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71a406f710 0x7f71a4158ef0 unknown :-1 s=CLOSED pgs=3453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 >> 172.21.15.44:0/1595342902 conn(0x7f71a40b0fb0 msgr2=0x7f71a4157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:06.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 shutdown_connections 2024-07-17T08:47:06.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:06.989+0000 7f71ab400640 1 -- 172.21.15.44:0/1595342902 wait complete. 2024-07-17T08:47:07.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 -- 172.21.15.44:0/2488068469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 msgr2=0x7fd9f015c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 --2- 172.21.15.44:0/2488068469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015c140 secure :-1 s=READY pgs=3454 cs=0 l=1 rev1=1 crypto rx=0x7fd9f015ebe0 tx=0x7fd9e402a560 comp rx=0 tx=0).stop 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 -- 172.21.15.44:0/2488068469 shutdown_connections 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 --2- 172.21.15.44:0/2488068469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 0x7fd9f006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 --2- 172.21.15.44:0/2488068469 >> v2:172.21.15.175:3300/0 conn(0x7fd9f0152300 0x7fd9f006f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 --2- 172.21.15.44:0/2488068469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015c140 unknown :-1 s=CLOSED pgs=3454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 -- 172.21.15.44:0/2488068469 >> 172.21.15.44:0/2488068469 conn(0x7fd9f00cf0d0 msgr2=0x7fd9f00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 -- 172.21.15.44:0/2488068469 shutdown_connections 2024-07-17T08:47:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.209+0000 7fd9f6000640 1 -- 172.21.15.44:0/2488068469 wait complete. 2024-07-17T08:47:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.210+0000 7fd9f6000640 1 Processor -- start 2024-07-17T08:47:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 -- start start 2024-07-17T08:47:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 0x7fd9f014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f0152300 0x7fd9f015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00cb410 con 0x7fd9f0151f20 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00cb290 con 0x7fd9f0152300 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.211+0000 7fd9f6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f00cb590 con 0x7fd9f00cd0b0 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38812/0 (socket says 172.21.15.44:38812) 2024-07-17T08:47:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 -- 172.21.15.44:0/3697920234 learned_addr learned my addr 172.21.15.44:0/3697920234 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9f5600640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f0152300 0x7fd9f015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9f4c00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 0x7fd9f014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 -- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 msgr2=0x7fd9f014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 0x7fd9f014a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 -- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f0152300 msgr2=0x7fd9f015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f0152300 0x7fd9f015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.212+0000 7fd9efe00640 1 -- 172.21.15.44:0/3697920234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f01f7760 con 0x7fd9f0151f20 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9efe00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b4a0 tx=0x7fd9dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc014070 con 0x7fd9f0151f20 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9dc010700 con 0x7fd9f0151f20 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc010b50 con 0x7fd9f0151f20 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f01f7a40 con 0x7fd9f0151f20 2024-07-17T08:47:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.213+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f01f8000 con 0x7fd9f0151f20 2024-07-17T08:47:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.214+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f00af3e0 con 0x7fd9f0151f20 2024-07-17T08:47:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.215+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fd9dc017070 con 0x7fd9f0151f20 2024-07-17T08:47:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.216+0000 7fd9ecc00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 0x7fd9d0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:07.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.216+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fd9dc098480 con 0x7fd9f0151f20 2024-07-17T08:47:07.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.217+0000 7fd9f4c00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 0x7fd9d0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:07.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.218+0000 7fd9f4c00640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 0x7fd9d0079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4033000 tx=0x7fd9e4004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:07.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.223+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd9f00af3e0 con 0x7fd9f0151f20 2024-07-17T08:47:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:07.481+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd9f00cbac0 con 0x7fd9f0151f20 2024-07-17T08:47:08.078 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-07-17T08:47:08.077+0000 7fb67ae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-17T08:47:08.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.138+0000 7fd9ecc00640 1 -- 172.21.15.44:0/3697920234 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v217) ==== 118+0+0 (secure 0 0 0) 0x7fd9dc0613a0 con 0x7fd9f0151f20 2024-07-17T08:47:08.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.143+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 msgr2=0x7fd9d0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.143+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 0x7fd9d0079190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4033000 tx=0x7fd9e4004730 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.143+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 msgr2=0x7fd9f015a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.143+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b4a0 tx=0x7fd9dc00b960 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.143+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 shutdown_connections 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd9d0076cd0 0x7fd9d0079190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9f0152300 0x7fd9f015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9f0151f20 0x7fd9f015a190 unknown :-1 s=CLOSED pgs=3455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 --2- 172.21.15.44:0/3697920234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9f00cd0b0 0x7fd9f014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 >> 172.21.15.44:0/3697920234 conn(0x7fd9f00cf0d0 msgr2=0x7fd9f00af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 shutdown_connections 2024-07-17T08:47:08.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.144+0000 7fd9f6000640 1 -- 172.21.15.44:0/3697920234 wait complete. 2024-07-17T08:47:08.168 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-17T08:47:08.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 -- 172.21.15.44:0/1584855228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b40acab0 msgr2=0x7f08b40b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 --2- 172.21.15.44:0/1584855228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b40acab0 0x7f08b40b3280 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7f08ac005fd0 tx=0x7f08ac02a8d0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 -- 172.21.15.44:0/1584855228 shutdown_connections 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 --2- 172.21.15.44:0/1584855228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b40acab0 0x7f08b40b3280 unknown :-1 s=CLOSED pgs=3457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 -- 172.21.15.44:0/1584855228 >> 172.21.15.44:0/1584855228 conn(0x7f08b40b0eb0 msgr2=0x7f08b40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 -- 172.21.15.44:0/1584855228 shutdown_connections 2024-07-17T08:47:08.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.386+0000 7f08ba800640 1 -- 172.21.15.44:0/1584855228 wait complete. 2024-07-17T08:47:08.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.388+0000 7f08ba800640 1 Processor -- start 2024-07-17T08:47:08.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.388+0000 7f08ba800640 1 -- start start 2024-07-17T08:47:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 0x7f08b414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b415a5a0 0x7f08b414c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:08.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08b414fd60 con 0x7f08b406eb90 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08b406f8e0 con 0x7f08b40acab0 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08ba800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08b414fee0 con 0x7f08b415a5a0 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38842/0 (socket says 172.21.15.44:38842) 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 -- 172.21.15.44:0/3975728856 learned_addr learned my addr 172.21.15.44:0/3975728856 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08abe00640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 0x7f08b414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 -- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b415a5a0 msgr2=0x7f08b414c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b415a5a0 0x7f08b414c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 -- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 msgr2=0x7f08b414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 0x7f08b414c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.389+0000 7f08b3400640 1 -- 172.21.15.44:0/3975728856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b41f5600 con 0x7f08b406eb90 2024-07-17T08:47:08.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08abe00640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 0x7f08b414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08b3400640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 crypto rx=0x7f08a40013c0 tx=0x7f08a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4016020 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a400a950 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b41f58e0 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400ada0 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.390+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41f5ea0 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.392+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b40b3280 con 0x7f08b406eb90 2024-07-17T08:47:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.393+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f08a4019020 con 0x7f08b406eb90 2024-07-17T08:47:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.394+0000 7f08ab400640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 0x7f0890079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.394+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f08a4098740 con 0x7f08b406eb90 2024-07-17T08:47:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.395+0000 7f08abe00640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 0x7f0890079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.396+0000 7f08abe00640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 0x7f0890079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f089c007fd0 tx=0x7f089c009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.399+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f08a4020030 con 0x7f08b406eb90 2024-07-17T08:47:08.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:08.642+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f08b40b4230 con 0x7f08b406eb90 2024-07-17T08:47:09.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.161+0000 7f08ab400640 1 -- 172.21.15.44:0/3975728856 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v218) ==== 122+0+0 (secure 0 0 0) 0x7f08a40616f0 con 0x7f08b406eb90 2024-07-17T08:47:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 msgr2=0x7f0890079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 0x7f0890079190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f089c007fd0 tx=0x7f089c009040 comp rx=0 tx=0).stop 2024-07-17T08:47:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 msgr2=0x7f08b415c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 crypto rx=0x7f08a40013c0 tx=0x7f08a400cb90 comp rx=0 tx=0).stop 2024-07-17T08:47:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 shutdown_connections 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f0890076cd0 0x7f0890079190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08b415a5a0 0x7f08b414c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08b40acab0 0x7f08b414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 --2- 172.21.15.44:0/3975728856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08b406eb90 0x7f08b415c0a0 unknown :-1 s=CLOSED pgs=3458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.165+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 >> 172.21.15.44:0/3975728856 conn(0x7f08b40b0eb0 msgr2=0x7f08b4153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.166+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 shutdown_connections 2024-07-17T08:47:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.166+0000 7f08ba800640 1 -- 172.21.15.44:0/3975728856 wait complete. 2024-07-17T08:47:09.191 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-17T08:47:09.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:47:09.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 -- 172.21.15.44:0/2550960096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 msgr2=0x7f759c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 --2- 172.21.15.44:0/2550960096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f759c003e90 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009e20 tx=0x7f759c005830 comp rx=0 tx=0).stop 2024-07-17T08:47:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75bd800640 1 -- 172.21.15.44:0/2550960096 reap_dead start 2024-07-17T08:47:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 -- 172.21.15.44:0/2550960096 shutdown_connections 2024-07-17T08:47:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 -- 172.21.15.44:0/2550960096 >> 172.21.15.44:0/2550960096 conn(0x7f75b80cfec0 msgr2=0x7f75b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 -- 172.21.15.44:0/2550960096 shutdown_connections 2024-07-17T08:47:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.405+0000 7f75be200640 1 -- 172.21.15.44:0/2550960096 wait complete. 2024-07-17T08:47:09.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.408+0000 7f75be200640 1 Processor -- start 2024-07-17T08:47:09.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.408+0000 7f75be200640 1 -- start start 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b80cb560 0x7f75b80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b815b7d0 0x7f75b815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80b4ec0 con 0x7f759c003a70 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80b9270 con 0x7f75b815b7d0 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75be200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75b80b5040 con 0x7f75b80cb560 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75bce00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50534/0 (socket says 172.21.15.44:50534) 2024-07-17T08:47:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.409+0000 7f75bce00640 1 -- 172.21.15.44:0/4103139161 learned_addr learned my addr 172.21.15.44:0/4103139161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:09.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 -- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b80cb560 msgr2=0x7f75b80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b80cb560 0x7f75b80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 -- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b815b7d0 msgr2=0x7f75b815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b815b7d0 0x7f75b815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 -- 172.21.15.44:0/4103139161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b815c190 con 0x7f759c003a70 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75bce00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d70 tx=0x7f75a80336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8038040 con 0x7f759c003a70 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75a8041070 con 0x7f759c003a70 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8033b40 con 0x7f759c003a70 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.410+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b80ccc90 con 0x7f759c003a70 2024-07-17T08:47:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.411+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b8159720 con 0x7f759c003a70 2024-07-17T08:47:09.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.412+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b806d9b0 con 0x7f759c003a70 2024-07-17T08:47:09.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.413+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f75a8033ce0 con 0x7f759c003a70 2024-07-17T08:47:09.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.415+0000 7f75b4c00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 0x7f75940791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.415+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f75a80b9090 con 0x7f759c003a70 2024-07-17T08:47:09.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.415+0000 7f75b7e00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 0x7f75940791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:09.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.416+0000 7f75b7e00640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 0x7f75940791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75a0001860 tx=0x7f75a00014d0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:09.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.418+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f75a8081fb0 con 0x7f759c003a70 2024-07-17T08:47:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.667+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75b80b5d70 con 0x7f759c003a70 2024-07-17T08:47:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.668+0000 7f75b4c00640 1 -- 172.21.15.44:0/4103139161 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+5247 (secure 0 0 0) 0x7f75a808a1f0 con 0x7f759c003a70 2024-07-17T08:47:09.670 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:09.670 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":218,"btime":"2024-07-17T08:47:09:148469+0000","default_fscid":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4026613862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4026613862},{"type":"v1","addr":"172.21.15.44:6835","nonce":4026613862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":10475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1040890779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1040890779},{"type":"v1","addr":"172.21.15.175:6836","nonce":1040890779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":14859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3392644589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3392644589},{"type":"v1","addr":"172.21.15.44:6834","nonce":3392644589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:47:05.578374+0000","modified":"2024-07-17T08:47:08.644730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14868":{"gid":14868,"name":"b","rank":0,"incarnation":214,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1064157662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1064157662},{"type":"v1","addr":"172.21.15.175:6801","nonce":1064157662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[52],"metadata_pool":51,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14868,"qdb_cluster":[14868]},"id":25}]} 2024-07-17T08:47:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 218 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 msgr2=0x7f75940791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 0x7f75940791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75a0001860 tx=0x7f75a00014d0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 msgr2=0x7f75b80b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d70 tx=0x7f75a80336f0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 shutdown_connections 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7594076d00 0x7f75940791c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b815b7d0 0x7f75b815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b80cb560 0x7f75b80cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 --2- 172.21.15.44:0/4103139161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f759c003a70 0x7f75b80b1910 unknown :-1 s=CLOSED pgs=3461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.671+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 >> 172.21.15.44:0/4103139161 conn(0x7f75b80cfec0 msgr2=0x7f75b80d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.672+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 shutdown_connections 2024-07-17T08:47:09.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.672+0000 7f75be200640 1 -- 172.21.15.44:0/4103139161 wait complete. 2024-07-17T08:47:09.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.904+0000 7fecc7e00640 1 -- 172.21.15.44:0/370986243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0160a80 msgr2=0x7fecc006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.904+0000 7fecc7e00640 1 --2- 172.21.15.44:0/370986243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0160a80 0x7fecc006e470 secure :-1 s=READY pgs=3463 cs=0 l=1 rev1=1 crypto rx=0x7fecb0005fd0 tx=0x7fecb002a5d0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.905+0000 7fecc7e00640 1 -- 172.21.15.44:0/370986243 shutdown_connections 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.905+0000 7fecc7e00640 1 --2- 172.21.15.44:0/370986243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc0160a80 0x7fecc006e470 unknown :-1 s=CLOSED pgs=3463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.905+0000 7fecc7e00640 1 -- 172.21.15.44:0/370986243 >> 172.21.15.44:0/370986243 conn(0x7fecc00b3120 msgr2=0x7fecc00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.905+0000 7fecc7e00640 1 -- 172.21.15.44:0/370986243 shutdown_connections 2024-07-17T08:47:09.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.905+0000 7fecc7e00640 1 -- 172.21.15.44:0/370986243 wait complete. 2024-07-17T08:47:09.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.907+0000 7fecc7e00640 1 Processor -- start 2024-07-17T08:47:09.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.907+0000 7fecc7e00640 1 -- start start 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 0x7fecc014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc0160a80 0x7fecc01f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fecc00b2df0 con 0x7fecc00b3c40 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecc00b2c70 con 0x7fecc00b1c50 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecc00b2f70 con 0x7fecc0160a80 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc4a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50540/0 (socket says 172.21.15.44:50540) 2024-07-17T08:47:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc4a00640 1 -- 172.21.15.44:0/4018998449 learned_addr learned my addr 172.21.15.44:0/4018998449 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:09.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.908+0000 7fecc5400640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 0x7fecc014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 -- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc0160a80 msgr2=0x7fecc01f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc0160a80 0x7fecc01f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 -- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 msgr2=0x7fecc014a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 0x7fecc014a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 -- 172.21.15.44:0/4018998449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc015f280 con 0x7fecc00b3c40 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc5400640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 0x7fecc014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc4a00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 secure :-1 s=READY pgs=3464 cs=0 l=1 rev1=1 crypto rx=0x7fecac00c460 tx=0x7fecac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac016020 con 0x7fecc00b3c40 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecac012960 con 0x7fecc00b3c40 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac012db0 con 0x7fecc00b3c40 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc015f560 con 0x7fecc00b3c40 2024-07-17T08:47:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.909+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc014b0e0 con 0x7fecc00b3c40 2024-07-17T08:47:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.910+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc006ec70 con 0x7fecc00b3c40 2024-07-17T08:47:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.912+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fecac019020 con 0x7fecc00b3c40 2024-07-17T08:47:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.913+0000 7fecbd600640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 0x7feca0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.913+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fecac098630 con 0x7fecc00b3c40 2024-07-17T08:47:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.913+0000 7fecc5400640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 0x7feca0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.915+0000 7fecc5400640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 0x7feca0079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecb0033000 tx=0x7fecb0004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:09.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:09.918+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fecac020030 con 0x7fecc00b3c40 2024-07-17T08:47:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.128+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecc00d1a60 con 0x7fecc00b3c40 2024-07-17T08:47:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.131+0000 7fecbd600640 1 -- 172.21.15.44:0/4018998449 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v379) ==== 74+0+17911 (secure 0 0 0) 0x7fecac0615e0 con 0x7fecc00b3c40 2024-07-17T08:47:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:10.133 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":379,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:47:07.124873+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":52,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":51,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:47:03.554223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":52,"pool_name":"cephfs_data","create_time":"2024-07-17T08:47:04.562957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"379","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.133+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 msgr2=0x7feca0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.133+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 0x7feca0079190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecb0033000 tx=0x7fecb0004730 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 msgr2=0x7fecc015e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 secure :-1 s=READY pgs=3464 cs=0 l=1 rev1=1 crypto rx=0x7fecac00c460 tx=0x7fecac00c920 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 shutdown_connections 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7feca0076cd0 0x7feca0079190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc0160a80 0x7fecc01f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc00b3c40 0x7fecc015e970 unknown :-1 s=CLOSED pgs=3464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 --2- 172.21.15.44:0/4018998449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecc00b1c50 0x7fecc014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 >> 172.21.15.44:0/4018998449 conn(0x7fecc00b3120 msgr2=0x7fecc00b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 shutdown_connections 2024-07-17T08:47:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.134+0000 7fecc7e00640 1 -- 172.21.15.44:0/4018998449 wait complete. 2024-07-17T08:47:10.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:47:10.156 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 -- 172.21.15.44:0/1666874964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc0cfa90 msgr2=0x7f7acc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 --2- 172.21.15.44:0/1666874964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc0cfa90 0x7f7acc06e060 secure :-1 s=READY pgs=3465 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005fd0 tx=0x7f7ab802adc0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 -- 172.21.15.44:0/1666874964 shutdown_connections 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 --2- 172.21.15.44:0/1666874964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc09b6e0 0x7f7acc159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 --2- 172.21.15.44:0/1666874964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc0cfa90 0x7f7acc06e060 unknown :-1 s=CLOSED pgs=3465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 -- 172.21.15.44:0/1666874964 >> 172.21.15.44:0/1666874964 conn(0x7f7acc0b0e80 msgr2=0x7f7acc0b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 -- 172.21.15.44:0/1666874964 shutdown_connections 2024-07-17T08:47:10.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.365+0000 7f7ad1800640 1 -- 172.21.15.44:0/1666874964 wait complete. 2024-07-17T08:47:10.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.367+0000 7f7ad1800640 1 Processor -- start 2024-07-17T08:47:10.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.367+0000 7f7ad1800640 1 -- start start 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 0x7f7acc158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 0x7f7acc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0b2120 con 0x7f7acc09b300 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0b1fa0 con 0x7f7acc09b6e0 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7ad1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc0b22a0 con 0x7f7acc0cfa90 2024-07-17T08:47:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50562/0 (socket says 172.21.15.44:50562) 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.368+0000 7f7acbe00640 1 -- 172.21.15.44:0/2072833791 learned_addr learned my addr 172.21.15.44:0/2072833791 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acb400640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 0x7f7acc158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ad0e00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 0x7f7acc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 -- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 msgr2=0x7f7acc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 0x7f7acc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 -- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 msgr2=0x7f7acc158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 0x7f7acc158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 -- 172.21.15.44:0/2072833791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc1f3ee0 con 0x7f7acc09b300 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acb400640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 0x7f7acc158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ad0e00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 0x7f7acc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7acbe00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c410 tx=0x7f7abc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc017070 con 0x7f7acc09b300 2024-07-17T08:47:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7abc0035f0 con 0x7f7acc09b300 2024-07-17T08:47:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc1f4180 con 0x7f7acc09b300 2024-07-17T08:47:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.369+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc003a80 con 0x7f7acc09b300 2024-07-17T08:47:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.370+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1f4740 con 0x7f7acc09b300 2024-07-17T08:47:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.371+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc159800 con 0x7f7acc09b300 2024-07-17T08:47:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.372+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7abc01a070 con 0x7f7acc09b300 2024-07-17T08:47:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.374+0000 7f7ac3e00640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 0x7f7aac079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.374+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f7abc097df0 con 0x7f7acc09b300 2024-07-17T08:47:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.374+0000 7f7acb400640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 0x7f7aac079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.376+0000 7f7acb400640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 0x7f7aac079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8034000 tx=0x7f7ab8004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:10.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.378+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7abc060d10 con 0x7f7acc09b300 2024-07-17T08:47:10.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.585+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7acc09bea0 con 0x7f7aac076cd0 2024-07-17T08:47:10.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.601+0000 7f7ac3e00640 1 -- 172.21.15.44:0/2072833791 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7f7acc09bea0 con 0x7f7aac076cd0 2024-07-17T08:47:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.606+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 msgr2=0x7f7aac079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.606+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 0x7f7aac079190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8034000 tx=0x7f7ab8004730 comp rx=0 tx=0).stop 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.606+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 msgr2=0x7f7acc14fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.606+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 secure :-1 s=READY pgs=3466 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00c410 tx=0x7f7abc00c8d0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 shutdown_connections 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7aac076cd0 0x7f7aac079190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7acc0cfa90 0x7f7acc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7acc09b6e0 0x7f7acc158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 --2- 172.21.15.44:0/2072833791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7acc09b300 0x7f7acc14fcb0 unknown :-1 s=CLOSED pgs=3466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 >> 172.21.15.44:0/2072833791 conn(0x7f7acc0b0e80 msgr2=0x7f7acc0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 shutdown_connections 2024-07-17T08:47:10.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.607+0000 7f7ad1800640 1 -- 172.21.15.44:0/2072833791 wait complete. 2024-07-17T08:47:10.627 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:47:09.884478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":633,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4718,"ondisk_log_size":4718,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546568,"kb_used_data":38908,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428152,"statfs":{"total":773094113280,"available":772534427648,"internally_reserved":0,"allocated":39841792,"data_stored":22541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.803254"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274062+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:47:07.274062+0000","last_peered":"2024-07-17T08:47:07.274062+0000","last_clean":"2024-07-17T08:47:07.274062+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:47:07.274062+0000","last_undegraded":"2024-07-17T08:47:07.274062+0000","last_fullsized":"2024-07-17T08:47:07.274062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221603+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:47:07.221603+0000","last_peered":"2024-07-17T08:47:07.221603+0000","last_clean":"2024-07-17T08:47:07.221603+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:47:07.221603+0000","last_undegraded":"2024-07-17T08:47:07.221603+0000","last_fullsized":"2024-07-17T08:47:07.221603+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:43:50.699357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.276429+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.276429+0000","last_peered":"2024-07-17T08:47:07.276429+0000","last_clean":"2024-07-17T08:47:07.276429+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.276429+0000","last_undegraded":"2024-07-17T08:47:07.276429+0000","last_fullsized":"2024-07-17T08:47:07.276429+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143704+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.143704+0000","last_peered":"2024-07-17T08:47:07.143704+0000","last_clean":"2024-07-17T08:47:07.143704+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.143704+0000","last_undegraded":"2024-07-17T08:47:07.143704+0000","last_fullsized":"2024-07-17T08:47:07.143704+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221716+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.221716+0000","last_peered":"2024-07-17T08:47:07.221716+0000","last_clean":"2024-07-17T08:47:07.221716+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.221716+0000","last_undegraded":"2024-07-17T08:47:07.221716+0000","last_fullsized":"2024-07-17T08:47:07.221716+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:03.150509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273776+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.273776+0000","last_peered":"2024-07-17T08:47:07.273776+0000","last_clean":"2024-07-17T08:47:07.273776+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.273776+0000","last_undegraded":"2024-07-17T08:47:07.273776+0000","last_fullsized":"2024-07-17T08:47:07.273776+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221537+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.221537+0000","last_peered":"2024-07-17T08:47:07.221537+0000","last_clean":"2024-07-17T08:47:07.221537+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.221537+0000","last_undegraded":"2024-07-17T08:47:07.221537+0000","last_fullsized":"2024-07-17T08:47:07.221537+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:08.644619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273038+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.273038+0000","last_peered":"2024-07-17T08:47:07.273038+0000","last_clean":"2024-07-17T08:47:07.273038+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.273038+0000","last_undegraded":"2024-07-17T08:47:07.273038+0000","last_fullsized":"2024-07-17T08:47:07.273038+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271623+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.271623+0000","last_peered":"2024-07-17T08:47:07.271623+0000","last_clean":"2024-07-17T08:47:07.271623+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.271623+0000","last_undegraded":"2024-07-17T08:47:07.271623+0000","last_fullsized":"2024-07-17T08:47:07.271623+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221733+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.221733+0000","last_peered":"2024-07-17T08:47:07.221733+0000","last_clean":"2024-07-17T08:47:07.221733+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.221733+0000","last_undegraded":"2024-07-17T08:47:07.221733+0000","last_fullsized":"2024-07-17T08:47:07.221733+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:19.898818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274204+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.274204+0000","last_peered":"2024-07-17T08:47:07.274204+0000","last_clean":"2024-07-17T08:47:07.274204+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.274204+0000","last_undegraded":"2024-07-17T08:47:07.274204+0000","last_fullsized":"2024-07-17T08:47:07.274204+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:29.623868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141167+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.141167+0000","last_peered":"2024-07-17T08:47:07.141167+0000","last_clean":"2024-07-17T08:47:07.141167+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.141167+0000","last_undegraded":"2024-07-17T08:47:07.141167+0000","last_fullsized":"2024-07-17T08:47:07.141167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144133+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.144133+0000","last_peered":"2024-07-17T08:47:07.144133+0000","last_clean":"2024-07-17T08:47:07.144133+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.144133+0000","last_undegraded":"2024-07-17T08:47:07.144133+0000","last_fullsized":"2024-07-17T08:47:07.144133+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221396+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:47:07.221396+0000","last_peered":"2024-07-17T08:47:07.221396+0000","last_clean":"2024-07-17T08:47:07.221396+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:47:07.221396+0000","last_undegraded":"2024-07-17T08:47:07.221396+0000","last_fullsized":"2024-07-17T08:47:07.221396+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:00:09.543515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140850+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.140850+0000","last_peered":"2024-07-17T08:47:07.140850+0000","last_clean":"2024-07-17T08:47:07.140850+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.140850+0000","last_undegraded":"2024-07-17T08:47:07.140850+0000","last_fullsized":"2024-07-17T08:47:07.140850+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140398+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.140398+0000","last_peered":"2024-07-17T08:47:07.140398+0000","last_clean":"2024-07-17T08:47:07.140398+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.140398+0000","last_undegraded":"2024-07-17T08:47:07.140398+0000","last_fullsized":"2024-07-17T08:47:07.140398+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143331+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.143331+0000","last_peered":"2024-07-17T08:47:07.143331+0000","last_clean":"2024-07-17T08:47:07.143331+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.143331+0000","last_undegraded":"2024-07-17T08:47:07.143331+0000","last_fullsized":"2024-07-17T08:47:07.143331+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144579+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.144579+0000","last_peered":"2024-07-17T08:47:07.144579+0000","last_clean":"2024-07-17T08:47:07.144579+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.144579+0000","last_undegraded":"2024-07-17T08:47:07.144579+0000","last_fullsized":"2024-07-17T08:47:07.144579+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272686+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.272686+0000","last_peered":"2024-07-17T08:47:07.272686+0000","last_clean":"2024-07-17T08:47:07.272686+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.272686+0000","last_undegraded":"2024-07-17T08:47:07.272686+0000","last_fullsized":"2024-07-17T08:47:07.272686+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273638+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.273638+0000","last_peered":"2024-07-17T08:47:07.273638+0000","last_clean":"2024-07-17T08:47:07.273638+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.273638+0000","last_undegraded":"2024-07-17T08:47:07.273638+0000","last_fullsized":"2024-07-17T08:47:07.273638+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273485+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.273485+0000","last_peered":"2024-07-17T08:47:07.273485+0000","last_clean":"2024-07-17T08:47:07.273485+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.273485+0000","last_undegraded":"2024-07-17T08:47:07.273485+0000","last_fullsized":"2024-07-17T08:47:07.273485+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273535+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.273535+0000","last_peered":"2024-07-17T08:47:07.273535+0000","last_clean":"2024-07-17T08:47:07.273535+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.273535+0000","last_undegraded":"2024-07-17T08:47:07.273535+0000","last_fullsized":"2024-07-17T08:47:07.273535+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273438+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.273438+0000","last_peered":"2024-07-17T08:47:07.273438+0000","last_clean":"2024-07-17T08:47:07.273438+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.273438+0000","last_undegraded":"2024-07-17T08:47:07.273438+0000","last_fullsized":"2024-07-17T08:47:07.273438+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274768+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.274768+0000","last_peered":"2024-07-17T08:47:07.274768+0000","last_clean":"2024-07-17T08:47:07.274768+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.274768+0000","last_undegraded":"2024-07-17T08:47:07.274768+0000","last_fullsized":"2024-07-17T08:47:07.274768+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274314+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.274314+0000","last_peered":"2024-07-17T08:47:07.274314+0000","last_clean":"2024-07-17T08:47:07.274314+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.274314+0000","last_undegraded":"2024-07-17T08:47:07.274314+0000","last_fullsized":"2024-07-17T08:47:07.274314+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221117+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.221117+0000","last_peered":"2024-07-17T08:47:07.221117+0000","last_clean":"2024-07-17T08:47:07.221117+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.221117+0000","last_undegraded":"2024-07-17T08:47:07.221117+0000","last_fullsized":"2024-07-17T08:47:07.221117+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:53.930624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273641+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.273641+0000","last_peered":"2024-07-17T08:47:07.273641+0000","last_clean":"2024-07-17T08:47:07.273641+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.273641+0000","last_undegraded":"2024-07-17T08:47:07.273641+0000","last_fullsized":"2024-07-17T08:47:07.273641+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221602+0000","last_change":"2024-07-17T08:47:05.129655+0000","last_active":"2024-07-17T08:47:07.221602+0000","last_peered":"2024-07-17T08:47:07.221602+0000","last_clean":"2024-07-17T08:47:07.221602+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:47:07.221602+0000","last_undegraded":"2024-07-17T08:47:07.221602+0000","last_fullsized":"2024-07-17T08:47:07.221602+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:09:03.283870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272723+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.272723+0000","last_peered":"2024-07-17T08:47:07.272723+0000","last_clean":"2024-07-17T08:47:07.272723+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.272723+0000","last_undegraded":"2024-07-17T08:47:07.272723+0000","last_fullsized":"2024-07-17T08:47:07.272723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:17.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273634+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.273634+0000","last_peered":"2024-07-17T08:47:07.273634+0000","last_clean":"2024-07-17T08:47:07.273634+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.273634+0000","last_undegraded":"2024-07-17T08:47:07.273634+0000","last_fullsized":"2024-07-17T08:47:07.273634+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272935+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.272935+0000","last_peered":"2024-07-17T08:47:07.272935+0000","last_clean":"2024-07-17T08:47:07.272935+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.272935+0000","last_undegraded":"2024-07-17T08:47:07.272935+0000","last_fullsized":"2024-07-17T08:47:07.272935+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221435+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.221435+0000","last_peered":"2024-07-17T08:47:07.221435+0000","last_clean":"2024-07-17T08:47:07.221435+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.221435+0000","last_undegraded":"2024-07-17T08:47:07.221435+0000","last_fullsized":"2024-07-17T08:47:07.221435+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:50.145513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221353+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.221353+0000","last_peered":"2024-07-17T08:47:07.221353+0000","last_clean":"2024-07-17T08:47:07.221353+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.221353+0000","last_undegraded":"2024-07-17T08:47:07.221353+0000","last_fullsized":"2024-07-17T08:47:07.221353+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:50.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143822+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.143822+0000","last_peered":"2024-07-17T08:47:07.143822+0000","last_clean":"2024-07-17T08:47:07.143822+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.143822+0000","last_undegraded":"2024-07-17T08:47:07.143822+0000","last_fullsized":"2024-07-17T08:47:07.143822+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271594+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.271594+0000","last_peered":"2024-07-17T08:47:07.271594+0000","last_clean":"2024-07-17T08:47:07.271594+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.271594+0000","last_undegraded":"2024-07-17T08:47:07.271594+0000","last_fullsized":"2024-07-17T08:47:07.271594+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273871+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.273871+0000","last_peered":"2024-07-17T08:47:07.273871+0000","last_clean":"2024-07-17T08:47:07.273871+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.273871+0000","last_undegraded":"2024-07-17T08:47:07.273871+0000","last_fullsized":"2024-07-17T08:47:07.273871+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272214+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.272214+0000","last_peered":"2024-07-17T08:47:07.272214+0000","last_clean":"2024-07-17T08:47:07.272214+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.272214+0000","last_undegraded":"2024-07-17T08:47:07.272214+0000","last_fullsized":"2024-07-17T08:47:07.272214+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"378'6","reported_seq":20,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141395+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:47:07.141395+0000","last_peered":"2024-07-17T08:47:07.141395+0000","last_clean":"2024-07-17T08:47:07.141395+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:47:07.141395+0000","last_undegraded":"2024-07-17T08:47:07.141395+0000","last_fullsized":"2024-07-17T08:47:07.141395+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221401+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.221401+0000","last_peered":"2024-07-17T08:47:07.221401+0000","last_clean":"2024-07-17T08:47:07.221401+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.221401+0000","last_undegraded":"2024-07-17T08:47:07.221401+0000","last_fullsized":"2024-07-17T08:47:07.221401+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:57.532859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221261+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:47:07.221261+0000","last_peered":"2024-07-17T08:47:07.221261+0000","last_clean":"2024-07-17T08:47:07.221261+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:47:07.221261+0000","last_undegraded":"2024-07-17T08:47:07.221261+0000","last_fullsized":"2024-07-17T08:47:07.221261+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:52.365125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274195+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:47:07.274195+0000","last_peered":"2024-07-17T08:47:07.274195+0000","last_clean":"2024-07-17T08:47:07.274195+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:47:07.274195+0000","last_undegraded":"2024-07-17T08:47:07.274195+0000","last_fullsized":"2024-07-17T08:47:07.274195+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273281+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.273281+0000","last_peered":"2024-07-17T08:47:07.273281+0000","last_clean":"2024-07-17T08:47:07.273281+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.273281+0000","last_undegraded":"2024-07-17T08:47:07.273281+0000","last_fullsized":"2024-07-17T08:47:07.273281+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144016+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.144016+0000","last_peered":"2024-07-17T08:47:07.144016+0000","last_clean":"2024-07-17T08:47:07.144016+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.144016+0000","last_undegraded":"2024-07-17T08:47:07.144016+0000","last_fullsized":"2024-07-17T08:47:07.144016+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274519+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.274519+0000","last_peered":"2024-07-17T08:47:07.274519+0000","last_clean":"2024-07-17T08:47:07.274519+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.274519+0000","last_undegraded":"2024-07-17T08:47:07.274519+0000","last_fullsized":"2024-07-17T08:47:07.274519+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141801+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.141801+0000","last_peered":"2024-07-17T08:47:07.141801+0000","last_clean":"2024-07-17T08:47:07.141801+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.141801+0000","last_undegraded":"2024-07-17T08:47:07.141801+0000","last_fullsized":"2024-07-17T08:47:07.141801+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275409+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.275409+0000","last_peered":"2024-07-17T08:47:07.275409+0000","last_clean":"2024-07-17T08:47:07.275409+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.275409+0000","last_undegraded":"2024-07-17T08:47:07.275409+0000","last_fullsized":"2024-07-17T08:47:07.275409+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273190+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.273190+0000","last_peered":"2024-07-17T08:47:07.273190+0000","last_clean":"2024-07-17T08:47:07.273190+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.273190+0000","last_undegraded":"2024-07-17T08:47:07.273190+0000","last_fullsized":"2024-07-17T08:47:07.273190+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141198+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.141198+0000","last_peered":"2024-07-17T08:47:07.141198+0000","last_clean":"2024-07-17T08:47:07.141198+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.141198+0000","last_undegraded":"2024-07-17T08:47:07.141198+0000","last_fullsized":"2024-07-17T08:47:07.141198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221468+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:47:07.221468+0000","last_peered":"2024-07-17T08:47:07.221468+0000","last_clean":"2024-07-17T08:47:07.221468+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:47:07.221468+0000","last_undegraded":"2024-07-17T08:47:07.221468+0000","last_fullsized":"2024-07-17T08:47:07.221468+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:37.618880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273673+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.273673+0000","last_peered":"2024-07-17T08:47:07.273673+0000","last_clean":"2024-07-17T08:47:07.273673+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.273673+0000","last_undegraded":"2024-07-17T08:47:07.273673+0000","last_fullsized":"2024-07-17T08:47:07.273673+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272804+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.272804+0000","last_peered":"2024-07-17T08:47:07.272804+0000","last_clean":"2024-07-17T08:47:07.272804+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.272804+0000","last_undegraded":"2024-07-17T08:47:07.272804+0000","last_fullsized":"2024-07-17T08:47:07.272804+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275184+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.275184+0000","last_peered":"2024-07-17T08:47:07.275184+0000","last_clean":"2024-07-17T08:47:07.275184+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.275184+0000","last_undegraded":"2024-07-17T08:47:07.275184+0000","last_fullsized":"2024-07-17T08:47:07.275184+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142040+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.142040+0000","last_peered":"2024-07-17T08:47:07.142040+0000","last_clean":"2024-07-17T08:47:07.142040+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.142040+0000","last_undegraded":"2024-07-17T08:47:07.142040+0000","last_fullsized":"2024-07-17T08:47:07.142040+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144992+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.144992+0000","last_peered":"2024-07-17T08:47:07.144992+0000","last_clean":"2024-07-17T08:47:07.144992+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.144992+0000","last_undegraded":"2024-07-17T08:47:07.144992+0000","last_fullsized":"2024-07-17T08:47:07.144992+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272363+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.272363+0000","last_peered":"2024-07-17T08:47:07.272363+0000","last_clean":"2024-07-17T08:47:07.272363+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.272363+0000","last_undegraded":"2024-07-17T08:47:07.272363+0000","last_fullsized":"2024-07-17T08:47:07.272363+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143598+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.143598+0000","last_peered":"2024-07-17T08:47:07.143598+0000","last_clean":"2024-07-17T08:47:07.143598+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.143598+0000","last_undegraded":"2024-07-17T08:47:07.143598+0000","last_fullsized":"2024-07-17T08:47:07.143598+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275708+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.275708+0000","last_peered":"2024-07-17T08:47:07.275708+0000","last_clean":"2024-07-17T08:47:07.275708+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.275708+0000","last_undegraded":"2024-07-17T08:47:07.275708+0000","last_fullsized":"2024-07-17T08:47:07.275708+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140726+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.140726+0000","last_peered":"2024-07-17T08:47:07.140726+0000","last_clean":"2024-07-17T08:47:07.140726+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.140726+0000","last_undegraded":"2024-07-17T08:47:07.140726+0000","last_fullsized":"2024-07-17T08:47:07.140726+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275327+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:47:07.275327+0000","last_peered":"2024-07-17T08:47:07.275327+0000","last_clean":"2024-07-17T08:47:07.275327+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:47:07.275327+0000","last_undegraded":"2024-07-17T08:47:07.275327+0000","last_fullsized":"2024-07-17T08:47:07.275327+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272277+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:47:07.272277+0000","last_peered":"2024-07-17T08:47:07.272277+0000","last_clean":"2024-07-17T08:47:07.272277+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:47:07.272277+0000","last_undegraded":"2024-07-17T08:47:07.272277+0000","last_fullsized":"2024-07-17T08:47:07.272277+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141849+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.141849+0000","last_peered":"2024-07-17T08:47:07.141849+0000","last_clean":"2024-07-17T08:47:07.141849+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.141849+0000","last_undegraded":"2024-07-17T08:47:07.141849+0000","last_fullsized":"2024-07-17T08:47:07.141849+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275827+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.275827+0000","last_peered":"2024-07-17T08:47:07.275827+0000","last_clean":"2024-07-17T08:47:07.275827+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.275827+0000","last_undegraded":"2024-07-17T08:47:07.275827+0000","last_fullsized":"2024-07-17T08:47:07.275827+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221217+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:47:07.221217+0000","last_peered":"2024-07-17T08:47:07.221217+0000","last_clean":"2024-07-17T08:47:07.221217+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:47:07.221217+0000","last_undegraded":"2024-07-17T08:47:07.221217+0000","last_fullsized":"2024-07-17T08:47:07.221217+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:28.605364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142847+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.142847+0000","last_peered":"2024-07-17T08:47:07.142847+0000","last_clean":"2024-07-17T08:47:07.142847+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.142847+0000","last_undegraded":"2024-07-17T08:47:07.142847+0000","last_fullsized":"2024-07-17T08:47:07.142847+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141536+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.141536+0000","last_peered":"2024-07-17T08:47:07.141536+0000","last_clean":"2024-07-17T08:47:07.141536+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.141536+0000","last_undegraded":"2024-07-17T08:47:07.141536+0000","last_fullsized":"2024-07-17T08:47:07.141536+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141587+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.141587+0000","last_peered":"2024-07-17T08:47:07.141587+0000","last_clean":"2024-07-17T08:47:07.141587+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.141587+0000","last_undegraded":"2024-07-17T08:47:07.141587+0000","last_fullsized":"2024-07-17T08:47:07.141587+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141133+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.141133+0000","last_peered":"2024-07-17T08:47:07.141133+0000","last_clean":"2024-07-17T08:47:07.141133+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.141133+0000","last_undegraded":"2024-07-17T08:47:07.141133+0000","last_fullsized":"2024-07-17T08:47:07.141133+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273486+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.273486+0000","last_peered":"2024-07-17T08:47:07.273486+0000","last_clean":"2024-07-17T08:47:07.273486+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.273486+0000","last_undegraded":"2024-07-17T08:47:07.273486+0000","last_fullsized":"2024-07-17T08:47:07.273486+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140715+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.140715+0000","last_peered":"2024-07-17T08:47:07.140715+0000","last_clean":"2024-07-17T08:47:07.140715+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.140715+0000","last_undegraded":"2024-07-17T08:47:07.140715+0000","last_fullsized":"2024-07-17T08:47:07.140715+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273611+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.273611+0000","last_peered":"2024-07-17T08:47:07.273611+0000","last_clean":"2024-07-17T08:47:07.273611+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.273611+0000","last_undegraded":"2024-07-17T08:47:07.273611+0000","last_fullsized":"2024-07-17T08:47:07.273611+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274659+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.274659+0000","last_peered":"2024-07-17T08:47:07.274659+0000","last_clean":"2024-07-17T08:47:07.274659+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.274659+0000","last_undegraded":"2024-07-17T08:47:07.274659+0000","last_fullsized":"2024-07-17T08:47:07.274659+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6211,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272213+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272213+0000","last_peered":"2024-07-17T08:47:07.272213+0000","last_clean":"2024-07-17T08:47:07.272213+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272213+0000","last_undegraded":"2024-07-17T08:47:07.272213+0000","last_fullsized":"2024-07-17T08:47:07.272213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543472,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80888,"kb_used_data":5288,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290952,"statfs":{"total":96636764160,"available":96553934848,"internally_reserved":0,"allocated":5414912,"data_stored":3254053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76160,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295680,"statfs":{"total":96636764160,"available":96558776320,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543473,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80332,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291508,"statfs":{"total":96636764160,"available":96554504192,"internally_reserved":0,"allocated":4845568,"data_stored":2672097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543471,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510768,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58116,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313724,"statfs":{"total":96636764160,"available":96577253376,"internally_reserved":0,"allocated":4841472,"data_stored":2672027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:47:10.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 -- 172.21.15.44:0/3320131786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4178003a80 msgr2=0x7f4178003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 --2- 172.21.15.44:0/3320131786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4178003a80 0x7f4178003ea0 secure :-1 s=READY pgs=3468 cs=0 l=1 rev1=1 crypto rx=0x7f418400c560 tx=0x7f4178005830 comp rx=0 tx=0).stop 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4193400640 1 -- 172.21.15.44:0/3320131786 reap_dead start 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 -- 172.21.15.44:0/3320131786 shutdown_connections 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 -- 172.21.15.44:0/3320131786 >> 172.21.15.44:0/3320131786 conn(0x7f41940b0fb0 msgr2=0x7f41940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 -- 172.21.15.44:0/3320131786 shutdown_connections 2024-07-17T08:47:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.884+0000 7f4199600640 1 -- 172.21.15.44:0/3320131786 wait complete. 2024-07-17T08:47:10.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.886+0000 7f4199600640 1 Processor -- start 2024-07-17T08:47:10.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.886+0000 7f4199600640 1 -- start start 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178003a80 0x7f419414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f419414dba0 0x7f41940b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f41940ac7e0 con 0x7f41940b3940 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41940ac660 con 0x7f4178003a80 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4199600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41940ac960 con 0x7f419414dba0 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4193400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4193400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50572/0 (socket says 172.21.15.44:50572) 2024-07-17T08:47:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4193400640 1 -- 172.21.15.44:0/3741304683 learned_addr learned my addr 172.21.15.44:0/3741304683 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:10.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f4192a00640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178003a80 0x7f419414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.887+0000 7f418a000640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f419414dba0 0x7f41940b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 -- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f419414dba0 msgr2=0x7f41940b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f419414dba0 0x7f41940b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 -- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178003a80 msgr2=0x7f419414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178003a80 0x7f419414f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 -- 172.21.15.44:0/3741304683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941f76c0 con 0x7f41940b3940 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4193400640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 secure :-1 s=READY pgs=3469 cs=0 l=1 rev1=1 crypto rx=0x7f4178004780 tx=0x7f418c00e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c018070 con 0x7f41940b3940 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f418c0135f0 con 0x7f41940b3940 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c00f040 con 0x7f41940b3940 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941f79a0 con 0x7f41940b3940 2024-07-17T08:47:10.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.888+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941f7f60 con 0x7f41940b3940 2024-07-17T08:47:10.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.890+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41940cdb20 con 0x7f41940b3940 2024-07-17T08:47:10.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.891+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f418c01b070 con 0x7f41940b3940 2024-07-17T08:47:10.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.892+0000 7f418b400640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 0x7f4170079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:10.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.892+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f418c097770 con 0x7f41940b3940 2024-07-17T08:47:10.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.892+0000 7f4192a00640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 0x7f4170079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:10.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.894+0000 7f4192a00640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 0x7f4170079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f418400c560 tx=0x7f4184001cb0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:10.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:10.896+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f418c050f50 con 0x7f41940b3940 2024-07-17T08:47:11.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.106+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4194072840 con 0x7f4170076cd0 2024-07-17T08:47:11.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.121+0000 7f418b400640 1 -- 172.21.15.44:0/3741304683 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7f4194072840 con 0x7f4170076cd0 2024-07-17T08:47:11.125 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:11.128 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:47:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 msgr2=0x7f4170079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 0x7f4170079190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f418400c560 tx=0x7f4184001cb0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 msgr2=0x7f4194071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 secure :-1 s=READY pgs=3469 cs=0 l=1 rev1=1 crypto rx=0x7f4178004780 tx=0x7f418c00e680 comp rx=0 tx=0).stop 2024-07-17T08:47:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 shutdown_connections 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4170076cd0 0x7f4170079190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41940b3940 0x7f4194071620 unknown :-1 s=CLOSED pgs=3469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f419414dba0 0x7f41940b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 --2- 172.21.15.44:0/3741304683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4178003a80 0x7f419414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.128+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 >> 172.21.15.44:0/3741304683 conn(0x7f41940b0fb0 msgr2=0x7f41940aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.129+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 shutdown_connections 2024-07-17T08:47:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.129+0000 7f4199600640 1 -- 172.21.15.44:0/3741304683 wait complete. 2024-07-17T08:47:11.150 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-07-17T08:47:09.884478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":633,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4718,"ondisk_log_size":4718,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546568,"kb_used_data":38908,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428152,"statfs":{"total":773094113280,"available":772534427648,"internally_reserved":0,"allocated":39841792,"data_stored":22541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.803254"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274062+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:47:07.274062+0000","last_peered":"2024-07-17T08:47:07.274062+0000","last_clean":"2024-07-17T08:47:07.274062+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:47:07.274062+0000","last_undegraded":"2024-07-17T08:47:07.274062+0000","last_fullsized":"2024-07-17T08:47:07.274062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221603+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:47:07.221603+0000","last_peered":"2024-07-17T08:47:07.221603+0000","last_clean":"2024-07-17T08:47:07.221603+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:47:07.221603+0000","last_undegraded":"2024-07-17T08:47:07.221603+0000","last_fullsized":"2024-07-17T08:47:07.221603+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:43:50.699357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.276429+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.276429+0000","last_peered":"2024-07-17T08:47:07.276429+0000","last_clean":"2024-07-17T08:47:07.276429+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.276429+0000","last_undegraded":"2024-07-17T08:47:07.276429+0000","last_fullsized":"2024-07-17T08:47:07.276429+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143704+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.143704+0000","last_peered":"2024-07-17T08:47:07.143704+0000","last_clean":"2024-07-17T08:47:07.143704+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.143704+0000","last_undegraded":"2024-07-17T08:47:07.143704+0000","last_fullsized":"2024-07-17T08:47:07.143704+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221716+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.221716+0000","last_peered":"2024-07-17T08:47:07.221716+0000","last_clean":"2024-07-17T08:47:07.221716+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.221716+0000","last_undegraded":"2024-07-17T08:47:07.221716+0000","last_fullsized":"2024-07-17T08:47:07.221716+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:03.150509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273776+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.273776+0000","last_peered":"2024-07-17T08:47:07.273776+0000","last_clean":"2024-07-17T08:47:07.273776+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.273776+0000","last_undegraded":"2024-07-17T08:47:07.273776+0000","last_fullsized":"2024-07-17T08:47:07.273776+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221537+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.221537+0000","last_peered":"2024-07-17T08:47:07.221537+0000","last_clean":"2024-07-17T08:47:07.221537+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.221537+0000","last_undegraded":"2024-07-17T08:47:07.221537+0000","last_fullsized":"2024-07-17T08:47:07.221537+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:08.644619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273038+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.273038+0000","last_peered":"2024-07-17T08:47:07.273038+0000","last_clean":"2024-07-17T08:47:07.273038+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.273038+0000","last_undegraded":"2024-07-17T08:47:07.273038+0000","last_fullsized":"2024-07-17T08:47:07.273038+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271623+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.271623+0000","last_peered":"2024-07-17T08:47:07.271623+0000","last_clean":"2024-07-17T08:47:07.271623+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.271623+0000","last_undegraded":"2024-07-17T08:47:07.271623+0000","last_fullsized":"2024-07-17T08:47:07.271623+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221733+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.221733+0000","last_peered":"2024-07-17T08:47:07.221733+0000","last_clean":"2024-07-17T08:47:07.221733+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.221733+0000","last_undegraded":"2024-07-17T08:47:07.221733+0000","last_fullsized":"2024-07-17T08:47:07.221733+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:19.898818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274204+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.274204+0000","last_peered":"2024-07-17T08:47:07.274204+0000","last_clean":"2024-07-17T08:47:07.274204+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.274204+0000","last_undegraded":"2024-07-17T08:47:07.274204+0000","last_fullsized":"2024-07-17T08:47:07.274204+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:29.623868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141167+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.141167+0000","last_peered":"2024-07-17T08:47:07.141167+0000","last_clean":"2024-07-17T08:47:07.141167+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.141167+0000","last_undegraded":"2024-07-17T08:47:07.141167+0000","last_fullsized":"2024-07-17T08:47:07.141167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144133+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.144133+0000","last_peered":"2024-07-17T08:47:07.144133+0000","last_clean":"2024-07-17T08:47:07.144133+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.144133+0000","last_undegraded":"2024-07-17T08:47:07.144133+0000","last_fullsized":"2024-07-17T08:47:07.144133+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221396+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:47:07.221396+0000","last_peered":"2024-07-17T08:47:07.221396+0000","last_clean":"2024-07-17T08:47:07.221396+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:47:07.221396+0000","last_undegraded":"2024-07-17T08:47:07.221396+0000","last_fullsized":"2024-07-17T08:47:07.221396+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:00:09.543515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140850+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.140850+0000","last_peered":"2024-07-17T08:47:07.140850+0000","last_clean":"2024-07-17T08:47:07.140850+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.140850+0000","last_undegraded":"2024-07-17T08:47:07.140850+0000","last_fullsized":"2024-07-17T08:47:07.140850+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140398+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.140398+0000","last_peered":"2024-07-17T08:47:07.140398+0000","last_clean":"2024-07-17T08:47:07.140398+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.140398+0000","last_undegraded":"2024-07-17T08:47:07.140398+0000","last_fullsized":"2024-07-17T08:47:07.140398+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143331+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.143331+0000","last_peered":"2024-07-17T08:47:07.143331+0000","last_clean":"2024-07-17T08:47:07.143331+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.143331+0000","last_undegraded":"2024-07-17T08:47:07.143331+0000","last_fullsized":"2024-07-17T08:47:07.143331+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144579+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.144579+0000","last_peered":"2024-07-17T08:47:07.144579+0000","last_clean":"2024-07-17T08:47:07.144579+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.144579+0000","last_undegraded":"2024-07-17T08:47:07.144579+0000","last_fullsized":"2024-07-17T08:47:07.144579+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272686+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.272686+0000","last_peered":"2024-07-17T08:47:07.272686+0000","last_clean":"2024-07-17T08:47:07.272686+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.272686+0000","last_undegraded":"2024-07-17T08:47:07.272686+0000","last_fullsized":"2024-07-17T08:47:07.272686+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273638+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.273638+0000","last_peered":"2024-07-17T08:47:07.273638+0000","last_clean":"2024-07-17T08:47:07.273638+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.273638+0000","last_undegraded":"2024-07-17T08:47:07.273638+0000","last_fullsized":"2024-07-17T08:47:07.273638+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273485+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.273485+0000","last_peered":"2024-07-17T08:47:07.273485+0000","last_clean":"2024-07-17T08:47:07.273485+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.273485+0000","last_undegraded":"2024-07-17T08:47:07.273485+0000","last_fullsized":"2024-07-17T08:47:07.273485+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273535+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.273535+0000","last_peered":"2024-07-17T08:47:07.273535+0000","last_clean":"2024-07-17T08:47:07.273535+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.273535+0000","last_undegraded":"2024-07-17T08:47:07.273535+0000","last_fullsized":"2024-07-17T08:47:07.273535+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273438+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.273438+0000","last_peered":"2024-07-17T08:47:07.273438+0000","last_clean":"2024-07-17T08:47:07.273438+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.273438+0000","last_undegraded":"2024-07-17T08:47:07.273438+0000","last_fullsized":"2024-07-17T08:47:07.273438+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274768+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.274768+0000","last_peered":"2024-07-17T08:47:07.274768+0000","last_clean":"2024-07-17T08:47:07.274768+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.274768+0000","last_undegraded":"2024-07-17T08:47:07.274768+0000","last_fullsized":"2024-07-17T08:47:07.274768+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274314+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.274314+0000","last_peered":"2024-07-17T08:47:07.274314+0000","last_clean":"2024-07-17T08:47:07.274314+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.274314+0000","last_undegraded":"2024-07-17T08:47:07.274314+0000","last_fullsized":"2024-07-17T08:47:07.274314+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221117+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.221117+0000","last_peered":"2024-07-17T08:47:07.221117+0000","last_clean":"2024-07-17T08:47:07.221117+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.221117+0000","last_undegraded":"2024-07-17T08:47:07.221117+0000","last_fullsized":"2024-07-17T08:47:07.221117+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:53.930624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273641+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.273641+0000","last_peered":"2024-07-17T08:47:07.273641+0000","last_clean":"2024-07-17T08:47:07.273641+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.273641+0000","last_undegraded":"2024-07-17T08:47:07.273641+0000","last_fullsized":"2024-07-17T08:47:07.273641+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221602+0000","last_change":"2024-07-17T08:47:05.129655+0000","last_active":"2024-07-17T08:47:07.221602+0000","last_peered":"2024-07-17T08:47:07.221602+0000","last_clean":"2024-07-17T08:47:07.221602+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:47:07.221602+0000","last_undegraded":"2024-07-17T08:47:07.221602+0000","last_fullsized":"2024-07-17T08:47:07.221602+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:09:03.283870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272723+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.272723+0000","last_peered":"2024-07-17T08:47:07.272723+0000","last_clean":"2024-07-17T08:47:07.272723+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.272723+0000","last_undegraded":"2024-07-17T08:47:07.272723+0000","last_fullsized":"2024-07-17T08:47:07.272723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:17.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273634+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.273634+0000","last_peered":"2024-07-17T08:47:07.273634+0000","last_clean":"2024-07-17T08:47:07.273634+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.273634+0000","last_undegraded":"2024-07-17T08:47:07.273634+0000","last_fullsized":"2024-07-17T08:47:07.273634+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272935+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.272935+0000","last_peered":"2024-07-17T08:47:07.272935+0000","last_clean":"2024-07-17T08:47:07.272935+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.272935+0000","last_undegraded":"2024-07-17T08:47:07.272935+0000","last_fullsized":"2024-07-17T08:47:07.272935+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221435+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.221435+0000","last_peered":"2024-07-17T08:47:07.221435+0000","last_clean":"2024-07-17T08:47:07.221435+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.221435+0000","last_undegraded":"2024-07-17T08:47:07.221435+0000","last_fullsized":"2024-07-17T08:47:07.221435+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:50.145513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221353+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.221353+0000","last_peered":"2024-07-17T08:47:07.221353+0000","last_clean":"2024-07-17T08:47:07.221353+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.221353+0000","last_undegraded":"2024-07-17T08:47:07.221353+0000","last_fullsized":"2024-07-17T08:47:07.221353+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:50.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143822+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.143822+0000","last_peered":"2024-07-17T08:47:07.143822+0000","last_clean":"2024-07-17T08:47:07.143822+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.143822+0000","last_undegraded":"2024-07-17T08:47:07.143822+0000","last_fullsized":"2024-07-17T08:47:07.143822+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271594+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.271594+0000","last_peered":"2024-07-17T08:47:07.271594+0000","last_clean":"2024-07-17T08:47:07.271594+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.271594+0000","last_undegraded":"2024-07-17T08:47:07.271594+0000","last_fullsized":"2024-07-17T08:47:07.271594+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273871+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.273871+0000","last_peered":"2024-07-17T08:47:07.273871+0000","last_clean":"2024-07-17T08:47:07.273871+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.273871+0000","last_undegraded":"2024-07-17T08:47:07.273871+0000","last_fullsized":"2024-07-17T08:47:07.273871+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272214+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.272214+0000","last_peered":"2024-07-17T08:47:07.272214+0000","last_clean":"2024-07-17T08:47:07.272214+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.272214+0000","last_undegraded":"2024-07-17T08:47:07.272214+0000","last_fullsized":"2024-07-17T08:47:07.272214+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"378'6","reported_seq":20,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141395+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:47:07.141395+0000","last_peered":"2024-07-17T08:47:07.141395+0000","last_clean":"2024-07-17T08:47:07.141395+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:47:07.141395+0000","last_undegraded":"2024-07-17T08:47:07.141395+0000","last_fullsized":"2024-07-17T08:47:07.141395+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221401+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.221401+0000","last_peered":"2024-07-17T08:47:07.221401+0000","last_clean":"2024-07-17T08:47:07.221401+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.221401+0000","last_undegraded":"2024-07-17T08:47:07.221401+0000","last_fullsized":"2024-07-17T08:47:07.221401+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:57.532859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221261+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:47:07.221261+0000","last_peered":"2024-07-17T08:47:07.221261+0000","last_clean":"2024-07-17T08:47:07.221261+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:47:07.221261+0000","last_undegraded":"2024-07-17T08:47:07.221261+0000","last_fullsized":"2024-07-17T08:47:07.221261+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:52.365125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274195+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:47:07.274195+0000","last_peered":"2024-07-17T08:47:07.274195+0000","last_clean":"2024-07-17T08:47:07.274195+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:47:07.274195+0000","last_undegraded":"2024-07-17T08:47:07.274195+0000","last_fullsized":"2024-07-17T08:47:07.274195+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273281+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.273281+0000","last_peered":"2024-07-17T08:47:07.273281+0000","last_clean":"2024-07-17T08:47:07.273281+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.273281+0000","last_undegraded":"2024-07-17T08:47:07.273281+0000","last_fullsized":"2024-07-17T08:47:07.273281+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144016+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.144016+0000","last_peered":"2024-07-17T08:47:07.144016+0000","last_clean":"2024-07-17T08:47:07.144016+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.144016+0000","last_undegraded":"2024-07-17T08:47:07.144016+0000","last_fullsized":"2024-07-17T08:47:07.144016+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274519+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.274519+0000","last_peered":"2024-07-17T08:47:07.274519+0000","last_clean":"2024-07-17T08:47:07.274519+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.274519+0000","last_undegraded":"2024-07-17T08:47:07.274519+0000","last_fullsized":"2024-07-17T08:47:07.274519+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141801+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.141801+0000","last_peered":"2024-07-17T08:47:07.141801+0000","last_clean":"2024-07-17T08:47:07.141801+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.141801+0000","last_undegraded":"2024-07-17T08:47:07.141801+0000","last_fullsized":"2024-07-17T08:47:07.141801+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275409+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.275409+0000","last_peered":"2024-07-17T08:47:07.275409+0000","last_clean":"2024-07-17T08:47:07.275409+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.275409+0000","last_undegraded":"2024-07-17T08:47:07.275409+0000","last_fullsized":"2024-07-17T08:47:07.275409+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273190+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.273190+0000","last_peered":"2024-07-17T08:47:07.273190+0000","last_clean":"2024-07-17T08:47:07.273190+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.273190+0000","last_undegraded":"2024-07-17T08:47:07.273190+0000","last_fullsized":"2024-07-17T08:47:07.273190+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141198+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.141198+0000","last_peered":"2024-07-17T08:47:07.141198+0000","last_clean":"2024-07-17T08:47:07.141198+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.141198+0000","last_undegraded":"2024-07-17T08:47:07.141198+0000","last_fullsized":"2024-07-17T08:47:07.141198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221468+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:47:07.221468+0000","last_peered":"2024-07-17T08:47:07.221468+0000","last_clean":"2024-07-17T08:47:07.221468+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:47:07.221468+0000","last_undegraded":"2024-07-17T08:47:07.221468+0000","last_fullsized":"2024-07-17T08:47:07.221468+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:37.618880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273673+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.273673+0000","last_peered":"2024-07-17T08:47:07.273673+0000","last_clean":"2024-07-17T08:47:07.273673+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.273673+0000","last_undegraded":"2024-07-17T08:47:07.273673+0000","last_fullsized":"2024-07-17T08:47:07.273673+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272804+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.272804+0000","last_peered":"2024-07-17T08:47:07.272804+0000","last_clean":"2024-07-17T08:47:07.272804+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.272804+0000","last_undegraded":"2024-07-17T08:47:07.272804+0000","last_fullsized":"2024-07-17T08:47:07.272804+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275184+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.275184+0000","last_peered":"2024-07-17T08:47:07.275184+0000","last_clean":"2024-07-17T08:47:07.275184+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.275184+0000","last_undegraded":"2024-07-17T08:47:07.275184+0000","last_fullsized":"2024-07-17T08:47:07.275184+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142040+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.142040+0000","last_peered":"2024-07-17T08:47:07.142040+0000","last_clean":"2024-07-17T08:47:07.142040+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.142040+0000","last_undegraded":"2024-07-17T08:47:07.142040+0000","last_fullsized":"2024-07-17T08:47:07.142040+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144992+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.144992+0000","last_peered":"2024-07-17T08:47:07.144992+0000","last_clean":"2024-07-17T08:47:07.144992+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.144992+0000","last_undegraded":"2024-07-17T08:47:07.144992+0000","last_fullsized":"2024-07-17T08:47:07.144992+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272363+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.272363+0000","last_peered":"2024-07-17T08:47:07.272363+0000","last_clean":"2024-07-17T08:47:07.272363+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.272363+0000","last_undegraded":"2024-07-17T08:47:07.272363+0000","last_fullsized":"2024-07-17T08:47:07.272363+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143598+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.143598+0000","last_peered":"2024-07-17T08:47:07.143598+0000","last_clean":"2024-07-17T08:47:07.143598+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.143598+0000","last_undegraded":"2024-07-17T08:47:07.143598+0000","last_fullsized":"2024-07-17T08:47:07.143598+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275708+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.275708+0000","last_peered":"2024-07-17T08:47:07.275708+0000","last_clean":"2024-07-17T08:47:07.275708+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.275708+0000","last_undegraded":"2024-07-17T08:47:07.275708+0000","last_fullsized":"2024-07-17T08:47:07.275708+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140726+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.140726+0000","last_peered":"2024-07-17T08:47:07.140726+0000","last_clean":"2024-07-17T08:47:07.140726+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.140726+0000","last_undegraded":"2024-07-17T08:47:07.140726+0000","last_fullsized":"2024-07-17T08:47:07.140726+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275327+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:47:07.275327+0000","last_peered":"2024-07-17T08:47:07.275327+0000","last_clean":"2024-07-17T08:47:07.275327+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:47:07.275327+0000","last_undegraded":"2024-07-17T08:47:07.275327+0000","last_fullsized":"2024-07-17T08:47:07.275327+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272277+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:47:07.272277+0000","last_peered":"2024-07-17T08:47:07.272277+0000","last_clean":"2024-07-17T08:47:07.272277+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:47:07.272277+0000","last_undegraded":"2024-07-17T08:47:07.272277+0000","last_fullsized":"2024-07-17T08:47:07.272277+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141849+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.141849+0000","last_peered":"2024-07-17T08:47:07.141849+0000","last_clean":"2024-07-17T08:47:07.141849+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.141849+0000","last_undegraded":"2024-07-17T08:47:07.141849+0000","last_fullsized":"2024-07-17T08:47:07.141849+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275827+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.275827+0000","last_peered":"2024-07-17T08:47:07.275827+0000","last_clean":"2024-07-17T08:47:07.275827+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.275827+0000","last_undegraded":"2024-07-17T08:47:07.275827+0000","last_fullsized":"2024-07-17T08:47:07.275827+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221217+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:47:07.221217+0000","last_peered":"2024-07-17T08:47:07.221217+0000","last_clean":"2024-07-17T08:47:07.221217+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:47:07.221217+0000","last_undegraded":"2024-07-17T08:47:07.221217+0000","last_fullsized":"2024-07-17T08:47:07.221217+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:28.605364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142847+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.142847+0000","last_peered":"2024-07-17T08:47:07.142847+0000","last_clean":"2024-07-17T08:47:07.142847+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.142847+0000","last_undegraded":"2024-07-17T08:47:07.142847+0000","last_fullsized":"2024-07-17T08:47:07.142847+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141536+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.141536+0000","last_peered":"2024-07-17T08:47:07.141536+0000","last_clean":"2024-07-17T08:47:07.141536+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.141536+0000","last_undegraded":"2024-07-17T08:47:07.141536+0000","last_fullsized":"2024-07-17T08:47:07.141536+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141587+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.141587+0000","last_peered":"2024-07-17T08:47:07.141587+0000","last_clean":"2024-07-17T08:47:07.141587+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.141587+0000","last_undegraded":"2024-07-17T08:47:07.141587+0000","last_fullsized":"2024-07-17T08:47:07.141587+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141133+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.141133+0000","last_peered":"2024-07-17T08:47:07.141133+0000","last_clean":"2024-07-17T08:47:07.141133+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.141133+0000","last_undegraded":"2024-07-17T08:47:07.141133+0000","last_fullsized":"2024-07-17T08:47:07.141133+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273486+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.273486+0000","last_peered":"2024-07-17T08:47:07.273486+0000","last_clean":"2024-07-17T08:47:07.273486+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.273486+0000","last_undegraded":"2024-07-17T08:47:07.273486+0000","last_fullsized":"2024-07-17T08:47:07.273486+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140715+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.140715+0000","last_peered":"2024-07-17T08:47:07.140715+0000","last_clean":"2024-07-17T08:47:07.140715+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.140715+0000","last_undegraded":"2024-07-17T08:47:07.140715+0000","last_fullsized":"2024-07-17T08:47:07.140715+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273611+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.273611+0000","last_peered":"2024-07-17T08:47:07.273611+0000","last_clean":"2024-07-17T08:47:07.273611+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.273611+0000","last_undegraded":"2024-07-17T08:47:07.273611+0000","last_fullsized":"2024-07-17T08:47:07.273611+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274659+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.274659+0000","last_peered":"2024-07-17T08:47:07.274659+0000","last_clean":"2024-07-17T08:47:07.274659+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.274659+0000","last_undegraded":"2024-07-17T08:47:07.274659+0000","last_fullsized":"2024-07-17T08:47:07.274659+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6211,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272213+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272213+0000","last_peered":"2024-07-17T08:47:07.272213+0000","last_clean":"2024-07-17T08:47:07.272213+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272213+0000","last_undegraded":"2024-07-17T08:47:07.272213+0000","last_fullsized":"2024-07-17T08:47:07.272213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543472,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80888,"kb_used_data":5288,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290952,"statfs":{"total":96636764160,"available":96553934848,"internally_reserved":0,"allocated":5414912,"data_stored":3254053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510768,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76160,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295680,"statfs":{"total":96636764160,"available":96558776320,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543473,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80332,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291508,"statfs":{"total":96636764160,"available":96554504192,"internally_reserved":0,"allocated":4845568,"data_stored":2672097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543471,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510768,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58116,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313724,"statfs":{"total":96636764160,"available":96577253376,"internally_reserved":0,"allocated":4841472,"data_stored":2672027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58692,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313148,"statfs":{"total":96636764160,"available":96576663552,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:47:11.154 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:47:11.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-17T08:47:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 -- 172.21.15.44:0/2348442439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 msgr2=0x7fa19c0b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 --2- 172.21.15.44:0/2348442439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c0b32e0 secure :-1 s=READY pgs=3471 cs=0 l=1 rev1=1 crypto rx=0x7fa19c070830 tx=0x7fa19402aac0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 -- 172.21.15.44:0/2348442439 shutdown_connections 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 --2- 172.21.15.44:0/2348442439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c0b32e0 unknown :-1 s=CLOSED pgs=3471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 --2- 172.21.15.44:0/2348442439 >> v2:172.21.15.175:3300/0 conn(0x7fa19c06f4d0 0x7fa19c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 -- 172.21.15.44:0/2348442439 >> 172.21.15.44:0/2348442439 conn(0x7fa19c0cf0d0 msgr2=0x7fa19c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 -- 172.21.15.44:0/2348442439 shutdown_connections 2024-07-17T08:47:11.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.423+0000 7fa19b200640 1 -- 172.21.15.44:0/2348442439 wait complete. 2024-07-17T08:47:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.425+0000 7fa19b200640 1 Processor -- start 2024-07-17T08:47:11.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.425+0000 7fa19b200640 1 -- start start 2024-07-17T08:47:11.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c06f4d0 0x7fa19c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 0x7fa19c1f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c06daf0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c06d970 con 0x7fa19c1503c0 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.426+0000 7fa19b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa19c06dc70 con 0x7fa19c06f4d0 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50584/0 (socket says 172.21.15.44:50584) 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 -- 172.21.15.44:0/1095793697 learned_addr learned my addr 172.21.15.44:0/1095793697 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199e00640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c06f4d0 0x7fa19c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa19a800640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 0x7fa19c1f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 -- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c06f4d0 msgr2=0x7fa19c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c06f4d0 0x7fa19c073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 -- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 msgr2=0x7fa19c1f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 0x7fa19c1f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa199400640 1 -- 172.21.15.44:0/1095793697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c15be20 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.427+0000 7fa19a800640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 0x7fa19c1f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa199400640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c450 tx=0x7fa18800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880052d0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa188005470 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188012870 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c073ad0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.428+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c074090 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.429+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19c0ce3b0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.430+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fa1880035f0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.431+0000 7fa192000640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 0x7fa17c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.431+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fa188098390 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.432+0000 7fa199e00640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 0x7fa17c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.433+0000 7fa199e00640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 0x7fa17c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005d70 tx=0x7fa18c005d00 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.436+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa19c0ce3b0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.731+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa19c15a450 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.738+0000 7fa192000640 1 -- 172.21.15.44:0/1095793697 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v86) ==== 190+0+147 (secure 0 0 0) 0x7fa1880613e0 con 0x7fa19c0b2eb0 2024-07-17T08:47:11.740 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-07-17T08:47:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4apdmi7YVJBAAnQSSeRQwB6cHGl28IW/UQg== 2024-07-17T08:47:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow" 2024-07-17T08:47:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow r" 2024-07-17T08:47:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-17T08:47:11.741 INFO:teuthology.orchestra.run.smithi044.stderr:updated caps for client.0 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.741+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 msgr2=0x7fa17c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.741+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 0x7fa17c079190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005d70 tx=0x7fa18c005d00 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 msgr2=0x7fa19c15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 secure :-1 s=READY pgs=3472 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c450 tx=0x7fa18800c910 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 shutdown_connections 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa17c076cd0 0x7fa17c079190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c1503c0 0x7fa19c1f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa19c0b2eb0 0x7fa19c15b8e0 unknown :-1 s=CLOSED pgs=3472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 --2- 172.21.15.44:0/1095793697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c06f4d0 0x7fa19c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 >> 172.21.15.44:0/1095793697 conn(0x7fa19c0cf0d0 msgr2=0x7fa19c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:11.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 shutdown_connections 2024-07-17T08:47:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.742+0000 7fa19b200640 1 -- 172.21.15.44:0/1095793697 wait complete. 2024-07-17T08:47:11.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:47:11.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 -- 172.21.15.44:0/1057642664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000cd690 msgr2=0x7fcc000cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 --2- 172.21.15.44:0/1057642664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000cd690 0x7fcc000cdae0 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 crypto rx=0x7fcc0015c950 tx=0x7fcbf802aa10 comp rx=0 tx=0).stop 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 -- 172.21.15.44:0/1057642664 shutdown_connections 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 --2- 172.21.15.44:0/1057642664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000cd690 0x7fcc000cdae0 unknown :-1 s=CLOSED pgs=3473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 --2- 172.21.15.44:0/1057642664 >> v2:172.21.15.175:3300/0 conn(0x7fcc000ac220 0x7fcc000ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 -- 172.21.15.44:0/1057642664 >> 172.21.15.44:0/1057642664 conn(0x7fcc000b0fb0 msgr2=0x7fcc000b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 -- 172.21.15.44:0/1057642664 shutdown_connections 2024-07-17T08:47:11.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.966+0000 7fcc04c00640 1 -- 172.21.15.44:0/1057642664 wait complete. 2024-07-17T08:47:11.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.968+0000 7fcc04c00640 1 Processor -- start 2024-07-17T08:47:11.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.968+0000 7fcc04c00640 1 -- start start 2024-07-17T08:47:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 0x7fcc000b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc001522c0 0x7fcc001f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc000ce020 con 0x7fcc000ac220 2024-07-17T08:47:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc000adf80 con 0x7fcc000cd690 2024-07-17T08:47:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.969+0000 7fcc04c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc000ce1a0 con 0x7fcc001522c0 2024-07-17T08:47:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50598/0 (socket says 172.21.15.44:50598) 2024-07-17T08:47:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 -- 172.21.15.44:0/784478657 learned_addr learned my addr 172.21.15.44:0/784478657 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfd400640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 0x7fcc000b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfe800640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc001522c0 0x7fcc001f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 -- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc001522c0 msgr2=0x7fcc001f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc001522c0 0x7fcc001f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 -- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 msgr2=0x7fcc000b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 0x7fcc000b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 -- 172.21.15.44:0/784478657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc000b3000 con 0x7fcc000ac220 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfd400640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 0x7fcc000b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.970+0000 7fcbfde00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000b3e0 tx=0x7fcbf000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.971+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0015020 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.971+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbf0011940 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.971+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0011d90 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.971+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc00153c00 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.971+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001541c0 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.972+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc000ce6c0 con 0x7fcc000ac220 2024-07-17T08:47:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.973+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fcbf0018020 con 0x7fcc000ac220 2024-07-17T08:47:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.974+0000 7fcbf6000640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 0x7fcbe0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.974+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fcbf0098830 con 0x7fcc000ac220 2024-07-17T08:47:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.975+0000 7fcbfd400640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 0x7fcbe0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.976+0000 7fcbfd400640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 0x7fcbe0079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc001f77d0 tx=0x7fcbec005d90 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:11.979+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fcbf001f020 con 0x7fcc000ac220 2024-07-17T08:47:12.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.234+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc000cdd20 con 0x7fcc000ac220 2024-07-17T08:47:12.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.234+0000 7fcbf6000640 1 -- 172.21.15.44:0/784478657 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+5247 (secure 0 0 0) 0x7fcc000b3000 con 0x7fcc000ac220 2024-07-17T08:47:12.236 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:47:12.236 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":218,"btime":"2024-07-17T08:47:09:148469+0000","default_fscid":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4026613862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4026613862},{"type":"v1","addr":"172.21.15.44:6835","nonce":4026613862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":10475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1040890779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1040890779},{"type":"v1","addr":"172.21.15.175:6836","nonce":1040890779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":14859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3392644589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3392644589},{"type":"v1","addr":"172.21.15.44:6834","nonce":3392644589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:47:05.578374+0000","modified":"2024-07-17T08:47:08.644730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14868":{"gid":14868,"name":"b","rank":0,"incarnation":214,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1064157662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1064157662},{"type":"v1","addr":"172.21.15.175:6801","nonce":1064157662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[52],"metadata_pool":51,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14868,"qdb_cluster":[14868]},"id":25}]} 2024-07-17T08:47:12.237 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 218 2024-07-17T08:47:12.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.237+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 msgr2=0x7fcbe0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.237+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 0x7fcbe0079190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc001f77d0 tx=0x7fcbec005d90 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.237+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 msgr2=0x7fcc001482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.237+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000b3e0 tx=0x7fcbf000b8a0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 shutdown_connections 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fcbe0076cd0 0x7fcbe0079190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc001522c0 0x7fcc001f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc000cd690 0x7fcc000b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 --2- 172.21.15.44:0/784478657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc000ac220 0x7fcc001482b0 unknown :-1 s=CLOSED pgs=3474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 >> 172.21.15.44:0/784478657 conn(0x7fcc000b0fb0 msgr2=0x7fcc000afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 shutdown_connections 2024-07-17T08:47:12.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.238+0000 7fcc04c00640 1 -- 172.21.15.44:0/784478657 wait complete. 2024-07-17T08:47:12.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-17T08:47:05.578374+0000', 'modified': '2024-07-17T08:47:08.644730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14868}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14868': {'gid': 14868, 'name': 'b', 'rank': 0, 'incarnation': 214, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6801/1064157662', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1064157662}, {'type': 'v1', 'addr': '172.21.15.175:6801', 'nonce': 1064157662}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [52], 'metadata_pool': 51, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 14868, 'qdb_cluster': [14868]} 2024-07-17T08:47:12.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-17T08:47:12.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 -- 172.21.15.44:0/1867752768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80ac3a0 msgr2=0x7efcb80ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1867752768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80ac3a0 0x7efcb80ac810 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 crypto rx=0x7efcb8070880 tx=0x7efcb002aa10 comp rx=0 tx=0).stop 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 -- 172.21.15.44:0/1867752768 shutdown_connections 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1867752768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80b3280 0x7efcb806eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1867752768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80ac3a0 0x7efcb80ac810 unknown :-1 s=CLOSED pgs=3475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1867752768 >> v2:172.21.15.175:3300/0 conn(0x7efcb806f6d0 0x7efcb815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 -- 172.21.15.44:0/1867752768 >> 172.21.15.44:0/1867752768 conn(0x7efcb80b0eb0 msgr2=0x7efcb80b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 -- 172.21.15.44:0/1867752768 shutdown_connections 2024-07-17T08:47:12.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.881+0000 7efcbfe00640 1 -- 172.21.15.44:0/1867752768 wait complete. 2024-07-17T08:47:12.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.883+0000 7efcbfe00640 1 Processor -- start 2024-07-17T08:47:12.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.883+0000 7efcbfe00640 1 -- start start 2024-07-17T08:47:12.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb806f6d0 0x7efcb8098330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80ac3a0 0x7efcb814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80ae8f0 con 0x7efcb80b3280 2024-07-17T08:47:12.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80ae770 con 0x7efcb806f6d0 2024-07-17T08:47:12.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efcb80aea70 con 0x7efcb80ac3a0 2024-07-17T08:47:12.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50610/0 (socket says 172.21.15.44:50610) 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbde00640 1 -- 172.21.15.44:0/1146959211 learned_addr learned my addr 172.21.15.44:0/1146959211 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.884+0000 7efcbca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80ac3a0 0x7efcb814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbd400640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb806f6d0 0x7efcb8098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 -- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80ac3a0 msgr2=0x7efcb814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80ac3a0 0x7efcb814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 -- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb806f6d0 msgr2=0x7efcb8098330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb806f6d0 0x7efcb8098330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 -- 172.21.15.44:0/1146959211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb8073370 con 0x7efcb80b3280 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.885+0000 7efcbde00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7efcb002b000 tx=0x7efcb0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.886+0000 7efcb5600640 1 -- 172.21.15.44:0/1146959211 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0046070 con 0x7efcb80b3280 2024-07-17T08:47:12.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.886+0000 7efcb5600640 1 -- 172.21.15.44:0/1146959211 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcb0004250 con 0x7efcb80b3280 2024-07-17T08:47:12.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.886+0000 7efcb5600640 1 -- 172.21.15.44:0/1146959211 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0033050 con 0x7efcb80b3280 2024-07-17T08:47:12.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.886+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb8073d20 con 0x7efcb80b3280 2024-07-17T08:47:12.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.886+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb8072fe0 con 0x7efcb80b3280 2024-07-17T08:47:12.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.888+0000 7efcb5600640 1 -- 172.21.15.44:0/1146959211 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7efcb0049070 con 0x7efcb80b3280 2024-07-17T08:47:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.889+0000 7efcb5600640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 0x7efc980791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.889+0000 7efcb5600640 1 -- 172.21.15.44:0/1146959211 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7efcb00b97d0 con 0x7efcb80b3280 2024-07-17T08:47:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.889+0000 7efcc2bf2740 1 Processor -- start 2024-07-17T08:47:12.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.889+0000 7efcc2bf2740 1 -- start start 2024-07-17T08:47:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcbd400640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 0x7efc980791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f45737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f457a090 0x5642f4573d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642f44ead60 con 0x5642f459c0b0 2024-07-17T08:47:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642f44da260 con 0x5642f457a090 2024-07-17T08:47:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.890+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5642f44de2a0 con 0x5642f459c490 2024-07-17T08:47:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50622/0 (socket says 172.21.15.44:50622) 2024-07-17T08:47:12.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 -- 172.21.15.44:0/3214010845 learned_addr learned my addr 172.21.15.44:0/3214010845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbde00640 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f457a090 0x5642f4573d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 -- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 msgr2=0x5642f45737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f45737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 -- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f457a090 msgr2=0x5642f4573d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f457a090 0x5642f4573d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 -- 172.21.15.44:0/3214010845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f423e850 con 0x5642f459c0b0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 0x7efc980791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b3b20 tx=0x7efcac009cb0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efcbd400640 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 secure :-1 s=READY pgs=3477 cs=0 l=1 rev1=1 crypto rx=0x7efcac00d8f0 tx=0x7efcac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7047c4fdae8e6531 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.891+0000 7efca9c00640 1 -- 172.21.15.44:0/3214010845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac024070 con 0x5642f459c0b0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efca9c00640 1 -- 172.21.15.44:0/3214010845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcac01b040 con 0x5642f459c0b0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efca9c00640 1 -- 172.21.15.44:0/3214010845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac017d50 con 0x5642f459c0b0 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3214010845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 msgr2=0x5642f4573290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 secure :-1 s=READY pgs=3477 cs=0 l=1 rev1=1 crypto rx=0x7efcac00d8f0 tx=0x7efcac018000 comp rx=0 tx=0).stop 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3214010845 shutdown_connections 2024-07-17T08:47:12.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f457a090 0x5642f4573d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f45737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3214010845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c0b0 0x5642f4573290 unknown :-1 s=CLOSED pgs=3477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3214010845 >> 172.21.15.44:0/3214010845 conn(0x5642f4575280 msgr2=0x5642f4578ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.892+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3214010845 shutdown_connections 2024-07-17T08:47:12.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.893+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3214010845 wait complete. 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.893+0000 7efcc2bf2740 1 -- start start 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.893+0000 7efcc2bf2740 1 Processor -- start 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 0x5642f45b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 0x5642f45b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642f424d620 con 0x5642f45b5a80 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642f4481f90 con 0x5642f459c490 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.894+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5642f4576b70 con 0x5642f459c0b0 2024-07-17T08:47:12.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcc2bf2740 10 client.0 _ll_register_callbacks cb 0x5642f44a7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50628/0 (socket says 172.21.15.44:50628) 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 -- 172.21.15.44:0/3785793467 learned_addr learned my addr 172.21.15.44:0/3785793467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbca00640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 0x5642f45b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbd400640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 0x5642f45b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 -- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 msgr2=0x5642f45b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 0x5642f45b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 -- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 msgr2=0x5642f45b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 0x5642f45b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbde00640 1 -- 172.21.15.44:0/3785793467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f42b56b0 con 0x5642f45b5a80 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbca00640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 0x5642f45b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.895+0000 7efcbd400640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 0x5642f45b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcbde00640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 crypto rx=0x7efcb007e610 tx=0x7efcb0004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9b2a237f6e300f server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0082020 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7efcb0081280 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642f44b1900 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7efcb00b5050 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 10 client.14991 fetch_fsmap learned FSMap version 218 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x5642f459cec0 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5642f44b1900 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 218) ==== 3502+0+0 (secure 0 0 0) 0x7efcb007d990 con 0x5642f45b5a80 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 10 client.14991 fetch_fsmap finished waiting for FSMap version 218 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 20 client.14991 populate_metadata read hostname 'smithi044' 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45a6160 0x5642f45a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 4 client.14991 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-17T08:47:12.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.896+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 --> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642f44eccb0 con 0x5642f45a6160 2024-07-17T08:47:12.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.897+0000 7efcbd400640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45a6160 0x5642f45a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.898+0000 7efcbd400640 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45a6160 0x5642f45a6540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd08eef456df92fb server_cookie=738f0dbf9b64b6a2 in_seq=0 out_seq=0 2024-07-17T08:47:12.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.898+0000 7efc9e000640 10 client.14991 ms_handle_connect on v2:172.21.15.175:6800/1064157662 2024-07-17T08:47:12.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efc9e000640 1 -- 172.21.15.44:0/3785793467 <== mds.0 v2:172.21.15.175:6800/1064157662 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5642f44eccb0 con 0x5642f45a6160 2024-07-17T08:47:12.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efc9e000640 10 client.14991 handle_command_reply: tid=0 2024-07-17T08:47:12.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 client.14991 shutdown 2024-07-17T08:47:12.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 msgr2=0x5642f45b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 secure :-1 s=READY pgs=3478 cs=0 l=1 rev1=1 crypto rx=0x7efcb007e610 tx=0x7efcb0004980 comp rx=0 tx=0).stop 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 shutdown_connections 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45b5f00 unknown :-1 s=CLOSED pgs=3478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45a6160 0x5642f45a6540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c490 0x5642f45b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/3785793467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c0b0 0x5642f45b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efc9e000640 0 client.14991 ms_handle_reset on v2:172.21.15.175:6800/1064157662 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 >> 172.21.15.44:0/3785793467 conn(0x5642f457a090 msgr2=0x5642f45a27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.901+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 shutdown_connections 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.902+0000 7efcc2bf2740 1 -- 172.21.15.44:0/3785793467 wait complete. 2024-07-17T08:47:12.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.902+0000 7efcc2bf2740 20 client.14991 trim_cache size 0 max 16384 2024-07-17T08:47:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.922+0000 7efcc2bf2740 1 Processor -- start 2024-07-17T08:47:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.922+0000 7efcc2bf2740 1 -- start start 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f459dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f4578e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81591e0 con 0x5642f45b5a80 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcb814ca20 con 0x5642f459c0b0 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efcb8158b30 con 0x5642f459c490 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.923+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50644/0 (socket says 172.21.15.44:50644) 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 -- 172.21.15.44:0/2281287046 learned_addr learned my addr 172.21.15.44:0/2281287046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbd400640 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f459dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbca00640 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f4578e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 -- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 msgr2=0x5642f4578e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f4578e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 -- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 msgr2=0x5642f459dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f459dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.924+0000 7efcbde00640 1 -- 172.21.15.44:0/2281287046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f45a70c0 con 0x5642f45b5a80 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efcbde00640 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 secure :-1 s=READY pgs=3479 cs=0 l=1 rev1=1 crypto rx=0x7efcb007e7c0 tx=0x7efcb0086430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092d2e2cc5c25ce server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efca9200640 1 -- 172.21.15.44:0/2281287046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb009f070 con 0x5642f45b5a80 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efca9200640 1 -- 172.21.15.44:0/2281287046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcb008af90 con 0x5642f45b5a80 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efca9200640 1 -- 172.21.15.44:0/2281287046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0086570 con 0x5642f45b5a80 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efcc2bf2740 1 -- 172.21.15.44:0/2281287046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 msgr2=0x5642f45e4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.925+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 secure :-1 s=READY pgs=3479 cs=0 l=1 rev1=1 crypto rx=0x7efcb007e7c0 tx=0x7efcb0086430 comp rx=0 tx=0).stop 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 -- 172.21.15.44:0/2281287046 shutdown_connections 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f45b5a80 0x5642f45e4730 unknown :-1 s=CLOSED pgs=3479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f459c490 0x5642f4578e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/2281287046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f459dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 -- 172.21.15.44:0/2281287046 >> 172.21.15.44:0/2281287046 conn(0x5642f457a090 msgr2=0x5642f45d40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 -- 172.21.15.44:0/2281287046 shutdown_connections 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 -- 172.21.15.44:0/2281287046 wait complete. 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.926+0000 7efcc2bf2740 1 -- start start 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.927+0000 7efcc2bf2740 1 Processor -- start 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f46355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f45b5a80 0x5642f45d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642f424d620 con 0x5642f459c490 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642f45a73a0 con 0x5642f459c0b0 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5642f4481f90 con 0x5642f45b5a80 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcc2bf2740 10 client.0 _ll_register_callbacks cb 0x7efcb0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50654/0 (socket says 172.21.15.44:50654) 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 -- 172.21.15.44:0/1962619305 learned_addr learned my addr 172.21.15.44:0/1962619305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbde00640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f45b5a80 0x5642f45d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbd400640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f46355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 -- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f45b5a80 msgr2=0x5642f45d78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f45b5a80 0x5642f45d78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 -- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 msgr2=0x5642f46355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f46355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.928+0000 7efcbca00640 1 -- 172.21.15.44:0/1962619305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f4635b10 con 0x5642f459c490 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efcbca00640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 secure :-1 s=READY pgs=3480 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fa0 tx=0x7efca4012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d53002a13154286 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efc9e000640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4019030 con 0x5642f459c490 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb008af90 con 0x5642f459c490 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7efca401e410 con 0x5642f459c490 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7efca400f040 con 0x5642f459c490 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efcc2bf2740 10 client.14997 fetch_fsmap learned FSMap version 218 2024-07-17T08:47:12.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.929+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x5642f45b3b00 con 0x5642f459c490 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcb008af90 con 0x5642f459c490 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 218) ==== 3502+0+0 (secure 0 0 0) 0x7efca4028080 con 0x5642f459c490 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 10 client.14997 fetch_fsmap finished waiting for FSMap version 218 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 20 client.14997 populate_metadata read hostname 'smithi044' 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45b6b40 0x5642f45a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 4 client.14997 mds_command: new command op to 14868 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-17T08:47:12.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.930+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 --> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] -- command(tid 0: {"prefix": "status"}) -- 0x5642f44e8750 con 0x5642f45b6b40 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.931+0000 7efcbd400640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45b6b40 0x5642f45a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.931+0000 7efcbd400640 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45b6b40 0x5642f45a27f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db39190a96554f8a server_cookie=be801e50208ec55e in_seq=0 out_seq=0 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.931+0000 7efc9e000640 10 client.14997 ms_handle_connect on v2:172.21.15.175:6800/1064157662 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efc9e000640 1 -- 172.21.15.44:0/1962619305 <== mds.0 v2:172.21.15.175:6800/1064157662 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5642f44e8750 con 0x5642f45b6b40 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efc9e000640 10 client.14997 handle_command_reply: tid=0 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 client.14997 shutdown 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 msgr2=0x5642f45d7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 secure :-1 s=READY pgs=3480 cs=0 l=1 rev1=1 crypto rx=0x7efca4009fa0 tx=0x7efca4012700 comp rx=0 tx=0).stop 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 shutdown_connections 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5642f45b5a80 0x5642f45d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642f459c490 0x5642f45d7360 unknown :-1 s=CLOSED pgs=3480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:6800/1064157662,v1:172.21.15.175:6801/1064157662] conn(0x5642f45b6b40 0x5642f45a27f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 --2- 172.21.15.44:0/1962619305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642f459c0b0 0x5642f46355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 >> 172.21.15.44:0/1962619305 conn(0x5642f457a090 msgr2=0x5642f45a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efc9e000640 0 client.14997 ms_handle_reset on v2:172.21.15.175:6800/1064157662 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 shutdown_connections 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.932+0000 7efcc2bf2740 1 -- 172.21.15.44:0/1962619305 wait complete. 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcc2bf2740 20 client.14997 trim_cache size 0 max 16384 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "4797a2db-a688-4e60-bfb8-d8a9e86b84a3", 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 14868, 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-07-17T08:47:12.937 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 6.8294054419999997, 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 218, 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 379, 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 379, 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 11.697440734000001 2024-07-17T08:47:12.938 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 msgr2=0x7efc980791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 0x7efc980791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b3b20 tx=0x7efcac009cb0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 msgr2=0x7efcb814a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7efcb002b000 tx=0x7efcb0003f10 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 shutdown_connections 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7efc98076d00 0x7efc980791c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efcb80b3280 0x7efcb814a4a0 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcb80ac3a0 0x7efcb814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 --2- 172.21.15.44:0/1146959211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcb806f6d0 0x7efcb8098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 >> 172.21.15.44:0/1146959211 conn(0x7efcb80b0eb0 msgr2=0x7efcb80afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 shutdown_connections 2024-07-17T08:47:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:12.933+0000 7efcbfe00640 1 -- 172.21.15.44:0/1146959211 wait complete. 2024-07-17T08:47:12.952 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 -- 172.21.15.44:0/1985615203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 msgr2=0x7f3304004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 --2- 172.21.15.44:0/1985615203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f3304004c80 secure :-1 s=READY pgs=3482 cs=0 l=1 rev1=1 crypto rx=0x7f330800c260 tx=0x7f3304005830 comp rx=0 tx=0).stop 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f331fe00640 1 -- 172.21.15.44:0/1985615203 reap_dead start 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 -- 172.21.15.44:0/1985615203 shutdown_connections 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 -- 172.21.15.44:0/1985615203 >> 172.21.15.44:0/1985615203 conn(0x7f33200d00a0 msgr2=0x7f33200d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 -- 172.21.15.44:0/1985615203 shutdown_connections 2024-07-17T08:47:13.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.159+0000 7f3325200640 1 -- 172.21.15.44:0/1985615203 wait complete. 2024-07-17T08:47:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.161+0000 7f3325200640 1 Processor -- start 2024-07-17T08:47:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.161+0000 7f3325200640 1 -- start start 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33200c9fb0 0x7f33200c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33200cddb0 0x7f33200ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33200b56c0 con 0x7f3304004800 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33200b5540 con 0x7f33200c9fb0 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f3325200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33200d14e0 con 0x7f33200cddb0 2024-07-17T08:47:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f331f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f331f400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50680/0 (socket says 172.21.15.44:50680) 2024-07-17T08:47:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f331f400640 1 -- 172.21.15.44:0/2736763943 learned_addr learned my addr 172.21.15.44:0/2736763943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.162+0000 7f331fe00640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33200cddb0 0x7f33200ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f3317e00640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33200c9fb0 0x7f33200c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 -- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33200cddb0 msgr2=0x7f33200ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33200cddb0 0x7f33200ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 -- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33200c9fb0 msgr2=0x7f33200c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33200c9fb0 0x7f33200c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 -- 172.21.15.44:0/2736763943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33201f3c40 con 0x7f3304004800 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f331f400640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7f331000d5f0 tx=0x7f331000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33100062e0 con 0x7f3304004800 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3310015020 con 0x7f3304004800 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310003830 con 0x7f3304004800 2024-07-17T08:47:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.163+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33201f3f20 con 0x7f3304004800 2024-07-17T08:47:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.164+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33201f44e0 con 0x7f3304004800 2024-07-17T08:47:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.165+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332015d6e0 con 0x7f3304004800 2024-07-17T08:47:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.166+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f33100039d0 con 0x7f3304004800 2024-07-17T08:47:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.167+0000 7f3317400640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 0x7f32fc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.167+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f3310098fe0 con 0x7f3304004800 2024-07-17T08:47:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.168+0000 7f3317e00640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 0x7f32fc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.169+0000 7f3317e00640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 0x7f32fc079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3308033000 tx=0x7f332015e8c0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:13.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.171+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f331001f020 con 0x7f3304004800 2024-07-17T08:47:13.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.424+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f332015eb40 con 0x7f3304004800 2024-07-17T08:47:13.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.425+0000 7f3317400640 1 -- 172.21.15.44:0/2736763943 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v218) ==== 53+0+73 (secure 0 0 0) 0x7f3310061ea0 con 0x7f3304004800 2024-07-17T08:47:13.426 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.427+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 msgr2=0x7f32fc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.427+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 0x7f32fc079190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3308033000 tx=0x7f332015e8c0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.427+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 msgr2=0x7f33200b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.427+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7f331000d5f0 tx=0x7f331000dab0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 shutdown_connections 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f32fc076cd0 0x7f32fc079190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33200cddb0 0x7f33200ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33200c9fb0 0x7f33200c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 --2- 172.21.15.44:0/2736763943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3304004800 0x7f33200b3d30 unknown :-1 s=CLOSED pgs=3483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 >> 172.21.15.44:0/2736763943 conn(0x7f33200d00a0 msgr2=0x7f33200b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:13.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 shutdown_connections 2024-07-17T08:47:13.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13.428+0000 7f3325200640 1 -- 172.21.15.44:0/2736763943 wait complete. 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-17T08:47:13.449 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:47:13.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:13.450 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-07-17T08:47:13.501 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 35824sec preferred_lft 35824sec 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-07-17T08:47:13.502 INFO:teuthology.orchestra.run.smithi044.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::a4c7:a9ff:fe54:87bc/64 scope link 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 16:e8:55:b4:51:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::14e8:55ff:feb4:517a/64 scope link 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-17T08:47:13.503 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 96:70:84:7f:30:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:47:13.504 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::9470:84ff:fe7f:30a5/64 scope link 2024-07-17T08:47:13.504 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-07-17T08:47:13.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:13.506 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-17T08:47:13.586 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-07-17T08:47:13.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:13.591 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-07-17T08:47:13.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:47:13.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:47:13.644 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-07-17T08:47:13.644 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-17T08:47:13.644 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-17T08:47:13.644 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-17T08:47:13.644 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-17T08:47:13.645 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-07-17T08:47:13.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-17T08:47:13.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:47:13.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:13.901 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:47:14.000 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:47:14.000 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:47:14.001 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:47:14.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:14.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-07-17T08:47:14.088 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:47:14.088 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-17T08:47:14.146 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-17T08:47:14.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-17T08:47:14.147 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:47:14.198 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-07-17T08:47:14.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:14.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:47:14.323 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:47:14.323 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:47:14.323 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:47:14.346 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:47:14.346 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:47:14.346 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:47:14.346 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:47:14.346 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:invalid new device string format 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: switching to using v1 address with old syntax 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.44:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-07-17T08:47:14.375 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4797a2db-a688-4e60-bfb8-d8a9e86b84a3" will pass to kernel 2024-07-17T08:47:14.380 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-17T08:47:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-07-17T08:47:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-17T08:47:14.380 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-07-17T08:47:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-17T08:47:14.384 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-17T08:47:14.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:47:14.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-17T08:47:14.457 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-17T08:47:14.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-17T08:47:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-17T08:47:14.604 INFO:teuthology.orchestra.run.smithi044.stdout:client10529 2024-07-17T08:47:14.604 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:47:14.604 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:47:14.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 -- 172.21.15.44:0/2371190519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c003ac0 msgr2=0x7fc25c003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 --2- 172.21.15.44:0/2371190519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c003ac0 0x7fc25c003ee0 secure :-1 s=READY pgs=3485 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c310 tx=0x7fc25c005830 comp rx=0 tx=0).stop 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 -- 172.21.15.44:0/2371190519 shutdown_connections 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 --2- 172.21.15.44:0/2371190519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c003ac0 0x7fc25c003ee0 unknown :-1 s=CLOSED pgs=3485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 -- 172.21.15.44:0/2371190519 >> 172.21.15.44:0/2371190519 conn(0x7fc2780b4ff0 msgr2=0x7fc2780b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.873+0000 7fc27e000640 1 -- 172.21.15.44:0/2371190519 shutdown_connections 2024-07-17T08:47:14.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.874+0000 7fc27e000640 1 -- 172.21.15.44:0/2371190519 wait complete. 2024-07-17T08:47:14.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.875+0000 7fc27e000640 1 Processor -- start 2024-07-17T08:47:14.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.875+0000 7fc27e000640 1 -- start start 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c003ac0 0x7fc2781f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc278160350 0x7fc2781535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278153b00 0x7fc278145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780b2860 con 0x7fc278160350 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780b26e0 con 0x7fc278153b00 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc27e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780cfdf0 con 0x7fc25c003ac0 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.876+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc278160350 0x7fc2781535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc278160350 0x7fc2781535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50696/0 (socket says 172.21.15.44:50696) 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 -- 172.21.15.44:0/4130512682 learned_addr learned my addr 172.21.15.44:0/4130512682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc27cc00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c003ac0 0x7fc2781f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc27d600640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278153b00 0x7fc278145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:14.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 -- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c003ac0 msgr2=0x7fc2781f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c003ac0 0x7fc2781f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 -- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278153b00 msgr2=0x7fc278145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:47:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278153b00 0x7fc278145250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:47:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 -- 172.21.15.44:0/4130512682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc278155740 con 0x7fc278160350 2024-07-17T08:47:14.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc27cc00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc25c003ac0 0x7fc2781f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.877+0000 7fc26fe00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc278160350 0x7fc2781535c0 secure :-1 s=READY pgs=3486 cs=0 l=1 rev1=1 crypto rx=0x7fc26000d5e0 tx=0x7fc26000daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.878+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2600064e0 con 0x7fc278160350 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.878+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc260003610 con 0x7fc278160350 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.878+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260003a60 con 0x7fc278160350 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.878+0000 7fc27e000640 1 -- 172.21.15.44:0/4130512682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc278155a20 con 0x7fc278160350 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.878+0000 7fc27e000640 1 -- 172.21.15.44:0/4130512682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc278155fe0 con 0x7fc278160350 2024-07-17T08:47:14.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.879+0000 7fc27e000640 1 -- 172.21.15.44:0/4130512682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2780d02c0 con 0x7fc278160350 2024-07-17T08:47:14.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.880+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fc260003c00 con 0x7fc278160350 2024-07-17T08:47:14.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.881+0000 7fc275600640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc254076cd0 0x7fc254079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:47:14.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.881+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fc260098bf0 con 0x7fc278160350 2024-07-17T08:47:14.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.882+0000 7fc27cc00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc254076cd0 0x7fc254079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:47:14.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.883+0000 7fc27cc00640 1 --2- 172.21.15.44:0/4130512682 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc254076cd0 0x7fc254079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc268033000 tx=0x7fc268001a60 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:47:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:14.885+0000 7fc275600640 1 -- 172.21.15.44:0/4130512682 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fc260061ba0 con 0x7fc278160350 2024-07-17T08:47:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:47:15.121+0000 7fc27e000640 1 -- 172.21.15.44:0/4130512682 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc2780d0e10 con 0x7fc254076cd0 2024-07-17T08:47:43.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:47:43.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:48:13.919 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:48:13.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:48:44.049 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:48:44.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:49:14.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:49:14.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:49:14.769 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:Test subvol schedule listing succeeds with a subvol and a subvolgroup argument. ... ERROR 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner: 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:====================================================================== 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:Test subvol schedule listing succeeds with a subvol and a subvolgroup argument. 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-17T08:49:14.821 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 619, in setUp 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: super(TestSnapSchedulesSubvolAndGroupArguments, self).setUp() 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 103, in setUp 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: self._create_or_reuse_test_volume() 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 79, in _create_or_reuse_test_volume 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: result = json.loads(self._fs_cmd("volume", "ls")) 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 63, in _fs_cmd 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_manager.py", line 1652, in run_cluster_cmd 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/remote.py", line 523, in run 2024-07-17T08:49:14.822 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 455, in run 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: r.wait() 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 161, in wait 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls' 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner:Ran 22 tests in 6820.137s 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-07-17T08:49:14.823 INFO:tasks.cephfs_test_runner: 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner:====================================================================== 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner:Test subvol schedule listing succeeds with a subvol and a subvolgroup argument. 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 619, in setUp 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: super(TestSnapSchedulesSubvolAndGroupArguments, self).setUp() 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 103, in setUp 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: self._create_or_reuse_test_volume() 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 79, in _create_or_reuse_test_volume 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: result = json.loads(self._fs_cmd("volume", "ls")) 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs/test_snap_schedules.py", line 63, in _fs_cmd 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-07-17T08:49:14.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/ceph_manager.py", line 1652, in run_cluster_cmd 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/remote.py", line 523, in run 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 455, in run 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: r.wait() 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 161, in wait 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls' 2024-07-17T08:49:14.825 INFO:tasks.cephfs_test_runner: 2024-07-17T08:49:14.826 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:49:15.106 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=513b20ac45f04564bb49a579c0eb9741 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_9c8b7e3d36b252529f99a61f43068893847721b3/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2024-07-17T08:49:15.108 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-07-17T08:49:15.118 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-17T08:49:15.125 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-17T08:49:15.126 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:49:15.126 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:49:15.188 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:49:15.189 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:49:15.241 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-17T08:49:15.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 -- 172.21.15.44:0/2782414901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe34006ed30 msgr2=0x7fe34015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 --2- 172.21.15.44:0/2782414901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe34006ed30 0x7fe34015c2f0 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005fd0 tx=0x7fe32c02a540 comp rx=0 tx=0).stop 2024-07-17T08:49:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 -- 172.21.15.44:0/2782414901 shutdown_connections 2024-07-17T08:49:15.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 --2- 172.21.15.44:0/2782414901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe34006ed30 0x7fe34015c2f0 unknown :-1 s=CLOSED pgs=3488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 --2- 172.21.15.44:0/2782414901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 0x7fe34006d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.457+0000 7fe347800640 1 -- 172.21.15.44:0/2782414901 >> 172.21.15.44:0/2782414901 conn(0x7fe3400b0eb0 msgr2=0x7fe3400b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.458+0000 7fe347800640 1 -- 172.21.15.44:0/2782414901 shutdown_connections 2024-07-17T08:49:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.458+0000 7fe347800640 1 -- 172.21.15.44:0/2782414901 wait complete. 2024-07-17T08:49:15.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.460+0000 7fe347800640 1 Processor -- start 2024-07-17T08:49:15.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.460+0000 7fe347800640 1 -- start start 2024-07-17T08:49:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe34006ed30 0x7fe3401f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 0x7fe34015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:15.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400cbbf0 con 0x7fe3400cb3a0 2024-07-17T08:49:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400cba70 con 0x7fe34006ed30 2024-07-17T08:49:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe347800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe34015f570 con 0x7fe340152220 2024-07-17T08:49:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.461+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34250/0 (socket says 172.21.15.44:34250) 2024-07-17T08:49:15.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 -- 172.21.15.44:0/3710734055 learned_addr learned my addr 172.21.15.44:0/3710734055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe345800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 0x7fe34015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe344e00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe34006ed30 0x7fe3401f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 -- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 msgr2=0x7fe34015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 0x7fe34015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 -- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe34006ed30 msgr2=0x7fe3401f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe34006ed30 0x7fe3401f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 -- 172.21.15.44:0/3710734055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe340152a80 con 0x7fe3400cb3a0 2024-07-17T08:49:15.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.462+0000 7fe33fe00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006010 tx=0x7fe32c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.463+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c03f070 con 0x7fe3400cb3a0 2024-07-17T08:49:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.463+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe32c004890 con 0x7fe3400cb3a0 2024-07-17T08:49:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.463+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c035030 con 0x7fe3400cb3a0 2024-07-17T08:49:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.463+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe340152d60 con 0x7fe3400cb3a0 2024-07-17T08:49:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.463+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe340153320 con 0x7fe3400cb3a0 2024-07-17T08:49:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.464+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34015fac0 con 0x7fe3400cb3a0 2024-07-17T08:49:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.465+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fe32c042070 con 0x7fe3400cb3a0 2024-07-17T08:49:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.466+0000 7fe33cc00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 0x7fe320079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.466+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fe32c0b97e0 con 0x7fe3400cb3a0 2024-07-17T08:49:15.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.467+0000 7fe344e00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 0x7fe320079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.468+0000 7fe344e00640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 0x7fe320079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe334005fd0 tx=0x7fe334005da0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:15.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.471+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe32c082680 con 0x7fe3400cb3a0 2024-07-17T08:49:15.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.696+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3400afe30 con 0x7fe3400cb3a0 2024-07-17T08:49:15.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.697+0000 7fe33cc00640 1 -- 172.21.15.44:0/3710734055 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v379) ==== 80+0+0 (secure 0 0 0) 0x7fe32c081f80 con 0x7fe3400cb3a0 2024-07-17T08:49:15.699 INFO:teuthology.orchestra.run.smithi044.stderr:listed 0 entries 2024-07-17T08:49:15.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 msgr2=0x7fe320079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:15.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 0x7fe320079190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe334005fd0 tx=0x7fe334005da0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 msgr2=0x7fe340073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:15.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006010 tx=0x7fe32c0039e0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 shutdown_connections 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe320076cd0 0x7fe320079190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe340152220 0x7fe34015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3400cb3a0 0x7fe340073e10 unknown :-1 s=CLOSED pgs=3489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 --2- 172.21.15.44:0/3710734055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe34006ed30 0x7fe3401f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 >> 172.21.15.44:0/3710734055 conn(0x7fe3400b0eb0 msgr2=0x7fe3400b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 shutdown_connections 2024-07-17T08:49:15.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:15.700+0000 7fe347800640 1 -- 172.21.15.44:0/3710734055 wait complete. 2024-07-17T08:49:15.720 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-17T08:49:15.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:15.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-17T08:49:15.811 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:49:15.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:15.812 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-17T08:49:15.863 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-07-17T08:49:15.863 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:49:15.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053457,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249608k,nr_inodes=812402,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499220k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-07-17T08:49:15.911 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:49:15.911 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:49:15.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:49:15.969 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-17T08:49:15.969 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-17T08:49:16.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052672,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248980k,nr_inodes=812245,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497964k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-07-17T08:49:16.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.022 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:49:16.079 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-17T08:49:16.079 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-07-17T08:49:16.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.079 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-17T08:49:16.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-07-17T08:49:16.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.517 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-07-17T08:49:16.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.897 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-07-17T08:49:16.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:16.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-07-17T08:49:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-07-17T08:49:17.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:49:17.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.072 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T08:49:17.129 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-07-17T08:49:17.129 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-07-17T08:49:17.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-17T08:49:17.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-07-17T08:49:17.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-17T08:49:17.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.936 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-07-17T08:49:17.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-17T08:49:17.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-07-17T08:49:18.107 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-07-17T08:49:18.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:49:18.111 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-07-17T08:49:18.121 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-17T08:49:18.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:49:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 -- 172.21.15.44:0/1624213852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381522c0 msgr2=0x7fa6380ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 --2- 172.21.15.44:0/1624213852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381522c0 0x7fa6380ad790 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 crypto rx=0x7fa628005fd0 tx=0x7fa62802aaa0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 -- 172.21.15.44:0/1624213852 shutdown_connections 2024-07-17T08:49:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 --2- 172.21.15.44:0/1624213852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6381522c0 0x7fa6380ad790 unknown :-1 s=CLOSED pgs=3491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 -- 172.21.15.44:0/1624213852 >> 172.21.15.44:0/1624213852 conn(0x7fa6380b0fb0 msgr2=0x7fa6380b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 -- 172.21.15.44:0/1624213852 shutdown_connections 2024-07-17T08:49:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.341+0000 7fa63f200640 1 -- 172.21.15.44:0/1624213852 wait complete. 2024-07-17T08:49:18.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.343+0000 7fa63f200640 1 Processor -- start 2024-07-17T08:49:18.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.343+0000 7fa63f200640 1 -- start start 2024-07-17T08:49:18.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380adcd0 0x7fa638150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6381522c0 0x7fa6381f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380ce1a0 con 0x7fa6380ac930 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380ce020 con 0x7fa6380adcd0 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa63f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380ce320 con 0x7fa6381522c0 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa637e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa637e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34266/0 (socket says 172.21.15.44:34266) 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.344+0000 7fa637e00640 1 -- 172.21.15.44:0/3098572737 learned_addr learned my addr 172.21.15.44:0/3098572737 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 -- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6381522c0 msgr2=0x7fa6381f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6381522c0 0x7fa6381f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 -- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380adcd0 msgr2=0x7fa638150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380adcd0 0x7fa638150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 -- 172.21.15.44:0/3098572737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380c8b70 con 0x7fa6380ac930 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa637e00640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7fa628000f80 tx=0x7fa628003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.345+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628044070 con 0x7fa6380ac930 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.346+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa628004250 con 0x7fa6380ac930 2024-07-17T08:49:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.346+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa638071870 con 0x7fa6380ac930 2024-07-17T08:49:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.346+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628004680 con 0x7fa6380ac930 2024-07-17T08:49:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.346+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa638071e30 con 0x7fa6380ac930 2024-07-17T08:49:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.347+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6380ce840 con 0x7fa6380ac930 2024-07-17T08:49:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.348+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fa628047070 con 0x7fa6380ac930 2024-07-17T08:49:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.349+0000 7fa634c00640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.349+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fa6280b9830 con 0x7fa6380ac930 2024-07-17T08:49:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.350+0000 7fa62f400640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.351+0000 7fa62f400640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 0x7fa614079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa620000f60 tx=0x7fa6200079b0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.354+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fa6380c8b70 con 0x7fa6380ac930 2024-07-17T08:49:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.557+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa638071620 con 0x7fa614076cd0 2024-07-17T08:49:18.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.571+0000 7fa634c00640 1 -- 172.21.15.44:0/3098572737 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252429 (secure 0 0 0) 0x7fa638071620 con 0x7fa614076cd0 2024-07-17T08:49:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:18.578 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:49:18.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.577+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 msgr2=0x7fa614079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.577+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 0x7fa614079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa620000f60 tx=0x7fa6200079b0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.577+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 msgr2=0x7fa6380c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7fa628000f80 tx=0x7fa628003f10 comp rx=0 tx=0).stop 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 shutdown_connections 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fa614076cd0 0x7fa614079190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6381522c0 0x7fa6381f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6380adcd0 0x7fa638150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 --2- 172.21.15.44:0/3098572737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6380ac930 0x7fa6380c8630 unknown :-1 s=CLOSED pgs=3492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 >> 172.21.15.44:0/3098572737 conn(0x7fa6380b0fb0 msgr2=0x7fa6380b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 shutdown_connections 2024-07-17T08:49:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.578+0000 7fa63f200640 1 -- 172.21.15.44:0/3098572737 wait complete. 2024-07-17T08:49:18.599 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-07-17T08:49:17.933331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594165,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":642,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4725,"ondisk_log_size":4725,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546552,"kb_used_data":38920,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428168,"statfs":{"total":773094113280,"available":772534444032,"internally_reserved":0,"allocated":39854080,"data_stored":22551474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004212"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884745+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:47:07.884745+0000","last_peered":"2024-07-17T08:47:07.884745+0000","last_clean":"2024-07-17T08:47:07.884745+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:47:07.884745+0000","last_undegraded":"2024-07-17T08:47:07.884745+0000","last_fullsized":"2024-07-17T08:47:07.884745+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890201+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:47:07.890201+0000","last_peered":"2024-07-17T08:47:07.890201+0000","last_clean":"2024-07-17T08:47:07.890201+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:47:07.890201+0000","last_undegraded":"2024-07-17T08:47:07.890201+0000","last_fullsized":"2024-07-17T08:47:07.890201+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:43:50.699357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890224+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.890224+0000","last_peered":"2024-07-17T08:47:07.890224+0000","last_clean":"2024-07-17T08:47:07.890224+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.890224+0000","last_undegraded":"2024-07-17T08:47:07.890224+0000","last_fullsized":"2024-07-17T08:47:07.890224+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888208+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.888208+0000","last_peered":"2024-07-17T08:47:07.888208+0000","last_clean":"2024-07-17T08:47:07.888208+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.888208+0000","last_undegraded":"2024-07-17T08:47:07.888208+0000","last_fullsized":"2024-07-17T08:47:07.888208+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221716+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.221716+0000","last_peered":"2024-07-17T08:47:07.221716+0000","last_clean":"2024-07-17T08:47:07.221716+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.221716+0000","last_undegraded":"2024-07-17T08:47:07.221716+0000","last_fullsized":"2024-07-17T08:47:07.221716+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:03.150509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273776+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.273776+0000","last_peered":"2024-07-17T08:47:07.273776+0000","last_clean":"2024-07-17T08:47:07.273776+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.273776+0000","last_undegraded":"2024-07-17T08:47:07.273776+0000","last_fullsized":"2024-07-17T08:47:07.273776+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221537+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.221537+0000","last_peered":"2024-07-17T08:47:07.221537+0000","last_clean":"2024-07-17T08:47:07.221537+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.221537+0000","last_undegraded":"2024-07-17T08:47:07.221537+0000","last_fullsized":"2024-07-17T08:47:07.221537+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:08.644619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273038+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.273038+0000","last_peered":"2024-07-17T08:47:07.273038+0000","last_clean":"2024-07-17T08:47:07.273038+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.273038+0000","last_undegraded":"2024-07-17T08:47:07.273038+0000","last_fullsized":"2024-07-17T08:47:07.273038+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271623+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.271623+0000","last_peered":"2024-07-17T08:47:07.271623+0000","last_clean":"2024-07-17T08:47:07.271623+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.271623+0000","last_undegraded":"2024-07-17T08:47:07.271623+0000","last_fullsized":"2024-07-17T08:47:07.271623+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221733+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.221733+0000","last_peered":"2024-07-17T08:47:07.221733+0000","last_clean":"2024-07-17T08:47:07.221733+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.221733+0000","last_undegraded":"2024-07-17T08:47:07.221733+0000","last_fullsized":"2024-07-17T08:47:07.221733+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:19.898818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274204+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.274204+0000","last_peered":"2024-07-17T08:47:07.274204+0000","last_clean":"2024-07-17T08:47:07.274204+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.274204+0000","last_undegraded":"2024-07-17T08:47:07.274204+0000","last_fullsized":"2024-07-17T08:47:07.274204+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:29.623868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141167+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.141167+0000","last_peered":"2024-07-17T08:47:07.141167+0000","last_clean":"2024-07-17T08:47:07.141167+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.141167+0000","last_undegraded":"2024-07-17T08:47:07.141167+0000","last_fullsized":"2024-07-17T08:47:07.141167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144133+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.144133+0000","last_peered":"2024-07-17T08:47:07.144133+0000","last_clean":"2024-07-17T08:47:07.144133+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.144133+0000","last_undegraded":"2024-07-17T08:47:07.144133+0000","last_fullsized":"2024-07-17T08:47:07.144133+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221396+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:47:07.221396+0000","last_peered":"2024-07-17T08:47:07.221396+0000","last_clean":"2024-07-17T08:47:07.221396+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:47:07.221396+0000","last_undegraded":"2024-07-17T08:47:07.221396+0000","last_fullsized":"2024-07-17T08:47:07.221396+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:00:09.543515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140850+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.140850+0000","last_peered":"2024-07-17T08:47:07.140850+0000","last_clean":"2024-07-17T08:47:07.140850+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.140850+0000","last_undegraded":"2024-07-17T08:47:07.140850+0000","last_fullsized":"2024-07-17T08:47:07.140850+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140398+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.140398+0000","last_peered":"2024-07-17T08:47:07.140398+0000","last_clean":"2024-07-17T08:47:07.140398+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.140398+0000","last_undegraded":"2024-07-17T08:47:07.140398+0000","last_fullsized":"2024-07-17T08:47:07.140398+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888113+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.888113+0000","last_peered":"2024-07-17T08:47:07.888113+0000","last_clean":"2024-07-17T08:47:07.888113+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.888113+0000","last_undegraded":"2024-07-17T08:47:07.888113+0000","last_fullsized":"2024-07-17T08:47:07.888113+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889751+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.889751+0000","last_peered":"2024-07-17T08:47:07.889751+0000","last_clean":"2024-07-17T08:47:07.889751+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.889751+0000","last_undegraded":"2024-07-17T08:47:07.889751+0000","last_fullsized":"2024-07-17T08:47:07.889751+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869495+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.869495+0000","last_peered":"2024-07-17T08:47:07.869495+0000","last_clean":"2024-07-17T08:47:07.869495+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.869495+0000","last_undegraded":"2024-07-17T08:47:07.869495+0000","last_fullsized":"2024-07-17T08:47:07.869495+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886129+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.886129+0000","last_peered":"2024-07-17T08:47:07.886129+0000","last_clean":"2024-07-17T08:47:07.886129+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.886129+0000","last_undegraded":"2024-07-17T08:47:07.886129+0000","last_fullsized":"2024-07-17T08:47:07.886129+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886055+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.886055+0000","last_peered":"2024-07-17T08:47:07.886055+0000","last_clean":"2024-07-17T08:47:07.886055+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.886055+0000","last_undegraded":"2024-07-17T08:47:07.886055+0000","last_fullsized":"2024-07-17T08:47:07.886055+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886018+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.886018+0000","last_peered":"2024-07-17T08:47:07.886018+0000","last_clean":"2024-07-17T08:47:07.886018+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.886018+0000","last_undegraded":"2024-07-17T08:47:07.886018+0000","last_fullsized":"2024-07-17T08:47:07.886018+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885983+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.885983+0000","last_peered":"2024-07-17T08:47:07.885983+0000","last_clean":"2024-07-17T08:47:07.885983+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.885983+0000","last_undegraded":"2024-07-17T08:47:07.885983+0000","last_fullsized":"2024-07-17T08:47:07.885983+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890062+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.890062+0000","last_peered":"2024-07-17T08:47:07.890062+0000","last_clean":"2024-07-17T08:47:07.890062+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.890062+0000","last_undegraded":"2024-07-17T08:47:07.890062+0000","last_fullsized":"2024-07-17T08:47:07.890062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889946+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.889946+0000","last_peered":"2024-07-17T08:47:07.889946+0000","last_clean":"2024-07-17T08:47:07.889946+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.889946+0000","last_undegraded":"2024-07-17T08:47:07.889946+0000","last_fullsized":"2024-07-17T08:47:07.889946+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221117+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.221117+0000","last_peered":"2024-07-17T08:47:07.221117+0000","last_clean":"2024-07-17T08:47:07.221117+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.221117+0000","last_undegraded":"2024-07-17T08:47:07.221117+0000","last_fullsized":"2024-07-17T08:47:07.221117+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:53.930624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":19,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886272+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.886272+0000","last_peered":"2024-07-17T08:47:07.886272+0000","last_clean":"2024-07-17T08:47:07.886272+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.886272+0000","last_undegraded":"2024-07-17T08:47:07.886272+0000","last_fullsized":"2024-07-17T08:47:07.886272+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890219+0000","last_change":"2024-07-17T08:47:05.129655+0000","last_active":"2024-07-17T08:47:07.890219+0000","last_peered":"2024-07-17T08:47:07.890219+0000","last_clean":"2024-07-17T08:47:07.890219+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:47:07.890219+0000","last_undegraded":"2024-07-17T08:47:07.890219+0000","last_fullsized":"2024-07-17T08:47:07.890219+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:09:03.283870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272723+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.272723+0000","last_peered":"2024-07-17T08:47:07.272723+0000","last_clean":"2024-07-17T08:47:07.272723+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.272723+0000","last_undegraded":"2024-07-17T08:47:07.272723+0000","last_fullsized":"2024-07-17T08:47:07.272723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:17.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273634+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.273634+0000","last_peered":"2024-07-17T08:47:07.273634+0000","last_clean":"2024-07-17T08:47:07.273634+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.273634+0000","last_undegraded":"2024-07-17T08:47:07.273634+0000","last_fullsized":"2024-07-17T08:47:07.273634+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272935+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.272935+0000","last_peered":"2024-07-17T08:47:07.272935+0000","last_clean":"2024-07-17T08:47:07.272935+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.272935+0000","last_undegraded":"2024-07-17T08:47:07.272935+0000","last_fullsized":"2024-07-17T08:47:07.272935+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221435+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.221435+0000","last_peered":"2024-07-17T08:47:07.221435+0000","last_clean":"2024-07-17T08:47:07.221435+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.221435+0000","last_undegraded":"2024-07-17T08:47:07.221435+0000","last_fullsized":"2024-07-17T08:47:07.221435+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:50.145513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221353+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.221353+0000","last_peered":"2024-07-17T08:47:07.221353+0000","last_clean":"2024-07-17T08:47:07.221353+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.221353+0000","last_undegraded":"2024-07-17T08:47:07.221353+0000","last_fullsized":"2024-07-17T08:47:07.221353+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:50.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889720+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.889720+0000","last_peered":"2024-07-17T08:47:07.889720+0000","last_clean":"2024-07-17T08:47:07.889720+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.889720+0000","last_undegraded":"2024-07-17T08:47:07.889720+0000","last_fullsized":"2024-07-17T08:47:07.889720+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271594+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.271594+0000","last_peered":"2024-07-17T08:47:07.271594+0000","last_clean":"2024-07-17T08:47:07.271594+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.271594+0000","last_undegraded":"2024-07-17T08:47:07.271594+0000","last_fullsized":"2024-07-17T08:47:07.271594+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869512+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.869512+0000","last_peered":"2024-07-17T08:47:07.869512+0000","last_clean":"2024-07-17T08:47:07.869512+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.869512+0000","last_undegraded":"2024-07-17T08:47:07.869512+0000","last_fullsized":"2024-07-17T08:47:07.869512+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272214+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.272214+0000","last_peered":"2024-07-17T08:47:07.272214+0000","last_clean":"2024-07-17T08:47:07.272214+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.272214+0000","last_undegraded":"2024-07-17T08:47:07.272214+0000","last_fullsized":"2024-07-17T08:47:07.272214+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"379'10","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:15.803589+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:49:15.803589+0000","last_peered":"2024-07-17T08:49:15.803589+0000","last_clean":"2024-07-17T08:49:15.803589+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:49:15.803589+0000","last_undegraded":"2024-07-17T08:49:15.803589+0000","last_fullsized":"2024-07-17T08:49:15.803589+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221401+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.221401+0000","last_peered":"2024-07-17T08:47:07.221401+0000","last_clean":"2024-07-17T08:47:07.221401+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.221401+0000","last_undegraded":"2024-07-17T08:47:07.221401+0000","last_fullsized":"2024-07-17T08:47:07.221401+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:57.532859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889969+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:47:07.889969+0000","last_peered":"2024-07-17T08:47:07.889969+0000","last_clean":"2024-07-17T08:47:07.889969+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:47:07.889969+0000","last_undegraded":"2024-07-17T08:47:07.889969+0000","last_fullsized":"2024-07-17T08:47:07.889969+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:52.365125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274195+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:47:07.274195+0000","last_peered":"2024-07-17T08:47:07.274195+0000","last_clean":"2024-07-17T08:47:07.274195+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:47:07.274195+0000","last_undegraded":"2024-07-17T08:47:07.274195+0000","last_fullsized":"2024-07-17T08:47:07.274195+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885960+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.885960+0000","last_peered":"2024-07-17T08:47:07.885960+0000","last_clean":"2024-07-17T08:47:07.885960+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.885960+0000","last_undegraded":"2024-07-17T08:47:07.885960+0000","last_fullsized":"2024-07-17T08:47:07.885960+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144016+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.144016+0000","last_peered":"2024-07-17T08:47:07.144016+0000","last_clean":"2024-07-17T08:47:07.144016+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.144016+0000","last_undegraded":"2024-07-17T08:47:07.144016+0000","last_fullsized":"2024-07-17T08:47:07.144016+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890124+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.890124+0000","last_peered":"2024-07-17T08:47:07.890124+0000","last_clean":"2024-07-17T08:47:07.890124+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.890124+0000","last_undegraded":"2024-07-17T08:47:07.890124+0000","last_fullsized":"2024-07-17T08:47:07.890124+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141801+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.141801+0000","last_peered":"2024-07-17T08:47:07.141801+0000","last_clean":"2024-07-17T08:47:07.141801+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.141801+0000","last_undegraded":"2024-07-17T08:47:07.141801+0000","last_fullsized":"2024-07-17T08:47:07.141801+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275409+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.275409+0000","last_peered":"2024-07-17T08:47:07.275409+0000","last_clean":"2024-07-17T08:47:07.275409+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.275409+0000","last_undegraded":"2024-07-17T08:47:07.275409+0000","last_fullsized":"2024-07-17T08:47:07.275409+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273190+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.273190+0000","last_peered":"2024-07-17T08:47:07.273190+0000","last_clean":"2024-07-17T08:47:07.273190+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.273190+0000","last_undegraded":"2024-07-17T08:47:07.273190+0000","last_fullsized":"2024-07-17T08:47:07.273190+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141198+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.141198+0000","last_peered":"2024-07-17T08:47:07.141198+0000","last_clean":"2024-07-17T08:47:07.141198+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.141198+0000","last_undegraded":"2024-07-17T08:47:07.141198+0000","last_fullsized":"2024-07-17T08:47:07.141198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221468+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:47:07.221468+0000","last_peered":"2024-07-17T08:47:07.221468+0000","last_clean":"2024-07-17T08:47:07.221468+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:47:07.221468+0000","last_undegraded":"2024-07-17T08:47:07.221468+0000","last_fullsized":"2024-07-17T08:47:07.221468+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:37.618880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273673+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.273673+0000","last_peered":"2024-07-17T08:47:07.273673+0000","last_clean":"2024-07-17T08:47:07.273673+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.273673+0000","last_undegraded":"2024-07-17T08:47:07.273673+0000","last_fullsized":"2024-07-17T08:47:07.273673+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886001+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.886001+0000","last_peered":"2024-07-17T08:47:07.886001+0000","last_clean":"2024-07-17T08:47:07.886001+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.886001+0000","last_undegraded":"2024-07-17T08:47:07.886001+0000","last_fullsized":"2024-07-17T08:47:07.886001+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275184+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.275184+0000","last_peered":"2024-07-17T08:47:07.275184+0000","last_clean":"2024-07-17T08:47:07.275184+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.275184+0000","last_undegraded":"2024-07-17T08:47:07.275184+0000","last_fullsized":"2024-07-17T08:47:07.275184+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889551+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.889551+0000","last_peered":"2024-07-17T08:47:07.889551+0000","last_clean":"2024-07-17T08:47:07.889551+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.889551+0000","last_undegraded":"2024-07-17T08:47:07.889551+0000","last_fullsized":"2024-07-17T08:47:07.889551+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144992+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.144992+0000","last_peered":"2024-07-17T08:47:07.144992+0000","last_clean":"2024-07-17T08:47:07.144992+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.144992+0000","last_undegraded":"2024-07-17T08:47:07.144992+0000","last_fullsized":"2024-07-17T08:47:07.144992+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885909+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.885909+0000","last_peered":"2024-07-17T08:47:07.885909+0000","last_clean":"2024-07-17T08:47:07.885909+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.885909+0000","last_undegraded":"2024-07-17T08:47:07.885909+0000","last_fullsized":"2024-07-17T08:47:07.885909+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888132+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.888132+0000","last_peered":"2024-07-17T08:47:07.888132+0000","last_clean":"2024-07-17T08:47:07.888132+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.888132+0000","last_undegraded":"2024-07-17T08:47:07.888132+0000","last_fullsized":"2024-07-17T08:47:07.888132+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890304+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.890304+0000","last_peered":"2024-07-17T08:47:07.890304+0000","last_clean":"2024-07-17T08:47:07.890304+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.890304+0000","last_undegraded":"2024-07-17T08:47:07.890304+0000","last_fullsized":"2024-07-17T08:47:07.890304+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877385+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.877385+0000","last_peered":"2024-07-17T08:47:07.877385+0000","last_clean":"2024-07-17T08:47:07.877385+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.877385+0000","last_undegraded":"2024-07-17T08:47:07.877385+0000","last_fullsized":"2024-07-17T08:47:07.877385+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"379'3","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:16.142642+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:49:16.142642+0000","last_peered":"2024-07-17T08:49:16.142642+0000","last_clean":"2024-07-17T08:49:16.142642+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:49:16.142642+0000","last_undegraded":"2024-07-17T08:49:16.142642+0000","last_fullsized":"2024-07-17T08:49:16.142642+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884480+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:47:07.884480+0000","last_peered":"2024-07-17T08:47:07.884480+0000","last_clean":"2024-07-17T08:47:07.884480+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:47:07.884480+0000","last_undegraded":"2024-07-17T08:47:07.884480+0000","last_fullsized":"2024-07-17T08:47:07.884480+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877661+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.877661+0000","last_peered":"2024-07-17T08:47:07.877661+0000","last_clean":"2024-07-17T08:47:07.877661+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.877661+0000","last_undegraded":"2024-07-17T08:47:07.877661+0000","last_fullsized":"2024-07-17T08:47:07.877661+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890188+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.890188+0000","last_peered":"2024-07-17T08:47:07.890188+0000","last_clean":"2024-07-17T08:47:07.890188+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.890188+0000","last_undegraded":"2024-07-17T08:47:07.890188+0000","last_fullsized":"2024-07-17T08:47:07.890188+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890168+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:47:07.890168+0000","last_peered":"2024-07-17T08:47:07.890168+0000","last_clean":"2024-07-17T08:47:07.890168+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:47:07.890168+0000","last_undegraded":"2024-07-17T08:47:07.890168+0000","last_fullsized":"2024-07-17T08:47:07.890168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:28.605364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889649+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.889649+0000","last_peered":"2024-07-17T08:47:07.889649+0000","last_clean":"2024-07-17T08:47:07.889649+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.889649+0000","last_undegraded":"2024-07-17T08:47:07.889649+0000","last_fullsized":"2024-07-17T08:47:07.889649+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877509+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.877509+0000","last_peered":"2024-07-17T08:47:07.877509+0000","last_clean":"2024-07-17T08:47:07.877509+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.877509+0000","last_undegraded":"2024-07-17T08:47:07.877509+0000","last_fullsized":"2024-07-17T08:47:07.877509+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877625+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.877625+0000","last_peered":"2024-07-17T08:47:07.877625+0000","last_clean":"2024-07-17T08:47:07.877625+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.877625+0000","last_undegraded":"2024-07-17T08:47:07.877625+0000","last_fullsized":"2024-07-17T08:47:07.877625+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877473+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.877473+0000","last_peered":"2024-07-17T08:47:07.877473+0000","last_clean":"2024-07-17T08:47:07.877473+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.877473+0000","last_undegraded":"2024-07-17T08:47:07.877473+0000","last_fullsized":"2024-07-17T08:47:07.877473+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886198+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.886198+0000","last_peered":"2024-07-17T08:47:07.886198+0000","last_clean":"2024-07-17T08:47:07.886198+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.886198+0000","last_undegraded":"2024-07-17T08:47:07.886198+0000","last_fullsized":"2024-07-17T08:47:07.886198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877423+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.877423+0000","last_peered":"2024-07-17T08:47:07.877423+0000","last_clean":"2024-07-17T08:47:07.877423+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.877423+0000","last_undegraded":"2024-07-17T08:47:07.877423+0000","last_fullsized":"2024-07-17T08:47:07.877423+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886235+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.886235+0000","last_peered":"2024-07-17T08:47:07.886235+0000","last_clean":"2024-07-17T08:47:07.886235+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.886235+0000","last_undegraded":"2024-07-17T08:47:07.886235+0000","last_fullsized":"2024-07-17T08:47:07.886235+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890029+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.890029+0000","last_peered":"2024-07-17T08:47:07.890029+0000","last_clean":"2024-07-17T08:47:07.890029+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.890029+0000","last_undegraded":"2024-07-17T08:47:07.890029+0000","last_fullsized":"2024-07-17T08:47:07.890029+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6212,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272409+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272409+0000","last_peered":"2024-07-17T08:47:07.272409+0000","last_clean":"2024-07-17T08:47:07.272409+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272409+0000","last_undegraded":"2024-07-17T08:47:07.272409+0000","last_fullsized":"2024-07-17T08:47:07.272409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":3797,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543498,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80904,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290936,"statfs":{"total":96636764160,"available":96553918464,"internally_reserved":0,"allocated":5427200,"data_stored":3261377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510794,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76152,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295688,"statfs":{"total":96636764160,"available":96558784512,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80324,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291516,"statfs":{"total":96636764160,"available":96554512384,"internally_reserved":0,"allocated":4845568,"data_stored":2673484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510794,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4841472,"data_stored":2673414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:49:18.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:49:18.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 -- 172.21.15.44:0/7909056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 msgr2=0x7fd1780071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 --2- 172.21.15.44:0/7909056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1780071d0 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0016c0 tx=0x7fd178003810 comp rx=0 tx=0).stop 2024-07-17T08:49:18.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd192800640 1 -- 172.21.15.44:0/7909056 reap_dead start 2024-07-17T08:49:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 -- 172.21.15.44:0/7909056 shutdown_connections 2024-07-17T08:49:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 -- 172.21.15.44:0/7909056 >> 172.21.15.44:0/7909056 conn(0x7fd1940b0fb0 msgr2=0x7fd1940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 -- 172.21.15.44:0/7909056 shutdown_connections 2024-07-17T08:49:18.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.863+0000 7fd198e00640 1 -- 172.21.15.44:0/7909056 wait complete. 2024-07-17T08:49:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.865+0000 7fd198e00640 1 Processor -- start 2024-07-17T08:49:18.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.865+0000 7fd198e00640 1 -- start start 2024-07-17T08:49:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd194071830 0x7fd1941f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd194158fb0 0x7fd194159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd194153fb0 con 0x7fd178004de0 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd194153e30 con 0x7fd194071830 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.866+0000 7fd198e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd19406f710 con 0x7fd194158fb0 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33394/0 (socket says 172.21.15.44:33394) 2024-07-17T08:49:18.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 -- 172.21.15.44:0/2782984011 learned_addr learned my addr 172.21.15.44:0/2782984011 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd192800640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd194158fb0 0x7fd194159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 -- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd194158fb0 msgr2=0x7fd194159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd194158fb0 0x7fd194159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 -- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd194071830 msgr2=0x7fd1941f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191400640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd194071830 0x7fd1941f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd194071830 0x7fd1941f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.867+0000 7fd191e00640 1 -- 172.21.15.44:0/2782984011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1940740d0 con 0x7fd178004de0 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd191e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0017e0 tx=0x7fd17c006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c038070 con 0x7fd178004de0 2024-07-17T08:49:18.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd17c003af0 con 0x7fd178004de0 2024-07-17T08:49:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c033420 con 0x7fd178004de0 2024-07-17T08:49:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd194074370 con 0x7fd178004de0 2024-07-17T08:49:18.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.868+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1941e4120 con 0x7fd178004de0 2024-07-17T08:49:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.870+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1940ac840 con 0x7fd178004de0 2024-07-17T08:49:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.870+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fd17c043020 con 0x7fd178004de0 2024-07-17T08:49:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.871+0000 7fd18a000640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 0x7fd170079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:18.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.872+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fd17c0b7f90 con 0x7fd178004de0 2024-07-17T08:49:18.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.872+0000 7fd191400640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 0x7fd170079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:18.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.873+0000 7fd191400640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 0x7fd170079190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd194143180 tx=0x7fd184009500 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:18.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:18.876+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd17c080eb0 con 0x7fd178004de0 2024-07-17T08:49:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.083+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd194071e80 con 0x7fd170076cd0 2024-07-17T08:49:19.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.096+0000 7fd18a000640 1 -- 172.21.15.44:0/2782984011 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252429 (secure 0 0 0) 0x7fd194071e80 con 0x7fd170076cd0 2024-07-17T08:49:19.101 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 msgr2=0x7fd170079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 0x7fd170079190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd194143180 tx=0x7fd184009500 comp rx=0 tx=0).stop 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 msgr2=0x7fd1940b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0017e0 tx=0x7fd17c006360 comp rx=0 tx=0).stop 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 shutdown_connections 2024-07-17T08:49:19.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd170076cd0 0x7fd170079190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd194158fb0 0x7fd194159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd194071830 0x7fd1941f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 --2- 172.21.15.44:0/2782984011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178004de0 0x7fd1940b3010 unknown :-1 s=CLOSED pgs=3495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 >> 172.21.15.44:0/2782984011 conn(0x7fd1940b0fb0 msgr2=0x7fd19415d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 shutdown_connections 2024-07-17T08:49:19.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.103+0000 7fd198e00640 1 -- 172.21.15.44:0/2782984011 wait complete. 2024-07-17T08:49:19.124 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-07-17T08:49:17.933331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594165,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":642,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4725,"ondisk_log_size":4725,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546552,"kb_used_data":38920,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428168,"statfs":{"total":773094113280,"available":772534444032,"internally_reserved":0,"allocated":39854080,"data_stored":22551474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004212"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884745+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:47:07.884745+0000","last_peered":"2024-07-17T08:47:07.884745+0000","last_clean":"2024-07-17T08:47:07.884745+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:47:07.884745+0000","last_undegraded":"2024-07-17T08:47:07.884745+0000","last_fullsized":"2024-07-17T08:47:07.884745+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890201+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:47:07.890201+0000","last_peered":"2024-07-17T08:47:07.890201+0000","last_clean":"2024-07-17T08:47:07.890201+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:47:07.890201+0000","last_undegraded":"2024-07-17T08:47:07.890201+0000","last_fullsized":"2024-07-17T08:47:07.890201+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:43:50.699357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890224+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.890224+0000","last_peered":"2024-07-17T08:47:07.890224+0000","last_clean":"2024-07-17T08:47:07.890224+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.890224+0000","last_undegraded":"2024-07-17T08:47:07.890224+0000","last_fullsized":"2024-07-17T08:47:07.890224+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888208+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.888208+0000","last_peered":"2024-07-17T08:47:07.888208+0000","last_clean":"2024-07-17T08:47:07.888208+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.888208+0000","last_undegraded":"2024-07-17T08:47:07.888208+0000","last_fullsized":"2024-07-17T08:47:07.888208+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221716+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.221716+0000","last_peered":"2024-07-17T08:47:07.221716+0000","last_clean":"2024-07-17T08:47:07.221716+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.221716+0000","last_undegraded":"2024-07-17T08:47:07.221716+0000","last_fullsized":"2024-07-17T08:47:07.221716+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:03.150509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273776+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.273776+0000","last_peered":"2024-07-17T08:47:07.273776+0000","last_clean":"2024-07-17T08:47:07.273776+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.273776+0000","last_undegraded":"2024-07-17T08:47:07.273776+0000","last_fullsized":"2024-07-17T08:47:07.273776+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221537+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.221537+0000","last_peered":"2024-07-17T08:47:07.221537+0000","last_clean":"2024-07-17T08:47:07.221537+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.221537+0000","last_undegraded":"2024-07-17T08:47:07.221537+0000","last_fullsized":"2024-07-17T08:47:07.221537+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:08.644619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273038+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.273038+0000","last_peered":"2024-07-17T08:47:07.273038+0000","last_clean":"2024-07-17T08:47:07.273038+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.273038+0000","last_undegraded":"2024-07-17T08:47:07.273038+0000","last_fullsized":"2024-07-17T08:47:07.273038+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271623+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.271623+0000","last_peered":"2024-07-17T08:47:07.271623+0000","last_clean":"2024-07-17T08:47:07.271623+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.271623+0000","last_undegraded":"2024-07-17T08:47:07.271623+0000","last_fullsized":"2024-07-17T08:47:07.271623+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221733+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.221733+0000","last_peered":"2024-07-17T08:47:07.221733+0000","last_clean":"2024-07-17T08:47:07.221733+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.221733+0000","last_undegraded":"2024-07-17T08:47:07.221733+0000","last_fullsized":"2024-07-17T08:47:07.221733+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:19.898818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274204+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.274204+0000","last_peered":"2024-07-17T08:47:07.274204+0000","last_clean":"2024-07-17T08:47:07.274204+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.274204+0000","last_undegraded":"2024-07-17T08:47:07.274204+0000","last_fullsized":"2024-07-17T08:47:07.274204+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:29.623868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141167+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.141167+0000","last_peered":"2024-07-17T08:47:07.141167+0000","last_clean":"2024-07-17T08:47:07.141167+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.141167+0000","last_undegraded":"2024-07-17T08:47:07.141167+0000","last_fullsized":"2024-07-17T08:47:07.141167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144133+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.144133+0000","last_peered":"2024-07-17T08:47:07.144133+0000","last_clean":"2024-07-17T08:47:07.144133+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.144133+0000","last_undegraded":"2024-07-17T08:47:07.144133+0000","last_fullsized":"2024-07-17T08:47:07.144133+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221396+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:47:07.221396+0000","last_peered":"2024-07-17T08:47:07.221396+0000","last_clean":"2024-07-17T08:47:07.221396+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:47:07.221396+0000","last_undegraded":"2024-07-17T08:47:07.221396+0000","last_fullsized":"2024-07-17T08:47:07.221396+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:00:09.543515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140850+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.140850+0000","last_peered":"2024-07-17T08:47:07.140850+0000","last_clean":"2024-07-17T08:47:07.140850+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.140850+0000","last_undegraded":"2024-07-17T08:47:07.140850+0000","last_fullsized":"2024-07-17T08:47:07.140850+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140398+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.140398+0000","last_peered":"2024-07-17T08:47:07.140398+0000","last_clean":"2024-07-17T08:47:07.140398+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.140398+0000","last_undegraded":"2024-07-17T08:47:07.140398+0000","last_fullsized":"2024-07-17T08:47:07.140398+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888113+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.888113+0000","last_peered":"2024-07-17T08:47:07.888113+0000","last_clean":"2024-07-17T08:47:07.888113+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.888113+0000","last_undegraded":"2024-07-17T08:47:07.888113+0000","last_fullsized":"2024-07-17T08:47:07.888113+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889751+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.889751+0000","last_peered":"2024-07-17T08:47:07.889751+0000","last_clean":"2024-07-17T08:47:07.889751+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.889751+0000","last_undegraded":"2024-07-17T08:47:07.889751+0000","last_fullsized":"2024-07-17T08:47:07.889751+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869495+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.869495+0000","last_peered":"2024-07-17T08:47:07.869495+0000","last_clean":"2024-07-17T08:47:07.869495+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.869495+0000","last_undegraded":"2024-07-17T08:47:07.869495+0000","last_fullsized":"2024-07-17T08:47:07.869495+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886129+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.886129+0000","last_peered":"2024-07-17T08:47:07.886129+0000","last_clean":"2024-07-17T08:47:07.886129+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.886129+0000","last_undegraded":"2024-07-17T08:47:07.886129+0000","last_fullsized":"2024-07-17T08:47:07.886129+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886055+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.886055+0000","last_peered":"2024-07-17T08:47:07.886055+0000","last_clean":"2024-07-17T08:47:07.886055+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.886055+0000","last_undegraded":"2024-07-17T08:47:07.886055+0000","last_fullsized":"2024-07-17T08:47:07.886055+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886018+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.886018+0000","last_peered":"2024-07-17T08:47:07.886018+0000","last_clean":"2024-07-17T08:47:07.886018+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.886018+0000","last_undegraded":"2024-07-17T08:47:07.886018+0000","last_fullsized":"2024-07-17T08:47:07.886018+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885983+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.885983+0000","last_peered":"2024-07-17T08:47:07.885983+0000","last_clean":"2024-07-17T08:47:07.885983+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.885983+0000","last_undegraded":"2024-07-17T08:47:07.885983+0000","last_fullsized":"2024-07-17T08:47:07.885983+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890062+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.890062+0000","last_peered":"2024-07-17T08:47:07.890062+0000","last_clean":"2024-07-17T08:47:07.890062+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.890062+0000","last_undegraded":"2024-07-17T08:47:07.890062+0000","last_fullsized":"2024-07-17T08:47:07.890062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889946+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.889946+0000","last_peered":"2024-07-17T08:47:07.889946+0000","last_clean":"2024-07-17T08:47:07.889946+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.889946+0000","last_undegraded":"2024-07-17T08:47:07.889946+0000","last_fullsized":"2024-07-17T08:47:07.889946+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221117+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.221117+0000","last_peered":"2024-07-17T08:47:07.221117+0000","last_clean":"2024-07-17T08:47:07.221117+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.221117+0000","last_undegraded":"2024-07-17T08:47:07.221117+0000","last_fullsized":"2024-07-17T08:47:07.221117+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:53.930624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":19,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886272+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.886272+0000","last_peered":"2024-07-17T08:47:07.886272+0000","last_clean":"2024-07-17T08:47:07.886272+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.886272+0000","last_undegraded":"2024-07-17T08:47:07.886272+0000","last_fullsized":"2024-07-17T08:47:07.886272+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890219+0000","last_change":"2024-07-17T08:47:05.129655+0000","last_active":"2024-07-17T08:47:07.890219+0000","last_peered":"2024-07-17T08:47:07.890219+0000","last_clean":"2024-07-17T08:47:07.890219+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:47:07.890219+0000","last_undegraded":"2024-07-17T08:47:07.890219+0000","last_fullsized":"2024-07-17T08:47:07.890219+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:09:03.283870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272723+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.272723+0000","last_peered":"2024-07-17T08:47:07.272723+0000","last_clean":"2024-07-17T08:47:07.272723+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.272723+0000","last_undegraded":"2024-07-17T08:47:07.272723+0000","last_fullsized":"2024-07-17T08:47:07.272723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:17.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273634+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.273634+0000","last_peered":"2024-07-17T08:47:07.273634+0000","last_clean":"2024-07-17T08:47:07.273634+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.273634+0000","last_undegraded":"2024-07-17T08:47:07.273634+0000","last_fullsized":"2024-07-17T08:47:07.273634+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272935+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.272935+0000","last_peered":"2024-07-17T08:47:07.272935+0000","last_clean":"2024-07-17T08:47:07.272935+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.272935+0000","last_undegraded":"2024-07-17T08:47:07.272935+0000","last_fullsized":"2024-07-17T08:47:07.272935+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221435+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.221435+0000","last_peered":"2024-07-17T08:47:07.221435+0000","last_clean":"2024-07-17T08:47:07.221435+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.221435+0000","last_undegraded":"2024-07-17T08:47:07.221435+0000","last_fullsized":"2024-07-17T08:47:07.221435+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:50.145513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221353+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.221353+0000","last_peered":"2024-07-17T08:47:07.221353+0000","last_clean":"2024-07-17T08:47:07.221353+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.221353+0000","last_undegraded":"2024-07-17T08:47:07.221353+0000","last_fullsized":"2024-07-17T08:47:07.221353+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:50.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889720+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.889720+0000","last_peered":"2024-07-17T08:47:07.889720+0000","last_clean":"2024-07-17T08:47:07.889720+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.889720+0000","last_undegraded":"2024-07-17T08:47:07.889720+0000","last_fullsized":"2024-07-17T08:47:07.889720+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271594+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.271594+0000","last_peered":"2024-07-17T08:47:07.271594+0000","last_clean":"2024-07-17T08:47:07.271594+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.271594+0000","last_undegraded":"2024-07-17T08:47:07.271594+0000","last_fullsized":"2024-07-17T08:47:07.271594+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869512+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.869512+0000","last_peered":"2024-07-17T08:47:07.869512+0000","last_clean":"2024-07-17T08:47:07.869512+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.869512+0000","last_undegraded":"2024-07-17T08:47:07.869512+0000","last_fullsized":"2024-07-17T08:47:07.869512+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272214+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.272214+0000","last_peered":"2024-07-17T08:47:07.272214+0000","last_clean":"2024-07-17T08:47:07.272214+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.272214+0000","last_undegraded":"2024-07-17T08:47:07.272214+0000","last_fullsized":"2024-07-17T08:47:07.272214+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"379'10","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:15.803589+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:49:15.803589+0000","last_peered":"2024-07-17T08:49:15.803589+0000","last_clean":"2024-07-17T08:49:15.803589+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:49:15.803589+0000","last_undegraded":"2024-07-17T08:49:15.803589+0000","last_fullsized":"2024-07-17T08:49:15.803589+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221401+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.221401+0000","last_peered":"2024-07-17T08:47:07.221401+0000","last_clean":"2024-07-17T08:47:07.221401+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.221401+0000","last_undegraded":"2024-07-17T08:47:07.221401+0000","last_fullsized":"2024-07-17T08:47:07.221401+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:57.532859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889969+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:47:07.889969+0000","last_peered":"2024-07-17T08:47:07.889969+0000","last_clean":"2024-07-17T08:47:07.889969+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:47:07.889969+0000","last_undegraded":"2024-07-17T08:47:07.889969+0000","last_fullsized":"2024-07-17T08:47:07.889969+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:52.365125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274195+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:47:07.274195+0000","last_peered":"2024-07-17T08:47:07.274195+0000","last_clean":"2024-07-17T08:47:07.274195+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:47:07.274195+0000","last_undegraded":"2024-07-17T08:47:07.274195+0000","last_fullsized":"2024-07-17T08:47:07.274195+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885960+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.885960+0000","last_peered":"2024-07-17T08:47:07.885960+0000","last_clean":"2024-07-17T08:47:07.885960+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.885960+0000","last_undegraded":"2024-07-17T08:47:07.885960+0000","last_fullsized":"2024-07-17T08:47:07.885960+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144016+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.144016+0000","last_peered":"2024-07-17T08:47:07.144016+0000","last_clean":"2024-07-17T08:47:07.144016+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.144016+0000","last_undegraded":"2024-07-17T08:47:07.144016+0000","last_fullsized":"2024-07-17T08:47:07.144016+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890124+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.890124+0000","last_peered":"2024-07-17T08:47:07.890124+0000","last_clean":"2024-07-17T08:47:07.890124+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.890124+0000","last_undegraded":"2024-07-17T08:47:07.890124+0000","last_fullsized":"2024-07-17T08:47:07.890124+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141801+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.141801+0000","last_peered":"2024-07-17T08:47:07.141801+0000","last_clean":"2024-07-17T08:47:07.141801+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.141801+0000","last_undegraded":"2024-07-17T08:47:07.141801+0000","last_fullsized":"2024-07-17T08:47:07.141801+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275409+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.275409+0000","last_peered":"2024-07-17T08:47:07.275409+0000","last_clean":"2024-07-17T08:47:07.275409+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.275409+0000","last_undegraded":"2024-07-17T08:47:07.275409+0000","last_fullsized":"2024-07-17T08:47:07.275409+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273190+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.273190+0000","last_peered":"2024-07-17T08:47:07.273190+0000","last_clean":"2024-07-17T08:47:07.273190+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.273190+0000","last_undegraded":"2024-07-17T08:47:07.273190+0000","last_fullsized":"2024-07-17T08:47:07.273190+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141198+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.141198+0000","last_peered":"2024-07-17T08:47:07.141198+0000","last_clean":"2024-07-17T08:47:07.141198+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.141198+0000","last_undegraded":"2024-07-17T08:47:07.141198+0000","last_fullsized":"2024-07-17T08:47:07.141198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221468+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:47:07.221468+0000","last_peered":"2024-07-17T08:47:07.221468+0000","last_clean":"2024-07-17T08:47:07.221468+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:47:07.221468+0000","last_undegraded":"2024-07-17T08:47:07.221468+0000","last_fullsized":"2024-07-17T08:47:07.221468+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:37.618880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273673+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.273673+0000","last_peered":"2024-07-17T08:47:07.273673+0000","last_clean":"2024-07-17T08:47:07.273673+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.273673+0000","last_undegraded":"2024-07-17T08:47:07.273673+0000","last_fullsized":"2024-07-17T08:47:07.273673+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886001+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.886001+0000","last_peered":"2024-07-17T08:47:07.886001+0000","last_clean":"2024-07-17T08:47:07.886001+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.886001+0000","last_undegraded":"2024-07-17T08:47:07.886001+0000","last_fullsized":"2024-07-17T08:47:07.886001+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275184+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.275184+0000","last_peered":"2024-07-17T08:47:07.275184+0000","last_clean":"2024-07-17T08:47:07.275184+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.275184+0000","last_undegraded":"2024-07-17T08:47:07.275184+0000","last_fullsized":"2024-07-17T08:47:07.275184+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889551+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.889551+0000","last_peered":"2024-07-17T08:47:07.889551+0000","last_clean":"2024-07-17T08:47:07.889551+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.889551+0000","last_undegraded":"2024-07-17T08:47:07.889551+0000","last_fullsized":"2024-07-17T08:47:07.889551+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144992+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.144992+0000","last_peered":"2024-07-17T08:47:07.144992+0000","last_clean":"2024-07-17T08:47:07.144992+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.144992+0000","last_undegraded":"2024-07-17T08:47:07.144992+0000","last_fullsized":"2024-07-17T08:47:07.144992+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885909+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.885909+0000","last_peered":"2024-07-17T08:47:07.885909+0000","last_clean":"2024-07-17T08:47:07.885909+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.885909+0000","last_undegraded":"2024-07-17T08:47:07.885909+0000","last_fullsized":"2024-07-17T08:47:07.885909+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888132+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.888132+0000","last_peered":"2024-07-17T08:47:07.888132+0000","last_clean":"2024-07-17T08:47:07.888132+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.888132+0000","last_undegraded":"2024-07-17T08:47:07.888132+0000","last_fullsized":"2024-07-17T08:47:07.888132+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890304+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.890304+0000","last_peered":"2024-07-17T08:47:07.890304+0000","last_clean":"2024-07-17T08:47:07.890304+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.890304+0000","last_undegraded":"2024-07-17T08:47:07.890304+0000","last_fullsized":"2024-07-17T08:47:07.890304+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877385+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.877385+0000","last_peered":"2024-07-17T08:47:07.877385+0000","last_clean":"2024-07-17T08:47:07.877385+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.877385+0000","last_undegraded":"2024-07-17T08:47:07.877385+0000","last_fullsized":"2024-07-17T08:47:07.877385+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"379'3","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:16.142642+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:49:16.142642+0000","last_peered":"2024-07-17T08:49:16.142642+0000","last_clean":"2024-07-17T08:49:16.142642+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:49:16.142642+0000","last_undegraded":"2024-07-17T08:49:16.142642+0000","last_fullsized":"2024-07-17T08:49:16.142642+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884480+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:47:07.884480+0000","last_peered":"2024-07-17T08:47:07.884480+0000","last_clean":"2024-07-17T08:47:07.884480+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:47:07.884480+0000","last_undegraded":"2024-07-17T08:47:07.884480+0000","last_fullsized":"2024-07-17T08:47:07.884480+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877661+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.877661+0000","last_peered":"2024-07-17T08:47:07.877661+0000","last_clean":"2024-07-17T08:47:07.877661+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.877661+0000","last_undegraded":"2024-07-17T08:47:07.877661+0000","last_fullsized":"2024-07-17T08:47:07.877661+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890188+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.890188+0000","last_peered":"2024-07-17T08:47:07.890188+0000","last_clean":"2024-07-17T08:47:07.890188+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.890188+0000","last_undegraded":"2024-07-17T08:47:07.890188+0000","last_fullsized":"2024-07-17T08:47:07.890188+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890168+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:47:07.890168+0000","last_peered":"2024-07-17T08:47:07.890168+0000","last_clean":"2024-07-17T08:47:07.890168+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:47:07.890168+0000","last_undegraded":"2024-07-17T08:47:07.890168+0000","last_fullsized":"2024-07-17T08:47:07.890168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:28.605364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889649+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.889649+0000","last_peered":"2024-07-17T08:47:07.889649+0000","last_clean":"2024-07-17T08:47:07.889649+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.889649+0000","last_undegraded":"2024-07-17T08:47:07.889649+0000","last_fullsized":"2024-07-17T08:47:07.889649+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877509+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.877509+0000","last_peered":"2024-07-17T08:47:07.877509+0000","last_clean":"2024-07-17T08:47:07.877509+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.877509+0000","last_undegraded":"2024-07-17T08:47:07.877509+0000","last_fullsized":"2024-07-17T08:47:07.877509+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877625+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.877625+0000","last_peered":"2024-07-17T08:47:07.877625+0000","last_clean":"2024-07-17T08:47:07.877625+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.877625+0000","last_undegraded":"2024-07-17T08:47:07.877625+0000","last_fullsized":"2024-07-17T08:47:07.877625+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877473+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.877473+0000","last_peered":"2024-07-17T08:47:07.877473+0000","last_clean":"2024-07-17T08:47:07.877473+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.877473+0000","last_undegraded":"2024-07-17T08:47:07.877473+0000","last_fullsized":"2024-07-17T08:47:07.877473+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886198+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.886198+0000","last_peered":"2024-07-17T08:47:07.886198+0000","last_clean":"2024-07-17T08:47:07.886198+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.886198+0000","last_undegraded":"2024-07-17T08:47:07.886198+0000","last_fullsized":"2024-07-17T08:47:07.886198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877423+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.877423+0000","last_peered":"2024-07-17T08:47:07.877423+0000","last_clean":"2024-07-17T08:47:07.877423+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.877423+0000","last_undegraded":"2024-07-17T08:47:07.877423+0000","last_fullsized":"2024-07-17T08:47:07.877423+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886235+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.886235+0000","last_peered":"2024-07-17T08:47:07.886235+0000","last_clean":"2024-07-17T08:47:07.886235+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.886235+0000","last_undegraded":"2024-07-17T08:47:07.886235+0000","last_fullsized":"2024-07-17T08:47:07.886235+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890029+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.890029+0000","last_peered":"2024-07-17T08:47:07.890029+0000","last_clean":"2024-07-17T08:47:07.890029+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.890029+0000","last_undegraded":"2024-07-17T08:47:07.890029+0000","last_fullsized":"2024-07-17T08:47:07.890029+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6212,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272409+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272409+0000","last_peered":"2024-07-17T08:47:07.272409+0000","last_clean":"2024-07-17T08:47:07.272409+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272409+0000","last_undegraded":"2024-07-17T08:47:07.272409+0000","last_fullsized":"2024-07-17T08:47:07.272409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":3797,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543498,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80904,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290936,"statfs":{"total":96636764160,"available":96553918464,"internally_reserved":0,"allocated":5427200,"data_stored":3261377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510794,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76152,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295688,"statfs":{"total":96636764160,"available":96558784512,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80324,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291516,"statfs":{"total":96636764160,"available":96554512384,"internally_reserved":0,"allocated":4845568,"data_stored":2673484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510794,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4841472,"data_stored":2673414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:49:19.129 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-17T08:49:19.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.377+0000 7f4df5a00640 1 -- 172.21.15.44:0/2111978994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df01522c0 msgr2=0x7f4df00ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.377+0000 7f4df5a00640 1 --2- 172.21.15.44:0/2111978994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df01522c0 0x7f4df00ad790 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009830 tx=0x7f4de002a510 comp rx=0 tx=0).stop 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 -- 172.21.15.44:0/2111978994 shutdown_connections 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 --2- 172.21.15.44:0/2111978994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df00adcd0 0x7f4df00ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 --2- 172.21.15.44:0/2111978994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df01522c0 0x7f4df00ad790 unknown :-1 s=CLOSED pgs=3497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 -- 172.21.15.44:0/2111978994 >> 172.21.15.44:0/2111978994 conn(0x7f4df00b0fb0 msgr2=0x7f4df00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:19.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 -- 172.21.15.44:0/2111978994 shutdown_connections 2024-07-17T08:49:19.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.378+0000 7f4df5a00640 1 -- 172.21.15.44:0/2111978994 wait complete. 2024-07-17T08:49:19.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.379+0000 7f4df5a00640 1 Processor -- start 2024-07-17T08:49:19.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 -- start start 2024-07-17T08:49:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 0x7f4df0147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 0x7f4df015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4df015f5a0 con 0x7f4df00adcd0 2024-07-17T08:49:19.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4df00ce8b0 con 0x7f4df00ac760 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.380+0000 7f4df5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4df015f720 con 0x7f4df01522c0 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33410/0 (socket says 172.21.15.44:33410) 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 -- 172.21.15.44:0/1448389514 learned_addr learned my addr 172.21.15.44:0/1448389514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 -- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 msgr2=0x7f4df015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4deea00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 0x7f4df0147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4def400640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 0x7f4df015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 0x7f4df015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 -- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 msgr2=0x7f4df0147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 0x7f4df0147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4de7e00640 1 -- 172.21.15.44:0/1448389514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df00c93b0 con 0x7f4df00adcd0 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4deea00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 0x7f4df0147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.381+0000 7f4def400640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 0x7f4df015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4de7e00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b450 tx=0x7f4dd800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:19.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8015020 con 0x7f4df00adcd0 2024-07-17T08:49:19.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dd8011980 con 0x7f4df00adcd0 2024-07-17T08:49:19.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011dd0 con 0x7f4df00adcd0 2024-07-17T08:49:19.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df00c9690 con 0x7f4df00adcd0 2024-07-17T08:49:19.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.382+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df00b2730 con 0x7f4df00adcd0 2024-07-17T08:49:19.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.383+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df015c4f0 con 0x7f4df00adcd0 2024-07-17T08:49:19.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.385+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f4dd8018020 con 0x7f4df00adcd0 2024-07-17T08:49:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.386+0000 7f4de6a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 0x7f4dcc079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.386+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f4dd8098ad0 con 0x7f4df00adcd0 2024-07-17T08:49:19.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.386+0000 7f4deea00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 0x7f4dcc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.388+0000 7f4deea00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 0x7f4dcc079190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4de0034000 tx=0x7f4de0004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:19.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.391+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f4df015c4f0 con 0x7f4df00adcd0 2024-07-17T08:49:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.599+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4df015fc40 con 0x7f4dcc076cd0 2024-07-17T08:49:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.612+0000 7f4de6a00640 1 -- 172.21.15.44:0/1448389514 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252429 (secure 0 0 0) 0x7f4df015fc40 con 0x7f4dcc076cd0 2024-07-17T08:49:19.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:19.620 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:49:19.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 msgr2=0x7f4dcc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 0x7f4dcc079190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4de0034000 tx=0x7f4de0004730 comp rx=0 tx=0).stop 2024-07-17T08:49:19.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 msgr2=0x7f4df014b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b450 tx=0x7f4dd800b910 comp rx=0 tx=0).stop 2024-07-17T08:49:19.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 shutdown_connections 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4dcc076cd0 0x7f4dcc079190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4df01522c0 0x7f4df015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df00adcd0 0x7f4df014b210 unknown :-1 s=CLOSED pgs=3498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 --2- 172.21.15.44:0/1448389514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4df00ac760 0x7f4df0147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.619+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 >> 172.21.15.44:0/1448389514 conn(0x7f4df00b0fb0 msgr2=0x7f4df006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.620+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 shutdown_connections 2024-07-17T08:49:19.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.620+0000 7f4df5a00640 1 -- 172.21.15.44:0/1448389514 wait complete. 2024-07-17T08:49:19.642 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-07-17T08:49:17.933331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594165,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":642,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4725,"ondisk_log_size":4725,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546552,"kb_used_data":38920,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428168,"statfs":{"total":773094113280,"available":772534444032,"internally_reserved":0,"allocated":39854080,"data_stored":22551474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004212"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884745+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:47:07.884745+0000","last_peered":"2024-07-17T08:47:07.884745+0000","last_clean":"2024-07-17T08:47:07.884745+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:47:07.884745+0000","last_undegraded":"2024-07-17T08:47:07.884745+0000","last_fullsized":"2024-07-17T08:47:07.884745+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890201+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:47:07.890201+0000","last_peered":"2024-07-17T08:47:07.890201+0000","last_clean":"2024-07-17T08:47:07.890201+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:47:07.890201+0000","last_undegraded":"2024-07-17T08:47:07.890201+0000","last_fullsized":"2024-07-17T08:47:07.890201+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:43:50.699357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890224+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.890224+0000","last_peered":"2024-07-17T08:47:07.890224+0000","last_clean":"2024-07-17T08:47:07.890224+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.890224+0000","last_undegraded":"2024-07-17T08:47:07.890224+0000","last_fullsized":"2024-07-17T08:47:07.890224+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888208+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.888208+0000","last_peered":"2024-07-17T08:47:07.888208+0000","last_clean":"2024-07-17T08:47:07.888208+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.888208+0000","last_undegraded":"2024-07-17T08:47:07.888208+0000","last_fullsized":"2024-07-17T08:47:07.888208+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221716+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.221716+0000","last_peered":"2024-07-17T08:47:07.221716+0000","last_clean":"2024-07-17T08:47:07.221716+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.221716+0000","last_undegraded":"2024-07-17T08:47:07.221716+0000","last_fullsized":"2024-07-17T08:47:07.221716+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:46:03.150509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273776+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.273776+0000","last_peered":"2024-07-17T08:47:07.273776+0000","last_clean":"2024-07-17T08:47:07.273776+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.273776+0000","last_undegraded":"2024-07-17T08:47:07.273776+0000","last_fullsized":"2024-07-17T08:47:07.273776+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221537+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.221537+0000","last_peered":"2024-07-17T08:47:07.221537+0000","last_clean":"2024-07-17T08:47:07.221537+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.221537+0000","last_undegraded":"2024-07-17T08:47:07.221537+0000","last_fullsized":"2024-07-17T08:47:07.221537+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:08.644619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273038+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.273038+0000","last_peered":"2024-07-17T08:47:07.273038+0000","last_clean":"2024-07-17T08:47:07.273038+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.273038+0000","last_undegraded":"2024-07-17T08:47:07.273038+0000","last_fullsized":"2024-07-17T08:47:07.273038+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271623+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.271623+0000","last_peered":"2024-07-17T08:47:07.271623+0000","last_clean":"2024-07-17T08:47:07.271623+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.271623+0000","last_undegraded":"2024-07-17T08:47:07.271623+0000","last_fullsized":"2024-07-17T08:47:07.271623+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221733+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.221733+0000","last_peered":"2024-07-17T08:47:07.221733+0000","last_clean":"2024-07-17T08:47:07.221733+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.221733+0000","last_undegraded":"2024-07-17T08:47:07.221733+0000","last_fullsized":"2024-07-17T08:47:07.221733+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:19.898818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274204+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.274204+0000","last_peered":"2024-07-17T08:47:07.274204+0000","last_clean":"2024-07-17T08:47:07.274204+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.274204+0000","last_undegraded":"2024-07-17T08:47:07.274204+0000","last_fullsized":"2024-07-17T08:47:07.274204+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:29.623868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141167+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.141167+0000","last_peered":"2024-07-17T08:47:07.141167+0000","last_clean":"2024-07-17T08:47:07.141167+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.141167+0000","last_undegraded":"2024-07-17T08:47:07.141167+0000","last_fullsized":"2024-07-17T08:47:07.141167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144133+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.144133+0000","last_peered":"2024-07-17T08:47:07.144133+0000","last_clean":"2024-07-17T08:47:07.144133+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.144133+0000","last_undegraded":"2024-07-17T08:47:07.144133+0000","last_fullsized":"2024-07-17T08:47:07.144133+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221396+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:47:07.221396+0000","last_peered":"2024-07-17T08:47:07.221396+0000","last_clean":"2024-07-17T08:47:07.221396+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:47:07.221396+0000","last_undegraded":"2024-07-17T08:47:07.221396+0000","last_fullsized":"2024-07-17T08:47:07.221396+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:00:09.543515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140850+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.140850+0000","last_peered":"2024-07-17T08:47:07.140850+0000","last_clean":"2024-07-17T08:47:07.140850+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.140850+0000","last_undegraded":"2024-07-17T08:47:07.140850+0000","last_fullsized":"2024-07-17T08:47:07.140850+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140398+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.140398+0000","last_peered":"2024-07-17T08:47:07.140398+0000","last_clean":"2024-07-17T08:47:07.140398+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.140398+0000","last_undegraded":"2024-07-17T08:47:07.140398+0000","last_fullsized":"2024-07-17T08:47:07.140398+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888113+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.888113+0000","last_peered":"2024-07-17T08:47:07.888113+0000","last_clean":"2024-07-17T08:47:07.888113+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.888113+0000","last_undegraded":"2024-07-17T08:47:07.888113+0000","last_fullsized":"2024-07-17T08:47:07.888113+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889751+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.889751+0000","last_peered":"2024-07-17T08:47:07.889751+0000","last_clean":"2024-07-17T08:47:07.889751+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.889751+0000","last_undegraded":"2024-07-17T08:47:07.889751+0000","last_fullsized":"2024-07-17T08:47:07.889751+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869495+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.869495+0000","last_peered":"2024-07-17T08:47:07.869495+0000","last_clean":"2024-07-17T08:47:07.869495+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.869495+0000","last_undegraded":"2024-07-17T08:47:07.869495+0000","last_fullsized":"2024-07-17T08:47:07.869495+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886129+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.886129+0000","last_peered":"2024-07-17T08:47:07.886129+0000","last_clean":"2024-07-17T08:47:07.886129+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.886129+0000","last_undegraded":"2024-07-17T08:47:07.886129+0000","last_fullsized":"2024-07-17T08:47:07.886129+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886055+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.886055+0000","last_peered":"2024-07-17T08:47:07.886055+0000","last_clean":"2024-07-17T08:47:07.886055+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.886055+0000","last_undegraded":"2024-07-17T08:47:07.886055+0000","last_fullsized":"2024-07-17T08:47:07.886055+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886018+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.886018+0000","last_peered":"2024-07-17T08:47:07.886018+0000","last_clean":"2024-07-17T08:47:07.886018+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.886018+0000","last_undegraded":"2024-07-17T08:47:07.886018+0000","last_fullsized":"2024-07-17T08:47:07.886018+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885983+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.885983+0000","last_peered":"2024-07-17T08:47:07.885983+0000","last_clean":"2024-07-17T08:47:07.885983+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.885983+0000","last_undegraded":"2024-07-17T08:47:07.885983+0000","last_fullsized":"2024-07-17T08:47:07.885983+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890062+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.890062+0000","last_peered":"2024-07-17T08:47:07.890062+0000","last_clean":"2024-07-17T08:47:07.890062+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.890062+0000","last_undegraded":"2024-07-17T08:47:07.890062+0000","last_fullsized":"2024-07-17T08:47:07.890062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889946+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.889946+0000","last_peered":"2024-07-17T08:47:07.889946+0000","last_clean":"2024-07-17T08:47:07.889946+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.889946+0000","last_undegraded":"2024-07-17T08:47:07.889946+0000","last_fullsized":"2024-07-17T08:47:07.889946+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221117+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.221117+0000","last_peered":"2024-07-17T08:47:07.221117+0000","last_clean":"2024-07-17T08:47:07.221117+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.221117+0000","last_undegraded":"2024-07-17T08:47:07.221117+0000","last_fullsized":"2024-07-17T08:47:07.221117+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:57:53.930624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":19,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886272+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.886272+0000","last_peered":"2024-07-17T08:47:07.886272+0000","last_clean":"2024-07-17T08:47:07.886272+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.886272+0000","last_undegraded":"2024-07-17T08:47:07.886272+0000","last_fullsized":"2024-07-17T08:47:07.886272+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890219+0000","last_change":"2024-07-17T08:47:05.129655+0000","last_active":"2024-07-17T08:47:07.890219+0000","last_peered":"2024-07-17T08:47:07.890219+0000","last_clean":"2024-07-17T08:47:07.890219+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:47:07.890219+0000","last_undegraded":"2024-07-17T08:47:07.890219+0000","last_fullsized":"2024-07-17T08:47:07.890219+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:09:03.283870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272723+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.272723+0000","last_peered":"2024-07-17T08:47:07.272723+0000","last_clean":"2024-07-17T08:47:07.272723+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.272723+0000","last_undegraded":"2024-07-17T08:47:07.272723+0000","last_fullsized":"2024-07-17T08:47:07.272723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:25:17.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273634+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.273634+0000","last_peered":"2024-07-17T08:47:07.273634+0000","last_clean":"2024-07-17T08:47:07.273634+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.273634+0000","last_undegraded":"2024-07-17T08:47:07.273634+0000","last_fullsized":"2024-07-17T08:47:07.273634+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272935+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.272935+0000","last_peered":"2024-07-17T08:47:07.272935+0000","last_clean":"2024-07-17T08:47:07.272935+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.272935+0000","last_undegraded":"2024-07-17T08:47:07.272935+0000","last_fullsized":"2024-07-17T08:47:07.272935+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221435+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.221435+0000","last_peered":"2024-07-17T08:47:07.221435+0000","last_clean":"2024-07-17T08:47:07.221435+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.221435+0000","last_undegraded":"2024-07-17T08:47:07.221435+0000","last_fullsized":"2024-07-17T08:47:07.221435+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:57:50.145513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221353+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.221353+0000","last_peered":"2024-07-17T08:47:07.221353+0000","last_clean":"2024-07-17T08:47:07.221353+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.221353+0000","last_undegraded":"2024-07-17T08:47:07.221353+0000","last_fullsized":"2024-07-17T08:47:07.221353+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:58:50.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889720+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.889720+0000","last_peered":"2024-07-17T08:47:07.889720+0000","last_clean":"2024-07-17T08:47:07.889720+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.889720+0000","last_undegraded":"2024-07-17T08:47:07.889720+0000","last_fullsized":"2024-07-17T08:47:07.889720+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.271594+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.271594+0000","last_peered":"2024-07-17T08:47:07.271594+0000","last_clean":"2024-07-17T08:47:07.271594+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.271594+0000","last_undegraded":"2024-07-17T08:47:07.271594+0000","last_fullsized":"2024-07-17T08:47:07.271594+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869512+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.869512+0000","last_peered":"2024-07-17T08:47:07.869512+0000","last_clean":"2024-07-17T08:47:07.869512+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.869512+0000","last_undegraded":"2024-07-17T08:47:07.869512+0000","last_fullsized":"2024-07-17T08:47:07.869512+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272214+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.272214+0000","last_peered":"2024-07-17T08:47:07.272214+0000","last_clean":"2024-07-17T08:47:07.272214+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.272214+0000","last_undegraded":"2024-07-17T08:47:07.272214+0000","last_fullsized":"2024-07-17T08:47:07.272214+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"379'10","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:15.803589+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:49:15.803589+0000","last_peered":"2024-07-17T08:49:15.803589+0000","last_clean":"2024-07-17T08:49:15.803589+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:49:15.803589+0000","last_undegraded":"2024-07-17T08:49:15.803589+0000","last_fullsized":"2024-07-17T08:49:15.803589+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221401+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.221401+0000","last_peered":"2024-07-17T08:47:07.221401+0000","last_clean":"2024-07-17T08:47:07.221401+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.221401+0000","last_undegraded":"2024-07-17T08:47:07.221401+0000","last_fullsized":"2024-07-17T08:47:07.221401+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:57.532859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889969+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:47:07.889969+0000","last_peered":"2024-07-17T08:47:07.889969+0000","last_clean":"2024-07-17T08:47:07.889969+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:47:07.889969+0000","last_undegraded":"2024-07-17T08:47:07.889969+0000","last_fullsized":"2024-07-17T08:47:07.889969+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:38:52.365125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274195+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:47:07.274195+0000","last_peered":"2024-07-17T08:47:07.274195+0000","last_clean":"2024-07-17T08:47:07.274195+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:47:07.274195+0000","last_undegraded":"2024-07-17T08:47:07.274195+0000","last_fullsized":"2024-07-17T08:47:07.274195+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885960+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.885960+0000","last_peered":"2024-07-17T08:47:07.885960+0000","last_clean":"2024-07-17T08:47:07.885960+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.885960+0000","last_undegraded":"2024-07-17T08:47:07.885960+0000","last_fullsized":"2024-07-17T08:47:07.885960+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144016+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.144016+0000","last_peered":"2024-07-17T08:47:07.144016+0000","last_clean":"2024-07-17T08:47:07.144016+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.144016+0000","last_undegraded":"2024-07-17T08:47:07.144016+0000","last_fullsized":"2024-07-17T08:47:07.144016+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890124+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.890124+0000","last_peered":"2024-07-17T08:47:07.890124+0000","last_clean":"2024-07-17T08:47:07.890124+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.890124+0000","last_undegraded":"2024-07-17T08:47:07.890124+0000","last_fullsized":"2024-07-17T08:47:07.890124+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141801+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.141801+0000","last_peered":"2024-07-17T08:47:07.141801+0000","last_clean":"2024-07-17T08:47:07.141801+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.141801+0000","last_undegraded":"2024-07-17T08:47:07.141801+0000","last_fullsized":"2024-07-17T08:47:07.141801+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275409+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.275409+0000","last_peered":"2024-07-17T08:47:07.275409+0000","last_clean":"2024-07-17T08:47:07.275409+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.275409+0000","last_undegraded":"2024-07-17T08:47:07.275409+0000","last_fullsized":"2024-07-17T08:47:07.275409+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273190+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.273190+0000","last_peered":"2024-07-17T08:47:07.273190+0000","last_clean":"2024-07-17T08:47:07.273190+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.273190+0000","last_undegraded":"2024-07-17T08:47:07.273190+0000","last_fullsized":"2024-07-17T08:47:07.273190+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141198+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.141198+0000","last_peered":"2024-07-17T08:47:07.141198+0000","last_clean":"2024-07-17T08:47:07.141198+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.141198+0000","last_undegraded":"2024-07-17T08:47:07.141198+0000","last_fullsized":"2024-07-17T08:47:07.141198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.221468+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:47:07.221468+0000","last_peered":"2024-07-17T08:47:07.221468+0000","last_clean":"2024-07-17T08:47:07.221468+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:47:07.221468+0000","last_undegraded":"2024-07-17T08:47:07.221468+0000","last_fullsized":"2024-07-17T08:47:07.221468+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:37.618880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273673+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.273673+0000","last_peered":"2024-07-17T08:47:07.273673+0000","last_clean":"2024-07-17T08:47:07.273673+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.273673+0000","last_undegraded":"2024-07-17T08:47:07.273673+0000","last_fullsized":"2024-07-17T08:47:07.273673+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886001+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.886001+0000","last_peered":"2024-07-17T08:47:07.886001+0000","last_clean":"2024-07-17T08:47:07.886001+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.886001+0000","last_undegraded":"2024-07-17T08:47:07.886001+0000","last_fullsized":"2024-07-17T08:47:07.886001+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275184+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.275184+0000","last_peered":"2024-07-17T08:47:07.275184+0000","last_clean":"2024-07-17T08:47:07.275184+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.275184+0000","last_undegraded":"2024-07-17T08:47:07.275184+0000","last_fullsized":"2024-07-17T08:47:07.275184+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889551+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.889551+0000","last_peered":"2024-07-17T08:47:07.889551+0000","last_clean":"2024-07-17T08:47:07.889551+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.889551+0000","last_undegraded":"2024-07-17T08:47:07.889551+0000","last_fullsized":"2024-07-17T08:47:07.889551+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.144992+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.144992+0000","last_peered":"2024-07-17T08:47:07.144992+0000","last_clean":"2024-07-17T08:47:07.144992+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.144992+0000","last_undegraded":"2024-07-17T08:47:07.144992+0000","last_fullsized":"2024-07-17T08:47:07.144992+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885909+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.885909+0000","last_peered":"2024-07-17T08:47:07.885909+0000","last_clean":"2024-07-17T08:47:07.885909+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.885909+0000","last_undegraded":"2024-07-17T08:47:07.885909+0000","last_fullsized":"2024-07-17T08:47:07.885909+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888132+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.888132+0000","last_peered":"2024-07-17T08:47:07.888132+0000","last_clean":"2024-07-17T08:47:07.888132+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.888132+0000","last_undegraded":"2024-07-17T08:47:07.888132+0000","last_fullsized":"2024-07-17T08:47:07.888132+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890304+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.890304+0000","last_peered":"2024-07-17T08:47:07.890304+0000","last_clean":"2024-07-17T08:47:07.890304+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.890304+0000","last_undegraded":"2024-07-17T08:47:07.890304+0000","last_fullsized":"2024-07-17T08:47:07.890304+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877385+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.877385+0000","last_peered":"2024-07-17T08:47:07.877385+0000","last_clean":"2024-07-17T08:47:07.877385+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.877385+0000","last_undegraded":"2024-07-17T08:47:07.877385+0000","last_fullsized":"2024-07-17T08:47:07.877385+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"379'3","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:16.142642+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:49:16.142642+0000","last_peered":"2024-07-17T08:49:16.142642+0000","last_clean":"2024-07-17T08:49:16.142642+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:49:16.142642+0000","last_undegraded":"2024-07-17T08:49:16.142642+0000","last_fullsized":"2024-07-17T08:49:16.142642+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884480+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:47:07.884480+0000","last_peered":"2024-07-17T08:47:07.884480+0000","last_clean":"2024-07-17T08:47:07.884480+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:47:07.884480+0000","last_undegraded":"2024-07-17T08:47:07.884480+0000","last_fullsized":"2024-07-17T08:47:07.884480+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877661+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.877661+0000","last_peered":"2024-07-17T08:47:07.877661+0000","last_clean":"2024-07-17T08:47:07.877661+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.877661+0000","last_undegraded":"2024-07-17T08:47:07.877661+0000","last_fullsized":"2024-07-17T08:47:07.877661+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890188+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.890188+0000","last_peered":"2024-07-17T08:47:07.890188+0000","last_clean":"2024-07-17T08:47:07.890188+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.890188+0000","last_undegraded":"2024-07-17T08:47:07.890188+0000","last_fullsized":"2024-07-17T08:47:07.890188+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890168+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:47:07.890168+0000","last_peered":"2024-07-17T08:47:07.890168+0000","last_clean":"2024-07-17T08:47:07.890168+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:47:07.890168+0000","last_undegraded":"2024-07-17T08:47:07.890168+0000","last_fullsized":"2024-07-17T08:47:07.890168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:13:28.605364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889649+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.889649+0000","last_peered":"2024-07-17T08:47:07.889649+0000","last_clean":"2024-07-17T08:47:07.889649+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.889649+0000","last_undegraded":"2024-07-17T08:47:07.889649+0000","last_fullsized":"2024-07-17T08:47:07.889649+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877509+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.877509+0000","last_peered":"2024-07-17T08:47:07.877509+0000","last_clean":"2024-07-17T08:47:07.877509+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.877509+0000","last_undegraded":"2024-07-17T08:47:07.877509+0000","last_fullsized":"2024-07-17T08:47:07.877509+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877625+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.877625+0000","last_peered":"2024-07-17T08:47:07.877625+0000","last_clean":"2024-07-17T08:47:07.877625+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.877625+0000","last_undegraded":"2024-07-17T08:47:07.877625+0000","last_fullsized":"2024-07-17T08:47:07.877625+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877473+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.877473+0000","last_peered":"2024-07-17T08:47:07.877473+0000","last_clean":"2024-07-17T08:47:07.877473+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.877473+0000","last_undegraded":"2024-07-17T08:47:07.877473+0000","last_fullsized":"2024-07-17T08:47:07.877473+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886198+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.886198+0000","last_peered":"2024-07-17T08:47:07.886198+0000","last_clean":"2024-07-17T08:47:07.886198+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.886198+0000","last_undegraded":"2024-07-17T08:47:07.886198+0000","last_fullsized":"2024-07-17T08:47:07.886198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877423+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.877423+0000","last_peered":"2024-07-17T08:47:07.877423+0000","last_clean":"2024-07-17T08:47:07.877423+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.877423+0000","last_undegraded":"2024-07-17T08:47:07.877423+0000","last_fullsized":"2024-07-17T08:47:07.877423+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886235+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.886235+0000","last_peered":"2024-07-17T08:47:07.886235+0000","last_clean":"2024-07-17T08:47:07.886235+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.886235+0000","last_undegraded":"2024-07-17T08:47:07.886235+0000","last_fullsized":"2024-07-17T08:47:07.886235+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890029+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.890029+0000","last_peered":"2024-07-17T08:47:07.890029+0000","last_clean":"2024-07-17T08:47:07.890029+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.890029+0000","last_undegraded":"2024-07-17T08:47:07.890029+0000","last_fullsized":"2024-07-17T08:47:07.890029+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6212,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272409+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272409+0000","last_peered":"2024-07-17T08:47:07.272409+0000","last_clean":"2024-07-17T08:47:07.272409+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272409+0000","last_undegraded":"2024-07-17T08:47:07.272409+0000","last_fullsized":"2024-07-17T08:47:07.272409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":3797,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543498,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80904,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290936,"statfs":{"total":96636764160,"available":96553918464,"internally_reserved":0,"allocated":5427200,"data_stored":3261377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510794,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76152,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295688,"statfs":{"total":96636764160,"available":96558784512,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80324,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291516,"statfs":{"total":96636764160,"available":96554512384,"internally_reserved":0,"allocated":4845568,"data_stored":2673484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510794,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4841472,"data_stored":2673414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510793,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:49:19.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-17T08:49:19.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 -- 172.21.15.44:0/14376050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680cf540 msgr2=0x7fb4680b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 --2- 172.21.15.44:0/14376050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680cf540 0x7fb4680b3870 secure :-1 s=READY pgs=3499 cs=0 l=1 rev1=1 crypto rx=0x7fb46813e5e0 tx=0x7fb45802a4d0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 -- 172.21.15.44:0/14376050 shutdown_connections 2024-07-17T08:49:19.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 --2- 172.21.15.44:0/14376050 >> v2:172.21.15.175:3300/0 conn(0x7fb4680b3db0 0x7fb4680ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 --2- 172.21.15.44:0/14376050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680cf540 0x7fb4680b3870 unknown :-1 s=CLOSED pgs=3499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 -- 172.21.15.44:0/14376050 >> 172.21.15.44:0/14376050 conn(0x7fb4680b0e80 msgr2=0x7fb4680b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 -- 172.21.15.44:0/14376050 shutdown_connections 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.898+0000 7fb46d800640 1 -- 172.21.15.44:0/14376050 wait complete. 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.900+0000 7fb46d800640 1 Processor -- start 2024-07-17T08:49:19.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.900+0000 7fb46d800640 1 -- start start 2024-07-17T08:49:19.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 0x7fb468155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4680cf920 0x7fb46813bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680ae8c0 con 0x7fb4680b3db0 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680ae740 con 0x7fb4680cf540 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb46d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4680aea40 con 0x7fb4680cf920 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.901+0000 7fb467e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33418/0 (socket says 172.21.15.44:33418) 2024-07-17T08:49:19.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 -- 172.21.15.44:0/2821079254 learned_addr learned my addr 172.21.15.44:0/2821079254 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467400640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 0x7fb468155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb46ce00640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4680cf920 0x7fb46813bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 -- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4680cf920 msgr2=0x7fb46813bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4680cf920 0x7fb46813bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 -- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 msgr2=0x7fb468155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 0x7fb468155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:19.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 -- 172.21.15.44:0/2821079254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681f77a0 con 0x7fb4680b3db0 2024-07-17T08:49:19.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467400640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 0x7fb468155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.902+0000 7fb467e00640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb458003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.903+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45803f070 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.903+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4580047a0 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.903+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681f7a40 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.903+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458035030 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.903+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb468073d70 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.904+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4681412e0 con 0x7fb4680b3db0 2024-07-17T08:49:19.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.905+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb458042070 con 0x7fb4680b3db0 2024-07-17T08:49:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.906+0000 7fb45fe00640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 0x7fb448079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.906+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fb4580b97a0 con 0x7fb4680b3db0 2024-07-17T08:49:19.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.907+0000 7fb467400640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 0x7fb448079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:19.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.908+0000 7fb467400640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 0x7fb448079190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb454005fd0 tx=0x7fb45400b5f0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:19.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:19.911+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb458082640 con 0x7fb4680b3db0 2024-07-17T08:49:20.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.112+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb46806f030 con 0x7fb4680b3db0 2024-07-17T08:49:20.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.115+0000 7fb45fe00640 1 -- 172.21.15.44:0/2821079254 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v379) ==== 74+0+17911 (secure 0 0 0) 0x7fb458081f40 con 0x7fb4680b3db0 2024-07-17T08:49:20.118 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:20.118 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":379,"fsid":"4797a2db-a688-4e60-bfb8-d8a9e86b84a3","created":"2024-07-17T06:54:28.487789+0000","modified":"2024-07-17T08:47:07.124873+0000","last_up_change":"2024-07-17T06:54:42.202505+0000","last_in_change":"2024-07-17T06:54:33.536877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":52,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-17T06:54:41.754444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":51,"pool_name":"cephfs_metadata","create_time":"2024-07-17T08:47:03.554223+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"378","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":52,"pool_name":"cephfs_data","create_time":"2024-07-17T08:47:04.562957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"379","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c5762a28-1148-44aa-b365-c565843eb11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6815","nonce":429081112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6819","nonce":429081112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6825","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6827","nonce":429081112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":429081112},{"type":"v1","addr":"172.21.15.44:6822","nonce":429081112}]},"public_addr":"172.21.15.44:6815/429081112","cluster_addr":"172.21.15.44:6819/429081112","heartbeat_back_addr":"172.21.15.44:6827/429081112","heartbeat_front_addr":"172.21.15.44:6822/429081112","state":["exists","up"]},{"osd":1,"uuid":"72f19265-9816-4d99-a295-93f784f36b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6804","nonce":3861011633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6805","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6807","nonce":3861011633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6816","nonce":3861011633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":3861011633},{"type":"v1","addr":"172.21.15.44:6811","nonce":3861011633}]},"public_addr":"172.21.15.44:6804/3861011633","cluster_addr":"172.21.15.44:6807/3861011633","heartbeat_back_addr":"172.21.15.44:6816/3861011633","heartbeat_front_addr":"172.21.15.44:6811/3861011633","state":["exists","up"]},{"osd":2,"uuid":"45d9e4e1-1225-4892-a141-48e9874da9f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6823","nonce":2788335734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6826","nonce":2788335734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6831","nonce":2788335734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2788335734},{"type":"v1","addr":"172.21.15.44:6829","nonce":2788335734}]},"public_addr":"172.21.15.44:6823/2788335734","cluster_addr":"172.21.15.44:6826/2788335734","heartbeat_back_addr":"172.21.15.44:6831/2788335734","heartbeat_front_addr":"172.21.15.44:6829/2788335734","state":["exists","up"]},{"osd":3,"uuid":"c27fa4b5-eeda-4985-b474-37c30575bb4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6801","nonce":2705860470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6803","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6806","nonce":2705860470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6817","nonce":2705860470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2705860470},{"type":"v1","addr":"172.21.15.44:6810","nonce":2705860470}]},"public_addr":"172.21.15.44:6801/2705860470","cluster_addr":"172.21.15.44:6806/2705860470","heartbeat_back_addr":"172.21.15.44:6817/2705860470","heartbeat_front_addr":"172.21.15.44:6810/2705860470","state":["exists","up"]},{"osd":4,"uuid":"237302da-caf3-4c4b-a188-f73614d97790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6804","nonce":2954959872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6808","nonce":2954959872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6817","nonce":2954959872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":2954959872},{"type":"v1","addr":"172.21.15.175:6813","nonce":2954959872}]},"public_addr":"172.21.15.175:6804/2954959872","cluster_addr":"172.21.15.175:6808/2954959872","heartbeat_back_addr":"172.21.15.175:6817/2954959872","heartbeat_front_addr":"172.21.15.175:6813/2954959872","state":["exists","up"]},{"osd":5,"uuid":"893f49bf-b33f-4a70-b75a-0448b881671d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6819","nonce":1413905252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6821","nonce":1413905252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6825","nonce":1413905252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":1413905252},{"type":"v1","addr":"172.21.15.175:6823","nonce":1413905252}]},"public_addr":"172.21.15.175:6819/1413905252","cluster_addr":"172.21.15.175:6821/1413905252","heartbeat_back_addr":"172.21.15.175:6825/1413905252","heartbeat_front_addr":"172.21.15.175:6823/1413905252","state":["exists","up"]},{"osd":6,"uuid":"8b6c3614-66eb-4cf1-aa72-b6e9c5ef443b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6805","nonce":302428054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6809","nonce":302428054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6816","nonce":302428054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":302428054},{"type":"v1","addr":"172.21.15.175:6812","nonce":302428054}]},"public_addr":"172.21.15.175:6805/302428054","cluster_addr":"172.21.15.175:6809/302428054","heartbeat_back_addr":"172.21.15.175:6816/302428054","heartbeat_front_addr":"172.21.15.175:6812/302428054","state":["exists","up"]},{"osd":7,"uuid":"bd85a235-c448-4dcc-9f46-b3ab6fe9be64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":377,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6827","nonce":2516790419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6829","nonce":2516790419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6833","nonce":2516790419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2516790419},{"type":"v1","addr":"172.21.15.175:6831","nonce":2516790419}]},"public_addr":"172.21.15.175:6827/2516790419","cluster_addr":"172.21.15.175:6829/2516790419","heartbeat_back_addr":"172.21.15.175:6833/2516790419","heartbeat_front_addr":"172.21.15.175:6831/2516790419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.953741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.926472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.689430+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.904531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.668346+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.873498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.942131+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-17T06:54:38.747712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1},{"begin":8,"length":1},{"begin":14,"length":2},{"begin":20,"length":1},{"begin":22,"length":1},{"begin":24,"length":1},{"begin":29,"length":1},{"begin":34,"length":1},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":49,"length":2},{"begin":52,"length":1},{"begin":55,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":2},{"begin":72,"length":1},{"begin":75,"length":1},{"begin":78,"length":1},{"begin":84,"length":1},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":94,"length":1},{"begin":98,"length":1},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":24,"snaps":[{"begin":2,"length":32}]},{"pool":38,"snaps":[{"begin":2,"length":2}]},{"pool":46,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-17T08:49:20.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 msgr2=0x7fb448079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:20.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 0x7fb448079190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb454005fd0 tx=0x7fb45400b5f0 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 msgr2=0x7fb468158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7fb458000c00 tx=0x7fb458003910 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 shutdown_connections 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb448076cd0 0x7fb448079190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4680cf920 0x7fb46813bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4680cf540 0x7fb468155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 --2- 172.21.15.44:0/2821079254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4680b3db0 0x7fb468158620 unknown :-1 s=CLOSED pgs=3500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.118+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 >> 172.21.15.44:0/2821079254 conn(0x7fb4680b0e80 msgr2=0x7fb4680b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.119+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 shutdown_connections 2024-07-17T08:49:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:20.119+0000 7fb46d800640 1 -- 172.21.15.44:0/2821079254 wait complete. 2024-07-17T08:49:21.141 INFO:tasks.ceph:Scrubbing osd.0 2024-07-17T08:49:21.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.366+0000 7f66e8a00640 1 -- 172.21.15.44:0/2615308384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 msgr2=0x7f66e406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.366+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2615308384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e406e6d0 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7f66e406db20 tx=0x7f66d402a980 comp rx=0 tx=0).stop 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 -- 172.21.15.44:0/2615308384 shutdown_connections 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2615308384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e406ec10 0x7f66e40ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2615308384 >> v2:172.21.15.175:3300/0 conn(0x7f66e4153bb0 0x7f66e40ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2615308384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e406e6d0 unknown :-1 s=CLOSED pgs=3501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 -- 172.21.15.44:0/2615308384 >> 172.21.15.44:0/2615308384 conn(0x7f66e40cf050 msgr2=0x7f66e40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 -- 172.21.15.44:0/2615308384 shutdown_connections 2024-07-17T08:49:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.367+0000 7f66e8a00640 1 -- 172.21.15.44:0/2615308384 wait complete. 2024-07-17T08:49:21.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.369+0000 7f66e8a00640 1 Processor -- start 2024-07-17T08:49:21.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.369+0000 7f66e8a00640 1 -- start start 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e406ec10 0x7f66e414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e414a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e4153bb0 0x7f66e40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40ac330 con 0x7f66e41537d0 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40ac1b0 con 0x7f66e4153bb0 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66e40ac4b0 con 0x7f66e406ec10 2024-07-17T08:49:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e414a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e1e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e414a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33434/0 (socket says 172.21.15.44:33434) 2024-07-17T08:49:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e1e00640 1 -- 172.21.15.44:0/2382140046 learned_addr learned my addr 172.21.15.44:0/2382140046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e2800640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e406ec10 0x7f66e414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.371+0000 7f66e3200640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e4153bb0 0x7f66e40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 -- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e406ec10 msgr2=0x7f66e414a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e406ec10 0x7f66e414a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 -- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e4153bb0 msgr2=0x7f66e40afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e4153bb0 0x7f66e40afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e414b170 con 0x7f66e41537d0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e1e00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e414a870 secure :-1 s=READY pgs=3502 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00d750 tx=0x7f66cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc017070 con 0x7f66e41537d0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66cc009cc0 con 0x7f66e41537d0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc005280 con 0x7f66e41537d0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.372+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e414b450 con 0x7f66e41537d0 2024-07-17T08:49:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.373+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e414ba10 con 0x7f66e41537d0 2024-07-17T08:49:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.373+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66e406e6d0 con 0x7f66e41537d0 2024-07-17T08:49:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.375+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f66cc005420 con 0x7f66e41537d0 2024-07-17T08:49:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.376+0000 7f66daa00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f66c0076cd0 0x7f66c0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.376+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f66cc098a80 con 0x7f66e41537d0 2024-07-17T08:49:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.377+0000 7f66daa00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7f66c007fc30 0x7f66c00820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.377+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66cc020e80 con 0x7f66c007fc30 2024-07-17T08:49:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.377+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7f66cc09d2a0 con 0x7f66e41537d0 2024-07-17T08:49:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.377+0000 7f66e3200640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7f66c007fc30 0x7f66c00820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.377+0000 7f66e2800640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f66c0076cd0 0x7f66c0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.378+0000 7f66e3200640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7f66c007fc30 0x7f66c00820a0 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.378+0000 7f66e2800640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f66c0076cd0 0x7f66c0079190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66d4035000 tx=0x7f66d4004ab0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.379+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== osd.0 v2:172.21.15.44:6812/429081112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66cc020e80 con 0x7f66c007fc30 2024-07-17T08:49:21.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.394+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 --> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66e414adb0 con 0x7f66c007fc30 2024-07-17T08:49:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.394+0000 7f66daa00640 1 -- 172.21.15.44:0/2382140046 <== osd.0 v2:172.21.15.44:6812/429081112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66e414adb0 con 0x7f66c007fc30 2024-07-17T08:49:21.396 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:21.396 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:21.396 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7f66c007fc30 msgr2=0x7f66c00820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:6812/429081112,v1:172.21.15.44:6815/429081112] conn(0x7f66c007fc30 0x7f66c00820a0 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f66c0076cd0 msgr2=0x7f66c0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f66c0076cd0 0x7f66c0079190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66d4035000 tx=0x7f66d4004ab0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 msgr2=0x7f66e414a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 --2- 172.21.15.44:0/2382140046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66e41537d0 0x7f66e414a870 secure :-1 s=READY pgs=3502 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00d750 tx=0x7f66cc00dc10 comp rx=0 tx=0).stop 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e3200640 1 -- 172.21.15.44:0/2382140046 reap_dead start 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 shutdown_connections 2024-07-17T08:49:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 >> 172.21.15.44:0/2382140046 conn(0x7f66e40cf050 msgr2=0x7f66e40b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 shutdown_connections 2024-07-17T08:49:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.395+0000 7f66e8a00640 1 -- 172.21.15.44:0/2382140046 wait complete. 2024-07-17T08:49:21.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-07-17T08:49:21.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.628+0000 7fb5f9800640 1 -- 172.21.15.44:0/134921379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f406ed30 msgr2=0x7fb5f415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.628+0000 7fb5f9800640 1 --2- 172.21.15.44:0/134921379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f406ed30 0x7fb5f415c2f0 secure :-1 s=READY pgs=3503 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec005fd0 tx=0x7fb5ec02a970 comp rx=0 tx=0).stop 2024-07-17T08:49:21.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 -- 172.21.15.44:0/134921379 shutdown_connections 2024-07-17T08:49:21.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 --2- 172.21.15.44:0/134921379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f406ed30 0x7fb5f415c2f0 unknown :-1 s=CLOSED pgs=3503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 --2- 172.21.15.44:0/134921379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f40cbab0 0x7fb5f406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 -- 172.21.15.44:0/134921379 >> 172.21.15.44:0/134921379 conn(0x7fb5f40b0eb0 msgr2=0x7fb5f40b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 -- 172.21.15.44:0/134921379 shutdown_connections 2024-07-17T08:49:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.629+0000 7fb5f9800640 1 -- 172.21.15.44:0/134921379 wait complete. 2024-07-17T08:49:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.630+0000 7fb5f9800640 1 Processor -- start 2024-07-17T08:49:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.631+0000 7fb5f9800640 1 -- start start 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 0x7fb5f41f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 0x7fb5f4155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f40cd720 con 0x7fb5f4152220 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f40cd5a0 con 0x7fb5f40cbab0 2024-07-17T08:49:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f40cd8a0 con 0x7fb5f406ed30 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f3400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33440/0 (socket says 172.21.15.44:33440) 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f3400640 1 -- 172.21.15.44:0/3257005262 learned_addr learned my addr 172.21.15.44:0/3257005262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f2000640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 0x7fb5f4155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.632+0000 7fb5f2a00640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 0x7fb5f41f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 -- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 msgr2=0x7fb5f41f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 0x7fb5f41f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 -- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 msgr2=0x7fb5f4155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 0x7fb5f4155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 -- 172.21.15.44:0/3257005262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f4155b00 con 0x7fb5f4152220 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f2a00640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 0x7fb5f41f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f2000640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 0x7fb5f4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f3400640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec034070 con 0x7fb5f4152220 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ec0046c0 con 0x7fb5f4152220 2024-07-17T08:49:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0396b0 con 0x7fb5f4152220 2024-07-17T08:49:21.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.633+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f40715e0 con 0x7fb5f4152220 2024-07-17T08:49:21.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.634+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f4071ba0 con 0x7fb5f4152220 2024-07-17T08:49:21.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.635+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f40cddc0 con 0x7fb5f4152220 2024-07-17T08:49:21.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.636+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb5ec037020 con 0x7fb5f4152220 2024-07-17T08:49:21.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.637+0000 7fb5eaa00640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:21.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.637+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fb5ec0b9fb0 con 0x7fb5f4152220 2024-07-17T08:49:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.638+0000 7fb5f2a00640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.639+0000 7fb5f2a00640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 0x7fb5d4079190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400a830 tx=0x7fb5e4008040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.642+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb5ec082dd0 con 0x7fb5f4152220 2024-07-17T08:49:21.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.850+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb5f415f9d0 con 0x7fb5d4076cd0 2024-07-17T08:49:21.853 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 0 to deep-scrub 2024-07-17T08:49:21.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.851+0000 7fb5eaa00640 1 -- 172.21.15.44:0/3257005262 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb5f415f9d0 con 0x7fb5d4076cd0 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.853+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 msgr2=0x7fb5d4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.853+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 0x7fb5d4079190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400a830 tx=0x7fb5e4008040 comp rx=0 tx=0).stop 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.853+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 msgr2=0x7fb5f414ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.853+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec003f90 comp rx=0 tx=0).stop 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 shutdown_connections 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb5d4076cd0 0x7fb5d4079190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f4152220 0x7fb5f414ffd0 unknown :-1 s=CLOSED pgs=3504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5f40cbab0 0x7fb5f4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 --2- 172.21.15.44:0/3257005262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5f406ed30 0x7fb5f41f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:21.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 >> 172.21.15.44:0/3257005262 conn(0x7fb5f40b0eb0 msgr2=0x7fb5f40b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:21.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 shutdown_connections 2024-07-17T08:49:21.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:21.854+0000 7fb5f9800640 1 -- 172.21.15.44:0/3257005262 wait complete. 2024-07-17T08:49:21.878 INFO:tasks.ceph:Scrubbing osd.1 2024-07-17T08:49:21.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.093+0000 7fc0a9600640 1 -- 172.21.15.44:0/1064742256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc088005210 msgr2=0x7fc08800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.093+0000 7fc0a9600640 1 --2- 172.21.15.44:0/1064742256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc088005210 0x7fc08800b9c0 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005fd0 tx=0x7fc088005830 comp rx=0 tx=0).stop 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.094+0000 7fc0a9600640 1 -- 172.21.15.44:0/1064742256 shutdown_connections 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.094+0000 7fc0a9600640 1 --2- 172.21.15.44:0/1064742256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc088005210 0x7fc08800b9c0 unknown :-1 s=CLOSED pgs=3506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.094+0000 7fc0a9600640 1 -- 172.21.15.44:0/1064742256 >> 172.21.15.44:0/1064742256 conn(0x7fc0a40b2160 msgr2=0x7fc0a40b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.094+0000 7fc0a9600640 1 -- 172.21.15.44:0/1064742256 shutdown_connections 2024-07-17T08:49:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.094+0000 7fc0a9600640 1 -- 172.21.15.44:0/1064742256 wait complete. 2024-07-17T08:49:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.096+0000 7fc0a9600640 1 Processor -- start 2024-07-17T08:49:22.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.096+0000 7fc0a9600640 1 -- start start 2024-07-17T08:49:22.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc088005210 0x7fc0a413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 0x7fc0a406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0a406f360 0x7fc0a414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a415eeb0 con 0x7fc0a40ae690 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a415c890 con 0x7fc088005210 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a415f030 con 0x7fc0a406f360 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc09be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 0x7fc0a406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc09be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 0x7fc0a406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33450/0 (socket says 172.21.15.44:33450) 2024-07-17T08:49:22.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc09be00640 1 -- 172.21.15.44:0/3530336912 learned_addr learned my addr 172.21.15.44:0/3530336912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a2a00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc088005210 0x7fc0a413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.097+0000 7fc0a3400640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0a406f360 0x7fc0a414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 -- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0a406f360 msgr2=0x7fc0a414fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0a406f360 0x7fc0a414fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 -- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc088005210 msgr2=0x7fc0a413aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc088005210 0x7fc0a413aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a40ad940 con 0x7fc0a40ae690 2024-07-17T08:49:22.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc0a2a00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc088005210 0x7fc0a413aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-17T08:49:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09be00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 0x7fc0a406ee20 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00d780 tx=0x7fc08c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c018070 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc08c0050d0 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a40adbb0 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.098+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00b6c0 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.099+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a40ae170 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.100+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc0a406c500 con 0x7fc0a40ae690 2024-07-17T08:49:22.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.101+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fc08c01b070 con 0x7fc0a40ae690 2024-07-17T08:49:22.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.102+0000 7fc09aa00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc080076d00 0x7fc0800791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.102+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fc08c097ea0 con 0x7fc0a40ae690 2024-07-17T08:49:22.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.102+0000 7fc09aa00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fc08007fc60 0x7fc0800820d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.102+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc08c015ec0 con 0x7fc08007fc60 2024-07-17T08:49:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.102+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7fc08c09a4c0 con 0x7fc0a40ae690 2024-07-17T08:49:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.103+0000 7fc0a2a00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc080076d00 0x7fc0800791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.103+0000 7fc0a3400640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fc08007fc60 0x7fc0800820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.103+0000 7fc0a3400640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fc08007fc60 0x7fc0800820d0 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.104+0000 7fc0a2a00640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc080076d00 0x7fc0800791c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094006c40 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.104+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== osd.1 v2:172.21.15.44:6802/3861011633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc08c015ec0 con 0x7fc08007fc60 2024-07-17T08:49:22.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.119+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 --> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc0a406cb70 con 0x7fc08007fc60 2024-07-17T08:49:22.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.119+0000 7fc09aa00640 1 -- 172.21.15.44:0/3530336912 <== osd.1 v2:172.21.15.44:6802/3861011633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc0a406cb70 con 0x7fc08007fc60 2024-07-17T08:49:22.121 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:22.121 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:22.121 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:22.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fc08007fc60 msgr2=0x7fc0800820d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:6802/3861011633,v1:172.21.15.44:6804/3861011633] conn(0x7fc08007fc60 0x7fc0800820d0 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc080076d00 msgr2=0x7fc0800791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fc080076d00 0x7fc0800791c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094006c40 comp rx=0 tx=0).stop 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 msgr2=0x7fc0a406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 --2- 172.21.15.44:0/3530336912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0a40ae690 0x7fc0a406ee20 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00d780 tx=0x7fc08c00dc40 comp rx=0 tx=0).stop 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a3400640 1 -- 172.21.15.44:0/3530336912 reap_dead start 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 shutdown_connections 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 >> 172.21.15.44:0/3530336912 conn(0x7fc0a40b2160 msgr2=0x7fc0a415d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 shutdown_connections 2024-07-17T08:49:22.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.120+0000 7fc0a9600640 1 -- 172.21.15.44:0/3530336912 wait complete. 2024-07-17T08:49:22.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-07-17T08:49:22.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 -- 172.21.15.44:0/2909119235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2f0004170 msgr2=0x7fe2f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 --2- 172.21.15.44:0/2909119235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2f0004170 0x7fe2f0004800 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40016c0 tx=0x7fe2f0005830 comp rx=0 tx=0).stop 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30a800640 1 -- 172.21.15.44:0/2909119235 reap_dead start 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 -- 172.21.15.44:0/2909119235 shutdown_connections 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 -- 172.21.15.44:0/2909119235 >> 172.21.15.44:0/2909119235 conn(0x7fe30c0cf0d0 msgr2=0x7fe30c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 -- 172.21.15.44:0/2909119235 shutdown_connections 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.348+0000 7fe30b200640 1 -- 172.21.15.44:0/2909119235 wait complete. 2024-07-17T08:49:22.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.350+0000 7fe30b200640 1 Processor -- start 2024-07-17T08:49:22.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.350+0000 7fe30b200640 1 -- start start 2024-07-17T08:49:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 0x7fe30c071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe30c0739d0 0x7fe30c09c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c152480 con 0x7fe30c1f8490 2024-07-17T08:49:22.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c152300 con 0x7fe2f0004170 2024-07-17T08:49:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe30b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c152600 con 0x7fe30c0739d0 2024-07-17T08:49:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe303e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe303e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33472/0 (socket says 172.21.15.44:33472) 2024-07-17T08:49:22.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.351+0000 7fe303e00640 1 -- 172.21.15.44:0/2152818339 learned_addr learned my addr 172.21.15.44:0/2152818339 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 -- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe30c0739d0 msgr2=0x7fe30c09c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe309e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 0x7fe30c071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe30c0739d0 0x7fe30c09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 -- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 msgr2=0x7fe30c071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 0x7fe30c071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 -- 172.21.15.44:0/2152818339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c0726f0 con 0x7fe30c1f8490 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe309e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 0x7fe30c071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe303e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fe2f402a930 tx=0x7fe2f4006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4038070 con 0x7fe30c1f8490 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2f4003af0 con 0x7fe30c1f8490 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.352+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4033420 con 0x7fe30c1f8490 2024-07-17T08:49:22.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.353+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c072ef0 con 0x7fe30c1f8490 2024-07-17T08:49:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.353+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c072300 con 0x7fe30c1f8490 2024-07-17T08:49:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.354+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c15c450 con 0x7fe30c1f8490 2024-07-17T08:49:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.355+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fe2f4043020 con 0x7fe30c1f8490 2024-07-17T08:49:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.356+0000 7fe302000640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 0x7fe2e8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.356+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fe2f40b7fe0 con 0x7fe30c1f8490 2024-07-17T08:49:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.357+0000 7fe309e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 0x7fe2e8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.358+0000 7fe309e00640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 0x7fe2e8079190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005fd0 tx=0x7fe2fc005d10 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.360+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fe2f4080f90 con 0x7fe30c1f8490 2024-07-17T08:49:22.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.576+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe30c0cbd00 con 0x7fe2e8076cd0 2024-07-17T08:49:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.578+0000 7fe302000640 1 -- 172.21.15.44:0/2152818339 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe30c0cbd00 con 0x7fe2e8076cd0 2024-07-17T08:49:22.579 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 1 to deep-scrub 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 msgr2=0x7fe2e8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 0x7fe2e8079190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005fd0 tx=0x7fe2fc005d10 comp rx=0 tx=0).stop 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 msgr2=0x7fe30c1f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fe2f402a930 tx=0x7fe2f4006360 comp rx=0 tx=0).stop 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 shutdown_connections 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fe2e8076cd0 0x7fe2e8079190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe30c0739d0 0x7fe30c09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe30c1f8490 0x7fe30c1f7b60 unknown :-1 s=CLOSED pgs=3510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 --2- 172.21.15.44:0/2152818339 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2f0004170 0x7fe30c071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 >> 172.21.15.44:0/2152818339 conn(0x7fe30c0cf0d0 msgr2=0x7fe30c15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 shutdown_connections 2024-07-17T08:49:22.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.581+0000 7fe30b200640 1 -- 172.21.15.44:0/2152818339 wait complete. 2024-07-17T08:49:22.602 INFO:tasks.ceph:Scrubbing osd.2 2024-07-17T08:49:22.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 -- 172.21.15.44:0/952890169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 msgr2=0x7ff018004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 --2- 172.21.15.44:0/952890169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff018004c80 secure :-1 s=READY pgs=3512 cs=0 l=1 rev1=1 crypto rx=0x7ff01c006fc0 tx=0x7ff018005830 comp rx=0 tx=0).stop 2024-07-17T08:49:22.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff033400640 1 -- 172.21.15.44:0/952890169 reap_dead start 2024-07-17T08:49:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 -- 172.21.15.44:0/952890169 shutdown_connections 2024-07-17T08:49:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 -- 172.21.15.44:0/952890169 >> 172.21.15.44:0/952890169 conn(0x7ff0340b0eb0 msgr2=0x7ff0340b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 -- 172.21.15.44:0/952890169 shutdown_connections 2024-07-17T08:49:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.807+0000 7ff039a00640 1 -- 172.21.15.44:0/952890169 wait complete. 2024-07-17T08:49:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.809+0000 7ff039a00640 1 Processor -- start 2024-07-17T08:49:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.809+0000 7ff039a00640 1 -- start start 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff034158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0340cab10 0x7ff0340ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0340caf90 0x7ff034072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff03415b5c0 con 0x7ff018004800 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff03415b440 con 0x7ff0340cab10 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff039a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff03406d620 con 0x7ff0340caf90 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff032a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff034158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff032a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff034158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33496/0 (socket says 172.21.15.44:33496) 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.810+0000 7ff032a00640 1 -- 172.21.15.44:0/1401263193 learned_addr learned my addr 172.21.15.44:0/1401263193 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 -- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0340caf90 msgr2=0x7ff034072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0340caf90 0x7ff034072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 -- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0340cab10 msgr2=0x7ff0340ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0340cab10 0x7ff0340ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0340c9940 con 0x7ff018004800 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff032a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff034158710 secure :-1 s=READY pgs=3513 cs=0 l=1 rev1=1 crypto rx=0x7ff024010a10 tx=0x7ff024010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400ece0 con 0x7ff018004800 2024-07-17T08:49:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff02400ee80 con 0x7ff018004800 2024-07-17T08:49:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024005260 con 0x7ff018004800 2024-07-17T08:49:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0340c9c20 con 0x7ff018004800 2024-07-17T08:49:22.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.811+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0340ca1e0 con 0x7ff018004800 2024-07-17T08:49:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.812+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff03406dbb0 con 0x7ff018004800 2024-07-17T08:49:22.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.814+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7ff024005400 con 0x7ff018004800 2024-07-17T08:49:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.815+0000 7ff02aa00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ff010076cd0 0x7ff010079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.815+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7ff024099d30 con 0x7ff018004800 2024-07-17T08:49:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.815+0000 7ff02aa00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7ff01007fc30 0x7ff0100820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.815+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0100826d0 con 0x7ff01007fc30 2024-07-17T08:49:22.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.815+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7ff02409a0e0 con 0x7ff018004800 2024-07-17T08:49:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.816+0000 7ff033400640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7ff01007fc30 0x7ff0100820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.816+0000 7ff02be00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ff010076cd0 0x7ff010079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:22.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.816+0000 7ff033400640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7ff01007fc30 0x7ff0100820a0 crc :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.817+0000 7ff02be00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ff010076cd0 0x7ff010079190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff01c033000 tx=0x7ff03406ec60 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:22.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.817+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== osd.2 v2:172.21.15.44:6821/2788335734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0100826d0 con 0x7ff01007fc30 2024-07-17T08:49:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.832+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 --> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff0340c9580 con 0x7ff01007fc30 2024-07-17T08:49:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.832+0000 7ff02aa00640 1 -- 172.21.15.44:0/1401263193 <== osd.2 v2:172.21.15.44:6821/2788335734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff0340c9580 con 0x7ff01007fc30 2024-07-17T08:49:22.834 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:22.834 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:22.834 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7ff01007fc30 msgr2=0x7ff0100820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:6821/2788335734,v1:172.21.15.44:6823/2788335734] conn(0x7ff01007fc30 0x7ff0100820a0 crc :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ff010076cd0 msgr2=0x7ff010079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ff010076cd0 0x7ff010079190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff01c033000 tx=0x7ff03406ec60 comp rx=0 tx=0).stop 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 msgr2=0x7ff034158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 --2- 172.21.15.44:0/1401263193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff018004800 0x7ff034158710 secure :-1 s=READY pgs=3513 cs=0 l=1 rev1=1 crypto rx=0x7ff024010a10 tx=0x7ff024010ed0 comp rx=0 tx=0).stop 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff033400640 1 -- 172.21.15.44:0/1401263193 reap_dead start 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 shutdown_connections 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 >> 172.21.15.44:0/1401263193 conn(0x7ff0340b0eb0 msgr2=0x7ff03415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 shutdown_connections 2024-07-17T08:49:22.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:22.833+0000 7ff039a00640 1 -- 172.21.15.44:0/1401263193 wait complete. 2024-07-17T08:49:22.853 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-07-17T08:49:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 -- 172.21.15.44:0/489500747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 msgr2=0x7f7b9c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 --2- 172.21.15.44:0/489500747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c15a5a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c02a5d0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 -- 172.21.15.44:0/489500747 shutdown_connections 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 --2- 172.21.15.44:0/489500747 >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0b3280 0x7f7b9c0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 --2- 172.21.15.44:0/489500747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c15a5a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 -- 172.21.15.44:0/489500747 >> 172.21.15.44:0/489500747 conn(0x7f7b9c0b0eb0 msgr2=0x7f7b9c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.063+0000 7f7ba2000640 1 -- 172.21.15.44:0/489500747 shutdown_connections 2024-07-17T08:49:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.064+0000 7f7ba2000640 1 -- 172.21.15.44:0/489500747 wait complete. 2024-07-17T08:49:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.065+0000 7f7ba2000640 1 Processor -- start 2024-07-17T08:49:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.065+0000 7f7ba2000640 1 -- start start 2024-07-17T08:49:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 0x7f7b9c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c06f850 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c06f6d0 con 0x7f7b9c0b3280 2024-07-17T08:49:23.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.066+0000 7f7ba2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c150470 con 0x7f7b9c06eb90 2024-07-17T08:49:23.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33500/0 (socket says 172.21.15.44:33500) 2024-07-17T08:49:23.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 -- 172.21.15.44:0/2926399175 learned_addr learned my addr 172.21.15.44:0/2926399175 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9be00640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 0x7f7b9c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9b400640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 -- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 msgr2=0x7f7b9c155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 -- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 msgr2=0x7f7b9c0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 0x7f7b9c0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 -- 172.21.15.44:0/2926399175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1510b0 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9be00640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 0x7f7b9c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.067+0000 7f7b9aa00640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 secure :-1 s=READY pgs=3514 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009d30 tx=0x7f7b8400c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.068+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84003b70 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.068+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b84003d10 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.068+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004290 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.068+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c151390 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.068+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c151950 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.069+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c15a5a0 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.070+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f7b84004430 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.072+0000 7f7b93400640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 0x7f7b78079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.072+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f7b84099380 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.073+0000 7f7b9b400640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 0x7f7b78079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.074+0000 7f7b9b400640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 0x7f7b78079190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c033000 tx=0x7f7b8c004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.077+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f7b84011070 con 0x7f7b9c0ac3a0 2024-07-17T08:49:23.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.288+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f7b9c150e50 con 0x7f7b78076cd0 2024-07-17T08:49:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.290+0000 7f7b93400640 1 -- 172.21.15.44:0/2926399175 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7b8c0042c0 con 0x7f7b78076cd0 2024-07-17T08:49:23.292 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 2 to deep-scrub 2024-07-17T08:49:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.292+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 msgr2=0x7f7b78079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 0x7f7b78079190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c033000 tx=0x7f7b8c004730 comp rx=0 tx=0).stop 2024-07-17T08:49:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 msgr2=0x7f7b9c150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 secure :-1 s=READY pgs=3514 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009d30 tx=0x7f7b8400c560 comp rx=0 tx=0).stop 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 shutdown_connections 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f7b78076cd0 0x7f7b78079190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0b3280 0x7f7b9c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b9c0ac3a0 0x7f7b9c150910 unknown :-1 s=CLOSED pgs=3514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 --2- 172.21.15.44:0/2926399175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c06eb90 0x7f7b9c155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 >> 172.21.15.44:0/2926399175 conn(0x7f7b9c0b0eb0 msgr2=0x7f7b9c0cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 shutdown_connections 2024-07-17T08:49:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.293+0000 7f7ba2000640 1 -- 172.21.15.44:0/2926399175 wait complete. 2024-07-17T08:49:23.316 INFO:tasks.ceph:Scrubbing osd.3 2024-07-17T08:49:23.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 -- 172.21.15.44:0/1433091008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 msgr2=0x7ffb7c0b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 --2- 172.21.15.44:0/1433091008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0b8670 secure :-1 s=READY pgs=3515 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0766b0 tx=0x7ffb7402a420 comp rx=0 tx=0).stop 2024-07-17T08:49:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 -- 172.21.15.44:0/1433091008 shutdown_connections 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 --2- 172.21.15.44:0/1433091008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 0x7ffb7c0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 --2- 172.21.15.44:0/1433091008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0b8670 unknown :-1 s=CLOSED pgs=3515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 --2- 172.21.15.44:0/1433091008 >> v2:172.21.15.175:3300/0 conn(0x7ffb7c15e370 0x7ffb7c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 -- 172.21.15.44:0/1433091008 >> 172.21.15.44:0/1433091008 conn(0x7ffb7c073a60 msgr2=0x7ffb7c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 -- 172.21.15.44:0/1433091008 shutdown_connections 2024-07-17T08:49:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.533+0000 7ffb82a00640 1 -- 172.21.15.44:0/1433091008 wait complete. 2024-07-17T08:49:23.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.535+0000 7ffb82a00640 1 Processor -- start 2024-07-17T08:49:23.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.535+0000 7ffb82a00640 1 -- start start 2024-07-17T08:49:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7c15e370 0x7ffb7c0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 0x7ffb7c150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c15e8d0 con 0x7ffb7c0d0970 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c15e750 con 0x7ffb7c15e370 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb82a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c0b8bb0 con 0x7ffb7c161b50 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb81600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb81600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33518/0 (socket says 172.21.15.44:33518) 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.536+0000 7ffb81600640 1 -- 172.21.15.44:0/3937711020 learned_addr learned my addr 172.21.15.44:0/3937711020 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb82000640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 0x7ffb7c150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb80c00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7c15e370 0x7ffb7c0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 -- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 msgr2=0x7ffb7c150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 0x7ffb7c150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 -- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7c15e370 msgr2=0x7ffb7c0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7c15e370 0x7ffb7c0cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c0c95a0 con 0x7ffb7c0d0970 2024-07-17T08:49:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb82000640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb7c161b50 0x7ffb7c150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb80c00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb7c15e370 0x7ffb7c0cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb81600640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0cafa0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7ffb70005fd0 tx=0x7ffb7000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000f650 con 0x7ffb7c0d0970 2024-07-17T08:49:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.537+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb7000f8e0 con 0x7ffb7c0d0970 2024-07-17T08:49:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.538+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7000fd50 con 0x7ffb7c0d0970 2024-07-17T08:49:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.538+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c0c9b40 con 0x7ffb7c0d0970 2024-07-17T08:49:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.538+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c0c9e20 con 0x7ffb7c0d0970 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.539+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb7c0ce500 con 0x7ffb7c0d0970 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.540+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7ffb70018400 con 0x7ffb7c0d0970 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.541+0000 7ffb79600640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffb5c076cd0 0x7ffb5c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.541+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7ffb70099960 con 0x7ffb7c0d0970 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.542+0000 7ffb79600640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7ffb5c07fc30 0x7ffb5c0820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.542+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb700059c0 con 0x7ffb5c07fc30 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.542+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7ffb70099d10 con 0x7ffb7c0d0970 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.542+0000 7ffb82000640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7ffb5c07fc30 0x7ffb5c0820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.542+0000 7ffb80c00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffb5c076cd0 0x7ffb5c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.543+0000 7ffb82000640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7ffb5c07fc30 0x7ffb5c0820a0 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.543+0000 7ffb80c00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffb5c076cd0 0x7ffb5c079190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c9240 tx=0x7ffb68005db0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.543+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== osd.3 v2:172.21.15.44:6800/2705860470 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb700059c0 con 0x7ffb5c07fc30 2024-07-17T08:49:23.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.558+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 --> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffb7c069460 con 0x7ffb5c07fc30 2024-07-17T08:49:23.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.559+0000 7ffb79600640 1 -- 172.21.15.44:0/3937711020 <== osd.3 v2:172.21.15.44:6800/2705860470 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffb7c069460 con 0x7ffb5c07fc30 2024-07-17T08:49:23.561 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:23.561 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.559+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7ffb5c07fc30 msgr2=0x7ffb5c0820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.559+0000 7ffb82a00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:6800/2705860470,v1:172.21.15.44:6801/2705860470] conn(0x7ffb5c07fc30 0x7ffb5c0820a0 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffb5c076cd0 msgr2=0x7ffb5c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7ffb5c076cd0 0x7ffb5c079190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c9240 tx=0x7ffb68005db0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 msgr2=0x7ffb7c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 --2- 172.21.15.44:0/3937711020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb7c0d0970 0x7ffb7c0cafa0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7ffb70005fd0 tx=0x7ffb7000a620 comp rx=0 tx=0).stop 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82000640 1 -- 172.21.15.44:0/3937711020 reap_dead start 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 shutdown_connections 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 >> 172.21.15.44:0/3937711020 conn(0x7ffb7c073a60 msgr2=0x7ffb7c160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 shutdown_connections 2024-07-17T08:49:23.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.560+0000 7ffb82a00640 1 -- 172.21.15.44:0/3937711020 wait complete. 2024-07-17T08:49:23.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-07-17T08:49:23.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.795+0000 7f1e5b200640 1 -- 172.21.15.44:0/1991451544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 msgr2=0x7f1e5c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.795+0000 7f1e5b200640 1 --2- 172.21.15.44:0/1991451544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c15c140 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005fd0 tx=0x7f1e4c02a4d0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 -- 172.21.15.44:0/1991451544 shutdown_connections 2024-07-17T08:49:23.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 --2- 172.21.15.44:0/1991451544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 0x7f1e5c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 --2- 172.21.15.44:0/1991451544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c15c140 unknown :-1 s=CLOSED pgs=3517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 -- 172.21.15.44:0/1991451544 >> 172.21.15.44:0/1991451544 conn(0x7f1e5c0cf0d0 msgr2=0x7f1e5c0cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 -- 172.21.15.44:0/1991451544 shutdown_connections 2024-07-17T08:49:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.796+0000 7f1e5b200640 1 -- 172.21.15.44:0/1991451544 wait complete. 2024-07-17T08:49:23.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.797+0000 7f1e5b200640 1 Processor -- start 2024-07-17T08:49:23.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.798+0000 7f1e5b200640 1 -- start start 2024-07-17T08:49:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 0x7f1e5c1ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5c0cd0b0 0x7f1e5c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0cb740 con 0x7f1e5c151f20 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0cb5c0 con 0x7f1e5c0cd0b0 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0cb8c0 con 0x7f1e5c06f380 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33542/0 (socket says 172.21.15.44:33542) 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 -- 172.21.15.44:0/2685812995 learned_addr learned my addr 172.21.15.44:0/2685812995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e59e00640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 0x7f1e5c1ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e59400640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5c0cd0b0 0x7f1e5c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 -- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 msgr2=0x7f1e5c1ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 0x7f1e5c1ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 -- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5c0cd0b0 msgr2=0x7f1e5c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5c0cd0b0 0x7f1e5c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:23.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.799+0000 7f1e5a800640 1 -- 172.21.15.44:0/2685812995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1effd0 con 0x7f1e5c151f20 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e5a800640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b9e0 tx=0x7f1e5400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54019070 con 0x7f1e5c151f20 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e5400cc10 con 0x7f1e5c151f20 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540146a0 con 0x7f1e5c151f20 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1f0310 con 0x7f1e5c151f20 2024-07-17T08:49:23.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.800+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c072430 con 0x7f1e5c151f20 2024-07-17T08:49:23.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.801+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c050670 con 0x7f1e5c151f20 2024-07-17T08:49:23.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.803+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f1e5401c070 con 0x7f1e5c151f20 2024-07-17T08:49:23.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.804+0000 7f1e52000640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 0x7f1e3c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:23.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.804+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f1e540986c0 con 0x7f1e5c151f20 2024-07-17T08:49:23.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.804+0000 7f1e59e00640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 0x7f1e3c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:23.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.805+0000 7f1e59e00640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 0x7f1e3c079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:23.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:23.808+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f1e5c050670 con 0x7f1e5c151f20 2024-07-17T08:49:24.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.022+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1e5c0cb080 con 0x7f1e3c076cd0 2024-07-17T08:49:24.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.024+0000 7f1e52000640 1 -- 172.21.15.44:0/2685812995 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1e5c0cb080 con 0x7f1e3c076cd0 2024-07-17T08:49:24.026 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 3 to deep-scrub 2024-07-17T08:49:24.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.027+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 msgr2=0x7f1e3c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.027+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 0x7f1e3c079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).stop 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.027+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 msgr2=0x7f1e5c1f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.027+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b9e0 tx=0x7f1e5400bea0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 shutdown_connections 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f1e3c076cd0 0x7f1e3c079190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e5c151f20 0x7f1e5c1f3220 unknown :-1 s=CLOSED pgs=3518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e5c0cd0b0 0x7f1e5c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 --2- 172.21.15.44:0/2685812995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e5c06f380 0x7f1e5c1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 >> 172.21.15.44:0/2685812995 conn(0x7f1e5c0cf0d0 msgr2=0x7f1e5c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 shutdown_connections 2024-07-17T08:49:24.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.028+0000 7f1e5b200640 1 -- 172.21.15.44:0/2685812995 wait complete. 2024-07-17T08:49:24.050 INFO:tasks.ceph:Scrubbing osd.4 2024-07-17T08:49:24.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:24.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.264+0000 7fb7c3e00640 1 -- 172.21.15.44:0/1260505204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 msgr2=0x7fb7c40b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.264+0000 7fb7c3e00640 1 --2- 172.21.15.44:0/1260505204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c40b8670 secure :-1 s=READY pgs=3520 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005fd0 tx=0x7fb7bc02a980 comp rx=0 tx=0).stop 2024-07-17T08:49:24.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.265+0000 7fb7c3e00640 1 -- 172.21.15.44:0/1260505204 shutdown_connections 2024-07-17T08:49:24.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.265+0000 7fb7c3e00640 1 --2- 172.21.15.44:0/1260505204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c40b8670 unknown :-1 s=CLOSED pgs=3520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.265+0000 7fb7c3e00640 1 -- 172.21.15.44:0/1260505204 >> 172.21.15.44:0/1260505204 conn(0x7fb7c4073a60 msgr2=0x7fb7c4073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.265+0000 7fb7c3e00640 1 -- 172.21.15.44:0/1260505204 shutdown_connections 2024-07-17T08:49:24.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.265+0000 7fb7c3e00640 1 -- 172.21.15.44:0/1260505204 wait complete. 2024-07-17T08:49:24.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.266+0000 7fb7c3e00640 1 Processor -- start 2024-07-17T08:49:24.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.266+0000 7fb7c3e00640 1 -- start start 2024-07-17T08:49:24.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c41f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c415e370 0x7fb7c41f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7c4161b50 0x7fb7c40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40b8d30 con 0x7fb7c40d0b40 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40b8bb0 con 0x7fb7c415e370 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.267+0000 7fb7c3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c40b8eb0 con 0x7fb7c4161b50 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c41f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c41f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33570/0 (socket says 172.21.15.44:33570) 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 -- 172.21.15.44:0/2236240789 learned_addr learned my addr 172.21.15.44:0/2236240789 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2000640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c415e370 0x7fb7c41f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c3400640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7c4161b50 0x7fb7c40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 -- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7c4161b50 msgr2=0x7fb7c40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7c4161b50 0x7fb7c40b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 -- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c415e370 msgr2=0x7fb7c41f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7c415e370 0x7fb7c41f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.268+0000 7fb7c2a00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c40b32e0 con 0x7fb7c40d0b40 2024-07-17T08:49:24.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7c2a00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c41f3e20 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c450 tx=0x7fb7b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40052d0 con 0x7fb7c40d0b40 2024-07-17T08:49:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7b4005470 con 0x7fb7c40d0b40 2024-07-17T08:49:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4012870 con 0x7fb7c40d0b40 2024-07-17T08:49:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c40b35c0 con 0x7fb7c40d0b40 2024-07-17T08:49:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.269+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c40b3b80 con 0x7fb7c40d0b40 2024-07-17T08:49:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.270+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7c40b9330 con 0x7fb7c40d0b40 2024-07-17T08:49:24.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.271+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb7b40035f0 con 0x7fb7c40d0b40 2024-07-17T08:49:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.272+0000 7fb7baa00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb7a0076cd0 0x7fb7a0079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.272+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fb7b4098460 con 0x7fb7c40d0b40 2024-07-17T08:49:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.273+0000 7fb7baa00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fb7a007fc30 0x7fb7a00820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.273+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7b401ae50 con 0x7fb7a007fc30 2024-07-17T08:49:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.273+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7fb7b409f050 con 0x7fb7c40d0b40 2024-07-17T08:49:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.273+0000 7fb7c2000640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb7a0076cd0 0x7fb7a0079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.273+0000 7fb7c3400640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fb7a007fc30 0x7fb7a00820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.274+0000 7fb7c2000640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb7a0076cd0 0x7fb7a0079190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006fd0 tx=0x7fb7ac008040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.274+0000 7fb7c3400640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fb7a007fc30 0x7fb7a00820a0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.276+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== osd.4 v2:172.21.15.175:6802/2954959872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7b401ae50 con 0x7fb7a007fc30 2024-07-17T08:49:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.292+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 --> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7c4069460 con 0x7fb7a007fc30 2024-07-17T08:49:24.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.293+0000 7fb7baa00640 1 -- 172.21.15.44:0/2236240789 <== osd.4 v2:172.21.15.175:6802/2954959872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7c4069460 con 0x7fb7a007fc30 2024-07-17T08:49:24.295 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:24.295 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:24.295 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:24.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.293+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fb7a007fc30 msgr2=0x7fb7a00820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.293+0000 7fb7c3e00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6802/2954959872,v1:172.21.15.175:6804/2954959872] conn(0x7fb7a007fc30 0x7fb7a00820a0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.293+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb7a0076cd0 msgr2=0x7fb7a0079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.293+0000 7fb7c3e00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb7a0076cd0 0x7fb7a0079190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006fd0 tx=0x7fb7ac008040 comp rx=0 tx=0).stop 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 msgr2=0x7fb7c41f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 --2- 172.21.15.44:0/2236240789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7c40d0b40 0x7fb7c41f3e20 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c450 tx=0x7fb7b400c910 comp rx=0 tx=0).stop 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3400640 1 -- 172.21.15.44:0/2236240789 reap_dead start 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 shutdown_connections 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 >> 172.21.15.44:0/2236240789 conn(0x7fb7c4073a60 msgr2=0x7fb7c415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 shutdown_connections 2024-07-17T08:49:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.294+0000 7fb7c3e00640 1 -- 172.21.15.44:0/2236240789 wait complete. 2024-07-17T08:49:24.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-07-17T08:49:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 -- 172.21.15.44:0/1140306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 msgr2=0x7fd5540d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 --2- 172.21.15.44:0/1140306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540d1e30 secure :-1 s=READY pgs=3523 cs=0 l=1 rev1=1 crypto rx=0x7fd5540b8bb0 tx=0x7fd54c02a820 comp rx=0 tx=0).stop 2024-07-17T08:49:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 -- 172.21.15.44:0/1140306510 shutdown_connections 2024-07-17T08:49:24.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 --2- 172.21.15.44:0/1140306510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540d1e30 unknown :-1 s=CLOSED pgs=3523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 --2- 172.21.15.44:0/1140306510 >> v2:172.21.15.175:3300/0 conn(0x7fd5540b55d0 0x7fd5540d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 -- 172.21.15.44:0/1140306510 >> 172.21.15.44:0/1140306510 conn(0x7fd5540cfec0 msgr2=0x7fd5540d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 -- 172.21.15.44:0/1140306510 shutdown_connections 2024-07-17T08:49:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.509+0000 7fd558a00640 1 -- 172.21.15.44:0/1140306510 wait complete. 2024-07-17T08:49:24.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.511+0000 7fd558a00640 1 Processor -- start 2024-07-17T08:49:24.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.511+0000 7fd558a00640 1 -- start start 2024-07-17T08:49:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5540b55d0 0x7fd554154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 0x7fd5541f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540d1220 con 0x7fd5540d2fa0 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540d2370 con 0x7fd55415d300 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd558a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5540cc6b0 con 0x7fd5540b55d0 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd552a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd552a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33582/0 (socket says 172.21.15.44:33582) 2024-07-17T08:49:24.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd552a00640 1 -- 172.21.15.44:0/2016935319 learned_addr learned my addr 172.21.15.44:0/2016935319 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd553400640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5540b55d0 0x7fd554154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.512+0000 7fd553e00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 0x7fd5541f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 -- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5540b55d0 msgr2=0x7fd554154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5540b55d0 0x7fd554154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 -- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 msgr2=0x7fd5541f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 0x7fd5541f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 -- 172.21.15.44:0/2016935319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55415f510 con 0x7fd5540d2fa0 2024-07-17T08:49:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd553e00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 0x7fd5541f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd552a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7fd54000d950 tx=0x7fd54000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540017070 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd540009d20 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400106c0 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.513+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5540cc950 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.514+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5540cc290 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.515+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5540ca2b0 con 0x7fd5540d2fa0 2024-07-17T08:49:24.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.516+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fd540010860 con 0x7fd5540d2fa0 2024-07-17T08:49:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.517+0000 7fd54b400640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 0x7fd534079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.517+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fd540098db0 con 0x7fd5540d2fa0 2024-07-17T08:49:24.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.518+0000 7fd553400640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 0x7fd534079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.519+0000 7fd553400640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 0x7fd534079190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ca210 tx=0x7fd544009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.521+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd5540ca2b0 con 0x7fd5540d2fa0 2024-07-17T08:49:24.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.734+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fd5540d13a0 con 0x7fd534076cd0 2024-07-17T08:49:24.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.736+0000 7fd54b400640 1 -- 172.21.15.44:0/2016935319 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd544015430 con 0x7fd534076cd0 2024-07-17T08:49:24.737 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 4 to deep-scrub 2024-07-17T08:49:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.738+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 msgr2=0x7fd534079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.738+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 0x7fd534079190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ca210 tx=0x7fd544009040 comp rx=0 tx=0).stop 2024-07-17T08:49:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 msgr2=0x7fd5540cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7fd54000d950 tx=0x7fd54000de10 comp rx=0 tx=0).stop 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 shutdown_connections 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd534076cd0 0x7fd534079190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55415d300 0x7fd5541f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5540d2fa0 0x7fd5540cbd50 unknown :-1 s=CLOSED pgs=3524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 --2- 172.21.15.44:0/2016935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5540b55d0 0x7fd554154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 >> 172.21.15.44:0/2016935319 conn(0x7fd5540cfec0 msgr2=0x7fd5540b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 shutdown_connections 2024-07-17T08:49:24.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.739+0000 7fd558a00640 1 -- 172.21.15.44:0/2016935319 wait complete. 2024-07-17T08:49:24.762 INFO:tasks.ceph:Scrubbing osd.5 2024-07-17T08:49:24.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 -- 172.21.15.44:0/2247389687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34300b1d40 msgr2=0x7f343006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 --2- 172.21.15.44:0/2247389687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34300b1d40 0x7f343006bf40 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f3430073300 tx=0x7f342002a560 comp rx=0 tx=0).stop 2024-07-17T08:49:24.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 -- 172.21.15.44:0/2247389687 shutdown_connections 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 --2- 172.21.15.44:0/2247389687 >> v2:172.21.15.175:3300/0 conn(0x7f34301545f0 0x7f343006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 --2- 172.21.15.44:0/2247389687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34300b1d40 0x7f343006bf40 unknown :-1 s=CLOSED pgs=3525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 --2- 172.21.15.44:0/2247389687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3430154210 0x7f343015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 -- 172.21.15.44:0/2247389687 >> 172.21.15.44:0/2247389687 conn(0x7f34300b55d0 msgr2=0x7f34300b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 -- 172.21.15.44:0/2247389687 shutdown_connections 2024-07-17T08:49:24.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.978+0000 7f3435c00640 1 -- 172.21.15.44:0/2247389687 wait complete. 2024-07-17T08:49:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.980+0000 7f3435c00640 1 Processor -- start 2024-07-17T08:49:24.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.980+0000 7f3435c00640 1 -- start start 2024-07-17T08:49:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3430154210 0x7f343014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34301545f0 0x7f34301f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 0x7f3430155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f34300d0c10 con 0x7f343014d9b0 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34300d0a90 con 0x7f34301545f0 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34300d0d90 con 0x7f3430154210 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 0x7f3430155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 0x7f3430155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33612/0 (socket says 172.21.15.44:33612) 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.981+0000 7f3435200640 1 -- 172.21.15.44:0/21040930 learned_addr learned my addr 172.21.15.44:0/21040930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f342f400640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34301545f0 0x7f34301f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f342fe00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3430154210 0x7f343014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 -- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3430154210 msgr2=0x7f343014f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3430154210 0x7f343014f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 -- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34301545f0 msgr2=0x7f34301f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34301545f0 0x7f34301f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34301f76a0 con 0x7f343014d9b0 2024-07-17T08:49:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435200640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 0x7f3430155210 secure :-1 s=READY pgs=3526 cs=0 l=1 rev1=1 crypto rx=0x7f342800e960 tx=0x7f342800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428004d20 con 0x7f343014d9b0 2024-07-17T08:49:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342800cb20 con 0x7f343014d9b0 2024-07-17T08:49:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34301f7980 con 0x7f343014d9b0 2024-07-17T08:49:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428005020 con 0x7f343014d9b0 2024-07-17T08:49:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.982+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3430154a10 con 0x7f343014d9b0 2024-07-17T08:49:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.983+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34301f7440 con 0x7f343014d9b0 2024-07-17T08:49:24.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.985+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f34280051c0 con 0x7f343014d9b0 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f3427e00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3410076cd0 0x7f3410079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f3428099200 con 0x7f343014d9b0 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f3427e00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f341007fc20 0x7f3410082090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3428018ec0 con 0x7f341007fc20 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7f34280a0050 con 0x7f343014d9b0 2024-07-17T08:49:24.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.986+0000 7f342fe00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3410076cd0 0x7f3410079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.987+0000 7f342f400640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f341007fc20 0x7f3410082090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:24.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.988+0000 7f342fe00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3410076cd0 0x7f3410079190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f341c00bb20 tx=0x7f341c005e50 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.988+0000 7f342f400640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f341007fc20 0x7f3410082090 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:24.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:24.990+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== osd.5 v2:172.21.15.175:6818/1413905252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3428018ec0 con 0x7f341007fc20 2024-07-17T08:49:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.005+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 --> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f343015d8f0 con 0x7f341007fc20 2024-07-17T08:49:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.006+0000 7f3427e00640 1 -- 172.21.15.44:0/21040930 <== osd.5 v2:172.21.15.175:6818/1413905252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f343015d8f0 con 0x7f341007fc20 2024-07-17T08:49:25.009 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:25.009 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:25.009 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f341007fc20 msgr2=0x7f3410082090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6818/1413905252,v1:172.21.15.175:6819/1413905252] conn(0x7f341007fc20 0x7f3410082090 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3410076cd0 msgr2=0x7f3410079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3410076cd0 0x7f3410079190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f341c00bb20 tx=0x7f341c005e50 comp rx=0 tx=0).stop 2024-07-17T08:49:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 msgr2=0x7f3430155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 --2- 172.21.15.44:0/21040930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f343014d9b0 0x7f3430155210 secure :-1 s=READY pgs=3526 cs=0 l=1 rev1=1 crypto rx=0x7f342800e960 tx=0x7f342800ee20 comp rx=0 tx=0).stop 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435200640 1 -- 172.21.15.44:0/21040930 reap_dead start 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 shutdown_connections 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.007+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 >> 172.21.15.44:0/21040930 conn(0x7f34300b55d0 msgr2=0x7f34300adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.008+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 shutdown_connections 2024-07-17T08:49:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.008+0000 7f3435c00640 1 -- 172.21.15.44:0/21040930 wait complete. 2024-07-17T08:49:25.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-07-17T08:49:25.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 -- 172.21.15.44:0/3634234778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 msgr2=0x7f2bb006ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 --2- 172.21.15.44:0/3634234778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb006ec70 secure :-1 s=READY pgs=3527 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005fd0 tx=0x7f2b9802a530 comp rx=0 tx=0).stop 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 -- 172.21.15.44:0/3634234778 shutdown_connections 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 --2- 172.21.15.44:0/3634234778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0152300 0x7f2bb006f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 --2- 172.21.15.44:0/3634234778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb006ec70 unknown :-1 s=CLOSED pgs=3527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 -- 172.21.15.44:0/3634234778 >> 172.21.15.44:0/3634234778 conn(0x7f2bb00cf0d0 msgr2=0x7f2bb00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 -- 172.21.15.44:0/3634234778 shutdown_connections 2024-07-17T08:49:25.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.237+0000 7f2bb6600640 1 -- 172.21.15.44:0/3634234778 wait complete. 2024-07-17T08:49:25.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.239+0000 7f2bb6600640 1 Processor -- start 2024-07-17T08:49:25.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.240+0000 7f2bb6600640 1 -- start start 2024-07-17T08:49:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 0x7f2bb013d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb0152300 0x7f2bb014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb00cb770 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb00cb5f0 con 0x7f2bb0152300 2024-07-17T08:49:25.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb00cb8f0 con 0x7f2bb0151f20 2024-07-17T08:49:25.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb5200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33624/0 (socket says 172.21.15.44:33624) 2024-07-17T08:49:25.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.241+0000 7f2bb5200640 1 -- 172.21.15.44:0/635271163 learned_addr learned my addr 172.21.15.44:0/635271163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5c00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb0152300 0x7f2bb014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bafe00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 0x7f2bb013d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 -- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 msgr2=0x7f2bb013d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 0x7f2bb013d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 -- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb0152300 msgr2=0x7f2bb014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb0152300 0x7f2bb014f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 -- 172.21.15.44:0/635271163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb014df70 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bafe00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 0x7f2bb013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.242+0000 7f2bb5200640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c430 tx=0x7f2ba000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.243+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0011790 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.243+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ba0017070 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.243+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb014e250 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.243+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0011d00 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.243+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb014eaf0 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.244+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb014e640 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.245+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f2ba001c460 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.246+0000 7f2bacc00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.246+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f2ba0098870 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.247+0000 7f2bafe00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.248+0000 7f2bafe00640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 0x7f2b8c079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b98033000 tx=0x7f2b98004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.250+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f2ba0061790 con 0x7f2bb00cd0b0 2024-07-17T08:49:25.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.464+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f2bb00b0840 con 0x7f2b8c076cd0 2024-07-17T08:49:25.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.466+0000 7f2bacc00640 1 -- 172.21.15.44:0/635271163 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2bb00b0840 con 0x7f2b8c076cd0 2024-07-17T08:49:25.467 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 5 to deep-scrub 2024-07-17T08:49:25.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 msgr2=0x7f2b8c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 0x7f2b8c079190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b98033000 tx=0x7f2b98004730 comp rx=0 tx=0).stop 2024-07-17T08:49:25.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 msgr2=0x7f2bb01f1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c430 tx=0x7f2ba000c8f0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 shutdown_connections 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f2b8c076cd0 0x7f2b8c079190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bb0152300 0x7f2bb014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bb0151f20 0x7f2bb013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 --2- 172.21.15.44:0/635271163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2bb00cd0b0 0x7f2bb01f1050 unknown :-1 s=CLOSED pgs=3528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 >> 172.21.15.44:0/635271163 conn(0x7f2bb00cf0d0 msgr2=0x7f2bb006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 shutdown_connections 2024-07-17T08:49:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.468+0000 7f2bb6600640 1 -- 172.21.15.44:0/635271163 wait complete. 2024-07-17T08:49:25.489 INFO:tasks.ceph:Scrubbing osd.6 2024-07-17T08:49:25.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 -- 172.21.15.44:0/2062480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 msgr2=0x7f4e240ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 --2- 172.21.15.44:0/2062480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e240ade70 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7f4e240b2e80 tx=0x7f4e1c02aac0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 -- 172.21.15.44:0/2062480391 shutdown_connections 2024-07-17T08:49:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 --2- 172.21.15.44:0/2062480391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e240ade70 unknown :-1 s=CLOSED pgs=3530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 --2- 172.21.15.44:0/2062480391 >> v2:172.21.15.175:3300/0 conn(0x7f4e2406bf40 0x7f4e2406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 -- 172.21.15.44:0/2062480391 >> 172.21.15.44:0/2062480391 conn(0x7f4e240b2240 msgr2=0x7f4e240b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 -- 172.21.15.44:0/2062480391 shutdown_connections 2024-07-17T08:49:25.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.706+0000 7f4e29600640 1 -- 172.21.15.44:0/2062480391 wait complete. 2024-07-17T08:49:25.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.708+0000 7f4e29600640 1 Processor -- start 2024-07-17T08:49:25.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.708+0000 7f4e29600640 1 -- start start 2024-07-17T08:49:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2406bf40 0x7f4e24156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e24157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e24140b70 0x7f4e24136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24140880 con 0x7f4e240ad9e0 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24140700 con 0x7f4e2406bf40 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e29600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e240ae3b0 con 0x7f4e24140b70 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e24157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e24157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33640/0 (socket says 172.21.15.44:33640) 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 -- 172.21.15.44:0/1779965834 learned_addr learned my addr 172.21.15.44:0/1779965834 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:25.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e23400640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e24140b70 0x7f4e24136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22a00640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2406bf40 0x7f4e24156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 -- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e24140b70 msgr2=0x7f4e24136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e24140b70 0x7f4e24136920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 -- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2406bf40 msgr2=0x7f4e24156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2406bf40 0x7f4e24156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.709+0000 7f4e22000640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241579e0 con 0x7f4e240ad9e0 2024-07-17T08:49:25.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e22000640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e24157300 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c3a0 tx=0x7f4e1000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10017070 con 0x7f4e240ad9e0 2024-07-17T08:49:25.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e100053c0 con 0x7f4e240ad9e0 2024-07-17T08:49:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e24155d70 con 0x7f4e240ad9e0 2024-07-17T08:49:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10003850 con 0x7f4e240ad9e0 2024-07-17T08:49:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.710+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241562f0 con 0x7f4e240ad9e0 2024-07-17T08:49:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.711+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e241566c0 con 0x7f4e240ad9e0 2024-07-17T08:49:25.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.713+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f4e1001a070 con 0x7f4e240ad9e0 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e1aa00640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4e04076d00 0x7f4e040791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f4e10097c10 con 0x7f4e240ad9e0 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e1aa00640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f4e0407fc50 0x7f4e040820c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e1001de50 con 0x7f4e0407fc50 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7f4e1009a310 con 0x7f4e240ad9e0 2024-07-17T08:49:25.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.714+0000 7f4e22a00640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4e04076d00 0x7f4e040791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.715+0000 7f4e23400640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f4e0407fc50 0x7f4e040820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.716+0000 7f4e22a00640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4e04076d00 0x7f4e040791c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005fd0 tx=0x7f4e14005850 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.716+0000 7f4e23400640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f4e0407fc50 0x7f4e040820c0 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.718+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== osd.6 v2:172.21.15.175:6803/302428054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e1001de50 con 0x7f4e0407fc50 2024-07-17T08:49:25.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.733+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 --> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4e2406c320 con 0x7f4e0407fc50 2024-07-17T08:49:25.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e1aa00640 1 -- 172.21.15.44:0/1779965834 <== osd.6 v2:172.21.15.175:6803/302428054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4e2406c320 con 0x7f4e0407fc50 2024-07-17T08:49:25.736 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:25.736 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:25.736 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:25.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f4e0407fc50 msgr2=0x7f4e040820c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6803/302428054,v1:172.21.15.175:6805/302428054] conn(0x7f4e0407fc50 0x7f4e040820c0 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4e04076d00 msgr2=0x7f4e040791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f4e04076d00 0x7f4e040791c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005fd0 tx=0x7f4e14005850 comp rx=0 tx=0).stop 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 msgr2=0x7f4e24157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 --2- 172.21.15.44:0/1779965834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e240ad9e0 0x7f4e24157300 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c3a0 tx=0x7f4e1000c860 comp rx=0 tx=0).stop 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e23400640 1 -- 172.21.15.44:0/1779965834 reap_dead start 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 shutdown_connections 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 >> 172.21.15.44:0/1779965834 conn(0x7f4e240b2240 msgr2=0x7f4e2413e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 shutdown_connections 2024-07-17T08:49:25.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.735+0000 7f4e29600640 1 -- 172.21.15.44:0/1779965834 wait complete. 2024-07-17T08:49:25.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-07-17T08:49:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 -- 172.21.15.44:0/2301069698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 msgr2=0x7f9d6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/2301069698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d6c004c80 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000ac30 tx=0x7f9d6c005830 comp rx=0 tx=0).stop 2024-07-17T08:49:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8a200640 1 -- 172.21.15.44:0/2301069698 reap_dead start 2024-07-17T08:49:25.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 -- 172.21.15.44:0/2301069698 shutdown_connections 2024-07-17T08:49:25.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 -- 172.21.15.44:0/2301069698 >> 172.21.15.44:0/2301069698 conn(0x7f9d840cf0d0 msgr2=0x7f9d840cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:25.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 -- 172.21.15.44:0/2301069698 shutdown_connections 2024-07-17T08:49:25.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.969+0000 7f9d8ac00640 1 -- 172.21.15.44:0/2301069698 wait complete. 2024-07-17T08:49:25.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.971+0000 7f9d8ac00640 1 Processor -- start 2024-07-17T08:49:25.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.971+0000 7f9d8ac00640 1 -- start start 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 0x7f9d84140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d84073ce0 0x7f9d841414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8406dc40 con 0x7f9d6c004800 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8406dac0 con 0x7f9d84073860 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d8ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d841503c0 con 0x7f9d84073ce0 2024-07-17T08:49:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d89800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.972+0000 7f9d89800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33668/0 (socket says 172.21.15.44:33668) 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 -- 172.21.15.44:0/609245073 learned_addr learned my addr 172.21.15.44:0/609245073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d88e00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 0x7f9d84140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d8a200640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d84073ce0 0x7f9d841414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 -- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d84073ce0 msgr2=0x7f9d841414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d84073ce0 0x7f9d841414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 -- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 msgr2=0x7f9d84140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 0x7f9d84140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 -- 172.21.15.44:0/609245073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d841f6300 con 0x7f9d6c004800 2024-07-17T08:49:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d88e00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 0x7f9d84140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:25.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.973+0000 7f9d89800640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800daf0 tx=0x7f9d7800b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.974+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78018070 con 0x7f9d6c004800 2024-07-17T08:49:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.974+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d78003b80 con 0x7f9d6c004800 2024-07-17T08:49:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.974+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800b910 con 0x7f9d6c004800 2024-07-17T08:49:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.974+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d841e3660 con 0x7f9d6c004800 2024-07-17T08:49:25.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.974+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841e3ec0 con 0x7f9d6c004800 2024-07-17T08:49:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.975+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d840ac010 con 0x7f9d6c004800 2024-07-17T08:49:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.976+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f9d7801b070 con 0x7f9d6c004800 2024-07-17T08:49:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.977+0000 7f9d81600640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 0x7f9d64079210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:25.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.977+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f9d78098480 con 0x7f9d6c004800 2024-07-17T08:49:25.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.978+0000 7f9d88e00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 0x7f9d64079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:25.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.979+0000 7f9d88e00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 0x7f9d64079210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036000 tx=0x7f9d700046c0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:25.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:25.981+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f9d841f6300 con 0x7f9d6c004800 2024-07-17T08:49:26.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.193+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f9d840cffe0 con 0x7f9d64076d50 2024-07-17T08:49:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.195+0000 7f9d81600640 1 -- 172.21.15.44:0/609245073 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9d840cffe0 con 0x7f9d64076d50 2024-07-17T08:49:26.196 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 6 to deep-scrub 2024-07-17T08:49:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 msgr2=0x7f9d64079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 0x7f9d64079210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036000 tx=0x7f9d700046c0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 msgr2=0x7f9d841f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800daf0 tx=0x7f9d7800b0e0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 shutdown_connections 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f9d64076d50 0x7f9d64079210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d84073ce0 0x7f9d841414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d84073860 0x7f9d84140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 --2- 172.21.15.44:0/609245073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d6c004800 0x7f9d841f5dc0 unknown :-1 s=CLOSED pgs=3534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 >> 172.21.15.44:0/609245073 conn(0x7f9d840cf0d0 msgr2=0x7f9d8415b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:26.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 shutdown_connections 2024-07-17T08:49:26.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.197+0000 7f9d8ac00640 1 -- 172.21.15.44:0/609245073 wait complete. 2024-07-17T08:49:26.219 INFO:tasks.ceph:Scrubbing osd.7 2024-07-17T08:49:26.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-07-17T08:49:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 -- 172.21.15.44:0/1104471658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 msgr2=0x7f604c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 --2- 172.21.15.44:0/1104471658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0d0430 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7f6038009830 tx=0x7f603802a3f0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 -- 172.21.15.44:0/1104471658 shutdown_connections 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 --2- 172.21.15.44:0/1104471658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0d0430 unknown :-1 s=CLOSED pgs=3536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 -- 172.21.15.44:0/1104471658 >> 172.21.15.44:0/1104471658 conn(0x7f604c073a60 msgr2=0x7f604c073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 -- 172.21.15.44:0/1104471658 shutdown_connections 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.439+0000 7f6050a00640 1 -- 172.21.15.44:0/1104471658 wait complete. 2024-07-17T08:49:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.441+0000 7f6050a00640 1 Processor -- start 2024-07-17T08:49:26.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.441+0000 7f6050a00640 1 -- start start 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0d0970 0x7f604c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f604c15e370 0x7f604c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0b8f00 con 0x7f604c161b50 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0b8d80 con 0x7f604c0d0970 2024-07-17T08:49:26.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f6050a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f604c0b9080 con 0x7f604c15e370 2024-07-17T08:49:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f604be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f604be00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33698/0 (socket says 172.21.15.44:33698) 2024-07-17T08:49:26.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.442+0000 7f604be00640 1 -- 172.21.15.44:0/2620672946 learned_addr learned my addr 172.21.15.44:0/2620672946 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604b400640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0d0970 0x7f604c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604aa00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f604c15e370 0x7f604c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 -- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f604c15e370 msgr2=0x7f604c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f604c15e370 0x7f604c0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 -- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0d0970 msgr2=0x7f604c1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f604c0d0970 0x7f604c1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c0cd430 con 0x7f604c161b50 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.443+0000 7f604be00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0b2d70 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7f604400c780 tx=0x7f604400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.444+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044017070 con 0x7f604c161b50 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.444+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6044005070 con 0x7f604c161b50 2024-07-17T08:49:26.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.444+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044005500 con 0x7f604c161b50 2024-07-17T08:49:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.444+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c0cd710 con 0x7f604c161b50 2024-07-17T08:49:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.444+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1f8bc0 con 0x7f604c161b50 2024-07-17T08:49:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.445+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f604c0cfbf0 con 0x7f604c161b50 2024-07-17T08:49:26.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.446+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f60440056a0 con 0x7f604c161b50 2024-07-17T08:49:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.447+0000 7f6043400640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f602c076cd0 0x7f602c079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.447+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f6044097ed0 con 0x7f604c161b50 2024-07-17T08:49:26.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.448+0000 7f6043400640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f602c07fc30 0x7f602c0820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.448+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f602c0826d0 con 0x7f602c07fc30 2024-07-17T08:49:26.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.448+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=379) ==== 24+0+0 (secure 0 0 0) 0x7f604409a4f0 con 0x7f604c161b50 2024-07-17T08:49:26.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.448+0000 7f604b400640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f602c076cd0 0x7f602c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.448+0000 7f604aa00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f602c07fc30 0x7f602c0820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.449+0000 7f604aa00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f602c07fc30 0x7f602c0820a0 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:26.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.449+0000 7f604b400640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f602c076cd0 0x7f602c079190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f603c001700 tx=0x7f603c009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:26.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.451+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== osd.7 v2:172.21.15.175:6826/2516790419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f602c0826d0 con 0x7f602c07fc30 2024-07-17T08:49:26.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.466+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 --> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f604c0d0430 con 0x7f602c07fc30 2024-07-17T08:49:26.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.467+0000 7f6043400640 1 -- 172.21.15.44:0/2620672946 <== osd.7 v2:172.21.15.175:6826/2516790419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f604c0d0430 con 0x7f602c07fc30 2024-07-17T08:49:26.469 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-07-17T08:49:26.469 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-17T08:49:26.469 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-07-17T08:49:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f602c07fc30 msgr2=0x7f602c0820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6826/2516790419,v1:172.21.15.175:6827/2516790419] conn(0x7f602c07fc30 0x7f602c0820a0 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f602c076cd0 msgr2=0x7f602c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f602c076cd0 0x7f602c079190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f603c001700 tx=0x7f603c009040 comp rx=0 tx=0).stop 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 msgr2=0x7f604c0b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 --2- 172.21.15.44:0/2620672946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f604c161b50 0x7f604c0b2d70 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7f604400c780 tx=0x7f604400cc40 comp rx=0 tx=0).stop 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f604be00640 1 -- 172.21.15.44:0/2620672946 reap_dead start 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 shutdown_connections 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 >> 172.21.15.44:0/2620672946 conn(0x7f604c073a60 msgr2=0x7f604c15b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 shutdown_connections 2024-07-17T08:49:26.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.468+0000 7f6050a00640 1 -- 172.21.15.44:0/2620672946 wait complete. 2024-07-17T08:49:26.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-07-17T08:49:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 -- 172.21.15.44:0/2361850123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3054003b10 msgr2=0x7f3054003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 --2- 172.21.15.44:0/2361850123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3054003b10 0x7f3054003f30 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f3060009b50 tx=0x7f3054005830 comp rx=0 tx=0).stop 2024-07-17T08:49:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 -- 172.21.15.44:0/2361850123 shutdown_connections 2024-07-17T08:49:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 --2- 172.21.15.44:0/2361850123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3054003b10 0x7f3054003f30 unknown :-1 s=CLOSED pgs=3539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 -- 172.21.15.44:0/2361850123 >> 172.21.15.44:0/2361850123 conn(0x7f30700b4ff0 msgr2=0x7f30700b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:26.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 -- 172.21.15.44:0/2361850123 shutdown_connections 2024-07-17T08:49:26.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.707+0000 7f3075000640 1 -- 172.21.15.44:0/2361850123 wait complete. 2024-07-17T08:49:26.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.709+0000 7f3075000640 1 Processor -- start 2024-07-17T08:49:26.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.709+0000 7f3075000640 1 -- start start 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3054003b10 0x7f30701566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070157cb0 0x7f307015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f30700b28c0 con 0x7f3070157830 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30700b2740 con 0x7f3054003b10 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f3075000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30700cfdf0 con 0x7f3070157cb0 2024-07-17T08:49:26.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f306ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f306ea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33716/0 (socket says 172.21.15.44:33716) 2024-07-17T08:49:26.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.710+0000 7f306ea00640 1 -- 172.21.15.44:0/3505041058 learned_addr learned my addr 172.21.15.44:0/3505041058 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:26.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 -- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070157cb0 msgr2=0x7f307015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070157cb0 0x7f307015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 -- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3054003b10 msgr2=0x7f30701566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3054003b10 0x7f30701566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 -- 172.21.15.44:0/3505041058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3070156c00 con 0x7f3070157830 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f306ea00640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c390 tx=0x7f305c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0052d0 con 0x7f3070157830 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f305c007020 con 0x7f3070157830 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c007470 con 0x7f3070157830 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3070158210 con 0x7f3070157830 2024-07-17T08:49:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.711+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3070153c80 con 0x7f3070157830 2024-07-17T08:49:26.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.713+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30700d0350 con 0x7f3070157830 2024-07-17T08:49:26.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.714+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f305c0067d0 con 0x7f3070157830 2024-07-17T08:49:26.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.715+0000 7f3067400640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 0x7f30500791c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.715+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f305c097d50 con 0x7f3070157830 2024-07-17T08:49:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.716+0000 7f306f400640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 0x7f30500791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:26.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.717+0000 7f306f400640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 0x7f30500791c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3060035000 tx=0x7f3060004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:26.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.720+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f305c09b030 con 0x7f3070157830 2024-07-17T08:49:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.930+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f30700d0e10 con 0x7f3050076d00 2024-07-17T08:49:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.931+0000 7f3067400640 1 -- 172.21.15.44:0/3505041058 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f30700d0e10 con 0x7f3050076d00 2024-07-17T08:49:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 7 to deep-scrub 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.933+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 msgr2=0x7f30500791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.933+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 0x7f30500791c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3060035000 tx=0x7f3060004730 comp rx=0 tx=0).stop 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.933+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 msgr2=0x7f307015b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.933+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f305c00c390 tx=0x7f305c00c850 comp rx=0 tx=0).stop 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 shutdown_connections 2024-07-17T08:49:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f3050076d00 0x7f30500791c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070157cb0 0x7f307015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3070157830 0x7f307015b4f0 unknown :-1 s=CLOSED pgs=3540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 --2- 172.21.15.44:0/3505041058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3054003b10 0x7f30701566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 >> 172.21.15.44:0/3505041058 conn(0x7f30700b4ff0 msgr2=0x7f30700b8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 shutdown_connections 2024-07-17T08:49:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:26.934+0000 7f3075000640 1 -- 172.21.15.44:0/3505041058 wait complete. 2024-07-17T08:49:26.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:49:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 -- 172.21.15.44:0/765155955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc06f710 msgr2=0x7f25dc153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 --2- 172.21.15.44:0/765155955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc06f710 0x7f25dc153a50 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00a940 tx=0x7f25cc02b850 comp rx=0 tx=0).stop 2024-07-17T08:49:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 -- 172.21.15.44:0/765155955 shutdown_connections 2024-07-17T08:49:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 --2- 172.21.15.44:0/765155955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc0ac3e0 0x7f25dc0ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 --2- 172.21.15.44:0/765155955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc06f710 0x7f25dc153a50 unknown :-1 s=CLOSED pgs=3542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 -- 172.21.15.44:0/765155955 >> 172.21.15.44:0/765155955 conn(0x7f25dc0b0fb0 msgr2=0x7f25dc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 -- 172.21.15.44:0/765155955 shutdown_connections 2024-07-17T08:49:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.168+0000 7f25e2400640 1 -- 172.21.15.44:0/765155955 wait complete. 2024-07-17T08:49:27.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.170+0000 7f25e2400640 1 Processor -- start 2024-07-17T08:49:27.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.170+0000 7f25e2400640 1 -- start start 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc06f710 0x7f25dc1f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 0x7f25dc1f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc0ae800 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc06f0a0 con 0x7f25dc13ed60 2024-07-17T08:49:27.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25e2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc0ae980 con 0x7f25dc06f710 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.171+0000 7f25daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33728/0 (socket says 172.21.15.44:33728) 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 -- 172.21.15.44:0/754132107 learned_addr learned my addr 172.21.15.44:0/754132107 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25db400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc06f710 0x7f25dc1f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25dbe00640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 0x7f25dc1f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 -- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc06f710 msgr2=0x7f25dc1f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc06f710 0x7f25dc1f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 -- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 msgr2=0x7f25dc1f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 0x7f25dc1f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 -- 172.21.15.44:0/754132107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc0cb3c0 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25dbe00640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 0x7f25dc1f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:27.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.172+0000 7f25daa00640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d950 tx=0x7f25c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.173+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.173+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c4009d20 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.173+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40106c0 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.173+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc142e80 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.173+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc099790 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.174+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc13fad0 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.175+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f25c4010860 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.176+0000 7f25d3400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 0x7f25b8079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.176+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f25c4098de0 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.177+0000 7f25db400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 0x7f25b8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.178+0000 7f25db400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 0x7f25b8079190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25cc036000 tx=0x7f25cc004730 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.181+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f25c4061bf0 con 0x7f25dc0ac3e0 2024-07-17T08:49:27.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.392+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25dc143050 con 0x7f25b8076cd0 2024-07-17T08:49:27.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.405+0000 7f25d3400640 1 -- 172.21.15.44:0/754132107 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251509 (secure 0 0 0) 0x7f25dc143050 con 0x7f25b8076cd0 2024-07-17T08:49:27.409 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:27.412 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:49:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 msgr2=0x7f25b8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 0x7f25b8079190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25cc036000 tx=0x7f25cc004730 comp rx=0 tx=0).stop 2024-07-17T08:49:27.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 msgr2=0x7f25dc09ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d950 tx=0x7f25c400de10 comp rx=0 tx=0).stop 2024-07-17T08:49:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 shutdown_connections 2024-07-17T08:49:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f25b8076cd0 0x7f25b8079190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc13ed60 0x7f25dc1f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25dc0ac3e0 0x7f25dc09ccf0 unknown :-1 s=CLOSED pgs=3543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 --2- 172.21.15.44:0/754132107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25dc06f710 0x7f25dc1f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:27.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 >> 172.21.15.44:0/754132107 conn(0x7f25dc0b0fb0 msgr2=0x7f25dc15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:27.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 shutdown_connections 2024-07-17T08:49:27.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:27.412+0000 7f25e2400640 1 -- 172.21.15.44:0/754132107 wait complete. 2024-07-17T08:49:27.434 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-07-17T08:49:25.936222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594165,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":642,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4725,"ondisk_log_size":4725,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546552,"kb_used_data":38920,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428168,"statfs":{"total":773094113280,"available":772534444032,"internally_reserved":0,"allocated":39854080,"data_stored":22551474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004207"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:12.905392+0000","last_change":"2024-07-17T08:47:06.140155+0000","last_active":"2024-07-17T08:49:12.905392+0000","last_peered":"2024-07-17T08:49:12.905392+0000","last_clean":"2024-07-17T08:49:12.905392+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:49:12.905392+0000","last_undegraded":"2024-07-17T08:49:12.905392+0000","last_fullsized":"2024-07-17T08:49:12.905392+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:12.911500+0000","last_change":"2024-07-17T08:47:06.143704+0000","last_active":"2024-07-17T08:49:12.911500+0000","last_peered":"2024-07-17T08:49:12.911500+0000","last_clean":"2024-07-17T08:49:12.911500+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:49:12.911500+0000","last_undegraded":"2024-07-17T08:49:12.911500+0000","last_fullsized":"2024-07-17T08:49:12.911500+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890224+0000","last_change":"2024-07-17T08:47:06.139258+0000","last_active":"2024-07-17T08:47:07.890224+0000","last_peered":"2024-07-17T08:47:07.890224+0000","last_clean":"2024-07-17T08:47:07.890224+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:47:07.890224+0000","last_undegraded":"2024-07-17T08:47:07.890224+0000","last_fullsized":"2024-07-17T08:47:07.890224+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:40:38.637384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888208+0000","last_change":"2024-07-17T08:47:06.137093+0000","last_active":"2024-07-17T08:47:07.888208+0000","last_peered":"2024-07-17T08:47:07.888208+0000","last_clean":"2024-07-17T08:47:07.888208+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:47:07.888208+0000","last_undegraded":"2024-07-17T08:47:07.888208+0000","last_fullsized":"2024-07-17T08:47:07.888208+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:06:33.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140912+0000","last_change":"2024-07-17T08:47:06.135709+0000","last_active":"2024-07-17T08:47:07.140912+0000","last_peered":"2024-07-17T08:47:07.140912+0000","last_clean":"2024-07-17T08:47:07.140912+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:47:07.140912+0000","last_undegraded":"2024-07-17T08:47:07.140912+0000","last_fullsized":"2024-07-17T08:47:07.140912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:34:58.434973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272970+0000","last_change":"2024-07-17T08:47:06.131922+0000","last_active":"2024-07-17T08:47:07.272970+0000","last_peered":"2024-07-17T08:47:07.272970+0000","last_clean":"2024-07-17T08:47:07.272970+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:47:07.272970+0000","last_undegraded":"2024-07-17T08:47:07.272970+0000","last_fullsized":"2024-07-17T08:47:07.272970+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:54:40.771270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143161+0000","last_change":"2024-07-17T08:47:06.123627+0000","last_active":"2024-07-17T08:47:07.143161+0000","last_peered":"2024-07-17T08:47:07.143161+0000","last_clean":"2024-07-17T08:47:07.143161+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:47:07.143161+0000","last_undegraded":"2024-07-17T08:47:07.143161+0000","last_fullsized":"2024-07-17T08:47:07.143161+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:22:46.658125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143081+0000","last_change":"2024-07-17T08:47:06.117911+0000","last_active":"2024-07-17T08:47:07.143081+0000","last_peered":"2024-07-17T08:47:07.143081+0000","last_clean":"2024-07-17T08:47:07.143081+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:47:07.143081+0000","last_undegraded":"2024-07-17T08:47:07.143081+0000","last_fullsized":"2024-07-17T08:47:07.143081+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:24:07.233067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890238+0000","last_change":"2024-07-17T08:47:06.147281+0000","last_active":"2024-07-17T08:47:07.890238+0000","last_peered":"2024-07-17T08:47:07.890238+0000","last_clean":"2024-07-17T08:47:07.890238+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:47:07.890238+0000","last_undegraded":"2024-07-17T08:47:07.890238+0000","last_fullsized":"2024-07-17T08:47:07.890238+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884680+0000","last_change":"2024-07-17T08:47:06.142034+0000","last_active":"2024-07-17T08:47:07.884680+0000","last_peered":"2024-07-17T08:47:07.884680+0000","last_clean":"2024-07-17T08:47:07.884680+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:47:07.884680+0000","last_undegraded":"2024-07-17T08:47:07.884680+0000","last_fullsized":"2024-07-17T08:47:07.884680+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143912+0000","last_change":"2024-07-17T08:47:06.147569+0000","last_active":"2024-07-17T08:47:07.143912+0000","last_peered":"2024-07-17T08:47:07.143912+0000","last_clean":"2024-07-17T08:47:07.143912+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:47:07.143912+0000","last_undegraded":"2024-07-17T08:47:07.143912+0000","last_fullsized":"2024-07-17T08:47:07.143912+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:29.200022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273393+0000","last_change":"2024-07-17T08:47:06.143086+0000","last_active":"2024-07-17T08:47:07.273393+0000","last_peered":"2024-07-17T08:47:07.273393+0000","last_clean":"2024-07-17T08:47:07.273393+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:47:07.273393+0000","last_undegraded":"2024-07-17T08:47:07.273393+0000","last_fullsized":"2024-07-17T08:47:07.273393+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:30:51.593361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890015+0000","last_change":"2024-07-17T08:47:06.142933+0000","last_active":"2024-07-17T08:47:07.890015+0000","last_peered":"2024-07-17T08:47:07.890015+0000","last_clean":"2024-07-17T08:47:07.890015+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:47:07.890015+0000","last_undegraded":"2024-07-17T08:47:07.890015+0000","last_fullsized":"2024-07-17T08:47:07.890015+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884362+0000","last_change":"2024-07-17T08:47:06.143856+0000","last_active":"2024-07-17T08:47:07.884362+0000","last_peered":"2024-07-17T08:47:07.884362+0000","last_clean":"2024-07-17T08:47:07.884362+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:47:07.884362+0000","last_undegraded":"2024-07-17T08:47:07.884362+0000","last_fullsized":"2024-07-17T08:47:07.884362+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884309+0000","last_change":"2024-07-17T08:47:06.148417+0000","last_active":"2024-07-17T08:47:07.884309+0000","last_peered":"2024-07-17T08:47:07.884309+0000","last_clean":"2024-07-17T08:47:07.884309+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:47:07.884309+0000","last_undegraded":"2024-07-17T08:47:07.884309+0000","last_fullsized":"2024-07-17T08:47:07.884309+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890255+0000","last_change":"2024-07-17T08:47:06.145798+0000","last_active":"2024-07-17T08:47:07.890255+0000","last_peered":"2024-07-17T08:47:07.890255+0000","last_clean":"2024-07-17T08:47:07.890255+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:47:07.890255+0000","last_undegraded":"2024-07-17T08:47:07.890255+0000","last_fullsized":"2024-07-17T08:47:07.890255+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884713+0000","last_change":"2024-07-17T08:47:06.133870+0000","last_active":"2024-07-17T08:47:07.884713+0000","last_peered":"2024-07-17T08:47:07.884713+0000","last_clean":"2024-07-17T08:47:07.884713+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:47:07.884713+0000","last_undegraded":"2024-07-17T08:47:07.884713+0000","last_fullsized":"2024-07-17T08:47:07.884713+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877608+0000","last_change":"2024-07-17T08:47:06.141701+0000","last_active":"2024-07-17T08:47:07.877608+0000","last_peered":"2024-07-17T08:47:07.877608+0000","last_clean":"2024-07-17T08:47:07.877608+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:47:07.877608+0000","last_undegraded":"2024-07-17T08:47:07.877608+0000","last_fullsized":"2024-07-17T08:47:07.877608+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:12:18.163822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888271+0000","last_change":"2024-07-17T08:47:06.131141+0000","last_active":"2024-07-17T08:47:07.888271+0000","last_peered":"2024-07-17T08:47:07.888271+0000","last_clean":"2024-07-17T08:47:07.888271+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:47:07.888271+0000","last_undegraded":"2024-07-17T08:47:07.888271+0000","last_fullsized":"2024-07-17T08:47:07.888271+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:12:49.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:21.853110+0000","last_change":"2024-07-17T08:47:06.146989+0000","last_active":"2024-07-17T08:49:21.853110+0000","last_peered":"2024-07-17T08:49:21.853110+0000","last_clean":"2024-07-17T08:49:21.853110+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:49:21.853110+0000","last_undegraded":"2024-07-17T08:49:21.853110+0000","last_fullsized":"2024-07-17T08:49:21.853110+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877440+0000","last_change":"2024-07-17T08:47:06.145026+0000","last_active":"2024-07-17T08:47:07.877440+0000","last_peered":"2024-07-17T08:47:07.877440+0000","last_clean":"2024-07-17T08:47:07.877440+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:47:07.877440+0000","last_undegraded":"2024-07-17T08:47:07.877440+0000","last_fullsized":"2024-07-17T08:47:07.877440+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:58:04.326976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877351+0000","last_change":"2024-07-17T08:47:06.139324+0000","last_active":"2024-07-17T08:47:07.877351+0000","last_peered":"2024-07-17T08:47:07.877351+0000","last_clean":"2024-07-17T08:47:07.877351+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:47:07.877351+0000","last_undegraded":"2024-07-17T08:47:07.877351+0000","last_fullsized":"2024-07-17T08:47:07.877351+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:17:59.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888113+0000","last_change":"2024-07-17T08:47:06.131071+0000","last_active":"2024-07-17T08:47:07.888113+0000","last_peered":"2024-07-17T08:47:07.888113+0000","last_clean":"2024-07-17T08:47:07.888113+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:47:07.888113+0000","last_undegraded":"2024-07-17T08:47:07.888113+0000","last_fullsized":"2024-07-17T08:47:07.888113+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:41:25.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889751+0000","last_change":"2024-07-17T08:47:06.142842+0000","last_active":"2024-07-17T08:47:07.889751+0000","last_peered":"2024-07-17T08:47:07.889751+0000","last_clean":"2024-07-17T08:47:07.889751+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:47:07.889751+0000","last_undegraded":"2024-07-17T08:47:07.889751+0000","last_fullsized":"2024-07-17T08:47:07.889751+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:28:14.465902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869495+0000","last_change":"2024-07-17T08:47:06.135178+0000","last_active":"2024-07-17T08:47:07.869495+0000","last_peered":"2024-07-17T08:47:07.869495+0000","last_clean":"2024-07-17T08:47:07.869495+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:47:07.869495+0000","last_undegraded":"2024-07-17T08:47:07.869495+0000","last_fullsized":"2024-07-17T08:47:07.869495+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:32:23.301241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886129+0000","last_change":"2024-07-17T08:47:06.136573+0000","last_active":"2024-07-17T08:47:07.886129+0000","last_peered":"2024-07-17T08:47:07.886129+0000","last_clean":"2024-07-17T08:47:07.886129+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:47:07.886129+0000","last_undegraded":"2024-07-17T08:47:07.886129+0000","last_fullsized":"2024-07-17T08:47:07.886129+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:51.094377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886055+0000","last_change":"2024-07-17T08:47:06.139264+0000","last_active":"2024-07-17T08:47:07.886055+0000","last_peered":"2024-07-17T08:47:07.886055+0000","last_clean":"2024-07-17T08:47:07.886055+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:47:07.886055+0000","last_undegraded":"2024-07-17T08:47:07.886055+0000","last_fullsized":"2024-07-17T08:47:07.886055+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:38:37.624417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886018+0000","last_change":"2024-07-17T08:47:06.138326+0000","last_active":"2024-07-17T08:47:07.886018+0000","last_peered":"2024-07-17T08:47:07.886018+0000","last_clean":"2024-07-17T08:47:07.886018+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:47:07.886018+0000","last_undegraded":"2024-07-17T08:47:07.886018+0000","last_fullsized":"2024-07-17T08:47:07.886018+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:47:59.971550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885983+0000","last_change":"2024-07-17T08:47:06.139702+0000","last_active":"2024-07-17T08:47:07.885983+0000","last_peered":"2024-07-17T08:47:07.885983+0000","last_clean":"2024-07-17T08:47:07.885983+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:47:07.885983+0000","last_undegraded":"2024-07-17T08:47:07.885983+0000","last_fullsized":"2024-07-17T08:47:07.885983+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:32:40.342156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890062+0000","last_change":"2024-07-17T08:47:06.140413+0000","last_active":"2024-07-17T08:47:07.890062+0000","last_peered":"2024-07-17T08:47:07.890062+0000","last_clean":"2024-07-17T08:47:07.890062+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:47:07.890062+0000","last_undegraded":"2024-07-17T08:47:07.890062+0000","last_fullsized":"2024-07-17T08:47:07.890062+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:09:25.230186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889946+0000","last_change":"2024-07-17T08:47:06.140908+0000","last_active":"2024-07-17T08:47:07.889946+0000","last_peered":"2024-07-17T08:47:07.889946+0000","last_clean":"2024-07-17T08:47:07.889946+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:47:07.889946+0000","last_undegraded":"2024-07-17T08:47:07.889946+0000","last_fullsized":"2024-07-17T08:47:07.889946+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:25:31.892696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889909+0000","last_change":"2024-07-17T08:47:06.141979+0000","last_active":"2024-07-17T08:47:07.889909+0000","last_peered":"2024-07-17T08:47:07.889909+0000","last_clean":"2024-07-17T08:47:07.889909+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:47:07.889909+0000","last_undegraded":"2024-07-17T08:47:07.889909+0000","last_fullsized":"2024-07-17T08:47:07.889909+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"378'4","reported_seq":19,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886272+0000","last_change":"2024-07-17T08:47:05.135506+0000","last_active":"2024-07-17T08:47:07.886272+0000","last_peered":"2024-07-17T08:47:07.886272+0000","last_clean":"2024-07-17T08:47:07.886272+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:47:07.886272+0000","last_undegraded":"2024-07-17T08:47:07.886272+0000","last_fullsized":"2024-07-17T08:47:07.886272+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:31:43.112544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141891+0000","last_change":"2024-07-17T08:47:06.115336+0000","last_active":"2024-07-17T08:47:07.141891+0000","last_peered":"2024-07-17T08:47:07.141891+0000","last_clean":"2024-07-17T08:47:07.141891+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:47:07.141891+0000","last_undegraded":"2024-07-17T08:47:07.141891+0000","last_fullsized":"2024-07-17T08:47:07.141891+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:59:02.099820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":25,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:22.334604+0000","last_change":"2024-07-17T08:49:22.334668+0000","last_active":"2024-07-17T08:49:22.334604+0000","last_peered":"2024-07-17T08:49:22.334604+0000","last_clean":"2024-07-17T08:49:22.334604+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:49:22.334604+0000","last_undegraded":"2024-07-17T08:49:22.334604+0000","last_fullsized":"2024-07-17T08:49:22.334604+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"378'1","last_scrub_stamp":"2024-07-17T08:49:22.334428+0000","last_deep_scrub":"378'1","last_deep_scrub_stamp":"2024-07-17T08:49:22.334428+0000","last_clean_scrub_stamp":"2024-07-17T08:49:22.334428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.012660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141805+0000","last_change":"2024-07-17T08:47:06.136418+0000","last_active":"2024-07-17T08:47:07.141805+0000","last_peered":"2024-07-17T08:47:07.141805+0000","last_clean":"2024-07-17T08:47:07.141805+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:47:07.141805+0000","last_undegraded":"2024-07-17T08:47:07.141805+0000","last_fullsized":"2024-07-17T08:47:07.141805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:17:52.714571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884641+0000","last_change":"2024-07-17T08:47:05.126823+0000","last_active":"2024-07-17T08:47:07.884641+0000","last_peered":"2024-07-17T08:47:07.884641+0000","last_clean":"2024-07-17T08:47:07.884641+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:47:07.884641+0000","last_undegraded":"2024-07-17T08:47:07.884641+0000","last_fullsized":"2024-07-17T08:47:07.884641+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884512+0000","last_change":"2024-07-17T08:47:06.143703+0000","last_active":"2024-07-17T08:47:07.884512+0000","last_peered":"2024-07-17T08:47:07.884512+0000","last_clean":"2024-07-17T08:47:07.884512+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:47:07.884512+0000","last_undegraded":"2024-07-17T08:47:07.884512+0000","last_fullsized":"2024-07-17T08:47:07.884512+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"378'2","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272936+0000","last_change":"2024-07-17T08:47:05.139116+0000","last_active":"2024-07-17T08:47:07.272936+0000","last_peered":"2024-07-17T08:47:07.272936+0000","last_clean":"2024-07-17T08:47:07.272936+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:47:07.272936+0000","last_undegraded":"2024-07-17T08:47:07.272936+0000","last_fullsized":"2024-07-17T08:47:07.272936+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:31:05.116408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884604+0000","last_change":"2024-07-17T08:47:06.152866+0000","last_active":"2024-07-17T08:47:07.884604+0000","last_peered":"2024-07-17T08:47:07.884604+0000","last_clean":"2024-07-17T08:47:07.884604+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:47:07.884604+0000","last_undegraded":"2024-07-17T08:47:07.884604+0000","last_fullsized":"2024-07-17T08:47:07.884604+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273399+0000","last_change":"2024-07-17T08:47:05.135397+0000","last_active":"2024-07-17T08:47:07.273399+0000","last_peered":"2024-07-17T08:47:07.273399+0000","last_clean":"2024-07-17T08:47:07.273399+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:47:07.273399+0000","last_undegraded":"2024-07-17T08:47:07.273399+0000","last_fullsized":"2024-07-17T08:47:07.273399+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:10:42.476368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890185+0000","last_change":"2024-07-17T08:47:06.141288+0000","last_active":"2024-07-17T08:47:07.890185+0000","last_peered":"2024-07-17T08:47:07.890185+0000","last_clean":"2024-07-17T08:47:07.890185+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:47:07.890185+0000","last_undegraded":"2024-07-17T08:47:07.890185+0000","last_fullsized":"2024-07-17T08:47:07.890185+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890057+0000","last_change":"2024-07-17T08:47:05.133953+0000","last_active":"2024-07-17T08:47:07.890057+0000","last_peered":"2024-07-17T08:47:07.890057+0000","last_clean":"2024-07-17T08:47:07.890057+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:47:07.890057+0000","last_undegraded":"2024-07-17T08:47:07.890057+0000","last_fullsized":"2024-07-17T08:47:07.890057+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889720+0000","last_change":"2024-07-17T08:47:06.132327+0000","last_active":"2024-07-17T08:47:07.889720+0000","last_peered":"2024-07-17T08:47:07.889720+0000","last_clean":"2024-07-17T08:47:07.889720+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:47:07.889720+0000","last_undegraded":"2024-07-17T08:47:07.889720+0000","last_fullsized":"2024-07-17T08:47:07.889720+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:57:00.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884415+0000","last_change":"2024-07-17T08:47:05.129247+0000","last_active":"2024-07-17T08:47:07.884415+0000","last_peered":"2024-07-17T08:47:07.884415+0000","last_clean":"2024-07-17T08:47:07.884415+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:47:07.884415+0000","last_undegraded":"2024-07-17T08:47:07.884415+0000","last_fullsized":"2024-07-17T08:47:07.884415+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869512+0000","last_change":"2024-07-17T08:47:06.134713+0000","last_active":"2024-07-17T08:47:07.869512+0000","last_peered":"2024-07-17T08:47:07.869512+0000","last_clean":"2024-07-17T08:47:07.869512+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:47:07.869512+0000","last_undegraded":"2024-07-17T08:47:07.869512+0000","last_fullsized":"2024-07-17T08:47:07.869512+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:46:14.357947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143776+0000","last_change":"2024-07-17T08:47:05.136499+0000","last_active":"2024-07-17T08:47:07.143776+0000","last_peered":"2024-07-17T08:47:07.143776+0000","last_clean":"2024-07-17T08:47:07.143776+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:47:07.143776+0000","last_undegraded":"2024-07-17T08:47:07.143776+0000","last_fullsized":"2024-07-17T08:47:07.143776+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:07:15.469344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274492+0000","last_change":"2024-07-17T08:47:06.146038+0000","last_active":"2024-07-17T08:47:07.274492+0000","last_peered":"2024-07-17T08:47:07.274492+0000","last_clean":"2024-07-17T08:47:07.274492+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:47:07.274492+0000","last_undegraded":"2024-07-17T08:47:07.274492+0000","last_fullsized":"2024-07-17T08:47:07.274492+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:37:00.211515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141476+0000","last_change":"2024-07-17T08:47:05.131754+0000","last_active":"2024-07-17T08:47:07.141476+0000","last_peered":"2024-07-17T08:47:07.141476+0000","last_clean":"2024-07-17T08:47:07.141476+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:47:07.141476+0000","last_undegraded":"2024-07-17T08:47:07.141476+0000","last_fullsized":"2024-07-17T08:47:07.141476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:58.528856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.140602+0000","last_change":"2024-07-17T08:47:06.137348+0000","last_active":"2024-07-17T08:47:07.140602+0000","last_peered":"2024-07-17T08:47:07.140602+0000","last_clean":"2024-07-17T08:47:07.140602+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:47:07.140602+0000","last_undegraded":"2024-07-17T08:47:07.140602+0000","last_fullsized":"2024-07-17T08:47:07.140602+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:36:31.729140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143835+0000","last_change":"2024-07-17T08:47:05.134270+0000","last_active":"2024-07-17T08:47:07.143835+0000","last_peered":"2024-07-17T08:47:07.143835+0000","last_clean":"2024-07-17T08:47:07.143835+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:47:07.143835+0000","last_undegraded":"2024-07-17T08:47:07.143835+0000","last_fullsized":"2024-07-17T08:47:07.143835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:42:55.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.884449+0000","last_change":"2024-07-17T08:47:06.144296+0000","last_active":"2024-07-17T08:47:07.884449+0000","last_peered":"2024-07-17T08:47:07.884449+0000","last_clean":"2024-07-17T08:47:07.884449+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:47:07.884449+0000","last_undegraded":"2024-07-17T08:47:07.884449+0000","last_fullsized":"2024-07-17T08:47:07.884449+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"378'3","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273674+0000","last_change":"2024-07-17T08:47:05.133283+0000","last_active":"2024-07-17T08:47:07.273674+0000","last_peered":"2024-07-17T08:47:07.273674+0000","last_clean":"2024-07-17T08:47:07.273674+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:47:07.273674+0000","last_undegraded":"2024-07-17T08:47:07.273674+0000","last_fullsized":"2024-07-17T08:47:07.273674+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:37:49.665371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142468+0000","last_change":"2024-07-17T08:47:06.114973+0000","last_active":"2024-07-17T08:47:07.142468+0000","last_peered":"2024-07-17T08:47:07.142468+0000","last_clean":"2024-07-17T08:47:07.142468+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:47:07.142468+0000","last_undegraded":"2024-07-17T08:47:07.142468+0000","last_fullsized":"2024-07-17T08:47:07.142468+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:09:34.320415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"379'10","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:15.803589+0000","last_change":"2024-07-17T08:47:05.130953+0000","last_active":"2024-07-17T08:49:15.803589+0000","last_peered":"2024-07-17T08:49:15.803589+0000","last_clean":"2024-07-17T08:49:15.803589+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:49:15.803589+0000","last_undegraded":"2024-07-17T08:49:15.803589+0000","last_fullsized":"2024-07-17T08:49:15.803589+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:43:16.504877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141039+0000","last_change":"2024-07-17T08:47:06.152556+0000","last_active":"2024-07-17T08:47:07.141039+0000","last_peered":"2024-07-17T08:47:07.141039+0000","last_clean":"2024-07-17T08:47:07.141039+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:47:07.141039+0000","last_undegraded":"2024-07-17T08:47:07.141039+0000","last_fullsized":"2024-07-17T08:47:07.141039+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:29:55.971545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890129+0000","last_change":"2024-07-17T08:47:05.135419+0000","last_active":"2024-07-17T08:47:07.890129+0000","last_peered":"2024-07-17T08:47:07.890129+0000","last_clean":"2024-07-17T08:47:07.890129+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:47:07.890129+0000","last_undegraded":"2024-07-17T08:47:07.890129+0000","last_fullsized":"2024-07-17T08:47:07.890129+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273619+0000","last_change":"2024-07-17T08:47:06.139400+0000","last_active":"2024-07-17T08:47:07.273619+0000","last_peered":"2024-07-17T08:47:07.273619+0000","last_clean":"2024-07-17T08:47:07.273619+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:47:07.273619+0000","last_undegraded":"2024-07-17T08:47:07.273619+0000","last_fullsized":"2024-07-17T08:47:07.273619+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:42:03.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.142074+0000","last_change":"2024-07-17T08:47:05.119500+0000","last_active":"2024-07-17T08:47:07.142074+0000","last_peered":"2024-07-17T08:47:07.142074+0000","last_clean":"2024-07-17T08:47:07.142074+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:47:07.142074+0000","last_undegraded":"2024-07-17T08:47:07.142074+0000","last_fullsized":"2024-07-17T08:47:07.142074+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:03:39.432998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.141742+0000","last_change":"2024-07-17T08:47:06.149735+0000","last_active":"2024-07-17T08:47:07.141742+0000","last_peered":"2024-07-17T08:47:07.141742+0000","last_clean":"2024-07-17T08:47:07.141742+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:47:07.141742+0000","last_undegraded":"2024-07-17T08:47:07.141742+0000","last_fullsized":"2024-07-17T08:47:07.141742+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:03:29.232655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.274408+0000","last_change":"2024-07-17T08:47:05.124238+0000","last_active":"2024-07-17T08:47:07.274408+0000","last_peered":"2024-07-17T08:47:07.274408+0000","last_clean":"2024-07-17T08:47:07.274408+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:47:07.274408+0000","last_undegraded":"2024-07-17T08:47:07.274408+0000","last_fullsized":"2024-07-17T08:47:07.274408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:14:55.782949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.275417+0000","last_change":"2024-07-17T08:47:06.147501+0000","last_active":"2024-07-17T08:47:07.275417+0000","last_peered":"2024-07-17T08:47:07.275417+0000","last_clean":"2024-07-17T08:47:07.275417+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:47:07.275417+0000","last_undegraded":"2024-07-17T08:47:07.275417+0000","last_fullsized":"2024-07-17T08:47:07.275417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:59:51.271960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"378'8","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.143945+0000","last_change":"2024-07-17T08:47:05.134196+0000","last_active":"2024-07-17T08:47:07.143945+0000","last_peered":"2024-07-17T08:47:07.143945+0000","last_clean":"2024-07-17T08:47:07.143945+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:47:07.143945+0000","last_undegraded":"2024-07-17T08:47:07.143945+0000","last_fullsized":"2024-07-17T08:47:07.143945+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:37:47.081835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:17.912336+0000","last_change":"2024-07-17T08:47:06.127658+0000","last_active":"2024-07-17T08:49:17.912336+0000","last_peered":"2024-07-17T08:49:17.912336+0000","last_clean":"2024-07-17T08:49:17.912336+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:49:17.912336+0000","last_undegraded":"2024-07-17T08:49:17.912336+0000","last_fullsized":"2024-07-17T08:49:17.912336+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"378'1","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:22.579921+0000","last_change":"2024-07-17T08:47:05.126263+0000","last_active":"2024-07-17T08:49:22.579921+0000","last_peered":"2024-07-17T08:49:22.579921+0000","last_clean":"2024-07-17T08:49:22.579921+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:49:22.579921+0000","last_undegraded":"2024-07-17T08:49:22.579921+0000","last_fullsized":"2024-07-17T08:49:22.579921+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885960+0000","last_change":"2024-07-17T08:47:06.146988+0000","last_active":"2024-07-17T08:47:07.885960+0000","last_peered":"2024-07-17T08:47:07.885960+0000","last_clean":"2024-07-17T08:47:07.885960+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:47:07.885960+0000","last_undegraded":"2024-07-17T08:47:07.885960+0000","last_fullsized":"2024-07-17T08:47:07.885960+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:06.911587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888349+0000","last_change":"2024-07-17T08:47:05.132995+0000","last_active":"2024-07-17T08:47:07.888349+0000","last_peered":"2024-07-17T08:47:07.888349+0000","last_clean":"2024-07-17T08:47:07.888349+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:47:07.888349+0000","last_undegraded":"2024-07-17T08:47:07.888349+0000","last_fullsized":"2024-07-17T08:47:07.888349+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:36:59.220995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890124+0000","last_change":"2024-07-17T08:47:06.128417+0000","last_active":"2024-07-17T08:47:07.890124+0000","last_peered":"2024-07-17T08:47:07.890124+0000","last_clean":"2024-07-17T08:47:07.890124+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:47:07.890124+0000","last_undegraded":"2024-07-17T08:47:07.890124+0000","last_fullsized":"2024-07-17T08:47:07.890124+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:10:45.633232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"378'1","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877642+0000","last_change":"2024-07-17T08:47:05.128285+0000","last_active":"2024-07-17T08:47:07.877642+0000","last_peered":"2024-07-17T08:47:07.877642+0000","last_clean":"2024-07-17T08:47:07.877642+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:47:07.877642+0000","last_undegraded":"2024-07-17T08:47:07.877642+0000","last_fullsized":"2024-07-17T08:47:07.877642+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:27:05.988092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890158+0000","last_change":"2024-07-17T08:47:06.145946+0000","last_active":"2024-07-17T08:47:07.890158+0000","last_peered":"2024-07-17T08:47:07.890158+0000","last_clean":"2024-07-17T08:47:07.890158+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:47:07.890158+0000","last_undegraded":"2024-07-17T08:47:07.890158+0000","last_fullsized":"2024-07-17T08:47:07.890158+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:34:12.169632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886094+0000","last_change":"2024-07-17T08:47:05.139284+0000","last_active":"2024-07-17T08:47:07.886094+0000","last_peered":"2024-07-17T08:47:07.886094+0000","last_clean":"2024-07-17T08:47:07.886094+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:47:07.886094+0000","last_undegraded":"2024-07-17T08:47:07.886094+0000","last_fullsized":"2024-07-17T08:47:07.886094+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:49.770736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877569+0000","last_change":"2024-07-17T08:47:06.152516+0000","last_active":"2024-07-17T08:47:07.877569+0000","last_peered":"2024-07-17T08:47:07.877569+0000","last_clean":"2024-07-17T08:47:07.877569+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:47:07.877569+0000","last_undegraded":"2024-07-17T08:47:07.877569+0000","last_fullsized":"2024-07-17T08:47:07.877569+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:12:03.878088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"378'2","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:21.852952+0000","last_change":"2024-07-17T08:47:05.133862+0000","last_active":"2024-07-17T08:49:21.852952+0000","last_peered":"2024-07-17T08:49:21.852952+0000","last_clean":"2024-07-17T08:49:21.852952+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:49:21.852952+0000","last_undegraded":"2024-07-17T08:49:21.852952+0000","last_fullsized":"2024-07-17T08:49:21.852952+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.869457+0000","last_change":"2024-07-17T08:47:06.145680+0000","last_active":"2024-07-17T08:47:07.869457+0000","last_peered":"2024-07-17T08:47:07.869457+0000","last_clean":"2024-07-17T08:47:07.869457+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:47:07.869457+0000","last_undegraded":"2024-07-17T08:47:07.869457+0000","last_fullsized":"2024-07-17T08:47:07.869457+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:02:08.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886001+0000","last_change":"2024-07-17T08:47:05.138515+0000","last_active":"2024-07-17T08:47:07.886001+0000","last_peered":"2024-07-17T08:47:07.886001+0000","last_clean":"2024-07-17T08:47:07.886001+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:47:07.886001+0000","last_undegraded":"2024-07-17T08:47:07.886001+0000","last_fullsized":"2024-07-17T08:47:07.886001+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:21.507368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890252+0000","last_change":"2024-07-17T08:47:06.137310+0000","last_active":"2024-07-17T08:47:07.890252+0000","last_peered":"2024-07-17T08:47:07.890252+0000","last_clean":"2024-07-17T08:47:07.890252+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:47:07.890252+0000","last_undegraded":"2024-07-17T08:47:07.890252+0000","last_fullsized":"2024-07-17T08:47:07.890252+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:10:09.730171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889551+0000","last_change":"2024-07-17T08:47:05.121651+0000","last_active":"2024-07-17T08:47:07.889551+0000","last_peered":"2024-07-17T08:47:07.889551+0000","last_clean":"2024-07-17T08:47:07.889551+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:47:07.889551+0000","last_undegraded":"2024-07-17T08:47:07.889551+0000","last_fullsized":"2024-07-17T08:47:07.889551+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:32:10.973962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889782+0000","last_change":"2024-07-17T08:47:06.144549+0000","last_active":"2024-07-17T08:47:07.889782+0000","last_peered":"2024-07-17T08:47:07.889782+0000","last_clean":"2024-07-17T08:47:07.889782+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:47:07.889782+0000","last_undegraded":"2024-07-17T08:47:07.889782+0000","last_fullsized":"2024-07-17T08:47:07.889782+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:20:02.933721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.885909+0000","last_change":"2024-07-17T08:47:05.134720+0000","last_active":"2024-07-17T08:47:07.885909+0000","last_peered":"2024-07-17T08:47:07.885909+0000","last_clean":"2024-07-17T08:47:07.885909+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:47:07.885909+0000","last_undegraded":"2024-07-17T08:47:07.885909+0000","last_fullsized":"2024-07-17T08:47:07.885909+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:39:07.886840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.888132+0000","last_change":"2024-07-17T08:47:06.122239+0000","last_active":"2024-07-17T08:47:07.888132+0000","last_peered":"2024-07-17T08:47:07.888132+0000","last_clean":"2024-07-17T08:47:07.888132+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:47:07.888132+0000","last_undegraded":"2024-07-17T08:47:07.888132+0000","last_fullsized":"2024-07-17T08:47:07.888132+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:13:39.841279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"378'2","reported_seq":17,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890304+0000","last_change":"2024-07-17T08:47:05.134021+0000","last_active":"2024-07-17T08:47:07.890304+0000","last_peered":"2024-07-17T08:47:07.890304+0000","last_clean":"2024-07-17T08:47:07.890304+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:47:07.890304+0000","last_undegraded":"2024-07-17T08:47:07.890304+0000","last_fullsized":"2024-07-17T08:47:07.890304+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:15:34.905626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877385+0000","last_change":"2024-07-17T08:47:06.137590+0000","last_active":"2024-07-17T08:47:07.877385+0000","last_peered":"2024-07-17T08:47:07.877385+0000","last_clean":"2024-07-17T08:47:07.877385+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:47:07.877385+0000","last_undegraded":"2024-07-17T08:47:07.877385+0000","last_fullsized":"2024-07-17T08:47:07.877385+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:08:38.057596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"379'3","reported_seq":18,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:16.142642+0000","last_change":"2024-07-17T08:47:05.128642+0000","last_active":"2024-07-17T08:49:16.142642+0000","last_peered":"2024-07-17T08:49:16.142642+0000","last_clean":"2024-07-17T08:49:16.142642+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:49:16.142642+0000","last_undegraded":"2024-07-17T08:49:16.142642+0000","last_fullsized":"2024-07-17T08:49:16.142642+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:53:06.786384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":14,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:12.905339+0000","last_change":"2024-07-17T08:47:06.136720+0000","last_active":"2024-07-17T08:49:12.905339+0000","last_peered":"2024-07-17T08:49:12.905339+0000","last_clean":"2024-07-17T08:49:12.905339+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:49:12.905339+0000","last_undegraded":"2024-07-17T08:49:12.905339+0000","last_fullsized":"2024-07-17T08:49:12.905339+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877661+0000","last_change":"2024-07-17T08:47:05.132981+0000","last_active":"2024-07-17T08:47:07.877661+0000","last_peered":"2024-07-17T08:47:07.877661+0000","last_clean":"2024-07-17T08:47:07.877661+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:47:07.877661+0000","last_undegraded":"2024-07-17T08:47:07.877661+0000","last_fullsized":"2024-07-17T08:47:07.877661+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:25:57.504370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890188+0000","last_change":"2024-07-17T08:47:06.133480+0000","last_active":"2024-07-17T08:47:07.890188+0000","last_peered":"2024-07-17T08:47:07.890188+0000","last_clean":"2024-07-17T08:47:07.890188+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:47:07.890188+0000","last_undegraded":"2024-07-17T08:47:07.890188+0000","last_fullsized":"2024-07-17T08:47:07.890188+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:22:58.551363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":16,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:12.911463+0000","last_change":"2024-07-17T08:47:05.133629+0000","last_active":"2024-07-17T08:49:12.911463+0000","last_peered":"2024-07-17T08:49:12.911463+0000","last_clean":"2024-07-17T08:49:12.911463+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:49:12.911463+0000","last_undegraded":"2024-07-17T08:49:12.911463+0000","last_fullsized":"2024-07-17T08:49:12.911463+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.889649+0000","last_change":"2024-07-17T08:47:06.115688+0000","last_active":"2024-07-17T08:47:07.889649+0000","last_peered":"2024-07-17T08:47:07.889649+0000","last_clean":"2024-07-17T08:47:07.889649+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:47:07.889649+0000","last_undegraded":"2024-07-17T08:47:07.889649+0000","last_fullsized":"2024-07-17T08:47:07.889649+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:35:21.869842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877509+0000","last_change":"2024-07-17T08:47:05.135440+0000","last_active":"2024-07-17T08:47:07.877509+0000","last_peered":"2024-07-17T08:47:07.877509+0000","last_clean":"2024-07-17T08:47:07.877509+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:47:07.877509+0000","last_undegraded":"2024-07-17T08:47:07.877509+0000","last_fullsized":"2024-07-17T08:47:07.877509+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:59:51.291675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877625+0000","last_change":"2024-07-17T08:47:06.141413+0000","last_active":"2024-07-17T08:47:07.877625+0000","last_peered":"2024-07-17T08:47:07.877625+0000","last_clean":"2024-07-17T08:47:07.877625+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:47:07.877625+0000","last_undegraded":"2024-07-17T08:47:07.877625+0000","last_fullsized":"2024-07-17T08:47:07.877625+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:40:51.181038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877473+0000","last_change":"2024-07-17T08:47:05.135095+0000","last_active":"2024-07-17T08:47:07.877473+0000","last_peered":"2024-07-17T08:47:07.877473+0000","last_clean":"2024-07-17T08:47:07.877473+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:47:07.877473+0000","last_undegraded":"2024-07-17T08:47:07.877473+0000","last_fullsized":"2024-07-17T08:47:07.877473+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:11:22.585047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886198+0000","last_change":"2024-07-17T08:47:06.145004+0000","last_active":"2024-07-17T08:47:07.886198+0000","last_peered":"2024-07-17T08:47:07.886198+0000","last_clean":"2024-07-17T08:47:07.886198+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:47:07.886198+0000","last_undegraded":"2024-07-17T08:47:07.886198+0000","last_fullsized":"2024-07-17T08:47:07.886198+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:25:15.953728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.877423+0000","last_change":"2024-07-17T08:47:05.131419+0000","last_active":"2024-07-17T08:47:07.877423+0000","last_peered":"2024-07-17T08:47:07.877423+0000","last_clean":"2024-07-17T08:47:07.877423+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:47:07.877423+0000","last_undegraded":"2024-07-17T08:47:07.877423+0000","last_fullsized":"2024-07-17T08:47:07.877423+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:48:55.721853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":13,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.886235+0000","last_change":"2024-07-17T08:47:06.145000+0000","last_active":"2024-07-17T08:47:07.886235+0000","last_peered":"2024-07-17T08:47:07.886235+0000","last_clean":"2024-07-17T08:47:07.886235+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:47:07.886235+0000","last_undegraded":"2024-07-17T08:47:07.886235+0000","last_fullsized":"2024-07-17T08:47:07.886235+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:27:17.195480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":15,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.890029+0000","last_change":"2024-07-17T08:47:05.127261+0000","last_active":"2024-07-17T08:47:07.890029+0000","last_peered":"2024-07-17T08:47:07.890029+0000","last_clean":"2024-07-17T08:47:07.890029+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:47:07.890029+0000","last_undegraded":"2024-07-17T08:47:07.890029+0000","last_fullsized":"2024-07-17T08:47:07.890029+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:04.061231+0000","last_clean_scrub_stamp":"2024-07-17T08:47:04.061231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:22:23.320660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":12,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.273276+0000","last_change":"2024-07-17T08:47:06.144622+0000","last_active":"2024-07-17T08:47:07.273276+0000","last_peered":"2024-07-17T08:47:07.273276+0000","last_clean":"2024-07-17T08:47:07.273276+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:47:07.273276+0000","last_undegraded":"2024-07-17T08:47:07.273276+0000","last_fullsized":"2024-07-17T08:47:07.273276+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:47:05.079735+0000","last_clean_scrub_stamp":"2024-07-17T08:47:05.079735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:36:37.687947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"370'4680","reported_seq":6212,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:47:07.272409+0000","last_change":"2024-07-17T06:54:43.872052+0000","last_active":"2024-07-17T08:47:07.272409+0000","last_peered":"2024-07-17T08:47:07.272409+0000","last_clean":"2024-07-17T08:47:07.272409+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:47:07.272409+0000","last_undegraded":"2024-07-17T08:47:07.272409+0000","last_fullsized":"2024-07-17T08:47:07.272409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T06:54:42.202505+0000","last_clean_scrub_stamp":"2024-07-17T06:54:42.202505+0000","objects_scrubbed":0,"log_size":4680,"log_dups_size":0,"ondisk_log_size":4680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:59:57.861269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":3797,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80904,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290936,"statfs":{"total":96636764160,"available":96553918464,"internally_reserved":0,"allocated":5427200,"data_stored":3261377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53948,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317892,"statfs":{"total":96636764160,"available":96581521408,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510795,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76152,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295688,"statfs":{"total":96636764160,"available":96558784512,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510794,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80324,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291516,"statfs":{"total":96636764160,"available":96554512384,"internally_reserved":0,"allocated":4845568,"data_stored":2673484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543498,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80320,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291520,"statfs":{"total":96636764160,"available":96554516480,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510795,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4841472,"data_stored":2673414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510794,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:49:27.442 INFO:tasks.ceph:pgid 52.3f last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.442 INFO:tasks.ceph:pgid 52.3e last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.442 INFO:tasks.ceph:pgid 52.3d last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.442 INFO:tasks.ceph:pgid 52.3c last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.443 INFO:tasks.ceph:pgid 52.3b last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.443 INFO:tasks.ceph:pgid 52.3a last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.443 INFO:tasks.ceph:pgid 52.39 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.443 INFO:tasks.ceph:pgid 52.38 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.444 INFO:tasks.ceph:pgid 52.37 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.444 INFO:tasks.ceph:pgid 52.36 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.444 INFO:tasks.ceph:pgid 52.34 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.445 INFO:tasks.ceph:pgid 52.33 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.445 INFO:tasks.ceph:pgid 52.32 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.445 INFO:tasks.ceph:pgid 52.31 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.445 INFO:tasks.ceph:pgid 52.30 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.445 INFO:tasks.ceph:pgid 52.2f last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.2e last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.2d last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.2c last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.2b last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.2a last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.446 INFO:tasks.ceph:pgid 52.29 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.447 INFO:tasks.ceph:pgid 52.28 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.447 INFO:tasks.ceph:pgid 52.27 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.447 INFO:tasks.ceph:pgid 52.26 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.447 INFO:tasks.ceph:pgid 52.25 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.447 INFO:tasks.ceph:pgid 52.24 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 52.23 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 52.22 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 52.21 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 52.20 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 52.18 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.448 INFO:tasks.ceph:pgid 51.1f last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 52.19 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 52.1a last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 51.1d last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 52.1b last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 51.1c last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.449 INFO:tasks.ceph:pgid 52.1c last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 51.1b last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 52.1d last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 51.1a last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 52.1e last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 51.19 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.450 INFO:tasks.ceph:pgid 52.1f last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.451 INFO:tasks.ceph:pgid 51.18 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.451 INFO:tasks.ceph:pgid 52.10 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.451 INFO:tasks.ceph:pgid 51.17 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.451 INFO:tasks.ceph:pgid 52.11 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 51.16 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 52.12 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 51.15 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 52.13 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 51.14 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.452 INFO:tasks.ceph:pgid 52.14 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 51.13 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 52.15 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 51.12 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 52.16 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 51.11 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 52.17 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.453 INFO:tasks.ceph:pgid 51.10 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.454 INFO:tasks.ceph:pgid 52.8 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.454 INFO:tasks.ceph:pgid 51.f last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.454 INFO:tasks.ceph:pgid 52.9 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.454 INFO:tasks.ceph:pgid 51.e last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.454 INFO:tasks.ceph:pgid 52.a last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 51.d last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 52.b last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 51.c last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 52.c last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 51.b last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.455 INFO:tasks.ceph:pgid 52.d last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 51.a last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 52.e last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 51.9 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 52.f last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 51.8 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.456 INFO:tasks.ceph:pgid 52.0 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.457 INFO:tasks.ceph:pgid 51.7 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.457 INFO:tasks.ceph:pgid 52.1 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.457 INFO:tasks.ceph:pgid 51.6 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.457 INFO:tasks.ceph:pgid 52.2 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.457 INFO:tasks.ceph:pgid 51.5 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.458 INFO:tasks.ceph:pgid 52.3 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.458 INFO:tasks.ceph:pgid 51.4 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.458 INFO:tasks.ceph:pgid 52.4 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.458 INFO:tasks.ceph:pgid 51.3 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 52.5 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 51.2 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 52.6 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 51.1 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 52.7 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.459 INFO:tasks.ceph:pgid 51.0 last_scrub_stamp 2024-07-17T08:47:04.061231+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=4, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.460 INFO:tasks.ceph:pgid 52.35 last_scrub_stamp 2024-07-17T08:47:05.079735+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=47, tm_sec=5, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.460 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-07-17T06:54:42.202505+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=6, tm_min=54, tm_sec=42, tm_wday=2, tm_yday=199, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=17, tm_hour=8, tm_min=49, tm_sec=20, tm_wday=2, tm_yday=199, tm_isdst=0) 2024-07-17T08:49:27.460 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-07-17T08:49:44.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:49:44.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:49:47.461 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-17T08:49:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.685+0000 7fb8cfe00640 1 -- 172.21.15.44:0/3847693944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 msgr2=0x7fb8b4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.685+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/3847693944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8b4003ec0 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0001940 tx=0x7fb8b4005830 comp rx=0 tx=0).stop 2024-07-17T08:49:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.685+0000 7fb8cf400640 1 -- 172.21.15.44:0/3847693944 reap_dead start 2024-07-17T08:49:47.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.685+0000 7fb8cfe00640 1 -- 172.21.15.44:0/3847693944 shutdown_connections 2024-07-17T08:49:47.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.685+0000 7fb8cfe00640 1 -- 172.21.15.44:0/3847693944 >> 172.21.15.44:0/3847693944 conn(0x7fb8d00b0e80 msgr2=0x7fb8d00b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:47.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.686+0000 7fb8cfe00640 1 -- 172.21.15.44:0/3847693944 shutdown_connections 2024-07-17T08:49:47.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.686+0000 7fb8cfe00640 1 -- 172.21.15.44:0/3847693944 wait complete. 2024-07-17T08:49:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.687+0000 7fb8cfe00640 1 Processor -- start 2024-07-17T08:49:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.687+0000 7fb8cfe00640 1 -- start start 2024-07-17T08:49:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:47.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d01f7800 0x7fb8d01f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d009aa30 0x7fb8d009aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d009b9a0 con 0x7fb8b4003aa0 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d009bb20 con 0x7fb8d009aa30 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.688+0000 7fb8cfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d009bca0 con 0x7fb8d01f7800 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59170/0 (socket says 172.21.15.44:59170) 2024-07-17T08:49:47.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 -- 172.21.15.44:0/518782329 learned_addr learned my addr 172.21.15.44:0/518782329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 -- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d01f7800 msgr2=0x7fb8d01f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d01f7800 0x7fb8d01f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 -- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d009aa30 msgr2=0x7fb8d009aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d009aa30 0x7fb8d009aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 -- 172.21.15.44:0/518782329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d01f4b40 con 0x7fb8b4003aa0 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.689+0000 7fb8cea00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 secure :-1 s=READY pgs=3546 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00068b0 tx=0x7fb8c00068e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.690+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0042020 con 0x7fb8b4003aa0 2024-07-17T08:49:47.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.690+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c003e460 con 0x7fb8b4003aa0 2024-07-17T08:49:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.690+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c003e830 con 0x7fb8b4003aa0 2024-07-17T08:49:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.690+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d01f3d80 con 0x7fb8b4003aa0 2024-07-17T08:49:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.690+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01f4340 con 0x7fb8b4003aa0 2024-07-17T08:49:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.691+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d015e5c0 con 0x7fb8b4003aa0 2024-07-17T08:49:47.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.692+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fb8c0045020 con 0x7fb8b4003aa0 2024-07-17T08:49:47.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.694+0000 7fb8c6a00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 0x7fb8ac079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:47.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.694+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fb8c00b9ec0 con 0x7fb8b4003aa0 2024-07-17T08:49:47.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.694+0000 7fb8ce000640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 0x7fb8ac079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:47.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.695+0000 7fb8ce000640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 0x7fb8ac079190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b3f60 tx=0x7fb8b8005af0 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:47.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.698+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fb8c0030070 con 0x7fb8b4003aa0 2024-07-17T08:49:47.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.898+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 --> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8d006f480 con 0x7fb8ac076cd0 2024-07-17T08:49:47.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.910+0000 7fb8c6a00640 1 -- 172.21.15.44:0/518782329 <== mgr.10343 v2:172.21.15.175:6834/294464184 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7fb8d006f480 con 0x7fb8ac076cd0 2024-07-17T08:49:47.915 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:47.917 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-07-17T08:49:47.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 msgr2=0x7fb8ac079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 0x7fb8ac079190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b3f60 tx=0x7fb8b8005af0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 msgr2=0x7fb8d0154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 secure :-1 s=READY pgs=3546 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00068b0 tx=0x7fb8c00068e0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 shutdown_connections 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fb8ac076cd0 0x7fb8ac079190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d009aa30 0x7fb8d009aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d01f7800 0x7fb8d01f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 --2- 172.21.15.44:0/518782329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8b4003aa0 0x7fb8d0154600 unknown :-1 s=CLOSED pgs=3546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:47.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.917+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 >> 172.21.15.44:0/518782329 conn(0x7fb8d00b0e80 msgr2=0x7fb8d01569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:47.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.918+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 shutdown_connections 2024-07-17T08:49:47.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:47.918+0000 7fb8cfe00640 1 -- 172.21.15.44:0/518782329 wait complete. 2024-07-17T08:49:47.937 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-07-17T08:49:45.943536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594165,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":642,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4746,"ondisk_log_size":4746,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":546504,"kb_used_data":38920,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754428216,"statfs":{"total":773094113280,"available":772534493184,"internally_reserved":0,"allocated":39854080,"data_stored":22551474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":519687728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004581"},"pg_stats":[{"pgid":"52.3f","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.379938+0000","last_change":"2024-07-17T08:49:28.379996+0000","last_active":"2024-07-17T08:49:28.379938+0000","last_peered":"2024-07-17T08:49:28.379938+0000","last_clean":"2024-07-17T08:49:28.379938+0000","last_became_active":"2024-07-17T08:47:06.136335+0000","last_became_peered":"2024-07-17T08:47:06.136335+0000","last_unstale":"2024-07-17T08:49:28.379938+0000","last_undegraded":"2024-07-17T08:49:28.379938+0000","last_fullsized":"2024-07-17T08:49:28.379938+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.379757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.379757+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.379757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:36:38.408334+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.3e","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.390457+0000","last_change":"2024-07-17T08:49:31.390513+0000","last_active":"2024-07-17T08:49:31.390457+0000","last_peered":"2024-07-17T08:49:31.390457+0000","last_clean":"2024-07-17T08:49:31.390457+0000","last_became_active":"2024-07-17T08:47:06.143415+0000","last_became_peered":"2024-07-17T08:47:06.143415+0000","last_unstale":"2024-07-17T08:49:31.390457+0000","last_undegraded":"2024-07-17T08:49:31.390457+0000","last_fullsized":"2024-07-17T08:49:31.390457+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.390280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.390280+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.390280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:14:30.188634+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.3d","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.391647+0000","last_change":"2024-07-17T08:49:33.391743+0000","last_active":"2024-07-17T08:49:33.391647+0000","last_peered":"2024-07-17T08:49:33.391647+0000","last_clean":"2024-07-17T08:49:33.391647+0000","last_became_active":"2024-07-17T08:47:06.138968+0000","last_became_peered":"2024-07-17T08:47:06.138968+0000","last_unstale":"2024-07-17T08:49:33.391647+0000","last_undegraded":"2024-07-17T08:49:33.391647+0000","last_fullsized":"2024-07-17T08:49:33.391647+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.391476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.391476+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.391476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:45.835123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.3c","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.307873+0000","last_change":"2024-07-17T08:49:34.308025+0000","last_active":"2024-07-17T08:49:34.307873+0000","last_peered":"2024-07-17T08:49:34.307873+0000","last_clean":"2024-07-17T08:49:34.307873+0000","last_became_active":"2024-07-17T08:47:06.136765+0000","last_became_peered":"2024-07-17T08:47:06.136765+0000","last_unstale":"2024-07-17T08:49:34.307873+0000","last_undegraded":"2024-07-17T08:49:34.307873+0000","last_fullsized":"2024-07-17T08:49:34.307873+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.307635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.307635+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.307635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:06:32.740125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.3b","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.604532+0000","last_change":"2024-07-17T08:49:33.604532+0000","last_active":"2024-07-17T08:49:33.604532+0000","last_peered":"2024-07-17T08:49:33.604532+0000","last_clean":"2024-07-17T08:49:33.604532+0000","last_became_active":"2024-07-17T08:47:06.135423+0000","last_became_peered":"2024-07-17T08:47:06.135423+0000","last_unstale":"2024-07-17T08:49:33.604532+0000","last_undegraded":"2024-07-17T08:49:33.604532+0000","last_fullsized":"2024-07-17T08:49:33.604532+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.604287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.604287+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.604287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:42:28.072906+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3a","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:25.302597+0000","last_change":"2024-07-17T08:49:25.302597+0000","last_active":"2024-07-17T08:49:25.302597+0000","last_peered":"2024-07-17T08:49:25.302597+0000","last_clean":"2024-07-17T08:49:25.302597+0000","last_became_active":"2024-07-17T08:47:06.131638+0000","last_became_peered":"2024-07-17T08:47:06.131638+0000","last_unstale":"2024-07-17T08:49:25.302597+0000","last_undegraded":"2024-07-17T08:49:25.302597+0000","last_fullsized":"2024-07-17T08:49:25.302597+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:25.302344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:25.302344+0000","last_clean_scrub_stamp":"2024-07-17T08:49:25.302344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:53:29.860854+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.39","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.286286+0000","last_change":"2024-07-17T08:49:31.286286+0000","last_active":"2024-07-17T08:49:31.286286+0000","last_peered":"2024-07-17T08:49:31.286286+0000","last_clean":"2024-07-17T08:49:31.286286+0000","last_became_active":"2024-07-17T08:47:06.123261+0000","last_became_peered":"2024-07-17T08:47:06.123261+0000","last_unstale":"2024-07-17T08:49:31.286286+0000","last_undegraded":"2024-07-17T08:49:31.286286+0000","last_fullsized":"2024-07-17T08:49:31.286286+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.286060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.286060+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.286060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:17:22.437036+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.38","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.279450+0000","last_change":"2024-07-17T08:49:29.279450+0000","last_active":"2024-07-17T08:49:29.279450+0000","last_peered":"2024-07-17T08:49:29.279450+0000","last_clean":"2024-07-17T08:49:29.279450+0000","last_became_active":"2024-07-17T08:47:06.117472+0000","last_became_peered":"2024-07-17T08:47:06.117472+0000","last_unstale":"2024-07-17T08:49:29.279450+0000","last_undegraded":"2024-07-17T08:49:29.279450+0000","last_fullsized":"2024-07-17T08:49:29.279450+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.279232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.279232+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.279232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:21:45.113184+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.37","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.425463+0000","last_change":"2024-07-17T08:49:34.425519+0000","last_active":"2024-07-17T08:49:34.425463+0000","last_peered":"2024-07-17T08:49:34.425463+0000","last_clean":"2024-07-17T08:49:34.425463+0000","last_became_active":"2024-07-17T08:47:06.146995+0000","last_became_peered":"2024-07-17T08:47:06.146995+0000","last_unstale":"2024-07-17T08:49:34.425463+0000","last_undegraded":"2024-07-17T08:49:34.425463+0000","last_fullsized":"2024-07-17T08:49:34.425463+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.425285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.425285+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.425285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:18:45.808650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.36","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.425306+0000","last_change":"2024-07-17T08:49:29.425306+0000","last_active":"2024-07-17T08:49:29.425306+0000","last_peered":"2024-07-17T08:49:29.425306+0000","last_clean":"2024-07-17T08:49:29.425306+0000","last_became_active":"2024-07-17T08:47:06.140623+0000","last_became_peered":"2024-07-17T08:47:06.140623+0000","last_unstale":"2024-07-17T08:49:29.425306+0000","last_undegraded":"2024-07-17T08:49:29.425306+0000","last_fullsized":"2024-07-17T08:49:29.425306+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.425072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.425072+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.425072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:16.655517+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.34","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:35.301017+0000","last_change":"2024-07-17T08:49:35.301070+0000","last_active":"2024-07-17T08:49:35.301017+0000","last_peered":"2024-07-17T08:49:35.301017+0000","last_clean":"2024-07-17T08:49:35.301017+0000","last_became_active":"2024-07-17T08:47:06.147301+0000","last_became_peered":"2024-07-17T08:47:06.147301+0000","last_unstale":"2024-07-17T08:49:35.301017+0000","last_undegraded":"2024-07-17T08:49:35.301017+0000","last_fullsized":"2024-07-17T08:49:35.301017+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:35.300834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:35.300834+0000","last_clean_scrub_stamp":"2024-07-17T08:49:35.300834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:07:17.040098+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.33","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.407507+0000","last_change":"2024-07-17T08:49:28.407507+0000","last_active":"2024-07-17T08:49:28.407507+0000","last_peered":"2024-07-17T08:49:28.407507+0000","last_clean":"2024-07-17T08:49:28.407507+0000","last_became_active":"2024-07-17T08:47:06.142681+0000","last_became_peered":"2024-07-17T08:47:06.142681+0000","last_unstale":"2024-07-17T08:49:28.407507+0000","last_undegraded":"2024-07-17T08:49:28.407507+0000","last_fullsized":"2024-07-17T08:49:28.407507+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.407198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.407198+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.407198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:16:38.028025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.32","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.422168+0000","last_change":"2024-07-17T08:49:30.422225+0000","last_active":"2024-07-17T08:49:30.422168+0000","last_peered":"2024-07-17T08:49:30.422168+0000","last_clean":"2024-07-17T08:49:30.422168+0000","last_became_active":"2024-07-17T08:47:06.142655+0000","last_became_peered":"2024-07-17T08:47:06.142655+0000","last_unstale":"2024-07-17T08:49:30.422168+0000","last_undegraded":"2024-07-17T08:49:30.422168+0000","last_fullsized":"2024-07-17T08:49:30.422168+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.421989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.421989+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.421989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:32:21.313799+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.31","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.444895+0000","last_change":"2024-07-17T08:49:31.444950+0000","last_active":"2024-07-17T08:49:31.444895+0000","last_peered":"2024-07-17T08:49:31.444895+0000","last_clean":"2024-07-17T08:49:31.444895+0000","last_became_active":"2024-07-17T08:47:06.138013+0000","last_became_peered":"2024-07-17T08:47:06.138013+0000","last_unstale":"2024-07-17T08:49:31.444895+0000","last_undegraded":"2024-07-17T08:49:31.444895+0000","last_fullsized":"2024-07-17T08:49:31.444895+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.444719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.444719+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.444719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:49.968536+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.30","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.448122+0000","last_change":"2024-07-17T08:49:33.448177+0000","last_active":"2024-07-17T08:49:33.448122+0000","last_peered":"2024-07-17T08:49:33.448122+0000","last_clean":"2024-07-17T08:49:33.448122+0000","last_became_active":"2024-07-17T08:47:06.144377+0000","last_became_peered":"2024-07-17T08:47:06.144377+0000","last_unstale":"2024-07-17T08:49:33.448122+0000","last_undegraded":"2024-07-17T08:49:33.448122+0000","last_fullsized":"2024-07-17T08:49:33.448122+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.447940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.447940+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.447940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:48:27.915566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2f","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.422862+0000","last_change":"2024-07-17T08:49:32.422915+0000","last_active":"2024-07-17T08:49:32.422862+0000","last_peered":"2024-07-17T08:49:32.422862+0000","last_clean":"2024-07-17T08:49:32.422862+0000","last_became_active":"2024-07-17T08:47:06.145518+0000","last_became_peered":"2024-07-17T08:47:06.145518+0000","last_unstale":"2024-07-17T08:49:32.422862+0000","last_undegraded":"2024-07-17T08:49:32.422862+0000","last_fullsized":"2024-07-17T08:49:32.422862+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.422682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.422682+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.422682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:05:56.575602+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2e","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.406417+0000","last_change":"2024-07-17T08:49:26.406417+0000","last_active":"2024-07-17T08:49:26.406417+0000","last_peered":"2024-07-17T08:49:26.406417+0000","last_clean":"2024-07-17T08:49:26.406417+0000","last_became_active":"2024-07-17T08:47:06.133598+0000","last_became_peered":"2024-07-17T08:47:06.133598+0000","last_unstale":"2024-07-17T08:49:26.406417+0000","last_undegraded":"2024-07-17T08:49:26.406417+0000","last_fullsized":"2024-07-17T08:49:26.406417+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:26.406272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:26.406272+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.406272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:37.475125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.2d","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:37.690879+0000","last_change":"2024-07-17T08:49:37.690933+0000","last_active":"2024-07-17T08:49:37.690879+0000","last_peered":"2024-07-17T08:49:37.690879+0000","last_clean":"2024-07-17T08:49:37.690879+0000","last_became_active":"2024-07-17T08:47:06.137488+0000","last_became_peered":"2024-07-17T08:47:06.137488+0000","last_unstale":"2024-07-17T08:49:37.690879+0000","last_undegraded":"2024-07-17T08:49:37.690879+0000","last_fullsized":"2024-07-17T08:49:37.690879+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:37.690691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:37.690691+0000","last_clean_scrub_stamp":"2024-07-17T08:49:37.690691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:01:12.170037+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.2c","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.275268+0000","last_change":"2024-07-17T08:49:33.275324+0000","last_active":"2024-07-17T08:49:33.275268+0000","last_peered":"2024-07-17T08:49:33.275268+0000","last_clean":"2024-07-17T08:49:33.275268+0000","last_became_active":"2024-07-17T08:47:06.130805+0000","last_became_peered":"2024-07-17T08:47:06.130805+0000","last_unstale":"2024-07-17T08:49:33.275268+0000","last_undegraded":"2024-07-17T08:49:33.275268+0000","last_fullsized":"2024-07-17T08:49:33.275268+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.275098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.275098+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.275098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:23:48.643552+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.2b","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.435167+0000","last_change":"2024-07-17T08:49:33.435279+0000","last_active":"2024-07-17T08:49:33.435167+0000","last_peered":"2024-07-17T08:49:33.435167+0000","last_clean":"2024-07-17T08:49:33.435167+0000","last_became_active":"2024-07-17T08:47:06.146056+0000","last_became_peered":"2024-07-17T08:47:06.146056+0000","last_unstale":"2024-07-17T08:49:33.435167+0000","last_undegraded":"2024-07-17T08:49:33.435167+0000","last_fullsized":"2024-07-17T08:49:33.435167+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.434957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.434957+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.434957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:50:35.747221+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.2a","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:38.686959+0000","last_change":"2024-07-17T08:49:38.687013+0000","last_active":"2024-07-17T08:49:38.686959+0000","last_peered":"2024-07-17T08:49:38.686959+0000","last_clean":"2024-07-17T08:49:38.686959+0000","last_became_active":"2024-07-17T08:47:06.143688+0000","last_became_peered":"2024-07-17T08:47:06.143688+0000","last_unstale":"2024-07-17T08:49:38.686959+0000","last_undegraded":"2024-07-17T08:49:38.686959+0000","last_fullsized":"2024-07-17T08:49:38.686959+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:38.686777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:38.686777+0000","last_clean_scrub_stamp":"2024-07-17T08:49:38.686777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:02:03.149430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.29","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:35.651325+0000","last_change":"2024-07-17T08:49:35.651380+0000","last_active":"2024-07-17T08:49:35.651325+0000","last_peered":"2024-07-17T08:49:35.651325+0000","last_clean":"2024-07-17T08:49:35.651325+0000","last_became_active":"2024-07-17T08:47:06.135754+0000","last_became_peered":"2024-07-17T08:47:06.135754+0000","last_unstale":"2024-07-17T08:49:35.651325+0000","last_undegraded":"2024-07-17T08:49:35.651325+0000","last_fullsized":"2024-07-17T08:49:35.651325+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:35.651145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:35.651145+0000","last_clean_scrub_stamp":"2024-07-17T08:49:35.651145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:21:18.968986+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.28","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.255805+0000","last_change":"2024-07-17T08:49:32.255858+0000","last_active":"2024-07-17T08:49:32.255805+0000","last_peered":"2024-07-17T08:49:32.255805+0000","last_clean":"2024-07-17T08:49:32.255805+0000","last_became_active":"2024-07-17T08:47:06.130785+0000","last_became_peered":"2024-07-17T08:47:06.130785+0000","last_unstale":"2024-07-17T08:49:32.255805+0000","last_undegraded":"2024-07-17T08:49:32.255805+0000","last_fullsized":"2024-07-17T08:49:32.255805+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.255596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.255596+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.255596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:50:01.466507+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.27","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.665964+0000","last_change":"2024-07-17T08:49:32.666083+0000","last_active":"2024-07-17T08:49:32.665964+0000","last_peered":"2024-07-17T08:49:32.665964+0000","last_clean":"2024-07-17T08:49:32.665964+0000","last_became_active":"2024-07-17T08:47:06.142521+0000","last_became_peered":"2024-07-17T08:47:06.142521+0000","last_unstale":"2024-07-17T08:49:32.665964+0000","last_undegraded":"2024-07-17T08:49:32.665964+0000","last_fullsized":"2024-07-17T08:49:32.665964+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.665757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.665757+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.665757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:28:36.230642+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.26","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.362997+0000","last_change":"2024-07-17T08:49:27.363036+0000","last_active":"2024-07-17T08:49:27.362997+0000","last_peered":"2024-07-17T08:49:27.362997+0000","last_clean":"2024-07-17T08:49:27.362997+0000","last_became_active":"2024-07-17T08:47:06.134918+0000","last_became_peered":"2024-07-17T08:47:06.134918+0000","last_unstale":"2024-07-17T08:49:27.362997+0000","last_undegraded":"2024-07-17T08:49:27.362997+0000","last_fullsized":"2024-07-17T08:49:27.362997+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.362886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.362886+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.362886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:59:01.360012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.25","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.843066+0000","last_change":"2024-07-17T08:49:34.843168+0000","last_active":"2024-07-17T08:49:34.843066+0000","last_peered":"2024-07-17T08:49:34.843066+0000","last_clean":"2024-07-17T08:49:34.843066+0000","last_became_active":"2024-07-17T08:47:06.135431+0000","last_became_peered":"2024-07-17T08:47:06.135431+0000","last_unstale":"2024-07-17T08:49:34.843066+0000","last_undegraded":"2024-07-17T08:49:34.843066+0000","last_fullsized":"2024-07-17T08:49:34.843066+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.842866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.842866+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.842866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:16:28.859313+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.24","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:36.875726+0000","last_change":"2024-07-17T08:49:36.875784+0000","last_active":"2024-07-17T08:49:36.875726+0000","last_peered":"2024-07-17T08:49:36.875726+0000","last_clean":"2024-07-17T08:49:36.875726+0000","last_became_active":"2024-07-17T08:47:06.138960+0000","last_became_peered":"2024-07-17T08:47:06.138960+0000","last_unstale":"2024-07-17T08:49:36.875726+0000","last_undegraded":"2024-07-17T08:49:36.875726+0000","last_fullsized":"2024-07-17T08:49:36.875726+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:36.875525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:36.875525+0000","last_clean_scrub_stamp":"2024-07-17T08:49:36.875525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:02.613191+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.23","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:35.859647+0000","last_change":"2024-07-17T08:49:35.859742+0000","last_active":"2024-07-17T08:49:35.859647+0000","last_peered":"2024-07-17T08:49:35.859647+0000","last_clean":"2024-07-17T08:49:35.859647+0000","last_became_active":"2024-07-17T08:47:06.137855+0000","last_became_peered":"2024-07-17T08:47:06.137855+0000","last_unstale":"2024-07-17T08:49:35.859647+0000","last_undegraded":"2024-07-17T08:49:35.859647+0000","last_fullsized":"2024-07-17T08:49:35.859647+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:35.859460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:35.859460+0000","last_clean_scrub_stamp":"2024-07-17T08:49:35.859460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:06:34.012305+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.22","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:38.873972+0000","last_change":"2024-07-17T08:49:38.874025+0000","last_active":"2024-07-17T08:49:38.873972+0000","last_peered":"2024-07-17T08:49:38.873972+0000","last_clean":"2024-07-17T08:49:38.873972+0000","last_became_active":"2024-07-17T08:47:06.137421+0000","last_became_peered":"2024-07-17T08:47:06.137421+0000","last_unstale":"2024-07-17T08:49:38.873972+0000","last_undegraded":"2024-07-17T08:49:38.873972+0000","last_fullsized":"2024-07-17T08:49:38.873972+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:38.873795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:38.873795+0000","last_clean_scrub_stamp":"2024-07-17T08:49:38.873795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:02:03.687946+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.21","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.382614+0000","last_change":"2024-07-17T08:49:34.382697+0000","last_active":"2024-07-17T08:49:34.382614+0000","last_peered":"2024-07-17T08:49:34.382614+0000","last_clean":"2024-07-17T08:49:34.382614+0000","last_became_active":"2024-07-17T08:47:06.140120+0000","last_became_peered":"2024-07-17T08:47:06.140120+0000","last_unstale":"2024-07-17T08:49:34.382614+0000","last_undegraded":"2024-07-17T08:49:34.382614+0000","last_fullsized":"2024-07-17T08:49:34.382614+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.382444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.382444+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.382444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:51:26.043067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.20","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:35.379888+0000","last_change":"2024-07-17T08:49:35.379941+0000","last_active":"2024-07-17T08:49:35.379888+0000","last_peered":"2024-07-17T08:49:35.379888+0000","last_clean":"2024-07-17T08:49:35.379888+0000","last_became_active":"2024-07-17T08:47:06.140560+0000","last_became_peered":"2024-07-17T08:47:06.140560+0000","last_unstale":"2024-07-17T08:49:35.379888+0000","last_undegraded":"2024-07-17T08:49:35.379888+0000","last_fullsized":"2024-07-17T08:49:35.379888+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:35.379716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:35.379716+0000","last_clean_scrub_stamp":"2024-07-17T08:49:35.379716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:13:53.903533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.18","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.431656+0000","last_change":"2024-07-17T08:49:29.431656+0000","last_active":"2024-07-17T08:49:29.431656+0000","last_peered":"2024-07-17T08:49:29.431656+0000","last_clean":"2024-07-17T08:49:29.431656+0000","last_became_active":"2024-07-17T08:47:06.141702+0000","last_became_peered":"2024-07-17T08:47:06.141702+0000","last_unstale":"2024-07-17T08:49:29.431656+0000","last_undegraded":"2024-07-17T08:49:29.431656+0000","last_fullsized":"2024-07-17T08:49:29.431656+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.431310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.431310+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.431310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:37:22.786548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1f","version":"379'6","reported_seq":30,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.870835+0000","last_change":"2024-07-17T08:49:30.870903+0000","last_active":"2024-07-17T08:49:30.870835+0000","last_peered":"2024-07-17T08:49:30.870835+0000","last_clean":"2024-07-17T08:49:30.870835+0000","last_became_active":"2024-07-17T08:47:05.135181+0000","last_became_peered":"2024-07-17T08:47:05.135181+0000","last_unstale":"2024-07-17T08:49:30.870835+0000","last_undegraded":"2024-07-17T08:49:30.870835+0000","last_fullsized":"2024-07-17T08:49:30.870835+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'6","last_scrub_stamp":"2024-07-17T08:49:30.870643+0000","last_deep_scrub":"379'6","last_deep_scrub_stamp":"2024-07-17T08:49:30.870643+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.870643+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:46:16.306802+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.19","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.607129+0000","last_change":"2024-07-17T08:49:29.607129+0000","last_active":"2024-07-17T08:49:29.607129+0000","last_peered":"2024-07-17T08:49:29.607129+0000","last_clean":"2024-07-17T08:49:29.607129+0000","last_became_active":"2024-07-17T08:47:06.114920+0000","last_became_peered":"2024-07-17T08:47:06.114920+0000","last_unstale":"2024-07-17T08:49:29.607129+0000","last_undegraded":"2024-07-17T08:49:29.607129+0000","last_fullsized":"2024-07-17T08:49:29.607129+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.606901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.606901+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.606901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:00:18.457791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1e","version":"378'1","reported_seq":25,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:22.334604+0000","last_change":"2024-07-17T08:49:22.334668+0000","last_active":"2024-07-17T08:49:22.334604+0000","last_peered":"2024-07-17T08:49:22.334604+0000","last_clean":"2024-07-17T08:49:22.334604+0000","last_became_active":"2024-07-17T08:47:05.128569+0000","last_became_peered":"2024-07-17T08:47:05.128569+0000","last_unstale":"2024-07-17T08:49:22.334604+0000","last_undegraded":"2024-07-17T08:49:22.334604+0000","last_fullsized":"2024-07-17T08:49:22.334604+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"378'1","last_scrub_stamp":"2024-07-17T08:49:22.334428+0000","last_deep_scrub":"378'1","last_deep_scrub_stamp":"2024-07-17T08:49:22.334428+0000","last_clean_scrub_stamp":"2024-07-17T08:49:22.334428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:28.012660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1a","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:25.679596+0000","last_change":"2024-07-17T08:49:25.679596+0000","last_active":"2024-07-17T08:49:25.679596+0000","last_peered":"2024-07-17T08:49:25.679596+0000","last_clean":"2024-07-17T08:49:25.679596+0000","last_became_active":"2024-07-17T08:47:06.136118+0000","last_became_peered":"2024-07-17T08:47:06.136118+0000","last_unstale":"2024-07-17T08:49:25.679596+0000","last_undegraded":"2024-07-17T08:49:25.679596+0000","last_fullsized":"2024-07-17T08:49:25.679596+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:25.679195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:25.679195+0000","last_clean_scrub_stamp":"2024-07-17T08:49:25.679195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:01:40.410027+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1d","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:24.405824+0000","last_change":"2024-07-17T08:49:24.405824+0000","last_active":"2024-07-17T08:49:24.405824+0000","last_peered":"2024-07-17T08:49:24.405824+0000","last_clean":"2024-07-17T08:49:24.405824+0000","last_became_active":"2024-07-17T08:47:05.126513+0000","last_became_peered":"2024-07-17T08:47:05.126513+0000","last_unstale":"2024-07-17T08:49:24.405824+0000","last_undegraded":"2024-07-17T08:49:24.405824+0000","last_fullsized":"2024-07-17T08:49:24.405824+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:24.405703+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:24.405703+0000","last_clean_scrub_stamp":"2024-07-17T08:49:24.405703+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:16.050292+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1b","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.455179+0000","last_change":"2024-07-17T08:49:30.455179+0000","last_active":"2024-07-17T08:49:30.455179+0000","last_peered":"2024-07-17T08:49:30.455179+0000","last_clean":"2024-07-17T08:49:30.455179+0000","last_became_active":"2024-07-17T08:47:06.141488+0000","last_became_peered":"2024-07-17T08:47:06.141488+0000","last_unstale":"2024-07-17T08:49:30.455179+0000","last_undegraded":"2024-07-17T08:49:30.455179+0000","last_fullsized":"2024-07-17T08:49:30.455179+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.454956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.454956+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.454956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:45:42.898602+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.899097+0000","last_change":"2024-07-17T08:49:32.899097+0000","last_active":"2024-07-17T08:49:32.899097+0000","last_peered":"2024-07-17T08:49:32.899097+0000","last_clean":"2024-07-17T08:49:32.899097+0000","last_became_active":"2024-07-17T08:47:05.138359+0000","last_became_peered":"2024-07-17T08:47:05.138359+0000","last_unstale":"2024-07-17T08:49:32.899097+0000","last_undegraded":"2024-07-17T08:49:32.899097+0000","last_fullsized":"2024-07-17T08:49:32.899097+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:32.898843+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:32.898843+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.898843+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:56:50.085847+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1c","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.491138+0000","last_change":"2024-07-17T08:49:34.491138+0000","last_active":"2024-07-17T08:49:34.491138+0000","last_peered":"2024-07-17T08:49:34.491138+0000","last_clean":"2024-07-17T08:49:34.491138+0000","last_became_active":"2024-07-17T08:47:06.152516+0000","last_became_peered":"2024-07-17T08:47:06.152516+0000","last_unstale":"2024-07-17T08:49:34.491138+0000","last_undegraded":"2024-07-17T08:49:34.491138+0000","last_fullsized":"2024-07-17T08:49:34.491138+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.490890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.490890+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.490890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:00:32.990903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1b","version":"378'1","reported_seq":25,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.859307+0000","last_change":"2024-07-17T08:49:29.859307+0000","last_active":"2024-07-17T08:49:29.859307+0000","last_peered":"2024-07-17T08:49:29.859307+0000","last_clean":"2024-07-17T08:49:29.859307+0000","last_became_active":"2024-07-17T08:47:05.134774+0000","last_became_peered":"2024-07-17T08:47:05.134774+0000","last_unstale":"2024-07-17T08:49:29.859307+0000","last_undegraded":"2024-07-17T08:49:29.859307+0000","last_fullsized":"2024-07-17T08:49:29.859307+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"378'1","last_scrub_stamp":"2024-07-17T08:49:29.858980+0000","last_deep_scrub":"378'1","last_deep_scrub_stamp":"2024-07-17T08:49:29.858980+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.858980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T08:51:10.355639+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.1d","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.402920+0000","last_change":"2024-07-17T08:49:28.402920+0000","last_active":"2024-07-17T08:49:28.402920+0000","last_peered":"2024-07-17T08:49:28.402920+0000","last_clean":"2024-07-17T08:49:28.402920+0000","last_became_active":"2024-07-17T08:47:06.133707+0000","last_became_peered":"2024-07-17T08:47:06.133707+0000","last_unstale":"2024-07-17T08:49:28.402920+0000","last_undegraded":"2024-07-17T08:49:28.402920+0000","last_fullsized":"2024-07-17T08:49:28.402920+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.402691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.402691+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.402691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:20:16.037527+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1a","version":"378'1","reported_seq":25,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:25.388942+0000","last_change":"2024-07-17T08:49:25.388942+0000","last_active":"2024-07-17T08:49:25.388942+0000","last_peered":"2024-07-17T08:49:25.388942+0000","last_clean":"2024-07-17T08:49:25.388942+0000","last_became_active":"2024-07-17T08:47:05.133681+0000","last_became_peered":"2024-07-17T08:47:05.133681+0000","last_unstale":"2024-07-17T08:49:25.388942+0000","last_undegraded":"2024-07-17T08:49:25.388942+0000","last_fullsized":"2024-07-17T08:49:25.388942+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"378'1","last_scrub_stamp":"2024-07-17T08:49:25.388524+0000","last_deep_scrub":"378'1","last_deep_scrub_stamp":"2024-07-17T08:49:25.388524+0000","last_clean_scrub_stamp":"2024-07-17T08:49:25.388524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:20:20.276566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.1e","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.681163+0000","last_change":"2024-07-17T08:49:31.681223+0000","last_active":"2024-07-17T08:49:31.681163+0000","last_peered":"2024-07-17T08:49:31.681163+0000","last_clean":"2024-07-17T08:49:31.681163+0000","last_became_active":"2024-07-17T08:47:06.132052+0000","last_became_peered":"2024-07-17T08:47:06.132052+0000","last_unstale":"2024-07-17T08:49:31.681163+0000","last_undegraded":"2024-07-17T08:49:31.681163+0000","last_fullsized":"2024-07-17T08:49:31.681163+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.680989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.680989+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.680989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:00:16.746767+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:25.357258+0000","last_change":"2024-07-17T08:49:25.357258+0000","last_active":"2024-07-17T08:49:25.357258+0000","last_peered":"2024-07-17T08:49:25.357258+0000","last_clean":"2024-07-17T08:49:25.357258+0000","last_became_active":"2024-07-17T08:47:05.128942+0000","last_became_peered":"2024-07-17T08:47:05.128942+0000","last_unstale":"2024-07-17T08:49:25.357258+0000","last_undegraded":"2024-07-17T08:49:25.357258+0000","last_fullsized":"2024-07-17T08:49:25.357258+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:25.357038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:25.357038+0000","last_clean_scrub_stamp":"2024-07-17T08:49:25.357038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:43:50.806269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.1f","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.324091+0000","last_change":"2024-07-17T08:49:26.324120+0000","last_active":"2024-07-17T08:49:26.324091+0000","last_peered":"2024-07-17T08:49:26.324091+0000","last_clean":"2024-07-17T08:49:26.324091+0000","last_became_active":"2024-07-17T08:47:06.134413+0000","last_became_peered":"2024-07-17T08:47:06.134413+0000","last_unstale":"2024-07-17T08:49:26.324091+0000","last_undegraded":"2024-07-17T08:49:26.324091+0000","last_fullsized":"2024-07-17T08:49:26.324091+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:26.323986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:26.323986+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.323986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:48.929544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.239466+0000","last_change":"2024-07-17T08:49:28.239466+0000","last_active":"2024-07-17T08:49:28.239466+0000","last_peered":"2024-07-17T08:49:28.239466+0000","last_clean":"2024-07-17T08:49:28.239466+0000","last_became_active":"2024-07-17T08:47:05.136191+0000","last_became_peered":"2024-07-17T08:47:05.136191+0000","last_unstale":"2024-07-17T08:49:28.239466+0000","last_undegraded":"2024-07-17T08:49:28.239466+0000","last_fullsized":"2024-07-17T08:49:28.239466+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.239231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.239231+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.239231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:07:45.393081+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.10","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:37.369375+0000","last_change":"2024-07-17T08:49:37.369434+0000","last_active":"2024-07-17T08:49:37.369375+0000","last_peered":"2024-07-17T08:49:37.369375+0000","last_clean":"2024-07-17T08:49:37.369375+0000","last_became_active":"2024-07-17T08:47:06.145736+0000","last_became_peered":"2024-07-17T08:47:06.145736+0000","last_unstale":"2024-07-17T08:49:37.369375+0000","last_undegraded":"2024-07-17T08:49:37.369375+0000","last_fullsized":"2024-07-17T08:49:37.369375+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:37.369204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:37.369204+0000","last_clean_scrub_stamp":"2024-07-17T08:49:37.369204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:48:31.836829+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.667849+0000","last_change":"2024-07-17T08:49:29.667849+0000","last_active":"2024-07-17T08:49:29.667849+0000","last_peered":"2024-07-17T08:49:29.667849+0000","last_clean":"2024-07-17T08:49:29.667849+0000","last_became_active":"2024-07-17T08:47:05.131469+0000","last_became_peered":"2024-07-17T08:47:05.131469+0000","last_unstale":"2024-07-17T08:49:29.667849+0000","last_undegraded":"2024-07-17T08:49:29.667849+0000","last_fullsized":"2024-07-17T08:49:29.667849+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.667612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.667612+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.667612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:54:11.718082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.11","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:24.721984+0000","last_change":"2024-07-17T08:49:24.721984+0000","last_active":"2024-07-17T08:49:24.721984+0000","last_peered":"2024-07-17T08:49:24.721984+0000","last_clean":"2024-07-17T08:49:24.721984+0000","last_became_active":"2024-07-17T08:47:06.135383+0000","last_became_peered":"2024-07-17T08:47:06.135383+0000","last_unstale":"2024-07-17T08:49:24.721984+0000","last_undegraded":"2024-07-17T08:49:24.721984+0000","last_fullsized":"2024-07-17T08:49:24.721984+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:24.721755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:24.721755+0000","last_clean_scrub_stamp":"2024-07-17T08:49:24.721755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:59:46.459595+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.16","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.244708+0000","last_change":"2024-07-17T08:49:27.244708+0000","last_active":"2024-07-17T08:49:27.244708+0000","last_peered":"2024-07-17T08:49:27.244708+0000","last_clean":"2024-07-17T08:49:27.244708+0000","last_became_active":"2024-07-17T08:47:05.132170+0000","last_became_peered":"2024-07-17T08:47:05.132170+0000","last_unstale":"2024-07-17T08:49:27.244708+0000","last_undegraded":"2024-07-17T08:49:27.244708+0000","last_fullsized":"2024-07-17T08:49:27.244708+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.244444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.244444+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.244444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:45:20.472858+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.12","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.489836+0000","last_change":"2024-07-17T08:49:32.489890+0000","last_active":"2024-07-17T08:49:32.489836+0000","last_peered":"2024-07-17T08:49:32.489836+0000","last_clean":"2024-07-17T08:49:32.489836+0000","last_became_active":"2024-07-17T08:47:06.143991+0000","last_became_peered":"2024-07-17T08:47:06.143991+0000","last_unstale":"2024-07-17T08:49:32.489836+0000","last_undegraded":"2024-07-17T08:49:32.489836+0000","last_fullsized":"2024-07-17T08:49:32.489836+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.489652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.489652+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.489652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:28:25.932792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.15","version":"379'5","reported_seq":29,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:24.294064+0000","last_change":"2024-07-17T08:49:24.294064+0000","last_active":"2024-07-17T08:49:24.294064+0000","last_peered":"2024-07-17T08:49:24.294064+0000","last_clean":"2024-07-17T08:49:24.294064+0000","last_became_active":"2024-07-17T08:47:05.132913+0000","last_became_peered":"2024-07-17T08:47:05.132913+0000","last_unstale":"2024-07-17T08:49:24.294064+0000","last_undegraded":"2024-07-17T08:49:24.294064+0000","last_fullsized":"2024-07-17T08:49:24.294064+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'5","last_scrub_stamp":"2024-07-17T08:49:24.293958+0000","last_deep_scrub":"379'5","last_deep_scrub_stamp":"2024-07-17T08:49:24.293958+0000","last_clean_scrub_stamp":"2024-07-17T08:49:24.293958+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:21:56.093543+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"52.13","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.653183+0000","last_change":"2024-07-17T08:49:28.653234+0000","last_active":"2024-07-17T08:49:28.653183+0000","last_peered":"2024-07-17T08:49:28.653183+0000","last_clean":"2024-07-17T08:49:28.653183+0000","last_became_active":"2024-07-17T08:47:06.114679+0000","last_became_peered":"2024-07-17T08:47:06.114679+0000","last_unstale":"2024-07-17T08:49:28.653183+0000","last_undegraded":"2024-07-17T08:49:28.653183+0000","last_fullsized":"2024-07-17T08:49:28.653183+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.652997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.652997+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.652997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:41:21.201636+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"379'12","reported_seq":36,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.698223+0000","last_change":"2024-07-17T08:49:27.698223+0000","last_active":"2024-07-17T08:49:27.698223+0000","last_peered":"2024-07-17T08:49:27.698223+0000","last_clean":"2024-07-17T08:49:27.698223+0000","last_became_active":"2024-07-17T08:47:05.130671+0000","last_became_peered":"2024-07-17T08:47:05.130671+0000","last_unstale":"2024-07-17T08:49:27.698223+0000","last_undegraded":"2024-07-17T08:49:27.698223+0000","last_fullsized":"2024-07-17T08:49:27.698223+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'12","last_scrub_stamp":"2024-07-17T08:49:27.697974+0000","last_deep_scrub":"379'12","last_deep_scrub_stamp":"2024-07-17T08:49:27.697974+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.697974+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:55:54.332742+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.14","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:41.681690+0000","last_change":"2024-07-17T08:49:41.681746+0000","last_active":"2024-07-17T08:49:41.681690+0000","last_peered":"2024-07-17T08:49:41.681690+0000","last_clean":"2024-07-17T08:49:41.681690+0000","last_became_active":"2024-07-17T08:47:06.151074+0000","last_became_peered":"2024-07-17T08:47:06.151074+0000","last_unstale":"2024-07-17T08:49:41.681690+0000","last_undegraded":"2024-07-17T08:49:41.681690+0000","last_fullsized":"2024-07-17T08:49:41.681690+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:41.681492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:41.681492+0000","last_clean_scrub_stamp":"2024-07-17T08:49:41.681492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:48:53.608471+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.13","version":"379'4","reported_seq":28,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.416850+0000","last_change":"2024-07-17T08:49:26.416850+0000","last_active":"2024-07-17T08:49:26.416850+0000","last_peered":"2024-07-17T08:49:26.416850+0000","last_clean":"2024-07-17T08:49:26.416850+0000","last_became_active":"2024-07-17T08:47:05.135091+0000","last_became_peered":"2024-07-17T08:47:05.135091+0000","last_unstale":"2024-07-17T08:49:26.416850+0000","last_undegraded":"2024-07-17T08:49:26.416850+0000","last_fullsized":"2024-07-17T08:49:26.416850+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'4","last_scrub_stamp":"2024-07-17T08:49:26.416754+0000","last_deep_scrub":"379'4","last_deep_scrub_stamp":"2024-07-17T08:49:26.416754+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.416754+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:10:59.394616+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.15","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:37.874545+0000","last_change":"2024-07-17T08:49:37.874603+0000","last_active":"2024-07-17T08:49:37.874545+0000","last_peered":"2024-07-17T08:49:37.874545+0000","last_clean":"2024-07-17T08:49:37.874545+0000","last_became_active":"2024-07-17T08:47:06.139081+0000","last_became_peered":"2024-07-17T08:47:06.139081+0000","last_unstale":"2024-07-17T08:49:37.874545+0000","last_undegraded":"2024-07-17T08:49:37.874545+0000","last_fullsized":"2024-07-17T08:49:37.874545+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:37.874372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:37.874372+0000","last_clean_scrub_stamp":"2024-07-17T08:49:37.874372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:34:44.980605+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.12","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.700406+0000","last_change":"2024-07-17T08:49:26.700406+0000","last_active":"2024-07-17T08:49:26.700406+0000","last_peered":"2024-07-17T08:49:26.700406+0000","last_clean":"2024-07-17T08:49:26.700406+0000","last_became_active":"2024-07-17T08:47:05.118661+0000","last_became_peered":"2024-07-17T08:47:05.118661+0000","last_unstale":"2024-07-17T08:49:26.700406+0000","last_undegraded":"2024-07-17T08:49:26.700406+0000","last_fullsized":"2024-07-17T08:49:26.700406+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:26.700169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:26.700169+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.700169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:35:16.523674+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.16","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:39.689103+0000","last_change":"2024-07-17T08:49:39.689158+0000","last_active":"2024-07-17T08:49:39.689103+0000","last_peered":"2024-07-17T08:49:39.689103+0000","last_clean":"2024-07-17T08:49:39.689103+0000","last_became_active":"2024-07-17T08:47:06.149409+0000","last_became_peered":"2024-07-17T08:47:06.149409+0000","last_unstale":"2024-07-17T08:49:39.689103+0000","last_undegraded":"2024-07-17T08:49:39.689103+0000","last_fullsized":"2024-07-17T08:49:39.689103+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:39.688915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:39.688915+0000","last_clean_scrub_stamp":"2024-07-17T08:49:39.688915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:04:17.178682+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.392035+0000","last_change":"2024-07-17T08:49:26.392035+0000","last_active":"2024-07-17T08:49:26.392035+0000","last_peered":"2024-07-17T08:49:26.392035+0000","last_clean":"2024-07-17T08:49:26.392035+0000","last_became_active":"2024-07-17T08:47:05.123769+0000","last_became_peered":"2024-07-17T08:47:05.123769+0000","last_unstale":"2024-07-17T08:49:26.392035+0000","last_undegraded":"2024-07-17T08:49:26.392035+0000","last_fullsized":"2024-07-17T08:49:26.392035+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:26.391735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:26.391735+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.391735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:30:47.377186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.17","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:38.347290+0000","last_change":"2024-07-17T08:49:38.347347+0000","last_active":"2024-07-17T08:49:38.347290+0000","last_peered":"2024-07-17T08:49:38.347290+0000","last_clean":"2024-07-17T08:49:38.347290+0000","last_became_active":"2024-07-17T08:47:06.146467+0000","last_became_peered":"2024-07-17T08:47:06.146467+0000","last_unstale":"2024-07-17T08:49:38.347290+0000","last_undegraded":"2024-07-17T08:49:38.347290+0000","last_fullsized":"2024-07-17T08:49:38.347290+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:38.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:38.347114+0000","last_clean_scrub_stamp":"2024-07-17T08:49:38.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:18:19.764975+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"379'12","reported_seq":36,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.294731+0000","last_change":"2024-07-17T08:49:26.294731+0000","last_active":"2024-07-17T08:49:26.294731+0000","last_peered":"2024-07-17T08:49:26.294731+0000","last_clean":"2024-07-17T08:49:26.294731+0000","last_became_active":"2024-07-17T08:47:05.132884+0000","last_became_peered":"2024-07-17T08:47:05.132884+0000","last_unstale":"2024-07-17T08:49:26.294731+0000","last_undegraded":"2024-07-17T08:49:26.294731+0000","last_fullsized":"2024-07-17T08:49:26.294731+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'12","last_scrub_stamp":"2024-07-17T08:49:26.294436+0000","last_deep_scrub":"379'12","last_deep_scrub_stamp":"2024-07-17T08:49:26.294436+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.294436+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:41:10.094977+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.8","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.404698+0000","last_change":"2024-07-17T08:49:27.404757+0000","last_active":"2024-07-17T08:49:27.404698+0000","last_peered":"2024-07-17T08:49:27.404698+0000","last_clean":"2024-07-17T08:49:27.404698+0000","last_became_active":"2024-07-17T08:47:06.127338+0000","last_became_peered":"2024-07-17T08:47:06.127338+0000","last_unstale":"2024-07-17T08:49:27.404698+0000","last_undegraded":"2024-07-17T08:49:27.404698+0000","last_fullsized":"2024-07-17T08:49:27.404698+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.404512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.404512+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.404512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:00:26.651753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.f","version":"379'2","reported_seq":26,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:23.414723+0000","last_change":"2024-07-17T08:49:23.414785+0000","last_active":"2024-07-17T08:49:23.414723+0000","last_peered":"2024-07-17T08:49:23.414723+0000","last_clean":"2024-07-17T08:49:23.414723+0000","last_became_active":"2024-07-17T08:47:05.125379+0000","last_became_peered":"2024-07-17T08:47:05.125379+0000","last_unstale":"2024-07-17T08:49:23.414723+0000","last_undegraded":"2024-07-17T08:49:23.414723+0000","last_fullsized":"2024-07-17T08:49:23.414723+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'2","last_scrub_stamp":"2024-07-17T08:49:23.414597+0000","last_deep_scrub":"379'2","last_deep_scrub_stamp":"2024-07-17T08:49:23.414597+0000","last_clean_scrub_stamp":"2024-07-17T08:49:23.414597+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:15:43.791975+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"52.9","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:41.959828+0000","last_change":"2024-07-17T08:49:41.959880+0000","last_active":"2024-07-17T08:49:41.959828+0000","last_peered":"2024-07-17T08:49:41.959828+0000","last_clean":"2024-07-17T08:49:41.959828+0000","last_became_active":"2024-07-17T08:47:06.146366+0000","last_became_peered":"2024-07-17T08:47:06.146366+0000","last_unstale":"2024-07-17T08:49:41.959828+0000","last_undegraded":"2024-07-17T08:49:41.959828+0000","last_fullsized":"2024-07-17T08:49:41.959828+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:41.959632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:41.959632+0000","last_clean_scrub_stamp":"2024-07-17T08:49:41.959632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:27:45.616707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.e","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:25.299592+0000","last_change":"2024-07-17T08:49:25.299661+0000","last_active":"2024-07-17T08:49:25.299592+0000","last_peered":"2024-07-17T08:49:25.299592+0000","last_clean":"2024-07-17T08:49:25.299592+0000","last_became_active":"2024-07-17T08:47:05.131783+0000","last_became_peered":"2024-07-17T08:47:05.131783+0000","last_unstale":"2024-07-17T08:49:25.299592+0000","last_undegraded":"2024-07-17T08:49:25.299592+0000","last_fullsized":"2024-07-17T08:49:25.299592+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:25.299428+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:25.299428+0000","last_clean_scrub_stamp":"2024-07-17T08:49:25.299428+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:59:38.942868+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"52.a","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.348329+0000","last_change":"2024-07-17T08:49:30.348382+0000","last_active":"2024-07-17T08:49:30.348329+0000","last_peered":"2024-07-17T08:49:30.348329+0000","last_clean":"2024-07-17T08:49:30.348329+0000","last_became_active":"2024-07-17T08:47:06.128095+0000","last_became_peered":"2024-07-17T08:47:06.128095+0000","last_unstale":"2024-07-17T08:49:30.348329+0000","last_undegraded":"2024-07-17T08:49:30.348329+0000","last_fullsized":"2024-07-17T08:49:30.348329+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.348142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.348142+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.348142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:00:25.557636+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.d","version":"379'2","reported_seq":26,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:26.652122+0000","last_change":"2024-07-17T08:49:26.652239+0000","last_active":"2024-07-17T08:49:26.652122+0000","last_peered":"2024-07-17T08:49:26.652122+0000","last_clean":"2024-07-17T08:49:26.652122+0000","last_became_active":"2024-07-17T08:47:05.126005+0000","last_became_peered":"2024-07-17T08:47:05.126005+0000","last_unstale":"2024-07-17T08:49:26.652122+0000","last_undegraded":"2024-07-17T08:49:26.652122+0000","last_fullsized":"2024-07-17T08:49:26.652122+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'2","last_scrub_stamp":"2024-07-17T08:49:26.651928+0000","last_deep_scrub":"379'2","last_deep_scrub_stamp":"2024-07-17T08:49:26.651928+0000","last_clean_scrub_stamp":"2024-07-17T08:49:26.651928+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:42:27.598548+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.b","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:36.399926+0000","last_change":"2024-07-17T08:49:36.399980+0000","last_active":"2024-07-17T08:49:36.399926+0000","last_peered":"2024-07-17T08:49:36.399926+0000","last_clean":"2024-07-17T08:49:36.399926+0000","last_became_active":"2024-07-17T08:47:06.145620+0000","last_became_peered":"2024-07-17T08:47:06.145620+0000","last_unstale":"2024-07-17T08:49:36.399926+0000","last_undegraded":"2024-07-17T08:49:36.399926+0000","last_fullsized":"2024-07-17T08:49:36.399926+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:36.399728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:36.399728+0000","last_clean_scrub_stamp":"2024-07-17T08:49:36.399728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:28:29.842867+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.892141+0000","last_change":"2024-07-17T08:49:33.892247+0000","last_active":"2024-07-17T08:49:33.892141+0000","last_peered":"2024-07-17T08:49:33.892141+0000","last_clean":"2024-07-17T08:49:33.892141+0000","last_became_active":"2024-07-17T08:47:05.138988+0000","last_became_peered":"2024-07-17T08:47:05.138988+0000","last_unstale":"2024-07-17T08:49:33.892141+0000","last_undegraded":"2024-07-17T08:49:33.892141+0000","last_fullsized":"2024-07-17T08:49:33.892141+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.891941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.891941+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.891941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:06:03.852318+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.c","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:40.648371+0000","last_change":"2024-07-17T08:49:40.648430+0000","last_active":"2024-07-17T08:49:40.648371+0000","last_peered":"2024-07-17T08:49:40.648371+0000","last_clean":"2024-07-17T08:49:40.648371+0000","last_became_active":"2024-07-17T08:47:06.150788+0000","last_became_peered":"2024-07-17T08:47:06.150788+0000","last_unstale":"2024-07-17T08:49:40.648371+0000","last_undegraded":"2024-07-17T08:49:40.648371+0000","last_fullsized":"2024-07-17T08:49:40.648371+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:40.648190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:40.648190+0000","last_clean_scrub_stamp":"2024-07-17T08:49:40.648190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:25:06.375372+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.b","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:24.380434+0000","last_change":"2024-07-17T08:49:24.380460+0000","last_active":"2024-07-17T08:49:24.380434+0000","last_peered":"2024-07-17T08:49:24.380434+0000","last_clean":"2024-07-17T08:49:24.380434+0000","last_became_active":"2024-07-17T08:47:05.133570+0000","last_became_peered":"2024-07-17T08:47:05.133570+0000","last_unstale":"2024-07-17T08:49:24.380434+0000","last_undegraded":"2024-07-17T08:49:24.380434+0000","last_fullsized":"2024-07-17T08:49:24.380434+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:24.380363+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:24.380363+0000","last_clean_scrub_stamp":"2024-07-17T08:49:24.380363+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T11:55:25.831737+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.d","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.382124+0000","last_change":"2024-07-17T08:49:30.382177+0000","last_active":"2024-07-17T08:49:30.382124+0000","last_peered":"2024-07-17T08:49:30.382124+0000","last_clean":"2024-07-17T08:49:30.382124+0000","last_became_active":"2024-07-17T08:47:06.144846+0000","last_became_peered":"2024-07-17T08:47:06.144846+0000","last_unstale":"2024-07-17T08:49:30.382124+0000","last_undegraded":"2024-07-17T08:49:30.382124+0000","last_fullsized":"2024-07-17T08:49:30.382124+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.381950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.381950+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.381950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:55:54.599428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.912120+0000","last_change":"2024-07-17T08:49:31.912174+0000","last_active":"2024-07-17T08:49:31.912120+0000","last_peered":"2024-07-17T08:49:31.912120+0000","last_clean":"2024-07-17T08:49:31.912120+0000","last_became_active":"2024-07-17T08:47:05.138121+0000","last_became_peered":"2024-07-17T08:47:05.138121+0000","last_unstale":"2024-07-17T08:49:31.912120+0000","last_undegraded":"2024-07-17T08:49:31.912120+0000","last_fullsized":"2024-07-17T08:49:31.912120+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.911941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.911941+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.911941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:57:06.224533+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.e","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.343606+0000","last_change":"2024-07-17T08:49:32.343661+0000","last_active":"2024-07-17T08:49:32.343606+0000","last_peered":"2024-07-17T08:49:32.343606+0000","last_clean":"2024-07-17T08:49:32.343606+0000","last_became_active":"2024-07-17T08:47:06.135115+0000","last_became_peered":"2024-07-17T08:47:06.135115+0000","last_unstale":"2024-07-17T08:49:32.343606+0000","last_undegraded":"2024-07-17T08:49:32.343606+0000","last_fullsized":"2024-07-17T08:49:32.343606+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.343430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.343430+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.343430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:35.857340+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.691200+0000","last_change":"2024-07-17T08:49:27.691262+0000","last_active":"2024-07-17T08:49:27.691200+0000","last_peered":"2024-07-17T08:49:27.691200+0000","last_clean":"2024-07-17T08:49:27.691200+0000","last_became_active":"2024-07-17T08:47:05.121265+0000","last_became_peered":"2024-07-17T08:47:05.121265+0000","last_unstale":"2024-07-17T08:49:27.691200+0000","last_undegraded":"2024-07-17T08:49:27.691200+0000","last_fullsized":"2024-07-17T08:49:27.691200+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.691026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.691026+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.691026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:22:03.444372+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"52.f","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:33.679401+0000","last_change":"2024-07-17T08:49:33.679455+0000","last_active":"2024-07-17T08:49:33.679401+0000","last_peered":"2024-07-17T08:49:33.679401+0000","last_clean":"2024-07-17T08:49:33.679401+0000","last_became_active":"2024-07-17T08:47:06.144255+0000","last_became_peered":"2024-07-17T08:47:06.144255+0000","last_unstale":"2024-07-17T08:49:33.679401+0000","last_undegraded":"2024-07-17T08:49:33.679401+0000","last_fullsized":"2024-07-17T08:49:33.679401+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:33.679219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:33.679219+0000","last_clean_scrub_stamp":"2024-07-17T08:49:33.679219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:46:19.298841+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.832576+0000","last_change":"2024-07-17T08:49:28.832632+0000","last_active":"2024-07-17T08:49:28.832576+0000","last_peered":"2024-07-17T08:49:28.832576+0000","last_clean":"2024-07-17T08:49:28.832576+0000","last_became_active":"2024-07-17T08:47:05.134405+0000","last_became_peered":"2024-07-17T08:47:05.134405+0000","last_unstale":"2024-07-17T08:49:28.832576+0000","last_undegraded":"2024-07-17T08:49:28.832576+0000","last_fullsized":"2024-07-17T08:49:28.832576+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.832391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.832391+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.832391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:34:16.048942+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"52.0","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.323389+0000","last_change":"2024-07-17T08:49:30.323444+0000","last_active":"2024-07-17T08:49:30.323389+0000","last_peered":"2024-07-17T08:49:30.323389+0000","last_clean":"2024-07-17T08:49:30.323389+0000","last_became_active":"2024-07-17T08:47:06.121944+0000","last_became_peered":"2024-07-17T08:47:06.121944+0000","last_unstale":"2024-07-17T08:49:30.323389+0000","last_undegraded":"2024-07-17T08:49:30.323389+0000","last_fullsized":"2024-07-17T08:49:30.323389+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.323210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.323210+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.323210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:26:58.486765+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.7","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.354178+0000","last_change":"2024-07-17T08:49:29.354240+0000","last_active":"2024-07-17T08:49:29.354178+0000","last_peered":"2024-07-17T08:49:29.354178+0000","last_clean":"2024-07-17T08:49:29.354178+0000","last_became_active":"2024-07-17T08:47:05.133432+0000","last_became_peered":"2024-07-17T08:47:05.133432+0000","last_unstale":"2024-07-17T08:49:29.354178+0000","last_undegraded":"2024-07-17T08:49:29.354178+0000","last_fullsized":"2024-07-17T08:49:29.354178+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:29.354010+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:29.354010+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.354010+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:26:34.797145+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.1","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:34.631847+0000","last_change":"2024-07-17T08:49:34.631900+0000","last_active":"2024-07-17T08:49:34.631847+0000","last_peered":"2024-07-17T08:49:34.631847+0000","last_clean":"2024-07-17T08:49:34.631847+0000","last_became_active":"2024-07-17T08:47:06.135523+0000","last_became_peered":"2024-07-17T08:47:06.135523+0000","last_unstale":"2024-07-17T08:49:34.631847+0000","last_undegraded":"2024-07-17T08:49:34.631847+0000","last_fullsized":"2024-07-17T08:49:34.631847+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:34.631678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:34.631678+0000","last_clean_scrub_stamp":"2024-07-17T08:49:34.631678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T17:51:14.444731+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.6","version":"379'3","reported_seq":27,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.374071+0000","last_change":"2024-07-17T08:49:28.374137+0000","last_active":"2024-07-17T08:49:28.374071+0000","last_peered":"2024-07-17T08:49:28.374071+0000","last_clean":"2024-07-17T08:49:28.374071+0000","last_became_active":"2024-07-17T08:47:05.128340+0000","last_became_peered":"2024-07-17T08:47:05.128340+0000","last_unstale":"2024-07-17T08:49:28.374071+0000","last_undegraded":"2024-07-17T08:49:28.374071+0000","last_fullsized":"2024-07-17T08:49:28.374071+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'3","last_scrub_stamp":"2024-07-17T08:49:28.373901+0000","last_deep_scrub":"379'3","last_deep_scrub_stamp":"2024-07-17T08:49:28.373901+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.373901+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:43:53.823132+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.2","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.420870+0000","last_change":"2024-07-17T08:49:27.420933+0000","last_active":"2024-07-17T08:49:27.420870+0000","last_peered":"2024-07-17T08:49:27.420870+0000","last_clean":"2024-07-17T08:49:27.420870+0000","last_became_active":"2024-07-17T08:47:06.136316+0000","last_became_peered":"2024-07-17T08:47:06.136316+0000","last_unstale":"2024-07-17T08:49:27.420870+0000","last_undegraded":"2024-07-17T08:49:27.420870+0000","last_fullsized":"2024-07-17T08:49:27.420870+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.420656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.420656+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.420656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T18:25:30.934565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.631478+0000","last_change":"2024-07-17T08:49:30.631536+0000","last_active":"2024-07-17T08:49:30.631478+0000","last_peered":"2024-07-17T08:49:30.631478+0000","last_clean":"2024-07-17T08:49:30.631478+0000","last_became_active":"2024-07-17T08:47:05.132453+0000","last_became_peered":"2024-07-17T08:47:05.132453+0000","last_unstale":"2024-07-17T08:49:30.631478+0000","last_undegraded":"2024-07-17T08:49:30.631478+0000","last_fullsized":"2024-07-17T08:49:30.631478+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.631304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.631304+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.631304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:05:25.540914+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.3","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.351072+0000","last_change":"2024-07-17T08:49:31.351129+0000","last_active":"2024-07-17T08:49:31.351072+0000","last_peered":"2024-07-17T08:49:31.351072+0000","last_clean":"2024-07-17T08:49:31.351072+0000","last_became_active":"2024-07-17T08:47:06.133197+0000","last_became_peered":"2024-07-17T08:47:06.133197+0000","last_unstale":"2024-07-17T08:49:31.351072+0000","last_undegraded":"2024-07-17T08:49:31.351072+0000","last_fullsized":"2024-07-17T08:49:31.351072+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.350901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.350901+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.350901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T13:40:42.419754+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:23.332891+0000","last_change":"2024-07-17T08:49:23.332912+0000","last_active":"2024-07-17T08:49:23.332891+0000","last_peered":"2024-07-17T08:49:23.332891+0000","last_clean":"2024-07-17T08:49:23.332891+0000","last_became_active":"2024-07-17T08:47:05.133291+0000","last_became_peered":"2024-07-17T08:47:05.133291+0000","last_unstale":"2024-07-17T08:49:23.332891+0000","last_undegraded":"2024-07-17T08:49:23.332891+0000","last_fullsized":"2024-07-17T08:49:23.332891+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:23.332816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:23.332816+0000","last_clean_scrub_stamp":"2024-07-17T08:49:23.332816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T19:56:41.887640+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"52.4","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:30.648250+0000","last_change":"2024-07-17T08:49:30.648303+0000","last_active":"2024-07-17T08:49:30.648250+0000","last_peered":"2024-07-17T08:49:30.648250+0000","last_clean":"2024-07-17T08:49:30.648250+0000","last_became_active":"2024-07-17T08:47:06.115359+0000","last_became_peered":"2024-07-17T08:47:06.115359+0000","last_unstale":"2024-07-17T08:49:30.648250+0000","last_undegraded":"2024-07-17T08:49:30.648250+0000","last_fullsized":"2024-07-17T08:49:30.648250+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:30.648065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:30.648065+0000","last_clean_scrub_stamp":"2024-07-17T08:49:30.648065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T09:02:35.111845+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:32.645498+0000","last_change":"2024-07-17T08:49:32.645572+0000","last_active":"2024-07-17T08:49:32.645498+0000","last_peered":"2024-07-17T08:49:32.645498+0000","last_clean":"2024-07-17T08:49:32.645498+0000","last_became_active":"2024-07-17T08:47:05.135168+0000","last_became_peered":"2024-07-17T08:47:05.135168+0000","last_unstale":"2024-07-17T08:49:32.645498+0000","last_undegraded":"2024-07-17T08:49:32.645498+0000","last_fullsized":"2024-07-17T08:49:32.645498+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:32.645327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:32.645327+0000","last_clean_scrub_stamp":"2024-07-17T08:49:32.645327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T20:45:43.464635+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.5","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:36.695237+0000","last_change":"2024-07-17T08:49:36.695294+0000","last_active":"2024-07-17T08:49:36.695237+0000","last_peered":"2024-07-17T08:49:36.695237+0000","last_clean":"2024-07-17T08:49:36.695237+0000","last_became_active":"2024-07-17T08:47:06.137168+0000","last_became_peered":"2024-07-17T08:47:06.137168+0000","last_unstale":"2024-07-17T08:49:36.695237+0000","last_undegraded":"2024-07-17T08:49:36.695237+0000","last_fullsized":"2024-07-17T08:49:36.695237+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:36.695067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:36.695067+0000","last_clean_scrub_stamp":"2024-07-17T08:49:36.695067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:13:09.654805+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:31.603434+0000","last_change":"2024-07-17T08:49:31.603492+0000","last_active":"2024-07-17T08:49:31.603434+0000","last_peered":"2024-07-17T08:49:31.603434+0000","last_clean":"2024-07-17T08:49:31.603434+0000","last_became_active":"2024-07-17T08:47:05.134400+0000","last_became_peered":"2024-07-17T08:47:05.134400+0000","last_unstale":"2024-07-17T08:49:31.603434+0000","last_undegraded":"2024-07-17T08:49:31.603434+0000","last_fullsized":"2024-07-17T08:49:31.603434+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:31.603253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:31.603253+0000","last_clean_scrub_stamp":"2024-07-17T08:49:31.603253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:28:36.020985+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.6","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:40.952842+0000","last_change":"2024-07-17T08:49:40.952897+0000","last_active":"2024-07-17T08:49:40.952842+0000","last_peered":"2024-07-17T08:49:40.952842+0000","last_clean":"2024-07-17T08:49:40.952842+0000","last_became_active":"2024-07-17T08:47:06.144725+0000","last_became_peered":"2024-07-17T08:47:06.144725+0000","last_unstale":"2024-07-17T08:49:40.952842+0000","last_undegraded":"2024-07-17T08:49:40.952842+0000","last_fullsized":"2024-07-17T08:49:40.952842+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:40.952643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:40.952643+0000","last_clean_scrub_stamp":"2024-07-17T08:49:40.952643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T12:33:47.669052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:28.645006+0000","last_change":"2024-07-17T08:49:28.645062+0000","last_active":"2024-07-17T08:49:28.645006+0000","last_peered":"2024-07-17T08:49:28.645006+0000","last_clean":"2024-07-17T08:49:28.645006+0000","last_became_active":"2024-07-17T08:47:05.131061+0000","last_became_peered":"2024-07-17T08:47:05.131061+0000","last_unstale":"2024-07-17T08:49:28.645006+0000","last_undegraded":"2024-07-17T08:49:28.645006+0000","last_fullsized":"2024-07-17T08:49:28.645006+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:28.644827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:28.644827+0000","last_clean_scrub_stamp":"2024-07-17T08:49:28.644827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T14:02:36.366910+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"52.7","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:39.905872+0000","last_change":"2024-07-17T08:49:39.905927+0000","last_active":"2024-07-17T08:49:39.905872+0000","last_peered":"2024-07-17T08:49:39.905872+0000","last_clean":"2024-07-17T08:49:39.905872+0000","last_became_active":"2024-07-17T08:47:06.144597+0000","last_became_peered":"2024-07-17T08:47:06.144597+0000","last_unstale":"2024-07-17T08:49:39.905872+0000","last_undegraded":"2024-07-17T08:49:39.905872+0000","last_fullsized":"2024-07-17T08:49:39.905872+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:39.905657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:39.905657+0000","last_clean_scrub_stamp":"2024-07-17T08:49:39.905657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:29:30.249842+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":24,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.365500+0000","last_change":"2024-07-17T08:49:27.365555+0000","last_active":"2024-07-17T08:49:27.365500+0000","last_peered":"2024-07-17T08:49:27.365500+0000","last_clean":"2024-07-17T08:49:27.365500+0000","last_became_active":"2024-07-17T08:47:05.126976+0000","last_became_peered":"2024-07-17T08:47:05.126976+0000","last_unstale":"2024-07-17T08:49:27.365500+0000","last_undegraded":"2024-07-17T08:49:27.365500+0000","last_fullsized":"2024-07-17T08:49:27.365500+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:27.365323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:27.365323+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.365323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T16:21:19.009913+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"52.35","version":"0'0","reported_seq":22,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:29.403747+0000","last_change":"2024-07-17T08:49:29.403747+0000","last_active":"2024-07-17T08:49:29.403747+0000","last_peered":"2024-07-17T08:49:29.403747+0000","last_clean":"2024-07-17T08:49:29.403747+0000","last_became_active":"2024-07-17T08:47:06.144067+0000","last_became_peered":"2024-07-17T08:47:06.144067+0000","last_unstale":"2024-07-17T08:49:29.403747+0000","last_undegraded":"2024-07-17T08:49:29.403747+0000","last_fullsized":"2024-07-17T08:49:29.403747+0000","mapping_epoch":377,"log_start":"0'0","ondisk_log_start":"0'0","created":377,"last_epoch_clean":378,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-17T08:49:29.403392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-17T08:49:29.403392+0000","last_clean_scrub_stamp":"2024-07-17T08:49:29.403392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T10:12:30.450749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"379'4682","reported_seq":6224,"reported_epoch":379,"state":"active+clean","last_fresh":"2024-07-17T08:49:27.872261+0000","last_change":"2024-07-17T08:49:27.872261+0000","last_active":"2024-07-17T08:49:27.872261+0000","last_peered":"2024-07-17T08:49:27.872261+0000","last_clean":"2024-07-17T08:49:27.872261+0000","last_became_active":"2024-07-17T06:54:43.866510+0000","last_became_peered":"2024-07-17T06:54:43.866510+0000","last_unstale":"2024-07-17T08:49:27.872261+0000","last_undegraded":"2024-07-17T08:49:27.872261+0000","last_fullsized":"2024-07-17T08:49:27.872261+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'4682","last_scrub_stamp":"2024-07-17T08:49:27.872017+0000","last_deep_scrub":"379'4682","last_deep_scrub_stamp":"2024-07-17T08:49:27.872017+0000","last_clean_scrub_stamp":"2024-07-17T08:49:27.872017+0000","objects_scrubbed":2,"log_size":4682,"log_dups_size":0,"ondisk_log_size":4682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-18T15:12:06.478036+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":52,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":3797,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2569,"num_read_kb":7318,"num_write":589,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4682,"ondisk_log_size":4682,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543503,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80892,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290948,"statfs":{"total":96636764160,"available":96553930752,"internally_reserved":0,"allocated":5427200,"data_stored":3261377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424510798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53936,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317904,"statfs":{"total":96636764160,"available":96581533696,"internally_reserved":0,"allocated":4825088,"data_stored":2670393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424510799,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76152,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295688,"statfs":{"total":96636764160,"available":96558784512,"internally_reserved":0,"allocated":4833280,"data_stored":2671045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129543504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58100,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313740,"statfs":{"total":96636764160,"available":96577269760,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424510798,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80324,"kb_used_data":4732,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291516,"statfs":{"total":96636764160,"available":96554512384,"internally_reserved":0,"allocated":4845568,"data_stored":2673484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129543502,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80308,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291532,"statfs":{"total":96636764160,"available":96554528768,"internally_reserved":0,"allocated":4829184,"data_stored":2670483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510799,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58112,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313728,"statfs":{"total":96636764160,"available":96577257472,"internally_reserved":0,"allocated":4841472,"data_stored":2673414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424510798,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":5423104,"data_stored":3260795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":52,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-17T08:49:47.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-07-17T08:49:48.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 -- 172.21.15.44:0/1000566877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 msgr2=0x7f24000ac7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 --2- 172.21.15.44:0/1000566877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000ac7f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f23f8007350 tx=0x7f23f802b1a0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 -- 172.21.15.44:0/1000566877 shutdown_connections 2024-07-17T08:49:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 --2- 172.21.15.44:0/1000566877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000ac7f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 --2- 172.21.15.44:0/1000566877 >> v2:172.21.15.175:3300/0 conn(0x7f240006eb90 0x7f240015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.180+0000 7f2408000640 1 -- 172.21.15.44:0/1000566877 >> 172.21.15.44:0/1000566877 conn(0x7f24000b0eb0 msgr2=0x7f24000b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.181+0000 7f2408000640 1 -- 172.21.15.44:0/1000566877 shutdown_connections 2024-07-17T08:49:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.181+0000 7f2408000640 1 -- 172.21.15.44:0/1000566877 wait complete. 2024-07-17T08:49:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.182+0000 7f2408000640 1 Processor -- start 2024-07-17T08:49:48.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.183+0000 7f2408000640 1 -- start start 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 0x7f24000b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f240006f850 con 0x7f240006eb90 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f240006f6d0 con 0x7f24000b3280 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2408000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f240014fd60 con 0x7f24000ac3a0 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2405600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2405600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59174/0 (socket says 172.21.15.44:59174) 2024-07-17T08:49:48.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2405600640 1 -- 172.21.15.44:0/959217382 learned_addr learned my addr 172.21.15.44:0/959217382 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:48.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2406000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 0x7f24000b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.184+0000 7f2404c00640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 -- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 msgr2=0x7f24000b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 -- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 msgr2=0x7f24000b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 0x7f24000b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 -- 172.21.15.44:0/959217382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2400155fc0 con 0x7f240006eb90 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2406000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 0x7f24000b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2405600640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9c0 tx=0x7f23f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400ccc0 con 0x7f240006eb90 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f4005020 con 0x7f240006eb90 2024-07-17T08:49:48.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400ce60 con 0x7f240006eb90 2024-07-17T08:49:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24001562a0 con 0x7f240006eb90 2024-07-17T08:49:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.185+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2400156b40 con 0x7f240006eb90 2024-07-17T08:49:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.187+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240015a5a0 con 0x7f240006eb90 2024-07-17T08:49:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.188+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7f23f4005450 con 0x7f240006eb90 2024-07-17T08:49:48.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.189+0000 7f23fd600640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 0x7f23e4079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.189+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7f23f4099370 con 0x7f240006eb90 2024-07-17T08:49:48.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.190+0000 7f2404c00640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 0x7f23e4079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.191+0000 7f2404c00640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 0x7f23e4079190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24000cee90 tx=0x7f23f0009040 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:48.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.193+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7f23f4062180 con 0x7f240006eb90 2024-07-17T08:49:48.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.406+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2400156470 con 0x7f240006eb90 2024-07-17T08:49:48.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.414+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v113)=0 v113) ==== 123+0+0 (secure 0 0 0) 0x7f23f406a3c0 con 0x7f240006eb90 2024-07-17T08:49:48.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.415+0000 7f23fd600640 1 -- 172.21.15.44:0/959217382 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f23f4061590 con 0x7f240006eb90 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 msgr2=0x7f23e4079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 0x7f23e4079190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24000cee90 tx=0x7f23f0009040 comp rx=0 tx=0).stop 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 msgr2=0x7f24001f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 secure :-1 s=READY pgs=3547 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9c0 tx=0x7f23f400ee80 comp rx=0 tx=0).stop 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 shutdown_connections 2024-07-17T08:49:48.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7f23e4076cd0 0x7f23e4079190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24000b3280 0x7f24000b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24000ac3a0 0x7f24000b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 --2- 172.21.15.44:0/959217382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f240006eb90 0x7f24001f14b0 unknown :-1 s=CLOSED pgs=3547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.418+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 >> 172.21.15.44:0/959217382 conn(0x7f24000b0eb0 msgr2=0x7f24000b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.419+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 shutdown_connections 2024-07-17T08:49:48.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.419+0000 7f2408000640 1 -- 172.21.15.44:0/959217382 wait complete. 2024-07-17T08:49:48.438 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-17T08:49:48.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-17T08:49:48.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.649+0000 7fd187e00640 1 -- 172.21.15.44:0/3588269364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 msgr2=0x7fd16c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.649+0000 7fd187e00640 1 --2- 172.21.15.44:0/3588269364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd16c00b9c0 secure :-1 s=READY pgs=3549 cs=0 l=1 rev1=1 crypto rx=0x7fd1800016c0 tx=0x7fd16c005830 comp rx=0 tx=0).stop 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.650+0000 7fd187e00640 1 -- 172.21.15.44:0/3588269364 shutdown_connections 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.650+0000 7fd187e00640 1 --2- 172.21.15.44:0/3588269364 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd16c00b9c0 unknown :-1 s=CLOSED pgs=3549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.650+0000 7fd187e00640 1 -- 172.21.15.44:0/3588269364 >> 172.21.15.44:0/3588269364 conn(0x7fd1880b0e80 msgr2=0x7fd1880b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.650+0000 7fd187e00640 1 -- 172.21.15.44:0/3588269364 shutdown_connections 2024-07-17T08:49:48.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.650+0000 7fd187e00640 1 -- 172.21.15.44:0/3588269364 wait complete. 2024-07-17T08:49:48.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.651+0000 7fd187e00640 1 Processor -- start 2024-07-17T08:49:48.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 -- start start 2024-07-17T08:49:48.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1880cb930 0x7fd188073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1880afd80 0x7fd1880b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd18809bb20 con 0x7fd16c005230 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd18809b9a0 con 0x7fd1880cb930 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.652+0000 7fd187e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd18809bca0 con 0x7fd1880afd80 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59192/0 (socket says 172.21.15.44:59192) 2024-07-17T08:49:48.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 -- 172.21.15.44:0/3523958393 learned_addr learned my addr 172.21.15.44:0/3523958393 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186000640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1880cb930 0x7fd188073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd187400640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1880afd80 0x7fd1880b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 -- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1880afd80 msgr2=0x7fd1880b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1880afd80 0x7fd1880b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 -- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1880cb930 msgr2=0x7fd188073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1880cb930 0x7fd188073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 -- 172.21.15.44:0/3523958393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd188154750 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.653+0000 7fd186a00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c790 tx=0x7fd17800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.654+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178017070 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.654+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fd178005290 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.654+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800a8a0 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.654+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd188154a30 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.654+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd188154ff0 con 0x7fd16c005230 2024-07-17T08:49:48.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.655+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1880cf620 con 0x7fd16c005230 2024-07-17T08:49:48.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.656+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 230) ==== 98931+0+0 (secure 0 0 0) 0x7fd17801a070 con 0x7fd16c005230 2024-07-17T08:49:48.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.657+0000 7fd17ea00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 0x7fd164079190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-17T08:49:48.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.657+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(379..379 src has 1..379) ==== 6733+0+0 (secure 0 0 0) 0x7fd178097db0 con 0x7fd16c005230 2024-07-17T08:49:48.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.658+0000 7fd186000640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 0x7fd164079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-17T08:49:48.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.659+0000 7fd186000640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 0x7fd164079190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1881546e0 tx=0x7fd170009500 comp rx=0 tx=0).ready entity=mgr.10343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-17T08:49:48.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.662+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192305 (secure 0 0 0) 0x7fd17809c070 con 0x7fd16c005230 2024-07-17T08:49:48.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.915+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1880ae070 con 0x7fd16c005230 2024-07-17T08:49:48.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.916+0000 7fd17ea00640 1 -- 172.21.15.44:0/3523958393 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+5247 (secure 0 0 0) 0x7fd178060c50 con 0x7fd16c005230 2024-07-17T08:49:48.917 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-07-17T08:49:48.918 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":218,"btime":"2024-07-17T08:47:09:148469+0000","default_fscid":25,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/4026613862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6833","nonce":4026613862},{"type":"v1","addr":"172.21.15.44:6835","nonce":4026613862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":10475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1040890779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":1040890779},{"type":"v1","addr":"172.21.15.175:6836","nonce":1040890779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":213},{"gid":14859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6834/3392644589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3392644589},{"type":"v1","addr":"172.21.15.44:6834","nonce":3392644589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-17T08:47:05.578374+0000","modified":"2024-07-17T08:47:08.644730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14868":{"gid":14868,"name":"b","rank":0,"incarnation":214,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6801/1064157662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1064157662},{"type":"v1","addr":"172.21.15.175:6801","nonce":1064157662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[52],"metadata_pool":51,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":14868,"qdb_cluster":[14868]},"id":25}]} 2024-07-17T08:49:48.918 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 218 2024-07-17T08:49:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 msgr2=0x7fd164079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 0x7fd164079190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd1881546e0 tx=0x7fd170009500 comp rx=0 tx=0).stop 2024-07-17T08:49:48.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 msgr2=0x7fd18814e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c790 tx=0x7fd17800cc50 comp rx=0 tx=0).stop 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 shutdown_connections 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:6834/294464184,v1:172.21.15.175:6837/294464184] conn(0x7fd164076cd0 0x7fd164079190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1880afd80 0x7fd1880b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1880cb930 0x7fd188073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 --2- 172.21.15.44:0/3523958393 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd16c005230 0x7fd18814e390 unknown :-1 s=CLOSED pgs=3550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-17T08:49:48.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 >> 172.21.15.44:0/3523958393 conn(0x7fd1880b0e80 msgr2=0x7fd18815cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-17T08:49:48.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 shutdown_connections 2024-07-17T08:49:48.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-07-17T08:49:48.919+0000 7fd187e00640 1 -- 172.21.15.44:0/3523958393 wait complete. 2024-07-17T08:49:48.938 DEBUG:tasks.cephfs.filesystem:already dead... 2024-07-17T08:49:48.938 INFO:teuthology.misc:Shutting down mds daemons... 2024-07-17T08:49:48.938 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-17T08:49:48.938 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:48.958 INFO:tasks.ceph.mds.a:Stopped 2024-07-17T08:49:48.958 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-17T08:49:48.959 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:49.024 INFO:tasks.ceph.mds.c:Stopped 2024-07-17T08:49:49.024 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-17T08:49:49.024 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:49.062 INFO:tasks.ceph.mds.b:Stopped 2024-07-17T08:49:49.063 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-17T08:49:49.063 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:49.125 INFO:tasks.ceph.mds.d:Stopped 2024-07-17T08:49:49.125 INFO:teuthology.misc:Shutting down osd daemons... 2024-07-17T08:49:49.126 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-07-17T08:49:49.126 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:49.219 INFO:tasks.ceph.osd.0:Stopped 2024-07-17T08:49:49.219 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-07-17T08:49:49.219 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:50.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-07-17T08:49:55.321 INFO:tasks.ceph.osd.1:Stopped 2024-07-17T08:49:55.321 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-07-17T08:49:55.322 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.388 INFO:tasks.ceph.osd.2:Stopped 2024-07-17T08:49:55.388 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-07-17T08:49:55.388 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.469 INFO:tasks.ceph.osd.3:Stopped 2024-07-17T08:49:55.469 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-07-17T08:49:55.469 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.541 INFO:tasks.ceph.osd.4:Stopped 2024-07-17T08:49:55.541 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-07-17T08:49:55.541 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.621 INFO:tasks.ceph.osd.5:Stopped 2024-07-17T08:49:55.622 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-07-17T08:49:55.622 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.681 INFO:tasks.ceph.osd.6:Stopped 2024-07-17T08:49:55.681 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-07-17T08:49:55.681 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:55.748 INFO:tasks.ceph.osd.7:Stopped 2024-07-17T08:49:55.748 INFO:teuthology.misc:Shutting down mgr daemons... 2024-07-17T08:49:55.749 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-07-17T08:49:55.749 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:49:56.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-07-17T08:49:56.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-07-17T08:50:01.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2024-07-17T08:50:01.851 INFO:tasks.ceph.mgr.y:Stopped 2024-07-17T08:50:01.851 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-07-17T08:50:01.851 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:50:01.936 INFO:tasks.ceph.mgr.x:Stopped 2024-07-17T08:50:01.936 INFO:teuthology.misc:Shutting down mon daemons... 2024-07-17T08:50:01.936 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-07-17T08:50:01.936 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:50:02.007 INFO:tasks.ceph.mon.a:Stopped 2024-07-17T08:50:02.008 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-07-17T08:50:02.008 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:50:02.074 INFO:tasks.ceph.mon.b:Stopped 2024-07-17T08:50:02.075 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-07-17T08:50:02.075 INFO:teuthology.orchestra.run:waiting for 300 2024-07-17T08:50:02.136 INFO:tasks.ceph.mon.c:Stopped 2024-07-17T08:50:02.136 INFO:tasks.ceph:Checking cluster log for badness... 2024-07-17T08:50:02.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-07-17T08:50:02.250 INFO:teuthology.orchestra.run.smithi044.stdout:2024-07-17T08:19:41.822445+0000 mds.a (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi175:x (7112), after 302.841 seconds 2024-07-17T08:50:02.251 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-07-17T08:50:02.251 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:50:02.251 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-07-17T08:50:02.451 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:50:02.451 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-07-17T08:50:02.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:50:02.615 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-07-17T08:50:02.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:50:02.895 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-07-17T08:50:03.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T08:50:03.069 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-07-17T08:50:03.395 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T08:50:03.396 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-07-17T08:50:03.532 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T08:50:03.533 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-07-17T08:50:03.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T08:50:03.665 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-07-17T08:50:03.844 INFO:tasks.ceph:Archiving mon data... 2024-07-17T08:50:03.845 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/data/mon.a.tgz 2024-07-17T08:50:03.846 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-07-17T08:50:03.906 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.L6u0WvCGQd 2024-07-17T08:50:03.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.L6u0WvCGQd 2024-07-17T08:50:04.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.L6u0WvCGQd 2024-07-17T08:50:04.971 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.L6u0WvCGQd is 5MB 2024-07-17T08:50:05.176 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.L6u0WvCGQd 2024-07-17T08:50:05.235 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/data/mon.b.tgz 2024-07-17T08:50:05.235 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-07-17T08:50:05.291 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.CrRZMBHJVB 2024-07-17T08:50:05.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.CrRZMBHJVB 2024-07-17T08:50:06.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.CrRZMBHJVB 2024-07-17T08:50:06.356 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.CrRZMBHJVB is 5MB 2024-07-17T08:50:06.547 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.CrRZMBHJVB 2024-07-17T08:50:06.600 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/data/mon.c.tgz 2024-07-17T08:50:06.601 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-07-17T08:50:06.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-07-17T08:50:06.647 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.rBCwXiHlzi 2024-07-17T08:50:06.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rBCwXiHlzi 2024-07-17T08:50:07.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.rBCwXiHlzi 2024-07-17T08:50:07.691 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.rBCwXiHlzi is 5MB 2024-07-17T08:50:07.886 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.rBCwXiHlzi 2024-07-17T08:50:07.944 INFO:tasks.ceph:Cleaning ceph cluster... 2024-07-17T08:50:07.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-07-17T08:50:07.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-07-17T08:50:08.089 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-07-17T08:50:08.144 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-07-17T08:50:08.144 INFO:tasks.ceph:Archiving crash dumps... 2024-07-17T08:50:08.146 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi044/crash 2024-07-17T08:50:08.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-07-17T08:50:08.235 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi175/crash 2024-07-17T08:50:08.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-07-17T08:50:08.319 INFO:tasks.ceph:Shutting down logrotate 2024-07-17T08:50:08.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:50:08.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-17T08:50:08.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-07-17T08:50:08.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-07-17T08:50:08.605 INFO:tasks.ceph:Compressing logs... 2024-07-17T08:50:08.605 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-17T08:50:08.613 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-17T08:50:08.687 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38161.log 2024-07-17T08:50:08.688 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-07-17T08:50:08.689 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43151.log 2024-07-17T08:50:08.689 INFO:teuthology.orchestra.run.smithi175.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38161.log.gz 2024-07-17T08:50:08.690 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-07-17T08:50:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.43151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43151.log.gz 2024-07-17T08:50:08.691 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43155.log 2024-07-17T08:50:08.692 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-07-17T08:50:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.43155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43155.log.gz 2024-07-17T08:50:08.693 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-07-17T08:50:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37672.log 2024-07-17T08:50:08.694 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-07-17T08:50:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117993.log 2024-07-17T08:50:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-07-17T08:50:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2024-07-17T08:50:08.696 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37672.log.gz 2024-07-17T08:50:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85347.log 2024-07-17T08:50:08.697 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.117993.log.gz 2024-07-17T08:50:08.698 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48163.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz -5 2024-07-17T08:50:08.698 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.109985.log 2024-07-17T08:50:08.698 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38040.log 2024-07-17T08:50:08.699 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-07-17T08:50:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111479.log 2024-07-17T08:50:08.700 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85347.log.gz 2024-07-17T08:50:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111966.log 2024-07-17T08:50:08.700 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109985.log.gz 2024-07-17T08:50:08.701 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38040.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38040.log.gz 2024-07-17T08:50:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111479.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111479.log.gz 2024-07-17T08:50:08.702 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51168.log 2024-07-17T08:50:08.703 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-07-17T08:50:08.703 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69392.log 2024-07-17T08:50:08.704 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111966.log.gz 2024-07-17T08:50:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61127.log 2024-07-17T08:50:08.704 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51168.log.gz 2024-07-17T08:50:08.704 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69392.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69392.log.gz 2024-07-17T08:50:08.705 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72334.log 2024-07-17T08:50:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-07-17T08:50:08.705 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38291.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-07-17T08:50:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106371.log 2024-07-17T08:50:08.706 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61127.log.gz 2024-07-17T08:50:08.706 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.72334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72334.log.gz 2024-07-17T08:50:08.707 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44751.log 2024-07-17T08:50:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42737.log 2024-07-17T08:50:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106371.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.106371.log.gz -5 2024-07-17T08:50:08.708 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80405.log 2024-07-17T08:50:08.708 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109481.log 2024-07-17T08:50:08.709 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-07-17T08:50:08.710 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52311.log 2024-07-17T08:50:08.710 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80405.log.gz 2024-07-17T08:50:08.710 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109481.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.109481.log.gz 2024-07-17T08:50:08.710 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86136.log 2024-07-17T08:50:08.711 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.c.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-07-17T08:50:08.711 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-07-17T08:50:08.712 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59980.log 2024-07-17T08:50:08.712 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-07-17T08:50:08.713 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86136.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.86136.log.gz --verbose 2024-07-17T08:50:08.713 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.103375.log 2024-07-17T08:50:08.713 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.42737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-07-17T08:50:08.714 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59980.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59980.log.gz 2024-07-17T08:50:08.714 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82859.log 2024-07-17T08:50:08.715 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.6.log: 93.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-07-17T08:50:08.715 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42737.log.gz 2024-07-17T08:50:08.716 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2024-07-17T08:50:08.716 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103375.log.gz 2024-07-17T08:50:08.716 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95705.log 2024-07-17T08:50:08.717 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82859.log.gz 2024-07-17T08:50:08.717 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50680.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-07-17T08:50:08.717 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104257.log 2024-07-17T08:50:08.718 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37962.log 2024-07-17T08:50:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38121.log 2024-07-17T08:50:08.718 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38080.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-07-17T08:50:08.720 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95705.log.gz 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96424.log 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.104257.log.gz 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37962.log.gz 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113307.log 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38000.log 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi175.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38121.log.gz 2024-07-17T08:50:08.721 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-07-17T08:50:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96424.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96424.log.gz 2024-07-17T08:50:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44457.log 2024-07-17T08:50:08.723 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104345.log 2024-07-17T08:50:08.723 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.113307.log.gz 2024-07-17T08:50:08.725 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.38000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42756.log 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44457.log.gz 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104345.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104345.log.gz 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108380.log 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113903.log 2024-07-17T08:50:08.726 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-07-17T08:50:08.729 INFO:teuthology.orchestra.run.smithi175.stderr: 86.5%/var/log/ceph/ceph-client.admin.38209.log: /var/log/ceph/ceph-client.1.42756.log: -- replaced with /var/log/ceph/ceph-client.admin.38000.log.gz 2024-07-17T08:50:08.729 INFO:teuthology.orchestra.run.smithi175.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-07-17T08:50:08.729 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42756.log.gz 2024-07-17T08:50:08.729 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108380.log.gz 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68972.log 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116548.log 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113903.log.gz 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48875.log 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68972.log.gz 2024-07-17T08:50:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116548.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116548.log.gz 2024-07-17T08:50:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2024-07-17T08:50:08.732 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82226.log 2024-07-17T08:50:08.732 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48875.log.gz 2024-07-17T08:50:08.732 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57041.log 2024-07-17T08:50:08.732 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz 2024-07-17T08:50:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-07-17T08:50:08.734 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38249.log 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82226.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82226.log.gz 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48743.log 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116013.log 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57041.log.gz 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106203.log 2024-07-17T08:50:08.735 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48743.log.gz 2024-07-17T08:50:08.736 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116013.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.116013.log.gz 2024-07-17T08:50:08.737 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62101.log 2024-07-17T08:50:08.738 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115930.log 2024-07-17T08:50:08.738 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106203.log.gz 2024-07-17T08:50:08.738 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90880.log 2024-07-17T08:50:08.739 INFO:teuthology.orchestra.run.smithi044.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.62101.log.gz 2024-07-17T08:50:08.739 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115930.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.115930.log.gz 2024-07-17T08:50:08.740 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97892.log 2024-07-17T08:50:08.740 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108632.log 2024-07-17T08:50:08.741 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90880.log.gz 2024-07-17T08:50:08.742 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.38249.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38249.log.gz 2024-07-17T08:50:08.742 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73716.log 2024-07-17T08:50:08.743 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.97892.log.gz 2024-07-17T08:50:08.743 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108632.log: gzip -5 --verbose 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108632.log.gz -- 2024-07-17T08:50:08.743 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.90670.log 2024-07-17T08:50:08.744 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73716.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73716.log.gz 2024-07-17T08:50:08.744 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118035.log 2024-07-17T08:50:08.746 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90670.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90670.log.gz 2024-07-17T08:50:08.746 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70634.log 2024-07-17T08:50:08.747 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63573.log 2024-07-17T08:50:08.747 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.118035.log.gz 2024-07-17T08:50:08.748 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79500.log 2024-07-17T08:50:08.748 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70634.log.gz 2024-07-17T08:50:08.749 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71785.log 2024-07-17T08:50:08.749 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63573.log.gz 2024-07-17T08:50:08.750 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79500.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79500.log.gz 2024-07-17T08:50:08.750 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2024-07-17T08:50:08.752 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98757.log 2024-07-17T08:50:08.752 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71785.log.gz 2024-07-17T08:50:08.753 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2024-07-17T08:50:08.753 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz 2024-07-17T08:50:08.754 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119248.log 2024-07-17T08:50:08.754 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98757.log.gz 2024-07-17T08:50:08.755 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66964.log 2024-07-17T08:50:08.756 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69272.log 2024-07-17T08:50:08.757 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119248.log.gz 2024-07-17T08:50:08.757 INFO:teuthology.orchestra.run.smithi044.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2024-07-17T08:50:08.757 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107768.log 2024-07-17T08:50:08.758 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66964.log.gz 2024-07-17T08:50:08.758 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.69272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69272.log.gz 2024-07-17T08:50:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64861.log 2024-07-17T08:50:08.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91263.log 2024-07-17T08:50:08.760 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107768.log.gz 2024-07-17T08:50:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71431.log 2024-07-17T08:50:08.761 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64861.log.gz 2024-07-17T08:50:08.762 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48457.log 2024-07-17T08:50:08.762 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.91263.log.gz 2024-07-17T08:50:08.763 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118982.log 2024-07-17T08:50:08.764 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71431.log.gz 2024-07-17T08:50:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101010.log 2024-07-17T08:50:08.765 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48457.log.gz 2024-07-17T08:50:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118982.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118982.log.gz 2024-07-17T08:50:08.765 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94898.log 2024-07-17T08:50:08.766 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49362.log 2024-07-17T08:50:08.767 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101010.log.gz 2024-07-17T08:50:08.768 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71560.log 2024-07-17T08:50:08.768 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94898.log.gz 2024-07-17T08:50:08.768 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49362.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2024-07-17T08:50:08.769 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112275.log 2024-07-17T08:50:08.770 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68762.log 2024-07-17T08:50:08.770 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71560.log.gz 2024-07-17T08:50:08.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-07-17T08:50:08.771 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112275.log.gz 2024-07-17T08:50:08.772 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68762.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68762.log.gz 2024-07-17T08:50:08.772 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109439.log 2024-07-17T08:50:08.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122297.log 2024-07-17T08:50:08.773 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-07-17T08:50:08.774 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109439.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.109439.log.gz 2024-07-17T08:50:08.774 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102899.log 2024-07-17T08:50:08.775 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-07-17T08:50:08.775 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.122297.log.gz 2024-07-17T08:50:08.776 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110975.log 2024-07-17T08:50:08.776 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102899.log.gz 2024-07-17T08:50:08.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53088.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-07-17T08:50:08.777 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80661.log 2024-07-17T08:50:08.778 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110975.log.gz 2024-07-17T08:50:08.779 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48205.log 2024-07-17T08:50:08.780 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118714.log 2024-07-17T08:50:08.780 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80661.log.gz 2024-07-17T08:50:08.781 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104484.log 2024-07-17T08:50:08.781 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48205.log.gz 2024-07-17T08:50:08.782 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118714.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118714.log.gz 2024-07-17T08:50:08.782 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98923.log 2024-07-17T08:50:08.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86560.log 2024-07-17T08:50:08.783 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104484.log.gz 2024-07-17T08:50:08.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96340.log 2024-07-17T08:50:08.784 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98923.log.gz 2024-07-17T08:50:08.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46155.log 2024-07-17T08:50:08.785 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86560.log.gz 2024-07-17T08:50:08.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96340.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96340.log.gz 2024-07-17T08:50:08.786 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49236.log 2024-07-17T08:50:08.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44937.log 2024-07-17T08:50:08.788 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46155.log.gz 2024-07-17T08:50:08.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75677.log 2024-07-17T08:50:08.789 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gz 2024-07-17T08:50:08.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44937.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44937.log.gz 2024-07-17T08:50:08.790 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52783.log 2024-07-17T08:50:08.792 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75677.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75677.log.gz 2024-07-17T08:50:08.792 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74503.log 2024-07-17T08:50:08.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114640.log 2024-07-17T08:50:08.793 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52783.log.gz 2024-07-17T08:50:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74503.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74503.log.gz 2024-07-17T08:50:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111437.log 2024-07-17T08:50:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114640.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.114640.log.gz --verbose 2024-07-17T08:50:08.795 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.74408.log 2024-07-17T08:50:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111437.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111437.log.gz 2024-07-17T08:50:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83256.log 2024-07-17T08:50:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92786.log 2024-07-17T08:50:08.797 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74408.log.gz 2024-07-17T08:50:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83256.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83256.log.gz 2024-07-17T08:50:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116734.log 2024-07-17T08:50:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92786.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92786.log.gz 2024-07-17T08:50:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103646.log 2024-07-17T08:50:08.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113265.log 2024-07-17T08:50:08.800 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116734.log.gz 2024-07-17T08:50:08.801 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103646.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103646.log.gz 2024-07-17T08:50:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2024-07-17T08:50:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113265.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.113265.log.gz 2024-07-17T08:50:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121793.log 2024-07-17T08:50:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-07-17T08:50:08.804 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2024-07-17T08:50:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121793.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.121793.log.gz 2024-07-17T08:50:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120578.log 2024-07-17T08:50:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44321.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-07-17T08:50:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98677.log 2024-07-17T08:50:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120578.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120578.log.gz 2024-07-17T08:50:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104886.log 2024-07-17T08:50:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98677.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98677.log.gz 2024-07-17T08:50:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105806.log 2024-07-17T08:50:08.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63093.log 2024-07-17T08:50:08.810 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104886.log.gz 2024-07-17T08:50:08.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105806.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105806.log.gz 2024-07-17T08:50:08.811 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52488.log 2024-07-17T08:50:08.811 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63093.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63093.log.gz 2024-07-17T08:50:08.812 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120282.log 2024-07-17T08:50:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107053.log 2024-07-17T08:50:08.813 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52488.log.gz 2024-07-17T08:50:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120282.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120282.log.gz 2024-07-17T08:50:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99011.log 2024-07-17T08:50:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107053.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107053.log.gz 2024-07-17T08:50:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91307.log 2024-07-17T08:50:08.816 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99011.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99011.log.gz 2024-07-17T08:50:08.816 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94357.log 2024-07-17T08:50:08.817 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91307.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91307.log.gz 2024-07-17T08:50:08.817 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105156.log 2024-07-17T08:50:08.818 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94357.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94357.log.gz 2024-07-17T08:50:08.818 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43467.log 2024-07-17T08:50:08.819 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105156.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.105156.log.gz 2024-07-17T08:50:08.819 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53614.log 2024-07-17T08:50:08.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105359.log 2024-07-17T08:50:08.821 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43467.log.gz 2024-07-17T08:50:08.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53614.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53614.log.gz 2024-07-17T08:50:08.824 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63703.log 2024-07-17T08:50:08.825 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105359.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105359.log.gz 2024-07-17T08:50:08.825 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91177.log 2024-07-17T08:50:08.826 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63703.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63703.log.gz 2024-07-17T08:50:08.826 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112233.log 2024-07-17T08:50:08.826 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59938.log 2024-07-17T08:50:08.826 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91177.log.gz 2024-07-17T08:50:08.827 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112233.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112233.log.gz 2024-07-17T08:50:08.827 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104396.log 2024-07-17T08:50:08.827 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59938.log.gz 2024-07-17T08:50:08.827 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52665.log 2024-07-17T08:50:08.828 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72416.log 2024-07-17T08:50:08.829 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104396.log.gz 2024-07-17T08:50:08.829 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52665.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52665.log.gz 2024-07-17T08:50:08.830 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115793.log 2024-07-17T08:50:08.830 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72416.log.gz 2024-07-17T08:50:08.831 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75089.log 2024-07-17T08:50:08.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81213.log 2024-07-17T08:50:08.832 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115793.log.gz 2024-07-17T08:50:08.833 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75089.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75089.log.gz 2024-07-17T08:50:08.833 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82184.log 2024-07-17T08:50:08.834 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.81213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81213.log.gz 2024-07-17T08:50:08.834 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49572.log 2024-07-17T08:50:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82184.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82184.log.gz 2024-07-17T08:50:08.835 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60605.log 2024-07-17T08:50:08.836 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49572.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49572.log.gz 2024-07-17T08:50:08.836 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66078.log 2024-07-17T08:50:08.836 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60605.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.60605.log.gz 2024-07-17T08:50:08.837 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48121.log 2024-07-17T08:50:08.838 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-07-17T08:50:08.838 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66078.log.gz 2024-07-17T08:50:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48121.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48121.log.gz 2024-07-17T08:50:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104694.log 2024-07-17T08:50:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45662.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-07-17T08:50:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99234.log 2024-07-17T08:50:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97096.log 2024-07-17T08:50:08.842 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104694.log.gz 2024-07-17T08:50:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99234.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99234.log.gz 2024-07-17T08:50:08.843 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93020.log 2024-07-17T08:50:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97096.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97096.log.gz 2024-07-17T08:50:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43083.log 2024-07-17T08:50:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93020.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.93020.log.gz 2024-07-17T08:50:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59561.log 2024-07-17T08:50:08.846 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102762.log 2024-07-17T08:50:08.846 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43083.log.gz 2024-07-17T08:50:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59561.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59561.log.gz 2024-07-17T08:50:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50562.log 2024-07-17T08:50:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102762.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.102762.log.gz 2024-07-17T08:50:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79710.log 2024-07-17T08:50:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85781.log 2024-07-17T08:50:08.849 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50562.log.gz 2024-07-17T08:50:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121288.log 2024-07-17T08:50:08.850 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79710.log.gz 2024-07-17T08:50:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90712.log 2024-07-17T08:50:08.851 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85781.log.gz 2024-07-17T08:50:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121288.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121288.log.gz 2024-07-17T08:50:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107413.log 2024-07-17T08:50:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90712.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90712.log.gz 2024-07-17T08:50:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108114.log 2024-07-17T08:50:08.854 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117131.log 2024-07-17T08:50:08.854 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107413.log.gz 2024-07-17T08:50:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108114.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108114.log.gz 2024-07-17T08:50:08.855 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65071.log 2024-07-17T08:50:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117131.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117131.log.gz 2024-07-17T08:50:08.856 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65851.log 2024-07-17T08:50:08.857 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62924.log 2024-07-17T08:50:08.858 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.65071.log.gz 2024-07-17T08:50:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65851.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65851.log.gz 2024-07-17T08:50:08.859 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75047.log 2024-07-17T08:50:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62924.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62924.log.gz 2024-07-17T08:50:08.860 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92828.log 2024-07-17T08:50:08.861 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56389.log 2024-07-17T08:50:08.861 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75047.log.gz 2024-07-17T08:50:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92828.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92828.log.gz 2024-07-17T08:50:08.862 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112739.log 2024-07-17T08:50:08.863 INFO:teuthology.orchestra.run.smithi175.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-07-17T08:50:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56389.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56389.log.gz 2024-07-17T08:50:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73842.log 2024-07-17T08:50:08.864 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.112739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2024-07-17T08:50:08.865 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112739.log.gz 2024-07-17T08:50:08.865 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116055.log 2024-07-17T08:50:08.866 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73842.log.gz 2024-07-17T08:50:08.867 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49614.log: gzip -5 --verbose 87.0% -- -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz /var/log/ceph/ceph-client.admin.118521.log 2024-07-17T08:50:08.867 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:08.868 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94772.log 2024-07-17T08:50:08.868 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.116055.log.gz 2024-07-17T08:50:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118521.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118521.log.gz 2024-07-17T08:50:08.869 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-07-17T08:50:08.870 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83508.log 2024-07-17T08:50:08.871 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94772.log.gz 2024-07-17T08:50:08.872 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75005.log 2024-07-17T08:50:08.873 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48002.log 2024-07-17T08:50:08.873 INFO:teuthology.orchestra.run.smithi044.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83508.log.gz 2024-07-17T08:50:08.874 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103333.log 2024-07-17T08:50:08.875 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75005.log.gz 2024-07-17T08:50:08.875 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48002.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48002.log.gz 2024-07-17T08:50:08.876 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117783.log 2024-07-17T08:50:08.877 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103333.log: gzip -5 86.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.103333.log.gz 2024-07-17T08:50:08.877 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.119645.log 2024-07-17T08:50:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117783.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117783.log.gz 2024-07-17T08:50:08.878 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74793.log 2024-07-17T08:50:08.880 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100042.log 2024-07-17T08:50:08.880 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119645.log.gz 2024-07-17T08:50:08.881 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74793.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74793.log.gz 2024-07-17T08:50:08.881 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114924.log 2024-07-17T08:50:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100042.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100042.log.gz 2024-07-17T08:50:08.882 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44379.log 2024-07-17T08:50:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114924.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114924.log.gz 2024-07-17T08:50:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75365.log 2024-07-17T08:50:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44379.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44379.log.gz 2024-07-17T08:50:08.884 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81548.log 2024-07-17T08:50:08.885 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75365.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75365.log.gz 2024-07-17T08:50:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100511.log 2024-07-17T08:50:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81548.log: 88.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-07-17T08:50:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43299.log 2024-07-17T08:50:08.887 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81548.log.gz 2024-07-17T08:50:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100511.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.100511.log.gz 2024-07-17T08:50:08.888 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66852.log 2024-07-17T08:50:08.889 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107221.log 2024-07-17T08:50:08.889 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43299.log.gz 2024-07-17T08:50:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66852.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66852.log.gz 2024-07-17T08:50:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67611.log 2024-07-17T08:50:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71073.log 2024-07-17T08:50:08.891 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107221.log.gz 2024-07-17T08:50:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117909.log 2024-07-17T08:50:08.892 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67611.log.gz 2024-07-17T08:50:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90628.log 2024-07-17T08:50:08.893 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71073.log.gz 2024-07-17T08:50:08.894 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60335.log 2024-07-17T08:50:08.894 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.117909.log.gz 2024-07-17T08:50:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90628.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90628.log.gz 2024-07-17T08:50:08.895 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94146.log 2024-07-17T08:50:08.896 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102683.log 2024-07-17T08:50:08.896 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60335.log.gz 2024-07-17T08:50:08.897 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94146.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94146.log.gz 2024-07-17T08:50:08.897 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79542.log 2024-07-17T08:50:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120326.log 2024-07-17T08:50:08.898 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.102683.log.gz 2024-07-17T08:50:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103688.log 2024-07-17T08:50:08.899 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79542.log.gz 2024-07-17T08:50:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115054.log 2024-07-17T08:50:08.900 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120326.log.gz 2024-07-17T08:50:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70905.log 2024-07-17T08:50:08.901 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103688.log.gz 2024-07-17T08:50:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100469.log 2024-07-17T08:50:08.902 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115054.log.gz 2024-07-17T08:50:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70905.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.70905.log.gz -- 2024-07-17T08:50:08.903 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.45859.log 2024-07-17T08:50:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85823.log 2024-07-17T08:50:08.904 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100469.log.gz 2024-07-17T08:50:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45859.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-07-17T08:50:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64251.log 2024-07-17T08:50:08.906 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93492.log 2024-07-17T08:50:08.906 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85823.log.gz 2024-07-17T08:50:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.64251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64251.log.gz 2024-07-17T08:50:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92912.log 2024-07-17T08:50:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37863.log 2024-07-17T08:50:08.908 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93492.log.gz 2024-07-17T08:50:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87604.log 2024-07-17T08:50:08.909 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92912.log.gz 2024-07-17T08:50:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37863.log.gz 2024-07-17T08:50:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45374.log 2024-07-17T08:50:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72872.log 2024-07-17T08:50:08.911 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87604.log.gz 2024-07-17T08:50:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108422.log 2024-07-17T08:50:08.911 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45374.log.gz 2024-07-17T08:50:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98837.log 2024-07-17T08:50:08.912 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.72872.log.gz 2024-07-17T08:50:08.913 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67176.log 2024-07-17T08:50:08.913 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108422.log.gz 2024-07-17T08:50:08.914 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47816.log 2024-07-17T08:50:08.914 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98837.log.gz 2024-07-17T08:50:08.915 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70361.log 2024-07-17T08:50:08.915 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67176.log.gz 2024-07-17T08:50:08.916 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81690.log 2024-07-17T08:50:08.916 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47816.log.gz 2024-07-17T08:50:08.917 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98582.log 2024-07-17T08:50:08.917 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70361.log.gz 2024-07-17T08:50:08.918 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81690.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.119290.log -- replaced with /var/log/ceph/ceph-client.admin.81690.log.gz 2024-07-17T08:50:08.918 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98582.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98582.log.gz 2024-07-17T08:50:08.919 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94273.log 2024-07-17T08:50:08.920 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121709.log 2024-07-17T08:50:08.920 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119290.log.gz 2024-07-17T08:50:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94273.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-07-17T08:50:08.921 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94273.log.gz 2024-07-17T08:50:08.921 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106329.log 2024-07-17T08:50:08.922 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121709.log.gz 2024-07-17T08:50:08.922 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64375.log 2024-07-17T08:50:08.923 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112359.log 2024-07-17T08:50:08.923 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106329.log.gz 2024-07-17T08:50:08.924 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-07-17T08:50:08.924 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64375.log.gz 2024-07-17T08:50:08.925 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96676.log 2024-07-17T08:50:08.926 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.112359.log.gz 2024-07-17T08:50:08.926 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47453.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-07-17T08:50:08.927 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102342.log 2024-07-17T08:50:08.928 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115560.log 2024-07-17T08:50:08.928 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96676.log.gz 2024-07-17T08:50:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.102342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102342.log.gz 2024-07-17T08:50:08.929 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119603.log 2024-07-17T08:50:08.930 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.115560.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.115560.log.gz --verbose 2024-07-17T08:50:08.930 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.79794.log 2024-07-17T08:50:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119603.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119603.log.gz 2024-07-17T08:50:08.931 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86890.log 2024-07-17T08:50:08.932 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61341.log 2024-07-17T08:50:08.933 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79794.log.gz 2024-07-17T08:50:08.933 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86890.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86890.log.gz 2024-07-17T08:50:08.933 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68050.log 2024-07-17T08:50:08.934 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82142.log 2024-07-17T08:50:08.934 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61341.log.gz 2024-07-17T08:50:08.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72754.log 2024-07-17T08:50:08.935 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68050.log.gz 2024-07-17T08:50:08.936 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106884.log 2024-07-17T08:50:08.937 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82142.log.gz 2024-07-17T08:50:08.937 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72754.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.72754.log.gz 2024-07-17T08:50:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112443.log 2024-07-17T08:50:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118413.log 2024-07-17T08:50:08.938 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106884.log.gz 2024-07-17T08:50:08.939 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106969.log 2024-07-17T08:50:08.939 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112443.log.gz 2024-07-17T08:50:08.940 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119729.log 87.5% 2024-07-17T08:50:08.940 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118413.log.gz 2024-07-17T08:50:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106969.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106969.log.gz 2024-07-17T08:50:08.941 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71911.log 2024-07-17T08:50:08.942 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119729.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.119729.log.gz /var/log/ceph/ceph-client.admin.95115.log 2024-07-17T08:50:08.943 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:08.943 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93534.log 2024-07-17T08:50:08.944 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71911.log.gz 2024-07-17T08:50:08.944 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95115.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95115.log.gz 2024-07-17T08:50:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63478.log 2024-07-17T08:50:08.945 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74323.log 2024-07-17T08:50:08.946 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.93534.log.gz 2024-07-17T08:50:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-07-17T08:50:08.947 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63478.log.gz 2024-07-17T08:50:08.947 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67013.log 2024-07-17T08:50:08.947 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74323.log.gz 2024-07-17T08:50:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120788.log 2024-07-17T08:50:08.948 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-07-17T08:50:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108506.log 2024-07-17T08:50:08.950 INFO:teuthology.orchestra.run.smithi044.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.67013.log.gz 2024-07-17T08:50:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-07-17T08:50:08.951 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.120788.log.gz 2024-07-17T08:50:08.952 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108506.log.gz 2024-07-17T08:50:08.952 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83424.log 2024-07-17T08:50:08.953 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63051.log 2024-07-17T08:50:08.953 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-07-17T08:50:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83424.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83424.log.gz 2024-07-17T08:50:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80703.log 2024-07-17T08:50:08.954 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72338.log 2024-07-17T08:50:08.955 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63051.log.gz 2024-07-17T08:50:08.955 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94086.log 2024-07-17T08:50:08.956 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80703.log.gz 2024-07-17T08:50:08.956 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.72338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72338.log.gz 2024-07-17T08:50:08.957 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57083.log 2024-07-17T08:50:08.958 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91743.log 2024-07-17T08:50:08.958 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.94086.log.gz 2024-07-17T08:50:08.958 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99822.log 2024-07-17T08:50:08.959 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57083.log.gz 2024-07-17T08:50:08.960 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98312.log 2024-07-17T08:50:08.960 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91743.log.gz 2024-07-17T08:50:08.960 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99822.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99822.log.gz 2024-07-17T08:50:08.961 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108926.log 2024-07-17T08:50:08.961 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98312.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.106550.log 2024-07-17T08:50:08.961 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98312.log.gz 2024-07-17T08:50:08.962 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93282.log 2024-07-17T08:50:08.963 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108926.log.gz 2024-07-17T08:50:08.963 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65131.log 2024-07-17T08:50:08.963 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106550.log.gz 2024-07-17T08:50:08.964 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2024-07-17T08:50:08.965 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.93282.log.gz 2024-07-17T08:50:08.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73113.log 2024-07-17T08:50:08.965 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65131.log.gz 2024-07-17T08:50:08.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44583.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2024-07-17T08:50:08.967 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81021.log 2024-07-17T08:50:08.967 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47621.log 2024-07-17T08:50:08.967 INFO:teuthology.orchestra.run.smithi044.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.73113.log.gz 2024-07-17T08:50:08.968 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109523.log 2024-07-17T08:50:08.968 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81021.log.gz 2024-07-17T08:50:08.969 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87142.log 2024-07-17T08:50:08.969 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47621.log.gz 2024-07-17T08:50:08.970 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109523.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.109523.log.gz 2024-07-17T08:50:08.970 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61509.log 2024-07-17T08:50:08.971 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71604.log 2024-07-17T08:50:08.971 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87142.log.gz 2024-07-17T08:50:08.972 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109351.log 2024-07-17T08:50:08.973 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61509.log.gz 2024-07-17T08:50:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112010.log 2024-07-17T08:50:08.973 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71604.log.gz 2024-07-17T08:50:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95369.log 2024-07-17T08:50:08.974 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.109351.log.gz 2024-07-17T08:50:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112010.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112010.log.gz 2024-07-17T08:50:08.976 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65342.log 2024-07-17T08:50:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48247.log 2024-07-17T08:50:08.977 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95369.log.gz 2024-07-17T08:50:08.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99864.log 2024-07-17T08:50:08.977 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65342.log.gz 2024-07-17T08:50:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62485.log 2024-07-17T08:50:08.978 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48247.log.gz 2024-07-17T08:50:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99864.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99864.log.gz 2024-07-17T08:50:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61085.log 2024-07-17T08:50:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62485.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62485.log.gz 2024-07-17T08:50:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73154.log 2024-07-17T08:50:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107684.log 2024-07-17T08:50:08.982 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61085.log.gz 2024-07-17T08:50:08.982 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82268.log 2024-07-17T08:50:08.983 INFO:teuthology.orchestra.run.smithi044.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.73154.log.gz 2024-07-17T08:50:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44895.log 2024-07-17T08:50:08.984 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107684.log.gz 2024-07-17T08:50:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82268.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82268.log.gz 2024-07-17T08:50:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113693.log 2024-07-17T08:50:08.985 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57185.log 2024-07-17T08:50:08.986 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44895.log.gz 2024-07-17T08:50:08.987 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95537.log 2024-07-17T08:50:08.987 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113693.log.gz 2024-07-17T08:50:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57797.log 2024-07-17T08:50:08.988 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57185.log.gz 2024-07-17T08:50:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103562.log 2024-07-17T08:50:08.989 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95537.log.gz 2024-07-17T08:50:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57797.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57797.log.gz 2024-07-17T08:50:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104301.log 2024-07-17T08:50:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79584.log 2024-07-17T08:50:08.991 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103562.log.gz 2024-07-17T08:50:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73072.log 2024-07-17T08:50:08.992 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104301.log.gz 2024-07-17T08:50:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79584.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79584.log.gz 2024-07-17T08:50:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100595.log 2024-07-17T08:50:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112191.log 2024-07-17T08:50:08.995 INFO:teuthology.orchestra.run.smithi044.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.73072.log.gz 2024-07-17T08:50:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62653.log 2024-07-17T08:50:08.995 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100595.log.gz 2024-07-17T08:50:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112191.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112191.log.gz 2024-07-17T08:50:08.997 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37729.log 2024-07-17T08:50:08.997 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70592.log 2024-07-17T08:50:08.997 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62653.log.gz 2024-07-17T08:50:08.998 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37729.log.gz 2024-07-17T08:50:08.998 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-07-17T08:50:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102420.log 2024-07-17T08:50:09.000 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70592.log.gz 2024-07-17T08:50:09.001 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107011.log 2024-07-17T08:50:09.001 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-07-17T08:50:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69080.log 2024-07-17T08:50:09.002 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.102420.log.gz 2024-07-17T08:50:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121877.log 2024-07-17T08:50:09.003 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107011.log.gz 2024-07-17T08:50:09.004 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61887.log 2024-07-17T08:50:09.004 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69080.log.gz 2024-07-17T08:50:09.005 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121877.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121877.log.gz 2024-07-17T08:50:09.005 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98018.log 2024-07-17T08:50:09.006 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2024-07-17T08:50:09.006 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61887.log.gz 2024-07-17T08:50:09.007 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116632.log 2024-07-17T08:50:09.007 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98018.log.gz 2024-07-17T08:50:09.008 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47705.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz -5 2024-07-17T08:50:09.008 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71827.log 2024-07-17T08:50:09.009 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117226.log 2024-07-17T08:50:09.010 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116632.log.gz 2024-07-17T08:50:09.010 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113735.log 2024-07-17T08:50:09.010 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71827.log.gz 2024-07-17T08:50:09.011 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117226.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-07-17T08:50:09.011 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117226.log.gz 2024-07-17T08:50:09.012 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49320.log 2024-07-17T08:50:09.013 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113735.log.gz 2024-07-17T08:50:09.013 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67485.log 2024-07-17T08:50:09.015 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49320.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2024-07-17T08:50:09.015 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86932.log 2024-07-17T08:50:09.016 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62159.log 2024-07-17T08:50:09.016 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67485.log.gz 2024-07-17T08:50:09.017 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55740.log 2024-07-17T08:50:09.017 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86932.log.gz 2024-07-17T08:50:09.018 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120704.log 2024-07-17T08:50:09.018 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62159.log.gz 2024-07-17T08:50:09.019 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115372.log 2024-07-17T08:50:09.019 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55740.log.gz 2024-07-17T08:50:09.020 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118463.log 2024-07-17T08:50:09.020 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120704.log.gz 2024-07-17T08:50:09.021 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115372.log: 90.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-07-17T08:50:09.022 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50058.log 2024-07-17T08:50:09.022 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115372.log.gz 2024-07-17T08:50:09.022 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67443.log 2024-07-17T08:50:09.023 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118463.log.gz 2024-07-17T08:50:09.023 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.50058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50058.log.gz 2024-07-17T08:50:09.024 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60377.log 2024-07-17T08:50:09.025 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81291.log 2024-07-17T08:50:09.025 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67443.log.gz 2024-07-17T08:50:09.026 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60377.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60377.log.gz 2024-07-17T08:50:09.026 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100968.log 2024-07-17T08:50:09.026 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100169.log 2024-07-17T08:50:09.027 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81291.log.gz 2024-07-17T08:50:09.028 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100968.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100968.log.gz 2024-07-17T08:50:09.028 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101476.log 2024-07-17T08:50:09.029 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100169.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.100169.log.gz 2024-07-17T08:50:09.029 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93104.log 2024-07-17T08:50:09.030 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60461.log 2024-07-17T08:50:09.030 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101476.log.gz 2024-07-17T08:50:09.031 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83298.log 2024-07-17T08:50:09.031 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93104.log.gz 2024-07-17T08:50:09.032 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111059.log 2024-07-17T08:50:09.033 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60461.log.gz 2024-07-17T08:50:09.033 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83298.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83298.log.gz 2024-07-17T08:50:09.033 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80829.log 2024-07-17T08:50:09.034 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101519.log 2024-07-17T08:50:09.035 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111059.log.gz 2024-07-17T08:50:09.035 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110500.log 2024-07-17T08:50:09.035 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80829.log.gz 2024-07-17T08:50:09.037 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101519.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.101519.log.gz 2024-07-17T08:50:09.037 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115012.log 2024-07-17T08:50:09.038 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37719.log 2024-07-17T08:50:09.038 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110500.log.gz 2024-07-17T08:50:09.039 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115012.log.gz 2024-07-17T08:50:09.039 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45118.log 2024-07-17T08:50:09.039 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37719.log.gz 2024-07-17T08:50:09.040 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57269.log 2024-07-17T08:50:09.041 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72458.log 2024-07-17T08:50:09.041 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45118.log.gz 2024-07-17T08:50:09.042 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57269.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57269.log.gz 2024-07-17T08:50:09.042 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91403.log 2024-07-17T08:50:09.043 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57647.log 2024-07-17T08:50:09.043 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72458.log.gz 2024-07-17T08:50:09.044 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95663.log 2024-07-17T08:50:09.044 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.91403.log.gz 2024-07-17T08:50:09.045 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-07-17T08:50:09.045 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57647.log.gz 2024-07-17T08:50:09.046 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60293.log 2024-07-17T08:50:09.046 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95663.log.gz 2024-07-17T08:50:09.047 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48701.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-07-17T08:50:09.047 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73884.log 2024-07-17T08:50:09.048 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46752.log 2024-07-17T08:50:09.049 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60293.log.gz 2024-07-17T08:50:09.049 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73884.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73884.log.gz 2024-07-17T08:50:09.050 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118839.log 2024-07-17T08:50:09.050 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.46752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46752.log.gz 2024-07-17T08:50:09.051 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82394.log 2024-07-17T08:50:09.052 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118839.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.118839.log.gz 2024-07-17T08:50:09.052 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96090.log 2024-07-17T08:50:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87562.log 2024-07-17T08:50:09.053 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82394.log.gz 2024-07-17T08:50:09.053 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.96090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96090.log.gz 2024-07-17T08:50:09.054 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114384.log 2024-07-17T08:50:09.055 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119771.log 2024-07-17T08:50:09.055 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87562.log.gz 2024-07-17T08:50:09.056 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114384.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114384.log.gz 2024-07-17T08:50:09.057 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70989.log 2024-07-17T08:50:09.058 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119771.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119771.log.gz 2024-07-17T08:50:09.058 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117516.log 2024-07-17T08:50:09.059 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70989.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70989.log.gz -5 2024-07-17T08:50:09.059 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50958.log 2024-07-17T08:50:09.060 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117516.log: gzip -5 86.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.117516.log.gz 2024-07-17T08:50:09.060 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.92558.log 2024-07-17T08:50:09.061 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50958.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50958.log.gz 2024-07-17T08:50:09.061 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114216.log 2024-07-17T08:50:09.062 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113067.log 2024-07-17T08:50:09.062 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.92558.log.gz 2024-07-17T08:50:09.063 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114216.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114216.log.gz 2024-07-17T08:50:09.063 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2024-07-17T08:50:09.064 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101094.log 2024-07-17T08:50:09.065 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.113067.log.gz 2024-07-17T08:50:09.065 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50298.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2024-07-17T08:50:09.065 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101136.log 2024-07-17T08:50:09.066 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101094.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101094.log.gz 2024-07-17T08:50:09.067 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65808.log 2024-07-17T08:50:09.068 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119561.log 2024-07-17T08:50:09.068 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101136.log.gz 2024-07-17T08:50:09.069 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65808.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65808.log.gz 2024-07-17T08:50:09.069 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94814.log 2024-07-17T08:50:09.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107605.log 2024-07-17T08:50:09.070 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119561.log.gz 2024-07-17T08:50:09.071 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116674.log 2024-07-17T08:50:09.071 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94814.log.gz 2024-07-17T08:50:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.107605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107605.log.gz 2024-07-17T08:50:09.072 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-07-17T08:50:09.073 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92600.log 2024-07-17T08:50:09.073 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116674.log.gz 2024-07-17T08:50:09.074 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73968.log 2024-07-17T08:50:09.074 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-07-17T08:50:09.075 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92600.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92600.log.gz 2024-07-17T08:50:09.075 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56564.log 2024-07-17T08:50:09.076 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71473.log 2024-07-17T08:50:09.076 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.73968.log.gz 2024-07-17T08:50:09.077 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110297.log 2024-07-17T08:50:09.077 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56564.log.gz 2024-07-17T08:50:09.078 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116156.log 87.7% 2024-07-17T08:50:09.078 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71473.log.gz 2024-07-17T08:50:09.079 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86760.log 2024-07-17T08:50:09.079 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110297.log.gz 2024-07-17T08:50:09.080 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115564.log 2024-07-17T08:50:09.080 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.116156.log.gz 2024-07-17T08:50:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86760.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86760.log.gz 2024-07-17T08:50:09.081 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55473.log 2024-07-17T08:50:09.082 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.115564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.115564.log.gz 2024-07-17T08:50:09.082 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45945.log 2024-07-17T08:50:09.083 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55473.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55473.log.gz 2024-07-17T08:50:09.083 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109266.log 2024-07-17T08:50:09.084 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98270.log 2024-07-17T08:50:09.084 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45945.log.gz 2024-07-17T08:50:09.085 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109266.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.109266.log.gz 2024-07-17T08:50:09.085 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.83340.log 2024-07-17T08:50:09.086 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70466.log 2024-07-17T08:50:09.086 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98270.log.gz 2024-07-17T08:50:09.087 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83340.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83340.log.gz 2024-07-17T08:50:09.087 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73593.log 2024-07-17T08:50:09.088 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70466.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70466.log.gz 2024-07-17T08:50:09.088 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-07-17T08:50:09.089 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73593.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73593.log.gz 2024-07-17T08:50:09.089 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87058.log 2024-07-17T08:50:09.090 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122411.log 2024-07-17T08:50:09.090 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz 2024-07-17T08:50:09.091 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87058.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.87058.log.gz 2024-07-17T08:50:09.091 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115845.log 2024-07-17T08:50:09.092 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79752.log 2024-07-17T08:50:09.092 INFO:teuthology.orchestra.run.smithi044.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.122411.log.gz 2024-07-17T08:50:09.093 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115845.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115845.log.gz 2024-07-17T08:50:09.093 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75215.log 2024-07-17T08:50:09.094 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79752.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.79752.log.gz 2024-07-17T08:50:09.094 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95035.log 2024-07-17T08:50:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75215.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.75215.log.gz 2024-07-17T08:50:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101052.log 2024-07-17T08:50:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95035.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95035.log.gz 2024-07-17T08:50:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82945.log 2024-07-17T08:50:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101052.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.101052.log.gz 2024-07-17T08:50:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83084.log 2024-07-17T08:50:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64483.log 2024-07-17T08:50:09.099 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82945.log.gz 2024-07-17T08:50:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83084.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83084.log.gz 2024-07-17T08:50:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109607.log 2024-07-17T08:50:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120854.log 2024-07-17T08:50:09.101 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64483.log.gz 2024-07-17T08:50:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109607.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109607.log.gz 2024-07-17T08:50:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114882.log 2024-07-17T08:50:09.103 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86262.log 2024-07-17T08:50:09.103 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120854.log.gz 2024-07-17T08:50:09.104 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114882.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114882.log.gz 2024-07-17T08:50:09.104 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-07-17T08:50:09.105 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50857.log 2024-07-17T08:50:09.105 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86262.log.gz 2024-07-17T08:50:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-07-17T08:50:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50857.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50857.log.gz 2024-07-17T08:50:09.107 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117825.log 2024-07-17T08:50:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62840.log 2024-07-17T08:50:09.109 INFO:teuthology.orchestra.run.smithi044.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-07-17T08:50:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117825.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117825.log.gz 2024-07-17T08:50:09.110 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108842.log 2024-07-17T08:50:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56957.log 2024-07-17T08:50:09.111 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62840.log.gz 2024-07-17T08:50:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108842.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108842.log.gz 2024-07-17T08:50:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103207.log 2024-07-17T08:50:09.114 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118289.log 2024-07-17T08:50:09.115 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56957.log.gz 2024-07-17T08:50:09.115 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103207.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103207.log.gz 2024-07-17T08:50:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104652.log 2024-07-17T08:50:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.118289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118289.log.gz 2024-07-17T08:50:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115264.log 2024-07-17T08:50:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104652.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104652.log.gz 2024-07-17T08:50:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49051.log 2024-07-17T08:50:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104778.log 2024-07-17T08:50:09.119 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115264.log.gz 2024-07-17T08:50:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49051.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49051.log.gz 2024-07-17T08:50:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79626.log 2024-07-17T08:50:09.121 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111269.log 2024-07-17T08:50:09.121 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104778.log.gz 2024-07-17T08:50:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79626.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79626.log.gz 2024-07-17T08:50:09.122 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49190.log 2024-07-17T08:50:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94646.log 2024-07-17T08:50:09.124 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.111269.log.gz 2024-07-17T08:50:09.124 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49190.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49190.log.gz 2024-07-17T08:50:09.125 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107095.log 2024-07-17T08:50:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57951.log 2024-07-17T08:50:09.126 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94646.log.gz 2024-07-17T08:50:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107095.log.gz 2024-07-17T08:50:09.127 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81974.log 2024-07-17T08:50:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94543.log 2024-07-17T08:50:09.128 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57951.log.gz 2024-07-17T08:50:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-07-17T08:50:09.129 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81974.log.gz 2024-07-17T08:50:09.130 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53351.log 2024-07-17T08:50:09.131 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94543.log.gz 2024-07-17T08:50:09.131 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2024-07-17T08:50:09.132 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-07-17T08:50:09.133 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53351.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53351.log.gz 2024-07-17T08:50:09.133 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74239.log 2024-07-17T08:50:09.134 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72566.log 2024-07-17T08:50:09.134 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2024-07-17T08:50:09.135 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74239.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74239.log.gz 2024-07-17T08:50:09.136 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96382.log 2024-07-17T08:50:09.137 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45117.log 2024-07-17T08:50:09.137 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72566.log.gz 2024-07-17T08:50:09.138 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48415.log 2024-07-17T08:50:09.138 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96382.log.gz 2024-07-17T08:50:09.139 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59896.log 2024-07-17T08:50:09.139 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45117.log.gz 2024-07-17T08:50:09.140 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56263.log 2024-07-17T08:50:09.140 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48415.log.gz 2024-07-17T08:50:09.141 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107726.log 2024-07-17T08:50:09.142 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59896.log.gz 2024-07-17T08:50:09.142 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56263.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56263.log.gzgzip 2024-07-17T08:50:09.143 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.119897.log 2024-07-17T08:50:09.144 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106926.log 2024-07-17T08:50:09.144 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107726.log.gz 2024-07-17T08:50:09.145 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93834.log 2024-07-17T08:50:09.145 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119897.log.gz 2024-07-17T08:50:09.146 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107305.log 2024-07-17T08:50:09.146 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106926.log.gz 2024-07-17T08:50:09.147 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93408.log 2024-07-17T08:50:09.148 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93834.log.gz 2024-07-17T08:50:09.149 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79962.log 2024-07-17T08:50:09.149 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107305.log.gz 2024-07-17T08:50:09.150 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68930.log 2024-07-17T08:50:09.150 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.93408.log.gz 2024-07-17T08:50:09.151 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122255.log 2024-07-17T08:50:09.151 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79962.log.gz 2024-07-17T08:50:09.152 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80022.log 2024-07-17T08:50:09.152 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68930.log.gz 2024-07-17T08:50:09.153 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79374.log 2024-07-17T08:50:09.153 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.122255.log.gz 2024-07-17T08:50:09.155 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80022.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80022.log.gz 2024-07-17T08:50:09.155 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-07-17T08:50:09.156 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110542.log 2024-07-17T08:50:09.156 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79374.log.gz 2024-07-17T08:50:09.157 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122171.log 2024-07-17T08:50:09.157 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-07-17T08:50:09.158 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69014.log 2024-07-17T08:50:09.159 INFO:teuthology.orchestra.run.smithi044.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.110542.log.gz 2024-07-17T08:50:09.160 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65895.log 2024-07-17T08:50:09.160 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122171.log.gz 2024-07-17T08:50:09.161 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48499.log 2024-07-17T08:50:09.161 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69014.log.gz 2024-07-17T08:50:09.162 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94189.log 2024-07-17T08:50:09.162 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65895.log.gz 2024-07-17T08:50:09.163 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83168.log 2024-07-17T08:50:09.164 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48499.log.gz 2024-07-17T08:50:09.165 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66204.log 2024-07-17T08:50:09.165 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94189.log.gz 2024-07-17T08:50:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51084.log 2024-07-17T08:50:09.166 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83168.log.gz 2024-07-17T08:50:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91491.log 2024-07-17T08:50:09.167 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66204.log.gz 2024-07-17T08:50:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117951.log 2024-07-17T08:50:09.168 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51084.log.gz 2024-07-17T08:50:09.169 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51210.log 2024-07-17T08:50:09.170 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91491.log.gz 2024-07-17T08:50:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64903.log 2024-07-17T08:50:09.171 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.117951.log.gz 2024-07-17T08:50:09.172 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82604.log 2024-07-17T08:50:09.172 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51210.log.gz 2024-07-17T08:50:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71115.log 2024-07-17T08:50:09.173 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64903.log.gz 2024-07-17T08:50:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112551.log 2024-07-17T08:50:09.174 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82604.log.gz 2024-07-17T08:50:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71115.log: gzip -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.94315.log -- replaced with /var/log/ceph/ceph-client.admin.71115.log.gz 2024-07-17T08:50:09.176 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2024-07-17T08:50:09.177 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112551.log.gz 2024-07-17T08:50:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117867.log 2024-07-17T08:50:09.178 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94315.log.gz 2024-07-17T08:50:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55950.log 2024-07-17T08:50:09.179 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2024-07-17T08:50:09.180 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59672.log 2024-07-17T08:50:09.180 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.117867.log.gz 2024-07-17T08:50:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92428.log 2024-07-17T08:50:09.182 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55950.log.gz 2024-07-17T08:50:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80871.log 2024-07-17T08:50:09.183 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59672.log.gz 2024-07-17T08:50:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110177.log 2024-07-17T08:50:09.184 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92428.log.gz 2024-07-17T08:50:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65990.log 2024-07-17T08:50:09.185 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80871.log.gz 2024-07-17T08:50:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.110177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110177.log.gz 2024-07-17T08:50:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86518.log 2024-07-17T08:50:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53877.log 2024-07-17T08:50:09.188 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65990.log.gz 2024-07-17T08:50:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-07-17T08:50:09.189 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86518.log.gz 2024-07-17T08:50:09.190 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.53877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-07-17T08:50:09.190 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53877.log.gz 2024-07-17T08:50:09.191 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115306.log 2024-07-17T08:50:09.191 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-07-17T08:50:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46113.log 2024-07-17T08:50:09.193 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-07-17T08:50:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66686.log 2024-07-17T08:50:09.194 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.115306.log.gz 2024-07-17T08:50:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64819.log 2024-07-17T08:50:09.195 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46113.log.gz 2024-07-17T08:50:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.66686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66686.log.gz 2024-07-17T08:50:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85613.log 2024-07-17T08:50:09.197 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69500.log 2024-07-17T08:50:09.197 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64819.log.gz 2024-07-17T08:50:09.199 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85613.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85613.log.gz 2024-07-17T08:50:09.199 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56911.log 2024-07-17T08:50:09.200 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71031.log 2024-07-17T08:50:09.200 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69500.log.gz 2024-07-17T08:50:09.201 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117047.log 2024-07-17T08:50:09.201 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56911.log.gz 2024-07-17T08:50:09.202 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71869.log 2024-07-17T08:50:09.202 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71031.log.gz 2024-07-17T08:50:09.204 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108758.log 2024-07-17T08:50:09.204 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117047.log.gz 2024-07-17T08:50:09.205 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45110.log 2024-07-17T08:50:09.205 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71869.log.gz 2024-07-17T08:50:09.206 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93366.log 2024-07-17T08:50:09.206 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108758.log.gz 2024-07-17T08:50:09.207 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90255.log 2024-07-17T08:50:09.208 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45110.log.gz 2024-07-17T08:50:09.209 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101788.log 2024-07-17T08:50:09.209 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93366.log.gz 2024-07-17T08:50:09.210 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84999.log 2024-07-17T08:50:09.210 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90255.log.gz 2024-07-17T08:50:09.211 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101788.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-07-17T08:50:09.211 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101788.log.gz 2024-07-17T08:50:09.212 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118371.log 2024-07-17T08:50:09.212 INFO:teuthology.orchestra.run.smithi044.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.84999.log.gz 2024-07-17T08:50:09.213 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-07-17T08:50:09.215 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104736.log 2024-07-17T08:50:09.215 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118371.log.gz 2024-07-17T08:50:09.216 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69268.log 2024-07-17T08:50:09.216 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-07-17T08:50:09.217 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.104736.log: -- /var/log/ceph/ceph-client.admin.56305.log 2024-07-17T08:50:09.218 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104736.log.gz 2024-07-17T08:50:09.219 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.69268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69268.log.gz 2024-07-17T08:50:09.219 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97766.log 2024-07-17T08:50:09.220 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56305.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56305.log.gz 2024-07-17T08:50:09.221 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68632.log 2024-07-17T08:50:09.222 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68846.log 2024-07-17T08:50:09.222 INFO:teuthology.orchestra.run.smithi044.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.97766.log.gz 2024-07-17T08:50:09.223 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60990.log 2024-07-17T08:50:09.223 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68632.log.gz 2024-07-17T08:50:09.225 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64987.log 2024-07-17T08:50:09.225 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.68846.log.gz 2024-07-17T08:50:09.226 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65216.log 2024-07-17T08:50:09.226 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60990.log.gz 2024-07-17T08:50:09.227 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106076.log 2024-07-17T08:50:09.227 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64987.log.gz 2024-07-17T08:50:09.229 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113385.log 2024-07-17T08:50:09.229 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.65216.log.gz 2024-07-17T08:50:09.230 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57143.log 2024-07-17T08:50:09.230 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106076.log.gz 2024-07-17T08:50:09.232 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110667.log 2024-07-17T08:50:09.232 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.113385.log.gz 2024-07-17T08:50:09.233 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101872.log 2024-07-17T08:50:09.233 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57143.log.gz 2024-07-17T08:50:09.234 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83574.log 2024-07-17T08:50:09.234 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110667.log.gz 2024-07-17T08:50:09.236 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101872.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.101872.log.gz --verbose 2024-07-17T08:50:09.236 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.97409.log 2024-07-17T08:50:09.237 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62569.log 2024-07-17T08:50:09.238 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83574.log.gz 2024-07-17T08:50:09.238 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103916.log 2024-07-17T08:50:09.239 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97409.log.gz 2024-07-17T08:50:09.240 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122213.log 2024-07-17T08:50:09.241 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62569.log.gz 2024-07-17T08:50:09.242 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68521.log 2024-07-17T08:50:09.242 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103916.log.gz 2024-07-17T08:50:09.243 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122339.log 2024-07-17T08:50:09.243 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122213.log.gz 2024-07-17T08:50:09.244 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75257.log 2024-07-17T08:50:09.244 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68521.log.gz 2024-07-17T08:50:09.246 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56053.log 2024-07-17T08:50:09.246 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122339.log.gz 2024-07-17T08:50:09.247 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110933.log 2024-07-17T08:50:09.247 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75257.log.gz 2024-07-17T08:50:09.248 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68572.log 2024-07-17T08:50:09.248 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56053.log.gz 2024-07-17T08:50:09.249 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80493.log 2024-07-17T08:50:09.250 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110933.log.gz 2024-07-17T08:50:09.251 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56347.log 2024-07-17T08:50:09.251 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68572.log.gz 2024-07-17T08:50:09.252 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102084.log 2024-07-17T08:50:09.252 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80493.log.gz 2024-07-17T08:50:09.253 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112821.log 2024-07-17T08:50:09.254 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56347.log.gz 2024-07-17T08:50:09.254 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111776.log 2024-07-17T08:50:09.255 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.102084.log.gz 2024-07-17T08:50:09.256 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112821.log: gzip -5 --verbose -- 86.8% /var/log/ceph/ceph-client.admin.51941.log 2024-07-17T08:50:09.256 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112821.log.gz 2024-07-17T08:50:09.258 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94940.log 2024-07-17T08:50:09.258 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111776.log.gz 2024-07-17T08:50:09.259 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56179.log 2024-07-17T08:50:09.259 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51941.log.gz 2024-07-17T08:50:09.260 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47194.log 2024-07-17T08:50:09.260 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94940.log.gz 2024-07-17T08:50:09.262 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85739.log 2024-07-17T08:50:09.262 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56179.log.gz 2024-07-17T08:50:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.2.47194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47194.log.gz 2024-07-17T08:50:09.263 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109649.log 2024-07-17T08:50:09.264 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90423.log 2024-07-17T08:50:09.264 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85739.log.gz 2024-07-17T08:50:09.266 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97976.log 2024-07-17T08:50:09.266 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.109649.log.gz 2024-07-17T08:50:09.267 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82310.log 2024-07-17T08:50:09.267 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90423.log.gz 2024-07-17T08:50:09.268 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109010.log 2024-07-17T08:50:09.268 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97976.log.gz 2024-07-17T08:50:09.269 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61845.log 2024-07-17T08:50:09.270 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82310.log.gz 2024-07-17T08:50:09.271 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109010.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-07-17T08:50:09.271 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109010.log.gz 2024-07-17T08:50:09.272 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44499.log 2024-07-17T08:50:09.272 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.61845.log.gz 2024-07-17T08:50:09.273 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95325.log 2024-07-17T08:50:09.275 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81384.log 2024-07-17T08:50:09.275 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44499.log.gz 2024-07-17T08:50:09.276 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109105.log 2024-07-17T08:50:09.276 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95325.log.gz 2024-07-17T08:50:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86357.log 2024-07-17T08:50:09.278 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81384.log.gz 2024-07-17T08:50:09.278 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109105.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-07-17T08:50:09.279 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.109105.log.gz 2024-07-17T08:50:09.280 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74837.log 2024-07-17T08:50:09.280 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86357.log.gz 2024-07-17T08:50:09.281 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49740.log 2024-07-17T08:50:09.283 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121919.log 2024-07-17T08:50:09.283 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74837.log.gz 2024-07-17T08:50:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49740.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49740.log.gz 2024-07-17T08:50:09.284 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93170.log 2024-07-17T08:50:09.285 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121919.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121919.log.gz 2024-07-17T08:50:09.286 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56774.log 2024-07-17T08:50:09.287 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93170.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119101.log 2024-07-17T08:50:09.287 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93170.log.gz 2024-07-17T08:50:09.288 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56774.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56774.log.gz 2024-07-17T08:50:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79416.log 2024-07-17T08:50:09.289 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119101.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119101.log.gz 2024-07-17T08:50:09.290 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111143.log 2024-07-17T08:50:09.291 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95495.log 2024-07-17T08:50:09.291 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79416.log.gz 2024-07-17T08:50:09.293 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52151.log 2024-07-17T08:50:09.293 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111143.log.gz 2024-07-17T08:50:09.294 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87520.log 2024-07-17T08:50:09.294 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95495.log.gz 2024-07-17T08:50:09.295 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-07-17T08:50:09.295 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52151.log.gz 2024-07-17T08:50:09.296 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71655.log 2024-07-17T08:50:09.297 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87520.log.gz 2024-07-17T08:50:09.297 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-07-17T08:50:09.298 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114831.log 2024-07-17T08:50:09.299 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62966.log 2024-07-17T08:50:09.299 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71655.log.gz 2024-07-17T08:50:09.300 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57569.log 2024-07-17T08:50:09.300 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114831.log.gz 2024-07-17T08:50:09.301 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62966.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62966.log.gz 2024-07-17T08:50:09.302 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90108.log 2024-07-17T08:50:09.302 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.57569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57569.log.gz 2024-07-17T08:50:09.304 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96718.log 2024-07-17T08:50:09.305 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63350.log 2024-07-17T08:50:09.305 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90108.log.gz 2024-07-17T08:50:09.307 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113777.log 2024-07-17T08:50:09.307 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96718.log.gz 2024-07-17T08:50:09.308 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82058.log 2024-07-17T08:50:09.308 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63350.log.gz 2024-07-17T08:50:09.309 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113777.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-07-17T08:50:09.310 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113777.log.gz 2024-07-17T08:50:09.310 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82058.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82058.log.gz 2024-07-17T08:50:09.313 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-07-17T08:50:09.315 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70821.log 2024-07-17T08:50:09.316 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37628.log: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-07-17T08:50:09.317 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118756.log/var/log/ceph/ceph-client.admin.70821.log: 2024-07-17T08:50:09.317 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70821.log.gz 2024-07-17T08:50:09.318 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74963.log 2024-07-17T08:50:09.319 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118756.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.118756.log.gz 2024-07-17T08:50:09.320 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95411.log 2024-07-17T08:50:09.321 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74963.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74963.log.gz 2024-07-17T08:50:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95411.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.95411.log.gz 2024-07-17T08:50:09.323 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91445.log 2024-07-17T08:50:09.325 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72695.log 2024-07-17T08:50:09.326 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91445.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91445.log.gz 2024-07-17T08:50:09.327 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72695.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72695.log.gz 2024-07-17T08:50:09.328 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106630.log 2024-07-17T08:50:09.330 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101353.log 2024-07-17T08:50:09.331 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106630.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106630.log.gz 2024-07-17T08:50:09.332 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101353.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101353.log.gz 2024-07-17T08:50:09.333 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61995.log 2024-07-17T08:50:09.334 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116506.log 2024-07-17T08:50:09.336 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61995.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61995.log.gz 2024-07-17T08:50:09.337 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116506.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116506.log.gz 2024-07-17T08:50:09.337 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79150.log 2024-07-17T08:50:09.339 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115685.log 2024-07-17T08:50:09.340 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79150.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79150.log.gz 2024-07-17T08:50:09.341 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115685.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115685.log.gz 2024-07-17T08:50:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80955.log 2024-07-17T08:50:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48079.log 2024-07-17T08:50:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80955.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.80955.log.gz 2024-07-17T08:50:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48079.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48079.log.gz 2024-07-17T08:50:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67840.log 2024-07-17T08:50:09.348 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70760.log 2024-07-17T08:50:09.349 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67840.log.gz 2024-07-17T08:50:09.350 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70760.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70760.log.gz 2024-07-17T08:50:09.351 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-07-17T08:50:09.352 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90754.log 2024-07-17T08:50:09.354 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45446.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-07-17T08:50:09.354 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.90754.log: -- /var/log/ceph/ceph-client.admin.75131.log 2024-07-17T08:50:09.355 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90754.log.gz 2024-07-17T08:50:09.356 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95897.log 2024-07-17T08:50:09.357 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75131.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.75131.log.gz 2024-07-17T08:50:09.358 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113861.log 2024-07-17T08:50:09.359 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95897.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95897.log.gz 2024-07-17T08:50:09.360 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66162.log 2024-07-17T08:50:09.361 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113861.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113861.log.gz 2024-07-17T08:50:09.362 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104091.log 2024-07-17T08:50:09.363 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66162.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66162.log.gz 2024-07-17T08:50:09.364 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37565.log/var/log/ceph/ceph-client.admin.104091.log: 2024-07-17T08:50:09.364 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104091.log.gz 2024-07-17T08:50:09.365 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115888.log 2024-07-17T08:50:09.368 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37565.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37565.log.gz 2024-07-17T08:50:09.368 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114725.log 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115888.log.gz 2024-07-17T08:50:09.368 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:09.369 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96252.log 2024-07-17T08:50:09.371 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114725.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114725.log.gz 2024-07-17T08:50:09.372 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96252.log: -- /var/log/ceph/ceph-client.admin.60419.log 2024-07-17T08:50:09.372 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96252.log.gz 2024-07-17T08:50:09.373 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64531.log 2024-07-17T08:50:09.374 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60419.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60419.log.gz 2024-07-17T08:50:09.375 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-07-17T08:50:09.376 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64531.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64531.log.gz 2024-07-17T08:50:09.377 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52025.log 2024-07-17T08:50:09.378 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-07-17T08:50:09.379 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72145.log 2024-07-17T08:50:09.380 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52025.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52025.log.gz 2024-07-17T08:50:09.381 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81209.log 2024-07-17T08:50:09.382 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72145.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72145.log.gz 2024-07-17T08:50:09.383 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92081.log 2024-07-17T08:50:09.383 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.81209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81209.log.gz 2024-07-17T08:50:09.384 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74071.log 2024-07-17T08:50:09.385 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.92081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92081.log.gz 2024-07-17T08:50:09.386 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65766.log 2024-07-17T08:50:09.387 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74071.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74071.log.gz 2024-07-17T08:50:09.389 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.65766.log: --verbose -- /var/log/ceph/ceph-client.admin.83904.log 2024-07-17T08:50:09.390 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65766.log.gz 2024-07-17T08:50:09.390 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91617.log 2024-07-17T08:50:09.392 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83904.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83904.log.gz 2024-07-17T08:50:09.393 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.91617.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60208.log 2024-07-17T08:50:09.393 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91617.log.gz 2024-07-17T08:50:09.394 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104820.log 2024-07-17T08:50:09.396 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60208.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60208.log.gz 2024-07-17T08:50:09.397 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38524.log 2024-07-17T08:50:09.398 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104820.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104820.log.gz 2024-07-17T08:50:09.399 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38524.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.38524.log.gz 2024-07-17T08:50:09.399 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50916.log 2024-07-17T08:50:09.401 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67966.log 2024-07-17T08:50:09.403 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50916.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50916.log.gz 2024-07-17T08:50:09.403 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113206.log 2024-07-17T08:50:09.404 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67966.log.gz 2024-07-17T08:50:09.405 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68804.log 2024-07-17T08:50:09.406 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113206.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.113206.log.gz 2024-07-17T08:50:09.407 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68804.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68804.log.gz 2024-07-17T08:50:09.408 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97640.log 2024-07-17T08:50:09.409 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79920.log 2024-07-17T08:50:09.410 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97640.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97640.log.gz 2024-07-17T08:50:09.411 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117699.log/var/log/ceph/ceph-client.admin.79920.log: 2024-07-17T08:50:09.412 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79920.log.gz 2024-07-17T08:50:09.413 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82016.log 2024-07-17T08:50:09.414 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117699.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117699.log.gz 2024-07-17T08:50:09.415 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.82016.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37644.log 2024-07-17T08:50:09.416 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82016.log.gz 2024-07-17T08:50:09.417 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117387.log 2024-07-17T08:50:09.419 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37644.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37644.log.gz 2024-07-17T08:50:09.419 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111227.log/var/log/ceph/ceph-client.admin.117387.log: 2024-07-17T08:50:09.420 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117387.log.gz 2024-07-17T08:50:09.420 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67569.log 2024-07-17T08:50:09.422 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111227.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111227.log.gz 2024-07-17T08:50:09.423 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.67569.log: /var/log/ceph/ceph-client.admin.112145.log 2024-07-17T08:50:09.423 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67569.log.gz 2024-07-17T08:50:09.424 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102150.log 2024-07-17T08:50:09.426 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112145.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112145.log.gz 2024-07-17T08:50:09.427 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79458.log 2024-07-17T08:50:09.427 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102150.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102150.log.gz 2024-07-17T08:50:09.428 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98456.log 2024-07-17T08:50:09.429 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79458.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79458.log.gz 2024-07-17T08:50:09.430 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.98456.log: --verbose -- /var/log/ceph/ceph-client.admin.89989.log 2024-07-17T08:50:09.431 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98456.log.gz 2024-07-17T08:50:09.432 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-07-17T08:50:09.433 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.89989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89989.log.gz 2024-07-17T08:50:09.434 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37989.log: -- /var/log/ceph/ceph-client.admin.103102.log 2024-07-17T08:50:09.434 INFO:teuthology.orchestra.run.smithi044.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-07-17T08:50:09.436 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68720.log 2024-07-17T08:50:09.437 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103102.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103102.log.gz 2024-07-17T08:50:09.437 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110583.log 2024-07-17T08:50:09.438 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68720.log.gz 2024-07-17T08:50:09.439 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63955.log 2024-07-17T08:50:09.440 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110583.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.110583.log.gz 2024-07-17T08:50:09.441 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92474.log/var/log/ceph/ceph-client.admin.63955.log: 2024-07-17T08:50:09.442 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63955.log.gz 2024-07-17T08:50:09.443 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100926.log 2024-07-17T08:50:09.447 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92474.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.92474.log.gz 2024-07-17T08:50:09.447 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122451.log 2024-07-17T08:50:09.447 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100926.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100926.log.gz 2024-07-17T08:50:09.448 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108548.log 2024-07-17T08:50:09.449 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122451.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122451.log.gz 2024-07-17T08:50:09.449 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71516.log 2024-07-17T08:50:09.450 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108548.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108548.log.gz 2024-07-17T08:50:09.450 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51815.log 2024-07-17T08:50:09.451 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71516.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71516.log.gz 2024-07-17T08:50:09.452 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60946.log/var/log/ceph/ceph-client.admin.51815.log: 2024-07-17T08:50:09.453 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51815.log.gz 2024-07-17T08:50:09.454 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log 2024-07-17T08:50:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60946.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60946.log.gz 2024-07-17T08:50:09.457 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51252.log: -- /var/log/ceph/ceph-client.0.75557.log 2024-07-17T08:50:09.457 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-07-17T08:50:09.459 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103832.log 2024-07-17T08:50:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.75557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75557.log.gz 2024-07-17T08:50:09.460 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100739.log 2024-07-17T08:50:09.461 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103832.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103832.log.gz 2024-07-17T08:50:09.461 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109877.log 2024-07-17T08:50:09.462 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100739.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100739.log.gz 2024-07-17T08:50:09.463 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99740.log 2024-07-17T08:50:09.464 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109877.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.109877.log.gz 2024-07-17T08:50:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64595.log 2024-07-17T08:50:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.99740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99740.log.gz 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108030.log 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64595.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64595.log.gz 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87819.log 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108030.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.108030.log.gz 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83762.log 2024-07-17T08:50:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87819.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.87819.log.gz 2024-07-17T08:50:09.473 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113427.log 2024-07-17T08:50:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.83762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83762.log.gz 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101563.log/var/log/ceph/ceph-client.admin.113427.log: 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113427.log.gz 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81632.log 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101563.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101563.log.gz 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81632.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.81632.log.gz 2024-07-17T08:50:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96862.log 2024-07-17T08:50:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43213.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-07-17T08:50:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83126.log 2024-07-17T08:50:09.481 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96862.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.96862.log.gz 2024-07-17T08:50:09.482 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121142.log 2024-07-17T08:50:09.483 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83126.log: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-07-17T08:50:09.483 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83126.log.gz 2024-07-17T08:50:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68674.log 2024-07-17T08:50:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108464.log 2024-07-17T08:50:09.485 INFO:teuthology.orchestra.run.smithi044.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.121142.log.gz 2024-07-17T08:50:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80159.log 2024-07-17T08:50:09.486 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68674.log.gz 2024-07-17T08:50:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113819.log 2024-07-17T08:50:09.487 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108464.log.gz 2024-07-17T08:50:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107347.log 2024-07-17T08:50:09.489 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80159.log.gz 2024-07-17T08:50:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101178.log 2024-07-17T08:50:09.490 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113819.log.gz 2024-07-17T08:50:09.491 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94002.log 2024-07-17T08:50:09.491 INFO:teuthology.orchestra.run.smithi044.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.107347.log.gz 2024-07-17T08:50:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101178.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101178.log.gz 2024-07-17T08:50:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112061.log 2024-07-17T08:50:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98498.log 2024-07-17T08:50:09.494 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94002.log.gz 2024-07-17T08:50:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78138.log 2024-07-17T08:50:09.495 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112061.log.gz 2024-07-17T08:50:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98498.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.98498.log.gz 2024-07-17T08:50:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120620.log 2024-07-17T08:50:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.78138.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78138.log.gz 2024-07-17T08:50:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66768.log 2024-07-17T08:50:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101700.log 2024-07-17T08:50:09.499 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120620.log.gz 2024-07-17T08:50:09.500 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66768.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66768.log.gz 2024-07-17T08:50:09.501 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.101700.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2024-07-17T08:50:09.502 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101700.log.gz 2024-07-17T08:50:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94688.log 2024-07-17T08:50:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74113.log 2024-07-17T08:50:09.504 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2024-07-17T08:50:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65300.log 2024-07-17T08:50:09.505 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94688.log.gz 2024-07-17T08:50:09.506 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74281.log 2024-07-17T08:50:09.506 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.74113.log.gz 2024-07-17T08:50:09.507 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65300.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114048.log 2024-07-17T08:50:09.507 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65300.log.gz 2024-07-17T08:50:09.509 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67756.log 2024-07-17T08:50:09.509 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74281.log.gz 2024-07-17T08:50:09.510 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114048.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114048.log.gz 2024-07-17T08:50:09.510 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112863.log 2024-07-17T08:50:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67756.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67756.log.gz 2024-07-17T08:50:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46886.log 2024-07-17T08:50:09.513 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72634.log 2024-07-17T08:50:09.513 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112863.log.gz 2024-07-17T08:50:09.514 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87184.log 2024-07-17T08:50:09.514 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46886.log.gz 2024-07-17T08:50:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60106.log 2024-07-17T08:50:09.516 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72634.log.gz 2024-07-17T08:50:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52825.log 2024-07-17T08:50:09.518 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.87184.log.gz 2024-07-17T08:50:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60106.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60106.log.gz 2024-07-17T08:50:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84012.log 2024-07-17T08:50:09.519 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82436.log 2024-07-17T08:50:09.519 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52825.log.gz 2024-07-17T08:50:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66372.log 2024-07-17T08:50:09.521 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84012.log.gz 2024-07-17T08:50:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105242.log 2024-07-17T08:50:09.522 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.82436.log.gz 2024-07-17T08:50:09.523 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66372.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66372.log.gz 2024-07-17T08:50:09.523 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61215.log 2024-07-17T08:50:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105242.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105242.log.gz 2024-07-17T08:50:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74706.log 2024-07-17T08:50:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2024-07-17T08:50:09.526 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61215.log.gz 2024-07-17T08:50:09.527 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57311.log 2024-07-17T08:50:09.527 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74706.log.gz 2024-07-17T08:50:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52067.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2024-07-17T08:50:09.529 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114090.log 2024-07-17T08:50:09.530 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57311.log: gzip -5 --verbose 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57311.log.gz 2024-07-17T08:50:09.530 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.97162.log 2024-07-17T08:50:09.531 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114090.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114090.log.gz 2024-07-17T08:50:09.531 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99552.log 2024-07-17T08:50:09.533 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100553.log 2024-07-17T08:50:09.533 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97162.log.gz 2024-07-17T08:50:09.534 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98540.log 2024-07-17T08:50:09.534 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99552.log.gz 2024-07-17T08:50:09.535 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100553.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.57689.log 2024-07-17T08:50:09.536 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100553.log.gz 2024-07-17T08:50:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98540.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98540.log.gz 2024-07-17T08:50:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-07-17T08:50:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99150.log 2024-07-17T08:50:09.538 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57689.log.gz 2024-07-17T08:50:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98879.log 2024-07-17T08:50:09.540 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-07-17T08:50:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116317.log 2024-07-17T08:50:09.541 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99150.log.gz 2024-07-17T08:50:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98879.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-07-17T08:50:09.542 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98879.log.gz 2024-07-17T08:50:09.543 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116317.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.116317.log.gz --verbose 2024-07-17T08:50:09.543 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.92291.log 2024-07-17T08:50:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-07-17T08:50:09.547 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92291.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92291.log.gz 2024-07-17T08:50:09.547 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.46071.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.92163.log 2024-07-17T08:50:09.547 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz 2024-07-17T08:50:09.548 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98354.log 2024-07-17T08:50:09.550 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92163.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92163.log.gz 2024-07-17T08:50:09.551 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log/var/log/ceph/ceph-client.admin.98354.log: 2024-07-17T08:50:09.551 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98354.log.gz 2024-07-17T08:50:09.552 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43255.log 2024-07-17T08:50:09.553 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43041.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2024-07-17T08:50:09.558 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64945.log 2024-07-17T08:50:09.558 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43255.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2024-07-17T08:50:09.558 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119206.log 2024-07-17T08:50:09.558 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64945.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64945.log.gz 2024-07-17T08:50:09.559 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114258.log 2024-07-17T08:50:09.560 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119206.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119206.log.gz 2024-07-17T08:50:09.560 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-07-17T08:50:09.560 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114258.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114258.log.gz 2024-07-17T08:50:09.561 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92085.log 2024-07-17T08:50:09.562 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47369.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-07-17T08:50:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115222.log 2024-07-17T08:50:09.563 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.92085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92085.log.gz 2024-07-17T08:50:09.564 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121064.log 2024-07-17T08:50:09.565 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115222.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.115222.log.gz 2024-07-17T08:50:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119855.log 2024-07-17T08:50:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.121064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121064.log.gz 2024-07-17T08:50:09.568 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98102.log 2024-07-17T08:50:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119855.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119855.log.gz 2024-07-17T08:50:09.570 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61767.log 2024-07-17T08:50:09.571 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98102.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98102.log.gz 2024-07-17T08:50:09.571 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81334.log 2024-07-17T08:50:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.61767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61767.log.gz 2024-07-17T08:50:09.573 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74921.log 2024-07-17T08:50:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81334.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-07-17T08:50:09.575 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.81334.log.gz 2024-07-17T08:50:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114174.log 2024-07-17T08:50:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74921.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102983.log 2024-07-17T08:50:09.576 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74921.log.gz 2024-07-17T08:50:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100655.log 2024-07-17T08:50:09.577 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114174.log.gz 2024-07-17T08:50:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119813.log 2024-07-17T08:50:09.579 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.102983.log.gz 2024-07-17T08:50:09.580 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101830.log 2024-07-17T08:50:09.580 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100655.log.gz 2024-07-17T08:50:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119813.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119813.log.gz 2024-07-17T08:50:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109565.log 2024-07-17T08:50:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101830.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101830.log.gz 2024-07-17T08:50:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97012.log 2024-07-17T08:50:09.584 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45385.log 2024-07-17T08:50:09.584 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.109565.log.gz 2024-07-17T08:50:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105074.log 2024-07-17T08:50:09.585 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97012.log.gz 2024-07-17T08:50:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43593.log 2024-07-17T08:50:09.587 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45385.log.gz 2024-07-17T08:50:09.587 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.105074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105074.log.gz 2024-07-17T08:50:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49095.log 2024-07-17T08:50:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50503.log 2024-07-17T08:50:09.590 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43593.log.gz 2024-07-17T08:50:09.590 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49095.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49095.log.gz 2024-07-17T08:50:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52252.log 2024-07-17T08:50:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94231.log 2024-07-17T08:50:09.592 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50503.log.gz 2024-07-17T08:50:09.593 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63913.log 2024-07-17T08:50:09.594 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52252.log.gz 2024-07-17T08:50:09.595 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94231.log: gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94231.log.gz 2024-07-17T08:50:09.595 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2024-07-17T08:50:09.595 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63913.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63913.log.gz 2024-07-17T08:50:09.596 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117741.log 2024-07-17T08:50:09.597 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63661.log 2024-07-17T08:50:09.598 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2024-07-17T08:50:09.599 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90465.log 2024-07-17T08:50:09.599 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117741.log.gz 2024-07-17T08:50:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63661.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63661.log.gz 2024-07-17T08:50:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120072.log 2024-07-17T08:50:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90465.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52606.log 2024-07-17T08:50:09.601 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90465.log.gz 2024-07-17T08:50:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120072.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.120072.log.gz /var/log/ceph/ceph-client.admin.72500.log 2024-07-17T08:50:09.603 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:09.604 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92386.log 2024-07-17T08:50:09.604 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52606.log.gz 2024-07-17T08:50:09.605 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83844.log 2024-07-17T08:50:09.605 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72500.log.gz 2024-07-17T08:50:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92386.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92386.log.gz 2024-07-17T08:50:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-07-17T08:50:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83844.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83844.log.gz 2024-07-17T08:50:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2024-07-17T08:50:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95453.log 2024-07-17T08:50:09.609 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-07-17T08:50:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43425.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2024-07-17T08:50:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95453.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95453.log.gz 2024-07-17T08:50:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93610.log 2024-07-17T08:50:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65605.log 2024-07-17T08:50:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82520.log 2024-07-17T08:50:09.615 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93610.log.gz 2024-07-17T08:50:09.616 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66810.log 2024-07-17T08:50:09.616 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.65605.log.gz 2024-07-17T08:50:09.617 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108968.log 2024-07-17T08:50:09.618 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82520.log.gz 2024-07-17T08:50:09.619 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103501.log 2024-07-17T08:50:09.619 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66810.log.gz 2024-07-17T08:50:09.620 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87250.log 2024-07-17T08:50:09.620 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108968.log.gz 2024-07-17T08:50:09.621 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103501.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103501.log.gz 2024-07-17T08:50:09.623 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87250.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87250.log.gz 2024-07-17T08:50:09.623 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50621.log 2024-07-17T08:50:09.624 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96508.log 2024-07-17T08:50:09.625 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110786.log 2024-07-17T08:50:09.625 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50621.log.gz 2024-07-17T08:50:09.626 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96508.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.96508.log.gz 2024-07-17T08:50:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92642.log 2024-07-17T08:50:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110786.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110786.log.gz 2024-07-17T08:50:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66032.log 2024-07-17T08:50:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92642.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92642.log.gz 2024-07-17T08:50:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65258.log 2024-07-17T08:50:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-07-17T08:50:09.632 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66032.log.gz 2024-07-17T08:50:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67705.log 2024-07-17T08:50:09.633 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65258.log.gz 2024-07-17T08:50:09.634 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98228.log 2024-07-17T08:50:09.634 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-07-17T08:50:09.635 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95831.log 2024-07-17T08:50:09.635 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67705.log.gz 2024-07-17T08:50:09.636 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106161.log 2024-07-17T08:50:09.636 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.98228.log.gz 2024-07-17T08:50:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95831.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.95831.log.gz 2024-07-17T08:50:09.638 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101658.log 2024-07-17T08:50:09.639 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106161.log.gz 2024-07-17T08:50:09.639 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116464.log 2024-07-17T08:50:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119518.log 2024-07-17T08:50:09.641 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101658.log.gz 2024-07-17T08:50:09.642 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116464.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81932.log 2024-07-17T08:50:09.643 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116464.log.gz 2024-07-17T08:50:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119518.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119518.log.gz 2024-07-17T08:50:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98186.log 2024-07-17T08:50:09.645 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55210.log 2024-07-17T08:50:09.645 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81932.log.gz 2024-07-17T08:50:09.646 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69434.log 2024-07-17T08:50:09.646 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98186.log.gz 2024-07-17T08:50:09.647 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106711.log 2024-07-17T08:50:09.649 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55210.log.gz 2024-07-17T08:50:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120662.log 2024-07-17T08:50:09.650 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69434.log.gz 2024-07-17T08:50:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106711.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106711.log.gz 2024-07-17T08:50:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122493.log 2024-07-17T08:50:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120662.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104438.log 2024-07-17T08:50:09.652 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120662.log.gz 2024-07-17T08:50:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122493.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122493.log.gz 2024-07-17T08:50:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117306.log 2024-07-17T08:50:09.654 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104438.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104438.log.gz 2024-07-17T08:50:09.654 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120494.log 2024-07-17T08:50:09.655 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57731.log 2024-07-17T08:50:09.656 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117306.log.gz 2024-07-17T08:50:09.657 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-07-17T08:50:09.657 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120494.log.gz 2024-07-17T08:50:09.658 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61383.log 2024-07-17T08:50:09.658 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57731.log.gz 2024-07-17T08:50:09.659 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2024-07-17T08:50:09.659 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-07-17T08:50:09.660 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99402.log 2024-07-17T08:50:09.661 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.61383.log.gz 2024-07-17T08:50:09.662 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99972.log 2024-07-17T08:50:09.662 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2024-07-17T08:50:09.663 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75173.log 2024-07-17T08:50:09.663 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99402.log.gz 2024-07-17T08:50:09.664 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2024-07-17T08:50:09.665 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99972.log.gz 2024-07-17T08:50:09.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2024-07-17T08:50:09.666 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75173.log.gz 2024-07-17T08:50:09.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45106.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-07-17T08:50:09.667 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87861.log 2024-07-17T08:50:09.668 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2024-07-17T08:50:09.668 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85655.log 2024-07-17T08:50:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102941.log 2024-07-17T08:50:09.670 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87861.log.gz 2024-07-17T08:50:09.671 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82562.log 2024-07-17T08:50:09.671 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85655.log.gz 2024-07-17T08:50:09.672 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106413.log 2024-07-17T08:50:09.672 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102941.log.gz 2024-07-17T08:50:09.673 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52547.log 2024-07-17T08:50:09.674 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82562.log.gz 2024-07-17T08:50:09.675 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110339.log 2024-07-17T08:50:09.675 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106413.log.gz 2024-07-17T08:50:09.676 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66918.log 2024-07-17T08:50:09.676 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52547.log.gz 2024-07-17T08:50:09.677 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110339.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110339.log.gz 2024-07-17T08:50:09.678 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.115727.log 2024-07-17T08:50:09.678 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66918.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66918.log.gz 2024-07-17T08:50:09.679 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-07-17T08:50:09.680 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63997.log 2024-07-17T08:50:09.680 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115727.log.gz 2024-07-17T08:50:09.681 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107179.log 2024-07-17T08:50:09.682 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-07-17T08:50:09.683 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85466.log 2024-07-17T08:50:09.683 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63997.log.gz 2024-07-17T08:50:09.684 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107179.log: gzip -5 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107179.log.gz 2024-07-17T08:50:09.684 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.120238.log 2024-07-17T08:50:09.685 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85466.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85466.log.gz 2024-07-17T08:50:09.686 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80361.log 2024-07-17T08:50:09.687 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48787.log 2024-07-17T08:50:09.687 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120238.log.gz 2024-07-17T08:50:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90796.log 2024-07-17T08:50:09.689 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80361.log.gz 2024-07-17T08:50:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101999.log 2024-07-17T08:50:09.690 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48787.log.gz 2024-07-17T08:50:09.691 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-07-17T08:50:09.691 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90796.log.gz 2024-07-17T08:50:09.692 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100842.log 2024-07-17T08:50:09.692 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101999.log.gz 2024-07-17T08:50:09.694 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97598.log 2024-07-17T08:50:09.694 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-07-17T08:50:09.695 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100842.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96904.log 2024-07-17T08:50:09.695 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100842.log.gz 2024-07-17T08:50:09.696 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118797.log 2024-07-17T08:50:09.696 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97598.log.gz 2024-07-17T08:50:09.697 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120746.log 2024-07-17T08:50:09.697 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96904.log.gz 2024-07-17T08:50:09.698 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63434.log 2024-07-17T08:50:09.699 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.118797.log.gz 2024-07-17T08:50:09.700 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120746.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.120746.log.gz /var/log/ceph/ceph-client.admin.92954.log 2024-07-17T08:50:09.700 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:09.701 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107601.log 2024-07-17T08:50:09.701 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63434.log.gz 2024-07-17T08:50:09.702 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67924.log 2024-07-17T08:50:09.703 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.92954.log.gz 2024-07-17T08:50:09.703 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.107601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107601.log.gz 2024-07-17T08:50:09.704 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106016.log 2024-07-17T08:50:09.705 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73474.log 2024-07-17T08:50:09.705 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67924.log.gz 2024-07-17T08:50:09.706 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61763.log 2024-07-17T08:50:09.706 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106016.log.gz 2024-07-17T08:50:09.707 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70508.log 2024-07-17T08:50:09.708 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73474.log.gz 2024-07-17T08:50:09.708 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.61763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61763.log.gz 2024-07-17T08:50:09.709 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98967.log 2024-07-17T08:50:09.710 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113588.log 2024-07-17T08:50:09.710 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70508.log.gz 2024-07-17T08:50:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98967.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98967.log.gz 2024-07-17T08:50:09.711 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81827.log 2024-07-17T08:50:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106753.log 2024-07-17T08:50:09.713 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113588.log.gz 2024-07-17T08:50:09.713 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81827.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81827.log.gz 2024-07-17T08:50:09.714 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49698.log 2024-07-17T08:50:09.715 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106753.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106753.log.gz 2024-07-17T08:50:09.715 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73800.log 2024-07-17T08:50:09.717 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99276.log 2024-07-17T08:50:09.717 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49698.log.gz 2024-07-17T08:50:09.718 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73800.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98060.log 2024-07-17T08:50:09.718 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73800.log.gz 2024-07-17T08:50:09.719 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99276.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-07-17T08:50:09.719 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99276.log.gz 2024-07-17T08:50:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98060.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98060.log.gz 2024-07-17T08:50:09.721 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114479.log 2024-07-17T08:50:09.722 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111838.log 2024-07-17T08:50:09.722 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-07-17T08:50:09.723 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120410.log 2024-07-17T08:50:09.723 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114479.log.gz 2024-07-17T08:50:09.724 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82699.log 2024-07-17T08:50:09.724 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111838.log.gz 2024-07-17T08:50:09.725 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120410.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118293.log 2024-07-17T08:50:09.726 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120410.log.gz 2024-07-17T08:50:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82699.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82699.log.gz 2024-07-17T08:50:09.727 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64255.log 2024-07-17T08:50:09.728 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.118293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118293.log.gz 2024-07-17T08:50:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73926.log 2024-07-17T08:50:09.729 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.64255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64255.log.gz 2024-07-17T08:50:09.730 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95789.log 2024-07-17T08:50:09.731 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109919.log 2024-07-17T08:50:09.731 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73926.log.gz 2024-07-17T08:50:09.732 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121060.log 87.3% 2024-07-17T08:50:09.733 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95789.log.gz 2024-07-17T08:50:09.734 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116903.log 2024-07-17T08:50:09.734 INFO:teuthology.orchestra.run.smithi044.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.109919.log.gz 2024-07-17T08:50:09.735 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.121060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121060.log.gz 2024-07-17T08:50:09.735 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68187.log 2024-07-17T08:50:09.737 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117429.log 2024-07-17T08:50:09.737 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116903.log.gz 2024-07-17T08:50:09.738 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56730.log 2024-07-17T08:50:09.738 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68187.log.gz 2024-07-17T08:50:09.739 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85968.log 2024-07-17T08:50:09.739 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117429.log.gz 2024-07-17T08:50:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56730.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56730.log.gz 2024-07-17T08:50:09.740 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80449.log 2024-07-17T08:50:09.742 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2024-07-17T08:50:09.742 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.85968.log.gz 2024-07-17T08:50:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80449.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80449.log.gz 2024-07-17T08:50:09.743 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110625.log 2024-07-17T08:50:09.745 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119687.log 2024-07-17T08:50:09.746 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz/var/log/ceph/ceph-client.admin.110625.log: 2024-07-17T08:50:09.746 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110625.log.gz 2024-07-17T08:50:09.746 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112103.log 2024-07-17T08:50:09.747 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119687.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119687.log.gz 2024-07-17T08:50:09.747 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70863.log 2024-07-17T08:50:09.749 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85571.log 2024-07-17T08:50:09.749 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.112103.log.gz 2024-07-17T08:50:09.750 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51983.log 2024-07-17T08:50:09.750 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70863.log.gz 2024-07-17T08:50:09.751 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101434.log 2024-07-17T08:50:09.751 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85571.log.gz 2024-07-17T08:50:09.753 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122003.log 2024-07-17T08:50:09.753 INFO:teuthology.orchestra.run.smithi044.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.51983.log.gz 2024-07-17T08:50:09.754 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115180.log 2024-07-17T08:50:09.754 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101434.log.gz 2024-07-17T08:50:09.755 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120153.log 2024-07-17T08:50:09.755 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122003.log.gz 2024-07-17T08:50:09.757 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-07-17T08:50:09.757 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115180.log.gz 2024-07-17T08:50:09.757 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120153.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120153.log.gz 2024-07-17T08:50:09.758 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82989.log 2024-07-17T08:50:09.759 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42877.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81590.log 2024-07-17T08:50:09.759 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-07-17T08:50:09.760 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103249.log 2024-07-17T08:50:09.760 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82989.log.gz 2024-07-17T08:50:09.761 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81590.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81590.log.gz 2024-07-17T08:50:09.762 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121835.log 2024-07-17T08:50:09.762 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103249.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103249.log.gz 2024-07-17T08:50:09.763 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86437.log 2024-07-17T08:50:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87100.log 2024-07-17T08:50:09.765 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.121835.log.gz 2024-07-17T08:50:09.765 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50137.log 2024-07-17T08:50:09.766 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86437.log.gz 2024-07-17T08:50:09.767 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91827.log 2024-07-17T08:50:09.767 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87100.log.gz 2024-07-17T08:50:09.768 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68390.log 2024-07-17T08:50:09.768 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50137.log.gz 2024-07-17T08:50:09.769 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103291.log 2024-07-17T08:50:09.770 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.91827.log.gz 2024-07-17T08:50:09.771 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80619.log 2024-07-17T08:50:09.771 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68390.log.gz 2024-07-17T08:50:09.772 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103291.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103291.log.gz 2024-07-17T08:50:09.772 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111311.log 2024-07-17T08:50:09.773 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80619.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80619.log.gz 2024-07-17T08:50:09.773 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-07-17T08:50:09.775 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94856.log 2024-07-17T08:50:09.775 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111311.log.gz 2024-07-17T08:50:09.776 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110255.log 2024-07-17T08:50:09.776 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-07-17T08:50:09.777 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95281.log 2024-07-17T08:50:09.777 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94856.log.gz 2024-07-17T08:50:09.778 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60700.log 2024-07-17T08:50:09.779 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110255.log.gz 2024-07-17T08:50:09.779 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95281.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95281.log.gz 2024-07-17T08:50:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100799.log 2024-07-17T08:50:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60700.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60700.log.gz 2024-07-17T08:50:09.781 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-07-17T08:50:09.782 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61467.log 2024-07-17T08:50:09.783 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.100799.log.gz 2024-07-17T08:50:09.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52429.log 2024-07-17T08:50:09.784 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-07-17T08:50:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61467.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61467.log.gz 2024-07-17T08:50:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-07-17T08:50:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52429.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52429.log.gz 2024-07-17T08:50:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90213.log 2024-07-17T08:50:09.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83466.log 2024-07-17T08:50:09.788 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-07-17T08:50:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90213.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90213.log.gz 2024-07-17T08:50:09.789 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87442.log 2024-07-17T08:50:09.790 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83466.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83466.log.gz 2024-07-17T08:50:09.791 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48621.log 2024-07-17T08:50:09.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.87442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87442.log.gz 2024-07-17T08:50:09.792 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69350.log 2024-07-17T08:50:09.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43127.log 2024-07-17T08:50:09.793 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48621.log.gz 2024-07-17T08:50:09.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-07-17T08:50:09.795 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69350.log.gz 2024-07-17T08:50:09.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43127.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43127.log.gz 2024-07-17T08:50:09.796 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49278.log 2024-07-17T08:50:09.797 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47327.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117560.log 2024-07-17T08:50:09.797 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-07-17T08:50:09.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49278.log: gzip 86.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.49278.log.gz --verbose 2024-07-17T08:50:09.799 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.47411.log 2024-07-17T08:50:09.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97682.log 2024-07-17T08:50:09.800 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.117560.log.gz 2024-07-17T08:50:09.801 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47411.log: gzip 86.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz --verbose 2024-07-17T08:50:09.801 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.56221.log 2024-07-17T08:50:09.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66288.log 2024-07-17T08:50:09.802 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97682.log.gz 2024-07-17T08:50:09.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46239.log 2024-07-17T08:50:09.804 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56221.log.gz 2024-07-17T08:50:09.805 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80787.log 2024-07-17T08:50:09.805 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66288.log.gz 2024-07-17T08:50:09.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61173.log 2024-07-17T08:50:09.807 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0%/var/log/ceph/ceph-client.admin.80787.log: -- replaced with /var/log/ceph/ceph-client.admin.46239.log.gz 2024-07-17T08:50:09.807 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80787.log.gz 2024-07-17T08:50:09.807 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-07-17T08:50:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61173.log: gzip 86.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.61173.log.gz --verbose 2024-07-17T08:50:09.809 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.84898.log 2024-07-17T08:50:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66690.log 2024-07-17T08:50:09.810 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-07-17T08:50:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92335.log 2024-07-17T08:50:09.811 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84898.log.gz 2024-07-17T08:50:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.66690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84178.log 2024-07-17T08:50:09.813 INFO:teuthology.orchestra.run.smithi044.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66690.log.gz 2024-07-17T08:50:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92335.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69612.log 2024-07-17T08:50:09.814 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92335.log.gz 2024-07-17T08:50:09.815 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86010.log 2024-07-17T08:50:09.816 INFO:teuthology.orchestra.run.smithi044.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.84178.log.gz 2024-07-17T08:50:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-07-17T08:50:09.817 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69612.log.gz 2024-07-17T08:50:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75553.log 2024-07-17T08:50:09.818 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.86010.log.gz 2024-07-17T08:50:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105764.log 2024-07-17T08:50:09.819 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-07-17T08:50:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.75553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75553.log.gz 2024-07-17T08:50:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44793.log 2024-07-17T08:50:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67527.log 2024-07-17T08:50:09.822 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105764.log.gz 2024-07-17T08:50:09.823 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116422.log 2024-07-17T08:50:09.823 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44793.log.gz 2024-07-17T08:50:09.824 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100697.log 2024-07-17T08:50:09.824 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67527.log.gz 2024-07-17T08:50:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-07-17T08:50:09.826 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116422.log.gz 2024-07-17T08:50:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100697.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100697.log.gz 2024-07-17T08:50:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97808.log 2024-07-17T08:50:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47537.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93876.log 2024-07-17T08:50:09.828 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-07-17T08:50:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92205.log 2024-07-17T08:50:09.830 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.97808.log.gz 2024-07-17T08:50:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103417.log 2024-07-17T08:50:09.831 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93876.log.gz 2024-07-17T08:50:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-07-17T08:50:09.832 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92205.log.gz 2024-07-17T08:50:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106287.log 2024-07-17T08:50:09.833 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103417.log.gz 2024-07-17T08:50:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61034.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-07-17T08:50:09.835 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.74365.log 2024-07-17T08:50:09.836 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106287.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106287.log.gz 2024-07-17T08:50:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56137.log 2024-07-17T08:50:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74365.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74365.log.gz 2024-07-17T08:50:09.837 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96778.log 2024-07-17T08:50:09.839 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104171.log 2024-07-17T08:50:09.839 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56137.log.gz 2024-07-17T08:50:09.840 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37903.log 2024-07-17T08:50:09.841 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96778.log.gz 2024-07-17T08:50:09.842 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63008.log 2024-07-17T08:50:09.842 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104171.log.gz 2024-07-17T08:50:09.842 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37903.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37903.log.gz 2024-07-17T08:50:09.843 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110173.log 2024-07-17T08:50:09.844 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114005.log 2024-07-17T08:50:09.844 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz 2024-07-17T08:50:09.845 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.110173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110173.log.gz 2024-07-17T08:50:09.846 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109395.log 2024-07-17T08:50:09.846 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114005.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.114005.log.gz 2024-07-17T08:50:09.847 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66330.log 2024-07-17T08:50:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63188.log 2024-07-17T08:50:09.848 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.109395.log.gz 2024-07-17T08:50:09.849 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66330.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.66330.log.gz 2024-07-17T08:50:09.850 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100127.log 2024-07-17T08:50:09.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63188.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64417.log 2024-07-17T08:50:09.851 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63188.log.gz 2024-07-17T08:50:09.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60251.log 2024-07-17T08:50:09.852 INFO:teuthology.orchestra.run.smithi044.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.100127.log.gz 2024-07-17T08:50:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45987.log 2024-07-17T08:50:09.854 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64417.log.gz 2024-07-17T08:50:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47579.log 2024-07-17T08:50:09.855 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60251.log.gz 2024-07-17T08:50:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49530.log 2024-07-17T08:50:09.856 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45987.log.gz 2024-07-17T08:50:09.857 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47579.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47579.log.gz 2024-07-17T08:50:09.857 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64333.log 2024-07-17T08:50:09.858 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49530.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49530.log.gz 2024-07-17T08:50:09.859 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122129.log 2024-07-17T08:50:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70550.log 2024-07-17T08:50:09.860 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64333.log.gz 2024-07-17T08:50:09.861 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61425.log 2024-07-17T08:50:09.861 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122129.log.gz 2024-07-17T08:50:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97232.log 2024-07-17T08:50:09.863 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70550.log.gz 2024-07-17T08:50:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61425.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61425.log.gz 2024-07-17T08:50:09.864 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113945.log 2024-07-17T08:50:09.865 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97232.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.97232.log.gz 2024-07-17T08:50:09.865 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108233.log 2024-07-17T08:50:09.867 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104610.log 2024-07-17T08:50:09.867 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.113945.log.gz 2024-07-17T08:50:09.868 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98144.log 2024-07-17T08:50:09.868 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108233.log.gz 2024-07-17T08:50:09.869 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92726.log 2024-07-17T08:50:09.869 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104610.log.gz 2024-07-17T08:50:09.870 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104526.log 2024-07-17T08:50:09.871 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.98144.log.gz 2024-07-17T08:50:09.872 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92726.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.84667.log 2024-07-17T08:50:09.872 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92726.log.gz 2024-07-17T08:50:09.873 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-07-17T08:50:09.873 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104526.log.gz 2024-07-17T08:50:09.874 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84667.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-07-17T08:50:09.874 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84667.log.gz 2024-07-17T08:50:09.876 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115096.log 2024-07-17T08:50:09.876 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gz 2024-07-17T08:50:09.877 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.46403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86848.log 2024-07-17T08:50:09.877 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-07-17T08:50:09.878 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83033.log 2024-07-17T08:50:09.878 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115096.log.gz 2024-07-17T08:50:09.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99104.log 2024-07-17T08:50:09.880 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86848.log.gz 2024-07-17T08:50:09.881 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99744.log 2024-07-17T08:50:09.881 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.83033.log.gz 2024-07-17T08:50:09.882 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109733.log 2024-07-17T08:50:09.882 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99104.log.gz 2024-07-17T08:50:09.883 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.99744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99744.log.gz 2024-07-17T08:50:09.883 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71697.log 2024-07-17T08:50:09.884 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91575.log 2024-07-17T08:50:09.884 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.109733.log.gz 2024-07-17T08:50:09.886 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71697.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71697.log.gz 2024-07-17T08:50:09.886 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103730.log 2024-07-17T08:50:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91575.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91575.log.gz 2024-07-17T08:50:09.887 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103874.log 2024-07-17T08:50:09.889 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66246.log 2024-07-17T08:50:09.889 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103730.log.gz 2024-07-17T08:50:09.890 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97514.log 2024-07-17T08:50:09.890 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103874.log.gz 2024-07-17T08:50:09.891 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43551.log 2024-07-17T08:50:09.891 INFO:teuthology.orchestra.run.smithi044.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.66246.log.gz 2024-07-17T08:50:09.892 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114342.log 2024-07-17T08:50:09.892 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97514.log.gz 2024-07-17T08:50:09.893 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91701.log 2024-07-17T08:50:09.894 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43551.log.gz 2024-07-17T08:50:09.895 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114342.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102504.log 2024-07-17T08:50:09.895 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114342.log.gz 2024-07-17T08:50:09.896 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70241.log 2024-07-17T08:50:09.896 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91701.log.gz 2024-07-17T08:50:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60902.log 2024-07-17T08:50:09.898 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102504.log.gz 2024-07-17T08:50:09.899 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99062.log 2024-07-17T08:50:09.899 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70241.log.gz 2024-07-17T08:50:09.900 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56999.log 2024-07-17T08:50:09.900 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60902.log.gz 2024-07-17T08:50:09.901 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98396.log 2024-07-17T08:50:09.901 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99062.log.gz 2024-07-17T08:50:09.903 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82478.log 2024-07-17T08:50:09.903 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56999.log.gz 2024-07-17T08:50:09.904 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.98396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2024-07-17T08:50:09.904 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98396.log.gz 2024-07-17T08:50:09.905 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82478.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82478.log.gz 2024-07-17T08:50:09.905 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121961.log 2024-07-17T08:50:09.906 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49656.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91097.log 2024-07-17T08:50:09.907 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2024-07-17T08:50:09.908 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120368.log 2024-07-17T08:50:09.908 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121961.log.gz 2024-07-17T08:50:09.909 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116777.log 2024-07-17T08:50:09.909 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91097.log.gz 2024-07-17T08:50:09.910 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118898.log 2024-07-17T08:50:09.911 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120368.log.gz 2024-07-17T08:50:09.911 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116777.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116777.log.gz 2024-07-17T08:50:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106455.log 2024-07-17T08:50:09.912 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118898.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.118898.log.gz 2024-07-17T08:50:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119332.log 2024-07-17T08:50:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114300.log 2024-07-17T08:50:09.914 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106455.log.gz 2024-07-17T08:50:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68267.log 2024-07-17T08:50:09.916 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119332.log.gz 2024-07-17T08:50:09.917 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56484.log 2024-07-17T08:50:09.917 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114300.log.gz 2024-07-17T08:50:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68267.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68267.log.gz 2024-07-17T08:50:09.918 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117611.log 2024-07-17T08:50:09.919 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56484.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71953.log 2024-07-17T08:50:09.920 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56484.log.gz 2024-07-17T08:50:09.921 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117472.log 2024-07-17T08:50:09.921 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117611.log.gz 2024-07-17T08:50:09.922 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43509.log 2024-07-17T08:50:09.922 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.71953.log.gz 2024-07-17T08:50:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111521.log 2024-07-17T08:50:09.923 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117472.log.gz 2024-07-17T08:50:09.924 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121667.log 2024-07-17T08:50:09.925 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43509.log.gz 2024-07-17T08:50:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37934.log 2024-07-17T08:50:09.926 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111521.log.gz 2024-07-17T08:50:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93062.log 2024-07-17T08:50:09.927 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121667.log.gz 2024-07-17T08:50:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37934.log.gz 2024-07-17T08:50:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79668.log 2024-07-17T08:50:09.929 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93062.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.93062.log.gz 2024-07-17T08:50:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91219.log 2024-07-17T08:50:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60860.log 2024-07-17T08:50:09.931 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79668.log.gz 2024-07-17T08:50:09.932 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80239.log 2024-07-17T08:50:09.932 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91219.log.gz 2024-07-17T08:50:09.934 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93729.log 2024-07-17T08:50:09.934 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60860.log.gz 2024-07-17T08:50:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51042.log 2024-07-17T08:50:09.935 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80239.log.gz 2024-07-17T08:50:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84625.log 2024-07-17T08:50:09.936 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93729.log.gz 2024-07-17T08:50:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62756.log 2024-07-17T08:50:09.937 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51042.log.gz 2024-07-17T08:50:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84625.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.84625.log.gz 2024-07-17T08:50:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109775.log 2024-07-17T08:50:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62756.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62756.log.gz 2024-07-17T08:50:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105890.log 2024-07-17T08:50:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86052.log 2024-07-17T08:50:09.942 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.109775.log.gz 2024-07-17T08:50:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87670.log 2024-07-17T08:50:09.943 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105890.log.gz 2024-07-17T08:50:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86052.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.86052.log.gz 2024-07-17T08:50:09.944 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65685.log 2024-07-17T08:50:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87670.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87670.log.gz 2024-07-17T08:50:09.945 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68092.log 2024-07-17T08:50:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65685.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68008.log 2024-07-17T08:50:09.947 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65685.log.gz 2024-07-17T08:50:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68092.log: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116819.log 2024-07-17T08:50:09.948 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68092.log.gz 2024-07-17T08:50:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68008.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.68008.log.gz 2024-07-17T08:50:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91533.log 2024-07-17T08:50:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116819.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116819.log.gz 2024-07-17T08:50:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94730.log 2024-07-17T08:50:09.953 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56818.log 2024-07-17T08:50:09.953 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91533.log.gz 2024-07-17T08:50:09.954 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60064.log 2024-07-17T08:50:09.954 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.94730.log.gz 2024-07-17T08:50:09.955 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96550.log 2024-07-17T08:50:09.955 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.56818.log.gz 2024-07-17T08:50:09.957 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121541.log 2024-07-17T08:50:09.957 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60064.log.gz 2024-07-17T08:50:09.958 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104011.log 2024-07-17T08:50:09.958 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96550.log.gz 2024-07-17T08:50:09.959 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121541.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.121541.log.gz 2024-07-17T08:50:09.959 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79836.log 2024-07-17T08:50:09.960 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104011.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104011.log.gz 2024-07-17T08:50:09.960 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62443.log 2024-07-17T08:50:09.962 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94604.log 2024-07-17T08:50:09.962 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79836.log.gz 2024-07-17T08:50:09.963 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62443.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62443.log.gz 2024-07-17T08:50:09.963 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121583.log 2024-07-17T08:50:09.964 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94604.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94604.log.gz 2024-07-17T08:50:09.964 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91893.log 2024-07-17T08:50:09.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121583.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-07-17T08:50:09.966 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121583.log.gz 2024-07-17T08:50:09.967 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91893.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91893.log.gz 2024-07-17T08:50:09.967 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80913.log 2024-07-17T08:50:09.968 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55698.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-07-17T08:50:09.968 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96820.log 2024-07-17T08:50:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60545.log 2024-07-17T08:50:09.970 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80913.log.gz 2024-07-17T08:50:09.971 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54140.log 2024-07-17T08:50:09.971 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96820.log.gz 2024-07-17T08:50:09.972 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102620.log 2024-07-17T08:50:09.972 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60545.log.gz 2024-07-17T08:50:09.973 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54140.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54140.log.gz 2024-07-17T08:50:09.974 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121751.log 2024-07-17T08:50:09.975 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102620.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99444.log 2024-07-17T08:50:09.975 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102620.log.gz 2024-07-17T08:50:09.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49446.log 2024-07-17T08:50:09.977 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121751.log.gz 2024-07-17T08:50:09.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65384.log 2024-07-17T08:50:09.978 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.99444.log.gz 2024-07-17T08:50:09.979 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114132.log 2024-07-17T08:50:09.979 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49446.log.gz 2024-07-17T08:50:09.980 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82100.log 2024-07-17T08:50:09.980 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65384.log.gz 2024-07-17T08:50:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72990.log 2024-07-17T08:50:09.981 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114132.log.gz 2024-07-17T08:50:09.982 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57377.log 2024-07-17T08:50:09.982 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82100.log.gz 2024-07-17T08:50:09.984 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59791.log 2024-07-17T08:50:09.984 INFO:teuthology.orchestra.run.smithi044.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.72990.log.gz 2024-07-17T08:50:09.985 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-07-17T08:50:09.985 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106119.log 2024-07-17T08:50:09.986 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57377.log.gz 2024-07-17T08:50:09.987 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.59791.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59791.log.gz 2024-07-17T08:50:09.987 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.111563.log 2024-07-17T08:50:09.987 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106119.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106119.log.gz 2024-07-17T08:50:09.988 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67092.log 2024-07-17T08:50:09.989 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86647.log 2024-07-17T08:50:09.989 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111563.log.gz 2024-07-17T08:50:09.990 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-07-17T08:50:09.990 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67092.log.gz 2024-07-17T08:50:09.991 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-07-17T08:50:09.992 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86647.log.gz 2024-07-17T08:50:09.993 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117005.log 2024-07-17T08:50:09.993 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-07-17T08:50:09.994 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52193.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-07-17T08:50:09.994 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107834.log 2024-07-17T08:50:09.995 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117005.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122087.log 2024-07-17T08:50:09.995 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117005.log.gz 2024-07-17T08:50:09.997 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69570.log 2024-07-17T08:50:09.997 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107834.log.gz 2024-07-17T08:50:09.998 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122087.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122087.log.gz 2024-07-17T08:50:09.998 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.107989.log 2024-07-17T08:50:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.69570.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69570.log.gz 2024-07-17T08:50:09.999 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-07-17T08:50:10.001 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109830.log 2024-07-17T08:50:10.001 INFO:teuthology.orchestra.run.smithi044.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.107989.log.gz 2024-07-17T08:50:10.002 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102570.log 2024-07-17T08:50:10.002 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-07-17T08:50:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109830.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.109830.log.gz 2024-07-17T08:50:10.003 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62611.log 2024-07-17T08:50:10.005 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102570.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73195.log 2024-07-17T08:50:10.005 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102570.log.gz 2024-07-17T08:50:10.006 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106840.log 2024-07-17T08:50:10.006 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62611.log.gz 2024-07-17T08:50:10.007 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65510.log 2024-07-17T08:50:10.007 INFO:teuthology.orchestra.run.smithi044.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.73195.log.gz 2024-07-17T08:50:10.008 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116963.log 2024-07-17T08:50:10.008 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106840.log.gz 2024-07-17T08:50:10.010 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91351.log 2024-07-17T08:50:10.010 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65510.log.gz 2024-07-17T08:50:10.011 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74749.log 2024-07-17T08:50:10.011 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116963.log.gz 2024-07-17T08:50:10.012 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-07-17T08:50:10.012 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.91351.log.gz 2024-07-17T08:50:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83954.log 2024-07-17T08:50:10.013 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74749.log.gz 2024-07-17T08:50:10.015 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96634.log 2024-07-17T08:50:10.015 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-07-17T08:50:10.016 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48289.log 2024-07-17T08:50:10.016 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83954.log.gz 2024-07-17T08:50:10.017 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110452.log 2024-07-17T08:50:10.017 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96634.log.gz 2024-07-17T08:50:10.018 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92870.log 2024-07-17T08:50:10.018 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48289.log.gz 2024-07-17T08:50:10.019 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.110452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80064.log 2024-07-17T08:50:10.020 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110452.log.gz 2024-07-17T08:50:10.021 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92870.log: gzip -5 --verbose 86.4% -- replaced with /var/log/ceph/ceph-client.admin.92870.log.gz 2024-07-17T08:50:10.021 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.100245.log 2024-07-17T08:50:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80064.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80064.log.gz 2024-07-17T08:50:10.022 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68477.log 2024-07-17T08:50:10.023 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105522.log 2024-07-17T08:50:10.024 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100245.log.gz 2024-07-17T08:50:10.025 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116198.log 2024-07-17T08:50:10.025 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68477.log.gz 2024-07-17T08:50:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105522.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105522.log.gz 2024-07-17T08:50:10.026 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96466.log 2024-07-17T08:50:10.027 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116198.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75915.log 2024-07-17T08:50:10.027 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116198.log.gz 2024-07-17T08:50:10.029 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84957.log 2024-07-17T08:50:10.029 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96466.log.gz 2024-07-17T08:50:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75915.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.75915.log.gz 2024-07-17T08:50:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45389.log 2024-07-17T08:50:10.031 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83214.log 2024-07-17T08:50:10.031 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.84957.log.gz 2024-07-17T08:50:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80577.log 2024-07-17T08:50:10.032 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45389.log.gz 2024-07-17T08:50:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118573.log 2024-07-17T08:50:10.034 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83214.log.gz 2024-07-17T08:50:10.034 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80577.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80577.log.gz 2024-07-17T08:50:10.035 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102042.log 2024-07-17T08:50:10.036 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118573.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73031.log 2024-07-17T08:50:10.036 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118573.log.gz 2024-07-17T08:50:10.037 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2024-07-17T08:50:10.038 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102042.log.gz 2024-07-17T08:50:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73031.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.73031.log.gz 2024-07-17T08:50:10.039 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47663.log 2024-07-17T08:50:10.040 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48373.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2024-07-17T08:50:10.040 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84136.log 2024-07-17T08:50:10.042 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.47663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82779.log 2024-07-17T08:50:10.042 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47663.log.gz 2024-07-17T08:50:10.043 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.84136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111880.log 2024-07-17T08:50:10.043 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.84136.log.gz 2024-07-17T08:50:10.044 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110405.log 2024-07-17T08:50:10.044 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82779.log.gz 2024-07-17T08:50:10.045 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111880.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.111880.log.gz 2024-07-17T08:50:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.110405.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83382.log 2024-07-17T08:50:10.047 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110405.log.gz 2024-07-17T08:50:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102821.log 2024-07-17T08:50:10.049 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.83382.log: gzip -5 --verbose 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83382.log.gz 2024-07-17T08:50:10.049 INFO:teuthology.orchestra.run.smithi044.stderr: -- /var/log/ceph/ceph-client.admin.67798.log 2024-07-17T08:50:10.050 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102821.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102821.log.gz 2024-07-17T08:50:10.051 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78096.log 2024-07-17T08:50:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55824.log 2024-07-17T08:50:10.052 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67798.log.gz 2024-07-17T08:50:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.78096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86974.log 2024-07-17T08:50:10.053 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.78096.log.gz 2024-07-17T08:50:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106796.log 2024-07-17T08:50:10.054 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55824.log.gz 2024-07-17T08:50:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74583.log 2024-07-17T08:50:10.056 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86974.log.gz 2024-07-17T08:50:10.057 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111922.log 2024-07-17T08:50:10.057 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106796.log.gz 2024-07-17T08:50:10.058 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82901.log 2024-07-17T08:50:10.058 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74583.log.gz 2024-07-17T08:50:10.059 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119416.log 2024-07-17T08:50:10.059 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111922.log.gz 2024-07-17T08:50:10.061 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75635.log 2024-07-17T08:50:10.061 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82901.log.gz 2024-07-17T08:50:10.062 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71157.log 2024-07-17T08:50:10.062 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119416.log.gz 2024-07-17T08:50:10.063 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52724.log 2024-07-17T08:50:10.063 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.75635.log.gz 2024-07-17T08:50:10.064 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63522.log 2024-07-17T08:50:10.064 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71157.log.gz 2024-07-17T08:50:10.065 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74879.log 2024-07-17T08:50:10.065 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52724.log.gz 2024-07-17T08:50:10.067 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79878.log 2024-07-17T08:50:10.067 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63522.log.gz 2024-07-17T08:50:10.068 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74879.log: gzip -5 --verbose 87.0% -- /var/log/ceph/ceph-client.admin.60503.log -- replaced with /var/log/ceph/ceph-client.admin.74879.log.gz 2024-07-17T08:50:10.068 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:10.069 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103604.log 2024-07-17T08:50:10.069 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79878.log.gz 2024-07-17T08:50:10.070 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99906.log 2024-07-17T08:50:10.071 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60503.log.gz 2024-07-17T08:50:10.072 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103604.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.103604.log.gz /var/log/ceph/ceph-client.0.83766.log 2024-07-17T08:50:10.072 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:10.073 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65174.log 2024-07-17T08:50:10.073 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99906.log.gz 2024-07-17T08:50:10.074 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.83766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83766.log.gz 2024-07-17T08:50:10.074 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62882.log 2024-07-17T08:50:10.075 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111395.log 2024-07-17T08:50:10.076 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65174.log.gz 2024-07-17T08:50:10.077 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67134.log 2024-07-17T08:50:10.077 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62882.log.gz 2024-07-17T08:50:10.078 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94399.log 2024-07-17T08:50:10.078 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111395.log.gz 2024-07-17T08:50:10.079 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74197.log 2024-07-17T08:50:10.079 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67134.log.gz 2024-07-17T08:50:10.080 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118940.log 2024-07-17T08:50:10.081 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94399.log.gz 2024-07-17T08:50:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86220.log 2024-07-17T08:50:10.082 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74197.log.gz 2024-07-17T08:50:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86806.log 2024-07-17T08:50:10.083 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118940.log.gz 2024-07-17T08:50:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72931.log 2024-07-17T08:50:10.084 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86220.log.gz 2024-07-17T08:50:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100884.log 2024-07-17T08:50:10.086 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86806.log.gz 2024-07-17T08:50:10.086 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101742.log 2024-07-17T08:50:10.087 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72931.log.gz 2024-07-17T08:50:10.088 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100884.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100884.log.gz 2024-07-17T08:50:10.088 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.101742.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80319.log 2024-07-17T08:50:10.089 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101742.log.gz 2024-07-17T08:50:10.089 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72813.log 2024-07-17T08:50:10.091 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115972.log 2024-07-17T08:50:10.091 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80319.log.gz 2024-07-17T08:50:10.092 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67882.log 2024-07-17T08:50:10.093 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72813.log.gz 2024-07-17T08:50:10.094 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62527.log 2024-07-17T08:50:10.094 INFO:teuthology.orchestra.run.smithi044.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.115972.log.gz 2024-07-17T08:50:10.095 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67882.log: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43659.log 2024-07-17T08:50:10.095 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67882.log.gz 2024-07-17T08:50:10.096 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62527.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62527.log.gz 2024-07-17T08:50:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96168.log 2024-07-17T08:50:10.097 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43659.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-07-17T08:50:10.098 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102462.log 2024-07-17T08:50:10.100 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96168.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96168.log.gzgzip 2024-07-17T08:50:10.100 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57227.log 2024-07-17T08:50:10.100 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.102462.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102462.log.gz 2024-07-17T08:50:10.101 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105199.log 2024-07-17T08:50:10.102 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75855.log 2024-07-17T08:50:10.102 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57227.log.gz 2024-07-17T08:50:10.103 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105199.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105199.log.gz 2024-07-17T08:50:10.104 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113023.log 2024-07-17T08:50:10.105 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75855.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75855.log.gz 2024-07-17T08:50:10.105 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66414.log 2024-07-17T08:50:10.106 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96210.log 2024-07-17T08:50:10.107 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113023.log.gz 2024-07-17T08:50:10.108 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66414.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.66414.log.gz 2024-07-17T08:50:10.109 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96210.log: gzip -5 87.5% --verbose -- /var/log/ceph/ceph-client.admin.99486.log 2024-07-17T08:50:10.109 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96210.log.gz 2024-07-17T08:50:10.110 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64063.log 2024-07-17T08:50:10.112 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72037.log 2024-07-17T08:50:10.112 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0%/var/log/ceph/ceph-client.admin.64063.log: -- replaced with /var/log/ceph/ceph-client.admin.99486.log.gz 2024-07-17T08:50:10.113 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64063.log.gz 2024-07-17T08:50:10.113 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61257.log 2024-07-17T08:50:10.114 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111658.log 2024-07-17T08:50:10.114 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72037.log.gz 2024-07-17T08:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116114.log 2024-07-17T08:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61257.log.gz 2024-07-17T08:50:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111658.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111658.log.gz 2024-07-17T08:50:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75299.log 2024-07-17T08:50:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116114.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.116114.log.gz 2024-07-17T08:50:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45107.log 2024-07-17T08:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81442.log 2024-07-17T08:50:10.120 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75299.log.gz 2024-07-17T08:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45107.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45107.log.gz 2024-07-17T08:50:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114769.log 2024-07-17T08:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.81442.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81442.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49404.log 2024-07-17T08:50:10.122 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72079.log 2024-07-17T08:50:10.124 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114769.log.gz 2024-07-17T08:50:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.49404.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49404.log.gz 2024-07-17T08:50:10.125 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63871.log 2024-07-17T08:50:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.72079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51857.log 2024-07-17T08:50:10.127 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72079.log.gz 2024-07-17T08:50:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65029.log 2024-07-17T08:50:10.128 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.63871.log.gz 2024-07-17T08:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51857.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.51857.log.gz 2024-07-17T08:50:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.65029.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93240.log 2024-07-17T08:50:10.130 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65029.log.gz 2024-07-17T08:50:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121499.log 2024-07-17T08:50:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112485.log 2024-07-17T08:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93240.log.gz 2024-07-17T08:50:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70148.log 2024-07-17T08:50:10.134 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.121499.log.gz 2024-07-17T08:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97934.log 2024-07-17T08:50:10.135 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.112485.log.gz 2024-07-17T08:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70148.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70148.log.gz 2024-07-17T08:50:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96946.log 2024-07-17T08:50:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97934.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103772.log 2024-07-17T08:50:10.138 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97934.log.gz 2024-07-17T08:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101914.log 2024-07-17T08:50:10.139 INFO:teuthology.orchestra.run.smithi044.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96946.log.gz 2024-07-17T08:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103772.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103772.log.gz 2024-07-17T08:50:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87016.log 2024-07-17T08:50:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101914.log: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120195.log 2024-07-17T08:50:10.142 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101914.log.gz 2024-07-17T08:50:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68348.log 2024-07-17T08:50:10.143 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87016.log.gz 2024-07-17T08:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90838.log 2024-07-17T08:50:10.144 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120195.log.gz 2024-07-17T08:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111185.log 2024-07-17T08:50:10.145 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68348.log.gz 2024-07-17T08:50:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50054.log 2024-07-17T08:50:10.147 INFO:teuthology.orchestra.run.smithi044.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90838.log.gz 2024-07-17T08:50:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92684.log 2024-07-17T08:50:10.148 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111185.log.gz 2024-07-17T08:50:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.50054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50054.log.gz 2024-07-17T08:50:10.149 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91659.log 2024-07-17T08:50:10.150 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108338.log 2024-07-17T08:50:10.150 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92684.log.gz 2024-07-17T08:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91659.log: gzip -5 --verbose -- 87.0% -- replaced with /var/log/ceph/ceph-client.admin.91659.log.gz 2024-07-17T08:50:10.152 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.107947.log 2024-07-17T08:50:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108338.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108338.log.gz 2024-07-17T08:50:10.153 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65468.log 2024-07-17T08:50:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107947.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107947.log.gz 2024-07-17T08:50:10.154 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120452.log 2024-07-17T08:50:10.156 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65468.log: gzip -5 --verbose -- 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65468.log.gz 2024-07-17T08:50:10.156 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.61929.log 2024-07-17T08:50:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120452.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.120452.log.gz -5 2024-07-17T08:50:10.158 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-07-17T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61929.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61929.log.gz 2024-07-17T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.67296.log 2024-07-17T08:50:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 2024-07-17T08:50:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95747.log 2024-07-17T08:50:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71332.log 2024-07-17T08:50:10.162 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67296.log.gz 2024-07-17T08:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119458.log 2024-07-17T08:50:10.163 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.95747.log.gz 2024-07-17T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71332.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71332.log.gz 2024-07-17T08:50:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101273.log 2024-07-17T08:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104568.log 2024-07-17T08:50:10.165 INFO:teuthology.orchestra.run.smithi044.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.119458.log.gz 2024-07-17T08:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105308.log 2024-07-17T08:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101273.log.gz 2024-07-17T08:50:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104568.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104568.log.gz 2024-07-17T08:50:10.168 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63268.log 2024-07-17T08:50:10.169 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105308.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62383.log 2024-07-17T08:50:10.170 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105308.log.gz 2024-07-17T08:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114966.log 2024-07-17T08:50:10.171 INFO:teuthology.orchestra.run.smithi044.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63268.log.gz 2024-07-17T08:50:10.172 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90507.log 2024-07-17T08:50:10.172 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62383.log.gz 2024-07-17T08:50:10.173 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114966.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114966.log.gz 2024-07-17T08:50:10.173 INFO:teuthology.orchestra.run.smithi044.stderr: --verbose -- /var/log/ceph/ceph-mds.a.log 2024-07-17T08:50:10.174 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90507.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90507.log.gz 2024-07-17T08:50:10.175 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63745.log 2024-07-17T08:50:10.176 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110891.log 2024-07-17T08:50:10.177 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63745.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63745.log.gz 2024-07-17T08:50:10.179 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.110891.log: --verbose -- /var/log/ceph/ceph-client.admin.38042.log 2024-07-17T08:50:10.179 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110891.log.gz 2024-07-17T08:50:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112317.log 2024-07-17T08:50:10.180 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38042.log.gz 2024-07-17T08:50:10.181 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93450.log 2024-07-17T08:50:10.182 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112317.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.112317.log.gz 2024-07-17T08:50:10.183 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93918.log 2024-07-17T08:50:10.184 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93450.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.93450.log.gz 2024-07-17T08:50:10.185 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108674.log 2024-07-17T08:50:10.186 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93918.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93918.log.gz 2024-07-17T08:50:10.187 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85865.log 2024-07-17T08:50:10.188 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108674.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108674.log.gz 2024-07-17T08:50:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105078.log 2024-07-17T08:50:10.189 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85865.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85865.log.gz 2024-07-17T08:50:10.191 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115643.log 2024-07-17T08:50:10.191 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.105078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105078.log.gz 2024-07-17T08:50:10.192 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85697.log 2024-07-17T08:50:10.193 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115643.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.115643.log.gz 2024-07-17T08:50:10.194 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.85697.log: -- /var/log/ceph/ceph-client.admin.115138.log 2024-07-17T08:50:10.195 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85697.log.gz 2024-07-17T08:50:10.196 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114559.log 2024-07-17T08:50:10.197 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.115138.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.115138.log.gz 2024-07-17T08:50:10.197 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62798.log 2024-07-17T08:50:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114559.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114559.log.gz 2024-07-17T08:50:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.62798.log: --verbose -- /var/log/ceph/ceph-client.admin.105974.log 2024-07-17T08:50:10.200 INFO:teuthology.orchestra.run.smithi044.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62798.log.gz 2024-07-17T08:50:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68433.log 2024-07-17T08:50:10.203 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105974.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105974.log.gz 2024-07-17T08:50:10.203 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85926.log 2024-07-17T08:50:10.204 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68433.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.68433.log.gz 2024-07-17T08:50:10.205 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.85926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61575.log 2024-07-17T08:50:10.206 INFO:teuthology.orchestra.run.smithi044.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85926.log.gz 2024-07-17T08:50:10.207 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114682.log 2024-07-17T08:50:10.208 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61575.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61575.log.gz 2024-07-17T08:50:10.209 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90922.log 2024-07-17T08:50:10.209 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.114682.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114682.log.gz 2024-07-17T08:50:10.211 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74028.log 2024-07-17T08:50:10.211 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90922.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90922.log.gz 2024-07-17T08:50:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.74028.log: --verbose -- /var/log/ceph/ceph-client.admin.75785.log 2024-07-17T08:50:10.213 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74028.log.gz 2024-07-17T08:50:10.214 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91017.log 2024-07-17T08:50:10.215 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75785.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75785.log.gz 2024-07-17T08:50:10.216 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108072.log 2024-07-17T08:50:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91017.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91017.log.gz 2024-07-17T08:50:10.217 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-07-17T08:50:10.219 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108072.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.108072.log.gz 2024-07-17T08:50:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95621.log 2024-07-17T08:50:10.220 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.48967.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-07-17T08:50:10.221 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37769.log 2024-07-17T08:50:10.222 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95621.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95621.log.gz 2024-07-17T08:50:10.223 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2024-07-17T08:50:10.224 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37769.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37769.log.gz 2024-07-17T08:50:10.225 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109691.log/var/log/ceph/ceph-client.admin.37542.log: 2024-07-17T08:50:10.225 INFO:teuthology.orchestra.run.smithi044.stderr: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2024-07-17T08:50:10.226 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95579.log 2024-07-17T08:50:10.227 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109691.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.109691.log.gz 2024-07-17T08:50:10.228 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44541.log 2024-07-17T08:50:10.229 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95579.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95579.log.gz 2024-07-17T08:50:10.230 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56869.log 2024-07-17T08:50:10.231 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44541.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44541.log.gz 2024-07-17T08:50:10.232 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102338.log/var/log/ceph/ceph-client.admin.56869.log: 2024-07-17T08:50:10.233 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56869.log.gz 2024-07-17T08:50:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82352.log 2024-07-17T08:50:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.102338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102338.log.gz 2024-07-17T08:50:10.235 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108590.log 2024-07-17T08:50:10.236 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.82352.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82352.log.gz 2024-07-17T08:50:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118101.log 2024-07-17T08:50:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108590.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.108590.log.gz 2024-07-17T08:50:10.239 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55908.log 2024-07-17T08:50:10.240 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118101.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118101.log.gz 2024-07-17T08:50:10.240 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2024-07-17T08:50:10.242 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55908.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55908.log.gz 2024-07-17T08:50:10.243 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73758.log 2024-07-17T08:50:10.243 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55782.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-07-17T08:50:10.244 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117089.log 2024-07-17T08:50:10.246 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.73758.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73758.log.gz 2024-07-17T08:50:10.246 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86603.log 2024-07-17T08:50:10.247 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117089.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117089.log.gz 2024-07-17T08:50:10.248 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97556.log 2024-07-17T08:50:10.249 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86603.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.86603.log.gz 2024-07-17T08:50:10.250 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100085.log 2024-07-17T08:50:10.250 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97556.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97556.log.gz 2024-07-17T08:50:10.251 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56095.log 2024-07-17T08:50:10.253 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100085.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.100085.log.gz 2024-07-17T08:50:10.253 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90297.log 2024-07-17T08:50:10.254 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56095.log.gz 2024-07-17T08:50:10.255 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-07-17T08:50:10.256 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90297.log.gz 2024-07-17T08:50:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63392.log 2024-07-17T08:50:10.257 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45103.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-07-17T08:50:10.258 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89896.log 2024-07-17T08:50:10.260 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63392.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63392.log.gz 2024-07-17T08:50:10.262 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.89896.log: --verbose -- /var/log/ceph/ceph-client.admin.112401.log 2024-07-17T08:50:10.262 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89896.log.gz 2024-07-17T08:50:10.263 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105848.log 2024-07-17T08:50:10.264 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112401.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.112401.log.gz 2024-07-17T08:50:10.264 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105932.log/var/log/ceph/ceph-client.admin.105848.log: 2024-07-17T08:50:10.265 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105848.log.gz 2024-07-17T08:50:10.266 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101607.log 2024-07-17T08:50:10.267 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105932.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105932.log.gz 2024-07-17T08:50:10.268 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99192.log 2024-07-17T08:50:10.268 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.101607.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101607.log.gz 2024-07-17T08:50:10.269 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43341.log 2024-07-17T08:50:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99192.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.99192.log.gz 2024-07-17T08:50:10.272 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43341.log: /var/log/ceph/ceph-client.admin.74155.log 2024-07-17T08:50:10.272 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43341.log.gz 2024-07-17T08:50:10.273 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56645.log 2024-07-17T08:50:10.274 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74155.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74155.log.gz 2024-07-17T08:50:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63615.log 2024-07-17T08:50:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56645.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56645.log.gz 2024-07-17T08:50:10.277 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93324.log 2024-07-17T08:50:10.277 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63615.log.gz 2024-07-17T08:50:10.278 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109308.log 2024-07-17T08:50:10.279 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93324.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.93324.log.gz 2024-07-17T08:50:10.280 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112971.log 2024-07-17T08:50:10.281 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109308.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.109308.log.gz 2024-07-17T08:50:10.282 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90339.log 2024-07-17T08:50:10.283 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112971.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112971.log.gz 2024-07-17T08:50:10.284 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2024-07-17T08:50:10.284 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90339.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90339.log.gz 2024-07-17T08:50:10.285 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120536.log 2024-07-17T08:50:10.286 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51336.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2024-07-17T08:50:10.287 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87777.log 2024-07-17T08:50:10.288 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.120536.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.120536.log.gz 2024-07-17T08:50:10.289 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103459.log 2024-07-17T08:50:10.290 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.87777.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87777.log.gz 2024-07-17T08:50:10.291 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55593.log 2024-07-17T08:50:10.292 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.103459.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103459.log.gz 2024-07-17T08:50:10.293 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62278.log 2024-07-17T08:50:10.293 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55593.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55593.log.gz 2024-07-17T08:50:10.294 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111017.log 2024-07-17T08:50:10.295 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62278.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62278.log.gz 2024-07-17T08:50:10.296 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.111017.log: --verbose -- /var/log/ceph/ceph-client.admin.60022.log 2024-07-17T08:50:10.297 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111017.log.gz 2024-07-17T08:50:10.298 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46731.log 2024-07-17T08:50:10.299 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60022.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60022.log.gz 2024-07-17T08:50:10.300 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105659.log 2024-07-17T08:50:10.300 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.46731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46731.log.gz 2024-07-17T08:50:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94441.log 2024-07-17T08:50:10.302 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105659.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105659.log.gz 2024-07-17T08:50:10.303 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70718.log 2024-07-17T08:50:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94441.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94441.log.gz 2024-07-17T08:50:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99360.log 2024-07-17T08:50:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70718.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.70718.log.gz 2024-07-17T08:50:10.307 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47190.log 2024-07-17T08:50:10.307 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.99360.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99360.log.gz 2024-07-17T08:50:10.308 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96592.log 2024-07-17T08:50:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.2.47190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47190.log.gz 2024-07-17T08:50:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71995.log 2024-07-17T08:50:10.311 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96592.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96592.log.gz 2024-07-17T08:50:10.312 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65426.log 2024-07-17T08:50:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71995.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71995.log.gz 2024-07-17T08:50:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118654.log 2024-07-17T08:50:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.65426.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65426.log.gz 2024-07-17T08:50:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90381.log 2024-07-17T08:50:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.118654.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.118654.log.gz 2024-07-17T08:50:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66480.log 2024-07-17T08:50:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.90381.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90381.log.gz 2024-07-17T08:50:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97290.log 2024-07-17T08:50:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66480.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66480.log.gz 2024-07-17T08:50:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119992.log 2024-07-17T08:50:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97290.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97290.log.gz 2024-07-17T08:50:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38626.log 2024-07-17T08:50:10.324 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119992.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.119992.log.gz 2024-07-17T08:50:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54684.log 2024-07-17T08:50:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38626.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38626.log.gz 2024-07-17T08:50:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101956.log 2024-07-17T08:50:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54684.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54684.log.gz 2024-07-17T08:50:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61299.log/var/log/ceph/ceph-client.admin.101956.log: 2024-07-17T08:50:10.329 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101956.log.gz 2024-07-17T08:50:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71739.log 2024-07-17T08:50:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.61299.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61299.log.gz 2024-07-17T08:50:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97054.log 2024-07-17T08:50:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71739.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71739.log.gz 2024-07-17T08:50:10.333 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67653.log 2024-07-17T08:50:10.334 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97054.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97054.log.gz 2024-07-17T08:50:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107263.log 2024-07-17T08:50:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67653.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.67653.log.gz 2024-07-17T08:50:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97850.log 2024-07-17T08:50:10.338 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.107263.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107263.log.gz 2024-07-17T08:50:10.339 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60780.log 2024-07-17T08:50:10.339 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97850.log.gz 2024-07-17T08:50:10.340 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57909.log 2024-07-17T08:50:10.341 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.60780.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60780.log.gz 2024-07-17T08:50:10.342 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86709.log 2024-07-17T08:50:10.343 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57909.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57909.log.gz 2024-07-17T08:50:10.344 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.86709.log: /var/log/ceph/ceph-client.admin.116861.log 2024-07-17T08:50:10.344 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86709.log.gz 2024-07-17T08:50:10.345 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107137.log 2024-07-17T08:50:10.346 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116861.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.116861.log.gz 2024-07-17T08:50:10.347 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66120.log/var/log/ceph/ceph-client.admin.107137.log: 2024-07-17T08:50:10.348 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107137.log.gz 2024-07-17T08:50:10.349 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-07-17T08:50:10.350 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.66120.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66120.log.gz 2024-07-17T08:50:10.351 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80745.log 2024-07-17T08:50:10.352 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.55992.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-07-17T08:50:10.353 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64714.log 2024-07-17T08:50:10.353 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80745.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80745.log.gz 2024-07-17T08:50:10.354 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90568.log 2024-07-17T08:50:10.355 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.64714.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64714.log.gz 2024-07-17T08:50:10.356 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113109.log/var/log/ceph/ceph-client.admin.90568.log: 2024-07-17T08:50:10.357 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90568.log.gz 2024-07-17T08:50:10.358 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63829.log 2024-07-17T08:50:10.359 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113109.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.113109.log.gz 2024-07-17T08:50:10.360 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-07-17T08:50:10.361 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63829.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63829.log.gz 2024-07-17T08:50:10.362 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67401.log 2024-07-17T08:50:10.362 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.54947.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-07-17T08:50:10.363 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95196.log 2024-07-17T08:50:10.364 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.67401.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67401.log.gz 2024-07-17T08:50:10.365 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87438.log 2024-07-17T08:50:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95196.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.95196.log.gz 2024-07-17T08:50:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50739.log 2024-07-17T08:50:10.368 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.87438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87438.log.gz 2024-07-17T08:50:10.368 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99318.log 2024-07-17T08:50:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50739.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50739.log.gz 2024-07-17T08:50:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-client.admin.99318.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.122045.log 2024-07-17T08:50:10.371 INFO:teuthology.orchestra.run.smithi044.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99318.log.gz 2024-07-17T08:50:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75727.log 2024-07-17T08:50:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.122045.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122045.log.gz 2024-07-17T08:50:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70676.log/var/log/ceph/ceph-client.admin.75727.log: 2024-07-17T08:50:10.375 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75727.log.gz 2024-07-17T08:50:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38322.log 2024-07-17T08:50:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.70676.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70676.log.gz 2024-07-17T08:50:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108716.log 2024-07-17T08:50:10.378 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38322.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38322.log.gz 2024-07-17T08:50:10.380 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.108716.log: --verbose -- /var/log/ceph/ceph-client.admin.93960.log 2024-07-17T08:50:10.380 INFO:teuthology.orchestra.run.smithi044.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108716.log.gz 2024-07-17T08:50:10.381 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96296.log 2024-07-17T08:50:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.93960.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93960.log.gz 2024-07-17T08:50:10.383 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45378.log 2024-07-17T08:50:10.384 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.96296.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.96296.log.gz 2024-07-17T08:50:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86094.log 2024-07-17T08:50:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45378.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45378.log.gz 2024-07-17T08:50:10.386 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37784.log 2024-07-17T08:50:10.387 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86094.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86094.log.gz 2024-07-17T08:50:10.388 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100364.log 2024-07-17T08:50:10.389 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.37784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37784.log.gz 2024-07-17T08:50:10.390 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112743.log 2024-07-17T08:50:10.391 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.100364.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100364.log.gz 2024-07-17T08:50:10.392 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31523.log 2024-07-17T08:50:10.392 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.112743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112743.log.gz 2024-07-17T08:50:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104213.log 2024-07-17T08:50:10.394 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.tmp-client.admin.31523.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31523.log.gz 2024-07-17T08:50:10.395 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107886.log 2024-07-17T08:50:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.104213.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104213.log.gz 2024-07-17T08:50:10.397 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.107886.log: --verbose -- /var/log/ceph/ceph-client.admin.80535.log 2024-07-17T08:50:10.397 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107886.log.gz 2024-07-17T08:50:10.398 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68888.log 2024-07-17T08:50:10.399 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.80535.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80535.log.gz 2024-07-17T08:50:10.400 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71252.log 2024-07-17T08:50:10.401 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.68888.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68888.log.gz 2024-07-17T08:50:10.402 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62713.log 2024-07-17T08:50:10.403 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.71252.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.71252.log.gz 2024-07-17T08:50:10.404 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94044.log 2024-07-17T08:50:10.405 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.62713.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62713.log.gz 2024-07-17T08:50:10.406 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111101.log 2024-07-17T08:50:10.406 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94044.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94044.log.gz 2024-07-17T08:50:10.407 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121625.log 2024-07-17T08:50:10.408 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111101.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111101.log.gz 2024-07-17T08:50:10.409 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112913.log 2024-07-17T08:50:10.410 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.121625.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121625.log.gz 2024-07-17T08:50:10.411 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92247.log 2024-07-17T08:50:10.412 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.112913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112913.log.gz 2024-07-17T08:50:10.413 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113469.log 2024-07-17T08:50:10.413 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92247.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92247.log.gz 2024-07-17T08:50:10.414 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108800.log 2024-07-17T08:50:10.415 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.113469.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113469.log.gz 2024-07-17T08:50:10.416 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95238.log/var/log/ceph/ceph-client.admin.108800.log: 2024-07-17T08:50:10.417 INFO:teuthology.orchestra.run.smithi044.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108800.log.gz 2024-07-17T08:50:10.418 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52109.log 2024-07-17T08:50:10.419 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.95238.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95238.log.gz 2024-07-17T08:50:10.420 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96086.log 2024-07-17T08:50:10.420 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.52109.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52109.log.gz 2024-07-17T08:50:10.421 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-07-17T08:50:10.422 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.96086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96086.log.gz 2024-07-17T08:50:10.423 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57565.log 2024-07-17T08:50:10.424 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45448.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-07-17T08:50:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75957.log 2024-07-17T08:50:10.425 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.0.57565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57565.log.gz 2024-07-17T08:50:10.426 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105480.log 2024-07-17T08:50:10.428 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.75957.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.75957.log.gz 2024-07-17T08:50:10.429 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.105480.log: /var/log/ceph/ceph-client.admin.51000.log 2024-07-17T08:50:10.429 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105480.log.gz 2024-07-17T08:50:10.430 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79269.log 2024-07-17T08:50:10.431 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51000.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51000.log.gz 2024-07-17T08:50:10.432 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106245.log 2024-07-17T08:50:10.433 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.79269.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79269.log.gz 2024-07-17T08:50:10.434 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108884.log 2024-07-17T08:50:10.434 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.106245.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106245.log.gz 2024-07-17T08:50:10.435 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51294.log 2024-07-17T08:50:10.436 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.108884.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108884.log.gz 2024-07-17T08:50:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109185.log 2024-07-17T08:50:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51294.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51294.log.gz 2024-07-17T08:50:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119374.log 2024-07-17T08:50:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.109185.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109185.log.gz 2024-07-17T08:50:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-07-17T08:50:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.119374.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119374.log.gz 2024-07-17T08:50:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94483.log 2024-07-17T08:50:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45682.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-07-17T08:50:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111353.log 2024-07-17T08:50:10.445 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.94483.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94483.log.gz 2024-07-17T08:50:10.446 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57866.log 2024-07-17T08:50:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.111353.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.111353.log.gz 2024-07-17T08:50:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92516.log 2024-07-17T08:50:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.57866.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57866.log.gz 2024-07-17T08:50:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97724.log 2024-07-17T08:50:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.92516.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92516.log.gz 2024-07-17T08:50:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105439.log 2024-07-17T08:50:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.97724.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97724.log.gz 2024-07-17T08:50:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116590.log 2024-07-17T08:50:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.105439.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.105439.log.gz 2024-07-17T08:50:10.455 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-07-17T08:50:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.116590.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116590.log.gz 2024-07-17T08:50:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91785.log 2024-07-17T08:50:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.51126.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-07-17T08:50:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-07-17T08:50:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.91785.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91785.log.gz 2024-07-17T08:50:10.460 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63787.log 2024-07-17T08:50:10.461 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.50181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-07-17T08:50:10.462 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86178.log 2024-07-17T08:50:10.463 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.63787.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.63787.log.gz 2024-07-17T08:50:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38020.log 2024-07-17T08:50:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.86178.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86178.log.gz 2024-07-17T08:50:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117653.log 2024-07-17T08:50:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.38020.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38020.log.gz 2024-07-17T08:50:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56687.log 2024-07-17T08:50:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.117653.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117653.log.gz 2024-07-17T08:50:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74664.log 2024-07-17T08:50:10.470 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.56687.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56687.log.gz 2024-07-17T08:50:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.74664.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74664.log.gz 2024-07-17T08:50:10.793 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-07-17T08:50:11.244 INFO:teuthology.orchestra.run.smithi175.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-07-17T08:50:11.343 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-07-17T08:50:12.137 INFO:teuthology.orchestra.run.smithi044.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-07-17T08:50:14.828 INFO:teuthology.orchestra.run.smithi175.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-07-17T08:50:15.373 INFO:teuthology.orchestra.run.smithi044.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-07-17T08:50:15.425 INFO:teuthology.orchestra.run.smithi175.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-07-17T08:50:15.460 INFO:teuthology.orchestra.run.smithi044.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-07-17T08:50:15.506 INFO:teuthology.orchestra.run.smithi175.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-07-17T08:50:15.741 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-07-17T08:50:16.172 INFO:teuthology.orchestra.run.smithi044.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-07-17T08:50:17.159 INFO:teuthology.orchestra.run.smithi175.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-07-17T08:50:17.163 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-07-17T08:50:17.163 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m8.503s 2024-07-17T08:50:17.163 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m34.088s 2024-07-17T08:50:17.163 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m2.689s 2024-07-17T08:50:18.783 INFO:teuthology.orchestra.run.smithi044.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-07-17T08:50:18.787 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-07-17T08:50:18.787 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m10.126s 2024-07-17T08:50:18.787 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m35.740s 2024-07-17T08:50:18.787 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m5.958s 2024-07-17T08:50:18.788 INFO:tasks.ceph:Archiving logs... 2024-07-17T08:50:18.789 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi044/log 2024-07-17T08:50:18.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-17T08:50:24.604 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi175/log 2024-07-17T08:50:24.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-17T08:50:29.250 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-17T08:50:29.260 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-17T08:50:29.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-17T08:50:29.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-17T08:50:29.413 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-17T08:50:29.432 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-17T08:50:29.432 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-17T08:50:29.440 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-17T08:50:29.489 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-07-17T08:50:29.492 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 8 377 88 +154us[ +95us] +/- 55ms 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 88 -772us[ -831us] +/- 58ms 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +749us[ +749us] +/- 81ms 2024-07-17T08:50:29.500 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T08:50:29.502 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-17T08:50:29.502 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-17T08:50:29.503 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 9 377 280 +176us[ +156us] +/- 52ms 2024-07-17T08:50:29.503 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 21 -784us[ -784us] +/- 59ms 2024-07-17T08:50:29.503 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +753us[ +753us] +/- 81ms 2024-07-17T08:50:29.503 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-07-17T08:50:29.504 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-17T08:50:29.513 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-17T08:50:29.514 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-17T08:50:29.522 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-17T08:50:29.790 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-17T08:50:29.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-17T08:50:30.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:50:30.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-17T08:50:30.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:50:30.178 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-17T08:50:30.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-17T08:50:30.197 INFO:teuthology.task.internal:Duration was 7539.389415 seconds 2024-07-17T08:50:30.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-17T08:50:30.214 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-17T08:50:30.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-17T08:50:30.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-17T08:50:30.348 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-17T08:50:30.360 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-17T08:50:30.679 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-17T08:50:30.679 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-07-17T08:50:30.679 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-17T08:50:30.748 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-07-17T08:50:30.748 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-17T08:50:30.820 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-17T08:50:30.820 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-17T08:50:30.827 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-17T08:50:30.908 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-17T08:50:30.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-17T08:50:30.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-17T08:50:31.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-17T08:50:31.208 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-17T08:50:31.209 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-17T08:50:31.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-17T08:50:31.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-17T08:50:31.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-17T08:50:31.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-17T08:50:31.506 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-07-17T08:50:31.507 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-07-17T08:50:31.550 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-17T08:50:31.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:50:31.598 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-17T08:50:31.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-17T08:50:31.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-17T08:50:31.667 INFO:teuthology.task.internal:Transferring archived files... 2024-07-17T08:50:31.668 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi044 2024-07-17T08:50:31.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-17T08:50:31.773 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-07-17_06:32:46-fs-main-distro-default-smithi/7805141/remote/smithi175 2024-07-17T08:50:31.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-17T08:50:31.947 INFO:teuthology.task.internal:Removing archive directory... 2024-07-17T08:50:31.947 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-17T08:50:31.955 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-17T08:50:32.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-17T08:50:32.020 INFO:teuthology.task.internal:Not uploading archives. 2024-07-17T08:50:32.020 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-17T08:50:32.058 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-17T08:50:32.058 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-17T08:50:32.065 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-17T08:50:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 17 08:50 /home/ubuntu/cephtest 2024-07-17T08:50:32.120 INFO:teuthology.orchestra.run.smithi175.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 17 08:50 /home/ubuntu/cephtest 2024-07-17T08:50:32.123 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-17T08:50:32.133 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-17T08:50:32.157 INFO:teuthology.run:Summary data: client.0-kernel-sha1: c30fd5683ea85fa16b09ca6926db2200d11edd1d client.1-kernel-sha1: c30fd5683ea85fa16b09ca6926db2200d11edd1d description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap-schedule} duration: 7539.389414548874 failure_reason: 'Test failure: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=513b20ac45f04564bb49a579c0eb9741 status: fail success: false 2024-07-17T08:50:32.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-17T08:50:32.237 INFO:teuthology.run:FAIL